diff --git a/HANDOFF.md b/HANDOFF.md index 8cd679c..0a4899d 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -1,9 +1,10 @@ # Jellyfin Web VR180 — 项目交接文档 > 给新对话用。读完这份就能接手,不需要重新调查环境。 -> 最后更新:已部署版本 `b9c3a5`(`NS.version = hud-8`;ADDENDUM 6 字幕深度/大小解耦 +> 最后更新:已部署版本 `e394bcb`(`NS.version = hud-9`;ADDENDUM 6 字幕深度/大小解耦 > + 7 唤出门限/前面板精简 + 7b 锁按钮与「锁定」面板模式 + 8/8b 画面搬迁(实时拖动) -> + 9 rAF 代理修复(VR 内字幕不再冻结)+ 普通/双目字幕自动识别) +> + 9 rAF 代理修复(VR 内字幕不再冻结)+ 普通/双目字幕自动识别 +> + 10 原生(SRT 等)字幕由插件自绘到 HUD;布局覆盖不再持久化) > > ⚠️ **Jellyfin web 注册了 service worker**(`serviceworker.js`,scope `/web/`),会把 > `index.html` 缓存在 SW cache 里。重新部署后如果只是普通刷新,浏览器可能仍用旧的 @@ -414,6 +415,7 @@ handsReady / handsErr / handsSources / handsFrames subMode / subCanvasSize / subCrop / subTex / subOpacity / subPlacement subLockAngle / subAngleDeg / subTextDeg / subYOverZ ← ADDENDUM 6 subLayout / subDual / subEyeW / subCopyND / subCenterInk ← ADDENDUM 9 +subSource / subNativeText ← ADDENDUM 10 ctlSubsOn / ctlSubDist / ctlSubAngle / ctlSubLock ← 控制条实际画出的值 ctlLock / ctlMode / ctlHold / pressAt / lastActivate ← ADDENDUM 7/7b 锁、面板保持与唤出判定 videoYaw / videoPitch / videoRotMode / videoTransformWritable ← ADDENDUM 8/8b 画面搬迁 @@ -569,3 +571,60 @@ for (const k of await caches.keys()) await caches.delete(k); - 下次戴上头显后要确认:`stats.subLayout=auto`、双目轨 `subDual=true/subEyeW=2160/ subCrop` 落在左半屏(≤2160)、普通轨 `subDual=false/subEyeW=4320/subCrop` 跨中线、 `subCopyND` 与 `subCenterInk` 是否符合上面的表。 + +--- + +## 14. ADDENDUM 10 — 普通字幕(SRT 等)为什么读不到(已部署 `e394bcb` / `hud-9`) + +### 14.1 根因:Jellyfin 只把 ssa/ass 交给 libass + +`htmlVideoPlayer-plugin` 的 `renderTracksEvents` 决策: + +```js +a = (n.Codec || '').toLowerCase(); +if (!X.includes(a)) { // X = ['ssa','ass'] + if (a === 'pgssub') -> renderPgs + if (Y.includes(a)) -> renderVobSub // Y = ['dvdsub','vobsub'] + if (!u(h.fe)) -> renderSubtitlesWithCustomElement // DOM 覆盖层 + /* 否则落到原生 TextTrack 分支 */ +} +return renderSsaAss(...); // 只有 ssa/ass +``` + +所以 `subrip`/`vtt` **根本不会创建 libass canvas**。设备实测(选中 `CN` SRT 时): +`canvas.libassjs-canvas` 数量 **0**、`.libassjs-canvas-parent` 0、`.videoSubtitles` 0, +只有 `video.textTracks[0] = { kind:'subtitles', label:'manualTrack', mode:'showing', +cues:315 }` —— 浏览器用 `::cue` 自己画。插件**无论改什么布局都读不到任何东西**。 + +### 14.2 修复(`src/10-subtitles.js`) + +`updateSource()` 每帧挑来源: + +- 有非空 `canvas.libassjs-canvas` → `srcKind='libass'`(原路径不变); +- 否则若有 showing 的 TextTrack 或 `.videoSubtitlesInner` → `srcKind='native'`: + 从 `textTracks[i].activeCues` 读当前 cue(兜底读 DOM 覆盖层 `innerText`), + 清洗 ASS/HTML 标签与双向控制符、按字符折行,用白字黑描边画进私有 1920×1080 + canvas,再走**完全相同的裁剪/上传/HUD 管线**; +- 都没有 → `srcKind='none'`,清一次字幕条。 + +`prepare()` 在没有 libass canvas 时跳过(放大播放器框只是为了让 libass 重渲染,原生 +canvas 是我们自己的)。新增诊断 `stats.subSource` / `stats.subNativeText` 和 +`Subtitles.getSourceKind()`。 + +### 14.3 血的教训:布局覆盖**不要持久化** + +ADDENDUM 9 把 `state.subLayout` 存进了 `jfvr.settings.v1`。结果一个残留的 `'mono'` +静默地把默认的**双目**内容强制成普通:`dual:false`、`eyeW:4320`、裁剪跨越两眼接缝 → +HUD 把两只眼的字幕并排显示。找这个 bug 花的时间比写检测还多。现在 `subLayout` +**不持久化**(和 `subtitles`、`subMode:'off'` 同理),开机恒为 `auto`,面板可临时改。 + +### 14.4 设备实测(`hud-9`) + +| 选中轨 | `subSource` | 结果 | +|---|---|---| +| `CN_dual_eye`(ass,idx 0) | `libass` | `dual:true`、`eyeW:2160`、`crop:[1040,768,1176,924]`(左半)、`copyND:0.055`、uploads 递增 | +| `CN`(subrip,idx 1) | `native` | `dual:false`、`eyeW:1920`、canvas 1920×1080、`text:"进去了呢"`、`crop` 256×88、uploads 递增 | + +私有 cue canvas 还读回做了 OCR(cue `怎么样?好好地` → OCR `怎么样? 好好地`); +注入/移除一个假的 `canvas.libassjs-canvas` 能让 `subSource` 在 `libass`/`native` +间切换且 `subErr` 为空。 diff --git a/INTERFACES.md b/INTERFACES.md index b24f89b..c80f1c0 100644 --- a/INTERFACES.md +++ b/INTERFACES.md @@ -1069,3 +1069,69 @@ __JFVR.Subtitles.getLayout(): string - On device (before the UI became unreachable): the broker change was observed to carry SubtitlesOctopus's loop through `immersive-vr` (`rafId` in the plugin's id range, `eventStart` tracking, `subUploads` 4 → 7 over 20 s in VR). + +# ADDENDUM 10 — normal (non-ASS) subtitles: Jellyfin does not use libass for them + +## 1. Why normal subtitles were invisible + +The HUD samples `canvas.libassjs-canvas`. That canvas only exists when Jellyfin +routes the subtitle through **SubtitlesOctopus (libass)**. From +`htmlVideoPlayer-plugin` (`renderTracksEvents`): + +```js +a = (n.Codec || '').toLowerCase(); +if (!X.includes(a)) { // X = ['ssa','ass'] + if (a === 'pgssub') -> renderPgs + if (Y.includes(a)) -> renderVobSub // Y = ['dvdsub','vobsub'] + if (!u(h.fe)) -> renderSubtitlesWithCustomElement // DOM overlay + /* else falls through to the native TextTrack path */ +} +return renderSsaAss(...); // ssa/ass ONLY +``` + +So `subrip`/`vtt`/… never reach libass. Observed on device with the `CN` SRT track +selected: `canvas.libassjs-canvas` count **0**, `.libassjs-canvas-parent` 0, +`.videoSubtitles` 0 — and `video.textTracks[0] = { kind:'subtitles', label:'manualTrack', +mode:'showing', cues:315 }`, i.e. the **browser** is rendering the cues through +`::cue`. There was nothing for the plugin to read, at any layout setting. + +## 2. What the module does now + +`updateSource()` runs every XR frame and picks the source: + +- a `canvas.libassjs-canvas` with a non-zero size → `srcKind = 'libass'` (unchanged path); +- otherwise, if the player has a showing TextTrack or a `.videoSubtitlesInner` + overlay → `srcKind = 'native'`. The active text comes from + `textTracks[i].activeCues` (fallback: the DOM overlay's `innerText`), is cleaned + of ASS/HTML tags and bidi marks, wrapped, and painted — white with a black outline, + centred — into a private 1920×1080 canvas. That canvas is fed through the exact + same crop/bbox/upload code as the libass one. +- neither → `srcKind = 'none'` and the strip is cleared once. + +`prepare()` is a no-op without a libass canvas (blowing up the player box is only +needed so libass re-renders bigger; the native canvas is ours). + +New diagnostics: `stats.subSource` (`'libass' | 'native' | 'none'`), +`stats.subNativeText`; `__JFVR.Subtitles.getSourceKind()`. + +## 3. The persisted layout override was a trap + +ADDENDUM 9 persisted `state.subLayout`. A stale `'mono'` in `jfvr.settings.v1` +silently forced the mono branch on the default **dual-eye** content: `dual:false`, +`eyeW:4320`, and a crop spanning the seam, so the HUD showed both eyes' text side by +side. Debugging that cost more time than the detection itself. + +`subLayout` is therefore **not persisted** (same reasoning as `subtitles` and +`subMode:'off'` — a stored value that leaves the layer looking broken). It is +`'auto'` at boot, toggleable per session from the panel. + +## 4. Verification (on device, build `hud-9`) + +| selected track | `subSource` | result | +|---|---|---| +| `CN_dual_eye` (ass, idx 0) | `libass` | `dual:true`, `eyeW:2160`, `crop:[1040,768,1176,924]` (left half), `copyND:0.055`, uploads climbing | +| `CN` (subrip, idx 1) | `native` | `dual:false`, `eyeW:1920`, canvas 1920×1080, `text:"进去了呢"`, `crop` 256×88, uploads climbing | + +The private cue canvas was read back and OCR'd ("怎么样? 好好地" for cue +"怎么样?好好地"), and injecting/removing a fake `canvas.libassjs-canvas` flips +`subSource` between `libass` and `native` with no errors. diff --git a/dist/jfvr.js b/dist/jfvr.js index 491b90b..86a18a8 100644 --- a/dist/jfvr.js +++ b/dist/jfvr.js @@ -19,7 +19,7 @@ var NS = window.__JFVR = window.__JFVR || {}; - NS.version = 'hud-8'; // hud-8 = ADDENDUM 9 (rAF broker always-on so the libass loop survives XR entry; auto dual/mono subtitle layout) + NS.version = 'hud-9'; // hud-9 = ADDENDUM 10 (native/TextTrack subtitles painted by the HUD; layout override per-session, ADDENDUM 9 rAF broker + auto dual/mono layout) NS.__core = true; /* ------------------------------------------------------------------ * @@ -439,6 +439,15 @@ var CENTER_BAND_PX = 40; // source px around the eye seam that count as "centre ink" var COPY_TOL = 0.25; // column-profile copy distance for "two copies of one line" // (measured: dual 0.02-0.08, mono 1.00) + // ADDENDUM 10 — Jellyfin only routes ssa/ass through SubtitlesOctopus (the + // libass canvas). Every other text codec (subrip, vtt, ...) is drawn by the + // BROWSER from a TextTrack (`video.addTextTrack` + mode 'showing'), so there is + // no canvas at all to sample. The HUD then paints the active cue itself into a + // private canvas and feeds it through the same crop/upload pipeline. + var NATIVE_W = 1920, NATIVE_H = 1080; + var NATIVE_FONT_H = 0.052; // cue font size as a fraction of the canvas height + var NATIVE_LINE_H = 1.32; // line advance + var NATIVE_MAX_W = 0.9; // max text width as a fraction of the canvas width var SPHERE_TEX_W = 4096; // legacy equirect layer size var SPHERE_TEX_H = 2048; var OPACITY_DEFAULT = 0.85; // dim the strip below full opacity by default @@ -515,6 +524,9 @@ copyND: null, // diagnostics: normalised column-profile copy distance centerInk: null, // diagnostics: ink pixels in the seam band colL: null, colR: null, // reused column-profile scratch (half width each) + // ADDENDUM 10: where the HUD's source pixels come from. + srcKind: 'none', // 'libass' | 'native' | 'none' + nativeCanvas: null, nativeCtx: null, nativeText: null, bbox: null, lastSample: null, hasInk: false, @@ -535,6 +547,174 @@ catch (e) { return null; } } + /* ================================================================ + * ADDENDUM 10 — native (non-libass) subtitle source + * Jellyfin asks the browser to render subrip/vtt cues from a TextTrack, so the + * HUD has no canvas to read. Read the active cue, paint it into a private + * canvas, and let the normal crop/upload path take over from there. + * ============================================================== */ + function ensureNativeCanvas() { + if (!M.nativeCanvas) { + try { + M.nativeCanvas = document.createElement('canvas'); + M.nativeCanvas.width = NATIVE_W; + M.nativeCanvas.height = NATIVE_H; + M.nativeCtx = M.nativeCanvas.getContext('2d'); + // Kept in the DOM but hidden: Canvas2D works detached too, and this makes + // the exact pixels the HUD uploads inspectable from DevTools. + M.nativeCanvas.className = 'jfvr-native-subs'; + M.nativeCanvas.style.display = 'none'; + (document.body || document.documentElement).appendChild(M.nativeCanvas); + } catch (e) { M.nativeCtx = null; } + } + return !!M.nativeCtx; + } + + // Jellyfin renders non-ASS text subtitles one of two ways depending on the + // build: into a TextTrack (video.addTextTrack + mode 'showing', the browser + // shows it through ::cue) or into a DOM overlay (div.videoSubtitlesInner). + // Both leave no canvas behind. + function hasNativeTracks() { + try { + var v = S.video; + if (v && v.textTracks) { + for (var i = 0; i < v.textTracks.length; i++) { + var tr = v.textTracks[i]; + if (tr && tr.mode === 'showing' && tr.cues && tr.cues.length) return true; + } + } + if (document.querySelector('.videoSubtitlesInner')) return true; + } catch (e) {} + return false; + } + + function activeNativeText() { + var out = []; + try { + var v = S.video; + if (v && v.textTracks) { + for (var i = 0; i < v.textTracks.length; i++) { + var tr = v.textTracks[i]; + if (!tr || tr.mode !== 'showing') continue; + var cues = tr.activeCues; // maintained by the browser + if (!cues) continue; + for (var j = 0; j < cues.length; j++) { + var s = cues[j] && cues[j].text; + if (s) out.push(s); + } + } + } + if (!out.length) { + var d = document.querySelector('.videoSubtitlesInner, .videoSubtitles'); + if (d) { var t = d.innerText || ''; if (t) out.push(t); } + } + } catch (e) { return ''; } + return out.join('\n'); + } + + function cleanCueText(s) { + return String(s == null ? '' : s) + .replace(/\{[^}]*\}/g, '') // ASS override tags + .replace(/<[^>]*>/g, '') // html-ish tags + .replace(/ /g, ' ').replace(/&/g, '&') + .replace(/</g, '<').replace(/>/g, '>') + .replace(/[\u200e\u200f\u202a-\u202e]/g, '') // bidi control marks + .replace(/\r/g, '') + .replace(/[ \t]+\n/g, '\n') + .replace(/\n{3,}/g, '\n\n') + .trim(); + } + + function wrapCueLines(ctx, text, maxW) { + var out = []; + var paras = text.split('\n'); + for (var p = 0; p < paras.length; p++) { + var para = paras[p]; + if (!para) { out.push(''); continue; } + var line = ''; + for (var i = 0; i < para.length; i++) { + var ch = para[i]; + if (line && ctx.measureText(line + ch).width > maxW) { + var sp = line.lastIndexOf(' '); + if (sp > 0 && /[A-Za-z0-9]/.test(line.charAt(line.length - 1))) { + out.push(line.slice(0, sp)); + line = line.slice(sp + 1); + } else { + out.push(line); + line = ''; + } + } + line += ch; + } + out.push(line); + } + return out.length ? out : ['']; + } + + function drawNativeCue(text) { + var c = M.nativeCanvas, ctx = M.nativeCtx; + if (!c || !ctx) return; + var W = c.width, H = c.height; + ctx.clearRect(0, 0, W, H); + if (!text) return; + var fs = Math.max(12, Math.round(H * NATIVE_FONT_H)); + ctx.font = '600 ' + fs + 'px "Noto Sans SC","Noto Sans CJK SC","PingFang SC",' + + '"Microsoft YaHei",system-ui,Arial,sans-serif'; + ctx.textAlign = 'center'; + ctx.textBaseline = 'middle'; + ctx.lineJoin = 'round'; + ctx.miterLimit = 2; + var lines = wrapCueLines(ctx, text, W * NATIVE_MAX_W); + var lh = fs * NATIVE_LINE_H; + var y0 = H / 2 - (lines.length - 1) * lh / 2; + ctx.lineWidth = Math.max(2, Math.round(fs * 0.1)); + ctx.strokeStyle = 'rgba(0,0,0,0.95)'; + ctx.fillStyle = '#ffffff'; + for (var i = 0; i < lines.length; i++) { + ctx.strokeText(lines[i], W / 2, y0 + i * lh); + ctx.fillText(lines[i], W / 2, y0 + i * lh); + } + } + + // Decide, per frame, which canvas the HUD samples. libass wins when Jellyfin + // created one; otherwise the native cue canvas. Returns false when there is no + // subtitle source at all. + function updateSource() { + var lib = getCanvas(); + if (lib && lib.width > 0 && lib.height > 0) { + if (M.srcKind !== 'libass') { + M.srcKind = 'libass'; + M.canvas = lib; + M.nativeText = null; + M.lastSample = null; M.hasInk = false; M.bbox = null; + computeTexDims(); + M.needRebuild = true; + } + return true; + } + if (!hasNativeTracks()) { + if (M.srcKind !== 'none') { M.srcKind = 'none'; M.hasInk = false; M.bbox = null; M.dirty = true; } + return false; + } + if (!ensureNativeCanvas()) return false; + var text = cleanCueText(activeNativeText()); + if (text !== M.nativeText) { + M.nativeText = text; + drawNativeCue(text); + M.dirty = true; + } + if (M.srcKind !== 'native') { + M.srcKind = 'native'; + M.canvas = M.nativeCanvas; + M.dualEye = false; // our own canvas is always one centred line + M.mismatch = 0; + M.lastSample = null; M.hasInk = false; M.bbox = null; + computeTexDims(); + M.needRebuild = true; + } + return true; + } + function getHolder(video) { var holder = null; try { holder = document.querySelector('.videoPlayerContainer'); } catch (e) {} @@ -549,6 +729,8 @@ st.subLayer = !!M.layer; st.subPrepared = !!M.prepared; st.subCanvasSize = M.canvas ? (M.canvas.width + 'x' + M.canvas.height) : null; + st.subSource = M.srcKind; + st.subNativeText = M.nativeText ? M.nativeText.slice(0, 60) : null; st.subLayout = M.layout; st.subDual = M.dualEye; st.subEyeW = M.eyeW; @@ -578,6 +760,10 @@ * ============================================================== */ function prepare() { if (M.prepared) return; // idempotent via a live flag, not a backup check + // Only libass needs the box blown up so it re-renders bigger. The native cue + // canvas is ours, so resizing the player would be pointless (and would move + // the browser's own cue overlay around). + if (!getCanvas()) { log('prepare skipped: native subtitle source'); return; } var video = S.video || null; var holder = getHolder(video); try { @@ -824,10 +1010,6 @@ if (M.mode === 'off') { log('create: subtitle mode off'); return null; } - var canvas = getCanvas(); - if (!canvas || !canvas.width || !canvas.height) { - setErr('no libass canvas'); log('create: no '); return null; - } if (typeof window.XRWebGLBinding !== 'function') { setErr('no XRWebGLBinding'); log('create: no XRWebGLBinding'); return null; } @@ -839,11 +1021,17 @@ var gl = NS.xrGl(true); if (!gl) { setErr('no webgl2'); log('create: no webgl2 context'); return null; } - M.canvas = canvas; M.gl = gl; M.glCanvas = gl.canvas; M.uploads = 0; M.samples = 0; + // ADDENDUM 10: the source may be the libass canvas OR our own cue canvas. + // Start on the native canvas so create() works even before Jellyfin has + // created its renderer, then updateSource() upgrades to libass if it shows up. + ensureNativeCanvas(); + M.canvas = M.nativeCanvas; + M.srcKind = 'none'; + updateSource(); computeTexDims(); if (!M.texW || !M.texH) { setErr('bad canvas size'); log('create: bad canvas size'); destroy(); return null; } @@ -890,13 +1078,15 @@ var layer = attachLayer(session); if (!layer) throw new Error('layer creation returned null'); + M.needRebuild = false; // the source was already resolved above sample(); // seed bbox / hasInk before the first XR frame M.dirty = true; setErr(null); updateStats(); log('subtitles layer created:', M.mode, M.texW + 'x' + M.texH, - 'alpha=' + M.alpha, 'canvas=' + canvas.width + 'x' + canvas.height); + 'alpha=' + M.alpha, 'source=' + M.srcKind, + 'canvas=' + (M.canvas ? M.canvas.width + 'x' + M.canvas.height : 'none')); return layer; } catch (e) { setErr((e && e.message) || e); @@ -922,6 +1112,7 @@ } catch (e) {} M.gl = null; M.glCanvas = null; M.prog = null; M.tex = null; M.fbo = null; M.binding = null; M.session = null; M.canvas = null; + M.srcKind = 'none'; M.nativeText = null; M.sampleCanvas = null; M.sampleCtx = null; M.cropCanvas = null; M.cropCtx = null; M.srcW = 0; M.srcH = 0; M.halfW = 1; M.fullW = 1; M.eyeW = 1; M.texW = 0; M.texH = 0; @@ -1071,7 +1262,8 @@ if (hasInk) { M.centerInk = centerInk; var dual = M.dualEye; - if (M.layout === 'dual') dual = true; + if (M.srcKind === 'native') { M.copyND = null; dual = false; } // our own canvas is always mono + else if (M.layout === 'dual') dual = true; else if (M.layout === 'mono') dual = false; else if (hasLeft && hasRight) { if (centerInk > 0) { M.copyND = null; dual = false; } @@ -1227,7 +1419,11 @@ : (frame.predictedDisplayTime != null ? frame.predictedDisplayTime : (window.performance ? performance.now() : Date.now())); - if (t - M.lastSampleT >= SAMPLE_MS || M.lastSampleT < 0) { + // Keep the source current every frame (native cue text changes on its own). + var hasSrc = updateSource(); + if (!hasSrc) { + if (M.hasInk) { M.hasInk = false; M.bbox = null; M.dirty = true; } + } else if (t - M.lastSampleT >= SAMPLE_MS || M.lastSampleT < 0) { M.lastSampleT = t; sample(); } @@ -1532,7 +1728,8 @@ setOpacity: setOpacity, getOpacity: getOpacity, getLayer: getLayer, - getMode: getMode + getMode: getMode, + getSourceKind: function () { return M.srcKind; } }; })(); @@ -4369,7 +4566,9 @@ S.subOpacity = num(o.subOpacity, 0.1, 1.0, S.subOpacity); S.ctlAutoHideMs = num(o.ctlAutoHideMs, 1000, 10000, S.ctlAutoHideMs); if (o.subMode === 'hud' || o.subMode === 'sphere') S.subMode = o.subMode; - if (o.subLayout === 'auto' || o.subLayout === 'dual' || o.subLayout === 'mono') S.subLayout = o.subLayout; + // subLayout is deliberately NOT restored: a persisted manual override + // silently ruined the default dual-eye content once already (forced 'mono' + // made the HUD show both eyes side by side). Layout always starts on 'auto'. // ADDENDUM 8: how the picture aim is applied. 'space' is the live path; the // 'rebuild' escape hatch exists because mutating a media layer's transform is // ignored by this runtime, so if an offset reference space turns out not to be @@ -4404,7 +4603,6 @@ subAngleDeg: S.subAngleDeg, subOpacity: S.subOpacity, subMode: S.subMode, - subLayout: S.subLayout, subAlpha: S.subAlpha, projection: S.projection, stereo: S.stereo, @@ -6005,6 +6203,7 @@ // `subtitles` and `subMode` could otherwise leave the layer uncreated. S.subtitles = true; if (S.subMode === 'off') S.subMode = 'hud'; + S.subLayout = 'auto'; // ADDENDUM 10: the override is per-session only try { if (NS.Subtitles && NS.Subtitles.init) NS.Subtitles.init(); } catch (e) { NS.recordError('Subtitles.init', e); } try { if (NS.Controls && NS.Controls.init) NS.Controls.init(); } catch (e) { NS.recordError('Controls.init', e); } try { if (NS.Hands && NS.Hands.init) NS.Hands.init(); } catch (e) { NS.recordError('Hands.init', e); } diff --git a/src/00-core.js b/src/00-core.js index 762a0b3..df393f2 100644 --- a/src/00-core.js +++ b/src/00-core.js @@ -11,7 +11,7 @@ var NS = window.__JFVR = window.__JFVR || {}; - NS.version = 'hud-8'; // hud-8 = ADDENDUM 9 (rAF broker always-on so the libass loop survives XR entry; auto dual/mono subtitle layout) + NS.version = 'hud-9'; // hud-9 = ADDENDUM 10 (native/TextTrack subtitles painted by the HUD; layout override per-session, ADDENDUM 9 rAF broker + auto dual/mono layout) NS.__core = true; /* ------------------------------------------------------------------ * diff --git a/src/10-subtitles.js b/src/10-subtitles.js index c0ed1c9..1b74701 100644 --- a/src/10-subtitles.js +++ b/src/10-subtitles.js @@ -72,6 +72,15 @@ var CENTER_BAND_PX = 40; // source px around the eye seam that count as "centre ink" var COPY_TOL = 0.25; // column-profile copy distance for "two copies of one line" // (measured: dual 0.02-0.08, mono 1.00) + // ADDENDUM 10 — Jellyfin only routes ssa/ass through SubtitlesOctopus (the + // libass canvas). Every other text codec (subrip, vtt, ...) is drawn by the + // BROWSER from a TextTrack (`video.addTextTrack` + mode 'showing'), so there is + // no canvas at all to sample. The HUD then paints the active cue itself into a + // private canvas and feeds it through the same crop/upload pipeline. + var NATIVE_W = 1920, NATIVE_H = 1080; + var NATIVE_FONT_H = 0.052; // cue font size as a fraction of the canvas height + var NATIVE_LINE_H = 1.32; // line advance + var NATIVE_MAX_W = 0.9; // max text width as a fraction of the canvas width var SPHERE_TEX_W = 4096; // legacy equirect layer size var SPHERE_TEX_H = 2048; var OPACITY_DEFAULT = 0.85; // dim the strip below full opacity by default @@ -148,6 +157,9 @@ copyND: null, // diagnostics: normalised column-profile copy distance centerInk: null, // diagnostics: ink pixels in the seam band colL: null, colR: null, // reused column-profile scratch (half width each) + // ADDENDUM 10: where the HUD's source pixels come from. + srcKind: 'none', // 'libass' | 'native' | 'none' + nativeCanvas: null, nativeCtx: null, nativeText: null, bbox: null, lastSample: null, hasInk: false, @@ -168,6 +180,174 @@ catch (e) { return null; } } + /* ================================================================ + * ADDENDUM 10 — native (non-libass) subtitle source + * Jellyfin asks the browser to render subrip/vtt cues from a TextTrack, so the + * HUD has no canvas to read. Read the active cue, paint it into a private + * canvas, and let the normal crop/upload path take over from there. + * ============================================================== */ + function ensureNativeCanvas() { + if (!M.nativeCanvas) { + try { + M.nativeCanvas = document.createElement('canvas'); + M.nativeCanvas.width = NATIVE_W; + M.nativeCanvas.height = NATIVE_H; + M.nativeCtx = M.nativeCanvas.getContext('2d'); + // Kept in the DOM but hidden: Canvas2D works detached too, and this makes + // the exact pixels the HUD uploads inspectable from DevTools. + M.nativeCanvas.className = 'jfvr-native-subs'; + M.nativeCanvas.style.display = 'none'; + (document.body || document.documentElement).appendChild(M.nativeCanvas); + } catch (e) { M.nativeCtx = null; } + } + return !!M.nativeCtx; + } + + // Jellyfin renders non-ASS text subtitles one of two ways depending on the + // build: into a TextTrack (video.addTextTrack + mode 'showing', the browser + // shows it through ::cue) or into a DOM overlay (div.videoSubtitlesInner). + // Both leave no canvas behind. + function hasNativeTracks() { + try { + var v = S.video; + if (v && v.textTracks) { + for (var i = 0; i < v.textTracks.length; i++) { + var tr = v.textTracks[i]; + if (tr && tr.mode === 'showing' && tr.cues && tr.cues.length) return true; + } + } + if (document.querySelector('.videoSubtitlesInner')) return true; + } catch (e) {} + return false; + } + + function activeNativeText() { + var out = []; + try { + var v = S.video; + if (v && v.textTracks) { + for (var i = 0; i < v.textTracks.length; i++) { + var tr = v.textTracks[i]; + if (!tr || tr.mode !== 'showing') continue; + var cues = tr.activeCues; // maintained by the browser + if (!cues) continue; + for (var j = 0; j < cues.length; j++) { + var s = cues[j] && cues[j].text; + if (s) out.push(s); + } + } + } + if (!out.length) { + var d = document.querySelector('.videoSubtitlesInner, .videoSubtitles'); + if (d) { var t = d.innerText || ''; if (t) out.push(t); } + } + } catch (e) { return ''; } + return out.join('\n'); + } + + function cleanCueText(s) { + return String(s == null ? '' : s) + .replace(/\{[^}]*\}/g, '') // ASS override tags + .replace(/<[^>]*>/g, '') // html-ish tags + .replace(/ /g, ' ').replace(/&/g, '&') + .replace(/</g, '<').replace(/>/g, '>') + .replace(/[\u200e\u200f\u202a-\u202e]/g, '') // bidi control marks + .replace(/\r/g, '') + .replace(/[ \t]+\n/g, '\n') + .replace(/\n{3,}/g, '\n\n') + .trim(); + } + + function wrapCueLines(ctx, text, maxW) { + var out = []; + var paras = text.split('\n'); + for (var p = 0; p < paras.length; p++) { + var para = paras[p]; + if (!para) { out.push(''); continue; } + var line = ''; + for (var i = 0; i < para.length; i++) { + var ch = para[i]; + if (line && ctx.measureText(line + ch).width > maxW) { + var sp = line.lastIndexOf(' '); + if (sp > 0 && /[A-Za-z0-9]/.test(line.charAt(line.length - 1))) { + out.push(line.slice(0, sp)); + line = line.slice(sp + 1); + } else { + out.push(line); + line = ''; + } + } + line += ch; + } + out.push(line); + } + return out.length ? out : ['']; + } + + function drawNativeCue(text) { + var c = M.nativeCanvas, ctx = M.nativeCtx; + if (!c || !ctx) return; + var W = c.width, H = c.height; + ctx.clearRect(0, 0, W, H); + if (!text) return; + var fs = Math.max(12, Math.round(H * NATIVE_FONT_H)); + ctx.font = '600 ' + fs + 'px "Noto Sans SC","Noto Sans CJK SC","PingFang SC",' + + '"Microsoft YaHei",system-ui,Arial,sans-serif'; + ctx.textAlign = 'center'; + ctx.textBaseline = 'middle'; + ctx.lineJoin = 'round'; + ctx.miterLimit = 2; + var lines = wrapCueLines(ctx, text, W * NATIVE_MAX_W); + var lh = fs * NATIVE_LINE_H; + var y0 = H / 2 - (lines.length - 1) * lh / 2; + ctx.lineWidth = Math.max(2, Math.round(fs * 0.1)); + ctx.strokeStyle = 'rgba(0,0,0,0.95)'; + ctx.fillStyle = '#ffffff'; + for (var i = 0; i < lines.length; i++) { + ctx.strokeText(lines[i], W / 2, y0 + i * lh); + ctx.fillText(lines[i], W / 2, y0 + i * lh); + } + } + + // Decide, per frame, which canvas the HUD samples. libass wins when Jellyfin + // created one; otherwise the native cue canvas. Returns false when there is no + // subtitle source at all. + function updateSource() { + var lib = getCanvas(); + if (lib && lib.width > 0 && lib.height > 0) { + if (M.srcKind !== 'libass') { + M.srcKind = 'libass'; + M.canvas = lib; + M.nativeText = null; + M.lastSample = null; M.hasInk = false; M.bbox = null; + computeTexDims(); + M.needRebuild = true; + } + return true; + } + if (!hasNativeTracks()) { + if (M.srcKind !== 'none') { M.srcKind = 'none'; M.hasInk = false; M.bbox = null; M.dirty = true; } + return false; + } + if (!ensureNativeCanvas()) return false; + var text = cleanCueText(activeNativeText()); + if (text !== M.nativeText) { + M.nativeText = text; + drawNativeCue(text); + M.dirty = true; + } + if (M.srcKind !== 'native') { + M.srcKind = 'native'; + M.canvas = M.nativeCanvas; + M.dualEye = false; // our own canvas is always one centred line + M.mismatch = 0; + M.lastSample = null; M.hasInk = false; M.bbox = null; + computeTexDims(); + M.needRebuild = true; + } + return true; + } + function getHolder(video) { var holder = null; try { holder = document.querySelector('.videoPlayerContainer'); } catch (e) {} @@ -182,6 +362,8 @@ st.subLayer = !!M.layer; st.subPrepared = !!M.prepared; st.subCanvasSize = M.canvas ? (M.canvas.width + 'x' + M.canvas.height) : null; + st.subSource = M.srcKind; + st.subNativeText = M.nativeText ? M.nativeText.slice(0, 60) : null; st.subLayout = M.layout; st.subDual = M.dualEye; st.subEyeW = M.eyeW; @@ -211,6 +393,10 @@ * ============================================================== */ function prepare() { if (M.prepared) return; // idempotent via a live flag, not a backup check + // Only libass needs the box blown up so it re-renders bigger. The native cue + // canvas is ours, so resizing the player would be pointless (and would move + // the browser's own cue overlay around). + if (!getCanvas()) { log('prepare skipped: native subtitle source'); return; } var video = S.video || null; var holder = getHolder(video); try { @@ -457,10 +643,6 @@ if (M.mode === 'off') { log('create: subtitle mode off'); return null; } - var canvas = getCanvas(); - if (!canvas || !canvas.width || !canvas.height) { - setErr('no libass canvas'); log('create: no '); return null; - } if (typeof window.XRWebGLBinding !== 'function') { setErr('no XRWebGLBinding'); log('create: no XRWebGLBinding'); return null; } @@ -472,11 +654,17 @@ var gl = NS.xrGl(true); if (!gl) { setErr('no webgl2'); log('create: no webgl2 context'); return null; } - M.canvas = canvas; M.gl = gl; M.glCanvas = gl.canvas; M.uploads = 0; M.samples = 0; + // ADDENDUM 10: the source may be the libass canvas OR our own cue canvas. + // Start on the native canvas so create() works even before Jellyfin has + // created its renderer, then updateSource() upgrades to libass if it shows up. + ensureNativeCanvas(); + M.canvas = M.nativeCanvas; + M.srcKind = 'none'; + updateSource(); computeTexDims(); if (!M.texW || !M.texH) { setErr('bad canvas size'); log('create: bad canvas size'); destroy(); return null; } @@ -523,13 +711,15 @@ var layer = attachLayer(session); if (!layer) throw new Error('layer creation returned null'); + M.needRebuild = false; // the source was already resolved above sample(); // seed bbox / hasInk before the first XR frame M.dirty = true; setErr(null); updateStats(); log('subtitles layer created:', M.mode, M.texW + 'x' + M.texH, - 'alpha=' + M.alpha, 'canvas=' + canvas.width + 'x' + canvas.height); + 'alpha=' + M.alpha, 'source=' + M.srcKind, + 'canvas=' + (M.canvas ? M.canvas.width + 'x' + M.canvas.height : 'none')); return layer; } catch (e) { setErr((e && e.message) || e); @@ -555,6 +745,7 @@ } catch (e) {} M.gl = null; M.glCanvas = null; M.prog = null; M.tex = null; M.fbo = null; M.binding = null; M.session = null; M.canvas = null; + M.srcKind = 'none'; M.nativeText = null; M.sampleCanvas = null; M.sampleCtx = null; M.cropCanvas = null; M.cropCtx = null; M.srcW = 0; M.srcH = 0; M.halfW = 1; M.fullW = 1; M.eyeW = 1; M.texW = 0; M.texH = 0; @@ -704,7 +895,8 @@ if (hasInk) { M.centerInk = centerInk; var dual = M.dualEye; - if (M.layout === 'dual') dual = true; + if (M.srcKind === 'native') { M.copyND = null; dual = false; } // our own canvas is always mono + else if (M.layout === 'dual') dual = true; else if (M.layout === 'mono') dual = false; else if (hasLeft && hasRight) { if (centerInk > 0) { M.copyND = null; dual = false; } @@ -860,7 +1052,11 @@ : (frame.predictedDisplayTime != null ? frame.predictedDisplayTime : (window.performance ? performance.now() : Date.now())); - if (t - M.lastSampleT >= SAMPLE_MS || M.lastSampleT < 0) { + // Keep the source current every frame (native cue text changes on its own). + var hasSrc = updateSource(); + if (!hasSrc) { + if (M.hasInk) { M.hasInk = false; M.bbox = null; M.dirty = true; } + } else if (t - M.lastSampleT >= SAMPLE_MS || M.lastSampleT < 0) { M.lastSampleT = t; sample(); } @@ -1165,6 +1361,7 @@ setOpacity: setOpacity, getOpacity: getOpacity, getLayer: getLayer, - getMode: getMode + getMode: getMode, + getSourceKind: function () { return M.srcKind; } }; })(); diff --git a/src/90-main.js b/src/90-main.js index f9a9593..dcc0920 100644 --- a/src/90-main.js +++ b/src/90-main.js @@ -170,7 +170,9 @@ S.subOpacity = num(o.subOpacity, 0.1, 1.0, S.subOpacity); S.ctlAutoHideMs = num(o.ctlAutoHideMs, 1000, 10000, S.ctlAutoHideMs); if (o.subMode === 'hud' || o.subMode === 'sphere') S.subMode = o.subMode; - if (o.subLayout === 'auto' || o.subLayout === 'dual' || o.subLayout === 'mono') S.subLayout = o.subLayout; + // subLayout is deliberately NOT restored: a persisted manual override + // silently ruined the default dual-eye content once already (forced 'mono' + // made the HUD show both eyes side by side). Layout always starts on 'auto'. // ADDENDUM 8: how the picture aim is applied. 'space' is the live path; the // 'rebuild' escape hatch exists because mutating a media layer's transform is // ignored by this runtime, so if an offset reference space turns out not to be @@ -205,7 +207,6 @@ subAngleDeg: S.subAngleDeg, subOpacity: S.subOpacity, subMode: S.subMode, - subLayout: S.subLayout, subAlpha: S.subAlpha, projection: S.projection, stereo: S.stereo, @@ -1806,6 +1807,7 @@ // `subtitles` and `subMode` could otherwise leave the layer uncreated. S.subtitles = true; if (S.subMode === 'off') S.subMode = 'hud'; + S.subLayout = 'auto'; // ADDENDUM 10: the override is per-session only try { if (NS.Subtitles && NS.Subtitles.init) NS.Subtitles.init(); } catch (e) { NS.recordError('Subtitles.init', e); } try { if (NS.Controls && NS.Controls.init) NS.Controls.init(); } catch (e) { NS.recordError('Controls.init', e); } try { if (NS.Hands && NS.Hands.init) NS.Hands.init(); } catch (e) { NS.recordError('Hands.init', e); }