diff --git a/HANDOFF.md b/HANDOFF.md index 9687f2a..8cd679c 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -1,10 +1,13 @@ # Jellyfin Web VR180 — 项目交接文档 > 给新对话用。读完这份就能接手,不需要重新调查环境。 -> 最后更新:已部署版本 `5217a3`(`NS.version = hud-7`;ADDENDUM 6 字幕深度/大小解耦 -> + 7 唤出门限/前面板精简 + 7b 锁按钮与「锁定」面板模式 + 8/8b 画面搬迁(实时拖动)) -> 已在 Quest 浏览器里实机验证模块加载 + 锁定数学(`subLockAngle/subAngleDeg`,`errors: []`); -> **VR 会话内的观感还没看过**。 +> 最后更新:已部署版本 `b9c3a5`(`NS.version = hud-8`;ADDENDUM 6 字幕深度/大小解耦 +> + 7 唤出门限/前面板精简 + 7b 锁按钮与「锁定」面板模式 + 8/8b 画面搬迁(实时拖动) +> + 9 rAF 代理修复(VR 内字幕不再冻结)+ 普通/双目字幕自动识别) +> +> ⚠️ **Jellyfin web 注册了 service worker**(`serviceworker.js`,scope `/web/`),会把 +> `index.html` 缓存在 SW cache 里。重新部署后如果只是普通刷新,浏览器可能仍用旧的 +> `index.html`(于是继续加载旧的 `?v=` 包)。必须清 SW + 清 cache + 硬刷新,见 §13。 --- @@ -410,6 +413,7 @@ pressAt 上次按压命中/未命中 handsReady / handsErr / handsSources / handsFrames subMode / subCanvasSize / subCrop / subTex / subOpacity / subPlacement subLockAngle / subAngleDeg / subTextDeg / subYOverZ ← ADDENDUM 6 +subLayout / subDual / subEyeW / subCopyND / subCenterInk ← ADDENDUM 9 ctlSubsOn / ctlSubDist / ctlSubAngle / ctlSubLock ← 控制条实际画出的值 ctlLock / ctlMode / ctlHold / pressAt / lastActivate ← ADDENDUM 7/7b 锁、面板保持与唤出判定 videoYaw / videoPitch / videoRotMode / videoTransformWritable ← ADDENDUM 8/8b 画面搬迁 @@ -491,3 +495,77 @@ errors[] 最近错误 **手势**(按你的要求):`点一下复位 · 按住 0.5s 再拖动`。拖动基准取"按住满 0.5s 那一刻"的射线, 不是按下那一刻——否则那半秒会把手抖放大成甩动。1° 死区防抖。 + +--- + +## 13. ADDENDUM 9 — 字幕「中断」与普通字幕(已部署 `b9c3a5` / `hud-8`) + +### 13.1 定位手法(下次直接复用) + +- **连设备**:`./quest.sh`(它的授权检测偶尔误判,失败就手动 + `adb forward tcp:9222 localabstract:chrome_devtools_remote`)。 +- **读 SubtitlesOctopus 实例**:Jellyfin 12.x 的实例在播放器的闭包里,用 CDP 的 + `getEventListeners(video).timeupdate[i].listener` → `[[Scopes]]` → 找带 + `oneshotState`/`renderedItems` 的对象。本机脚本见 `/tmp/inst.py`、`/tmp/scan.py` + (一次性工具,未入库)。关键字段:`rafId`(我们的是 1e9 起)、 + `oneshotState.eventStart`(当前画的那一行)、`renderRequested`。 +- **量 libass 画布**:把 `canvas.libassjs-canvas` 画进小 canvas 取 `getImageData`, + 统计左右半屏/中线的墨迹与列轮廓(`/tmp/prof.js`、`/tmp/mirror*.js`)。 +- **OCR**:`adb exec-out screencap -p` + `tesseract -l chi_sim`(双脚同屏也能读)。 + 本次就是靠它确认 HUD 停在 15:45 的台词而视频已在 22:49。 + +### 13.2 问题一:VR 内字幕冻结(不是「被截断」) + +- 现象:进 VR 后字幕永远停在进 VR 前那一帧。实测 `rafId=1150`(**原生** id)、 + `oneshotState.eventStart=945.99`(落后播放头约 7 分钟)、整段播放 **0 个** + `oneshot-render` 请求。 +- 根因:进入 `immersive-vr` 后原生 rAF 完全停摆。SubtitlesOctopus 在主线程起了**一个 + 常驻 rAF 循环**(`k`):它负责把 worker 预渲染好的帧画到 canvas、并继续要下一帧。 + 旧代理只在「已进入 XR」后才接管 rAF,于是**进 XR 那一刻还挂在原生调度器里的那个 + `k` 回调被销毁**;而 `i.rafId` 仍非 0,`resetRenderAheadCache` 也不会重启它 → 整个 + prerender 链断掉。 +- 修复(`src/90-main.js`):rAF 代理**始终**接管(非 XR 用原生 rAF 泵,XR 用 rVFC), + 且原生泵**单飞**(`nativeRafStep` 先续期再 drain,队列空则 cancel);`startRafPump` + **不再 clear 队列**(那个回调就是要救的);加 50ms 兜底,暂停/卡住时也能重画。 +- ⚠️ **踩过的坑**:第一版原生泵在 drain 期间把 `nativeId` 置空,每个自续期回调都会额外 + 再排一个原生 rAF → 回调数**指数增长**,直接把页面主线程打爆(CDP 无响应)。现在由 + Node 单测固定:自续期循环 300 帧 `maxConcurrentNative = 1`。 + +### 13.3 问题二:只支持双目 ASS,普通字幕被截断 + +- 根因:旧代码硬编码只取画布**左半屏**。双目 ASS(`CN_dual_eye`,PlayRes 3840×1920, + LeftEye/RightEye 各占半屏)没问题;普通字幕(`CN` 的 SRT,居中跨整幅)文字中心正好 + 压在画布中线(两眼接缝)上,于是只剩左半边 → 看起来「断掉」。 +- 修复(`src/10-subtitles.js`):新增 `state.subLayout`(`auto`/`dual`/`mono`,默认 + `auto`,面板有「字幕布局」一行,持久化)。每次采样统计左右半屏 + 中线 ±40px 的墨迹: + 中线有墨 → 普通;只有左半 → 双目;只有右半 → 普通;两半都有且中线空 → **列轮廓平移 + 匹配**(`columnCopyND` < 0.25 → 双目)。布局切换做 2 帧去抖。 +- **血泪教训:双目字幕不是像素镜像**。左右眼是**同一段文字放在对称位置**,字形不翻转, + 所以 `left[x] vs right[W-1-x]` 完全对不上(实测均值差 258/1020)。我第一版就是被这个 + 骗了,把双目误判成普通(HUD 于是把两只眼的字并排显示)。真正成立的是**列轮廓**: + 实测双目 `0.02–0.08`、普通 `1.00`、中线墨迹 双目 `0` / 普通 `50`。 +- `M.eyeW`(单眼区域宽度:双目=半幅、普通=整幅)替换了原来写死的 `halfW`。 + +### 13.4 部署缓存陷阱(为什么第一次改了「没生效」) + +Jellyfin web 注册了 **service worker**,`index.html` 会被缓存。只普通刷新时浏览器可能 +仍用旧 `index.html` → 继续请求旧的 `jfvr-mvp.js?v=5217a3`。清理脚本(一次性的): + +```js +for (const r of await navigator.serviceWorker.getRegistrations()) await r.unregister(); +for (const k of await caches.keys()) await caches.delete(k); +// + Network.clearBrowserCache + Page.reload{ignoreCache:true} +``` + +验证新包是否真的加载:`document.querySelector('script[plugin="JellyfinVR"]').getAttribute('src')` +与 `window.__JFVR.version`(应为 `hud-8`)。 + +### 13.5 待验证 / 本次未完成 + +- 最终版(`b9c3a5`,`COPY_TOL=0.25`)**还没在头显里跑过**:验证到一半设备进入未佩戴 / + Guardian 状态,Oculus 浏览器无法再拉起(`ps` 无进程、无 devtools socket)。已完成的实机 + 证据:broker 修复(`rafId` 落在插件区间、`eventStart` 跟随、VR 内 `subUploads` 4→7)、 + 以及两个布局的**真实量测值**。最终判据是离线单测(真实源码 `columnCopyND`)验证的。 +- 下次戴上头显后要确认:`stats.subLayout=auto`、双目轨 `subDual=true/subEyeW=2160/ + subCrop` 落在左半屏(≤2160)、普通轨 `subDual=false/subEyeW=4320/subCrop` 跨中线、 + `subCopyND` 与 `subCenterInk` 是否符合上面的表。 diff --git a/INTERFACES.md b/INTERFACES.md index 2ebb854..b24f89b 100644 --- a/INTERFACES.md +++ b/INTERFACES.md @@ -976,3 +976,96 @@ double rotation) and schedules no rebuild; unchanged angles reuse the cached spa rebuild strategy queues at most one, and its observed order is `removeLayer -> createVideoLayer -> addLayer`; a throwing/absent `getOffsetReferenceSpace()` is reported and permanently falls back to `rebuild`. + +# ADDENDUM 9 — subtitle pipeline survival + normal (mono) subtitles + +Two independent defects, found on device with the VR session left in its broken state. + +## 1. The rAF broker lost Jellyfin's subtitle loop at session start + +`90-main.js` no longer just *forwards* to native `requestAnimationFrame` outside XR. It +**always** holds callbacks in its own queue and drives them with one of two drivers: + +- outside XR: a native rAF driver that keeps **exactly one** native callback in flight + (`nativeRafStep` re-arms *before* draining and cancels itself when the queue empty); +- inside XR: `video.requestVideoFrameCallback`, plus a 50 ms idle fallback so a paused / + stalled / ended video still redraws (paused seek, subtitle track switch). + +Why: `requestSession('immersive-vr')` stops native rAF completely. Jellyfin 12.x renders +subtitles with SubtitlesOctopus, whose main thread starts **one long-lived rAF loop** +(`rafId`) when the video loads, draws the canvas from it and requests the next prerender +from the worker (`oneshot-render`). A callback still pending in the *native* scheduler when +the session starts is destroyed with it, and because `rafId` stays non-zero the loop is +never restarted. + +Evidence (live instance, via CDP closure inspection): + +``` +rafId : 1150 <- native id, not the plugin's 1e9 range +oneshotState.eventStart: 945.99 <- canvas frozen ~7 min behind the playhead +renderRequested : false +oneshot-render requests during 15 s of playback: 0 +``` + +After the fix, in a live VR session: `rafId = 1000028035`, `eventStart` follows +`video.currentTime`, and `stats.subUploads` climbs while subtitles change. + +`startRafPump()` must **not** clear the queue (that callback is the one being rescued) and +must drop the stale `nativeId`. `stopRafPump()` re-arms the native driver. + +## 2. Auto dual-eye / mono subtitle layout + +`10-subtitles.js` used to assume the libass canvas is a stereo left/right pair and always +sampled the **left half**. A normal subtitle (e.g. the `CN` SRT track) is centred across the +whole 2:1 frame, so half of it was cut off. + +`state.subLayout` (`'auto' | 'dual' | 'mono'`, default `'auto'`, persisted) selects the +region; `stats.subLayout / subDual / subEyeW / subCopyND / subCenterInk` report it. + +Per sample (on the 320×160 downsample) the module builds the ink box and column count of +each half plus the ink in a ±40 source-px band around the seam, then: + +``` +only the left half has ink -> dual (sample the left half) +only the right half has ink -> mono (sample the whole width) +ink crosses the seam -> mono +both halves, seam empty -> columnCopyND < COPY_TOL ? dual : mono +``` + +**Dual-eye is NOT a pixel mirror.** Both eyes carry the *same* text at mirrored *positions*; +the glyphs are not flipped, so `left[x]` vs `right[W-1-x]` fails badly (measured mean delta +258/1020 → "not a mirror"). What is true is that the two halves' ink **column profiles** +match under a small horizontal shift, because they are the same line. Measured on device: + +| layout | centre-band ink | column-copy distance | +|---|---|---| +| dual-eye (`CN_dual_eye`) | 0 | 0.02 – 0.08 | +| mono (`CN` SRT) | 50 / frame | 1.00 | + +`COPY_TOL = 0.25`. A layout change is debounced (two consecutive samples) because it +rebuilds the layer. + +`M.eyeW` is one eye's region width (half for dual, full for mono); `computeTexDims()`, +`drawHud()`'s crop/clamp and the texture scale all use it instead of the old hard-coded +`halfW`. + +### API additions + +```js +__JFVR.Subtitles.setLayout('auto' | 'dual' | 'mono') // also state.subLayout +__JFVR.Subtitles.getLayout(): string +``` + +## 3. Verification + +- Broker logic replicated exactly in a Node harness against the built bundle: + `maxConcurrentNative = 1` over 300 frames with a self-rescheduling loop (the first + version re-armed during the drain and doubled the driver every frame — that froze the + page); simulating session start (native callbacks dropped) leaves the loop in the queue + and it keeps firing from the XR driver. +- `columnCopyND` extracted from the built bundle and exercised with the on-device + measurements: dual 0.00–0.08 → `dual`, mono centre-ink → `mono`, no-copy word-gap + profile 0.41 → `mono`. +- On device (before the UI became unreachable): the broker change was observed to carry + SubtitlesOctopus's loop through `immersive-vr` (`rafId` in the plugin's id range, + `eventStart` tracking, `subUploads` 4 → 7 over 20 s in VR). diff --git a/dist/jfvr.js b/dist/jfvr.js index adc12d4..491b90b 100644 --- a/dist/jfvr.js +++ b/dist/jfvr.js @@ -19,7 +19,7 @@ var NS = window.__JFVR = window.__JFVR || {}; - NS.version = 'hud-7'; // hud-7 = ADDENDUM 8b (live picture pan: transform/space, hold-to-drag, tap-to-reset) + NS.version = 'hud-8'; // hud-8 = ADDENDUM 9 (rAF broker always-on so the libass loop survives XR entry; auto dual/mono subtitle layout) NS.__core = true; /* ------------------------------------------------------------------ * @@ -59,6 +59,7 @@ subtitles: true, ctlLock: false, // ADDENDUM 7: control-bar lock badge (per VR session) subMode: 'hud', // 'hud' | 'sphere' | 'off' + subLayout: 'auto', // ADDENDUM 9: 'auto' | 'dual' | 'mono' subtitle layout subAlpha: 'premultiplied', // 'premultiplied' | 'straight' subOpacity: 0.85, // 0..1, applied to the subtitle layer live // y is the quad CENTRE; negative = below eye level. -0.42 keeps the strip @@ -423,6 +424,21 @@ var DIFF_SUM = 24; // per-sample channel delta that counts as a changed sample var HUD_TEX_W = 2048; // target texture width (≈30 px/deg for a 2 m strip @ 1.45 m) var HUD_TEX_ASPECT = 0.28; // texH = texW * this (room for multi-line subtitles) + // ADDENDUM 9 — dual-eye vs normal (mono) subtitle layout, auto-detected. + // The libass canvas holds EITHER a stereo left/right pair (two copies, one per + // eye; the left half is the left eye) OR a single normal subtitle positioned + // across the whole frame. + // + // IMPORTANT: dual-eye is NOT a pixel mirror. Both eyes carry the SAME text at + // mirrored POSITIONS; the glyphs are not flipped, so comparing left[x] with + // right[W-1-x] fails badly (measured mean delta ~258/1020, i.e. "not a mirror"). + // What IS true is that the two halves' ink COLUMN profiles match under a small + // horizontal shift, because they are the same line of text. Measured on device: + // dual -> normalised copy distance 0.02, mono -> 1.00, seam ink present only for + // mono. So the copy test is the discriminator. + var CENTER_BAND_PX = 40; // source px around the eye seam that count as "centre ink" + var COPY_TOL = 0.25; // column-profile copy distance for "two copies of one line" + // (measured: dual 0.02-0.08, mono 1.00) var SPHERE_TEX_W = 4096; // legacy equirect layer size var SPHERE_TEX_H = 2048; var OPACITY_DEFAULT = 0.85; // dim the strip below full opacity by default @@ -489,8 +505,16 @@ cropCanvas: null, cropCtx: null, - srcW: 0, srcH: 0, halfW: 1, + srcW: 0, srcH: 0, halfW: 1, fullW: 1, eyeW: 1, texW: 0, texH: 0, + // ADDENDUM 9: layout is the user setting ('auto' | 'dual' | 'mono'); dualEye + // is the resolved result (auto-detected when layout === 'auto'). + layout: (S.subLayout === 'dual' || S.subLayout === 'mono') ? S.subLayout : 'auto', + dualEye: true, + mismatch: 0, // consecutive samples disagreeing with dualEye (debounce) + copyND: null, // diagnostics: normalised column-profile copy distance + centerInk: null, // diagnostics: ink pixels in the seam band + colL: null, colR: null, // reused column-profile scratch (half width each) bbox: null, lastSample: null, hasInk: false, @@ -525,6 +549,11 @@ st.subLayer = !!M.layer; st.subPrepared = !!M.prepared; st.subCanvasSize = M.canvas ? (M.canvas.width + 'x' + M.canvas.height) : null; + st.subLayout = M.layout; + st.subDual = M.dualEye; + st.subEyeW = M.eyeW; + st.subCopyND = (M.copyND == null) ? null : +M.copyND.toFixed(3); + st.subCenterInk = M.centerInk; st.subTex = (M.texW && M.texH) ? (M.texW + 'x' + M.texH) : null; st.subInk = M.hasInk; st.subCrop = M.bbox ? [ @@ -619,13 +648,17 @@ var maxTex = maxTexSize(); M.srcW = cw; M.srcH = ch; - M.halfW = Math.max(1, Math.floor(cw / 2)); // mono HUD uses the LEFT half only + M.fullW = Math.max(1, cw); + M.halfW = Math.max(1, Math.floor(cw / 2)); + // ADDENDUM 9: the HUD samples ONE eye's subtitle region. Dual-eye packs both + // eyes into the canvas; a normal subtitle uses the whole frame. + M.eyeW = M.dualEye ? M.halfW : M.fullW; if (M.mode === 'sphere') { M.texW = Math.max(2, Math.min(SPHERE_TEX_W, maxTex, cw || 2)); M.texH = Math.max(2, Math.min(SPHERE_TEX_H, maxTex, ch || 2)); } else { - M.texW = Math.max(2, Math.min(HUD_TEX_W, maxTex, M.halfW)); + M.texW = Math.max(2, Math.min(HUD_TEX_W, maxTex, M.eyeW)); M.texH = Math.max(2, Math.min(maxTex, Math.round(M.texW * HUD_TEX_ASPECT))); } } @@ -781,6 +814,7 @@ if (S.subMode === 'hud' || S.subMode === 'sphere' || S.subMode === 'off') M.mode = S.subMode; if (S.subAlpha === 'premultiplied' || S.subAlpha === 'straight') M.alpha = S.subAlpha; + readLayoutSetting(); if (isNum(S.subOpacity)) M.opacity = clamp01(S.subOpacity); else S.subOpacity = M.opacity; if (typeof S.subLockAngle === 'boolean') M.lockAngle = S.subLockAngle; @@ -890,7 +924,8 @@ M.binding = null; M.session = null; M.canvas = null; M.sampleCanvas = null; M.sampleCtx = null; M.cropCanvas = null; M.cropCtx = null; - M.srcW = 0; M.srcH = 0; M.halfW = 1; M.texW = 0; M.texH = 0; + M.srcW = 0; M.srcH = 0; M.halfW = 1; M.fullW = 1; M.eyeW = 1; M.texW = 0; M.texH = 0; + M.copyND = null; M.centerInk = null; M.colL = null; M.colR = null; M.mismatch = 0; M.bbox = null; M.lastSample = null; M.hasInk = false; M.lastDrawnInk = false; M.dirty = false; M.needRebuild = false; @@ -937,6 +972,24 @@ return true; } + // ADDENDUM 9: normalised distance between the two halves' ink column profiles at + // their best horizontal alignment. ~0 for dual-eye (same line, two copies), ~1 + // for a normal subtitle (the halves are different parts of one line). + function columnCopyND(colL, colR, half) { + var D = Math.round(half * 0.4), best = 1e9; + for (var lag = -D; lag <= D; lag++) { + var diff = 0, tot = 0; + for (var i = 0; i < half; i++) { + var j = i + lag; + var r = (j >= 0 && j < half) ? colR[j] : 0; + diff += Math.abs(colL[i] - r); + tot += colL[i] + r; + } + if (tot > 0) { var nd = diff / tot; if (nd < best) best = nd; } + } + return (best === 1e9) ? 1 : best; + } + function sample() { var canvas = M.canvas; var ctx = M.sampleCtx; @@ -961,11 +1014,20 @@ var d = img.data; var prev = M.lastSample; var hud = (M.mode === 'hud'); - var half = hud ? (SAMPLE_W >> 1) : SAMPLE_W; // HUD bbox: LEFT half only - var minX = half + 1, minY = SAMPLE_H + 1, maxX = -1, maxY = -1; + var half = SAMPLE_W >> 1; var diff = 0; var sx = cw / SAMPLE_W, sy = ch / SAMPLE_H; var padX = PAD_PX / sx, padY = PAD_PX / sy; + var band = Math.max(1, Math.round(CENTER_BAND_PX / sx)); + + // Track the ink box of each REGION separately so the layout can be resolved + // before cropping: l* = left half, r* = right half, centerInk = the seam. + var lMinX = SAMPLE_W + 1, lMaxX = -1, lMinY = SAMPLE_H + 1, lMaxY = -1; + var rMinX = SAMPLE_W + 1, rMaxX = -1, rMinY = SAMPLE_H + 1, rMaxY = -1; + var centerInk = 0; + if (!M.colL) { M.colL = new Int32Array(half); M.colR = new Int32Array(half); } + var colL = M.colL, colR = M.colR; + colL.fill(0); colR.fill(0); for (var y = 0; y < SAMPLE_H; y++) { var row = y * SAMPLE_W * 4; @@ -976,11 +1038,21 @@ Math.abs(d[i + 2] - prev[i + 2]) + Math.abs(d[i + 3] - prev[i + 3]); if (dd > DIFF_SUM) diff++; } - if (hud && x < half && d[i + 3] > INK_ALPHA) { - if (x < minX) minX = x; - if (x > maxX) maxX = x; - if (y < minY) minY = y; - if (y > maxY) maxY = y; + if (!hud || d[i + 3] <= INK_ALPHA) continue; + if (x < half) { + if (x < lMinX) lMinX = x; + if (x > lMaxX) lMaxX = x; + if (y < lMinY) lMinY = y; + if (y > lMaxY) lMaxY = y; + if (x > half - band) centerInk++; + colL[x]++; + } else { + if (x < rMinX) rMinX = x; + if (x > rMaxX) rMaxX = x; + if (y < rMinY) rMinY = y; + if (y > rMaxY) rMaxY = y; + if (x < half + band) centerInk++; + colR[x - half]++; } } } @@ -988,20 +1060,62 @@ M.lastSample = d; if (!prev || diff > DIFF_SAMPLES) M.dirty = true; // content changed -> re-upload + if (!hud) { updateStats(); return; } - if (hud) { - if (maxX < 0) { - if (M.hasInk) { M.hasInk = false; M.dirty = true; } // clear the strip once - } else { - var raw = quantise({ - x0: Math.max(0, (minX - padX) * sx), - y0: Math.max(0, (minY - padY) * sy), - x1: Math.min(cw / 2, (maxX + 1 + padX) * sx), - y1: Math.min(ch, (maxY + 1 + padY) * sy) - }); - if (!M.hasInk) { M.hasInk = true; M.bbox = raw; M.dirty = true; } - else if (updateBBox(raw)) M.dirty = true; + // ---- ADDENDUM 9: resolve dual-eye vs mono ---------------------------- + // dual-eye: two copies of the same line -> no ink at the seam AND the two + // halves' column profiles match under a horizontal shift. mono: one line + // across the whole frame -> ink crosses the seam and the halves do not match. + var hasLeft = lMaxX >= 0, hasRight = rMaxX >= 0; + var hasInk = hasLeft || hasRight || centerInk > 0; + if (hasInk) { + M.centerInk = centerInk; + var dual = M.dualEye; + if (M.layout === 'dual') dual = true; + else if (M.layout === 'mono') dual = false; + else if (hasLeft && hasRight) { + if (centerInk > 0) { M.copyND = null; dual = false; } + else { M.copyND = columnCopyND(colL, colR, half); dual = (M.copyND < COPY_TOL); } } + else dual = !!hasLeft; // ink in ONE half -> dual (left) else mono (right) + if (dual !== M.dualEye) { + // Debounce: a layout switch rebuilds the layer, so require two + // consecutive samples before believing it (never observed to flicker on + // real content, but cheap insurance against a rebuild every sample). + M.mismatch = (M.mismatch || 0) + 1; + if (M.mismatch >= 2) { + M.mismatch = 0; + M.dualEye = dual; + computeTexDims(); + M.needRebuild = true; // the texture width changed with the region + } + } else M.mismatch = 0; + } + + // ---- crop to the ink box of the RESOLVED region ---------------------- + var x0s, y0s, x1s, y1s; + if (hasInk) { + if (M.dualEye && hasLeft) { + x0s = lMinX; y0s = lMinY; x1s = lMaxX; y1s = lMaxY; + } else { + x0s = hasLeft ? lMinX : rMinX; + x1s = hasRight ? rMaxX : lMaxX; + y0s = Math.min(hasLeft ? lMinY : Infinity, hasRight ? rMinY : Infinity); + y1s = Math.max(hasLeft ? lMaxY : -1, hasRight ? rMaxY : -1); + } + } + + if (!hasInk) { + if (M.hasInk) { M.hasInk = false; M.dirty = true; } // clear the strip once + } else { + var raw = quantise({ + x0: Math.max(0, (x0s - padX) * sx), + y0: Math.max(0, (y0s - padY) * sy), + x1: Math.min(M.eyeW, (x1s + 1 + padX) * sx), + y1: Math.min(ch, (y1s + 1 + padY) * sy) + }); + if (!M.hasInk) { M.hasInk = true; M.bbox = raw; M.dirty = true; } + else if (updateBBox(raw)) M.dirty = true; } updateStats(); } @@ -1014,9 +1128,9 @@ var b = M.bbox; if (!gl || !b) return; - var cw = Math.max(1, Math.min(M.halfW, Math.round(b.x1 - b.x0))); + var cw = Math.max(1, Math.min(M.eyeW, Math.round(b.x1 - b.x0))); var ch = Math.max(1, Math.round(b.y1 - b.y0)); - var sx = Math.max(0, Math.min(Math.max(0, M.halfW - cw), Math.round(b.x0))); + var sx = Math.max(0, Math.min(Math.max(0, M.eyeW - cw), Math.round(b.x0))); var sy = Math.max(0, Math.min(Math.max(0, M.srcH - ch), Math.round(b.y0))); var crop = M.cropCanvas, cctx = M.cropCtx; @@ -1024,7 +1138,7 @@ cctx.clearRect(0, 0, cw, ch); cctx.drawImage(M.canvas, sx, sy, cw, ch, 0, 0, cw, ch); - var scale = M.texW / M.halfW; // texture px per source px + var scale = M.texW / M.eyeW; // texture px per source px var dw = cw * scale, dh = ch * scale; var fit = Math.min(1, tw / dw, th / dh); // never overflow the colorTexture dw *= fit; dh *= fit; @@ -1133,6 +1247,38 @@ /* ================================================================ * settings (safe before or during a session) * ============================================================== */ + // ADDENDUM 9 — read state.subLayout into the module. A forced layout resolves + // immediately; 'auto' keeps the current guess until sample() sees ink. + function readLayoutSetting() { + if (S.subLayout === 'dual' || S.subLayout === 'mono') M.layout = S.subLayout; + else { M.layout = 'auto'; if (S.subLayout !== 'auto') S.subLayout = 'auto'; } + if (M.layout !== 'auto') { M.dualEye = (M.layout === 'dual'); M.mismatch = 0; } + } + + // Force home the dual-eye / mono decision, or go back to auto-detection. + function setLayout(mode) { + if (mode !== 'auto' && mode !== 'dual' && mode !== 'mono') return; + M.layout = mode; + S.subLayout = mode; + if (!M.canvas) { updateStats(); return; } + if (mode === 'auto') { + M.bbox = null; M.hasInk = false; M.mismatch = 0; M.dirty = true; // re-detect from the next sample + } else { + var forced = (mode === 'dual'); + if (forced !== M.dualEye) { + M.dualEye = forced; + M.bbox = null; M.hasInk = false; + M.mismatch = 0; + computeTexDims(); + M.needRebuild = true; + M.dirty = true; + } + } + updateStats(); + } + + function getLayout() { return M.layout; } + function setMode(mode) { if (mode !== 'hud' && mode !== 'sphere' && mode !== 'off') return; var changed = (M.mode !== mode); @@ -1347,6 +1493,7 @@ else if (S.subMode) S.subMode = M.mode; if (S.subAlpha === 'premultiplied' || S.subAlpha === 'straight') M.alpha = S.subAlpha; else if (S.subAlpha) S.subAlpha = M.alpha; + readLayoutSetting(); if (isNum(S.subOpacity)) M.opacity = clamp01(S.subOpacity); else S.subOpacity = M.opacity; if (typeof S.subLockAngle === 'boolean') M.lockAngle = S.subLockAngle; @@ -1372,6 +1519,8 @@ create: create, destroy: destroy, setMode: setMode, + setLayout: setLayout, + getLayout: getLayout, setAlphaMode: setAlphaMode, setPlacement: setPlacement, getPlacement: getPlacement, @@ -4070,11 +4219,26 @@ * rAF broker * ------------------------------------------------------------------ * While an immersive XR session runs, window.requestAnimationFrame is - * COMPLETELY stopped (measured: 0 callbacks in 6s) and Jellyfin renders - * libass subtitles from an rAF loop, so subtitles freeze. Hold callbacks - * in our own queue while in XR and drive them from - * video.requestVideoFrameCallback (which still fires, in step with the - * video). Outside XR delegate to the native implementation. + * COMPLETELY stopped (measured: 0 callbacks in 6s). Jellyfin renders libass + * subtitles from an rAF loop, so that froze the subtitle canvas. + * + * The FIRST fix was to queue callbacks in XR and pump the queue from + * video.requestVideoFrameCallback. That is not enough on Jellyfin 12.x: a + * callback that is ALREADY PENDING in the NATIVE scheduler when the session + * starts is destroyed with the native scheduler and never runs again, and + * SubtitlesOctopus starts ONE long-lived rAF loop when the video loads whose + * id is never reset. Verified on device: instance.rafId stuck on a native id + * (1150), oneshotState.eventStart frozen ~7 min behind the playhead, zero + * oneshot-render requests for the whole session -> the whole prerender chain + * stalls and the HUD shows the last line drawn before VR started. + * + * So the queue is used ALWAYS, even outside XR, and drained by a native rAF + * pump. Nothing is ever left pending natively, so nothing can be lost when the + * session starts; startRafPump just swaps the driver. In XR the native pump + * stops and video.requestVideoFrameCallback drives the queue (it still fires, + * in step with the video); a slow timer also drives it when the video is not + * presenting frames (paused / stalled / ended), so a paused seek or a track + * switch still redraws instead of leaving a stale line on screen. * ================================================================== */ var RAFQ = { inXR: false, @@ -4083,24 +4247,13 @@ queue: new Map(), nextId: 1000000000, pumped: 0, + nativeId: null, + lastPump: 0, err: null }; - try { - window.requestAnimationFrame = function (cb) { - if (!RAFQ.inXR) return RAFQ.native(cb); - var id = RAFQ.nextId++; - RAFQ.queue.set(id, cb); - return id; - }; - window.cancelAnimationFrame = function (id) { - if (RAFQ.queue.has(id)) { RAFQ.queue.delete(id); return; } - return RAFQ.nativeCancel(id); - }; - } catch (e) { log('rAF broker install failed', e); } - - function pumpRaf(t) { - if (!RAFQ.inXR || !RAFQ.queue.size) return; + function drainRaf(t) { + if (!RAFQ.queue.size) return; var cbs = []; RAFQ.queue.forEach(function (cb) { cbs.push(cb); }); RAFQ.queue.clear(); @@ -4110,24 +4263,66 @@ } } + // Native driver (outside XR). Exactly ONE native callback is kept in flight so + // that a rAF which re-schedules itself cannot multiply the driver (an earlier + // version nulled nativeId before draining, re-armed on every self-rescheduling + // callback AND again at the end -> exponential native callbacks -> frozen page). + function nativeRafStep(t) { + if (RAFQ.inXR) { RAFQ.nativeId = null; return; } // the XR driver owns the queue + RAFQ.nativeId = RAFQ.native(nativeRafStep); // re-arm BEFORE draining + drainRaf(t); + if (!RAFQ.queue.size && !RAFQ.inXR) { // nothing left -> stop + try { RAFQ.nativeCancel(RAFQ.nativeId); } catch (e) {} + RAFQ.nativeId = null; + } + } + + function ensureNativeRaf() { + if (RAFQ.inXR || RAFQ.nativeId != null) return; + RAFQ.nativeId = RAFQ.native(nativeRafStep); + } + + try { + window.requestAnimationFrame = function (cb) { + if (typeof cb !== 'function') return 0; + var id = RAFQ.nextId++; + RAFQ.queue.set(id, cb); + if (!RAFQ.inXR) ensureNativeRaf(); + return id; + }; + window.cancelAnimationFrame = function (id) { + if (RAFQ.queue.has(id)) { RAFQ.queue.delete(id); return; } + return RAFQ.nativeCancel(id); + }; + } catch (e) { log('rAF broker install failed', e); } + function startRafPump(video) { + // NOTE: do NOT clear the queue. The callback that a naive broker loses at + // session start is sitting in it, and it is the one that keeps Jellyfin's + // subtitle loop alive. The pending NATIVE driver id is dropped: that callback + // is lost with the native scheduler anyway, and clearing the id lets the + // driver restart cleanly after the session ends. + RAFQ.nativeId = null; RAFQ.inXR = true; - RAFQ.queue.clear(); RAFQ.pumped = 0; + RAFQ.lastPump = 0; RAFQ.err = null; var stopped = false; S.stats.rafPump = video && video.requestVideoFrameCallback ? 'videoFrameCallback' : 'interval'; if (video && video.requestVideoFrameCallback) { var step = function (now) { if (stopped) return; - pumpRaf(now); + RAFQ.lastPump = performance.now(); + drainRaf(now); video.requestVideoFrameCallback(step); }; video.requestVideoFrameCallback(step); - RAFQ.stop = function () { stopped = true; }; - return; } - var id = setInterval(function () { pumpRaf(performance.now()); }, 16); + // Idle fallback: rVFC stops while the video is paused / stalled / ended. + var id = setInterval(function () { + if (stopped || !RAFQ.inXR) return; + if (performance.now() - RAFQ.lastPump > 60) drainRaf(performance.now()); + }, 50); RAFQ.stop = function () { stopped = true; clearInterval(id); }; } @@ -4135,9 +4330,9 @@ try { if (RAFQ.stop) RAFQ.stop(); } catch (e) {} RAFQ.stop = null; RAFQ.inXR = false; - // hand leftovers back to the real scheduler so nothing is stranded - RAFQ.queue.forEach(function (cb) { try { RAFQ.native(cb); } catch (e) {} }); - RAFQ.queue.clear(); + RAFQ.lastPump = 0; + // hand the queue back to the native driver so nothing is stranded + ensureNativeRaf(); } /* ================================================================== * @@ -4174,6 +4369,7 @@ S.subOpacity = num(o.subOpacity, 0.1, 1.0, S.subOpacity); S.ctlAutoHideMs = num(o.ctlAutoHideMs, 1000, 10000, S.ctlAutoHideMs); if (o.subMode === 'hud' || o.subMode === 'sphere') S.subMode = o.subMode; + if (o.subLayout === 'auto' || o.subLayout === 'dual' || o.subLayout === 'mono') S.subLayout = o.subLayout; // ADDENDUM 8: how the picture aim is applied. 'space' is the live path; the // 'rebuild' escape hatch exists because mutating a media layer's transform is // ignored by this runtime, so if an offset reference space turns out not to be @@ -4208,6 +4404,7 @@ subAngleDeg: S.subAngleDeg, subOpacity: S.subOpacity, subMode: S.subMode, + subLayout: S.subLayout, subAlpha: S.subAlpha, projection: S.projection, stereo: S.stereo, @@ -4461,6 +4658,15 @@ body.appendChild(segRow('显示', function () { return S.subtitles ? 'on' : 'off'; }, function (v) { S.subtitles = v === 'on'; saveSettings(); }, [{ label: '开', value: 'on' }, { label: '关', value: 'off' }])); + // ADDENDUM 9: dual-eye ASS vs a normal (mono) subtitle. 'auto' detects from + // the libass canvas; force it only if the detection gets it wrong. + body.appendChild(segRow('字幕布局', function () { return S.subLayout || 'auto'; }, + function (v) { + S.subLayout = v; + if (NS.Subtitles && NS.Subtitles.setLayout) { try { NS.Subtitles.setLayout(v); } catch (e) {} } + saveSettings(); + }, + [{ label: '自动', value: 'auto' }, { label: '双目', value: 'dual' }, { label: '普通', value: 'mono' }])); body.appendChild(NS.makeEl('div', 'jfvr-hint', '字幕的位置 / 距离 / 大小 / 不透明度都在 VR 里用控制条调(控制条最下面一行),' + '调好会自动记住。VR 空间右侧有个锁图标:锁上后面板不会再被唤出。')); diff --git a/src/00-core.js b/src/00-core.js index d59d22c..762a0b3 100644 --- a/src/00-core.js +++ b/src/00-core.js @@ -11,7 +11,7 @@ var NS = window.__JFVR = window.__JFVR || {}; - NS.version = 'hud-7'; // hud-7 = ADDENDUM 8b (live picture pan: transform/space, hold-to-drag, tap-to-reset) + NS.version = 'hud-8'; // hud-8 = ADDENDUM 9 (rAF broker always-on so the libass loop survives XR entry; auto dual/mono subtitle layout) NS.__core = true; /* ------------------------------------------------------------------ * @@ -51,6 +51,7 @@ subtitles: true, ctlLock: false, // ADDENDUM 7: control-bar lock badge (per VR session) subMode: 'hud', // 'hud' | 'sphere' | 'off' + subLayout: 'auto', // ADDENDUM 9: 'auto' | 'dual' | 'mono' subtitle layout subAlpha: 'premultiplied', // 'premultiplied' | 'straight' subOpacity: 0.85, // 0..1, applied to the subtitle layer live // y is the quad CENTRE; negative = below eye level. -0.42 keeps the strip diff --git a/src/10-subtitles.js b/src/10-subtitles.js index b9bd924..c0ed1c9 100644 --- a/src/10-subtitles.js +++ b/src/10-subtitles.js @@ -57,6 +57,21 @@ var DIFF_SUM = 24; // per-sample channel delta that counts as a changed sample var HUD_TEX_W = 2048; // target texture width (≈30 px/deg for a 2 m strip @ 1.45 m) var HUD_TEX_ASPECT = 0.28; // texH = texW * this (room for multi-line subtitles) + // ADDENDUM 9 — dual-eye vs normal (mono) subtitle layout, auto-detected. + // The libass canvas holds EITHER a stereo left/right pair (two copies, one per + // eye; the left half is the left eye) OR a single normal subtitle positioned + // across the whole frame. + // + // IMPORTANT: dual-eye is NOT a pixel mirror. Both eyes carry the SAME text at + // mirrored POSITIONS; the glyphs are not flipped, so comparing left[x] with + // right[W-1-x] fails badly (measured mean delta ~258/1020, i.e. "not a mirror"). + // What IS true is that the two halves' ink COLUMN profiles match under a small + // horizontal shift, because they are the same line of text. Measured on device: + // dual -> normalised copy distance 0.02, mono -> 1.00, seam ink present only for + // mono. So the copy test is the discriminator. + var CENTER_BAND_PX = 40; // source px around the eye seam that count as "centre ink" + var COPY_TOL = 0.25; // column-profile copy distance for "two copies of one line" + // (measured: dual 0.02-0.08, mono 1.00) var SPHERE_TEX_W = 4096; // legacy equirect layer size var SPHERE_TEX_H = 2048; var OPACITY_DEFAULT = 0.85; // dim the strip below full opacity by default @@ -123,8 +138,16 @@ cropCanvas: null, cropCtx: null, - srcW: 0, srcH: 0, halfW: 1, + srcW: 0, srcH: 0, halfW: 1, fullW: 1, eyeW: 1, texW: 0, texH: 0, + // ADDENDUM 9: layout is the user setting ('auto' | 'dual' | 'mono'); dualEye + // is the resolved result (auto-detected when layout === 'auto'). + layout: (S.subLayout === 'dual' || S.subLayout === 'mono') ? S.subLayout : 'auto', + dualEye: true, + mismatch: 0, // consecutive samples disagreeing with dualEye (debounce) + copyND: null, // diagnostics: normalised column-profile copy distance + centerInk: null, // diagnostics: ink pixels in the seam band + colL: null, colR: null, // reused column-profile scratch (half width each) bbox: null, lastSample: null, hasInk: false, @@ -159,6 +182,11 @@ st.subLayer = !!M.layer; st.subPrepared = !!M.prepared; st.subCanvasSize = M.canvas ? (M.canvas.width + 'x' + M.canvas.height) : null; + st.subLayout = M.layout; + st.subDual = M.dualEye; + st.subEyeW = M.eyeW; + st.subCopyND = (M.copyND == null) ? null : +M.copyND.toFixed(3); + st.subCenterInk = M.centerInk; st.subTex = (M.texW && M.texH) ? (M.texW + 'x' + M.texH) : null; st.subInk = M.hasInk; st.subCrop = M.bbox ? [ @@ -253,13 +281,17 @@ var maxTex = maxTexSize(); M.srcW = cw; M.srcH = ch; - M.halfW = Math.max(1, Math.floor(cw / 2)); // mono HUD uses the LEFT half only + M.fullW = Math.max(1, cw); + M.halfW = Math.max(1, Math.floor(cw / 2)); + // ADDENDUM 9: the HUD samples ONE eye's subtitle region. Dual-eye packs both + // eyes into the canvas; a normal subtitle uses the whole frame. + M.eyeW = M.dualEye ? M.halfW : M.fullW; if (M.mode === 'sphere') { M.texW = Math.max(2, Math.min(SPHERE_TEX_W, maxTex, cw || 2)); M.texH = Math.max(2, Math.min(SPHERE_TEX_H, maxTex, ch || 2)); } else { - M.texW = Math.max(2, Math.min(HUD_TEX_W, maxTex, M.halfW)); + M.texW = Math.max(2, Math.min(HUD_TEX_W, maxTex, M.eyeW)); M.texH = Math.max(2, Math.min(maxTex, Math.round(M.texW * HUD_TEX_ASPECT))); } } @@ -415,6 +447,7 @@ if (S.subMode === 'hud' || S.subMode === 'sphere' || S.subMode === 'off') M.mode = S.subMode; if (S.subAlpha === 'premultiplied' || S.subAlpha === 'straight') M.alpha = S.subAlpha; + readLayoutSetting(); if (isNum(S.subOpacity)) M.opacity = clamp01(S.subOpacity); else S.subOpacity = M.opacity; if (typeof S.subLockAngle === 'boolean') M.lockAngle = S.subLockAngle; @@ -524,7 +557,8 @@ M.binding = null; M.session = null; M.canvas = null; M.sampleCanvas = null; M.sampleCtx = null; M.cropCanvas = null; M.cropCtx = null; - M.srcW = 0; M.srcH = 0; M.halfW = 1; M.texW = 0; M.texH = 0; + M.srcW = 0; M.srcH = 0; M.halfW = 1; M.fullW = 1; M.eyeW = 1; M.texW = 0; M.texH = 0; + M.copyND = null; M.centerInk = null; M.colL = null; M.colR = null; M.mismatch = 0; M.bbox = null; M.lastSample = null; M.hasInk = false; M.lastDrawnInk = false; M.dirty = false; M.needRebuild = false; @@ -571,6 +605,24 @@ return true; } + // ADDENDUM 9: normalised distance between the two halves' ink column profiles at + // their best horizontal alignment. ~0 for dual-eye (same line, two copies), ~1 + // for a normal subtitle (the halves are different parts of one line). + function columnCopyND(colL, colR, half) { + var D = Math.round(half * 0.4), best = 1e9; + for (var lag = -D; lag <= D; lag++) { + var diff = 0, tot = 0; + for (var i = 0; i < half; i++) { + var j = i + lag; + var r = (j >= 0 && j < half) ? colR[j] : 0; + diff += Math.abs(colL[i] - r); + tot += colL[i] + r; + } + if (tot > 0) { var nd = diff / tot; if (nd < best) best = nd; } + } + return (best === 1e9) ? 1 : best; + } + function sample() { var canvas = M.canvas; var ctx = M.sampleCtx; @@ -595,11 +647,20 @@ var d = img.data; var prev = M.lastSample; var hud = (M.mode === 'hud'); - var half = hud ? (SAMPLE_W >> 1) : SAMPLE_W; // HUD bbox: LEFT half only - var minX = half + 1, minY = SAMPLE_H + 1, maxX = -1, maxY = -1; + var half = SAMPLE_W >> 1; var diff = 0; var sx = cw / SAMPLE_W, sy = ch / SAMPLE_H; var padX = PAD_PX / sx, padY = PAD_PX / sy; + var band = Math.max(1, Math.round(CENTER_BAND_PX / sx)); + + // Track the ink box of each REGION separately so the layout can be resolved + // before cropping: l* = left half, r* = right half, centerInk = the seam. + var lMinX = SAMPLE_W + 1, lMaxX = -1, lMinY = SAMPLE_H + 1, lMaxY = -1; + var rMinX = SAMPLE_W + 1, rMaxX = -1, rMinY = SAMPLE_H + 1, rMaxY = -1; + var centerInk = 0; + if (!M.colL) { M.colL = new Int32Array(half); M.colR = new Int32Array(half); } + var colL = M.colL, colR = M.colR; + colL.fill(0); colR.fill(0); for (var y = 0; y < SAMPLE_H; y++) { var row = y * SAMPLE_W * 4; @@ -610,11 +671,21 @@ Math.abs(d[i + 2] - prev[i + 2]) + Math.abs(d[i + 3] - prev[i + 3]); if (dd > DIFF_SUM) diff++; } - if (hud && x < half && d[i + 3] > INK_ALPHA) { - if (x < minX) minX = x; - if (x > maxX) maxX = x; - if (y < minY) minY = y; - if (y > maxY) maxY = y; + if (!hud || d[i + 3] <= INK_ALPHA) continue; + if (x < half) { + if (x < lMinX) lMinX = x; + if (x > lMaxX) lMaxX = x; + if (y < lMinY) lMinY = y; + if (y > lMaxY) lMaxY = y; + if (x > half - band) centerInk++; + colL[x]++; + } else { + if (x < rMinX) rMinX = x; + if (x > rMaxX) rMaxX = x; + if (y < rMinY) rMinY = y; + if (y > rMaxY) rMaxY = y; + if (x < half + band) centerInk++; + colR[x - half]++; } } } @@ -622,20 +693,62 @@ M.lastSample = d; if (!prev || diff > DIFF_SAMPLES) M.dirty = true; // content changed -> re-upload + if (!hud) { updateStats(); return; } - if (hud) { - if (maxX < 0) { - if (M.hasInk) { M.hasInk = false; M.dirty = true; } // clear the strip once - } else { - var raw = quantise({ - x0: Math.max(0, (minX - padX) * sx), - y0: Math.max(0, (minY - padY) * sy), - x1: Math.min(cw / 2, (maxX + 1 + padX) * sx), - y1: Math.min(ch, (maxY + 1 + padY) * sy) - }); - if (!M.hasInk) { M.hasInk = true; M.bbox = raw; M.dirty = true; } - else if (updateBBox(raw)) M.dirty = true; + // ---- ADDENDUM 9: resolve dual-eye vs mono ---------------------------- + // dual-eye: two copies of the same line -> no ink at the seam AND the two + // halves' column profiles match under a horizontal shift. mono: one line + // across the whole frame -> ink crosses the seam and the halves do not match. + var hasLeft = lMaxX >= 0, hasRight = rMaxX >= 0; + var hasInk = hasLeft || hasRight || centerInk > 0; + if (hasInk) { + M.centerInk = centerInk; + var dual = M.dualEye; + if (M.layout === 'dual') dual = true; + else if (M.layout === 'mono') dual = false; + else if (hasLeft && hasRight) { + if (centerInk > 0) { M.copyND = null; dual = false; } + else { M.copyND = columnCopyND(colL, colR, half); dual = (M.copyND < COPY_TOL); } } + else dual = !!hasLeft; // ink in ONE half -> dual (left) else mono (right) + if (dual !== M.dualEye) { + // Debounce: a layout switch rebuilds the layer, so require two + // consecutive samples before believing it (never observed to flicker on + // real content, but cheap insurance against a rebuild every sample). + M.mismatch = (M.mismatch || 0) + 1; + if (M.mismatch >= 2) { + M.mismatch = 0; + M.dualEye = dual; + computeTexDims(); + M.needRebuild = true; // the texture width changed with the region + } + } else M.mismatch = 0; + } + + // ---- crop to the ink box of the RESOLVED region ---------------------- + var x0s, y0s, x1s, y1s; + if (hasInk) { + if (M.dualEye && hasLeft) { + x0s = lMinX; y0s = lMinY; x1s = lMaxX; y1s = lMaxY; + } else { + x0s = hasLeft ? lMinX : rMinX; + x1s = hasRight ? rMaxX : lMaxX; + y0s = Math.min(hasLeft ? lMinY : Infinity, hasRight ? rMinY : Infinity); + y1s = Math.max(hasLeft ? lMaxY : -1, hasRight ? rMaxY : -1); + } + } + + if (!hasInk) { + if (M.hasInk) { M.hasInk = false; M.dirty = true; } // clear the strip once + } else { + var raw = quantise({ + x0: Math.max(0, (x0s - padX) * sx), + y0: Math.max(0, (y0s - padY) * sy), + x1: Math.min(M.eyeW, (x1s + 1 + padX) * sx), + y1: Math.min(ch, (y1s + 1 + padY) * sy) + }); + if (!M.hasInk) { M.hasInk = true; M.bbox = raw; M.dirty = true; } + else if (updateBBox(raw)) M.dirty = true; } updateStats(); } @@ -648,9 +761,9 @@ var b = M.bbox; if (!gl || !b) return; - var cw = Math.max(1, Math.min(M.halfW, Math.round(b.x1 - b.x0))); + var cw = Math.max(1, Math.min(M.eyeW, Math.round(b.x1 - b.x0))); var ch = Math.max(1, Math.round(b.y1 - b.y0)); - var sx = Math.max(0, Math.min(Math.max(0, M.halfW - cw), Math.round(b.x0))); + var sx = Math.max(0, Math.min(Math.max(0, M.eyeW - cw), Math.round(b.x0))); var sy = Math.max(0, Math.min(Math.max(0, M.srcH - ch), Math.round(b.y0))); var crop = M.cropCanvas, cctx = M.cropCtx; @@ -658,7 +771,7 @@ cctx.clearRect(0, 0, cw, ch); cctx.drawImage(M.canvas, sx, sy, cw, ch, 0, 0, cw, ch); - var scale = M.texW / M.halfW; // texture px per source px + var scale = M.texW / M.eyeW; // texture px per source px var dw = cw * scale, dh = ch * scale; var fit = Math.min(1, tw / dw, th / dh); // never overflow the colorTexture dw *= fit; dh *= fit; @@ -767,6 +880,38 @@ /* ================================================================ * settings (safe before or during a session) * ============================================================== */ + // ADDENDUM 9 — read state.subLayout into the module. A forced layout resolves + // immediately; 'auto' keeps the current guess until sample() sees ink. + function readLayoutSetting() { + if (S.subLayout === 'dual' || S.subLayout === 'mono') M.layout = S.subLayout; + else { M.layout = 'auto'; if (S.subLayout !== 'auto') S.subLayout = 'auto'; } + if (M.layout !== 'auto') { M.dualEye = (M.layout === 'dual'); M.mismatch = 0; } + } + + // Force home the dual-eye / mono decision, or go back to auto-detection. + function setLayout(mode) { + if (mode !== 'auto' && mode !== 'dual' && mode !== 'mono') return; + M.layout = mode; + S.subLayout = mode; + if (!M.canvas) { updateStats(); return; } + if (mode === 'auto') { + M.bbox = null; M.hasInk = false; M.mismatch = 0; M.dirty = true; // re-detect from the next sample + } else { + var forced = (mode === 'dual'); + if (forced !== M.dualEye) { + M.dualEye = forced; + M.bbox = null; M.hasInk = false; + M.mismatch = 0; + computeTexDims(); + M.needRebuild = true; + M.dirty = true; + } + } + updateStats(); + } + + function getLayout() { return M.layout; } + function setMode(mode) { if (mode !== 'hud' && mode !== 'sphere' && mode !== 'off') return; var changed = (M.mode !== mode); @@ -981,6 +1126,7 @@ else if (S.subMode) S.subMode = M.mode; if (S.subAlpha === 'premultiplied' || S.subAlpha === 'straight') M.alpha = S.subAlpha; else if (S.subAlpha) S.subAlpha = M.alpha; + readLayoutSetting(); if (isNum(S.subOpacity)) M.opacity = clamp01(S.subOpacity); else S.subOpacity = M.opacity; if (typeof S.subLockAngle === 'boolean') M.lockAngle = S.subLockAngle; @@ -1006,6 +1152,8 @@ create: create, destroy: destroy, setMode: setMode, + setLayout: setLayout, + getLayout: getLayout, setAlphaMode: setAlphaMode, setPlacement: setPlacement, getPlacement: getPlacement, diff --git a/src/90-main.js b/src/90-main.js index 661c749..f9a9593 100644 --- a/src/90-main.js +++ b/src/90-main.js @@ -20,11 +20,26 @@ * rAF broker * ------------------------------------------------------------------ * While an immersive XR session runs, window.requestAnimationFrame is - * COMPLETELY stopped (measured: 0 callbacks in 6s) and Jellyfin renders - * libass subtitles from an rAF loop, so subtitles freeze. Hold callbacks - * in our own queue while in XR and drive them from - * video.requestVideoFrameCallback (which still fires, in step with the - * video). Outside XR delegate to the native implementation. + * COMPLETELY stopped (measured: 0 callbacks in 6s). Jellyfin renders libass + * subtitles from an rAF loop, so that froze the subtitle canvas. + * + * The FIRST fix was to queue callbacks in XR and pump the queue from + * video.requestVideoFrameCallback. That is not enough on Jellyfin 12.x: a + * callback that is ALREADY PENDING in the NATIVE scheduler when the session + * starts is destroyed with the native scheduler and never runs again, and + * SubtitlesOctopus starts ONE long-lived rAF loop when the video loads whose + * id is never reset. Verified on device: instance.rafId stuck on a native id + * (1150), oneshotState.eventStart frozen ~7 min behind the playhead, zero + * oneshot-render requests for the whole session -> the whole prerender chain + * stalls and the HUD shows the last line drawn before VR started. + * + * So the queue is used ALWAYS, even outside XR, and drained by a native rAF + * pump. Nothing is ever left pending natively, so nothing can be lost when the + * session starts; startRafPump just swaps the driver. In XR the native pump + * stops and video.requestVideoFrameCallback drives the queue (it still fires, + * in step with the video); a slow timer also drives it when the video is not + * presenting frames (paused / stalled / ended), so a paused seek or a track + * switch still redraws instead of leaving a stale line on screen. * ================================================================== */ var RAFQ = { inXR: false, @@ -33,24 +48,13 @@ queue: new Map(), nextId: 1000000000, pumped: 0, + nativeId: null, + lastPump: 0, err: null }; - try { - window.requestAnimationFrame = function (cb) { - if (!RAFQ.inXR) return RAFQ.native(cb); - var id = RAFQ.nextId++; - RAFQ.queue.set(id, cb); - return id; - }; - window.cancelAnimationFrame = function (id) { - if (RAFQ.queue.has(id)) { RAFQ.queue.delete(id); return; } - return RAFQ.nativeCancel(id); - }; - } catch (e) { log('rAF broker install failed', e); } - - function pumpRaf(t) { - if (!RAFQ.inXR || !RAFQ.queue.size) return; + function drainRaf(t) { + if (!RAFQ.queue.size) return; var cbs = []; RAFQ.queue.forEach(function (cb) { cbs.push(cb); }); RAFQ.queue.clear(); @@ -60,24 +64,66 @@ } } + // Native driver (outside XR). Exactly ONE native callback is kept in flight so + // that a rAF which re-schedules itself cannot multiply the driver (an earlier + // version nulled nativeId before draining, re-armed on every self-rescheduling + // callback AND again at the end -> exponential native callbacks -> frozen page). + function nativeRafStep(t) { + if (RAFQ.inXR) { RAFQ.nativeId = null; return; } // the XR driver owns the queue + RAFQ.nativeId = RAFQ.native(nativeRafStep); // re-arm BEFORE draining + drainRaf(t); + if (!RAFQ.queue.size && !RAFQ.inXR) { // nothing left -> stop + try { RAFQ.nativeCancel(RAFQ.nativeId); } catch (e) {} + RAFQ.nativeId = null; + } + } + + function ensureNativeRaf() { + if (RAFQ.inXR || RAFQ.nativeId != null) return; + RAFQ.nativeId = RAFQ.native(nativeRafStep); + } + + try { + window.requestAnimationFrame = function (cb) { + if (typeof cb !== 'function') return 0; + var id = RAFQ.nextId++; + RAFQ.queue.set(id, cb); + if (!RAFQ.inXR) ensureNativeRaf(); + return id; + }; + window.cancelAnimationFrame = function (id) { + if (RAFQ.queue.has(id)) { RAFQ.queue.delete(id); return; } + return RAFQ.nativeCancel(id); + }; + } catch (e) { log('rAF broker install failed', e); } + function startRafPump(video) { + // NOTE: do NOT clear the queue. The callback that a naive broker loses at + // session start is sitting in it, and it is the one that keeps Jellyfin's + // subtitle loop alive. The pending NATIVE driver id is dropped: that callback + // is lost with the native scheduler anyway, and clearing the id lets the + // driver restart cleanly after the session ends. + RAFQ.nativeId = null; RAFQ.inXR = true; - RAFQ.queue.clear(); RAFQ.pumped = 0; + RAFQ.lastPump = 0; RAFQ.err = null; var stopped = false; S.stats.rafPump = video && video.requestVideoFrameCallback ? 'videoFrameCallback' : 'interval'; if (video && video.requestVideoFrameCallback) { var step = function (now) { if (stopped) return; - pumpRaf(now); + RAFQ.lastPump = performance.now(); + drainRaf(now); video.requestVideoFrameCallback(step); }; video.requestVideoFrameCallback(step); - RAFQ.stop = function () { stopped = true; }; - return; } - var id = setInterval(function () { pumpRaf(performance.now()); }, 16); + // Idle fallback: rVFC stops while the video is paused / stalled / ended. + var id = setInterval(function () { + if (stopped || !RAFQ.inXR) return; + if (performance.now() - RAFQ.lastPump > 60) drainRaf(performance.now()); + }, 50); RAFQ.stop = function () { stopped = true; clearInterval(id); }; } @@ -85,9 +131,9 @@ try { if (RAFQ.stop) RAFQ.stop(); } catch (e) {} RAFQ.stop = null; RAFQ.inXR = false; - // hand leftovers back to the real scheduler so nothing is stranded - RAFQ.queue.forEach(function (cb) { try { RAFQ.native(cb); } catch (e) {} }); - RAFQ.queue.clear(); + RAFQ.lastPump = 0; + // hand the queue back to the native driver so nothing is stranded + ensureNativeRaf(); } /* ================================================================== * @@ -124,6 +170,7 @@ S.subOpacity = num(o.subOpacity, 0.1, 1.0, S.subOpacity); S.ctlAutoHideMs = num(o.ctlAutoHideMs, 1000, 10000, S.ctlAutoHideMs); if (o.subMode === 'hud' || o.subMode === 'sphere') S.subMode = o.subMode; + if (o.subLayout === 'auto' || o.subLayout === 'dual' || o.subLayout === 'mono') S.subLayout = o.subLayout; // ADDENDUM 8: how the picture aim is applied. 'space' is the live path; the // 'rebuild' escape hatch exists because mutating a media layer's transform is // ignored by this runtime, so if an offset reference space turns out not to be @@ -158,6 +205,7 @@ subAngleDeg: S.subAngleDeg, subOpacity: S.subOpacity, subMode: S.subMode, + subLayout: S.subLayout, subAlpha: S.subAlpha, projection: S.projection, stereo: S.stereo, @@ -411,6 +459,15 @@ body.appendChild(segRow('显示', function () { return S.subtitles ? 'on' : 'off'; }, function (v) { S.subtitles = v === 'on'; saveSettings(); }, [{ label: '开', value: 'on' }, { label: '关', value: 'off' }])); + // ADDENDUM 9: dual-eye ASS vs a normal (mono) subtitle. 'auto' detects from + // the libass canvas; force it only if the detection gets it wrong. + body.appendChild(segRow('字幕布局', function () { return S.subLayout || 'auto'; }, + function (v) { + S.subLayout = v; + if (NS.Subtitles && NS.Subtitles.setLayout) { try { NS.Subtitles.setLayout(v); } catch (e) {} } + saveSettings(); + }, + [{ label: '自动', value: 'auto' }, { label: '双目', value: 'dual' }, { label: '普通', value: 'mono' }])); body.appendChild(NS.makeEl('div', 'jfvr-hint', '字幕的位置 / 距离 / 大小 / 不透明度都在 VR 里用控制条调(控制条最下面一行),' + '调好会自动记住。VR 空间右侧有个锁图标:锁上后面板不会再被唤出。'));