6 Commits
Author SHA1 Message Date
cat-shark 0713f35513 hud-11: one pointer owner (dominant hand, press takes over) + hand isolation
ADDENDUM 12. The pointer was symmetric: whoever acted last owned it, and when
neither had acted the first source in runtime order won - the LEFT hand on this
runtime, so a right-handed user's guide line sat on the non-dominant hand.

- state.handPref ('right'|'left', default 'right') is the default owner. Set from the
  pre-VR panel (操作 -> 主用手), persisted. pickPointerCand(cands, preferKey,
  ownerKey, prefHand): acting source owns the shot (or MISSes), else owner aiming ->
  preferred hand aiming -> any hand aiming -> owner -> preferred hand -> first.
- A press takes the pointer over and the owner sticks while it is aiming.
- Hold exclusion: while a grab/viewDrag started by one source is active, a press from
  the other source is ignored (pressAt "IGNORED (other hand is holding)").
- Source-matched release: pointerUp(src) only ends the hold src started. It used to be
  argument-less, so either hand's release cancelled the other hand's subtitle grab or
  picture pan. grab/viewDrag now carry a key; onSelectEnd(ev) and pollGamepads pass
  their source.
- dropStaleHolds() runs each frame and ends a hold whose source left
  session.inputSources (tracking lost), so nothing sticks.

New diagnostics: stats.handPref, stats.owner.

Verified with injected synthetic input sources:
  default (right), both on panel     pointer hand:right HIT 0.78,0.50  (was hand:left)
  handPref=left, fresh session       pointer hand:left  HIT 0.22,0.50
  left press, release                owner hand:left, stays left
  right holds picture-pan, left press    "IGNORED (other hand is holding)"
  right holds picture-pan, LEFT release  viewDrag stays armed (drag survives)
  right holds picture-pan, RIGHT release viewDrag reset (tap) (drag ends)
2026-10-12 21:36:03 +08:00
cat-shark e18c3d51bd hud-10 (rev3): cursor on the guide line; a press is attributed to the acting hand
ADDENDUM 11b, both reported against ADDENDUM 11.

1. The cursor drifted off the guide line, worse near the panel edge. panelHit()
   clamps an off-panel intersection to the border and flags it via inBounds, but
   updatePointer() published that clamped nx,ny as pointer.hit, so the ring was
   drawn on the border while the beam ended at the true (off-panel) intersection.
   pointer.hit is now published only when inBounds; distance is still published so
   an edge-aimed beam reaches the plane (beamLength no longer requires p.hit).
   Near-misses still summon the bar, but no longer press.

2. A right-hand press could still act at the left hand's position, for three
   separate reasons: the activate cooldown was global (a gamepad edge from one hand
   swallowed the other hand's select - now per input source); a missing ray fell
   back to another hand (with a preferKey the acting source now owns the shot and
   a missing candidate MISSes); and a null frame skipped re-attribution entirely
   (now falls back to S.lastFrame and MISSes if the acting source yields no
   pointer).

New diagnostics: stats.pressSrc, stats.pressPtr.

Verified with injected synthetic input sources:
  both on panel, right press      pressSrc hand:right  pressAt 0.78,0.50  seek 79.5%
  left press then right 100ms     pressSrc hand:right  pressAt 0.78,0.50  (not swallowed)
  both off panel                  state.pointer.hit = null (no clamped cursor)
  off panel, right press          pressPtr "hand:right nohit"  MISS (no hit on the panel)
2026-10-12 21:07:05 +08:00
cat-shark 5eb679f6ca hud-10 (rev2): verify the hand switch with injected synthetic XR input sources
Adds the on-device verification for ADDENDUM 11 and a diagnostics nit
(stats.activeSrc reports '(none)' while no candidate is picked).

There is no official way to fake hand input on a Quest for a WebXR page: Meta's
simulator is a desktop tool for Unity/Unreal native apps, adb has no XR input
injection and CDP has no WebXR domain (WebXR.enable does not exist). So the test
injects synthetic input sources whose targetRaySpace is a real XRRigidTransform
offset reference space of the live session, and drives a press through the real
gamepad path (rising edge on gamepad.buttons[0]) - the same pointerDown(src, frame,
...) the select event reaches. XRSession.prototype.inputSources is overridden for
the test and restored after.

Measured in a live immersive-vr session:
  initial            pointer hand:left  HIT 0.22,0.50
  right press        pointer hand:right HIT 0.78,0.50  activeSrc hand:right
  right release      pointer hand:right HIT 0.78,0.50  (sticky)
  left press         pointer hand:left  HIT 0.22,0.50
  right press, right hand off panel -> pressAt "MISS (not aiming at the panel)"
                                       (did NOT fall back to the left hand's position)
2026-10-12 20:29:51 +08:00
cat-shark 2459b999c6 hud-10: the VR pointer follows the hand that is acting or aiming
ADDENDUM 11. With two hands both report a tracked-pointer ray and the runtime's order
is not "the hand the user is using". updatePointer() took the first valid source and
broke, and pointerDown() called updatePointer(frame) without the select event's
inputSource. On device that pinned the guide line/cursor to one hand while the user
aimed with the other, and made the other hand's pinch land wherever the pinned hand
pointed (left hand controlling the click position, right hand clicking).

updatePointer(frame, preferSrc) now builds a candidate per tracked-pointer source and
pickPointerCand() chooses: the acting source, else the last-active source while it is
on the panel, else any source on the panel, else the last-active source, else the
first. pointerDown() passes the event's inputSource, so an action is always evaluated
with the acting hand's ray, and the choice sticks afterwards. stats.activeSrc reports
it; session end resets it.

pickPointerCand is pure and unit-tested against the built bundle (9 cases, including
the reported one: guide on the left + right pinch -> hand:right, and it stays).

Verified in the headset only up to "no hands tracked at the time"; the live hand
switch still needs a worn headset.
2026-10-12 20:22:29 +08:00
cat-shark 22dd482d69 hud-9: render native (SRT) subtitles in the HUD; stop persisting the layout override
ADDENDUM 10. Two more independent subtitle problems found on device.

1. Normal (non-ASS) subtitles never reached the HUD. Jellyfin's
   htmlVideoPlayer-plugin only routes ssa/ass through SubtitlesOctopus
   (`if (!['ssa','ass'].includes(codec))` -> pgssub/vobsub/custom-element/native);
   subrip falls through to the browser's own TextTrack rendering, so there is no
   libass canvas at all (measured with the CN SRT selected: 0 libass canvases,
   0 .videoSubtitles overlays, textTracks[0] = manualTrack/mode=showing/315 cues).
   No layout setting could have made that work.

   updateSource() now picks the source every XR frame: the libass canvas when it
   exists, otherwise the active TextTrack cue (fallback: the .videoSubtitlesInner
   overlay). The native text is cleaned, wrapped and painted white-on-black into a
   private 1920x1080 canvas that flows through the unchanged crop/upload/HUD path.
   prepare() is skipped when there is no libass canvas. New diagnostics:
   stats.subSource / stats.subNativeText, Subtitles.getSourceKind().

2. ADDENDUM 9 persisted state.subLayout, and a stale 'mono' in jfvr.settings.v1
   silently forced the mono branch on the default dual-eye content (dual:false,
   eyeW:4320, crop across the seam -> both eyes side by side). The override is now
   per-session: not saved, not restored, 'auto' at boot.

Verified on device with hud-9:
  CN_dual_eye (ass)   -> subSource=libass, dual=true, eyeW=2160,
                         crop=[1040,768,1176,924], copyND=0.055
  CN (subrip)         -> subSource=native, dual=false, eyeW=1920, text="进去了呢",
                         crop 256x88, uploads climbing
The private cue canvas was read back and OCR'd (cue "怎么样?好好地" -> "怎么样? 好好地"),
and injecting/removing a fake libass canvas flips subSource with no errors.
2026-10-12 19:53:10 +08:00
cat-shark 164d692791 hud-8: keep the libass loop alive across XR entry; auto dual/mono subtitles
ADDENDUM 9 — two independent defects found on device.

1. VR subtitles froze on whatever line was on screen when the session started.
   Jellyfin 12.x renders with SubtitlesOctopus, which starts ONE long-lived rAF
   loop when the video loads; entering immersive-vr stops native rAF and destroys
   the callback still pending in the native scheduler, and because the instance's
   rafId stays non-zero it is never restarted (measured live: rafId=1150 native,
   oneshotState.eventStart frozen ~7 min behind the playhead, zero oneshot-render
   requests). The broker now ALWAYS queues rAF (native driver outside XR, rVFC +
   50 ms idle fallback inside), keeps exactly one native callback in flight, and
   no longer clears the queue at session start. A first version re-armed during
   the drain and doubled the driver every frame, freezing the page; that is now
   fixed and locked down by a Node harness (maxConcurrentNative = 1 over 300
   frames, self-rescheduling loop survives a simulated session start).

2. Only dual-eye ASS subtitles worked; a normal subtitle (e.g. the CN SRT track)
   is centred across the whole 2:1 frame and was cut at the eye seam because the
   HUD always sampled the left half. state.subLayout ('auto'|'dual'|'mono',
   default auto, persisted, panel row) plus per-sample detection: seam ink -> mono,
   one half only -> dual/mono, both halves with an empty seam -> column-profile
   copy test. Dual-eye is NOT a pixel mirror (same text at mirrored positions, no
   glyph flip): measured dual copy distance 0.02-0.08 vs mono 1.00, seam ink
   0 vs 50. M.eyeW (half for dual, full for mono) replaces the hard-coded halfW.

Docs: INTERFACES.md ADDENDUM 9, HANDOFF.md §13 (including the Jellyfin service
worker cache trap that made the first deploy look like a no-op).

The final build was not re-checked in the headset: the device left the worn state
mid-verification and the browser could not be relaunched. Verified on device: the
loop survives XR (rafId in the plugin range, eventStart tracking, subUploads 4->7
over 20 s in VR) and the layout metrics above; the final decision wiring is
covered by the offline suites.
2026-10-12 16:32:25 +08:00
7 changed files with 1850 additions and 164 deletions
+293 -4
View File
@@ -1,10 +1,16 @@
# Jellyfin Web VR180 — 项目交接文档
> 给新对话用。读完这份就能接手,不需要重新调查环境。
> 最后更新:已部署版本 `5217a3`(`NS.version = hud-7`;ADDENDUM 6 字幕深度/大小解耦
> + 7 唤出门限/前面板精简 + 7b 锁按钮与「锁定」面板模式 + 8/8b 画面搬迁(实时拖动))
> 已在 Quest 浏览器里实机验证模块加载 + 锁定数学(`subLockAngle/subAngleDeg`,`errors: []`);
> **VR 会话内的观感还没看过**。
> 最后更新:已部署版本 `917fde`(`NS.version = hud-11`;ADDENDUM 6 字幕深度/大小解耦
> + 7 唤出门限/前面板精简 + 7b 锁按钮与「锁定」面板模式 + 8/8b 画面搬迁(实时拖动)
> + 9 rAF 代理修复(VR 内字幕不再冻结)+ 普通/双目字幕自动识别
> + 10 原生(SRT 等)字幕由插件自绘到 HUD;布局覆盖不再持久化
> + 11 指针跟随正在操作/瞄准的那只手;11b 光标落在引导线上 + 点击归属操作手
> + 12 单一指针主人(主用手默认,按下接管,按住互斥,抬起按源匹配))
>
> ⚠️ **Jellyfin web 注册了 service worker**(`serviceworker.js`,scope `/web/`),会把
> `index.html` 缓存在 SW cache 里。重新部署后如果只是普通刷新,浏览器可能仍用旧的
> `index.html`(于是继续加载旧的 `?v=` 包)。必须清 SW + 清 cache + 硬刷新,见 §13。
---
@@ -410,6 +416,9 @@ pressAt 上次按压命中/未命中
handsReady / handsErr / handsSources / handsFrames
subMode / subCanvasSize / subCrop / subTex / subOpacity / subPlacement
subLockAngle / subAngleDeg / subTextDeg / subYOverZ ← ADDENDUM 6
subLayout / subDual / subEyeW / subCopyND / subCenterInk ← ADDENDUM 9
subSource / subNativeText ← ADDENDUM 10
activeSrc / pressSrc / pressPtr / handPref ← ADDENDUM 11/12
ctlSubsOn / ctlSubDist / ctlSubAngle / ctlSubLock ← 控制条实际画出的值
ctlLock / ctlMode / ctlHold / pressAt / lastActivate ← ADDENDUM 7/7b 锁、面板保持与唤出判定
videoYaw / videoPitch / videoRotMode / videoTransformWritable ← ADDENDUM 8/8b 画面搬迁
@@ -491,3 +500,283 @@ errors[] 最近错误
**手势**(按你的要求):`点一下复位 · 按住 0.5s 再拖动`。拖动基准取"按住满 0.5s 那一刻"的射线,
不是按下那一刻——否则那半秒会把手抖放大成甩动。1° 死区防抖。
---
## 13. ADDENDUM 9 — 字幕「中断」与普通字幕(已部署 `b9c3a5` / `hud-8`)
### 13.1 定位手法(下次直接复用)
- **连设备**:`./quest.sh`(它的授权检测偶尔误判,失败就手动
`adb forward tcp:9222 localabstract:chrome_devtools_remote`)。
- **读 SubtitlesOctopus 实例**:Jellyfin 12.x 的实例在播放器的闭包里,用 CDP 的
`getEventListeners(video).timeupdate[i].listener` → `[[Scopes]]` → 找带
`oneshotState`/`renderedItems` 的对象。本机脚本见 `/tmp/inst.py`、`/tmp/scan.py`
(一次性工具,未入库)。关键字段:`rafId`(我们的是 1e9 起)、
`oneshotState.eventStart`(当前画的那一行)、`renderRequested`。
- **量 libass 画布**:把 `canvas.libassjs-canvas` 画进小 canvas 取 `getImageData`,
统计左右半屏/中线的墨迹与列轮廓(`/tmp/prof.js`、`/tmp/mirror*.js`)。
- **OCR**:`adb exec-out screencap -p` + `tesseract -l chi_sim`(双脚同屏也能读)。
本次就是靠它确认 HUD 停在 15:45 的台词而视频已在 22:49。
### 13.2 问题一:VR 内字幕冻结(不是「被截断」)
- 现象:进 VR 后字幕永远停在进 VR 前那一帧。实测 `rafId=1150`(**原生** id)、
`oneshotState.eventStart=945.99`(落后播放头约 7 分钟)、整段播放 **0 个**
`oneshot-render` 请求。
- 根因:进入 `immersive-vr` 后原生 rAF 完全停摆。SubtitlesOctopus 在主线程起了**一个
常驻 rAF 循环**(`k`):它负责把 worker 预渲染好的帧画到 canvas、并继续要下一帧。
旧代理只在「已进入 XR」后才接管 rAF,于是**进 XR 那一刻还挂在原生调度器里的那个
`k` 回调被销毁**;而 `i.rafId` 仍非 0,`resetRenderAheadCache` 也不会重启它 → 整个
prerender 链断掉。
- 修复(`src/90-main.js`):rAF 代理**始终**接管(非 XR 用原生 rAF 泵,XR 用 rVFC),
且原生泵**单飞**(`nativeRafStep` 先续期再 drain,队列空则 cancel);`startRafPump`
**不再 clear 队列**(那个回调就是要救的);加 50ms 兜底,暂停/卡住时也能重画。
- ⚠️ **踩过的坑**:第一版原生泵在 drain 期间把 `nativeId` 置空,每个自续期回调都会额外
再排一个原生 rAF → 回调数**指数增长**,直接把页面主线程打爆(CDP 无响应)。现在由
Node 单测固定:自续期循环 300 帧 `maxConcurrentNative = 1`。
### 13.3 问题二:只支持双目 ASS,普通字幕被截断
- 根因:旧代码硬编码只取画布**左半屏**。双目 ASS(`CN_dual_eye`,PlayRes 3840×1920,
LeftEye/RightEye 各占半屏)没问题;普通字幕(`CN` 的 SRT,居中跨整幅)文字中心正好
压在画布中线(两眼接缝)上,于是只剩左半边 → 看起来「断掉」。
- 修复(`src/10-subtitles.js`):新增 `state.subLayout`(`auto`/`dual`/`mono`,默认
`auto`,面板有「字幕布局」一行,持久化)。每次采样统计左右半屏 + 中线 ±40px 的墨迹:
中线有墨 → 普通;只有左半 → 双目;只有右半 → 普通;两半都有且中线空 → **列轮廓平移
匹配**(`columnCopyND` < 0.25 → 双目)。布局切换做 2 帧去抖。
- **血泪教训:双目字幕不是像素镜像**。左右眼是**同一段文字放在对称位置**,字形不翻转,
所以 `left[x] vs right[W-1-x]` 完全对不上(实测均值差 258/1020)。我第一版就是被这个
骗了,把双目误判成普通(HUD 于是把两只眼的字并排显示)。真正成立的是**列轮廓**:
实测双目 `0.02–0.08`、普通 `1.00`、中线墨迹 双目 `0` / 普通 `50`。
- `M.eyeW`(单眼区域宽度:双目=半幅、普通=整幅)替换了原来写死的 `halfW`。
### 13.4 部署缓存陷阱(为什么第一次改了「没生效」)
Jellyfin web 注册了 **service worker**,`index.html` 会被缓存。只普通刷新时浏览器可能
仍用旧 `index.html` → 继续请求旧的 `jfvr-mvp.js?v=5217a3`。清理脚本(一次性的):
```js
for (const r of await navigator.serviceWorker.getRegistrations()) await r.unregister();
for (const k of await caches.keys()) await caches.delete(k);
// + Network.clearBrowserCache + Page.reload{ignoreCache:true}
```
验证新包是否真的加载:`document.querySelector('script[plugin="JellyfinVR"]').getAttribute('src')`
与 `window.__JFVR.version`(应为 `hud-8`)。
### 13.5 待验证 / 本次未完成
- 最终版(`b9c3a5`,`COPY_TOL=0.25`)**还没在头显里跑过**:验证到一半设备进入未佩戴 /
Guardian 状态,Oculus 浏览器无法再拉起(`ps` 无进程、无 devtools socket)。已完成的实机
证据:broker 修复(`rafId` 落在插件区间、`eventStart` 跟随、VR 内 `subUploads` 4→7)、
以及两个布局的**真实量测值**。最终判据是离线单测(真实源码 `columnCopyND`)验证的。
- 下次戴上头显后要确认:`stats.subLayout=auto`、双目轨 `subDual=true/subEyeW=2160/
subCrop` 落在左半屏(≤2160)、普通轨 `subDual=false/subEyeW=4320/subCrop` 跨中线、
`subCopyND` 与 `subCenterInk` 是否符合上面的表。
---
## 14. ADDENDUM 10 — 普通字幕(SRT 等)为什么读不到(已部署 `e394bcb` / `hud-9`)
### 14.1 根因:Jellyfin 只把 ssa/ass 交给 libass
`htmlVideoPlayer-plugin` 的 `renderTracksEvents` 决策:
```js
a = (n.Codec || '').toLowerCase();
if (!X.includes(a)) { // X = ['ssa','ass']
if (a === 'pgssub') -> renderPgs
if (Y.includes(a)) -> renderVobSub // Y = ['dvdsub','vobsub']
if (!u(h.fe)) -> renderSubtitlesWithCustomElement // DOM 覆盖层
/* 否则落到原生 TextTrack 分支 */
}
return renderSsaAss(...); // 只有 ssa/ass
```
所以 `subrip`/`vtt` **根本不会创建 libass canvas**。设备实测(选中 `CN` SRT 时):
`canvas.libassjs-canvas` 数量 **0**、`.libassjs-canvas-parent` 0、`.videoSubtitles` 0,
只有 `video.textTracks[0] = { kind:'subtitles', label:'manualTrack', mode:'showing',
cues:315 }` —— 浏览器用 `::cue` 自己画。插件**无论改什么布局都读不到任何东西**。
### 14.2 修复(`src/10-subtitles.js`)
`updateSource()` 每帧挑来源:
- 有非空 `canvas.libassjs-canvas` → `srcKind='libass'`(原路径不变);
- 否则若有 showing 的 TextTrack 或 `.videoSubtitlesInner` → `srcKind='native'`:
从 `textTracks[i].activeCues` 读当前 cue(兜底读 DOM 覆盖层 `innerText`),
清洗 ASS/HTML 标签与双向控制符、按字符折行,用白字黑描边画进私有 1920×1080
canvas,再走**完全相同的裁剪/上传/HUD 管线**;
- 都没有 → `srcKind='none'`,清一次字幕条。
`prepare()` 在没有 libass canvas 时跳过(放大播放器框只是为了让 libass 重渲染,原生
canvas 是我们自己的)。新增诊断 `stats.subSource` / `stats.subNativeText` 和
`Subtitles.getSourceKind()`。
### 14.3 血的教训:布局覆盖**不要持久化**
ADDENDUM 9 把 `state.subLayout` 存进了 `jfvr.settings.v1`。结果一个残留的 `'mono'`
静默地把默认的**双目**内容强制成普通:`dual:false`、`eyeW:4320`、裁剪跨越两眼接缝 →
HUD 把两只眼的字幕并排显示。找这个 bug 花的时间比写检测还多。现在 `subLayout`
**不持久化**(和 `subtitles`、`subMode:'off'` 同理),开机恒为 `auto`,面板可临时改。
### 14.4 设备实测(`hud-9`)
| 选中轨 | `subSource` | 结果 |
|---|---|---|
| `CN_dual_eye`(ass,idx 0) | `libass` | `dual:true`、`eyeW:2160`、`crop:[1040,768,1176,924]`(左半)、`copyND:0.055`、uploads 递增 |
| `CN`(subrip,idx 1) | `native` | `dual:false`、`eyeW:1920`、canvas 1920×1080、`text:"进去了呢"`、`crop` 256×88、uploads 递增 |
私有 cue canvas 还读回做了 OCR(cue `怎么样?好好地` → OCR `怎么样? 好好地`);
注入/移除一个假的 `canvas.libassjs-canvas` 能让 `subSource` 在 `libass`/`native`
间切换且 `subErr` 为空。
---
## 15. ADDENDUM 11 — 指针跟随「正在操作/正在瞄准」的那只手(已部署 `6bf1021` / `hud-10`)
**根因**:双手都会报 `tracked-pointer` 射线,而运行时的顺序不是「用户正在用的那只手」。
`updatePointer()` 取**第一个**有有效射线的输入源就 `break`;而 `pointerDown()` 调用的是
`updatePointer(frame)`——**没有把 `select` 事件里的 `inputSource` 传进去**。于是设备上:
- 引导线/光标永远钉在一只手上(用户用另一只手指也不跟)→「引导线并没有指向光标」;
- 另一只手的捏合**落在被钉住那只手所指的位置**:左手控制点击位置,右手完成点击。
**修复**(`src/90-main.js`):`updatePointer(frame, preferSrc)` 为每个 tracked-pointer
源各算一次命中,再用纯函数 `pickPointerCand(cands, preferKey, activeKey)` 选:
```
1. preferSrc —— 当前正在操作的那个源(select / gamepad 按下)
2. 上次操作的源 —— 前提是它正瞄准面板(hit.inBounds)
3. 任一个正在瞄准面板的源(运行时顺序)
4. 上次操作的源(粘滞)
5. 第一个源
```
`pointerDown()` 传入事件的 `inputSource`,所以每次操作都用**操作那只手**的射线判定;
选中的手之后保持粘滞。诊断新增 `stats.activeSrc`,会话结束时清空。
**验证**:`pickPointerCand` 是纯函数,直接从构建产物提取做了单测——包含你描述的场景
(引导线在左手 → 右手捏合)现在得到 `hand:right` 并保持;另有「左手接管」「操作手不在
面板上、另一只手在瞄 → 跟随瞄准手」「都没瞄 → 保持上次操作手」等 9 个用例全过。
**验证手法(重要,可复用)**:官方文档里只有 **Meta XR Simulator** 能模拟 aim/poke/pinch/grab,
但它是**开发机上的桌面工具**,面向 Unity/Unreal 原生应用;`adb` 没有 XR 输入注入,CDP 也没有
WebXR 域(`WebXR.enable` 不存在)。所以改成**从页面注入合成输入源**:
```js
// 每个 targetRaySpace 都是本会话真实的 XRRigidTransform 偏移参考空间
var off = (await session.requestReferenceSpace('local'))
.getOffsetReferenceSpace(new XRRigidTransform({x:±0.2, y:-0.34, z:-1.0}));
window.__FAKE = [ {handedness:'left', targetRayMode:'tracked-pointer', hand:{}, targetRaySpace:off,
gamepad:{buttons:[{pressed:false,value:0}]}} , /* right */ ];
Object.defineProperty(XRSession.prototype,'inputSources',{configurable:true,get:function(){return window.__FAKE;}});
// 触发:window.__FAKE[1].gamepad.buttons[0].pressed = true → pollGamepads 上升沿 → pointerDown(src,frame,'gamepad')
```
走的是**真实**的 `pointerDown(src, frame, …)`(和 `select` 同一条路径),测完把
`inputSources` 描述符还原。实测(`immersive-vr` 活动会话):
```
初始 pointer hand:left HIT 0.22,0.50 activeSrc hand:left
右手按下 pointer hand:right HIT 0.78,0.50 activeSrc hand:right
右手松开 pointer hand:right HIT 0.78,0.50 (粘滞保持)
左手按下 pointer hand:left HIT 0.22,0.50 activeSrc hand:left
右手没瞄面板时按下 pressAt "MISS (not aiming at the panel)" ← 没有偷用左手位置
```
> `XRInputSourceEvent('select', …)` 构造器要求真实 `XRInputSource`,所以无法伪造 select 事件;
> 但 select 与 gamepad 共用 `pointerDown`,已被上面覆盖。
---
## 16. ADDENDUM 11b — 光标落在引导线上;点击归属操作那只手(已部署 `3d3b5f`)
针对 ADDENDUM 11 实测后反馈的两点。
### 16.1 光标偏离引导线(越靠边缘越明显)
`panelHit()` 会把**打在面板外**的交点**夹取到边框**,并用 `inBounds` 标出来;但
`updatePointer()` 把夹取后的 `nx,ny` 当 `pointer.hit` 发布了。于是光标环画在**面板边框**上,
而光束末端在**真实交点**(面板之外)——射线越偏离面板,两者差得越远,与反馈完全一致。
现在 **只有 `hit.inBounds` 才发布 `pointer.hit`**:光标只在射线真正打在面板上时出现,必然落在
光束上。只要射线与面板平面相交就仍然发布 `distance`,且 `beamLength()` 不再要求 `p.hit`,
所以瞄边缘时引导线仍指到面板平面。擦边仍能**唤出**控制条(宽判 `panelHit(...,0.5,0.9)` 未动),
但擦边不再能**按下**。
### 16.2 右手点击仍会落在左手位置
三个独立原因,全部修掉:
- **冷却是全局的**:`ACTIVATE_COOLDOWN_MS`(250ms) 会压掉 250ms 内的任意一次按下,所以一只手的
gamepad 抖动会**吃掉另一只手的 `select`**,动作随即用了第一只手的位置。现改为**按输入源分别
计时**(`lastActivateByKey`):同一次捏合产生的 gamepad+select 仍会去重,但绝不会吃掉另一只手。
- **取不到射线时静默回退到另一只手**:现在只要给了 `preferKey`(正在操作的那个源),它就
**独占这次射击**;若它不在候选里,`pick` 返回 `null`,这次按下算 **MISS**。
- **`frame` 为空时根本没重新归属**:`pointerDown()` 原来写 `if (frame) updatePointer(...)`,
frame 为空时就沿用上一帧的指针(可能是另一只手)。现在回退到 `S.lastFrame`,且若仍算不出指针,
直接记 `MISS (acting source has no ray)`。
新增诊断:`stats.pressSrc`(本次按下归属的源)、`stats.pressPtr`(`hand:right HIT` /
`hand:right nohit` / `(no ray)`)。
### 16.3 实测(注入合成输入源,活动会话)
```
都在面板上,右手按下 pressSrc hand:right pressPtr "hand:right HIT" pressAt 0.78,0.50 seek 79.5%
左手按下后 100ms 右手按下 pressSrc hand:right pressAt 0.78,0.50 (右手没被吃掉)
两只手都在面板外 state.pointer.hit = null (不再夹取光标)
面板外右手按下 pressSrc hand:right pressPtr "hand:right nohit"
pressAt "MISS (no hit on the panel)" (没有用左手位置)
```
---
## 17. ADDENDUM 12 — 单一指针“主人”:主用手默认,按下接管(已部署 `917fde` / `hud-11`)
之前的指针是**对称**的:谁最后操作谁拥有;谁都没操作时用「运行时第一个源」——本设备上就是**左手**,
所以右撇子一抬手引导线就在非惯用手上。而且两只手会互相打扰彼此的按住操作。
### 17.1 单一主人(像鼠标只有一个光标)
`pickPointerCand(cands, preferKey, ownerKey, prefHand)`:
```
preferKey(正在操作的那个源) → 就是它;取不到 → null ⇒ 这次按下 MISS
主人且正瞄面板 → 主人
主用手且正瞄面板 → 主用手
任一只正瞄面板的手 → 它
主人(没瞄面板) → 主人
主用手(没瞄面板) → 主用手
→ 第一个源
```
- **`state.handPref`**(`'right' | 'left'`,默认 `'right'`)= 指针的默认主人(惯用手)。
在**进 VR 前的面板**「操作 → 主用手」里设置,持久化在 `jfvr.settings.v1`。会话开始生效;
会话中主人只会粘滞。
- **按下即接管**(`preferKey`),之后主人粘在那只手上(只要它在瞄面板)。
### 17.2 两只手不再互相打扰
- **按住期间互斥**:某只手发起的长按/拖动(`grab` / `viewDrag`)进行中时,**另一只手的按下被忽略**
(`stats.pressAt = 'IGNORED (other hand is holding)'`)。
- **抬起按源匹配**:`pointerUp(src)` 只结束 `src` 自己发起的那次抓取/拖动。之前它是无参的,
所以**任一只手松手都会取消另一只手的字幕抓拖/画面搬迁**。现在 `onSelectEnd(ev)` 和
`pollGamepads` 都会传源,`grab` / `viewDrag` 各自带 `key`。
- **失效收尾**:`dropStaleHolds()` 每帧检查,若发起抓取的那只手从 `session.inputSources` 消失
(手部追踪丢失)就主动结束,避免永远没有 release 事件而卡住。
新增诊断:`stats.handPref`、`stats.owner`(= `stats.activeSrc`),以及已有的 `pressSrc` / `pressPtr`。
### 17.3 实测(注入合成输入源,活动会话)
```
默认(handPref=right),两手都在面板上 pointer hand:right HIT 0.78,0.50 (原来是 hand:left)
handPref=left(持久化后重开会话) pointer hand:left HIT 0.22,0.50
左手按下后松开 主人 hand:left,之后仍是 left
右手按住画面拖动条,左手按下 pressAt "IGNORED (other hand is holding)"
右手按住画面拖动条,左手松开 viewDrag 仍 "armed"(拖动存活)
右手按住画面拖动条,右手松开 viewDrag "reset (tap)"(拖动结束)
```
+315
View File
@@ -976,3 +976,318 @@ double rotation) and schedules no rebuild; unchanged angles reuse the cached spa
rebuild strategy queues at most one, and its observed order is
`removeLayer -> createVideoLayer -> addLayer`; a throwing/absent
`getOffsetReferenceSpace()` is reported and permanently falls back to `rebuild`.
# ADDENDUM 9 — subtitle pipeline survival + normal (mono) subtitles
Two independent defects, found on device with the VR session left in its broken state.
## 1. The rAF broker lost Jellyfin's subtitle loop at session start
`90-main.js` no longer just *forwards* to native `requestAnimationFrame` outside XR. It
**always** holds callbacks in its own queue and drives them with one of two drivers:
- outside XR: a native rAF driver that keeps **exactly one** native callback in flight
(`nativeRafStep` re-arms *before* draining and cancels itself when the queue empty);
- inside XR: `video.requestVideoFrameCallback`, plus a 50 ms idle fallback so a paused /
stalled / ended video still redraws (paused seek, subtitle track switch).
Why: `requestSession('immersive-vr')` stops native rAF completely. Jellyfin 12.x renders
subtitles with SubtitlesOctopus, whose main thread starts **one long-lived rAF loop**
(`rafId`) when the video loads, draws the canvas from it and requests the next prerender
from the worker (`oneshot-render`). A callback still pending in the *native* scheduler when
the session starts is destroyed with it, and because `rafId` stays non-zero the loop is
never restarted.
Evidence (live instance, via CDP closure inspection):
```
rafId : 1150 <- native id, not the plugin's 1e9 range
oneshotState.eventStart: 945.99 <- canvas frozen ~7 min behind the playhead
renderRequested : false
oneshot-render requests during 15 s of playback: 0
```
After the fix, in a live VR session: `rafId = 1000028035`, `eventStart` follows
`video.currentTime`, and `stats.subUploads` climbs while subtitles change.
`startRafPump()` must **not** clear the queue (that callback is the one being rescued) and
must drop the stale `nativeId`. `stopRafPump()` re-arms the native driver.
## 2. Auto dual-eye / mono subtitle layout
`10-subtitles.js` used to assume the libass canvas is a stereo left/right pair and always
sampled the **left half**. A normal subtitle (e.g. the `CN` SRT track) is centred across the
whole 2:1 frame, so half of it was cut off.
`state.subLayout` (`'auto' | 'dual' | 'mono'`, default `'auto'`, persisted) selects the
region; `stats.subLayout / subDual / subEyeW / subCopyND / subCenterInk` report it.
Per sample (on the 320×160 downsample) the module builds the ink box and column count of
each half plus the ink in a ±40 source-px band around the seam, then:
```
only the left half has ink -> dual (sample the left half)
only the right half has ink -> mono (sample the whole width)
ink crosses the seam -> mono
both halves, seam empty -> columnCopyND < COPY_TOL ? dual : mono
```
**Dual-eye is NOT a pixel mirror.** Both eyes carry the *same* text at mirrored *positions*;
the glyphs are not flipped, so `left[x]` vs `right[W-1-x]` fails badly (measured mean delta
258/1020 → "not a mirror"). What is true is that the two halves' ink **column profiles**
match under a small horizontal shift, because they are the same line. Measured on device:
| layout | centre-band ink | column-copy distance |
|---|---|---|
| dual-eye (`CN_dual_eye`) | 0 | 0.02 – 0.08 |
| mono (`CN` SRT) | 50 / frame | 1.00 |
`COPY_TOL = 0.25`. A layout change is debounced (two consecutive samples) because it
rebuilds the layer.
`M.eyeW` is one eye's region width (half for dual, full for mono); `computeTexDims()`,
`drawHud()`'s crop/clamp and the texture scale all use it instead of the old hard-coded
`halfW`.
### API additions
```js
__JFVR.Subtitles.setLayout('auto' | 'dual' | 'mono') // also state.subLayout
__JFVR.Subtitles.getLayout(): string
```
## 3. Verification
- Broker logic replicated exactly in a Node harness against the built bundle:
`maxConcurrentNative = 1` over 300 frames with a self-rescheduling loop (the first
version re-armed during the drain and doubled the driver every frame — that froze the
page); simulating session start (native callbacks dropped) leaves the loop in the queue
and it keeps firing from the XR driver.
- `columnCopyND` extracted from the built bundle and exercised with the on-device
measurements: dual 0.00–0.08 → `dual`, mono centre-ink → `mono`, no-copy word-gap
profile 0.41 → `mono`.
- On device (before the UI became unreachable): the broker change was observed to carry
SubtitlesOctopus's loop through `immersive-vr` (`rafId` in the plugin's id range,
`eventStart` tracking, `subUploads` 4 → 7 over 20 s in VR).
# ADDENDUM 10 — normal (non-ASS) subtitles: Jellyfin does not use libass for them
## 1. Why normal subtitles were invisible
The HUD samples `canvas.libassjs-canvas`. That canvas only exists when Jellyfin
routes the subtitle through **SubtitlesOctopus (libass)**. From
`htmlVideoPlayer-plugin` (`renderTracksEvents`):
```js
a = (n.Codec || '').toLowerCase();
if (!X.includes(a)) { // X = ['ssa','ass']
if (a === 'pgssub') -> renderPgs
if (Y.includes(a)) -> renderVobSub // Y = ['dvdsub','vobsub']
if (!u(h.fe)) -> renderSubtitlesWithCustomElement // DOM overlay
/* else falls through to the native TextTrack path */
}
return renderSsaAss(...); // ssa/ass ONLY
```
So `subrip`/`vtt`/… never reach libass. Observed on device with the `CN` SRT track
selected: `canvas.libassjs-canvas` count **0**, `.libassjs-canvas-parent` 0,
`.videoSubtitles` 0 — and `video.textTracks[0] = { kind:'subtitles', label:'manualTrack',
mode:'showing', cues:315 }`, i.e. the **browser** is rendering the cues through
`::cue`. There was nothing for the plugin to read, at any layout setting.
## 2. What the module does now
`updateSource()` runs every XR frame and picks the source:
- a `canvas.libassjs-canvas` with a non-zero size → `srcKind = 'libass'` (unchanged path);
- otherwise, if the player has a showing TextTrack or a `.videoSubtitlesInner`
overlay → `srcKind = 'native'`. The active text comes from
`textTracks[i].activeCues` (fallback: the DOM overlay's `innerText`), is cleaned
of ASS/HTML tags and bidi marks, wrapped, and painted — white with a black outline,
centred — into a private 1920×1080 canvas. That canvas is fed through the exact
same crop/bbox/upload code as the libass one.
- neither → `srcKind = 'none'` and the strip is cleared once.
`prepare()` is a no-op without a libass canvas (blowing up the player box is only
needed so libass re-renders bigger; the native canvas is ours).
New diagnostics: `stats.subSource` (`'libass' | 'native' | 'none'`),
`stats.subNativeText`; `__JFVR.Subtitles.getSourceKind()`.
## 3. The persisted layout override was a trap
ADDENDUM 9 persisted `state.subLayout`. A stale `'mono'` in `jfvr.settings.v1`
silently forced the mono branch on the default **dual-eye** content: `dual:false`,
`eyeW:4320`, and a crop spanning the seam, so the HUD showed both eyes' text side by
side. Debugging that cost more time than the detection itself.
`subLayout` is therefore **not persisted** (same reasoning as `subtitles` and
`subMode:'off'` — a stored value that leaves the layer looking broken). It is
`'auto'` at boot, toggleable per session from the panel.
## 4. Verification (on device, build `hud-9`)
| selected track | `subSource` | result |
|---|---|---|
| `CN_dual_eye` (ass, idx 0) | `libass` | `dual:true`, `eyeW:2160`, `crop:[1040,768,1176,924]` (left half), `copyND:0.055`, uploads climbing |
| `CN` (subrip, idx 1) | `native` | `dual:false`, `eyeW:1920`, canvas 1920×1080, `text:"进去了呢"`, `crop` 256×88, uploads climbing |
The private cue canvas was read back and OCR'd ("怎么样? 好好地" for cue
"怎么样?好好地"), and injecting/removing a fake `canvas.libassjs-canvas` flips
`subSource` between `libass` and `native` with no errors.
# ADDENDUM 11 — the pointer follows the hand that is acting / aiming
With two hands, BOTH report a `tracked-pointer` ray, and the runtime's order is not
"the hand the user is using". `updatePointer()` took the **first** input source with
a valid ray and `break`ed, and `pointerDown()` called `updatePointer(frame)` without
passing the source from the `select` event. Consequences on device:
- the guide line / cursor stayed pinned to one hand even while the user aimed with
the other (so the guide did not point where they were pointing);
- the other hand's pinch landed **wherever the pinned hand pointed** — the left hand
controlled the click position while the right hand performed the click.
`updatePointer(frame, preferSrc)` now builds a candidate per tracked-pointer source
(with its `panelHit`), and `pickPointerCand()` chooses:
```
1. preferSrc the source acting right now (select / gamepad press)
2. last-active source while it is aiming at the panel (hit.inBounds)
3. any source aiming at the panel (first in runtime order)
4. last-active source (sticky)
5. the first source
```
`pointerDown()` passes the event's `inputSource`, so an action is always evaluated
with the acting hand's ray, and the choice sticks afterwards (`state.stats.activeSrc`).
`pointerUp`/session end reset it.
### Verification
There is no official way to fake hand input on a Quest for a WebXR page: Meta's
simulator (the only "simulate aim/poke/pinch/grab" tool in the docs) is a desktop
tool for Unity/Unreal native apps, `adb` has no input-injection for XR, and CDP has
no WebXR domain. So the live check injected **synthetic input sources** from the page:
each `targetRaySpace` is a real `XRRigidTransform` offset reference space of the live
session, and a press is driven through the real gamepad path (a rising edge on
`gamepad.buttons[0]`), which reaches the same `pointerDown(src, frame, …)` the
`select` event uses. `XRSession.prototype.inputSources` is overridden for the test and
restored afterwards.
Measured in a live `immersive-vr` session:
```
initial pointer hand:left HIT 0.22,0.50 activeSrc hand:left
right hand press pointer hand:right HIT 0.78,0.50 activeSrc hand:right lastActivate gamepad:right
right hand release pointer hand:right HIT 0.78,0.50 (sticky)
left hand press pointer hand:left HIT 0.22,0.50 activeSrc hand:left
right press, right hand NOT on panel
pressAt "MISS (not aiming at the panel)" <- did NOT use the left hand's position
```
`pickPointerCand` is also extracted from the built bundle and unit-tested (9 cases).
# ADDENDUM 11b — the cursor sits ON the guide line; a press is attributed to the acting hand
Two follow-ups reported against ADDENDUM 11.
## 1. The cursor drifted off the guide line, worse near the panel edge
`panelHit()` clamps an intersection that misses the panel to the border and returns
`inBounds` to say so, but `updatePointer()` published that clamped `nx,ny` as
`pointer.hit`. The cursor ring was therefore drawn **on the panel border** while the
beam ended at the true intersection, which is off the panel: the further off the panel
the ray pointed, the larger the gap — exactly as reported.
`pointer.hit` is now published **only when `hit.inBounds`** the cursor exists only where
the ray really meets the panel, so it always lies on the beam. `distance` is still
published whenever the ray meets the panel plane, and `beamLength()` no longer requires
`p.hit`, so an edge-aimed beam still reaches the plane. Near-misses keep summoning the
bar (the wide `panelHit(...,0.5,0.9)` check is unchanged), but they no longer press.
## 2. A right-hand press could still act at the left hand's position
Three separate causes, all fixed:
- **The cooldown was global.** `ACTIVATE_COOLDOWN_MS` (250 ms) suppressed *any* press
within 250 ms of the previous one, so a stray gamepad edge from one hand swallowed the
other hand's `select` — and the action then used the first hand's pointer. The cooldown
is now **per input source** (`lastActivateByKey`), so it still de-duplicates the
gamepad/`select` pair of one pinch but never eats the other hand's press.
- **A missing ray silently fell back to the other hand.** If the acting source's ray
could not be resolved, `pickPointerCand` skipped to the next candidate. Now, when a
`preferKey` (the acting source) is supplied, it **owns the shot**: if it is not among
the candidates the pick is `null` and the press MISSES.
- **A null frame meant no re-attribution at all.** `pointerDown()` did
`if (frame) updatePointer(...)`, so with no frame the per-frame pointer (possibly the
other hand) was used silently. It now falls back to `S.lastFrame`, and when the acting
source still yields no pointer it reports `MISS (acting source has no ray)`.
New diagnostics: `stats.pressSrc` (the source key the press was attributed to) and
`stats.pressPtr` (`hand:right HIT` / `hand:right nohit` / `(no ray)`).
## Verification (injected synthetic input sources, live session)
```
both hands on panel, right press pressSrc hand:right pressPtr "hand:right HIT" pressAt 0.78,0.50 seek 79.5%
left press then right 100 ms later pressSrc hand:right pressAt 0.78,0.50 (right not swallowed)
both hands off panel state.pointer.hit = null (no clamped cursor)
both off panel, right press pressSrc hand:right pressPtr "hand:right nohit"
pressAt "MISS (no hit on the panel)" (did NOT use the left hand)
```
# ADDENDUM 12 — one pointer owner: the dominant hand, taken over by a press
The pointer was symmetric: whichever hand acted last owned it, and when neither had
acted the first source in runtime order won — which is the LEFT hand on this runtime, so
a right-handed user's guide line sat on the non-dominant hand. Two hands could also
disturb each other's holds.
## 1. One owner (a mouse-like single cursor)
`pickPointerCand(cands, preferKey, ownerKey, prefHand)`:
```
preferKey (the acting source) -> that source, or null => the press MISSES
owner aiming at the panel -> owner
the preferred hand aiming at the panel -> preferred hand
any hand aiming at the panel -> that hand
owner (not aiming) -> owner
the preferred hand (not aiming) -> preferred hand
-> the first source
```
- **`state.handPref`** (`'right' | 'left'`, default `'right'`) is the pointer's default
owner — the dominant hand. Set from the pre-VR panel ("操作 → 主用手"), persisted in
`jfvr.settings.v1`. It applies at the start of a session; during a session the owner
simply sticks.
- **A press takes the pointer over** (`preferKey`), and the owner then sticks to that
hand while it is aiming.
## 2. The hands can no longer disturb each other
- **Hold exclusion:** while a hold/drag started by one source is in progress
(`grab` / `viewDrag`), a press from the OTHER source is ignored
(`stats.pressAt = 'IGNORED (other hand is holding)'`).
- **Source-matched release:** `pointerUp(src)` only ends a hold that `src` started.
It used to be argument-less, so *either* hand's release cancelled the other hand's
subtitle grab / picture pan. `onSelectEnd(ev)` and `pollGamepads` now pass their
source. `grab` / `viewDrag` carry a `key`.
- **Stale holds:** `dropStaleHolds()` runs each frame and ends a hold whose source has
left `session.inputSources` (hand tracking lost) — otherwise no release would ever
arrive and the drag would stick.
New diagnostics: `stats.handPref`, `stats.owner` (= `stats.activeSrc`), plus the
existing `pressSrc` / `pressPtr`.
## Verification (injected synthetic input sources, live session)
```
default (handPref=right), both hands on panel pointer hand:right HIT 0.78,0.50 (was hand:left)
handPref=left (persisted, fresh session) pointer hand:left HIT 0.22,0.50
left press, release owner hand:left, still left afterwards
right holds picture-pan, left press pressAt "IGNORED (other hand is holding)"
right holds picture-pan, LEFT releases viewDrag stays "armed" (drag survives)
right holds picture-pan, RIGHT releases viewDrag "reset (tap)" (drag ends)
```
+621 -80
View File
File diff suppressed because it is too large Load Diff
+3 -1
View File
@@ -11,7 +11,7 @@
var NS = window.__JFVR = window.__JFVR || {};
NS.version = 'hud-7'; // hud-7 = ADDENDUM 8b (live picture pan: transform/space, hold-to-drag, tap-to-reset)
NS.version = 'hud-11'; // hud-11 = ADDENDUM 12 (single pointer owner: dominant hand by default, press takes over, hold exclusion) + 11/10/9
NS.__core = true;
/* ------------------------------------------------------------------ *
@@ -51,6 +51,7 @@
subtitles: true,
ctlLock: false, // ADDENDUM 7: control-bar lock badge (per VR session)
subMode: 'hud', // 'hud' | 'sphere' | 'off'
subLayout: 'auto', // ADDENDUM 9: 'auto' | 'dual' | 'mono' subtitle layout
subAlpha: 'premultiplied', // 'premultiplied' | 'straight'
subOpacity: 0.85, // 0..1, applied to the subtitle layer live
// y is the quad CENTRE; negative = below eye level. -0.42 keeps the strip
@@ -66,6 +67,7 @@
subAngleDeg: 69.2,
controls: true,
handPref: 'right', // ADDENDUM 12: dominant hand = the pointer's default owner
ctlAutoHideMs: 4000, // in-VR control bar auto-hide
// written every XR frame by 90-main.js, read by modules
+379 -34
View File
@@ -57,6 +57,30 @@
var DIFF_SUM = 24; // per-sample channel delta that counts as a changed sample
var HUD_TEX_W = 2048; // target texture width (≈30 px/deg for a 2 m strip @ 1.45 m)
var HUD_TEX_ASPECT = 0.28; // texH = texW * this (room for multi-line subtitles)
// ADDENDUM 9 — dual-eye vs normal (mono) subtitle layout, auto-detected.
// The libass canvas holds EITHER a stereo left/right pair (two copies, one per
// eye; the left half is the left eye) OR a single normal subtitle positioned
// across the whole frame.
//
// IMPORTANT: dual-eye is NOT a pixel mirror. Both eyes carry the SAME text at
// mirrored POSITIONS; the glyphs are not flipped, so comparing left[x] with
// right[W-1-x] fails badly (measured mean delta ~258/1020, i.e. "not a mirror").
// What IS true is that the two halves' ink COLUMN profiles match under a small
// horizontal shift, because they are the same line of text. Measured on device:
// dual -> normalised copy distance 0.02, mono -> 1.00, seam ink present only for
// mono. So the copy test is the discriminator.
var CENTER_BAND_PX = 40; // source px around the eye seam that count as "centre ink"
var COPY_TOL = 0.25; // column-profile copy distance for "two copies of one line"
// (measured: dual 0.02-0.08, mono 1.00)
// ADDENDUM 10 — Jellyfin only routes ssa/ass through SubtitlesOctopus (the
// libass canvas). Every other text codec (subrip, vtt, ...) is drawn by the
// BROWSER from a TextTrack (`video.addTextTrack` + mode 'showing'), so there is
// no canvas at all to sample. The HUD then paints the active cue itself into a
// private canvas and feeds it through the same crop/upload pipeline.
var NATIVE_W = 1920, NATIVE_H = 1080;
var NATIVE_FONT_H = 0.052; // cue font size as a fraction of the canvas height
var NATIVE_LINE_H = 1.32; // line advance
var NATIVE_MAX_W = 0.9; // max text width as a fraction of the canvas width
var SPHERE_TEX_W = 4096; // legacy equirect layer size
var SPHERE_TEX_H = 2048;
var OPACITY_DEFAULT = 0.85; // dim the strip below full opacity by default
@@ -123,8 +147,19 @@
cropCanvas: null,
cropCtx: null,
srcW: 0, srcH: 0, halfW: 1,
srcW: 0, srcH: 0, halfW: 1, fullW: 1, eyeW: 1,
texW: 0, texH: 0,
// ADDENDUM 9: layout is the user setting ('auto' | 'dual' | 'mono'); dualEye
// is the resolved result (auto-detected when layout === 'auto').
layout: (S.subLayout === 'dual' || S.subLayout === 'mono') ? S.subLayout : 'auto',
dualEye: true,
mismatch: 0, // consecutive samples disagreeing with dualEye (debounce)
copyND: null, // diagnostics: normalised column-profile copy distance
centerInk: null, // diagnostics: ink pixels in the seam band
colL: null, colR: null, // reused column-profile scratch (half width each)
// ADDENDUM 10: where the HUD's source pixels come from.
srcKind: 'none', // 'libass' | 'native' | 'none'
nativeCanvas: null, nativeCtx: null, nativeText: null,
bbox: null,
lastSample: null,
hasInk: false,
@@ -145,6 +180,174 @@
catch (e) { return null; }
}
/* ================================================================
* ADDENDUM 10 — native (non-libass) subtitle source
* Jellyfin asks the browser to render subrip/vtt cues from a TextTrack, so the
* HUD has no canvas to read. Read the active cue, paint it into a private
* canvas, and let the normal crop/upload path take over from there.
* ============================================================== */
function ensureNativeCanvas() {
if (!M.nativeCanvas) {
try {
M.nativeCanvas = document.createElement('canvas');
M.nativeCanvas.width = NATIVE_W;
M.nativeCanvas.height = NATIVE_H;
M.nativeCtx = M.nativeCanvas.getContext('2d');
// Kept in the DOM but hidden: Canvas2D works detached too, and this makes
// the exact pixels the HUD uploads inspectable from DevTools.
M.nativeCanvas.className = 'jfvr-native-subs';
M.nativeCanvas.style.display = 'none';
(document.body || document.documentElement).appendChild(M.nativeCanvas);
} catch (e) { M.nativeCtx = null; }
}
return !!M.nativeCtx;
}
// Jellyfin renders non-ASS text subtitles one of two ways depending on the
// build: into a TextTrack (video.addTextTrack + mode 'showing', the browser
// shows it through ::cue) or into a DOM overlay (div.videoSubtitlesInner).
// Both leave no canvas behind.
function hasNativeTracks() {
try {
var v = S.video;
if (v && v.textTracks) {
for (var i = 0; i < v.textTracks.length; i++) {
var tr = v.textTracks[i];
if (tr && tr.mode === 'showing' && tr.cues && tr.cues.length) return true;
}
}
if (document.querySelector('.videoSubtitlesInner')) return true;
} catch (e) {}
return false;
}
function activeNativeText() {
var out = [];
try {
var v = S.video;
if (v && v.textTracks) {
for (var i = 0; i < v.textTracks.length; i++) {
var tr = v.textTracks[i];
if (!tr || tr.mode !== 'showing') continue;
var cues = tr.activeCues; // maintained by the browser
if (!cues) continue;
for (var j = 0; j < cues.length; j++) {
var s = cues[j] && cues[j].text;
if (s) out.push(s);
}
}
}
if (!out.length) {
var d = document.querySelector('.videoSubtitlesInner, .videoSubtitles');
if (d) { var t = d.innerText || ''; if (t) out.push(t); }
}
} catch (e) { return ''; }
return out.join('\n');
}
function cleanCueText(s) {
return String(s == null ? '' : s)
.replace(/\{[^}]*\}/g, '') // ASS override tags
.replace(/<[^>]*>/g, '') // html-ish tags
.replace(/&nbsp;/g, ' ').replace(/&amp;/g, '&')
.replace(/&lt;/g, '<').replace(/&gt;/g, '>')
.replace(/[\u200e\u200f\u202a-\u202e]/g, '') // bidi control marks
.replace(/\r/g, '')
.replace(/[ \t]+\n/g, '\n')
.replace(/\n{3,}/g, '\n\n')
.trim();
}
function wrapCueLines(ctx, text, maxW) {
var out = [];
var paras = text.split('\n');
for (var p = 0; p < paras.length; p++) {
var para = paras[p];
if (!para) { out.push(''); continue; }
var line = '';
for (var i = 0; i < para.length; i++) {
var ch = para[i];
if (line && ctx.measureText(line + ch).width > maxW) {
var sp = line.lastIndexOf(' ');
if (sp > 0 && /[A-Za-z0-9]/.test(line.charAt(line.length - 1))) {
out.push(line.slice(0, sp));
line = line.slice(sp + 1);
} else {
out.push(line);
line = '';
}
}
line += ch;
}
out.push(line);
}
return out.length ? out : [''];
}
function drawNativeCue(text) {
var c = M.nativeCanvas, ctx = M.nativeCtx;
if (!c || !ctx) return;
var W = c.width, H = c.height;
ctx.clearRect(0, 0, W, H);
if (!text) return;
var fs = Math.max(12, Math.round(H * NATIVE_FONT_H));
ctx.font = '600 ' + fs + 'px "Noto Sans SC","Noto Sans CJK SC","PingFang SC",' +
'"Microsoft YaHei",system-ui,Arial,sans-serif';
ctx.textAlign = 'center';
ctx.textBaseline = 'middle';
ctx.lineJoin = 'round';
ctx.miterLimit = 2;
var lines = wrapCueLines(ctx, text, W * NATIVE_MAX_W);
var lh = fs * NATIVE_LINE_H;
var y0 = H / 2 - (lines.length - 1) * lh / 2;
ctx.lineWidth = Math.max(2, Math.round(fs * 0.1));
ctx.strokeStyle = 'rgba(0,0,0,0.95)';
ctx.fillStyle = '#ffffff';
for (var i = 0; i < lines.length; i++) {
ctx.strokeText(lines[i], W / 2, y0 + i * lh);
ctx.fillText(lines[i], W / 2, y0 + i * lh);
}
}
// Decide, per frame, which canvas the HUD samples. libass wins when Jellyfin
// created one; otherwise the native cue canvas. Returns false when there is no
// subtitle source at all.
function updateSource() {
var lib = getCanvas();
if (lib && lib.width > 0 && lib.height > 0) {
if (M.srcKind !== 'libass') {
M.srcKind = 'libass';
M.canvas = lib;
M.nativeText = null;
M.lastSample = null; M.hasInk = false; M.bbox = null;
computeTexDims();
M.needRebuild = true;
}
return true;
}
if (!hasNativeTracks()) {
if (M.srcKind !== 'none') { M.srcKind = 'none'; M.hasInk = false; M.bbox = null; M.dirty = true; }
return false;
}
if (!ensureNativeCanvas()) return false;
var text = cleanCueText(activeNativeText());
if (text !== M.nativeText) {
M.nativeText = text;
drawNativeCue(text);
M.dirty = true;
}
if (M.srcKind !== 'native') {
M.srcKind = 'native';
M.canvas = M.nativeCanvas;
M.dualEye = false; // our own canvas is always one centred line
M.mismatch = 0;
M.lastSample = null; M.hasInk = false; M.bbox = null;
computeTexDims();
M.needRebuild = true;
}
return true;
}
function getHolder(video) {
var holder = null;
try { holder = document.querySelector('.videoPlayerContainer'); } catch (e) {}
@@ -159,6 +362,13 @@
st.subLayer = !!M.layer;
st.subPrepared = !!M.prepared;
st.subCanvasSize = M.canvas ? (M.canvas.width + 'x' + M.canvas.height) : null;
st.subSource = M.srcKind;
st.subNativeText = M.nativeText ? M.nativeText.slice(0, 60) : null;
st.subLayout = M.layout;
st.subDual = M.dualEye;
st.subEyeW = M.eyeW;
st.subCopyND = (M.copyND == null) ? null : +M.copyND.toFixed(3);
st.subCenterInk = M.centerInk;
st.subTex = (M.texW && M.texH) ? (M.texW + 'x' + M.texH) : null;
st.subInk = M.hasInk;
st.subCrop = M.bbox ? [
@@ -183,6 +393,10 @@
* ============================================================== */
function prepare() {
if (M.prepared) return; // idempotent via a live flag, not a backup check
// Only libass needs the box blown up so it re-renders bigger. The native cue
// canvas is ours, so resizing the player would be pointless (and would move
// the browser's own cue overlay around).
if (!getCanvas()) { log('prepare skipped: native subtitle source'); return; }
var video = S.video || null;
var holder = getHolder(video);
try {
@@ -253,13 +467,17 @@
var maxTex = maxTexSize();
M.srcW = cw;
M.srcH = ch;
M.halfW = Math.max(1, Math.floor(cw / 2)); // mono HUD uses the LEFT half only
M.fullW = Math.max(1, cw);
M.halfW = Math.max(1, Math.floor(cw / 2));
// ADDENDUM 9: the HUD samples ONE eye's subtitle region. Dual-eye packs both
// eyes into the canvas; a normal subtitle uses the whole frame.
M.eyeW = M.dualEye ? M.halfW : M.fullW;
if (M.mode === 'sphere') {
M.texW = Math.max(2, Math.min(SPHERE_TEX_W, maxTex, cw || 2));
M.texH = Math.max(2, Math.min(SPHERE_TEX_H, maxTex, ch || 2));
} else {
M.texW = Math.max(2, Math.min(HUD_TEX_W, maxTex, M.halfW));
M.texW = Math.max(2, Math.min(HUD_TEX_W, maxTex, M.eyeW));
M.texH = Math.max(2, Math.min(maxTex, Math.round(M.texW * HUD_TEX_ASPECT)));
}
}
@@ -415,6 +633,7 @@
if (S.subMode === 'hud' || S.subMode === 'sphere' || S.subMode === 'off') M.mode = S.subMode;
if (S.subAlpha === 'premultiplied' || S.subAlpha === 'straight') M.alpha = S.subAlpha;
readLayoutSetting();
if (isNum(S.subOpacity)) M.opacity = clamp01(S.subOpacity);
else S.subOpacity = M.opacity;
if (typeof S.subLockAngle === 'boolean') M.lockAngle = S.subLockAngle;
@@ -424,10 +643,6 @@
if (M.mode === 'off') { log('create: subtitle mode off'); return null; }
var canvas = getCanvas();
if (!canvas || !canvas.width || !canvas.height) {
setErr('no libass canvas'); log('create: no <canvas.libassjs-canvas>'); return null;
}
if (typeof window.XRWebGLBinding !== 'function') {
setErr('no XRWebGLBinding'); log('create: no XRWebGLBinding'); return null;
}
@@ -439,11 +654,17 @@
var gl = NS.xrGl(true);
if (!gl) { setErr('no webgl2'); log('create: no webgl2 context'); return null; }
M.canvas = canvas;
M.gl = gl;
M.glCanvas = gl.canvas;
M.uploads = 0;
M.samples = 0;
// ADDENDUM 10: the source may be the libass canvas OR our own cue canvas.
// Start on the native canvas so create() works even before Jellyfin has
// created its renderer, then updateSource() upgrades to libass if it shows up.
ensureNativeCanvas();
M.canvas = M.nativeCanvas;
M.srcKind = 'none';
updateSource();
computeTexDims();
if (!M.texW || !M.texH) { setErr('bad canvas size'); log('create: bad canvas size'); destroy(); return null; }
@@ -490,13 +711,15 @@
var layer = attachLayer(session);
if (!layer) throw new Error('layer creation returned null');
M.needRebuild = false; // the source was already resolved above
sample(); // seed bbox / hasInk before the first XR frame
M.dirty = true;
setErr(null);
updateStats();
log('subtitles layer created:', M.mode, M.texW + 'x' + M.texH,
'alpha=' + M.alpha, 'canvas=' + canvas.width + 'x' + canvas.height);
'alpha=' + M.alpha, 'source=' + M.srcKind,
'canvas=' + (M.canvas ? M.canvas.width + 'x' + M.canvas.height : 'none'));
return layer;
} catch (e) {
setErr((e && e.message) || e);
@@ -522,9 +745,11 @@
} catch (e) {}
M.gl = null; M.glCanvas = null; M.prog = null; M.tex = null; M.fbo = null;
M.binding = null; M.session = null; M.canvas = null;
M.srcKind = 'none'; M.nativeText = null;
M.sampleCanvas = null; M.sampleCtx = null;
M.cropCanvas = null; M.cropCtx = null;
M.srcW = 0; M.srcH = 0; M.halfW = 1; M.texW = 0; M.texH = 0;
M.srcW = 0; M.srcH = 0; M.halfW = 1; M.fullW = 1; M.eyeW = 1; M.texW = 0; M.texH = 0;
M.copyND = null; M.centerInk = null; M.colL = null; M.colR = null; M.mismatch = 0;
M.bbox = null; M.lastSample = null;
M.hasInk = false; M.lastDrawnInk = false;
M.dirty = false; M.needRebuild = false;
@@ -571,6 +796,24 @@
return true;
}
// ADDENDUM 9: normalised distance between the two halves' ink column profiles at
// their best horizontal alignment. ~0 for dual-eye (same line, two copies), ~1
// for a normal subtitle (the halves are different parts of one line).
function columnCopyND(colL, colR, half) {
var D = Math.round(half * 0.4), best = 1e9;
for (var lag = -D; lag <= D; lag++) {
var diff = 0, tot = 0;
for (var i = 0; i < half; i++) {
var j = i + lag;
var r = (j >= 0 && j < half) ? colR[j] : 0;
diff += Math.abs(colL[i] - r);
tot += colL[i] + r;
}
if (tot > 0) { var nd = diff / tot; if (nd < best) best = nd; }
}
return (best === 1e9) ? 1 : best;
}
function sample() {
var canvas = M.canvas;
var ctx = M.sampleCtx;
@@ -595,11 +838,20 @@
var d = img.data;
var prev = M.lastSample;
var hud = (M.mode === 'hud');
var half = hud ? (SAMPLE_W >> 1) : SAMPLE_W; // HUD bbox: LEFT half only
var minX = half + 1, minY = SAMPLE_H + 1, maxX = -1, maxY = -1;
var half = SAMPLE_W >> 1;
var diff = 0;
var sx = cw / SAMPLE_W, sy = ch / SAMPLE_H;
var padX = PAD_PX / sx, padY = PAD_PX / sy;
var band = Math.max(1, Math.round(CENTER_BAND_PX / sx));
// Track the ink box of each REGION separately so the layout can be resolved
// before cropping: l* = left half, r* = right half, centerInk = the seam.
var lMinX = SAMPLE_W + 1, lMaxX = -1, lMinY = SAMPLE_H + 1, lMaxY = -1;
var rMinX = SAMPLE_W + 1, rMaxX = -1, rMinY = SAMPLE_H + 1, rMaxY = -1;
var centerInk = 0;
if (!M.colL) { M.colL = new Int32Array(half); M.colR = new Int32Array(half); }
var colL = M.colL, colR = M.colR;
colL.fill(0); colR.fill(0);
for (var y = 0; y < SAMPLE_H; y++) {
var row = y * SAMPLE_W * 4;
@@ -610,11 +862,21 @@
Math.abs(d[i + 2] - prev[i + 2]) + Math.abs(d[i + 3] - prev[i + 3]);
if (dd > DIFF_SUM) diff++;
}
if (hud && x < half && d[i + 3] > INK_ALPHA) {
if (x < minX) minX = x;
if (x > maxX) maxX = x;
if (y < minY) minY = y;
if (y > maxY) maxY = y;
if (!hud || d[i + 3] <= INK_ALPHA) continue;
if (x < half) {
if (x < lMinX) lMinX = x;
if (x > lMaxX) lMaxX = x;
if (y < lMinY) lMinY = y;
if (y > lMaxY) lMaxY = y;
if (x > half - band) centerInk++;
colL[x]++;
} else {
if (x < rMinX) rMinX = x;
if (x > rMaxX) rMaxX = x;
if (y < rMinY) rMinY = y;
if (y > rMaxY) rMaxY = y;
if (x < half + band) centerInk++;
colR[x - half]++;
}
}
}
@@ -622,20 +884,63 @@
M.lastSample = d;
if (!prev || diff > DIFF_SAMPLES) M.dirty = true; // content changed -> re-upload
if (!hud) { updateStats(); return; }
if (hud) {
if (maxX < 0) {
if (M.hasInk) { M.hasInk = false; M.dirty = true; } // clear the strip once
} else {
var raw = quantise({
x0: Math.max(0, (minX - padX) * sx),
y0: Math.max(0, (minY - padY) * sy),
x1: Math.min(cw / 2, (maxX + 1 + padX) * sx),
y1: Math.min(ch, (maxY + 1 + padY) * sy)
});
if (!M.hasInk) { M.hasInk = true; M.bbox = raw; M.dirty = true; }
else if (updateBBox(raw)) M.dirty = true;
// ---- ADDENDUM 9: resolve dual-eye vs mono ----------------------------
// dual-eye: two copies of the same line -> no ink at the seam AND the two
// halves' column profiles match under a horizontal shift. mono: one line
// across the whole frame -> ink crosses the seam and the halves do not match.
var hasLeft = lMaxX >= 0, hasRight = rMaxX >= 0;
var hasInk = hasLeft || hasRight || centerInk > 0;
if (hasInk) {
M.centerInk = centerInk;
var dual = M.dualEye;
if (M.srcKind === 'native') { M.copyND = null; dual = false; } // our own canvas is always mono
else if (M.layout === 'dual') dual = true;
else if (M.layout === 'mono') dual = false;
else if (hasLeft && hasRight) {
if (centerInk > 0) { M.copyND = null; dual = false; }
else { M.copyND = columnCopyND(colL, colR, half); dual = (M.copyND < COPY_TOL); }
}
else dual = !!hasLeft; // ink in ONE half -> dual (left) else mono (right)
if (dual !== M.dualEye) {
// Debounce: a layout switch rebuilds the layer, so require two
// consecutive samples before believing it (never observed to flicker on
// real content, but cheap insurance against a rebuild every sample).
M.mismatch = (M.mismatch || 0) + 1;
if (M.mismatch >= 2) {
M.mismatch = 0;
M.dualEye = dual;
computeTexDims();
M.needRebuild = true; // the texture width changed with the region
}
} else M.mismatch = 0;
}
// ---- crop to the ink box of the RESOLVED region ----------------------
var x0s, y0s, x1s, y1s;
if (hasInk) {
if (M.dualEye && hasLeft) {
x0s = lMinX; y0s = lMinY; x1s = lMaxX; y1s = lMaxY;
} else {
x0s = hasLeft ? lMinX : rMinX;
x1s = hasRight ? rMaxX : lMaxX;
y0s = Math.min(hasLeft ? lMinY : Infinity, hasRight ? rMinY : Infinity);
y1s = Math.max(hasLeft ? lMaxY : -1, hasRight ? rMaxY : -1);
}
}
if (!hasInk) {
if (M.hasInk) { M.hasInk = false; M.dirty = true; } // clear the strip once
} else {
var raw = quantise({
x0: Math.max(0, (x0s - padX) * sx),
y0: Math.max(0, (y0s - padY) * sy),
x1: Math.min(M.eyeW, (x1s + 1 + padX) * sx),
y1: Math.min(ch, (y1s + 1 + padY) * sy)
});
if (!M.hasInk) { M.hasInk = true; M.bbox = raw; M.dirty = true; }
else if (updateBBox(raw)) M.dirty = true;
}
updateStats();
}
@@ -648,9 +953,9 @@
var b = M.bbox;
if (!gl || !b) return;
var cw = Math.max(1, Math.min(M.halfW, Math.round(b.x1 - b.x0)));
var cw = Math.max(1, Math.min(M.eyeW, Math.round(b.x1 - b.x0)));
var ch = Math.max(1, Math.round(b.y1 - b.y0));
var sx = Math.max(0, Math.min(Math.max(0, M.halfW - cw), Math.round(b.x0)));
var sx = Math.max(0, Math.min(Math.max(0, M.eyeW - cw), Math.round(b.x0)));
var sy = Math.max(0, Math.min(Math.max(0, M.srcH - ch), Math.round(b.y0)));
var crop = M.cropCanvas, cctx = M.cropCtx;
@@ -658,7 +963,7 @@
cctx.clearRect(0, 0, cw, ch);
cctx.drawImage(M.canvas, sx, sy, cw, ch, 0, 0, cw, ch);
var scale = M.texW / M.halfW; // texture px per source px
var scale = M.texW / M.eyeW; // texture px per source px
var dw = cw * scale, dh = ch * scale;
var fit = Math.min(1, tw / dw, th / dh); // never overflow the colorTexture
dw *= fit; dh *= fit;
@@ -747,7 +1052,11 @@
: (frame.predictedDisplayTime != null ? frame.predictedDisplayTime
: (window.performance ? performance.now() : Date.now()));
if (t - M.lastSampleT >= SAMPLE_MS || M.lastSampleT < 0) {
// Keep the source current every frame (native cue text changes on its own).
var hasSrc = updateSource();
if (!hasSrc) {
if (M.hasInk) { M.hasInk = false; M.bbox = null; M.dirty = true; }
} else if (t - M.lastSampleT >= SAMPLE_MS || M.lastSampleT < 0) {
M.lastSampleT = t;
sample();
}
@@ -767,6 +1076,38 @@
/* ================================================================
* settings (safe before or during a session)
* ============================================================== */
// ADDENDUM 9 — read state.subLayout into the module. A forced layout resolves
// immediately; 'auto' keeps the current guess until sample() sees ink.
function readLayoutSetting() {
if (S.subLayout === 'dual' || S.subLayout === 'mono') M.layout = S.subLayout;
else { M.layout = 'auto'; if (S.subLayout !== 'auto') S.subLayout = 'auto'; }
if (M.layout !== 'auto') { M.dualEye = (M.layout === 'dual'); M.mismatch = 0; }
}
// Force home the dual-eye / mono decision, or go back to auto-detection.
function setLayout(mode) {
if (mode !== 'auto' && mode !== 'dual' && mode !== 'mono') return;
M.layout = mode;
S.subLayout = mode;
if (!M.canvas) { updateStats(); return; }
if (mode === 'auto') {
M.bbox = null; M.hasInk = false; M.mismatch = 0; M.dirty = true; // re-detect from the next sample
} else {
var forced = (mode === 'dual');
if (forced !== M.dualEye) {
M.dualEye = forced;
M.bbox = null; M.hasInk = false;
M.mismatch = 0;
computeTexDims();
M.needRebuild = true;
M.dirty = true;
}
}
updateStats();
}
function getLayout() { return M.layout; }
function setMode(mode) {
if (mode !== 'hud' && mode !== 'sphere' && mode !== 'off') return;
var changed = (M.mode !== mode);
@@ -981,6 +1322,7 @@
else if (S.subMode) S.subMode = M.mode;
if (S.subAlpha === 'premultiplied' || S.subAlpha === 'straight') M.alpha = S.subAlpha;
else if (S.subAlpha) S.subAlpha = M.alpha;
readLayoutSetting();
if (isNum(S.subOpacity)) M.opacity = clamp01(S.subOpacity);
else S.subOpacity = M.opacity;
if (typeof S.subLockAngle === 'boolean') M.lockAngle = S.subLockAngle;
@@ -1006,6 +1348,8 @@
create: create,
destroy: destroy,
setMode: setMode,
setLayout: setLayout,
getLayout: getLayout,
setAlphaMode: setAlphaMode,
setPlacement: setPlacement,
getPlacement: getPlacement,
@@ -1017,6 +1361,7 @@
setOpacity: setOpacity,
getOpacity: getOpacity,
getLayer: getLayer,
getMode: getMode
getMode: getMode,
getSourceKind: function () { return M.srcKind; }
};
})();
+4 -1
View File
@@ -1532,8 +1532,11 @@
return d > 0 ? d : null;
}
// ADDENDUM 11b: the beam reaches the panel plane whenever the ray meets it,
// even when the intersection is off the panel (there is simply no cursor then).
// Previously it fell back to a fixed length without a hit, so an edge-aimed
// beam stopped short of the plane while the cursor was clamped to the border.
function beamLength(p) {
if (!p.hit) return BEAM_L_DEFAULT;
var d = num(p.distance);
if (d == null) d = panelDistance(p);
if (d == null || !(d > BEAM_L_MIN)) return BEAM_L_DEFAULT;
+235 -44
View File
@@ -20,11 +20,26 @@
* rAF broker
* ------------------------------------------------------------------
* While an immersive XR session runs, window.requestAnimationFrame is
* COMPLETELY stopped (measured: 0 callbacks in 6s) and Jellyfin renders
* libass subtitles from an rAF loop, so subtitles freeze. Hold callbacks
* in our own queue while in XR and drive them from
* video.requestVideoFrameCallback (which still fires, in step with the
* video). Outside XR delegate to the native implementation.
* COMPLETELY stopped (measured: 0 callbacks in 6s). Jellyfin renders libass
* subtitles from an rAF loop, so that froze the subtitle canvas.
*
* The FIRST fix was to queue callbacks in XR and pump the queue from
* video.requestVideoFrameCallback. That is not enough on Jellyfin 12.x: a
* callback that is ALREADY PENDING in the NATIVE scheduler when the session
* starts is destroyed with the native scheduler and never runs again, and
* SubtitlesOctopus starts ONE long-lived rAF loop when the video loads whose
* id is never reset. Verified on device: instance.rafId stuck on a native id
* (1150), oneshotState.eventStart frozen ~7 min behind the playhead, zero
* oneshot-render requests for the whole session -> the whole prerender chain
* stalls and the HUD shows the last line drawn before VR started.
*
* So the queue is used ALWAYS, even outside XR, and drained by a native rAF
* pump. Nothing is ever left pending natively, so nothing can be lost when the
* session starts; startRafPump just swaps the driver. In XR the native pump
* stops and video.requestVideoFrameCallback drives the queue (it still fires,
* in step with the video); a slow timer also drives it when the video is not
* presenting frames (paused / stalled / ended), so a paused seek or a track
* switch still redraws instead of leaving a stale line on screen.
* ================================================================== */
var RAFQ = {
inXR: false,
@@ -33,24 +48,13 @@
queue: new Map(),
nextId: 1000000000,
pumped: 0,
nativeId: null,
lastPump: 0,
err: null
};
try {
window.requestAnimationFrame = function (cb) {
if (!RAFQ.inXR) return RAFQ.native(cb);
var id = RAFQ.nextId++;
RAFQ.queue.set(id, cb);
return id;
};
window.cancelAnimationFrame = function (id) {
if (RAFQ.queue.has(id)) { RAFQ.queue.delete(id); return; }
return RAFQ.nativeCancel(id);
};
} catch (e) { log('rAF broker install failed', e); }
function pumpRaf(t) {
if (!RAFQ.inXR || !RAFQ.queue.size) return;
function drainRaf(t) {
if (!RAFQ.queue.size) return;
var cbs = [];
RAFQ.queue.forEach(function (cb) { cbs.push(cb); });
RAFQ.queue.clear();
@@ -60,24 +64,66 @@
}
}
// Native driver (outside XR). Exactly ONE native callback is kept in flight so
// that a rAF which re-schedules itself cannot multiply the driver (an earlier
// version nulled nativeId before draining, re-armed on every self-rescheduling
// callback AND again at the end -> exponential native callbacks -> frozen page).
function nativeRafStep(t) {
if (RAFQ.inXR) { RAFQ.nativeId = null; return; } // the XR driver owns the queue
RAFQ.nativeId = RAFQ.native(nativeRafStep); // re-arm BEFORE draining
drainRaf(t);
if (!RAFQ.queue.size && !RAFQ.inXR) { // nothing left -> stop
try { RAFQ.nativeCancel(RAFQ.nativeId); } catch (e) {}
RAFQ.nativeId = null;
}
}
function ensureNativeRaf() {
if (RAFQ.inXR || RAFQ.nativeId != null) return;
RAFQ.nativeId = RAFQ.native(nativeRafStep);
}
try {
window.requestAnimationFrame = function (cb) {
if (typeof cb !== 'function') return 0;
var id = RAFQ.nextId++;
RAFQ.queue.set(id, cb);
if (!RAFQ.inXR) ensureNativeRaf();
return id;
};
window.cancelAnimationFrame = function (id) {
if (RAFQ.queue.has(id)) { RAFQ.queue.delete(id); return; }
return RAFQ.nativeCancel(id);
};
} catch (e) { log('rAF broker install failed', e); }
function startRafPump(video) {
// NOTE: do NOT clear the queue. The callback that a naive broker loses at
// session start is sitting in it, and it is the one that keeps Jellyfin's
// subtitle loop alive. The pending NATIVE driver id is dropped: that callback
// is lost with the native scheduler anyway, and clearing the id lets the
// driver restart cleanly after the session ends.
RAFQ.nativeId = null;
RAFQ.inXR = true;
RAFQ.queue.clear();
RAFQ.pumped = 0;
RAFQ.lastPump = 0;
RAFQ.err = null;
var stopped = false;
S.stats.rafPump = video && video.requestVideoFrameCallback ? 'videoFrameCallback' : 'interval';
if (video && video.requestVideoFrameCallback) {
var step = function (now) {
if (stopped) return;
pumpRaf(now);
RAFQ.lastPump = performance.now();
drainRaf(now);
video.requestVideoFrameCallback(step);
};
video.requestVideoFrameCallback(step);
RAFQ.stop = function () { stopped = true; };
return;
}
var id = setInterval(function () { pumpRaf(performance.now()); }, 16);
// Idle fallback: rVFC stops while the video is paused / stalled / ended.
var id = setInterval(function () {
if (stopped || !RAFQ.inXR) return;
if (performance.now() - RAFQ.lastPump > 60) drainRaf(performance.now());
}, 50);
RAFQ.stop = function () { stopped = true; clearInterval(id); };
}
@@ -85,9 +131,9 @@
try { if (RAFQ.stop) RAFQ.stop(); } catch (e) {}
RAFQ.stop = null;
RAFQ.inXR = false;
// hand leftovers back to the real scheduler so nothing is stranded
RAFQ.queue.forEach(function (cb) { try { RAFQ.native(cb); } catch (e) {} });
RAFQ.queue.clear();
RAFQ.lastPump = 0;
// hand the queue back to the native driver so nothing is stranded
ensureNativeRaf();
}
/* ================================================================== *
@@ -124,6 +170,9 @@
S.subOpacity = num(o.subOpacity, 0.1, 1.0, S.subOpacity);
S.ctlAutoHideMs = num(o.ctlAutoHideMs, 1000, 10000, S.ctlAutoHideMs);
if (o.subMode === 'hud' || o.subMode === 'sphere') S.subMode = o.subMode;
// subLayout is deliberately NOT restored: a persisted manual override
// silently ruined the default dual-eye content once already (forced 'mono'
// made the HUD show both eyes side by side). Layout always starts on 'auto'.
// ADDENDUM 8: how the picture aim is applied. 'space' is the live path; the
// 'rebuild' escape hatch exists because mutating a media layer's transform is
// ignored by this runtime, so if an offset reference space turns out not to be
@@ -138,6 +187,7 @@
if (o.subAlpha === 'premultiplied' || o.subAlpha === 'straight') S.subAlpha = o.subAlpha;
if (o.projection === '180' || o.projection === '360') S.projection = o.projection;
if (o.stereo === 'mono' || o.stereo === 'sbs' || o.stereo === 'ou') S.stereo = o.stereo;
if (o.handPref === 'right' || o.handPref === 'left') S.handPref = o.handPref;
if (typeof o.controls === 'boolean') S.controls = o.controls;
// NOTE: `subtitles` is deliberately NOT restored. Persisting it meant a
// session where the user toggled subtitles off came back with them off,
@@ -161,6 +211,7 @@
subAlpha: S.subAlpha,
projection: S.projection,
stereo: S.stereo,
handPref: S.handPref,
videoRotMode: S.videoRotMode,
controls: S.controls,
ctlAutoHideMs: S.ctlAutoHideMs
@@ -411,10 +462,26 @@
body.appendChild(segRow('显示', function () { return S.subtitles ? 'on' : 'off'; },
function (v) { S.subtitles = v === 'on'; saveSettings(); },
[{ label: '开', value: 'on' }, { label: '关', value: 'off' }]));
// ADDENDUM 9: dual-eye ASS vs a normal (mono) subtitle. 'auto' detects from
// the libass canvas; force it only if the detection gets it wrong.
body.appendChild(segRow('字幕布局', function () { return S.subLayout || 'auto'; },
function (v) {
S.subLayout = v;
if (NS.Subtitles && NS.Subtitles.setLayout) { try { NS.Subtitles.setLayout(v); } catch (e) {} }
saveSettings();
},
[{ label: '自动', value: 'auto' }, { label: '双目', value: 'dual' }, { label: '普通', value: 'mono' }]));
body.appendChild(NS.makeEl('div', 'jfvr-hint',
'字幕的位置 / 距离 / 大小 / 不透明度都在 VR 里用控制条调(控制条最下面一行),' +
'调好会自动记住。VR 空间右侧有个锁图标:锁上后面板不会再被唤出。'));
body.appendChild(NS.makeEl('div', 'jfvr-sec', '操作'));
body.appendChild(segRow('主用手', function () { return S.handPref || 'right'; },
function (v) { S.handPref = v; saveSettings(); },
[{ label: '右手', value: 'right' }, { label: '左手', value: 'left' }]));
body.appendChild(NS.makeEl('div', 'jfvr-hint',
'引导线/光标默认在主用手上;另一只手按下会立刻接管那一次操作。'));
// ADDENDUM 8: the picture-pan mechanism. Only touch this if panning does not
// move the picture live (see stats.videoRotMode).
body.appendChild(segRow('画面旋转', function () { return S.videoRotMode || 'auto'; },
@@ -705,6 +772,7 @@
try {
updatePointer(frame);
pollGamepads(frame);
dropStaleHolds();
updatePanelPose(frame, dt);
updateGrabFrame(frame);
updateViewDrag(frame);
@@ -792,6 +860,8 @@
S.ctlLock = false;
panelPose = null;
pointer = { visible: false, beam: false, source: 'none', origin: { x: 0, y: 0, z: 0 }, dir: { x: 0, y: 0, z: -1 }, hit: null };
activeSrcKey = null; // ADDENDUM 11: fresh hand choice every session
lastActivateByKey = {};
S.pointer = null;
S.session = null;
S.refSpace = null;
@@ -978,7 +1048,63 @@
// the only mode (an index-finger ray carries fingertip jitter straight into
// the cursor, and nobody wants to point with one finger).
function updatePointer(frame) {
// Identity for an input source that survives the runtime re-listing it.
function srcKey(src) {
if (!src) return null;
return (src.hand ? 'hand' : 'ctrl') + ':' + (src.handedness || '?');
}
function candByKey(cands, key) {
for (var i = 0; i < cands.length; i++) if (cands[i].key === key) return cands[i];
return null;
}
function candByHand(cands, hand) {
for (var i = 0; i < cands.length; i++) if (cands[i].hand === hand) return cands[i];
return null;
}
// ADDENDUM 12 — pure: choose which input source owns the pointer.
// cands [{ key, hand, onPanel, ... }] in runtime order
// preferKey the source that is acting right now (select / gamepad press)
// ownerKey the source that owns the pointer (last accepted press)
// prefHand the dominant hand ('right' | 'left') — the default owner
//
// The acting source OWNS the shot: if it cannot be resolved the pick is null and
// the press MISSES (never another hand's position). Otherwise:
// owner aiming -> preferred hand aiming -> any hand aiming -> owner ->
// preferred hand -> first.
// A mouse-style single cursor: exactly ONE hand ever drives the guide line.
function pickPointerCand(cands, preferKey, ownerKey, prefHand) {
if (preferKey) return candByKey(cands, preferKey);
if (!cands || !cands.length) return null;
var owner = ownerKey ? candByKey(cands, ownerKey) : null;
var pref = prefHand ? candByHand(cands, prefHand) : null;
if (owner && owner.onPanel) return owner;
if (pref && pref.onPanel) return pref;
for (var i = 0; i < cands.length; i++) if (cands[i].onPanel) return cands[i];
if (owner) return owner;
if (pref) return pref;
return cands[0];
}
// When the source that started a hold disappears (hand tracking lost), the hold
// would otherwise stick forever because no release event ever arrives.
function dropStaleHolds() {
var srcs = (S.session && S.session.inputSources) ? S.session.inputSources : [];
var i;
function present(src) {
for (i = 0; i < srcs.length; i++) if (srcs[i] === src) return true;
return false;
}
if (grab && !present(grab.src)) endGrab();
if (viewDrag && !present(viewDrag.src)) endViewDrag();
}
// ADDENDUM 11: which hand owns the pointer.
var activeSrcKey = null;
function updatePointer(frame, preferSrc) {
if (!S.session) return;
var next = {
visible: false, beam: false, source: 'none',
@@ -996,24 +1122,49 @@
// platform; an index-finger ray computed from joint poses carries the
// per-frame fingertip jitter straight into the cursor, so its landing point
// is visibly less steady. Meta's own samples use targetRaySpace too.
//
// ADDENDUM 11: with two hands BOTH report a tracked-pointer ray, and the
// runtime's order is not "the hand the user is using". Taking the first one
// pinned the guide line to one hand AND made the OTHER hand's select land
// wherever the pinned hand pointed. The pointer now follows, in order:
// 1. the source acting right now (select / gamepad press),
// 2. the last acting source, while it is aiming at the panel,
// 3. any source aiming at the panel,
// 4. the last acting source, then the first one.
var cands = [];
for (var i = 0; i < srcs.length; i++) {
var src = srcs[i];
if (src.targetRayMode !== 'tracked-pointer') continue;
var isHand = !!src.hand;
var ray = rayOf(frame, src);
if (!ray) continue;
var hit = panelHit(frame, ray, layer, 0.25, 0.6);
var hit = null;
try { hit = panelHit(frame, ray, layer, 0.25, 0.6); } catch (e) { hit = null; }
cands.push({
src: src, ray: ray, hit: hit, key: srcKey(src), hand: src.handedness,
onPanel: !!(hit && hit.inBounds)
});
}
var pick = pickPointerCand(cands, preferSrc ? srcKey(preferSrc) : null, activeSrcKey, S.handPref);
if (pick) {
activeSrcKey = pick.key;
next = {
visible: true,
beam: true,
source: (isHand ? 'hand' : 'controller') + ':' + src.handedness,
origin: ray.origin,
dir: ray.dir,
hit: hit ? { nx: hit.nx, ny: hit.ny } : null,
distance: hit ? hit.distance : null,
source: (pick.src.hand ? 'hand' : 'controller') + ':' + pick.src.handedness,
origin: pick.ray.origin,
dir: pick.ray.dir,
// ADDENDUM 11b: only a hit that is really ON the panel becomes the
// cursor. panelHit() clamps an off-panel intersection to the border, and
// drawing the cursor at that clamped point while the beam ended at the
// true intersection made the two drift apart - the further off the panel
// the ray pointed, the worse the gap. `distance` is still published so the
// beam still reaches the panel plane.
hit: (pick.hit && pick.hit.inBounds) ? { nx: pick.hit.nx, ny: pick.hit.ny } : null,
distance: pick.hit ? pick.hit.distance : null,
pressed: performance.now() < pressUntil
};
break;
}
pointer = next;
@@ -1022,6 +1173,8 @@
if (NS.Controls && NS.Controls.setPointer) {
try { NS.Controls.setPointer(next); } catch (e) {}
}
S.stats.activeSrc = next.visible ? (activeSrcKey || '(none)') : '(none)';
S.stats.handPref = S.handPref || 'right';
S.stats.pointer = next.source +
(next.hit ? ' HIT ' + next.hit.nx.toFixed(2) + ',' + next.hit.ny.toFixed(2) : ' (no hit)');
S.stats.inputSources = srcs.map(function (s) {
@@ -1273,7 +1426,7 @@
var a = rayAngles(ray);
if (!a) { S.stats.viewDrag = 'no ray'; return; }
viewDrag = {
src: src, t0: performance.now(), armed: false,
src: src, key: srcKey(src), t0: performance.now(), armed: false,
a0: null, yaw0: S.videoYaw, pitch0: S.videoPitch, moved: false
};
S.stats.viewDrag = 'pressed (hold ' + VIEW_HOLD_MS + 'ms to pan)';
@@ -1369,6 +1522,10 @@
* ------------------------------------------------------------------ */
var ACTIVATE_COOLDOWN_MS = 250;
var lastActivate = 0;
// ADDENDUM 11b: the cooldown is per INPUT SOURCE, not global. With hands, a
// stray gamepad edge from one hand used to swallow the other hand's `select`
// within 250 ms, so the click silently used the first hand's position.
var lastActivateByKey = {};
var padPrev = new WeakMap();
/* ---------------- grab & drag the subtitle strip ---------------- *
@@ -1410,6 +1567,7 @@
if (!p0) { S.stats.grab = 'no source point'; return; }
grab = {
src: src,
key: srcKey(src),
p0: p0,
place0: { y: S.subPlacement.y, z: S.subPlacement.z, width: S.subPlacement.width },
moved: false
@@ -1513,15 +1671,40 @@
function pointerDown(src, frame, via) {
if (!S.session) return;
var now = performance.now();
if (now - lastActivate < ACTIVATE_COOLDOWN_MS) return; // dedupe select+gamepad
var actKey = srcKey(src) || (via || '?');
if (now - (lastActivateByKey[actKey] || 0) < ACTIVATE_COOLDOWN_MS) return;
lastActivateByKey[actKey] = now;
lastActivate = now;
S.stats.lastActivate = (via || '?') + (src && src.handedness ? ':' + src.handedness : '');
S.stats.pressSrc = actKey;
// Captured BEFORE show(): when the control bar is already up, a pinch belongs
// to the bar, so it must not also yank the subtitles.
var barWasShown = NS.Controls && NS.Controls.isVisible ? !!NS.Controls.isVisible() : false;
if (frame) { try { updatePointer(frame); } catch (e) {} }
// ADDENDUM 11b: attribute the shot to the acting source. If its ray cannot be
// resolved, MISS - never fall back to whatever hand the per-frame pointer was
// on, which is exactly how a right-hand pinch ended up landing on the left
// hand's position.
frame = frame || S.lastFrame;
if (src) {
var excludeKey = srcKey(src);
if (excludeKey &&
((grab && grab.key && grab.key !== excludeKey) ||
(viewDrag && viewDrag.key && viewDrag.key !== excludeKey))) {
S.stats.pressAt = 'IGNORED (other hand is holding)';
return;
}
if (frame) { try { updatePointer(frame, src); } catch (e) {} }
if (!pointer || !pointer.visible) {
S.stats.pressAt = 'MISS (acting source has no ray)';
S.stats.pressPtr = '(no ray)';
return;
}
S.stats.pressPtr = pointer.source + (pointer.hit ? ' HIT' : ' nohit');
} else if (frame) {
try { updatePointer(frame); } catch (e) {}
}
// 1) grabbing the subtitle strip wins, but only while the bar is down
if (frame && !barWasShown) {
@@ -1579,7 +1762,14 @@
var pressUntil = 0;
function pointerUp() {
// ADDENDUM 12: only the source that STARTED a hold ends it. A release from the
// other hand used to cancel the first hand's drag (subtitle grab / picture pan).
function pointerUp(src) {
var key = srcKey(src);
if (key) {
if (grab && grab.key && grab.key !== key) return;
if (viewDrag && viewDrag.key && viewDrag.key !== key) return;
}
endGrab();
endViewDrag();
}
@@ -1599,7 +1789,7 @@
var prev = padPrev.get(src) || false;
padPrev.set(src, pressed);
if (pressed && !prev) pointerDown(src, frame, 'gamepad');
else if (!pressed && prev) pointerUp();
else if (!pressed && prev) pointerUp(src);
}
}
@@ -1607,8 +1797,8 @@
pointerDown(ev && ev.inputSource, S.lastFrame, 'select');
}
function onSelectEnd() {
pointerUp();
function onSelectEnd(ev) {
pointerUp(ev && ev.inputSource);
}
/* ---------------- live subtitle setters (single source of truth) ---------------- */
// state.subPlacement / subAngleDeg are re-read from the module after every
@@ -1749,6 +1939,7 @@
// `subtitles` and `subMode` could otherwise leave the layer uncreated.
S.subtitles = true;
if (S.subMode === 'off') S.subMode = 'hud';
S.subLayout = 'auto'; // ADDENDUM 10: the override is per-session only
try { if (NS.Subtitles && NS.Subtitles.init) NS.Subtitles.init(); } catch (e) { NS.recordError('Subtitles.init', e); }
try { if (NS.Controls && NS.Controls.init) NS.Controls.init(); } catch (e) { NS.recordError('Controls.init', e); }
try { if (NS.Hands && NS.Hands.init) NS.Hands.init(); } catch (e) { NS.recordError('Hands.init', e); }