hud-8: keep the libass loop alive across XR entry; auto dual/mono subtitles
ADDENDUM 9 — two independent defects found on device.
1. VR subtitles froze on whatever line was on screen when the session started.
Jellyfin 12.x renders with SubtitlesOctopus, which starts ONE long-lived rAF
loop when the video loads; entering immersive-vr stops native rAF and destroys
the callback still pending in the native scheduler, and because the instance's
rafId stays non-zero it is never restarted (measured live: rafId=1150 native,
oneshotState.eventStart frozen ~7 min behind the playhead, zero oneshot-render
requests). The broker now ALWAYS queues rAF (native driver outside XR, rVFC +
50 ms idle fallback inside), keeps exactly one native callback in flight, and
no longer clears the queue at session start. A first version re-armed during
the drain and doubled the driver every frame, freezing the page; that is now
fixed and locked down by a Node harness (maxConcurrentNative = 1 over 300
frames, self-rescheduling loop survives a simulated session start).
2. Only dual-eye ASS subtitles worked; a normal subtitle (e.g. the CN SRT track)
is centred across the whole 2:1 frame and was cut at the eye seam because the
HUD always sampled the left half. state.subLayout ('auto'|'dual'|'mono',
default auto, persisted, panel row) plus per-sample detection: seam ink -> mono,
one half only -> dual/mono, both halves with an empty seam -> column-profile
copy test. Dual-eye is NOT a pixel mirror (same text at mirrored positions, no
glyph flip): measured dual copy distance 0.02-0.08 vs mono 1.00, seam ink
0 vs 50. M.eyeW (half for dual, full for mono) replaces the hard-coded halfW.
Docs: INTERFACES.md ADDENDUM 9, HANDOFF.md §13 (including the Jellyfin service
worker cache trap that made the first deploy look like a no-op).
The final build was not re-checked in the headset: the device left the worn state
mid-verification and the browser could not be relaunched. Verified on device: the
loop survives XR (rafId in the plugin range, eventStart tracking, subUploads 4->7
over 20 s in VR) and the layout metrics above; the final decision wiring is
covered by the offline suites.
This commit is contained in:
@@ -976,3 +976,96 @@ double rotation) and schedules no rebuild; unchanged angles reuse the cached spa
|
||||
rebuild strategy queues at most one, and its observed order is
|
||||
`removeLayer -> createVideoLayer -> addLayer`; a throwing/absent
|
||||
`getOffsetReferenceSpace()` is reported and permanently falls back to `rebuild`.
|
||||
|
||||
# ADDENDUM 9 — subtitle pipeline survival + normal (mono) subtitles
|
||||
|
||||
Two independent defects, found on device with the VR session left in its broken state.
|
||||
|
||||
## 1. The rAF broker lost Jellyfin's subtitle loop at session start
|
||||
|
||||
`90-main.js` no longer just *forwards* to native `requestAnimationFrame` outside XR. It
|
||||
**always** holds callbacks in its own queue and drives them with one of two drivers:
|
||||
|
||||
- outside XR: a native rAF driver that keeps **exactly one** native callback in flight
|
||||
(`nativeRafStep` re-arms *before* draining and cancels itself when the queue empty);
|
||||
- inside XR: `video.requestVideoFrameCallback`, plus a 50 ms idle fallback so a paused /
|
||||
stalled / ended video still redraws (paused seek, subtitle track switch).
|
||||
|
||||
Why: `requestSession('immersive-vr')` stops native rAF completely. Jellyfin 12.x renders
|
||||
subtitles with SubtitlesOctopus, whose main thread starts **one long-lived rAF loop**
|
||||
(`rafId`) when the video loads, draws the canvas from it and requests the next prerender
|
||||
from the worker (`oneshot-render`). A callback still pending in the *native* scheduler when
|
||||
the session starts is destroyed with it, and because `rafId` stays non-zero the loop is
|
||||
never restarted.
|
||||
|
||||
Evidence (live instance, via CDP closure inspection):
|
||||
|
||||
```
|
||||
rafId : 1150 <- native id, not the plugin's 1e9 range
|
||||
oneshotState.eventStart: 945.99 <- canvas frozen ~7 min behind the playhead
|
||||
renderRequested : false
|
||||
oneshot-render requests during 15 s of playback: 0
|
||||
```
|
||||
|
||||
After the fix, in a live VR session: `rafId = 1000028035`, `eventStart` follows
|
||||
`video.currentTime`, and `stats.subUploads` climbs while subtitles change.
|
||||
|
||||
`startRafPump()` must **not** clear the queue (that callback is the one being rescued) and
|
||||
must drop the stale `nativeId`. `stopRafPump()` re-arms the native driver.
|
||||
|
||||
## 2. Auto dual-eye / mono subtitle layout
|
||||
|
||||
`10-subtitles.js` used to assume the libass canvas is a stereo left/right pair and always
|
||||
sampled the **left half**. A normal subtitle (e.g. the `CN` SRT track) is centred across the
|
||||
whole 2:1 frame, so half of it was cut off.
|
||||
|
||||
`state.subLayout` (`'auto' | 'dual' | 'mono'`, default `'auto'`, persisted) selects the
|
||||
region; `stats.subLayout / subDual / subEyeW / subCopyND / subCenterInk` report it.
|
||||
|
||||
Per sample (on the 320×160 downsample) the module builds the ink box and column count of
|
||||
each half plus the ink in a ±40 source-px band around the seam, then:
|
||||
|
||||
```
|
||||
only the left half has ink -> dual (sample the left half)
|
||||
only the right half has ink -> mono (sample the whole width)
|
||||
ink crosses the seam -> mono
|
||||
both halves, seam empty -> columnCopyND < COPY_TOL ? dual : mono
|
||||
```
|
||||
|
||||
**Dual-eye is NOT a pixel mirror.** Both eyes carry the *same* text at mirrored *positions*;
|
||||
the glyphs are not flipped, so `left[x]` vs `right[W-1-x]` fails badly (measured mean delta
|
||||
258/1020 → "not a mirror"). What is true is that the two halves' ink **column profiles**
|
||||
match under a small horizontal shift, because they are the same line. Measured on device:
|
||||
|
||||
| layout | centre-band ink | column-copy distance |
|
||||
|---|---|---|
|
||||
| dual-eye (`CN_dual_eye`) | 0 | 0.02 – 0.08 |
|
||||
| mono (`CN` SRT) | 50 / frame | 1.00 |
|
||||
|
||||
`COPY_TOL = 0.25`. A layout change is debounced (two consecutive samples) because it
|
||||
rebuilds the layer.
|
||||
|
||||
`M.eyeW` is one eye's region width (half for dual, full for mono); `computeTexDims()`,
|
||||
`drawHud()`'s crop/clamp and the texture scale all use it instead of the old hard-coded
|
||||
`halfW`.
|
||||
|
||||
### API additions
|
||||
|
||||
```js
|
||||
__JFVR.Subtitles.setLayout('auto' | 'dual' | 'mono') // also state.subLayout
|
||||
__JFVR.Subtitles.getLayout(): string
|
||||
```
|
||||
|
||||
## 3. Verification
|
||||
|
||||
- Broker logic replicated exactly in a Node harness against the built bundle:
|
||||
`maxConcurrentNative = 1` over 300 frames with a self-rescheduling loop (the first
|
||||
version re-armed during the drain and doubled the driver every frame — that froze the
|
||||
page); simulating session start (native callbacks dropped) leaves the loop in the queue
|
||||
and it keeps firing from the XR driver.
|
||||
- `columnCopyND` extracted from the built bundle and exercised with the on-device
|
||||
measurements: dual 0.00–0.08 → `dual`, mono centre-ink → `mono`, no-copy word-gap
|
||||
profile 0.41 → `mono`.
|
||||
- On device (before the UI became unreachable): the broker change was observed to carry
|
||||
SubtitlesOctopus's loop through `immersive-vr` (`rafId` in the plugin's id range,
|
||||
`eventStart` tracking, `subUploads` 4 → 7 over 20 s in VR).
|
||||
|
||||
Reference in New Issue
Block a user