Commit Graph
9 Commits
Author SHA1 Message Date
cat-shark c25e40070a caps: treat "decodes it, but not in real time" as unsupported
The previous gate only acted on decodingInfo's supported flag, so a device with
an HEVC decoder that cannot keep up at 8K still claimed level 6.1, Jellyfin
picked DirectPlay, and the picture never appeared. That is the realistic desktop
case and it was deliberately left unhandled; it is now handled.

A tier is blocked when:

    supported === false   cannot decode the format at all
    smooth    === false   can decode it, but not at that resolution and frame
                          rate -- exactly the "decoder exists, 8K does not work"
                          case

powerEfficient:false on its own still does not block. Nothing is asked of the
user: the point is that the client reports accurately and Jellyfin's own
PlaybackInfo decision then picks Transcode by itself. This is the mechanism the
bitrate prompt was compensating for.

Also:
  - each tier is now probed at a realistic bitrate (8K 30Mbps, 4K 20Mbps,
    1080p 8Mbps) instead of a flat 20Mbps
  - the diagnostics show WHY each tier was blocked (unsupported vs not-smooth)
  - new pin list: localStorage['jfvr-cpt-keep'] = '2:183' keeps a tier claimed
    even if the probe says no, so a single misreported device does not require
    switching the whole mechanism off

Verified against the local harness in five modes, all passing:
  pc    supported=false      -> capped at 153 -> TRANSCODE
  slow  supported, !smooth   -> capped at 153 -> TRANSCODE   (the real PC case)
  quest all smooth           -> 8K kept       -> DIRECT PLAY
  pin   2:183 pinned         -> 183 restored  -> DIRECT PLAY
  off   kill switch          -> untouched
2026-10-18 01:34:17 +08:00
cat-shark ab36d2f25c caps: ask the real decoder, and stop the client from over-claiming HEVC
The capability probe only ever asked canPlayType about Main profile (8-bit)
codec strings, which is why the project notes said "8K HEVC: supported" right up
until a Main 10 file showed up and could not be decoded at all. It now also asks
about Main 10, and -- more importantly -- asks the decoder itself.

The override: Jellyfin Web derives its entire HEVC claim from canPlayType
(browserDeviceProfile.js):

    if (canPlayType('video/mp4; codecs="hvc1.2.4.L183"')) {
        maxHevcLevel = 183;              // level 6.1 = 8K
        hevcProfiles = 'main|main 10';
    }

canPlayType answers "does this system have an HEVC decoder", not "can it do
8192x4096 Main 10 at 60fps". A desktop that merely ships an HEVC decoder
therefore claims 8K Main 10, the server picks DirectPlay, and playback dies on a
black screen with no player UI at all. MediaCapabilities.decodingInfo asks the
actual decoder, so its verdict is fed back through canPlayType and Jellyfin's own
logic lands on the right profile -- the desktop drops to what it can really do
and the server transcodes, while a Quest that genuinely decodes 8K Main 10 keeps
DirectPlay. Meta's browser-video docs recommend exactly this API over UA sniffing.

Deliberately conservative: only supported === false downgrades.
supported:true/smooth:false is recorded but changes nothing -- too close to the
edge to act on, and the watchdog already covers "claimed to work, did not".
Kill switch if a device reports nonsense: localStorage['jfvr-cpt'] = 'off'.

Each level is probed at the resolution it exists for (L120 -> 1080p30,
L153 -> 4K60, L183/L186 -> 8K60), because a level string carries no dimensions.

Verified against a local harness in three modes, all passing:
  pc    -> 8K Main10 blocked, Jellyfin caps at level 153, the 8K Main 10 file
           exceeds the claim and the server transcodes
  quest -> nothing blocked, claim stays >= 183, DirectPlay preserved
  off   -> hook not installed, canPlayType untouched, old behaviour restored
Also re-ran the playback watchdog on the new bundle: still fires on media error.

Also widens the harness gitignore to _harness*.html for the new codec harness.
2026-10-18 01:07:44 +08:00
cat-shark 0713f35513 hud-11: one pointer owner (dominant hand, press takes over) + hand isolation
ADDENDUM 12. The pointer was symmetric: whoever acted last owned it, and when
neither had acted the first source in runtime order won - the LEFT hand on this
runtime, so a right-handed user's guide line sat on the non-dominant hand.

- state.handPref ('right'|'left', default 'right') is the default owner. Set from the
  pre-VR panel (操作 -> 主用手), persisted. pickPointerCand(cands, preferKey,
  ownerKey, prefHand): acting source owns the shot (or MISSes), else owner aiming ->
  preferred hand aiming -> any hand aiming -> owner -> preferred hand -> first.
- A press takes the pointer over and the owner sticks while it is aiming.
- Hold exclusion: while a grab/viewDrag started by one source is active, a press from
  the other source is ignored (pressAt "IGNORED (other hand is holding)").
- Source-matched release: pointerUp(src) only ends the hold src started. It used to be
  argument-less, so either hand's release cancelled the other hand's subtitle grab or
  picture pan. grab/viewDrag now carry a key; onSelectEnd(ev) and pollGamepads pass
  their source.
- dropStaleHolds() runs each frame and ends a hold whose source left
  session.inputSources (tracking lost), so nothing sticks.

New diagnostics: stats.handPref, stats.owner.

Verified with injected synthetic input sources:
  default (right), both on panel     pointer hand:right HIT 0.78,0.50  (was hand:left)
  handPref=left, fresh session       pointer hand:left  HIT 0.22,0.50
  left press, release                owner hand:left, stays left
  right holds picture-pan, left press    "IGNORED (other hand is holding)"
  right holds picture-pan, LEFT release  viewDrag stays armed (drag survives)
  right holds picture-pan, RIGHT release viewDrag reset (tap) (drag ends)
2026-10-12 21:36:03 +08:00
cat-shark 2459b999c6 hud-10: the VR pointer follows the hand that is acting or aiming
ADDENDUM 11. With two hands both report a tracked-pointer ray and the runtime's order
is not "the hand the user is using". updatePointer() took the first valid source and
broke, and pointerDown() called updatePointer(frame) without the select event's
inputSource. On device that pinned the guide line/cursor to one hand while the user
aimed with the other, and made the other hand's pinch land wherever the pinned hand
pointed (left hand controlling the click position, right hand clicking).

updatePointer(frame, preferSrc) now builds a candidate per tracked-pointer source and
pickPointerCand() chooses: the acting source, else the last-active source while it is
on the panel, else any source on the panel, else the last-active source, else the
first. pointerDown() passes the event's inputSource, so an action is always evaluated
with the acting hand's ray, and the choice sticks afterwards. stats.activeSrc reports
it; session end resets it.

pickPointerCand is pure and unit-tested against the built bundle (9 cases, including
the reported one: guide on the left + right pinch -> hand:right, and it stays).

Verified in the headset only up to "no hands tracked at the time"; the live hand
switch still needs a worn headset.
2026-10-12 20:22:29 +08:00
cat-shark 22dd482d69 hud-9: render native (SRT) subtitles in the HUD; stop persisting the layout override
ADDENDUM 10. Two more independent subtitle problems found on device.

1. Normal (non-ASS) subtitles never reached the HUD. Jellyfin's
   htmlVideoPlayer-plugin only routes ssa/ass through SubtitlesOctopus
   (`if (!['ssa','ass'].includes(codec))` -> pgssub/vobsub/custom-element/native);
   subrip falls through to the browser's own TextTrack rendering, so there is no
   libass canvas at all (measured with the CN SRT selected: 0 libass canvases,
   0 .videoSubtitles overlays, textTracks[0] = manualTrack/mode=showing/315 cues).
   No layout setting could have made that work.

   updateSource() now picks the source every XR frame: the libass canvas when it
   exists, otherwise the active TextTrack cue (fallback: the .videoSubtitlesInner
   overlay). The native text is cleaned, wrapped and painted white-on-black into a
   private 1920x1080 canvas that flows through the unchanged crop/upload/HUD path.
   prepare() is skipped when there is no libass canvas. New diagnostics:
   stats.subSource / stats.subNativeText, Subtitles.getSourceKind().

2. ADDENDUM 9 persisted state.subLayout, and a stale 'mono' in jfvr.settings.v1
   silently forced the mono branch on the default dual-eye content (dual:false,
   eyeW:4320, crop across the seam -> both eyes side by side). The override is now
   per-session: not saved, not restored, 'auto' at boot.

Verified on device with hud-9:
  CN_dual_eye (ass)   -> subSource=libass, dual=true, eyeW=2160,
                         crop=[1040,768,1176,924], copyND=0.055
  CN (subrip)         -> subSource=native, dual=false, eyeW=1920, text="进去了呢",
                         crop 256x88, uploads climbing
The private cue canvas was read back and OCR'd (cue "怎么样?好好地" -> "怎么样? 好好地"),
and injecting/removing a fake libass canvas flips subSource with no errors.
2026-10-12 19:53:10 +08:00
cat-shark 164d692791 hud-8: keep the libass loop alive across XR entry; auto dual/mono subtitles
ADDENDUM 9 — two independent defects found on device.

1. VR subtitles froze on whatever line was on screen when the session started.
   Jellyfin 12.x renders with SubtitlesOctopus, which starts ONE long-lived rAF
   loop when the video loads; entering immersive-vr stops native rAF and destroys
   the callback still pending in the native scheduler, and because the instance's
   rafId stays non-zero it is never restarted (measured live: rafId=1150 native,
   oneshotState.eventStart frozen ~7 min behind the playhead, zero oneshot-render
   requests). The broker now ALWAYS queues rAF (native driver outside XR, rVFC +
   50 ms idle fallback inside), keeps exactly one native callback in flight, and
   no longer clears the queue at session start. A first version re-armed during
   the drain and doubled the driver every frame, freezing the page; that is now
   fixed and locked down by a Node harness (maxConcurrentNative = 1 over 300
   frames, self-rescheduling loop survives a simulated session start).

2. Only dual-eye ASS subtitles worked; a normal subtitle (e.g. the CN SRT track)
   is centred across the whole 2:1 frame and was cut at the eye seam because the
   HUD always sampled the left half. state.subLayout ('auto'|'dual'|'mono',
   default auto, persisted, panel row) plus per-sample detection: seam ink -> mono,
   one half only -> dual/mono, both halves with an empty seam -> column-profile
   copy test. Dual-eye is NOT a pixel mirror (same text at mirrored positions, no
   glyph flip): measured dual copy distance 0.02-0.08 vs mono 1.00, seam ink
   0 vs 50. M.eyeW (half for dual, full for mono) replaces the hard-coded halfW.

Docs: INTERFACES.md ADDENDUM 9, HANDOFF.md §13 (including the Jellyfin service
worker cache trap that made the first deploy look like a no-op).

The final build was not re-checked in the headset: the device left the worn state
mid-verification and the browser could not be relaunched. Verified on device: the
loop survives XR (rafId in the plugin range, eventStart tracking, subUploads 4->7
over 20 s in VR) and the layout metrics above; the final decision wiring is
covered by the offline suites.
2026-10-12 16:32:25 +08:00
cat-shark 7f751b7eee hud-7: live picture pan; fix the reset crash; tap-to-reset / hold-to-drag
Reported after hud-6:
  触发复位后直接崩溃了;我拖动后,我希望画面立刻就有反应,而不是我松手之后才改变。
  按住0.5s后拖动才是拖动,否则算复位

1. Why the picture only moved on release — a bad probe
The transform-write probe compared quaternion COMPONENTS. q and -q are the same
rotation and a runtime may hand back a renormalised quaternion, so a perfectly
capable runtime could be judged "read-only". When it was, the drag took the
rebuild branch, and that branch was throttled to "on release only" — hence the
picture sitting still until the finger came up. The probe now compares the
POSITION (exact, no sign ambiguity).

2. Two mechanisms, auto-selected and mutually exclusive
   'transform' -> layer.transform = q              (one write per change)
   'space'     -> layer.space = refSpace.getOffsetReferenceSpace(q)
                     (the recentering path every WebXR app uses; a layer pose is
                      recomputed from its space each frame)
   'rebuild'   -> recreate the media layer (manual escape hatch, "兼容" in the 2D panel)
createVideoLayer() only bakes the transform in for 'rebuild' — otherwise the
probe's own write would apply the aim twice. layer.transform is never written on
the 'space' path and vice versa. stats.videoRotMode / videoTransformWritable
report what was chosen.

3. The crash — create-before-destroy
The reset created the replacement layer BEFORE destroying the old one: two
XRMediaBinding equirect layers bound to one <video> element at once, then a
second rebuild microseconds later (release rebuilt, then reset rebuilt again),
took the browser down. A rebuild now runs in a setTimeout OUTSIDE the XR frame
callback, destroys FIRST, is throttled to 120 ms, and never queues more than one.
The transform/space paths never rebuild at all.

Two robustness holes found while testing:
  * videoSpaceFor() let a throwing getOffsetReferenceSpace() escape into the frame
    loop, where an exception kills the loop for good (rAF is only re-armed at the
    end of the callback) — a frozen session. Caught now, demoting to 'rebuild'.
  * the whole per-frame body is wrapped in try/catch for the same reason.

4. The gesture the owner asked for
   press the drag strip
     released before 500 ms, no movement -> RESET to dead ahead
     held past 500 ms, then moved        -> PAN
     held past 500 ms, never moved       -> RESET
The pan reference is captured when the hold threshold passes, not at the press,
so the hold itself cannot fling the picture; a ~1 deg dead zone absorbs jitter.
The strip now reads 点一下复位 · 按住 0.5s 再拖动 (the triple-tap hint is gone).

Verified offline against the real sources by three suites (build.py --check too):
  test-gesture  the state machine above, stubbed ray/clock: quick tap resets;
                hold-without-move resets; movement inside the hold window is
                ignored; after arming a +10/+5 deg ray move gives exactly
                +10/+5 deg; a pan does not reset; dead zone respected; pitch guard
  test-mech     position probe true/false for writable/ignored; transform
                strategy writes with no rebuild; space strategy assigns the offset
                space and leaves the transform alone; unchanged angle reuses the
                cached space; rebuild queues at most one and runs
                removeLayer -> createVideoLayer -> addLayer; throwing or absent
                getOffsetReferenceSpace is reported and falls back permanently
  test-ad8      quaternion aim/wrap/guard maths, panel bands on the 1440x520
                canvas, the strip's drawn readout, hold()
Deployed as mvp-5217a3 (hud-7); confirmed live on the Quest, errors empty.
2026-10-11 21:47:55 +08:00
cat-shark 263e270747 hud-6: pan the 180° video window about the eye; padlock as a panel button
ADDENDUM 8 — panning the video window (yaw/pitch about the eye)
The 180° window was hard-wired dead ahead, so scenes staged below the horizon
could only be watched by physically lowering the head. The video layer is an
XREquirectLayer with radius 0, i.e. the content sits at infinity: only its
ORIENTATION is visible and translating it does nothing. So panning is purely
layer.transform's rotation, which rotates the content about the eye point — no
parallax, no swim when the head moves, and no decode/upload cost (8K HEVC
DirectPlay untouched). Only the video rotates; the panel (refSpace, fixed pose)
and the subtitles (viewer space, head-locked) stay in front of the user.

    state.videoYaw   (+ = picture moved RIGHT)
    state.videoPitch (+ = picture moved UP)
    q = qy(-yaw) * qx(pitch)   -> aim = (cos p·sin yaw, sin p, -cos p·cos yaw)

Interaction is one affordance in the panel, above the progress bar (the owner's
requested placement): a drag strip. Pinch and drag -> the picture follows the
hand 1:1; three taps within 1.6 s reset it (taps counted on RELEASE, so a drag
cannot be mistaken for one); the live angles are printed on the strip. The drag
is driven by the RAY's own direction delta rather than a cursor position on the
panel: a panel-space cursor leaves the panel after a few degrees, while ray
rotation has no range limit and keeps the content point under the cursor.
Controls.hold() freezes the panel auto-hide for the drag's duration.

Pitch is limited only by a gimbal guard (±(π/2 - 0.02)) per "不限"; yaw wraps to
[-π, π]. Both reset to 0 on every VR session — not persisted.

Runtime risk: XREquirectLayer.transform is settable per spec but a runtime may
ignore the write, so it is probed once per session (write a known rotation, read
it back) -> stats.videoRotMode = 'live' (transform write per frame) or
'rebuild' (recreate the media layer, but only on creation/release/reset, never
per frame). THIS PROBE NEEDS A REAL XR SESSION — it is the one thing not
verifiable offline.

ADDENDUM 7b (previous pass, in this commit's tree) — the padlock
The padlock had been given its own quad layer parked 0.45 m to the right of the
panel. On device it could not be selected while the panel was shown but could
when it was hidden. Aiming is the binding constraint: with hands the only
pointing device is a ray, and the cursor's acceptance window reaches the panel's
own right edge, so the badge was outside every reachable ray; its hit test also
used the panel's layer geometry, which is why the behaviour flipped with panel
visibility. It is now an ordinary panel button (media row, fifth slot) using the
same hitTest/handleSelect path as every other button, and "locked" is a third
panel mode (0.26 × 0.10 m) that draws and hit-tests ONLY the padlock. Locking
swaps the panel in place and lets it auto-hide; the next summon shows the padlock
alone; unlocking restores the full panel and ignores presses for 500 ms.
handleSelect() no longer falls back to pressing the panel centre when no ray hit
exists.

Panel: expanded grew 0.72 × 0.21 m -> 0.72 × 0.26 m (canvas 1440 × 520) for the
drag strip; every row re-proportioned.

Verified offline against the real bundle: quaternion/drag maths (aim direction
for ±yaw/±pitch, unit norm, azimuth/elevation recovery, yaw wrap, pitch clamp,
"aimed point ends up under the cursor"), panel hit bands on the new canvas
(strip 0..0.20, inert readout 0.20..0.30, seek 0.30..0.52, media 0.52..0.74,
subtitle 0.84..1.0, no dead column), the strip's drawn readout, and hold().
build.py --check passes; deployed as mvp-2e32e6 and confirmed live on the Quest
(hud-6, videoYaw/videoPitch 0, errors empty).
2026-10-11 19:59:25 +08:00
cat-shark 838675baa6 Jellyfin Web VR plugin for Quest 3 (VR180 stereo playback)
Injects a 「VR 播放」 button into Jellyfin Web's video player page and plays
VR180 / VR360 stereo content on Meta Quest 3 directly in the browser, with
DirectPlay 8K HEVC, in-VR subtitles and an in-VR control panel.

Modules (src/, concatenated in filename order into dist/jfvr.js):
  00-core.js       namespace, shared state, GL helpers, layer/frame-hook registry
  10-subtitles.js  libass canvas -> head-locked quad layer, auto-cropped to ink,
                   with depth/size decoupling (angular-size lock)
  20-controls.js   in-VR control panel: progress/seek, transport, subtitle rows,
                   aiming beam, padlock, pure 2D canvas -> XR quad layer
  30-hands.js      system hand model (Three.js + generic-hand GLB) via a manually
                   driven projection layer
  90-main.js       boot, XR session lifecycle, rAF broker (rAF stops during XR),
                   input dispatch, settings panel, localStorage persistence

Key on-device findings are documented in HANDOFF.md §6 and README.md addenda:
requestAnimationFrame stops during an XR session, 3:2 pulldown at 90 Hz needs
updateTargetFrameRate(120), libass must be resized before entering VR, a quad's
local -Z is its plane normal, and hand aiming must use the gaze ray because the
emulated targetRaySpace points sideways. Quest does not expose eye tracking.

Docs: INTERFACES.md (module contract + addenda), README.md (measurements),
HANDOFF.md (handover notes). Credentials are NOT in the repo: deploy.py reads
the SSH password from the JF_PW environment variable.
2026-10-11 19:19:06 +08:00