Commit Graph
12 Commits
Author SHA1 Message Date
cat-shark ac85de2cb8 plugin: publish 0.1.0.0 (bundle 917fde) v0.1.0.1 2026-10-17 23:25:43 +08:00
cat-shark 7e25861498 plugin: publish 0.1.0.0 (bundle 917fde) 2026-10-17 23:25:01 +08:00
cat-shark f4fe3d61fc plugin: ship JFVR as a real Jellyfin plugin (self-hosted catalog)
Replaces the deploy.py approach (which sed-patched index.html inside the
container, and was therefore lost on every jellyfin/jellyfin upgrade) with a
proper plugin installable from a Jellyfin plugin repository.

The web bundle is unchanged: the same src/*.js -> dist/jfvr.js is now embedded
in Jellyfin.Plugin.JFVR.dll and served from /JFVR/jfvr.js with a content-hash
cache key. index.html is rewritten at request time by an IStartupFilter
middleware, so nothing on disk is touched.

Verified before this commit:
  - dotnet build: 0 warnings, 0 errors (net10.0, Jellyfin.Controller 12.1.0)
  - the DLL embeds dist/jfvr.js byte for byte
  - 19/19 checks on the rewrite logic (injection, idempotency, legacy-tag
    takeover, no </body> safety, other scripts preserved, URL override)
  - manifest.json passes Jellyfin's PackageInfo/VersionInfo expectations
  - the packaged zip's MD5 matches the checksum the manifest advertises

Details that were checked against the Jellyfin source rather than assumed:
  - a catalog version survives when targetAbi <= serverVersion, so a single
    targetAbi 12.0.0.0 package covers all of 12.x (server is 12.1.0)
  - Jellyfin writes meta.json itself at install time, so the zip needs only
    the DLL
  - Jellyfin verifies the MD5 at install time

Also adds pack.py (build -> package -> manifest -> Gitea release) and documents
the plugin route in README/HANDOFF, keeping deploy.py as a fallback.
v0.1.0.0
2026-10-17 23:17:00 +08:00
cat-shark 0713f35513 hud-11: one pointer owner (dominant hand, press takes over) + hand isolation
ADDENDUM 12. The pointer was symmetric: whoever acted last owned it, and when
neither had acted the first source in runtime order won - the LEFT hand on this
runtime, so a right-handed user's guide line sat on the non-dominant hand.

- state.handPref ('right'|'left', default 'right') is the default owner. Set from the
  pre-VR panel (操作 -> 主用手), persisted. pickPointerCand(cands, preferKey,
  ownerKey, prefHand): acting source owns the shot (or MISSes), else owner aiming ->
  preferred hand aiming -> any hand aiming -> owner -> preferred hand -> first.
- A press takes the pointer over and the owner sticks while it is aiming.
- Hold exclusion: while a grab/viewDrag started by one source is active, a press from
  the other source is ignored (pressAt "IGNORED (other hand is holding)").
- Source-matched release: pointerUp(src) only ends the hold src started. It used to be
  argument-less, so either hand's release cancelled the other hand's subtitle grab or
  picture pan. grab/viewDrag now carry a key; onSelectEnd(ev) and pollGamepads pass
  their source.
- dropStaleHolds() runs each frame and ends a hold whose source left
  session.inputSources (tracking lost), so nothing sticks.

New diagnostics: stats.handPref, stats.owner.

Verified with injected synthetic input sources:
  default (right), both on panel     pointer hand:right HIT 0.78,0.50  (was hand:left)
  handPref=left, fresh session       pointer hand:left  HIT 0.22,0.50
  left press, release                owner hand:left, stays left
  right holds picture-pan, left press    "IGNORED (other hand is holding)"
  right holds picture-pan, LEFT release  viewDrag stays armed (drag survives)
  right holds picture-pan, RIGHT release viewDrag reset (tap) (drag ends)
2026-10-12 21:36:03 +08:00
cat-shark e18c3d51bd hud-10 (rev3): cursor on the guide line; a press is attributed to the acting hand
ADDENDUM 11b, both reported against ADDENDUM 11.

1. The cursor drifted off the guide line, worse near the panel edge. panelHit()
   clamps an off-panel intersection to the border and flags it via inBounds, but
   updatePointer() published that clamped nx,ny as pointer.hit, so the ring was
   drawn on the border while the beam ended at the true (off-panel) intersection.
   pointer.hit is now published only when inBounds; distance is still published so
   an edge-aimed beam reaches the plane (beamLength no longer requires p.hit).
   Near-misses still summon the bar, but no longer press.

2. A right-hand press could still act at the left hand's position, for three
   separate reasons: the activate cooldown was global (a gamepad edge from one hand
   swallowed the other hand's select - now per input source); a missing ray fell
   back to another hand (with a preferKey the acting source now owns the shot and
   a missing candidate MISSes); and a null frame skipped re-attribution entirely
   (now falls back to S.lastFrame and MISSes if the acting source yields no
   pointer).

New diagnostics: stats.pressSrc, stats.pressPtr.

Verified with injected synthetic input sources:
  both on panel, right press      pressSrc hand:right  pressAt 0.78,0.50  seek 79.5%
  left press then right 100ms     pressSrc hand:right  pressAt 0.78,0.50  (not swallowed)
  both off panel                  state.pointer.hit = null (no clamped cursor)
  off panel, right press          pressPtr "hand:right nohit"  MISS (no hit on the panel)
2026-10-12 21:07:05 +08:00
cat-shark 5eb679f6ca hud-10 (rev2): verify the hand switch with injected synthetic XR input sources
Adds the on-device verification for ADDENDUM 11 and a diagnostics nit
(stats.activeSrc reports '(none)' while no candidate is picked).

There is no official way to fake hand input on a Quest for a WebXR page: Meta's
simulator is a desktop tool for Unity/Unreal native apps, adb has no XR input
injection and CDP has no WebXR domain (WebXR.enable does not exist). So the test
injects synthetic input sources whose targetRaySpace is a real XRRigidTransform
offset reference space of the live session, and drives a press through the real
gamepad path (rising edge on gamepad.buttons[0]) - the same pointerDown(src, frame,
...) the select event reaches. XRSession.prototype.inputSources is overridden for
the test and restored after.

Measured in a live immersive-vr session:
  initial            pointer hand:left  HIT 0.22,0.50
  right press        pointer hand:right HIT 0.78,0.50  activeSrc hand:right
  right release      pointer hand:right HIT 0.78,0.50  (sticky)
  left press         pointer hand:left  HIT 0.22,0.50
  right press, right hand off panel -> pressAt "MISS (not aiming at the panel)"
                                       (did NOT fall back to the left hand's position)
2026-10-12 20:29:51 +08:00
cat-shark 2459b999c6 hud-10: the VR pointer follows the hand that is acting or aiming
ADDENDUM 11. With two hands both report a tracked-pointer ray and the runtime's order
is not "the hand the user is using". updatePointer() took the first valid source and
broke, and pointerDown() called updatePointer(frame) without the select event's
inputSource. On device that pinned the guide line/cursor to one hand while the user
aimed with the other, and made the other hand's pinch land wherever the pinned hand
pointed (left hand controlling the click position, right hand clicking).

updatePointer(frame, preferSrc) now builds a candidate per tracked-pointer source and
pickPointerCand() chooses: the acting source, else the last-active source while it is
on the panel, else any source on the panel, else the last-active source, else the
first. pointerDown() passes the event's inputSource, so an action is always evaluated
with the acting hand's ray, and the choice sticks afterwards. stats.activeSrc reports
it; session end resets it.

pickPointerCand is pure and unit-tested against the built bundle (9 cases, including
the reported one: guide on the left + right pinch -> hand:right, and it stays).

Verified in the headset only up to "no hands tracked at the time"; the live hand
switch still needs a worn headset.
2026-10-12 20:22:29 +08:00
cat-shark 22dd482d69 hud-9: render native (SRT) subtitles in the HUD; stop persisting the layout override
ADDENDUM 10. Two more independent subtitle problems found on device.

1. Normal (non-ASS) subtitles never reached the HUD. Jellyfin's
   htmlVideoPlayer-plugin only routes ssa/ass through SubtitlesOctopus
   (`if (!['ssa','ass'].includes(codec))` -> pgssub/vobsub/custom-element/native);
   subrip falls through to the browser's own TextTrack rendering, so there is no
   libass canvas at all (measured with the CN SRT selected: 0 libass canvases,
   0 .videoSubtitles overlays, textTracks[0] = manualTrack/mode=showing/315 cues).
   No layout setting could have made that work.

   updateSource() now picks the source every XR frame: the libass canvas when it
   exists, otherwise the active TextTrack cue (fallback: the .videoSubtitlesInner
   overlay). The native text is cleaned, wrapped and painted white-on-black into a
   private 1920x1080 canvas that flows through the unchanged crop/upload/HUD path.
   prepare() is skipped when there is no libass canvas. New diagnostics:
   stats.subSource / stats.subNativeText, Subtitles.getSourceKind().

2. ADDENDUM 9 persisted state.subLayout, and a stale 'mono' in jfvr.settings.v1
   silently forced the mono branch on the default dual-eye content (dual:false,
   eyeW:4320, crop across the seam -> both eyes side by side). The override is now
   per-session: not saved, not restored, 'auto' at boot.

Verified on device with hud-9:
  CN_dual_eye (ass)   -> subSource=libass, dual=true, eyeW=2160,
                         crop=[1040,768,1176,924], copyND=0.055
  CN (subrip)         -> subSource=native, dual=false, eyeW=1920, text="进去了呢",
                         crop 256x88, uploads climbing
The private cue canvas was read back and OCR'd (cue "怎么样?好好地" -> "怎么样? 好好地"),
and injecting/removing a fake libass canvas flips subSource with no errors.
2026-10-12 19:53:10 +08:00
cat-shark 164d692791 hud-8: keep the libass loop alive across XR entry; auto dual/mono subtitles
ADDENDUM 9 — two independent defects found on device.

1. VR subtitles froze on whatever line was on screen when the session started.
   Jellyfin 12.x renders with SubtitlesOctopus, which starts ONE long-lived rAF
   loop when the video loads; entering immersive-vr stops native rAF and destroys
   the callback still pending in the native scheduler, and because the instance's
   rafId stays non-zero it is never restarted (measured live: rafId=1150 native,
   oneshotState.eventStart frozen ~7 min behind the playhead, zero oneshot-render
   requests). The broker now ALWAYS queues rAF (native driver outside XR, rVFC +
   50 ms idle fallback inside), keeps exactly one native callback in flight, and
   no longer clears the queue at session start. A first version re-armed during
   the drain and doubled the driver every frame, freezing the page; that is now
   fixed and locked down by a Node harness (maxConcurrentNative = 1 over 300
   frames, self-rescheduling loop survives a simulated session start).

2. Only dual-eye ASS subtitles worked; a normal subtitle (e.g. the CN SRT track)
   is centred across the whole 2:1 frame and was cut at the eye seam because the
   HUD always sampled the left half. state.subLayout ('auto'|'dual'|'mono',
   default auto, persisted, panel row) plus per-sample detection: seam ink -> mono,
   one half only -> dual/mono, both halves with an empty seam -> column-profile
   copy test. Dual-eye is NOT a pixel mirror (same text at mirrored positions, no
   glyph flip): measured dual copy distance 0.02-0.08 vs mono 1.00, seam ink
   0 vs 50. M.eyeW (half for dual, full for mono) replaces the hard-coded halfW.

Docs: INTERFACES.md ADDENDUM 9, HANDOFF.md §13 (including the Jellyfin service
worker cache trap that made the first deploy look like a no-op).

The final build was not re-checked in the headset: the device left the worn state
mid-verification and the browser could not be relaunched. Verified on device: the
loop survives XR (rafId in the plugin range, eventStart tracking, subUploads 4->7
over 20 s in VR) and the layout metrics above; the final decision wiring is
covered by the offline suites.
2026-10-12 16:32:25 +08:00
cat-shark 7f751b7eee hud-7: live picture pan; fix the reset crash; tap-to-reset / hold-to-drag
Reported after hud-6:
  触发复位后直接崩溃了;我拖动后,我希望画面立刻就有反应,而不是我松手之后才改变。
  按住0.5s后拖动才是拖动,否则算复位

1. Why the picture only moved on release — a bad probe
The transform-write probe compared quaternion COMPONENTS. q and -q are the same
rotation and a runtime may hand back a renormalised quaternion, so a perfectly
capable runtime could be judged "read-only". When it was, the drag took the
rebuild branch, and that branch was throttled to "on release only" — hence the
picture sitting still until the finger came up. The probe now compares the
POSITION (exact, no sign ambiguity).

2. Two mechanisms, auto-selected and mutually exclusive
   'transform' -> layer.transform = q              (one write per change)
   'space'     -> layer.space = refSpace.getOffsetReferenceSpace(q)
                     (the recentering path every WebXR app uses; a layer pose is
                      recomputed from its space each frame)
   'rebuild'   -> recreate the media layer (manual escape hatch, "兼容" in the 2D panel)
createVideoLayer() only bakes the transform in for 'rebuild' — otherwise the
probe's own write would apply the aim twice. layer.transform is never written on
the 'space' path and vice versa. stats.videoRotMode / videoTransformWritable
report what was chosen.

3. The crash — create-before-destroy
The reset created the replacement layer BEFORE destroying the old one: two
XRMediaBinding equirect layers bound to one <video> element at once, then a
second rebuild microseconds later (release rebuilt, then reset rebuilt again),
took the browser down. A rebuild now runs in a setTimeout OUTSIDE the XR frame
callback, destroys FIRST, is throttled to 120 ms, and never queues more than one.
The transform/space paths never rebuild at all.

Two robustness holes found while testing:
  * videoSpaceFor() let a throwing getOffsetReferenceSpace() escape into the frame
    loop, where an exception kills the loop for good (rAF is only re-armed at the
    end of the callback) — a frozen session. Caught now, demoting to 'rebuild'.
  * the whole per-frame body is wrapped in try/catch for the same reason.

4. The gesture the owner asked for
   press the drag strip
     released before 500 ms, no movement -> RESET to dead ahead
     held past 500 ms, then moved        -> PAN
     held past 500 ms, never moved       -> RESET
The pan reference is captured when the hold threshold passes, not at the press,
so the hold itself cannot fling the picture; a ~1 deg dead zone absorbs jitter.
The strip now reads 点一下复位 · 按住 0.5s 再拖动 (the triple-tap hint is gone).

Verified offline against the real sources by three suites (build.py --check too):
  test-gesture  the state machine above, stubbed ray/clock: quick tap resets;
                hold-without-move resets; movement inside the hold window is
                ignored; after arming a +10/+5 deg ray move gives exactly
                +10/+5 deg; a pan does not reset; dead zone respected; pitch guard
  test-mech     position probe true/false for writable/ignored; transform
                strategy writes with no rebuild; space strategy assigns the offset
                space and leaves the transform alone; unchanged angle reuses the
                cached space; rebuild queues at most one and runs
                removeLayer -> createVideoLayer -> addLayer; throwing or absent
                getOffsetReferenceSpace is reported and falls back permanently
  test-ad8      quaternion aim/wrap/guard maths, panel bands on the 1440x520
                canvas, the strip's drawn readout, hold()
Deployed as mvp-5217a3 (hud-7); confirmed live on the Quest, errors empty.
2026-10-11 21:47:55 +08:00
cat-shark 263e270747 hud-6: pan the 180° video window about the eye; padlock as a panel button
ADDENDUM 8 — panning the video window (yaw/pitch about the eye)
The 180° window was hard-wired dead ahead, so scenes staged below the horizon
could only be watched by physically lowering the head. The video layer is an
XREquirectLayer with radius 0, i.e. the content sits at infinity: only its
ORIENTATION is visible and translating it does nothing. So panning is purely
layer.transform's rotation, which rotates the content about the eye point — no
parallax, no swim when the head moves, and no decode/upload cost (8K HEVC
DirectPlay untouched). Only the video rotates; the panel (refSpace, fixed pose)
and the subtitles (viewer space, head-locked) stay in front of the user.

    state.videoYaw   (+ = picture moved RIGHT)
    state.videoPitch (+ = picture moved UP)
    q = qy(-yaw) * qx(pitch)   -> aim = (cos p·sin yaw, sin p, -cos p·cos yaw)

Interaction is one affordance in the panel, above the progress bar (the owner's
requested placement): a drag strip. Pinch and drag -> the picture follows the
hand 1:1; three taps within 1.6 s reset it (taps counted on RELEASE, so a drag
cannot be mistaken for one); the live angles are printed on the strip. The drag
is driven by the RAY's own direction delta rather than a cursor position on the
panel: a panel-space cursor leaves the panel after a few degrees, while ray
rotation has no range limit and keeps the content point under the cursor.
Controls.hold() freezes the panel auto-hide for the drag's duration.

Pitch is limited only by a gimbal guard (±(π/2 - 0.02)) per "不限"; yaw wraps to
[-π, π]. Both reset to 0 on every VR session — not persisted.

Runtime risk: XREquirectLayer.transform is settable per spec but a runtime may
ignore the write, so it is probed once per session (write a known rotation, read
it back) -> stats.videoRotMode = 'live' (transform write per frame) or
'rebuild' (recreate the media layer, but only on creation/release/reset, never
per frame). THIS PROBE NEEDS A REAL XR SESSION — it is the one thing not
verifiable offline.

ADDENDUM 7b (previous pass, in this commit's tree) — the padlock
The padlock had been given its own quad layer parked 0.45 m to the right of the
panel. On device it could not be selected while the panel was shown but could
when it was hidden. Aiming is the binding constraint: with hands the only
pointing device is a ray, and the cursor's acceptance window reaches the panel's
own right edge, so the badge was outside every reachable ray; its hit test also
used the panel's layer geometry, which is why the behaviour flipped with panel
visibility. It is now an ordinary panel button (media row, fifth slot) using the
same hitTest/handleSelect path as every other button, and "locked" is a third
panel mode (0.26 × 0.10 m) that draws and hit-tests ONLY the padlock. Locking
swaps the panel in place and lets it auto-hide; the next summon shows the padlock
alone; unlocking restores the full panel and ignores presses for 500 ms.
handleSelect() no longer falls back to pressing the panel centre when no ray hit
exists.

Panel: expanded grew 0.72 × 0.21 m -> 0.72 × 0.26 m (canvas 1440 × 520) for the
drag strip; every row re-proportioned.

Verified offline against the real bundle: quaternion/drag maths (aim direction
for ±yaw/±pitch, unit norm, azimuth/elevation recovery, yaw wrap, pitch clamp,
"aimed point ends up under the cursor"), panel hit bands on the new canvas
(strip 0..0.20, inert readout 0.20..0.30, seek 0.30..0.52, media 0.52..0.74,
subtitle 0.84..1.0, no dead column), the strip's drawn readout, and hold().
build.py --check passes; deployed as mvp-2e32e6 and confirmed live on the Quest
(hud-6, videoYaw/videoPitch 0, errors empty).
2026-10-11 19:59:25 +08:00
cat-shark 838675baa6 Jellyfin Web VR plugin for Quest 3 (VR180 stereo playback)
Injects a 「VR 播放」 button into Jellyfin Web's video player page and plays
VR180 / VR360 stereo content on Meta Quest 3 directly in the browser, with
DirectPlay 8K HEVC, in-VR subtitles and an in-VR control panel.

Modules (src/, concatenated in filename order into dist/jfvr.js):
  00-core.js       namespace, shared state, GL helpers, layer/frame-hook registry
  10-subtitles.js  libass canvas -> head-locked quad layer, auto-cropped to ink,
                   with depth/size decoupling (angular-size lock)
  20-controls.js   in-VR control panel: progress/seek, transport, subtitle rows,
                   aiming beam, padlock, pure 2D canvas -> XR quad layer
  30-hands.js      system hand model (Three.js + generic-hand GLB) via a manually
                   driven projection layer
  90-main.js       boot, XR session lifecycle, rAF broker (rAF stops during XR),
                   input dispatch, settings panel, localStorage persistence

Key on-device findings are documented in HANDOFF.md §6 and README.md addenda:
requestAnimationFrame stops during an XR session, 3:2 pulldown at 90 Hz needs
updateTargetFrameRate(120), libass must be resized before entering VR, a quad's
local -Z is its plane normal, and hand aiming must use the gaze ray because the
emulated targetRaySpace points sideways. Quest does not expose eye tracking.

Docs: INTERFACES.md (module contract + addenda), README.md (measurements),
HANDOFF.md (handover notes). Credentials are NOT in the repo: deploy.py reads
the SSH password from the JF_PW environment variable.
2026-10-11 19:19:06 +08:00