ab36d2f25c6181797e0b0ef07f57d2c9a8bab3ce
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
0713f35513 |
hud-11: one pointer owner (dominant hand, press takes over) + hand isolation
ADDENDUM 12. The pointer was symmetric: whoever acted last owned it, and when
neither had acted the first source in runtime order won - the LEFT hand on this
runtime, so a right-handed user's guide line sat on the non-dominant hand.
- state.handPref ('right'|'left', default 'right') is the default owner. Set from the
pre-VR panel (操作 -> 主用手), persisted. pickPointerCand(cands, preferKey,
ownerKey, prefHand): acting source owns the shot (or MISSes), else owner aiming ->
preferred hand aiming -> any hand aiming -> owner -> preferred hand -> first.
- A press takes the pointer over and the owner sticks while it is aiming.
- Hold exclusion: while a grab/viewDrag started by one source is active, a press from
the other source is ignored (pressAt "IGNORED (other hand is holding)").
- Source-matched release: pointerUp(src) only ends the hold src started. It used to be
argument-less, so either hand's release cancelled the other hand's subtitle grab or
picture pan. grab/viewDrag now carry a key; onSelectEnd(ev) and pollGamepads pass
their source.
- dropStaleHolds() runs each frame and ends a hold whose source left
session.inputSources (tracking lost), so nothing sticks.
New diagnostics: stats.handPref, stats.owner.
Verified with injected synthetic input sources:
default (right), both on panel pointer hand:right HIT 0.78,0.50 (was hand:left)
handPref=left, fresh session pointer hand:left HIT 0.22,0.50
left press, release owner hand:left, stays left
right holds picture-pan, left press "IGNORED (other hand is holding)"
right holds picture-pan, LEFT release viewDrag stays armed (drag survives)
right holds picture-pan, RIGHT release viewDrag reset (tap) (drag ends)
|
||
|
|
e18c3d51bd |
hud-10 (rev3): cursor on the guide line; a press is attributed to the acting hand
ADDENDUM 11b, both reported against ADDENDUM 11. 1. The cursor drifted off the guide line, worse near the panel edge. panelHit() clamps an off-panel intersection to the border and flags it via inBounds, but updatePointer() published that clamped nx,ny as pointer.hit, so the ring was drawn on the border while the beam ended at the true (off-panel) intersection. pointer.hit is now published only when inBounds; distance is still published so an edge-aimed beam reaches the plane (beamLength no longer requires p.hit). Near-misses still summon the bar, but no longer press. 2. A right-hand press could still act at the left hand's position, for three separate reasons: the activate cooldown was global (a gamepad edge from one hand swallowed the other hand's select - now per input source); a missing ray fell back to another hand (with a preferKey the acting source now owns the shot and a missing candidate MISSes); and a null frame skipped re-attribution entirely (now falls back to S.lastFrame and MISSes if the acting source yields no pointer). New diagnostics: stats.pressSrc, stats.pressPtr. Verified with injected synthetic input sources: both on panel, right press pressSrc hand:right pressAt 0.78,0.50 seek 79.5% left press then right 100ms pressSrc hand:right pressAt 0.78,0.50 (not swallowed) both off panel state.pointer.hit = null (no clamped cursor) off panel, right press pressPtr "hand:right nohit" MISS (no hit on the panel) |
||
|
|
5eb679f6ca |
hud-10 (rev2): verify the hand switch with injected synthetic XR input sources
Adds the on-device verification for ADDENDUM 11 and a diagnostics nit
(stats.activeSrc reports '(none)' while no candidate is picked).
There is no official way to fake hand input on a Quest for a WebXR page: Meta's
simulator is a desktop tool for Unity/Unreal native apps, adb has no XR input
injection and CDP has no WebXR domain (WebXR.enable does not exist). So the test
injects synthetic input sources whose targetRaySpace is a real XRRigidTransform
offset reference space of the live session, and drives a press through the real
gamepad path (rising edge on gamepad.buttons[0]) - the same pointerDown(src, frame,
...) the select event reaches. XRSession.prototype.inputSources is overridden for
the test and restored after.
Measured in a live immersive-vr session:
initial pointer hand:left HIT 0.22,0.50
right press pointer hand:right HIT 0.78,0.50 activeSrc hand:right
right release pointer hand:right HIT 0.78,0.50 (sticky)
left press pointer hand:left HIT 0.22,0.50
right press, right hand off panel -> pressAt "MISS (not aiming at the panel)"
(did NOT fall back to the left hand's position)
|
||
|
|
2459b999c6 |
hud-10: the VR pointer follows the hand that is acting or aiming
ADDENDUM 11. With two hands both report a tracked-pointer ray and the runtime's order is not "the hand the user is using". updatePointer() took the first valid source and broke, and pointerDown() called updatePointer(frame) without the select event's inputSource. On device that pinned the guide line/cursor to one hand while the user aimed with the other, and made the other hand's pinch land wherever the pinned hand pointed (left hand controlling the click position, right hand clicking). updatePointer(frame, preferSrc) now builds a candidate per tracked-pointer source and pickPointerCand() chooses: the acting source, else the last-active source while it is on the panel, else any source on the panel, else the last-active source, else the first. pointerDown() passes the event's inputSource, so an action is always evaluated with the acting hand's ray, and the choice sticks afterwards. stats.activeSrc reports it; session end resets it. pickPointerCand is pure and unit-tested against the built bundle (9 cases, including the reported one: guide on the left + right pinch -> hand:right, and it stays). Verified in the headset only up to "no hands tracked at the time"; the live hand switch still needs a worn headset. |
||
|
|
22dd482d69 |
hud-9: render native (SRT) subtitles in the HUD; stop persisting the layout override
ADDENDUM 10. Two more independent subtitle problems found on device.
1. Normal (non-ASS) subtitles never reached the HUD. Jellyfin's
htmlVideoPlayer-plugin only routes ssa/ass through SubtitlesOctopus
(`if (!['ssa','ass'].includes(codec))` -> pgssub/vobsub/custom-element/native);
subrip falls through to the browser's own TextTrack rendering, so there is no
libass canvas at all (measured with the CN SRT selected: 0 libass canvases,
0 .videoSubtitles overlays, textTracks[0] = manualTrack/mode=showing/315 cues).
No layout setting could have made that work.
updateSource() now picks the source every XR frame: the libass canvas when it
exists, otherwise the active TextTrack cue (fallback: the .videoSubtitlesInner
overlay). The native text is cleaned, wrapped and painted white-on-black into a
private 1920x1080 canvas that flows through the unchanged crop/upload/HUD path.
prepare() is skipped when there is no libass canvas. New diagnostics:
stats.subSource / stats.subNativeText, Subtitles.getSourceKind().
2. ADDENDUM 9 persisted state.subLayout, and a stale 'mono' in jfvr.settings.v1
silently forced the mono branch on the default dual-eye content (dual:false,
eyeW:4320, crop across the seam -> both eyes side by side). The override is now
per-session: not saved, not restored, 'auto' at boot.
Verified on device with hud-9:
CN_dual_eye (ass) -> subSource=libass, dual=true, eyeW=2160,
crop=[1040,768,1176,924], copyND=0.055
CN (subrip) -> subSource=native, dual=false, eyeW=1920, text="进去了呢",
crop 256x88, uploads climbing
The private cue canvas was read back and OCR'd (cue "怎么样?好好地" -> "怎么样? 好好地"),
and injecting/removing a fake libass canvas flips subSource with no errors.
|
||
|
|
164d692791 |
hud-8: keep the libass loop alive across XR entry; auto dual/mono subtitles
ADDENDUM 9 — two independent defects found on device.
1. VR subtitles froze on whatever line was on screen when the session started.
Jellyfin 12.x renders with SubtitlesOctopus, which starts ONE long-lived rAF
loop when the video loads; entering immersive-vr stops native rAF and destroys
the callback still pending in the native scheduler, and because the instance's
rafId stays non-zero it is never restarted (measured live: rafId=1150 native,
oneshotState.eventStart frozen ~7 min behind the playhead, zero oneshot-render
requests). The broker now ALWAYS queues rAF (native driver outside XR, rVFC +
50 ms idle fallback inside), keeps exactly one native callback in flight, and
no longer clears the queue at session start. A first version re-armed during
the drain and doubled the driver every frame, freezing the page; that is now
fixed and locked down by a Node harness (maxConcurrentNative = 1 over 300
frames, self-rescheduling loop survives a simulated session start).
2. Only dual-eye ASS subtitles worked; a normal subtitle (e.g. the CN SRT track)
is centred across the whole 2:1 frame and was cut at the eye seam because the
HUD always sampled the left half. state.subLayout ('auto'|'dual'|'mono',
default auto, persisted, panel row) plus per-sample detection: seam ink -> mono,
one half only -> dual/mono, both halves with an empty seam -> column-profile
copy test. Dual-eye is NOT a pixel mirror (same text at mirrored positions, no
glyph flip): measured dual copy distance 0.02-0.08 vs mono 1.00, seam ink
0 vs 50. M.eyeW (half for dual, full for mono) replaces the hard-coded halfW.
Docs: INTERFACES.md ADDENDUM 9, HANDOFF.md §13 (including the Jellyfin service
worker cache trap that made the first deploy look like a no-op).
The final build was not re-checked in the headset: the device left the worn state
mid-verification and the browser could not be relaunched. Verified on device: the
loop survives XR (rafId in the plugin range, eventStart tracking, subUploads 4->7
over 20 s in VR) and the layout metrics above; the final decision wiring is
covered by the offline suites.
|
||
|
|
7f751b7eee |
hud-7: live picture pan; fix the reset crash; tap-to-reset / hold-to-drag
Reported after hud-6:
触发复位后直接崩溃了;我拖动后,我希望画面立刻就有反应,而不是我松手之后才改变。
按住0.5s后拖动才是拖动,否则算复位
1. Why the picture only moved on release — a bad probe
The transform-write probe compared quaternion COMPONENTS. q and -q are the same
rotation and a runtime may hand back a renormalised quaternion, so a perfectly
capable runtime could be judged "read-only". When it was, the drag took the
rebuild branch, and that branch was throttled to "on release only" — hence the
picture sitting still until the finger came up. The probe now compares the
POSITION (exact, no sign ambiguity).
2. Two mechanisms, auto-selected and mutually exclusive
'transform' -> layer.transform = q (one write per change)
'space' -> layer.space = refSpace.getOffsetReferenceSpace(q)
(the recentering path every WebXR app uses; a layer pose is
recomputed from its space each frame)
'rebuild' -> recreate the media layer (manual escape hatch, "兼容" in the 2D panel)
createVideoLayer() only bakes the transform in for 'rebuild' — otherwise the
probe's own write would apply the aim twice. layer.transform is never written on
the 'space' path and vice versa. stats.videoRotMode / videoTransformWritable
report what was chosen.
3. The crash — create-before-destroy
The reset created the replacement layer BEFORE destroying the old one: two
XRMediaBinding equirect layers bound to one <video> element at once, then a
second rebuild microseconds later (release rebuilt, then reset rebuilt again),
took the browser down. A rebuild now runs in a setTimeout OUTSIDE the XR frame
callback, destroys FIRST, is throttled to 120 ms, and never queues more than one.
The transform/space paths never rebuild at all.
Two robustness holes found while testing:
* videoSpaceFor() let a throwing getOffsetReferenceSpace() escape into the frame
loop, where an exception kills the loop for good (rAF is only re-armed at the
end of the callback) — a frozen session. Caught now, demoting to 'rebuild'.
* the whole per-frame body is wrapped in try/catch for the same reason.
4. The gesture the owner asked for
press the drag strip
released before 500 ms, no movement -> RESET to dead ahead
held past 500 ms, then moved -> PAN
held past 500 ms, never moved -> RESET
The pan reference is captured when the hold threshold passes, not at the press,
so the hold itself cannot fling the picture; a ~1 deg dead zone absorbs jitter.
The strip now reads 点一下复位 · 按住 0.5s 再拖动 (the triple-tap hint is gone).
Verified offline against the real sources by three suites (build.py --check too):
test-gesture the state machine above, stubbed ray/clock: quick tap resets;
hold-without-move resets; movement inside the hold window is
ignored; after arming a +10/+5 deg ray move gives exactly
+10/+5 deg; a pan does not reset; dead zone respected; pitch guard
test-mech position probe true/false for writable/ignored; transform
strategy writes with no rebuild; space strategy assigns the offset
space and leaves the transform alone; unchanged angle reuses the
cached space; rebuild queues at most one and runs
removeLayer -> createVideoLayer -> addLayer; throwing or absent
getOffsetReferenceSpace is reported and falls back permanently
test-ad8 quaternion aim/wrap/guard maths, panel bands on the 1440x520
canvas, the strip's drawn readout, hold()
Deployed as mvp-5217a3 (hud-7); confirmed live on the Quest, errors empty.
|
||
|
|
263e270747 |
hud-6: pan the 180° video window about the eye; padlock as a panel button
ADDENDUM 8 — panning the video window (yaw/pitch about the eye)
The 180° window was hard-wired dead ahead, so scenes staged below the horizon
could only be watched by physically lowering the head. The video layer is an
XREquirectLayer with radius 0, i.e. the content sits at infinity: only its
ORIENTATION is visible and translating it does nothing. So panning is purely
layer.transform's rotation, which rotates the content about the eye point — no
parallax, no swim when the head moves, and no decode/upload cost (8K HEVC
DirectPlay untouched). Only the video rotates; the panel (refSpace, fixed pose)
and the subtitles (viewer space, head-locked) stay in front of the user.
state.videoYaw (+ = picture moved RIGHT)
state.videoPitch (+ = picture moved UP)
q = qy(-yaw) * qx(pitch) -> aim = (cos p·sin yaw, sin p, -cos p·cos yaw)
Interaction is one affordance in the panel, above the progress bar (the owner's
requested placement): a drag strip. Pinch and drag -> the picture follows the
hand 1:1; three taps within 1.6 s reset it (taps counted on RELEASE, so a drag
cannot be mistaken for one); the live angles are printed on the strip. The drag
is driven by the RAY's own direction delta rather than a cursor position on the
panel: a panel-space cursor leaves the panel after a few degrees, while ray
rotation has no range limit and keeps the content point under the cursor.
Controls.hold() freezes the panel auto-hide for the drag's duration.
Pitch is limited only by a gimbal guard (±(π/2 - 0.02)) per "不限"; yaw wraps to
[-π, π]. Both reset to 0 on every VR session — not persisted.
Runtime risk: XREquirectLayer.transform is settable per spec but a runtime may
ignore the write, so it is probed once per session (write a known rotation, read
it back) -> stats.videoRotMode = 'live' (transform write per frame) or
'rebuild' (recreate the media layer, but only on creation/release/reset, never
per frame). THIS PROBE NEEDS A REAL XR SESSION — it is the one thing not
verifiable offline.
ADDENDUM 7b (previous pass, in this commit's tree) — the padlock
The padlock had been given its own quad layer parked 0.45 m to the right of the
panel. On device it could not be selected while the panel was shown but could
when it was hidden. Aiming is the binding constraint: with hands the only
pointing device is a ray, and the cursor's acceptance window reaches the panel's
own right edge, so the badge was outside every reachable ray; its hit test also
used the panel's layer geometry, which is why the behaviour flipped with panel
visibility. It is now an ordinary panel button (media row, fifth slot) using the
same hitTest/handleSelect path as every other button, and "locked" is a third
panel mode (0.26 × 0.10 m) that draws and hit-tests ONLY the padlock. Locking
swaps the panel in place and lets it auto-hide; the next summon shows the padlock
alone; unlocking restores the full panel and ignores presses for 500 ms.
handleSelect() no longer falls back to pressing the panel centre when no ray hit
exists.
Panel: expanded grew 0.72 × 0.21 m -> 0.72 × 0.26 m (canvas 1440 × 520) for the
drag strip; every row re-proportioned.
Verified offline against the real bundle: quaternion/drag maths (aim direction
for ±yaw/±pitch, unit norm, azimuth/elevation recovery, yaw wrap, pitch clamp,
"aimed point ends up under the cursor"), panel hit bands on the new canvas
(strip 0..0.20, inert readout 0.20..0.30, seek 0.30..0.52, media 0.52..0.74,
subtitle 0.84..1.0, no dead column), the strip's drawn readout, and hold().
build.py --check passes; deployed as mvp-2e32e6 and confirmed live on the Quest
(hud-6, videoYaw/videoPitch 0, errors empty).
|
||
|
|
838675baa6 |
Jellyfin Web VR plugin for Quest 3 (VR180 stereo playback)
Injects a 「VR 播放」 button into Jellyfin Web's video player page and plays
VR180 / VR360 stereo content on Meta Quest 3 directly in the browser, with
DirectPlay 8K HEVC, in-VR subtitles and an in-VR control panel.
Modules (src/, concatenated in filename order into dist/jfvr.js):
00-core.js namespace, shared state, GL helpers, layer/frame-hook registry
10-subtitles.js libass canvas -> head-locked quad layer, auto-cropped to ink,
with depth/size decoupling (angular-size lock)
20-controls.js in-VR control panel: progress/seek, transport, subtitle rows,
aiming beam, padlock, pure 2D canvas -> XR quad layer
30-hands.js system hand model (Three.js + generic-hand GLB) via a manually
driven projection layer
90-main.js boot, XR session lifecycle, rAF broker (rAF stops during XR),
input dispatch, settings panel, localStorage persistence
Key on-device findings are documented in HANDOFF.md §6 and README.md addenda:
requestAnimationFrame stops during an XR session, 3:2 pulldown at 90 Hz needs
updateTargetFrameRate(120), libass must be resized before entering VR, a quad's
local -Z is its plane normal, and hand aiming must use the gaze ray because the
emulated targetRaySpace points sideways. Quest does not expose eye tracking.
Docs: INTERFACES.md (module contract + addenda), README.md (measurements),
HANDOFF.md (handover notes). Credentials are NOT in the repo: deploy.py reads
the SSH password from the JF_PW environment variable.
|