Files
jellyfin_vr_web_plugin/INTERFACES.md
T
cat-shark 5eb679f6ca hud-10 (rev2): verify the hand switch with injected synthetic XR input sources
Adds the on-device verification for ADDENDUM 11 and a diagnostics nit
(stats.activeSrc reports '(none)' while no candidate is picked).

There is no official way to fake hand input on a Quest for a WebXR page: Meta's
simulator is a desktop tool for Unity/Unreal native apps, adb has no XR input
injection and CDP has no WebXR domain (WebXR.enable does not exist). So the test
injects synthetic input sources whose targetRaySpace is a real XRRigidTransform
offset reference space of the live session, and drives a press through the real
gamepad path (rising edge on gamepad.buttons[0]) - the same pointerDown(src, frame,
...) the select event reaches. XRSession.prototype.inputSources is overridden for
the test and restored after.

Measured in a live immersive-vr session:
  initial            pointer hand:left  HIT 0.22,0.50
  right press        pointer hand:right HIT 0.78,0.50  activeSrc hand:right
  right release      pointer hand:right HIT 0.78,0.50  (sticky)
  left press         pointer hand:left  HIT 0.22,0.50
  right press, right hand off panel -> pressAt "MISS (not aiming at the panel)"
                                       (did NOT fall back to the left hand's position)
2026-10-12 20:29:51 +08:00

55 KiB
Raw Blame History

JFVR module interfaces (contract)

Everything lives under a single global namespace window.__JFVR, created by src/00-core.js before any other module runs. Modules are plain (function(){ ... })() IIFEs that attach themselves to __JFVR.

Load order (build script concatenates in this order):

00-core.js      -> __JFVR core: state, helpers, layer/frame-hook registry
10-subtitles.js -> __JFVR.Subtitles
20-controls.js  -> __JFVR.Controls
90-main.js      -> boot, DOM scan, XR session lifecycle, settings panel, wiring

Nothing may read document / navigator.xr at load time except 00-core.js and 90-main.js. The two module files must only define functions and are called later with a session/frame.


00-core.js — always available as window.__JFVR

__JFVR.state            // shared mutable state object (see below)
__JFVR.log(...args)     // console.log with [JFVR] prefix + colour
__JFVR.version          // string

// --- helpers ---
__JFVR.makeEl(tag, className?, text?)      // document.createElement wrapper
__JFVR.compileShader(gl, type, source)     // throws with the info log on failure
__JFVR.invert4(mat4ArrayLike)              // -> Float32Array(16)
__JFVR.QUAD_VS                             // vertex shader source: fullscreen triangle,
                                           //   `in vec2 aPos; out vec2 vUv;`
                                           //   vUv = vec2(aPos.x*.5+.5, .5 - aPos.y*.5)
__JFVR.bindFullscreenTriangle(gl, program) // creates the VBO, enables aPos, returns attribLoc

// --- XR layer registry (owned by 90-main.js) ---
__JFVR.addLayer(layer)        // register + push updateRenderState if a session is live
__JFVR.removeLayer(layer)     // unregister + push
__JFVR.addFrameHook(fn)       // fn(frame, time, session) every XR frame
__JFVR.removeFrameHook(fn)

// --- capabilities (filled by 00-core.js at boot) ---
__JFVR.caps = {
  secureContext, hasXR, immersiveVR, hasXRMediaBinding, hasEquirectLayer,
  hasXRWebGLBinding, hasXRHand, hasXRJointSpace, maxTextureSize,
  hevc: { ... }, ua
}

// --- live handles ---
__JFVR.session      // current XRSession or null
__JFVR.video        // current <video> element or null
__JFVR.refSpace     // 'local' XRReferenceSpace for the running session
__JFVR.viewerSpace  // 'viewer' XRReferenceSpace (head-locked layers)

__JFVR.state fields that modules may use

state.video          // <video> or null
state.session        // XRSession or null
state.refSpace       // local space
state.viewerSpace    // viewer space (head-locked)
state.projection     // '180' | '360'
state.stereo         // 'mono' | 'sbs' | 'ou'
state.subtitles      // bool — render subtitles in VR
state.subMode        // 'hud'   (head-locked quad, default)
                     // 'sphere'(on the equirect layer, legacy)
state.subAlpha       // 'premultiplied' | 'straight'
state.subPlacement   // { y, z, width } metres, hud mode only
state.stats          // diagnostics bag, free to add keys
state.gl             // { refSpace, session } set by main during a session

Layer / frame-hook lifecycle a module must follow

  1. create(session, glContextOrNull) — build GL resources + the XR layer, call __JFVR.addLayer(layer), __JFVR.addFrameHook(hook). Return the layer, or null when it cannot run (missing canvas, missing API…). Never throw — return null and __JFVR.log() the reason.
  2. hook(frame, time, session) — per-frame work. Must be cheap; skip work when nothing changed.
  3. destroy() — remove the frame hook, remove the layer, delete GL resources, drop canvas references. Must be safe to call twice and safe to call when create() returned null.

10-subtitles.js — __JFVR.Subtitles

Renders Jellyfin's libass subtitle canvas into VR.

__JFVR.Subtitles = {
  // Called once at boot, before any session.
  init(): void,

  // BEFORE entering VR. Jellyfin sizes the libass canvas to the on-screen video
  // box (~1250x625), which is far too small for VR. Blowing up the player box
  // makes libass re-render bigger. Must be called while NOT in an immersive
  // session (libass does not recompute its backing store during XR).
  // Idempotent: calling twice must not double-apply.
  prepare(): void,

  // Undo prepare(). Safe when prepare() was never called.
  restore(): void,

  // True when a libass canvas exists and has non-zero size.
  isReady(): boolean,

  // Current libass canvas size, for diagnostics: { w, h } or null
  canvasSize(): { w, h } | null,

  // --- session lifecycle ---
  create(session): XRLayer | null,
  destroy(): void,

  // --- settings, safe to call before or during a session ---
  setMode(mode),                        // 'hud' | 'sphere' | 'off'
  setAlphaMode(mode),                   // 'premultiplied' | 'straight'
  setPlacement({ y, z, width }),        // partial update, hud only
  getMode(): string
};

HUD mode requirements (the default, and the point of this module)

  • A head-locked quad layer in __JFVR.viewerSpace (XRWebGLBinding.createQuadLayer({ space: viewerSpace, layout: 'mono', ... })).
  • Source pixels come from canvas.libassjs-canvas. That canvas holds a stereo left-right image: the left half is the LeftEye subtitle, the right half the RightEye one. For a mono HUD use one half only (the left half is correct for this content).
  • Auto-crop to the ink bounding box so the strip is tight and the text is as large as possible. Recompute the bbox at a low rate (≈4 Hz) on a downscaled copy — never getImageData the full-resolution canvas every frame (a 4320x2160 full read is ~37 MB and has already crashed a renderer once). Add hysteresis / smoothing so the strip does not jitter between frames.
  • Upload only when the source changed (or at most ~30 Hz) — the earlier implementation burned a 4K texture upload every frame.
  • Target ≈30 px/degree. A 2.0 m wide strip at 1.45 m is ≈66° → texture width 2048 is comfortable on Quest 3.
  • Respect state.subAlpha:
    • 'premultiplied': gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, true)
    • 'straight': gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, false)
  • Draw with XRWebGLBinding.getSubImage(layer, frame) → bind sub.colorTexture to a framebuffer → viewport to sub.colorTextureWidth/Height → clear to (0,0,0,0) → draw. getSubImage takes two arguments.

prepare() detail

The measured behaviour: setting the player box to 7680x3840 makes libass produce a 4320x2160 canvas (it does not go higher). The previous bug was that this was only applied on "open panel" and then restored on session end while the panel stayed open, so a re-entry got a 1250x625 canvas. prepare() must therefore be idempotent and re-callable.

Target DOM: document.querySelector('.videoPlayerContainer') and the <video>. Save/restore their inline style.cssText exactly.


20-controls.js — __JFVR.Controls

A head-locked control strip: progress bar + seek, play/pause, skip ±10 s, subtitle toggle. Pure 2D canvas rendering + a hit-test function, plus a thin XR quad layer wrapper.

__JFVR.Controls = {
  init(): void,

  // --- pure rendering, no WebXR, must be unit-testable in isolation ---
  // Draws the bar into `ctx` sized `w` x `h` (CSS pixels).
  // `view` = {
  //   playing, currentTime, duration, buffered, subtitlesOn,
  //   mode: 'collapsed' | 'expanded',
  //   hover: null | 'playpause' | 'back10' | 'fwd10' | 'subs' | 'track'
  // }
  draw(ctx, w, h, view): void,

  // Maps a point in normalized panel coords (0..1, origin top-left) to an action.
  // Returns one of:
  //   { type: 'seek',    value: 0..1 }
  //   { type: 'playpause' | 'back10' | 'fwd10' | 'subs' }
  //   null
  hitTest(nx, ny, w, h): action | null,

  // --- XR lifecycle ---
  create(session): XRLayer | null,
  destroy(): void,

  // --- state / input ---
  setVideo(videoEl, getExtra | null): void,   // main provides time/duration/buffered readers
  onAction(cb): void,                         // cb(action) — main performs the media op
  show(autoHideMs?), hide(): void,            // collapsed <-> expanded + auto-hide
  isVisible(): boolean,
  handleSelect(inputSource, frame): void      // called on XR `select`
};

Panel design requirements

  • Two visual states: collapsed = a slim progress bar only (≈0.9 m x 0.07 m, centre-bottom of view). expanded = progress row + buttons (≈1.1 m x 0.22 m).
  • Hit targets must be generous: the progress bar should be visually ~1 cm tall but hit-testable over ~3–4 cm. Meta's docs warn that the emulated hand target ray for hand tracking is only a coarse forward pointer, so do not rely on pixel-precise aiming.
  • Colours must read on top of arbitrary video: dark translucent pill background, white text, accent #4ea1ff (matches the existing in-page panel).
  • Everything must be drawn from scratch on a canvas — no DOM, no images, no external fonts (use a generic stack: 'system-ui, -apple-system, sans-serif').
  • draw() must be deterministic and side-effect free (no timers inside).
  • hitTest must be a pure geometry function of (nx, ny, w, h).

Input model (main.js supplies)

  • Gaze: the panel is head-locked, so the reticle is the view centre. main.js converts that into (nx, ny) and calls hitTest.
  • Controller ray: main.js intersects the controller ray with the panel plane and converts to (nx, ny).
  • Both end in handleSelect() → the action from hitTest → onAction callback.

90-main.js — owned by the integrator

Boot, DOM scan, the in-page VR 播放 button + settings panel, the XR session lifecycle, the rAF broker (see below), input dispatch, and the settings UI for subtitle mode / alpha mode / placement.

rAF broker (already proven, keep it)

While an immersive XR session is running window.requestAnimationFrame is completely stopped (measured: 0 callbacks in 6 s), and Jellyfin renders libass subtitles from an rAF loop. 90-main.js intercepts requestAnimationFrame/cancelAnimationFrame, holds callbacks while in XR and drives them from video.requestVideoFrameCallback. Outside XR it delegates to the native implementation.


ADDENDUM 2 — in-VR UX pass (after first on-device test)

Owner feedback from the first headset test:

  1. The control bar is permanently visible — wrong. It must be hidden by default and only appear on interaction, then auto-hide.
  2. Hand/controller input has no guide line. There is no visible ray and no cursor, so the user cannot tell where they are aiming.
  3. Subtitles must sit below the visual centre by default, and both the position and the opacity must be user-adjustable.

New shared state (owned by 90-main.js, read by modules)

state.subOpacity      // 0..1, default 0.85
state.subPlacement    // { y, z, width }  — y is the quad CENTRE, negative = below eye level
state.ctlAutoHideMs   // default 4000
state.pointer         // { visible, source, origin:{x,y,z}, dir:{x,y,z}, hit:{nx,ny}|null }
                      // written every XR frame by 90-main.js; modules must treat it as read-only

10-subtitles.js — additions

setOpacity(a)              // 0..1; applied to the XR layer (layer.opacity or the
                           // shader's alpha), NOT by rebuilding the layer
getOpacity(): number
  • Default state.subPlacement becomes { y: -0.42, z: -1.45, width: 2.0 }.
  • setPlacement() must take effect live (update layer.transform if the runtime allows mutating it, otherwise rebuild just this layer and keep the frame hook).
  • Keep the existing tight-crop behaviour and the 4320x2160 canvas preparation.

20-controls.js — additions

setPointer(p)     // p as in state.pointer; called every frame by main.
                  // Draw a beam from origin along dir and, when hit != null, a
                  // cursor dot at (hit.nx, hit.ny) on the panel.
setOpacity(a)
getOpacity(): number
  • Default state is HIDDEN. create() must not make the bar visible.
    • show(autoHideMs) → visible + expanded, auto-hides after autoHideMs (default state.ctlAutoHideMs).
    • hide() → fully hidden (no bar, no beam, no cursor) until show() again.
    • isVisible() → true only while shown.
  • The beam must be a separate, always-updated visual that is only visible while the bar is shown. Implement it as a thin quad layer whose transform is placed at origin + dir * (L/2) and oriented so its local -Z follows dir (build the quaternion from the direction; L = distance to the hit, or a fixed 1.2 m when there is no hit). Height ≈ 0.006 m, width = L, with a soft gradient texture (bright at the origin end, fading out) so it reads as a laser.
  • The cursor: when hit is non-null, draw a ring/dot on the control panel canvas at (hit.nx, hit.ny) so the user sees exactly what will be activated.
  • Do not re-upload either texture more than ~30 Hz; skip everything when hidden.
  • Add stats keys ctlPointer, ctlHidden, ctlBeamOn.

Input model change (90-main.js) — unified pointing

Hand tracking and controllers both expose targetRaySpace. Use one code path:

  • For every input source with targetRayMode === 'tracked-pointer' (controllers and hands), intersect its ray with the control quad plane and produce hit.
  • If no source produces a hit, fall back to gaze: hit = {nx:0.5, ny:0.5} centred.
  • state.pointer.source records which one won.
  • Show the beam for the winning source; hide it when the controls are hidden.

Meta's docs note the hand target ray is an emulated forward-pointing ray, so hit targets stay generous — that is already the case.

Beam orientation — CORRECTED convention (my first spec was wrong)

Do not map the quad's local -Z onto the ray direction. A quad's local -Z is its plane normal, so that would put the quad's plane perpendicular to the ray and lay its width extent across the beam — a thin face-on bar, not a beam.

Correct:

position   = origin + dir * (L / 2)          // L = hit distance, else 1.2 m
width      = L        (the long axis)
height     = 0.006
local +X   -> dir                            // the beam axis lies IN the plane
local -Z   -> perpendicular to dir, biased up so the ribbon is never edge-on
gradient   -> bright at the origin end, fading to transparent at the hit end

Assertion for tests: R(q)·(1,0,0) ≈ dir and R(q)·(0,0,-1) ⟂ dir.


ADDENDUM 4 — in-VR subtitle placement (grab & drag)

Owner requirement: adjust the subtitles without leaving VR. Chosen model:

  • Position: grab the subtitle strip with the laser and drag it in space.
  • Reset button restores the defaults.
  • Size and opacity also adjustable in VR.
  • Values persist in localStorage (per browser, no server changes).

Grab & drag model (owned by 90-main.js)

The strip is a head-locked quad in viewer space, so both the ray and the strip live in the same space — no world-space math needed.

on grab start:
  grabActive   = true
  grabOrigin0  = ray.origin            // viewer space
  grabHit0     = ray ∩ plane(z = placement.z)
  grabPlace0   = { y, z, width }

per frame while grabbed:
  P  = currentRay ∩ plane(z = grabPlace0.z)          // fixed reference plane
  y' = grabPlace0.y + (P.y - grabHit0.y)             // drag up/down
  z' = grabPlace0.z + (ray.origin.z - grabOrigin0.z) // pull toward / push away
  clamp y' to [-0.95, -0.05], z' to [-2.6, -0.7]
  Subtitles.setPlacement({ y: y', z: z', width: current })

on grab end: clear grab state, persist

Rationale for z: in viewer space forward is -Z, so pulling the hand back (increasing origin.z) makes z' less negative, i.e. the strip comes closer — which matches how pulling a real object behaves.

Grab must not steal the normal control-bar interaction: only start a grab when the ray actually hits the subtitle quad; otherwise fall through to the bar.

Input: press and release are now needed

The existing activation path in 90-main.js only detects a rising edge. Grabbing needs the held state too, so track per input source:

  • selectstart / selectend session events, and
  • inputSource.gamepad button press/release (hand pinch; see ADDENDUM 3)

Both feed one pointerDown(src) / pointerUp(src) pair, deduped the same way.

10-subtitles.js — additions

getLayer(): XRLayer | null      // so main can ray-cast against the strip
resetPlacement(): void          // back to { y: -0.42, z: -1.45, width: 2.0 }
  • width is the size control: setPlacement({ width }) already rescales the quad while keeping the angular text height consistent with the texture aspect.
  • All of setPlacement / setOpacity / resetPlacement stay live during a session.

20-controls.js — additions

The expanded control bar gains a subtitle row with large −/+ buttons:

Action returned by hitTest Meaning
sub_smaller / sub_bigger width by ±0.15 m, clamped to [0.8, 4.0]
sub_opaque_down / sub_opaque_up opacity by ∓0.05, clamped to [0.1, 1.0]
sub_reset restore all subtitle defaults
  • Add view.sub = { width, opacity } so draw() can render the current values.
  • Keep hit targets generous; the rows may overlap visually but must not overlap in hit-test order (buttons before track).
  • Add a new expanded height that fits the extra row.

90-main.js — persistence

localStorage key: 'jfvr.settings.v1'
{ subPlacement: {y,z,width}, subOpacity, subMode, subAlpha,
  projection, stereo, ctlAutoHideMs }

Load at boot (merge over defaults, validate ranges), save debounced ~400 ms after any change. A corrupt/absent value must fall back to defaults silently.


ADDENDUM 5 — gaze pointing + lazy-follow panel + panel size

Why (measured on device)

The hand's targetRaySpace is an emulated ray (Meta documents this). Measured on a real Quest 3 while the panel was straight ahead:

hand:right  origin=(-0.317,-0.054,-0.248)  dir=(-0.874, 0.461, -0.154)
                                                    ^ 87 % sideways, 15 % forward

Aiming with it lands somewhere the user is not looking. Gaze + pinch is Meta's documented recommendation for hands and needs no ray accuracy at all.

Changes

# Change
1 Pointer rays are computed in LOCAL space (__JFVR.refSpace), not viewer space.
2 Hands aim with the gaze ray (head forward in local space). No beam - a guide line from the hand would be misleading because the hand is not aiming. The on-panel reticle is the indicator.
3 Controllers keep their own targetRaySpace ray and the laser beam; a controller ray is accurate and a laser is expected. Selected by pointer.beam.
4 The control panel layer's space is refSpace and holds a stable world pose. A head-locked panel would always put the view centre on its own centre, so no button could ever be targeted by gaze.
5 90-main.js drives a lazy follow: it only re-centres the panel once the head has turned > 26 deg away from it, easing with a 0.45 s time constant.
6 subtitles is no longer persisted. Persisting it meant a session where the user toggled subtitles off came back with the layer never created - which looked like a load failure. Always ON at boot; the toggle is per-session.
7 Subtitle distance range extended: the near clamp is now 0.5 m (was 0.7 m).

ADDENDUM 6 — subtitle depth vs size decoupling (angular-size lock)

Why (owner requirement)

当前 vr 视频中可能出现 vr 空间中物体可能距离我很近,如果字幕还很远,我将无法在 看清物体的同时看清字幕。我希望可以让字幕的空间距离和物体的距离尽量接近。

Eye tracking is not exposed to WebXR on Quest (HANDOFF §4), so the strip's depth cannot follow the user's gaze automatically. What matters instead is that the DEPTH control must not change the strip's apparent size: otherwise pulling the subtitles to a nearby object makes the text huge (and, with the head-locked quad, drops it out of the lower view). See HANDOFF §8 for the literature this is based on.

Model

state.subLockAngle (default true) + state.subAngleDeg (default 69.2).

While locked, a depth change holds TWO things:

width = 2 * |z| * tan(subAngleDeg / 2)     angular width  (theta)
y     = (y0 / z0) * z                      angular height (the y/z ratio)

so the strip keeps both its angular size and its screen position — only depth changes. The default pair is bit-for-bit the pre-ADDENDUM-6 look (2.0 m at 1.45 m → 69.2°), so nothing moves for anyone who does not touch the control.

While unlocked everything behaves exactly as at ADDENDUM 4: width and z are independent ("pull closer = bigger").

Ranges: theta ∈ [20, 120]°, z ∈ [-2.6, -0.35] m (near clamp moved from 0.5 m to 0.35 m — the whole point is matching very close objects). Under the lock the width floor drops to 0.05 m (the angle clamp is the real limit); free mode keeps 0.2 m.

10-subtitles.js — API additions

setLockAngle(on) / getLockAngle(): boolean
setAngleDeg(deg) / getAngleDeg(): number     // locked angular width, degrees
getPlacement(): { y, z, width }              // the resolved, clamped triple
  • setPlacement({ y, z, width }) under the lock:
    • z without y → depth-only: y is re-derived from the held y/z ratio and width from subAngleDeg;
    • width without z → size change: subAngleDeg is re-derived from the requested width at the current depth (the derived width then wins);
    • y + z (grab & drag) → the caller's y wins, only width is derived.
  • resetPlacement() restores { y:-0.42, z:-1.45, width:2.0 } and subAngleDeg = 69.2; the lock flag is left alone.
  • New diagnostics: stats.subAngleDeg, stats.subLockAngle, stats.subYOverZ, and stats.subTextDeg — the ACTUAL angular height of the drawn subtitle ink, in degrees. arXiv 2004.01545 measures preferred angular size (character height) at ~16–18 dmm ≈ 0.9–1.0° for medium/long text and ~32–40 dmm ≈ 1.8–2.3° for titles/headlines; Google's dmm is 1 mm of character height at 1 m. Use subTextDeg to pick a subAngleDeg default from real measurements later; the shipped default is deliberately the on-device-proven look.

20-controls.js — subtitle row is now NINE buttons

sub_up  sub_down  sub_near  sub_far  sub_smaller  sub_bigger  sub_opaque_down  sub_opaque_up  sub_reset
  上       下        近        远        小            大            暗               亮           复位
  • The vertical-position pair (up/down) and the depth pair (near/far) are the ADDENDUM-6 additions.
  • view.sub gains angleDeg, lock and dist (= |z|). The caption ALWAYS leads with the depth in metres — 字幕 距1.45m · 69° while locked, 字幕 距1.45m · 2.0m while free — because the depth is the number the user has to match against the object being watched. Opacity stays right-aligned.
  • readSubtitles() bug fixed: it only read the module-local S.subtitlesOn, which 90-main.js never set (it calls setVideo(video) with no getExtra), so the CC button always rendered as OFF. It now falls back to the shared state.subtitles. The button also got an unambiguous on/off treatment: an accent pill + 字幕开 when on, a slashed CC glyph + 字幕关 when off.
  • Hit pills: pitch 0.10, half width 0.052 (a hair of overlap → no dead zone); visual width 0.092; caption font 0.048·h, button label font 0.040·h. The bar itself is unchanged (PANEL_M.expanded = 0.72 × 0.21 m, canvas 1440 × 420 px).

90-main.js

  • Persists subLockAngle / subAngleDeg in jfvr.settings.v1; while locked the stored subPlacement.width is re-derived on load, so subAngleDeg + z is what restores the user's look.
  • In-page panel: a 距离与大小 toggle (锁定视角尺寸 / 自由), and a 视角宽度 slider in degrees shown while locked / the old 宽度 slider in metres while free. All sliders re-read their value whenever a setter runs, so VR-bar and grab changes show up in an already-open panel.
  • sub_up / sub_down step y by ±0.04 m. sub_near / sub_far step z by ±0.15 m. sub_smaller / sub_bigger step subAngleDeg by ∓5° while locked, else width by ∓0.15 m.
  • Grab & drag: while locked, a depth drag also carries y by the grab-start y/z ratio (vertical hand motion still adds on top), so the strip cannot slide out of the lower view when it is pulled close.

ADDENDUM 7 — summon gating, lock badge, pre-VR panel trim

Owner feedback after the ADDENDUM 6 pass:

控制面版的唤出太灵敏了,只有当指示线指向面板附近时,面板再显示出来即可;在 vr 空间 右侧边缘增加一个锁的图标,当选中上锁后,面板不会启动;在刚唤出面板时,0.5s 内的操作 应该忽略,否则可能误触。在进入"vr 模式"时,字幕相关有很多滑块配置,移除哪些配置, 在没有在真实 vr 空间中的调整,都是不准确的,没必要在进入前指定。视角宽度保持默认即可。 字幕相关只保留开启和关闭即可。"vr 播放"这个按钮,不要挡住原有播放器的选项,现在似乎 挡住了原播放器的全屏按钮;手部指向选项直接移除吧,没人会只使用食指作为指向。

1. Summon gating (90-main.js)

There were two independent reasons the bar appeared on every pinch, and both are gone:

Was Now
onSelectStart called Controls.show() unconditionally listener removed; summoning is decided in pointerDown() from the actual aim
pointerDown() called Controls.show() before checking the hit show() is called only when the ray lands on / near the bar
bar hidden  + aim ON the bar  -> show(), record barShownAt, return  (the summoning press never clicks)
bar hidden  + aim NEAR the bar-> same (padded window, see below)
bar hidden  + aim elsewhere   -> nothing
bar hidden  + locked          -> nothing
bar shown   + < PANEL_ARM_MS(500) since it appeared -> ignore
bar shown   + aim ON the bar  -> show() (extends auto-hide) + handleSelect

Barely-visible: the summoning press is consumed — it does not also press a button, and for 500 ms after the bar appears every activation is ignored. Otherwise the hand travelling onto the bar right after the summon fires a random button.

The summon window is deliberately wider than the button window: pointer.hit (used for the cursor and for button hits) uses panelHit(..., 0.25, 0.6), while summoning falls back to panelHit(..., 0.5, 0.9). A ray that merely grazes the bar can still bring it up; the 500 ms arm is what protects against mis-clicks.

Stats: stats.pressAt now reads SUMMON (armed 500ms), IGNORED (arming), MISS (not aiming at the panel) or MISS (panel locked).

2. Lock badge (20-controls.js + 90-main.js)

An always-visible small quad (LOCK_X/Y/Z = 0.86 / -0.34 / -1.45, LOCK_M = 0.075, texture 128²) placed in the same refSpace as the control bar — to the right of it. While state.ctlLock is true, nothing summons the bar; the badge stays visible and hit-testable so it can be unlocked again.

Why refSpace and not a head-locked HUD corner: on this setup the only pointing device is the hand's ray, and a head-locked badge could never be aimed at. "Right-hand edge of the VR space" is therefore implemented as "right of the control bar, in the same world-locked space".

  • Unlocked: dim (60 % white) padlock with the shackle open.
  • Locked: accent (#4ea1ff) filled padlock, shackle closed, keyhole punched out.
  • Hit window: panelHit(..., 0.6, 0.6) ≈ 6.5° across at 1.45 m — the badge itself is only ~3°, so it is easy to hit with a coarse hand ray.
  • The lock is per VR session: enterVR() sets ctlLock = false, onSessionEnd() resets it too. A stuck lock would look exactly like a broken plugin, so it is never restored from localStorage.

New API:

Controls.getLockLayer(): XRLayer | null
Controls.setLock(on) / getLock(): boolean

Stats: stats.ctlLock.

3. Pre-VR settings panel trimmed (90-main.js)

Sliders set on a flat screen do not transfer to a headset, so the only subtitle setting left before entering VR is 开 / 关:

Removed Why
模式 (hud/sphere/off) legacy A/B debug toggle
透明合成, 不透明度 judge-able only in VR
垂直位置, 距离, 宽度 now fully replaced by the in-VR row
距离与大小 (lock toggle) the lock is always ON; it is the whole point
视角宽度 keeps its default; in-VR 小/大 is the control
手部指向 the index-ray mode was removed outright

视角宽度 / subAngleDeg still persists — the in-VR 小/大 buttons write it, and the panel just no longer offers a second, less accurate way to set it. Same for placement / opacity: set in VR, remembered next time.

Removed with it: setSubModeLive(), setSubLockLive(), handRay() and every handRayMode read/write.

4. "VR 播放" button position (90-main.js CSS)

.jfvr-btn moved from bottom: 24px to bottom: 120px — at the viewport's bottom-right it sat on top of Jellyfin's own OSD, covering its fullscreen button. .jfvr-panel moved to bottom: 180px and max-height: calc(100vh - 200px) so it still stacks above the button.


ADDENDUM 7b — the padlock is a PANEL BUTTON, not its own layer

What was wrong with ADDENDUM 7's badge

On device the badge could not be selected while the panel was shown, but could be while it was hidden. ADDENDUM 7 had given the padlock its own quad layer parked at (0.45, -0.34, -1.45) — 0.45 m to the right of the panel centre, i.e. 0.09 m to the right of the panel's own right edge.

Aiming is the binding constraint and it was underestimated: the only pointing device here is the hand ray, and the cursor's own acceptance window (panelHit padding 0.25/0.6 on a 0.72 m panel) reaches x ≈ 0.45 — the panel's right edge. The badge was outside every reachable ray. Worse, its hit test ran on the panel's layer geometry, so whether it fired depended on where the panel happened to be, which is exactly the "works when hidden, not when shown" report.

What it is now

An ordinary button on the panel canvas — the lock action, fifth in the media row:

expanded:  [<<<] [>||] [>>>] [CC] [padlock]
             0.18  0.36  0.54  0.72   0.90     (hit pill half width 0.092)

It travels the same code path as every button that already works: cursor → hitTest → handleSelect → onAction → 90-main.js handleControlAction('lock').

The "locked" panel mode

show() picks the mode from the lock state, so locking answers "what does a summon look like now?" with a third mode:

mode size (m) canvas drawn / hit-testable
expanded 0.72 × 0.21 1440 × 420 everything
locked 0.26 × 0.10 520 × 200 padlock + 解锁 label only
collapsed 0.60 × 0.05 1200 × 100 (legacy; no longer entered)

hitTest(nx, ny, w, h, modeHint) now takes the mode explicitly, because "collapsed" (a thin strip) and "locked" (a small square-ish pill) share no aspect-ratio invariant. handleSelect passes S.mode; the aspect-ratio modeFor() remains only as a fallback for external callers.

hide() no longer resets the mode. The layer therefore keeps the size it will be shown at, so the summon window that 90-main.js ray-casts is the full panel rect (0.72 × 0.21 unlocked, 0.26 × 0.10 locked) instead of a 5 cm strip — and re-showing never needs a resize pop.

Lifecycle (all of it in 90-main.js toggleControlsLock)

lock:    S.ctlLock = true; Controls.setLock(true)  -> mode 'locked'
         Controls.show()                           -> padlock-only panel, auto-hide restarts
         barShownAt = now
unlock:  S.ctlLock = false; Controls.setLock(false) -> mode 'expanded'
         Controls.show()                           -> full panel
         barShownAt = now                          -> 0.5 s ignored, so the unlocking
                                                       press cannot land on a button

Summoning while locked is allowed (it is the only way back out): pointerDown no longer early-returns on S.ctlLock, and the summon shows the padlock panel alone.

Also fixed

handleSelect() used to fall back to (nx, ny) = (0.5, 0.5) when there was no pointer hit and no reticle — which silently pressed whatever button lives at the centre. It now returns null and performs nothing.

API

Controls.setLock(on) / getLock(): boolean    // setLock also swaps the panel in place
Controls.hitTest(nx, ny, w, h, mode?)        // mode: 'expanded' | 'locked' | 'collapsed'
Controls.PANEL_M / PANEL_PX                  // now include .locked

Stats: stats.ctlMode (expanded | locked), stats.ctlLock, stats.pressAt (SUMMON (locked panel), LOCKED/UNLOCKED (armed 500ms), …).


ADDENDUM 8 — panning the video window (yaw / pitch about the eye)

Why

我在观看过程中,有时想调整180度视频的播放位置,让其更适合我的环境,我希望画面可以不是 固定在我的正前方180度,我希望能调整画面内容在整个vr空间中以我眼睛为旋转点进行上下左右 的水平移动(为了解决一些场景需要物理低头才能看到的问题,我不想真的低头去看)

The 180° window was hard-wired dead ahead. Scenes staged below the horizon could only be watched by physically lowering the head.

Model

The video layer is an XREquirectLayer with radius 0 — the content sits at infinity. Two consequences drive the whole design:

  • Only the layer's orientation decides which direction is visible. Translating it does nothing, and there is no parallax, so the picture does not swim when the head moves.
  • So panning the picture is purely layer.transform's rotation, which rotates the content about the eye point — exactly what was asked for. No decode, no texture upload, no extra GPU work: 8K HEVC DirectPlay is untouched.

Only the video rotates. The panel lives at a fixed pose in refSpace and the subtitles are head-locked in viewerSpace, so the UI stays in front of the user while the picture moves.

state.videoYaw     // radians, + = the picture has been moved RIGHT
state.videoPitch   // radians, + = the picture has been moved UP
q = qy(-yaw) * qx(pitch)          // pitch in the content frame, then yaw about world Y
layer.transform = XRRigidTransform({0,0,0}, q)

Because of that composition the window's aim direction is exactly (cos p·sin yaw, sin p, −cos p·cos yaw), so yaw is a pure azimuth and pitch a pure elevation — no gimbal weirdness in the useful range.

Limits: pitch ±(π/2 − 0.02) — i.e. effectively unlimited, per the owner's "看用户拖动幅度来定"; the guard only avoids the degenerate pole. Yaw wraps to [−π, π]. Not persisted: every VR session starts dead ahead.

Interaction — one affordance, in the panel

A drag strip above the progress bar (the owner's requested placement), part of the panel canvas like every other control:

┌─ 画面 → the drag strip (panel y 0.00–0.20) ────────────────────────┐
│ ⠿  拖动调整画面                        偏航 +12°  俯仰 -8°        │
│    连点 3 次复位                       0° / 0° = 正前方            │
└──────────────────────────────────────────────────────────────────┘
  • Pinch the strip and drag → the picture follows the hand 1:1.
  • Three taps inside 1.6 s → reset to dead ahead. Taps are counted on RELEASE, so a real drag can never be mistaken for one.
  • The right side always shows the live angles, so the state is readable even when you are not moving anything.

The drag is driven by the pointer ray's own direction, not by a position on the panel:

yaw'   = wrap(yaw0   + (rayYaw   - rayYaw0))
pitch' = clamp(pitch0 + (rayPitch - rayPitch0))     rayYaw = atan2(dx, -dz)
                                                    rayPitch = asin(dy)

A panel-space cursor would run off the panel's edge after a few degrees of movement; ray rotation has no range limit. It also makes the content point under the cursor stay under the cursor, which is what "grab the picture" should feel like.

Controls.hold(true/false) freezes the panel auto-hide for the duration of the drag — otherwise the panel would vanish mid-drag because the timer show() armed keeps running.

Runtime risk and how it is handled

SUPERSEDED by ADDENDUM 8b — the probe below compared quaternion components, which is sign-sensitive and produced a false "read-only" verdict; and the gesture is now tap = reset / hold 0.5 s = drag. See the addendum at the end of this file.

XRCompositionLayer.transform / XREquirectLayer.transform are settable in the spec, but a runtime may ignore the write. Since the whole interaction depends on it, the module probes once per session:

probeVideoAngleMutable(layer)   // write a known rotation, read it back
  -> stats.videoRotMode = 'live' | 'rebuild'
  • live — a transform write per frame while dragging. Free.
  • rebuild — the media layer is recreated. Far too expensive per frame, so rebuilds happen only on creation / release / reset; during a drag the state updates and the picture catches up when the finger comes up. stats.videoRebuilds counts them.

20-controls.js — changes

  • The expanded panel grew from 0.72 × 0.21 m to 0.72 × 0.26 m (canvas 1440 × 420 → 1440 × 520) to make room for the strip; every row was re-proportioned.
  • New hit band viewHitY0/Y1 = 0..0.20 returning {type:'view_drag'}, spanning the panel width. It does not overlap the seek band (0.30–0.52) or the button bands.
  • view.view = { yawDeg, pitchDeg } feeds the readout (drawViewStrip), and is part of the redraw signature.
  • New hold(on) for the auto-hide pause; stats.ctlHold.

90-main.js — changes

  • state.videoYaw/videoPitch reset to 0 at session start and at session end.
  • S.videoLayer handle (was local), applyVideoAngle(force) / rebuildVideoLayer(), resetVideoAngle(), videoQuat(), rayAngles(), beginViewDrag/updateViewDrag/ endViewDrag, and viewTaps for the triple-tap reset.
  • The frame loop calls updateViewDrag(frame); pointerUp() calls endViewDrag().
  • handleControlAction('view_drag') starts the drag using the press context (lastPressSrc / lastPressFrame) captured in pointerDown.
  • Stats: videoYaw, videoPitch, videoAngleLive, videoRotMode, videoRebuilds, videoReset, viewDrag, viewTaps.

ADDENDUM 8b — live picture pan: two mechanisms, the crash, and the real gesture

Owner feedback on ADDENDUM 8:

触发复位后直接崩溃了;另外,我拖动后,我希望画面立刻就有反应,而不是我松手之后才改变。 按住0.5s后拖动才是拖动,否则算复位

1. Why the picture only moved on release

ADDENDUM 8 decided whether it could write layer.transform with this probe:

layer.transform = new XRRigidTransform(origin, q);
var o = layer.transform.orientation;
live = Math.abs(o.y - q.y) < 1e-4 && Math.abs(o.w - q.w) < 1e-4;

Comparing quaternion components is sign-sensitive — q and −q are the same rotation — and a runtime is free to hand back a renormalised quaternion. So the probe could report "read-only" for a runtime that was perfectly capable of the write. When it did, the drag path took the rebuild branch, and because that branch was throttled to "on release only", the picture sat still until the finger came up. That is exactly what was observed.

The probe now compares the position, which is exact and has no sign ambiguity:

layer.transform = new XRRigidTransform({x:0.123, y:-0.456, z:0.789}, identity);
writable = layer.transform.position matches that marker;

2. Two mechanisms, chosen automatically

videoRotStrategy() -> 'transform'   // preferred: one cheap write per change
                   |  'space'       // layer.space = refSpace.getOffsetReferenceSpace(q)
                   |  'rebuild'     // manual escape hatch ("兼容" in the 2D panel)
  • transform — post-creation layer.transform writes are honoured.
  • space — they are not, so the aim lives in an offset reference space instead. A layer's pose is recomputed from its space every frame, and getOffsetReferenceSpace() is the recentering path every WebXR app uses, so this is the more likely of the two to be fully implemented.
  • The two are mutually exclusive: createVideoLayer() must NOT bake the transform in while the probe is about to write it, or the aim would be applied twice. Only the rebuild strategy bakes it in at creation.
  • layer.transform is never written on the space path, and vice versa.

Switching is automatic per session; stats.videoRotMode reports what was chosen and stats.videoTransformWritable reports the raw probe result. The 2D panel has 「画面旋转: 自动 / 兼容」 to force the rebuild path without a redeploy.

3. The crash

The reset path rebuilt the video layer, and it created the replacement before destroying the old one. Two XRMediaBinding equirect layers bound to the same <video> element at the same time, plus a second rebuild microseconds later (the release path also rebuilt, then the reset rebuilt again), took the browser down.

Every rebuild now:

  1. runs in a setTimeout outside the XR frame callback (never inline in a frame),
  2. destroys the old layer first, then creates the replacement,
  3. is throttled to at most one per REBUILD_MIN_MS (120 ms),
  4. never queues more than one pending rebuild.

The space/transform strategies rebuild not at all, so in the normal case the crash path is not even reachable.

Two more robustness fixes found while testing this:

  • videoSpaceFor() used to let a throwing getOffsetReferenceSpace() escape. It runs inside the XR frame loop, where an exception would stop the loop and freeze the session. It is caught now, and the strategy falls back to rebuild.
  • The whole per-frame body (updatePointer / pollGamepads / updatePanelPose / updateGrabFrame / updateViewDrag / frame hooks) is wrapped in a try/catch. A single bad frame could previously kill the loop: requestAnimationFrame is only re-armed at the end of the callback, so any throw froze the session permanently.

4. The real gesture

press the drag strip
  ├─ release before 500 ms, no movement   -> RESET to dead ahead
  ├─ held past 500 ms, then moved         -> PAN (picture follows the hand)
  └─ held past 500 ms, never moved        -> RESET
  • The pan reference (a0) is captured at the moment the hold threshold passes, not at the press. Otherwise the half second the user spends holding would fling the picture the instant dragging armed.
  • A ~1° dead zone keeps hand jitter from turning a reset tap into a tiny pan.
  • The strip now reads 点一下复位 · 按住 0.5s 再拖动 instead of the retired triple-tap hint.

stats.viewDrag walks through it: pressed (hold 500ms to pan) → armed (drag to pan) → ended (moved) / reset (tap).

5. Verification

Three offline suites, all against the real sources (build.py --check passes too):

test-ad8 — quaternion + panel geometry: aim for ±yaw/±pitch, unit norm, exact azimuth/elevation recovery, yaw wrap, pitch guard, "aimed point stays under the cursor"; panel bands on the 1440×520 canvas (strip 0–0.20, inert readout 0.20–0.30, seek 0.30–0.52, media 0.52–0.74, subtitle 0.84–1.0, no dead column); the strip's drawn readout; hold().

test-gesture — the state machine above, driven with a stubbed ray: quick tap resets; hold-without-movement resets; movement inside the hold window is ignored (no fling); after arming a +10°/+5° ray move yields exactly +10°/+5°; a real pan does not reset; the dead zone is respected; pitch cannot pass the guard.

test-mech — the mechanism: the position-based probe returns true for a writable transform and false for an ignored one; the transform strategy writes and schedules no rebuild; the space strategy assigns an offset space, leaves the transform alone (no double rotation) and schedules no rebuild; unchanged angles reuse the cached space; the rebuild strategy queues at most one, and its observed order is removeLayer -> createVideoLayer -> addLayer; a throwing/absent getOffsetReferenceSpace() is reported and permanently falls back to rebuild.

ADDENDUM 9 — subtitle pipeline survival + normal (mono) subtitles

Two independent defects, found on device with the VR session left in its broken state.

1. The rAF broker lost Jellyfin's subtitle loop at session start

90-main.js no longer just forwards to native requestAnimationFrame outside XR. It always holds callbacks in its own queue and drives them with one of two drivers:

  • outside XR: a native rAF driver that keeps exactly one native callback in flight (nativeRafStep re-arms before draining and cancels itself when the queue empty);
  • inside XR: video.requestVideoFrameCallback, plus a 50 ms idle fallback so a paused / stalled / ended video still redraws (paused seek, subtitle track switch).

Why: requestSession('immersive-vr') stops native rAF completely. Jellyfin 12.x renders subtitles with SubtitlesOctopus, whose main thread starts one long-lived rAF loop (rafId) when the video loads, draws the canvas from it and requests the next prerender from the worker (oneshot-render). A callback still pending in the native scheduler when the session starts is destroyed with it, and because rafId stays non-zero the loop is never restarted.

Evidence (live instance, via CDP closure inspection):

rafId                  : 1150            <- native id, not the plugin's 1e9 range
oneshotState.eventStart: 945.99          <- canvas frozen ~7 min behind the playhead
renderRequested        : false
oneshot-render requests during 15 s of playback: 0

After the fix, in a live VR session: rafId = 1000028035, eventStart follows video.currentTime, and stats.subUploads climbs while subtitles change.

startRafPump() must not clear the queue (that callback is the one being rescued) and must drop the stale nativeId. stopRafPump() re-arms the native driver.

2. Auto dual-eye / mono subtitle layout

10-subtitles.js used to assume the libass canvas is a stereo left/right pair and always sampled the left half. A normal subtitle (e.g. the CN SRT track) is centred across the whole 2:1 frame, so half of it was cut off.

state.subLayout ('auto' | 'dual' | 'mono', default 'auto', persisted) selects the region; stats.subLayout / subDual / subEyeW / subCopyND / subCenterInk report it.

Per sample (on the 320×160 downsample) the module builds the ink box and column count of each half plus the ink in a ±40 source-px band around the seam, then:

only the left half has ink      -> dual   (sample the left half)
only the right half has ink     -> mono   (sample the whole width)
ink crosses the seam            -> mono
both halves, seam empty         -> columnCopyND < COPY_TOL ? dual : mono

Dual-eye is NOT a pixel mirror. Both eyes carry the same text at mirrored positions; the glyphs are not flipped, so left[x] vs right[W-1-x] fails badly (measured mean delta 258/1020 → "not a mirror"). What is true is that the two halves' ink column profiles match under a small horizontal shift, because they are the same line. Measured on device:

layout centre-band ink column-copy distance
dual-eye (CN_dual_eye) 0 0.02 – 0.08
mono (CN SRT) 50 / frame 1.00

COPY_TOL = 0.25. A layout change is debounced (two consecutive samples) because it rebuilds the layer.

M.eyeW is one eye's region width (half for dual, full for mono); computeTexDims(), drawHud()'s crop/clamp and the texture scale all use it instead of the old hard-coded halfW.

API additions

__JFVR.Subtitles.setLayout('auto' | 'dual' | 'mono')   // also state.subLayout
__JFVR.Subtitles.getLayout(): string

3. Verification

  • Broker logic replicated exactly in a Node harness against the built bundle: maxConcurrentNative = 1 over 300 frames with a self-rescheduling loop (the first version re-armed during the drain and doubled the driver every frame — that froze the page); simulating session start (native callbacks dropped) leaves the loop in the queue and it keeps firing from the XR driver.
  • columnCopyND extracted from the built bundle and exercised with the on-device measurements: dual 0.00–0.08 → dual, mono centre-ink → mono, no-copy word-gap profile 0.41 → mono.
  • On device (before the UI became unreachable): the broker change was observed to carry SubtitlesOctopus's loop through immersive-vr (rafId in the plugin's id range, eventStart tracking, subUploads 4 → 7 over 20 s in VR).

ADDENDUM 10 — normal (non-ASS) subtitles: Jellyfin does not use libass for them

1. Why normal subtitles were invisible

The HUD samples canvas.libassjs-canvas. That canvas only exists when Jellyfin routes the subtitle through SubtitlesOctopus (libass). From htmlVideoPlayer-plugin (renderTracksEvents):

a = (n.Codec || '').toLowerCase();
if (!X.includes(a)) {              // X = ['ssa','ass']
    if (a === 'pgssub') -> renderPgs
    if (Y.includes(a)) -> renderVobSub      // Y = ['dvdsub','vobsub']
    if (!u(h.fe))      -> renderSubtitlesWithCustomElement   // DOM overlay
    /* else falls through to the native TextTrack path */
}
return renderSsaAss(...);          // ssa/ass ONLY

So subrip/vtt/… never reach libass. Observed on device with the CN SRT track selected: canvas.libassjs-canvas count 0, .libassjs-canvas-parent 0, .videoSubtitles 0 — and video.textTracks[0] = { kind:'subtitles', label:'manualTrack', mode:'showing', cues:315 }, i.e. the browser is rendering the cues through ::cue. There was nothing for the plugin to read, at any layout setting.

2. What the module does now

updateSource() runs every XR frame and picks the source:

  • a canvas.libassjs-canvas with a non-zero size → srcKind = 'libass' (unchanged path);
  • otherwise, if the player has a showing TextTrack or a .videoSubtitlesInner overlay → srcKind = 'native'. The active text comes from textTracks[i].activeCues (fallback: the DOM overlay's innerText), is cleaned of ASS/HTML tags and bidi marks, wrapped, and painted — white with a black outline, centred — into a private 1920×1080 canvas. That canvas is fed through the exact same crop/bbox/upload code as the libass one.
  • neither → srcKind = 'none' and the strip is cleared once.

prepare() is a no-op without a libass canvas (blowing up the player box is only needed so libass re-renders bigger; the native canvas is ours).

New diagnostics: stats.subSource ('libass' | 'native' | 'none'), stats.subNativeText; __JFVR.Subtitles.getSourceKind().

3. The persisted layout override was a trap

ADDENDUM 9 persisted state.subLayout. A stale 'mono' in jfvr.settings.v1 silently forced the mono branch on the default dual-eye content: dual:false, eyeW:4320, and a crop spanning the seam, so the HUD showed both eyes' text side by side. Debugging that cost more time than the detection itself.

subLayout is therefore not persisted (same reasoning as subtitles and subMode:'off' — a stored value that leaves the layer looking broken). It is 'auto' at boot, toggleable per session from the panel.

4. Verification (on device, build hud-9)

selected track subSource result
CN_dual_eye (ass, idx 0) libass dual:true, eyeW:2160, crop:[1040,768,1176,924] (left half), copyND:0.055, uploads climbing
CN (subrip, idx 1) native dual:false, eyeW:1920, canvas 1920×1080, text:"进去了呢", crop 256×88, uploads climbing

The private cue canvas was read back and OCR'd ("怎么样? 好好地" for cue "怎么样?好好地"), and injecting/removing a fake canvas.libassjs-canvas flips subSource between libass and native with no errors.

ADDENDUM 11 — the pointer follows the hand that is acting / aiming

With two hands, BOTH report a tracked-pointer ray, and the runtime's order is not "the hand the user is using". updatePointer() took the first input source with a valid ray and breaked, and pointerDown() called updatePointer(frame) without passing the source from the select event. Consequences on device:

  • the guide line / cursor stayed pinned to one hand even while the user aimed with the other (so the guide did not point where they were pointing);
  • the other hand's pinch landed wherever the pinned hand pointed — the left hand controlled the click position while the right hand performed the click.

updatePointer(frame, preferSrc) now builds a candidate per tracked-pointer source (with its panelHit), and pickPointerCand() chooses:

1. preferSrc                     the source acting right now (select / gamepad press)
2. last-active source            while it is aiming at the panel (hit.inBounds)
3. any source aiming at the panel (first in runtime order)
4. last-active source            (sticky)
5. the first source

pointerDown() passes the event's inputSource, so an action is always evaluated with the acting hand's ray, and the choice sticks afterwards (state.stats.activeSrc). pointerUp/session end reset it.

Verification

There is no official way to fake hand input on a Quest for a WebXR page: Meta's simulator (the only "simulate aim/poke/pinch/grab" tool in the docs) is a desktop tool for Unity/Unreal native apps, adb has no input-injection for XR, and CDP has no WebXR domain. So the live check injected synthetic input sources from the page: each targetRaySpace is a real XRRigidTransform offset reference space of the live session, and a press is driven through the real gamepad path (a rising edge on gamepad.buttons[0]), which reaches the same pointerDown(src, frame, …) the select event uses. XRSession.prototype.inputSources is overridden for the test and restored afterwards.

Measured in a live immersive-vr session:

initial              pointer hand:left  HIT 0.22,0.50    activeSrc hand:left
right hand press     pointer hand:right HIT 0.78,0.50    activeSrc hand:right   lastActivate gamepad:right
right hand release   pointer hand:right HIT 0.78,0.50    (sticky)
left hand press      pointer hand:left  HIT 0.22,0.50    activeSrc hand:left
right press, right hand NOT on panel
                     pressAt "MISS (not aiming at the panel)"   <- did NOT use the left hand's position

pickPointerCand is also extracted from the built bundle and unit-tested (9 cases).