hud-7: live picture pan; fix the reset crash; tap-to-reset / hold-to-drag

Reported after hud-6:
  触发复位后直接崩溃了;我拖动后,我希望画面立刻就有反应,而不是我松手之后才改变。
  按住0.5s后拖动才是拖动,否则算复位

1. Why the picture only moved on release — a bad probe
The transform-write probe compared quaternion COMPONENTS. q and -q are the same
rotation and a runtime may hand back a renormalised quaternion, so a perfectly
capable runtime could be judged "read-only". When it was, the drag took the
rebuild branch, and that branch was throttled to "on release only" — hence the
picture sitting still until the finger came up. The probe now compares the
POSITION (exact, no sign ambiguity).

2. Two mechanisms, auto-selected and mutually exclusive
   'transform' -> layer.transform = q              (one write per change)
   'space'     -> layer.space = refSpace.getOffsetReferenceSpace(q)
                     (the recentering path every WebXR app uses; a layer pose is
                      recomputed from its space each frame)
   'rebuild'   -> recreate the media layer (manual escape hatch, "兼容" in the 2D panel)
createVideoLayer() only bakes the transform in for 'rebuild' — otherwise the
probe's own write would apply the aim twice. layer.transform is never written on
the 'space' path and vice versa. stats.videoRotMode / videoTransformWritable
report what was chosen.

3. The crash — create-before-destroy
The reset created the replacement layer BEFORE destroying the old one: two
XRMediaBinding equirect layers bound to one <video> element at once, then a
second rebuild microseconds later (release rebuilt, then reset rebuilt again),
took the browser down. A rebuild now runs in a setTimeout OUTSIDE the XR frame
callback, destroys FIRST, is throttled to 120 ms, and never queues more than one.
The transform/space paths never rebuild at all.

Two robustness holes found while testing:
  * videoSpaceFor() let a throwing getOffsetReferenceSpace() escape into the frame
    loop, where an exception kills the loop for good (rAF is only re-armed at the
    end of the callback) — a frozen session. Caught now, demoting to 'rebuild'.
  * the whole per-frame body is wrapped in try/catch for the same reason.

4. The gesture the owner asked for
   press the drag strip
     released before 500 ms, no movement -> RESET to dead ahead
     held past 500 ms, then moved        -> PAN
     held past 500 ms, never moved       -> RESET
The pan reference is captured when the hold threshold passes, not at the press,
so the hold itself cannot fling the picture; a ~1 deg dead zone absorbs jitter.
The strip now reads 点一下复位 · 按住 0.5s 再拖动 (the triple-tap hint is gone).

Verified offline against the real sources by three suites (build.py --check too):
  test-gesture  the state machine above, stubbed ray/clock: quick tap resets;
                hold-without-move resets; movement inside the hold window is
                ignored; after arming a +10/+5 deg ray move gives exactly
                +10/+5 deg; a pan does not reset; dead zone respected; pitch guard
  test-mech     position probe true/false for writable/ignored; transform
                strategy writes with no rebuild; space strategy assigns the offset
                space and leaves the transform alone; unchanged angle reuses the
                cached space; rebuild queues at most one and runs
                removeLayer -> createVideoLayer -> addLayer; throwing or absent
                getOffsetReferenceSpace is reported and falls back permanently
  test-ad8      quaternion aim/wrap/guard maths, panel bands on the 1440x520
                canvas, the strip's drawn readout, hold()
Deployed as mvp-5217a3 (hud-7); confirmed live on the Quest, errors empty.
This commit is contained in:
2026-10-11 21:47:55 +08:00
parent 263e270747
commit 7f751b7eee
7 changed files with 672 additions and 161 deletions
+207 -74
View File
@@ -124,6 +124,13 @@
S.subOpacity = num(o.subOpacity, 0.1, 1.0, S.subOpacity);
S.ctlAutoHideMs = num(o.ctlAutoHideMs, 1000, 10000, S.ctlAutoHideMs);
if (o.subMode === 'hud' || o.subMode === 'sphere') S.subMode = o.subMode;
// ADDENDUM 8: how the picture aim is applied. 'space' is the live path; the
// 'rebuild' escape hatch exists because mutating a media layer's transform is
// ignored by this runtime, so if an offset reference space turns out not to be
// honoured either, this is how we get movement back without a redeploy.
if (o.videoRotMode === 'auto' || o.videoRotMode === 'space' || o.videoRotMode === 'rebuild') {
S.videoRotMode = o.videoRotMode;
}
// 'off' is deliberately NOT restored. Persisting it had the same effect as
// persisting subtitles:false - the subtitle layer was never created on the
// next session, which looked like a loading failure. Same for subAlpha,
@@ -154,6 +161,7 @@
subAlpha: S.subAlpha,
projection: S.projection,
stereo: S.stereo,
videoRotMode: S.videoRotMode,
controls: S.controls,
ctlAutoHideMs: S.ctlAutoHideMs
}));
@@ -407,6 +415,18 @@
'字幕的位置 / 距离 / 大小 / 不透明度都在 VR 里用控制条调(控制条最下面一行),' +
'调好会自动记住。VR 空间右侧有个锁图标:锁上后面板不会再被唤出。'));
// ADDENDUM 8: the picture-pan mechanism. Only touch this if panning does not
// move the picture live (see stats.videoRotMode).
body.appendChild(segRow('画面旋转', function () { return S.videoRotMode || 'auto'; },
function (v) {
S.videoRotMode = v;
if (S.session) {
try { videoSpaceCache = null; S.videoTransformLive = undefined; applyVideoAngle(true); } catch (e) {}
}
saveSettings();
},
[{ label: '自动', value: 'auto' }, { label: '兼容', value: 'rebuild' }]));
body.appendChild(NS.makeEl('div', 'jfvr-sec', '控制'));
body.appendChild(segRow('VR 内控制条', function () { return S.controls ? 'on' : 'off'; },
function (v) { S.controls = v === 'on'; saveSettings(); },
@@ -623,8 +643,11 @@
// then the transform is applied so probe + apply see the real layer.
S.videoYaw = 0;
S.videoPitch = 0;
probeVideoAngleMutable(vlayer);
applyVideoAngle(false); // undo the probe's test rotation
videoSpaceCache = null;
S.videoTransformLive = probeVideoTransformWritable(vlayer);
S.stats.videoTransformWritable = !!S.videoTransformLive;
applyVideoAngle(true); // restores the probe marker, sets the aim
log('video aim: ' + videoRotStrategy() + ' (transform writable: ' + !!S.videoTransformLive + ')');
updateVideoAngleStats();
}
} catch (e) { NS.recordError('video layer', e); made.push('video:err'); }
@@ -677,12 +700,19 @@
frames = 0; lastT = time;
}
if (frame) {
updatePointer(frame);
pollGamepads(frame);
updatePanelPose(frame, dt);
updateGrabFrame(frame);
updateViewDrag(frame);
NS.runFrameHooks(frame, time, session);
// One bad frame must never stop the loop: if this throws, rAF is never
// requested again and the whole session freezes.
try {
updatePointer(frame);
pollGamepads(frame);
updatePanelPose(frame, dt);
updateGrabFrame(frame);
updateViewDrag(frame);
NS.runFrameHooks(frame, time, session);
} catch (e) {
NS.recordError('frame', e);
S.stats.frameErr = (e && e.message) || String(e);
}
}
session.requestAnimationFrame(loop);
};
@@ -713,12 +743,20 @@
upperVerticalAngle: Math.PI / 2,
lowerVerticalAngle: -Math.PI / 2
};
// ADDENDUM 8: the transform is the picture's aim. radius stays 0 (content at
// infinity) so this is a pure rotation about the eye point. The transform is
// what makes a rebuild-based runtime possible, and what makes the very first
// frame already point where the user left it (a mid-session rebuild).
var t = videoAngleTransform();
if (t) opts.transform = t;
// ADDENDUM 8: the picture's aim. radius stays 0 (content at infinity), so this
// is a pure rotation about the eye point.
// 'space' -> the rotation lives in an offset reference space, which a layer
// pose is recomputed from every frame (live updates).
// 'rebuild' -> the rotation is supplied as the creation transform, which is
// the only form this runtime honours.
// The transform must NOT be baked in for the transform strategy: the probe
// writes to it right after creation, and baking it in as well would apply the
// aim twice. Only the rebuild strategy bakes it in, because a fresh layer is the
// only surviving form of the write it can rely on.
if (videoRotStrategy() === 'rebuild') {
var t = videoAngleTransform();
if (t) opts.transform = t;
}
var layer = new XRMediaBinding(session).createEquirectLayer(video, opts);
log('video layer', JSON.stringify({ layout: opts.layout, central: opts.centralHorizontalAngle,
yaw: Math.round(S.videoYaw * 180 / Math.PI), pitch: Math.round(S.videoPitch * 180 / Math.PI) }));
@@ -744,9 +782,10 @@
grab = null;
S.stats.grab = 'reset';
viewDrag = null;
viewTaps = [];
cancelVideoRebuild();
videoSpaceCache = null;
S.videoTransformLive = undefined;
S.videoLayer = null;
S.videoAngleLive = false;
S.videoYaw = 0;
S.videoPitch = 0;
if (NS.Controls && NS.Controls.hold) { try { NS.Controls.hold(false); } catch (e) {} }
@@ -1049,67 +1088,154 @@
function updateVideoAngleStats() {
S.stats.videoYaw = Math.round(S.videoYaw * 180 / Math.PI);
S.stats.videoPitch = Math.round(S.videoPitch * 180 / Math.PI);
S.stats.videoAngleLive = !!S.videoAngleLive;
S.stats.videoRotMode = videoRotStrategy();
}
// XREquirectLayer.transform is settable in the spec, but a runtime is free to
// ignore the write, so probe it once: write a known rotation and read it back.
function probeVideoAngleMutable(layer) {
S.videoAngleLive = false;
if (!layer || typeof XRRigidTransform === 'undefined') return;
/* ------------------------------------------------------------------
* HOW the aim is applied. Measured on the Quest 3 browser:
*
* * A transform passed to createEquirectLayer IS honoured -> rebuilding the
* media layer visibly moves the picture. That is the only reason the first
* version appeared to work "on release".
* * Writing layer.transform AFTER creation is ignored: the drag updated the
* angle every frame but nothing moved until the layer was rebuilt.
*
* So the primary strategy changes the layer's SPACE instead. getOffsetReference-
* Space is the recentering path every WebXR app uses for teleport/roomscale, and
* a layer's pose is recomputed from its space each frame, so assigning a rotated
* offset space rotates the picture live:
*
* layer.space = refSpace.getOffsetReferenceSpace(XRRigidTransform(0, q))
*
* The fallback ('rebuild') recreates the media layer with the transform supplied
* at creation. It is throttled, done OUTSIDE the XR frame callback, and destroys
* the old layer BEFORE creating the replacement - doing it inline with two
* XRMediaBinding layers alive on the same <video> crashed the browser.
* ------------------------------------------------------------------ */
function videoRotStrategy() {
if (S.videoRotMode === 'rebuild') return 'rebuild'; // manual escape hatch
if (S.videoRotMode === 'space') return 'space'; // manual
return S.videoTransformLive === false ? 'space' : 'transform';
}
// Is a post-creation `layer.transform` write honoured?
//
// Compare the POSITION, not the orientation. The first version compared
// orientation components and was sign/normalisation sensitive, so a perfectly
// good runtime could look read-only - and that false negative is what made the
// drag fall back to rebuilding the layer. Positions are exact.
function probeVideoTransformWritable(layer) {
if (!layer || typeof XRRigidTransform === 'undefined') return false;
var marker = { x: 0.123, y: -0.456, z: 0.789 };
try {
var q = videoQuat(0.05, 0.05);
layer.transform = new XRRigidTransform(videoOrigin(), q);
var o = layer.transform && layer.transform.orientation;
S.videoAngleLive = !!(o && Math.abs(o.y - q.y) < 1e-4 && Math.abs(o.w - q.w) < 1e-4);
layer.transform = new XRRigidTransform(marker, { x: 0, y: 0, z: 0, w: 1 });
var p = layer.transform && layer.transform.position;
var okPos = !!p && Math.abs(p.x - marker.x) < 1e-6 &&
Math.abs(p.y - marker.y) < 1e-6 && Math.abs(p.z - marker.z) < 1e-6;
return okPos;
} catch (e) {
S.videoAngleLive = false;
}
S.stats.videoRotMode = S.videoAngleLive ? 'live' : 'rebuild';
if (!S.videoAngleLive) {
log('video layer transform is read-only -> rebuilding the layer on changes');
return false;
}
}
// Push the current angles into the layer. On a live runtime this is just a
// transform write. On a runtime that ignores the write we must REBUILD the media
// layer — which is far too expensive to do once per drag frame, so rebuilds
// happen only when `force` is set (creation, release, reset).
function applyVideoAngle(force) {
var layer = S.videoLayer;
if (!layer || !S.session) return;
if (S.videoAngleLive) {
try {
layer.transform = videoAngleTransform();
if ('needsRedraw' in layer) layer.needsRedraw = true;
} catch (e) {
S.videoAngleLive = false;
S.stats.videoRotMode = 'rebuild';
}
var videoSpaceCache = null;
function videoSpaceFor(yaw, pitch, force) {
if (typeof XRRigidTransform === 'undefined' || !S.refSpace) return null;
if (typeof S.refSpace.getOffsetReferenceSpace !== 'function') return null;
var c = videoSpaceCache;
if (!force && c && Math.abs(c.yaw - yaw) < 1e-4 && Math.abs(c.pitch - pitch) < 1e-4) return c.space;
// A throwing getOffsetReferenceSpace must not escape: this runs inside the XR
// frame loop, where an exception would stop the loop and freeze the session.
var sp;
try {
sp = S.refSpace.getOffsetReferenceSpace(
new XRRigidTransform(videoOrigin(), videoQuat(yaw, pitch)));
} catch (e) {
S.stats.videoSpace = 'throw: ' + ((e && e.message) || e);
return null;
}
if (!S.videoAngleLive && force) rebuildVideoLayer();
updateVideoAngleStats();
if (!sp) return null;
videoSpaceCache = { space: sp, yaw: yaw, pitch: pitch };
return sp;
}
function rebuildVideoLayer() {
var REBUILD_MIN_MS = 120; // rebuild strategy: at most ~8 recreations/second
var rebuildTimer = null;
var lastRebuildAt = 0;
function scheduleVideoRebuild() {
if (rebuildTimer != null) return; // one pending is enough
var wait = Math.max(0, REBUILD_MIN_MS - (performance.now() - lastRebuildAt));
rebuildTimer = setTimeout(function () {
rebuildTimer = null;
lastRebuildAt = performance.now();
rebuildVideoLayerNow();
}, wait);
}
function cancelVideoRebuild() {
if (rebuildTimer != null) { clearTimeout(rebuildTimer); rebuildTimer = null; }
}
function rebuildVideoLayerNow() {
var session = S.session;
if (!session || !S.video) return;
// Destroy FIRST: two XRMediaBinding layers bound to one <video> element at the
// same time is what took the browser down.
var old = S.videoLayer;
S.videoLayer = null;
if (old) {
try { NS.removeLayer(old); } catch (e) {}
try { if (typeof old.destroy === 'function') old.destroy(); } catch (e) {}
}
try {
var fresh = createVideoLayer(session);
if (!fresh) return;
var old = S.videoLayer;
S.videoLayer = fresh;
NS.addLayer(fresh);
if (old) {
NS.removeLayer(old);
try { if (typeof old.destroy === 'function') old.destroy(); } catch (e) {}
}
S.stats.videoRebuilds = (S.stats.videoRebuilds || 0) + 1;
} catch (e) {
NS.recordError('videoRebuild', e);
}
}
// Called for every angle change (and every drag frame). In 'space' mode this is a
// single space assignment; in 'rebuild' mode it schedules a throttled rebuild.
function applyVideoAngle(force) {
var layer = S.videoLayer;
if (!layer || !S.session) return;
if (videoRotStrategy() === 'transform') {
try {
layer.transform = videoAngleTransform();
if ('needsRedraw' in layer) layer.needsRedraw = true;
S.stats.videoRotMode = 'transform';
} catch (e) {
// Demote to the space path for the rest of the session.
S.videoTransformLive = false;
S.stats.videoTransformErr = (e && e.message) || String(e);
}
}
if (videoRotStrategy() === 'space') {
var sp = videoSpaceFor(S.videoYaw, S.videoPitch, !!force);
if (sp) {
try {
layer.space = sp;
S.stats.videoSpace = 'assigned';
} catch (e) {
S.stats.videoSpace = 'err: ' + ((e && e.message) || e);
S.videoRotMode = 'rebuild'; // fall back for good
}
} else {
S.stats.videoSpace = 'unavailable';
S.videoRotMode = 'rebuild';
}
}
if (videoRotStrategy() === 'rebuild') scheduleVideoRebuild();
updateVideoAngleStats();
void force;
}
function resetVideoAngle(why) {
S.videoYaw = 0;
S.videoPitch = 0;
@@ -1124,10 +1250,12 @@
* no range limit (a panel-space cursor would run off the panel's edge).
* -------------------------------------------------------------- */
var viewDrag = null;
var VIEW_TAP_MS = 1600; // three taps inside this window reset the view
var viewTaps = [];
var lastPressSrc = null; // source / frame of the last accepted press, for
var lastPressFrame = null; // hold-style actions (the picture drag)
// Owner's gesture: HOLD the strip for half a second and move -> pan the picture.
// Anything else (a plain tap, or a hold with no movement) resets to dead ahead.
var VIEW_HOLD_MS = 500;
var VIEW_DEADZONE = 0.017; // ~1 deg of ray movement before a drag counts
function rayAngles(ray) {
if (!ray || !ray.dir) return null;
@@ -1144,8 +1272,11 @@
var ray = rayOf(frame, src);
var a = rayAngles(ray);
if (!a) { S.stats.viewDrag = 'no ray'; return; }
viewDrag = { src: src, a0: a, yaw0: S.videoYaw, pitch0: S.videoPitch, moved: false };
S.stats.viewDrag = 'started';
viewDrag = {
src: src, t0: performance.now(), armed: false,
a0: null, yaw0: S.videoYaw, pitch0: S.videoPitch, moved: false
};
S.stats.viewDrag = 'pressed (hold ' + VIEW_HOLD_MS + 'ms to pan)';
if (NS.Controls && NS.Controls.hold) { try { NS.Controls.hold(true); } catch (e) {} }
}
@@ -1155,8 +1286,21 @@
if (!ray) return;
var a = rayAngles(ray);
if (!a) return;
if (!viewDrag.armed) {
// Still inside the hold window: this may yet turn out to be a reset tap.
if (performance.now() - viewDrag.t0 < VIEW_HOLD_MS) return;
// Armed. Measure deltas from HERE, not from the press, so the half second the
// user spent holding does not fling the picture.
viewDrag.armed = true;
viewDrag.a0 = a;
S.stats.viewDrag = 'armed (drag to pan)';
return;
}
var dyaw = wrapYaw(a.yaw - viewDrag.a0.yaw);
var dpitch = a.pitch - viewDrag.a0.pitch;
if (Math.abs(dyaw) < VIEW_DEADZONE && Math.abs(dpitch) < VIEW_DEADZONE) return;
var ny = wrapYaw(viewDrag.yaw0 + dyaw);
var np = clampPitch(viewDrag.pitch0 + dpitch);
if (Math.abs(ny - S.videoYaw) < 1e-4 && Math.abs(np - S.videoPitch) < 1e-4) return;
@@ -1166,31 +1310,20 @@
applyVideoAngle(false);
}
// Release. A press that never moved counts as a TAP: three taps inside
// VIEW_TAP_MS reset the view (the owner's "连点 3 次复位"). Counting taps on
// RELEASE rather than on press means a real drag cannot be mistaken for one.
// Release. A press that never panned is a reset (owner: "按住 0.5s 后拖动才是
// 拖动,否则算复位"), which also means a hold with no movement resets.
function endViewDrag() {
if (!viewDrag) return;
var moved = viewDrag.moved;
viewDrag = null;
if (NS.Controls && NS.Controls.hold) { try { NS.Controls.hold(false); } catch (e) {} }
if (!S.videoAngleLive) applyVideoAngle(true); // pay the rebuild once, on release
applyVideoAngle(true); // settle the final angle
if (moved) {
viewTaps = [];
S.stats.viewDrag = 'ended (moved)';
return;
}
var now = performance.now();
viewTaps = viewTaps.filter(function (t) { return now - t < VIEW_TAP_MS; });
viewTaps.push(now);
S.stats.viewTaps = viewTaps.length;
if (viewTaps.length >= 3) {
viewTaps = [];
resetVideoAngle('triple-tap');
S.stats.viewDrag = 'reset (triple tap)';
return;
}
S.stats.viewDrag = 'tap ' + viewTaps.length + '/3';
resetVideoAngle('tap');
S.stats.viewDrag = 'reset (tap)';
}
/* ================================================================== *