hud-6: pan the 180° video window about the eye; padlock as a panel button

ADDENDUM 8 — panning the video window (yaw/pitch about the eye)
The 180° window was hard-wired dead ahead, so scenes staged below the horizon
could only be watched by physically lowering the head. The video layer is an
XREquirectLayer with radius 0, i.e. the content sits at infinity: only its
ORIENTATION is visible and translating it does nothing. So panning is purely
layer.transform's rotation, which rotates the content about the eye point — no
parallax, no swim when the head moves, and no decode/upload cost (8K HEVC
DirectPlay untouched). Only the video rotates; the panel (refSpace, fixed pose)
and the subtitles (viewer space, head-locked) stay in front of the user.

    state.videoYaw   (+ = picture moved RIGHT)
    state.videoPitch (+ = picture moved UP)
    q = qy(-yaw) * qx(pitch)   -> aim = (cos p·sin yaw, sin p, -cos p·cos yaw)

Interaction is one affordance in the panel, above the progress bar (the owner's
requested placement): a drag strip. Pinch and drag -> the picture follows the
hand 1:1; three taps within 1.6 s reset it (taps counted on RELEASE, so a drag
cannot be mistaken for one); the live angles are printed on the strip. The drag
is driven by the RAY's own direction delta rather than a cursor position on the
panel: a panel-space cursor leaves the panel after a few degrees, while ray
rotation has no range limit and keeps the content point under the cursor.
Controls.hold() freezes the panel auto-hide for the drag's duration.

Pitch is limited only by a gimbal guard (±(π/2 - 0.02)) per "不限"; yaw wraps to
[-π, π]. Both reset to 0 on every VR session — not persisted.

Runtime risk: XREquirectLayer.transform is settable per spec but a runtime may
ignore the write, so it is probed once per session (write a known rotation, read
it back) -> stats.videoRotMode = 'live' (transform write per frame) or
'rebuild' (recreate the media layer, but only on creation/release/reset, never
per frame). THIS PROBE NEEDS A REAL XR SESSION — it is the one thing not
verifiable offline.

ADDENDUM 7b (previous pass, in this commit's tree) — the padlock
The padlock had been given its own quad layer parked 0.45 m to the right of the
panel. On device it could not be selected while the panel was shown but could
when it was hidden. Aiming is the binding constraint: with hands the only
pointing device is a ray, and the cursor's acceptance window reaches the panel's
own right edge, so the badge was outside every reachable ray; its hit test also
used the panel's layer geometry, which is why the behaviour flipped with panel
visibility. It is now an ordinary panel button (media row, fifth slot) using the
same hitTest/handleSelect path as every other button, and "locked" is a third
panel mode (0.26 × 0.10 m) that draws and hit-tests ONLY the padlock. Locking
swaps the panel in place and lets it auto-hide; the next summon shows the padlock
alone; unlocking restores the full panel and ignores presses for 500 ms.
handleSelect() no longer falls back to pressing the panel centre when no ray hit
exists.

Panel: expanded grew 0.72 × 0.21 m -> 0.72 × 0.26 m (canvas 1440 × 520) for the
drag strip; every row re-proportioned.

Verified offline against the real bundle: quaternion/drag maths (aim direction
for ±yaw/±pitch, unit norm, azimuth/elevation recovery, yaw wrap, pitch clamp,
"aimed point ends up under the cursor"), panel hit bands on the new canvas
(strip 0..0.20, inert readout 0.20..0.30, seek 0.30..0.52, media 0.52..0.74,
subtitle 0.84..1.0, no dead column), the strip's drawn readout, and hold().
build.py --check passes; deployed as mvp-2e32e6 and confirmed live on the Quest
(hud-6, videoYaw/videoPitch 0, errors empty).
This commit is contained in:
2026-10-11 19:59:25 +08:00
parent 838675baa6
commit 263e270747
7 changed files with 1103 additions and 87 deletions
+231 -2
View File
@@ -615,7 +615,18 @@
// video (bottom) -> hands -> subtitles -> controls (top)
try {
var vlayer = createVideoLayer(session);
if (vlayer) { NS.addLayer(vlayer); made.push('video'); }
if (vlayer) {
NS.addLayer(vlayer);
S.videoLayer = vlayer;
made.push('video');
// ADDENDUM 8: angles start dead ahead every session (never restored),
// then the transform is applied so probe + apply see the real layer.
S.videoYaw = 0;
S.videoPitch = 0;
probeVideoAngleMutable(vlayer);
applyVideoAngle(false); // undo the probe's test rotation
updateVideoAngleStats();
}
} catch (e) { NS.recordError('video layer', e); made.push('video:err'); }
// System hand models (WebXR Input Profiles 'generic-hand'), rendered into
@@ -670,6 +681,7 @@
pollGamepads(frame);
updatePanelPose(frame, dt);
updateGrabFrame(frame);
updateViewDrag(frame);
NS.runFrameHooks(frame, time, session);
}
session.requestAnimationFrame(loop);
@@ -701,8 +713,15 @@
upperVerticalAngle: Math.PI / 2,
lowerVerticalAngle: -Math.PI / 2
};
// ADDENDUM 8: the transform is the picture's aim. radius stays 0 (content at
// infinity) so this is a pure rotation about the eye point. The transform is
// what makes a rebuild-based runtime possible, and what makes the very first
// frame already point where the user left it (a mid-session rebuild).
var t = videoAngleTransform();
if (t) opts.transform = t;
var layer = new XRMediaBinding(session).createEquirectLayer(video, opts);
log('video layer', JSON.stringify({ layout: opts.layout, central: opts.centralHorizontalAngle }));
log('video layer', JSON.stringify({ layout: opts.layout, central: opts.centralHorizontalAngle,
yaw: Math.round(S.videoYaw * 180 / Math.PI), pitch: Math.round(S.videoPitch * 180 / Math.PI) }));
return layer;
}
@@ -724,6 +743,13 @@
S.lastFrame = null;
grab = null;
S.stats.grab = 'reset';
viewDrag = null;
viewTaps = [];
S.videoLayer = null;
S.videoAngleLive = false;
S.videoYaw = 0;
S.videoPitch = 0;
if (NS.Controls && NS.Controls.hold) { try { NS.Controls.hold(false); } catch (e) {} }
S.ctlLock = false;
panelPose = null;
pointer = { visible: false, beam: false, source: 'none', origin: { x: 0, y: 0, z: 0 }, dir: { x: 0, y: 0, z: -1 }, hit: null };
@@ -973,6 +999,200 @@
}).join(' ') || '(none)';
}
/* ================================================================== *
* ADDENDUM 8 — panning the video window
*
* The video layer is an equirect sphere with radius 0, i.e. the content sits at
* infinity. That has an important consequence: only the sphere's ORIENTATION
* decides which direction you see, and translating it does nothing. So panning
* the picture is purely `layer.transform`'s rotation, which rotates the content
* about the eye point — no parallax, no swimming when the head moves, and no
* decode/upload cost (8K HEVC DirectPlay is untouched).
*
* Only the video rotates. The control panel lives at a fixed pose in refSpace
* and the subtitles are head-locked in viewer space, so the UI stays in front of
* the user while the picture moves.
* ================================================================== */
var VIDEO_PITCH_LIMIT = Math.PI / 2 - 0.02; // "unlimited", minus a gimbal guard
function wrapYaw(a) {
while (a > Math.PI) a -= Math.PI * 2;
while (a < -Math.PI) a += Math.PI * 2;
return a;
}
function clampPitch(p) { return NS.clamp(p, -VIDEO_PITCH_LIMIT, VIDEO_PITCH_LIMIT); }
// q = qy(-yaw) * qx(pitch): pitch about the content's own X first, then yaw about
// the world Y — the standard look-around composition. The yaw sign is flipped so
// the STATE reads the way the user thinks about it: +yaw = the picture has been
// moved to the RIGHT, +pitch = UP (so a scene that used to need a physical
// head-down is brought up to eye level).
//
// The window's aim direction (the layer's local -Z) then works out to
// (cos p · sin yaw, sin p, -cos p · cos yaw)
// and the drag adds the ray's own yaw/pitch deltas directly, so the content point
// under the cursor stays under the cursor.
function videoQuat(yaw, pitch) {
var qy = { x: 0, y: Math.sin(-yaw / 2), z: 0, w: Math.cos(-yaw / 2) };
var qx = { x: Math.sin(pitch / 2), y: 0, z: 0, w: Math.cos(pitch / 2) };
return qMul(qy, qx);
}
function videoOrigin() { return { x: 0, y: 0, z: 0 }; }
function videoAngleTransform() {
if (typeof XRRigidTransform === 'undefined') return undefined;
return new XRRigidTransform(videoOrigin(), videoQuat(S.videoYaw, S.videoPitch));
}
function updateVideoAngleStats() {
S.stats.videoYaw = Math.round(S.videoYaw * 180 / Math.PI);
S.stats.videoPitch = Math.round(S.videoPitch * 180 / Math.PI);
S.stats.videoAngleLive = !!S.videoAngleLive;
}
// XREquirectLayer.transform is settable in the spec, but a runtime is free to
// ignore the write, so probe it once: write a known rotation and read it back.
function probeVideoAngleMutable(layer) {
S.videoAngleLive = false;
if (!layer || typeof XRRigidTransform === 'undefined') return;
try {
var q = videoQuat(0.05, 0.05);
layer.transform = new XRRigidTransform(videoOrigin(), q);
var o = layer.transform && layer.transform.orientation;
S.videoAngleLive = !!(o && Math.abs(o.y - q.y) < 1e-4 && Math.abs(o.w - q.w) < 1e-4);
} catch (e) {
S.videoAngleLive = false;
}
S.stats.videoRotMode = S.videoAngleLive ? 'live' : 'rebuild';
if (!S.videoAngleLive) {
log('video layer transform is read-only -> rebuilding the layer on changes');
}
}
// Push the current angles into the layer. On a live runtime this is just a
// transform write. On a runtime that ignores the write we must REBUILD the media
// layer — which is far too expensive to do once per drag frame, so rebuilds
// happen only when `force` is set (creation, release, reset).
function applyVideoAngle(force) {
var layer = S.videoLayer;
if (!layer || !S.session) return;
if (S.videoAngleLive) {
try {
layer.transform = videoAngleTransform();
if ('needsRedraw' in layer) layer.needsRedraw = true;
} catch (e) {
S.videoAngleLive = false;
S.stats.videoRotMode = 'rebuild';
}
}
if (!S.videoAngleLive && force) rebuildVideoLayer();
updateVideoAngleStats();
}
function rebuildVideoLayer() {
var session = S.session;
if (!session || !S.video) return;
try {
var fresh = createVideoLayer(session);
if (!fresh) return;
var old = S.videoLayer;
S.videoLayer = fresh;
NS.addLayer(fresh);
if (old) {
NS.removeLayer(old);
try { if (typeof old.destroy === 'function') old.destroy(); } catch (e) {}
}
S.stats.videoRebuilds = (S.stats.videoRebuilds || 0) + 1;
} catch (e) {
NS.recordError('videoRebuild', e);
}
}
function resetVideoAngle(why) {
S.videoYaw = 0;
S.videoPitch = 0;
if (S.session) { try { applyVideoAngle(true); } catch (e) {} }
S.stats.videoReset = why || 'reset';
updateVideoAngleStats();
}
/* ---------------- drag the picture (ADDENDUM 8) ----------------
* The pointer ray's own direction is the drag handle: rotating the aiming ray by
* δ rotates the picture by the same δ, so the content follows the hand 1:1 with
* no range limit (a panel-space cursor would run off the panel's edge).
* -------------------------------------------------------------- */
var viewDrag = null;
var VIEW_TAP_MS = 1600; // three taps inside this window reset the view
var viewTaps = [];
var lastPressSrc = null; // source / frame of the last accepted press, for
var lastPressFrame = null; // hold-style actions (the picture drag)
function rayAngles(ray) {
if (!ray || !ray.dir) return null;
var d = ray.dir;
var len = Math.sqrt(d.x * d.x + d.y * d.y + d.z * d.z) || 1;
var dx = d.x / len, dy = d.y / len, dz = d.z / len;
return {
yaw: Math.atan2(dx, -dz),
pitch: Math.asin(NS.clamp(dy, -1, 1))
};
}
function beginViewDrag(src, frame) {
var ray = rayOf(frame, src);
var a = rayAngles(ray);
if (!a) { S.stats.viewDrag = 'no ray'; return; }
viewDrag = { src: src, a0: a, yaw0: S.videoYaw, pitch0: S.videoPitch, moved: false };
S.stats.viewDrag = 'started';
if (NS.Controls && NS.Controls.hold) { try { NS.Controls.hold(true); } catch (e) {} }
}
function updateViewDrag(frame) {
if (!viewDrag || !frame) return;
var ray = rayOf(frame, viewDrag.src);
if (!ray) return;
var a = rayAngles(ray);
if (!a) return;
var dyaw = wrapYaw(a.yaw - viewDrag.a0.yaw);
var dpitch = a.pitch - viewDrag.a0.pitch;
var ny = wrapYaw(viewDrag.yaw0 + dyaw);
var np = clampPitch(viewDrag.pitch0 + dpitch);
if (Math.abs(ny - S.videoYaw) < 1e-4 && Math.abs(np - S.videoPitch) < 1e-4) return;
viewDrag.moved = true;
S.videoYaw = ny;
S.videoPitch = np;
applyVideoAngle(false);
}
// Release. A press that never moved counts as a TAP: three taps inside
// VIEW_TAP_MS reset the view (the owner's "连点 3 次复位"). Counting taps on
// RELEASE rather than on press means a real drag cannot be mistaken for one.
function endViewDrag() {
if (!viewDrag) return;
var moved = viewDrag.moved;
viewDrag = null;
if (NS.Controls && NS.Controls.hold) { try { NS.Controls.hold(false); } catch (e) {} }
if (!S.videoAngleLive) applyVideoAngle(true); // pay the rebuild once, on release
if (moved) {
viewTaps = [];
S.stats.viewDrag = 'ended (moved)';
return;
}
var now = performance.now();
viewTaps = viewTaps.filter(function (t) { return now - t < VIEW_TAP_MS; });
viewTaps.push(now);
S.stats.viewTaps = viewTaps.length;
if (viewTaps.length >= 3) {
viewTaps = [];
resetVideoAngle('triple-tap');
S.stats.viewDrag = 'reset (triple tap)';
return;
}
S.stats.viewDrag = 'tap ' + viewTaps.length + '/3';
}
/* ================================================================== *
* Fixed panel pose
*
@@ -1207,6 +1427,10 @@
if (NS.Controls && NS.Controls.show) { try { NS.Controls.show(); } catch (e) {} } // extend auto-hide
pressUntil = now + 300;
S.stats.pressAt = pointer.hit.nx.toFixed(2) + ',' + pointer.hit.ny.toFixed(2);
// Remembered for hold-style actions (the ADDENDUM 8 picture drag), which need
// the input source and frame that started them.
lastPressSrc = src;
lastPressFrame = frame;
try { NS.Controls.handleSelect(src, frame); }
catch (e) { NS.recordError('handleSelect', e); }
return;
@@ -1224,6 +1448,7 @@
function pointerUp() {
endGrab();
endViewDrag();
}
function pollGamepads(frame) {
@@ -1307,6 +1532,10 @@
// The padlock and the subtitle actions do not need a <video>.
switch (action.type) {
case 'lock': toggleControlsLock(); break;
case 'view_drag':
if (lastPressSrc && lastPressFrame) beginViewDrag(lastPressSrc, lastPressFrame);
else S.stats.viewDrag = 'no press context';
break;
case 'sub_up': setSubPlacementLive({ y: S.subPlacement.y + 0.04 }); break;
case 'sub_down': setSubPlacementLive({ y: S.subPlacement.y - 0.04 }); break;
case 'sub_near': setSubPlacementLive({ z: S.subPlacement.z + 0.15 }); break;