hud-6: pan the 180° video window about the eye; padlock as a panel button
ADDENDUM 8 — panning the video window (yaw/pitch about the eye)
The 180° window was hard-wired dead ahead, so scenes staged below the horizon
could only be watched by physically lowering the head. The video layer is an
XREquirectLayer with radius 0, i.e. the content sits at infinity: only its
ORIENTATION is visible and translating it does nothing. So panning is purely
layer.transform's rotation, which rotates the content about the eye point — no
parallax, no swim when the head moves, and no decode/upload cost (8K HEVC
DirectPlay untouched). Only the video rotates; the panel (refSpace, fixed pose)
and the subtitles (viewer space, head-locked) stay in front of the user.
state.videoYaw (+ = picture moved RIGHT)
state.videoPitch (+ = picture moved UP)
q = qy(-yaw) * qx(pitch) -> aim = (cos p·sin yaw, sin p, -cos p·cos yaw)
Interaction is one affordance in the panel, above the progress bar (the owner's
requested placement): a drag strip. Pinch and drag -> the picture follows the
hand 1:1; three taps within 1.6 s reset it (taps counted on RELEASE, so a drag
cannot be mistaken for one); the live angles are printed on the strip. The drag
is driven by the RAY's own direction delta rather than a cursor position on the
panel: a panel-space cursor leaves the panel after a few degrees, while ray
rotation has no range limit and keeps the content point under the cursor.
Controls.hold() freezes the panel auto-hide for the drag's duration.
Pitch is limited only by a gimbal guard (±(π/2 - 0.02)) per "不限"; yaw wraps to
[-π, π]. Both reset to 0 on every VR session — not persisted.
Runtime risk: XREquirectLayer.transform is settable per spec but a runtime may
ignore the write, so it is probed once per session (write a known rotation, read
it back) -> stats.videoRotMode = 'live' (transform write per frame) or
'rebuild' (recreate the media layer, but only on creation/release/reset, never
per frame). THIS PROBE NEEDS A REAL XR SESSION — it is the one thing not
verifiable offline.
ADDENDUM 7b (previous pass, in this commit's tree) — the padlock
The padlock had been given its own quad layer parked 0.45 m to the right of the
panel. On device it could not be selected while the panel was shown but could
when it was hidden. Aiming is the binding constraint: with hands the only
pointing device is a ray, and the cursor's acceptance window reaches the panel's
own right edge, so the badge was outside every reachable ray; its hit test also
used the panel's layer geometry, which is why the behaviour flipped with panel
visibility. It is now an ordinary panel button (media row, fifth slot) using the
same hitTest/handleSelect path as every other button, and "locked" is a third
panel mode (0.26 × 0.10 m) that draws and hit-tests ONLY the padlock. Locking
swaps the panel in place and lets it auto-hide; the next summon shows the padlock
alone; unlocking restores the full panel and ignores presses for 500 ms.
handleSelect() no longer falls back to pressing the panel centre when no ray hit
exists.
Panel: expanded grew 0.72 × 0.21 m -> 0.72 × 0.26 m (canvas 1440 × 520) for the
drag strip; every row re-proportioned.
Verified offline against the real bundle: quaternion/drag maths (aim direction
for ±yaw/±pitch, unit norm, azimuth/elevation recovery, yaw wrap, pitch clamp,
"aimed point ends up under the cursor"), panel hit bands on the new canvas
(strip 0..0.20, inert readout 0.20..0.30, seek 0.30..0.52, media 0.52..0.74,
subtitle 0.84..1.0, no dead column), the strip's drawn readout, and hold().
build.py --check passes; deployed as mvp-2e32e6 and confirmed live on the Quest
(hud-6, videoYaw/videoPitch 0, errors empty).
This commit is contained in:
+117
@@ -732,3 +732,120 @@ Controls.PANEL_M / PANEL_PX // now include .locked
|
||||
|
||||
Stats: `stats.ctlMode` (`expanded` | `locked`), `stats.ctlLock`,
|
||||
`stats.pressAt` (`SUMMON (locked panel)`, `LOCKED`/`UNLOCKED (armed 500ms)`, …).
|
||||
|
||||
---
|
||||
|
||||
# ADDENDUM 8 — panning the video window (yaw / pitch about the eye)
|
||||
|
||||
## Why
|
||||
|
||||
> 我在观看过程中,有时想调整180度视频的播放位置,让其更适合我的环境,我希望画面可以不是
|
||||
> 固定在我的正前方180度,我希望能调整画面内容在整个vr空间中以我眼睛为旋转点进行上下左右
|
||||
> 的水平移动(为了解决一些场景需要物理低头才能看到的问题,我不想真的低头去看)
|
||||
|
||||
The 180° window was hard-wired dead ahead. Scenes staged below the horizon could only
|
||||
be watched by physically lowering the head.
|
||||
|
||||
## Model
|
||||
|
||||
The video layer is an `XREquirectLayer` with `radius` 0 — the content sits **at
|
||||
infinity**. Two consequences drive the whole design:
|
||||
|
||||
- Only the layer's **orientation** decides which direction is visible. Translating it
|
||||
does nothing, and there is no parallax, so the picture does not swim when the head
|
||||
moves.
|
||||
- So panning the picture is *purely* `layer.transform`'s rotation, which rotates the
|
||||
content **about the eye point** — exactly what was asked for. No decode, no texture
|
||||
upload, no extra GPU work: 8K HEVC DirectPlay is untouched.
|
||||
|
||||
Only the video rotates. The panel lives at a fixed pose in `refSpace` and the
|
||||
subtitles are head-locked in `viewerSpace`, so **the UI stays in front of the user**
|
||||
while the picture moves.
|
||||
|
||||
```js
|
||||
state.videoYaw // radians, + = the picture has been moved RIGHT
|
||||
state.videoPitch // radians, + = the picture has been moved UP
|
||||
q = qy(-yaw) * qx(pitch) // pitch in the content frame, then yaw about world Y
|
||||
layer.transform = XRRigidTransform({0,0,0}, q)
|
||||
```
|
||||
|
||||
Because of that composition the window's aim direction is exactly
|
||||
`(cos p·sin yaw, sin p, −cos p·cos yaw)`, so yaw is a pure azimuth and pitch a pure
|
||||
elevation — no gimbal weirdness in the useful range.
|
||||
|
||||
Limits: pitch ±(π/2 − 0.02) — i.e. effectively unlimited, per the owner's
|
||||
"看用户拖动幅度来定"; the guard only avoids the degenerate pole. Yaw wraps to
|
||||
[−π, π]. **Not persisted**: every VR session starts dead ahead.
|
||||
|
||||
## Interaction — one affordance, in the panel
|
||||
|
||||
A **drag strip above the progress bar** (the owner's requested placement), part of the
|
||||
panel canvas like every other control:
|
||||
|
||||
```
|
||||
┌─ 画面 → the drag strip (panel y 0.00–0.20) ────────────────────────┐
|
||||
│ ⠿ 拖动调整画面 偏航 +12° 俯仰 -8° │
|
||||
│ 连点 3 次复位 0° / 0° = 正前方 │
|
||||
└──────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
- **Pinch the strip and drag** → the picture follows the hand 1:1.
|
||||
- **Three taps inside 1.6 s** → reset to dead ahead. Taps are counted on RELEASE, so a
|
||||
real drag can never be mistaken for one.
|
||||
- The right side always shows the live angles, so the state is readable even when you
|
||||
are not moving anything.
|
||||
|
||||
The drag is driven by the **pointer ray's own direction**, not by a position on the
|
||||
panel:
|
||||
|
||||
```
|
||||
yaw' = wrap(yaw0 + (rayYaw - rayYaw0))
|
||||
pitch' = clamp(pitch0 + (rayPitch - rayPitch0)) rayYaw = atan2(dx, -dz)
|
||||
rayPitch = asin(dy)
|
||||
```
|
||||
|
||||
A panel-space cursor would run off the panel's edge after a few degrees of movement;
|
||||
ray rotation has no range limit. It also makes the content point under the cursor stay
|
||||
under the cursor, which is what "grab the picture" should feel like.
|
||||
|
||||
`Controls.hold(true/false)` freezes the panel auto-hide for the duration of the drag —
|
||||
otherwise the panel would vanish mid-drag because the timer `show()` armed keeps
|
||||
running.
|
||||
|
||||
## Runtime risk and how it is handled
|
||||
|
||||
`XRCompositionLayer.transform` / `XREquirectLayer.transform` are settable in the spec,
|
||||
but a runtime may ignore the write. Since the whole interaction depends on it, the
|
||||
module probes once per session:
|
||||
|
||||
```js
|
||||
probeVideoAngleMutable(layer) // write a known rotation, read it back
|
||||
-> stats.videoRotMode = 'live' | 'rebuild'
|
||||
```
|
||||
|
||||
- `live` — a transform write per frame while dragging. Free.
|
||||
- `rebuild` — the media layer is recreated. Far too expensive per frame, so rebuilds
|
||||
happen only on **creation / release / reset**; during a drag the state updates and
|
||||
the picture catches up when the finger comes up. `stats.videoRebuilds` counts them.
|
||||
|
||||
## 20-controls.js — changes
|
||||
|
||||
- The expanded panel grew from **0.72 × 0.21 m to 0.72 × 0.26 m** (canvas 1440 × 420 →
|
||||
1440 × 520) to make room for the strip; every row was re-proportioned.
|
||||
- New hit band `viewHitY0/Y1 = 0..0.20` returning `{type:'view_drag'}`, spanning the
|
||||
panel width. It does not overlap the seek band (0.30–0.52) or the button bands.
|
||||
- `view.view = { yawDeg, pitchDeg }` feeds the readout (`drawViewStrip`), and is part
|
||||
of the redraw signature.
|
||||
- New `hold(on)` for the auto-hide pause; `stats.ctlHold`.
|
||||
|
||||
## 90-main.js — changes
|
||||
|
||||
- `state.videoYaw/videoPitch` reset to 0 at session start and at session end.
|
||||
- `S.videoLayer` handle (was local), `applyVideoAngle(force)` / `rebuildVideoLayer()`,
|
||||
`resetVideoAngle()`, `videoQuat()`, `rayAngles()`, `beginViewDrag/updateViewDrag/
|
||||
endViewDrag`, and `viewTaps` for the triple-tap reset.
|
||||
- The frame loop calls `updateViewDrag(frame)`; `pointerUp()` calls `endViewDrag()`.
|
||||
- `handleControlAction('view_drag')` starts the drag using the press context
|
||||
(`lastPressSrc` / `lastPressFrame`) captured in `pointerDown`.
|
||||
- Stats: `videoYaw`, `videoPitch`, `videoAngleLive`, `videoRotMode`, `videoRebuilds`,
|
||||
`videoReset`, `viewDrag`, `viewTaps`.
|
||||
|
||||
Reference in New Issue
Block a user