diff --git a/HANDOFF.md b/HANDOFF.md index 83bacd3..6ec007d 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -1,8 +1,8 @@ # Jellyfin Web VR180 — 项目交接文档 > 给新对话用。读完这份就能接手,不需要重新调查环境。 -> 最后更新:已部署版本 `7616c9`(`NS.version = hud-5`;ADDENDUM 6 字幕深度/大小解耦 -> + ADDENDUM 7 唤出门限/进 VR 前面板精简 + ADDENDUM 7b 锁按钮与「锁定」面板模式) +> 最后更新:已部署版本 `2e32e6`(`NS.version = hud-6`;ADDENDUM 6 字幕深度/大小解耦 +> + 7 唤出门限/前面板精简 + 7b 锁按钮与「锁定」面板模式 + 8 画面拖动搬迁) > 已在 Quest 浏览器里实机验证模块加载 + 锁定数学(`subLockAngle/subAngleDeg`,`errors: []`); > **VR 会话内的观感还没看过**。 @@ -132,11 +132,11 @@ jelllyfin_vr_web_plugin/ ├── README.md ← 技术记录(含全部实测数据) ├── HANDOFF.md ← 本文档 ├── src/ -│ ├── 00-core.js (343 行) 命名空间、共享状态、GL 助手、层/帧钩子注册表、能力探测 +│ ├── 00-core.js (353 行) 命名空间、共享状态、GL 助手、层/帧钩子注册表、能力探测 │ ├── 10-subtitles.js (1022 行) 字幕:libass 画布 → 头锁定 quad 层(HUD)+ 视角尺寸锁定 -│ ├── 20-controls.js (1930 行) 控制面板:进度条/按钮/光束/光标/锁按钮,canvas 2D → quad 层 +│ ├── 20-controls.js (2078 行) 控制面板:拖动条/进度条/按钮/光束/光标/锁,canvas 2D → quad 层 │ ├── 30-hands.js (569 行) **系统手部模型**:Three.js + 官方 generic-hand GLB → projection 层 -│ └── 90-main.js (1420 行) 引导、会话生命周期、rAF 代理、输入分发、设置面板、持久化 +│ └── 90-main.js (1641 行) 引导、会话生命周期、rAF 代理、输入分发、设置面板、持久化、画面搬迁 ├── build.py ← src/*.js 按文件名顺序拼接 → dist/jfvr.js;`--check` 只做语法检查 ├── deploy.py ← 构建 + 上传进容器 + 注入 script 标签;`rollback` 还原 ├── quest.sh ← 一键重连 Quest 的 ADB + CDP 转发 @@ -285,6 +285,7 @@ WebXR 规范:需要 *transient activation*。前面不能有 `await`,否则 | 引导线(十字双光束) | ✅ 任何时候都显示 | | 面板唤出门限 | ✅ 只有指示线指向/靠近面板时才唤出;唤出那一下不算点击;500ms 内忽略 | | 锁按钮(面板不启动) | ✅ 面板媒体行的第五个按钮:上锁后切到「只显示锁」的最小面板,并同样自动隐藏;每次进 VR 重置为未锁 | +| 画面搬迁(180°窗口) | ✅ 面板进度条上方的拖动条:按住拖动,画面绕眼点左右上下转;连点 3 次复位;每次进 VR 回正前方 | | 按压反馈 | ✅ 圆环放大 1.9× + 蓝填充 + 变白,300ms | | 字幕抓拖 | ✅ 注视选中 + 手移动拖(与视线解耦,避免转头就飘) | | 字幕深度/大小解耦 | ✅ 默认「锁定视角尺寸」:拉近/拉远只改深度,视角大小与画面位置不变 | @@ -410,7 +411,9 @@ handsReady / handsErr / handsSources / handsFrames subMode / subCanvasSize / subCrop / subTex / subOpacity / subPlacement subLockAngle / subAngleDeg / subTextDeg / subYOverZ ← ADDENDUM 6 ctlSubsOn / ctlSubDist / ctlSubAngle / ctlSubLock ← 控制条实际画出的值 -ctlLock / ctlMode / pressAt / lastActivate ← ADDENDUM 7/7b 锁与唤出判定 +ctlLock / ctlMode / ctlHold / pressAt / lastActivate ← ADDENDUM 7/7b 锁、面板保持与唤出判定 +videoYaw / videoPitch / videoAngleLive / videoRotMode ← ADDENDUM 8 画面搬迁 +videoRebuilds / videoReset / viewDrag / viewTaps ctlHidden / ctlVisible / ctlBeamOn / ctlDraws / ctlErr panelPose / grab / grabValue / saved errors[] 最近错误 @@ -452,3 +455,22 @@ errors[] 最近错误 [ ] 确认 fps 仍在 ~110+,targetFrameRate 120 [ ] 确认面板/字幕/手模型均无异常 ``` + +### 6.14 画面搬迁为什么是"纯旋转"(ADDENDUM 8) + +等距柱状图层 `radius = 0`,语义是**内容在无穷远**。这决定了三件事: + +- 平移这个球**看不出任何变化**,所以搬画面只能是 `layer.transform` 的**旋转** +- 远景没有视差,所以转头/走动时画面**不会游动** +- 旋转中心就是**球心=眼点**,正好是"以我眼睛为旋转点" + +而且只转视频层:面板在 `refSpace` 固定姿态、字幕在 viewer 空间头锁定,所以**UI 不会跟着 +画面跑**。代价为零(不解码、不重传纹理、不动 8K HEVC DirectPlay)。 + +拖动用**射线自身的方向差**而不是面板上的坐标:面板坐标挪几度就出界了,射线旋转无行程限制, +而且"光标下的那个点始终在光标下",才像"抓住画面拖"。 + +**唯一的未知**:`XREquirectLayer.transform` 规范上可写,但运行时可能忽略。所以每会话探测一次 +(写一个已知旋转再读回):`live` = 每帧写 transform(零成本);`rebuild` = 只能重建媒体层, +那就只在**创建/松手/复位**时重建,拖动过程中画面在松手后追上。见 `stats.videoRotMode` / +`videoRebuilds`。**这条必须真机验证,离线测不出来。** diff --git a/INTERFACES.md b/INTERFACES.md index 5ff7a8a..c0ffba4 100644 --- a/INTERFACES.md +++ b/INTERFACES.md @@ -732,3 +732,120 @@ Controls.PANEL_M / PANEL_PX // now include .locked Stats: `stats.ctlMode` (`expanded` | `locked`), `stats.ctlLock`, `stats.pressAt` (`SUMMON (locked panel)`, `LOCKED`/`UNLOCKED (armed 500ms)`, …). + +--- + +# ADDENDUM 8 — panning the video window (yaw / pitch about the eye) + +## Why + +> 我在观看过程中,有时想调整180度视频的播放位置,让其更适合我的环境,我希望画面可以不是 +> 固定在我的正前方180度,我希望能调整画面内容在整个vr空间中以我眼睛为旋转点进行上下左右 +> 的水平移动(为了解决一些场景需要物理低头才能看到的问题,我不想真的低头去看) + +The 180° window was hard-wired dead ahead. Scenes staged below the horizon could only +be watched by physically lowering the head. + +## Model + +The video layer is an `XREquirectLayer` with `radius` 0 — the content sits **at +infinity**. Two consequences drive the whole design: + +- Only the layer's **orientation** decides which direction is visible. Translating it + does nothing, and there is no parallax, so the picture does not swim when the head + moves. +- So panning the picture is *purely* `layer.transform`'s rotation, which rotates the + content **about the eye point** — exactly what was asked for. No decode, no texture + upload, no extra GPU work: 8K HEVC DirectPlay is untouched. + +Only the video rotates. The panel lives at a fixed pose in `refSpace` and the +subtitles are head-locked in `viewerSpace`, so **the UI stays in front of the user** +while the picture moves. + +```js +state.videoYaw // radians, + = the picture has been moved RIGHT +state.videoPitch // radians, + = the picture has been moved UP +q = qy(-yaw) * qx(pitch) // pitch in the content frame, then yaw about world Y +layer.transform = XRRigidTransform({0,0,0}, q) +``` + +Because of that composition the window's aim direction is exactly +`(cos p·sin yaw, sin p, −cos p·cos yaw)`, so yaw is a pure azimuth and pitch a pure +elevation — no gimbal weirdness in the useful range. + +Limits: pitch ±(π/2 − 0.02) — i.e. effectively unlimited, per the owner's +"看用户拖动幅度来定"; the guard only avoids the degenerate pole. Yaw wraps to +[−π, π]. **Not persisted**: every VR session starts dead ahead. + +## Interaction — one affordance, in the panel + +A **drag strip above the progress bar** (the owner's requested placement), part of the +panel canvas like every other control: + +``` +┌─ 画面 → the drag strip (panel y 0.00–0.20) ────────────────────────┐ +│ ⠿ 拖动调整画面 偏航 +12° 俯仰 -8° │ +│ 连点 3 次复位 0° / 0° = 正前方 │ +└──────────────────────────────────────────────────────────────────┘ +``` + +- **Pinch the strip and drag** → the picture follows the hand 1:1. +- **Three taps inside 1.6 s** → reset to dead ahead. Taps are counted on RELEASE, so a + real drag can never be mistaken for one. +- The right side always shows the live angles, so the state is readable even when you + are not moving anything. + +The drag is driven by the **pointer ray's own direction**, not by a position on the +panel: + +``` +yaw' = wrap(yaw0 + (rayYaw - rayYaw0)) +pitch' = clamp(pitch0 + (rayPitch - rayPitch0)) rayYaw = atan2(dx, -dz) + rayPitch = asin(dy) +``` + +A panel-space cursor would run off the panel's edge after a few degrees of movement; +ray rotation has no range limit. It also makes the content point under the cursor stay +under the cursor, which is what "grab the picture" should feel like. + +`Controls.hold(true/false)` freezes the panel auto-hide for the duration of the drag — +otherwise the panel would vanish mid-drag because the timer `show()` armed keeps +running. + +## Runtime risk and how it is handled + +`XRCompositionLayer.transform` / `XREquirectLayer.transform` are settable in the spec, +but a runtime may ignore the write. Since the whole interaction depends on it, the +module probes once per session: + +```js +probeVideoAngleMutable(layer) // write a known rotation, read it back + -> stats.videoRotMode = 'live' | 'rebuild' +``` + +- `live` — a transform write per frame while dragging. Free. +- `rebuild` — the media layer is recreated. Far too expensive per frame, so rebuilds + happen only on **creation / release / reset**; during a drag the state updates and + the picture catches up when the finger comes up. `stats.videoRebuilds` counts them. + +## 20-controls.js — changes + +- The expanded panel grew from **0.72 × 0.21 m to 0.72 × 0.26 m** (canvas 1440 × 420 → + 1440 × 520) to make room for the strip; every row was re-proportioned. +- New hit band `viewHitY0/Y1 = 0..0.20` returning `{type:'view_drag'}`, spanning the + panel width. It does not overlap the seek band (0.30–0.52) or the button bands. +- `view.view = { yawDeg, pitchDeg }` feeds the readout (`drawViewStrip`), and is part + of the redraw signature. +- New `hold(on)` for the auto-hide pause; `stats.ctlHold`. + +## 90-main.js — changes + +- `state.videoYaw/videoPitch` reset to 0 at session start and at session end. +- `S.videoLayer` handle (was local), `applyVideoAngle(force)` / `rebuildVideoLayer()`, + `resetVideoAngle()`, `videoQuat()`, `rayAngles()`, `beginViewDrag/updateViewDrag/ + endViewDrag`, and `viewTaps` for the triple-tap reset. +- The frame loop calls `updateViewDrag(frame)`; `pointerUp()` calls `endViewDrag()`. +- `handleControlAction('view_drag')` starts the drag using the press context + (`lastPressSrc` / `lastPressFrame`) captured in `pointerDown`. +- Stats: `videoYaw`, `videoPitch`, `videoAngleLive`, `videoRotMode`, `videoRebuilds`, + `videoReset`, `viewDrag`, `viewTaps`. diff --git a/README.md b/README.md index 25fa37f..dc1b6da 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ VR180 / VR360 沉浸播放,不用再手动切全屏 + 选投影。 ## 1. 当前部署状态 -已注入到 `192.168.123.199` 上的 Jellyfin 容器 `Jellyfin`(当前版本 `mvp-7616c9`,`NS.version = hud-5`,含 ADDENDUM 6 + 7 + 7b): +已注入到 `192.168.123.199` 上的 Jellyfin 容器 `Jellyfin`(当前版本 `mvp-2e32e6`,`NS.version = hud-6`,含 ADDENDUM 6 + 7 + 7b + 8): | 项 | 值 | |---|---| @@ -811,3 +811,126 @@ Live Quest 3 over CDP after deploy: `__JFVR.version = 'hud-5'`, `stats.ctlMode = The *feel* — how easily the padlock is aimed with the hand ray, and whether 0.5 s is the right arm delay — can only be judged in the headset. + +--- + +# ADDENDUM 8 — panning the 180° video window about the eye + +## The requirement + +> 我希望画面可以不是固定在我的正前方180度,我希望能调整画面内容在整个vr空间中以我眼睛为 +> 旋转点进行上下左右的水平移动(为了解决一些场景需要物理低头才能看到的问题,我不想真的 +> 低头去看) + +The window was hard-wired dead ahead, so anything staged below the horizon cost a +physical head-down. + +## Why a pure rotation is the right mechanism + +The video layer is an `XREquirectLayer` with `radius` 0, i.e. the content is **at +infinity**. That single fact decides everything: + +| Property of radius-0 equirect | Consequence for this feature | +| --- | --- | +| Only the orientation is visible; translation has no effect | Panning = rotating `layer.transform`, nothing else | +| No parallax | The picture does not swim when the head moves | +| It is a sphere "around everywhere" | The rotation is about the **eye point** — literally what was asked | +| No new texture / no re-upload | 8K HEVC DirectPlay, 120 Hz and the ~112 fps budget are untouched | + +Only the video layer rotates. The control panel is at a fixed pose in `refSpace`, the +subtitles are head-locked in `viewerSpace`, so **the UI does not move with the +picture**. + +Rejected alternatives: + +| Alternative | Why not | +| --- | --- | +| `refSpace.getOffsetReferenceSpace()` | Rotates the panel, the hands and every ray too — the UI would leave the user's view | +| Moving the layer's position | An infinite-radius sphere shows no visible change (and would add parallax/swim) | +| Changing `centralHorizontalAngle` | That changes FOV/distortion, not where the window points | + +## Controls + +`state.videoYaw` (+ = picture right) and `state.videoPitch` (+ = picture up): + +``` +q = qy(-yaw) * qx(pitch) // pitch in the content frame, then yaw about world Y +aim = (cos p·sin yaw, sin p, -cos p·cos yaw) // the window's aim direction +``` + +The yaw sign is flipped so the two numbers read the way a user thinks about them, and +every value shown in the panel is a plain "moved this many degrees right/up". +Pitch is limited only by a gimbal guard (±(π/2 − 0.02)) per the owner's "不限,看用户拖动 +幅度来定"; yaw wraps to [−π, π]. **Reset to 0/0 on every VR session** — not persisted. + +Interaction is a single affordance that lives in the panel, above the progress bar +(the owner's requested placement): + +``` +⠿ 拖动调整画面 偏航 +12° 俯仰 -8° + 连点 3 次复位 0° / 0° = 正前方 +``` + +- Pinch it and drag — the picture follows the hand 1:1. +- Three taps within 1.6 s reset to dead ahead; taps are counted on **release**, so a + drag cannot be mistaken for one. +- The live angles are printed on the right at all times. + +The drag uses the **ray's direction**, not a cursor position on the panel: + +``` +yaw' = wrap(yaw0 + (rayYaw - rayYaw0)), pitch' = clamp(pitch0 + (rayPitch - rayPitch0)) +rayYaw = atan2(dx, -dz), rayPitch = asin(dy) +``` + +A panel-space cursor leaves the panel after a few degrees; ray rotation has no range +limit, and the content point under the cursor stays under the cursor — which is what +"grab the picture" has to feel like. `Controls.hold(true)` freezes the panel auto-hide +for the duration, otherwise the panel would auto-hide mid-drag. + +## The one runtime unknown + +`XREquirectLayer.transform` is settable per spec but a runtime may ignore the write. It +is probed once per session (write a known rotation, read it back): + +``` +stats.videoRotMode = 'live' -> a transform write per frame, free + = 'rebuild' -> recreate the media layer, but only on + creation / release / reset (never per frame) +``` + +`stats.videoRebuilds` counts rebuilds. **This probe needs a real XR session, so it is +the one thing that cannot be checked offline.** + +## Offline verification (real bundle, no headset) + +Quaternion / drag maths, extracted from `src/90-main.js` and driven directly: + +``` +yaw=0 pitch=0 -> aim (0, 0, -1) dead ahead +yaw=+20 -> aim x = +0.342 picture moved RIGHT +yaw=-20 -> aim x = -0.342 picture moved LEFT +pitch=+20 -> aim y = +0.342 picture moved UP +pitch=-20 -> aim y = -0.342 picture moved DOWN +yaw=30 pitch=40 -> unit direction, azimuth 30.000° and elevation 40.000° recovered +wrapYaw(±3π/2) -> ∓π/2 wraps into [-π, π] +clampPitch(±9) -> ±(π/2-0.02) gimbal guard holds +drag -> the aimed point ends up under the cursor (yaw and pitch) +``` + +Panel geometry via `Controls.hitTest` on the 1440×520 canvas: + +``` +y 0.05 / 0.18 -> view_drag (the whole strip) +y 0.25 -> null (time readout: intentionally inert) +y 0.40 -> seek +y 0.60 -> media row (back10|playpause|fwd10|subs|lock) +y 0.90 -> subtitle row (9 actions) +x sweep 0.04..0.96 at y 0.05 -> no dead column +``` + +`Controls.draw` renders `拖动调整画面` / `连点 3 次复位` / `偏航 +12° 俯仰 -8°` / +`0° / 0° = 正前方`, and `hold(true|false)` is reflected in `stats.ctlHold`. + +Live Quest 3 over CDP after deploy: `__JFVR.version = 'hud-6'`, +`state.videoYaw = state.videoPitch = 0`, `errors: []`. diff --git a/dist/jfvr.js b/dist/jfvr.js index 18fc625..f4b6dab 100644 --- a/dist/jfvr.js +++ b/dist/jfvr.js @@ -19,7 +19,7 @@ var NS = window.__JFVR = window.__JFVR || {}; - NS.version = 'hud-5'; // hud-4 = ADDENDUM 7; hud-5 = ADDENDUM 7b (padlock in the panel + locked mode) + NS.version = 'hud-6'; // hud-6 = ADDENDUM 8 (drag the picture window: yaw/pitch about the eye) NS.__core = true; /* ------------------------------------------------------------------ * @@ -46,6 +46,16 @@ projection: '180', // '180' | '360' stereo: 'sbs', // 'mono' | 'sbs' | 'ou' + // ADDENDUM 8 — where the 180/360 window points. The video layer is an + // equirect sphere with radius 0 (content "at infinity"), so rotating its + // transform rotates the CONTENT about the eye point without any parallax: + // exactly "pan the picture so I do not have to crane my neck". Only the video + // rotates — the panel (refSpace, fixed pose) and the subtitles (viewer space, + // head-locked) stay in front of the user. Radians. NOT persisted: the owner + // wants a fresh forward-facing view every time VR starts. + videoYaw: 0, // + = the picture has been moved to the RIGHT + videoPitch: 0, // + = the picture has been moved UP + subtitles: true, ctlLock: false, // ADDENDUM 7: control-bar lock badge (per VR session) subMode: 'hud', // 'hud' | 'sphere' | 'off' @@ -1438,7 +1448,7 @@ // 0.60 m x 0.05 m collapsed. var PANEL_M = { collapsed: { w: 0.60, h: 0.05 }, - expanded: { w: 0.72, h: 0.21 }, + expanded: { w: 0.72, h: 0.26 }, // ADDENDUM 7b: the minimal panel shown while the controls are LOCKED. It // carries only the padlock, so nothing adjustable is reachable. locked: { w: 0.26, h: 0.10 } @@ -1448,14 +1458,14 @@ // ratio of (w, h), so pass these (or any matching ratio). var PANEL_PX = { collapsed: { w: 1200, h: 100 }, - expanded: { w: 1440, h: 420 }, + expanded: { w: 1440, h: 520 }, locked: { w: 520, h: 200 } }; // Layer placement in viewer space (head-locked, centred, lower part of view). var PANEL_Y = -0.28; var PANEL_Z = 1.4; - // hitTest mode threshold: collapsed ratio 12.0, expanded ratio 3.43. + // hitTest mode threshold: collapsed ratio 12.0, expanded 2.77, locked 2.6. var COLLAPSED_RATIO = 8; // Pointer beam (ADDENDUM 2): a separate thin quad layer running from the @@ -1561,53 +1571,68 @@ trackHitY0: 0.0, // full strip height is a generous seek band trackHitY1: 1.0 }, - // Vertical bands of the 420 px (= 0.21 m) expanded panel, top to bottom: - // time readout, progress track, media buttons, subtitle caption, subtitle - // buttons. The three hit bands are contiguous and never overlap, so the seek - // band is untouched by the button rows (hitTest still tests every button - // first). Every offset is a fraction of the panel, so draw() and hitTest() - // scale with whatever w/h they are handed. + // Vertical bands of the 520 px (= 0.26 m) expanded panel, top to bottom: + // the ADDENDUM 8 drag strip, time readout, progress track, media buttons, + // subtitle caption, subtitle buttons. The hit bands are contiguous and never + // overlap, so the seek band is untouched by the button rows (hitTest still + // tests every button first). Every offset is a fraction of the panel, so + // draw() and hitTest() scale with whatever w/h they are handed. expanded: { - trackCy: 0.32, // 134 px - trackVisH: 0.04, // 17 px, ~8 mm of a 21 cm panel - trackHitY0: 0.20, // 84 px band 84..185 px, far larger than the bar - trackHitY1: 0.44, // 185 px - textBaseline: 0.165, // 69 px - textFont: 0.095, // 40 px - btnY: 0.58, // 244 px + // ADDENDUM 8: the picture-pan drag strip sits ABOVE the progress bar (the + // owner's requested place). It is the whole picture-pan affordance: pinch it + // and drag, and the video content rotates with your hand. Three taps on it + // within VIEW_TAP_MS reset the view to dead ahead. + viewStripX0: 0.02, + viewStripX1: 0.98, + viewStripY0: 0.03, + viewStripY1: 0.17, + viewHitY0: 0.0, // 0 px band 0..104 px + viewHitY1: 0.20, // 104 px + viewLabelFont: 0.040, + viewValueFont: 0.046, + viewHintFont: 0.030, + viewLabelDy: 0.075, // 39 px first line baseline + viewHintDy: 0.155, // 81 px second (hint) baseline + trackCy: 0.42, // 218 px + trackVisH: 0.030, // 16 px, ~8 mm of a 26 cm panel + trackHitY0: 0.30, // 156 px band 156..270 px, far larger than the bar + trackHitY1: 0.52, // 270 px + textBaseline: 0.275, // 143 px + textFont: 0.075, // 39 px + btnY: 0.62, // 322 px btnVisualW: 0.105, // 151 px - btnVisualH: 0.155, // 65 px - btnHitY0: 0.44, // 185 px band 185..302 px - btnHitY1: 0.72, // 302 px - // ADDENDUM 7b: five buttons now — the padlock joined the row at the far - // right (centres 0.18 / 0.36 / 0.54 / 0.72 / 0.90, pills 0.088..0.992). + btnVisualH: 0.125, // 65 px + btnHitY0: 0.52, // 270 px band 270..385 px + btnHitY1: 0.74, // 385 px + // Five buttons - the padlock joined the row at the far right (centres + // 0.18 / 0.36 / 0.54 / 0.72 / 0.90, pills 0.088..0.992). btnHitHalfW: 0.092, btnSpacing: 0.18, btnFirstCx: 0.18, // Icon over label inside each media button: the label says what the // button does, the icon keeps the quick visual cue. - btnIconDy: -0.032, // -13.4 px, icon centre above the button centre - btnIconSize: 0.080, // 34 px icon box - btnLabelFont: 0.050, // 21 px label - btnLabelDy: 0.050, // 21 px label baseline below the button centre + btnIconDy: -0.026, // -13.5 px, icon centre above the button centre + btnIconSize: 0.065, // 34 px icon box + btnLabelFont: 0.040, // 21 px label + btnLabelDy: 0.041, // 21 px label baseline below the button centre // ADDENDUM 4 subtitle row (ADDENDUM 6: nine buttons), tiling the full // width (centres 0.10 .. 0.90, pitch 0.10, symmetric). The hit pills are // 0.104 wide on a 0.10 pitch, so adjacent hit regions overlap by a hair // instead of leaving a dead zone between. - subLabelBaseline: 0.757, // 318 px caption baseline - subCaptionFont: 0.048, // 20 px caption (distance + size + opacity readout) - subBtnY: 0.90, // 378 px button centre + subLabelBaseline: 0.795, // 413 px caption baseline + subCaptionFont: 0.040, // 21 px caption (distance + size + opacity readout) + subBtnY: 0.925, // 481 px button centre subVisualW: 0.092, // 132 px - subVisualH: 0.145, // 61 px - subHitY0: 0.80, // 336 px band 336..420 px (below the caption) - subHitY1: 1.0, // 420 px + subVisualH: 0.120, // 62 px + subHitY0: 0.84, // 437 px band 437..520 px (below the caption) + subHitY1: 1.0, // 520 px subHitHalfW: 0.052, subSpacing: 0.10, subFirstCx: 0.10, - subIconDy: -0.024, // -10.1 px icon centre above the button centre - subIconSize: 0.058, // 24.4 px icon box - subLabelFont: 0.040, // 16.8 px label - subLabelDy: 0.044 // 18.5 px label baseline below the button centre + subIconDy: -0.020, // -10.4 px icon centre above the button centre + subIconSize: 0.046, // 24 px icon box + subLabelFont: 0.034, // 17.7 px label + subLabelDy: 0.036 // 18.7 px label baseline below the button centre }, // ADDENDUM 7b: the locked panel. ONE target — the padlock — centred, with a // label that says what pressing it does. Nothing else is drawn or hit-tested, @@ -1784,6 +1809,7 @@ ctx.fill(); if (mode === 'expanded') { + drawViewStrip(ctx, w, h, view, lay); drawExpanded(ctx, w, h, view, lay, dur, cur); } @@ -1833,6 +1859,79 @@ ctx.fill(); } + // ADDENDUM 8 — the picture-pan drag strip, ABOVE the progress bar. + // It is one self-contained affordance: the grip says "grab me", the values say + // where the picture currently points, and the hint says how to reset. Pure like + // the rest of draw(): no state, no timers. + function drawViewStrip(ctx, w, h, view, lay) { + var v = view.view || {}; + var yaw = num(v.yawDeg); + var pitch = num(v.pitchDeg); + if (yaw == null) yaw = 0; + if (pitch == null) pitch = 0; + + var x0 = num(lay.viewStripX0) * w; + var x1 = num(lay.viewStripX1) * w; + var y0 = num(lay.viewStripY0) * h; + var y1 = num(lay.viewStripY1) * h; + var sw = x1 - x0, sh = y1 - y0; + if (sw <= 0 || sh <= 0) return; + + var dragging = view.hover === 'view_drag'; + ctx.fillStyle = dragging ? 'rgba(78,161,255,0.26)' : 'rgba(255,255,255,0.10)'; + rr(ctx, x0, y0, sw, sh, sh * 0.30); + ctx.fill(); + ctx.strokeStyle = dragging ? ACCENT : 'rgba(255,255,255,0.26)'; + ctx.lineWidth = Math.max(1.5, sh * 0.035); + if (typeof ctx.setLineDash === 'function') ctx.setLineDash([sh * 0.22, sh * 0.16]); + rr(ctx, x0, y0, sw, sh, sh * 0.30); + ctx.stroke(); + if (typeof ctx.setLineDash === 'function') ctx.setLineDash([]); + + // Grip: a 2x3 dot pad, the universal "this is draggable" mark. + var gy = (y0 + y1) / 2; + var gx = x0 + sh * 0.72; + var dot = Math.max(1.5, sh * 0.075); + var dx = sh * 0.30, dy = sh * 0.20; + ctx.fillStyle = dragging ? '#ffffff' : 'rgba(255,255,255,0.75)'; + for (var r = -1; r <= 1; r++) { + for (var c = -1; c <= 1; c += 2) { + ctx.beginPath(); + ctx.arc(gx + c * dx * 0.5, gy + r * dy, dot, 0, Math.PI * 2); + ctx.fill(); + } + } + + var textX = gx + sh * 0.60; + ctx.textBaseline = 'alphabetic'; + var f1 = Math.max(10, Math.round(num(lay.viewLabelFont) * h)); + ctx.textAlign = 'left'; + ctx.fillStyle = '#ffffff'; + ctx.font = '600 ' + f1 + 'px ' + FONT; + ctx.fillText('拖动调整画面', textX, num(lay.viewLabelDy) * h); + ctx.font = Math.max(9, Math.round(num(lay.viewHintFont) * h)) + 'px ' + FONT; + ctx.fillStyle = 'rgba(255,255,255,0.70)'; + ctx.fillText('连点 3 次复位', textX, num(lay.viewHintDy) * h); + + // Current angles, right-aligned: the numbers to watch while dragging. + var f2 = Math.max(10, Math.round(num(lay.viewValueFont) * h)); + ctx.textAlign = 'right'; + ctx.font = '700 ' + f2 + 'px ' + FONT; + ctx.fillStyle = dragging ? ACCENT : '#ffffff'; + ctx.fillText('偏航 ' + fmtSigned(yaw) + '\u00b0 俯仰 ' + fmtSigned(pitch) + '\u00b0', + x1 - sh * 0.30, num(lay.viewLabelDy) * h); + ctx.font = Math.max(9, Math.round(num(lay.viewHintFont) * h)) + 'px ' + FONT; + ctx.fillStyle = 'rgba(255,255,255,0.60)'; + ctx.fillText('0\u00b0 / 0\u00b0 = 正前方', x1 - sh * 0.30, num(lay.viewHintDy) * h); + } + + // Signed degree readout for the drag strip: "+12" / "-8" / "0". + function fmtSigned(deg) { + var r = Math.round(deg); + if (r > 0) return '+' + r; + return String(r); + } + function drawExpanded(ctx, w, h, view, lay, dur, cur) { var fontSize = Math.max(10, Math.round(lay.textFont * h)); @@ -2358,6 +2457,12 @@ } if (mode === 'expanded') { + // ADDENDUM 8: the drag strip first. Its band (0..0.20) does not overlap the + // seek band (0.30..0.52) or the button bands (0.52+), so order is cosmetic. + if (ny >= lay.viewHitY0 && ny <= lay.viewHitY1 && + nx >= lay.viewStripX0 && nx <= lay.viewStripX1) { + return { type: 'view_drag' }; + } var i, cx, y0 = lay.btnHitY0, y1 = lay.btnHitY1; for (i = 0; i < BUTTONS.length; i++) { cx = lay.btnFirstCx + i * lay.btnSpacing; @@ -2415,6 +2520,7 @@ beamTex: null, // padlock (ADDENDUM 7b) — panel state only, no own layer lockOn: false, + hold: false, // ADDENDUM 8: drag in progress -> freeze auto-hide // XR resources canvas: null, ctx2d: null, @@ -2552,6 +2658,19 @@ // pointer position was already reduced to a hitTest() action by the draw path, // so this is just that lookup - and it means hover feedback needs no extra // plumbing from 90-main.js. + // ADDENDUM 8: current video-window angles in DEGREES, read from the shared + // state that 90-main.js owns (it owns the video layer's transform). + function readViewAngles() { + var n = ns(); + var st = (n && n.state) || {}; + var yaw = num(st.videoYaw); + var pitch = num(st.videoPitch); + return { + yawDeg: yaw == null ? 0 : yaw * 180 / Math.PI, + pitchDeg: pitch == null ? 0 : pitch * 180 / Math.PI + }; + } + function readHover() { var extra = reader(S.getExtra, 'hover'); if (typeof extra === 'string') return extra; @@ -2603,6 +2722,8 @@ hover: readHover(), pointer: viewPointer(), lock: !!S.lockOn, + // ADDENDUM 8: the video window's angles, in degrees, for the drag strip. + view: readViewAngles(), sub: readSubValues() }; // Diagnostics: what the bar actually rendered, so the CC state can be checked @@ -2633,7 +2754,9 @@ Math.round(v.sub.opacity * 100), Math.round(v.sub.angleDeg), v.sub.lock ? 1 : 0, - Math.round(v.sub.dist * 100) + Math.round(v.sub.dist * 100), + Math.round(v.view.yawDeg), + Math.round(v.view.pitchDeg) ].join('|'); } @@ -3152,7 +3275,13 @@ S.beamSig = null; // re-place the beam for this show applyBeamVisible(); if (S.beamOn) updateBeamTransform(); + restartAutoHide(autoHideMs); + updateVisibilityStats(); + } + + function restartAutoHide(autoHideMs) { if (S.autoHideTimer != null) { clearTimeout(S.autoHideTimer); S.autoHideTimer = null; } + if (S.hold) return; // a drag is in progress: stay up var n = ns(); var def = (n && n.state && typeof n.state.ctlAutoHideMs === 'number') ? n.state.ctlAutoHideMs @@ -3162,7 +3291,22 @@ S.autoHideTimer = null; hide(); }, ms); - updateVisibilityStats(); + } + + // ADDENDUM 8: pause the auto-hide while a hold-style interaction (the picture + // drag) is running. Without this the panel would disappear mid-drag, because + // the timer that show() armed keeps running. + function hold(on) { + var v = !!on; + if (v === S.hold) return; + S.hold = v; + if (v) { + if (S.autoHideTimer != null) { clearTimeout(S.autoHideTimer); S.autoHideTimer = null; } + } else if (S.shown) { + restartAutoHide(); + } + var s = stats(); + if (s) s.ctlHold = v; } // Fully hidden: no bar, no beam, no cursor, and no further drawing/uploading @@ -3172,6 +3316,9 @@ function hide() { if (S.autoHideTimer != null) { clearTimeout(S.autoHideTimer); S.autoHideTimer = null; } S.shown = false; + S.hold = false; + var hs = stats(); + if (hs) hs.ctlHold = false; applyBarVisible(); applyBeamVisible(); updateVisibilityStats(); @@ -3301,6 +3448,7 @@ show: show, hide: hide, isVisible: isVisible, + hold: hold, setOpacity: setOpacity, getOpacity: getOpacity, setLock: setLock, @@ -4517,7 +4665,18 @@ // video (bottom) -> hands -> subtitles -> controls (top) try { var vlayer = createVideoLayer(session); - if (vlayer) { NS.addLayer(vlayer); made.push('video'); } + if (vlayer) { + NS.addLayer(vlayer); + S.videoLayer = vlayer; + made.push('video'); + // ADDENDUM 8: angles start dead ahead every session (never restored), + // then the transform is applied so probe + apply see the real layer. + S.videoYaw = 0; + S.videoPitch = 0; + probeVideoAngleMutable(vlayer); + applyVideoAngle(false); // undo the probe's test rotation + updateVideoAngleStats(); + } } catch (e) { NS.recordError('video layer', e); made.push('video:err'); } // System hand models (WebXR Input Profiles 'generic-hand'), rendered into @@ -4572,6 +4731,7 @@ pollGamepads(frame); updatePanelPose(frame, dt); updateGrabFrame(frame); + updateViewDrag(frame); NS.runFrameHooks(frame, time, session); } session.requestAnimationFrame(loop); @@ -4603,8 +4763,15 @@ upperVerticalAngle: Math.PI / 2, lowerVerticalAngle: -Math.PI / 2 }; + // ADDENDUM 8: the transform is the picture's aim. radius stays 0 (content at + // infinity) so this is a pure rotation about the eye point. The transform is + // what makes a rebuild-based runtime possible, and what makes the very first + // frame already point where the user left it (a mid-session rebuild). + var t = videoAngleTransform(); + if (t) opts.transform = t; var layer = new XRMediaBinding(session).createEquirectLayer(video, opts); - log('video layer', JSON.stringify({ layout: opts.layout, central: opts.centralHorizontalAngle })); + log('video layer', JSON.stringify({ layout: opts.layout, central: opts.centralHorizontalAngle, + yaw: Math.round(S.videoYaw * 180 / Math.PI), pitch: Math.round(S.videoPitch * 180 / Math.PI) })); return layer; } @@ -4626,6 +4793,13 @@ S.lastFrame = null; grab = null; S.stats.grab = 'reset'; + viewDrag = null; + viewTaps = []; + S.videoLayer = null; + S.videoAngleLive = false; + S.videoYaw = 0; + S.videoPitch = 0; + if (NS.Controls && NS.Controls.hold) { try { NS.Controls.hold(false); } catch (e) {} } S.ctlLock = false; panelPose = null; pointer = { visible: false, beam: false, source: 'none', origin: { x: 0, y: 0, z: 0 }, dir: { x: 0, y: 0, z: -1 }, hit: null }; @@ -4875,6 +5049,200 @@ }).join(' ') || '(none)'; } + /* ================================================================== * + * ADDENDUM 8 — panning the video window + * + * The video layer is an equirect sphere with radius 0, i.e. the content sits at + * infinity. That has an important consequence: only the sphere's ORIENTATION + * decides which direction you see, and translating it does nothing. So panning + * the picture is purely `layer.transform`'s rotation, which rotates the content + * about the eye point — no parallax, no swimming when the head moves, and no + * decode/upload cost (8K HEVC DirectPlay is untouched). + * + * Only the video rotates. The control panel lives at a fixed pose in refSpace + * and the subtitles are head-locked in viewer space, so the UI stays in front of + * the user while the picture moves. + * ================================================================== */ + var VIDEO_PITCH_LIMIT = Math.PI / 2 - 0.02; // "unlimited", minus a gimbal guard + + function wrapYaw(a) { + while (a > Math.PI) a -= Math.PI * 2; + while (a < -Math.PI) a += Math.PI * 2; + return a; + } + + function clampPitch(p) { return NS.clamp(p, -VIDEO_PITCH_LIMIT, VIDEO_PITCH_LIMIT); } + + // q = qy(-yaw) * qx(pitch): pitch about the content's own X first, then yaw about + // the world Y — the standard look-around composition. The yaw sign is flipped so + // the STATE reads the way the user thinks about it: +yaw = the picture has been + // moved to the RIGHT, +pitch = UP (so a scene that used to need a physical + // head-down is brought up to eye level). + // + // The window's aim direction (the layer's local -Z) then works out to + // (cos p · sin yaw, sin p, -cos p · cos yaw) + // and the drag adds the ray's own yaw/pitch deltas directly, so the content point + // under the cursor stays under the cursor. + function videoQuat(yaw, pitch) { + var qy = { x: 0, y: Math.sin(-yaw / 2), z: 0, w: Math.cos(-yaw / 2) }; + var qx = { x: Math.sin(pitch / 2), y: 0, z: 0, w: Math.cos(pitch / 2) }; + return qMul(qy, qx); + } + + function videoOrigin() { return { x: 0, y: 0, z: 0 }; } + + function videoAngleTransform() { + if (typeof XRRigidTransform === 'undefined') return undefined; + return new XRRigidTransform(videoOrigin(), videoQuat(S.videoYaw, S.videoPitch)); + } + + function updateVideoAngleStats() { + S.stats.videoYaw = Math.round(S.videoYaw * 180 / Math.PI); + S.stats.videoPitch = Math.round(S.videoPitch * 180 / Math.PI); + S.stats.videoAngleLive = !!S.videoAngleLive; + } + + // XREquirectLayer.transform is settable in the spec, but a runtime is free to + // ignore the write, so probe it once: write a known rotation and read it back. + function probeVideoAngleMutable(layer) { + S.videoAngleLive = false; + if (!layer || typeof XRRigidTransform === 'undefined') return; + try { + var q = videoQuat(0.05, 0.05); + layer.transform = new XRRigidTransform(videoOrigin(), q); + var o = layer.transform && layer.transform.orientation; + S.videoAngleLive = !!(o && Math.abs(o.y - q.y) < 1e-4 && Math.abs(o.w - q.w) < 1e-4); + } catch (e) { + S.videoAngleLive = false; + } + S.stats.videoRotMode = S.videoAngleLive ? 'live' : 'rebuild'; + if (!S.videoAngleLive) { + log('video layer transform is read-only -> rebuilding the layer on changes'); + } + } + + // Push the current angles into the layer. On a live runtime this is just a + // transform write. On a runtime that ignores the write we must REBUILD the media + // layer — which is far too expensive to do once per drag frame, so rebuilds + // happen only when `force` is set (creation, release, reset). + function applyVideoAngle(force) { + var layer = S.videoLayer; + if (!layer || !S.session) return; + if (S.videoAngleLive) { + try { + layer.transform = videoAngleTransform(); + if ('needsRedraw' in layer) layer.needsRedraw = true; + } catch (e) { + S.videoAngleLive = false; + S.stats.videoRotMode = 'rebuild'; + } + } + if (!S.videoAngleLive && force) rebuildVideoLayer(); + updateVideoAngleStats(); + } + + function rebuildVideoLayer() { + var session = S.session; + if (!session || !S.video) return; + try { + var fresh = createVideoLayer(session); + if (!fresh) return; + var old = S.videoLayer; + S.videoLayer = fresh; + NS.addLayer(fresh); + if (old) { + NS.removeLayer(old); + try { if (typeof old.destroy === 'function') old.destroy(); } catch (e) {} + } + S.stats.videoRebuilds = (S.stats.videoRebuilds || 0) + 1; + } catch (e) { + NS.recordError('videoRebuild', e); + } + } + + function resetVideoAngle(why) { + S.videoYaw = 0; + S.videoPitch = 0; + if (S.session) { try { applyVideoAngle(true); } catch (e) {} } + S.stats.videoReset = why || 'reset'; + updateVideoAngleStats(); + } + + /* ---------------- drag the picture (ADDENDUM 8) ---------------- + * The pointer ray's own direction is the drag handle: rotating the aiming ray by + * δ rotates the picture by the same δ, so the content follows the hand 1:1 with + * no range limit (a panel-space cursor would run off the panel's edge). + * -------------------------------------------------------------- */ + var viewDrag = null; + var VIEW_TAP_MS = 1600; // three taps inside this window reset the view + var viewTaps = []; + var lastPressSrc = null; // source / frame of the last accepted press, for + var lastPressFrame = null; // hold-style actions (the picture drag) + + function rayAngles(ray) { + if (!ray || !ray.dir) return null; + var d = ray.dir; + var len = Math.sqrt(d.x * d.x + d.y * d.y + d.z * d.z) || 1; + var dx = d.x / len, dy = d.y / len, dz = d.z / len; + return { + yaw: Math.atan2(dx, -dz), + pitch: Math.asin(NS.clamp(dy, -1, 1)) + }; + } + + function beginViewDrag(src, frame) { + var ray = rayOf(frame, src); + var a = rayAngles(ray); + if (!a) { S.stats.viewDrag = 'no ray'; return; } + viewDrag = { src: src, a0: a, yaw0: S.videoYaw, pitch0: S.videoPitch, moved: false }; + S.stats.viewDrag = 'started'; + if (NS.Controls && NS.Controls.hold) { try { NS.Controls.hold(true); } catch (e) {} } + } + + function updateViewDrag(frame) { + if (!viewDrag || !frame) return; + var ray = rayOf(frame, viewDrag.src); + if (!ray) return; + var a = rayAngles(ray); + if (!a) return; + var dyaw = wrapYaw(a.yaw - viewDrag.a0.yaw); + var dpitch = a.pitch - viewDrag.a0.pitch; + var ny = wrapYaw(viewDrag.yaw0 + dyaw); + var np = clampPitch(viewDrag.pitch0 + dpitch); + if (Math.abs(ny - S.videoYaw) < 1e-4 && Math.abs(np - S.videoPitch) < 1e-4) return; + viewDrag.moved = true; + S.videoYaw = ny; + S.videoPitch = np; + applyVideoAngle(false); + } + + // Release. A press that never moved counts as a TAP: three taps inside + // VIEW_TAP_MS reset the view (the owner's "连点 3 次复位"). Counting taps on + // RELEASE rather than on press means a real drag cannot be mistaken for one. + function endViewDrag() { + if (!viewDrag) return; + var moved = viewDrag.moved; + viewDrag = null; + if (NS.Controls && NS.Controls.hold) { try { NS.Controls.hold(false); } catch (e) {} } + if (!S.videoAngleLive) applyVideoAngle(true); // pay the rebuild once, on release + if (moved) { + viewTaps = []; + S.stats.viewDrag = 'ended (moved)'; + return; + } + var now = performance.now(); + viewTaps = viewTaps.filter(function (t) { return now - t < VIEW_TAP_MS; }); + viewTaps.push(now); + S.stats.viewTaps = viewTaps.length; + if (viewTaps.length >= 3) { + viewTaps = []; + resetVideoAngle('triple-tap'); + S.stats.viewDrag = 'reset (triple tap)'; + return; + } + S.stats.viewDrag = 'tap ' + viewTaps.length + '/3'; + } + /* ================================================================== * * Fixed panel pose * @@ -5109,6 +5477,10 @@ if (NS.Controls && NS.Controls.show) { try { NS.Controls.show(); } catch (e) {} } // extend auto-hide pressUntil = now + 300; S.stats.pressAt = pointer.hit.nx.toFixed(2) + ',' + pointer.hit.ny.toFixed(2); + // Remembered for hold-style actions (the ADDENDUM 8 picture drag), which need + // the input source and frame that started them. + lastPressSrc = src; + lastPressFrame = frame; try { NS.Controls.handleSelect(src, frame); } catch (e) { NS.recordError('handleSelect', e); } return; @@ -5126,6 +5498,7 @@ function pointerUp() { endGrab(); + endViewDrag(); } function pollGamepads(frame) { @@ -5209,6 +5582,10 @@ // The padlock and the subtitle actions do not need a