Files
cat-shark 0270cd1bc1 docs: record the smooth/throughput dimension (0.5.0.0)
README 1.3 now documents both blocking conditions (supported=false and
smooth=false), the pin list, and the fact that nothing is asked of the user.
HANDOFF 18.7.1 explains why smooth was deliberately left out in 0.3.0.0 and why
that was wrong: it is precisely the realistic desktop case (decoder present, 8K
too slow), and it is what this machine actually hit.
2026-10-18 01:35:15 +08:00

1193 lines
53 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Jellyfin Web VR — Quest 3 / WebXR MVP
给 Jellyfin Web 播放页注入一个「VR 播放」按钮,在 Meta Quest 3 浏览器里直接进入
VR180 / VR360 沉浸播放,不用再手动切全屏 + 选投影。
---
## 1. 当前部署状态
已注入到 `192.168.123.199` 上的 Jellyfin 容器 `Jellyfin`(当前版本 `mvp-2e32e6`,`NS.version = hud-6`,含 ADDENDUM 6 + 7 + 7b + 8):
| 项 | 值 |
|---|---|
| 脚本 | `/jellyfin/jellyfin-web/jfvr-mvp.js`(容器内) |
| 注入点 | `/jellyfin/jellyfin-web/index.html` 的 `</body>` 之前 |
| 备份 | `/vol2/jellyfin/config/_vr_backup/index.html.orig` |
| 两个入口都生效 | `http://192.168.123.199:8096/web/` 与 `https://jellyfin.cat-shark.xyz/web/` |
> ⚠️ 老路子(`deploy.py`):注入写在**容器内文件**里,`jellyfin/jellyfin:latest` 一升级就没了,
> 得重跑 `deploy.py`。**现在已经有插件版了,见下面 §1.1,优先用插件。**
### 1.1 插件化(推荐,已实现)
JFVR 同时也是一个标准的 **Jellyfin 插件**,通过自建 Gitea 仓库分发,走 Dashboard 的
「添加存储库 → Catalog 安装」,**不再改容器内任何文件**。
| 项 | 值 |
|---|---|
| 插件仓库地址 | `https://git.cat-shark.xyz/cat-shark/jellyfin_vr_web_plugin/raw/branch/main/manifest.json` |
| 插件名 / GUID | `Jellyfin Web VR` / `7d33ef59-f9b0-412b-93bd-5cd5353018de` |
| 目标 ABI | `12.0.0.0`(Jellyfin 只保留 `targetAbi <= 服务器版本` 的条目,所以一份包覆盖整个 12.x) |
| 插件代码 | `plugin/Jellyfin.Plugin.JFVR/`(C#,net10.0,对 `Jellyfin.Controller 12.1.0`) |
| 打包 / 发布 | `python3 pack.py` / `python3 pack.py --bump --publish` |
| 脚本端点 | `/JFVR/jfvr.js`(bundle 内嵌在 DLL 里,URL 带内容哈希做缓存键) |
| 状态查询 | `/JFVR/status` |
| 设置页 | Dashboard → 插件 → Jellyfin Web VR(开关注入 / 覆盖脚本 URL) |
| 页面按钮 | 「VR 播放」按钮**可拖动**(位置记住;双击回到默认右下角) |
**安装**:Dashboard → 插件 → 目录 → ⚙️ → ➕ → 粘贴上面的仓库地址 → 保存 →
在 Catalog 里找到 **Jellyfin Web VR** → 安装 → **重启 Jellyfin**。
**工作方式**:`ScriptInjectionStartupFilter` 在**请求时**把
`<script plugin="JFVR" src="../JFVR/jfvr.js?v=<hash>" defer></script>`
插到 `index.html` 的 `</body>` 之前。磁盘上的 `/jellyfin/jellyfin-web/` 一个字节都不动,
所以升级 Jellyfin 不会丢。如果检测到老 `deploy.py` 留下的 `<script plugin="JellyfinVR" ...>`,
会**自动清掉**,不会重复加载 bundle。
**改完 web 代码后的流程**:
```bash
python3 pack.py --bump --publish # 构建 → 打包 → 建 Release → 推 manifest
# 然后在 Jellyfin 里:插件页点更新(或重装),重启,刷新浏览器
```
> 老路子仍然可用,作为插件的兜底:`JF_PW=... python3 deploy.py`。
> 但**两者不要同时开** —— 装了插件就别再跑 `deploy.py`(跑完插件也会把那个标签清掉)。
### 1.2 播放看门狗(卡住检测)
`src/40-watchdog.js`。针对一种很坑的失败:**某些设备/浏览器放不出来时,Jellyfin 的播放器界面根本不会被创建**。
原因(已从 Jellyfin Web 源码确认):播放器 UI 只在 `playbackstart` 事件之后构建,而 `playbackstart` 要等媒体真的起来了才会触发。加载顺序是:
```
HtmlVideoPlayer.play() htmlVideoPlayer/plugin.js:502
createMediaElement() :1802 转圈 + <div.videoPlayerContainer> + <video>
(此时还没 src)
setCurrentSrc() :593 elem.src = url
→ loadedmetadata → canplay → play → 'playbackstart'
→ OSD 控制器(apps/legacy/controllers/playback/video/index.js:591)
```
媒体起不来 → `playbackstart` 不触发 → **码率菜单根本不存在**,页面上只剩一个转圈和一个 `videoWidth === 0` 的 `<video>`。所以「卡住之后调不了码率」不是菜单藏起来了,是菜单还没被创建。
看门狗就盯这个状态(`.videoPlayerContainer` 里的 `<video>` 仍然是 `videoWidth === 0`):
| | 阈值 |
|---|---|
| 媒体报错(`video.error`) | 立即弹 |
| 直连没画面 | 15 秒 |
| 转码(src 已经是 hls) | 45 秒(服务器要先出片) |
| 已经有画面 | 立即退出,不打扰 |
| VR 会话中 | 完全不跑 |
弹窗会写明**具体格式**(走 Jellyfin API 查 MediaStreams),并给 20/8/3 Mbps 三档 + 「继续等待」。选一档后写入 Jellyfin 自己那套 localStorage 键然后重载:
```
maxbitrate-Video-<true|false> = 20000000
enableautobitratebitrate-Video-<true|false> = false
```
> 注意 `enableautobitratebitrate` 里 `bitrate` 确实出现了两次 —— 这是 Jellyfin 上游的拼写,照抄的。
> 之所以不用 `playbackManager.setMaxStreamingBitrate()`:那个模块没暴露到 window(已实测 `window.playbackManager === undefined`,`window.Emby` 只剩 `Page`),所以只能写设置 + 重载。
选过之后 120 秒内不再打扰(记在 sessionStorage)。
### 1.3 让客户端如实上报解码能力(`canPlayType` 覆盖)
`src/00-core.js`。解决的问题:**Jellyfin 的直连/转码决策 100% 看客户端上报的设备档案**,而 Jellyfin Web 是从 `canPlayType` 推出整个 HEVC 声明的(`browserDeviceProfile.js`):
```js
if (canPlayType('video/mp4; codecs="hvc1.2.4.L183"')) {
maxHevcLevel = 183; // level 6.1 = 8K
hevcProfiles = 'main|main 10';
}
```
`canPlayType` 回答的是「这台机器*有没有* HEVC 解码器」,不是「能不能解 8192×4096 的 Main 10 @60fps」。所以一个只是「系统里有 HEVC 解码器」的桌面浏览器会声称支持 8K Main 10 → 服务器选 DirectPlay → 黑屏(而 OSD 压根不会创建,见 §1.2)。
插件的做法:
1. 页面加载时用 **`MediaCapabilities.decodingInfo`** 对 HEVC 各档做真机探测(每档在它对应的分辨率和码率上测:L120→1080p30/8Mbps、L153→4K60/20Mbps、L183/L186→8K60/30Mbps,因为 level 字符串本身不带尺寸);
2. 把结果回灌给 `HTMLVideoElement.prototype.canPlayType`。
**拦掉一档的条件有两个,都是「确定的不能」**:
| 条件 | 含义 |
|---|---|
| `supported === false` | 完全解不了这个格式 |
| `smooth === false` | 能解,但**在那种分辨率和帧率下跟不上** —— 就是「有 HEVC 解码器,但 8K 跑不动」这种桌面场景 |
只有 `powerEfficient:false`(纯软解)**本身不拦**。
这样 Jellyfin Web 自己那套 `maxHevcLevel` 逻辑就会得出**准确**档案:
| 设备 | 结果 |
|---|---|
| 桌面(声称支持但实际解不动 8K Main 10) | 降到 level 153 → 8K 文件超出声明 → **服务器自动转码** |
| Quest 3(真能解 8K Main 10) | 保持 ≥183 → **继续 Direct Play** |
注意:这里**不问用户任何东西**。客户端如实上报,Jellyfin 在 `PlaybackInfo` 那一步自己选 Transcode —— 这正是「提示用户切码率」想要达到的目的,只是做在了正确的层。
逃生开关:
```js
localStorage['jfvr-cpt'] = 'off'; // 完全关掉钩子
localStorage['jfvr-cpt-keep'] = '2:183'; // 只保留某一档(单台设备误报时用,不用整个关掉)
```
当前状态可以在诊断面板里看到,每档会写明**为什么**被拦(`unsupported` / `not-smooth`)或是否被 pin:
```
canPlayType hook : on (blocked: Main10 L6.1 (8K) (not-smooth), ...)
HEVC real (decodingInfo):
Main10 L6.1 (8K) supported=true smooth=false powerEfficient=true [8192x4096@60 30Mbps] <-- BLOCKED (not-smooth)
```
这也是 Meta 官方文档推荐的做法 —— 他们明确说「用 Media Capabilities API 探测,不要靠 UA 嗅探」。
### 1.4 「VR 播放」按钮可以拖了
原来钉死在右下角(`right:16px; bottom:120px`),窗口矮的时候正好压在播放看门狗弹窗的码率按钮上,没地方躲。
现在:
- 用 `left/top` 定位(不再用 `right/bottom`),用 pointer 事件拖动,位置记在 `localStorage['jfvr-btn-pos']`
- `touch-action:none` —— 不加这个,触屏上拖动会变成滚动页面
- **拖动结束仍然会触发一次 click**,代码会把那一次吃掉,所以拖完不会意外弹出面板
- 限制在视窗内,窗口缩放时会重新夹紧
- **双击回到默认右下角**(拖到尴尬位置时的逃生口)
- 设置面板跟着按钮走(不再是钉在视窗角落),优先放在按钮上方,上方放不下就放下方,再放不下就夹在屏幕内
---
## 2. Quest 3 实测数据(已用 ADB + CDP 在真机验证)
### 2.0 设备与能力
```
Quest 3 Android 14 (SDK 34) build UP1A.231005.007.A1
Quest Browser 152.1.0.34.52.1079628208 (Chromium 152.0.7977.83)
UA Mozilla/5.0 (X11; Linux x86_64; Quest 3) ... Chrome/152 ... VR Safari/537.36
```
| 检测项 | `http://192.168.123.199:8096` | `https://jellyfin.cat-shark.xyz` |
|---|---|---|
| `isSecureContext` | false | **true** |
| `navigator.xr` | false | **true** |
| `immersive-vr` | — | **true** |
| **`XRMediaBinding.createEquirectLayer`** | — | **true** ← 首选路径可用 |
| `XRWebGLBinding` | — | true ← 回退路径可用 |
| `MAX_TEXTURE_SIZE` | 8192 | 8192 |
| `MAX_RENDERBUFFER_SIZE` | 16384 | 16384 |
| 8K HEVC `decodingInfo` | supported / powerEfficient | **supported:true, powerEfficient:true**(仅 8-bit Main) |
结论:**HTTPS 下 Quest 浏览器提供全部所需 WebXR 能力,8K HEVC 走硬件解码。**
> ⚠️ 上面那行 `decodingInfo` 当年只测了 **8-bit Main** 的 codec string(`hev1.1.6.L183.B0`),
> **10-bit Main 10(`hev1.2.4.L183.B0`)从来没测过**。插件现在会分档探测(见 §1.3),
> 诊断面板里会列出 Main/Main10 × 4K/8K 的真实结果。
### 2.1 网络实测:HTTPS 域名在 Quest 上就是内网直连
```
Quest 3 IPv6 : 2409:8a14:44a:e360:70b9:446e:d5f7:10a7/64
NAS IPv6 : 2409:8a14:44a:e360:520:8aa2:7736:4f12/64 ← 同一 /64
domain AAAA → NAS 自己的局域网 IPv6
服务端连接表 : [NAS]:443 ← [Quest]:35860 (ESTAB, IPv6)
```
60 MB 文件在 Quest 浏览器上实测:
| Run | 吞吐 | TTFB |
|---|---|---|
| 1 | 450 Mbps | 11 ms |
| 2 | 620 Mbps | 7 ms |
| 3 | **702 Mbps** | 12 ms |
对比:从局域网 PC 用 curl 走 IPv6 只有 4.29 MB/s(34 Mbps),走 IPv4 阿里云中转只有 94 KB/s。
> 慢的根因:NAS 上跑着 **`npc` 内网穿透客户端**,域名的 **A 记录 `39.105.149.197` 是阿里云上的 nps 服务端**,
> 走 IPv4 就是 NAS → 阿里云 → 你。**走 IPv6 则完全不出内网。**
>
> 所以 **不需要为了速度退回 HTTP**。8K 片源码率仅 21 Mbps,702 Mbps 有 30 倍余量。
### 2.2 如果仍想用 `192.168.123.199:8096`
WebXR 要求「安全上下文」。实测结果:
| 入口 | `isSecureContext` | `navigator.xr` | WebXR |
|---|---|---|---|
| `http://192.168.123.199:8096` | **false** | **false** | ❌ 需先做 §2.1 |
| `https://jellyfin.cat-shark.xyz` | true | true | ✅ 直接可用 |
用内网 HTTP 地址 —— Quest 浏览器开 flags
Meta Quest 2/3/3S(**OS 79+**)浏览器支持在 HTTP 模式下使用 WebXR,需显式授权:
1. Quest 浏览器地址栏输入 `chrome://flags`
2. 搜索 `insecure`
3. 启用 **`unsafely-treat-insecure-origin-as-secure`**
4. 在下方输入框填:`http://192.168.123.199:8096`
(**必须带 `http://` 和端口号**;多个用逗号分隔)
5. 点空白处失焦 → 屏幕底部出现 **Relaunch** → 点它
6. 回到 `chrome://flags` 确认 flag 仍启用且 URL 已保存
代价:这台设备专属、浏览器更新后可能需要重设。
### 2.3 兜底:让局域网内解析到内网 IPv4
如果某天 IPv6 不可用,且 Quest 落到了 IPv4(阿里云中转)导致慢,最一劳永逸的修法是在
**OpenWrt 路由器(192.168.123.1)** 上加一条静态解析,让局域网内所有设备都解析到内网 IP:
- LuCI:`网络 → DHCP/DNS → 一般设置 → 地址`,添加 `/jellyfin.cat-shark.xyz/192.168.123.199`
- 或 SSH 到路由器:
```sh
uci add_list dhcp.@dnsmasq[0].address='/jellyfin.cat-shark.xyz/192.168.123.199'
uci commit dhcp && /etc/init.d/dnsmasq restart
```
之后 `https://jellyfin.cat-shark.xyz` 在局域网内全程走内网,证书依然有效,所有设备(不只 Quest)都受益。
---
## 3. 实操:如何在真机上验证(ADB + CDP)
```bash
ADB=/opt/SideQuest/resources/platform-tools/adb # SideQuest 自带的 adb
# 1) 看设备(SideQuest 的无线 ADB)
$ADB devices -l
# 2) 把 Quest 浏览器的 DevTools 端口转发出来
$ADB forward tcp:9222 localabstract:chrome_devtools_remote
curl -s http://127.0.0.1:9222/json/version
# 3) 让 Quest 打开某个页面
$ADB shell am start -a android.intent.action.VIEW \
-d "http://192.168.123.199:8096/web/index.html" \
-n com.oculus.browser/.OculusLauncherActivity
# 4) 截图 / OCR
$ADB exec-out screencap -p > /tmp/quest/shot.png # 4128x2208
tesseract /tmp/quest/shot.png - --psm 6
```
拿到 `9222` 后可以直接用 `agent_browser --cdp http://127.0.0.1:9222` 接管 Quest 浏览器,
在页面上跑 `Runtime.evaluate`、读网络、发输入事件。
> 注:Quest 浏览器**不把 `console.log` 转发到 logcat**,所以诊断要走 CDP `Runtime.evaluate` 或页面 UI。
---
## 4. 使用步骤(Quest 3 里)
1. 打开 Jellyfin(§2.1 或 §2.2 任一入口)
2. 进媒体库,播放一个 VR 视频
3. 右下角出现蓝色 **「VR 播放」** 按钮 → 点它
4. 面板里确认 **投影**(180°/360°)和 **立体**(单目/左右/上下)
- 自动按文件名 + 分辨率判断;**8192×4096(2:1)会默认选 180° + 左右**
5. 点 **「进入 VR」** → 进入沉浸模式;摘头显即退出
进不去时展开面板里的 **「诊断信息」**,把内容贴回来。
---
## 5. 在 VR 里修掉的两个关键缺陷
真机(Quest 3 + CDP)实测发现并修复:
### 5.1 卡顿 —— 帧率不匹配(3:2 pulldown)
| 项 | 实测值 |
|---|---|
| 视频帧率 | **59.94 fps** |
| XR 会话默认 | **90 Hz**(实测 87.9 fps) |
| `session.supportedFrameRates` | `[72, 80, 90, 120]` |
| `session.frameRate`(初始) | **0**(未指定) |
90 ÷ 59.94 = 1.5 → 视频帧在显示上被交替重复 1 次、2 次 —— 经典 3:2 pulldown 抖动。
2D 播放时系统会自动匹配刷新率,所以「web 端流畅、进 VR 就卡」。
**修复**:进 VR 后调用 `session.updateTargetFrameRate(120)`。120 ÷ 60 = 2,每帧均匀显示两次。
```
修复前 fps : 87.9
修复后 fps : 114.9 (20 秒稳定 111–115,丢帧 0.000%)
```
### 5.2 字幕丢失 —— libass 被冻结
两个独立原因叠加:
**(a) 字幕画布不在 XR 场景里。** Jellyfin Web 12 用 libass 渲染 ASS 到独立的
`canvas.libassjs-canvas`,和 `<video>` 是两个元素。只把 `<video>` 放进 XR layer,字幕自然没有。
**(b) 进了 VR,libass 就停止渲染。** 实测:
```
window.requestAnimationFrame : 6 秒内 0 次调用 ← 完全停摆(XR 接管了帧循环)
timeupdate : 22 次 ✅
requestVideoFrameCallback : 288 次 ✅
```
Jellyfin 的字幕渲染由 rAF 驱动 → 画布冻结在进 VR 前的最后一帧。
**修复**:安装一个 rAF 代理 —— 进 VR 后把 `requestAnimationFrame` 的回调收进自己的队列,
用 `video.requestVideoFrameCallback` 驱动(它跟着视频帧走,正好是字幕需要的节奏);
不在 VR 时直接透传原生实现,2D 播放完全不受影响。
**字幕层**:用 `XRWebGLBinding.createEquirectLayer` 建第二个动态等距柱状层
(同一立体几何),把画布拷进 `getSubImage(layer, frame).colorTexture` 后经 GL 合成。
### 5.3 字幕分辨率
libass 按屏幕上的视频框尺寸渲染(**1416×708**,在 180° 球面上只有约 **4 px/度**,
而 Quest 3 约 **19 px/度** → 非常糊)。
必须**在进入 VR 之前**放大:XR 会话运行期间 libass 不会重算画布尺寸。
所以放大动作挂在「打开 VR 面板」这个点击上(进 VR 本来就要先开面板)。
```
放大前 : 1416x708
放大后 : 4320x2160 (仍是精确 2:1,约 24 px/度)
```
### 5.4 你的字幕文件本身就是为 SBS 设计的
```
Title: VR Dual-Eye Subtitle
PlayResX: 3840 PlayResY: 1920 ← 2:1,与视频同比例
Style: LeftEye , Alignment=8, MarginL=50 , MarginR=1920, MarginV=700
Style: RightEye, Alignment=8, MarginL=1920 , MarginR=50 , MarginV=700
956 条对白,每条成对出现(LeftEye + RightEye)
```
实测墨迹分布 `left=30, right=30`(完全对称)—— 左半屏给左眼、右半屏给右眼,
正好匹配 `stereo-left-right` 层。**不需要任何转换。**
---
## 6. 硬性约束:必须由用户手势触发
WebXR 规范原文:
> Wait for the window to have *transient activation* …
> If the request was not made while the global object has transient activation, return `false`.
所以 `navigator.xr.requestSession()` **必须同步写在点击处理函数里**,前面不能有任何 `await`
(否则 user activation 丢失,被 UA 拒绝)。代码里这一点是硬性保证的,改动时不要破坏。
---
## 7. 渲染路径
| 路径 | 条件 | 说明 |
|---|---|---|
| **A. `XRMediaBinding.createEquirectLayer`**(首选) | 浏览器支持 WebXR Media Layers | Meta 官方推荐:视频直接交给系统合成器,不经 WebGL 采样。**8K 下唯一靠谱的路** |
| **B. WebGL2 等距柱状光线重建**(回退) | 上面不可用 | 每帧 `texImage2D(video)` 上传整帧,8K 会吃力,仅兜底 |
投影参数:
- VR180 SBS → `layout:'stereo-left-right'`, `centralHorizontalAngle:π`
- VR360 SBS → `layout:'stereo-left-right'`, `centralHorizontalAngle:2π`
- 上下 3D → `layout:'stereo-top-bottom'`
---
## 8. 修改 / 部署 / 回滚
### 8.1 插件(推荐)
```bash
python3 pack.py --bump --publish # 构建 + 打包 + 建 Release + 推 manifest.json
```
然后在 Jellyfin 里更新插件并重启,刷新浏览器。
### 8.2 老路子:直接注入容器内文件(兜底)
```bash
# 改 jfvr-mvp.js 后重新部署
JF_PW='<ssh密码>' python3 deploy.py
# 完全回滚(恢复原始 index.html 并删除脚本)
JF_PW='<ssh密码>' python3 deploy.py rollback
```
`deploy.py` 用文件 md5 前 6 位作为版本号写进 script 标签的 query,自动破缓存。
插件走的是同一套破缓存思路,只是 hash 由 DLL 内嵌 bundle 的 md5 算出来。
---
## 9. Meta 官方规格(决定性依据)
出处:`developers.meta.com/horizon/documentation/web/browser-video/`
- 支持的容器:**MP4 / Ogg / WebM**(**MKV 不在列表**)
- 支持的编码:h.264 / **h.265** / vp8 / vp9 / AV1
- > **For all formats, Browser supports 4K and supports 8K on h.265 and AV1.**
- > **WebXR Media Layers 是显示沉浸式视频最高效的方式。如果可能,围绕这个 API 设计你的体验。**
- Quest 浏览器还内置 2D 页面全屏重投影(可手动选 360/180 + 单目/左右/上下),
但 2D 路径最大显示尺寸 ≤ 7680×3840,且每次要手动选 —— 所以走 WebXR 才有意义。
WebXR 规范(`immersive-web/webxr` index.bs):
> Wait for the window to have **transient activation** …
> If the request was not made while the global object has transient activation, return `false`.
所以 `navigator.xr.requestSession()` 必须**同步**写在点击处理里,前面不能有 `await` ——
代码里是硬保证,改动时不要破坏。
---
## 10. 片源情况(实测)
```
/mnt1/123 → 61 个 VR 文件,全部:
8192 × 4096 | HEVC Main / level 6.1 | ~60 fps | 8–25 Mbps | .mp4 (faststart 已确认)
```
即 **VR180 SBS**,每眼 4096×4096 的 180° 等距柱状。
> ⚠️ **必须保持 Direct Play**。服务器的 Pentium Gold 8505 + UHD 核显编不了 8K
> (Alder Lake-N 编码上限约 4K),一旦 Jellyfin 触发转码,分辨率腰斩且 CPU 打满。
> 诊断面板里的 `playbackMode` 会显示当前是 DirectPlay 还是 HLS。
---
## 11. 相关现成实现(备查)
| 项目 | 技术 | 备注 |
|---|---|---|
| `allexxex/jellyfinvr` | Three.js + WebXR | VR180/360/SBS/OU;每帧上传纹理,8K 下是瓶颈 |
| `Mix1C/Jellyfin-360VR-Player` | A-Frame | 仅 360°,无 180 |
| `Lqlsoftware/Jellyfin-vr-plugin` | C# 插件 | 独立播放页 `/VR/Video/{id}/Play` |
| `Toastyice/DeoVRDeeplink` | C# 插件 + `deovr://` | 可完全自动唤起 Quest 上的 DeoVR(deeplink 走系统,不受 user gesture 限制) |
| `IAmParadox27/jellyfin-plugin-file-transformation` | C# 插件 | 正式版注入通道 |
| `n00bcodr/jellyfin-plugins` → JavaScript Injector | 插件 | `12/manifest.json`,targetAbi 12.0.0.0 |
---
# ADDENDUM 2 — in-VR UX pass
Feedback from the first headset test, and what changed.
## Fixed
| Feedback | Fix |
|---|---|
| The progress bar was permanently visible | **Hidden by default.** `show()` appears on any pinch/trigger and auto-hides after `ctlAutoHideMs` (default 4 s, slider 1–10 s). While hidden: no draw, no texture upload, and the layer is dropped from the render state. |
| No guide line for hand/controller input | **Laser beam + cursor.** A thin quad beam (height 6 mm) runs from the input source to the panel, bright at the origin fading to transparent, plus a ring/dot cursor drawn at the exact hit point. |
| Subtitles should sit below the visual centre | Default `subPlacement.y` moved `-0.34 → -0.42` (quad centre, negative = below eye level). |
| Subtitle opacity and position should be adjustable | 4 live sliders in the panel: opacity, vertical position, distance, width. |
Input is now **one code path for hands and controllers**: both expose
`XRInputSource.targetRaySpace`, so each is ray-cast against the panel plane.
Gaze (view centre) is only the fallback when no ray reaches the panel.
## Beam orientation — corrected convention
The first spec said "orient the quad so its local `-Z` follows the ray". That is
geometrically wrong: a quad's local `-Z` is its plane **normal**, so the quad's
plane would end up perpendicular to the ray and its `width` would lie *across*
the beam — a thin face-on bar, not a laser.
Correct:
```
position = origin + dir * (L/2) // L = hit distance, else 1.2 m
width = L, height = 0.006
local +X -> dir // beam axis lies IN the quad plane
local -Z -> perpendicular to dir, biased up so it is never edge-on
```
Verified on device against two different rays — the measured beam midpoint
matched `origin + dir·L/2` to 1 cm, and the beam layer is added/removed with the
pointer.
## UI bug found and fixed this round
Adding 5 sliders grew the settings panel to 784 px while the viewport was 708 px,
so `进入 VR` ended up **off-screen and unclickable** (`elementFromPoint` returned
`null`). The panel is now a flex column with a scrolling body and a **sticky
footer** holding the status line and the CTA, and `max-height: calc(100vh - 104px)`.
## Verified on Quest 3 after this pass
```
layers : [equirect, quad(op=0.85)] ← control bar correctly absent
sub.placement : [-0.42, -1.45, 2.0]
sub.opacity : 0.85 (applied to layer.opacity)
ctl.hidden : true ← default hidden
fps : 110.9 targetFrameRate: 120
errors : []
```
Beam check (pointer frozen so main's per-frame update could not overwrite it):
```
controller ray -> beam w=1.22 pos=(0.13,-0.46,-0.86) matches origin+dir·L/2
hand ray -> beam w=1.45 pos=(-0.12,-0.47,-0.89) (different dir, follows)
pointer null -> beam removed (layers 4 -> 3, beamOn false)
```
---
# ADDENDUM 3 — aiming guide + system gestures
## Root cause of "no guide line" AND "cursor click does nothing"
Both symptoms were one chain of three bugs:
1. `applyBeamVisible()` required `S.shown` — the beam only existed while the
control bar was visible. But the bar only appears on activation, so the user
could never aim at it before pinching. **Chicken-and-egg.**
2. `hook()` early-returned on `!S.shown`, so `presentBeam()` never ran and the
beam texture was never uploaded — the beam could not render even when flagged
visible. **Second, independent blocker.**
3. Only the `select` event was handled. Meta documents hand pinch as
"implemented in the emulated Gamepad attribute for the hand as a **button
press**", which does not reliably raise `select`. So on hands **nothing fired
at all** — no bar, no beam, no click.
Also: `pointer.visible` used to require an actual panel hit, so the aiming guide
would only appear once you were already on target. Now `visible` means "an input
source is pointing" and `hit` is separate (may be null).
## Use the platform's gesture recognition — do not reimplement it
| Platform facility | What it gives us |
| --- | --- |
| `XRInputSource.gamepad` emulated buttons | **hand pinch** (per Meta's WebXR Hands doc) |
| `selectstart` / `select` / `selectend` | controller trigger, and pinch on some builds |
| `targetRaySpace` | one ray API for hands **and** controllers |
So `90-main.js` now polls gamepad buttons for a rising edge **and** listens for
`select`, with a 250 ms dedupe so both paths cannot double-fire. No thumb↔index
joint-distance math is done anywhere.
System-reserved gesture to avoid: **palm pinch** (left hand = menu button, leaves
the session). Ordinary point-and-pinch is ours.
Activation now requires a real ray hit (`pointer.visible && pointer.hit`), so a
stray pinch can no longer fall back to gaze-centre and seek to 50%.
## Verified on device after the fix
```
pointing, NOT hitting the panel -> beamOn: true, ctlHidden: true
beam quad present (h=0.006, w=1.2)
```
## Feasibility: auto-adjusting subtitle depth from scene content
Measured on a real 8K VR180 frame (SBS pair, 320x160 downsample, SAD block
matching +-24 px):
| Band | median disparity | match residual | disparity IQR |
| --- | --- | --- | --- |
| mid frame (y 40-88) | 1 px | 6.1 | — |
| **subtitle band (y 96-144)** | -5 .. -8 px | **13.5-15.3** | **[-16, +1]** |
- Disparity IS extractable and temporally stable (mid band held 1 px across 6 frames).
- But the **subtitle band is the least reliable region**: 2.3x the matching
residual and a 17 px inter-quartile spread — i.e. several depths coexist there,
so no single depth represents it.
- Disparity -> metres needs the VR180 rig's focal length and baseline, which are
not known a priori.
- Compute is NOT the blocker: the naive JS test cost ~250 ms/frame, but a WebGL2
SAD shader would be ~1-2 ms. Not the limiting factor.
Prior art (the depth question is studied; the answer is not "match scene depth"):
- Sidenmark, Kiefer & Gellersen, *Subtitles in Interactive VR: Using Gaze to
Address Depth Conflicts* (IEEE VR'19, Lancaster): depth conflicts between
environment and subtitles cause **occlusion and strain**; they propose
**gaze-based** techniques.
- Shimizu et al. (IEEE 2021): four dynamic subtitle placement methods, balancing
vertical/horizontal/**depth** of eye movement.
- *Gaze-Adaptive Subtitles for 360° Videos in VR* (IEEE 2025): adaptive placement
reduces visual fatigue versus fixed below-the-FOV placement.
**But**: the literature's best technique needs **eye tracking**, and Meta does
not expose `eye-tracking` to WebXR — our negotiated `enabledFeatures` were
`[local, viewer, bounded-floor, layers, hand-tracking, webxr, local-floor]`.
So gaze-adaptive placement is not available to us.
Recommendation if this is pursued: option **A** — use the *mid-band* (reliable)
disparity to bias the depth toward the scene, blended ~30/70 with the comfortable
default, clamped to 1.4-3.0 m, smoothed with a 2-3 s time constant, and scale the
quad proportionally so the **angular text size stays constant** (otherwise the
text visibly pulses). Default OFF. Chasing per-frame scene depth (option C) is
not recommended: disparity noise plus vergence conflict produces discomfort.
---
# ADDENDUM 4 — in-VR subtitle placement
Adjust the subtitles **without leaving VR**. Verified on device this round.
## Grab & drag the strip
Point at the subtitle text, pinch, and move:
- **up / down** → vertical position (drag on a fixed reference plane)
- **pull toward / push away** → depth (the hand's `origin.z` in viewer space;
forward is `-Z`, so pulling back brings the strip closer — like a real object)
- release → the new position is saved
Both the ray and the strip live in **viewer space**, so no world-space maths is
needed. The reference plane is fixed at grab time so the drag stays stable.
### Two deliberate restrictions (both needed)
The strip sits in the lower centre of the view by default and the control bar is
0.30 m tall at `y = -0.28`, so the two **overlap in y**. Without restrictions the
strip would swallow every aim in the lower half of the view.
1. Grabbing only happens when the control bar is **not already shown** — a pinch
while operating the bar never yanks the subtitles.
2. Only the **central 40 %** of the strip counts as grabbable (`nx,ny ∈ 0.30–0.70`),
which is where the text actually is (the crop is letterboxed into the texture).
## In-VR buttons on the control bar
The expanded bar grew from 1.1 × 0.22 m to **1.1 × 0.30 m** and gained a subtitle
row. Verified reachable layout (grid scan, no ambiguous overlaps):
```
y 0.25-0.45 seek track (full width)
y 0.50-0.70 [<<<<][PPPP][>>>>][ZZZZ] back10 / playpause / fwd10 / subs
y 0.85-1.00 [aa][AA][oo][OO][RR] size −/+ · opacity −/+ · reset
```
Actions: `sub_smaller`, `sub_bigger`, `sub_opaque_down`, `sub_opaque_up`, `sub_reset`.
## Persistence
`localStorage['jfvr.settings.v1']`, debounced 400 ms, loaded at boot with range
validation and silent fallback on corruption.
```json
{"subPlacement":{"y":-0.78,"z":-1.45,"width":2.8},"subOpacity":0.45,
"subMode":"hud","subAlpha":"premultiplied","projection":"180","stereo":"sbs",
"subtitles":true,"controls":true,"ctlAutoHideMs":4000}
```
## Verified this round
```
live setPlacement -> layer [-0.7, -2.1, 2.6] immediately
live setOpacity -> 0.35
resetPlacement() -> [-0.42, -1.45, 2]
slider -> localStorage written, and after a page reload the values come back
errors: []
```
Note: the four subtitle sliders in the 2D panel and the in-VR buttons now share
one code path (`setSubPlacementLive` / `setSubOpacityLive` / `setSubModeLive`),
so both save automatically.
---
# ADDENDUM 6 — subtitle depth vs size decoupling (angular-size lock)
## The problem, as the owner put it
> 当前 vr 视频中可能出现 vr 空间中物体可能距离我很近,如果字幕还很远,我将无法在
> 看清物体的同时看清字幕。我希望可以让字幕的空间距离和物体的距离尽量接近。
Fixation is a **vergence** problem: an object at 0.4 m and subtitles at 1.45 m cannot
both be looked at comfortably at the same time. Eye tracking is not available to
WebXR on Quest (`enabledFeatures` has no `eye-tracking`), so the depth cannot be
driven by gaze — see ADDENDUM 3 above, which already concluded the gaze-adaptive
literature is out of reach for us.
What IS in reach is making **depth a pure depth knob**: keeping the strip's angular
size (and its angular position) constant while it moves in depth. That is exactly
the "scale the quad proportionally so the angular text size stays constant" note
from ADDENDUM 3, applied to the manual control.
Measured behaviour of the old control (free mode), which is why it did not work:
| depth | width | strip angular width |
| --- | --- | --- |
| 1.45 m | 2.00 m | 69° |
| 0.70 m | 2.00 m | 110° ← text blows up |
| 0.35 m | 2.00 m | 141° ← text leaves the view |
## What shipped
`state.subLockAngle = true`, `state.subAngleDeg = 69.2` (defaults). While locked:
```
width = 2 · |z| · tan(subAngleDeg / 2) constant angular width
y = (y0 / z0) · z constant angular height
```
so the strip stays at the same spot on the retina and only its **vergence
distance** changes. 69.2° is exactly the old `2.0 m @ 1.45 m`, so the default look
is unchanged. Depth range extended: near clamp **0.5 m → 0.35 m**.
In-VR subtitle row grew from five to nine buttons (the bar itself is unchanged —
`PANEL_M.expanded` = 0.72 × 0.21 m, canvas 1440 × 420 px). The row now carries
vertical position as well as depth, and the caption always leads with the depth:
```
y 0.80-1.00 [↑][↓][<<][>>][aa][AA][oo][OO][RR] 上 下 · 近 远 · 小 大 · 暗 亮 · 复位
caption 字幕 距1.45m · 69° 85%
```
`上`/`下` step the vertical position ±0.04 m (that control did not exist in the bar
before). `近`/`远` step depth ±0.15 m. `小`/`大` step the locked angle ∓5° (or
`width` ∓0.15 m when unlocked). Grab & drag carries `y` by the grab-start y/z
ratio while locked, so a strip pulled close does not slide below the view.
## Two bugs fixed in the same pass
1. **The CC (字幕) button never showed its state.** `Controls.readSubtitles()` only
read the controls module's own `S.subtitlesOn`, which `90-main.js` never sets —
it calls `setVideo(video)` with no `getExtra`. It now falls back to the shared
`state.subtitles`. The button is also unambiguous now: accent pill + `字幕开`
when on, slashed CC glyph + `字幕关` when off.
2. **There was no vertical-position control in VR.** Only grab & drag could move
the strip. The row gained `上`/`下`. (The 2D panel's `垂直位置` slider was
verified working — slider → `-0.70 m`, distance → `-0.80 m` re-derived
`y = -0.385` — so the gap was VR-only.)
## Verification (logic, offline)
The module was driven headlessly in node (state + maths only, no WebGL), holding
`subAngleDeg` at 69.2° and stepping depth:
```
z=-1.45 -> width 2.00, y -0.42 (69.2°)
z=-0.70 -> width 0.97, y -0.205 (69.2°)
z=-0.35 -> width 0.48, y -0.100 (69.2°)
z=-2.60 -> width 3.59, y -0.745 (69.2°)
```
and the 9-button row was grid-scanned through `Controls.hitTest` — 9 distinct
actions, no dead zone between pills. `Controls.draw` renders
`字幕开 · 字幕 距1.45m · 69°` while subtitles are on and locked, `字幕关` when
off, and `字幕 距1.45m · 2.0m` when unlocked. `build.py --check` passes.
Then verified in the **live Quest 3 browser** over CDP after deploying:
```
window.__JFVR.version -> 'hud-2'
state.stats.subLockAngle -> true
state.stats.subAngleDeg -> 69.2
state.errors -> []
Sub.setPlacement({z:-0.70}) -> width 0.97, y -0.20 (still 69.2°)
Sub.setPlacement({z:-0.35}) -> width 0.48, y -0.10 (still 69.2°)
Sub.setPlacement({z:-9}) -> width 3.59, y -0.745 (clamped to -2.6, still 69.2°)
Sub.setAngleDeg(50) -> width 2.42 (size control)
```
## Literature for the default value (not yet acted on)
All of these measure the **character height** angular size; our control is the
whole strip's angular width, so they need `stats.subTextDeg` (new) to compare.
- Kojić et al., *User Experience of Reading in Virtual Reality* (arXiv 2004.01545):
preferred angular size ≈ **16–18 dmm for medium/long text** (0.9–1.0° character
height), 27–32 dmm for 2-word text, vs Google's guidelines of 32 dmm (title) and
40 dmm (headline). Participants also rated **too-large** angular size among the
worst experiences.
- Dingler et al., *VR Reading UIs* (CHI'18 EA): **41 ± 14 dmm** (≈2.35° ± 0.8°).
- The default stays 69.2° deliberately: it is the on-device-proven look. Read
`stats.subTextDeg` in the headset, then decide.
## Persistence
`jfvr.settings.v1` gains `subLockAngle` and `subAngleDeg`; while locked the stored
`subPlacement.width` is re-derived from `subAngleDeg` + `z` on load.
```json
{"subPlacement":{"y":-0.42,"z":-0.35,"width":0.48},
"subLockAngle":true,"subAngleDeg":69.2,"subOpacity":0.85,
"subMode":"hud","subAlpha":"premultiplied","projection":"180","stereo":"sbs",
"controls":true,"ctlAutoHideMs":4000}
```
---
# ADDENDUM 7 — summon gating, lock badge, pre-VR panel trim
## The bar appeared on every pinch
Two independent causes, both removed:
1. `session.addEventListener('selectstart', onSelectStart)` called `Controls.show()`
unconditionally — any pinch anywhere in the view-summoned the bar.
2. `pointerDown()` called `Controls.show()` *before* it looked at the hit.
Now the bar is summoned only when the pointing ray lands on (or near) the bar:
```
bar hidden + aim on/near the bar -> show(), consume the press, arm for 500 ms
bar hidden + aim elsewhere -> nothing
bar shown + within 500 ms -> ignored
bar shown + aim on the bar -> handleSelect (and show() to extend auto-hide)
```
While LOCKED the same rule applies, except that what comes up is the padlock panel
alone — see ADDENDUM 7b below.
The summon window is wider than the hit window (`panelHit` padding 0.5/0.9 vs
0.25/0.6), because a ray that grazes the bar should still be able to bring it up —
the 500 ms arm is what prevents the accidental button press, not pixel precision.
**Superseded by ADDENDUM 7b:** the padlock was originally a separate quad layer to the
right of the bar. That did not work on device — the hand ray cannot reach that far
right. The padlock is now a normal panel button and "locked" is a third panel mode.
## Pre-VR settings panel trimmed
Sliders set on a flat screen do not predict what reads well in a headset, so the
subtitle section is now just 开 / 关. Removed: 模式, 透明合成, 不透明度, 垂直位置,
距离, 宽度, 距离与大小 (lock toggle), 视角宽度, 手部指向. `subAngleDeg`, placement
and opacity still persist — they are written by the in-VR controls.
The 手部指向 option (system ray vs index-finger ray) is gone together with its code
path; the system ray is the only mode (an index-finger ray injects fingertip jitter
into the cursor — see HANDOFF §6.6).
## "VR 播放" no longer covers the player's own controls
`.jfvr-btn` moved from `bottom: 24px` to `bottom: 120px`; at the viewport corner it
sat on Jellyfin's OSD fullscreen button. `.jfvr-panel` moved to `bottom: 180px`
(`max-height: calc(100vh - 200px)`) so the panel still stacks above the button.
## Verification
`build.py --check` passes. A local harness (a `file://` page with a fake 8192x4096
`<video>`, a fake libass canvas and stub `XRWebGLBinding`/`XRXRigidTransform`/
`createQuadLayer` classes) exercised the real bundle without a headset:
```
Controls.create(stubSession) -> ok, 4 quad layers (bar, beam x2, lock)
Controls.getLockLayer() -> the lock quad
Controls.setLock(true) -> stats.ctlLock = true, no throw
runFrameHooks(fakeFrame) -> presentLock/present ran, errors: []
panel DOM -> 画面: 投影/立体 · 字幕: 显示 · 控制: 控制条/自动收起
.jfvr-btn computed bottom -> 120px
drawLockBadge() -> 16 canvas ops, no throw (locked and unlocked)
```
Live Quest 3 over CDP after deploy: `__JFVR.version = 'hud-4'`,
`stats.subLockAngle = true`, `errors: []`.
---
# ADDENDUM 7b — padlock as a panel button + the "locked" panel mode
## The failure on device
> 我有新发现,在面板唤出状态时其无法点击,但面板消失后,其可以点击。
ADDENDUM 7's padlock was its own quad layer at `(0.45, -0.34, -1.45)` — 0.45 m right of
the panel centre, 0.09 m right of the panel's own right edge. Two things were wrong:
1. **It was outside every reachable ray.** Aiming is the binding constraint with hands:
a hand ray plus the cursor's acceptance window (`panelHit` 0.25/0.6 on a 0.72 m panel)
reaches x ≈ 0.45 — the panel's right edge. Nothing could aim 0.09 m past it.
2. **Its hit test used the panel's layer geometry.** Whether it fired therefore depended
on the panel's own size/position, which is why it behaved differently shown vs hidden.
## The fix: make it an ordinary button
The padlock is now the fifth button in the media row and uses the same code path as
everybutton that already worked (cursor → `hitTest` → `handleSelect` → `onAction`):
```
expanded: [<<<] [>||] [>>>] [CC] [padlock]
0.18 0.36 0.54 0.72 0.90 (hit pill half width 0.092)
```
## The behavior the owner asked for
> 想要的是:唤出面板,然后可以上锁,上锁后锁头图标和面板一样自动隐藏,下次触发面板
> 唤出时,只展示锁本身,不展示进度条等能调整的项目。(解锁后的0.5s的动做也忽略)
A third panel mode does exactly that:
| mode | size (m) | canvas | drawn / hit-testable |
| --- | --- | --- | --- |
| `expanded` | 0.72 × 0.21 | 1440 × 420 | everything |
| `locked` | 0.26 × 0.10 | 520 × 200 | the padlock + 解锁 label ONLY |
```
summon (unlocked) -> expanded panel, auto-hides after ctlAutoHideMs
press 上锁 -> panel becomes padlock-only IN PLACE, auto-hide timer restarts
(so the padlock disappears on its own, like the panel always did)
summon (locked) -> padlock alone; no progress bar, no rows, nothing adjustable
press 解锁 -> full panel returns, and 500 ms of presses are ignored
```
Because `hide()` no longer resets the mode, the layer keeps the size it will be shown
at: the summon window stays the full panel rect (0.72 × 0.21 unlocked, 0.26 × 0.10
locked) and re-showing never pops a resize. `hitTest()` takes the mode explicitly now
(`collapsed` and `locked` share no safe aspect-ratio invariant), and `handleSelect()`
no longer falls back to pressing the panel centre when there is no ray hit.
## Verification (offline, real bundle, no headset)
A temporary `file://` harness fed the real `dist/jfvr.js` a fake 8K `<video>`, a fake
libass canvas and stub `XRWebGLBinding` / `XRXRigidTransform` / `createQuadLayer`
classes. The harness file was deleted afterwards.
```
Controls.create(stubSession) -> ok, layers: bar + beam x2
expanded media row (5 centres) -> back10|playpause|fwd10|subs|lock, no dead zone
expanded subtitle row (9 centres) -> 9 distinct actions
locked panel: centre -> {type:'lock'} -> bottom edge / left edge -> null
draw() locked -> draws ONLY '解锁'
draw() expanded -> 上锁 + 字幕 距1.45m · 69° + 00:10 (all intact)
setLock(true) -> stats.ctlMode=locked, layer 0.72x0.21 -> 0.26x0.10, canvas resized
setLock(false) -> stats.ctlMode=expanded, layer back to 0.72x0.21
hide() -> mode KEPT (summon window stays the panel rect)
runFrameHooks() after each switch -> no errors, stats.hookErr / ctlErr null
handleSelect() with no ray hit -> null (was: silently pressed the centre)
```
Also standalone (`node`): the padlock hit region and the mode table above were asserted
directly against `Controls.hitTest` / `Controls.draw` with mock 2D contexts.
Live Quest 3 over CDP after deploy: `__JFVR.version = 'hud-5'`, `stats.ctlMode =
'expanded'`, `stats.ctlLock = false`, `errors: []`.
## Still unverified
The *feel* — how easily the padlock is aimed with the hand ray, and whether 0.5 s is the
right arm delay — can only be judged in the headset.
---
# ADDENDUM 8 — panning the 180° video window about the eye
## The requirement
> 我希望画面可以不是固定在我的正前方180度,我希望能调整画面内容在整个vr空间中以我眼睛为
> 旋转点进行上下左右的水平移动(为了解决一些场景需要物理低头才能看到的问题,我不想真的
> 低头去看)
The window was hard-wired dead ahead, so anything staged below the horizon cost a
physical head-down.
## Why a pure rotation is the right mechanism
The video layer is an `XREquirectLayer` with `radius` 0, i.e. the content is **at
infinity**. That single fact decides everything:
| Property of radius-0 equirect | Consequence for this feature |
| --- | --- |
| Only the orientation is visible; translation has no effect | Panning = rotating `layer.transform`, nothing else |
| No parallax | The picture does not swim when the head moves |
| It is a sphere "around everywhere" | The rotation is about the **eye point** — literally what was asked |
| No new texture / no re-upload | 8K HEVC DirectPlay, 120 Hz and the ~112 fps budget are untouched |
Only the video layer rotates. The control panel is at a fixed pose in `refSpace`, the
subtitles are head-locked in `viewerSpace`, so **the UI does not move with the
picture**.
Rejected alternatives:
| Alternative | Why not |
| --- | --- |
| `refSpace.getOffsetReferenceSpace()` | Rotates the panel, the hands and every ray too — the UI would leave the user's view |
| Moving the layer's position | An infinite-radius sphere shows no visible change (and would add parallax/swim) |
| Changing `centralHorizontalAngle` | That changes FOV/distortion, not where the window points |
## Controls
`state.videoYaw` (+ = picture right) and `state.videoPitch` (+ = picture up):
```
q = qy(-yaw) * qx(pitch) // pitch in the content frame, then yaw about world Y
aim = (cos p·sin yaw, sin p, -cos p·cos yaw) // the window's aim direction
```
The yaw sign is flipped so the two numbers read the way a user thinks about them, and
every value shown in the panel is a plain "moved this many degrees right/up".
Pitch is limited only by a gimbal guard (±(π/2 − 0.02)) per the owner's "不限,看用户拖动
幅度来定"; yaw wraps to [−π, π]. **Reset to 0/0 on every VR session** — not persisted.
Interaction is a single affordance that lives in the panel, above the progress bar
(the owner's requested placement):
```
⠿ 拖动调整画面 偏航 +12° 俯仰 -8°
连点 3 次复位 0° / 0° = 正前方
```
- Pinch it and drag — the picture follows the hand 1:1.
- Three taps within 1.6 s reset to dead ahead; taps are counted on **release**, so a
drag cannot be mistaken for one.
- The live angles are printed on the right at all times.
The drag uses the **ray's direction**, not a cursor position on the panel:
```
yaw' = wrap(yaw0 + (rayYaw - rayYaw0)), pitch' = clamp(pitch0 + (rayPitch - rayPitch0))
rayYaw = atan2(dx, -dz), rayPitch = asin(dy)
```
A panel-space cursor leaves the panel after a few degrees; ray rotation has no range
limit, and the content point under the cursor stays under the cursor — which is what
"grab the picture" has to feel like. `Controls.hold(true)` freezes the panel auto-hide
for the duration, otherwise the panel would auto-hide mid-drag.
## The one runtime unknown
> **SUPERSEDED by ADDENDUM 8b** — this probe compared quaternion components (sign- and
> normalisation-sensitive) and so could report "read-only" on a perfectly good runtime.
> The gesture also changed: tap = reset, hold 0.5 s then drag = pan.
`XREquirectLayer.transform` is settable per spec but a runtime may ignore the write. It
is probed once per session (write a known rotation, read it back):
```
stats.videoRotMode = 'live' -> a transform write per frame, free
= 'rebuild' -> recreate the media layer, but only on
creation / release / reset (never per frame)
```
`stats.videoRebuilds` counts rebuilds. **This probe needs a real XR session, so it is
the one thing that cannot be checked offline.**
## Offline verification (real bundle, no headset)
Quaternion / drag maths, extracted from `src/90-main.js` and driven directly:
```
yaw=0 pitch=0 -> aim (0, 0, -1) dead ahead
yaw=+20 -> aim x = +0.342 picture moved RIGHT
yaw=-20 -> aim x = -0.342 picture moved LEFT
pitch=+20 -> aim y = +0.342 picture moved UP
pitch=-20 -> aim y = -0.342 picture moved DOWN
yaw=30 pitch=40 -> unit direction, azimuth 30.000° and elevation 40.000° recovered
wrapYaw(±3π/2) -> ∓π/2 wraps into [-π, π]
clampPitch(±9) -> ±(π/2-0.02) gimbal guard holds
drag -> the aimed point ends up under the cursor (yaw and pitch)
```
Panel geometry via `Controls.hitTest` on the 1440×520 canvas:
```
y 0.05 / 0.18 -> view_drag (the whole strip)
y 0.25 -> null (time readout: intentionally inert)
y 0.40 -> seek
y 0.60 -> media row (back10|playpause|fwd10|subs|lock)
y 0.90 -> subtitle row (9 actions)
x sweep 0.04..0.96 at y 0.05 -> no dead column
```
`Controls.draw` renders `拖动调整画面` / `连点 3 次复位` / `偏航 +12° 俯仰 -8°` /
`0° / 0° = 正前方`, and `hold(true|false)` is reflected in `stats.ctlHold`.
Live Quest 3 over CDP after deploy: `__JFVR.version = 'hud-6'`,
`state.videoYaw = state.videoPitch = 0`, `errors: []`.
---
# ADDENDUM 8b — live pan, the crash, and the real gesture
## Why the picture only moved on release
ADDENDUM 8 gated the transform write on a probe that compared **quaternion components**:
```js
live = Math.abs(readBack.orientation.y - q.y) < 1e-4 && Math.abs(readBack.orientation.w - q.w) < 1e-4;
```
`q` and `−q` are the same rotation, and a runtime may return a renormalised quaternion,
so this could answer "read-only" on a runtime that would have accepted the write. When it
did, the drag fell into the rebuild branch — which was throttled to release-only — so the
picture did not move until the finger came up.
The probe now compares the **position** (exact, no sign ambiguity):
```
layer.transform = XRRigidTransform({0.123, -0.456, 0.789}, identity)
writable <=> layer.transform.position equals that marker
```
## Two mechanisms, mutually exclusive
| `stats.videoRotMode` | Mechanism | Cost per change |
| --- | --- | --- |
| `transform` | `layer.transform = q` | one write |
| `space` | `layer.space = refSpace.getOffsetReferenceSpace(q)` | one space alloc |
| `rebuild` | recreate the media layer (manual escape hatch) | ~8/second max |
`space` exists because if the transform write is genuinely ignored,
`getOffsetReferenceSpace()` is the recentering path every WebXR app relies on, and a
layer's pose is recomputed from its space each frame. The two are never combined:
`createVideoLayer()` only bakes the transform in for the `rebuild` strategy, otherwise the
probe's write would double the aim. `stats.videoTransformWritable` shows the raw probe
result; the 2D panel's 「画面旋转: 自动 / 兼容」 can force the rebuild path.
## The crash, and every fix around it
The reset rebuilt the layer and created the replacement **before** destroying the old
one — two `XRMediaBinding` equirect layers on one `<video>` element at once, then a
second rebuild microseconds later (release rebuilt, then reset rebuilt again) took the
browser down.
A rebuild now: runs in a `setTimeout` **outside** the XR frame callback, **destroys
first**, is throttled to 120 ms, and never queues more than one. The `transform`/`space`
paths never rebuild at all, so the normal case cannot reach the crash.
Two robustness holes found while testing:
- `videoSpaceFor()` let a throwing `getOffsetReferenceSpace()` escape into the frame
loop, where an exception stops it for good (`requestAnimationFrame` is only re-armed at
the end of the callback) — a frozen session. It is caught, and the strategy demotes to
`rebuild`.
- The whole per-frame body is now wrapped in a try/catch for the same reason.
## The gesture
```
press the drag strip
├─ released before 500 ms with no movement -> RESET to dead ahead
├─ held past 500 ms, then moved -> PAN
└─ held past 500 ms, never moved -> RESET
```
The pan reference is taken **when the hold threshold passes**, not at the press, so the
hold itself cannot fling the picture; a ~1° dead zone absorbs hand jitter. The strip reads
`点一下复位 · 按住 0.5s 再拖动`.
## Verification
Everything below was asserted offline against the real sources with a stubbed ray, stubbed
WebXR layer objects and a stubbed clock (`build.py --check` passes as well):
```
test-gesture quick tap -> reset; hold-without-move -> reset; movement inside the hold
window ignored; after arming, +10°/+5° ray -> exactly +10°/+5°; a pan does
not reset; dead zone respected; pitch guard holds
test-mech probe true for a writable transform, false for an ignored one;
transform strategy: writes, no rebuild; space strategy: assigns the offset
space, leaves the transform alone, no rebuild; unchanged angle reuses the
cached space; rebuild strategy: one pending max, observed sequence
removeLayer -> createVideoLayer -> addLayer; throwing/absent
getOffsetReferenceSpace -> reported + permanent fallback
test-ad8 quaternion aim/wrap/guard maths, panel bands on the 1440x520 canvas,
the strip's drawn readout, hold()
```
Deployed as `mvp-5217a3` (hud-7) and confirmed live on the Quest.
## Still to confirm on device
Which of `transform` / `space` this runtime actually honours — `stats.videoRotMode` and
`stats.videoTransformWritable` say so once a session is running.