Jellyfin Web VR plugin for Quest 3 (VR180 stereo playback)
Injects a 「VR 播放」 button into Jellyfin Web's video player page and plays
VR180 / VR360 stereo content on Meta Quest 3 directly in the browser, with
DirectPlay 8K HEVC, in-VR subtitles and an in-VR control panel.
Modules (src/, concatenated in filename order into dist/jfvr.js):
00-core.js namespace, shared state, GL helpers, layer/frame-hook registry
10-subtitles.js libass canvas -> head-locked quad layer, auto-cropped to ink,
with depth/size decoupling (angular-size lock)
20-controls.js in-VR control panel: progress/seek, transport, subtitle rows,
aiming beam, padlock, pure 2D canvas -> XR quad layer
30-hands.js system hand model (Three.js + generic-hand GLB) via a manually
driven projection layer
90-main.js boot, XR session lifecycle, rAF broker (rAF stops during XR),
input dispatch, settings panel, localStorage persistence
Key on-device findings are documented in HANDOFF.md §6 and README.md addenda:
requestAnimationFrame stops during an XR session, 3:2 pulldown at 90 Hz needs
updateTargetFrameRate(120), libass must be resized before entering VR, a quad's
local -Z is its plane normal, and hand aiming must use the gaze ray because the
emulated targetRaySpace points sideways. Quest does not expose eye tracking.
Docs: INTERFACES.md (module contract + addenda), README.md (measurements),
HANDOFF.md (handover notes). Credentials are NOT in the repo: deploy.py reads
the SSH password from the JF_PW environment variable.
This commit is contained in:
+20
@@ -0,0 +1,20 @@
|
|||||||
|
# --- secrets (never commit) -------------------------------------------
|
||||||
|
# Local copy of the NAS / Jellyfin credentials. The repo only carries
|
||||||
|
# placeholders; see HANDOFF.md §2.1.1.
|
||||||
|
secrets.local.md
|
||||||
|
secrets*.md
|
||||||
|
|
||||||
|
# --- duplicate / scratch ----------------------------------------------
|
||||||
|
# Root copy is byte-identical to legacy/jfvr-mvp-v6-working.js (which
|
||||||
|
# deploy.py --legacy actually uses), so only the legacy one is tracked.
|
||||||
|
/jfvr-mvp.js
|
||||||
|
# Temporary offline test page (fake <video> + libass canvas + XR stubs).
|
||||||
|
_harness.html
|
||||||
|
|
||||||
|
# --- build / tooling noise --------------------------------------------
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
*.swp
|
||||||
|
*~
|
||||||
+454
@@ -0,0 +1,454 @@
|
|||||||
|
# Jellyfin Web VR180 — 项目交接文档
|
||||||
|
|
||||||
|
> 给新对话用。读完这份就能接手,不需要重新调查环境。
|
||||||
|
> 最后更新:已部署版本 `7616c9`(`NS.version = hud-5`;ADDENDUM 6 字幕深度/大小解耦
|
||||||
|
> + ADDENDUM 7 唤出门限/进 VR 前面板精简 + ADDENDUM 7b 锁按钮与「锁定」面板模式)
|
||||||
|
> 已在 Quest 浏览器里实机验证模块加载 + 锁定数学(`subLockAngle/subAngleDeg`,`errors: []`);
|
||||||
|
> **VR 会话内的观感还没看过**。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 项目目标
|
||||||
|
|
||||||
|
给 Jellyfin Web 播放页注入插件,让 **Meta Quest 3 浏览器**能直接播放内网 Jellyfin 里的
|
||||||
|
**VR180 立体视频**,不用手动切全屏、选投影。在网页里点「VR 播放」→ 选格式 → 进 VR。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 环境事实(已核实,不用重查)
|
||||||
|
|
||||||
|
### 2.1 服务器
|
||||||
|
|
||||||
|
| 项 | 值 |
|
||||||
|
|---|---|
|
||||||
|
| 主机 | `192.168.123.199`(fnOS NAS,CW8505fnOS,Intel Pentium Gold 8505 + UHD 核显) |
|
||||||
|
| **SSH** | `admin` / `<NAS 密码>`(密码不写进仓库,见 §2.1.1) |
|
||||||
|
| Jellyfin | **12.1.0**,Docker 容器名 `Jellyfin`,`jellyfin/jellyfin:latest` |
|
||||||
|
| 配置目录 | `/vol2/jellyfin/config` |
|
||||||
|
| web 目录(注入点) | 容器内 `/jellyfin/jellyfin-web/` |
|
||||||
|
| 备份 | `/vol2/jellyfin/config/_vr_backup/index.html.orig` |
|
||||||
|
| 反向代理 | Traefik v3.0(`http://192.168.123.199:8080/api/rawdata` 可读配置) |
|
||||||
|
|
||||||
|
#### 2.1.1 密码怎么取
|
||||||
|
|
||||||
|
**仓库里不存密码。** 真实密码在本机一个被 `.gitignore` 忽略的文件里:
|
||||||
|
|
||||||
|
```
|
||||||
|
secrets.local.md # 不入库,只存在于开发机
|
||||||
|
```
|
||||||
|
|
||||||
|
`deploy.py` 也只从环境变量 `JF_PW` 读,代码里没有硬编码。换机器时要么把那份
|
||||||
|
`secrets.local.md` 拷过去,要么手动 `export JF_PW=...`。
|
||||||
|
|
||||||
|
### 2.2 两个访问入口(关键区别)
|
||||||
|
|
||||||
|
| 入口 | 安全上下文 | WebXR | 速度 |
|
||||||
|
|---|---|---|---|
|
||||||
|
| **`https://jellyfin.cat-shark.xyz`** | ✅ | **直接可用,零配置** | **IPv6 局域网直连,实测 700 Mbps,TTFB 7–12ms** |
|
||||||
|
| `http://192.168.123.199:8096` | ❌ | ❌ 需每台设备在 Quest 浏览器 `chrome://flags` 里启用 `unsafely-treat-insecure-origin-as-secure` 并填该地址 → Relaunch | 内网 |
|
||||||
|
|
||||||
|
> **推荐一律用 HTTPS 域名。** 该域名的 IPv6 AAAA 记录就是 NAS 自己的局域网地址
|
||||||
|
> (`2409:8a14:44a:e360:520:8aa2:7736:4f12`),Quest 与 NAS 在同一个 /64,纯内网直连。
|
||||||
|
> IPv4 的 A 记录 `39.105.149.197` 是阿里云中转(NAS 上跑着 `npc` 内网穿透客户端),走 IPv4 才慢。
|
||||||
|
|
||||||
|
### 2.3 Quest 3
|
||||||
|
|
||||||
|
| 项 | 值 |
|
||||||
|
|---|---|
|
||||||
|
| 系统 | Android 14 (SDK 34),build `UP1A.231005.007.A1` |
|
||||||
|
| 浏览器 | **`152.1.0.34.52.1079628208`**(Chromium 152.0.7977.83) |
|
||||||
|
| 无线 ADB | `192.168.123.229:34485`,由 SideQuest 维持 |
|
||||||
|
|
||||||
|
**ADB 与 CDP 通路:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ADB=/opt/SideQuest/resources/platform-tools/adb # SideQuest 自带的 adb
|
||||||
|
$ADB devices -l # 确认设备在线
|
||||||
|
$ADB forward tcp:9222 localabstract:chrome_devtools_remote
|
||||||
|
curl -s http://127.0.0.1:9222/json/version # 确认 CDP 通
|
||||||
|
```
|
||||||
|
|
||||||
|
项目里有 `quest.sh` 一键做上面这些。拿到 9222 后用 `agent_browser --cdp http://127.0.0.1:9222` 接管 Quest 浏览器。
|
||||||
|
|
||||||
|
> ⚠️ **CDP 转发会被无线 ADB 重置**,断了就重跑 `quest.sh`。
|
||||||
|
|
||||||
|
**截图与「看」画面:**
|
||||||
|
```bash
|
||||||
|
$ADB exec-out screencap -p > shot.png # 4128x2208,双眼并排合成输出
|
||||||
|
tesseract shot.png - -l chi_sim --psm 6 # OCR 可读中文
|
||||||
|
```
|
||||||
|
> ⚠️ 当前模型**看不到图片**,只能靠 OCR + 像素统计 + 差分分析。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 片源
|
||||||
|
|
||||||
|
```
|
||||||
|
61+ 个 VR180 文件,全部:
|
||||||
|
8192 x 4096 | HEVC Main / level 6.1 | ~60 fps | 8–25 Mbps | .mp4 (faststart)
|
||||||
|
```
|
||||||
|
|
||||||
|
即 **VR180 SBS**,每眼 4096×4096 的 180° 等距柱状。
|
||||||
|
|
||||||
|
**字幕文件是天生的双目设计**(853 部是这样的):
|
||||||
|
|
||||||
|
```
|
||||||
|
Title: VR Dual-Eye Subtitle
|
||||||
|
PlayResX: 3840 PlayResY: 1920 ← 2:1,与视频同比例
|
||||||
|
Style: LeftEye , Alignment=8, MarginL=50 , MarginR=1920, MarginV=700
|
||||||
|
Style: RightEye, Alignment=8, MarginL=1920, MarginR=50 , MarginV=700
|
||||||
|
```
|
||||||
|
|
||||||
|
左半屏=左眼、右半屏=右眼,正好匹配 `stereo-left-right`。
|
||||||
|
|
||||||
|
> ⚠️ **必须保持 DirectPlay。** Pentium 8505 + UHD 核显编不了 8K(Alder Lake-N 编码上限约 4K),
|
||||||
|
> 一旦 Jellyfin 转码,分辨率腰斩且 CPU 打满。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Quest 3 实测能力(真机数据)
|
||||||
|
|
||||||
|
| 能力 | 结果 |
|
||||||
|
|---|---|
|
||||||
|
| `isSecureContext` / `navigator.xr`(HTTPS 下) | true / true |
|
||||||
|
| `immersive-vr` | **true** |
|
||||||
|
| `XRMediaBinding.createEquirectLayer` | **true** |
|
||||||
|
| `XRWebGLBinding.createQuadLayer` | **true** |
|
||||||
|
| `XRWebGLBinding.getSubImage` | **需要 2 个参数** `(layer, frame)` |
|
||||||
|
| `MAX_TEXTURE_SIZE` / `MAX_RENDERBUFFER_SIZE` | **8192 / 16384** |
|
||||||
|
| 8K HEVC `decodingInfo` | `supported:true, powerEfficient:true` |
|
||||||
|
| `supportedFrameRates` | **[72, 80, 90, 120]** |
|
||||||
|
| `enabledFeatures` | `[local, viewer, bounded-floor, layers, hand-tracking, webxr, local-floor]` |
|
||||||
|
| 手部追踪 | ✅ 25 关节齐全,profiles `["oculus-hand","generic-hand","generic-hand-select","generic-trigger"]` |
|
||||||
|
| **眼动追踪** | ❌ **Meta 不向 WebXR 暴露** |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 架构
|
||||||
|
|
||||||
|
```
|
||||||
|
jelllyfin_vr_web_plugin/
|
||||||
|
├── INTERFACES.md ← 绑定契约(所有模块必须遵守)+ 5 个 ADDENDUM
|
||||||
|
├── README.md ← 技术记录(含全部实测数据)
|
||||||
|
├── HANDOFF.md ← 本文档
|
||||||
|
├── src/
|
||||||
|
│ ├── 00-core.js (343 行) 命名空间、共享状态、GL 助手、层/帧钩子注册表、能力探测
|
||||||
|
│ ├── 10-subtitles.js (1022 行) 字幕:libass 画布 → 头锁定 quad 层(HUD)+ 视角尺寸锁定
|
||||||
|
│ ├── 20-controls.js (1930 行) 控制面板:进度条/按钮/光束/光标/锁按钮,canvas 2D → quad 层
|
||||||
|
│ ├── 30-hands.js (569 行) **系统手部模型**:Three.js + 官方 generic-hand GLB → projection 层
|
||||||
|
│ └── 90-main.js (1420 行) 引导、会话生命周期、rAF 代理、输入分发、设置面板、持久化
|
||||||
|
├── build.py ← src/*.js 按文件名顺序拼接 → dist/jfvr.js;`--check` 只做语法检查
|
||||||
|
├── deploy.py ← 构建 + 上传进容器 + 注入 script 标签;`rollback` 还原
|
||||||
|
├── quest.sh ← 一键重连 Quest 的 ADB + CDP 转发
|
||||||
|
├── dist/jfvr.js ← 构建产物
|
||||||
|
└── legacy/jfvr-mvp-v6-working.js ← 早期验证可用的单文件版(回退用)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.1 层栈(**顺序即合成顺序,后的在上**)
|
||||||
|
|
||||||
|
```
|
||||||
|
video equirect XRMediaBinding.createEquirectLayer, 180°, stereo-left-right
|
||||||
|
hands projection Three.js 渲染官方手模型(手动渲染循环,不占 baseLayer)
|
||||||
|
subtitles quad viewer 空间,头锁定,自动裁剪墨迹包围盒
|
||||||
|
controls quad local 空间,固定位置 (0,-0.34,-1.45),身份旋转
|
||||||
|
```
|
||||||
|
|
||||||
|
> ⚠️ **Three.js 不能接管 XR 会话**:`renderer.xr.setSession()` 会设 `baseLayer`,
|
||||||
|
> 与 `layers` 数组冲突。所以手模型走**手动渲染循环**(自建 projection layer、
|
||||||
|
> 自绑帧缓冲、自遍历 view 设投影矩阵)。
|
||||||
|
|
||||||
|
### 5.2 关键 API
|
||||||
|
|
||||||
|
```js
|
||||||
|
__JFVR.addLayer(layer) / removeLayer(layer) // 层注册表,自动推 updateRenderState
|
||||||
|
__JFVR.addFrameHook(fn) / removeFrameHook(fn) // fn(frame, time, session) 每 XR 帧
|
||||||
|
__JFVR.state // 共享状态;window.__JFVR_STATE__ 也是它(供 CDP 读取)
|
||||||
|
__JFVR.stats // 诊断信息,全部落在 state.stats
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 关键技术发现(**血泪教训,不要重犯**)
|
||||||
|
|
||||||
|
### 6.1 `window.requestAnimationFrame` 在 XR 会话中完全停摆
|
||||||
|
|
||||||
|
```
|
||||||
|
实测:6 秒内 0 次回调
|
||||||
|
timeupdate : 22 次 ✅
|
||||||
|
requestVideoFrameCallback : 288 次 ✅
|
||||||
|
```
|
||||||
|
|
||||||
|
Jellyfin 用 rAF 渲染 libass 字幕 → **进 VR 后字幕画布冻结在最后一帧**。
|
||||||
|
**解法**:90-main.js 劫持 `requestAnimationFrame`,XR 中把回调收进私有队列,
|
||||||
|
用 `video.requestVideoFrameCallback` 驱动;不在 XR 时透传原生实现。
|
||||||
|
|
||||||
|
### 6.2 帧率不匹配导致"卡顿"
|
||||||
|
|
||||||
|
59.94fps 视频 + 90Hz 会话 = **1.5 倍** → 每帧交替显示 1/2 次 = 3:2 pulldown 抖动。
|
||||||
|
**解法**:`session.updateTargetFrameRate(120)`(120/60 = 2 整除)。
|
||||||
|
实测 87.9 → 114.9 fps,20 秒稳定、丢帧 0.000%。
|
||||||
|
|
||||||
|
### 6.3 libass 画布必须在**进入 VR 之前**放大
|
||||||
|
|
||||||
|
libass 按屏幕视频框尺寸渲染(约 1250×708 = **3.5 px/度**,糊)。
|
||||||
|
**XR 会话运行期间 libass 不重算画布尺寸**,所以放大动作挂在「打开 VR 面板」这个点击上。
|
||||||
|
实测放大到 **4320×2160**(libass 封顶)。
|
||||||
|
|
||||||
|
**踩过的坑**:退出 VR 时如果还原尺寸,而面板还开着,再进 VR 就不会重新放大 → 字幕糊。
|
||||||
|
现在:面板开着就不还原。
|
||||||
|
|
||||||
|
### 6.4 quad 的局部 `-Z` 是**面法线**,不是轴向
|
||||||
|
|
||||||
|
想在光束朝向里把 `-Z` 对齐射线是**错的** —— 会让平面垂直于射线、`width` 横在射线上,
|
||||||
|
看到的是一条正对眼睛的细横条。正确:**`+X`(长轴)对齐射线**,`-Z` 与射线垂直。
|
||||||
|
|
||||||
|
### 6.5 单条平面条带沿视线方向时**必然侧对** → 十字双条带
|
||||||
|
|
||||||
|
光束指向"远离你"的方向时,任何包含光束轴的平面都包含视线方向 → 侧对 → 投影宽度趋零。
|
||||||
|
实测有效宽度只剩 `cos(68.7°) = 0.36` 倍 ≈ 2mm ≈ **3 像素,等于隐身**。
|
||||||
|
**解法**:**两条正交条带**(第二条绕光束轴转 90°),`max(cosθ, sinθ) ≥ 0.707` 恒成立。
|
||||||
|
|
||||||
|
### 6.6 手部 `targetRaySpace` 是**模拟射线**
|
||||||
|
|
||||||
|
Meta 官方文档原文:
|
||||||
|
|
||||||
|
> the `targetRaySpace` for hands is populated with an **emulated ray** that matches the
|
||||||
|
> behaviors of the pointing ray of hands present in **Meta Horizon Home**, which assumes
|
||||||
|
> that the UI is in front of the user, so the target rays will **always point in that
|
||||||
|
> general direction**
|
||||||
|
|
||||||
|
- **优点**:系统做了**稳定化**,落点平稳
|
||||||
|
- **缺点**:不跟手指
|
||||||
|
|
||||||
|
> ❌ 我一度改成用食指关节自己算射线 → **指尖逐帧抖动直接传导到落点,明显比系统射线不稳**。
|
||||||
|
> **默认必须用系统射线。** 面板里留了「手部指向:系统射线 / 食指」开关供切换。
|
||||||
|
|
||||||
|
### 6.7 `requestSession()` 必须**同步**写在点击处理里
|
||||||
|
|
||||||
|
WebXR 规范:需要 *transient activation*。前面不能有 `await`,否则被拒。
|
||||||
|
实测用 `element.click()` 从 CDP eval 调用会失败(`The requested session requires user activation`),
|
||||||
|
但用 `agent_browser click` 可以(CDP 输入事件算用户手势)。
|
||||||
|
|
||||||
|
### 6.8 `frame.getPose()` 的结果**只对该帧有效**
|
||||||
|
|
||||||
|
反复读同一个旧 `XRFrame` 会一直返回缓存值 —— 我因此误判过"射线冻结"。
|
||||||
|
**采样必须在帧循环内做。**
|
||||||
|
|
||||||
|
### 6.9 其余坐标陷阱
|
||||||
|
|
||||||
|
- 面板带 yaw 旋转后,"平面垂直于局部 z 轴"的假设失效 → `panelHit` 必须
|
||||||
|
**把射线变换到层自身空间**再求交(含四元数旋转/共轭/姿态合成)
|
||||||
|
- 字幕层在 **viewer 空间**而指针射线在 **local 空间** → 两者不同坐标系,必须转换
|
||||||
|
|
||||||
|
### 6.10 系统保留手势
|
||||||
|
|
||||||
|
**掌心肌肤捏合**(palm pinch)是系统保留:左手 = 菜单键,**会直接退出会话**。
|
||||||
|
只能用普通点捏。
|
||||||
|
|
||||||
|
### 6.11 面板“一捏就弹”有两个独立原因(ADDENDUM 7)
|
||||||
|
|
||||||
|
只修 `pointerDown` 里的 `show()` 不够——`selectstart` 监听里也有一句无条件 `show()`。
|
||||||
|
两处删掉后才真的变成“指哪儿弹哪儿”。另外**唤出那一下不算点击**,否则手从远处
|
||||||
|
扫到面板的路径上会顺手碰到一个按钮。
|
||||||
|
|
||||||
|
### 6.12 锁按钮为什么一度做成了独立图层(错误,已改回)
|
||||||
|
|
||||||
|
第一版把锁做成了控制条右侧的**独立 quad 图层**(x=+0.45),实测选不中。教训:
|
||||||
|
|
||||||
|
- **能用手的射线扫到的范围 = 光标自己的容差窗口**(`panelHit` 对 0.72m 面板用
|
||||||
|
0.25/0.6 → 只能到 x≈0.45,即面板右边缘)。放在更外面就是**任何射线都到不了**。
|
||||||
|
- 锁的命中检测当时用的是**面板的图层几何**,所以“面板显示时点不中、隐藏时能点中”——
|
||||||
|
行为随面板尺寸变化。
|
||||||
|
|
||||||
|
正确做法:锁就是**面板上的一个普通按钮**(`lock` 动作),走光标→hitTest→handleSelect
|
||||||
|
→onAction 这条已验证能用的路。锁定后切到第三种面板模式 `locked`(0.26×0.10),
|
||||||
|
**只画锁、只命中锁**。
|
||||||
|
|
||||||
|
### 6.13 「头部锁定」的东西永远瞄不到
|
||||||
|
|
||||||
|
“VR 空间右侧边缘”听起来像视野边缘的头锁定 HUD,但这个项目**只能用手的射线**指向
|
||||||
|
(手柄连不上)。头锁定元素的相对角度永远不变,射线**永远追不上它**,所以任何可点的
|
||||||
|
元素都必须和控制条在同一个 `refSpace` 里。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 当前状态(部署 `8ae73b`)
|
||||||
|
|
||||||
|
### 7.1 可用
|
||||||
|
|
||||||
|
| 功能 | 状态 |
|
||||||
|
|---|---|
|
||||||
|
| VR180 立体播放 | ✅ DirectPlay 8K HEVC,120Hz,~112 fps |
|
||||||
|
| 字幕 HUD(头锁定 quad) | ✅ 自动裁剪墨迹包围盒、双目半屏提取 |
|
||||||
|
| 控制面板(fixed 位置) | ✅ 进度条/播放暂停/±10s/字幕开关/字号±/透明±/复位 |
|
||||||
|
| 中文按钮标签 | ✅ |
|
||||||
|
| 引导线(十字双光束) | ✅ 任何时候都显示 |
|
||||||
|
| 面板唤出门限 | ✅ 只有指示线指向/靠近面板时才唤出;唤出那一下不算点击;500ms 内忽略 |
|
||||||
|
| 锁按钮(面板不启动) | ✅ 面板媒体行的第五个按钮:上锁后切到「只显示锁」的最小面板,并同样自动隐藏;每次进 VR 重置为未锁 |
|
||||||
|
| 按压反馈 | ✅ 圆环放大 1.9× + 蓝填充 + 变白,300ms |
|
||||||
|
| 字幕抓拖 | ✅ 注视选中 + 手移动拖(与视线解耦,避免转头就飘) |
|
||||||
|
| 字幕深度/大小解耦 | ✅ 默认「锁定视角尺寸」:拉近/拉远只改深度,视角大小与画面位置不变 |
|
||||||
|
| VR 内「近/远」字幕按钮 | ✅ 控制条字幕行现为 9 键:上 下 近 远 小 大 暗 亮 复位 |
|
||||||
|
| 字幕行题注显示距离 | ✅ `字幕 距1.45m · 69°`(距离永远是第一个数字,便于和物体对深度) |
|
||||||
|
| 字幕 CC 按钮状态 | ✅ 开=蓝底+`字幕开`,关=斜杠 CC+`字幕关` |
|
||||||
|
| 设置持久化 | ✅ localStorage `jfvr.settings.v1` |
|
||||||
|
| 系统手模型 | ✅ 官方 `generic-hand` GLB,Three.js 渲染,帧率无损失 |
|
||||||
|
|
||||||
|
### 7.2 已知未完成 / 待验证
|
||||||
|
|
||||||
|
- **锁定角尺寸的默认值还没定** —— 代码默认 69.2°,但头显上已存的是 **85°**
|
||||||
|
(之前用面板滑杆设的,ADDENDUM 7 已移除那个滑杆)。要回默认就在 VR 里按几次「小」
|
||||||
|
(每次 −5°),或者直接改 localStorage。
|
||||||
|
- **ADDENDUM 7/7b 的 VR 手感未验证** —— 唤出灵不灵、锁按钮好不好矒、500ms 合不合适,
|
||||||
|
只能戴上去试。模块级冒烟测试已过(见 README ADDENDUM 7b 的 Verification)。
|
||||||
|
- **手柄无法连接** —— **系统层问题,不是插件问题**。
|
||||||
|
`adb shell getevent -i` 只列出 5 个设备(无任何控制器),WebXR 的 `inputSources`
|
||||||
|
也只有双手。需要在头显里检查手柄电量 / 重新配对 / 确认「自动切换」开关。
|
||||||
|
- 手模型实际渲染效果**还没在头显里确认**(subagent 交付后只验证了加载成功 `handsReady: true`)
|
||||||
|
- 可疑现象:不捏合时 `inputSources` 仍显示左手 `@0`(按钮 0 卡在按下)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. 字幕距离 / 角尺寸 —— 已解决(ADDENDUM 6)
|
||||||
|
|
||||||
|
### 用户要解决的问题
|
||||||
|
|
||||||
|
> 当前 vr 视频中可能出现 vr 空间中物体可能距离我很近,如果字幕还很远,我将无法在
|
||||||
|
> 看清物体的同时看清字幕。我希望可以让字幕的空间距离和物体的距离尽量接近。
|
||||||
|
|
||||||
|
Quest 不给 WebXR 眼动(§4),**没法自动跟视线距离**;而且近处物体足够强行的方案是
|
||||||
|
让「距离」变成一个**纯深度**旋钮:不管拉到多近,字幕的角尺寸和画面位置都不变。
|
||||||
|
|
||||||
|
### 实现(默认开启)
|
||||||
|
|
||||||
|
`state.subLockAngle = true`,`state.subAngleDeg = 69.2`。锁定时拉近/拉远:
|
||||||
|
|
||||||
|
```
|
||||||
|
width = 2 · |z| · tan(subAngleDeg / 2) 保持角尺寸 θ
|
||||||
|
y = (y0 / z0) · z 保持角高度 y/z
|
||||||
|
```
|
||||||
|
|
||||||
|
69.2° 就是旧的 `2.0 m @ 1.45 m`,所以**默认观感零变化**。近端限位从 0.5 m 收到
|
||||||
|
**0.35 m**。ADDENDUM 7 之后锁**固定在开**(面板里的「距离与大小」开关已按用户要求移除),
|
||||||
|
所以不再有「拉近=变大」的自由模式入口。
|
||||||
|
|
||||||
|
- VR 内控制条字幕那一行现在是 **9 个按钮**:`上 下 近 远 小 大 暗 亮 复位`。
|
||||||
|
- `上/下`:垂直位置 ±0.04 m(以前 VR 里只能抓拖,没有按钮)
|
||||||
|
- `近/远`:深度 ±0.15 m(锁定下只改深度,角尺寸不变)
|
||||||
|
- `小/大`:锁定下视角宽度 ±5°(解锁后是宽度 ±0.15 m)
|
||||||
|
- 题注永远先报**深度**:`字幕 距1.45m · 69°`,右边是不透明度。要用它和眼前物体对深度。
|
||||||
|
- 顺手修了两个 bug:① 控制条 CC 按钮以前读的是 controls 模块自己的、main 从不写入的
|
||||||
|
`S.subtitlesOn`,所以**永远显示为关**——现在改为回退读共享的 `state.subtitles`,
|
||||||
|
并把开/关做成蓝底 `字幕开` / 斜杠 CC `字幕关`;② VR 里没有垂直位置按钮(现已补上)。
|
||||||
|
2D 面板的「垂直位置」滑杆实测是好的,所以缺的只是 VR 侧。
|
||||||
|
- 抓拖在锁定下会把深度位移按抓取起点的 y/z 比例同步到 y,所以拉到很近时字幕
|
||||||
|
不会掉到视野下方。
|
||||||
|
- 新增诊断 `stats.subTextDeg`:**实际画出来的字幕墨迹角高度(度)**。用它对照
|
||||||
|
文献就能定默认 `subAngleDeg`。
|
||||||
|
|
||||||
|
### 文献结论(保留供参考)
|
||||||
|
|
||||||
|
核心指标是 **角尺寸(angular size)**= 距离 × 字体大小的组合,不是单独某一个:
|
||||||
|
|
||||||
|
- *User Experience of Reading in Virtual Reality — Finding Values for Text Distance, Size and Contrast*
|
||||||
|
(arXiv 2004.01545,全文表 I 已复现):以 dmm(1 dmm = 1 m 处 1 mm 字高)为单位,
|
||||||
|
**中/长文本偏好 16–18 dmm ≈ 0.9–1.0° 字高**,2 词短文本 27–32 dmm ≈ 1.6–1.8°;
|
||||||
|
Google 指南:正文/说明 ~15–20 dmm,标题 32 dmm ≈ 1.8°,大标题 40 dmm ≈ 2.3°。
|
||||||
|
参与者明确认为**角尺寸过大**是「最差」读数体验之一 → 存在上限。
|
||||||
|
- *VR Reading UIs: Assessing Text Parameters for Reading in VR*(Dingler 等,CHI 2018 EA):
|
||||||
|
偏好字高 **41 ± 14 dmm ≈ 2.35° ± 0.8°**。
|
||||||
|
- *The influence of text rotation, font and distance on legibility in VR*(IEEE VR 2020)
|
||||||
|
与 *An effective method for measuring text legibility in XR*(Frontiers in VR 2023,含 Quest 2)
|
||||||
|
都用 dmm/角度测最小可读字号,结论同向:可读性由角尺寸决定。
|
||||||
|
|
||||||
|
上述都是**字高**角尺寸,而本插件的滑杆/参数是**字幕条整幅角宽**,两者靠 `subTextDeg`
|
||||||
|
换算。默认值暂不动(保持已调好的观感),等头显实测后再定。
|
||||||
|
|
||||||
|
### 待办
|
||||||
|
|
||||||
|
1. 头显里读 `stats.subTextDeg`,若明显偏离 1–2° 再考虑改默认 `subAngleDeg`
|
||||||
|
2. (可选)如果实际用下来「近/远」每次 0.15 m 太粗,再加长按连续调整
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. 工作流速查
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /home/cat/Desktop/myProject/jelllyfin_vr_web_plugin
|
||||||
|
|
||||||
|
# 语法检查(全部模块)
|
||||||
|
python3 build.py --check
|
||||||
|
|
||||||
|
# 构建 + 部署(自动构建 → 上传 → 注入 script 标签)
|
||||||
|
# 密码从环境变量读;真实值放在本地 secrets.local.md(不入库)
|
||||||
|
export JF_PW='<NAS 密码>'
|
||||||
|
python3 deploy.py
|
||||||
|
|
||||||
|
# 回滚到原始 index.html
|
||||||
|
python3 deploy.py rollback
|
||||||
|
|
||||||
|
# 部署早期单文件版
|
||||||
|
python3 deploy.py --legacy
|
||||||
|
|
||||||
|
# 重连 Quest 的 ADB + CDP
|
||||||
|
./quest.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
**部署后必须让用户刷新页面**(脚本按页面加载缓存,URL 里的 `?r=<md5前6位>` 是破缓存用的)。
|
||||||
|
|
||||||
|
### 诊断信息
|
||||||
|
|
||||||
|
页面里「VR 播放」面板 → 展开「诊断信息」,或经 CDP 读 `window.__JFVR_STATE__.stats`。关键字段:
|
||||||
|
|
||||||
|
```
|
||||||
|
layersBuilt 实际建立的层
|
||||||
|
fps / targetFrameRate
|
||||||
|
pointer 当前瞄准源与命中坐标
|
||||||
|
inputSources 系统报告的输入源
|
||||||
|
pressAt 上次按压命中/未命中
|
||||||
|
handsReady / handsErr / handsSources / handsFrames
|
||||||
|
subMode / subCanvasSize / subCrop / subTex / subOpacity / subPlacement
|
||||||
|
subLockAngle / subAngleDeg / subTextDeg / subYOverZ ← ADDENDUM 6
|
||||||
|
ctlSubsOn / ctlSubDist / ctlSubAngle / ctlSubLock ← 控制条实际画出的值
|
||||||
|
ctlLock / ctlMode / pressAt / lastActivate ← ADDENDUM 7/7b 锁与唤出判定
|
||||||
|
ctlHidden / ctlVisible / ctlBeamOn / ctlDraws / ctlErr
|
||||||
|
panelPose / grab / grabValue / saved
|
||||||
|
errors[] 最近错误
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. 长期隐患
|
||||||
|
|
||||||
|
1. **注入写在容器内文件**(`/jellyfin/jellyfin-web/index.html`)→
|
||||||
|
`jellyfin/jellyfin:latest` 一升级就没了,需重跑 `deploy.py`。
|
||||||
|
正式版应改用 **JavaScript Injector**(`n00bcodr/jellyfin-plugins` 的 `12/manifest.json`,
|
||||||
|
targetAbi 12.0.0.0)或 **File Transformation**(`IAmParadox27/jellyfin-plugin-file-transformation`,554⭐)。
|
||||||
|
2. **Three.js 从 CDN 加载**(1.27MB)→ 依赖外网。可改为本地托管。
|
||||||
|
3. **Quest 浏览器版本会随系统更新变化** → 能力探测(诊断信息里的能力段)是唯一的自检手段。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. 参考项目
|
||||||
|
|
||||||
|
| 项目 | 形态 | 备注 |
|
||||||
|
|---|---|---|
|
||||||
|
| `JiaruiGe/jellyfinvr`(镜像 `allexxex`) | 单 JS,Three.js | VR180/360/SBS/OU;每帧上传纹理,8K 下是瓶颈 |
|
||||||
|
| `Mix1C/Jellyfin-360VR-Player` | 单 JS,A-Frame | 仅 360°,无 180 |
|
||||||
|
| `Lqlsoftware/Jellyfin-vr-plugin` | C# DLL | 独立播放页 `/VR/Video/{id}/Play` |
|
||||||
|
| `Toastyice/DeoVRDeeplink` | C# DLL + `deovr://` | 可完全自动唤起 Quest 上的 DeoVR(deeplink 走系统,不受 user gesture 限制) |
|
||||||
|
| `IAmParadox27/jellyfin-plugin-file-transformation` | C# 插件 | 正式版注入通道 |
|
||||||
|
| `n00bcodr/jellyfin-plugins` → JavaScript Injector | 插件 | `12/manifest.json` |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 12. 自检清单(改完代码后)
|
||||||
|
|
||||||
|
```
|
||||||
|
[ ] python3 build.py --check 全部模块语法通过
|
||||||
|
[ ] JF_PW=... python3 deploy.py 部署成功,md5 一致
|
||||||
|
[ ] 让用户刷新页面
|
||||||
|
[ ] 读 window.__JFVR_STATE__.stats 确认 layersBuilt / errors 为空
|
||||||
|
[ ] 确认 fps 仍在 ~110+,targetFrameRate 120
|
||||||
|
[ ] 确认面板/字幕/手模型均无异常
|
||||||
|
```
|
||||||
+734
@@ -0,0 +1,734 @@
|
|||||||
|
# JFVR module interfaces (contract)
|
||||||
|
|
||||||
|
Everything lives under a single global namespace `window.__JFVR`, created by
|
||||||
|
`src/00-core.js` before any other module runs. Modules are plain
|
||||||
|
`(function(){ ... })()` IIFEs that attach themselves to `__JFVR`.
|
||||||
|
|
||||||
|
Load order (build script concatenates in this order):
|
||||||
|
|
||||||
|
```
|
||||||
|
00-core.js -> __JFVR core: state, helpers, layer/frame-hook registry
|
||||||
|
10-subtitles.js -> __JFVR.Subtitles
|
||||||
|
20-controls.js -> __JFVR.Controls
|
||||||
|
90-main.js -> boot, DOM scan, XR session lifecycle, settings panel, wiring
|
||||||
|
```
|
||||||
|
|
||||||
|
Nothing may read `document` / `navigator.xr` at load time except `00-core.js`
|
||||||
|
and `90-main.js`. The two module files must only *define* functions and are
|
||||||
|
called later with a session/frame.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 00-core.js — always available as `window.__JFVR`
|
||||||
|
|
||||||
|
```js
|
||||||
|
__JFVR.state // shared mutable state object (see below)
|
||||||
|
__JFVR.log(...args) // console.log with [JFVR] prefix + colour
|
||||||
|
__JFVR.version // string
|
||||||
|
|
||||||
|
// --- helpers ---
|
||||||
|
__JFVR.makeEl(tag, className?, text?) // document.createElement wrapper
|
||||||
|
__JFVR.compileShader(gl, type, source) // throws with the info log on failure
|
||||||
|
__JFVR.invert4(mat4ArrayLike) // -> Float32Array(16)
|
||||||
|
__JFVR.QUAD_VS // vertex shader source: fullscreen triangle,
|
||||||
|
// `in vec2 aPos; out vec2 vUv;`
|
||||||
|
// vUv = vec2(aPos.x*.5+.5, .5 - aPos.y*.5)
|
||||||
|
__JFVR.bindFullscreenTriangle(gl, program) // creates the VBO, enables aPos, returns attribLoc
|
||||||
|
|
||||||
|
// --- XR layer registry (owned by 90-main.js) ---
|
||||||
|
__JFVR.addLayer(layer) // register + push updateRenderState if a session is live
|
||||||
|
__JFVR.removeLayer(layer) // unregister + push
|
||||||
|
__JFVR.addFrameHook(fn) // fn(frame, time, session) every XR frame
|
||||||
|
__JFVR.removeFrameHook(fn)
|
||||||
|
|
||||||
|
// --- capabilities (filled by 00-core.js at boot) ---
|
||||||
|
__JFVR.caps = {
|
||||||
|
secureContext, hasXR, immersiveVR, hasXRMediaBinding, hasEquirectLayer,
|
||||||
|
hasXRWebGLBinding, hasXRHand, hasXRJointSpace, maxTextureSize,
|
||||||
|
hevc: { ... }, ua
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- live handles ---
|
||||||
|
__JFVR.session // current XRSession or null
|
||||||
|
__JFVR.video // current <video> element or null
|
||||||
|
__JFVR.refSpace // 'local' XRReferenceSpace for the running session
|
||||||
|
__JFVR.viewerSpace // 'viewer' XRReferenceSpace (head-locked layers)
|
||||||
|
```
|
||||||
|
|
||||||
|
### `__JFVR.state` fields that modules may use
|
||||||
|
|
||||||
|
```js
|
||||||
|
state.video // <video> or null
|
||||||
|
state.session // XRSession or null
|
||||||
|
state.refSpace // local space
|
||||||
|
state.viewerSpace // viewer space (head-locked)
|
||||||
|
state.projection // '180' | '360'
|
||||||
|
state.stereo // 'mono' | 'sbs' | 'ou'
|
||||||
|
state.subtitles // bool — render subtitles in VR
|
||||||
|
state.subMode // 'hud' (head-locked quad, default)
|
||||||
|
// 'sphere'(on the equirect layer, legacy)
|
||||||
|
state.subAlpha // 'premultiplied' | 'straight'
|
||||||
|
state.subPlacement // { y, z, width } metres, hud mode only
|
||||||
|
state.stats // diagnostics bag, free to add keys
|
||||||
|
state.gl // { refSpace, session } set by main during a session
|
||||||
|
```
|
||||||
|
|
||||||
|
### Layer / frame-hook lifecycle a module must follow
|
||||||
|
|
||||||
|
1. `create(session, glContextOrNull)` — build GL resources + the XR layer,
|
||||||
|
call `__JFVR.addLayer(layer)`, `__JFVR.addFrameHook(hook)`.
|
||||||
|
Return the layer, or `null` when it cannot run (missing canvas, missing API…).
|
||||||
|
**Never throw** — return null and `__JFVR.log()` the reason.
|
||||||
|
2. `hook(frame, time, session)` — per-frame work. Must be cheap; skip work when
|
||||||
|
nothing changed.
|
||||||
|
3. `destroy()` — remove the frame hook, remove the layer, delete GL resources,
|
||||||
|
drop canvas references. Must be safe to call twice and safe to call when
|
||||||
|
`create()` returned null.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10-subtitles.js — `__JFVR.Subtitles`
|
||||||
|
|
||||||
|
Renders Jellyfin's libass subtitle canvas into VR.
|
||||||
|
|
||||||
|
```js
|
||||||
|
__JFVR.Subtitles = {
|
||||||
|
// Called once at boot, before any session.
|
||||||
|
init(): void,
|
||||||
|
|
||||||
|
// BEFORE entering VR. Jellyfin sizes the libass canvas to the on-screen video
|
||||||
|
// box (~1250x625), which is far too small for VR. Blowing up the player box
|
||||||
|
// makes libass re-render bigger. Must be called while NOT in an immersive
|
||||||
|
// session (libass does not recompute its backing store during XR).
|
||||||
|
// Idempotent: calling twice must not double-apply.
|
||||||
|
prepare(): void,
|
||||||
|
|
||||||
|
// Undo prepare(). Safe when prepare() was never called.
|
||||||
|
restore(): void,
|
||||||
|
|
||||||
|
// True when a libass canvas exists and has non-zero size.
|
||||||
|
isReady(): boolean,
|
||||||
|
|
||||||
|
// Current libass canvas size, for diagnostics: { w, h } or null
|
||||||
|
canvasSize(): { w, h } | null,
|
||||||
|
|
||||||
|
// --- session lifecycle ---
|
||||||
|
create(session): XRLayer | null,
|
||||||
|
destroy(): void,
|
||||||
|
|
||||||
|
// --- settings, safe to call before or during a session ---
|
||||||
|
setMode(mode), // 'hud' | 'sphere' | 'off'
|
||||||
|
setAlphaMode(mode), // 'premultiplied' | 'straight'
|
||||||
|
setPlacement({ y, z, width }), // partial update, hud only
|
||||||
|
getMode(): string
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
### HUD mode requirements (the default, and the point of this module)
|
||||||
|
|
||||||
|
- A **head-locked quad layer** in `__JFVR.viewerSpace`
|
||||||
|
(`XRWebGLBinding.createQuadLayer({ space: viewerSpace, layout: 'mono', ... })`).
|
||||||
|
- Source pixels come from `canvas.libassjs-canvas`. That canvas holds a
|
||||||
|
**stereo left-right** image: the left half is the `LeftEye` subtitle, the
|
||||||
|
right half the `RightEye` one. For a mono HUD use **one half only**
|
||||||
|
(the left half is correct for this content).
|
||||||
|
- **Auto-crop to the ink bounding box** so the strip is tight and the text is
|
||||||
|
as large as possible. Recompute the bbox at a low rate (≈4 Hz) on a
|
||||||
|
downscaled copy — never `getImageData` the full-resolution canvas every frame
|
||||||
|
(a 4320x2160 full read is ~37 MB and has already crashed a renderer once).
|
||||||
|
Add hysteresis / smoothing so the strip does not jitter between frames.
|
||||||
|
- Upload only when the source changed (or at most ~30 Hz) — the earlier
|
||||||
|
implementation burned a 4K texture upload every frame.
|
||||||
|
- Target ≈30 px/degree. A 2.0 m wide strip at 1.45 m is ≈66° → texture width
|
||||||
|
2048 is comfortable on Quest 3.
|
||||||
|
- Respect `state.subAlpha`:
|
||||||
|
- `'premultiplied'`: `gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, true)`
|
||||||
|
- `'straight'`: `gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, false)`
|
||||||
|
- Draw with `XRWebGLBinding.getSubImage(layer, frame)` → bind
|
||||||
|
`sub.colorTexture` to a framebuffer → viewport to
|
||||||
|
`sub.colorTextureWidth/Height` → clear to `(0,0,0,0)` → draw.
|
||||||
|
`getSubImage` takes **two** arguments.
|
||||||
|
|
||||||
|
### `prepare()` detail
|
||||||
|
|
||||||
|
The measured behaviour: setting the player box to `7680x3840` makes libass
|
||||||
|
produce a **4320x2160** canvas (it does not go higher). The previous bug was
|
||||||
|
that this was only applied on "open panel" and then restored on session end
|
||||||
|
while the panel stayed open, so a re-entry got a 1250x625 canvas.
|
||||||
|
`prepare()` must therefore be idempotent and re-callable.
|
||||||
|
|
||||||
|
Target DOM: `document.querySelector('.videoPlayerContainer')` and the `<video>`.
|
||||||
|
Save/restore their inline `style.cssText` exactly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 20-controls.js — `__JFVR.Controls`
|
||||||
|
|
||||||
|
A head-locked control strip: progress bar + seek, play/pause, skip ±10 s,
|
||||||
|
subtitle toggle. **Pure 2D canvas rendering + a hit-test function, plus a thin
|
||||||
|
XR quad layer wrapper.**
|
||||||
|
|
||||||
|
```js
|
||||||
|
__JFVR.Controls = {
|
||||||
|
init(): void,
|
||||||
|
|
||||||
|
// --- pure rendering, no WebXR, must be unit-testable in isolation ---
|
||||||
|
// Draws the bar into `ctx` sized `w` x `h` (CSS pixels).
|
||||||
|
// `view` = {
|
||||||
|
// playing, currentTime, duration, buffered, subtitlesOn,
|
||||||
|
// mode: 'collapsed' | 'expanded',
|
||||||
|
// hover: null | 'playpause' | 'back10' | 'fwd10' | 'subs' | 'track'
|
||||||
|
// }
|
||||||
|
draw(ctx, w, h, view): void,
|
||||||
|
|
||||||
|
// Maps a point in normalized panel coords (0..1, origin top-left) to an action.
|
||||||
|
// Returns one of:
|
||||||
|
// { type: 'seek', value: 0..1 }
|
||||||
|
// { type: 'playpause' | 'back10' | 'fwd10' | 'subs' }
|
||||||
|
// null
|
||||||
|
hitTest(nx, ny, w, h): action | null,
|
||||||
|
|
||||||
|
// --- XR lifecycle ---
|
||||||
|
create(session): XRLayer | null,
|
||||||
|
destroy(): void,
|
||||||
|
|
||||||
|
// --- state / input ---
|
||||||
|
setVideo(videoEl, getExtra | null): void, // main provides time/duration/buffered readers
|
||||||
|
onAction(cb): void, // cb(action) — main performs the media op
|
||||||
|
show(autoHideMs?), hide(): void, // collapsed <-> expanded + auto-hide
|
||||||
|
isVisible(): boolean,
|
||||||
|
handleSelect(inputSource, frame): void // called on XR `select`
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
### Panel design requirements
|
||||||
|
|
||||||
|
- **Two visual states**: `collapsed` = a slim progress bar only (≈0.9 m x 0.07 m,
|
||||||
|
centre-bottom of view). `expanded` = progress row + buttons
|
||||||
|
(≈1.1 m x 0.22 m).
|
||||||
|
- Hit targets must be **generous**: the progress bar should be visually ~1 cm
|
||||||
|
tall but hit-testable over ~3–4 cm. Meta's docs warn that the emulated hand
|
||||||
|
target ray for hand tracking is only a coarse forward pointer, so **do not
|
||||||
|
rely on pixel-precise aiming**.
|
||||||
|
- Colours must read on top of arbitrary video: dark translucent pill background,
|
||||||
|
white text, accent `#4ea1ff` (matches the existing in-page panel).
|
||||||
|
- Everything must be drawn from scratch on a canvas — no DOM, no images, no
|
||||||
|
external fonts (use a generic stack: `'system-ui, -apple-system, sans-serif'`).
|
||||||
|
- `draw()` must be deterministic and side-effect free (no timers inside).
|
||||||
|
- `hitTest` must be a pure geometry function of `(nx, ny, w, h)`.
|
||||||
|
|
||||||
|
### Input model (main.js supplies)
|
||||||
|
|
||||||
|
- **Gaze**: the panel is head-locked, so the reticle is the view centre.
|
||||||
|
`main.js` converts that into `(nx, ny)` and calls `hitTest`.
|
||||||
|
- **Controller ray**: `main.js` intersects the controller ray with the panel
|
||||||
|
plane and converts to `(nx, ny)`.
|
||||||
|
- Both end in `handleSelect()` → the action from `hitTest` → `onAction` callback.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 90-main.js — owned by the integrator
|
||||||
|
|
||||||
|
Boot, DOM scan, the in-page `VR 播放` button + settings panel, the XR session
|
||||||
|
lifecycle, the rAF broker (see below), input dispatch, and the settings UI for
|
||||||
|
subtitle mode / alpha mode / placement.
|
||||||
|
|
||||||
|
### rAF broker (already proven, keep it)
|
||||||
|
|
||||||
|
While an immersive XR session is running `window.requestAnimationFrame` is
|
||||||
|
**completely stopped** (measured: 0 callbacks in 6 s), and Jellyfin renders
|
||||||
|
libass subtitles from an rAF loop. `90-main.js` intercepts
|
||||||
|
`requestAnimationFrame`/`cancelAnimationFrame`, holds callbacks while in XR and
|
||||||
|
drives them from `video.requestVideoFrameCallback`. Outside XR it delegates to
|
||||||
|
the native implementation.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 2 — in-VR UX pass (after first on-device test)
|
||||||
|
|
||||||
|
Owner feedback from the first headset test:
|
||||||
|
|
||||||
|
1. **The control bar is permanently visible — wrong.** It must be hidden by
|
||||||
|
default and only appear on interaction, then auto-hide.
|
||||||
|
2. **Hand/controller input has no guide line.** There is no visible ray and no
|
||||||
|
cursor, so the user cannot tell where they are aiming.
|
||||||
|
3. **Subtitles must sit below the visual centre** by default, and both the
|
||||||
|
**position and the opacity must be user-adjustable**.
|
||||||
|
|
||||||
|
## New shared state (owned by 90-main.js, read by modules)
|
||||||
|
|
||||||
|
```js
|
||||||
|
state.subOpacity // 0..1, default 0.85
|
||||||
|
state.subPlacement // { y, z, width } — y is the quad CENTRE, negative = below eye level
|
||||||
|
state.ctlAutoHideMs // default 4000
|
||||||
|
state.pointer // { visible, source, origin:{x,y,z}, dir:{x,y,z}, hit:{nx,ny}|null }
|
||||||
|
// written every XR frame by 90-main.js; modules must treat it as read-only
|
||||||
|
```
|
||||||
|
|
||||||
|
## 10-subtitles.js — additions
|
||||||
|
|
||||||
|
```js
|
||||||
|
setOpacity(a) // 0..1; applied to the XR layer (layer.opacity or the
|
||||||
|
// shader's alpha), NOT by rebuilding the layer
|
||||||
|
getOpacity(): number
|
||||||
|
```
|
||||||
|
- Default `state.subPlacement` becomes `{ y: -0.42, z: -1.45, width: 2.0 }`.
|
||||||
|
- `setPlacement()` must take effect live (update `layer.transform` if the runtime
|
||||||
|
allows mutating it, otherwise rebuild just this layer and keep the frame hook).
|
||||||
|
- Keep the existing tight-crop behaviour and the 4320x2160 canvas preparation.
|
||||||
|
|
||||||
|
## 20-controls.js — additions
|
||||||
|
|
||||||
|
```js
|
||||||
|
setPointer(p) // p as in state.pointer; called every frame by main.
|
||||||
|
// Draw a beam from origin along dir and, when hit != null, a
|
||||||
|
// cursor dot at (hit.nx, hit.ny) on the panel.
|
||||||
|
setOpacity(a)
|
||||||
|
getOpacity(): number
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Default state is HIDDEN.** `create()` must not make the bar visible.
|
||||||
|
- `show(autoHideMs)` → visible + expanded, auto-hides after `autoHideMs`
|
||||||
|
(default `state.ctlAutoHideMs`).
|
||||||
|
- `hide()` → fully hidden (no bar, no beam, no cursor) until `show()` again.
|
||||||
|
- `isVisible()` → true only while shown.
|
||||||
|
- The beam must be a **separate, always-updated visual** that is only visible
|
||||||
|
while the bar is shown. Implement it as a thin quad layer whose transform is
|
||||||
|
placed at `origin + dir * (L/2)` and oriented so its local -Z follows `dir`
|
||||||
|
(build the quaternion from the direction; L = distance to the hit, or a fixed
|
||||||
|
1.2 m when there is no hit). Height ≈ 0.006 m, width = L, with a soft gradient
|
||||||
|
texture (bright at the origin end, fading out) so it reads as a laser.
|
||||||
|
- The cursor: when `hit` is non-null, draw a ring/dot on the control panel canvas
|
||||||
|
at `(hit.nx, hit.ny)` so the user sees exactly what will be activated.
|
||||||
|
- Do not re-upload either texture more than ~30 Hz; skip everything when hidden.
|
||||||
|
- Add stats keys `ctlPointer`, `ctlHidden`, `ctlBeamOn`.
|
||||||
|
|
||||||
|
## Input model change (90-main.js) — unified pointing
|
||||||
|
|
||||||
|
Hand tracking and controllers both expose `targetRaySpace`. Use one code path:
|
||||||
|
|
||||||
|
- For **every** input source with `targetRayMode === 'tracked-pointer'`
|
||||||
|
(controllers *and* hands), intersect its ray with the control quad plane and
|
||||||
|
produce `hit`.
|
||||||
|
- If no source produces a hit, fall back to gaze: `hit = {nx:0.5, ny:0.5}` centred.
|
||||||
|
- `state.pointer.source` records which one won.
|
||||||
|
- Show the beam for the winning source; hide it when the controls are hidden.
|
||||||
|
|
||||||
|
Meta's docs note the hand target ray is an *emulated* forward-pointing ray, so
|
||||||
|
hit targets stay generous — that is already the case.
|
||||||
|
|
||||||
|
### Beam orientation — CORRECTED convention (my first spec was wrong)
|
||||||
|
|
||||||
|
Do **not** map the quad's local `-Z` onto the ray direction. A quad's local `-Z`
|
||||||
|
is its plane **normal**, so that would put the quad's plane perpendicular to the
|
||||||
|
ray and lay its `width` extent across the beam — a thin face-on bar, not a beam.
|
||||||
|
|
||||||
|
Correct:
|
||||||
|
|
||||||
|
```
|
||||||
|
position = origin + dir * (L / 2) // L = hit distance, else 1.2 m
|
||||||
|
width = L (the long axis)
|
||||||
|
height = 0.006
|
||||||
|
local +X -> dir // the beam axis lies IN the plane
|
||||||
|
local -Z -> perpendicular to dir, biased up so the ribbon is never edge-on
|
||||||
|
gradient -> bright at the origin end, fading to transparent at the hit end
|
||||||
|
```
|
||||||
|
|
||||||
|
Assertion for tests: `R(q)·(1,0,0) ≈ dir` and `R(q)·(0,0,-1) ⟂ dir`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 4 — in-VR subtitle placement (grab & drag)
|
||||||
|
|
||||||
|
Owner requirement: adjust the subtitles **without leaving VR**. Chosen model:
|
||||||
|
|
||||||
|
- **Position**: grab the subtitle strip with the laser and drag it in space.
|
||||||
|
- **Reset** button restores the defaults.
|
||||||
|
- **Size** and **opacity** also adjustable in VR.
|
||||||
|
- Values **persist in localStorage** (per browser, no server changes).
|
||||||
|
|
||||||
|
## Grab & drag model (owned by 90-main.js)
|
||||||
|
|
||||||
|
The strip is a head-locked quad in viewer space, so both the ray and the strip
|
||||||
|
live in the same space — no world-space math needed.
|
||||||
|
|
||||||
|
```
|
||||||
|
on grab start:
|
||||||
|
grabActive = true
|
||||||
|
grabOrigin0 = ray.origin // viewer space
|
||||||
|
grabHit0 = ray ∩ plane(z = placement.z)
|
||||||
|
grabPlace0 = { y, z, width }
|
||||||
|
|
||||||
|
per frame while grabbed:
|
||||||
|
P = currentRay ∩ plane(z = grabPlace0.z) // fixed reference plane
|
||||||
|
y' = grabPlace0.y + (P.y - grabHit0.y) // drag up/down
|
||||||
|
z' = grabPlace0.z + (ray.origin.z - grabOrigin0.z) // pull toward / push away
|
||||||
|
clamp y' to [-0.95, -0.05], z' to [-2.6, -0.7]
|
||||||
|
Subtitles.setPlacement({ y: y', z: z', width: current })
|
||||||
|
|
||||||
|
on grab end: clear grab state, persist
|
||||||
|
```
|
||||||
|
|
||||||
|
Rationale for `z`: in viewer space forward is `-Z`, so pulling the hand back
|
||||||
|
(increasing `origin.z`) makes `z'` less negative, i.e. the strip comes **closer**
|
||||||
|
— which matches how pulling a real object behaves.
|
||||||
|
|
||||||
|
Grab **must not** steal the normal control-bar interaction: only start a grab when
|
||||||
|
the ray actually hits the subtitle quad; otherwise fall through to the bar.
|
||||||
|
|
||||||
|
## Input: press **and release** are now needed
|
||||||
|
|
||||||
|
The existing activation path in `90-main.js` only detects a rising edge. Grabbing
|
||||||
|
needs the held state too, so track per input source:
|
||||||
|
|
||||||
|
- `selectstart` / `selectend` session events, **and**
|
||||||
|
- `inputSource.gamepad` button press/release (hand pinch; see ADDENDUM 3)
|
||||||
|
|
||||||
|
Both feed one `pointerDown(src)` / `pointerUp(src)` pair, deduped the same way.
|
||||||
|
|
||||||
|
## 10-subtitles.js — additions
|
||||||
|
|
||||||
|
```js
|
||||||
|
getLayer(): XRLayer | null // so main can ray-cast against the strip
|
||||||
|
resetPlacement(): void // back to { y: -0.42, z: -1.45, width: 2.0 }
|
||||||
|
```
|
||||||
|
- `width` **is** the size control: `setPlacement({ width })` already rescales the
|
||||||
|
quad while keeping the angular text height consistent with the texture aspect.
|
||||||
|
- All of `setPlacement` / `setOpacity` / `resetPlacement` stay live during a session.
|
||||||
|
|
||||||
|
## 20-controls.js — additions
|
||||||
|
|
||||||
|
The expanded control bar gains a subtitle row with large −/+ buttons:
|
||||||
|
|
||||||
|
| Action returned by `hitTest` | Meaning |
|
||||||
|
| --- | --- |
|
||||||
|
| `sub_smaller` / `sub_bigger` | width by ±0.15 m, clamped to [0.8, 4.0] |
|
||||||
|
| `sub_opaque_down` / `sub_opaque_up` | opacity by ∓0.05, clamped to [0.1, 1.0] |
|
||||||
|
| `sub_reset` | restore all subtitle defaults |
|
||||||
|
|
||||||
|
- Add `view.sub = { width, opacity }` so `draw()` can render the current values.
|
||||||
|
- Keep hit targets generous; the rows may overlap visually but must not overlap
|
||||||
|
in hit-test order (buttons before track).
|
||||||
|
- Add a new expanded height that fits the extra row.
|
||||||
|
|
||||||
|
## 90-main.js — persistence
|
||||||
|
|
||||||
|
```
|
||||||
|
localStorage key: 'jfvr.settings.v1'
|
||||||
|
{ subPlacement: {y,z,width}, subOpacity, subMode, subAlpha,
|
||||||
|
projection, stereo, ctlAutoHideMs }
|
||||||
|
```
|
||||||
|
Load at boot (merge over defaults, validate ranges), save debounced ~400 ms after
|
||||||
|
any change. A corrupt/absent value must fall back to defaults silently.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 5 — gaze pointing + lazy-follow panel + panel size
|
||||||
|
|
||||||
|
## Why (measured on device)
|
||||||
|
|
||||||
|
The hand's `targetRaySpace` is an **emulated** ray (Meta documents this). Measured
|
||||||
|
on a real Quest 3 while the panel was straight ahead:
|
||||||
|
|
||||||
|
```
|
||||||
|
hand:right origin=(-0.317,-0.054,-0.248) dir=(-0.874, 0.461, -0.154)
|
||||||
|
^ 87 % sideways, 15 % forward
|
||||||
|
```
|
||||||
|
|
||||||
|
Aiming with it lands somewhere the user is not looking. Gaze + pinch is Meta's
|
||||||
|
documented recommendation for hands and needs no ray accuracy at all.
|
||||||
|
|
||||||
|
## Changes
|
||||||
|
|
||||||
|
| # | Change |
|
||||||
|
| --- | --- |
|
||||||
|
| 1 | Pointer rays are computed in **LOCAL space** (`__JFVR.refSpace`), not viewer space. |
|
||||||
|
| 2 | Hands aim with the **gaze ray** (head forward in local space). No beam - a guide line from the hand would be misleading because the hand is not aiming. The on-panel reticle is the indicator. |
|
||||||
|
| 3 | Controllers keep their own `targetRaySpace` ray **and** the laser beam; a controller ray is accurate and a laser is expected. Selected by `pointer.beam`. |
|
||||||
|
| 4 | The control panel layer's **space is `refSpace`** and holds a stable world pose. A head-locked panel would always put the view centre on its own centre, so no button could ever be targeted by gaze. |
|
||||||
|
| 5 | 90-main.js drives a **lazy follow**: it only re-centres the panel once the head has turned > 26 deg away from it, easing with a 0.45 s time constant. |
|
||||||
|
| 6 | `subtitles` is **no longer persisted**. Persisting it meant a session where the user toggled subtitles off came back with the layer never created - which looked like a load failure. Always ON at boot; the toggle is per-session. |
|
||||||
|
| 7 | Subtitle **distance** range extended: the near clamp is now **0.5 m** (was 0.7 m). |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 6 — subtitle depth vs size decoupling (angular-size lock)
|
||||||
|
|
||||||
|
## Why (owner requirement)
|
||||||
|
|
||||||
|
> 当前 vr 视频中可能出现 vr 空间中物体可能距离我很近,如果字幕还很远,我将无法在
|
||||||
|
> 看清物体的同时看清字幕。我希望可以让字幕的空间距离和物体的距离尽量接近。
|
||||||
|
|
||||||
|
Eye tracking is **not** exposed to WebXR on Quest (HANDOFF §4), so the strip's depth
|
||||||
|
cannot follow the user's gaze automatically. What matters instead is that the DEPTH
|
||||||
|
control must not change the strip's apparent size: otherwise pulling the subtitles to
|
||||||
|
a nearby object makes the text huge (and, with the head-locked quad, drops it out of
|
||||||
|
the lower view). See HANDOFF §8 for the literature this is based on.
|
||||||
|
|
||||||
|
## Model
|
||||||
|
|
||||||
|
`state.subLockAngle` (default `true`) + `state.subAngleDeg` (default `69.2`).
|
||||||
|
|
||||||
|
While **locked**, a depth change holds TWO things:
|
||||||
|
|
||||||
|
```
|
||||||
|
width = 2 * |z| * tan(subAngleDeg / 2) angular width (theta)
|
||||||
|
y = (y0 / z0) * z angular height (the y/z ratio)
|
||||||
|
```
|
||||||
|
|
||||||
|
so the strip keeps both its angular size and its screen position — only depth
|
||||||
|
changes. The default pair is bit-for-bit the pre-ADDENDUM-6 look (2.0 m at 1.45 m →
|
||||||
|
69.2°), so nothing moves for anyone who does not touch the control.
|
||||||
|
|
||||||
|
While **unlocked** everything behaves exactly as at ADDENDUM 4: `width` and `z` are
|
||||||
|
independent ("pull closer = bigger").
|
||||||
|
|
||||||
|
Ranges: `theta` ∈ [20, 120]°, `z` ∈ [-2.6, -0.35] m (near clamp moved from 0.5 m to
|
||||||
|
0.35 m — the whole point is matching very close objects). Under the lock the width
|
||||||
|
floor drops to 0.05 m (the angle clamp is the real limit); free mode keeps 0.2 m.
|
||||||
|
|
||||||
|
## 10-subtitles.js — API additions
|
||||||
|
|
||||||
|
```js
|
||||||
|
setLockAngle(on) / getLockAngle(): boolean
|
||||||
|
setAngleDeg(deg) / getAngleDeg(): number // locked angular width, degrees
|
||||||
|
getPlacement(): { y, z, width } // the resolved, clamped triple
|
||||||
|
```
|
||||||
|
|
||||||
|
- `setPlacement({ y, z, width })` under the lock:
|
||||||
|
- `z` without `y` → depth-only: `y` is re-derived from the held y/z ratio and
|
||||||
|
`width` from `subAngleDeg`;
|
||||||
|
- `width` without `z` → size change: `subAngleDeg` is re-derived from the requested
|
||||||
|
width at the current depth (the derived width then wins);
|
||||||
|
- `y` + `z` (grab & drag) → the caller's `y` wins, only `width` is derived.
|
||||||
|
- `resetPlacement()` restores `{ y:-0.42, z:-1.45, width:2.0 }` **and**
|
||||||
|
`subAngleDeg = 69.2`; the lock flag is left alone.
|
||||||
|
- New diagnostics: `stats.subAngleDeg`, `stats.subLockAngle`, `stats.subYOverZ`, and
|
||||||
|
`stats.subTextDeg` — the ACTUAL angular height of the drawn subtitle ink, in
|
||||||
|
degrees. arXiv 2004.01545 measures preferred angular size (character height) at
|
||||||
|
~16–18 dmm ≈ 0.9–1.0° for medium/long text and ~32–40 dmm ≈ 1.8–2.3° for
|
||||||
|
titles/headlines; Google's dmm is 1 mm of character height at 1 m. Use
|
||||||
|
`subTextDeg` to pick a `subAngleDeg` default from real measurements later; the
|
||||||
|
shipped default is deliberately the on-device-proven look.
|
||||||
|
|
||||||
|
## 20-controls.js — subtitle row is now NINE buttons
|
||||||
|
|
||||||
|
```
|
||||||
|
sub_up sub_down sub_near sub_far sub_smaller sub_bigger sub_opaque_down sub_opaque_up sub_reset
|
||||||
|
上 下 近 远 小 大 暗 亮 复位
|
||||||
|
```
|
||||||
|
|
||||||
|
- The vertical-position pair (up/down) and the depth pair (near/far) are the
|
||||||
|
ADDENDUM-6 additions.
|
||||||
|
- `view.sub` gains `angleDeg`, `lock` and `dist` (= `|z|`). The caption ALWAYS
|
||||||
|
leads with the depth in metres — `字幕 距1.45m · 69°` while locked,
|
||||||
|
`字幕 距1.45m · 2.0m` while free — because the depth is the number the user has
|
||||||
|
to match against the object being watched. Opacity stays right-aligned.
|
||||||
|
- **`readSubtitles()` bug fixed**: it only read the module-local `S.subtitlesOn`,
|
||||||
|
which `90-main.js` never set (it calls `setVideo(video)` with no `getExtra`), so
|
||||||
|
the CC button always rendered as OFF. It now falls back to the shared
|
||||||
|
`state.subtitles`. The button also got an unambiguous on/off treatment: an
|
||||||
|
accent pill + `字幕开` when on, a slashed CC glyph + `字幕关` when off.
|
||||||
|
- Hit pills: pitch 0.10, half width 0.052 (a hair of overlap → no dead zone);
|
||||||
|
visual width 0.092; caption font 0.048·h, button label font 0.040·h. The bar
|
||||||
|
itself is unchanged (`PANEL_M.expanded` = 0.72 × 0.21 m, canvas 1440 × 420 px).
|
||||||
|
|
||||||
|
## 90-main.js
|
||||||
|
|
||||||
|
- Persists `subLockAngle` / `subAngleDeg` in `jfvr.settings.v1`; while locked the
|
||||||
|
stored `subPlacement.width` is re-derived on load, so `subAngleDeg` + `z` is what
|
||||||
|
restores the user's look.
|
||||||
|
- In-page panel: a `距离与大小` toggle (锁定视角尺寸 / 自由), and a `视角宽度`
|
||||||
|
slider in degrees shown while locked / the old `宽度` slider in metres while free.
|
||||||
|
All sliders re-read their value whenever a setter runs, so VR-bar and grab changes
|
||||||
|
show up in an already-open panel.
|
||||||
|
- `sub_up` / `sub_down` step `y` by ±0.04 m. `sub_near` / `sub_far` step `z` by
|
||||||
|
±0.15 m. `sub_smaller` / `sub_bigger` step `subAngleDeg` by ∓5° while locked,
|
||||||
|
else `width` by ∓0.15 m.
|
||||||
|
- Grab & drag: while locked, a depth drag also carries `y` by the grab-start y/z
|
||||||
|
ratio (vertical hand motion still adds on top), so the strip cannot slide out of
|
||||||
|
the lower view when it is pulled close.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 7 — summon gating, lock badge, pre-VR panel trim
|
||||||
|
|
||||||
|
Owner feedback after the ADDENDUM 6 pass:
|
||||||
|
|
||||||
|
> 控制面版的唤出太灵敏了,只有当指示线指向面板附近时,面板再显示出来即可;在 vr 空间
|
||||||
|
> 右侧边缘增加一个锁的图标,当选中上锁后,面板不会启动;在刚唤出面板时,0.5s 内的操作
|
||||||
|
> 应该忽略,否则可能误触。在进入"vr 模式"时,字幕相关有很多滑块配置,移除哪些配置,
|
||||||
|
> 在没有在真实 vr 空间中的调整,都是不准确的,没必要在进入前指定。视角宽度保持默认即可。
|
||||||
|
> 字幕相关只保留开启和关闭即可。"vr 播放"这个按钮,不要挡住原有播放器的选项,现在似乎
|
||||||
|
> 挡住了原播放器的全屏按钮;手部指向选项直接移除吧,没人会只使用食指作为指向。
|
||||||
|
|
||||||
|
## 1. Summon gating (90-main.js)
|
||||||
|
|
||||||
|
There were **two** independent reasons the bar appeared on *every* pinch, and both
|
||||||
|
are gone:
|
||||||
|
|
||||||
|
| Was | Now |
|
||||||
|
| --- | --- |
|
||||||
|
| `onSelectStart` called `Controls.show()` unconditionally | listener removed; summoning is decided in `pointerDown()` from the actual aim |
|
||||||
|
| `pointerDown()` called `Controls.show()` before checking the hit | `show()` is called only when the ray lands on / near the bar |
|
||||||
|
|
||||||
|
```
|
||||||
|
bar hidden + aim ON the bar -> show(), record barShownAt, return (the summoning press never clicks)
|
||||||
|
bar hidden + aim NEAR the bar-> same (padded window, see below)
|
||||||
|
bar hidden + aim elsewhere -> nothing
|
||||||
|
bar hidden + locked -> nothing
|
||||||
|
bar shown + < PANEL_ARM_MS(500) since it appeared -> ignore
|
||||||
|
bar shown + aim ON the bar -> show() (extends auto-hide) + handleSelect
|
||||||
|
```
|
||||||
|
|
||||||
|
Barely-visible: the summoning press is *consumed* — it does not also press a button,
|
||||||
|
and for 500 ms after the bar appears every activation is ignored. Otherwise the hand
|
||||||
|
travelling onto the bar right after the summon fires a random button.
|
||||||
|
|
||||||
|
The summon window is deliberately wider than the button window: `pointer.hit` (used
|
||||||
|
for the cursor and for button hits) uses `panelHit(..., 0.25, 0.6)`, while summoning
|
||||||
|
falls back to `panelHit(..., 0.5, 0.9)`. A ray that merely grazes the bar can still
|
||||||
|
bring it up; the 500 ms arm is what protects against mis-clicks.
|
||||||
|
|
||||||
|
Stats: `stats.pressAt` now reads `SUMMON (armed 500ms)`, `IGNORED (arming)`,
|
||||||
|
`MISS (not aiming at the panel)` or `MISS (panel locked)`.
|
||||||
|
|
||||||
|
## 2. Lock badge (20-controls.js + 90-main.js)
|
||||||
|
|
||||||
|
An **always-visible** small quad (`LOCK_X/Y/Z = 0.86 / -0.34 / -1.45`, `LOCK_M = 0.075`,
|
||||||
|
texture 128²) placed in the **same `refSpace`** as the control bar — to the right of
|
||||||
|
it. While `state.ctlLock` is true, nothing summons the bar; the badge stays visible
|
||||||
|
and hit-testable so it can be unlocked again.
|
||||||
|
|
||||||
|
Why `refSpace` and not a head-locked HUD corner: on this setup the only pointing
|
||||||
|
device is the hand's ray, and a head-locked badge could never be aimed at. "Right-hand
|
||||||
|
edge of the VR space" is therefore implemented as "right of the control bar, in the
|
||||||
|
same world-locked space".
|
||||||
|
|
||||||
|
- Unlocked: dim (60 % white) padlock with the shackle open.
|
||||||
|
- Locked: accent (`#4ea1ff`) filled padlock, shackle closed, keyhole punched out.
|
||||||
|
- Hit window: `panelHit(..., 0.6, 0.6)` ≈ 6.5° across at 1.45 m — the badge itself is
|
||||||
|
only ~3°, so it is easy to hit with a coarse hand ray.
|
||||||
|
- The lock is **per VR session**: `enterVR()` sets `ctlLock = false`, `onSessionEnd()`
|
||||||
|
resets it too. A stuck lock would look exactly like a broken plugin, so it is never
|
||||||
|
restored from `localStorage`.
|
||||||
|
|
||||||
|
New API:
|
||||||
|
|
||||||
|
```js
|
||||||
|
Controls.getLockLayer(): XRLayer | null
|
||||||
|
Controls.setLock(on) / getLock(): boolean
|
||||||
|
```
|
||||||
|
|
||||||
|
Stats: `stats.ctlLock`.
|
||||||
|
|
||||||
|
## 3. Pre-VR settings panel trimmed (90-main.js)
|
||||||
|
|
||||||
|
Sliders set on a flat screen do not transfer to a headset, so the only subtitle
|
||||||
|
setting left before entering VR is **开 / 关**:
|
||||||
|
|
||||||
|
| Removed | Why |
|
||||||
|
| --- | --- |
|
||||||
|
| 模式 (hud/sphere/off) | legacy A/B debug toggle |
|
||||||
|
| 透明合成, 不透明度 | judge-able only in VR |
|
||||||
|
| 垂直位置, 距离, 宽度 | now fully replaced by the in-VR row |
|
||||||
|
| 距离与大小 (lock toggle) | the lock is always ON; it is the whole point |
|
||||||
|
| 视角宽度 | keeps its default; in-VR `小/大` is the control |
|
||||||
|
| 手部指向 | the index-ray mode was removed outright |
|
||||||
|
|
||||||
|
`视角宽度` / `subAngleDeg` still **persists** — the in-VR `小/大` buttons write it,
|
||||||
|
and the panel just no longer offers a second, less accurate way to set it. Same for
|
||||||
|
placement / opacity: set in VR, remembered next time.
|
||||||
|
|
||||||
|
Removed with it: `setSubModeLive()`, `setSubLockLive()`, `handRay()` and every
|
||||||
|
`handRayMode` read/write.
|
||||||
|
|
||||||
|
## 4. "VR 播放" button position (90-main.js CSS)
|
||||||
|
|
||||||
|
`.jfvr-btn` moved from `bottom: 24px` to `bottom: 120px` — at the viewport's
|
||||||
|
bottom-right it sat on top of Jellyfin's own OSD, covering its fullscreen button.
|
||||||
|
`.jfvr-panel` moved to `bottom: 180px` and `max-height: calc(100vh - 200px)` so it
|
||||||
|
still stacks above the button.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 7b — the padlock is a PANEL BUTTON, not its own layer
|
||||||
|
|
||||||
|
## What was wrong with ADDENDUM 7's badge
|
||||||
|
|
||||||
|
On device the badge could not be selected while the panel was shown, but could be
|
||||||
|
while it was hidden. ADDENDUM 7 had given the padlock **its own quad layer** parked at
|
||||||
|
`(0.45, -0.34, -1.45)` — 0.45 m to the right of the panel centre, i.e. 0.09 m to the
|
||||||
|
right of the panel's own right edge.
|
||||||
|
|
||||||
|
Aiming is the binding constraint and it was underestimated: the only pointing device
|
||||||
|
here is the hand ray, and the cursor's own acceptance window (`panelHit` padding
|
||||||
|
0.25/0.6 on a 0.72 m panel) reaches `x ≈ 0.45` — the panel's right edge. The badge was
|
||||||
|
**outside every reachable ray**. Worse, its hit test ran on the *panel's* layer
|
||||||
|
geometry, so whether it fired depended on where the panel happened to be, which is
|
||||||
|
exactly the "works when hidden, not when shown" report.
|
||||||
|
|
||||||
|
## What it is now
|
||||||
|
|
||||||
|
An ordinary button on the panel canvas — the `lock` action, fifth in the media row:
|
||||||
|
|
||||||
|
```
|
||||||
|
expanded: [<<<] [>||] [>>>] [CC] [padlock]
|
||||||
|
0.18 0.36 0.54 0.72 0.90 (hit pill half width 0.092)
|
||||||
|
```
|
||||||
|
|
||||||
|
It travels the same code path as every button that already works: cursor → `hitTest`
|
||||||
|
→ `handleSelect` → `onAction` → `90-main.js handleControlAction('lock')`.
|
||||||
|
|
||||||
|
## The "locked" panel mode
|
||||||
|
|
||||||
|
`show()` picks the mode from the lock state, so locking answers "what does a summon
|
||||||
|
look like now?" with a third mode:
|
||||||
|
|
||||||
|
| mode | size (m) | canvas | drawn / hit-testable |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `expanded` | 0.72 × 0.21 | 1440 × 420 | everything |
|
||||||
|
| `locked` | 0.26 × 0.10 | 520 × 200 | padlock + 解锁 label only |
|
||||||
|
| `collapsed` | 0.60 × 0.05 | 1200 × 100 | (legacy; no longer entered) |
|
||||||
|
|
||||||
|
`hitTest(nx, ny, w, h, modeHint)` now takes the mode explicitly, because "collapsed" (a
|
||||||
|
thin strip) and "locked" (a small square-ish pill) share no aspect-ratio invariant.
|
||||||
|
`handleSelect` passes `S.mode`; the aspect-ratio `modeFor()` remains only as a fallback
|
||||||
|
for external callers.
|
||||||
|
|
||||||
|
`hide()` **no longer resets the mode**. The layer therefore keeps the size it will be
|
||||||
|
shown at, so the summon window that `90-main.js` ray-casts is the full panel rect
|
||||||
|
(0.72 × 0.21 unlocked, 0.26 × 0.10 locked) instead of a 5 cm strip — and re-showing
|
||||||
|
never needs a resize pop.
|
||||||
|
|
||||||
|
## Lifecycle (all of it in 90-main.js `toggleControlsLock`)
|
||||||
|
|
||||||
|
```
|
||||||
|
lock: S.ctlLock = true; Controls.setLock(true) -> mode 'locked'
|
||||||
|
Controls.show() -> padlock-only panel, auto-hide restarts
|
||||||
|
barShownAt = now
|
||||||
|
unlock: S.ctlLock = false; Controls.setLock(false) -> mode 'expanded'
|
||||||
|
Controls.show() -> full panel
|
||||||
|
barShownAt = now -> 0.5 s ignored, so the unlocking
|
||||||
|
press cannot land on a button
|
||||||
|
```
|
||||||
|
|
||||||
|
Summoning while locked is allowed (it is the only way back out): `pointerDown` no longer
|
||||||
|
early-returns on `S.ctlLock`, and the summon shows the padlock panel alone.
|
||||||
|
|
||||||
|
## Also fixed
|
||||||
|
|
||||||
|
`handleSelect()` used to fall back to `(nx, ny) = (0.5, 0.5)` when there was no pointer
|
||||||
|
hit and no reticle — which silently pressed whatever button lives at the centre. It now
|
||||||
|
returns `null` and performs nothing.
|
||||||
|
|
||||||
|
## API
|
||||||
|
|
||||||
|
```js
|
||||||
|
Controls.setLock(on) / getLock(): boolean // setLock also swaps the panel in place
|
||||||
|
Controls.hitTest(nx, ny, w, h, mode?) // mode: 'expanded' | 'locked' | 'collapsed'
|
||||||
|
Controls.PANEL_M / PANEL_PX // now include .locked
|
||||||
|
```
|
||||||
|
|
||||||
|
Stats: `stats.ctlMode` (`expanded` | `locked`), `stats.ctlLock`,
|
||||||
|
`stats.pressAt` (`SUMMON (locked panel)`, `LOCKED`/`UNLOCKED (armed 500ms)`, …).
|
||||||
@@ -0,0 +1,813 @@
|
|||||||
|
# Jellyfin Web VR — Quest 3 / WebXR MVP
|
||||||
|
|
||||||
|
给 Jellyfin Web 播放页注入一个「VR 播放」按钮,在 Meta Quest 3 浏览器里直接进入
|
||||||
|
VR180 / VR360 沉浸播放,不用再手动切全屏 + 选投影。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 当前部署状态
|
||||||
|
|
||||||
|
已注入到 `192.168.123.199` 上的 Jellyfin 容器 `Jellyfin`(当前版本 `mvp-7616c9`,`NS.version = hud-5`,含 ADDENDUM 6 + 7 + 7b):
|
||||||
|
|
||||||
|
| 项 | 值 |
|
||||||
|
|---|---|
|
||||||
|
| 脚本 | `/jellyfin/jellyfin-web/jfvr-mvp.js`(容器内) |
|
||||||
|
| 注入点 | `/jellyfin/jellyfin-web/index.html` 的 `</body>` 之前 |
|
||||||
|
| 备份 | `/vol2/jellyfin/config/_vr_backup/index.html.orig` |
|
||||||
|
| 两个入口都生效 | `http://192.168.123.199:8096/web/` 与 `https://jellyfin.cat-shark.xyz/web/` |
|
||||||
|
|
||||||
|
> ⚠️ 注入写在**容器内文件**里,`jellyfin/jellyfin:latest` 一升级就没了,重跑 `deploy.py` 即可。
|
||||||
|
> 正式版应改用 **JavaScript Injector 插件**(`n00bcodr/jellyfin-plugins` 的 `12/manifest.json`,targetAbi 12.0.0.0)
|
||||||
|
> 或 **File Transformation**,这样升级不丢、可开关。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Quest 3 实测数据(已用 ADB + CDP 在真机验证)
|
||||||
|
|
||||||
|
### 2.0 设备与能力
|
||||||
|
|
||||||
|
```
|
||||||
|
Quest 3 Android 14 (SDK 34) build UP1A.231005.007.A1
|
||||||
|
Quest Browser 152.1.0.34.52.1079628208 (Chromium 152.0.7977.83)
|
||||||
|
UA Mozilla/5.0 (X11; Linux x86_64; Quest 3) ... Chrome/152 ... VR Safari/537.36
|
||||||
|
```
|
||||||
|
|
||||||
|
| 检测项 | `http://192.168.123.199:8096` | `https://jellyfin.cat-shark.xyz` |
|
||||||
|
|---|---|---|
|
||||||
|
| `isSecureContext` | false | **true** |
|
||||||
|
| `navigator.xr` | false | **true** |
|
||||||
|
| `immersive-vr` | — | **true** |
|
||||||
|
| **`XRMediaBinding.createEquirectLayer`** | — | **true** ← 首选路径可用 |
|
||||||
|
| `XRWebGLBinding` | — | true ← 回退路径可用 |
|
||||||
|
| `MAX_TEXTURE_SIZE` | 8192 | 8192 |
|
||||||
|
| `MAX_RENDERBUFFER_SIZE` | 16384 | 16384 |
|
||||||
|
| 8K HEVC `decodingInfo` | supported / powerEfficient | **supported:true, powerEfficient:true** |
|
||||||
|
|
||||||
|
结论:**HTTPS 下 Quest 浏览器提供全部所需 WebXR 能力,8K HEVC 走硬件解码。**
|
||||||
|
|
||||||
|
### 2.1 网络实测:HTTPS 域名在 Quest 上就是内网直连
|
||||||
|
|
||||||
|
```
|
||||||
|
Quest 3 IPv6 : 2409:8a14:44a:e360:70b9:446e:d5f7:10a7/64
|
||||||
|
NAS IPv6 : 2409:8a14:44a:e360:520:8aa2:7736:4f12/64 ← 同一 /64
|
||||||
|
|
||||||
|
domain AAAA → NAS 自己的局域网 IPv6
|
||||||
|
服务端连接表 : [NAS]:443 ← [Quest]:35860 (ESTAB, IPv6)
|
||||||
|
```
|
||||||
|
|
||||||
|
60 MB 文件在 Quest 浏览器上实测:
|
||||||
|
|
||||||
|
| Run | 吞吐 | TTFB |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | 450 Mbps | 11 ms |
|
||||||
|
| 2 | 620 Mbps | 7 ms |
|
||||||
|
| 3 | **702 Mbps** | 12 ms |
|
||||||
|
|
||||||
|
对比:从局域网 PC 用 curl 走 IPv6 只有 4.29 MB/s(34 Mbps),走 IPv4 阿里云中转只有 94 KB/s。
|
||||||
|
|
||||||
|
> 慢的根因:NAS 上跑着 **`npc` 内网穿透客户端**,域名的 **A 记录 `39.105.149.197` 是阿里云上的 nps 服务端**,
|
||||||
|
> 走 IPv4 就是 NAS → 阿里云 → 你。**走 IPv6 则完全不出内网。**
|
||||||
|
>
|
||||||
|
> 所以 **不需要为了速度退回 HTTP**。8K 片源码率仅 21 Mbps,702 Mbps 有 30 倍余量。
|
||||||
|
|
||||||
|
### 2.2 如果仍想用 `192.168.123.199:8096`
|
||||||
|
|
||||||
|
WebXR 要求「安全上下文」。实测结果:
|
||||||
|
|
||||||
|
| 入口 | `isSecureContext` | `navigator.xr` | WebXR |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `http://192.168.123.199:8096` | **false** | **false** | ❌ 需先做 §2.1 |
|
||||||
|
| `https://jellyfin.cat-shark.xyz` | true | true | ✅ 直接可用 |
|
||||||
|
|
||||||
|
用内网 HTTP 地址 —— Quest 浏览器开 flags
|
||||||
|
|
||||||
|
Meta Quest 2/3/3S(**OS 79+**)浏览器支持在 HTTP 模式下使用 WebXR,需显式授权:
|
||||||
|
|
||||||
|
1. Quest 浏览器地址栏输入 `chrome://flags`
|
||||||
|
2. 搜索 `insecure`
|
||||||
|
3. 启用 **`unsafely-treat-insecure-origin-as-secure`**
|
||||||
|
4. 在下方输入框填:`http://192.168.123.199:8096`
|
||||||
|
(**必须带 `http://` 和端口号**;多个用逗号分隔)
|
||||||
|
5. 点空白处失焦 → 屏幕底部出现 **Relaunch** → 点它
|
||||||
|
6. 回到 `chrome://flags` 确认 flag 仍启用且 URL 已保存
|
||||||
|
|
||||||
|
代价:这台设备专属、浏览器更新后可能需要重设。
|
||||||
|
|
||||||
|
### 2.3 兜底:让局域网内解析到内网 IPv4
|
||||||
|
|
||||||
|
如果某天 IPv6 不可用,且 Quest 落到了 IPv4(阿里云中转)导致慢,最一劳永逸的修法是在
|
||||||
|
**OpenWrt 路由器(192.168.123.1)** 上加一条静态解析,让局域网内所有设备都解析到内网 IP:
|
||||||
|
|
||||||
|
- LuCI:`网络 → DHCP/DNS → 一般设置 → 地址`,添加 `/jellyfin.cat-shark.xyz/192.168.123.199`
|
||||||
|
- 或 SSH 到路由器:
|
||||||
|
```sh
|
||||||
|
uci add_list dhcp.@dnsmasq[0].address='/jellyfin.cat-shark.xyz/192.168.123.199'
|
||||||
|
uci commit dhcp && /etc/init.d/dnsmasq restart
|
||||||
|
```
|
||||||
|
|
||||||
|
之后 `https://jellyfin.cat-shark.xyz` 在局域网内全程走内网,证书依然有效,所有设备(不只 Quest)都受益。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 实操:如何在真机上验证(ADB + CDP)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ADB=/opt/SideQuest/resources/platform-tools/adb # SideQuest 自带的 adb
|
||||||
|
|
||||||
|
# 1) 看设备(SideQuest 的无线 ADB)
|
||||||
|
$ADB devices -l
|
||||||
|
|
||||||
|
# 2) 把 Quest 浏览器的 DevTools 端口转发出来
|
||||||
|
$ADB forward tcp:9222 localabstract:chrome_devtools_remote
|
||||||
|
curl -s http://127.0.0.1:9222/json/version
|
||||||
|
|
||||||
|
# 3) 让 Quest 打开某个页面
|
||||||
|
$ADB shell am start -a android.intent.action.VIEW \
|
||||||
|
-d "http://192.168.123.199:8096/web/index.html" \
|
||||||
|
-n com.oculus.browser/.OculusLauncherActivity
|
||||||
|
|
||||||
|
# 4) 截图 / OCR
|
||||||
|
$ADB exec-out screencap -p > /tmp/quest/shot.png # 4128x2208
|
||||||
|
tesseract /tmp/quest/shot.png - --psm 6
|
||||||
|
```
|
||||||
|
|
||||||
|
拿到 `9222` 后可以直接用 `agent_browser --cdp http://127.0.0.1:9222` 接管 Quest 浏览器,
|
||||||
|
在页面上跑 `Runtime.evaluate`、读网络、发输入事件。
|
||||||
|
|
||||||
|
> 注:Quest 浏览器**不把 `console.log` 转发到 logcat**,所以诊断要走 CDP `Runtime.evaluate` 或页面 UI。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 使用步骤(Quest 3 里)
|
||||||
|
|
||||||
|
1. 打开 Jellyfin(§2.1 或 §2.2 任一入口)
|
||||||
|
2. 进媒体库,播放一个 VR 视频
|
||||||
|
3. 右下角出现蓝色 **「VR 播放」** 按钮 → 点它
|
||||||
|
4. 面板里确认 **投影**(180°/360°)和 **立体**(单目/左右/上下)
|
||||||
|
- 自动按文件名 + 分辨率判断;**8192×4096(2:1)会默认选 180° + 左右**
|
||||||
|
5. 点 **「进入 VR」** → 进入沉浸模式;摘头显即退出
|
||||||
|
|
||||||
|
进不去时展开面板里的 **「诊断信息」**,把内容贴回来。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 在 VR 里修掉的两个关键缺陷
|
||||||
|
|
||||||
|
真机(Quest 3 + CDP)实测发现并修复:
|
||||||
|
|
||||||
|
### 5.1 卡顿 —— 帧率不匹配(3:2 pulldown)
|
||||||
|
|
||||||
|
| 项 | 实测值 |
|
||||||
|
|---|---|
|
||||||
|
| 视频帧率 | **59.94 fps** |
|
||||||
|
| XR 会话默认 | **90 Hz**(实测 87.9 fps) |
|
||||||
|
| `session.supportedFrameRates` | `[72, 80, 90, 120]` |
|
||||||
|
| `session.frameRate`(初始) | **0**(未指定) |
|
||||||
|
|
||||||
|
90 ÷ 59.94 = 1.5 → 视频帧在显示上被交替重复 1 次、2 次 —— 经典 3:2 pulldown 抖动。
|
||||||
|
2D 播放时系统会自动匹配刷新率,所以「web 端流畅、进 VR 就卡」。
|
||||||
|
|
||||||
|
**修复**:进 VR 后调用 `session.updateTargetFrameRate(120)`。120 ÷ 60 = 2,每帧均匀显示两次。
|
||||||
|
|
||||||
|
```
|
||||||
|
修复前 fps : 87.9
|
||||||
|
修复后 fps : 114.9 (20 秒稳定 111–115,丢帧 0.000%)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.2 字幕丢失 —— libass 被冻结
|
||||||
|
|
||||||
|
两个独立原因叠加:
|
||||||
|
|
||||||
|
**(a) 字幕画布不在 XR 场景里。** Jellyfin Web 12 用 libass 渲染 ASS 到独立的
|
||||||
|
`canvas.libassjs-canvas`,和 `<video>` 是两个元素。只把 `<video>` 放进 XR layer,字幕自然没有。
|
||||||
|
|
||||||
|
**(b) 进了 VR,libass 就停止渲染。** 实测:
|
||||||
|
|
||||||
|
```
|
||||||
|
window.requestAnimationFrame : 6 秒内 0 次调用 ← 完全停摆(XR 接管了帧循环)
|
||||||
|
timeupdate : 22 次 ✅
|
||||||
|
requestVideoFrameCallback : 288 次 ✅
|
||||||
|
```
|
||||||
|
|
||||||
|
Jellyfin 的字幕渲染由 rAF 驱动 → 画布冻结在进 VR 前的最后一帧。
|
||||||
|
|
||||||
|
**修复**:安装一个 rAF 代理 —— 进 VR 后把 `requestAnimationFrame` 的回调收进自己的队列,
|
||||||
|
用 `video.requestVideoFrameCallback` 驱动(它跟着视频帧走,正好是字幕需要的节奏);
|
||||||
|
不在 VR 时直接透传原生实现,2D 播放完全不受影响。
|
||||||
|
|
||||||
|
**字幕层**:用 `XRWebGLBinding.createEquirectLayer` 建第二个动态等距柱状层
|
||||||
|
(同一立体几何),把画布拷进 `getSubImage(layer, frame).colorTexture` 后经 GL 合成。
|
||||||
|
|
||||||
|
### 5.3 字幕分辨率
|
||||||
|
|
||||||
|
libass 按屏幕上的视频框尺寸渲染(**1416×708**,在 180° 球面上只有约 **4 px/度**,
|
||||||
|
而 Quest 3 约 **19 px/度** → 非常糊)。
|
||||||
|
|
||||||
|
必须**在进入 VR 之前**放大:XR 会话运行期间 libass 不会重算画布尺寸。
|
||||||
|
所以放大动作挂在「打开 VR 面板」这个点击上(进 VR 本来就要先开面板)。
|
||||||
|
|
||||||
|
```
|
||||||
|
放大前 : 1416x708
|
||||||
|
放大后 : 4320x2160 (仍是精确 2:1,约 24 px/度)
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5.4 你的字幕文件本身就是为 SBS 设计的
|
||||||
|
|
||||||
|
```
|
||||||
|
Title: VR Dual-Eye Subtitle
|
||||||
|
PlayResX: 3840 PlayResY: 1920 ← 2:1,与视频同比例
|
||||||
|
Style: LeftEye , Alignment=8, MarginL=50 , MarginR=1920, MarginV=700
|
||||||
|
Style: RightEye, Alignment=8, MarginL=1920 , MarginR=50 , MarginV=700
|
||||||
|
956 条对白,每条成对出现(LeftEye + RightEye)
|
||||||
|
```
|
||||||
|
|
||||||
|
实测墨迹分布 `left=30, right=30`(完全对称)—— 左半屏给左眼、右半屏给右眼,
|
||||||
|
正好匹配 `stereo-left-right` 层。**不需要任何转换。**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 硬性约束:必须由用户手势触发
|
||||||
|
|
||||||
|
WebXR 规范原文:
|
||||||
|
|
||||||
|
> Wait for the window to have *transient activation* …
|
||||||
|
> If the request was not made while the global object has transient activation, return `false`.
|
||||||
|
|
||||||
|
所以 `navigator.xr.requestSession()` **必须同步写在点击处理函数里**,前面不能有任何 `await`
|
||||||
|
(否则 user activation 丢失,被 UA 拒绝)。代码里这一点是硬性保证的,改动时不要破坏。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 渲染路径
|
||||||
|
|
||||||
|
| 路径 | 条件 | 说明 |
|
||||||
|
|---|---|---|
|
||||||
|
| **A. `XRMediaBinding.createEquirectLayer`**(首选) | 浏览器支持 WebXR Media Layers | Meta 官方推荐:视频直接交给系统合成器,不经 WebGL 采样。**8K 下唯一靠谱的路** |
|
||||||
|
| **B. WebGL2 等距柱状光线重建**(回退) | 上面不可用 | 每帧 `texImage2D(video)` 上传整帧,8K 会吃力,仅兜底 |
|
||||||
|
|
||||||
|
投影参数:
|
||||||
|
|
||||||
|
- VR180 SBS → `layout:'stereo-left-right'`, `centralHorizontalAngle:π`
|
||||||
|
- VR360 SBS → `layout:'stereo-left-right'`, `centralHorizontalAngle:2π`
|
||||||
|
- 上下 3D → `layout:'stereo-top-bottom'`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 8. 修改 / 部署 / 回滚
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 改 jfvr-mvp.js 后重新部署
|
||||||
|
JF_PW='<ssh密码>' python3 deploy.py
|
||||||
|
|
||||||
|
# 完全回滚(恢复原始 index.html 并删除脚本)
|
||||||
|
JF_PW='<ssh密码>' python3 deploy.py rollback
|
||||||
|
```
|
||||||
|
|
||||||
|
`deploy.py` 用文件 md5 前 6 位作为版本号写进 script 标签的 query,自动破缓存。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 9. Meta 官方规格(决定性依据)
|
||||||
|
|
||||||
|
出处:`developers.meta.com/horizon/documentation/web/browser-video/`
|
||||||
|
|
||||||
|
- 支持的容器:**MP4 / Ogg / WebM**(**MKV 不在列表**)
|
||||||
|
- 支持的编码:h.264 / **h.265** / vp8 / vp9 / AV1
|
||||||
|
- > **For all formats, Browser supports 4K and supports 8K on h.265 and AV1.**
|
||||||
|
- > **WebXR Media Layers 是显示沉浸式视频最高效的方式。如果可能,围绕这个 API 设计你的体验。**
|
||||||
|
- Quest 浏览器还内置 2D 页面全屏重投影(可手动选 360/180 + 单目/左右/上下),
|
||||||
|
但 2D 路径最大显示尺寸 ≤ 7680×3840,且每次要手动选 —— 所以走 WebXR 才有意义。
|
||||||
|
|
||||||
|
WebXR 规范(`immersive-web/webxr` index.bs):
|
||||||
|
|
||||||
|
> Wait for the window to have **transient activation** …
|
||||||
|
> If the request was not made while the global object has transient activation, return `false`.
|
||||||
|
|
||||||
|
所以 `navigator.xr.requestSession()` 必须**同步**写在点击处理里,前面不能有 `await` ——
|
||||||
|
代码里是硬保证,改动时不要破坏。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 10. 片源情况(实测)
|
||||||
|
|
||||||
|
```
|
||||||
|
/mnt1/123 → 61 个 VR 文件,全部:
|
||||||
|
8192 × 4096 | HEVC Main / level 6.1 | ~60 fps | 8–25 Mbps | .mp4 (faststart 已确认)
|
||||||
|
```
|
||||||
|
|
||||||
|
即 **VR180 SBS**,每眼 4096×4096 的 180° 等距柱状。
|
||||||
|
|
||||||
|
> ⚠️ **必须保持 Direct Play**。服务器的 Pentium Gold 8505 + UHD 核显编不了 8K
|
||||||
|
> (Alder Lake-N 编码上限约 4K),一旦 Jellyfin 触发转码,分辨率腰斩且 CPU 打满。
|
||||||
|
> 诊断面板里的 `playbackMode` 会显示当前是 DirectPlay 还是 HLS。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 11. 相关现成实现(备查)
|
||||||
|
|
||||||
|
| 项目 | 技术 | 备注 |
|
||||||
|
|---|---|---|
|
||||||
|
| `allexxex/jellyfinvr` | Three.js + WebXR | VR180/360/SBS/OU;每帧上传纹理,8K 下是瓶颈 |
|
||||||
|
| `Mix1C/Jellyfin-360VR-Player` | A-Frame | 仅 360°,无 180 |
|
||||||
|
| `Lqlsoftware/Jellyfin-vr-plugin` | C# 插件 | 独立播放页 `/VR/Video/{id}/Play` |
|
||||||
|
| `Toastyice/DeoVRDeeplink` | C# 插件 + `deovr://` | 可完全自动唤起 Quest 上的 DeoVR(deeplink 走系统,不受 user gesture 限制) |
|
||||||
|
| `IAmParadox27/jellyfin-plugin-file-transformation` | C# 插件 | 正式版注入通道 |
|
||||||
|
| `n00bcodr/jellyfin-plugins` → JavaScript Injector | 插件 | `12/manifest.json`,targetAbi 12.0.0.0 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 2 — in-VR UX pass
|
||||||
|
|
||||||
|
Feedback from the first headset test, and what changed.
|
||||||
|
|
||||||
|
## Fixed
|
||||||
|
|
||||||
|
| Feedback | Fix |
|
||||||
|
|---|---|
|
||||||
|
| The progress bar was permanently visible | **Hidden by default.** `show()` appears on any pinch/trigger and auto-hides after `ctlAutoHideMs` (default 4 s, slider 1–10 s). While hidden: no draw, no texture upload, and the layer is dropped from the render state. |
|
||||||
|
| No guide line for hand/controller input | **Laser beam + cursor.** A thin quad beam (height 6 mm) runs from the input source to the panel, bright at the origin fading to transparent, plus a ring/dot cursor drawn at the exact hit point. |
|
||||||
|
| Subtitles should sit below the visual centre | Default `subPlacement.y` moved `-0.34 → -0.42` (quad centre, negative = below eye level). |
|
||||||
|
| Subtitle opacity and position should be adjustable | 4 live sliders in the panel: opacity, vertical position, distance, width. |
|
||||||
|
|
||||||
|
Input is now **one code path for hands and controllers**: both expose
|
||||||
|
`XRInputSource.targetRaySpace`, so each is ray-cast against the panel plane.
|
||||||
|
Gaze (view centre) is only the fallback when no ray reaches the panel.
|
||||||
|
|
||||||
|
## Beam orientation — corrected convention
|
||||||
|
|
||||||
|
The first spec said "orient the quad so its local `-Z` follows the ray". That is
|
||||||
|
geometrically wrong: a quad's local `-Z` is its plane **normal**, so the quad's
|
||||||
|
plane would end up perpendicular to the ray and its `width` would lie *across*
|
||||||
|
the beam — a thin face-on bar, not a laser.
|
||||||
|
|
||||||
|
Correct:
|
||||||
|
|
||||||
|
```
|
||||||
|
position = origin + dir * (L/2) // L = hit distance, else 1.2 m
|
||||||
|
width = L, height = 0.006
|
||||||
|
local +X -> dir // beam axis lies IN the quad plane
|
||||||
|
local -Z -> perpendicular to dir, biased up so it is never edge-on
|
||||||
|
```
|
||||||
|
|
||||||
|
Verified on device against two different rays — the measured beam midpoint
|
||||||
|
matched `origin + dir·L/2` to 1 cm, and the beam layer is added/removed with the
|
||||||
|
pointer.
|
||||||
|
|
||||||
|
## UI bug found and fixed this round
|
||||||
|
|
||||||
|
Adding 5 sliders grew the settings panel to 784 px while the viewport was 708 px,
|
||||||
|
so `进入 VR` ended up **off-screen and unclickable** (`elementFromPoint` returned
|
||||||
|
`null`). The panel is now a flex column with a scrolling body and a **sticky
|
||||||
|
footer** holding the status line and the CTA, and `max-height: calc(100vh - 104px)`.
|
||||||
|
|
||||||
|
## Verified on Quest 3 after this pass
|
||||||
|
|
||||||
|
```
|
||||||
|
layers : [equirect, quad(op=0.85)] ← control bar correctly absent
|
||||||
|
sub.placement : [-0.42, -1.45, 2.0]
|
||||||
|
sub.opacity : 0.85 (applied to layer.opacity)
|
||||||
|
ctl.hidden : true ← default hidden
|
||||||
|
fps : 110.9 targetFrameRate: 120
|
||||||
|
errors : []
|
||||||
|
```
|
||||||
|
|
||||||
|
Beam check (pointer frozen so main's per-frame update could not overwrite it):
|
||||||
|
|
||||||
|
```
|
||||||
|
controller ray -> beam w=1.22 pos=(0.13,-0.46,-0.86) matches origin+dir·L/2
|
||||||
|
hand ray -> beam w=1.45 pos=(-0.12,-0.47,-0.89) (different dir, follows)
|
||||||
|
pointer null -> beam removed (layers 4 -> 3, beamOn false)
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 3 — aiming guide + system gestures
|
||||||
|
|
||||||
|
## Root cause of "no guide line" AND "cursor click does nothing"
|
||||||
|
|
||||||
|
Both symptoms were one chain of three bugs:
|
||||||
|
|
||||||
|
1. `applyBeamVisible()` required `S.shown` — the beam only existed while the
|
||||||
|
control bar was visible. But the bar only appears on activation, so the user
|
||||||
|
could never aim at it before pinching. **Chicken-and-egg.**
|
||||||
|
2. `hook()` early-returned on `!S.shown`, so `presentBeam()` never ran and the
|
||||||
|
beam texture was never uploaded — the beam could not render even when flagged
|
||||||
|
visible. **Second, independent blocker.**
|
||||||
|
3. Only the `select` event was handled. Meta documents hand pinch as
|
||||||
|
"implemented in the emulated Gamepad attribute for the hand as a **button
|
||||||
|
press**", which does not reliably raise `select`. So on hands **nothing fired
|
||||||
|
at all** — no bar, no beam, no click.
|
||||||
|
|
||||||
|
Also: `pointer.visible` used to require an actual panel hit, so the aiming guide
|
||||||
|
would only appear once you were already on target. Now `visible` means "an input
|
||||||
|
source is pointing" and `hit` is separate (may be null).
|
||||||
|
|
||||||
|
## Use the platform's gesture recognition — do not reimplement it
|
||||||
|
|
||||||
|
| Platform facility | What it gives us |
|
||||||
|
| --- | --- |
|
||||||
|
| `XRInputSource.gamepad` emulated buttons | **hand pinch** (per Meta's WebXR Hands doc) |
|
||||||
|
| `selectstart` / `select` / `selectend` | controller trigger, and pinch on some builds |
|
||||||
|
| `targetRaySpace` | one ray API for hands **and** controllers |
|
||||||
|
|
||||||
|
So `90-main.js` now polls gamepad buttons for a rising edge **and** listens for
|
||||||
|
`select`, with a 250 ms dedupe so both paths cannot double-fire. No thumb↔index
|
||||||
|
joint-distance math is done anywhere.
|
||||||
|
|
||||||
|
System-reserved gesture to avoid: **palm pinch** (left hand = menu button, leaves
|
||||||
|
the session). Ordinary point-and-pinch is ours.
|
||||||
|
|
||||||
|
Activation now requires a real ray hit (`pointer.visible && pointer.hit`), so a
|
||||||
|
stray pinch can no longer fall back to gaze-centre and seek to 50%.
|
||||||
|
|
||||||
|
## Verified on device after the fix
|
||||||
|
|
||||||
|
```
|
||||||
|
pointing, NOT hitting the panel -> beamOn: true, ctlHidden: true
|
||||||
|
beam quad present (h=0.006, w=1.2)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Feasibility: auto-adjusting subtitle depth from scene content
|
||||||
|
|
||||||
|
Measured on a real 8K VR180 frame (SBS pair, 320x160 downsample, SAD block
|
||||||
|
matching +-24 px):
|
||||||
|
|
||||||
|
| Band | median disparity | match residual | disparity IQR |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| mid frame (y 40-88) | 1 px | 6.1 | — |
|
||||||
|
| **subtitle band (y 96-144)** | -5 .. -8 px | **13.5-15.3** | **[-16, +1]** |
|
||||||
|
|
||||||
|
- Disparity IS extractable and temporally stable (mid band held 1 px across 6 frames).
|
||||||
|
- But the **subtitle band is the least reliable region**: 2.3x the matching
|
||||||
|
residual and a 17 px inter-quartile spread — i.e. several depths coexist there,
|
||||||
|
so no single depth represents it.
|
||||||
|
- Disparity -> metres needs the VR180 rig's focal length and baseline, which are
|
||||||
|
not known a priori.
|
||||||
|
- Compute is NOT the blocker: the naive JS test cost ~250 ms/frame, but a WebGL2
|
||||||
|
SAD shader would be ~1-2 ms. Not the limiting factor.
|
||||||
|
|
||||||
|
Prior art (the depth question is studied; the answer is not "match scene depth"):
|
||||||
|
|
||||||
|
- Sidenmark, Kiefer & Gellersen, *Subtitles in Interactive VR: Using Gaze to
|
||||||
|
Address Depth Conflicts* (IEEE VR'19, Lancaster): depth conflicts between
|
||||||
|
environment and subtitles cause **occlusion and strain**; they propose
|
||||||
|
**gaze-based** techniques.
|
||||||
|
- Shimizu et al. (IEEE 2021): four dynamic subtitle placement methods, balancing
|
||||||
|
vertical/horizontal/**depth** of eye movement.
|
||||||
|
- *Gaze-Adaptive Subtitles for 360° Videos in VR* (IEEE 2025): adaptive placement
|
||||||
|
reduces visual fatigue versus fixed below-the-FOV placement.
|
||||||
|
|
||||||
|
**But**: the literature's best technique needs **eye tracking**, and Meta does
|
||||||
|
not expose `eye-tracking` to WebXR — our negotiated `enabledFeatures` were
|
||||||
|
`[local, viewer, bounded-floor, layers, hand-tracking, webxr, local-floor]`.
|
||||||
|
So gaze-adaptive placement is not available to us.
|
||||||
|
|
||||||
|
Recommendation if this is pursued: option **A** — use the *mid-band* (reliable)
|
||||||
|
disparity to bias the depth toward the scene, blended ~30/70 with the comfortable
|
||||||
|
default, clamped to 1.4-3.0 m, smoothed with a 2-3 s time constant, and scale the
|
||||||
|
quad proportionally so the **angular text size stays constant** (otherwise the
|
||||||
|
text visibly pulses). Default OFF. Chasing per-frame scene depth (option C) is
|
||||||
|
not recommended: disparity noise plus vergence conflict produces discomfort.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 4 — in-VR subtitle placement
|
||||||
|
|
||||||
|
Adjust the subtitles **without leaving VR**. Verified on device this round.
|
||||||
|
|
||||||
|
## Grab & drag the strip
|
||||||
|
|
||||||
|
Point at the subtitle text, pinch, and move:
|
||||||
|
|
||||||
|
- **up / down** → vertical position (drag on a fixed reference plane)
|
||||||
|
- **pull toward / push away** → depth (the hand's `origin.z` in viewer space;
|
||||||
|
forward is `-Z`, so pulling back brings the strip closer — like a real object)
|
||||||
|
- release → the new position is saved
|
||||||
|
|
||||||
|
Both the ray and the strip live in **viewer space**, so no world-space maths is
|
||||||
|
needed. The reference plane is fixed at grab time so the drag stays stable.
|
||||||
|
|
||||||
|
### Two deliberate restrictions (both needed)
|
||||||
|
|
||||||
|
The strip sits in the lower centre of the view by default and the control bar is
|
||||||
|
0.30 m tall at `y = -0.28`, so the two **overlap in y**. Without restrictions the
|
||||||
|
strip would swallow every aim in the lower half of the view.
|
||||||
|
|
||||||
|
1. Grabbing only happens when the control bar is **not already shown** — a pinch
|
||||||
|
while operating the bar never yanks the subtitles.
|
||||||
|
2. Only the **central 40 %** of the strip counts as grabbable (`nx,ny ∈ 0.30–0.70`),
|
||||||
|
which is where the text actually is (the crop is letterboxed into the texture).
|
||||||
|
|
||||||
|
## In-VR buttons on the control bar
|
||||||
|
|
||||||
|
The expanded bar grew from 1.1 × 0.22 m to **1.1 × 0.30 m** and gained a subtitle
|
||||||
|
row. Verified reachable layout (grid scan, no ambiguous overlaps):
|
||||||
|
|
||||||
|
```
|
||||||
|
y 0.25-0.45 seek track (full width)
|
||||||
|
y 0.50-0.70 [<<<<][PPPP][>>>>][ZZZZ] back10 / playpause / fwd10 / subs
|
||||||
|
y 0.85-1.00 [aa][AA][oo][OO][RR] size −/+ · opacity −/+ · reset
|
||||||
|
```
|
||||||
|
|
||||||
|
Actions: `sub_smaller`, `sub_bigger`, `sub_opaque_down`, `sub_opaque_up`, `sub_reset`.
|
||||||
|
|
||||||
|
## Persistence
|
||||||
|
|
||||||
|
`localStorage['jfvr.settings.v1']`, debounced 400 ms, loaded at boot with range
|
||||||
|
validation and silent fallback on corruption.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"subPlacement":{"y":-0.78,"z":-1.45,"width":2.8},"subOpacity":0.45,
|
||||||
|
"subMode":"hud","subAlpha":"premultiplied","projection":"180","stereo":"sbs",
|
||||||
|
"subtitles":true,"controls":true,"ctlAutoHideMs":4000}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Verified this round
|
||||||
|
|
||||||
|
```
|
||||||
|
live setPlacement -> layer [-0.7, -2.1, 2.6] immediately
|
||||||
|
live setOpacity -> 0.35
|
||||||
|
resetPlacement() -> [-0.42, -1.45, 2]
|
||||||
|
slider -> localStorage written, and after a page reload the values come back
|
||||||
|
errors: []
|
||||||
|
```
|
||||||
|
|
||||||
|
Note: the four subtitle sliders in the 2D panel and the in-VR buttons now share
|
||||||
|
one code path (`setSubPlacementLive` / `setSubOpacityLive` / `setSubModeLive`),
|
||||||
|
so both save automatically.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 6 — subtitle depth vs size decoupling (angular-size lock)
|
||||||
|
|
||||||
|
## The problem, as the owner put it
|
||||||
|
|
||||||
|
> 当前 vr 视频中可能出现 vr 空间中物体可能距离我很近,如果字幕还很远,我将无法在
|
||||||
|
> 看清物体的同时看清字幕。我希望可以让字幕的空间距离和物体的距离尽量接近。
|
||||||
|
|
||||||
|
Fixation is a **vergence** problem: an object at 0.4 m and subtitles at 1.45 m cannot
|
||||||
|
both be looked at comfortably at the same time. Eye tracking is not available to
|
||||||
|
WebXR on Quest (`enabledFeatures` has no `eye-tracking`), so the depth cannot be
|
||||||
|
driven by gaze — see ADDENDUM 3 above, which already concluded the gaze-adaptive
|
||||||
|
literature is out of reach for us.
|
||||||
|
|
||||||
|
What IS in reach is making **depth a pure depth knob**: keeping the strip's angular
|
||||||
|
size (and its angular position) constant while it moves in depth. That is exactly
|
||||||
|
the "scale the quad proportionally so the angular text size stays constant" note
|
||||||
|
from ADDENDUM 3, applied to the manual control.
|
||||||
|
|
||||||
|
Measured behaviour of the old control (free mode), which is why it did not work:
|
||||||
|
|
||||||
|
| depth | width | strip angular width |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| 1.45 m | 2.00 m | 69° |
|
||||||
|
| 0.70 m | 2.00 m | 110° ← text blows up |
|
||||||
|
| 0.35 m | 2.00 m | 141° ← text leaves the view |
|
||||||
|
|
||||||
|
## What shipped
|
||||||
|
|
||||||
|
`state.subLockAngle = true`, `state.subAngleDeg = 69.2` (defaults). While locked:
|
||||||
|
|
||||||
|
```
|
||||||
|
width = 2 · |z| · tan(subAngleDeg / 2) constant angular width
|
||||||
|
y = (y0 / z0) · z constant angular height
|
||||||
|
```
|
||||||
|
|
||||||
|
so the strip stays at the same spot on the retina and only its **vergence
|
||||||
|
distance** changes. 69.2° is exactly the old `2.0 m @ 1.45 m`, so the default look
|
||||||
|
is unchanged. Depth range extended: near clamp **0.5 m → 0.35 m**.
|
||||||
|
|
||||||
|
In-VR subtitle row grew from five to nine buttons (the bar itself is unchanged —
|
||||||
|
`PANEL_M.expanded` = 0.72 × 0.21 m, canvas 1440 × 420 px). The row now carries
|
||||||
|
vertical position as well as depth, and the caption always leads with the depth:
|
||||||
|
|
||||||
|
```
|
||||||
|
y 0.80-1.00 [↑][↓][<<][>>][aa][AA][oo][OO][RR] 上 下 · 近 远 · 小 大 · 暗 亮 · 复位
|
||||||
|
caption 字幕 距1.45m · 69° 85%
|
||||||
|
```
|
||||||
|
|
||||||
|
`上`/`下` step the vertical position ±0.04 m (that control did not exist in the bar
|
||||||
|
before). `近`/`远` step depth ±0.15 m. `小`/`大` step the locked angle ∓5° (or
|
||||||
|
`width` ∓0.15 m when unlocked). Grab & drag carries `y` by the grab-start y/z
|
||||||
|
ratio while locked, so a strip pulled close does not slide below the view.
|
||||||
|
|
||||||
|
## Two bugs fixed in the same pass
|
||||||
|
|
||||||
|
1. **The CC (字幕) button never showed its state.** `Controls.readSubtitles()` only
|
||||||
|
read the controls module's own `S.subtitlesOn`, which `90-main.js` never sets —
|
||||||
|
it calls `setVideo(video)` with no `getExtra`. It now falls back to the shared
|
||||||
|
`state.subtitles`. The button is also unambiguous now: accent pill + `字幕开`
|
||||||
|
when on, slashed CC glyph + `字幕关` when off.
|
||||||
|
2. **There was no vertical-position control in VR.** Only grab & drag could move
|
||||||
|
the strip. The row gained `上`/`下`. (The 2D panel's `垂直位置` slider was
|
||||||
|
verified working — slider → `-0.70 m`, distance → `-0.80 m` re-derived
|
||||||
|
`y = -0.385` — so the gap was VR-only.)
|
||||||
|
|
||||||
|
## Verification (logic, offline)
|
||||||
|
|
||||||
|
The module was driven headlessly in node (state + maths only, no WebGL), holding
|
||||||
|
`subAngleDeg` at 69.2° and stepping depth:
|
||||||
|
|
||||||
|
```
|
||||||
|
z=-1.45 -> width 2.00, y -0.42 (69.2°)
|
||||||
|
z=-0.70 -> width 0.97, y -0.205 (69.2°)
|
||||||
|
z=-0.35 -> width 0.48, y -0.100 (69.2°)
|
||||||
|
z=-2.60 -> width 3.59, y -0.745 (69.2°)
|
||||||
|
```
|
||||||
|
|
||||||
|
and the 9-button row was grid-scanned through `Controls.hitTest` — 9 distinct
|
||||||
|
actions, no dead zone between pills. `Controls.draw` renders
|
||||||
|
`字幕开 · 字幕 距1.45m · 69°` while subtitles are on and locked, `字幕关` when
|
||||||
|
off, and `字幕 距1.45m · 2.0m` when unlocked. `build.py --check` passes.
|
||||||
|
|
||||||
|
Then verified in the **live Quest 3 browser** over CDP after deploying:
|
||||||
|
|
||||||
|
```
|
||||||
|
window.__JFVR.version -> 'hud-2'
|
||||||
|
state.stats.subLockAngle -> true
|
||||||
|
state.stats.subAngleDeg -> 69.2
|
||||||
|
state.errors -> []
|
||||||
|
|
||||||
|
Sub.setPlacement({z:-0.70}) -> width 0.97, y -0.20 (still 69.2°)
|
||||||
|
Sub.setPlacement({z:-0.35}) -> width 0.48, y -0.10 (still 69.2°)
|
||||||
|
Sub.setPlacement({z:-9}) -> width 3.59, y -0.745 (clamped to -2.6, still 69.2°)
|
||||||
|
Sub.setAngleDeg(50) -> width 2.42 (size control)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Literature for the default value (not yet acted on)
|
||||||
|
|
||||||
|
All of these measure the **character height** angular size; our control is the
|
||||||
|
whole strip's angular width, so they need `stats.subTextDeg` (new) to compare.
|
||||||
|
|
||||||
|
- Kojić et al., *User Experience of Reading in Virtual Reality* (arXiv 2004.01545):
|
||||||
|
preferred angular size ≈ **16–18 dmm for medium/long text** (0.9–1.0° character
|
||||||
|
height), 27–32 dmm for 2-word text, vs Google's guidelines of 32 dmm (title) and
|
||||||
|
40 dmm (headline). Participants also rated **too-large** angular size among the
|
||||||
|
worst experiences.
|
||||||
|
- Dingler et al., *VR Reading UIs* (CHI'18 EA): **41 ± 14 dmm** (≈2.35° ± 0.8°).
|
||||||
|
- The default stays 69.2° deliberately: it is the on-device-proven look. Read
|
||||||
|
`stats.subTextDeg` in the headset, then decide.
|
||||||
|
|
||||||
|
## Persistence
|
||||||
|
|
||||||
|
`jfvr.settings.v1` gains `subLockAngle` and `subAngleDeg`; while locked the stored
|
||||||
|
`subPlacement.width` is re-derived from `subAngleDeg` + `z` on load.
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"subPlacement":{"y":-0.42,"z":-0.35,"width":0.48},
|
||||||
|
"subLockAngle":true,"subAngleDeg":69.2,"subOpacity":0.85,
|
||||||
|
"subMode":"hud","subAlpha":"premultiplied","projection":"180","stereo":"sbs",
|
||||||
|
"controls":true,"ctlAutoHideMs":4000}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 7 — summon gating, lock badge, pre-VR panel trim
|
||||||
|
|
||||||
|
## The bar appeared on every pinch
|
||||||
|
|
||||||
|
Two independent causes, both removed:
|
||||||
|
|
||||||
|
1. `session.addEventListener('selectstart', onSelectStart)` called `Controls.show()`
|
||||||
|
unconditionally — any pinch anywhere in the view-summoned the bar.
|
||||||
|
2. `pointerDown()` called `Controls.show()` *before* it looked at the hit.
|
||||||
|
|
||||||
|
Now the bar is summoned only when the pointing ray lands on (or near) the bar:
|
||||||
|
|
||||||
|
```
|
||||||
|
bar hidden + aim on/near the bar -> show(), consume the press, arm for 500 ms
|
||||||
|
bar hidden + aim elsewhere -> nothing
|
||||||
|
bar shown + within 500 ms -> ignored
|
||||||
|
bar shown + aim on the bar -> handleSelect (and show() to extend auto-hide)
|
||||||
|
```
|
||||||
|
|
||||||
|
While LOCKED the same rule applies, except that what comes up is the padlock panel
|
||||||
|
alone — see ADDENDUM 7b below.
|
||||||
|
|
||||||
|
The summon window is wider than the hit window (`panelHit` padding 0.5/0.9 vs
|
||||||
|
0.25/0.6), because a ray that grazes the bar should still be able to bring it up —
|
||||||
|
the 500 ms arm is what prevents the accidental button press, not pixel precision.
|
||||||
|
|
||||||
|
**Superseded by ADDENDUM 7b:** the padlock was originally a separate quad layer to the
|
||||||
|
right of the bar. That did not work on device — the hand ray cannot reach that far
|
||||||
|
right. The padlock is now a normal panel button and "locked" is a third panel mode.
|
||||||
|
|
||||||
|
## Pre-VR settings panel trimmed
|
||||||
|
|
||||||
|
Sliders set on a flat screen do not predict what reads well in a headset, so the
|
||||||
|
subtitle section is now just 开 / 关. Removed: 模式, 透明合成, 不透明度, 垂直位置,
|
||||||
|
距离, 宽度, 距离与大小 (lock toggle), 视角宽度, 手部指向. `subAngleDeg`, placement
|
||||||
|
and opacity still persist — they are written by the in-VR controls.
|
||||||
|
|
||||||
|
The 手部指向 option (system ray vs index-finger ray) is gone together with its code
|
||||||
|
path; the system ray is the only mode (an index-finger ray injects fingertip jitter
|
||||||
|
into the cursor — see HANDOFF §6.6).
|
||||||
|
|
||||||
|
## "VR 播放" no longer covers the player's own controls
|
||||||
|
|
||||||
|
`.jfvr-btn` moved from `bottom: 24px` to `bottom: 120px`; at the viewport corner it
|
||||||
|
sat on Jellyfin's OSD fullscreen button. `.jfvr-panel` moved to `bottom: 180px`
|
||||||
|
(`max-height: calc(100vh - 200px)`) so the panel still stacks above the button.
|
||||||
|
|
||||||
|
## Verification
|
||||||
|
|
||||||
|
`build.py --check` passes. A local harness (a `file://` page with a fake 8192x4096
|
||||||
|
`<video>`, a fake libass canvas and stub `XRWebGLBinding`/`XRXRigidTransform`/
|
||||||
|
`createQuadLayer` classes) exercised the real bundle without a headset:
|
||||||
|
|
||||||
|
```
|
||||||
|
Controls.create(stubSession) -> ok, 4 quad layers (bar, beam x2, lock)
|
||||||
|
Controls.getLockLayer() -> the lock quad
|
||||||
|
Controls.setLock(true) -> stats.ctlLock = true, no throw
|
||||||
|
runFrameHooks(fakeFrame) -> presentLock/present ran, errors: []
|
||||||
|
panel DOM -> 画面: 投影/立体 · 字幕: 显示 · 控制: 控制条/自动收起
|
||||||
|
.jfvr-btn computed bottom -> 120px
|
||||||
|
drawLockBadge() -> 16 canvas ops, no throw (locked and unlocked)
|
||||||
|
```
|
||||||
|
|
||||||
|
Live Quest 3 over CDP after deploy: `__JFVR.version = 'hud-4'`,
|
||||||
|
`stats.subLockAngle = true`, `errors: []`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# ADDENDUM 7b — padlock as a panel button + the "locked" panel mode
|
||||||
|
|
||||||
|
## The failure on device
|
||||||
|
|
||||||
|
> 我有新发现,在面板唤出状态时其无法点击,但面板消失后,其可以点击。
|
||||||
|
|
||||||
|
ADDENDUM 7's padlock was its own quad layer at `(0.45, -0.34, -1.45)` — 0.45 m right of
|
||||||
|
the panel centre, 0.09 m right of the panel's own right edge. Two things were wrong:
|
||||||
|
|
||||||
|
1. **It was outside every reachable ray.** Aiming is the binding constraint with hands:
|
||||||
|
a hand ray plus the cursor's acceptance window (`panelHit` 0.25/0.6 on a 0.72 m panel)
|
||||||
|
reaches x ≈ 0.45 — the panel's right edge. Nothing could aim 0.09 m past it.
|
||||||
|
2. **Its hit test used the panel's layer geometry.** Whether it fired therefore depended
|
||||||
|
on the panel's own size/position, which is why it behaved differently shown vs hidden.
|
||||||
|
|
||||||
|
## The fix: make it an ordinary button
|
||||||
|
|
||||||
|
The padlock is now the fifth button in the media row and uses the same code path as
|
||||||
|
everybutton that already worked (cursor → `hitTest` → `handleSelect` → `onAction`):
|
||||||
|
|
||||||
|
```
|
||||||
|
expanded: [<<<] [>||] [>>>] [CC] [padlock]
|
||||||
|
0.18 0.36 0.54 0.72 0.90 (hit pill half width 0.092)
|
||||||
|
```
|
||||||
|
|
||||||
|
## The behavior the owner asked for
|
||||||
|
|
||||||
|
> 想要的是:唤出面板,然后可以上锁,上锁后锁头图标和面板一样自动隐藏,下次触发面板
|
||||||
|
> 唤出时,只展示锁本身,不展示进度条等能调整的项目。(解锁后的0.5s的动做也忽略)
|
||||||
|
|
||||||
|
A third panel mode does exactly that:
|
||||||
|
|
||||||
|
| mode | size (m) | canvas | drawn / hit-testable |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `expanded` | 0.72 × 0.21 | 1440 × 420 | everything |
|
||||||
|
| `locked` | 0.26 × 0.10 | 520 × 200 | the padlock + 解锁 label ONLY |
|
||||||
|
|
||||||
|
```
|
||||||
|
summon (unlocked) -> expanded panel, auto-hides after ctlAutoHideMs
|
||||||
|
press 上锁 -> panel becomes padlock-only IN PLACE, auto-hide timer restarts
|
||||||
|
(so the padlock disappears on its own, like the panel always did)
|
||||||
|
summon (locked) -> padlock alone; no progress bar, no rows, nothing adjustable
|
||||||
|
press 解锁 -> full panel returns, and 500 ms of presses are ignored
|
||||||
|
```
|
||||||
|
|
||||||
|
Because `hide()` no longer resets the mode, the layer keeps the size it will be shown
|
||||||
|
at: the summon window stays the full panel rect (0.72 × 0.21 unlocked, 0.26 × 0.10
|
||||||
|
locked) and re-showing never pops a resize. `hitTest()` takes the mode explicitly now
|
||||||
|
(`collapsed` and `locked` share no safe aspect-ratio invariant), and `handleSelect()`
|
||||||
|
no longer falls back to pressing the panel centre when there is no ray hit.
|
||||||
|
|
||||||
|
## Verification (offline, real bundle, no headset)
|
||||||
|
|
||||||
|
A temporary `file://` harness fed the real `dist/jfvr.js` a fake 8K `<video>`, a fake
|
||||||
|
libass canvas and stub `XRWebGLBinding` / `XRXRigidTransform` / `createQuadLayer`
|
||||||
|
classes. The harness file was deleted afterwards.
|
||||||
|
|
||||||
|
```
|
||||||
|
Controls.create(stubSession) -> ok, layers: bar + beam x2
|
||||||
|
expanded media row (5 centres) -> back10|playpause|fwd10|subs|lock, no dead zone
|
||||||
|
expanded subtitle row (9 centres) -> 9 distinct actions
|
||||||
|
locked panel: centre -> {type:'lock'} -> bottom edge / left edge -> null
|
||||||
|
draw() locked -> draws ONLY '解锁'
|
||||||
|
draw() expanded -> 上锁 + 字幕 距1.45m · 69° + 00:10 (all intact)
|
||||||
|
setLock(true) -> stats.ctlMode=locked, layer 0.72x0.21 -> 0.26x0.10, canvas resized
|
||||||
|
setLock(false) -> stats.ctlMode=expanded, layer back to 0.72x0.21
|
||||||
|
hide() -> mode KEPT (summon window stays the panel rect)
|
||||||
|
runFrameHooks() after each switch -> no errors, stats.hookErr / ctlErr null
|
||||||
|
handleSelect() with no ray hit -> null (was: silently pressed the centre)
|
||||||
|
```
|
||||||
|
|
||||||
|
Also standalone (`node`): the padlock hit region and the mode table above were asserted
|
||||||
|
directly against `Controls.hitTest` / `Controls.draw` with mock 2D contexts.
|
||||||
|
|
||||||
|
Live Quest 3 over CDP after deploy: `__JFVR.version = 'hud-5'`, `stats.ctlMode =
|
||||||
|
'expanded'`, `stats.ctlLock = false`, `errors: []`.
|
||||||
|
|
||||||
|
## Still unverified
|
||||||
|
|
||||||
|
The *feel* — how easily the padlock is aimed with the hand ray, and whether 0.5 s is the
|
||||||
|
right arm delay — can only be judged in the headset.
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Concatenate src/*.js (in filename order) into dist/jfvr.js.
|
||||||
|
|
||||||
|
Each module is already a self-contained IIFE that attaches to window.__JFVR,
|
||||||
|
so the build is a straight concatenation with a banner. No minification: the
|
||||||
|
deployed file stays readable in DevTools, which matters because on-device
|
||||||
|
debugging happens over CDP.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python3 build.py # build
|
||||||
|
python3 build.py --check # build + syntax check every module
|
||||||
|
"""
|
||||||
|
import hashlib
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
SRC = os.path.join(HERE, "src")
|
||||||
|
DIST = os.path.join(HERE, "dist")
|
||||||
|
OUT = os.path.join(DIST, "jfvr.js")
|
||||||
|
|
||||||
|
ORDER = ["00-core.js", "10-subtitles.js", "20-controls.js", "30-hands.js", "90-main.js"]
|
||||||
|
|
||||||
|
|
||||||
|
def sources():
|
||||||
|
"""Return (path, text) for every module, in load order."""
|
||||||
|
found = sorted(f for f in os.listdir(SRC) if f.endswith(".js"))
|
||||||
|
ordered = [f for f in ORDER if f in found]
|
||||||
|
ordered += [f for f in found if f not in ORDER]
|
||||||
|
if not ordered:
|
||||||
|
sys.exit("no sources in %s" % SRC)
|
||||||
|
return [(f, open(os.path.join(SRC, f), encoding="utf-8").read()) for f in ordered]
|
||||||
|
|
||||||
|
|
||||||
|
def syntax_check():
|
||||||
|
bad = 0
|
||||||
|
for name, _ in sources():
|
||||||
|
p = os.path.join(SRC, name)
|
||||||
|
r = subprocess.run(["node", "--check", p], capture_output=True, text=True)
|
||||||
|
status = "ok" if r.returncode == 0 else "FAIL"
|
||||||
|
print(" %-18s %s" % (name, status))
|
||||||
|
if r.returncode != 0:
|
||||||
|
print(r.stderr.strip())
|
||||||
|
bad += 1
|
||||||
|
return bad
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
if "--check" in sys.argv:
|
||||||
|
print("syntax check:")
|
||||||
|
sys.exit(1 if syntax_check() else 0)
|
||||||
|
|
||||||
|
mods = sources()
|
||||||
|
parts = [
|
||||||
|
"/*!",
|
||||||
|
" * JFVR (Jellyfin Web VR) — built bundle. DO NOT EDIT.",
|
||||||
|
" * Edit src/*.js and run: python3 build.py",
|
||||||
|
" * Modules: " + ", ".join(n for n, _ in mods),
|
||||||
|
" */",
|
||||||
|
]
|
||||||
|
for name, text in mods:
|
||||||
|
parts.append("\n/* ===== %s ===== */\n" % name)
|
||||||
|
parts.append(text.rstrip("\n"))
|
||||||
|
parts.append("\n")
|
||||||
|
bundle = "\n".join(parts)
|
||||||
|
|
||||||
|
os.makedirs(DIST, exist_ok=True)
|
||||||
|
with open(OUT, "w", encoding="utf-8") as fh:
|
||||||
|
fh.write(bundle)
|
||||||
|
|
||||||
|
digest = hashlib.md5(bundle.encode()).hexdigest()
|
||||||
|
print("built %s" % OUT)
|
||||||
|
print(" modules : %d (%s)" % (len(mods), ", ".join(n for n, _ in mods)))
|
||||||
|
print(" bytes : %d" % len(bundle))
|
||||||
|
print(" md5 : %s" % digest)
|
||||||
|
print(" version : %s" % digest[:6])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Deploy jfvr-mvp.js into the Jellyfin container and inject the script tag
|
||||||
|
into jellyfin-web/index.html.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
JF_PW='<ssh-password>' python3 deploy.py # deploy current jfvr-mvp.js
|
||||||
|
JF_PW='<ssh-password>' python3 deploy.py rollback # restore original index.html
|
||||||
|
|
||||||
|
Env:
|
||||||
|
JF_HOST (default 192.168.123.199)
|
||||||
|
JF_USER (default admin)
|
||||||
|
JF_PW (required) SSH password
|
||||||
|
JF_CONTAINER (default Jellyfin)
|
||||||
|
"""
|
||||||
|
import base64
|
||||||
|
import hashlib
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
import paramiko
|
||||||
|
|
||||||
|
HOST = os.environ.get("JF_HOST", "192.168.123.199")
|
||||||
|
USER = os.environ.get("JF_USER", "admin")
|
||||||
|
PW = os.environ.get("JF_PW")
|
||||||
|
CONTAINER = os.environ.get("JF_CONTAINER", "Jellyfin")
|
||||||
|
|
||||||
|
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
BUILT_JS = os.path.join(HERE, "dist", "jfvr.js")
|
||||||
|
LEGACY_JS = os.path.join(HERE, "legacy", "jfvr-mvp-v6-working.js")
|
||||||
|
REMOTE_JS = "/tmp/jfvr-mvp.js"
|
||||||
|
WEB_DIR = "/jellyfin/jellyfin-web"
|
||||||
|
BACKUP_DIR = "/vol2/jellyfin/config/_vr_backup"
|
||||||
|
|
||||||
|
TAG = '<script plugin="JellyfinVR" version="%s" src="jfvr-mvp.js?v=%s" defer></script>'
|
||||||
|
|
||||||
|
|
||||||
|
def build():
|
||||||
|
"""Concatenate src/*.js into dist/jfvr.js. Aborts on failure."""
|
||||||
|
r = subprocess.run(
|
||||||
|
[sys.executable, os.path.join(HERE, "build.py")],
|
||||||
|
capture_output=True, text=True, cwd=HERE,
|
||||||
|
)
|
||||||
|
print(r.stdout.strip() or r.stderr.strip())
|
||||||
|
if r.returncode != 0:
|
||||||
|
sys.exit("build failed")
|
||||||
|
r = subprocess.run(
|
||||||
|
[sys.executable, os.path.join(HERE, "build.py"), "--check"],
|
||||||
|
capture_output=True, text=True, cwd=HERE,
|
||||||
|
)
|
||||||
|
if r.returncode != 0:
|
||||||
|
print(r.stdout)
|
||||||
|
sys.exit("syntax check failed")
|
||||||
|
|
||||||
|
|
||||||
|
def run(client, script, timeout=180):
|
||||||
|
"""Run a bash script as root on the remote host."""
|
||||||
|
b64 = base64.b64encode(script.encode()).decode()
|
||||||
|
wrapped = (
|
||||||
|
"echo '%s' | sudo -S -p '' bash -c "
|
||||||
|
"'echo %s | base64 -d > /tmp/_jfvr.sh; bash /tmp/_jfvr.sh'" % (PW, b64)
|
||||||
|
)
|
||||||
|
_, out, err = client.exec_command(wrapped, timeout=timeout)
|
||||||
|
text = out.read().decode("utf-8", "replace")
|
||||||
|
errs = err.read().decode("utf-8", "replace")
|
||||||
|
errs = "\n".join(
|
||||||
|
line for line in errs.splitlines()
|
||||||
|
if "password for" not in line and not line.startswith("[sudo]")
|
||||||
|
)
|
||||||
|
return text, errs
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
if not PW:
|
||||||
|
sys.exit("JF_PW env var is required")
|
||||||
|
|
||||||
|
rollback = len(sys.argv) > 1 and sys.argv[1] == "rollback"
|
||||||
|
legacy = "--legacy" in sys.argv
|
||||||
|
|
||||||
|
if not rollback:
|
||||||
|
if legacy:
|
||||||
|
BUILT_JS_LOCAL = LEGACY_JS
|
||||||
|
print("deploying LEGACY bundle")
|
||||||
|
else:
|
||||||
|
build()
|
||||||
|
BUILT_JS_LOCAL = BUILT_JS
|
||||||
|
else:
|
||||||
|
BUILT_JS_LOCAL = None
|
||||||
|
|
||||||
|
client = paramiko.SSHClient()
|
||||||
|
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
|
||||||
|
client.connect(
|
||||||
|
HOST, username=USER, password=PW, timeout=15,
|
||||||
|
look_for_keys=False, allow_agent=False,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
if rollback:
|
||||||
|
script = f"""
|
||||||
|
set -e
|
||||||
|
if [ ! -f "{BACKUP_DIR}/index.html.orig" ]; then
|
||||||
|
echo "no backup found at {BACKUP_DIR}/index.html.orig"; exit 1
|
||||||
|
fi
|
||||||
|
docker cp "{BACKUP_DIR}/index.html.orig" {CONTAINER}:{WEB_DIR}/index.html
|
||||||
|
docker exec {CONTAINER} rm -f {WEB_DIR}/jfvr-mvp.js
|
||||||
|
echo "rolled back. tail:"
|
||||||
|
docker exec {CONTAINER} tail -c 200 {WEB_DIR}/index.html
|
||||||
|
"""
|
||||||
|
out, err = run(client, script)
|
||||||
|
print(out)
|
||||||
|
if err.strip():
|
||||||
|
print("--- STDERR ---\n" + err)
|
||||||
|
return
|
||||||
|
|
||||||
|
data = open(BUILT_JS_LOCAL, "rb").read()
|
||||||
|
md5 = hashlib.md5(data).hexdigest()
|
||||||
|
version = "mvp-%s" % md5[:6]
|
||||||
|
tag = TAG % (version, md5[:6])
|
||||||
|
|
||||||
|
sftp = client.open_sftp()
|
||||||
|
with sftp.open(REMOTE_JS, "wb") as fh:
|
||||||
|
fh.write(data)
|
||||||
|
sftp.close()
|
||||||
|
|
||||||
|
print("uploaded %d bytes, md5=%s, tag=%s" % (len(data), md5, version))
|
||||||
|
|
||||||
|
script = f"""
|
||||||
|
set -e
|
||||||
|
mkdir -p "{BACKUP_DIR}"
|
||||||
|
[ -f "{BACKUP_DIR}/index.html.orig" ] || docker cp {CONTAINER}:{WEB_DIR}/index.html "{BACKUP_DIR}/index.html.orig"
|
||||||
|
docker cp {REMOTE_JS} {CONTAINER}:{WEB_DIR}/jfvr-mvp.js
|
||||||
|
docker exec {CONTAINER} sed -i -E 's|<script plugin="JellyfinVR"[^>]*></script>||g' {WEB_DIR}/index.html
|
||||||
|
docker exec {CONTAINER} sed -i 's|</body>|{tag}</body>|' {WEB_DIR}/index.html
|
||||||
|
echo "--- tail of index.html ---"
|
||||||
|
docker exec {CONTAINER} tail -c 300 {WEB_DIR}/index.html
|
||||||
|
echo
|
||||||
|
echo "--- md5 inside container ---"
|
||||||
|
docker exec {CONTAINER} md5sum {WEB_DIR}/jfvr-mvp.js
|
||||||
|
"""
|
||||||
|
out, err = run(client, script)
|
||||||
|
print(out)
|
||||||
|
if err.strip():
|
||||||
|
print("--- STDERR ---\n" + err)
|
||||||
|
finally:
|
||||||
|
client.close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
Vendored
+5315
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,28 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Reconnect to the Quest 3 via SideQuest's ADB and expose its browser CDP on :9222
|
||||||
|
set -u
|
||||||
|
ADB=${ADB:-/opt/SideQuest/resources/platform-tools/adb}
|
||||||
|
CDP_PORT=${CDP_PORT:-9222}
|
||||||
|
|
||||||
|
if ! [ -x "$ADB" ]; then echo "adb not found at $ADB" >&2; exit 1; fi
|
||||||
|
|
||||||
|
echo "== devices =="
|
||||||
|
$ADB devices -l
|
||||||
|
|
||||||
|
if ! $ADB devices | grep -qE '\bdevice$'; then
|
||||||
|
echo "!! no authorized device. Reconnect SideQuest (wireless ADB) and retry." >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "== forward tcp:$CDP_PORT =="
|
||||||
|
$ADB forward --remove-all >/dev/null 2>&1
|
||||||
|
$ADB forward "tcp:$CDP_PORT" localabstract:chrome_devtools_remote
|
||||||
|
|
||||||
|
echo "== verify =="
|
||||||
|
for i in 1 2 3 4 5; do
|
||||||
|
v=$(curl -s --max-time 3 "http://127.0.0.1:$CDP_PORT/json/version" || true)
|
||||||
|
if [ -n "$v" ]; then echo "$v"; echo "OK: CDP ready on 127.0.0.1:$CDP_PORT"; exit 0; fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
echo "!! CDP not answering on $CDP_PORT" >&2
|
||||||
|
exit 3
|
||||||
+343
@@ -0,0 +1,343 @@
|
|||||||
|
/*!
|
||||||
|
* JFVR core — shared namespace, state, helpers, registries.
|
||||||
|
* Loaded first. Everything else attaches to window.__JFVR.
|
||||||
|
*
|
||||||
|
* See INTERFACES.md for the binding contract.
|
||||||
|
*/
|
||||||
|
(function () {
|
||||||
|
'use strict';
|
||||||
|
|
||||||
|
if (window.__JFVR && window.__JFVR.__core) return; // idempotent
|
||||||
|
|
||||||
|
var NS = window.__JFVR = window.__JFVR || {};
|
||||||
|
|
||||||
|
NS.version = 'hud-5'; // hud-4 = ADDENDUM 7; hud-5 = ADDENDUM 7b (padlock in the panel + locked mode)
|
||||||
|
NS.__core = true;
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* logging
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
NS.log = function () {
|
||||||
|
var a = Array.prototype.slice.call(arguments);
|
||||||
|
try {
|
||||||
|
console.log.apply(console, ['%c[JFVR ' + NS.version + ']', 'color:#4ea1ff;font-weight:bold'].concat(a));
|
||||||
|
} catch (e) {
|
||||||
|
try { console.log('[JFVR]', a.join(' ')); } catch (e2) {}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* shared state (modules read/write; integrator owns lifecycle)
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
var S = NS.state = {
|
||||||
|
video: null,
|
||||||
|
session: null,
|
||||||
|
refSpace: null,
|
||||||
|
viewerSpace: null,
|
||||||
|
|
||||||
|
projection: '180', // '180' | '360'
|
||||||
|
stereo: 'sbs', // 'mono' | 'sbs' | 'ou'
|
||||||
|
|
||||||
|
subtitles: true,
|
||||||
|
ctlLock: false, // ADDENDUM 7: control-bar lock badge (per VR session)
|
||||||
|
subMode: 'hud', // 'hud' | 'sphere' | 'off'
|
||||||
|
subAlpha: 'premultiplied', // 'premultiplied' | 'straight'
|
||||||
|
subOpacity: 0.85, // 0..1, applied to the subtitle layer live
|
||||||
|
// y is the quad CENTRE; negative = below eye level. -0.42 keeps the strip
|
||||||
|
// clearly under the visual centre rather than crowding it.
|
||||||
|
subPlacement: { y: -0.42, z: -1.45, width: 2.0 },
|
||||||
|
// ADDENDUM 6 — depth / size decoupling. With subLockAngle ON, changing the
|
||||||
|
// strip's DEPTH holds its angular size (subAngleDeg, the strip's full
|
||||||
|
// angular width in degrees) and its angular height (the y/z ratio), so the
|
||||||
|
// user can pull the subtitles to the depth of a nearby object without the
|
||||||
|
// text blowing up or dropping out of the lower view. 69.2 deg is exactly
|
||||||
|
// the pre-ADDENDUM-6 look: 2.0 m wide at 1.45 m.
|
||||||
|
subLockAngle: true,
|
||||||
|
subAngleDeg: 69.2,
|
||||||
|
|
||||||
|
controls: true,
|
||||||
|
ctlAutoHideMs: 4000, // in-VR control bar auto-hide
|
||||||
|
|
||||||
|
// written every XR frame by 90-main.js, read by modules
|
||||||
|
pointer: null, // { visible, source, origin:{x,y,z}, dir:{x,y,z}, hit:{nx,ny}|null }
|
||||||
|
|
||||||
|
stats: {},
|
||||||
|
errors: []
|
||||||
|
};
|
||||||
|
|
||||||
|
// live handles (mirrors of state, convenient for modules)
|
||||||
|
Object.defineProperty(NS, 'session', {
|
||||||
|
get: function () { return S.session; },
|
||||||
|
set: function (v) { S.session = v; }
|
||||||
|
});
|
||||||
|
Object.defineProperty(NS, 'video', {
|
||||||
|
get: function () { return S.video; },
|
||||||
|
set: function (v) { S.video = v; }
|
||||||
|
});
|
||||||
|
Object.defineProperty(NS, 'refSpace', {
|
||||||
|
get: function () { return S.refSpace; },
|
||||||
|
set: function (v) { S.refSpace = v; }
|
||||||
|
});
|
||||||
|
Object.defineProperty(NS, 'viewerSpace', {
|
||||||
|
get: function () { return S.viewerSpace; },
|
||||||
|
set: function (v) { S.viewerSpace = v; }
|
||||||
|
});
|
||||||
|
|
||||||
|
NS.recordError = function (where, e) {
|
||||||
|
var msg = where + ': ' + ((e && e.message) || e);
|
||||||
|
S.errors.push(msg);
|
||||||
|
if (S.errors.length > 40) S.errors.shift();
|
||||||
|
return msg;
|
||||||
|
};
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* DOM helpers
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
NS.makeEl = function (tag, className, text) {
|
||||||
|
var el = document.createElement(tag);
|
||||||
|
if (className) el.className = className;
|
||||||
|
if (text != null) el.textContent = text;
|
||||||
|
return el;
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.clamp = function (v, lo, hi) { return v < lo ? lo : (v > hi ? hi : v); };
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* GL helpers
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
NS.compileShader = function (gl, type, src) {
|
||||||
|
var sh = gl.createShader(type);
|
||||||
|
gl.shaderSource(sh, src);
|
||||||
|
gl.compileShader(sh);
|
||||||
|
if (!gl.getShaderParameter(sh, gl.COMPILE_STATUS)) {
|
||||||
|
var info = gl.getShaderInfoLog(sh) || 'shader compile failed';
|
||||||
|
gl.deleteShader(sh);
|
||||||
|
throw new Error(info);
|
||||||
|
}
|
||||||
|
return sh;
|
||||||
|
};
|
||||||
|
|
||||||
|
// Fullscreen triangle in clip space. Fragment shader gets `in vec2 vUv`
|
||||||
|
// with vUv.y already flipped so (0,0) is the top-left of the source image.
|
||||||
|
NS.QUAD_VS = [
|
||||||
|
'#version 300 es',
|
||||||
|
'in vec2 aPos;',
|
||||||
|
'out vec2 vUv;',
|
||||||
|
'void main(){',
|
||||||
|
' vUv = vec2(aPos.x * 0.5 + 0.5, 0.5 - aPos.y * 0.5);',
|
||||||
|
' gl_Position = vec4(aPos, 0.0, 1.0);',
|
||||||
|
'}'
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
NS.bindFullscreenTriangle = function (gl, program) {
|
||||||
|
var buf = gl.createBuffer();
|
||||||
|
gl.bindBuffer(gl.ARRAY_BUFFER, buf);
|
||||||
|
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([-1, -1, 3, -1, -1, 3]), gl.STATIC_DRAW);
|
||||||
|
var loc = gl.getAttribLocation(program, 'aPos');
|
||||||
|
gl.enableVertexAttribArray(loc);
|
||||||
|
gl.vertexAttribPointer(loc, 2, gl.FLOAT, false, 0, 0);
|
||||||
|
return loc;
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.linkProgram = function (gl, vsSrc, fsSrc) {
|
||||||
|
var prog = gl.createProgram();
|
||||||
|
gl.attachShader(prog, NS.compileShader(gl, gl.VERTEX_SHADER, vsSrc));
|
||||||
|
gl.attachShader(prog, NS.compileShader(gl, gl.FRAGMENT_SHADER, fsSrc));
|
||||||
|
gl.linkProgram(prog);
|
||||||
|
if (!gl.getProgramParameter(prog, gl.LINK_STATUS)) {
|
||||||
|
var info = gl.getProgramInfoLog(prog) || 'program link failed';
|
||||||
|
throw new Error(info);
|
||||||
|
}
|
||||||
|
gl.useProgram(prog);
|
||||||
|
return prog;
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.makeTexture = function (gl) {
|
||||||
|
var t = gl.createTexture();
|
||||||
|
gl.bindTexture(gl.TEXTURE_2D, t);
|
||||||
|
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
|
||||||
|
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
|
||||||
|
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
|
||||||
|
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
|
||||||
|
return t;
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.xrGl = function (alpha) {
|
||||||
|
var c = document.createElement('canvas');
|
||||||
|
return c.getContext('webgl2', {
|
||||||
|
xrCompatible: true,
|
||||||
|
alpha: alpha !== false,
|
||||||
|
premultipliedAlpha: true,
|
||||||
|
antialias: false,
|
||||||
|
depth: false,
|
||||||
|
desynchronized: true
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
// 4x4 matrix inverse (column-major, as used by WebXR).
|
||||||
|
NS.invert4 = function (m) {
|
||||||
|
var a00 = m[0], a01 = m[1], a02 = m[2], a03 = m[3],
|
||||||
|
a10 = m[4], a11 = m[5], a12 = m[6], a13 = m[7],
|
||||||
|
a20 = m[8], a21 = m[9], a22 = m[10], a23 = m[11],
|
||||||
|
a30 = m[12], a31 = m[13], a32 = m[14], a33 = m[15],
|
||||||
|
b00 = a00 * a11 - a01 * a10, b01 = a00 * a12 - a02 * a10,
|
||||||
|
b02 = a00 * a13 - a03 * a10, b03 = a01 * a12 - a02 * a11,
|
||||||
|
b04 = a01 * a13 - a03 * a11, b05 = a02 * a13 - a03 * a12,
|
||||||
|
b06 = a20 * a31 - a21 * a30, b07 = a20 * a32 - a22 * a30,
|
||||||
|
b08 = a20 * a33 - a23 * a30, b09 = a21 * a32 - a22 * a31,
|
||||||
|
b10 = a21 * a33 - a23 * a31, b11 = a22 * a33 - a23 * a32,
|
||||||
|
det = b00 * b11 - b01 * b10 + b02 * b09 + b03 * b08 - b04 * b07 + b05 * b06;
|
||||||
|
if (!det) return new Float32Array(16);
|
||||||
|
det = 1.0 / det;
|
||||||
|
return new Float32Array([
|
||||||
|
(a11 * b11 - a12 * b10 + a13 * b09) * det,
|
||||||
|
(a02 * b10 - a01 * b11 - a03 * b09) * det,
|
||||||
|
(a31 * b05 - a32 * b04 + a33 * b03) * det,
|
||||||
|
(a22 * b04 - a21 * b05 - a23 * b03) * det,
|
||||||
|
(a12 * b08 - a10 * b11 - a13 * b07) * det,
|
||||||
|
(a00 * b11 - a02 * b08 + a03 * b07) * det,
|
||||||
|
(a32 * b02 - a30 * b05 - a33 * b01) * det,
|
||||||
|
(a20 * b05 - a22 * b02 + a23 * b01) * det,
|
||||||
|
(a10 * b10 - a11 * b08 + a13 * b06) * det,
|
||||||
|
(a01 * b08 - a00 * b10 - a03 * b06) * det,
|
||||||
|
(a30 * b04 - a31 * b02 + a33 * b00) * det,
|
||||||
|
(a21 * b02 - a20 * b04 - a23 * b00) * det,
|
||||||
|
(a11 * b07 - a10 * b09 - a12 * b06) * det,
|
||||||
|
(a00 * b09 - a01 * b07 + a02 * b06) * det,
|
||||||
|
(a31 * b01 - a30 * b03 - a32 * b00) * det,
|
||||||
|
(a20 * b03 - a21 * b01 + a22 * b00) * det
|
||||||
|
]);
|
||||||
|
};
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* XR layer + frame-hook registry
|
||||||
|
*
|
||||||
|
* Layers are owned by modules; main.js owns pushing the render state.
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
var layers = [];
|
||||||
|
var hooks = [];
|
||||||
|
var pushRenderState = null; // set by main.js once a session exists
|
||||||
|
|
||||||
|
NS.setRenderStatePusher = function (fn) { pushRenderState = fn; };
|
||||||
|
|
||||||
|
NS.addLayer = function (layer) {
|
||||||
|
if (!layer || layers.indexOf(layer) >= 0) return layer;
|
||||||
|
layers.push(layer);
|
||||||
|
NS.log('layer added (' + layers.length + ' total)');
|
||||||
|
if (pushRenderState) { try { pushRenderState(); } catch (e) { NS.recordError('addLayer', e); } }
|
||||||
|
return layer;
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.removeLayer = function (layer) {
|
||||||
|
var i = layers.indexOf(layer);
|
||||||
|
if (i < 0) return;
|
||||||
|
layers.splice(i, 1);
|
||||||
|
NS.log('layer removed (' + layers.length + ' left)');
|
||||||
|
if (pushRenderState) { try { pushRenderState(); } catch (e) { NS.recordError('removeLayer', e); } }
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.getLayers = function () { return layers.slice(); };
|
||||||
|
|
||||||
|
NS.clearLayers = function () {
|
||||||
|
layers.length = 0;
|
||||||
|
if (pushRenderState) { try { pushRenderState(); } catch (e) {} }
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.addFrameHook = function (fn) {
|
||||||
|
if (typeof fn !== 'function' || hooks.indexOf(fn) >= 0) return fn;
|
||||||
|
hooks.push(fn);
|
||||||
|
return fn;
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.removeFrameHook = function (fn) {
|
||||||
|
var i = hooks.indexOf(fn);
|
||||||
|
if (i >= 0) hooks.splice(i, 1);
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.runFrameHooks = function (frame, time, session) {
|
||||||
|
for (var i = 0; i < hooks.length; i++) {
|
||||||
|
try { hooks[i](frame, time, session); }
|
||||||
|
catch (e) {
|
||||||
|
var msg = NS.recordError('frameHook', e);
|
||||||
|
if (!S.stats.hookErr) S.stats.hookErr = msg;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.clearFrameHooks = function () { hooks.length = 0; };
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* stereo / projection helpers shared by modules
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
NS.stereoLayout = function () {
|
||||||
|
return S.stereo === 'sbs' ? 'stereo-left-right'
|
||||||
|
: S.stereo === 'ou' ? 'stereo-top-bottom'
|
||||||
|
: 'mono';
|
||||||
|
};
|
||||||
|
|
||||||
|
NS.centralAngle = function () {
|
||||||
|
return S.projection === '360' ? Math.PI * 2 : Math.PI;
|
||||||
|
};
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* capabilities probe (safe at load time: no user gesture needed)
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
NS.caps = {
|
||||||
|
ua: navigator.userAgent,
|
||||||
|
secureContext: !!window.isSecureContext,
|
||||||
|
hasXR: !!navigator.xr,
|
||||||
|
immersiveVR: null,
|
||||||
|
hasXRMediaBinding: typeof window.XRMediaBinding !== 'undefined',
|
||||||
|
hasEquirectLayer: false,
|
||||||
|
hasXRWebGLBinding: typeof window.XRWebGLBinding !== 'undefined',
|
||||||
|
hasHandTracking: typeof window.XRHand !== 'undefined',
|
||||||
|
hasXRJointSpace: typeof window.XRJointSpace !== 'undefined',
|
||||||
|
hasGetJointPose: typeof window.XRFrame !== 'undefined' && typeof XRFrame.prototype.getJointPose === 'function',
|
||||||
|
maxTextureSize: null,
|
||||||
|
hevc: {},
|
||||||
|
decoding8k: null
|
||||||
|
};
|
||||||
|
|
||||||
|
(function probe() {
|
||||||
|
try {
|
||||||
|
var mb = window.XRMediaBinding;
|
||||||
|
if (mb && mb.prototype) NS.caps.hasEquirectLayer = typeof mb.prototype.createEquirectLayer === 'function';
|
||||||
|
var wg = window.XRWebGLBinding;
|
||||||
|
if (wg && wg.prototype) NS.caps.hasQuadLayer = typeof wg.prototype.createQuadLayer === 'function';
|
||||||
|
} catch (e) {}
|
||||||
|
try {
|
||||||
|
var gl = document.createElement('canvas').getContext('webgl2') ||
|
||||||
|
document.createElement('canvas').getContext('webgl');
|
||||||
|
if (gl) NS.caps.maxTextureSize = gl.getParameter(gl.MAX_TEXTURE_SIZE);
|
||||||
|
} catch (e) {}
|
||||||
|
try {
|
||||||
|
['video/mp4; codecs="hev1.1.6.L183.B0"',
|
||||||
|
'video/mp4; codecs="hvc1.1.6.L183.B0"',
|
||||||
|
'video/mp4; codecs="hev1"',
|
||||||
|
'video/mp4; codecs="hvc1"'].forEach(function (cs) {
|
||||||
|
NS.caps.hevc[cs] = (typeof MediaSource !== 'undefined' && MediaSource.isTypeSupported(cs)) ||
|
||||||
|
document.createElement('video').canPlayType(cs) || 'no';
|
||||||
|
});
|
||||||
|
} catch (e) {}
|
||||||
|
if (navigator.xr && navigator.xr.isSessionSupported) {
|
||||||
|
navigator.xr.isSessionSupported('immersive-vr')
|
||||||
|
.then(function (v) { NS.caps.immersiveVR = v; })
|
||||||
|
.catch(function (e) { NS.caps.immersiveVR = 'err:' + e.message; });
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
|
||||||
|
NS.capsDiag = function () {
|
||||||
|
var c = NS.caps;
|
||||||
|
return [
|
||||||
|
'origin : ' + location.origin,
|
||||||
|
'secureContext : ' + c.secureContext,
|
||||||
|
'immersive-vr : ' + c.immersiveVR,
|
||||||
|
'XRMediaBinding : ' + c.hasXRMediaBinding,
|
||||||
|
'XRWebGLBinding : ' + c.hasXRWebGLBinding,
|
||||||
|
'createQuadLayer : ' + c.hasQuadLayer,
|
||||||
|
'hand tracking API : ' + c.hasHandTracking + ' / jointSpace=' + c.hasXRJointSpace + ' / getJointPose=' + c.hasGetJointPose,
|
||||||
|
'MAX_TEXTURE_SIZE : ' + c.maxTextureSize,
|
||||||
|
'HEVC : ' + JSON.stringify(c.hevc)
|
||||||
|
].join('\n');
|
||||||
|
};
|
||||||
|
})();
|
||||||
+1022
File diff suppressed because it is too large
Load Diff
+1940
File diff suppressed because it is too large
Load Diff
+569
@@ -0,0 +1,569 @@
|
|||||||
|
/* 30-hands.js — __JFVR.Hands
|
||||||
|
*
|
||||||
|
* Draws the official Meta generic-hand models (the @webxr-input-profiles GLBs)
|
||||||
|
* so the user sees their real hands in VR, posed from the 25 WebXR joint poses
|
||||||
|
* the runtime reports. This replaces the hand-rolled laser beam / cursor.
|
||||||
|
*
|
||||||
|
* Three.js draws the models, but it does NOT own the XR session. This page
|
||||||
|
* already composes several layers (an XRMediaBinding equirect video layer plus
|
||||||
|
* quad layers for subtitles and the control panel); calling
|
||||||
|
* renderer.xr.setSession() would install a baseLayer, which is illegal next to
|
||||||
|
* `layers`. So instead:
|
||||||
|
* - we build our own xrCompatible WebGL2 context,
|
||||||
|
* - create a projection layer with XRWebGLBinding and RETURN it, so
|
||||||
|
* 90-main.js can put it in updateRenderState({ layers }),
|
||||||
|
* - drive renderer.render() by hand from a __JFVR frame hook, binding the
|
||||||
|
* layer's swapchain framebuffer and each view's viewport ourselves.
|
||||||
|
*
|
||||||
|
* Interface (INTERFACES.md): init() / create(session) / destroy() / isReady() /
|
||||||
|
* getLayer(). Nothing here reads document, the network or WebXR at load time;
|
||||||
|
* only init()/create() do.
|
||||||
|
*/
|
||||||
|
(function () {
|
||||||
|
'use strict';
|
||||||
|
|
||||||
|
var NS = (typeof window !== 'undefined' && window.__JFVR) ? window.__JFVR : null;
|
||||||
|
if (!NS || NS.Hands) return; // core missing, or already installed
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* constants
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
|
||||||
|
var CDN = 'https://cdn.jsdelivr.net/npm/';
|
||||||
|
var THREE_VERSION = 'three@0.160.0';
|
||||||
|
var THREE_URL = CDN + THREE_VERSION + '/build/three.module.js';
|
||||||
|
var ADDONS_URL = CDN + THREE_VERSION + '/examples/jsm/';
|
||||||
|
var IMPORT_MAP = {
|
||||||
|
imports: {
|
||||||
|
'three': THREE_URL,
|
||||||
|
'three/addons/': ADDONS_URL
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
var HAND_URL = {
|
||||||
|
left: CDN + '@webxr-input-profiles/assets@1.0/dist/profiles/generic-hand/left.glb',
|
||||||
|
right: CDN + '@webxr-input-profiles/assets@1.0/dist/profiles/generic-hand/right.glb'
|
||||||
|
};
|
||||||
|
|
||||||
|
// The 25 WebXR hand joints. The generic-hand GLB names its joint nodes after
|
||||||
|
// these exact strings, so a normalised name lookup maps bone -> joint.
|
||||||
|
var JOINT_NAMES = [
|
||||||
|
'wrist',
|
||||||
|
'thumb-metacarpal', 'thumb-phalanx-proximal', 'thumb-phalanx-distal', 'thumb-tip',
|
||||||
|
'index-finger-metacarpal', 'index-finger-phalanx-proximal', 'index-finger-phalanx-intermediate', 'index-finger-phalanx-distal', 'index-finger-tip',
|
||||||
|
'middle-finger-metacarpal', 'middle-finger-phalanx-proximal', 'middle-finger-phalanx-intermediate', 'middle-finger-phalanx-distal', 'middle-finger-tip',
|
||||||
|
'ring-finger-metacarpal', 'ring-finger-phalanx-proximal', 'ring-finger-phalanx-intermediate', 'ring-finger-phalanx-distal', 'ring-finger-tip',
|
||||||
|
'pinky-finger-metacarpal', 'pinky-finger-phalanx-proximal', 'pinky-finger-phalanx-intermediate', 'pinky-finger-phalanx-distal', 'pinky-finger-tip'
|
||||||
|
];
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* state
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
|
||||||
|
var S = {
|
||||||
|
// asset state: survives sessions, so a re-entry reuses the loaded GLBs
|
||||||
|
inited: false,
|
||||||
|
ready: false,
|
||||||
|
THREE: null,
|
||||||
|
models: { left: null, right: null },
|
||||||
|
|
||||||
|
// session state: torn down by destroy()
|
||||||
|
session: null,
|
||||||
|
gl: null,
|
||||||
|
binding: null,
|
||||||
|
projectionLayer: null,
|
||||||
|
fbo: null,
|
||||||
|
renderer: null,
|
||||||
|
scene: null,
|
||||||
|
camera: null,
|
||||||
|
hookFn: null,
|
||||||
|
frames: 0
|
||||||
|
};
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* helpers
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
|
||||||
|
function log() {
|
||||||
|
try {
|
||||||
|
if (NS && typeof NS.log === 'function') { NS.log.apply(NS, arguments); return; }
|
||||||
|
} catch (e) { /* fall through */ }
|
||||||
|
try { console.log.apply(console, ['[JFVR hands]'].concat(Array.prototype.slice.call(arguments))); } catch (e2) {}
|
||||||
|
}
|
||||||
|
|
||||||
|
function msg(e) {
|
||||||
|
return (e && e.message) ? e.message : String(e);
|
||||||
|
}
|
||||||
|
|
||||||
|
function stats() {
|
||||||
|
if (!NS.state) NS.state = {};
|
||||||
|
if (!NS.state.stats) NS.state.stats = {};
|
||||||
|
return NS.state.stats;
|
||||||
|
}
|
||||||
|
|
||||||
|
function setStat(k, v) {
|
||||||
|
try { stats()[k] = v; } catch (e) {}
|
||||||
|
}
|
||||||
|
|
||||||
|
function setErr(text) {
|
||||||
|
setStat('handsErr', text);
|
||||||
|
}
|
||||||
|
|
||||||
|
function refSpaceOf() {
|
||||||
|
var proj = S.projectionLayer;
|
||||||
|
if (proj && proj.space) return proj.space; // layers with a space
|
||||||
|
if (NS.refSpace) return NS.refSpace;
|
||||||
|
return (NS.state && NS.state.refSpace) || null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function normalizeName(n) {
|
||||||
|
return String(n == null ? '' : n).toLowerCase().replace(/_/g, '-');
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* asset loading (async, kicked off by init())
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
|
||||||
|
// 90-main.js has no import map, so Three's modules cannot resolve their bare
|
||||||
|
// `three` import until we install one. Must happen before the first dynamic
|
||||||
|
// import of a bare specifier, which is why init() does it up front.
|
||||||
|
function ensureImportMap() {
|
||||||
|
if (!document || !document.head) return;
|
||||||
|
if (document.querySelector('script[type="importmap"]')) return;
|
||||||
|
var im = document.createElement('script');
|
||||||
|
im.type = 'importmap';
|
||||||
|
im.textContent = JSON.stringify(IMPORT_MAP);
|
||||||
|
document.head.appendChild(im);
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildModel(gltf, handedness) {
|
||||||
|
var root = gltf && (gltf.scene || (gltf.scenes && gltf.scenes[0]));
|
||||||
|
if (!root) { log('hands[' + handedness + ']: GLB has no scene'); return null; }
|
||||||
|
|
||||||
|
// The mesh is skinned: its bind-pose bounds are wrong once the joints move,
|
||||||
|
// so frustum culling would make the hand pop in and out. Never cull it.
|
||||||
|
root.traverse(function (o) {
|
||||||
|
if (o.isMesh || o.isSkinnedMesh) o.frustumCulled = false;
|
||||||
|
});
|
||||||
|
|
||||||
|
var byName = {};
|
||||||
|
root.traverse(function (o) {
|
||||||
|
if (o.name) byName[normalizeName(o.name)] = o;
|
||||||
|
});
|
||||||
|
|
||||||
|
var bones = {};
|
||||||
|
var missing = [];
|
||||||
|
for (var i = 0; i < JOINT_NAMES.length; i++) {
|
||||||
|
var joint = JOINT_NAMES[i];
|
||||||
|
var bone = byName[normalizeName(joint)];
|
||||||
|
if (bone) bones[joint] = bone;
|
||||||
|
else missing.push(joint);
|
||||||
|
}
|
||||||
|
// Tolerate a renamed/partial skeleton: report once, pose what we can.
|
||||||
|
if (missing.length) {
|
||||||
|
log('hands[' + handedness + ']: no GLB bone for ' + missing.join(', ') + ' (joint skipped)');
|
||||||
|
}
|
||||||
|
return { root: root, bones: bones, missing: missing };
|
||||||
|
}
|
||||||
|
|
||||||
|
function loadModels(THREE, GLTFLoader) {
|
||||||
|
var loader = new GLTFLoader();
|
||||||
|
function one(handedness) {
|
||||||
|
return new Promise(function (resolve, reject) {
|
||||||
|
loader.load(HAND_URL[handedness], function (gltf) {
|
||||||
|
var model = buildModel(gltf, handedness);
|
||||||
|
if (!model) { reject(new Error(handedness + ': empty scene')); return; }
|
||||||
|
S.models[handedness] = model;
|
||||||
|
if (S.scene) S.scene.add(model.root); // layer may already be rendering
|
||||||
|
resolve();
|
||||||
|
}, undefined, function (e) {
|
||||||
|
reject(new Error(handedness + ': ' + msg(e)));
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return Promise.all([one('left'), one('right')]);
|
||||||
|
}
|
||||||
|
|
||||||
|
function load() {
|
||||||
|
return import('three').then(function (THREE) {
|
||||||
|
S.THREE = THREE;
|
||||||
|
return import('three/addons/loaders/GLTFLoader.js').then(function (mod) {
|
||||||
|
return loadModels(THREE, mod.GLTFLoader);
|
||||||
|
});
|
||||||
|
}).then(function () {
|
||||||
|
S.ready = true;
|
||||||
|
setStat('handsReady', true);
|
||||||
|
setErr(null);
|
||||||
|
ensureRenderer(); // if create() already ran, start drawing the models
|
||||||
|
log('hand models ready');
|
||||||
|
}).catch(function (e) {
|
||||||
|
setErr('load: ' + msg(e));
|
||||||
|
log('hand model load failed:', msg(e));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function init() {
|
||||||
|
if (S.inited) return;
|
||||||
|
S.inited = true;
|
||||||
|
setStat('handsReady', false);
|
||||||
|
setStat('handsErr', null);
|
||||||
|
setStat('handsSources', 'none');
|
||||||
|
setStat('handsFrames', 0);
|
||||||
|
try {
|
||||||
|
ensureImportMap();
|
||||||
|
var p = load();
|
||||||
|
if (p && typeof p.catch === 'function') {
|
||||||
|
p.catch(function (e) { setErr('init: ' + msg(e)); }); // belt and braces
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
// Never propagate: a missing import map / document must not kill boot.
|
||||||
|
setErr('init: ' + msg(e));
|
||||||
|
log('hands init failed:', msg(e));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* three.js + scene (built lazily, once THREE and a session are available)
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
|
||||||
|
function makeGl() {
|
||||||
|
if (!document || !document.createElement) return null;
|
||||||
|
var canvas = document.createElement('canvas');
|
||||||
|
// A dedicated context: the other modules keep their own for the quad
|
||||||
|
// layers, and the projection layer must render into its own swapchain.
|
||||||
|
return canvas.getContext('webgl2', {
|
||||||
|
xrCompatible: true,
|
||||||
|
alpha: true,
|
||||||
|
premultipliedAlpha: true,
|
||||||
|
antialias: true,
|
||||||
|
depth: true,
|
||||||
|
stencil: false
|
||||||
|
}) || null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function ensureRenderer() {
|
||||||
|
if (S.renderer || !S.gl || !S.THREE) return S.renderer;
|
||||||
|
try {
|
||||||
|
var THREE = S.THREE;
|
||||||
|
var renderer = new THREE.WebGLRenderer({
|
||||||
|
canvas: S.gl.canvas,
|
||||||
|
context: S.gl,
|
||||||
|
alpha: true,
|
||||||
|
antialias: true
|
||||||
|
});
|
||||||
|
// NEVER renderer.xr.setSession(): that sets a baseLayer and would clash
|
||||||
|
// with the page's `layers` array. We bind the projection framebuffer and
|
||||||
|
// render manually from the frame hook instead.
|
||||||
|
renderer.autoClear = true;
|
||||||
|
renderer.setClearColor(0x000000, 0); // transparent: the video shows through
|
||||||
|
|
||||||
|
var scene = new THREE.Scene();
|
||||||
|
// The generic-hand GLBs use PBR materials; without a light they render
|
||||||
|
// black, so give them a hemisphere fill plus a key light.
|
||||||
|
scene.add(new THREE.HemisphereLight(0xffffff, 0x445566, 2.5));
|
||||||
|
var key = new THREE.DirectionalLight(0xffffff, 1.4);
|
||||||
|
key.position.set(0.3, 0.8, 0.5);
|
||||||
|
scene.add(key);
|
||||||
|
|
||||||
|
var camera = new THREE.PerspectiveCamera();
|
||||||
|
// The hook writes this camera's matrices straight from the XR view, so
|
||||||
|
// stop three from recomputing matrixWorld from position/quaternion.
|
||||||
|
camera.matrixAutoUpdate = false;
|
||||||
|
camera.matrixWorldAutoUpdate = false;
|
||||||
|
|
||||||
|
S.renderer = renderer;
|
||||||
|
S.scene = scene;
|
||||||
|
S.camera = camera;
|
||||||
|
['left', 'right'].forEach(function (h) {
|
||||||
|
if (S.models[h]) scene.add(S.models[h].root);
|
||||||
|
});
|
||||||
|
log('three renderer ready for hands');
|
||||||
|
} catch (e) {
|
||||||
|
setErr('renderer: ' + msg(e));
|
||||||
|
log('renderer setup failed:', msg(e));
|
||||||
|
S.renderer = null;
|
||||||
|
S.scene = null;
|
||||||
|
S.camera = null;
|
||||||
|
}
|
||||||
|
return S.renderer;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* XR projection layer + manual render
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
|
||||||
|
// Resolve the framebuffer + viewport to render `view` into.
|
||||||
|
//
|
||||||
|
// Two shapes are accepted, in order:
|
||||||
|
// 1. a projection layer that exposes `framebuffer` + `getViewport(view)`;
|
||||||
|
// 2. the WebXR Layers spec path, XRWebGLBinding.getViewSubImage(layer, view),
|
||||||
|
// which returns this view's viewport plus the swapchain colour/depth
|
||||||
|
// textures; we attach those to a framebuffer of our own. (The spec gives
|
||||||
|
// XRProjectionLayer no framebuffer and no getViewport(), so on a real
|
||||||
|
// runtime branch 2 is the one that runs.)
|
||||||
|
function bindViewTarget(view) {
|
||||||
|
var gl = S.gl;
|
||||||
|
var proj = S.projectionLayer;
|
||||||
|
var binding = S.binding;
|
||||||
|
if (!gl || !proj) return null;
|
||||||
|
|
||||||
|
if (proj.framebuffer && typeof proj.getViewport === 'function') {
|
||||||
|
var direct = null;
|
||||||
|
try { direct = proj.getViewport(view); } catch (e) { direct = null; }
|
||||||
|
if (direct) {
|
||||||
|
gl.bindFramebuffer(gl.FRAMEBUFFER, proj.framebuffer);
|
||||||
|
return { framebuffer: proj.framebuffer, viewport: direct };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (binding && typeof binding.getViewSubImage === 'function') {
|
||||||
|
var sub = null;
|
||||||
|
try { sub = binding.getViewSubImage(proj, view); } catch (e) { sub = null; }
|
||||||
|
if (sub && sub.colorTexture) {
|
||||||
|
if (!S.fbo) S.fbo = gl.createFramebuffer();
|
||||||
|
gl.bindFramebuffer(gl.FRAMEBUFFER, S.fbo);
|
||||||
|
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.COLOR_ATTACHMENT0, gl.TEXTURE_2D, sub.colorTexture, 0);
|
||||||
|
if (sub.depthStencilTexture) {
|
||||||
|
gl.framebufferTexture2D(gl.FRAMEBUFFER, gl.DEPTH_ATTACHMENT, gl.TEXTURE_2D, sub.depthStencilTexture, 0);
|
||||||
|
}
|
||||||
|
if (sub.viewport) return { framebuffer: S.fbo, viewport: sub.viewport };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Blank the layer while Three has not finished loading, so the compositor
|
||||||
|
// does not show stale content.
|
||||||
|
function clearLayer(pose) {
|
||||||
|
var gl = S.gl;
|
||||||
|
var target = bindViewTarget(pose.views[0]);
|
||||||
|
if (!target) return;
|
||||||
|
var vp = target.viewport;
|
||||||
|
gl.viewport(Math.round(vp.x), Math.round(vp.y), Math.round(vp.width), Math.round(vp.height));
|
||||||
|
gl.clearColor(0, 0, 0, 0);
|
||||||
|
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderFrame(frame) {
|
||||||
|
var space = refSpaceOf();
|
||||||
|
if (!space) return;
|
||||||
|
var pose = null;
|
||||||
|
try { pose = frame.getViewerPose(space); } catch (e) { pose = null; }
|
||||||
|
if (!pose || !pose.views || !pose.views.length) return;
|
||||||
|
|
||||||
|
var renderer = S.renderer || ensureRenderer();
|
||||||
|
if (!renderer) { clearLayer(pose); return; }
|
||||||
|
|
||||||
|
var gl = S.gl;
|
||||||
|
var camera = S.camera;
|
||||||
|
|
||||||
|
// The layer colour texture holds BOTH eyes in one framebuffer, so a bare
|
||||||
|
// gl.clear() would erase the eye already drawn. autoClear stays on (it
|
||||||
|
// clears to the transparent clear colour above) and the scissor box keeps
|
||||||
|
// each clear + draw inside that eye's viewport.
|
||||||
|
gl.enable(gl.SCISSOR_TEST);
|
||||||
|
for (var i = 0; i < pose.views.length; i++) {
|
||||||
|
var view = pose.views[i];
|
||||||
|
var target = bindViewTarget(view);
|
||||||
|
if (!target) break;
|
||||||
|
var vp = target.viewport;
|
||||||
|
var x = Math.round(vp.x), y = Math.round(vp.y);
|
||||||
|
var w = Math.round(vp.width), h = Math.round(vp.height);
|
||||||
|
if (w <= 0 || h <= 0) continue;
|
||||||
|
|
||||||
|
gl.viewport(x, y, w, h);
|
||||||
|
gl.scissor(x, y, w, h);
|
||||||
|
|
||||||
|
camera.projectionMatrix.fromArray(view.projectionMatrix);
|
||||||
|
camera.projectionMatrixInverse.copy(camera.projectionMatrix).invert();
|
||||||
|
camera.matrixWorldInverse.fromArray(view.transform.inverse.matrix);
|
||||||
|
camera.matrixWorld.copy(camera.matrixWorldInverse).invert();
|
||||||
|
|
||||||
|
renderer.render(S.scene, camera);
|
||||||
|
}
|
||||||
|
gl.disable(gl.SCISSOR_TEST);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* joint posing
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
|
||||||
|
function poseHand(frame, handedness, hand, space) {
|
||||||
|
var model = S.models[handedness];
|
||||||
|
if (!model || !model.bones || !space) return;
|
||||||
|
|
||||||
|
for (var i = 0; i < JOINT_NAMES.length; i++) {
|
||||||
|
var joint = JOINT_NAMES[i];
|
||||||
|
var bone = model.bones[joint];
|
||||||
|
if (!bone) continue; // unmatched bone: logged once at load
|
||||||
|
|
||||||
|
var jointSpace = null;
|
||||||
|
try { jointSpace = hand.get(joint); } catch (e) { jointSpace = null; }
|
||||||
|
if (!jointSpace) continue;
|
||||||
|
|
||||||
|
var jp = null;
|
||||||
|
try { jp = frame.getJointPose(jointSpace, space); } catch (e) { jp = null; }
|
||||||
|
if (!jp || !jp.transform) continue;
|
||||||
|
|
||||||
|
// The generic-hand skeleton is FLAT: every joint node is a direct child
|
||||||
|
// of the armature root, which sits at identity. A joint's pose in the
|
||||||
|
// reference space is therefore also its local transform, so feeding the
|
||||||
|
// matrix to the bone's TRS and letting three propagate is exact. The
|
||||||
|
// model is metre-scaled (hand span ~0.2 m), so jp.radius is NOT applied:
|
||||||
|
// the mesh already matches the joint scale.
|
||||||
|
bone.matrix.fromArray(jp.transform.matrix);
|
||||||
|
bone.matrix.decompose(bone.position, bone.quaternion, bone.scale);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function poseHands(frame, session) {
|
||||||
|
var left = false, right = false;
|
||||||
|
var sources = (session && session.inputSources) || null;
|
||||||
|
var space = refSpaceOf();
|
||||||
|
|
||||||
|
if (sources) {
|
||||||
|
for (var i = 0; i < sources.length; i++) {
|
||||||
|
var src = sources[i];
|
||||||
|
var hand = src && src.hand;
|
||||||
|
if (!hand || hand.size !== 25) continue; // XRHand: exactly 25 joints
|
||||||
|
var handedness = src.handedness;
|
||||||
|
if (handedness !== 'left' && handedness !== 'right') continue;
|
||||||
|
if (handedness === 'left') left = true; else right = true;
|
||||||
|
poseHand(frame, handedness, hand, space);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Show a hand only while its input source is present.
|
||||||
|
if (S.models.left && S.models.left.root) S.models.left.root.visible = left;
|
||||||
|
if (S.models.right && S.models.right.root) S.models.right.root.visible = right;
|
||||||
|
|
||||||
|
setStat('handsSources', (left || right)
|
||||||
|
? (left ? 'left' : '') + (left && right ? ',' : '') + (right ? 'right' : '')
|
||||||
|
: 'none');
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* frame hook
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
|
||||||
|
function hook(frame, time, session) {
|
||||||
|
if (!S.projectionLayer || !frame) return;
|
||||||
|
S.frames++;
|
||||||
|
setStat('handsFrames', S.frames);
|
||||||
|
poseHands(frame, session);
|
||||||
|
renderFrame(frame);
|
||||||
|
void time;
|
||||||
|
}
|
||||||
|
|
||||||
|
function registerHook() {
|
||||||
|
if (S.hookFn) return;
|
||||||
|
S.hookFn = hook;
|
||||||
|
if (typeof NS.addFrameHook === 'function') NS.addFrameHook(hook);
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* lifecycle
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
|
||||||
|
function create(session) {
|
||||||
|
try {
|
||||||
|
if (S.projectionLayer && S.session === session) return S.projectionLayer; // idempotent
|
||||||
|
if (!session) { log('hands.create: no session'); return null; }
|
||||||
|
|
||||||
|
destroy(); // drop anything left from a previous session
|
||||||
|
|
||||||
|
if (typeof XRWebGLBinding === 'undefined') {
|
||||||
|
setErr('no XRWebGLBinding');
|
||||||
|
log('hands.create: XRWebGLBinding unavailable');
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
var gl = makeGl();
|
||||||
|
if (!gl) { setErr('no webgl2 context'); log('hands.create: no webgl2 context'); return null; }
|
||||||
|
|
||||||
|
var binding = new XRWebGLBinding(session, gl);
|
||||||
|
var proj = binding.createProjectionLayer({
|
||||||
|
textureType: 'texture',
|
||||||
|
colorFormat: gl.RGBA8,
|
||||||
|
depthFormat: gl.DEPTH_COMPONENT24,
|
||||||
|
scaleFactor: 1,
|
||||||
|
alpha: true
|
||||||
|
});
|
||||||
|
if (!proj) { setErr('createProjectionLayer returned null'); log('hands.create: no projection layer'); return null; }
|
||||||
|
|
||||||
|
S.session = session;
|
||||||
|
S.gl = gl;
|
||||||
|
S.binding = binding;
|
||||||
|
S.projectionLayer = proj;
|
||||||
|
S.fbo = null;
|
||||||
|
S.frames = 0;
|
||||||
|
|
||||||
|
ensureRenderer(); // no-op until THREE has loaded; the hook retries
|
||||||
|
registerHook();
|
||||||
|
setErr(null);
|
||||||
|
log('hands projection layer created (models ' + (S.ready ? 'ready' : 'loading') + ')');
|
||||||
|
return proj;
|
||||||
|
} catch (e) {
|
||||||
|
setErr('create: ' + msg(e));
|
||||||
|
log('hands.create failed:', msg(e));
|
||||||
|
try { destroy(); } catch (e2) { void e2; }
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function destroy() {
|
||||||
|
try {
|
||||||
|
if (S.hookFn && typeof NS.removeFrameHook === 'function') NS.removeFrameHook(S.hookFn);
|
||||||
|
} catch (e) { void e; }
|
||||||
|
S.hookFn = null;
|
||||||
|
|
||||||
|
try {
|
||||||
|
if (S.projectionLayer && typeof NS.removeLayer === 'function') NS.removeLayer(S.projectionLayer);
|
||||||
|
} catch (e) { void e; }
|
||||||
|
|
||||||
|
try { if (S.gl && S.fbo) S.gl.deleteFramebuffer(S.fbo); } catch (e) { void e; }
|
||||||
|
S.fbo = null;
|
||||||
|
|
||||||
|
var renderer = S.renderer;
|
||||||
|
S.renderer = null;
|
||||||
|
if (renderer) {
|
||||||
|
// Drop the context with the session: browsers cap live WebGL contexts,
|
||||||
|
// and the next create() builds a bright new one. Only our own resources
|
||||||
|
// are touched - the loaded models are kept for reuse.
|
||||||
|
try { if (typeof renderer.forceContextLoss === 'function') renderer.forceContextLoss(); } catch (e) { void e; }
|
||||||
|
try { renderer.dispose(); } catch (e) { void e; }
|
||||||
|
}
|
||||||
|
|
||||||
|
S.scene = null;
|
||||||
|
S.camera = null;
|
||||||
|
S.binding = null;
|
||||||
|
S.projectionLayer = null;
|
||||||
|
S.gl = null;
|
||||||
|
S.session = null;
|
||||||
|
S.frames = 0;
|
||||||
|
setStat('handsFrames', 0);
|
||||||
|
setStat('handsSources', 'none');
|
||||||
|
}
|
||||||
|
|
||||||
|
function isReady() {
|
||||||
|
// Models loaded AND a live projection layer + renderer: genuinely drawing.
|
||||||
|
return !!(S.ready && S.renderer && S.projectionLayer);
|
||||||
|
}
|
||||||
|
|
||||||
|
function getLayer() {
|
||||||
|
return S.projectionLayer || null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ------------------------------------------------------------------ *
|
||||||
|
* public API
|
||||||
|
* ------------------------------------------------------------------ */
|
||||||
|
|
||||||
|
NS.Hands = {
|
||||||
|
init: init,
|
||||||
|
create: create,
|
||||||
|
destroy: destroy,
|
||||||
|
isReady: isReady,
|
||||||
|
getLayer: getLayer
|
||||||
|
};
|
||||||
|
})();
|
||||||
+1412
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user