hud-10: the VR pointer follows the hand that is acting or aiming
ADDENDUM 11. With two hands both report a tracked-pointer ray and the runtime's order is not "the hand the user is using". updatePointer() took the first valid source and broke, and pointerDown() called updatePointer(frame) without the select event's inputSource. On device that pinned the guide line/cursor to one hand while the user aimed with the other, and made the other hand's pinch land wherever the pinned hand pointed (left hand controlling the click position, right hand clicking). updatePointer(frame, preferSrc) now builds a candidate per tracked-pointer source and pickPointerCand() chooses: the acting source, else the last-active source while it is on the panel, else any source on the panel, else the last-active source, else the first. pointerDown() passes the event's inputSource, so an action is always evaluated with the acting hand's ray, and the choice sticks afterwards. stats.activeSrc reports it; session end resets it. pickPointerCand is pure and unit-tested against the built bundle (9 cases, including the reported one: guide on the left + right pinch -> hand:right, and it stays). Verified in the headset only up to "no hands tracked at the time"; the live hand switch still needs a worn headset.
This commit is contained in:
+35
-2
@@ -1,10 +1,11 @@
|
||||
# Jellyfin Web VR180 — 项目交接文档
|
||||
|
||||
> 给新对话用。读完这份就能接手,不需要重新调查环境。
|
||||
> 最后更新:已部署版本 `e394bcb`(`NS.version = hud-9`;ADDENDUM 6 字幕深度/大小解耦
|
||||
> 最后更新:已部署版本 `6bf1021`(`NS.version = hud-10`;ADDENDUM 6 字幕深度/大小解耦
|
||||
> + 7 唤出门限/前面板精简 + 7b 锁按钮与「锁定」面板模式 + 8/8b 画面搬迁(实时拖动)
|
||||
> + 9 rAF 代理修复(VR 内字幕不再冻结)+ 普通/双目字幕自动识别
|
||||
> + 10 原生(SRT 等)字幕由插件自绘到 HUD;布局覆盖不再持久化)
|
||||
> + 10 原生(SRT 等)字幕由插件自绘到 HUD;布局覆盖不再持久化
|
||||
> + 11 指针跟随正在操作/瞄准的那只手)
|
||||
>
|
||||
> ⚠️ **Jellyfin web 注册了 service worker**(`serviceworker.js`,scope `/web/`),会把
|
||||
> `index.html` 缓存在 SW cache 里。重新部署后如果只是普通刷新,浏览器可能仍用旧的
|
||||
@@ -416,6 +417,7 @@ subMode / subCanvasSize / subCrop / subTex / subOpacity / subPlacement
|
||||
subLockAngle / subAngleDeg / subTextDeg / subYOverZ ← ADDENDUM 6
|
||||
subLayout / subDual / subEyeW / subCopyND / subCenterInk ← ADDENDUM 9
|
||||
subSource / subNativeText ← ADDENDUM 10
|
||||
activeSrc ← ADDENDUM 11
|
||||
ctlSubsOn / ctlSubDist / ctlSubAngle / ctlSubLock ← 控制条实际画出的值
|
||||
ctlLock / ctlMode / ctlHold / pressAt / lastActivate ← ADDENDUM 7/7b 锁、面板保持与唤出判定
|
||||
videoYaw / videoPitch / videoRotMode / videoTransformWritable ← ADDENDUM 8/8b 画面搬迁
|
||||
@@ -628,3 +630,34 @@ HUD 把两只眼的字幕并排显示。找这个 bug 花的时间比写检测
|
||||
私有 cue canvas 还读回做了 OCR(cue `怎么样?好好地` → OCR `怎么样? 好好地`);
|
||||
注入/移除一个假的 `canvas.libassjs-canvas` 能让 `subSource` 在 `libass`/`native`
|
||||
间切换且 `subErr` 为空。
|
||||
|
||||
---
|
||||
|
||||
## 15. ADDENDUM 11 — 指针跟随「正在操作/正在瞄准」的那只手(已部署 `6bf1021` / `hud-10`)
|
||||
|
||||
**根因**:双手都会报 `tracked-pointer` 射线,而运行时的顺序不是「用户正在用的那只手」。
|
||||
`updatePointer()` 取**第一个**有有效射线的输入源就 `break`;而 `pointerDown()` 调用的是
|
||||
`updatePointer(frame)`——**没有把 `select` 事件里的 `inputSource` 传进去**。于是设备上:
|
||||
|
||||
- 引导线/光标永远钉在一只手上(用户用另一只手指也不跟)→「引导线并没有指向光标」;
|
||||
- 另一只手的捏合**落在被钉住那只手所指的位置**:左手控制点击位置,右手完成点击。
|
||||
|
||||
**修复**(`src/90-main.js`):`updatePointer(frame, preferSrc)` 为每个 tracked-pointer
|
||||
源各算一次命中,再用纯函数 `pickPointerCand(cands, preferKey, activeKey)` 选:
|
||||
|
||||
```
|
||||
1. preferSrc —— 当前正在操作的那个源(select / gamepad 按下)
|
||||
2. 上次操作的源 —— 前提是它正瞄准面板(hit.inBounds)
|
||||
3. 任一个正在瞄准面板的源(运行时顺序)
|
||||
4. 上次操作的源(粘滞)
|
||||
5. 第一个源
|
||||
```
|
||||
|
||||
`pointerDown()` 传入事件的 `inputSource`,所以每次操作都用**操作那只手**的射线判定;
|
||||
选中的手之后保持粘滞。诊断新增 `stats.activeSrc`,会话结束时清空。
|
||||
|
||||
**验证**:`pickPointerCand` 是纯函数,直接从构建产物提取做了单测——包含你描述的场景
|
||||
(引导线在左手 → 右手捏合)现在得到 `hand:right` 并保持;另有「左手接管」「操作手不在
|
||||
面板上、另一只手在瞄 → 跟随瞄准手」「都没瞄 → 保持上次操作手」等 9 个用例全过。
|
||||
|
||||
> 真机复核需要戴上手(当时 `inputSources` 为空、没有手部源,无法现场触发)。
|
||||
|
||||
@@ -1135,3 +1135,33 @@ side. Debugging that cost more time than the detection itself.
|
||||
The private cue canvas was read back and OCR'd ("怎么样? 好好地" for cue
|
||||
"怎么样?好好地"), and injecting/removing a fake `canvas.libassjs-canvas` flips
|
||||
`subSource` between `libass` and `native` with no errors.
|
||||
|
||||
# ADDENDUM 11 — the pointer follows the hand that is acting / aiming
|
||||
|
||||
With two hands, BOTH report a `tracked-pointer` ray, and the runtime's order is not
|
||||
"the hand the user is using". `updatePointer()` took the **first** input source with
|
||||
a valid ray and `break`ed, and `pointerDown()` called `updatePointer(frame)` without
|
||||
passing the source from the `select` event. Consequences on device:
|
||||
|
||||
- the guide line / cursor stayed pinned to one hand even while the user aimed with
|
||||
the other (so the guide did not point where they were pointing);
|
||||
- the other hand's pinch landed **wherever the pinned hand pointed** — the left hand
|
||||
controlled the click position while the right hand performed the click.
|
||||
|
||||
`updatePointer(frame, preferSrc)` now builds a candidate per tracked-pointer source
|
||||
(with its `panelHit`), and `pickPointerCand()` chooses:
|
||||
|
||||
```
|
||||
1. preferSrc the source acting right now (select / gamepad press)
|
||||
2. last-active source while it is aiming at the panel (hit.inBounds)
|
||||
3. any source aiming at the panel (first in runtime order)
|
||||
4. last-active source (sticky)
|
||||
5. the first source
|
||||
```
|
||||
|
||||
`pointerDown()` passes the event's `inputSource`, so an action is always evaluated
|
||||
with the acting hand's ray, and the choice sticks afterwards (`state.stats.activeSrc`).
|
||||
`pointerUp`/session end reset it.
|
||||
|
||||
`pickPointerCand` is pure and unit-tested against the built bundle; the reported case
|
||||
(guide on the left, right hand pinches) now yields `hand:right` and stays there.
|
||||
|
||||
Vendored
+64
-11
@@ -19,7 +19,7 @@
|
||||
|
||||
var NS = window.__JFVR = window.__JFVR || {};
|
||||
|
||||
NS.version = 'hud-9'; // hud-9 = ADDENDUM 10 (native/TextTrack subtitles painted by the HUD; layout override per-session, ADDENDUM 9 rAF broker + auto dual/mono layout)
|
||||
NS.version = 'hud-10'; // hud-10 = ADDENDUM 11 (pointer follows the acting/aiming hand) + ADDENDUM 10 (native subtitles) + 9
|
||||
NS.__core = true;
|
||||
|
||||
/* ------------------------------------------------------------------ *
|
||||
@@ -5246,6 +5246,7 @@
|
||||
S.ctlLock = false;
|
||||
panelPose = null;
|
||||
pointer = { visible: false, beam: false, source: 'none', origin: { x: 0, y: 0, z: 0 }, dir: { x: 0, y: 0, z: -1 }, hit: null };
|
||||
activeSrcKey = null; // ADDENDUM 11: fresh hand choice every session
|
||||
S.pointer = null;
|
||||
S.session = null;
|
||||
S.refSpace = null;
|
||||
@@ -5432,7 +5433,39 @@
|
||||
// the only mode (an index-finger ray carries fingertip jitter straight into
|
||||
// the cursor, and nobody wants to point with one finger).
|
||||
|
||||
function updatePointer(frame) {
|
||||
// Identity for an input source that survives the runtime re-listing it.
|
||||
function srcKey(src) {
|
||||
if (!src) return null;
|
||||
return (src.hand ? 'hand' : 'ctrl') + ':' + (src.handedness || '?');
|
||||
}
|
||||
|
||||
function candByKey(cands, key) {
|
||||
for (var i = 0; i < cands.length; i++) if (cands[i].key === key) return cands[i];
|
||||
return null;
|
||||
}
|
||||
|
||||
// ADDENDUM 11 — pure: choose which input source owns the pointer.
|
||||
// cands [{ key, onPanel, ... }] in runtime order
|
||||
// preferKey the source that is acting right now (select / gamepad press)
|
||||
// activeKey the source that acted last
|
||||
// Order: acting -> last-active-while-on-panel -> any-on-panel -> last-active -> first.
|
||||
function pickPointerCand(cands, preferKey, activeKey) {
|
||||
if (!cands || !cands.length) return null;
|
||||
var i, c;
|
||||
if (preferKey) { c = candByKey(cands, preferKey); if (c) return c; }
|
||||
if (activeKey) {
|
||||
c = candByKey(cands, activeKey);
|
||||
if (c && c.onPanel) return c;
|
||||
}
|
||||
for (i = 0; i < cands.length; i++) if (cands[i].onPanel) return cands[i];
|
||||
if (activeKey) { c = candByKey(cands, activeKey); if (c) return c; }
|
||||
return cands[0];
|
||||
}
|
||||
|
||||
// ADDENDUM 11: which hand owns the pointer.
|
||||
var activeSrcKey = null;
|
||||
|
||||
function updatePointer(frame, preferSrc) {
|
||||
if (!S.session) return;
|
||||
var next = {
|
||||
visible: false, beam: false, source: 'none',
|
||||
@@ -5450,24 +5483,43 @@
|
||||
// platform; an index-finger ray computed from joint poses carries the
|
||||
// per-frame fingertip jitter straight into the cursor, so its landing point
|
||||
// is visibly less steady. Meta's own samples use targetRaySpace too.
|
||||
//
|
||||
// ADDENDUM 11: with two hands BOTH report a tracked-pointer ray, and the
|
||||
// runtime's order is not "the hand the user is using". Taking the first one
|
||||
// pinned the guide line to one hand AND made the OTHER hand's select land
|
||||
// wherever the pinned hand pointed. The pointer now follows, in order:
|
||||
// 1. the source acting right now (select / gamepad press),
|
||||
// 2. the last acting source, while it is aiming at the panel,
|
||||
// 3. any source aiming at the panel,
|
||||
// 4. the last acting source, then the first one.
|
||||
var cands = [];
|
||||
for (var i = 0; i < srcs.length; i++) {
|
||||
var src = srcs[i];
|
||||
if (src.targetRayMode !== 'tracked-pointer') continue;
|
||||
var isHand = !!src.hand;
|
||||
var ray = rayOf(frame, src);
|
||||
if (!ray) continue;
|
||||
var hit = panelHit(frame, ray, layer, 0.25, 0.6);
|
||||
var hit = null;
|
||||
try { hit = panelHit(frame, ray, layer, 0.25, 0.6); } catch (e) { hit = null; }
|
||||
cands.push({
|
||||
src: src, ray: ray, hit: hit, key: srcKey(src),
|
||||
onPanel: !!(hit && hit.inBounds)
|
||||
});
|
||||
}
|
||||
|
||||
var pick = pickPointerCand(cands, preferSrc ? srcKey(preferSrc) : null, activeSrcKey);
|
||||
|
||||
if (pick) {
|
||||
activeSrcKey = pick.key;
|
||||
next = {
|
||||
visible: true,
|
||||
beam: true,
|
||||
source: (isHand ? 'hand' : 'controller') + ':' + src.handedness,
|
||||
origin: ray.origin,
|
||||
dir: ray.dir,
|
||||
hit: hit ? { nx: hit.nx, ny: hit.ny } : null,
|
||||
distance: hit ? hit.distance : null,
|
||||
source: (pick.src.hand ? 'hand' : 'controller') + ':' + pick.src.handedness,
|
||||
origin: pick.ray.origin,
|
||||
dir: pick.ray.dir,
|
||||
hit: pick.hit ? { nx: pick.hit.nx, ny: pick.hit.ny } : null,
|
||||
distance: pick.hit ? pick.hit.distance : null,
|
||||
pressed: performance.now() < pressUntil
|
||||
};
|
||||
break;
|
||||
}
|
||||
|
||||
pointer = next;
|
||||
@@ -5476,6 +5528,7 @@
|
||||
if (NS.Controls && NS.Controls.setPointer) {
|
||||
try { NS.Controls.setPointer(next); } catch (e) {}
|
||||
}
|
||||
S.stats.activeSrc = activeSrcKey || '(none)';
|
||||
S.stats.pointer = next.source +
|
||||
(next.hit ? ' HIT ' + next.hit.nx.toFixed(2) + ',' + next.hit.ny.toFixed(2) : ' (no hit)');
|
||||
S.stats.inputSources = srcs.map(function (s) {
|
||||
@@ -5975,7 +6028,7 @@
|
||||
// to the bar, so it must not also yank the subtitles.
|
||||
var barWasShown = NS.Controls && NS.Controls.isVisible ? !!NS.Controls.isVisible() : false;
|
||||
|
||||
if (frame) { try { updatePointer(frame); } catch (e) {} }
|
||||
if (frame) { try { updatePointer(frame, src); } catch (e) {} }
|
||||
|
||||
// 1) grabbing the subtitle strip wins, but only while the bar is down
|
||||
if (frame && !barWasShown) {
|
||||
|
||||
+1
-1
@@ -11,7 +11,7 @@
|
||||
|
||||
var NS = window.__JFVR = window.__JFVR || {};
|
||||
|
||||
NS.version = 'hud-9'; // hud-9 = ADDENDUM 10 (native/TextTrack subtitles painted by the HUD; layout override per-session, ADDENDUM 9 rAF broker + auto dual/mono layout)
|
||||
NS.version = 'hud-10'; // hud-10 = ADDENDUM 11 (pointer follows the acting/aiming hand) + ADDENDUM 10 (native subtitles) + 9
|
||||
NS.__core = true;
|
||||
|
||||
/* ------------------------------------------------------------------ *
|
||||
|
||||
+63
-10
@@ -850,6 +850,7 @@
|
||||
S.ctlLock = false;
|
||||
panelPose = null;
|
||||
pointer = { visible: false, beam: false, source: 'none', origin: { x: 0, y: 0, z: 0 }, dir: { x: 0, y: 0, z: -1 }, hit: null };
|
||||
activeSrcKey = null; // ADDENDUM 11: fresh hand choice every session
|
||||
S.pointer = null;
|
||||
S.session = null;
|
||||
S.refSpace = null;
|
||||
@@ -1036,7 +1037,39 @@
|
||||
// the only mode (an index-finger ray carries fingertip jitter straight into
|
||||
// the cursor, and nobody wants to point with one finger).
|
||||
|
||||
function updatePointer(frame) {
|
||||
// Identity for an input source that survives the runtime re-listing it.
|
||||
function srcKey(src) {
|
||||
if (!src) return null;
|
||||
return (src.hand ? 'hand' : 'ctrl') + ':' + (src.handedness || '?');
|
||||
}
|
||||
|
||||
function candByKey(cands, key) {
|
||||
for (var i = 0; i < cands.length; i++) if (cands[i].key === key) return cands[i];
|
||||
return null;
|
||||
}
|
||||
|
||||
// ADDENDUM 11 — pure: choose which input source owns the pointer.
|
||||
// cands [{ key, onPanel, ... }] in runtime order
|
||||
// preferKey the source that is acting right now (select / gamepad press)
|
||||
// activeKey the source that acted last
|
||||
// Order: acting -> last-active-while-on-panel -> any-on-panel -> last-active -> first.
|
||||
function pickPointerCand(cands, preferKey, activeKey) {
|
||||
if (!cands || !cands.length) return null;
|
||||
var i, c;
|
||||
if (preferKey) { c = candByKey(cands, preferKey); if (c) return c; }
|
||||
if (activeKey) {
|
||||
c = candByKey(cands, activeKey);
|
||||
if (c && c.onPanel) return c;
|
||||
}
|
||||
for (i = 0; i < cands.length; i++) if (cands[i].onPanel) return cands[i];
|
||||
if (activeKey) { c = candByKey(cands, activeKey); if (c) return c; }
|
||||
return cands[0];
|
||||
}
|
||||
|
||||
// ADDENDUM 11: which hand owns the pointer.
|
||||
var activeSrcKey = null;
|
||||
|
||||
function updatePointer(frame, preferSrc) {
|
||||
if (!S.session) return;
|
||||
var next = {
|
||||
visible: false, beam: false, source: 'none',
|
||||
@@ -1054,24 +1087,43 @@
|
||||
// platform; an index-finger ray computed from joint poses carries the
|
||||
// per-frame fingertip jitter straight into the cursor, so its landing point
|
||||
// is visibly less steady. Meta's own samples use targetRaySpace too.
|
||||
//
|
||||
// ADDENDUM 11: with two hands BOTH report a tracked-pointer ray, and the
|
||||
// runtime's order is not "the hand the user is using". Taking the first one
|
||||
// pinned the guide line to one hand AND made the OTHER hand's select land
|
||||
// wherever the pinned hand pointed. The pointer now follows, in order:
|
||||
// 1. the source acting right now (select / gamepad press),
|
||||
// 2. the last acting source, while it is aiming at the panel,
|
||||
// 3. any source aiming at the panel,
|
||||
// 4. the last acting source, then the first one.
|
||||
var cands = [];
|
||||
for (var i = 0; i < srcs.length; i++) {
|
||||
var src = srcs[i];
|
||||
if (src.targetRayMode !== 'tracked-pointer') continue;
|
||||
var isHand = !!src.hand;
|
||||
var ray = rayOf(frame, src);
|
||||
if (!ray) continue;
|
||||
var hit = panelHit(frame, ray, layer, 0.25, 0.6);
|
||||
var hit = null;
|
||||
try { hit = panelHit(frame, ray, layer, 0.25, 0.6); } catch (e) { hit = null; }
|
||||
cands.push({
|
||||
src: src, ray: ray, hit: hit, key: srcKey(src),
|
||||
onPanel: !!(hit && hit.inBounds)
|
||||
});
|
||||
}
|
||||
|
||||
var pick = pickPointerCand(cands, preferSrc ? srcKey(preferSrc) : null, activeSrcKey);
|
||||
|
||||
if (pick) {
|
||||
activeSrcKey = pick.key;
|
||||
next = {
|
||||
visible: true,
|
||||
beam: true,
|
||||
source: (isHand ? 'hand' : 'controller') + ':' + src.handedness,
|
||||
origin: ray.origin,
|
||||
dir: ray.dir,
|
||||
hit: hit ? { nx: hit.nx, ny: hit.ny } : null,
|
||||
distance: hit ? hit.distance : null,
|
||||
source: (pick.src.hand ? 'hand' : 'controller') + ':' + pick.src.handedness,
|
||||
origin: pick.ray.origin,
|
||||
dir: pick.ray.dir,
|
||||
hit: pick.hit ? { nx: pick.hit.nx, ny: pick.hit.ny } : null,
|
||||
distance: pick.hit ? pick.hit.distance : null,
|
||||
pressed: performance.now() < pressUntil
|
||||
};
|
||||
break;
|
||||
}
|
||||
|
||||
pointer = next;
|
||||
@@ -1080,6 +1132,7 @@
|
||||
if (NS.Controls && NS.Controls.setPointer) {
|
||||
try { NS.Controls.setPointer(next); } catch (e) {}
|
||||
}
|
||||
S.stats.activeSrc = activeSrcKey || '(none)';
|
||||
S.stats.pointer = next.source +
|
||||
(next.hit ? ' HIT ' + next.hit.nx.toFixed(2) + ',' + next.hit.ny.toFixed(2) : ' (no hit)');
|
||||
S.stats.inputSources = srcs.map(function (s) {
|
||||
@@ -1579,7 +1632,7 @@
|
||||
// to the bar, so it must not also yank the subtitles.
|
||||
var barWasShown = NS.Controls && NS.Controls.isVisible ? !!NS.Controls.isVisible() : false;
|
||||
|
||||
if (frame) { try { updatePointer(frame); } catch (e) {} }
|
||||
if (frame) { try { updatePointer(frame, src); } catch (e) {} }
|
||||
|
||||
// 1) grabbing the subtitle strip wins, but only while the bar is down
|
||||
if (frame && !barWasShown) {
|
||||
|
||||
Reference in New Issue
Block a user