Pose the cockpit hands from the headset's hand tracking

- New [vr] hand_tracking (default off, Quest only for now): the Quest launcher's Settings > VR and
  the headset panel's VR tab, under hand steering. Two XR_EXT_hand_tracking trackers are located
  every XR frame: with the controllers held the Quest builds the joints from their touch sensors
  (XR_EXT_hand_tracking_data_source's controller source), once they are put down from its
  cameras. The trackers exist only while the option and hand steering are on and also serve the
  runtime hand mesh; the extensions (plus XR_FB_hand_tracking_aim) are asked for when either is
  on at launch.
- Aurora skins the runtime mesh with the joints themselves (tracked pose times inverse bind pose,
  no curl, no grip); runtimes with joints but no mesh get a skeleton; non-finite joints put only
  that hand back on its grip curl. AuroraCockpitHand carries the 26 seated-frame joints and radii.
- The manifest declares horizonos.permission.HAND_TRACKING (and the deprecated
  com.oculus.permission.HAND_TRACKING), both normal permissions with no prompt on a Quest 3, and
  oculus.software.handtracking as optional; without it Horizon OS keeps the app controllers-only.
  Bare hands then drive khr/simple_controller, so on Android a hand whose squeeze action is
  inactive and select active is treated as bare: with the option off it presses nothing but the
  menu gesture, is not drawn and feeds no Wii Remote motion.
- Interaction-profile changes and tracker sources are logged. The pure rules (grasp from finger
  flexion, bare latch, pinch gate, bare-hand buttons, flick) live in the OpenXR-free
  vr/openxr_hand_tracking.h with mkw_vr_hand_tracking_tests; the driving and flick parts are
  wired in the next commits. Docs: OPENXR.md, docs/quest-port.md, README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
iChris4andClaude Opus 5.5 committed 2026-09-25 00:05:48 +02:00
1 parent c990c595f0
commit a21e22b605
24 files changed
+1549 -49

No files matched your search

+19 -2
View File
@@ -196,6 +196,23 @@ shoulders, thumbsticks → sticks (clicks → stick buttons), left menu → Star
and every existing binding, dead zone and overlay setting applies. Bindings are
suggested for `oculus/touch_controller` and `khr/simple_controller`.
The manifest declares hand tracking (`horizonos.permission.HAND_TRACKING`, the
deprecated `com.oculus.permission.HAND_TRACKING` for older builds, and
`oculus.software.handtracking` as optional). Both permissions are `normal` on a
Quest 3 (`adb shell pm list permissions -g -f`), so there is no prompt. Without
the feature flag Horizon OS keeps the app controllers-only: the game process logs
`setting hand mode control settings to ControllersOnly` / `sethandmanifest 0`,
and putting the controllers down logs `going to controller mode because hands are
disabled by manifest flag`; with it, `Is hands or controller` /
`sethandmanifest 2`. Bare hands then drive `khr/simple_controller` (a pinch is
select, the left palm-up pinch the menu), which the runtime ignores apart from
the menu gesture unless `[vr] hand_tracking` is on; see "Tracked hands" in
`OPENXR.md`. The Quest 3 runtime (`libvrapiimpl.so` in the `com.meta.xr` APEX's
VrDriver.apk) implements `XR_EXT_hand_tracking_data_source`,
`XR_FB_hand_tracking_aim`, `XR_EXT_hand_interaction`, microgestures and the
wide-motion modes; Meta's manifest filter skips the wide-motion modes unless the
app also declares `com.oculus.software.body_tracking`.
### Android platform glue
- `runtime/src/vr/openxr_android.cpp`: `xrInitializeLoaderKHR` with the
@@ -273,8 +290,8 @@ palette. **Home** has the Play button and reports a missing or incomplete `DATA`
(the check is the runtime's own `IsDvdDataRoot`: `files/` and `sys/fst.bin`).
**Settings** edits `Config.toml` in tabs: VR (race view: immersive, immersive window or flat
screen, camera, rotation, driver hiding,
seat, hand steering, lean back, render scale, VR interpolation, virtual screen
size and distance),
seat, hand steering, tracked hands, lean back, render scale, VR interpolation,
virtual screen size and distance),
Graphics (resolution, widescreen, bloom, shader stutter), Controls (controller
mode, vibration, the Wii Remote mapping), Audio, and About (paths, OpenXR
logging). The launch-time geometry (`render_scale`, `hud_distance_meters`,