Pose the cockpit hands from the headset's hand tracking

- New [vr] hand_tracking (default off, Quest only for now): the Quest launcher's Settings > VR and
  the headset panel's VR tab, under hand steering. Two XR_EXT_hand_tracking trackers are located
  every XR frame: with the controllers held the Quest builds the joints from their touch sensors
  (XR_EXT_hand_tracking_data_source's controller source), once they are put down from its
  cameras. The trackers exist only while the option and hand steering are on and also serve the
  runtime hand mesh; the extensions (plus XR_FB_hand_tracking_aim) are asked for when either is
  on at launch.
- Aurora skins the runtime mesh with the joints themselves (tracked pose times inverse bind pose,
  no curl, no grip); runtimes with joints but no mesh get a skeleton; non-finite joints put only
  that hand back on its grip curl. AuroraCockpitHand carries the 26 seated-frame joints and radii.
- The manifest declares horizonos.permission.HAND_TRACKING (and the deprecated
  com.oculus.permission.HAND_TRACKING), both normal permissions with no prompt on a Quest 3, and
  oculus.software.handtracking as optional; without it Horizon OS keeps the app controllers-only.
  Bare hands then drive khr/simple_controller, so on Android a hand whose squeeze action is
  inactive and select active is treated as bare: with the option off it presses nothing but the
  menu gesture, is not drawn and feeds no Wii Remote motion.
- Interaction-profile changes and tracker sources are logged. The pure rules (grasp from finger
  flexion, bare latch, pinch gate, bare-hand buttons, flick) live in the OpenXR-free
  vr/openxr_hand_tracking.h with mkw_vr_hand_tracking_tests; the driving and flick parts are
  wired in the next commits. Docs: OPENXR.md, docs/quest-port.md, README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
iChris4andClaude Opus 5.5 committed 2026-09-25 00:05:48 +02:00
1 parent c990c595f0
commit a21e22b605
24 files changed
+1549 -49

No files matched your search

+12 -1
View File
@@ -123,11 +123,20 @@ typedef enum {
* immersive base position. Aurora draws it per eye after the scene, depth-tested
* against the scene with the scene's own depth mapping.
*/
enum { AURORA_VR_HAND_JOINT_COUNT = 26 };
typedef struct {
bool tracked;
bool held;
// The hand-tracking joints below are valid: the hand is drawn from them
// instead of curling from squeeze at seatFromGrip.
bool jointsValid;
float squeeze;
float seatFromGrip[12];
// XR_EXT_hand_tracking joints in XR_HAND_JOINT_* order, each a row-major 3x4
// in the seated frame, and their radii in metres.
float seatFromJoint[AURORA_VR_HAND_JOINT_COUNT][12];
float jointRadii[AURORA_VR_HAND_JOINT_COUNT];
} AuroraCockpitHand;
typedef struct {
@@ -159,7 +168,9 @@ typedef struct {
void aurora_set_stereo_panel_layer(bool enabled);
// Copies optional runtime-provided hand meshes (XR_FB_hand_tracking_mesh, 26
// joints). Null clears to the procedural glove. Bind poses: x,y,z,w,px,py,pz.
// joints). Null clears to the procedural glove. Bind poses: x,y,z,w,px,py,pz,
// in the space of the mesh's vertices, as xrLocateHandJointsEXT reports poses:
// a hand with tracked joints is skinned with seatFromJoint * inverse(bind).
void aurora_set_vr_hand_mesh(uint32_t hand, const AuroraVRHandVertex* vertices, uint32_t vertexCount,
const uint16_t* indices, uint32_t indexCount, const float* bindPoses,
const int32_t* parents, uint32_t jointCount);