From d3ad9fbd564e6f4d43a0ec2c0903296f412e029c Mon Sep 17 00:00:00 2001 From: iChris4 Date: Fri, 25 Sep 2026 00:40:56 +0200 Subject: [PATCH] Hold the gas while a bare hand holds the wheel - On a Quest 3 the race controls never engaged: grabbing the wheel gave no gas and a right pinch stayed A. They waited for the game to switch the remote's pointer off, and MKW keeps it on in a race (the log only ever said "the game's pointer is on"). - The race controls now apply while a bare hand holds the wheel: that holds the gas, and a free hand's pinch (either hand) uses an item. With no bare hand on the wheel a right pinch stays A, which is what the pause menu and the results need. The flick still works in the whole cockpit. - The game-pointer publication (KPAD hook and bridge) is gone. The "tracked hands" source line is logged at most once a second: camera-tracked hands drop in and out of view often. Co-Authored-By: Claude Opus 5.5 --- OPENXR.md | 29 ++++++------ runtime/include/vr/openxr_hand_tracking.h | 25 +++++----- runtime/include/vr/openxr_input.h | 4 +- runtime/include/vr/openxr_wii_remote.h | 8 ---- runtime/src/hle/input/kpad.cpp | 6 --- runtime/src/vr/openxr_input.cpp | 57 +++++++++++++---------- runtime/src/vr/openxr_wii_remote.cpp | 11 ----- runtime/tests/vr_hand_tracking_tests.cpp | 18 ++++--- 8 files changed, 72 insertions(+), 86 deletions(-) diff --git a/OPENXR.md b/OPENXR.md index 712b64e..25db266 100644 --- a/OPENXR.md +++ b/OPENXR.md @@ -277,8 +277,8 @@ own motion detection still reads them, so a sharp enough turn can read as a shak **Bare hands.** On the Quest, with `hand_tracking` on and the controllers put down, the hands drive `khr/simple_controller`: a right pinch is A with the pointer on the hand's aim ray, the left -palm-up pinch is + (pause), and in a cockpit race a hand on the wheel holds A and a free hand's -pinch is Z; see "Tracked hands" under [Steering wheel and hand +palm-up pinch is + (pause), and in the cockpit, while a hand holds the wheel, that hand holds A and +a free hand's pinch is Z; see "Tracked hands" under [Steering wheel and hand steering](#steering-wheel-and-hand-steering). A bare hand feeds no motion; flicking the hands up plays one shake instead. With `hand_tracking` off a bare hand presses nothing but +. @@ -542,19 +542,18 @@ stands in for the grip and a grasp for the squeeze: the middle, ring and little summed over each finger's three joints, read as 0 below 1.2 radians (a relaxed hand) and 1 from 3.0 (a hand closed on a rim), so the wheel's own 55 % press and 15 % release apply, and closing a hand on the rim takes hold. The grasp never reaches the game's buttons (as a squeeze it would press -the gamepad's shoulders). In a cockpit race a bare hand holding the wheel holds the gas (A, the -gamepad's South) until both hands let go, and a pinch from a free bare hand uses an item (Z, the -gamepad's L) if the hand has been off the wheel for 0.15 s, so opening a hand off the rim does not -fire one; holding the pinch holds the button. The left palm-up pinch pauses (a pinch made in that -gesture is not an item: `XR_FB_hand_tracking_aim` reports the system gesture). Outside a race, and -in a race whenever the game has the remote's pointer on (the pause menu, the results), a right -pinch is A and the pointer follows the hand's aim ray; a left pinch does nothing, so the headset -panel cannot be opened with bare hands (an open one takes a right pinch, and the palm-up pinch -closes it). The game's pointer switch is read where KPAD polls the VR remote, and the log notes -each change (`OpenXR bare hands: the game's pointer is off (driving)`); with the gamepad -presentation there is no pointer, so the race controls apply throughout the cockpit. Manual drift -has no gesture: choose Automatic drift. The headset panel reads out each hand's source, grasp, -hold and pinch under the checkbox, for tuning. +the gamepad's shoulders). In the cockpit, while a bare hand holds the wheel it holds the gas (A, +the gamepad's South) until both hands let go, and a pinch from a free bare hand, either one, uses +an item (Z, the gamepad's L) if the hand has been off the wheel for 0.15 s, so opening a hand off +the rim does not fire one; holding the pinch holds the button. The left palm-up pinch pauses (a +pinch made in that gesture is not an item: `XR_FB_hand_tracking_aim` reports the system gesture). +With no bare hand on the wheel, in the cockpit or anywhere else, a right pinch is A and the +pointer follows the hand's aim ray, which is what the pause menu and the results need; in a race +it also gives gas without steering. A left pinch then does nothing, so the headset panel cannot be +opened with bare hands (an open one takes a right pinch, and the palm-up pinch closes it). The +game's own pointer switch cannot tell driving from those menus: MKW keeps it on in a race (checked +on a Quest 3, 2026-09-25). Manual drift has no gesture: choose Automatic drift. The headset panel +reads out each hand's source, grasp, hold and pinch under the checkbox, for tuning. A bare hand feeds no Wii Remote motion: camera-tracked poses are too noisy to differentiate twice, and turning the wheel would trick and wheelie. Tricks come from a flick instead. Both hands on diff --git a/runtime/include/vr/openxr_hand_tracking.h b/runtime/include/vr/openxr_hand_tracking.h index 8ca0ce7..1ec0a6e 100644 --- a/runtime/include/vr/openxr_hand_tracking.h +++ b/runtime/include/vr/openxr_hand_tracking.h @@ -256,23 +256,24 @@ inline void ApplyHandDrivenButtons(std::array& hands, } } -// In a cockpit race, after the wheel: a bare hand on the wheel holds the gas -// (A: the right primary button), and an item pinch uses an item (the left -// trigger: the Wii Remote's Z, the GameCube's L). A hand-driven right hand's -// pinch stops meaning A there. The grasp itself never becomes a squeeze, which -// would press the gamepad's shoulders (GameCube R drifts). -inline void ApplyBareHandRace(std::array& hands, - const std::array& hand_driven, const std::array& bare_held, +// In the cockpit, after the wheel. While a bare hand holds the wheel it holds +// the gas (A: the right primary button, whatever a right pinch says), and an +// item pinch uses an item (the left trigger: the Wii Remote's Z, the +// GameCube's L). With no bare hand on the wheel nothing changes, so a right +// pinch stays A for the menus inside a race (the pause menu, the results), +// which the game's pointer cannot tell from driving: MKW keeps it on in a +// race. The grasp itself never becomes a squeeze, which would press the +// gamepad's shoulders (GameCube R drifts). True while a bare hand holds. +inline bool ApplyBareHandRace(std::array& hands, const std::array& bare_held, const std::array& item_pinch) noexcept { - if (hand_driven[1]) { - hands[1].primary = false; - } - if (bare_held[0] || bare_held[1]) { - hands[1].primary = true; + if (!bare_held[0] && !bare_held[1]) { + return false; } + hands[1].primary = true; if (item_pinch[0] || item_pinch[1]) { hands[0].trigger = 1.0f; } + return true; } // One bare hand for the flick detector: its palm's height in the seated frame. diff --git a/runtime/include/vr/openxr_input.h b/runtime/include/vr/openxr_input.h index c920e84..ebb757d 100644 --- a/runtime/include/vr/openxr_input.h +++ b/runtime/include/vr/openxr_input.h @@ -232,16 +232,16 @@ private: hand_tracking::HandJointFrame m_joint_frame{}; std::array m_logged_sources{hand_tracking::Source::None, hand_tracking::Source::None}; + XrTime m_sources_logged_at = 0; // Per frame, from the actions: a controller is in the hand (its squeeze is // bound), and the hand drives simple_controller (Android only). std::array m_squeeze_active{}; std::array m_hand_driven{}; std::array m_pinch{}; // Bare-hand driving: each hand's bare latch (camera joints, last grasp) and - // item pinch gate, and the game's pointer switch as last logged. + // item pinch gate. std::array m_bare_latch{}; std::array m_pinch_gate{}; - int m_logged_game_pointer = -2; // A flick of the bare hands plays one shake on the remote's accelerometer // from this input time (0 when none is playing). hand_tracking::FlickDetector m_flick; diff --git a/runtime/include/vr/openxr_wii_remote.h b/runtime/include/vr/openxr_wii_remote.h index 0b3804c..ffd28d3 100644 --- a/runtime/include/vr/openxr_wii_remote.h +++ b/runtime/include/vr/openxr_wii_remote.h @@ -56,18 +56,10 @@ bool OpenXRReadWiiRemote(OpenXRWiiRemoteSample& sample) noexcept; // WPADControlMotor for the emulated remote. void OpenXRSetWiiRemoteRumble(bool active) noexcept; -// Guest side: whether the game has the remote's pointer switched on, as KPAD -// reads it each time the game polls the VR remote (Input::WiiController:: -// TogglePointer flips it). Tracked bare hands use it to tell driving from a -// menu inside a race; OpenXRInput logs every change. -void OpenXRPublishGamePointer(bool enabled) noexcept; - // XR side. void OpenXRPublishWiiRemote(uint32_t sdl_joystick_id, const OpenXRWiiRemoteSample& sample) noexcept; void OpenXRWithdrawWiiRemote() noexcept; bool OpenXRWiiRemoteRumbleRequested() noexcept; -// The game's pointer switch: 1 on, 0 off, -1 not known (no remote polled yet). -int OpenXRGamePointerState() noexcept; // The geometry and signal conditioning behind a sample, kept free of OpenXR // types so it can be checked headlessly (tests/vr_wii_remote_tests.cpp). diff --git a/runtime/src/hle/input/kpad.cpp b/runtime/src/hle/input/kpad.cpp index 1c2165f..dd3fc3b 100644 --- a/runtime/src/hle/input/kpad.cpp +++ b/runtime/src/hle/input/kpad.cpp @@ -1,6 +1,5 @@ #include "hle_stubs.h" #include "memory.h" -#include "vr/openxr_wii_remote.h" #include "wii_remote_input.h" #include @@ -277,11 +276,6 @@ extern "C" int32_t KPAD__Read_HLE(uint32_t chan, uint32_t statusPtr, uint32_t co WiiRemoteInput::KpadSample sample; const bool have = WiiRemoteInput::ReadKpadSample(chan, sample); try { - if (have && WiiRemoteInput::IsVrControllerChannel(chan)) { - // Tracked bare hands drive with the pointer off and point at menus - // with it on (openxr_hand_tracking.h). - mkw::vr::OpenXRPublishGamePointer(DpdEnabled(chan)); - } return WriteStatus(chan, statusPtr, have ? &sample : nullptr); } catch (const Memory::AccessViolation&) { return 0; diff --git a/runtime/src/vr/openxr_input.cpp b/runtime/src/vr/openxr_input.cpp index 501906f..9474afc 100644 --- a/runtime/src/vr/openxr_input.cpp +++ b/runtime/src/vr/openxr_input.cpp @@ -645,7 +645,12 @@ void OpenXRInput::LocateHands(XrTime time, const driving::SeatFrame& seat) { } tracked.seated = seat.valid; } - if (m_tracked_hands[0].source != m_logged_sources[0] || m_tracked_hands[1].source != m_logged_sources[1]) { + // Camera-tracked hands drop in and out of view often; a change is logged at + // most once a second, so the log still ends on the settled state. + constexpr XrTime kSourceLogIntervalNs = 1'000'000'000; + if ((m_tracked_hands[0].source != m_logged_sources[0] || m_tracked_hands[1].source != m_logged_sources[1]) && + (m_sources_logged_at == 0 || time - m_sources_logged_at >= kSourceLogIntervalNs)) { + m_sources_logged_at = time; m_logged_sources = {m_tracked_hands[0].source, m_tracked_hands[1].source}; std::ostringstream message; message << "OpenXR tracked hands: left " << hand_tracking::SourceLabel(m_logged_sources[0]) << ", right " @@ -744,8 +749,8 @@ void OpenXRInput::Destroy() { m_hand_data_source = m_hand_aim = false; m_hand_trackers_failed = m_logged_hand_restart = false; m_logged_sources = {}; + m_sources_logged_at = 0; m_squeeze_active = m_hand_driven = m_pinch = {}; - m_logged_game_pointer = -2; m_injected_flick_held = false; m_profile_serial = 0; DestroyPoseSpaces(); @@ -974,38 +979,40 @@ void OpenXRInput::Sync(XrTime predicted_display_time, const OpenXRPointerScreen& UpdateDriving(predicted_display_time, seat, hands, panel.withheld); #if defined(__ANDROID__) - // Bare hands in a cockpit race: a hand on the wheel holds the gas and a free - // hand's pinch uses an item. Only while the game's pointer is off: the - // pause menu and the results keep the cockpit but need A from a pinch. - const bool wii_remote = OpenXRGetControllerMode() == OpenXRControllerMode::WiiRemote; - const int game_pointer = wii_remote ? OpenXRGamePointerState() : -1; - if (hand_tracking_on && game_pointer != m_logged_game_pointer) { - m_logged_game_pointer = game_pointer; - Log(OpenXRLogLevel::Info, game_pointer > 0 ? "OpenXR bare hands: the game's pointer is on (menu)" - : game_pointer == 0 ? "OpenXR bare hands: the game's pointer is off (driving)" - : "OpenXR bare hands: the game's pointer is unknown"); + // Bare hands in the cockpit. While one of them holds the wheel it holds the + // gas and a free hand's pinch uses an item; with none on the wheel (the + // pause menu, the results, coasting) a right pinch stays A. The game's own + // pointer cannot tell the two apart: MKW keeps it on while driving. + const bool cockpit_hands = + hand_tracking_on && m_driving.cockpit_active && m_driving.hand_steering && !panel.withheld; + std::array bare_held{}; + for (uint32_t hand = 0; hand < kHands; ++hand) { + bare_held[hand] = cockpit_hands && m_bare_latch[hand].Bare() && m_wheel_held[hand]; } - const bool bare_race = hand_tracking_on && m_driving.cockpit_active && m_driving.hand_steering && - !panel.withheld && game_pointer <= 0; - if (bare_race) { - std::array bare_held{}, item_pinch{}; - std::array flick{}; + if (bare_held[0] || bare_held[1]) { + std::array item_pinch{}; for (uint32_t hand = 0; hand < kHands; ++hand) { - bare_held[hand] = m_bare_latch[hand].Bare() && m_wheel_held[hand]; item_pinch[hand] = m_hand_driven[hand] && m_pinch_gate[hand].Update(m_pinch[hand], m_wheel_held[hand], dt_seconds); - flick[hand] = {m_bare_latch[hand].Tracked(), m_wheel_held[hand], - m_joint_frame.seat_from_joint[hand][hand_tracking::kPalm][7]}; - } - hand_tracking::ApplyBareHandRace(hands, m_hand_driven, bare_held, item_pinch); - // A trick or a wheelie: the remote's shake, which the gamepad cannot give. - if (m_flick.Update(flick, dt_seconds) && wii_remote) { - m_flick_start = input_time; } + hand_tracking::ApplyBareHandRace(hands, bare_held, item_pinch); } else { + // A pinch already held when a hand takes the wheel is not an item. for (auto& gate : m_pinch_gate) { gate.Reset(); } + } + // A trick or a wheelie: the remote's shake, which the gamepad cannot give. + if (cockpit_hands) { + std::array flick{}; + for (uint32_t hand = 0; hand < kHands; ++hand) { + flick[hand] = {m_bare_latch[hand].Tracked(), m_wheel_held[hand], + m_joint_frame.seat_from_joint[hand][hand_tracking::kPalm][7]}; + } + if (m_flick.Update(flick, dt_seconds) && OpenXRGetControllerMode() == OpenXRControllerMode::WiiRemote) { + m_flick_start = input_time; + } + } else { m_flick.Reset(); } // `debug.wiicompiled.inject :flick` plays the same shake, with the diff --git a/runtime/src/vr/openxr_wii_remote.cpp b/runtime/src/vr/openxr_wii_remote.cpp index 24cbec2..ebe14de 100644 --- a/runtime/src/vr/openxr_wii_remote.cpp +++ b/runtime/src/vr/openxr_wii_remote.cpp @@ -25,8 +25,6 @@ std::atomic g_mode{OpenXRControllerMode::WiiRemote}; // SDL_JoystickID of the virtual gamepad the samples belong to; 0 when none. std::atomic g_joystick_id{0}; std::atomic g_rumble{false}; -// -1 until the game polls the VR remote, then its pointer switch. -std::atomic g_game_pointer{-1}; } // namespace wii_remote_bridge @@ -75,18 +73,9 @@ void OpenXRPublishWiiRemote(uint32_t sdl_joystick_id, const OpenXRWiiRemoteSampl wii_remote_bridge::g_joystick_id.store(sdl_joystick_id, std::memory_order_relaxed); } -void OpenXRPublishGamePointer(bool enabled) noexcept { - wii_remote_bridge::g_game_pointer.store(enabled ? 1 : 0, std::memory_order_relaxed); -} - -int OpenXRGamePointerState() noexcept { - return wii_remote_bridge::g_game_pointer.load(std::memory_order_relaxed); -} - void OpenXRWithdrawWiiRemote() noexcept { wii_remote_bridge::g_joystick_id.store(0, std::memory_order_relaxed); wii_remote_bridge::g_rumble.store(false, std::memory_order_relaxed); - wii_remote_bridge::g_game_pointer.store(-1, std::memory_order_relaxed); auto& published = wii_remote_bridge::Published(); std::lock_guard lock(published.mutex); published.available = false; diff --git a/runtime/tests/vr_hand_tracking_tests.cpp b/runtime/tests/vr_hand_tracking_tests.cpp index 0117f87..845628d 100644 --- a/runtime/tests/vr_hand_tracking_tests.cpp +++ b/runtime/tests/vr_hand_tracking_tests.cpp @@ -173,20 +173,24 @@ void TestHandDrivenButtons() { void TestBareHandRace() { std::array hands{}; - hands[1].primary = true; // a right pinch, already A from the menus' mapping - ApplyBareHandRace(hands, {true, true}, {false, false}, {false, false}); - Check(!hands[1].primary, "in a race a bare right pinch is not A"); - ApplyBareHandRace(hands, {true, true}, {true, false}, {false, false}); - Check(hands[1].primary, "a bare hand on the wheel holds the gas"); + hands[1].primary = true; // a right pinch, A from the menus' mapping + Check(!ApplyBareHandRace(hands, {false, false}, {false, false}) && hands[1].primary, + "with no hand on the wheel a right pinch stays A (the pause menu, the results)"); + hands = {}; + Check(ApplyBareHandRace(hands, {true, false}, {false, false}) && hands[1].primary, + "a bare hand on the wheel holds the gas"); Check(hands[0].trigger == 0.0f && hands[0].squeeze == 0.0f && hands[1].squeeze == 0.0f, "holding presses no item and no shoulder"); hands = {}; - ApplyBareHandRace(hands, {true, true}, {false, true}, {true, false}); + ApplyBareHandRace(hands, {false, true}, {true, false}); Check(hands[1].primary && hands[0].trigger == 1.0f, "a free hand's pinch uses an item while the other drives"); + hands = {}; + ApplyBareHandRace(hands, {true, false}, {false, true}); + Check(hands[1].primary && hands[0].trigger == 1.0f, "a free right hand's pinch is an item, not A, while driving"); std::array controllers{}; controllers[1].primary = true; controllers[0].trigger = 0.3f; - ApplyBareHandRace(controllers, {false, false}, {false, false}, {false, false}); + ApplyBareHandRace(controllers, {false, false}, {false, false}); Check(controllers[1].primary && controllers[0].trigger == 0.3f, "controller hands keep their own buttons"); }