Hold the gas while a bare hand holds the wheel

- On a Quest 3 the race controls never engaged: grabbing the wheel gave no gas and a right pinch
  stayed A. They waited for the game to switch the remote's pointer off, and MKW keeps it on in a
  race (the log only ever said "the game's pointer is on").
- The race controls now apply while a bare hand holds the wheel: that holds the gas, and a free
  hand's pinch (either hand) uses an item. With no bare hand on the wheel a right pinch stays A,
  which is what the pause menu and the results need. The flick still works in the whole cockpit.
- The game-pointer publication (KPAD hook and bridge) is gone. The "tracked hands" source line is
  logged at most once a second: camera-tracked hands drop in and out of view often.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
iChris4andClaude Opus 5.5 committed 2026-09-25 00:40:56 +02:00
1 parent 771a143254
commit d3ad9fbd56
8 files changed
+72 -86

No files matched your search

+14 -15
View File
@@ -277,8 +277,8 @@ own motion detection still reads them, so a sharp enough turn can read as a shak
**Bare hands.** On the Quest, with `hand_tracking` on and the controllers put down, the hands drive **Bare hands.** On the Quest, with `hand_tracking` on and the controllers put down, the hands drive
`khr/simple_controller`: a right pinch is A with the pointer on the hand's aim ray, the left `khr/simple_controller`: a right pinch is A with the pointer on the hand's aim ray, the left
palm-up pinch is + (pause), and in a cockpit race a hand on the wheel holds A and a free hand's palm-up pinch is + (pause), and in the cockpit, while a hand holds the wheel, that hand holds A and
pinch is Z; see "Tracked hands" under [Steering wheel and hand a free hand's pinch is Z; see "Tracked hands" under [Steering wheel and hand
steering](#steering-wheel-and-hand-steering). A bare hand feeds no motion; flicking the hands up steering](#steering-wheel-and-hand-steering). A bare hand feeds no motion; flicking the hands up
plays one shake instead. With `hand_tracking` off a bare hand presses nothing but +. plays one shake instead. With `hand_tracking` off a bare hand presses nothing but +.
@@ -542,19 +542,18 @@ stands in for the grip and a grasp for the squeeze: the middle, ring and little
summed over each finger's three joints, read as 0 below 1.2 radians (a relaxed hand) and 1 from summed over each finger's three joints, read as 0 below 1.2 radians (a relaxed hand) and 1 from
3.0 (a hand closed on a rim), so the wheel's own 55 % press and 15 % release apply, and closing a 3.0 (a hand closed on a rim), so the wheel's own 55 % press and 15 % release apply, and closing a
hand on the rim takes hold. The grasp never reaches the game's buttons (as a squeeze it would press hand on the rim takes hold. The grasp never reaches the game's buttons (as a squeeze it would press
the gamepad's shoulders). In a cockpit race a bare hand holding the wheel holds the gas (A, the the gamepad's shoulders). In the cockpit, while a bare hand holds the wheel it holds the gas (A,
gamepad's South) until both hands let go, and a pinch from a free bare hand uses an item (Z, the the gamepad's South) until both hands let go, and a pinch from a free bare hand, either one, uses
gamepad's L) if the hand has been off the wheel for 0.15 s, so opening a hand off the rim does not an item (Z, the gamepad's L) if the hand has been off the wheel for 0.15 s, so opening a hand off
fire one; holding the pinch holds the button. The left palm-up pinch pauses (a pinch made in that the rim does not fire one; holding the pinch holds the button. The left palm-up pinch pauses (a
gesture is not an item: `XR_FB_hand_tracking_aim` reports the system gesture). Outside a race, and pinch made in that gesture is not an item: `XR_FB_hand_tracking_aim` reports the system gesture).
in a race whenever the game has the remote's pointer on (the pause menu, the results), a right With no bare hand on the wheel, in the cockpit or anywhere else, a right pinch is A and the
pinch is A and the pointer follows the hand's aim ray; a left pinch does nothing, so the headset pointer follows the hand's aim ray, which is what the pause menu and the results need; in a race
panel cannot be opened with bare hands (an open one takes a right pinch, and the palm-up pinch it also gives gas without steering. A left pinch then does nothing, so the headset panel cannot be
closes it). The game's pointer switch is read where KPAD polls the VR remote, and the log notes opened with bare hands (an open one takes a right pinch, and the palm-up pinch closes it). The
each change (`OpenXR bare hands: the game's pointer is off (driving)`); with the gamepad game's own pointer switch cannot tell driving from those menus: MKW keeps it on in a race (checked
presentation there is no pointer, so the race controls apply throughout the cockpit. Manual drift on a Quest 3, 2026-09-25). Manual drift has no gesture: choose Automatic drift. The headset panel
has no gesture: choose Automatic drift. The headset panel reads out each hand's source, grasp, reads out each hand's source, grasp, hold and pinch under the checkbox, for tuning.
hold and pinch under the checkbox, for tuning.
A bare hand feeds no Wii Remote motion: camera-tracked poses are too noisy to differentiate twice, A bare hand feeds no Wii Remote motion: camera-tracked poses are too noisy to differentiate twice,
and turning the wheel would trick and wheelie. Tricks come from a flick instead. Both hands on and turning the wheel would trick and wheelie. Tricks come from a flick instead. Both hands on
+13 -12
View File
@@ -256,23 +256,24 @@ inline void ApplyHandDrivenButtons(std::array<wii_remote::HandInputs, 2>& hands,
} }
} }
// In a cockpit race, after the wheel: a bare hand on the wheel holds the gas // In the cockpit, after the wheel. While a bare hand holds the wheel it holds
// (A: the right primary button), and an item pinch uses an item (the left // the gas (A: the right primary button, whatever a right pinch says), and an
// trigger: the Wii Remote's Z, the GameCube's L). A hand-driven right hand's // item pinch uses an item (the left trigger: the Wii Remote's Z, the
// pinch stops meaning A there. The grasp itself never becomes a squeeze, which // GameCube's L). With no bare hand on the wheel nothing changes, so a right
// would press the gamepad's shoulders (GameCube R drifts). // pinch stays A for the menus inside a race (the pause menu, the results),
inline void ApplyBareHandRace(std::array<wii_remote::HandInputs, 2>& hands, // which the game's pointer cannot tell from driving: MKW keeps it on in a
const std::array<bool, 2>& hand_driven, const std::array<bool, 2>& bare_held, // race. The grasp itself never becomes a squeeze, which would press the
// gamepad's shoulders (GameCube R drifts). True while a bare hand holds.
inline bool ApplyBareHandRace(std::array<wii_remote::HandInputs, 2>& hands, const std::array<bool, 2>& bare_held,
const std::array<bool, 2>& item_pinch) noexcept { const std::array<bool, 2>& item_pinch) noexcept {
if (hand_driven[1]) { if (!bare_held[0] && !bare_held[1]) {
hands[1].primary = false; return false;
}
if (bare_held[0] || bare_held[1]) {
hands[1].primary = true;
} }
hands[1].primary = true;
if (item_pinch[0] || item_pinch[1]) { if (item_pinch[0] || item_pinch[1]) {
hands[0].trigger = 1.0f; hands[0].trigger = 1.0f;
} }
return true;
} }
// One bare hand for the flick detector: its palm's height in the seated frame. // One bare hand for the flick detector: its palm's height in the seated frame.
+2 -2
View File
@@ -232,16 +232,16 @@ private:
hand_tracking::HandJointFrame m_joint_frame{}; hand_tracking::HandJointFrame m_joint_frame{};
std::array<hand_tracking::Source, kHands> m_logged_sources{hand_tracking::Source::None, std::array<hand_tracking::Source, kHands> m_logged_sources{hand_tracking::Source::None,
hand_tracking::Source::None}; hand_tracking::Source::None};
XrTime m_sources_logged_at = 0;
// Per frame, from the actions: a controller is in the hand (its squeeze is // Per frame, from the actions: a controller is in the hand (its squeeze is
// bound), and the hand drives simple_controller (Android only). // bound), and the hand drives simple_controller (Android only).
std::array<bool, kHands> m_squeeze_active{}; std::array<bool, kHands> m_squeeze_active{};
std::array<bool, kHands> m_hand_driven{}; std::array<bool, kHands> m_hand_driven{};
std::array<bool, kHands> m_pinch{}; std::array<bool, kHands> m_pinch{};
// Bare-hand driving: each hand's bare latch (camera joints, last grasp) and // Bare-hand driving: each hand's bare latch (camera joints, last grasp) and
// item pinch gate, and the game's pointer switch as last logged. // item pinch gate.
std::array<hand_tracking::BareLatch, kHands> m_bare_latch{}; std::array<hand_tracking::BareLatch, kHands> m_bare_latch{};
std::array<hand_tracking::PinchGate, kHands> m_pinch_gate{}; std::array<hand_tracking::PinchGate, kHands> m_pinch_gate{};
int m_logged_game_pointer = -2;
// A flick of the bare hands plays one shake on the remote's accelerometer // A flick of the bare hands plays one shake on the remote's accelerometer
// from this input time (0 when none is playing). // from this input time (0 when none is playing).
hand_tracking::FlickDetector m_flick; hand_tracking::FlickDetector m_flick;
-8
View File
@@ -56,18 +56,10 @@ bool OpenXRReadWiiRemote(OpenXRWiiRemoteSample& sample) noexcept;
// WPADControlMotor for the emulated remote. // WPADControlMotor for the emulated remote.
void OpenXRSetWiiRemoteRumble(bool active) noexcept; void OpenXRSetWiiRemoteRumble(bool active) noexcept;
// Guest side: whether the game has the remote's pointer switched on, as KPAD
// reads it each time the game polls the VR remote (Input::WiiController::
// TogglePointer flips it). Tracked bare hands use it to tell driving from a
// menu inside a race; OpenXRInput logs every change.
void OpenXRPublishGamePointer(bool enabled) noexcept;
// XR side. // XR side.
void OpenXRPublishWiiRemote(uint32_t sdl_joystick_id, const OpenXRWiiRemoteSample& sample) noexcept; void OpenXRPublishWiiRemote(uint32_t sdl_joystick_id, const OpenXRWiiRemoteSample& sample) noexcept;
void OpenXRWithdrawWiiRemote() noexcept; void OpenXRWithdrawWiiRemote() noexcept;
bool OpenXRWiiRemoteRumbleRequested() noexcept; bool OpenXRWiiRemoteRumbleRequested() noexcept;
// The game's pointer switch: 1 on, 0 off, -1 not known (no remote polled yet).
int OpenXRGamePointerState() noexcept;
// The geometry and signal conditioning behind a sample, kept free of OpenXR // The geometry and signal conditioning behind a sample, kept free of OpenXR
// types so it can be checked headlessly (tests/vr_wii_remote_tests.cpp). // types so it can be checked headlessly (tests/vr_wii_remote_tests.cpp).
-6
View File
@@ -1,6 +1,5 @@
#include "hle_stubs.h" #include "hle_stubs.h"
#include "memory.h" #include "memory.h"
#include "vr/openxr_wii_remote.h"
#include "wii_remote_input.h" #include "wii_remote_input.h"
#include <algorithm> #include <algorithm>
@@ -277,11 +276,6 @@ extern "C" int32_t KPAD__Read_HLE(uint32_t chan, uint32_t statusPtr, uint32_t co
WiiRemoteInput::KpadSample sample; WiiRemoteInput::KpadSample sample;
const bool have = WiiRemoteInput::ReadKpadSample(chan, sample); const bool have = WiiRemoteInput::ReadKpadSample(chan, sample);
try { try {
if (have && WiiRemoteInput::IsVrControllerChannel(chan)) {
// Tracked bare hands drive with the pointer off and point at menus
// with it on (openxr_hand_tracking.h).
mkw::vr::OpenXRPublishGamePointer(DpdEnabled(chan));
}
return WriteStatus(chan, statusPtr, have ? &sample : nullptr); return WriteStatus(chan, statusPtr, have ? &sample : nullptr);
} catch (const Memory::AccessViolation&) { } catch (const Memory::AccessViolation&) {
return 0; return 0;
+32 -25
View File
@@ -645,7 +645,12 @@ void OpenXRInput::LocateHands(XrTime time, const driving::SeatFrame& seat) {
} }
tracked.seated = seat.valid; tracked.seated = seat.valid;
} }
if (m_tracked_hands[0].source != m_logged_sources[0] || m_tracked_hands[1].source != m_logged_sources[1]) { // Camera-tracked hands drop in and out of view often; a change is logged at
// most once a second, so the log still ends on the settled state.
constexpr XrTime kSourceLogIntervalNs = 1'000'000'000;
if ((m_tracked_hands[0].source != m_logged_sources[0] || m_tracked_hands[1].source != m_logged_sources[1]) &&
(m_sources_logged_at == 0 || time - m_sources_logged_at >= kSourceLogIntervalNs)) {
m_sources_logged_at = time;
m_logged_sources = {m_tracked_hands[0].source, m_tracked_hands[1].source}; m_logged_sources = {m_tracked_hands[0].source, m_tracked_hands[1].source};
std::ostringstream message; std::ostringstream message;
message << "OpenXR tracked hands: left " << hand_tracking::SourceLabel(m_logged_sources[0]) << ", right " message << "OpenXR tracked hands: left " << hand_tracking::SourceLabel(m_logged_sources[0]) << ", right "
@@ -744,8 +749,8 @@ void OpenXRInput::Destroy() {
m_hand_data_source = m_hand_aim = false; m_hand_data_source = m_hand_aim = false;
m_hand_trackers_failed = m_logged_hand_restart = false; m_hand_trackers_failed = m_logged_hand_restart = false;
m_logged_sources = {}; m_logged_sources = {};
m_sources_logged_at = 0;
m_squeeze_active = m_hand_driven = m_pinch = {}; m_squeeze_active = m_hand_driven = m_pinch = {};
m_logged_game_pointer = -2;
m_injected_flick_held = false; m_injected_flick_held = false;
m_profile_serial = 0; m_profile_serial = 0;
DestroyPoseSpaces(); DestroyPoseSpaces();
@@ -974,38 +979,40 @@ void OpenXRInput::Sync(XrTime predicted_display_time, const OpenXRPointerScreen&
UpdateDriving(predicted_display_time, seat, hands, panel.withheld); UpdateDriving(predicted_display_time, seat, hands, panel.withheld);
#if defined(__ANDROID__) #if defined(__ANDROID__)
// Bare hands in a cockpit race: a hand on the wheel holds the gas and a free // Bare hands in the cockpit. While one of them holds the wheel it holds the
// hand's pinch uses an item. Only while the game's pointer is off: the // gas and a free hand's pinch uses an item; with none on the wheel (the
// pause menu and the results keep the cockpit but need A from a pinch. // pause menu, the results, coasting) a right pinch stays A. The game's own
const bool wii_remote = OpenXRGetControllerMode() == OpenXRControllerMode::WiiRemote; // pointer cannot tell the two apart: MKW keeps it on while driving.
const int game_pointer = wii_remote ? OpenXRGamePointerState() : -1; const bool cockpit_hands =
if (hand_tracking_on && game_pointer != m_logged_game_pointer) { hand_tracking_on && m_driving.cockpit_active && m_driving.hand_steering && !panel.withheld;
m_logged_game_pointer = game_pointer; std::array<bool, kHands> bare_held{};
Log(OpenXRLogLevel::Info, game_pointer > 0 ? "OpenXR bare hands: the game's pointer is on (menu)" for (uint32_t hand = 0; hand < kHands; ++hand) {
: game_pointer == 0 ? "OpenXR bare hands: the game's pointer is off (driving)" bare_held[hand] = cockpit_hands && m_bare_latch[hand].Bare() && m_wheel_held[hand];
: "OpenXR bare hands: the game's pointer is unknown");
} }
const bool bare_race = hand_tracking_on && m_driving.cockpit_active && m_driving.hand_steering && if (bare_held[0] || bare_held[1]) {
!panel.withheld && game_pointer <= 0; std::array<bool, kHands> item_pinch{};
if (bare_race) {
std::array<bool, kHands> bare_held{}, item_pinch{};
std::array<hand_tracking::FlickHand, kHands> flick{};
for (uint32_t hand = 0; hand < kHands; ++hand) { for (uint32_t hand = 0; hand < kHands; ++hand) {
bare_held[hand] = m_bare_latch[hand].Bare() && m_wheel_held[hand];
item_pinch[hand] = m_hand_driven[hand] && m_pinch_gate[hand].Update(m_pinch[hand], m_wheel_held[hand], item_pinch[hand] = m_hand_driven[hand] && m_pinch_gate[hand].Update(m_pinch[hand], m_wheel_held[hand],
dt_seconds); dt_seconds);
flick[hand] = {m_bare_latch[hand].Tracked(), m_wheel_held[hand],
m_joint_frame.seat_from_joint[hand][hand_tracking::kPalm][7]};
}
hand_tracking::ApplyBareHandRace(hands, m_hand_driven, bare_held, item_pinch);
// A trick or a wheelie: the remote's shake, which the gamepad cannot give.
if (m_flick.Update(flick, dt_seconds) && wii_remote) {
m_flick_start = input_time;
} }
hand_tracking::ApplyBareHandRace(hands, bare_held, item_pinch);
} else { } else {
// A pinch already held when a hand takes the wheel is not an item.
for (auto& gate : m_pinch_gate) { for (auto& gate : m_pinch_gate) {
gate.Reset(); gate.Reset();
} }
}
// A trick or a wheelie: the remote's shake, which the gamepad cannot give.
if (cockpit_hands) {
std::array<hand_tracking::FlickHand, kHands> flick{};
for (uint32_t hand = 0; hand < kHands; ++hand) {
flick[hand] = {m_bare_latch[hand].Tracked(), m_wheel_held[hand],
m_joint_frame.seat_from_joint[hand][hand_tracking::kPalm][7]};
}
if (m_flick.Update(flick, dt_seconds) && OpenXRGetControllerMode() == OpenXRControllerMode::WiiRemote) {
m_flick_start = input_time;
}
} else {
m_flick.Reset(); m_flick.Reset();
} }
// `debug.wiicompiled.inject <n>:flick` plays the same shake, with the // `debug.wiicompiled.inject <n>:flick` plays the same shake, with the
-11
View File
@@ -25,8 +25,6 @@ std::atomic<OpenXRControllerMode> g_mode{OpenXRControllerMode::WiiRemote};
// SDL_JoystickID of the virtual gamepad the samples belong to; 0 when none. // SDL_JoystickID of the virtual gamepad the samples belong to; 0 when none.
std::atomic<uint32_t> g_joystick_id{0}; std::atomic<uint32_t> g_joystick_id{0};
std::atomic<bool> g_rumble{false}; std::atomic<bool> g_rumble{false};
// -1 until the game polls the VR remote, then its pointer switch.
std::atomic<int> g_game_pointer{-1};
} // namespace wii_remote_bridge } // namespace wii_remote_bridge
@@ -75,18 +73,9 @@ void OpenXRPublishWiiRemote(uint32_t sdl_joystick_id, const OpenXRWiiRemoteSampl
wii_remote_bridge::g_joystick_id.store(sdl_joystick_id, std::memory_order_relaxed); wii_remote_bridge::g_joystick_id.store(sdl_joystick_id, std::memory_order_relaxed);
} }
void OpenXRPublishGamePointer(bool enabled) noexcept {
wii_remote_bridge::g_game_pointer.store(enabled ? 1 : 0, std::memory_order_relaxed);
}
int OpenXRGamePointerState() noexcept {
return wii_remote_bridge::g_game_pointer.load(std::memory_order_relaxed);
}
void OpenXRWithdrawWiiRemote() noexcept { void OpenXRWithdrawWiiRemote() noexcept {
wii_remote_bridge::g_joystick_id.store(0, std::memory_order_relaxed); wii_remote_bridge::g_joystick_id.store(0, std::memory_order_relaxed);
wii_remote_bridge::g_rumble.store(false, std::memory_order_relaxed); wii_remote_bridge::g_rumble.store(false, std::memory_order_relaxed);
wii_remote_bridge::g_game_pointer.store(-1, std::memory_order_relaxed);
auto& published = wii_remote_bridge::Published(); auto& published = wii_remote_bridge::Published();
std::lock_guard lock(published.mutex); std::lock_guard lock(published.mutex);
published.available = false; published.available = false;
+11 -7
View File
@@ -173,20 +173,24 @@ void TestHandDrivenButtons() {
void TestBareHandRace() { void TestBareHandRace() {
std::array<wii_remote::HandInputs, 2> hands{}; std::array<wii_remote::HandInputs, 2> hands{};
hands[1].primary = true; // a right pinch, already A from the menus' mapping hands[1].primary = true; // a right pinch, A from the menus' mapping
ApplyBareHandRace(hands, {true, true}, {false, false}, {false, false}); Check(!ApplyBareHandRace(hands, {false, false}, {false, false}) && hands[1].primary,
Check(!hands[1].primary, "in a race a bare right pinch is not A"); "with no hand on the wheel a right pinch stays A (the pause menu, the results)");
ApplyBareHandRace(hands, {true, true}, {true, false}, {false, false}); hands = {};
Check(hands[1].primary, "a bare hand on the wheel holds the gas"); Check(ApplyBareHandRace(hands, {true, false}, {false, false}) && hands[1].primary,
"a bare hand on the wheel holds the gas");
Check(hands[0].trigger == 0.0f && hands[0].squeeze == 0.0f && hands[1].squeeze == 0.0f, Check(hands[0].trigger == 0.0f && hands[0].squeeze == 0.0f && hands[1].squeeze == 0.0f,
"holding presses no item and no shoulder"); "holding presses no item and no shoulder");
hands = {}; hands = {};
ApplyBareHandRace(hands, {true, true}, {false, true}, {true, false}); ApplyBareHandRace(hands, {false, true}, {true, false});
Check(hands[1].primary && hands[0].trigger == 1.0f, "a free hand's pinch uses an item while the other drives"); Check(hands[1].primary && hands[0].trigger == 1.0f, "a free hand's pinch uses an item while the other drives");
hands = {};
ApplyBareHandRace(hands, {true, false}, {false, true});
Check(hands[1].primary && hands[0].trigger == 1.0f, "a free right hand's pinch is an item, not A, while driving");
std::array<wii_remote::HandInputs, 2> controllers{}; std::array<wii_remote::HandInputs, 2> controllers{};
controllers[1].primary = true; controllers[1].primary = true;
controllers[0].trigger = 0.3f; controllers[0].trigger = 0.3f;
ApplyBareHandRace(controllers, {false, false}, {false, false}, {false, false}); ApplyBareHandRace(controllers, {false, false}, {false, false});
Check(controllers[1].primary && controllers[0].trigger == 0.3f, "controller hands keep their own buttons"); Check(controllers[1].primary && controllers[0].trigger == 0.3f, "controller hands keep their own buttons");
} }