With [vr] eye_tracked_foveation (on by default on the Steam Frame, off
elsewhere) the runtime asks for XR_EXT_eye_gaze_interaction. When the
system reports an eye tracker, OpenXRInput binds the gaze pose and
locates it for each packet's display time, in the space the eye views
are located in; vr/eye_gaze.h turns it into tangents of each eye's own
view, which AuroraStereoFrame now carries (appended, after the existing
prefix).
Aurora centres the eye's fragment density map on the gaze snapped to a
cell of two map texels (about 3 degrees). Each eye keeps up to 32 maps,
one per cell looked at, so a glance back reuses its map; a new map is
bound once its upload completes, and until then the eye keeps the map
it had. Without a tracked gaze (a blink, no tracker, the setting off)
foveation centres on the forward direction exactly as before: the
forward maps are byte-identical.
Also logs every extension the OpenXR runtime offers at startup, so the
first Steam Frame session shows what SteamVR's Android runtime has.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HBRGKTE1GnN2ah8gcZKr3
The Steam Frame runs Android apps through Lepton with SteamVR's OpenXR
runtime. A third headset flavour, steamFrame, targets it:
- -mcpu=cortex-x4+nosve for the Snapdragon 8 Gen 3 (its firmware does
not expose SVE, which clang otherwise auto-vectorises with), from one
flavour-to-CPU map that the kit export now reads instead of guessing
from the variant name.
- MKW_ANDROID_HEADSET=steam_frame defines MKW_HEADSET_STEAM_FRAME for
the runtime's own targets, for Frame-specific defaults.
- A manifest without the Horizon OS entries, and FrameEntryActivity as
the single real MAIN/LAUNCHER activity with the Khronos and Oculus VR
categories, which Lepton needs to start an app in VR. It opens the
setup panel and, when the selected game can start, the game on top.
- XR_VALVE_frame_controller_interaction: the Frame controller profile
with its left D-pad (new dpad_* actions, the Wii Remote's D-pad or the
gamepad's), View as menu and the left shoulder as the panel button.
Also requested on Windows for SteamVR streaming to a Frame.
- Build-Quest.ps1, Build-QuestGame.ps1 and Run-Quest.ps1 take
-Headset frame.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HBRGKTE1GnN2ah8gcZKr3
- Introduced `vrCockpitItemThrow` configuration option to enable throwing items in VR.
- Implemented `ThrowDetector` and `ThrowSequence` classes to manage item throwing mechanics based on hand movements.
- Updated `OpenXRInput` to handle item throwing based on hand swings, integrating with existing VR input systems.
- Enhanced settings overlay to allow users to toggle item throwing functionality.
- Added tests for item throwing mechanics, ensuring correct behavior for various hand movements and conditions.
- Implemented DVDReadVrAsset function to read mapped disc paths for VR assets.
- Created HeldItem structure and ReadHeldItem function for managing held items in the game.
- Developed unit tests for ReadHeldItem to ensure correct functionality and edge case handling.
- Added cockpit item data tests to validate model indexing and parsing of archives.
- Introduced a new configuration option for object culling in VR, allowing the game to hide objects outside the camera's view.
- Implemented native replacements for the Mario Kart functions responsible for scene culling, ensuring accurate behavior in VR.
- Added a new header file `mkw_vr_culling.h` to define the culling logic and structures.
- Created `mkw_vr_culling.cpp` to implement the culling logic, including frustum intersection checks and screen info updates.
- Updated runtime configuration to include the new object culling option, with appropriate getters and setters.
- Enhanced the settings overlay to allow users to toggle object culling in VR.
- Added tests in `vr_culling_tests.cpp` to validate the frustum intersection logic and ensure compliance with original behavior.
- Updated CMake files to include new source files and tests.
- Added support for hiding model arrays in the rendering pipeline to optimize performance during VR gameplay.
- Introduced new functions in `gx_model_visibility` to manage hidden model arrays and their visibility based on game state.
- Enhanced cockpit recentering logic to ensure accurate seat measurements and eye positioning during gameplay.
- Updated `FirstPersonState` to track cockpit height and forward direction, improving VR experience.
- Added tests for Bullet Bill model visibility and cockpit height adjustments to ensure functionality and stability.
- Refactored existing code to accommodate new features and improve overall code organization.
- New [vr] hand_tracking (default off, Quest only for now): the Quest launcher's Settings > VR and
the headset panel's VR tab, under hand steering. Two XR_EXT_hand_tracking trackers are located
every XR frame: with the controllers held the Quest builds the joints from their touch sensors
(XR_EXT_hand_tracking_data_source's controller source), once they are put down from its
cameras. The trackers exist only while the option and hand steering are on and also serve the
runtime hand mesh; the extensions (plus XR_FB_hand_tracking_aim) are asked for when either is
on at launch.
- Aurora skins the runtime mesh with the joints themselves (tracked pose times inverse bind pose,
no curl, no grip); runtimes with joints but no mesh get a skeleton; non-finite joints put only
that hand back on its grip curl. AuroraCockpitHand carries the 26 seated-frame joints and radii.
- The manifest declares horizonos.permission.HAND_TRACKING (and the deprecated
com.oculus.permission.HAND_TRACKING), both normal permissions with no prompt on a Quest 3, and
oculus.software.handtracking as optional; without it Horizon OS keeps the app controllers-only.
Bare hands then drive khr/simple_controller, so on Android a hand whose squeeze action is
inactive and select active is treated as bare: with the option off it presses nothing but the
menu gesture, is not drawn and feeds no Wii Remote motion.
- Interaction-profile changes and tracker sources are logged. The pure rules (grasp from finger
flexion, bare latch, pinch gate, bare-hand buttons, flick) live in the OpenXR-free
vr/openxr_hand_tracking.h with mkw_vr_hand_tracking_tests; the driving and flick parts are
wired in the next commits. Docs: OPENXR.md, docs/quest-port.md, README.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Eyes render under a VK_EXT_fragment_density_map: full rate around each eye's forward direction,
2x2 then 4x4 pixel blocks towards the edges ([vr] foveation = off|low|medium|high, default off).
XR_FB_foveation cannot help here: the runtime's maps only shape passes drawing into its
swapchain, and the eyes reach it through a copy.
- aurora-main/patches/dawn/aurora_fdm.inc: Dawn enables the extension only on request and for
dynamic rendering, flags every render pipeline, and chains an immutable RG8 map into any pass
whose first color attachment is a view bound to one (ABI: include/aurora/dawn_fdm_abi.h).
- android/Build-QuestDawn.ps1 builds the pinned Dawn revision with those patches for arm64
(dawn-build CI flags, protobuf off) into a cached package; Build-Quest.ps1 links it
(-StockDawn opts out) and AuroraDawnProvider.cmake enables the ABI from its manifest.
- lib/gfx/foveation.hpp generates the maps (32 px per texel, densities 255/127/63); an eye is
foveated only when single_pass_eyes draws it in one render pass. Menus never are.
- Live level from the headset panel's VR tab and the launcher; the launch decides whether the
device has maps. debug.wiicompiled.foveation and debug.wiicompiled.fdm for A/B.
- Tests: Foveation cases in gx_fifo_tests, mkw_vr_config_tests. Docs: OPENXR.md, quest-port.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Clicking the right thumbstick flips first person exactly as the F10
checkbox does, and saves it the same way. It works on the VR controllers
in either presentation (with a short tick on the right controller) and on
any other gamepad while VR is running. It never reached the game: SDL's
stick click only lands in Aurora's extended buttons.
A click counts on release, and only if the left thumbstick stayed up and
the headset settings panel stayed closed throughout, so the two-thumbstick
panel chord in gamepad mode never toggles the camera. A gamepad whose
right stick click is bound to a GameCube control on its port, as a button
or in an input expression, keeps it for the game. The XR side only posts
a request; the toggle itself runs on the game thread with the checkbox.
[vr] first_person_toggle_click (default true) and a checkbox under the
camera toggle turn the click off. mkw_vr_camera_toggle_tests covers the
click rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ported from heurazy's mario-kart-wii-VR-port. F10 > Controllers > USB
wheel and pedals (also in the headset panel) picks and calibrates the
steering axis and both pedals by moving them, and assigns buttons by
pressing them, over raw SDL joysticks: no per-model table or gamepad
mapping, and separate pedals, reversed axes and combined pedal axes all
calibrate the same way. Settings live in PhysicalWheel.toml beside
Config.toml; heurazy's five-button files load unchanged.
The wheel is player 1's GameCube controller. In a race it owns port 0
(steering on the stick, the accelerator on A, the brake pedal braking then
reversing over A and drift, the paddles on R and L), keeping only the
other source's pause and item aim. In menus it adds only deliberate
presses. It arms once every pedal and button is released, stays neutral
in a race while its setup is incomplete, and nothing is opened until it
is enabled. Optional light rumble follows the game's own, capped at 15 %.
Added here: its confirm, a new back button and the steering device's
first hat (D-pad) work the menus, since our VR controllers default to a
Wii Remote and the wheel needs to navigate on its own. Devices SDL
classifies as wheels are marked in the list.
In the VR cockpit the wheel follows the hardware steering and hand
steering steps aside while it drives. README documents setup and
Logitech notes (G HUB, the G29's PS3 switch, shifter gears as buttons);
not yet tried on a physical wheel.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The pure pieces of the first-person cockpit and hand steering, ported from
heurazy's mario-kart-wii-VR-port: the SteeringWheel grab/turn model, the
native wheel vertex rotation, the level seat stabiliser, the seated-eye and
wheel/handlebar geometry, and the XR_FB_hand_tracking_mesh loader. Adds
openxr_driving.h, the OpenXR-free snapshot the pacing thread will publish for
the guest thread, with the hand-off rule (a held wheel replaces the left
stick's X and releases that hand's grip for the game) and the wheel's
displayed angle.
New [vr] keys: first_person_seat (cockpit), cockpit_units_per_meter (100),
steering_wheel (true), native_steering_wheel (true), hand_steering (false)
and the seven wheel_* tuning keys. Nothing reads them yet.
The fresh-config template now writes the first-person defaults the
constants hold (50 / 1.5 / 0); d86dcb0 updated the constants but not the
template.
Tests: mkw_steering_wheel_tests (the fork's), mkw_vr_cockpit_tests,
mkw_vr_hand_steering_tests.
Brings in 25 upstream commits: Dolphin-compatible input expressions and
GCPadNew.ini import, NAND setting.txt console identity, empty-MKW-save
handling, rumble toggle, LLVM 22, and CI caching.
The only conflict was runtime/CMakeLists.txt, where both sides appended
test targets after mkw_platform_paths_tests. Both blocks are kept: the VR
first-person test alongside upstream's NAND save/settings, SC serial, and
input expression tests.
runtime_config.h and settings_overlay.cpp auto-merged; the VR settings
menu, recenter hotkey, and stereo/first-person init calls are intact
alongside upstream's InputBindings wiring.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* treat empty mkw save as missing
first-run format zero-fills rksys.dat before any real save; a quit before
the first save left an all-zero file that read back as corrupt and trapped
the user in a delete/recreate loop. read opens now treat an all-zero
rksys.dat as absent (a real save always begins with the RKSD0006 header),
so the game recreates it from scratch. also ignore native build output.
* shorten
* I dont really want to change this to be honest.
* extra safety
---------
Co-authored-by: patchzyy <64382339+patchzyy@users.noreply.github.com>
* Add Dolphin-compatible input expressions and GCPadNew.ini import
Rebased onto current main; addresses both CodeRabbit reviews on #89.
- Expression engine matching Dolphin's semantics: doubles rather than
booleans, 0.5 press threshold, & as min, | as max, and the functions if,
min, max, clamp, abs, sqrt, pow, sin, cos, tan, deadzone, timer, toggle,
hold, tap, pulse and smooth. Timing uses a steady clock in seconds, as
Dolphin does, so a copied expression behaves identically.
- Expressions bind to the GameCube buttons and triggers, combined with the
existing button mapping rather than replacing it, and are skipped while
the settings overlay holds input.
- Import reads [GCPadN] from the Dolphin config directory or from
GCPadNew.ini beside the executable. Stick axes are not expression driven
and keep their normal mapping.
- Fixes#74: a digital button bound to L or R now reports a fully pulled
analog trigger, plus a PlayStation preset and a vibration toggle.
Review fixes: config paths round-trip through RuntimeConfigFile::PathToUtf8
and PathFromUtf8 so non-ASCII paths open correctly on Windows, and the
duplicated exists branch is gone; the tap count is clamped before the
unsigned conversion; the expression editor uses resizable storage via
ImGuiInputTextFlags_CallbackResize so a long expression cannot be saved
truncated; clamp bounds are ordered before std::clamp; <cstdlib> is included
for std::strtod; non-finite values are rejected at the evaluator boundary as
well as at the deadzone and timer divisions; and InputBindings::Reload() runs
from InitializeRuntimeSettings rather than the vibration handler.
runtime/tests/test_expr.cpp covers operator precedence, each stateful
function and every case raised in review.
Third review round: smooth() guards NaN as well as infinity so a zero rate
cannot latch a non-finite value in node state; division evaluates both operands
so stateful functions in the left subtree still update when the divisor is zero;
the expression editor clears stale errors when the port changes; and
runtime/tests/test_expr.cpp is registered with CTest as mkw_input_expr_tests,
following the existing test targets.
* Update runtime/src/input_expr.cpp
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update runtime/src/input_expr.cpp
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
* Update runtime/src/input_expr.cpp
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
---------
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Claude