mirror of
https://github.com/mitch030504/Wiicompiled_VR_Frame.git
synced 2026-10-06 05:00:27 +02:00
Document the Steam Frame build
docs/steam-frame.md covers the steamFrame flavour: the CPU target and why it excludes SVE, the Lepton entry activity and manifest, the Frame controller map, the refresh rate and eye-tracked foveation, building and installing, the device checklist (what to collect from the headset, the log lines a first session should show, the open questions), and what a native SteamOS build would need. OPENXR.md documents refresh_rate and eye_tracked_foveation and the Frame controller profile; the Quest doc, the Android README and the README point to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019HBRGKTE1GnN2ah8gcZKr3
This commit is contained in:
5 files changed
+282
-7
No files matched your search
@@ -40,6 +40,7 @@ required = false
|
|||||||
mirror_view = "normal"
|
mirror_view = "normal"
|
||||||
controller_mode = "wii_remote"
|
controller_mode = "wii_remote"
|
||||||
frame_interpolation_fps = 0
|
frame_interpolation_fps = 0
|
||||||
|
refresh_rate = 0
|
||||||
render_scale = 1.0
|
render_scale = 1.0
|
||||||
world_units_per_meter = 500.0
|
world_units_per_meter = 500.0
|
||||||
hud_distance_meters = 2.0
|
hud_distance_meters = 2.0
|
||||||
@@ -217,11 +218,19 @@ its CPU and GPU domains: `boost`, `sustained_high`, `sustained_low`, `power_savi
|
|||||||
request (see `docs/quest-port.md`); desktop runtimes rarely offer the extension, and the setting
|
request (see `docs/quest-port.md`); desktop runtimes rarely offer the extension, and the setting
|
||||||
then does nothing. It is read at launch, and the session log records whether the runtime accepted
|
then does nothing. It is read at launch, and the session log records whether the runtime accepted
|
||||||
it and any later performance notification (a thermal or rendering warning).
|
it and any later performance notification (a thermal or rendering warning).
|
||||||
`foveation` (Quest only, default `medium`) shades the edges of the immersive race view more coarsely:
|
`foveation` (Quest and Steam Frame only, default `medium`) shades the edges of the immersive race view more coarsely:
|
||||||
`off`, `low`, `medium` or `high`, see [Foveated rendering](#foveated-rendering). A session launched
|
`off`, `low`, `medium` or `high`, see [Foveated rendering](#foveated-rendering). A session launched
|
||||||
with it off runs without fragment density maps, so going from `off` to a level takes a restart;
|
with it off runs without fragment density maps, so going from `off` to a level takes a restart;
|
||||||
between levels, and back to `off`, it is live from the headset panel's VR tab. The launcher's
|
between levels, and back to `off`, it is live from the headset panel's VR tab. The launcher's
|
||||||
Settings page has it too.
|
Settings page has it too. `eye_tracked_foveation` (default on for the Steam Frame, off elsewhere)
|
||||||
|
centres it on the player's gaze where the runtime offers `XR_EXT_eye_gaze_interaction` with an eye
|
||||||
|
tracker; see [Eye-tracked foveation](#eye-tracked-foveation).
|
||||||
|
`refresh_rate` is the display rate in Hz asked of the runtime through `XR_FB_display_refresh_rate`
|
||||||
|
each time the session starts and whenever it changes, or `0` (the default, `120` on the Steam
|
||||||
|
Frame) to leave the headset's own. The game renders 60 frames a second, so 120 Hz shows each frame
|
||||||
|
for exactly two refreshes. A rate the runtime does not list, or declines, is logged and leaves its
|
||||||
|
own; setting `0` again restores the rate the session started at. It is live from F10 / the headset
|
||||||
|
panel (*Headset refresh rate*) and the Quest launcher; runtimes without the extension ignore it.
|
||||||
|
|
||||||
## Controllers
|
## Controllers
|
||||||
|
|
||||||
@@ -269,6 +278,12 @@ because hand steering holds a grip down for a whole corner, and C is the game's
|
|||||||
game's Wii Remote rumble vibrates both controllers, subject to the ordinary controller-vibration
|
game's Wii Remote rumble vibrates both controllers, subject to the ordinary controller-vibration
|
||||||
switch.
|
switch.
|
||||||
|
|
||||||
|
The Steam Frame's controllers get their own profile where the runtime offers it
|
||||||
|
(`XR_VALVE_frame_controller_interaction`, `/interaction_profiles/valve/frame_controller_valve`):
|
||||||
|
right A, B, trigger and stick as above, left View as the left menu (+), the left shoulder as left Y
|
||||||
|
(the settings panel), and the left D-pad as the Wii Remote's D-pad (the gamepad's D-pad in
|
||||||
|
`"gamepad"` mode). The table is in `docs/steam-frame.md`.
|
||||||
|
|
||||||
**Motion.** Each XR frame the aim and grip poses are located at the measured current time
|
**Motion.** Each XR frame the aim and grip poses are located at the measured current time
|
||||||
(`XR_KHR_win32_convert_performance_counter_time`, `XR_KHR_convert_timespec_time` on Android), not
|
(`XR_KHR_win32_convert_performance_counter_time`, `XR_KHR_convert_timespec_time` on Android), not
|
||||||
the predicted display time, whose extrapolation sprays fast wrist motion. The grip's linear
|
the predicted display time, whose extrapolation sprays fast wrist motion. The grip's linear
|
||||||
@@ -1068,6 +1083,17 @@ ripples give way to a smoother look. It is no fix for a heavy track: on Retro Re
|
|||||||
2 at 1.0, GPU-bound at about 40 FPS, no level raised the frame rate, while merging the eye passes
|
2 at 1.0, GPU-bound at about 40 FPS, no level raised the frame rate, while merging the eye passes
|
||||||
did (39 to 41.5 FPS).
|
did (39 to 41.5 FPS).
|
||||||
|
|
||||||
|
### Eye-tracked foveation
|
||||||
|
|
||||||
|
With `eye_tracked_foveation`, a runtime that offers `XR_EXT_eye_gaze_interaction` and reports an eye
|
||||||
|
tracker (the Steam Frame's SteamVR) has its gaze pose located for each packet's display time and
|
||||||
|
turned into tangents of each eye's view (`vr/eye_gaze.h`), which `AuroraStereoFrame` carries as
|
||||||
|
`gaze`/`gazeValid`. Aurora centres the level's rings on the gaze snapped to a cell of two map texels
|
||||||
|
(about 3 degrees), keeping up to 32 maps per eye, one per cell looked at, and binds a new one once
|
||||||
|
its upload completes, the previous map staying bound meanwhile. Without a tracked gaze (a blink, no
|
||||||
|
tracker, the setting off) the map is the forward one above, unchanged. Details and the Steam Frame
|
||||||
|
checks are in `docs/steam-frame.md`.
|
||||||
|
|
||||||
## Diagnostics
|
## Diagnostics
|
||||||
|
|
||||||
**F10 > Diagnostics** holds three bug-report aids.
|
**F10 > Diagnostics** holds three bug-report aids.
|
||||||
@@ -1250,6 +1276,7 @@ custom DLL's ABI through three borrowed-image copy/readback cycles; run it with
|
|||||||
| Windows D3D12 | Implemented: same-adapter, same-device asynchronous OpenXR submission. |
|
| Windows D3D12 | Implemented: same-adapter, same-device asynchronous OpenXR submission. |
|
||||||
| Windows Vulkan | Implemented, opt-in (`video.graphics_api = "vulkan"`): the runtime creates Dawn's Vulkan instance and device through `XR_KHR_vulkan_enable2`, eyes are copied on the same queue, and Dawn's device guard is held around the four queue-touching OpenXR calls. Needs the custom Dawn from `Launcher/Build-DawnVulkan.ps1`. Raced on SteamVR/PSVR2 at the headset's full rate; other runtimes unexercised. See [Windows Vulkan](#windows-vulkan). |
|
| Windows Vulkan | Implemented, opt-in (`video.graphics_api = "vulkan"`): the runtime creates Dawn's Vulkan instance and device through `XR_KHR_vulkan_enable2`, eyes are copied on the same queue, and Dawn's device guard is held around the four queue-touching OpenXR calls. Needs the custom Dawn from `Launcher/Build-DawnVulkan.ps1`. Raced on SteamVR/PSVR2 at the headset's full rate; other runtimes unexercised. See [Windows Vulkan](#windows-vulkan). |
|
||||||
| Android Vulkan (Meta Quest) | Implemented and running on a Quest 3: the OpenXR side owns its own Vulkan device (`XR_KHR_vulkan_enable2`, `XR_KHR_vulkan_enable` fallback) and shares eyes with Dawn through `AHardwareBuffer`s ordered by sync-fd fences. Controllers arrive through OpenXR actions as a virtual SDL gamepad. See `docs/quest-port.md`. |
|
| Android Vulkan (Meta Quest) | Implemented and running on a Quest 3: the OpenXR side owns its own Vulkan device (`XR_KHR_vulkan_enable2`, `XR_KHR_vulkan_enable` fallback) and shares eyes with Dawn through `AHardwareBuffer`s ordered by sync-fd fences. Controllers arrive through OpenXR actions as a virtual SDL gamepad. See `docs/quest-port.md`. |
|
||||||
|
| Android Vulkan (Steam Frame) | The same backend in the `steamFrame` flavour, for SteamVR's Android runtime under Lepton: Frame controller profile, 120 Hz request, eye-tracked foveation. Built and unit-tested, not yet run on the headset. See `docs/steam-frame.md`. |
|
||||||
| Linux Vulkan | Not wired. The pinned Dawn package does not expose a native Vulkan device, and the AHardwareBuffer bridge is Android-only; a dma-buf/opaque-fd variant of the same design would cover desktop Linux. |
|
| Linux Vulkan | Not wired. The pinned Dawn package does not expose a native Vulkan device, and the AHardwareBuffer bridge is Android-only; a dma-buf/opaque-fd variant of the same design would cover desktop Linux. |
|
||||||
| Other platforms | Not wired yet. |
|
| Other platforms | Not wired yet. |
|
||||||
|
|
||||||
|
|||||||
@@ -57,6 +57,9 @@ to immersive stereo rendering. VR is opt-in and falls back to the normal desktop
|
|||||||
runtime or headset is unavailable. In first person you sit in the cockpit, where the steering wheel
|
runtime or headset is unavailable. In first person you sit in the cockpit, where the steering wheel
|
||||||
or handlebar turns with your steering, and hand steering by heurazy lets you grab it with the
|
or handlebar turns with your steering, and hand steering by heurazy lets you grab it with the
|
||||||
tracked controllers and turn it. On a Quest the hands can follow the headset's own hand tracking.
|
tracked controllers and turn it. On a Quest the hands can follow the headset's own hand tracking.
|
||||||
|
A Steam Frame build of the Android app (not yet tested on the headset) adds the Frame controllers'
|
||||||
|
D-pad, a 120 Hz display for the game's 60 FPS, and foveation that follows your eyes; see
|
||||||
|
[`docs/steam-frame.md`](docs/steam-frame.md).
|
||||||
See [`OPENXR.md`](OPENXR.md) for setup, configuration, and the current limitations.
|
See [`OPENXR.md`](OPENXR.md) for setup, configuration, and the current limitations.
|
||||||
|
|
||||||
**Music ducking.**
|
**Music ducking.**
|
||||||
|
|||||||
+4
-2
@@ -1,8 +1,9 @@
|
|||||||
# WiiCompiled VR for Meta Quest (Android)
|
# WiiCompiled VR for Meta Quest (Android)
|
||||||
|
|
||||||
Standalone Android/OpenXR build of the Mario Kart Wii recompilation for Quest 1,
|
Standalone Android/OpenXR build of the Mario Kart Wii recompilation for Quest 1,
|
||||||
Quest 2, Quest 3, Quest 3S and Quest Pro. The full design, build walkthrough and current
|
Quest 2, Quest 3, Quest 3S and Quest Pro, and, as the `steamFrame` flavour, for Valve's Steam
|
||||||
status live in [docs/quest-port.md](../docs/quest-port.md); this directory only
|
Frame under Lepton ([docs/steam-frame.md](../docs/steam-frame.md)). The full design, build
|
||||||
|
walkthrough and current status live in [docs/quest-port.md](../docs/quest-port.md); this directory only
|
||||||
holds the Gradle project, its helper scripts, the game kit tooling
|
holds the Gradle project, its helper scripts, the game kit tooling
|
||||||
(`QuestGameKit.psm1`, `Build-QuestGame.ps1`), the on-headset build toolchain
|
(`QuestGameKit.psm1`, `Build-QuestGame.ps1`), the on-headset build toolchain
|
||||||
(`Prepare-QuestToolchain.ps1`, `toolchain/`) and `nod-jni`.
|
(`Prepare-QuestToolchain.ps1`, `toolchain/`) and `nod-jni`.
|
||||||
@@ -10,6 +11,7 @@ holds the Gradle project, its helper scripts, the game kit tooling
|
|||||||
```powershell
|
```powershell
|
||||||
powershell -ExecutionPolicy Bypass -File android/Prepare-QuestDependencies.ps1 # stages the SDL3 3.4.4 AAR once
|
powershell -ExecutionPolicy Bypass -File android/Prepare-QuestDependencies.ps1 # stages the SDL3 3.4.4 AAR once
|
||||||
powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Install # the app, debug-signed, installs over adb
|
powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Install # the app, debug-signed, installs over adb
|
||||||
|
powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Headset frame # Steam Frame flavour
|
||||||
powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Headset quest1 -Install # Quest 1 flavour
|
powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Headset quest1 -Install # Quest 1 flavour
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
+6
-3
@@ -819,9 +819,11 @@ Android facts this design rests on, all measured on a Quest 3:
|
|||||||
`generate-data-init` and `translate-mod`, for pipelines that generate on
|
`generate-data-init` and `translate-mod`, for pipelines that generate on
|
||||||
another host.
|
another host.
|
||||||
- `android/`: the Gradle project, with `modernQuest` (the default script
|
- `android/`: the Gradle project, with `modernQuest` (the default script
|
||||||
target) and `quest1` headset flavours. They share the application ID and
|
target), `quest1` and `steamFrame` headset flavours. They share the
|
||||||
storage, but select the appropriate CPU baseline, supported-device manifest,
|
application ID and storage, but select the appropriate CPU baseline (one
|
||||||
and launcher behavior.
|
`headsetCpus` map in `app/build.gradle.kts`), manifest and launcher
|
||||||
|
behavior. `steamFrame` is Valve's Steam Frame under Lepton, SteamOS's
|
||||||
|
Android layer; see `docs/steam-frame.md`.
|
||||||
`app/src/main/cpp/CMakeLists.txt` adds the repository's `runtime/` as a
|
`app/src/main/cpp/CMakeLists.txt` adds the repository's `runtime/` as a
|
||||||
subdirectory with those Android choices and builds both game kit probes
|
subdirectory with those Android choices and builds both game kit probes
|
||||||
(the Retro Rewind one only when the translation includes the mod), which
|
(the Retro Rewind one only when the translation includes the mod), which
|
||||||
@@ -850,6 +852,7 @@ powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Headset quest1
|
|||||||
powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Install # your game, against that kit, into Import (or WheelWizard VR's Build for Quest)
|
powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Install # your game, against that kit, into Import (or WheelWizard VR's Build for Quest)
|
||||||
powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Product retro_rewind -Mod <RetroRewind6> -Install # the mod and its pack (needs translate-mod output with --retro-wfc-payload)
|
powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Product retro_rewind -Mod <RetroRewind6> -Install # the mod and its pack (needs translate-mod output with --retro-wfc-payload)
|
||||||
powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Headset quest1 -Install # game package from the Quest 1 kit
|
powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Headset quest1 -Install # game package from the Quest 1 kit
|
||||||
|
powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Headset frame # Steam Frame flavour (docs/steam-frame.md)
|
||||||
adb push MarioKart.iso /sdcard/Download/ # then Select disc image in the launcher
|
adb push MarioKart.iso /sdcard/Download/ # then Select disc image in the launcher
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,240 @@
|
|||||||
|
# WiiCompiled VR on the Steam Frame
|
||||||
|
|
||||||
|
Valve's Steam Frame runs SteamOS on a Snapdragon 8 Gen 3 (Cortex-X4, A720 and A520 cores, Adreno 750),
|
||||||
|
with 2160x2160 panels per eye at 72 to 144 Hz, eye tracking, and SteamVR as its OpenXR runtime. It runs
|
||||||
|
Android apps through Lepton, SteamOS's Android layer, where SteamVR provides an Android OpenXR runtime
|
||||||
|
(OpenXR 1.0, through the Khronos loader's runtime broker). The Steam Frame build is therefore a third
|
||||||
|
flavour of the Quest app, `steamFrame`: everything in `docs/quest-port.md` below the app shell (the
|
||||||
|
Vulkan backend, the game kit, `.wcgame` packages, the on-headset build) applies unchanged, and this
|
||||||
|
document covers what differs.
|
||||||
|
|
||||||
|
A native SteamOS ARM64 build is a separate, later piece of work: Linux has no OpenXR graphics backend
|
||||||
|
yet (see [A native SteamOS build](#a-native-steamos-build)).
|
||||||
|
|
||||||
|
**Status: not yet run on a Steam Frame.** Everything below compiles and is unit-tested, but the
|
||||||
|
device checks at the end are still to do.
|
||||||
|
|
||||||
|
## What the flavour changes
|
||||||
|
|
||||||
|
| | Quest flavours | `steamFrame` |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| CPU target (`kit.json` `androidCpu`) | `cortex-a77` (`kryo` on Quest 1) | `cortex-x4+nosve` |
|
||||||
|
| `MKW_ANDROID_HEADSET` | `quest` | `steam_frame` (defines `MKW_HEADSET_STEAM_FRAME`) |
|
||||||
|
| Library entry | `LauncherActivity` (Quest 1: `QuestActivity`) | `FrameEntryActivity` |
|
||||||
|
| Horizon OS manifest entries | present | removed |
|
||||||
|
| `[vr] refresh_rate` default | `0` (the headset's own) | `120` |
|
||||||
|
| `[vr] passthrough` | default on (`XR_FB_passthrough`) | not asked for, default off, setting hidden |
|
||||||
|
| `[vr] eye_tracked_foveation` default | off | on |
|
||||||
|
|
||||||
|
The application ID stays `org.wiicompiled.quest`, so the storage paths in `docs/quest-port.md` hold
|
||||||
|
as they are. The kit's CPU string differs from the Quest ones, which gives the Frame its own kit
|
||||||
|
fingerprint: a game built for a Quest is refused on the Frame and the other way round, by the same
|
||||||
|
checks that keep Quest 1 and modern Quest games apart.
|
||||||
|
|
||||||
|
**CPU.** Every core of the 8 Gen 3 implements ARMv9.2, so the products are tuned for the Cortex-X4.
|
||||||
|
`+nosve` matters: clang auto-vectorises with SVE for a `cortex-x4` (a simple loop compiled with
|
||||||
|
`-O3` used SVE registers ten times), and Qualcomm's firmware does not expose SVE on this chip, so
|
||||||
|
those instructions would end the game with `SIGILL`. With `+nosve` the target features read
|
||||||
|
`-sve -sve2 -sve2-bitperm` and the same loop uses NEON only. The flavour-to-CPU map lives once in
|
||||||
|
`android/app/build.gradle.kts` (`headsetCpus`), which the kit export also reads now instead of
|
||||||
|
guessing from the variant name.
|
||||||
|
|
||||||
|
**Launch under Lepton.** Lepton starts the one real activity that is both `MAIN` and `LAUNCHER`, and
|
||||||
|
runs the app in VR when that activity carries a VR category; it ignores `activity-alias` entries.
|
||||||
|
Quest builds put `LAUNCHER` on the 2D panel (or, on Quest 1, add an alias), so neither works there.
|
||||||
|
`src/steamFrame/AndroidManifest.xml` makes `FrameEntryActivity` the only `MAIN`/`LAUNCHER` activity,
|
||||||
|
with `org.khronos.openxr.intent.category.IMMERSIVE_HMD` and `com.oculus.intent.category.VR`. It
|
||||||
|
shows nothing: it always opens the setup panel (`LauncherActivity`), and when the selected game can
|
||||||
|
start as it is (game files, a game built for this kit, Retro Rewind's pack, and no enabled mods
|
||||||
|
still to copy into the pack), it opens `QuestActivity` on top of it. The headset therefore goes
|
||||||
|
straight into VR, and quitting the game returns to the panel for setup, imports and mods.
|
||||||
|
`adb logcat -s WiiCompiledLauncher` shows which way it went.
|
||||||
|
|
||||||
|
The manifest also removes Horizon OS's own entries (`com.oculus.supportedDevices`, `focusaware`,
|
||||||
|
`trade_cpu_for_gpu_amount`, the passthrough feature and the hand tracking permissions and feature)
|
||||||
|
and keeps the Khronos broker queries and the `OPENXR_SYSTEM` permission, which any Android OpenXR
|
||||||
|
runtime needs.
|
||||||
|
|
||||||
|
## Controllers
|
||||||
|
|
||||||
|
With `XR_VALVE_frame_controller_interaction` the runtime offers the Frame controller's own profile,
|
||||||
|
`/interaction_profiles/valve/frame_controller_valve`. Without it SteamVR presents the controllers as
|
||||||
|
Touch controllers, which loses the left D-pad. `openxr_input.cpp` suggests it after Touch and the
|
||||||
|
simple controller. Each hand has a thumbstick, trigger, grip and shoulder button. The right hand has
|
||||||
|
A, B, X, Y and a menu button; the left hand has a D-pad and a View button. Binding paths follow
|
||||||
|
DolphinXR's port (iChris4/dolphinXR#9). Windows asks for the same extension, so a Frame streaming
|
||||||
|
from a PC through SteamVR gets the D-pad too.
|
||||||
|
|
||||||
|
| Frame controller | Wii Remote mode | Gamepad mode |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Right A | A | South (A) |
|
||||||
|
| Right B | C (look behind) | East (B) |
|
||||||
|
| Right trigger | B | Right trigger |
|
||||||
|
| Right stick up / down | 1 / 2 | Right stick |
|
||||||
|
| Left View | + (pause) | Start |
|
||||||
|
| Left shoulder | Settings panel (Touch's left Y) | North (Y) |
|
||||||
|
| Left D-pad | Wii Remote D-pad | D-pad |
|
||||||
|
| Left stick, left trigger | Nunchuk stick, Z | Left stick, left trigger |
|
||||||
|
| Grips, stick clicks, motion, aim | as on Touch (`OPENXR.md`, Controllers) | as on Touch |
|
||||||
|
| Right X, Y, menu and shoulder | unbound | unbound |
|
||||||
|
|
||||||
|
The four D-pad actions are new and also reach the virtual gamepad's D-pad; Touch leaves them unbound,
|
||||||
|
so nothing changes on a Quest.
|
||||||
|
|
||||||
|
## Refresh rate
|
||||||
|
|
||||||
|
`[vr] refresh_rate` (Hz, `0` = the headset's own rate) is asked of the runtime through
|
||||||
|
`XR_FB_display_refresh_rate` each time the session starts running and whenever the setting changes.
|
||||||
|
The request uses the runtime's own value within half a hertz of the setting (runtimes report 119.98
|
||||||
|
for 120). Setting it back to `0` restores the rate the session started at. The game renders 60 frames
|
||||||
|
a second, so at 120 Hz each frame shows for exactly two refreshes. At 72 or 90 Hz some frames show
|
||||||
|
for one refresh and others for two, which judders. The Frame starts at 120. Render-first pacing
|
||||||
|
(`docs/quest-port.md`) already waits for each sealed game frame, so on the Frame the pacing summary
|
||||||
|
should read about 60 `skipped-slots` a second with no `late` cycles.
|
||||||
|
|
||||||
|
Lepton may decline the request (frame-control found SteamVR keeping its own rate there). The session
|
||||||
|
log then says `display refresh rate 120 Hz refused` with the rates it offers, and nothing else
|
||||||
|
changes. The setting is in the headset panel and in the launcher (VR → Headset); other headsets
|
||||||
|
offer it as well, at their own default of `0`.
|
||||||
|
|
||||||
|
## Eye-tracked foveation
|
||||||
|
|
||||||
|
`[vr] eye_tracked_foveation` (default on for the Frame) moves the foveation level's full-density
|
||||||
|
region to where the player looks:
|
||||||
|
|
||||||
|
1. At launch the runtime asks for `XR_EXT_eye_gaze_interaction`. If the system reports an eye tracker,
|
||||||
|
`OpenXRInput` binds the gaze pose (`/user/eyes_ext/input/gaze_ext/pose`) and locates it for each
|
||||||
|
packet's display time, in the space the eye views are located in.
|
||||||
|
2. `vr/eye_gaze.h` turns the gaze into tangents of each eye's own view, which may be canted.
|
||||||
|
`AuroraStereoFrame` carries them as `gaze` and `gazeValid`, appended after its existing fields.
|
||||||
|
3. Aurora snaps the gaze to a cell of two map texels (64 pixels, about 3 degrees) and builds that
|
||||||
|
cell's density map with the level's rings centred on the gaze (`gfx/foveation.hpp`).
|
||||||
|
|
||||||
|
Each eye keeps up to 32 maps, one per cell looked at, so a glance back reuses its map instead of
|
||||||
|
uploading a new one. A new map is bound once its upload has completed, and until then the eye keeps
|
||||||
|
the map it had. A blink, lost tracking or the setting turned off returns to the map centred on the
|
||||||
|
forward direction, which is byte-identical to the fixed foveation map. No change to the Dawn patch
|
||||||
|
was needed: maps stay immutable, and the patch already lets a view be rebound to another map.
|
||||||
|
|
||||||
|
The session log reports `OpenXR eye gaze: available` (or that the runtime has no eye tracker),
|
||||||
|
`OpenXR eye gaze: tracking` at the first tracked sample, and `eye foveation medium following the
|
||||||
|
gaze` for each eye's first gaze map. Foveation pays only when an eye is pixel-bound
|
||||||
|
(`OPENXR.md`, Foveated rendering), which the Frame's larger eyes make more likely.
|
||||||
|
|
||||||
|
If the Frame's driver offers `VK_QCOM_fragment_density_map_offset` or
|
||||||
|
`VK_EXT_fragment_density_map_offset`, shifting one map per pass would replace switching between
|
||||||
|
maps. That needs the Dawn patch to create the eye textures with the offset flag, so it waits for
|
||||||
|
the device's extension list.
|
||||||
|
|
||||||
|
## Building and installing
|
||||||
|
|
||||||
|
On the Windows build host described in `docs/quest-port.md`:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Headset frame # the steamFrame APK; checks kit.json says cortex-x4+nosve
|
||||||
|
powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Headset frame -Product base -Data <DATA> # a .wcgame for the Frame's kit
|
||||||
|
```
|
||||||
|
|
||||||
|
The APK lands in `android/app/build/outputs/apk/steamFrame/<configuration>`. WheelWizard VR's
|
||||||
|
"Build for Quest" builds a Frame game unchanged once it is given the Frame APK (Setup takes the
|
||||||
|
headset from the APK's kit). WheelWizard itself still needs a Steam Frame choice that fetches that
|
||||||
|
APK from the release, published as `…-SteamFrame.apk`.
|
||||||
|
|
||||||
|
Lepton opens an adb port (5555 and up) for each running Android instance, reachable over the
|
||||||
|
network. With an Android app running on the Frame, `adb connect <frame-ip>:5555` reaches it from the
|
||||||
|
build PC. How the APK reaches the Steam library (adb into a Lepton instance, frame-control, or
|
||||||
|
Steam's own sideloading) is to be confirmed on the device.
|
||||||
|
|
||||||
|
## Device checklist
|
||||||
|
|
||||||
|
Information to collect first, from the build PC over Lepton's adb port, with an Android app running
|
||||||
|
on the Frame:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
adb connect <frame-ip>:5555
|
||||||
|
$f = "<frame-ip>:5555"
|
||||||
|
adb -s $f shell "getprop ro.build.version.release; getprop ro.build.version.sdk; getprop ro.product.manufacturer; getprop ro.product.model; getprop ro.product.device; getprop ro.hardware.vulkan; getprop ro.board.platform"
|
||||||
|
adb -s $f shell "uname -a; getconf PAGE_SIZE; grep -m1 Features /proc/cpuinfo; grep -c processor /proc/cpuinfo"
|
||||||
|
adb -s $f shell cmd gpu vkjson > frame-vkjson.json
|
||||||
|
adb -s $f shell "pm list packages | grep -i -E 'xr|valve|steam|khronos|openxr'"
|
||||||
|
```
|
||||||
|
|
||||||
|
and on the Frame itself (Desktop Mode, Konsole):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
uname -a; getconf PAGESIZE; grep -m1 Features /proc/cpuinfo; head -5 /etc/os-release
|
||||||
|
cat ~/.config/openxr/1/active_runtime.json 2>/dev/null; ls /usr/share/openxr/1/ /etc/xdg/openxr/1/ 2>/dev/null
|
||||||
|
```
|
||||||
|
|
||||||
|
What each answers:
|
||||||
|
|
||||||
|
- `vkjson`: whether Lepton's Vulkan driver has what the Android bridge needs:
|
||||||
|
- `VK_ANDROID_external_memory_android_hardware_buffer`;
|
||||||
|
- `VK_KHR_external_semaphore_fd` and `VK_KHR_external_fence_fd` with sync fd handles.
|
||||||
|
|
||||||
|
Without these the APK route cannot present at all, and the native route becomes the way. It also
|
||||||
|
shows `VK_EXT_fragment_density_map` (foveation), the density map offset extensions, and which
|
||||||
|
driver Lepton uses.
|
||||||
|
- The CPU features line: no `sve`, as the CPU target assumes.
|
||||||
|
- The page size: on a 16 KB-page kernel the runtime finds out at launch and routes translated memory
|
||||||
|
accesses through the checked path (`guest_flat_memory.h`, `RequiresCheckedAccess`), which works but
|
||||||
|
is slower. That is worth knowing before measuring.
|
||||||
|
- The Android version: the app needs API 29 (Android 10) or newer.
|
||||||
|
|
||||||
|
Then, on the first launch, the session log (`Logs/<product>_<stamp>_pid<pid>/console.log` next to
|
||||||
|
`DATA`) and `adb logcat -s SDL WiiCompiledQuest WiiCompiledLauncher` should show, in order:
|
||||||
|
|
||||||
|
1. `OpenXR Android loader initialized`;
|
||||||
|
2. `OpenXR runtime offers N extensions: ...`, the whole list SteamVR's Android runtime has;
|
||||||
|
3. `OpenXR initialized: runtime '...'` with SteamVR's name;
|
||||||
|
4. the negotiated Vulkan binding, then `OpenXR Vulkan swapchains ready`;
|
||||||
|
5. `display refresh rate 120 Hz requested` (or `refused`, with the rates on offer);
|
||||||
|
6. the session reaching `FOCUSED`;
|
||||||
|
7. `OpenXR interaction profiles: left /interaction_profiles/valve/frame_controller_valve, right ...`;
|
||||||
|
8. `OpenXR eye gaze: available`, then `tracking`;
|
||||||
|
9. `presentation=virtual-screen` for the menus, and `first immersive packet consumed` on race entry.
|
||||||
|
|
||||||
|
Then, in the game:
|
||||||
|
|
||||||
|
- the D-pad does tricks, the left shoulder opens the settings panel, and View pauses;
|
||||||
|
- `debug.wiicompiled.fpslog 1` on a race start shows the pacing summary and the GPU time per pass
|
||||||
|
with foveation off, fixed and following the gaze;
|
||||||
|
- `render_scale` starts at 0.8, the Quest's value. The runtime's recommended eye size (logged at
|
||||||
|
startup) and those measurements decide whether the Frame keeps it.
|
||||||
|
|
||||||
|
A black headset with a working mirror points at the AHardwareBuffer copy, as on the Quest
|
||||||
|
(`docs/quest-port.md`, Validation status).
|
||||||
|
|
||||||
|
Open questions only the device can answer:
|
||||||
|
|
||||||
|
- Whether Lepton's driver supports the AHardwareBuffer and sync fd bridge.
|
||||||
|
- Whether Lepton shows the 2D setup panel while the app runs in VR mode. If it does not, the game
|
||||||
|
still starts directly once a `.wcgame` with the game files has been imported, but importing needs
|
||||||
|
the panel.
|
||||||
|
- Whether SteamVR grants 120 Hz.
|
||||||
|
- Whether the gaze needs an Android permission under Lepton.
|
||||||
|
- Whether "Build on this headset" can run its toolchain through `/system/bin/linker64` inside
|
||||||
|
Lepton. A game built on the PC does not depend on it.
|
||||||
|
|
||||||
|
## A native SteamOS build
|
||||||
|
|
||||||
|
Valve recommends native Linux ARM64 builds for the Frame, and one would avoid Lepton and the
|
||||||
|
two-device copy. The pieces:
|
||||||
|
|
||||||
|
- **Backend.** The Windows Vulkan backend (`openxr_vulkan_win32.cpp`), where OpenXR creates Dawn's
|
||||||
|
own device and eyes are copied on Dawn's queue, ports almost as it is. Only its `_WIN32` guards are
|
||||||
|
platform-specific.
|
||||||
|
- **Interop.** Aurora's `vulkan_win32_interop.cpp` finds the patched Dawn's exports with
|
||||||
|
`GetModuleHandleW`. A static Linux Dawn would reference them directly, and `aurora_core.cmake`
|
||||||
|
compiles that file on Windows only.
|
||||||
|
- **Dawn.** A linux-aarch64 Dawn built with `aurora-main/patches/dawn` (the hook ABI and density
|
||||||
|
maps), as `android/Build-QuestDawn.ps1` already does for Android.
|
||||||
|
- **`openxr_integration.cpp`.** It needs a Linux branch asking for `XR_KHR_vulkan_enable2` and
|
||||||
|
`XR_KHR_convert_timespec_time`. Today its `#else` is Android's and requires
|
||||||
|
`XR_KHR_android_create_instance`.
|
||||||
|
- **Controller timing.** `openxr_input.cpp` needs a `__linux__` branch for the input clock.
|
||||||
|
- **CPU target.** An `MKW_LINUX_CPU` knob in place of `-mcpu=native`, for cross-builds.
|
||||||
|
- **Android-gated fixes.** The Adreno vertex padding, `headset_owns_display` and the
|
||||||
|
`last_pass_feeding_replay` saving are gated on `__ANDROID__`. They would follow the GPU or the
|
||||||
|
headset instead.
|
||||||
|
- **Packaging.** SteamOS packaging, and a way to build the player's game for it.
|
||||||
Reference in new issue
Block a user