From 0a3bcc99bd9fca17885e3f0bcf9c7081ed8a10fc Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 5 Oct 2026 06:22:35 +0000 Subject: [PATCH] Fold the Steam Frame guide into the README, with a quick start docs/steam-frame.md is gone. The README now opens with a quick start that goes from the disc to playing (emulation setup, disc extraction, the container build, installing with Frame Control, updating), and carries what the guide held that a player or contributor needs: recommended settings, the Frame controller map, known issues, troubleshooting (log lines, the pacing line, crash backtraces), building on the Frame or with Docker (including Unraid's qemu registration), how the native build works and why the Android flavour cannot show a picture. Links to the old guide point at the README. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_019HBRGKTE1GnN2ah8gcZKr3 --- Launcher/build-dawn-linux.sh | 2 +- Launcher/local-build.sh | 2 +- OPENXR.md | 10 +- README.md | 339 ++++++++++++++++------ android/README.md | 2 +- docs/quest-port.md | 4 +- docs/steam-frame.md | 548 ----------------------------------- 7 files changed, 263 insertions(+), 644 deletions(-) delete mode 100644 docs/steam-frame.md diff --git a/Launcher/build-dawn-linux.sh b/Launcher/build-dawn-linux.sh index 4885c96..b19eb0a 100755 --- a/Launcher/build-dawn-linux.sh +++ b/Launcher/build-dawn-linux.sh @@ -3,7 +3,7 @@ # Vulkan hooks through which the OpenXR runtime creates Dawn's own instance and device (the # same-device OpenXR backend, runtime/src/vr/openxr_vulkan_win32.cpp) and fragment density maps for # foveated rendering. The stock prebuilt package has neither. This is the Linux counterpart of -# android/Build-QuestDawn.ps1, for the Steam Frame's native SteamOS build (docs/steam-frame.md). +# android/Build-QuestDawn.ps1, for the Steam Frame's native SteamOS build (README.md, Quick start). # # Launcher/build-dawn-linux.sh [--work-dir DIR] [--cc PATH --cxx PATH] [--cmake PATH] # [--ninja PATH] [--python PATH] [--jobs N] [--force] diff --git a/Launcher/local-build.sh b/Launcher/local-build.sh index 042f65d..d08758f 100755 --- a/Launcher/local-build.sh +++ b/Launcher/local-build.sh @@ -99,7 +99,7 @@ Usage: local-build.sh --output-dir DIR [options] --dawn-package DIR A Dawn built with Aurora's patches (Launcher/build-dawn-linux.sh's WORK_DIR/package); build it with the same --cc/--cxx --headset NAME steam_frame: the Steam Frame's native SteamOS build and its defaults - (docs/steam-frame.md); empty for any other PC headset + (README.md); empty for any other PC headset --cpu NAME AArch64 -mcpu target (default: cortex-x4 with --headset steam_frame, else native) EOF diff --git a/OPENXR.md b/OPENXR.md index 53d4bb0..48d447c 100644 --- a/OPENXR.md +++ b/OPENXR.md @@ -287,7 +287,7 @@ The Steam Frame's controllers get their own profile where the runtime offers it (`XR_VALVE_frame_controller_interaction`, `/interaction_profiles/valve/frame_controller_valve`): right A, B, trigger and stick as above, left View as the left menu (+), the left shoulder as left Y (the settings panel), and the left D-pad as the Wii Remote's D-pad (the gamepad's D-pad in -`"gamepad"` mode). The table is in `docs/steam-frame.md`. +`"gamepad"` mode). The table is in the README, Controls. **Motion.** Each XR frame the aim and grip poses are located at the measured current time (`XR_KHR_win32_convert_performance_counter_time`, `XR_KHR_convert_timespec_time` on Android), not @@ -1097,8 +1097,8 @@ turned into tangents of each eye's view (`vr/eye_gaze.h`), which `AuroraStereoFr and error of tracking, on the gaze snapped to a cell of two map texels (about 3 degrees), keeping up to 128 maps per eye, one per cell looked at, and binds a new one once its upload completes, the previous map staying bound meanwhile. Without a tracked gaze (a blink, no -tracker, the setting off) the map is the forward one above, unchanged. Details and the Steam Frame -checks are in `docs/steam-frame.md`. +tracker, the setting off) the map is the forward one above, unchanged. The README's How it works +covers the Steam Frame. ## Diagnostics @@ -1282,8 +1282,8 @@ custom DLL's ABI through three borrowed-image copy/readback cycles; run it with | Windows D3D12 | Implemented: same-adapter, same-device asynchronous OpenXR submission. | | Windows Vulkan | Implemented, opt-in (`video.graphics_api = "vulkan"`): the runtime creates Dawn's Vulkan instance and device through `XR_KHR_vulkan_enable2`, eyes are copied on the same queue, and Dawn's device guard is held around the four queue-touching OpenXR calls. Needs the custom Dawn from `Launcher/Build-DawnVulkan.ps1`. Raced on SteamVR/PSVR2 at the headset's full rate; other runtimes unexercised. See [Windows Vulkan](#windows-vulkan). | | Android Vulkan (Meta Quest) | Implemented and running on a Quest 3: the OpenXR side owns its own Vulkan device (`XR_KHR_vulkan_enable2`, `XR_KHR_vulkan_enable` fallback) and shares eyes with Dawn through `AHardwareBuffer`s ordered by sync-fd fences. Controllers arrive through OpenXR actions as a virtual SDL gamepad. See `docs/quest-port.md`. | -| Android Vulkan (Steam Frame) | The same backend in the `steamFrame` flavour, for SteamVR's Android runtime under Lepton: Frame controller profile, 120 Hz request, eye-tracked foveation. Blocked on the headset: Lepton's Turnip exposes no AHardwareBuffer or fd external memory, so the eyes cannot cross devices. The native build below is the supported route. See `docs/steam-frame.md`. | -| Linux Vulkan (Steam Frame, SteamOS) | Implemented, not yet run on the headset: the Windows Vulkan design compiled for Linux, so the runtime creates Dawn's own instance and device and the eyes are copied on Dawn's queue with no sharing. Adds the Frame controller profile, 120 Hz and gaze-centred foveation. Needs a Dawn built with Aurora's patches (`Launcher/build-dawn-linux.sh`), then `Launcher/local-build.sh --openxr --dawn-package --headset steam_frame`. See `docs/steam-frame.md`. | +| Android Vulkan (Steam Frame) | The same backend in the `steamFrame` flavour, for SteamVR's Android runtime under Lepton: Frame controller profile, 120 Hz request, eye-tracked foveation. Blocked on the headset: Lepton's Turnip exposes no AHardwareBuffer or fd external memory, so the eyes cannot cross devices. The native build below is the supported route. See the README, The Android flavour. | +| Linux Vulkan (Steam Frame, SteamOS) | Implemented and played on a Steam Frame (beta): the Windows Vulkan design compiled for Linux, so the runtime creates Dawn's own instance and device and the eyes are copied on Dawn's queue with no sharing. Adds the Frame controller profile, 120 Hz and gaze-centred foveation. Needs a Dawn built with Aurora's patches (`Launcher/build-dawn-linux.sh`), then `Launcher/local-build.sh --openxr --dawn-package --headset steam_frame`. See the README, Quick start. | | Other platforms | Not wired yet. | Both bindings share `openxr_integration.cpp`: the pacing thread, policy evaluation, the diff --git a/README.md b/README.md index 1f4be52..83df59e 100644 --- a/README.md +++ b/README.md @@ -29,122 +29,289 @@ C++ and compiled for the Frame's ARM64 CPU, and it renders through SteamVR's Ope --- -## What this fork adds +## Quick start -Everything below is in [`docs/steam-frame.md`](docs/steam-frame.md), with the reasoning and the -readings it is based on. +This is the whole way from your disc to playing in the headset, building on an x86_64 Linux PC. The +first build takes a few hours, most of it compiling Dawn under emulation; later builds reuse it. +[Other ways to build](#other-ways-to-build) covers a faster machine and the Frame itself. -- **A native SteamOS build.** The game links SteamVR's OpenXR runtime directly and lets it create - the GPU device it renders with, so each eye is copied straight into the headset's swapchain with - no second device and no sharing between them. It runs fullscreen in the headset only; there is - no desktop window to draw. -- **Built for the Frame's Snapdragon 8 Gen 3**: compiled with `-mcpu=cortex-x4`. -- **The Frame's controllers.** Its own interaction profile is bound, so beside what the Quest Touch - layout already does, the left D-pad is the Wii Remote's D-pad (tricks, menus), the left View - button pauses and the left shoulder opens the settings panel. -- **120 Hz.** The headset is asked for 120 Hz, exactly two display refreshes per game frame at the - game's 60 FPS, so motion is even. `[vr] refresh_rate` changes it. -- **Every refresh from the game.** SteamVR halved the app's rate and filled every other refresh - itself. The game now submits its last frame again, at the pose it was rendered for, on each - refresh it has no new frame for (`[vr] repeat_frames`), so SteamVR runs it at the full 120 Hz. -- **Eye-tracked foveation.** Variable-resolution rendering whose sharp centre follows your gaze - through the Frame's eye tracking, instead of staying fixed straight ahead. `[vr] foveation` sets - the strength and `[vr] eye_tracked_foveation` turns the tracking off. -- **Standalone defaults**: the game's own object culling and medium foveation, as on the Quest. - The render scale defaults to 0.8; on the Frame 1.0 is SteamVR's recommended 1728x1728 and **1.25 - is the panels' native 2160x2160**, which the Frame renders with time to spare. -- **Crash diagnostics on Linux.** A native crash logs the faulting thread, its pc and a backtrace, - which is how a crash on race restart was traced to the density maps sharing memory Dawn unmaps. -- **Build tooling.** `Launcher/build-dawn-linux.sh` builds the patched Dawn (Aurora's WebGPU - layer) the VR backend needs, and `Launcher/local-build.sh` gained `--openxr`, `--dawn-package`, - `--headset steam_frame` and `--cpu`. +**You need:** +- a Steam Frame with Developer Mode on (Steam Settings → System → Enable Developer Mode, then set a + user password), on the same network as your PC; +- your own clean PAL `RMCP01` disc image of Mario Kart Wii (ISO, WBFS, RVZ, ...); +- an x86_64 Linux PC with about 20 GB free and 16 GB of memory or more. -The Steam Frame also runs Android apps through its Lepton layer, and the Quest app gained a -`steamFrame` flavour for it, but it cannot show a picture there: Lepton's graphics driver lacks -the memory-sharing extensions the Android backend needs. The native build is the way to play. - -## Requirements - -- A Steam Frame with Developer Mode on and SSH set up (Steam Settings → System → Enable Developer - Mode, then set a user password). [Frame Control](https://github.com/saphid/frame-control) makes the - rest easier: it sets up an SSH key and installs the game into your Steam library. -- A clean, unmodified **PAL `RMCP01`** disc image of Mario Kart Wii, dumped by you. ISO, GCM, - GCZ, CISO, WBFS, WIA and RVZ can all be extracted. Other regions and patched executables are - rejected. -- Somewhere to build, all of them running an ARM64 Debian container: - - **an x86_64 Linux PC** with podman and qemu (emulated, so slow: the first build takes hours); - - **a stronger Linux machine or server** with Docker, the same way and faster; - - **the Frame itself**, in a podman container there. -- About 20 GB of free disk space on the build machine for the toolchain, Dawn and the game. +The PC commands below work in bash, zsh and fish. > [!NOTE] > Nobody here will tell you where to get the game. Dumping your own disc is on you, and links to > game files won't be provided or tolerated. For the same reason there is no ready-built game to > download: releases hold the source, and the game is always built from your own disc. -## Building and installing +### 1. Set up ARM64 emulation on the PC -The commands are in [`docs/steam-frame.md`](docs/steam-frame.md): [on a Linux -PC](docs/steam-frame.md#building-it-on-a-linux-pc), [with Docker on another -machine](docs/steam-frame.md#building-it-with-docker-on-another-machine) or [on the -Frame](docs/steam-frame.md#building-it-on-the-frame). In short: +```bash +sudo pacman -S --needed podman qemu-user-static qemu-user-static-binfmt # Arch, CachyOS +sudo systemctl restart systemd-binfmt +podman run --rm --platform linux/arm64 docker.io/library/debian:trixie uname -m +``` -1. Extract your disc with [nodtool](https://github.com/encounter/nod) and copy `sys/main.dol` and - `files/rel/StaticR.rel` into `Assets/`. Keep the extracted disc: the game reads it at run time. -2. Start a Debian trixie ARM64 container and install the build packages, the bundled clang 22, - CMake and Ninja (`Launcher/prepare-portable-tools.sh --arch aarch64`), and .NET 8. -3. Build the patched Dawn once: `Launcher/build-dawn-linux.sh`. Under emulation give it `--jobs 4` - on a 16 GB machine; more parallel jobs can run it out of memory. -4. Build the game: `Launcher/local-build.sh ... --openxr --dawn-package /package --headset - steam_frame`, with `--parallel` sized to the machine's memory (8 for 32 GB). -5. Copy the extracted disc to the Frame, and either send the `out` folder to the Frame with - [Frame Control](docs/steam-frame.md#installing-it-with-frame-control), which adds it to your - Steam library, or copy it over with `scp`. -6. Write `~/.local/share/WiiCompiled/Config.toml` on the Frame with your disc's path, then start - the game from the library in the headset: +The last command must print `aarch64`. On Debian or Ubuntu install `podman qemu-user-static +binfmt-support` instead. If podman complains about subordinate ids, run +`sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 $USER` and log in again. - ```toml - [paths] - dvd_root = "/home/steamos/wiicompiled/disc" - ``` +### 2. Get the code and extract your disc -### Recommended settings +```bash +mkdir -p ~/wiicompiled; cd ~/wiicompiled +git clone https://github.com/mitch030504/Wiicompiled_VR_Frame.git +curl -fL -o nodtool https://github.com/encounter/nod/releases/download/v2.0.0-alpha.10/nodtool-linux-x86_64 +chmod +x nodtool +./nodtool extract "/path/to/Mario Kart Wii.wbfs" disc-extract +mkdir -p Wiicompiled_VR_Frame/Assets +cp disc-extract/sys/main.dol disc-extract/files/rel/StaticR.rel Wiicompiled_VR_Frame/Assets/ +``` -All of them are in the headset's settings panel (left shoulder button, **VR** tab) as well as in -`Config.toml`: +`disc-extract` must hold `sys/` and `files/` directly; keep it, the game reads it when it runs. + +### 3. Build, inside an ARM64 container + +```bash +podman run -it --name wiicompiled-frame --platform linux/arm64 -v ~/wiicompiled:/work docker.io/library/debian:trixie bash +``` + +Then, in the container's bash prompt: + +```bash +apt-get update && apt-get install -y --no-install-recommends \ + ca-certificates curl git python3 xz-utils unzip file pkg-config g++ binutils libicu-dev zlib1g-dev \ + libvulkan-dev libx11-dev libx11-xcb-dev libxcb1-dev libxext-dev libxrandr-dev libxinerama-dev \ + libxcursor-dev libxi-dev libxss-dev libxtst-dev libxkbcommon-dev libwayland-dev wayland-protocols \ + libdecor-0-dev libegl-dev libgl-dev libgles-dev libdrm-dev libgbm-dev libasound2-dev libpulse-dev \ + libpipewire-0.3-dev libudev-dev libdbus-1-dev libusb-1.0-0-dev +cd /work/Wiicompiled_VR_Frame +Launcher/prepare-portable-tools.sh --arch aarch64 --destination /work/tools # clang 22, CMake, Ninja +T=/work/tools/toolchain-aarch64/bin +curl -fsSL https://dot.net/v1/dotnet-install.sh | bash -s -- --channel 8.0 --install-dir /work/dotnet +Launcher/build-dawn-linux.sh --work-dir /work/dawn --cc $T/clang --cxx $T/clang++ \ + --cmake $T/cmake --ninja $T/ninja --jobs 4 2>&1 | tee -a /work/dawn.log +Launcher/local-build.sh --output-dir /work/out --cc $T/clang --cxx $T/clang++ --fuse-ld lld \ + --cmake $T/cmake --ninja $T/ninja --dotnet /work/dotnet/dotnet --parallel 4 \ + --openxr --dawn-package /work/dawn/package --headset steam_frame 2>&1 | tee -a /work/game.log +exit +``` + +- Dawn is done when it prints `Patched Dawn for Linux ready`, the game when it prints + `MKWCBUILD:OUTPUT=/work/out`. +- `--jobs 4` and `--parallel 4` suit 16 GB of memory. Compiles take more memory under emulation, and + 16 at once froze a 16 GB laptop. Use 8 with 32 GB. +- If it stops, `podman start -ai wiicompiled-frame` gets you back in. Run `cd /work/Wiicompiled_VR_Frame; + T=/work/tools/toolchain-aarch64/bin`, then the step that stopped: both scripts resume. + +### 4. Install on the Frame + +The easiest way is [Frame Control](https://github.com/saphid/frame-control): set up its connection +to the Frame, then drag `~/wiicompiled/out` onto **Send to Frame**. Name it `WiiCompiled` and keep +**Launches** on `WiiCompiled`. It adds the game to your Steam library, in `~/devkit-game/WiiCompiled/` +on the Frame. + +Then copy the disc over and, on a first install, tell the game where it is (Frame Control's SSH +key answers to `frame`; use `steamos@` otherwise): + +```bash +ssh frame mkdir -p wiicompiled +scp -r ~/wiicompiled/disc-extract frame:wiicompiled/disc +ssh frame 'mkdir -p ~/.local/share/WiiCompiled && printf "[paths]\ndvd_root = \"/home/steamos/wiicompiled/disc\"\n" > ~/.local/share/WiiCompiled/Config.toml' +``` + +Without Frame Control, `scp -r ~/wiicompiled/out frame:wiicompiled/` and start +`~/wiicompiled/out/WiiCompiled` from a terminal in the Frame's Desktop Mode, with SteamVR running. + +### 5. Play + +Start **WiiCompiled** from the library in the headset. Open the settings panel with the left +shoulder button, go to the **VR** tab and set the [recommended settings](#recommended-settings). + +### Updating + +Pull the new code, rebuild, and replace only the executable. Copying to a new name and moving it +into place works even while an old copy is open: + +```bash +cd ~/wiicompiled/Wiicompiled_VR_Frame && git pull +podman start -ai wiicompiled-frame +# in the container: cd /work/Wiicompiled_VR_Frame; T=/work/tools/toolchain-aarch64/bin, then the +# build-dawn-linux.sh and local-build.sh lines from step 3 (only what changed is rebuilt), then exit +scp ~/wiicompiled/out/WiiCompiled frame:devkit-game/WiiCompiled/WiiCompiled.new +ssh frame 'cd ~/devkit-game/WiiCompiled && chmod 755 WiiCompiled.new && mv -f WiiCompiled.new WiiCompiled' +``` + +## Recommended settings + +All of these are in the headset's settings panel (left shoulder button, **VR** tab) and in +`~/.local/share/WiiCompiled/Config.toml`. Quit the game before editing the file; it writes its +settings back when it closes. | Setting | Value | Why | | --- | --- | --- | -| `[vr] render_scale` | `1.25` | The panels' native 2160x2160 per eye. | -| `[vr] foveation` | `medium` | `off` costs the most GPU time. | -| `[vr] repeat_frames` | `true` (default) | Keeps SteamVR at 120 Hz. | -| `[vr] frame_interpolation_fps` | `0` | Interpolation made things worse on the Frame. | -| `[video] resolution_multiplier` | `2` | The game's own frame; 4x is too heavy for the Frame's GPU. | +| `[vr] render_scale` | `1.25` | Scales SteamVR's recommended eye size, 1728x1728 on the Frame: 1.25 is the panels' native 2160x2160, which the Frame renders in 9 to 11 ms a frame with medium foveation. | +| `[vr] foveation` | `medium` | `off` shades every pixel and costs the most. See [Known issues](#known-issues) if images double. | +| `[vr] repeat_frames` | `true` (default) | Without it SteamVR halves the game's rate and fills refreshes itself. | +| `[vr] frame_interpolation_fps` | `0` | Rendering in-between frames needs 120 eye pairs a second, which made things worse on the Frame. | +| `[video] resolution_multiplier` | `2` | The game's own frame, which the eyes are made from. 4x is far too heavy for the Frame's GPU. | -Keep SteamVR's own refresh rate at 120 Hz. Motion Smoothing makes no difference here. +Keep SteamVR's refresh rate at 120 Hz. Motion Smoothing makes no difference to this game. + +## Controls + +The Frame's controllers are bound through their own profile, so the left D-pad works: + +| Frame controller | Wii Remote mode | Gamepad mode | +| --- | --- | --- | +| Right A | A | South (A) | +| Right B | C (look behind) | East (B) | +| Right trigger | B | Right trigger | +| Right stick up / down | 1 / 2 | Right stick | +| Left View | + (pause) | Start | +| Left shoulder | Settings panel | North (Y) | +| Left D-pad | Wii Remote D-pad | D-pad | +| Left stick, left trigger | Nunchuk stick, Z | Left stick, left trigger | +| Grips, stick clicks, motion, aim | as on Quest Touch ([`OPENXR.md`](OPENXR.md), Controllers) | as on Touch | +| Right X, Y, menu and shoulder | unbound | unbound | ## Known issues - **Doubled images** in races and on the HUD, worst while racing, sometimes in the right eye only. - The suspect is foveation on the Frame's graphics driver. If it bothers you, try foveation **Off** - in the panel (it costs GPU time), and report whether it helped. -- **Foveation follows the right eye less well** than the left, and its sharp area may feel small - on Low and Medium. + Both eyes get the same frames, so a doubling in one eye points at foveation: the Frame's driver + (Turnip) draws a foveated screen tile at lower resolution and scales it back up, and each eye's + density maps change with its gaze. Try foveation **Off** if it bothers you, and report whether it + helped. +- **Foveation follows the right eye less well** than the left. Convergence on near content (the HUD + screen at 2 m, the cockpit) is not yet corrected for. - **VR frame interpolation** is not recommended on the Frame. -- **The Quest app's `steamFrame` flavour** (an Android build for the Frame's Lepton layer) cannot - show a picture: Lepton's graphics driver lacks the memory sharing it needs. Use the native build. +- **The Quest app's `steamFrame` flavour** cannot show a picture in the Frame's Android layer + ([below](#the-android-flavour)). Use the native build. + +## Troubleshooting + +**Logs.** Each run gets a folder under `~/.local/share/WiiCompiled/Logs/` on the Frame, holding +`console.log` and, after a crash, `crash_sigsegv.txt`. A working start logs, in order: + +1. `OpenXR initialized: runtime 'SteamVR/OpenXR'`; +2. `Dawn will create its device through the runtime`; +3. `Fragment density maps: enabled`; +4. `OpenXR Vulkan swapchains ready ... same-queue native eye copies`; +5. `display refresh rate 120 Hz requested`, then `OpenXR session state -> FOCUSED`; +6. `OpenXR interaction profiles: left /interaction_profiles/valve/frame_controller_valve`; +7. `OpenXR eye gaze: available`, then `tracking`. + +The first one missing says where it stopped. `Linux Vulkan OpenXR requires a Dawn built with +Aurora's patches` means the game was built without `--dawn-package`. + +**Smoothness.** With `[diagnostics] openxr_logging = true`, the log gets a pacing line every second: + +```bash +ssh frame 'cd ~/.local/share/WiiCompiled/Logs && d=$(ls -t | head -1) && grep -h "xr-diag\] 1.0" "$d/console.log" | tail -5' +``` + +A healthy race reads `predicted-rate=120.0Hz`, `new=60 repeat=60` and `late=0`. Fewer than 60 `new` +frames means the GPU is over budget: lower `render_scale` or `resolution_multiplier`. + +**Crashes.** `console.log` then names the faulting thread and gives its pc and a backtrace as +`module+offset`. On the build machine, +`addr2line -f -C -e Wiicompiled_VR_Frame/native-build/WiiCompiled 0x` turns an offset in +`WiiCompiled` into a function: the executable is not stripped. + +**Building.** A build that freezes the machine has run out of memory: lower `--jobs` or +`--parallel`. A missing CMake package means installing its `-dev` package in the container and +running the same command again. + +## Other ways to build + +**On the Frame itself.** SteamOS's root file system is read-only, but it ships podman. Over SSH: + +```bash +mkdir -p ~/wiicompiled && cd ~/wiicompiled +git clone https://github.com/mitch030504/Wiicompiled_VR_Frame.git +podman run -it --name wiicompiled-frame -v ~/wiicompiled:/work:Z docker.io/library/debian:trixie bash +``` + +It is native ARM64, so no emulation, but the Frame has less memory and cooling than a PC. Extract +your disc as in step 2, with `nodtool-linux-aarch64` instead of `nodtool-linux-x86_64`, copy +`main.dol` and `StaticR.rel` into `Assets/`, and run step 3's commands in the container. The game +lands in `~/wiicompiled/out`. + +**With Docker on a stronger machine** (a server, for example). The same container and commands +work, and much faster with more cores and memory. Copy `~/wiicompiled` over without Dawn's build +tree, and run the build detached so a closed SSH session does not stop it: + +```bash +rsync -a --exclude dawn/build --exclude 'dawn/dawn-*' --exclude disc-extract \ + ~/wiicompiled/ root@:/srv/wiicompiled/ # from the PC +docker run -d --name wiicompiled-frame --platform linux/arm64 -v /srv/wiicompiled:/work \ + debian:trixie bash -c 'bash /work/build-game.sh >> /work/game.log 2>&1' +``` + +`build-game.sh` holds step 3's `apt-get` line and `local-build.sh` command, with `--parallel 8` for +32 GB. `docker start wiicompiled-frame` runs it again after an update. + +On some hosts, for example Unraid 7 with kernel 6.18, `binfmt_misc` registrations are per container. +`tonistiigi/binfmt --install arm64` then reports success, but Debian answers `exec format error`. +Register qemu on the host instead, with the `P` flag the tonistiigi build of qemu expects. Without it, +every program loses its first argument, and `uname -m` prints `Linux`: + +```bash +docker create --name qemu-src tonistiigi/binfmt +docker cp qemu-src:/usr/bin/qemu-aarch64 /usr/local/bin/qemu-aarch64 +docker rm qemu-src +echo ':qemu-aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/local/bin/qemu-aarch64:POCF' > /proc/sys/fs/binfmt_misc/register +docker run --rm --platform linux/arm64 debian:trixie uname -m # aarch64 +``` + +Unraid keeps `/usr/local/bin` in memory, so repeat this after a reboot. + +## How it works + +- **Same-device rendering.** SteamVR's OpenXR runtime creates the Vulkan instance and device that + Dawn (Aurora's WebGPU layer) renders with, through Aurora's patches to Dawn + (`aurora-main/patches/dawn`). Each eye is copied into SteamVR's swapchain on Dawn's own queue, so + nothing is shared between GPU devices. This is the PC's Vulkan backend + (`runtime/src/vr/openxr_vulkan_win32.cpp`), compiled for Linux. Dawn links statically, so the + patched Dawn is built once (`Launcher/build-dawn-linux.sh`). Its `aurora-dawn.json` declares the + Vulkan hook and density map ABIs that Aurora compiles against. +- **Headset only.** The game neither shows nor finishes rendering a desktop window, and is compiled + for the Frame's Cortex-X4 cores (`-mcpu=cortex-x4`, overridable with `--cpu`). +- **Every refresh from the game.** The game draws 60 frames a second. When it handed SteamVR only + those, SteamVR ran it at half rate and made up every other refresh itself, even with Motion + Smoothing off. With `[vr] repeat_frames`, the pacing thread waits for the next frame until 1.5 ms + before SteamVR's next wake, then resubmits the last frame at the pose it was rendered for. + SteamVR turns that to the current head pose. +- **Eye-tracked foveation.** `XR_EXT_eye_gaze_interaction` gives the gaze, which + `runtime/include/vr/eye_gaze.h` turns into each eye's view. Aurora picks a fragment density map + centred on the gaze, snapped to cells of about 3 degrees (`aurora-main/lib/gfx/foveation.hpp`), + and keeps up to 128 per eye. Each map has a memory block of its own: Turnip reads a map through a + host mapping, and Dawn's buffer uploads unmap the shared blocks they sub-allocate from. That + crashed the game when a race restarted, until each map got its own block. +- **120 Hz.** `[vr] refresh_rate` (default 120 on the Frame, `0` keeps the headset's own) is asked of + SteamVR through `XR_FB_display_refresh_rate`. SteamVR only offers the rate set in its own settings. + +### The Android flavour + +The Quest app also has a `steamFrame` flavour (`android/Build-Quest.ps1 -Headset frame`), for the +Frame's Android layer, Lepton. It builds and starts in VR there, but cannot show a picture. The +Android backend hands each eye from Dawn's device to its own OpenXR device through +`VK_ANDROID_external_memory_android_hardware_buffer` and sync fds, and Lepton's Turnip has neither, +nor `VK_KHR_external_memory_fd`. The native build needs no sharing, which is why it is the one to +play. [`docs/quest-port.md`](docs/quest-port.md) covers the Android build. ## Reporting problems Open an issue on this repository with: - what you did and what you saw (which eye, where in the picture, racing or menus); -- the run log: each run has a folder under `~/.local/share/WiiCompiled/Logs/` on the Frame, with - `console.log` and, after a crash, `crash_sigsegv.txt`; +- the run's `console.log` (and `crash_sigsegv.txt` after a crash) from + `~/.local/share/WiiCompiled/Logs/` on the Frame; - or, for a build problem, the last lines of the failing step. -[Running it](docs/steam-frame.md#running-it) lists the log lines a working start shows, in order; -the first one missing says where it stopped. - Problems that also happen on a PC or a Quest belong upstream, in [WiiCompiled OpenXR VR](https://github.com/iChris4/Wiicompiled_VR). diff --git a/android/README.md b/android/README.md index fc85b0e..8634120 100644 --- a/android/README.md +++ b/android/README.md @@ -2,7 +2,7 @@ Standalone Android/OpenXR build of the Mario Kart Wii recompilation for Quest 1, Quest 2, Quest 3, Quest 3S and Quest Pro, and, as the `steamFrame` flavour, for Valve's Steam -Frame under Lepton ([docs/steam-frame.md](../docs/steam-frame.md)). The full design, build +Frame under Lepton (the [README](../README.md#the-android-flavour); the Frame is played with the native SteamOS build). The full design, build walkthrough and current status live in [docs/quest-port.md](../docs/quest-port.md); this directory only holds the Gradle project, its helper scripts, the game kit tooling (`QuestGameKit.psm1`, `Build-QuestGame.ps1`), the on-headset build toolchain diff --git a/docs/quest-port.md b/docs/quest-port.md index 074fb78..129cea8 100644 --- a/docs/quest-port.md +++ b/docs/quest-port.md @@ -823,7 +823,7 @@ Android facts this design rests on, all measured on a Quest 3: application ID and storage, but select the appropriate CPU baseline (one `headsetCpus` map in `app/build.gradle.kts`), manifest and launcher behavior. `steamFrame` is Valve's Steam Frame under Lepton, SteamOS's - Android layer; see `docs/steam-frame.md`. + Android layer, where it cannot show a picture; see the README, The Android flavour. `app/src/main/cpp/CMakeLists.txt` adds the repository's `runtime/` as a subdirectory with those Android choices and builds both game kit probes (the Retro Rewind one only when the translation includes the mod), which @@ -852,7 +852,7 @@ powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Headset quest1 powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Install # your game, against that kit, into Import (or WheelWizard VR's Build for Quest) powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Product retro_rewind -Mod -Install # the mod and its pack (needs translate-mod output with --retro-wfc-payload) powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Headset quest1 -Install # game package from the Quest 1 kit -powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Headset frame # Steam Frame flavour (docs/steam-frame.md) +powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Headset frame # Steam Frame flavour (README, The Android flavour) adb push MarioKart.iso /sdcard/Download/ # then Select disc image in the launcher ``` diff --git a/docs/steam-frame.md b/docs/steam-frame.md deleted file mode 100644 index 86e710f..0000000 --- a/docs/steam-frame.md +++ /dev/null @@ -1,548 +0,0 @@ -# WiiCompiled VR on the Steam Frame - -Valve's Steam Frame runs SteamOS on a Snapdragon 8 Gen 3 (Cortex-X4, A720 and A520 cores, Adreno 750), -with 2160x2160 panels per eye at 72 to 144 Hz, eye tracking, and SteamVR as its OpenXR runtime. A game -can run on it two ways, and this project has both: - -- **Natively on SteamOS** (Linux ARM64), with SteamVR's own OpenXR runtime. This is the Frame's build: - [The native SteamOS build](#the-native-steamos-build) says how to make and run it. -- **As an Android app in Lepton**, SteamOS's Android layer, as a third flavour of the Quest app - (`steamFrame`). It is built, but it cannot show a picture there: Lepton's Vulkan driver has no - external memory or sync fd extensions, and the Quest backend's two-device eye handoff needs them - (see [What the Frame reported](#what-the-frame-reported)). - -Most of what this document describes is shared by both: the Frame controller profile, the 120 Hz -request, eye-tracked foveation and the Frame's defaults. The native build gets them through -`MKW_HEADSET=steam_frame` (`MKW_HEADSET_STEAM_FRAME`), as the Android flavour does. - -**Status: beta, played on a Steam Frame.** The native build starts in VR under SteamVR, renders -both eyes on the device Dawn shares with the runtime, binds the Frame's controllers, runs at 120 Hz -with every game frame shown (`new=60 repeat=60` a second, no late frames, at `render_scale = 1.25`, -the panels' 2160x2160), and follows the eyes with its foveation. Two device runs found and fixed a -crash on race restart (see [Eye-tracked foveation](#eye-tracked-foveation)) and SteamVR halving the -app's rate (see [Refresh rate](#refresh-rate)). Still open, in [Known issues](#known-issues): doubled -images in races and on the HUD, sometimes in one eye only, and foveation that follows the right eye -less well than the left. - -## The native SteamOS build - -The backend is the PC's same-device Vulkan backend (`openxr_vulkan_win32.cpp`, on Linux too): the -OpenXR runtime creates Dawn's own Vulkan instance and device through Aurora's patches to Dawn, and -each eye is copied into SteamVR's swapchain on Dawn's queue, so nothing is shared between devices. -On Linux, Dawn links statically, so the patched Dawn is built once on the build machine -(`Launcher/build-dawn-linux.sh`); `aurora-dawn.json` in its package declares the Vulkan hook and -density map ABIs, and only against such a package does Aurora compile the bridge -(`AURORA_DAWN_VULKAN_HOOKS`) and the density maps (`AURORA_DAWN_FDM`). Against a stock Dawn the -build still links, and VR falls back to the desktop. - -What the Frame build changes, beyond the Android flavour's settings: - -- `-mcpu=cortex-x4` (`MKW_LINUX_CPU`, which `--cpu` overrides). -- `AuroraConfig::xrHeadsetOnly`: Aurora neither presents the desktop window nor renders it past the - last pass the eyes sample, as on Android (4 to 6 ms of a 12 ms GPU frame on a Quest 3). -- Fragment density maps are asked for on Linux as on the Quest; `AURORA_FDM=0` or `1` overrides the - settings, as `debug.wiicompiled.fdm` does there. -- Controller motion uses `XR_KHR_convert_timespec_time` when SteamVR offers it. - -### Building it on the Frame - -SteamOS's root file system is read-only, so the build runs in a Debian container on the Frame, -started with the `podman` SteamOS already ships. Over SSH (`ssh steamos@`): - -```bash -mkdir -p ~/wiicompiled && cd ~/wiicompiled -git clone https://github.com/mitch030504/Wiicompiled_VR_Frame.git -podman run -it --name wiicompiled-build -v ~/wiicompiled:/work:Z docker.io/library/debian:trixie bash -``` - -Inside the container (`podman start -ai wiicompiled-build` gets back into it later): - -```bash -apt-get update && apt-get install -y --no-install-recommends \ - ca-certificates curl git python3 xz-utils unzip file pkg-config g++ binutils libicu-dev zlib1g-dev \ - libvulkan-dev libx11-dev libx11-xcb-dev libxcb1-dev libxext-dev libxrandr-dev libxinerama-dev \ - libxcursor-dev libxi-dev libxss-dev libxtst-dev libxkbcommon-dev libwayland-dev wayland-protocols \ - libdecor-0-dev libegl-dev libgl-dev libgles-dev libdrm-dev libgbm-dev libasound2-dev libpulse-dev \ - libpipewire-0.3-dev libudev-dev libdbus-1-dev libusb-1.0-0-dev -cd /work/Wiicompiled_VR_Frame -Launcher/prepare-portable-tools.sh --arch aarch64 --destination /work/tools # clang 22, CMake, Ninja -T=/work/tools/toolchain-aarch64/bin -curl -fsSL https://dot.net/v1/dotnet-install.sh | bash -s -- --channel 8.0 --install-dir /work/dotnet -Launcher/build-dawn-linux.sh --work-dir /work/dawn --cc $T/clang --cxx $T/clang++ --cmake $T/cmake --ninja $T/ninja -``` - -Then the game. `local-build.sh` translates your own disc, so it needs `main.dol` and `StaticR.rel` from -your extracted PAL `RMCP01` disc in `Assets/` (the extracted disc's `sys/main.dol` and -`files/rel/StaticR.rel`; `translator/README.md` explains): - -```bash -mkdir -p Assets && cp /sys/main.dol /files/rel/StaticR.rel Assets/ -Launcher/local-build.sh --output-dir /work/out --cc $T/clang --cxx $T/clang++ --fuse-ld lld \ - --cmake $T/cmake --ninja $T/ninja --dotnet /work/dotnet/dotnet \ - --openxr --dawn-package /work/dawn/package --headset steam_frame -``` - -The game lands in `~/wiicompiled/out` on the Frame. Debian trixie's C library is not newer than -SteamOS's, so the binary runs on SteamOS outside the container. If CMake reports a missing package, -install its `-dev` package in the container and run the same command again; both scripts resume -where they stopped. - -### Building it on a Linux PC - -The same container runs on an x86_64 Linux PC as an emulated ARM64 one, which spares the Frame's -storage and battery; the result is copied over. Emulation makes it several times slower: the -first Dawn build takes hours. On the PC (these commands also work in fish): - -```bash -sudo pacman -S --needed podman qemu-user-static qemu-user-static-binfmt # Arch, CachyOS -sudo systemctl restart systemd-binfmt -podman run --rm --platform linux/arm64 docker.io/library/debian:trixie uname -m # prints aarch64 -``` - -Other distributions name the packages differently (on Debian and Ubuntu: `podman -qemu-user-static binfmt-support`). If rootless podman complains about subordinate ids, run -`sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 $USER` and log in again. - -Extract your disc image with [nodtool](https://github.com/encounter/nod), the extractor the -installer uses, and put the two files the build reads into `Assets/`: - -```bash -mkdir -p ~/wiicompiled; cd ~/wiicompiled -git clone https://github.com/mitch030504/Wiicompiled_VR_Frame.git -curl -fL -o nodtool https://github.com/encounter/nod/releases/download/v2.0.0-alpha.10/nodtool-linux-x86_64 -chmod +x nodtool -./nodtool extract "/path/to/Mario Kart Wii.wbfs" disc-extract -mkdir -p Wiicompiled_VR_Frame/Assets -cp disc-extract/sys/main.dol disc-extract/files/rel/StaticR.rel Wiicompiled_VR_Frame/Assets/ -podman run -it --name wiicompiled-frame --platform linux/arm64 -v ~/wiicompiled:/work docker.io/library/debian:trixie bash -``` - -Inside the container, run the commands of [Building it on the Frame](#building-it-on-the-frame) -from `apt-get` on, skipping the `cp` into `Assets/`, which is done. `podman start -ai -wiicompiled-frame` gets back into it; set `T` again before resuming. Then copy the game and the -extracted disc to the Frame: - -```bash -ssh steamos@ mkdir -p wiicompiled -scp -r ~/wiicompiled/out steamos@:wiicompiled/ -scp -r ~/wiicompiled/disc-extract steamos@:wiicompiled/disc -``` - -### Building it with Docker on another machine - -A faster x86_64 machine helps most with the game itself (the translated code is a few thousand large -files). Docker works as podman does, with one difference on hosts whose `binfmt_misc` registrations -are per container (Unraid 7 with kernel 6.18): `tonistiigi/binfmt --install arm64` then succeeds but -only inside its own container, and Debian answers `exec format error`. Register the emulator on the -host instead, with the `P` flag the tonistiigi build of qemu expects (without it every program loses -its first argument: `uname -m` prints `Linux`): - -```bash -docker create --name qemu-src tonistiigi/binfmt -docker cp qemu-src:/usr/bin/qemu-aarch64 /usr/local/bin/qemu-aarch64 -docker rm qemu-src -echo ':qemu-aarch64:M::\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/local/bin/qemu-aarch64:POCF' > /proc/sys/fs/binfmt_misc/register -docker run --rm --platform linux/arm64 debian:trixie uname -m # aarch64 -``` - -Unraid keeps `/usr/local/bin` in memory, so this is repeated after a reboot. Copy the work directory -over, without Dawn's build tree (the package is what the game links), and run the build detached so -that a closed SSH session does not stop it: - -```bash -rsync -a --exclude dawn/build --exclude 'dawn/dawn-*' --exclude disc-extract \ - ~/wiicompiled/ root@:/mnt/user/appdata/wiicompiled/ # from the PC -docker run -d --name wiicompiled-frame --platform linux/arm64 \ - -v /mnt/user/appdata/wiicompiled:/work debian:trixie \ - bash -c 'bash /work/build-game.sh >> /work/game.log 2>&1' -``` - -`build-game.sh` holds the `apt-get` line of [Building it on the Frame](#building-it-on-the-frame) -and the `local-build.sh` command, with `--parallel 8` for 32 GB of memory (compiles take more memory -under emulation: Dawn's build at 16 jobs froze a 16 GB laptop, and finished at 4). `docker start -wiicompiled-frame` runs it again after a change: the translation is reused and only what changed -is compiled. - -### Installing it with Frame Control - -[Frame Control](https://github.com/saphid/frame-control) adds a folder to the Frame's Steam library as -a Devkit Game: drop the `out` folder on **Send to Frame** and keep **Launches** on `WiiCompiled`. -It picks `SteamLinuxRuntime_4-arm64` for an ARM64 program, but Steam starts such a title natively, -which this build needs (it uses the system's `libpng16`, `libstdc++` and `libz`). The game then lands -in `~/devkit-game//` and starts from the library inside the headset, where SteamVR is already -running. An update only replaces the executable; copy it under another name and move it into place, -which also works while an old copy is open: - -```bash -scp WiiCompiled frame:devkit-game/WiiCompiled/WiiCompiled.new -ssh frame 'cd ~/devkit-game/WiiCompiled && chmod 755 WiiCompiled.new && mv -f WiiCompiled.new WiiCompiled' -``` - -### Running it - -The game reads its `Config.toml` from `~/.local/share/WiiCompiled/` on SteamOS (it is created on the first start): set -`[paths] dvd_root` there to your extracted disc (the directory holding `sys/` and `files/`; -`/home/steamos/wiicompiled/disc` when it was copied as above). Start -SteamVR on the Frame, then start `~/wiicompiled/out/WiiCompiled`, from Desktop Mode or as a -non-Steam game added to the library. The run log is in `Logs/` next to `Config.toml`; it should show, -in order: - -1. `OpenXR runtime offers N extensions: ...`, and `OpenXR initialized: runtime 'SteamVR/OpenXR'`; -2. `OpenXR Vulkan requirements: ... Dawn will create its device through the runtime`; -3. `Fragment density maps: enabled` (the patched Dawn and Turnip's density maps); -4. `OpenXR Vulkan swapchains ready ... same-queue native eye copies`; -5. `display refresh rate 120 Hz requested`, the session reaching `FOCUSED`, and - `OpenXR interaction profiles: left /interaction_profiles/valve/frame_controller_valve`; -6. `OpenXR eye gaze: available`, then `tracking`. - -`Linux Vulkan OpenXR requires a Dawn built with Aurora's patches` means the build used a stock Dawn: -check that `--dawn-package` pointed at `build-dawn-linux.sh`'s `package` directory. - -The game only writes a `Config.toml` when there is none, so a file holding just `[paths]` and -`dvd_root` can be written before the first start. Each run gets its own folder under `Logs/`; a -native crash leaves `crash_sigsegv.txt` there and, in `console.log`, the faulting thread, its pc and -lr and a backtrace as module + offset, which `addr2line -f -C -e native-build/WiiCompiled ` -turns into functions on the build machine (the executable is not stripped). - -### Settings that matter on the Frame - -| Setting | Recommended | Why | -| --- | --- | --- | -| `[vr] render_scale` | `1.25` | Scales SteamVR's recommended eye size, 1728x1728 on the Frame; 1.25 is the panels' 2160x2160. The eye passes took 9 to 11 ms a game frame there, with medium foveation. | -| `[vr] foveation` | `medium` | `off` shades every pixel and costs the most. See [Known issues](#known-issues) if images double. | -| `[vr] repeat_frames` | `true` (default) | Without it SteamVR halves the app's rate and fills refreshes itself. | -| `[vr] frame_interpolation_fps` | `0` | Rendering in-between frames needs 120 eye pairs a second; at these resolutions it made things worse. | -| `[video] resolution_multiplier` | `2` | The game's own frame, which the eyes are made from, at 2x the Wii's. 4x is far too heavy for the Adreno 750. | - -## Known issues - -- **Doubled images.** Images double in races and on the HUD, worst while racing, at first in the - right eye only and later in both. Both eyes get the same frames and repeats, so a one-eyed doubling - is not the 60 FPS cadence. The suspect is foveation: Turnip draws a coarse bin at lower resolution - and scales it back up, and each eye's density maps change with its gaze. Comparing foveation off, - on without eye tracking, and on with it is the next test. -- **Foveation and the right eye.** The full-density region follows the left eye better than the - right. Each eye already gets its own gaze direction; convergence on near content (the HUD screen at - 2 m, the cockpit) is not yet corrected for. -- **VR frame interpolation** is not recommended on the Frame (see the table above). -- **The Android flavour** cannot show a picture in Lepton (below). - -## The Android flavour in Lepton - -Everything from here to [Building and installing](#building-and-installing) is the `steamFrame` -flavour of the Quest app. Its controller, refresh rate and foveation work is shared with the native -build; its launch and manifest are Lepton's. - -### What the flavour changes - -| | Quest flavours | `steamFrame` | -| --- | --- | --- | -| CPU target (`kit.json` `androidCpu`) | `cortex-a77` (`kryo` on Quest 1) | `cortex-x4` | -| `MKW_HEADSET` | empty | `steam_frame` (defines `MKW_HEADSET_STEAM_FRAME`) | -| Library entry | `LauncherActivity` (Quest 1: `QuestActivity`) | `FrameEntryActivity` | -| Horizon OS manifest entries | present | removed | -| `[vr] refresh_rate` default | `0` (the headset's own) | `120` | -| `[vr] passthrough` | default on (`XR_FB_passthrough`) | not asked for, default off, setting hidden | -| `[vr] eye_tracked_foveation` default | off | on | -| `[vr] repeat_frames` default | off | on | - -The application ID stays `org.wiicompiled.quest`, so the storage paths in `docs/quest-port.md` hold -as they are. The kit's CPU string differs from the Quest ones, which gives the Frame its own kit -fingerprint: a game built for a Quest is refused on the Frame and the other way round, by the same -checks that keep Quest 1 and modern Quest games apart. - -**CPU.** Every core of the 8 Gen 3 implements ARMv9.2, so the products target the Cortex-X4 with -its whole feature set. That includes SVE and SVE2, which clang auto-vectorises with (a simple loop -compiled with `-O3` used SVE registers ten times). Phones with this chip do not expose SVE, but the -Frame's kernel does: `/proc/cpuinfo` lists `sve`, `sve2`, `svei8mm`, `svebf16` and the SVE2 crypto -extensions, on SteamOS and inside Lepton alike. A build for a device without SVE would need -`cortex-x4+nosve`. The flavour-to-CPU map lives once in -`android/app/build.gradle.kts` (`headsetCpus`), which the kit export also reads now instead of -guessing from the variant name. - -**Launch under Lepton.** Lepton starts the one real activity that is both `MAIN` and `LAUNCHER`, and -runs the app in VR when that activity carries a VR category; it ignores `activity-alias` entries. -Quest builds put `LAUNCHER` on the 2D panel (or, on Quest 1, add an alias), so neither works there. -`src/steamFrame/AndroidManifest.xml` makes `FrameEntryActivity` the only `MAIN`/`LAUNCHER` activity, -with `org.khronos.openxr.intent.category.IMMERSIVE_HMD` and `com.oculus.intent.category.VR`. It -shows nothing: it always opens the setup panel (`LauncherActivity`), and when the selected game can -start as it is (game files, a game built for this kit, Retro Rewind's pack, and no enabled mods -still to copy into the pack), it opens `QuestActivity` on top of it. The headset therefore goes -straight into VR, and quitting the game returns to the panel for setup, imports and mods. -`adb logcat -s WiiCompiledLauncher` shows which way it went. - -The manifest also removes Horizon OS's own entries (`com.oculus.supportedDevices`, `focusaware`, -`trade_cpu_for_gpu_amount`, the passthrough feature and the hand tracking permissions and feature) -and keeps the Khronos broker queries and the `OPENXR_SYSTEM` permission, which any Android OpenXR -runtime needs. - -## Controllers - -With `XR_VALVE_frame_controller_interaction` the runtime offers the Frame controller's own profile, -`/interaction_profiles/valve/frame_controller_valve`. Without it SteamVR presents the controllers as -Touch controllers, which loses the left D-pad. `openxr_input.cpp` suggests it after Touch and the -simple controller. Each hand has a thumbstick, trigger, grip and shoulder button. The right hand has -A, B, X, Y and a menu button; the left hand has a D-pad and a View button. Binding paths follow -DolphinXR's port (iChris4/dolphinXR#9). Windows asks for the same extension, so a Frame streaming -from a PC through SteamVR gets the D-pad too. - -| Frame controller | Wii Remote mode | Gamepad mode | -| --- | --- | --- | -| Right A | A | South (A) | -| Right B | C (look behind) | East (B) | -| Right trigger | B | Right trigger | -| Right stick up / down | 1 / 2 | Right stick | -| Left View | + (pause) | Start | -| Left shoulder | Settings panel (Touch's left Y) | North (Y) | -| Left D-pad | Wii Remote D-pad | D-pad | -| Left stick, left trigger | Nunchuk stick, Z | Left stick, left trigger | -| Grips, stick clicks, motion, aim | as on Touch (`OPENXR.md`, Controllers) | as on Touch | -| Right X, Y, menu and shoulder | unbound | unbound | - -The four D-pad actions are new and also reach the virtual gamepad's D-pad; Touch leaves them unbound, -so nothing changes on a Quest. - -## Refresh rate - -`[vr] refresh_rate` (Hz, `0` = the headset's own rate) is asked of the runtime through -`XR_FB_display_refresh_rate` each time the session starts running and whenever the setting changes. -The request uses the runtime's own value within half a hertz of the setting (runtimes report 119.98 -for 120). Setting it back to `0` restores the rate the session started at. The game renders 60 frames -a second, so at 120 Hz each frame shows for exactly two refreshes. At 72 or 90 Hz some frames show -for one refresh and others for two, which judders. The Frame starts at 120. - -Render-first pacing (`docs/quest-port.md`) submits a frame once the game has sealed one, 60 times a -second. On the Frame, SteamVR answered that by running the app at half rate (the pacing summary read -`predicted-rate=60.0Hz`) and filling every other refresh itself, even with Motion Smoothing off, which -doubled the HUD and the menu screen while the head turned. `[vr] repeat_frames` (default on for the -Frame, off elsewhere, live in the headset panel's VR tab) therefore submits the retained layer, with -the poses it was rendered for, on every refresh the next eyes are not ready for: the pacing thread -waits for them until 1.5 ms before the runtime's next wake (a period after the last xrWaitFrame -returned) and otherwise spends that refresh on a keep-alive cycle, which xrWaitFrame paces. The -summary should then read `predicted-rate=120.0Hz`, about 60 `keepalive` a second and 60 `new` -layers. A first version waited only a millisecond, so every packet also spent a refresh on a repeat -it did not need, the next packet missed the game's next frame, and `new` fell to about 35 a second. - -Lepton may decline the request (frame-control found SteamVR keeping its own rate there). The session -log then says `display refresh rate 120 Hz refused` with the rates it offers, and nothing else -changes. The setting is in the headset panel and in the launcher (VR → Headset); other headsets -offer it as well, at their own default of `0`. - -## Eye-tracked foveation - -`[vr] eye_tracked_foveation` (default on for the Frame) moves the foveation level's full-density -region to where the player looks: - -1. At launch the runtime asks for `XR_EXT_eye_gaze_interaction`. If the system reports an eye tracker, - `OpenXRInput` binds the gaze pose (`/user/eyes_ext/input/gaze_ext/pose`) and locates it for each - packet's display time, in the space the eye views are located in. -2. `vr/eye_gaze.h` turns the gaze into tangents of each eye's own view, which may be canted. - `AuroraStereoFrame` carries them as `gaze` and `gazeValid`, appended after its existing fields. -3. Aurora snaps the gaze to a cell of two map texels (64 pixels, about 3 degrees) and builds that - cell's density map with the level's rings centred on the gaze (`gfx/foveation.hpp`). Both rings - are 8 degrees wider than the fixed map's: the full-density region has to cover where a saccade - lands until the next game frame's map is bound, the tracker's error, and Turnip shading a whole - render-pass bin at one density. - -Each eye keeps up to 128 maps, one per cell looked at, so a glance back reuses its map instead of -uploading a new one. Each map has a memory block of its own: Turnip reads a map through a host -mapping of its memory, and Dawn's buffer uploads unmap the shared blocks they sub-allocate from. A new map is bound once its upload has completed, and until then the eye keeps -the map it had. A blink, lost tracking or the setting turned off returns to the map centred on the -forward direction, which is byte-identical to the fixed foveation map. Maps stay immutable, and the -Dawn patch lets a view be rebound to another map. - -The session log reports `OpenXR eye gaze: available` (or that the runtime has no eye tracker), -`OpenXR eye gaze: tracking` at the first tracked sample, and `eye foveation medium following the -gaze` for each eye's first gaze map. Foveation pays only when an eye is pixel-bound -(`OPENXR.md`, Foveated rendering), which the Frame's larger eyes make more likely. - -If the Frame's driver offers `VK_QCOM_fragment_density_map_offset` or -`VK_EXT_fragment_density_map_offset`, shifting one map per pass would replace switching between -maps. That needs the Dawn patch to create the eye textures with the offset flag, so it waits for -the device's extension list. - -## Building and installing - -On the Windows build host described in `docs/quest-port.md`: - -```powershell -powershell -ExecutionPolicy Bypass -File android/Build-Quest.ps1 -Headset frame # the steamFrame APK; checks kit.json says cortex-x4 -powershell -ExecutionPolicy Bypass -File android/Build-QuestGame.ps1 -Headset frame -Product base -Data # a .wcgame for the Frame's kit -``` - -The APK lands in `android/app/build/outputs/apk/steamFrame/`. WheelWizard VR's -"Build for Quest" builds a Frame game unchanged once it is given the Frame APK (Setup takes the -headset from the APK's kit). WheelWizard itself still needs a Steam Frame choice that fetches that -APK from the release, published as `…-SteamFrame.apk`. - -Lepton opens an adb port (5555 and up) for each running Android instance, reachable over the -network. With an Android app running on the Frame, `adb connect :5555` reaches it from the -build PC. How the APK reaches the Steam library (adb into a Lepton instance, frame-control, or -Steam's own sideloading) is to be confirmed on the device. - -## What the Frame reported - -Read on 2026-10-04 from a Steam Frame running SteamOS (`holo`), kernel 6.18.0, with the commands -below: - -| Reading | SteamOS | Lepton | -| --- | --- | --- | -| Page size | 4096 | 4096 | -| CPU | 8 cores; `sve sve2 svei8mm svebf16 sveaes svepmull svebitperm svesha3 svesm4 i8mm bf16 bti paca pacg ...` | the same | -| Android | — | 11 (API 30, LineageOS), `ro.product.model` Lepton, device `lepton_arm64_only`, platform `waydroid`, `ro.steam.running_in_app_container=true` | -| Vulkan driver | Turnip, Mesa 26.3.0-devel, Vulkan 1.4.362: `VK_EXT_fragment_density_map` (non-subsampled images, not dynamic), `VK_EXT_fragment_density_map_offset` and `VK_QCOM_fragment_density_map_offset`, `VK_KHR_external_{memory,semaphore,fence}_fd`, `VK_EXT_external_memory_dma_buf`, `VK_EXT_queue_family_foreign`, `VK_KHR_dynamic_rendering`; Valve's `fdm_injection` and `rpo` Vulkan layers | `ro.hardware.vulkan=freedreno`: the same Mesa 26.3.0-devel Turnip built for Android (`vulkan.pastel.so` is also present, not selected). `/dev/kgsl-3d0` is the DRM render node | -| OpenXR runtime | SteamVR, `bin/linuxarm64/vrclient.so` (`~/.config/openxr/1/active_runtime.json`); SteamOS ships the SDK headers, `libopenxr_loader.a` and `openxr.pc` | `/vendor/etc/openxr/1/active_runtime.json`, from the host's `/usr/share/guestos/android/vendor/etc/openxr`; no runtime broker package | -| Implicit OpenXR layers | `XrApiLayer_VALVE_fdm_injection` (also listed as explicit) | `XrApiLayer_VALVE_fdm_injection` | - -What follows from them: - -- **Fast memory path.** 4 KB pages keep the translated code's flat memory path. A 16 KB kernel - would have sent it through the checked path. -- **CPU target.** The Frame exposes SVE, so the build targets the whole `cortex-x4` (above). -- **Android version.** API 30 meets the app's minimum of 29. -- **Driver workarounds.** Lepton is a Waydroid container, and its Vulkan driver is Turnip. The - Adreno workarounds in `docs/quest-port.md` were found on Qualcomm's own driver. The vertex padding - stays on (it is correct either way); `debug.wiicompiled.vtxpad 0` can check whether Turnip needs it. -- **Foveation.** The host's Turnip has density maps for non-subsampled images through dynamic - rendering, what the Dawn patch needs, and both density map offset extensions, which would let - eye-tracked foveation shift one map instead of switching maps. -- **No buffer sharing between devices in Lepton.** The Vulkan Hardware Capability Viewer (4.03, - the last release for Android 11), run inside Lepton, reports Turnip `26.2.99` (Vulkan 1.4.362, - display name "Valve Lepton") with `VK_EXT_fragment_density_map`, both density map offset - extensions, `VK_VALVE_fragment_density_map_layered`, `VK_KHR_timeline_semaphore` and the - maintenance extensions, but **no** `VK_ANDROID_external_memory_android_hardware_buffer`, - `VK_KHR_external_memory_fd`, `VK_KHR_external_semaphore_fd` or `VK_KHR_external_fence_fd`. The - Quest backend (`openxr_vulkan.cpp`) hands each eye from Dawn's device to its own OpenXR device - through exactly those, so it cannot present under Lepton. What can: binding Dawn's own device to - the session, as the Windows Vulkan backend (`openxr_vulkan_win32.cpp`) does, so the eyes are - copied into the swapchain on Dawn's queue with no sharing at all. That backend is also the core - of a native SteamOS build ([above](#the-native-steamos-build)). -- **Finding the runtime.** An app inside Lepton reaches SteamVR's OpenXR runtime through the system - runtime file, not a broker. The Khronos loader the game links statically (`DYNAMIC_LOADER OFF`) - tries the runtime brokers first, then reads `/{product,odm,oem,vendor,system}/etc/openxr/1/active_runtime.json`, - so no app change is needed; the manifest's broker queries are simply unused here. Walkabout Mini - Golf, an Android VR game, runs in the same Lepton. -- **Valve's foveation layer.** `XrApiLayer_VALVE_fdm_injection` is implicit, so it wraps every - Android OpenXR app. By its name it adds fragment density maps to apps' own render passes. This - game draws its eyes on Dawn's device and only copies them into the swapchain on the OpenXR - device, so the layer has no render pass of the game's to change; the game's own maps - (eye-tracked foveation, above) do that. If the layer gets in the way, - `DISABLE_VULKAN_FDM_INJECTION_LAYER` turns it off (its manifest loads - `libVkLayer_VALVE_fdm_injection.so`; the runtime itself is - `/data/steamvr/runtime/bin/androidarm64/vrclient.so`). - -### SteamVR's Android OpenXR extensions - -Walkabout Mini Golf, a Unity game in the same Lepton, logs what the runtime offers (`adb logcat -d | -grep -F '[XR]'`). Its extensions: - -``` -XR_EXT_active_action_set_priority XR_EXT_debug_utils XR_EXT_dpad_binding XR_EXT_eye_gaze_interaction -XR_EXT_frame_composition_report XR_EXT_frame_synthesis XR_EXT_hand_interaction XR_EXT_hand_joints_motion_range -XR_EXT_hand_tracking XR_EXT_hand_tracking_data_source XR_EXT_hp_mixed_reality_controller -XR_EXT_interaction_profile_battery_state_display XR_EXT_interaction_render_model XR_EXT_local_floor -XR_EXT_palm_pose XR_EXT_performance_settings XR_EXT_render_model XR_EXT_user_presence XR_EXT_uuid -XR_EXT_view_configuration_views_change XR_FB_display_refresh_rate XR_FB_foveation -XR_FB_foveation_configuration XR_FB_foveation_vulkan XR_FB_space_warp XR_FB_swapchain_update_state -XR_HTC_vive_cosmos_controller_interaction XR_HTC_vive_focus3_controller_interaction -XR_HTC_vive_wrist_tracker_interaction XR_HTCX_vive_tracker_interaction XR_KHR_android_create_instance -XR_KHR_binding_modification XR_KHR_composition_layer_depth XR_KHR_generic_controller XR_KHR_locate_spaces -XR_KHR_opengl_enable XR_KHR_opengl_es_enable XR_KHR_visibility_mask XR_KHR_vulkan_enable -XR_KHR_vulkan_enable2 XR_META_foveation_eye_tracked XR_META_performance_metrics -XR_META_recommended_layer_resolution XR_META_vulkan_swapchain_create_info XR_MND_headless -XR_MNDX_egl_enable XR_VALVE_analog_threshold XR_VALVE_app_space_delta_pose -XR_VALVE_frame_controller_interaction XR_VALVE_timing_utils -``` - -Environment blend modes `OPAQUE` and `ALPHA_BLEND`; reference spaces `LOCAL`, `STAGE` and `VIEW`. -What this build asks for and gets: - -| Extension | Offered | What it means here | -| --- | --- | --- | -| `XR_KHR_android_create_instance`, `XR_KHR_vulkan_enable2` | yes | The Android backend's instance and Vulkan binding | -| `XR_VALVE_frame_controller_interaction` | yes | The Frame controller profile and its D-pad | -| `XR_FB_display_refresh_rate` | yes | `[vr] refresh_rate` (120 Hz) can be requested | -| `XR_EXT_eye_gaze_interaction` | yes | Eye-tracked foveation | -| `XR_EXT_performance_settings` | yes | `performance_level` | -| `XR_EXT_hand_tracking`, `XR_EXT_hand_tracking_data_source` | yes | Tracked hands; `XR_FB_hand_tracking_mesh` and `_aim` are not offered, so the cockpit draws its procedural gloves and bare hands get no pinch gestures | -| `XR_KHR_convert_timespec_time` | **no** | VR frame interpolation is unavailable, and controller motion is sampled at the frame's display time rather than the current time | -| `XR_KHR_android_thread_settings` | no | Thread hints are skipped (logged as refused) | -| `XR_FB_passthrough` | no | As expected; the build does not ask for it | - -Later candidates the runtime offers: `ALPHA_BLEND` could bring back the room around the menu screen -without `XR_FB_passthrough`, `XR_KHR_visibility_mask` would skip the pixels the lenses never show, -and `XR_FB_foveation` with `XR_META_foveation_eye_tracked` only shapes render passes into the -runtime's swapchain images, which this game's eyes reach by copy, so it does not apply. - -## Device checklist - -Information to collect first, from any PC over Lepton's adb port (the commands work in bash, zsh and -fish; in PowerShell only the adb lines do), with an Android app running on the Frame: - -```sh -adb connect :5555 -adb -s :5555 shell 'getprop ro.build.version.release; getprop ro.build.version.sdk; getprop ro.product.manufacturer; getprop ro.product.model; getprop ro.product.device; getprop ro.hardware.vulkan; getprop ro.board.platform' -adb -s :5555 shell 'uname -a; getconf PAGE_SIZE; grep -m1 Features /proc/cpuinfo; grep -c processor /proc/cpuinfo' -adb -s :5555 shell cmd gpu vkjson > frame-vkjson.json -adb -s :5555 shell "pm list packages | grep -i -E 'xr|valve|steam|khronos|openxr'" -# The parts of frame-vkjson.json that matter: -grep -oE '"(deviceName|driverName|driverInfo|apiVersion|driverVersion)": *[^,]*' frame-vkjson.json | sort -u -grep -oE '"extensionName": *"[^"]*"' frame-vkjson.json | grep -iE 'hardware_buffer|external_semaphore|external_fence|external_memory|fragment_density|shading_rate|dynamic_rendering' | sort -u -``` - -and on the Frame itself (Desktop Mode, Konsole): - -```bash -uname -a; getconf PAGESIZE; grep -m1 Features /proc/cpuinfo; head -5 /etc/os-release -cat ~/.config/openxr/1/active_runtime.json 2>/dev/null; ls /usr/share/openxr/1/ /etc/xdg/openxr/1/ 2>/dev/null -``` - -What each answers: - -- `vkjson`: whether Lepton's Vulkan driver has what the Android bridge needs: - - `VK_ANDROID_external_memory_android_hardware_buffer`; - - `VK_KHR_external_semaphore_fd` and `VK_KHR_external_fence_fd` with sync fd handles. - - Without these the APK route cannot present at all, and the native route becomes the way. It also - shows `VK_EXT_fragment_density_map` (foveation), the density map offset extensions, and which - driver Lepton uses. -- The CPU features line: no `sve`, as the CPU target assumes. -- The page size: on a 16 KB-page kernel the runtime finds out at launch and routes translated memory - accesses through the checked path (`guest_flat_memory.h`, `RequiresCheckedAccess`), which works but - is slower. That is worth knowing before measuring. -- The Android version: the app needs API 29 (Android 10) or newer. - -Then, on the first launch, the session log (`Logs/__pid/console.log` next to -`DATA`) and `adb logcat -s SDL WiiCompiledQuest WiiCompiledLauncher` should show, in order: - -1. `OpenXR Android loader initialized`; -2. `OpenXR runtime offers N extensions: ...`, the whole list SteamVR's Android runtime has; -3. `OpenXR initialized: runtime '...'` with SteamVR's name; -4. the negotiated Vulkan binding, then `OpenXR Vulkan swapchains ready`; -5. `display refresh rate 120 Hz requested` (or `refused`, with the rates on offer); -6. the session reaching `FOCUSED`; -7. `OpenXR interaction profiles: left /interaction_profiles/valve/frame_controller_valve, right ...`; -8. `OpenXR eye gaze: available`, then `tracking`; -9. `presentation=virtual-screen` for the menus, and `first immersive packet consumed` on race entry. - -Then, in the game: - -- the D-pad does tricks, the left shoulder opens the settings panel, and View pauses; -- `debug.wiicompiled.fpslog 1` on a race start shows the pacing summary and the GPU time per pass - with foveation off, fixed and following the gaze; -- `render_scale` starts at 0.8, the Quest's value. The runtime's recommended eye size (logged at - startup) and those measurements decide whether the Frame keeps it. - -A black headset with a working mirror points at the AHardwareBuffer copy, as on the Quest -(`docs/quest-port.md`, Validation status). - -Open questions only the device can answer: - -- Whether Lepton's driver supports the AHardwareBuffer and sync fd bridge. -- Whether Lepton shows the 2D setup panel while the app runs in VR mode. If it does not, the game - still starts directly once a `.wcgame` with the game files has been imported, but importing needs - the panel. -- Whether SteamVR grants 120 Hz. -- Whether the gaze needs an Android permission under Lepton. -- Whether "Build on this headset" can run its toolchain through `/system/bin/linker64` inside - Lepton. A game built on the PC does not depend on it.