Merge pull request #36 from saphid/linux-vr-streaming

docs: Linux VR feasibility and first-party streaming options

docs/apks.md

@@ -5,6 +5,12 @@ in **Lepton**, Valve's Waydroid-based container. Lepton is built for games,
 not general Android use
 ([GamingOnLinux](https://www.gamingonlinux.com/2026/09/lepton-from-valve-to-run-android-games-on-linux-is-now-open-source/)).
 
+**VR streaming clients:** WiVRn 26.9 and ALVR 20.14.1 install, but both fail
+OpenXR instance creation on the checked Frame because its Android runtime lacks
+`XR_KHR_convert_timespec_time` (**verified** 2026-09-28, SteamOS 0.4.1,
+BUILD_ID 20260925.6191901). They are not Frame Control dependencies. See
+[the feasibility results and options](linux-vr-streaming.md).
+
 ## Install from the Mac: one app, one Lepton instance (verified 2026-09-25)
 
 Use Frame Control's **Android apps** section (search, Install, Test, Report), drop

docs/evidence/linux-vr/2026-09-28.txt

@@ -0,0 +1,47 @@
+Frame client feasibility, 2026-09-28
+SteamOS VERSION_ID=0.4.1 BUILD_ID=20260925.6191901; uname -m=aarch64
+SteamVR: vrserver log reports 2.18.1; process 2256 remained alive across checks.
+Base: dcf9689f6459e576d35fc507eb702ca6b2bf4dad (main).
+Unmodified upstream release APKs; installed with ui/frame_android.py install APK --vr.
+No Linux host attached. No pairing, streamed video, input, audio or worn-headset checks.
+Selected logcat lines only; timestamps in Android logs are UTC.
+
+WiVRn-release.apk
+https://github.com/WiVRn/WiVRn/releases/tag/v26.9
+sha256=1df6649ec77224fcc821af0ab4897222bdf3d7eb6ce6ad636461336724111331
+E/OpenXR-Loader( 1140): Error [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance, no support found for requested extension: XR_KHR_convert_timespec_time
+I/WiVRn   ( 1140): [2026-09-28 12:07:58.987] [WiVRn] [info] Failed to create OpenXR instance version 1.1.58: XR_ERROR_EXTENSION_NOT_PRESENT
+E/OpenXR-Loader( 1140): Error [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance, no support found for requested extension: XR_KHR_convert_timespec_time
+I/WiVRn   ( 1140): [2026-09-28 12:07:59.034] [WiVRn] [info] Failed to create OpenXR instance version 1.0.58: XR_ERROR_EXTENSION_NOT_PRESENT
+E/WiVRn   ( 1140): [2026-09-28 12:07:59.035] [WiVRn] [error] Error during initialization: Failed to create OpenXR instance: XR_ERROR_EXTENSION_NOT_PRESENT
+
+alvr_client_android.apk
+https://github.com/alvr-org/ALVR/releases/tag/v20.14.1
+sha256=be68feeb02665e3d69f1cdbcabf38ea4d15c42868a7ec6b5e698dbefee4e4e36
+E/OpenXR-Loader( 1139): Error [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance, no support found for requested extension: XR_KHR_convert_timespec_time
+I/RustStdoutStderr( 1139): Error [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance, no support found for requested extension: XR_KHR_convert_timespec_time
+E/[ALVR NATIVE-RUST]( 1139): ALVR panicked: What happened:
+E/[ALVR NATIVE-RUST]( 1139): panicked at alvr/client_openxr/src/lib.rs:220:10:
+E/[ALVR NATIVE-RUST]( 1139): called `Result::unwrap()` on an `Err` value: ERROR_EXTENSION_NOT_PRESENT
+
+Cleanup verified: both app directories, compatdata directories and Steam shortcuts absent.
+Both test containers stopped and removed. No Steam/SteamVR restart or global setting changes.
+
+Valve release notes fetched from ISteamNews/GetNewsForApp/v2 (appid=250820).
+
+SteamVR Beta Updated - 2.18.1
+https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1844751498219787
+Added tethered Quest support over USB (must be used with Steam Link Beta)
+
+Introducing SteamVR 2.17
+https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1843481262693486
+Adds initial support for USB streaming. Note: Requires new Steam Client Beta.
+Fix crash using Steam Link on Linux when games submit invalid textures.
+Improve streaming recovery when using Steam Link on Linux.
+
+SteamVR Beta Updated - 2.17.8
+https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1842212951314598
+Fix crash using Steam Link on Linux when games submit invalid textures.
+Improve streaming recovery when using Steam Link on Linux.
+Adds initial support for USB streaming.
+USB streaming can be used without WiFi by opting into the Steam Client Beta.

docs/linux-vr-streaming.md

@@ -0,0 +1,195 @@
+# PC VR streaming from Linux
+
+**Recommendation, 2026-09-28:** test Valve's current SteamVR/Steam Link path
+on a Linux gaming PC before building another streamer. Valve now documents
+Linux streaming fixes and USB support. We have no Linux host attached, so
+Linux-to-Frame VR streaming remains **unverified here**.
+
+This is the feasibility and options report for
+[#24](https://github.com/saphid/frame-control/issues/24), not a shipped streaming
+feature. Frame Control's features must use our own implementation or standard
+platform components. WiVRn and ALVR are research comparisons, not dependencies.
+An optional install shortcut is the most we would offer for a third-party app.
+Our own streamer requires Alex's choice before implementation.
+
+## What was checked on the Frame
+
+**Verified** on 2026-09-28: aarch64, SteamOS **0.4.1**, BUILD_ID
+`20260925.6191901`, SteamVR **2.18.1**. Version and build are recorded separately;
+earlier docs associate this build with other SteamOS version labels.
+
+| Client | Installation | Runtime result |
+|---|---|---|
+| WiVRn **26.9**, upstream `WiVRn-release.apk` | API 29, arm64-v8a; installed in its own immersive Lepton instance | OpenXR instance creation fails: missing `XR_KHR_convert_timespec_time`. Both 1.1.58 and 1.0.58 attempts return `XR_ERROR_EXTENSION_NOT_PRESENT` |
+| ALVR **20.14.1**, upstream `alvr_client_android.apk` | API 26, arm64-v8a; installed in its own immersive Lepton instance | Same missing extension. Client panics at `client_openxr/src/lib.rs:220` with `ERROR_EXTENSION_NOT_PRESENT` |
+
+The [evidence excerpt](evidence/linux-vr/2026-09-28.txt) includes APK SHA-256s,
+upstream release links, loader errors and cleanup results. These are failures
+before an OpenXR session, not successful VR clients. WiVRn was launched twice;
+ALVR's container remained up despite its client panic. Container liveness alone
+does not establish VR compatibility.
+
+Both APKs already declare `MAIN` and `LAUNCHER`. They were installed unmodified
+using this branch's existing `python3 ui/frame_android.py install APK --vr`,
+then launched through their Steam shortcuts. Logs came from the instance's
+`podman exec … /system/bin/logcat`; the user journal also retained WiVRn's errors
+after its container exited. No headset was worn and no host was connected.
+
+All test app files, compatdata, shortcuts and containers were removed afterwards.
+SteamVR's original process remained running. No global settings changed.
+
+### Relation to the VR APK branch
+
+**Documented from source:** [PR #20](https://github.com/saphid/frame-control/pull/20)
+was read, not edited (branch inspected at
+[`038dcd4`](https://github.com/saphid/frame-control/commit/038dcd48cd75336f6a86c63c7878bfc9c52deec9)).
+Its compatibility layer handles OpenXR version negotiation, some controller
+profiles and refresh-rate requests. It does **not** implement
+`XR_KHR_convert_timespec_time`. Its launcher fix is unnecessary for these APKs.
+This report has **no unmerged code dependency** on that PR, and neither APK was
+tested with its layer injected.
+
+**Documented from upstream source:** WiVRn requests the extension in
+[`application.cpp`](https://github.com/WiVRn/WiVRn/blob/bbc6e4cc36c355fa6180980abd231673dc15115d/client/application.cpp#L1286)
+and uses it to convert `CLOCK_MONOTONIC` into `XrTime` in
+[`instance::now()`](https://github.com/WiVRn/WiVRn/blob/bbc6e4cc36c355fa6180980abd231673dc15115d/client/xr/instance.cpp#L335).
+ALVR also [requests it unconditionally](https://github.com/alvr-org/ALVR/blob/a9f6542fa507a841f40ab4f3fcb531427cd02550/alvr/client_openxr/src/lib.rs#L188).
+Simply deleting the extension request or returning made-up timestamps would
+not prove correct tracking or timing. A real fix needs a valid clock mapping
+and further runtime tests. No such patch was made.
+
+### Native SteamOS aarch64 clients
+
+**Verified:** the Frame has a native OpenXR runtime manifest at
+`~/.config/openxr/1/active_runtime.json`, pointing to SteamVR's
+`bin/linuxarm64/vrclient.so`.
+
+**Documented:** WiVRn's [26.9 README](https://github.com/WiVRn/WiVRn/blob/bbc6e4cc36c355fa6180980abd231673dc15115d/README.md)
+describes its Linux client as debugging-only, without audio or hardware decode.
+ALVR 20.14.1's [non-Android decoder](https://github.com/alvr-org/ALVR/blob/a9f6542fa507a841f40ab4f3fcb531427cd02550/alvr/client_core/src/video_decoder/mod.rs)
+returns no decoded frames. The inspected releases ship Android clients, not a
+ready-to-run native Frame client.
+
+**Inferred:** a native port is possible research, but neither release offers a
+demonstrated native alternative to the blocked APKs. Native builds, native
+extension enumeration, hardware decoding and audio were **not tested**. The
+Android extension failure does not establish that the native runtime lacks it.
+
+## (a) Valve's own path — recommended first
+
+**Documented**, from Valve's release notes rather than launch-window reports:
+
+- [SteamVR 2.17.8 beta](https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1842212951314598)
+  says “Fix crash using Steam Link on Linux when games submit invalid textures”
+  and “Improve streaming recovery when using Steam Link on Linux.” It also
+  adds initial USB streaming, with Steam Client Beta required to use USB
+  without Wi-Fi.
+- [SteamVR 2.17 release](https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1843481262693486)
+  repeats Linux streaming fixes and initial USB support. USB is no longer
+  solely a claim about an old beta, but version/channel requirements still
+  need checking on the actual host.
+- [SteamVR 2.18.1 beta](https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1844751498219787)
+  adds USB-tethered **Quest** support with Steam Link Beta. That entry is not
+  proof of a Frame/Linux combination.
+- The [Steam Link page](https://store.steampowered.com/app/353380/Steam_Link/)
+  lists Linux desktop clients, while its Quest VR requirements still say
+  Windows 10 or newer. Desktop Steam Link support is not equivalent to VR host
+  support, and the Quest requirements are not a Frame support matrix.
+
+**Inferred:** Valve has a Linux VR streaming path worth testing. The old blanket
+claim “Linux cannot stream VR” is no longer justified by the evidence. These
+release notes do not establish which Linux GPU/driver/Frame combinations work.
+USB changes the transport; it does not by itself prove host encoder support.
+
+**Not verified:** Linux host discovery, pairing, wireless or USB streaming,
+stereo rendering, controllers, haptics, audio, latency, or a game. Frame-only
+inspection cannot establish any of these. Flat Remote Play and a desktop shown
+on a panel are not substitutes for this test.
+
+Next test, once a Linux gaming PC is available: record distro, GPU/driver,
+Steam client channel/version and SteamVR version; use the Frame's built-in
+Steam connection flow, first wirelessly and then over a data-capable USB cable.
+Launch a free OpenXR sample or developer-consented VR game. Verify stereo,
+head/controller tracking, haptics and audio while worn; retain both ends' logs
+and measure latency and recovery after a link interruption. Restore any test
+channel changes. Do not change the shared headset's channel just for this report.
+
+If that works, Frame Control can provide our own host checks, setup guidance
+and session controls around Valve's existing platform. First establish which
+controls have a usable interface; no stable automated pairing API has been
+verified. **Estimate (inferred):** 2–5 engineer-days for the hardware feasibility
+pass; another 1–2 weeks for a small integration if those interfaces exist.
+
+## (b) Our own streaming — proposal only
+
+This is a new VR transport and device integration, not a desktop capture feature.
+A plausible first target is **one Linux GPU family, one host, one Frame**, using
+SteamVR on both ends. Our host driver would expose a remote HMD/controllers,
+receive poses and inputs, and obtain stereo textures for hardware encoding.
+Our Frame OpenXR app would decode, submit the correct eye views and render poses
+at predicted display times, and return tracking/input. SteamVR/OpenXR, bundled
+codec/transport libraries and platform GPU APIs fit the ownership rule; a
+WiVRn/ALVR/Monado server dependency would not.
+
+**Inferred design risks:** Linux SteamVR texture-sharing/driver interfaces and
+Frame decode-to-GPU interoperability need a spike before committing to this
+architecture. Sending an already-composited desktop mirror loses the stereo,
+pose and timing information we need. Late reprojection, clock conversion,
+backpressure, controller bindings, audio sync and reconnects are substantial
+work. A runtime shim must not assume `XrTime` equals monotonic nanoseconds.
+
+### Reuse from `mac-in-headset`
+
+**Documented from our code**, read-only at
+[`1b90c64`](https://github.com/saphid/frame-control/commit/1b90c64b73bace54c63a3aae154c5d29a9448d72):
+
+- Reuse the ideas for low-latency encoding without B-frames, dropping work
+  before encoding, bounded queues, keyframe recovery, adaptive bitrate,
+  per-frame timing and authenticated session setup.
+- Its VideoToolbox encoder and ScreenCaptureKit capture are macOS-specific.
+  Linux needs a new GPU encoder path (for example VA-API or NVENC via bundled
+  libraries) and VR texture capture, not a port of window capture.
+- Its WebSocket over SSH is useful for a first controlled transport experiment
+  and control messages. Reliable TCP can stall behind lost packets; a VR media
+  path needs measured deadline behaviour, likely datagrams with loss recovery
+  using an ordinary bundled transport library. Do not invent cryptography.
+- Its Chromium/WebCodecs panel viewer is not a VR client. That branch reports
+  software H.264 decoding and occasional long Wi-Fi stalls on the Frame.
+  Its desktop latency measurements are not motion-to-photon measurements or
+  evidence that a 90/120 Hz stereo stream will work.
+
+**Size/effort estimate (inferred, one experienced full-time engineer, hardware
+available):**
+
+| Phase | Deliverable / stop condition | Effort |
+|---|---|---|
+| Feasibility | Linux driver texture access, Frame hardware decode into OpenXR, pose/clock loop; stop if any cannot meet frame deadlines | 2–4 weeks |
+| First end-to-end prototype | One GPU/codec, stereo sample over a controlled LAN, head/controllers, logs and teardown | 4–8 additional weeks |
+| Usable limited beta | Audio/haptics, pairing, recovery, bitrate/loss handling, installer, worn testing and latency work | 6–12 additional weeks |
+| Wider support | Multiple GPU vendors/distros, USB and Wi-Fi variation, long-session stability | 2–4 additional months |
+
+Planning range: **12–24 engineer-weeks for a limited beta**, roughly
+**10–25k lines of our code plus tests/tooling**, excluding bundled libraries.
+This is a low-confidence scope estimate, not a delivery promise; an unsupported
+driver or decode interface could block it entirely. Foveated streaming,
+eye tracking and parity with Valve are excluded. A Linux gaming PC and repeatable
+worn-headset testing are prerequisites. **Do not build this until Alex chooses.**
+
+## (c) Optional “install WiVRn” shortcut only
+
+Allowed as a clearly optional convenience, never a prerequisite for a Frame
+Control feature. **Documented:** WiVRn's server Flatpak ID is
+`io.github.wivrn.wivrn`; its client/server versions must match, and its Flatpak
+includes xrizer/OpenComposite. Those are properties of an independently
+installed third-party stack, not components of our implementation.
+
+**Recommendation:** defer the shortcut while the current client fails before
+session creation. If offered later, label that compatibility result and let
+the user choose the install; do not present “install” as “streaming works.”
+**Estimate (inferred):** 1–2 engineer-days for an optional host-side shortcut
+with package/version detection and honest status, excluding third-party fixes.
+No shortcut, host install, pairing automation or streaming UI was built here.
+
+Choose **(a)** for the next hardware test. Keep **(b)** as a separately approved
+project if Valve's path fails or lacks a required capability. **(c)** does not
+solve the verified client blocker and should not be the product's foundation.

docs/streaming.md

@@ -5,6 +5,8 @@ This covers three directions, plus input:
 - **A. Frame → Mac**: see and control the headset from the Mac.
 - **B. Mac → Frame**: use the Mac's desktop inside the headset.
 - **C. iPhone → Frame**: mirror the phone inside the headset.
+- **PC VR from Linux**: [feasibility and options](linux-vr-streaming.md),
+  including Valve's streaming and USB support. No Linux host tested yet.
 - **Input**: type and point in the Frame from the Mac or iPhone.
 
 The confidence labels are the same as in [ssh.md](ssh.md).
@@ -24,11 +26,13 @@ Mac with keyboard, mouse, and clipboard.
 
 ## B. Show the Mac's desktop inside the Frame
 
-The Frame's streaming features are built around a **Windows PC running
-SteamVR** plus the USB Wi-Fi 6E dongle. Even Linux hosts had VR-streaming
-problems at launch
+The Frame's VR streaming uses **SteamVR** on the host. Linux hosts had
+VR-streaming problems at launch
 ([Steam discussion](https://steamcommunity.com/app/4165890/discussions/0/528765047224280796/),
 [gbl08ma](https://gbl08ma.com/posts/steam-frame-a-linux-machine-doesnt-support-linux/)).
+Valve's later 2.17.8 notes explicitly describe Steam Link fixes on Linux and
+initial USB streaming support (**documented**, not tested from a Linux host
+here). See [the current comparison](linux-vr-streaming.md#a-valves-own-path--recommended-first).
 **macOS isn't a supported SteamVR host**, so for the Mac we're only looking at
 flat 2D desktop streaming into a window on the Frame's Linux desktop.