Compare commits

..
4 Commits
Author SHA1 Message Date
saphidandClaude Opus 5.5 6d73912f8c Report an unreachable headset as 502, not a server error
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 07:40:48 +10:00
saphidandClaude Opus 5.5 60571dbfac Prepare Frame Control 0.2.0 for public testing
- Without the maintainer's key, Android compatibility reports stay on the
  Mac and the UI says so; the shared database is never contacted.
- Remove personal infrastructure details from scripts and docs: the Drive
  folder and gog wrapper now come from the environment, and the Chromium
  build host is required instead of defaulted.
- Add an MIT license, tester instructions in the README, and bump to 0.2.0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 07:37:37 +10:00
saphidandClaude Opus 5.5 f324aac690 Live H.264 video of the headset view in Frame Control
Live in the headset view now streams video instead of polling stereo
screenshots (~2 fps). SteamVR's steamvr-v4l2cam.service mirrors the headset
view into /dev/video99; ffmpeg on the Frame encodes it with x264 (720p30 by
default, AUD + repeated SPS/PPS), /api/stream relays the raw H.264 over SSH,
and the page splits it on access unit delimiters and decodes it with
WebCodecs into the existing viewer. Capture still takes a stereo still; the
desktop panel keeps capture polling, and the page falls back to it if the
video can't start.

The remote ffmpeg runs under a shell that kills it when the SSH channel
closes, stderr goes to a temp file, and a 10 s stall ends the stream. One
stream at a time; a new one supersedes the last.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 01:11:33 +10:00
saphidandClaude Opus 5.5 8b7c46a63a Add WebXR Chromium build and install scripts
Flathub Chromium can't enter immersive WebXR on Linux because upstream
only wires the OpenXR device on Windows. Document why, and add scripts to
cross-compile arm64 Chromium with the unmerged Linux OpenXR CLs and to
install, launch and check it on the Frame.

The first build is still running; immersive-vr support is unverified.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 00:29:27 +10:00
18 changed files with 673 additions and 60 deletions

No files matched your search

+2 -2
View File
@@ -1,11 +1,11 @@
---
name: steam-frame
description: Operate the user's Valve Steam Frame headset from the Mac through the ~/projects/steam-frame helpers and field notes. Use for Steam Frame SSH, screen streaming, clipboard, file push, APK or Flatpak installs, launching apps on the headset, arranging floating windows or panels in VR space, or debugging SteamOS/gamescope/SteamVR on the Frame.
description: Operate the user's Valve Steam Frame headset from the Mac through this repo's helpers and field notes. Use for Steam Frame SSH, screen streaming, clipboard, file push, APK or Flatpak installs, launching apps on the headset, arranging floating windows or panels in VR space, or debugging SteamOS/gamescope/SteamVR on the Frame.
---
# Steam Frame
The repo is `~/projects/steam-frame`. SSH works through the `frame` alias
SSH works through the `frame` alias
(user `steamos`). The headset has to be awake for anything that touches its
desktop or panels.
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 saphid
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+63 -17
View File
@@ -1,8 +1,13 @@
# Steam Frame ↔ Mac
This repo holds notes and Mac-side helpers for controlling a Valve Steam Frame
(standalone VR headset: SteamOS 3, Arch-based, arm64, Snapdragon 8 Gen 3) from
this Mac, with as little typing on the headset's virtual keyboard as possible.
**Frame Control** is a Mac app for managing a Valve Steam Frame (standalone VR
headset: SteamOS 3, Arch-based, arm64, Snapdragon 8 Gen 3) over SSH: live
headset view, battery and status, your Steam library, Android (Lepton) apps,
file and clipboard transfer. This repo also holds the scripts behind it and
field notes on how the Frame's software works, all aimed at as little typing
on the headset's virtual keyboard as possible.
It's an unofficial hobby project, not affiliated with Valve.
Status: written 2026-09-25 and checked against a real Frame the same day
(SteamOS 0.3.0, variant `vr`, build 20260922). The **Frame Control** Mac app
@@ -10,8 +15,41 @@ and most scripts are **verified** on the device. The scripts table below marks
each one, and [docs/open-questions.md](docs/open-questions.md#verified-on-device-2026-09-25)
lists what's still unchecked.
**Quick start:** set up SSH once (next section), then install
[Frame Control](#frame-control-mac-app) from the DMG.
## Trying it out
You need:
- A Steam Frame with **Developer Mode** on (next section; it's a toggle).
- A Mac with Apple Silicon (M1 or later). Tested on macOS 26. There's no Intel
build.
- `python3` on the Mac (`xcode-select --install` provides it).
- Optional: `adb` for Android apps (`brew install android-platform-tools`).
Steps:
1. Download the DMG from the
[latest release](https://github.com/saphid/steam-frame/releases/latest),
open it and drag **Frame Control** to Applications.
2. The app isn't notarized (no paid Apple developer account), so macOS will
say it's damaged or can't be checked. Clear the download quarantine once:
```sh
xattr -dr com.apple.quarantine "/Applications/Frame Control.app"
```
3. Open it. With no `frame` SSH alias yet, it offers to run the connection
setup in Terminal. That asks for the Developer Mode password once, then
uses a key from then on.
**Feedback:** please open a
[GitHub issue](https://github.com/saphid/steam-frame/issues) with what you
tried, your SteamOS build (Steam Settings → System) and the server log
(**Frame → Show Server Log**, at `~/Library/Logs/Frame Control/server.log`).
Features are marked **verified** or not below; the unverified ones are the
most useful to hear about.
**What it changes on your Frame:** only what you click. Installs go to your
user account (`--user` Flatpaks, Lepton instances, Steam downloads), and
nothing needs `sudo` except the power buttons. On the Mac it adds a `Host
frame` entry to `~/.ssh/config` and a key at `~/.ssh/id_ed25519_frame`.
## Minimum typing on the headset
@@ -33,8 +71,10 @@ On the Frame:
On the Mac:
To use the scripts from a checkout instead of the app:
```sh
cd ~/projects/steam-frame
git clone https://github.com/saphid/steam-frame.git && cd steam-frame
./scripts/connect.sh # or: ./scripts/connect.sh 192.168.1.50
ssh frame # passwordless from now on
```
@@ -105,10 +145,12 @@ the same UI in a browser without packaging:
- **Headset view**: what the lenses show, as SteamVR composites it (the room,
floating panels, dashboard and controllers). Shows the left eye, like pointing
a camera into one lens, or both eyes; single shot or about 2 fps live; saves
as PNG. The viewer fits the whole frame; zoom with − / + (or scroll, or
double-click), drag to pan, `0` to fit, `F` for full screen. It uses OpenVR's `IVRScreenshots` API through Python `ctypes`
(`ui/frame_vrshot.py`), so nothing is installed on the Frame. **Desktop panel**
a camera into one lens, or both eyes, as a single shot; saves as PNG. **Live**
is 720p video at about 30 fps: `ffmpeg` on the Frame encodes SteamVR's
headset-view device (`/dev/video99`) to H.264 over SSH, and the page decodes
it with WebCodecs. Live video is one eye; Capture still gets both. The viewer fits the whole frame; zoom with − / + (or scroll, or
double-click), drag to pan, `0` to fit, `F` for full screen. Capture uses OpenVR's `IVRScreenshots` API through Python `ctypes`
(`ui/frame_vrshot.py`). Nothing extra is installed on the Frame (SteamOS ships `ffmpeg`). **Desktop panel**
captures gamescope's flat layer instead.
- Battery with charging state: charge rate in watts, time to full or empty,
charger type and wattage (for example USB-C PD 20 W), and battery temperature
@@ -125,9 +167,10 @@ the same UI in a browser without packaging:
- **Android apps**: search about 4,500 F-Droid apps rated for the Frame, install
one with a click as its own Lepton instance (it keeps its data and shows in the
Steam library), then launch, stop, test or remove it. **Report an APK** records whether any APK
worked (F-Droid or not: pick a file, type a package, or use an installed app). The
ratings go into our private compatibility database (a Lakebed capsule only the
app can use, backed up daily to Google Drive; see `compat-db/README.md`)
worked (F-Droid or not: pick a file, type a package, or use an installed app). Your
reports are saved on your Mac and change the verdicts you see. They aren't
uploaded anywhere: the shared database is maintainer-only for now (see
`compat-db/README.md`)
- **Android display**: pick a running Lepton instance (by the app in it) and set
its resolution (Native 1920×1080, or Sharp 2560×1440 with density scaled to
match), UI scale (Smaller / Default / Larger, or an exact dpi) and text size
@@ -175,9 +218,7 @@ Finder. Each time it starts while there's no `frame` SSH alias, the app offers
to run `connect.sh` in Terminal. **Frame → Set Up Connection…** does the same
at any time. The Frame menu also shows the server log at
`~/Library/Logs/Frame Control/server.log`. Installing APKs needs `adb`
(`brew install android-platform-tools`). The Android ratings database needs
its key in the Keychain (see `compat-db/README.md`); without it, the app uses
its offline copy.
(`brew install android-platform-tools`). The F-Droid ratings are bundled with the app.
The build is ad-hoc signed and not notarized. A copy you build yourself opens
normally. A copy downloaded from GitHub Releases is quarantined; clear it with
@@ -200,7 +241,7 @@ showed live status and the library.
| `scripts/run-on-frame.sh` | Mac → Frame | Start an app on the headset desktop, e.g. `mac-screen` opens Remmina straight into the Mac (**verified**) |
| `scripts/frame-ui.sh` | Mac | Start the Frame Control web UI (`ui/server.py`) and open it (**verified**) |
| `scripts/apk-catalog.sh` | Mac | Refresh the rated F-Droid catalogue that Frame Control's Android section shows (**verified**) |
| `scripts/compat-db-backup.sh` | Mac | Back up the compatibility database locally and to Google Drive (daily LaunchAgent) (**verified**) |
| `scripts/compat-db-backup.sh` | Mac | Maintainer-only: back up the shared compatibility database locally and to Google Drive (**verified**) |
| `scripts/push-vr-video.sh` | Mac → Frame | Upload VR180/360 videos to `~/Videos/VR`, linked into DeoVR's Proton prefix; `--launch` starts DeoVR (**verified**: upload and link; in-headset playback of local files not yet checked). See [docs/vr-video.md](docs/vr-video.md) |
| `scripts/push.sh` | Mac → Frame | `rsync` files to `~/Downloads` (or a given path) on the Frame (**verified**) |
| `scripts/serve-bootstrap.sh` | Mac | Fallback: serve `bootstrap-on-frame.sh` with your public key embedded |
@@ -236,3 +277,8 @@ may find (Xcode Command Line Tools), plus syntax checks for every script and the
Electron main process (`.github/workflows/checks.yml`). Anything that touches the
headset is verified by hand against a real Frame, and the docs label it
**verified** or **inferred**.
## License
[MIT](LICENSE). Steam, Steam Frame and SteamVR are trademarks of Valve
Corporation. This project isn't affiliated with or endorsed by Valve.
+2 -2
View File
@@ -27,8 +27,8 @@ rules and the evidence behind them are in [docs/apks.md](../docs/apks.md).
## Compatibility reports
Reports live in Frame Control's private database, a Lakebed capsule at
`https://frame-compat.lakebed.app` that only the app can read or write (see
Reports are saved on your Mac. The maintainer's copy of Frame Control also
syncs them to a private Lakebed database (see
[compat-db/README.md](../compat-db/README.md), including backups). **Test**
records whether an app stays up in its own instance (`result`); **Report**
(on any installed app, catalogue card, or **+ Report an APK** for anything else, e.g. an
+3 -3
View File
@@ -1,13 +1,13 @@
{
"name": "frame-control",
"version": "0.1.1",
"version": "0.2.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "frame-control",
"version": "0.1.1",
"license": "UNLICENSED",
"version": "0.2.0",
"license": "MIT",
"devDependencies": {
"electron": "^44.4.5",
"electron-builder": "^26.15.3"
+2 -2
View File
@@ -1,11 +1,11 @@
{
"name": "frame-control",
"productName": "Frame Control",
"version": "0.1.1",
"version": "0.2.0",
"description": "Mac app for managing a Valve Steam Frame over SSH",
"private": true,
"main": "main.js",
"license": "UNLICENSED",
"license": "MIT",
"scripts": {
"start": "env -u ELECTRON_RUN_AS_NODE electron .",
"icon": "env -u ELECTRON_RUN_AS_NODE electron build/make-icon.js",
+7 -7
View File
@@ -1,11 +1,12 @@
# compat-db: Frame Control's compatibility database
A private [Lakebed](https://docs.lakebed.dev/) capsule holding compatibility
reports for Android apps on the Steam Frame. Only Frame Control can read or
write it.
reports for Android apps on the Steam Frame. For now only the maintainer's
copy of Frame Control has the key to read or write it. Everyone else's reports
stay on their own Mac (see `shared()` in `ui/frame_compat_db.py`).
- Live: `https://frame-compat.lakebed.app` (deploy `dep_dDmcsosVSiFirpW6`,
owned by `saphid`, doesn't expire). The browser page only says it's private.
claimed, so it doesn't expire). The browser page only says it's private.
- Access: `GET /v1/reports?since=<createdAt>` and `POST /v1/reports` with
`{"reports": [...]}`. Both need the `x-frame-control-key` header. There are
no Lakebed queries or mutations, so nothing else can reach the rows.
@@ -22,10 +23,9 @@ write it.
`scripts/compat-db-backup.sh` exports every report through the app key and
keeps dated copies in
`~/Library/Application Support/Frame Control/compat-db/backups` (newest 60).
When the data has changed, it also uploads them to Google Drive
(**the backup folder**, folder
`<drive-folder-id>`) with `gog`. The LaunchAgent
`frame-compat-backup` runs it daily at 03:40; the log is
When the data has changed, it also uploads them with `gog` to the Google
Drive folder named by `DRIVE_FOLDER_ID` (set it in the LaunchAgent's
`EnvironmentVariables`). A LaunchAgent runs it daily at 03:40 and logs to
`~/Library/Logs/frame-compat-backup.log`. If an export has fewer reports than
the last good backup (`backups/.last-good`), it's kept as `refused-*.json`,
nothing is uploaded, and every later run refuses too until you rerun with
+7 -8
View File
@@ -135,11 +135,11 @@ collects community reports for Steam games only and has no public API, and
Valve's "Great on Frame" badges and each Steam app's `recommended_runtime`
(for example `lepton-stable`) also cover Steam games only
([VR.org](https://vr.org/articles/steam-frame-lepton-android-runtime-52-of-130-certified-2026)).
So we keep our own, in a private Lakebed database
(`https://frame-compat.lakebed.app`) that only Frame Control can read or write.
So Frame Control keeps its own. Your reports are saved on your Mac; the
maintainer's copy also syncs them to a private Lakebed database.
**Test** records whether the app stays up in its own instance, and **Report**
(for any APK, F-Droid or not) records whether it worked, how it was run, where it came from, and notes, each with the SteamOS and Lepton build ids.
A daily job backs it up locally and to Google Drive. See
See
[compat-db/README.md](../compat-db/README.md) and
[apk-catalog/README.md](../apk-catalog/README.md).
@@ -188,13 +188,12 @@ But pasta runs with `--map-gw`, so the **gateway address inside Lepton
T3 Code v2 on the Mac listens only on `127.0.0.1:3873`. To reach it:
1. The LaunchAgent `~/Library/LaunchAgents/frame-t3-tunnel.plist`
keeps `ssh -N -R 127.0.0.1:3873:127.0.0.1:3873 frame` running. launchd
restarts it if it drops. Log: `~/Library/Logs/frame-t3-tunnel.log`.
1. Keep `ssh -N -R 127.0.0.1:3873:127.0.0.1:3873 frame` running on the Mac,
for example from a LaunchAgent with `KeepAlive`, so launchd restarts it if
it drops.
2. In the app on the Frame, the environment host is `192.168.1.1:3873`.
The app on the Frame was built from the v2 nightly source (fork commit
`d0c468e3`) with `expo prebuild` and `gradlew assembleRelease
The app on the Frame was built from the T3 Code v2 nightly source with `expo prebuild` and `gradlew assembleRelease
-PreactNativeArchitectures=arm64-v8a`, using Homebrew `openjdk@17` and the
`android-commandlinetools` SDK. It's signed with the debug key.
+1
View File
@@ -30,6 +30,7 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600
| Handy gamescope root properties on `:0`: `GAMESCOPE_FOCUSABLE_APPS`, `GAMESCOPE_FOCUSABLE_WINDOWS` (triples: window, app id, pid), `GAMESCOPE_FOCUSED_APP`. Read them with `DISPLAY=:0 xprop -root`. | Debugging panels |
| `gamescopectl screenshot <file>` (with `WAYLAND_DISPLAY=gamescope-0`) captures gamescope's flat layer. | Frame Control's capture |
| The **headset view** (both eyes, fully composited: room, panels, dashboard, controllers) comes from OpenVR `IVRScreenshots::RequestScreenshot(VRScreenshotType_Stereo)`. It's callable from `python3` with `ctypes` against `/opt/steamvr/bin/linuxarm64/libopenvr_api.so` as an overlay app. The compositor appends `.png`, writing a 1920×1080 side-by-side image (960×1080 per eye) plus a left-eye preview, in about 0.3s. In standby the frame is blank. `vrcmd --screenshot` and `vrcmd --compositorcmd screenshot_request` wrote nothing, even with `steamvr/rawCapturePath` set. | `ui/frame_vrshot.py` |
| SteamVR's `steamvr-v4l2cam.service` (`/opt/steamvr/bin/linuxarm64/v4l2cam --output=99`) copies the headset view (the `system.HeadsetView` mirror, one undistorted image) into the v4l2loopback device `/dev/video99` ("SteamVR"), 1920×1080 RGB24. `ffmpeg -f v4l2 -i /dev/video99` reads it at about 70 new frames/s; the first frame read can be black. The Frame's hardware encoder (`iris_encoder`, `/dev/video-enc0`) crashes ffmpeg's `h264_v4l2m2m`, so encode with `libx264 -preset ultrafast -tune zerolatency`: 720p30 takes about 0.7 of a core and 1080p60 about 1.7 (of 8). gamescope also publishes a PipeWire `gamescope` video source, but the Frame's GStreamer has no `pipewiresrc`. **Verified 2026-09-26.** | Frame Control's live video (`/api/stream`) |
| Battery: `/sys/class/power_supply/max1720x_bat_7-36` gives µV/µA (current is positive while charging), `time_to_full_now`/`time_to_empty_now` in seconds, and `temp` in tenths of °C. The charger shows up as `tcpm-source-psy-…` (`type=USB`, `usb_type=C PD [PD_PPS]`), for example 12 V × 1.67 A. | Frame Control's battery card |
| `vrcmd --stats` reports `activity_level` (3 = standby). | Telling whether the headset is being worn |
| The SteamVR dashboard has docking: Float in World, Move, Size, Curvature, controller docking, Theater, Multitasking View. **Inferred** from `/opt/steamvr/resources/webinterface/dashboard/` and not yet driven by hand. | [panels.md](panels.md) |
+82
View File
@@ -0,0 +1,82 @@
# WebXR in Chromium on the Frame
Goal: open a web VR180 or 360 player (DeoVR and DL8 embeds, WebXR samples),
press its VR button, and watch in 3D in the headset.
## Why Flathub Chromium can't
**Verified 2026-09-25** (Frame BUILD_ID 20260922.6101926, Flathub
`org.chromium.Chromium` 154.0.8037.57 aarch64):
- `navigator.xr` exists, but `isSessionSupported("immersive-vr")` is `false`.
Flags don't change that, and neither does `--force-webxr-runtime=openxr`
with SteamVR exposed to the Flatpak.
- The binary has no OpenXR loader: no `XR_RUNTIME_JSON`,
`xrGetInstanceProcAddr` or `XR_LOADER_DEBUG` strings. The only OpenXR
strings are the `chrome://flags` entries.
**Cause (verified against Chromium source, same date).** M154 is the first
release that compiles OpenXR on Linux:
- `device/vr/buildflags/buildflags.gni` adds `is_linux` to `enable_openxr`.
- Flathub's tarball sets `checkout_openxr = true`.
- Flathub's GN args don't turn it off.
But `content/services/isolated_xr_device/xr_runtime_provider.cc` only creates
the OpenXR device under `ENABLE_OPENXR && IS_WIN`. That's true on 154, 155 and
`main`. Nothing on Linux calls the OpenXR code, so the linker drops it. The
rest of the Linux port is in two unmerged CLs (bug 506004811):
- [8441736](https://chromium-review.googlesource.com/c/chromium/src/+/8441736)
runs the XR device service in a Linux sandbox that allows SteamVR's
sockets, `/dev/shm` and `flock`.
- [8132979](https://chromium-review.googlesource.com/c/chromium/src/+/8132979)
wires the provider to `OpenXrPlatformHelperLinux`. `kOpenXR` stays off by
default, so it needs `--enable-features=OpenXR`.
8132979 sits on top of 8441736, so fetching `refs/changes/79/8132979/<ps>`
gets both.
**The Frame side is ready.** `~/.config/openxr/1/active_runtime.json` names
SteamVR (`bin/linuxarm64/vrclient.so`, `VALVE_runtime_is_steamvr`). The
Linux backend uses Vulkan (`XR_USE_GRAPHICS_API_VULKAN`).
## Building it
[`scripts/build-chromium-xr.sh`](../scripts/build-chromium-xr.sh)
cross-compiles arm64 Linux Chromium on an x64 Linux host. It doesn't need
sudo: the arm64 sysroot comes from Chromium's own script. It needs about
90 GB of disk. It shallow-fetches the CL ref, runs `gclient sync --no-history`,
installs the sysroot, builds `chrome` with `symbol_level=0` and proprietary
codecs, and packs `chromium-xr-arm64.tar.xz`. Progress is logged to
`~/chromium-xr/stage`. The build aborts if `/` drops below 12 GB free.
First run: a 12-core, 31 GB x64 Linux box, started 2026-09-25.
## Running it on the Frame
[`scripts/chromium-xr.sh`](../scripts/chromium-xr.sh):
```sh
BUILD_HOST=my-linux-box scripts/chromium-xr.sh install # your build host; scp, unpack to ~/chromium-xr
scripts/chromium-xr.sh launch [URL] # headset desktop, --enable-features=OpenXR
scripts/chromium-xr.sh check # prints isSessionSupported('immersive-vr')
```
It runs natively, not as a Flatpak, so the XR sandbox and SteamVR's IPC work
as the CL expects. It uses its own profile (`~/.config/chromium-xr`) and
DevTools on loopback port 9223, so it doesn't collide with the Flatpak's 9222.
**Verified 2026-09-25:**
- Vulkan is there: Turnip (Mesa) on Adreno 750, API 1.4.359.
- Unprivileged user namespaces work (`unshare -Ur true`), so Chromium's
namespace sandbox shouldn't need the setuid `chrome_sandbox`.
**Unverified (inferred):**
- Chromium's GPU process may still fall back from Vulkan to GL on Turnip.
- An immersive session started from a window on the nested desktop may not
hand over cleanly to the SteamVR compositor.
- If the sandbox fails to start, `--no-sandbox` is the fallback for a first
test.
+79
View File
@@ -0,0 +1,79 @@
#!/bin/bash
# Linux-side (x64 host): cross-compile arm64 Chromium with the Linux OpenXR CLs
# (8441736 + 8132979, bug 506004811) so WebXR immersive-vr works on the Frame.
# Needs ~90 GB free, no sudo. Takes hours; run it detached on the build host:
# scp scripts/build-chromium-xr.sh buildhost:chromium-xr/build.sh
# ssh buildhost 'cd ~/chromium-xr && tmux new -d -s chromium-xr "./build.sh > build.log 2>&1"'
# Progress: ~/chromium-xr/stage. Output: ~/chromium-xr/chromium-xr-arm64.tar.xz,
# which scripts/chromium-xr.sh install copies to the Frame.
# Re-running resumes: existing checkout and out/XR are reused.
set -euo pipefail
W=~/chromium-xr
cd "$W"
stage(){ echo "$(date -Is) $*" | tee -a "$W/stage"; }
# Returns non-zero below 12 GB free; set -e turns that into an exit at top level.
guard(){ avail=$(df --output=avail -BG "$W" | tail -n 1 | tr -dc 0-9); if [ "$avail" -lt 12 ]; then stage "ABORT: only ${avail}G free for $W"; return 3; fi; }
[ -d depot_tools ] || git clone -q https://chromium.googlesource.com/chromium/tools/depot_tools.git
export PATH="$W/depot_tools:$PATH" DEPOT_TOOLS_UPDATE=1 DEPOT_TOOLS_METRICS=0
CL_REF=refs/changes/79/8132979/44
if [ ! -f .gclient ]; then
cat > .gclient <<'G'
solutions = [{ "name": "src", "url": "https://chromium.googlesource.com/chromium/src.git",
"managed": False, "custom_deps": {}, "custom_vars": { "checkout_nacl": False } }]
target_os = ["linux"]
target_cpu = ["arm64"]
G
fi
# Keyed on a real commit, so an interrupted first fetch is retried on re-run.
if ! git -C src rev-parse -q --verify HEAD >/dev/null 2>&1; then
stage "clone src at $CL_REF"
mkdir -p src
[ -d src/.git ] || git -C src init -q
git -C src remote get-url origin >/dev/null 2>&1 || git -C src remote add origin https://chromium.googlesource.com/chromium/src.git
git -C src fetch -q --depth=1 origin "$CL_REF"
git -C src checkout -q FETCH_HEAD
fi
guard
stage "src at $(git -C src log -1 --format='%h %s')"
stage "gclient sync"
gclient sync --nohooks --no-history -D --shallow --revision "src@$(git -C src rev-parse HEAD)" -j 8
guard
stage "runhooks"
gclient runhooks
src/build/linux/sysroot_scripts/install-sysroot.py --arch=arm64
guard
cd src
mkdir -p out/XR
cat > out/XR/args.gn <<'A'
target_os = "linux"
target_cpu = "arm64"
is_debug = false
is_official_build = false
is_component_build = false
dcheck_always_on = false
symbol_level = 0
blink_symbol_level = 0
v8_symbol_level = 0
proprietary_codecs = true
ffmpeg_branding = "Chrome"
use_remoteexec = false
use_siso = true
treat_warnings_as_errors = false
A
stage "gn gen"
gn gen out/XR
gn args out/XR --list=enable_openxr --short | tee -a "$W/stage"
stage "build"
( while sleep 600; do guard || { pkill -u "$(id -u)" -f "siso|ninja"; exit 3; }; done ) &
GUARD=$!
trap 'kill $GUARD 2>/dev/null || true' EXIT
autoninja -C out/XR chrome chrome_sandbox chrome_crashpad_handler
stage "package"
cd out/XR
files=(chrome chrome_sandbox chrome_crashpad_handler *.pak *.bin icudtl.dat locales)
# GPU libraries aren't produced by every config; pack the ones that exist.
for f in libEGL.so libGLESv2.so libvk_swiftshader.so libvulkan.so.1 vk_swiftshader_icd.json; do
[ -e "$f" ] && files+=("$f")
done
tar -cJf "$W/chromium-xr-arm64.tar.xz" "${files[@]}"
stage "DONE $(ls -la $W/chromium-xr-arm64.tar.xz)"
+98
View File
@@ -0,0 +1,98 @@
#!/usr/bin/env zsh
# Mac-side: install and launch the WebXR-enabled Chromium build on the Frame.
#
# Flathub Chromium can't enter immersive WebXR on Linux: upstream only wires
# the OpenXR device on Windows (see docs/webxr-chromium.md). This deploys an
# arm64 build with the Linux OpenXR CLs, made on a Linux host by
# scripts/build-chromium-xr.sh, into ~/chromium-xr on the Frame (not a
# Flatpak, so SteamVR's sockets and the XR sandbox work unmodified).
#
# Usage:
# scripts/chromium-xr.sh install [TARBALL] # default: scp from $BUILD_HOST
# scripts/chromium-xr.sh launch [URL] # opens in the headset desktop
# scripts/chromium-xr.sh check # isSessionSupported via DevTools
set -euo pipefail
FRAME_ALIAS=${FRAME_ALIAS:-frame}
BUILD_HOST=${BUILD_HOST:-}
BUILD_TARBALL=${BUILD_TARBALL:-chromium-xr/chromium-xr-arm64.tar.xz}
DEVTOOLS_PORT=${DEVTOOLS_PORT:-9223}
here=${0:A:h}
case "${1:-}" in
install)
tarball=${2:-}
if [[ -z "$tarball" ]]; then
[[ -n "$BUILD_HOST" ]] || { print -u2 "Pass a tarball, or set BUILD_HOST to the build machine"; exit 2; }
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
tarball=$tmp/chromium-xr-arm64.tar.xz
scp -q "$BUILD_HOST:$BUILD_TARBALL" "$tarball"
fi
ssh "$FRAME_ALIAS" 'rm -rf ~/chromium-xr.new && mkdir -p ~/chromium-xr.new'
ssh "$FRAME_ALIAS" 'tar -xJf - -C ~/chromium-xr.new' < "$tarball"
# Check the new build runs before replacing the old one.
ssh "$FRAME_ALIAS" '~/chromium-xr.new/chrome --version && rm -rf ~/chromium-xr && mv ~/chromium-xr.new ~/chromium-xr'
;;
launch)
# run-on-frame starts in $HOME on the Frame, so the profile path is relative.
exec "$here/run-on-frame.sh" -- '~/chromium-xr/chrome' \
--user-data-dir=.config/chromium-xr \
--enable-features=OpenXR \
--remote-debugging-port="$DEVTOOLS_PORT" \
"${2:-https://immersive-web.github.io/webxr-samples/}"
;;
check)
# DevTools listens on the Frame's loopback only; evaluate there.
ssh "$FRAME_ALIAS" python3 - "$DEVTOOLS_PORT" <<'EOF'
import json, sys, urllib.request, base64, os, socket, struct
port = int(sys.argv[1])
tabs = json.load(urllib.request.urlopen(f"http://127.0.0.1:{port}/json", timeout=10))
page = next((t for t in tabs if t["type"] == "page"), None)
if page is None:
sys.exit("no open page: run 'chromium-xr.sh launch' first")
path = page["webSocketDebuggerUrl"].split(f":{port}", 1)[1]
s = socket.create_connection(("127.0.0.1", port), timeout=30)
key = base64.b64encode(os.urandom(16)).decode()
s.sendall(f"GET {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nUpgrade: websocket\r\n"
f"Connection: Upgrade\r\nSec-WebSocket-Key: {key}\r\n"
"Sec-WebSocket-Version: 13\r\n\r\n".encode())
s.recv(4096)
msg = json.dumps({"id": 1, "method": "Runtime.evaluate", "params": {
"expression": "navigator.xr ? navigator.xr.isSessionSupported('immersive-vr') : 'no navigator.xr'",
"awaitPromise": True}}).encode()
mask = os.urandom(4)
hdr = bytes([0x81]) + (bytes([0x80 | len(msg)]) if len(msg) < 126
else bytes([0x80 | 126]) + struct.pack(">H", len(msg)))
s.sendall(hdr + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(msg)))
buf = b""
reply = None
while reply is None:
chunk = s.recv(65536)
if not chunk:
sys.exit("DevTools closed the connection")
buf += chunk
# Consume every complete frame already buffered before reading again.
while len(buf) >= 2:
n = buf[1] & 0x7F
off = 2
if n == 126:
if len(buf) < 4:
break
n, off = struct.unpack(">H", buf[2:4])[0], 4
elif n == 127:
if len(buf) < 10:
break
n, off = struct.unpack(">Q", buf[2:10])[0], 10
if len(buf) < off + n:
break
frame, buf = buf[off:off + n], buf[off + n:]
msg = json.loads(frame)
if msg.get("id") == 1:
reply = msg
break
print("immersive-vr supported:", reply["result"]["result"].get("value"))
EOF
;;
*) sed -n '2,13p' "$0"; exit 2 ;;
esac
+8 -7
View File
@@ -4,9 +4,9 @@
#
# Exports every report through the app's own key (ui/frame_compat_db.py), keeps
# dated copies in ~/Library/Application Support/Frame Control/compat-db/backups (newest
# 60), and uploads to Google Drive (the backup folder) when
# the data changed since the last upload. Run daily by the LaunchAgent
# frame-compat-backup (see docs/apks.md).
# 60), and uploads to the Google Drive folder DRIVE_FOLDER_ID when the data
# changed since the last upload. Maintainer-only: it needs the database key.
# Run it daily from a LaunchAgent (see compat-db/README.md).
#
# Usage: scripts/compat-db-backup.sh [--no-upload] [--force-upload] [--accept-shrink]
# Env: DRIVE_FOLDER_ID, GOG_WRAPPER
@@ -14,8 +14,8 @@ set -euo pipefail
ROOT="${0:A:h}/.."
DEST="$HOME/Library/Application Support/Frame Control/compat-db/backups"
DRIVE_FOLDER_ID=${DRIVE_FOLDER_ID:-<drive-folder-id>}
GOG_WRAPPER=${GOG_WRAPPER:-$HOME/bin/gog-with-keyring.sh}
DRIVE_FOLDER_ID=${DRIVE_FOLDER_ID:-}
GOG_WRAPPER=${GOG_WRAPPER:-$(command -v gog || true)}
upload=1 force=0 accept_shrink=0
for arg in "$@"; do
case "$arg" in
@@ -60,9 +60,10 @@ if (( upload )); then
print "==> Unchanged since the last Drive upload; skipped"
exit 0
fi
[[ -x "$GOG_WRAPPER" ]] || { print -u2 "gog wrapper not found at $GOG_WRAPPER"; exit 1; }
[[ -n "$DRIVE_FOLDER_ID" ]] || { print -u2 "Set DRIVE_FOLDER_ID, or pass --no-upload"; exit 1; }
[[ -n "$GOG_WRAPPER" && -x "$GOG_WRAPPER" ]] || { print -u2 "gog not found; install it or set GOG_WRAPPER"; exit 1; }
"$GOG_WRAPPER" drive upload "$out" --parent "$DRIVE_FOLDER_ID" --json --no-input >/dev/null
"$GOG_WRAPPER" drive upload "$out.sha256" --parent "$DRIVE_FOLDER_ID" --json --no-input >/dev/null
print -r -- "$digest" > "$last"
print "==> Uploaded to Google Drive (the backup folder)"
print "==> Uploaded to Google Drive"
fi
+51
View File
@@ -0,0 +1,51 @@
"""Compatibility reports without the maintainer's key: saved locally, never sent.
Run: python3 -m unittest discover -s tests
"""
import os
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT / "ui"))
import frame_compat_db as db # noqa: E402
class NoKey(unittest.TestCase):
def setUp(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
state = tmp.name
for name, value in (("STATE", state), ("OUTBOX", os.path.join(state, "outbox.jsonl")),
("MIRROR", os.path.join(state, "mirror.json"))):
p = mock.patch.object(db, name, value)
p.start()
self.addCleanup(p.stop)
db._mem.update(at=0, reports=None, source=None)
env = mock.patch.dict(os.environ, {}, clear=False)
env.start()
self.addCleanup(env.stop)
os.environ.pop("FRAME_CONTROL_KEY", None)
# No Keychain entry, and any network use fails the test.
no_key = mock.patch.object(db.subprocess, "run",
return_value=mock.Mock(returncode=44, stdout=""))
no_key.start()
self.addCleanup(no_key.stop)
net = mock.patch.object(db._opener, "open", side_effect=AssertionError("network used"))
net.start()
self.addCleanup(net.stop)
def test_report_is_kept_locally(self):
self.assertFalse(db.shared())
r = db.add({"package": "org.example.app", "date": "2026-09-26T10:00:00", "rating": "works"})
reports = db.load()
self.assertEqual([x["id"] for x in reports], [r["id"]])
self.assertEqual(db._mem["source"], "mirror")
if __name__ == "__main__":
unittest.main()
+6
View File
@@ -83,6 +83,7 @@ class ServerGuards(unittest.TestCase):
self.assertEqual(self.request("GET", "/api/status")[0], 403)
self.assertEqual(self.request("GET", "/api/screenshot?view=headset")[0], 403)
self.assertEqual(self.request("GET", "/api/shots")[0], 403)
self.assertEqual(self.request("GET", "/api/stream")[0], 403)
self.assertEqual(self.request("GET", "/api/shots/image?id=1/250820/20260925225208_1.jpg")[0], 403)
self.assertEqual(self.request("POST", "/api/launch", {"appid": "620"})[0], 403)
@@ -116,6 +117,11 @@ class ServerGuards(unittest.TestCase):
status, _, _ = self.request("GET", f"/api/shots/image?id={quote(shot)}", headers={"X-Frame-UI": "1"})
self.assertEqual(status, 400, shot)
def test_stream_settings_checked_before_ssh(self):
for query in ("h=480", "fps=24", "h=abc", "h=1080&fps=120"):
status, _, _ = self.request("GET", f"/api/stream?{query}", headers={"X-Frame-UI": "1"})
self.assertEqual(status, 400, query)
def test_bad_bodies(self):
conn = http.client.HTTPConnection("127.0.0.1", self.port, timeout=10)
conn.request("POST", "/api/launch", body=b"{not json", headers={"X-Frame-UI": "1"})
+13
View File
@@ -40,6 +40,16 @@ def key():
return p.stdout.strip()
def shared():
"""Whether reports reach the shared database. Without the key (anyone but the
maintainer), reports stay in this Mac's outbox and ratings come from the catalogue."""
try:
key()
return True
except DBError:
return False
class _NoRedirect(urllib.request.HTTPRedirectHandler):
"""Never follow redirects: urllib would copy the key header to the new host."""
def redirect_request(self, *args, **kwargs):
@@ -169,6 +179,8 @@ def load():
now = time.time()
if _mem['reports'] is None or now - _mem['at'] > TTL:
try:
if not shared():
raise DBError('no key')
try:
flush()
except Exception:
@@ -196,6 +208,7 @@ def add(report):
with _lock, open(OUTBOX, 'a') as f:
f.write(json.dumps(r, ensure_ascii=False) + '\n')
try:
if shared():
flush()
_mem['at'] = 0 # refetch on next load
except Exception:
+122 -8
View File
@@ -321,7 +321,7 @@
</div>
<div class="spacer"></div>
<button class="action" id="shotBtn">Capture</button>
<button id="liveBtn" title="Keep capturing">Live</button>
<button id="liveBtn" title="Keep updating: video of the headset view, or repeated captures of the desktop panel">Live</button>
<button id="saveBtn" disabled>Save</button>
</div>
<div class="viewer" id="viewer">
@@ -434,8 +434,8 @@
<div class="panel">
<div class="shelf-head"><h2>Recent reports</h2><span class="count" id="repCount"></span></div>
<div class="list" id="repList"><div class="sub">Loading…</div></div>
<div class="hint">Reports go to Frame Control's private compatibility database and change the
verdicts in the catalogue. Any APK can be reported, including ones not on F-Droid.</div>
<div class="hint" id="repHint">Reports change the verdicts in the catalogue. Any APK can be
reported, including ones not on F-Droid.</div>
</div>
</div>
<div class="panel">
@@ -575,7 +575,7 @@ const QUICK = [["Remmina", "org.remmina.Remmina"], ["Moonlight", "com.moonlight_
["Firefox", "org.mozilla.firefox"], ["VLC", "org.videolan.VLC"]];
const CDN = "https://cdn.cloudflare.steamstatic.com/steam/apps";
const HINTS = {
headset: "What the lenses show, composited by SteamVR: the room, floating panels, dashboard and controllers. Live refreshes about twice a second. Captures show everything on screen, including anything private.",
headset: "What the lenses show, composited by SteamVR: the room, floating panels, dashboard and controllers. Live streams it as video (one eye, about 30 fps); Capture takes a still of both eyes. Captures show everything on screen, including anything private.",
flat: "gamescope's 2D layer: the desktop panel and Steam's flat UI, without the room or VR scene.",
};
const SOURCE_LABEL = { steamvr: "Headset view", gamescope: "Desktop panel", shot: "Screenshot" };
@@ -751,10 +751,12 @@ function render(s) {
// ---- view ----
function setView(v) {
const changed = v !== view;
view = v;
document.querySelectorAll("[data-view]").forEach(b => b.classList.toggle("on", b.dataset.view === v));
$("eyeSeg").style.visibility = v === "headset" ? "visible" : "hidden";
$("eyeSeg").style.visibility = v === "headset" && !video.ctl ? "visible" : "hidden";
$("viewHint").textContent = HINTS[v];
if (live && changed) { toggleLive(false); toggleLive(true); } // video for the headset, captures for the panel
}
function setEye(e) {
eye = e;
@@ -841,6 +843,7 @@ function isBlank(ctx, w, h) {
return max - min < 6;
}
async function capture() {
if (video.ctl) toggleLive(false); // Capture during live video takes a still of both eyes instead
const gen = viewGen; // a screenshot opened meanwhile wins over this capture
if (!live) $("viewer").classList.add("busy"); // no spinner flashing over a live stream
let url = null;
@@ -883,7 +886,115 @@ function toggleLive(on) {
liveFailures = 0;
$("liveBtn").classList.toggle("on", live);
$("liveBadge").hidden = !live;
if (live) liveLoop(); else clearTimeout(liveTimer);
clearTimeout(liveTimer);
stopVideo();
if (!live) return;
if (view === "headset" && "VideoDecoder" in window) {
const started = startVideo();
const ctl = video.ctl;
started.catch(e => {
if (!live || ctl !== video.ctl || e.name === "AbortError") return;
log("Live video failed: " + e.message, "e");
toast("Live video failed, using captures instead: " + e.message, true);
stopVideo();
liveLoop();
});
} else liveLoop();
}
// ---- live video of the headset view ----
// The server relays raw H.264 (Annex B) from SteamVR's headset-view device.
// Every frame starts with an access unit delimiter (NAL type 9), which is how
// the stream is cut into frames for WebCodecs.
const STREAM_QUERY = "h=720&fps=30", STREAM_FPS = 30;
const video = { ctl: null, dec: null, gen: 0 };
function startCode(b, i) { return b[i] === 0 && b[i + 1] === 0 && b[i + 2] === 1; }
function nalTypes(au) {
const types = [];
for (let i = 0; i + 3 < au.length; i++) if (startCode(au, i)) { types.push(au[i + 3] & 0x1f); i += 3; }
return types;
}
async function startVideo() {
const gen = ++video.gen, ctl = new AbortController();
video.ctl = ctl;
$("eyeSeg").style.visibility = "hidden";
if (!lastImg) $("viewer").classList.add("busy");
let frames = 0, shown = 0, second = performance.now(), needKey = true, ts = 0;
const c = $("canvas"), ctx = c.getContext("2d");
const dec = video.dec = new VideoDecoder({
output: frame => {
if (gen !== video.gen) return frame.close();
if (c.width !== frame.displayWidth || c.height !== frame.displayHeight) {
c.width = frame.displayWidth; c.height = frame.displayHeight;
$("viewer").style.aspectRatio = `${c.width} / ${c.height}`;
setZoom(1);
}
ctx.drawImage(frame, 0, 0);
frame.close();
if (!shown++) {
viewGen++; // a capture still in flight mustn't replace the video
lastImg = null; lastSource = "video"; lastShot = null;
$("viewer").classList.remove("busy");
c.hidden = false; $("viewerEmpty").hidden = true; $("zoombar").hidden = false; $("asleep").hidden = true;
$("srcBadge").hidden = false; $("srcBadge").textContent = "Headset view · video";
$("stamp").hidden = false; $("saveBtn").disabled = false;
}
frames++;
const now = performance.now();
if (now - second >= 1000) {
$("stamp").textContent = `${Math.round(frames * 1000 / (now - second))} fps`;
frames = 0; second = now;
}
},
error: e => log("Video decoder: " + e.message, "e"),
});
const r = await fetch(`/api/stream?${STREAM_QUERY}`, { headers: {"X-Frame-UI": "1"}, signal: ctl.signal });
if (!r.ok) throw new Error((await r.json().catch(() => ({}))).error || `HTTP ${r.status}`);
const reader = r.body.getReader();
let buf = new Uint8Array(0), scan = 0, auStart = -1;
const onAU = au => {
const types = nalTypes(au), key = types.includes(5);
if (dec.state === "unconfigured") {
const sps = types.indexOf(7);
if (sps < 0) return;
let i = 0, n = -1; // find the SPS bytes: profile, constraints, level follow its header
for (; i + 3 < au.length; i++) if (startCode(au, i) && ++n === sps) break;
if (i + 6 >= au.length) return;
const hex = [au[i + 4], au[i + 5], au[i + 6]].map(b => b.toString(16).padStart(2, "0")).join("");
dec.configure({ codec: `avc1.${hex}`, optimizeForLatency: true });
}
// If decoding falls behind, drop frames until the next keyframe rather than lag.
if (dec.decodeQueueSize > 3) needKey = true;
if (needKey && !key) return;
needKey = false;
dec.decode(new EncodedVideoChunk({ type: key ? "key" : "delta", timestamp: ts, data: au }));
ts += 1e6 / STREAM_FPS;
};
for (;;) {
const { value, done } = await reader.read();
if (done || gen !== video.gen) break;
const next = new Uint8Array(buf.length + value.length);
next.set(buf); next.set(value, buf.length);
buf = next;
if (buf.length > 8 << 20) throw new Error("the stream isn't split into frames");
for (; scan + 3 < buf.length; scan++) {
if (!startCode(buf, scan) || (buf[scan + 3] & 0x1f) !== 9) continue;
if (auStart >= 0) onAU(buf.subarray(auStart, scan));
auStart = scan;
scan += 3;
}
if (auStart > 0) { buf = buf.slice(auStart); scan -= auStart; auStart = 0; }
}
if (gen === video.gen && live) throw new Error(shown ? "the stream ended" : "no video arrived");
}
function stopVideo() {
video.gen++;
if (video.ctl) video.ctl.abort();
if (video.dec && video.dec.state !== "closed") video.dec.close();
video.ctl = video.dec = null;
$("viewer").classList.remove("busy");
if (lastSource === "video") { $("srcBadge").textContent = "Headset view · video (stopped)"; $("stamp").hidden = true; }
$("eyeSeg").style.visibility = view === "headset" ? "visible" : "hidden";
}
$("shotBtn").onclick = () => capture();
$("liveBtn").onclick = () => toggleLive(!live);
@@ -1345,9 +1456,12 @@ const RLABEL = { works: "Works", issues: "Problems", broken: "Doesn't work", run
crashes: "Crashed (test)", install_failed: "Won't install", instance_failed: "Didn't start (test)" };
const RCLASS = { works: "works", runs: "works", issues: "maybe" };
async function loadReports() {
let reps;
try { reps = (await api("/api/android/reports")).reports; }
let reps, shared;
try { ({ reports: reps, shared } = await api("/api/android/reports")); }
catch (e) { $("repList").innerHTML = `<div class="sub">${esc(e.message)}</div>`; return; }
$("repHint").textContent = shared
? "Reports go to Frame Control's shared compatibility database and change the verdicts in the catalogue. Any APK can be reported, including ones not on F-Droid."
: "Reports are saved on this Mac and change the verdicts you see. They aren't uploaded: the shared database is maintainer-only for now. Any APK can be reported, including ones not on F-Droid.";
$("repCount").textContent = reps.length ? `${reps.length} newest` : "";
$("repList").innerHTML = reps.length ? reps.slice(0, 40).map(r => {
const k = r.rating || r.result;
+106 -4
View File
@@ -12,6 +12,7 @@ import http.client
import json
import os
import re
import select
import shlex
import shutil
import signal
@@ -272,6 +273,41 @@ def save_shots(body):
return {"message": msg + (f" ({skipped} already there)" if skipped else ""), "saved": n}
# Live video of the headset view. SteamVR's steamvr-v4l2cam.service copies the
# headset view (one undistorted 1920x1080 image) into the v4l2loopback device
# /dev/video99. ffmpeg encodes it with x264 (the hardware encoder crashes
# ffmpeg) and the raw H.264 comes back over SSH for the page to decode with
# WebCodecs. An access unit delimiter starts every frame so the page can split
# the stream, and repeated SPS/PPS let it start at any keyframe. ffmpeg runs in
# the background while the shell waits for our stdin to close: when the local
# ssh goes, the channel closes and the shell kills ffmpeg, even one that has
# stopped writing (and so would never get SIGPIPE).
STREAM_DEVICE = "/dev/video99"
STREAM_HEIGHTS = (720, 1080)
STREAM_FPS = (30, 60)
STREAM_STALL = 10 # seconds without video before the stream is dropped
_stream_lock = threading.Lock()
_stream_proc = None
def stream_command(query):
q = parse_qs(query)
try:
height = int((q.get("h") or ["720"])[0])
fps = int((q.get("fps") or ["30"])[0])
except ValueError:
raise Failure("h and fps must be integers", 400)
if height not in STREAM_HEIGHTS or fps not in STREAM_FPS:
raise Failure(f"h must be one of {STREAM_HEIGHTS} and fps one of {STREAM_FPS}", 400)
rate = 3 if height == 720 else 6 # Mbit/s
return (f"[ -e {STREAM_DEVICE} ] || {{ echo 'No headset view device ({STREAM_DEVICE}). Is SteamVR running?' >&2; exit 3; }}; "
f"ffmpeg -hide_banner -loglevel error -nostdin -f v4l2 -video_size 1920x1080 -i {STREAM_DEVICE} "
f"-vf fps={fps},scale=-2:{height},format=yuv420p -c:v libx264 -preset ultrafast -tune zerolatency "
f"-g {fps * 2} -bf 0 -b:v {rate}M -maxrate {rate}M -bufsize {rate // 2 or 1}M "
f"-x264-params aud=1:repeat-headers=1 -f h264 - & p=$!; "
f"exec >&-; cat >/dev/null; kill $p 2>/dev/null; wait $p")
def launch(body):
appid = str(body.get("appid", ""))
if not APPID.match(appid):
@@ -402,7 +438,8 @@ def android(body):
runtime=body.get("runtime") or "instance",
label=body.get("label"), source=body.get("source"))
name = r.get("label") or pkg
return {"message": f"Saved your report for {name}", "report": r}
where = "" if frame_catalog.compat_db.shared() else " on this Mac"
return {"message": f"Saved your report for {name}{where}", "report": r}
except frame_android.FrameError as e:
raise Failure(str(e))
raise Failure("unknown action", 400)
@@ -423,7 +460,7 @@ FONT_RANGE = (0.5, 2.0)
KNOWN_LABELS = {"com.t3tools.t3code": "T3 Code", "org.fdroid.fdroid": "F-Droid"}
# One ADB session at a time: requests are rare, and it keeps adb's state simple.
_adb_lock = threading.Lock()
_live_tunnels = set() # ssh processes to kill if the server stops mid-request
_live_tunnels = set() # ssh processes (ADB forwards, live video) to kill if the server stops mid-request
def adb_path():
@@ -757,7 +794,8 @@ class Handler(BaseHTTPRequestHandler):
elif path == "/api/android/displays":
self.send_json(android_displays())
elif path == "/api/android/reports":
self.send_json({"reports": frame_catalog.recent_reports()})
self.send_json({"reports": frame_catalog.recent_reports(),
"shared": frame_catalog.compat_db.shared()})
elif path == "/api/android/catalog":
self.send_json({"apps": frame_catalog.catalog()})
elif path == "/api/status":
@@ -770,6 +808,8 @@ class Handler(BaseHTTPRequestHandler):
self.send_json(list_shots())
elif path == "/api/shots/image":
self.send_bytes(*shot_image(url.query))
elif path == "/api/stream":
self.stream_video(url.query)
elif path == "/api/screenshot" and parse_qs(url.query).get("view") == ["headset"]:
self.send_bytes(headset_view(), "image/png", headers=[("X-Capture-Source", "steamvr")])
elif path == "/api/screenshot":
@@ -779,6 +819,8 @@ class Handler(BaseHTTPRequestHandler):
self.send_json({"error": "not found"}, 404)
except Failure as e:
self.send_json({"error": str(e)}, e.status)
except frame_android.FrameError as e:
self.send_json({"error": str(e)}, 502)
except Exception as e:
self.send_json({"error": f"{type(e).__name__}: {e}"}, 500)
@@ -805,9 +847,69 @@ class Handler(BaseHTTPRequestHandler):
self.send_json({"error": str(e)}, e.status)
except (ValueError, TypeError) as e:
self.send_json({"error": f"bad request: {e}"}, 400)
except frame_android.FrameError as e:
self.send_json({"error": str(e)}, 502)
except Exception as e:
self.send_json({"error": f"{type(e).__name__}: {e}"}, 500)
def stream_video(self, query):
"""Raw H.264 of the headset view until the page disconnects (see stream_command)."""
global _stream_proc
remote = stream_command(query)
ensure_master()
# stderr goes to a file: nothing reads it while streaming, and a full
# pipe would stall ffmpeg. It's only read if the stream fails to start.
errors = tempfile.TemporaryFile()
proc = subprocess.Popen([*SSH, FRAME, remote], stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=errors)
try:
_live_tunnels.add(proc)
# One viewer at a time: a new stream (another tab, a reload) ends the last one.
with _stream_lock:
old, _stream_proc = _stream_proc, proc
if old and old.poll() is None:
old.terminate()
fd = proc.stdout.fileno()
# Nothing is sent until the first bytes arrive, so a failure to
# start still comes back as a JSON error.
ready, _, _ = select.select([fd], [], [], 20)
first = os.read(fd, 1 << 16) if ready else b""
if not first:
proc.kill()
proc.wait()
errors.seek(0)
err = strip_ansi(errors.read().decode(errors="replace")).strip()
raise Failure(err or "The headset view sent no video for 20 s")
chunk = first
try:
self.send_response(200)
self.send_header("Content-Type", "video/h264")
self.send_header("Cache-Control", "no-store")
self.send_header("X-Frame-Options", "DENY")
self.send_header("Content-Security-Policy", "frame-ancestors 'none'")
self.end_headers()
self.close_connection = True # the body ends when the connection does
while chunk:
self.wfile.write(chunk)
self.wfile.flush()
# A stalled headset view ends the stream rather than
# holding this thread (and the page) forever.
ready, _, _ = select.select([fd], [], [], STREAM_STALL)
chunk = os.read(fd, 1 << 16) if ready else b""
except OSError:
pass # the page stopped watching (or stopped reading); the body has started, so no JSON
finally:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait()
for f in (proc.stdin, proc.stdout, errors):
f.close()
_live_tunnels.discard(proc)
def upload(self):
"""Raw file body. X-Filename names it; X-Mode is 'push', 'apk' (install) or 'apkinfo' (read only)."""
name = os.path.basename(unquote(self.headers.get("X-Filename", "")))
@@ -871,7 +973,7 @@ def main():
subprocess.run([*MUX, "-O", "exit", FRAME], capture_output=True)
if _master and _master.poll() is None:
_master.terminate()
for proc in list(_live_tunnels): # ADB forwards of requests cut off mid-way
for proc in list(_live_tunnels): # ADB forwards and video streams cut off mid-way
if proc.poll() is None:
proc.terminate()