mirror of
https://github.com/saphid/frame-control.git
synced 2026-10-06 05:02:50 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6d73912f8c | ||
|
|
60571dbfac | ||
|
|
f324aac690 | ||
|
|
8b7c46a63a |
No files matched your search
@@ -1,11 +1,11 @@
|
||||
---
|
||||
name: steam-frame
|
||||
description: Operate the user's Valve Steam Frame headset from the Mac through the ~/projects/steam-frame helpers and field notes. Use for Steam Frame SSH, screen streaming, clipboard, file push, APK or Flatpak installs, launching apps on the headset, arranging floating windows or panels in VR space, or debugging SteamOS/gamescope/SteamVR on the Frame.
|
||||
description: Operate the user's Valve Steam Frame headset from the Mac through this repo's helpers and field notes. Use for Steam Frame SSH, screen streaming, clipboard, file push, APK or Flatpak installs, launching apps on the headset, arranging floating windows or panels in VR space, or debugging SteamOS/gamescope/SteamVR on the Frame.
|
||||
---
|
||||
|
||||
# Steam Frame
|
||||
|
||||
The repo is `~/projects/steam-frame`. SSH works through the `frame` alias
|
||||
SSH works through the `frame` alias
|
||||
(user `steamos`). The headset has to be awake for anything that touches its
|
||||
desktop or panels.
|
||||
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 saphid
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -1,8 +1,13 @@
|
||||
# Steam Frame ↔ Mac
|
||||
|
||||
This repo holds notes and Mac-side helpers for controlling a Valve Steam Frame
|
||||
(standalone VR headset: SteamOS 3, Arch-based, arm64, Snapdragon 8 Gen 3) from
|
||||
this Mac, with as little typing on the headset's virtual keyboard as possible.
|
||||
**Frame Control** is a Mac app for managing a Valve Steam Frame (standalone VR
|
||||
headset: SteamOS 3, Arch-based, arm64, Snapdragon 8 Gen 3) over SSH: live
|
||||
headset view, battery and status, your Steam library, Android (Lepton) apps,
|
||||
file and clipboard transfer. This repo also holds the scripts behind it and
|
||||
field notes on how the Frame's software works, all aimed at as little typing
|
||||
on the headset's virtual keyboard as possible.
|
||||
|
||||
It's an unofficial hobby project, not affiliated with Valve.
|
||||
|
||||
Status: written 2026-09-25 and checked against a real Frame the same day
|
||||
(SteamOS 0.3.0, variant `vr`, build 20260922). The **Frame Control** Mac app
|
||||
@@ -10,8 +15,41 @@ and most scripts are **verified** on the device. The scripts table below marks
|
||||
each one, and [docs/open-questions.md](docs/open-questions.md#verified-on-device-2026-09-25)
|
||||
lists what's still unchecked.
|
||||
|
||||
**Quick start:** set up SSH once (next section), then install
|
||||
[Frame Control](#frame-control-mac-app) from the DMG.
|
||||
## Trying it out
|
||||
|
||||
You need:
|
||||
|
||||
- A Steam Frame with **Developer Mode** on (next section; it's a toggle).
|
||||
- A Mac with Apple Silicon (M1 or later). Tested on macOS 26. There's no Intel
|
||||
build.
|
||||
- `python3` on the Mac (`xcode-select --install` provides it).
|
||||
- Optional: `adb` for Android apps (`brew install android-platform-tools`).
|
||||
|
||||
Steps:
|
||||
|
||||
1. Download the DMG from the
|
||||
[latest release](https://github.com/saphid/steam-frame/releases/latest),
|
||||
open it and drag **Frame Control** to Applications.
|
||||
2. The app isn't notarized (no paid Apple developer account), so macOS will
|
||||
say it's damaged or can't be checked. Clear the download quarantine once:
|
||||
```sh
|
||||
xattr -dr com.apple.quarantine "/Applications/Frame Control.app"
|
||||
```
|
||||
3. Open it. With no `frame` SSH alias yet, it offers to run the connection
|
||||
setup in Terminal. That asks for the Developer Mode password once, then
|
||||
uses a key from then on.
|
||||
|
||||
**Feedback:** please open a
|
||||
[GitHub issue](https://github.com/saphid/steam-frame/issues) with what you
|
||||
tried, your SteamOS build (Steam Settings → System) and the server log
|
||||
(**Frame → Show Server Log**, at `~/Library/Logs/Frame Control/server.log`).
|
||||
Features are marked **verified** or not below; the unverified ones are the
|
||||
most useful to hear about.
|
||||
|
||||
**What it changes on your Frame:** only what you click. Installs go to your
|
||||
user account (`--user` Flatpaks, Lepton instances, Steam downloads), and
|
||||
nothing needs `sudo` except the power buttons. On the Mac it adds a `Host
|
||||
frame` entry to `~/.ssh/config` and a key at `~/.ssh/id_ed25519_frame`.
|
||||
|
||||
## Minimum typing on the headset
|
||||
|
||||
@@ -33,8 +71,10 @@ On the Frame:
|
||||
|
||||
On the Mac:
|
||||
|
||||
To use the scripts from a checkout instead of the app:
|
||||
|
||||
```sh
|
||||
cd ~/projects/steam-frame
|
||||
git clone https://github.com/saphid/steam-frame.git && cd steam-frame
|
||||
./scripts/connect.sh # or: ./scripts/connect.sh 192.168.1.50
|
||||
ssh frame # passwordless from now on
|
||||
```
|
||||
@@ -105,10 +145,12 @@ the same UI in a browser without packaging:
|
||||
|
||||
- **Headset view**: what the lenses show, as SteamVR composites it (the room,
|
||||
floating panels, dashboard and controllers). Shows the left eye, like pointing
|
||||
a camera into one lens, or both eyes; single shot or about 2 fps live; saves
|
||||
as PNG. The viewer fits the whole frame; zoom with − / + (or scroll, or
|
||||
double-click), drag to pan, `0` to fit, `F` for full screen. It uses OpenVR's `IVRScreenshots` API through Python `ctypes`
|
||||
(`ui/frame_vrshot.py`), so nothing is installed on the Frame. **Desktop panel**
|
||||
a camera into one lens, or both eyes, as a single shot; saves as PNG. **Live**
|
||||
is 720p video at about 30 fps: `ffmpeg` on the Frame encodes SteamVR's
|
||||
headset-view device (`/dev/video99`) to H.264 over SSH, and the page decodes
|
||||
it with WebCodecs. Live video is one eye; Capture still gets both. The viewer fits the whole frame; zoom with − / + (or scroll, or
|
||||
double-click), drag to pan, `0` to fit, `F` for full screen. Capture uses OpenVR's `IVRScreenshots` API through Python `ctypes`
|
||||
(`ui/frame_vrshot.py`). Nothing extra is installed on the Frame (SteamOS ships `ffmpeg`). **Desktop panel**
|
||||
captures gamescope's flat layer instead.
|
||||
- Battery with charging state: charge rate in watts, time to full or empty,
|
||||
charger type and wattage (for example USB-C PD 20 W), and battery temperature
|
||||
@@ -125,9 +167,10 @@ the same UI in a browser without packaging:
|
||||
- **Android apps**: search about 4,500 F-Droid apps rated for the Frame, install
|
||||
one with a click as its own Lepton instance (it keeps its data and shows in the
|
||||
Steam library), then launch, stop, test or remove it. **Report an APK** records whether any APK
|
||||
worked (F-Droid or not: pick a file, type a package, or use an installed app). The
|
||||
ratings go into our private compatibility database (a Lakebed capsule only the
|
||||
app can use, backed up daily to Google Drive; see `compat-db/README.md`)
|
||||
worked (F-Droid or not: pick a file, type a package, or use an installed app). Your
|
||||
reports are saved on your Mac and change the verdicts you see. They aren't
|
||||
uploaded anywhere: the shared database is maintainer-only for now (see
|
||||
`compat-db/README.md`)
|
||||
- **Android display**: pick a running Lepton instance (by the app in it) and set
|
||||
its resolution (Native 1920×1080, or Sharp 2560×1440 with density scaled to
|
||||
match), UI scale (Smaller / Default / Larger, or an exact dpi) and text size
|
||||
@@ -175,9 +218,7 @@ Finder. Each time it starts while there's no `frame` SSH alias, the app offers
|
||||
to run `connect.sh` in Terminal. **Frame → Set Up Connection…** does the same
|
||||
at any time. The Frame menu also shows the server log at
|
||||
`~/Library/Logs/Frame Control/server.log`. Installing APKs needs `adb`
|
||||
(`brew install android-platform-tools`). The Android ratings database needs
|
||||
its key in the Keychain (see `compat-db/README.md`); without it, the app uses
|
||||
its offline copy.
|
||||
(`brew install android-platform-tools`). The F-Droid ratings are bundled with the app.
|
||||
|
||||
The build is ad-hoc signed and not notarized. A copy you build yourself opens
|
||||
normally. A copy downloaded from GitHub Releases is quarantined; clear it with
|
||||
@@ -200,7 +241,7 @@ showed live status and the library.
|
||||
| `scripts/run-on-frame.sh` | Mac → Frame | Start an app on the headset desktop, e.g. `mac-screen` opens Remmina straight into the Mac (**verified**) |
|
||||
| `scripts/frame-ui.sh` | Mac | Start the Frame Control web UI (`ui/server.py`) and open it (**verified**) |
|
||||
| `scripts/apk-catalog.sh` | Mac | Refresh the rated F-Droid catalogue that Frame Control's Android section shows (**verified**) |
|
||||
| `scripts/compat-db-backup.sh` | Mac | Back up the compatibility database locally and to Google Drive (daily LaunchAgent) (**verified**) |
|
||||
| `scripts/compat-db-backup.sh` | Mac | Maintainer-only: back up the shared compatibility database locally and to Google Drive (**verified**) |
|
||||
| `scripts/push-vr-video.sh` | Mac → Frame | Upload VR180/360 videos to `~/Videos/VR`, linked into DeoVR's Proton prefix; `--launch` starts DeoVR (**verified**: upload and link; in-headset playback of local files not yet checked). See [docs/vr-video.md](docs/vr-video.md) |
|
||||
| `scripts/push.sh` | Mac → Frame | `rsync` files to `~/Downloads` (or a given path) on the Frame (**verified**) |
|
||||
| `scripts/serve-bootstrap.sh` | Mac | Fallback: serve `bootstrap-on-frame.sh` with your public key embedded |
|
||||
@@ -236,3 +277,8 @@ may find (Xcode Command Line Tools), plus syntax checks for every script and the
|
||||
Electron main process (`.github/workflows/checks.yml`). Anything that touches the
|
||||
headset is verified by hand against a real Frame, and the docs label it
|
||||
**verified** or **inferred**.
|
||||
|
||||
## License
|
||||
|
||||
[MIT](LICENSE). Steam, Steam Frame and SteamVR are trademarks of Valve
|
||||
Corporation. This project isn't affiliated with or endorsed by Valve.
|
||||
@@ -27,8 +27,8 @@ rules and the evidence behind them are in [docs/apks.md](../docs/apks.md).
|
||||
|
||||
## Compatibility reports
|
||||
|
||||
Reports live in Frame Control's private database, a Lakebed capsule at
|
||||
`https://frame-compat.lakebed.app` that only the app can read or write (see
|
||||
Reports are saved on your Mac. The maintainer's copy of Frame Control also
|
||||
syncs them to a private Lakebed database (see
|
||||
[compat-db/README.md](../compat-db/README.md), including backups). **Test**
|
||||
records whether an app stays up in its own instance (`result`); **Report**
|
||||
(on any installed app, catalogue card, or **+ Report an APK** for anything else, e.g. an
|
||||
|
||||
Generated
+3
-3
@@ -1,13 +1,13 @@
|
||||
{
|
||||
"name": "frame-control",
|
||||
"version": "0.1.1",
|
||||
"version": "0.2.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "frame-control",
|
||||
"version": "0.1.1",
|
||||
"license": "UNLICENSED",
|
||||
"version": "0.2.0",
|
||||
"license": "MIT",
|
||||
"devDependencies": {
|
||||
"electron": "^44.4.5",
|
||||
"electron-builder": "^26.15.3"
|
||||
|
||||
+2
-2
@@ -1,11 +1,11 @@
|
||||
{
|
||||
"name": "frame-control",
|
||||
"productName": "Frame Control",
|
||||
"version": "0.1.1",
|
||||
"version": "0.2.0",
|
||||
"description": "Mac app for managing a Valve Steam Frame over SSH",
|
||||
"private": true,
|
||||
"main": "main.js",
|
||||
"license": "UNLICENSED",
|
||||
"license": "MIT",
|
||||
"scripts": {
|
||||
"start": "env -u ELECTRON_RUN_AS_NODE electron .",
|
||||
"icon": "env -u ELECTRON_RUN_AS_NODE electron build/make-icon.js",
|
||||
|
||||
+7
-7
@@ -1,11 +1,12 @@
|
||||
# compat-db: Frame Control's compatibility database
|
||||
|
||||
A private [Lakebed](https://docs.lakebed.dev/) capsule holding compatibility
|
||||
reports for Android apps on the Steam Frame. Only Frame Control can read or
|
||||
write it.
|
||||
reports for Android apps on the Steam Frame. For now only the maintainer's
|
||||
copy of Frame Control has the key to read or write it. Everyone else's reports
|
||||
stay on their own Mac (see `shared()` in `ui/frame_compat_db.py`).
|
||||
|
||||
- Live: `https://frame-compat.lakebed.app` (deploy `dep_dDmcsosVSiFirpW6`,
|
||||
owned by `saphid`, doesn't expire). The browser page only says it's private.
|
||||
claimed, so it doesn't expire). The browser page only says it's private.
|
||||
- Access: `GET /v1/reports?since=<createdAt>` and `POST /v1/reports` with
|
||||
`{"reports": [...]}`. Both need the `x-frame-control-key` header. There are
|
||||
no Lakebed queries or mutations, so nothing else can reach the rows.
|
||||
@@ -22,10 +23,9 @@ write it.
|
||||
`scripts/compat-db-backup.sh` exports every report through the app key and
|
||||
keeps dated copies in
|
||||
`~/Library/Application Support/Frame Control/compat-db/backups` (newest 60).
|
||||
When the data has changed, it also uploads them to Google Drive
|
||||
(**the backup folder**, folder
|
||||
`<drive-folder-id>`) with `gog`. The LaunchAgent
|
||||
`frame-compat-backup` runs it daily at 03:40; the log is
|
||||
When the data has changed, it also uploads them with `gog` to the Google
|
||||
Drive folder named by `DRIVE_FOLDER_ID` (set it in the LaunchAgent's
|
||||
`EnvironmentVariables`). A LaunchAgent runs it daily at 03:40 and logs to
|
||||
`~/Library/Logs/frame-compat-backup.log`. If an export has fewer reports than
|
||||
the last good backup (`backups/.last-good`), it's kept as `refused-*.json`,
|
||||
nothing is uploaded, and every later run refuses too until you rerun with
|
||||
|
||||
+7
-8
@@ -135,11 +135,11 @@ collects community reports for Steam games only and has no public API, and
|
||||
Valve's "Great on Frame" badges and each Steam app's `recommended_runtime`
|
||||
(for example `lepton-stable`) also cover Steam games only
|
||||
([VR.org](https://vr.org/articles/steam-frame-lepton-android-runtime-52-of-130-certified-2026)).
|
||||
So we keep our own, in a private Lakebed database
|
||||
(`https://frame-compat.lakebed.app`) that only Frame Control can read or write.
|
||||
So Frame Control keeps its own. Your reports are saved on your Mac; the
|
||||
maintainer's copy also syncs them to a private Lakebed database.
|
||||
**Test** records whether the app stays up in its own instance, and **Report**
|
||||
(for any APK, F-Droid or not) records whether it worked, how it was run, where it came from, and notes, each with the SteamOS and Lepton build ids.
|
||||
A daily job backs it up locally and to Google Drive. See
|
||||
See
|
||||
[compat-db/README.md](../compat-db/README.md) and
|
||||
[apk-catalog/README.md](../apk-catalog/README.md).
|
||||
|
||||
@@ -188,13 +188,12 @@ But pasta runs with `--map-gw`, so the **gateway address inside Lepton
|
||||
|
||||
T3 Code v2 on the Mac listens only on `127.0.0.1:3873`. To reach it:
|
||||
|
||||
1. The LaunchAgent `~/Library/LaunchAgents/frame-t3-tunnel.plist`
|
||||
keeps `ssh -N -R 127.0.0.1:3873:127.0.0.1:3873 frame` running. launchd
|
||||
restarts it if it drops. Log: `~/Library/Logs/frame-t3-tunnel.log`.
|
||||
1. Keep `ssh -N -R 127.0.0.1:3873:127.0.0.1:3873 frame` running on the Mac,
|
||||
for example from a LaunchAgent with `KeepAlive`, so launchd restarts it if
|
||||
it drops.
|
||||
2. In the app on the Frame, the environment host is `192.168.1.1:3873`.
|
||||
|
||||
The app on the Frame was built from the v2 nightly source (fork commit
|
||||
`d0c468e3`) with `expo prebuild` and `gradlew assembleRelease
|
||||
The app on the Frame was built from the T3 Code v2 nightly source with `expo prebuild` and `gradlew assembleRelease
|
||||
-PreactNativeArchitectures=arm64-v8a`, using Homebrew `openjdk@17` and the
|
||||
`android-commandlinetools` SDK. It's signed with the debug key.
|
||||
|
||||
|
||||
@@ -30,6 +30,7 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600
|
||||
| Handy gamescope root properties on `:0`: `GAMESCOPE_FOCUSABLE_APPS`, `GAMESCOPE_FOCUSABLE_WINDOWS` (triples: window, app id, pid), `GAMESCOPE_FOCUSED_APP`. Read them with `DISPLAY=:0 xprop -root`. | Debugging panels |
|
||||
| `gamescopectl screenshot <file>` (with `WAYLAND_DISPLAY=gamescope-0`) captures gamescope's flat layer. | Frame Control's capture |
|
||||
| The **headset view** (both eyes, fully composited: room, panels, dashboard, controllers) comes from OpenVR `IVRScreenshots::RequestScreenshot(VRScreenshotType_Stereo)`. It's callable from `python3` with `ctypes` against `/opt/steamvr/bin/linuxarm64/libopenvr_api.so` as an overlay app. The compositor appends `.png`, writing a 1920×1080 side-by-side image (960×1080 per eye) plus a left-eye preview, in about 0.3s. In standby the frame is blank. `vrcmd --screenshot` and `vrcmd --compositorcmd screenshot_request` wrote nothing, even with `steamvr/rawCapturePath` set. | `ui/frame_vrshot.py` |
|
||||
| SteamVR's `steamvr-v4l2cam.service` (`/opt/steamvr/bin/linuxarm64/v4l2cam --output=99`) copies the headset view (the `system.HeadsetView` mirror, one undistorted image) into the v4l2loopback device `/dev/video99` ("SteamVR"), 1920×1080 RGB24. `ffmpeg -f v4l2 -i /dev/video99` reads it at about 70 new frames/s; the first frame read can be black. The Frame's hardware encoder (`iris_encoder`, `/dev/video-enc0`) crashes ffmpeg's `h264_v4l2m2m`, so encode with `libx264 -preset ultrafast -tune zerolatency`: 720p30 takes about 0.7 of a core and 1080p60 about 1.7 (of 8). gamescope also publishes a PipeWire `gamescope` video source, but the Frame's GStreamer has no `pipewiresrc`. **Verified 2026-09-26.** | Frame Control's live video (`/api/stream`) |
|
||||
| Battery: `/sys/class/power_supply/max1720x_bat_7-36` gives µV/µA (current is positive while charging), `time_to_full_now`/`time_to_empty_now` in seconds, and `temp` in tenths of °C. The charger shows up as `tcpm-source-psy-…` (`type=USB`, `usb_type=C PD [PD_PPS]`), for example 12 V × 1.67 A. | Frame Control's battery card |
|
||||
| `vrcmd --stats` reports `activity_level` (3 = standby). | Telling whether the headset is being worn |
|
||||
| The SteamVR dashboard has docking: Float in World, Move, Size, Curvature, controller docking, Theater, Multitasking View. **Inferred** from `/opt/steamvr/resources/webinterface/dashboard/` and not yet driven by hand. | [panels.md](panels.md) |
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
# WebXR in Chromium on the Frame
|
||||
|
||||
Goal: open a web VR180 or 360 player (DeoVR and DL8 embeds, WebXR samples),
|
||||
press its VR button, and watch in 3D in the headset.
|
||||
|
||||
## Why Flathub Chromium can't
|
||||
|
||||
**Verified 2026-09-25** (Frame BUILD_ID 20260922.6101926, Flathub
|
||||
`org.chromium.Chromium` 154.0.8037.57 aarch64):
|
||||
|
||||
- `navigator.xr` exists, but `isSessionSupported("immersive-vr")` is `false`.
|
||||
Flags don't change that, and neither does `--force-webxr-runtime=openxr`
|
||||
with SteamVR exposed to the Flatpak.
|
||||
- The binary has no OpenXR loader: no `XR_RUNTIME_JSON`,
|
||||
`xrGetInstanceProcAddr` or `XR_LOADER_DEBUG` strings. The only OpenXR
|
||||
strings are the `chrome://flags` entries.
|
||||
|
||||
**Cause (verified against Chromium source, same date).** M154 is the first
|
||||
release that compiles OpenXR on Linux:
|
||||
|
||||
- `device/vr/buildflags/buildflags.gni` adds `is_linux` to `enable_openxr`.
|
||||
- Flathub's tarball sets `checkout_openxr = true`.
|
||||
- Flathub's GN args don't turn it off.
|
||||
|
||||
But `content/services/isolated_xr_device/xr_runtime_provider.cc` only creates
|
||||
the OpenXR device under `ENABLE_OPENXR && IS_WIN`. That's true on 154, 155 and
|
||||
`main`. Nothing on Linux calls the OpenXR code, so the linker drops it. The
|
||||
rest of the Linux port is in two unmerged CLs (bug 506004811):
|
||||
|
||||
- [8441736](https://chromium-review.googlesource.com/c/chromium/src/+/8441736)
|
||||
runs the XR device service in a Linux sandbox that allows SteamVR's
|
||||
sockets, `/dev/shm` and `flock`.
|
||||
- [8132979](https://chromium-review.googlesource.com/c/chromium/src/+/8132979)
|
||||
wires the provider to `OpenXrPlatformHelperLinux`. `kOpenXR` stays off by
|
||||
default, so it needs `--enable-features=OpenXR`.
|
||||
|
||||
8132979 sits on top of 8441736, so fetching `refs/changes/79/8132979/<ps>`
|
||||
gets both.
|
||||
|
||||
**The Frame side is ready.** `~/.config/openxr/1/active_runtime.json` names
|
||||
SteamVR (`bin/linuxarm64/vrclient.so`, `VALVE_runtime_is_steamvr`). The
|
||||
Linux backend uses Vulkan (`XR_USE_GRAPHICS_API_VULKAN`).
|
||||
|
||||
## Building it
|
||||
|
||||
[`scripts/build-chromium-xr.sh`](../scripts/build-chromium-xr.sh)
|
||||
cross-compiles arm64 Linux Chromium on an x64 Linux host. It doesn't need
|
||||
sudo: the arm64 sysroot comes from Chromium's own script. It needs about
|
||||
90 GB of disk. It shallow-fetches the CL ref, runs `gclient sync --no-history`,
|
||||
installs the sysroot, builds `chrome` with `symbol_level=0` and proprietary
|
||||
codecs, and packs `chromium-xr-arm64.tar.xz`. Progress is logged to
|
||||
`~/chromium-xr/stage`. The build aborts if `/` drops below 12 GB free.
|
||||
|
||||
First run: a 12-core, 31 GB x64 Linux box, started 2026-09-25.
|
||||
|
||||
## Running it on the Frame
|
||||
|
||||
[`scripts/chromium-xr.sh`](../scripts/chromium-xr.sh):
|
||||
|
||||
```sh
|
||||
BUILD_HOST=my-linux-box scripts/chromium-xr.sh install # your build host; scp, unpack to ~/chromium-xr
|
||||
scripts/chromium-xr.sh launch [URL] # headset desktop, --enable-features=OpenXR
|
||||
scripts/chromium-xr.sh check # prints isSessionSupported('immersive-vr')
|
||||
```
|
||||
|
||||
It runs natively, not as a Flatpak, so the XR sandbox and SteamVR's IPC work
|
||||
as the CL expects. It uses its own profile (`~/.config/chromium-xr`) and
|
||||
DevTools on loopback port 9223, so it doesn't collide with the Flatpak's 9222.
|
||||
|
||||
**Verified 2026-09-25:**
|
||||
|
||||
- Vulkan is there: Turnip (Mesa) on Adreno 750, API 1.4.359.
|
||||
- Unprivileged user namespaces work (`unshare -Ur true`), so Chromium's
|
||||
namespace sandbox shouldn't need the setuid `chrome_sandbox`.
|
||||
|
||||
**Unverified (inferred):**
|
||||
|
||||
- Chromium's GPU process may still fall back from Vulkan to GL on Turnip.
|
||||
- An immersive session started from a window on the nested desktop may not
|
||||
hand over cleanly to the SteamVR compositor.
|
||||
- If the sandbox fails to start, `--no-sandbox` is the fallback for a first
|
||||
test.
|
||||
Executable
+79
@@ -0,0 +1,79 @@
|
||||
#!/bin/bash
|
||||
# Linux-side (x64 host): cross-compile arm64 Chromium with the Linux OpenXR CLs
|
||||
# (8441736 + 8132979, bug 506004811) so WebXR immersive-vr works on the Frame.
|
||||
# Needs ~90 GB free, no sudo. Takes hours; run it detached on the build host:
|
||||
# scp scripts/build-chromium-xr.sh buildhost:chromium-xr/build.sh
|
||||
# ssh buildhost 'cd ~/chromium-xr && tmux new -d -s chromium-xr "./build.sh > build.log 2>&1"'
|
||||
# Progress: ~/chromium-xr/stage. Output: ~/chromium-xr/chromium-xr-arm64.tar.xz,
|
||||
# which scripts/chromium-xr.sh install copies to the Frame.
|
||||
# Re-running resumes: existing checkout and out/XR are reused.
|
||||
set -euo pipefail
|
||||
W=~/chromium-xr
|
||||
cd "$W"
|
||||
stage(){ echo "$(date -Is) $*" | tee -a "$W/stage"; }
|
||||
# Returns non-zero below 12 GB free; set -e turns that into an exit at top level.
|
||||
guard(){ avail=$(df --output=avail -BG "$W" | tail -n 1 | tr -dc 0-9); if [ "$avail" -lt 12 ]; then stage "ABORT: only ${avail}G free for $W"; return 3; fi; }
|
||||
[ -d depot_tools ] || git clone -q https://chromium.googlesource.com/chromium/tools/depot_tools.git
|
||||
export PATH="$W/depot_tools:$PATH" DEPOT_TOOLS_UPDATE=1 DEPOT_TOOLS_METRICS=0
|
||||
CL_REF=refs/changes/79/8132979/44
|
||||
if [ ! -f .gclient ]; then
|
||||
cat > .gclient <<'G'
|
||||
solutions = [{ "name": "src", "url": "https://chromium.googlesource.com/chromium/src.git",
|
||||
"managed": False, "custom_deps": {}, "custom_vars": { "checkout_nacl": False } }]
|
||||
target_os = ["linux"]
|
||||
target_cpu = ["arm64"]
|
||||
G
|
||||
fi
|
||||
# Keyed on a real commit, so an interrupted first fetch is retried on re-run.
|
||||
if ! git -C src rev-parse -q --verify HEAD >/dev/null 2>&1; then
|
||||
stage "clone src at $CL_REF"
|
||||
mkdir -p src
|
||||
[ -d src/.git ] || git -C src init -q
|
||||
git -C src remote get-url origin >/dev/null 2>&1 || git -C src remote add origin https://chromium.googlesource.com/chromium/src.git
|
||||
git -C src fetch -q --depth=1 origin "$CL_REF"
|
||||
git -C src checkout -q FETCH_HEAD
|
||||
fi
|
||||
guard
|
||||
stage "src at $(git -C src log -1 --format='%h %s')"
|
||||
stage "gclient sync"
|
||||
gclient sync --nohooks --no-history -D --shallow --revision "src@$(git -C src rev-parse HEAD)" -j 8
|
||||
guard
|
||||
stage "runhooks"
|
||||
gclient runhooks
|
||||
src/build/linux/sysroot_scripts/install-sysroot.py --arch=arm64
|
||||
guard
|
||||
cd src
|
||||
mkdir -p out/XR
|
||||
cat > out/XR/args.gn <<'A'
|
||||
target_os = "linux"
|
||||
target_cpu = "arm64"
|
||||
is_debug = false
|
||||
is_official_build = false
|
||||
is_component_build = false
|
||||
dcheck_always_on = false
|
||||
symbol_level = 0
|
||||
blink_symbol_level = 0
|
||||
v8_symbol_level = 0
|
||||
proprietary_codecs = true
|
||||
ffmpeg_branding = "Chrome"
|
||||
use_remoteexec = false
|
||||
use_siso = true
|
||||
treat_warnings_as_errors = false
|
||||
A
|
||||
stage "gn gen"
|
||||
gn gen out/XR
|
||||
gn args out/XR --list=enable_openxr --short | tee -a "$W/stage"
|
||||
stage "build"
|
||||
( while sleep 600; do guard || { pkill -u "$(id -u)" -f "siso|ninja"; exit 3; }; done ) &
|
||||
GUARD=$!
|
||||
trap 'kill $GUARD 2>/dev/null || true' EXIT
|
||||
autoninja -C out/XR chrome chrome_sandbox chrome_crashpad_handler
|
||||
stage "package"
|
||||
cd out/XR
|
||||
files=(chrome chrome_sandbox chrome_crashpad_handler *.pak *.bin icudtl.dat locales)
|
||||
# GPU libraries aren't produced by every config; pack the ones that exist.
|
||||
for f in libEGL.so libGLESv2.so libvk_swiftshader.so libvulkan.so.1 vk_swiftshader_icd.json; do
|
||||
[ -e "$f" ] && files+=("$f")
|
||||
done
|
||||
tar -cJf "$W/chromium-xr-arm64.tar.xz" "${files[@]}"
|
||||
stage "DONE $(ls -la $W/chromium-xr-arm64.tar.xz)"
|
||||
Executable
+98
@@ -0,0 +1,98 @@
|
||||
#!/usr/bin/env zsh
|
||||
# Mac-side: install and launch the WebXR-enabled Chromium build on the Frame.
|
||||
#
|
||||
# Flathub Chromium can't enter immersive WebXR on Linux: upstream only wires
|
||||
# the OpenXR device on Windows (see docs/webxr-chromium.md). This deploys an
|
||||
# arm64 build with the Linux OpenXR CLs, made on a Linux host by
|
||||
# scripts/build-chromium-xr.sh, into ~/chromium-xr on the Frame (not a
|
||||
# Flatpak, so SteamVR's sockets and the XR sandbox work unmodified).
|
||||
#
|
||||
# Usage:
|
||||
# scripts/chromium-xr.sh install [TARBALL] # default: scp from $BUILD_HOST
|
||||
# scripts/chromium-xr.sh launch [URL] # opens in the headset desktop
|
||||
# scripts/chromium-xr.sh check # isSessionSupported via DevTools
|
||||
set -euo pipefail
|
||||
|
||||
FRAME_ALIAS=${FRAME_ALIAS:-frame}
|
||||
BUILD_HOST=${BUILD_HOST:-}
|
||||
BUILD_TARBALL=${BUILD_TARBALL:-chromium-xr/chromium-xr-arm64.tar.xz}
|
||||
DEVTOOLS_PORT=${DEVTOOLS_PORT:-9223}
|
||||
here=${0:A:h}
|
||||
|
||||
case "${1:-}" in
|
||||
install)
|
||||
tarball=${2:-}
|
||||
if [[ -z "$tarball" ]]; then
|
||||
[[ -n "$BUILD_HOST" ]] || { print -u2 "Pass a tarball, or set BUILD_HOST to the build machine"; exit 2; }
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
tarball=$tmp/chromium-xr-arm64.tar.xz
|
||||
scp -q "$BUILD_HOST:$BUILD_TARBALL" "$tarball"
|
||||
fi
|
||||
ssh "$FRAME_ALIAS" 'rm -rf ~/chromium-xr.new && mkdir -p ~/chromium-xr.new'
|
||||
ssh "$FRAME_ALIAS" 'tar -xJf - -C ~/chromium-xr.new' < "$tarball"
|
||||
# Check the new build runs before replacing the old one.
|
||||
ssh "$FRAME_ALIAS" '~/chromium-xr.new/chrome --version && rm -rf ~/chromium-xr && mv ~/chromium-xr.new ~/chromium-xr'
|
||||
;;
|
||||
launch)
|
||||
# run-on-frame starts in $HOME on the Frame, so the profile path is relative.
|
||||
exec "$here/run-on-frame.sh" -- '~/chromium-xr/chrome' \
|
||||
--user-data-dir=.config/chromium-xr \
|
||||
--enable-features=OpenXR \
|
||||
--remote-debugging-port="$DEVTOOLS_PORT" \
|
||||
"${2:-https://immersive-web.github.io/webxr-samples/}"
|
||||
;;
|
||||
check)
|
||||
# DevTools listens on the Frame's loopback only; evaluate there.
|
||||
ssh "$FRAME_ALIAS" python3 - "$DEVTOOLS_PORT" <<'EOF'
|
||||
import json, sys, urllib.request, base64, os, socket, struct
|
||||
port = int(sys.argv[1])
|
||||
tabs = json.load(urllib.request.urlopen(f"http://127.0.0.1:{port}/json", timeout=10))
|
||||
page = next((t for t in tabs if t["type"] == "page"), None)
|
||||
if page is None:
|
||||
sys.exit("no open page: run 'chromium-xr.sh launch' first")
|
||||
path = page["webSocketDebuggerUrl"].split(f":{port}", 1)[1]
|
||||
s = socket.create_connection(("127.0.0.1", port), timeout=30)
|
||||
key = base64.b64encode(os.urandom(16)).decode()
|
||||
s.sendall(f"GET {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nUpgrade: websocket\r\n"
|
||||
f"Connection: Upgrade\r\nSec-WebSocket-Key: {key}\r\n"
|
||||
"Sec-WebSocket-Version: 13\r\n\r\n".encode())
|
||||
s.recv(4096)
|
||||
msg = json.dumps({"id": 1, "method": "Runtime.evaluate", "params": {
|
||||
"expression": "navigator.xr ? navigator.xr.isSessionSupported('immersive-vr') : 'no navigator.xr'",
|
||||
"awaitPromise": True}}).encode()
|
||||
mask = os.urandom(4)
|
||||
hdr = bytes([0x81]) + (bytes([0x80 | len(msg)]) if len(msg) < 126
|
||||
else bytes([0x80 | 126]) + struct.pack(">H", len(msg)))
|
||||
s.sendall(hdr + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(msg)))
|
||||
buf = b""
|
||||
reply = None
|
||||
while reply is None:
|
||||
chunk = s.recv(65536)
|
||||
if not chunk:
|
||||
sys.exit("DevTools closed the connection")
|
||||
buf += chunk
|
||||
# Consume every complete frame already buffered before reading again.
|
||||
while len(buf) >= 2:
|
||||
n = buf[1] & 0x7F
|
||||
off = 2
|
||||
if n == 126:
|
||||
if len(buf) < 4:
|
||||
break
|
||||
n, off = struct.unpack(">H", buf[2:4])[0], 4
|
||||
elif n == 127:
|
||||
if len(buf) < 10:
|
||||
break
|
||||
n, off = struct.unpack(">Q", buf[2:10])[0], 10
|
||||
if len(buf) < off + n:
|
||||
break
|
||||
frame, buf = buf[off:off + n], buf[off + n:]
|
||||
msg = json.loads(frame)
|
||||
if msg.get("id") == 1:
|
||||
reply = msg
|
||||
break
|
||||
print("immersive-vr supported:", reply["result"]["result"].get("value"))
|
||||
EOF
|
||||
;;
|
||||
*) sed -n '2,13p' "$0"; exit 2 ;;
|
||||
esac
|
||||
@@ -4,9 +4,9 @@
|
||||
#
|
||||
# Exports every report through the app's own key (ui/frame_compat_db.py), keeps
|
||||
# dated copies in ~/Library/Application Support/Frame Control/compat-db/backups (newest
|
||||
# 60), and uploads to Google Drive (the backup folder) when
|
||||
# the data changed since the last upload. Run daily by the LaunchAgent
|
||||
# frame-compat-backup (see docs/apks.md).
|
||||
# 60), and uploads to the Google Drive folder DRIVE_FOLDER_ID when the data
|
||||
# changed since the last upload. Maintainer-only: it needs the database key.
|
||||
# Run it daily from a LaunchAgent (see compat-db/README.md).
|
||||
#
|
||||
# Usage: scripts/compat-db-backup.sh [--no-upload] [--force-upload] [--accept-shrink]
|
||||
# Env: DRIVE_FOLDER_ID, GOG_WRAPPER
|
||||
@@ -14,8 +14,8 @@ set -euo pipefail
|
||||
|
||||
ROOT="${0:A:h}/.."
|
||||
DEST="$HOME/Library/Application Support/Frame Control/compat-db/backups"
|
||||
DRIVE_FOLDER_ID=${DRIVE_FOLDER_ID:-<drive-folder-id>}
|
||||
GOG_WRAPPER=${GOG_WRAPPER:-$HOME/bin/gog-with-keyring.sh}
|
||||
DRIVE_FOLDER_ID=${DRIVE_FOLDER_ID:-}
|
||||
GOG_WRAPPER=${GOG_WRAPPER:-$(command -v gog || true)}
|
||||
upload=1 force=0 accept_shrink=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
@@ -60,9 +60,10 @@ if (( upload )); then
|
||||
print "==> Unchanged since the last Drive upload; skipped"
|
||||
exit 0
|
||||
fi
|
||||
[[ -x "$GOG_WRAPPER" ]] || { print -u2 "gog wrapper not found at $GOG_WRAPPER"; exit 1; }
|
||||
[[ -n "$DRIVE_FOLDER_ID" ]] || { print -u2 "Set DRIVE_FOLDER_ID, or pass --no-upload"; exit 1; }
|
||||
[[ -n "$GOG_WRAPPER" && -x "$GOG_WRAPPER" ]] || { print -u2 "gog not found; install it or set GOG_WRAPPER"; exit 1; }
|
||||
"$GOG_WRAPPER" drive upload "$out" --parent "$DRIVE_FOLDER_ID" --json --no-input >/dev/null
|
||||
"$GOG_WRAPPER" drive upload "$out.sha256" --parent "$DRIVE_FOLDER_ID" --json --no-input >/dev/null
|
||||
print -r -- "$digest" > "$last"
|
||||
print "==> Uploaded to Google Drive (the backup folder)"
|
||||
print "==> Uploaded to Google Drive"
|
||||
fi
|
||||
@@ -0,0 +1,51 @@
|
||||
"""Compatibility reports without the maintainer's key: saved locally, never sent.
|
||||
|
||||
Run: python3 -m unittest discover -s tests
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT / "ui"))
|
||||
|
||||
import frame_compat_db as db # noqa: E402
|
||||
|
||||
|
||||
class NoKey(unittest.TestCase):
|
||||
def setUp(self):
|
||||
tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(tmp.cleanup)
|
||||
state = tmp.name
|
||||
for name, value in (("STATE", state), ("OUTBOX", os.path.join(state, "outbox.jsonl")),
|
||||
("MIRROR", os.path.join(state, "mirror.json"))):
|
||||
p = mock.patch.object(db, name, value)
|
||||
p.start()
|
||||
self.addCleanup(p.stop)
|
||||
db._mem.update(at=0, reports=None, source=None)
|
||||
env = mock.patch.dict(os.environ, {}, clear=False)
|
||||
env.start()
|
||||
self.addCleanup(env.stop)
|
||||
os.environ.pop("FRAME_CONTROL_KEY", None)
|
||||
# No Keychain entry, and any network use fails the test.
|
||||
no_key = mock.patch.object(db.subprocess, "run",
|
||||
return_value=mock.Mock(returncode=44, stdout=""))
|
||||
no_key.start()
|
||||
self.addCleanup(no_key.stop)
|
||||
net = mock.patch.object(db._opener, "open", side_effect=AssertionError("network used"))
|
||||
net.start()
|
||||
self.addCleanup(net.stop)
|
||||
|
||||
def test_report_is_kept_locally(self):
|
||||
self.assertFalse(db.shared())
|
||||
r = db.add({"package": "org.example.app", "date": "2026-09-26T10:00:00", "rating": "works"})
|
||||
reports = db.load()
|
||||
self.assertEqual([x["id"] for x in reports], [r["id"]])
|
||||
self.assertEqual(db._mem["source"], "mirror")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -83,6 +83,7 @@ class ServerGuards(unittest.TestCase):
|
||||
self.assertEqual(self.request("GET", "/api/status")[0], 403)
|
||||
self.assertEqual(self.request("GET", "/api/screenshot?view=headset")[0], 403)
|
||||
self.assertEqual(self.request("GET", "/api/shots")[0], 403)
|
||||
self.assertEqual(self.request("GET", "/api/stream")[0], 403)
|
||||
self.assertEqual(self.request("GET", "/api/shots/image?id=1/250820/20260925225208_1.jpg")[0], 403)
|
||||
self.assertEqual(self.request("POST", "/api/launch", {"appid": "620"})[0], 403)
|
||||
|
||||
@@ -116,6 +117,11 @@ class ServerGuards(unittest.TestCase):
|
||||
status, _, _ = self.request("GET", f"/api/shots/image?id={quote(shot)}", headers={"X-Frame-UI": "1"})
|
||||
self.assertEqual(status, 400, shot)
|
||||
|
||||
def test_stream_settings_checked_before_ssh(self):
|
||||
for query in ("h=480", "fps=24", "h=abc", "h=1080&fps=120"):
|
||||
status, _, _ = self.request("GET", f"/api/stream?{query}", headers={"X-Frame-UI": "1"})
|
||||
self.assertEqual(status, 400, query)
|
||||
|
||||
def test_bad_bodies(self):
|
||||
conn = http.client.HTTPConnection("127.0.0.1", self.port, timeout=10)
|
||||
conn.request("POST", "/api/launch", body=b"{not json", headers={"X-Frame-UI": "1"})
|
||||
|
||||
@@ -40,6 +40,16 @@ def key():
|
||||
return p.stdout.strip()
|
||||
|
||||
|
||||
def shared():
|
||||
"""Whether reports reach the shared database. Without the key (anyone but the
|
||||
maintainer), reports stay in this Mac's outbox and ratings come from the catalogue."""
|
||||
try:
|
||||
key()
|
||||
return True
|
||||
except DBError:
|
||||
return False
|
||||
|
||||
|
||||
class _NoRedirect(urllib.request.HTTPRedirectHandler):
|
||||
"""Never follow redirects: urllib would copy the key header to the new host."""
|
||||
def redirect_request(self, *args, **kwargs):
|
||||
@@ -169,6 +179,8 @@ def load():
|
||||
now = time.time()
|
||||
if _mem['reports'] is None or now - _mem['at'] > TTL:
|
||||
try:
|
||||
if not shared():
|
||||
raise DBError('no key')
|
||||
try:
|
||||
flush()
|
||||
except Exception:
|
||||
@@ -196,6 +208,7 @@ def add(report):
|
||||
with _lock, open(OUTBOX, 'a') as f:
|
||||
f.write(json.dumps(r, ensure_ascii=False) + '\n')
|
||||
try:
|
||||
if shared():
|
||||
flush()
|
||||
_mem['at'] = 0 # refetch on next load
|
||||
except Exception:
|
||||
|
||||
+122
-8
@@ -321,7 +321,7 @@
|
||||
</div>
|
||||
<div class="spacer"></div>
|
||||
<button class="action" id="shotBtn">Capture</button>
|
||||
<button id="liveBtn" title="Keep capturing">Live</button>
|
||||
<button id="liveBtn" title="Keep updating: video of the headset view, or repeated captures of the desktop panel">Live</button>
|
||||
<button id="saveBtn" disabled>Save</button>
|
||||
</div>
|
||||
<div class="viewer" id="viewer">
|
||||
@@ -434,8 +434,8 @@
|
||||
<div class="panel">
|
||||
<div class="shelf-head"><h2>Recent reports</h2><span class="count" id="repCount"></span></div>
|
||||
<div class="list" id="repList"><div class="sub">Loading…</div></div>
|
||||
<div class="hint">Reports go to Frame Control's private compatibility database and change the
|
||||
verdicts in the catalogue. Any APK can be reported, including ones not on F-Droid.</div>
|
||||
<div class="hint" id="repHint">Reports change the verdicts in the catalogue. Any APK can be
|
||||
reported, including ones not on F-Droid.</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="panel">
|
||||
@@ -575,7 +575,7 @@ const QUICK = [["Remmina", "org.remmina.Remmina"], ["Moonlight", "com.moonlight_
|
||||
["Firefox", "org.mozilla.firefox"], ["VLC", "org.videolan.VLC"]];
|
||||
const CDN = "https://cdn.cloudflare.steamstatic.com/steam/apps";
|
||||
const HINTS = {
|
||||
headset: "What the lenses show, composited by SteamVR: the room, floating panels, dashboard and controllers. Live refreshes about twice a second. Captures show everything on screen, including anything private.",
|
||||
headset: "What the lenses show, composited by SteamVR: the room, floating panels, dashboard and controllers. Live streams it as video (one eye, about 30 fps); Capture takes a still of both eyes. Captures show everything on screen, including anything private.",
|
||||
flat: "gamescope's 2D layer: the desktop panel and Steam's flat UI, without the room or VR scene.",
|
||||
};
|
||||
const SOURCE_LABEL = { steamvr: "Headset view", gamescope: "Desktop panel", shot: "Screenshot" };
|
||||
@@ -751,10 +751,12 @@ function render(s) {
|
||||
|
||||
// ---- view ----
|
||||
function setView(v) {
|
||||
const changed = v !== view;
|
||||
view = v;
|
||||
document.querySelectorAll("[data-view]").forEach(b => b.classList.toggle("on", b.dataset.view === v));
|
||||
$("eyeSeg").style.visibility = v === "headset" ? "visible" : "hidden";
|
||||
$("eyeSeg").style.visibility = v === "headset" && !video.ctl ? "visible" : "hidden";
|
||||
$("viewHint").textContent = HINTS[v];
|
||||
if (live && changed) { toggleLive(false); toggleLive(true); } // video for the headset, captures for the panel
|
||||
}
|
||||
function setEye(e) {
|
||||
eye = e;
|
||||
@@ -841,6 +843,7 @@ function isBlank(ctx, w, h) {
|
||||
return max - min < 6;
|
||||
}
|
||||
async function capture() {
|
||||
if (video.ctl) toggleLive(false); // Capture during live video takes a still of both eyes instead
|
||||
const gen = viewGen; // a screenshot opened meanwhile wins over this capture
|
||||
if (!live) $("viewer").classList.add("busy"); // no spinner flashing over a live stream
|
||||
let url = null;
|
||||
@@ -883,7 +886,115 @@ function toggleLive(on) {
|
||||
liveFailures = 0;
|
||||
$("liveBtn").classList.toggle("on", live);
|
||||
$("liveBadge").hidden = !live;
|
||||
if (live) liveLoop(); else clearTimeout(liveTimer);
|
||||
clearTimeout(liveTimer);
|
||||
stopVideo();
|
||||
if (!live) return;
|
||||
if (view === "headset" && "VideoDecoder" in window) {
|
||||
const started = startVideo();
|
||||
const ctl = video.ctl;
|
||||
started.catch(e => {
|
||||
if (!live || ctl !== video.ctl || e.name === "AbortError") return;
|
||||
log("Live video failed: " + e.message, "e");
|
||||
toast("Live video failed, using captures instead: " + e.message, true);
|
||||
stopVideo();
|
||||
liveLoop();
|
||||
});
|
||||
} else liveLoop();
|
||||
}
|
||||
|
||||
// ---- live video of the headset view ----
|
||||
// The server relays raw H.264 (Annex B) from SteamVR's headset-view device.
|
||||
// Every frame starts with an access unit delimiter (NAL type 9), which is how
|
||||
// the stream is cut into frames for WebCodecs.
|
||||
const STREAM_QUERY = "h=720&fps=30", STREAM_FPS = 30;
|
||||
const video = { ctl: null, dec: null, gen: 0 };
|
||||
function startCode(b, i) { return b[i] === 0 && b[i + 1] === 0 && b[i + 2] === 1; }
|
||||
function nalTypes(au) {
|
||||
const types = [];
|
||||
for (let i = 0; i + 3 < au.length; i++) if (startCode(au, i)) { types.push(au[i + 3] & 0x1f); i += 3; }
|
||||
return types;
|
||||
}
|
||||
async function startVideo() {
|
||||
const gen = ++video.gen, ctl = new AbortController();
|
||||
video.ctl = ctl;
|
||||
$("eyeSeg").style.visibility = "hidden";
|
||||
if (!lastImg) $("viewer").classList.add("busy");
|
||||
let frames = 0, shown = 0, second = performance.now(), needKey = true, ts = 0;
|
||||
const c = $("canvas"), ctx = c.getContext("2d");
|
||||
const dec = video.dec = new VideoDecoder({
|
||||
output: frame => {
|
||||
if (gen !== video.gen) return frame.close();
|
||||
if (c.width !== frame.displayWidth || c.height !== frame.displayHeight) {
|
||||
c.width = frame.displayWidth; c.height = frame.displayHeight;
|
||||
$("viewer").style.aspectRatio = `${c.width} / ${c.height}`;
|
||||
setZoom(1);
|
||||
}
|
||||
ctx.drawImage(frame, 0, 0);
|
||||
frame.close();
|
||||
if (!shown++) {
|
||||
viewGen++; // a capture still in flight mustn't replace the video
|
||||
lastImg = null; lastSource = "video"; lastShot = null;
|
||||
$("viewer").classList.remove("busy");
|
||||
c.hidden = false; $("viewerEmpty").hidden = true; $("zoombar").hidden = false; $("asleep").hidden = true;
|
||||
$("srcBadge").hidden = false; $("srcBadge").textContent = "Headset view · video";
|
||||
$("stamp").hidden = false; $("saveBtn").disabled = false;
|
||||
}
|
||||
frames++;
|
||||
const now = performance.now();
|
||||
if (now - second >= 1000) {
|
||||
$("stamp").textContent = `${Math.round(frames * 1000 / (now - second))} fps`;
|
||||
frames = 0; second = now;
|
||||
}
|
||||
},
|
||||
error: e => log("Video decoder: " + e.message, "e"),
|
||||
});
|
||||
const r = await fetch(`/api/stream?${STREAM_QUERY}`, { headers: {"X-Frame-UI": "1"}, signal: ctl.signal });
|
||||
if (!r.ok) throw new Error((await r.json().catch(() => ({}))).error || `HTTP ${r.status}`);
|
||||
const reader = r.body.getReader();
|
||||
let buf = new Uint8Array(0), scan = 0, auStart = -1;
|
||||
const onAU = au => {
|
||||
const types = nalTypes(au), key = types.includes(5);
|
||||
if (dec.state === "unconfigured") {
|
||||
const sps = types.indexOf(7);
|
||||
if (sps < 0) return;
|
||||
let i = 0, n = -1; // find the SPS bytes: profile, constraints, level follow its header
|
||||
for (; i + 3 < au.length; i++) if (startCode(au, i) && ++n === sps) break;
|
||||
if (i + 6 >= au.length) return;
|
||||
const hex = [au[i + 4], au[i + 5], au[i + 6]].map(b => b.toString(16).padStart(2, "0")).join("");
|
||||
dec.configure({ codec: `avc1.${hex}`, optimizeForLatency: true });
|
||||
}
|
||||
// If decoding falls behind, drop frames until the next keyframe rather than lag.
|
||||
if (dec.decodeQueueSize > 3) needKey = true;
|
||||
if (needKey && !key) return;
|
||||
needKey = false;
|
||||
dec.decode(new EncodedVideoChunk({ type: key ? "key" : "delta", timestamp: ts, data: au }));
|
||||
ts += 1e6 / STREAM_FPS;
|
||||
};
|
||||
for (;;) {
|
||||
const { value, done } = await reader.read();
|
||||
if (done || gen !== video.gen) break;
|
||||
const next = new Uint8Array(buf.length + value.length);
|
||||
next.set(buf); next.set(value, buf.length);
|
||||
buf = next;
|
||||
if (buf.length > 8 << 20) throw new Error("the stream isn't split into frames");
|
||||
for (; scan + 3 < buf.length; scan++) {
|
||||
if (!startCode(buf, scan) || (buf[scan + 3] & 0x1f) !== 9) continue;
|
||||
if (auStart >= 0) onAU(buf.subarray(auStart, scan));
|
||||
auStart = scan;
|
||||
scan += 3;
|
||||
}
|
||||
if (auStart > 0) { buf = buf.slice(auStart); scan -= auStart; auStart = 0; }
|
||||
}
|
||||
if (gen === video.gen && live) throw new Error(shown ? "the stream ended" : "no video arrived");
|
||||
}
|
||||
function stopVideo() {
|
||||
video.gen++;
|
||||
if (video.ctl) video.ctl.abort();
|
||||
if (video.dec && video.dec.state !== "closed") video.dec.close();
|
||||
video.ctl = video.dec = null;
|
||||
$("viewer").classList.remove("busy");
|
||||
if (lastSource === "video") { $("srcBadge").textContent = "Headset view · video (stopped)"; $("stamp").hidden = true; }
|
||||
$("eyeSeg").style.visibility = view === "headset" ? "visible" : "hidden";
|
||||
}
|
||||
$("shotBtn").onclick = () => capture();
|
||||
$("liveBtn").onclick = () => toggleLive(!live);
|
||||
@@ -1345,9 +1456,12 @@ const RLABEL = { works: "Works", issues: "Problems", broken: "Doesn't work", run
|
||||
crashes: "Crashed (test)", install_failed: "Won't install", instance_failed: "Didn't start (test)" };
|
||||
const RCLASS = { works: "works", runs: "works", issues: "maybe" };
|
||||
async function loadReports() {
|
||||
let reps;
|
||||
try { reps = (await api("/api/android/reports")).reports; }
|
||||
let reps, shared;
|
||||
try { ({ reports: reps, shared } = await api("/api/android/reports")); }
|
||||
catch (e) { $("repList").innerHTML = `<div class="sub">${esc(e.message)}</div>`; return; }
|
||||
$("repHint").textContent = shared
|
||||
? "Reports go to Frame Control's shared compatibility database and change the verdicts in the catalogue. Any APK can be reported, including ones not on F-Droid."
|
||||
: "Reports are saved on this Mac and change the verdicts you see. They aren't uploaded: the shared database is maintainer-only for now. Any APK can be reported, including ones not on F-Droid.";
|
||||
$("repCount").textContent = reps.length ? `${reps.length} newest` : "";
|
||||
$("repList").innerHTML = reps.length ? reps.slice(0, 40).map(r => {
|
||||
const k = r.rating || r.result;
|
||||
|
||||
+106
-4
@@ -12,6 +12,7 @@ import http.client
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import select
|
||||
import shlex
|
||||
import shutil
|
||||
import signal
|
||||
@@ -272,6 +273,41 @@ def save_shots(body):
|
||||
return {"message": msg + (f" ({skipped} already there)" if skipped else ""), "saved": n}
|
||||
|
||||
|
||||
# Live video of the headset view. SteamVR's steamvr-v4l2cam.service copies the
|
||||
# headset view (one undistorted 1920x1080 image) into the v4l2loopback device
|
||||
# /dev/video99. ffmpeg encodes it with x264 (the hardware encoder crashes
|
||||
# ffmpeg) and the raw H.264 comes back over SSH for the page to decode with
|
||||
# WebCodecs. An access unit delimiter starts every frame so the page can split
|
||||
# the stream, and repeated SPS/PPS let it start at any keyframe. ffmpeg runs in
|
||||
# the background while the shell waits for our stdin to close: when the local
|
||||
# ssh goes, the channel closes and the shell kills ffmpeg, even one that has
|
||||
# stopped writing (and so would never get SIGPIPE).
|
||||
STREAM_DEVICE = "/dev/video99"
|
||||
STREAM_HEIGHTS = (720, 1080)
|
||||
STREAM_FPS = (30, 60)
|
||||
STREAM_STALL = 10 # seconds without video before the stream is dropped
|
||||
_stream_lock = threading.Lock()
|
||||
_stream_proc = None
|
||||
|
||||
|
||||
def stream_command(query):
|
||||
q = parse_qs(query)
|
||||
try:
|
||||
height = int((q.get("h") or ["720"])[0])
|
||||
fps = int((q.get("fps") or ["30"])[0])
|
||||
except ValueError:
|
||||
raise Failure("h and fps must be integers", 400)
|
||||
if height not in STREAM_HEIGHTS or fps not in STREAM_FPS:
|
||||
raise Failure(f"h must be one of {STREAM_HEIGHTS} and fps one of {STREAM_FPS}", 400)
|
||||
rate = 3 if height == 720 else 6 # Mbit/s
|
||||
return (f"[ -e {STREAM_DEVICE} ] || {{ echo 'No headset view device ({STREAM_DEVICE}). Is SteamVR running?' >&2; exit 3; }}; "
|
||||
f"ffmpeg -hide_banner -loglevel error -nostdin -f v4l2 -video_size 1920x1080 -i {STREAM_DEVICE} "
|
||||
f"-vf fps={fps},scale=-2:{height},format=yuv420p -c:v libx264 -preset ultrafast -tune zerolatency "
|
||||
f"-g {fps * 2} -bf 0 -b:v {rate}M -maxrate {rate}M -bufsize {rate // 2 or 1}M "
|
||||
f"-x264-params aud=1:repeat-headers=1 -f h264 - & p=$!; "
|
||||
f"exec >&-; cat >/dev/null; kill $p 2>/dev/null; wait $p")
|
||||
|
||||
|
||||
def launch(body):
|
||||
appid = str(body.get("appid", ""))
|
||||
if not APPID.match(appid):
|
||||
@@ -402,7 +438,8 @@ def android(body):
|
||||
runtime=body.get("runtime") or "instance",
|
||||
label=body.get("label"), source=body.get("source"))
|
||||
name = r.get("label") or pkg
|
||||
return {"message": f"Saved your report for {name}", "report": r}
|
||||
where = "" if frame_catalog.compat_db.shared() else " on this Mac"
|
||||
return {"message": f"Saved your report for {name}{where}", "report": r}
|
||||
except frame_android.FrameError as e:
|
||||
raise Failure(str(e))
|
||||
raise Failure("unknown action", 400)
|
||||
@@ -423,7 +460,7 @@ FONT_RANGE = (0.5, 2.0)
|
||||
KNOWN_LABELS = {"com.t3tools.t3code": "T3 Code", "org.fdroid.fdroid": "F-Droid"}
|
||||
# One ADB session at a time: requests are rare, and it keeps adb's state simple.
|
||||
_adb_lock = threading.Lock()
|
||||
_live_tunnels = set() # ssh processes to kill if the server stops mid-request
|
||||
_live_tunnels = set() # ssh processes (ADB forwards, live video) to kill if the server stops mid-request
|
||||
|
||||
|
||||
def adb_path():
|
||||
@@ -757,7 +794,8 @@ class Handler(BaseHTTPRequestHandler):
|
||||
elif path == "/api/android/displays":
|
||||
self.send_json(android_displays())
|
||||
elif path == "/api/android/reports":
|
||||
self.send_json({"reports": frame_catalog.recent_reports()})
|
||||
self.send_json({"reports": frame_catalog.recent_reports(),
|
||||
"shared": frame_catalog.compat_db.shared()})
|
||||
elif path == "/api/android/catalog":
|
||||
self.send_json({"apps": frame_catalog.catalog()})
|
||||
elif path == "/api/status":
|
||||
@@ -770,6 +808,8 @@ class Handler(BaseHTTPRequestHandler):
|
||||
self.send_json(list_shots())
|
||||
elif path == "/api/shots/image":
|
||||
self.send_bytes(*shot_image(url.query))
|
||||
elif path == "/api/stream":
|
||||
self.stream_video(url.query)
|
||||
elif path == "/api/screenshot" and parse_qs(url.query).get("view") == ["headset"]:
|
||||
self.send_bytes(headset_view(), "image/png", headers=[("X-Capture-Source", "steamvr")])
|
||||
elif path == "/api/screenshot":
|
||||
@@ -779,6 +819,8 @@ class Handler(BaseHTTPRequestHandler):
|
||||
self.send_json({"error": "not found"}, 404)
|
||||
except Failure as e:
|
||||
self.send_json({"error": str(e)}, e.status)
|
||||
except frame_android.FrameError as e:
|
||||
self.send_json({"error": str(e)}, 502)
|
||||
except Exception as e:
|
||||
self.send_json({"error": f"{type(e).__name__}: {e}"}, 500)
|
||||
|
||||
@@ -805,9 +847,69 @@ class Handler(BaseHTTPRequestHandler):
|
||||
self.send_json({"error": str(e)}, e.status)
|
||||
except (ValueError, TypeError) as e:
|
||||
self.send_json({"error": f"bad request: {e}"}, 400)
|
||||
except frame_android.FrameError as e:
|
||||
self.send_json({"error": str(e)}, 502)
|
||||
except Exception as e:
|
||||
self.send_json({"error": f"{type(e).__name__}: {e}"}, 500)
|
||||
|
||||
def stream_video(self, query):
|
||||
"""Raw H.264 of the headset view until the page disconnects (see stream_command)."""
|
||||
global _stream_proc
|
||||
remote = stream_command(query)
|
||||
ensure_master()
|
||||
# stderr goes to a file: nothing reads it while streaming, and a full
|
||||
# pipe would stall ffmpeg. It's only read if the stream fails to start.
|
||||
errors = tempfile.TemporaryFile()
|
||||
proc = subprocess.Popen([*SSH, FRAME, remote], stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE, stderr=errors)
|
||||
try:
|
||||
_live_tunnels.add(proc)
|
||||
# One viewer at a time: a new stream (another tab, a reload) ends the last one.
|
||||
with _stream_lock:
|
||||
old, _stream_proc = _stream_proc, proc
|
||||
if old and old.poll() is None:
|
||||
old.terminate()
|
||||
fd = proc.stdout.fileno()
|
||||
# Nothing is sent until the first bytes arrive, so a failure to
|
||||
# start still comes back as a JSON error.
|
||||
ready, _, _ = select.select([fd], [], [], 20)
|
||||
first = os.read(fd, 1 << 16) if ready else b""
|
||||
if not first:
|
||||
proc.kill()
|
||||
proc.wait()
|
||||
errors.seek(0)
|
||||
err = strip_ansi(errors.read().decode(errors="replace")).strip()
|
||||
raise Failure(err or "The headset view sent no video for 20 s")
|
||||
chunk = first
|
||||
try:
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "video/h264")
|
||||
self.send_header("Cache-Control", "no-store")
|
||||
self.send_header("X-Frame-Options", "DENY")
|
||||
self.send_header("Content-Security-Policy", "frame-ancestors 'none'")
|
||||
self.end_headers()
|
||||
self.close_connection = True # the body ends when the connection does
|
||||
while chunk:
|
||||
self.wfile.write(chunk)
|
||||
self.wfile.flush()
|
||||
# A stalled headset view ends the stream rather than
|
||||
# holding this thread (and the page) forever.
|
||||
ready, _, _ = select.select([fd], [], [], STREAM_STALL)
|
||||
chunk = os.read(fd, 1 << 16) if ready else b""
|
||||
except OSError:
|
||||
pass # the page stopped watching (or stopped reading); the body has started, so no JSON
|
||||
finally:
|
||||
if proc.poll() is None:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
proc.wait()
|
||||
for f in (proc.stdin, proc.stdout, errors):
|
||||
f.close()
|
||||
_live_tunnels.discard(proc)
|
||||
|
||||
def upload(self):
|
||||
"""Raw file body. X-Filename names it; X-Mode is 'push', 'apk' (install) or 'apkinfo' (read only)."""
|
||||
name = os.path.basename(unquote(self.headers.get("X-Filename", "")))
|
||||
@@ -871,7 +973,7 @@ def main():
|
||||
subprocess.run([*MUX, "-O", "exit", FRAME], capture_output=True)
|
||||
if _master and _master.poll() is None:
|
||||
_master.terminate()
|
||||
for proc in list(_live_tunnels): # ADB forwards of requests cut off mid-way
|
||||
for proc in list(_live_tunnels): # ADB forwards and video streams cut off mid-way
|
||||
if proc.poll() is None:
|
||||
proc.terminate()
|
||||
|
||||
|
||||
Reference in new issue
Block a user