Merge pull request #5 from saphid/iphone-app

iPhone and iPad app: the same features, served from the Frame
This commit is contained in:
Alex Southwell authored and GitHub committed 2026-09-28 17:20:18 +10:00
commit 03322d3166
89 files changed
+8409 -75

No files matched your search

+9 -1
View File
@@ -51,7 +51,13 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600
| Tailscale runs without root as a userspace `tailscaled` user service (static arm64 build in `~/.local/share/tailscale`, lingering on). In userspace mode, inbound tailnet connections reach the Frame's **loopback**, so every port, including DevTools on 8080, is reachable from the tailnet. **Verified 2026-09-25.** | [tailscale.md](tailscale.md), `scripts/tailscale-on-frame.sh` |
| **T3 Code desktop runs natively.** The stock release `T3-Code-0.0.42-arm64.AppImage` in `~/Applications/T3CodeDesktop/` starts with no extra setup: glibc 2.39, `libfuse.so.2`, GTK 3, NSS and libsecret are on the image. `panel-on-frame.sh --name t3code-desktop -- '~/Applications/T3CodeDesktop/T3-Code.AppImage'` gives it its own panel (`valve.steam.desktopgame.2000281357`, `--ozone-platform=x11`). Its bundled server listens on `127.0.0.1:3773` and shows up in onboarding as the `frame` computer, with `passwordStore: gnome-libsecret`. The image has no agent CLI and no `node`. Agents run through the LAN CLIProxyAPI (`llm-proxy.lan:8317`, which resolves on the Frame). Claude Code 2.1.283 comes from `claude.ai/install.sh`, and Codex 0.157.1 from the `codex-aarch64-unknown-linux-musl` release tarball, both into `~/.local/bin`. `with-cliproxy` and a mode-600 `~/.config/cliproxyapi/secrets.env` are copied from the Mac. The wrappers `claude-cliproxy` and `codex-cliproxy` (a `-c model_provider=cliproxy`, `wire_api="responses"`, `env_key="CLIPROXY_API_KEY"`) are set as `providers.claudeAgent.binaryPath` and `providers.codex.binaryPath` in `~/.t3/userdata/settings.json`, and T3 picked that up without a restart. Through the wrappers, `claude auth status` reports `loggedIn: true` (`oauth_token`), and both CLIs answered a prompt with `kimi-k3`. `gamescopectl screenshot` captured another layer (the Lepton T3 app) rather than this panel. `DISPLAY=:0 xwd -id <win>` piped to `ffmpeg` captures the window itself (1920×1080). **Verified 2026-09-26**, BUILD_ID 20260922.6101926. | Running T3 Code as a host on the Frame |
| Power actions need `sudo`, which asks for the Developer Mode password over SSH. | Frame Control's power buttons |
| **Boot / recovery menu.** Hold Power ~10 s until the LED goes off, then power on while holding the **AUX button on top of the Power button** (not the volume keys) until a text menu appears. Entries: `Current` (SteamOS-A/B + build), `Previous` (the other A/B slot), `Boot from USB`, `Repair Steam Installation`, `Erase User Data` (factory reset), `ADB mode`, `Battery Ship Mode`. It auto-boots `Current` after a ~15 s countdown. **Volume Up/Down (left side) move, AUX (right side) selects.** For a boot loop, Valve says pick `Previous` (keeps user data); then `Repair Steam Installation`; `Erase User Data` wipes `~` (SSH keys, Tailscale, Flatpaks, T3 setup). Last resort is a full re-image, two ways: (1) USB: write `steamframe-repair-latest.img.bz2` to an 8 GB+ USB-C stick (Balena Etcher on the Mac), pick `Boot from USB`, then use "Wipe Device & Install SteamOS" / "Repair SteamOS" (keeps games and personal content) from the recovery desktop; (2) cable/EDL: `steamframe-repair-qdl-latest.tar.gz`, run `flash.sh` (Linux) or `flash.cmd` (Windows), then with the Frame off for 10 s hold Power + Vol Up + Vol Down for 10 s and plug it in; it reflashes and reboots. Both images: `https://steamdeck-images.steamos.cloud/recovery/` (build 20260922.5153644, 0.3.0, ~4 GB each, no published checksums); local copies in `~/Downloads/steam-frame-recovery/`. Source: Valve's [SteamOS Recovery FAQ](https://help.steampowered.com/en/faqs/view/1B71-EDF2-EB6D-2BB3) and [Installation and Repair FAQ](https://help.steampowered.com/en/faqs/view/65B4-2AA3-5F37-4227), plus a menu photo in [EloiStree/HelloSteamFrame#9](https://github.com/EloiStree/HelloSteamFrame/issues/9). **Inferred** (Valve docs, 2026-09-26); not yet tried on our Frame. | Recovering from a boot loop |
| **SSH server:** OpenSSH 9.7p1. It offers `publickey,password` (keyboard-interactive is off, PAM on) and also asks `userdbctl ssh-authorized-keys` for keys. OpenSSH ≥ 8.8 rejects SHA-1 `ssh-rsa` signatures by default, so a client whose RSA support is SHA-1 only (the Swift library Citadel, for one) can't log in with the RSA key that devkit pairing installs; use ed25519 (**inferred** from OpenSSH defaults). **Verified 2026-09-27**, BUILD_ID 20260922.6101926. | [iphone.md](iphone.md), `ui/frame_connect.py` |
| **Tools on the image:** Python 3.12.3, `ffmpeg`, `openssl`, `curl`, `rsync`, `zip`/`unzip`, `flatpak`, `wpctl`, `podman`. **No `adb`.** `steamos` is uid 1000, in `wheel`, and sudoers has `%wheel ALL=(ALL) ALL`, so `sudo -S` takes the Developer Mode password on stdin. **Verified 2026-09-27.** | Running Frame Control's server on the Frame (`FRAME_LOCAL=1`, [iphone.md](iphone.md)) |
| **Each Lepton instance is a podman container** named `lepton-steamlaunch-<instance id>`, labelled with its ADB port (`podman ps --format '{{.Names}} {{.Labels.adb_port}}'`). `podman exec <container> /system/bin/sh -c '…'` runs Android's shell inside it with no adb at all (used for `pidof` and `logcat` by the app tester). Running `wm size`/`wm density` that way is untested. **Verified 2026-09-27.** | `ui/frame_android.py`, the iPhone app's display settings |
| **Asleep means off the network.** In standby the Frame stops answering on its LAN address, `frame.local` and Tailscale alike (`Host is down`, `No route to host`, timeouts), and ping fails. It was unreachable for about 2.5 hours until woken. Nothing over SSH can wake it. **Verified 2026-09-27.** | Frame Control's offline banner and retries |
| **Battery at full on a charger** can read `Discharging` at about 0 W (for example 99 %, 0.0 W, USB-C PD 18 W). Treat under 0.5 W on a charger as "not charging", not "draining". **Verified 2026-09-27.** | Frame Control's battery card |
| **The OS image is downloadable.** Valve's recovery images for the Frame are at `https://steamdeck-images.steamos.cloud/recovery/`. The root filesystem inside is btrfs, and it runs as an SSH test target on ARM64 Linux without the headset (`tests/frame-container/frame-image.sh`). **Verified 2026-09-27.** | [recovery-and-images.md](recovery-and-images.md) |
| **Boot / recovery menu.** Hold Power ~10 s until the LED goes off, then power on while holding the **AUX button on top of the Power button** (not the volume keys) until a text menu appears. Entries: `Current` (SteamOS-A/B + build), `Previous` (the other A/B slot), `Boot from USB`, `Repair Steam Installation`, `Erase User Data` (factory reset), `ADB mode`, `Battery Ship Mode`. It auto-boots `Current` after a ~15 s countdown. **Volume Up/Down (left side) move, AUX (right side) selects.** For a boot loop, Valve says pick `Previous` (keeps user data); then `Repair Steam Installation`; `Erase User Data` wipes `~` (SSH keys, Tailscale, Flatpaks, T3 setup). Last resort is a full re-image, two ways: (1) USB: write `steamframe-oobe-repair-<build>.img.bz2` to an 8 GB+ USB-C stick (Balena Etcher on the Mac), pick `Boot from USB`, then use "Wipe Device & Install SteamOS" / "Repair SteamOS" (keeps games and personal content) from the recovery desktop; (2) cable/EDL: `steamframe-oobe-repair-qdl-<build>.tar.gz`, run `flash.sh` (Linux) or `flash.cmd` (Windows), then with the Frame off for 10 s hold Power + Vol Up + Vol Down for 10 s and plug it in; it reflashes and reboots. Both images: `https://steamdeck-images.steamos.cloud/recovery/` (build 20260922.5153644, 0.3.0, 3.8 GiB each, no published checksums); local copies in `~/Downloads/steam-frame-recovery/`. File names, checksums and what's inside: [recovery-and-images.md](recovery-and-images.md). Source: Valve's [SteamOS Recovery FAQ](https://help.steampowered.com/en/faqs/view/1B71-EDF2-EB6D-2BB3) and [Installation and Repair FAQ](https://help.steampowered.com/en/faqs/view/65B4-2AA3-5F37-4227), plus a menu photo in [EloiStree/HelloSteamFrame#9](https://github.com/EloiStree/HelloSteamFrame/issues/9). **Inferred** (Valve docs, 2026-09-26); not yet tried on our Frame. | Recovering from a boot loop |
| **Boot loop cause: the SteamVR health check.** `steamvr.service` runs `/usr/share/deckard/steamvr-health-check`, which appends `frog:glasses:` to `$XDG_RUNTIME_DIR/steamvr-short-session-tracker` on every failed or <10 s SteamVR run. At 3 it runs `steam-health-check --repair-now`, which **deletes all of `~/.local/share/Steam` (games, login, Developer Mode) and `~/.steam`**, keeping only `registry.vdf`. At 4 it also tries `steamos-bootconf set-mode reboot-other` (fails as the user: `bootenv: Permission denied`). SteamVR normally fails 1–2 times per boot while it waits for the Steam client (`SteamAPI_InitEx failed … Steam is probably not running`, then `fatal stalled cross-thread pipe`). Once Steam has been wiped, it has to re-download a ~210 MB client on every boot, so SteamVR keeps failing, Steam keeps getting wiped and the Frame reboots, in a loop. Also, the Steam updater can deadlock at `Installing update...` (main process blocked writing to the `-child-update-ui` process, which is stuck in `drm_syncobj_array_wait_timeout`). Killing only the `-child-update-ui` process lets the install finish (`package/*.installed` appears). **Fix without sudo:** over USB-C ADB (`adb -s frame shell` works as `steamos` while the Frame is looping; SSH is refused once Developer Mode is lost), truncate both `/run/user/1000/steam{,vr}-short-session-tracker` files and `chmod 444` them (the health check then logs `Permission denied` and does nothing; this is tmpfs, so it resets on reboot). Unstick the updater if needed, let Steam finish installing, then hold Power 10 s and start the Frame normally. `systemctl reboot` over ADB needs interactive auth. After the fix, sign in to Steam and turn Developer Mode back on. **Verified 2026-09-26**, BUILD_ID 20260922.6101926, slot B (clean boot: 0 SteamVR failures, SSH and Tailscale back). | Diagnosing a boot loop |
## Debug recipes
@@ -79,4 +85,6 @@ ssh frame 'cat /opt/steamvr/resources/webinterface/dashboard/localization/dashbo
- Installing and buying Steam games: [steam-games.md](steam-games.md)
- Remote access from anywhere: [tailscale.md](tailscale.md)
- Floating windows in space: [panels.md](panels.md)
- Recovery images, what's in them, testing without the headset: [recovery-and-images.md](recovery-and-images.md)
- Frame Control on iPhone (the server running on the Frame itself): [iphone.md](iphone.md)
- What's still unverified: [open-questions.md](open-questions.md)
Binary file not shown.

After

Width:  |  Height:  |  Size: 305 KiB

+109
View File
@@ -0,0 +1,109 @@
# Frame Control for iPhone
The iPhone (and iPad) app does what the desktop app does, from the phone:
headset view and live video, battery and status, screenshots, Steam games,
Android apps and their display settings, sideloading, files, clipboard,
Flatpaks, and power. Source: [`ios/`](../ios).
## How it works
An iPhone can't run Python or `ssh`, but the Frame can. So the app:
1. connects to the Frame over SSH itself (the [Citadel](https://github.com/orlandos-nl/Citadel)
Swift SSH library), with its own ed25519 key from the Keychain;
2. copies Frame Control's server and helpers (`ios/scripts/make_frame_bundle.py`,
under 1 MB) to `~/.cache/frame-control/<version>` on the Frame, once per version;
3. starts `ui/server.py` there with `FRAME_LOCAL=1`. It listens only on the
Frame's own 127.0.0.1, and it stops when the phone disconnects (`--exit-on-eof`);
4. tunnels to it through the SSH session and shows the same page as the desktop
app, in a web view. The page carries a fresh key each session, which the
server requires on every request.
With `FRAME_LOCAL=1`, every `ssh frame COMMAND` the server runs goes to
`ui/local-bin/ssh`, which runs the command on the Frame directly (rsync uses it
as its transport too), so the desktop and phone share one code path. Android
display settings use `podman exec` into each Lepton container instead of adb,
which the Frame doesn't have.
Nothing is left running on the Frame after the phone disconnects; the copied
files stay in `~/.cache/frame-control` (delete it any time).
## Pairing
On the Frame, turn on Developer Mode and set a user password (Steam Settings →
System, then Developer → Set User Password). In the app, enter the headset's
address (`frame.local`, its IP, or its Tailscale name) and that password once.
The app adds its own key to `~/.ssh/authorized_keys` and remembers the Frame's
host key; the password isn't saved. If you already reach the Frame over SSH,
**Or add the key yourself** shows the phone's key to paste into
`authorized_keys`, and connects without a password.
Valve's tap-to-approve devkit pairing isn't used: it only takes RSA keys, and
the Frame's OpenSSH 9.7 rejects the SHA-1 RSA signatures the Swift SSH library
makes.
## What's different on the phone
| Desktop | iPhone |
|---|---|
| Drop files anywhere | Tap **Send to Frame** (or Add a game) and pick files; folders need zipping |
| Screenshots save to `~/Pictures/SteamFrame` | Save opens the share sheet: Save Image puts it in Photos |
| SSH and SFTP open a terminal | They open an app that handles `ssh://` / `sftp://` (Blink Shell, Termius) |
| Steam Link, remote desktop | Open the Steam Link and Windows App apps |
| Sleep, restart, shut down ask in a terminal | The page asks for the Developer Mode password |
| Compatibility reports kept on the computer | Kept on the Frame (`~/.local/share/Frame Control`) |
## Building
```sh
cd ios
xcodegen generate # after changing project.yml
open FrameControl.xcodeproj
```
The build packs the Frame bundle from the checkout, so the phone always runs
the page and server from the same commit. Running on a phone needs your own
signing team in Xcode (Signing & Capabilities).
## Verified
<img src="img/iphone-tabs.jpg" alt="The four tabs in the iPhone app, connected to a Frame" width="900">
In the iOS Simulator (iOS 26.5) against a real Frame, 2026-09-27: the app connected
with its key, copied the bundle over SFTP, started the server on the Frame and
showed all four tabs with live data. In the app's web view, Capture returned a
headset still and Live played H.264 video at 31 fps (WebCodecs works in
WKWebView). Through the app's tunnel: status, games, Steam library, Android apps,
screenshots, a file upload (checked on the Frame), a background install job, and
the power password check (a wrong password is refused). The server on the Frame
exits within seconds of the app closing.
Against Valve's own Steam Frame OS (SteamOS 0.3.0 build 20260922.5152327, the
`rootfs-A` partition of the Frame recovery image, run with its own sshd; see
[tests/frame-container](../tests/frame-container)), and a Holo Core stand-in:
pairing with the password (key added with the right
permissions, host key pinned, password stored nowhere), the power password
check (a wrong or missing password refused; the right one reaches `systemctl`),
a changed host key refused with "Pair with the Frame again", and a wrong
pairing password reported the same way.
Also verified in the Simulator against the Frame (2026-09-27): the setup screen
found the Frame by itself over Bonjour (`frame · 192.168.1.237`); a paired app
waiting for a sleeping Frame connected 4 s after it answered; an upload from the
app's web view landed in `~/Downloads`; the share sheet offers Save Image
(needs `NSPhotoLibraryAddUsageDescription`, now declared); an install link opens
the confirm dialog and downloads nothing until Install; Steam Link without the
app installed opens its App Store page.
Things iOS asks the first time: **Local Network** (tap Allow, or the app can't
see the Frame), and **Paste** when you send the iPhone's clipboard (tap Allow
Paste, or set Settings → Apps → Frame Control → Paste from Other Apps → Allow).
Sending text to the Frame's clipboard needs the desktop panel open in the
headset, as on the desktop app.
Not yet exercised: Android display changes through podman (no Android app was
running), a real sleep/restart/shut down on the Frame, and a physical iPhone.
Debug builds have Simulator test hooks (`FRAME_TEST_HOST`, `FRAME_TEST_PAGE`,
`FRAME_TEST_JS`, and the tunnel URL in the app's Caches folder); release builds
don't.
+22
View File
@@ -103,6 +103,28 @@ Still open: 4, 6, 7, 11 (in-headset connect), 12–15, 16 (off-LAN and after a r
the colour-coded test clips play in 3D (red left eye, cyan right) for both
H.264 and H.265? Does the DLNA browser find a server on the Mac?
## Verified 2026-09-27
- **Recovery images exist** for the Frame at
`https://steamdeck-images.steamos.cloud/recovery/`; the root filesystem inside
is btrfs and runs, as a userland, on ARM64 Linux. See
[recovery-and-images.md](recovery-and-images.md).
- **Frame Control's server runs on the Frame itself** (the iPhone app does
this), including headset capture, 31 fps live video and file uploads. See
[iphone.md](iphone.md).
- **Password pairing and `sudo -S`** work against the recovery image's own
sshd and sudo (not yet against the headset, whose password we don't hold).
## Still open (2026-09-27)
- Does `podman exec <lepton container> /system/bin/sh -c 'wm size'` change an
instance's display the way `adb shell wm size` does?
- Can the recovery image, or its kernel, boot in a VM at all?
- Does a real sleep, restart or shut down from the iPhone app work (via
`sudo -S systemctl`)?
- The Mac EDL flashing script in `~/Downloads/steam-frame-recovery/` hasn't
been run against a Frame.
## Unconfirmed claims made in these docs
- `/home` and `/etc` persist across Frame OS updates. This is inferred from
+109
View File
@@ -0,0 +1,109 @@
# Recovery images and OS images for the Frame
Where to get the Steam Frame's operating system, what's inside it, and how to
run it for testing without the headset. For recovering a Frame that won't boot,
see the boot menu and boot-loop entries in
[how-the-frame-works.md](how-the-frame-works.md#facts-worth-knowing).
## Downloads
Valve's SteamOS download page (`store.steampowered.com/steamos/download`)
redirects to the [Installation and Repair FAQ](https://help.steampowered.com/en/faqs/view/65B4-2AA3-5F37-4227),
which offers the Steam Deck image. The **Steam Frame images are on the same
server** but aren't linked from that page:
**https://steamdeck-images.steamos.cloud/recovery/** (a plain directory
listing, checked 2026-09-27).
| File | Size | Use |
|---|---|---|
| `steamframe-oobe-repair-20260922.5153644-0.3.0.img.bz2` (or `.img.zip`) | 3.8 GiB | Write to an 8 GB+ USB-C stick, then **Boot from USB** in the Frame's boot menu |
| `steamframe-oobe-repair-qdl-20260922.5153644-0.3.0.tar.gz` (or `.zip`) | 3.8 GiB | Flash over a USB-C cable in Qualcomm EDL mode with `flash.sh` (Linux) or `flash.cmd` (Windows), which use [qdl](https://github.com/linux-msm/qdl). **Wipes everything** |
All four are dated 2026-09-22. Everything else there is for the Steam Deck
(`steamdeck-…`, x86-64), which won't run on the Frame. Valve publishes **no
checksums**. These are the SHA-256s of our downloads (2026-09-26), which passed
`bzip2 -t` and `tar -t`:
```
3a4a077f1b1f40688ab3279affcb56776bd97c54db1573e7c65fc52a97106676 steamframe-oobe-repair-20260922.5153644-0.3.0.img.bz2
d3323bfa8efe9ece1954948421cdf5f705e8942eb50c960e2916d935d1b850ab steamframe-oobe-repair-qdl-20260922.5153644-0.3.0.tar.gz
```
Our copies, with a Mac EDL flashing script built on qdl (untested), are in
`~/Downloads/steam-frame-recovery/` on the Mac.
## What's inside the USB image
A GPT disk with 512-byte sectors and one A slot (a Frame has A and B slots;
the installer makes the rest). **Verified 2026-09-27** from
`steamframe-oobe-repair-20260922.5153644-0.3.0.img.bz2`:
| # | Name | Start sector | Size | Type GUID |
|---|---|---|---|---|
| 1 | `esp` | 34 | 256 MiB | `c12a7328-f81f-11d2-ba4b-00a0c93ec93b` (EFI system) |
| 2 | `efi-A` | 524322 | 64 MiB | `ebd0a0a2-b9e5-4433-87c0-68b6b72699c7` |
| 3 | `rootfs-A` | 655394 | 5120 MiB | `4f68bce3-e8cd-4db1-96e7-fbcaf984b709` |
| 4 | `var-A` | 11141154 | 256 MiB | `4d21b016-b534-45c2-a9fb-5c16e091fd2d` |
| 5 | `home` | 11665442 | 100 MiB | `933ac7e1-2eb4-4f13-b844-0e14e2aef915` |
The partitions start at sector 34, not on MiB boundaries, so compute offsets
from the table (sector × 512), not from rounded sizes. `rootfs-A` is **btrfs**
(label `rootfs-A`, 9.2 GB of files), mounted read-only on the Frame.
Its `/etc/os-release` says `NAME="SteamOS"`, `ID=steamos`, `ID_LIKE=arch`,
`VERSION_CODENAME=holo`; the running system reports version 0.3.0, variant
`vr`, build **20260922.5152327**, which is a different number from the
`5153644` in the file name. Our headset reports build 20260922.6101926.
Inside, it matches a real Frame:
- User `steamos` (uid 1000) is in `wheel` (gid 998), and sudoers has
`%wheel ALL=(ALL) ALL`, so sudo asks for the Developer Mode password.
- `sshd_config` includes `sshd_config.d/*.conf`, uses `.ssh/authorized_keys`
plus `AuthorizedKeysCommand /usr/bin/userdbctl ssh-authorized-keys %u`,
and sets `KbdInteractiveAuthentication no` and `UsePAM yes`. So sshd offers
`publickey,password`, the same as the headset.
- `/usr/bin` has `sshd`, `sudo`, `python3` and `podman`.
Get just the root filesystem without unpacking the whole 5.8 GB image (the
partition's start and size, in sectors, come from the table above):
```sh
bzcat steamframe-oobe-repair-*.img.bz2 | tail -c +$((655394 * 512 + 1)) | head -c $((10485760 * 512)) > rootfs-A.img
```
A Mac can't mount btrfs; a Linux machine or VM can (`mount -o ro -t btrfs`).
## Running it without the headset
The image can't boot in a generic virtual machine: its kernel and bootloader
are built for the Frame's Qualcomm Snapdragon 8 Gen 3 (**inferred**; not
attempted). Its **userland** runs fine on any ARM64 Linux, which covers
anything that talks to the Frame over SSH.
[`tests/frame-container/frame-image.sh`](../tests/frame-container/frame-image.sh)
extracts `rootfs-A`, mounts it read-only with a throwaway writable layer, and
starts the image's own `sshd` on port 2223 (user `steamos`, a test password;
`systemctl` only records requests). On a Mac, run it in Colima's ARM64 VM (see
[tests/frame-container/README.md](../tests/frame-container/README.md)).
**Verified 2026-09-27:** the iPhone app paired with it by password (the image's
sshd logged `Accepted password`, then `Accepted publickey … ED25519`), ran
Frame Control's server on the image's Python, and the image's sudo rejected a
wrong power password and passed the right one to `systemctl`. Without the
Frame's hardware there's no SteamVR, Steam client, battery or Lepton, so those
parts stay untested this way.
## Holo Core aarch64 (Valve and Collabora)
The ARM64 port of Arch Linux that the Frame's SteamOS is built on, published as
a preview in July 2026 ([Collabora's announcement](https://www.collabora.com/news-and-blog/news-and-events/building-an-arch-linux-aarch64-port-for-holo-core.html)).
It's a base system and build environment, not the Frame's OS:
- Source: `https://gitlab.steamos.cloud/holo/holo-core-aarch64-preview`
- Packages: `https://holo-packages.steamos.cloud/holo-core-aarch64-preview/mash-20251118`
- Container: `registry.gitlab.steamos.cloud/holo/holo-core-aarch64-preview/base-devel:latest`
(1.7 GB; `/etc/os-release` says "Holo core Aarch64 port (preview)"; `pacman`
installs OpenSSH 10.2, Python 3.13 and sudo from its repositories. Checked 2026-09-27.)
[`tests/frame-container/Dockerfile`](../tests/frame-container/Dockerfile) builds a
lighter Frame stand-in on it (a `steamos` user with a password and sudo, sshd
with keys and passwords), handy when you don't have the 4 GB image.
+10 -4
View File
@@ -21,7 +21,8 @@ desktop app, which knows where a dropped folder lives; in a plain browser, zip
it.) A dialog shows:
- **Name**: what Steam shows. Steam uses the title id as the name, so it's
limited to letters, digits, `_` and `-`; the dialog shows the result.
limited to letters, digits and `_`, and can't start with a digit; the
dialog shows the result.
- **Launches**: the program picked to start the game, with the other
candidates in the list.
- **Runtime**: picked from the program, see below. Windows programs can switch
@@ -133,10 +134,15 @@ splits that string is **not checked**.
with the same rule, because `scp -r` would follow a link out of the folder
and upload whatever it points at.
- Installs run one at a time, and Remove is refused while one runs.
- The title id is limited to `[A-Za-z0-9_-]`, at most 64 characters. Valve's
scripts pass it to a shell (`steamos-delete` runs `rm -r` on it). Valve's
- The title id is limited to letters, digits and `_`, doesn't start with a
digit (one that would gets `_` in front), and is 2 to 64 characters. That's
what Steam's `create-shortcut` accepts: on the Frame it refused
`fc-smoke-exe` with `missing/invalid arguments` and registered the same
program as `FCSmokeProbe` (2026-09-27, BUILD_ID 20260922.6101926), and
Valve's client only allows `^[A-Za-z_][A-Za-z0-9_.]+$`. Valve's scripts
also pass the id to a shell (`steamos-delete` runs `rm -r` on it). Valve's
reserved sideload names (`steam`, `steamvr`, and their `deckard` forms,
which would replace the Steam client itself) get `-game` added.
which would replace the Steam client itself) get `_game` added.
- Nothing needs `sudo`; everything goes to your home folder on the Frame.
- In the app, a dropped folder is read from its local path by the app's own
server, which only accepts requests from its own page (see
+12
View File
@@ -102,6 +102,18 @@ started. `curl http://<frame-ip>:32000/properties.json` shows whether the servic
`~/.ssh/authorized_keys` lives under `/home`, which SteamOS keeps across OS
updates (inferred from Deck; the Frame uses the same A/B image scheme).
## From an iPhone or iPad
The iPhone app ([iphone.md](iphone.md)) makes its own ed25519 key and adds it
with the Developer Mode password, once, over a password login; the Frame's sshd
offers `publickey,password` (OpenSSH 9.7p1, keyboard-interactive off). It can't
use the devkit pairing above: that installs an RSA key, and the Swift SSH
library signs RSA only with SHA-1, which OpenSSH 8.8 and later refuse by default.
The app pins the Frame's host key on first use and asks you to pair again if it
changes. **Verified 2026-09-27** against the Frame's recovery image
([recovery-and-images.md](recovery-and-images.md)); on the headset, the add-the-key-yourself
route was used.
## Keeping `sshd` enabled across updates
- **Frame**: SSH is tied to the Developer Mode toggle, so it should survive
+34 -9
View File
@@ -1,9 +1,11 @@
# Screen and desktop streaming
This covers two directions:
This covers three directions, plus input:
- **A. Frame → Mac**: see and control the headset from the Mac.
- **B. Mac → Frame**: use the Mac's desktop inside the headset.
- **C. iPhone → Frame**: mirror the phone inside the headset.
- **Input**: type and point in the Frame from the Mac or iPhone.
The confidence labels are the same as in [ssh.md](ssh.md).
@@ -51,14 +53,37 @@ choose to save it. Remmina stores passwords encrypted with a per-install key,
so the script doesn't try to write the password. (The Remmina file format is
standard; the Flatpak data path is inferred.)
## Input and text entry without the virtual keyboard
## C. Show the iPhone's screen inside the Frame
- **A Bluetooth keyboard and mouse** paired to the Frame is the obvious way to
avoid the virtual keyboard. Road to VR says there are "only a few things
you'd actually want to do" on the Linux desktop unless you connect a
keyboard and mouse.
(Pairing a BT keyboard on the Frame is inferred from SteamOS; not verified.)
- **Clipboard from the Mac**: `scripts/paste-to-frame.sh` (see
[file-transfer.md](file-transfer.md#clipboard)).
iOS only shares its screen two ways: **AirPlay** (Screen Mirroring in Control
Centre) or a **ReplayKit broadcast extension** in an app. Nothing else can
capture it.
| Option | What it takes | Confidence | Verdict |
|---|---|---|---|
| **UxPlay** (an open-source AirPlay receiver) on the Frame | Build it for aarch64 (no Flathub package; there's a Snap and distro packages), run it in `~` or a podman container, and advertise it over mDNS. The iPhone *and* the Mac then see "Frame" in Screen Mirroring, with nothing to install on either | **Inferred.** It runs on ARM64 Linux such as the Raspberry Pi ([UxPlay](https://github.com/FDH2/UxPlay)). Not tried on the Frame: needs mDNS registration and its ports (7000, 7001, 7100 and a UDP range) reachable | **Recommended to try first.** It's the only receiver-side option, and it covers the Mac too. The window shows in the Frame's Linux desktop panel |
| A broadcast extension in Frame Control | ReplayKit sends the screen to a small extension (50 MB memory limit), which encodes H.264 and sends it through the app's SSH tunnel to the page, shown the same way as the Frame's live view in reverse | **Inferred** from Apple's ReplayKit docs | Full control and no network setup, but several days' work, and the picture only shows where Frame Control's page is open in the headset |
## Input: type and point in the Frame from the Mac or iPhone
**Verified 2026-09-27** on the headset: `steamos` is in the `input` group and
`/dev/uinput` is `crw-rw-r-- root input`, so **our own code can create a
virtual keyboard and mouse without sudo**. The Frame has no `python-evdev`,
`ydotool`, `wtype` or KDE Connect; `kwin_wayland` and `plasmashell` run only
while the desktop panel is open in the headset.
| Option | Mac | iPhone | Notes |
|---|---|---|---|
| **A uinput keyboard and mouse in Frame Control's server** | ✓ | ✓ | **Recommended.** The server opens `/dev/uinput` with `ctypes` (standard library only) and the page sends key and pointer events through the tunnel it already has. On the phone: a trackpad area (drag to move, tap to click, two fingers to scroll) and the iOS keyboard for typing. On the Mac: a "control the Frame" mode that captures the keyboard and pointer (Esc to release). Uinput devices look like real hardware to the kernel, so libinput, KWin and gamescope should take them; [frame-voice](https://github.com/DeeJanuz/frame-voice) already types into a Frame through a uinput keyboard. **Untested**: which surfaces in VR (desktop panel, SteamVR dashboard, games, Android apps in Lepton) accept the pointer. About a day or two of work |
| **Bluetooth keyboard and mouse** | – | – | Real hardware paired in SteamOS settings. The iPhone can't pretend to be a Bluetooth keyboard: iOS won't advertise the HID service ([Apple forums](https://developer.apple.com/forums/thread/733916)) |
| **Deskflow** (formerly Input Leap / Barrier) | ✓ | – | Moves the Mac's own mouse and keyboard onto the Frame's screen edge. Flathub has an aarch64 build ([Flathub](https://flathub.org/apps/org.deskflow.deskflow)); on Wayland it needs the InputCapture/libei portal, and only works while Plasma is running. No iPhone client |
| **KDE Connect** | ~ | ✓ | Its iOS app has a remote touchpad and keyboard, but the Frame would need KDE Connect installed (not on Flathub; `pacman` on a read-only root). More moving parts than the uinput route |
| **Remmina / Steam Link / RDP** | ✓ | – | Input only reaches the streamed session, not the headset's own apps |
Other ways to get text in:
- **Clipboard from the Mac**: `scripts/paste-to-frame.sh`, or Frame Control's
clipboard box (see [file-transfer.md](file-transfer.md#clipboard)). Needs
the desktop panel open.
- **RDP session**: Windows App syncs the clipboard with xrdp, but only inside
that RDP session.
+190
View File
@@ -0,0 +1,190 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Frame Control 0.3.1: features by OS</title>
<style>
:root {
--bg: #1b2838; --panel: #16202d; --line: #2a3f5a; --text: #c7d5e0; --dim: #8f98a0;
--tested: #5ba32b; --partial: #d9a33a; --auto: #4b8bbe; --built: #3d4f63; --no: #6b2b2b;
}
* { box-sizing: border-box; }
body { margin: 0; background: linear-gradient(#171a21, var(--bg) 320px); color: var(--text);
font: 15px/1.5 "Motiva Sans", -apple-system, "Segoe UI", Roboto, sans-serif; }
main { max-width: 1180px; margin: 0 auto; padding: 40px 24px 80px; }
h1 { color: #fff; font-size: 30px; margin: 0 0 4px; font-weight: 600; }
h2 { color: #fff; font-size: 18px; margin: 40px 0 12px; font-weight: 600;
text-transform: uppercase; letter-spacing: .06em; }
.sub { color: var(--dim); margin: 0 0 28px; }
a { color: #66c0f4; }
.cards { display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); gap: 14px; }
.card { background: var(--panel); border: 1px solid var(--line); border-radius: 6px; padding: 16px 18px; }
.card h3 { margin: 0 0 8px; color: #fff; font-size: 16px; }
.card dl { margin: 0; display: grid; grid-template-columns: 76px 1fr; gap: 3px 10px; font-size: 13.5px; }
.card dt { color: var(--dim); }
.card dd { margin: 0; }
.legend { display: flex; flex-wrap: wrap; gap: 10px 20px; margin: 0 0 14px; font-size: 13.5px; }
.legend span { display: inline-flex; align-items: center; gap: 7px; }
table { width: 100%; border-collapse: collapse; background: var(--panel);
border: 1px solid var(--line); border-radius: 6px; overflow: hidden; }
th, td { padding: 9px 12px; border-bottom: 1px solid var(--line); vertical-align: top; text-align: left; }
thead th { background: #0e141b; color: #fff; font-weight: 600; position: sticky; top: 0; z-index: 1; }
thead th.os { width: 150px; text-align: center; }
tr.group td { background: #203044; color: #fff; font-weight: 600; font-size: 13px;
text-transform: uppercase; letter-spacing: .05em; }
td.os { text-align: center; }
td .feat { color: #fff; }
td .note { color: var(--dim); font-size: 13px; }
.pill { display: inline-block; min-width: 92px; padding: 2px 9px; border-radius: 999px;
font-size: 12.5px; font-weight: 600; color: #fff; white-space: nowrap; }
.t { background: var(--tested); }
.p { background: var(--partial); color: #1b1b1b; }
.a { background: var(--auto); }
.b { background: var(--built); color: #c7d5e0; }
.n { background: var(--no); }
.dot { width: 12px; height: 12px; border-radius: 50%; display: inline-block; }
ul { margin: 6px 0 0; padding-left: 20px; }
li { margin: 3px 0; }
footer { color: var(--dim); font-size: 13px; margin-top: 36px; }
</style>
</head>
<body>
<main>
<h1>Frame Control 0.3.1: features by OS</h1>
<p class="sub">Which features are built for each OS, and which were tested against a real Steam Frame
(SteamOS 0.3.0, build 20260922.6101926). Status as of 26 September 2026, for
<a href="https://github.com/saphid/steam-frame/pull/2">PR #2</a> (0.3.1). Every build now bundles its own
Python 3.12, <code>adb</code> and CA certificates, so nothing else needs installing (only <code>ssh</code> on Linux,
plus the system <code>adb</code> on arm64 Linux).</p>
<h2>Builds and test machines</h2>
<div class="cards">
<div class="card"><h3>macOS</h3><dl>
<dt>Built</dt><dd>Apple Silicon (arm64): <code>.dmg</code>, <code>.zip</code>. No Intel build.</dd>
<dt>Signing</dt><dd>Ad-hoc signed, not notarised</dd>
<dt>Tested on</dt><dd>Apple Silicon Mac, macOS 26, using a local 0.3.1 build with the bundled Python and <code>adb</code>. All 15 calls it made to the Frame at startup returned OK.</dd>
</dl></div>
<div class="card"><h3>Windows</h3><dl>
<dt>Built</dt><dd>x64: NSIS installer <code>.exe</code> and <code>.zip</code></dd>
<dt>Signing</dt><dd>Unsigned. SmartScreen shows a warning.</dd>
<dt>Tested on</dt><dd>Windows 11 x64 VM. Real-Frame results below are from 0.3.0. The 0.3.1 installer from CI installs cleanly (31 s) and reinstalls over itself (38 s). The server starts on the bundled Python, and HTTPS to Steam and F-Droid works. The Frame went offline before its 0.3.1 run on the headset.</dd>
</dl></div>
<div class="card"><h3>Linux</h3><dl>
<dt>Built</dt><dd>x86_64 and arm64: <code>AppImage</code> and <code>.deb</code></dd>
<dt>Signing</dt><dd>n/a</dd>
<dt>Tested on</dt><dd>x86_64 Ubuntu 26.04 with no <code>adb</code> and no clipboard tools, using the 0.3.1 AppImage under Xvfb with the bundled Python and <code>adb</code>. The arm64 builds and the <code>.deb</code> packages weren't run; the arm64 package was only checked to contain an ARM Python.</dd>
</dl></div>
</div>
<h2>Features</h2>
<div class="legend">
<span><i class="dot" style="background:var(--tested)"></i><b>Tested</b>: worked against the real Frame on that OS</span>
<span><i class="dot" style="background:var(--partial)"></i><b>Partial</b>: only part of the feature was tested (see note)</span>
<span><i class="dot" style="background:var(--auto)"></i><b>Automated</b>: covered by CI tests on that OS, not tried on a real Frame</span>
<span><i class="dot" style="background:var(--built)"></i><b>Built</b>: in the build, not tested</span>
<span><i class="dot" style="background:var(--no)"></i><b>Not built</b></span>
</div>
<table>
<thead><tr><th>Feature</th><th class="os">macOS</th><th class="os">Windows</th><th class="os">Linux</th></tr></thead>
<tbody>
<tr class="group"><td colspan="4">Connection</td></tr>
<tr><td><div class="feat">Set Up Connection</div><div class="note">Finds the Frame, writes the <code>frame</code> SSH alias, copies your key using the Frame's password. macOS runs <code>connect.sh</code> in Terminal; Windows and Linux run <code>frame_connect.py</code>.</div></td>
<td class="os"><span class="pill p">Partial</span><div class="note">Existing alias used, script not re-run</div></td>
<td class="os"><span class="pill t">Tested</span></td>
<td class="os"><span class="pill t">Tested</span></td></tr>
<tr><td><div class="feat">Shared SSH connection</div><div class="note">A single SSH connection is reused, so each request takes about 0.3 s. Windows OpenSSH can't do this, so there each request opens its own connection (about 0.5 to 1 s).</div></td>
<td class="os"><span class="pill t">Tested</span></td>
<td class="os"><span class="pill n">Not built</span><div class="note">OpenSSH limitation</div></td>
<td class="os"><span class="pill t">Tested</span></td></tr>
<tr class="group"><td colspan="4">Headset view</td></tr>
<tr><td><div class="feat">Capture headset view</div><div class="note">The left eye or both eyes as the lenses show them, saved as PNG</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td></tr>
<tr><td><div class="feat">Capture desktop panel</div><div class="note">gamescope's flat layer</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td></tr>
<tr><td><div class="feat">Live view</div><div class="note">720p H.264 at about 30 fps, decoded with WebCodecs</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td></tr>
<tr><td><div class="feat">Headset screenshots</div><div class="note">Browse the screenshots you took with Steam's shortcut, and save them to <code>~/Pictures/SteamFrame</code></div></td>
<td class="os"><span class="pill t">Tested</span></td>
<td class="os"><span class="pill p">Partial</span><div class="note">Listed (5 found); saving not tried</div></td>
<td class="os"><span class="pill p">Partial</span><div class="note">Listed with thumbnails; saving not tried</div></td></tr>
<tr class="group"><td colspan="4">Status</td></tr>
<tr><td><div class="feat">Battery and charging</div><div class="note">Percentage, watts, time to full or empty, charger type, temperature</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td></tr>
<tr><td><div class="feat">System status</div><div class="note">Storage, memory, temperature, Wi-Fi, uptime, running services</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td></tr>
<tr><td><div class="feat">Volume and mute</div></td>
<td class="os"><span class="pill t">Tested</span></td>
<td class="os"><span class="pill p">Partial</span><div class="note">Read only</div></td>
<td class="os"><span class="pill p">Partial</span><div class="note">Read only</div></td></tr>
<tr class="group"><td colspan="4">Games</td></tr>
<tr><td><div class="feat">Owned games with Frame ratings</div><div class="note">Verified, Playable, Unsupported or Unknown</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td></tr>
<tr><td><div class="feat">Install a game on the Frame</div><div class="note">Uses the headset's Steam client, with live progress</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td></tr>
<tr><td><div class="feat">Store search, Buy, Store on Frame</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td></tr>
<tr><td><div class="feat">Library shelf and Play button</div></td>
<td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td></tr>
<tr class="group"><td colspan="4">Android apps</td></tr>
<tr><td><div class="feat">Installed Android apps list</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td></tr>
<tr><td><div class="feat">F-Droid catalogue search</div><div class="note">About 4,500 apps with Frame ratings, bundled with the app</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td></tr>
<tr><td><div class="feat">Install, launch, stop, test, remove an app</div><div class="note">Each app runs as its own Lepton instance, using the bundled <code>adb</code>. APK files are read by a built-in parser (no <code>aapt2</code>) that matched <code>aapt2</code> on 9 F-Droid APKs.</div></td>
<td class="os"><span class="pill t">Tested</span><div class="note">Diary: read, install, launch, test, remove</div></td>
<td class="os"><span class="pill b">Built</span></td>
<td class="os"><span class="pill p">Partial</span><div class="note">Launch and stop</div></td></tr>
<tr><td><div class="feat">Report an APK</div><div class="note">Reports are saved on your computer; the shared database is maintainer-only</div></td>
<td class="os"><span class="pill a">Automated</span></td><td class="os"><span class="pill a">Automated</span></td><td class="os"><span class="pill a">Automated</span></td></tr>
<tr><td><div class="feat">Android display settings</div><div class="note">Resolution, UI scale, text size</div></td>
<td class="os"><span class="pill t">Tested</span><div class="note">Density and text size set, then reset</div></td>
<td class="os"><span class="pill b">Built</span></td>
<td class="os"><span class="pill p">Partial</span><div class="note">Read over the bundled adb</div></td></tr>
<tr class="group"><td colspan="4">Transfer</td></tr>
<tr><td><div class="feat">Send files to ~/Downloads</div><div class="note">Test files had non-English characters in their names (é, ✓). macOS and Linux copy with rsync; Windows uses scp.</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td></tr>
<tr><td><div class="feat">Drop an APK to install it</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td></tr>
<tr><td><div class="feat">Send text or clipboard to the Frame</div><div class="note">Needs the headset desktop open. The app reads your clipboard through Electron, so no extra tools are needed.</div></td>
<td class="os"><span class="pill t">Tested</span><div class="note">Reading the clipboard retested in 0.3.1</div></td>
<td class="os"><span class="pill p">Partial</span><div class="note">Reached the Frame; desktop was closed</div></td>
<td class="os"><span class="pill p">Partial</span><div class="note">Clipboard read with no xclip; Frame desktop was closed</div></td></tr>
<tr><td><div class="feat">Flatpak install and remove</div></td>
<td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td></tr>
<tr class="group"><td colspan="4">One-click tools</td></tr>
<tr><td><div class="feat">SSH or SFTP in a terminal</div><div class="note">macOS: Terminal. Windows: cmd. Linux: GNOME Terminal, Konsole, xterm and others.</div></td>
<td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td></tr>
<tr><td><div class="feat">Steam Link</div></td>
<td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td></tr>
<tr><td><div class="feat">Remote desktop</div><div class="note">macOS: Windows App. Windows: Remote Desktop. Linux: Remmina or FreeRDP.</div></td>
<td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td></tr>
<tr><td><div class="feat">Sleep, restart, shut down</div><div class="note">Opens a terminal because SteamOS asks for the sudo password</div></td>
<td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td><td class="os"><span class="pill b">Built</span></td></tr>
<tr class="group"><td colspan="4">App</td></tr>
<tr><td><div class="feat">Local server test suite</div><div class="note">Runs in GitHub Actions on every push (Python 3.12 on macOS and Windows, Python 3.13 on Ubuntu), including the APK reader tests</div></td>
<td class="os"><span class="pill a">Automated</span></td><td class="os"><span class="pill a">Automated</span></td><td class="os"><span class="pill a">Automated</span></td></tr>
<tr><td><div class="feat">Mac or PC wording</div><div class="note">The UI says Finder or File Explorer, and Mac or PC, to match your system</div></td>
<td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td><td class="os"><span class="pill t">Tested</span></td></tr>
</tbody>
</table>
<h2>Notes</h2>
<ul>
<li><b>Tested</b> means the app, running on that OS, got a successful response from the real Frame: for example, a PNG from a capture, 868 owned games, or the Android apps listed.</li>
<li>The Windows VM tests ran in its desktop session. <code>ssh.exe</code> hangs when it's started from a remote SSH session, but a normal desktop user won't hit that.</li>
<li>The macOS test from 25 September also covered the capture shown when the headset is in standby, input validation, and using the clipboard with the headset desktop open.</li>
<li>Everything marked <b>Built</b> runs a command that works on its own. It just hasn't been tried end to end from the app on that OS yet.</li>
</ul>
<footer>Frame Control is an unofficial tool, not made by Valve. MIT licence.</footer>
</main>
</body>
</html>
+150
View File
@@ -0,0 +1,150 @@
# Testing
Frame Control is tested in three layers, from fast and fake to slow and real.
A fourth, a SteamOS VM, may come later ([issue #6](https://github.com/saphid/steam-frame/issues/6)).
| Layer | Runs | Needs | Covers |
|---|---|---|---|
| Unit tests (`tests/*.py`) | `python3 -m unittest discover -s tests` | Nothing | Parsing, validation, request guards; SSH and HTTP are mocked |
| Fake Frame (`tests/e2e`) | `scripts/e2e.sh` | Linux with Docker | The real server and scripts against a container that behaves like a Frame |
| Headset smoke test | `scripts/frame-smoke.sh` | A Frame on the `frame` alias | Install, launch and remove on the real device, recorded with its BUILD_ID |
## Unit tests
```sh
python3 -m unittest discover -s tests
```
About 120 tests, a few seconds, on Python 3.9 and newer. GitHub Actions runs
them on macOS, Windows and Linux. They don't pick up `tests/e2e`.
## The fake Frame
`tests/fakeframe/` builds a container that stands in for the headset, and a
second one for the computer Frame Control runs on. `scripts/e2e.sh` builds
both, starts them with `docker compose`, runs `tests/e2e` in the host
container and takes everything down, exiting with the tests' status:
```sh
scripts/e2e.sh # everything, about 2 minutes plus the first build
scripts/e2e.sh test_titles # one module
scripts/e2e.sh test_faults.Faults.test_disk_full # one test
FAKEFRAME_KEEP=1 scripts/e2e.sh # leave it running afterwards
```
It needs a Linux host with Docker and `docker compose`, and zsh. The images
are `fakeframe-frame` and `fakeframe-host`; the compose project, network and
volumes are `fakeframe-e2e*`. CI runs it on a native arm64 runner
(`ubuntu-24.04-arm`, the `e2e` job in `.github/workflows/checks.yml`).
The host container exists because OpenSSH reads `~/.ssh/config` from the
passwd home directory, not `$HOME`. There, `ssh frame` reaches the fake Frame
through the same `Host frame` block `ui/frame_connect.py` writes, the
repository is mounted read-only at `/repo`, and each test module starts the
real `ui/server.py` (Python 3.9) and talks to it over HTTP with the headers
its guards want.
### What's real and what's fake
| On the fake Frame | |
|---|---|
| Arch Linux (`archlinux:base`, or Valve's Holo Core aarch64 preview on arm64), user `steamos`, `/etc/os-release` with BUILD_ID 20260922.6101926 | Real OS, Frame's identity |
| `sshd` with key and password logins, `rsync`, `python3` | Real |
| Valve's steamos-devkit-service on port 32000 and its hooks, vendored unmodified in `tests/fakeframe/steamos-devkit-service` | Real; only its `dbus` import (for mDNS through systemd-resolved) is a stand-in that logs the registration |
| Valve's devkit-utils, copied over by Frame Control itself | Real |
| **fakesteam**: `~/.steam/steam.pid`, `steam.token` and the `steam.pipe` FIFO; answers `approve-ssh-key`, `create-shortcut`, `run-game`, `list-shortcuts` and `delete-shortcut` with the response files devkit-utils waits for; takes `steam://rungameid`, `install` and `store` URLs | Fake |
| DevTools on `127.0.0.1:8080` with a `SharedJSContext` target. The JavaScript Frame Control sends runs for real in Node against stand-in `SteamClient`, `appStore` and `downloadsStore` objects (`cef_shim.js`), so async functions, optional chaining and `Map`s behave as in Steam's CEF | The JS engine is real; the objects are fake |
| `steam`, `wpctl`, `flatpak`, `podman`, `nmcli`, `qdbus6`, `gamescopectl`, SteamOS's `steamos-enable-sshd` helper, and Lepton's launcher | Stubs that record their calls |
| Battery, charger and thermal zones under `/sys/class` | Files the supervisor writes. `/sys` is read-only in a container and Docker's AppArmor profile refuses writes under it, so each folder is a volume mounted twice: over `/sys/class/...` for `frame_status.py` to read, and under `/var/lib/fakeframe/sys` for the supervisor to write |
| `vrserver` and `plasmashell` | Renamed `sleep` processes, so the status page and the clipboard find them |
Every fake behaviour copied from the device has a comment citing the doc or
observation and the BUILD_ID it came from; anything not seen on a headset is
marked as a guess. The fake keeps its state in `/var/lib/fakeframe/state.json`
(shortcuts, devkit titles, compat tool mapping, launches, pairing requests,
Lepton instances, volume, Flatpaks, clipboard) and logs stub calls to
`calls.jsonl` beside it.
Native programs really run: a launched aarch64 title executes on an arm64
host, and an x86-64 one on x86-64 (the container shares the host's kernel).
Proton titles are recorded with the command Steam would run, not run.
### Fault switches
`fakeframe-ctl` works over SSH (`ssh frame fakeframe-ctl help`) and from the
host container (`FAKEFRAME_CTL=http://fakeframe:9999`), so a test can flip a
switch while SSH is down:
| Command | Effect |
|---|---|
| `pairing on\|off` | Steam's **Pair new host** screen open or not; off gives the device's 403 text |
| `answer approve\|deny\|timeout` | How the pairing prompt is answered |
| `steam on\|off` | Steam client running (pid file, pipe, DevTools) |
| `sleep on\|off` | Headset asleep: ports 22 and 32000 accept and never answer, so SSH times out |
| `sshd on\|off` | sshd stopped: new connections are refused, open ones stay |
| `devkit-service on\|off` | Port 32000 closed |
| `disk-full on\|off` | Fills the small (64 MB) filesystem on `~/devkit-game` |
| `runtime NAME installed\|missing` | Proton, the Steam Linux Runtimes, Lepton |
| `battery KEY=VALUE...` | e.g. `capacity=15 status=Discharging current_now=-900000` |
| `keys harness\|none`, `authorized-keys` | Set or read `~/.ssh/authorized_keys` |
| `reset`, `state`, `calls [TOOL]` | Start over; read the state and call log |
### What the fake can't show
- Rendering: the headset view, desktop capture content, live video, SteamVR,
gamescope and panels. The capture stub returns a placeholder PNG.
- Proton and FEX: whether a Windows or x86-64 program actually runs.
- Android: there's no Android in the Lepton stand-in, so no ADB, display
settings, probes or app crashes.
- The real Steam client's UI and anything it does that isn't modelled, and
mDNS discovery.
- `sudo` and the power buttons, Tailscale, and the Windows and macOS sides of
the app (the host container is Linux, so the `rsync` paths are tested and the
`scp` fallback isn't).
## Headset smoke test
```sh
scripts/frame-smoke.sh # needs `ssh frame` to work without a password
scripts/frame-smoke.sh --pair # also pairs a throwaway key: approve it in the headset
```
It checks `properties.json` and the status, then installs, launches and
removes three tiny titles built from bytes by `tests/smoke/tiny_programs.py`
(an ARM64 and an x86-64 static Linux program that sleep for ten seconds, and
an x86-64 `.exe` that exits at once). A launch passes only with fresh evidence:
the ARM64 program running, the `.exe` started (its process or Steam's log),
and the x86-64 program running or Steam logging that its runtime isn't
installed, which is what the Frame does today. Steam's log lines about each
title are kept.
Everything it installs is removed again, also after a failure: the titles and
their Steam shortcuts, a paired key, and `~/devkit-utils` if it wasn't there
before (if it was, it stays, synced to this checkout as Frame Control always
does). A cleanup that fails counts as a failed step. Results go to
`tests/smoke/results/<time>-<BUILD_ID>.json` (not committed) with a summary on
screen; it exits 0 when every step passed, 1 if one failed, 2 if the headset
isn't reachable.
`--pair` asks the devkit service to pair a new RSA key, which needs someone
in the headset to open **Settings → Developer → Pair new host** and approve
it; the key is checked and then taken out of `authorized_keys` again.
## When the device disagrees with the fake
The fake is only as good as what's been seen on a headset. When the smoke
test (or anyone) finds the Frame doing something else:
1. Record what the device did, with the date and BUILD_ID, in the doc that
covers it (`docs/sideloading.md`, `docs/ssh.md` and so on).
2. Change the fake to match, with a comment citing that observation. The
behaviours are in `tests/fakeframe/rootfs/usr/local/lib/fakeframe/`
(`fakesteam.py` for Steam, `cef_shim.js` for DevTools, `init.py` for the
switches, the stubs in `rootfs/usr/local/bin`).
3. Run `scripts/e2e.sh`. If the app is wrong, the tests now fail the way the
device did; fix the app and add a unit test.
For example, on 2026-09-27 the smoke test found that Steam's `create-shortcut`
refuses ids with a hyphen (`missing/invalid arguments`), which the fake had
accepted. The fake now refuses them the same way, and Frame Control makes ids
Steam accepts.