iPhone and iPad app: the same features, served from the Frame

An iPhone can't run Python or ssh, but the Frame can. The app (ios/, SwiftUI)
connects with its own SSH key (Citadel), copies the server and helpers to
~/.cache/frame-control/<version> on the Frame once per version, starts
ui/server.py there with FRAME_LOCAL=1 on the Frame's 127.0.0.1, and shows the
page through an SSH tunnel. The server exits when the phone disconnects.

Server: FRAME_LOCAL=1 puts ui/local-bin on PATH, whose ssh stand-in runs each
`ssh frame COMMAND` locally (and serves as rsync's transport), so desktop and
phone share one code path. Android display goes through podman exec there, as
the Frame has no adb. FRAME_UI_KEY replaces the fixed X-Frame-UI value with a
per-session key. Power actions take the Developer Mode password via sudo -S.
--port 0 now prints the port it took.

Page: a bottom tab bar and safe areas on phones, Play buttons visible on touch
screens, saving through the share sheet, SSH/SFTP/Steam Link/remote desktop
opening in their iOS apps, and a password dialog for power.

App: pairing with the Developer Mode password once (never stored) or with a
key the user adds; host key pinned on first use; plain-language connection
errors with quiet retries; frame-control://install links; alerts and confirms.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
saphidandClaude Opus 5.5 committed 2026-09-27 11:18:15 +10:00
1 parent 0f770dc88a
commit 13187e8f6a
33 files changed
+2459 -34

No files matched your search

+73
View File
@@ -0,0 +1,73 @@
# Frame Control for iPhone
The iPhone (and iPad) app does what the desktop app does, from the phone:
headset view and live video, battery and status, screenshots, Steam games,
Android apps and their display settings, sideloading, files, clipboard,
Flatpaks, and power. Source: [`ios/`](../ios).
## How it works
An iPhone can't run Python or `ssh`, but the Frame can. So the app:
1. connects to the Frame over SSH itself (the [Citadel](https://github.com/orlandos-nl/Citadel)
Swift SSH library), with its own ed25519 key from the Keychain;
2. copies Frame Control's server and helpers (`ios/scripts/make_frame_bundle.py`,
under 1 MB) to `~/.cache/frame-control/<version>` on the Frame, once per version;
3. starts `ui/server.py` there with `FRAME_LOCAL=1`. It listens only on the
Frame's own 127.0.0.1, and it stops when the phone disconnects (`--exit-on-eof`);
4. tunnels to it through the SSH session and shows the same page as the desktop
app, in a web view. The page carries a fresh key each session, which the
server requires on every request.
With `FRAME_LOCAL=1`, every `ssh frame COMMAND` the server runs goes to
`ui/local-bin/ssh`, which runs the command on the Frame directly (rsync uses it
as its transport too), so the desktop and phone share one code path. Android
display settings use `podman exec` into each Lepton container instead of adb,
which the Frame doesn't have.
Nothing is left running on the Frame after the phone disconnects; the copied
files stay in `~/.cache/frame-control` (delete it any time).
## Pairing
On the Frame, turn on Developer Mode and set a user password (Steam Settings →
System, then Developer → Set User Password). In the app, enter the headset's
address (`frame.local`, its IP, or its Tailscale name) and that password once.
The app adds its own key to `~/.ssh/authorized_keys` and remembers the Frame's
host key; the password isn't saved. If you already reach the Frame over SSH,
**Or add the key yourself** shows the phone's key to paste into
`authorized_keys`, and connects without a password.
Valve's tap-to-approve devkit pairing isn't used: it only takes RSA keys, and
the Frame's OpenSSH 9.7 rejects the SHA-1 RSA signatures the Swift SSH library
makes.
## What's different on the phone
| Desktop | iPhone |
|---|---|
| Drop files anywhere | Tap **Send to Frame** (or Add a game) and pick files; folders need zipping |
| Screenshots save to `~/Pictures/SteamFrame` | Save opens the share sheet: Save Image puts it in Photos |
| SSH and SFTP open a terminal | They open an app that handles `ssh://` / `sftp://` (Blink Shell, Termius) |
| Steam Link, remote desktop | Open the Steam Link and Windows App apps |
| Sleep, restart, shut down ask in a terminal | The page asks for the Developer Mode password |
| Compatibility reports kept on the computer | Kept on the Frame (`~/.local/share/Frame Control`) |
## Building
```sh
cd ios
xcodegen generate # after changing project.yml
open FrameControl.xcodeproj
```
The build packs the Frame bundle from the checkout, so the phone always runs
the page and server from the same commit. Running on a phone needs your own
signing team in Xcode (Signing & Capabilities).
## Verified
In the iOS Simulator (iOS 26.5) and against a Frame (2026-09-27): the page and
all its reads, headset capture, live video, and file upload through the
on-Frame server. Not yet exercised: pairing with the password, Android display
changes through podman, and power actions.