Merge remote-tracking branch 'origin/main' into theatre-media

# Conflicts:
#	ui/server.py

docs/frame-control.md

@@ -72,6 +72,10 @@ counts them while they run.
   Runtime picked from the program's header), listed under **Sideloaded titles**
   with Launch and Remove; see [sideloading.md](sideloading.md). Send typed text, or your computer's clipboard, to the
   Frame clipboard.
+- **Mac in the headset** (macOS): show any Mac window, or a whole screen, as
+  its own panel in the headset. Place it with the SteamVR dashboard, click and
+  scroll with the laser, and type on the Mac. Streams hardware H.264 through
+  an SSH tunnel; see [mac-in-headset.md](mac-in-headset.md).
 - **Flatpaks**: install and remove them (quick picks: Moonlight, Firefox, VLC,
   Remmina).
 - **One-click tools**: SSH or SFTP in a terminal window, Steam Link, and remote

docs/how-the-frame-works.md

@@ -52,6 +52,9 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600
 | **T3 Code desktop runs natively.** The stock release `T3-Code-0.0.42-arm64.AppImage` in `~/Applications/T3CodeDesktop/` starts with no extra setup: glibc 2.39, `libfuse.so.2`, GTK 3, NSS and libsecret are on the image. `panel-on-frame.sh --name t3code-desktop -- '~/Applications/T3CodeDesktop/T3-Code.AppImage'` gives it its own panel (`valve.steam.desktopgame.2000281357`, `--ozone-platform=x11`). Its bundled server listens on `127.0.0.1:3773` and shows up in onboarding as the `frame` computer, with `passwordStore: gnome-libsecret`. The image has no agent CLI and no `node`. Agents run through the LAN CLIProxyAPI (`llm-proxy.lan:8317`, which resolves on the Frame). Claude Code 2.1.283 comes from `claude.ai/install.sh`, and Codex 0.157.1 from the `codex-aarch64-unknown-linux-musl` release tarball, both into `~/.local/bin`. `with-cliproxy` and a mode-600 `~/.config/cliproxyapi/secrets.env` are copied from the Mac. The wrappers `claude-cliproxy` and `codex-cliproxy` (a `-c model_provider=cliproxy`, `wire_api="responses"`, `env_key="CLIPROXY_API_KEY"`) are set as `providers.claudeAgent.binaryPath` and `providers.codex.binaryPath` in `~/.t3/userdata/settings.json`, and T3 picked that up without a restart. Through the wrappers, `claude auth status` reports `loggedIn: true` (`oauth_token`), and both CLIs answered a prompt with `kimi-k3`. `gamescopectl screenshot` captured another layer (the Lepton T3 app) rather than this panel. `DISPLAY=:0 xwd -id <win>` piped to `ffmpeg` captures the window itself (1920×1080). **Verified 2026-09-26**, BUILD_ID 20260922.6101926. | Running T3 Code as a host on the Frame |
 | Power actions need `sudo`, which asks for the Developer Mode password over SSH. | Frame Control's power buttons |
 | **SSH server:** OpenSSH 9.7p1. It offers `publickey,password` (keyboard-interactive is off, PAM on) and also asks `userdbctl ssh-authorized-keys` for keys. OpenSSH ≥ 8.8 rejects SHA-1 `ssh-rsa` signatures by default, so a client whose RSA support is SHA-1 only (the Swift library Citadel, for one) can't log in with the RSA key that devkit pairing installs; use ed25519 (**inferred** from OpenSSH defaults). **Verified 2026-09-27**, BUILD_ID 20260922.6101926. | [iphone.md](iphone.md), `ui/frame_connect.py` |
+| **A Chromium app window on gamescope's `:0` with its own `STEAM_GAME` becomes a panel, even when started over SSH.** `chromium-xr/chrome --ozone-platform=x11 --app=URL --window-size=1280,720` got a 1920×1080 window, and tagging it produced `[Overlays] Created: valve.steam.desktopgame.<id>` in `~/.local/share/Steam/logs/vrwebhelper_systemui.txt`. Chromium XR decoded a 1080p H.264 WebCodecs stream from the Mac at about 60 fps. XTest events sent to `:0` (libXtst through Python ctypes) did not reach the page. The Frame has `libXtst`, `xprop`, `xwininfo`, `curl` and the `C.utf8` locale. **Verified 2026-09-28**, BUILD_ID 20260925.6191901. | [mac-in-headset.md](mac-in-headset.md) |
+| **Streaming video into a panel: what the Frame adds.** Chromium XR decodes H.264 in **software** (1920×1290 at about 9 Mbit/s took 4–6 ms per frame); its GL is ANGLE → zink → Turnip on the Adreno 750, and it logs `GetVSyncParametersIfAvailable() failed`. An idle page's `requestAnimationFrame` runs at about 140 Hz; when the headset is worn or woken, SteamVR picks the 4320×2160 @ 144 Hz mode. **An unworn headset throttles panel apps** whatever they draw: a local canvas page ran at 58 fps for about 6 s, then 28, then about 15 once in standby (vrserver logs `entering standby`, with `power.pauseCompositorOnStandby 1`). `vrcmd --handlewakeup` gave 137–144 fps for about 2 s, then 36. So a panel's frame rate and compositor delay can only be measured while it's worn. `tailscaled` runs with `--tun=userspace-networking` and cost about 8% of a core at 9 Mbit/s (0.5% over the LAN address). Wi-Fi power saving is on (`iw … get power_save`). **Verified 2026-09-28**, BUILD_ID 20260925.6191901. | [mac-in-headset.md](mac-in-headset.md#measuring) |
+| **USB-C networking.** Plugged into a Mac, the Frame is a USB network device: macOS names the port "Steam Frame" (here `en9`, 10.86.200.234/29), and the Frame's `usb0` is 10.86.200.233. Ping is about 0.9 ms, and SSH works with the usual host key (`-o HostName=10.86.200.233 -o HostKeyAlias=<tailscale name>`). Steam's Remote Play discovery also broadcasts over it. **Verified 2026-09-28**, BUILD_ID 20260925.6191901. | Frame Control's Mac stream uses it when present ([mac-in-headset.md](mac-in-headset.md)) |
 | **Tools on the image:** Python 3.12.3, `ffmpeg`, `openssl`, `curl`, `rsync`, `zip`/`unzip`, `flatpak`, `wpctl`, `podman`. **No `adb`.** `steamos` is uid 1000, in `wheel`, and sudoers has `%wheel ALL=(ALL) ALL`, so `sudo -S` takes the Developer Mode password on stdin. **Verified 2026-09-27.** | Running Frame Control's server on the Frame (`FRAME_LOCAL=1`, [iphone.md](iphone.md)) |
 | **Each Lepton instance is a podman container** named `lepton-steamlaunch-<instance id>`, labelled with its ADB port (`podman ps --format '{{.Names}} {{.Labels.adb_port}}'`). `podman exec <container> /system/bin/sh -c '…'` runs Android's shell inside it with no adb at all (used for `pidof` and `logcat` by the app tester). Running `wm size`/`wm density` that way is untested. **Verified 2026-09-27.** | `ui/frame_android.py`, the iPhone app's display settings |
 | **Asleep means off the network.** In standby the Frame stops answering on its LAN address, `frame.local` and Tailscale alike (`Host is down`, `No route to host`, timeouts), and ping fails. It was unreachable for about 2.5 hours until woken. Nothing over SSH can wake it. **Verified 2026-09-27.** | Frame Control's offline banner and retries |
@@ -59,7 +62,7 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600
 | **Battery at full on a charger** can read `Discharging` at about 0 W (for example 99 %, 0.0 W, USB-C PD 18 W). Treat under 0.5 W on a charger as "not charging", not "draining". **Verified 2026-09-27.** | Frame Control's battery card |
 | **The OS image is downloadable.** Valve's recovery images for the Frame are at `https://steamdeck-images.steamos.cloud/recovery/`. The root filesystem inside is btrfs, and it runs as an SSH test target on ARM64 Linux without the headset (`tests/frame-container/frame-image.sh`). **Verified 2026-09-27.** | [recovery-and-images.md](recovery-and-images.md) |
 | **Boot / recovery menu.** Hold Power ~10 s until the LED goes off, then power on while holding the **AUX button on top of the Power button** (not the volume keys) until a text menu appears. Entries: `Current` (SteamOS-A/B + build), `Previous` (the other A/B slot), `Boot from USB`, `Repair Steam Installation`, `Erase User Data` (factory reset), `ADB mode`, `Battery Ship Mode`. It auto-boots `Current` after a ~15 s countdown. **Volume Up/Down (left side) move, AUX (right side) selects.** For a boot loop, Valve says pick `Previous` (keeps user data); then `Repair Steam Installation`; `Erase User Data` wipes `~` (SSH keys, Tailscale, Flatpaks, T3 setup). Last resort is a full re-image, two ways: (1) USB: write `steamframe-oobe-repair-<build>.img.bz2` to an 8 GB+ USB-C stick (Balena Etcher on the Mac), pick `Boot from USB`, then use "Wipe Device & Install SteamOS" / "Repair SteamOS" (keeps games and personal content) from the recovery desktop; (2) cable/EDL: `steamframe-oobe-repair-qdl-<build>.tar.gz`, run `flash.sh` (Linux) or `flash.cmd` (Windows), then with the Frame off for 10 s hold Power + Vol Up + Vol Down for 10 s and plug it in; it reflashes and reboots. Both images: `https://steamdeck-images.steamos.cloud/recovery/` (build 20260922.5153644, 0.3.0, 3.8 GiB each, no published checksums); local copies in `~/Downloads/steam-frame-recovery/`. File names, checksums and what's inside: [recovery-and-images.md](recovery-and-images.md). Source: Valve's [SteamOS Recovery FAQ](https://help.steampowered.com/en/faqs/view/1B71-EDF2-EB6D-2BB3) and [Installation and Repair FAQ](https://help.steampowered.com/en/faqs/view/65B4-2AA3-5F37-4227), plus a menu photo in [EloiStree/HelloSteamFrame#9](https://github.com/EloiStree/HelloSteamFrame/issues/9). **Inferred** (Valve docs, 2026-09-26); not yet tried on our Frame. | Recovering from a boot loop |
-| **Boot loop cause: the SteamVR health check.** `steamvr.service` runs `/usr/share/deckard/steamvr-health-check`, which appends `frog:glasses:` to `$XDG_RUNTIME_DIR/steamvr-short-session-tracker` on every failed or <10 s SteamVR run. At 3 it runs `steam-health-check --repair-now`, which **deletes all of `~/.local/share/Steam` (games, login, Developer Mode) and `~/.steam`**, keeping only `registry.vdf`. At 4 it also tries `steamos-bootconf set-mode reboot-other` (fails as the user: `bootenv: Permission denied`). SteamVR normally fails 1–2 times per boot while it waits for the Steam client (`SteamAPI_InitEx failed … Steam is probably not running`, then `fatal stalled cross-thread pipe`). Once Steam has been wiped, it has to re-download a ~210 MB client on every boot, so SteamVR keeps failing, Steam keeps getting wiped and the Frame reboots, in a loop. Also, the Steam updater can deadlock at `Installing update...` (main process blocked writing to the `-child-update-ui` process, which is stuck in `drm_syncobj_array_wait_timeout`). Killing only the `-child-update-ui` process lets the install finish (`package/*.installed` appears). **Fix without sudo:** over USB-C ADB (`adb -s frame shell` works as `steamos` while the Frame is looping; SSH is refused once Developer Mode is lost), truncate both `/run/user/1000/steam{,vr}-short-session-tracker` files and `chmod 444` them (the health check then logs `Permission denied` and does nothing; this is tmpfs, so it resets on reboot). Unstick the updater if needed, let Steam finish installing, then hold Power 10 s and start the Frame normally. `systemctl reboot` over ADB needs interactive auth. After the fix, sign in to Steam and turn Developer Mode back on. **Verified 2026-09-26**, BUILD_ID 20260922.6101926, slot B (clean boot: 0 SteamVR failures, SSH and Tailscale back). | Diagnosing a boot loop |
+| **Boot loop cause: the SteamVR health check.** `steamvr.service` runs `/usr/share/deckard/steamvr-health-check`, which appends `frog:glasses:` to `$XDG_RUNTIME_DIR/steamvr-short-session-tracker` on every failed or <10 s SteamVR run. At 3 it runs `steam-health-check --repair-now`, which **deletes all of `~/.local/share/Steam` (games, login, Developer Mode) and `~/.steam`**, keeping only `registry.vdf`. At 4 it also tries `steamos-bootconf set-mode reboot-other` (fails as the user: `bootenv: Permission denied`). SteamVR normally fails 1–2 times per boot while it waits for the Steam client (`SteamAPI_InitEx failed … Steam is probably not running`, then `fatal stalled cross-thread pipe`). Once Steam has been wiped, it has to re-download a ~210 MB client on every boot, so SteamVR keeps failing, Steam keeps getting wiped and the Frame reboots, in a loop. Also, the Steam updater can deadlock at `Installing update...` (main process blocked writing to the `-child-update-ui` process, which is stuck in `drm_syncobj_array_wait_timeout`). Killing only the `-child-update-ui` process lets the install finish (`package/*.installed` appears). **Fix without sudo:** over USB-C ADB (`adb -s frame shell` works as `steamos` while the Frame is looping; SSH is refused once Developer Mode is lost), truncate both `/run/user/1000/steam{,vr}-short-session-tracker` files and `chmod 444` them (the health check then logs `Permission denied` and does nothing; this is tmpfs, so it resets on reboot). Unstick the updater if needed, let Steam finish installing, then hold Power 10 s and start the Frame normally. `systemctl reboot` over ADB needs interactive auth. After the fix, sign in to Steam and turn Developer Mode back on. **Verified 2026-09-26**, BUILD_ID 20260922.6101926, slot B (clean boot: 0 SteamVR failures, SSH and Tailscale back). **Seen again 2026-09-28** on BUILD_ID 20260925.6191901, beta client 1790377368. The Frame rebooted by itself while off the network. At 21:13 the check tried `reboot-other` (`bootenv: Permission denied`) and reset the unpacked Steam install, and the tracker had 15 entries. The tracker fix above, applied over SSH, stopped the resets (the checks then log `Permission denied`), but Steam itself kept exiting about 17 s after each start (34 restarts). Cause not found yet. | Diagnosing a boot loop |
 
 ## Debug recipes
 

docs/mac-in-headset.md

@@ -0,0 +1,470 @@
+# Mac in the headset
+
+Frame Control can show any Mac window, or a whole Mac screen, as its own panel
+in the Steam Frame. You place each panel anywhere in the room with the SteamVR
+dashboard. The laser clicks and drags, the thumbstick scrolls, and you type on
+the Mac's own keyboard. Find it under **Tools → Mac in the headset** (macOS
+only).
+
+The confidence labels are the same as in [ssh.md](ssh.md).
+
+## Why this design
+
+First-party options come first, as the repo's rule asks, with the reason
+each one was or wasn't chosen. The full list for every device is in
+[streaming.md](streaming.md#first-party-options-and-why-they-do-or-dont-fit).
+Checked 2026-09-28.
+
+| Goal | First-party option | Chosen? | Why |
+|---|---|---|---|
+| One Mac screen in the headset | **Apple Screen Sharing** (VNC) → Remmina (Remmina 1.4.43 is already installed on this Frame) | Kept as the fallback (`panel-on-frame.sh mac-screen`) | It's the closest to first-party and needs nothing new. But VNC sends compressed tiles rather than video, so moving content is slow: noticeable lag even on a good 5 GHz link (**verified** 2026-09-27, see [streaming.md](streaming.md)), and the Mac's pointer isn't in the picture without a helper. It shows only whole screens |
+| One Mac screen | **Steam Remote Play**, Mac as host (Valve) | For Mac games only; see [Steam's own streaming](#steams-own-streaming) | The Mac's and the Frame's Steam clients already find each other (**verified**). But Remote Play streams a game (the whole desktop only while the game is out of focus, untested from a Mac), never single windows, and a Mac can't host the Frame's VR streaming |
+| One Mac screen | **AirPlay** (Apple) | No | Apple licenses AirPlay receivers only to TV and speaker makers, and nothing official runs on Linux. UxPlay is an unofficial receiver, and it mirrors a whole screen, not single windows |
+| One Mac screen | **Sidecar / Mac Virtual Display** (Apple) | No | These work only with an iPad or Apple Vision Pro |
+| **Each Mac window as its own panel** | None | – | No first-party way does this: Apple's per-app streaming is only for Vision Pro, and Valve's desktop streaming needs a Windows SteamVR host. So Frame Control does it itself |
+| Mac keyboard and trackpad driving the headset | **Bluetooth HID** | No | macOS can't act as a Bluetooth keyboard or mouse. A real Bluetooth keyboard paired with the Frame still works |
+| Mac keyboard and trackpad | **KDE Connect** (KDE) | No | The Frame has no `kdeconnectd` and it isn't on Flathub. Its Mac app has no keyboard or mouse sharing (**inferred**), and on Wayland it can only reach the desktop panel |
+| Mac keyboard and trackpad | **xrdp** (Valve, Developer Mode) | No | It runs a separate Linux session that you view on the Mac. It isn't the headset's view, and it doesn't carry input the other way |
+
+What that leaves is our own stream: nothing to install on the Mac or the
+Frame, and whole screens or single windows. Here the Mac's own keyboard and
+trackpad need no forwarding, because the windows are still on the Mac. The
+laser is the only input that has to be sent back.
+
+Other routes that were compared:
+
+| Option | One screen | Each window | Speed | Verdict |
+|---|---|---|---|---|
+| Sunshine → Moonlight | ✓ | – | Good | Sunshine's macOS support is still experimental ([discussion #777](https://github.com/orgs/LizardByte/discussions/777)), and it captures whole screens only |
+| Virtual Desktop, Immersed | – | – | – | No Frame client as of September 2026 |
+| **Frame Control's own stream** | ✓ | ✓ | Hardware H.264, sending only changed frames | **Built** |
+
+To type into VR surfaces other than these panels (SteamVR's dashboard,
+games), the Frame supports a uinput keyboard and mouse without sudo
+(verified 2026-09-27: `steamos` is in `input`, and `/dev/uinput` is
+`root:input 660`). That's a separate feature, not part of this one.
+
+## Steam's own streaming
+
+The Frame is built around Steam streaming, so this was checked first
+(2026-09-28). It fits Mac games, not Mac windows.
+
+- **VR streaming from the Mac: no.** The Frame streams VR from a PC running
+  SteamVR ("Steam Link" with foveated streaming). SteamVR dropped macOS in
+  2020, and Valve lists PCs, laptops, Steam Deck and Steam Machine as
+  hosts, never a Mac (**documented**:
+  [UploadVR](https://www.uploadvr.com/steamvr-drops-mac-support/),
+  [Road to VR](https://roadtovr.com/steam-frame-game-certification-specs/)).
+- **Flat Remote Play from the Mac: probably, for Steam games.**
+  - Steam on this Mac has streaming on, and the two Steam clients already
+    see each other. The Frame's `remote_connections.txt` shows it
+    connecting directly to "Alexs-MacBook-Pro-7" at 192.168.1.211:27036,
+    and the Mac's shows the Frame connecting over Wi-Fi and over the USB-C
+    link (**verified** in both clients' logs).
+  - Whether a stream then starts, and how a flat game looks in the headset
+    (reviews describe a theater screen), is **not tested yet**. The Frame's
+    Steam was crash-looping during this session (below).
+  - Mac-hosted Remote Play has a long-standing report of the stream
+    closing as the game loads
+    ([Steam forum](https://steamcommunity.com/groups/homestream/discussions/1/574921459914429988/),
+    **reported**).
+- **The Mac desktop through Steam: untested; single windows: no.** Valve
+  says Remote Play shows the host's desktop when the game loses focus
+  ([Steam Remote Play FAQ](https://help.steampowered.com/en/faqs/view/0689-74B8-92AC-10F2),
+  **documented**), so a whole Mac screen may be reachable by starting a
+  game, then switching away from it. Nobody has tried that from a Mac
+  host. It would still be one screen in one panel: Remote Play has
+  nothing like one panel per Mac window, so Frame Control's own stream
+  stays the way to see separate windows.
+- **Steam has a "stream desktop" call, and it pairs with a Mac
+  (verified 2026-09-28).** In the Remote Play device list, the Frame's
+  Steam UI calls `SteamClient.RemotePlay.StartDesktopStream(<client id>)`
+  for a connected device. Called over CDP with the Mac's client ID, it made
+  the Mac's Steam show "Authorize Device" and ask for a 4-digit code shown
+  on the Frame. Once the code was entered, the Mac logged
+  `k_ERemoteDeviceAuthorizationSuccess`. No stream started in that attempt,
+  and a second attempt, now that the device is authorized, is the next
+  test. If it streams the Mac's desktop, that's a whole-screen option built
+  into Steam: one panel, Valve's encoder and transport. It still wouldn't
+  give each window its own panel.
+- **What Steam's work did give us: the USB-C link.** Plugged into the Mac,
+  the Frame appears as a network port called "Steam Frame". Steam's Remote
+  Play discovery uses it, and so does Frame Control's stream now (see
+  "USB-C, when it's plugged in" below).
+
+Next steps, once Steam on the Frame is healthy:
+
+1. Call `StartDesktopStream` again now that the Frame is authorized, and
+   compare its latency and sharpness with Frame Control's stream of the same
+   screen.
+2. Stream the one Mac game installed here (Fortune Mill) from the Frame's
+   library, over Wi-Fi and over USB-C.
+3. Record whether it starts, how it's shown, and its latency. Steam's
+   streaming overlay shows this; our benchmark can't measure it.
+4. While streaming, switch away from the game on the Mac, and see whether
+   the Mac's desktop appears in the headset, and whether its keyboard and
+   pointer work.
+5. If it works, Frame Control's Games page could offer "Stream from the
+   Mac" for Mac-installed games.
+
+## How it works
+
+```
+Mac                                              Frame
+ScreenCaptureKit (one window or display)
+  → VideoToolbox H.264 (hardware, low-latency,
+    no B-frames)
+  → frame-mac-view, 127.0.0.1 ──ssh -R──→ 127.0.0.1:479xx
+                                             → Chromium app window per stream
+                                               (WebCodecs decode), on gamescope's
+                                               X display, tagged STEAM_GAME
+                                             → its own SteamVR panel
+  ← CGEvent (clicks, drags, wheel, keys) ←──── pointer, wheel and key events
+```
+
+- **The agent** is `mac/bin/frame-mac-view`, built from `mac/frame-mac-view`
+  (Swift, no dependencies; `build.sh`). Frame Control's server starts it on
+  first use and stops it on quit.
+  - It captures with ScreenCaptureKit, which sends frames only when something
+    changes, so idle windows cost nothing.
+  - It encodes in hardware with VideoToolbox's low-latency rate control (plain
+    real-time mode where that's unavailable).
+  - While anyone is watching, it keeps the Mac's display awake. A sleeping
+    display isn't drawn, so there would be nothing to capture.
+- **The link** is an `ssh -R` tunnel on its own connection. It's encrypted and
+  works anywhere `ssh frame` works, Tailscale included, with no firewall
+  changes on the Mac. If the headset sleeps or the network drops, Frame
+  Control reopens the tunnel on the same port, and open viewers reconnect by
+  themselves.
+- **USB-C, when it's plugged in.** Connected to the Mac by cable, the Frame
+  is also a USB network device: macOS lists a network port called "Steam
+  Frame", and the Frame's `usb0` answers in under 1 ms. Frame Control
+  checks for it each time it opens the tunnel and uses it when it's there,
+  with the Frame's usual SSH host key. Otherwise it uses the normal path.
+  `FRAME_MACVIEW_USB=0` turns this off. **Verified** 2026-09-28, in two
+  interleaved pairs of runs:
+
+  | | USB-C | Wi-Fi (Tailscale) |
+  |---|---|---|
+  | test: content p50 / p95 | 6.9–7.3 / 8.4–8.8 ms | 9.8–10.1 / 12.1–12.3 ms |
+  | test: click to drawn p50 | 16.6–16.9 ms | 26.4–27.8 ms |
+  | scroll: content p95 | 23.0–23.5 ms | 31.4–36.9 ms |
+  | scroll: late frames | 3.2–3.3% | 4.7–7.0% |
+
+  (`bench/results/2026-09-28-*-usb1.json`, `-usb2`, `-wifi1`, `-wifi2`.)
+- **Access.**
+  - Frame Control's own key never leaves the Mac.
+  - Each viewer is opened with a **single-use ticket**. It's tied to one
+    window or display and expires after a minute. It's spent as soon as the
+    viewer confirms it has received its reconnect key. Until then, a retry
+    gets the same key, so a connection lost at that moment doesn't strand
+    the viewer. Stop revokes tickets that haven't been used yet.
+  - After that, the viewer holds a reconnect key for that one source, in
+    memory only. **Stop** revokes it.
+  - Remaining risk: a program running as `steamos` on the Frame could read a
+    ticket from Chromium's command line in the first second or so and use it
+    first. That gets it the one source being opened, not the Mac, and the
+    real viewer would then fail to connect. Android apps in Lepton run in
+    their own podman container, so they shouldn't see the Frame's process
+    list (inferred, not checked).
+- **The viewer** is `ui/mac-view.html`, served by the agent. It opens on the
+  Frame as a Chromium app window, preferring Chromium XR (`~/chromium-xr`,
+  built with H.264) over Flathub Chromium.
+  - The page puts `[fcNNNNN]` in its title. The launcher finds the window by
+    that tag and sets `STEAM_GAME` to a stable id per source, which gives it
+    its own panel (see [panels.md](panels.md)). The same Mac window gets the
+    same panel id each time.
+  - It decodes with WebCodecs. If it falls behind, it skips to the next
+    keyframe instead of showing old frames late.
+  - It falls back to JPEG stills (**Compatible** quality) where H.264 isn't
+    available.
+- **Flow control.** The agent never lets frames queue up anywhere on the
+  way. It skips capture frames *before* encoding, so no reference frame goes
+  missing, and it lowers the bitrate, then the frame rate, then the size, to
+  fit the link (see [Adapting to the network](#adapting-to-the-network)).
+- **Input.**
+  - A click on a window's panel brings that Mac window to the front
+    (Accessibility API), then clicks at the same point. Double clicks, right
+    clicks, drags and the wheel work too.
+  - Keys typed into the panel are sent as Mac key codes. Any keys or buttons
+    still held down are released if the viewer loses focus or disconnects, or
+    when the stream stops. Characters the key
+    table doesn't know, such as those from other keyboard layouts, are typed
+    as text.
+  - The Mac's own keyboard and trackpad keep working as normal. Click a panel
+    with the laser, then type on the Mac.
+
+## Permissions (Mac)
+
+- **Screen Recording**, to see windows. Without it, the card asks for it.
+- **Accessibility**, so input from the headset reaches the Mac. Without it the
+  stream still works, and the viewer says clicks won't go through.
+
+Both are granted to Frame Control. After granting, press **Refresh**, which
+restarts the helper so it picks them up. The app is ad-hoc signed, so macOS
+may ask again after an update.
+
+## Quality settings
+
+| Setting | Long side | fps | Codec | Use |
+|---|---|---|---|---|
+| Sharp | 2560 | 60 | H.264, ~0.14 bits/pixel | Text-heavy windows on a strong link |
+| Balanced (default) | 1920 | 60 | H.264, ~0.1 bits/pixel | Most things |
+| Light | 1280 | 30 | H.264 | Weak Wi-Fi or Tailscale off the LAN |
+| Compatible | 1280 | 20 | JPEG | A Frame browser without H.264 |
+
+## Measuring
+
+Every frame carries a sequence number, and the agent records its journey on
+the Mac's clock (`Sources/Stats.swift`):
+
+| Stage | From → to |
+|---|---|
+| capture | the Mac composited it (ScreenCaptureKit's display time) → the agent got it |
+| queue, encode | → encoding started → the encoder finished |
+| network | → the viewer received it |
+| decode, draw | → WebCodecs decoded it → it was drawn on the page's canvas |
+| present | → the page's next animation frame |
+
+- **Clock sync.** The viewer syncs its clock to the Mac's the way NTP does:
+  it pings over the stream's own WebSocket and keeps the sample with the
+  shortest round trip. It then reports, in Mac time, when each frame arrived
+  (right away, so the agent can pace itself) and when it was decoded and
+  drawn (in batches every 250 ms).
+- **Input.** The first frame captured after a click or key carries that
+  event's id. So input latency is the viewer's event → injected on the Mac →
+  the first frame after it → drawn in the headset.
+- **Where to see it.**
+  - `GET /stats` (key required) returns every frame and input record.
+  - `/status` includes a two-second summary, which Frame Control's card
+    shows next to each live stream.
+  - In the headset, add `?stats=1` to the viewer or press
+    Ctrl+Alt+Shift+S for an overlay.
+- **The benchmark.** `scripts/macview-bench.py` runs fixed scenarios on the
+  real Frame, from the Mac, with nobody wearing the headset:
+  - **test** is the moving test pattern.
+  - **scroll** is a Chrome page on its own display scrolling at 240 pt/s,
+    which gives about 9 Mbit/s of real 1920×1290 video.
+  - **type** types into a Chrome text box, first fast and then with pauses.
+
+  It writes `bench/results/<date>-<commit>-<label>.json`, compares two
+  results, and runs interleaved A/B tests between agent settings (`ab`).
+  Wi-Fi changes from minute to minute, so single runs at different times
+  aren't comparable. Throttled links come from a shaping relay on the Mac,
+  which needs no sudo (`--net 50@0,3@8,50@16` means 50 Mbit/s, then 3 from
+  8 s, then 50 from 16 s).
+- **What's graded.** "Content" runs from when the Mac composited a frame
+  (or when ScreenCaptureKit delivered it, if that was earlier) to when it was
+  drawn in the viewer. The Frame's compositor adds its own delay after that.
+  That part is reported, but not graded: an unworn Frame throttles panels to
+  about 36 fps after a few seconds, and to 15 fps in standby, whatever they
+  draw. This was **verified** with a local canvas page that ran on the Frame
+  with no network involved (`bench/pages/present.html`). The compositor's
+  share needs a run with the headset worn.
+
+Targets: content p50 ≤ 25 ms (p95 ≤ 40), click to photon p50 ≤ 50 ms
+(p95 ≤ 70), 60 fps with ≤ 1% late frames, no stall over 100 ms, and adapting
+to a new link rate within 1 s.
+
+Baseline on 2026-09-28 (**verified**, home Wi-Fi, Tailscale, Balanced,
+`bench/results/2026-09-28-a3c6e5c-dirty-baseline-fixed.json`; ms p50/p95):
+
+| Scenario | Content | Input to drawn | fps drawn | Notes |
+|---|---|---|---|---|
+| test (1280×720) | 9.9 | 28.8 / 39.0 | 59 | encode 4.0, network 4.0, decode 1.3 |
+| scroll (1920×1290) | 14.7 | – | 50.4 | encode 6.7, decode 6.4 (software), 9.4 Mbit/s, 16% late |
+| type (1920×1290) | 16.7 | 51.2 / 73.2 | – | most of the input time is the Mac app reacting |
+
+After this work, with the controller on (**verified**, same setup,
+`bench/results/2026-09-28-9e4dcdd-final.json`; ms p50/p95). Content is
+now measured from the earlier of display time and delivery, which adds
+about 5 ms to scroll compared with the baseline's way of measuring:
+
+| Scenario | Content | Input to drawn | fps drawn | Grades |
+|---|---|---|---|---|
+| test | 10.5 / 16.7 | 29.6 / 36.3 | 60 | all within target |
+| scroll | 19.8 / 27.4 | – | 55.9 | fps, late frames (4.8%) and worst gap (222 ms) only "acceptable": Wi-Fi stalls (the Mac captured 57 fps in this run; earlier runs got 46–53 from virtual displays) |
+| type | 15.0 / 21.4 | 43.0 / 60.5 | – | all within target |
+
+Of the targets, click to photon is met without the Frame's compositor (the
+headset has to be worn to measure its share), and so is content latency.
+The frame-rate and no-stall targets aren't yet met while scrolling.
+
+**Worn** (**verified** 2026-09-28 22:00, `vrcmd --stats` activity level 1,
+home Wi-Fi over Tailscale, `bench/results/2026-09-28-39afb23-worn.json`;
+ms p50/p95):
+
+| Scenario | Content | Input to drawn | fps drawn | What happened |
+|---|---|---|---|---|
+| test | 10.8 / 15.3 | 25.7 / 34.4 | 58 | as unworn |
+| scroll | 22.2 / 141 | – | 37.7 | Wi-Fi queued 56–364 ms and held frames for 220–270 ms; the controller went to 2.6 Mbit/s and 45 fps |
+| type | 13.5 / 24.3 | 56.8 / 106 | – | slower replies than unworn (43 / 60) |
+
+The Frame's CPU wasn't the limit: about 27% in total, and the viewer took
+45% of one core. The link to a headset on someone's head is much rougher
+than to one lying still. The adaptation keeps latency bounded there, but
+the frame rate drops. The USB-C cable avoids Wi-Fi entirely. Frames
+actually shown were still about 39 fps for the test pattern while worn, so
+the unworn throttling isn't the whole story. The cause is **unknown**.
+
+What was learned (all **verified**, unless marked):
+
+- The biggest costs are encoding (4–7 ms), network (4–6 ms), and decoding
+  on the Frame. Chromium XR on the Frame decodes H.264 in **software**.
+- Wi-Fi alone stalls for 240–580 ms now and then, over Tailscale and over
+  the LAN alike. Over the LAN (`--host 192.168.1.237`) latency was no
+  better, but the Frame used about 8% less CPU, because tailscaled runs in
+  userspace there.
+- With no controller, a link that slows down queues without limit. In one
+  run, frames arrived 1.9 s late, and at worst 9.4 s late.
+- Tried, and no help, so not kept: VideoToolbox options (require hardware,
+  no frame delay, prioritise speed, a hard data-rate cap), Chromium flags
+  (`--disable-gpu-vsync`, `--disable-frame-rate-limit`,
+  `--use-angle=vulkan`), and a 120 Hz virtual display.
+- Inconclusive, so off by default: keeping the Frame's Wi-Fi awake during
+  typing (`FRAME_MAC_VIEW_WARM=40`, a tiny message every 40 ms for 5 s
+  after input). Over three interleaved runs each, input p50 went 44 → 47 ms
+  and p95 92 → 71 ms, and the ranges overlapped widely
+  (`…-ab-keepwarm.json`). In that run, and in the baseline's typing, the
+  harness typed spaces as "+" (a URL-encoding bug, since fixed), so they
+  went through the text path rather than as space keys.
+- Pointer moves now go out on an 8 ms timer, not on the page's next
+  animation frame, which an unworn Frame slows to 15–36 Hz. This is
+  **inferred** to help dragging; the benchmark has no drag scenario yet.
+- Kept: the encoder's timestamps never jump more than two frame intervals.
+  Before this, the first frame after a pause got a quarter of a second's
+  bit budget, and one P-frame reached 204 KB.
+
+## Adapting to the network
+
+`Sources/Controller.swift`, per stream, latency first:
+
+- **The gate.** The viewer acknowledges every frame as it arrives. A new
+  frame is sent only while the oldest unacknowledged one is younger than
+  the path's usual round trip, plus one frame interval, plus room for this
+  link's normal jitter (1.5 times its recent spread, 25–80 ms). So frames
+  never queue in SSH, TCP or the Wi-Fi driver. While the link is stuck, the
+  newest picture waits and goes out as soon as it moves.
+- **The bitrate.** The link counts as congested when, for two checks in a
+  row (100 ms apart), round trips grow by more than 40 ms while the stream
+  uses much of its budget, or the gate holds frames back, or a frame is
+  stuck for 100 ms. Then the bitrate drops to a bit under what actually got
+  through: at least a fifth off, and at most half. Once the link has been
+  clear for a second, it rises by 10% steps, never above the quality
+  setting's bitrate.
+- **The tier.** When the bitrate stays low, and the stream is really
+  limited by the link rather than having little to send, it steps down:
+  60 → 45 → 30 fps, then 75%, then 50% of the pixels. It goes straight to
+  the tier the bitrate supports after half a second, and steps back up one
+  tier at a time after two seconds with room to spare.
+- `FRAME_MAC_VIEW_ADAPT=0` turns it off, for comparison.
+
+Measured on the real Frame, 2026-09-28 (**verified**; interleaved A/B, off
+versus on, medians of the runs, ms):
+
+| Link | Scenario | Content p95, off → on | fps drawn, off → on | Result file |
+|---|---|---|---|---|
+| Clean Wi-Fi (3 runs each) | scroll | 32.3 → 33.6 | 57 → 56.2 | `…-ab-adapt-clean2.json` |
+| Clean Wi-Fi | test | 15 → 14.2 | 60 → 59.7 | same |
+| 50 → 3 → 50 Mbit/s at 8 s and 16 s (2 runs each) | scroll | **4670 → 72** | 45 → 42 | `…-ab-adapt-step3.json` |
+| 50 → 3 → 50 Mbit/s | test | 66 → 16 | 60 → 60 | same |
+
+- **Clean link.** On a clean link it costs nothing measurable. An earlier
+  version with a fixed gate slack lost 9 fps to Wi-Fi jitter while
+  scrolling (47 → 38 fps), and that's why the slack now follows the link's
+  jitter.
+- **Throttled link.** Without the controller, frames queued for up to 5.6 s
+  and never caught up while the link was slow (p95 1.8–5.6 s, second by
+  second). With it, in the two runs:
+
+  | | Run 1 | Run 2 |
+  |---|---|---|
+  | Worst second's p95 just after the drop | 219 ms | 428 ms |
+  | p95 back under 100 ms for 3 s in a row | after 1 s | after 4 s |
+  | Stepped down to 1440 px at 30 fps | 1.8 s after the drop | 3.5 s after |
+  | p95 per second after that, on the 3 Mbit/s link | 55–94 ms | 60–148 ms |
+
+  Sending one frame takes about 27 ms on that link by itself. After the
+  link recovered, the stream was back at full size and 60 fps in about
+  7.5 s. It steps up one tier every two seconds, on purpose, so it doesn't
+  bounce. The 1 s adaptation target was met in one run of two.
+- **A hiccup on a small stream.** In one clean run, a Wi-Fi hiccup made an
+  earlier version halve the test pattern's bitrate five times and drop it
+  to half size. That cut couldn't help: the stream only sends
+  0.47 Mbit/s. Now the controller estimates what a stream wants (captures
+  per second × average frame size). While a stream wants about half its
+  budget or less, no cut takes it below twice what it wants, so it doesn't change
+  tier.
+
+## Checked so far (2026-09-28)
+
+- **Mac (checked by hand, macOS 26.5.2).**
+  - The agent builds, and lists windows and displays.
+  - In a browser, the test pattern decoded at about 60 fps (H.264) and 30 fps
+    (JPEG, about 22 Mbps).
+  - A click in the viewer arrived at the same point in the source.
+  - `pmset -g assertions` showed the display-awake assertion only while a
+    stream was being watched.
+- **Automated** (`tests/test_macview.py`, on CI's macOS runner): the agent
+  builds (a build failure fails the job).
+  - `/ping` and the page are open; everything else needs the key.
+  - Tickets work once and only for their own source, and Stop revokes
+    reconnect keys.
+  - A WebSocket frame claiming 2^63 bytes closes that socket, and the agent
+    keeps running.
+  - A stream sends an SPS-led H.264 keyframe, and sends another when asked.
+  - Stop ends the stream on the Mac even if the viewer ignores it.
+  - The test doesn't decode video, time it, or check where clicks land.
+- **Linux aarch64 (verified in a stand-in, not on the Frame).** In an Arch
+  Linux ARM container with sshd, Xvfb as `:0` and Chromium 153:
+  - The real `show` path worked in 1–2.3 s: tunnel, ticket, launcher,
+    window found and tagged `STEAM_GAME`.
+  - Frame Control's key didn't appear anywhere in the stand-in's process
+    list.
+  - After the tunnel was killed, it came back on the same port within 4 s,
+    and the viewer reconnected by itself.
+  - Chromium decoded the stream in software with the GPU off.
+  - Stop closed the window, and Chromium exited.
+  - This test found and fixed a bug: in a C locale, `xwininfo` can't print a
+    title with non-ASCII characters, so the launcher reads `_NET_WM_NAME`
+    with `xprop`.
+- **On the Frame (verified 2026-09-28, SteamOS build 20260925.6191901,
+  from the Mac, nobody wearing the headset).** Frame Control's **Show** with
+  the test pattern:
+  - The panel was ready in 1.45 s. SteamVR logged `[Overlays] Created:
+    valve.steam.desktopgame.2001639889` (in `vrwebhelper_systemui.txt`).
+  - The window was tagged `STEAM_GAME`, and gamescope sized it to 1920×1080
+    although 1280×720 was asked for.
+  - Chromium XR decoded it live at about 60 fps: the frame counter advanced
+    62 in 1.04 s.
+  - Over home Wi-Fi, the frames in the viewer window had been drawn on the
+    Mac about 11–17 ms earlier, plus `xwd`'s own time. That's measured
+    against the two clocks, which were 37–39 ms apart (±4 ms, measured over
+    one SSH session). SteamVR's compositor and the display come on top.
+  - Clicks and keys injected with XTest into gamescope's Xwayland didn't
+    reach the page. That's inconclusive, not a failure: XTest on gamescope
+    isn't how real input arrives. The laser should arrive as a left mouse
+    button and the thumbstick as a wheel, because the window has an app id
+    (from gamescope's source; see [panels.md](panels.md)).
+- **Not yet checked:**
+  - Clicking, dragging and scrolling with the laser while wearing the
+    headset.
+  - Real window capture and input on a Mac with both permissions granted.
+  - Keys from SteamVR's on-screen keyboard.
+  - Whether Flathub Chromium has H.264. Chromium XR is used when it's
+    installed, as it is on this Frame.
+  - Latency with real, busy windows at Sharp.
+  - A benchmark run while wearing the headset. Only then does the Frame show
+    panels at full rate, so only then can the compositor's share of the
+    latency, and the frame rate you actually see, be measured.
+
+## Limits
+
+- Only windows on the Mac's current desktop (Space) are listed, and
+  minimised windows can't be captured.
+- A window's panel shows only that window. Its menus and sheets are separate
+  windows on the Mac, so open them from the Mac or use **Whole screen**.
+- Keys go to whichever Mac window is in front. Clicking a panel brings its
+  window to the front first.
+- Ctrl stays Ctrl. On the Mac, copy is ⌘C, so use Meta+C on a keyboard paired
+  with the Frame.

docs/mac-window-separation.md

@@ -0,0 +1,200 @@
+# Real separation of Mac windows in the headset
+
+Research and experiments on 2026-09-28 (macOS 26.5.2, Apple M5 Pro), for
+making each streamed Mac window truly independent. Builds on
+[mac-in-headset.md](mac-in-headset.md). Labels: **verified** (tried here),
+**documented**, **source** (read in someone's code) or **reported**.
+
+## What "separate" lacks today
+
+Today each window is captured on its own
+(`SCContentFilter(desktopIndependentWindow:)`), but it still lives on the
+Mac's one desktop:
+
+- **Clicks.** A click has to raise the window first, so it reorders the
+  Mac's windows, steals focus and moves the real cursor.
+- **Child windows.** A window's menus, popovers, sheets and tooltips are
+  separate windows, so they aren't in its panel. Apple documents that the
+  single-window filter leaves them out.
+- **Size.** A panel's size is tied to the window's size on the Mac screen.
+- **Hidden windows.** Minimised windows, and windows on other Spaces, can't
+  be shown live.
+
+## How the Mac draws, and where pixels can be read
+
+Apps draw with Core Animation into IOSurfaces. WindowServer's compositor
+stacks every window onto each display, including virtual ones, and sends
+the result to the screen. Pixels can be read at three points:
+
+| Where | How | Gets | Doesn't get |
+|---|---|---|---|
+| One window's content | ScreenCaptureKit `desktopIndependentWindow` (public, what we use) | Live, zero-copy, even when covered by other windows (**documented**) | Menus, popovers, sheets. It pauses while the window is minimised (**reported**) |
+| One window plus its children | `SCStreamConfiguration.includeChildWindows` (macOS 14.2+, **reported**) | Menus, popovers and sheets attached to the window | Anything the app puts outside the window's bounds |
+| A whole display | ScreenCaptureKit display filter, with apps or windows included or excluded | Everything on that display, cursor included | Only what's on that display |
+| A snapshot of any window | Private `CGSHWCaptureWindowList`, which AltTab uses for minimised windows and other Spaces (**source**: `alt-tab-macos` `PrivateApis.swift`) | Minimised windows and other Spaces | It's one still image, not a live stream |
+| Another app's layer tree | Private `CALayerHost` with a context id | A live, zero-copy picture | It only works when the other app cooperates, so it's no good for arbitrary windows (**reported**) |
+
+`CGWindowListCreateImage` and `CGDisplayStream` are obsolete in the macOS 15
+SDK (**reported** by MacPorts and JUCE). Nothing reads another app's pixels
+without the Screen Recording permission.
+
+## The idea: each window gets its own virtual display
+
+A virtual display (the private `CGVirtualDisplay`, as used by BetterDisplay,
+DeskPad and quest-display) is a compositor target with no physical screen.
+Put one streamed window alone on its own virtual display, sized to its
+panel, and capture the whole display:
+
+- **Nothing can overlap it.** A plain click at that point always lands on
+  that window, so input doesn't need raising or background-event tricks.
+- **Menus, sheets, popovers, tooltips and context menus appear on the same
+  display, so they're in the panel.** With "Displays have separate Spaces"
+  on (it is on this Mac), each display also has its own menu bar, so the
+  app's menu bar can be part of the panel.
+- **The panel's size is the display's size,** in HiDPI. Resizing the panel
+  means changing the display mode and resizing the window to fill it
+  (Accessibility API).
+- **Minimised windows and other Spaces stop being a problem,** because
+  streamed windows live on their own displays.
+- **The cursor goes where the laser points.** That display's panel is the
+  one being used, much as Vision Pro's Mac Virtual Display works. Keys from
+  the Mac's keyboard go to the window last clicked.
+
+### Verified here (no permission needed)
+
+- **Creating one.** It needs no permission or entitlement. 1920×1080 HiDPI
+  gives a 3840×2160-pixel, 60 Hz display, placed next to the built-in
+  screen, and `NSScreen.screensHaveSeparateSpaces` was true.
+- **Many at once.** 16 were created at once with no error.
+- **Placement.** `CGConfigureDisplayOrigin` far away failed with error
+  1014: macOS keeps displays edge to edge. Disabling one with the private
+  `CGSConfigureDisplayEnabled` from a command-line tool also failed with 1014.
+- **Removal is deferred while the physical screen sleeps.** With the
+  built-in display asleep, virtual displays were not removed when released,
+  or when their process exited, even from their own `.app`. A second display
+  with the same vendor, product and serial couldn't be created. All of them
+  disappeared as soon as the screen woke (`caffeinate -u`). The helper must
+  therefore:
+  - keep a fixed pool of displays and reuse them rather than create new ones;
+  - keep the screen awake while they exist, which it already does while
+    streaming.
+
+### Built: "Give each window its own display"
+
+Frame Control's helper now does this (`mac/frame-mac-view/Sources/Separate.swift`),
+and it's the default in the Tools card. Streams of this kind are named
+`separate:<window id>`.
+
+- For each window shown, it creates a HiDPI virtual display. The display is
+  sized to the window plus a menu bar, with a fresh serial each time, so a
+  display left over while the screen slept can't block a new one.
+- The window is moved onto the display and resized to fill it (Accessibility
+  API), and the whole display is captured.
+- On **Stop** the window goes back where it was, and the display is released.
+- Plain per-window capture is still there: untick "Give each window its own
+  display". Separate mode needs Accessibility (to move the window), and
+  without it, Show says so rather than quietly falling back.
+
+Verified 2026-09-28 (macOS 26.5.2, M5 Pro), using a TextEdit test document
+and the benchmark's Chrome windows:
+
+- **Separation works.** The window moved onto its own display and streamed,
+  with its first frames in 3.8 s. The display showed TextEdit's own menu bar.
+- **Child windows come along.** A context menu and the Page Setup sheet
+  opened on the same display and were in the capture.
+- **Input.** Typing through the stream reached the window. The benchmark's
+  typing scenario does this on every run.
+- **Stop.** Stop put the window back at exactly its old size (656×422), and
+  the display was removed.
+- **The helper must run a real Cocoa event loop.** Earlier, it ran only a
+  `RunLoop`. With that, AppKit never learned about new displays:
+  - `NSScreen.screens` never listed them, so placing the window always waited
+    3 s and then gave up;
+  - the display's HiDPI mode never applied, so windows were captured at 1×
+    (1280×860 instead of 1920×1290 pixels) and text was soft.
+
+  Running `NSApplication` (with no Dock icon) fixed both. The screen now
+  appears within 100 ms, and captures are at 2× (**verified** in
+  `bench/results/*-baseline.json` against `*-baseline-fixed.json`).
+- **Frame rate.** Chrome drew at 60–61 fps on the virtual display, but
+  ScreenCaptureKit delivered only about 46–53 fps from it, whereas the
+  built-in ProMotion screen gave 121 fps (**verified**). Neither a 120 Hz
+  virtual display (`FRAME_MAC_VIEW_VD_HZ=120`) nor a looser
+  `minimumFrameInterval` changed that. Cause unknown.
+- **Windows that keep their own size** (Calculator, 230×408). The window
+  moved and streamed, but stayed small in the display's corner, so the panel
+  was mostly wallpaper. Now only that corner is captured
+  (`SCStreamConfiguration.sourceRect`): the menu bar and the window, at least
+  480×360 points so menus fit. Clicks map to the cropped area. The first
+  frame arrived in 0.6–1.1 s, and on Stop the window went back and the
+  display was removed. A display's menu bar names the active app, so it
+  shows the window's own menus only once the panel has been clicked.
+- **Several windows at once** (on the Mac, with a local stand-in viewer that
+  acknowledges frames). Three and four Chrome windows, each scrolling on its
+  own display at 1920×1290 pixels, streamed at 53–56 fps and about
+  9 Mbit/s each. The helper used 20% (three) or 24% (four) of one CPU core.
+  The hardware encoder is shared: its time per frame went from 6.7 ms for
+  one stream to 7–15 ms for three and 11–25 ms for four, so each extra
+  moving window adds latency to the others. Once in four runs, before a fix,
+  one window left its display when several displays appeared at the same
+  moment, and its stream stopped: nothing on that display was changing any
+  more. The helper now checks every second and puts the window back. Three
+  further runs with four windows were clean, and every display was gone
+  afterwards (checked with `CGGetOnlineDisplayList`).
+- **Quitting.** On SIGTERM, or when Frame Control goes away, the helper ends
+  every stream first, so windows go back before their displays disappear.
+  Verified: a 900×600 window was back at 900×600 after SIGTERM. Closing a
+  viewer 0.05–1.5 s into startup left the window at its old size and no
+  extra display.
+- **A consent prompt.** macOS 26 asks whether to let ScreenCaptureKit apps
+  "bypass the system private window picker". The prompt appeared *on the
+  virtual display*, so it would show up inside the headset panel. Allow it
+  once on the Mac.
+
+### Still to check
+
+1. That a context menu opens over the text being clicked, not just somewhere
+   on the display. This needs a retest after the consent prompt above is
+   allowed.
+2. Stage Manager, which is reported to undo Accessibility resizes.
+3. Where the Dock and the cursor go on a virtual display.
+4. Closing the lid with virtual displays attached (clamshell).
+5. Many moving windows at once in the headset: whether to lower the bitrate
+   or frame rate of panels you aren't using, so the one you are using keeps
+   the encoder to itself.
+6. All of it in the headset, with the laser.
+
+## Input without disturbing the Mac (a finer option)
+
+For windows that stay on the Mac's own screen, input can go to a window
+behind others without raising it:
+
+- **Background click.** `CGEventPostToPid` with the CGEvent fields 91 and
+  92, which say which window a click is for (`kCGMouseEventWindowUnderMousePointer`
+  and `...ThatCanHandleThisEvent`), plus a window-relative location
+  (**reported**, reverse-engineered, needs testing).
+- **Focus without raise.** yabai makes a window key without raising it by
+  posting event records with the private `SLPSPostEventRecordTo` (**source**:
+  `yabai/src/window_manager.c`).
+- **Known failures.**
+  - Chromium and Electron ignore background clicks unless they're built
+    with the 2026 `acceptsFirstMouse` fix (electron/electron#54493).
+  - Games and canvas apps often need real activation.
+  - Password fields (Secure Event Input) drop synthetic keys.
+  - IME composition, as for Chinese or Japanese input, isn't reliable.
+
+With a virtual display per window, most of this isn't needed. It's worth
+having for hovering over one panel while typing in another.
+
+## Encoding and transport
+
+- **Codec.** Keep H.264 4:2:0. Apple's High Performance Screen Sharing uses
+  4:4:4 over two virtual displays (**documented**), but the Frame's Chromium
+  decodes H.264 in software, and no 4:4:4 decode path is confirmed there.
+- **Sharper static text.** When a panel has been still for a moment, send a
+  high-quality refresh, either a keyframe at low quantisation or a lossless
+  WebP overlay, so static text is sharp. Moving content stays as video.
+- **Transport.** Keep WebSocket over SSH for now, as it works everywhere.
+  WebRTC (UDP, congestion control) is the proven step up for Wi-Fi.
+  WebTransport over QUIC is promising, but its server side on macOS is
+  unverified.

docs/open-questions.md

@@ -130,6 +130,19 @@ Still open: 4, 6, 7, 12–15, 16 (off-LAN and after a reboot), 17–21.
 - The Mac EDL flashing script in `~/Downloads/steam-frame-recovery/` hasn't
   been run against a Frame.
 
+## Mac in the headset (2026-09-28)
+
+The test pattern streams to the Frame as its own panel at about 60 fps
+(verified, build 20260925.6191901; see
+[mac-in-headset.md](mac-in-headset.md#checked-so-far-2026-09-28)). Still to
+check in the headset:
+
+- Laser clicks, drags and thumbstick scrolling in a viewer panel.
+- Real window capture and input once Screen Recording and Accessibility are
+  granted to Frame Control.
+- Keys from SteamVR's on-screen keyboard.
+- Whether Flathub Chromium decodes H.264 (otherwise use Compatible).
+
 ## Unconfirmed claims made in these docs
 
 - `/home` and `/etc` persist across Frame OS updates. This is inferred from

docs/streaming.md

@@ -38,7 +38,8 @@ flat 2D desktop streaming into a window on the Frame's Linux desktop.
 
 | Option | Setup | Confidence | Verdict |
 |---|---|---|---|
-| **macOS Screen Sharing (VNC) → Remmina on the Frame** | **Mac:** System Settings → General → Sharing → Screen Sharing on → (i) → enable "VNC viewers may control screen with password". **Frame:** `./scripts/install-apps.sh remmina` from the Mac, then open Remmina in the headset and connect to `vnc://<mac>.local` | **Verified 2026-09-27** (Frame BUILD_ID 20260925.6191901, macOS 27.0), in its own panel via `panel-on-frame.sh mac-screen`. Remmina is on Flathub for **aarch64** with VNC and RDP ([Flathub](https://flathub.org/apps/org.remmina.Remmina)). The Frame desktop runs Flatpaks ([UploadVR](https://www.uploadvr.com/flatpaks-open-source-steam-frame/)). macOS VNC is built in. | **Recommended.** Nothing to install on the Mac, and it's easy to set up. Noticeable lag, even at lower Remmina quality settings on a good 5 GHz link, where neither Wi-Fi nor the Frame's CPU was the bottleneck. Usable for reading and coding, but not for games. You'll type the Mac's hostname once in Remmina on the headset, then save the profile. To avoid even that, the script can pre-seed a Remmina profile over SSH (see below). |
+| **Frame Control → Tools → Mac in the headset** | Nothing to install. Allow Screen Recording and Accessibility for Frame Control, then press **Show** next to any window or screen | **Verified 2026-09-28** on the Frame (panel in 1.5 s, measured with `scripts/macview-bench.py`: about 10–20 ms from the Mac drawing a frame to the viewer drawing it); laser input not yet tried while wearing it | **Recommended.** Hardware H.264 over an SSH tunnel, adapting to the link. Each window becomes its own panel you can place anywhere. Laser clicks and scrolls, and the Mac's keyboard types. See [mac-in-headset.md](mac-in-headset.md) |
+| **macOS Screen Sharing (VNC) → Remmina on the Frame** | **Mac:** System Settings → General → Sharing → Screen Sharing on → (i) → enable "VNC viewers may control screen with password". **Frame:** `./scripts/install-apps.sh remmina` from the Mac, then open Remmina in the headset and connect to `vnc://<mac>.local` | **Verified 2026-09-27** (Frame BUILD_ID 20260925.6191901, macOS 27.0), in its own panel via `panel-on-frame.sh mac-screen`. Remmina is on Flathub for **aarch64** with VNC and RDP ([Flathub](https://flathub.org/apps/org.remmina.Remmina)). The Frame desktop runs Flatpaks ([UploadVR](https://www.uploadvr.com/flatpaks-open-source-steam-frame/)). macOS VNC is built in. | **Fallback** (whole screens only). Nothing to install on the Mac, and it's easy to set up. Noticeable lag, even at lower Remmina quality settings on a good 5 GHz link, where neither Wi-Fi nor the Frame's CPU was the bottleneck. Usable for reading and coding, but not for games. You'll type the Mac's hostname once in Remmina on the headset, then save the profile. To avoid even that, the script can pre-seed a Remmina profile over SSH (see below). |
 | Sunshine (Mac) → Moonlight (Frame Flatpak) | `brew install` Sunshine on the Mac, then `./scripts/install-apps.sh moonlight` | Moonlight Flatpak supports **aarch64** ([Flathub](https://flathub.org/apps/com.moonlight_stream.Moonlight)). **Sunshine on macOS is poorly supported**: install problems on Apple Silicon/Sequoia, and no virtual gamepads ([LizardByte discussion #777](https://github.com/orgs/LizardByte/discussions/777)). | Try it if VNC is too laggy. Expect some friction. |
 | Steam Remote Play with the Mac as host | Steam on the Mac, Steam Link/Remote Play on the Frame | macOS-hosted Remote Play is reported broken or flaky in 2024–2026 ([Steam discussion](https://steamcommunity.com/groups/homestream/discussions/1/574921459914429988/)) | Not recommended. It's only for games, if it works at all. |
 | Immersed / Virtual Desktop | Vendor apps | Immersed has a Mac agent but no known Frame client. Virtual Desktop's developer said he'd "try" to port it ([NewsBreak](https://www.newsbreak.com/news/4892834783961-virtual-desktop-dev-says-he-ll-try-to-bring-the-app-to-steam-frame)). | Not available as of 2026-09-25. Check again later. |
@@ -86,6 +87,21 @@ its header. (Verified 2026-09-27.)
 Going the other way, pointing a controller at the panel moves the Mac's mouse,
 because Remmina forwards input (`viewonly=0`).
 
+## First-party options, and why they do or don't fit
+
+Checked 2026-09-28. The first-party way is usually the best one, so these are
+listed first; the sections above and below explain the alternatives.
+
+| Goal | First-party option | Fits? | Why, and what would make it easier |
+|---|---|---|---|
+| Type and point from the **iPhone** | **KDE Connect** (KDE; official [iOS app](https://apps.apple.com/app/kde-connect/id1580245991)) remote touchpad and keyboard, plus clipboard and files | **Best candidate, untested on the Frame** | The Frame doesn't have it (verified: no `kdeconnectd`), it isn't on Flathub, and the root is read-only, so it would have to run from `~` or a container. On Wayland it types through KWin, so it can only reach the desktop panel, not SteamVR or games (**inferred**). Steam Deck users report its remote input breaking after SteamOS updates ([SteamOS #1939](https://github.com/ValveSoftware/SteamOS/issues/1939)). If it works, Frame Control could install it and pair it for you. |
+| Type and point from the **Mac** | KDE Connect for macOS (KDE builds) | Same as above | Its Mac app sends clipboard and files but has no keyboard/mouse sharing (**inferred**). |
+| Either | **Bluetooth keyboard and mouse** paired in SteamOS (Valve) | Yes, with real hardware | Neither device can pretend to be one: iOS refuses the HID service ([Apple forums](https://developer.apple.com/forums/thread/733916)), and macOS has no built-in way. |
+| Either | **xrdp** in Developer Mode (Valve) | No | Input goes into a *separate* desktop shown on the Mac, not into what you see in the headset. |
+| **Mac screen** in the Frame | **Screen Sharing** (Apple's VNC server) + Remmina (already installed on this Frame, profile pre-seeded by `install-apps.sh`) | **Yes, closest to first-party** | Only the Mac side is first-party; Remmina is the client. Turn on System Settings → General → Sharing → Screen Sharing → (i) → "VNC viewers may control screen with password". Still to test in the headset (open question 11). |
+| Mac screen | **Steam Remote Play** with the Mac as host (Valve) | Probably not | macOS isn't a SteamVR host, and Mac-hosted Remote Play is reported broken ([Steam forum](https://steamcommunity.com/groups/homestream/discussions/1/574921459914429988/)). One quick test is worth doing: Steam on the Mac, then Remote Play from the Frame's Steam. |
+| Mac or **iPhone screen** | **AirPlay** (Apple) | Not officially | It's Apple's own mirroring for both, but Apple only licenses receivers to TV and speaker makers; nothing official runs on Linux. UxPlay (below) is the unofficial receiver. |
+
 ## C. Show the iPhone's screen inside the Frame
 
 iOS only shares its screen two ways: **AirPlay** (Screen Mirroring in Control

docs/vr-apks.md

@@ -0,0 +1,148 @@
+# VR APKs and Quest games in Lepton
+
+What it takes to run an immersive (OpenXR) Android app, including Meta Quest
+builds, on the Frame. Checked on SteamOS BUILD_ID 20260925.6191901, Lepton
+v2.8.14 (rootfs v2.8.11), SteamVR 2.18.1, on 2026-09-28, unless marked
+**inferred**.
+
+## How a VR APK reaches SteamVR (verified)
+
+- Lepton ships a standard Khronos system runtime manifest,
+  `/vendor/etc/openxr/1/active_runtime.json`, pointing at SteamVR's Android
+  client, `/data/steamvr/runtime/bin/androidarm64/vrclient.so`. That is the
+  host's `/opt/steamvr/bin/androidarm64/`, bind-mounted in.
+- An APK's own Khronos-style `libopenxr_loader.so` tries the runtime brokers
+  (`org.khronos.openxr.runtime_broker`, `…system_runtime_broker`), finds
+  neither, then falls back to that manifest. Nothing in the APK has to change
+  for discovery.
+- Install the APK without the flatscreen marker
+  (`python3 ui/frame_android.py install app.apk --vr`). The marker only
+  controls Lepton's 2D Android surface; the app itself has to start an
+  OpenXR session.
+- Lepton also loads Valve's `XR_APILAYER_VALVE_fdm_injection` layer from
+  `/vendor/etc/openxr/1/api_layers/implicit.d/`. Only layers in Valve's own
+  directories are picked up (`liblepton/vulkan_layers.sh`), so a third-party
+  layer has to ship inside the APK.
+
+**Open Brush 2.32.29, the Quest APK from its GitHub release (Unity OpenXR,
+Vulkan), works unmodified.** Its manifest already has `LAUNCHER` next to
+`com.oculus.intent.category.VR`. Unity asked for OpenXR 1.1, got
+`XR_ERROR_API_VERSION_UNSUPPORTED`, retried with 1.0 and succeeded. SteamVR
+took it as the scene app, created Touch, simple-controller and Frame-controller
+bindings, and the session reached `XR_SESSION_STATE_FOCUSED`. A headset capture
+(`ui/frame_vrshot.py`, after waking the compositor and closing the dashboard)
+showed a dark sky over a mountain horizon; nobody wore the headset to confirm
+it was Open Brush's scene or to try drawing.
+
+**Khronos `hello_xr` (Vulkan, 1.1.63 release APK) works unmodified:**
+`Instance RuntimeName=SteamVR/OpenXR RuntimeVersion=2.18.1`, 1728×1728
+swapchains per eye, session `IDLE → READY → SYNCHRONIZED` (the headset was
+not being worn, so it did not reach `FOCUSED`).
+
+## What SteamVR's Android runtime supports (verified, from `vrclient.so`)
+
+- **OpenXR 1.0 only.** An app requesting `XR_API_VERSION_1_0` works; one
+  requesting 1.1 (`XR_CURRENT_API_VERSION` in a 1.1 SDK) gets
+  `XR_ERROR_API_VERSION_UNSUPPORTED` from the runtime.
+- Extensions include `XR_KHR_opengl_es_enable`, `XR_KHR_vulkan_enable{,2}`,
+  `XR_KHR_composition_layer_depth`, `XR_KHR_locate_spaces`,
+  `XR_EXT_local_floor`, `XR_EXT_uuid`, `XR_EXT_palm_pose`,
+  `XR_EXT_hand_tracking`, `XR_EXT_eye_gaze_interaction`, and these Meta ones:
+  `XR_FB_display_refresh_rate`, `XR_FB_foveation{,_configuration,_vulkan}`,
+  `XR_FB_space_warp`, `XR_FB_swapchain_update_state`,
+  `XR_META_foveation_eye_tracked`, `XR_META_recommended_layer_resolution`,
+  `XR_META_vulkan_swapchain_create_info`, `XR_META_performance_metrics`.
+- Not present: `XR_FB_passthrough`, `XR_FB_hand_tracking_*`,
+  `XR_FB_spatial_entity*`, `XR_FB_color_space`,
+  `XR_KHR_android_thread_settings`, `XR_OCULUS_*`.
+- Interaction profiles include `oculus/touch_controller`, `khr/simple_controller`,
+  `valve/frame_controller` and the usual PC controllers. Valve documents Touch
+  bindings as a working fallback on the Frame controllers.
+
+## What stops a Quest APK (verified with Wolvic 1.9, `oculusvr` build)
+
+1. **Lepton won't start it.** Lepton's `apk-info-extractor` only accepts an
+   activity whose intent filter has `android.intent.action.MAIN` and
+   `android.intent.category.LAUNCHER`. Quest apps use
+   `com.oculus.intent.category.VR` instead, so Lepton logs `APP_ACTIVITY is
+   empty` and exits. There is no override. **Fix:** add the `LAUNCHER`
+   category to that intent filter and re-sign. After that, Wolvic started.
+2. **OpenXR 1.1.** Wolvic's Quest build then requested OpenXR 1.1 and aborted
+   on `XR_ERROR_API_VERSION_UNSUPPORTED`. Unity's OpenXR plugin retries with
+   1.0 (Open Brush, above), so this mostly bites native and non-Unity apps. **Fix (inferred):** an API layer
+   inside the APK that asks the runtime for 1.0 and maps the 1.1 core
+   functions to the extensions the runtime does have (`XR_KHR_locate_spaces`,
+   `XR_EXT_local_floor`, `XR_EXT_uuid`, `XR_EXT_palm_pose`).
+3. **Lepton's missing clipboard service** still applies to VR apps. The Godot
+   XR Tools demo's Quest build (itch.io) dies in `Godot.<init>` casting the
+   null clipboard service to `ClipboardManager`, before any OpenXR call. See
+   the clipboard table in [apks.md](apks.md).
+4. **Not yet reached:** required Meta-only extensions (each app differs),
+   swapchain formats (the Lynx Wolvic build needed `GL_SRGB8_ALPHA8`), and
+   Meta platform services.
+
+The loader was never the problem: Wolvic's Quest `libopenxr_loader.so` is a
+Khronos-style loader and found SteamVR through `/vendor`.
+
+## Out of scope
+
+- **Meta entitlement.** Apps that call the Oculus Platform SDK
+  (`libovrplatformloader.so`) to check the Quest store licence need Meta's
+  services. Frame Control won't work around that.
+- **VrApi-era apps** (`libvrapi.so`, before OpenXR) need an API translator,
+  not a patch.
+
+## Frame Control does this for you
+
+APK uploads and `python3 ui/frame_android.py install app.apk` detect VR
+manifest categories, Samsung's `vr_only` flag and the arm64 OpenXR loader.
+VR apps default to immersive mode without the flatscreen marker. The upload
+selector or CLI `--flat` / `--vr` overrides that choice. Compatibility notes
+identify legacy VrApi, Meta platform SDK and OpenXR libraries.
+
+Lepton only starts an `<activity>` whose MAIN intent filter has LAUNCHER; it
+ignores `<activity-alias>`, which is where Godot 4 exports put LAUNCHER. When no
+real activity qualifies, Frame Control adds LAUNCHER to the VR activity's MAIN
+filter, or to the activity the launcher alias targets, then repacks and v2-signs
+the APK locally before copying it; `meta.json` records
+`"patched": ["launcher"]`. Unchanged ZIP members retain their compressed
+bytes; stored libraries are aligned to 16 KiB. The RSA signing identity lives
+in Frame Control's per-user app-data directory as `apk-signing-key.json`
+(mode 0600). Keep this key to preserve the signer on subsequent patched
+updates. A re-signed APK cannot update an installation signed by its original
+publisher; Android also treats it as a different signer for signature checks.
+
+VR apps with an arm64 OpenXR loader also get the OpenXR compatibility layer
+([frame/openxr-compat](../frame/openxr-compat/README.md)): an implicit API
+layer in the APK's `assets/openxr/1/api_layers/implicit.d/`, which the app's
+own loader picks up next to Valve's layer. It asks SteamVR for OpenXR 1.0 when
+the app wants 1.1 and enables the extensions that became 1.1 core; maps
+`xrLocateSpaces` to `xrLocateSpacesKHR` and `grip_surface` to `palm_ext`; drops
+1.1 controller profiles SteamVR doesn't know; stubs
+`XR_KHR_android_thread_settings` and `XR_OCULUS_android_session_state_enable`;
+and keeps the current refresh rate when SteamVR refuses a requested one.
+`meta.json` records `"patched": ["openxr-compat"]`. Skip it with
+`install … --no-xr-compat`. Its decisions go to logcat under `FrameXrCompat`.
+
+Verified on the headset (2026-09-28):
+
+- **Wolvic 1.9, Quest build**, installed as downloaded: Frame Control added
+  `LAUNCHER` and the layer. The layer turned OpenXR 1.1.48 into 1.0.63, the
+  instance and session were created, and a 144 Hz refresh request that SteamVR
+  refused was kept at the current rate. The session reached `SYNCHRONIZED`;
+  then Wolvic's Gecko engine crashed (null SIGSEGV on its Gecko thread, the
+  same crash its Lynx build has), which is Wolvic's, not OpenXR's.
+- **Open Brush, Quest build**, with the layer: 1.1.54 → 1.0.63, the thread
+  settings stub in use, `bytedance/pico4_controller` bindings dropped, and the
+  session reached `FOCUSED`, the same as without the layer.
+
+Inspect or prepare an APK without contacting the headset:
+
+```sh
+python3 ui/frame_android.py info app.apk
+python3 ui/frame_android.py patch app.apk patched.apk
+python3 ui/frame_android.py patch app.apk patched.apk --add assets/openxr/1/api_layers/implicit.d/X.json=X.json --add lib/arm64-v8a/libX.so=libX.so
+```
+
+The patch fixes Lepton's launch-category requirement. It does not supply an
+OpenXR 1.1 translation layer, Meta services or a VrApi implementation.