Settings: HANDS_SWAP_SIDES defaults to auto, and old untouched configs move to it

frametop.conf.example said HANDS_SWAP_SIDES=0, and desktops.sh copies it on a
fresh install, so every new install forced ft-camd's side camera names and
turned the hand tracker's own side check off. Some SteamVR restarts swap those
names, and then hands land beside their cutouts and recordings are mislabelled.

The example now says auto. scripts/conf-migrate.sh, run by install.sh and
hands/rec/install.sh, replaces the old line only where it's still exactly as
the example wrote it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

AGENTS.md

@@ -25,10 +25,14 @@ scripts/frame.sh --host '<cmd>'          # runs on the SteamOS host
 A Steam Frame is someone's personal headset, and they may be wearing it while you work.
 
 - Don't kill or restart `gamescope`, `steam`, `vrserver`, `vrcompositor`, the gamescope session, or the Frametop desktop without asking. Each one ends or disrupts whatever is happening in VR.
-- Don't run host `sudo`, `steamos-readonly disable`, `steamos-devmode` changes, pacman installs, or reboots without explicit approval. Only the Bluetooth fixes need host `sudo`, and they ask.
-- Write only inside the repo, `/tmp`, and the container unless told otherwise. The installers are the exception: they write the user services, launchers, and the SteamVR driver into the home folder.
+- Don't run host `sudo`, `steamos-readonly disable`, `steamos-devmode` changes, pacman installs, or reboots without explicit approval. Three installers need host `sudo`, and they ask for it: the Bluetooth fixes (`setup/bluetooth/install.sh`), hand tracking (`hands/run.sh install` and `caps`, which set ft-camd's file capabilities with `setcap`), and our own eye tracker's frame grabber (`gaze/tracker/install.sh`).
+- Write only inside the repo, `/tmp`, and the container unless told otherwise. The installers are the exception: they write the user services, launchers, and the SteamVR driver into the home folder. The Bluetooth fixes and the eye tracker's frame grabber also install root-owned files and system services under `/etc` (`/etc/steamframe`, `/etc/frametop`, `/etc/systemd/system`). When an installer starts writing something new outside the repo, add it to `uninstall.sh` too: users uninstall with that script, not with each installer's `uninstall`.
 - Never copy `.netrc`, SSH keys, or Steam config off the Frame or into this repo.
 
+## SteamOS updates
+
+A SteamOS update replaces SteamVR, KWin, and gamescope with the rest of the OS image. When a change starts depending on something from the image (a host file, an OpenVR interface outside the bundled header, an undocumented layout or output format, a SteamVR or KWin quirk), add a check for it to `scripts/update-check.py`, or a retest hint for its package there. [docs/design.md](docs/design.md) has the background.
+
 ## Names
 
 User-facing names are "Frametop", "Frametop Display Settings", and "Frametop Input Settings". Programs and files use the `ft-` / `ft_` prefix (`ft-screens`, `ft-pointer`, `ft-layout`, the `ft_pointer` driver); config, units, and overlay keys use `frametop`. Program names must stay within 15 characters: Linux truncates process names there, and the scripts find programs with `pgrep -x` / `pkill -x`.

README.md

@@ -1,32 +1,37 @@
 # Frametop
 
-Frametop puts a multi-monitor KDE Plasma desktop into SteamVR on the Valve Steam Frame, and lets a Bluetooth mouse drive all of SteamVR. It installs and runs on the headset itself.
+Frametop is a desktop for the Steam Frame that runs on the headset itself, with no PC. Put several monitors around you, pull app windows out to float on their own, and point with a mouse, or with your eyes.
 
-Each screen is its own monitor with its own resolution, so you can have an ultrawide in the middle and two portrait screens beside it, at whatever size and distance you like. The screens come back to your saved layout when the desktop starts. You can move, resize, curve, and roll them, pin one to your wrist, and put them all back with a shortcut.
+- **Screens that are real monitors.** Each KDE Plasma screen has its own resolution and shape: an ultrawide in front, portrait screens beside it. Move, resize, curve, and roll them, or pin one to your wrist or your head. They come back to your layout when the desktop starts.
+- **Windows that float on their own.** Take any app window off the screens into a panel of its own with Meta+Shift+F, its title bar button, or Launch as Standalone. It stays part of the desktop, so drag and drop and the clipboard still work between floating windows and the screens.
+- **Profiles.** Save where your screens are, which ones show, and which apps are open where. Switch to a profile from Display Settings, a key, or its launcher entry, or start the desktop in one.
+- **One mouse for all of SteamVR.** A Bluetooth mouse drives a small dot anchored in the room. It works the screens, the dashboard, Steam, and overlays, and hands the laser back when you pick up a controller.
+- **Look and click (experimental).** In gaze mode the pointer goes where you look. Meta+J and Meta+K, or the mouse buttons, click and fine-tune, and each correction teaches the tracker. Calibration runs in the headset.
+- **Made for long sessions.** The displays turn off when the headset isn't used, even on a stand that makes it seem worn. It stays awake on the charger, and you can reach the desktop remotely over VNC.
 
-The mouse shows up as a small dot anchored in the room. It works on the SteamVR dashboard, Steam, overlays, and the desktop, and it hands the laser back to your controllers when you pick one up.
-
-It comes with two settings apps, Frametop Display Settings for the screens and Frametop Input Settings for mice, keyboards, and button mappings, plus fixes that let Bluetooth LE mice and keyboards like the Swiftpoint Z3 reconnect after they sleep.
-
-When you're not wearing the headset, Frametop can turn its displays off and keep it awake on the charger, so you can still reach it remotely. This works even on a stand or mount that makes the headset seem worn.
+Two settings apps come with it: Frametop Display Settings for the screens, profiles, and power, and Frametop Input Settings for mice, keyboards, gaze, and button mappings. Optional fixes let Bluetooth LE mice and keyboards like the Swiftpoint Z3 reconnect after they sleep. Install it with one command: see [Install on the headset](#install-on-the-headset).
 
 Frametop is an independent project, not made by or affiliated with Valve.
 
+Join the [Frametop Discord](https://discord.gg/W3X9f7z3Bc) for questions, ideas, and help with your setup.
+
 ## Install on the headset
 
+> **Frametop doesn't work on the SteamOS beta right now.** On the beta (SteamOS 0.4.3), gaze mode can't read the eye tracker, and the desktop has started without its taskbar ([#15](https://github.com/DeeJanuz/frametop/issues/15)). Use the stable SteamOS release until this note is gone.
+
 You need a Steam Frame with an internet connection, a keyboard (Bluetooth, or the on-screen one), and about 3 GB of free space.
 
 1. In the launcher, choose Launch a program → Desktop.
 2. In the application menu, open System → Konsole.
-3. Clone the repo and run the installer:
+3. Run:
 
    ```
-   git clone https://github.com/DeeJanuz/frametop.git ~/frametop
-   cd ~/frametop
-   ./install.sh
+   curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash
    ```
 
-   The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks you two things along the way. The Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
+   It asks which version you want: stable (the `main` branch, tested releases) or experimental (the `experimental` branch, the newest features, less tested). Then it clones the repo into `~/frametop` and runs `install.sh`. To choose without the question, add `-s -- --stable` or `-s -- --experimental` after `bash`. By hand, the same is `git clone https://github.com/DeeJanuz/frametop.git ~/frametop`, then `cd ~/frametop` and `./install.sh` (add `--branch experimental` to the clone for experimental).
+
+   The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks you four things along the way: whether to install gaze mode (experimental, yes by default), our own eye tracker for it (yes by default), and the Bluetooth fixes, then whether to restart SteamVR. The eye tracker and the Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
 
 After the restart, Launch a program → Desktop opens the multi-screen desktop, with its screens arranged around where you're facing. Frametop Display Settings and Frametop Input Settings are in the desktop's application menu, under Settings.
 
@@ -40,11 +45,10 @@ If you work in the desktop for long stretches, or leave the headset on a stand,
 
 ## Use
 
+### Screens
+
 | Do this | To get this |
 | --- | --- |
-| Move the mouse | The dot moves around you and snaps onto whatever panel it's over |
-| Click, right-click, scroll | Acts on the panel under the dot |
-| Pick up a controller | The controller gets its laser back; move the mouse to take over again |
 | Point near the bottom of a screen | Its controls fade in: the bar, the curve and roll buttons, and the resize tab on the corner |
 | Drag the bar under a screen | Moves the screen; scroll while dragging to push it away or pull it closer. With the mouse, hold right while dragging to tilt it |
 | Drag the tab on a screen's bottom right corner | Resizes the screen |
@@ -52,15 +56,71 @@ If you work in the desktop for long stretches, or leave the headset on a stand,
 | Drag the roll button sideways, or scroll on it | Rolls the screen; it snaps level near straight |
 | While carrying a screen, sweep its laser across your other controller's ring, then let go | Pins it to that wrist, at its size and distance, as you hold it when you let go; it shows while you see its front. Grab its bar to adjust it (it stays pinned); sweep across the ring again to take it off |
 | Set a screen to On your head (Frametop Display Settings, Visibility & pins) | Pins it to your head where it is, like a HUD. Grab its bar to move it; it stays on your head |
-| Save current arrangement… (Frametop Display Settings, Layout) | Saves where the screens are, with their sizes and pins, under a name. Pick a saved layout under Arrangement and press Arrange now to switch to it |
 | Meta+Shift+R in the desktop | Puts the screens back in their layout (also in the menu as Reset Screen Layout, and mappable to a mouse button) |
+| Meta+Alt+Tab, or Meta+Alt+Shift+Tab | Spins every screen and floating window around you together, like a lazy susan, so the next one on your right (or left) glides to straight ahead, with the pointer and typing going to it. Their arrangement stays the same: the room turns instead of you. Tap again to keep going; Meta+Shift+R puts the screens back. Pinned screens stay where they are |
 | Meta+Shift+H in the desktop | Hides or shows all screens (also in the menu as Hide/Show Screens, and mappable). The Visibility & pins tab of Frametop Display Settings can instead show them only with the dashboard open, or while you look at your wrist |
-| Leave the headset on a stand | Its displays turn off once it has gone unused for the time set in Frametop Display Settings → Power, even if the stand covers its proximity sensor. Pick it up, or use any mouse, keyboard, or button, and they come back on |
-| Play a VR game | The screens hide and your controllers stay in the game. Open the SteamVR dashboard, or press Meta+Shift+H, to see and use them. To keep them visible over games, change During VR games on the Visibility & pins tab; the controllers still stay in the game, and you use the screens with the mouse or the dashboard |
+| Tap Meta, on any keyboard | Opens the Steam menu in the SteamVR dashboard, or closes the dashboard, wherever you are. The desktop's launcher is still on the taskbar and Alt+F1. Change it in Frametop Input Settings (Keyboard page), where any key combination or modifier tap can do a Frametop or Steam action, open a profile, or run a command of your own |
+| Switch a screen to Hidden (Frametop Display Settings, Visibility & pins → Screens shown) | Hides just that screen until you switch it back, whatever the other visibility settings say; new windows that would open on it float instead |
+| Play a VR game | Frametop pauses so the game gets the headset to itself (see [Pause for VR games](#pause-for-vr-games)): the screens hide and your controllers stay in the game. Click both thumbsticks together twice to bring Frametop back. With the automatic pause off, the screens still hide, and the SteamVR dashboard or Meta+Shift+H shows them; to keep them visible over games, change During VR games on the Visibility & pins tab |
+
+Restarting the desktop (Restart desktop in Frametop Display Settings) closes its windows, but background work you started in it, such as servers, tmux sessions, or builds, keeps running.
+
+### Mouse and controllers
+
+| Do this | To get this |
+| --- | --- |
+| Move the mouse | The dot moves around you and snaps onto whatever panel it's over |
+| Click, right-click, scroll | Acts on the panel under the dot |
+| Pick up a controller | The controller gets its laser back; move the mouse to take over again |
 
 You can map the mouse's extra buttons to actions such as Toggle SteamVR dashboard, Recenter pointer, or Head follow on/off on the Buttons page of Frametop Input Settings, and the Frame controllers' buttons on its Controllers page. Pointer speed, dot size, and the rest are on its Pointer page and take effect immediately. If a panel you only look at, such as a performance overlay that follows your view, keeps catching the dot, tick it (or its whole app) on the Ignored panels page, and the pointer passes through it. Head follow, which is experimental and off by default, makes the pointer come along when you turn your head: it stays put until your head turns past the leash angle, then glides back to its place in your view, and a leash of 0 keeps it fixed in your view. It's only lightly tested and not polished; tuning its settings, or improving how it feels, is open to anyone who wants to take it further.
 
-Restarting the desktop (Restart desktop in Frametop Display Settings) closes its windows, but background work you started in it, such as servers, tmux sessions, or builds, keeps running.
+### Floating windows
+
+Any desktop window can float in VR as a panel of its own. It stays a window of the same desktop, so drag and drop and the clipboard work between floating windows and the screens, and it's still in the taskbar and Alt+Tab.
+
+| Do this | To get this |
+| --- | --- |
+| Meta+Shift+F over a desktop window | Floats that window, or puts it back on its screen if it floats. It acts on the window under the pointer, or the active one if the pointer is over the wallpaper. Rebind it, or map it to a mouse or controller button, in Frametop Input Settings (Keyboard page, or Buttons and Controllers as Float window in VR) |
+| Click the float button, left of Close in a window's title bar | The same. Float in VR is also in every window's menu (Alt+F3). Apps that draw their own title bar, like Chromium and Electron apps, don't have the button: use Meta+Shift+F |
+| Right-click an app in the Application Launcher (or the taskbar) and pick Launch as Standalone | Starts the app with its window floating, where that app last floated, or in front of you the first time. From a terminal: `float/ft-float launch org.kde.dolphin`, or `float/ft-float run <command>` |
+| Meta+scroll over a floating window | Scales it up or down |
+
+### Profiles
+
+A profile is a named setup: where the screens are, with their sizes and pins, which ones are hidden, and which apps are open and where their windows are, on a screen or floating.
+
+| Do this | To get this |
+| --- | --- |
+| Save as profile… (Frametop Display Settings, Layout & profiles) | Saves the current setup under a name, or updates the profile you're in |
+| Pick a profile under Arrangement and press Open profile | Switches to it: the screens move, open windows of its apps go to their places, and the apps that aren't open start. Nothing closes |
+| Pick a profile under Start in profile, or run its entry (Frametop: NAME) from SteamVR's Launch a program list | The desktop starts in that profile, or switches to it if it's running. A profile can also go on a key combination, mouse button, or controller button in Frametop Input Settings |
+
+### Pause for VR games
+
+Frametop pauses while a VR game runs, so the game gets the headset's CPU and GPU, and comes back a few seconds after the game ends. Paused, the screens hide and the desktop nearly stops drawing, but its windows stay open. Gaze mode's eye tracking stops, and so do remote desktop and hand tracking if they run. The mouse works as a plain mouse in SteamVR.
+
+| Do this | To get this |
+| --- | --- |
+| Click both thumbsticks together, twice | Pauses Frametop, or brings it back, in a game or not. You hear a short sound. The game sees the clicks too |
+| Start a VR game | Frametop pauses, and comes back 5 seconds after the game ends. Bring it back during the game, and it stays on until that game ends |
+| Map Pause/resume Frametop to a mouse button, key combination, or controller button | The same, from that button (Frametop Input Settings) |
+
+The Game optimization page of Frametop Input Settings turns the automatic pause off, changes the gesture, closes the desktop instead of hiding it (more for the game, but its windows close), and turns the sound off. From a terminal: `input/ft-pause on`, `off`, or `status`.
+
+### Gaze mode (experimental)
+
+In gaze mode the pointer goes where you look, and the mouse or the keyboard does the last bit. The installer offers it (or run `gaze/run.sh install` later), and then our own eye tracker for it, which is more accurate than SteamVR's (or run `gaze/tracker/install.sh` later; it needs `sudo`). Gaze mode uses ours once it's installed, and SteamVR's until then. Turn it on and calibrate it on the Gaze page of Frametop Input Settings.
+
+| Do this | To get this |
+| --- | --- |
+| Tap Meta+J, or Meta+K | A left or right click where you look |
+| Hold Meta+J, turn your head onto what you meant, and let go | A click there. Held still for half a second, it becomes a press, and turning your head drags |
+| Hold the left mouse button, move the mouse onto what you meant, and let go | A click there. The right button does the same for a right click. Held still, the left button drags |
+| Double right click (or double Meta+K) while dragging a screen's bar | Pans and tilts the screen |
+| Put the headset on | A quick check: look at the dot for a moment, and the pointer lines up again |
+
+By default the mouse only corrects: while the gaze has the pointer, moving the mouse does nothing until you hold a button. Each correction before a click teaches the tracker where it was off. A correction past the learning limit (55 degrees by default, about half of what you can see) starts a quick check instead. Calibrate, on the Gaze page, runs a full calibration in a panel in front of you: look at each dot and click. Check headset fit shows how well the eye tracker sees your eyes. [gaze/README.md](gaze/README.md) has the details.
 
 ### Leave the headset on a stand and reach it remotely
 
@@ -75,14 +135,18 @@ With the displays off, the headset keeps tracking and rendering, so it uses abou
 
 This is an early release, tested on one Steam Frame (SteamOS 0.3.0 build 20260922, SteamVR 2.17.10).
 
-- A SteamOS or SteamVR update can break parts of it until Frametop catches up. If something stops working after an update, please report it.
+- A SteamOS or SteamVR update can break parts of it until Frametop catches up. After an update, run `cd ~/frametop && scripts/doctor.sh` in a terminal. It checks what Frametop needs from SteamOS, and says what changed since the versions you last marked as working and what to try. Once everything works, `scripts/doctor.sh --mark-good` records the versions. If something stops working, please report it.
 - The first install downloads 1–2 GB for the build container and compiles everything on the headset, which takes several minutes.
-- During a VR game you can't show the screens with a controller button, because the game owns the buttons. Open the SteamVR dashboard, press Meta+Shift+H, or use a mapped mouse button instead.
-- Flatscreen games aren't detected as games. If your controllers end up working the screens instead of the game, set Controllers on the screens to "Only with the SteamVR dashboard open" (Frametop Display Settings, Visibility & pins tab).
+- During a VR game, mapped controller buttons belong to the game, so they can't bring the screens up. The pause gesture still works: Frametop reads it without taking the thumbsticks from the game. With the automatic pause off, open the SteamVR dashboard, press Meta+Shift+H, or use a mapped mouse button instead.
+- Flatscreen games aren't detected as games, so they don't pause Frametop by themselves: click both thumbsticks twice to pause it. If your controllers end up working the screens instead of the game, set Controllers on the screens to "Only with the SteamVR dashboard open" (Frametop Display Settings, Visibility & pins tab).
 - Typing follows your last click. A controller click on a panel other than the screens (the dashboard, a Steam app) doesn't move typing there; click it with the mouse, or click a screen to bring typing back.
 - The screens don't draw a mouse cursor of their own. The 3D mouse's dot or SteamVR's laser shows where you're pointing.
+- Profiles reopen apps, not what the apps had open. Tabs, files, and folders are left to each app's own restore.
+- Dragging something from one panel to another (a screen and a floating window) works, but the dragged item's icon doesn't show while the pointer is between panels.
+- Gaze mode is only as good as its calibration, and that depends on how the headset sits on your face. If the pointer lands off after you adjust the headset, run Quick check or Calibrate on the Gaze page of Frametop Input Settings.
 - On SteamVR's Settings page, the 3D mouse shows a laser beam and a larger hit dot, like a controller. SteamVR doesn't tell other programs where that page is (unlike Steam's pages, such as Library), so the mouse used to miss most of it: clicks went through to a desktop screen behind, and the dot disappeared. As a workaround, on that page only, the laser starts near your eye and SteamVR finds the page itself. See docs/design.md.
-- Remote desktop over VNC (Frametop Remote Access in the app menu, or `./desktops.sh remote on`) needs Tailscale on the Frame. It shows the primary screen only. The app turns it on and off, shows the address, and shows, copies, or changes the VNC password. The password is made at random on the Frame and kept in `~/.config/frametop-remote` (only you can read it); VNC limits it to 8 characters, and the tailnet encrypts the connection. Turning it on in a desktop that started with it off takes a desktop restart.
+- Remote desktop over VNC (Frametop Remote Access in the app menu, or `./desktops.sh remote on`) needs Tailscale on the Frame. It shows the primary screen only. The app turns it on and off, shows the address, and shows, copies, or changes the VNC password. The password is made at random on the Frame and kept in `~/.config/frametop-remote` (only you can read it); VNC limits it to 8 characters, and the tailnet encrypts the connection. Turning it on in a desktop that started with it off takes a desktop restart. It costs almost nothing until a viewer connects; the picture then takes a few seconds to appear.
+- The desktop has no blur behind panels and menus, and no window animations, so it leaves the headset's GPU to SteamVR. Turn them back on in the Frametop desktop's System Settings (Desktop Effects, and Animation speed under General Behavior); Frametop won't turn them off again.
 - Turning the displays off on a stand only turns their backlight off. SteamVR has no way for other programs to put the headset in standby, so tracking and rendering keep running, and the headset draws nearly its full power.
 
 ## Reporting problems
@@ -93,42 +157,59 @@ In a terminal on the headset, run:
 cd ~/frametop && scripts/report.sh
 ```
 
-This writes `frametop-report-<date>.txt` with version numbers, service states, settings, and recent logs. Bluetooth addresses and the headset's serial number are masked. Then [open an issue](https://github.com/DeeJanuz/frametop/issues), describe what you did, what you expected, and what happened, and attach the file.
+This writes `frametop-report-<date>.txt` with version numbers, service states, settings, and recent logs. Bluetooth addresses and the headset's serial number are masked. Then [open an issue](https://github.com/DeeJanuz/frametop/issues), describe what you did, what you expected, and what happened, and attach the file. Quick questions can go to [Discord](https://discord.gg/W3X9f7z3Bc) instead.
 
 ## Update
 
+Run the same command again. It updates `~/frametop` to the latest of the version you have (or switches, if you pick the other one) and installs it:
+
 ```
-cd ~/frametop && git pull && ./install.sh
+curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash
 ```
 
+Or by hand: `cd ~/frametop && git pull && ./install.sh`.
+
 ## Uninstall
 
+In a terminal on the headset, run:
+
 ```
-./desktops.sh uninstall                # the launcher's Desktop entry goes back to the stock desktop
-./desktops.sh relay uninstall
-pointer/helper/run.sh uninstall
-power/run.sh uninstall
-pointer/driver/install.sh uninstall    # then restart SteamVR
-input-settings/install.sh uninstall
-display-settings/install.sh uninstall
-setup/bluetooth/install.sh uninstall   # if you installed the Bluetooth fixes
+curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash
 ```
 
+It works in two steps, so it never takes away the keyboard, mouse, or desktop you're using while it runs:
+
+1. It stops Frametop from starting. Launch a program → Desktop opens the stock desktop again, and Frametop's services, its SteamVR driver, its menu entries, and the system files of our eye tracker and the Bluetooth fixes are removed (those need your `sudo` password). Everything running now keeps running until you restart the headset, and it offers to restart it for you.
+2. After the restart, run the same command again. It deletes the code in `~/frametop`, and asks whether to delete your settings, any eye or hand recordings, and the build container (1–2 GB) too.
+
+To see what it would do without changing anything, add `-s -- --dry-run` after `bash`. If the code isn't in `~/frametop`, add `-s -- --dir <folder>`. From the repo, the same script is `./uninstall.sh`.
+
+Don't delete `~/frametop` by hand before you uninstall and restart: the desktop and the input relay run from it, and without it Launch a program → Desktop no longer opens anything. If you've already deleted it, the command above still works, since it doesn't need the repo.
+
+Unless you ask for them to go, your settings stay: `~/.config/frametop.conf`, `frametop-input.json` (button maps and key combinations), `frametop-layout.json` (the layout and profiles), `frametop-float.json`, and `frametop-remote/` in `~/.config`, the gaze calibration in `~/.local/state/frametop`, and the desktop's own Plasma setup in `~/.config/frametop`. A later install picks them up again.
+
 ## How it works
 
 A Plasma session runs nested inside ft-screens (`screens/`), a small Wayland compositor. KWin opens one window per screen, ft-screens sets each window's size, and each frame goes to SteamVR as an overlay without being copied. An input relay (`input/`) keeps Bluetooth mice working in SteamVR and feeds the mouse to the 3D pointer, which drives a virtual SteamVR controller (`pointer/`). A power service (`power/`) turns the displays off while the headset isn't used. [docs/reference.md](docs/reference.md) covers each piece, and [docs/design.md](docs/design.md) explains the design and what we learned about SteamVR on the Frame. [docs/hazards.md](docs/hazards.md) lists known ways the input handling can go wrong.
 
 | Folder | What it is |
 | --- | --- |
+| `get.sh` | The one-line installer: picks stable or experimental, clones or updates the repo, and runs `install.sh`. |
 | `install.sh` | The one-step installer. Safe to re-run. |
+| `uninstall.sh` | The uninstaller: run it, restart the headset, and run it again. It doesn't need the rest of the repo. |
 | `desktops.sh` | Start, stop, and configure the desktop, and install the input relay. |
 | `screens/` | ft-screens, the compositor (wlroots and OpenVR). |
 | `session/` | The desktop session script and its config example. |
 | `layout/` | ft-layout: where the screens float, and their sizes. |
+| `float/` | Floating windows: ft-floatd and the KWin script that float a desktop window in VR. |
+| `decoration/` | The desktop's window decoration: Breeze's look plus the float button. |
 | `input/` | The input relay (Bluetooth mice and keyboards, button maps). |
 | `pointer/` | The 3D mouse: SteamVR driver, helper service, and a probe tool. |
 | `power/` | ft-powerd: turns the displays off while the headset isn't used. |
+| `gaze/` | Gaze mode (experimental): the gaze service, its calibration panel, our own eye tracker, and the gaze probe. See [gaze/README.md](gaze/README.md). |
+| `hands/` | Hand tracking (experimental, deferred: the installer doesn't offer it). See [hands/README.md](hands/README.md). |
 | `display-settings/`, `input-settings/` | The two settings apps (Kirigami, Python). |
+| `remote/` | Frametop Remote Access, the app that turns remote desktop over VNC on and off. |
 | `setup/` | The build container and the Bluetooth fixes. See [setup/README.md](setup/README.md). |
 | `scripts/` | Helpers the installers use. They run commands locally on the Frame, or over SSH from a PC. |
 
@@ -146,16 +227,17 @@ The scripts also work from a Linux or WSL PC over SSH, which is easier for editi
        IdentityFile ~/.ssh/<your-key>
    ```
 
-3. The Bluetooth fixes need `sudo`, and there's no terminal on the Frame to type the password into, so put it in `.env` at the repo root. It's gitignored and never synced:
+3. The Bluetooth fixes need `sudo` on the Frame. The installer asks for the password in your terminal (over `ssh -t`). To skip the question, or to install with no terminal, put it in `.env` at the repo root instead. It's gitignored and never synced:
 
    ```
    steamos_root_pwd="<password>"
    ```
 
-Then run `./install.sh` from the PC. Daily use:
+Then run `./install.sh` from the PC. If SteamVR isn't running on the Frame, the services that need it start with it later. Daily use:
 
 ```
 scripts/doctor.sh                  # is the Frame reachable and ready?
+scripts/doctor.sh --mark-good      # and record the versions Frametop works with
 scripts/sync.sh                    # copy the repo to ~/dev/frametop on the Frame
 scripts/frame.sh '<cmd>'           # run in the dev container, in the Frame's copy
 scripts/frame.sh -C <dir> '<cmd>'  # same, in a folder of the repo

decoration/apply.sh

@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+# Switch the running Frametop desktop's window decoration without restarting it: Frametop's
+# own (this folder, with the float button) or back to Breeze. The session script does the
+# same at every desktop start (session/frametop-session.sh), so this is for trying changes.
+#   decoration/apply.sh        install this folder's copy and use it
+#   decoration/apply.sh --off  back to Breeze (until the next desktop start)
+set -euo pipefail
+here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+deco=kwin4_decoration_qml_frametop
+cfg=$HOME/.config/frametop
+kwinrc=$cfg/kwinrc
+
+# The desktop's D-Bus: from its Plasma shell, which uses Frametop's config folder (KWin's own
+# environment isn't readable: it runs with extra capabilities).
+bus=
+for pid in $(pgrep -x plasmashell); do
+  env=$( (tr '\0' '\n' < "/proc/$pid/environ") 2>/dev/null) || continue
+  grep -qx "XDG_CONFIG_HOME=$cfg" <<<"$env" || continue
+  bus=$(sed -n 's/^DBUS_SESSION_BUS_ADDRESS=//p' <<<"$env")
+done
+[ -n "$bus" ] || { echo "the Frametop desktop isn't running" >&2; exit 1; }
+
+if [ "${1:-}" = --off ]; then
+  kwriteconfig6 --file "$kwinrc" --group org.kde.kdecoration2 --key library --delete
+  kwriteconfig6 --file "$kwinrc" --group org.kde.kdecoration2 --key theme --delete
+else
+  # Under a new name each time: KWin keeps a decoration's QML (even a broken one) by name
+  # until it restarts. The next desktop start goes back to the plain name.
+  decos=${XDG_DATA_HOME:-$HOME/.local/share}/kwin/decorations
+  rm -rf "$decos/${deco}"_try*
+  deco=${deco}_try$(date +%s)
+  mkdir -p "$decos"
+  cp -r "$here" "$decos/$deco"
+  rm -f "$decos/$deco/apply.sh"
+  sed -i "s/\"Id\": \"[^\"]*\"/\"Id\": \"$deco\"/" "$decos/$deco/metadata.json"
+  kwriteconfig6 --file "$kwinrc" --group org.kde.kdecoration2 --key library org.kde.kwin.aurorae
+  kwriteconfig6 --file "$kwinrc" --group org.kde.kdecoration2 --key theme "$deco"
+fi
+DBUS_SESSION_BUS_ADDRESS=$bus gdbus call --session -d org.kde.KWin -o /KWin -m org.kde.KWin.reconfigure >/dev/null
+echo "decoration: $(kreadconfig6 --file "$kwinrc" --group org.kde.kdecoration2 --key theme --default Breeze)"

decoration/contents/ui/FtButton.qml

@@ -0,0 +1,123 @@
+/*
+    A title bar button drawn the way Breeze draws its own: a glyph in the title's colour,
+    a soft circle behind it on hover, and a red circle for Close.
+
+    The float button is KWin's Keep Below button with a glyph of its own: Frametop's KWin
+    script (float/frametop-float.js) floats a window when keep-below is set on it and docks
+    it when it's cleared, and keeps the flag set on every floating window, so "toggled" here
+    means "floating" and the glyph turns into "back to the desktop".
+
+    SPDX-License-Identifier: GPL-2.0-or-later
+*/
+import QtQuick
+import QtQuick.Shapes
+import org.kde.kwin.decoration
+
+DecorationButton {
+    id: button
+    property real size: 20
+    property color fg: "white"
+    property color bg: "black"
+    readonly property bool isClose: buttonType === DecorationOptions.DecorationButtonClose
+    readonly property bool isFloat: buttonType === DecorationOptions.DecorationButtonKeepBelow
+    // The glyph, as polylines in an 18 x 18 box (Breeze's own sizes).
+    readonly property var glyph: {
+        switch (buttonType) {
+        case DecorationOptions.DecorationButtonClose:
+            return [[[5, 5], [13, 13]], [[13, 5], [5, 13]]];
+        case DecorationOptions.DecorationButtonMaximizeRestore:
+            return decoration.client.maximized ? [[[4.5, 9], [9, 4.5], [13.5, 9], [9, 13.5], [4.5, 9]]]
+                                               : [[[4, 11.5], [9, 6.5], [14, 11.5]]];
+        case DecorationOptions.DecorationButtonMinimize:
+            return [[[4, 7], [9, 12], [14, 7]]];
+        case DecorationOptions.DecorationButtonKeepAbove:
+            return [[[4, 9], [9, 4], [14, 9]], [[4, 14], [9, 9], [14, 14]]];
+        case DecorationOptions.DecorationButtonShade:
+            return [[[4, 5], [14, 5]], [[4, 13], [9, 8], [14, 13]]];
+        case DecorationOptions.DecorationButtonApplicationMenu:
+            return [[[3.5, 5], [14.5, 5]], [[3.5, 9], [14.5, 9]], [[3.5, 13], [14.5, 13]]];
+        case DecorationOptions.DecorationButtonKeepBelow:
+            // A box with an arrow leaving it (float in VR), or coming back into it (put it back).
+            return button.toggled
+                ? [[[8, 3.5], [3.5, 3.5], [3.5, 14.5], [14.5, 14.5], [14.5, 10]], [[14.5, 3.5], [8.5, 9.5]],
+                   [[8.5, 5], [8.5, 9.5], [13, 9.5]]]
+                : [[[8, 3.5], [3.5, 3.5], [3.5, 14.5], [14.5, 14.5], [14.5, 10]], [[8.5, 9.5], [14.5, 3.5]],
+                   [[10, 3.5], [14.5, 3.5], [14.5, 8]]];
+        }
+        return [];
+    }
+    readonly property string tip: isFloat ? (toggled ? "Back to Desktop" : "Float in VR") : ""
+
+    width: size
+    height: size
+
+    Rectangle {
+        anchors.fill: parent
+        radius: width / 2
+        visible: button.hovered || button.pressed || (button.toggled && !button.isFloat)
+        color: button.isClose ? (button.pressed ? "#c0392b" : "#da4453")
+                              : Qt.rgba(button.fg.r, button.fg.g, button.fg.b,
+                                        button.pressed ? 0.35 : button.hovered ? 0.2 : 0.12)
+    }
+
+    // On all desktops: a dot, filled while the window is on all of them. Help: a question mark.
+    Rectangle {
+        visible: button.buttonType === DecorationOptions.DecorationButtonOnAllDesktops
+        anchors.centerIn: parent
+        width: button.size * 0.3
+        height: width
+        radius: width / 2
+        color: button.toggled ? button.fg : "transparent"
+        border.width: Math.max(1, button.size / 18)
+        border.color: button.fg
+    }
+    Text {
+        visible: button.buttonType === DecorationOptions.DecorationButtonQuickHelp
+        anchors.centerIn: parent
+        text: "?"
+        color: button.fg
+        font.pixelSize: button.size * 0.65
+        font.bold: true
+    }
+
+    // Up to three strokes per glyph.
+    function stroke(i) {
+        const k = button.size / 18;
+        return i < glyph.length ? glyph[i].map(p => Qt.point(p[0] * k, p[1] * k)) : [];
+    }
+    readonly property color strokeColor: isClose && (hovered || pressed) ? "white" : fg
+    readonly property real strokeWidth: Math.max(1, size / 18 * 1.1)
+    // (An inline component can't see the ids around it, so everything comes in as properties.)
+    component Stroke: ShapePath {
+        property var points: []
+        fillColor: "transparent"
+        capStyle: ShapePath.RoundCap
+        joinStyle: ShapePath.RoundJoin
+        PathPolyline { path: points }
+    }
+    Shape {
+        anchors.fill: parent
+        preferredRendererType: Shape.CurveRenderer
+        Stroke { points: button.stroke(0); strokeColor: button.strokeColor; strokeWidth: button.strokeWidth }
+        Stroke { points: button.stroke(1); strokeColor: button.strokeColor; strokeWidth: button.strokeWidth }
+        Stroke { points: button.stroke(2); strokeColor: button.strokeColor; strokeWidth: button.strokeWidth }
+    }
+
+    onHoveredChanged: {
+        if (!tip || typeof decoration.requestShowToolTip !== "function")
+            return;
+        if (hovered)
+            decoration.requestShowToolTip(tip);
+        else
+            decoration.requestHideToolTip();
+    }
+    Component.onCompleted: {
+        if (buttonType === DecorationOptions.DecorationButtonQuickHelp)
+            visible = Qt.binding(() => decoration.client.providesContextHelp);
+        if (buttonType === DecorationOptions.DecorationButtonApplicationMenu)
+            visible = Qt.binding(() => decoration.client.hasApplicationMenu);
+        // Like Breeze: no On All Desktops button with only one virtual desktop.
+        if (buttonType === DecorationOptions.DecorationButtonOnAllDesktops)
+            visible = Qt.binding(() => decorationSettings.onAllDesktopsAvailable);
+    }
+}

decoration/contents/ui/main.qml

@@ -0,0 +1,270 @@
+/*
+    Frametop's window decoration: Breeze's flat title bar, drawn in QML for KWin's Aurorae
+    engine, plus a button left of Close that floats the window in VR (docs/floating-windows.md,
+    decision 25). Aurorae loads QML without compiling anything, so this keeps working across
+    SteamOS's KWin updates, which a C++ decoration wouldn't.
+
+    The float button is the Keep Below button (FtButton.qml): Frametop's KWin script floats a
+    window when keep-below is set and docks it when it's cleared. A Keep Below button in the
+    configured button order is left out, since the float button stands in for it.
+
+    SPDX-License-Identifier: GPL-2.0-or-later
+*/
+import QtQuick
+import org.kde.kwin.decoration
+
+Decoration {
+    id: root
+    alpha: false
+
+    DecorationOptions {
+        id: options
+        deco: decoration
+    }
+    TextMetrics {
+        id: metrics
+        font: options.titleFont
+        text: "Mj"
+    }
+
+    readonly property bool maximized: decoration.client.maximized
+    readonly property int buttonSize: Math.max(16, Math.round(metrics.height * 1.25))
+    readonly property int titleHeight: buttonSize + 8
+    readonly property int borderSize: decorationSettings.borderSize
+    readonly property int side: {
+        switch (borderSize) {
+        case DecorationOptions.BorderNone:
+        case DecorationOptions.BorderNoSides: return 0;
+        case DecorationOptions.BorderTiny: return 2;
+        case DecorationOptions.BorderLarge: return 6;
+        case DecorationOptions.BorderVeryLarge: return 8;
+        case DecorationOptions.BorderHuge: return 12;
+        case DecorationOptions.BorderVeryHuge: return 18;
+        case DecorationOptions.BorderOversized: return 27;
+        default: return 4;
+        }
+    }
+    readonly property int bottomBorder: borderSize === DecorationOptions.BorderNone ? 0
+                                : borderSize === DecorationOptions.BorderNoSides ? 4 : side
+    readonly property color outline: Qt.tint(options.titleBarColor, Qt.rgba(0, 0, 0, 0.35))
+
+    function applyBorders() {
+        borders.left = side;
+        borders.right = side;
+        borders.bottom = bottomBorder;
+        borders.top = titleHeight;
+        maximizedBorders.top = titleHeight;
+        // Without visible side borders, keep a strip to grab for resizing.
+        extendedBorders.left = side ? 0 : 4;
+        extendedBorders.right = side ? 0 : 4;
+        extendedBorders.bottom = bottomBorder ? 0 : 4;
+    }
+    onTitleHeightChanged: applyBorders()
+    onSideChanged: applyBorders()
+    onBottomBorderChanged: applyBorders()
+    Component.onCompleted: applyBorders()
+
+    // The configured buttons, with the float button left of Close (or first on the right
+    // when there's no Close), and no Keep Below of their own.
+    function order(list, right) {
+        const out = [];
+        let placed = false;
+        for (let i = 0; i < list.length; ++i) {
+            const t = list[i];
+            if (t === DecorationOptions.DecorationButtonKeepBelow)
+                continue;
+            if (t === DecorationOptions.DecorationButtonClose && !placed) {
+                if (right) {
+                    out.push(DecorationOptions.DecorationButtonKeepBelow, t);
+                } else {
+                    out.push(t, DecorationOptions.DecorationButtonKeepBelow);
+                }
+                placed = true;
+                continue;
+            }
+            out.push(t);
+        }
+        return {buttons: out, placed: placed};
+    }
+    readonly property var leftOrder: order(options.titleButtonsLeft || [], false)
+    readonly property var rightOrder: {
+        const r = order(options.titleButtonsRight || [], true);
+        if (!r.placed && !leftOrder.placed)
+            r.buttons.unshift(DecorationOptions.DecorationButtonKeepBelow);
+        return r;
+    }
+
+    function componentFor(t) {
+        switch (t) {
+        case DecorationOptions.DecorationButtonMenu: return menuButton;
+        case DecorationOptions.DecorationButtonExplicitSpacer: return spacer;
+        case DecorationOptions.DecorationButtonClose: return closeButton;
+        case DecorationOptions.DecorationButtonMaximizeRestore: return maximizeButton;
+        case DecorationOptions.DecorationButtonMinimize: return minimizeButton;
+        case DecorationOptions.DecorationButtonKeepBelow: return floatButton;
+        case DecorationOptions.DecorationButtonKeepAbove: return keepAboveButton;
+        case DecorationOptions.DecorationButtonShade: return shadeButton;
+        case DecorationOptions.DecorationButtonOnAllDesktops: return allDesktopsButton;
+        case DecorationOptions.DecorationButtonQuickHelp: return helpButton;
+        case DecorationOptions.DecorationButtonApplicationMenu: return appMenuButton;
+        }
+        return null;
+    }
+
+    Rectangle {
+        anchors.fill: parent
+        color: options.titleBarColor
+        border.width: root.maximized ? 0 : 1
+        border.color: root.outline
+    }
+
+    Item {
+        id: titleBar
+        x: root.maximized ? 0 : Math.max(root.side, 1)
+        y: root.maximized ? 0 : 1
+        width: root.width - 2 * x
+        height: root.titleHeight - y
+
+        Row {
+            id: leftButtons
+            anchors.left: parent.left
+            anchors.leftMargin: 4
+            anchors.verticalCenter: parent.verticalCenter
+            spacing: 4
+            Repeater {
+                model: root.leftOrder.buttons
+                delegate: Loader {
+                    required property var modelData
+                    sourceComponent: root.componentFor(modelData)
+                }
+            }
+        }
+        Row {
+            id: rightButtons
+            anchors.right: parent.right
+            anchors.rightMargin: 4
+            anchors.verticalCenter: parent.verticalCenter
+            spacing: 4
+            layoutDirection: Qt.LeftToRight
+            Repeater {
+                model: root.rightOrder.buttons
+                delegate: Loader {
+                    required property var modelData
+                    sourceComponent: root.componentFor(modelData)
+                }
+            }
+        }
+        // Centred over the whole bar, like Breeze, but never under the buttons.
+        Text {
+            id: caption
+            readonly property real free: rightButtons.x - (leftButtons.x + leftButtons.width) - 16
+            width: Math.min(implicitWidth, free)
+            x: Math.max(leftButtons.x + leftButtons.width + 8,
+                        Math.min((parent.width - width) / 2, rightButtons.x - 8 - width))
+            anchors.verticalCenter: parent.verticalCenter
+            text: decoration.client.caption
+            textFormat: Text.PlainText
+            font: options.titleFont
+            color: options.fontColor
+            elide: Text.ElideRight
+            renderType: Text.NativeRendering
+        }
+        Component.onCompleted: decoration.installTitleItem(titleBar)
+    }
+
+    Component {
+        id: menuButton
+        MenuButton {
+            width: root.buttonSize
+            height: root.buttonSize
+        }
+    }
+    Component {
+        id: spacer
+        Item {
+            width: root.buttonSize
+            height: root.buttonSize
+        }
+    }
+    Component {
+        id: closeButton
+        FtButton {
+            buttonType: DecorationOptions.DecorationButtonClose
+            size: root.buttonSize
+            fg: options.fontColor
+            bg: options.titleBarColor
+        }
+    }
+    Component {
+        id: maximizeButton
+        FtButton {
+            buttonType: DecorationOptions.DecorationButtonMaximizeRestore
+            size: root.buttonSize
+            fg: options.fontColor
+            bg: options.titleBarColor
+        }
+    }
+    Component {
+        id: minimizeButton
+        FtButton {
+            buttonType: DecorationOptions.DecorationButtonMinimize
+            size: root.buttonSize
+            fg: options.fontColor
+            bg: options.titleBarColor
+        }
+    }
+    Component {
+        id: floatButton
+        FtButton {
+            buttonType: DecorationOptions.DecorationButtonKeepBelow
+            size: root.buttonSize
+            fg: options.fontColor
+            bg: options.titleBarColor
+        }
+    }
+    Component {
+        id: keepAboveButton
+        FtButton {
+            buttonType: DecorationOptions.DecorationButtonKeepAbove
+            size: root.buttonSize
+            fg: options.fontColor
+            bg: options.titleBarColor
+        }
+    }
+    Component {
+        id: shadeButton
+        FtButton {
+            buttonType: DecorationOptions.DecorationButtonShade
+            size: root.buttonSize
+            fg: options.fontColor
+            bg: options.titleBarColor
+        }
+    }
+    Component {
+        id: allDesktopsButton
+        FtButton {
+            buttonType: DecorationOptions.DecorationButtonOnAllDesktops
+            size: root.buttonSize
+            fg: options.fontColor
+            bg: options.titleBarColor
+        }
+    }
+    Component {
+        id: helpButton
+        FtButton {
+            buttonType: DecorationOptions.DecorationButtonQuickHelp
+            size: root.buttonSize
+            fg: options.fontColor
+            bg: options.titleBarColor
+        }
+    }
+    Component {
+        id: appMenuButton
+        FtButton {
+            buttonType: DecorationOptions.DecorationButtonApplicationMenu
+            size: root.buttonSize
+            fg: options.fontColor
+            bg: options.titleBarColor
+        }
+    }
+}

decoration/metadata.json

@@ -0,0 +1,14 @@
+{
+    "KPackageStructure": "KWin/Decoration",
+    "KPlugin": {
+        "Authors": [
+            {
+                "Name": "Frametop"
+            }
+        ],
+        "Description": "Breeze-style window decoration with a button that floats the window in VR (Frametop desktop)",
+        "Id": "kwin4_decoration_qml_frametop",
+        "License": "GPL",
+        "Name": "Frametop"
+    }
+}

desktops.sh

@@ -40,7 +40,12 @@ $running && echo 'started' || { echo 'failed:'; tail -20 $log; exit 1; }" ;;
 sed 's|@SESSION@|$session/frametop-session.sh|' $session/deckard-nested-desktop.desktop > ~/$override
 [ -f ~/.config/frametop.conf ] || cp $session/frametop.conf.example ~/.config/frametop.conf
 echo \"installed ~/$override\"; grep ^Exec= ~/$override; echo; cat ~/.config/frametop.conf" ;;
-  uninstall) "$frame" --host "rm -f ~/$override && echo 'removed; the launcher uses the stock desktop again'" ;;
+  uninstall)
+    # Also what the session puts in place at each start: Launch as Standalone's app copies and
+    # the title bar decoration (float/ft_apps.py, decoration/).
+    "$frame" --host "rm -f ~/$override
+rm -rf ~/.local/share/frametop/apps ~/.local/share/kwin/decorations/kwin4_decoration_qml_frametop
+rmdir ~/.local/share/frametop 2>/dev/null; echo 'removed; the launcher uses the stock desktop again'" ;;
   screens)
     [[ ${2:-} =~ ^[1-9]$ ]] || { echo "usage: $0 screens N   (1-9)" >&2; exit 2; }
     "$frame" --host "set -e; f=~/.config/frametop.conf

display-settings/ft-layout-reset.desktop

@@ -3,7 +3,7 @@ Type=Application
 Name=Reset Screen Layout
 GenericName=Put the VR desktop's screens back in their layout
 Comment=Float the screens and arrange them in the layout from Frametop Display Settings
-Exec=@REPO@/layout/ft-layout apply
+Exec=@REPO@/layout/ft-layout-reset
 Icon=view-restore
 Categories=Settings;
 Keywords=display;screen;layout;arrange;reset;steamvr;frametop;

display-settings/ft-screens-toggle.desktop

@@ -3,7 +3,7 @@ Type=Application
 Name=Hide/Show Screens
 GenericName=Hide or show the VR desktop's screens
 Comment=Hide the screens (and SteamVR's laser) for a VR game; press again to bring them back
-Exec=@REPO@/layout/ft-layout toggle
+Exec=@REPO@/layout/ft-hide-show
 Icon=view-visible
 Categories=Settings;
 Keywords=display;screen;hide;show;steamvr;frametop;

display-settings/ft_display_settings.py

@@ -409,6 +409,31 @@ class Backend(QObject):
     def pins(self):
         return self._pins
 
+    @Property("QVariantList", notify=changed)
+    def screensShown(self):
+        """For each screen, whether it shows (False: hidden on its own, ft-layout hide N)."""
+        layout = ft_layout.load_layout()
+        return [not ft_layout.screen_entry(layout, i).get("hidden") for i in range(ft_layout.screen_count(layout))]
+
+    @Slot(int, bool)
+    def setScreenShown(self, index, shown):
+        """Hide screen `index` (0-based) on its own, whatever the visibility mode, or show it."""
+        layout = ft_layout.load_layout()
+        screens = layout.setdefault("screens", [])
+        while len(screens) <= index:
+            screens.append({})
+        if shown:
+            screens[index].pop("hidden", None)
+        else:
+            screens[index]["hidden"] = True
+        ft_layout.save_layout(layout)
+        self.changed.emit()
+        if self._running:
+            reply = self._ask_screens(f"{'reveal' if shown else 'conceal'} {index + 1}")
+            if not (reply and reply.startswith("ok")):
+                self.message.emit("Saved; the desktop applies it when it next starts "
+                                  "(its compositor is older than hiding screens one at a time)", False)
+
     @Slot(str, str)
     def pin(self, which, where):
         """Pin screen `which` (1-based, or "all") to "left", "right", or "head" as it is
@@ -420,7 +445,7 @@ class Backend(QObject):
                               f"{reply or 'the desktop is not running'}", True)
         elif which == "all" and where != "none":
             place = "on your head" if where == "head" else f"on your {where} wrist"
-            self.message.emit(f"All screens ride {place} now. Save current arrangement (Layout) keeps it.", False)
+            self.message.emit(f"All screens ride {place} now. Save as profile… (Layout & profiles) keeps it.", False)
         self._check_running()
 
     # --- power: ft-powerd and Steam's sleep setting ---
@@ -559,16 +584,55 @@ class Backend(QObject):
     def renameLayout(self, old, new):
         try:
             self._edit_layout(lambda l: ft_layout.rename_named(l, old, new))
-        except RuntimeError as e:
+            ft_layout.write_launchers(ft_layout.load_layout())
+        except (RuntimeError, OSError) as e:
             self.message.emit(str(e), True)
 
     @Slot(str)
     def deleteLayout(self, name):
         try:
             self._edit_layout(lambda l: ft_layout.delete_named(l, name))
-        except RuntimeError as e:
+            ft_layout.write_launchers(ft_layout.load_layout())
+        except (RuntimeError, OSError) as e:
             self.message.emit(str(e), True)
 
+    # --- profiles (docs/profiles.md): a named layout's apps and hidden screens ---
+    @Slot(str, result="QVariantList")
+    def profileWindows(self, name):
+        """A profile's windows, as "app" and "where" for the list."""
+        out = []
+        for e in ft_layout.load_layout().get("profiles", {}).get(name, {}).get("windows", []):
+            app = e.get("app") or os.path.basename((e.get("cmd") or ["?"])[0])
+            app = app.rsplit(".", 1)[-1] if "." in app and not e.get("cmd") else app
+            where = "floating" if "float" in e else f"screen {e.get('screen', 1)}" + (", maximized" if e.get("maximized") else "")
+            out.append({"app": app, "where": where})
+        return out
+
+    @Slot(str, result="QVariantList")
+    def profileHidden(self, name):
+        return ft_layout.load_layout().get("profiles", {}).get(name, {}).get("hidden", [])
+
+    @Slot(str, int)
+    def removeProfileWindow(self, name, index):
+        def edit(layout):
+            windows = layout.get("profiles", {}).get(name, {}).get("windows", [])
+            if 0 <= index < len(windows):
+                windows.pop(index)
+        self._edit_layout(edit)
+
+    @Property(str, notify=changed)
+    def defaultProfile(self):
+        return ft_layout.load_layout().get("default_profile", "")
+
+    @Slot(str)
+    def setDefaultProfile(self, name):
+        def edit(layout):
+            if name:
+                layout["default_profile"] = name
+            else:
+                layout.pop("default_profile", None)
+        self._edit_layout(edit)
+
     @Slot(str, "QVariant")
     def setPreset(self, key, value):
         def edit(layout):
@@ -582,7 +646,13 @@ class Backend(QObject):
 
     @Slot()
     def arrange(self):
-        self._run("Arranging the screens", "apply")
+        """Arrange the screens; in a profile, also open its apps (ft-layout use)."""
+        layout = ft_layout.load_layout()
+        name = layout.get("active")
+        if layout.get("mode") == "custom" and name in layout.get("layouts", {}):
+            self._run(f"Opening {name}", "use", name)
+        else:
+            self._run("Arranging the screens", "apply")
 
     @Slot()
     def capture(self):

display-settings/install.sh

@@ -17,10 +17,12 @@ case ${1:-install} in
     on_frame "chmod +x display-settings/ft-display-settings layout/ft-layout layout/ft_layout.py
 mkdir -p ~/.config/frametop
 kwriteconfig6 --file ~/$shortcuts --group services --group ft-layout-reset.desktop --key _launch 'Meta+Shift+R'
-kwriteconfig6 --file ~/$shortcuts --group services --group ft-screens-toggle.desktop --key _launch 'Meta+Shift+H'"
+kwriteconfig6 --file ~/$shortcuts --group services --group ft-screens-toggle.desktop --key _launch 'Meta+Shift+H'
+layout/ft-layout launchers  # each profile's entry (Frametop: NAME), if there are profiles"
     echo "installed: Frametop Display Settings, Reset Screen Layout (Meta+Shift+R), Hide/Show Screens (Meta+Shift+H)" ;;
   uninstall)
     on_frame "rm -f ~/$apps/ft-display-settings.desktop ~/$apps/ft-layout-reset.desktop ~/$apps/ft-screens-toggle.desktop
+rm -f ~/$apps/frametop-profile-*.desktop  # the profiles' entries (the profiles stay in ~/.config/frametop-layout.json)
 [ -f ~/$shortcuts ] && for f in ft-layout-reset ft-screens-toggle; do kwriteconfig6 --file ~/$shortcuts --group services --group \$f.desktop --key _launch --delete; done
 echo removed" ;;
   *) echo "usage: $0 [install|uninstall]" >&2; exit 2 ;;

display-settings/main.qml

@@ -411,8 +411,8 @@ Kirigami.ApplicationWindow {
                 text: spage.md
                       ? "Each screen is a real monitor of its own: any resolution, portrait by choosing a tall one. Resolution, "
                         + "width, and curve apply at once. In VR: move a screen by the bar underneath, curve it with the round "
-                        + "button next to the bar, resize it by the tab on its bottom right corner; Save current arrangement on the "
-                        + "Layout page keeps all of it."
+                        + "button next to the bar, resize it by the tab on its bottom right corner; Save as profile… on the "
+                        + "Layout & profiles page keeps all of it."
                       : "gamescope draws every screen at the same resolution, at most 1920 × 1080 worth of pixels. Portrait turns "
                         + "a screen on its side. Rotation and scale apply at once; the rest when the desktop starts."
             }
@@ -424,7 +424,7 @@ Kirigami.ApplicationWindow {
         id: layoutPage
         Kirigami.ScrollablePage {
             id: lpage
-            title: "Layout"
+            title: "Layout & profiles"
             property var layout: backend.layout
             property var preset: layout.preset || {}
             property bool hasCustom: (layout.screens || []).some(s => s.pos !== undefined)
@@ -440,16 +440,17 @@ Kirigami.ApplicationWindow {
 
             actions: [
                 Kirigami.Action {
-                    text: "Arrange now"
+                    text: lpage.named ? "Open profile" : "Arrange now"
                     icon.name: "view-restore"
-                    tooltip: "Float the screens out of the dashboard and put them in this layout, around where you're facing"
+                    tooltip: lpage.named ? "Put the screens in this profile's places, around where you're facing, and open its apps (windows already open move; nothing closes)"
+                                         : "Float the screens out of the dashboard and put them in this layout, around where you're facing"
                     enabled: backend.desktopRunning && backend.busy === ""
                     onTriggered: backend.arrange()
                 },
                 Kirigami.Action {
-                    text: "Save current arrangement…"
+                    text: "Save as profile…"
                     icon.name: "document-save"
-                    tooltip: "Save where the screens are now (placed by hand) as a named layout, and use it"
+                    tooltip: "Save where the screens are now, which ones are hidden, and the open apps and where their windows are, under a name"
                     enabled: backend.desktopRunning && backend.busy === ""
                     onTriggered: nameDialog.openFor("save", lpage.named ? lpage.layout.active
                                                                         : "Layout " + (lpage.names.length + 1))
@@ -507,16 +508,61 @@ Kirigami.ApplicationWindow {
                     Controls.Label {
                         visible: lpage.layout.mode === "custom"
                         Kirigami.FormData.label: ""
-                        text: lpage.named ? "Where the screens were when you saved it. Arrange now puts them there. "
-                                            + "Save current arrangement updates it or saves a new one."
-                              : lpage.hasCustom ? "Where the screens were when you saved. Save current arrangement "
+                        text: lpage.named ? "Where the screens were when you saved it, and the apps that were open. Open "
+                                            + "profile puts the screens there and opens the apps. Save as profile updates "
+                                            + "it or saves a new one."
+                              : lpage.hasCustom ? "Where the screens were when you saved. Save as profile "
                                                   + "names it. Pick a preset to edit."
-                              : "Nothing saved yet: place the screens by hand, then Save current arrangement."
+                              : "Nothing saved yet: place the screens by hand, open your apps, then Save as profile."
                         opacity: 0.7
                         wrapMode: Text.Wrap
                         Layout.maximumWidth: Kirigami.Units.gridUnit * 20
                     }
 
+                    // The profile's apps (docs/profiles.md): each window and where it goes.
+                    ColumnLayout {
+                        id: profileApps
+                        visible: lpage.named
+                        Kirigami.FormData.label: "Apps:"
+                        property var windows: lpage.named ? backend.profileWindows(lpage.layout.active) : []
+                        property var hidden: lpage.named ? backend.profileHidden(lpage.layout.active) : []
+                        Connections {
+                            target: backend
+                            function onChanged() {
+                                profileApps.windows = lpage.named ? backend.profileWindows(lpage.layout.active) : []
+                                profileApps.hidden = lpage.named ? backend.profileHidden(lpage.layout.active) : []
+                            }
+                        }
+                        Controls.Label {
+                            visible: profileApps.windows.length === 0
+                            text: "None saved. Open the apps you want, place their windows, then Save as profile."
+                            opacity: 0.7
+                            wrapMode: Text.Wrap
+                            Layout.maximumWidth: Kirigami.Units.gridUnit * 20
+                        }
+                        Repeater {
+                            model: profileApps.windows
+                            delegate: RowLayout {
+                                required property var modelData
+                                required property int index
+                                Controls.Label { text: modelData.app + " (" + modelData.where + ")" }
+                                Controls.ToolButton {
+                                    icon.name: "list-remove"
+                                    text: "Leave out"
+                                    display: Controls.AbstractButton.IconOnly
+                                    Controls.ToolTip.text: "Leave this window out of the profile"
+                                    Controls.ToolTip.visible: hovered
+                                    onClicked: backend.removeProfileWindow(lpage.layout.active, index)
+                                }
+                            }
+                        }
+                        Controls.Label {
+                            visible: profileApps.hidden.length > 0
+                            text: "Hides screen" + (profileApps.hidden.length > 1 ? "s " : " ") + profileApps.hidden.join(", ")
+                            opacity: 0.7
+                        }
+                    }
+
                     Controls.SpinBox {
                         Kirigami.FormData.label: "Rows:"
                         visible: lpage.layout.mode !== "custom"
@@ -554,8 +600,19 @@ Kirigami.ApplicationWindow {
                         Kirigami.FormData.label: "When the desktop starts:"
                         text: "Float the screens and arrange them"
                         checked: lpage.layout.auto !== false
+                        enabled: backend.defaultProfile === ""
                         onToggled: backend.setAuto(checked)
                     }
+                    Controls.ComboBox {
+                        Kirigami.FormData.label: "Start in profile:"
+                        model: [{ text: "None", value: "" }].concat(lpage.names.map(n => ({ text: n, value: n })))
+                        textRole: "text"
+                        valueRole: "value"
+                        currentIndex: Math.max(0, indexOfValue(backend.defaultProfile))
+                        onActivated: backend.setDefaultProfile(currentValue)
+                        Controls.ToolTip.text: "The desktop starts in this profile: its screens, and its apps open. Each profile also has its own entry in SteamVR's Launch a program list"
+                        Controls.ToolTip.visible: hovered
+                    }
                 }
 
                 // Preview: from above (you at the bottom) and from the front.
@@ -745,7 +802,7 @@ Kirigami.ApplicationWindow {
                     Repeater {
                         model: [
                             { value: "hide", text: "Hide them unless the SteamVR dashboard is open", help: "The game has the view to itself; open the dashboard (or press Meta+Shift+H) to see the screens." },
-                            { value: "visible", text: "Keep them visible over the game", help: "They float over the game as they are outside it." }
+                            { value: "visible", text: "Keep them visible over the game", help: "They float over the game as they are outside it. Turn off Pause while a VR game runs in Frametop Input Settings (Game optimization), or Frametop pauses and hides them anyway." }
                         ]
                         delegate: ColumnLayout {
                             required property var modelData
@@ -802,6 +859,27 @@ Kirigami.ApplicationWindow {
                         }
                     }
 
+                    Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Screens shown" }
+
+                    Repeater {
+                        model: backend.screensShown
+                        delegate: Controls.Switch {
+                            required property var modelData
+                            required property int index
+                            Kirigami.FormData.label: "Screen " + (index + 1) + ":"
+                            text: modelData ? "Shown" : "Hidden"
+                            checked: modelData
+                            onToggled: backend.setScreenShown(index, checked)
+                        }
+                    }
+                    Controls.Label {
+                        text: "A hidden screen stays hidden whatever the choices above say, and Meta+Shift+H doesn't bring it back. Windows on it stay there; new ones that would open on it float instead."
+                        opacity: 0.7
+                        font: Kirigami.Theme.smallFont
+                        wrapMode: Text.Wrap
+                        Layout.maximumWidth: Kirigami.Units.gridUnit * 26
+                    }
+
                     Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Pinned screens" }
 
                     Repeater {
@@ -871,7 +949,7 @@ Kirigami.ApplicationWindow {
                           + "it.\n\nPin a screen to your head: choose On your head above. It rides on the headset where it "
                           + "is now, like a HUD, and shows whenever the screens do. Grab its bar to move it; it stays on "
                           + "your head where you let go. Choosing a pin above keeps the screen where it is now, so place "
-                          + "it first. Save current arrangement (Layout) keeps pins."
+                          + "it first. Save as profile… (Layout) keeps pins."
                 }
             }
         }

display-settings/steam_settings.py

@@ -4,117 +4,26 @@ Steam, not systemd, puts the Frame to sleep: after "When Plugged In and Idle ->
 (an hour by default) without input, even while it charges. That's a Steam client setting,
 `system_idle_suspend_ac_sec` (0 = never), with no file or command line to change it. Steam
 on the Frame runs with -cef-enable-debugging, so its UI's JavaScript context
-(SharedJSContext) is reachable over the Chrome DevTools Protocol on 127.0.0.1:8080. There
-`settingsStore.clientSettings` has the current values, and `SteamClient.Settings.SetSetting`
-takes a change as a serialized CMsgClientSettings protobuf, which is what Steam's own
-Settings -> Power page sends. Standard library only (a minimal WebSocket client).
+(SharedJSContext) is reachable over the Chrome DevTools Protocol on 127.0.0.1:8080
+(steam/steamui.py). There `settingsStore.clientSettings` has the current values, and
+`SteamClient.Settings.SetSetting` takes a change as a serialized CMsgClientSettings protobuf,
+which is what Steam's own Settings -> Power page sends.
 """
-import base64
-import json
 import os
-import socket
-import struct
-import urllib.request
+import sys
+
+sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "steam"))
+from steamui import SteamUnreachable, evaluate  # noqa: E402,F401  (callers catch SteamUnreachable here)
 
-CDP_PORT = 8080
 # CMsgClientSettings field numbers (Steam's UI bundle maps the names to these).
 FIELDS = {"system_idle_suspend_ac_sec": 24004, "system_idle_suspend_battery_sec": 24003}
 
 
-class SteamUnreachable(Exception):
-    pass
-
-
-class _WebSocket:
-    def __init__(self, url, timeout=5):
-        host_port, path = url[len("ws://"):].split("/", 1)
-        host, port = host_port.rsplit(":", 1)
-        self.sock = socket.create_connection((host, int(port)), timeout=timeout)
-        key = base64.b64encode(os.urandom(16)).decode()
-        self.sock.sendall((f"GET /{path} HTTP/1.1\r\nHost: {host_port}\r\nUpgrade: websocket\r\n"
-                           f"Connection: Upgrade\r\nSec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n\r\n").encode())
-        head = b""
-        while b"\r\n\r\n" not in head:
-            chunk = self.sock.recv(4096)
-            if not chunk:
-                raise SteamUnreachable("Steam closed the connection")
-            head += chunk
-        if b" 101 " not in head.split(b"\r\n", 1)[0]:
-            raise SteamUnreachable(head.split(b"\r\n", 1)[0].decode(errors="replace"))
-        self.buf = head.split(b"\r\n\r\n", 1)[1]
-
-    def send(self, text):
-        data, mask = text.encode(), os.urandom(4)
-        n = len(data)
-        if n < 126:
-            head = struct.pack(">BB", 0x81, 0x80 | n)
-        elif n < 65536:
-            head = struct.pack(">BBH", 0x81, 0x80 | 126, n)
-        else:
-            head = struct.pack(">BBQ", 0x81, 0x80 | 127, n)
-        self.sock.sendall(head + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(data)))
-
-    def _take(self, n):
-        while len(self.buf) < n:
-            chunk = self.sock.recv(65536)
-            if not chunk:
-                raise SteamUnreachable("Steam closed the connection")
-            self.buf += chunk
-        out, self.buf = self.buf[:n], self.buf[n:]
-        return out
-
-    def recv(self):
-        message = b""
-        while True:
-            b0, b1 = self._take(2)
-            n = b1 & 0x7F
-            if n == 126:
-                n = struct.unpack(">H", self._take(2))[0]
-            elif n == 127:
-                n = struct.unpack(">Q", self._take(8))[0]
-            message += self._take(n)
-            if b0 & 0x80:
-                return message.decode(errors="replace")
-
-    def close(self):
-        self.sock.close()
-
-
-def _evaluate(expression):
-    """Runs JavaScript in Steam's SharedJSContext and returns its (awaited) value."""
-    try:
-        with urllib.request.urlopen(f"http://127.0.0.1:{CDP_PORT}/json", timeout=3) as r:
-            targets = json.load(r)
-    except OSError as e:
-        raise SteamUnreachable(f"Steam isn't reachable on port {CDP_PORT} ({e})") from e
-    url = next((t["webSocketDebuggerUrl"] for t in targets if t.get("title") == "SharedJSContext"), None)
-    if not url:
-        raise SteamUnreachable("Steam's UI isn't running")
-    try:
-        ws = _WebSocket(url)
-        try:
-            ws.send(json.dumps({"id": 1, "method": "Runtime.evaluate",
-                                "params": {"expression": expression, "awaitPromise": True, "returnByValue": True}}))
-            while True:
-                reply = json.loads(ws.recv())
-                if reply.get("id") == 1:
-                    break
-        finally:
-            ws.close()
-    except OSError as e:
-        raise SteamUnreachable(str(e)) from e
-    result = reply.get("result", {})
-    if "exceptionDetails" in result:
-        details = result["exceptionDetails"]
-        raise SteamUnreachable(details.get("exception", {}).get("description") or details.get("text", "error"))
-    return result.get("result", {}).get("value")
-
-
 def sleep_settings():
     """{"ac": seconds, "battery": seconds}: when Steam puts the Frame to sleep without input,
     plugged in and on battery (0 = never)."""
-    value = _evaluate("(() => { const c = settingsStore.clientSettings; "
-                      "return {ac: c.system_idle_suspend_ac_sec, battery: c.system_idle_suspend_battery_sec}; })()")
+    value = evaluate("(() => { const c = settingsStore.clientSettings; "
+                     "return {ac: c.system_idle_suspend_ac_sec, battery: c.system_idle_suspend_battery_sec}; })()")
     if not isinstance(value, dict) or not all(isinstance(value.get(k), int) for k in ("ac", "battery")):
         raise SteamUnreachable("Steam's settings don't have the sleep timeouts")
     return value
@@ -125,7 +34,7 @@ def set_sleep_setting(name, seconds):
     field, seconds = FIELDS[name], int(seconds)
     if seconds < 0:
         raise ValueError("seconds must be 0 (never) or more")
-    ok = _evaluate(f"""(async () => {{
+    ok = evaluate(f"""(async () => {{
         const bytes = [];
         const varint = n => {{ while (n > 127) {{ bytes.push((n & 127) | 128); n = Math.floor(n / 128); }} bytes.push(n); }};
         varint({field} * 8); varint({seconds});

docs/controller-click-stability.md

@@ -0,0 +1,34 @@
+# Controller desktop click stability
+
+Trigger presses reach KDE immediately, but controller motion within 32 logical
+pixels of the press stays at that position until release. Releasing without a motion outside this
+zone delivers the click at the original position, even if the hand moved during
+release. Moving outside the zone begins a normal drag immediately; returning to
+the zone does not turn it back into a click. There is no hold-duration timer.
+
+This filters overlay pointer content events on desktop monitors only, and only
+presses from hand controllers start it. The 3D mouse (whose laser comes from the
+`ft_pointer` virtual controller), SteamVR UI, separate screen grab bars and
+floating-app title-bar carrying are unaffected. Multi-button gestures keep their
+existing behavior. A motion onto another desktop monitor starts a drag;
+cross-monitor motion is not stabilized.
+
+CLI (runtime preferences, reset to 32 on desktop restart):
+
+```sh
+input/ft-clickctl status
+input/ft-clickctl threshold 32
+input/ft-clickctl threshold 0  # disable without a restart
+```
+
+Thresholds are 0–64 logical pixels, normalized to each panel's KDE scale.
+Status reports held state, suppressed motions, stabilized clicks and drags.
+Changing the threshold while a controller button is held is refused.
+
+This is a separate contribution from desktop mouse/controller ownership. Its
+hardware validation must check small controls, intentional text selection,
+long presses, cross-monitor dragging and simultaneous mouse use. The existing
+renderer laser remains tracked; this change stabilizes desktop input rather
+than smoothing the visual laser. The default was 8 at first. That's about 0.2° on a 3.4 m wide 3440-pixel screen 2 m away, and clicking took a very still hand, so it's 32 (about 0.9°) since 2026-10-03.
+
+Run `scripts/test-controller-click.sh` for the isolated gesture-state tests.

docs/design.md

@@ -38,7 +38,7 @@ The curved layout chains screens edge to edge, like monitors on a desk: the midd
 
 A resize handle has to be able to shrink a screen from any direction, so the dragged corner follows the laser along the screen's diagonal rather than taking the larger of its horizontal and vertical reach. Pushing and pulling a carried screen moves it along the line from your head, because the 3D mouse's virtual controller sits just in front of the bar, below the screen's centre, so the line from the device points mostly upward.
 
-Wherever ft-screens needs to know where a laser points (showing the controls, the resize tab, the roll knob), it uses the laser's own pose, the render model's `tip` component, rather than the controller's pose. On the Frame's controllers the tip points 40° below the pose's forward axis, so rays from the pose missed what the laser was actually on. The 3D mouse's virtual controller has no tip, and its laser runs along its pose.
+Wherever ft-screens needs to know where a laser points (showing the controls, the resize tab, the roll knob), it uses the laser's own pose, the render model's `tip` component, rather than the controller's pose. On the Frame's controllers the tip points 40° below the pose's forward axis, so rays from the pose missed what the laser was actually on. The 3D mouse's virtual controller has no tip, and its laser runs along its pose. ft-screens reads the tip with `GetComponentState`: `GetComponentStateForDevicePath` without an input source handle fails for every component while a VR game runs, so in games the rays came from the pose, 40° too high.
 
 `ComputeOverlayIntersection` ignores `SetOverlayIntersectionMask`, and a control can't be allowed to cover part of its screen, so the resize tab sits entirely outside the corner.
 
@@ -52,7 +52,9 @@ A head pin is the same pin on the headset (device index 0): the screen's transfo
 
 ### Named layouts
 
-A named layout is the custom arrangement under a name: each screen's pose relative to your head, width, curve, and pin, but not its resolution or scale, which need a desktop restart or belong to KWin. Using one copies it into the custom arrangement, so everything that applies the layout (desktop start, Meta+Shift+R, Arrange now) works unchanged, and `active` remembers which name it came from. Saving without a name (`ft-layout capture`) clears `active`, because the screens have been placed by hand since. Layouts are kept per screen number, so one saved with a different screen count still applies: missing screens keep their last saved place or the preset's.
+Named layouts are now profiles, which also hold which screens are hidden and which apps to open, with where their windows go. See [profiles.md](profiles.md).
+
+A profile's screen part is the custom arrangement under a name: each screen's pose relative to your head, width, curve, and pin, but not its resolution or scale, which need a desktop restart or belong to KWin. Using one copies it into the custom arrangement, so everything that applies the layout (desktop start, Meta+Shift+R, Arrange now) works unchanged, and `active` remembers which name it came from. Saving without a name (`ft-layout capture`) clears `active`, because the screens have been placed by hand since. Layouts are kept per screen number, so one saved with a different screen count still applies: missing screens keep their last saved place or the preset's.
 
 ### Visibility and VR games
 
@@ -60,6 +62,46 @@ A named layout is the custom arrangement under a name: each screen's pose relati
 
 `IVRApplications::GetCurrentSceneProcessId()` is 0 when no game is running (the Frame's home environment isn't a scene app) and the game's process ID while one is. ft-screens checks it twice a second, turns the flag off while a game runs, and by default hides the screens unless the dashboard is open. Flatscreen games run inside Steam's gamescope overlay and aren't scene apps, which is why "only with the dashboard open" is offered as a controller setting.
 
+In a game, Frametop's panels work like SteamVR's own floating windows: point a controller at one and its laser comes on, point away and the game has the controllers again. ft-screens turns the flag on for a panel while a hand controller's laser pose meets it, its controls, or a floating window's popups. It finds that from the poses it already reads to show the controls, so SteamVR's laser doesn't have to be on first. Leaving takes a margin two control-sizes wide and 0.3 s, a drag or a held button keeps the flag on, and the keyboard, a single overlay, uses SteamVR's `ComputeOverlayIntersection`. The 3D mouse doesn't need any of this: it has its own laser mode.
+
+## Floating windows
+
+[floating-windows.md](floating-windows.md) describes the feature and its parts. Drag and drop and the clipboard only work between windows of one compositor, so a floating window stays a KWin window and gets a KWin output of its own: one of the spare outputs KWin opens after the screens, shown by ft-screens as a panel cropped to the window. What follows is how KWin 6.2.5 behaves underneath that, from its source (`src/backends/wayland/`) and from trying it on the Frametop desktop.
+
+### KWin's nested outputs
+
+- Disabling a nested output keeps its host window. `Output::applyChanges` only flips `enabled`, KWin stops rendering it (no more commits), and Plasma drops its desktop view. So ft-screens keeps the same toplevel, and its screen numbers stay put. Enabling the output again resumes on the same toplevel.
+- Each output's host window is titled `KDE Wayland Compositor WL-<n>`, with `- Output disabled` appended while it's disabled (`WaylandOutput::updateWindowTitle`, on every `enabledChanged`). ft-screens reads the title to tell screens (`WL-0` to `WL-<SCREENS-1>`) from spares, and to see a spare turn on and off.
+- A spare resized while it's disabled comes up at the new size on its first frame, so floating a window needn't blink. Outputs with gaps between them are accepted, so ft-floatd places spares apart from the screens and from each other, within Xwayland's 32767-pixel limit.
+- KWin keeps a Wayland popup inside its parent's output (`XdgPopupWindow::updateRelativePlacement` uses the output's placement area), and X11 apps place their menus within the monitor. That's why a floating window's output has a margin around the window: menus and dropdowns open past the window's edges, into the margin.
+- KWin makes a nested output the size it's configured to times its scale, rounded (at 1.5 it lays out 1067 × 667 on a 1600 × 1000 buffer), and gives the buffer a whole buffer scale (1.2 becomes 2). A buffer whose size isn't a multiple of that is a protocol error that disconnects KWin, so ft-floatd sizes spares in multiples of it. After a scale change, ft-floatd asks for the output's size again in the new scale's terms, or the next configure would make it the old size times the scale.
+- **Virtual outputs don't work.** `createVirtualOutput` makes an output window but never adds it to the backend's `m_outputs`, so `findOutput()` returns null when the pointer enters it, and the next line dereferences it (`Q_ASSERT` is compiled out). A click on such a panel would crash KWin. This rules out virtual outputs (`stream_virtual_output`) for floating windows without a patched KWin.
+
+### The pointer
+
+- Pointer positions reach KWin only through motion events: the output's position in the layout plus the position on its window. When ft-screens stops sending motion, KWin's pointer stays put.
+- KWin starts an interactive move on the press itself, before any motion. So ft-screens stops sending motion as soon as a press lands in a floating window's title bar (from the frame and client rectangles ft-floatd sends it), with no round trip, and carries the panel instead. KWin gets the release at the press point, and the window moves by nothing on its output.
+- KWin's nested backend ignores the position in `wl_pointer.enter`, and wlroots drops a motion to the position it entered at, so the first click after crossing onto another panel landed where KWin's pointer had been. ft-screens enters one unit off.
+
+### The KWin script
+
+The KWin side is a script (`float/frametop-float.js`), not a C++ effect, because a script keeps working across KWin updates and an effect would have to match the host's exact KWin build. KWin scripts can call D-Bus but can't serve it, so ft-floatd's commands come back through a long poll: the script calls `NextCommand`, which answers when a command is ready, or empty after 20 seconds, under KWin's 25-second D-Bus timeout. A few things about KWin's script engine:
+
+- `windowAdded` reports popups as windows of their own (`popupWindow` true, `transient` true) with their geometry.
+- Setting `frameGeometry` applies asynchronously: the app has to answer the new size first.
+- A script can't read a window's maximize mode, so the script counts a window as maximized when it fills its output's maximize area.
+- `globalThis` isn't defined. `print` goes to the journal unless `QT_FORCE_STDERR_LOGGING=1`.
+
+The title bar's float button is Frametop's own window decoration (`decoration/`), written in QML for KWin's Aurorae engine, which loads it without compiling. A C++ fork of Breeze would have to match SteamOS's exact KDecoration build. A decoration can only make the window requests KWin offers it, so the button toggles keep-below, which has no visible effect on a window alone on its own output, and the script treats keep-below as the floating flag.
+
+### KWin's placement memory
+
+KWin keeps each window's geometry, full screen, and maximized state for each layout of the outputs (its `PlacementTracker`, keyed by every enabled output's name and geometry). When the outputs come back to a layout it has seen, it puts every window back as it was in it. That's for plugging monitors in and out, and it does harm here. A spare output changes size after its window does, so what KWin keeps for a spare's size is the window's next size. Resizing a floating window back to an earlier size made the window and its output flip between two sizes for good (Dolphin went between 1187 and 1424 logical pixels wide, its output between 1687 and 1925). Full screen flipped the same way, and floating or docking one window could move others, even onto a spare or off one.
+
+So the KWin script keeps where each window belongs: where ft-floatd put it, or where it went outside an output change. While KWin changes the outputs, it reports nothing to ft-floatd. Once KWin is done (`screensChanged` comes after its restore), it puts floating windows back, and the screens' windows too when only spares changed. KWin's resize request hasn't reached the app by then, so the app never sees it. A size asked for is held for a second, since an app can still answer an older request, and then the script takes the size the window has. If `screensChanged` doesn't come within 2 seconds, the script stops waiting for it.
+
+KWin also ends an interactive move or resize whenever the outputs change. So during a resize by a floating window's edge, ft-floatd only crops the panel to the window, and resizes the output when the drag ends. The margin is the room to grow until then.
+
 ## The 3D mouse
 
 The mouse works like the pointer on the Apple Vision Pro: a small cursor floats in the room, lands on whatever panel it meets, and acts on it like a controller's laser.
@@ -76,9 +118,9 @@ The driver starts disconnected, because holding the right-hand role while SteamV
 
 ### The cursor
 
-Mouse motion turns into yaw and pitch around an anchor, the head position at the last recenter. A ray from the anchor is tested against every visible overlay with `ComputeOverlayIntersection`. On a hit, the cursor sits on that surface; otherwise it floats at `POINTER_DISTANCE`. Since the anchor isn't your current eye position, a second test runs along your line of sight to the cursor point, and anything nearer wins, so the cursor always lands on what you see under it. Overlays in `POINTER_IGNORE` are left out of both tests. A display-only panel, like a performance overlay locked to your view, has no input method, so SteamVR's laser passes through it, but `ComputeOverlayIntersection` still hits it, and the cursor stuck to it. The laser starts just before the cursor point, so an ignored panel nearer to you doesn't catch it either.
+Mouse motion turns into yaw and pitch around an anchor, the head position at the last recenter. A ray from the anchor is tested against every visible overlay with `ComputeOverlayIntersection`. On a hit, the cursor sits on that surface; otherwise it floats at `POINTER_DISTANCE`. Since the anchor isn't your current eye position, a second test runs along your line of sight to the cursor point, and anything nearer wins, so the cursor always lands on what you see under it. Overlays in `POINTER_IGNORE` are left out of both tests. A display-only panel, like a performance overlay locked to your view, has no input method, so SteamVR's laser passes through it, but `ComputeOverlayIntersection` still hits it, and the cursor stuck to it. The laser starts just before the cursor point, so an ignored panel nearer to you doesn't catch it either. Both tests ask SteamVR about every visible overlay, so a frame where nothing moved (the mouse, the anchor, the eye by more than 5 mm, which overlays show) reuses the last result, for up to 100 ms, since overlays can also move on their own. The dots' overlay settings go to SteamVR only when they change, and the pose goes to the driver, which keeps the last one, only when the laser would land 0.1 mm or more elsewhere, and at least every 100 ms.
 
-OpenVR has no call to list other programs' overlays, so the helper runs `vrcmd --overlays` in the background. It includes hidden overlays, because a floating window's controls only appear while something hovers the window, and the cursor has to find them immediately.
+OpenVR has no call to list other programs' overlays, so the helper runs `vrcmd --overlays` in the background. It includes hidden overlays, because a floating window's controls only appear while something hovers the window, and the cursor has to find them immediately. Each run is a shell and a new SteamVR client, about 30 ms of CPU, and it ran every second while the pointer was awake, which in gaze mode is all the time. Now it runs every 20 seconds, and at once when the pointer wakes, when the dashboard opens or closes, when a game starts or ends, and when a left click hits nothing (a panel that came up since). An overlay already on the list showing or hiding needs no new list: the helper checks the visibility of the ones it knows every 50 ms.
 
 The laser starts partway along your line of sight to the cursor rather than at your eye. SteamVR sizes its hit dot by distance from the laser's origin, and a laser from the eye still shows a beam in each eye. Starting it close to the target makes the beam and the dot tiny, while `POINTER_ORIGIN_MARGIN` keeps the origin in front of the small window controls, which float a few centimetres in front of their panels. The helper's own white dot is the visible cursor. In empty space it's an interactive overlay that the laser lands on, so SteamVR never draws a laser into nothing.
 
@@ -91,7 +133,7 @@ A few overlays need special handling:
 
 Head follow is experimental and off by default. It works, but it's only lightly tested, and the feel is mostly a matter of its settings; polishing it is left open. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction, where you were facing when your head last settled, and keeps its offset from it. The mouse can put the cursor anywhere up to `POINTER_FOLLOW_REACH` (70 degrees) from the reference, a corner of your view included. While your head stays within `POINTER_LEASH_DEG` of the reference, nothing moves on its own. Once your head has been past the leash for `POINTER_LEASH_DELAY` (0.2 s, so a glance out and back doesn't count), the reference eases to where you're facing (time constant `POINTER_LEASH_RETURN`, 0.2 s), never falling further behind than the leash, and the cursor ends up back where it was in your view. Then it waits for the leash again. Two earlier versions didn't work out. Moving the reference only while your head pulled at the end of the leash left it up to the leash off after you turned back, and getting it centred again meant overshooting with your head. Easing it toward your facing all the time moved the cursor on every small head movement. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping.
 
-Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse or controller button mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. Moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: it needs accessibility (AT-SPI) on in the Frametop session, where it's off (no registry runs), plus app restarts, and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze; the mouse is gaze mode's only pointer device for now. The dot shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. The gaze moving the pointer doesn't show it: you know where you're looking. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge of up to `POINTER_GAZE_NUDGE_MAX` (8 degrees) before a click is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. See `gaze/README.md` for the service, the calibration, and what was measured.
+Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. By default (`POINTER_GAZE_MOUSE_MOVE=held`, the Gaze page's Mouse movement switch) moving the mouse does nothing while the gaze has the pointer: it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so the lessons aren't polluted by mouse moves to somewhere else (they used to be kept out by an 8 degree limit, which also dropped real corrections when the tracker was further off). With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual; with `free`, moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: the session now starts an AT-SPI registry, but apps still need to expose useful accessibility trees (and may need restarting), and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. The right button works the same way, with the right click on the release, and pressing it while the left press is held back starts a drag where the pointer is, like Meta+J then Meta+K. That drag lasts while either button (or key) is held, so a second right press, or a second Meta+K, is free to pan and tilt the panel being dragged; with the keyboard, the head turns it. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge before a click whose correction is within `POINTER_GAZE_NUDGE_MAX` (55 degrees, half of what the headset shows across) is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on, when our tracker thinks it moved, and when a correction is past that limit (the tracker is far off, so a click there isn't trusted as a lesson), and the full calibration and the headset fit check run in the same panel, so everything a user does to calibrate happens in one place in the headset; the gaze probe, a fullscreen GTK app, is the development tool. The limit was 8 degrees, which dropped every correction while our tracker was 12 off. Its dots sit at known directions from the headset, so the panel needs no screen geometry. The quick check's dot takes the gaze once it has held still, so what the tracker says doesn't have to be close for the capture to work. The full calibration's and the five-dot check's dots wait for a click while you look at the dot (a left click or Meta+J), because a steady gaze isn't always on the dot, and take the gaze held still up to the click; a rightLine truncated
 
 Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place.
 
@@ -101,7 +143,7 @@ Replacing a loaded driver's files, as re-running the installer used to do, leave
 
 The dashboard follows whichever device summoned it or last pressed its trigger. Frametop adds "last used wins": moving a real controller releases the pointer, and the next mouse movement takes the laser back. Moving means faster than 0.35 m/s or 2 rad/s (both times `POINTER_CONTROLLER_PICKUP`, 1 by default) for 100 ms in a row, while the controller is tracked normally. A single sample over the limit used to be enough, and controllers resting on a desk took the laser back on a knock or a tracking jump while the mouse was in use. Small movements don't count; waking needs `POINTER_WAKE_COUNTS` of mouse motion within a second, so desk jitter doesn't steal the laser. While the pointer is awake, a tiny transparent overlay with `MakeOverlaysInteractiveIfVisible` keeps SteamVR's laser mouse on, since otherwise the first click would only switch the laser on.
 
-When the headset comes off, SteamVR reports its activity level as idle at once and turns the displays off 5 seconds later (`power.turnOffScreensTimeout`), unless something keeps it awake. An awake pointer did, and so did the helper's `vrcmd` runs: each is a new SteamVR client, and a new client every second kept SteamVR out of standby. The helper now releases the pointer as soon as the headset is idle, ignores the mouse until you're wearing it again, and pauses the overlay list whenever the pointer is off.
+When the headset comes off, SteamVR reports its activity level as idle at once and turns the displays off 5 seconds later (`power.turnOffScreensTimeout`), unless something keeps it awake. An awake pointer did, and so did the helper's `vrcmd` runs: each is a new SteamVR client, and a new client every second kept SteamVR out of standby. The helper now releases the pointer as soon as the headset is idle, ignores the mouse until you're wearing it again, and pauses the overlay list whenever the pointer is off. With the pointer off it also stops running its loop every 8 ms, about 116 wakeups a second for nothing: it waits up to 250 ms for a command on its socket (20 ms while it reads mapped controller buttons, which SteamVR input only offers by polling), and leaves the overlay lookups until the pointer wakes.
 
 ### Moving floating windows
 
@@ -113,6 +155,8 @@ SteamVR opens every input device only when it starts. When a Bluetooth mouse sle
 
 Keyboards aren't grabbed by default, because a grabbed keyboard's keys went into a virtual keyboard nothing typed from; the relay forwards them to ft-screens instead.
 
+The relay never waits on the pointer helper. Its socket to the helper used to block, so when the helper stalled (a layout placement or `grabprobe` holds it for seconds, and the gaze service fills its socket 90 times a second meanwhile), the whole relay stopped with it: keyboards, the volume keys, and pausing. Now what the helper doesn't take waits in order and goes out on the next loops. Mouse moves add up into one while they wait, and a scroll notch is dropped, since scrolling seconds late is no use; presses and releases are kept, so no button stays down. Mouse motion goes to the helper at most every 4 ms, rather than once per report, which from a 1000 Hz mouse was 1000 datagrams a second to a helper that runs every 8 ms; a button sends the motion before it first, so the click lands where the pointer was.
+
 An ungrabbed keyboard reaches both sides at once. In VR, gamescope reads every input device itself (the SteamOS build's `InputStealer`, libinput with udev hotplug, so new devices too) and types into its focused app, and ft-screens types the same keys into the desktop. So Space in the desktop also paused Spotify on the dashboard. Typing now follows the last click. ft-screens sees clicks on its own screens, from the mouse or a controller. A click anywhere else is only visible for the mouse: overlay apps get SteamVR's `OverlayFocusChanged` (which panel the laser is on) but no controller button events, so the pointer helper reports the panel under the dot on each left press. ft-screens tells the relay where typing goes every second, from an unbound socket so the relay's replies can't loop back into its control socket, and the relay grabs pass-through keyboards while it's the desktop. A grab waits until the keyboard has no key down, so no key stays held on either side, and the relay lets go if ft-screens stops reporting. A program that reads every keyboard for a hotkey (a dictation tool, say) loses a grabbed keyboard. Repeating the keys on another input device doesn't work: gamescope reads that device too, whether it's the relay's virtual keyboard or one created later, and every Space, typed or dictated, paused Spotify again. So with `SHARE_KEYS=1` the relay sends a grabbed keyboard's keys to `@frametop_keys` as datagrams (`key <code> <value> <device name>`). It's off by default, because the relay can't tell who is listening: abstract sockets have no permissions, and any local process that binds the name first gets every key typed into the desktop, passwords included. A listener should accept only its own user (`SO_PASSCRED`) and skip any keyboard of its own that the relay grabs too.
 
 Volume keys must never reach gamescope. With the openvr backend, gamescope sends volume up and down to Steam by moving keyboard focus to Steam for the key and then back to the previously focused surface. When nothing had focus, the one it moves back to is null, and wlroots aborts on a null focus surface (`wlr_seat_keyboard_notify_enter: Assertion 'surface' failed`), which ends the whole VR session. Keyboard focus is often empty while you work in VR, so one press of the headset's volume button could take everything down. gamescope reads the headset's buttons and every keyboard itself (`InputStealer`), as do SteamVR's processes, so the relay has to stop volume keys at the device. Grabbing `gpio-keys` would also take the headset's click button, so the relay remaps the volume entries in each device's keymap (`EVIOCSKEYCODE`) and handles the stand-in codes itself. That fix covers every device at once, including keyboards that aren't grabbed.
@@ -125,17 +169,37 @@ The Frame controllers can be mapped like mouse buttons, but they aren't input de
 
 ## The desktop session
 
-The session is modeled on SteamOS's `steamos-nested-desktop` and runs beside it. It has its own runtime directory, config (`~/.config/frametop`), and state, so it never disturbs the stock desktop's layout or panels. It runs on a private D-Bus from `dbus-run-session`, which has two consequences. KDE only launches apps in systemd scopes when systemd is on the session bus, so everything started in the desktop lands in its systemd unit, and stopping the unit would kill all of it; `session/keep-apps.sh` moves those programs out first. And tools that need the real user bus, like podman and `distrobox-host-exec`, have to be pointed at it explicitly.
+The session is modeled on SteamOS's `steamos-nested-desktop` and runs beside it. It has its own runtime directory, config (`~/.config/frametop`), and state, so it never disturbs the stock desktop's layout or panels. It runs on a private D-Bus from `dbus-run-session`, which has two consequences. KDE only launches apps in systemd scopes when systemd is on the session bus, so everything started in the desktop lands in its systemd unit, and stopping the unit would kill all of it; `session/keep-apps.sh` moves those programs out first. And tools that need the real user bus, like podman and `distrobox-host-exec`, have to be pointed at it explicitly. Its own config folder also hides SteamVR's path registry (`~/.config/openvr/openvrpaths.vrpath`) from everything started in it: OpenVR programs there fail with `VRInitError_Init_PathRegistryNotFound`, and `vrpathreg adddriver` writes a new registry under `~/.config/frametop/openvr` that has no SteamVR in it and that SteamVR never reads. So Frametop's scripts run SteamVR's tools with `XDG_CONFIG_HOME=~/.config`.
 
 The VR launcher starts the session from the Steam client, and the client's environment came along: `LD_LIBRARY_PATH` pointing at Steam's own runtime, whose `libavcodec` has no H.264 decoder, so VLC in the desktop couldn't play most videos, plus the client's overlay and launch settings. The session script drops the client's variables before it starts anything. SteamOS's global Mesa settings (`/usr/share/deckard/mesavars.sh`) stay, and the gamescope session's Vulkan layer (`ENABLE_GAMESCOPE_WSI`) is only kept for the gamescope backend.
 
+### Nested accessibility
+
+The session drops an inherited `AT_SPI_BUS_ADDRESS`, so apps cannot accidentally use the host desktop's registry. It autostarts `session/ft-atspi` in Plasma phase 2, after KWin has set the nested display environment. The helper gets the live accessibility address from `org.a11y.Bus` on the private session bus, preserves any existing registry owner, updates the accessibility bus's activation environment, and tries `StartServiceByName` first.
+
+On SteamOS 0.3.0 with at-spi2-core 2.52.0, the native launcher can choose dbus-broker because its process belongs to a systemd user unit. Registry activation then fails: this desktop's private session bus does not have a systemd activation manager. In that case the helper starts only `at-spi2-registryd` on the already-existing accessibility bus. The registry refuses duplicate ownership. Unlike native activation's `--use-gnome-session`, the fallback does not try to register with GNOME's session manager; that flag did not explain the observed native activation failure.
+
+The fallback registry does not exit merely when its bus disconnects in the isolated SteamOS test. Its small watcher checks both private buses every 5 seconds, and terminates and reaps only the child it started when either bus disappears or the watcher is stopped. Each check runs `gdbus` twice; once a second, that cost about 1% of a core. `keep-apps.sh` keeps the watcher in the desktop unit when `desktops.sh start` runs the desktop as `frametop-desktop`. Started from the VR launcher, the desktop runs in steam.service, which doesn't stop with it, so there the watcher is the only thing that stops the registry. There is no second accessibility bus, global systemd environment update, process-name kill, or host registry replacement. Missing accessibility files or bus errors are nonfatal; the desktop still starts. Toolkit-specific accessibility opt-ins and pointer snapping are separate work.
+
+Run the isolated checks on the host with `/usr/bin/python3 session/test/test_accessibility.py`. They use private D-Bus buses, Xvfb and a GTK3 app, never the production display or input. Native activation uses a small `org.a11y.Bus` test provider pointing to a real private dbus-daemon with the installed registry service; the SteamOS fallback uses the installed bus launcher and broker. The tests check real app-tree discovery, existing owners, concurrent starts, session stop/restart, and teardown. They require test-only PyGObject (Gio and GTK3), Xvfb, and at-spi2-core; the runtime helper uses Python's standard library and the existing host `gdbus`. Actual Plasma autostart and VR desktop restart still require an approved hardware test.
+
+### Other session behavior
+
 Steam, not systemd, suspends the Frame: after `system_idle_suspend_ac_sec` (an hour by default) without input on AC power, it logs `Switching to power state: k_ESystemPowerState_Sleep` and suspends, even while charging. It's a Steam setting (Settings → Power → When Plugged In and Idle → Sleep after), which the Stay awake while plugged in switch in Frametop Display Settings sets to Never. SteamVR's standby, which turns the displays off when the headset comes off, is separate; see below.
 
 Flatpak apps need `XDG_DATA_DIRS` to include Flatpak's exports, or Plasma opens Discover instead of launching them, so the session sources `/etc/profile.d/flatpak.sh`.
 
 The private runtime directory also moves the session's document portal to `$XDG_RUNTIME_DIR/frametop/doc`, and that broke saving and uploading in Flatpak apps. The file picker (xdg-desktop-portal 1.18.4 on SteamOS) gives a sandboxed app the host path of the file it picked, `/run/user/1000/frametop/doc/ID/NAME`. Inside the sandbox the portal is at `/run/flatpak/doc`, and `/run/user/1000` is a private per-app folder (`.flatpak/APP/xdg-run` in the runtime directory). So Brave created the missing folder there, "finished" the download into it, and the file vanished when the session cleaned up. The session script now links that path to `/run/flatpak/doc` in each installed app's folder before Plasma starts. Upstream xdg-desktop-portal fixed this after 1.22.1 (commit `69ba5e1`) by handing Flatpak apps `/run/flatpak/doc` paths, after which the links go unused.
 
-A podman container's monitor process (conmon) stays in the cgroup of whatever started the container, and `distrobox enter` starts it on demand. When a Frametop service happened to start the `dev` container, stopping that service stopped the container and everything in it, including the desktop's compositor. `scripts/container-up.sh` starts the container in a systemd scope of its own before anything enters it.
+A podman container's monitor process (conmon) stays in the cgroup of whatever started the container, and `distrobox enter` starts it on demand. When a Frametop service happened to start the `dev` container, stopping that service stopped the container and everything in it, including the desktop's compositor. `scripts/container-up.sh` starts the container in a systemd scope of its own before anything enters it. It then waits for distrobox-init to log `container_setup_done`, as `distrobox enter` does only for containers it starts itself. A new container's first start takes a minute or more (it installs distrobox's dependencies and sets up passwordless sudo), and an install that entered right away met a sudo password prompt with no terminal to answer it ([#9](https://github.com/DeeJanuz/frametop/issues/9)).
+
+KWin renders with OpenGL through zink on Turnip, Vulkan on the same GPU vrcompositor needs to hit its frame time, and on the Frame that costs CPU too. The nested session started with KWin's defaults: blur and background contrast on (no `[Plugins]` group in its kwinrc) and animations at full length. Blur re-renders what's behind every translucent panel and menu each time it changes, and every animated frame is one more frame for KWin and ft-screens to draw and send. They're off by default in the Frametop desktop. The session script writes them before KWin starts, only where the desktop's own config has no value, once: System Settings deletes a setting put back to KDE's default rather than writing it, so without the marker in `frametoprc` a user who turned blur back on would lose it at the next start. The effect ids (`blur`, `contrast`) are the ones built into KWin 6.2.5 on SteamOS; KWin reads `<id>Enabled` from `[Plugins]`.
+
+The nested session also runs the system's XDG autostart entries, being a KDE session. Discover's update notifier started `plasma-discover --mode update` in it (520 to 620 MB resident and about 9% of a core, plus `flatpak-system-helper` and AppStream downloads), and IBus started a daemon, the kimpanel panel and its GTK extension that nothing can use: KWin hands text input to the one input method it starts (`ft-textinput`), and the session drops `QT_IM_MODULE`, `GTK_IM_MODULE` and `XMODIFIERS`. The session hides both for this desktop only, with `Hidden=true` copies in its own autostart folder. The geoclue demo agent stays: it's what answers apps' location requests to Geoclue outside GNOME, and it costs nothing while idle. Orca's entry only starts in GNOME-family desktops.
+
+Plasma 6.2.5 keeps each panel on a screen number (`lastScreen` in `plasma-org.kde.plasma.desktop-appletsrc`), and the numbers rank the enabled outputs by priority, so 0 is the primary screen. A panel whose number is past the screen count gets no view, and Plasma never moves it: the remap it runs at every start only moves a panel whose number has no desktop, and this desktop keeps a desktop for every output it has seen, spares included. So the taskbar was lost when the number of screens went down, and once it was found saved on a spare output, number 8 of a desktop with three screens ([#18](https://github.com/DeeJanuz/frametop/issues/18)). Before Plasma starts, the session runs `session/fix-panels.py`, which moves any panel numbered past the screen count, with its system tray's containment, to screen 0, keeping its widgets and settings. A panel stays put when screen 0 already has one on that edge, and comes back by itself if the screens do. The file is backed up to `<file>.ft-bak` first. Plasma's scripting can't do this while it runs (`panel.screen` is read-only in 6.2.5), so a lost taskbar comes back at the desktop's next start. `scripts/doctor.sh` and `scripts/report.sh` list the panels and their screens.
+
+Remote desktop is a chain (krdp, then FreeRDP inside Xvnc) because nothing on SteamOS serves KWin over VNC directly. Kept connected all the time, it cost about a core with nobody watching: krdpserver 55 to 78% (it encodes H.264 in software with openh264: VA-API finds no driver for the Frame's GPU in the container), FreeRDP 16 to 27%, Xvnc 6 to 11%, and the bridge's layout check every 5 seconds another 4%. krdp creates its screencast session per RDP connection and drops it when the connection closes (`SessionController::onNewConnection` in krdp 6.7), so an idle krdpserver costs nothing and can stay up; only the RDP connection has to go. The bridge connects FreeRDP when a VNC client appears and disconnects 45 seconds after the last one leaves. Xvnc has no hook for its clients, so the bridge counts established connections to its port with `ss`, woken early by Xvnc's log output; looking with `ss` once a second cost about 0.9% of a core in bash, against about 0.1% this way. `Xvnc -inetd` from a systemd socket would start a server per connection and lose sharing between viewers. The layout check (`ft-layout remote-view`, which scans `/proc` for plasmashell and runs `kscreen-doctor -j`) now runs only while FreeRDP runs, and then only after `kwinoutputconfig.json` or `frametop-layout.json` changes, with one check a minute in case a change touched neither.
 
 Program names stay within 15 characters, because Linux truncates process names there and the scripts find programs with `pgrep -x` and `pkill -x`. That's why the prefix is `ft-`.
 
@@ -149,6 +213,21 @@ Movement is judged within 10-second windows. On the mount, the head pose jittere
 
 Staying awake while charging uses Steam's own setting rather than a logind sleep inhibitor. Steam suspends with `dbus-send ... login1.Manager.Suspend boolean:true`, and a block inhibitor does stop that (`CanSuspend` answers "challenge" while one is held), but it stops the power button too. `system_idle_suspend_ac_sec` is field 24004 of Steam's CMsgClientSettings. In Steam's SharedJSContext, reachable over CDP on port 8080 because Steam runs with `-cef-enable-debugging`, `SteamClient.Settings.SetSetting` takes a change as a base64 protobuf, the way Steam's Power page sends it (0 is never), and `settingsStore.clientSettings` has the current values.
 
+## Pausing for VR games
+
+Hiding the screens during a game kept them out of view, but Frametop kept using the headset. Measured on 2026-10-02 with gaze mode off and no game running, in shares of one core: our eye tracker (ft-eyes) about 60%, ft-eyegrab, ft-gaze and ft-gazed about 3 to 4% each; remote desktop (krdpserver, FreeRDP, Xvnc) about 2 cores while it ran; KWin about 13%, ft-screens about 4%. The gaze service ran at full rate whether gaze mode was on or not; now it idles while the gaze isn't used (gaze/README.md), and pausing stops it outright. Reading SteamVR's eye tracking 90 times a second also made it restart every 10 to 13 seconds during Beat Saber, and each restart took input focus from the game, which paused it (PR #13; since then ft-gaze skips SteamVR's gaze action during games, but our own tracker kept running). So pausing stops what costs the most and leaves windows where they are.
+
+- A hidden screen still cost as much as a visible one. ft-screens sent every committed screen its frame callback at 90 Hz whether its overlay showed or not (since then, a hidden screen always gets one a second; see the frame rates in reference.md), so KWin kept drawing, and its apps with it. Paused, ft-screens sends the callbacks once a second. A Wayland client draws again only after its last frame's callback, so KWin's output stalls, KWin's own clients stop getting theirs, and the whole desktop idles, without anything losing its connection. A second's pace, rather than none, keeps any client that waits on a callback from waiting forever. Stopping KWin or the apps with SIGSTOP would free the same, but a Wayland peer that stops reading overflows the other side's 4 KB socket buffer, which ends the connection: that's how the live desktop died once when its KWin stalled (`Data too big for buffer`). They also sit in different cgroups (KWin under steam.service when the VR launcher starts it, ft-screens in the dev container's), so no single freeze stops them together.
+- The relay does the pausing because it's the one part that always runs, and the pointer helper keeps running because stopping it leaves its virtual controller connected with its last pose (the driver has no staleness timeout), maybe holding a hand role, with the 3D mouse dead. Releasing it does the job. The helper already checks for a scene app twice a second, so it's what tells the relay a game started.
+- The gesture has to work during a game, but SteamVR input reaches only the app with input focus, and an overlay with global input (`steamvr/globalActionSetPriority`) takes the buttons it binds from the game. vrserver's web socket on 127.0.0.1:27062, which its controller binding page uses for the live view, reports every controller component whatever has focus, and reading it takes nothing. The game sees the clicks too, so the default is a gesture games hardly use: both thumbsticks, together, twice. "Together" means within 0.3 seconds of each other, so a stick held down to sprint while the other clicks doesn't count. The stream is about 160 messages a second, nearly all capacitive sensing, so the reader parses only the few that mention a gesture's button. A controller's root path changes while the 3D mouse holds its hand role (`/devices/cv/<serial>` instead of `/user/hand/right`), so the reader looks the controllers up again (an HTTP request to vrserver): when the relay's 3D mouse connects or lets go, when a message comes from a device it doesn't know, and every 30 seconds. It used to be every 3 seconds.
+- Resuming starts remote desktop through `systemd-run --scope`: started straight from the relay, it would join the relay's cgroup and end with the next relay restart.
+
+## SteamOS updates
+
+On the Frame, SteamVR is part of the OS image (`/opt/steamvr`, the `deckard-steamvr-rel` package), next to KWin, gamescope, and the kernel, so every SteamOS update can bring a new SteamVR too. Frametop survives updates: it lives in the home folder and the `dev` container, the Bluetooth fixes are in `/etc`, which SteamOS keeps across updates, and nothing goes into `/usr`. What an update can break is what Frametop uses from the image. The public OpenVR API is versioned and stays put. The rest is less certain: `IVRIPCResourceManagerClient`, which is newer than the header SteamVR ships; the text `vrcmd --overlays` prints; the eye tracker's shared memory layout; XRService's camera buffers; KWin's nested backend; and behavior Frametop works around, such as the SteamVR Settings page that `ComputeOverlayIntersection` can't find or the scale KWin's nested backend doesn't undo.
+
+`scripts/update-check.py`, which `scripts/doctor.sh` runs, checks what it can directly: that SteamVR still serves every OpenVR interface version the installed programs were built against (read from the binaries), that `vrcmd`'s format still parses, that the eye tracker's sample timestamp is still at the offset ft-gaze reads, and the host files, services, sockets, and driver registration. Behavior can't be checked without someone in the headset, so it records the versions of the packages that matter once things work (`--mark-good`), and after an update names what changed and what to try by hand.
+
 ## Approaches we dropped
 
 - WayVR, an existing Wayland desktop for VR. It built and connected to SteamVR on the Frame, but nothing showed in the headset. It has no bindings for the Frame's controllers, and its KDE screen capture needs `xdg-desktop-portal-kde`, which SteamOS doesn't ship.
@@ -157,9 +236,7 @@ Staying awake while charging uses Steam's own setting rather than a logind sleep
 
 ## Open questions
 
-- A head-locked screen, like a HUD.
 - A controller button that shows the screens during a game. Games own the controllers, so this needs SteamVR input actions for ft-screens.
 - Drawing KWin's cursor on the screens.
-- Plasma can lose its panels when the number of screens goes down, because they're saved against a screen that no longer exists. Removing `plasma-org.kde.plasma.desktop-appletsrc` and `plasmashellrc` from `~/.config/frametop` brings the default panels back.
 - Frame pacing and GPU cost with several busy screens haven't been measured.
 - Real standby on a stand, with rendering and tracking paused, not just the backlight off. SteamVR has no call for it, and its activity level follows the proximity sensor.

docs/floating-windows.md

@@ -1,76 +1,79 @@
-# Floating windows (plan)
+# Floating windows
 
-Status: design settled 2026-09-29 (see "Decisions"); being built on the `floating-windows` branch.
+Any desktop app can float in VR in a panel of its own, like SteamVR's floating windows, while it stays part of the Frametop desktop. Drag and drop, the clipboard, and focus keep working between floating windows and the screens.
 
-Built so far (2026-09-29; the KWin side tested on the headless test desktop, `screens/test/headless.sh`; nothing yet tried in the headset):
-
-- The catcher (a release off every panel still reaches KWin), and the pointer helper's "up" backstop.
-- `float/frametop-float.js` (the KWin script), `float/ft-floatd`, and `float/ft-float`. "Float in VR" is in the window menu under Extensions, and Meta+Shift+F toggles the active window. Floating, docking (back where it came from), closing, full screen, per-window scale (Meta+scroll), popups reported with their rectangles, windows of a floating app floating too, and the notification when every spare is in use.
-- ft-screens: a panel per spare output (`frametop.float.N`) with the crop, density, popups and dialogs as small panels over it, title-bar carrying, the corner tab resizing the window, and dock and close buttons. `--spares`, and the commands `float`, `unfloat`, `pose`, `sub`, `minimized`, `carry`.
-- The session adds `FLOAT_SLOTS` spares and starts ft-floatd from the desktop's autostart; ft-layout leaves the spares alone.
-- The 3D mouse's drag lock crosses onto other Frametop panels (not while carrying one).
-
-Not built yet: phase 2 (the ghost, tear-off by dragging, push-flush docking), phase 3 (launching floating, the Frametop Apps entry and picker, remembered placement), and phase 4. Frametop Apps (decision 10) needs a per-screen hide, which ft-screens doesn't have yet: its hide and show are for all screens.
-
-The goal is to let any desktop app float in VR in a panel of its own, like SteamVR's floating windows, while it stays part of the Frametop desktop. That means drag and drop, the clipboard, and focus keep working between floating windows and the screens.
-
-- There are two ways to get a floating window. Launch the app floating, or drag a desktop window by its title bar off a screen and let go in the air.
-- There are two ways to put one back. Push it flush against a screen and let go, or press its "back to desktop" button.
+- **Float a window** with "Float in VR" in its window menu (Alt+F3), the float button left of Close in its title bar, or the float key (Meta+Shift+F by default). Start an app floating with "Launch as Standalone" in its right-click menu in the Application Launcher or the taskbar, with `ft-float launch` or `ft-float run`, or from a profile ([profiles.md](profiles.md)).
+- **Put it back** with the dock button under its panel, the title bar button, the float key, or "Back to Desktop" in the window menu. It returns to the screen, position, and size it came from. The `dock_all` action docks every floating window.
+- **Move it** by its title bar or the bar under its panel, **resize it** by its edges or the corner tab, and **change its scale** with Meta+scroll over it. Each app's last floating place, size, and scale are remembered.
 - Files, text, and images drag between any two floating windows, and between floating windows and the screens.
 
+How KWin behaves underneath all this, and what the KWin script does about it, is in [design.md](design.md#floating-windows).
+
+## Not built
+
+These were decided (see the table) but aren't built:
+
+- Tearing a window off a screen by dragging its title bar into the air, with a ghost of it on the laser (decision 3; see "Tearing a window off a screen").
+- Docking by pushing a floating window flush against a screen, with the landing spot highlighted (decisions 4 and 17).
+- New windows of a floating app placed where that app's windows went last time, or to the parent's right (decision 15).
+- +/- scale buttons on the floating panel's bar (decision 18). Meta+scroll changes the scale.
+- The glow at the edge of your view toward a floating window activated out of sight, and the setting that brings it in front of you instead (decision 19).
+- A Floating windows section in Frametop Display Settings (decision 1). `FLOAT_SLOTS` and `FLOAT_MARGIN` are set in `~/.config/frametop.conf`.
+- A drag proxy on the catcher, so a drag's icon shows while the laser is between panels.
+- Keeping floating windows out of Show Desktop (Meta+D) (decision 9). Nothing handles it yet.
+
 ## Decisions
 
-Settled with the user on 2026-09-29. The sections below follow them.
+The numbers are cited in the code, so they stay as they are. Struck-out text was replaced by a later decision.
 
 | # | Question | Decision |
 |---|---|---|
-| 1 | How many windows can float at once | 8 spare outputs by default, configurable (`FLOAT_SLOTS`, and Display Settings); a change needs a desktop restart |
+| 1 | How many windows can float at once | 8 spare outputs by default, configurable with `FLOAT_SLOTS` (at most 16); a change needs a desktop restart. A Display Settings control isn't built |
 | 2 | Menus and dropdowns | Each floating output has a margin around the window. The panel shows only the window, and each open popup gets a small overlay of its own, cut from the same buffer |
-| 3 | Tearing off | Drag the title bar past a screen's edge and let go in the air, with a small dead zone past the edge |
-| 4 | Docking by dragging | Push the window flush against a screen (within about 10 cm), with the landing spot highlighted, and let go |
+| 3 | Tearing off | Not built. Drag the title bar past a screen's edge and let go in the air, with a small dead zone past the edge |
+| 4 | Docking by dragging | Not built. Push the window flush against a screen (within about 10 cm), with the landing spot highlighted, and let go |
 | 5 | Visibility | Floating windows follow the same rules as the screens: the hide hotkey, the visibility modes, and the games rule |
 | 6 | Windows a floating app opens | They float too |
-| 7 | Launching floating from the headset | One "Frametop Apps" launcher entry with a picker |
-| 8 | Build order | The catcher first, as a fix that stands on its own; `pointer-ignore` and `layouts-headpin` merged before phase 1; the hand cutouts stay out |
-| 9 | Show Desktop (Meta+D) | Floating windows stay |
-| 10 | Frametop Apps and visibility | The entry starts the desktop with each screen hidden on its own (the existing per-screen hide), so only floating windows show. No special mode |
+| 7 | Launching floating from the headset | ~~One "Frametop Apps" launcher entry with a picker~~ Replaced by 26 and profiles (27) |
+| 8 | Build order | Not kept here: it only set the order of the work |
+| 9 | Show Desktop (Meta+D) | Floating windows stay. Not built |
+| 10 | Frametop Apps and visibility | ~~The entry starts the desktop with each screen hidden on its own, so only floating windows show~~ Replaced: a profile can hide screens (27) |
 | 11 | Window frame | KWin's title bar and border stay. Frametop's bar, close, and "back to desktop" are extras |
 | 12 | Resizing | The window's own edges and Frametop's corner tab both change the size in pixels at the same density; the output follows |
-| 13 | Margin | 300 px on each side, configurable |
-| 14 | All spares in use | The window opens on the screens, with a notification |
-| 15 | Where a floating app's new windows go | Where that app's windows went last time; otherwise to the parent's right, curving around you |
-| 16 | Where the code is written | A branch in the PC's clone of the repo |
-| 17 | Size when docked by dragging | The current floating size in pixels, shrunk to fit the screen |
-| 18 | Bigger text | A scale for each window (KWin's output scale): Meta+scroll over the window, or +/- on its bar. Remembered for each app |
-| 19 | Switching to a window you can't see | It's focused, and a glow at the edge of your view points to it. Moving it in front of you is a setting |
+| 13 | Margin | 300 px on each side, configurable (`FLOAT_MARGIN`) |
+| 14 | All spares in use | The window stays on the screens, with a notification |
+| 15 | Where a floating app's new windows go | Not built: where that app's windows went last time, otherwise to the parent's right, curving around you. For now, a new window that opens on a floating window's output floats a little in front of it |
+| 16 | Where the code is written | Not kept here: it was about the work, not about Frametop |
+| 17 | Size when docked by dragging | Not built (see 4): the current floating size in pixels, shrunk to fit the screen |
+| 18 | Bigger text | A scale for each window (KWin's output scale): Meta+scroll over the window. Remembered for each app. +/- buttons on its bar aren't built |
+| 19 | Switching to a window you can't see | It's focused. Not built: a glow at the edge of your view that points to it, and a setting that moves it in front of you |
 | 20 | Full screen | The window fills its own panel. The margin drops to zero while it's full screen, and the panel keeps its size and place |
-| 21 | Named layouts | They cover the screens only. Floating windows use the placement remembered for each app |
+| 21 | Named layouts | ~~They cover the screens only~~ Replaced by profiles (27). Outside a profile, floating windows use the placement remembered for each app |
+| 22 | The float key | One toggle: it floats a window, or docks it if it already floats. The input relay owns it (`float_toggle`), Meta+Shift+F by default, rebindable in Frametop Input Settings and mappable to mouse and controller buttons. KWin has no shortcut of its own for it, so one press can't fire twice |
+| 23 | Which window the key acts on | The window under the desktop's pointer; the active window if there's none there (the wallpaper, the taskbar) |
+| 24 | Docking everything | A `dock_all` action, with no default binding |
+| 25 | A button on every window | A float button left of Close in the title bar, from Frametop's own QML window decoration, made to look like Breeze. It shows a dock icon on floating windows. Apps that draw their own title bar (Chromium, Electron, GTK) use the key |
+| 26 | Launching one app floating | "Launch as Standalone" in the right-click menu of every app in the Application Launcher and the taskbar, from copies of the apps' desktop files that only the Frametop desktop reads. It replaces the Frametop Apps entry (7, 10) |
+| 27 | Profiles | Named layouts become profiles: the screens' places, which screens show, and the apps and their windows, floating or not. See `docs/profiles.md` |
 
-Also assumed: floating windows get the wrist pin, the head pin, and pass-through (`pointer-ignore`) like screens. Every gesture works with the controllers as well as the 3D mouse. A window launched floating uses the primary screen's density. VNC shows only the primary screen, as now. Anything that restarts the live desktop waits for the user's OK.
+Also: floating windows get the wrist pin, the head pin, and pass-through (`pointer-ignore`) like screens. Every gesture works with the controllers as well as the 3D mouse. A window launched floating uses the primary screen's density. VNC shows only the primary screen.
 
 ## The approach: each floating window gets a KWin output of its own
 
 Drag and drop and the clipboard only work between windows of the same compositor. A Wayland window can't move from one compositor to another. So a floating window has to stay a KWin window.
 
-ft-screens already shows each KWin output as a panel. It sets the output's size with an `xdg_toplevel` configure, and KWin resizes the output to match. So a floating window can get an output of its own, sized to fit it, and ft-screens shows that output as a panel with its own controls. To KWin this is an ordinary desktop with more monitors. Dragging between two floating windows is the same as dragging between two monitors, which KWin already handles. ft-screens already moves the pointer between panels in the middle of a drag: `handle_vr_event` moves pointer focus to another KWin window even while a button is held.
+ft-screens already shows each KWin output as a panel. It sets the output's size with an `xdg_toplevel` configure, and KWin resizes the output to match. So a floating window gets an output of its own, sized to fit it, and ft-screens shows that output as a panel with its own controls. To KWin this is an ordinary desktop with more monitors. Dragging between two floating windows is the same as dragging between two monitors, which KWin already handles. ft-screens moves the pointer between panels in the middle of a drag: `handle_vr_event` moves pointer focus to another KWin window even while a button is held.
 
-Alternatives we considered:
+Alternatives considered:
 
-- **Run floating apps directly on ft-screens.** It's a wlroots compositor, so apps could connect to it and get a panel per window. But they would get no drag and drop or clipboard with desktop apps unless we wrote a bridge. Also, a window that's already on the desktop could never be torn off, because a Wayland client can't change compositors. Rejected.
-- **One large hidden "canvas" output.** Every floating window would sit on one big output, and each panel would show a crop of it (`SetOverlayTextureBounds`). That needs only one extra output, with no copies. But an 8K canvas uses about 128 MB per buffer, with two or three buffers in KWin's swapchain. It would also have to repack windows whenever one resized, full screen would fill the whole canvas, and every window would share one scale. This is the fallback if per-window outputs don't work.
+- **Run floating apps directly on ft-screens.** It's a wlroots compositor, so apps could connect to it and get a panel per window. But they would get no drag and drop or clipboard with desktop apps without a bridge. Also, a window that's already on the desktop could never float, because a Wayland client can't change compositors. Rejected.
+- **One large hidden "canvas" output.** Every floating window would sit on one big output, and each panel would show a crop of it (`SetOverlayTextureBounds`). That needs only one extra output, with no copies. But an 8K canvas uses about 128 MB per buffer, with two or three buffers in KWin's swapchain. It would also have to repack windows whenever one resized, full screen would fill the whole canvas, and every window would share one scale. It was the fallback in case per-window outputs didn't work.
 - **Screencast single windows** (`zkde_screencast` `stream_window`, over PipeWire). This adds copies and latency, and the window still needs a real place in KWin's layout to receive input. Rejected.
 - **SteamOS's own floating windows** (Launch a program from the dashboard). Those apps run in gamescope, outside KWin, so they can't drag and drop with the desktop.
 
 ### Where the extra outputs come from: spare outputs
 
-KWin's nested backend opens its outputs at start (`--output-count`). The session starts KWin with the screen count plus `FLOAT_SLOTS` outputs (default 8). Each spare is disabled until it's needed, with `kscreen-doctor` (or in the session's `kwinoutputconfig.json`, so it starts disabled). Floating a window enables a spare, and docking the window disables it again. `FLOAT_SLOTS` limits how many windows can float at once, and changing it means restarting the desktop.
-
-Checked in KWin 6.2.5's source (`src/backends/wayland/`, 2026-09-29):
-
-- Disabling a nested output keeps its host window. `Output::applyChanges` only flips `enabled`, KWin stops rendering it, and Plasma drops its desktop view. So ft-screens keeps the same toplevel, and its screen numbers stay put.
-- Each output's host window is titled `KDE Wayland Compositor WL-<n>`, with `- Output disabled` appended while it's disabled (`WaylandOutput::updateWindowTitle`, on every `enabledChanged`). ft-screens reads the title to tell screens (`WL-0` to `WL-<SCREENS-1>`) from spares, and to see a spare turn on and off.
-- Pointer positions reach KWin only through motion events: the output's position in the layout plus the position on its window. When ft-screens stops sending motion, KWin's pointer stays put.
-- **Virtual outputs don't work.** `createVirtualOutput` makes an output window but never adds it to the backend's `m_outputs`, so `findOutput()` returns null when the pointer enters it, and the next line dereferences it (`Q_ASSERT` is compiled out). A click on such a panel would crash KWin. This rules out the virtual-output fallback (`stream_virtual_output`) without a patched KWin.
+KWin's nested backend opens its outputs at start (`--output-count`). The session starts KWin with the screen count plus `FLOAT_SLOTS` outputs (default 8, at most 16). ft-floatd turns off the spares nothing floats on with `kscreen-doctor` once it starts. Floating a window enables a spare, and docking the window disables it again. `FLOAT_SLOTS` limits how many windows can float at once, and changing it means restarting the desktop. How KWin's nested backend treats disabled outputs, and why its virtual outputs can't be used instead, is in [design.md](design.md#floating-windows).
 
 ft-screens creates a `screen` for each toplevel in the order they appear, and indexes its settings by that order. Spares come after the screens, so they get indices `SCREENS` and up. Their panels are hidden while their output is disabled.
 
@@ -79,35 +82,38 @@ ft-screens creates a `screen` for each toplevel in the order they appear, and in
 ```
   KWin script "frametop-float"                ft-floatd (host, Python)                 ft-screens
   window events, moves, menus  ── D-Bus ──▶   window ↔ output ↔ panel table   ── @ft_screens ──▶   panels, controls,
-  runs commands                ◀─ long poll ─  spare outputs (kscreen-doctor)  ◀─ @frametop_float ─  lasers, ghost, catcher
+  runs commands                ◀─ long poll ─  spare outputs (kscreen-doctor)  ◀─ @frametop_float ─  lasers, catcher
 ```
 
-- **KWin script `frametop-float`** (JavaScript). A script keeps working across KWin updates. A C++ effect would have to match the host's exact KWin build, and our build container is Fedora, not SteamOS. The script watches windows (`windowAdded`/`windowRemoved`, `interactiveMoveResizeStarted`/`Stepped`/`Finished`, `outputChanged`, `minimizedChanged`, `windowActivated`, `fullScreenChanged`). It runs commands: move a window to an output, set its geometry, put it on all virtual desktops, and restore it. It adds "Float in VR" to the window menu (`registerUserActionsMenu`) and registers a shortcut (`registerShortcut`, Meta+Shift+F). KWin scripts can call D-Bus but can't serve it, so commands come back through a long poll. The script calls ft-floatd's `NextCommand`, which answers when a command is ready, and then the script calls it again. The fallback is loading one-shot scripts through `org.kde.kwin.Scripting`, the way kdotool does. All of these API names are present in the host's KWin 6.2.5.
-- **ft-floatd** (Python). The host has dbus-python and PyGObject. It owns `org.frametop.Float` on the session's private bus, and it keeps the table of which window is on which output and panel. It enables and disables spare outputs and sets their size, scale, and position with `kscreen-doctor`, as ft-layout does. It tells ft-screens where each floating window goes and tells the script which window goes where. It also remembers each app's placement and scale, keyed by desktop file name.
-- **ft-screens.** `Screen` becomes a panel with a kind: screen or floating window. Floating panels get the same bar, curve, roll, resize tab, and wrist and head pins, plus close, "back to desktop", and scale buttons. New parts are the tear-off ghost, the catcher, popup overlays, carrying a panel during a KWin move, and the dock target highlight. `MAX_SCREENS` goes from 8 to 16. Commands arrive on `@ft_screens`. Events go out to `@frametop_float` from an unbound socket, the same way ft-screens talks to the input relay.
-- **Session script.** Adds `FLOAT_SLOTS` to the output count, starts ft-floatd, and enables the KWin script in the session's `kwinrc`.
-- **ft-layout.** Arranges only the screens' outputs. Today it arranges everything in `kscreen-doctor -j`, so it has to skip the spares (`WL-<SCREENS>` and up), enabled or not.
-- **ft-pointer.** Changes to the drag lock (see "Drag and drop between panels").
-- **Frametop Display Settings.** Gets a Floating windows section: slots, margin, and "bring a window in front of you when it's activated".
+- **KWin script `frametop-float`** (`float/frametop-float.js`). ft-floatd loads it into the desktop's KWin over D-Bus (`org.kde.kwin.Scripting`). A script keeps working across KWin updates. A C++ effect would have to match the host's exact KWin build, and the build container is Fedora, not SteamOS. The script watches windows (`windowAdded`/`windowRemoved`, `frameGeometryChanged`, `outputChanged`, `interactiveMoveResizeStarted`/`Finished`, `fullScreenChanged`, `maximizedChanged`, `minimizedChanged`, `keepBelowChanged`, `windowActivated`) and the outputs (`screensChanged`). It runs commands: move a window to an output, set its geometry, put it on all virtual desktops, and restore it. It adds "Float in VR" ("Back to Desktop" on a floating window) to the window menu (`registerUserActionsMenu`). It registers no shortcut: the float key belongs to the input relay. KWin scripts can call D-Bus but can't serve it, so commands come back through a long poll. The script calls ft-floatd's `NextCommand`, which answers when a command is ready, and then the script calls it again. It also keeps KWin's placement memory from moving windows (see design.md).
+- **ft-floatd** (`float/ft-floatd`, Python). The host has dbus-python and PyGObject. It runs inside the desktop's Plasma session, started from its autostart. It owns `org.frametop.Float` on the session's private bus, and it keeps the table of which window is on which output and panel. It enables and disables spare outputs and sets their scale and position with `kscreen-doctor`, and their size through ft-screens. It tells ft-screens where each floating window goes and tells the script which window goes where. It launches apps floating, opens profiles' apps, and remembers each app's placement and scale, keyed by desktop file name. Commands come in on `@frametop_float`, from `ft-float`, the input relay, and ft-screens.
+- **ft-screens.** A spare output's panel is a floating window's. Floating panels get the same bar, curve, roll, resize tab, and wrist and head pins as screens, plus dock and close buttons left of the bar. Other parts: the catcher, popup and dialog overlays, and carrying a panel during a KWin move. `MAX_SCREENS` (screens and spares together) is 24. Commands arrive on `@ft_screens`. Events go out to `@frametop_float` from an unbound socket, the same way ft-screens talks to the input relay.
+- **Session script.** Adds `FLOAT_SLOTS` to KWin's output count, starts ft-floatd from the desktop's autostart, installs Frametop's window decoration and chooses it in the session's `kwinrc`, and writes the Launch as Standalone copies of the apps' desktop files.
+- **ft-layout.** Arranges only the screens' outputs, and leaves the spares (`WL-<SCREENS>` and up) to ft-floatd, enabled or not.
+- **ft-pointer.** The drag lock crosses onto other Frametop panels (see "Drag and drop between panels"), and a left release also goes to ft-screens as a backstop for the catcher.
+- **Input relay.** Owns the float key: `float_toggle` and `dock_all` send `float pointer` and `dock all` to ft-floatd.
 
-Program names stay within 15 characters (`ft-floatd`). Overlay keys are `frametop.float.N` and `frametop.float.N.bar`, and so on.
+Program names stay within 15 characters (`ft-floatd`). Overlay keys are `frametop.float.N` and `frametop.float.N.bar`, and so on; a floating window's popups and dialogs are `frametop.float.N.sub.K`.
 
 ## A floating window
 
-- **Output and margin.** Its output is the window's frame plus a margin on each side (`FLOAT_MARGIN`, default 300 px). KWin keeps a Wayland popup inside its parent's output (`XdgPopupWindow::updateRelativePlacement` uses the output's placement area), so the margin gives menus and dropdowns room past the window's edges. X11 apps place their own menus within the monitor, so the same applies. Enabled spares sit apart from the screens and from each other in KWin's layout, so nothing spills from one to the next. Memory: a 1600 × 1000 window with a 300 px margin is about 14 MB per buffer, 42 MB for three.
-- **What the panel shows.** Only the window's frame: ft-screens crops the output's buffer with `SetOverlayTextureBounds` and maps mouse positions through the crop. Each open popup gets a small overlay of its own, cut from the same buffer and placed a few millimetres in front of the window, so the main panel never changes size. KWin tells scripts about popups as windows of their own (`windowAdded` with `popupWindow`), so the script reports their rectangles.
-- **Size and scale.** The panel's width is the window's pixel width times the source screen's metres per pixel, so text stays the same size in VR. A window launched floating uses the primary screen's density. Each window also has a scale (KWin's output scale), changed with Meta+scroll over the window or +/- on its bar and remembered for each app. A bigger scale makes the content bigger at the same panel size.
+- **Output and margin.** Its output is the window's frame plus a margin on each side (`FLOAT_MARGIN`, default 300 px). KWin keeps a Wayland popup inside its parent's output, so the margin gives menus and dropdowns room past the window's edges. X11 apps place their own menus within the monitor, so the same applies. Enabled spares sit apart from the screens and from each other in KWin's layout, so nothing spills from one to the next. Memory: a 1600 × 1000 window with a 300 px margin is about 14 MB per buffer, 42 MB for three.
+- **What the panel shows.** Only the window's frame: ft-screens crops the output's buffer with `SetOverlayTextureBounds` and maps mouse positions through the crop. Each open popup or dialog gets a small overlay of its own, cut from the same buffer and placed a few millimetres in front of the window, so the main panel never changes size. KWin tells scripts about popups as windows of their own (`windowAdded` with `popupWindow`), so the script reports their rectangles.
+- **Where it appears.** Floated from a screen, the panel starts where the window was on that screen, 30 cm in front of it. Launched floating, it goes where that app last floated, or in front of you, 0.8 to 2 m away.
+- **Size and scale.** The panel's width is the window's pixel width times the source screen's metres per pixel, so text stays the same size in VR. A window launched floating uses the primary screen's density. Each window also has a scale (KWin's output scale), changed with Meta+scroll over the window in steps of 10% and remembered for each app. A bigger scale makes the content bigger at the same panel size.
 - **Window state.** An ordinary window, not maximized, placed inside its output with the margin around it, and set to show on all virtual desktops. It keeps its title bar and border. Apps that draw their own title bar (GTK, Chromium) keep theirs.
-- **Moving.** Press the title bar. KWin starts an interactive move and the script reports it. ft-screens then stops forwarding pointer motion to KWin, so KWin's pointer stays at the press point and the window moves by nothing. Meanwhile ft-screens carries the panel with the pressing device, the same way the bar does today: it follows rigidly, scroll pushes and pulls, and the 3D mouse's right-drag tilts. When the button comes up, KWin gets the release at the press point. The bar under the panel works too.
-- **Resizing.** The window's own edges (inside the margin, so KWin's resize works as on the desktop) and Frametop's corner tab both change the window's size in pixels at the same density, so the app lays itself out again. ft-floatd resizes the output to keep the margin, and the panel grows or shrinks around the window's top-left corner. A screen's tab only scales the panel. Resizing is throttled to about 20 updates a second, with a minimum of 320 × 200, like screens.
+- **Moving.** Press the title bar. KWin starts an interactive move on the press itself, before any motion, so ft-screens stops forwarding pointer motion to KWin as soon as a press lands in a floating window's title bar (from the frame and client rectangles ft-floatd sends it). KWin's pointer stays at the press point and the window moves by nothing. For apps that draw their own title bar, the script reports the move and ft-screens stops then (`carry`); ft-floatd puts back any few pixels the window slipped before that. Meanwhile ft-screens carries the panel with the pressing device, the same way the bar does: it follows rigidly, scroll pushes and pulls, and the 3D mouse's right-drag tilts. When the button comes up, KWin gets the release at the press point. The bar under the panel works too.
+- **Resizing.** The window's own edges (inside the margin, so KWin's resize works as on the desktop) and Frametop's corner tab both change the window's size in pixels at the same density, so the app lays itself out again. ft-floatd resizes the output to keep the margin, and the panel grows or shrinks around the window's top-left corner. KWin ends a resize by the window's edge whenever an output changes, so during one the panel follows the window and the output follows only when the drag ends: the margin is the room to grow until then. A screen's tab only scales the panel. Resizing is throttled to about 20 updates a second, with a minimum of 320 × 200, like screens.
+- **KWin's placement memory.** KWin puts windows back where they were for each layout of the outputs it has seen, which fights spare outputs that follow their windows' sizes. The KWin script undoes it ([design.md](design.md#kwins-placement-memory)).
 - **Full screen.** The window fills its own panel: the margin drops to zero while it's full screen, and the output is the panel's size in pixels. The panel keeps its size and place. On leaving full screen, the margin comes back.
-- **Buttons.** The close button closes the window. "Back to desktop" docks it where it came from.
+- **Buttons.** The close button closes the window. The dock button docks it where it came from.
 - **Minimize.** Minimizing, from the title bar or the taskbar, hides the panel, and restoring it shows the panel again. Floating windows stay in the desktop's taskbar and in Alt+Tab.
-- **Activated out of view.** When a floating window is activated (taskbar, Alt+Tab, a notification) and it's more than about 60° from where you're looking, it's focused and a glow at the edge of your view points to it. A setting moves it in front of you instead.
-- **New windows.** A dialog of a floating window (`transientFor`) floats in front of its parent. Other new windows of a floating app float too: where that app's windows went last time, otherwise to the parent's right at the same distance, curving around you, and to its left if that's taken. When every spare is in use, the window opens on the screen used last and a notification says so.
-- **Show Desktop.** Meta+D leaves floating windows alone.
+- **New windows.** Popups and dialogs of a floating window (`transientFor`) show as small overlays over it. Another window of a floating app that opens on its output floats too, a little in front of it. Any other window that opens on a floating window's output goes to the first screen that shows. When every spare is in use, the window stays on the screens and a notification says so.
+- **On a hidden screen.** A new window that opens on a screen hidden on its own floats instead, where that app last floated or in front of you.
 
-## Tearing a window off a screen
+## Tearing a window off a screen (not built)
+
+The design for decision 3:
 
 1. Press a desktop window's title bar and drag it. KWin starts a move, and the script tells ft-floatd, which tells ft-screens: `move-start <output> <window> <rect>`.
 2. While the button is held, the laser leaves every Frametop panel by more than a small dead zone (a few centimetres past the edge). Letting go inside the dead zone is an ordinary drop.
@@ -117,93 +123,48 @@ Program names stay within 15 characters (`ft-floatd`). Overlay keys are `frameto
 
 ## Putting it back
 
-- **Button.** "Back to desktop" returns the window to the screen, position, and size it had before it was torn off (saved at tear-off).
-- **Dragging.** Carry the floating window, by its title bar or its bar, until the spot you're pointing at is on a screen. Then push it flush with the screen, within about 10 cm of its surface: scroll away with the mouse, or move the controller forward. The screen shows where the window will land, and letting go docks it there at its current size in pixels, shrunk to fit if the screen is smaller. A carried panel keeps its distance, so moving a floating window in front of a screen never docks it by accident.
-- Docking disables the output and removes the panel.
+- **Button.** The dock button returns the window to the screen, position, and size it had before it floated. If that screen is hidden now, it goes onto the first screen that shows.
+- **Dragging (not built).** Carry the floating window, by its title bar or its bar, until the spot you're pointing at is on a screen. Then push it flush with the screen, within about 10 cm of its surface: scroll away with the mouse, or move the controller forward. The screen shows where the window will land, and letting go docks it there at its current size in pixels, shrunk to fit if the screen is smaller. A carried panel keeps its distance, so moving a floating window in front of a screen never docks it by accident.
+- Docking disables the output and hides the panel.
+
+## Getting at it: the float key and the title bar button
+
+Decisions 22 to 25.
+
+- **The float key.** The input relay owns it: the action `float_toggle`, bound to Meta+Shift+F unless the rules file says otherwise (a rules file with no `key_bindings` gets that default; one with its own list, even an empty one, doesn't). It can be rebound or removed in Frametop Input Settings, and mapped to a mouse button or a Frame controller button like any other action. The relay takes the combination before it reaches the desktop and sends `float pointer` to ft-floatd, which asks the script for the window under KWin's pointer (`workspace.cursorPos`, top of `workspace.stackingOrder`, popups and dialogs counting as their parent). With none there, the wallpaper or the taskbar, it's the active window. KWin's pointer is where the 3D mouse or a laser last was on a Frametop panel. A window that floats docks; any other floats. The KWin script has no shortcut of its own (ft-floatd removes one that an older script registered), so one press can't float a window and dock it again.
+- **Docking everything.** `dock_all` (no default binding) sends `dock all`, which docks every floating window where it came from.
+- **The title bar button.** Breeze can't take a button of its own, and a C++ fork of it would have to match SteamOS's exact KDecoration build (Plasma 6.3 replaces KDecoration2 with KDecoration3). So the Frametop desktop gets its own window decoration, written in QML for KWin's Aurorae engine, which loads it without compiling (`decoration/`, installed to `~/.local/share/kwin/decorations/kwin4_decoration_qml_frametop`, chosen in the session's `kwinrc` only, so Desktop Mode keeps Breeze; `decoration/apply.sh` switches the running desktop to it or back to Breeze). It's drawn to look like Breeze, with a float button left of Close. The button calls `requestToggleKeepBelow()`, the one window request a decoration can make that has no visible effect here, and the KWin script reads the change: keep-below set on a window on the screens floats it, cleared on a floating window docks it. The script keeps keep-below set on every floating window, however it was floated, so the button shows its dock icon there. A window alone on its own output loses nothing by being kept below (only the wallpaper is under it). If the window can't float (every spare is in use), the script clears the flag again. Keep Below Others in a window's menu does the same as the button.
+- **Apps that draw their own title bar** (Chromium and Electron apps, GTK apps) never show KWin's decoration, so they don't get the button. They use the key, or the window menu (Alt+F3).
 
 ## Launching an app floating
 
-- **In the desktop.** Use "Float in VR" in any window's menu, or press Meta+Shift+F for the active window.
-- **From a command.** `ft-float run <command>` and `ft-float launch <app.desktop>` start an app and float its first window. ft-floatd records the process it started, and the script matches new windows by PID, including child processes. Some single-instance apps (Firefox, D-Bus-activated apps) open the window from a process that was already running. Those are matched by desktop file name within a few seconds, or by `XDG_ACTIVATION_TOKEN` where the app honors it.
-- **From the headset without the desktop open.** One new launcher entry, "Frametop Apps". It starts the Frametop session with each screen hidden on its own (the per-screen hide that already exists), and opens an app picker as a floating window. Picking an app launches it floating. The hide hotkey and visibility modes still apply to everything, and a screen comes back with one click. Everything runs in one session, so dragging between a standalone app and a desktop app works. If the desktop is already running, the entry just opens the picker.
-- **The picker.** Either KRunner, floated, or a small Kirigami app like the settings apps, with a grid of apps and their icons.
-- **Remembered placement.** Each app's last floating pose, size, and scale, keyed by desktop file name. Named layouts don't include floating windows.
+- **In the desktop.** Use "Float in VR" in any window's menu, its title bar button, or the float key.
+- **From the menu.** Right-click an app in the Application Launcher, or in the taskbar (where it starts another window of that app), and pick "Launch as Standalone" (decision 26). The launcher has no way to add an entry to every app's menu, but its menu shows each app's own desktop actions. So the Frametop desktop reads copies of the apps' desktop files with one more action added (`float/ft_apps.py`). They're written to `~/.local/share/frametop/apps/applications` from every desktop file in `XDG_DATA_DIRS`: by the session script before Plasma starts, and by ft-floatd whenever an app is installed, changed, or removed. The session puts `~/.local/share/frametop/apps` first in `XDG_DATA_DIRS`. Plasma's app cache is keyed by those directories, so Desktop Mode never sees the copies. Desktop files in `~/.local/share/applications` come before every data dir, so an app you've customized there keeps your copy and has no Launch as Standalone. The action runs `ft-float launch <desktop file name>`.
+- **From a command.** `ft-float run <command>` and `ft-float launch <app.desktop>` start an app and float its first window. ft-floatd records the process it started, and new windows are matched by PID, including child processes. Some single-instance apps (Firefox, D-Bus-activated apps) open the window from a process that was already running. Those are matched by desktop file name. Either way, the window has to show up within 30 seconds.
+- **From the headset with the desktop off.** A profile's launcher entry starts the desktop in that profile, and a profile can hide every screen and hold only floating apps (`docs/profiles.md`). There's no separate Frametop Apps entry or picker.
+- **Remembered placement.** Each app's last floating pose (relative to the primary screen's panel, so it moves with the screens' layout), size in pixels, and scale, keyed by desktop file name, in `~/.config/frametop-float.json`. It's kept whenever one of the app's windows stops floating. With nothing remembered, the window opens in front of you, at the primary screen's density, 0.8 to 2 m away. A profile's own placement wins when the profile opens the app.
 
 ## Drag and drop between panels
 
 KWin handles the protocols: Wayland, X11 through Xwayland, and the portal's file transfer. Frametop has to get the pointer right between panels.
 
 - **Crossing panels.** When the laser moves onto another panel mid-drag, ft-screens gives that panel's KWin window pointer focus. KWin puts its cursor at that output's position, and the drop target gets enter and motion events. Floating windows add nothing new here, but they make gaps between panels the normal case.
-- **Gaps (the catcher).** While the laser is between panels, none of our overlays get its events, and ft-screens clears pointer focus on `FT_LEAVE` even with a button held. If you let go in empty space, the release never reaches KWin, and the drag or move stays stuck until the next click. The fix: while a button is held on a Frametop panel and the laser leaves all of them, ft-screens puts an invisible catcher overlay on the laser (the tear-off ghost is the same thing with a picture on it). A release on the catcher releases in KWin wherever the pointer last was. Dropping in a gap cancels, just as dropping outside any window does. The pointer helper also tells ft-screens when the mouse's left button comes up, in case the catcher misses it. This also fixes window moves and drags that end off a panel today.
-- **The 3D mouse's drag lock.** While the button is held, the drag lock keeps the cursor at its distance and stops hit tests. So a drag onto a nearer panel passes behind it, and a farther panel works only if SteamVR's laser happens to reach it. The change: while the button is held, keep testing the other panels (not the one pressed on) and move onto a panel the ray meets. Off the edge of the pressed panel, keep that panel's plane, as now, so moves and resizes past the edge still work.
-- **Drag icon.** KWin 6 draws the drag icon as part of its scene, so it should show on the panel under the laser (to check). In a gap it stops at the source panel's edge. A later version can show a small drag proxy on the catcher.
-- **Flatpak apps.** Dropping files into a sandboxed app goes through the document portal, the same path that the session script's file-picker fix covers. Test it explicitly, for example Dolphin to Brave.
+- **Gaps (the catcher).** While the laser is between panels, none of Frametop's overlays get its events, and ft-screens clears pointer focus on `FT_LEAVE` even with a button held. A release in empty space would never reach KWin, and the drag or move would stay stuck until the next click. So while a button is held on a Frametop panel and the laser leaves all of them, ft-screens puts an invisible catcher overlay on the laser. A release on the catcher releases in KWin wherever the pointer last was. Dropping in a gap cancels, just as dropping outside any window does. The pointer helper also tells ft-screens when the mouse's left button comes up ("up"), in case the catcher misses it. This also covers window moves and drags on the screens that end off a panel.
+- **The 3D mouse's drag lock.** While the button is held, the drag lock keeps the cursor at its distance and stops hit tests, so a drag onto a nearer panel would pass behind it. So while the button is held, the helper keeps testing the other Frametop panels (not the one pressed on, and not while carrying one) and moves onto a panel the ray meets. Off the edge of the pressed panel, it keeps that panel's plane, so moves and resizes past the edge still work.
+- **Drag icon.** KWin 6 draws the drag icon as part of its scene, on the output its pointer is on. In a gap it stays at the source panel's edge.
+- **Flatpak apps.** Dropping files into a sandboxed app goes through the document portal, the same path that the session script's file-picker fix covers ([design.md](design.md#the-desktop-session)).
 
 ## Things that must keep working
 
 - **Typing follows the last click.** A click on a floating panel counts as a click on the desktop, since the window is a KWin window.
 - **Visibility.** Floating windows follow the screens' rules: the hide hotkey, the visibility modes, and hiding during a VR game unless the dashboard is open. Controllers' lasers are off in games.
 - **Headset standby.** Nothing new may poll SteamVR with new clients, so no new `vrcmd` loops.
-- **The pointer helper's overlay list.** The helper learns about overlays by running `vrcmd --overlays` in the background, so a new floating panel appears in its next listing. Check the delay after a tear-off. If it's too long, ft-screens can send the helper new overlay keys directly.
+- **The pointer helper's overlay list.** The helper learns about overlays by running `vrcmd --overlays` in the background, so a new floating panel appears in its next listing.
 - **Plasma.** An enabled floating output gets a desktop view (wallpaper) under its window, hidden by the crop. Plasma doesn't add panels to new outputs by default. A floating output must never become primary. With spare outputs, the output count stays the same, which avoids the lost-taskbar problem in design.md's open questions.
 - **Restarting the desktop** closes every window, floating ones included. Each app's placement is remembered, so an app launched floating again comes back where it was.
 
-## Plan
-
-### Step 1: the catcher and the branches
-
-- The catcher (see "Gaps"), as a fix that stands on its own, so it can go to `main` by itself.
-- Merge `pointer-ignore` and `layouts-headpin`.
-
-### Phase 0: find out
-
-Each item has a pass condition. Items that need a desktop restart with extra outputs wait until the headset is free, or run in the headless test mode from 0.1.
-
-- **0.1 Headless test mode.** `ft-screens --no-vr` runs the compositor without SteamVR. It logs toplevels, titles, and sizes, and answers commands on a separate control socket name. This lets KWin and output experiments run without the headset, and without touching the running desktop.
-- **0.2 Outputs.** Start KWin with spare outputs, then disable and re-enable one with `kscreen-doctor`. Pass: the toplevel stays and its title changes (as the source says), a disabled output costs no frames, resizing a spare through configure works, and outputs with gaps between them are accepted. Also: an output larger than its window with the window placed inside, and a popup placed in the margin.
-- **0.3 KWin script API on 6.2.5.** Move signals fire for moves from both KWin's title bars and apps' own. `sendClientToScreen` and `frameGeometry` work on another output, the `callDBus` long poll works, and `registerUserActionsMenu` works. Popups show up in `windowAdded` with their geometry. The observers can load into the running desktop through `org.kde.kwin.Scripting` and move nothing, so this is safe while the headset is in use.
-- **0.4 Frozen-pointer move.** Pass: a KWin move with no pointer movement doesn't shift the window.
-- **0.5 SteamVR's laser.** Find out which overlay gets MouseMove and ButtonUp when a held laser moves from overlay A to overlay B, and when it's released over nothing, for both a controller and the 3D mouse. Pass: an interactive overlay placed on the laser reliably catches the release.
-- **0.6 Texture bounds.** Pass: `SetOverlayTextureBounds` crops a DMA-BUF (`SharedTextureHandle`) overlay correctly, mouse positions map to the cropped area, and two overlays can show different crops of one buffer.
-
-#### Results (2026-09-29, headless, KWin 6.2.5)
-
-`screens/test/headless.sh` runs these: ft-screens `--no-vr` with a bare nested KWin next to the running desktop, with an `input` command that feeds pointer events as if from a panel, KWin scripts loaded over D-Bus, and screenshots through ScreenShot2.
-
-- **0.1 passes.** `--no-vr`, `--control`, `toplevels`, and `input` are in ft-screens.
-- **0.2 passes.** Disabling a spare with `kscreen-doctor` keeps its toplevel, its title gets `- Output disabled`, and it stops committing; enabling it resumes on the same toplevel. A spare resized (`size`) while disabled comes up at the new size on its first frame, so a tear-off needn't blink. Live resizing works, output scale works (1.5: KWin lays out 1067 × 667 on a 1600 × 1000 buffer), and outputs with gaps between them (x = 5000, 8000, 10000) are accepted. A window placed inside a 1600 × 1200 output with a 300 px margin opens a context menu past its bottom and right edges, into the margin.
-- **0.3 mostly passes.** `workspace.screens`, `sendClientToScreen`, `windowList`, `frameGeometry` (set; it applies asynchronously), `callDBus`, `registerShortcut`, `registerUserActionsMenu`, and `readConfig` exist. `windowAdded` reports popups (`popupWindow` true, `transient` true) with their geometry. Move signals fire for KWin's title bars (`interactiveMoveResizeStarted` with `move` true, `Stepped` with the geometry, `Finished`). Not yet checked: apps' own title bars, the `callDBus` long poll, and the window menu entry. `globalThis` isn't defined in KWin's script engine. `print` goes to the journal unless `QT_FORCE_STDERR_LOGGING=1`.
-- **0.4: KWin starts the move on the press itself,** before any motion. So ft-screens freezes pointer motion as soon as a press lands in a floating window's title bar band (from the frame and client rectangles ft-floatd sends it), with no round trip. For apps that draw their own title bars, the script reports the move and ft-screens freezes then; the script puts back any few pixels the window slipped before that.
-- **Found and fixed:** KWin's nested backend ignores the position in `wl_pointer.enter`, and wlroots drops a motion to the position it entered at, so the first click after crossing onto another screen landed where KWin's pointer had been. ft-screens now enters one unit off.
-- **0.5 and 0.6** need SteamVR and the headset.
-
-### Phase 1: float a window from its menu
-
-Build the KWin script, ft-floatd, and floating panels in ft-screens: the margin and popup overlays, controls, moving by the title bar, resizing (edges and tab), scale, full screen, close, and back to desktop. Add drag and drop across panels, with the pointer helper change.
-
-Done when:
-
-- Dolphin and Kate float from "Float in VR".
-- A file drags from the floating Dolphin to the floating Kate, to a screen, and back.
-- The clipboard works between them.
-- A menu near a floating window's edge opens past the edge.
-- Closing and docking give the output back.
-- Frame pacing and GPU memory are measured with several floating windows.
-
-### Phase 2: tear off and dock by dragging
-
-The ghost and tear-off, and the dock highlight with push-flush docking.
-
-### Phase 3: launch floating
-
-`ft-float run` and `ft-float launch`, window matching, new windows of floating apps, the Frametop Apps launcher entry, the picker, remembered placement, and the notification when every spare is in use.
-
-### Phase 4: polish
-
-The drag proxy, the glow toward a window activated out of view (and the setting to bring it in front), the Display Settings section, and docs (design.md, reference.md, and the Use table in the README).
-
 ## Risks
 
-- A SteamOS update can change KWin's script API or its nested backend. The script and the output handling are the parts to recheck after one.
+- A SteamOS update can change KWin's script API or its nested backend. The script and the output handling are the parts to recheck after one, and KWin's placement memory with them (the script undoes it, see [design.md](design.md#kwins-placement-memory)).
 - GPU memory: each floating output has its own swapchain of two or three buffers, including the margin. The Frame has 16 GB shared, with about 4 GB free in normal use (2026-09-29).
 - Frame pacing with many panels hasn't been measured (already an open question in design.md). Each output is a separate render pass in KWin.

docs/gaze-controllers.md

@@ -0,0 +1,28 @@
+# Gaze with the controllers: a dead end
+
+Gaze mode is a mouse and keyboard feature. On 2026-09-30 we tried to make the Frame controllers its buttons: with gaze mode on and no game running, either controller's trigger would click where you look (a tap clicks, moving the hand steers the pointer, holding still drags), the controllers' lasers would be muted, and SteamVR's dashboard would follow the gaze too. It can't be done cleanly, for the reasons below. The work wasn't merged, and it's kept outside the published history.
+
+## What worked
+
+- Our `ft_pointer` device can hold SteamVR's laser without a hand role, in the treadmill role. It has to hint that role when SteamVR activates it; a hint changed later never gets the `/user/treadmill` path.
+- A trimmed copy of the Frame controller's compositor binding mutes the controllers' laser buttons. It's chosen with `POST /input/selectconfig.action` on vrserver's port 27062 (a JSON body; a form-encoded one gets "Parse failed"). The helper's global action sets can't do it, because SteamVR marks them inactive while its laser mouse has focus.
+- vrserver's web socket on 127.0.0.1:27062 reports every controller component without taking it from anyone.
+- Steam's UI can be kept from acting on the controllers by wrapping its gamepad input source (webpack module 17900) through Steam's CEF debugger.
+- Our device takes the laser back 10 to 15 ms after a controller takes it.
+
+## What broke it
+
+- Steam reads the Frame controllers itself. They aren't devices on the host: vrserver owns their radio and passes their raw reports to Steam through SteamVR's private Steam interface. Steam's client library turns them into a virtual device ("SteamFrameVirtual", at `/steamvr/virtual`), outside every SteamVR binding. Steam's own SteamVR action manifest asks only for haptics.
+- Every press and every release that Steam sees takes SteamVR's dashboard, and Frametop's panels, out of laser mode about 40 ms later. That happens whatever the bindings say and whatever Steam's UI does with the event. None of these stopped it: dropping the events in Steam's UI, removing the controller's `dualanalog` bindings, binding every button to a harmless compositor action, or setting `dashboard.modalGamepadAndLaser` to false.
+- Taking the laser back after each switch leaves a gap of 20 to 40 ms, and panels treat it as the pointer leaving, so clicks and drags break.
+- Steam can't be told to ignore the controllers. It won't save a controller layout for the virtual controller ("Saved Binding Selection Failed - No Identity"), and its menus don't go through the layout anyway: a live preview of the empty layout for Steam's UI (app 769) changed nothing.
+- SteamVR hands the controllers to a VR app instead of Steam only while that app has the input focus, as games do. A dashboard overlay with overlay flag `1 << 4` is given the focus only in gamepad mode, and only for gamepad input.
+
+## Options not taken
+
+- Frametop as a transparent VR app (a scene application using OpenXR's alpha blend mode) whenever gaze mode is on. That would cut Steam off while the dashboard is closed, but Steam's dashboard pages would still take the controllers, and it costs a scene layer all the time.
+- Patching Steam's running process, with an eBPF probe that writes its memory or an injected hook, to drop the controller reports while gaze mode is on. It would cover everything, but it changes Valve's software, needs Steam's client library reverse-engineered, and breaks with Steam updates.
+
+## What was built
+
+The attempt had a plan and a test log, probes for the laser, input focus, and SteamVR settings, a reader for vrserver's web socket, a filter for Steam's UI, and controller code in the relay and the helper. None of it is in this repo. Only the gaze dot setting (`POINTER_GAZE_DOT`) came over.

docs/hands-migration.md

@@ -1,140 +0,0 @@
-# Hands in Frametop: migration plan
-
-Hand tracking from the headset's own cameras has been built as a separate project, frame-hands (`~/Desktop/Projects/frame-hands` on the Frame, a local git repo with no remote). The plan is to make it a native Frametop component, like `gaze/` and `power/`, instead of a separate module. The work happens on branch `hands-migration` (worktree `frametop-hands/` in the PC workspace) and is merged into `experimental` after it's been tested in the headset.
-
-Builds from this worktree must sync to their own folder on the Frame, never `~/dev/frametop`: run every script with `FRAME_REPO=/home/steamos/dev/frametop-hands`.
-
-## Status (2026-09-30)
-
-Steps 1-5 are done:
-
-- frame-hands' pending work was committed there (6c63c9e).
-- Its filtered history was merged under `hands/` (1a76d15), then laid out (`trackd/` to `track/`).
-- The renames, the Frametop paths, and ft-camd's file capabilities are done. So are `hands/Makefile`, `build.sh`, `run.sh`, the two units, the README, the settings, the installer step, and ft-screens on the shared header.
-- Built in the dev container on the Frame, and on the 7i.
-- Checked without the headset:
-  - `ft-handreplay` against frame-hands' `fh-replay`, both x86 with `--cost`, on the whole dim recording and the first 60 s of the bright one: identical summaries and byte-identical depth dumps. The Makefile's own ncnn build is included in that.
-  - `ft-ringplay` into `ft-hands` on the 7i tracked, pinched, and wrote `/run/user/UID/frametop-hands/{hands,gestures}`.
-  - On the Frame, ft-hands in the container finds the calibration through `/run/host/persist`, and ft-camd without its capabilities refuses with a clear message.
-
-Step 6 has started (2026-09-30 10:30):
-- `hands/run.sh install` is done, and both services run from this worktree.
-- The files moved to `/run/user/UID/frametop-hands/`, because `/run/user/UID/frametop` is the desktop session's own runtime folder, deleted at every desktop start.
-- Until the desktop restarts from a build with this branch's ft-screens, the link `/run/user/UID/frame-hands -> frametop-hands` feeds the running one. It's tmpfs, so it's gone at reboot.
-
-Found in the headset:
-- The side cameras were swapped (`HANDS_SWAP_SIDES=1`).
-- The cutout copy shader lost resolution at `mediump` (now `highp`).
-- Colour capture isn't reliable (see the README).
-- Two pinch fixes: one hand no longer pinches both sides, and the palm-down limit stops typing pinches.
-
-## What frame-hands is today
-
-| Part | What it is | Size |
-| --- | --- | --- |
-| `camd/` | `fh-camd`, the camera broker (C). It borrows XRService's camera DMA-BUFs read-only with `pidfd_getfd`, times them with the `v4l2_dqbuf` tracepoint, and publishes the four IR cameras (and optionally the two colour cameras) to a shared-memory ring. It starts as root and drops to the user after setup. Adapted in part from FrameEyeCameraFeed (MIT, licence file kept). `fh-camprobe` is its discovery and recording probe. | camd 1.1k lines, tp 0.4k, xrcams 0.8k, camprobe 1.1k |
-| `trackd/` | `fh-tracker` (C++): the tracker, the models on ncnn, the calibration (jsoncpp), the pinch detector, the publisher, and the recorder. Also `fh-replay` (offline replay and scoring), `fh-ringplay` (plays a recording into a ring), and `nettest`. | 2.9k lines |
-| `include/` | The hands file (`fh_hands.h`, read by ft-screens) and the gestures file (`fh_gestures.h`, pinches). | |
-| `models/ncnn/` | MediaPipe's palm detector and hand landmark model, from the OpenCV Zoo ONNX ports (Apache-2.0), converted to ncnn in float and int8. | 5.9 MB |
-| `tools/` | Python analysis: side-camera check, colour calibration check, frame viewer, gesture watcher, depth report, model comparison, int8 calibration, model conversion. | ~1.1k lines |
-| `tracker/` | The Python prototype of the tracker. Some tools import its `calib.py` and `models.py`. | 1.3k lines |
-| `probes/`, `notes/`, `re/`, `shim/` | One-off experiments, reverse-engineering notes on SteamVR's passthrough internals, a disassembly (not in git), and a header from an abandoned XRService shim approach. | |
-| `vendor/`, `captures/` | ncnn and FrameEyeCameraFeed clones, and recordings of the user's hands and room (tens of GB). Neither is in git. | |
-
-Today it runs by hand: `sudo camd/fh-camd`, then `trackd/fh-tracker`. There are no units and no installer. Files: `/run/frame-hands/ir-ring` (the ring, in a root-owned folder), and `$XDG_RUNTIME_DIR/frame-hands/hands` and `gestures`.
-
-Frametop already has the consumer side on `experimental`: `screens/handcut.{h,cpp}` cuts the hands out of the screens, with its own copy of the hands file layout, and `screens/handtest.cpp` tries it on a test panel.
-
-## Where it goes
-
-A top-level `hands/` folder, laid out like `gaze/`:
-
-```
-hands/
-  README.md                 # from trackd/README.md and camd/README.md
-  build.sh                  # ft-camd, ft-hands; --tools also builds the replay tools
-  run.sh                    # install|uninstall|start|stop|restart|status|log
-  frametop-camd.service     # user units (templates, @REPO@)
-  frametop-hands.service
-  include/                  # fhring.h, fh_hands.h, fh_gestures.h: shared with screens/ and pointer/
-  camd/                     # ft-camd: camd.c tp.c xrcams.c, LICENSE.FrameEyeCameraFeed
-  track/                    # ft-hands: tracker, nets, calib, pinch, publish, record; replay.cpp
-                            #   (ft-handreplay) and ringplay.cpp (ft-ringplay) for recordings
-  models/                   # the ncnn models, with NOTICE (Apache-2.0, MediaPipe / OpenCV Zoo)
-  tools/                    # the Python checks, watch_gestures, depth_report, calib.py, ring.py
-```
-
-Left behind in frame-hands, which stays as the lab: the recordings, the Python prototype (the tools that need `calib.py` or `models.py` get a trimmed copy in `hands/tools/`), `probes/`, `notes/`, `re/`, `shim/`, `camprobe`, and `vendor/`. The reverse-engineering notes don't belong in a public repo, and recordings are images of the user's hands and room, so they never go into git.
-
-## Names
-
-Programs within 15 characters, `ft-` prefix; files under `frametop`:
-
-| Now | In Frametop |
-| --- | --- |
-| `fh-camd` | `ft-camd` |
-| `fh-tracker` | `ft-hands` |
-| `fh-replay`, `fh-ringplay` | `ft-handreplay`, `ft-ringplay` |
-| `/run/frame-hands/ir-ring` | `/run/user/UID/frametop-hands/cam-ring` |
-| `$XDG_RUNTIME_DIR/frame-hands/hands`, `gestures` | `/run/user/UID/frametop-hands/hands`, `gestures` |
-
-The source keeps its `fh_` identifiers and header names (`fh_hands.h`, `fh_gestures.h`, `fhring.h`), and the file formats keep their magic strings, so recordings and tools from frame-hands keep working. Programs, units and runtime paths change.
-
-## Build
-
-- `hands/build.sh` builds in the dev container through `scripts/frame.sh --build`, into `hands/build/`, like the other components. `FRAME_BUILDER=pc` can take the ncnn build.
-- ncnn: fetched at a pinned tag (20260526, as now) into `hands/build/ncnn` and built once, the way `screens/build.sh` fetches the OpenVR header, with frame-hands' options so results match. `NCNN=` points the build at an existing install instead. Every net runs single-threaded (`num_threads = 1`), with the tracker spreading nets over its own pinned threads, so OpenMP could go later.
-- ft-hands runs in the dev container like ft-pointer and ft-powerd (`distrobox enter dev --`, after `scripts/container-up.sh`). Today's fh-tracker runs on the host and works only because the host happens to have the same `libjsoncpp.so.25` and libgomp as the container. Inside the container the calibration is at `/run/host/persist`, and calib.cpp (and `tools/calib.py`) fall back to it when `/persist` isn't there.
-- ft-camd has to run on the host (below), so it's linked statically (only libc and libm; `glibc-static` goes into `setup/dev-container.sh`). The host has an older glibc than the container.
-
-## Running it
-
-**ft-camd needs privileges**, only while it sets up: `pidfd_getfd` on XRService (the Frame has `ptrace_scope=1`), system-wide tracepoints (`perf_event_paranoid=2`), and the tracepoint files, which are root-only (`/sys/kernel/tracing/events/v4l2/v4l2_dqbuf/{id,format}` are mode 0440). A rootless container's root can't do any of that, so it runs on the host. Two ways:
-
-- **A. File capabilities (chosen, 2026-09-30).** The installer runs `sudo setcap cap_sys_ptrace,cap_perfmon,cap_dac_read_search+ep hands/build/ft-camd` once. ft-camd then runs as the user, in a user unit `PartOf=steamvr.service`, so it starts and stops with SteamVR, and its ring lives in the user's runtime folder. It drops all capabilities after setup, as it drops root today. Nothing ever runs as root. Writing the file clears its capabilities, so a rebuilt ft-camd needs the setcap again. It changes rarely. `/home` on the Frame is ext4 without `nosuid`, so file capabilities work there.
-- **B. Root system service**, like the Bluetooth fixes: a root-owned copy in `/var/lib/frametop/`, a unit in `/etc/systemd/system/`. It would have to watch for XRService itself, because a system unit can't follow the user's `steamvr.service`.
-
-Either way the password is needed once at install, through the same `sudo -S` path the Bluetooth fixes use, and only after asking.
-
-**ft-hands** is a user unit, `frametop-hands.service`: after `frametop-camd.service`, `PartOf=steamvr.service`, nice 5, model threads on CPUs 5-7 (measured best on 2026-09-29).
-
-**Settings** in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES=1` and `HANDS_CPUS=5,6,7`, read by ft-hands. It's on while its services are installed (`hands/run.sh install`, `uninstall`), so there's no `HANDS` switch. There's no setting for colour yet. Later, a switch in Frametop Display Settings.
-
-**Installer:** an optional last step in `install.sh`, off by default, which asks first because it needs sudo.
-
-## Interfaces
-
-- `screens/handcut.cpp` includes `hands/include/ft_hands.h` instead of its own copy of the layout, and reads the new path. ft-screens and ft-hands change together on this branch.
-- Pinches go to the pointer helper. It maps the gestures file and checks the begin and end counters each tick. A begin is a press, an end the release, and the pinch point's movement a drag. In gaze mode, the press lands where you look. The counters mean a quick tap between two ticks isn't missed. The tracker knows nothing about the pointer.
-
-## Open items that aren't part of the move
-
-These block shipping hands to other people, not the migration:
-
-- **The side-camera swap.** After some XRService restarts, fh-camd publishes the two side cameras under each other's names. Today it's caught by hand (`tools/check_sides.py --ring`, then `--swap-sides`). It needs fixing at the source (tell the buffers apart by the `dqbuf` tracepoint's device, the way the colour pair is split), or at least an automatic check at start-up.
-- **The colour cameras' calibration mapping** (`tools/check_color.py` on a recording with texture).
-- **Depth when one camera loses the hand.** From the 2026-09-30 replay measurements: drifting 10% per update toward the one-camera guess (`kMonoDepthGain`) makes the depth worse than keeping the last distance. Try 0.02.
-
-## Public repo
-
-Frametop is public. **Not pushed to GitHub until the user says it's ready** (user decision, 2026-09-30). When it is, it publishes:
-
-- The camera borrowing (`pidfd_getfd` on XRService's buffers) and the tracepoint timing. FrameEyeCameraFeed already does the same publicly. Its MIT licence and credit stay with the code.
-- The models, under Apache-2.0, with a NOTICE.
-
-It doesn't publish the reverse-engineering notes, the probes, or any recording. They stay in frame-hands.
-
-## History
-
-frame-hands' work was committed there first (6c63c9e, its 4th commit). Its history was then filtered to drop what stays behind (`notes/`, `probes/`, `shim/`, `camd/camprobe.c`, the camprobe tools, the Python prototype except `calib.py` and `models.py`, and `.frame-job`) from every commit. It was merged into this branch under `hands/` (a subtree merge), so blame still leads to where each line came from. The renames come after, as their own commits.
-
-## Steps
-
-1. In frame-hands: commit the pending work, as its last state before the move (needs the user's OK).
-2. On this branch: import it under `hands/`, then rename the programs and paths. The behaviour stays identical.
-3. `hands/build.sh`, `run.sh`, the two units, the README, the settings, and the installer step.
-4. ft-screens' hand cutouts on the shared header and the new path.
-5. Check without the headset. `ft-handreplay` on the 2026-09-29 recordings with `--cost` is repeatable, so its summary must match `fh-replay`'s exactly. And `ft-ringplay` into ft-hands must publish the same hands as into fh-tracker.
-6. In the headset, with the user and after asking: stop fh-camd and fh-tracker, install the services from `~/dev/frametop-hands`, and restart the desktop from this branch so ft-screens reads the new path.
-7. Pinch into the pointer helper (it can also follow the merge). The gaze work is on the Frame's `~/frametop` main: on 2026-09-30 that branch had 4 commits `experimental` doesn't have, plus uncommitted work in the pointer helper's gaze mode. Build this step on wherever that work lands, not on this branch's older copy.
-8. Merge into `experimental`. It's checked out in a worktree on the Frame (`~/frametop/.worktrees/experimental`, where the live desktop runs), so the merge happens there, or `experimental` is switched away first.

docs/hazards.md

@@ -16,8 +16,8 @@ The input relay takes the volume keys from every device that has them, so gamesc
 
 ft-screens drops keys while no screen has focus or the SteamVR dashboard is open, but always lets through the release of a key the desktop saw pressed, so a modifier held as the dashboard opens doesn't stay down.
 
-- **A release that never arrives leaves the key held in the desktop.** KWin repeats held keys itself, so a stuck letter repeats and a stuck modifier changes every later key (Ctrl+Alt held turns T into Konsole). Pressing and releasing the key again clears it.
-- **A keyboard that disconnects mid-press is one way to get there.** The relay forgets the held key without telling ft-screens. The same goes for the relay restarting while a key is down.
+- **A key whose release never arrives stays held in the desktop until the relay clears it, within about a second.** KWin repeats held keys itself, so a stuck letter repeats and a stuck modifier changes every later key (Ctrl+Alt held turns T into Konsole). The relay remembers which keys it told the desktop went down, and once a second it releases any that no keyboard holds (`reconcile_desktop_keys`, which asks the kernel with `EVIOCGKEY`). Pressing and releasing the key again also clears it.
+- **A keyboard that disconnects mid-press, or a relay restart with a key down, is how it happens.** The once-a-second check catches the first. A relay that starts doesn't know what an earlier one left down, so it releases the modifiers on the desktop; another key left down that way stays until it's pressed and released again.
 - **To see where a key went,** run `scripts/keys-report.py` and reproduce the problem while it records. It logs the modifiers, Tab, and Esc (no other keys) as the relay reads them and as its virtual keyboard sends them on, with the device roles and grabs, which programs have each keyboard open, and the relay's and desktop's logs.
 - **Switching where typing goes waits for keys to come up.** The relay changes a keyboard's grab only while none of its keys are down, so a press and its release go to the same side. A key held for a long time delays the switch until it's let go.
 

docs/profiles.md

@@ -0,0 +1,54 @@
+# Profiles
+
+Profiles are built: Display Settings, `ft-layout`, each profile's launcher entry, and the input relay's `profile:NAME` action open them, and the desktop can start in one.
+
+A profile is a named layout that also opens apps. It holds:
+
+- where each screen goes, with its size in metres, curve, roll, and pin (what a named layout held before profiles);
+- which screens show and which are hidden;
+- the apps, one entry per window: on a screen at a place and size, or floating at a pose, size, and scale.
+
+So a "Work" profile can put three screens around you with a browser, two terminals, and an editor on them, and a "Couch" profile can hide every screen and float one video player in front of you.
+
+## Decisions
+
+| # | Question | Decision |
+|---|---|---|
+| 1 | Profiles and named layouts | One list. Named layouts grow into profiles; a layout saved before profiles existed is a profile with no apps and every screen shown |
+| 2 | Making one | Capture what's open: the screens and every app's windows. Display Settings lists a profile's apps, so one can be removed. No editor beyond that |
+| 3 | Apps with several windows | One entry per window. The app is launched once; when its first window shows up, it's launched again for each window still missing. A browser that restores its own windows gets launched once, a terminal twice |
+| 4 | What's recorded of an app | Its desktop file name, or its command line if it has none. Not what it had open: tabs, files, and folders are left to the app's own restore |
+| 5 | Switching while apps are open | Additive: launch what's missing, move the windows that match into place, leave the rest alone. Nothing is ever closed |
+| 6 | Saving changes | Only on an explicit save. Moving things after switching doesn't change the profile |
+| 7 | Starting one | Four ways: a default profile when the desktop starts, Display Settings, a launcher entry for each profile, and a mappable action |
+| 8 | Plasma's session restore | Off in the Frametop session, so a profile is the only thing that reopens apps |
+| 9 | Screen count and resolution | Not part of a profile. They're global, because changing them restarts the desktop, which closes every window |
+
+## Where profiles live
+
+`~/.config/frametop-layout.json` keeps its `layouts` as they are (`{"Work": [screen places]}`), so older copies of ft-layout still read it. What a profile adds goes in a parallel `profiles` map under the same names:
+
+```
+"layouts":  {"Work": [{"pos": ..., "face": ..., "roll": ..., "metres": ..., "curve": ..., "pin": ...}, ...]},
+"profiles": {"Work": {"hidden": [3],
+                      "windows": [
+                        {"app": "org.kde.konsole", "screen": 2, "rect": [40, 60, 1200, 800], "maximized": false},
+                        {"app": "com.brave.Browser", "screen": 1, "maximized": true},
+                        {"cmd": ["/opt/tool/run"], "class": "tool", "screen": 1, "rect": [...]},
+                        {"app": "org.kde.dolphin", "float": {"rel": [12 numbers], "pixels": [1400, 900],
+                                                             "scale": 1.2, "mpp": 0.00097}}]}},
+"default_profile": "Work"
+```
+
+- `screen` is 1-based, as everywhere in Frametop. `rect` is the window's frame in KWin's logical units, relative to its screen's output, so it survives the screens being arranged differently.
+- A floating window's place (`rel`) is its panel's centre and axes in the frame of the primary screen's panel, the same way ft-floatd remembers each app's place. The screens go relative to your head when the profile is applied, and the floating windows follow them. `mpp` is its density in metres per pixel, and `scale` its scale, put back with the rest.
+- Renaming or deleting a layout renames or deletes its profile entry with it.
+
+## How it works
+
+- **Capture** (`ft-layout save NAME`, and Save as profile… in Display Settings). ft-layout captures the screens as before, then asks ft-floatd for the windows (`windows` on @frametop_float). ft-floatd has the KWin script report every window as it is now (`report-all`), then answers with every normal window: its desktop file name, the screen it's on, its rectangle there, and whether it's maximized. For floating windows it gives their panel's place, their size in pixels, and their scale. A window whose app id has no desktop file (a Flatpak app's X11 window can give its own: RustDesk's says `com.carriez.flutter_hbb`, its desktop file is `com.rustdesk.RustDesk`) is kept by the desktop file whose `StartupWMClass` names its window class. Windows with no desktop file name are kept by their process's command line (`/proc/<pid>/cmdline`) and window class. Windows of Plasma itself, the Frametop settings apps, and dialogs aren't recorded. If ft-floatd doesn't answer, the profile keeps the apps it had.
+- **Apply** (`ft-layout use NAME`, Open profile in Display Settings). ft-layout makes the profile's hidden screens the screens' own setting, arranges the screens (which hides and shows them: ft-screens' `conceal` and `reveal`), then has ft-floatd open the apps (`profile NAME`; ft-floatd reads the windows from the layout file). If the screens can't be arranged, for example with the headset off and no head pose, the apps still open: the screens stay where they are, the profile's hidden screens still hide, and floating windows go relative to the screens wherever they are. ft-floatd goes through the entries app by app. It claims windows of that app already open (oldest first, each claimed once), and moves each to its entry's place: onto its screen at its rect (or maximized), or floating at its pose. For the entries left over, it launches the app once (`ft-float launch`, the same path as Launch as Standalone) and waits up to 30 seconds for its first window. Each window that shows up goes to the next entry's place. Once the first window has been up for 3 seconds (time for an app that restores its own windows to show them), ft-floatd launches the app again for each entry still waiting, and waits up to 30 seconds more. New windows are matched to the launch by process (or a child of it), or by desktop file name (found the same way as at capture): single-instance and D-Bus-activated apps open their windows from a process that was already running.
+- **Default at start.** The session script runs `ft-layout start --wait 90`. That opens the profile in `FT_PROFILE` or `default_profile` (screens, then the apps once ft-floatd is up), or runs `apply --wait` if there's none. Start in profile on the Layout & profiles page sets `default_profile` (`ft-layout default NAME|none`). Plasma's session restore is turned off in the session (`ksmserverrc`: `loginMode=emptySession`).
+- **Launcher entries.** Each profile gets `~/.local/share/applications/frametop-profile-<name>.desktop` ("Frametop: Work"), written when it's saved and removed when it's deleted. They show in SteamVR's Launch a program list, the Application Launcher, and KRunner. Running one (`ft-layout open NAME`) switches to that profile if the desktop runs. Otherwise it starts the desktop with `FT_PROFILE` set (`systemd-run`, as `desktops.sh start` does), which overrides `default_profile` for that start. That needs SteamVR to be running.
+- **The action.** `profile:NAME` in the input relay (it runs `ft-layout use NAME`) for key combinations, mouse buttons, and controller buttons, with or without pointer mode. Input Settings lists one "Open profile NAME" action per profile.
+- **Display Settings.** On the Layout & profiles page, the arrangement list has the profiles, which can be renamed and deleted. Open profile and Save as profile… are the page's actions. A profile's apps are listed with where each goes and a button to leave one out, plus which screens it hides. Start in profile picks the one the desktop starts with. The Visibility tab's Screens shown switches hide screens one at a time.

docs/reference.md

@@ -18,6 +18,18 @@ desktops.sh start | stop | restart | status | log [lines]
 
 When the VR launcher starts the desktop, it inherits the Steam client's environment. The session script drops the client's runtime from it (`LD_LIBRARY_PATH`, the `STEAM_*` settings, and the Steam overlay's Vulkan layer), so apps in the desktop use the system's libraries, including its video codecs, just as they would after a normal login.
 
+The nested session also starts an AT-SPI accessibility registry through `session/ft-atspi` in Plasma's autostart. It discovers the bus from this session, ignores an inherited host accessibility address, and leaves an existing registry alone. Accessibility errors do not stop the desktop. This supplies the registry infrastructure for apps that expose AT-SPI trees; it does not enable gaze snapping or force Chromium/Electron accessibility. After an approved desktop restart, an AT-SPI-aware app should be visible on the nested bus. See [design.md](design.md#nested-accessibility) for native activation, fallback lifecycle, and the isolated test command.
+
+KWin's blur and background contrast effects and its animations are off in this desktop, because KWin draws on the headset's GPU, which SteamVR needs. The session script turns them off once, the first time it starts (it leaves a setting you already have alone, and marks it done in `~/.config/frametop/frametoprc`), so turning them back on sticks. In the Frametop desktop, System Settings → Window Management → Desktop Effects has Blur and Background Contrast, and General Behavior has Animation speed. Or from a terminal, then restart the desktop:
+
+```
+kwriteconfig6 --file ~/.config/frametop/kwinrc --group Plugins --key blurEnabled true
+kwriteconfig6 --file ~/.config/frametop/kwinrc --group Plugins --key contrastEnabled true
+kwriteconfig6 --file ~/.config/frametop/kdeglobals --group KDE --key AnimationDurationFactor 1
+```
+
+Two of the system's autostart programs don't start in this desktop: Discover's update notifier (`org.kde.discover.notifier`), which starts Discover to check for updates, and IBus (`ibus`), which can't reach the desktop's apps because KWin's input method is `input/ft-textinput`. The session script puts copies with `Hidden=true` in `~/.config/frametop/autostart` once (marked in `frametoprc`), and skips a name you already have a file for. Delete a copy to start that program again.
+
 Settings are in two files, and Frametop Display Settings edits both. The screens (resolution, width in metres, scale, curve, which one has the taskbar) and their layout are in `~/.config/frametop-layout.json`. The backend, remote desktop, and pointer settings are in `~/.config/frametop.conf`; `session/frametop.conf.example` lists every key.
 
 Restarting the desktop closes its windows. Before the unit stops, `session/keep-apps.sh` moves every program started in the desktop into a systemd scope of its own, so background work such as servers, tmux, and builds keeps running. An app that shuts down its own helper processes when its window closes will still lose them; run that kind of work outside the desktop, for example as a systemd user service.
@@ -28,12 +40,13 @@ Restarting the desktop closes its windows. Before the unit stops, `session/keep-
 
 Each KWin window is one screen. ft-screens sets its size with an `xdg_toplevel` configure and KWin resizes the output to match, live. Frames arrive as DMA-BUFs and go to SteamVR through OpenVR's `IVRIPCResourceManagerClient::ImportDmabuf`, with no copy and no size limit.
 
-Every screen is an overlay named `frametop.screen.N` with four controls:
+Every screen is an overlay named `frametop.screen.N` with five controls:
 
 - `.bar` moves the screen. Drag it with any laser or with the 3D mouse, whose right-drag tilts. Scrolling while you drag pushes the screen away or pulls it closer, along the line from your head.
 - `.curve` bends the screen into a cylinder around you, using your current distance as the radius, or makes it flat again.
 - `.roll` rolls the screen when you drag it sideways, like a knob. It snaps level within 2.5°, and scrolling on it turns 5° per notch.
 - `.resize`, the tab on the bottom right corner, sets the width. Screens go down to 15 cm wide.
+- `.reset`, left of the bar, puts every screen back in its layout around where you are now, like Meta+Shift+R (`ft-layout apply`).
 
 The controls are sized from both the screen's width and its distance from you, follow the surface of a curved screen, and stay invisible until a laser or the 3D mouse's cursor lands on one or comes within about 1.5 times a button's size of it. While invisible they're still there, fully transparent, so SteamVR's laser can find them. They're translucent until a laser is on them, like SteamVR's own window controls.
 
@@ -48,10 +61,10 @@ The Visibility & pins tab of Frametop Display Settings decides when the screens
 - While you look at a chosen controller (the wrist gesture).
 - Only after you show them with the hotkey.
 
-In the last three modes the hotkey shows the screens anyway. Two more settings on the same tab cover VR games, which ft-screens detects as SteamVR scene apps:
+In the last three modes the hotkey shows the screens anyway. A screen can also be hidden on its own (Screens shown on the same tab, or `ft-layout hide N`): it stays hidden whatever the mode or the hotkey says, until it's shown again there. Windows on it stay put, and a new window that would open on it floats instead (ft-floatd). Profiles use this to show only some screens. Two more settings on the same tab cover VR games, which ft-screens detects as SteamVR scene apps:
 
 - During VR games, the Always mode hides the screens unless the dashboard is open (the default), or leaves them up.
-- Controllers on the screens. Visible screens can keep SteamVR's laser mouse on, so controllers work them with the dashboard closed, but that also takes the controllers away from a game. By default this is off while a VR game runs, and the 3D mouse or the dashboard works the screens. The other choices are always on, or only with the dashboard open, which also suits flatscreen games since they aren't scene apps.
+- Controllers on the screens. Visible screens can keep SteamVR's laser mouse on, so controllers work them with the dashboard closed, but that also takes the controllers away from a game. By default this is off while a VR game runs, and the 3D mouse or the dashboard works the screens. Pointing a controller at a screen, a floating window, or the keyboard still turns its laser on, like SteamVR's own floating windows, and pointing away gives the game the controllers back. The other choices are always on, or only with the dashboard open, which also suits flatscreen games since they aren't scene apps.
 
 Input from the lasers reaches KWin through ft-screens' own seat. Keys come from the input relay, from pass-through keyboards and any key a pointer device passes through. Typing follows your last click: after a click on a screen it goes to the desktop, even with the SteamVR dashboard open, and after a mouse click on any other panel (the dashboard, Steam, an app like Spotify) it goes there instead. While it goes to the desktop, the relay grabs pass-through keyboards so gamescope, which reads every keyboard itself, doesn't type them into the Steam app too. A program that watches every keyboard for a hotkey loses a grabbed one; with `SHARE_KEYS=1` in `~/.config/frametop.conf`, their keys also go to `@frametop_keys` for it. That's off by default, since any local process that binds the name first would get everything typed into the desktop. Hidden screens don't take typing.
 
@@ -64,11 +77,18 @@ ft-screens listens for datagrams on the abstract socket `@ft_screens` and replie
 ```
 place N x y z yaw pitch roll     width N metres          curve N radius|on|off
 pin N|all left|right|head [matrix]    unpin N|all        size N w h
-get N    screens    head    state    key code value    scale N s    vrkeyboard show|hide|toggle
+get N    screens    head    state    key code value    scale N s    vrkeyboard show|hide|toggle|close
 visibility always|dashboard|gesture|toggle    wrist degrees    gesture left|right degrees
-hide | show | toggle    controllers always|outside_games|dashboard    ingames hide|visible
+hide | show | toggle    controllers always|outside_games|dashboard    ingames hide|visible    pause on|off|state
+conceal N|all    reveal N|all    concealed    cutouts on|off|state    cutouts predict on|off    cutouts lead ms
+float N mpp x y w h title    unfloat N    pose N matrix    sub N k x y w h | sub N k off    minimized N 0|1    carry N
+rates focused in_view hidden    rates?    watch seconds    phase ms
 ```
 
+Each screen draws at a frame rate for how much of it you see. KWin draws a screen only after ft-screens gives it a frame callback, and its apps wait for theirs, so the rate of callbacks is the screen's frame rate, for KWin and the apps on it alike. A screen is focused while you look at it (within 12 degrees of where your head points), while a laser or the mouse is on it or was in the last 1.5 seconds, while it's carried, and while you type on it; it gets every display frame. The rest of what you can see (within 60 degrees) gets 15 frames a second, and a hidden screen, one behind you, and everything while paused get one a second. A level goes up at once and comes down after a moment (1.5 s from focused, 0.5 s from in view). A video, or anything moving over a large part of a screen (6% or more of it, redrawn on 8 commits in a row, 10 or more a second), keeps every frame while in view. A floating window is a screen of its own here. Nothing that stands still costs anything at any rate: KWin sends a frame only when something on the screen changed. `rates F V H` sets the three rates in Hz (0: every display frame; default `0 15 1`, also `ft-screens --rates 0,15,1`), and `rates?` shows them, the display's rate, and for each screen its level, its milliseconds between frames, and whether it counts as a video. `watch S` gives every screen full rate for S seconds: remote desktop renews it while a VNC client is connected, since a viewer sees what KWin draws. The ticks (SteamVR events and the callbacks) come once per display frame, 1 ms after the vsync (`phase ms` changes that, for tuning), in step with the display rather than on a timer that drifted through the frame.
+
+`conceal` and `reveal` hide and show one screen on its own (`ft-layout hide` and `show` send them), and `concealed` lists those screens. `pause on` (from the input relay, when Frametop pauses for a VR game) hides every screen and floating window whatever else says, and slows the desktop down; `pause off` undoes it. `cutouts` turns the hand cutouts on and off (`ft-handsctl cutouts`). The last line is ft-floatd's, for floating windows: N is a floating window's panel, numbered on from the screens, one per spare output. `float` gives the window's rectangle in its output, metres per pixel, and the title bar's height, and shows the panel; `unfloat` hides it. `pose` places it (a 3x4 matrix, standing universe), `sub` shows popup or dialog k over it, `minimized` hides it while its window is minimized, and `carry` moves it with the laser that pressed the window's own title bar.
+
 ## Input relay
 
 SteamVR opens input devices only when it starts. A Bluetooth mouse that sleeps and reconnects gets new device nodes, SteamVR keeps reading the dead ones, and the mouse stops working until SteamVR restarts. `input/input-relay.py` avoids this. It creates two virtual devices, `frametop virtual mouse` and `frametop virtual keyboard`, through `/dev/uinput` before SteamVR starts. It then grabs USB and Bluetooth mice and keyboards as they come and go and forwards their events, so SteamVR only ever sees the virtual devices, which never go away.
@@ -81,6 +101,8 @@ The relay also owns the volume keys, on every device that has them, the headset'
 desktops.sh relay install     # enable it (starts with the next reboot or SteamVR start)
 desktops.sh relay status | log | uninstall
 input/input-relay.py --no-grab   # try it without taking devices from SteamVR
+input/test/keys-test.py          # key combinations and modifier taps, against fake devices (safe next to the live relay)
+steam/ft-steam menu              # what Open Steam menu does; ft-steam check: Steam's UI still has the calls
 ```
 
 The first time, the relay has to start before SteamVR, so reboot or restart SteamVR after installing it. After that it's safe to restart on its own: systemd keeps the virtual devices open in its file descriptor store (`FileDescriptorStorePreserve=yes`), so SteamVR keeps the same devices.
@@ -95,7 +117,7 @@ A mouse drives SteamVR the way a controller's laser does, but shows up as a smal
 
 Whichever device you used last wins. Picking up a controller hands the laser back at once, and moving the mouse takes it again. When the headset comes off, the pointer lets go, so the displays can sleep, and it stays off until you're wearing the headset again.
 
-To move a floating panel, left-drag its grab bar. The scroll wheel pushes and pulls it while you drag. Hold the right button while dragging and move the mouse to tilt the panel around the grab point; the right press isn't sent as a click. The tilt stays for the rest of the drag, and releasing the left button drops the panel as it is. A mapped Toggle dashboard button (or a Meta tap, with `META_DASHBOARD=1`) wakes the pointer if needed and holds the virtual system button for 0.12 s, because SteamVR ignores a press and release in the same instant.
+To move a floating panel, left-drag its grab bar. The scroll wheel pushes and pulls it while you drag. Hold the right button while dragging and move the mouse to tilt the panel around the grab point; the right press isn't sent as a click. The tilt stays for the rest of the drag, and releasing the left button drops the panel as it is. A mapped Toggle dashboard button wakes the pointer if needed and holds the virtual system button for 0.12 s, because SteamVR ignores a press and release in the same instant. Open Steam menu / close dashboard (`steam_menu`) needs no pointer: `steam/ft-steam menu` asks Steam's UI, over its debugging port (`steam/steamui.py`), to show its dashboard overlay and focus the Steam frame's menu, or to hide the dashboard if it's up.
 
 ```
 pointer/driver/build.sh && pointer/driver/install.sh install   # then restart SteamVR
@@ -104,33 +126,34 @@ pointer/helper/run.sh status | log | restart
 pointer/driver/install.sh probe     # devices, hand roles, who owns the dashboard pointer
 ```
 
-The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_CONTROLLER_PICKUP`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, `POINTER_IGNORE`, the head follow settings `POINTER_FOLLOW`, `POINTER_LEASH_DEG`, `POINTER_LEASH_DELAY`, `POINTER_LEASH_RETURN`, and `POINTER_FOLLOW_REACH`, and the gaze mode settings `POINTER_GAZE`, `POINTER_GAZE_RETAKE`, `POINTER_GAZE_NUDGE_MAX`, `POINTER_GAZE_HOLD`, and `POINTER_GAZE_SHOW`, and the gaze service's `GAZE_TRACKER` (SteamVR's eye tracker or our own) and `GAZE_EYE` (the eye bias). The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper (the gaze service reads its two again when the file changes).
+The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_CONTROLLER_PICKUP`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, `POINTER_IGNORE`, the head follow settings `POINTER_FOLLOW`, `POINTER_LEASH_DEG`, `POINTER_LEASH_DELAY`, `POINTER_LEASH_RETURN`, and `POINTER_FOLLOW_REACH`, and the gaze mode settings `POINTER_GAZE`, `POINTER_GAZE_RETAKE`, `POINTER_GAZE_NUDGE_MAX`, `POINTER_GAZE_HOLD`, `POINTER_GAZE_DOT`, `POINTER_GAZE_SHOW`, `POINTER_GAZE_MOUSE`, `POINTER_GAZE_MOUSE_MOVE`, and the keyboard clicks' `POINTER_HEAD_DEADZONE` and `POINTER_KEY_TAP`, and the gaze service's `GAZE_TRACKER` (`auto`, the default: our own eye tracker when it's installed, else SteamVR's; or `own` or `steam`) and `GAZE_EYE` (the eye bias). The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper (the gaze service reads its two again when the file changes).
 
 ## Frametop Input Settings
 
-A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has eight pages:
+A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has nine pages:
 
-- Devices lists every USB and Bluetooth mouse and keyboard, with a light that flashes when the device is used. Each device gets a role: 3D pointer (grabbed, drives the pointer; the default for anything with a mouse), Pass through (grabbed only while typing goes to the desktop; the default for keyboards, where a Meta tap toggles the dashboard if `META_DASHBOARD=1` is in `~/.config/frametop.conf`), or Ignore. A device is identified by its Bluetooth address, or its USB ids and name, so all of its input nodes share one role. Forget drops everything saved for a device.
-- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, open or close the keyboard, head follow on or off, gaze pointer on or off, faster or slower, pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
-- Controllers maps the Frame controllers' buttons (every button but the system button) to the same actions, except passing a key through. Capture a button and press it on a controller, or pick it from the list. The controllers aren't input devices on the host; only SteamVR sees them. So the pointer helper reads them with SteamVR input (`pointer/helper/vrbuttons.h`, `pointer/helper/actions/`) and sends presses to the relay (`vrbtn right/a 1`), which does the mapped action. The helper only takes the buttons that are mapped (the relay tells it with `vrbind`), at an overlay-global priority, and only while no game (scene application) runs, so games keep every button; with In games on (`controller_in_games`), a mapped button is taken from games too. That needs SteamVR's "Enable global input from overlays (Experimental)" setting (`steamvr/globalActionSetPriority`), which the page's Global input switch turns on and off. Mappings are saved as `controller_buttons` in `~/.config/frametop-input.json`.
-- Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput, like frame-voice's, don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected.
+- Devices lists every USB and Bluetooth mouse and keyboard, with a light that flashes when the device is used. Each device gets a role: 3D pointer (grabbed, drives the pointer; the default for anything with a mouse), Pass through (grabbed only while typing goes to the desktop; the default for keyboards, whose key combinations work everywhere), or Ignore. A device is identified by its Bluetooth address, or its USB ids and name, so all of its input nodes share one role. Forget drops everything saved for a device.
+- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, gaze pointer on or off, gaze precision, gaze drag, gaze quick check, faster or slower, reset the screen layout, hide or show the screens, open or close the keyboard, float a window in VR or put it back, put all floating windows back, pause or resume Frametop ([Pausing for VR games](#pausing-for-vr-games)), Open profile NAME (one per profile, [profiles.md](profiles.md)), pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
+- Controllers maps the Frame controllers' buttons (every button but the system button) to the same actions, except passing a key through and the gaze actions: gaze mode is a mouse and keyboard feature ([gaze-controllers.md](gaze-controllers.md)). Capture a button and press it on a controller, or pick it from the list. The controllers aren't input devices on the host; only SteamVR sees them. So the pointer helper reads them with SteamVR input (`pointer/helper/vrbuttons.h`, `pointer/helper/actions/`) and sends presses to the relay (`vrbtn right/a 1`), which does the mapped action. The helper only takes the buttons that are mapped (the relay tells it with `vrbind`), at an overlay-global priority, and only while no game (scene application) runs, so games keep every button; with In games on (`controller_in_games`), a mapped button is taken from games too. That needs SteamVR's "Enable global input from overlays (Experimental)" setting (`steamvr/globalActionSetPriority`), which the page's Global input switch turns on and off. Mappings are saved as `controller_buttons` in `~/.config/frametop-input.json`.
+- Game optimization has the pause for VR games: its state with Pause now or Resume, whether VR games pause Frametop by themselves, the controller gesture (one or two buttons, pressed once or twice; one button always takes two presses), what happens to the desktop, and the sound. They're saved as `pause_auto`, `pause_gesture`, `pause_desktop`, and `pause_sound` in `~/.config/frametop-input.json`. See [Pausing for VR games](#pausing-for-vr-games).
+- Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput, like frame-voice's, don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected. Its Key combinations section maps modifiers plus a key, or one modifier tapped on its own, on any keyboard, to any action but passing a key through or nothing, or to Run a command…: a command line the input relay runs with `sh -c` when you press the keys (`command:CMD`). The command runs as the relay's user service, outside the desktop's session, with `layout/`, `float/` and `steam/` on its `PATH` (so `ft-layout use Work` or `ft-float launch org.kde.dolphin` work as they are), and its output goes to the relay's journal. The gaze clicks (Gaze left click and Gaze right click) only go on key combinations. The defaults are a Meta tap (open the Steam menu, or close the dashboard), Meta+J (gaze left click), Meta+K (gaze right click), Meta+Shift+F (float window in VR or put it back), and Meta+Alt+Tab and Meta+Alt+Shift+Tab (spin the panels: every screen and floating window turns about your head, so the next one on the right or left comes to the front; ft-screens' `spin next|prev|<degrees>`); remove them or add others there. A tap is a press and release with no other key, mouse button, or scroll in between; a bound one sends the desktop F24 before the release, so Plasma's launcher doesn't open on it. The combination's last key isn't typed, and the modifiers still reach the app; while typing goes to Steam rather than the desktop, keyboards aren't grabbed, so Steam or the game sees the keys too. They're saved as `key_bindings` in the same file; a file with its own list, even an empty one, gets no defaults.
 - Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button.
 - Ignored panels lists the SteamVR overlays that are showing, grouped by app (the first two parts of the overlay key, such as `sasaken.frame-perf-overlay`), from the pointer helper (`overlays`). Tick a panel, or Ignore the whole app, and the pointer passes through it to what's behind. It's for panels you only look at, like a performance overlay that follows your view. The list is saved as `POINTER_IGNORE` in `~/.config/frametop.conf`: comma-separated overlay keys, where a shell pattern like `vendor.app*` covers a whole app, including panels it opens later. The helper reloads at once. Frametop's own screens aren't listed, and entries for apps that aren't open are listed below, to remove.
-- Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), the gaze mode sliders, the gaze service's state (headset, samples per second, how often the tracker is losing each eye, the calibration, the nudges learned), and Calibrate (opens the gaze probe), Check headset fit (opens the probe's Headset fit mode), Reload calibration, and Forget nudges.
+- Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), what the mouse's left button and movement do, the gaze dot, the eye tracker and eye bias, the gaze mode sliders, the gaze service's state (headset, samples per second, how often the tracker is losing each eye, the calibration, the nudges learned), and Quick check, Calibrate, and Check headset fit (each in a panel in the headset), Reload calibration, and Forget nudges. The gaze probe, a development tool, is in the page's overflow menu.
 - Bluetooth lists paired devices and has Apply Bluetooth fixes, which runs `/etc/steamframe/bt-fixups.sh` through `pkexec`. Pair new devices in Steam.
 
 Device rules are saved in `~/.config/frametop-input.json`. `input-settings/install.sh` installs the menu entry. Its launcher hands podman the real `XDG_RUNTIME_DIR` and user bus and gives the app the session's Wayland socket, because the desktop session runs on a private D-Bus and podman fails on it.
 
 ## Frametop Display Settings and ft-layout
 
-When the desktop starts, its screens arrange themselves around where you're facing. You can move them by hand at any time and put them back with Meta+Shift+R, the Reset Screen Layout menu entry, Arrange now in the app, or a mouse button mapped to Reset desktop screen layout.
+When the desktop starts, its screens arrange themselves around where you're facing. You can move them by hand at any time and put them back with Meta+Shift+R, the reset button left of any screen's bar, the Reset Screen Layout menu entry, Arrange now in the app, or a mouse button mapped to Reset desktop screen layout.
 
 The desktop's own screen arrangement follows where the screens are around you, whatever their numbers: a screen you see to the left of another is to its left in Plasma too, so the pointer and dragged windows cross straight to it. Screens one above the other stack, and screens pinned to a wrist or your head come last. It's updated at startup, after arranging or saving the layout, and half a second after you let go of a screen you moved. With the headset off there's no head pose to go by, and the arrangement stays as it was.
 
 Frametop Display Settings has four tabs (three with the gamescope backend, which has no Visibility & pins):
 
 - Screens: add and remove screens, and set each one's resolution (presets from 1080p to 4K, ultrawide, super ultrawide, portrait, or custom), its width in VR (0.5 to 6 m), its scale, whether it's curved, and whether it has the taskbar. Resolution, width, and curve apply at once. Adding or removing a screen takes a desktop restart, which the app offers.
-- Layout: a curve around you, with the screens hinged edge to edge like monitors on a desk and each turned to face you, or a flat wall. Both take rows, distance, gap, and height. Save current arrangement saves the positions, sizes, curves, and pins you set by hand under a name instead. Named layouts are listed with the presets: pick one and Arrange now to switch to it, and rename or delete it with the buttons next to the list. A layout saved with fewer screens than you have now leaves the others where they were saved last, or where the preset would put them. A preview shows the layout from above and from the front, and a switch turns auto-arrange at startup on or off.
+- Layout (the Layout & profiles page): the Arrangement list starts with two presets: Curved around you, with the screens hinged edge to edge like monitors on a desk and each turned to face you, and Flat wall. Both take rows, distance, gap, and height, and Arrange now applies them. Save as profile… saves where the screens are now (positions, sizes, curves, and pins), which ones are hidden, and the open apps and where their windows are, under a name. Profiles are listed in Arrangement after the presets: pick one and Open profile switches to it, and the buttons next to the list rename or delete it. Below it, Apps lists the profile's windows and where they go, and Leave out drops one. A profile saved with fewer screens than you have now leaves the others where they were saved last, or where the preset would put them. Start in profile picks the profile the desktop starts in; with None, a switch turns auto-arrange at startup on or off. A preview shows the layout from above and from the front. See [profiles.md](profiles.md).
 - Visibility & pins: the visibility, game, and controller settings described above, the wrist angle, where each screen is pinned (in the room, a wrist, or your head), and buttons to pin all screens or unpin them.
 - Power: when the displays turn off while the headset isn't used, their state now, Turn displays off now (to try it), and Stay awake while plugged in. See [Displays off and sleep](#displays-off-and-sleep).
 
@@ -139,18 +162,39 @@ Frametop Display Settings has four tabs (three with the gamescope backend, which
 ```
 layout/ft-layout apply      # arrange every screen
 layout/ft-layout capture    # save the current arrangement and sizes as the layout
-layout/ft-layout save NAME  # ...under a name too, and use it
-layout/ft-layout use NAME   # switch to a named layout and arrange the screens in it
+layout/ft-layout save NAME  # ...under a name too, with the open apps and hidden screens (a profile, docs/profiles.md), and use it
+layout/ft-layout use NAME   # switch to a profile: arrange the screens in it and open its apps
+layout/ft-layout open NAME  # a profile's launcher entry: use it, or start the desktop in it
+layout/ft-layout default NAME|none  # the profile the desktop starts with (start --wait runs it at desktop start)
 layout/ft-layout layouts    # list the named layouts (* = in use); rename OLD NEW, delete NAME
 layout/ft-layout pin N|all left|right|head   # pin as they are now; unpin N|all
 layout/ft-layout plan       # print the arrangement as JSON (no VR needed)
 layout/ft-layout scale      # per-screen scale, positions (as the screens are around you), and taskbar screen, to KWin
 layout/ft-layout toggle     # hide or show all screens
+layout/ft-layout hide N|all # hide a screen on its own, whatever the visibility mode; show N|all brings it back, hidden lists them
 display-settings/install.sh # menu entries and the Meta+Shift+R and Meta+Shift+H shortcuts
 ```
 
 The layout is stored relative to your head when it's applied. `/tmp/frametop-layout.log` has the run from the last desktop start.
 
+## Floating windows
+
+A desktop window can float in VR as a panel of its own, away from the screens. Meta+Shift+F floats the window under the pointer (or the active one, over the wallpaper), or puts it back on its screen if it floats. So do Float in VR in every window's menu (Alt+F3; Back to Desktop on a floating one), the button left of Close in its title bar, and a mouse button, controller button, or key combination mapped to Float window in VR in Frametop Input Settings; Put all floating windows back is mappable too. Launch as Standalone, in an app's right-click menu in the Application Launcher or the taskbar, starts the app with its first window floating, where that app last floated or in front of you. [floating-windows.md](floating-windows.md) explains how it works.
+
+`float/ft-floatd` does this. It runs inside the desktop's Plasma session (log: `/tmp/frametop-floatd.log`), loads the KWin script `float/frametop-float.js`, and moves each floating window to a spare KWin output of its own, which ft-screens shows as a panel cropped to the window. The settings are in `~/.config/frametop.conf`: `FLOAT_SLOTS` is how many windows can float at once (8, at most 16; 0 turns floating off; restart the desktop after a change), and `FLOAT_MARGIN` the pixels around each window on its output, so menus have room past its edges (300). Each app's last floating place, size, and scale are kept in `~/.config/frametop-float.json` by desktop file name, relative to the primary screen, so they move with the screens. `FT_FLOAT_DEBUG=1` in ft-floatd's environment logs every event from the KWin script.
+
+With floating on, the session gives the desktop's windows Frametop's own decoration (`decoration/`): Breeze's look plus the float button. Apps that draw their own title bar, like Chromium and Electron apps, don't have the button. `decoration/apply.sh` puts a changed copy into the running desktop, and `decoration/apply.sh --off` goes back to Breeze until the next desktop start.
+
+```
+float/ft-float float [ID|active]      # float a window (default: the active one, as a toggle)
+float/ft-float float pointer          # the float key: the window under the pointer, floated or put back
+float/ft-float dock [ID|active|all]   # put a floating window back (default: the active one), or all of them
+float/ft-float launch org.kde.dolphin # start an app (its desktop file name) floating
+float/ft-float run COMMAND [ARG...]   # the same for a command
+float/ft-float close ID               # close a window
+float/ft-float list                   # the spare outputs and what floats on them
+```
+
 ## Displays off and sleep
 
 SteamVR turns the displays off a few seconds after the headset's proximity sensor says it came off. A stand or display mount that covers the sensor makes the headset seem worn, so its displays stay on, and Steam, which then counts someone as present, never puts it to sleep either.
@@ -172,14 +216,54 @@ power/run.sh off | on      # the displays off now, or back on
 power/run.sh log
 ```
 
-## Hand tracking (experimental)
+## Pausing for VR games
 
-Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen. The same tracker also detects pinches, for clicking where you look with the gaze pointer (not wired to the pointer yet). It's optional: `hands/run.sh install`, or the last step of `install.sh`.
+Paused, Frametop leaves the headset's CPU and GPU to a VR game. The input relay does it (`input/game_pause.py`), since it's the one part that always runs:
+
+- The gaze service stops (`frametop-gaze`: ft-gazed, ft-gaze, our own eye tracker, the gaze panel), so nothing reads SteamVR's eye tracking. Our frame grabber, the root service `ft-eyegrab`, goes idle by itself 3 seconds after our eye tracker stops asking it for frames.
+- Hand tracking stops if it runs (`frametop-camd`, `frametop-hands`).
+- The desktop, as the Game optimization page of Frametop Input Settings says (`pause_desktop`): hidden (the default) or closed. Hidden, ft-screens hides every screen and floating window whatever the visibility mode, the hotkey, or the dashboard says, and gives KWin a frame callback once a second instead of every display frame. KWin draws a screen only after its frame callback, and its apps wait for theirs, so the desktop hardly draws, but its windows stay open. Remote desktop stops if it runs (`session/remote-ctl.sh`). Closed, `desktops.sh stop` closes the desktop and its windows, and resuming starts it again (about 12 seconds), in its start profile if it has one.
+- The relay lets go of the 3D mouse and feeds pointer devices to its virtual mouse and keyboard, as with `POINTER=0`. Typing goes to Steam. Mapped buttons and key combinations do nothing but pausing, the Steam menu, and commands; a key combination that does nothing is typed as usual.
+
+Resuming starts again only what pausing stopped, and plays a second sound. The pointer helper and ft-powerd keep running: they cost little, the helper is what says a game started, and stopping it would leave its virtual controller connected with its last pose.
+
+Ways to pause and resume:
+
+- The controller gesture, by default both thumbsticks clicked together twice: both go down within 0.3 seconds of each other, and the second time within 0.7 seconds of the first. The relay reads it from vrserver's web socket (`input/vrws.py`), which works whatever has input focus and takes nothing from the game, so the game sees the clicks too. The Game optimization page changes it: one or two of the buttons the Controllers page lists, pressed once or twice (one button always takes two), or none.
+- The Pause/resume Frametop action, on a mouse button, a key combination, or a controller button (outside games, like every mapped controller button).
+- VR games, with Pause while a VR game runs on (`pause_auto`, the default). The pointer helper tells the relay when a scene app starts and ends (`vrgame 1|0`, repeated every 5 seconds). A game starting pauses Frametop. A pause that starts while a game runs ends 5 seconds after the game does, unless another game starts first. Resumed during a game, Frametop stays on until that game ends. A pause that starts outside a game lasts until you resume. Flatscreen games aren't scene apps, so they don't pause it.
+- From a terminal or a script:
+
+```
+input/ft-pause on | off | toggle   # pause or resume
+input/ft-pause status              # the state as JSON (the relay's "pause ?")
+input/vrws.py 10                   # the controllers' buttons from vrserver's web socket, for 10 s
+input/test/pause-test.py           # the gesture and the automatic pause, offline
+```
+
+The state outlives a relay restart, in `/run/user/UID/frametop-pause.json`. A SteamVR restart while paused starts the gaze service with it, and the relay stops it again when the pointer helper comes back.
+
+## Gaze pointer (experimental)
+
+In gaze mode the 3D mouse's pointer goes where you look, and the mouse or the keyboard does the last bit. It needs the gaze service, which `install.sh` offers (yes by default) and `gaze/run.sh install` installs on its own: it builds it and runs `gaze/ft-gazed` as `frametop-gaze.service`, which starts with SteamVR. The service idles while the gaze isn't used: its eye tracker reader and our own eye tracker run only while gaze mode is on and someone wears the headset, while a check or the calibration runs, or while the Gaze page of Frametop Input Settings is open, and stop 30 seconds after ([gaze/README.md](../gaze/README.md)). Turn gaze mode on with the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1`, or a button or key combination mapped to Gaze pointer on/off.
+
+- Meta+J left-clicks and Meta+K right-clicks where you look. A quick tap clicks where the dot was at the press. Hold instead, and the dot stays put in your view: turn your head until it's on what you meant, and let go to click there. Held still for `POINTER_GAZE_HOLD` (0.5 s), the press becomes a real one, and your head drags. Meta+K with Meta+J held presses where the dot is now, to drag from there, and a second Meta+K during that drag (a double Meta+K) pans and tilts what you're dragging while it's held.
+- The mouse's buttons work the same way, with the mouse steering instead of your head (`POINTER_GAZE_MOUSE=precision`, the default): the right button with the left held starts a drag, and a double right click pans and tilts what you're dragging. With `POINTER_GAZE_MOUSE_MOVE=held`, the default, the mouse only corrects: while the gaze has the pointer, moving it does nothing unless a button is held. `free` lets the mouse take the pointer any time. With the gaze stale for a second, in a game, or with the headset off, the mouse works as usual.
+- A correction before a click teaches the gaze service the tracker's error there. A correction bigger than `POINTER_GAZE_NUDGE_MAX` (55 degrees) isn't learned; it opens a quick check instead.
+- Calibration and checks run in a panel fixed to the headset (`gaze/panel/ft-gazepanel`, which the gaze service runs), from the Gaze page: Quick check is one dot, and also opens when you put the headset on. Calibrate is three rounds of dots, dark to bright; look at each dot and left click or press Meta+J to take it. Check headset fit shows, live, how well the tracker sees each eye. A right click or Meta+K closes the panel. Gaze mode on without a calibration opens Calibrate by itself, as soon as your eyes are seen. If gaze mode is on but can't follow your eyes yet (no calibration, the calibration can't open, the gaze service not running), the Gaze page says why under the Gaze pointer switch, and `gaze/ft-gazectl on` notes it.
+
+[gaze/README.md](../gaze/README.md) has the details, our own eye tracker, and the gaze probe, a development tool.
+
+## Hand tracking (experimental, deferred)
+
+Deferred: it costs a lot of the headset's CPU and needs more work, so `install.sh` doesn't offer it. It still builds and runs, installed by hand, for working on it.
+
+Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen. The same tracker detects pinches and grips, and with `POINTER_HANDS=1` in `~/.config/frametop.conf` they work the pointer. In gaze mode a pinch clicks where you look when it opens; hold it and move the hand to correct the pointer first. Without gaze mode a pinch is a press like the mouse's button, so a held pinch drags. A grip (closing the hand) presses and drags. To install it: `hands/run.sh install`.
 
 - `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`.
-- `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches) next to the ring.
-- Both start and stop with SteamVR. `hands/run.sh status` and `hands/run.sh log` show how they're doing.
-- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (after some SteamVR restarts the side cameras' names come out swapped, and hands land beside the holes; `hands/tools/check_sides.py --ring` tells) and `HANDS_CPUS`.
+- `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches and grips, read by the pointer helper) next to the ring.
+- The install leaves both off, and they don't start with SteamVR. `ft-handsctl on` starts them while SteamVR runs, and `ft-handsctl off` stops them; they also stop with SteamVR. The install links `ft-handsctl` into `~/.local/bin`. `ft-handsctl status` and `ft-handsctl log` (or `hands/run.sh status` and `log`) show how they're doing, `ft-handsctl cutouts on|off` turns just the cutouts off, and `ft-handsctl gestures` shows pinches and grips live.
+- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (`auto`, the default: ft-hands tells from the hands when some SteamVR restart has swapped the side cameras' names, and fixes them; `0` or `1` force them, and `hands/tools/check_sides.py --ring` tells which is right), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, and `POINTER_PINCH_TYPING`. The example config explains each.
 
 Details, options, and the recording and replay tools are in [hands/README.md](../hands/README.md).
 
@@ -191,6 +275,8 @@ It listens on port 5900 on the Frame's Tailscale address only, not the LAN, so i
 
 No VNC server can capture KWin on SteamOS directly: `krfb` needs `xdg-desktop-portal-kde`, which SteamOS doesn't ship, and `wayvnc` only works with wlroots compositors. So `session/remote-desktop.sh` captures the desktop with KDE's `krdpserver --plasma` on `127.0.0.1:3390`, and `session/vnc-bridge.sh` runs TigerVNC's `Xvnc` on display `:20` with a FreeRDP client inside it and serves that. Both run in the `dev` container, and the extra hop adds a little latency. krdp streams every screen; the VNC screen is the primary's size, and the FreeRDP window is shifted so the primary fills it (`ft-layout remote-view` gives the offset). krdp's own `--monitor` would stream just one screen, but it maps the pointer as if that screen sat at 0,0, so clicks would miss. When the layout changes, the VNC screen resizes and FreeRDP reconnects within a few seconds.
 
+FreeRDP runs only while a VNC viewer is connected, because while it's connected krdp captures and encodes every redraw. With no viewer, krdp has no RDP connection and so captures nothing, and Xvnc shows a black screen. When a viewer connects, the bridge starts FreeRDP, and the desktop appears about 3 seconds later; FreeRDP stops 45 seconds after the last viewer leaves (`VNC_IDLE_SEC` in the bridge's environment). The bridge looks for viewers with `ss` whenever Xvnc logs something, as it does for every connection, and every 5 seconds otherwise. While a viewer is connected, the bridge asks ft-screens to draw every screen at full rate (`watch 15` on `@ft_screens`, renewed every 5 seconds), so screens you aren't looking at in the headset, or a headset on a stand, don't stream at a low rate. It reads the primary screen's place again (`ft-layout remote-view`) only while FreeRDP runs, after `~/.config/frametop/kwinoutputconfig.json` or `~/.config/frametop-layout.json` changes, and once a minute.
+
 With remote access on, the nested KWin runs with `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1` and `KWIN_SCREENSHOT_NO_PERMISSION_CHECKS=1`, so any app in the Frametop desktop could capture its screens or inject input. The second one lets scripts take screenshots through KWin's `org.kde.KWin.ScreenShot2` D-Bus interface. This applies only to that desktop, not the stock one. Port 3389 is SteamOS's own `xrdp`, which starts a separate X11 session rather than showing the VR desktop.
 
 ## Limits

float/frametop-float.js

@@ -2,7 +2,7 @@
 // loads it into the desktop's KWin over D-Bus (org.kde.kwin.Scripting) and talks to it:
 //   - events go to ft-floatd as JSON strings (org.frametop.Float.Event), for the windows it
 //     cares about: floating windows (the ones on a spare output, WL-<screens> and up), their
-//     popups and dialogs, new windows, and "Float in VR" requests;
+//     popups and dialogs, new windows, and requests to float or dock one;
 //   - commands come back through a long poll: the script calls NextCommand, ft-floatd
 //     answers when it has one (or after a while with nothing), and the script calls again.
 // KWin scripts can call D-Bus but can't serve it, hence the poll. Window ids are KWin's
@@ -12,6 +12,11 @@ const SERVICE = "org.frametop.Float", PATH = "/Float", IFACE = "org.frametop.Flo
 let screens = 0;       // outputs WL-0 .. WL-<screens - 1> are screens; the rest are spares
 let polling = false;
 const watched = {};    // id -> true once its signals are connected
+let marking = false;   // the script itself is setting keep-below (see mark)
+const settled = {};    // id -> {output, frame, fullScreen, maximized}: where a window belongs (see putBack)
+const held = {};       // id -> {w, h, until, asked}: a size asked for, for a second (see hold)
+const moved = {};      // id -> true, or "back" once put back: it moved while KWin changed the outputs
+let layout = "", layoutOutputs = {}, layoutSince = 0;  // the outputs at the last screensChanged
 
 function send(ev) {
     callDBus(SERVICE, PATH, IFACE, "Event", JSON.stringify(ev));
@@ -47,29 +52,203 @@ function info(w) {
         frame: rect(w.frameGeometry), client: rect(w.clientGeometry), popup: w.popupWindow,
         transient: w.transient, parent: w.transientFor ? String(w.transientFor.internalId) : "",
         normal: w.normalWindow, dialog: w.dialog, fullScreen: w.fullScreen, minimized: w.minimized,
-        onAllDesktops: w.onAllDesktops
+        onAllDesktops: w.onAllDesktops, maximized: isMaximized(w)
     };
 }
 
+// KWin 6.2's scripts have no maximize mode to read: a window is maximized when it fills its
+// output's maximize area.
+function isMaximized(w) {
+    if (!w.normalWindow || !w.output) return false;
+    const a = workspace.clientArea(KWin.MaximizeArea, w), g = w.frameGeometry;
+    return g.x === a.x && g.y === a.y && g.width === a.width && g.height === a.height;
+}
+
 function report(type, w) {
+    if (w.deleted) return;  // a window on its way out still changes output and size
     const ev = info(w);
     ev.ev = type;
     send(ev);
 }
 
+// KWin's placement memory (its PlacementTracker) keeps each window's geometry for each layout of
+// the outputs (every enabled output's name and geometry), and when the outputs come back to a
+// layout it has seen, it puts the windows back where they were in it. That's for plugging monitors
+// in and out, and it does harm here. A spare output changes size after its window does, so what
+// KWin keeps for a spare's size is the window's next size: resizing a floating window back to a
+// size it had set off an endless flip between two sizes. And floating or docking one window could
+// move others, even onto a spare or off one. So the script keeps where each window belongs
+// (settled), tells ft-floatd nothing while KWin changes the outputs, and once KWin is done
+// (screensChanged comes after its restore) puts the floating windows back, and the screens' windows
+// too when only spares changed.
+function outputsNow() {
+    const all = workspace.screens, out = {};
+    for (let i = 0; i < all.length; ++i) {
+        const g = all[i].geometry;
+        out[all[i].name] = g.x + "," + g.y + " " + g.width + "x" + g.height;
+    }
+    return out;
+}
+function keyOf(outputs) {
+    return Object.keys(outputs).sort().map(n => n + "=" + outputs[n]).join(" ");
+}
+function takeLayout() {
+    layoutOutputs = outputsNow();
+    layout = keyOf(layoutOutputs);
+    layoutSince = 0;
+}
+// KWin is changing the outputs: they differ from the last screensChanged.
+function changingOutputs() {
+    if (keyOf(outputsNow()) === layout) {
+        layoutSince = 0;
+        return false;
+    }
+    if (!layoutSince) {
+        layoutSince = Date.now();
+    } else if (Date.now() - layoutSince > 2000) {
+        takeLayout();  // screensChanged should have come by now: don't stay quiet for good
+        return false;
+    }
+    return true;
+}
+function settle(w) {
+    if (w.output) {
+        settled[String(w.internalId)] = {output: w.output.name, frame: rect(w.frameGeometry), fullScreen: w.fullScreen};
+    }
+}
+// ft-floatd put the window here: it's where it belongs now.
+function expect(w, output, c, fullScreen) {
+    settled[String(w.internalId)] = {output: output, frame: {x: c.x, y: c.y, w: c.w, h: c.h}, fullScreen: fullScreen};
+    hold(w, c.w, c.h);
+}
+
+// A size asked for (by ft-floatd, or by the script putting a window back) comes in when the app
+// answers, and until then the app can still answer older requests: one KWin's restore made, or,
+// just after it opened, its own. For a second, the script asks again instead of taking those;
+// then it takes the size the window has (an app can refuse a size, below its minimum).
+const holdTimer = new QTimer();
+holdTimer.singleShot = true;
+holdTimer.timeout.connect(() => {
+    const now = Date.now();
+    Object.keys(held).forEach(id => {
+        const h = held[id];
+        if (h.until > now) return;
+        delete held[id];
+        const w = byId(id);
+        if (!h.asked || !w || w.deleted) return;  // (nothing held back: nothing to tell)
+        settle(w);
+        if (isSpare(w.output)) report("geometry", w);
+    });
+    if (Object.keys(held).length) holdTimer.start();
+});
+function hold(w, width, height) {
+    held[String(w.internalId)] = {w: width, h: height, until: Date.now() + 1000};
+    holdTimer.interval = 1100;
+    holdTimer.start();
+}
+// A size change while a size is held: true when it isn't that size (the script asked again).
+// (ft-floatd's sizes can be fractional, the window's are whole: within a pixel is the same.)
+function holding(w) {
+    const id = String(w.internalId), h = held[id], s = settled[id];
+    if (!h) return false;
+    const g = w.frameGeometry;
+    if (!s || h.until < Date.now() || w.move || w.resize || (Math.abs(g.width - h.w) < 1 && Math.abs(g.height - h.h) < 1)) {
+        delete held[id];
+        return false;
+    }
+    w.frameGeometry = {x: s.frame.x, y: s.frame.y, width: h.w, height: h.h};
+    h.asked = true;
+    return true;
+}
+
+// Runs while KWin's output change still counts as going on (nothing reported), so the steps on
+// the way don't reach ft-floatd: told only where the window ends up (see reportMoved).
+function putBack(w, screensChanged) {
+    const id = String(w.internalId), s = settled[id];
+    if (w.deleted || !s || screens === 0 || !marked(w)) return;
+    const o = outputByName(s.output);
+    // KWin's restore sets full screen (and maximized) as it was in that layout too: with a
+    // floating window that flipped forever, its output changing size with it. Ask for the
+    // state it had: KWin's request hasn't reached the app yet, so it never sees it.
+    w.fullScreen = s.fullScreen;
+    if (o && isSpare(o) && !s.fullScreen) w.setMaximize(false, false);
+    // Not where KWin had to move it: its output went, a screen changed, or it's full screen or
+    // maximized (KWin fits those to their output).
+    const back = o && !s.fullScreen && !w.fullScreen && !w.move && !w.resize && !s.maximized
+        && (isSpare(o) || (!screensChanged && !isMaximized(w)));
+    if (!back) return;
+    if (!w.output || w.output.name !== s.output) workspace.sendClientToScreen(w, o);
+    w.frameGeometry = {x: s.frame.x, y: s.frame.y, width: s.frame.w, height: s.frame.h};
+    if (isSpare(o)) hold(w, s.frame.w, s.frame.h);
+    // Moved during the change (by KWin, by the lines above, or the size ft-floatd asked for came
+    // in): report where it is, and keep settled as it is.
+    if (moved[id]) moved[id] = "back";
+}
+// After an output change: tell ft-floatd where the windows that moved during it are now.
+function reportMoved(w) {
+    const id = String(w.internalId), s = settled[id], how = moved[id];
+    if (!how) return;
+    delete moved[id];
+    if (w.deleted) return;
+    if (how !== "back") settle(w);
+    if (!w.output || !s || w.output.name !== s.output) {
+        report("output", w);
+        mark(w);
+    } else if (isSpare(w.output)) {
+        report("geometry", w);
+    }
+}
+workspace.screensChanged.connect(() => {
+    const before = layoutOutputs, now = outputsNow();
+    if (keyOf(now) === layout) return;
+    let screensChanged = screens === 0;
+    Object.keys(Object.assign({}, before, now)).forEach(name => {
+        const m = /^WL-(\d+)$/.exec(name);
+        if (before[name] !== now[name] && !(m && parseInt(m[1]) >= screens)) screensChanged = true;
+    });
+    const all = workspace.windowList();
+    all.forEach(w => putBack(w, screensChanged));
+    takeLayout();
+    all.forEach(reportMoved);
+});
+
 // Floating windows, and popups and dialogs on a spare output: tell ft-floatd about changes.
 function watch(w) {
     const id = String(w.internalId);
     if (watched[id]) return;
     watched[id] = true;
     const onSpare = () => isSpare(w.output);
-    w.frameGeometryChanged.connect(() => { if (onSpare()) report("geometry", w); });
-    w.outputChanged.connect(() => report("output", w));
+    w.frameGeometryChanged.connect(() => {
+        if (changingOutputs()) {
+            moved[id] = true;
+            return;
+        }
+        if (holding(w)) return;
+        settle(w);
+        if (onSpare()) report("geometry", w);
+    });
+    w.outputChanged.connect(() => {
+        if (changingOutputs()) {
+            moved[id] = true;
+            return;
+        }
+        if (!held[id]) settle(w);  // (held: the place asked for is settled already)
+        report("output", w);
+        mark(w);
+    });
+    w.keepBelowChanged.connect(() => keepBelowChanged(w));
     w.interactiveMoveResizeStarted.connect(() => {
         if (onSpare()) send({ev: "move-start", id: id, move: w.move, resize: w.resize, frame: rect(w.frameGeometry)});
     });
     w.interactiveMoveResizeFinished.connect(() => { if (onSpare()) report("move-end", w); });
-    w.fullScreenChanged.connect(() => { if (onSpare()) report("fullscreen", w); });
+    w.fullScreenChanged.connect(() => {
+        if (changingOutputs()) {
+            moved[id] = true;
+            return;
+        }
+        if (settled[id]) settled[id].fullScreen = w.fullScreen;
+        if (onSpare()) report("fullscreen", w);
+    });
     w.minimizedChanged.connect(() => { if (onSpare()) report("minimized", w); });
     w.maximizedChanged.connect(() => {
         // A floating window stays an ordinary window: its output is its size plus a margin.
@@ -79,16 +258,52 @@ function watch(w) {
 
 workspace.windowAdded.connect(w => {
     watch(w);
+    settle(w);
     report("added", w);
 });
 workspace.windowRemoved.connect(w => {
-    send({ev: "removed", id: String(w.internalId)});
-    delete watched[String(w.internalId)];
+    const id = String(w.internalId);
+    send({ev: "removed", id: id});
+    delete watched[id];
+    delete settled[id];
+    delete held[id];
+    delete moved[id];
 });
 workspace.windowActivated.connect(w => {
     if (w && isSpare(w.output)) send({ev: "activated", id: String(w.internalId)});
 });
-workspace.windowList().forEach(watch);
+takeLayout();
+workspace.windowList().forEach(w => {
+    watch(w);
+    settle(w);
+});
+
+// Keep-below means "floating" in the Frametop desktop. The title bar's float button (Frametop's
+// window decoration, decoration/) is the Keep Below button, so setting the flag on a window on
+// the screens floats it, and clearing it on a floating one docks it. The script keeps the flag
+// set on every floating window, its dialogs included, and cleared everywhere else, however the
+// window got there. Kept below, a window alone on its own output only has the wallpaper under it.
+function marked(w) {
+    return w.managed && !w.deleted && !w.specialWindow && !w.popupWindow;
+}
+function topOf(w) {
+    let top = w;
+    for (let n = 0; top.transientFor && n < 10; ++n) top = top.transientFor;
+    return top;
+}
+function mark(w) {
+    if (screens === 0 || !marked(w)) return;
+    const want = isSpare(w.output);
+    if (w.keepBelow === want) return;
+    marking = true;
+    w.keepBelow = want;
+    marking = false;
+}
+function keepBelowChanged(w) {
+    if (marking || screens === 0 || !marked(w)) return;
+    const top = topOf(w);
+    if (w.keepBelow !== isSpare(top.output)) requestFloat(top);
+}
 
 function requestFloat(w) {
     if (!w || !w.normalWindow || w.popupWindow) return;
@@ -104,15 +319,35 @@ registerUserActionsMenu(w => {
         triggered: () => requestFloat(w)
     };
 });
-registerShortcut("Frametop Float Window", "Frametop: Float Window in VR (or put it back)", "Meta+Shift+F",
-                 () => requestFloat(workspace.activeWindow));
+// The float key is the input relay's (float_toggle, Meta+Shift+F by default): it reaches us as
+// "request-pointer". No shortcut of KWin's own, so one press can't float a window and dock it again.
+
+// The window under KWin's pointer (where the 3D mouse or a laser last was on a panel): the top
+// one there, a popup or dialog standing for the window it belongs to. Null over the wallpaper
+// or the taskbar.
+function underPointer() {
+    const p = workspace.cursorPos;
+    const order = workspace.stackingOrder;
+    for (let i = order.length - 1; i >= 0; --i) {
+        const w = order[i];
+        if (w.deleted || w.minimized || w.hidden || !w.managed) continue;
+        const g = w.frameGeometry;
+        if (p.x < g.x || p.y < g.y || p.x >= g.x + g.width || p.y >= g.y + g.height) continue;
+        const top = topOf(w);
+        return top.normalWindow && !top.popupWindow ? top : null;
+    }
+    return null;
+}
 
 function run(c) {
     const w = c.id ? byId(c.id) : null;
     switch (c.cmd) {
         case "config":
             screens = c.screens;
-            workspace.windowList().forEach(w => report("window", w));
+            workspace.windowList().forEach(w => { report("window", w); mark(w); });
+            break;
+        case "mark":  // after a float that didn't happen: keep-below back as it was
+            if (w) mark(w);
             break;
         case "place": {  // onto an output, at a frame rectangle (logical, global)
             if (!w) break;
@@ -120,13 +355,22 @@ function run(c) {
             if (!o) break;
             if (w.fullScreen && !c.keepFullScreen) w.fullScreen = false;
             w.setMaximize(false, false);
+            expect(w, o.name, c, w.fullScreen && !!c.keepFullScreen);
             workspace.sendClientToScreen(w, o);
             w.frameGeometry = {x: c.x, y: c.y, width: c.w, height: c.h};
             if (c.onAllDesktops !== undefined) w.onAllDesktops = c.onAllDesktops;
+            if (c.maximized) {
+                // Maximized: KWin picks the size, and the place above is only where it goes.
+                delete held[c.id];
+                settled[c.id].maximized = true;
+                w.setMaximize(true, true);
+            }
             break;
         }
         case "geometry":
-            if (w) w.frameGeometry = {x: c.x, y: c.y, width: c.w, height: c.h};
+            if (!w) break;
+            expect(w, settled[c.id] ? settled[c.id].output : (w.output ? w.output.name : ""), c, w.fullScreen);
+            w.frameGeometry = {x: c.x, y: c.y, width: c.w, height: c.h};
             break;
         case "close":
             if (w) w.closeWindow();
@@ -134,15 +378,36 @@ function run(c) {
         case "activate":
             if (w) workspace.activeWindow = w;
             break;
+        case "activate-output": {  // the top window on that output (a spin brought it to the front)
+            const order = workspace.stackingOrder;
+            for (let i = order.length - 1; i >= 0; --i) {
+                const o = order[i];
+                if (o.deleted || o.minimized || o.hidden || !o.managed || !o.output) continue;
+                if (o.output.name !== c.output || !o.normalWindow || o.popupWindow) continue;
+                workspace.activeWindow = o;
+                break;
+            }
+            break;
+        }
         case "minimize":
             if (w) w.minimized = c.on;
             break;
         case "info":
             if (w) report("window", w);
             break;
-        case "request-active":  // ft-float float|dock active: like the shortcut
+        case "report-all":  // a profile's capture: every window as it is now, then a marker
+            workspace.windowList().forEach(w => report("window", w));
+            send({ev: "reported", token: c.token});
+            break;
+        case "request-float":  // ft-float float ID: float it, if it isn't floating
+            if (w && !isSpare(w.output)) requestFloat(w);
+            break;
+        case "request-active":  // ft-float float|dock active
             requestFloat(workspace.activeWindow);
             break;
+        case "request-pointer":  // the float key: the window under the pointer, else the active one
+            requestFloat(underPointer() || workspace.activeWindow);
+            break;
     }
 }
 

float/ft-float

@@ -2,11 +2,18 @@
 """ft-float: talk to ft-floatd (floating windows in the Frametop desktop).
 
   ft-float float [ID|active]   float a window (default: the active one; for it, a toggle)
+  ft-float float pointer       the float key: float the window under the pointer (else the
+                               active one), or put it back if it floats
   ft-float dock [ID|active]    put a floating window back on the desktop
+  ft-float dock all            put every floating window back
+  ft-float launch APP          start an app (its desktop file name, e.g. org.kde.dolphin) and
+                               float its first window where that app last floated
+  ft-float run COMMAND [ARG...]  the same for a command
   ft-float close ID            close a window
   ft-float list                the spare outputs and what floats on them
 FT_FLOAT_SOCKET names ft-floatd's socket (default frametop_float).
 """
+import json
 import os
 import socket
 import sys
@@ -17,7 +24,8 @@ s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
 s.bind("")
 s.settimeout(5)
 try:
-    s.sendto(" ".join(sys.argv[1:]).encode(), "\0" + os.environ.get("FT_FLOAT_SOCKET", "frametop_float"))
+    text = "run " + json.dumps(sys.argv[2:]) if sys.argv[1] == "run" else " ".join(sys.argv[1:])
+    s.sendto(text.encode(), "\0" + os.environ.get("FT_FLOAT_SOCKET", "frametop_float"))
     reply = s.recv(8192).decode()
 except OSError as e:
     sys.exit(f"ft-floatd didn't answer ({e}); is the Frametop desktop running?")

float/ft-float-apps

@@ -0,0 +1,3 @@
+#!/bin/sh
+# Launch as Standalone: write the apps' desktop file copies (see ft_apps.py).
+exec python3 "$(dirname "$(readlink -f "$0")")/ft_apps.py" "$@"

float/ft_apps.py

@@ -0,0 +1,123 @@
+#!/usr/bin/env python3
+"""Launch as Standalone: an action on every app in the Frametop desktop's menus.
+
+Plasma's Application Launcher has no way to add an entry to every app's right-click menu,
+but its menu (and the taskbar's) shows each app's own desktop actions. So the Frametop
+desktop reads copies of the apps' desktop files with one more action, "Launch as
+Standalone", which runs `ft-float launch <desktop file name>`: the app starts and its first
+window floats in VR (docs/floating-windows.md, decision 26).
+
+The copies go in OUT (~/.local/share/frametop/apps/applications), and the session puts
+OUT_ROOT first in XDG_DATA_DIRS, so a copy wins over the app's own file. Plasma's app cache
+is keyed by those directories, so Desktop Mode never sees the copies. Files in
+XDG_DATA_HOME/applications come before every data dir, so an app customized there keeps
+its own file and has no Launch as Standalone. A copy drops DBusActivatable: a D-Bus
+activated app would be asked to run the action itself, and it doesn't know ours.
+
+  ft-float-apps        write the copies (the session script runs it before Plasma starts;
+                       ft-floatd again whenever an app's desktop file changes)
+"""
+import os
+import sys
+
+from gi.repository import GLib
+
+ACTION = "frametop-standalone"
+OUT_ROOT = os.path.expanduser("~/.local/share/frametop/apps")
+OUT = os.path.join(OUT_ROOT, "applications")
+FT_FLOAT = os.path.join(os.path.dirname(os.path.realpath(__file__)), "ft-float")
+GROUP = "Desktop Entry"
+
+
+def data_dirs():
+    """(XDG_DATA_HOME, the XDG_DATA_DIRS other than ours)."""
+    home = os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share")
+    ours = os.path.realpath(OUT_ROOT)
+    dirs = [d for d in os.environ.get("XDG_DATA_DIRS", "/usr/local/share:/usr/share").split(":")
+            if d and os.path.realpath(d) != ours]
+    return home, dirs
+
+
+def app_dirs():
+    """Every applications folder that holds the apps' own desktop files, home first."""
+    home, dirs = data_dirs()
+    return [os.path.join(d, "applications") for d in [home] + dirs]
+
+
+def scan():
+    """Desktop file name -> (its path, whether it's in XDG_DATA_HOME), the first one found
+    winning, as the desktop spec has it."""
+    found = {}
+    home_apps = app_dirs()[0]
+    for root in app_dirs():
+        for dirpath, _dirs, files in os.walk(root, followlinks=True):
+            for name in files:
+                if name.endswith(".desktop"):
+                    path = os.path.join(dirpath, name)
+                    found.setdefault(os.path.relpath(path, root).replace("/", "-"), (path, root == home_apps))
+    return found
+
+
+def standalone(path, desktop_id):
+    """The desktop file with Launch as Standalone added, or None for one that isn't a
+    visible app."""
+    kf = GLib.KeyFile()
+    try:
+        kf.load_from_file(path, GLib.KeyFileFlags.KEEP_TRANSLATIONS | GLib.KeyFileFlags.KEEP_COMMENTS)
+    except GLib.Error:
+        return None
+
+    def get(key):
+        try:
+            return kf.get_string(GROUP, key)
+        except GLib.Error:
+            return None
+    if get("Type") != "Application" or not get("Exec"):
+        return None
+    if (get("NoDisplay") or "").lower() == "true" or (get("Hidden") or "").lower() == "true":
+        return None
+    actions = [a for a in (get("Actions") or "").split(";") if a and a != ACTION]
+    kf.set_string(GROUP, "Actions", ";".join(actions + [ACTION]) + ";")
+    try:
+        kf.remove_key(GROUP, "DBusActivatable")
+    except GLib.Error:
+        pass
+    group = "Desktop Action " + ACTION
+    kf.set_string(group, "Name", "Launch as Standalone")
+    kf.set_string(group, "Icon", "window-new")
+    exe = FT_FLOAT if " " not in FT_FLOAT else '"' + FT_FLOAT + '"'
+    kf.set_string(group, "Exec", f"{exe} launch {desktop_id}")
+    return kf.to_data()[0]
+
+
+def write_all():
+    """Write the copies, and remove the ones whose app is gone. Returns how many there are."""
+    os.makedirs(OUT, exist_ok=True)
+    keep = set()
+    for desktop_id, (path, in_home) in sorted(scan().items()):
+        if in_home:
+            continue  # XDG_DATA_HOME's own file wins over a copy anyway
+        data = standalone(path, desktop_id)
+        if data is None:
+            continue
+        keep.add(desktop_id)
+        out = os.path.join(OUT, desktop_id)
+        try:
+            with open(out) as f:
+                if f.read() == data:
+                    continue
+        except OSError:
+            pass
+        with open(out + ".tmp", "w") as f:
+            f.write(data)
+        os.replace(out + ".tmp", out)
+    for name in os.listdir(OUT):
+        if name.endswith(".desktop") and name not in keep:
+            os.remove(os.path.join(OUT, name))
+    return len(keep)
+
+
+if __name__ == "__main__":
+    if len(sys.argv) > 1:
+        sys.exit(__doc__)
+    print(f"{write_all()} apps with Launch as Standalone in {OUT}")

float/ft_floatd.py

@@ -11,8 +11,15 @@ which window floats on which spare output and panel, and connects three parts:
   - kscreen-doctor, to turn spare outputs on and off and place them in KWin's layout.
 Commands and ft-screens' events arrive as datagrams on @frametop_float (ft-float is the
 command-line side). Replies go to the sender:
-  float [ID|active]   dock [ID|active]   close ID   list   quit          (ft-float)
+  float [ID|active|pointer]   dock [ID|active|all]   close ID   list   quit   (ft-float)
+  launch APP.desktop   run ["argv", ...]   (start an app and float its first window)
+  windows   (-> JSON: the open apps' windows and where they are, for a profile; ft-layout save)
+  profile NAME   (open a profile's apps: move the matching windows, launch the missing ones)
+  ("float pointer" is the float key: the window under the pointer, else the active one,
+  floated or docked; the input relay sends it for float_toggle, and "dock all" for dock_all)
   dock N | close N | resize N W H | scale N STEPS                      (ft-screens, N = its screen)
+  front N   (ft-screens: a spin brought panel N to the front: make its window, or the top one
+            on a screen, KWin's active window)
 
 Spare outputs are WL-<screens> .. WL-<screens + slots - 1>. A floating window's output is its
 frame plus a margin on each side (FLOAT_MARGIN pixels), so menus have room; the panel shows
@@ -22,6 +29,25 @@ layout, all within Xwayland's 32767-pixel limit.
 Usage: ft-floatd [--screens N] [--slots N] [--margin PX] [--control NAME] [--socket NAME]
 Defaults: FT_SCREEN_COUNT (from the session) or the layout's count, FLOAT_SLOTS and
 FLOAT_MARGIN from ~/.config/frametop.conf (8 and 300), @ft_screens, @frametop_float.
+
+Launching floating: "launch" starts a desktop file's app (Gio), "run" a command. The next
+normal window from that process (or a child), or with that desktop file name, within
+LAUNCH_SECONDS floats: where that app last floated, or in front of you. Single-instance and
+D-Bus-activated apps open their window from a process that was already running, hence the
+name. Each app's last floating place (its pose relative to the primary screen, so it moves
+with the screens' layout), size in pixels, and scale are kept in PLACES_PATH, by desktop
+file name, whenever one of its windows stops floating.
+
+Profiles (docs/profiles.md): "windows" lists every app window with where it is: on a screen
+(its rectangle on that output, maximized or not) or floating (its pose relative to the primary
+screen, size, scale). "profile NAME" takes a profile's windows from the layout file: the
+windows of each app already open (oldest first) move to its entries, and for the entries
+left the app is launched once, and again for each window still missing 3 seconds after its
+first one shows up (a browser restores its own windows; a terminal opens one each time).
+Nothing is ever closed.
+
+Launch as Standalone (float/ft_apps.py): copies of the apps' desktop files with that action,
+which only the Frametop desktop reads. ft-floatd rewrites them when apps change.
 """
 import argparse
 import json
@@ -36,11 +62,12 @@ import time
 import dbus
 import dbus.mainloop.glib
 import dbus.service
-from gi.repository import GLib
+from gi.repository import Gio, GLib
 
 HERE = os.path.dirname(os.path.realpath(__file__))
 sys.path.insert(0, os.path.join(HERE, "..", "layout"))
 import ft_layout  # noqa: E402  (config and layout)
+import ft_apps  # noqa: E402  (Launch as Standalone)
 
 SERVICE = IFACE = "org.frametop.Float"
 PATH = "/Float"
@@ -49,6 +76,9 @@ POLL_SECONDS = 20        # NextCommand answers empty after this (KWin's D-Bus ti
 SPARE_X, SPARE_CELL = 12000, 5000  # spares in KWin's layout: a grid from here, 4 across
 PULL_OUT = 0.3           # a floated window starts this far in front of its screen (metres), clear of it
 DEFAULT_MPP = 1.6 / 1920  # metres per pixel when ft-screens can't say (no SteamVR)
+PLACES_PATH = os.path.expanduser("~/.config/frametop-float.json")  # each app's last floating place
+LAUNCH_SECONDS = 30      # a launched app's window must show up within this
+IN_FRONT = 2.0           # at most this far in front of you, for an app with no remembered place
 DEBUG = os.environ.get("FT_FLOAT_DEBUG") == "1"  # log every event from the script
 
 
@@ -108,6 +138,20 @@ def kscreen(*args):
         return ""
 
 
+def output_rects():
+    """Each output's place and size in KWin's layout (logical)."""
+    try:
+        data = json.loads(kscreen("-j") or "{}")
+    except ValueError:
+        return {}
+    out = {}
+    for o in data.get("outputs", []):
+        pos, size, scale = o.get("pos") or {}, o.get("size") or {}, float(o.get("scale", 1)) or 1.0
+        if o.get("name") and size:
+            out[o["name"]] = (pos.get("x", 0), pos.get("y", 0), size.get("width", 0) / scale, size.get("height", 0) / scale)
+    return out
+
+
 def output_scales():
     try:
         data = json.loads(kscreen("-j") or "{}")
@@ -116,6 +160,108 @@ def output_scales():
     return {o["name"]: float(o.get("scale", 1)) for o in data.get("outputs", []) if o.get("name")}
 
 
+def load_places():
+    try:
+        with open(PLACES_PATH) as f:
+            places = json.load(f)
+        return places if isinstance(places, dict) else {}
+    except (OSError, ValueError):
+        return {}
+
+
+def save_places(places):
+    tmp = PLACES_PATH + ".tmp"
+    with open(tmp, "w") as f:
+        json.dump(places, f, indent=1)
+    os.replace(tmp, PLACES_PATH)
+
+
+def to_local(ref, g):
+    """A panel's pose (ft_layout.parse_get) in the frame of another panel (ref): its centre
+    and axes, 12 numbers."""
+    axes = (ref["x"], ref["y"], ref["z"])
+    d = [g["center"][k] - ref["center"][k] for k in range(3)]
+    out = [ft_layout.dot(d, a) for a in axes]
+    for v in (g["x"], g["y"], g["z"]):
+        out += [ft_layout.dot(v, a) for a in axes]
+    return [round(v, 5) for v in out]
+
+
+def to_world(ref, local):
+    """to_local's inverse: (centre, x, y, z) in the world."""
+    axes = (ref["x"], ref["y"], ref["z"])
+
+    def turn(v):
+        return [sum(v[i] * axes[i][k] for i in range(3)) for k in range(3)]
+    c = turn(local[0:3])
+    c = [c[k] + ref["center"][k] for k in range(3)]
+    return c, turn(local[3:6]), turn(local[6:9]), turn(local[9:12])
+
+
+def pid_chain(pid):
+    """A process and its ancestors, nearest first."""
+    out = []
+    while pid > 1 and len(out) < 32:
+        out.append(pid)
+        try:
+            with open(f"/proc/{pid}/stat") as f:
+                pid = int(f.read().rsplit(")", 1)[1].split()[1])
+        except (OSError, ValueError, IndexError):
+            break
+    return out
+
+
+# Windows a profile doesn't keep: Plasma's own, the Frametop settings apps, the login splash.
+SKIP_APPS = ("org.kde.plasmashell", "org.kde.krunner", "org.kde.ksplashqml", "org.kde.polkit-kde-authentication-agent-1")
+
+
+def recordable(ev):
+    """An app's top-level window, the kind a profile keeps."""
+    if not ev.get("normal") or ev.get("popup") or ev.get("transient"):
+        return False
+    app, cls = ev.get("app", ""), ev.get("cls", "")
+    return app not in SKIP_APPS and not any(n.startswith(("ft-", "frametop", "ksplash")) for n in (app, cls))
+
+
+def cmdline(pid):
+    try:
+        with open(f"/proc/{int(pid)}/cmdline", "rb") as f:
+            return [a.decode(errors="replace") for a in f.read().split(b"\0") if a]
+    except (OSError, ValueError, TypeError):
+        return []
+
+
+def desktop_name(app, cls):
+    """The desktop file name for a window's app id and class. A Flatpak app's window can give an
+    id with no desktop file: RustDesk's (an X11 window) says com.carriez.flutter_hbb, and its
+    desktop file is com.rustdesk.RustDesk. That file's StartupWMClass names the window's class
+    then. The app id itself when nothing matches."""
+    try:
+        if not app or Gio.DesktopAppInfo.new(app + ".desktop"):
+            return app
+    except TypeError:  # PyGObject raises for the NULL a missing desktop file returns
+        pass
+    names = {n.lower() for n in (app, cls) if n}
+    for info in Gio.AppInfo.get_all():
+        wm = info.get_startup_wm_class() if isinstance(info, Gio.DesktopAppInfo) else None
+        if wm and wm.lower() in names:
+            return info.get_id().removesuffix(".desktop")
+    return app
+
+
+class Launch:
+    """An app we started: its first window floats, or (for a profile) its windows go to the
+    profile's entries for it, in order."""
+
+    def __init__(self, app, pid, entries=None, argv=None):
+        self.app = app            # its desktop file name, without .desktop ("" for a command)
+        self.argv = argv          # the command, when it isn't a desktop file's app
+        self.pids = [pid] if pid else []  # the processes we started (none when D-Bus started it)
+        self.entries = entries    # a profile's entries still waiting for a window, or None
+        self.relaunched = False
+        self.until = time.monotonic() + LAUNCH_SECONDS
+
+
 class Float:
     """A floating window."""
 
@@ -131,7 +277,9 @@ class Float:
         self.normal = None        # its size in pixels when last not full screen
         self.unfull_until = 0.0   # left full screen just now (see follow)
         self.move_from = None     # frame when a title-bar move started (put back after)
+        self.resizing = False     # KWin's resize by the window's edge is going on (see follow)
         self.subs = {}            # popup or dialog id -> number on the panel
+        self.app = ""             # its desktop file name (for its remembered place)
 
 
 class Slot:
@@ -153,11 +301,17 @@ class Daemon:
         self.slots = [Slot(k, args.screens) for k in range(args.slots)]
         self.floats = {}         # window id -> Float
         self.windows = {}        # window id -> last info from the script
+        self.gone = set()        # ids of windows that closed (KWin's ids aren't reused)
         self.pending = []        # commands for the script
         self.waiter = None       # (reply callback, timeout source) while the script waits
         self.screens = Screens(args.control)
         self.sub_numbers = {}    # popup/dialog id -> (window id, number)
         self.next_sub = 1
+        self.launches = []       # Launch: apps we started, waiting for their window
+        self.captures = {}       # token -> sender: "windows" requests waiting for the script's report
+        self.next_token = 1
+        self.sock = None         # our socket (set by main), for deferred replies
+        self.places = load_places()  # app -> {"rel": pose by the primary screen, "pixels", "scale"}
 
     # ------------------------------------------------------------ the script
 
@@ -195,8 +349,19 @@ class Daemon:
         sid = int(kwin.loadScript(os.path.join(HERE, "frametop-float.js"), SCRIPT, signature="ss"))
         if sid < 0:
             raise RuntimeError("KWin didn't load the script")
-        bus.get_object("org.kde.KWin", f"/Scripting/Script{sid}").run(dbus_interface="org.kde.kwin.Script")
+        # Scripting.start runs every loaded script that isn't running. Not /Scripting/Script<id>'s
+        # run: a reloaded script gets the old one's id while the old one is still being deleted,
+        # so that path is still the old script's, and the new one would never run.
+        kwin.start()
         log(f"script loaded ({sid})")
+        # Older scripts registered the float key with KWin; the input relay owns it now. Drop
+        # that shortcut, so System Settings doesn't list one that does nothing.
+        try:
+            accel = dbus.Interface(bus.get_object("org.kde.kglobalaccel", "/kglobalaccel"), "org.kde.KGlobalAccel")
+            if accel.unregister("kwin", "Frametop Float Window"):
+                log("dropped KWin's old float shortcut")
+        except dbus.DBusException:
+            pass
 
     # ------------------------------------------------------------ events from the script
 
@@ -211,7 +376,10 @@ class Daemon:
             # The script reports every window after "config"; spares nothing floats on are off.
             GLib.timeout_add(1500, self.disable_unused)
             return
+        if wid in self.gone:
+            return  # an event that came after the window closed (it would bring it back)
         if kind == "removed":
+            self.gone.add(wid)
             self.windows.pop(wid, None)
             if wid in self.floats:
                 log(f"{wid[:9]} closed")
@@ -221,6 +389,24 @@ class Daemon:
         if wid:
             self.windows[wid] = ev
         f = self.floats.get(wid)
+        if kind == "reported":
+            self.captured(ev.get("token"))
+            return
+        if kind == "added" and self.launches and not f:
+            launch = self.launched(ev)
+            if launch and launch.entries is None:
+                self.float_launched(ev, launch)
+                return
+            if launch:
+                self.place_entry(ev, launch.entries.pop(0))
+                if launch.entries and not launch.relaunched:
+                    GLib.timeout_add_seconds(3, lambda: self.relaunch(launch) and False)
+                return
+        if kind == "added" and not f and self.on_hidden_screen(ev):
+            # Nobody would see it there (a profile can hide every screen): it floats instead.
+            log(f"{wid[:9]} ({ev.get('cls')}) opened on a hidden screen")
+            self.float_launched(ev, None)
+            return
         if kind == "float-request":
             self.float_window(ev)
         elif kind == "dock-request":
@@ -236,6 +422,11 @@ class Daemon:
         elif kind == "move-start" and f and ev.get("move"):
             f.move_from = ev["frame"]
             self.screens.ask(f"carry {f.slot.index}", quiet=True)
+        elif kind == "move-start" and f and ev.get("resize"):
+            f.resizing = True
+        elif kind == "move-end" and f and f.resizing:
+            f.resizing = False
+            self.follow(f, ev)
         elif kind == "move-end" and f and f.move_from:
             # The panel carried the window; KWin may have slipped it a few pixels first.
             m, f.move_from = f.move_from, None
@@ -278,9 +469,14 @@ class Daemon:
                 return
             # Floating when ft-floatd (re)started: take it over where it is.
             f = Float(wid, slot, None)
+            f.app = ev.get("app", "")
             slot.window = f
             self.floats[wid] = f
             f.scale = slot.kscale = output_scales().get(slot.output, 1.0)
+            # Its panel's density as it is (follow below sets the panel again).
+            g, px = self.panel_get(slot.index), round(ev["frame"]["w"] * f.scale)
+            if g and g["metres"] > 0 and px > 0:
+                f.mpp = g["metres"] / px
             log(f"{wid[:9]} ({ev.get('cls')}) already floats on {slot.output}")
             self.follow(f, ev)
             return
@@ -290,8 +486,16 @@ class Daemon:
                                        for o in self.floats.values()):
             self.float_window(ev)
         else:
-            self.command(cmd="place", id=wid, output="WL-0", x=ev["frame"]["x"] % 400 + 100,
-                         y=ev["frame"]["y"] % 300 + 100, w=ev["frame"]["w"], h=ev["frame"]["h"])
+            # Onto the first screen that shows (or floating, if none does).
+            hidden = self.concealed()
+            shown = [i for i in range(self.screens_n) if i + 1 not in hidden]
+            if not shown:
+                self.float_launched(ev, None)
+                return
+            name = f"WL-{shown[0]}"
+            x0, y0, _, _ = output_rects().get(name, (0, 0, 0, 0))
+            self.command(cmd="place", id=wid, output=name, x=x0 + ev["frame"]["x"] % 400 + 100,
+                         y=y0 + ev["frame"]["y"] % 300 + 100, w=ev["frame"]["w"], h=ev["frame"]["h"])
 
     # ------------------------------------------------------------ floating and docking
 
@@ -339,22 +543,27 @@ class Daemon:
                     return float(metres) / max(1, int(size.split("x")[0]))
         return DEFAULT_MPP
 
-    def float_window(self, ev):
+    def float_window(self, ev, place=None, mpp=None, scale=None):
+        """Float a window: its panel in front of where it was on its screen, or at place (centre
+        and axes in the world), at the density of its screen, or mpp, and at its screen's scale,
+        or scale (then ev's frame is the window's size at that scale). Returns its Float."""
         wid = ev["id"]
         if wid in self.floats:
-            return
+            return self.floats[wid]
         slot = self.free_slot()
         if slot is None:
+            self.command(cmd="mark", id=wid)  # the title bar button set keep-below for nothing
             self.notify(f"All {len(self.slots)} floating windows are in use. Put one back on the desktop "
                         "to float another.")
-            return
+            return None
         f = Float(wid, slot, {"output": ev["output"], "frame": ev["frame"], "onAllDesktops": ev.get("onAllDesktops")})
+        f.app = ev.get("app", "")
         slot.window = f
         self.floats[wid] = f
         scales = output_scales()
-        f.scale = scales.get(ev["output"], 1.0)
+        f.scale = scale or scales.get(ev["output"], 1.0)
         slot.kscale = scales.get(slot.output, slot.kscale)
-        f.mpp = self.screen_mpp(ev["output"])
+        f.mpp = mpp or self.screen_mpp(ev["output"])
         fr, s, m = ev["frame"], f.scale, self.margin
         w, h = round(fr["w"] * s), round(fr["h"] * s)
         log(f"{wid[:9]} ({ev.get('cls')}) floats on {slot.output}: {w}x{h} px, scale {s:g}")
@@ -365,12 +574,20 @@ class Daemon:
         self.set_size(slot, (w + 2 * m, h + 2 * m), whole(slot.kscale, s))
         self.screens.ask(f"scale {slot.index} {s:g}")  # for pointer positions (KWin's units)
         self.set_panel(f, (m, m, w, h), title=round((ev["client"]["y"] - fr["y"]) * s))
-        self.place_panel(f, ev)
+        if place:
+            self.pose(f, *place)
+        else:
+            self.place_panel(f, ev)
         kscreen(f"output.{slot.output}.enable", f"output.{slot.output}.scale.{s:g}",
                 f"output.{slot.output}.position.{slot.pos[0]},{slot.pos[1]}")
         self.rescaled(slot, s)
         self.command(cmd="place", id=wid, output=slot.output, x=slot.pos[0] + m / s, y=slot.pos[1] + m / s,
                      w=fr["w"], h=fr["h"], onAllDesktops=True)
+        return f
+
+    def pose(self, f, c, xa, ya, za):
+        rows = [f"{xa[k]:.5f} {ya[k]:.5f} {za[k]:.5f} {c[k]:.4f}" for k in range(3)]
+        self.screens.ask(f"pose {f.slot.index} {' '.join(rows)}")
 
     def set_panel(self, f, crop, title=0):
         x, y, w, h = crop
@@ -389,8 +606,7 @@ class Daemon:
         dx = ((fr["x"] - out["x"]) + fr["w"] / 2 - out["w"] / 2) * s * f.mpp
         dy = ((fr["y"] - out["y"]) + fr["h"] / 2 - out["h"] / 2) * s * f.mpp
         p = [c[k] + xa[k] * dx - ya[k] * dy + za[k] * PULL_OUT for k in range(3)]
-        rows = [f"{xa[k]:.5f} {ya[k]:.5f} {za[k]:.5f} {p[k]:.4f}" for k in range(3)]
-        self.screens.ask(f"pose {f.slot.index} {' '.join(rows)}")
+        self.pose(f, p, xa, ya, za)
 
     def follow(self, f, ev):
         """The window moved or resized on its output: crop the panel to it, and keep the
@@ -406,9 +622,17 @@ class Daemon:
         full = bool(ev.get("fullScreen")) or (fills and time.monotonic() > f.unfull_until)
         f.full = full
         w, h = round(fr["w"] * s), round(fr["h"] * s)
-        if not full:
+        if not full and not (f.normal and abs(f.normal[0] - w) <= math.ceil(s) and abs(f.normal[1] - h) <= math.ceil(s)):
+            # (Within a logical pixel it's the same size: a size asked for in pixels comes out
+            # rounded to whole logical pixels. Keeping it stops scale changes from creeping.)
             f.normal = (w, h)
         m = 0 if full else self.margin
+        if f.resizing:
+            # KWin ends a resize by the window's edge whenever an output changes: the output
+            # follows when it's done (the margin is room to grow until then).
+            x, y = round((fr["x"] - out["x"]) * s), round((fr["y"] - out["y"]) * s)
+            self.set_panel(f, (x, y, w, h), title=round((cl["y"] - fr["y"]) * s))
+            return
         self.set_size(slot, f.normal if full and f.normal else (w + 2 * m, h + 2 * m))
         if not full:
             x0, y0 = slot.pos[0] + m / s, slot.pos[1] + m / s
@@ -423,10 +647,14 @@ class Daemon:
         panel stays the same size (KWin's output scale, in steps of 10%)."""
         if not f.frame or f.full or steps == 0:
             return
-        s = min(3.0, max(0.5, round(f.scale * 1.1 ** steps * 20) / 20))
-        if s == f.scale:
+        self.set_scale(f, round(f.scale * 1.1 ** steps * 20) / 20)
+
+    def set_scale(self, f, s):
+        """The window's scale (KWin's output scale), at the same size in pixels."""
+        s = min(3.0, max(0.5, s))
+        if not f.frame or f.full or s == f.scale or self.floats.get(f.id) is not f:
             return
-        w, h = round(f.frame["w"] * f.scale), round(f.frame["h"] * f.scale)
+        w, h = f.normal or (round(f.frame["w"] * f.scale), round(f.frame["h"] * f.scale))
         slot, m = f.slot, self.margin
         log(f"{f.id[:9]} scale {f.scale:g} -> {s:g}")
         f.scale = s
@@ -450,17 +678,45 @@ class Daemon:
         want, slot.want = slot.size, None
         self.set_size(slot, want)
 
-    def dock(self, f, frame=None):
-        """Back where it came from (or onto screen 1 if we don't know)."""
+    def dock(self, f, frame=None, output=None, maximized=False):
+        """Back where it came from (or onto screen 1 if we don't know), or onto output at frame."""
         saved = f.saved or {"output": "WL-0", "frame": dict(f.frame or {"x": 100, "y": 100, "w": 800, "h": 600}),
                             "onAllDesktops": False}
         fr = frame or saved["frame"]
-        log(f"{f.id[:9]} back to {saved['output']}")
-        self.command(cmd="place", id=f.id, output=saved["output"], x=fr["x"], y=fr["y"], w=fr["w"], h=fr["h"],
-                     onAllDesktops=bool(saved.get("onAllDesktops")))
+        if not output:
+            output = saved["output"]
+            m = re.match(r"WL-(\d+)$", output)
+            hidden = self.concealed()
+            if m and int(m.group(1)) + 1 in hidden:
+                # It came from a screen that's hidden now: onto the first one that shows.
+                shown = [i for i in range(self.screens_n) if i + 1 not in hidden]
+                if shown:
+                    x0, y0, ow, oh = output_rects().get(f"WL-{shown[0]}", (0, 0, 0, 0))
+                    output = f"WL-{shown[0]}"
+                    w, h = min(fr["w"], ow or fr["w"]), min(fr["h"], oh or fr["h"])
+                    fr = {"x": x0 + max(0, (ow - w) / 2), "y": y0 + max(0, (oh - h) / 2), "w": w, "h": h}
+        log(f"{f.id[:9]} back to {output}")
+        self.command(cmd="place", id=f.id, output=output, x=fr["x"], y=fr["y"], w=fr["w"], h=fr["h"],
+                     onAllDesktops=bool(saved.get("onAllDesktops")), maximized=maximized)
+
+    def remember(self, f):
+        """Keep where an app's window floated (before its panel goes)."""
+        if not f.app or not f.frame:
+            return
+        g, ref = self.panel_get(f.slot.index), self.reference()
+        if not g or not ref:
+            return
+        pixels = list(f.normal or (round(f.frame["w"] * f.scale), round(f.frame["h"] * f.scale)))
+        self.places[f.app] = {"rel": to_local(ref, g), "pixels": pixels, "scale": f.scale,
+                              "mpp": round(f.mpp, 8)}
+        try:
+            save_places(self.places)
+        except OSError as e:
+            log(f"couldn't save {PLACES_PATH}: {e}")
 
     def release(self, f):
         """Its window left: hide the panel and turn the spare off."""
+        self.remember(f)
         slot = f.slot
         if slot.window is f:
             slot.window = None
@@ -470,6 +726,274 @@ class Daemon:
         self.screens.ask(f"unfloat {slot.index}", quiet=True)
         kscreen(f"output.{slot.output}.disable")
 
+    # ------------------------------------------------------------ launching floating
+
+    def panel_get(self, index):
+        reply = self.screens.ask(f"get {index}", quiet=True)
+        return ft_layout.parse_get(reply) if reply.startswith("ok") else None
+
+    def primary(self):
+        """The primary screen (0-based): the one with the taskbar."""
+        try:
+            return min(ft_layout.primary_screen(ft_layout.load_layout()), self.screens_n - 1)
+        except (OSError, ValueError, KeyError):
+            return 0
+
+    def reference(self):
+        """The primary screen's panel: remembered places are relative to it, so they move with
+        the screens' layout."""
+        return self.panel_get(self.primary() + 1)
+
+    def in_front(self):
+        """A place in front of you, facing you, a little nearer than the primary screen (so text
+        looks as big as on the screens, at their density)."""
+        reply = self.screens.ask("head", quiet=True)
+        ref = self.reference()
+        if not reply.startswith("ok"):
+            return None
+        h = reply.split()
+        eye, heading = [float(v) for v in h[1:4]], float(h[4])
+        # (Within reach: you may have walked away from the screens since they were arranged.)
+        d = min(IN_FRONT, max(0.8, math.dist(eye, ref["center"]) - PULL_OUT)) if ref else IN_FRONT
+        fwd = ft_layout.turn_yaw((0.0, 0.0, -1.0), heading)
+        c = [eye[k] + fwd[k] * d for k in range(3)]
+        c[1] -= 0.1 * d  # a little below eye level, like a screen
+        return c, ft_layout.turn_yaw((1.0, 0.0, 0.0), heading), (0.0, 1.0, 0.0), ft_layout.turn_yaw((0.0, 0.0, 1.0), heading)
+
+    def launch(self, app=None, argv=None):
+        """Start an app (a desktop file name) or a command; its first window will float.
+        Returns (reply, pid or None)."""
+        if app:
+            app = app.removesuffix(".desktop")
+            try:
+                info = Gio.DesktopAppInfo.new(app + ".desktop")
+            except TypeError:  # PyGObject raises for the NULL a missing desktop file returns
+                info = None
+            if info is None:
+                return f"error no app {app}", None
+            pids = []
+            try:
+                info.launch_uris_as_manager([], Gio.AppLaunchContext(), GLib.SpawnFlags.SEARCH_PATH,
+                                            None, None, lambda _info, pid, *_: pids.append(pid), None)
+            except GLib.Error as e:
+                return f"error {app}: {e.message}", None
+            self.launches.append(Launch(app, pids[0] if pids else None))
+            log(f"launched {app} (pid {pids[0] if pids else 'by D-Bus'})")
+            return "ok", (pids[0] if pids else None)
+        else:
+            try:
+                proc = subprocess.Popen(argv, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
+                                        stderr=subprocess.DEVNULL, start_new_session=True)
+            except OSError as e:
+                return f"error {argv[0]}: {e}", None
+            GLib.child_watch_add(GLib.PRIORITY_DEFAULT, proc.pid, lambda *_: None)  # reap it
+            self.launches.append(Launch("", proc.pid))
+            log(f"started {argv[0]} (pid {proc.pid})")
+            return "ok", proc.pid
+
+    def launched(self, ev):
+        """Is this new window the one an app we started was to open? Takes it off the list."""
+        now = time.monotonic()
+        self.launches = [la for la in self.launches if la.until > now]
+        if not ev.get("normal") or ev.get("popup") or ev.get("transient"):
+            return None
+        chain = pid_chain(int(ev.get("pid") or 0))
+        app = ev.get("app", "")
+        # (An X11 window in a Flatpak gives its pid in the sandbox: only its app id can match.)
+        apps = {app, desktop_name(app, ev.get("cls", ""))} if app and any(la.app for la in self.launches) else {app}
+        for la in self.launches:
+            if any(p in chain for p in la.pids) or (la.app and app and la.app in apps):
+                if not la.entries or len(la.entries) <= 1:
+                    self.launches.remove(la)
+                return la
+        return None
+
+    def concealed(self):
+        """The screens (1-based) hidden on their own (ft-layout hide N)."""
+        reply = self.screens.ask("concealed", quiet=True)
+        return {int(w) for w in reply.split()[1:] if w.isdigit()} if reply.startswith("ok") else set()
+
+    def on_hidden_screen(self, ev):
+        """A new top-level window on a screen that's hidden on its own."""
+        m = re.match(r"WL-(\d+)$", ev.get("output", ""))
+        if not m or int(m.group(1)) >= self.screens_n:
+            return False
+        if not ev.get("normal") or ev.get("popup") or ev.get("transient") or ev.get("cls") == "ksplashqml":
+            return False
+        return int(m.group(1)) + 1 in self.concealed()
+
+    def float_launched(self, ev, launch):
+        """Float a launched app's window (or one that opened on a hidden screen) where that app
+        last floated (its size too), or in front of you, at the primary screen's density."""
+        app = ev.get("app") or (launch.app if launch else "")
+        known = self.places.get(app) if app else None
+        ref = self.reference()
+        place = to_world(ref, known["rel"]) if known and ref and len(known.get("rel", [])) == 12 else self.in_front()
+        mpp = (known or {}).get("mpp") or self.screen_mpp(f"WL-{self.primary()}")
+        scale = None
+        if known and known.get("pixels"):
+            scale = known.get("scale") or output_scales().get(ev["output"], 1.0)
+            ev = dict(ev, frame=dict(ev["frame"], w=known["pixels"][0] / scale, h=known["pixels"][1] / scale))
+        self.float_window(dict(ev, app=app), place=place, mpp=mpp, scale=scale)
+
+    # ------------------------------------------------------------ profiles (docs/profiles.md)
+
+    def capture(self, sender):
+        """"windows": have the script report every window as it is now, and answer when the
+        report is in (or after 3 seconds with what we know)."""
+        if not sender:
+            return "error windows needs a reply address"
+        token = self.next_token
+        self.next_token += 1
+        self.captures[token] = sender
+        self.command(cmd="report-all", token=token)
+        GLib.timeout_add(3000, lambda: self.captured(token) and False)
+        return None  # answered by captured
+
+    def captured(self, token):
+        sender = self.captures.pop(token, None)
+        if sender and self.sock:
+            try:
+                self.sock.sendto(("ok " + json.dumps(self.window_entries())).encode(), sender)
+            except OSError as e:
+                log(f"windows: {e}")
+
+    def window_entries(self):
+        """Every app window and where it is, as a profile keeps it."""
+        ref = self.reference()
+        rects = None
+        out = []
+        for wid, ev in self.windows.items():
+            if not recordable(ev):
+                continue
+            entry = {"app": desktop_name(ev["app"], ev.get("cls", ""))} if ev.get("app") else {"cmd": cmdline(ev.get("pid")), "class": ev.get("cls", "")}
+            if not entry.get("app") and not entry.get("cmd"):
+                continue
+            f = self.floats.get(wid)
+            if f:
+                g = self.panel_get(f.slot.index)
+                if not g or not ref or not f.frame:
+                    continue
+                pixels = list(f.normal or (round(f.frame["w"] * f.scale), round(f.frame["h"] * f.scale)))
+                entry["float"] = {"rel": to_local(ref, g), "pixels": pixels, "scale": f.scale, "mpp": round(f.mpp, 8)}
+            else:
+                m = re.match(r"WL-(\d+)$", ev.get("output", ""))
+                if not m or int(m.group(1)) >= self.screens_n:
+                    continue
+                fr, o = ev["frame"], ev.get("outputRect")
+                if not o:
+                    rects = rects if rects is not None else output_rects()
+                    x0, y0, _, _ = rects.get(ev["output"], (0, 0, 0, 0))
+                    o = {"x": x0, "y": y0}
+                entry["screen"] = int(m.group(1)) + 1
+                entry["rect"] = [round(fr["x"] - o["x"]), round(fr["y"] - o["y"]), round(fr["w"]), round(fr["h"])]
+                if ev.get("maximized"):
+                    entry["maximized"] = True
+            out.append(entry)
+        return out
+
+    def window_key(self, ev):
+        return desktop_name(ev.get("app", ""), ev.get("cls", "")) or json.dumps(cmdline(ev.get("pid")))
+
+    def open_profile(self, name):
+        """Open a profile's apps (additive: nothing closes)."""
+        try:
+            profile = ft_layout.load_layout().get("profiles", {}).get(name)
+        except (OSError, ValueError):
+            profile = None
+        if profile is None:
+            return f"error no profile {name!r}"
+        groups = {}
+        for e in profile.get("windows", []):
+            key = e.get("app") or json.dumps(e.get("cmd") or [])
+            if key != "[]":
+                groups.setdefault(key, []).append(e)
+        claimed = set()
+        keys = {wid: self.window_key(ev) for wid, ev in self.windows.items() if recordable(ev)}
+        for key, entries in groups.items():
+            have = [ev for wid, ev in self.windows.items() if wid not in claimed and keys.get(wid) == key]
+            for e, ev in zip(entries, have):
+                claimed.add(ev["id"])
+                self.place_entry(ev, e)
+            rest = entries[len(have):]
+            if rest:
+                self.launch_entries(rest)
+        log(f"profile {name!r}: {sum(len(v) for v in groups.values())} windows, {len(claimed)} already open")
+        return "ok"
+
+    def launch_entries(self, entries):
+        """Launch an app for a profile's entries that have no window yet."""
+        e = entries[0]
+        if e.get("app"):
+            reply, pid = self.launch(app=e["app"])
+            app = e["app"]
+        else:
+            reply, pid = self.launch(argv=e["cmd"])
+            app = ""
+        if not reply.startswith("ok"):
+            log(f"profile: {reply}")
+            return
+        la = self.launches[-1]  # the one launch() just added
+        la.entries, la.argv = list(entries), (None if app else e["cmd"])
+
+    def relaunch(self, la):
+        """3 seconds after a profile's app showed its first window: start it again for each
+        entry still waiting (an app that restores its own windows has shown them by now)."""
+        if la.relaunched or not la.entries or la not in self.launches:
+            return
+        la.relaunched = True
+        la.until = time.monotonic() + LAUNCH_SECONDS
+        for _ in la.entries:
+            if la.app:
+                reply, pid = self.launch(app=la.app)
+            else:
+                reply, pid = self.launch(argv=la.argv)
+            if reply.startswith("ok"):
+                self.launches.pop()  # launch() added one of its own; this one waits for them all
+                if pid:
+                    la.pids.append(pid)
+
+    def place_entry(self, ev, e):
+        """Move a window to a profile entry's place: on a screen, or floating."""
+        f = self.floats.get(ev["id"])
+        if "float" in e:
+            fl, ref = e["float"], self.reference()
+            if not ref or len(fl.get("rel", [])) != 12:
+                return
+            place = to_world(ref, fl["rel"])
+            pixels, scale = fl.get("pixels"), fl.get("scale")
+            if f:
+                self.pose(f, *place)
+                if scale:
+                    self.set_scale(f, scale)
+                if pixels and f.frame:
+                    m = self.margin
+                    self.command(cmd="geometry", id=f.id, x=f.slot.pos[0] + m / f.scale,
+                                 y=f.slot.pos[1] + m / f.scale, w=pixels[0] / f.scale, h=pixels[1] / f.scale)
+                return
+            if pixels:
+                scale = scale or output_scales().get(ev["output"], 1.0)
+                ev = dict(ev, frame=dict(ev["frame"], w=pixels[0] / scale, h=pixels[1] / scale))
+            else:
+                scale = None
+            self.float_window(ev, place=place, mpp=fl.get("mpp"), scale=scale)
+            return
+        n = int(e.get("screen", 1)) - 1
+        if not 0 <= n < self.screens_n:
+            n = 0
+        name = f"WL-{n}"
+        x0, y0, ow, oh = output_rects().get(name, (0, 0, 0, 0))
+        rx, ry, rw, rh = (e.get("rect") or [100, 100, ev["frame"]["w"], ev["frame"]["h"]])[:4]
+        if ow and oh:  # keep it on the screen if the screen got smaller
+            rw, rh = min(rw, ow), min(rh, oh)
+            rx, ry = max(0, min(rx, ow - rw)), max(0, min(ry, oh - rh))
+        frame = {"x": x0 + rx, "y": y0 + ry, "w": rw, "h": rh}
+        if f:
+            self.dock(f, frame=frame, output=name, maximized=bool(e.get("maximized")))
+        else:
+            self.command(cmd="place", id=ev["id"], output=name, x=frame["x"], y=frame["y"], w=rw, h=rh,
+                         maximized=bool(e.get("maximized")))
+
     # ------------------------------------------------------------ popups and dialogs
 
     def sub(self, ev):
@@ -507,11 +1031,25 @@ class Daemon:
                 return s.window
         return None
 
-    def request(self, text):
+    def request(self, text, sender=None):
         words = text.split()
         if not words:
             return "error empty"
         cmd, rest = words[0], words[1:]
+        if cmd == "launch" and len(rest) == 1:
+            return self.launch(app=rest[0])[0]
+        if cmd == "run" and rest:
+            try:
+                argv = json.loads(text.split(None, 1)[1])
+            except ValueError:
+                return "error run takes a JSON list"
+            if not isinstance(argv, list) or not argv or not all(isinstance(a, str) for a in argv):
+                return "error run takes a JSON list"
+            return self.launch(argv=argv)[0]
+        if cmd == "windows":
+            return self.capture(sender)
+        if cmd == "profile" and rest:
+            return self.open_profile(text.split(None, 1)[1])
         if cmd == "list":
             return "ok " + " ".join(f"{s.output}:{s.window.id if s.window else '-'}" for s in self.slots)
         if cmd == "quit":
@@ -520,6 +1058,20 @@ class Daemon:
         if cmd in ("float", "dock") and (not rest or rest[0] == "active"):
             self.command(cmd="request-active")
             return "ok"
+        if cmd == "float" and rest == ["pointer"]:
+            self.command(cmd="request-pointer")
+            return "ok"
+        if cmd == "dock" and rest == ["all"]:
+            for f in list(self.floats.values()):
+                self.dock(f)
+            return "ok"
+        if cmd == "front" and len(rest) == 1 and rest[0].isdigit():
+            f = self.by_panel(int(rest[0]))
+            if f:
+                self.command(cmd="activate", id=f.id)
+            else:  # a screen: its output is WL-<N - 1>
+                self.command(cmd="activate-output", output=f"WL-{int(rest[0]) - 1}")
+            return "ok"
         if cmd in ("dock", "close", "resize", "scale") and rest and rest[0].isdigit():
             f = self.by_panel(int(rest[0]))
             if not f:
@@ -539,7 +1091,8 @@ class Daemon:
             if not ev:
                 return f"error no window {rest[0]}"
             if cmd == "float":
-                self.float_window(ev)
+                # The script's word for where it is now: a window on a screen isn't followed here.
+                self.command(cmd="request-float", id=rest[0])
             elif cmd == "dock" and rest[0] in self.floats:
                 self.dock(self.floats[rest[0]])
             elif cmd == "close":
@@ -575,6 +1128,34 @@ class Service(dbus.service.Object):
         self.daemon.wait(reply)
 
 
+def watch_apps(daemon):
+    """Keep Launch as Standalone's desktop file copies up to date (float/ft_apps.py), in a
+    session that reads them (the session script puts them in XDG_DATA_DIRS)."""
+    ours = os.path.realpath(ft_apps.OUT_ROOT)
+    if ours not in (os.path.realpath(d) for d in os.environ.get("XDG_DATA_DIRS", "").split(":") if d):
+        return
+    later = []
+
+    def write():
+        later.clear()
+        try:
+            log(f"Launch as Standalone: {ft_apps.write_all()} apps")
+        except OSError as e:
+            log(f"Launch as Standalone: {e}")
+        return False
+
+    def changed(*_):
+        if not later:  # wait for an install or update to settle
+            later.append(GLib.timeout_add_seconds(3, write))
+    write()
+    daemon.app_monitors = []
+    for d in ft_apps.app_dirs():
+        if os.path.isdir(d):
+            m = Gio.File.new_for_path(d).monitor_directory(Gio.FileMonitorFlags.NONE, None)
+            m.connect("changed", changed)
+            daemon.app_monitors.append(m)
+
+
 def main():
     conf = ft_layout.read_conf()
     p = argparse.ArgumentParser(description="Floating windows for the Frametop desktop")
@@ -598,6 +1179,7 @@ def main():
     sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
     sock.bind("\0" + args.socket)
     sock.setblocking(False)
+    daemon.sock = sock
 
     def readable(*_):
         while True:
@@ -605,8 +1187,8 @@ def main():
                 data, sender = sock.recvfrom(4096)
             except BlockingIOError:
                 return True
-            reply = daemon.request(data.decode(errors="replace").strip())
-            if sender:
+            reply = daemon.request(data.decode(errors="replace").strip(), sender)
+            if sender and reply is not None:
                 try:
                     sock.sendto(reply.encode(), sender)
                 except OSError:
@@ -615,6 +1197,7 @@ def main():
 
     log(f"{args.screens} screens, {args.slots} floating slots (WL-{args.screens} and up), margin {args.margin} px")
     daemon.load_script(bus)
+    watch_apps(daemon)
     daemon.loop.run()
 
 

gaze/README.md

@@ -2,18 +2,20 @@
 
 The Steam Frame's eye tracking as pointer input: a gaze mode for the 3D mouse (the pointer goes where you look, and the mouse does the last bit), and the tools to calibrate and measure it.
 
-- `ft-gaze` (C++, OpenVR, runs in the dev container) reads the eye tracker and prints one JSON line per sample (90 Hz). For each source, it gives the gaze direction relative to the head and the Frametop screen pixel it lands on.
+- `ft-gaze` (C++, OpenVR, runs in the dev container) reads the eye tracker and prints one JSON line per sample (90 Hz). For each source, it gives the gaze direction relative to the head and the Frametop screen pixel it lands on. The gaze service asks it for only the sources it uses (`--sources`, and `sources LIST` on its stdin): our tracker and mmap set 1 with Own tracker, about 700 bytes a line instead of 1.3 KB, and every source while a check or the calibration runs. So SteamVR's gaze action, the only source that calls into vrserver (twice a sample), is read only then. The probe gets them all.
 - `gazecal.py` has what the probe and the gaze service share: the correction models, filters, and the reader for SteamVR's eye tracking log.
 - `tracker/` is our own eye tracker, an alternative to SteamVR's: `ft-eyes` finds the pupils and glints in the eye-camera frames that `ft-eyegrab` (a small root service) copies out of SteamVR's tracker. See "Our own eye tracker" below.
-- `probe/ft-gazeprobe` (GTK 4, host Python) is a fullscreen playground. It runs ft-gaze, draws where you're looking, measures accuracy, and tries out hold-to-adjust clicking with a calibration that learns from your adjustments.
+- `probe/ft-gazeprobe` (GTK 4, host Python) is a fullscreen playground, for developing the gaze tracking: day to day, the calibration and the checks run in the headset panel (Quick check, Calibrate, and Check headset fit on the Gaze page). It runs ft-gaze, draws where you're looking, measures accuracy, and tries out hold-to-adjust clicking with a calibration that learns from your adjustments.
+
+Day to day, install the gaze service, then turn gaze mode on and calibrate on the Gaze page of Frametop Input Settings (Calibrate). The installer offers the gaze service (`gaze/run.sh install`) and then our own tracker (`gaze/tracker/install.sh`), which gaze mode uses once it's installed; the probe is installed by hand.
 
 ```
-gaze/build.sh                 # build ft-gaze
-gaze/probe/install.sh         # build, and add Frametop Gaze Probe to the app menu
-gaze/probe/ft-gazeprobe --screen 1
-gaze/run.sh install           # the gaze service, with SteamVR
+gaze/run.sh install           # the gaze service: builds ft-gaze and the panel, starts with SteamVR
 gaze/ft-gazectl on            # the pointer follows your gaze (off: the mouse alone)
 gaze/tracker/install.sh       # our own eye tracker's frame grabber (asks for sudo)
+gaze/build.sh                 # build ft-gaze and the panel by hand
+gaze/probe/install.sh         # development: build, and add Frametop Gaze Probe to the app menu
+gaze/probe/ft-gazeprobe --screen 1
 ```
 
 ## Gaze pointer
@@ -21,15 +23,19 @@ gaze/tracker/install.sh       # our own eye tracker's frame grabber (asks for su
 Gaze as an input method for the whole desktop, without replacing anything of SteamVR's:
 
 - `ft-gazed` (host Python, a user service: `gaze/run.sh install`) runs ft-gaze and corrects its gaze. Two settings on the Gaze page of Frametop Input Settings (`GAZE_TRACKER` and `GAZE_EYE` in `~/.config/frametop.conf`, read again when the file changes) pick whose eye tracking it uses and how it weights the eyes:
-  - **Eye tracker:** SteamVR's (the default), or our own (Own tracker: see "Our own eye tracker" below). The gaze service runs ours while this is on. It keeps its own calibration: calibrate it in the probe with the tracker toggle on Own tracker. The gaze pointer's settings (hand back, nudges, hold to drag, the dot) are the pointer helper's, so they're the same with either.
+  - **Eye tracker:** our own (Own tracker: see "Our own eye tracker" below) or SteamVR's. The default, `GAZE_TRACKER=auto`, is ours when it's installed (its frame grabber, and ft-eyes' Python in the gaze service's checkout), else SteamVR's, and it switches when ours is installed or removed; picking one on the Gaze page sets it for good. The gaze service runs ours while it's the one in use. It keeps its own calibration: with Own tracker chosen, Calibrate on the Gaze page calibrates it. The gaze pointer's settings (hand back, nudges, hold to drag, the dot) are the pointer helper's, so they're the same with either.
   - **Eye bias:** Auto, Left, or Right. The gaze combines both eyes, each calibrated on its own, because their errors partly cancel: on 306 clicks with our tracker, the eyes' sideways errors were correlated -0.37, and both together were 0.65 degrees off (median) against 0.96 for the left eye alone and 1.11 for the right. So Left or Right leans instead of choosing: that eye counts twice as much as the other (0.03 degrees worse there toward the better eye, 0.13 toward the worse). Auto weights each eye by the inverse square of how far off it was at your last 20 nudges, once each eye has 5, and evenly before that. Each eye's miss is measured before that nudge teaches anything, so each is a fresh test. The calibration's own fit isn't used for this: on SteamVR's test of 2026-09-29, the calibration dots said the left eye was the better one, and new spots said the right. Either eye carries the gaze alone while the other is closed or lost.
 
   With SteamVR, each eye is its own reading (set 2), corrected by its calibration from the probe (the Left eye and Right eye sources) plus what the pointer has taught that eye since. On that test, the two eyes each calibrated and averaged were 1.70 degrees off (median; mean 1.62) against 1.72 (mean 1.84) for SteamVR's combined gaze with its calibration. A calibration from before the probe had the eyes as sources, or `--source`, uses the older path. That path runs on SteamVR's combined gaze (mmap set 1), corrected as a whole. When the tracker loses one eye (its variance for that eye jumps from about 0.001 to 0.02), the gaze comes from the other eye instead: that eye's own reading (set 2) plus what it usually reads against the combined gaze, learned while both eyes are seen, in 10 degree cells of where it looks. Set 1 keeps going on one eye too, but it holds the lost eye's yaw where it was, so the gaze moves half as far sideways as your eyes do. On a recording, one eye alone came out a median 0.8 degrees from both eyes' gaze over a steady look, a little more jittery.
 
   Looks down past the screens (under 20 degrees down, on no Frametop screen: a glance at the keyboard) aren't sent, so the pointer stays where it was instead of following you down, and eyes lost there aren't counted. It drops blinks (both eyes closing or lost), smooths with a fixation lock, and sends the result to the pointer helper 90 times a second. It follows SteamVR's eye tracking log, and when the headset goes back on (SteamVR starts its eye model over, and the error moves), older lessons count less, so the first few after relearn the offset.
-- The pointer helper's **gaze mode** (off by default: the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a mouse or controller button mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look. Move the mouse and it's the mouse's, from where the gaze put it, for the last bit. Look well away (5 degrees) and the gaze takes it back. A press isn't sent at once: the pointer stops where the gaze put it, and if that's wrong, drag it onto what you meant with the button still held; the click happens where you let go. To drag something, hold the press still for half a second first (`POINTER_GAZE_HOLD`), then move. Outside games the pointer stays on while gaze mode is on. The dot only shows while the mouse moves it, while a press is held, and as a pulse when you click.
-- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it a little (0.2 to 8 degrees) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the probe's one-dot check does the same). The helper only sends nudges up to `POINTER_GAZE_NUDGE_MAX` (8 degrees), and right after putting the headset back on our tracker can be further off than that. If so, raise it for a moment, or do the probe's check. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
+- The pointer helper's **gaze mode** (off by default: the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a mouse button or key combination mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look, and the mouse does the last bit. By default the mouse moves it only while a button is held (see "The mouse only corrects" below). With `POINTER_GAZE_MOUSE_MOVE=free`, moving the mouse takes the pointer, from where the gaze put it, and looking well away (5 degrees) gives it back to the gaze. A press isn't sent at once: the pointer stops where the gaze put it, and if that's wrong, drag it onto what you meant with the button still held; the click happens where you let go. To drag something, hold the press still for half a second first (`POINTER_GAZE_HOLD`), then move. Outside games the pointer stays on while gaze mode is on, until a controller is picked up. The dot shows all the time (`POINTER_GAZE_DOT=moving`: only while the mouse moves it, while a press is held, and as a pulse when you click). Gaze mode works with the mouse and the keyboard, not the controllers ([docs/gaze-controllers.md](../docs/gaze-controllers.md) explains why).
+- **The mouse only corrects** (the default; the Gaze page's Mouse movement switch, `POINTER_GAZE_MOUSE_MOVE=held`): while the gaze has the pointer, moving the mouse does nothing. The buttons work like Meta+J and Meta+K: press and hold one and the pointer stops where you look; move the mouse onto what you meant and let go to click there (a left or a right click). Held still for half a second, a press is a real one (to drag). Once you've moved, the left button alone only clicks: press the right one while still holding the left to start a drag there; it lasts while either button is held. Press the right one again (a double right click, the left still held) to pan and tilt what you're dragging, as a right press does during any drag. A bumped or drifting mouse can't pull the pointer away, and every mouse move is a correction, so the tracker only learns from real ones. With the gaze stale for a second (the tracker stopped, eyes lost), in a game, or with the headset off, the mouse moves the pointer as usual. `free` (the switch off) lets the mouse take the pointer any time.
+- **Keyboard clicks** (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, `POINTER_KEY_TAP`) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse, under the same limit: past `POINTER_GAZE_NUDGE_MAX` it opens the quick check instead). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; the drag lasts while either key is held. Meta+K during a Meta+J drag (again, after starting it with Meta+K: a double Meta+K) pans and tilts what you're dragging while it's held: turn your head to turn it.
+- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it (0.2 degrees or more, and the correction within `POINTER_GAZE_NUDGE_MAX`: 55 degrees by default, half of the 109 the headset shows across, and 1 to 110; the same limit for mouse, keyboard, and pinch clicks) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the quick check's dot does the same). A correction past `POINTER_GAZE_NUDGE_MAX` isn't learned: the helper asks ft-gazed for the quick check instead ("recheck", after its 2-minute cooldown). Tested on our tracker's 409 clicks since its Sep 29 calibration: a one-dot check set from any one of them put the next 2 minutes' clicks within 15 degrees (99% within 4.2) and the next 10 minutes' within 25 (the far ones after the headset moved), so the check gets back well under it. The limit used to be 8 degrees, and live on 2026-10-01 our tracker was 12 off after the headset went on, so every correction was dropped. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
+- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. Our tracker's first calibration has no gaze to go on, since ft-eyes maps pupils to a gaze only once it has a calibration: it opens once SteamVR's tracker sees an eye and ft-eyes answers, and a click takes the 0.6 s up to it, as long as ft-eyes saw each pupil held still then (before 2026-10-05 it waited for a gaze, so a fresh install could never calibrate ours). A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
 - Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
+- **Idle while the gaze isn't used:** ft-gaze and our own tracker run only while gaze mode is on and someone wears the headset (the pointer helper says both: SteamVR drops the headset's activity level as soon as it comes off), while a check or the calibration is open or asked for, or while the Gaze page of Frametop Input Settings is open (it renews a `wake` lease). 30 seconds after the last use they stop, and our frame grabber goes idle with our tracker. With our tracker, that saves over half a core: on 2026-10-02, with gaze mode off, ft-eyes took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. A check asked for while it idles starts the tracker and opens once it sends. When the gaze is used again, it takes a few seconds to come back, and our tracker's first click re-seats it, as after the headset was off: so the quick check opens when gaze mode comes on after the service idled, as it does when you put the headset on. `ft-gazectl status` says `"awake"`, and `"idle"` says why it isn't. A stand that covers the proximity sensor makes the headset seem worn, so with gaze mode on it doesn't idle there.
 
 Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction, the correction at the time, and how far off it was.
 
@@ -40,7 +46,7 @@ Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction,
 | SteamVR action | An `eyetracking` action bound to `/user/head/eyetracking` (`actions/`), read with `IVRInput::GetEyeTrackingDataRelativeToNow`. This is the supported way. |
 | mmap set 1, set 2 | `/dev/shm/eye-server.mmap`, which SteamVR's eyetracking process writes for the HMD driver (`driver_cv.so`). It has two sets of per-eye directions in head space: set 1 is filtered, and its two eyes always share one pitch; set 2 is each eye's own reading. After each set come the tracker's variances for each eye, and at the end each eye's raw measurement and its variance (the tracker's confidence in that frame), which ft-gaze passes on for the fit check. |
 | Left eye, right eye | Each eye alone, from set 2: calibrate and test them to see what one eye is worth against both. The layout is undocumented (offsets are in `ft-gaze.cpp`) and may change with a SteamVR update. ft-gaze maps it read-only; the file also carries calibration clicks to the tracker and must never be written. |
-| Own tracker | Our own tracker (`tracker/`, experimental; see "Our own eye tracker"). It keeps its own calibration, not SteamVR's: the probe's calibration with the tracker toggle on Own tracker fits it (its dots go out to the Calibration ring angle each way, on an oval, since the fit goes wrong past its dots), and practice clicks teach it how far the headset has moved on your face since. After the headset was off, the probe first asks for one look at a centre dot, which resets that. With Own tracker on, the probe hides SteamVR's gaze and draws a red dot where each eye alone puts it, and asks the gaze service to keep the tracker running. The gaze pointer can use it too (Eye tracker: Own tracker, on the Gaze page of Frametop Input Settings). ft-gaze reports it as `own` while it's running, and as `{"ok":0}` otherwise. |
+| Own tracker | Our own tracker (`tracker/`, experimental; see "Our own eye tracker"). It keeps its own calibration, not SteamVR's: Calibrate on the Gaze page fits it while Eye tracker is Own tracker (eight dots to a ring instead of six, on a slight oval, since the fit goes wrong past its dots; the probe's calibration with its tracker toggle on Own tracker does the same), and clicks teach it how far the headset has moved on your face since. After the headset was off, one look at a centre dot (the quick check, or the probe's first dot) resets that. With Own tracker on, the probe hides SteamVR's gaze and draws a red dot where each eye alone puts it, and asks the gaze service to keep the tracker running. The gaze pointer can use it too (Eye tracker: Own tracker, on the Gaze page of Frametop Input Settings). ft-gaze reports it as `own` while it's running, and as `{"ok":0}` otherwise. |
 
 The tracker stops when the headset is off your head. SteamVR also calibrates gaze on its own from laser-mouse clicks, treating each click as a spot you were looking at. That includes mouse clicks through the Frametop pointer, so a click where the pointer's dot isn't what you're looking at teaches SteamVR a wrong sample (it only takes clicks within 5 degrees of your gaze). In the probe, use Enter or Space as the trigger: keys don't go through SteamVR's laser. See `Accept usercal` in `~/.local/share/Steam/logs/eyetracking.txt`. When the tracker loses an eye, the same log says `CEyePoseUKF L: Large dt` (or `R`) as it starts that eye over.
 
@@ -50,23 +56,25 @@ The tracker stops when the headset is off your head. SteamVR also calibrates gaz
 
 - `ft-eyegrab` (C, root, the system service `frametop-eyegrab.service`) copies the eye-camera frames (512x400, 90 fps per eye) out of the DMA-BUFs SteamVR's `eyetracking` process holds into `/dev/shm/frametop-eyes-cams`, owned by you. It maps them read-only, and it only copies while someone touches `/dev/shm/frametop-eyes-want` (ft-eyes and the recorder do, every second). Otherwise it holds none of the tracker's buffers. Its unit keeps only the capabilities that needs (`CAP_SYS_PTRACE`, `CAP_DAC_READ_SEARCH`, `CAP_CHOWN`). `gaze/tracker/install.sh` builds it and installs it to `/etc/frametop` with sudo, which it asks for (`uninstall`, `status`, and `log` too).
 - `ft-eyes` (Python with numpy and OpenCV, in the dev container: `gaze/tracker/build.sh` puts the pinned `requirements.txt` in `gaze/tracker/build/venv`) finds each eye's pupil (dark threshold, closing, ellipse fit) and glint pair (`eyes_pupil.py`), and maps them to a gaze with a quadratic fit per eye (`eyes_model.py`). It follows the headset moving on your face with a per-eye shift, which your clicks teach, and uses the glints only to notice a sudden jump. It publishes the gaze in `/dev/shm/frametop-eyes-gaze` (ft-gaze's source `own`) and takes calibration dots and clicks on `@ft_eyes`. The gaze service runs it while Eye tracker is Own tracker, or while the probe uses it. State (the calibration, each eye's shift, the clicks) is in `~/.local/state/frametop/gaze/eyes/`.
-- `lab/` has the tools for improving it on recordings. `ft-eyes-record NAME` (or `ft-eyes-session`, with SteamVR's gaze alongside) records the cameras. `ft-eyes-score` fits and scores on recordings against the probe's practice clicks. `ft-eyes-e2e` runs the whole live path on two recordings (calibrate on one, click through the other). `ft-eyes-replay` plays a recording into a scratch share. Heavy ones run on a PC through `frame-job` (`gaze/tracker/.frame-job`). `lab/py` runs them with that Python (in the dev container on the Frame; frame-job's setup makes the same venv on the PC).
+- `lab/` has the tools for improving it on recordings. `ft-eyes-record NAME` (or `ft-eyes-session`, with SteamVR's gaze alongside) records the cameras. `ft-eyes-score` fits and scores on recordings against the probe's practice clicks. `ft-eyes-e2e` runs the whole live path on two recordings (calibrate on one, click through the other). `ft-eyes-replay` plays a recording into a scratch share. Heavy ones are meant for a PC: if you have `frame-job` (a personal tool, not in this repo), `gaze/tracker/.frame-job` sends them there. `lab/py` runs them with that Python (in the dev container on the Frame; on a PC, the same venv from `requirements.txt`, which frame-job's setup makes).
 
 Ground rules, for anyone changing it:
 
 - **Clean room.** Nothing of Valve's goes in: we don't decompile, disassemble, or patch the `eyetracking` binary or its network weights, and we don't copy their code or weights. Its public output (eye-server.mmap, read-only) is fair game as a baseline and as labels, and so are published papers and openly licensed pupil detectors (check each one's license: PuRe, PuReST, ElSe, and ExCuSe are non-commercial only).
 - **Root only reads.** ft-eyegrab never writes to, stops, or signals the `eyetracking` process, vrserver, or vrcompositor, never opens `/dev/adsp`, `/dev/cdsp`, or `/dev/spidev0.1`, and never writes to `/dev/shm/eye-server.mmap` (it also carries calibration clicks into SteamVR's tracker), `/opt`, or `/persist`.
 - **Eye images are biometric data.** Recordings live outside the repo, in `~/.local/share/frametop/eyes/captures` (0700), and `.gitignore` catches stray frame dumps. They go nowhere but the machine that runs your offline jobs.
-- **Mind the headset's budget.** Finding a pupil takes about 0.4 ms a frame while ft-eyes follows it, and 1.4-2.1 ms when it searches the whole frame. Replays, scoring, and training go to a PC.
+- **Mind the headset's budget.** Finding a pupil takes about 0.4 ms a frame while ft-eyes follows it, and 1.4-2.1 ms when it searches the whole frame. ft-eyes keeps OpenCV and numpy to one thread: their pools of one per core spun idle workers at about a quarter of a core, for frames this small. It also runs at nice 10 with SCHED_BATCH, and ft-gaze at nice 5 (not batch, since each sample goes on to the pointer): both run in the dev container's podman scope, out of reach of the gaze service's unit, on the cores vrcompositor and vrserver use at nice 0. Replays, scoring, and training go to a PC.
 
 ## Headset fit
 
-The probe's Headset fit mode (Check headset fit on the Gaze page, or `ft-gazeprobe --mode fit`) shows, for each eye, whether the tracker has it, how open it is, and the tracker's confidence in it, and a map of where you looked coloured by how often it lost that eye there. Hints under the maps say which eye gets lost where, and what to try. Enter runs a guided check: dots around the screen, then looking down at the keyboard, up, left and right. R starts over. Adjust the headset while you watch it.
+Check headset fit on the Gaze page opens it in the headset panel: a card per eye (tracked or lost, the tracker's signal, how much of the last 10 s it was seen) and the hints, live while you adjust the headset. A left click or Meta+J runs the guided check (dots, then looks down, up, left and right), and a right click or Meta+K closes it. The probe's Headset fit mode (`ft-gazeprobe --mode fit`, for development) has the same check with maps: it shows, for each eye, whether the tracker has it, how open it is, and the tracker's confidence in it, and a map of where you looked coloured by how often it lost that eye there. Hints under the maps say which eye gets lost where, and what to try. Enter runs a guided check: dots around the screen, then looking down at the keyboard, up, left and right. R starts over. Adjust the headset while you watch it.
 
 Losing an eye is usually about where you look, not the tracker. On this Frame the left eye was lost 57 to 64 % of the time looking 30 to 50 degrees down (at the keyboard) and the right eye never; at screen height both were seen over 98 % of the time. Looking down, the lids come down over the eyes. That's harmless, since the gaze service ignores looks down past the screens: they show on the maps, but not in the counts or as a problem.
 
 ## Probe
 
+The probe is a development tool (in the Gaze page's overflow menu): calibration experiments, accuracy tests, and practice modes. Users calibrate and check in the headset panel instead.
+
 The trigger is Enter, Space, or a mouse button. Right-click anywhere in the window (or press the Menu key or Shift+F10) for a menu with Run calibration, Start accuracy test, Calibrate from last test, Reset calibration, the modes, the panel, fullscreen, and Quit. The buttons at the top right show and hide the panel, leave fullscreen, and quit. The arrow in the panel's title bar collapses it to just that bar, so the dot and targets behind it stay visible; the collapsed bar stays through tests. The keys do the same (Tab, C, F11, Esc), but only after you click the window once, since Frametop sends typing to the panel you clicked last. If ft-gaze stops, the probe starts it again after 3 s and shows why it stopped. Windowed mode stays on the screen it was on, and a small KWin script tells the probe where the window is, so the dot and targets are still in the right place.
 
 - **Run calibration (start here):** the initial calibration, modeled on Apple Vision Pro's eye setup. Face the centre and keep your head still. Look at one dot and press the trigger, then at each of six dots in a circle. That happens in three rounds, and the screen goes dark, then medium, then bright, because pupil size changes with brightness and the tracker's error with it. Each round turns the ring 20 degrees, and the middle round's ring is half the size, so the 21 dots cover the middle, halfway out, and the edge of your view. The ring's size is `Calibration ring` (degrees, 20 by default, less if the window is too small). Error grows toward the edge, and the calibration can only correct as far out as it has seen dots. The current dot is a bright pulsing dot with a point in the middle; finished dots fade to specks, so your eyes don't go back to them. Samples from blinks and from moments when the tracker lost an eye are dropped: openness under half of what it was during that look (not a fixed level, because your lids come down when you look down, and you squint in the bright round), or the angle between the eyes jumping more than 1.5 degrees from its median (that angle depends on how far away you're looking, so only a jump counts). Each dot is measured with medians, so one bad sample can't fail it. A look that lands where the gaze was for another dot of the round is refused as a look at the wrong dot. Mouse clicks don't count during a calibration run or a test: use Enter or Space. If a dot still fails, the message says why and the next try listens longer. After two failures, S (or the menu) skips the dot. Every attempt is logged to `calibration-attempts.jsonl`. At the end it fits every source's calibration from all the dots, replacing what it had learned (quadratic if the model was none). Esc cancels. The run is saved as `calibration-*.json`. With "Test after calibration" on (the default), the accuracy test starts right after, on new spots.

gaze/build.sh

@@ -1,5 +1,7 @@
 #!/usr/bin/env bash
-# Build ft-gaze in the dev container on the Frame (gaze/build/ft-gaze; it also runs there).
+# Build ft-gaze and the calibration panel ft-gazepanel in the dev container on the Frame
+# (gaze/build/; they also run there). The panel draws its text with stb_truetype (public
+# domain, one header, pinned as in screens/build.sh).
 # The eye tracking API (IVRInput::GetEyeTrackingDataRelativeToNow) is newer than the header
 # shipped with SteamVR's samples, so this uses the pinned public header ft-screens fetches.
 set -euo pipefail
@@ -10,4 +12,9 @@ openvr=v2.15.6
 [ -f build/include/openvr-$openvr ] || { curl -fsSL "https://raw.githubusercontent.com/ValveSoftware/openvr/$openvr/headers/openvr.h" -o build/include/openvr.h && touch build/include/openvr-$openvr; }
 g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -Wno-missing-field-initializers -Ibuild/include -I../pointer/common \
   -o build/ft-gaze ft-gaze.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64 -lpthread
-echo "built build/ft-gaze"'
+stb=2c980bb59875b0d32144a71867fbdebb2f77cd20
+[ -f build/include/stb-$stb ] || { curl -fsSL "https://raw.githubusercontent.com/nothings/stb/$stb/stb_truetype.h" -o build/include/stb_truetype.h && touch build/include/stb-$stb; }
+g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -Wno-missing-field-initializers -Ibuild/include $(pkg-config --cflags gbm libdrm) \
+  -o build/ft-gazepanel panel/ft-gazepanel.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64 \
+  $(pkg-config --libs gbm libdrm) -lpthread
+echo "built build/ft-gaze build/ft-gazepanel"'

gaze/fitcheck.py

@@ -1,5 +1,5 @@
-"""fitcheck: how well the eye tracker sees each eye, for fitting the headset (ft-gazeprobe's
-Headset fit mode).
+"""fitcheck: how well the eye tracker sees each eye, for fitting the headset (the headset
+panel's fit check, gazecheck.py, and ft-gazeprobe's Headset fit mode).
 
 From each ft-gaze sample it takes, per eye, whether the tracker has that eye (its variance
 for the eye's direction, "unc", under EYE_LOST; see gazecal), how open the eye is, and the

gaze/ft-gaze.cpp

@@ -1,10 +1,17 @@
 // ft-gaze: the headset's eye tracking as rays and Frametop screen pixels (OpenVR overlay
-// client, runs in the dev container). An experiment for gaze input; ft-gazeprobe reads it.
+// client, runs in the dev container). The gaze service (ft-gazed) and the gaze probe run it.
 //
 // Every eye tracker sample (90 Hz) becomes one JSON line on stdout with each gaze source
 // hit-tested against the Frametop screens:
 //
-// Options: -v (log action errors), --watch-stdin (quit when stdin closes).
+// Options: -v (log action errors), --watch-stdin (quit when stdin closes; until then, a line
+// "sources LIST" on stdin switches the sources as --sources does), --sources LIST (comma-
+// separated: action, mmap1, mmap2, left, right, own, and eye for EYE; or all, the default).
+// Sources left out are read not at all and print as {"ok":0} ("eye" as null), so every line
+// keeps the same keys. The gaze service asks for the ones it uses (own and mmap1 with our
+// tracker), and all of them while a check or the calibration runs. Only the action costs
+// SteamVR anything (two calls into vrserver per sample), and a line with every source is
+// about 1.5 KB, 130 KB a second through podman's relay.
 //
 //   {"t":<sample time, CLOCK_MONOTONIC_RAW s>,"age":<ms old when read>,"n":<sample counter>,
 //    "head":{"yaw":..,"pitch":..,"hit":HIT},          head forward ray (for head nudging)
@@ -58,6 +65,7 @@
 
 #include <algorithm>
 #include <atomic>
+#include <cerrno>
 #include <chrono>
 #include <climits>
 #include <cmath>
@@ -72,6 +80,7 @@
 
 #include <fcntl.h>
 #include <sys/mman.h>
+#include <sys/resource.h>
 #include <sys/socket.h>
 #include <sys/stat.h>
 #include <sys/un.h>
@@ -389,6 +398,30 @@ std::string SrcJson(const std::vector<Screen> &screens, const vr::HmdMatrix34_t
     return buf + extra + "\"hit\":" + HitJson(screens, head, yaw, pitch) + "}";
 }
 
+// Sources (--sources, "sources LIST" on stdin): a bit each.
+enum : unsigned { kAction = 1, kMmap1 = 2, kMmap2 = 4, kLeft = 8, kRight = 16, kOwn = 32, kEye = 64, kAll = 127 };
+
+bool ParseSources(const std::string &list, unsigned &mask) {
+    static const struct {
+        const char *name;
+        unsigned bit;
+    } names[] = {{"action", kAction}, {"mmap1", kMmap1}, {"mmap2", kMmap2}, {"left", kLeft},
+                 {"right", kRight},   {"own", kOwn},     {"eye", kEye},     {"all", kAll}};
+    unsigned m = 0;
+    size_t at = 0;
+    while (at <= list.size()) {
+        const size_t comma = std::min(list.find(',', at), list.size());
+        const std::string name = list.substr(at, comma - at);
+        bool known = false;
+        for (const auto &n : names)
+            if (name == n.name) m |= n.bit, known = true;
+        if (!known) return false;
+        at = comma + 1;
+    }
+    mask = m;
+    return true;
+}
+
 // Head poses of the last half second, so a sample can use the pose at its own time.
 class PoseHistory {
 public:
@@ -426,17 +459,42 @@ std::string ExeDir() {
 
 int main(int argc, char **argv) {
     bool verbose = false, watchStdin = false;
+    std::atomic<unsigned> sources{kAll};
     for (int i = 1; i < argc; ++i) {
         if (std::strcmp(argv[i], "-v") == 0) verbose = true;
         if (std::strcmp(argv[i], "--watch-stdin") == 0) watchStdin = true;
+        if (std::strcmp(argv[i], "--sources") == 0 && i + 1 < argc) {
+            unsigned m;
+            if (ParseSources(argv[++i], m))
+                sources = m;
+            else
+                std::fprintf(stderr, "ft-gaze: --sources %s: unknown source (all used)\n", argv[i]);
+        }
     }
+    // Nice 5, before any thread starts (they inherit it): we run in the dev container's podman
+    // scope, beside vrcompositor and vrserver at nice 0, and the gaze service's unit doesn't
+    // reach us. Not SCHED_BATCH, as ft-eyes is: that would let each wakeup wait out another
+    // task's turn, and each sample goes on to the pointer.
+    errno = 0;
+    const int nice0 = getpriority(PRIO_PROCESS, 0);
+    if (errno == 0 && nice0 < 5 && setpriority(PRIO_PROCESS, 0, 5) != 0)
+        std::fprintf(stderr, "ft-gaze: nice: %s\n", std::strerror(errno));
     // --watch-stdin: quit when stdin closes. The probe runs us through distrobox, which
     // passes neither its signals nor a closed stdout on to us, but does pass stdin's end.
+    // Lines on stdin until then: "sources LIST".
     std::atomic<bool> stdinClosed{false};
     if (watchStdin)
-        std::thread([&stdinClosed] {
+        std::thread([&stdinClosed, &sources] {
             char c[256];
-            while (read(0, c, sizeof c) > 0) {
+            std::string line;
+            ssize_t n;
+            while ((n = read(0, c, sizeof c)) > 0) {
+                line.append(c, size_t(n));
+                for (size_t nl; (nl = line.find('\n')) != std::string::npos; line.erase(0, nl + 1)) {
+                    unsigned m;
+                    if (line.compare(0, 8, "sources ") == 0 && ParseSources(line.substr(8, nl - 8), m)) sources = m;
+                }
+                if (line.size() > 4096) line.clear();  // no newline in sight: not ours
             }
             stdinClosed = true;
         }).detach();
@@ -461,7 +519,10 @@ int main(int argc, char **argv) {
     vr::VRActionSetHandle_t set = vr::k_ulInvalidActionSetHandle;
     input->GetActionHandle("/actions/gaze/in/gaze", &gaze);
     input->GetActionSetHandle("/actions/gaze", &set);
-    std::fprintf(stderr, "ft-gaze: action manifest %s: error %d\n", manifest.c_str(), int(me));
+    if (me == vr::VRInputError_None)
+        std::fprintf(stderr, "ft-gaze: action manifest %s: ok\n", manifest.c_str());
+    else
+        std::fprintf(stderr, "ft-gaze: action manifest %s: error %d\n", manifest.c_str(), int(me));
 
     EyeFile eyes;
     const bool haveMmap = eyes.Open();
@@ -476,9 +537,24 @@ int main(int argc, char **argv) {
     double lastEmit = 0;
     int actionErrors = 0;
     vr::EVRInputError lastActionError = vr::VRInputError_None;
+    // During a VR game, SteamVR's gaze action is left alone. With ft-gaze reading the eyes, SteamVR
+    // restarted its eye tracker every 10 s or so in a game, as if the headset came off, and each
+    // restart took input focus from the game: Beat Saber paused (PR #13). Of what ft-gaze reads,
+    // only the action reaches SteamVR (the mmap and our tracker are files), so gaze still works
+    // over the dashboard. Games are told apart the way ft-screens does it, by the scene app.
+    bool inGame = false;
+    double nextGameCheck = 0;
 
     while (true) {
         const double now = NowRaw();
+        if (now >= nextGameCheck) {
+            nextGameCheck = now + 0.5;
+            const bool game = vr::VRApplications()->GetCurrentSceneProcessId() != 0;
+            if (game != inGame)
+                std::fprintf(stderr, "ft-gaze: %s\n",
+                             game ? "a VR game is running: SteamVR's gaze action left alone" : "the VR game ended");
+            inGame = game;
+        }
         vr::TrackedDevicePose_t hp;
         sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &hp, 1);
         if (hp.bPoseIsValid) history.Add(now, hp.mDeviceToAbsoluteTracking);
@@ -491,6 +567,7 @@ int main(int argc, char **argv) {
 
         if (fresh && hp.bPoseIsValid) {
             lastEmit = now;
+            const unsigned want = sources;
             const auto list = screens.Get();
             const vr::HmdMatrix34_t &headNow = hp.mDeviceToAbsoluteTracking;
             vr::HmdMatrix34_t headThen = headNow;
@@ -499,24 +576,26 @@ int main(int argc, char **argv) {
             // SteamVR's action: a room-space origin and fixation point, turned into the head
             // frame so every source reports the same kind of angles.
             std::string action = "{\"ok\":0}";
-            vr::VRActiveActionSet_t active{};
-            active.ulActionSet = set;
-            active.nPriority = vr::k_nActionSetOverlayGlobalPriorityMin;
-            input->UpdateActionState(&active, sizeof active, 1);
-            vr::VREyeTrackingData_t e{};
-            const vr::EVRInputError ae =
-                input->GetEyeTrackingDataRelativeToNow(gaze, vr::TrackingUniverseStanding, 0, &e, sizeof e);
-            if (ae == vr::VRInputError_None && e.bActive && e.bValid) {
-                const Vec3 o{e.vGazeOrigin.v[0], e.vGazeOrigin.v[1], e.vGazeOrigin.v[2]};
-                const Vec3 t{e.vGazeTarget.v[0], e.vGazeTarget.v[1], e.vGazeTarget.v[2]};
-                const Vec3 dHead = RotateInverse(headNow, Normalize(t - o));
-                char extra[96];
-                std::snprintf(extra, sizeof extra, "\"tracked\":%d,\"dist\":%.3f,", int(e.bTracked), Length(t - o));
-                action = SrcJson(list, headNow, dHead, extra);
-            } else if (ae != lastActionError || (verbose && ++actionErrors % 90 == 1)) {
-                std::fprintf(stderr, "ft-gaze: action: error %d active %d valid %d\n", int(ae), int(e.bActive),
-                             int(e.bValid));
-                lastActionError = ae;
+            if (!inGame && (want & kAction)) {
+                vr::VRActiveActionSet_t active{};
+                active.ulActionSet = set;
+                active.nPriority = vr::k_nActionSetOverlayGlobalPriorityMin;
+                input->UpdateActionState(&active, sizeof active, 1);
+                vr::VREyeTrackingData_t e{};
+                const vr::EVRInputError ae =
+                    input->GetEyeTrackingDataRelativeToNow(gaze, vr::TrackingUniverseStanding, 0, &e, sizeof e);
+                if (ae == vr::VRInputError_None && e.bActive && e.bValid) {
+                    const Vec3 o{e.vGazeOrigin.v[0], e.vGazeOrigin.v[1], e.vGazeOrigin.v[2]};
+                    const Vec3 t{e.vGazeTarget.v[0], e.vGazeTarget.v[1], e.vGazeTarget.v[2]};
+                    const Vec3 dHead = RotateInverse(headNow, Normalize(t - o));
+                    char extra[96];
+                    std::snprintf(extra, sizeof extra, "\"tracked\":%d,\"dist\":%.3f,", int(e.bTracked), Length(t - o));
+                    action = SrcJson(list, headNow, dHead, extra);
+                } else if (ae != lastActionError || (verbose && ++actionErrors % 90 == 1)) {
+                    std::fprintf(stderr, "ft-gaze: action: error %d active %d valid %d\n", int(ae), int(e.bActive),
+                                 int(e.bValid));
+                    lastActionError = ae;
+                }
             }
 
             std::string m1 = "{\"ok\":0}", m2 = m1, left = m1, right = m1, eye = "null";
@@ -540,26 +619,33 @@ int main(int argc, char **argv) {
                     return std::string(b);
                 };
                 char extra[256];
-                std::snprintf(extra, sizeof extra, "\"dist\":%.3f,\"open\":[%.3f,%.3f],\"lr\":%.3f,", Length(s.fix1),
-                              s.open[0], s.open[1], lr(s.left1, s.right1));
-                m1 = SrcJson(list, headThen, s.left1 + s.right1, extra + eyes(s.left1, s.right1) + unc(s.var1));
-                std::snprintf(extra, sizeof extra, "\"lr\":%.3f,", lr(s.left2, s.right2));
-                m2 = SrcJson(list, headThen, s.left2 + s.right2, extra + eyes(s.left2, s.right2) + unc(s.var2));
-                left = SrcJson(list, headThen, s.left2);
-                right = SrcJson(list, headThen, s.right2);
+                if (want & kMmap1) {
+                    std::snprintf(extra, sizeof extra, "\"dist\":%.3f,\"open\":[%.3f,%.3f],\"lr\":%.3f,", Length(s.fix1),
+                                  s.open[0], s.open[1], lr(s.left1, s.right1));
+                    m1 = SrcJson(list, headThen, s.left1 + s.right1, extra + eyes(s.left1, s.right1) + unc(s.var1));
+                }
+                if (want & kMmap2) {
+                    std::snprintf(extra, sizeof extra, "\"lr\":%.3f,", lr(s.left2, s.right2));
+                    m2 = SrcJson(list, headThen, s.left2 + s.right2, extra + eyes(s.left2, s.right2) + unc(s.var2));
+                }
+                if (want & kLeft) left = SrcJson(list, headThen, s.left2);
+                if (want & kRight) right = SrcJson(list, headThen, s.right2);
+                // "new" compares with the last sample, so the last measurement is kept either way.
                 const float *m = s.meas;
                 const bool newL = m[0] != lastMeas[0] || m[1] != lastMeas[1];
                 const bool newR = m[2] != lastMeas[2] || m[3] != lastMeas[3];
                 std::memcpy(lastMeas, m, sizeof lastMeas);
-                std::snprintf(extra, sizeof extra, "{\"q\":[%.3g,%.3g],\"m\":[[%.4f,%.4f],[%.4f,%.4f]],\"new\":[%d,%d]}",
-                              (m[4] + m[5]) / 2, (m[6] + m[7]) / 2, m[0], m[1], m[2], m[3], int(newL), int(newR));
-                eye = extra;
+                if (want & kEye) {
+                    std::snprintf(extra, sizeof extra, "{\"q\":[%.3g,%.3g],\"m\":[[%.4f,%.4f],[%.4f,%.4f]],\"new\":[%d,%d]}",
+                                  (m[4] + m[5]) / 2, (m[6] + m[7]) / 2, m[0], m[1], m[2], m[3], int(newL), int(newR));
+                    eye = extra;
+                }
             }
 
             // Our tracker: its own sample time picks the head pose, like the mmap's.
             std::string own = "{\"ok\":0}";
             OwnSample o;
-            if (ownFile.Read(o) && now - o.t < 0.1) {
+            if ((want & kOwn) && ownFile.Read(o) && now - o.t < 0.1) {
                 vr::HmdMatrix34_t headOwn = headNow;
                 history.At(o.t, headOwn);
                 auto pair = [](bool ok, float a, float b) {
@@ -604,7 +690,10 @@ int main(int argc, char **argv) {
             break;
         }
         if (stdinClosed) break;
-        std::this_thread::sleep_for(std::chrono::milliseconds(2));
+        // 250 passes a second: a new sample is printed within 4 ms (2 on average) of appearing,
+        // and the pose history has a pose within 2 ms of any sample's time (a 0.2 degree head
+        // turn at 100 degrees a second). Every 2 ms read the pose and the events twice as often.
+        std::this_thread::sleep_for(std::chrono::milliseconds(4));
     }
     screens.Stop();
     vr::VR_Shutdown();

gaze/ft-gazectl

@@ -36,8 +36,17 @@ def main():
         if r is None:
             sys.exit("the pointer helper isn't running")
         print(f"gaze mode {r.removeprefix('ok ')}" if r else "no answer (an older pointer helper without gaze mode?)")
-        if ask("ft_gazed", "status", 0.3) is None:
+        st = ask("ft_gazed", "status", 0.3)
+        if st is None:
             print("note: the gaze service (ft-gazed) isn't running, so the pointer has no gaze to follow")
+        elif r == "ok on":
+            try:
+                calibrated = (json.loads(st).get("checks") or {}).get("calibrated")
+            except ValueError:
+                calibrated = None
+            if calibrated is False:
+                print("note: no calibration for this eye tracker yet; it opens in the headset "
+                      "(if it can't, ft-gazectl status says why: checks.problem)")
     elif cmd in ("status", "forget", "reload"):
         r = ask("ft_gazed", cmd)
         if r is None:

gaze/ft-gazed

@@ -3,8 +3,12 @@
 
 Two settings in ~/.config/frametop.conf (the Gaze page of Frametop Input Settings), read
 again when the file changes:
-  GAZE_TRACKER=steam|own   SteamVR's eye tracker (default), or our own (gaze/tracker/ft-eyes,
-                           ft-gaze's source "own"; this service runs it, see below)
+  GAZE_TRACKER=auto|own|steam
+                           our own eye tracker (gaze/tracker/ft-eyes, ft-gaze's source "own";
+                           this service runs it, see below) or SteamVR's. auto (the default)
+                           is ours when it's installed (gaze/tracker/install.sh: the frame
+                           grabber, and ft-eyes' Python in this checkout), else SteamVR's; it
+                           switches when ours is installed or removed.
   GAZE_EYE=auto|left|right the eye bias (gazecal.EyeWeights): auto weights each eye by how far
                            off it was at your recent nudges; left or right counts that eye twice
                            as much as the other. Either eye alone carries the gaze when the
@@ -44,9 +48,11 @@ click, it sends "lesson <raw yaw> <raw pitch> <true yaw> <true pitch>": where th
 was when the mouse took over, and where the pointer was when you clicked (you were looking
 there). The gap is the tracker's error there. The raw gaze is the one sent, so it also says
 when that look was (the history of what was sent), and so what each eye read then:
-  - SteamVR: each eye learns its own error, unless the gaze was more than LESSON_MAX degrees
+  - SteamVR: each eye learns its own error, unless the gaze was more than
+    POINTER_GAZE_NUDGE_MAX degrees (frametop.conf, 55 by default: the helper's limit too)
     past the correction (then it wasn't a nudge onto what you looked at);
-  - our tracker: the look goes to it as a click ("click T YAW PITCH" on @ft_eyes), as the
+  - our tracker: unless it was more than POINTER_GAZE_NUDGE_MAX off, the look goes to it as
+    a click ("click T YAW PITCH" on @ft_eyes), as the
     probe's clicks do, and it learns how far the headset has moved on your face. That's
     what it gets wrong, and after the headset was off, the first click resets it;
   - either way, how far off each eye was (before the lesson taught it anything) goes to the
@@ -57,20 +63,48 @@ and the error moves): lessons from before count less then, so the first few afte
 relearn the offset.
 
 Our own tracker (gaze/tracker/ft-eyes) runs here too, in the dev container, while
-GAZE_TRACKER=own or the gaze probe asks for it ("eyes SECONDS", a lease the probe renews). It
-reads the eye-camera frames the root service frametop-eyegrab copies (gaze/tracker/install.sh),
-which copies them only while ft-eyes runs.
+GAZE_TRACKER=own and the gaze is in use (below), or the gaze probe asks for it ("eyes SECONDS",
+a lease the probe renews). It reads the eye-camera frames the root service frametop-eyegrab
+copies (gaze/tracker/install.sh), which copies them only while ft-eyes runs.
+
+Idle: ft-gaze and our own tracker run only while the gaze is in use: gaze mode on with someone
+wearing the headset (the pointer helper says both: "gaze ? headset" -> "ok on|off worn|away"), a
+check or calibration open or asked for, or a "wake" lease (the Gaze page of Frametop Input
+Settings renews one while it's open). IDLE_AFTER after the last use they stop, and the frame
+grabber goes idle with our tracker; with our tracker, that was over half a core with gaze mode
+off. A check asked for while idle waits for the tracker to start (at most WAKE_SETTLE). Our
+tracker's next click after it starts again re-seats it, as after the headset was off, so turning
+gaze mode on after a while opens the quick check.
+
+ft-gaze prints only the sources this uses ("sources LIST" on its stdin, see wanted_sources):
+own and mmap1 with our tracker, left, right and mmap1 with SteamVR's eyes, and every source
+while a check or the calibration runs or is asked for, since those record them all. So
+SteamVR's gaze action, the one source that calls into vrserver, is read only then (or with
+--source action).
+
+Checks and calibration (gaze/gazecheck.py): a one-dot quick check when the headset goes on or
+our tracker asks for a click, and the full calibration when gaze mode comes on without one,
+both in a panel fixed to the headset (gaze/panel/ft-gazepanel, which this service runs too).
 
 Nothing here writes to SteamVR, its eye tracker, or its files: ft-gaze reads the eye
 tracker's shared memory read-only.
 
 Control socket: abstract unix datagram "@ft_gazed":
   lesson <rhy> <rhp> <thy> <thp>   from the pointer helper (see above)
+  recheck <deg>                    from the pointer helper: a click's correction was past
+                                   POINTER_GAZE_NUDGE_MAX, so the quick check (gazecheck.py)
   status                           reply: one JSON object
   forget                           drop what the lessons taught (the calibration stays)
   reload                           read calibration.json and the settings again
   eyes <seconds>                   keep our own tracker running that much longer (at most 120),
                                    whatever GAZE_TRACKER says: the probe's lease. Reply: "ok"
+  wake <seconds>                   keep the gaze running (not idle) that much longer (at most 120),
+                                   whatever gaze mode says: Input Settings' Gaze page. Reply: "ok"
+  quickcal                         the one-dot check now (the calibration if there's none)
+  calibrate                        the full calibration in the panel
+  fitcheck                         the headset fit check in the panel
+  calaccept | calquit              from the pointer helper while the panel is up: take this dot
+                                   now (a left click, Meta+J) | close it (a right click, Meta+K)
 
 Options: --source action|mmap1|mmap2 (the older one-source path with that source, whatever
 the settings say; set 2 was a little quieter in the probe, but loses the pointer whenever
@@ -96,6 +130,7 @@ from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent))
 from gazecal import (DEFAULT_MODEL, EYE_FOUND, EYE_LOST, MODELS, STATE, Correction, EyeFallback,  # noqa: E402
                      EyeWeights, Fixation, LiveCorrection, SteamEyeLog)
+from gazecheck import Checks  # noqa: E402
 
 REPO = Path(__file__).resolve().parents[1]
 HELPER = REPO / "gaze" / "build" / "ft-gaze"
@@ -105,6 +140,7 @@ EYES_PROG = REPO / "gaze" / "tracker" / "ft-eyes"  # our own tracker
 EYES_PYTHON = REPO / "gaze" / "tracker" / "build" / "venv" / "bin" / "python"  # numpy, OpenCV (build.sh)
 EYES_SOCKET = "\0ft_eyes"                      # its control socket
 EYES_CAMS = Path("/dev/shm/frametop-eyes-cams")  # the frames it reads (frametop-eyegrab.service)
+EYEGRAB = (Path("/etc/frametop/ft-eyegrab"), Path("/etc/systemd/system/frametop-eyegrab.service"))  # tracker/install.sh
 CONF = Path.home() / ".config" / "frametop.conf"
 CALIBRATION = STATE / "calibration.json"
 LESSONS = STATE / "pointer-lessons.json"
@@ -112,15 +148,18 @@ LESSON_LOG = STATE / "pointer-lessons.jsonl"
 SOURCES = ("action", "mmap1", "mmap2", "left", "right")  # the ones with a calibration here
 SIDES = ("left", "right")  # ft-gaze's order, and the sources for each eye alone
 TRACKERS = ("steam", "own")
+TRACKER_SETTINGS = ("auto",) + TRACKERS
 BIASES = ("auto", "left", "right")
-LESSON_MAX = 8.0  # degrees past the correction
-OWN_LESSON_MAX = 25.0  # our tracker: after the headset was off, its first clicks can be 10-17 off
+NUDGE_MAX = 55.0  # degrees: POINTER_GAZE_NUDGE_MAX's default, the largest lesson taken
 HISTORY = 12.0    # seconds of the gaze sent, to find a lesson's look (the helper sends it up to 10 s later)
 LOOK = 0.3        # seconds of samples before that moment make the look (the probe's fixation)
 RETRY = 3.0       # seconds before starting ft-gaze again
 EYES_RETRY = 10.0  # seconds before starting ft-eyes again after it stopped on its own
 EYES_LEASE_MAX = 120.0
 SETTLE = 0.3      # seconds after an eye is found again before the fallback learns from it
+IDLE_AFTER = 30.0  # seconds after the gaze was last in use before ft-gaze and our tracker stop
+WAKE_SETTLE = 15.0  # seconds after waking that the tracker may take to start sending
+WAKE_MAX = 120.0
 KEYBOARD_PITCH = -20.0  # degrees: gaze under this, on no screen, is a look at the keyboard
 
 
@@ -140,8 +179,14 @@ def log(msg):
     print(f"ft-gazed: {msg}", file=sys.stderr, flush=True)
 
 
+def own_installed():
+    """Is our own tracker installed: its frame grabber, and ft-eyes' Python here?"""
+    return all(p.exists() for p in EYEGRAB) and EYES_PYTHON.exists()
+
+
 def read_settings():
-    """(tracker, eye bias) from frametop.conf, defaults for anything missing or unknown."""
+    """(tracker, GAZE_TRACKER, eye bias, nudge max) from frametop.conf, defaults for anything
+    missing or unknown. The tracker is "steam" or "own": auto picks ours when it's installed."""
     conf = {}
     try:
         for line in CONF.read_text().splitlines():
@@ -151,9 +196,15 @@ def read_settings():
                 conf[k.strip()] = v.strip().lower()
     except OSError:
         pass
-    tracker = conf.get("GAZE_TRACKER", "steam")
+    setting = conf.get("GAZE_TRACKER", "auto")
+    setting = setting if setting in TRACKER_SETTINGS else "auto"
+    tracker = setting if setting != "auto" else "own" if own_installed() else "steam"
     bias = conf.get("GAZE_EYE", "auto")
-    return tracker if tracker in TRACKERS else "steam", bias if bias in BIASES else "auto"
+    try:
+        nudge = min(max(float(conf.get("POINTER_GAZE_NUDGE_MAX", NUDGE_MAX)), 1.0), 110.0)  # the helper's range
+    except ValueError:
+        nudge = NUDGE_MAX
+    return tracker, setting, bias if bias in BIASES else "auto", nudge
 
 
 def mtime(path):
@@ -168,7 +219,7 @@ class Service:
         self.override, self.verbose, self.to = source, verbose, to
         self.source = source or "mmap1"  # the older path's source
         STATE.mkdir(parents=True, exist_ok=True)
-        self.tracker, self.bias = read_settings()
+        self.tracker, self.tracker_setting, self.bias, self.nudge_max = read_settings()
         self.conf_mtime = mtime(CONF)
         self.models = {name: Correction() for name in SOURCES}
         self.mode = DEFAULT_MODEL
@@ -199,6 +250,10 @@ class Service:
         self.eyes_proc = None  # ft-eyes, while it runs
         self.eyes_until = 0.0  # the probe's lease (monotonic time)
         self.eyes_restart_at = 0.0
+        self.awake = False      # ft-gaze (and our tracker) run: the gaze is in use (see the top)
+        self.idle_at = 0.0      # idle from then, unless it's in use again first
+        self.woke_at = 0.0
+        self.wake_until = 0.0   # a "wake" lease
 
         self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC | socket.SOCK_NONBLOCK)
         self.sock.bind(ME)
@@ -209,7 +264,9 @@ class Service:
         self.sel = selectors.DefaultSelector()
         self.sel.register(self.sock, selectors.EVENT_READ, "control")
         self.sel.register(self.eyes_sock, selectors.EVENT_READ, "own")
+        self.checks = Checks(self, self.sel)
         self.proc = None
+        self.proc_sources = None  # what ft-gaze was last told to print
         self.buf = b""
         self.restart_at = 0.0
         self.running = True
@@ -228,9 +285,12 @@ class Service:
 
     def load_settings(self):
         self.conf_mtime = mtime(CONF)
-        tracker, bias = read_settings()
+        tracker, self.tracker_setting, bias, self.nudge_max = read_settings()
         if (tracker, bias) != (self.tracker, self.bias):
-            log(f"tracker {tracker}, eye bias {bias}" + (f" (--source {self.override} wins)" if self.override else ""))
+            auto = ""
+            if self.tracker_setting == "auto":
+                auto = " (auto: ours is installed)" if tracker == "own" else " (auto: ours isn't installed)"
+            log(f"tracker {tracker}{auto}, eye bias {bias}" + (f" (--source {self.override} wins)" if self.override else ""))
             self.tracker, self.bias = tracker, bias
             for w in self.weights.values():
                 w.bias = bias
@@ -273,6 +333,13 @@ class Service:
         tmp.replace(LESSONS)
         self.dirty = False
 
+    def forget_lessons(self):
+        """Drop what the lessons taught (the calibration stays)."""
+        self.lives = {name: PointerLessons() for name in SOURCES}
+        self.weights = {t: EyeWeights(self.bias) for t in TRACKERS}
+        self.refit()
+        self.save_lessons()
+
     def refit(self):
         for name, live in self.lives.items():
             live.wear_time = self.steam.worn()
@@ -304,8 +371,8 @@ class Service:
             cy, cp = self.correction(self.source, rhy, rhp)
             left = math.hypot(dy - cy, dp - cp)
             rec.update(source=self.source, model=self.mode, correction=[cy, cp], lesson_deg=left)
-            if left > LESSON_MAX:
-                rec["refused"] = f"more than {LESSON_MAX} deg past the correction"
+            if left > self.nudge_max:
+                rec["refused"] = f"more than {self.nudge_max:g} deg past the correction"
             else:
                 self.lives[self.source].add({"time": rec["time"], "hy": rhy, "hp": rhp, "dy": dy, "dp": dp,
                                              "wy": 1.0, "wp": 1.0, "how": "pointer"}, self.models[self.source], self.mode)
@@ -316,11 +383,10 @@ class Service:
         weights = self.weights[self.tracker if kind == "own" else "steam"]
         rec.update(tracker=self.tracker if kind == "own" else "steam", bias=self.bias, lesson_deg=left, look_t=t,
                    eyes=eyes, weights=[round(w, 3) for w in weights.weights()])
-        limit = OWN_LESSON_MAX if kind == "own" else LESSON_MAX
         if t is None:
             rec["refused"] = "that gaze isn't in the last few seconds sent"
-        elif left > limit:
-            rec["refused"] = f"more than {limit} deg off"
+        elif left > self.nudge_max:
+            rec["refused"] = f"more than {self.nudge_max:g} deg off"
         if "refused" in rec:
             return self.taken(rec)
         if kind == "own":
@@ -358,8 +424,73 @@ class Service:
             log(f"lesson log: {e}")
         return rec
 
+    # --- Idle (see the top) ---
+
+    def use_reason(self):
+        """Why the gaze is in use now, or None."""
+        c = self.checks
+        if c.gaze_on and c.headset is not False:
+            return "gaze mode is on"
+        if c.active or c.pending:
+            return "a check"
+        if time.monotonic() < self.wake_until:
+            return "asked to stay awake"
+        return None
+
+    def idle_reason(self):
+        c = self.checks
+        if c.gaze_on is None:
+            return "the pointer helper isn't answering (SteamVR not running?)"
+        if c.gaze_on and c.headset is False:
+            return "nobody is wearing the headset"
+        return "gaze mode is off"
+
+    def waking(self):
+        """Idle, or awake too briefly for the tracker to be sending yet."""
+        return not self.awake or time.monotonic() - self.woke_at < WAKE_SETTLE
+
+    def update_awake(self):
+        now = time.monotonic()
+        why = self.use_reason()
+        if why:
+            self.idle_at = now + IDLE_AFTER
+        if why and not self.awake:
+            self.awake, self.woke_at, self.restart_at = True, now, 0.0
+            log(f"awake: {why}")
+        elif not why and self.awake and now >= self.idle_at:
+            self.awake = False
+            log(f"idle: {self.idle_reason()}; ft-gaze and our tracker stop until the gaze is used again")
+            self.stop_helper()
+            if self.eyes_proc and not self.eyes_wanted():
+                self.stop_eyes()
+
     # --- ft-gaze ---
 
+    def wanted_sources(self):
+        """The sources ft-gaze should print (its --sources): what on_sample and the checks read.
+        mmap1 always (blinks, lost eyes, and the headset going on, from its openness and
+        variances); everything while a check runs or waits, since they record every source."""
+        c = self.checks
+        if c.active or c.pending:
+            return "all"
+        kind = self.kind
+        if kind == "own":
+            return "own,mmap1"
+        if kind == "eyes":
+            return "left,right,mmap1"
+        return ",".join(dict.fromkeys((self.source, "mmap1", "mmap2")))  # mmap2: each eye, for the fallback
+
+    def sync_sources(self):
+        want = self.wanted_sources()
+        if not self.proc or want == self.proc_sources:
+            return
+        try:
+            self.proc.stdin.write(f"sources {want}\n".encode())
+            self.proc.stdin.flush()
+        except (OSError, ValueError):
+            return  # it's stopping: read_stdout notices
+        self.proc_sources = want
+
     def start_helper(self):
         if not HELPER.exists():
             log(f"ft-gaze isn't built: run {REPO}/gaze/build.sh")
@@ -370,9 +501,11 @@ class Service:
         subprocess.run([str(REPO / "scripts" / "container-up.sh")], env=env, check=False)
         distrobox = Path.home() / ".local" / "bin" / "distrobox"
         # ft-gaze quits when its stdin closes: the one thing distrobox passes on.
-        self.proc = subprocess.Popen([str(distrobox), "enter", "dev", "--", str(HELPER), "--watch-stdin"], env=env,
-                                     stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                                     start_new_session=True)
+        sources = self.wanted_sources()
+        self.proc = subprocess.Popen([str(distrobox), "enter", "dev", "--", str(HELPER), "--watch-stdin",
+                                      "--sources", sources], env=env, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+                                     stderr=subprocess.PIPE, start_new_session=True)
+        self.proc_sources = sources
         os.set_blocking(self.proc.stdout.fileno(), False)
         os.set_blocking(self.proc.stderr.fileno(), False)
         self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
@@ -400,7 +533,7 @@ class Service:
         self.proc = None
 
     def eyes_wanted(self):
-        return (self.tracker == "own" and not self.override) or time.monotonic() < self.eyes_until
+        return (self.tracker == "own" and not self.override and self.awake) or time.monotonic() < self.eyes_until
 
     def start_eyes(self):
         """Our own tracker, in the dev container, with build/venv's numpy and OpenCV. Like
@@ -505,6 +638,7 @@ class Service:
         return low
 
     def on_sample(self, s):
+        self.checks.on_sample(s)
         kind = self.kind
         if kind != self.last_kind:
             log({"own": "our own tracker", "eyes": "SteamVR's eyes, each calibrated",
@@ -636,6 +770,7 @@ class Service:
             if words[:1] == ["lesson"] and len(words) == 5:
                 try:
                     rec = self.lesson(*map(float, words[1:]))
+                    self.checks.after_lesson(rec)
                     reply = "refused" if "refused" in rec else f"ok {rec['lesson_deg']:.2f}"
                     log(f"lesson {rec['lesson_deg']:.2f} deg at {rec['raw'][0]:+.1f},{rec['raw'][1]:+.1f}"
                         + (f", eyes off {', '.join('-' if m is None else f'{m:.2f}' for m in rec['miss'])}"
@@ -646,11 +781,10 @@ class Service:
             elif words[:1] == ["status"]:
                 reply = json.dumps(self.status())
             elif words[:1] == ["forget"]:
-                self.lives = {name: PointerLessons() for name in SOURCES}
-                self.weights = {t: EyeWeights(self.bias) for t in TRACKERS}
-                self.refit()
-                self.save_lessons()
+                self.forget_lessons()
                 reply = "ok"
+            elif words[:1] and words[0] in ("quickcal", "calibrate", "fitcheck", "calaccept", "calquit", "recheck"):
+                reply = self.checks.command(words)
             elif words[:1] == ["eyes"] and len(words) == 2:
                 try:
                     secs = min(max(float(words[1]), 0.0), EYES_LEASE_MAX)
@@ -658,6 +792,14 @@ class Service:
                     reply = "ok"
                 except ValueError:
                     reply = "error bad seconds"
+            elif words[:1] == ["wake"] and len(words) == 2:
+                try:
+                    secs = min(max(float(words[1]), 0.0), WAKE_MAX)
+                    self.wake_until = max(self.wake_until, time.monotonic() + secs)
+                    self.update_awake()
+                    reply = "ok"
+                except ValueError:
+                    reply = "error bad seconds"
             elif words[:1] == ["reload"]:
                 self.load_settings()
                 self.load_calibration()
@@ -708,12 +850,14 @@ class Service:
             st.update(calibration_samples=cal.get("dots", 0), calibration_made=cal.get("made"),
                       lessons=max(len(m) for m in w.misses), own_running=bool(own),
                       own_reseat=any(e.get("reseat") for e in own.get("eyes", {}).values()))
-        st.update(eyes_process=self.eyes_proc is not None, eyegrab=EYES_CAMS.exists())
+        st.update(awake=self.awake, idle=None if self.awake else self.idle_reason())
+        st.update(eyes_process=self.eyes_proc is not None, eyegrab=EYES_CAMS.exists(),
+                  tracker_setting=self.tracker_setting, own_installed=own_installed())
         st.update({"ft_gaze": self.proc is not None, "sample_age_s": round(time.monotonic() - self.last_sample, 2)
                    if self.last_sample else None, "headset_on": self.steam.wearing(),
                    "headset_on_since": self.steam.worn(), "last": [round(v, 2) for v in self.last] if self.last else None,
                    "eyes_lost": self.lost, "fallback_ready": [self.fallback.ready(0), self.fallback.ready(1)],
-                   **self.counts})
+                   "checks": self.checks.status(), **self.counts})
         return st
 
     def periodic(self):
@@ -724,8 +868,10 @@ class Service:
         if mtime(CALIBRATION) != self.cal_mtime:
             self.load_calibration()
             self.refit()
-        if mtime(CONF) != self.conf_mtime:
+        if mtime(CONF) != self.conf_mtime or (self.tracker_setting == "auto"
+                                              and own_installed() != (self.tracker == "own")):
             self.load_settings()
+        self.update_awake()
         want = self.eyes_wanted()
         if want and not self.eyes_proc and time.monotonic() >= self.eyes_restart_at:
             self.start_eyes()
@@ -737,6 +883,7 @@ class Service:
                 self.eyes_sock.sendto(b"status", EYES_SOCKET)
             except OSError:
                 pass  # not up yet: status() says so once the last answer is old
+        self.checks.periodic()
         if self.dirty:
             self.save_lessons()
 
@@ -745,11 +892,15 @@ class Service:
         next_verbose = time.monotonic() + 5
         while self.running:
             now = time.monotonic()
-            if not self.proc and now >= self.restart_at:
+            if self.awake and not self.proc and now >= self.restart_at:
                 self.start_helper()
-            for key, _ in self.sel.select(timeout=0.5):
+            for key, _ in self.sel.select(timeout=0.05 if self.checks.active else 0.5):
                 if key.data == "control":
                     self.on_control()
+                elif key.data == "checks":
+                    self.checks.on_readable()
+                elif key.data == "panel" and self.checks.panel_proc:
+                    self.checks.read_panel()
                 elif key.data == "own":
                     self.on_own()
                 elif key.data == "eyes" and self.eyes_proc:
@@ -758,12 +909,15 @@ class Service:
                     self.read_stdout()
                 elif key.data == "stderr" and self.proc:
                     self.read_stderr()
+            self.checks.tick()
+            self.sync_sources()
             if now >= next_periodic:
                 self.periodic()
                 next_periodic = now + 1.0
             if self.verbose and now >= next_verbose:
                 log(json.dumps(self.status()))
                 next_verbose = now + 5
+        self.checks.stop()
         self.stop_helper()
         self.stop_eyes()
         if self.dirty:

gaze/gazecal.py

@@ -544,6 +544,9 @@ class SteamEyeLog:
     each time the headset goes on ("HMD on"): the eye model starts over then too."""
 
     PATH = Path.home() / ".local" / "share" / "Steam" / "logs" / "eyetracking.txt"
+    # It writes "HMD on" again every minute or so while on, and flickers off for 0.01-0.3 s:
+    # only an on after an off of BLIP or longer counts.
+    BLIP = 1.5
 
     def __init__(self):
         self.pos = 0
@@ -595,9 +598,14 @@ class SteamEyeLog:
                 self.starts.append(t)
                 restarted = not first
             elif "HMD on" in line:
-                self.wears.append(t)
+                off = bool(self.offs) and (not self.wears or self.offs[-1] > self.wears[-1])
+                if off and self.wears and t - self.offs[-1] < self.BLIP:
+                    self.offs.pop()  # the sensor flickering: it never came off
+                elif off or not self.wears:
+                    self.wears.append(t)
             elif "HMD off" in line:
-                self.offs.append(t)
+                if not self.offs or (self.wears and self.wears[-1] > self.offs[-1]):
+                    self.offs.append(t)
             elif "Accept usercal" in line:
                 self.accepts.append(t)
             elif "Reject usercal" in line:
@@ -632,31 +640,44 @@ def cross_validate(points, mode):
     return errs
 
 
-def steady_samples(samples, vergence_jump=1.5):
+def steady_samples(samples, vergence_jump=1.5, why=None):
     """The samples of one look at one spot where the tracker had both eyes: none in a blink
     (openness under half its median over the samples), none where it had lost an eye (its
     variance over EYE_LOST), and none where the angle between the eyes' directions (`lr`, the
     vergence) is more than `vergence_jump` degrees from its median over the samples. The
     vergence itself depends on distance (about 2.8 degrees for a screen 1.3 m away, a
     fraction of one far off), so only a jump away from what it was during this look means
-    the tracker lost an eye. Without the mmap there's nothing to judge by: all are kept."""
+    the tracker lost an eye. Without the mmap there's nothing to judge by: all are kept.
+    `why`, a dict, gets how many were dropped for each reason: "lost_left", "lost_right",
+    "lost_both", "blink" and "vergence" (each sample once, for the first that applies)."""
+    if why is None:
+        why = {}
     # Openness: a blink is a sharp drop from what it was during this look. Not a fixed
     # level: looking down, the upper lids come down with the eyes, and in bright light you
     # squint, so the reading can stay under 0.5 for the whole look while the tracker follows
     # the eyes fine (a calibration dot at the bottom of the bright round failed that way).
     opens = [min(o) for o in ((smp["src"].get("mmap1") or {}).get("open") for smp in samples) if o]
     floor = max(0.12, 0.5 * statistics.median(opens)) if len(opens) >= 5 else 0.12
-    opened = []
+    seen = []
     for smp in samples:
-        o = (smp["src"].get("mmap1") or {}).get("open")
-        if not o or min(o) >= floor:
-            opened.append(smp)
+        m1 = smp["src"].get("mmap1") or {}
+        o = m1.get("open")
+        lost = [u > EYE_LOST for u in m1.get("unc") or [0, 0]]
+        # A lost eye's openness reads 0 too, so a lost eye is named before a blink.
+        key = ("lost_both" if all(lost) else "lost_left" if lost[0] else "lost_right") if any(lost) else \
+            "blink" if o and min(o) < floor else None
+        if key:
+            why[key] = why.get(key, 0) + 1
+        else:
+            seen.append(smp)
 
     def vergence(smp):
         return (smp["src"].get("mmap1") or {}).get("lr", (smp["src"].get("mmap2") or {}).get("lr"))
-    opened = [smp for smp in opened if max((smp["src"].get("mmap1") or {}).get("unc") or [0]) <= EYE_LOST]
-    have = [v for v in map(vergence, opened) if v is not None]
+    have = [v for v in map(vergence, seen) if v is not None]
     if len(have) < 5:
-        return opened
+        return seen
     med = statistics.median(have)
-    return [smp for smp in opened if vergence(smp) is None or abs(vergence(smp) - med) <= vergence_jump]
+    kept = [smp for smp in seen if vergence(smp) is None or abs(vergence(smp) - med) <= vergence_jump]
+    if len(kept) < len(seen):
+        why["vergence"] = why.get("vergence", 0) + len(seen) - len(kept)
+    return kept

gaze/gazecheck.py

@@ -0,0 +1,926 @@
+"""gazecheck: the gaze service's checks and calibration, in the panel fixed to the headset
+(gaze/panel/ft-gazepanel; ft-gazed runs it). Every kind is made of dots shown at head-relative
+directions: look at each one.
+
+  quick   one dot in the middle of your view. It opens when the headset goes on: eyes seen
+          for DON_DELAY after none for AWAY_MIN (SteamVR's tracker's variance for an eye under
+          EYE_LOST). SteamVR's "HMD on" can't say: it repeats every minute or so, and it can
+          stay on for hours with nobody in the headset. It also opens when our own tracker asks
+          for a click (its "reseat": the headset may sit differently on your face now), at most
+          once every QUICK_COOLDOWN, and on "quickcal" (Frametop Input Settings, or a mouse
+          button or key combination mapped to Gaze quick check), and when a click's correction
+          was past POINTER_GAZE_NUDGE_MAX (55 degrees; the helper's "recheck"): the tracker is
+          far off. Ignored, it closes after QUICK_TIMEOUT and changes nothing. The headset going
+          on is seen only while the gaze service is awake (gaze mode on, someone wearing it: see
+          ft-gazed), so it also opens when gaze mode comes on after the service idled.
+  five    the middle and four around it, when the first FIVE_COUNT lessons after a quick check
+          were all over FIVE_LIMIT degrees off: the quick check didn't fix it.
+  full    the calibration, as the gaze probe's: three rounds, dark, medium and bright (pupil
+          size, and the tracker's error with it, changes with brightness), each the middle and
+          a ring of six (SteamVR's tracker) or eight (ours, whose fit goes wrong past its dots)
+          RING degrees out, half that in the middle round, turned 20 degrees a round. It opens
+          whenever gaze mode is on without a calibration for the tracker in use and someone's
+          in the headset, and on "calibrate". One that closes unfinished (ignored, too few
+          dots) opens again only once the headset comes off and on, or gaze mode off and on.
+          Frametop's screens hide while it runs. Quitting it while there's still no
+          calibration turns gaze mode off (POINTER_GAZE=0); turning it on again reopens it.
+          Why gaze mode can't work yet goes in the status ("problem"), for Input Settings.
+
+  fit     the headset fit check (on "fitcheck", Check headset fit on the Gaze page): live, a
+          card per eye (tracked or lost, the tracker's signal, how much of the last 10 s it was
+          seen) and hints, from the gaze probe's Headset fit (gaze/fitcheck.py), while you
+          adjust the headset. A left click or Meta+J runs its guided check (dots, then looks
+          down, up, left and right); a right click or Meta+K closes it, as does FIT_TIMEOUT.
+
+A check asked for while the gaze service idles (quickcal, calibrate, fitcheck) wakes it and
+waits until the tracker sends, at most ft-gazed's WAKE_SETTLE; then it opens, or logs why not.
+
+The quick check's dot captures itself: from CHECK_SETTLE after it shows (the eyes getting
+there), once the gaze has held within CHECK_SPREAD for CHECK_WINDOW (the probe's max spread and
+capture time). It's the gaze holding still that counts, not where the tracker puts it, so it
+works however far off the tracker is; a left click or Meta+J (the pointer helper's
+"calaccept") takes it now. The full calibration's and five's dots wait for that click: you
+click when you're looking at the dot (the user asked for that: a steady gaze isn't always on
+the dot), and the gaze held still up to then is taken (ACCEPT_SPREAD). They wait as long as
+it takes, up to CLICK_IDLE. A dot not taken says why in the panel's note line (reject_reason:
+gazecal.steady_samples' drop counts for SteamVR's tracker, ft-eyes' reply for ours), as does a
+click with nothing taken after ACCEPT_WAIT, and a failed calibration names its most common
+reason there and in the status. A right click or Meta+K ("calquit") closes the panel. The pointer hides meanwhile ("calpanel 1",
+renewed every second; the helper shows it again by itself when that stops).
+
+Our tracker's first calibration: before it has one, ft-eyes publishes no gaze (it maps pupils
+to a gaze only with a calibration), so there's no gaze to hold still. Its calibration runs
+anyway ("blind"): someone in the headset (SteamVR's tracker sees an eye) and ft-eyes answering
+are enough to start it, each dot stands in for the gaze, and a click takes the CHECK_WINDOW up
+to it. ft-eyes then checks that each pupil was seen and held still in that window (calib-point)
+and says why not. Without this, a fresh install could never calibrate our tracker.
+
+What a capture teaches:
+  our tracker   quick and five: a click ("click T YAW PITCH", like a pointer lesson); full:
+                calib-start, a calib-point for each dot, calib-fit (its calibration)
+  SteamVR's     quick and five: a lesson for each eye, like the pointer's; full: each source's
+                calibration fitted from the dots (gazecal.Correction, the probe's way), saved
+                to calibration.json, and the lessons start over on top of it
+Either way, how far off each eye was goes to the eye bias, and each capture to the lesson log.
+"""
+
+import json
+import math
+import os
+import selectors
+import signal
+import socket
+import statistics
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+from fitcheck import MIN_REGION, FitCheck, wrap
+from gazecal import DEFAULT_MODEL, EYE_LOST, STATE, steady_samples
+
+REPO = Path(__file__).resolve().parents[1]
+PANEL_PROG = REPO / "gaze" / "build" / "ft-gazepanel"
+PANEL = "\0ft_gazepanel"
+POINTER = "\0ft_pointer_helper"
+SCREENS = "\0ft_screens"
+EYES = "\0ft_eyes"
+CONF = Path.home() / ".config" / "frametop.conf"
+CALIBRATION = STATE / "calibration.json"
+POINTS = STATE / "points.jsonl"   # the probe's record of calibration dots, for its refine
+CHECK_LOG = STATE / "checks.jsonl"
+
+CHECK_SETTLE = 0.45
+CHECK_WINDOW = 0.6
+CHECK_SPREAD = 1.0     # degrees
+ACCEPT_SPREAD = 2.5    # degrees: a capture asked for (calaccept) takes this much
+CLICK_IDLE = 120.0     # seconds a dot of five or full waits for its click; then the check closes
+QUICK_TIMEOUT = 6.0
+QUICK_COOLDOWN = 120.0
+DON_DELAY = 3.0        # seconds of eyes after AWAY_MIN without: the headset went on
+AWAY_MIN = 3.0
+EYES_GONE = 2.0        # seconds without eyes that close a check: the headset came off
+FIVE_LIMIT = 2.0
+FIVE_COUNT = 3
+DONE_PAUSE = 0.35      # seconds the filled dot shows before the next
+RING = 20.0
+ROUND_BG = (0.03, 0.33, 0.8)
+ROUND_NAMES = ("dark", "medium", "bright")
+RING_SCALE = (1.0, 0.5, 1.0)
+PANEL_RETRY = 10.0
+FULL_RETRY = 10.0      # seconds before an automatic calibration that failed to start tries again
+FIT_TIMEOUT = 300.0    # seconds the fit check stays up
+FIT_EVERY = 0.5        # seconds between its cards' updates (each is a new picture for the panel)
+FIT_HINT_WIDTH = 95    # characters a hint line holds in the panel
+
+
+def log(msg):
+    print(f"ft-gazed: {msg}", file=sys.stderr, flush=True)
+
+
+def check_dots(kind, own):
+    """(yaw, pitch, round): head-relative degrees, yaw +left, pitch +up."""
+    if kind == "quick":
+        return [(0.0, 0.0, 0)]
+    if kind == "five":
+        return [(0.0, 0.0, 0), (12.0, 0.0, 0), (-12.0, 0.0, 0), (0.0, 9.0, 0), (0.0, -9.0, 0)]
+    out = []
+    n = 8 if own else 6
+    for rnd in range(3):
+        out.append((0.0, 0.0, rnd))
+        r = RING * RING_SCALE[rnd]
+        for i in range(n):
+            a = math.radians(-90 + rnd * 20 + i * 360 / n)  # as the probe: x right, y down
+            x, y = r * math.cos(a), r * math.sin(a) * (0.9 if own else 1.0)
+            out.append((-x, -y, rnd))
+    return out
+
+
+# Why SteamVR's samples for a look were dropped (gazecal.steady_samples' counts) -> (short, long):
+# short for the small panel, long for the calibration's. FIT_HINT goes after the ones the
+# headset's fit causes, in the calibration's panel.
+STEAM_REASONS = {"lost_left": ("left eye lost", "SteamVR lost your left eye"),
+                 "lost_right": ("right eye lost", "SteamVR lost your right eye"),
+                 "lost_both": ("both eyes lost", "SteamVR lost both eyes"),
+                 "blink": ("blinked", "you blinked"),
+                 "vergence": ("eyes disagreed", "SteamVR's two eyes disagreed")}
+FIT_HINT = "check the headset fit"
+ACCEPT_WAIT = 1.5      # seconds after a click with no capture before the panel says what it waits for
+
+
+def reject_reason(reply, why):
+    """Why a dot wasn't taken -> (short, long, fit): our tracker's reply (`reply`), or SteamVR's
+    drop counts (`why`, when `reply` is None). `fit`: the headset's fit is the likely cause."""
+    if reply is None:
+        if not why:
+            return "no reading", "SteamVR sent no reading for that look", False
+        key = max(why, key=why.get)
+        return *STEAM_REASONS[key], key.startswith("lost")
+    words = reply.removeprefix("fail ").split()
+    # ft-eyes: "fail the left eye was seen in only 3 frames", "fail the left eye moved (4.2 px)"
+    if len(words) >= 3 and words[0] == "the" and words[2] == "eye":
+        eye = words[1]
+        if "seen" in words:
+            return f"{eye} eye not seen", f"our tracker saw your {eye} eye in only {words[-2]} frames", True
+        if "moved" in words:
+            return f"{eye} eye moved", f"your {eye} eye moved while you looked", False
+    if not reply:
+        return "no answer", "our tracker didn't answer", False
+    if reply.startswith("our tracker isn't running"):
+        return "tracker not running", "our tracker isn't running", False
+    text = reply.removeprefix("fail ")
+    return text[:24], f"our tracker said: {text}", False
+
+
+def spread(points):
+    """The median point and the spread around it (1.4826 x the median distance: a standard
+    deviation that one stray sample can't move far)."""
+    mx = statistics.median(p[0] for p in points)
+    my = statistics.median(p[1] for p in points)
+    return 1.4826 * statistics.median(math.hypot(p[0] - mx, p[1] - my) for p in points), (mx, my)
+
+
+def write_gaze(on, path=CONF):
+    """POINTER_GAZE=1|0 in the config, every other line kept."""
+    try:
+        lines = path.read_text().splitlines()
+    except OSError:
+        lines = []
+    value = f"POINTER_GAZE={1 if on else 0}"
+    for i, line in enumerate(lines):
+        if line.split("#", 1)[0].split("=", 1)[0].strip() == "POINTER_GAZE":
+            lines[i] = value
+            break
+    else:
+        lines.append(value)
+    tmp = path.with_suffix(".tmp")
+    tmp.write_text("\n".join(lines) + "\n")
+    tmp.replace(path)
+
+
+def ask(addr, command, timeout=1.0):
+    """A command to a local datagram socket, and its reply ("" without one)."""
+    s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC)
+    try:
+        s.bind("")
+        s.settimeout(timeout)
+        s.sendto(command.encode(), addr)
+        return s.recv(4096).decode("utf-8", "replace")
+    except OSError:
+        return ""
+    finally:
+        s.close()
+
+
+class Checks:
+    def __init__(self, svc, sel):
+        self.svc = svc
+        self.sel = sel
+        self.out = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC | socket.SOCK_NONBLOCK)
+        self.out.bind("")  # the panel's and the helper's replies come back here
+        sel.register(self.out, selectors.EVENT_READ, "checks")
+        self.check = None
+        self.gaze_on = None
+        self.headset = None      # someone wears it (the helper's "worn"), False "away", None unknown
+        self.gaze_heard = 0.0
+        self.pending = None      # (command words, when): asked for while the gaze service idled
+        self.full_armed = True    # gaze mode on without a calibration opens the full one (need_full)
+        self.full_blocked = None  # why it can't open now
+        self.full_retry_at = 0.0
+        self.full_failed = None   # why the last calibration failed (too few dots), for problem()
+        self.last_quick = 0.0
+        self.sample_at = 0.0     # the tracker last sent anything
+        self.seen_at = 0.0       # eyes last seen (SteamVR's tracker's variance for them, "unc")
+        self.away = True         # no eyes for AWAY_MIN: their coming back is the headset going on
+        self.back_since = None
+        self.reseat_seen = False
+        self.after_quick = None
+        self.panel_proc = None
+        self.panel_restart_at = 0.0
+        self.screens_shown = None
+        self.last_progress = 0.0
+
+    @property
+    def active(self):
+        return self.check is not None
+
+    # --- The panel process ---
+
+    def start_panel(self):
+        if not PANEL_PROG.exists():
+            log(f"ft-gazepanel isn't built: run {REPO}/gaze/build.sh")
+            self.panel_restart_at = time.monotonic() + 60
+            return
+        env = dict(os.environ)
+        env["XDG_RUNTIME_DIR"] = f"/run/user/{os.getuid()}"
+        distrobox = Path.home() / ".local" / "bin" / "distrobox"
+        self.panel_proc = subprocess.Popen([str(distrobox), "enter", "dev", "--", str(PANEL_PROG), "--watch-stdin"],
+                                           env=env, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL,
+                                           stderr=subprocess.PIPE, start_new_session=True)
+        os.set_blocking(self.panel_proc.stderr.fileno(), False)
+        self.sel.register(self.panel_proc.stderr, selectors.EVENT_READ, "panel")
+        log("ft-gazepanel started")
+
+    def read_panel(self):
+        try:
+            data = os.read(self.panel_proc.stderr.fileno(), 65536)
+        except BlockingIOError:
+            return
+        if not data:
+            log(f"ft-gazepanel stopped (exit {self.panel_proc.poll()}); again in {PANEL_RETRY:.0f} s")
+            self.stop_panel()
+            self.panel_restart_at = time.monotonic() + PANEL_RETRY
+            if self.check:
+                self.close("the panel stopped")
+            return
+        for line in data.decode("utf-8", "replace").splitlines():
+            if line.strip():
+                log(line)
+
+    def stop_panel(self):
+        if not self.panel_proc:
+            return
+        try:
+            self.sel.unregister(self.panel_proc.stderr)
+        except (KeyError, ValueError):
+            pass
+        if self.panel_proc.stdin and not self.panel_proc.stdin.closed:
+            self.panel_proc.stdin.close()
+        try:
+            self.panel_proc.wait(timeout=2)
+        except subprocess.TimeoutExpired:
+            try:
+                os.killpg(self.panel_proc.pid, signal.SIGTERM)
+            except ProcessLookupError:
+                pass
+        self.panel_proc = None
+
+    def to_panel(self, command):
+        try:
+            self.out.sendto(command.encode(), PANEL)
+        except OSError:
+            pass
+
+    def to_helper(self, command):
+        try:
+            self.out.sendto(command.encode(), POINTER)
+        except OSError:
+            pass
+
+    def on_readable(self):
+        """Replies on our socket: the helper's "ok on|off [worn|away]" to "gaze ? headset" (an
+        older helper leaves the headset out); the panel's are dropped."""
+        while True:
+            try:
+                words = self.out.recv(4096).decode("utf-8", "replace").split()
+            except (BlockingIOError, OSError):
+                return
+            if words[:1] == ["ok"] and len(words) in (2, 3) and words[1] in ("on", "off"):
+                on = words[1] == "on"
+                headset = None if len(words) == 2 else words[2] == "worn"
+                was = (self.gaze_on, self.headset)
+                self.gaze_on, self.headset, self.gaze_heard = on, headset, time.monotonic()
+                if on and was[0] is False:
+                    self.on_gaze_on()
+                if was != (on, headset):
+                    self.svc.update_awake()
+
+    # --- State ---
+
+    def calibrated(self):
+        """Does the tracker in use have a calibration? None: our tracker hasn't said yet."""
+        svc = self.svc
+        kind = svc.kind
+        if kind == "own":
+            if time.monotonic() - svc.own_at > 5 or not svc.own:
+                return None
+            return bool((svc.own.get("calibration") or {}).get("dots"))
+        if kind == "eyes":
+            return True
+        return svc.models[svc.source].samples > 0
+
+    def eyes_seen(self, within=1.0):
+        return time.monotonic() - self.seen_at < within
+
+    def can_run(self):
+        """Someone's in the headset and the tracker is sending. Our tracker sends no gaze before
+        its first calibration: then ft-eyes answering (calibrated() is False only once it has)
+        is enough, since the calibration is what it needs (see "first calibration" at the top)."""
+        if self.blind():
+            return self.eyes_seen()
+        return self.eyes_seen() and time.monotonic() - self.svc.last_sample < 2
+
+    def blind(self):
+        """Our tracker is in use, running, and not calibrated yet: it has no gaze to send."""
+        return self.svc.kind == "own" and self.calibrated() is False
+
+    def on_gaze_on(self):
+        self.full_armed = True
+        self.need_full("gaze mode came on without a calibration")
+
+    def need_full(self, reason):
+        """Gaze mode is on without a calibration: open the full one, once per arming (see the top),
+        or note why it can't open."""
+        cal = self.calibrated()
+        if cal:
+            self.full_armed = True  # missing one later (the other tracker picked) is news again
+        if not self.gaze_on or self.check or not self.full_armed or cal is not False:
+            self.full_blocked = None
+            return
+        now = time.monotonic()
+        if not self.can_run() and self.svc.waking():
+            return  # the tracker is still starting (the service idled)
+        if not self.can_run():
+            why = ("the eye tracker isn't sending" if now - self.svc.last_sample >= 2 and not self.blind()
+                   else "no eyes seen (is the headset on?)")
+        elif now < self.full_retry_at:
+            return
+        else:
+            reply = self.start("full", reason)
+            why = None if reply == "ok" else reply.removeprefix("error ")
+            if why:
+                self.full_retry_at = now + FULL_RETRY
+        if why and why != self.full_blocked:
+            log(f"the calibration can't open: {why}")
+        self.full_blocked = why
+        if not why:
+            self.full_armed = False
+
+    def problem(self):
+        """Why gaze mode, on, can't follow your eyes yet, or None. Our tracker not having said
+        yet is None: Input Settings has its own line for our tracker."""
+        if not self.gaze_on or self.calibrated() is not False:
+            return None
+        if self.check and self.check["kind"] == "full":
+            return "Not calibrated yet: the calibration is open in the headset"
+        if self.full_blocked:
+            return f"Not calibrated, and the calibration can't open: {self.full_blocked}"
+        if not self.full_armed:
+            if self.full_failed:
+                return f"Not calibrated: the calibration failed (most dots: {self.full_failed}). Use Calibrate"
+            return "Not calibrated: the calibration closed unfinished. Use Calibrate"
+        return "Not calibrated: the calibration opens in the headset"
+
+    def auto_quick(self, reason):
+        now = time.monotonic()
+        if self.check or not self.gaze_on or not self.can_run() or self.calibrated() is not True:
+            return
+        if now - self.last_quick < QUICK_COOLDOWN:
+            log(f"quick check skipped ({reason}): one ran {now - self.last_quick:.0f} s ago")
+            return
+        self.start("quick", reason)
+
+    # --- Running a check ---
+
+    def start(self, kind, reason):
+        svc = self.svc
+        if self.check:
+            return "error a check is running"
+        if not self.panel_proc:
+            return "error the panel isn't running (gaze/build.sh builds it)"
+        if kind == "fit":
+            return self.start_fit(reason)
+        if not self.can_run():
+            return "error the headset is off or the tracker isn't sending"
+        own = svc.kind == "own"
+        blind = self.blind()
+        if blind and kind != "full":
+            return "error our tracker isn't calibrated yet: use Calibrate"
+        if kind == "full" and own:
+            reply = ask(EYES, "calib-start", 3.0)
+            if not reply.startswith("ok"):
+                log(f"calibration not started: our tracker says {reply or 'nothing'}")
+                return f"error our tracker: {reply or 'no reply'}"
+        now = time.monotonic()
+        self.check = {"kind": kind, "reason": reason, "own": own, "dots": check_dots(kind, own), "i": 0,
+                      "started": now, "shown": now, "run": [], "accept": False, "done_at": None, "tries": 0,
+                      "skipped": 0, "captured": 0, "points": {}, "reasons": {}, "fit_reasons": set(), "note": "",
+                      "blind": blind}
+        log(f"{kind} check: {reason}" + (" (our tracker's first: no gaze yet, so each click takes the look "
+                                         "up to it)" if blind else ""))
+        if kind == "full":
+            st = ask(SCREENS, "state", 0.5).split()
+            if len(st) >= 3 and st[0] == "ok":
+                self.screens_shown = (st[2] == "0") if st[1] == "always" else (st[2] == "1")
+                ask(SCREENS, "hide", 0.5)
+        self.to_helper("calpanel 1")
+        self.to_panel(f"show {'full' if kind == 'full' else 'quick'}")
+        self.show_dot()
+        if kind == "quick":
+            self.last_quick = now
+        return "ok"
+
+    def start_fit(self, reason):
+        # Eyes lost are what it's there to show, so it needs only the tracker sending.
+        if time.monotonic() - self.sample_at > 2:
+            return "error the headset is off or the eye tracker isn't sending"
+        now = time.monotonic()
+        self.check = {"kind": "fit", "reason": reason, "own": False, "dots": [], "i": 0, "started": now, "shown": now,
+                      "run": [], "accept": False, "done_at": None, "tries": 0, "skipped": 0, "captured": 0,
+                      "points": {}, "fit": FitCheck(), "drawn": {}, "drawn_at": 0.0, "step": None}
+        log(f"fit check: {reason}")
+        self.to_helper("calpanel 1")
+        self.to_panel("show fit")
+        self.to_panel("title Headset fit: adjust the headset while you watch")
+        self.to_panel("text Left click or Meta+J: guided check · Right click or Meta+K: done")
+        return "ok"
+
+    def fit_tick(self, now):
+        c = self.check
+        fit = c["fit"]
+        # The guided check: its dots at head-relative directions in the panel (the probe's
+        # screen fractions, spread over the middle of the panel), and its looks as the title.
+        step = fit.guide_step(now)
+        key = None if step is None else (step[0], step[1])
+        if key != c["step"]:
+            c["step"] = key
+            if step is None:
+                self.to_panel("dot 0 0 off")
+                self.to_panel("title Headset fit: adjust the headset while you watch")
+            elif step[0] == "dot":
+                fx, fy = step[1]
+                self.to_panel(f"dot {(0.5 - fx) * 32:.2f} {(0.5 - fy) * 24:.2f} look")
+                self.to_panel("title Look at the dot")
+            else:
+                self.to_panel("dot 0 0 off")
+                self.to_panel(f"title {step[1]}")
+        if now - c["drawn_at"] < FIT_EVERY:
+            return
+        c["drawn_at"] = now
+        drawn = c["drawn"]
+        for k in (0, 1):
+            word, (r, g, b) = fit.status(k)
+            sig, seen = fit.signal(k), fit.tracked_share(k, now)
+            cmd = (f"eye {k} {r:.2f} {g:.2f} {b:.2f} {-1 if sig is None else round(sig, 1):g} "
+                   f"{-1 if seen is None else round(seen * 20) / 20:g} {word.capitalize()}")
+            if drawn.get(k) != cmd:
+                drawn[k] = cmd
+                self.to_panel(cmd)
+        if fit.have_eye_data and fit.samples < 3 * MIN_REGION:
+            hints = ["Look around slowly: up, down, left and right. Or left click (Meta+J) for a guided check."]
+        else:
+            hints = fit.hints()
+        lines = [line for h in hints for line in wrap(h, FIT_HINT_WIDTH)][:8]
+        cmd = "hints " + "|".join(lines)
+        if drawn.get("hints") != cmd:
+            drawn["hints"] = cmd
+            self.to_panel(cmd)
+
+    def show_dot(self):
+        c = self.check
+        yaw, pitch, rnd = c["dots"][c["i"]]
+        if c["kind"] == "full":
+            self.to_panel(f"bg {ROUND_BG[rnd]}")
+            self.to_panel(f"title Gaze calibration: {ROUND_NAMES[rnd]} round, {rnd + 1} of 3")
+            self.to_panel("text Look at the dot and click (left click or Meta+J). Right click or Meta+K: stop")
+        elif c["kind"] == "five":
+            self.to_panel(f"text Look at the dot and click ({c['i'] + 1} of {len(c['dots'])})")
+        else:
+            self.to_panel("text Look at the dot")
+        self.to_panel(f"dot {yaw:.3f} {pitch:.3f} look")
+        self.last_progress = None
+        c["shown"] = time.monotonic()
+        c["run"], c["accept"], c["done_at"], c["accept_at"] = [], False, None, None
+
+    def on_sample(self, s):
+        self.sample_at = time.monotonic()
+        if self.pending:
+            self.run_pending()
+        unc = (s["src"].get("mmap1") or {}).get("unc")
+        if unc and min(unc) <= EYE_LOST:
+            self.seen_at = time.monotonic()
+            if self.away and self.back_since is None:
+                self.back_since = self.seen_at
+        c = self.check
+        if c and c["kind"] == "fit":
+            c["fit"].feed(s, self.sample_at)
+            return
+        if not c or c["done_at"]:
+            return
+        if c["own"]:
+            src = s["src"].get("own") or {}
+            if c["blind"]:
+                # Our tracker's first calibration: no gaze yet, so the dot stands in for it (the
+                # gaze can't seem to move) and a click takes the look up to it. ft-eyes checks
+                # the pupils held still (see the top).
+                yaw, pitch, _ = c["dots"][c["i"]]
+                src = {"hy": yaw, "hp": pitch}
+        else:
+            src = s["src"].get("mmap1") or {}
+            if "hy" not in src:
+                per = [s["src"].get(n) or {} for n in ("left", "right")]
+                per = [p for p in per if "hy" in p]
+                src = {"hy": statistics.fmean(p["hy"] for p in per), "hp": statistics.fmean(p["hp"] for p in per)} if per else {}
+        if "hy" not in src:
+            return
+        now = time.monotonic()
+        c["gaze_at"] = now
+        if now < c["shown"] + CHECK_SETTLE:
+            return
+        g = (src["hy"], src["hp"])
+        run = c["run"]
+        if run:
+            recent = run[-30:]
+            my, mp = statistics.median(p[2] for p in recent), statistics.median(p[3] for p in recent)
+            if math.hypot(g[0] - my, g[1] - mp) > 2.5 * CHECK_SPREAD:
+                run.clear()  # the eyes moved on (a saccade, a blink): start over
+        run.append((now, s, g[0], g[1]))
+        window = [p for p in run if p[0] >= now - CHECK_WINDOW]
+        held = now - run[0][0]
+        # The quick check's ring fills in quarters: each step is a new picture for the panel, so
+        # few of them keep it solid. The others wait for the click (see the top): no ring.
+        progress = math.floor(min(1.0, held / CHECK_WINDOW) * 4) / 4
+        if c["kind"] == "quick" and progress != self.last_progress:
+            yaw, pitch, _ = c["dots"][c["i"]]
+            self.to_panel(f"dot {yaw:.3f} {pitch:.3f} capture {progress:.2f}")
+            self.last_progress = progress
+        if c["accept"] and held >= 0.3 and len(window) >= 10:
+            sd, _ = spread([(p[2], p[3]) for p in window])
+            if sd <= ACCEPT_SPREAD:
+                self.capture(window)
+            return
+        if c["kind"] == "quick" and held >= CHECK_WINDOW and len(window) >= 20:
+            sd, _ = spread([(p[2], p[3]) for p in window])
+            if sd <= CHECK_SPREAD:
+                self.capture(window)
+            else:
+                del run[:len(run) // 2]  # not steady enough yet: keep trying with the newer half
+
+    def capture(self, window):
+        svc = self.svc
+        c = self.check
+        yaw, pitch, rnd = c["dots"][c["i"]]
+        samples = [p[1] for p in window]
+        rec = {"time": time.time(), "check": c["kind"], "dot": c["i"], "round": rnd, "true": [yaw, pitch],
+               "samples": len(samples), "own": c["own"]}
+        ok = True
+        if c["own"]:
+            eyes = []
+            for k in (0, 1):
+                seen = [smp["src"]["own"]["eyes"][k] for smp in samples
+                        if (smp["src"].get("own") or {}).get("eyes") and smp["src"]["own"]["eyes"][k]]
+                eyes.append((statistics.median(e[0] for e in seen), statistics.median(e[1] for e in seen)) if seen else None)
+            miss = [math.hypot(yaw - e[0], pitch - e[1]) if e else None for e in eyes]
+            rec.update(eyes=eyes, miss=miss)
+            t0, t1 = samples[0]["t"], samples[-1]["t"]
+            if c["kind"] == "full":
+                reply = ask(EYES, f"calib-point {t0:.6f} {t1:.6f} {yaw:.4f} {pitch:.4f}", 3.0)
+                rec["reply"] = reply
+                ok = reply.startswith("ok")
+            else:
+                try:
+                    svc.eyes_sock.sendto(f"click {t1:.6f} {yaw:.4f} {pitch:.4f}".encode(), EYES)
+                except OSError as e:
+                    rec["reply"], ok = f"our tracker isn't running ({e})", False
+            if ok:
+                svc.weights["own"].add(miss)
+        else:
+            why = {}
+            steady = steady_samples(samples, why=why)
+            rec["dropped"] = why
+            reads = {}
+            for name in ("action", "mmap1", "mmap2", "left", "right"):
+                pts = [(smp["src"][name]["hy"], smp["src"][name]["hp"]) for smp in steady
+                       if "hy" in (smp["src"].get(name) or {})]
+                if len(pts) >= 15:
+                    reads[name] = (statistics.median(p[0] for p in pts), statistics.median(p[1] for p in pts))
+            rec["reads"] = reads
+            main = ("left", "right") if svc.kind == "eyes" else (svc.source,)
+            if not all(n in reads for n in main):
+                ok = False
+                rec["reply"] = f"only {len(steady)} of {len(samples)} samples had both eyes"
+            elif c["kind"] == "full":
+                for name, (hy, hp) in reads.items():
+                    c["points"].setdefault(name, []).append((hy, hp, yaw - hy, pitch - hp))
+                try:
+                    with open(POINTS, "a") as f:
+                        for name, (hy, hp) in reads.items():
+                            f.write(json.dumps({"time": time.time(), "source": name, "hy": hy, "hp": hp,
+                                                "off": [yaw - hy, pitch - hp], "layout": None, "how": "panel"}) + "\n")
+                except OSError:
+                    pass
+            else:
+                miss = []
+                for name in main:
+                    hy, hp = reads[name]
+                    cy, cp = svc.correction(name, hy, hp)
+                    miss.append(math.hypot(yaw - hy - cy, pitch - hp - cp))
+                    svc.lives[name].add({"time": time.time(), "hy": hy, "hp": hp, "dy": yaw - hy, "dp": pitch - hp,
+                                         "wy": 1.0, "wp": 1.0, "how": "check"}, svc.models[name], svc.mode)
+                rec["miss"] = miss
+                if svc.kind == "eyes":
+                    svc.weights["steam"].add(miss)
+                svc.dirty = True
+        if not ok:
+            short, long, fit = reject_reason(rec.get("reply", "") if c["own"] else None, rec.get("dropped"))
+            rec["reason"] = long
+            c["reasons"][long] = c["reasons"].get(long, 0) + 1
+            if fit:
+                c["fit_reasons"].add(long)
+        rec["accepted"] = ok
+        self.log_check(rec)
+        if not ok:
+            c["tries"] += 1
+            log(f"{c['kind']} check dot {c['i'] + 1}: not taken: {rec['reason']} ({rec.get('reply', '')})")
+            full = c["kind"] == "full"
+            if c["tries"] >= 2 or not full:
+                self.note(f"Dot skipped: {long}" + (f" ({FIT_HINT})" if fit else "") if full else f"Skipped: {short}")
+                self.skip()
+            else:
+                self.note(f"Not taken: {long}. Look at the dot and click again")
+                self.to_panel(f"dot {yaw:.3f} {pitch:.3f} fail")
+                c["run"], c["accept"], c["accept_at"] = [], False, None
+                c["shown"] = time.monotonic()  # settle again, then retry
+            return
+        c["captured"] += 1
+        c["tries"] = 0
+        self.note("")
+        self.to_panel(f"dot {yaw:.3f} {pitch:.3f} done")
+        c["done_at"] = time.monotonic() + DONE_PAUSE
+
+    def note(self, text):
+        """The panel's warning line, over the instructions (empty: none). It stays until the
+        next dot is taken, so a skipped dot's reason is still there at the one after it."""
+        c = self.check
+        if c.get("note") != text:
+            c["note"] = text
+            self.to_panel(f"note {text}".rstrip())
+
+    def skip(self):
+        c = self.check
+        yaw, pitch, _ = c["dots"][c["i"]]
+        c["skipped"] += 1
+        c["tries"] = 0
+        self.to_panel(f"dot {yaw:.3f} {pitch:.3f} fail")
+        c["done_at"] = time.monotonic() + DONE_PAUSE
+
+    def advance(self):
+        c = self.check
+        c["i"] += 1
+        if c["i"] < len(c["dots"]):
+            self.show_dot()
+            return
+        self.finish()
+
+    def finish(self):
+        svc = self.svc
+        c = self.check
+        kind = c["kind"]
+        if kind in ("quick", "five"):
+            if c["captured"]:
+                log(f"{kind} check done ({c['captured']} of {len(c['dots'])} dots)")
+                self.after_quick = [] if kind == "quick" else None
+            self.close()
+            return
+        n = len(c["dots"])
+        if c["captured"] < n * 2 / 3:
+            reasons = c["reasons"]
+            main = max(reasons, key=reasons.get) if reasons else None
+            self.full_failed = main
+            log(f"calibration failed: only {c['captured']} of {n} dots; the old one stays"
+                + (f". Not taken: {', '.join(f'{r} ({k})' for r, k in reasons.items())}" if reasons else ""))
+            self.to_panel(f"text Calibration failed: only {c['captured']} of {n} dots. Try again from Input Settings")
+            if main:
+                fit = main in c["fit_reasons"]
+                self.note(f"Most dots: {main}" + (". Check headset fit on the Gaze page" if fit else ""))
+            self.to_panel("dot 0 0 off")
+            c["done_at"] = time.monotonic() + (8.0 if main else 3.0)  # time to read why
+            c["closing"] = True
+            return
+        self.full_failed = None
+        if c["own"]:
+            reply = ask(EYES, "calib-fit", 10.0)
+            log(f"calibration ({c['captured']} of {n} dots): our tracker says {reply or 'nothing'}")
+        else:
+            mode = svc.mode if svc.mode != "none" else DEFAULT_MODEL
+            for name, pts in c["points"].items():
+                if name in svc.models and pts:
+                    svc.models[name].fit(pts, mode)
+            d = {name: m.to_json() for name, m in svc.models.items()}
+            d["_meta"] = {"calibrated_at": time.time(), "model": mode, "how": "panel"}
+            d["_live"] = {}
+            tmp = CALIBRATION.with_suffix(".tmp")
+            tmp.write_text(json.dumps(d, indent=1))
+            tmp.replace(CALIBRATION)
+            svc.forget_lessons()  # a new calibration: the pointer's lessons start over on top of it
+            svc.load_calibration()
+            svc.refit()
+            log(f"calibration ({c['captured']} of {n} dots): {mode}, saved")
+        self.close()
+
+    def close(self, why=None):
+        if not self.check:
+            return
+        if why:
+            log(f"{self.check['kind']} check closed: {why}")
+        self.to_panel("hide")
+        self.to_helper("calpanel 0")
+        if self.check["kind"] == "full" and self.screens_shown:
+            ask(SCREENS, "show", 0.5)
+        self.screens_shown = None
+        self.check = None
+
+    def quit(self):
+        c = self.check
+        if not c:
+            return
+        self.close("quit")
+        if c["kind"] == "full" and self.calibrated() is False:
+            # No calibration still: gaze mode can't work, so it goes off until it's turned on again.
+            try:
+                write_gaze(False)
+            except OSError as e:
+                log(f"can't write {CONF}: {e}")
+            self.to_helper("gaze off")
+            self.gaze_on = False
+            log("gaze mode off: no calibration")
+
+    def log_check(self, rec):
+        try:
+            with open(CHECK_LOG, "a") as f:
+                f.write(json.dumps(rec) + "\n")
+        except OSError:
+            pass
+
+    # --- From the service ---
+
+    def run_pending(self):
+        """A check asked for while the service idled: once the tracker sends (and our tracker has
+        said whether it's calibrated), or WAKE_SETTLE after waking, when its error is the real one."""
+        svc = self.svc
+        words, _ = self.pending
+        ready = (time.monotonic() - svc.last_sample < 2 if words[0] == "fitcheck" else self.can_run()) \
+            and (svc.kind != "own" or self.calibrated() is not None)
+        if not ready and svc.waking():
+            return
+        self.pending = None
+        reply = self.command(words, queue=False)
+        if reply != "ok":
+            log(f"{words[0]}, asked for while idle: {reply.removeprefix('error ')}")
+
+    def command(self, words, queue=True):
+        """quickcal, calibrate, calaccept, calquit -> a reply."""
+        cmd = words[0]
+        if cmd in ("quickcal", "calibrate", "fitcheck") and queue and self.svc.waking() and not self.check:
+            # The tracker isn't running (or only just started): wake it, and do this once it sends.
+            self.pending = (words, time.monotonic())
+            self.svc.update_awake()
+            return "ok waking the eye tracker first"
+        if cmd == "quickcal":
+            return self.start("full" if self.calibrated() is False else "quick", "asked for")
+        if cmd == "calibrate":
+            return self.start("full", "asked for")
+        if cmd == "fitcheck":
+            return self.start("fit", "asked for")
+        if cmd == "calaccept":
+            if self.check and self.check["kind"] == "fit":
+                self.check["fit"].toggle_guide(time.monotonic())
+            elif self.check:
+                if not self.check["accept"]:
+                    self.check["accept_at"] = time.monotonic()
+                self.check["accept"] = True
+            return "ok"
+        if cmd == "calquit":
+            self.quit()
+            return "ok"
+        if cmd == "recheck" and len(words) == 2:
+            self.auto_quick(f"a click was corrected {words[1]} deg")
+            return "ok"
+        return "error unknown command"
+
+    def after_lesson(self, rec):
+        if self.after_quick is None or "refused" in rec:
+            return
+        self.after_quick.append(rec.get("lesson_deg", 0.0))
+        if len(self.after_quick) < FIVE_COUNT:
+            return
+        off = self.after_quick
+        self.after_quick = None
+        if all(d > FIVE_LIMIT for d in off):
+            log(f"the {FIVE_COUNT} lessons after the quick check were {', '.join(f'{d:.1f}' for d in off)} deg off")
+            if self.gaze_on and self.can_run() and not self.check:
+                self.start("five", "the quick check didn't fix it")
+
+    def tick(self):
+        c = self.check
+        if not c:
+            return
+        now = time.monotonic()
+        if c["kind"] == "fit":
+            # Not closed when the eyes go: adjusting the headset loses them.
+            if now - c["started"] > FIT_TIMEOUT:
+                self.close("timed out")
+            else:
+                self.fit_tick(now)
+            return
+        if c["done_at"] and now >= c["done_at"]:
+            if c.get("closing"):
+                self.close()
+            else:
+                self.advance()
+            return
+        if not self.eyes_seen(EYES_GONE):
+            self.close("the headset came off")
+            return
+        if c["done_at"]:
+            return
+        if c["accept"] and c["accept_at"] and now - c["accept_at"] > ACCEPT_WAIT:
+            # Clicked, but no capture yet (see on_sample): say what it's waiting for.
+            full = c["kind"] == "full"
+            if now - c.get("gaze_at", 0.0) > 0.5:
+                self.note("Waiting: the eye tracker isn't sending a gaze" if full else "Waiting: no gaze")
+            else:
+                self.note("Waiting for your gaze to hold still on the dot" if full else "Hold your look still")
+        if c["kind"] == "quick" and now - c["started"] > QUICK_TIMEOUT:
+            self.close("ignored")
+        elif c["kind"] != "quick" and now - c["shown"] > CLICK_IDLE:
+            self.close(f"no click on dot {c['i'] + 1} for {CLICK_IDLE:.0f} s")
+
+    def periodic(self):
+        svc = self.svc
+        now = time.monotonic()
+        if not self.panel_proc and now >= self.panel_restart_at:
+            self.start_panel()
+        self.to_helper("gaze ? headset")
+        if now - self.gaze_heard > 5:
+            self.gaze_on = self.headset = None  # the helper isn't answering
+        if self.pending:
+            self.run_pending()
+        if self.check:
+            self.to_helper("calpanel 1")
+        if not self.eyes_seen(AWAY_MIN):
+            self.away, self.back_since = True, None
+        elif self.back_since is not None and not self.eyes_seen():
+            self.back_since = None  # gone again before DON_DELAY
+        elif self.back_since is not None and now - self.back_since >= DON_DELAY:
+            self.away, self.back_since = False, None
+            if not self.check:
+                # A calibration that closed unfinished opens again. Not one still open: gaze mode
+                # coming on wakes our tracker, so its eyes come back just as the calibration
+                # opens, and re-arming then opened a second one when the first ended.
+                self.full_armed = True
+            self.auto_quick("the headset went on")
+        if svc.kind == "own" and now - svc.own_at < 5:
+            reseat = any(e.get("reseat") for e in (svc.own.get("eyes") or {}).values())
+            if not reseat:
+                self.reseat_seen = False
+            elif not self.reseat_seen and self.can_run():
+                self.reseat_seen = True
+                self.auto_quick("our tracker asked for a click")
+        self.need_full("gaze mode is on without a calibration")
+
+    def status(self):
+        c = self.check
+        st = {"check": None, "gaze_mode": self.gaze_on, "headset_worn": self.headset, "pending": self.pending[0][0]
+              if self.pending else None, "calibrated": self.calibrated(), "eyes": self.eyes_seen(),
+              "problem": self.problem(),
+              "panel": self.panel_proc is not None,
+              "last_quick_s": round(time.monotonic() - self.last_quick) if self.last_quick else None}
+        if c:
+            st["check"] = {"kind": c["kind"], "dot": c["i"] + 1, "dots": len(c["dots"]), "captured": c["captured"],
+                           "skipped": c["skipped"], "reason": c["reason"]}
+        return st
+
+    def stop(self):
+        self.close("the gaze service is stopping")
+        self.stop_panel()

gaze/panel/ft-gazepanel.cpp

@@ -0,0 +1,538 @@
+// ft-gazepanel: the gaze calibration panel (docs/design.md, gaze/README.md). A SteamVR overlay
+// fixed to the headset, so wherever you turn your head it stays in the same place in your
+// view: a dot drawn at a head-relative direction is exactly that direction from the headset,
+// which is what the gaze service needs to know where you were looking. It's drawn on the CPU
+// and takes no input: the gaze service (gaze/ft-gazed) drives it, and the pointer helper
+// passes it your presses (calaccept, calquit).
+//
+// The panel sits POINTER-like at --distance (1.5 m, about where Frametop's screens are, so
+// the eyes converge as they do in use). "quick" is a small square, QUICK_DEG across, for the
+// one-dot check; "full" is FULL_DEG across (4:3), with a solid background whose brightness the
+// service sets per round (pupil size changes with it, and the tracker's error with it); "fit"
+// is FIT_DEG across (4:3), see-through like quick, for the headset fit check: a card per eye
+// (tracked or lost, the tracker's signal, how much of the last 10 s it was seen) and hints.
+//
+// Control socket: abstract unix datagram "@ft_gazepanel" (--socket NAME); a sender with an
+// address gets "ok" or "error ...":
+//   show quick|full|fit          the panel, empty, in front of you
+//   hide
+//   bg <0..1>                    the background's brightness (full)
+//   dot <yaw> <pitch> <state> [<progress 0..1>]
+//                                the dot, head-relative degrees (yaw +left, pitch +up); state:
+//                                look, capture (a ring filling to progress), done,
+//                                fail, off
+//   title <text> / text <text>   a line at the top / at the bottom (empty to clear)
+//   note <text>                  a warning line just above the bottom one, in orange (red on
+//                                the bright round): why a dot wasn't taken (empty to clear)
+//   eye <0|1> <r> <g> <b> <signal 0..1|-1> <seen 0..1|-1> <word>
+//                                fit: an eye's card (0 left): its state in that colour, the
+//                                tracker's signal, the share of the last 10 s it was seen
+//   hints <line>|<line>|...      fit: lines under the cards (empty to clear)
+//   ping
+//
+// Each picture goes into the next of three shared buffers (linear DMA-BUFs SteamVR imported
+// once, the size of the biggest panel; the texture bounds show the part in use), and the panel
+// switches to it, as screens/keyboard.cpp does. SetOverlayRaw, which uploads a new texture each
+// time, flickered on every change of the full calibration's 1024x768 picture, and in a live
+// test the headset kept showing an old picture after the panel had drawn new ones (2026-10-01).
+// It's only the fallback. A "show" makes the panel visible once its first picture is in.
+//
+// Options: --watch-stdin (quit when stdin closes: the service runs it), --socket NAME,
+// --distance METRES. Runs in the dev container (gaze/build.sh builds it into gaze/build).
+#include <openvr.h>
+
+#define STB_TRUETYPE_IMPLEMENTATION
+#include "stb_truetype.h"
+
+#include <drm_fourcc.h>
+#include <fcntl.h>
+#include <gbm.h>
+#include <poll.h>
+#include <sys/socket.h>
+#include <sys/un.h>
+#include <unistd.h>
+
+#include <algorithm>
+#include <atomic>
+#include <chrono>
+#include <cmath>
+#include <csignal>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <map>
+#include <string>
+#include <thread>
+#include <vector>
+
+namespace {
+
+using Clock = std::chrono::steady_clock;
+constexpr double kQuickDeg = 16;      // QUICK_DEG: the one-dot check's square
+constexpr double kFullDeg = 64;       // FULL_DEG: the full calibration's width (4:3)
+constexpr double kFitDeg = 40;        // FIT_DEG: the headset fit check's width (4:3)
+constexpr int kQuickPx = 320, kFullW = 1024, kFullH = 768, kFitW = 800, kFitH = 600;
+std::atomic<bool> g_stop{false};
+
+// ---------------------------------------------------------------- text (as screens/keyboard.cpp)
+
+stbtt_fontinfo g_font;
+std::vector<unsigned char> g_fontData;
+bool g_fontOk = false;
+constexpr int kMaxW = kFullW, kMaxH = kFullH;  // the buffers' size: the biggest panel
+struct Glyph {
+    std::vector<unsigned char> bitmap;
+    int w = 0, h = 0, xoff = 0, yoff = 0, advance = 0;
+};
+std::map<std::pair<uint32_t, int>, Glyph> g_glyphs;
+
+void LoadFont() {
+    std::string path;
+    if (FILE *p = popen("fc-match -f '%{file}' 'Noto Sans' 2>/dev/null", "r")) {
+        char buf[512];
+        if (std::fgets(buf, sizeof buf, p)) path = buf;
+        pclose(p);
+    }
+    if (path.empty()) path = "/usr/share/fonts/google-noto-vf/NotoSans[wght].ttf";
+    if (FILE *f = std::fopen(path.c_str(), "rb")) {
+        std::fseek(f, 0, SEEK_END);
+        g_fontData.resize(size_t(std::ftell(f)));
+        std::fseek(f, 0, SEEK_SET);
+        g_fontOk = std::fread(g_fontData.data(), 1, g_fontData.size(), f) == g_fontData.size() &&
+                   stbtt_InitFont(&g_font, g_fontData.data(), stbtt_GetFontOffsetForIndex(g_fontData.data(), 0));
+        std::fclose(f);
+    }
+    if (!g_fontOk) std::fprintf(stderr, "ft-gazepanel: no font (%s); no text\n", path.c_str());
+}
+
+const Glyph &GetGlyph(uint32_t cp, int size) {
+    auto [it, fresh] = g_glyphs.try_emplace({cp, size});
+    Glyph &g = it->second;
+    if (fresh) {
+        const float scale = stbtt_ScaleForPixelHeight(&g_font, float(size));
+        unsigned char *b = stbtt_GetCodepointBitmap(&g_font, 0, scale, int(cp), &g.w, &g.h, &g.xoff, &g.yoff);
+        if (b) g.bitmap.assign(b, b + size_t(g.w) * g.h), stbtt_FreeBitmap(b, nullptr);
+        int adv, lsb;
+        stbtt_GetCodepointHMetrics(&g_font, int(cp), &adv, &lsb);
+        g.advance = int(std::lround(adv * scale));
+    }
+    return g;
+}
+
+std::vector<uint32_t> Codepoints(const std::string &s) {
+    std::vector<uint32_t> out;
+    for (const unsigned char *p = (const unsigned char *)s.c_str(); *p;) {
+        uint32_t c = *p++;
+        int more = c >= 0xF0 ? 3 : c >= 0xE0 ? 2 : c >= 0xC0 ? 1 : 0;
+        if (more) c &= 0x3Fu >> more;
+        for (; more && (*p & 0xC0) == 0x80; --more) c = c << 6 | (*p++ & 0x3F);
+        out.push_back(c);
+    }
+    return out;
+}
+
+// ---------------------------------------------------------------- the picture
+
+struct EyeCard {
+    std::string word = "no data";
+    double r = 0.6, g = 0.6, b = 0.6, signal = -1, seen = -1;
+};
+
+struct Panel {
+    bool full = false, fit = false;
+    EyeCard eyes[2];
+    std::vector<std::string> hints;
+    int w = kQuickPx, h = kQuickPx;
+    double wDeg = kQuickDeg;  // across
+    std::vector<uint8_t> px;
+    double bg = 0.05;
+    std::string title, text, note;
+    bool dotOn = false;
+    double dotYaw = 0, dotPitch = 0, progress = 0;
+    std::string state = "off";
+};
+
+void Blend(Panel &p, int x, int y, double r, double g, double b, double a) {
+    if (x < 0 || y < 0 || x >= p.w || y >= p.h || a <= 0) return;
+    uint8_t *q = &p.px[(size_t(y) * p.w + x) * 4];
+    a = std::min(a, 1.0);
+    q[0] = uint8_t(std::lround(q[0] + (r * 255 - q[0]) * a));
+    q[1] = uint8_t(std::lround(q[1] + (g * 255 - q[1]) * a));
+    q[2] = uint8_t(std::lround(q[2] + (b * 255 - q[2]) * a));
+    q[3] = uint8_t(std::lround(q[3] + (255 - q[3]) * a));
+}
+
+// A filled disc, or (inner > 0) a ring from inner to outer radius, anti-aliased; with sweep < 1,
+// only that share of the ring, clockwise from the top.
+void Disc(Panel &p, double cx, double cy, double outer, double inner, double r, double g, double b, double a,
+          double sweep = 1) {
+    const int x0 = int(cx - outer - 1), x1 = int(cx + outer + 1), y0 = int(cy - outer - 1), y1 = int(cy + outer + 1);
+    for (int y = y0; y <= y1; ++y)
+        for (int x = x0; x <= x1; ++x) {
+            const double dx = x + 0.5 - cx, dy = y + 0.5 - cy, d = std::hypot(dx, dy);
+            double cover = std::clamp(outer - d + 0.5, 0.0, 1.0);
+            if (inner > 0) cover = std::min(cover, std::clamp(d - inner + 0.5, 0.0, 1.0));
+            if (sweep < 1) {
+                double ang = std::atan2(dx, -dy) / (2 * M_PI);  // 0 at the top, clockwise
+                if (ang < 0) ang += 1;
+                if (ang > sweep) continue;
+            }
+            Blend(p, x, y, r, g, b, a * cover);
+        }
+}
+
+void Rect(Panel &p, int x0, int y0, int x1, int y1, double r, double g, double b, double a) {
+    for (int y = std::max(y0, 0); y < std::min(y1, p.h); ++y)
+        for (int x = std::max(x0, 0); x < std::min(x1, p.w); ++x) Blend(p, x, y, r, g, b, a);
+}
+
+// Text centred on (x, cy), or starting at x (left).
+void Text(Panel &p, const std::string &s, int size, int x, int cy, double r, double g, double b, bool left = false) {
+    if (!g_fontOk || s.empty()) return;
+    const auto cps = Codepoints(s);
+    int width = 0;
+    for (uint32_t cp : cps) width += GetGlyph(cp, size).advance;
+    int ascent, descent, gap;
+    stbtt_GetFontVMetrics(&g_font, &ascent, &descent, &gap);
+    const float scale = stbtt_ScaleForPixelHeight(&g_font, float(size));
+    if (!left) x -= width / 2;
+    const int baseline = cy + int(std::lround((ascent + descent) * scale / 2));
+    for (uint32_t cp : cps) {
+        const Glyph &gl = GetGlyph(cp, size);
+        for (int gy = 0; gy < gl.h; ++gy)
+            for (int gx = 0; gx < gl.w; ++gx)
+                Blend(p, x + gl.xoff + gx, baseline + gl.yoff + gy, r, g, b, gl.bitmap[size_t(gy) * gl.w + gx] / 255.0);
+        x += gl.advance;
+    }
+}
+
+void Text(Panel &p, const std::string &s, int size, int x, int cy, double lum, bool left = false) {
+    Text(p, s, size, x, cy, lum, lum, lum, left);
+}
+
+// The headset fit check (see the top): a card per eye, then the hints.
+void DrawFit(Panel &p, double pxPerDeg, int textSize, double faint) {
+    const int margin = int(p.w * 0.06), cw = int(p.w * 0.41), ch = int(p.h * 0.32), top = int(p.h * 0.12);
+    const int pad = int(pxPerDeg * 0.9), big = int(pxPerDeg * 1.6), small = int(pxPerDeg * 0.8);
+    for (int k = 0; k < 2; ++k) {
+        const EyeCard &e = p.eyes[k];
+        const int x0 = k == 0 ? margin : p.w - margin - cw;
+        Rect(p, x0, top, x0 + cw, top + ch, 1, 1, 1, 0.07);
+        Text(p, k ? "Right eye" : "Left eye", textSize, x0 + pad, top + pad + textSize / 2, faint, true);
+        Text(p, e.word, big, x0 + pad, top + int(ch * 0.40), e.r, e.g, e.b, true);
+        // The tracker's signal: a bar, red to green.
+        const int by = top + int(ch * 0.62), bh = std::max(4, int(pxPerDeg * 0.35)), bw = cw - 2 * pad;
+        Text(p, "Signal", small, x0 + pad, by - small, faint, true);
+        Rect(p, x0 + pad, by, x0 + pad + bw, by + bh, 1, 1, 1, 0.15);
+        if (e.signal >= 0) {
+            const double v = std::clamp(e.signal, 0.0, 1.0);
+            const double r = v < 0.5 ? 1.0 : 1.0 - 1.3 * (v - 0.5), g = v < 0.5 ? 0.3 + 0.9 * v : 0.75 + 0.5 * (v - 0.5);
+            Rect(p, x0 + pad, by, x0 + pad + int(bw * v), by + bh, r, g, 0.35, 0.95);
+        }
+        char seen[64] = "Seen: not yet";
+        if (e.seen >= 0) std::snprintf(seen, sizeof seen, "Seen %d%% of the last 10 s", int(std::lround(e.seen * 100)));
+        Text(p, seen, small, x0 + pad, top + ch - pad, faint, true);
+    }
+    int y = top + ch + pad * 2;
+    for (const std::string &line : p.hints) {
+        Text(p, line, textSize, margin, y, faint, true);
+        y += int(textSize * 1.4);
+    }
+}
+
+// Head-relative direction -> panel pixel: the panel is a plane `d` in front of the headset.
+void ToPixel(const Panel &p, double yaw, double pitch, double &x, double &y) {
+    const double yr = yaw * M_PI / 180, pr = pitch * M_PI / 180;
+    const double half = std::tan(p.wDeg * M_PI / 360);  // half the width, per unit of distance
+    const double X = -std::tan(yr), Y = std::tan(pr) / std::cos(yr);
+    x = (X / (2 * half) + 0.5) * p.w;
+    y = (0.5 - Y / (2 * half) * p.w / p.h) * p.h;
+}
+
+void Draw(Panel &p) {
+    p.px.assign(size_t(p.w) * p.h * 4, 0);
+    const double pxPerDeg = p.w / p.wDeg;
+    if (p.full) {
+        for (size_t i = 0; i < p.px.size(); i += 4)
+            p.px[i] = p.px[i + 1] = p.px[i + 2] = uint8_t(std::lround(p.bg * 255)), p.px[i + 3] = 255;
+    } else {
+        // The quick check and the fit check: a dim rounded panel, see-through, so it's clear of what's behind.
+        const double r = std::min(p.w, p.h) * 0.12;
+        for (int y = 0; y < p.h; ++y)
+            for (int x = 0; x < p.w; ++x) {
+                const double dx = std::max({r - x - 0.5, x + 0.5 - (p.w - r), 0.0});
+                const double dy = std::max({r - y - 0.5, y + 0.5 - (p.h - r), 0.0});
+                const double cover = std::clamp(r - std::hypot(dx, dy) + 0.5, 0.0, 1.0);
+                Blend(p, x, y, 0.06, 0.06, 0.07, 0.82 * cover);
+            }
+    }
+    const bool light = p.full && p.bg > 0.5;  // a dark dot on the bright round
+    const double ink = light ? 0.0 : 1.0, faint = light ? 0.2 : 0.75;
+    const int titleSize = int(pxPerDeg * (p.full ? 1.5 : 1.1)), textSize = int(pxPerDeg * (p.full ? 1.2 : 0.9));
+    Text(p, p.title, titleSize, p.w / 2, int(titleSize * 1.2), faint);
+    Text(p, p.text, textSize, p.w / 2, p.h - int(textSize * 1.3), faint);
+    // Under the full calibration's lowest dots (RING degrees down) and over the text.
+    if (light) Text(p, p.note, textSize, p.w / 2, p.h - int(textSize * 2.8), 0.7, 0.12, 0.05);
+    else Text(p, p.note, textSize, p.w / 2, p.h - int(textSize * 2.8), 1.0, 0.62, 0.3);
+    if (p.fit) DrawFit(p, pxPerDeg, textSize, faint);
+    if (!p.dotOn || p.state == "off") return;
+    double x, y;
+    ToPixel(p, p.dotYaw, p.dotPitch, x, y);
+    const double core = 0.22 * pxPerDeg;
+    if (p.state == "look") {
+        // Still, so the panel looks solid and is drawn again only when something changes.
+        const double rr = 0.75 * pxPerDeg;
+        Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, ink, ink, ink, 0.6);
+        Disc(p, x, y, core, 0, ink, ink, ink, 1);
+    } else if (p.state == "capture") {
+        const double rr = 0.75 * pxPerDeg;
+        Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, ink, ink, ink, 0.25);
+        Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, 0.3, 0.85, 1.0, 1, std::clamp(p.progress, 0.0, 1.0));
+        Disc(p, x, y, core, 0, ink, ink, ink, 1);
+    } else if (p.state == "done") {
+        Disc(p, x, y, 0.75 * pxPerDeg, 0, 0.25, 0.85, 0.4, 0.9);
+        Disc(p, x, y, core, 0, 1, 1, 1, 1);
+    } else if (p.state == "fail") {
+        Disc(p, x, y, 0.75 * pxPerDeg, 0.6 * pxPerDeg, 0.95, 0.35, 0.3, 0.9);
+        Disc(p, x, y, core, 0, ink, ink, ink, 1);
+    }
+}
+
+
+// ---------------------------------------------------------------- the buffers (see the top)
+
+struct Buffer {
+    gbm_bo *bo = nullptr;
+    int fd = -1;
+    vr::SharedTextureHandle_t handle = 0;
+};
+
+struct Buffers {
+    int drm = -1;
+    gbm_device *gbm = nullptr;
+    Buffer b[3];
+    int next = 0;
+    bool ok = false;
+
+    bool Make() {
+        drm = open("/dev/dri/renderD128", O_RDWR | O_CLOEXEC);
+        if (drm >= 0) gbm = gbm_create_device(drm);
+        for (Buffer &x : b) {
+            // ABGR8888 is R, G, B, A in memory, like Panel::px.
+            if (gbm) x.bo = gbm_bo_create(gbm, kMaxW, kMaxH, GBM_FORMAT_ABGR8888, GBM_BO_USE_RENDERING | GBM_BO_USE_LINEAR);
+            if (!x.bo || (x.fd = gbm_bo_get_fd(x.bo)) < 0) break;
+            vr::DmabufAttributes_t a{};
+            a.unWidth = kMaxW, a.unHeight = kMaxH;
+            a.unDepth = a.unMipLevels = a.unArrayLayers = a.unSampleCount = 1;
+            a.unFormat = DRM_FORMAT_ABGR8888;
+            a.ulModifier = DRM_FORMAT_MOD_LINEAR;
+            a.unPlaneCount = 1;
+            a.plane[0].unOffset = gbm_bo_get_offset(x.bo, 0);
+            a.plane[0].unStride = gbm_bo_get_stride(x.bo);
+            a.plane[0].nFd = x.fd;
+            if (!vr::VRIPCResourceManager()->ImportDmabuf(vr::VRApplication_Overlay, &a, &x.handle)) x.handle = 0;
+            if (!x.handle) break;
+        }
+        ok = b[2].handle != 0;
+        if (!ok) {
+            std::fprintf(stderr, "ft-gazepanel: no shared buffers; falling back to SetOverlayRaw (it flickers)\n");
+            Drop();
+        }
+        return ok;
+    }
+
+    void Drop() {
+        for (Buffer &x : b) {
+            if (x.handle) vr::VRIPCResourceManager()->UnrefResource(x.handle);
+            if (x.fd >= 0) close(x.fd);
+            if (x.bo) gbm_bo_destroy(x.bo);
+            x = Buffer{};
+        }
+        if (gbm) gbm_device_destroy(gbm);
+        if (drm >= 0) close(drm);
+        gbm = nullptr, drm = -1, ok = false;
+    }
+
+    // p's picture to the overlay: into the next buffer, premultiplied (the overlay's flag says
+    // so), then the overlay switches to it.
+    void Present(vr::IVROverlay *ov, vr::VROverlayHandle_t h, const Panel &p) {
+        vr::EVROverlayError e;
+        if (!ok) {
+            e = ov->SetOverlayRaw(h, const_cast<uint8_t *>(p.px.data()), uint32_t(p.w), uint32_t(p.h), 4);
+        } else {
+            Buffer &x = b[next];
+            next = (next + 1) % 3;
+            uint32_t stride = 0;
+            void *mapping = nullptr;
+            auto *dst = static_cast<uint8_t *>(gbm_bo_map(x.bo, 0, 0, p.w, p.h, GBM_BO_TRANSFER_WRITE, &stride, &mapping));
+            if (!dst) {
+                std::fprintf(stderr, "ft-gazepanel: can't map a buffer\n");
+                return;
+            }
+            for (int y = 0; y < p.h; ++y) {
+                const uint8_t *src = &p.px[size_t(y) * p.w * 4];
+                uint8_t *row = dst + size_t(y) * stride;
+                for (int i = 0; i < p.w * 4; i += 4) {
+                    const unsigned a = src[i + 3];
+                    row[i] = uint8_t(src[i] * a / 255), row[i + 1] = uint8_t(src[i + 1] * a / 255);
+                    row[i + 2] = uint8_t(src[i + 2] * a / 255), row[i + 3] = uint8_t(a);
+                }
+            }
+            gbm_bo_unmap(x.bo, mapping);
+            const vr::VRTextureBounds_t bounds{0, 0, float(p.w) / kMaxW, float(p.h) / kMaxH};
+            ov->SetOverlayTextureBounds(h, &bounds);
+            vr::Texture_t tex = {&x.handle, vr::TextureType_SharedTextureHandle, vr::ColorSpace_Gamma};
+            e = ov->SetOverlayTexture(h, &tex);
+        }
+        if (e != vr::VROverlayError_None)
+            std::fprintf(stderr, "ft-gazepanel: the picture didn't go to SteamVR: %s\n", ov->GetOverlayErrorNameFromEnum(e));
+    }
+};
+
+}  // namespace
+
+int main(int argc, char **argv) {
+    bool watchStdin = false;
+    std::string sockName = "ft_gazepanel";
+    double distance = 1.5;
+    for (int i = 1; i < argc; ++i) {
+        if (!std::strcmp(argv[i], "--watch-stdin")) watchStdin = true;
+        else if (!std::strcmp(argv[i], "--socket") && i + 1 < argc) sockName = argv[++i];
+        else if (!std::strcmp(argv[i], "--distance") && i + 1 < argc) distance = std::clamp(std::atof(argv[++i]), 0.5, 5.0);
+        else {
+            std::fprintf(stderr, "usage: %s [--watch-stdin] [--socket NAME] [--distance METRES]\n", argv[0]);
+            return 2;
+        }
+    }
+    std::signal(SIGINT, [](int) { g_stop = true; });
+    std::signal(SIGTERM, [](int) { g_stop = true; });
+    if (watchStdin)
+        std::thread([] {
+            char c[256];
+            while (read(0, c, sizeof c) > 0) {
+            }
+            g_stop = true;
+        }).detach();
+
+    int sock = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
+    sockaddr_un addr{};
+    addr.sun_family = AF_UNIX;
+    std::memcpy(addr.sun_path + 1, sockName.data(), std::min(sockName.size(), sizeof addr.sun_path - 2));
+    if (bind(sock, reinterpret_cast<sockaddr *>(&addr), socklen_t(offsetof(sockaddr_un, sun_path) + 1 + sockName.size())) != 0) {
+        std::fprintf(stderr, "ft-gazepanel: @%s is taken (another copy running?)\n", sockName.c_str());
+        return 1;
+    }
+
+    // As Frametop's other SteamVR clients: background first, so we never start vrserver.
+    vr::EVRInitError err = vr::VRInitError_None;
+    vr::VR_Init(&err, vr::VRApplication_Background);
+    if (err == vr::VRInitError_None) {
+        vr::VR_Shutdown();
+        vr::VR_Init(&err, vr::VRApplication_Overlay);
+    }
+    if (err != vr::VRInitError_None) {
+        std::fprintf(stderr, "ft-gazepanel: SteamVR: %s\n", vr::VR_GetVRInitErrorAsEnglishDescription(err));
+        return 1;
+    }
+    vr::IVROverlay *ov = vr::VROverlay();
+    vr::VROverlayHandle_t h = vr::k_ulOverlayHandleInvalid;
+    if (ov->CreateOverlay("frametop.gazepanel", "Frametop gaze calibration", &h) != vr::VROverlayError_None) {
+        std::fprintf(stderr, "ft-gazepanel: can't create the overlay (another copy running?)\n");
+        return 1;
+    }
+    ov->SetOverlaySortOrder(h, 250);  // in front of Frametop's screens and the pointer's dot
+    LoadFont();
+    Buffers buffers;
+    if (buffers.Make()) ov->SetOverlayFlag(h, vr::VROverlayFlags_IsPremultiplied, true);
+
+    Panel p;
+    bool visible = false, dirty = false, shown = false;  // shown: SteamVR shows it (after its first picture)
+    auto place = [&] {
+        const double half = std::tan(p.wDeg * M_PI / 360);
+        vr::HmdMatrix34_t m{};
+        m.m[0][0] = m.m[1][1] = m.m[2][2] = 1;
+        m.m[2][3] = float(-distance);
+        ov->SetOverlayTransformTrackedDeviceRelative(h, vr::k_unTrackedDeviceIndex_Hmd, &m);
+        ov->SetOverlayWidthInMeters(h, float(2 * distance * half));
+    };
+    std::fprintf(stderr, "ft-gazepanel running: @%s, %.2f m\n", sockName.c_str(), distance);
+
+    while (!g_stop) {
+        char buf[512];
+        sockaddr_un from{};
+        socklen_t fromLen = sizeof from;
+        ssize_t n;
+        while ((n = recvfrom(sock, buf, sizeof buf - 1, 0, reinterpret_cast<sockaddr *>(&from), &fromLen)) > 0) {
+            buf[n] = 0;
+            std::string reply = "ok";
+            char word[16] = "", state[16] = "";
+            double a = 0, b = 0, c = 0, d = 0, e = 0;
+            int rest = 0;
+            if (!std::strncmp(buf, "show ", 5)) {
+                p.full = !std::strcmp(buf + 5, "full");
+                p.fit = !std::strcmp(buf + 5, "fit");
+                p.w = p.full ? kFullW : p.fit ? kFitW : kQuickPx;
+                p.h = p.full ? kFullH : p.fit ? kFitH : kQuickPx;
+                p.wDeg = p.full ? kFullDeg : p.fit ? kFitDeg : kQuickDeg;
+                p.title.clear(), p.text.clear(), p.note.clear(), p.dotOn = false, p.state = "off";
+                p.eyes[0] = p.eyes[1] = EyeCard{}, p.hints.clear();
+                place();
+                visible = dirty = true;  // shown with its first picture
+            } else if (!std::strcmp(buf, "hide")) {
+                ov->HideOverlay(h);
+                visible = shown = false;
+            } else if (std::sscanf(buf, "bg %lf", &a) == 1) {
+                p.bg = std::clamp(a, 0.0, 1.0), dirty = true;
+            } else if (std::sscanf(buf, "dot %lf %lf %15s %lf", &a, &b, state, &c) >= 3) {
+                p.dotYaw = a, p.dotPitch = b, p.state = state, p.progress = c;
+                p.dotOn = std::strcmp(state, "off") != 0, dirty = true;
+            } else if (int k; std::sscanf(buf, "eye %d %lf %lf %lf %lf %lf %n", &k, &a, &b, &c, &d, &e, &rest) >= 6 &&
+                       rest > 0 && (k == 0 || k == 1)) {
+                p.eyes[k] = EyeCard{buf + rest, a, b, c, d, e}, dirty = true;
+            } else if (!std::strncmp(buf, "hints", 5)) {
+                p.hints.clear();
+                std::string s = buf[5] == ' ' ? buf + 6 : "";
+                for (size_t at = 0; !s.empty() && at <= s.size();) {
+                    const size_t bar = std::min(s.find('|', at), s.size());
+                    p.hints.push_back(s.substr(at, bar - at));
+                    at = bar + 1;
+                }
+                dirty = true;
+            } else if (!std::strncmp(buf, "title", 5)) {
+                p.title = buf[5] == ' ' ? buf + 6 : "", dirty = true;
+            } else if (!std::strncmp(buf, "text", 4)) {
+                p.text = buf[4] == ' ' ? buf + 5 : "", dirty = true;
+            } else if (!std::strncmp(buf, "note", 4)) {
+                p.note = buf[4] == ' ' ? buf + 5 : "", dirty = true;
+            } else if (std::sscanf(buf, "%15s", word) == 1 && !std::strcmp(word, "ping")) {
+                reply = visible ? "ok shown" : "ok hidden";
+            } else {
+                reply = "error unknown command";
+            }
+            if (fromLen > offsetof(sockaddr_un, sun_path))
+                sendto(sock, reply.data(), reply.size(), 0, reinterpret_cast<sockaddr *>(&from), fromLen);
+            fromLen = sizeof from;
+        }
+        vr::VREvent_t ev;
+        while (vr::VRSystem()->PollNextEvent(&ev, sizeof ev))
+            if (ev.eventType == vr::VREvent_Quit) {
+                vr::VRSystem()->AcknowledgeQuit_Exiting();
+                g_stop = true;
+            }
+        if (visible && dirty) {
+            Draw(p);
+            buffers.Present(ov, h, p);
+            if (!shown) ov->ShowOverlay(h), shown = true;
+            dirty = false;
+        }
+        // Until a command comes, or 10 ms while shown (SteamVR's events). Hidden, it waits up to a
+        // second: it woke 20 to 30 times a second for nothing, the main cost left with gaze idle.
+        // A closed stdin (--watch-stdin) wakes it too, so quitting doesn't wait.
+        pollfd fds[2] = {{sock, POLLIN, 0}, {0, POLLIN, 0}};
+        poll(fds, watchStdin ? 2 : 1, visible ? 10 : 1000);
+    }
+    ov->DestroyOverlay(h);
+    buffers.Drop();
+    vr::VR_Shutdown();
+    return 0;
+}

gaze/probe/ft-gazeprobe.desktop

@@ -1,9 +1,9 @@
 [Desktop Entry]
 Type=Application
-Name=Frametop Gaze Probe
-GenericName=Eye tracking playground
-Comment=How accurate the headset's eye tracking is on your screens, and gaze clicking with calibration
+Name=Frametop Gaze Probe (development)
+GenericName=Eye tracking development tool
+Comment=For developing Frametop's gaze tracking. Day to day, the calibration and checks are on the Gaze page of Frametop Input Settings
 Exec=@REPO@/gaze/probe/ft-gazeprobe
 Icon=view-visible
-Categories=Utility;Development;
+Categories=Development;
 Keywords=eye;gaze;tracking;calibration;pointer;steamvr;frametop;

gaze/run.sh

@@ -12,8 +12,8 @@ case ${1:-status} in
     "$root/gaze/build.sh"
     fill_template "$root/gaze/$unit" | on_frame "mkdir -p ~/.config/systemd/user && cat > ~/.config/systemd/user/$unit"
     on_frame "chmod +x gaze/ft-gazed gaze/ft-gazectl"
-    "$frame" --host "set -e; systemctl --user daemon-reload; systemctl --user enable --now $unit
-sleep 2; echo \"$unit: \$(systemctl --user is-active $unit)\"; journalctl --user -u $unit --no-pager -o cat -n 5" ;;
+    "$frame" --host "set -e; systemctl --user daemon-reload; systemctl --user enable $unit
+$(start_with_steamvr $unit)" ;;
   uninstall) "$frame" --host "systemctl --user disable --now $unit 2>/dev/null; rm -f ~/.config/systemd/user/$unit; systemctl --user daemon-reload; echo removed" ;;
   start|stop|restart) "$frame" --host "systemctl --user $1 $unit; systemctl --user is-active $unit" ;;
   status) "$frame" --host "systemctl --user is-active $unit" || true; on_frame "gaze/ft-gazectl status" || true ;;

gaze/test/first-calibration-test.py

@@ -0,0 +1,238 @@
+#!/usr/bin/env python3
+"""Offline test of our own eye tracker's first calibration (gaze/gazecheck.py, "blind"): before
+it has a calibration, ft-eyes publishes no gaze, and the calibration must still open and take
+its dots. Until 2026-10-05 it couldn't: a fresh install that chose our tracker never calibrated.
+
+Runs ft-gazed's Service with its sockets renamed and HOME in a temp folder (state and settings
+go there), a fake pointer helper (gaze mode on, headset worn), a fake ft-gaze (SteamVR sees both
+eyes; "own" is {"ok":0}, as with an uncalibrated ft-eyes), a fake ft-eyes control socket, and no
+panel (a stand-in process). Nothing reaches the live gaze service, the pointer helper, ft-eyes,
+or SteamVR, so it's safe next to them.
+
+  gaze/test/first-calibration-test.py
+"""
+import os
+import tempfile
+
+HOME = tempfile.mkdtemp(prefix="ft-gaze-first-cal-test-")
+os.environ["HOME"] = HOME  # before gazecal: its STATE, and frametop.conf, follow HOME
+
+import importlib.machinery  # noqa: E402
+import importlib.util  # noqa: E402
+import json  # noqa: E402
+import selectors  # noqa: E402
+import shutil  # noqa: E402
+import socket  # noqa: E402
+import subprocess  # noqa: E402
+import sys  # noqa: E402
+import threading  # noqa: E402
+import time  # noqa: E402
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+GAZE = os.path.join(HERE, "..")
+sys.path.insert(0, GAZE)
+loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(GAZE, "ft-gazed"))
+gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
+loader.exec_module(gazed)
+import gazecheck  # noqa: E402  (the module ft-gazed imported)
+
+tag = f"ft_gaze_first_cal_test_{os.getpid()}"
+gazed.ME = f"\0{tag}_gazed"
+gazed.POINTER = gazecheck.POINTER = f"\0{tag}_helper"
+gazecheck.SCREENS = f"\0{tag}_screens"
+gazecheck.PANEL = f"\0{tag}_panel"
+gazed.EYES_SOCKET = gazecheck.EYES = f"\0{tag}_eyes"
+gazed.read_settings = lambda: ("own", "auto", "auto", 55.0)
+DOTS = 3
+real_dots = gazecheck.check_dots
+gazecheck.check_dots = lambda kind, own: real_dots(kind, own)[:DOTS]  # a short calibration
+logs = []
+gazed.log = gazecheck.log = lambda msg: logs.append(msg)
+
+# The fake ft-gaze: 90 samples a second, SteamVR sees both eyes, no gaze from ours.
+FAKE = os.path.join(HOME, "ft-gaze")
+with open(FAKE, "w") as f:
+    f.write('''import json, os, select, sys, time
+while True:
+    if select.select([sys.stdin], [], [], 1 / 90)[0]:
+        if not os.read(0, 4096):
+            break
+    print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
+          "open": [0.8, 0.8]}, "own": {"ok": 0}}}), flush=True)
+''')
+SLEEPER = [sys.executable, "-c", "import sys; sys.stdin.read()"]  # quits when its stdin closes
+
+
+def start_helper(self):
+    """ft-gaze, straight from here instead of the dev container."""
+    self.proc = subprocess.Popen([sys.executable, FAKE], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+                                 stderr=subprocess.PIPE)
+    self.proc_sources = self.wanted_sources()
+    os.set_blocking(self.proc.stdout.fileno(), False)
+    os.set_blocking(self.proc.stderr.fileno(), False)
+    self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
+    self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
+    self.buf = b""
+
+
+def start_eyes(self):
+    """ft-eyes' process: a stand-in. Its control socket is the fake below."""
+    self.eyes_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+    os.set_blocking(self.eyes_proc.stderr.fileno(), False)
+    self.sel.register(self.eyes_proc.stderr, selectors.EVENT_READ, "eyes")
+
+
+def start_panel(self):
+    self.panel_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+    os.set_blocking(self.panel_proc.stderr.fileno(), False)
+    self.sel.register(self.panel_proc.stderr, selectors.EVENT_READ, "panel")
+
+
+gazed.Service.start_helper = start_helper
+gazed.Service.start_eyes = start_eyes
+gazecheck.Checks.start_panel = start_panel
+
+# The fake ft-eyes: uncalibrated until calib-fit. "fail" answers the next calib-point with that.
+eyes_state = {"cal": None, "points": [], "fail": None}
+eyes = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+eyes.bind(gazed.EYES_SOCKET)
+eyes.settimeout(0.2)
+
+
+def eyes_answer():
+    while True:
+        try:
+            data, addr = eyes.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        w = data.decode().split()
+        if w[0] == "status":
+            reply = json.dumps({"calibration": eyes_state["cal"], "calibrating": False, "dots": 0,
+                                "eyes": {"right": {"reseat": False}, "left": {"reseat": False}}})
+        elif w[0] == "calib-start":
+            eyes_state["points"] = []
+            reply = "ok"
+        elif w[0] == "calib-point":
+            eyes_state["points"].append(tuple(map(float, w[1:5])))
+            reply, eyes_state["fail"] = eyes_state["fail"] or "ok 50 50 1.00 1.00", None
+        elif w[0] == "calib-fit":
+            eyes_state["cal"] = {"made": "test", "dots": len(eyes_state["points"])}
+            reply = f"ok {len(eyes_state['points'])} dots"
+        else:
+            reply = f"fail unknown command {w[0]}"
+        if addr:
+            eyes.sendto(reply.encode(), addr)
+
+
+threading.Thread(target=eyes_answer, daemon=True).start()
+
+helper_state = {"reply": "ok off worn", "heard": []}
+helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+helper.bind(gazed.POINTER)
+helper.settimeout(0.2)
+
+
+def helper_answer():
+    while True:
+        try:
+            data, addr = helper.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        if data.startswith(b"gaze ?") and addr:
+            helper.sendto(helper_state["reply"].encode(), addr)
+        else:
+            helper_state["heard"].append(data.decode())
+
+
+threading.Thread(target=helper_answer, daemon=True).start()
+svc = gazed.Service(None, False, gazed.POINTER)
+threading.Thread(target=svc.run, daemon=True).start()
+ctl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+ctl.bind("")
+ctl.settimeout(3)
+
+
+def ask(cmd):
+    ctl.sendto(cmd.encode(), gazed.ME)
+    return ctl.recv(65536).decode()
+
+
+failures = []
+
+
+def check(label, got, want):
+    ok = got == want
+    print(("ok    " if ok else "FAIL  ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
+    if not ok:
+        failures.append(label)
+
+
+def wait(cond, seconds):
+    end = time.monotonic() + seconds
+    while time.monotonic() < end:
+        if cond():
+            return True
+        time.sleep(0.05)
+    return cond()
+
+
+def check_state():
+    return svc.checks.check or {}
+
+
+def take_dot(i):
+    """Wait for dot i to settle, then click: is it taken?"""
+    wait(lambda: check_state().get("i") == i and not check_state().get("done_at"), 3)
+    time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1)
+    before = check_state().get("captured", 0)
+    ask("calaccept")
+    return wait(lambda: check_state().get("captured", 0) > before, 2)
+
+
+check("gaze mode off, Gaze page open (wake): ours runs, uncalibrated", ask("wake 60"), "ok")
+check("the service knows ours has no calibration", wait(lambda: svc.checks.calibrated() is False, 6), True)
+check("a quick check is refused while ours has no calibration", svc.checks.start("quick", "test"),
+      "error our tracker isn't calibrated yet: use Calibrate")
+helper_state["reply"] = "ok on worn"
+check("gaze mode on: the calibration opens by itself", wait(lambda: check_state().get("kind") == "full", 6), True)
+check("it runs blind (no gaze from ours yet)", check_state().get("blind"), True)
+check("nothing says it can't open", any("can't open" in m for m in logs), False)
+# Live 2026-10-05: turning gaze mode on woke our tracker, and the calibration opened before
+# DON_DELAY of eyes had passed, so "the headset went on" came while it was open. That re-armed
+# it, and a second calibration opened as soon as the first ended.
+svc.checks.away, svc.checks.back_since = True, None
+
+check("dot 1: a click takes it", take_dot(0), True)
+check("the headset went on while it was open",
+      wait(lambda: not svc.checks.away, gazecheck.DON_DELAY + 2) and bool(svc.checks.check), True)
+t0, t1, yaw, pitch = eyes_state["points"][0]
+dot = gazecheck.check_dots("full", True)[0]
+check("its window is the look up to the click (about CHECK_WINDOW)",
+      abs((t1 - t0) - gazecheck.CHECK_WINDOW) < 0.15, True)
+check("ft-eyes got that dot's direction", (round(yaw, 3), round(pitch, 3)), (round(dot[0], 3), round(dot[1], 3)))
+
+eyes_state["fail"] = "fail the left eye was seen in only 3 frames"
+wait(lambda: check_state().get("i") == 1 and not check_state().get("done_at"), 3)
+time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1)
+ask("calaccept")
+check("ft-eyes refusing a dot: its reason reaches the panel's note",
+      wait(lambda: "left eye in only 3 frames" in check_state().get("note", ""), 2), True)
+check("dot 2, second try: taken", take_dot(1), True)
+check("dot 3: taken", take_dot(2), True)
+
+check("all dots: ours fits its calibration (calib-fit)",
+      wait(lambda: eyes_state["cal"] is not None and not svc.checks.check, 4), True)
+check("the service sees it calibrated", wait(lambda: svc.checks.calibrated() is True, 4), True)
+check("and gaze mode stays on", "gaze off" in helper_state["heard"], False)
+check("and no second calibration opens", wait(lambda: svc.checks.check is not None, 3), False)
+check("one calibration in the log", sum(m.startswith("full check:") for m in logs), 1)
+
+print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
+svc.running = False
+time.sleep(0.7)
+shutil.rmtree(HOME, ignore_errors=True)
+os._exit(1 if failures else 0)

gaze/test/idle-test.py

@@ -0,0 +1,178 @@
+#!/usr/bin/env python3
+"""Offline test of the gaze service idling (gaze/ft-gazed, gaze/gazecheck.py): ft-gaze runs only
+while the gaze is in use, and a check asked for while it idles waits for the tracker.
+
+Runs ft-gazed's Service with its sockets renamed, a fake pointer helper (answers "gaze ?
+headset" as the test says), and a fake ft-gaze (prints samples, quits when its stdin closes).
+SteamVR's tracker is the one in use, so our own isn't started; the panel isn't either. Nothing
+reaches the live gaze service, the pointer helper, or SteamVR, so it's safe next to them.
+
+  gaze/test/idle-test.py
+"""
+import importlib.machinery
+import importlib.util
+import os
+import socket
+import subprocess
+import sys
+import tempfile
+import threading
+import time
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+GAZE = os.path.join(HERE, "..")
+sys.path.insert(0, GAZE)
+loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(GAZE, "ft-gazed"))
+gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
+loader.exec_module(gazed)
+import gazecheck  # noqa: E402  (the module ft-gazed imported)
+
+tag = f"ft_gaze_idle_test_{os.getpid()}"
+gazed.ME = f"\0{tag}_gazed"
+gazed.POINTER = gazecheck.POINTER = f"\0{tag}_helper"
+gazecheck.SCREENS = f"\0{tag}_screens"
+gazecheck.PANEL_PROG = gazed.REPO / "nonexistent-panel"  # "isn't built": no panel
+gazed.IDLE_AFTER, gazed.WAKE_SETTLE = 1.0, 3.0
+gazed.read_settings = lambda: ("steam", "steam", "auto", 55.0)
+logs = []
+gazed.log = gazecheck.log = lambda msg: logs.append(msg)
+
+# The fake ft-gaze: 90 samples a second, both eyes seen, until its stdin closes. It logs the
+# sources it was asked for: "argv LIST" (--sources), then "line LIST" for each "sources LIST".
+tmp = tempfile.mkdtemp(prefix="ft-gaze-idle-test-")
+FAKE = os.path.join(tmp, "ft-gaze")
+SOURCES_LOG = os.path.join(tmp, "sources.log")
+with open(FAKE, "w") as f:
+    f.write('''import json, os, select, sys, time
+log = open(sys.argv[1], "a", buffering=1)
+log.write("argv " + (sys.argv[sys.argv.index("--sources") + 1] if "--sources" in sys.argv else "-") + "\\n")
+buf = b""
+while True:
+    if select.select([sys.stdin], [], [], 1 / 90)[0]:
+        data = os.read(0, 4096)
+        if not data:
+            break
+        buf += data
+        while b"\\n" in buf:
+            line, buf = buf.split(b"\\n", 1)
+            if line.startswith(b"sources "):
+                log.write("line " + line[8:].decode() + "\\n")
+    eye = {"hy": 1.0, "hp": 2.0}
+    print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
+          "open": [0.8, 0.8]}, "left": eye, "right": eye}}), flush=True)
+''')
+started = []
+
+
+def start_helper(self):
+    """ft-gaze, straight from here instead of the dev container."""
+    import selectors
+    self.proc = subprocess.Popen([sys.executable, FAKE, SOURCES_LOG, "--sources", self.wanted_sources()],
+                                 stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+    self.proc_sources = self.wanted_sources()
+    os.set_blocking(self.proc.stdout.fileno(), False)
+    os.set_blocking(self.proc.stderr.fileno(), False)
+    self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
+    self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
+    self.buf = b""
+    started.append(time.monotonic())
+
+
+gazed.Service.start_helper = start_helper
+
+# The fake pointer helper.
+helper_state = {"reply": "ok off worn"}
+helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+helper.bind(gazed.POINTER)
+helper.settimeout(0.2)
+
+
+def answer():
+    while True:
+        try:
+            data, addr = helper.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        if data.startswith(b"gaze ?") and addr:
+            helper.sendto(helper_state["reply"].encode(), addr)
+
+
+threading.Thread(target=answer, daemon=True).start()
+svc = gazed.Service(None, False, gazed.POINTER)
+threading.Thread(target=svc.run, daemon=True).start()
+ctl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+ctl.bind("")
+ctl.settimeout(2)
+
+
+def ask(cmd):
+    ctl.sendto(cmd.encode(), gazed.ME)
+    return ctl.recv(4096).decode()
+
+
+failures = []
+
+
+def check(label, got, want):
+    ok = got == want
+    print(("ok    " if ok else "FAIL  ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
+    if not ok:
+        failures.append(label)
+
+
+def wait(cond, seconds):
+    end = time.monotonic() + seconds
+    while time.monotonic() < end:
+        if cond():
+            return True
+        time.sleep(0.05)
+    return cond()
+
+
+time.sleep(2.5)
+check("gaze mode off: idle, no ft-gaze", (svc.awake, svc.proc is None), (False, True))
+check("status says why", ask("status").count('"idle": "gaze mode is off"'), 1)
+
+helper_state["reply"] = "ok on worn"
+check("gaze mode on, headset worn: awake within 2 s", wait(lambda: svc.awake and svc.proc is not None, 2.5), True)
+check("samples come in", wait(lambda: svc.counts["samples"] > 20, 2), True)
+
+helper_state["reply"] = "ok on away"
+check("headset off: still awake for IDLE_AFTER", wait(lambda: not svc.awake, 0.5), False)
+check("then idle, ft-gaze stopped", wait(lambda: not svc.awake and svc.proc is None, 3.5), True)
+check("why: nobody wears it", ask("status").count('"idle": "nobody is wearing the headset"'), 1)
+
+helper_state["reply"] = "ok on"
+check("an older helper (no headset word): awake with gaze mode on", wait(lambda: svc.awake, 2.5), True)
+helper_state["reply"] = "ok off worn"
+check("gaze mode off again: idle", wait(lambda: not svc.awake and svc.proc is None, 4.5), True)
+
+check("wake lease", ask("wake 2"), "ok")
+check("wake: awake at once", (svc.awake, wait(lambda: svc.proc is not None, 1)), (True, True))
+check("lease over (2 s + IDLE_AFTER): idle", wait(lambda: not svc.awake, 4.5), True)
+
+time.sleep(0.5)
+logs.clear()
+open(SOURCES_LOG, "w").close()
+count = len(started)
+check("quick check while idle: queued, waking", ask("quickcal"), "ok waking the eye tracker first")
+check("it woke", wait(lambda: svc.awake and len(started) > count, 1.5), True)
+check("once the tracker sends, it ran (and says why it couldn't open)",
+      wait(lambda: any("quickcal, asked for while idle: the panel isn't running" in m for m in logs), 3), True)
+check("nothing left pending", svc.checks.pending, None)
+sources = open(SOURCES_LOG).read().split("\n")
+check("ft-gaze started with every source for the check", sources[0], "argv all")
+in_use = svc.wanted_sources()
+check("then only those in use (SteamVR's tracker: not the action, not own)",
+      (f"line {in_use}" in sources, in_use != "all", "action" in in_use, "own" in in_use), (True, True, False, False))
+check("idle again after", wait(lambda: not svc.awake, 3), True)
+
+print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
+svc.running = False
+time.sleep(0.7)
+os.remove(FAKE)
+os.remove(SOURCES_LOG)
+os.rmdir(tmp)
+os._exit(1 if failures else 0)

gaze/tracker/eyes_pupil.py

@@ -9,7 +9,12 @@ the search runs again with a smaller closing.
 import cv2
 import numpy as np
 
-DARK = 30          # pupil pixels are below this (the face around it is 40-180)
+# One thread: OpenCV's pool of one per core costs more than it saves on a frame this small
+# (a 140-240 px window while it follows the pupil). Its idle workers spun and yielded about
+# 14,000 times a second each, a quarter of a core, beside SteamVR's compositor.
+cv2.setNumThreads(1)
+
+DARK = 30         # pupil pixels are below this (the face around it is 40-180)
 MIN_AREA = 150     # pupil area range in pixels
 MAX_AREA = 20000
 MIN_FILL = 0.75    # blob area / fitted-ellipse area

gaze/tracker/findings.md

@@ -63,6 +63,13 @@ reading only.
     frame when its camera starts the frame after next (no slot is rewritten sooner than three
     frames), ignores late writes to the frame just finished, and saves from a separate
     thread. fit1 may hold a few percent of torn frames.
+  - `--share` (2026-10-03) checks only the slot each camera writes next, from the two before
+    (the orders above, learned again if they change; all four slots for 2 s after a frame turns
+    up elsewhere), sleeps until 2.5 ms before the next frame is due, then looks every 1 ms with
+    0.5 ms of timer slack. Against the 0.3 ms poll of all eight slots, on simulated cameras:
+    207 wakeups a second instead of 1,486, 0.8% of a core instead of 3.6% (more on the real
+    DMA-BUF memory), no torn or skipped frames, and a frame's start seen 1.35 ms late on
+    average instead of 0.76. `--rec` still polls all slots every 0.3 ms, for its times.
   - Eye tracking stops when the headset is off ("HMD off, stopping eye tracking"), so
     recordings are empty then.
 - **Which camera is which eye** (capture fit1, 2026-09-29, closing one eye at a time):

gaze/tracker/ft-eyegrab.c

@@ -46,6 +46,7 @@
 #include <stdlib.h>
 #include <string.h>
 #include <sys/mman.h>
+#include <sys/prctl.h>
 #include <sys/stat.h>
 #include <sys/syscall.h>
 #include <time.h>
@@ -288,20 +289,53 @@ static int eye_buffer(void) {
 
 // Calls done(frame, slot, time) for every complete eye-camera frame until `seconds` pass
 // (forever if negative), a stop signal comes, the tracker process goes away, or keep()
-// (checked about every 0.25 s, when given) says to stop.
+// (checked about every 0.25 s, when given) says to stop. Looks every `poll_us`.
 //
 // A frame lands over several milliseconds, in bursts, and its last bursts can come after
 // the camera has started its next frame. A slot isn't rewritten until at least three frames
 // later (camera 0 cycles 3,0,1,2; camera 1 7,5,4,6,5,7,6,4), so a frame is passed on when
 // its camera starts the frame after next. Changes to the slot just finished are late bursts,
 // not a new frame. A frame's time is when its slot first changed.
+//
+// Each look checks only the slot each camera writes next: the one that followed the last two
+// before (after[][], seeded with the orders above and learned as frames come). Every check
+// reads 256 words spread over a frame in DMA-BUF memory, so all eight slots each time was
+// most of the cost. A camera with nothing in its expected slot for 1.5 frames, or with no
+// order known yet, has all its slots checked until a frame comes. When that finds a frame
+// somewhere else (the order changed, or a frame was missed), all its slots are checked for
+// SCAN_AFTER_MISS, as before, while after[][] learns the new order. A slot's fingerprint is
+// taken again when it stops being one of the two in use, so a check later sees only a new frame.
+// Between frames it sleeps until FRAME_EARLY before the next one is due (the cameras run at
+// 90 fps, within a ms of each other), then looks every `poll_us`.
+#define FRAME_DUE (1.5 / 90)  // s: a camera that hasn't started a frame by then gets all its slots checked
+#define SCAN_AFTER_MISS 2.0    // s of checking all slots after a frame came in an unexpected one
+#define CAMS_IDLE 0.5          // s without a frame from either camera: look 4 times less often
+#define FRAME_EARLY 0.0025     // s before a frame is due to start looking for it
+
+// When to start looking for the next frame: FRAME_EARLY before the first camera's is due. A
+// camera already late (it lost its order, or stopped) means now.
+static double next_due(const double last[2], double t) {
+    double due = 1e300;
+    for (int cam = 0; cam < 2; cam++) {
+        double d = last[cam] + 1.0 / 90 - FRAME_EARLY;
+        if (t - last[cam] > CAMS_IDLE) continue;  // stopped: the other one sets the pace
+        if (d < due) due = d;
+    }
+    return due < 1e300 ? due : t;
+}
+
 static void poll_frames(int b, double seconds, int pid, void (*done)(const uint8_t *, int, double),
-                        int (*keep)(void)) {
+                        int (*keep)(void), unsigned poll_us) {
+    static const int order[2][8] = {{3, 0, 1, 2, 3, 0, 1, 2}, {7, 5, 4, 6, 5, 7, 6, 4}};
+    int after[EYE_SLOTS][EYE_SLOTS];
+    memset(after, -1, sizeof after);
+    for (int c = 0; c < 2; c++)
+        for (int i = 0; i < 8; i++) after[order[c][i]][order[c][(i + 1) % 8]] = order[c][(i + 2) % 8];
     uint64_t sig[EYE_SLOTS];
     double first[EYE_SLOTS];
     int cur[2] = {-1, -1}, prev[2] = {-1, -1};
     for (int k = 0; k < EYE_SLOTS; k++) sig[k] = frame_sig(bufs[b].p + slot_start(k)), first[k] = 0;
-    double start = now(), checked = start, kept = start;
+    double start = now(), checked = start, kept = start, last[2] = {start, start}, scan_until[2] = {0, 0};
     char proc[64];
     snprintf(proc, sizeof proc, "/proc/%d", pid);
     while ((seconds < 0 || now() - start < seconds) && !stop_rec) {
@@ -315,18 +349,36 @@ static void poll_frames(int b, double seconds, int pid, void (*done)(const uint8
             if (!keep()) return;
             kept = t;
         }
-        for (int k = 0; k < EYE_SLOTS; k++) {
-            uint64_t s = frame_sig(bufs[b].p + slot_start(k));
-            if (s == sig[k]) continue;
-            sig[k] = s;
-            int cam = k >= 4;
-            if (k == cur[cam] || k == prev[cam]) continue;  // landing, or a late burst
-            if (prev[cam] >= 0) done(bufs[b].p + slot_start(prev[cam]), prev[cam], first[prev[cam]]);
-            prev[cam] = cur[cam];
-            cur[cam] = k;
-            first[k] = t;
+        for (int cam = 0; cam < 2; cam++) {
+            int next = prev[cam] >= 0 ? after[prev[cam]][cur[cam]] : -1;
+            int all = next < 0 || t - last[cam] > FRAME_DUE || t < scan_until[cam];
+            int from = all ? cam * 4 : next, to = all ? cam * 4 + 4 : next + 1;
+            for (int k = from; k < to; k++) {
+                uint64_t s = frame_sig(bufs[b].p + slot_start(k));
+                if (s == sig[k]) continue;
+                sig[k] = s;
+                if (k == cur[cam] || k == prev[cam]) continue;  // landing, or a late burst
+                if (next >= 0 && k != next) scan_until[cam] = t + SCAN_AFTER_MISS;
+                if (prev[cam] >= 0) {
+                    done(bufs[b].p + slot_start(prev[cam]), prev[cam], first[prev[cam]]);
+                    after[prev[cam]][cur[cam]] = k;
+                    // Out of use from now on: later changes are a new frame.
+                    sig[prev[cam]] = frame_sig(bufs[b].p + slot_start(prev[cam]));
+                }
+                prev[cam] = cur[cam];
+                cur[cam] = k;
+                first[k] = t;
+                last[cam] = t;
+                next = prev[cam] >= 0 ? after[prev[cam]][cur[cam]] : -1;
+            }
         }
-        usleep(300);
+        double wait = poll_us * 1e-6;
+        if (t - last[0] > CAMS_IDLE && t - last[1] > CAMS_IDLE) {
+            wait *= 4;
+        } else if (next_due(last, t) - t > wait) {
+            wait = next_due(last, t) - t;
+        }
+        usleep((useconds_t)(wait * 1e6));
     }
 }
 
@@ -356,7 +408,7 @@ static int rec(double seconds, const char *dir, int pid) {
     pthread_t writer;
     pthread_create(&writer, NULL, ring_writer, NULL);
     double start = now();
-    poll_frames(b, seconds, pid, rec_frame, NULL);
+    poll_frames(b, seconds, pid, rec_frame, NULL, 300);  // 0.3 ms: recordings' times
     pthread_mutex_lock(&ring.mu);
     ring.done = 1;
     pthread_cond_signal(&ring.cv);
@@ -383,6 +435,10 @@ static int rec(double seconds, const char *dir, int pid) {
 #define SHARE_SLOTS 8
 #define SHARE_MAGIC 0x31434546u  // "FEC1"
 #define WANT_FRESH 3.0           // seconds a touch of the --want file lasts
+// How often --share looks for frames, with a timer slack that lets the kernel group the
+// wakeups: a frame reaches ft-eyes 1 to 1.5 ms after its camera starts the next, not 0.3.
+#define SHARE_POLL_US 1000
+#define SHARE_SLACK_NS 500000
 
 typedef struct {
     uint32_t magic, version, width, height, slots, entry_size;
@@ -462,6 +518,7 @@ static int share_loop(const char *path) {
                         .slots = SHARE_SLOTS, .entry_size = (uint32_t)esize};
     signal(SIGINT, on_stop);
     signal(SIGTERM, on_stop);
+    if (prctl(PR_SET_TIMERSLACK, SHARE_SLACK_NS, 0, 0, 0) != 0) perror("PR_SET_TIMERSLACK");
     fprintf(stderr, "ft-eyegrab: sharing frames in %s%s%s\n", path, want_path ? " while wanted by " : "",
             want_path ? want_path : "");
     int pid = -1, idle = -1, missing = 0;
@@ -486,7 +543,7 @@ static int share_loop(const char *path) {
         if (idle != 0 || missing) fprintf(stderr, "ft-eyegrab: copying frames from eyetracking %d\n", pid);
         idle = 0, missing = 0;
         h->tracker_pid = pid;
-        poll_frames(eye_buffer(), -1, pid, share_frame, wanted);
+        poll_frames(eye_buffer(), -1, pid, share_frame, wanted, SHARE_POLL_US);
         struct stat st;
         char proc[64];
         snprintf(proc, sizeof proc, "/proc/%d", pid);

gaze/tracker/ft-eyes

@@ -50,6 +50,7 @@ import json
 import math
 import mmap
 import os
+import select
 import socket
 import struct
 import sys
@@ -58,7 +59,12 @@ import time
 from collections import deque
 from pathlib import Path
 
-import numpy as np
+# One thread for numpy's BLAS and OpenMP, set before numpy loads: it would start one per core
+# (8 here) for small arrays that never need them. eyes_pupil keeps OpenCV to one as well.
+for _var in ("OPENBLAS_NUM_THREADS", "OMP_NUM_THREADS"):
+    os.environ.setdefault(_var, "1")
+
+import numpy as np  # noqa: E402
 
 sys.path.insert(0, str(Path(__file__).resolve().parent))
 import eyes_model  # noqa: E402
@@ -82,6 +88,15 @@ GAP = 3.0             # s without frames: the headset was off, and may sit diffe
 CLICK_BEFORE = 0.3    # a click's frames: the 300 ms before it (like the probe's fixation)
 CALIB_MIN = 15        # frames an eye needs in a calibration dot's window
 CALIB_SPREAD = 4.0    # px: more than this and the eye moved during the dot
+# Waiting for frames. They come only as a counter in shared memory, so there's nothing to block
+# on: sleep until a camera's next frame is due, then look every POLL. ft-eyegrab passes each one
+# on within a ms or two of when its camera starts the one after, so they arrive 11.1 ms apart,
+# give or take that.
+PERIOD = 1 / 90       # s between a camera's frames
+EARLY = 0.002         # start looking this long before a frame is due
+POLL = 0.001          # then this often until it comes
+STALLED = 0.1         # s without a frame: that camera stopped (headset off), and isn't waited for
+IDLE_POLL = 0.02      # how often to look while both are stopped
 
 
 class Cams:
@@ -352,7 +367,7 @@ def wait_for_cams(sock, tracker, want):
             return Cams()
         except (OSError, ValueError, RuntimeError):
             serve(sock, tracker)
-            time.sleep(0.2)
+            select.select([sock], [], [], 0.2)
 
 
 def serve(sock, tracker):
@@ -372,7 +387,24 @@ def serve(sock, tracker):
                 pass
 
 
+def below_steamvr():
+    """Nice 10 and SCHED_BATCH, for this thread and those it starts. ft-eyes runs in the dev
+    container's podman scope, where the gaze service's unit doesn't reach it, so it ran at
+    nice 0 on the cores vrcompositor and vrserver use. Batch also lets a waking ft-eyes wait
+    for the running task's turn instead of taking the core: a frame a few ms late costs the
+    gaze little, a late compositor frame costs a dropped frame in the headset."""
+    try:
+        os.setpriority(os.PRIO_PROCESS, 0, max(os.getpriority(os.PRIO_PROCESS, 0), 10))
+    except OSError as e:
+        print(f"ft-eyes: nice: {e}", file=sys.stderr, flush=True)
+    try:
+        os.sched_setscheduler(0, os.SCHED_BATCH, os.sched_param(0))
+    except (OSError, AttributeError) as e:
+        print(f"ft-eyes: SCHED_BATCH: {e}", file=sys.stderr, flush=True)
+
+
 def main():
+    below_steamvr()
     verbose = "-v" in sys.argv
     if "--watch-stdin" in sys.argv:
         # Run by ft-gazed through distrobox, which doesn't pass a stop on: quit when our
@@ -398,6 +430,7 @@ def main():
     print("ft-eyes: frames found, tracking", file=sys.stderr, flush=True)
     seen = [cams.count(0), cams.count(1)]
     report = time.monotonic()
+    arrived = [0.0, 0.0]  # when each camera's newest frame was seen (monotonic)
     while True:
         serve(sock, tracker)
         want()
@@ -407,6 +440,7 @@ def main():
             if n == seen[c]:
                 continue
             seen[c] = n
+            arrived[c] = time.monotonic()
             got = cams.frame(c, n - 1)  # only the newest: never fall behind
             if got:
                 eyes[c].feed(*got)
@@ -425,8 +459,6 @@ def main():
                     shifts += [float(v) for v in e.shift.value]
                     pupils += [e.gaze[3], e.gaze[4]] if fresh else [math.nan, math.nan]
                 out.write(latest, (yaw, pitch), flags, per, shifts, pupils)
-        else:
-            time.sleep(0.001)
         now = time.monotonic()
         if now - report >= 5:
             if verbose:
@@ -444,6 +476,12 @@ def main():
                 print("ft-eyes: frames went away; waiting", file=sys.stderr, flush=True)
                 cams = wait_for_cams(sock, tracker, want)
                 seen = [cams.count(0), cams.count(1)]
+        # Until the next frame is due (a command on the socket wakes us sooner).
+        wait = IDLE_POLL
+        for c in (0, 1):
+            if now - arrived[c] < STALLED:
+                wait = min(wait, max(arrived[c] + PERIOD - EARLY - now, POLL))
+        select.select([sock], [], [], wait)
 
 
 if __name__ == "__main__":

gaze/tracker/install.sh

@@ -2,12 +2,12 @@
 # Install (or remove) the frame grabber our own eye tracker needs: ft-eyegrab, as the system
 # service frametop-eyegrab.service. It copies the eye-camera frames, read-only, out of
 # SteamVR's eyetracking process into /dev/shm/frametop-eyes-cams for ft-eyes, and only while
-# ft-eyes wants them. The gaze service (gaze/ft-gazed) runs ft-eyes itself, when Eye tracker
-# is Own tracker or the gaze probe uses it.
-# Needs host sudo, for the binary (/etc/frametop/ft-eyegrab, root's) and the unit. On the
-# Frame, sudo asks for the password in the terminal, or runs SUDO_ASKPASS when that's set.
-# From a PC (or with no terminal), the password comes from steamos_root_pwd in the repo's .env
-# and is sent to sudo -S on stdin, never on a command line.
+# ft-eyes wants them. The gaze service (gaze/ft-gazed) runs ft-eyes itself, when ours is the
+# tracker in use (GAZE_TRACKER=auto, the default, picks it once this is installed) or the gaze
+# probe uses it. install.sh offers this after gaze mode.
+# Needs host sudo, for the binary (/etc/frametop/ft-eyegrab, root's) and the unit: it asks for
+# the password in the terminal, on the Frame or from a PC, or runs SUDO_ASKPASS when that's set
+# (frame_sudo in scripts/_env.sh, which also takes it from the repo's .env).
 # Usage: gaze/tracker/install.sh [install|uninstall|status|log [lines]]
 set -euo pipefail
 
@@ -16,21 +16,7 @@ root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
 src=$FRAME_REPO/gaze/tracker
 unit=frametop-eyegrab.service
 
-sudo_run() {
-  if [ "$FRAME_LOCAL" = 1 ] && [ -n "${SUDO_ASKPASS:-}" ]; then
-    sudo -A bash -c "$1"  # SUDO_ASKPASS supplies the password
-    return
-  fi
-  if [ "$FRAME_LOCAL" = 1 ] && [ -t 0 ]; then
-    sudo bash -c "$1"  # asks for the password here
-    return
-  fi
-  local pw
-  pw=$(sed -n 's/^steamos_root_pwd=//p' "$root/.env" 2>/dev/null)
-  pw=${pw#[\"\']}; pw=${pw%[\"\']}  # .env values may be quoted
-  [ -n "$pw" ] || { echo "no terminal for sudo, and steamos_root_pwd is missing from $root/.env" >&2; exit 1; }
-  printf '%s\n' "$pw" | on_frame "sudo -S -p '' bash -c $(printf %q "$1")"
-}
+sudo_run() { frame_sudo "$1"; }
 
 case ${1:-install} in
   install)

gaze/tracker/lab/eyes_track.py

@@ -1,41 +0,0 @@
-"""Per-frame pupil ellipses through a recording, cached in the capture (numbers only).
-
-ellipses(cap) -> {camera: array of rows (t, x, y, a, b, major, fill, glint mid x, y)}, every
-EVERY-th frame of each camera; the glint midpoint is NaN when the pair isn't seen.
-"""
-import numpy as np
-
-import eyes_lab  # noqa: F401  (puts gaze/tracker on the path)
-import eyes_pupil
-
-EVERY = 2
-VERSION = 2
-
-
-def load_index(cap):
-    return np.array([[float(v) for v in l.split()]
-                     for l in (cap / "index.txt").read_text().splitlines() if len(l.split()) == 4])
-
-
-def ellipses(cap):
-    cache = cap / f"ellipses-v{VERSION}.npz"
-    if cache.exists():
-        z = np.load(cache)
-        return {0: z["cam0"], 1: z["cam1"]}
-    idx = load_index(cap)
-    frames = np.memmap(cap / "frames.raw", dtype=np.uint8, mode="r").reshape(-1, 400, 512)
-    idx = idx[:len(frames)]
-    out = {}
-    for c in (0, 1):
-        rows, prev = [], None
-        for i in np.where(idx[:, 2] == c)[0][::EVERY]:
-            p = eyes_pupil.find_pupil(frames[i], prev)
-            prev = p
-            if p is None:
-                continue
-            pair = eyes_pupil.glint_pair(p)
-            mx, my = ((pair[0][0] + pair[1][0]) / 2, (pair[0][1] + pair[1][1]) / 2) if pair else (np.nan, np.nan)
-            rows.append((idx[i, 3], p["x"], p["y"], p["a"], p["b"], p["major"], p["fill"], mx, my))
-        out[c] = np.array(rows).reshape(-1, 9)
-    np.savez(cache, cam0=out[0], cam1=out[1])
-    return out

get.sh

@@ -0,0 +1,124 @@
+#!/usr/bin/env bash
+# Frametop's one-line installer. In a terminal on the Steam Frame (Konsole in the desktop, or
+# over SSH):
+#
+#   curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash
+#
+# It asks which version to install, clones the repo into ~/frametop (or updates the clone
+# that's there), and runs its install.sh. Run it again to update, or to switch versions.
+# Options (piped, they go after "bash -s --"):
+#   --stable        the main branch: tested releases (the default for a new install)
+#   --experimental  the experimental branch: the newest features, less tested
+#   --branch NAME   another branch, such as a fix to test before it's released
+#   --dir DIR       where the repo goes (default ~/frametop)
+#   --clone-only    get or update the repo, but don't run install.sh
+#   --yes, --no-bluetooth   passed to install.sh (--yes also answers this script's question:
+#                   the version already there, or stable)
+set -euo pipefail
+
+usage() {
+  cat <<'EOF'
+usage: get.sh [--stable | --experimental | --branch NAME] [--dir DIR] [--clone-only] [--yes] [--no-bluetooth]
+piped: curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash -s -- [options]
+EOF
+}
+
+# Everything happens in main, called on the last line, so a download cut short runs nothing.
+main() {
+  local repo=https://github.com/DeeJanuz/frametop.git dir=$HOME/frametop branch= clone_only=0
+  local yes=0 tty=0 current= def answer
+  local pass=()
+  while [ $# -gt 0 ]; do
+    case $1 in
+      --stable) branch=main ;;
+      --experimental) branch=experimental ;;
+      --branch) branch=${2:?--branch needs a branch name}; shift ;;
+      --dir) dir=${2:?--dir needs a folder}; shift ;;
+      --clone-only) clone_only=1 ;;
+      --yes) yes=1; pass+=("$1") ;;
+      --no-bluetooth) pass+=("$1") ;;
+      -h|--help) usage; return 0 ;;
+      *) echo "unknown option: $1" >&2; usage >&2; return 2 ;;
+    esac
+    shift
+  done
+
+  if ! { grep -qx 'ID=steamos' /etc/os-release && grep -qE '^VARIANT_ID="?vr"?$' /etc/os-release; } 2>/dev/null; then
+    echo "Frametop installs on a Steam Frame (SteamOS, VR variant). Run this in a terminal on the headset." >&2
+    return 1
+  fi
+  # Piped into bash, stdin is this script: the questions (here and install.sh's) read the terminal.
+  { : </dev/tty; } 2>/dev/null && tty=1
+  if [ "$tty" = 0 ] && [ "$yes" = 0 ]; then
+    echo "This asks questions, and there's no terminal to ask in: run it in one, or add --yes." >&2
+    return 1
+  fi
+
+  if [ -e "$dir/.git" ]; then
+    git -C "$dir" remote get-url origin 2>/dev/null | grep -qi 'frametop' ||
+      { echo "$dir is a git repo, but not Frametop's. Pick another folder with --dir." >&2; return 1; }
+    current=$(git -C "$dir" branch --show-current)
+  elif [ -e "$dir" ]; then
+    echo "$dir is there and isn't Frametop's repo. Move it, or pick another folder with --dir." >&2
+    return 1
+  fi
+
+  if [ -z "$branch" ]; then
+    def=main
+    [ "$current" = experimental ] && def=experimental
+    if [ "$yes" = 1 ]; then
+      branch=$def
+    else
+      echo "Which version of Frametop?"
+      echo "  1) stable: the main branch, tested releases"
+      echo "  2) experimental: the newest features, less tested"
+      [ -n "$current" ] && echo "(installed now: $current)"
+      read -r -p "Choose 1 or 2 [$([ "$def" = main ] && echo 1 || echo 2)]: " answer </dev/tty || answer=
+      case ${answer:-$def} in
+        1|main|s*) branch=main ;;
+        2|experimental|e*) branch=experimental ;;
+        *) echo "not 1 or 2: $answer" >&2; return 2 ;;
+      esac
+    fi
+  fi
+
+  if ! git ls-remote --exit-code --heads "$repo" "$branch" >/dev/null; then
+    echo "Frametop has no branch called $branch (or GitHub can't be reached)." >&2
+    return 1
+  fi
+
+  if [ ! -e "$dir" ]; then
+    echo "Cloning Frametop ($branch) into $dir"
+    git clone --branch "$branch" "$repo" "$dir"
+  else
+    if ! git -C "$dir" diff --quiet || ! git -C "$dir" diff --cached --quiet; then
+      echo "$dir has changes of its own. Commit or stash them first (git -C $dir status)." >&2
+      return 1
+    fi
+    echo "Updating $dir to the latest $branch"
+    git -C "$dir" fetch --quiet origin
+    if [ "$current" != "$branch" ]; then
+      if git -C "$dir" show-ref --verify --quiet "refs/heads/$branch"; then
+        git -C "$dir" switch --quiet "$branch"
+      else
+        git -C "$dir" switch --quiet --track -c "$branch" "origin/$branch"
+      fi
+    fi
+    git -C "$dir" merge --ff-only --quiet "origin/$branch" ||
+      { echo "$dir has commits of its own on $branch, so it can't just move to the latest. Update it by hand." >&2; return 1; }
+  fi
+  echo "Frametop $branch: $(git -C "$dir" log -1 --format='%h %s')"
+
+  if [ "$clone_only" = 1 ]; then
+    echo "Install with: cd $dir && ./install.sh"
+    return 0
+  fi
+  cd "$dir"
+  if [ "$tty" = 1 ]; then
+    ./install.sh "${pass[@]}" </dev/tty
+  else
+    ./install.sh "${pass[@]}" </dev/null
+  fi
+}
+
+main "$@"

hand-recorder.md

@@ -0,0 +1,68 @@
+---
+title: Install the Frametop Hand Recorder
+---
+
+# Install the Frametop Hand Recorder
+
+The Hand Recorder records your hands with the Steam Frame's tracking cameras for Frametop's open hand dataset ([DeeJanuz/frametop-hands](https://huggingface.co/datasets/DeeJanuz/frametop-hands) on Hugging Face). These are the Konsole commands to install it.
+
+> **Fixed in Frametop 0.2.1 (October 5, 2026):** the recorder now works on headsets without the Arcturus color passthrough module too. If you installed it earlier and the camera check stopped with "Not all of the headset's tracking cameras are running (ft-camd publishes only 2 of 4 mono cameras ...)", run the commands under [Update](#update).
+
+Join the [Frametop Discord](https://discord.gg/W3X9f7z3Bc) for questions and help with recording.
+
+For now the recorder runs inside Frametop's desktop, so these steps install Frametop first. A standalone recorder that runs from the SteamVR dashboard without Frametop is planned.
+
+## Before you start
+
+- You must be 18 or older, and for now you can't take part if you live in Illinois, Texas or Washington (USA). The [consent text](https://github.com/DeeJanuz/frametop/blob/main/hands/rec/CONSENT.md) explains what's recorded and what you agree to. The recorder shows it again before your first session.
+- You need a Steam Frame on the stable SteamOS release (not the beta), an internet connection, and a keyboard (Bluetooth, or the on-screen one).
+- You need a `sudo` password. If you've never set one, run `passwd` in Konsole first.
+- Recordings are several gigabytes per round, and uploading one needs about the same again free while it runs. `df -h ~` shows your free space.
+
+## Open Konsole
+
+1. In the launcher, choose Launch a program → Desktop.
+2. In the application menu, open System → Konsole.
+
+## 1. Install Frametop
+
+```
+curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash -s -- --stable
+```
+
+This clones Frametop into `~/frametop` and runs its installer. The first run downloads 1–2 GB. The installer asks a few questions (gaze mode, the eye tracker, the Bluetooth fixes); the defaults are fine. At the end SteamVR restarts, which closes Konsole. If Frametop is already installed, this updates it.
+
+## 2. Install the Hand Recorder
+
+After SteamVR restarts, choose Launch a program → Desktop again, open Konsole, and run:
+
+```
+~/frametop/hands/rec/install.sh
+```
+
+This builds the camera broker, the hand tracker and the headset panel (the first build takes a few minutes), asks for your `sudo` password once to let the camera broker read the cameras, and adds Frametop Hand Recorder to the application menu.
+
+## 3. Record
+
+Open Frametop Hand Recorder from the desktop's application menu. It walks you through the consent text, a short checklist, the recording, a review of what you recorded, and the upload to Hugging Face. Nothing leaves the headset until you press Upload.
+
+## Update
+
+```
+curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash -s -- --stable
+~/frametop/hands/rec/install.sh
+```
+
+Run the second command after SteamVR has restarted, as in the install.
+
+## Uninstall
+
+```
+~/frametop/hands/rec/install.sh uninstall
+```
+
+This removes the menu entry. Your recordings stay in `~/.local/share/frametop/hands/contrib`; delete that folder to remove them. To remove Frametop as well, follow [Uninstall](https://github.com/DeeJanuz/frametop#uninstall) in the README.
+
+## Help
+
+Ask in the [Frametop Discord](https://discord.gg/W3X9f7z3Bc), the [Frametop issues](https://github.com/DeeJanuz/frametop/issues), or the dataset's [discussion page](https://huggingface.co/datasets/DeeJanuz/frametop-hands/discussions). All three are public.

hands/.gitignore

@@ -1,3 +1,5 @@
 # Model sources that tools/convert_models.py downloads; the converted ncnn models are kept
 models/onnx/
 models/*.task
+# frame-job settings (where replays run, and the lab's capture folder): personal, not shipped
+.frame-job

hands/Makefile

@@ -1,6 +1,7 @@
 # Hand tracking, built into build/ (hands/build.sh runs this in the dev container):
 #   make          ft-camd (camd/: runs on the host, so linked statically) and ft-hands (track/)
 #   make tools    ft-handreplay and ft-ringplay, for recordings
+#   make check    builds and runs the C++ unit tests (tests/sides_test.cpp)
 # The first build fetches ncnn (NCNN_TAG) and builds it into build/ncnn, which takes a few
 # minutes. NCNN=DIR uses an ncnn install already built instead.
 NCNN_TAG  = 20260526
@@ -11,7 +12,7 @@ CXXFLAGS += -std=c++17 -fopenmp -I$(NCNN)/include/ncnn
 LDLIBS    = $(NCNN)/lib/libncnn.a -ljsoncpp -fopenmp -lpthread
 
 CAMD  = camd/camd.c camd/tp.c camd/xrcams.c
-TRACK = track/calib.cpp track/nets.cpp track/tracker.cpp track/io.cpp track/record.cpp track/pinch.cpp
+TRACK = track/calib.cpp track/nets.cpp track/tracker.cpp track/io.cpp track/record.cpp track/pinch.cpp track/sides.cpp
 HDR   = $(wildcard track/*.h) camd/fhring.h include/fh_hands.h include/fh_gestures.h
 
 all: build/ft-camd build/ft-hands
@@ -29,6 +30,13 @@ build/ft-handreplay: track/replay.cpp $(TRACK) $(HDR) $(NCNN)/lib/libncnn.a
 	@mkdir -p build
 	$(CXX) $(CXXFLAGS) -o $@ track/replay.cpp $(TRACK) $(LDLIBS)
 
+build/sides-test: tests/sides_test.cpp $(TRACK) $(HDR) $(NCNN)/lib/libncnn.a
+	@mkdir -p build
+	$(CXX) $(CXXFLAGS) -o $@ tests/sides_test.cpp $(TRACK) $(LDLIBS)
+
+check: build/sides-test
+	build/sides-test
+
 build/ft-ringplay: track/ringplay.cpp track/record.h camd/fhring.h
 	@mkdir -p build
 	$(CXX) $(CXXFLAGS) -o $@ track/ringplay.cpp
@@ -44,6 +52,6 @@ build/ncnn/install/lib/libncnn.a:
 	cmake --build build/ncnn/build --target install > build/ncnn/build.log
 
 clean:
-	rm -f build/ft-camd build/ft-hands build/ft-handreplay build/ft-ringplay
+	rm -f build/ft-camd build/ft-hands build/ft-handreplay build/ft-ringplay build/sides-test
 
-.PHONY: all tools clean
+.PHONY: all tools check clean

hands/README.md

@@ -1,28 +1,47 @@
-# Hands (experimental)
+# Hands (experimental, deferred)
 
-Hand tracking from the headset's own cameras. It serves two things in Frametop:
+Hand tracking from the headset's own cameras. It's deferred: it costs a lot of the headset's CPU and needs more work, so `install.sh` doesn't offer it and the README doesn't list it. It still builds and runs, installed by hand (below), for working on it.
+
+It serves two things in Frametop:
 
 - **Hand cutouts:** where your hand is between an eye and a screen, that eye sees the room through the screen (ft-screens, `screens/handcut.cpp`), so your hands show over the screens the way they do on a Vision Pro.
-- **Pinches:** look at something and pinch to click it, pinch and move to drag, with the eye tracker doing the looking (`gaze/`). The tracker publishes the pinches. The pointer helper doesn't read them yet.
+- **Pinches and grips:** with `POINTER_HANDS=1`, the pointer helper takes them as clicks and drags. Look at something and pinch to click it, with the eye tracker doing the looking (`gaze/`), or close your hand to press and drag what the pointer is on. See "Pinches and grips in the pointer" below.
 
-Two programs, each a user service that starts and stops with SteamVR:
+Two programs, each a user service that stops when SteamVR does:
 
 - `ft-camd` (`camd/`, C) borrows XRService's camera buffers and publishes the four IR tracking cameras' frames to a shared-memory ring. It runs on the host.
 - `ft-hands` (`track/`, C++) finds hands in those frames with MediaPipe's palm and landmark models on ncnn, triangulates them, and publishes them. It runs in the dev container.
 
+They don't start with SteamVR. `hands/run.sh install` builds them, gives ft-camd its capabilities, installs both services disabled, and links `hands/ft-handsctl` into `~/.local/bin`. Then `ft-handsctl on` starts hand tracking and `ft-handsctl off` stops it. `install.sh` doesn't install it.
+
+For the cutouts alone, `hands/ft-cutouts on` starts the same two programs with ft-hands' `--no-gestures`: your hands show through the screens, and no pinch or grip is detected, so nothing clicks or drags. It runs this checkout's build as transient user units, so it needs `hands/build.sh` and ft-camd's capabilities (`hands/run.sh caps`) but not `hands/run.sh install`. It and `ft-handsctl on` stop each other's services, and it stops with SteamVR too.
+
 ```
-hands/run.sh install            # build, give ft-camd its capabilities (sudo, once per build), enable
-hands/run.sh status             # the services, and ft-hands' last status lines
-hands/run.sh log [lines]
+ft-handsctl on | off            # on the Frame: start or stop hand tracking (SteamVR must be running)
+ft-handsctl status              # the services, and ft-hands' last status lines
+ft-handsctl log [lines]
+ft-handsctl cutouts on|off|state  # ft-screens' hand cutouts, without stopping tracking
+ft-handsctl gestures            # pinches and grips, live (tools/watch_gestures.py --distance)
+
+hands/ft-cutouts on | off | status   # the cutouts only: no pinches or grips
+
+hands/run.sh install            # build, give ft-camd its capabilities (sudo, once per build), install disabled
+hands/run.sh start|stop         # start or stop the services
 hands/run.sh restart            # after changing a setting
+hands/run.sh status
+hands/run.sh log [lines]
 hands/run.sh caps               # after rebuilding ft-camd (a rebuild clears its capabilities)
 hands/run.sh uninstall
 ```
 
-Settings in `~/.config/frametop.conf` (`FT_<name>` in the environment overrides them):
+Settings in `~/.config/frametop.conf` (`FT_<name>` in the environment overrides them), read when ft-camd and ft-hands start:
 
-- `HANDS_SWAP_SIDES=1`: the two side cameras' names are swapped (see ft-camd below). Check with `tools/check_sides.py --ring`.
+- `HANDS_SWAP_SIDES=auto` (the default): ft-hands tells from the hands which side camera is which, and corrects ft-camd's names when they're backwards (see "Which camera is which" below). `1` forces them exchanged and `0` forces ft-camd's names; ft-hands still checks, and if the hands disagree it logs a warning and publishes the hands' answer as the truth (`sides.json`), so recordings are labelled right. The example config said `0` until 2026-10-05; `scripts/conf-migrate.sh` (run by `install.sh` and `hands/rec/install.sh`) turns that untouched line into `auto`.
 - `HANDS_CPUS=5,6,7`: the CPUs the model threads run on (below).
+- `HANDS_CAMERAS` (`auto`), `HANDS_BRIGHT` (`all`), `HANDS_BRIGHT_ON` (40), `HANDS_BRIGHT_OFF` (25): which cameras ft-hands tracks with, as `--cams`, `--bright`, `--bright-on` and `--bright-off` (see ft-hands). `HANDS_CAMERAS=mono` also keeps ft-camd off the colour cameras.
+- `HANDS_COLOR_LEFT` (`color_video0`), `HANDS_COLOR_CROP` (`subtract`): how the colour module's calibration maps onto its images, as `--color-left` and `--color-crop`.
+
+The pointer helper's `POINTER_HANDS` and `POINTER_PINCH_*`/`POINTER_GRIP_*` settings are in "Pinches and grips in the pointer" below.
 
 Files, all in `/run/user/UID/frametop-hands/` (private to the user; not `/run/user/UID/frametop/`, which the desktop session deletes whenever it starts):
 
@@ -30,9 +49,9 @@ Files, all in `/run/user/UID/frametop-hands/` (private to the user; not `/run/us
 | --- | --- | --- | --- |
 | `cam-ring` | ft-camd | `camd/fhring.h` | ft-hands, `tools/ring.py` |
 | `hands` | ft-hands | `include/fh_hands.h` | ft-screens (`screens/handcut.cpp`) |
-| `gestures` | ft-hands | `include/fh_gestures.h` | `tools/watch_gestures.py`; the pointer helper, later |
+| `gestures` | ft-hands | `include/fh_gestures.h` | the pointer helper (`pointer/helper/ft-pointer.cpp`), `tools/watch_gestures.py` |
 
-The source keeps the `fh_` names and magic strings of frame-hands, where this was developed (`~/Desktop/Projects/frame-hands` on the developer's Frame, which keeps the recordings, probes and Python prototype). So its recordings and tools still work.
+The source keeps the `fh_` names and magic strings of frame-hands, the project it started as, so recordings made with it still work.
 
 ## ft-camd
 
@@ -45,7 +64,7 @@ Polling buffers for changes can catch a frame while the camera is still writing
 - The two upper cameras share one run of buffers. For them, only allocation order can tell the cameras apart.
 - It also re-maps an index on the fly when its buffer holds no new frame.
 
-**Privileges.** Setting up needs three things. `pidfd_getfd` on XRService needs `CAP_SYS_PTRACE`, because the Frame has `ptrace_scope=1`. The system-wide tracepoint needs `CAP_PERFMON`, because `perf_event_paranoid` is 2. Its format files are root-only, which needs `CAP_DAC_READ_SEARCH`. `hands/run.sh install` gives the binary those capabilities with `sudo setcap`. ft-camd drops them all once it has set up, before it reads a frame, and then runs as you. XRService runs as you too. It also runs under `sudo`, for trying it by hand, and then drops to the user who ran sudo. It reads nothing from the ring's readers.
+**Privileges.** Setting up needs three things. `pidfd_getfd` on XRService needs `CAP_SYS_PTRACE`, because the Frame has `ptrace_scope=1`. The system-wide tracepoint needs `CAP_PERFMON`, because `perf_event_paranoid` is 2. Its format files are root-only, which needs `CAP_DAC_READ_SEARCH`. `hands/run.sh install` gives the binary those capabilities with `sudo setcap`. File capabilities need a filesystem mounted without `nosuid`. The Frame's `/home` (ext4) has no `nosuid`. ft-camd drops them all once it has set up, before it reads a frame, and then runs as you. XRService runs as you too. It also runs under `sudo`, for trying it by hand, and then drops to the user who ran sudo. It reads nothing from the ring's readers.
 
 The ring is mode 0600, in a folder only you can write. Frame handling:
 
@@ -55,17 +74,26 @@ The ring is mode 0600, in a folder only you can write. Frame handling:
 
 Options:
 
+- `--dark R`: a frame dimmer than R times the camera's recent brightest counts as near-black. Default 0.4.
 - `--with-dark`: also publish the near-black frames, as extra ring cameras flagged `FH_CAM_DARK`. They show only light sources, so they're no use for hands.
-- `--with-color` (the service uses it): also publish the two Arcturus colour cameras, flagged `FH_CAM_COLOR`. Each is the luma of the 10-bit frame's valid 1972x2464 (the top 8 bits), at half size (`--color-scale 2`: 986x1232). They run at `--color-idle` (2 fps), enough for ft-hands to tell how bright it is, until a reader asks for more in `/run/user/UID/frametop-hands/color-fps` (ft-hands writes 30 while it tracks or records with them), up to `--color-fps` (30; the cameras run at 60). `HANDS_CAMERAS=mono` leaves them out. Frames that carry the module's warped half-size copy are dropped. Their `capture_ns` is on the colour module's clock (2.2 s off the mono cameras' on 2026-09-29), so line them up with the mono cameras by `dqbuf_ns`. Each frame costs about 0.65 ms of cache sync and 1.1 ms of decoding, so both cameras at 30 fps take about 11% of a core.
+- `--with-color`: also publish the two Arcturus colour cameras, flagged `FH_CAM_COLOR`. The service leaves it off and runs the mono cameras only (see "Known issues"). To try the colour cameras, add it to `ExecStart` in `hands/frametop-camd.service` and run `hands/run.sh install` again; ft-hands then picks the cameras by the light. Each is the luma of the 10-bit frame's valid 1972x2464 (the top 8 bits), at half size (`--color-scale 2`: 986x1232). They run at `--color-idle` (2 fps), enough for ft-hands to tell how bright it is, until a reader asks for more in `/run/user/UID/frametop-hands/color-fps` (ft-hands writes 30 while it tracks or records with them), up to `--color-fps` (30; the cameras run at 60). `HANDS_CAMERAS=mono` leaves them out. Frames that carry the module's warped half-size copy are dropped. Their `capture_ns` is on the colour module's clock (2.2 s off the mono cameras' on 2026-09-29), so line them up with the mono cameras by `dqbuf_ns`. Each frame costs about 0.65 ms of cache sync and 1.1 ms of decoding, so both cameras at 30 fps take about 11% of a core.
 - Each mono camera's latest near-black frame's mean goes in the ring (`dark_mean`): a short fixed exposure, so it follows the room's IR light, sunlight above all.
 - The ring holds 8 cameras: 4 mono, plus 4 dark twins or 2 colour cameras.
-- Colour isn't reliable yet. In the lit-room test of 2026-09-30, the colour cameras kept losing their buffer mapping while the headset was worn: 30 frames in a row looked unchanged, the camera relearned, and after 5 relearns ft-camd exited. Each relearn probed all 32 colour buffers, a whole-buffer cache sync each, which also made the mono cameras miss frames. Runs with the headset idle had none of this. So the passthrough compositor may be writing into the colour buffers while Room View shows. Since then a colour camera never takes the mono ones down: it probes at most 4 buffers a frame, and one that goes stale twice in a row is paused (10 s, doubling up to 160 s) and learned again, without ft-camd exiting. Whether a frame is new is judged on the luma rows only: the chroma after them hardly changes in a lit room. `FT_CAMD_DEBUG=1` prints, at each colour stale frame, how many sampled words changed in every candidate buffer.
+- Colour isn't reliable yet. In the lit-room test of 2026-09-30, the colour cameras kept losing their buffer mapping while the headset was worn: 30 frames in a row looked unchanged, the camera relearned, and after 5 relearns ft-camd exited. Each relearn probed all 32 colour buffers, a whole-buffer cache sync each, which also made the mono cameras miss frames. Runs with the headset idle had none of this. So the passthrough compositor may be writing into the colour buffers while Room View shows. Since then a colour camera never takes the mono ones down: it probes at most 4 buffers a frame, and one that goes stale twice in a row is paused (10 s, doubling up to 160 s) and learned again, without ft-camd exiting. Whether a frame is new is judged on the luma rows only: the chroma after them hardly changes in a lit room.
+- `FT_CAMD_DEBUG=1` in the environment: at each stale colour frame, ft-camd logs to stderr the camera, the frame's time, V4L2 index and sequence number, and, for every candidate buffer, how many of its sampled words changed, in all and in the last eighth of the samples.
 - `--sensor S`: only the mono cameras whose sensor name contains S.
 - `--status S`: a status line every S seconds (0: never).
 
 It exits when XRService exits, or when a camera's buffers keep going stale, which means XRService has reallocated them. The service starts it again, and it attaches to the new buffers.
 
-**Which camera is which:** video9 is `slam_left`, video13 is `slam_right`, video6 is `upper_left` and video7 is `upper_right`. This was checked by rendering the same view from each camera with the factory calibration. But ft-camd tells the side cameras' buffers apart only by XRService's allocation order, and after some XRService restarts it gets them backwards. Then every hand is seen by one camera only, at the wrong depth, and the hand holes land beside the hands. With the headset on, looking at a room with some texture, `tools/check_sides.py --ring` says whether the names are right (exit 0), swapped (exit 3), or it can't tell (exit 2). When they're swapped, set `HANDS_SWAP_SIDES=1`. The colour cameras are video3 (`arcimx616 0-0010`) and video0 (`0-001a`); which of them is `passthrough_left` in the module's calibration is for `tools/check_color.py` to settle, on a recording with texture in view.
+**Which camera is which:** video9 is `slam_left`, video13 is `slam_right`, video6 is `upper_left` and video7 is `upper_right`. This was checked by rendering the same view from each camera with the factory calibration. But ft-camd tells the side cameras' buffers apart only by XRService's allocation order, and after some XRService restarts it gets them backwards. Then every hand is seen by one camera only, at the wrong depth, and the hand holes land beside the hands. ft-hands now catches this by itself (`track/sides.h`, `HANDS_SWAP_SIDES=auto`):
+- Whenever a hand's landmarks are found in two cameras at once (one of them a side camera), it intersects the rays through the 21 landmarks twice: once with the calibrations as named, once with the two side cameras exchanged. The same hand seen the right way meets within a few mm, in front of both cameras and as far away as its size says. The wrong way misses by centimetres or meets behind a camera.
+- With the names wrong, the tracker never gets such pairs on its own: it hands the hand over to where the wrong calibration puts it and finds nothing there. So 5 times a second while undecided, the check places a tracked hand in 3D under the other naming and runs the landmark model where that puts it in the other side camera.
+- It decides after 10 votes one way and none the other, or 20 with at most a fifth the other way, over at least 1 s. That takes about 1-2 s of hands in view. If the names are backwards, it exchanges them; the tracked views move with their images. Then it checks once more, more strictly.
+- The log says what it found (`side cameras: SWAPPED, now exchanged after 3.2 s (votes ...)`). So does `/run/user/UID/frametop-hands/sides.json`, which the hand recorder reads. Recordings get a `DIR/sides.json` (hands/rec/sides.py has the rules).
+- `--record-only` can't tell (it tracks nothing): it records ft-camd's names unless `--sides 0|1` says otherwise.
+
+`tools/check_sides.py --ring` is the independent check from the scene (ORB matches meeting under each naming): exit 0 as named, 3 swapped, 2 can't tell. `--pair upper` checks the upper pair the same way: in every recording so far (3 XRService starts, both side namings) the upper pair was named right. The colour cameras are video3 (`arcimx616 0-0010`) and video0 (`0-001a`); which of them is `passthrough_left` in the module's calibration is for `tools/check_color.py` to settle, on a recording with texture in view.
 
 ## ft-hands
 
@@ -81,24 +109,26 @@ Options:
 - `--threads N`: model threads, pinned to the `--cpus` list. Default 3.
 - `--cpus LIST`: CPUs for the model threads and the main loop. Default `5,6,7` (`HANDS_CPUS`). SteamOS starts user processes on CPUs 0-4, and XRService's head tracking runs on 2-3. With the headset on, a step took 8.4 ms on 5-7 against 13.2 ms on 2-4, and SteamVR's frame timing didn't change (2026-09-29, three rounds of the same replayed frames).
 - `--contrast MODE` or `PALM/HAND`: how crops are equalized before the models see them: `clahe[:CLIP]`, `none`, or `stretch` (1st-99th percentile). Default `clahe:2/none`. In the dim recording, CLAHE let the palm search find about 10% more hands, but it made the landmarks jitter more (published median 6.9 mm, against 6.0 mm with plain landmark crops).
-- `--swap-sides`: swap the two side cameras (`HANDS_SWAP_SIDES`, see ft-camd).
+- `--sides auto|0|1`: which side camera is which (`HANDS_SWAP_SIDES`, see "Which camera is which"); `--swap-sides` is `--sides 1`.
 - `--seconds N`: stop after N seconds.
 - `--status S`: how often to print status, in seconds.
 - `--models DIR`: where the models are.
 - `--nice N`: niceness. Default 5, so the VR stack wins contested CPUs.
 - `--no-publish`: don't write the hands and gestures files.
-- `--record DIR`, `--record-for S`: save every frame set for S seconds (default 120) to `DIR/sets.bin`. That's about 80 MB/s. Sending the tracker SIGUSR1 (`pkill -USR1 -x ft-hands`) starts a recording in `~/.local/share/frametop/hands/rec-<time>` without a restart. Recordings are images of your hands and room: they stay on the headset unless you move them.
+- `--no-gestures`: hands for the cutouts only. No pinch or grip detection, so nothing reaches the pointer and a closing hand doesn't raise the rate to 30 Hz. The gestures file is removed at start. `ft-cutouts` runs it this way.
+- `--record DIR`, `--record-for S`, `--record-hz N`: save every frame set for S seconds (default 120) to `DIR/sets.bin`, or at most N sets a second with `--record-hz` (the hand recorder uses 10). Every set is about 80 MB/s. Sending the tracker SIGUSR1 (`pkill -USR1 -x ft-hands`) starts a recording in `~/.local/share/frametop/hands/rec-<time>` without a restart. Recordings are images of your hands and room: they stay on the headset unless you move them.
 - `--record-only`: record without tracking or publishing, so it can run beside the live tracker. Give it `--record DIR`, since SIGUSR1 would reach both trackers. With `ft-camd --with-dark`, recordings also hold each camera's newest dark frame as `<name>_dk`, which doubles the rate. With `--with-color`, each colour camera's newest frame is saved with every set, as `color_video<N>`, which adds about 70 MB/s. Run the recorder at normal I/O priority: idle I/O priority stalled a 165 MB/s recording.
 - `--keep-presence P`: the landmark presence a tracked view needs to stay tracked. New views always need 0.5. Default 0.5. Lowering it to 0.2 barely helped in the bright recording, because lost hands drop to near-zero presence.
 - `--ring PATH`: read frames from another ring, such as `ft-ringplay`'s.
-- `--cams auto|mono|color|all` (`HANDS_CAMERAS`, default `auto`): which cameras to track with. The mono IR cameras light the hands themselves and track well in dim rooms, but in bright light they expose for the room and the hands come out dark. The colour pair is the other way round. `auto` goes by the colour frames' mean brightness: at `--bright-on` (`HANDS_BRIGHT_ON`, 40) or over for 2 s it tracks with `--bright` (`HANDS_BRIGHT`: `all`, every camera, the default, or `color`), and under `--bright-off` (`HANDS_BRIGHT_OFF`, 25) for 2 s with the mono cameras again. A dim evening room read 9. The switch is logged (`cameras: mono -> all (...)`), and the status line gives the colour level, the mono cameras' ambient IR, and how many steps had colour frames. Colour frames arrive on their own schedule, so a step holds the mono set, the colour pair, or both, and views wait in their camera for its next frame.
+- `--cams auto|mono|color|all` (`HANDS_CAMERAS`, default `auto`): which cameras to track with. The mono IR cameras light the hands themselves and track well in dim rooms, but in bright light they expose for the room and the hands come out dark. The colour pair is the other way round. `auto` goes by the colour frames' mean brightness: at `--bright-on` (`HANDS_BRIGHT_ON`, 40) or over for 2 s it tracks with `--bright` (`HANDS_BRIGHT`: `all`, every camera, the default, or `color`), and under `--bright-off` (`HANDS_BRIGHT_OFF`, 25) for 2 s with the mono cameras again. A dim evening room read 9. The switch is logged (`cameras: mono -> all (...)`), and the status line gives the colour level, the mono cameras' ambient IR, and how many steps had colour frames. Colour frames arrive on their own schedule, so a step holds the mono set, the colour pair, or both, and views wait in their camera for its next frame. With no colour cameras in the ring (ft-camd without `--with-color`, as the service runs it), ft-hands tracks with the mono cameras whatever this says.
 - `--color-left NODE` (`HANDS_COLOR_LEFT`, `color_video0`) and `--color-crop subtract|none` (`HANDS_COLOR_CROP`, `subtract`): how the colour module's calibration maps onto the images. Not settled yet: `tools/check_color.py` on a recording with a lit, textured view tells.
-- `--grip-begin R`, `--grip-end R`: the grip detector (below).
+- `--grip-begin R`, `--grip-end R`: the grip detector (below). Defaults 1.2 and 1.45.
+- `--gesture-log`: print what the pinch and grip detectors measure, 10 times a second: each hand's thumb-to-index distance (world and triangulated), its palm-down reading and its finger curl.
 
 **Gestures** (`/run/user/UID/frametop-hands/gestures`, `include/fh_gestures.h`), for the pointer helper:
 
-- A pinch: the thumb and index tips within 2 cm, ending past 3.5 cm. Not begun with the palm facing down (`--pinch-palm-down`, 0.6), which is how typing looks.
-- A grip, a closed hand: every finger's tip nearer the wrist than 1.2 times its knuckle is (from the model's 3D hand, so hand size doesn't matter), ending when they open past 1.45 on average. It begins only on a hand seen open within the last second (closing it is the gesture), with the palm at most 35 degrees below straight ahead and at least 15 cm in front of the eyes. A grip ends a pinch on the same hand, as lost. In the 2026-09-30 lit recording (no deliberate fists), the checks cut false grips from 14 to 6, all with the hands on the desk while looking down at it; the pointer helper ignores grips that begin more than 30 cm below the eyes, which it can tell and ft-hands can't.
+- A pinch: the thumb and index tips within 2 cm, ending past 3.5 cm (see "Pinch" below).
+- A grip, a closed hand: every finger's tip nearer the wrist than 1.2 times its knuckle is (from the model's 3D hand, so hand size doesn't matter), ending when they open past 1.45 on average. It begins only on a hand seen open within the last second (closing it is the gesture), with the palm at most 35 degrees below straight ahead and at least 15 cm in front of the eyes, and not with the thumb within 3 cm of the index tip (that's a pinch with the other fingers curled). A grip ends a pinch on the same hand, as lost. In the 2026-09-30 lit recording (no deliberate fists), the checks cut false grips from 14 to 6, all with the hands on the desk while looking down at it. The pointer helper ignores grips that begin more than `POINTER_GRIP_BELOW` (0.35 m) below the eyes, which it can tell and ft-hands can't.
 - `tools/watch_gestures.py --distance` shows both live; `ft-handreplay --timeline` logs them and each hand's finger curl.
 
 The status line also says how often a hand was on each side (by where the wrist is), and why views and hands came and went: views lost (the landmark model stopped seeing the hand), handoff misses (a crop projected from the hand's 3D position found nothing), duplicates, splits (two views disagreed in 3D), and hands created, merged and forgotten.
@@ -125,13 +155,30 @@ How good the depth is, measured from recordings (2026-09-30, `--depth` below): t
 ft-hands detects a pinch per hand (`track/pinch.h`) and publishes it to the gestures file. The layout, and how to read it without missing quick taps, is in `include/fh_gestures.h`.
 
 - A pinch begins when the thumb and index tips come within `--pinch-begin` (default 0.020 m). It ends when they open past `--pinch-end` (0.035 m) for 2 processed frames in a row, or when the hand stays lost for 0.25 s (flagged lost).
-- The distance comes from MediaPipe's world landmarks: the model's own 3D hand pose, averaged over the hand's views, at the user's hand size. `--pinch-triangulated` uses the triangulated tips instead. On two recordings without deliberate pinches, the world landmarks came under 2 cm in 0.2-1% of frames, against 3.3-4.5% for the triangulated tips. In the dim recording, typing still gave 2 pinches a minute before the palm check below.
-- No pinch begins while the palm faces down (`--pinch-palm-down MAX`: the palm normal's share of the head's up axis, default 0.6; 1 turns it off), and a close held back that way has to open again before a pinch can begin. Typing curls the thumb onto the index. In the lit recording of 2026-09-30, typing on a keyboard in the lap began 23 pinches in about 2 minutes, all with the palm facing down (0.69-1.00), while the 26 deliberate ones read 0.00-0.50. The limit held back every typing pinch and none of the deliberate ones. Looking down tilts the head frame, which lowers the reading for a hand on a keyboard, so the consumer's gaze check stays the other guard.
+- The distance comes from MediaPipe's world landmarks: the model's own 3D hand pose, averaged over the hand's views, at the user's hand size. `--pinch-triangulated` uses the triangulated tips instead. On two recordings without deliberate pinches, the world landmarks came under 2 cm in 0.2-1% of frames, against 3.3-4.5% for the triangulated tips. In the dim recording, typing still gave 2 pinches a minute (see the next point).
+- `--pinch-palm-down MAX` holds back pinches begun with the palm facing down (MAX is the palm normal's share of the head's up axis). The default, 1, turns it off. A close held back that way has to open again before a pinch can begin. Typing curls the thumb onto the index: in the lit recording of 2026-09-30, typing on a keyboard in the lap began 23 pinches in about 2 minutes, all with the palm facing down (0.69-1.00), while the 26 deliberate ones read 0.00-0.50. But in the headset, deliberate pinches with the hand raised in front read 0.90-0.99 too, so the limit is off. Typing is caught by the pointer helper instead: the input relay tells it when you press a key, and no pinch begins within `POINTER_PINCH_TYPING` of one.
 - A hand a pinch is down on stays with that side until the pinch ends. The left/right call is a running average of the model's, and when it flipped mid-pinch, the other side took the same hand and both sides pinched at once.
-- The pinch point is midway between the thumb and index tips. A drag is the pinch point now, minus where it was when the pinch began, both turned into the room with the HMD pose at their capture times.
+- The pinch point is between the index and middle knuckles, which hold still while the fingers open and close. The tips' midpoint moved 1-2 cm as a pinch opened, which dragged every release off its press. A drag is the pinch point now, minus where it was when the pinch began, both turned into the room with the HMD pose at their capture times.
 - `tools/watch_gestures.py` prints begins, ends and drag offsets live, and `--distance` prints each hand's distance.
 
-The pointer helper is the natural consumer. Its gaze mode already treats a press as "stop where the gaze put it, drag onto the target, click on release", and "hold still for half a second, then move" as a drag. A pinch begin would be the press, the end the release, and the pinch point's movement the drag.
+## Pinches and grips in the pointer
+
+With `POINTER_HANDS=1`, the pointer helper (`pointer/helper/ft-pointer.cpp`) reads the gestures file every frame. It's off by default.
+
+- **Pinch to click.** In gaze mode a pinch works like the mouse's press: the pointer stops where the gaze put it, and the click comes when the pinch opens, where the pointer is then. A quick tap clicks where you looked. Held, the pinching hand moves the pointer to correct the gaze, and the correction is a lesson for the gaze tracker, as with the mouse.
+- **Without gaze mode,** a pinch is a real press, like the mouse's button: pressed when it closes, released when it opens, and while it's held the hand drags the pointer. A tap is still a click where the pointer is.
+- **Grip to drag.** Closing the hand presses where the pointer is, the hand moves the pointer, and opening the hand releases. So a title bar moves its window, a panel's grab bar carries the panel, and text gets selected.
+- A pinch ended by losing the hand, or by a grip taking over, doesn't click.
+- The hand's movement is taken in the room, from where the eye was when the gesture began, so turning your head doesn't move the pointer. The first gesture while the pointer is off only wakes it. Gestures are ignored in a VR game (unless the dashboard is up), with the headset off, and while the mouse's button is held.
+
+Settings in `~/.config/frametop.conf`:
+
+- `POINTER_HANDS` (0): 1 turns pinches and grips on.
+- `POINTER_PINCH_GAIN` (0.5): a held pinch moves the pointer this many times the hand's angle, seen from the eye. Under 1 gives precision.
+- `POINTER_PINCH_DEADZONE` (1.5): how many degrees the pinching hand moves before the pointer does, so a tap's jitter and the pinch point shifting as the fingers close don't move it.
+- `POINTER_GRIP_GAIN` (1): a grip moves the pointer this many times the hand's angle.
+- `POINTER_GRIP_BELOW` (0.35): grips that begin more than this many metres below the eyes are ignored, because hands resting on a desk curl like a loose fist. Pinches have no such limit: deliberate ones sat 0.35-0.45 m below the eyes with the elbow resting.
+- `POINTER_PINCH_TYPING` (1): no pinch begins within this many seconds of a key press, because typing touches thumb to index.
 
 ## Recordings
 
@@ -155,15 +202,71 @@ hands/build/ft-handreplay ~/.local/share/frametop/hands/rec-20260929-120000 --co
 
 ## Tools
 
-Python, with NumPy and OpenCV (in the dev container: `python3-numpy`, `python3-opencv`, which `setup/dev-container.sh` installs). Off the Frame, `FRAME_JOB_DEVICE_ROOT` can point at a folder with copies of the headset's calibration files.
+Python, with NumPy and OpenCV. `setup/dev-container.sh` doesn't install them, because Fedora's `python3-opencv` pulls in over a gigabyte; in the dev container, run `sudo dnf install python3-numpy python3-opencv` once. Off the Frame, `FRAME_JOB_DEVICE_ROOT` can point at a folder with copies of the headset's calibration files.
 
-- `tools/check_sides.py --ring` (or a recording): are the side cameras named right?
+- `tools/check_sides.py --ring` (or a recording, a sets file, `--pair upper`, `--calib DIR`, `--json`): are the side cameras named right, from the scene? ft-handreplay's `--sides file|0|1|auto` replays with DIR/sides.json's names (the default), as recorded, exchanged, or as auto decides, and reports what the side check found and when.
 - `tools/check_color.py REC`: how the colour module's calibration maps onto its images.
 - `tools/show_set.py REC`: a recording's frame sets as images.
-- `tools/watch_gestures.py [--distance]`: pinches, live.
+- `tools/watch_gestures.py [--distance]`: pinches and grips, live.
 - `tools/depth_report.py DEPTH`: the depth measures above.
+- `tools/cut_sets.py REC OUT [--sets N | --at I,J,...]`: copies a few frame sets (by default 8, spread evenly) out of a recording into a small one, to look at or check elsewhere without moving gigabytes. Plain Python, so it also runs on the Frame's host.
 - `tools/convert_models.py`: how `models/ncnn` was made from the OpenCV Zoo ONNX ports of MediaPipe's models (see `models/NOTICE`).
 
+To try the hand cutouts without restarting the desktop, `screens/build/ft-handtest [--distance m] [--width m] [--seconds s]` (built by `screens/build.sh`, run in the dev container, with hand tracking on) shows a test panel of its own, a light grid 1 m wide and 0.8 m ahead by default, and cuts your hands out of it the way ft-screens cuts them out of the screens.
+
+## Camera check
+
+Hand tracking needs all four mono cameras and the headset's IR light. With the Arcturus colour module attached, SteamVR's XRService loads an FPGA image ("VCINT") onto the module every time it opens the cameras: when SteamVR starts and after every wake. When that load fails, XRService runs only the two side cameras, the frames come out darker and noisier, and ft-hands finds no hands at all. It happened on 2026-10-02 at 17:02, after the headset slept; the hand recorder then said "I can't see your hands" for a whole session.
+
+`hands/camcheck.py` (system Python, standard library) tells whether the four cameras run: `ok`, `degraded: upper cameras and IR light off (VCINT FPGA failed to load)` (or another `degraded:` reason), or `unknown` (SteamVR not running, the cameras closed while the headset sleeps). It prints the log lines and other evidence it used; `--json` is for programs; the exit status is 0, 1 or 2. It reads:
+- the running XRService's log (`~/.local/share/Steam/logs/xrservice.txt`): the last camera start (from the FPGA check to the next "Closing tracking camera interfaces"), its VCINT result, `Upper cameras FPGA interleaving support: N`, `Created N tasks (T tracking, P passthrough)` and the `TrackingCameraInit` lines. A wake that works prints no "Created N tasks", so an older one doesn't count;
+- which `/dev/video*` XRService has open (`/proc/PID/fd`; video9 and video13 are the side pair, video6 and video7 the upper pair). Only on the host: the dev container can't read another process's open files, so there it's skipped;
+- ft-camd's ring header, when it runs: how many mono cameras it publishes.
+
+The hand recorder runs it before a session (DESIGN.md, "Camera check"). `hands/tests/test_camcheck.py` runs it on the 2026-10-02 log cut at several points, and on made-up logs.
+
+### The watcher (off by default)
+
+`hands/ft-camwatch` follows the XRService log (a stat every 2 s, reading only what's new). On a VCINT failure it posts a notification in the Frametop desktop, on the desktop's own D-Bus, found through its plasmashell as `decoration/apply.sh` does. With `CAMWATCH_AUTO_RESTART=1` in `~/.config/frametop.conf` it also restarts SteamVR, but only:
+- while the headset isn't worn (frame-job's check: `vrcompositor` runs and a `/sys/class/backlight/*/brightness` is over 0), and after it has been off for `CAMWATCH_IDLE_S` (60);
+- with no app Steam launched (`SteamLaunch AppId=N` in a process's arguments; `CAMWATCH_IGNORE_APPIDS` lists ids that don't count) and nobody on the remote desktop (an established connection to the VNC port, `VNC_PORT`, 5900);
+- once per failure, and not again within `CAMWATCH_COOLDOWN_MIN` (30) of the last automatic restart. It remembers both in `~/.local/state/frametop/camwatch.json`, so its own restart doesn't reset them.
+
+It logs every decision to the journal. `ft-camwatch --once` prints the state and what it would do, and does nothing; `--dry-run` keeps watching without acting. `hands/frametop-camwatch.service` is the unit (a template, `@REPO@` as in the others; nothing installs or enables it yet). It isn't `PartOf=steamvr.service`, so it outlives the restart it asks for. `CAMWATCH_NOTIFY=0` turns the notification off. `hands/tests/test_camwatch.py` tests its decisions with made-up inputs.
+
+### What a SteamVR restart does to Frametop
+
+Read from the code on the experimental branch, not tried live:
+- ft-screens quits when SteamVR does: on `VREvent_Quit` it ends its Wayland display (`screens/vr.cpp`, `ft_vr_poll`; `screens/compositor.c`, `handle_vr_event`). It never connects to SteamVR again: `ft_vr_init` runs once, at its start.
+- KWin runs nested in ft-screens, so the Frametop desktop ends with it, every window in it too (the hand recorder's as well). Its unit, `frametop-desktop`, is a transient `systemd-run` unit with `Restart=no`, so the desktop doesn't come back by itself: start it again (Desktop in the library, or `desktops.sh start`).
+- When the unit stops, systemd ends what's left in it. `session/keep-apps.sh` moves programs started in the desktop out of the unit first, but only `desktops.sh stop` runs it; here they stop too. Programs in the dev container (ft-screens, the hand recorder) are in the container's cgroup and end when their Wayland connection goes.
+- The units that are `PartOf=steamvr.service` restart with it: `frametop-camd`, `frametop-hands`, the pointer helper, gaze and power, and the hand recorder's own transient ft-camd and ft-hands units.
+
+### Verified, and what's a guess
+
+Verified, from the XRService logs of 2026-10-01 and 2026-10-02 and the running system:
+- The failure's log lines and its effect: "Failed to load VCINT FPGA image when passthrough cameras are connected", interleaving support 0, "Created 4 tasks (2 tracking, 2 passthrough)", and only video9 and video13 opened. At 19:38-19:59 XRService held only those two of the four (plus video0 and video3), and ft-camd published two mono cameras.
+- A wake's load can work and can fail. Both wakes in the logs started from an FPGA that answered nothing ("ERROR/UNKNOWN"): the one at 2026-10-01 16:39 loaded VCINT, the one at 2026-10-02 17:02 failed ("FPGA config_done signal did not assert").
+- After a reboot the FPGA reads PASSTHRU and SteamVR's start loads VCINT (2026-10-01 21:53, 2026-10-02 13:39).
+- A SteamVR restart within a boot found VCINT still loaded and loaded nothing (2026-10-01 15:27): XRService checks the FPGA when it starts and loads only when it must.
+
+Guesses, not tested:
+- **Whether a SteamVR restart fixes it.** After a failed load the FPGA doesn't answer, so a new XRService would run the same load a wake runs, which has worked once and failed once. It's never been tried after a failure. If it doesn't help, only a reboot is known to work (the FPGA comes up as PASSTHRU, and the load at SteamVR's start has worked both times).
+- That the IR light is off because of the FPGA: the frames are darker and the illuminator ring isn't seen, and the FPGA loader lists a `room_led_en` pin, but nothing shows the light's state directly.
+- That a sleep and wake (taking the headset off long enough) would retry the load too: it should, since every wake loads VCINT, but no failure has been followed by a wake yet.
+- How the Frametop desktop behaves on a SteamVR restart (above): read from the code only.
+- That Steam-launched apps carry `SteamLaunch AppId=N`: from Steam on other Linux systems; no VR game has run on the Frame to confirm it.
+
 ## Build
 
-`hands/build.sh` builds in the dev container on the Frame, into `hands/build/`, with `hands/Makefile`. The first build fetches ncnn at a pinned tag and builds it into `hands/build/ncnn`, which takes a few minutes; `NCNN=DIR` points at an ncnn install already built instead. ft-camd is linked statically, because it runs on the host, which has an older glibc than the container.
+`hands/build.sh` builds in the dev container on the Frame, into `hands/build/`, with `hands/Makefile`. The first build fetches ncnn at a pinned tag (`NCNN_TAG` in the Makefile) and builds it into `hands/build/ncnn`, which takes a few minutes; `NCNN=DIR` points at an ncnn install already built instead. ft-camd is linked statically, because it runs on the host, which has an older glibc than the container.
+
+## Known issues
+
+- **The side cameras can come out swapped.** ft-camd tells the side cameras' buffers apart only by XRService's allocation order, and some XRService restarts reverse it. ft-hands corrects it from the hands (`HANDS_SWAP_SIDES=auto`, the default). Until it has seen about 1-2 s of hands in both namings' reach, the cutouts may sit beside the hands. ft-camd itself still can't tell.
+- **The colour cameras can't be used while the headset is worn.** The colour module then writes only a half-size image into the top-left quarter of its buffers, and ft-camd drops those frames. So the service runs the mono cameras only, and tracking in bright light, where the mono cameras see dark hands, doesn't get the colour pair's help.
+- **The colour calibration mapping isn't settled.** Which colour camera is `passthrough_left` (`HANDS_COLOR_LEFT`) and how the module's crop applies (`HANDS_COLOR_CROP`) still need `tools/check_color.py` on a recording with a lit, textured view.
+- **Depth when one camera loses the hand.** A hand seen in one camera drifts 10% per update toward the one-camera depth guess (`kMonoDepthGain`, 0.1, in `track/tracker.cpp`). In the 2026-09-30 replays that was worse than keeping the last distance (see "3D" above). A smaller gain, such as 0.02, is the next thing to try.
+- **Pinches aren't reliable enough for everyday use yet.** That's why hand tracking stays off until `ft-handsctl on`, and `POINTER_HANDS` is 0 by default.
+- **SteamVR can leave the upper cameras and the IR light off after a wake**, and then no hands are found. See "Camera check" above: `camcheck.py` tells, the hand recorder won't start a session, and the fix is a SteamVR restart or a reboot.
+- **Floating windows don't get hand cutouts.** Their panels show crops of the client buffer, which the cutouts' side-by-side buffer doesn't match (`screens/vr.cpp`, `UpdateCutouts`).

hands/camcheck.py

@@ -0,0 +1,459 @@
+#!/usr/bin/env python3
+"""camcheck: are the headset's four mono tracking cameras running, so that ft-camd and
+ft-hands see them all?
+
+The Frame has four mono IR tracking cameras: the side pair slam_left and slam_right
+(/dev/video9 and /dev/video13) and the upper pair (/dev/video6 and /dev/video7). With the
+Arcturus colour module attached, SteamVR's XRService loads an FPGA image ("VCINT") onto the
+module whenever it opens the cameras (at start and after every wake). When that load fails
+(seen 2026-10-02 17:02, after a sleep), XRService runs only the two side cameras, the IR
+illuminator seems to stay off, and ft-hands finds no hands at all.
+
+What it looks at, cheapest first, all read-only:
+  1. The newest XRService log (~/.local/share/Steam/logs/xrservice.txt, a symlink to the
+     running instance's log): the last camera start, its VCINT result, "Upper cameras FPGA
+     interleaving support: N", "Created N tasks (T tracking, P passthrough)" and the
+     TrackingCameraInit lines. A wake doesn't always print "Created N tasks", so the parser
+     tracks each camera start ("episode") from the FPGA check to the next close.
+  2. Which /dev/video* XRService has open (/proc/PID/fd). Only on the host: the dev container
+     can't read another process's fd table (checked 2026-10-02), and then this is skipped.
+  3. A running ft-camd's ring header (/run/user/UID/frametop-hands/cam-ring): how many mono
+     cameras it publishes.
+
+Status: "ok", "degraded: <why>" or "unknown" (SteamVR not running, the cameras closed while
+the headset sleeps, no log). Exit status 0, 1, 2 for those.
+
+  python3 hands/camcheck.py             # the status and its evidence
+  python3 hands/camcheck.py --json      # for programs
+  python3 hands/camcheck.py --log FILE --no-proc --no-ring   # a saved log only (tests)
+
+System Python, standard library only; session.py and ft-camwatch import it.
+"""
+import argparse
+import glob
+import json
+import os
+import re
+import struct
+import sys
+import time
+
+LOG_DIR = os.path.expanduser("~/.local/share/Steam/logs")
+LOG_LINK = os.path.join(LOG_DIR, "xrservice.txt")
+SIDE_NODES = (9, 13)     # slam_left, slam_right (TrackingCameraInit index 0 and 1)
+UPPER_NODES = (6, 7)     # the upper pair (index 2 and 3)
+TRACKING = 4
+
+VCINT_REASON = "upper cameras and IR light off (VCINT FPGA failed to load)"
+DEGRADED_VCINT = "degraded: " + VCINT_REASON
+
+# What the window and the recorder say when the check fails this way.
+USER_TEXT = ("The headset's upper cameras and IR light are off. SteamVR couldn't start the colour camera "
+             "module (it happens sometimes after the headset sleeps). Restart SteamVR, or restart the headset "
+             "if that doesn't fix it.")
+
+ANSI = re.compile(r"\x1b\[[0-9;]*m")
+STAMP = re.compile(r"^\w{3} \w{3} \d{2} \d{4} (\d{2}:\d{2}:\d{2})\.\d+ (\w+): ?(.*)$")
+# Lines worth reading; anything else is skipped before the regexes (the log grows by MBs a day).
+KEYS = ("FPGA", "VCINT", "Created", "TrackingCameraInit", "Closing tracking camera", "Streaming",
+        "systemd suspend", "systemd resume", "XRService logging to", "Exiting XRService", "ISP ")
+# XRService's numbering of its tracking cameras (the TrackingCameraInit index): ft-hands names
+# them this way too (track/main.cpp, cameras_from_xrservice_log).
+NAMES = ("slam_left", "slam_right", "upper_left", "upper_right")
+RE_PASSTHRU = re.compile(r"Passthrough connected but FPGA is (\S+) - loading VCINT")
+RE_INTERLEAVE = re.compile(r"Upper cameras FPGA interleaving support: (\d)")
+RE_TASKS = re.compile(r"Created (\d+) tasks \((\d+) tracking, (\d+) passthrough\)")
+RE_INIT = re.compile(r"TrackingCameraInit: index: (\d+)\. video device: /dev/video(\d+)")
+RE_STREAM = re.compile(r"Streaming resumed \(FPGA: (\S+), VC interleaving: (\w+)\)")
+RE_STATE = re.compile(r"FPGA state check: (\S+)")
+# Without the colour module XRService runs the side cameras through the ISP, as NV12 on other
+# capture pipes (vfe0 and vfe1), and the upper pair on vfe3 and vfe4.
+RE_ISP = re.compile(r"ISP (enabled|disabled) for tracking cameras")
+
+
+class LogState:
+    """Reads an XRService log line by line (feed), so the watcher can follow it as it grows.
+
+    An episode is one opening of the cameras: from the first FPGA, task or camera-init line
+    after the log starts or after "Closing tracking camera interfaces", to the next close."""
+
+    def __init__(self, path=""):
+        self.path = path
+        self.instance = ""        # the "XRService logging to" line's time
+        self.exited = False
+        self.closed = False       # the cameras were closed and haven't opened again
+        self.closed_at = ""
+        self.episode = None
+        self.nodes = {}           # TrackingCameraInit index -> /dev/videoN, from the whole log
+        self.failures = []        # [(time, line)]: every VCINT failure in this log
+        self.lines = 0
+
+    def _new_episode(self, t):
+        self.closed = False
+        self.episode = {"start": t, "fpga_before": "", "vcint": "", "interleave": None, "tasks": None,
+                        "inits": {}, "stream": "", "isp": None, "failure": "", "evidence": []}
+        if self.closed_at:
+            self.episode["evidence"].append(self.closed_at)
+
+    def _ep(self, t):
+        if self.episode is None or self.closed:
+            self._new_episode(t)
+        return self.episode
+
+    def feed(self, raw):
+        self.lines += 1
+        if not any(k in raw for k in KEYS):
+            return
+        line = ANSI.sub("", raw).rstrip("\n")
+        m = STAMP.match(line)
+        if not m:
+            return   # the FPGA loader's own output, without a time
+        t, _level, text = m.groups()
+        short = ("%s %s" % (t, text))[:220]
+        if "XRService logging to" in text:
+            lines = self.lines
+            self.__init__(self.path)
+            self.instance, self.lines = t, lines
+            return
+        if "Exiting XRService" in text:
+            self.exited = True
+            return
+        if "Closing tracking camera interfaces" in text:
+            self.closed, self.closed_at = True, short
+            return
+        if "systemd suspend notification" in text or "systemd resume notification" in text:
+            if "resume" in text:
+                self.closed_at = (self.closed_at + " / " if self.closed_at else "") + short
+            return
+        m = RE_PASSTHRU.search(text)
+        if m:
+            ep = self._ep(t)
+            ep["fpga_before"] = m.group(1)
+            ep["evidence"].append(short)
+            return
+        if "FPGA image VCINT loaded and verified successfully" in text:
+            ep = self._ep(t)
+            ep["vcint"] = "ok"
+            ep["evidence"].append(short)
+            return
+        if "Failed to load VCINT FPGA image" in text:
+            ep = self._ep(t)
+            ep["vcint"] = "failed"
+            ep["failure"] = t
+            ep["evidence"].append(short)
+            self.failures.append((t, short))
+            return
+        if "FPGA load failed" in text:
+            self._ep(t)["evidence"].append(short)
+            return
+        m = RE_INTERLEAVE.search(text)
+        if m:
+            ep = self._ep(t)
+            ep["interleave"] = int(m.group(1))
+            ep["evidence"].append(short)
+            return
+        m = RE_ISP.search(text)
+        if m:
+            ep = self._ep(t)
+            ep["isp"] = m.group(1) == "enabled"
+            ep["evidence"].append(short)
+            return
+        m = RE_TASKS.search(text)
+        if m:
+            ep = self._ep(t)
+            ep["tasks"] = tuple(int(v) for v in m.groups())
+            ep["evidence"].append(short)
+            return
+        m = RE_INIT.search(text)
+        if m:
+            ep = self._ep(t)
+            idx, node = int(m.group(1)), int(m.group(2))
+            ep["inits"][idx] = node
+            self.nodes[idx] = node
+            ep["evidence"].append(short)
+            return
+        m = RE_STREAM.search(text)
+        if m:
+            ep = self._ep(t)
+            ep["stream"] = "%s, interleaving %s" % m.groups()
+            ep["evidence"].append(short)
+            return
+        m = RE_STATE.search(text)
+        if m:
+            ep = self._ep(t)
+            if not ep["fpga_before"] and not ep["vcint"]:
+                ep["fpga_before"] = m.group(1)
+                if m.group(1) == "VCINT":
+                    ep["vcint"] = "loaded"   # already there: no load needed (a SteamVR restart in the same boot)
+                ep["evidence"].append(short)
+
+    def feed_text(self, text):
+        for line in text.splitlines():
+            self.feed(line)
+        return self
+
+    def upper_nodes(self):
+        got = tuple(self.nodes[i] for i in (2, 3) if i in self.nodes)
+        return got if len(got) == 2 else UPPER_NODES
+
+    def camera_map(self):
+        """{calibration name: /dev/videoN's N} from the latest camera start's TrackingCameraInit
+        lines (the whole log's when that start has none yet)."""
+        inits = (self.episode or {}).get("inits") or self.nodes
+        return {NAMES[i]: node for i, node in sorted(inits.items()) if 0 <= i < len(NAMES)}
+
+    def tracking_nodes(self):
+        got = tuple(self.nodes[i] for i in range(TRACKING) if i in self.nodes)
+        return got if len(got) == TRACKING else SIDE_NODES + UPPER_NODES
+
+    def verdict(self):
+        """(status, reason, evidence): status "ok", "degraded" or "unknown"."""
+        if self.lines == 0:
+            return "unknown", "the XRService log is empty", []
+        if self.exited:
+            return "unknown", "XRService has exited (SteamVR isn't running)", []
+        if self.episode is None:
+            return "unknown", "the cameras haven't started yet in this log", []
+        ep = self.episode
+        ev = ep["evidence"][-14:]
+        if self.closed:
+            return "unknown", "the cameras are closed (the headset is asleep, or SteamVR is stopping)", \
+                ev + [self.closed_at]
+        tracking = len(ep["inits"]) if ep["inits"] else (ep["tasks"][1] if ep["tasks"] else None)
+        if ep["vcint"] == "failed":
+            return "degraded", VCINT_REASON, ev
+        if tracking is not None and tracking < TRACKING:
+            why = "only %d of %d tracking cameras running" % (tracking, TRACKING)
+            if ep["interleave"] == 0:
+                why += " (upper cameras' FPGA interleaving off)"
+            return "degraded", why, ev
+        if tracking == TRACKING:
+            return "ok", "%d tracking cameras running" % TRACKING, ev
+        return "unknown", "the cameras are starting", ev
+
+    def snapshot(self):
+        status, reason, ev = self.verdict()
+        ep = self.episode or {}
+        return {"status": status, "reason": reason, "evidence": ev, "log": self.path, "instance": self.instance,
+                "episode": {k: (list(v) if isinstance(v, tuple) else v) for k, v in ep.items() if k != "evidence"},
+                "failure": "%s@%s" % (self.path, ep["failure"]) if ep.get("vcint") == "failed" else ""}
+
+
+# ------------------------------------------------------------------------------------------
+# Processes
+
+def proc_argv(pid):
+    try:
+        with open("/proc/%s/cmdline" % pid, "rb") as f:
+            return [a.decode(errors="replace") for a in f.read().split(b"\0") if a]
+    except OSError:
+        return []
+
+
+def xrservice_pid():
+    """XRService's pid (its main thread renames itself XRServiceLoopTh), or None."""
+    for pid in os.listdir("/proc"):
+        if not pid.isdigit():
+            continue
+        try:
+            with open("/proc/%s/comm" % pid) as f:
+                if not f.read().startswith("XRService"):
+                    continue
+        except OSError:
+            continue
+        argv = proc_argv(pid)
+        if argv and os.path.basename(argv[0]) == "XRService":
+            return int(pid)
+    return None
+
+
+def xrservice_fds(pid):
+    """{"videos": [N, ...], "log": path or ""} from /proc/PID/fd, or None if it can't be read
+    (the dev container can't)."""
+    try:
+        fds = os.listdir("/proc/%d/fd" % pid)
+    except OSError:
+        return None
+    videos, log = set(), ""
+    for fd in fds:
+        try:
+            target = os.readlink("/proc/%d/fd/%s" % (pid, fd))
+        except OSError:
+            continue
+        m = re.match(r"/dev/video(\d+)$", target)
+        if m:
+            videos.add(int(m.group(1)))
+        elif re.search(r"/XRService-[^/]*\.log$", target):
+            log = target
+    if not videos and not log:
+        return None   # nothing readable: as good as no access
+    return {"videos": sorted(videos), "log": log}
+
+
+def newest_log():
+    """The running XRService's log: the xrservice.txt symlink, else the newest by time."""
+    if os.path.exists(LOG_LINK):
+        return os.path.realpath(LOG_LINK)
+    found = glob.glob(os.path.join(LOG_DIR, "XRService-*", "XRService-*.log"))
+    found += glob.glob(os.path.join(LOG_DIR, "XRService-*.log"))
+    found = [p for p in found if os.path.isfile(p)]
+    return max(found, key=os.path.getmtime) if found else ""
+
+
+def read_log(path):
+    st = LogState(path)
+    with open(path, "r", errors="replace") as f:
+        for line in f:
+            st.feed(line)
+    return st
+
+
+# ------------------------------------------------------------------------------------------
+# ft-camd's ring (camd/fhring.h; the header only, as session.py's Ring reads it)
+
+RING_HDR = struct.Struct("<8sIIIIQqQ16x")
+RING_CAM = struct.Struct("<32s32siIIIIIQQQQQIf24x")
+FH_CAM_DARK, FH_CAM_COLOR = 1, 2
+
+
+def default_ring():
+    return "/run/user/%d/frametop-hands/cam-ring" % os.getuid()
+
+
+def read_ring(path):
+    """{"alive", "writer_pid", "mono": [{"name", "sensor", "node"}]} or None (no ring)."""
+    try:
+        with open(path, "rb") as f:
+            data = f.read(RING_HDR.size + 8 * RING_CAM.size)
+    except OSError:
+        return None
+    if len(data) < RING_HDR.size:
+        return None
+    magic, version, _, ncams, _, _, writer, _ = RING_HDR.unpack_from(data, 0)
+    if magic != b"FHRING01" or version != 1:
+        return None
+    hb = struct.unpack_from("<Q", data, 40)[0]
+    alive = hb != 0 and (time.clock_gettime_ns(time.CLOCK_MONOTONIC) - hb) / 1e9 < 2.0
+    mono = []
+    for i in range(min(ncams, 8)):
+        off = RING_HDR.size + i * RING_CAM.size
+        if off + RING_CAM.size > len(data):
+            break
+        f = RING_CAM.unpack_from(data, off)
+        sensor = f[0].split(b"\0", 1)[0].decode(errors="replace")
+        name = f[1].split(b"\0", 1)[0].decode(errors="replace")
+        if f[13] & (FH_CAM_DARK | FH_CAM_COLOR) or name.endswith("_dk") or name.startswith("color"):
+            continue
+        mono.append({"name": name, "sensor": sensor, "node": f[2]})
+    return {"alive": alive, "writer_pid": writer, "mono": mono}
+
+
+# ------------------------------------------------------------------------------------------
+# The check
+
+def check(log=None, proc=True, ring=True, ring_path=None):
+    """The cameras' state: {"status": "ok"|"degraded"|"unknown", "summary": "ok" or
+    "degraded: ..." or "unknown: ...", "reason", "evidence": [lines], "log", "xrservice", "ring"}."""
+    evidence = []
+    pid = xrservice_pid() if proc else None
+    fds = xrservice_fds(pid) if pid else None
+    path = log or (fds or {}).get("log") or newest_log()
+    state = None
+    if path:
+        try:
+            state = read_log(path)
+        except OSError as e:
+            evidence.append("log %s: %s" % (path, e))
+    if state:
+        status, reason, ev = state.verdict()
+        evidence += ["log %s:" % path] + ["  " + e for e in ev]
+    else:
+        status, reason = "unknown", "no XRService log in %s" % LOG_DIR
+    out = {"log": path, "xrservice": None, "ring": None,
+           "episode": state.snapshot()["episode"] if state else {},
+           "failure": state.snapshot()["failure"] if state else "",
+           "map": {name: {"node": n, "pipe": pipe_name(n)} for name, n in state.camera_map().items()} if state else {},
+           "ring_missing": []}
+
+    if proc:
+        if pid is None:
+            # The log can't tell a killed XRService from a running one; no process settles it.
+            evidence.append("XRService isn't running")
+            status, reason = "unknown", "SteamVR isn't running (no XRService)"
+        elif fds is None:
+            evidence.append("XRService pid %d: its open files can't be read here (in the dev container?)" % pid)
+            out["xrservice"] = {"pid": pid, "videos": None}
+        else:
+            videos = fds["videos"]
+            want = state.tracking_nodes() if state else SIDE_NODES + UPPER_NODES
+            upper = state.upper_nodes() if state else UPPER_NODES
+            have = [n for n in want if n in videos]
+            evidence.append("XRService pid %d has open: %s (tracking cameras: %s; upper: %s)" % (
+                pid, " ".join("video%d" % n for n in videos) or "no cameras",
+                " ".join("video%d" % n for n in want), " ".join("video%d" % n for n in upper)))
+            out["xrservice"] = {"pid": pid, "videos": videos, "tracking_open": len(have)}
+            closed = state is not None and state.closed
+            if not closed and len(have) == TRACKING and status == "unknown":
+                status, reason = "ok", "XRService has all %d tracking cameras open" % TRACKING
+            elif not closed and videos and not all(n in videos for n in upper) and status != "degraded":
+                status, reason = "degraded", ("XRService has %d of %d tracking cameras open (the upper pair "
+                                              "is missing)" % (len(have), TRACKING))
+
+    if ring:
+        r = read_ring(ring_path or default_ring())
+        out["ring"] = r
+        if r is None:
+            evidence.append("ft-camd: no camera ring (not running)")
+        else:
+            names = " ".join(c["name"] for c in r["mono"]) or "none"
+            evidence.append("ft-camd (pid %d, %s): %d mono cameras: %s" % (
+                r["writer_pid"], "running" if r["alive"] else "stale ring", len(r["mono"]), names))
+            if r["alive"] and len(r["mono"]) < TRACKING and status == "ok":
+                have = {c["node"] for c in r["mono"]}
+                want = state.tracking_nodes() if state else SIDE_NODES + UPPER_NODES
+                out["ring_missing"] = [n for n in want if n not in have]
+                status, reason = "degraded", ("ft-camd publishes only %d of %d mono cameras, missing %s" % (
+                    len(r["mono"]), TRACKING, " ".join("video%d" % n for n in out["ring_missing"]) or "?"))
+    out.update(status=status, reason=reason, evidence=evidence,
+               summary="ok" if status == "ok" else "%s: %s" % (status, reason))
+    return out
+
+
+def pipe_name(node):
+    """The capture pipe behind /dev/videoN (msm_vfe3_video0 and so on), or ""."""
+    try:
+        with open("/sys/class/video4linux/video%d/name" % node) as f:
+            return f.read().strip()
+    except OSError:
+        return ""
+
+
+def is_ring_short(result):
+    """XRService runs all the tracking cameras, but ft-camd doesn't publish them all."""
+    return bool(result) and result.get("status") == "degraded" and bool(result.get("ring_missing"))
+
+
+def is_vcint_failure(result):
+    return bool(result) and result.get("status") == "degraded" and result.get("reason") == VCINT_REASON
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(description="Are the headset's four mono tracking cameras running?")
+    ap.add_argument("--json", action="store_true", help="print the result as JSON")
+    ap.add_argument("--log", help="read this XRService log (default: the running instance's)")
+    ap.add_argument("--no-proc", action="store_true", help="don't look at XRService's process")
+    ap.add_argument("--no-ring", action="store_true", help="don't look at ft-camd's ring")
+    ap.add_argument("--ring", help="ft-camd's ring (default /run/user/UID/frametop-hands/cam-ring)")
+    a = ap.parse_args(argv)
+    r = check(log=a.log, proc=not a.no_proc, ring=not a.no_ring, ring_path=a.ring)
+    if a.json:
+        print(json.dumps(r, indent=1))
+    else:
+        print(r["summary"])
+        for line in r["evidence"]:
+            print("  " + line)
+    return {"ok": 0, "degraded": 1}.get(r["status"], 2)
+
+
+if __name__ == "__main__":
+    sys.exit(main())

hands/camd/camd.c

@@ -379,9 +379,15 @@ static bool resolve_block(cam_t *c)
     return true;
 }
 
+/*
+ * Through the ISP (no colour module), the near-black exposures come out all zeros,
+ * the same every time, so their dequeues change no buffer and get no votes. Such
+ * an index is left unmapped (slot -1): on_frame counts its frames as dark without
+ * reading them. Most of a camera's indices silent means it isn't streaming yet.
+ */
 static bool resolve_each(cam_t *c)
 {
-    int depth = c->maxindex + 1;
+    int depth = c->maxindex + 1, silent = 0;
     bool taken[MAX_SLOTS] = { false };
 
     for (int i = 0; i < depth; i++) {
@@ -396,6 +402,12 @@ static bool resolve_each(cam_t *c)
             }
         }
 
+        if (c->nobs[i] >= 3 && v1 <= 0.2 * c->nobs[i]) {
+            c->slot_of[i] = -1;
+            silent++;
+            continue;
+        }
+
         if (c->nobs[i] < 3 || best < 0 || taken[best] || v1 < 0.8 * c->nobs[i] || v2 > 0.3 * c->nobs[i])
             return false;
 
@@ -403,9 +415,14 @@ static bool resolve_each(cam_t *c)
         c->slot_of[i] = best;
     }
 
+    if (silent * 2 > depth)
+        return false;
+
     printf("%s: queue depth %d, buffers mapped one by one:", c->slug, depth);
     for (int i = 0; i < depth; i++)
-        printf(" %d", c->slot_of[i]);
+        c->slot_of[i] < 0 ? printf(" -") : printf(" %d", c->slot_of[i]);
+    if (silent)
+        printf(" (- : %d indices whose frames are all zeros, skipped as dark)", silent);
     printf("\n");
     return true;
 }
@@ -677,6 +694,12 @@ static void on_frame(cam_t *c, int64_t index, uint32_t seq, uint64_t ts, uint64_
 
     int slot = c->slot_of[index];
 
+    if (slot < 0) {     /* an index whose frames are all zeros (resolve_each): a dark one */
+        c->dark++;
+        c->rc->dropped++;
+        return;
+    }
+
     /* color runs at 60 fps: skip frames early enough that the asked rate holds, before any sync */
     if (c->color && color_fps > 0 && evtime - c->last_pub_ns < (uint64_t)(1e9 / color_fps) - 3000000) {
         c->paced++;

hands/camd/xrcams.c

@@ -544,22 +544,26 @@ static void probe_cameras(xr_state_t *st)
 /*
  * qcom-camss can report bytesperline as the visible width while the VFE
  * writes a larger aligned pitch. sizeimage is right, so derive the pitch.
+ * For NV12, plane 0 normally holds the chroma rows after the luma (the side
+ * cameras through the ISP: 1056 wide, 1152 bytes a row); if it's too small for
+ * that, it holds the luma alone.
  */
 unsigned xr_camera_stride(const xr_camera_t *c)
 {
     if (!c->height || !c->planesize[0])
         return c->bytesperline ? c->bytesperline : c->width;
 
-    double bpp = 1.0;
-
-    if (c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21)
-        bpp = 1.5;
-
-    unsigned s = (unsigned)((double)c->planesize[0] / ((double)c->height * bpp));
+    bool yuv = c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21;
+    unsigned s = (unsigned)((double)c->planesize[0] / ((double)c->height * (yuv ? 1.5 : 1.0)));
 
     if (s >= c->width && s <= c->width * 4)
         return s;
 
+    s = (unsigned)(c->planesize[0] / c->height);
+
+    if (yuv && s >= c->width && s <= c->width * 4)
+        return s;
+
     return c->bytesperline ? c->bytesperline : c->width;
 }
 
@@ -591,7 +595,20 @@ void xr_camera_layout(const xr_camera_t *c, xr_layout_t *l)
     l->pitch = xr_camera_stride(c);
     l->width = c->width < l->pitch ? c->width : l->pitch;
 
-    if (c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21) {
+    /*
+     * Without the colour module, XRService runs the side cameras through the
+     * ISP ("ISP enabled for tracking cameras (main VFE available)" in its log),
+     * and they come out NV12. They're mono sensors, so the luma is the image.
+     */
+    bool yuv = c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21;
+
+    if (yuv && c->role && !strcmp(c->role, "tracking")) {
+        l->fmt  = XR_FMT_GREY8;
+        l->rows = c->height;
+        return;
+    }
+
+    if (yuv) {
         l->fmt  = XR_FMT_NV12;
         l->rows = c->height + c->height / 2;
     } else {

hands/frametop-camwatch.service

@@ -0,0 +1,21 @@
+# Template: the installer replaces @REPO@ with the repo path on the Frame. Not installed or
+# enabled by anything yet (hands/README.md, "Camera check").
+[Unit]
+Description=Frametop camera watch: tells you when SteamVR leaves the headset's upper cameras off
+Documentation=file://@REPO@/hands/README.md
+# Not PartOf=steamvr.service: it has to outlive the SteamVR restart it may ask for.
+
+[Service]
+# On the host, system Python, standard library only. It stats the XRService log every 2 s and
+# reads only what's new. Notifications go to the Frametop desktop's own D-Bus (found through its
+# plasmashell). With CAMWATCH_AUTO_RESTART=1 in ~/.config/frametop.conf it may restart SteamVR
+# while the headset isn't worn: that also closes the Frametop desktop.
+ExecStart=/usr/bin/python3 @REPO@/hands/ft-camwatch
+Restart=on-failure
+RestartSec=30
+Nice=10
+CPUQuota=5%
+MemoryMax=64M
+
+[Install]
+WantedBy=default.target

hands/ft-camwatch

@@ -0,0 +1,378 @@
+#!/usr/bin/env python3
+"""ft-camwatch: watches SteamVR's XRService log for the camera failure that turns off the
+headset's upper cameras and IR light (a VCINT FPGA load that fails, usually after a wake;
+hands/camcheck.py), tells the Frametop desktop, and can restart SteamVR by itself.
+
+Off by default: hands/frametop-camwatch.service is a template the installer doesn't enable.
+Settings in ~/.config/frametop.conf:
+  CAMWATCH_AUTO_RESTART=1    restart SteamVR on a failure (default 0: only a notification). Only
+                             while the headset isn't worn (frame-job's check: vrcompositor runs and
+                             a backlight is on), after it has been off for CAMWATCH_IDLE_S, with no
+                             Steam-launched app running and nobody on the remote desktop (VNC).
+                             At most once per failure, and not again within CAMWATCH_COOLDOWN_MIN.
+  CAMWATCH_IDLE_S=60         how long the headset must be off first
+  CAMWATCH_COOLDOWN_MIN=30   the least time between two automatic restarts
+  CAMWATCH_IGNORE_APPIDS=    Steam app ids that don't count as a running VR app (comma-separated)
+  CAMWATCH_NOTIFY=1          post a notification in the Frametop desktop (0: log only)
+
+A SteamVR restart also closes the Frametop desktop and every window in it (ft-screens quits
+with SteamVR, and the desktop's unit doesn't restart: see hands/README.md, "Camera check").
+
+It follows the log with a stat every 2 s (no inotify, nothing else while all is well), and
+logs every decision to stdout (the journal). --dry-run never notifies or restarts; --once
+prints the state and what it would do now, and exits.
+"""
+import argparse
+import json
+import os
+import subprocess
+import sys
+import time
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+import camcheck  # noqa: E402
+
+CONF = os.path.expanduser("~/.config/frametop.conf")
+STATE = os.path.expanduser("~/.local/state/frametop/camwatch.json")
+BACKLIGHTS = "/sys/class/backlight"
+POLL_S = 2.0
+MAX_READ = 16 << 20     # a poll reads at most this much of a log that grew
+NOTIFY_TITLE = "Hand tracking: cameras off"
+NOTIFY_TEXT = ("The headset's upper cameras and IR light are off: SteamVR couldn't start the colour camera "
+               "module (it happens sometimes after the headset sleeps). Restart SteamVR, or the headset if "
+               "that doesn't fix it. Restarting SteamVR closes this desktop and its windows.")
+DEFAULTS = {"CAMWATCH_AUTO_RESTART": "0", "CAMWATCH_IDLE_S": "60", "CAMWATCH_COOLDOWN_MIN": "30",
+            "CAMWATCH_IGNORE_APPIDS": "", "CAMWATCH_NOTIFY": "1", "VNC_PORT": "5900"}
+
+
+def log(text):
+    print("%s %s" % (time.strftime("%H:%M:%S"), text), flush=True)
+
+
+def read_conf(path=CONF):
+    """KEY=VALUE lines of a shell-style file (comments and quotes stripped), over DEFAULTS."""
+    conf = dict(DEFAULTS)
+    try:
+        with open(path) as f:
+            for line in f:
+                line = line.split("#", 1)[0].strip()
+                if "=" not in line:
+                    continue
+                k, v = line.split("=", 1)
+                k, v = k.strip(), v.strip().strip("'\"")
+                if k.replace("_", "").isalnum():
+                    conf[k] = v
+    except OSError:
+        pass
+    return conf
+
+
+def conf_int(conf, key):
+    try:
+        return int(float(conf.get(key, DEFAULTS.get(key, "0"))))
+    except ValueError:
+        return int(DEFAULTS.get(key, "0") or 0)
+
+
+# ------------------------------------------------------------------------------------------
+# What's going on around (all read-only, from /proc and /sys)
+
+def process_running(name):
+    for pid in os.listdir("/proc"):
+        if pid.isdigit():
+            try:
+                with open("/proc/%s/comm" % pid) as f:
+                    if f.read().strip() == name:
+                        return True
+            except OSError:
+                pass
+    return False
+
+
+def headset_worn(backlights=BACKLIGHTS):
+    """frame-job's check: vrcompositor runs and any panel's backlight is on (SteamVR turns the
+    panels off 5 s after the headset comes off)."""
+    lit = False
+    try:
+        for name in os.listdir(backlights):
+            try:
+                with open(os.path.join(backlights, name, "brightness")) as f:
+                    lit = lit or int(f.read().strip()) > 0
+            except (OSError, ValueError):
+                pass
+    except OSError:
+        return False
+    return lit and process_running("vrcompositor")
+
+
+def vr_apps(ignore=()):
+    """Apps Steam launched ("SteamLaunch AppId=N" in a process's arguments, as Steam's reaper
+    runs them): ["AppId=N ..."]. Steam's own processes, SteamVR's and Frametop's services aren't
+    launched this way. A guess: no VR game has been run on the Frame to confirm the form."""
+    out = set()
+    for pid in os.listdir("/proc"):
+        if not pid.isdigit():
+            continue
+        argv = camcheck.proc_argv(pid)
+        if "SteamLaunch" not in argv:
+            continue
+        ids = [a.split("=", 1)[1] for a in argv if a.startswith("AppId=")]
+        if ids and ids[0] not in ignore:
+            out.add("AppId=" + ids[0])
+    return sorted(out)
+
+
+def remote_viewers(port=5900, tables=("/proc/net/tcp", "/proc/net/tcp6")):
+    """Established connections to the remote desktop's VNC port, from /proc/net/tcp(6)."""
+    out = []
+    for path in tables:
+        try:
+            with open(path) as f:
+                next(f)
+                for line in f:
+                    p = line.split()
+                    if len(p) > 3 and p[3] == "01" and int(p[1].rsplit(":", 1)[1], 16) == port:
+                        out.append(p[2])
+        except (OSError, StopIteration, ValueError):
+            pass
+    return out
+
+
+def frametop_bus():
+    """The Frametop desktop's D-Bus (from its plasmashell, as decoration/apply.sh finds it), or None."""
+    for pid in os.listdir("/proc"):
+        if not pid.isdigit():
+            continue
+        try:
+            with open("/proc/%s/comm" % pid) as f:
+                if f.read().strip() != "plasmashell":
+                    continue
+            with open("/proc/%s/environ" % pid, "rb") as f:
+                env = dict(kv.split(b"=", 1) for kv in f.read().split(b"\0") if b"=" in kv)
+        except OSError:
+            continue
+        if env.get(b"XDG_RUNTIME_DIR", b"").endswith(b"/frametop") and b"DBUS_SESSION_BUS_ADDRESS" in env:
+            return env[b"DBUS_SESSION_BUS_ADDRESS"].decode()
+    return None
+
+
+def notify(title, text):
+    bus = frametop_bus()
+    if not bus:
+        log("no Frametop desktop running: no notification")
+        return False
+    r = subprocess.run(["notify-send", "-a", "Frametop", "-u", "critical", "-i", "dialog-warning", title, text],
+                       env=dict(os.environ, DBUS_SESSION_BUS_ADDRESS=bus), capture_output=True, text=True,
+                       timeout=10)
+    if r.returncode:
+        log("notify-send failed (%d): %s" % (r.returncode, r.stderr.strip()))
+    return r.returncode == 0
+
+
+def restart_steamvr():
+    # --no-block: the job runs in systemd; the Frametop desktop closing doesn't cut it short.
+    r = subprocess.run(["systemctl", "--user", "restart", "--no-block", "steamvr.service"],
+                       capture_output=True, text=True, timeout=30)
+    log("systemctl --user restart steamvr.service: exit %d %s" % (r.returncode, (r.stderr or "").strip()))
+    return r.returncode == 0
+
+
+def environment(conf):
+    ignore = tuple(a.strip() for a in conf.get("CAMWATCH_IGNORE_APPIDS", "").replace(",", " ").split() if a.strip())
+    return {"steamvr": camcheck.xrservice_pid() is not None, "worn": headset_worn(),
+            "vr_apps": vr_apps(ignore), "remote": remote_viewers(conf_int(conf, "VNC_PORT"))}
+
+
+# ------------------------------------------------------------------------------------------
+# The decision (pure: tests feed it made-up states)
+
+def new_memory():
+    return {"pending": "", "notified": [], "restarted": {}, "last_restart": 0.0, "idle_since": None, "why": ""}
+
+
+def decide(now, failure, env, mem, conf):
+    """What to do now. failure: the current VCINT failure's id ("" if none: the cameras run,
+    or they're closed); env: {"steamvr", "worn", "vr_apps", "remote"} (only looked at while a
+    failure is current); mem: new_memory(), updated in place; now: wall-clock seconds.
+    Returns [("log", text) | ("notify", failure) | ("restart", failure)]."""
+    acts = []
+    if not failure:
+        if mem["pending"]:
+            acts.append(("log", "failure %s is no longer current" % mem["pending"]))
+        mem.update(pending="", idle_since=None, why="")
+        return acts
+    if mem["pending"] != failure:
+        mem.update(pending=failure, why="")
+        acts.append(("log", "VCINT failure: %s (upper cameras and IR light off)" % failure))
+    if failure not in mem["notified"]:
+        mem["notified"] = (mem["notified"] + [failure])[-20:]
+        if conf.get("CAMWATCH_NOTIFY", "1") != "0":
+            acts.append(("notify", failure))
+    if env.get("worn"):
+        mem["idle_since"] = None
+    elif mem["idle_since"] is None:
+        mem["idle_since"] = now
+    idle_s = conf_int(conf, "CAMWATCH_IDLE_S")
+    cooldown = conf_int(conf, "CAMWATCH_COOLDOWN_MIN") * 60
+    if conf.get("CAMWATCH_AUTO_RESTART", "0") != "1":
+        why = "no automatic restart (CAMWATCH_AUTO_RESTART=1 in ~/.config/frametop.conf turns it on)"
+    elif failure in mem["restarted"]:
+        why = "SteamVR was restarted once for this failure already: restart the headset"
+    elif mem["last_restart"] and now - mem["last_restart"] < cooldown:
+        why = "no restart: the last automatic one was %d min ago (cooldown %d min)" % (
+            (now - mem["last_restart"]) // 60, cooldown // 60)
+    elif not env.get("steamvr"):
+        why = "no restart: SteamVR isn't running"
+    elif env.get("worn"):
+        why = "no restart while the headset is worn"
+    elif env.get("vr_apps"):
+        why = "no restart: a VR app is running (%s)" % ", ".join(env["vr_apps"])
+    elif env.get("remote"):
+        why = "no restart: someone is on the remote desktop (%s)" % ", ".join(env["remote"])
+    elif now - mem["idle_since"] < idle_s:
+        why = "waiting for the headset to stay off for %d s" % idle_s
+    else:
+        mem["restarted"][failure] = now
+        mem["last_restart"] = now
+        why = "restarting SteamVR (the headset is off, nothing else in VR)"
+        acts.append(("log", why))
+        acts.append(("restart", failure))
+        mem["why"] = why
+        return acts
+    if why != mem["why"]:
+        mem["why"] = why
+        acts.append(("log", why))
+    return acts
+
+
+def load_memory(path=STATE):
+    mem = new_memory()
+    try:
+        with open(path) as f:
+            saved = json.load(f)
+        mem["notified"] = list(saved.get("notified", []))[-20:]
+        mem["restarted"] = dict(saved.get("restarted", {}))
+        mem["last_restart"] = float(saved.get("last_restart", 0.0))
+    except (OSError, ValueError, TypeError, AttributeError):
+        pass
+    return mem
+
+
+def save_memory(mem, path=STATE):
+    os.makedirs(os.path.dirname(path), exist_ok=True)
+    tmp = path + ".tmp"
+    with open(tmp, "w") as f:
+        json.dump({"notified": mem["notified"], "restarted": mem["restarted"],
+                   "last_restart": mem["last_restart"]}, f)
+    os.replace(tmp, path)
+
+
+# ------------------------------------------------------------------------------------------
+# Following the log
+
+class Follower:
+    """The running XRService's log, read as it grows. A new log (SteamVR restarted) starts afresh."""
+
+    def __init__(self, path=None):
+        self.fixed = path
+        self.path, self.ino, self.offset, self.rest = "", None, 0, ""
+        self.state = None
+
+    def poll(self):
+        path = self.fixed or camcheck.newest_log()
+        if not path:
+            return None
+        try:
+            st = os.stat(path)
+        except OSError:
+            return self.state
+        if path != self.path or st.st_ino != self.ino or st.st_size < self.offset:
+            self.path, self.ino, self.offset, self.rest = path, st.st_ino, 0, ""
+            self.state = camcheck.LogState(path)
+            log("following %s" % path)
+        if st.st_size > self.offset:
+            with open(path, "rb") as f:
+                f.seek(self.offset)
+                data = f.read(MAX_READ)
+            self.offset += len(data)
+            text = self.rest + data.decode(errors="replace")
+            lines = text.split("\n")
+            self.rest = lines.pop()
+            for line in lines:
+                self.state.feed(line)
+        return self.state
+
+
+def current_failure(state, steamvr_running=True):
+    """The failure id if the log's current camera start is a VCINT failure, else ""."""
+    if state is None or not steamvr_running:
+        return ""
+    snap = state.snapshot()
+    return snap["failure"] if snap["status"] == "degraded" and snap["reason"] == camcheck.VCINT_REASON else ""
+
+
+def act(acts, dry):
+    changed = False
+    for kind, arg in acts:
+        if kind == "log":
+            log(arg)
+        elif kind == "notify":
+            changed = True
+            if dry:
+                log("(dry run) would notify: %s" % NOTIFY_TITLE)
+            else:
+                notify(NOTIFY_TITLE, NOTIFY_TEXT)
+        elif kind == "restart":
+            changed = True
+            if dry:
+                log("(dry run) would restart SteamVR")
+            else:
+                restart_steamvr()
+    return changed
+
+
+def main(argv=None):
+    ap = argparse.ArgumentParser(description="Watch for the camera failure that turns off the upper cameras.")
+    ap.add_argument("--dry-run", action="store_true", help="log what it would do; never notify or restart")
+    ap.add_argument("--once", action="store_true", help="look once, print the decision (a dry run), and exit")
+    ap.add_argument("--log", help="follow this file instead of the running XRService's log (tests)")
+    ap.add_argument("--state", default=STATE, help="where it remembers past failures (default %(default)s)")
+    a = ap.parse_args(argv)
+    dry = a.dry_run or a.once
+    mem = load_memory(a.state)
+    follower = Follower(a.log)
+    conf = read_conf()
+    conf_mtime = 0.0
+    log("watching; automatic restart %s" % ("on" if conf.get("CAMWATCH_AUTO_RESTART") == "1" else "off"))
+    while True:
+        try:
+            m = os.stat(CONF).st_mtime
+        except OSError:
+            m = 0.0
+        if m != conf_mtime:
+            conf, conf_mtime = read_conf(), m
+        state = follower.poll()
+        failure = current_failure(state)
+        env = {}
+        if failure or mem["pending"]:
+            env = environment(conf)
+            if not env["steamvr"] and not a.log:
+                failure = ""   # the log's last word, but XRService is gone
+        acts = decide(time.time(), failure, env, mem, conf)
+        if a.once:
+            snap = state.snapshot() if state else {"status": "unknown", "reason": "no log"}
+            print("log: %s\nstate: %s %s\nenvironment: %s" % (follower.path, snap["status"], snap["reason"],
+                                                              json.dumps(env)))
+            act(acts, True)
+            return 0
+        if act(acts, dry) and not dry:
+            save_memory(mem, a.state)
+        time.sleep(POLL_S)
+
+
+if __name__ == "__main__":
+    try:
+        sys.exit(main())
+    except KeyboardInterrupt:
+        sys.exit(0)

hands/ft-cutouts

@@ -0,0 +1,57 @@
+#!/usr/bin/env bash
+# ft-cutouts: the hand cutouts without pinches and grips. Your hands show through Frametop's
+# screens, but ft-hands runs with --no-gestures, so nothing clicks or drags. ft-handsctl on is
+# the full version; each stops the other. Like it, this doesn't start with SteamVR, and it
+# stops when SteamVR does. On the Frame.
+#
+#   ft-cutouts on | off | status
+#
+# Runs this checkout's hands/build as transient user units, so it doesn't need hands/run.sh
+# install: only a build (hands/build.sh) and ft-camd's capabilities (hands/run.sh caps).
+set -euo pipefail
+here=$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)
+camd=frametop-cutouts-camd.service hands=frametop-cutouts-hands.service
+
+die() { echo "$*" >&2; exit 1; }
+
+run_unit() {  # unit, the ft-handsctl unit it replaces, description, then the command
+  local unit=$1 conflicts=$2 what=$3
+  shift 3
+  systemctl --user -q is-active "$unit" && return
+  systemd-run --user --quiet --collect --unit="$unit" --description="Frametop hand cutouts: $what" \
+    -p PartOf=steamvr.service -p After=steamvr.service -p Conflicts="$conflicts" \
+    -p Restart=always -p RestartSec=5 -p TimeoutStopSec=5 "$@"
+}
+
+case ${1:-status} in
+  on)
+    systemctl --user -q is-active steamvr.service || die "SteamVR isn't running"
+    [ -x "$here/build/ft-camd" ] && [ -x "$here/build/ft-hands" ] || die "not built: hands/build.sh"
+    grep -qa -- --no-gestures "$here/build/ft-hands" || die "ft-hands predates --no-gestures: hands/build.sh"
+    getcap "$here/build/ft-camd" | grep -q cap_sys_ptrace ||
+      die "ft-camd needs its capabilities: hands/run.sh caps (asks for sudo)"
+    if systemctl --user -q is-active frametop-camd.service frametop-hands.service; then
+      echo "stopping ft-handsctl's hand tracking (it has pinches and grips)"
+    fi
+    run_unit $camd frametop-camd.service "the camera broker" "$here/build/ft-camd" --status 60
+    if ! systemctl --user -q is-active $hands; then
+      "$here/../scripts/container-up.sh"  # so stopping the unit can't take the container down
+      pkill -x ft-hands || true
+    fi
+    run_unit $hands frametop-hands.service "hand tracking, no gestures" "$HOME/.local/bin/distrobox" enter dev -- \
+      "$here/build/ft-hands" --status 60 --no-gestures
+    "$here/ft-handsctl" cutouts on >/dev/null || true
+    sleep 4
+    "$0" status ;;
+  off)
+    systemctl --user stop $hands $camd 2>/dev/null || true
+    systemctl --user -q is-active frametop-hands.service || pkill -x ft-hands || true
+    echo "hand cutouts off" ;;
+  status)
+    for u in $camd $hands; do echo "$u: $(systemctl --user is-active $u || true)"; done
+    echo "ft-screens cutouts: $("$here/ft-handsctl" cutouts state 2>&1)"
+    inv=$(systemctl --user show -p InvocationID --value $hands)  # this run's lines only
+    [ -n "$inv" ] && journalctl --user _SYSTEMD_INVOCATION_ID="$inv" --no-pager -o cat |
+      grep -E '^ *[0-9.]+s |sets with a hand|cameras:' | tail -2 | cut -c1-160 || true ;;
+  *) sed -n '2,10p' "$0" | sed 's/^# \{0,1\}//'; exit 2 ;;
+esac

hands/rec/CONSENT.md

@@ -0,0 +1,97 @@
+# Recording your hands for the Frametop hand dataset
+
+Version: 2026-10-03
+
+Frametop's hand tracking needs a small model that finds hands in the headset's camera images. To train it, we're collecting recordings of many people's hands into a public dataset. This page explains what the hand recorder records, where it goes, and what you agree to if you take part. Please read all of it.
+
+## Who runs this
+
+Frametop is an independent open-source project, maintained by DeeJanuz (<https://github.com/DeeJanuz>). It isn't affiliated with or endorsed by Valve or Hugging Face. Steam and Steam Frame are trademarks of Valve Corporation. This text was written without a lawyer, in good faith; if something in it seems wrong or unclear, please say so before you take part.
+
+You can reach the maintainer through the Frametop repository's issues (<https://github.com/DeeJanuz/frametop/issues>) or the dataset's discussion page on Hugging Face. Both are public.
+
+## Who can take part
+
+- You must be **18 or older**.
+- For now, you can't take part if you live in **Illinois, Texas or Washington** (USA). Those states have laws about biometric data (such as hand geometry) that need more than this project can provide yet.
+- Take part only if you're comfortable with images of your hands and the room in front of you being public.
+
+## What is recorded
+
+While a session runs, the hand recorder saves:
+
+- **Camera images.** Infrared images from the headset's 4 tracking cameras, about 10 sets a second. They show your hands, your arms, your clothing and whatever is in front of you: your room, your desk and the things on it. They're grey, low-detail images, but people and things can be recognised in them. The size and shape of your hands can be measured from them: that's part of what they're for.
+- **Motion.** The position and rotation of the headset, many times a second, and of the controllers when you use them in the recording.
+- **Prompts.** What you were asked to do and when, and what the live hand tracker saw at the time.
+- **Calibration.** Where the cameras sit on the headset and how their lenses bend the image. Serial numbers and other fields that identify your headset are removed first, and the export lists what was removed.
+- **Your answers.** Which objects you had, the lighting, whether you wore sleeves, rings or a watch, your handedness if you gave it, and any notes you typed in the checklist or in Review. Notes are uploaded with the recordings and read by the maintainer: don't put anything in them that identifies you.
+- **Times.** When each session was recorded, with your time zone.
+- **A contributor id.** A random number made on your headset the first time you agree to this page. It isn't made from your name, your Steam account or your headset. It keeps your sessions together and lets you withdraw them.
+
+The recorder doesn't record sound, your name, your email address or your Steam account. The tracking cameras look outward, so your eyes and face aren't recorded, unless a mirror or something shiny shows them: avoid those.
+
+**Your Hugging Face account.** You upload with your own Hugging Face account, and your pull request shows its username next to your contributor id, publicly. So anyone can see which Hugging Face account contributed which recordings. Use an account you're happy to have linked to them.
+
+## What it's used for
+
+- Training and testing hand-tracking models: finding hands, their keypoints, their shape and how far away they are. Mainly for Frametop's hand cutouts, and by anyone else for non-commercial work under the dataset's license.
+- It isn't used to recognise or identify people, and the dataset's terms forbid anyone from trying.
+
+## Nothing leaves your headset unless you send it
+
+- Recordings stay on your headset, in `~/.local/share/frametop/hands/contrib`. Nothing is uploaded automatically.
+- Before you share anything, you can watch every recording in the Review page. You can delete any stretch of a recording, a whole take or a whole session. Export leaves out what you deleted.
+- You upload the export yourself, from the Upload page. That opens a pull request on the dataset. The maintainer checks it before it becomes part of the dataset, and may decline it. Until it's merged, you can close the pull request yourself.
+
+## Where it goes
+
+- **The dataset is public.** It's hosted on Hugging Face (<https://huggingface.co/datasets/DeeJanuz/frametop-hands>), whose servers may be outside your country, for example in the USA. Anyone who accepts the dataset's terms can download it.
+- People who download it agree not to try to identify anyone or anything in it, and to delete recordings that are later withdrawn. We can't enforce that against everyone: assume copies may exist.
+- Recordings stay in the dataset until they're withdrawn or the dataset is taken down.
+
+## Keep other people and private things out of view
+
+While recording, please:
+
+- face away from other people, mirrors, screens showing private things, papers, letters, photos and anything else you wouldn't want public;
+- make sure nobody else's face or hands are in view, and record only where the people you share the space with are fine with it;
+- don't record children.
+
+If something private got into a recording, delete that stretch in Review before you export. If you notice it after uploading, withdraw the session (below).
+
+## Safety
+
+The sessions ask you to move your hands and arms around you, out to full reach. Sit where you normally use the headset, clear an arm's reach around you, and stop whenever anything is uncomfortable. You take part at your own risk.
+
+## The license
+
+- **The dataset is published under Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0).** Anyone may use it for non-commercial purposes, with attribution. Your contribution is credited by its contributor id, not your name.
+- **You also give the maintainer of Frametop a non-exclusive, worldwide, royalty-free license to use your contribution for any purpose, including commercially.** That includes copying it, changing it, and training, publishing and selling models made from it, in Frametop and elsewhere. The maintainer today is DeeJanuz. If someone else, or an organization, takes over maintaining Frametop, this license passes to them. It's non-exclusive: you keep any rights you have in your recordings and can do what you like with your own copies. It ends for a recording when you withdraw it, except for models already trained with it.
+- You confirm that you have the right to give these licenses: the recordings are yours, and nothing in them belongs to someone who hasn't agreed.
+- There is no payment. The dataset and the hand recorder come with no warranty, and as far as the law allows, the maintainer isn't liable for any loss or damage from taking part.
+
+## Withdrawing
+
+You can withdraw a contribution at any time, without giving a reason:
+
+- **Before it's merged:** close your pull request on Hugging Face. The maintainer deletes its files.
+- **After it's merged:** post on the dataset's discussion page, or in the Frametop repository's issues, with your contributor id (shown in the hand recorder) and which sessions, or "all". Post from the Hugging Face account that uploaded them if you can, so the maintainer can tell it's you; otherwise, say how to check. These requests are public, so don't add anything else that identifies you.
+
+Then, usually within 30 days:
+
+- your recordings are deleted from the dataset and purged from its repository's history, so they can't be downloaded from there again;
+- they're left out of anything trained after that.
+
+What can't be undone: copies others downloaded before the withdrawal, and models already trained with them.
+
+## Your rights
+
+Depending on where you live (for example in the EU or the UK, under the GDPR), the law may give you more rights over this data: to get a copy of it, to have it corrected or deleted, to object to its use, and to complain to your data protection authority. The data is used because you agreed to it, and you can withdraw that agreement at any time, as above. Withdrawing doesn't make earlier use unlawful. To use any of these rights, contact the maintainer as above.
+
+## Changes to this text
+
+If this text changes, the hand recorder shows the new version and asks you again before your next session. Each upload records the version you agreed to, and recordings you already uploaded stay under that version, unless you agree to a newer one or withdraw them.
+
+## Agreeing
+
+By ticking the three boxes and "Agree and continue", you confirm that you're 18 or older, that you don't live in Illinois, Texas or Washington, and that you agree to the above. You can still decide not to upload anything.

hands/rec/DESIGN.md

@@ -0,0 +1,366 @@
+# Hand recorder: design (Phase 1 of the hands plan)
+
+The hand recorder guides a person through recording their hands with the headset's cameras. It saves the recordings as files, lets them review and delete anything, then exports a package to contribute to the open hand dataset. Recordings never leave the headset unless the person uploads them.
+
+The plan this belongs to is `~/Desktop/Projects/frame-hands/notes/hands-plan.md` (on the maintainer's Frame). In short, the dataset trains a small hand model for Frametop's hand cutouts.
+
+## Parts
+
+| Part | What it does |
+|---|---|
+| `hands/rec/panel/ft-handpanel.cpp` (C++, dev container) | The headset panel: a SteamVR overlay fixed to the head that shows the prompts. It also places the "touch the dot" target in the room and logs head and controller poses. Driven over `@ft_handpanel`. |
+| `hands/rec/session.py` (Python, no Qt) | The session runner. It reads `script.json`, starts and stops the recordings, and drives the panel. It reads the live hands file for feedback and writes each take's files. It also runs from the command line (`--dry-run`) for testing. |
+| `hands/rec/script.json` | The guided script: sections, prompts, timings. |
+| `hands/rec/poses/` | The pose pictures, `poses.json` and its PNGs (below). |
+| `hands/rec/takes.py` (Python, no Qt) | Reads sessions and takes from disk: frame sets for review, deleted ranges, export (compress, strip, manifest, checksums). |
+| `hands/rec/ft_handrec.py` + `main.qml` (PySide6 + Kirigami, dev container) | The desktop window: consent, the before-you-start checklist, the session controls, review, export and upload instructions. |
+| `hands/rec/ft-handrec` | The host launcher, like `input-settings/ft-input-settings`. |
+| `hands/rec/build.sh` | Builds `ft-handpanel` into `hands/rec/build/`, like `gaze/build.sh`. |
+| `hands/rec/CONSENT.md`, `hands/rec/UPLOAD.md` | The texts the window shows. |
+| `hands/rec/install.sh` | Installs the recorder on a Frame with Frametop: the dev container's packages, hands/build.sh, the panel, ft-camd's capabilities, and the menu entry (`uninstall` removes the entry). |
+| ft-hands `--record-hz N` (done) | Records at most N frame sets a second. The recorder uses 10. |
+| `hands/camcheck.py` (shared, standard library) | Are all four mono cameras running? The recorder runs it before a session and when a step sees no hands (below; `hands/README.md`, "Camera check"). |
+
+Every part runs in the dev container, as ft-hands and Input Settings do. The host Python isn't used: it lacks PySide6 and zstd. `setup/dev-container.sh` gains `zstd`.
+
+## Processes during a session
+
+- **ft-camd** publishes the camera ring. If it isn't running, the session starts it as the transient user unit `frametop-handrec-camd.service`, the way `hands/ft-cutouts` starts `frametop-cutouts-camd.service` (needs `hands/build/ft-camd` with capabilities: `hands/run.sh caps`). An ft-camd already running from ft-cutouts or ft-handsctl is used as it is.
+- **A tracking ft-hands** gives feedback through the hands file: which hands are seen, the palm's distance, the index tip. If none is running, the session starts `ft-hands --no-gestures --status 0` (unit `frametop-handrec-hands.service`). An ft-hands already running is used as it is.
+- **A recording ft-hands** runs once per recording part: `ft-hands --record-only --record DIR --record-for SECONDS --record-hz 10 --status 0 --sides auto|0|1` (below, "Side cameras"). It runs as a plain child process of the session, ended with SIGTERM when the part ends. SIGTERM ends ft-hands' loop, and `Recorder` writes out its queue when it's destroyed. In step mode (below) a part is one step's countdown and hold, so a take has one part per step (about 40 in the hand poses); in auto mode a take is one part, plus one more after each pause. `--record-for` is only a safety net.
+  - Why a process per part rather than one kept alive and paused: measured in the dev container with `ft-ringplay`'s ring (2026-10-02), `ft-hands --record-only` writes its first set 16-27 ms after it starts and ends 4-6 ms after SIGTERM, so a new part costs nothing the 3 s countdown doesn't cover. Every reader already takes parts in order (`takes.py`, `validate.py` through the export's single stream, the labeller's `fhl_io.py`, numbering `sets-10.bin` after `sets-9.bin`), ft-hands needs no new control, and nothing is written while a step waits. Before the hold starts the session checks that the part has written a set (`Recorder.has_data`, up to 3 s more), so the hold is recorded from its first frame.
+- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. With `HANDS_SWAP_SIDES` forced to `0` or `1`, the hands still decide what's published once they disagree (state `"forced, disagrees"`; `read_live` also corrects an ft-hands built before that). The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand.
+- **ft-handpanel** runs as a child process with `--watch-stdin`. It shows the panel and logs poses during each take.
+- **The headset button's reader** is a thread of the session (`ButtonReader`, below), not a process.
+
+Test hooks:
+- `--ring PATH` goes to both ft-hands (`ft-ringplay` publishes a recording there, so the whole flow runs without the headset).
+- `--no-start` uses only what's already running.
+- `--dry-run` runs no processes and only prints the panel commands, with timing sped up by `--speed X`.
+- `--next-after S` presses Next by itself after S seconds of waiting (real time), so a step-mode session runs unattended. Lines on stdin steer it too: `n` or an empty line is Next, `p` pause or resume, `r` redo, `s` skip the section, `q` stop.
+- `--no-headset-button` leaves the headset's button alone (`ft-handrec --no-headset-button` too). `--button-device PATH` reads it from PATH instead, an event device or a FIFO of `input_event` structs, also in a dry run: a simulated button for tests.
+- `--auto` runs the timed flow; `--poses DIR` takes the pose pictures from DIR; `--plan` prints the sections, their steps and length.
+- `--ignore-cameras` (`ft-handrec --ignore-cameras` too) starts even when the camera check fails. A dry run and `--ring` skip the check by themselves.
+
+## Files
+
+```
+~/.local/share/frametop/hands/contrib/
+  profile.json                  consent and contributor id (below)
+  sessions/<YYYYMMDD-HHMMSS>/   (a second session started in the same second gets -2, and so on)
+    session.json                the session: checklist answers, lighting, versions, mode, takes
+    calibration.json            /persist/xrservice.json with identifying fields removed (below)
+    device.json                 the rig's pose in the CAD frame from /persist/device_config.json (below)
+    takes/<NN>-<section>/
+      sets.bin                  ft-hands' recording (FHSET01, hands/track/record.h), 10 sets/s: the first part
+      sets-2.bin, sets-3.bin, ...  the next parts (step mode: one per step; any mode: after a pause)
+      prompts.jsonl             what the person was asked to do, when (below)
+      poses.jsonl               head and controller poses from ft-handpanel (below)
+      take.json                 {"section", "title", "started_ns", "ended_ns", "status": "complete"|"stopped"|"skipped",
+                                 "deleted": [[from_ns, to_ns], ...], "notes": "",
+                                 "parts": {"sets.bin": {"names_swapped": false}, "sets-2.bin": {...}},
+                                 "clock": [[mono_ns, raw_minus_mono_ns], ...]}
+  exports/<session>/            what export writes (below)
+```
+
+All `_ns` times are CLOCK_MONOTONIC nanoseconds, the clock of `dqbuf_ns` in sets.bin and of `capture_ns` in the hands file. sets.bin's `capture_ns` is the camera clock (CLOCK_MONOTONIC_RAW). The two drift apart with NTP's corrections: on 2026-10-03 RAW ran 0.80 s ahead, gaining about 10 ppm. take.json `"clock"` samples the difference (RAW minus MONOTONIC, as ft-hands' `raw_minus_mono_ns()`) when each part starts and stops, so a set's exposure time on the poses' clock is `capture_ns - raw_minus_mono_ns`, interpolated between samples. `dqbuf_ns` is on the right clock already, but a few ms after the exposure. Takes recorded before 2026-10-03 have no `"clock"`.
+
+### profile.json
+
+```json
+{"schema": 1, "contributor": "<random uuid4>", "consent": {"version": "2026-10-02", "accepted": "<ISO time>", "adult": true},
+ "optional": {"handedness": "right|left|both|", "notes": ""}}
+```
+
+No name, email, or account. The contributor id is random, so several sessions from one person can be held out together in evaluation. Withdrawal also goes by that id.
+
+### session.json
+
+```json
+{"schema": 1, "tool": "ft-handrec <git describe>", "started": "<ISO>", "contributor": "<uuid>",
+ "lighting": {"chosen": "dim|room|daylight|indoor", "source": "measured|picked", "measured": "indoor|daylight|",
+              "ambient_ir": 0.0, "ring": {"<cam>": {"mean": 0.0, "dark_mean": 0.0}}},
+ "checklist": {"objects": ["pencil", "phone", "cup", "keyboard", "mouse", "gamepad", "small"], "own_objects": ["..."],
+               "controllers": "straps|none", "sleeves": "short|long|", "rings": false, "watch": false, "notes": ""},
+ "device": {"steamos": "<VERSION_ID from /etc/os-release>", "steamvr": "<version if known>", "cameras": [{"name", "width", "height"}]},
+ "mode": "step|auto", "quick": false, "shuffle": {"seed": 123, "sweeps": {"pose-sweeps": [{"id", "hands", "cues"}]}},
+ "takes": ["01-hand-size", "..."],
+ "camera": {"status": "ok|unknown|degraded", "reason": "..."}, "stop_reason": "...",
+ "sides": {"swapped": true|false|null, "decided_by": "auto|config|option|manual", "state": "decided|confirmed|...",
+           "evidence": {"as_named": 0, "swapped": 10, "seconds": 1.8, "miss_mm": [-1, 4.2], "probes": 30, "found": 10},
+           "decided_at": "<ISO>", "decided_ns": 0}}
+```
+
+`sides`: whether ft-camd's side camera names were backwards during the session (`swapped`), as the live tracker decided it (above, "Side cameras"); `null` while nobody knows. A part's names are right when its take.json `names_swapped` equals `swapped`. Parts without a `parts` entry were recorded with ft-camd's names. Sessions from before this have no `sides`.
+
+`camera` is the camera check's verdict at the start (no paths or log lines: those go to `session.log`). `stop_reason` is there when a session was stopped at the no-hands screen, with the check's result.
+
+A session started before step mode existed has no `mode`: it ran as `auto`. `quick` and `shuffle` (below, "Sweeps" and "Quick round") are missing from sessions before the sweeps.
+
+### calibration.json
+
+This is a copy of `/persist/xrservice.json` (`/run/host/persist/` in the container). Keep the cameras' intrinsics and extrinsics, and drop anything that identifies the unit: keys containing `serial`, `sn`, `uuid`, `mac` or `id`, or values that look like serial numbers. List what was removed in `session.json` (`"calibration_removed": [...]`), so a reviewer can check it.
+
+### device.json
+
+The head frame needs the rig's pose in the CAD frame, from `/persist/device_config.json`. Only two of its keys are kept, `cv.cad_from_cal` (Cam0 in the CAD frame) and `head` (the head in CAD), in the shape the labeller in frame-hands `train/label` reads, as its `cut.py` writes it:
+
+```json
+{"cv": {"cad_from_cal": {"method": "FrontAndUpperCamPositions", "plus_x": [x, y, z], "plus_z": [x, y, z], "position": [x, y, z]}},
+ "head": {"plus_x": [x, y, z], "plus_z": [x, y, z], "position": [x, y, z]}}
+```
+
+The rest of that file identifies the unit (serial number, display EDID) and is never copied. The two kept keys go through `strip_calibration` as well, and anything it removes is listed in `calibration_removed` as `device.json:<path>`. Sessions recorded before device.json existed have none: they still validate, with a warning, and the labeller falls back to another unit's pose.
+
+### prompts.jsonl
+
+One JSON object per line:
+
+```json
+{"t": 123, "event": "take", "section": "static-poses", "take": "03-static-poses"}
+{"t": 123, "event": "prompt", "id": "static-poses/fist/left/near", "text": "...", "hands": "left|right|both|none",
+ "pose": "fist", "distance": "near|mid|far|", "position": "centre|left|right|up|down|", "object": "", "controller": false}
+{"t": 123, "event": "target", "id": "touch/3", "head": [x, y, z], "room": [x, y, z], "state": "show|hold|done|timeout"}
+{"t": 123, "event": "bar", "target": 0.0}
+{"t": 123, "event": "feedback", "left": true, "right": false, "palm_m": [0.0, 0.0]}
+{"t": 123, "event": "pause"}
+{"t": 123, "event": "resume"}
+{"t": 123, "event": "ready", "id": "static-poses/fist/left/near", "seconds": 3}
+{"t": 123, "event": "wait"}
+{"t": 123, "event": "redo", "id": "static-poses/fist/left/near", "from": 123, "to": 123}
+{"t": 123, "event": "nohands", "id": "hand-size/flat/both", "reads": 120, "published": 118}
+{"t": 123, "event": "end", "status": "complete|stopped|skipped"}
+```
+
+`feedback` is written about twice a second while recording. It's the live tracker's view, kept for later checks; it's not a label.
+
+A prompt holds from its `prompt` until the next `prompt`, `ready`, `wait` or `end`. A step in step mode reads:
+
+```
+ready      Next pressed: recording part N starts, the 3-2-1 countdown runs (recorded, no label)
+prompt     the hold: its labels start here
+(bar, target, feedback, pause/resume during the hold)
+wait       the hold is over: no labels from here; part N stops
+```
+
+The touch-the-dot targets after the first follow straight on: no `ready` or `wait` between them. `redo` marks a try done again (R): `from` is that step's `ready` (or its `prompt` if it had none), `to` its end. Its prompt is skipped; the sets stay. In auto mode there's no `ready` or `wait`, and the intro is a recorded prompt `<section>/intro`. `nohands` marks the first hand-size step stopped because no hand was seen (below, "Camera check"); a `redo` over the same range follows it, so the try gets no labels. A sweep step (below, "Sweeps") has a `prompt` per cue, each with its `pose`, `"cue": true` and `"step"`, the step's id; its `ready`, `wait` and `redo` carry the step's id. Readers that knew only `prompt` and `end` keep working, but they'd give the countdown to the step before: `hub_review.py` (frame-hands `train/hub`) shows it as "(countdown)", redone prompts as "(redone)" and cues as "[pose] text"; `FORMAT.md` in the dataset repo has the rules.
+
+### poses.jsonl
+
+ft-handpanel writes one line per sample, 250 a second, from `GetDeviceToAbsoluteTrackingPose(TrackingUniverseStanding, 0)`:
+
+```json
+{"t": 123, "hmd": {"m": [12 floats, row-major 3x4], "r": 200, "ok": true},
+ "left": {"m": [...], "r": 200, "ok": true}, "right": null}
+```
+
+`r` is `ETrackingResult` (200 = Running_OK, 201 = Running_OutOfRange, and so on). `left` and `right` are the devices holding those controller roles, or null. No device serials are logged.
+
+## The panel (`ft-handpanel`)
+
+- **Placement.** A SteamVR overlay fixed to the head, like `gaze/panel/ft-gazepanel.cpp`: key `frametop.handpanel`, sort order 250. It sits 1.2 m ahead, centred 12 degrees above straight ahead, so the hands stay clear below it. It's 36 degrees wide, 4:3, 1024x768 pixels, dim and see-through. It's drawn on the CPU with stb_truetype (and stb_image for the pose pictures, PNG only, the same pinned stb commit) into three shared DMA-BUFs SteamVR imports once, as ft-gazepanel does, and is drawn again only when something changes.
+- **Layout.** The title and step on top (a red "Rec" by the step while recording), a rule. With a pose picture or a diagram, a 14-degree column on the left holds the picture (or the flipped copy and the picture side by side) and the where-to diagram under it; the text takes the right. The text column holds the instruction, the orange note, the big countdown ("3", "2", "1", then "Hold" or "Go") and the cyan action line ("Ready? Press Space or click Next"), centred together. At the bottom: the near/far bar, the hand chips, the time-left bar and the key hints.
+- **Socket.** Abstract unix datagram `@ft_handpanel` (`--socket NAME`). A sender with an address gets `ok ...` or `error ...`.
+- **Options:** `--watch-stdin` (quit when stdin closes), `--socket NAME`, `--distance M`, `--no-vr`. `--no-vr` makes no SteamVR connection and prints each picture's text to stdout: for testing without a headset.
+
+Commands (UTF-8; `|` starts a new line in text):
+
+| Command | Effect |
+|---|---|
+| `show` / `hide` | The panel. A "show" makes it visible with its first picture. |
+| `title <text>` | Big line at the top. |
+| `step <text>` | Small line at the top right, e.g. `Section 3 of 11`. |
+| `text <text>` | The instruction, large, wrapped to the panel's width, centred. |
+| `note <text>` | An orange line under the instruction: a warning ("I can't see your left hand"). Empty clears it. |
+| `countdown <0..1>` / `countdown off` | A thin bar along the bottom: the share of this prompt's time left. |
+| `hands <left> <right>` | Two chips, "Left hand" and "Right hand", each `seen` (green), `lost` (orange) or `off` (hidden). |
+| `bar <target 0..1> <current 0..1 or -1> [near label] [far label]` / `bar off` | The near/far bar for the push out and back: a horizontal track with a target marker and the hand's current position. |
+| `paused on` / `paused off` | A "Paused" overlay over the picture. |
+| `image <path> [mirror\|both]` / `image off` | The pose picture, a PNG (below), in the left column. `mirror` flips it (a left hand); `both` draws a flipped copy on its left. A file that can't be read: `error ...` and no picture. |
+| `where <position\|-> <distance\|->` / `where off` | The where-to diagram under the picture: a 3x3 front view with the asked cell lit (`centre`, `left`, `right`, `up`, `down`, and the push sections' `chest`, `desk`, `eye`), and a side view of the head and three marks for `near` ("Close"), `mid` ("Halfway out") and `far` ("Arm out"). `-` leaves that half out. |
+| `action <text>` | The cyan line under the instruction (empty clears it). |
+| `big <text>` | Large cyan text under the instruction: the countdown (empty clears it). |
+| `keys <text>` | The faint key hints along the bottom. |
+| `rec on` / `rec off` | The red "Rec" by the step line. |
+| `strip <cue> <path>\|<mode>\|<label>;...` / `strip off` | A sweep's row of pose pictures (path `-`: none; mode `-` or `mirror`), each with its label under it; the one at index `cue` (from 0, -1 for none) framed in cyan and bright, the others dim. It sits along the bottom of the space left, the where-to diagram at its right end if there is one, and the text above it; it takes the left column's place. Each picture is loaded once. A file that can't be read: `error can't read ...`, and that item shows an empty frame. |
+| `target <x> <y> <z> [show\|hold <0..1>\|done]` / `target off` | The touch target: a small sphere-like dot about 2 cm across, in its own overlay (`frametop.handpanel.target`). The point is in the head frame (metres, +x right, +y up, -z forward). The first `target` with a new point places it in the room with the current HMD pose, and it stays there. Later commands with the same point change only the state. `hold` draws a filling ring, `done` turns it green. Reply: `ok <room x> <room y> <room z>`. |
+| `poses start <path>` / `poses stop` | Log poses to the path (appending, `poses.jsonl` format above) from a thread at 250 Hz, until stopped. |
+| `devices` | Reply: `ok hmd <r> left <r or -> right <r or ->`, the current `ETrackingResult` values (`-` for no device in that role). |
+| `head` | Reply: `ok <12 floats>`, the current HMD pose (standing universe). |
+| `ping` | `ok shown` or `ok hidden`. |
+
+It quits on SteamVR's quit event, as ft-gazepanel does. `--no-vr --dump DIR` writes each picture to `DIR/panel.pam`, to check the layout without a headset.
+
+## The pose pictures (`poses/`)
+
+`poses/poses.json` maps the script's `pose` ids to pictures: `{"<pose>": {"file": "<name>.png", "two_hands": false, "caption": "..."}}`. Each PNG is RGBA, square (512x512), drawn as the wearer sees it: a right hand, unless `two_hands` (then it shows both). How a prompt shows it (`session.pose_view`):
+
+| Prompt's `hands` | Picture |
+|---|---|
+| `right` (and `any`, `none`, empty) | as drawn |
+| `left` | flipped left to right (`image ... mirror`) |
+| `both`, not `two_hands` | a flipped copy on the left, the picture on the right (`image ... both`) |
+| anything, `two_hands` | as drawn |
+
+A pose with no entry, or whose file is missing, shows no picture: the text alone. The session reads `poses.json` when it starts. The window shows the same picture (QML `Image.mirror`), its caption, and the same diagram.
+
+## The script (`script.json`)
+
+```json
+{"version": 1,
+ "cue_names": {"thumbs-up": "Thumbs up", "...": "..."},
+ "sections": [
+   {"id": "hand-size", "title": "Hand size", "requires": [], "intro": "text shown before the first prompt: 4 s, or until Next", "go": "Hold",
+    "quick": true, "prompts": [{"text": "...", "cues": ["flat", "flat-back", "spread"], "cue_s": 5, "hands": "both", "distance": "mid"}]},
+   {"id": "pose-sweeps", "title": "Hand poses", "kind": "sweep", "quick": true, "cue_s": 4, "step_s": 20,
+    "groups": [["open", "fist", "point", "pinch"], ["ok", "thumbs-up", "spread", "claw"], ["count-1", "...", "count-5"]],
+    "sweeps": [{"hands": "both", "group": "next", "text": "..."}, {"hands": "left", "group": "any", "quick": false, "text": "..."}]},
+   {"id": "objects", "title": "Things you hold", "requires": ["objects"], "for_each": "object",
+    "prompts": [{"text": "Pick up the {object} and use it the way you normally would.", "seconds": 15, "hands": "both", "object": "{object}"}]},
+   {"id": "touch", "title": "Touch the dot", "kind": "targets", "hold_s": 1.0, "timeout_s": 8,
+    "targets": [[0.0, -0.15, -0.40], ...], "text": "Touch the dot with your index fingertip and hold still."},
+   {"id": "controller-push", "title": "Depth with controllers", "requires": ["controllers"], "kind": "bar",
+    "intro": "...", "heights": ["chest", "desk", "eye", "left", "right"], "reps": 5, "period_s": 6, "near_m": 0.2, "far_m": 0.6}
+ ]}
+```
+
+- `requires`: `objects` (at least one object ticked), `controllers` (straps ticked). A section whose requirements aren't met is skipped and logged.
+- `go`: the word the countdown ends on, "Go" unless set ("Hold" for the still poses).
+- `quick`: the section is in a quick round; a step with `"quick": false` isn't (below).
+- `kind`:
+  - `prompts` (the default): each prompt shows for its `seconds` with a countdown. A prompt with `cues` (and `cue_s`) is a sweep step with those cues in that order, `seconds` = cues x cue_s (hand size; the mouse and switch step).
+  - `sweep`: steps built from `sweeps` (below).
+  - `targets`: each target shows until the live index tip is within 3 cm of it for `hold_s`, or `timeout_s` passes.
+  - `bar`: the target marker sweeps near to far and back, `reps` times per height, at `period_s` per sweep. The current marker follows the live palm distance.
+- Each section is one take. Prompts within it are marked in `prompts.jsonl`.
+- `cue_names`: the short labels under the strip's pictures (otherwise the pose id).
+
+### Sweeps
+
+The second in-headset session (sessions/20261002-202734) took about 16 min and was "super long and kind of annoying": the 36 held poses alone took 7.9 min, 3.2 of it reading, and the person skipped the wrist turns and gave up on the bare push after 3 steps. The labels come from the auto-labeller (teacher model and triangulation, frame-hands `train/label`), not from the prompts, so what the recordings need is variety (shapes, distances, angles), not clean holds. So the poses are swept:
+
+- A sweep step shows a strip of 2-5 pose pictures and asks for slow, continuous movement (near and far, all around) while the hands change shape with the lit picture. The light moves on every `cue_s` (4 s), cycling, over `step_s` (20 s): 5 cues for a group of 4 or 5. Each cue is a `prompt` event with that `pose`, `"cue": true` and `"step"` (the step's id); `distance` and `position` are "" (varied). So the timeline tags each pose roughly, and the countdown, `wait` and `redo` work per step as before. The time-left bar covers the whole step.
+- `groups`: lists of poses. A step takes `"group": "next"` (the groups in turn) or `"any"` (one drawn at random, a different one for each "any" while there are groups left), or names its own `cues`. `"shuffle": false` (gestures) keeps the script's order; `"cycle": false` runs the cues once; `"fixed": true` keeps one step's order.
+- The pose sweeps: 3 two-hand sweeps, one per group ([open, fist, point, pinch], [ok, thumbs-up, spread, claw], [count-1 ... count-5]); then a left-hand and a right-hand sweep (the other hand in the lap) on groups drawn from those three, which also turn the wrist as they go, so the wrist angles come for free (the old wrist-turns section is gone).
+- **The shuffle, per session.** The groups' order, the "any" draws and the cues' order in each step are shuffled with a seed from the session's id (`session_seed`: the first 8 hex digits of its SHA-256), separately per section (`random.Random("<seed>/<section>")`). The sections' order isn't shuffled (the controller sections need their before and after). session.json records `"shuffle": {"seed", "sweeps": {section: [{"id", "hands", "cues"}]}}`, and `build_plan(script, checklist, seed=...)` gives the same again. `--seed N` overrides it; `--plan` without one shows the script's order.
+- The strip has one picture per pose, a single hand: both hands make the same shape, and five pairs wouldn't fit. A left hand's pictures are flipped.
+- Gestures are sweeps too, in order and once: tap then drag; grab, cross, overlap; near the face then a screen's distance. Hand size is one step of three held shapes (flat, backs, spread; 5 s each), still the no-hands check's first step.
+
+### Quick round
+
+For more lighting rounds: "Quick round (about 3 min)" on the checklist page, `session.py --quick`. It has the sections marked `quick` (hand size, the pose sweeps without the one-hand ones, touch the dot, no hands), about 2 min recorded in 6 steps. session.json gets `"quick": true`. The checklist page suggests it (and picks it) once a full session has gone to the end (`Backend.hasFullSession`: a session.json with status `done`, not quick, not a dry run).
+
+### Step mode (the default) and auto mode
+
+The first in-headset session (2026-10-02) went too fast: each prompt advanced after 4-8 s, before there was time to read it and find the hand shape. So by default every step waits:
+
+1. **Ready.** The panel shows the step: section title, "step N of M", the instruction, the pose picture, the where-to diagram, and the Next hint ("Ready? Press the button on the right side of the headset", or with a mouse also Space and Next: see Controls). The hand chips show which hands are seen, with no warnings yet. It waits as long as it takes, and nothing records.
+2. **Countdown.** Next starts a new recording part and a "ready" event, and the panel counts 3, 2, 1 (big), recorded so the hold is captured from its first frame. In the push sections the bar sits at near meanwhile.
+3. **Hold.** The `prompt` event, the section's word ("Hold" or "Go") and the time-left bar for the prompt's seconds (or the bar's sweeps, or the targets). Then a `wait` event and the part stops.
+
+Steps that wait: each prompt, each bar height, and the first touch-the-dot target (the others follow straight on, as each waits for the touch anyway). Before a section, one screen shows the section's intro (with its `before` text, such as putting on the controllers) and waits for Next too; the welcome screen as well. The take starts with the section's first countdown, so a section skipped at its intro leaves no take. A pause in a hold works as before (the part stops; resume starts the next one).
+
+Auto mode ("Advance by itself" on the checklist page, `session.py --auto`) is the old timed flow: the welcome, the between and before screens, the intro (recorded) and each prompt for its seconds, one recording per take. R still works there: it restarts the step running.
+
+Steps are 20 s sweeps, 16-18 s gestures, 10-12 s desk and object steps, the touch targets (6) and the push heights (2, 3 reps of 6 s). The core session (no objects, no controllers) records about 5 min in 15 steps; with 5 s of reading a step that's about 6 min. Everything ticked (four objects, controllers): about 7.3 min recorded in 24 steps. A quick round: about 2 min in 6 steps. The window and `session.py --plan` give these (`plan_summary`); in step mode they leave the reading time out and say so.
+
+The sections, in this order (see the plan):
+1. hand size (one step: flat, backs, spread)
+2. pose sweeps (above: 3 with both hands, one with each hand)
+3. gestures (3 sweeps: pinch taps and drags; grabs, crossing and overlapping; near the face, then pointing at screen distance)
+4. desk work (typing or pretend typing; the mouse, with switching to the keyboard when both are there)
+5. objects (one prompt per ticked object, own objects included)
+6. touch the dot (6 dots spread near and far, left and right, low)
+7. controller depth, straps (push out and back at chest and eye level, 3 reps each; then the wrists and fingers with the controllers on). "Out and back" is straight away from the headset and back toward it: the first in-headset session took the left-right bar for sideways. The texts say so, the bar's ends read "At your chest" and "Arm out" (`near_label`, `far_label`, sent as `bar ... At your chest|Arm out`), and the picture is a side view.
+8. bridge (one controller on, the bare fingertip on its thumbstick while that hand moves from close to arm's length and back, then swap; then a controller on the desk, touched from the front and above, then from the sides: 4 steps, the controller changes during the ready screens)
+9. bare repeat of section 7's pushes, controllers off
+10. no hands (10 s)
+
+Before section 7: "Put on both controllers and tighten the straps". Before section 9: "Take the controllers off and put them out of view".
+
+### Feedback while recording
+
+- **Hands seen:** from the hands file. A hand counts as seen if its flags match the side and the file is fresh (publish within 0.3 s). Prompts with `hands` set show the `hands` chips. If an asked-for hand is lost for more than 1.5 s, the note says "I can't see your left hand: bring it into view".
+- **Controller tracking:** in sections 8 and 9, `devices` is polled once a second. A result other than 200 for more than 1 s says "The left controller lost tracking: turn your palm slightly toward you". Each such stretch goes into `prompts.jsonl` as `feedback` with `"controller": {...}`.
+- **Lighting, measured:** the checklist page measures the light when it opens, starting ft-camd (`frametop-handrec-camd.service`) if nothing runs it; the window stops it again on quit if it started it. The mean of every mono camera's `mean` and `dark_mean` from the ring (`hands/tools/ring.py` layout; struct only, no numpy) is compared with the person's earlier sessions. If it matches an earlier round's within 15%, the window says so before starting.
+- **Lighting label:** "Measured by the cameras" is the default. `ambient_ir`, the mono cameras' mean `dark_mean` (the room's infrared), labels the round `daylight` from 6.0 and `indoor` below (`session.classify_lighting`). Lamps and LEDs give off hardly any infrared, so a dim room and a lit one read about the same (1.8 by one lamp, 2.2 in a lamp-lit room) and the cameras can't tell them apart; the person can pick dim, room or daylight instead (`source: "picked"`). The 6.0 threshold is a guess until a daylight round is measured.
+
+### Camera check
+
+On 2026-10-02 a whole session showed "I can't see your hands": after the headset slept, SteamVR had failed to load the colour module's FPGA image, which left the upper cameras and the IR light off (`hands/README.md`, "Camera check"). Two checks keep that from wasting a session:
+
+- **Before the session.** The window runs `hands/camcheck.py` when the checklist page opens ("Tracking cameras:", with the evidence under Details and Check again), and again when Start is pressed; `session.py` runs it first thing, before it makes the session's folder or starts anything (`Session._preflight`). If it finds the cameras degraded, the session doesn't start. The window says: "The headset's upper cameras and IR light are off. SteamVR couldn't start the colour camera module (it happens sometimes after the headset sleeps). Restart SteamVR, or restart the headset if that doesn't fix it." (another `degraded:` reason gets a sentence naming it), and Start stays off. `unknown` (SteamVR not running, the cameras asleep) doesn't stop it: the session's own start fails clearly then. From the command line, `session.py` prints the check and exits with status 3.
+- **Restart SteamVR.** The message has a Restart SteamVR button. After a confirmation it runs `systemctl --user restart --no-block steamvr.service` on the host (through `distrobox-host-exec`), then checks the cameras every 3 s, for up to 2 minutes, until a new XRService has opened its cameras. The confirmation says what really happens: every VR app closes, and so does the Frametop desktop with all its windows, this one included, and it doesn't come back by itself (`hands/README.md`, "What a SteamVR restart does to Frametop"). So the check after the restart mostly happens when the recorder is opened again; it re-runs when the checklist page opens. `--no-block` lets the restart finish after the window is gone.
+- **No hands in the first step.** The first step of the hand-size section has both hands up, about 40 cm away. During its hold, the session counts the hands file's reads (about 20 a second). If the tracker published in at least half of at least 10 reads and never saw a hand, the step stops: a `wait` and the recording stops as usual, then a `nohands` event and a `redo` over the try. The session runs the camera check and shows "I can't see your hands" with its result (the camera text above when it's the VCINT failure, else "The camera check found nothing wrong"). The state is `nohands`, waiting: Next (the headset button, Space, Try again) or R starts the same step's countdown at once; Stop (Esc) ends the session with `stop_reason` set; S skips the section. One missed step costs a retry, not the session, and every hold of that step is logged ("hands check ...: a hand in N of M reads"). Without a tracker publishing the session can't tell, logs that, and goes on. Only that one step is checked: later steps have their notes ("I can't see your left hand") as before. Auto mode does the same; its recording pauses meanwhile.
+
+### Controls
+
+- The window has Start, a big Next (while a step waits; its hint is the panel's), Pause/Resume, Redo step, Skip section and Stop. While it has focus: Space is Next, P pauses or resumes, R redoes, S skips the section, Esc stops. The panel's bottom line and the window list them. The window also shows the step's picture, diagram, countdown and "Hold".
+- **The headset's button.** The Frame has a click button on its right side, for use without controllers: `KEY_SELECT` (353) on the evdev device `gpio-keys`. While a step waits (the welcome, a section's intro, a step's ready screen) a press is Next; during a countdown or hold (and auto mode's timed screens) it pauses; while paused it resumes. The session finds the device in `/proc/bus/input/devices` by name and its KEY bitmap (`event3` on the maintainer's Frame) and reads `input_event` structs with plain `struct` (no python-evdev), from a thread. Only key-downs count (value 1: releases and autorepeat, value 2, don't), and presses closer than 0.3 s count once.
+  - It's read, never grabbed (`EVIOCGRAB`). Frametop's input relay (`input/input-relay.py`), ft-powerd, SteamVR and gamescope read the same device, and the relay remaps its volume keys (the experimental branch's `docs/hazards.md`). A grab would take the volume keys from the relay.
+  - `steamos` is in the `input` group, so it needs no sudo. The dev container sees the host's `/dev/input` and `/proc/bus/input/devices`, and the group carries over (checked 2026-10-02), so it works from the window there and from `session.py` on the host. If the device is missing or can't be opened, the session logs it, keeps trying every 5 s, and the hints don't mention the button.
+  - **Not known yet (needs the headset):** what SteamVR and gamescope do with the same press. They read it too, so it may also click whatever is under the head or gaze pointer in VR, or open something. Check on the first session; if it does, the fix is on their side or a different button, not a grab.
+- **The Next hint follows what's there.** No mouse connected: "Ready? Press the button on the right side of the headset" (the window's Next can't be clicked, and Space needs the window focused). A mouse: "Ready? Press Space, click Next, or press the headset button". No button: "Ready? Press Space or click Next". The panel's bottom line leads with "Headset button: next, pause". A mouse is a device in `/proc/bus/input/devices` with EV_REL, REL_X and REL_Y that isn't made in software: uinput devices (Frametop's virtual mouse, frame-voice's keyboard) sit under `/devices/virtual/input` or on the virtual bus (6), and are left out; Bluetooth mice come through uhid, under `/devices/virtual/misc/uhid`, and count. It's looked at again at each step, so a mouse plugged in mid-session counts from the next step.
+- **R (redo).** During a step's countdown or hold: that step starts again from its ready screen. At a step's ready screen: the step before it (in this section) goes again. Either way a `redo` event marks the range of the try being redone, so its labels are skipped; the sets stay, to delete in review if wanted.
+- A pause stops the take's recording and starts it again on resume as the next part of the same take (`sets-2.bin`, and so on). takes.py reads all parts in order. A pause while a step waits only shows "Paused"; a Next pressed while paused doesn't count.
+
+## Review and export (`takes.py`, the window)
+
+- **Review.** Each session lists its takes: title, duration, status, a thumbnail (the first set's `slam_left`). A viewer shows one frame set (all cameras side by side, 8-bit grey) at a time with a slider. It can mark a range and delete it. Deleted ranges go into `take.json`, and export leaves them out; the files keep everything until export. A whole take or session can be deleted (files removed, after a confirmation).
+- **Export.** It writes `exports/<session>/`:
+  - `manifest.json`: profile fields except `optional.notes` unless kept, session.json (without its `uploads` records), takes, schema, tool version, the consent version.
+  - `calibration.json` and `device.json`, when the session has them.
+  - Per take: `prompts.jsonl`, `poses.jsonl`, `take.json`, and `sets.bin.zst` (sets in deleted ranges removed, then zstd -10 with 2 threads).
+  - The side cameras are named right in every exported set when the session's `sides.swapped` is known: parts that need it get slam_left and slam_right (and their `_dk`) exchanged in the set headers as they're compressed. The exported take.json says `"parts": {"sets.bin": {"names_swapped": <swapped>}}`, and the manifest's take entry `"sides": {"names_swapped", "renamed_sets"}`. Unknown, the names stay as recorded.
+  - `poses.jsonl` and `prompts.jsonl` without what's in the deleted ranges: no poses and no live-tracker `feedback` there (the prompt timeline stays). With the checklist's controllers at `none`, the controllers' poses are null and `feedback` has no `controller` (`takes.export_jsonl`): controllers left switched on still get tracked, and their poses would pass for the hands' ground truth.
+  - `SHA256SUMS`.
+
+  Compression runs at nice 19. While it runs with the headset worn, the window notes that VR may stutter a little (exports are done in the headset). Worn is judged the way `frame-job` does: `vrcompositor` runs and a `/sys/class/backlight/*/brightness` reads over 0 (SteamVR turns the panel off 5 s after the headset comes off). CPU work while in VR causes stutter. The proximity sensor is no use here: it read 9-43 with the headset sitting unworn.
+- **Upload.** The Upload page uploads an export from the window, logging in included: no terminal. Below its steps it shows `UPLOAD.md` ("About uploading"), with the export's path, size and contributor id filled in.
+  - **`hands/rec/validate.py`** (standard library) checks an export. The window runs it before an upload, and the maintainer runs it on each submission (`validate.py DIR [--json]`, exit status 1 on errors). It checks:
+    - `SHA256SUMS`: every file listed and matching.
+    - An allow-list: `manifest.json`, `calibration.json`, `device.json` and `SHA256SUMS` at the top, and `prompts.jsonl`, `poses.jsonl`, `take.json` and `sets.bin.zst` in `takes/<NN>-<section>/`. Anything else is an error, and so is a symlink.
+    - The manifest's schema, keys and types, and that it matches the files.
+    - The consent version is present and the contributor confirmed being an adult. The contributor id is a uuid4.
+    - `calibration.json` has nothing that `session.py`'s `strip_calibration` would still remove.
+    - `device.json` holds only `cv.cad_from_cal` and `head`, each `plus_x`, `plus_z` and `position` as 3 numbers (plus `cad_from_cal`'s `method`). Without it: a warning.
+    - Each `sets.bin.zst` decompresses to its end, so a truncated one fails, and every set's FHSET01 header is sane: camera names, sizes, record length. Set counts and raw bytes match the manifest. Pixels aren't decoded.
+    - Every jsonl line parses.
+    - `session.sides` is `{"swapped": true|false|null}`, and every take's `sides.names_swapped` equals it (else an error: export again). Without `sides`, or `null`: a warning (the maintainer's check tells).
+    - The total size: a warning over 15 GB, an error over 40 GB.
+
+    Warnings cover notes kept in the export, a home folder path in the manifest, and missing `poses.jsonl` files.
+
+    It runs on Linux and on Windows (the maintainer's PC) with Python 3.12 or later. It decompresses with Python 3.14's `compression.zstd`, else the `zstandard` package, else the `zstd` program, and handles several zstd frames in a row.
+  - **`hands/rec/hub.py`** does the upload. It runs as a child process of the window (Cancel ends it), or from the command line (`hub.py [--base DIR] whoami | login | upload SESSION [--dry-run] [--again] [--json]`). It uses `huggingface_hub` (`python3-huggingface-hub` in the dev container) with the login saved in huggingface_hub's token file.
+    - **`login`** is huggingface_hub's browser login (OAuth device code), the same as `hf auth login`'s default since huggingface_hub 1.x. It asks Hugging Face for a link (`https://hf.co/oauth/device`) and a short code, prints them (`{"phase": "code", "url", "code", "expires_in"}`), and waits while the person enters the code in their browser and approves. Then it saves the token, which can refresh itself. Nobody types or pastes a token, and the window never sees one. It uses huggingface_hub's own helpers (`request_device_code`, `poll_device_token`, `_save_oauth_token`), as there's no public call that hands back the code. On 2026-10-03 the code lasted 5 minutes and wasn't filled into the link. The consent screen lists the person's organizations: none are needed, as a pull request on a public dataset comes from the person's own account.
+
+    An upload goes through these steps: An upload goes through these steps:
+    1. Validate, and stop on errors.
+    2. Stop if the same export was uploaded before (same `SHA256SUMS`), unless asked again.
+    3. Stop while the texts are drafts, unless `FT_HANDREC_ALLOW_UPLOAD=1`.
+    4. `whoami`: a read-only token is refused.
+    5. `auth_check` on the dataset: a gated dataset whose terms aren't accepted gives "accept the dataset's terms first", with the link.
+    6. Open the pull request first: `create_pull_request(HF_DATASET, title, description=...)`, an empty draft. Its link goes to the window right away (`{"phase": "opened", "pr_url"}`), which tells the person to plug the headset in and leave it. Record `{"repo", "pr_url", "pr_num", "started", "export_sha", "status": "started"}` in `uploads` in `session.json`. A retry of the same export goes on in that pull request while it's draft or open.
+    7. `upload_folder(repo_id=HF_DATASET, repo_type="dataset", folder_path=EXPORT, path_in_repo="contributions/<contributor>/<session>", revision="refs/pr/N", commit_message=..., commit_description=...)`. The description summarizes the manifest: takes, minutes, sets, lighting, objects, controllers, the consent and tool versions, the size, and validate's warnings. Then mark the pull request open if it's still a draft (the maintainer's `list` shows open ones, so drafts are uploads still in progress).
+    8. Mark the record `"status": "done"` with `uploaded`. `export_sha` is the SHA256 of `SHA256SUMS`. The file keeps its modification time, so the export doesn't count as out of date. Only finished records count as "uploaded before" (records without a status are from before 2026-10-03 and finished).
+
+    Errors get a plain explanation: not logged in, a login Hugging Face rejects (401), a login that can't open a pull request (403), terms not accepted, dataset not found, network errors. `--dry-run` does everything except the network calls and the record, and lists what it would upload.
+  - **The page** has three numbered steps:
+    1. Choose the export, with its size.
+    2. Log in to Hugging Face. The page shows whether someone is logged in (`whoami`). Log in runs `hub.py login`, opens the link in the browser, and shows the code large, with Copy code and Cancel. "Use another account" logs in again. A classic read-only token counts as not logged in.
+    3. Upload, with a note to accept the dataset's terms the first time.
+
+    Upload has Cancel, the phase with a progress bar (a share while the export is checked, a sweep while it's sent, as `huggingface_hub` reports no progress), the pull request's link once it's open, with "plug in the headset and leave it plugged in until this says Uploaded", and Uploaded at the end. If this export was uploaded before, the page says so, and uploading it again asks first. A stale export can't be uploaded.
+  - **Staying awake:** while an export or an upload runs, the window holds a host unit, `frametop-handrec-awake.service`, running `systemd-inhibit --what=sleep:idle --mode=block ... sleep infinity`, and stops it when the last of them ends (or on quit). It's meant to keep Steam from putting the Frame to sleep with the headset off; whether Steam's sleep honours a logind block inhibitor is still to be checked on the device. Exports and uploads are done in the headset: the export page only notes that VR may stutter a little while it compresses.
+  - **While the texts are drafts**, Upload stays off unless `FT_HANDREC_ALLOW_UPLOAD=1`, so the maintainer can rehearse against a private test repo. `FT_HANDREC_DATASET` overrides `HF_DATASET`. `ft-handrec --hub-dry-run` makes Upload a dry run: no network, so it isn't held back by the drafts.
+  - **Rehearsal: `hands/rec/rehearse.sh [--repo ID]`** runs it all without the headset, in the dev container, in one `frame-job --local` scope when frame-job is installed. `ft-ringplay` plays 30 s of a recording into a ring in `/run/user/UID`. A tracking ft-hands that's already running is used, or one is started on that ring. `ft-handpanel --no-vr` stands in for the panel. `session.py --no-start --next-after 0.3` records a three-section test script (a prompt, a two-cue sweep, no hands) in step mode, three parts of 6 s (countdown and hold), about 360 MB once exported. Then `takes.py` exports, `validate.py` checks, and `hub.py` uploads: a dry run by default, or for real to `--repo ID` with `FT_HANDREC_ALLOW_UPLOAD=1`. It prints a summary, deletes its temporary folders (camera images of a room) and stops everything it started, Ctrl+C included. The `--no-vr` panel logs no poses, so `poses.jsonl` is missing there (a warning).
+
+## Licensing and consent (texts in `CONSENT.md`)
+
+- The dataset is CC BY-NC 4.0.
+- Contributors also grant the maintainer (DeeJanuz) a broad, non-exclusive license to their contribution.
+- Contributors confirm they're 18 or older.
+- The text explains what's recorded and that nothing uploads automatically, how review works, how to withdraw (by contributor id), and that a withdrawal is purged from the repo's history.
+- The texts need a legal review before the dataset launches. Until then they carry a "draft" banner, and the Upload page says contributions aren't open yet.
+- The dataset repo: `HF_DATASET` in `hub.py`, `DeeJanuz/frametop-hands` (private until launch).

hands/rec/UPLOAD.md

@@ -0,0 +1,9 @@
+### About uploading
+
+- **What's sent:** the export in `@EXPORT_PATH@` (@EXPORT_SIZE@), into `contributions/@CONTRIBUTOR@/@SESSION@` in [@DATASET@](https://huggingface.co/datasets/@DATASET@). Upload first checks that every file is complete and matches its checksum, and that nothing identifying is left in.
+- **Your account:** the pull request comes from your Hugging Face account, and anyone can see its username next to your contributor id, `@CONTRIBUTOR@`. The dataset itself credits the contributor id, not your name. The login stays saved on this headset, so you only log in once.
+- **The dataset's terms:** the first time, open the dataset's page and accept its terms. Until you have, Upload stops with "accept the dataset's terms first".
+- **Once your pull request's link shows, plug in the headset and leave it plugged in until this page says Uploaded.** A round is several gigabytes, so this can take a while. You can take the headset off: the Hand Recorder keeps it awake until the upload is done. Keep the Hand Recorder open, since closing it stops the upload.
+- **If it stops partway** (Cancel, or the network drops), press Upload again. It carries on in the same pull request, and files already sent aren't sent twice.
+- **The maintainer's review:** they check that the files are complete, and that nobody else and nothing private is in view, before merging. You can follow it and answer questions on the pull request's page. Once it's merged, you can delete the session and its export here to free the space.
+- **Withdrawing:** see "Withdrawing" in the consent text. You'll need your contributor id, `@CONTRIBUTOR@`.

hands/rec/build.sh

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+# Build the hand recorder's headset panel ft-handpanel in the dev container on the Frame
+# (hands/rec/build/; it also runs there). Like gaze/build.sh: the pinned public OpenVR header
+# (the DMA-BUF import is newer than the header shipped with SteamVR's samples), stb_truetype
+# for the text and stb_image (PNG only) for the pose pictures, at the same stb commit.
+set -euo pipefail
+root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
+"$root/scripts/sync.sh" >/dev/null
+exec "$root/scripts/frame.sh" -C hands/rec 'set -e; mkdir -p build/include
+openvr=v2.15.6
+[ -f build/include/openvr-$openvr ] || { curl -fsSL "https://raw.githubusercontent.com/ValveSoftware/openvr/$openvr/headers/openvr.h" -o build/include/openvr.h && touch build/include/openvr-$openvr; }
+stb=2c980bb59875b0d32144a71867fbdebb2f77cd20
+[ -f build/include/stb-$stb ] || { curl -fsSL "https://raw.githubusercontent.com/nothings/stb/$stb/stb_truetype.h" -o build/include/stb_truetype.h && touch build/include/stb-$stb; }
+[ -f build/include/stb_image-$stb ] || { curl -fsSL "https://raw.githubusercontent.com/nothings/stb/$stb/stb_image.h" -o build/include/stb_image.h && touch build/include/stb_image-$stb; }
+g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -Wno-missing-field-initializers -Ibuild/include $(pkg-config --cflags gbm libdrm) \
+  -o build/ft-handpanel panel/ft-handpanel.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64 \
+  $(pkg-config --libs gbm libdrm) -lpthread
+echo "built build/ft-handpanel"'

hands/rec/ft-handrec

@@ -0,0 +1,21 @@
+#!/bin/bash
+# Launch the Frametop Hand Recorder from a Plasma session on the Frame host.
+# The app runs in the dev container (PySide6, Kirigami and zstd come from Fedora there).
+# podman needs the real XDG_RUNTIME_DIR and the real user bus (to reach systemd for
+# the container's cgroup; the Frametop session runs on a private bus from
+# dbus-run-session). The session's Wayland socket and bus go to the app itself.
+# Options go to ft_handrec.py: --base DIR, --page NAME, and --dry-run for testing.
+here=$(cd "$(dirname "$(readlink -f "$0")")" && pwd)
+wl=${WAYLAND_DISPLAY:-wayland-0}
+case $wl in /*) ;; *) wl="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/$wl" ;; esac
+session_bus=${DBUS_SESSION_BUS_ADDRESS:-}
+export XDG_RUNTIME_DIR=/run/user/$(id -u)
+export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
+# From the home folder: the app's host commands (distrobox-host-exec) run in its folder on the
+# host, and a folder only the container has, such as /run/host/tmp, makes them all fail.
+cd "$HOME" || exit 1
+"$here/../../scripts/container-up.sh"
+exec "$HOME/.local/bin/distrobox" enter dev -- env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
+  XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \
+  QT_QPA_PLATFORM="wayland;xcb" \
+  python3 "$here/ft_handrec.py" "$@"

hands/rec/ft-handrec.desktop

@@ -0,0 +1,9 @@
+[Desktop Entry]
+Type=Application
+Name=Frametop Hand Recorder
+GenericName=Record your hands for the hand dataset
+Comment=Record, review and export hand recordings for Frametop's open hand dataset
+Exec=@REPO@/hands/rec/ft-handrec
+Icon=camera-video
+Categories=Utility;
+Keywords=hands;hand tracking;dataset;record;frametop;

hands/rec/hub.py

@@ -0,0 +1,468 @@
+#!/usr/bin/env python3
+"""Upload a hand recorder export to the hand dataset on Hugging Face (DESIGN.md "Upload").
+
+The window runs this as a child process (so Cancel can end it, and huggingface_hub stays out
+of the window's process); it also runs from the command line. It uses huggingface_hub (in the
+dev container: python3-huggingface-hub, from setup/dev-container.sh) with the login that
+`hub.py login` (the window's Log in) or `hf auth login` saved. Nobody types a token anywhere.
+
+`login` is huggingface_hub's browser login (OAuth device code, as `hf auth login` does): it gets a
+link and a short code, the person enters the code in their browser and approves, and the token goes
+straight from Hugging Face into huggingface_hub's token file. This process saves it; it never
+prints it, and the window never sees it.
+
+An upload:
+  1. checks the export with validate.py, and stops on errors;
+  2. stops if this export (same SHA256SUMS) was uploaded before, unless --again;
+  3. stops while CONSENT.md or UPLOAD.md is a draft, unless FT_HANDREC_ALLOW_UPLOAD=1;
+  4. checks the login (whoami: a read-only token can't open a pull request) and access to the
+     dataset (auth_check: a gated dataset's terms must be accepted first);
+  5. opens the pull request first (a draft, empty), so its link can be shown while the files go,
+     and records it under "uploads" in session.json with "status": "started";
+  6. upload_folder(..., revision="refs/pr/N") to contributions/<contributor>/<session>; a retry of
+     the same export goes on in the same pull request while it's still open;
+  7. marks the record {"repo", "pr_url", "pr_num", "uploaded", "export_sha", "status": "done"}.
+--dry-run does all of it except the network calls (4 to 6) and recording (5, 7), and says what
+it would upload.
+
+The dataset is HF_DATASET; FT_HANDREC_DATASET overrides it (a test repo, for rehearsals).
+
+usage: hub.py [--base DIR] whoami [--json]
+       hub.py [--base DIR] login [--json]
+       hub.py [--base DIR] upload SESSION [--dry-run] [--again] [--json]
+With --json, each line of output is one JSON object: {"phase", "text", "fraction"} as it goes,
+with {"phase": "opened", "pr_url"} once the pull request exists, then {"done": {...}} or
+{"error": {"kind", "text", "link", "errors"}}. login says {"phase": "code", "url", "code",
+"expires_in"} once it has the link, then {"done": {"name", "role"}}. Exit status 0: done.
+"""
+import argparse
+import datetime
+import hashlib
+import json
+import os
+import re
+import sys
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+import takes  # noqa: E402  (next to this file)
+
+# The dataset contributions go to (DESIGN.md "Licensing and consent").
+HF_DATASET = "DeeJanuz/frametop-hands"
+DATASET_ENV = "FT_HANDREC_DATASET"
+ALLOW_ENV = "FT_HANDREC_ALLOW_UPLOAD"
+CONSENT_PATH = os.path.join(HERE, "CONSENT.md")
+UPLOAD_PATH = os.path.join(HERE, "UPLOAD.md")
+REPO_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*/[A-Za-z0-9][A-Za-z0-9._-]*$")
+TOKENS_URL = "https://huggingface.co/settings/tokens"
+# Quiet huggingface_hub: no progress bars on stderr, no telemetry or update hints.
+HF_ENV = {"HF_HUB_DISABLE_PROGRESS_BARS": "1", "HF_HUB_DISABLE_TELEMETRY": "1", "HF_HUB_DISABLE_UPDATE_CHECK": "1"}
+
+
+class HubError(Exception):
+    """What went wrong, for people: kind (below), text, a link to open, validation errors.
+    Kinds: missing (no huggingface_hub), login, read-token, terms, not-found, permission,
+    network, hub, invalid, duplicate, closed, no-export."""
+
+    def __init__(self, kind, text, link="", errors=None, extra=None):
+        super().__init__(text)
+        self.kind, self.text, self.link = kind, text, link
+        self.errors = errors or []
+        self.extra = extra or {}
+
+    def as_dict(self):
+        return dict(self.extra, kind=self.kind, text=self.text, link=self.link, errors=self.errors)
+
+
+def dataset_id():
+    """HF_DATASET, or FT_HANDREC_DATASET when set."""
+    repo = os.environ.get(DATASET_ENV, "").strip() or HF_DATASET
+    if not REPO_RE.match(repo):
+        raise HubError("not-found", f"{DATASET_ENV}={repo!r} isn't a dataset id like owner/name")
+    return repo
+
+
+def dataset_url(repo=None):
+    return "https://huggingface.co/datasets/" + (repo or dataset_id())
+
+
+def is_draft(path):
+    try:
+        with open(path, encoding="utf-8") as f:
+            return "DRAFT" in f.readline()
+    except OSError:
+        return False
+
+
+def texts_draft():
+    return is_draft(CONSENT_PATH) or is_draft(UPLOAD_PATH)
+
+
+def upload_allowed():
+    """Real uploads: once the texts aren't drafts, or for the maintainer's rehearsal against a
+    test repo (FT_HANDREC_ALLOW_UPLOAD=1)."""
+    return not texts_draft() or os.environ.get(ALLOW_ENV) == "1"
+
+
+def now_iso():
+    return datetime.datetime.now().astimezone().isoformat(timespec="seconds")
+
+
+# ---------------------------------------------------------------- records in session.json
+
+def export_sha(export_dir):
+    """The export's identity: the SHA256 of its SHA256SUMS ("" if there's none)."""
+    try:
+        with open(os.path.join(export_dir, "SHA256SUMS"), "rb") as f:
+            return hashlib.sha256(f.read()).hexdigest()
+    except OSError:
+        return ""
+
+
+def uploads(store, session):
+    meta = takes.read_json(os.path.join(store.session_dir(session), "session.json"))
+    return [u for u in meta.get("uploads") or [] if isinstance(u, dict)]
+
+
+def previous_upload(store, session, sha):
+    """The latest finished upload of this same export, or None. (Records from before
+    2026-10-03 have no status: they were written only when an upload finished.)"""
+    same = [u for u in uploads(store, session) if sha and u.get("export_sha") == sha
+            and u.get("status", "done") == "done"]
+    return same[-1] if same else None
+
+
+def unfinished_upload(store, session, sha, repo):
+    """The latest upload of this same export to repo that opened its pull request and didn't
+    finish, or None."""
+    same = [u for u in uploads(store, session) if sha and u.get("export_sha") == sha and u.get("repo") == repo
+            and u.get("status") == "started" and u.get("pr_num")]
+    return same[-1] if same else None
+
+
+def record_upload(store, session, record):
+    """Add record to session.json's "uploads". The file keeps its time: an upload doesn't
+    change the recordings, so the export mustn't count as out of date because of it
+    (takes.Store.sessions compares times)."""
+    path = os.path.join(store.session_dir(session), "session.json")
+    try:
+        st = os.stat(path)
+    except OSError:
+        st = None
+    meta = takes.read_json(path)
+    records = meta.setdefault("uploads", [])
+    same = [i for i, u in enumerate(records) if isinstance(u, dict) and record.get("pr_num")
+            and u.get("pr_num") == record["pr_num"] and u.get("repo") == record.get("repo")]
+    if same:
+        records[same[-1]] = record   # the same pull request: started, then done
+    else:
+        records.append(record)
+    takes.write_json(path, meta)
+    if st:
+        os.utime(path, ns=(st.st_atime_ns, st.st_mtime_ns))
+
+
+# ---------------------------------------------------------------- the pull request's text
+
+def describe(summary, report, sha, nbytes):
+    """(commit message, commit description) from validate's summary of the manifest."""
+    s = summary
+    objects = ", ".join(s.get("objects") or []) or "none"
+    if s.get("own_objects"):
+        objects += f" (+{s['own_objects']} of their own)"
+    message = f"Hands: session {s.get('session')} from {s.get('contributor')}"
+    lines = ["Contribution to the Frametop hand dataset, uploaded from the hand recorder.", "",
+             f"- Session: {s.get('session')}",
+             f"- Contributor: {s.get('contributor')}",
+             f"- Takes: {s.get('takes')} ({s.get('minutes')} minutes, {s.get('sets')} frame sets)",
+             f"- Lighting: {s.get('lighting') or 'not given'}",
+             f"- Objects: {objects}",
+             f"- Controllers: {s.get('controllers') or 'not given'}",
+             f"- Consent version: {s.get('consent_version')}",
+             f"- Tool: {s.get('tool')}",
+             f"- Size: {takes.human_bytes(nbytes)}",
+             f"- SHA256 of SHA256SUMS: {sha}", "",
+             "Checked with hands/rec/validate.py: no errors" + (f", {len(report.warnings)} warnings:"
+                                                                 if report.warnings else ".")]
+    lines += [f"- {w}" for w in report.warnings]
+    return message, "\n".join(lines) + "\n"
+
+
+# ---------------------------------------------------------------- talking to the Hub
+
+def _hf():
+    os.environ.update({k: v for k, v in HF_ENV.items() if k not in os.environ})
+    try:
+        import huggingface_hub
+    except ImportError:
+        raise HubError("missing", "huggingface_hub isn't installed in the dev container: run setup/dev-container.sh "
+                                  "(or: pip install --user huggingface_hub)") from None
+    return huggingface_hub
+
+
+def explain(e, repo):
+    """A huggingface_hub (or network) exception as a HubError."""
+    if isinstance(e, HubError):
+        return e
+    try:
+        from huggingface_hub import errors as hf
+    except ImportError:
+        hf = None
+    url = dataset_url(repo)
+    if hf is not None:
+        if isinstance(e, hf.LocalTokenNotFoundError):
+            return HubError("login", "You're not logged in to Hugging Face. Press Log in.")
+        if isinstance(e, hf.GatedRepoError):
+            return HubError("terms", f"Accept the dataset's terms first: open {url}, read them and accept them, "
+                                     "then try again.", url)
+        if isinstance(e, hf.RepositoryNotFoundError):
+            return HubError("not-found", f"The dataset {repo} wasn't found, or it's private and your account has "
+                                         "no access to it.", url)
+        if isinstance(e, hf.HfHubHTTPError):
+            status = getattr(getattr(e, "response", None), "status_code", None)
+            first = str(e).strip().splitlines()[0] if str(e).strip() else type(e).__name__
+            if status == 401:
+                return HubError("login", "Hugging Face didn't accept your login (it may have expired or been "
+                                         "revoked). Log in again.")
+            if status == 403:
+                return HubError("permission", "Your login isn't allowed to open a pull request. Log in again, "
+                                              "and allow everything the Hugging Face page asks for.")
+            return HubError("hub", f"Hugging Face refused the upload ({status or 'no status'}): {first}")
+    try:
+        import httpx
+        net = (httpx.TransportError, OSError)
+    except ImportError:
+        net = (OSError,)
+    if isinstance(e, net):
+        return HubError("network", f"Couldn't reach huggingface.co ({type(e).__name__}: {e}). Check the network "
+                                   "and try again: files already sent usually aren't sent twice.")
+    return HubError("hub", f"{type(e).__name__}: {e}")
+
+
+def whoami():
+    """{"name", "role"} for the saved login; HubError if there's none or it doesn't work.
+    role: "write", "read", "fineGrained" or ""."""
+    hf = _hf()
+    try:
+        info = hf.HfApi().whoami()
+    except Exception as e:
+        raise explain(e, dataset_id()) from None
+    token = ((info.get("auth") or {}).get("accessToken") or {})
+    return {"name": info.get("name", ""), "role": token.get("role", "")}
+
+
+def login(progress=None):
+    """The browser login: progress("code", text, url=, code=, expires_in=) once the link is ready,
+    then wait (up to the code's expiry: 5 minutes on 2026-10-03) for the person to approve it, and save
+    the token. Returns whoami(). Uses huggingface_hub's own device code helpers (1.x)."""
+    tell = progress or (lambda phase, text, **extra: None)
+    _hf()
+    try:
+        from huggingface_hub._login import _save_oauth_token
+        from huggingface_hub.errors import DeviceCodeError
+        from huggingface_hub.utils._oauth_device import poll_device_token, request_device_code
+    except ImportError:
+        raise HubError("missing", "This huggingface_hub has no browser login: update the dev container "
+                                  "(setup/dev-container.sh).") from None
+    try:
+        info = request_device_code()
+        tell("code", "Approve the login in your browser", url=info["verification_uri_complete"],
+             code=info["user_code"], expires_in=info["expires_in"])
+        _save_oauth_token(poll_device_token(info))
+    except DeviceCodeError as e:
+        raise HubError("login", f"The login didn't go through: {e}. Press Log in to try again.") from None
+    except Exception as e:
+        raise explain(e, dataset_id()) from None
+    return whoami()
+
+
+def upload(store, session, dry_run=False, again=False, progress=None, log=None):
+    """Upload exports/<session> (the steps in this file's docstring). progress(phase, text,
+    fraction or None); log(text) for the dry run's account. Returns the result; raises HubError."""
+    tell = progress or (lambda phase, text, fraction=None, **extra: None)
+    say = log or (lambda text: None)
+    import validate
+    repo = dataset_id()
+    try:
+        export = store.export_dir(session)
+    except ValueError as e:
+        raise HubError("no-export", str(e)) from None
+    if not os.path.isfile(os.path.join(export, "manifest.json")):
+        raise HubError("no-export", f"No export of session {session}: export it first")
+
+    tell("check", "Checking the export", 0.0)
+    report = validate.validate(export, progress=lambda f, text: tell("check", text, f))
+    if not report.ok:
+        raise HubError("invalid", f"The export has {len(report.errors)} problems, so it can't be uploaded. Export "
+                                  "the session again; if that doesn't help, report it.", errors=report.errors)
+    summary = report.summary
+    contributor = summary.get("contributor", "")
+    sha = export_sha(export)
+    before = previous_upload(store, session, sha)
+    if before and not again:
+        raise HubError("duplicate", f"This export was uploaded already, on {before.get('uploaded', '?')}: "
+                                    f"{before.get('pr_url', '')}", before.get("pr_url", ""), extra={"previous": before})
+    if not dry_run and not upload_allowed():
+        raise HubError("closed", "Contributions aren't open yet: the texts are drafts waiting for a legal review. "
+                                 f"(For a rehearsal against a test repo: {ALLOW_ENV}=1.)")
+    path_in_repo = f"contributions/{contributor}/{session}"
+    message, description = describe(summary, report, sha, report.bytes)
+    files = []
+    for root, _, names in os.walk(export):
+        files += [os.path.relpath(os.path.join(root, n), export) for n in names]
+    plan = {"repo": repo, "repo_type": "dataset", "folder_path": export, "path_in_repo": path_in_repo,
+            "create_pr": True, "commit_message": message, "commit_description": description,
+            "files": len(files), "bytes": report.bytes, "warnings": report.warnings}
+
+    if dry_run:
+        say(f"dry run: would check the login (whoami) and access to {repo} (auth_check)")
+        say(f"dry run: would upload_folder {len(files)} files, {takes.human_bytes(report.bytes)}, "
+            f"from {export} to datasets/{repo}/{path_in_repo}, as a pull request")
+        for rel in sorted(files):
+            say(f"  {rel}  {takes.human_bytes(os.path.getsize(os.path.join(export, rel)))}")
+        say(f"dry run: commit message: {message}")
+        say("dry run: commit description:\n" + description.rstrip())
+        record = {"repo": repo, "pr_url": "", "uploaded": now_iso(), "export_sha": sha, "status": "done"}
+        say(f"dry run: would record in session.json's uploads: {json.dumps(record)}")
+        tell("done", "Dry run: nothing was uploaded", 1.0)
+        return dict(plan, dry_run=True, pr_url="", record=record)
+
+    hf = _hf()
+    api = hf.HfApi()
+    tell("login", "Checking your Hugging Face login", None)
+    try:
+        who = api.whoami()
+    except Exception as e:
+        raise explain(e, repo) from None
+    role = ((who.get("auth") or {}).get("accessToken") or {}).get("role", "")
+    if role == "read":
+        raise HubError("read-token", "Your saved login is a read-only token, so it can't open a pull request. "
+                                     "Log in again.")
+    tell("access", f"Checking access to {repo}", None)
+    try:
+        api.auth_check(repo, repo_type="dataset")
+    except Exception as e:
+        raise explain(e, repo) from None
+    # The pull request first, so its link shows while the files go (and the person can plug the
+    # headset in and leave it). A retry of this export goes on in its pull request if it's open.
+    tell("open", "Opening your pull request", None)
+    pr = None
+    before = unfinished_upload(store, session, sha, repo)
+    if before:
+        try:
+            d = api.get_discussion_details(repo, int(before["pr_num"]), repo_type="dataset")
+            if d.is_pull_request and d.status in ("draft", "open"):
+                pr = d
+        except Exception:
+            pr = None
+    if pr is None:
+        try:
+            pr = api.create_pull_request(repo, message, description=description, repo_type="dataset")
+        except Exception as e:
+            raise explain(e, repo) from None
+    pr_url = getattr(pr, "url", "") or f"{dataset_url(repo)}/discussions/{pr.num}"
+    record = {"repo": repo, "pr_url": pr_url, "pr_num": pr.num, "started": now_iso(), "export_sha": sha,
+              "status": "started"}
+    record_upload(store, session, record)
+    tell("opened", f"Pull request #{pr.num} is open. Uploading {len(files)} files ({takes.human_bytes(report.bytes)})",
+         None, pr_url=pr_url)
+    try:
+        api.upload_folder(repo_id=repo, repo_type="dataset", folder_path=export, path_in_repo=path_in_repo,
+                          revision=f"refs/pr/{pr.num}", commit_message=message, commit_description=description)
+    except Exception as e:
+        raise explain(e, repo) from None
+    try:   # a pull request opened through the API stays a draft until it's marked open
+        if api.get_discussion_details(repo, pr.num, repo_type="dataset").status == "draft":
+            api.change_discussion_status(repo, pr.num, "open", repo_type="dataset")
+    except Exception:
+        pass   # the maintainer can open it
+    record = dict(record, uploaded=now_iso(), status="done")
+    record_upload(store, session, record)
+    tell("done", "Uploaded", 1.0)
+    return dict(plan, dry_run=False, pr_url=pr_url, pr_num=pr.num, user=who.get("name", ""), record=record)
+
+
+# ---------------------------------------------------------------- the command line
+
+def main():
+    ap = argparse.ArgumentParser(description="Upload a hand recorder export to the hand dataset on Hugging Face.")
+    ap.add_argument("--base", default=takes.DEFAULT_BASE)
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    w = sub.add_parser("whoami", help="show the saved Hugging Face login")
+    w.add_argument("--json", action="store_true")
+    li = sub.add_parser("login", help="log in to Hugging Face in the browser")
+    li.add_argument("--json", action="store_true")
+    u = sub.add_parser("upload", help="upload exports/SESSION as a pull request")
+    u.add_argument("session")
+    u.add_argument("--dry-run", action="store_true", help="no network: say what would be uploaded")
+    u.add_argument("--again", action="store_true", help="upload even if this export was uploaded before")
+    u.add_argument("--json", action="store_true", help="JSON lines, for the window")
+    a = ap.parse_args()
+
+    def emit(obj):
+        print(json.dumps(obj), flush=True)
+
+    try:
+        if a.cmd == "whoami":
+            who = whoami()
+            if a.json:
+                emit({"done": who})
+            else:
+                print(f"logged in as {who['name']} (token role: {who['role'] or 'unknown'})")
+            return 0
+        if a.cmd == "login":
+            def code(phase, text, **extra):
+                if a.json:
+                    emit(dict(extra, phase=phase, text=text))
+                else:
+                    print(f"Open {extra['url']} and approve the code {extra['code']}. Waiting...", flush=True)
+            who = login(code)
+            if a.json:
+                emit({"done": who})
+            else:
+                print(f"logged in as {who['name']}")
+            return 0
+        if a.json:
+            def progress(phase, text, fraction=None, **extra):
+                emit(dict(extra, phase=phase, text=text, fraction=fraction))
+
+            def log(text):
+                emit({"log": text})
+        else:
+            last = {}
+
+            def progress(phase, text, fraction=None, **extra):
+                if extra.get("pr_url"):
+                    print(f"pull request: {extra['pr_url']}", flush=True)
+                line = text if fraction is None else f"{fraction * 100:5.1f}% {text}"
+                if (phase, text) != last.get("key") or fraction in (0.0, 1.0):
+                    print(line, file=sys.stderr, flush=True)
+                last["key"] = (phase, text)
+
+            def log(text):
+                print(text, flush=True)
+        if hasattr(os, "setpriority"):
+            try:
+                os.setpriority(os.PRIO_PROCESS, 0, 19)   # validation reads and hashes everything: stay out of VR's way
+            except OSError:
+                pass
+        result = upload(takes.Store(a.base), a.session, dry_run=a.dry_run, again=a.again, progress=progress, log=log)
+        if a.json:
+            emit({"done": result})
+        elif result["dry_run"]:
+            print(f"dry run done: {result['files']} files would go to datasets/{result['repo']}/{result['path_in_repo']}")
+        else:
+            print(f"uploaded: {result['pr_url']}")
+        return 0
+    except HubError as e:
+        if a.json:
+            emit({"error": e.as_dict()})
+        else:
+            print(f"error ({e.kind}): {e.text}", file=sys.stderr)
+            for line in e.errors:
+                print(f"  {line}", file=sys.stderr)
+        return 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

hands/rec/install.sh

@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+# Install the hand recorder on a Frame that has Frametop (get.sh --experimental): bring the dev
+# container's packages up to date, build hand tracking's camera broker and tracker (hands/build.sh:
+# the first build fetches and builds ncnn, a few minutes) and the headset panel (hands/rec/build.sh),
+# give ft-camd its capabilities (hands/run.sh caps: asks for the password, once per build), and add
+# "Frametop Hand Recorder" to the app menu (for Frametop's desktop: it isn't tested from SteamVR's
+# "Launch a program", which runs apps outside it; the standalone recorder is for that).
+# It doesn't turn on Frametop's live hand tracking (that's hands/run.sh install, still deferred).
+# Usage: hands/rec/install.sh            install or update
+#        hands/rec/install.sh uninstall  remove the menu entry (recordings stay where they are)
+set -euo pipefail
+root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
+. "$root/scripts/_env.sh"
+entry='~/.local/share/applications/frametop-handrec.desktop'
+
+case ${1:-install} in
+  install)
+    echo "== 1/4 dev container packages"
+    "$root/setup/dev-container.sh"
+    echo "== 2/4 hand tracking: ft-camd and ft-hands"
+    "$root/hands/build.sh"
+    echo "== 3/4 the headset panel: ft-handpanel"
+    "$root/hands/rec/build.sh"
+    echo "== 4/4 ft-camd's capabilities (asks for your password) and the menu entry"
+    "$root/hands/run.sh" caps
+    "$root/scripts/conf-migrate.sh"   # HANDS_SWAP_SIDES=0, the old default, becomes auto
+    fill_template "$root/hands/rec/ft-handrec.desktop" |
+      on_frame "mkdir -p ~/.local/share/applications && cat > $entry"
+    echo
+    echo "Installed. On Frametop's desktop, open \"Frametop Hand Recorder\" from the app menu."
+    echo "Recordings go to ~/.local/share/frametop/hands/contrib."
+    ;;
+  uninstall)
+    on_frame "rm -f $entry"
+    echo "Removed the menu entry. Your recordings are still in ~/.local/share/frametop/hands/contrib:"
+    echo "delete that folder to remove them."
+    ;;
+  *) echo "usage: $0 [install|uninstall]" >&2; exit 2 ;;
+esac

hands/rec/poses/README.md

@@ -0,0 +1,52 @@
+# Pose images for the hand recorder
+
+Small pictures of each hand pose in `script.json`, shown on the headset panel next to the prompt text.
+
+- `<id>.png`: one image per pose, 512x512 RGBA with a transparent background. Made for a dark panel and still readable at about 250 px.
+- `poses.json`: maps each pose id to `{"file", "two_hands", "caption"}`.
+- `contact-sheet.png`: every image in a grid with its id, for review. The panel doesn't use it.
+- `make_poses.py`: the generator that makes all of the above.
+
+## How the images are drawn
+
+Every image shows a right hand as the wearer sees it from their own eyes. "Palm toward you" shows the palm, with its creases. "Back toward you" shows the back of the hand, with the nails and knuckles. Fingers point up unless the pose says otherwise.
+
+- **Left-hand prompts:** the panel mirrors the image horizontally.
+- **"Both" prompts:** the panel shows two copies, one of them mirrored.
+- **`two_hands: true`:** the image already shows the whole scene: both hands, or one hand with an object, as in `hold`. Show it once, without mirroring or copying. This applies to `cross`, `overlap`, `near-face`, `typing`, `lift`, `switch`, `hold`, `push`, `push-controller`, `touch-stick` and `touch-stick-desk`.
+- **`touch-stick`:** shows the left hand holding the controller while the right index touches its thumbstick. For prompts where the right hand holds the controller (`controllers: ["right"]`), mirror it.
+
+`push` and `push-controller` are side views: the wearer's head with a headset on, one arm out in front with the palm out, a ghost of the hand further out, and a straight double arrow from the headset outward, labelled "near" and "arm out". The labels use Pillow's built-in font.
+
+Other motion poses show the start pose, a faint blue "ghost" of the end pose, and orange arrows. Objects (keyboard, mouse, bottle, bar, controller, screen, head and headset) are plain grey shapes.
+
+Besides the pose ids in `script.json`, these extra ids exist for prompts that need a different picture:
+
+| id | for |
+| --- | --- |
+| `push` | the bar sections: push straight out from the headset and back, palms out |
+| `push-controller` | the same with controllers on |
+| `touch-stick-desk` | touching the thumbstick of a controller lying on the desk |
+| `no-hands` | the no-hands section: hands down, out of view |
+| `open-close` | already used by the bar sections |
+
+`count-5` is the same picture as `spread`.
+
+## Regenerating
+
+You need Python 3 with numpy and Pillow. The script uses about one core-minute per image at the default quality, so don't run it on the headset. Run it on a build machine:
+
+```sh
+python3 -m venv venv && venv/bin/pip install numpy pillow
+venv/bin/python make_poses.py --jobs 6              # all images, poses.json, contact sheet
+venv/bin/python make_poses.py --only fist,ok        # just some (poses.json is left alone)
+venv/bin/python make_poses.py --ss 1 --jobs 6       # rough and about 8x faster, for trying things
+```
+
+`--out DIR` writes somewhere other than this folder.
+
+Each pose is a short entry in the `POSES` table in `make_poses.py`: a caption, the `two_hands` flag, and a function that returns the scene. A scene is the camera, layers of hands and objects (a layer can be a faint ghost), and arrows. Hands are built from joint angles: flexion at each finger's three joints, sideways spread, and four thumb angles. A thumb can also be given a target point, such as "touch the index fingertip", and a small solver finds the angles. The presets near `FLAT`, `FIST` and `OK` are a good place to start a new pose.
+
+## License
+
+MIT, like the rest of the repository. The script draws everything itself. A hand made of a palm slab and tapered capsules is ray-marched as a signed distance field, then shaded and outlined. No photos, downloaded images, scanned or research hand models, or AI image generators are involved, so the images carry no other terms.

hands/rec/poses/poses.json

@@ -0,0 +1,182 @@
+{
+  "flat": {
+    "file": "flat.png",
+    "two_hands": false,
+    "caption": "Palm toward you, fingers together"
+  },
+  "flat-back": {
+    "file": "flat-back.png",
+    "two_hands": false,
+    "caption": "Back of the hand toward you"
+  },
+  "spread": {
+    "file": "spread.png",
+    "two_hands": false,
+    "caption": "Fingers spread wide"
+  },
+  "open": {
+    "file": "open.png",
+    "two_hands": false,
+    "caption": "Open and relaxed"
+  },
+  "fist": {
+    "file": "fist.png",
+    "two_hands": false,
+    "caption": "A fist"
+  },
+  "point": {
+    "file": "point.png",
+    "two_hands": false,
+    "caption": "Point with your index finger"
+  },
+  "pinch": {
+    "file": "pinch.png",
+    "two_hands": false,
+    "caption": "Thumb and index tips touching"
+  },
+  "ok": {
+    "file": "ok.png",
+    "two_hands": false,
+    "caption": "OK sign: a ring, three fingers up"
+  },
+  "thumbs-up": {
+    "file": "thumbs-up.png",
+    "two_hands": false,
+    "caption": "Thumbs up"
+  },
+  "claw": {
+    "file": "claw.png",
+    "two_hands": false,
+    "caption": "Fingers curled like a claw"
+  },
+  "count-1": {
+    "file": "count-1.png",
+    "two_hands": false,
+    "caption": "One: index finger"
+  },
+  "count-2": {
+    "file": "count-2.png",
+    "two_hands": false,
+    "caption": "Two: index and middle"
+  },
+  "count-3": {
+    "file": "count-3.png",
+    "two_hands": false,
+    "caption": "Three: index, middle, ring"
+  },
+  "count-4": {
+    "file": "count-4.png",
+    "two_hands": false,
+    "caption": "Four: thumb folded in"
+  },
+  "count-5": {
+    "file": "count-5.png",
+    "two_hands": false,
+    "caption": "Five: all fingers spread"
+  },
+  "turn-in-out": {
+    "file": "turn-in-out.png",
+    "two_hands": false,
+    "caption": "Turn your wrist: palm in, palm out"
+  },
+  "turn-up-down": {
+    "file": "turn-up-down.png",
+    "two_hands": false,
+    "caption": "Palm down, turn it up, then back"
+  },
+  "bend": {
+    "file": "bend.png",
+    "two_hands": false,
+    "caption": "Bend your wrist up, down, side to side"
+  },
+  "pinch-tap": {
+    "file": "pinch-tap.png",
+    "two_hands": false,
+    "caption": "Tap thumb and index together"
+  },
+  "pinch-drag": {
+    "file": "pinch-drag.png",
+    "two_hands": false,
+    "caption": "Pinch, move to the side, let go"
+  },
+  "grab": {
+    "file": "grab.png",
+    "two_hands": false,
+    "caption": "Close your hand around a bar, open it"
+  },
+  "open-close": {
+    "file": "open-close.png",
+    "two_hands": false,
+    "caption": "Open and close your hand"
+  },
+  "cross": {
+    "file": "cross.png",
+    "two_hands": true,
+    "caption": "Cross your hands, then uncross"
+  },
+  "overlap": {
+    "file": "overlap.png",
+    "two_hands": true,
+    "caption": "One hand over the other, slide around"
+  },
+  "near-face": {
+    "file": "near-face.png",
+    "two_hands": true,
+    "caption": "Hands near your face, not touching"
+  },
+  "screen-point": {
+    "file": "screen-point.png",
+    "two_hands": false,
+    "caption": "Point at a screen at arm's length"
+  },
+  "typing": {
+    "file": "typing.png",
+    "two_hands": true,
+    "caption": "Type on the keyboard"
+  },
+  "lift": {
+    "file": "lift.png",
+    "two_hands": true,
+    "caption": "Lift your hands off the keyboard, put them back"
+  },
+  "mouse": {
+    "file": "mouse.png",
+    "two_hands": false,
+    "caption": "Hand on the mouse, move and click"
+  },
+  "switch": {
+    "file": "switch.png",
+    "two_hands": true,
+    "caption": "Switch between keyboard and mouse"
+  },
+  "hold": {
+    "file": "hold.png",
+    "two_hands": true,
+    "caption": "Pick it up, use it, put it down"
+  },
+  "push": {
+    "file": "push.png",
+    "two_hands": true,
+    "caption": "Push straight out, away from the headset, and back"
+  },
+  "push-controller": {
+    "file": "push-controller.png",
+    "two_hands": true,
+    "caption": "Controllers on: push straight out from the headset, and back"
+  },
+  "touch-stick": {
+    "file": "touch-stick.png",
+    "two_hands": true,
+    "caption": "Touch the thumbstick with your other index"
+  },
+  "touch-stick-desk": {
+    "file": "touch-stick-desk.png",
+    "two_hands": true,
+    "caption": "Touch the thumbstick of the controller on the desk"
+  },
+  "no-hands": {
+    "file": "no-hands.png",
+    "two_hands": false,
+    "caption": "Hands down, out of view"
+  }
+}