Merge experimental: eye tracker first calibration, accessibility registry, hand recorder without the colour module (#37)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 01:00:06 +02:00 · 2026-10-05 14:38:43 -06:00
commit a0fe3bb55e
@@ -133,7 +133,7 @@ A few overlays need special handling:

 Head follow is experimental and off by default. It works, but it's only lightly tested, and the feel is mostly a matter of its settings; polishing it is left open. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction, where you were facing when your head last settled, and keeps its offset from it. The mouse can put the cursor anywhere up to `POINTER_FOLLOW_REACH` (70 degrees) from the reference, a corner of your view included. While your head stays within `POINTER_LEASH_DEG` of the reference, nothing moves on its own. Once your head has been past the leash for `POINTER_LEASH_DELAY` (0.2 s, so a glance out and back doesn't count), the reference eases to where you're facing (time constant `POINTER_LEASH_RETURN`, 0.2 s), never falling further behind than the leash, and the cursor ends up back where it was in your view. Then it waits for the leash again. Two earlier versions didn't work out. Moving the reference only while your head pulled at the end of the leash left it up to the leash off after you turned back, and getting it centred again meant overshooting with your head. Easing it toward your facing all the time moved the cursor on every small head movement. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping.

-Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. By default (`POINTER_GAZE_MOUSE_MOVE=held`, the Gaze page's Mouse movement switch) moving the mouse does nothing while the gaze has the pointer: it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so the lessons aren't polluted by mouse moves to somewhere else (they used to be kept out by an 8 degree limit, which also dropped real corrections when the tracker was further off). With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual; with `free`, moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: it needs accessibility (AT-SPI) on in the Frametop session, where it's off (no registry runs), plus app restarts, and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. The right button works the same way, with the right click on the release, and pressing it while the left press is held back starts a drag where the pointer is, like Meta+J then Meta+K. That drag lasts while either button (or key) is held, so a second right press, or a second Meta+K, is free to pan and tilt the panel being dragged; with the keyboard, the head turns it. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge before a click whose correction is within `POINTER_GAZE_NUDGE_MAX` (55 degrees, half of what the headset shows across) is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on, when our tracker thinks it moved, and when a correction is past that limit (the tracker is far off, so a click there isn't trusted as a lesson), and the full calibration and the headset fit check run in the same panel, so everything a user does to calibrate happens in one place in the headset; the gaze probe, a fullscreen GTK app, is the development tool. The limit was 8 degrees, which dropped every correction while our tracker was 12 off. Its dots sit at known directions from the headset, so the panel needs no screen geometry. The quick check's dot takes the gaze once it has held still, so what the tracker says doesn't have to be close for the capture to work. The full calibration's and the five-dot check's dots wait for a click while you look at the dot (a left click or Meta+J), because a steady gaze isn't always on the dot, and take the gaze held still up to the click; a right click or MetLine truncated
+Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. By default (`POINTER_GAZE_MOUSE_MOVE=held`, the Gaze page's Mouse movement switch) moving the mouse does nothing while the gaze has the pointer: it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so the lessons aren't polluted by mouse moves to somewhere else (they used to be kept out by an 8 degree limit, which also dropped real corrections when the tracker was further off). With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual; with `free`, moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: the session now starts an AT-SPI registry, but apps still need to expose useful accessibility trees (and may need restarting), and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. The right button works the same way, with the right click on the release, and pressing it while the left press is held back starts a drag where the pointer is, like Meta+J then Meta+K. That drag lasts while either button (or key) is held, so a second right press, or a second Meta+K, is free to pan and tilt the panel being dragged; with the keyboard, the head turns it. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge before a click whose correction is within `POINTER_GAZE_NUDGE_MAX` (55 degrees, half of what the headset shows across) is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on, when our tracker thinks it moved, and when a correction is past that limit (the tracker is far off, so a click there isn't trusted as a lesson), and the full calibration and the headset fit check run in the same panel, so everything a user does to calibrate happens in one place in the headset; the gaze probe, a fullscreen GTK app, is the development tool. The limit was 8 degrees, which dropped every correction while our tracker was 12 off. Its dots sit at known directions from the headset, so the panel needs no screen geometry. The quick check's dot takes the gaze once it has held still, so what the tracker says doesn't have to be close for the capture to work. The full calibration's and the five-dot check's dots wait for a click while you look at the dot (a left click or Meta+J), because a steady gaze isn't always on the dot, and take the gaze held still up to the click; a rightLine truncated

 Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place.

@@ -169,10 +169,22 @@ The Frame controllers can be mapped like mouse buttons, but they aren't input de

 ## The desktop session

-The session is modeled on SteamOS's `steamos-nested-desktop` and runs beside it. It has its own runtime directory, config (`~/.config/frametop`), and state, so it never disturbs the stock desktop's layout or panels. It runs on a private D-Bus from `dbus-run-session`, which has two consequences. KDE only launches apps in systemd scopes when systemd is on the session bus, so everything started in the desktop lands in its systemd unit, and stopping the unit would kill all of it; `session/keep-apps.sh` moves those programs out first. And tools that need the real user bus, like podman and `distrobox-host-exec`, have to be pointed at it explicitly.
+The session is modeled on SteamOS's `steamos-nested-desktop` and runs beside it. It has its own runtime directory, config (`~/.config/frametop`), and state, so it never disturbs the stock desktop's layout or panels. It runs on a private D-Bus from `dbus-run-session`, which has two consequences. KDE only launches apps in systemd scopes when systemd is on the session bus, so everything started in the desktop lands in its systemd unit, and stopping the unit would kill all of it; `session/keep-apps.sh` moves those programs out first. And tools that need the real user bus, like podman and `distrobox-host-exec`, have to be pointed at it explicitly. Its own config folder also hides SteamVR's path registry (`~/.config/openvr/openvrpaths.vrpath`) from everything started in it: OpenVR programs there fail with `VRInitError_Init_PathRegistryNotFound`, and `vrpathreg adddriver` writes a new registry under `~/.config/frametop/openvr` that has no SteamVR in it and that SteamVR never reads. So Frametop's scripts run SteamVR's tools with `XDG_CONFIG_HOME=~/.config`.

 The VR launcher starts the session from the Steam client, and the client's environment came along: `LD_LIBRARY_PATH` pointing at Steam's own runtime, whose `libavcodec` has no H.264 decoder, so VLC in the desktop couldn't play most videos, plus the client's overlay and launch settings. The session script drops the client's variables before it starts anything. SteamOS's global Mesa settings (`/usr/share/deckard/mesavars.sh`) stay, and the gamescope session's Vulkan layer (`ENABLE_GAMESCOPE_WSI`) is only kept for the gamescope backend.

+### Nested accessibility
+
+The session drops an inherited `AT_SPI_BUS_ADDRESS`, so apps cannot accidentally use the host desktop's registry. It autostarts `session/ft-atspi` in Plasma phase 2, after KWin has set the nested display environment. The helper gets the live accessibility address from `org.a11y.Bus` on the private session bus, preserves any existing registry owner, updates the accessibility bus's activation environment, and tries `StartServiceByName` first.
+
+On SteamOS 0.3.0 with at-spi2-core 2.52.0, the native launcher can choose dbus-broker because its process belongs to a systemd user unit. Registry activation then fails: this desktop's private session bus does not have a systemd activation manager. In that case the helper starts only `at-spi2-registryd` on the already-existing accessibility bus. The registry refuses duplicate ownership. Unlike native activation's `--use-gnome-session`, the fallback does not try to register with GNOME's session manager; that flag did not explain the observed native activation failure.
+
+The fallback registry does not exit merely when its bus disconnects in the isolated SteamOS test. Its small watcher checks both private buses every 5 seconds, and terminates and reaps only the child it started when either bus disappears or the watcher is stopped. Each check runs `gdbus` twice; once a second, that cost about 1% of a core. `keep-apps.sh` keeps the watcher in the desktop unit when `desktops.sh start` runs the desktop as `frametop-desktop`. Started from the VR launcher, the desktop runs in steam.service, which doesn't stop with it, so there the watcher is the only thing that stops the registry. There is no second accessibility bus, global systemd environment update, process-name kill, or host registry replacement. Missing accessibility files or bus errors are nonfatal; the desktop still starts. Toolkit-specific accessibility opt-ins and pointer snapping are separate work.
+
+Run the isolated checks on the host with `/usr/bin/python3 session/test/test_accessibility.py`. They use private D-Bus buses, Xvfb and a GTK3 app, never the production display or input. Native activation uses a small `org.a11y.Bus` test provider pointing to a real private dbus-daemon with the installed registry service; the SteamOS fallback uses the installed bus launcher and broker. The tests check real app-tree discovery, existing owners, concurrent starts, session stop/restart, and teardown. They require test-only PyGObject (Gio and GTK3), Xvfb, and at-spi2-core; the runtime helper uses Python's standard library and the existing host `gdbus`. Actual Plasma autostart and VR desktop restart still require an approved hardware test.
+
+### Other session behavior
+
 Steam, not systemd, suspends the Frame: after `system_idle_suspend_ac_sec` (an hour by default) without input on AC power, it logs `Switching to power state: k_ESystemPowerState_Sleep` and suspends, even while charging. It's a Steam setting (Settings → Power → When Plugged In and Idle → Sleep after), which the Stay awake while plugged in switch in Frametop Display Settings sets to Never. SteamVR's standby, which turns the displays off when the headset comes off, is separate; see below.

 Flatpak apps need `XDG_DATA_DIRS` to include Flatpak's exports, or Plasma opens Discover instead of launching them, so the session sources `/etc/profile.d/flatpak.sh`.
@@ -18,6 +18,8 @@ desktops.sh start | stop | restart | status | log [lines]

 When the VR launcher starts the desktop, it inherits the Steam client's environment. The session script drops the client's runtime from it (`LD_LIBRARY_PATH`, the `STEAM_*` settings, and the Steam overlay's Vulkan layer), so apps in the desktop use the system's libraries, including its video codecs, just as they would after a normal login.

+The nested session also starts an AT-SPI accessibility registry through `session/ft-atspi` in Plasma's autostart. It discovers the bus from this session, ignores an inherited host accessibility address, and leaves an existing registry alone. Accessibility errors do not stop the desktop. This supplies the registry infrastructure for apps that expose AT-SPI trees; it does not enable gaze snapping or force Chromium/Electron accessibility. After an approved desktop restart, an AT-SPI-aware app should be visible on the nested bus. See [design.md](design.md#nested-accessibility) for native activation, fallback lifecycle, and the isolated test command.
+
 KWin's blur and background contrast effects and its animations are off in this desktop, because KWin draws on the headset's GPU, which SteamVR needs. The session script turns them off once, the first time it starts (it leaves a setting you already have alone, and marks it done in `~/.config/frametop/frametoprc`), so turning them back on sticks. In the Frametop desktop, System Settings → Window Management → Desktop Effects has Blur and Background Contrast, and General Behavior has Animation speed. Or from a terminal, then restart the desktop:

 ```
@@ -33,7 +33,7 @@ Gaze as an input method for the whole desktop, without replacing anything of Ste
 - **The mouse only corrects** (the default; the Gaze page's Mouse movement switch, `POINTER_GAZE_MOUSE_MOVE=held`): while the gaze has the pointer, moving the mouse does nothing. The buttons work like Meta+J and Meta+K: press and hold one and the pointer stops where you look; move the mouse onto what you meant and let go to click there (a left or a right click). Held still for half a second, a press is a real one (to drag). Once you've moved, the left button alone only clicks: press the right one while still holding the left to start a drag there; it lasts while either button is held. Press the right one again (a double right click, the left still held) to pan and tilt what you're dragging, as a right press does during any drag. A bumped or drifting mouse can't pull the pointer away, and every mouse move is a correction, so the tracker only learns from real ones. With the gaze stale for a second (the tracker stopped, eyes lost), in a game, or with the headset off, the mouse moves the pointer as usual. `free` (the switch off) lets the mouse take the pointer any time.
 - **Keyboard clicks** (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, `POINTER_KEY_TAP`) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse, under the same limit: past `POINTER_GAZE_NUDGE_MAX` it opens the quick check instead). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; the drag lasts while either key is held. Meta+K during a Meta+J drag (again, after starting it with Meta+K: a double Meta+K) pans and tilts what you're dragging while it's held: turn your head to turn it.
 - **Learning from nudges:** if the mouse took the pointer from the gaze and moved it (0.2 degrees or more, and the correction within `POINTER_GAZE_NUDGE_MAX`: 55 degrees by default, half of the 109 the headset shows across, and 1 to 110; the same limit for mouse, keyboard, and pinch clicks) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the quick check's dot does the same). A correction past `POINTER_GAZE_NUDGE_MAX` isn't learned: the helper asks ft-gazed for the quick check instead ("recheck", after its 2-minute cooldown). Tested on our tracker's 409 clicks since its Sep 29 calibration: a one-dot check set from any one of them put the next 2 minutes' clicks within 15 degrees (99% within 4.2) and the next 10 minutes' within 25 (the far ones after the headset moved), so the check gets back well under it. The limit used to be 8 degrees, and live on 2026-10-01 our tracker was 12 off after the headset went on, so every correction was dropped. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
+- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. Our tracker's first calibration has no gaze to go on, since ft-eyes maps pupils to a gaze only once it has a calibration: it opens once SteamVR's tracker sees an eye and ft-eyes answers, and a click takes the 0.6 s up to it, as long as ft-eyes saw each pupil held still then (before 2026-10-05 it waited for a gaze, so a fresh install could never calibrate ours). A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
 - Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
 - **Idle while the gaze isn't used:** ft-gaze and our own tracker run only while gaze mode is on and someone wears the headset (the pointer helper says both: SteamVR drops the headset's activity level as soon as it comes off), while a check or the calibration is open or asked for, or while the Gaze page of Frametop Input Settings is open (it renews a `wake` lease). 30 seconds after the last use they stop, and our frame grabber goes idle with our tracker. With our tracker, that saves over half a core: on 2026-10-02, with gaze mode off, ft-eyes took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. A check asked for while it idles starts the tracker and opens once it sends. When the gaze is used again, it takes a few seconds to come back, and our tracker's first click re-seats it, as after the headset was off: so the quick check opens when gaze mode comes on after the service idled, as it does when you put the headset on. `ft-gazectl status` says `"awake"`, and `"idle"` says why it isn't. A stand that covers the proximity sensor makes the headset seem worn, so with gaze mode on it doesn't idle there.

@@ -48,6 +48,13 @@ click with nothing taken after ACCEPT_WAIT, and a failed calibration names its m
 reason there and in the status. A right click or Meta+K ("calquit") closes the panel. The pointer hides meanwhile ("calpanel 1",
 renewed every second; the helper shows it again by itself when that stops).

+Our tracker's first calibration: before it has one, ft-eyes publishes no gaze (it maps pupils
+to a gaze only with a calibration), so there's no gaze to hold still. Its calibration runs
+anyway ("blind"): someone in the headset (SteamVR's tracker sees an eye) and ft-eyes answering
+are enough to start it, each dot stands in for the gaze, and a click takes the CHECK_WINDOW up
+to it. ft-eyes then checks that each pupil was seen and held still in that window (calib-point)
+and says why not. Without this, a fresh install could never calibrate our tracker.
+
 What a capture teaches:
  our tracker   quick and five: a click ("click T YAW PITCH", like a pointer lesson); full:
                calib-start, a calib-point for each dot, calib-fit (its calibration)
@@ -336,9 +343,17 @@ class Checks:
        return time.monotonic() - self.seen_at < within

    def can_run(self):
-        """Someone's in the headset and the tracker is sending."""
+        """Someone's in the headset and the tracker is sending. Our tracker sends no gaze before
+        its first calibration: then ft-eyes answering (calibrated() is False only once it has)
+        is enough, since the calibration is what it needs (see "first calibration" at the top)."""
+        if self.blind():
+            return self.eyes_seen()
        return self.eyes_seen() and time.monotonic() - self.svc.last_sample < 2

+    def blind(self):
+        """Our tracker is in use, running, and not calibrated yet: it has no gaze to send."""
+        return self.svc.kind == "own" and self.calibrated() is False
+
    def on_gaze_on(self):
        self.full_armed = True
        self.need_full("gaze mode came on without a calibration")
@@ -356,7 +371,7 @@ class Checks:
        if not self.can_run() and self.svc.waking():
            return  # the tracker is still starting (the service idled)
        if not self.can_run():
-            why = ("the eye tracker isn't sending" if now - self.svc.last_sample >= 2
+            why = ("the eye tracker isn't sending" if now - self.svc.last_sample >= 2 and not self.blind()
                   else "no eyes seen (is the headset on?)")
        elif now < self.full_retry_at:
            return
@@ -408,6 +423,9 @@ class Checks:
        if not self.can_run():
            return "error the headset is off or the tracker isn't sending"
        own = svc.kind == "own"
+        blind = self.blind()
+        if blind and kind != "full":
+            return "error our tracker isn't calibrated yet: use Calibrate"
        if kind == "full" and own:
            reply = ask(EYES, "calib-start", 3.0)
            if not reply.startswith("ok"):
@@ -416,8 +434,10 @@ class Checks:
        now = time.monotonic()
        self.check = {"kind": kind, "reason": reason, "own": own, "dots": check_dots(kind, own), "i": 0,
                      "started": now, "shown": now, "run": [], "accept": False, "done_at": None, "tries": 0,
-                      "skipped": 0, "captured": 0, "points": {}, "reasons": {}, "fit_reasons": set(), "note": ""}
-        log(f"{kind} check: {reason}")
+                      "skipped": 0, "captured": 0, "points": {}, "reasons": {}, "fit_reasons": set(), "note": "",
+                      "blind": blind}
+        log(f"{kind} check: {reason}" + (" (our tracker's first: no gaze yet, so each click takes the look "
+                                         "up to it)" if blind else ""))
        if kind == "full":
            st = ask(SCREENS, "state", 0.5).split()
            if len(st) >= 3 and st[0] == "ok":
@@ -519,6 +539,12 @@ class Checks:
            return
        if c["own"]:
            src = s["src"].get("own") or {}
+            if c["blind"]:
+                # Our tracker's first calibration: no gaze yet, so the dot stands in for it (the
+                # gaze can't seem to move) and a click takes the look up to it. ft-eyes checks
+                # the pupils held still (see the top).
+                yaw, pitch, _ = c["dots"][c["i"]]
+                src = {"hy": yaw, "hp": pitch}
        else:
            src = s["src"].get("mmap1") or {}
            if "hy" not in src:
@@ -868,7 +894,11 @@ class Checks:
            self.back_since = None  # gone again before DON_DELAY
        elif self.back_since is not None and now - self.back_since >= DON_DELAY:
            self.away, self.back_since = False, None
-            self.full_armed = True  # a calibration that closed unfinished opens again
+            if not self.check:
+                # A calibration that closed unfinished opens again. Not one still open: gaze mode
+                # coming on wakes our tracker, so its eyes come back just as the calibration
+                # opens, and re-arming then opened a second one when the first ended.
+                self.full_armed = True
            self.auto_quick("the headset went on")
        if svc.kind == "own" and now - svc.own_at < 5:
            reseat = any(e.get("reseat") for e in (svc.own.get("eyes") or {}).values())
@@ -0,0 +1,238 @@
+#!/usr/bin/env python3
+"""Offline test of our own eye tracker's first calibration (gaze/gazecheck.py, "blind"): before
+it has a calibration, ft-eyes publishes no gaze, and the calibration must still open and take
+its dots. Until 2026-10-05 it couldn't: a fresh install that chose our tracker never calibrated.
+
+Runs ft-gazed's Service with its sockets renamed and HOME in a temp folder (state and settings
+go there), a fake pointer helper (gaze mode on, headset worn), a fake ft-gaze (SteamVR sees both
+eyes; "own" is {"ok":0}, as with an uncalibrated ft-eyes), a fake ft-eyes control socket, and no
+panel (a stand-in process). Nothing reaches the live gaze service, the pointer helper, ft-eyes,
+or SteamVR, so it's safe next to them.
+
+  gaze/test/first-calibration-test.py
+"""
+import os
+import tempfile
+
+HOME = tempfile.mkdtemp(prefix="ft-gaze-first-cal-test-")
+os.environ["HOME"] = HOME  # before gazecal: its STATE, and frametop.conf, follow HOME
+
+import importlib.machinery  # noqa: E402
+import importlib.util  # noqa: E402
+import json  # noqa: E402
+import selectors  # noqa: E402
+import shutil  # noqa: E402
+import socket  # noqa: E402
+import subprocess  # noqa: E402
+import sys  # noqa: E402
+import threading  # noqa: E402
+import time  # noqa: E402
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+GAZE = os.path.join(HERE, "..")
+sys.path.insert(0, GAZE)
+loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(GAZE, "ft-gazed"))
+gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
+loader.exec_module(gazed)
+import gazecheck  # noqa: E402  (the module ft-gazed imported)
+
+tag = f"ft_gaze_first_cal_test_{os.getpid()}"
+gazed.ME = f"\0{tag}_gazed"
+gazed.POINTER = gazecheck.POINTER = f"\0{tag}_helper"
+gazecheck.SCREENS = f"\0{tag}_screens"
+gazecheck.PANEL = f"\0{tag}_panel"
+gazed.EYES_SOCKET = gazecheck.EYES = f"\0{tag}_eyes"
+gazed.read_settings = lambda: ("own", "auto", "auto", 55.0)
+DOTS = 3
+real_dots = gazecheck.check_dots
+gazecheck.check_dots = lambda kind, own: real_dots(kind, own)[:DOTS]  # a short calibration
+logs = []
+gazed.log = gazecheck.log = lambda msg: logs.append(msg)
+
+# The fake ft-gaze: 90 samples a second, SteamVR sees both eyes, no gaze from ours.
+FAKE = os.path.join(HOME, "ft-gaze")
+with open(FAKE, "w") as f:
+    f.write('''import json, os, select, sys, time
+while True:
+    if select.select([sys.stdin], [], [], 1 / 90)[0]:
+        if not os.read(0, 4096):
+            break
+    print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
+          "open": [0.8, 0.8]}, "own": {"ok": 0}}}), flush=True)
+''')
+SLEEPER = [sys.executable, "-c", "import sys; sys.stdin.read()"]  # quits when its stdin closes
+
+
+def start_helper(self):
+    """ft-gaze, straight from here instead of the dev container."""
+    self.proc = subprocess.Popen([sys.executable, FAKE], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+                                 stderr=subprocess.PIPE)
+    self.proc_sources = self.wanted_sources()
+    os.set_blocking(self.proc.stdout.fileno(), False)
+    os.set_blocking(self.proc.stderr.fileno(), False)
+    self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
+    self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
+    self.buf = b""
+
+
+def start_eyes(self):
+    """ft-eyes' process: a stand-in. Its control socket is the fake below."""
+    self.eyes_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+    os.set_blocking(self.eyes_proc.stderr.fileno(), False)
+    self.sel.register(self.eyes_proc.stderr, selectors.EVENT_READ, "eyes")
+
+
+def start_panel(self):
+    self.panel_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+    os.set_blocking(self.panel_proc.stderr.fileno(), False)
+    self.sel.register(self.panel_proc.stderr, selectors.EVENT_READ, "panel")
+
+
+gazed.Service.start_helper = start_helper
+gazed.Service.start_eyes = start_eyes
+gazecheck.Checks.start_panel = start_panel
+
+# The fake ft-eyes: uncalibrated until calib-fit. "fail" answers the next calib-point with that.
+eyes_state = {"cal": None, "points": [], "fail": None}
+eyes = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+eyes.bind(gazed.EYES_SOCKET)
+eyes.settimeout(0.2)
+
+
+def eyes_answer():
+    while True:
+        try:
+            data, addr = eyes.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        w = data.decode().split()
+        if w[0] == "status":
+            reply = json.dumps({"calibration": eyes_state["cal"], "calibrating": False, "dots": 0,
+                                "eyes": {"right": {"reseat": False}, "left": {"reseat": False}}})
+        elif w[0] == "calib-start":
+            eyes_state["points"] = []
+            reply = "ok"
+        elif w[0] == "calib-point":
+            eyes_state["points"].append(tuple(map(float, w[1:5])))
+            reply, eyes_state["fail"] = eyes_state["fail"] or "ok 50 50 1.00 1.00", None
+        elif w[0] == "calib-fit":
+            eyes_state["cal"] = {"made": "test", "dots": len(eyes_state["points"])}
+            reply = f"ok {len(eyes_state['points'])} dots"
+        else:
+            reply = f"fail unknown command {w[0]}"
+        if addr:
+            eyes.sendto(reply.encode(), addr)
+
+
+threading.Thread(target=eyes_answer, daemon=True).start()
+
+helper_state = {"reply": "ok off worn", "heard": []}
+helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+helper.bind(gazed.POINTER)
+helper.settimeout(0.2)
+
+
+def helper_answer():
+    while True:
+        try:
+            data, addr = helper.recvfrom(512)
+        except socket.timeout:
+            continue
+        except OSError:
+            return
+        if data.startswith(b"gaze ?") and addr:
+            helper.sendto(helper_state["reply"].encode(), addr)
+        else:
+            helper_state["heard"].append(data.decode())
+
+
+threading.Thread(target=helper_answer, daemon=True).start()
+svc = gazed.Service(None, False, gazed.POINTER)
+threading.Thread(target=svc.run, daemon=True).start()
+ctl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+ctl.bind("")
+ctl.settimeout(3)
+
+
+def ask(cmd):
+    ctl.sendto(cmd.encode(), gazed.ME)
+    return ctl.recv(65536).decode()
+
+
+failures = []
+
+
+def check(label, got, want):
+    ok = got == want
+    print(("ok    " if ok else "FAIL  ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
+    if not ok:
+        failures.append(label)
+
+
+def wait(cond, seconds):
+    end = time.monotonic() + seconds
+    while time.monotonic() < end:
+        if cond():
+            return True
+        time.sleep(0.05)
+    return cond()
+
+
+def check_state():
+    return svc.checks.check or {}
+
+
+def take_dot(i):
+    """Wait for dot i to settle, then click: is it taken?"""
+    wait(lambda: check_state().get("i") == i and not check_state().get("done_at"), 3)
+    time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1)
+    before = check_state().get("captured", 0)
+    ask("calaccept")
+    return wait(lambda: check_state().get("captured", 0) > before, 2)
+
+
+check("gaze mode off, Gaze page open (wake): ours runs, uncalibrated", ask("wake 60"), "ok")
+check("the service knows ours has no calibration", wait(lambda: svc.checks.calibrated() is False, 6), True)
+check("a quick check is refused while ours has no calibration", svc.checks.start("quick", "test"),
+      "error our tracker isn't calibrated yet: use Calibrate")
+helper_state["reply"] = "ok on worn"
+check("gaze mode on: the calibration opens by itself", wait(lambda: check_state().get("kind") == "full", 6), True)
+check("it runs blind (no gaze from ours yet)", check_state().get("blind"), True)
+check("nothing says it can't open", any("can't open" in m for m in logs), False)
+# Live 2026-10-05: turning gaze mode on woke our tracker, and the calibration opened before
+# DON_DELAY of eyes had passed, so "the headset went on" came while it was open. That re-armed
+# it, and a second calibration opened as soon as the first ended.
+svc.checks.away, svc.checks.back_since = True, None
+
+check("dot 1: a click takes it", take_dot(0), True)
+check("the headset went on while it was open",
+      wait(lambda: not svc.checks.away, gazecheck.DON_DELAY + 2) and bool(svc.checks.check), True)
+t0, t1, yaw, pitch = eyes_state["points"][0]
+dot = gazecheck.check_dots("full", True)[0]
+check("its window is the look up to the click (about CHECK_WINDOW)",
+      abs((t1 - t0) - gazecheck.CHECK_WINDOW) < 0.15, True)
+check("ft-eyes got that dot's direction", (round(yaw, 3), round(pitch, 3)), (round(dot[0], 3), round(dot[1], 3)))
+
+eyes_state["fail"] = "fail the left eye was seen in only 3 frames"
+wait(lambda: check_state().get("i") == 1 and not check_state().get("done_at"), 3)
+time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1)
+ask("calaccept")
+check("ft-eyes refusing a dot: its reason reaches the panel's note",
+      wait(lambda: "left eye in only 3 frames" in check_state().get("note", ""), 2), True)
+check("dot 2, second try: taken", take_dot(1), True)
+check("dot 3: taken", take_dot(2), True)
+
+check("all dots: ours fits its calibration (calib-fit)",
+      wait(lambda: eyes_state["cal"] is not None and not svc.checks.check, 4), True)
+check("the service sees it calibrated", wait(lambda: svc.checks.calibrated() is True, 4), True)
+check("and gaze mode stays on", "gaze off" in helper_state["heard"], False)
+check("and no second calibration opens", wait(lambda: svc.checks.check is not None, 3), False)
+check("one calibration in the log", sum(m.startswith("full check:") for m in logs), 1)
+
+print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
+svc.running = False
+time.sleep(0.7)
+shutil.rmtree(HOME, ignore_errors=True)
+os._exit(1 if failures else 0)
@@ -6,7 +6,7 @@ title: Install the Frametop Hand Recorder

 The Hand Recorder records your hands with the Steam Frame's tracking cameras for Frametop's open hand dataset ([DeeJanuz/frametop-hands](https://huggingface.co/datasets/DeeJanuz/frametop-hands) on Hugging Face). These are the Konsole commands to install it.

-> **Known issue (October 5, 2026):** the recorder currently works only on headsets with the Arcturus color passthrough module attached. Without the module, the camera check stops with "Not all of the headset's tracking cameras are running (ft-camd publishes only 2 of 4 mono cameras ...)", and restarting SteamVR or the headset doesn't help. A fix for headsets without the module will be published within 24 hours (by October 6). Once it's out, run the commands under [Update](#update).
+> **Fixed (October 5, 2026):** the recorder now works on headsets without the Arcturus color passthrough module too. If you installed it earlier and the camera check stopped with "Not all of the headset's tracking cameras are running (ft-camd publishes only 2 of 4 mono cameras ...)", run the commands under [Update](#update).

 Join the [Frametop Discord](https://discord.gg/W3X9f7z3Bc) for questions and help with recording.

@@ -56,13 +56,19 @@ ANSI = re.compile(r"\x1b\[[0-9;]*m")
 STAMP = re.compile(r"^\w{3} \w{3} \d{2} \d{4} (\d{2}:\d{2}:\d{2})\.\d+ (\w+): ?(.*)$")
 # Lines worth reading; anything else is skipped before the regexes (the log grows by MBs a day).
 KEYS = ("FPGA", "VCINT", "Created", "TrackingCameraInit", "Closing tracking camera", "Streaming",
-        "systemd suspend", "systemd resume", "XRService logging to", "Exiting XRService")
+        "systemd suspend", "systemd resume", "XRService logging to", "Exiting XRService", "ISP ")
+# XRService's numbering of its tracking cameras (the TrackingCameraInit index): ft-hands names
+# them this way too (track/main.cpp, cameras_from_xrservice_log).
+NAMES = ("slam_left", "slam_right", "upper_left", "upper_right")
 RE_PASSTHRU = re.compile(r"Passthrough connected but FPGA is (\S+) - loading VCINT")
 RE_INTERLEAVE = re.compile(r"Upper cameras FPGA interleaving support: (\d)")
 RE_TASKS = re.compile(r"Created (\d+) tasks \((\d+) tracking, (\d+) passthrough\)")
 RE_INIT = re.compile(r"TrackingCameraInit: index: (\d+)\. video device: /dev/video(\d+)")
 RE_STREAM = re.compile(r"Streaming resumed \(FPGA: (\S+), VC interleaving: (\w+)\)")
 RE_STATE = re.compile(r"FPGA state check: (\S+)")
+# Without the colour module XRService runs the side cameras through the ISP, as NV12 on other
+# capture pipes (vfe0 and vfe1), and the upper pair on vfe3 and vfe4.
+RE_ISP = re.compile(r"ISP (enabled|disabled) for tracking cameras")


 class LogState:
@@ -85,7 +91,7 @@ class LogState:
    def _new_episode(self, t):
        self.closed = False
        self.episode = {"start": t, "fpga_before": "", "vcint": "", "interleave": None, "tasks": None,
-                        "inits": {}, "stream": "", "failure": "", "evidence": []}
+                        "inits": {}, "stream": "", "isp": None, "failure": "", "evidence": []}
        if self.closed_at:
            self.episode["evidence"].append(self.closed_at)

@@ -146,6 +152,12 @@ class LogState:
            ep["interleave"] = int(m.group(1))
            ep["evidence"].append(short)
            return
+        m = RE_ISP.search(text)
+        if m:
+            ep = self._ep(t)
+            ep["isp"] = m.group(1) == "enabled"
+            ep["evidence"].append(short)
+            return
        m = RE_TASKS.search(text)
        if m:
            ep = self._ep(t)
@@ -184,6 +196,12 @@ class LogState:
        got = tuple(self.nodes[i] for i in (2, 3) if i in self.nodes)
        return got if len(got) == 2 else UPPER_NODES

+    def camera_map(self):
+        """{calibration name: /dev/videoN's N} from the latest camera start's TrackingCameraInit
+        lines (the whole log's when that start has none yet)."""
+        inits = (self.episode or {}).get("inits") or self.nodes
+        return {NAMES[i]: node for i, node in sorted(inits.items()) if 0 <= i < len(NAMES)}
+
    def tracking_nodes(self):
        got = tuple(self.nodes[i] for i in range(TRACKING) if i in self.nodes)
        return got if len(got) == TRACKING else SIDE_NODES + UPPER_NODES
@@ -353,7 +371,9 @@ def check(log=None, proc=True, ring=True, ring_path=None):
        status, reason = "unknown", "no XRService log in %s" % LOG_DIR
    out = {"log": path, "xrservice": None, "ring": None,
           "episode": state.snapshot()["episode"] if state else {},
-           "failure": state.snapshot()["failure"] if state else ""}
+           "failure": state.snapshot()["failure"] if state else "",
+           "map": {name: {"node": n, "pipe": pipe_name(n)} for name, n in state.camera_map().items()} if state else {},
+           "ring_missing": []}

    if proc:
        if pid is None:
@@ -389,13 +409,30 @@ def check(log=None, proc=True, ring=True, ring_path=None):
            evidence.append("ft-camd (pid %d, %s): %d mono cameras: %s" % (
                r["writer_pid"], "running" if r["alive"] else "stale ring", len(r["mono"]), names))
            if r["alive"] and len(r["mono"]) < TRACKING and status == "ok":
-                status, reason = "degraded", ("ft-camd publishes only %d of %d mono cameras (it started while "
-                                              "they were missing: restart it)" % (len(r["mono"]), TRACKING))
+                have = {c["node"] for c in r["mono"]}
+                want = state.tracking_nodes() if state else SIDE_NODES + UPPER_NODES
+                out["ring_missing"] = [n for n in want if n not in have]
+                status, reason = "degraded", ("ft-camd publishes only %d of %d mono cameras, missing %s" % (
+                    len(r["mono"]), TRACKING, " ".join("video%d" % n for n in out["ring_missing"]) or "?"))
    out.update(status=status, reason=reason, evidence=evidence,
               summary="ok" if status == "ok" else "%s: %s" % (status, reason))
    return out


+def pipe_name(node):
+    """The capture pipe behind /dev/videoN (msm_vfe3_video0 and so on), or ""."""
+    try:
+        with open("/sys/class/video4linux/video%d/name" % node) as f:
+            return f.read().strip()
+    except OSError:
+        return ""
+
+
+def is_ring_short(result):
+    """XRService runs all the tracking cameras, but ft-camd doesn't publish them all."""
+    return bool(result) and result.get("status") == "degraded" and bool(result.get("ring_missing"))
+
+
 def is_vcint_failure(result):
    return bool(result) and result.get("status") == "degraded" and result.get("reason") == VCINT_REASON

@@ -379,9 +379,15 @@ static bool resolve_block(cam_t *c)
    return true;
 }

+/*
+ * Through the ISP (no colour module), the near-black exposures come out all zeros,
+ * the same every time, so their dequeues change no buffer and get no votes. Such
+ * an index is left unmapped (slot -1): on_frame counts its frames as dark without
+ * reading them. Most of a camera's indices silent means it isn't streaming yet.
+ */
 static bool resolve_each(cam_t *c)
 {
-    int depth = c->maxindex + 1;
+    int depth = c->maxindex + 1, silent = 0;
    bool taken[MAX_SLOTS] = { false };

    for (int i = 0; i < depth; i++) {
@@ -396,6 +402,12 @@ static bool resolve_each(cam_t *c)
            }
        }

+        if (c->nobs[i] >= 3 && v1 <= 0.2 * c->nobs[i]) {
+            c->slot_of[i] = -1;
+            silent++;
+            continue;
+        }
+
        if (c->nobs[i] < 3 || best < 0 || taken[best] || v1 < 0.8 * c->nobs[i] || v2 > 0.3 * c->nobs[i])
            return false;

@@ -403,9 +415,14 @@ static bool resolve_each(cam_t *c)
        c->slot_of[i] = best;
    }

+    if (silent * 2 > depth)
+        return false;
+
    printf("%s: queue depth %d, buffers mapped one by one:", c->slug, depth);
    for (int i = 0; i < depth; i++)
-        printf(" %d", c->slot_of[i]);
+        c->slot_of[i] < 0 ? printf(" -") : printf(" %d", c->slot_of[i]);
+    if (silent)
+        printf(" (- : %d indices whose frames are all zeros, skipped as dark)", silent);
    printf("\n");
    return true;
 }
@@ -677,6 +694,12 @@ static void on_frame(cam_t *c, int64_t index, uint32_t seq, uint64_t ts, uint64_

    int slot = c->slot_of[index];

+    if (slot < 0) {     /* an index whose frames are all zeros (resolve_each): a dark one */
+        c->dark++;
+        c->rc->dropped++;
+        return;
+    }
+
    /* color runs at 60 fps: skip frames early enough that the asked rate holds, before any sync */
    if (c->color && color_fps > 0 && evtime - c->last_pub_ns < (uint64_t)(1e9 / color_fps) - 3000000) {
        c->paced++;
@@ -544,22 +544,26 @@ static void probe_cameras(xr_state_t *st)
 /*
 * qcom-camss can report bytesperline as the visible width while the VFE
 * writes a larger aligned pitch. sizeimage is right, so derive the pitch.
+ * For NV12, plane 0 normally holds the chroma rows after the luma (the side
+ * cameras through the ISP: 1056 wide, 1152 bytes a row); if it's too small for
+ * that, it holds the luma alone.
 */
 unsigned xr_camera_stride(const xr_camera_t *c)
 {
    if (!c->height || !c->planesize[0])
        return c->bytesperline ? c->bytesperline : c->width;

-    double bpp = 1.0;
-
-    if (c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21)
-        bpp = 1.5;
-
-    unsigned s = (unsigned)((double)c->planesize[0] / ((double)c->height * bpp));
+    bool yuv = c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21;
+    unsigned s = (unsigned)((double)c->planesize[0] / ((double)c->height * (yuv ? 1.5 : 1.0)));

    if (s >= c->width && s <= c->width * 4)
        return s;

+    s = (unsigned)(c->planesize[0] / c->height);
+
+    if (yuv && s >= c->width && s <= c->width * 4)
+        return s;
+
    return c->bytesperline ? c->bytesperline : c->width;
 }

@@ -591,7 +595,20 @@ void xr_camera_layout(const xr_camera_t *c, xr_layout_t *l)
    l->pitch = xr_camera_stride(c);
    l->width = c->width < l->pitch ? c->width : l->pitch;

-    if (c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21) {
+    /*
+     * Without the colour module, XRService runs the side cameras through the
+     * ISP ("ISP enabled for tracking cameras (main VFE available)" in its log),
+     * and they come out NV12. They're mono sensors, so the luma is the image.
+     */
+    bool yuv = c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21;
+
+    if (yuv && c->role && !strcmp(c->role, "tracking")) {
+        l->fmt  = XR_FMT_GREY8;
+        l->rows = c->height;
+        return;
+    }
+
+    if (yuv) {
        l->fmt  = XR_FMT_NV12;
        l->rows = c->height + c->height / 2;
    } else {
@@ -457,11 +457,12 @@ class Backend(QObject):
        """Start stays off: the check found the cameras degraded (unless --ignore-cameras)."""
        return not self._ignore_cameras and self._camera.get("status") == "degraded"

-    def _check_now(self):
+    def _check_now(self, repair=False):
+        """repair (off this thread only: it may restart ft-camd, up to 15 s): see session.camera_check."""
        mod = self._runner()
        if not mod or self._session_options.get("dry_run"):
            return {"status": "unknown", "summary": "not checked (dry run)", "reason": "dry run", "evidence": []}
-        return mod.camera_check()
+        return mod.camera_check(repair=repair)

    @Slot()
    def checkCameras(self):
@@ -470,7 +471,7 @@ class Backend(QObject):
            return
        self._camera_busy = True
        self.cameraChanged.emit()
-        self._thread(lambda: self._cameraArrived.emit(self._check_now()))
+        self._thread(lambda: self._cameraArrived.emit(self._check_now(repair=True)))

    def _on_camera(self, result):
        self._camera = result
@@ -499,7 +500,7 @@ class Backend(QObject):
                end = time.monotonic() + RECHECK_FOR_S
                time.sleep(RECHECK_S * 2)
                while time.monotonic() < end:
-                    result = self._check_now()
+                    result = self._check_now(repair=True)
                    # Done once a new XRService (a new log) has opened its cameras, ok or not.
                    if result.get("status") in ("ok", "degraded") and result.get("log") != before:
                        break
@@ -1153,11 +1153,33 @@ def _steamvr_version():
    return ""


-def camera_check():
-    """camcheck.check() (the XRService log, its open cameras where readable, ft-camd's ring);
-    never raises: a failure is "unknown"."""
+_camd_restarted = set()   # ft-camd pids camera_check(repair=True) has restarted: once each
+
+
+def _unit_pid(unit):
+    r = subprocess.run(host_command("systemctl", "--user", "show", "-p", "MainPID", "--value", unit),
+                       capture_output=True, text=True, timeout=30)
    try:
-        return camcheck.check()
+        return int(r.stdout.strip() or 0)
+    except ValueError:
+        return 0
+
+
+def camera_check(repair=False):
+    """camcheck.check() (the XRService log, its open cameras where readable, ft-camd's ring);
+    never raises: a failure is "unknown". repair (the window, never during a session): when
+    XRService runs every tracking camera but the recorder's own ft-camd doesn't publish them all
+    (it started while some were missing), restart it, once per ft-camd, and check again."""
+    try:
+        r = camcheck.check()
+        pid = (r.get("ring") or {}).get("writer_pid")
+        if repair and camcheck.is_ring_short(r) and pid not in _camd_restarted and _unit_pid(CAMD_UNIT) == pid:
+            _camd_restarted.add(pid)
+            stop_unit(CAMD_UNIT)
+            start_camd()
+            r = camcheck.check()
+            r["evidence"].append("restarted ft-camd (pid %d) because it published only some of the cameras" % pid)
+        return r
    except Exception as e:
        return {"status": "unknown", "summary": "unknown: the camera check failed (%s)" % e,
                "reason": str(e), "evidence": []}
@@ -1169,6 +1191,10 @@ def camera_text(result):
        return ""
    if camcheck.is_vcint_failure(result):
        return camcheck.USER_TEXT
+    if camcheck.is_ring_short(result):
+        return ("The headset's tracking cameras are all running, but the recorder can't read some of them (%s). "
+                "Close the Hand Recorder, run ~/frametop/hands/rec/install.sh again, and open it again. If that "
+                "doesn't help, ask in the Frametop Discord." % result.get("reason", ""))
    return ("Not all of the headset's tracking cameras are running (%s). Restart SteamVR, or restart the "
            "headset if that doesn't fix it." % result.get("reason", ""))

@@ -1451,6 +1477,10 @@ class Session:
        cam = self._status.get("camera")
        if cam:
            self._session_json["camera"] = {"status": cam.get("status"), "reason": cam.get("reason", "")}
+            # which device each calibrated camera was (XRService's numbering) and whether XRService
+            # ran the side cameras through the ISP (no colour module): to check the names later
+            self._session_json["device"]["camera_map"] = cam.get("map") or {}
+            self._session_json["device"]["isp"] = (cam.get("episode") or {}).get("isp")
        if self.dry_run:
            self._session_json["dry_run"] = True
        if self.speed != 1:
@@ -66,6 +66,22 @@ RESUME_OK = [L("12:30:00", "[SystemdInhibitor] Received systemd resume notificat
             for i, n in enumerate((9, 13, 6, 7))] + \
            [L("12:30:02", "[DeckardCaptureSource] Streaming resumed (FPGA: VCINT, VC interleaving: enabled)")]
 EXIT = [L("13:00:00", "XRService - main thread exiting"), L("13:00:00", "Exiting XRService")]
+# The colour module unplugged while SteamVR runs (2026-10-05 12:42): XRService reopens the
+# cameras with the side pair through the ISP on vfe0 and vfe1 (NV12); VCINT stays loaded, so the
+# upper pair stays on vfe2.
+UNPLUG = [L("12:42:25", "Received passthrough camera connection event (connected=0)"),
+          L("12:42:25", "[DeckardCaptureSource] Closing tracking camera interfaces camerasToUse: 1111"),
+          L("12:42:26", "FPGA state check: VCINT (register value: 0x00021211)"),
+          L("12:42:26", "Upper cameras FPGA interleaving support: 1 (Driver features available = 1 | VCINT loaded = 1)"),
+          L("12:42:26", "[buildMediaCtlSetupTasks] ISP enabled for tracking cameras (main VFE available)"),
+          L("12:42:26", "[buildMediaCtlSetupTasks] Created 4 tasks (4 tracking, 0 passthrough)")] + \
+         [L("12:42:26", "TrackingCameraInit: index: %d. video device: /dev/video%d. v4l subdevice: x" % (i, n))
+          for i, n in enumerate((0, 3, 6, 7))]
+# Started without the module (FrameEyeCameraFeed's layout): the upper pair on vfe3 and vfe4.
+NO_MODULE = [L("10:00:02", "[buildMediaCtlSetupTasks] ISP enabled for tracking cameras (main VFE available)"),
+             L("10:00:02", "[buildMediaCtlSetupTasks] Created 4 tasks (4 tracking, 0 passthrough)")] + \
+            [L("10:00:03", "TrackingCameraInit: index: %d. video device: /dev/video%d. v4l subdevice: x" % (i, n))
+             for i, n in enumerate((0, 3, 9, 13))]


 def state(lines):
@@ -128,6 +144,23 @@ class SyntheticLogs(unittest.TestCase):
        self.assertEqual(status, "degraded")
        self.assertEqual(reason, "only 2 of 4 tracking cameras running")

+    def test_camera_map_with_module(self):
+        st = state(START + GOOD_OPEN)
+        self.assertEqual(st.camera_map(), {"slam_left": 9, "slam_right": 13, "upper_left": 6, "upper_right": 7})
+
+    def test_module_unplugged(self):
+        st = state(START + GOOD_OPEN + UNPLUG)
+        self.assertEqual(st.verdict()[0], "ok")
+        self.assertIs(st.episode["isp"], True)
+        self.assertEqual(st.camera_map(), {"slam_left": 0, "slam_right": 3, "upper_left": 6, "upper_right": 7})
+        self.assertEqual(st.tracking_nodes(), (0, 3, 6, 7))
+
+    def test_started_without_module(self):
+        st = state(START + NO_MODULE)
+        self.assertEqual(st.verdict()[0], "ok")
+        self.assertEqual(st.camera_map(), {"slam_left": 0, "slam_right": 3, "upper_left": 9, "upper_right": 13})
+        self.assertEqual(st.upper_nodes(), (9, 13))
+
    def test_exited_and_empty(self):
        self.assertEqual(state(START + GOOD_OPEN + EXIT).verdict()[0], "unknown")
        self.assertEqual(state([]).verdict()[0], "unknown")
@@ -223,9 +256,9 @@ class RealLog(unittest.TestCase):
        self.assertEqual(out.getvalue().split("\n")[0], "ok")


-def make_ring(path, mono_names, alive=True):
+def make_ring(path, mono_names, alive=True, nodes=None):
    """A ring header as ft-camd writes it (camd/fhring.h), no frames."""
-    cams = [(b"og01a1b", n.encode(), 9 + i) for i, n in enumerate(mono_names)]
+    cams = [(b"og01a1b", n.encode(), nodes[i] if nodes else 9 + i) for i, n in enumerate(mono_names)]
    hb = time.clock_gettime_ns(time.CLOCK_MONOTONIC) if alive else 1
    data = bytearray(camcheck.RING_HDR.pack(b"FHRING01", 1, 0, len(cams), 0, 0, 4242, 0))
    struct.pack_into("<Q", data, 40, hb)
@@ -252,6 +285,19 @@ class Ring(unittest.TestCase):
        self.assertEqual(r["status"], "degraded")
        self.assertIn("ft-camd publishes only 2 of 4", r["reason"])

+    def test_ring_missing_the_side_cameras(self):
+        # an ft-camd from before NV12 support, without the colour module: only the upper pair
+        with open(self.log, "w") as f:
+            f.write("\n".join(START + NO_MODULE) + "\n")
+        make_ring(self.ring, ["og0ve10_5-003e_video9", "og0ve10_5-0060_video13"], nodes=[9, 13])
+        r = camcheck.check(log=self.log, proc=False, ring_path=self.ring)
+        self.assertEqual(r["status"], "degraded")
+        self.assertEqual(r["ring_missing"], [0, 3])
+        self.assertIn("missing video0 video3", r["reason"])
+        self.assertTrue(camcheck.is_ring_short(r))
+        self.assertFalse(camcheck.is_vcint_failure(r))
+        self.assertEqual(r["map"]["slam_left"]["node"], 0)
+
    def test_four_cameras_in_ring(self):
        make_ring(self.ring, ["a_video9", "b_video13", "c_video6", "d_video7", "a_video9_dk"])
        r = camcheck.check(log=self.log, proc=False, ring_path=self.ring)
@@ -34,8 +34,28 @@ from tools.show_set import index, read_set  # noqa: E402
 from tools import calib  # noqa: E402


-PIPES = {'msm_vfe3_video0': 'slam_left', 'msm_vfe4_video0': 'slam_right',    # as ft-hands maps them
+PIPES = {'msm_vfe3_video0': 'slam_left', 'msm_vfe4_video0': 'slam_right',    # with the colour module
         'msm_vfe2_video0': 'upper_left', 'msm_vfe2_video1': 'upper_right'}
+
+
+def ring_names():
+    """{/dev/videoN's N: calibration name} as ft-hands names them: by XRService's log
+    (camcheck.py), else by capture pipe (PIPES)."""
+    import camcheck
+    try:
+        by_log = {node: name for name, node in camcheck.read_log(camcheck.newest_log()).camera_map().items()}
+    except (OSError, ValueError):
+        by_log = {}
+    if by_log:
+        return by_log
+    out = {}
+    for path in os.listdir('/sys/class/video4linux'):
+        if path.startswith('video'):
+            with open('/sys/class/video4linux/%s/name' % path) as f:
+                name = PIPES.get(f.read().strip())
+            if name:
+                out[int(path[5:])] = name
+    return out
 PAIRS = {'side': ('slam_left', 'slam_right'), 'upper': ('upper_left', 'upper_right')}


@@ -100,8 +120,9 @@ def live_pairs(count, names=PAIRS['side']):
    if not ring.alive():
        sys.exit('ft-camd isn\'t running (no heartbeat)')
    cams = {}
+    names_of = ring_names()
    for c in ring.cams:
-        name = PIPES.get(open('/sys/class/video4linux/video%d/name' % c.node).read().strip())
+        name = names_of.get(c.node)
        if name and not c.name.endswith('-dark'):
            cams[name] = c
    for k in range(count):
@@ -63,7 +63,34 @@ namespace {

 volatile std::sig_atomic_t g_stop = 0, g_record = 0;

-// which calibrated camera each capture pipe carries (XRService's fixed routing)
+// Which calibrated camera each video device carries. XRService numbers its tracking cameras
+// (index 0 to 3: slam_left, slam_right, upper_left, upper_right) and logs the device each one
+// opened ("TrackingCameraInit: index: 0. video device: /dev/video9"). The devices depend on
+// the colour module: with it, the side cameras are on vfe3 and vfe4 and the upper pair on
+// vfe2; without it, XRService runs the side cameras through the ISP on vfe0 and vfe1, and the
+// upper pair on vfe3 and vfe4. So the running XRService's log decides; when it can't be read,
+// the capture pipes as they are with the module. {} if the log has no cameras.
+std::map<int, std::string> cameras_from_xrservice_log() {
+    static const char *const names[] = {"slam_left", "slam_right", "upper_left", "upper_right"};
+    const char *home = std::getenv("HOME");
+    std::ifstream in(std::string(home ? home : "") + "/.local/share/Steam/logs/xrservice.txt");
+    const std::string key = "TrackingCameraInit: index: ";
+    std::map<int, int> node_of;   // index -> N of /dev/videoN, from the latest camera start
+    std::string line;
+    while (std::getline(in, line)) {
+        if (line.find("XRService logging to") != std::string::npos) node_of.clear();
+        const auto at = line.find(key);
+        int index = -1, node = -1;
+        if (at != std::string::npos &&
+            std::sscanf(line.c_str() + at + key.size(), "%d. video device: /dev/video%d", &index, &node) == 2 &&
+            index >= 0 && index < 4)
+            node_of[index] = node;
+    }
+    std::map<int, std::string> out;
+    for (auto &[index, node] : node_of) out[node] = names[index];
+    return out;
+}
+
 const char *camera_for_pipe(int node) {
    char path[64], name[64] = "";
    std::snprintf(path, sizeof path, "/sys/class/video4linux/video%d/name", node);
@@ -302,6 +329,7 @@ int main(int argc, char **argv) {
    double decided_after_s = -1;
    if (sides_mode != "auto") truth = names_swapped, decided_by = sides_from, sides_state = "forced";
    std::string rec_dir;
+    std::string cams_json;   // which device each calibrated camera is (set below), for the sides files
    std::vector<std::pair<size_t, bool>> rec_names;   // from which recorded set on, names_swapped was what
    // DIR/sides.json beside a recording's sets.bin: how its side cameras are named. A set's
    // names are right when its names_swapped equals swapped (null: not known when recorded).
@@ -312,7 +340,8 @@ int main(int argc, char **argv) {
        write_file(rec_dir + "/sides.json",
                   "{\"swapped\": " + json_bool(truth) + ", \"decided_by\": " + json_str(truth ? decided_by : "") +
                       ", \"names_swapped\": [" + runs + "]" +
-                       (decision_evidence.empty() ? "" : ", \"evidence\": " + decision_evidence) + "}\n");
+                       (decision_evidence.empty() ? "" : ", \"evidence\": " + decision_evidence) +
+                       (cams_json.empty() ? "" : ", \"cameras\": " + cams_json) + "}\n");
    };
    auto start_recording = [&](const std::string &dir, std::string &e) {
        rec = std::make_unique<Recorder>();
@@ -340,19 +369,42 @@ int main(int argc, char **argv) {
    // (--with-color). Recorded names hold 15 characters, so "upper_right_dark" wouldn't fit.
    std::map<std::string, int> dark;
    std::map<std::string, Camera> used;
+    // ft-camd's cameras by XRService's numbering, else by capture pipe (see camera_for_pipe);
+    // ft-ringplay's (no device) by the name it gives
+    const std::map<int, std::string> by_log = cameras_from_xrservice_log();
+    const char *named_by = by_log.empty() ? "capture pipe" : "XRService's log";
    for (int i = 0; i < ring.cameras(); ++i) {
-        if (ring.camera(i).flags & FH_CAM_COLOR) {
-            const std::string name = "color_video" + std::to_string(ring.camera(i).node);
+        const fh_ring_cam_t &rc = ring.camera(i);
+        if (rc.flags & FH_CAM_COLOR) {
+            const std::string name = "color_video" + std::to_string(rc.node);
            dark[name] = i, color[name] = i;
            continue;
        }
-        // ft-camd's cameras by capture pipe; ft-ringplay's (no device) by the name it gives
-        const char *name = camera_for_pipe(ring.camera(i).node);
-        if (!name && ring.camera(i).node < 0) name = ring.camera(i).name;
-        if (!name || !calib.count(name)) continue;
-        if (ring.camera(i).flags & FH_CAM_DARK) dark[std::string(name) + "_dk"] = i;
-        else index[name] = i, used[name] = calib[name];
+        const auto it = by_log.find(rc.node);
+        const char *name = rc.node < 0                ? rc.name
+                           : !by_log.empty()          ? (it == by_log.end() ? nullptr : it->second.c_str())
+                                                      : camera_for_pipe(rc.node);
+        if (!name || !calib.count(name)) {
+            if (!(rc.flags & FH_CAM_DARK)) std::fprintf(stderr, "video%d (%s): not one of the calibrated cameras, left out\n", rc.node, rc.name);
+            continue;
+        }
+        if (int(rc.width) != calib[name].width || int(rc.height) != calib[name].height) {
+            std::fprintf(stderr, "video%d (%s) is %ux%u, but %s is calibrated at %dx%d: left out\n", rc.node, rc.name,
+                         rc.width, rc.height, name, calib[name].width, calib[name].height);
+            continue;
+        }
+        if (rc.flags & FH_CAM_DARK) {
+            dark[std::string(name) + "_dk"] = i;
+        } else if (index.count(name)) {
+            std::fprintf(stderr, "video%d (%s) would be %s too (video%d is): left out\n", rc.node, rc.name, name,
+                         ring.camera(index[name]).node);
+        } else {
+            index[name] = i, used[name] = calib[name];
+            cams_json += std::string(cams_json.empty() ? "" : ", ") + json_str(name) + ": {\"node\": " +
+                         std::to_string(rc.node) + ", \"ring\": " + json_str(rc.name) + "}";
+        }
    }
+    cams_json = "{\"named_by\": " + json_str(named_by) + ", \"cameras\": {" + cams_json + "}}";
    // ft-camd tells the side cameras' buffers apart by XRService's allocation order, which
    // some XRService restarts reverse (see the top).
    const bool have_sides = index.count("slam_left") && index.count("slam_right");
@@ -397,7 +449,7 @@ int main(int argc, char **argv) {
    }
    const bool switching = automatic && !color.empty();
    Cams mode = switching || color.empty() ? Cams::Mono : fixed;
-    std::printf("cameras:");
+    std::printf("cameras (by %s):", named_by);
    for (auto &[name, i] : index) std::printf(" %s=video%d", name.c_str(), ring.camera(i).node);
    for (auto &[name, i] : color) std::printf(" %s", name.c_str());
    std::printf("  tracking with %s%s  models: %s%s, %d threads on CPUs", cams_name(mode),
@@ -446,6 +498,7 @@ int main(int argc, char **argv) {
                       ", \"decided_after_s\": " + std::to_string(decided_after_s) +
                       ", \"evidence\": " + (decision_evidence.empty() ? "null" : decision_evidence) +
                       ", \"checking\": " + (checking ? side_check.json() : "null") +
+                       ", \"cameras\": " + (cams_json.empty() ? "null" : cams_json) +
                       ", \"updated_ns\": " + std::to_string(mono_ns()) + "}\n");
    };
    write_sides();
@@ -75,7 +75,7 @@ step "3/10 input relay (keeps Bluetooth mice working in SteamVR, device roles, b

 step "4/10 3D mouse: SteamVR driver"
 "$root/pointer/driver/build.sh"
-"$root/pointer/driver/install.sh" install 2>&1 | grep -v xdg-open
+"$root/pointer/driver/install.sh" install

 step "5/10 3D mouse: pointer helper service"
 "$root/pointer/helper/build.sh"
@@ -534,7 +534,8 @@ def capture_screens():
 # ---------------------------------------------------------------- gamescope (dashboard panels)

 def vrcmd(*args, timeout=10):
-    env = dict(os.environ, LD_LIBRARY_PATH=os.path.dirname(VRCMD))
+    # SteamVR's config folder: in the desktop, XDG_CONFIG_HOME is its own (docs/design.md).
+    env = dict(os.environ, LD_LIBRARY_PATH=os.path.dirname(VRCMD), XDG_CONFIG_HOME=os.path.expanduser("~/.config"))
    try:
        return subprocess.run([VRCMD, *args], capture_output=True, text=True, timeout=timeout, env=env).stdout
    except (OSError, subprocess.TimeoutExpired):
@@ -14,6 +14,10 @@ frame="$root/scripts/frame.sh"
 src=$FRAME_REPO/pointer/driver
 dest='$HOME/.local/share/frametop/ft_pointer'  # expanded on the Frame, in the commands below
 reg='/opt/steamvr/bin/linuxarm64/vrpathreg'
+# SteamVR's tools with SteamVR's config folder and libraries. A terminal in the Frametop desktop
+# has XDG_CONFIG_HOME=~/.config/frametop (session/frametop-session.sh), where vrpathreg finds no
+# registry, so it made one there that SteamVR never reads.
+steamvr='env XDG_CONFIG_HOME=$HOME/.config LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64'  # expanded on the Frame

 case ${1:-install} in
  install)
@@ -21,6 +25,11 @@ case ${1:-install} in
    # Replacing the files of a driver SteamVR has loaded leaves its input bindings in a bad
    # state (the 3D mouse no longer gets the laser) until SteamVR restarts, so an unchanged
    # driver is left alone.
+    # A registry an earlier install made in the desktop's config folder (see steamvr above) has
+    # no SteamVR in it, and hid SteamVR from OpenVR programs started in the desktop
+    # (VRInitError_Init_InstallationNotFound): it goes.
+    # vrpathreg runs 'xdg-open vrmonitor://driverinstalled' for SteamVR's desktop monitor, which
+    # the Frame doesn't have, so the desktop said "could not read file": a stand-in takes it.
    "$frame" --host "set -e; test -f $src/build/driver_ft_pointer.so
 rm -rf $dest.new; mkdir -p $dest.new/bin/linuxarm64
 cp -r $src/ft_pointer/. $dest.new/
@@ -31,15 +40,22 @@ else
  rm -rf $dest; mv $dest.new $dest
  echo 'installed; restart SteamVR to load it'
 fi
-LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg adddriver $dest
-LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg show | grep -A3 -i 'external'" ;;
+stray=\$HOME/.config/frametop/openvr/openvrpaths.vrpath
+if grep -qs '\"jsonid\" : \"vrpathreg\"' \$stray && grep -qsE '\"runtime\"[[:space:]]*:[[:space:]]*null' \$stray; then
+  rm -f \$stray; rmdir \$HOME/.config/frametop/openvr 2>/dev/null || true
+  echo 'removed a SteamVR path registry an earlier install left in ~/.config/frametop/openvr'
+fi
+noopen=\$(mktemp -d); trap 'rm -rf \$noopen' EXIT
+printf '#!/bin/sh\nexit 0\n' > \$noopen/xdg-open; chmod +x \$noopen/xdg-open
+PATH=\$noopen:\$PATH $steamvr $reg adddriver $dest
+$steamvr $reg show | grep -A3 -i 'external'" ;;
  uninstall)
-    "$frame" --host "LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg removedriver $dest; rm -rf $dest; echo 'removed; restart SteamVR to unload it'" ;;
+    "$frame" --host "$steamvr $reg removedriver $dest; rm -rf $dest; echo 'removed; restart SteamVR to unload it'" ;;
  send)
    "$frame" --host "python3 -c 'import socket,sys; s=socket.socket(socket.AF_UNIX,socket.SOCK_DGRAM); s.sendto(sys.argv[1].encode(), \"\\0ft_pointer\")' $(printf %q "${2:?command}")" ;;
-  probe) "$frame" -C pointer/probe 'LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 ./build/vrprobe' ;;
+  probe) "$frame" -C pointer/probe "$steamvr ./build/vrprobe" ;;
  aimhere)
-    read -r yaw pitch < <("$frame" -C pointer/probe 'LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 ./build/vrprobe' | sed -n 's/^head yaw = \([-0-9.]*\) pitch = \([-0-9.]*\)$/\1 \2/p')
+    read -r yaw pitch < <("$frame" -C pointer/probe "$steamvr ./build/vrprobe" | sed -n 's/^head yaw = \([-0-9.]*\) pitch = \([-0-9.]*\)$/\1 \2/p')
    [ -n "${yaw:-}" ] || { echo "head pose not valid (headset off?)" >&2; exit 1; }
    "$0" send "aim $yaw $pitch" && echo "aimed at yaw $yaw pitch $pitch" ;;
  log) "$frame" --host "grep -iE 'ft_pointer' ~/.local/share/Steam/logs/vrserver.txt | tail -n ${2:-20}" ;;
@@ -26,7 +26,8 @@ for u in frametop-input-relay frametop-pointer frametop-power; do
 done
 echo "desktop: $(pgrep -x ft-screens >/dev/null && echo running || echo 'not running'), plasmashell: $(pgrep -c plasmashell || true)"
 echo "paused for VR games: $(cat /run/user/$(id -u)/frametop-pause.json 2>/dev/null || echo 'no (never paused since boot)')"
-LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrpathreg show 2>/dev/null | sed -n '/xternal/,$p'
+# SteamVR's config folder: a terminal in the desktop has the session's own (docs/design.md).
+XDG_CONFIG_HOME=$HOME/.config LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrpathreg show 2>/dev/null | sed -n '/xternal/,$p'

 section "What Frametop needs from SteamOS (scripts/update-check.py)"
 python3 "$repo/scripts/update-check.py" 2>&1 || true
@@ -67,6 +68,28 @@ section "Pointer helper (last 60 lines)"
 journalctl --user -u frametop-pointer -n 60 --no-pager -o short 2>/dev/null
 section "Power service (last 30 lines)"
 journalctl --user -u frametop-power -n 30 --no-pager -o short 2>/dev/null
+section "Gaze"
+echo "frametop-gaze: $(systemctl --user is-enabled frametop-gaze 2>/dev/null) / $(systemctl --user is-active frametop-gaze 2>/dev/null)"
+echo "frametop-eyegrab (our eye tracker's frame grabber): $(systemctl is-enabled frametop-eyegrab 2>/dev/null) / $(systemctl is-active frametop-eyegrab 2>/dev/null)"
+for f in calibration.json eyes/calibration.json; do  # SteamVR's correction, our tracker's calibration
+  p=~/.local/state/frametop/gaze/$f
+  echo "$f: $([ -f "$p" ] && date -r "$p" '+%F %T' || echo none)"
+done
+python3 - "$repo" <<'PY' 2>&1 || true
+import json, subprocess, sys
+out = subprocess.run([sys.executable, sys.argv[1] + "/gaze/ft-gazectl", "status"], capture_output=True, text=True, timeout=10)
+text = (out.stdout or out.stderr).strip()
+try:
+    print("status:", json.dumps(json.loads(text), separators=(",", ":")))
+except ValueError:
+    print("status:", text or "no answer from the gaze service")
+PY
+section "Gaze checks and calibration dots (last 10)"
+tail -n 10 ~/.local/state/frametop/gaze/checks.jsonl 2>/dev/null | cut -c1-400 || true
+section "Gaze service (last 60 lines, podman's left out)"
+journalctl --user -u frametop-gaze -n 300 --no-pager -o short 2>/dev/null | grep -v ' podman\[' | tail -n 60
+section "Our eye tracker's frame grabber (last 20 lines)"
+journalctl -u frametop-eyegrab -n 20 --no-pager -o short 2>/dev/null
 for f in /tmp/frametop-session.log /tmp/frametop-screens.log /tmp/frametop-layout.log; do
  section "$f (last 60 lines)"
  tail -n 60 "$f" 2>/dev/null || echo "missing"
@@ -35,6 +35,9 @@ KNOWN_GOOD = os.path.join(HOME, ".local/state/frametop/known-good.json")
 STEAMVR_BIN = "/opt/steamvr/bin/linuxarm64"
 LAUNCHER = os.path.join(HOME, ".local/share/applications/deckard-nested-desktop.desktop")
 VRPATHS = os.path.join(HOME, ".config/openvr/openvrpaths.vrpath")
+# The Frametop desktop has its own XDG_CONFIG_HOME (session/frametop-session.sh). An install from a
+# terminal there, before pointer/driver/install.sh set SteamVR's, left vrpathreg's registry here.
+STRAY_VRPATHS = os.path.join(HOME, ".config/frametop/openvr/openvrpaths.vrpath")
 BACKLIGHT = "/sys/class/backlight/ae94000.dsi.0/brightness"  # as in power/ft-powerd.cpp
 EYE_MMAP = "/dev/shm/eye-server.mmap"
 VRSERVER = "http://127.0.0.1:27062"  # its web socket: input/vrws.py
@@ -49,6 +52,8 @@ PACKAGES = {
            "after a desktop restart; floating a window; no blur behind the taskbar's menus",
    "plasma-workspace": "the taskbar and panels after a desktop restart; no DiscoverNotifier or "
                        "ibus-daemon inside the desktop",
+    "at-spi2-core": "an AT-SPI-aware app appears on the nested desktop's accessibility bus; "
+                    "the registry stops and comes back after a desktop restart",
    "gamescope": "the headset's volume buttons with nothing focused; typing goes where you last clicked",
    "bluez": "a Bluetooth mouse reconnecting after it sleeps",
 }
@@ -162,6 +167,14 @@ def check_host():
        ("/usr/bin/kwin_wayland_wrapper", "FAIL", "the desktop can't start KWin"),
        ("/usr/bin/startplasma-wayland", "FAIL", "the desktop can't start Plasma"),
        ("/usr/bin/dbus-run-session", "FAIL", "the desktop can't start its session bus"),
+        ("/usr/bin/python3", "warn", "nested accessibility can't run its startup helper"),
+        ("/usr/bin/gdbus", "warn", "nested accessibility can't discover or check its bus"),
+        ("/usr/lib/at-spi-bus-launcher", "warn", "nested accessibility can't start its bus"),
+        ("/usr/lib/at-spi2-registryd", "warn", "nested accessibility has no fallback registry"),
+        ("/usr/share/dbus-1/services/org.a11y.Bus.service", "warn",
+         "nested accessibility can't activate its bus"),
+        ("/usr/share/dbus-1/accessibility-services/org.a11y.atspi.Registry.service", "warn",
+         "nested accessibility can't activate its registry natively"),
        (f"{STEAMVR_BIN}/vrcmd", "FAIL", "the 3D mouse can't find panels"),
        (f"{STEAMVR_BIN}/vrpathreg", "warn", "the pointer driver can't be installed or removed"),
        ("/usr/share/deckard/mesavars.sh", "warn", "the desktop starts without SteamOS's Mesa settings"),
@@ -217,6 +230,14 @@ def check_host():
    else:
        report("FAIL", "pointer driver", "not registered with SteamVR; run pointer/driver/install.sh install, "
               "then restart SteamVR")
+    try:
+        with open(STRAY_VRPATHS) as f:
+            stray = json.load(f).get("runtime") is None
+    except (OSError, ValueError, AttributeError):
+        stray = False
+    if stray:
+        report("warn", "OpenVR path registry", f"{STRAY_VRPATHS} has no SteamVR in it, and hides SteamVR from "
+               "OpenVR programs started in the Frametop desktop; pointer/driver/install.sh install removes it")

    session = launcher_session()
    if session is None:
@@ -464,6 +485,8 @@ def main():
    # A terminal in the desktop has its session's runtime dir and bus; systemctl needs the real ones.
    os.environ["XDG_RUNTIME_DIR"] = f"/run/user/{UID}"
    os.environ["DBUS_SESSION_BUS_ADDRESS"] = f"unix:path=/run/user/{UID}/bus"
+    # And its own config folder, where SteamVR's path registry isn't: OpenVR and vrcmd need ours.
+    os.environ["XDG_CONFIG_HOME"] = os.path.join(HOME, ".config")

    versions = current_versions()
    check_versions(versions)
@@ -29,7 +29,7 @@ for var in $(compgen -e); do
  case $var in
    LD_LIBRARY_PATH | LD_PRELOAD | STEAM_* | Steam* | SRT_* | PRESSURE_VESSEL_* | MANGOHUD_* | \
      ENABLE_VK_LAYER_VALVE_steam_overlay_* | STEAMVIDEOTOKEN | QT_IM_MODULE | GTK_IM_MODULE | \
-      XMODIFIERS) unset "$var" ;;
+      XMODIFIERS | AT_SPI_BUS_ADDRESS) unset "$var" ;;
  esac
 done

@@ -190,6 +190,21 @@ if [ "$remote" = 1 ]; then
  "$here/remote-ctl.sh" start
 fi

+# AT-SPI: start the registry inside Plasma's autostart, after KWin has published the
+# nested displays. Starting it before startplasma could bind to the host's X display.
+# This config belongs only to Frametop; the host desktop's autostart is unchanged.
+autostart=$XDG_CONFIG_HOME/autostart/frametop-atspi.desktop
+mkdir -p "$(dirname "$autostart")"
+cat > "$autostart" <<EOF
+[Desktop Entry]
+Type=Application
+Name=Frametop accessibility
+Exec="$here/ft-atspi"
+X-KDE-autostart-phase=2
+OnlyShowIn=KDE;
+NoDisplay=true
+EOF
+
 # ft-floatd (floating windows) runs inside the Plasma session, on its D-Bus: started from
 # the session's autostart, which only this desktop reads (XDG_CONFIG_HOME above).
 autostart=$XDG_CONFIG_HOME/autostart/frametop-floatd.desktop
@@ -0,0 +1,87 @@
+#!/usr/bin/python3
+"""Start AT-SPI on this nested desktop's live bus, after KWin has set its displays."""
+import ast
+import os
+import signal
+import subprocess
+import sys
+import time
+
+DBUS = "org.freedesktop.DBus"
+DBUS_PATH = "/org/freedesktop/DBus"
+REGISTRY = "org.a11y.atspi.Registry"
+
+
+def call(address, destination, path, method, *args):
+    return subprocess.run(
+        ["gdbus", "call", "--address", address, "--dest", destination,
+         "--object-path", path, "--method", method, "--timeout", "5", *args],
+        check=True, capture_output=True, text=True, timeout=6).stdout.strip()
+
+
+def stop(child):
+    if child.poll() is None:
+        child.terminate()
+    try:
+        child.wait(timeout=3)
+    except subprocess.TimeoutExpired:
+        child.kill()
+        child.wait()
+
+
+def run_registry(address, session):
+    # SteamOS's registryd stays alive after a bus disconnect. Keep only our own
+    # child tied to both private buses; no host PID lookup or broad process kill.
+    registry = subprocess.Popen(["/usr/lib/at-spi2-registryd"])
+    try:
+        while registry.poll() is None:
+            try:
+                call(session, DBUS, DBUS_PATH, DBUS + ".GetId")
+                call(address, DBUS, DBUS_PATH, DBUS + ".GetId")
+            except subprocess.SubprocessError:
+                break  # Normal session shutdown or loss of its accessibility bus.
+            # Each check starts two gdbus processes, and SteamVR needs the CPU.
+            time.sleep(5)
+    finally:
+        stop(registry)
+
+
+def start():
+    session = os.environ.get("DBUS_SESSION_BUS_ADDRESS")
+    runtime = os.environ.get("XDG_RUNTIME_DIR", "")
+    if not session or os.path.basename(runtime) != "frametop":
+        return  # Do not autolaunch a bus or touch the host desktop.
+    os.environ.pop("AT_SPI_BUS_ADDRESS", None)
+    reply = call(session, "org.a11y.Bus", "/org/a11y/bus", "org.a11y.Bus.GetAddress")
+    address, = ast.literal_eval(reply)
+    if not isinstance(address, str) or not address.startswith("unix:"):
+        raise ValueError("org.a11y.Bus returned no local accessibility bus")
+    if call(address, DBUS, DBUS_PATH, DBUS + ".NameHasOwner", REGISTRY) == "(true,)":
+        return
+    # Activation otherwise inherits the accessibility daemon's old environment (and
+    # DBUS_SESSION_BUS_ADDRESS is the accessibility bus itself). Pin the live bus.
+    env = {key: os.environ.get(key, "") for key in (
+        "DISPLAY", "WAYLAND_DISPLAY", "XAUTHORITY", "XDG_RUNTIME_DIR",
+        "XDG_CONFIG_HOME", "DBUS_SESSION_BUS_ADDRESS")}
+    env["AT_SPI_BUS_ADDRESS"] = address
+    call(address, DBUS, DBUS_PATH, DBUS + ".UpdateActivationEnvironment", repr(env))
+    try:
+        call(address, DBUS, DBUS_PATH, DBUS + ".StartServiceByName", REGISTRY, "0")
+    except subprocess.CalledProcessError:
+        # SteamOS's launcher can select dbus-broker because we are in a user unit,
+        # but the private session bus has no systemd activation manager. Reuse its
+        # bus rather than starting another launcher/bus. Never replace an owner.
+        if call(address, DBUS, DBUS_PATH, DBUS + ".NameHasOwner", REGISTRY) == "(true,)":
+            return
+        os.environ["AT_SPI_BUS_ADDRESS"] = address
+        # registryd itself refuses to queue behind an existing registry, including
+        # races with native activation. The watcher cleans up only our own child.
+        run_registry(address, session)
+
+
+if __name__ == "__main__":
+    signal.signal(signal.SIGTERM, lambda signum, frame: sys.exit(0))
+    try:
+        start()
+    except (OSError, ValueError, SyntaxError, subprocess.SubprocessError) as error:
+        print(f"frametop: accessibility unavailable: {error}", file=sys.stderr)
@@ -10,7 +10,7 @@ cg=$(systemctl --user show -p ControlGroup --value "$unit" 2>/dev/null)

 # The desktop's own processes stay in the unit and stop with it: the session, KWin,
 # Plasma, and the session services it started (portals, input methods, kded, wallet...).
-own='^(frametop-sessi|dbus-|startplasma|plasma|kwin|Xwayland|ksmserver|krdpserver|Xvnc|xfreerdp|ft-layout|'
+own='^(frametop-sessi|dbus-|startplasma|plasma|kwin|Xwayland|ksmserver|krdpserver|Xvnc|xfreerdp|ft-layout|ft-atspi|'
 own+='kded|kactivitymanage|kaccess|kglobalaccel|kscreen|kwalletd|ksecretd|polkit-kde|org_kde_|baloo|'
 own+='xembedsniproxy|gmenudbusmenu|DiscoverNotifie|kimpanel|ibus|xdg-|at-spi|dconf-service|fusermount|agent)'
 keep=()