Gaze checks and calibration in a panel fixed to the headset

ft-gazepanel (gaze/panel) is a SteamVR overlay that stays put in your view and
shows dots at known head-relative directions; the gaze service runs it and
drives it (gaze/gazecheck.py):
- a one-dot quick check 3 s after the headset goes on, when our tracker asks
  for a click (reseat), at most every 2 minutes, and from Quick check on the
  Gaze page; five dots follow if the next 3 lessons are still over 2 degrees off
- the full calibration (the probe's three rounds, dark to bright) when gaze mode
  comes on without one, or from Calibrate; quitting it with still no calibration
  turns gaze mode off
Each dot takes the gaze once it has held still for 0.6 s; a left click or Meta+J
takes it at once, a right click or Meta+K closes the panel (the pointer helper
hides its dot and passes those presses on while "calpanel" lasts). Our tracker
gets clicks and its own calibration; SteamVR's gets lessons per eye, or a new
calibration.json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/design.md

@@ -91,7 +91,7 @@ A few overlays need special handling:
 
 Head follow is experimental and off by default. It works, but it's only lightly tested, and the feel is mostly a matter of its settings; polishing it is left open. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction, where you were facing when your head last settled, and keeps its offset from it. The mouse can put the cursor anywhere up to `POINTER_FOLLOW_REACH` (70 degrees) from the reference, a corner of your view included. While your head stays within `POINTER_LEASH_DEG` of the reference, nothing moves on its own. Once your head has been past the leash for `POINTER_LEASH_DELAY` (0.2 s, so a glance out and back doesn't count), the reference eases to where you're facing (time constant `POINTER_LEASH_RETURN`, 0.2 s), never falling further behind than the leash, and the cursor ends up back where it was in your view. Then it waits for the leash again. Two earlier versions didn't work out. Moving the reference only while your head pulled at the end of the leash left it up to the leash off after you turned back, and getting it centred again meant overshooting with your head. Easing it toward your facing all the time moved the cursor on every small head movement. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping.
 
-Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. Moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: it needs accessibility (AT-SPI) on in the Frametop session, where it's off (no registry runs), plus app restarts, and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge of up to `POINTER_GAZE_NUDGE_MAX` (8 degrees) before a click is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. See `gaze/README.md` for the service, the calibration, and what was measured.
+Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. Moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: it needs accessibility (AT-SPI) on in the Frametop session, where it's off (no registry runs), plus app restarts, and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge of up to `POINTER_GAZE_NUDGE_MAX` (8 degrees) before a click is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on or our tracker thinks it moved, and the full calibration runs in the same panel. Its dots sit at known directions from the headset, so the panel needs no screen geometry, and each dot takes the gaze when it has held still rather than at a press: what the tracker says doesn't have to be close for the capture to work. See `gaze/README.md` for the service, the calibration, and what was measured.
 
 Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place.
 

docs/reference.md

@@ -104,7 +104,7 @@ pointer/helper/run.sh status | log | restart
 pointer/driver/install.sh probe     # devices, hand roles, who owns the dashboard pointer
 ```
 
-The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_CONTROLLER_PICKUP`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, `POINTER_IGNORE`, the head follow settings `POINTER_FOLLOW`, `POINTER_LEASH_DEG`, `POINTER_LEASH_DELAY`, `POINTER_LEASH_RETURN`, and `POINTER_FOLLOW_REACH`, and the gaze mode settings `POINTER_GAZE`, `POINTER_GAZE_RETAKE`, `POINTER_GAZE_NUDGE_MAX`, `POINTER_GAZE_HOLD`, `POINTER_GAZE_DOT`, `POINTER_GAZE_SHOW`, and `POINTER_GAZE_MOUSE`, and the gaze service's `GAZE_TRACKER` (SteamVR's eye tracker or our own) and `GAZE_EYE` (the eye bias). The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper (the gaze service reads its two again when the file changes).
+The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_CONTROLLER_PICKUP`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, `POINTER_IGNORE`, the head follow settings `POINTER_FOLLOW`, `POINTER_LEASH_DEG`, `POINTER_LEASH_DELAY`, `POINTER_LEASH_RETURN`, and `POINTER_FOLLOW_REACH`, and the gaze mode settings `POINTER_GAZE`, `POINTER_GAZE_RETAKE`, `POINTER_GAZE_NUDGE_MAX`, `POINTER_GAZE_HOLD`, `POINTER_GAZE_DOT`, `POINTER_GAZE_SHOW`, `POINTER_GAZE_MOUSE`, and the keyboard clicks' `POINTER_HEAD_DEADZONE` and `POINTER_KEY_TAP`, and the gaze service's `GAZE_TRACKER` (SteamVR's eye tracker or our own) and `GAZE_EYE` (the eye bias). The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper (the gaze service reads its two again when the file changes).
 
 ## Frametop Input Settings
 

gaze/README.md

@@ -27,8 +27,10 @@ Gaze as an input method for the whole desktop, without replacing anything of Ste
   With SteamVR, each eye is its own reading (set 2), corrected by its calibration from the probe (the Left eye and Right eye sources) plus what the pointer has taught that eye since. On that test, the two eyes each calibrated and averaged were 1.70 degrees off (median; mean 1.62) against 1.72 (mean 1.84) for SteamVR's combined gaze with its calibration. A calibration from before the probe had the eyes as sources, or `--source`, uses the older path. That path runs on SteamVR's combined gaze (mmap set 1), corrected as a whole. When the tracker loses one eye (its variance for that eye jumps from about 0.001 to 0.02), the gaze comes from the other eye instead: that eye's own reading (set 2) plus what it usually reads against the combined gaze, learned while both eyes are seen, in 10 degree cells of where it looks. Set 1 keeps going on one eye too, but it holds the lost eye's yaw where it was, so the gaze moves half as far sideways as your eyes do. On a recording, one eye alone came out a median 0.8 degrees from both eyes' gaze over a steady look, a little more jittery.
 
   Looks down past the screens (under 20 degrees down, on no Frametop screen: a glance at the keyboard) aren't sent, so the pointer stays where it was instead of following you down, and eyes lost there aren't counted. It drops blinks (both eyes closing or lost), smooths with a fixation lock, and sends the result to the pointer helper 90 times a second. It follows SteamVR's eye tracking log, and when the headset goes back on (SteamVR starts its eye model over, and the error moves), older lessons count less, so the first few after relearn the offset.
-- The pointer helper's **gaze mode** (off by default: the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a mouse button or key combination mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look. Move the mouse and it's the mouse's, from where the gaze put it, for the last bit. Look well away (5 degrees) and the gaze takes it back. A press isn't sent at once: the pointer stops where the gaze put it, and if that's wrong, drag it onto what you meant with the button still held; the click happens where you let go. To drag something, hold the press still for half a second first (`POINTER_GAZE_HOLD`), then move. Outside games the pointer stays on while gaze mode is on, until a controller is picked up. The dot shows all the time (`POINTER_GAZE_DOT=moving`: only while the mouse moves it, while a press is held, and as a pulse when you click). Gaze mode works with the mouse only; [docs/gaze-controllers.md](../docs/gaze-controllers.md) explains why the controllers can't click at the gaze.
-- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it a little (0.2 to 8 degrees) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the probe's one-dot check does the same). The helper only sends nudges up to `POINTER_GAZE_NUDGE_MAX` (8 degrees), and right after putting the headset back on our tracker can be further off than that. If so, raise it for a moment, or do the probe's check. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
+- The pointer helper's **gaze mode** (off by default: the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a mouse button or key combination mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look. Move the mouse and it's the mouse's, from where the gaze put it, for the last bit. Look well away (5 degrees) and the gaze takes it back. A press isn't sent at once: the pointer stops where the gaze put it, and if that's wrong, drag it onto what you meant with the button still held; the click happens where you let go. To drag something, hold the press still for half a second first (`POINTER_GAZE_HOLD`), then move. Outside games the pointer stays on while gaze mode is on, until a controller is picked up. The dot shows all the time (`POINTER_GAZE_DOT=moving`: only while the mouse moves it, while a press is held, and as a pulse when you click). Gaze mode works with the mouse and the keyboard, not the controllers ([docs/gaze-controllers.md](../docs/gaze-controllers.md) explains why).
+- **Keyboard clicks** (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, `POINTER_KEY_TAP`) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; let go of either to drop.
+- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it a little (0.2 to 8 degrees) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the probe's one-dot check does the same). The helper only sends nudges up to `POINTER_GAZE_NUDGE_MAX` (8 degrees), and right after putting the headset back on our tracker can be further off than that. If so, raise it for a moment, or do a quick check. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
+- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens 3 s after you put the headset on, and when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself when gaze mode comes on without one) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
 - Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
 
 Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction, the correction at the time, and how far off it was.

gaze/build.sh

@@ -1,5 +1,7 @@
 #!/usr/bin/env bash
-# Build ft-gaze in the dev container on the Frame (gaze/build/ft-gaze; it also runs there).
+# Build ft-gaze and the calibration panel ft-gazepanel in the dev container on the Frame
+# (gaze/build/; they also run there). The panel draws its text with stb_truetype (public
+# domain, one header, pinned as in screens/build.sh).
 # The eye tracking API (IVRInput::GetEyeTrackingDataRelativeToNow) is newer than the header
 # shipped with SteamVR's samples, so this uses the pinned public header ft-screens fetches.
 set -euo pipefail
@@ -10,4 +12,8 @@ openvr=v2.15.6
 [ -f build/include/openvr-$openvr ] || { curl -fsSL "https://raw.githubusercontent.com/ValveSoftware/openvr/$openvr/headers/openvr.h" -o build/include/openvr.h && touch build/include/openvr-$openvr; }
 g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -Wno-missing-field-initializers -Ibuild/include -I../pointer/common \
   -o build/ft-gaze ft-gaze.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64 -lpthread
-echo "built build/ft-gaze"'
+stb=2c980bb59875b0d32144a71867fbdebb2f77cd20
+[ -f build/include/stb-$stb ] || { curl -fsSL "https://raw.githubusercontent.com/nothings/stb/$stb/stb_truetype.h" -o build/include/stb_truetype.h && touch build/include/stb-$stb; }
+g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -Wno-missing-field-initializers -Ibuild/include \
+  -o build/ft-gazepanel panel/ft-gazepanel.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64 -lpthread
+echo "built build/ft-gaze build/ft-gazepanel"'

gaze/ft-gazed

@@ -61,6 +61,10 @@ GAZE_TRACKER=own or the gaze probe asks for it ("eyes SECONDS", a lease the prob
 reads the eye-camera frames the root service frametop-eyegrab copies (gaze/tracker/install.sh),
 which copies them only while ft-eyes runs.
 
+Checks and calibration (gaze/gazecheck.py): a one-dot quick check when the headset goes on or
+our tracker asks for a click, and the full calibration when gaze mode comes on without one,
+both in a panel fixed to the headset (gaze/panel/ft-gazepanel, which this service runs too).
+
 Nothing here writes to SteamVR, its eye tracker, or its files: ft-gaze reads the eye
 tracker's shared memory read-only.
 
@@ -71,6 +75,10 @@ Control socket: abstract unix datagram "@ft_gazed":
   reload                           read calibration.json and the settings again
   eyes <seconds>                   keep our own tracker running that much longer (at most 120),
                                    whatever GAZE_TRACKER says: the probe's lease. Reply: "ok"
+  quickcal                         the one-dot check now (the calibration if there's none)
+  calibrate                        the full calibration in the panel
+  calaccept | calquit              from the pointer helper while the panel is up: take this dot
+                                   now (a left click, Meta+J) | close it (a right click, Meta+K)
 
 Options: --source action|mmap1|mmap2 (the older one-source path with that source, whatever
 the settings say; set 2 was a little quieter in the probe, but loses the pointer whenever
@@ -96,6 +104,7 @@ from pathlib import Path
 sys.path.insert(0, str(Path(__file__).resolve().parent))
 from gazecal import (DEFAULT_MODEL, EYE_FOUND, EYE_LOST, MODELS, STATE, Correction, EyeFallback,  # noqa: E402
                      EyeWeights, Fixation, LiveCorrection, SteamEyeLog)
+from gazecheck import Checks  # noqa: E402
 
 REPO = Path(__file__).resolve().parents[1]
 HELPER = REPO / "gaze" / "build" / "ft-gaze"
@@ -209,6 +218,7 @@ class Service:
         self.sel = selectors.DefaultSelector()
         self.sel.register(self.sock, selectors.EVENT_READ, "control")
         self.sel.register(self.eyes_sock, selectors.EVENT_READ, "own")
+        self.checks = Checks(self, self.sel)
         self.proc = None
         self.buf = b""
         self.restart_at = 0.0
@@ -273,6 +283,13 @@ class Service:
         tmp.replace(LESSONS)
         self.dirty = False
 
+    def forget_lessons(self):
+        """Drop what the lessons taught (the calibration stays)."""
+        self.lives = {name: PointerLessons() for name in SOURCES}
+        self.weights = {t: EyeWeights(self.bias) for t in TRACKERS}
+        self.refit()
+        self.save_lessons()
+
     def refit(self):
         for name, live in self.lives.items():
             live.wear_time = self.steam.worn()
@@ -505,6 +522,7 @@ class Service:
         return low
 
     def on_sample(self, s):
+        self.checks.on_sample(s)
         kind = self.kind
         if kind != self.last_kind:
             log({"own": "our own tracker", "eyes": "SteamVR's eyes, each calibrated",
@@ -636,6 +654,7 @@ class Service:
             if words[:1] == ["lesson"] and len(words) == 5:
                 try:
                     rec = self.lesson(*map(float, words[1:]))
+                    self.checks.after_lesson(rec)
                     reply = "refused" if "refused" in rec else f"ok {rec['lesson_deg']:.2f}"
                     log(f"lesson {rec['lesson_deg']:.2f} deg at {rec['raw'][0]:+.1f},{rec['raw'][1]:+.1f}"
                         + (f", eyes off {', '.join('-' if m is None else f'{m:.2f}' for m in rec['miss'])}"
@@ -646,11 +665,10 @@ class Service:
             elif words[:1] == ["status"]:
                 reply = json.dumps(self.status())
             elif words[:1] == ["forget"]:
-                self.lives = {name: PointerLessons() for name in SOURCES}
-                self.weights = {t: EyeWeights(self.bias) for t in TRACKERS}
-                self.refit()
-                self.save_lessons()
+                self.forget_lessons()
                 reply = "ok"
+            elif words[:1] and words[0] in ("quickcal", "calibrate", "calaccept", "calquit"):
+                reply = self.checks.command(words)
             elif words[:1] == ["eyes"] and len(words) == 2:
                 try:
                     secs = min(max(float(words[1]), 0.0), EYES_LEASE_MAX)
@@ -713,7 +731,7 @@ class Service:
                    if self.last_sample else None, "headset_on": self.steam.wearing(),
                    "headset_on_since": self.steam.worn(), "last": [round(v, 2) for v in self.last] if self.last else None,
                    "eyes_lost": self.lost, "fallback_ready": [self.fallback.ready(0), self.fallback.ready(1)],
-                   **self.counts})
+                   "checks": self.checks.status(), **self.counts})
         return st
 
     def periodic(self):
@@ -737,6 +755,7 @@ class Service:
                 self.eyes_sock.sendto(b"status", EYES_SOCKET)
             except OSError:
                 pass  # not up yet: status() says so once the last answer is old
+        self.checks.periodic()
         if self.dirty:
             self.save_lessons()
 
@@ -747,9 +766,13 @@ class Service:
             now = time.monotonic()
             if not self.proc and now >= self.restart_at:
                 self.start_helper()
-            for key, _ in self.sel.select(timeout=0.5):
+            for key, _ in self.sel.select(timeout=0.05 if self.checks.active else 0.5):
                 if key.data == "control":
                     self.on_control()
+                elif key.data == "checks":
+                    self.checks.on_readable()
+                elif key.data == "panel" and self.checks.panel_proc:
+                    self.checks.read_panel()
                 elif key.data == "own":
                     self.on_own()
                 elif key.data == "eyes" and self.eyes_proc:
@@ -758,12 +781,14 @@ class Service:
                     self.read_stdout()
                 elif key.data == "stderr" and self.proc:
                     self.read_stderr()
+            self.checks.tick()
             if now >= next_periodic:
                 self.periodic()
                 next_periodic = now + 1.0
             if self.verbose and now >= next_verbose:
                 log(json.dumps(self.status()))
                 next_verbose = now + 5
+        self.checks.stop()
         self.stop_helper()
         self.stop_eyes()
         if self.dirty:

gaze/gazecheck.py

@@ -0,0 +1,636 @@
+"""gazecheck: the gaze service's checks and calibration, in the panel fixed to the headset
+(gaze/panel/ft-gazepanel; ft-gazed runs it). Every kind is made of dots shown at head-relative
+directions: look at each one.
+
+  quick   one dot in the middle of your view. It opens DON_DELAY after the headset goes on
+          (SteamVR's eye tracking log says when), when our own tracker asks for a click (its
+          "reseat": the headset may sit differently on your face now), at most once every
+          QUICK_COOLDOWN, and on "quickcal" (Frametop Input Settings, or a mouse button or key
+          combination mapped to Gaze quick check). Ignored, it closes after QUICK_TIMEOUT and
+          changes nothing.
+  five    the middle and four around it, when the first FIVE_COUNT lessons after a quick check
+          were all over FIVE_LIMIT degrees off: the quick check didn't fix it.
+  full    the calibration, as the gaze probe's: three rounds, dark, medium and bright (pupil
+          size, and the tracker's error with it, changes with brightness), each the middle and
+          a ring of six (SteamVR's tracker) or eight (ours, whose fit goes wrong past its dots)
+          RING degrees out, half that in the middle round, turned 20 degrees a round. It opens
+          when gaze mode comes on without a calibration for the tracker in use, and on
+          "calibrate". Frametop's screens hide while it runs. Quitting it while there's still
+          no calibration turns gaze mode off (POINTER_GAZE=0); turning it on again reopens it.
+
+A dot captures itself: from CHECK_SETTLE after it shows (the eyes getting there), once the gaze
+has held within CHECK_SPREAD for CHECK_WINDOW (the probe's max spread and capture time). It's
+the gaze holding still that counts, not where the tracker puts it, so it works however far off
+the tracker is. A left click or Meta+J (the pointer helper's "calaccept") takes it now; a right
+click or Meta+K ("calquit") closes the panel. The pointer hides meanwhile ("calpanel 1",
+renewed every second; the helper shows it again by itself when that stops).
+
+What a capture teaches:
+  our tracker   quick and five: a click ("click T YAW PITCH", like a pointer lesson); full:
+                calib-start, a calib-point for each dot, calib-fit (its calibration)
+  SteamVR's     quick and five: a lesson for each eye, like the pointer's; full: each source's
+                calibration fitted from the dots (gazecal.Correction, the probe's way), saved
+                to calibration.json, and the lessons start over on top of it
+Either way, how far off each eye was goes to the eye bias, and each capture to the lesson log.
+"""
+
+import json
+import math
+import os
+import selectors
+import signal
+import socket
+import statistics
+import subprocess
+import sys
+import time
+from pathlib import Path
+
+from gazecal import DEFAULT_MODEL, STATE, steady_samples
+
+REPO = Path(__file__).resolve().parents[1]
+PANEL_PROG = REPO / "gaze" / "build" / "ft-gazepanel"
+PANEL = "\0ft_gazepanel"
+POINTER = "\0ft_pointer_helper"
+SCREENS = "\0ft_screens"
+EYES = "\0ft_eyes"
+CONF = Path.home() / ".config" / "frametop.conf"
+CALIBRATION = STATE / "calibration.json"
+POINTS = STATE / "points.jsonl"   # the probe's record of calibration dots, for its refine
+CHECK_LOG = STATE / "checks.jsonl"
+
+CHECK_SETTLE = 0.45
+CHECK_WINDOW = 0.6
+CHECK_SPREAD = 1.0     # degrees
+ACCEPT_SPREAD = 2.5    # degrees: a capture asked for (calaccept) takes this much
+DOT_TIMEOUT = 8.0      # seconds a dot of five or full waits; then it's skipped
+QUICK_TIMEOUT = 6.0
+QUICK_COOLDOWN = 120.0
+DON_DELAY = 3.0
+FIVE_LIMIT = 2.0
+FIVE_COUNT = 3
+DONE_PAUSE = 0.35      # seconds the filled dot shows before the next
+RING = 20.0
+ROUND_BG = (0.03, 0.33, 0.8)
+ROUND_NAMES = ("dark", "medium", "bright")
+RING_SCALE = (1.0, 0.5, 1.0)
+PANEL_RETRY = 10.0
+
+
+def log(msg):
+    print(f"ft-gazed: {msg}", file=sys.stderr, flush=True)
+
+
+def check_dots(kind, own):
+    """(yaw, pitch, round): head-relative degrees, yaw +left, pitch +up."""
+    if kind == "quick":
+        return [(0.0, 0.0, 0)]
+    if kind == "five":
+        return [(0.0, 0.0, 0), (12.0, 0.0, 0), (-12.0, 0.0, 0), (0.0, 9.0, 0), (0.0, -9.0, 0)]
+    out = []
+    n = 8 if own else 6
+    for rnd in range(3):
+        out.append((0.0, 0.0, rnd))
+        r = RING * RING_SCALE[rnd]
+        for i in range(n):
+            a = math.radians(-90 + rnd * 20 + i * 360 / n)  # as the probe: x right, y down
+            x, y = r * math.cos(a), r * math.sin(a) * (0.9 if own else 1.0)
+            out.append((-x, -y, rnd))
+    return out
+
+
+def spread(points):
+    """The median point and the spread around it (1.4826 x the median distance: a standard
+    deviation that one stray sample can't move far)."""
+    mx = statistics.median(p[0] for p in points)
+    my = statistics.median(p[1] for p in points)
+    return 1.4826 * statistics.median(math.hypot(p[0] - mx, p[1] - my) for p in points), (mx, my)
+
+
+def write_gaze(on, path=CONF):
+    """POINTER_GAZE=1|0 in the config, every other line kept."""
+    try:
+        lines = path.read_text().splitlines()
+    except OSError:
+        lines = []
+    value = f"POINTER_GAZE={1 if on else 0}"
+    for i, line in enumerate(lines):
+        if line.split("#", 1)[0].split("=", 1)[0].strip() == "POINTER_GAZE":
+            lines[i] = value
+            break
+    else:
+        lines.append(value)
+    tmp = path.with_suffix(".tmp")
+    tmp.write_text("\n".join(lines) + "\n")
+    tmp.replace(path)
+
+
+def ask(addr, command, timeout=1.0):
+    """A command to a local datagram socket, and its reply ("" without one)."""
+    s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC)
+    try:
+        s.bind("")
+        s.settimeout(timeout)
+        s.sendto(command.encode(), addr)
+        return s.recv(4096).decode("utf-8", "replace")
+    except OSError:
+        return ""
+    finally:
+        s.close()
+
+
+class Checks:
+    def __init__(self, svc, sel):
+        self.svc = svc
+        self.sel = sel
+        self.out = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC | socket.SOCK_NONBLOCK)
+        self.out.bind("")  # the panel's and the helper's replies come back here
+        sel.register(self.out, selectors.EVENT_READ, "checks")
+        self.check = None
+        self.gaze_on = None
+        self.gaze_heard = 0.0
+        self.want_full_until = 0.0  # gaze mode came on before the tracker said whether it's calibrated
+        self.last_quick = 0.0
+        self.worn_seen = svc.steam.worn()
+        self.don_at = None
+        self.reseat_seen = False
+        self.after_quick = None
+        self.panel_proc = None
+        self.panel_restart_at = 0.0
+        self.screens_shown = None
+        self.last_progress = 0.0
+
+    @property
+    def active(self):
+        return self.check is not None
+
+    # --- The panel process ---
+
+    def start_panel(self):
+        if not PANEL_PROG.exists():
+            log(f"ft-gazepanel isn't built: run {REPO}/gaze/build.sh")
+            self.panel_restart_at = time.monotonic() + 60
+            return
+        env = dict(os.environ)
+        env["XDG_RUNTIME_DIR"] = f"/run/user/{os.getuid()}"
+        distrobox = Path.home() / ".local" / "bin" / "distrobox"
+        self.panel_proc = subprocess.Popen([str(distrobox), "enter", "dev", "--", str(PANEL_PROG), "--watch-stdin"],
+                                           env=env, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL,
+                                           stderr=subprocess.PIPE, start_new_session=True)
+        os.set_blocking(self.panel_proc.stderr.fileno(), False)
+        self.sel.register(self.panel_proc.stderr, selectors.EVENT_READ, "panel")
+        log("ft-gazepanel started")
+
+    def read_panel(self):
+        try:
+            data = os.read(self.panel_proc.stderr.fileno(), 65536)
+        except BlockingIOError:
+            return
+        if not data:
+            log(f"ft-gazepanel stopped (exit {self.panel_proc.poll()}); again in {PANEL_RETRY:.0f} s")
+            self.stop_panel()
+            self.panel_restart_at = time.monotonic() + PANEL_RETRY
+            if self.check:
+                self.close("the panel stopped")
+            return
+        for line in data.decode("utf-8", "replace").splitlines():
+            if line.strip():
+                log(line)
+
+    def stop_panel(self):
+        if not self.panel_proc:
+            return
+        try:
+            self.sel.unregister(self.panel_proc.stderr)
+        except (KeyError, ValueError):
+            pass
+        if self.panel_proc.stdin and not self.panel_proc.stdin.closed:
+            self.panel_proc.stdin.close()
+        try:
+            self.panel_proc.wait(timeout=2)
+        except subprocess.TimeoutExpired:
+            try:
+                os.killpg(self.panel_proc.pid, signal.SIGTERM)
+            except ProcessLookupError:
+                pass
+        self.panel_proc = None
+
+    def to_panel(self, command):
+        try:
+            self.out.sendto(command.encode(), PANEL)
+        except OSError:
+            pass
+
+    def to_helper(self, command):
+        try:
+            self.out.sendto(command.encode(), POINTER)
+        except OSError:
+            pass
+
+    def on_readable(self):
+        """Replies on our socket: the helper's "ok on|off" to "gaze ?"; the panel's are dropped."""
+        while True:
+            try:
+                data = self.out.recv(4096).decode("utf-8", "replace")
+            except (BlockingIOError, OSError):
+                return
+            if data in ("ok on", "ok off"):
+                on = data == "ok on"
+                was, self.gaze_on, self.gaze_heard = self.gaze_on, on, time.monotonic()
+                if on and was is False:
+                    self.on_gaze_on()
+
+    # --- State ---
+
+    def calibrated(self):
+        """Does the tracker in use have a calibration? None: our tracker hasn't said yet."""
+        svc = self.svc
+        kind = svc.kind
+        if kind == "own":
+            if time.monotonic() - svc.own_at > 5 or not svc.own:
+                return None
+            return bool((svc.own.get("calibration") or {}).get("dots"))
+        if kind == "eyes":
+            return True
+        return svc.models[svc.source].samples > 0
+
+    def can_run(self):
+        """The headset is on and the tracker is sending."""
+        return self.svc.steam.wearing() is not False and time.monotonic() - self.svc.last_sample < 2
+
+    def on_gaze_on(self):
+        cal = self.calibrated()
+        if cal is False:
+            self.start("full", "gaze mode came on without a calibration")
+        elif cal is None:
+            self.want_full_until = time.monotonic() + 20
+
+    def auto_quick(self, reason):
+        now = time.monotonic()
+        if self.check or not self.gaze_on or not self.can_run() or self.calibrated() is not True:
+            return
+        if now - self.last_quick < QUICK_COOLDOWN:
+            log(f"quick check skipped ({reason}): one ran {now - self.last_quick:.0f} s ago")
+            return
+        self.start("quick", reason)
+
+    # --- Running a check ---
+
+    def start(self, kind, reason):
+        svc = self.svc
+        if self.check:
+            return "error a check is running"
+        if not self.panel_proc:
+            return "error the panel isn't running (gaze/build.sh builds it)"
+        if not self.can_run():
+            return "error the headset is off or the tracker isn't sending"
+        own = svc.kind == "own"
+        if kind == "full" and own:
+            reply = ask(EYES, "calib-start", 3.0)
+            if not reply.startswith("ok"):
+                log(f"calibration not started: our tracker says {reply or 'nothing'}")
+                return f"error our tracker: {reply or 'no reply'}"
+        now = time.monotonic()
+        self.check = {"kind": kind, "reason": reason, "own": own, "dots": check_dots(kind, own), "i": 0,
+                      "started": now, "shown": now, "run": [], "accept": False, "done_at": None, "tries": 0,
+                      "skipped": 0, "captured": 0, "points": {}}
+        log(f"{kind} check: {reason}")
+        if kind == "full":
+            st = ask(SCREENS, "state", 0.5).split()
+            if len(st) >= 3 and st[0] == "ok":
+                self.screens_shown = (st[2] == "0") if st[1] == "always" else (st[2] == "1")
+                ask(SCREENS, "hide", 0.5)
+        self.to_helper("calpanel 1")
+        self.to_panel(f"show {'full' if kind == 'full' else 'quick'}")
+        self.show_dot()
+        if kind == "quick":
+            self.last_quick = now
+        return "ok"
+
+    def show_dot(self):
+        c = self.check
+        yaw, pitch, rnd = c["dots"][c["i"]]
+        if c["kind"] == "full":
+            self.to_panel(f"bg {ROUND_BG[rnd]}")
+            self.to_panel(f"title Gaze calibration: {ROUND_NAMES[rnd]} round, {rnd + 1} of 3")
+            self.to_panel("text Look at the dot. Left click or Meta+J: now. Right click or Meta+K: stop")
+        elif c["kind"] == "five":
+            self.to_panel(f"text Look at the dot ({c['i'] + 1} of {len(c['dots'])})")
+        else:
+            self.to_panel("text Look at the dot")
+        self.to_panel(f"dot {yaw:.3f} {pitch:.3f} look")
+        c["shown"] = time.monotonic()
+        c["run"], c["accept"], c["done_at"] = [], False, None
+
+    def on_sample(self, s):
+        c = self.check
+        if not c or c["done_at"]:
+            return
+        if c["own"]:
+            src = s["src"].get("own") or {}
+        else:
+            src = s["src"].get("mmap1") or {}
+            if "hy" not in src:
+                per = [s["src"].get(n) or {} for n in ("left", "right")]
+                per = [p for p in per if "hy" in p]
+                src = {"hy": statistics.fmean(p["hy"] for p in per), "hp": statistics.fmean(p["hp"] for p in per)} if per else {}
+        if "hy" not in src:
+            return
+        now = time.monotonic()
+        if now < c["shown"] + CHECK_SETTLE:
+            return
+        g = (src["hy"], src["hp"])
+        run = c["run"]
+        if run:
+            recent = run[-30:]
+            my, mp = statistics.median(p[2] for p in recent), statistics.median(p[3] for p in recent)
+            if math.hypot(g[0] - my, g[1] - mp) > 2.5 * CHECK_SPREAD:
+                run.clear()  # the eyes moved on (a saccade, a blink): start over
+        run.append((now, s, g[0], g[1]))
+        window = [p for p in run if p[0] >= now - CHECK_WINDOW]
+        held = now - run[0][0]
+        if now - self.last_progress > 0.07:
+            yaw, pitch, _ = c["dots"][c["i"]]
+            self.to_panel(f"dot {yaw:.3f} {pitch:.3f} capture {min(1.0, held / CHECK_WINDOW):.2f}")
+            self.last_progress = now
+        if c["accept"] and held >= 0.3 and len(window) >= 10:
+            sd, _ = spread([(p[2], p[3]) for p in window])
+            if sd <= ACCEPT_SPREAD:
+                self.capture(window)
+            return
+        if held >= CHECK_WINDOW and len(window) >= 20:
+            sd, _ = spread([(p[2], p[3]) for p in window])
+            if sd <= CHECK_SPREAD:
+                self.capture(window)
+            else:
+                del run[:len(run) // 2]  # not steady enough yet: keep trying with the newer half
+
+    def capture(self, window):
+        svc = self.svc
+        c = self.check
+        yaw, pitch, rnd = c["dots"][c["i"]]
+        samples = [p[1] for p in window]
+        rec = {"time": time.time(), "check": c["kind"], "dot": c["i"], "round": rnd, "true": [yaw, pitch],
+               "samples": len(samples), "own": c["own"]}
+        ok = True
+        if c["own"]:
+            eyes = []
+            for k in (0, 1):
+                seen = [smp["src"]["own"]["eyes"][k] for smp in samples
+                        if (smp["src"].get("own") or {}).get("eyes") and smp["src"]["own"]["eyes"][k]]
+                eyes.append((statistics.median(e[0] for e in seen), statistics.median(e[1] for e in seen)) if seen else None)
+            miss = [math.hypot(yaw - e[0], pitch - e[1]) if e else None for e in eyes]
+            rec.update(eyes=eyes, miss=miss)
+            t0, t1 = samples[0]["t"], samples[-1]["t"]
+            if c["kind"] == "full":
+                reply = ask(EYES, f"calib-point {t0:.6f} {t1:.6f} {yaw:.4f} {pitch:.4f}", 3.0)
+                rec["reply"] = reply
+                ok = reply.startswith("ok")
+            else:
+                try:
+                    svc.eyes_sock.sendto(f"click {t1:.6f} {yaw:.4f} {pitch:.4f}".encode(), EYES)
+                except OSError as e:
+                    rec["reply"], ok = f"our tracker isn't running ({e})", False
+            if ok:
+                svc.weights["own"].add(miss)
+        else:
+            steady = steady_samples(samples)
+            reads = {}
+            for name in ("action", "mmap1", "mmap2", "left", "right"):
+                pts = [(smp["src"][name]["hy"], smp["src"][name]["hp"]) for smp in steady
+                       if "hy" in (smp["src"].get(name) or {})]
+                if len(pts) >= 15:
+                    reads[name] = (statistics.median(p[0] for p in pts), statistics.median(p[1] for p in pts))
+            rec["reads"] = reads
+            main = ("left", "right") if svc.kind == "eyes" else (svc.source,)
+            if not all(n in reads for n in main):
+                ok = False
+                rec["reply"] = f"only {len(steady)} of {len(samples)} samples had both eyes"
+            elif c["kind"] == "full":
+                for name, (hy, hp) in reads.items():
+                    c["points"].setdefault(name, []).append((hy, hp, yaw - hy, pitch - hp))
+                try:
+                    with open(POINTS, "a") as f:
+                        for name, (hy, hp) in reads.items():
+                            f.write(json.dumps({"time": time.time(), "source": name, "hy": hy, "hp": hp,
+                                                "off": [yaw - hy, pitch - hp], "layout": None, "how": "panel"}) + "\n")
+                except OSError:
+                    pass
+            else:
+                miss = []
+                for name in main:
+                    hy, hp = reads[name]
+                    cy, cp = svc.correction(name, hy, hp)
+                    miss.append(math.hypot(yaw - hy - cy, pitch - hp - cp))
+                    svc.lives[name].add({"time": time.time(), "hy": hy, "hp": hp, "dy": yaw - hy, "dp": pitch - hp,
+                                         "wy": 1.0, "wp": 1.0, "how": "check"}, svc.models[name], svc.mode)
+                rec["miss"] = miss
+                if svc.kind == "eyes":
+                    svc.weights["steam"].add(miss)
+                svc.dirty = True
+        rec["accepted"] = ok
+        self.log_check(rec)
+        if not ok:
+            c["tries"] += 1
+            log(f"{c['kind']} check dot {c['i'] + 1}: not taken ({rec.get('reply', '')})")
+            if c["tries"] >= 2 or c["kind"] != "full":
+                self.skip()
+            else:
+                self.to_panel(f"dot {yaw:.3f} {pitch:.3f} fail")
+                c["run"], c["accept"] = [], False
+                c["shown"] = time.monotonic()  # settle again, then retry
+            return
+        c["captured"] += 1
+        c["tries"] = 0
+        self.to_panel(f"dot {yaw:.3f} {pitch:.3f} done")
+        c["done_at"] = time.monotonic() + DONE_PAUSE
+
+    def skip(self):
+        c = self.check
+        yaw, pitch, _ = c["dots"][c["i"]]
+        c["skipped"] += 1
+        c["tries"] = 0
+        self.to_panel(f"dot {yaw:.3f} {pitch:.3f} fail")
+        c["done_at"] = time.monotonic() + DONE_PAUSE
+
+    def advance(self):
+        c = self.check
+        c["i"] += 1
+        if c["i"] < len(c["dots"]):
+            self.show_dot()
+            return
+        self.finish()
+
+    def finish(self):
+        svc = self.svc
+        c = self.check
+        kind = c["kind"]
+        if kind in ("quick", "five"):
+            if c["captured"]:
+                log(f"{kind} check done ({c['captured']} of {len(c['dots'])} dots)")
+                self.after_quick = [] if kind == "quick" else None
+            self.close()
+            return
+        n = len(c["dots"])
+        if c["captured"] < n * 2 / 3:
+            log(f"calibration failed: only {c['captured']} of {n} dots; the old one stays")
+            self.to_panel(f"text Calibration failed: only {c['captured']} of {n} dots. Try again from Input Settings")
+            self.to_panel("dot 0 0 off")
+            c["done_at"] = time.monotonic() + 3.0
+            c["closing"] = True
+            return
+        if c["own"]:
+            reply = ask(EYES, "calib-fit", 10.0)
+            log(f"calibration ({c['captured']} of {n} dots): our tracker says {reply or 'nothing'}")
+        else:
+            mode = svc.mode if svc.mode != "none" else DEFAULT_MODEL
+            for name, pts in c["points"].items():
+                if name in svc.models and pts:
+                    svc.models[name].fit(pts, mode)
+            d = {name: m.to_json() for name, m in svc.models.items()}
+            d["_meta"] = {"calibrated_at": time.time(), "model": mode, "how": "panel"}
+            d["_live"] = {}
+            tmp = CALIBRATION.with_suffix(".tmp")
+            tmp.write_text(json.dumps(d, indent=1))
+            tmp.replace(CALIBRATION)
+            svc.forget_lessons()  # a new calibration: the pointer's lessons start over on top of it
+            svc.load_calibration()
+            svc.refit()
+            log(f"calibration ({c['captured']} of {n} dots): {mode}, saved")
+        self.close()
+
+    def close(self, why=None):
+        if not self.check:
+            return
+        if why:
+            log(f"{self.check['kind']} check closed: {why}")
+        self.to_panel("hide")
+        self.to_helper("calpanel 0")
+        if self.check["kind"] == "full" and self.screens_shown:
+            ask(SCREENS, "show", 0.5)
+        self.screens_shown = None
+        self.check = None
+
+    def quit(self):
+        c = self.check
+        if not c:
+            return
+        self.close("quit")
+        if c["kind"] == "full" and self.calibrated() is False:
+            # No calibration still: gaze mode can't work, so it goes off until it's turned on again.
+            try:
+                write_gaze(False)
+            except OSError as e:
+                log(f"can't write {CONF}: {e}")
+            self.to_helper("gaze off")
+            self.gaze_on = False
+            log("gaze mode off: no calibration")
+
+    def log_check(self, rec):
+        try:
+            with open(CHECK_LOG, "a") as f:
+                f.write(json.dumps(rec) + "\n")
+        except OSError:
+            pass
+
+    # --- From the service ---
+
+    def command(self, words):
+        """quickcal, calibrate, calaccept, calquit -> a reply."""
+        cmd = words[0]
+        if cmd == "quickcal":
+            return self.start("full" if self.calibrated() is False else "quick", "asked for")
+        if cmd == "calibrate":
+            return self.start("full", "asked for")
+        if cmd == "calaccept":
+            if self.check:
+                self.check["accept"] = True
+            return "ok"
+        if cmd == "calquit":
+            self.quit()
+            return "ok"
+        return "error unknown command"
+
+    def after_lesson(self, rec):
+        if self.after_quick is None or "refused" in rec:
+            return
+        self.after_quick.append(rec.get("lesson_deg", 0.0))
+        if len(self.after_quick) < FIVE_COUNT:
+            return
+        off = self.after_quick
+        self.after_quick = None
+        if all(d > FIVE_LIMIT for d in off):
+            log(f"the {FIVE_COUNT} lessons after the quick check were {', '.join(f'{d:.1f}' for d in off)} deg off")
+            if self.gaze_on and self.can_run() and not self.check:
+                self.start("five", "the quick check didn't fix it")
+
+    def tick(self):
+        c = self.check
+        if not c:
+            return
+        now = time.monotonic()
+        if c["done_at"] and now >= c["done_at"]:
+            if c.get("closing"):
+                self.close()
+            else:
+                self.advance()
+            return
+        if self.svc.steam.wearing() is False:
+            self.close("the headset came off")
+            return
+        if c["done_at"]:
+            return
+        if c["kind"] == "quick" and now - c["started"] > QUICK_TIMEOUT:
+            self.close("ignored")
+        elif c["kind"] != "quick" and now - c["shown"] > DOT_TIMEOUT:
+            log(f"{c['kind']} check dot {c['i'] + 1}: skipped (the gaze didn't hold still)")
+            self.skip()
+
+    def periodic(self):
+        svc = self.svc
+        now = time.monotonic()
+        if not self.panel_proc and now >= self.panel_restart_at:
+            self.start_panel()
+        self.to_helper("gaze ?")
+        if now - self.gaze_heard > 5:
+            self.gaze_on = None  # the helper isn't answering
+        if self.check:
+            self.to_helper("calpanel 1")
+        if self.want_full_until:
+            cal = self.calibrated()
+            if cal is not None or now > self.want_full_until:
+                self.want_full_until = 0.0
+                if cal is False and self.gaze_on:
+                    self.start("full", "gaze mode came on without a calibration")
+        worn = svc.steam.worn()
+        if worn and worn != self.worn_seen:
+            self.worn_seen = worn
+            self.don_at = worn + DON_DELAY
+        # Both wait for the tracker to send (it takes a moment after the headset goes on).
+        if self.don_at and time.time() >= self.don_at:
+            if self.can_run():
+                self.don_at = None
+                self.auto_quick("the headset went on")
+            elif time.time() > self.don_at + 20:
+                self.don_at = None
+        if svc.kind == "own" and now - svc.own_at < 5:
+            reseat = any(e.get("reseat") for e in (svc.own.get("eyes") or {}).values())
+            if not reseat:
+                self.reseat_seen = False
+            elif not self.reseat_seen and self.can_run():
+                self.reseat_seen = True
+                self.auto_quick("our tracker asked for a click")
+
+    def status(self):
+        c = self.check
+        st = {"check": None, "gaze_mode": self.gaze_on, "calibrated": self.calibrated(),
+              "panel": self.panel_proc is not None,
+              "last_quick_s": round(time.monotonic() - self.last_quick) if self.last_quick else None}
+        if c:
+            st["check"] = {"kind": c["kind"], "dot": c["i"] + 1, "dots": len(c["dots"]), "captured": c["captured"],
+                           "skipped": c["skipped"], "reason": c["reason"]}
+        return st
+
+    def stop(self):
+        self.close("the gaze service is stopping")
+        self.stop_panel()

gaze/panel/ft-gazepanel.cpp

@@ -0,0 +1,360 @@
+// ft-gazepanel: the gaze calibration panel (docs/design.md, gaze/README.md). A SteamVR overlay
+// fixed to the headset, so wherever you turn your head it stays in the same place in your
+// view: a dot drawn at a head-relative direction is exactly that direction from the headset,
+// which is what the gaze service needs to know where you were looking. It's drawn on the CPU
+// and takes no input: the gaze service (gaze/ft-gazed) drives it, and the pointer helper
+// passes it your presses (calaccept, calquit).
+//
+// The panel sits POINTER-like at --distance (1.5 m, about where Frametop's screens are, so
+// the eyes converge as they do in use). "quick" is a small square, QUICK_DEG across, for the
+// one-dot check; "full" is FULL_DEG across (4:3), with a solid background whose brightness the
+// service sets per round (pupil size changes with it, and the tracker's error with it).
+//
+// Control socket: abstract unix datagram "@ft_gazepanel" (--socket NAME); a sender with an
+// address gets "ok" or "error ...":
+//   show quick|full              the panel, empty, in front of you
+//   hide
+//   bg <0..1>                    the background's brightness (full)
+//   dot <yaw> <pitch> <state> [<progress 0..1>]
+//                                the dot, head-relative degrees (yaw +left, pitch +up); state:
+//                                look (pulsing), capture (a ring filling to progress), done,
+//                                fail, off
+//   title <text> / text <text>   a line at the top / at the bottom (empty to clear)
+//   ping
+//
+// Options: --watch-stdin (quit when stdin closes: the service runs it), --socket NAME,
+// --distance METRES. Runs in the dev container (gaze/build.sh builds it into gaze/build).
+#include <openvr.h>
+
+#define STB_TRUETYPE_IMPLEMENTATION
+#include "stb_truetype.h"
+
+#include <sys/socket.h>
+#include <sys/un.h>
+#include <unistd.h>
+
+#include <algorithm>
+#include <atomic>
+#include <chrono>
+#include <cmath>
+#include <csignal>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <map>
+#include <string>
+#include <thread>
+#include <vector>
+
+namespace {
+
+using Clock = std::chrono::steady_clock;
+constexpr double kQuickDeg = 16;      // QUICK_DEG: the one-dot check's square
+constexpr double kFullDeg = 64;       // FULL_DEG: the full calibration's width (4:3)
+constexpr int kQuickPx = 320, kFullW = 1024, kFullH = 768;
+std::atomic<bool> g_stop{false};
+
+// ---------------------------------------------------------------- text (as screens/keyboard.cpp)
+
+stbtt_fontinfo g_font;
+std::vector<unsigned char> g_fontData;
+bool g_fontOk = false;
+struct Glyph {
+    std::vector<unsigned char> bitmap;
+    int w = 0, h = 0, xoff = 0, yoff = 0, advance = 0;
+};
+std::map<std::pair<uint32_t, int>, Glyph> g_glyphs;
+
+void LoadFont() {
+    std::string path;
+    if (FILE *p = popen("fc-match -f '%{file}' 'Noto Sans' 2>/dev/null", "r")) {
+        char buf[512];
+        if (std::fgets(buf, sizeof buf, p)) path = buf;
+        pclose(p);
+    }
+    if (path.empty()) path = "/usr/share/fonts/google-noto-vf/NotoSans[wght].ttf";
+    if (FILE *f = std::fopen(path.c_str(), "rb")) {
+        std::fseek(f, 0, SEEK_END);
+        g_fontData.resize(size_t(std::ftell(f)));
+        std::fseek(f, 0, SEEK_SET);
+        g_fontOk = std::fread(g_fontData.data(), 1, g_fontData.size(), f) == g_fontData.size() &&
+                   stbtt_InitFont(&g_font, g_fontData.data(), stbtt_GetFontOffsetForIndex(g_fontData.data(), 0));
+        std::fclose(f);
+    }
+    if (!g_fontOk) std::fprintf(stderr, "ft-gazepanel: no font (%s); no text\n", path.c_str());
+}
+
+const Glyph &GetGlyph(uint32_t cp, int size) {
+    auto [it, fresh] = g_glyphs.try_emplace({cp, size});
+    Glyph &g = it->second;
+    if (fresh) {
+        const float scale = stbtt_ScaleForPixelHeight(&g_font, float(size));
+        unsigned char *b = stbtt_GetCodepointBitmap(&g_font, 0, scale, int(cp), &g.w, &g.h, &g.xoff, &g.yoff);
+        if (b) g.bitmap.assign(b, b + size_t(g.w) * g.h), stbtt_FreeBitmap(b, nullptr);
+        int adv, lsb;
+        stbtt_GetCodepointHMetrics(&g_font, int(cp), &adv, &lsb);
+        g.advance = int(std::lround(adv * scale));
+    }
+    return g;
+}
+
+std::vector<uint32_t> Codepoints(const std::string &s) {
+    std::vector<uint32_t> out;
+    for (const unsigned char *p = (const unsigned char *)s.c_str(); *p;) {
+        uint32_t c = *p++;
+        int more = c >= 0xF0 ? 3 : c >= 0xE0 ? 2 : c >= 0xC0 ? 1 : 0;
+        if (more) c &= 0x3Fu >> more;
+        for (; more && (*p & 0xC0) == 0x80; --more) c = c << 6 | (*p++ & 0x3F);
+        out.push_back(c);
+    }
+    return out;
+}
+
+// ---------------------------------------------------------------- the picture
+
+struct Panel {
+    bool full = false;
+    int w = kQuickPx, h = kQuickPx;
+    double wDeg = kQuickDeg;  // across
+    std::vector<uint8_t> px;
+    double bg = 0.05;
+    std::string title, text;
+    bool dotOn = false;
+    double dotYaw = 0, dotPitch = 0, progress = 0;
+    std::string state = "off";
+    Clock::time_point stateAt = Clock::now();
+};
+
+void Blend(Panel &p, int x, int y, double r, double g, double b, double a) {
+    if (x < 0 || y < 0 || x >= p.w || y >= p.h || a <= 0) return;
+    uint8_t *q = &p.px[(size_t(y) * p.w + x) * 4];
+    a = std::min(a, 1.0);
+    q[0] = uint8_t(std::lround(q[0] + (r * 255 - q[0]) * a));
+    q[1] = uint8_t(std::lround(q[1] + (g * 255 - q[1]) * a));
+    q[2] = uint8_t(std::lround(q[2] + (b * 255 - q[2]) * a));
+    q[3] = uint8_t(std::lround(q[3] + (255 - q[3]) * a));
+}
+
+// A filled disc, or (inner > 0) a ring from inner to outer radius, anti-aliased; with sweep < 1,
+// only that share of the ring, clockwise from the top.
+void Disc(Panel &p, double cx, double cy, double outer, double inner, double r, double g, double b, double a,
+          double sweep = 1) {
+    const int x0 = int(cx - outer - 1), x1 = int(cx + outer + 1), y0 = int(cy - outer - 1), y1 = int(cy + outer + 1);
+    for (int y = y0; y <= y1; ++y)
+        for (int x = x0; x <= x1; ++x) {
+            const double dx = x + 0.5 - cx, dy = y + 0.5 - cy, d = std::hypot(dx, dy);
+            double cover = std::clamp(outer - d + 0.5, 0.0, 1.0);
+            if (inner > 0) cover = std::min(cover, std::clamp(d - inner + 0.5, 0.0, 1.0));
+            if (sweep < 1) {
+                double ang = std::atan2(dx, -dy) / (2 * M_PI);  // 0 at the top, clockwise
+                if (ang < 0) ang += 1;
+                if (ang > sweep) continue;
+            }
+            Blend(p, x, y, r, g, b, a * cover);
+        }
+}
+
+void Text(Panel &p, const std::string &s, int size, int cx, int cy, double lum) {
+    if (!g_fontOk || s.empty()) return;
+    const auto cps = Codepoints(s);
+    int width = 0;
+    for (uint32_t cp : cps) width += GetGlyph(cp, size).advance;
+    int ascent, descent, gap;
+    stbtt_GetFontVMetrics(&g_font, &ascent, &descent, &gap);
+    const float scale = stbtt_ScaleForPixelHeight(&g_font, float(size));
+    int x = cx - width / 2;
+    const int baseline = cy + int(std::lround((ascent + descent) * scale / 2));
+    for (uint32_t cp : cps) {
+        const Glyph &g = GetGlyph(cp, size);
+        for (int gy = 0; gy < g.h; ++gy)
+            for (int gx = 0; gx < g.w; ++gx)
+                Blend(p, x + g.xoff + gx, baseline + g.yoff + gy, lum, lum, lum, g.bitmap[size_t(gy) * g.w + gx] / 255.0);
+        x += g.advance;
+    }
+}
+
+// Head-relative direction -> panel pixel: the panel is a plane `d` in front of the headset.
+void ToPixel(const Panel &p, double yaw, double pitch, double &x, double &y) {
+    const double yr = yaw * M_PI / 180, pr = pitch * M_PI / 180;
+    const double half = std::tan(p.wDeg * M_PI / 360);  // half the width, per unit of distance
+    const double X = -std::tan(yr), Y = std::tan(pr) / std::cos(yr);
+    x = (X / (2 * half) + 0.5) * p.w;
+    y = (0.5 - Y / (2 * half) * p.w / p.h) * p.h;
+}
+
+void Draw(Panel &p) {
+    p.px.assign(size_t(p.w) * p.h * 4, 0);
+    const double pxPerDeg = p.w / p.wDeg;
+    if (p.full) {
+        for (size_t i = 0; i < p.px.size(); i += 4)
+            p.px[i] = p.px[i + 1] = p.px[i + 2] = uint8_t(std::lround(p.bg * 255)), p.px[i + 3] = 255;
+    } else {
+        // The quick check: a dim rounded square, see-through, so it's clear of what's behind.
+        const double r = p.w * 0.12;
+        for (int y = 0; y < p.h; ++y)
+            for (int x = 0; x < p.w; ++x) {
+                const double dx = std::max({r - x - 0.5, x + 0.5 - (p.w - r), 0.0});
+                const double dy = std::max({r - y - 0.5, y + 0.5 - (p.h - r), 0.0});
+                const double cover = std::clamp(r - std::hypot(dx, dy) + 0.5, 0.0, 1.0);
+                Blend(p, x, y, 0.06, 0.06, 0.07, 0.82 * cover);
+            }
+    }
+    const bool light = p.full && p.bg > 0.5;  // a dark dot on the bright round
+    const double ink = light ? 0.0 : 1.0, faint = light ? 0.2 : 0.75;
+    const int titleSize = int(pxPerDeg * (p.full ? 1.5 : 1.1)), textSize = int(pxPerDeg * (p.full ? 1.2 : 0.9));
+    Text(p, p.title, titleSize, p.w / 2, int(titleSize * 1.2), faint);
+    Text(p, p.text, textSize, p.w / 2, p.h - int(textSize * 1.3), faint);
+    if (!p.dotOn || p.state == "off") return;
+    double x, y;
+    ToPixel(p, p.dotYaw, p.dotPitch, x, y);
+    const double core = 0.22 * pxPerDeg;
+    const double t = std::chrono::duration<double>(Clock::now() - p.stateAt).count();
+    if (p.state == "look") {
+        // A ring that draws in toward the dot, again and again, and a still centre to look at.
+        const double phase = std::fmod(t * 1.4, 1.0);
+        const double rr = (1.6 - 1.1 * phase) * pxPerDeg;
+        Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, ink, ink, ink, 0.55 * (1 - phase) + 0.2);
+        Disc(p, x, y, core, 0, ink, ink, ink, 1);
+    } else if (p.state == "capture") {
+        const double rr = 0.75 * pxPerDeg;
+        Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, ink, ink, ink, 0.25);
+        Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, 0.3, 0.85, 1.0, 1, std::clamp(p.progress, 0.0, 1.0));
+        Disc(p, x, y, core, 0, ink, ink, ink, 1);
+    } else if (p.state == "done") {
+        Disc(p, x, y, 0.75 * pxPerDeg, 0, 0.25, 0.85, 0.4, 0.9);
+        Disc(p, x, y, core, 0, 1, 1, 1, 1);
+    } else if (p.state == "fail") {
+        Disc(p, x, y, 0.75 * pxPerDeg, 0.6 * pxPerDeg, 0.95, 0.35, 0.3, 0.9);
+        Disc(p, x, y, core, 0, ink, ink, ink, 1);
+    }
+}
+
+bool Animating(const Panel &p) { return p.dotOn && (p.state == "look"); }
+
+}  // namespace
+
+int main(int argc, char **argv) {
+    bool watchStdin = false;
+    std::string sockName = "ft_gazepanel";
+    double distance = 1.5;
+    for (int i = 1; i < argc; ++i) {
+        if (!std::strcmp(argv[i], "--watch-stdin")) watchStdin = true;
+        else if (!std::strcmp(argv[i], "--socket") && i + 1 < argc) sockName = argv[++i];
+        else if (!std::strcmp(argv[i], "--distance") && i + 1 < argc) distance = std::clamp(std::atof(argv[++i]), 0.5, 5.0);
+        else {
+            std::fprintf(stderr, "usage: %s [--watch-stdin] [--socket NAME] [--distance METRES]\n", argv[0]);
+            return 2;
+        }
+    }
+    std::signal(SIGINT, [](int) { g_stop = true; });
+    std::signal(SIGTERM, [](int) { g_stop = true; });
+    if (watchStdin)
+        std::thread([] {
+            char c[256];
+            while (read(0, c, sizeof c) > 0) {
+            }
+            g_stop = true;
+        }).detach();
+
+    int sock = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
+    sockaddr_un addr{};
+    addr.sun_family = AF_UNIX;
+    std::memcpy(addr.sun_path + 1, sockName.data(), std::min(sockName.size(), sizeof addr.sun_path - 2));
+    if (bind(sock, reinterpret_cast<sockaddr *>(&addr), socklen_t(offsetof(sockaddr_un, sun_path) + 1 + sockName.size())) != 0) {
+        std::fprintf(stderr, "ft-gazepanel: @%s is taken (another copy running?)\n", sockName.c_str());
+        return 1;
+    }
+
+    // As Frametop's other SteamVR clients: background first, so we never start vrserver.
+    vr::EVRInitError err = vr::VRInitError_None;
+    vr::VR_Init(&err, vr::VRApplication_Background);
+    if (err == vr::VRInitError_None) {
+        vr::VR_Shutdown();
+        vr::VR_Init(&err, vr::VRApplication_Overlay);
+    }
+    if (err != vr::VRInitError_None) {
+        std::fprintf(stderr, "ft-gazepanel: SteamVR: %s\n", vr::VR_GetVRInitErrorAsEnglishDescription(err));
+        return 1;
+    }
+    vr::IVROverlay *ov = vr::VROverlay();
+    vr::VROverlayHandle_t h = vr::k_ulOverlayHandleInvalid;
+    if (ov->CreateOverlay("frametop.gazepanel", "Frametop gaze calibration", &h) != vr::VROverlayError_None) {
+        std::fprintf(stderr, "ft-gazepanel: can't create the overlay (another copy running?)\n");
+        return 1;
+    }
+    ov->SetOverlaySortOrder(h, 250);  // in front of Frametop's screens and the pointer's dot
+    LoadFont();
+
+    Panel p;
+    bool visible = false, dirty = false;
+    auto place = [&] {
+        const double half = std::tan(p.wDeg * M_PI / 360);
+        vr::HmdMatrix34_t m{};
+        m.m[0][0] = m.m[1][1] = m.m[2][2] = 1;
+        m.m[2][3] = float(-distance);
+        ov->SetOverlayTransformTrackedDeviceRelative(h, vr::k_unTrackedDeviceIndex_Hmd, &m);
+        ov->SetOverlayWidthInMeters(h, float(2 * distance * half));
+    };
+    std::fprintf(stderr, "ft-gazepanel running: @%s, %.2f m\n", sockName.c_str(), distance);
+
+    Clock::time_point lastDraw{};
+    while (!g_stop) {
+        char buf[512];
+        sockaddr_un from{};
+        socklen_t fromLen = sizeof from;
+        ssize_t n;
+        while ((n = recvfrom(sock, buf, sizeof buf - 1, 0, reinterpret_cast<sockaddr *>(&from), &fromLen)) > 0) {
+            buf[n] = 0;
+            std::string reply = "ok";
+            char word[16] = "", state[16] = "";
+            double a = 0, b = 0, c = 0;
+            if (!std::strncmp(buf, "show ", 5)) {
+                p.full = !std::strcmp(buf + 5, "full");
+                p.w = p.full ? kFullW : kQuickPx;
+                p.h = p.full ? kFullH : kQuickPx;
+                p.wDeg = p.full ? kFullDeg : kQuickDeg;
+                p.title.clear(), p.text.clear(), p.dotOn = false, p.state = "off";
+                place();
+                ov->ShowOverlay(h);
+                visible = dirty = true;
+            } else if (!std::strcmp(buf, "hide")) {
+                ov->HideOverlay(h);
+                visible = false;
+            } else if (std::sscanf(buf, "bg %lf", &a) == 1) {
+                p.bg = std::clamp(a, 0.0, 1.0), dirty = true;
+            } else if (std::sscanf(buf, "dot %lf %lf %15s %lf", &a, &b, state, &c) >= 3) {
+                if (p.state != state || a != p.dotYaw || b != p.dotPitch) p.stateAt = Clock::now();
+                p.dotYaw = a, p.dotPitch = b, p.state = state, p.progress = c;
+                p.dotOn = std::strcmp(state, "off") != 0, dirty = true;
+            } else if (!std::strncmp(buf, "title", 5)) {
+                p.title = buf[5] == ' ' ? buf + 6 : "", dirty = true;
+            } else if (!std::strncmp(buf, "text", 4)) {
+                p.text = buf[4] == ' ' ? buf + 5 : "", dirty = true;
+            } else if (std::sscanf(buf, "%15s", word) == 1 && !std::strcmp(word, "ping")) {
+                reply = visible ? "ok shown" : "ok hidden";
+            } else {
+                reply = "error unknown command";
+            }
+            if (fromLen > offsetof(sockaddr_un, sun_path))
+                sendto(sock, reply.data(), reply.size(), 0, reinterpret_cast<sockaddr *>(&from), fromLen);
+            fromLen = sizeof from;
+        }
+        vr::VREvent_t ev;
+        while (vr::VRSystem()->PollNextEvent(&ev, sizeof ev))
+            if (ev.eventType == vr::VREvent_Quit) {
+                vr::VRSystem()->AcknowledgeQuit_Exiting();
+                g_stop = true;
+            }
+        const auto now = Clock::now();
+        if (visible && (dirty || (Animating(p) && now - lastDraw > std::chrono::milliseconds(40)))) {
+            Draw(p);
+            ov->SetOverlayRaw(h, p.px.data(), uint32_t(p.w), uint32_t(p.h), 4);
+            lastDraw = now;
+            dirty = false;
+        }
+        std::this_thread::sleep_for(std::chrono::milliseconds(visible ? 10 : 50));
+    }
+    ov->DestroyOverlay(h);
+    vr::VR_Shutdown();
+    return 0;
+}

input-settings/ft_input_settings.py

@@ -274,6 +274,7 @@ class Backend(QObject):
         self._gaze_prev = None  # the status before, for rates
         self._gaze_at = 0.0
         self._gaze_mode = None  # the helper's gaze mode: True, False, None (no answer)
+        self._check_asked = 0.0  # when quickcal or calibrate went to the gaze service (its errors)
         self._driver_block = ""  # set by _check_driver
         self._panels = None  # SteamVR's overlays, from the helper; None until it answers
         self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
@@ -337,6 +338,10 @@ class Backend(QObject):
             except BlockingIOError:
                 return
             text = data.decode(errors="replace")
+            if text.startswith("error ") and self._check_asked and time.monotonic() - self._check_asked < 3:
+                self._check_asked = 0.0  # the gaze service's answer to quickcal or calibrate
+                self.message.emit("Gaze check: " + text[6:], True)
+                continue
             if text in ("ok on", "ok off"):  # the helper's answer to "gaze ?" (from an unbound socket)
                 self._gaze_mode = text == "ok on"
                 self._gaze_at = time.monotonic()
@@ -958,6 +963,23 @@ class Backend(QObject):
         else:
             self.message.emit("The gaze service isn't running (frametop-gaze.service)", True)
 
+    @Slot()
+    def gazeQuickCheck(self):
+        """The gaze service's one-dot check, in the panel fixed to the headset."""
+        self._gaze_check("quickcal", "Quick check: look at the dot in front of you")
+
+    @Slot()
+    def gazeCalibrate(self):
+        """The full calibration in the panel fixed to the headset (gaze/gazecheck.py)."""
+        self._gaze_check("calibrate", "Calibration: look at each dot in the headset; right click or Meta+K stops")
+
+    def _gaze_check(self, command, done):
+        if self._send(command, GAZED):
+            self._check_asked = time.monotonic()
+            self.message.emit(done, False)
+        else:
+            self.message.emit("The gaze service isn't running (frametop-gaze.service)", True)
+
     @Slot()
     def openGazeProbe(self):
         """Calibrate in ft-gazeprobe (a GTK app on the host, fullscreen on a Frametop screen)."""

input-settings/main.qml

@@ -815,9 +815,23 @@ Kirigami.ApplicationWindow {
             property var status: backend.gazeStatus
             actions: [
                 Kirigami.Action {
-                    text: "Calibrate…"
+                    text: "Quick check"
                     icon.name: "crosshairs"
-                    tooltip: "Open the gaze probe to calibrate (fullscreen on a Frametop screen)"
+                    enabled: backend.gazeServiceRunning
+                    tooltip: "One dot in front of you in the headset: look at it, and the gaze tracker relearns where it sits"
+                    onTriggered: backend.gazeQuickCheck()
+                },
+                Kirigami.Action {
+                    text: "Calibrate"
+                    icon.name: "crosshairs"
+                    enabled: backend.gazeServiceRunning
+                    tooltip: "The full calibration in the headset: 21 dots in three rounds, dark to bright"
+                    onTriggered: backend.gazeCalibrate()
+                },
+                Kirigami.Action {
+                    text: "Gaze probe…"
+                    icon.name: "crosshairs"
+                    tooltip: "The lab tool: calibrate, test, and practise on a Frametop screen"
                     onTriggered: backend.openGazeProbe()
                 },
                 Kirigami.Action {

pointer/helper/ft-pointer.cpp

@@ -187,6 +187,10 @@
 // with no correction). Pressing gaze_right while gaze_left aims (Meta+K with Meta+J held)
 // presses the left button where the dot is now, so you can correct first and then drag; the
 // first key let go drops it. Without gaze mode they work from wherever the pointer is.
+//   The gaze calibration panel (gaze/panel/ft-gazepanel, run by the gaze service): while
+// ft-gazed says it's up ("calpanel 1", renewed every second; it lapses 3 s after the last),
+// the dot hides and a press answers the panel instead of clicking: a left click or gaze_left
+// sends "calaccept" to @ft_gazed (take this dot now), a right click or gaze_right "calquit".
 //   POINTER_ROLE (right, left, or stylus): the hand role our device takes while connected. A
 // Frame controller in your hand counts as used through its touch sensors and takes its hand's
 // role back, and then no click lands (see "no hand role" in the main loop): with a controller
@@ -533,6 +537,7 @@ private:
             if (rest.find("Thumbnail") != std::string::npos || rest.find("Subview") != std::string::npos) continue;
             if (key.rfind("system.pointer", 0) == 0 || key.rfind("system.cursor", 0) == 0 ||
                 key.rfind("frametop.pointer", 0) == 0 || key.rfind("frametop.guide", 0) == 0 ||
+                key == "frametop.gazepanel" ||  // the gaze calibration panel, fixed to the headset
                 key == "frametop.catcher" ||  // ft-screens' release catcher: only on a laser mid-drag
                 key == "system.HeadsetView" || key == "system.toast")
                 continue;
@@ -900,6 +905,10 @@ int main() {
     bool pressRight = false;
     std::string heldButton = "trigger";
     bool confirmLesson = false;  // a keyboard click's quick tap: a lesson with no correction (see the top)
+    // The gaze calibration panel is up until then ("calpanel 1"; see the top); calOpened: it just
+    // came up, so a press in progress ends without a click.
+    Clock::time_point calPanelUntil{};
+    bool calOpened = false;
     auto pressLeft = [&] {
         // ft-screens sends the keyboard to the panel clicked last; it sees clicks on
         // its own screens, but only we know when one lands on another panel.
@@ -1188,6 +1197,27 @@ int main() {
                 gz = {g[0], g[1], g[2], g[3], Clock::now()};
                 continue;
             }
+            // The gaze calibration panel (see the top): presses answer it instead of clicking.
+            {
+                int on;
+                if (std::sscanf(buf, "calpanel %d", &on) == 1) {
+                    const bool was = Clock::now() < calPanelUntil;
+                    calPanelUntil = on ? Clock::now() + std::chrono::seconds(3) : Clock::time_point{};
+                    if (on && !was) calOpened = true;
+                    continue;
+                }
+            }
+            if (Clock::now() < calPanelUntil) {
+                const bool accept = !std::strncmp(buf, "btn trigger 1", 13) || !std::strncmp(buf, "gazekey left 1", 14);
+                const bool quit = !std::strncmp(buf, "btn b 1", 7) || !std::strncmp(buf, "gazekey right 1", 15);
+                if (accept || quit) {
+                    SendTo(out, "ft_gazed", accept ? "calaccept" : "calquit");
+                    continue;
+                }
+                if (!std::strncmp(buf, "btn ", 4) || !std::strncmp(buf, "gazekey ", 8) ||
+                    !std::strncmp(buf, "precision ", 10) || !std::strncmp(buf, "gazedrag ", 9))
+                    continue;  // their releases, and the other buttons: nothing to click now
+            }
             {
                 char kind[16], source[16];
                 int v;
@@ -1641,6 +1671,12 @@ int main() {
             if (debug) std::fflush(stdout);
         }
         keyPresses.clear();
+        if (calOpened) {  // the calibration panel came up: a press in progress ends, no click
+            calOpened = false;
+            if (hold.src != Src::None) endHold(true);
+            if (leftHeld) releaseLeft();
+            aimHeld = aimHand = clickPress = false;
+        }
         if (hold.src == Src::Head) {
             double hy, hp;
             if (headAngles(hy, hp)) steer(hy, hp, hold.pressed ? 0.3 : headDeadzone, 1.0);
@@ -1951,6 +1987,7 @@ int main() {
                     auto secs = [&](Clock::time_point t) { return std::chrono::duration<double>(tnow - t).count(); };
                     alpha = std::clamp(1 - std::min(secs(lastMove) - gazeShow, secs(lastHeld)) / 0.25, 0.0, 1.0);
                 }
+                if (tnow < calPanelUntil) alpha = 0;
                 overlay->SetOverlayAlpha(marker, float(alpha));
                 overlay->SetOverlayWidthInMeters(marker, float(2 * SETTINGS_DOT * std::tan(cursorDeg * M_PI / 360)));
                 auto mm = Billboard(near, eye);
@@ -1980,6 +2017,7 @@ int main() {
                         alpha = std::max(alpha, std::clamp((0.6 - pulse) / 0.3, 0.0, 1.0));
                     }
                 }
+                if (tnow < calPanelUntil) alpha = 0;  // the calibration panel is up (see the top)
                 overlay->SetOverlayAlpha(show, float(alpha));
                 overlay->SetOverlayWidthInMeters(show, float(2 * dist * std::tan(scale * cursorDeg * M_PI / 360)));
                 auto m = Billboard(at, eye);

session/frametop.conf.example

@@ -34,6 +34,8 @@ POINTER_GAZE_HOLD=0.5      # gaze mode: a press held this long (s) without movin
 POINTER_GAZE_DOT=always    # gaze mode: the dot shows all the time (always) | only while the mouse moves it (moving)
 POINTER_GAZE_SHOW=1        # gaze mode, with POINTER_GAZE_DOT=moving: the dot shows this long (s) after the mouse moves it; also while a press is held, and a pulse per click
 POINTER_GAZE_MOUSE=precision # gaze mode: the left button holds back its press, the mouse steers, the release clicks (precision) | clicks right away (direct)
+POINTER_HEAD_DEADZONE=0.5  # keyboard clicks at the gaze (Meta+J, Meta+K): degrees the head turns before the held dot moves with it
+POINTER_KEY_TAP=0.25       # keyboard clicks: let go within this (s) and it clicks where the dot was at the press, and tells the gaze tracker it was right
 GAZE_TRACKER=steam         # gaze service: steam = SteamVR's eye tracker | own = our own (gaze/tracker: its frame grabber needs gaze/tracker/install.sh; calibrated in the gaze probe with Own tracker)
 GAZE_EYE=auto              # gaze service: eye bias. auto = each eye weighted by how far off it was at your recent nudges | left | right = that eye counts twice
 HANDS_SWAP_SIDES=0         # hand tracking (hands/run.sh install): 1 = the side cameras' names are swapped, which some SteamVR restarts cause (hands/tools/check_sides.py --ring tells)