Calibration all in the headset panel; the gaze probe is a development tool

Check headset fit now runs in the headset panel too ("fitcheck"): a card per
eye (tracked or lost, the tracker's signal, how much of the last 10 s it was
seen) and the hints, from the probe's fitcheck.py, updated at most twice a
second; left click or Meta+J runs its guided check, right click or Meta+K closes
it. So Quick check, Calibrate, and Check headset fit all happen in one place.

The gaze probe moves to the Gaze page's overflow menu as "Gaze probe
(development)", and its app menu entry says it's a development tool. Texts that
sent users to it ("use Calibrate… with Own tracker") point at Calibrate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/design.md

@@ -91,7 +91,7 @@ A few overlays need special handling:
 
 Head follow is experimental and off by default. It works, but it's only lightly tested, and the feel is mostly a matter of its settings; polishing it is left open. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction, where you were facing when your head last settled, and keeps its offset from it. The mouse can put the cursor anywhere up to `POINTER_FOLLOW_REACH` (70 degrees) from the reference, a corner of your view included. While your head stays within `POINTER_LEASH_DEG` of the reference, nothing moves on its own. Once your head has been past the leash for `POINTER_LEASH_DELAY` (0.2 s, so a glance out and back doesn't count), the reference eases to where you're facing (time constant `POINTER_LEASH_RETURN`, 0.2 s), never falling further behind than the leash, and the cursor ends up back where it was in your view. Then it waits for the leash again. Two earlier versions didn't work out. Moving the reference only while your head pulled at the end of the leash left it up to the leash off after you turned back, and getting it centred again meant overshooting with your head. Easing it toward your facing all the time moved the cursor on every small head movement. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping.
 
-Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. By default (`POINTER_GAZE_MOUSE_MOVE=held`, the Gaze page's Mouse movement switch) moving the mouse does nothing while the gaze has the pointer: it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so the lessons aren't polluted by mouse moves to somewhere else (they used to be kept out by an 8 degree limit, which also dropped real corrections when the tracker was further off). With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual; with `free`, moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: it needs accessibility (AT-SPI) on in the Frametop session, where it's off (no registry runs), plus app restarts, and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. The right button works the same way, with the right click on the release, and pressing it while the left press is held back starts a drag where the pointer is, like Meta+J then Meta+K. That drag lasts while either button (or key) is held, so a second right press, or a second Meta+K, is free to pan and tilt the panel being dragged; with the keyboard, the head turns it. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge before a click whose correction is within `POINTER_GAZE_NUDGE_MAX` (55 degrees, half of what the headset shows across) is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on, when our tracker thinks it moved, and when a correction is past that limit (the tracker is far off, so a click there isn't trusted as a lesson), and the full calibration runs in the same panel. The limit was 8 degrees, which dropped every correction while our tracker was 12 off. Its dots sit at known directions from the headset, so the panel needs no screen geometry, and each dot takes the gaze when it has held still rather than at a press: what the tracker says doesn't have to be close for the capture to work. See `gaze/README.md` for the service, the calibration, and what was measured.
+Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. By default (`POINTER_GAZE_MOUSE_MOVE=held`, the Gaze page's Mouse movement switch) moving the mouse does nothing while the gaze has the pointer: it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so the lessons aren't polluted by mouse moves to somewhere else (they used to be kept out by an 8 degree limit, which also dropped real corrections when the tracker was further off). With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual; with `free`, moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: it needs accessibility (AT-SPI) on in the Frametop session, where it's off (no registry runs), plus app restarts, and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. The right button works the same way, with the right click on the release, and pressing it while the left press is held back starts a drag where the pointer is, like Meta+J then Meta+K. That drag lasts while either button (or key) is held, so a second right press, or a second Meta+K, is free to pan and tilt the panel being dragged; with the keyboard, the head turns it. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge before a click whose correction is within `POINTER_GAZE_NUDGE_MAX` (55 degrees, half of what the headset shows across) is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on, when our tracker thinks it moved, and when a correction is past that limit (the tracker is far off, so a click there isn't trusted as a lesson), and the full calibration and the headset fit check run in the same panel, so everything a user does to calibrate happens in one place in the headset; the gaze probe, a fullscreen GTK app, is the development tool. The limit was 8 degrees, which dropped every correction while our tracker was 12 off. Its dots sit at known directions from the headset, so the panel needs no screen geometry, and each dot takes the gaze when it has held still rather than at a press: what the tracker says doesn't have to be close for the capture to work. See `gaze/README.md` for the service, the calibration, and what was measured.
 
 Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place.
 

docs/reference.md

@@ -116,7 +116,7 @@ A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs
 - Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput, like frame-voice's, don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected.
 - Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button.
 - Ignored panels lists the SteamVR overlays that are showing, grouped by app (the first two parts of the overlay key, such as `sasaken.frame-perf-overlay`), from the pointer helper (`overlays`). Tick a panel, or Ignore the whole app, and the pointer passes through it to what's behind. It's for panels you only look at, like a performance overlay that follows your view. The list is saved as `POINTER_IGNORE` in `~/.config/frametop.conf`: comma-separated overlay keys, where a shell pattern like `vendor.app*` covers a whole app, including panels it opens later. The helper reloads at once. Frametop's own screens aren't listed, and entries for apps that aren't open are listed below, to remove.
-- Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), the gaze mode sliders, the gaze service's state (headset, samples per second, how often the tracker is losing each eye, the calibration, the nudges learned), and Calibrate (opens the gaze probe), Check headset fit (opens the probe's Headset fit mode), Reload calibration, and Forget nudges.
+- Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), the gaze mode sliders, the gaze service's state (headset, samples per second, how often the tracker is losing each eye, the calibration, the nudges learned), and Quick check, Calibrate, and Check headset fit (each in a panel in the headset), Reload calibration, and Forget nudges. The gaze probe, a development tool, is in the page's overflow menu.
 - Bluetooth lists paired devices and has Apply Bluetooth fixes, which runs `/etc/steamframe/bt-fixups.sh` through `pkexec`. Pair new devices in Steam.
 
 Device rules are saved in `~/.config/frametop-input.json`. `input-settings/install.sh` installs the menu entry. Its launcher hands podman the real `XDG_RUNTIME_DIR` and user bus and gives the app the session's Wayland socket, because the desktop session runs on a private D-Bus and podman fails on it.

gaze/README.md

@@ -5,7 +5,7 @@ The Steam Frame's eye tracking as pointer input: a gaze mode for the 3D mouse (t
 - `ft-gaze` (C++, OpenVR, runs in the dev container) reads the eye tracker and prints one JSON line per sample (90 Hz). For each source, it gives the gaze direction relative to the head and the Frametop screen pixel it lands on.
 - `gazecal.py` has what the probe and the gaze service share: the correction models, filters, and the reader for SteamVR's eye tracking log.
 - `tracker/` is our own eye tracker, an alternative to SteamVR's: `ft-eyes` finds the pupils and glints in the eye-camera frames that `ft-eyegrab` (a small root service) copies out of SteamVR's tracker. See "Our own eye tracker" below.
-- `probe/ft-gazeprobe` (GTK 4, host Python) is a fullscreen playground. It runs ft-gaze, draws where you're looking, measures accuracy, and tries out hold-to-adjust clicking with a calibration that learns from your adjustments.
+- `probe/ft-gazeprobe` (GTK 4, host Python) is a fullscreen playground, for developing the gaze tracking: day to day, the calibration and the checks run in the headset panel (Quick check, Calibrate, and Check headset fit on the Gaze page). It runs ft-gaze, draws where you're looking, measures accuracy, and tries out hold-to-adjust clicking with a calibration that learns from your adjustments.
 
 ```
 gaze/build.sh                 # build ft-gaze
@@ -64,12 +64,14 @@ Ground rules, for anyone changing it:
 
 ## Headset fit
 
-The probe's Headset fit mode (Check headset fit on the Gaze page, or `ft-gazeprobe --mode fit`) shows, for each eye, whether the tracker has it, how open it is, and the tracker's confidence in it, and a map of where you looked coloured by how often it lost that eye there. Hints under the maps say which eye gets lost where, and what to try. Enter runs a guided check: dots around the screen, then looking down at the keyboard, up, left and right. R starts over. Adjust the headset while you watch it.
+Check headset fit on the Gaze page opens it in the headset panel: a card per eye (tracked or lost, the tracker's signal, how much of the last 10 s it was seen) and the hints, live while you adjust the headset. A left click or Meta+J runs the guided check (dots, then looks down, up, left and right), and a right click or Meta+K closes it. The probe's Headset fit mode (`ft-gazeprobe --mode fit`, for development) has the same check with maps: it shows, for each eye, whether the tracker has it, how open it is, and the tracker's confidence in it, and a map of where you looked coloured by how often it lost that eye there. Hints under the maps say which eye gets lost where, and what to try. Enter runs a guided check: dots around the screen, then looking down at the keyboard, up, left and right. R starts over. Adjust the headset while you watch it.
 
 Losing an eye is usually about where you look, not the tracker. On this Frame the left eye was lost 57 to 64 % of the time looking 30 to 50 degrees down (at the keyboard) and the right eye never; at screen height both were seen over 98 % of the time. Looking down, the lids come down over the eyes. That's harmless, since the gaze service ignores looks down past the screens: they show on the maps, but not in the counts or as a problem.
 
 ## Probe
 
+The probe is a development tool (in the Gaze page's overflow menu): calibration experiments, accuracy tests, and practice modes. Users calibrate and check in the headset panel instead.
+
 The trigger is Enter, Space, or a mouse button. Right-click anywhere in the window (or press the Menu key or Shift+F10) for a menu with Run calibration, Start accuracy test, Calibrate from last test, Reset calibration, the modes, the panel, fullscreen, and Quit. The buttons at the top right show and hide the panel, leave fullscreen, and quit. The arrow in the panel's title bar collapses it to just that bar, so the dot and targets behind it stay visible; the collapsed bar stays through tests. The keys do the same (Tab, C, F11, Esc), but only after you click the window once, since Frametop sends typing to the panel you clicked last. If ft-gaze stops, the probe starts it again after 3 s and shows why it stopped. Windowed mode stays on the screen it was on, and a small KWin script tells the probe where the window is, so the dot and targets are still in the right place.
 
 - **Run calibration (start here):** the initial calibration, modeled on Apple Vision Pro's eye setup. Face the centre and keep your head still. Look at one dot and press the trigger, then at each of six dots in a circle. That happens in three rounds, and the screen goes dark, then medium, then bright, because pupil size changes with brightness and the tracker's error with it. Each round turns the ring 20 degrees, and the middle round's ring is half the size, so the 21 dots cover the middle, halfway out, and the edge of your view. The ring's size is `Calibration ring` (degrees, 20 by default, less if the window is too small). Error grows toward the edge, and the calibration can only correct as far out as it has seen dots. The current dot is a bright pulsing dot with a point in the middle; finished dots fade to specks, so your eyes don't go back to them. Samples from blinks and from moments when the tracker lost an eye are dropped: openness under half of what it was during that look (not a fixed level, because your lids come down when you look down, and you squint in the bright round), or the angle between the eyes jumping more than 1.5 degrees from its median (that angle depends on how far away you're looking, so only a jump counts). Each dot is measured with medians, so one bad sample can't fail it. A look that lands where the gaze was for another dot of the round is refused as a look at the wrong dot. Mouse clicks don't count during a calibration run or a test: use Enter or Space. If a dot still fails, the message says why and the next try listens longer. After two failures, S (or the menu) skips the dot. Every attempt is logged to `calibration-attempts.jsonl`. At the end it fits every source's calibration from all the dots, replacing what it had learned (quadratic if the model was none). Esc cancels. The run is saved as `calibration-*.json`. With "Test after calibration" on (the default), the accuracy test starts right after, on new spots.

gaze/fitcheck.py

@@ -1,5 +1,5 @@
-"""fitcheck: how well the eye tracker sees each eye, for fitting the headset (ft-gazeprobe's
-Headset fit mode).
+"""fitcheck: how well the eye tracker sees each eye, for fitting the headset (the headset
+panel's fit check, gazecheck.py, and ft-gazeprobe's Headset fit mode).
 
 From each ft-gaze sample it takes, per eye, whether the tracker has that eye (its variance
 for the eye's direction, "unc", under EYE_LOST; see gazecal), how open the eye is, and the

gaze/ft-gazed

@@ -81,6 +81,7 @@ Control socket: abstract unix datagram "@ft_gazed":
                                    whatever GAZE_TRACKER says: the probe's lease. Reply: "ok"
   quickcal                         the one-dot check now (the calibration if there's none)
   calibrate                        the full calibration in the panel
+  fitcheck                         the headset fit check in the panel
   calaccept | calquit              from the pointer helper while the panel is up: take this dot
                                    now (a left click, Meta+J) | close it (a right click, Meta+K)
 
@@ -673,7 +674,7 @@ class Service:
             elif words[:1] == ["forget"]:
                 self.forget_lessons()
                 reply = "ok"
-            elif words[:1] and words[0] in ("quickcal", "calibrate", "calaccept", "calquit", "recheck"):
+            elif words[:1] and words[0] in ("quickcal", "calibrate", "fitcheck", "calaccept", "calquit", "recheck"):
                 reply = self.checks.command(words)
             elif words[:1] == ["eyes"] and len(words) == 2:
                 try:

gaze/gazecheck.py

@@ -21,6 +21,12 @@ directions: look at each one.
           "calibrate". Frametop's screens hide while it runs. Quitting it while there's still
           no calibration turns gaze mode off (POINTER_GAZE=0); turning it on again reopens it.
 
+  fit     the headset fit check (on "fitcheck", Check headset fit on the Gaze page): live, a
+          card per eye (tracked or lost, the tracker's signal, how much of the last 10 s it was
+          seen) and hints, from the gaze probe's Headset fit (gaze/fitcheck.py), while you
+          adjust the headset. A left click or Meta+J runs its guided check (dots, then looks
+          down, up, left and right); a right click or Meta+K closes it, as does FIT_TIMEOUT.
+
 A dot captures itself: from CHECK_SETTLE after it shows (the eyes getting there), once the gaze
 has held within CHECK_SPREAD for CHECK_WINDOW (the probe's max spread and capture time). It's
 the gaze holding still that counts, not where the tracker puts it, so it works however far off
@@ -49,6 +55,7 @@ import sys
 import time
 from pathlib import Path
 
+from fitcheck import MIN_REGION, FitCheck, wrap
 from gazecal import DEFAULT_MODEL, EYE_LOST, STATE, steady_samples
 
 REPO = Path(__file__).resolve().parents[1]
@@ -80,6 +87,9 @@ ROUND_BG = (0.03, 0.33, 0.8)
 ROUND_NAMES = ("dark", "medium", "bright")
 RING_SCALE = (1.0, 0.5, 1.0)
 PANEL_RETRY = 10.0
+FIT_TIMEOUT = 300.0    # seconds the fit check stays up
+FIT_EVERY = 0.5        # seconds between its cards' updates (each is a new picture for the panel)
+FIT_HINT_WIDTH = 95    # characters a hint line holds in the panel
 
 
 def log(msg):
@@ -156,6 +166,7 @@ class Checks:
         self.gaze_heard = 0.0
         self.want_full_until = 0.0  # gaze mode came on before the tracker said whether it's calibrated
         self.last_quick = 0.0
+        self.sample_at = 0.0     # the tracker last sent anything
         self.seen_at = 0.0       # eyes last seen (SteamVR's tracker's variance for them, "unc")
         self.away = True         # no eyes for AWAY_MIN: their coming back is the headset going on
         self.back_since = None
@@ -291,6 +302,8 @@ class Checks:
             return "error a check is running"
         if not self.panel_proc:
             return "error the panel isn't running (gaze/build.sh builds it)"
+        if kind == "fit":
+            return self.start_fit(reason)
         if not self.can_run():
             return "error the headset is off or the tracker isn't sending"
         own = svc.kind == "own"
@@ -316,6 +329,62 @@ class Checks:
             self.last_quick = now
         return "ok"
 
+    def start_fit(self, reason):
+        # Eyes lost are what it's there to show, so it needs only the tracker sending.
+        if time.monotonic() - self.sample_at > 2:
+            return "error the headset is off or the eye tracker isn't sending"
+        now = time.monotonic()
+        self.check = {"kind": "fit", "reason": reason, "own": False, "dots": [], "i": 0, "started": now, "shown": now,
+                      "run": [], "accept": False, "done_at": None, "tries": 0, "skipped": 0, "captured": 0,
+                      "points": {}, "fit": FitCheck(), "drawn": {}, "drawn_at": 0.0, "step": None}
+        log(f"fit check: {reason}")
+        self.to_helper("calpanel 1")
+        self.to_panel("show fit")
+        self.to_panel("title Headset fit: adjust the headset while you watch")
+        self.to_panel("text Left click or Meta+J: guided check · Right click or Meta+K: done")
+        return "ok"
+
+    def fit_tick(self, now):
+        c = self.check
+        fit = c["fit"]
+        # The guided check: its dots at head-relative directions in the panel (the probe's
+        # screen fractions, spread over the middle of the panel), and its looks as the title.
+        step = fit.guide_step(now)
+        key = None if step is None else (step[0], step[1])
+        if key != c["step"]:
+            c["step"] = key
+            if step is None:
+                self.to_panel("dot 0 0 off")
+                self.to_panel("title Headset fit: adjust the headset while you watch")
+            elif step[0] == "dot":
+                fx, fy = step[1]
+                self.to_panel(f"dot {(0.5 - fx) * 32:.2f} {(0.5 - fy) * 24:.2f} look")
+                self.to_panel("title Look at the dot")
+            else:
+                self.to_panel("dot 0 0 off")
+                self.to_panel(f"title {step[1]}")
+        if now - c["drawn_at"] < FIT_EVERY:
+            return
+        c["drawn_at"] = now
+        drawn = c["drawn"]
+        for k in (0, 1):
+            word, (r, g, b) = fit.status(k)
+            sig, seen = fit.signal(k), fit.tracked_share(k, now)
+            cmd = (f"eye {k} {r:.2f} {g:.2f} {b:.2f} {-1 if sig is None else round(sig, 1):g} "
+                   f"{-1 if seen is None else round(seen * 20) / 20:g} {word.capitalize()}")
+            if drawn.get(k) != cmd:
+                drawn[k] = cmd
+                self.to_panel(cmd)
+        if fit.have_eye_data and fit.samples < 3 * MIN_REGION:
+            hints = ["Look around slowly: up, down, left and right. Or left click (Meta+J) for a guided check."]
+        else:
+            hints = fit.hints()
+        lines = [line for h in hints for line in wrap(h, FIT_HINT_WIDTH)][:8]
+        cmd = "hints " + "|".join(lines)
+        if drawn.get("hints") != cmd:
+            drawn["hints"] = cmd
+            self.to_panel(cmd)
+
     def show_dot(self):
         c = self.check
         yaw, pitch, rnd = c["dots"][c["i"]]
@@ -333,12 +402,16 @@ class Checks:
         c["run"], c["accept"], c["done_at"] = [], False, None
 
     def on_sample(self, s):
+        self.sample_at = time.monotonic()
         unc = (s["src"].get("mmap1") or {}).get("unc")
         if unc and min(unc) <= EYE_LOST:
             self.seen_at = time.monotonic()
             if self.away and self.back_since is None:
                 self.back_since = self.seen_at
         c = self.check
+        if c and c["kind"] == "fit":
+            c["fit"].feed(s, self.sample_at)
+            return
         if not c or c["done_at"]:
             return
         if c["own"]:
@@ -560,8 +633,12 @@ class Checks:
             return self.start("full" if self.calibrated() is False else "quick", "asked for")
         if cmd == "calibrate":
             return self.start("full", "asked for")
+        if cmd == "fitcheck":
+            return self.start("fit", "asked for")
         if cmd == "calaccept":
-            if self.check:
+            if self.check and self.check["kind"] == "fit":
+                self.check["fit"].toggle_guide(time.monotonic())
+            elif self.check:
                 self.check["accept"] = True
             return "ok"
         if cmd == "calquit":
@@ -590,6 +667,13 @@ class Checks:
         if not c:
             return
         now = time.monotonic()
+        if c["kind"] == "fit":
+            # Not closed when the eyes go: adjusting the headset loses them.
+            if now - c["started"] > FIT_TIMEOUT:
+                self.close("timed out")
+            else:
+                self.fit_tick(now)
+            return
         if c["done_at"] and now >= c["done_at"]:
             if c.get("closing"):
                 self.close()

gaze/panel/ft-gazepanel.cpp

@@ -8,11 +8,13 @@
 // The panel sits POINTER-like at --distance (1.5 m, about where Frametop's screens are, so
 // the eyes converge as they do in use). "quick" is a small square, QUICK_DEG across, for the
 // one-dot check; "full" is FULL_DEG across (4:3), with a solid background whose brightness the
-// service sets per round (pupil size changes with it, and the tracker's error with it).
+// service sets per round (pupil size changes with it, and the tracker's error with it); "fit"
+// is FIT_DEG across (4:3), see-through like quick, for the headset fit check: a card per eye
+// (tracked or lost, the tracker's signal, how much of the last 10 s it was seen) and hints.
 //
 // Control socket: abstract unix datagram "@ft_gazepanel" (--socket NAME); a sender with an
 // address gets "ok" or "error ...":
-//   show quick|full              the panel, empty, in front of you
+//   show quick|full|fit          the panel, empty, in front of you
 //   hide
 //   bg <0..1>                    the background's brightness (full)
 //   dot <yaw> <pitch> <state> [<progress 0..1>]
@@ -20,6 +22,10 @@
 //                                look, capture (a ring filling to progress), done,
 //                                fail, off
 //   title <text> / text <text>   a line at the top / at the bottom (empty to clear)
+//   eye <0|1> <r> <g> <b> <signal 0..1|-1> <seen 0..1|-1> <word>
+//                                fit: an eye's card (0 left): its state in that colour, the
+//                                tracker's signal, the share of the last 10 s it was seen
+//   hints <line>|<line>|...      fit: lines under the cards (empty to clear)
 //   ping
 //
 // Options: --watch-stdin (quit when stdin closes: the service runs it), --socket NAME,
@@ -51,7 +57,8 @@ namespace {
 using Clock = std::chrono::steady_clock;
 constexpr double kQuickDeg = 16;      // QUICK_DEG: the one-dot check's square
 constexpr double kFullDeg = 64;       // FULL_DEG: the full calibration's width (4:3)
-constexpr int kQuickPx = 320, kFullW = 1024, kFullH = 768;
+constexpr double kFitDeg = 40;        // FIT_DEG: the headset fit check's width (4:3)
+constexpr int kQuickPx = 320, kFullW = 1024, kFullH = 768, kFitW = 800, kFitH = 600;
 std::atomic<bool> g_stop{false};
 
 // ---------------------------------------------------------------- text (as screens/keyboard.cpp)
@@ -112,8 +119,15 @@ std::vector<uint32_t> Codepoints(const std::string &s) {
 
 // ---------------------------------------------------------------- the picture
 
+struct EyeCard {
+    std::string word = "no data";
+    double r = 0.6, g = 0.6, b = 0.6, signal = -1, seen = -1;
+};
+
 struct Panel {
-    bool full = false;
+    bool full = false, fit = false;
+    EyeCard eyes[2];
+    std::vector<std::string> hints;
     int w = kQuickPx, h = kQuickPx;
     double wDeg = kQuickDeg;  // across
     std::vector<uint8_t> px;
@@ -153,7 +167,13 @@ void Disc(Panel &p, double cx, double cy, double outer, double inner, double r,
         }
 }
 
-void Text(Panel &p, const std::string &s, int size, int cx, int cy, double lum) {
+void Rect(Panel &p, int x0, int y0, int x1, int y1, double r, double g, double b, double a) {
+    for (int y = std::max(y0, 0); y < std::min(y1, p.h); ++y)
+        for (int x = std::max(x0, 0); x < std::min(x1, p.w); ++x) Blend(p, x, y, r, g, b, a);
+}
+
+// Text centred on (x, cy), or starting at x (left).
+void Text(Panel &p, const std::string &s, int size, int x, int cy, double r, double g, double b, bool left = false) {
     if (!g_fontOk || s.empty()) return;
     const auto cps = Codepoints(s);
     int width = 0;
@@ -161,14 +181,48 @@ void Text(Panel &p, const std::string &s, int size, int cx, int cy, double lum)
     int ascent, descent, gap;
     stbtt_GetFontVMetrics(&g_font, &ascent, &descent, &gap);
     const float scale = stbtt_ScaleForPixelHeight(&g_font, float(size));
-    int x = cx - width / 2;
+    if (!left) x -= width / 2;
     const int baseline = cy + int(std::lround((ascent + descent) * scale / 2));
     for (uint32_t cp : cps) {
-        const Glyph &g = GetGlyph(cp, size);
-        for (int gy = 0; gy < g.h; ++gy)
-            for (int gx = 0; gx < g.w; ++gx)
-                Blend(p, x + g.xoff + gx, baseline + g.yoff + gy, lum, lum, lum, g.bitmap[size_t(gy) * g.w + gx] / 255.0);
-        x += g.advance;
+        const Glyph &gl = GetGlyph(cp, size);
+        for (int gy = 0; gy < gl.h; ++gy)
+            for (int gx = 0; gx < gl.w; ++gx)
+                Blend(p, x + gl.xoff + gx, baseline + gl.yoff + gy, r, g, b, gl.bitmap[size_t(gy) * gl.w + gx] / 255.0);
+        x += gl.advance;
+    }
+}
+
+void Text(Panel &p, const std::string &s, int size, int x, int cy, double lum, bool left = false) {
+    Text(p, s, size, x, cy, lum, lum, lum, left);
+}
+
+// The headset fit check (see the top): a card per eye, then the hints.
+void DrawFit(Panel &p, double pxPerDeg, int textSize, double faint) {
+    const int margin = int(p.w * 0.06), cw = int(p.w * 0.41), ch = int(p.h * 0.32), top = int(p.h * 0.12);
+    const int pad = int(pxPerDeg * 0.9), big = int(pxPerDeg * 1.6), small = int(pxPerDeg * 0.8);
+    for (int k = 0; k < 2; ++k) {
+        const EyeCard &e = p.eyes[k];
+        const int x0 = k == 0 ? margin : p.w - margin - cw;
+        Rect(p, x0, top, x0 + cw, top + ch, 1, 1, 1, 0.07);
+        Text(p, k ? "Right eye" : "Left eye", textSize, x0 + pad, top + pad + textSize / 2, faint, true);
+        Text(p, e.word, big, x0 + pad, top + int(ch * 0.40), e.r, e.g, e.b, true);
+        // The tracker's signal: a bar, red to green.
+        const int by = top + int(ch * 0.62), bh = std::max(4, int(pxPerDeg * 0.35)), bw = cw - 2 * pad;
+        Text(p, "Signal", small, x0 + pad, by - small, faint, true);
+        Rect(p, x0 + pad, by, x0 + pad + bw, by + bh, 1, 1, 1, 0.15);
+        if (e.signal >= 0) {
+            const double v = std::clamp(e.signal, 0.0, 1.0);
+            const double r = v < 0.5 ? 1.0 : 1.0 - 1.3 * (v - 0.5), g = v < 0.5 ? 0.3 + 0.9 * v : 0.75 + 0.5 * (v - 0.5);
+            Rect(p, x0 + pad, by, x0 + pad + int(bw * v), by + bh, r, g, 0.35, 0.95);
+        }
+        char seen[64] = "Seen: not yet";
+        if (e.seen >= 0) std::snprintf(seen, sizeof seen, "Seen %d%% of the last 10 s", int(std::lround(e.seen * 100)));
+        Text(p, seen, small, x0 + pad, top + ch - pad, faint, true);
+    }
+    int y = top + ch + pad * 2;
+    for (const std::string &line : p.hints) {
+        Text(p, line, textSize, margin, y, faint, true);
+        y += int(textSize * 1.4);
     }
 }
 
@@ -188,8 +242,8 @@ void Draw(Panel &p) {
         for (size_t i = 0; i < p.px.size(); i += 4)
             p.px[i] = p.px[i + 1] = p.px[i + 2] = uint8_t(std::lround(p.bg * 255)), p.px[i + 3] = 255;
     } else {
-        // The quick check: a dim rounded square, see-through, so it's clear of what's behind.
-        const double r = p.w * 0.12;
+        // The quick check and the fit check: a dim rounded panel, see-through, so it's clear of what's behind.
+        const double r = std::min(p.w, p.h) * 0.12;
         for (int y = 0; y < p.h; ++y)
             for (int x = 0; x < p.w; ++x) {
                 const double dx = std::max({r - x - 0.5, x + 0.5 - (p.w - r), 0.0});
@@ -203,6 +257,7 @@ void Draw(Panel &p) {
     const int titleSize = int(pxPerDeg * (p.full ? 1.5 : 1.1)), textSize = int(pxPerDeg * (p.full ? 1.2 : 0.9));
     Text(p, p.title, titleSize, p.w / 2, int(titleSize * 1.2), faint);
     Text(p, p.text, textSize, p.w / 2, p.h - int(textSize * 1.3), faint);
+    if (p.fit) DrawFit(p, pxPerDeg, textSize, faint);
     if (!p.dotOn || p.state == "off") return;
     double x, y;
     ToPixel(p, p.dotYaw, p.dotPitch, x, y);
@@ -302,13 +357,16 @@ int main(int argc, char **argv) {
             buf[n] = 0;
             std::string reply = "ok";
             char word[16] = "", state[16] = "";
-            double a = 0, b = 0, c = 0;
+            double a = 0, b = 0, c = 0, d = 0, e = 0;
+            int rest = 0;
             if (!std::strncmp(buf, "show ", 5)) {
                 p.full = !std::strcmp(buf + 5, "full");
-                p.w = p.full ? kFullW : kQuickPx;
-                p.h = p.full ? kFullH : kQuickPx;
-                p.wDeg = p.full ? kFullDeg : kQuickDeg;
+                p.fit = !std::strcmp(buf + 5, "fit");
+                p.w = p.full ? kFullW : p.fit ? kFitW : kQuickPx;
+                p.h = p.full ? kFullH : p.fit ? kFitH : kQuickPx;
+                p.wDeg = p.full ? kFullDeg : p.fit ? kFitDeg : kQuickDeg;
                 p.title.clear(), p.text.clear(), p.dotOn = false, p.state = "off";
+                p.eyes[0] = p.eyes[1] = EyeCard{}, p.hints.clear();
                 place();
                 ov->ShowOverlay(h);
                 visible = dirty = true;
@@ -320,6 +378,18 @@ int main(int argc, char **argv) {
             } else if (std::sscanf(buf, "dot %lf %lf %15s %lf", &a, &b, state, &c) >= 3) {
                 p.dotYaw = a, p.dotPitch = b, p.state = state, p.progress = c;
                 p.dotOn = std::strcmp(state, "off") != 0, dirty = true;
+            } else if (int k; std::sscanf(buf, "eye %d %lf %lf %lf %lf %lf %n", &k, &a, &b, &c, &d, &e, &rest) >= 6 &&
+                       rest > 0 && (k == 0 || k == 1)) {
+                p.eyes[k] = EyeCard{buf + rest, a, b, c, d, e}, dirty = true;
+            } else if (!std::strncmp(buf, "hints", 5)) {
+                p.hints.clear();
+                std::string s = buf[5] == ' ' ? buf + 6 : "";
+                for (size_t at = 0; !s.empty() && at <= s.size();) {
+                    const size_t bar = std::min(s.find('|', at), s.size());
+                    p.hints.push_back(s.substr(at, bar - at));
+                    at = bar + 1;
+                }
+                dirty = true;
             } else if (!std::strncmp(buf, "title", 5)) {
                 p.title = buf[5] == ' ' ? buf + 6 : "", dirty = true;
             } else if (!std::strncmp(buf, "text", 4)) {

gaze/probe/ft-gazeprobe.desktop

@@ -1,9 +1,9 @@
 [Desktop Entry]
 Type=Application
-Name=Frametop Gaze Probe
-GenericName=Eye tracking playground
-Comment=How accurate the headset's eye tracking is on your screens, and gaze clicking with calibration
+Name=Frametop Gaze Probe (development)
+GenericName=Eye tracking development tool
+Comment=For developing Frametop's gaze tracking. Day to day, the calibration and checks are on the Gaze page of Frametop Input Settings
 Exec=@REPO@/gaze/probe/ft-gazeprobe
 Icon=view-visible
-Categories=Utility;Development;
+Categories=Development;
 Keywords=eye;gaze;tracking;calibration;pointer;steamvr;frametop;

input-settings/ft_input_settings.py

@@ -993,14 +993,15 @@ class Backend(QObject):
             self.message.emit("The gaze service isn't running (frametop-gaze.service)", True)
 
     @Slot()
-    def openGazeProbe(self):
-        """Calibrate in ft-gazeprobe (a GTK app on the host, fullscreen on a Frametop screen)."""
-        self._open_probe([], "Opening the gaze probe: calibrate there, then close it")
+    def gazeFitCheck(self):
+        """The headset fit check, in the panel fixed to the headset (gaze/gazecheck.py)."""
+        self._gaze_check("fitcheck", "Headset fit: in the headset, adjust it while you watch; right click or Meta+K closes it")
 
     @Slot()
-    def openHeadsetFit(self):
-        """The probe's Headset fit mode: how well the tracker sees each eye, as you adjust."""
-        self._open_probe(["--mode", "fit"], "Opening the headset fit check in the gaze probe")
+    def openGazeProbe(self):
+        """ft-gazeprobe, the gaze tracking's development tool (a GTK app on the host, fullscreen on
+        a Frametop screen). Day to day, the checks and the calibration run in the headset panel."""
+        self._open_probe([], "Opening the gaze probe (a development tool)")
 
     def _open_probe(self, args, done):
         runner = ["distrobox-host-exec"] if shutil.which("distrobox-host-exec") else []

input-settings/main.qml

@@ -825,20 +825,15 @@ Kirigami.ApplicationWindow {
                     text: "Calibrate"
                     icon.name: "crosshairs"
                     enabled: backend.gazeServiceRunning
-                    tooltip: "The full calibration in the headset: 21 dots in three rounds, dark to bright"
+                    tooltip: "The full calibration in the headset: dots in three rounds, dark to bright"
                     onTriggered: backend.gazeCalibrate()
                 },
                 Kirigami.Action {
-                    text: "Gaze probe…"
-                    icon.name: "crosshairs"
-                    tooltip: "The lab tool: calibrate, test, and practise on a Frametop screen"
-                    onTriggered: backend.openGazeProbe()
-                },
-                Kirigami.Action {
-                    text: "Check headset fit…"
+                    text: "Check headset fit"
                     icon.name: "view-visible"
-                    tooltip: "How well the eye tracker sees each eye, and where it loses one, while you adjust the headset"
-                    onTriggered: backend.openHeadsetFit()
+                    enabled: backend.gazeServiceRunning
+                    tooltip: "In the headset: how well the eye tracker sees each eye, live, while you adjust it"
+                    onTriggered: backend.gazeFitCheck()
                 },
                 Kirigami.Action {
                     text: "Reload calibration"
@@ -852,6 +847,14 @@ Kirigami.ApplicationWindow {
                     enabled: backend.gazeServiceRunning
                     tooltip: "Drop what your mouse nudges taught; the calibration stays"
                     onTriggered: backend.forgetGazeLessons()
+                },
+                Kirigami.Action {
+                    // A development tool: the checks and the calibration run in the headset panel.
+                    text: "Gaze probe (development)…"
+                    icon.name: "tools"
+                    displayHint: Kirigami.DisplayHint.AlwaysHide
+                    tooltip: "The gaze tracking's lab tool, for developing it: experiments, accuracy tests, practice"
+                    onTriggered: backend.openGazeProbe()
                 }
             ]
 
@@ -943,7 +946,7 @@ Kirigami.ApplicationWindow {
                              && (!gpage.status.own_running || gpage.status.own_reseat || !gpage.status.calibration_samples)
                     text: !gpage.status.eyegrab ? "Needs its frame grabber: run gaze/tracker/install.sh (asks for sudo)"
                           : !gpage.status.own_running ? "Starting…"
-                          : !gpage.status.calibration_samples ? "Not calibrated: use Calibrate… with Own tracker"
+                          : !gpage.status.calibration_samples ? "Not calibrated: use Calibrate"
                           : "The headset was off: your first nudge and click resets where it sits"
                     color: gpage.status.own_running ? Kirigami.Theme.neutralTextColor : Kirigami.Theme.negativeTextColor
                     font: Kirigami.Theme.smallFont
@@ -1031,7 +1034,7 @@ Kirigami.ApplicationWindow {
                             + (gpage.status.one_eye_share > 0.5 ? " · only one eye tracked" : "")
                     color: gpage.status.one_eye_share > 0.5 ? Kirigami.Theme.neutralTextColor : Kirigami.Theme.textColor
                     Controls.ToolTip.text: "Only one eye tracked: the gaze comes from the other eye, a little less "
-                                           + "precisely. Check headset fit… shows where the tracker loses it."
+                                           + "precisely. Check headset fit shows where the tracker loses it."
                     Controls.ToolTip.visible: gpage.status.one_eye_share > 0.5 && ghover.hovered
                     HoverHandler { id: ghover }
                 }
@@ -1055,7 +1058,7 @@ Kirigami.ApplicationWindow {
                           ? gpage.status.calibration_samples + " points ("
                             + (gpage.status.tracker === "own" ? "Own tracker, " + gpage.status.calibration_made
                                : gpage.status.kind === "eyes" ? gpage.status.model + ", each eye" : gpage.status.model) + ")"
-                          : "none yet: use Calibrate…"
+                          : "none yet: use Calibrate"
                 }
                 Controls.Label {
                     visible: backend.gazeServiceRunning
@@ -1075,7 +1078,7 @@ Kirigami.ApplicationWindow {
                       + "to drag something instead. Look away to hand back: how far from the pointer you look before the "
                       + "gaze takes it back from the mouse. Largest correction to learn: a click corrected further than "
                       + "this isn't learned; the tracker is that far off, so the quick check runs instead. Eye tracker: SteamVR's, or our own "
-                      + "(gaze/tracker), which keeps its own calibration (Calibrate… with Own tracker). Eye bias: the gaze "
+                      + "(gaze/tracker), which keeps its own calibration. Eye bias: the gaze "
                       + "combines both eyes, since their errors partly cancel; Left or Right counts that eye twice as "
                       + "much, and Auto weights each by how far off it was at your recent nudges."
             }