Keep the gaze going when the tracker loses one eye

Live, the tracker had lost the left eye (openness 0, set 2's eyes 9 to 14
degrees apart) and ft-gazed dropped 96% of samples as blinks. A blink is now
both eyes closing, each judged against its own recent good readings, and the
service defaults to mmap set 1, SteamVR's combined gaze, which keeps going
with one eye. With set 2, one-eye samples are still dropped, since its
combined direction is off by half of what the lost eye reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

gaze/README.md

@@ -18,7 +18,7 @@ gaze/ft-gazectl on            # the pointer follows your gaze (off: the mouse al
 
 Gaze as an input method for the whole desktop, without replacing anything of SteamVR's:
 
-- `ft-gazed` (host Python, a user service: `gaze/run.sh install`) runs ft-gaze and corrects its gaze. It drops blinks and dropouts, smooths with a fixation lock, and applies the calibration from the probe (`calibration.json`, reloaded when the probe changes it) plus what the pointer has learned since (`pointer-lessons.json`). It sends the result to the pointer helper 90 times a second. It follows SteamVR's eye tracking log, and when the headset goes back on (SteamVR starts its eye model over, and the error moves), older lessons count less, so the first few after relearn the offset.
+- `ft-gazed` (host Python, a user service: `gaze/run.sh install`) runs ft-gaze and corrects its gaze. It uses SteamVR's combined gaze (mmap set 1), which keeps going when the tracker loses one eye; set 2's combined direction is off by half of whatever the lost eye reads, and the tracker does lose an eye for minutes at a time. It drops blinks (both eyes closing), smooths with a fixation lock, and applies the calibration from the probe (`calibration.json`, reloaded when the probe changes it) plus what the pointer has learned since (`pointer-lessons.json`). It sends the result to the pointer helper 90 times a second. It follows SteamVR's eye tracking log, and when the headset goes back on (SteamVR starts its eye model over, and the error moves), older lessons count less, so the first few after relearn the offset.
 - The pointer helper's **gaze mode** (off by default: `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a button mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look. Move the mouse and it's the mouse's, from where the gaze put it, for the last bit. Look well away (5 degrees) and the gaze takes it back. Clicks and drags work as usual.
 - **Learning from nudges:** if the mouse took the pointer from the gaze and moved it a little (0.2 to 8 degrees) before you clicked, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
 - Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.

gaze/ft-gazed

@@ -3,8 +3,13 @@
 
 Runs ft-gaze (in the dev container), and for every eye tracker sample (90 Hz):
 
-  1. drops blinks and moments the tracker lost an eye: openness under half its running
+  1. drops blinks: both eyes' openness under half its running median (each eye its own).
-     median, or the angle between the eyes jumping more than 1.5 degrees from its median;
+     With the default source, mmap set 1, that's all: set 1 is SteamVR's combined gaze,
+     which keeps going when the tracker loses one eye (its two directions stay together).
+     Set 2's combined direction is the mean of the eyes' own, so with one eye lost it's
+     off by half of whatever that eye reads (9 to 14 degrees apart were seen): with set 2,
+     samples with an eye under its floor, or the angle between the eyes jumping more than
+     1.5 degrees from its median, are dropped too;
   2. smooths it with a fixation lock (the running mean of the current fixation, 1 degree);
   3. corrects it: the calibration from ft-gazeprobe (calibration.json, reloaded when the
      probe changes it) plus what the pointer's corrections have taught since (LiveCorrection,
@@ -31,7 +36,8 @@ Control socket: abstract unix datagram "@ft_gazed":
   forget                           drop what the lessons taught (the calibration stays)
   reload                           read calibration.json again
 
-Options: --source action|mmap1|mmap2 (default mmap2, the quietest), -v (a status line every
+Options: --source action|mmap1|mmap2 (default mmap1; set 2 was a little quieter in the probe,
+but loses the pointer whenever the tracker loses an eye), -v (a status line every
 5 s on stderr), --to NAME (send the gaze to the abstract socket @NAME instead of the pointer
 helper; for testing: a helper without gaze mode forwards what it doesn't know to its driver).
 """
@@ -83,9 +89,9 @@ class Service:
         self.live.wear_time = self.steam.worn()
         self.live.refit(self.base, self.mode)
         self.fix = Fixation(radius=1.0)
-        self.opens = deque(maxlen=90)
+        self.opens = (deque(maxlen=90), deque(maxlen=90))  # left, right
         self.vergence = deque(maxlen=90)
-        self.counts = {"samples": 0, "sent": 0, "dropped": 0, "lessons": 0, "refused": 0}
+        self.counts = {"samples": 0, "sent": 0, "blinks": 0, "one_eye": 0, "dropped": 0, "lessons": 0, "refused": 0}
         self.last_sample = 0.0
         self.last = None
 
@@ -232,18 +238,30 @@ class Service:
         m1 = s["src"].get("mmap1") or {}
         o = m1.get("open")
         lr = src.get("lr", m1.get("lr"))
-        if o:
+        # Blinks and lost eyes, judged against the last second (see steady_samples: relative,
-            self.opens.append(min(o))
-        if lr is not None:
-            self.vergence.append(lr)
-        # Blinks and dropouts, judged against the last second (see steady_samples: relative,
         # because the lids come down looking down, and the vergence depends on distance).
-        if o and len(self.opens) >= 30 and min(o) < max(0.12, 0.5 * statistics.median(self.opens)):
+        # An eye's floor comes from its good readings, so a lost eye doesn't drag it to 0.
-            self.counts["dropped"] += 1
+        low = [False, False]
-            return
+        if o and len(o) == 2:
-        if lr is not None and len(self.vergence) >= 30 and abs(lr - statistics.median(self.vergence)) > 1.5:
+            for k in (0, 1):
-            self.counts["dropped"] += 1
+                hist = self.opens[k]
+                good = [v for v in hist if v >= 0.12]
+                floor = max(0.12, 0.5 * statistics.median(good)) if len(good) >= 30 else 0.12
+                low[k] = o[k] < floor
+                hist.append(o[k])
+        if all(low):
+            self.counts["blinks"] += 1
             return
+        if any(low):
+            self.counts["one_eye"] += 1
+        if self.source == "mmap2":
+            jump = (lr is not None and len(self.vergence) >= 30
+                    and abs(lr - statistics.median(self.vergence)) > 1.5)
+            if lr is not None and not any(low):
+                self.vergence.append(lr)
+            if any(low) or jump:
+                self.counts["dropped"] += 1
+                return
         # The fixation lock works in degrees here (1 degree per "pixel").
         fy, fp = self.fix(src["hy"], src["hp"], s["t"], 1.0)
         cy, cp = self.correction(fy, fp)
@@ -342,7 +360,7 @@ class Service:
 
 def main():
     ap = argparse.ArgumentParser(description="The gaze service: corrected eye tracking for the pointer")
-    ap.add_argument("--source", choices=["action", "mmap1", "mmap2"], default="mmap2")
+    ap.add_argument("--source", choices=["action", "mmap1", "mmap2"], default="mmap1")
     ap.add_argument("-v", "--verbose", action="store_true")
     ap.add_argument("--to", default="ft_pointer_helper", help="abstract socket to send the gaze to")
     args = ap.parse_args()