Let the gaze pointer use our own eye tracker, and weight the eyes

Frametop Input Settings' Gaze page gets two settings, saved in frametop.conf and
read again by ft-gazed when the file changes: Eye tracker (GAZE_TRACKER: SteamVR's
or our own, frame-eyes' fe-trackd) and Eye bias (GAZE_EYE: auto, left, right).

ft-gazed now combines the eyes, each calibrated on its own: SteamVR's set 2 eyes
with the probe's Left eye and Right eye calibrations, or our tracker's eyes as
they come. Without per-eye calibrations, or with --source, it keeps the older
one-source path. A pointer nudge finds its look from the raw gaze the helper
echoes back; with our tracker it goes to fe-trackd as a click.

The bias leans instead of choosing (gazecal.EyeWeights): on 306 live clicks the
eyes' errors partly cancelled, both together 0.65 deg off against 0.96 and 1.11
for either alone. Left or Right counts that eye twice; auto weights each eye by
its RMS miss at its last 20 nudges, since the calibration's fit picked the wrong
eye on SteamVR's test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

input-settings/ft_input_settings.py

@@ -13,7 +13,8 @@ to the input relay over its control socket (@frametop_relay):
   - Ignored panels: SteamVR overlays the pointer passes through (POINTER_IGNORE), by app or
     one by one. The helper lists them (@ft_pointer_helper "overlays").
   - Gaze: the pointer's gaze mode (@ft_pointer_helper "gaze") and the gaze service
-    (gaze/ft-gazed, @ft_gazed: status, forget, reload).
+    (gaze/ft-gazed, @ft_gazed: status, forget, reload), with its eye tracker (SteamVR's or
+    our own) and eye bias (GAZE_TRACKER, GAZE_EYE in frametop.conf).
   - Bluetooth: paired devices, and re-applying the Bluetooth LE fixes after pairing.
   - A warning on every page when SteamVR won't load the ft_pointer driver (blocked after a
     crash, disabled, or SteamVR in safe mode) or hasn't loaded it (@ft_pointer doesn't answer
@@ -83,6 +84,9 @@ GAZE_SETTINGS = [
     ("POINTER_GAZE_HOLD", "Hold still to drag", 0.5, 0.1, 2.0, 0.05, "s"),
     ("POINTER_GAZE_SHOW", "Dot shows after moving", 1.0, 0.0, 5.0, 0.1, "s"),
 ]
+# The gaze service's settings (gaze/ft-gazed): whose eye tracking, and the eye bias.
+GAZE_TRACKERS = {"steam": "SteamVR's eye tracker", "own": "our own eye tracker (frame-eyes)"}
+GAZE_EYES = {"auto": "auto", "left": "left eye", "right": "right eye"}
 # Pointer settings: key, label, default, min, max, step, unit.
 POINTER_SETTINGS = [
     ("POINTER_SENSITIVITY", "Speed", 0.03, 0.005, 0.12, 0.001, "°/count"),
@@ -721,6 +725,36 @@ class Backend(QObject):
         self.reload_timer.start()
         self.gazeChanged.emit()
 
+    @Property(str, notify=gazeChanged)
+    def gazeTracker(self):
+        """Whose eye tracking the gaze service uses: "steam" or "own" (GAZE_TRACKER)."""
+        v = read_conf().get("GAZE_TRACKER", "steam")
+        return v if v in GAZE_TRACKERS else "steam"
+
+    @Property(str, notify=gazeChanged)
+    def gazeEye(self):
+        """The eye bias: "auto", "left" or "right" (GAZE_EYE)."""
+        v = read_conf().get("GAZE_EYE", "auto")
+        return v if v in GAZE_EYES else "auto"
+
+    @Slot(str)
+    def setGazeTracker(self, tracker):
+        if tracker in GAZE_TRACKERS:
+            self._set_gaze("GAZE_TRACKER", tracker, f"Gaze from {GAZE_TRACKERS[tracker]}")
+
+    @Slot(str)
+    def setGazeEye(self, eye):
+        if eye in GAZE_EYES:
+            self._set_gaze("GAZE_EYE", eye, f"Eye bias: {GAZE_EYES[eye]}")
+
+    def _set_gaze(self, key, value, done):
+        """ft-gazed reads these from frametop.conf; "reload" makes it do so now."""
+        write_conf_value(key, value)
+        running = self._send("reload", GAZED)
+        self.message.emit(done if running else f"{done} (the gaze service isn't running: it takes it when it starts)",
+                          False)
+        self.gazeChanged.emit()
+
     @Property("QVariantList", notify=pointerChanged)
     def gazeSettings(self):
         conf = read_conf()

input-settings/main.qml

@@ -771,6 +771,61 @@ Kirigami.ApplicationWindow {
                     opacity: 0.7
                     font: Kirigami.Theme.smallFont
                 }
+                RowLayout {
+                    Kirigami.FormData.label: "Eye tracker:"
+                    Controls.RadioButton {
+                        text: "SteamVR"
+                        checked: backend.gazeTracker === "steam"
+                        onToggled: if (checked) backend.setGazeTracker("steam")
+                    }
+                    Controls.RadioButton {
+                        text: "Own tracker (frame-eyes)"
+                        checked: backend.gazeTracker === "own"
+                        onToggled: if (checked) backend.setGazeTracker("own")
+                    }
+                }
+                Controls.Label {
+                    // Ours runs apart from Frametop (frame-eyes: tools/fe-live) and keeps its own calibration.
+                    visible: backend.gazeTracker === "own" && backend.gazeServiceRunning
+                             && (!gpage.status.own_running || gpage.status.own_reseat || !gpage.status.calibration_samples)
+                    text: !gpage.status.own_running ? "Not running: start it in frame-eyes (tools/fe-live), then calibrate"
+                          : !gpage.status.calibration_samples ? "Not calibrated: use Calibrate… with Own tracker"
+                          : "The headset was off: your first nudge and click resets where it sits"
+                    color: gpage.status.own_running ? Kirigami.Theme.neutralTextColor : Kirigami.Theme.negativeTextColor
+                    font: Kirigami.Theme.smallFont
+                }
+                RowLayout {
+                    Kirigami.FormData.label: "Eye bias:"
+                    Controls.RadioButton {
+                        text: "Auto"
+                        checked: backend.gazeEye === "auto"
+                        onToggled: if (checked) backend.setGazeEye("auto")
+                    }
+                    Controls.RadioButton {
+                        text: "Left"
+                        checked: backend.gazeEye === "left"
+                        onToggled: if (checked) backend.setGazeEye("left")
+                    }
+                    Controls.RadioButton {
+                        text: "Right"
+                        checked: backend.gazeEye === "right"
+                        onToggled: if (checked) backend.setGazeEye("right")
+                    }
+                }
+                Controls.Label {
+                    // What the bias comes to now; on auto, from how far off each eye was at the last nudges.
+                    property var w: gpage.status.eye_weights
+                    property var n: gpage.status.eye_misses || [0, 0]
+                    property var rms: gpage.status.eye_rms || [null, null]
+                    visible: backend.gazeServiceRunning && w !== undefined
+                    text: w === undefined ? "" : "Left " + Math.round(w[0] * 100) + "% · right " + Math.round(w[1] * 100) + "%"
+                          + (backend.gazeEye !== "auto" ? ""
+                             : n[0] >= 5 && n[1] >= 5
+                               ? " (off by " + rms[0].toFixed(1) + "° and " + rms[1].toFixed(1) + "° at your last nudges)"
+                               : " (even until each eye has 5 nudges: " + n[0] + " and " + n[1] + " so far)")
+                    opacity: 0.7
+                    font: Kirigami.Theme.smallFont
+                }
                 Repeater {
                     model: backend.gazeSettings
                     delegate: RowLayout {
@@ -843,7 +898,9 @@ Kirigami.ApplicationWindow {
                     visible: backend.gazeServiceRunning
                     Kirigami.FormData.label: "Calibration:"
                     text: gpage.status.calibration_samples > 0
-                          ? gpage.status.calibration_samples + " points (" + gpage.status.model + ")"
+                          ? gpage.status.calibration_samples + " points ("
+                            + (gpage.status.tracker === "own" ? "Own tracker, " + gpage.status.calibration_made
+                               : gpage.status.kind === "eyes" ? gpage.status.model + ", each eye" : gpage.status.model) + ")"
                           : "none yet: use Calibrate…"
                 }
                 Controls.Label {
@@ -863,7 +920,10 @@ Kirigami.ApplicationWindow {
                       + "with the button held and let go there. Hold still to drag: hold a press this long without moving "
                       + "to drag something instead. Look away to hand back: how far from the pointer you look before the "
                       + "gaze takes it back from the mouse. Largest nudge to learn: bigger mouse moves before a click "
-                      + "are treated as using the mouse, not correcting the gaze."
+                      + "are treated as using the mouse, not correcting the gaze. Eye tracker: SteamVR's, or our own "
+                      + "(frame-eyes), which keeps its own calibration (Calibrate… with Own tracker). Eye bias: the gaze "
+                      + "combines both eyes, since their errors partly cancel; Left or Right counts that eye twice as "
+                      + "much, and Auto weights each by how far off it was at your recent nudges."
             }
         }
     }