Merge gaze-calibration into experimental

The gaze checks and calibration in a headset panel (quick, five, full, headset
fit; shared-buffer drawing, click-to-capture), keyboard and mouse gaze clicks
(Meta+J/K, the mouse's buttons alike, mouse moves only while a button is held,
double right/Meta+K tilts a drag), one 55 degree learning limit with a quick
check past it, eye presence for "the headset went on", the gaze probe as a
development tool, and the relay releasing keys the desktop has down that no
keyboard holds.

Conflicts with the profiles: the relay keeps known_action/needs_pointer and
the gaze defaults (Meta+J/K and the float key); Input Settings keeps the
profile actions everywhere and the gaze actions in key combinations only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

input/input-relay.py

@@ -23,7 +23,11 @@ pointer_toggle, follow_toggle = head follow on or off, gaze_toggle = gaze mode o
 (the pointer goes where you look; see pointer/helper/ft-pointer.cpp), gaze_precision = while
 held, the pointer stops where you look and the mouse steers it, and the release clicks there,
 gaze_drag = the same, but pressed at once, so it drags ("precision|gazedrag mouse|keyboard 1|0"
-to the helper), sens_up, sens_down,
+to the helper), gaze_left and gaze_right = keyboard clicks at the gaze: a tap clicks where you
+look; held, the pointer stops there and your head steers it (it stays put in your view), and
+the release clicks; held still for half a second, it's a real press that your head drags
+("gazekey left|right 1|0" to the helper; by default Meta+J and Meta+K, DEFAULT_KEY_BINDINGS),
+gaze_quickcal = the gaze service's one-dot check ("quickcal" to @ft_gazed), sens_up, sens_down,
 layout_reset = put the desktop screens back in their saved layout, screens_toggle = hide or show the desktop screens,
 keyboard_toggle = open or close Frametop's keyboard, float_toggle = float the desktop window under the
 pointer (else the active one) in VR, or put it back if it floats, dock_all = put every floating
@@ -35,8 +39,12 @@ rules file; any action but key and the gaze ones, GAZE_ACTIONS: gaze mode is a m
 docs/gaze-controllers.md). So can key combinations on any keyboard ("key_bindings":
 {"29+56+34": action}, evdev codes joined by "+", modifiers first and left-hand codes for
 either side, here Ctrl+Alt+G): the combination does the action, and its last key isn't typed.
-A rules file without "key_bindings" gets DEFAULT_KEY_BINDINGS (Meta+Shift+F: float_toggle); one
-with its own, even an empty one, doesn't. The float actions work without pointer mode too. The controllers aren't input devices here, only SteamVR sees
+A rules file without "key_bindings" gets DEFAULT_KEY_BINDINGS (Meta+J: gaze_left, Meta+K:
+gaze_right, Meta+Shift+F: float_toggle); one with its own, even an empty one, doesn't. The float
+actions work without pointer mode too. A combination with Meta also sends the desktop an F24 press
+and Meta's release right away: so letting go of Meta doesn't open Plasma's launcher, and a gaze
+click isn't Meta+click (KWin's window move and resize). Another key while Meta is still held gives
+the desktop Meta back. The controllers aren't input devices here, only SteamVR sees
 them, so the pointer helper reads them with SteamVR input and sends "vrbtn <button> 1|0".
 It only takes the buttons the relay tells it to ("vrbind <button>..." to @ft_pointer_helper,
 sent on start, reload, and when the helper says "vrhello"), and only while no game runs,
@@ -179,10 +187,15 @@ def eviocguniq(length):
 VIRTUAL_PREFIX = "frametop virtual"
 RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
 ACTIONS = ("left", "right", "middle", "back", "scroll_up", "scroll_down", "dashboard", "recenter",
-           "pointer_toggle", "follow_toggle", "gaze_toggle", "gaze_precision", "gaze_drag", "sens_up", "sens_down",
-           "layout_reset", "screens_toggle", "keyboard_toggle", "float_toggle", "dock_all", "key", "none")
+           "pointer_toggle", "follow_toggle", "gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right",
+           "gaze_quickcal", "sens_up", "sens_down", "layout_reset", "screens_toggle", "keyboard_toggle", "float_toggle",
+           "dock_all", "key", "none")
 # Gaze mode is a mouse feature: these never come from a controller button (docs/gaze-controllers.md).
-GAZE_ACTIONS = ("gaze_toggle", "gaze_precision", "gaze_drag")
+GAZE_ACTIONS = ("gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right", "gaze_quickcal")
+# Key combinations a rules file without "key_bindings" gets: Meta+J and Meta+K click at the gaze
+# (free on the Frametop desktop, and apps don't use Meta), Meta+Shift+F floats a window.
+DEFAULT_KEY_BINDINGS = {"125+36": "gaze_left", "125+37": "gaze_right", "42+125+33": "float_toggle"}
+KEY_F24 = 194  # sent to the desktop with a Meta combination (see the top)
 # Key combinations ("key_bindings"): modifiers, each side's code folded into the left one's.
 MODIFIERS = {29: 29, 97: 29, 42: 42, 54: 42, 56: 56, 100: 56, 125: 125, 126: 125}
 VR_KEYBOARD_MODES = ("always", "no_keyboard", "button", "never")  # when Frametop's keyboard opens
@@ -193,6 +206,7 @@ VR_BUTTONS = ("left/view", "left/dpad_up", "left/dpad_down", "left/dpad_left", "
               "right/bumper", "right/trigger", "right/grip", "right/thumbstick")
 VR_DEVICE = "frame_controller"  # the id controller buttons have in watch events
 SCREENS = "\0ft_screens"
+GAZED = "\0ft_gazed"
 FLOAT = "\0frametop_float"  # ft-floatd, floating windows in the Frametop desktop
 # Actions for ft-floatd ("float_toggle", "dock_all"): they don't need pointer mode.
 FLOAT_ACTIONS = {"float_toggle": b"float pointer", "dock_all": b"dock all"}
@@ -205,8 +219,6 @@ def known_action(a):
 
 def needs_pointer(a):
     return a not in FLOAT_ACTIONS and not a.startswith(PROFILE)
-# Key combinations a rules file without "key_bindings" gets: Meta+Shift+F floats a window.
-DEFAULT_KEY_BINDINGS = {"42+125+33": "float_toggle"}
 KEYS = "\0frametop_keys"  # keys of keyboards grabbed for the desktop, for other readers
 FT_LAYOUT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "layout", "ft-layout")
 DEFAULT_BUTTONS = {BTN_LEFT: "left", BTN_RIGHT: "right", BTN_MIDDLE: "middle",
@@ -470,6 +482,12 @@ class Pointer:
         which take the mouse or the keyboard only."""
         if value == 2:
             return
+        if name in ("gaze_left", "gaze_right"):
+            if value == 1:
+                self.wake(now)
+            self.flush()
+            self.send(f"gazekey {name[5:]} {value}")
+            return
         if name in ("gaze_precision", "gaze_drag"):
             if source not in ("mouse", "keyboard"):
                 return  # gaze mode is a mouse feature (GAZE_ACTIONS)
@@ -507,6 +525,11 @@ class Pointer:
         elif name == "gaze_toggle":
             self.send("gaze toggle")  # until the next restart; the setting is POINTER_GAZE
             log("gaze mode toggled")
+        elif name == "gaze_quickcal":
+            try:
+                self.sock.sendto(b"quickcal", GAZED)
+            except OSError:
+                pass  # the gaze service isn't running
         elif name == "screens_toggle":
             try:
                 self.sock.sendto(b"toggle", SCREENS)
@@ -746,14 +769,28 @@ def main():
             state["pointer"].action(action, value, now, source)
 
     held_modifiers = set()  # on any keyboard, folded (MODIFIERS)
+    held_meta = set()  # the Meta keys held, as they are (KEY_LEFTMETA, KEY_RIGHTMETA)
+    meta_hidden = set()  # held Meta keys the desktop was told came up (a combination; key_binding)
     combos_down = {}  # key code -> the action its combination started (released with it)
 
     def key_binding(code, value, now):
         """A key from a keyboard: does it complete a key combination ("key_bindings")? True if
         it was taken for one (then it isn't typed)."""
+        nonlocal meta_down
         if code in MODIFIERS:
             (held_modifiers.add if value else held_modifiers.discard)(MODIFIERS[code])
+            if code in (KEY_LEFTMETA, KEY_RIGHTMETA):
+                (held_meta.add if value else held_meta.discard)(code)
+                if value == 0 and code in meta_hidden:
+                    meta_hidden.discard(code)
+                    return True  # the desktop already had it come up (below)
             return False
+        if value == 1 and code not in combos_down and meta_hidden:
+            # Another key while Meta is still held after a combination: the desktop gets Meta
+            # back first, so Meta+that key still works there.
+            for c in sorted(meta_hidden):
+                to_screens(c, 1)
+            meta_hidden.clear()
         if value == 0 and code in combos_down:
             action = combos_down.pop(code)
             if state["pointer"] or not needs_pointer(action):
@@ -766,19 +803,33 @@ def main():
         if not known_action(action) or action in ("key", "none"):
             return False
         combos_down[code] = action
+        if held_meta - meta_hidden:
+            # The desktop saw Meta go down. Another key in between keeps its release from
+            # opening Plasma's launcher (and Meta from toggling the dashboard here), and Meta
+            # comes up there now: KWin takes Meta with a mouse button for moving or resizing
+            # windows, which would swallow a gaze click. Its real release is dropped (above).
+            meta_down = False
+            to_screens(KEY_F24, 1)
+            to_screens(KEY_F24, 0)
+            for c in sorted(held_meta - meta_hidden):
+                to_screens(c, 0)
+            meta_hidden.update(held_meta)
         if state["pointer"] or not needs_pointer(action):
             do_action(action, 1, now, "keyboard")
         log(f"key combination {combo}: {action}")
         return True
 
 
+    screens_down = set()  # keys the desktop was told went down and not yet up (see reconcile_desktop_keys)
+
     def to_screens(code, value):
         """A key for the desktop screens (ft-screens decides whether it types)."""
         if value in (0, 1) and code < BTN_MISC:
             try:
                 screens_sock.sendto(f"key {code} {value}".encode(), SCREENS)
             except OSError:
-                pass  # ft-screens not running
+                return  # ft-screens not running
+            (screens_down.add if value else screens_down.discard)(code)
     nodes = {}  # fd -> Node
     # Nodes already probed (rejected or open): path -> inode. A device that disconnects and
     # reconnects between two scans often gets the same event numbers back, so the path alone
@@ -860,6 +911,35 @@ def main():
         mouse.sync()
         keyboard.sync()
 
+    def physically_down():
+        """The keys down on every device read here, as the kernel has them (EVIOCGKEY)."""
+        down = set()
+        for node in nodes.values():
+            buf = bytearray((KEY_MAX + 8) // 8)
+            try:
+                fcntl.ioctl(node.fd, EVIOCGKEY, buf)
+            except OSError:
+                continue
+            down.update(i * 8 + bit for i, b in enumerate(buf) if b for bit in range(8) if b >> bit & 1)
+        return down
+
+    def reconcile_desktop_keys():
+        """A key the desktop has down that no device holds comes up there, and the key
+        combinations forget a Meta or modifier no device holds. A key can be left down when
+        its device vanishes with it held (release_held only lets go of it here) or a release
+        goes astray: on 2026-10-01, after a calibration, Meta stayed down in KWin, so typing
+        opened the overview and clicks on the desktop did other things."""
+        if not screens_down and not held_meta and not held_modifiers:
+            return
+        down = physically_down()
+        for code in sorted(screens_down - down):
+            to_screens(code, 0)
+            log(f"key {code} released on the desktop: no keyboard holds it")
+        for code in [c for c in held_meta if c not in down]:
+            held_meta.discard(code)
+            meta_hidden.discard(code)
+        held_modifiers.intersection_update({MODIFIERS[c] for c in down if c in MODIFIERS})
+
     def keys_down(node):
         buf = bytearray((KEY_MAX + 8) // 8)
         try:
@@ -988,11 +1068,16 @@ def main():
 
     vr_bind(time.monotonic())  # a helper that's already running keeps its buttons in step
     waiting = False  # a keyboard's grab waits for its keys to come up
+    # A relay that went away with a key down left it down on the desktop, where this one
+    # never sent it: modifiers come up there now (a release of a key that isn't down is nothing).
+    for code in sorted(MODIFIERS):
+        to_screens(code, 0)
     while True:
         now = time.monotonic()
         pointer = state["pointer"]
         if now >= next_scan:
             next_scan = now + 1.0
+            reconcile_desktop_keys()
             current = {}
             for name in os.listdir("/dev/input"):
                 if name.startswith("event"):