Keyboard clicks at the gaze: Meta+J and Meta+K

gaze_left and gaze_right (Meta+J and Meta+K by default) click where you look.
A tap clicks where the dot was at the press and tells the gaze tracker it was
right there. Held, the pointer stays put in your view, so turning your head
carries it onto the target; the release clicks there and the correction is a
lesson (judged by the net correction, not the head's path). Held still for
POINTER_GAZE_HOLD it's a real press that the head drags, and Meta+K while
Meta+J aims presses where the dot is now, to correct and then drag.

A Meta combination now also sends the desktop an F24 press and Meta's release
at once: Meta's release no longer opens Plasma's launcher, and the click isn't
Meta+click (KWin's window move and resize, which swallowed right clicks). The
desktop gets Meta back for the next key if it's still held.

Also fixes e16b758, which had taken the gaze actions off the key combination
list along with the controllers'; and gaze_quickcal (the gaze service's
one-dot check) joins the actions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

input/input-relay.py

@@ -23,7 +23,11 @@ pointer_toggle, follow_toggle = head follow on or off, gaze_toggle = gaze mode o
 (the pointer goes where you look; see pointer/helper/ft-pointer.cpp), gaze_precision = while
 held, the pointer stops where you look and the mouse steers it, and the release clicks there,
 gaze_drag = the same, but pressed at once, so it drags ("precision|gazedrag mouse|keyboard 1|0"
-to the helper), sens_up, sens_down,
+to the helper), gaze_left and gaze_right = keyboard clicks at the gaze: a tap clicks where you
+look; held, the pointer stops there and your head steers it (it stays put in your view), and
+the release clicks; held still for half a second, it's a real press that your head drags
+("gazekey left|right 1|0" to the helper; by default Meta+J and Meta+K, DEFAULT_KEY_BINDINGS),
+gaze_quickcal = the gaze service's one-dot check ("quickcal" to @ft_gazed), sens_up, sens_down,
 layout_reset = put the desktop screens back in their saved layout, screens_toggle = hide or show the desktop screens,
 keyboard_toggle = open or close Frametop's keyboard, key = pass through as a key, none).
 
@@ -31,7 +35,10 @@ Frame controller buttons can be mapped too ("controller_buttons": {"right/a": ac
 rules file; any action but key and the gaze ones, GAZE_ACTIONS: gaze mode is a mouse feature,
 docs/gaze-controllers.md). So can key combinations on any keyboard ("key_bindings":
 {"29+56+34": action}, evdev codes joined by "+", modifiers first and left-hand codes for
-either side, here Ctrl+Alt+G): the combination does the action, and its last key isn't typed. The controllers aren't input devices here, only SteamVR sees
+either side, here Ctrl+Alt+G): the combination does the action, and its last key isn't typed.
+A combination with Meta also sends the desktop an F24 press and Meta's release right away: so
+letting go of Meta doesn't open Plasma's launcher, and a gaze click isn't Meta+click (KWin's
+window move and resize). Another key while Meta is still held gives the desktop Meta back. The controllers aren't input devices here, only SteamVR sees
 them, so the pointer helper reads them with SteamVR input and sends "vrbtn <button> 1|0".
 It only takes the buttons the relay tells it to ("vrbind <button>..." to @ft_pointer_helper,
 sent on start, reload, and when the helper says "vrhello"), and only while no game runs,
@@ -174,10 +181,14 @@ def eviocguniq(length):
 VIRTUAL_PREFIX = "frametop virtual"
 RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
 ACTIONS = ("left", "right", "middle", "back", "scroll_up", "scroll_down", "dashboard", "recenter",
-           "pointer_toggle", "follow_toggle", "gaze_toggle", "gaze_precision", "gaze_drag", "sens_up", "sens_down",
-           "layout_reset", "screens_toggle", "keyboard_toggle", "key", "none")
+           "pointer_toggle", "follow_toggle", "gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right",
+           "gaze_quickcal", "sens_up", "sens_down", "layout_reset", "screens_toggle", "keyboard_toggle", "key", "none")
 # Gaze mode is a mouse feature: these never come from a controller button (docs/gaze-controllers.md).
-GAZE_ACTIONS = ("gaze_toggle", "gaze_precision", "gaze_drag")
+GAZE_ACTIONS = ("gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right", "gaze_quickcal")
+# Key combinations when the rules file has none: Meta+J and Meta+K click at the gaze (free on the
+# Frametop desktop, and apps don't use Meta).
+DEFAULT_KEY_BINDINGS = {"125+36": "gaze_left", "125+37": "gaze_right"}
+KEY_F24 = 194  # sent to the desktop with a Meta combination (see the top)
 # Key combinations ("key_bindings"): modifiers, each side's code folded into the left one's.
 MODIFIERS = {29: 29, 97: 29, 42: 42, 54: 42, 56: 56, 100: 56, 125: 125, 126: 125}
 VR_KEYBOARD_MODES = ("always", "no_keyboard", "button", "never")  # when Frametop's keyboard opens
@@ -188,6 +199,7 @@ VR_BUTTONS = ("left/view", "left/dpad_up", "left/dpad_down", "left/dpad_left", "
               "right/bumper", "right/trigger", "right/grip", "right/thumbstick")
 VR_DEVICE = "frame_controller"  # the id controller buttons have in watch events
 SCREENS = "\0ft_screens"
+GAZED = "\0ft_gazed"
 KEYS = "\0frametop_keys"  # keys of keyboards grabbed for the desktop, for other readers
 FT_LAYOUT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "layout", "ft-layout")
 DEFAULT_BUTTONS = {BTN_LEFT: "left", BTN_RIGHT: "right", BTN_MIDDLE: "middle",
@@ -295,7 +307,7 @@ def read_rules(path=RULES_PATH):
     rules.setdefault("devices", {})
     rules.setdefault("buttons", {})
     rules.setdefault("controller_buttons", {})
-    rules.setdefault("key_bindings", {})
+    rules.setdefault("key_bindings", dict(DEFAULT_KEY_BINDINGS))
     return rules
 
 
@@ -437,6 +449,12 @@ class Pointer:
         which take the mouse or the keyboard only."""
         if value == 2:
             return
+        if name in ("gaze_left", "gaze_right"):
+            if value == 1:
+                self.wake(now)
+            self.flush()
+            self.send(f"gazekey {name[5:]} {value}")
+            return
         if name in ("gaze_precision", "gaze_drag"):
             if source not in ("mouse", "keyboard"):
                 return  # gaze mode is a mouse feature (GAZE_ACTIONS)
@@ -474,6 +492,11 @@ class Pointer:
         elif name == "gaze_toggle":
             self.send("gaze toggle")  # until the next restart; the setting is POINTER_GAZE
             log("gaze mode toggled")
+        elif name == "gaze_quickcal":
+            try:
+                self.sock.sendto(b"quickcal", GAZED)
+            except OSError:
+                pass  # the gaze service isn't running
         elif name == "screens_toggle":
             try:
                 self.sock.sendto(b"toggle", SCREENS)
@@ -698,14 +721,28 @@ def main():
             state["pointer"].action(action, value, now, source)
 
     held_modifiers = set()  # on any keyboard, folded (MODIFIERS)
+    held_meta = set()  # the Meta keys held, as they are (KEY_LEFTMETA, KEY_RIGHTMETA)
+    meta_hidden = set()  # held Meta keys the desktop was told came up (a combination; key_binding)
     combos_down = {}  # key code -> the action its combination started (released with it)
 
     def key_binding(code, value, now):
         """A key from a keyboard: does it complete a key combination ("key_bindings")? True if
         it was taken for one (then it isn't typed)."""
+        nonlocal meta_down
         if code in MODIFIERS:
             (held_modifiers.add if value else held_modifiers.discard)(MODIFIERS[code])
+            if code in (KEY_LEFTMETA, KEY_RIGHTMETA):
+                (held_meta.add if value else held_meta.discard)(code)
+                if value == 0 and code in meta_hidden:
+                    meta_hidden.discard(code)
+                    return True  # the desktop already had it come up (below)
             return False
+        if value == 1 and code not in combos_down and meta_hidden:
+            # Another key while Meta is still held after a combination: the desktop gets Meta
+            # back first, so Meta+that key still works there.
+            for c in sorted(meta_hidden):
+                to_screens(c, 1)
+            meta_hidden.clear()
         if value == 0 and code in combos_down:
             action = combos_down.pop(code)
             if state["pointer"]:
@@ -718,6 +755,17 @@ def main():
         if action not in ACTIONS or action in ("key", "none"):
             return False
         combos_down[code] = action
+        if held_meta - meta_hidden:
+            # The desktop saw Meta go down. Another key in between keeps its release from
+            # opening Plasma's launcher (and Meta from toggling the dashboard here), and Meta
+            # comes up there now: KWin takes Meta with a mouse button for moving or resizing
+            # windows, which would swallow a gaze click. Its real release is dropped (above).
+            meta_down = False
+            to_screens(KEY_F24, 1)
+            to_screens(KEY_F24, 0)
+            for c in sorted(held_meta - meta_hidden):
+                to_screens(c, 0)
+            meta_hidden.update(held_meta)
         if state["pointer"]:
             do_action(action, 1, now, "keyboard")
         log(f"key combination {combo}: {action}")