Input relay: never block on the pointer helper's socket

The relay sent to @ft_pointer_helper on a blocking socket. When the helper stalled, a
layout placement or grabprobe holds it for seconds while ft-gazed keeps filling its socket at
90 Hz, the relay's one loop blocked with it: keyboards, the volume keys (which must never
reach gamescope), and pausing all stopped until the helper read again.

The socket is non-blocking now. A command the helper doesn't take (EAGAIN) waits in a queue,
and everything after it queues behind it so the order holds; tick() sends what it can on each
loop, and the select timeout drops to 20 ms while anything waits. Mouse moves add up into one
queued move. A scroll notch is dropped rather than queued, since scrolling seconds late is no
use; its release still goes. Presses, releases, show, hide, and the rest are kept, so no
button stays down. The queue holds at most 512 commands. While paused, the configured
pointer's queue still drains, so the releases and "hide" from standing down arrive. A
"vrbind" that hits a full socket is sent again on the next loop instead of being lost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/design.md

@@ -153,6 +153,8 @@ SteamVR opens every input device only when it starts. When a Bluetooth mouse sle
 
 Keyboards aren't grabbed by default, because a grabbed keyboard's keys went into a virtual keyboard nothing typed from; the relay forwards them to ft-screens instead.
 
+The relay never waits on the pointer helper. Its socket to the helper used to block, so when the helper stalled (a layout placement or `grabprobe` holds it for seconds, and the gaze service fills its socket 90 times a second meanwhile), the whole relay stopped with it: keyboards, the volume keys, and pausing. Now what the helper doesn't take waits in order and goes out on the next loops. Mouse moves add up into one while they wait, and a scroll notch is dropped, since scrolling seconds late is no use; presses and releases are kept, so no button stays down.
+
 An ungrabbed keyboard reaches both sides at once. In VR, gamescope reads every input device itself (the SteamOS build's `InputStealer`, libinput with udev hotplug, so new devices too) and types into its focused app, and ft-screens types the same keys into the desktop. So Space in the desktop also paused Spotify on the dashboard. Typing now follows the last click. ft-screens sees clicks on its own screens, from the mouse or a controller. A click anywhere else is only visible for the mouse: overlay apps get SteamVR's `OverlayFocusChanged` (which panel the laser is on) but no controller button events, so the pointer helper reports the panel under the dot on each left press. ft-screens tells the relay where typing goes every second, from an unbound socket so the relay's replies can't loop back into its control socket, and the relay grabs pass-through keyboards while it's the desktop. A grab waits until the keyboard has no key down, so no key stays held on either side, and the relay lets go if ft-screens stops reporting. A program that reads every keyboard for a hotkey (a dictation tool, say) loses a grabbed keyboard. Repeating the keys on another input device doesn't work: gamescope reads that device too, whether it's the relay's virtual keyboard or one created later, and every Space, typed or dictated, paused Spotify again. So with `SHARE_KEYS=1` the relay sends a grabbed keyboard's keys to `@frametop_keys` as datagrams (`key <code> <value> <device name>`). It's off by default, because the relay can't tell who is listening: abstract sockets have no permissions, and any local process that binds the name first gets every key typed into the desktop, passwords included. A listener should accept only its own user (`SO_PASSCRED`) and skip any keyboard of its own that the relay grabs too.
 
 Volume keys must never reach gamescope. With the openvr backend, gamescope sends volume up and down to Steam by moving keyboard focus to Steam for the key and then back to the previously focused surface. When nothing had focus, the one it moves back to is null, and wlroots aborts on a null focus surface (`wlr_seat_keyboard_notify_enter: Assertion 'surface' failed`), which ends the whole VR session. Keyboard focus is often empty while you work in VR, so one press of the headset's volume button could take everything down. gamescope reads the headset's buttons and every keyboard itself (`InputStealer`), as do SteamVR's processes, so the relay has to stop volume keys at the device. Grabbing `gpio-keys` would also take the headset's click button, so the relay remaps the volume entries in each device's keymap (`EVIOCSKEYCODE`) and handles the stand-in codes itself. That fix covers every device at once, including keyboards that aren't grabbed.

input/input-relay.py

@@ -128,6 +128,7 @@ kernel's evdev and uinput interfaces.
 """
 import array
 import atexit
+import collections
 import errno
 import fcntl
 import json
@@ -459,9 +460,15 @@ class Pointer:
     CLAIM_PULSE = 0.06  # seconds the claim button (switchlaserhand, no click) is held
     RESUME_PAUSE = 1.5  # mouse idle this long, then moving again, re-claims the laser
     WAKE_WINDOW = 1.0  # seconds in which WAKE_COUNTS of motion must add up
+    QUEUE_MAX = 512  # commands kept while the helper is behind (see send)
 
     def __init__(self, sensitivity, idle, wake_counts=40):
-        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+        # Never blocks (see send): a stalled helper must not stall the keyboard, volume keys and pausing.
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_NONBLOCK)
+        # Commands the helper's full socket didn't take yet, in order: text, or [dyaw, dpitch] for
+        # mouse moves, which add up into one while they wait.
+        self.queue = collections.deque()
+        self.behind_logged = -60.0
         self.sensitivity = sensitivity  # degrees per mouse count
         self.idle = idle
         self.active = False
@@ -479,11 +486,59 @@ class Pointer:
         self.pending_since = 0.0
         self.gaze_awake_until = 0.0  # the helper's gaze mode keeps the pointer until then
 
-    def send(self, command):
-        try:
-            self.sock.sendto(command.encode(), HELPER)
-        except OSError:
-            pass  # helper not running (SteamVR not running)
+    def send(self, command, droppable=False):
+        """To the helper, in order, without blocking. While the helper doesn't keep up (place and
+        grabprobe hold it for seconds, and ft-gazed's 90 Hz gaze fills its socket meanwhile), commands
+        wait in the queue and go out from tick(). A droppable one (a scroll notch: scrolling seconds
+        late is no use) is dropped instead; presses, releases and the rest are kept, so no button
+        stays down. Moves add up (_move)."""
+        if not self.queue:
+            try:
+                self.sock.sendto(command.encode(), HELPER)
+                return
+            except BlockingIOError:
+                pass
+            except OSError:
+                return  # helper not running (SteamVR not running)
+        if not droppable:
+            self._queue(command)
+
+    def _move(self, dyaw, dpitch):
+        if self.queue and isinstance(self.queue[-1], list):
+            self.queue[-1][0] += dyaw
+            self.queue[-1][1] += dpitch
+            return
+        if not self.queue:
+            try:
+                self.sock.sendto(f"move {dyaw:.4f} {dpitch:.4f}".encode(), HELPER)
+                return
+            except BlockingIOError:
+                pass
+            except OSError:
+                return
+        self._queue([dyaw, dpitch])
+
+    def _queue(self, item):
+        if not self.queue and time.monotonic() - self.behind_logged > 60:
+            self.behind_logged = time.monotonic()
+            log("pointer helper is behind: holding its commands (logged once a minute)")
+        self.queue.append(item)
+        if len(self.queue) > self.QUEUE_MAX:
+            self.queue.popleft()  # stalled for long: the oldest goes
+
+    def drain(self):
+        """Send what waits, in order, as far as the helper takes it."""
+        while self.queue:
+            item = self.queue[0]
+            text = f"move {item[0]:.4f} {item[1]:.4f}" if isinstance(item, list) else item
+            try:
+                self.sock.sendto(text.encode(), HELPER)
+            except BlockingIOError:
+                return
+            except OSError:
+                self.queue.clear()  # the helper went away: nothing to deliver to
+                return
+            self.queue.popleft()
 
     def wake(self, now):
         if not self.active:
@@ -511,7 +566,7 @@ class Pointer:
         elif code == REL_Y:
             self.dy += value
         elif code == REL_WHEEL and value:
-            self.send(f"scroll 0 {1 if value > 0 else -1}")
+            self.send(f"scroll 0 {1 if value > 0 else -1}", droppable=True)
             self.scroll_until = now + self.SCROLL_PULSE
 
     def action(self, name, value, now, source="mouse"):
@@ -543,7 +598,7 @@ class Pointer:
             return  # the rest act on press
         elif name in ("scroll_up", "scroll_down"):
             self.wake(now)
-            self.send(f"scroll 0 {1 if name == 'scroll_up' else -1}")
+            self.send(f"scroll 0 {1 if name == 'scroll_up' else -1}", droppable=True)
             self.scroll_until = now + self.SCROLL_PULSE
         elif name == "dashboard":
             self.dashboard(now)
@@ -585,7 +640,7 @@ class Pointer:
     def flush(self):
         if self.dx or self.dy:
             # Mouse right turns the ray right (negative yaw); mouse down tilts it down.
-            self.send(f"move {-self.dx * self.sensitivity:.4f} {-self.dy * self.sensitivity:.4f}")
+            self._move(-self.dx * self.sensitivity, -self.dy * self.sensitivity)
             self.dx = self.dy = 0
 
     def dashboard(self, now=None):
@@ -601,6 +656,7 @@ class Pointer:
         self.system_at = now + (0.4 if woke else 0.0)
 
     def tick(self, now):
+        self.drain()
         if self.system_at is not None and now >= self.system_at:
             self.send("btn system 1")
             self.system_at = None
@@ -625,7 +681,7 @@ class Pointer:
 
     def timeout(self):
         pending = (self.scroll_until, self.claim_at, self.claim_release, self.system_at, self.system_release)
-        return 0.02 if any(t is not None for t in pending) else 0.5
+        return 0.02 if self.queue or any(t is not None for t in pending) else 0.5
 
     def stand_down(self):
         """Frametop is pausing: a pulse under way ends now, and the pointer lets go."""
@@ -729,7 +785,7 @@ def main():
     # pointer: the 3D mouse while it's in use, pointer_conf: the one the config asks for (they
     # differ while Frametop is paused).
     state = {"pointer": None, "pointer_conf": None, "rules": {}, "share_keys": False,
-             "desktop_until": 0.0, "typing_applied": None, "vr_capture_until": 0.0}
+             "desktop_until": 0.0, "typing_applied": None, "vr_capture_until": 0.0, "vr_bind_retry": False}
 
     def pause_changed(paused):
         """Frametop paused or resumed (game_pause.py): the relay's own part."""
@@ -779,8 +835,11 @@ def main():
                                and a not in GAZE_ACTIONS and (not pause.paused or works_paused(a))) or "-"
             if state["rules"].get("controller_in_games"):
                 buttons = "+games " + buttons
+        state["vr_bind_retry"] = False
         try:
             screens_sock.sendto(f"vrbind {buttons}".encode(), HELPER)
+        except BlockingIOError:
+            state["vr_bind_retry"] = True  # the helper is behind: again on the next loop
         except OSError:
             pass  # helper not running; it says vrhello when it starts
 
@@ -1238,6 +1297,10 @@ def main():
         now = time.monotonic()
         if pointer:
             pointer.tick(now)
+        elif state["pointer_conf"]:
+            state["pointer_conf"].drain()  # paused: what waited still goes, in order
+        if state["vr_bind_retry"]:
+            vr_bind(now)
         volume.tick(now)
         pause.tick(now)
         if state["vr_capture_until"] and now >= state["vr_capture_until"]: