Map the Frame controllers' buttons; make gaze clicks correctable

The Frame controllers aren't input devices on the host, so the pointer
helper reads them with SteamVR input (vrbuttons.h, actions/), one action
set per button, and sends presses to the relay, which does the mapped
action like for a mouse button. Only mapped buttons are taken, at an
overlay-global priority (SteamVR's experimental "Enable global input from
overlays"), and only outside games unless In games is on. Frametop Input
Settings gets a Controllers page to map them.

Gaze mode: a left press while the gaze has the pointer isn't sent at
once. The pointer stops, you drag it onto what you meant with the button
held, and the release clicks there; a press held still for
POINTER_GAZE_HOLD (0.5 s) becomes a real press, for drags. The dot shows
only while the mouse moves it (POINTER_GAZE_SHOW), while a press is held,
and as a pulse per click. Outside games the pointer stays on while gaze
mode is on. ft-gazed takes a third of each lesson's offset instead of all
of it: in the first live test one 6 degree lesson moved everything and put
the next target 7 degrees off. Frametop Input Settings gets a Gaze page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

README.md

@@ -53,7 +53,7 @@ If you work in the desktop for long stretches, stop Steam from putting the heads
 | Meta+Shift+H in the desktop | Hides or shows all screens (also in the menu as Hide/Show Screens, and mappable). The Visibility & wrist tab of Frametop Display Settings can instead show them only with the dashboard open, or while you look at your wrist |
 | Play a VR game | The screens hide and your controllers stay in the game. Open the SteamVR dashboard, or press Meta+Shift+H, to see and use them. To keep them visible over games, change During VR games on the Visibility & wrist tab; the controllers still stay in the game, and you use the screens with the mouse or the dashboard |
 
-You can map the mouse's extra buttons to actions such as Toggle SteamVR dashboard, Recenter pointer, or Head follow on/off on the Buttons page of Frametop Input Settings. Pointer speed, dot size, and the rest are on its Pointer page and take effect immediately. Head follow, which is experimental and off by default, makes the pointer come along when you turn your head: it stays put until your head turns past the leash angle, then glides back to its place in your view, and a leash of 0 keeps it fixed in your view. It's only lightly tested and not polished; tuning its settings, or improving how it feels, is open to anyone who wants to take it further.
+You can map the mouse's extra buttons to actions such as Toggle SteamVR dashboard, Recenter pointer, or Head follow on/off on the Buttons page of Frametop Input Settings, and the Frame controllers' buttons on its Controllers page. Pointer speed, dot size, and the rest are on its Pointer page and take effect immediately. Head follow, which is experimental and off by default, makes the pointer come along when you turn your head: it stays put until your head turns past the leash angle, then glides back to its place in your view, and a leash of 0 keeps it fixed in your view. It's only lightly tested and not polished; tuning its settings, or improving how it feels, is open to anyone who wants to take it further.
 
 Restarting the desktop (Restart desktop in Frametop Display Settings) closes its windows, but background work you started in it, such as servers, tmux sessions, or builds, keeps running.
 

docs/design.md

@@ -82,7 +82,7 @@ A few overlays need special handling:
 
 Head follow is experimental and off by default. It works, but it's only lightly tested, and the feel is mostly a matter of its settings; polishing it is left open. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction, where you were facing when your head last settled, and keeps its offset from it. The mouse can put the cursor anywhere up to `POINTER_FOLLOW_REACH` (70 degrees) from the reference, a corner of your view included. While your head stays within `POINTER_LEASH_DEG` of the reference, nothing moves on its own. Once your head has been past the leash for `POINTER_LEASH_DELAY` (0.2 s, so a glance out and back doesn't count), the reference eases to where you're facing (time constant `POINTER_LEASH_RETURN`, 0.2 s), never falling further behind than the leash, and the cursor ends up back where it was in your view. Then it waits for the leash again. Two earlier versions didn't work out. Moving the reference only while your head pulled at the end of the leash left it up to the leash off after you turned back, and getting it centred again meant overshooting with your head. Easing it toward your facing all the time moved the cursor on every small head movement. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping.
 
-Gaze mode is experimental and off by default (`POINTER_GAZE=1`, `gaze/ft-gazectl on`, or a mouse button mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz, and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. Moving the mouse takes the pointer from the gaze. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge of up to `POINTER_GAZE_NUDGE_MAX` (8 degrees) before a click is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. See `gaze/README.md` for the service, the calibration, and what was measured.
+Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse or controller button mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz, and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. Moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze; the mouse is gaze mode's only pointer device for now. The dot shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. The gaze moving the pointer doesn't show it: you know where you're looking. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge of up to `POINTER_GAZE_NUDGE_MAX` (8 degrees) before a click is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. See `gaze/README.md` for the service, the calibration, and what was measured.
 
 Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place.
 
@@ -108,6 +108,8 @@ An ungrabbed keyboard reaches both sides at once. In VR, gamescope reads every i
 
 Volume keys must never reach gamescope. With the openvr backend, gamescope sends volume up and down to Steam by moving keyboard focus to Steam for the key and then back to the previously focused surface. When nothing had focus, the one it moves back to is null, and wlroots aborts on a null focus surface (`wlr_seat_keyboard_notify_enter: Assertion 'surface' failed`), which ends the whole VR session. Keyboard focus is often empty while you work in VR, so one press of the headset's volume button could take everything down. gamescope reads the headset's buttons and every keyboard itself (`InputStealer`), as do SteamVR's processes, so the relay has to stop volume keys at the device. Grabbing `gpio-keys` would also take the headset's click button, so the relay remaps the volume entries in each device's keymap (`EVIOCSKEYCODE`) and handles the stand-in codes itself. That fix covers every device at once, including keyboards that aren't grabbed.
 
+The Frame controllers can be mapped like mouse buttons, but they aren't input devices on the host: they reach SteamVR over the headset's own radio, and no evdev or hidraw node exists for them. So only a SteamVR client can read them. Overlay apps normally get controller input only while they have input focus, which a background helper never has. SteamVR's experimental global action set priority (`steamvr/globalActionSetPriority`, "Enable global input from overlays") lets an overlay's action set receive input anyway, and takes the inputs it binds from the scene app. Binding every button would take them all from games, so the helper's action manifest puts each button in an action set of its own, and it activates only the sets of mapped buttons. The mapping itself stays in the relay, which does the action, so mice and controllers share one list of actions.
+
 ## The desktop session
 
 The session is modeled on SteamOS's `steamos-nested-desktop` and runs beside it. It has its own runtime directory, config (`~/.config/frametop`), and state, so it never disturbs the stock desktop's layout or panels. It runs on a private D-Bus from `dbus-run-session`, which has two consequences. KDE only launches apps in systemd scopes when systemd is on the session bus, so everything started in the desktop lands in its systemd unit, and stopping the unit would kill all of it; `session/keep-apps.sh` moves those programs out first. And tools that need the real user bus, like podman and `distrobox-host-exec`, have to be pointed at it explicitly.

docs/reference.md

@@ -98,15 +98,17 @@ pointer/helper/run.sh status | log | restart
 pointer/driver/install.sh probe     # devices, hand roles, who owns the dashboard pointer
 ```
 
-The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, the head follow settings `POINTER_FOLLOW`, `POINTER_LEASH_DEG`, `POINTER_LEASH_DELAY`, `POINTER_LEASH_RETURN`, and `POINTER_FOLLOW_REACH`, and the gaze mode settings `POINTER_GAZE`, `POINTER_GAZE_RETAKE`, and `POINTER_GAZE_NUDGE_MAX`. The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper.
+The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, the head follow settings `POINTER_FOLLOW`, `POINTER_LEASH_DEG`, `POINTER_LEASH_DELAY`, `POINTER_LEASH_RETURN`, and `POINTER_FOLLOW_REACH`, and the gaze mode settings `POINTER_GAZE`, `POINTER_GAZE_RETAKE`, `POINTER_GAZE_NUDGE_MAX`, `POINTER_GAZE_HOLD`, and `POINTER_GAZE_SHOW`. The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper.
 
 ## Frametop Input Settings
 
-A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has four pages:
+A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has six pages:
 
 - Devices lists every USB and Bluetooth mouse and keyboard, with a light that flashes when the device is used. Each device gets a role: 3D pointer (grabbed, drives the pointer; the default for anything with a mouse), Pass through (grabbed only while typing goes to the desktop; the default for keyboards, where a Meta tap toggles the dashboard if `META_DASHBOARD=1` is in `~/.config/frametop.conf`), or Ignore. A device is identified by its Bluetooth address, or its USB ids and name, so all of its input nodes share one role. Forget drops everything saved for a device.
 - Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, gaze pointer on or off, faster or slower, pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
+- Controllers maps the Frame controllers' buttons (every button but the system button) to the same actions, except passing a key through. Capture a button and press it on a controller, or pick it from the list. The controllers aren't input devices on the host; only SteamVR sees them. So the pointer helper reads them with SteamVR input (`pointer/helper/vrbuttons.h`, `pointer/helper/actions/`) and sends presses to the relay (`vrbtn right/a 1`), which does the mapped action. The helper only takes the buttons that are mapped (the relay tells it with `vrbind`), at an overlay-global priority, and only while no game (scene application) runs, so games keep every button; with In games on (`controller_in_games`), a mapped button is taken from games too. That needs SteamVR's "Enable global input from overlays (Experimental)" setting (`steamvr/globalActionSetPriority`); the page offers to turn it on. Mappings are saved as `controller_buttons` in `~/.config/frametop-input.json`.
 - Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button.
+- Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), the gaze mode sliders, the gaze service's state (headset, samples per second, whether only one eye is tracked, the calibration, the nudges learned), and Calibrate (opens the gaze probe), Reload calibration, and Forget nudges.
 - Bluetooth lists paired devices and has Apply Bluetooth fixes, which runs `/etc/steamframe/bt-fixups.sh` through `pkexec`. Pair new devices in Steam.
 
 Device rules are saved in `~/.config/frametop-input.json`. `input-settings/install.sh` installs the menu entry. Its launcher hands podman the real `XDG_RUNTIME_DIR` and user bus and gives the app the session's Wayland socket, because the desktop session runs on a private D-Bus and podman fails on it.

gaze/README.md

@@ -19,8 +19,8 @@ gaze/ft-gazectl on            # the pointer follows your gaze (off: the mouse al
 Gaze as an input method for the whole desktop, without replacing anything of SteamVR's:
 
 - `ft-gazed` (host Python, a user service: `gaze/run.sh install`) runs ft-gaze and corrects its gaze. It uses SteamVR's combined gaze (mmap set 1), which keeps going when the tracker loses one eye; set 2's combined direction is off by half of whatever the lost eye reads, and the tracker does lose an eye for minutes at a time. It drops blinks (both eyes closing), smooths with a fixation lock, and applies the calibration from the probe (`calibration.json`, reloaded when the probe changes it) plus what the pointer has learned since (`pointer-lessons.json`). It sends the result to the pointer helper 90 times a second. It follows SteamVR's eye tracking log, and when the headset goes back on (SteamVR starts its eye model over, and the error moves), older lessons count less, so the first few after relearn the offset.
-- The pointer helper's **gaze mode** (off by default: `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a button mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look. Move the mouse and it's the mouse's, from where the gaze put it, for the last bit. Look well away (5 degrees) and the gaze takes it back. Clicks and drags work as usual.
-- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it a little (0.2 to 8 degrees) before you clicked, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
+- The pointer helper's **gaze mode** (off by default: the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a mouse or controller button mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look. Move the mouse and it's the mouse's, from where the gaze put it, for the last bit. Look well away (5 degrees) and the gaze takes it back. A press isn't sent at once: the pointer stops where the gaze put it, and if that's wrong, drag it onto what you meant with the button still held; the click happens where you let go. To drag something, hold the press still for half a second first (`POINTER_GAZE_HOLD`), then move. Outside games the pointer stays on while gaze mode is on. The dot only shows while the mouse moves it, while a press is held, and as a pulse when you click.
+- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it a little (0.2 to 8 degrees) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
 - Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
 
 Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction, the correction at the time, and how far off it was.

gaze/ft-gazed

@@ -69,6 +69,18 @@ LESSON_MAX = 8.0  # degrees past the correction
 RETRY = 3.0       # seconds before starting ft-gaze again
 
 
+class PointerLessons(LiveCorrection):
+    """LiveCorrection, with the offset held back: one lesson moves the whole correction by a
+    third of what it measured, not all of it (two alike, half; three, three fifths). In the
+    probe, clicks came thick and fast on a stale calibration, where the error was mostly one
+    offset. Here lessons are few and the calibration is often fresh: in the first live test a
+    6 degree lesson shifted everything 6 degrees, and the next target, 10 degrees away and
+    1.4 off before, was 7.1 off (3.6 with this). Near the lesson, the kernel still takes up
+    most of it (5.1 of the 6 degrees)."""
+
+    RIDGE = [2.0] + LiveCorrection.RIDGE[1:]
+
+
 def log(msg):
     print(f"ft-gazed: {msg}", file=sys.stderr, flush=True)
 
@@ -80,7 +92,7 @@ class Service:
         self.base = Correction()
         self.mode = DEFAULT_MODEL
         self.cal_mtime = None
-        self.live = LiveCorrection()
+        self.live = PointerLessons()
         self.dirty = False
         self.load_calibration()
         self.load_lessons()
@@ -91,7 +103,7 @@ class Service:
         self.fix = Fixation(radius=1.0)
         self.opens = (deque(maxlen=90), deque(maxlen=90))  # left, right
         self.vergence = deque(maxlen=90)
-        self.counts = {"samples": 0, "sent": 0, "blinks": 0, "one_eye": 0, "dropped": 0, "lessons": 0, "refused": 0}
+        self.counts = {"samples": 0, "sent": 0, "blinks": 0, "one_eye": 0, "dropped": 0, "lessons_taken": 0, "refused": 0}
         self.last_sample = 0.0
         self.last = None
 
@@ -124,7 +136,7 @@ class Service:
         try:
             d = json.loads(LESSONS.read_text())
             if d.get("source") == self.source:
-                self.live.samples = d.get("samples", [])[-LiveCorrection.KEEP:]
+                self.live.samples = d.get("samples", [])[-PointerLessons.KEEP:]
         except (OSError, ValueError):
             pass
         log(f"{len(self.live.samples)} lessons")
@@ -152,7 +164,7 @@ class Service:
         else:
             self.live.add({"time": rec["time"], "hy": rhy, "hp": rhp, "dy": dy, "dp": dp, "wy": 1.0, "wp": 1.0,
                            "how": "pointer"}, self.base, self.mode)
-            self.counts["lessons"] += 1
+            self.counts["lessons_taken"] += 1
             self.dirty = True
         try:
             with open(LESSON_LOG, "a") as f:
@@ -293,7 +305,7 @@ class Service:
             elif words[:1] == ["status"]:
                 reply = json.dumps(self.status())
             elif words[:1] == ["forget"]:
-                self.live = LiveCorrection()
+                self.live = PointerLessons()
                 self.live.wear_time = self.steam.worn()
                 self.save_lessons()
                 reply = "ok"

input-settings/ft_input_settings.py

@@ -6,7 +6,12 @@ to the input relay over its control socket (@frametop_relay):
   - Devices: every USB/Bluetooth mouse and keyboard, a live activity light to
     identify them, and a role for each (3D pointer, pass through, ignore).
   - Buttons: press a button or key on a pointer device, then pick an action.
+  - Controllers: the same for the Frame controllers' buttons. They're read by the pointer
+    helper through SteamVR input (@ft_pointer_helper: vrstatus, vrglobal), and a mapped
+    button is taken from games.
   - Pointer: speed, dot size, distance and the rest, applied live.
+  - Gaze: the pointer's gaze mode (@ft_pointer_helper "gaze") and the gaze service
+    (gaze/ft-gazed, @ft_gazed: status, forget, reload).
   - Bluetooth: paired devices, and re-applying the Bluetooth LE fixes after pairing.
 Rules go to ~/.config/frametop-input.json and pointer settings to
 ~/.config/frametop.conf; then the relay (and through it the helper) reloads.
@@ -19,6 +24,7 @@ import shutil
 import socket
 import subprocess
 import sys
+import time
 
 from PySide6.QtCore import Property, QObject, QSocketNotifier, QTimer, QUrl, Signal, Slot
 from PySide6.QtGui import QGuiApplication, QIcon
@@ -29,6 +35,9 @@ RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
 CONF_PATH = os.path.expanduser("~/.config/frametop.conf")
 RELAY = "\0frametop_relay"
 HELPER = "\0ft_pointer_helper"
+GAZED = "\0ft_gazed"
+REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+GAZE_PROBE = os.path.join(REPO, "gaze", "probe", "ft-gazeprobe")
 BTN_MISC = 0x100
 # Header names that mark the start of a range, not a real key (BTN_MOUSE == BTN_LEFT).
 RANGE_ALIASES = {"BTN_MISC", "BTN_MOUSE", "BTN_JOYSTICK", "BTN_GAMEPAD", "BTN_DIGI", "BTN_WHEEL",
@@ -45,6 +54,24 @@ ACTION_LABELS = {
     "none": "Do nothing",
 }
 ROLE_LABELS = {"pointer": "3D pointer", "passthrough": "Pass through", "ignore": "Ignore"}
+# Frame controller buttons the pointer helper can read (pointer/helper/vrbuttons.h). The
+# system button stays SteamVR's.
+CONTROLLER_BUTTONS = {
+    "left/view": "Left View", "left/dpad_up": "Left D-pad up", "left/dpad_down": "Left D-pad down",
+    "left/dpad_left": "Left D-pad left", "left/dpad_right": "Left D-pad right", "left/bumper": "Left bumper",
+    "left/trigger": "Left trigger", "left/grip": "Left grip", "left/thumbstick": "Left stick click",
+    "right/menu": "Right Menu", "right/a": "Right A", "right/b": "Right B", "right/x": "Right X", "right/y": "Right Y",
+    "right/bumper": "Right bumper", "right/trigger": "Right trigger", "right/grip": "Right grip",
+    "right/thumbstick": "Right stick click",
+}
+CONTROLLER_ACTIONS = [a for a in ACTION_LABELS if a not in ("key", "none")]
+# Gaze mode settings (pointer helper), like POINTER_SETTINGS.
+GAZE_SETTINGS = [
+    ("POINTER_GAZE_RETAKE", "Look away to hand back", 5, 1, 45, 0.5, "°"),
+    ("POINTER_GAZE_NUDGE_MAX", "Largest nudge to learn", 8, 1, 30, 0.5, "°"),
+    ("POINTER_GAZE_HOLD", "Hold still to drag", 0.5, 0.1, 2.0, 0.05, "s"),
+    ("POINTER_GAZE_SHOW", "Dot shows after moving", 1.0, 0.0, 5.0, 0.1, "s"),
+]
 # Pointer settings: key, label, default, min, max, step, unit.
 POINTER_SETTINGS = [
     ("POINTER_SENSITIVITY", "Speed", 0.03, 0.005, 0.12, 0.001, "°/count"),
@@ -134,8 +161,11 @@ class Backend(QObject):
     mappingsChanged = Signal()
     pointerChanged = Signal()
     bluetoothChanged = Signal()
+    controllersChanged = Signal()
+    gazeChanged = Signal()
     activity = Signal(str)  # device id
     captured = Signal(int, str)  # code, name
+    capturedController = Signal(str, str)  # button, label
     message = Signal(str, bool)  # text, is error
 
     def __init__(self):
@@ -146,6 +176,13 @@ class Backend(QObject):
         self._capture_id = ""
         self._bluetooth = []
         self._relay_ok = False
+        self._capture_vr = False
+        self._vr = {}  # the helper's vrstatus, {} when it doesn't answer
+        self._vr_at = 0.0
+        self._gaze = {}  # ft-gazed's status, {} when it isn't running
+        self._gaze_prev = None  # the status before, for rates
+        self._gaze_at = 0.0
+        self._gaze_mode = None  # the helper's gaze mode: True, False, None (no answer)
         self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
         self.sock.bind("")  # autobind an abstract address the relay can reply to
         self.sock.setblocking(False)
@@ -173,6 +210,19 @@ class Backend(QObject):
 
     def _refresh(self):
         self._send("devices")
+        self._send("vrstatus", HELPER)
+        self._send("gaze ?", HELPER)
+        if not self._send("status", GAZED) and self._gaze:
+            self._gaze, self._gaze_prev = {}, None
+            self.gazeChanged.emit()
+        now = time.monotonic()
+        # No answer for a while: that side isn't running (any more).
+        if self._vr and now - self._vr_at > 5:
+            self._vr = {}
+            self.controllersChanged.emit()
+        if self._gaze_mode is not None and now - self._gaze_at > 5:
+            self._gaze_mode = None
+            self.gazeChanged.emit()
 
     def _read(self):
         while True:
@@ -180,16 +230,37 @@ class Backend(QObject):
                 data = self.sock.recv(65536)
             except BlockingIOError:
                 return
+            text = data.decode(errors="replace")
+            if text in ("ok on", "ok off"):  # the helper's answer to "gaze ?" (from an unbound socket)
+                self._gaze_mode = text == "ok on"
+                self._gaze_at = time.monotonic()
+                self.gazeChanged.emit()
+                continue
             try:
                 msg = json.loads(data)
             except ValueError:
                 continue
+            if isinstance(msg, dict) and "samples" in msg and "t" not in msg:  # ft-gazed's status
+                self._gaze_status(msg)
+                continue
+            if not isinstance(msg, dict):
+                continue
             t = msg.get("t")
             if t == "devices":
                 self._nodes = msg.get("nodes", [])
                 self._pointer_mode = bool(msg.get("pointer_mode"))
                 self._relay_ok = True
                 self.devicesChanged.emit()
+            elif t == "vrstatus":
+                self._vr = msg
+                self._vr_at = time.monotonic()
+                self.controllersChanged.emit()
+            elif t == "event" and msg.get("type") == "vr":
+                self.activity.emit(msg["id"])
+                if self._capture_vr and msg["value"] == 1 and msg["code"] in CONTROLLER_BUTTONS:
+                    self._capture_vr = False
+                    self._send("vrcapture 0")
+                    self.capturedController.emit(msg["code"], CONTROLLER_BUTTONS[msg["code"]])
             elif t == "event":
                 self.activity.emit(msg["id"])
                 if (self._capture_id and msg["id"] == self._capture_id and msg["type"] == "key"
@@ -381,6 +452,167 @@ class Backend(QObject):
     def recenter(self):
         return self._send("recenter", HELPER)
 
+    # --- controllers ---
+    @Property("QVariantList", constant=True)
+    def controllerActions(self):
+        return [{"value": a, "text": ACTION_LABELS[a]} for a in CONTROLLER_ACTIONS]
+
+    @Property("QVariantList", constant=True)
+    def controllerButtons(self):
+        return [{"value": b, "text": label} for b, label in CONTROLLER_BUTTONS.items()]
+
+    @Property("QVariantList", notify=mappingsChanged)
+    def controllerMappings(self):
+        mapped = read_json(RULES_PATH).get("controller_buttons", {})
+        return [{"button": b, "label": label, "action": mapped[b],
+                 "actionLabel": ACTION_LABELS.get(mapped[b], mapped[b])}
+                for b, label in CONTROLLER_BUTTONS.items() if b in mapped]
+
+    @Property("QVariantMap", notify=controllersChanged)
+    def controllerStatus(self):
+        """helper: answering; manifest: its SteamVR input is set up; global: SteamVR's
+        "Enable global input from overlays"; active: mapped buttons SteamVR delivers now."""
+        vr = self._vr
+        return {"helper": bool(vr), "manifest": bool(vr.get("manifest")), "global": bool(vr.get("global")),
+                "inGame": bool(vr.get("in_game")), "bound": vr.get("bound", []), "active": vr.get("active", [])}
+
+    @Property(bool, notify=mappingsChanged)
+    def controllerInGames(self):
+        return bool(read_json(RULES_PATH).get("controller_in_games"))
+
+    @Slot(bool)
+    def setControllerInGames(self, on):
+        rules = read_json(RULES_PATH)
+        rules["controller_in_games"] = bool(on)
+        self._save_rules(rules)
+        self.message.emit("Mapped controller buttons: " + ("taken in games too" if on else "left to games"), False)
+
+    @Slot(str, str)
+    def setControllerMapping(self, button, action):
+        if button not in CONTROLLER_BUTTONS or action not in CONTROLLER_ACTIONS:
+            return
+        rules = read_json(RULES_PATH)
+        rules.setdefault("controller_buttons", {})[button] = action
+        self._save_rules(rules)
+        self.message.emit(f"{CONTROLLER_BUTTONS[button]} → {ACTION_LABELS[action]}", False)
+
+    @Slot(str)
+    def removeControllerMapping(self, button):
+        rules = read_json(RULES_PATH)
+        rules.get("controller_buttons", {}).pop(button, None)
+        self._save_rules(rules)
+        self.message.emit(f"{CONTROLLER_BUTTONS.get(button, button)}: back to games", False)
+
+    @Slot()
+    def clearControllerMappings(self):
+        rules = read_json(RULES_PATH)
+        removed = len(rules.pop("controller_buttons", {}) or {})
+        self._save_rules(rules)
+        self.message.emit(f"Removed {removed} controller binding{'s' if removed != 1 else ''}", False)
+
+    @Slot()
+    def startControllerCapture(self):
+        self._capture_vr = True
+        self._send("watch 60")
+        self._send("vrcapture 30")
+
+    @Slot()
+    def cancelControllerCapture(self):
+        if self._capture_vr:
+            self._capture_vr = False
+            self._send("vrcapture 0")
+
+    @Slot(bool)
+    def setGlobalInput(self, on):
+        """SteamVR's "Enable global input from overlays (Experimental)", which the helper needs
+        to get controller buttons while a game or the dashboard has focus."""
+        if self._send(f"vrglobal {'on' if on else 'off'}", HELPER):
+            self.message.emit(f"SteamVR global input from overlays {'on' if on else 'off'}", False)
+        else:
+            self.message.emit("The pointer helper isn't running (frametop-pointer.service)", True)
+
+    # --- gaze ---
+    def _gaze_status(self, status):
+        now = time.monotonic()
+        prev = self._gaze_prev
+        # Rates over the last poll: samples per second, and the share with only one eye.
+        if prev and now - prev[0] > 0.5 and status["samples"] >= prev[1]["samples"]:
+            n = status["samples"] - prev[1]["samples"]
+            status["rate"] = n / (now - prev[0])
+            status["one_eye_share"] = (status["one_eye"] - prev[1]["one_eye"]) / n if n else 0.0
+        elif self._gaze:
+            status.setdefault("rate", self._gaze.get("rate"))
+            status.setdefault("one_eye_share", self._gaze.get("one_eye_share"))
+        if not prev or now - prev[0] > 0.5:
+            self._gaze_prev = (now, status)
+        self._gaze = status
+        self.gazeChanged.emit()
+
+    @Property("QVariantMap", notify=gazeChanged)
+    def gazeStatus(self):
+        return self._gaze
+
+    @Property(bool, notify=gazeChanged)
+    def gazeServiceRunning(self):
+        return bool(self._gaze)
+
+    @Property(int, notify=gazeChanged)
+    def gazeMode(self):
+        """The helper's gaze mode now: 1 on, 0 off, -1 no answer (helper not running)."""
+        return -1 if self._gaze_mode is None else int(self._gaze_mode)
+
+    @Property(bool, notify=gazeChanged)
+    def gazeDefault(self):
+        return read_conf().get("POINTER_GAZE", "0") not in ("", "0")
+
+    @Slot(bool)
+    def setGazeMode(self, on):
+        """On or off now and from now on (POINTER_GAZE); a mapped button toggles it until restart."""
+        write_conf_value("POINTER_GAZE", "1" if on else "0")
+        self._send(f"gaze {'on' if on else 'off'}", HELPER)
+        self.reload_timer.start()
+        self.gazeChanged.emit()
+
+    @Property("QVariantList", notify=pointerChanged)
+    def gazeSettings(self):
+        conf = read_conf()
+        out = []
+        for key, label, default, lo, hi, step, unit in GAZE_SETTINGS:
+            try:
+                value = float(conf.get(key, default))
+            except ValueError:
+                value = default
+            out.append({"key": key, "label": label, "value": value, "min": lo, "max": hi, "step": step,
+                        "unit": unit, "default": default})
+        return out
+
+    @Slot()
+    def forgetGazeLessons(self):
+        if self._send("forget", GAZED):
+            self.message.emit("Forgot what the pointer's nudges taught; the calibration stays", False)
+        else:
+            self.message.emit("The gaze service isn't running (frametop-gaze.service)", True)
+
+    @Slot()
+    def reloadGazeCalibration(self):
+        if self._send("reload", GAZED):
+            self.message.emit("The gaze service read the calibration again", False)
+        else:
+            self.message.emit("The gaze service isn't running (frametop-gaze.service)", True)
+
+    @Slot()
+    def openGazeProbe(self):
+        """Calibrate in ft-gazeprobe (a GTK app on the host, fullscreen on a Frametop screen)."""
+        runner = ["distrobox-host-exec"] if shutil.which("distrobox-host-exec") else []
+        env = [f"{k}={os.environ[k]}" for k in ("WAYLAND_DISPLAY", "DISPLAY", "XAUTHORITY", "DBUS_SESSION_BUS_ADDRESS")
+               if os.environ.get(k)]
+        try:
+            subprocess.Popen(runner + ["env"] + env + [GAZE_PROBE], stdin=subprocess.DEVNULL,
+                             stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=True)
+            self.message.emit("Opening the gaze probe: calibrate there, then close it", False)
+        except OSError as e:
+            self.message.emit(f"Couldn't open the gaze probe: {e}", True)
+
     # --- bluetooth ---
     @Property("QVariantList", notify=bluetoothChanged)
     def bluetooth(self):

input-settings/main.qml

@@ -18,7 +18,9 @@ Kirigami.ApplicationWindow {
         actions: [
             Kirigami.Action { text: "Devices"; icon.name: "input-mouse"; onTriggered: root.show(devicesPage) },
             Kirigami.Action { text: "Buttons"; icon.name: "input-keyboard"; onTriggered: root.show(buttonsPage) },
+            Kirigami.Action { text: "Controllers"; icon.name: "input-gamepad"; onTriggered: root.show(controllersPage) },
             Kirigami.Action { text: "Pointer"; icon.name: "transform-move"; onTriggered: root.show(pointerPage) },
+            Kirigami.Action { text: "Gaze"; icon.name: "view-visible"; onTriggered: root.show(gazePage) },
             Kirigami.Action { text: "Bluetooth"; icon.name: "preferences-system-bluetooth"; onTriggered: root.show(bluetoothPage) }
         ]
     }
@@ -28,8 +30,9 @@ Kirigami.ApplicationWindow {
         pageStack.push(page)
     }
 
-    // FT_INPUT_PAGE=buttons|pointer|bluetooth opens the app on that page.
-    pageStack.initialPage: ({ buttons: buttonsPage, pointer: pointerPage, bluetooth: bluetoothPage })[startPage] || devicesPage
+    // FT_INPUT_PAGE=buttons|controllers|pointer|gaze|bluetooth opens the app on that page.
+    pageStack.initialPage: ({ buttons: buttonsPage, controllers: controllersPage, pointer: pointerPage, gaze: gazePage,
+                              bluetooth: bluetoothPage })[startPage] || devicesPage
 
     Connections {
         target: backend
@@ -294,7 +297,177 @@ Kirigami.ApplicationWindow {
                     wrapMode: Text.Wrap
                     opacity: 0.7
                     text: "Buttons without a mapping pass through (mouse buttons as clicks, keys as keys). "
-                          + "The Z3's extra buttons show up as keys from its keyboard node."
+                          + "The Z3's extra buttons show up as keys from its keyboard node. "
+                          + "The Frame controllers' buttons are on the Controllers page."
+                }
+            }
+        }
+    }
+
+    // ---------------------------------------------------------------- Controllers
+    Component {
+        id: controllersPage
+        Kirigami.ScrollablePage {
+            id: cpage
+            title: "Controllers"
+            actions: [
+                Kirigami.Action {
+                    text: "Remove all"
+                    icon.name: "edit-clear-all"
+                    tooltip: "Give every controller button back to games"
+                    enabled: backend.controllerMappings.length > 0
+                    onTriggered: backend.clearControllerMappings()
+                }
+            ]
+            property string capturedButton: ""
+            property string capturedLabel: ""
+            property bool capturing: false
+            property var status: backend.controllerStatus
+            Component.onDestruction: backend.cancelControllerCapture()
+
+            Connections {
+                target: backend
+                function onCapturedController(button, label) {
+                    cpage.capturedButton = button; cpage.capturedLabel = label; cpage.capturing = false
+                    buttonBox.currentIndex = buttonBox.indexOfValue(button)
+                }
+            }
+            Timer {
+                // The relay takes every button for 30 s while capturing.
+                running: cpage.capturing
+                interval: 30000
+                onTriggered: { backend.cancelControllerCapture(); cpage.capturing = false }
+            }
+
+            ColumnLayout {
+                spacing: Kirigami.Units.largeSpacing
+
+                Kirigami.InlineMessage {
+                    Layout.fillWidth: true
+                    visible: !cpage.status.helper
+                    type: Kirigami.MessageType.Error
+                    text: "The pointer helper isn't answering (frametop-pointer.service, needs SteamVR). "
+                          + "It reads the controller buttons."
+                }
+                Kirigami.InlineMessage {
+                    Layout.fillWidth: true
+                    visible: cpage.status.helper && !cpage.status.manifest
+                    type: Kirigami.MessageType.Error
+                    text: "The pointer helper couldn't set up SteamVR input (pointer/helper/actions). See its log."
+                }
+                Kirigami.InlineMessage {
+                    Layout.fillWidth: true
+                    visible: cpage.status.helper && !cpage.status.global
+                    type: Kirigami.MessageType.Warning
+                    text: "SteamVR's \"Enable global input from overlays (Experimental)\" is off. Without it the "
+                          + "buttons only reach Frametop when nothing else has focus, if at all."
+                    actions: [
+                        Kirigami.Action {
+                            text: "Turn it on"
+                            icon.name: "dialog-ok-apply"
+                            onTriggered: backend.setGlobalInput(true)
+                        }
+                    ]
+                }
+
+                Kirigami.FormLayout {
+                    Layout.fillWidth: true
+
+                    Controls.Switch {
+                        Kirigami.FormData.label: "In games:"
+                        text: "Mapped buttons work while a game is running too (the game loses them)"
+                        checked: backend.controllerInGames
+                        onToggled: backend.setControllerInGames(checked)
+                    }
+                    RowLayout {
+                        Kirigami.FormData.label: "New mapping:"
+                        Controls.Button {
+                            text: cpage.capturing ? "Press a button on a controller…" : "Capture a button"
+                            icon.name: "input-gamepad"
+                            highlighted: cpage.capturing
+                            enabled: cpage.status.helper
+                            onClicked: {
+                                if (cpage.capturing) { backend.cancelControllerCapture(); cpage.capturing = false }
+                                else { cpage.capturedButton = ""; cpage.capturing = true; backend.startControllerCapture() }
+                            }
+                        }
+                        Controls.Label { text: "or" ; opacity: 0.7 }
+                        Controls.ComboBox {
+                            id: buttonBox
+                            model: backend.controllerButtons
+                            textRole: "text"
+                            valueRole: "value"
+                            Layout.preferredWidth: Kirigami.Units.gridUnit * 10
+                        }
+                        Controls.ComboBox {
+                            id: newControllerAction
+                            model: backend.controllerActions
+                            textRole: "text"
+                            valueRole: "value"
+                            Layout.preferredWidth: Kirigami.Units.gridUnit * 13
+                        }
+                        Controls.Button {
+                            text: "Map"
+                            icon.name: "dialog-ok-apply"
+                            onClicked: { backend.setControllerMapping(buttonBox.currentValue, newControllerAction.currentValue); cpage.capturedButton = "" }
+                        }
+                    }
+                }
+
+                Kirigami.Heading {
+                    visible: backend.controllerMappings.length > 0
+                    level: 3
+                    text: "Current mappings"
+                }
+
+                Repeater {
+                    model: backend.controllerMappings
+                    delegate: RowLayout {
+                        id: crow
+                        required property var modelData
+                        Layout.fillWidth: true
+                        spacing: Kirigami.Units.largeSpacing
+                        Controls.Label {
+                            text: crow.modelData.label
+                            Layout.preferredWidth: Kirigami.Units.gridUnit * 10
+                        }
+                        Controls.ComboBox {
+                            model: backend.controllerActions
+                            textRole: "text"
+                            valueRole: "value"
+                            Layout.preferredWidth: Kirigami.Units.gridUnit * 13
+                            Component.onCompleted: currentIndex = indexOfValue(crow.modelData.action)
+                            onActivated: backend.setControllerMapping(crow.modelData.button, currentValue)
+                        }
+                        Controls.Label {
+                            // Active: SteamVR delivers it to Frametop now (a controller is on).
+                            text: cpage.status.active.indexOf(crow.modelData.button) >= 0 ? "active"
+                                  : cpage.status.inGame && !backend.controllerInGames ? "game's" : "waiting"
+                            opacity: 0.6
+                            Layout.preferredWidth: Kirigami.Units.gridUnit * 4
+                            Controls.ToolTip.text: text === "active" ? "SteamVR gives this button to Frametop"
+                                                   : text === "game's" ? "A game is running: the button is the game's until it quits"
+                                                   : "No controller with this button is on, or SteamVR doesn't give it to Frametop"
+                            Controls.ToolTip.visible: hover.hovered
+                            HoverHandler { id: hover }
+                        }
+                        Controls.Button {
+                            text: "Remove"
+                            icon.name: "edit-delete-remove"
+                            onClicked: backend.removeControllerMapping(crow.modelData.button)
+                        }
+                        Item { Layout.fillWidth: true }
+                    }
+                }
+
+                Controls.Label {
+                    Layout.fillWidth: true
+                    wrapMode: Text.Wrap
+                    opacity: 0.7
+                    text: "Outside games, a mapped button is Frametop's; while a game runs it's the game's, unless "
+                          + "\"In games\" is on. Unmapped buttons are left alone, and the system button stays SteamVR's. "
+                          + "Clicks and scrolling go to the 3D pointer. The trigger and grip also move SteamVR's laser "
+                          + "to that controller, so they're better left unmapped."
                 }
             }
         }
@@ -370,6 +543,142 @@ Kirigami.ApplicationWindow {
         }
     }
 
+    // ---------------------------------------------------------------- Gaze
+    Component {
+        id: gazePage
+        Kirigami.ScrollablePage {
+            id: gpage
+            title: "Gaze"
+            property var status: backend.gazeStatus
+            actions: [
+                Kirigami.Action {
+                    text: "Calibrate…"
+                    icon.name: "crosshairs"
+                    tooltip: "Open the gaze probe to calibrate (fullscreen on a Frametop screen)"
+                    onTriggered: backend.openGazeProbe()
+                },
+                Kirigami.Action {
+                    text: "Reload calibration"
+                    icon.name: "view-refresh"
+                    enabled: backend.gazeServiceRunning
+                    onTriggered: backend.reloadGazeCalibration()
+                },
+                Kirigami.Action {
+                    text: "Forget nudges"
+                    icon.name: "edit-clear-history"
+                    enabled: backend.gazeServiceRunning
+                    tooltip: "Drop what your mouse nudges taught; the calibration stays"
+                    onTriggered: backend.forgetGazeLessons()
+                }
+            ]
+
+            Kirigami.FormLayout {
+                Kirigami.InlineMessage {
+                    Layout.fillWidth: true
+                    Layout.maximumWidth: Kirigami.Units.gridUnit * 30
+                    visible: true
+                    type: Kirigami.MessageType.Warning
+                    text: "Gaze mode is experimental. The pointer goes where you look and the mouse does the last bit; "
+                          + "moving the mouse takes over, looking well away hands it back. A small nudge and a click "
+                          + "teach the gaze service where it was off."
+                }
+                Controls.Switch {
+                    Kirigami.FormData.label: "Gaze pointer:"
+                    text: backend.gazeMode < 0 ? "Pointer helper not running" : "Pointer goes where you look"
+                    enabled: backend.gazeMode >= 0
+                    checked: backend.gazeMode > 0
+                    onToggled: backend.setGazeMode(checked)
+                }
+                Controls.Label {
+                    visible: backend.gazeMode >= 0 && (backend.gazeMode > 0) !== backend.gazeDefault
+                    text: "Toggled by a button; it starts " + (backend.gazeDefault ? "on" : "off") + " after a restart."
+                    opacity: 0.7
+                    font: Kirigami.Theme.smallFont
+                }
+                Repeater {
+                    model: backend.gazeSettings
+                    delegate: RowLayout {
+                        required property var modelData
+                        Kirigami.FormData.label: modelData.label + ":"
+                        Controls.Slider {
+                            id: gslider
+                            from: modelData.min
+                            to: modelData.max
+                            stepSize: modelData.step
+                            value: modelData.value
+                            Layout.preferredWidth: Kirigami.Units.gridUnit * 14
+                            onMoved: backend.setPointerSetting(modelData.key, value)
+                        }
+                        Controls.Label {
+                            text: gslider.value.toFixed(modelData.step < 0.1 ? 2 : 1) + " " + modelData.unit
+                            Layout.preferredWidth: Kirigami.Units.gridUnit * 5
+                        }
+                        Controls.ToolButton {
+                            icon.name: "edit-undo"
+                            display: Controls.AbstractButton.IconOnly
+                            text: "Default (" + modelData.default + ")"
+                            Controls.ToolTip.text: text
+                            Controls.ToolTip.visible: hovered
+                            onClicked: { gslider.value = modelData.default; backend.setPointerSetting(modelData.key, modelData.default) }
+                        }
+                    }
+                }
+
+                Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Gaze service" }
+
+                Controls.Label {
+                    Kirigami.FormData.label: "Service:"
+                    text: backend.gazeServiceRunning
+                          ? (gpage.status.ft_gaze ? "running" : "running, eye tracker reader restarting")
+                          : "not running (frametop-gaze.service, starts with SteamVR)"
+                    color: backend.gazeServiceRunning ? Kirigami.Theme.textColor : Kirigami.Theme.negativeTextColor
+                }
+                Controls.Label {
+                    visible: backend.gazeServiceRunning
+                    Kirigami.FormData.label: "Headset:"
+                    text: gpage.status.headset_on ? "on" : "off"
+                }
+                Controls.Label {
+                    visible: backend.gazeServiceRunning
+                    Kirigami.FormData.label: "Tracking:"
+                    text: gpage.status.rate === undefined || gpage.status.rate === null ? "…"
+                          : Math.round(gpage.status.rate) + " samples/s"
+                            + (gpage.status.one_eye_share > 0.5 ? " · only one eye tracked" : "")
+                    color: gpage.status.one_eye_share > 0.5 ? Kirigami.Theme.neutralTextColor : Kirigami.Theme.textColor
+                    Controls.ToolTip.text: "Only one eye tracked: reseat the headset or check the lenses. It still works, "
+                                           + "probably less precisely."
+                    Controls.ToolTip.visible: gpage.status.one_eye_share > 0.5 && ghover.hovered
+                    HoverHandler { id: ghover }
+                }
+                Controls.Label {
+                    visible: backend.gazeServiceRunning
+                    Kirigami.FormData.label: "Calibration:"
+                    text: gpage.status.calibration_samples > 0
+                          ? gpage.status.calibration_samples + " points (" + gpage.status.model + ")"
+                          : "none yet: use Calibrate…"
+                }
+                Controls.Label {
+                    visible: backend.gazeServiceRunning
+                    Kirigami.FormData.label: "Learned from nudges:"
+                    text: gpage.status.lessons + (gpage.status.lessons === 1 ? " nudge" : " nudges")
+                          + (gpage.status.refused > 0 ? " (" + gpage.status.refused + " too far off, ignored)" : "")
+                }
+            }
+
+            footer: Controls.Label {
+                padding: Kirigami.Units.largeSpacing
+                wrapMode: Text.Wrap
+                opacity: 0.7
+                text: "Map a mouse button (Buttons) or a controller button (Controllers) to \"Gaze pointer on/off\" "
+                      + "to switch it on the fly. A click waits for the release: if the gaze is off, drag onto the target "
+                      + "with the button held and let go there. Hold still to drag: hold a press this long without moving "
+                      + "to drag something instead. Look away to hand back: how far from the pointer you look before the "
+                      + "gaze takes it back from the mouse. Largest nudge to learn: bigger mouse moves before a click "
+                      + "are treated as using the mouse, not correcting the gaze."
+            }
+        }
+    }
+
     // ---------------------------------------------------------------- Bluetooth
     Component {
         id: bluetoothPage

input/input-relay.py

@@ -24,6 +24,17 @@ pointer_toggle, follow_toggle = head follow on or off, gaze_toggle = gaze mode o
 layout_reset = put the desktop screens back in their saved layout, screens_toggle = hide or show the desktop screens, key = pass
 through as a key, none).
 
+Frame controller buttons can be mapped too ("controller_buttons": {"right/a": action} in the
+rules file; any action but key). The controllers aren't input devices here, only SteamVR sees
+them, so the pointer helper reads them with SteamVR input and sends "vrbtn <button> 1|0".
+It only takes the buttons the relay tells it to ("vrbind <button>..." to @ft_pointer_helper,
+sent on start, reload, and when the helper says "vrhello"), and only while no game runs,
+unless "controller_in_games" is true in the rules file (then a mapped button no longer
+reaches games; see pointer/helper/vrbuttons.h).
+
+In gaze mode outside games, the helper keeps the pointer ("gazeawake 1", repeated every 5
+seconds; "gazeawake 0" or silence ends it): the pointer isn't released when the mouse is idle.
+
 Keys also go to ft-screens (@ft_screens, the Frametop desktop's compositor), which
 types them into the desktop screen that has focus: from pass-through keyboards, and
 keys a pointer device passes through. Typing goes to the panel clicked last, and
@@ -53,6 +64,9 @@ Control socket (abstract datagram @frametop_relay, JSON replies to the sender):
   devices           list event nodes with id, name, kinds, role, grabbed
   watch <seconds>   stream input events from every candidate node (identification)
   reload            re-read both config files, re-apply roles, tell the helper
+  vrcapture <s>     take every controller button for s seconds (0: stop), so the settings
+                    app can capture one; watchers see them as events with id frame_controller
+  vrbtn, vrhello, gazeawake   from the pointer helper (above)
 
 Runs on the Frame host as a user service (frametop-input-relay.service). The
 virtual devices are parked in systemd's file descriptor store, so a relay
@@ -143,6 +157,12 @@ RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
 ACTIONS = ("left", "right", "middle", "back", "scroll_up", "scroll_down", "dashboard", "recenter",
            "pointer_toggle", "follow_toggle", "gaze_toggle", "sens_up", "sens_down", "layout_reset", "screens_toggle",
            "key", "none")
+HELPER = "\0ft_pointer_helper"
+# Frame controller buttons the pointer helper can read (pointer/helper/vrbuttons.h).
+VR_BUTTONS = ("left/view", "left/dpad_up", "left/dpad_down", "left/dpad_left", "left/dpad_right", "left/bumper",
+              "left/trigger", "left/grip", "left/thumbstick", "right/menu", "right/a", "right/b", "right/x", "right/y",
+              "right/bumper", "right/trigger", "right/grip", "right/thumbstick")
+VR_DEVICE = "frame_controller"  # the id controller buttons have in watch events
 SCREENS = "\0ft_screens"
 KEYS = "\0frametop_keys"  # keys of keyboards grabbed for the desktop, for other readers
 FT_LAYOUT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "layout", "ft-layout")
@@ -239,7 +259,8 @@ def read_config(path=os.path.expanduser("~/.config/frametop.conf")):
 
 
 def read_rules(path=RULES_PATH):
-    """{"devices": {id: {"role", "name"}}, "buttons": {id: {"<code>": action}}}."""
+    """{"devices": {id: {"role", "name"}}, "buttons": {id: {"<code>": action}},
+    "controller_buttons": {"<hand>/<button>": action}}."""
     try:
         with open(path) as f:
             rules = json.load(f)
@@ -247,6 +268,7 @@ def read_rules(path=RULES_PATH):
         rules = {}
     rules.setdefault("devices", {})
     rules.setdefault("buttons", {})
+    rules.setdefault("controller_buttons", {})
     return rules
 
 
@@ -345,10 +367,11 @@ class Pointer:
         self.wake_counts = wake_counts
         self.pending = 0
         self.pending_since = 0.0
+        self.gaze_awake_until = 0.0  # the helper's gaze mode keeps the pointer until then
 
     def send(self, command):
         try:
-            self.sock.sendto(command.encode(), "\0ft_pointer_helper")
+            self.sock.sendto(command.encode(), HELPER)
         except OSError:
             pass  # helper not running (SteamVR not running)
 
@@ -464,7 +487,7 @@ class Pointer:
         if self.scroll_until is not None and now >= self.scroll_until:
             self.send("scroll 0 0")
             self.scroll_until = None
-        if self.active and now - self.last_used > self.idle:
+        if self.active and now - self.last_used > self.idle and now >= self.gaze_awake_until:
             self.send("hide")
             self.active = False
             log("pointer off (idle)")
@@ -552,8 +575,9 @@ def main():
 
     # desktop_until: typing goes to the Frametop desktop until then (ft-screens says so
     # every second); typing_applied: the grabs match that as of the last apply_roles().
+    # vr_capture_until: every controller button is taken until then (the settings app capturing one).
     state = {"pointer": None, "rules": {}, "meta_dashboard": False, "share_keys": False,
-             "desktop_until": 0.0, "typing_applied": None}
+             "desktop_until": 0.0, "typing_applied": None, "vr_capture_until": 0.0}
 
     def load_config():
         conf = read_config()
@@ -575,6 +599,34 @@ def main():
     meta_down = False  # Meta pressed with no other key yet: a tap toggles the dashboard
     screens_sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_NONBLOCK)
 
+    def vr_bind(now):
+        """Tell the pointer helper which controller buttons to take from games."""
+        if now < state["vr_capture_until"]:
+            buttons = "*"
+        else:
+            state["vr_capture_until"] = 0.0
+            buttons = " ".join(b for b, a in state["rules"]["controller_buttons"].items()
+                               if b in VR_BUTTONS and a in ACTIONS and a not in ("key", "none")) or "-"
+            if state["rules"].get("controller_in_games"):
+                buttons = "+games " + buttons
+        try:
+            screens_sock.sendto(f"vrbind {buttons}".encode(), HELPER)
+        except OSError:
+            pass  # helper not running; it says vrhello when it starts
+
+    def vr_button(button, value, now):
+        """A Frame controller button from the pointer helper."""
+        for addr, until in list(watchers.items()):
+            if now > until:
+                del watchers[addr]
+            else:
+                reply(addr, {"t": "event", "id": VR_DEVICE, "path": "", "name": "Steam Frame controllers",
+                             "type": "vr", "code": button, "value": value})
+        action = state["rules"]["controller_buttons"].get(button)
+        if state["pointer"] and action in ACTIONS and action not in ("key", "none"):
+            state["pointer"].action(action, value, now)
+
+
     def to_screens(code, value):
         """A key for the desktop screens (ft-screens decides whether it types)."""
         if value in (0, 1) and code < BTN_MISC:
@@ -731,6 +783,16 @@ def main():
                 desktop = len(words) > 1 and words[1] == "desktop"
                 state["desktop_until"] = now + 3.0 if desktop else 0.0
                 continue
+            if cmd == "vrbtn" and len(words) == 3 and words[1] in VR_BUTTONS and words[2] in ("0", "1"):
+                vr_button(words[1], int(words[2]), now)
+                continue
+            if cmd == "vrhello":
+                vr_bind(now)
+                continue
+            if cmd == "gazeawake" and len(words) == 2:
+                if state["pointer"]:
+                    state["pointer"].gaze_awake_until = now + 12.0 if words[1] == "1" else 0.0
+                continue
             if not addr:
                 continue  # unbound sender, nowhere to reply
             if cmd == "devices":
@@ -746,7 +808,13 @@ def main():
                 apply_roles()
                 if state["pointer"]:
                     state["pointer"].send("reload")
+                vr_bind(now)
                 reply(addr, {"t": "reloaded"})
+            elif cmd == "vrcapture":
+                seconds = float(words[1]) if len(words) > 1 else 30
+                state["vr_capture_until"] = now + min(seconds, 120) if seconds > 0 else 0.0
+                vr_bind(now)
+                reply(addr, {"t": "vrcapture", "seconds": seconds})
             else:
                 reply(addr, {"t": "error", "error": f"unknown command {cmd!r}"})
 
@@ -764,6 +832,7 @@ def main():
             else:
                 reply(addr, msg)
 
+    vr_bind(time.monotonic())  # a helper that's already running keeps its buttons in step
     waiting = False  # a keyboard's grab waits for its keys to come up
     while True:
         now = time.monotonic()
@@ -803,6 +872,8 @@ def main():
         if pointer:
             pointer.tick(now)
         volume.tick(now)
+        if state["vr_capture_until"] and now >= state["vr_capture_until"]:
+            vr_bind(now)  # capture over: back to the mapped buttons
         if (now < state["desktop_until"]) != state["typing_applied"] or waiting:
             waiting = apply_roles()
         for fd in ready:

pointer/helper/actions/bindings_frame_controller.json

@@ -0,0 +1,242 @@
+{
+  "action_manifest_version": 0,
+  "controller_type": "frame_controller",
+  "description": "Frametop: each button in an action set of its own, so only mapped ones are taken",
+  "name": "Frametop pointer",
+  "bindings": {
+    "/actions/left_view": {
+      "sources": [
+        {
+          "path": "/user/hand/left/input/view",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/left_view/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/left_dpad_up": {
+      "sources": [
+        {
+          "path": "/user/hand/left/input/dpad_up",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/left_dpad_up/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/left_dpad_down": {
+      "sources": [
+        {
+          "path": "/user/hand/left/input/dpad_down",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/left_dpad_down/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/left_dpad_left": {
+      "sources": [
+        {
+          "path": "/user/hand/left/input/dpad_left",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/left_dpad_left/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/left_dpad_right": {
+      "sources": [
+        {
+          "path": "/user/hand/left/input/dpad_right",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/left_dpad_right/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/left_bumper": {
+      "sources": [
+        {
+          "path": "/user/hand/left/input/bumper",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/left_bumper/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/left_trigger": {
+      "sources": [
+        {
+          "path": "/user/hand/left/input/trigger",
+          "mode": "trigger",
+          "inputs": {
+            "click": {
+              "output": "/actions/left_trigger/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/left_grip": {
+      "sources": [
+        {
+          "path": "/user/hand/left/input/grip",
+          "mode": "trigger",
+          "inputs": {
+            "click": {
+              "output": "/actions/left_grip/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/left_thumbstick": {
+      "sources": [
+        {
+          "path": "/user/hand/left/input/thumbstick",
+          "mode": "joystick",
+          "inputs": {
+            "click": {
+              "output": "/actions/left_thumbstick/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/right_menu": {
+      "sources": [
+        {
+          "path": "/user/hand/right/input/menu",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/right_menu/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/right_a": {
+      "sources": [
+        {
+          "path": "/user/hand/right/input/a",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/right_a/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/right_b": {
+      "sources": [
+        {
+          "path": "/user/hand/right/input/b",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/right_b/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/right_x": {
+      "sources": [
+        {
+          "path": "/user/hand/right/input/x",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/right_x/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/right_y": {
+      "sources": [
+        {
+          "path": "/user/hand/right/input/y",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/right_y/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/right_bumper": {
+      "sources": [
+        {
+          "path": "/user/hand/right/input/bumper",
+          "mode": "button",
+          "inputs": {
+            "click": {
+              "output": "/actions/right_bumper/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/right_trigger": {
+      "sources": [
+        {
+          "path": "/user/hand/right/input/trigger",
+          "mode": "trigger",
+          "inputs": {
+            "click": {
+              "output": "/actions/right_trigger/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/right_grip": {
+      "sources": [
+        {
+          "path": "/user/hand/right/input/grip",
+          "mode": "trigger",
+          "inputs": {
+            "click": {
+              "output": "/actions/right_grip/in/press"
+            }
+          }
+        }
+      ]
+    },
+    "/actions/right_thumbstick": {
+      "sources": [
+        {
+          "path": "/user/hand/right/input/thumbstick",
+          "mode": "joystick",
+          "inputs": {
+            "click": {
+              "output": "/actions/right_thumbstick/in/press"
+            }
+          }
+        }
+      ]
+    }
+  }
+}

pointer/helper/actions/ft_pointer_actions.json

@@ -0,0 +1,197 @@
+{
+  "default_bindings": [
+    {
+      "controller_type": "frame_controller",
+      "binding_url": "bindings_frame_controller.json"
+    }
+  ],
+  "action_sets": [
+    {
+      "name": "/actions/left_view",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/left_dpad_up",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/left_dpad_down",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/left_dpad_left",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/left_dpad_right",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/left_bumper",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/left_trigger",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/left_grip",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/left_thumbstick",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/right_menu",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/right_a",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/right_b",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/right_x",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/right_y",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/right_bumper",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/right_trigger",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/right_grip",
+      "usage": "hidden"
+    },
+    {
+      "name": "/actions/right_thumbstick",
+      "usage": "hidden"
+    }
+  ],
+  "actions": [
+    {
+      "name": "/actions/left_view/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/left_dpad_up/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/left_dpad_down/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/left_dpad_left/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/left_dpad_right/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/left_bumper/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/left_trigger/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/left_grip/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/left_thumbstick/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/right_menu/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/right_a/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/right_b/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/right_x/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/right_y/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/right_bumper/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/right_trigger/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/right_grip/in/press",
+      "type": "boolean"
+    },
+    {
+      "name": "/actions/right_thumbstick/in/press",
+      "type": "boolean"
+    }
+  ],
+  "localization": [
+    {
+      "language_tag": "en_US",
+      "/actions/left_view": "Frametop: left view",
+      "/actions/left_view/in/press": "Press",
+      "/actions/left_dpad_up": "Frametop: left dpad up",
+      "/actions/left_dpad_up/in/press": "Press",
+      "/actions/left_dpad_down": "Frametop: left dpad down",
+      "/actions/left_dpad_down/in/press": "Press",
+      "/actions/left_dpad_left": "Frametop: left dpad left",
+      "/actions/left_dpad_left/in/press": "Press",
+      "/actions/left_dpad_right": "Frametop: left dpad right",
+      "/actions/left_dpad_right/in/press": "Press",
+      "/actions/left_bumper": "Frametop: left bumper",
+      "/actions/left_bumper/in/press": "Press",
+      "/actions/left_trigger": "Frametop: left trigger",
+      "/actions/left_trigger/in/press": "Press",
+      "/actions/left_grip": "Frametop: left grip",
+      "/actions/left_grip/in/press": "Press",
+      "/actions/left_thumbstick": "Frametop: left thumbstick",
+      "/actions/left_thumbstick/in/press": "Press",
+      "/actions/right_menu": "Frametop: right menu",
+      "/actions/right_menu/in/press": "Press",
+      "/actions/right_a": "Frametop: right a",
+      "/actions/right_a/in/press": "Press",
+      "/actions/right_b": "Frametop: right b",
+      "/actions/right_b/in/press": "Press",
+      "/actions/right_x": "Frametop: right x",
+      "/actions/right_x/in/press": "Press",
+      "/actions/right_y": "Frametop: right y",
+      "/actions/right_y/in/press": "Press",
+      "/actions/right_bumper": "Frametop: right bumper",
+      "/actions/right_bumper/in/press": "Press",
+      "/actions/right_trigger": "Frametop: right trigger",
+      "/actions/right_trigger/in/press": "Press",
+      "/actions/right_grip": "Frametop: right grip",
+      "/actions/right_grip/in/press": "Press",
+      "/actions/right_thumbstick": "Frametop: right thumbstick",
+      "/actions/right_thumbstick/in/press": "Press"
+    }
+  ]
+}

pointer/helper/ft-pointer.cpp

@@ -114,14 +114,28 @@
 // that gaze from the eye: nothing is steered, so nothing can pile up. Moving the mouse takes
 // the pointer from the gaze, and it moves from where the gaze left it, as usual. Looking
 // well away from it (more than POINTER_GAZE_RETAKE, 5 deg, for 120 ms, with the mouse still
-// for 300 ms) gives it back to the gaze; small eye movements around the pointer don't. A
-// click or a drag holds the pointer where it is, like any other press.
+// for 300 ms) gives it back to the gaze; small eye movements around the pointer don't.
+//   A left press while the gaze has the pointer isn't sent yet: the pointer stops where the
+// gaze put it, and if the gaze is off, you drag it onto what you meant with the mouse
+// (still holding the button; panels only see it hover). The release clicks there, a press
+// and a release 40 ms apart. Held still for POINTER_GAZE_HOLD (0.5 s) instead, it becomes
+// a real press where the pointer is, so drags work: hold, then move. After a click that
+// didn't need correcting, and after a drag, the gaze has the pointer again.
+//   Outside games (no scene application), gaze mode keeps the pointer: the relay doesn't
+// release it when the mouse is idle ("gazeawake 1|0" tells it). A controller that moves
+// still releases it (the mouse is gaze mode's only pointer device), and in games the mouse
+// wakes it and idling releases it, as without gaze.
+//   The dot only shows while the mouse moves it (within POINTER_GAZE_SHOW, 1 s), while a
+// press is held, and briefly for each click (a pulse);
+// otherwise it's transparent (still there for the laser to land on). The gaze moving it
+// doesn't show it: you know where you're looking.
 //   When the mouse took the pointer and you then click, the nudge was probably onto what
 // you were looking at: from the raw gaze when the mouse took over to where you clicked is
 // the tracker's error there. The helper sends it to ft-gazed as a lesson ("lesson <raw yaw>
 // <raw pitch> <true yaw> <true pitch>", the true direction relative to the head as it was
 // when the mouse took over) if the mouse moved between 0.2 deg and POINTER_GAZE_NUDGE_MAX
-// (8 deg) and the click came within 10 s; more is using the mouse, not a nudge. With no
+// (8 deg) and the click came within 10 s; more is using the mouse, not a nudge. A held
+// press dragged onto the target is the same: from the raw gaze at the press to the release. With no
 // fresh gaze (a blink, the service stopped, the headset off), the pointer stays put.
 //
 // Placement (for layout): SteamVR keeps a floating panel's position inside the
@@ -139,8 +153,9 @@
 //
 // Commands (datagrams on @ft_pointer_helper): show, hide, recenter, move <dyaw> <dpitch>,
 // follow on|off|toggle (head follow, until the next restart or a change to POINTER_FOLLOW),
-// gaze on|off|toggle (gaze mode, likewise with POINTER_GAZE), gz ... (the gaze, from ft-gazed),
+// gaze on|off|toggle|? (gaze mode, likewise with POINTER_GAZE; ? only asks), gz ... (the gaze, from ft-gazed),
 // reload (re-read the settings below), debug (toggle a twice-a-second state log),
+// vrbind/vrglobal/vrstatus (Frame controller buttons, see vrbuttons.h),
 // and btn/scroll lines, which are forwarded to the driver unchanged. For layouts, with a
 // reply datagram to the sender's (abstract) address:
 //   place <overlay> <x> <y> <z> <yaw> <pitch> [roll [grab]]: centre in the standing
@@ -160,9 +175,11 @@
 // panel, and a laser that starts behind them can't hit them. POINTER_FOLLOW (0) and
 // POINTER_LEASH_DEG (10), POINTER_LEASH_DELAY (0.2 s), POINTER_LEASH_RETURN (0.2 s),
 // POINTER_FOLLOW_REACH (70 deg): head follow, above. POINTER_GAZE (0), POINTER_GAZE_RETAKE
-// (5 deg), POINTER_GAZE_NUDGE_MAX (8 deg): gaze mode, above.
+// (5 deg), POINTER_GAZE_NUDGE_MAX (8 deg), POINTER_GAZE_HOLD (0.5 s), POINTER_GAZE_SHOW (1 s):
+// gaze mode, above.
 #include <openvr.h>
 
+#include "vrbuttons.h"
 #include "vrmath.h"
 
 #include <algorithm>
@@ -180,6 +197,8 @@
 #include <tuple>
 #include <vector>
 
+#include <climits>
+
 #include <sys/socket.h>
 #include <sys/un.h>
 #include <unistd.h>
@@ -227,6 +246,15 @@ int AbstractSocket(const char *name, bool bindIt) {
     return fd;
 }
 
+std::string ExeDir() {
+    char buf[PATH_MAX];
+    const ssize_t n = readlink("/proc/self/exe", buf, sizeof buf - 1);
+    if (n <= 0) return ".";
+    buf[n] = 0;
+    std::string p(buf);
+    return p.substr(0, p.rfind('/'));
+}
+
 void SendTo(int fd, const char *name, const std::string &msg) {
     sockaddr_un addr{};
     addr.sun_family = AF_UNIX;
@@ -361,7 +389,7 @@ int main() {
     bool follow = false, followConf = false, followReset = true;
     // Gaze mode (see the top); gazeConf is POINTER_GAZE as last read, like followConf.
     bool gazeOn = false, gazeConf = false;
-    double gazeRetake = 5, gazeNudgeMax = 8;
+    double gazeRetake = 5, gazeNudgeMax = 8, gazeHold = 0.5, gazeShow = 1;
     auto loadConfig = [&] {
         const auto conf = ReadConfig();
         freeDistance = std::clamp(ConfDouble(conf, "POINTER_DISTANCE", 1.5), 0.3, 10.0);
@@ -380,6 +408,8 @@ int main() {
         if (wantFollow != followConf) follow = followConf = wantFollow, followReset = true;
         gazeRetake = std::clamp(ConfDouble(conf, "POINTER_GAZE_RETAKE", 5), 1.0, 45.0);
         gazeNudgeMax = std::clamp(ConfDouble(conf, "POINTER_GAZE_NUDGE_MAX", 8), 1.0, 30.0);
+        gazeHold = std::clamp(ConfDouble(conf, "POINTER_GAZE_HOLD", 0.5), 0.1, 5.0);
+        gazeShow = std::clamp(ConfDouble(conf, "POINTER_GAZE_SHOW", 1), 0.0, 30.0);
         const bool wantGaze = ConfDouble(conf, "POINTER_GAZE", 0) != 0;
         if (wantGaze != gazeConf) gazeOn = gazeConf = wantGaze;
     };
@@ -427,6 +457,14 @@ int main() {
 
     const int in = AbstractSocket("ft_pointer_helper", true);
     const int out = AbstractSocket(nullptr, false);
+    // Frame controller buttons (vrbuttons.h). The build puts the binary in pointer/helper/build.
+    ControllerButtons controllerButtons;
+    {
+        const std::string manifest = ExeDir() + "/../actions/ft_pointer_actions.json";
+        char real[PATH_MAX];
+        controllerButtons.Init(realpath(manifest.c_str(), real) ? real : manifest);
+    }
+    SendTo(out, "frametop_relay", "vrhello");  // the relay answers with the mapped buttons
     OverlayList overlays;
     overlays.Start();
     std::map<std::string, vr::VROverlayHandle_t> handles;
@@ -481,7 +519,72 @@ int main() {
     vr::HmdMatrix34_t nudgeHead{}, lastHead{};
     bool haveHead = false, havePoint = false;
     Clock::time_point nudgeAt{}, retakeSince{};
+    // A held-back press (see the top): aimHeld while the button is down; then the click
+    // (clickPress at the end of the next frame, clickRelease 40 ms later). gazeBack: give
+    // the gaze the pointer again when the press or click is over.
     vr::TrackedDeviceIndex_t ours = vr::k_unTrackedDeviceIndexInvalid;
+    bool aimHeld = false, clickPress = false, clickRelease = false, gazeBack = false;
+    Clock::time_point aimSince{}, clickReleaseAt{};
+    // Gaze mode outside games, and its dot (see the top): lastMove/lastHeld/pulseAt.
+    bool inGame = false, gazeAwake = false;
+    Clock::time_point inGameAt{}, gazeAwakeAt{};
+    Clock::time_point lastMove{}, lastHeld{}, pulseAt{};
+    // The left button, as sent to the driver.
+    auto pressLeft = [&] {
+        // ft-screens sends the keyboard to the panel clicked last; it sees clicks on
+        // its own screens, but only we know when one lands on another panel.
+        SendTo(out, "ft_screens", "click " + (lastHit.empty() ? std::string("-") : lastHit));
+        // A click after nudging the gaze-placed pointer: the nudge is a lesson (see the top).
+        if (gazeOn && nudging && !gazeOwns && havePoint && Clock::now() - nudgeAt < std::chrono::seconds(10) &&
+            nudgeMoved >= 0.2 && nudgeMoved <= gazeNudgeMax) {
+            const Vec3 d = RotateInverse(nudgeHead, Normalize(lastPoint - Position(nudgeHead)));
+            char msg[160];
+            std::snprintf(msg, sizeof msg, "lesson %.3f %.3f %.3f %.3f", nudgeRawHy, nudgeRawHp,
+                          std::atan2(-d.x, -d.z) * 180 / M_PI, std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI);
+            SendTo(out, "ft_gazed", msg);
+            if (debug) std::printf("gaze %s (nudged %.2f deg)\n", msg, nudgeMoved);
+            if (debug) std::fflush(stdout);
+        }
+        nudging = false;
+        leftHeld = true;
+        dragDistance = lastDistance;
+        tiltYaw = tiltPitch = 0;  // a new drag starts untilted
+        dropHoldUntil = {};
+        pulseAt = Clock::now();
+        SendTo(out, "ft_pointer", "btn trigger 1");
+    };
+    auto releaseLeft = [&] {
+        leftHeld = false;
+        tilting = false;
+        // Hold the drag pose (tilt, frozen distance) while SteamVR finishes the drop.
+        dropHoldUntil = Clock::now() + std::chrono::milliseconds(500);
+        SendTo(out, "ft_pointer", "btn trigger 0");
+        if (gazeBack) gazeOwns = true, gazeBack = false;
+    };
+    // The left button, from the relay's "btn trigger", with gaze mode's held-back press (see
+    // the top).
+    auto leftButton = [&](bool down) {
+        if (down) {
+            if (gazeOn && gazeOwns && !aimHeld && !clickPress && !clickRelease) {
+                // Hold the press back: the pointer stops where the gaze put it.
+                gazeOwns = false;
+                nudging = haveHead && Clock::now() - gz.at < std::chrono::milliseconds(200);
+                nudgeRawHy = gz.rhy, nudgeRawHp = gz.rhp, nudgeHead = lastHead;
+                nudgeAt = aimSince = Clock::now(), nudgeMoved = 0;
+                aimHeld = true;
+                return;
+            }
+            pressLeft();
+            return;
+        }
+        if (aimHeld) {
+            aimHeld = false;
+            clickPress = true;  // after this frame's pose, so it lands where the pointer was moved to
+            gazeBack = nudgeMoved < 0.2;
+            return;
+        }
+        if (leftHeld) releaseLeft();
+    };
     auto lastSlow = std::chrono::steady_clock::now() - std::chrono::seconds(10);
 
     // --- Panel placement (see "Placement" at the top of the file) ---
@@ -659,6 +762,23 @@ int main() {
             std::fflush(stdout);
         }
 
+        // Outside games, gaze mode keeps the pointer (see the top); the relay needs to know.
+        {
+            const auto t = Clock::now();
+            if (t - inGameAt > std::chrono::milliseconds(500)) {
+                inGameAt = t;
+                inGame = vr::VRApplications()->GetCurrentSceneProcessId() != 0;
+            }
+            const bool awake = gazeOn && !inGame && !headsetOff;
+            if (awake != gazeAwake || t - gazeAwakeAt > std::chrono::seconds(5)) {
+                if (awake != gazeAwake) std::printf("gaze keeps the pointer: %s\n", awake ? "yes" : "no (off, in a game, or headset off)");
+                if (awake != gazeAwake) std::fflush(stdout);
+                gazeAwake = awake;
+                gazeAwakeAt = t;
+                SendTo(out, "frametop_relay", awake ? "gazeawake 1" : "gazeawake 0");
+            }
+        }
+
         // Commands from the relay.
         char buf[256];
         ssize_t n;
@@ -680,13 +800,33 @@ int main() {
                 gz = {g[0], g[1], g[2], g[3], Clock::now()};
                 continue;
             }
+            if (std::strncmp(buf, "vrbind", 6) == 0) {
+                std::printf("controller buttons: %s\n", controllerButtons.Bind(buf + 6).c_str());
+                std::fflush(stdout);
+                continue;
+            }
+            if (std::strncmp(buf, "vrglobal", 8) == 0) {
+                const char *arg = buf + 8;
+                while (*arg == ' ') ++arg;
+                ControllerButtons::SetGlobal(std::strncmp(arg, "off", 3) != 0);
+                reply(controllerButtons.Status());
+                continue;
+            }
+            if (std::strncmp(buf, "vrstatus", 8) == 0) {
+                reply(controllerButtons.Status());
+                continue;
+            }
             if (std::strncmp(buf, "gaze", 4) == 0) {
                 const char *arg = buf + 4;
                 while (*arg == ' ') ++arg;
-                gazeOn = std::strncmp(arg, "on", 2) == 0 ? true : std::strncmp(arg, "off", 3) == 0 ? false : !gazeOn;
-                gazeOwns = true, nudging = false;
-                std::printf("gaze mode %s\n", gazeOn ? "on" : "off");
-                std::fflush(stdout);
+                if (*arg != '?') {  // "gaze ?" only asks
+                    gazeOn = std::strncmp(arg, "on", 2) == 0    ? true
+                             : std::strncmp(arg, "off", 3) == 0 ? false
+                                                                : !gazeOn;
+                    gazeOwns = true, nudging = false;
+                    std::printf("gaze mode %s\n", gazeOn ? "on" : "off");
+                    std::fflush(stdout);
+                }
                 reply(gazeOn ? "ok on" : "ok off");
                 continue;
             }
@@ -738,30 +878,11 @@ int main() {
                 continue;
             }
             if (std::strncmp(buf, "btn trigger 1", 13) == 0) {
-                // ft-screens sends the keyboard to the panel clicked last; it sees clicks on
-                // its own screens, but only we know when one lands on another panel.
-                SendTo(out, "ft_screens", "click " + (lastHit.empty() ? std::string("-") : lastHit));
-                // A click after nudging the gaze-placed pointer: the nudge is a lesson (see the top).
-                if (gazeOn && nudging && !gazeOwns && havePoint && Clock::now() - nudgeAt < std::chrono::seconds(10) &&
-                    nudgeMoved >= 0.2 && nudgeMoved <= gazeNudgeMax) {
-                    const Vec3 d = RotateInverse(nudgeHead, Normalize(lastPoint - Position(nudgeHead)));
-                    char msg[160];
-                    std::snprintf(msg, sizeof msg, "lesson %.3f %.3f %.3f %.3f", nudgeRawHy, nudgeRawHp,
-                                  std::atan2(-d.x, -d.z) * 180 / M_PI, std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI);
-                    SendTo(out, "ft_gazed", msg);
-                    if (debug) std::printf("gaze %s (nudged %.2f deg)\n", msg, nudgeMoved);
-                    if (debug) std::fflush(stdout);
-                }
-                nudging = false;
-                leftHeld = true;
-                dragDistance = lastDistance;
-                tiltYaw = tiltPitch = 0;  // a new drag starts untilted
-                dropHoldUntil = {};
+                leftButton(true);
+                continue;
             } else if (std::strncmp(buf, "btn trigger 0", 13) == 0) {
-                leftHeld = false;
-                tilting = false;
-                // Hold the drag pose (tilt, frozen distance) while SteamVR finishes the drop.
-                dropHoldUntil = Clock::now() + std::chrono::milliseconds(500);
+                leftButton(false);
+                continue;
             } else if (std::strncmp(buf, "btn b 1", 7) == 0 && leftHeld) {
                 tilting = tiltStart = swallowedRight = true;  // right press while dragging: tilt, no right-click
                 continue;
@@ -775,6 +896,7 @@ int main() {
                 continue;
             }
             if (std::sscanf(buf, "move %lf %lf", &a, &b) == 2) {
+                lastMove = Clock::now();  // the dot shows while the mouse moves it (gaze mode)
                 if (gazeOn && gazeOwns) {
                     // The mouse takes the pointer from the gaze, from where the gaze left it.
                     gazeOwns = false;
@@ -782,7 +904,7 @@ int main() {
                     nudgeRawHy = gz.rhy, nudgeRawHp = gz.rhp, nudgeHead = lastHead;
                     nudgeAt = Clock::now(), nudgeMoved = 0;
                 }
-                if (nudging) nudgeMoved += std::hypot(a, b);
+                if (nudging || aimHeld) nudgeMoved += std::hypot(a, b);
                 if (!anchored) recenter = true;
                 yaw += a;
                 while (yaw > 180) yaw -= 360;
@@ -896,7 +1018,8 @@ int main() {
         // Gaze mode (see the top): the gaze has the pointer, or takes it back when you look
         // well away from it. Not while a press holds the pointer, and only on fresh gaze.
         if (hmd.bPoseIsValid) lastHead = hmd.mDeviceToAbsoluteTracking, haveHead = true;
-        if (gazeOn && active && hmd.bPoseIsValid && !tilting && !leftHeld && tnow >= dropHoldUntil &&
+        if (gazeOn && active && hmd.bPoseIsValid && !tilting && !leftHeld && !aimHeld && !clickPress && !clickRelease &&
+            tnow >= dropHoldUntil &&
             tnow - gz.at < std::chrono::milliseconds(150)) {
             const Vec3 g = Rotate(hmd.mDeviceToAbsoluteTracking, Direction(gz.hy, gz.hp));
             if (!gazeOwns && havePoint) {
@@ -917,6 +1040,8 @@ int main() {
             }
         }
 
+        if (aimHeld || leftHeld || clickPress || clickRelease) lastHeld = tnow;
+
         // Head follow (see the top): past the leash (for the delay), ease the reference to the
         // head's facing, and turn the cursor with it. Not in gaze mode: the gaze places it.
         if (follow && !gazeOn && active && anchored && hmd.bPoseIsValid) {
@@ -1117,7 +1242,20 @@ int main() {
                 const vr::VROverlayHandle_t show = onPanel ? marker : cursor, hide = onPanel ? cursor : marker;
                 const Vec3 at = onPanel ? point + Normalize(eye - point) * 0.005 : onScene ? point + dir * 0.05 : point;
                 const double dist = std::sqrt(Dot(at - eye, at - eye));
-                overlay->SetOverlayWidthInMeters(show, float(2 * dist * std::tan(cursorDeg * M_PI / 360)));
+                // Gaze mode: shown only while something moves it or a press holds it, and a pulse
+                // for each click (see the top); transparent otherwise, the laser still lands on it.
+                double scale = 1, alpha = 1;
+                if (gazeOn) {
+                    auto secs = [&](Clock::time_point t) { return std::chrono::duration<double>(tnow - t).count(); };
+                    alpha = std::clamp(1 - std::min(secs(lastMove) - gazeShow, secs(lastHeld)) / 0.25, 0.0, 1.0);
+                    const double pulse = secs(pulseAt);
+                    if (pulse < 0.6) {
+                        scale = 1 + 1.5 * std::max(0.0, 1 - pulse / 0.3);
+                        alpha = std::max(alpha, std::clamp((0.6 - pulse) / 0.3, 0.0, 1.0));
+                    }
+                }
+                overlay->SetOverlayAlpha(show, float(alpha));
+                overlay->SetOverlayWidthInMeters(show, float(2 * dist * std::tan(scale * cursorDeg * M_PI / 360)));
                 auto m = Billboard(at, eye);
                 overlay->SetOverlayTransformAbsolute(show, vr::TrackingUniverseStanding, &m);
                 overlay->ShowOverlay(show);
@@ -1172,6 +1310,30 @@ int main() {
             }
         }
 
+        // A held-back press (see the top): held still long enough, it's a real press (a drag);
+        // released, it's a click where the pointer is now (this frame's pose has gone out).
+        if (!active) aimHeld = clickPress = false;  // released meanwhile: nothing to click
+        if (aimHeld && nudgeMoved < 0.2 && tnow - aimSince >= std::chrono::duration<double>(gazeHold)) {
+            aimHeld = false;
+            gazeBack = true;
+            pressLeft();
+        }
+        if (clickPress) {
+            clickPress = false;
+            pressLeft();
+            clickRelease = true;
+            clickReleaseAt = tnow + std::chrono::milliseconds(40);
+        } else if (clickRelease && tnow >= clickReleaseAt) {
+            clickRelease = false;
+            releaseLeft();
+        }
+
+        controllerButtons.Poll(
+            [&](const char *button, bool down) {
+                SendTo(out, "frametop_relay", std::string("vrbtn ") + button + (down ? " 1" : " 0"));
+            },
+            inGame);
+
         vr::VREvent_t ev;
         while (sys->PollNextEvent(&ev, sizeof ev)) {
             if (ev.eventType == vr::VREvent_Quit) {

pointer/helper/vrbuttons.h

@@ -0,0 +1,141 @@
+// Frame controller buttons for the relay's button mappings (SteamVR input, see ft-pointer.cpp).
+//
+// The Frame controllers aren't input devices on the host (no evdev node, no hidraw): only
+// SteamVR sees them. So the helper, an overlay client anyway, reads them with SteamVR input
+// and sends each press and release to the relay ("vrbtn <hand>/<button> 1|0" on
+// @frametop_relay), which does the mapped action, like for a mouse button.
+//
+// actions/ft_pointer_actions.json has one action set per button, each with one boolean action
+// bound to that button's click (actions/bindings_frame_controller.json). Only the sets of
+// mapped buttons are active: the relay says which ("vrbind <button>..."; "vrbind *" for all,
+// while the settings app captures a button; "vrbind -" for none), and asks again with
+// "vrhello" when the helper connects. While a game runs (a scene application), the mapped
+// buttons are left to it, unless the list starts with "+games"; capturing takes them anyway. The sets are active at an overlay-global priority, so
+// SteamVR delivers the buttons without the helper having input focus, and takes those
+// buttons (only those) from the game: a mapped button is Frametop's. That needs SteamVR's
+// "Enable global input from overlays (Experimental)" (steamvr/globalActionSetPriority);
+// "vrglobal on|off" sets it, "vrstatus" replies with the state as JSON.
+#pragma once
+
+#include <openvr.h>
+
+#include <cstdio>
+#include <cstring>
+#include <string>
+#include <vector>
+
+class ControllerButtons {
+  public:
+    static constexpr const char *kButtons[] = {
+        "left/view",   "left/dpad_up", "left/dpad_down", "left/dpad_left", "left/dpad_right", "left/bumper",
+        "left/trigger", "left/grip",   "left/thumbstick", "right/menu",    "right/a",         "right/b",
+        "right/x",     "right/y",      "right/bumper",   "right/trigger",  "right/grip",      "right/thumbstick"};
+    static constexpr int kCount = sizeof kButtons / sizeof kButtons[0];
+    // Above SteamVR's own overlay input, inside the overlay-global range.
+    static constexpr int32_t kPriority = vr::k_nActionSetOverlayGlobalPriorityMin + 0x100;
+
+    // After VR_Init. The manifest can only be set once per connection.
+    void Init(const std::string &manifest) {
+        input_ = vr::VRInput();
+        const vr::EVRInputError e = input_->SetActionManifestPath(manifest.c_str());
+        ok_ = e == vr::VRInputError_None;
+        std::printf("controller buttons: action manifest %s: %s\n", manifest.c_str(), ok_ ? "ok" : "error");
+        if (!ok_) std::printf("controller buttons: SetActionManifestPath error %d\n", int(e));
+        for (int i = 0; i < kCount; ++i) {
+            std::string name = kButtons[i];
+            name[name.find('/')] = '_';
+            input_->GetActionSetHandle(("/actions/" + name).c_str(), &set_[i]);
+            input_->GetActionHandle(("/actions/" + name + "/in/press").c_str(), &action_[i]);
+        }
+        std::fflush(stdout);
+    }
+
+    // "vrbind" arguments: [+games] then button names, "*" (all) or "-" (none). Returns what's
+    // bound now.
+    std::string Bind(const char *args) {
+        bool want[kCount] = {};
+        char word[64];
+        int used = 0;
+        games_ = capture_ = false;
+        while (std::sscanf(args, " %63s%n", word, &used) == 1) {
+            args += used;
+            if (std::strcmp(word, "+games") == 0) games_ = true;
+            if (std::strcmp(word, "*") == 0) capture_ = true;
+            for (int i = 0; i < kCount; ++i)
+                if (std::strcmp(word, "*") == 0 || std::strcmp(word, kButtons[i]) == 0) want[i] = true;
+        }
+        for (int i = 0; i < kCount; ++i) bound_[i] = want[i];
+        return Bound();
+    }
+
+    std::string Bound() const {
+        std::string s;
+        for (int i = 0; i < kCount; ++i)
+            if (bound_[i]) s += (s.empty() ? "" : " ") + std::string(kButtons[i]);
+        return (s.empty() ? "-" : s) + (games_ ? " (in games too)" : " (outside games)");
+    }
+
+    // Every frame. send(button, pressed) for each change; a button that's unbound (or can't
+    // be read any more) while down is released, so no action stays held. inGame: a scene
+    // application is running.
+    template <class Send> void Poll(Send send, bool inGame) {
+        if (!ok_) return;
+        inGame_ = inGame;
+        const bool take = !inGame || games_ || capture_;
+        std::vector<vr::VRActiveActionSet_t> sets;
+        for (int i = 0; i < kCount; ++i)
+            if (bound_[i] && take) {
+                vr::VRActiveActionSet_t s{};
+                s.ulActionSet = set_[i];
+                s.nPriority = kPriority;
+                sets.push_back(s);
+            }
+        const bool updated =
+            !sets.empty() && input_->UpdateActionState(sets.data(), sizeof sets[0], uint32_t(sets.size())) ==
+                                 vr::VRInputError_None;
+        for (int i = 0; i < kCount; ++i) {
+            bool down = false;
+            active_[i] = false;
+            if (bound_[i] && take && updated) {
+                vr::InputDigitalActionData_t d{};
+                if (input_->GetDigitalActionData(action_[i], &d, sizeof d, vr::k_ulInvalidInputValueHandle) ==
+                    vr::VRInputError_None) {
+                    active_[i] = d.bActive;
+                    down = d.bActive && d.bState;
+                }
+            }
+            if (down != down_[i]) {
+                down_[i] = down;
+                send(kButtons[i], down);
+            }
+        }
+    }
+
+    // {"manifest":true,"global":false,"bound":[...],"active":[...]}: active = bound and
+    // delivered (a controller that has the button is on, and SteamVR lets us have it).
+    std::string Status() const {
+        const bool global = vr::VRSettings()->GetBool("steamvr", "globalActionSetPriority", nullptr);
+        std::string bound, active;
+        for (int i = 0; i < kCount; ++i) {
+            if (bound_[i]) bound += std::string(bound.empty() ? "" : ",") + "\"" + kButtons[i] + "\"";
+            if (active_[i]) active += std::string(active.empty() ? "" : ",") + "\"" + kButtons[i] + "\"";
+        }
+        return std::string("{\"t\":\"vrstatus\",\"manifest\":") + (ok_ ? "true" : "false") +
+               ",\"global\":" + (global ? "true" : "false") + ",\"in_game\":" + (inGame_ ? "true" : "false") +
+               ",\"games\":" + (games_ ? "true" : "false") + ",\"bound\":[" + bound + "],\"active\":[" + active + "]}";
+    }
+
+    static void SetGlobal(bool on) {
+        vr::VRSettings()->SetBool("steamvr", "globalActionSetPriority", on);
+        std::printf("controller buttons: SteamVR global input from overlays %s\n", on ? "on" : "off");
+        std::fflush(stdout);
+    }
+
+  private:
+    vr::IVRInput *input_ = nullptr;
+    bool ok_ = false;
+    vr::VRActionSetHandle_t set_[kCount] = {};
+    vr::VRActionHandle_t action_[kCount] = {};
+    bool bound_[kCount] = {}, down_[kCount] = {}, active_[kCount] = {};
+    bool games_ = false, capture_ = false, inGame_ = false;
+};

session/frametop.conf.example

@@ -26,6 +26,8 @@ POINTER_FOLLOW_REACH=70    # head follow: how far (degrees) from the middle of y
 POINTER_GAZE=0             # 1 = the pointer goes where you look, the mouse does the last bit (needs the gaze service: gaze/run.sh install)
 POINTER_GAZE_RETAKE=5      # gaze mode: how far (degrees) you look away from the pointer before the gaze takes it back from the mouse
 POINTER_GAZE_NUDGE_MAX=8   # gaze mode: a mouse nudge up to this far (degrees) before a click is learned as the eye tracker's error
+POINTER_GAZE_HOLD=0.5      # gaze mode: a press held this long (s) without moving becomes a real press (to drag); moved or released sooner, it clicks where you let go
+POINTER_GAZE_SHOW=1        # gaze mode: the dot shows this long (s) after the mouse moves it; also while a press is held, and a pulse per click
 META_DASHBOARD=0   # 1 = a Meta tap on a pass-through keyboard toggles the SteamVR dashboard (pointer mode only)
 SHARE_KEYS=0       # 1 = keys of keyboards grabbed for the desktop also go to @frametop_keys, for hotkey tools (any local process can listen)
 LAYOUT_GRAB_OFFSET=0.075   # arranging screens: where the floating window's grab bar is, in metres below the screen