From 0d50efec459758d221d79e9bd3b8b4c711ee29bf Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Mon, 28 Sep 2026 16:26:24 -0600 Subject: [PATCH] Map the Frame controllers' buttons; make gaze clicks correctable The Frame controllers aren't input devices on the host, so the pointer helper reads them with SteamVR input (vrbuttons.h, actions/), one action set per button, and sends presses to the relay, which does the mapped action like for a mouse button. Only mapped buttons are taken, at an overlay-global priority (SteamVR's experimental "Enable global input from overlays"), and only outside games unless In games is on. Frametop Input Settings gets a Controllers page to map them. Gaze mode: a left press while the gaze has the pointer isn't sent at once. The pointer stops, you drag it onto what you meant with the button held, and the release clicks there; a press held still for POINTER_GAZE_HOLD (0.5 s) becomes a real press, for drags. The dot shows only while the mouse moves it (POINTER_GAZE_SHOW), while a press is held, and as a pulse per click. Outside games the pointer stays on while gaze mode is on. ft-gazed takes a third of each lesson's offset instead of all of it: in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. Frametop Input Settings gets a Gaze page. Co-Authored-By: Claude Opus 5.5 --- README.md | 2 +- docs/design.md | 4 +- docs/reference.md | 6 +- gaze/README.md | 4 +- gaze/ft-gazed | 22 +- input-settings/ft_input_settings.py | 232 +++++++++++++ input-settings/main.qml | 315 +++++++++++++++++- input/input-relay.py | 79 ++++- .../actions/bindings_frame_controller.json | 242 ++++++++++++++ .../helper/actions/ft_pointer_actions.json | 197 +++++++++++ pointer/helper/ft-pointer.cpp | 234 +++++++++++-- pointer/helper/vrbuttons.h | 141 ++++++++ session/frametop.conf.example | 2 + 13 files changed, 1426 insertions(+), 54 deletions(-) create mode 100644 pointer/helper/actions/bindings_frame_controller.json create mode 100644 pointer/helper/actions/ft_pointer_actions.json create mode 100644 pointer/helper/vrbuttons.h diff --git a/README.md b/README.md index 83dbeb8..f29ff81 100644 --- a/README.md +++ b/README.md @@ -53,7 +53,7 @@ If you work in the desktop for long stretches, stop Steam from putting the heads | Meta+Shift+H in the desktop | Hides or shows all screens (also in the menu as Hide/Show Screens, and mappable). The Visibility & wrist tab of Frametop Display Settings can instead show them only with the dashboard open, or while you look at your wrist | | Play a VR game | The screens hide and your controllers stay in the game. Open the SteamVR dashboard, or press Meta+Shift+H, to see and use them. To keep them visible over games, change During VR games on the Visibility & wrist tab; the controllers still stay in the game, and you use the screens with the mouse or the dashboard | -You can map the mouse's extra buttons to actions such as Toggle SteamVR dashboard, Recenter pointer, or Head follow on/off on the Buttons page of Frametop Input Settings. Pointer speed, dot size, and the rest are on its Pointer page and take effect immediately. Head follow, which is experimental and off by default, makes the pointer come along when you turn your head: it stays put until your head turns past the leash angle, then glides back to its place in your view, and a leash of 0 keeps it fixed in your view. It's only lightly tested and not polished; tuning its settings, or improving how it feels, is open to anyone who wants to take it further. +You can map the mouse's extra buttons to actions such as Toggle SteamVR dashboard, Recenter pointer, or Head follow on/off on the Buttons page of Frametop Input Settings, and the Frame controllers' buttons on its Controllers page. Pointer speed, dot size, and the rest are on its Pointer page and take effect immediately. Head follow, which is experimental and off by default, makes the pointer come along when you turn your head: it stays put until your head turns past the leash angle, then glides back to its place in your view, and a leash of 0 keeps it fixed in your view. It's only lightly tested and not polished; tuning its settings, or improving how it feels, is open to anyone who wants to take it further. Restarting the desktop (Restart desktop in Frametop Display Settings) closes its windows, but background work you started in it, such as servers, tmux sessions, or builds, keeps running. diff --git a/docs/design.md b/docs/design.md index 1c80931..18c4ce6 100644 --- a/docs/design.md +++ b/docs/design.md @@ -82,7 +82,7 @@ A few overlays need special handling: Head follow is experimental and off by default. It works, but it's only lightly tested, and the feel is mostly a matter of its settings; polishing it is left open. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction, where you were facing when your head last settled, and keeps its offset from it. The mouse can put the cursor anywhere up to `POINTER_FOLLOW_REACH` (70 degrees) from the reference, a corner of your view included. While your head stays within `POINTER_LEASH_DEG` of the reference, nothing moves on its own. Once your head has been past the leash for `POINTER_LEASH_DELAY` (0.2 s, so a glance out and back doesn't count), the reference eases to where you're facing (time constant `POINTER_LEASH_RETURN`, 0.2 s), never falling further behind than the leash, and the cursor ends up back where it was in your view. Then it waits for the leash again. Two earlier versions didn't work out. Moving the reference only while your head pulled at the end of the leash left it up to the leash off after you turned back, and getting it centred again meant overshooting with your head. Easing it toward your facing all the time moved the cursor on every small head movement. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping. -Gaze mode is experimental and off by default (`POINTER_GAZE=1`, `gaze/ft-gazectl on`, or a mouse button mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz, and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. Moving the mouse takes the pointer from the gaze. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge of up to `POINTER_GAZE_NUDGE_MAX` (8 degrees) before a click is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. See `gaze/README.md` for the service, the calibration, and what was measured. +Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse or controller button mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz, and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. Moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze; the mouse is gaze mode's only pointer device for now. The dot shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. The gaze moving the pointer doesn't show it: you know where you're looking. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge of up to `POINTER_GAZE_NUDGE_MAX` (8 degrees) before a click is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. See `gaze/README.md` for the service, the calibration, and what was measured. Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place. @@ -108,6 +108,8 @@ An ungrabbed keyboard reaches both sides at once. In VR, gamescope reads every i Volume keys must never reach gamescope. With the openvr backend, gamescope sends volume up and down to Steam by moving keyboard focus to Steam for the key and then back to the previously focused surface. When nothing had focus, the one it moves back to is null, and wlroots aborts on a null focus surface (`wlr_seat_keyboard_notify_enter: Assertion 'surface' failed`), which ends the whole VR session. Keyboard focus is often empty while you work in VR, so one press of the headset's volume button could take everything down. gamescope reads the headset's buttons and every keyboard itself (`InputStealer`), as do SteamVR's processes, so the relay has to stop volume keys at the device. Grabbing `gpio-keys` would also take the headset's click button, so the relay remaps the volume entries in each device's keymap (`EVIOCSKEYCODE`) and handles the stand-in codes itself. That fix covers every device at once, including keyboards that aren't grabbed. +The Frame controllers can be mapped like mouse buttons, but they aren't input devices on the host: they reach SteamVR over the headset's own radio, and no evdev or hidraw node exists for them. So only a SteamVR client can read them. Overlay apps normally get controller input only while they have input focus, which a background helper never has. SteamVR's experimental global action set priority (`steamvr/globalActionSetPriority`, "Enable global input from overlays") lets an overlay's action set receive input anyway, and takes the inputs it binds from the scene app. Binding every button would take them all from games, so the helper's action manifest puts each button in an action set of its own, and it activates only the sets of mapped buttons. The mapping itself stays in the relay, which does the action, so mice and controllers share one list of actions. + ## The desktop session The session is modeled on SteamOS's `steamos-nested-desktop` and runs beside it. It has its own runtime directory, config (`~/.config/frametop`), and state, so it never disturbs the stock desktop's layout or panels. It runs on a private D-Bus from `dbus-run-session`, which has two consequences. KDE only launches apps in systemd scopes when systemd is on the session bus, so everything started in the desktop lands in its systemd unit, and stopping the unit would kill all of it; `session/keep-apps.sh` moves those programs out first. And tools that need the real user bus, like podman and `distrobox-host-exec`, have to be pointed at it explicitly. diff --git a/docs/reference.md b/docs/reference.md index c59eaa4..fe298e3 100644 --- a/docs/reference.md +++ b/docs/reference.md @@ -98,15 +98,17 @@ pointer/helper/run.sh status | log | restart pointer/driver/install.sh probe # devices, hand roles, who owns the dashboard pointer ``` -The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, the head follow settings `POINTER_FOLLOW`, `POINTER_LEASH_DEG`, `POINTER_LEASH_DELAY`, `POINTER_LEASH_RETURN`, and `POINTER_FOLLOW_REACH`, and the gaze mode settings `POINTER_GAZE`, `POINTER_GAZE_RETAKE`, and `POINTER_GAZE_NUDGE_MAX`. The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper. +The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, the head follow settings `POINTER_FOLLOW`, `POINTER_LEASH_DEG`, `POINTER_LEASH_DELAY`, `POINTER_LEASH_RETURN`, and `POINTER_FOLLOW_REACH`, and the gaze mode settings `POINTER_GAZE`, `POINTER_GAZE_RETAKE`, `POINTER_GAZE_NUDGE_MAX`, `POINTER_GAZE_HOLD`, and `POINTER_GAZE_SHOW`. The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper. ## Frametop Input Settings -A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has four pages: +A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has six pages: - Devices lists every USB and Bluetooth mouse and keyboard, with a light that flashes when the device is used. Each device gets a role: 3D pointer (grabbed, drives the pointer; the default for anything with a mouse), Pass through (grabbed only while typing goes to the desktop; the default for keyboards, where a Meta tap toggles the dashboard if `META_DASHBOARD=1` is in `~/.config/frametop.conf`), or Ignore. A device is identified by its Bluetooth address, or its USB ids and name, so all of its input nodes share one role. Forget drops everything saved for a device. - Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, gaze pointer on or off, faster or slower, pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep. +- Controllers maps the Frame controllers' buttons (every button but the system button) to the same actions, except passing a key through. Capture a button and press it on a controller, or pick it from the list. The controllers aren't input devices on the host; only SteamVR sees them. So the pointer helper reads them with SteamVR input (`pointer/helper/vrbuttons.h`, `pointer/helper/actions/`) and sends presses to the relay (`vrbtn right/a 1`), which does the mapped action. The helper only takes the buttons that are mapped (the relay tells it with `vrbind`), at an overlay-global priority, and only while no game (scene application) runs, so games keep every button; with In games on (`controller_in_games`), a mapped button is taken from games too. That needs SteamVR's "Enable global input from overlays (Experimental)" setting (`steamvr/globalActionSetPriority`); the page offers to turn it on. Mappings are saved as `controller_buttons` in `~/.config/frametop-input.json`. - Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button. +- Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), the gaze mode sliders, the gaze service's state (headset, samples per second, whether only one eye is tracked, the calibration, the nudges learned), and Calibrate (opens the gaze probe), Reload calibration, and Forget nudges. - Bluetooth lists paired devices and has Apply Bluetooth fixes, which runs `/etc/steamframe/bt-fixups.sh` through `pkexec`. Pair new devices in Steam. Device rules are saved in `~/.config/frametop-input.json`. `input-settings/install.sh` installs the menu entry. Its launcher hands podman the real `XDG_RUNTIME_DIR` and user bus and gives the app the session's Wayland socket, because the desktop session runs on a private D-Bus and podman fails on it. diff --git a/gaze/README.md b/gaze/README.md index d1f0938..7330feb 100644 --- a/gaze/README.md +++ b/gaze/README.md @@ -19,8 +19,8 @@ gaze/ft-gazectl on # the pointer follows your gaze (off: the mouse al Gaze as an input method for the whole desktop, without replacing anything of SteamVR's: - `ft-gazed` (host Python, a user service: `gaze/run.sh install`) runs ft-gaze and corrects its gaze. It uses SteamVR's combined gaze (mmap set 1), which keeps going when the tracker loses one eye; set 2's combined direction is off by half of whatever the lost eye reads, and the tracker does lose an eye for minutes at a time. It drops blinks (both eyes closing), smooths with a fixation lock, and applies the calibration from the probe (`calibration.json`, reloaded when the probe changes it) plus what the pointer has learned since (`pointer-lessons.json`). It sends the result to the pointer helper 90 times a second. It follows SteamVR's eye tracking log, and when the headset goes back on (SteamVR starts its eye model over, and the error moves), older lessons count less, so the first few after relearn the offset. -- The pointer helper's **gaze mode** (off by default: `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a button mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look. Move the mouse and it's the mouse's, from where the gaze put it, for the last bit. Look well away (5 degrees) and the gaze takes it back. Clicks and drags work as usual. -- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it a little (0.2 to 8 degrees) before you clicked, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them. +- The pointer helper's **gaze mode** (off by default: the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a mouse or controller button mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look. Move the mouse and it's the mouse's, from where the gaze put it, for the last bit. Look well away (5 degrees) and the gaze takes it back. A press isn't sent at once: the pointer stops where the gaze put it, and if that's wrong, drag it onto what you meant with the button still held; the click happens where you let go. To drag something, hold the press still for half a second first (`POINTER_GAZE_HOLD`), then move. Outside games the pointer stays on while gaze mode is on. The dot only shows while the mouse moves it, while a press is held, and as a pulse when you click. +- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it a little (0.2 to 8 degrees) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them. - Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always. Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction, the correction at the time, and how far off it was. diff --git a/gaze/ft-gazed b/gaze/ft-gazed index 470c6d6..b864543 100755 --- a/gaze/ft-gazed +++ b/gaze/ft-gazed @@ -69,6 +69,18 @@ LESSON_MAX = 8.0 # degrees past the correction RETRY = 3.0 # seconds before starting ft-gaze again +class PointerLessons(LiveCorrection): + """LiveCorrection, with the offset held back: one lesson moves the whole correction by a + third of what it measured, not all of it (two alike, half; three, three fifths). In the + probe, clicks came thick and fast on a stale calibration, where the error was mostly one + offset. Here lessons are few and the calibration is often fresh: in the first live test a + 6 degree lesson shifted everything 6 degrees, and the next target, 10 degrees away and + 1.4 off before, was 7.1 off (3.6 with this). Near the lesson, the kernel still takes up + most of it (5.1 of the 6 degrees).""" + + RIDGE = [2.0] + LiveCorrection.RIDGE[1:] + + def log(msg): print(f"ft-gazed: {msg}", file=sys.stderr, flush=True) @@ -80,7 +92,7 @@ class Service: self.base = Correction() self.mode = DEFAULT_MODEL self.cal_mtime = None - self.live = LiveCorrection() + self.live = PointerLessons() self.dirty = False self.load_calibration() self.load_lessons() @@ -91,7 +103,7 @@ class Service: self.fix = Fixation(radius=1.0) self.opens = (deque(maxlen=90), deque(maxlen=90)) # left, right self.vergence = deque(maxlen=90) - self.counts = {"samples": 0, "sent": 0, "blinks": 0, "one_eye": 0, "dropped": 0, "lessons": 0, "refused": 0} + self.counts = {"samples": 0, "sent": 0, "blinks": 0, "one_eye": 0, "dropped": 0, "lessons_taken": 0, "refused": 0} self.last_sample = 0.0 self.last = None @@ -124,7 +136,7 @@ class Service: try: d = json.loads(LESSONS.read_text()) if d.get("source") == self.source: - self.live.samples = d.get("samples", [])[-LiveCorrection.KEEP:] + self.live.samples = d.get("samples", [])[-PointerLessons.KEEP:] except (OSError, ValueError): pass log(f"{len(self.live.samples)} lessons") @@ -152,7 +164,7 @@ class Service: else: self.live.add({"time": rec["time"], "hy": rhy, "hp": rhp, "dy": dy, "dp": dp, "wy": 1.0, "wp": 1.0, "how": "pointer"}, self.base, self.mode) - self.counts["lessons"] += 1 + self.counts["lessons_taken"] += 1 self.dirty = True try: with open(LESSON_LOG, "a") as f: @@ -293,7 +305,7 @@ class Service: elif words[:1] == ["status"]: reply = json.dumps(self.status()) elif words[:1] == ["forget"]: - self.live = LiveCorrection() + self.live = PointerLessons() self.live.wear_time = self.steam.worn() self.save_lessons() reply = "ok" diff --git a/input-settings/ft_input_settings.py b/input-settings/ft_input_settings.py index 4e3adab..631ed05 100644 --- a/input-settings/ft_input_settings.py +++ b/input-settings/ft_input_settings.py @@ -6,7 +6,12 @@ to the input relay over its control socket (@frametop_relay): - Devices: every USB/Bluetooth mouse and keyboard, a live activity light to identify them, and a role for each (3D pointer, pass through, ignore). - Buttons: press a button or key on a pointer device, then pick an action. + - Controllers: the same for the Frame controllers' buttons. They're read by the pointer + helper through SteamVR input (@ft_pointer_helper: vrstatus, vrglobal), and a mapped + button is taken from games. - Pointer: speed, dot size, distance and the rest, applied live. + - Gaze: the pointer's gaze mode (@ft_pointer_helper "gaze") and the gaze service + (gaze/ft-gazed, @ft_gazed: status, forget, reload). - Bluetooth: paired devices, and re-applying the Bluetooth LE fixes after pairing. Rules go to ~/.config/frametop-input.json and pointer settings to ~/.config/frametop.conf; then the relay (and through it the helper) reloads. @@ -19,6 +24,7 @@ import shutil import socket import subprocess import sys +import time from PySide6.QtCore import Property, QObject, QSocketNotifier, QTimer, QUrl, Signal, Slot from PySide6.QtGui import QGuiApplication, QIcon @@ -29,6 +35,9 @@ RULES_PATH = os.path.expanduser("~/.config/frametop-input.json") CONF_PATH = os.path.expanduser("~/.config/frametop.conf") RELAY = "\0frametop_relay" HELPER = "\0ft_pointer_helper" +GAZED = "\0ft_gazed" +REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +GAZE_PROBE = os.path.join(REPO, "gaze", "probe", "ft-gazeprobe") BTN_MISC = 0x100 # Header names that mark the start of a range, not a real key (BTN_MOUSE == BTN_LEFT). RANGE_ALIASES = {"BTN_MISC", "BTN_MOUSE", "BTN_JOYSTICK", "BTN_GAMEPAD", "BTN_DIGI", "BTN_WHEEL", @@ -45,6 +54,24 @@ ACTION_LABELS = { "none": "Do nothing", } ROLE_LABELS = {"pointer": "3D pointer", "passthrough": "Pass through", "ignore": "Ignore"} +# Frame controller buttons the pointer helper can read (pointer/helper/vrbuttons.h). The +# system button stays SteamVR's. +CONTROLLER_BUTTONS = { + "left/view": "Left View", "left/dpad_up": "Left D-pad up", "left/dpad_down": "Left D-pad down", + "left/dpad_left": "Left D-pad left", "left/dpad_right": "Left D-pad right", "left/bumper": "Left bumper", + "left/trigger": "Left trigger", "left/grip": "Left grip", "left/thumbstick": "Left stick click", + "right/menu": "Right Menu", "right/a": "Right A", "right/b": "Right B", "right/x": "Right X", "right/y": "Right Y", + "right/bumper": "Right bumper", "right/trigger": "Right trigger", "right/grip": "Right grip", + "right/thumbstick": "Right stick click", +} +CONTROLLER_ACTIONS = [a for a in ACTION_LABELS if a not in ("key", "none")] +# Gaze mode settings (pointer helper), like POINTER_SETTINGS. +GAZE_SETTINGS = [ + ("POINTER_GAZE_RETAKE", "Look away to hand back", 5, 1, 45, 0.5, "°"), + ("POINTER_GAZE_NUDGE_MAX", "Largest nudge to learn", 8, 1, 30, 0.5, "°"), + ("POINTER_GAZE_HOLD", "Hold still to drag", 0.5, 0.1, 2.0, 0.05, "s"), + ("POINTER_GAZE_SHOW", "Dot shows after moving", 1.0, 0.0, 5.0, 0.1, "s"), +] # Pointer settings: key, label, default, min, max, step, unit. POINTER_SETTINGS = [ ("POINTER_SENSITIVITY", "Speed", 0.03, 0.005, 0.12, 0.001, "°/count"), @@ -134,8 +161,11 @@ class Backend(QObject): mappingsChanged = Signal() pointerChanged = Signal() bluetoothChanged = Signal() + controllersChanged = Signal() + gazeChanged = Signal() activity = Signal(str) # device id captured = Signal(int, str) # code, name + capturedController = Signal(str, str) # button, label message = Signal(str, bool) # text, is error def __init__(self): @@ -146,6 +176,13 @@ class Backend(QObject): self._capture_id = "" self._bluetooth = [] self._relay_ok = False + self._capture_vr = False + self._vr = {} # the helper's vrstatus, {} when it doesn't answer + self._vr_at = 0.0 + self._gaze = {} # ft-gazed's status, {} when it isn't running + self._gaze_prev = None # the status before, for rates + self._gaze_at = 0.0 + self._gaze_mode = None # the helper's gaze mode: True, False, None (no answer) self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) self.sock.bind("") # autobind an abstract address the relay can reply to self.sock.setblocking(False) @@ -173,6 +210,19 @@ class Backend(QObject): def _refresh(self): self._send("devices") + self._send("vrstatus", HELPER) + self._send("gaze ?", HELPER) + if not self._send("status", GAZED) and self._gaze: + self._gaze, self._gaze_prev = {}, None + self.gazeChanged.emit() + now = time.monotonic() + # No answer for a while: that side isn't running (any more). + if self._vr and now - self._vr_at > 5: + self._vr = {} + self.controllersChanged.emit() + if self._gaze_mode is not None and now - self._gaze_at > 5: + self._gaze_mode = None + self.gazeChanged.emit() def _read(self): while True: @@ -180,16 +230,37 @@ class Backend(QObject): data = self.sock.recv(65536) except BlockingIOError: return + text = data.decode(errors="replace") + if text in ("ok on", "ok off"): # the helper's answer to "gaze ?" (from an unbound socket) + self._gaze_mode = text == "ok on" + self._gaze_at = time.monotonic() + self.gazeChanged.emit() + continue try: msg = json.loads(data) except ValueError: continue + if isinstance(msg, dict) and "samples" in msg and "t" not in msg: # ft-gazed's status + self._gaze_status(msg) + continue + if not isinstance(msg, dict): + continue t = msg.get("t") if t == "devices": self._nodes = msg.get("nodes", []) self._pointer_mode = bool(msg.get("pointer_mode")) self._relay_ok = True self.devicesChanged.emit() + elif t == "vrstatus": + self._vr = msg + self._vr_at = time.monotonic() + self.controllersChanged.emit() + elif t == "event" and msg.get("type") == "vr": + self.activity.emit(msg["id"]) + if self._capture_vr and msg["value"] == 1 and msg["code"] in CONTROLLER_BUTTONS: + self._capture_vr = False + self._send("vrcapture 0") + self.capturedController.emit(msg["code"], CONTROLLER_BUTTONS[msg["code"]]) elif t == "event": self.activity.emit(msg["id"]) if (self._capture_id and msg["id"] == self._capture_id and msg["type"] == "key" @@ -381,6 +452,167 @@ class Backend(QObject): def recenter(self): return self._send("recenter", HELPER) + # --- controllers --- + @Property("QVariantList", constant=True) + def controllerActions(self): + return [{"value": a, "text": ACTION_LABELS[a]} for a in CONTROLLER_ACTIONS] + + @Property("QVariantList", constant=True) + def controllerButtons(self): + return [{"value": b, "text": label} for b, label in CONTROLLER_BUTTONS.items()] + + @Property("QVariantList", notify=mappingsChanged) + def controllerMappings(self): + mapped = read_json(RULES_PATH).get("controller_buttons", {}) + return [{"button": b, "label": label, "action": mapped[b], + "actionLabel": ACTION_LABELS.get(mapped[b], mapped[b])} + for b, label in CONTROLLER_BUTTONS.items() if b in mapped] + + @Property("QVariantMap", notify=controllersChanged) + def controllerStatus(self): + """helper: answering; manifest: its SteamVR input is set up; global: SteamVR's + "Enable global input from overlays"; active: mapped buttons SteamVR delivers now.""" + vr = self._vr + return {"helper": bool(vr), "manifest": bool(vr.get("manifest")), "global": bool(vr.get("global")), + "inGame": bool(vr.get("in_game")), "bound": vr.get("bound", []), "active": vr.get("active", [])} + + @Property(bool, notify=mappingsChanged) + def controllerInGames(self): + return bool(read_json(RULES_PATH).get("controller_in_games")) + + @Slot(bool) + def setControllerInGames(self, on): + rules = read_json(RULES_PATH) + rules["controller_in_games"] = bool(on) + self._save_rules(rules) + self.message.emit("Mapped controller buttons: " + ("taken in games too" if on else "left to games"), False) + + @Slot(str, str) + def setControllerMapping(self, button, action): + if button not in CONTROLLER_BUTTONS or action not in CONTROLLER_ACTIONS: + return + rules = read_json(RULES_PATH) + rules.setdefault("controller_buttons", {})[button] = action + self._save_rules(rules) + self.message.emit(f"{CONTROLLER_BUTTONS[button]} → {ACTION_LABELS[action]}", False) + + @Slot(str) + def removeControllerMapping(self, button): + rules = read_json(RULES_PATH) + rules.get("controller_buttons", {}).pop(button, None) + self._save_rules(rules) + self.message.emit(f"{CONTROLLER_BUTTONS.get(button, button)}: back to games", False) + + @Slot() + def clearControllerMappings(self): + rules = read_json(RULES_PATH) + removed = len(rules.pop("controller_buttons", {}) or {}) + self._save_rules(rules) + self.message.emit(f"Removed {removed} controller binding{'s' if removed != 1 else ''}", False) + + @Slot() + def startControllerCapture(self): + self._capture_vr = True + self._send("watch 60") + self._send("vrcapture 30") + + @Slot() + def cancelControllerCapture(self): + if self._capture_vr: + self._capture_vr = False + self._send("vrcapture 0") + + @Slot(bool) + def setGlobalInput(self, on): + """SteamVR's "Enable global input from overlays (Experimental)", which the helper needs + to get controller buttons while a game or the dashboard has focus.""" + if self._send(f"vrglobal {'on' if on else 'off'}", HELPER): + self.message.emit(f"SteamVR global input from overlays {'on' if on else 'off'}", False) + else: + self.message.emit("The pointer helper isn't running (frametop-pointer.service)", True) + + # --- gaze --- + def _gaze_status(self, status): + now = time.monotonic() + prev = self._gaze_prev + # Rates over the last poll: samples per second, and the share with only one eye. + if prev and now - prev[0] > 0.5 and status["samples"] >= prev[1]["samples"]: + n = status["samples"] - prev[1]["samples"] + status["rate"] = n / (now - prev[0]) + status["one_eye_share"] = (status["one_eye"] - prev[1]["one_eye"]) / n if n else 0.0 + elif self._gaze: + status.setdefault("rate", self._gaze.get("rate")) + status.setdefault("one_eye_share", self._gaze.get("one_eye_share")) + if not prev or now - prev[0] > 0.5: + self._gaze_prev = (now, status) + self._gaze = status + self.gazeChanged.emit() + + @Property("QVariantMap", notify=gazeChanged) + def gazeStatus(self): + return self._gaze + + @Property(bool, notify=gazeChanged) + def gazeServiceRunning(self): + return bool(self._gaze) + + @Property(int, notify=gazeChanged) + def gazeMode(self): + """The helper's gaze mode now: 1 on, 0 off, -1 no answer (helper not running).""" + return -1 if self._gaze_mode is None else int(self._gaze_mode) + + @Property(bool, notify=gazeChanged) + def gazeDefault(self): + return read_conf().get("POINTER_GAZE", "0") not in ("", "0") + + @Slot(bool) + def setGazeMode(self, on): + """On or off now and from now on (POINTER_GAZE); a mapped button toggles it until restart.""" + write_conf_value("POINTER_GAZE", "1" if on else "0") + self._send(f"gaze {'on' if on else 'off'}", HELPER) + self.reload_timer.start() + self.gazeChanged.emit() + + @Property("QVariantList", notify=pointerChanged) + def gazeSettings(self): + conf = read_conf() + out = [] + for key, label, default, lo, hi, step, unit in GAZE_SETTINGS: + try: + value = float(conf.get(key, default)) + except ValueError: + value = default + out.append({"key": key, "label": label, "value": value, "min": lo, "max": hi, "step": step, + "unit": unit, "default": default}) + return out + + @Slot() + def forgetGazeLessons(self): + if self._send("forget", GAZED): + self.message.emit("Forgot what the pointer's nudges taught; the calibration stays", False) + else: + self.message.emit("The gaze service isn't running (frametop-gaze.service)", True) + + @Slot() + def reloadGazeCalibration(self): + if self._send("reload", GAZED): + self.message.emit("The gaze service read the calibration again", False) + else: + self.message.emit("The gaze service isn't running (frametop-gaze.service)", True) + + @Slot() + def openGazeProbe(self): + """Calibrate in ft-gazeprobe (a GTK app on the host, fullscreen on a Frametop screen).""" + runner = ["distrobox-host-exec"] if shutil.which("distrobox-host-exec") else [] + env = [f"{k}={os.environ[k]}" for k in ("WAYLAND_DISPLAY", "DISPLAY", "XAUTHORITY", "DBUS_SESSION_BUS_ADDRESS") + if os.environ.get(k)] + try: + subprocess.Popen(runner + ["env"] + env + [GAZE_PROBE], stdin=subprocess.DEVNULL, + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=True) + self.message.emit("Opening the gaze probe: calibrate there, then close it", False) + except OSError as e: + self.message.emit(f"Couldn't open the gaze probe: {e}", True) + # --- bluetooth --- @Property("QVariantList", notify=bluetoothChanged) def bluetooth(self): diff --git a/input-settings/main.qml b/input-settings/main.qml index 06e41ff..ab13372 100644 --- a/input-settings/main.qml +++ b/input-settings/main.qml @@ -18,7 +18,9 @@ Kirigami.ApplicationWindow { actions: [ Kirigami.Action { text: "Devices"; icon.name: "input-mouse"; onTriggered: root.show(devicesPage) }, Kirigami.Action { text: "Buttons"; icon.name: "input-keyboard"; onTriggered: root.show(buttonsPage) }, + Kirigami.Action { text: "Controllers"; icon.name: "input-gamepad"; onTriggered: root.show(controllersPage) }, Kirigami.Action { text: "Pointer"; icon.name: "transform-move"; onTriggered: root.show(pointerPage) }, + Kirigami.Action { text: "Gaze"; icon.name: "view-visible"; onTriggered: root.show(gazePage) }, Kirigami.Action { text: "Bluetooth"; icon.name: "preferences-system-bluetooth"; onTriggered: root.show(bluetoothPage) } ] } @@ -28,8 +30,9 @@ Kirigami.ApplicationWindow { pageStack.push(page) } - // FT_INPUT_PAGE=buttons|pointer|bluetooth opens the app on that page. - pageStack.initialPage: ({ buttons: buttonsPage, pointer: pointerPage, bluetooth: bluetoothPage })[startPage] || devicesPage + // FT_INPUT_PAGE=buttons|controllers|pointer|gaze|bluetooth opens the app on that page. + pageStack.initialPage: ({ buttons: buttonsPage, controllers: controllersPage, pointer: pointerPage, gaze: gazePage, + bluetooth: bluetoothPage })[startPage] || devicesPage Connections { target: backend @@ -294,7 +297,177 @@ Kirigami.ApplicationWindow { wrapMode: Text.Wrap opacity: 0.7 text: "Buttons without a mapping pass through (mouse buttons as clicks, keys as keys). " - + "The Z3's extra buttons show up as keys from its keyboard node." + + "The Z3's extra buttons show up as keys from its keyboard node. " + + "The Frame controllers' buttons are on the Controllers page." + } + } + } + } + + // ---------------------------------------------------------------- Controllers + Component { + id: controllersPage + Kirigami.ScrollablePage { + id: cpage + title: "Controllers" + actions: [ + Kirigami.Action { + text: "Remove all" + icon.name: "edit-clear-all" + tooltip: "Give every controller button back to games" + enabled: backend.controllerMappings.length > 0 + onTriggered: backend.clearControllerMappings() + } + ] + property string capturedButton: "" + property string capturedLabel: "" + property bool capturing: false + property var status: backend.controllerStatus + Component.onDestruction: backend.cancelControllerCapture() + + Connections { + target: backend + function onCapturedController(button, label) { + cpage.capturedButton = button; cpage.capturedLabel = label; cpage.capturing = false + buttonBox.currentIndex = buttonBox.indexOfValue(button) + } + } + Timer { + // The relay takes every button for 30 s while capturing. + running: cpage.capturing + interval: 30000 + onTriggered: { backend.cancelControllerCapture(); cpage.capturing = false } + } + + ColumnLayout { + spacing: Kirigami.Units.largeSpacing + + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: !cpage.status.helper + type: Kirigami.MessageType.Error + text: "The pointer helper isn't answering (frametop-pointer.service, needs SteamVR). " + + "It reads the controller buttons." + } + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: cpage.status.helper && !cpage.status.manifest + type: Kirigami.MessageType.Error + text: "The pointer helper couldn't set up SteamVR input (pointer/helper/actions). See its log." + } + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: cpage.status.helper && !cpage.status.global + type: Kirigami.MessageType.Warning + text: "SteamVR's \"Enable global input from overlays (Experimental)\" is off. Without it the " + + "buttons only reach Frametop when nothing else has focus, if at all." + actions: [ + Kirigami.Action { + text: "Turn it on" + icon.name: "dialog-ok-apply" + onTriggered: backend.setGlobalInput(true) + } + ] + } + + Kirigami.FormLayout { + Layout.fillWidth: true + + Controls.Switch { + Kirigami.FormData.label: "In games:" + text: "Mapped buttons work while a game is running too (the game loses them)" + checked: backend.controllerInGames + onToggled: backend.setControllerInGames(checked) + } + RowLayout { + Kirigami.FormData.label: "New mapping:" + Controls.Button { + text: cpage.capturing ? "Press a button on a controller…" : "Capture a button" + icon.name: "input-gamepad" + highlighted: cpage.capturing + enabled: cpage.status.helper + onClicked: { + if (cpage.capturing) { backend.cancelControllerCapture(); cpage.capturing = false } + else { cpage.capturedButton = ""; cpage.capturing = true; backend.startControllerCapture() } + } + } + Controls.Label { text: "or" ; opacity: 0.7 } + Controls.ComboBox { + id: buttonBox + model: backend.controllerButtons + textRole: "text" + valueRole: "value" + Layout.preferredWidth: Kirigami.Units.gridUnit * 10 + } + Controls.ComboBox { + id: newControllerAction + model: backend.controllerActions + textRole: "text" + valueRole: "value" + Layout.preferredWidth: Kirigami.Units.gridUnit * 13 + } + Controls.Button { + text: "Map" + icon.name: "dialog-ok-apply" + onClicked: { backend.setControllerMapping(buttonBox.currentValue, newControllerAction.currentValue); cpage.capturedButton = "" } + } + } + } + + Kirigami.Heading { + visible: backend.controllerMappings.length > 0 + level: 3 + text: "Current mappings" + } + + Repeater { + model: backend.controllerMappings + delegate: RowLayout { + id: crow + required property var modelData + Layout.fillWidth: true + spacing: Kirigami.Units.largeSpacing + Controls.Label { + text: crow.modelData.label + Layout.preferredWidth: Kirigami.Units.gridUnit * 10 + } + Controls.ComboBox { + model: backend.controllerActions + textRole: "text" + valueRole: "value" + Layout.preferredWidth: Kirigami.Units.gridUnit * 13 + Component.onCompleted: currentIndex = indexOfValue(crow.modelData.action) + onActivated: backend.setControllerMapping(crow.modelData.button, currentValue) + } + Controls.Label { + // Active: SteamVR delivers it to Frametop now (a controller is on). + text: cpage.status.active.indexOf(crow.modelData.button) >= 0 ? "active" + : cpage.status.inGame && !backend.controllerInGames ? "game's" : "waiting" + opacity: 0.6 + Layout.preferredWidth: Kirigami.Units.gridUnit * 4 + Controls.ToolTip.text: text === "active" ? "SteamVR gives this button to Frametop" + : text === "game's" ? "A game is running: the button is the game's until it quits" + : "No controller with this button is on, or SteamVR doesn't give it to Frametop" + Controls.ToolTip.visible: hover.hovered + HoverHandler { id: hover } + } + Controls.Button { + text: "Remove" + icon.name: "edit-delete-remove" + onClicked: backend.removeControllerMapping(crow.modelData.button) + } + Item { Layout.fillWidth: true } + } + } + + Controls.Label { + Layout.fillWidth: true + wrapMode: Text.Wrap + opacity: 0.7 + text: "Outside games, a mapped button is Frametop's; while a game runs it's the game's, unless " + + "\"In games\" is on. Unmapped buttons are left alone, and the system button stays SteamVR's. " + + "Clicks and scrolling go to the 3D pointer. The trigger and grip also move SteamVR's laser " + + "to that controller, so they're better left unmapped." } } } @@ -370,6 +543,142 @@ Kirigami.ApplicationWindow { } } + // ---------------------------------------------------------------- Gaze + Component { + id: gazePage + Kirigami.ScrollablePage { + id: gpage + title: "Gaze" + property var status: backend.gazeStatus + actions: [ + Kirigami.Action { + text: "Calibrate…" + icon.name: "crosshairs" + tooltip: "Open the gaze probe to calibrate (fullscreen on a Frametop screen)" + onTriggered: backend.openGazeProbe() + }, + Kirigami.Action { + text: "Reload calibration" + icon.name: "view-refresh" + enabled: backend.gazeServiceRunning + onTriggered: backend.reloadGazeCalibration() + }, + Kirigami.Action { + text: "Forget nudges" + icon.name: "edit-clear-history" + enabled: backend.gazeServiceRunning + tooltip: "Drop what your mouse nudges taught; the calibration stays" + onTriggered: backend.forgetGazeLessons() + } + ] + + Kirigami.FormLayout { + Kirigami.InlineMessage { + Layout.fillWidth: true + Layout.maximumWidth: Kirigami.Units.gridUnit * 30 + visible: true + type: Kirigami.MessageType.Warning + text: "Gaze mode is experimental. The pointer goes where you look and the mouse does the last bit; " + + "moving the mouse takes over, looking well away hands it back. A small nudge and a click " + + "teach the gaze service where it was off." + } + Controls.Switch { + Kirigami.FormData.label: "Gaze pointer:" + text: backend.gazeMode < 0 ? "Pointer helper not running" : "Pointer goes where you look" + enabled: backend.gazeMode >= 0 + checked: backend.gazeMode > 0 + onToggled: backend.setGazeMode(checked) + } + Controls.Label { + visible: backend.gazeMode >= 0 && (backend.gazeMode > 0) !== backend.gazeDefault + text: "Toggled by a button; it starts " + (backend.gazeDefault ? "on" : "off") + " after a restart." + opacity: 0.7 + font: Kirigami.Theme.smallFont + } + Repeater { + model: backend.gazeSettings + delegate: RowLayout { + required property var modelData + Kirigami.FormData.label: modelData.label + ":" + Controls.Slider { + id: gslider + from: modelData.min + to: modelData.max + stepSize: modelData.step + value: modelData.value + Layout.preferredWidth: Kirigami.Units.gridUnit * 14 + onMoved: backend.setPointerSetting(modelData.key, value) + } + Controls.Label { + text: gslider.value.toFixed(modelData.step < 0.1 ? 2 : 1) + " " + modelData.unit + Layout.preferredWidth: Kirigami.Units.gridUnit * 5 + } + Controls.ToolButton { + icon.name: "edit-undo" + display: Controls.AbstractButton.IconOnly + text: "Default (" + modelData.default + ")" + Controls.ToolTip.text: text + Controls.ToolTip.visible: hovered + onClicked: { gslider.value = modelData.default; backend.setPointerSetting(modelData.key, modelData.default) } + } + } + } + + Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Gaze service" } + + Controls.Label { + Kirigami.FormData.label: "Service:" + text: backend.gazeServiceRunning + ? (gpage.status.ft_gaze ? "running" : "running, eye tracker reader restarting") + : "not running (frametop-gaze.service, starts with SteamVR)" + color: backend.gazeServiceRunning ? Kirigami.Theme.textColor : Kirigami.Theme.negativeTextColor + } + Controls.Label { + visible: backend.gazeServiceRunning + Kirigami.FormData.label: "Headset:" + text: gpage.status.headset_on ? "on" : "off" + } + Controls.Label { + visible: backend.gazeServiceRunning + Kirigami.FormData.label: "Tracking:" + text: gpage.status.rate === undefined || gpage.status.rate === null ? "…" + : Math.round(gpage.status.rate) + " samples/s" + + (gpage.status.one_eye_share > 0.5 ? " · only one eye tracked" : "") + color: gpage.status.one_eye_share > 0.5 ? Kirigami.Theme.neutralTextColor : Kirigami.Theme.textColor + Controls.ToolTip.text: "Only one eye tracked: reseat the headset or check the lenses. It still works, " + + "probably less precisely." + Controls.ToolTip.visible: gpage.status.one_eye_share > 0.5 && ghover.hovered + HoverHandler { id: ghover } + } + Controls.Label { + visible: backend.gazeServiceRunning + Kirigami.FormData.label: "Calibration:" + text: gpage.status.calibration_samples > 0 + ? gpage.status.calibration_samples + " points (" + gpage.status.model + ")" + : "none yet: use Calibrate…" + } + Controls.Label { + visible: backend.gazeServiceRunning + Kirigami.FormData.label: "Learned from nudges:" + text: gpage.status.lessons + (gpage.status.lessons === 1 ? " nudge" : " nudges") + + (gpage.status.refused > 0 ? " (" + gpage.status.refused + " too far off, ignored)" : "") + } + } + + footer: Controls.Label { + padding: Kirigami.Units.largeSpacing + wrapMode: Text.Wrap + opacity: 0.7 + text: "Map a mouse button (Buttons) or a controller button (Controllers) to \"Gaze pointer on/off\" " + + "to switch it on the fly. A click waits for the release: if the gaze is off, drag onto the target " + + "with the button held and let go there. Hold still to drag: hold a press this long without moving " + + "to drag something instead. Look away to hand back: how far from the pointer you look before the " + + "gaze takes it back from the mouse. Largest nudge to learn: bigger mouse moves before a click " + + "are treated as using the mouse, not correcting the gaze." + } + } + } + // ---------------------------------------------------------------- Bluetooth Component { id: bluetoothPage diff --git a/input/input-relay.py b/input/input-relay.py index 5c80ae2..cc7a04b 100755 --- a/input/input-relay.py +++ b/input/input-relay.py @@ -24,6 +24,17 @@ pointer_toggle, follow_toggle = head follow on or off, gaze_toggle = gaze mode o layout_reset = put the desktop screens back in their saved layout, screens_toggle = hide or show the desktop screens, key = pass through as a key, none). +Frame controller buttons can be mapped too ("controller_buttons": {"right/a": action} in the +rules file; any action but key). The controllers aren't input devices here, only SteamVR sees +them, so the pointer helper reads them with SteamVR input and sends "vrbtn