Pointer helper: split the member functions into files by topic

Structure only, no behaviour change; the code moves as it is (git's
--color-moved shows it). build.sh compiles ft-pointer.cpp and
pointer_*.cpp.

- pointer.h: the includes, the helpers more than one file uses
  (HandGestures, PoseHistory, FramePanel, SendTo, JsonQuote, OverlayList;
  the three functions are now inline), Clock, Frame, and Pointer.
- ft-pointer.cpp: the header comment, setup (Init, ApplyConfig, and the
  helpers only it uses), the short per-frame sections, and main().
- pointer_input.cpp: waking, the mouse's buttons, the relay's commands.
- pointer_place.cpp: panel placement for layouts.
- pointer_gaze.cpp: gaze mode, the held-back press, the click.
- pointer_holds.cpp: holds (hands, gaze precision buttons).
- pointer_keys.cpp: keyboard clicks.
- pointer_cursor.cpp: recenter, head follow, slow work, the cursor.

Helpers only one file uses stay in its anonymous namespace. Each new file
says that "the top" in its comments is ft-pointer.cpp's header comment.
hands/README.md and update-check.py now name the files that hold the code
they point at.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

README.md

@@ -17,8 +17,6 @@ Join the [Frametop Discord](https://discord.gg/W3X9f7z3Bc) for questions, ideas,
 
 ## Install on the headset
 
-> **Frametop doesn't work on the SteamOS beta right now.** On the beta (SteamOS 0.4.3), gaze mode can't read the eye tracker, and the desktop has started without its taskbar ([#15](https://github.com/DeeJanuz/frametop/issues/15)). Use the stable SteamOS release until this note is gone.
-
 You need a Steam Frame with an internet connection, a keyboard (Bluetooth, or the on-screen one), and about 3 GB of free space.
 
 1. In the launcher, choose Launch a program → Desktop.
@@ -60,7 +58,7 @@ If you work in the desktop for long stretches, or leave the headset on a stand,
 | Meta+Shift+H in the desktop | Hides or shows all screens (also in the menu as Hide/Show Screens, and mappable). The Visibility & pins tab of Frametop Display Settings can instead show them only with the dashboard open, or while you look at your wrist |
 | Tap Meta, on any keyboard | Opens the Steam menu in the SteamVR dashboard, or closes the dashboard, wherever you are. The desktop's launcher is still on the taskbar and Alt+F1. Change it in Frametop Input Settings (Keyboard page), where any key combination or modifier tap can do a Frametop or Steam action, open a profile, or run a command of your own |
 | Switch a screen to Hidden (Frametop Display Settings, Visibility & pins → Screens shown) | Hides just that screen until you switch it back, whatever the other visibility settings say; new windows that would open on it float instead |
-| Play a VR game | Frametop pauses so the game gets the headset to itself (see [Pause for VR games](#pause-for-vr-games)): the screens hide and your controllers stay in the game. Click both thumbsticks together twice to bring Frametop back. With the automatic pause off, the screens still hide, and the SteamVR dashboard or Meta+Shift+H shows them; to keep them visible over games, change During VR games on the Visibility & pins tab |
+| Play a VR game | The screens hide and your controllers stay in the game. Open the SteamVR dashboard, or press Meta+Shift+H, to see and use them. To keep them visible over games, change During VR games on the Visibility & pins tab; the controllers still stay in the game, and you use the screens with the mouse or the dashboard |
 
 Restarting the desktop (Restart desktop in Frametop Display Settings) closes its windows, but background work you started in it, such as servers, tmux sessions, or builds, keeps running.
 
@@ -95,18 +93,6 @@ A profile is a named setup: where the screens are, with their sizes and pins, wh
 | Pick a profile under Arrangement and press Open profile | Switches to it: the screens move, open windows of its apps go to their places, and the apps that aren't open start. Nothing closes |
 | Pick a profile under Start in profile, or run its entry (Frametop: NAME) from SteamVR's Launch a program list | The desktop starts in that profile, or switches to it if it's running. A profile can also go on a key combination, mouse button, or controller button in Frametop Input Settings |
 
-### Pause for VR games
-
-Frametop pauses while a VR game runs, so the game gets the headset's CPU and GPU, and comes back a few seconds after the game ends. Paused, the screens hide and the desktop nearly stops drawing, but its windows stay open. Gaze mode's eye tracking stops, and so do remote desktop and hand tracking if they run. The mouse works as a plain mouse in SteamVR.
-
-| Do this | To get this |
-| --- | --- |
-| Click both thumbsticks together, twice | Pauses Frametop, or brings it back, in a game or not. You hear a short sound. The game sees the clicks too |
-| Start a VR game | Frametop pauses, and comes back 5 seconds after the game ends. Bring it back during the game, and it stays on until that game ends |
-| Map Pause/resume Frametop to a mouse button, key combination, or controller button | The same, from that button (Frametop Input Settings) |
-
-The Game optimization page of Frametop Input Settings turns the automatic pause off, changes the gesture, closes the desktop instead of hiding it (more for the game, but its windows close), and turns the sound off. From a terminal: `input/ft-pause on`, `off`, or `status`.
-
 ### Gaze mode (experimental)
 
 In gaze mode the pointer goes where you look, and the mouse or the keyboard does the last bit. The installer offers it (or run `gaze/run.sh install` later), and then our own eye tracker for it, which is more accurate than SteamVR's (or run `gaze/tracker/install.sh` later; it needs `sudo`). Gaze mode uses ours once it's installed, and SteamVR's until then. Turn it on and calibrate it on the Gaze page of Frametop Input Settings.
@@ -136,16 +122,15 @@ This is an early release, tested on one Steam Frame (SteamOS 0.3.0 build 2026092
 
 - A SteamOS or SteamVR update can break parts of it until Frametop catches up. After an update, run `cd ~/frametop && scripts/doctor.sh` in a terminal. It checks what Frametop needs from SteamOS, and says what changed since the versions you last marked as working and what to try. Once everything works, `scripts/doctor.sh --mark-good` records the versions. If something stops working, please report it.
 - The first install downloads 1–2 GB for the build container and compiles everything on the headset, which takes several minutes.
-- During a VR game, mapped controller buttons belong to the game, so they can't bring the screens up. The pause gesture still works: Frametop reads it without taking the thumbsticks from the game. With the automatic pause off, open the SteamVR dashboard, press Meta+Shift+H, or use a mapped mouse button instead.
+- During a VR game you can't show the screens with a controller button, because the game owns the buttons. Open the SteamVR dashboard, press Meta+Shift+H, or use a mapped mouse button instead.
-- Flatscreen games aren't detected as games, so they don't pause Frametop by themselves: click both thumbsticks twice to pause it. If your controllers end up working the screens instead of the game, set Controllers on the screens to "Only with the SteamVR dashboard open" (Frametop Display Settings, Visibility & pins tab).
+- Flatscreen games aren't detected as games. If your controllers end up working the screens instead of the game, set Controllers on the screens to "Only with the SteamVR dashboard open" (Frametop Display Settings, Visibility & pins tab).
 - Typing follows your last click. A controller click on a panel other than the screens (the dashboard, a Steam app) doesn't move typing there; click it with the mouse, or click a screen to bring typing back.
 - The screens don't draw a mouse cursor of their own. The 3D mouse's dot or SteamVR's laser shows where you're pointing.
 - Profiles reopen apps, not what the apps had open. Tabs, files, and folders are left to each app's own restore.
 - Dragging something from one panel to another (a screen and a floating window) works, but the dragged item's icon doesn't show while the pointer is between panels.
 - Gaze mode is only as good as its calibration, and that depends on how the headset sits on your face. If the pointer lands off after you adjust the headset, run Quick check or Calibrate on the Gaze page of Frametop Input Settings.
 - On SteamVR's Settings page, the 3D mouse shows a laser beam and a larger hit dot, like a controller. SteamVR doesn't tell other programs where that page is (unlike Steam's pages, such as Library), so the mouse used to miss most of it: clicks went through to a desktop screen behind, and the dot disappeared. As a workaround, on that page only, the laser starts near your eye and SteamVR finds the page itself. See docs/design.md.
-- Remote desktop over VNC (Frametop Remote Access in the app menu, or `./desktops.sh remote on`) needs Tailscale on the Frame. It shows the primary screen only. The app turns it on and off, shows the address, and shows, copies, or changes the VNC password. The password is made at random on the Frame and kept in `~/.config/frametop-remote` (only you can read it); VNC limits it to 8 characters, and the tailnet encrypts the connection. Turning it on in a desktop that started with it off takes a desktop restart. It costs almost nothing until a viewer connects; the picture then takes a few seconds to appear.
+- Remote desktop over VNC (Frametop Remote Access in the app menu, or `./desktops.sh remote on`) needs Tailscale on the Frame. It shows the primary screen only. The app turns it on and off, shows the address, and shows, copies, or changes the VNC password. The password is made at random on the Frame and kept in `~/.config/frametop-remote` (only you can read it); VNC limits it to 8 characters, and the tailnet encrypts the connection. Turning it on in a desktop that started with it off takes a desktop restart.
-- The desktop has no blur behind panels and menus, and no window animations, so it leaves the headset's GPU to SteamVR. Turn them back on in the Frametop desktop's System Settings (Desktop Effects, and Animation speed under General Behavior); Frametop won't turn them off again.
 - Turning the displays off on a stand only turns their backlight off. SteamVR has no way for other programs to put the headset in standby, so tracking and rendering keep running, and the headset draws nearly its full power.
 
 ## Reporting problems

display-settings/main.qml

@@ -802,7 +802,7 @@ Kirigami.ApplicationWindow {
                     Repeater {
                         model: [
                             { value: "hide", text: "Hide them unless the SteamVR dashboard is open", help: "The game has the view to itself; open the dashboard (or press Meta+Shift+H) to see the screens." },
-                            { value: "visible", text: "Keep them visible over the game", help: "They float over the game as they are outside it. Turn off Pause while a VR game runs in Frametop Input Settings (Game optimization), or Frametop pauses and hides them anyway." }
+                            { value: "visible", text: "Keep them visible over the game", help: "They float over the game as they are outside it." }
                         ]
                         delegate: ColumnLayout {
                             required property var modelData

docs/design.md

@@ -116,9 +116,9 @@ The driver starts disconnected, because holding the right-hand role while SteamV
 
 ### The cursor
 
-Mouse motion turns into yaw and pitch around an anchor, the head position at the last recenter. A ray from the anchor is tested against every visible overlay with `ComputeOverlayIntersection`. On a hit, the cursor sits on that surface; otherwise it floats at `POINTER_DISTANCE`. Since the anchor isn't your current eye position, a second test runs along your line of sight to the cursor point, and anything nearer wins, so the cursor always lands on what you see under it. Overlays in `POINTER_IGNORE` are left out of both tests. A display-only panel, like a performance overlay locked to your view, has no input method, so SteamVR's laser passes through it, but `ComputeOverlayIntersection` still hits it, and the cursor stuck to it. The laser starts just before the cursor point, so an ignored panel nearer to you doesn't catch it either. Both tests ask SteamVR about every visible overlay, so a frame where nothing moved (the mouse, the anchor, the eye by more than 5 mm, which overlays show) reuses the last result, for up to 100 ms, since overlays can also move on their own. The dots' overlay settings go to SteamVR only when they change, and the pose goes to the driver, which keeps the last one, only when the laser would land 0.1 mm or more elsewhere, and at least every 100 ms.
+Mouse motion turns into yaw and pitch around an anchor, the head position at the last recenter. A ray from the anchor is tested against every visible overlay with `ComputeOverlayIntersection`. On a hit, the cursor sits on that surface; otherwise it floats at `POINTER_DISTANCE`. Since the anchor isn't your current eye position, a second test runs along your line of sight to the cursor point, and anything nearer wins, so the cursor always lands on what you see under it. Overlays in `POINTER_IGNORE` are left out of both tests. A display-only panel, like a performance overlay locked to your view, has no input method, so SteamVR's laser passes through it, but `ComputeOverlayIntersection` still hits it, and the cursor stuck to it. The laser starts just before the cursor point, so an ignored panel nearer to you doesn't catch it either.
 
-OpenVR has no call to list other programs' overlays, so the helper runs `vrcmd --overlays` in the background. It includes hidden overlays, because a floating window's controls only appear while something hovers the window, and the cursor has to find them immediately. Each run is a shell and a new SteamVR client, about 30 ms of CPU, and it ran every second while the pointer was awake, which in gaze mode is all the time. Now it runs every 20 seconds, and at once when the pointer wakes, when the dashboard opens or closes, when a game starts or ends, and when a left click hits nothing (a panel that came up since). An overlay already on the list showing or hiding needs no new list: the helper checks the visibility of the ones it knows every 50 ms.
+OpenVR has no call to list other programs' overlays, so the helper runs `vrcmd --overlays` in the background. It includes hidden overlays, because a floating window's controls only appear while something hovers the window, and the cursor has to find them immediately.
 
 The laser starts partway along your line of sight to the cursor rather than at your eye. SteamVR sizes its hit dot by distance from the laser's origin, and a laser from the eye still shows a beam in each eye. Starting it close to the target makes the beam and the dot tiny, while `POINTER_ORIGIN_MARGIN` keeps the origin in front of the small window controls, which float a few centimetres in front of their panels. The helper's own white dot is the visible cursor. In empty space it's an interactive overlay that the laser lands on, so SteamVR never draws a laser into nothing.
 
@@ -141,7 +141,7 @@ Replacing a loaded driver's files, as re-running the installer used to do, leave
 
 The dashboard follows whichever device summoned it or last pressed its trigger. Frametop adds "last used wins": moving a real controller releases the pointer, and the next mouse movement takes the laser back. Moving means faster than 0.35 m/s or 2 rad/s (both times `POINTER_CONTROLLER_PICKUP`, 1 by default) for 100 ms in a row, while the controller is tracked normally. A single sample over the limit used to be enough, and controllers resting on a desk took the laser back on a knock or a tracking jump while the mouse was in use. Small movements don't count; waking needs `POINTER_WAKE_COUNTS` of mouse motion within a second, so desk jitter doesn't steal the laser. While the pointer is awake, a tiny transparent overlay with `MakeOverlaysInteractiveIfVisible` keeps SteamVR's laser mouse on, since otherwise the first click would only switch the laser on.
 
-When the headset comes off, SteamVR reports its activity level as idle at once and turns the displays off 5 seconds later (`power.turnOffScreensTimeout`), unless something keeps it awake. An awake pointer did, and so did the helper's `vrcmd` runs: each is a new SteamVR client, and a new client every second kept SteamVR out of standby. The helper now releases the pointer as soon as the headset is idle, ignores the mouse until you're wearing it again, and pauses the overlay list whenever the pointer is off. With the pointer off it also stops running its loop every 8 ms, about 116 wakeups a second for nothing: it waits up to 250 ms for a command on its socket (20 ms while it reads mapped controller buttons, which SteamVR input only offers by polling), and leaves the overlay lookups until the pointer wakes.
+When the headset comes off, SteamVR reports its activity level as idle at once and turns the displays off 5 seconds later (`power.turnOffScreensTimeout`), unless something keeps it awake. An awake pointer did, and so did the helper's `vrcmd` runs: each is a new SteamVR client, and a new client every second kept SteamVR out of standby. The helper now releases the pointer as soon as the headset is idle, ignores the mouse until you're wearing it again, and pauses the overlay list whenever the pointer is off.
 
 ### Moving floating windows
 
@@ -153,8 +153,6 @@ SteamVR opens every input device only when it starts. When a Bluetooth mouse sle
 
 Keyboards aren't grabbed by default, because a grabbed keyboard's keys went into a virtual keyboard nothing typed from; the relay forwards them to ft-screens instead.
 
-The relay never waits on the pointer helper. Its socket to the helper used to block, so when the helper stalled (a layout placement or `grabprobe` holds it for seconds, and the gaze service fills its socket 90 times a second meanwhile), the whole relay stopped with it: keyboards, the volume keys, and pausing. Now what the helper doesn't take waits in order and goes out on the next loops. Mouse moves add up into one while they wait, and a scroll notch is dropped, since scrolling seconds late is no use; presses and releases are kept, so no button stays down. Mouse motion goes to the helper at most every 4 ms, rather than once per report, which from a 1000 Hz mouse was 1000 datagrams a second to a helper that runs every 8 ms; a button sends the motion before it first, so the click lands where the pointer was.
-
 An ungrabbed keyboard reaches both sides at once. In VR, gamescope reads every input device itself (the SteamOS build's `InputStealer`, libinput with udev hotplug, so new devices too) and types into its focused app, and ft-screens types the same keys into the desktop. So Space in the desktop also paused Spotify on the dashboard. Typing now follows the last click. ft-screens sees clicks on its own screens, from the mouse or a controller. A click anywhere else is only visible for the mouse: overlay apps get SteamVR's `OverlayFocusChanged` (which panel the laser is on) but no controller button events, so the pointer helper reports the panel under the dot on each left press. ft-screens tells the relay where typing goes every second, from an unbound socket so the relay's replies can't loop back into its control socket, and the relay grabs pass-through keyboards while it's the desktop. A grab waits until the keyboard has no key down, so no key stays held on either side, and the relay lets go if ft-screens stops reporting. A program that reads every keyboard for a hotkey (a dictation tool, say) loses a grabbed keyboard. Repeating the keys on another input device doesn't work: gamescope reads that device too, whether it's the relay's virtual keyboard or one created later, and every Space, typed or dictated, paused Spotify again. So with `SHARE_KEYS=1` the relay sends a grabbed keyboard's keys to `@frametop_keys` as datagrams (`key <code> <value> <device name>`). It's off by default, because the relay can't tell who is listening: abstract sockets have no permissions, and any local process that binds the name first gets every key typed into the desktop, passwords included. A listener should accept only its own user (`SO_PASSCRED`) and skip any keyboard of its own that the relay grabs too.
 
 Volume keys must never reach gamescope. With the openvr backend, gamescope sends volume up and down to Steam by moving keyboard focus to Steam for the key and then back to the previously focused surface. When nothing had focus, the one it moves back to is null, and wlroots aborts on a null focus surface (`wlr_seat_keyboard_notify_enter: Assertion 'surface' failed`), which ends the whole VR session. Keyboard focus is often empty while you work in VR, so one press of the headset's volume button could take everything down. gamescope reads the headset's buttons and every keyboard itself (`InputStealer`), as do SteamVR's processes, so the relay has to stop volume keys at the device. Grabbing `gpio-keys` would also take the headset's click button, so the relay remaps the volume entries in each device's keymap (`EVIOCSKEYCODE`) and handles the stand-in codes itself. That fix covers every device at once, including keyboards that aren't grabbed.
@@ -179,12 +177,6 @@ The private runtime directory also moves the session's document portal to `$XDG_
 
 A podman container's monitor process (conmon) stays in the cgroup of whatever started the container, and `distrobox enter` starts it on demand. When a Frametop service happened to start the `dev` container, stopping that service stopped the container and everything in it, including the desktop's compositor. `scripts/container-up.sh` starts the container in a systemd scope of its own before anything enters it. It then waits for distrobox-init to log `container_setup_done`, as `distrobox enter` does only for containers it starts itself. A new container's first start takes a minute or more (it installs distrobox's dependencies and sets up passwordless sudo), and an install that entered right away met a sudo password prompt with no terminal to answer it ([#9](https://github.com/DeeJanuz/frametop/issues/9)).
 
-KWin renders with OpenGL through zink on Turnip, Vulkan on the same GPU vrcompositor needs to hit its frame time, and on the Frame that costs CPU too. The nested session started with KWin's defaults: blur and background contrast on (no `[Plugins]` group in its kwinrc) and animations at full length. Blur re-renders what's behind every translucent panel and menu each time it changes, and every animated frame is one more frame for KWin and ft-screens to draw and send. They're off by default in the Frametop desktop. The session script writes them before KWin starts, only where the desktop's own config has no value, once: System Settings deletes a setting put back to KDE's default rather than writing it, so without the marker in `frametoprc` a user who turned blur back on would lose it at the next start. The effect ids (`blur`, `contrast`) are the ones built into KWin 6.2.5 on SteamOS; KWin reads `<id>Enabled` from `[Plugins]`.
-
-The nested session also runs the system's XDG autostart entries, being a KDE session. Discover's update notifier started `plasma-discover --mode update` in it (520 to 620 MB resident and about 9% of a core, plus `flatpak-system-helper` and AppStream downloads), and IBus started a daemon, the kimpanel panel and its GTK extension that nothing can use: KWin hands text input to the one input method it starts (`ft-textinput`), and the session drops `QT_IM_MODULE`, `GTK_IM_MODULE` and `XMODIFIERS`. The session hides both for this desktop only, with `Hidden=true` copies in its own autostart folder. The geoclue demo agent stays: it's what answers apps' location requests to Geoclue outside GNOME, and it costs nothing while idle. Orca's entry only starts in GNOME-family desktops.
-
-Remote desktop is a chain (krdp, then FreeRDP inside Xvnc) because nothing on SteamOS serves KWin over VNC directly. Kept connected all the time, it cost about a core with nobody watching: krdpserver 55 to 78% (it encodes H.264 in software with openh264: VA-API finds no driver for the Frame's GPU in the container), FreeRDP 16 to 27%, Xvnc 6 to 11%, and the bridge's layout check every 5 seconds another 4%. krdp creates its screencast session per RDP connection and drops it when the connection closes (`SessionController::onNewConnection` in krdp 6.7), so an idle krdpserver costs nothing and can stay up; only the RDP connection has to go. The bridge connects FreeRDP when a VNC client appears and disconnects 45 seconds after the last one leaves. Xvnc has no hook for its clients, so the bridge counts established connections to its port with `ss`, woken early by Xvnc's log output; looking with `ss` once a second cost about 0.9% of a core in bash, against about 0.1% this way. `Xvnc -inetd` from a systemd socket would start a server per connection and lose sharing between viewers. The layout check (`ft-layout remote-view`, which scans `/proc` for plasmashell and runs `kscreen-doctor -j`) now runs only while FreeRDP runs, and then only after `kwinoutputconfig.json` or `frametop-layout.json` changes, with one check a minute in case a change touched neither.
-
 Program names stay within 15 characters, because Linux truncates process names there and the scripts find programs with `pgrep -x` and `pkill -x`. That's why the prefix is `ft-`.
 
 ## Displays off on a stand
@@ -197,15 +189,6 @@ Movement is judged within 10-second windows. On the mount, the head pose jittere
 
 Staying awake while charging uses Steam's own setting rather than a logind sleep inhibitor. Steam suspends with `dbus-send ... login1.Manager.Suspend boolean:true`, and a block inhibitor does stop that (`CanSuspend` answers "challenge" while one is held), but it stops the power button too. `system_idle_suspend_ac_sec` is field 24004 of Steam's CMsgClientSettings. In Steam's SharedJSContext, reachable over CDP on port 8080 because Steam runs with `-cef-enable-debugging`, `SteamClient.Settings.SetSetting` takes a change as a base64 protobuf, the way Steam's Power page sends it (0 is never), and `settingsStore.clientSettings` has the current values.
 
-## Pausing for VR games
-
-Hiding the screens during a game kept them out of view, but Frametop kept using the headset. Measured on 2026-10-02 with gaze mode off and no game running, in shares of one core: our eye tracker (ft-eyes) about 60%, ft-eyegrab, ft-gaze and ft-gazed about 3 to 4% each; remote desktop (krdpserver, FreeRDP, Xvnc) about 2 cores while it ran; KWin about 13%, ft-screens about 4%. The gaze service ran at full rate whether gaze mode was on or not; now it idles while the gaze isn't used (gaze/README.md), and pausing stops it outright. Reading SteamVR's eye tracking 90 times a second also made it restart every 10 to 13 seconds during Beat Saber, and each restart took input focus from the game, which paused it (PR #13; since then ft-gaze skips SteamVR's gaze action during games, but our own tracker kept running). So pausing stops what costs the most and leaves windows where they are.
-
-- A hidden screen still cost as much as a visible one. ft-screens sent every committed screen its frame callback at 90 Hz whether its overlay showed or not (since then, a hidden screen always gets one a second; see the frame rates in reference.md), so KWin kept drawing, and its apps with it. Paused, ft-screens sends the callbacks once a second. A Wayland client draws again only after its last frame's callback, so KWin's output stalls, KWin's own clients stop getting theirs, and the whole desktop idles, without anything losing its connection. A second's pace, rather than none, keeps any client that waits on a callback from waiting forever. Stopping KWin or the apps with SIGSTOP would free the same, but a Wayland peer that stops reading overflows the other side's 4 KB socket buffer, which ends the connection: that's how the live desktop died once when its KWin stalled (`Data too big for buffer`). They also sit in different cgroups (KWin under steam.service when the VR launcher starts it, ft-screens in the dev container's), so no single freeze stops them together.
-- The relay does the pausing because it's the one part that always runs, and the pointer helper keeps running because stopping it leaves its virtual controller connected with its last pose (the driver has no staleness timeout), maybe holding a hand role, with the 3D mouse dead. Releasing it does the job. The helper already checks for a scene app twice a second, so it's what tells the relay a game started.
-- The gesture has to work during a game, but SteamVR input reaches only the app with input focus, and an overlay with global input (`steamvr/globalActionSetPriority`) takes the buttons it binds from the game. vrserver's web socket on 127.0.0.1:27062, which its controller binding page uses for the live view, reports every controller component whatever has focus, and reading it takes nothing. The game sees the clicks too, so the default is a gesture games hardly use: both thumbsticks, together, twice. "Together" means within 0.3 seconds of each other, so a stick held down to sprint while the other clicks doesn't count. The stream is about 160 messages a second, nearly all capacitive sensing, so the reader parses only the few that mention a gesture's button. A controller's root path changes while the 3D mouse holds its hand role (`/devices/cv/<serial>` instead of `/user/hand/right`), so the reader looks the controllers up again (an HTTP request to vrserver): when the relay's 3D mouse connects or lets go, when a message comes from a device it doesn't know, and every 30 seconds. It used to be every 3 seconds.
-- Resuming starts remote desktop through `systemd-run --scope`: started straight from the relay, it would join the relay's cgroup and end with the next relay restart.
-
 ## SteamOS updates
 
 On the Frame, SteamVR is part of the OS image (`/opt/steamvr`, the `deckard-steamvr-rel` package), next to KWin, gamescope, and the kernel, so every SteamOS update can bring a new SteamVR too. Frametop survives updates: it lives in the home folder and the `dev` container, the Bluetooth fixes are in `/etc`, which SteamOS keeps across updates, and nothing goes into `/usr`. What an update can break is what Frametop uses from the image. The public OpenVR API is versioned and stays put. The rest is less certain: `IVRIPCResourceManagerClient`, which is newer than the header SteamVR ships; the text `vrcmd --overlays` prints; the eye tracker's shared memory layout; XRService's camera buffers; KWin's nested backend; and behavior Frametop works around, such as the SteamVR Settings page that `ComputeOverlayIntersection` can't find or the scale KWin's nested backend doesn't undo.

docs/profiles.md

@@ -46,8 +46,8 @@ So a "Work" profile can put three screens around you with a browser, two termina
 
 ## How it works
 
-- **Capture** (`ft-layout save NAME`, and Save as profile… in Display Settings). ft-layout captures the screens as before, then asks ft-floatd for the windows (`windows` on @frametop_float). ft-floatd has the KWin script report every window as it is now (`report-all`), then answers with every normal window: its desktop file name, the screen it's on, its rectangle there, and whether it's maximized. For floating windows it gives their panel's place, their size in pixels, and their scale. A window whose app id has no desktop file (a Flatpak app's X11 window can give its own: RustDesk's says `com.carriez.flutter_hbb`, its desktop file is `com.rustdesk.RustDesk`) is kept by the desktop file whose `StartupWMClass` names its window class. Windows with no desktop file name are kept by their process's command line (`/proc/<pid>/cmdline`) and window class. Windows of Plasma itself, the Frametop settings apps, and dialogs aren't recorded. If ft-floatd doesn't answer, the profile keeps the apps it had.
+- **Capture** (`ft-layout save NAME`, and Save as profile… in Display Settings). ft-layout captures the screens as before, then asks ft-floatd for the windows (`windows` on @frametop_float). ft-floatd has the KWin script report every window as it is now (`report-all`), then answers with every normal window: its desktop file name, the screen it's on, its rectangle there, and whether it's maximized. For floating windows it gives their panel's place, their size in pixels, and their scale. Windows with no desktop file name are kept by their process's command line (`/proc/<pid>/cmdline`) and window class. Windows of Plasma itself, the Frametop settings apps, and dialogs aren't recorded. If ft-floatd doesn't answer, the profile keeps the apps it had.
-- **Apply** (`ft-layout use NAME`, Open profile in Display Settings). ft-layout makes the profile's hidden screens the screens' own setting, arranges the screens (which hides and shows them: ft-screens' `conceal` and `reveal`), then has ft-floatd open the apps (`profile NAME`; ft-floatd reads the windows from the layout file). If the screens can't be arranged, for example with the headset off and no head pose, the apps still open: the screens stay where they are, the profile's hidden screens still hide, and floating windows go relative to the screens wherever they are. ft-floatd goes through the entries app by app. It claims windows of that app already open (oldest first, each claimed once), and moves each to its entry's place: onto its screen at its rect (or maximized), or floating at its pose. For the entries left over, it launches the app once (`ft-float launch`, the same path as Launch as Standalone) and waits up to 30 seconds for its first window. Each window that shows up goes to the next entry's place. Once the first window has been up for 3 seconds (time for an app that restores its own windows to show them), ft-floatd launches the app again for each entry still waiting, and waits up to 30 seconds more. New windows are matched to the launch by process (or a child of it), or by desktop file name (found the same way as at capture): single-instance and D-Bus-activated apps open their windows from a process that was already running.
+- **Apply** (`ft-layout use NAME`, Open profile in Display Settings). ft-layout makes the profile's hidden screens the screens' own setting, arranges the screens (which hides and shows them: ft-screens' `conceal` and `reveal`), then has ft-floatd open the apps (`profile NAME`; ft-floatd reads the windows from the layout file). If the screens can't be arranged, for example with the headset off and no head pose, the apps still open: the screens stay where they are, the profile's hidden screens still hide, and floating windows go relative to the screens wherever they are. ft-floatd goes through the entries app by app. It claims windows of that app already open (oldest first, each claimed once), and moves each to its entry's place: onto its screen at its rect (or maximized), or floating at its pose. For the entries left over, it launches the app once (`ft-float launch`, the same path as Launch as Standalone) and waits up to 30 seconds for its first window. Each window that shows up goes to the next entry's place. Once the first window has been up for 3 seconds (time for an app that restores its own windows to show them), ft-floatd launches the app again for each entry still waiting, and waits up to 30 seconds more. New windows are matched to the launch by process (or a child of it), or by desktop file name: single-instance and D-Bus-activated apps open their windows from a process that was already running.
 - **Default at start.** The session script runs `ft-layout start --wait 90`. That opens the profile in `FT_PROFILE` or `default_profile` (screens, then the apps once ft-floatd is up), or runs `apply --wait` if there's none. Start in profile on the Layout & profiles page sets `default_profile` (`ft-layout default NAME|none`). Plasma's session restore is turned off in the session (`ksmserverrc`: `loginMode=emptySession`).
 - **Launcher entries.** Each profile gets `~/.local/share/applications/frametop-profile-<name>.desktop` ("Frametop: Work"), written when it's saved and removed when it's deleted. They show in SteamVR's Launch a program list, the Application Launcher, and KRunner. Running one (`ft-layout open NAME`) switches to that profile if the desktop runs. Otherwise it starts the desktop with `FT_PROFILE` set (`systemd-run`, as `desktops.sh start` does), which overrides `default_profile` for that start. That needs SteamVR to be running.
 - **The action.** `profile:NAME` in the input relay (it runs `ft-layout use NAME`) for key combinations, mouse buttons, and controller buttons, with or without pointer mode. Input Settings lists one "Open profile NAME" action per profile.

docs/reference.md

@@ -18,16 +18,6 @@ desktops.sh start | stop | restart | status | log [lines]
 
 When the VR launcher starts the desktop, it inherits the Steam client's environment. The session script drops the client's runtime from it (`LD_LIBRARY_PATH`, the `STEAM_*` settings, and the Steam overlay's Vulkan layer), so apps in the desktop use the system's libraries, including its video codecs, just as they would after a normal login.
 
-KWin's blur and background contrast effects and its animations are off in this desktop, because KWin draws on the headset's GPU, which SteamVR needs. The session script turns them off once, the first time it starts (it leaves a setting you already have alone, and marks it done in `~/.config/frametop/frametoprc`), so turning them back on sticks. In the Frametop desktop, System Settings → Window Management → Desktop Effects has Blur and Background Contrast, and General Behavior has Animation speed. Or from a terminal, then restart the desktop:
-
-```
-kwriteconfig6 --file ~/.config/frametop/kwinrc --group Plugins --key blurEnabled true
-kwriteconfig6 --file ~/.config/frametop/kwinrc --group Plugins --key contrastEnabled true
-kwriteconfig6 --file ~/.config/frametop/kdeglobals --group KDE --key AnimationDurationFactor 1
-```
-
-Two of the system's autostart programs don't start in this desktop: Discover's update notifier (`org.kde.discover.notifier`), which starts Discover to check for updates, and IBus (`ibus`), which can't reach the desktop's apps because KWin's input method is `input/ft-textinput`. The session script puts copies with `Hidden=true` in `~/.config/frametop/autostart` once (marked in `frametoprc`), and skips a name you already have a file for. Delete a copy to start that program again.
-
 Settings are in two files, and Frametop Display Settings edits both. The screens (resolution, width in metres, scale, curve, which one has the taskbar) and their layout are in `~/.config/frametop-layout.json`. The backend, remote desktop, and pointer settings are in `~/.config/frametop.conf`; `session/frametop.conf.example` lists every key.
 
 Restarting the desktop closes its windows. Before the unit stops, `session/keep-apps.sh` moves every program started in the desktop into a systemd scope of its own, so background work such as servers, tmux, and builds keeps running. An app that shuts down its own helper processes when its window closes will still lose them; run that kind of work outside the desktop, for example as a systemd user service.
@@ -76,15 +66,12 @@ place N x y z yaw pitch roll     width N metres          curve N radius|on|off
 pin N|all left|right|head [matrix]    unpin N|all        size N w h
 get N    screens    head    state    key code value    scale N s    vrkeyboard show|hide|toggle|close
 visibility always|dashboard|gesture|toggle    wrist degrees    gesture left|right degrees
-hide | show | toggle    controllers always|outside_games|dashboard    ingames hide|visible    pause on|off|state
+hide | show | toggle    controllers always|outside_games|dashboard    ingames hide|visible
 conceal N|all    reveal N|all    concealed    cutouts on|off|state    cutouts predict on|off    cutouts lead ms
 float N mpp x y w h title    unfloat N    pose N matrix    sub N k x y w h | sub N k off    minimized N 0|1    carry N
-rates focused in_view hidden    rates?    watch seconds    phase ms
 ```
 
-Each screen draws at a frame rate for how much of it you see. KWin draws a screen only after ft-screens gives it a frame callback, and its apps wait for theirs, so the rate of callbacks is the screen's frame rate, for KWin and the apps on it alike. A screen is focused while you look at it (within 12 degrees of where your head points), while a laser or the mouse is on it or was in the last 1.5 seconds, while it's carried, and while you type on it; it gets every display frame. The rest of what you can see (within 60 degrees) gets 15 frames a second, and a hidden screen, one behind you, and everything while paused get one a second. A level goes up at once and comes down after a moment (1.5 s from focused, 0.5 s from in view). A video, or anything moving over a large part of a screen (6% or more of it, redrawn on 8 commits in a row, 10 or more a second), keeps every frame while in view. A floating window is a screen of its own here. Nothing that stands still costs anything at any rate: KWin sends a frame only when something on the screen changed. `rates F V H` sets the three rates in Hz (0: every display frame; default `0 15 1`, also `ft-screens --rates 0,15,1`), and `rates?` shows them, the display's rate, and for each screen its level, its milliseconds between frames, and whether it counts as a video. `watch S` gives every screen full rate for S seconds: remote desktop renews it while a VNC client is connected, since a viewer sees what KWin draws. The ticks (SteamVR events and the callbacks) come once per display frame, 1 ms after the vsync (`phase ms` changes that, for tuning), in step with the display rather than on a timer that drifted through the frame.
+`conceal` and `reveal` hide and show one screen on its own (`ft-layout hide` and `show` send them), and `concealed` lists those screens. `cutouts` turns the hand cutouts on and off (`ft-handsctl cutouts`). The last line is ft-floatd's, for floating windows: N is a floating window's panel, numbered on from the screens, one per spare output. `float` gives the window's rectangle in its output, metres per pixel, and the title bar's height, and shows the panel; `unfloat` hides it. `pose` places it (a 3x4 matrix, standing universe), `sub` shows popup or dialog k over it, `minimized` hides it while its window is minimized, and `carry` moves it with the laser that pressed the window's own title bar.
-
-`conceal` and `reveal` hide and show one screen on its own (`ft-layout hide` and `show` send them), and `concealed` lists those screens. `pause on` (from the input relay, when Frametop pauses for a VR game) hides every screen and floating window whatever else says, and slows the desktop down; `pause off` undoes it. `cutouts` turns the hand cutouts on and off (`ft-handsctl cutouts`). The last line is ft-floatd's, for floating windows: N is a floating window's panel, numbered on from the screens, one per spare output. `float` gives the window's rectangle in its output, metres per pixel, and the title bar's height, and shows the panel; `unfloat` hides it. `pose` places it (a 3x4 matrix, standing universe), `sub` shows popup or dialog k over it, `minimized` hides it while its window is minimized, and `carry` moves it with the laser that pressed the window's own title bar.
 
 ## Input relay
 
@@ -127,12 +114,11 @@ The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `P
 
 ## Frametop Input Settings
 
-A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has nine pages:
+A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has eight pages:
 
 - Devices lists every USB and Bluetooth mouse and keyboard, with a light that flashes when the device is used. Each device gets a role: 3D pointer (grabbed, drives the pointer; the default for anything with a mouse), Pass through (grabbed only while typing goes to the desktop; the default for keyboards, whose key combinations work everywhere), or Ignore. A device is identified by its Bluetooth address, or its USB ids and name, so all of its input nodes share one role. Forget drops everything saved for a device.
-- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, gaze pointer on or off, gaze precision, gaze drag, gaze quick check, faster or slower, reset the screen layout, hide or show the screens, open or close the keyboard, float a window in VR or put it back, put all floating windows back, pause or resume Frametop ([Pausing for VR games](#pausing-for-vr-games)), Open profile NAME (one per profile, [profiles.md](profiles.md)), pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
+- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, gaze pointer on or off, gaze precision, gaze drag, gaze quick check, faster or slower, reset the screen layout, hide or show the screens, open or close the keyboard, float a window in VR or put it back, put all floating windows back, Open profile NAME (one per profile, [profiles.md](profiles.md)), pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
 - Controllers maps the Frame controllers' buttons (every button but the system button) to the same actions, except passing a key through and the gaze actions: gaze mode is a mouse and keyboard feature ([gaze-controllers.md](gaze-controllers.md)). Capture a button and press it on a controller, or pick it from the list. The controllers aren't input devices on the host; only SteamVR sees them. So the pointer helper reads them with SteamVR input (`pointer/helper/vrbuttons.h`, `pointer/helper/actions/`) and sends presses to the relay (`vrbtn right/a 1`), which does the mapped action. The helper only takes the buttons that are mapped (the relay tells it with `vrbind`), at an overlay-global priority, and only while no game (scene application) runs, so games keep every button; with In games on (`controller_in_games`), a mapped button is taken from games too. That needs SteamVR's "Enable global input from overlays (Experimental)" setting (`steamvr/globalActionSetPriority`), which the page's Global input switch turns on and off. Mappings are saved as `controller_buttons` in `~/.config/frametop-input.json`.
-- Game optimization has the pause for VR games: its state with Pause now or Resume, whether VR games pause Frametop by themselves, the controller gesture (one or two buttons, pressed once or twice; one button always takes two presses), what happens to the desktop, and the sound. They're saved as `pause_auto`, `pause_gesture`, `pause_desktop`, and `pause_sound` in `~/.config/frametop-input.json`. See [Pausing for VR games](#pausing-for-vr-games).
 - Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput, like frame-voice's, don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected. Its Key combinations section maps modifiers plus a key, or one modifier tapped on its own, on any keyboard, to any action but passing a key through or nothing, or to Run a command…: a command line the input relay runs with `sh -c` when you press the keys (`command:CMD`). The command runs as the relay's user service, outside the desktop's session, with `layout/`, `float/` and `steam/` on its `PATH` (so `ft-layout use Work` or `ft-float launch org.kde.dolphin` work as they are), and its output goes to the relay's journal. The gaze clicks (Gaze left click and Gaze right click) only go on key combinations. The defaults are a Meta tap (open the Steam menu, or close the dashboard), Meta+J (gaze left click), Meta+K (gaze right click), and Meta+Shift+F (float window in VR or put it back); remove them or add others there. A tap is a press and release with no other key, mouse button, or scroll in between; a bound one sends the desktop F24 before the release, so Plasma's launcher doesn't open on it. The combination's last key isn't typed, and the modifiers still reach the app; while typing goes to Steam rather than the desktop, keyboards aren't grabbed, so Steam or the game sees the keys too. They're saved as `key_bindings` in the same file; a file with its own list, even an empty one, gets no defaults.
 - Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button.
 - Ignored panels lists the SteamVR overlays that are showing, grouped by app (the first two parts of the overlay key, such as `sasaken.frame-perf-overlay`), from the pointer helper (`overlays`). Tick a panel, or Ignore the whole app, and the pointer passes through it to what's behind. It's for panels you only look at, like a performance overlay that follows your view. The list is saved as `POINTER_IGNORE` in `~/.config/frametop.conf`: comma-separated overlay keys, where a shell pattern like `vendor.app*` covers a whole app, including panels it opens later. The helper reloads at once. Frametop's own screens aren't listed, and entries for apps that aren't open are listed below, to remove.
@@ -213,36 +199,9 @@ power/run.sh off | on      # the displays off now, or back on
 power/run.sh log
 ```
 
-## Pausing for VR games
-
-Paused, Frametop leaves the headset's CPU and GPU to a VR game. The input relay does it (`input/game_pause.py`), since it's the one part that always runs:
-
-- The gaze service stops (`frametop-gaze`: ft-gazed, ft-gaze, our own eye tracker, the gaze panel), so nothing reads SteamVR's eye tracking. Our frame grabber, the root service `ft-eyegrab`, goes idle by itself 3 seconds after our eye tracker stops asking it for frames.
-- Hand tracking stops if it runs (`frametop-camd`, `frametop-hands`).
-- The desktop, as the Game optimization page of Frametop Input Settings says (`pause_desktop`): hidden (the default) or closed. Hidden, ft-screens hides every screen and floating window whatever the visibility mode, the hotkey, or the dashboard says, and gives KWin a frame callback once a second instead of every display frame. KWin draws a screen only after its frame callback, and its apps wait for theirs, so the desktop hardly draws, but its windows stay open. Remote desktop stops if it runs (`session/remote-ctl.sh`). Closed, `desktops.sh stop` closes the desktop and its windows, and resuming starts it again (about 12 seconds), in its start profile if it has one.
-- The relay lets go of the 3D mouse and feeds pointer devices to its virtual mouse and keyboard, as with `POINTER=0`. Typing goes to Steam. Mapped buttons and key combinations do nothing but pausing, the Steam menu, and commands; a key combination that does nothing is typed as usual.
-
-Resuming starts again only what pausing stopped, and plays a second sound. The pointer helper and ft-powerd keep running: they cost little, the helper is what says a game started, and stopping it would leave its virtual controller connected with its last pose.
-
-Ways to pause and resume:
-
-- The controller gesture, by default both thumbsticks clicked together twice: both go down within 0.3 seconds of each other, and the second time within 0.7 seconds of the first. The relay reads it from vrserver's web socket (`input/vrws.py`), which works whatever has input focus and takes nothing from the game, so the game sees the clicks too. The Game optimization page changes it: one or two of the buttons the Controllers page lists, pressed once or twice (one button always takes two), or none.
-- The Pause/resume Frametop action, on a mouse button, a key combination, or a controller button (outside games, like every mapped controller button).
-- VR games, with Pause while a VR game runs on (`pause_auto`, the default). The pointer helper tells the relay when a scene app starts and ends (`vrgame 1|0`, repeated every 5 seconds). A game starting pauses Frametop. A pause that starts while a game runs ends 5 seconds after the game does, unless another game starts first. Resumed during a game, Frametop stays on until that game ends. A pause that starts outside a game lasts until you resume. Flatscreen games aren't scene apps, so they don't pause it.
-- From a terminal or a script:
-
-```
-input/ft-pause on | off | toggle   # pause or resume
-input/ft-pause status              # the state as JSON (the relay's "pause ?")
-input/vrws.py 10                   # the controllers' buttons from vrserver's web socket, for 10 s
-input/test/pause-test.py           # the gesture and the automatic pause, offline
-```
-
-The state outlives a relay restart, in `/run/user/UID/frametop-pause.json`. A SteamVR restart while paused starts the gaze service with it, and the relay stops it again when the pointer helper comes back.
-
 ## Gaze pointer (experimental)
 
-In gaze mode the 3D mouse's pointer goes where you look, and the mouse or the keyboard does the last bit. It needs the gaze service, which `install.sh` offers (yes by default) and `gaze/run.sh install` installs on its own: it builds it and runs `gaze/ft-gazed` as `frametop-gaze.service`, which starts with SteamVR. The service idles while the gaze isn't used: its eye tracker reader and our own eye tracker run only while gaze mode is on and someone wears the headset, while a check or the calibration runs, or while the Gaze page of Frametop Input Settings is open, and stop 30 seconds after ([gaze/README.md](../gaze/README.md)). Turn gaze mode on with the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1`, or a button or key combination mapped to Gaze pointer on/off.
+In gaze mode the 3D mouse's pointer goes where you look, and the mouse or the keyboard does the last bit. It needs the gaze service, which `install.sh` offers (yes by default) and `gaze/run.sh install` installs on its own: it builds it and runs `gaze/ft-gazed` as `frametop-gaze.service`, which starts with SteamVR. Turn gaze mode on with the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1`, or a button or key combination mapped to Gaze pointer on/off.
 
 - Meta+J left-clicks and Meta+K right-clicks where you look. A quick tap clicks where the dot was at the press. Hold instead, and the dot stays put in your view: turn your head until it's on what you meant, and let go to click there. Held still for `POINTER_GAZE_HOLD` (0.5 s), the press becomes a real one, and your head drags. Meta+K with Meta+J held presses where the dot is now, to drag from there, and a second Meta+K during that drag (a double Meta+K) pans and tilts what you're dragging while it's held.
 - The mouse's buttons work the same way, with the mouse steering instead of your head (`POINTER_GAZE_MOUSE=precision`, the default): the right button with the left held starts a drag, and a double right click pans and tilts what you're dragging. With `POINTER_GAZE_MOUSE_MOVE=held`, the default, the mouse only corrects: while the gaze has the pointer, moving it does nothing unless a button is held. `free` lets the mouse take the pointer any time. With the gaze stale for a second, in a game, or with the headset off, the mouse works as usual.
@@ -272,8 +231,6 @@ It listens on port 5900 on the Frame's Tailscale address only, not the LAN, so i
 
 No VNC server can capture KWin on SteamOS directly: `krfb` needs `xdg-desktop-portal-kde`, which SteamOS doesn't ship, and `wayvnc` only works with wlroots compositors. So `session/remote-desktop.sh` captures the desktop with KDE's `krdpserver --plasma` on `127.0.0.1:3390`, and `session/vnc-bridge.sh` runs TigerVNC's `Xvnc` on display `:20` with a FreeRDP client inside it and serves that. Both run in the `dev` container, and the extra hop adds a little latency. krdp streams every screen; the VNC screen is the primary's size, and the FreeRDP window is shifted so the primary fills it (`ft-layout remote-view` gives the offset). krdp's own `--monitor` would stream just one screen, but it maps the pointer as if that screen sat at 0,0, so clicks would miss. When the layout changes, the VNC screen resizes and FreeRDP reconnects within a few seconds.
 
-FreeRDP runs only while a VNC viewer is connected, because while it's connected krdp captures and encodes every redraw. With no viewer, krdp has no RDP connection and so captures nothing, and Xvnc shows a black screen. When a viewer connects, the bridge starts FreeRDP, and the desktop appears about 3 seconds later; FreeRDP stops 45 seconds after the last viewer leaves (`VNC_IDLE_SEC` in the bridge's environment). The bridge looks for viewers with `ss` whenever Xvnc logs something, as it does for every connection, and every 5 seconds otherwise. While a viewer is connected, the bridge asks ft-screens to draw every screen at full rate (`watch 15` on `@ft_screens`, renewed every 5 seconds), so screens you aren't looking at in the headset, or a headset on a stand, don't stream at a low rate. It reads the primary screen's place again (`ft-layout remote-view`) only while FreeRDP runs, after `~/.config/frametop/kwinoutputconfig.json` or `~/.config/frametop-layout.json` changes, and once a minute.
-
 With remote access on, the nested KWin runs with `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1` and `KWIN_SCREENSHOT_NO_PERMISSION_CHECKS=1`, so any app in the Frametop desktop could capture its screens or inject input. The second one lets scripts take screenshots through KWin's `org.kde.KWin.ScreenShot2` D-Bus interface. This applies only to that desktop, not the stock one. Port 3389 is SteamOS's own `xrdp`, which starts a separate X11 session rather than showing the VR desktop.
 
 ## Limits

float/ft_floatd.py

@@ -229,24 +229,6 @@ def cmdline(pid):
         return []
 
 
-def desktop_name(app, cls):
-    """The desktop file name for a window's app id and class. A Flatpak app's window can give an
-    id with no desktop file: RustDesk's (an X11 window) says com.carriez.flutter_hbb, and its
-    desktop file is com.rustdesk.RustDesk. That file's StartupWMClass names the window's class
-    then. The app id itself when nothing matches."""
-    try:
-        if not app or Gio.DesktopAppInfo.new(app + ".desktop"):
-            return app
-    except TypeError:  # PyGObject raises for the NULL a missing desktop file returns
-        pass
-    names = {n.lower() for n in (app, cls) if n}
-    for info in Gio.AppInfo.get_all():
-        wm = info.get_startup_wm_class() if isinstance(info, Gio.DesktopAppInfo) else None
-        if wm and wm.lower() in names:
-            return info.get_id().removesuffix(".desktop")
-    return app
-
-
 class Launch:
     """An app we started: its first window floats, or (for a profile) its windows go to the
     profile's entries for it, in order."""
@@ -763,10 +745,7 @@ class Daemon:
         Returns (reply, pid or None)."""
         if app:
             app = app.removesuffix(".desktop")
-            try:
+            info = Gio.DesktopAppInfo.new(app + ".desktop")
-                info = Gio.DesktopAppInfo.new(app + ".desktop")
-            except TypeError:  # PyGObject raises for the NULL a missing desktop file returns
-                info = None
             if info is None:
                 return f"error no app {app}", None
             pids = []
@@ -797,10 +776,8 @@ class Daemon:
             return None
         chain = pid_chain(int(ev.get("pid") or 0))
         app = ev.get("app", "")
-        # (An X11 window in a Flatpak gives its pid in the sandbox: only its app id can match.)
-        apps = {app, desktop_name(app, ev.get("cls", ""))} if app and any(la.app for la in self.launches) else {app}
         for la in self.launches:
-            if any(p in chain for p in la.pids) or (la.app and app and la.app in apps):
+            if any(p in chain for p in la.pids) or (la.app and app and la.app == app):
                 if not la.entries or len(la.entries) <= 1:
                     self.launches.remove(la)
                 return la
@@ -864,7 +841,7 @@ class Daemon:
         for wid, ev in self.windows.items():
             if not recordable(ev):
                 continue
-            entry = {"app": desktop_name(ev["app"], ev.get("cls", ""))} if ev.get("app") else {"cmd": cmdline(ev.get("pid")), "class": ev.get("cls", "")}
+            entry = {"app": ev["app"]} if ev.get("app") else {"cmd": cmdline(ev.get("pid")), "class": ev.get("cls", "")}
             if not entry.get("app") and not entry.get("cmd"):
                 continue
             f = self.floats.get(wid)
@@ -891,7 +868,7 @@ class Daemon:
         return out
 
     def window_key(self, ev):
-        return desktop_name(ev.get("app", ""), ev.get("cls", "")) or json.dumps(cmdline(ev.get("pid")))
+        return ev.get("app") or json.dumps(cmdline(ev.get("pid")))
 
     def open_profile(self, name):
         """Open a profile's apps (additive: nothing closes)."""
@@ -907,9 +884,9 @@ class Daemon:
             if key != "[]":
                 groups.setdefault(key, []).append(e)
         claimed = set()
-        keys = {wid: self.window_key(ev) for wid, ev in self.windows.items() if recordable(ev)}
         for key, entries in groups.items():
-            have = [ev for wid, ev in self.windows.items() if wid not in claimed and keys.get(wid) == key]
+            have = [ev for wid, ev in self.windows.items()
+                    if wid not in claimed and recordable(ev) and self.window_key(ev) == key]
             for e, ev in zip(entries, have):
                 claimed.add(ev["id"])
                 self.place_entry(ev, e)

gaze/README.md

@@ -2,7 +2,7 @@
 
 The Steam Frame's eye tracking as pointer input: a gaze mode for the 3D mouse (the pointer goes where you look, and the mouse does the last bit), and the tools to calibrate and measure it.
 
-- `ft-gaze` (C++, OpenVR, runs in the dev container) reads the eye tracker and prints one JSON line per sample (90 Hz). For each source, it gives the gaze direction relative to the head and the Frametop screen pixel it lands on. The gaze service asks it for only the sources it uses (`--sources`, and `sources LIST` on its stdin): our tracker and mmap set 1 with Own tracker, about 700 bytes a line instead of 1.3 KB, and every source while a check or the calibration runs. So SteamVR's gaze action, the only source that calls into vrserver (twice a sample), is read only then. The probe gets them all.
+- `ft-gaze` (C++, OpenVR, runs in the dev container) reads the eye tracker and prints one JSON line per sample (90 Hz). For each source, it gives the gaze direction relative to the head and the Frametop screen pixel it lands on.
 - `gazecal.py` has what the probe and the gaze service share: the correction models, filters, and the reader for SteamVR's eye tracking log.
 - `tracker/` is our own eye tracker, an alternative to SteamVR's: `ft-eyes` finds the pupils and glints in the eye-camera frames that `ft-eyegrab` (a small root service) copies out of SteamVR's tracker. See "Our own eye tracker" below.
 - `probe/ft-gazeprobe` (GTK 4, host Python) is a fullscreen playground, for developing the gaze tracking: day to day, the calibration and the checks run in the headset panel (Quick check, Calibrate, and Check headset fit on the Gaze page). It runs ft-gaze, draws where you're looking, measures accuracy, and tries out hold-to-adjust clicking with a calibration that learns from your adjustments.
@@ -35,7 +35,6 @@ Gaze as an input method for the whole desktop, without replacing anything of Ste
 - **Learning from nudges:** if the mouse took the pointer from the gaze and moved it (0.2 degrees or more, and the correction within `POINTER_GAZE_NUDGE_MAX`: 55 degrees by default, half of the 109 the headset shows across, and 1 to 110; the same limit for mouse, keyboard, and pinch clicks) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the quick check's dot does the same). A correction past `POINTER_GAZE_NUDGE_MAX` isn't learned: the helper asks ft-gazed for the quick check instead ("recheck", after its 2-minute cooldown). Tested on our tracker's 409 clicks since its Sep 29 calibration: a one-dot check set from any one of them put the next 2 minutes' clicks within 15 degrees (99% within 4.2) and the next 10 minutes' within 25 (the far ones after the headset moved), so the check gets back well under it. The limit used to be 8 degrees, and live on 2026-10-01 our tracker was 12 off after the headset went on, so every correction was dropped. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
 - **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
 - Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
-- **Idle while the gaze isn't used:** ft-gaze and our own tracker run only while gaze mode is on and someone wears the headset (the pointer helper says both: SteamVR drops the headset's activity level as soon as it comes off), while a check or the calibration is open or asked for, or while the Gaze page of Frametop Input Settings is open (it renews a `wake` lease). 30 seconds after the last use they stop, and our frame grabber goes idle with our tracker. With our tracker, that saves over half a core: on 2026-10-02, with gaze mode off, ft-eyes took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. A check asked for while it idles starts the tracker and opens once it sends. When the gaze is used again, it takes a few seconds to come back, and our tracker's first click re-seats it, as after the headset was off: so the quick check opens when gaze mode comes on after the service idled, as it does when you put the headset on. `ft-gazectl status` says `"awake"`, and `"idle"` says why it isn't. A stand that covers the proximity sensor makes the headset seem worn, so with gaze mode on it doesn't idle there.
 
 Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction, the correction at the time, and how far off it was.
 
@@ -63,7 +62,7 @@ Ground rules, for anyone changing it:
 - **Clean room.** Nothing of Valve's goes in: we don't decompile, disassemble, or patch the `eyetracking` binary or its network weights, and we don't copy their code or weights. Its public output (eye-server.mmap, read-only) is fair game as a baseline and as labels, and so are published papers and openly licensed pupil detectors (check each one's license: PuRe, PuReST, ElSe, and ExCuSe are non-commercial only).
 - **Root only reads.** ft-eyegrab never writes to, stops, or signals the `eyetracking` process, vrserver, or vrcompositor, never opens `/dev/adsp`, `/dev/cdsp`, or `/dev/spidev0.1`, and never writes to `/dev/shm/eye-server.mmap` (it also carries calibration clicks into SteamVR's tracker), `/opt`, or `/persist`.
 - **Eye images are biometric data.** Recordings live outside the repo, in `~/.local/share/frametop/eyes/captures` (0700), and `.gitignore` catches stray frame dumps. They go nowhere but the machine that runs your offline jobs.
-- **Mind the headset's budget.** Finding a pupil takes about 0.4 ms a frame while ft-eyes follows it, and 1.4-2.1 ms when it searches the whole frame. ft-eyes keeps OpenCV and numpy to one thread: their pools of one per core spun idle workers at about a quarter of a core, for frames this small. It also runs at nice 10 with SCHED_BATCH, and ft-gaze at nice 5 (not batch, since each sample goes on to the pointer): both run in the dev container's podman scope, out of reach of the gaze service's unit, on the cores vrcompositor and vrserver use at nice 0. Replays, scoring, and training go to a PC.
+- **Mind the headset's budget.** Finding a pupil takes about 0.4 ms a frame while ft-eyes follows it, and 1.4-2.1 ms when it searches the whole frame. Replays, scoring, and training go to a PC.
 
 ## Headset fit
 

gaze/ft-gaze.cpp

@@ -4,14 +4,7 @@
 // Every eye tracker sample (90 Hz) becomes one JSON line on stdout with each gaze source
 // hit-tested against the Frametop screens:
 //
-// Options: -v (log action errors), --watch-stdin (quit when stdin closes; until then, a line
+// Options: -v (log action errors), --watch-stdin (quit when stdin closes).
-// "sources LIST" on stdin switches the sources as --sources does), --sources LIST (comma-
-// separated: action, mmap1, mmap2, left, right, own, and eye for EYE; or all, the default).
-// Sources left out are read not at all and print as {"ok":0} ("eye" as null), so every line
-// keeps the same keys. The gaze service asks for the ones it uses (own and mmap1 with our
-// tracker), and all of them while a check or the calibration runs. Only the action costs
-// SteamVR anything (two calls into vrserver per sample), and a line with every source is
-// about 1.5 KB, 130 KB a second through podman's relay.
 //
 //   {"t":<sample time, CLOCK_MONOTONIC_RAW s>,"age":<ms old when read>,"n":<sample counter>,
 //    "head":{"yaw":..,"pitch":..,"hit":HIT},          head forward ray (for head nudging)
@@ -65,7 +58,6 @@
 
 #include <algorithm>
 #include <atomic>
-#include <cerrno>
 #include <chrono>
 #include <climits>
 #include <cmath>
@@ -80,7 +72,6 @@
 
 #include <fcntl.h>
 #include <sys/mman.h>
-#include <sys/resource.h>
 #include <sys/socket.h>
 #include <sys/stat.h>
 #include <sys/un.h>
@@ -398,30 +389,6 @@ std::string SrcJson(const std::vector<Screen> &screens, const vr::HmdMatrix34_t
     return buf + extra + "\"hit\":" + HitJson(screens, head, yaw, pitch) + "}";
 }
 
-// Sources (--sources, "sources LIST" on stdin): a bit each.
-enum : unsigned { kAction = 1, kMmap1 = 2, kMmap2 = 4, kLeft = 8, kRight = 16, kOwn = 32, kEye = 64, kAll = 127 };
-
-bool ParseSources(const std::string &list, unsigned &mask) {
-    static const struct {
-        const char *name;
-        unsigned bit;
-    } names[] = {{"action", kAction}, {"mmap1", kMmap1}, {"mmap2", kMmap2}, {"left", kLeft},
-                 {"right", kRight},   {"own", kOwn},     {"eye", kEye},     {"all", kAll}};
-    unsigned m = 0;
-    size_t at = 0;
-    while (at <= list.size()) {
-        const size_t comma = std::min(list.find(',', at), list.size());
-        const std::string name = list.substr(at, comma - at);
-        bool known = false;
-        for (const auto &n : names)
-            if (name == n.name) m |= n.bit, known = true;
-        if (!known) return false;
-        at = comma + 1;
-    }
-    mask = m;
-    return true;
-}
-
 // Head poses of the last half second, so a sample can use the pose at its own time.
 class PoseHistory {
 public:
@@ -459,42 +426,17 @@ std::string ExeDir() {
 
 int main(int argc, char **argv) {
     bool verbose = false, watchStdin = false;
-    std::atomic<unsigned> sources{kAll};
     for (int i = 1; i < argc; ++i) {
         if (std::strcmp(argv[i], "-v") == 0) verbose = true;
         if (std::strcmp(argv[i], "--watch-stdin") == 0) watchStdin = true;
-        if (std::strcmp(argv[i], "--sources") == 0 && i + 1 < argc) {
-            unsigned m;
-            if (ParseSources(argv[++i], m))
-                sources = m;
-            else
-                std::fprintf(stderr, "ft-gaze: --sources %s: unknown source (all used)\n", argv[i]);
-        }
     }
-    // Nice 5, before any thread starts (they inherit it): we run in the dev container's podman
-    // scope, beside vrcompositor and vrserver at nice 0, and the gaze service's unit doesn't
-    // reach us. Not SCHED_BATCH, as ft-eyes is: that would let each wakeup wait out another
-    // task's turn, and each sample goes on to the pointer.
-    errno = 0;
-    const int nice0 = getpriority(PRIO_PROCESS, 0);
-    if (errno == 0 && nice0 < 5 && setpriority(PRIO_PROCESS, 0, 5) != 0)
-        std::fprintf(stderr, "ft-gaze: nice: %s\n", std::strerror(errno));
     // --watch-stdin: quit when stdin closes. The probe runs us through distrobox, which
     // passes neither its signals nor a closed stdout on to us, but does pass stdin's end.
-    // Lines on stdin until then: "sources LIST".
     std::atomic<bool> stdinClosed{false};
     if (watchStdin)
-        std::thread([&stdinClosed, &sources] {
+        std::thread([&stdinClosed] {
             char c[256];
-            std::string line;
+            while (read(0, c, sizeof c) > 0) {
-            ssize_t n;
-            while ((n = read(0, c, sizeof c)) > 0) {
-                line.append(c, size_t(n));
-                for (size_t nl; (nl = line.find('\n')) != std::string::npos; line.erase(0, nl + 1)) {
-                    unsigned m;
-                    if (line.compare(0, 8, "sources ") == 0 && ParseSources(line.substr(8, nl - 8), m)) sources = m;
-                }
-                if (line.size() > 4096) line.clear();  // no newline in sight: not ours
             }
             stdinClosed = true;
         }).detach();
@@ -567,7 +509,6 @@ int main(int argc, char **argv) {
 
         if (fresh && hp.bPoseIsValid) {
             lastEmit = now;
-            const unsigned want = sources;
             const auto list = screens.Get();
             const vr::HmdMatrix34_t &headNow = hp.mDeviceToAbsoluteTracking;
             vr::HmdMatrix34_t headThen = headNow;
@@ -576,7 +517,7 @@ int main(int argc, char **argv) {
             // SteamVR's action: a room-space origin and fixation point, turned into the head
             // frame so every source reports the same kind of angles.
             std::string action = "{\"ok\":0}";
-            if (!inGame && (want & kAction)) {
+            if (!inGame) {
                 vr::VRActiveActionSet_t active{};
                 active.ulActionSet = set;
                 active.nPriority = vr::k_nActionSetOverlayGlobalPriorityMin;
@@ -619,33 +560,26 @@ int main(int argc, char **argv) {
                     return std::string(b);
                 };
                 char extra[256];
-                if (want & kMmap1) {
+                std::snprintf(extra, sizeof extra, "\"dist\":%.3f,\"open\":[%.3f,%.3f],\"lr\":%.3f,", Length(s.fix1),
-                    std::snprintf(extra, sizeof extra, "\"dist\":%.3f,\"open\":[%.3f,%.3f],\"lr\":%.3f,", Length(s.fix1),
+                              s.open[0], s.open[1], lr(s.left1, s.right1));
-                                  s.open[0], s.open[1], lr(s.left1, s.right1));
+                m1 = SrcJson(list, headThen, s.left1 + s.right1, extra + eyes(s.left1, s.right1) + unc(s.var1));
-                    m1 = SrcJson(list, headThen, s.left1 + s.right1, extra + eyes(s.left1, s.right1) + unc(s.var1));
+                std::snprintf(extra, sizeof extra, "\"lr\":%.3f,", lr(s.left2, s.right2));
-                }
+                m2 = SrcJson(list, headThen, s.left2 + s.right2, extra + eyes(s.left2, s.right2) + unc(s.var2));
-                if (want & kMmap2) {
+                left = SrcJson(list, headThen, s.left2);
-                    std::snprintf(extra, sizeof extra, "\"lr\":%.3f,", lr(s.left2, s.right2));
+                right = SrcJson(list, headThen, s.right2);
-                    m2 = SrcJson(list, headThen, s.left2 + s.right2, extra + eyes(s.left2, s.right2) + unc(s.var2));
-                }
-                if (want & kLeft) left = SrcJson(list, headThen, s.left2);
-                if (want & kRight) right = SrcJson(list, headThen, s.right2);
-                // "new" compares with the last sample, so the last measurement is kept either way.
                 const float *m = s.meas;
                 const bool newL = m[0] != lastMeas[0] || m[1] != lastMeas[1];
                 const bool newR = m[2] != lastMeas[2] || m[3] != lastMeas[3];
                 std::memcpy(lastMeas, m, sizeof lastMeas);
-                if (want & kEye) {
+                std::snprintf(extra, sizeof extra, "{\"q\":[%.3g,%.3g],\"m\":[[%.4f,%.4f],[%.4f,%.4f]],\"new\":[%d,%d]}",
-                    std::snprintf(extra, sizeof extra, "{\"q\":[%.3g,%.3g],\"m\":[[%.4f,%.4f],[%.4f,%.4f]],\"new\":[%d,%d]}",
+                              (m[4] + m[5]) / 2, (m[6] + m[7]) / 2, m[0], m[1], m[2], m[3], int(newL), int(newR));
-                                  (m[4] + m[5]) / 2, (m[6] + m[7]) / 2, m[0], m[1], m[2], m[3], int(newL), int(newR));
+                eye = extra;
-                    eye = extra;
-                }
             }
 
             // Our tracker: its own sample time picks the head pose, like the mmap's.
             std::string own = "{\"ok\":0}";
             OwnSample o;
-            if ((want & kOwn) && ownFile.Read(o) && now - o.t < 0.1) {
+            if (ownFile.Read(o) && now - o.t < 0.1) {
                 vr::HmdMatrix34_t headOwn = headNow;
                 history.At(o.t, headOwn);
                 auto pair = [](bool ok, float a, float b) {
@@ -690,10 +624,7 @@ int main(int argc, char **argv) {
             break;
         }
         if (stdinClosed) break;
-        // 250 passes a second: a new sample is printed within 4 ms (2 on average) of appearing,
+        std::this_thread::sleep_for(std::chrono::milliseconds(2));
-        // and the pose history has a pose within 2 ms of any sample's time (a 0.2 degree head
-        // turn at 100 degrees a second). Every 2 ms read the pose and the events twice as often.
-        std::this_thread::sleep_for(std::chrono::milliseconds(4));
     }
     screens.Stop();
     vr::VR_Shutdown();

gaze/ft-gazed

@@ -63,24 +63,9 @@ and the error moves): lessons from before count less then, so the first few afte
 relearn the offset.
 
 Our own tracker (gaze/tracker/ft-eyes) runs here too, in the dev container, while
-GAZE_TRACKER=own and the gaze is in use (below), or the gaze probe asks for it ("eyes SECONDS",
+GAZE_TRACKER=own or the gaze probe asks for it ("eyes SECONDS", a lease the probe renews). It
-a lease the probe renews). It reads the eye-camera frames the root service frametop-eyegrab
+reads the eye-camera frames the root service frametop-eyegrab copies (gaze/tracker/install.sh),
-copies (gaze/tracker/install.sh), which copies them only while ft-eyes runs.
+which copies them only while ft-eyes runs.
-
-Idle: ft-gaze and our own tracker run only while the gaze is in use: gaze mode on with someone
-wearing the headset (the pointer helper says both: "gaze ? headset" -> "ok on|off worn|away"), a
-check or calibration open or asked for, or a "wake" lease (the Gaze page of Frametop Input
-Settings renews one while it's open). IDLE_AFTER after the last use they stop, and the frame
-grabber goes idle with our tracker; with our tracker, that was over half a core with gaze mode
-off. A check asked for while idle waits for the tracker to start (at most WAKE_SETTLE). Our
-tracker's next click after it starts again re-seats it, as after the headset was off, so turning
-gaze mode on after a while opens the quick check.
-
-ft-gaze prints only the sources this uses ("sources LIST" on its stdin, see wanted_sources):
-own and mmap1 with our tracker, left, right and mmap1 with SteamVR's eyes, and every source
-while a check or the calibration runs or is asked for, since those record them all. So
-SteamVR's gaze action, the one source that calls into vrserver, is read only then (or with
---source action).
 
 Checks and calibration (gaze/gazecheck.py): a one-dot quick check when the headset goes on or
 our tracker asks for a click, and the full calibration when gaze mode comes on without one,
@@ -98,8 +83,6 @@ Control socket: abstract unix datagram "@ft_gazed":
   reload                           read calibration.json and the settings again
   eyes <seconds>                   keep our own tracker running that much longer (at most 120),
                                    whatever GAZE_TRACKER says: the probe's lease. Reply: "ok"
-  wake <seconds>                   keep the gaze running (not idle) that much longer (at most 120),
-                                   whatever gaze mode says: Input Settings' Gaze page. Reply: "ok"
   quickcal                         the one-dot check now (the calibration if there's none)
   calibrate                        the full calibration in the panel
   fitcheck                         the headset fit check in the panel
@@ -157,9 +140,6 @@ RETRY = 3.0       # seconds before starting ft-gaze again
 EYES_RETRY = 10.0  # seconds before starting ft-eyes again after it stopped on its own
 EYES_LEASE_MAX = 120.0
 SETTLE = 0.3      # seconds after an eye is found again before the fallback learns from it
-IDLE_AFTER = 30.0  # seconds after the gaze was last in use before ft-gaze and our tracker stop
-WAKE_SETTLE = 15.0  # seconds after waking that the tracker may take to start sending
-WAKE_MAX = 120.0
 KEYBOARD_PITCH = -20.0  # degrees: gaze under this, on no screen, is a look at the keyboard
 
 
@@ -250,10 +230,6 @@ class Service:
         self.eyes_proc = None  # ft-eyes, while it runs
         self.eyes_until = 0.0  # the probe's lease (monotonic time)
         self.eyes_restart_at = 0.0
-        self.awake = False      # ft-gaze (and our tracker) run: the gaze is in use (see the top)
-        self.idle_at = 0.0      # idle from then, unless it's in use again first
-        self.woke_at = 0.0
-        self.wake_until = 0.0   # a "wake" lease
 
         self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC | socket.SOCK_NONBLOCK)
         self.sock.bind(ME)
@@ -266,7 +242,6 @@ class Service:
         self.sel.register(self.eyes_sock, selectors.EVENT_READ, "own")
         self.checks = Checks(self, self.sel)
         self.proc = None
-        self.proc_sources = None  # what ft-gaze was last told to print
         self.buf = b""
         self.restart_at = 0.0
         self.running = True
@@ -424,73 +399,8 @@ class Service:
             log(f"lesson log: {e}")
         return rec
 
-    # --- Idle (see the top) ---
-
-    def use_reason(self):
-        """Why the gaze is in use now, or None."""
-        c = self.checks
-        if c.gaze_on and c.headset is not False:
-            return "gaze mode is on"
-        if c.active or c.pending:
-            return "a check"
-        if time.monotonic() < self.wake_until:
-            return "asked to stay awake"
-        return None
-
-    def idle_reason(self):
-        c = self.checks
-        if c.gaze_on is None:
-            return "the pointer helper isn't answering (SteamVR not running?)"
-        if c.gaze_on and c.headset is False:
-            return "nobody is wearing the headset"
-        return "gaze mode is off"
-
-    def waking(self):
-        """Idle, or awake too briefly for the tracker to be sending yet."""
-        return not self.awake or time.monotonic() - self.woke_at < WAKE_SETTLE
-
-    def update_awake(self):
-        now = time.monotonic()
-        why = self.use_reason()
-        if why:
-            self.idle_at = now + IDLE_AFTER
-        if why and not self.awake:
-            self.awake, self.woke_at, self.restart_at = True, now, 0.0
-            log(f"awake: {why}")
-        elif not why and self.awake and now >= self.idle_at:
-            self.awake = False
-            log(f"idle: {self.idle_reason()}; ft-gaze and our tracker stop until the gaze is used again")
-            self.stop_helper()
-            if self.eyes_proc and not self.eyes_wanted():
-                self.stop_eyes()
-
     # --- ft-gaze ---
 
-    def wanted_sources(self):
-        """The sources ft-gaze should print (its --sources): what on_sample and the checks read.
-        mmap1 always (blinks, lost eyes, and the headset going on, from its openness and
-        variances); everything while a check runs or waits, since they record every source."""
-        c = self.checks
-        if c.active or c.pending:
-            return "all"
-        kind = self.kind
-        if kind == "own":
-            return "own,mmap1"
-        if kind == "eyes":
-            return "left,right,mmap1"
-        return ",".join(dict.fromkeys((self.source, "mmap1", "mmap2")))  # mmap2: each eye, for the fallback
-
-    def sync_sources(self):
-        want = self.wanted_sources()
-        if not self.proc or want == self.proc_sources:
-            return
-        try:
-            self.proc.stdin.write(f"sources {want}\n".encode())
-            self.proc.stdin.flush()
-        except (OSError, ValueError):
-            return  # it's stopping: read_stdout notices
-        self.proc_sources = want
-
     def start_helper(self):
         if not HELPER.exists():
             log(f"ft-gaze isn't built: run {REPO}/gaze/build.sh")
@@ -501,11 +411,9 @@ class Service:
         subprocess.run([str(REPO / "scripts" / "container-up.sh")], env=env, check=False)
         distrobox = Path.home() / ".local" / "bin" / "distrobox"
         # ft-gaze quits when its stdin closes: the one thing distrobox passes on.
-        sources = self.wanted_sources()
+        self.proc = subprocess.Popen([str(distrobox), "enter", "dev", "--", str(HELPER), "--watch-stdin"], env=env,
-        self.proc = subprocess.Popen([str(distrobox), "enter", "dev", "--", str(HELPER), "--watch-stdin",
+                                     stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
-                                      "--sources", sources], env=env, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
+                                     start_new_session=True)
-                                     stderr=subprocess.PIPE, start_new_session=True)
-        self.proc_sources = sources
         os.set_blocking(self.proc.stdout.fileno(), False)
         os.set_blocking(self.proc.stderr.fileno(), False)
         self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
@@ -533,7 +441,7 @@ class Service:
         self.proc = None
 
     def eyes_wanted(self):
-        return (self.tracker == "own" and not self.override and self.awake) or time.monotonic() < self.eyes_until
+        return (self.tracker == "own" and not self.override) or time.monotonic() < self.eyes_until
 
     def start_eyes(self):
         """Our own tracker, in the dev container, with build/venv's numpy and OpenCV. Like
@@ -792,14 +700,6 @@ class Service:
                     reply = "ok"
                 except ValueError:
                     reply = "error bad seconds"
-            elif words[:1] == ["wake"] and len(words) == 2:
-                try:
-                    secs = min(max(float(words[1]), 0.0), WAKE_MAX)
-                    self.wake_until = max(self.wake_until, time.monotonic() + secs)
-                    self.update_awake()
-                    reply = "ok"
-                except ValueError:
-                    reply = "error bad seconds"
             elif words[:1] == ["reload"]:
                 self.load_settings()
                 self.load_calibration()
@@ -850,7 +750,6 @@ class Service:
             st.update(calibration_samples=cal.get("dots", 0), calibration_made=cal.get("made"),
                       lessons=max(len(m) for m in w.misses), own_running=bool(own),
                       own_reseat=any(e.get("reseat") for e in own.get("eyes", {}).values()))
-        st.update(awake=self.awake, idle=None if self.awake else self.idle_reason())
         st.update(eyes_process=self.eyes_proc is not None, eyegrab=EYES_CAMS.exists(),
                   tracker_setting=self.tracker_setting, own_installed=own_installed())
         st.update({"ft_gaze": self.proc is not None, "sample_age_s": round(time.monotonic() - self.last_sample, 2)
@@ -871,7 +770,6 @@ class Service:
         if mtime(CONF) != self.conf_mtime or (self.tracker_setting == "auto"
                                               and own_installed() != (self.tracker == "own")):
             self.load_settings()
-        self.update_awake()
         want = self.eyes_wanted()
         if want and not self.eyes_proc and time.monotonic() >= self.eyes_restart_at:
             self.start_eyes()
@@ -892,7 +790,7 @@ class Service:
         next_verbose = time.monotonic() + 5
         while self.running:
             now = time.monotonic()
-            if self.awake and not self.proc and now >= self.restart_at:
+            if not self.proc and now >= self.restart_at:
                 self.start_helper()
             for key, _ in self.sel.select(timeout=0.05 if self.checks.active else 0.5):
                 if key.data == "control":
@@ -910,7 +808,6 @@ class Service:
                 elif key.data == "stderr" and self.proc:
                     self.read_stderr()
             self.checks.tick()
-            self.sync_sources()
             if now >= next_periodic:
                 self.periodic()
                 next_periodic = now + 1.0

gaze/gazecheck.py

@@ -10,9 +10,7 @@ directions: look at each one.
           once every QUICK_COOLDOWN, and on "quickcal" (Frametop Input Settings, or a mouse
           button or key combination mapped to Gaze quick check), and when a click's correction
           was past POINTER_GAZE_NUDGE_MAX (55 degrees; the helper's "recheck"): the tracker is
-          far off. Ignored, it closes after QUICK_TIMEOUT and changes nothing. The headset going
+          far off. Ignored, it closes after QUICK_TIMEOUT and changes nothing.
-          on is seen only while the gaze service is awake (gaze mode on, someone wearing it: see
-          ft-gazed), so it also opens when gaze mode comes on after the service idled.
   five    the middle and four around it, when the first FIVE_COUNT lessons after a quick check
           were all over FIVE_LIMIT degrees off: the quick check didn't fix it.
   full    the calibration, as the gaze probe's: three rounds, dark, medium and bright (pupil
@@ -32,9 +30,6 @@ directions: look at each one.
           adjust the headset. A left click or Meta+J runs its guided check (dots, then looks
           down, up, left and right); a right click or Meta+K closes it, as does FIT_TIMEOUT.
 
-A check asked for while the gaze service idles (quickcal, calibrate, fitcheck) wakes it and
-waits until the tracker sends, at most ft-gazed's WAKE_SETTLE; then it opens, or logs why not.
-
 The quick check's dot captures itself: from CHECK_SETTLE after it shows (the eyes getting
 there), once the gaze has held within CHECK_SPREAD for CHECK_WINDOW (the probe's max spread and
 capture time). It's the gaze holding still that counts, not where the tracker puts it, so it
@@ -214,9 +209,7 @@ class Checks:
         sel.register(self.out, selectors.EVENT_READ, "checks")
         self.check = None
         self.gaze_on = None
-        self.headset = None      # someone wears it (the helper's "worn"), False "away", None unknown
         self.gaze_heard = 0.0
-        self.pending = None      # (command words, when): asked for while the gaze service idled
         self.full_armed = True    # gaze mode on without a calibration opens the full one (need_full)
         self.full_blocked = None  # why it can't open now
         self.full_retry_at = 0.0
@@ -301,22 +294,17 @@ class Checks:
             pass
 
     def on_readable(self):
-        """Replies on our socket: the helper's "ok on|off [worn|away]" to "gaze ? headset" (an
+        """Replies on our socket: the helper's "ok on|off" to "gaze ?"; the panel's are dropped."""
-        older helper leaves the headset out); the panel's are dropped."""
         while True:
             try:
-                words = self.out.recv(4096).decode("utf-8", "replace").split()
+                data = self.out.recv(4096).decode("utf-8", "replace")
             except (BlockingIOError, OSError):
                 return
-            if words[:1] == ["ok"] and len(words) in (2, 3) and words[1] in ("on", "off"):
+            if data in ("ok on", "ok off"):
-                on = words[1] == "on"
+                on = data == "ok on"
-                headset = None if len(words) == 2 else words[2] == "worn"
+                was, self.gaze_on, self.gaze_heard = self.gaze_on, on, time.monotonic()
-                was = (self.gaze_on, self.headset)
+                if on and was is False:
-                self.gaze_on, self.headset, self.gaze_heard = on, headset, time.monotonic()
-                if on and was[0] is False:
                     self.on_gaze_on()
-                if was != (on, headset):
-                    self.svc.update_awake()
 
     # --- State ---
 
@@ -353,8 +341,6 @@ class Checks:
             self.full_blocked = None
             return
         now = time.monotonic()
-        if not self.can_run() and self.svc.waking():
-            return  # the tracker is still starting (the service idled)
         if not self.can_run():
             why = ("the eye tracker isn't sending" if now - self.svc.last_sample >= 2
                    else "no eyes seen (is the headset on?)")
@@ -504,8 +490,6 @@ class Checks:
 
     def on_sample(self, s):
         self.sample_at = time.monotonic()
-        if self.pending:
-            self.run_pending()
         unc = (s["src"].get("mmap1") or {}).get("unc")
         if unc and min(unc) <= EYE_LOST:
             self.seen_at = time.monotonic()
@@ -758,28 +742,9 @@ class Checks:
 
     # --- From the service ---
 
-    def run_pending(self):
+    def command(self, words):
-        """A check asked for while the service idled: once the tracker sends (and our tracker has
-        said whether it's calibrated), or WAKE_SETTLE after waking, when its error is the real one."""
-        svc = self.svc
-        words, _ = self.pending
-        ready = (time.monotonic() - svc.last_sample < 2 if words[0] == "fitcheck" else self.can_run()) \
-            and (svc.kind != "own" or self.calibrated() is not None)
-        if not ready and svc.waking():
-            return
-        self.pending = None
-        reply = self.command(words, queue=False)
-        if reply != "ok":
-            log(f"{words[0]}, asked for while idle: {reply.removeprefix('error ')}")
-
-    def command(self, words, queue=True):
         """quickcal, calibrate, calaccept, calquit -> a reply."""
         cmd = words[0]
-        if cmd in ("quickcal", "calibrate", "fitcheck") and queue and self.svc.waking() and not self.check:
-            # The tracker isn't running (or only just started): wake it, and do this once it sends.
-            self.pending = (words, time.monotonic())
-            self.svc.update_awake()
-            return "ok waking the eye tracker first"
         if cmd == "quickcal":
             return self.start("full" if self.calibrated() is False else "quick", "asked for")
         if cmd == "calibrate":
@@ -855,11 +820,9 @@ class Checks:
         now = time.monotonic()
         if not self.panel_proc and now >= self.panel_restart_at:
             self.start_panel()
-        self.to_helper("gaze ? headset")
+        self.to_helper("gaze ?")
         if now - self.gaze_heard > 5:
-            self.gaze_on = self.headset = None  # the helper isn't answering
+            self.gaze_on = None  # the helper isn't answering
-        if self.pending:
-            self.run_pending()
         if self.check:
             self.to_helper("calpanel 1")
         if not self.eyes_seen(AWAY_MIN):
@@ -881,8 +844,7 @@ class Checks:
 
     def status(self):
         c = self.check
-        st = {"check": None, "gaze_mode": self.gaze_on, "headset_worn": self.headset, "pending": self.pending[0][0]
+        st = {"check": None, "gaze_mode": self.gaze_on, "calibrated": self.calibrated(), "eyes": self.eyes_seen(),
-              if self.pending else None, "calibrated": self.calibrated(), "eyes": self.eyes_seen(),
               "problem": self.problem(),
               "panel": self.panel_proc is not None,
               "last_quick_s": round(time.monotonic() - self.last_quick) if self.last_quick else None}

gaze/panel/ft-gazepanel.cpp

@@ -47,7 +47,6 @@
 #include <drm_fourcc.h>
 #include <fcntl.h>
 #include <gbm.h>
-#include <poll.h>
 #include <sys/socket.h>
 #include <sys/un.h>
 #include <unistd.h>
@@ -525,11 +524,7 @@ int main(int argc, char **argv) {
             if (!shown) ov->ShowOverlay(h), shown = true;
             dirty = false;
         }
-        // Until a command comes, or 10 ms while shown (SteamVR's events). Hidden, it waits up to a
+        std::this_thread::sleep_for(std::chrono::milliseconds(visible ? 10 : 50));
-        // second: it woke 20 to 30 times a second for nothing, the main cost left with gaze idle.
-        // A closed stdin (--watch-stdin) wakes it too, so quitting doesn't wait.
-        pollfd fds[2] = {{sock, POLLIN, 0}, {0, POLLIN, 0}};
-        poll(fds, watchStdin ? 2 : 1, visible ? 10 : 1000);
     }
     ov->DestroyOverlay(h);
     buffers.Drop();

gaze/test/idle-test.py

@@ -1,178 +0,0 @@
-#!/usr/bin/env python3
-"""Offline test of the gaze service idling (gaze/ft-gazed, gaze/gazecheck.py): ft-gaze runs only
-while the gaze is in use, and a check asked for while it idles waits for the tracker.
-
-Runs ft-gazed's Service with its sockets renamed, a fake pointer helper (answers "gaze ?
-headset" as the test says), and a fake ft-gaze (prints samples, quits when its stdin closes).
-SteamVR's tracker is the one in use, so our own isn't started; the panel isn't either. Nothing
-reaches the live gaze service, the pointer helper, or SteamVR, so it's safe next to them.
-
-  gaze/test/idle-test.py
-"""
-import importlib.machinery
-import importlib.util
-import os
-import socket
-import subprocess
-import sys
-import tempfile
-import threading
-import time
-
-HERE = os.path.dirname(os.path.abspath(__file__))
-GAZE = os.path.join(HERE, "..")
-sys.path.insert(0, GAZE)
-loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(GAZE, "ft-gazed"))
-gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
-loader.exec_module(gazed)
-import gazecheck  # noqa: E402  (the module ft-gazed imported)
-
-tag = f"ft_gaze_idle_test_{os.getpid()}"
-gazed.ME = f"\0{tag}_gazed"
-gazed.POINTER = gazecheck.POINTER = f"\0{tag}_helper"
-gazecheck.SCREENS = f"\0{tag}_screens"
-gazecheck.PANEL_PROG = gazed.REPO / "nonexistent-panel"  # "isn't built": no panel
-gazed.IDLE_AFTER, gazed.WAKE_SETTLE = 1.0, 3.0
-gazed.read_settings = lambda: ("steam", "steam", "auto", 55.0)
-logs = []
-gazed.log = gazecheck.log = lambda msg: logs.append(msg)
-
-# The fake ft-gaze: 90 samples a second, both eyes seen, until its stdin closes. It logs the
-# sources it was asked for: "argv LIST" (--sources), then "line LIST" for each "sources LIST".
-tmp = tempfile.mkdtemp(prefix="ft-gaze-idle-test-")
-FAKE = os.path.join(tmp, "ft-gaze")
-SOURCES_LOG = os.path.join(tmp, "sources.log")
-with open(FAKE, "w") as f:
-    f.write('''import json, os, select, sys, time
-log = open(sys.argv[1], "a", buffering=1)
-log.write("argv " + (sys.argv[sys.argv.index("--sources") + 1] if "--sources" in sys.argv else "-") + "\\n")
-buf = b""
-while True:
-    if select.select([sys.stdin], [], [], 1 / 90)[0]:
-        data = os.read(0, 4096)
-        if not data:
-            break
-        buf += data
-        while b"\\n" in buf:
-            line, buf = buf.split(b"\\n", 1)
-            if line.startswith(b"sources "):
-                log.write("line " + line[8:].decode() + "\\n")
-    eye = {"hy": 1.0, "hp": 2.0}
-    print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
-          "open": [0.8, 0.8]}, "left": eye, "right": eye}}), flush=True)
-''')
-started = []
-
-
-def start_helper(self):
-    """ft-gaze, straight from here instead of the dev container."""
-    import selectors
-    self.proc = subprocess.Popen([sys.executable, FAKE, SOURCES_LOG, "--sources", self.wanted_sources()],
-                                 stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
-    self.proc_sources = self.wanted_sources()
-    os.set_blocking(self.proc.stdout.fileno(), False)
-    os.set_blocking(self.proc.stderr.fileno(), False)
-    self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
-    self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
-    self.buf = b""
-    started.append(time.monotonic())
-
-
-gazed.Service.start_helper = start_helper
-
-# The fake pointer helper.
-helper_state = {"reply": "ok off worn"}
-helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
-helper.bind(gazed.POINTER)
-helper.settimeout(0.2)
-
-
-def answer():
-    while True:
-        try:
-            data, addr = helper.recvfrom(512)
-        except socket.timeout:
-            continue
-        except OSError:
-            return
-        if data.startswith(b"gaze ?") and addr:
-            helper.sendto(helper_state["reply"].encode(), addr)
-
-
-threading.Thread(target=answer, daemon=True).start()
-svc = gazed.Service(None, False, gazed.POINTER)
-threading.Thread(target=svc.run, daemon=True).start()
-ctl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
-ctl.bind("")
-ctl.settimeout(2)
-
-
-def ask(cmd):
-    ctl.sendto(cmd.encode(), gazed.ME)
-    return ctl.recv(4096).decode()
-
-
-failures = []
-
-
-def check(label, got, want):
-    ok = got == want
-    print(("ok    " if ok else "FAIL  ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
-    if not ok:
-        failures.append(label)
-
-
-def wait(cond, seconds):
-    end = time.monotonic() + seconds
-    while time.monotonic() < end:
-        if cond():
-            return True
-        time.sleep(0.05)
-    return cond()
-
-
-time.sleep(2.5)
-check("gaze mode off: idle, no ft-gaze", (svc.awake, svc.proc is None), (False, True))
-check("status says why", ask("status").count('"idle": "gaze mode is off"'), 1)
-
-helper_state["reply"] = "ok on worn"
-check("gaze mode on, headset worn: awake within 2 s", wait(lambda: svc.awake and svc.proc is not None, 2.5), True)
-check("samples come in", wait(lambda: svc.counts["samples"] > 20, 2), True)
-
-helper_state["reply"] = "ok on away"
-check("headset off: still awake for IDLE_AFTER", wait(lambda: not svc.awake, 0.5), False)
-check("then idle, ft-gaze stopped", wait(lambda: not svc.awake and svc.proc is None, 3.5), True)
-check("why: nobody wears it", ask("status").count('"idle": "nobody is wearing the headset"'), 1)
-
-helper_state["reply"] = "ok on"
-check("an older helper (no headset word): awake with gaze mode on", wait(lambda: svc.awake, 2.5), True)
-helper_state["reply"] = "ok off worn"
-check("gaze mode off again: idle", wait(lambda: not svc.awake and svc.proc is None, 4.5), True)
-
-check("wake lease", ask("wake 2"), "ok")
-check("wake: awake at once", (svc.awake, wait(lambda: svc.proc is not None, 1)), (True, True))
-check("lease over (2 s + IDLE_AFTER): idle", wait(lambda: not svc.awake, 4.5), True)
-
-time.sleep(0.5)
-logs.clear()
-open(SOURCES_LOG, "w").close()
-count = len(started)
-check("quick check while idle: queued, waking", ask("quickcal"), "ok waking the eye tracker first")
-check("it woke", wait(lambda: svc.awake and len(started) > count, 1.5), True)
-check("once the tracker sends, it ran (and says why it couldn't open)",
-      wait(lambda: any("quickcal, asked for while idle: the panel isn't running" in m for m in logs), 3), True)
-check("nothing left pending", svc.checks.pending, None)
-sources = open(SOURCES_LOG).read().split("\n")
-check("ft-gaze started with every source for the check", sources[0], "argv all")
-in_use = svc.wanted_sources()
-check("then only those in use (SteamVR's tracker: not the action, not own)",
-      (f"line {in_use}" in sources, in_use != "all", "action" in in_use, "own" in in_use), (True, True, False, False))
-check("idle again after", wait(lambda: not svc.awake, 3), True)
-
-print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
-svc.running = False
-time.sleep(0.7)
-os.remove(FAKE)
-os.remove(SOURCES_LOG)
-os.rmdir(tmp)
-os._exit(1 if failures else 0)

gaze/tracker/eyes_pupil.py

@@ -9,12 +9,7 @@ the search runs again with a smaller closing.
 import cv2
 import numpy as np
 
-# One thread: OpenCV's pool of one per core costs more than it saves on a frame this small
+DARK = 30          # pupil pixels are below this (the face around it is 40-180)
-# (a 140-240 px window while it follows the pupil). Its idle workers spun and yielded about
-# 14,000 times a second each, a quarter of a core, beside SteamVR's compositor.
-cv2.setNumThreads(1)
-
-DARK = 30         # pupil pixels are below this (the face around it is 40-180)
 MIN_AREA = 150     # pupil area range in pixels
 MAX_AREA = 20000
 MIN_FILL = 0.75    # blob area / fitted-ellipse area

gaze/tracker/findings.md

@@ -63,13 +63,6 @@ reading only.
     frame when its camera starts the frame after next (no slot is rewritten sooner than three
     frames), ignores late writes to the frame just finished, and saves from a separate
     thread. fit1 may hold a few percent of torn frames.
-  - `--share` (2026-10-03) checks only the slot each camera writes next, from the two before
-    (the orders above, learned again if they change; all four slots for 2 s after a frame turns
-    up elsewhere), sleeps until 2.5 ms before the next frame is due, then looks every 1 ms with
-    0.5 ms of timer slack. Against the 0.3 ms poll of all eight slots, on simulated cameras:
-    207 wakeups a second instead of 1,486, 0.8% of a core instead of 3.6% (more on the real
-    DMA-BUF memory), no torn or skipped frames, and a frame's start seen 1.35 ms late on
-    average instead of 0.76. `--rec` still polls all slots every 0.3 ms, for its times.
   - Eye tracking stops when the headset is off ("HMD off, stopping eye tracking"), so
     recordings are empty then.
 - **Which camera is which eye** (capture fit1, 2026-09-29, closing one eye at a time):

gaze/tracker/ft-eyegrab.c

@@ -46,7 +46,6 @@
 #include <stdlib.h>
 #include <string.h>
 #include <sys/mman.h>
-#include <sys/prctl.h>
 #include <sys/stat.h>
 #include <sys/syscall.h>
 #include <time.h>
@@ -289,53 +288,20 @@ static int eye_buffer(void) {
 
 // Calls done(frame, slot, time) for every complete eye-camera frame until `seconds` pass
 // (forever if negative), a stop signal comes, the tracker process goes away, or keep()
-// (checked about every 0.25 s, when given) says to stop. Looks every `poll_us`.
+// (checked about every 0.25 s, when given) says to stop.
 //
 // A frame lands over several milliseconds, in bursts, and its last bursts can come after
 // the camera has started its next frame. A slot isn't rewritten until at least three frames
 // later (camera 0 cycles 3,0,1,2; camera 1 7,5,4,6,5,7,6,4), so a frame is passed on when
 // its camera starts the frame after next. Changes to the slot just finished are late bursts,
 // not a new frame. A frame's time is when its slot first changed.
-//
-// Each look checks only the slot each camera writes next: the one that followed the last two
-// before (after[][], seeded with the orders above and learned as frames come). Every check
-// reads 256 words spread over a frame in DMA-BUF memory, so all eight slots each time was
-// most of the cost. A camera with nothing in its expected slot for 1.5 frames, or with no
-// order known yet, has all its slots checked until a frame comes. When that finds a frame
-// somewhere else (the order changed, or a frame was missed), all its slots are checked for
-// SCAN_AFTER_MISS, as before, while after[][] learns the new order. A slot's fingerprint is
-// taken again when it stops being one of the two in use, so a check later sees only a new frame.
-// Between frames it sleeps until FRAME_EARLY before the next one is due (the cameras run at
-// 90 fps, within a ms of each other), then looks every `poll_us`.
-#define FRAME_DUE (1.5 / 90)  // s: a camera that hasn't started a frame by then gets all its slots checked
-#define SCAN_AFTER_MISS 2.0    // s of checking all slots after a frame came in an unexpected one
-#define CAMS_IDLE 0.5          // s without a frame from either camera: look 4 times less often
-#define FRAME_EARLY 0.0025     // s before a frame is due to start looking for it
-
-// When to start looking for the next frame: FRAME_EARLY before the first camera's is due. A
-// camera already late (it lost its order, or stopped) means now.
-static double next_due(const double last[2], double t) {
-    double due = 1e300;
-    for (int cam = 0; cam < 2; cam++) {
-        double d = last[cam] + 1.0 / 90 - FRAME_EARLY;
-        if (t - last[cam] > CAMS_IDLE) continue;  // stopped: the other one sets the pace
-        if (d < due) due = d;
-    }
-    return due < 1e300 ? due : t;
-}
-
 static void poll_frames(int b, double seconds, int pid, void (*done)(const uint8_t *, int, double),
-                        int (*keep)(void), unsigned poll_us) {
+                        int (*keep)(void)) {
-    static const int order[2][8] = {{3, 0, 1, 2, 3, 0, 1, 2}, {7, 5, 4, 6, 5, 7, 6, 4}};
-    int after[EYE_SLOTS][EYE_SLOTS];
-    memset(after, -1, sizeof after);
-    for (int c = 0; c < 2; c++)
-        for (int i = 0; i < 8; i++) after[order[c][i]][order[c][(i + 1) % 8]] = order[c][(i + 2) % 8];
     uint64_t sig[EYE_SLOTS];
     double first[EYE_SLOTS];
     int cur[2] = {-1, -1}, prev[2] = {-1, -1};
     for (int k = 0; k < EYE_SLOTS; k++) sig[k] = frame_sig(bufs[b].p + slot_start(k)), first[k] = 0;
-    double start = now(), checked = start, kept = start, last[2] = {start, start}, scan_until[2] = {0, 0};
+    double start = now(), checked = start, kept = start;
     char proc[64];
     snprintf(proc, sizeof proc, "/proc/%d", pid);
     while ((seconds < 0 || now() - start < seconds) && !stop_rec) {
@@ -349,36 +315,18 @@ static void poll_frames(int b, double seconds, int pid, void (*done)(const uint8
             if (!keep()) return;
             kept = t;
         }
-        for (int cam = 0; cam < 2; cam++) {
+        for (int k = 0; k < EYE_SLOTS; k++) {
-            int next = prev[cam] >= 0 ? after[prev[cam]][cur[cam]] : -1;
+            uint64_t s = frame_sig(bufs[b].p + slot_start(k));
-            int all = next < 0 || t - last[cam] > FRAME_DUE || t < scan_until[cam];
+            if (s == sig[k]) continue;
-            int from = all ? cam * 4 : next, to = all ? cam * 4 + 4 : next + 1;
+            sig[k] = s;
-            for (int k = from; k < to; k++) {
+            int cam = k >= 4;
-                uint64_t s = frame_sig(bufs[b].p + slot_start(k));
+            if (k == cur[cam] || k == prev[cam]) continue;  // landing, or a late burst
-                if (s == sig[k]) continue;
+            if (prev[cam] >= 0) done(bufs[b].p + slot_start(prev[cam]), prev[cam], first[prev[cam]]);
-                sig[k] = s;
+            prev[cam] = cur[cam];
-                if (k == cur[cam] || k == prev[cam]) continue;  // landing, or a late burst
+            cur[cam] = k;
-                if (next >= 0 && k != next) scan_until[cam] = t + SCAN_AFTER_MISS;
+            first[k] = t;
-                if (prev[cam] >= 0) {
-                    done(bufs[b].p + slot_start(prev[cam]), prev[cam], first[prev[cam]]);
-                    after[prev[cam]][cur[cam]] = k;
-                    // Out of use from now on: later changes are a new frame.
-                    sig[prev[cam]] = frame_sig(bufs[b].p + slot_start(prev[cam]));
-                }
-                prev[cam] = cur[cam];
-                cur[cam] = k;
-                first[k] = t;
-                last[cam] = t;
-                next = prev[cam] >= 0 ? after[prev[cam]][cur[cam]] : -1;
-            }
         }
-        double wait = poll_us * 1e-6;
+        usleep(300);
-        if (t - last[0] > CAMS_IDLE && t - last[1] > CAMS_IDLE) {
-            wait *= 4;
-        } else if (next_due(last, t) - t > wait) {
-            wait = next_due(last, t) - t;
-        }
-        usleep((useconds_t)(wait * 1e6));
     }
 }
 
@@ -408,7 +356,7 @@ static int rec(double seconds, const char *dir, int pid) {
     pthread_t writer;
     pthread_create(&writer, NULL, ring_writer, NULL);
     double start = now();
-    poll_frames(b, seconds, pid, rec_frame, NULL, 300);  // 0.3 ms: recordings' times
+    poll_frames(b, seconds, pid, rec_frame, NULL);
     pthread_mutex_lock(&ring.mu);
     ring.done = 1;
     pthread_cond_signal(&ring.cv);
@@ -435,10 +383,6 @@ static int rec(double seconds, const char *dir, int pid) {
 #define SHARE_SLOTS 8
 #define SHARE_MAGIC 0x31434546u  // "FEC1"
 #define WANT_FRESH 3.0           // seconds a touch of the --want file lasts
-// How often --share looks for frames, with a timer slack that lets the kernel group the
-// wakeups: a frame reaches ft-eyes 1 to 1.5 ms after its camera starts the next, not 0.3.
-#define SHARE_POLL_US 1000
-#define SHARE_SLACK_NS 500000
 
 typedef struct {
     uint32_t magic, version, width, height, slots, entry_size;
@@ -518,7 +462,6 @@ static int share_loop(const char *path) {
                         .slots = SHARE_SLOTS, .entry_size = (uint32_t)esize};
     signal(SIGINT, on_stop);
     signal(SIGTERM, on_stop);
-    if (prctl(PR_SET_TIMERSLACK, SHARE_SLACK_NS, 0, 0, 0) != 0) perror("PR_SET_TIMERSLACK");
     fprintf(stderr, "ft-eyegrab: sharing frames in %s%s%s\n", path, want_path ? " while wanted by " : "",
             want_path ? want_path : "");
     int pid = -1, idle = -1, missing = 0;
@@ -543,7 +486,7 @@ static int share_loop(const char *path) {
         if (idle != 0 || missing) fprintf(stderr, "ft-eyegrab: copying frames from eyetracking %d\n", pid);
         idle = 0, missing = 0;
         h->tracker_pid = pid;
-        poll_frames(eye_buffer(), -1, pid, share_frame, wanted, SHARE_POLL_US);
+        poll_frames(eye_buffer(), -1, pid, share_frame, wanted);
         struct stat st;
         char proc[64];
         snprintf(proc, sizeof proc, "/proc/%d", pid);

gaze/tracker/ft-eyes

@@ -50,7 +50,6 @@ import json
 import math
 import mmap
 import os
-import select
 import socket
 import struct
 import sys
@@ -59,12 +58,7 @@ import time
 from collections import deque
 from pathlib import Path
 
-# One thread for numpy's BLAS and OpenMP, set before numpy loads: it would start one per core
+import numpy as np
-# (8 here) for small arrays that never need them. eyes_pupil keeps OpenCV to one as well.
-for _var in ("OPENBLAS_NUM_THREADS", "OMP_NUM_THREADS"):
-    os.environ.setdefault(_var, "1")
-
-import numpy as np  # noqa: E402
 
 sys.path.insert(0, str(Path(__file__).resolve().parent))
 import eyes_model  # noqa: E402
@@ -88,15 +82,6 @@ GAP = 3.0             # s without frames: the headset was off, and may sit diffe
 CLICK_BEFORE = 0.3    # a click's frames: the 300 ms before it (like the probe's fixation)
 CALIB_MIN = 15        # frames an eye needs in a calibration dot's window
 CALIB_SPREAD = 4.0    # px: more than this and the eye moved during the dot
-# Waiting for frames. They come only as a counter in shared memory, so there's nothing to block
-# on: sleep until a camera's next frame is due, then look every POLL. ft-eyegrab passes each one
-# on within a ms or two of when its camera starts the one after, so they arrive 11.1 ms apart,
-# give or take that.
-PERIOD = 1 / 90       # s between a camera's frames
-EARLY = 0.002         # start looking this long before a frame is due
-POLL = 0.001          # then this often until it comes
-STALLED = 0.1         # s without a frame: that camera stopped (headset off), and isn't waited for
-IDLE_POLL = 0.02      # how often to look while both are stopped
 
 
 class Cams:
@@ -367,7 +352,7 @@ def wait_for_cams(sock, tracker, want):
             return Cams()
         except (OSError, ValueError, RuntimeError):
             serve(sock, tracker)
-            select.select([sock], [], [], 0.2)
+            time.sleep(0.2)
 
 
 def serve(sock, tracker):
@@ -387,24 +372,7 @@ def serve(sock, tracker):
                 pass
 
 
-def below_steamvr():
-    """Nice 10 and SCHED_BATCH, for this thread and those it starts. ft-eyes runs in the dev
-    container's podman scope, where the gaze service's unit doesn't reach it, so it ran at
-    nice 0 on the cores vrcompositor and vrserver use. Batch also lets a waking ft-eyes wait
-    for the running task's turn instead of taking the core: a frame a few ms late costs the
-    gaze little, a late compositor frame costs a dropped frame in the headset."""
-    try:
-        os.setpriority(os.PRIO_PROCESS, 0, max(os.getpriority(os.PRIO_PROCESS, 0), 10))
-    except OSError as e:
-        print(f"ft-eyes: nice: {e}", file=sys.stderr, flush=True)
-    try:
-        os.sched_setscheduler(0, os.SCHED_BATCH, os.sched_param(0))
-    except (OSError, AttributeError) as e:
-        print(f"ft-eyes: SCHED_BATCH: {e}", file=sys.stderr, flush=True)
-
-
 def main():
-    below_steamvr()
     verbose = "-v" in sys.argv
     if "--watch-stdin" in sys.argv:
         # Run by ft-gazed through distrobox, which doesn't pass a stop on: quit when our
@@ -430,7 +398,6 @@ def main():
     print("ft-eyes: frames found, tracking", file=sys.stderr, flush=True)
     seen = [cams.count(0), cams.count(1)]
     report = time.monotonic()
-    arrived = [0.0, 0.0]  # when each camera's newest frame was seen (monotonic)
     while True:
         serve(sock, tracker)
         want()
@@ -440,7 +407,6 @@ def main():
             if n == seen[c]:
                 continue
             seen[c] = n
-            arrived[c] = time.monotonic()
             got = cams.frame(c, n - 1)  # only the newest: never fall behind
             if got:
                 eyes[c].feed(*got)
@@ -459,6 +425,8 @@ def main():
                     shifts += [float(v) for v in e.shift.value]
                     pupils += [e.gaze[3], e.gaze[4]] if fresh else [math.nan, math.nan]
                 out.write(latest, (yaw, pitch), flags, per, shifts, pupils)
+        else:
+            time.sleep(0.001)
         now = time.monotonic()
         if now - report >= 5:
             if verbose:
@@ -476,12 +444,6 @@ def main():
                 print("ft-eyes: frames went away; waiting", file=sys.stderr, flush=True)
                 cams = wait_for_cams(sock, tracker, want)
                 seen = [cams.count(0), cams.count(1)]
-        # Until the next frame is due (a command on the socket wakes us sooner).
-        wait = IDLE_POLL
-        for c in (0, 1):
-            if now - arrived[c] < STALLED:
-                wait = min(wait, max(arrived[c] + PERIOD - EARLY - now, POLL))
-        select.select([sock], [], [], wait)
 
 
 if __name__ == "__main__":

hands/README.md

@@ -49,7 +49,7 @@ Files, all in `/run/user/UID/frametop-hands/` (private to the user; not `/run/us
 | --- | --- | --- | --- |
 | `cam-ring` | ft-camd | `camd/fhring.h` | ft-hands, `tools/ring.py` |
 | `hands` | ft-hands | `include/fh_hands.h` | ft-screens (`screens/handcut.cpp`) |
-| `gestures` | ft-hands | `include/fh_gestures.h` | the pointer helper (`pointer/helper/ft-pointer.cpp`), `tools/watch_gestures.py` |
+| `gestures` | ft-hands | `include/fh_gestures.h` | the pointer helper (`pointer/helper/pointer_holds.cpp`), `tools/watch_gestures.py` |
 
 The source keeps the `fh_` names and magic strings of frame-hands, the project it started as, so recordings made with it still work.
 
@@ -156,7 +156,7 @@ ft-hands detects a pinch per hand (`track/pinch.h`) and publishes it to the gest
 
 ## Pinches and grips in the pointer
 
-With `POINTER_HANDS=1`, the pointer helper (`pointer/helper/ft-pointer.cpp`) reads the gestures file every frame. It's off by default.
+With `POINTER_HANDS=1`, the pointer helper (`pointer/helper/pointer_holds.cpp`) reads the gestures file every frame. It's off by default.
 
 - **Pinch to click.** In gaze mode a pinch works like the mouse's press: the pointer stops where the gaze put it, and the click comes when the pinch opens, where the pointer is then. A quick tap clicks where you looked. Held, the pinching hand moves the pointer to correct the gaze, and the correction is a lesson for the gaze tracker, as with the mouse.
 - **Without gaze mode,** a pinch is a real press, like the mouse's button: pressed when it closes, released when it opens, and while it's held the hand drags the pointer. A tap is still a click where the pointer is.

input-settings/ft_input_settings.py

@@ -15,9 +15,6 @@ to the input relay over its control socket (@frametop_relay):
   - Pointer: speed, dot size, distance and the rest, applied live.
   - Ignored panels: SteamVR overlays the pointer passes through (POINTER_IGNORE), by app or
     one by one. The helper lists them (@ft_pointer_helper "overlays").
-  - Game optimization: pausing Frametop while a VR game runs, so it leaves the CPU and GPU to the game
-    (input/game_pause.py, in the relay: "pause on|off|?"): pause now, pause by itself during VR
-    games, the controller gesture that pauses and resumes, what happens to the desktop, a sound.
   - Gaze: the pointer's gaze mode (@ft_pointer_helper "gaze") and the gaze service
     (gaze/ft-gazed, @ft_gazed: status, forget, reload), with its eye tracker (SteamVR's or
     our own) and eye bias (GAZE_TRACKER, GAZE_EYE in frametop.conf).
@@ -76,7 +73,6 @@ ACTION_LABELS = {
     "sens_down": "Slower pointer", "layout_reset": "Reset desktop screen layout",
     "screens_toggle": "Hide/show desktop screens", "keyboard_toggle": "Open/close keyboard",
     "float_toggle": "Float window in VR / put it back", "dock_all": "Put all floating windows back",
-    "pause_toggle": "Pause/resume Frametop (for VR games)",
     "key": "Pass through as key",
     "none": "Do nothing",
 }
@@ -99,13 +95,6 @@ CONTROLLER_BUTTONS = {
     "right/bumper": "Right bumper", "right/trigger": "Right trigger", "right/grip": "Right grip",
     "right/thumbstick": "Right stick click",
 }
-# Pausing for VR games (input/game_pause.py): the gesture a rules file without "pause_gesture" gets,
-# both thumbsticks clicked together twice, and what can happen to the desktop.
-DEFAULT_PAUSE_GESTURE = {"buttons": ["left/thumbstick", "right/thumbstick"], "presses": 2}
-PAUSE_DESKTOP_MODES = {"hide": "Hide it and slow it down (windows stay open)",
-                       "close": "Close it (windows close; it starts again when you resume)"}
-PAUSE_REASONS = {"game": "a VR game started", "gesture": "controller gesture", "command": "ft-pause",
-                 "settings": "from here"}
 # Gaze mode is a mouse and keyboard feature (docs/gaze-controllers.md; the relay's GAZE_ACTIONS).
 GAZE_ACTIONS = ("gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right", "gaze_quickcal")
 CONTROLLER_ACTIONS = [a for a in ACTION_LABELS if a not in ("key", "none") + GAZE_ACTIONS]
@@ -304,7 +293,6 @@ class Backend(QObject):
     gazeChanged = Signal()
     driverChanged = Signal()
     panelsChanged = Signal()
-    pauseChanged = Signal()
     activity = Signal(str)  # device id
     captured = Signal(int, str)  # code, name
     capturedController = Signal(str, str)  # button, label
@@ -332,8 +320,6 @@ class Backend(QObject):
         self._check_asked = 0.0  # when quickcal or calibrate went to the gaze service (its errors)
         self._driver_block = ""  # set by _check_driver
         self._panels = None  # SteamVR's overlays, from the helper; None until it answers
-        self._pause = {}  # the relay's pause state ("pause ?"), {} when it doesn't answer
-        self._pause_at = 0.0
         self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
         self.sock.bind("")  # autobind an abstract address the relay can reply to
         self.sock.setblocking(False)
@@ -366,7 +352,6 @@ class Backend(QObject):
 
     def _refresh(self):
         self._send("devices")
-        self._send("pause ?")
         self._send("vrstatus", HELPER)
         self._send("gaze ?", HELPER)
         if not self._send("status", GAZED) and self._gaze:
@@ -380,9 +365,6 @@ class Backend(QObject):
         if self._gaze_mode is not None and now - self._gaze_at > 5:
             self._gaze_mode = None
             self.gazeChanged.emit()
-        if self._pause and now - self._pause_at > 5:
-            self._pause = {}
-            self.pauseChanged.emit()
 
     def _check_driver(self):
         block = driver_block()
@@ -421,20 +403,13 @@ class Backend(QObject):
             if t == "devices":
                 self._nodes = msg.get("nodes", [])
                 self._pointer_mode = bool(msg.get("pointer_mode"))
-                was_ok, self._relay_ok = self._relay_ok, True
+                self._relay_ok = True
-                if not was_ok:
-                    self.pauseChanged.emit()  # (pauseStatus' "old")
                 self.devicesChanged.emit()
             elif t == "overlays":
                 panels = [o for o in msg.get("list", []) if isinstance(o, dict) and isinstance(o.get("key"), str)]
                 if panels != self._panels:
                     self._panels = panels
                     self.panelsChanged.emit()
-            elif t == "pause":
-                self._pause_at = time.monotonic()
-                if msg != self._pause:
-                    self._pause = msg
-                    self.pauseChanged.emit()
             elif t == "vrstatus":
                 self._vr = msg
                 self._vr_at = time.monotonic()
@@ -842,108 +817,6 @@ class Backend(QObject):
         else:
             self.message.emit("The pointer helper isn't running (frametop-pointer.service)", True)
 
-    # --- pausing for VR games ---
-    @Property("QVariantMap", notify=pauseChanged)
-    def pauseStatus(self):
-        """The relay's pause state, and a line saying it ("relay": False when it doesn't answer)."""
-        st = self._pause
-        if not st:
-            # "old": the relay answers, but not about pausing (it predates it).
-            return {"relay": False, "old": self._relay_ok, "paused": False,
-                    "summary": "Unknown: the input relay " + ("can't pause" if self._relay_ok else "isn't answering")}
-        if st.get("paused"):
-            reason = PAUSE_REASONS.get(st.get("reason"), st.get("reason") or "")
-            summary = f"Paused since {time.strftime('%H:%M', time.localtime(st.get('since') or time.time()))}"
-            summary += f" ({reason})" if reason else ""
-            if st.get("ends_with_game"):
-                summary += ", until the game ends"
-        else:
-            summary = "Running" + (", with a VR game" if st.get("game") else "")
-        if st.get("busy"):
-            summary += "; working on it…"
-        return {"relay": True, "paused": bool(st.get("paused")), "summary": summary,
-                "gestureReader": bool(st.get("gesture_reader"))}
-
-    @Slot(bool)
-    def setPaused(self, on):
-        if self._send(f"pause {'on' if on else 'off'} settings"):
-            QTimer.singleShot(300, self._refresh)
-        else:
-            self.message.emit("The input relay isn't running (frametop-input-relay.service)", True)
-
-    def _set_rule(self, key, value, text):
-        rules = read_json(RULES_PATH)
-        rules[key] = value
-        self._save_rules(rules)
-        self.message.emit(text, False)
-
-    @Property(bool, notify=mappingsChanged)
-    def pauseAuto(self):
-        return read_json(RULES_PATH).get("pause_auto", True) is not False
-
-    @Slot(bool)
-    def setPauseAuto(self, on):
-        self._set_rule("pause_auto", bool(on), "VR games pause Frametop" if on else "VR games no longer pause Frametop")
-
-    @Property("QVariantList", constant=True)
-    def pauseDesktopModes(self):
-        return [{"value": k, "text": v} for k, v in PAUSE_DESKTOP_MODES.items()]
-
-    @Property(str, notify=mappingsChanged)
-    def pauseDesktop(self):
-        mode = read_json(RULES_PATH).get("pause_desktop")
-        return mode if mode in PAUSE_DESKTOP_MODES else "hide"
-
-    @Slot(str)
-    def setPauseDesktop(self, mode):
-        if mode in PAUSE_DESKTOP_MODES:
-            self._set_rule("pause_desktop", mode, "Paused, the desktop: " + PAUSE_DESKTOP_MODES[mode].split(" (")[0].lower())
-
-    @Property(bool, notify=mappingsChanged)
-    def pauseSound(self):
-        return read_json(RULES_PATH).get("pause_sound", True) is not False
-
-    @Slot(bool)
-    def setPauseSound(self, on):
-        self._set_rule("pause_sound", bool(on), "A sound on pause and resume" if on else "No sound on pause and resume")
-
-    @Property("QVariantList", constant=True)
-    def gestureButtons(self):
-        return [{"value": "", "text": "None"}] + [{"value": b, "text": label} for b, label in CONTROLLER_BUTTONS.items()]
-
-    @Property("QVariantMap", notify=mappingsChanged)
-    def pauseGesture(self):
-        """{"first", "second" ("" for none), "presses"} (the relay's game_pause.gesture_of)."""
-        spec = read_json(RULES_PATH).get("pause_gesture", DEFAULT_PAUSE_GESTURE)
-        buttons = [b for b in (spec or {}).get("buttons", []) if b in CONTROLLER_BUTTONS][:2] \
-            if isinstance(spec, dict) and isinstance(spec.get("buttons"), list) else []
-        presses = 1 if len(buttons) == 2 and spec.get("presses") == 1 else 2
-        return {"first": buttons[0] if buttons else "", "second": buttons[1] if len(buttons) > 1 else "",
-                "presses": presses}
-
-    @Slot(str, str, int)
-    def setPauseGesture(self, first, second, presses):
-        """No first button: no gesture. One button always takes two presses."""
-        buttons = list(dict.fromkeys(b for b in (first, second) if b in CONTROLLER_BUTTONS))
-        if not buttons:
-            self._set_rule("pause_gesture", None, "No controller gesture pauses Frametop")
-            return
-        presses = 1 if len(buttons) == 2 and presses == 1 else 2
-        self._set_rule("pause_gesture", {"buttons": buttons, "presses": presses},
-                       "Pause gesture: " + self.gestureText(buttons, presses))
-
-    @staticmethod
-    def gestureText(buttons, presses):
-        names = " and ".join(CONTROLLER_BUTTONS[b] for b in buttons)
-        together = " together" if len(buttons) == 2 else ""
-        return f"{names}{together}, {'twice' if presses == 2 else 'once'}"
-
-    @Property(str, notify=mappingsChanged)
-    def pauseGestureText(self):
-        g = self.pauseGesture
-        buttons = [b for b in (g["first"], g["second"]) if b]
-        return self.gestureText(buttons, g["presses"]) if buttons else "None"
-
     # --- gaze ---
     def _gaze_status(self, status):
         now = time.monotonic()
@@ -1163,12 +1036,6 @@ class Backend(QObject):
         else:
             self.message.emit("The gaze service isn't running (frametop-gaze.service)", True)
 
-    @Slot()
-    def keepGazeAwake(self):
-        """The Gaze page is open: the gaze service doesn't idle meanwhile (its status stays live,
-        and a check opens without waiting for the tracker). The page renews this every 10 s."""
-        self._send("wake 30", GAZED)
-
     @Slot()
     def gazeQuickCheck(self):
         """The gaze service's one-dot check, in the panel fixed to the headset."""

input-settings/main.qml

@@ -19,7 +19,6 @@ Kirigami.ApplicationWindow {
             Kirigami.Action { text: "Devices"; icon.name: "input-mouse"; onTriggered: root.show(devicesPage) },
             Kirigami.Action { text: "Buttons"; icon.name: "input-keyboard"; onTriggered: root.show(buttonsPage) },
             Kirigami.Action { text: "Controllers"; icon.name: "input-gamepad"; onTriggered: root.show(controllersPage) },
-            Kirigami.Action { text: "Game optimization"; icon.name: "applications-games"; onTriggered: root.show(gamesPage) },
             Kirigami.Action { text: "Keyboard"; icon.name: "input-keyboard-virtual"; onTriggered: root.show(keyboardPage) },
             Kirigami.Action { text: "Pointer"; icon.name: "transform-move"; onTriggered: root.show(pointerPage) },
             Kirigami.Action { text: "Ignored panels"; icon.name: "view-hidden"; onTriggered: root.show(ignorePage) },
@@ -57,8 +56,8 @@ Kirigami.ApplicationWindow {
         pageStack.push(page)
     }
 
-    // FT_INPUT_PAGE=buttons|controllers|games (Game optimization)|keyboard|pointer|ignore|gaze|bluetooth opens the app on that page.
+    // FT_INPUT_PAGE=buttons|controllers|keyboard|pointer|ignore|gaze|bluetooth opens the app on that page.
-    pageStack.initialPage: ({ buttons: buttonsPage, controllers: controllersPage, games: gamesPage, keyboard: keyboardPage,
+    pageStack.initialPage: ({ buttons: buttonsPage, controllers: controllersPage, keyboard: keyboardPage,
                               pointer: pointerPage, ignore: ignorePage, gaze: gazePage,
                               bluetooth: bluetoothPage })[startPage] || devicesPage
 
@@ -827,140 +826,6 @@ Kirigami.ApplicationWindow {
         }
     }
 
-    // ---------------------------------------------------------------- Games
-    Component {
-        id: gamesPage
-        Kirigami.ScrollablePage {
-            id: gmpage
-            title: "Game optimization"
-            property var st: backend.pauseStatus
-
-            // The gesture's boxes follow the saved gesture (which may tidy what was picked).
-            function syncGesture() {
-                const g = backend.pauseGesture
-                firstBox.currentIndex = firstBox.indexOfValue(g.first)
-                secondBox.currentIndex = secondBox.indexOfValue(g.second)
-                pressBox.currentIndex = pressBox.indexOfValue(g.presses)
-            }
-            function saveGesture() {
-                backend.setPauseGesture(firstBox.currentValue, secondBox.currentValue, pressBox.currentValue)
-            }
-            Component.onCompleted: syncGesture()
-            Connections {
-                target: backend
-                function onMappingsChanged() { gmpage.syncGesture() }
-            }
-
-            ColumnLayout {
-                spacing: Kirigami.Units.largeSpacing
-
-                Kirigami.InlineMessage {
-                    Layout.fillWidth: true
-                    visible: !gmpage.st.relay
-                    type: Kirigami.MessageType.Error
-                    text: gmpage.st.old
-                          ? "The input relay that runs is older and can't pause. Restart it: "
-                            + "systemctl --user restart frametop-input-relay"
-                          : "The input relay isn't answering (frametop-input-relay.service). It does the pausing."
-                }
-
-                Kirigami.FormLayout {
-                    Layout.fillWidth: true
-
-                    RowLayout {
-                        Kirigami.FormData.label: "Frametop:"
-                        Controls.Label { text: gmpage.st.summary }
-                        Controls.Button {
-                            text: gmpage.st.paused ? "Resume" : "Pause now"
-                            icon.name: gmpage.st.paused ? "media-playback-start" : "media-playback-pause"
-                            enabled: gmpage.st.relay
-                            onClicked: backend.setPaused(!gmpage.st.paused)
-                        }
-                    }
-                    Controls.Switch {
-                        Kirigami.FormData.label: "VR games:"
-                        text: "Pause while a VR game runs, and resume when it ends"
-                        checked: backend.pauseAuto
-                        onToggled: backend.setPauseAuto(checked)
-                    }
-
-                    Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Controller gesture" }
-                    Controls.ComboBox {
-                        id: firstBox
-                        Kirigami.FormData.label: "Button:"
-                        model: backend.gestureButtons
-                        textRole: "text"
-                        valueRole: "value"
-                        Layout.preferredWidth: Kirigami.Units.gridUnit * 10
-                        onActivated: gmpage.saveGesture()
-                    }
-                    Controls.ComboBox {
-                        id: secondBox
-                        Kirigami.FormData.label: "Together with:"
-                        model: backend.gestureButtons
-                        textRole: "text"
-                        valueRole: "value"
-                        enabled: firstBox.currentValue !== ""
-                        Layout.preferredWidth: Kirigami.Units.gridUnit * 10
-                        onActivated: gmpage.saveGesture()
-                    }
-                    Controls.ComboBox {
-                        id: pressBox
-                        Kirigami.FormData.label: "Pressed:"
-                        model: [{ value: 2, text: "Twice" }, { value: 1, text: "Once" }]
-                        textRole: "text"
-                        valueRole: "value"
-                        // One button pressed once would go off far too easily in games.
-                        enabled: firstBox.currentValue !== "" && secondBox.currentValue !== ""
-                        Layout.preferredWidth: Kirigami.Units.gridUnit * 10
-                        onActivated: gmpage.saveGesture()
-                    }
-                    Controls.Label {
-                        Layout.fillWidth: true
-                        wrapMode: Text.Wrap
-                        opacity: 0.7
-                        text: (backend.pauseGesture.first === "" ? "No gesture: a mapped button, a key combination, or ft-pause still work."
-                               : "Pauses and resumes Frametop, in games too. The game still gets the presses, so pick buttons it "
-                                 + "doesn't use this way. Two buttons count as together when both go down within a third of a "
-                                 + "second, and twice means within 0.7 s.")
-                              + (gmpage.st.relay && backend.pauseGesture.first !== "" && !gmpage.st.gestureReader
-                                 ? " Not reading the controllers right now (SteamVR isn't running?)." : "")
-                    }
-
-                    Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "While paused" }
-                    Controls.ComboBox {
-                        Kirigami.FormData.label: "The desktop:"
-                        model: backend.pauseDesktopModes
-                        textRole: "text"
-                        valueRole: "value"
-                        Layout.preferredWidth: Kirigami.Units.gridUnit * 20
-                        Component.onCompleted: currentIndex = indexOfValue(backend.pauseDesktop)
-                        onActivated: backend.setPauseDesktop(currentValue)
-                    }
-                    Controls.Switch {
-                        Kirigami.FormData.label: "Sound:"
-                        text: "Play a sound on pause and resume"
-                        checked: backend.pauseSound
-                        onToggled: backend.setPauseSound(checked)
-                    }
-                }
-
-                Controls.Label {
-                    Layout.fillWidth: true
-                    wrapMode: Text.Wrap
-                    opacity: 0.7
-                    text: "Paused, Frametop leaves the headset's CPU and GPU to the game. The gaze service and our eye "
-                          + "tracker stop, and nothing reads SteamVR's eye tracking. Hand tracking and remote desktop stop "
-                          + "if they run. The desktop hides and slows down, or closes, as chosen above. The 3D mouse lets go, "
-                          + "so the mouse is a plain one for SteamVR. Mapped buttons and key combinations do nothing but "
-                          + "pausing, the Steam menu, and your own commands. Resuming brings back what pausing stopped. "
-                          + "Map \"Pause/resume Frametop\" to a button or key combination on the other pages, or run "
-                          + "input/ft-pause."
-                }
-            }
-        }
-    }
-
     // ---------------------------------------------------------------- Gaze
     Component {
         id: gazePage
@@ -969,14 +834,6 @@ Kirigami.ApplicationWindow {
             title: "Gaze"
             header: DriverWarning {}
             property var status: backend.gazeStatus
-            // The gaze service idles while the gaze isn't used; open, this page keeps it going.
-            Timer {
-                running: true
-                repeat: true
-                interval: 10000
-                triggeredOnStart: true
-                onTriggered: backend.keepGazeAwake()
-            }
             actions: [
                 Kirigami.Action {
                     text: "Quick check"
@@ -1210,8 +1067,7 @@ Kirigami.ApplicationWindow {
                 Controls.Label {
                     Kirigami.FormData.label: "Service:"
                     text: backend.gazeServiceRunning
-                          ? (gpage.status.awake === false ? "idle: " + gpage.status.idle + " (it starts when the gaze is used)"
+                          ? (gpage.status.ft_gaze ? "running" : "running, eye tracker reader restarting")
-                             : gpage.status.ft_gaze ? "running" : "running, eye tracker reader starting")
                           : backend.gazeServiceInstalled ? "not running (frametop-gaze.service, starts with SteamVR)"
                           : "not installed: run gaze/run.sh install in the Frametop folder, in a terminal"
                     color: backend.gazeServiceRunning ? Kirigami.Theme.textColor : Kirigami.Theme.negativeTextColor

input/ft-pause

@@ -1,45 +0,0 @@
-#!/usr/bin/python3
-"""ft-pause: pause Frametop for a VR game, so it leaves the CPU and GPU to the game, or resume it.
-
-  ft-pause on|off|toggle   pause or resume (input/game_pause.py has what pausing stops)
-  ft-pause status          the pause state, as JSON (the default)
-
-The input relay does it (frametop-input-relay.service). So do the pause_toggle action, the
-controller gesture (by default both thumbsticks clicked together twice), and, with auto pause on,
-a VR game starting; Frametop Input Settings has the settings, on its Game optimization page.
-"""
-
-import json
-import os
-import socket
-import sys
-
-
-def ask(msg, timeout=2.0):
-    s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC)
-    s.bind(f"\0ft_pause.{os.getpid()}")
-    s.settimeout(timeout)
-    try:
-        s.sendto(msg.encode(), "\0frametop_relay")
-        return json.loads(s.recv(65536))
-    except ConnectionRefusedError:
-        sys.exit("the input relay isn't running (frametop-input-relay.service)")
-    except (socket.timeout, ValueError):
-        sys.exit("no answer from the input relay (an older one, without pausing?)")
-    finally:
-        s.close()
-
-
-def main():
-    cmd = sys.argv[1] if len(sys.argv) > 1 else "status"
-    if cmd in ("on", "off", "toggle"):
-        r = ask(f"pause {cmd} command")
-        print("Frametop paused" if r.get("paused") else "Frametop running")
-    elif cmd == "status":
-        print(json.dumps(ask("pause ?"), indent=1))
-    else:
-        sys.exit(__doc__)
-
-
-if __name__ == "__main__":
-    main()

input/game_pause.py

@@ -1,515 +0,0 @@
-"""Pause Frametop for VR games, so it leaves the headset's CPU and GPU to the game. The input
-relay (input-relay.py) runs this: it's the one part of Frametop that always runs.
-
-Paused:
-  - The gaze service stops (frametop-gaze: ft-gazed, ft-gaze, our own eye tracker ft-eyes, the
-    gaze panel), and with it every read of SteamVR's eye tracking. Our frame grabber, the root
-    service ft-eyegrab, goes idle by itself 3 seconds after ft-eyes stops asking for frames.
-  - Hand tracking stops if it runs (frametop-camd, frametop-hands).
-  - The desktop, by "pause_desktop": "hide" (the default) hides every screen and floating window
-    and slows the desktop down ("pause on" to ft-screens: KWin gets one frame callback a second
-    instead of 90, so it and the apps on it hardly draw), and remote desktop stops if it runs
-    (session/remote-ctl.sh: krdp, Xvnc, FreeRDP). Windows stay open. "close" closes the desktop
-    (desktops.sh stop): its windows close, and resuming starts it again (about 12 seconds), in
-    its start profile if it has one.
-  - In the relay: the 3D mouse lets go and the mouse is a plain mouse for SteamVR, typing goes to
-    Steam, and mapped buttons and key combinations do nothing but PAUSED_ACTIONS and commands.
-Resuming starts again only what pausing stopped. The pointer helper and ft-powerd keep running:
-they cost little, and the helper is what says a VR game started.
-
-Toggled by:
-  - The pause_toggle action: a mouse button, a key combination, or a controller button outside
-    games (the pointer helper only reads those outside games).
-  - A controller gesture, by default both thumbsticks clicked together twice ("pause_gesture").
-    It's read from vrserver's web socket (vrws.py), which works in games and takes nothing from
-    them, so the game sees the clicks too.
-  - input/ft-pause on|off|toggle|status, which uses the relay's control socket
-    ("pause on|off|toggle|?").
-  - VR games, with "pause_auto" (on by default): the pointer helper says "vrgame 1|0" as a SteamVR
-    scene app starts and ends (and every 5 seconds). A game starting pauses; a pause that starts
-    while a game runs ends RESUME_DELAY after the game does, unless another game starts first.
-    Toggled back on during a game, Frametop stays on until that game ends. A pause that starts
-    outside a game lasts until it's toggled off.
-
-Settings in ~/.config/frametop-input.json (Frametop Input Settings, Game optimization page): "pause_auto"
-(bool), "pause_gesture" ({"buttons": one or two controller buttons, "presses": 1 or 2}, or null
-for none; one button always takes two presses), "pause_desktop" ("hide" or "close"), and
-"pause_sound" (a sound on pause and resume; bool). The state outlives a relay restart in
-STATE_PATH.
-"""
-import json
-import os
-import queue
-import socket
-import subprocess
-import threading
-import time
-
-import vrws
-
-REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
-DESKTOPS = os.path.join(REPO, "desktops.sh")
-REMOTE_CTL = os.path.join(REPO, "session", "remote-ctl.sh")
-RELAY = "\0frametop_relay"
-SCREENS = "\0ft_screens"
-STATE_PATH = f"/run/user/{os.getuid()}/frametop-pause.json"
-# User services that pausing stops, if they run.
-UNITS = ("frametop-gaze.service", "frametop-hands.service", "frametop-camd.service")
-# What still works while paused, besides commands ("command:CMD"): the toggle itself, and Steam's menu.
-PAUSED_ACTIONS = ("pause_toggle", "steam_menu")
-DEFAULT_GESTURE = {"buttons": ["left/thumbstick", "right/thumbstick"], "presses": 2}
-DESKTOP_MODES = ("hide", "close")
-SOUNDS = {True: "/usr/share/sounds/ocean/stereo/device-removed.oga",
-          False: "/usr/share/sounds/ocean/stereo/device-added.oga"}
-RESUME_DELAY = 5.0  # after a VR game ends: loading the next one can end one scene app and start another
-GAME_STALE = 12.0  # the helper repeats "vrgame" every 5 s; silence this long means no game (or no SteamVR)
-CHORD_WINDOW = 0.3  # a two-button gesture's buttons go down within this of each other
-DOUBLE_WINDOW = 0.7  # a double press's second press comes within this of the first
-LOOKUP_EVERY = 30.0  # the controllers are looked up again this often (and sooner when they may have changed)
-LOOKUP_GAP = 3.0  # but not more often than this for a message from a device it doesn't know
-
-
-def device_of(text):
-    """A web socket message's "device", without parsing all of it (there are about 160 a second)."""
-    at = text.find('"device"')
-    if at < 0:
-        return None
-    start = text.find('"', text.find(":", at) + 1)
-    end = text.find('"', start + 1)
-    return text[start + 1:end] if 0 <= start < end else None
-
-
-def gesture_of(rules, buttons):
-    """The rules' pause gesture as (buttons, presses), or None for none. `buttons` are the
-    controller buttons there are (the relay's VR_BUTTONS)."""
-    spec = rules.get("pause_gesture", DEFAULT_GESTURE)
-    if not isinstance(spec, dict) or not isinstance(spec.get("buttons"), list):
-        return None
-    chosen = tuple(dict.fromkeys(b for b in spec["buttons"] if b in buttons))[:2]
-    if not chosen:
-        return None
-    return chosen, 1 if len(chosen) == 2 and spec.get("presses") == 1 else 2
-
-
-def settings_of(rules, buttons):
-    return {"auto": rules.get("pause_auto", True) is not False,
-            "gesture": gesture_of(rules, buttons),
-            "desktop": rules.get("pause_desktop") if rules.get("pause_desktop") in DESKTOP_MODES else "hide",
-            "sound": rules.get("pause_sound", True) is not False}
-
-
-class Gesture:
-    """The pause gesture, from button presses and releases. A press of a two-button gesture is
-    both buttons going down within CHORD_WINDOW of each other, so a button held for a while (to
-    sprint, say) and the other clicked doesn't count. Two presses within DOUBLE_WINDOW make a
-    double press."""
-
-    def __init__(self, buttons, presses):
-        self.buttons, self.presses = tuple(buttons), presses
-        self.down = {}  # button -> when it went down
-        self.first = None  # when the first press of a double press was
-        self.fired = None
-
-    def reset(self):
-        self.down.clear()
-        self.first = None
-
-    def feed(self, button, pressed, now):
-        """A button's state; True when it completes the gesture."""
-        if button not in self.buttons:
-            return False
-        if not pressed:
-            self.down.pop(button, None)
-            return False
-        if button in self.down:
-            return False  # still down
-        self.down[button] = now
-        if len(self.down) < len(self.buttons) or now - min(self.down.values()) > CHORD_WINDOW:
-            return False
-        if self.presses == 2 and (self.first is None or now - self.first > DOUBLE_WINDOW):
-            self.first = now
-            return False
-        self.first = None
-        if self.fired is not None and now - self.fired < 1.0:
-            return False  # a triple press is still one
-        self.fired = now
-        return True
-
-
-class ControllerWatch(threading.Thread):
-    """Follows the Frame controllers on vrserver's web socket for the pause gesture, and sends
-    the relay "pause toggle gesture" when it's made. Reconnects every 5 seconds while SteamVR
-    is away. With no gesture set, it doesn't connect."""
-
-    def __init__(self, log):
-        super().__init__(name="pause-gesture", daemon=True)
-        self.log = log
-        self.lock = threading.Lock()
-        self.gesture = None
-        self.changed = threading.Event()
-        self.connected = False
-        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
-        self.lookup_at = None  # look the controllers up again then (see recheck)
-
-    def recheck(self, delay=1.5):
-        """The controllers' root paths may change soon: the 3D mouse's virtual controller is taking
-        or giving back its hand role. SteamVR takes a moment to hand the role over."""
-        self.lookup_at = time.monotonic() + delay
-
-    def set_gesture(self, spec):
-        """(buttons, presses), or None for no gesture."""
-        with self.lock:
-            g = self.gesture
-            if spec is None:
-                self.gesture = None
-            elif g is None or (g.buttons, g.presses) != spec:
-                self.gesture = Gesture(*spec)
-        self.changed.set()
-
-    def current(self):
-        with self.lock:
-            return self.gesture
-
-    def run(self):
-        while True:
-            if self.current() is None:
-                if self.connected:
-                    self.log("pause gesture: none set, stopped reading the controllers")
-                self.connected = False
-                self.changed.wait()
-                self.changed.clear()
-                continue
-            try:
-                self.session()
-            except (OSError, ValueError) as e:  # urllib's errors are OSErrors
-                if self.connected:
-                    self.log(f"pause gesture: SteamVR's web socket went away ({e}); retrying")
-                self.connected = False
-            self.changed.wait(5)
-            self.changed.clear()
-
-    def session(self):
-        ws = vrws.VrSocket()
-        try:
-            ws.open(f"frametop_pause_{os.getpid()}")
-            sides = {}  # root path -> side
-            next_poll = 0.0
-            last_poll = 0.0
-            while True:
-                g = self.current()
-                if g is None:
-                    return
-                now = time.monotonic()
-                lookup_at = self.lookup_at
-                if lookup_at is not None and now >= lookup_at:
-                    self.lookup_at = None
-                    next_poll = now
-                if now >= next_poll:
-                    # A controller connects later, or changes its root path when the 3D mouse's
-                    # virtual controller takes or gives back its hand role. Each lookup is an HTTP
-                    # request to vrserver, so it's at connect, every LOOKUP_EVERY, when a message
-                    # comes from a device not on the list, and when the relay says (recheck).
-                    next_poll, last_poll = now + LOOKUP_EVERY, now
-                    found = vrws.controllers()
-                    if found != sides:
-                        for path in sides.keys() - found.keys():
-                            ws.subscribe(path, on=False)
-                        for path in found.keys() - sides.keys():
-                            ws.subscribe(path)
-                        sides = found
-                        g.reset()
-                    if not self.connected:
-                        self.connected = True
-                        self.log(f"pause gesture: following {' and '.join(sorted(sides.values())) or 'no'} "
-                                 f"controller{'s' if len(sides) != 1 else ''} on SteamVR's web socket")
-                wanted = {b: (b.split("/", 1)[0], f"/input/{b.split('/', 1)[1]}/click") for b in g.buttons}
-                for text in ws.messages(timeout=1.0):
-                    device = device_of(text)
-                    if device and device not in sides:
-                        next_poll = min(next_poll, last_poll + LOOKUP_GAP)  # its root path changed, maybe
-                    # About 160 messages a second, nearly all capacitive sensing: parse only the
-                    # few that carry one of the gesture's buttons.
-                    if not any(c in text for _, c in wanted.values()):
-                        continue
-                    try:
-                        msg = json.loads(text)
-                    except ValueError:
-                        continue
-                    side = sides.get(msg.get("device"))
-                    components = msg.get("components") or {}
-                    for button, (s, component) in wanted.items():
-                        if s == side and component in components:
-                            if g.feed(button, bool(components[component]), time.monotonic()):
-                                self.log("pause gesture")
-                                try:
-                                    self.sock.sendto(b"pause toggle gesture", RELAY)
-                                except OSError:
-                                    pass
-        finally:
-            ws.close()
-
-
-def run(cmd, timeout=30):
-    """A command's exit status (None if it couldn't run), its output going nowhere."""
-    try:
-        return subprocess.run(cmd, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
-                              stderr=subprocess.DEVNULL, timeout=timeout).returncode
-    except (OSError, subprocess.TimeoutExpired):
-        return None
-
-
-def output(cmd, timeout=10):
-    try:
-        return subprocess.run(cmd, stdin=subprocess.DEVNULL, capture_output=True, text=True,
-                              timeout=timeout).stdout
-    except (OSError, subprocess.TimeoutExpired):
-        return ""
-
-
-def active(unit):
-    return run(["systemctl", "--user", "-q", "is-active", unit], timeout=10) == 0
-
-
-def desktop_running():
-    return run(["pgrep", "-x", "ft-screens"], timeout=5) == 0
-
-
-class Worker(threading.Thread):
-    """Does the slow part of pausing and resuming, in order, one job at a time: stopping and
-    starting services, remote desktop, and closing or starting the desktop. `stopped` is what
-    the last pause stopped, for the resume to start again."""
-
-    def __init__(self, log, save):
-        super().__init__(name="pause-worker", daemon=True)
-        self.log, self.save = log, save
-        self.jobs = queue.Queue()
-        self.lock = threading.Lock()
-        self.stopped = {"units": [], "remote": False, "desktop": None}
-        self.busy = False
-
-    def submit(self, job, *args):
-        self.busy = True
-        self.jobs.put((job, args))
-
-    def run(self):
-        while True:
-            job, args = self.jobs.get()
-            try:
-                job(*args)
-            except Exception as e:  # one failure mustn't stop the next pause or resume
-                self.log(f"pause: {job.__name__} failed: {e!r}")
-            self.busy = not self.jobs.empty()
-            self.save()
-
-    def snapshot(self):
-        with self.lock:
-            return {"units": list(self.stopped["units"]), "remote": self.stopped["remote"],
-                    "desktop": self.stopped["desktop"]}
-
-    def restore(self, stopped):
-        with self.lock:
-            self.stopped = {"units": [u for u in stopped.get("units", []) if u in UNITS],
-                            "remote": bool(stopped.get("remote")),
-                            "desktop": stopped.get("desktop") if stopped.get("desktop") in ("hidden", "closed") else None}
-
-    def stop_units(self):
-        """Stop the UNITS that run, adding them to what the resume starts again."""
-        units = [u for u in UNITS if active(u)]
-        if units:
-            run(["systemctl", "--user", "stop", *units], timeout=30)
-            with self.lock:
-                self.stopped["units"] += [u for u in units if u not in self.stopped["units"]]
-            self.log(f"pause: stopped {', '.join(units)}")
-
-    def pause(self, desktop):
-        if desktop == "close":
-            if desktop_running():
-                run([DESKTOPS, "stop"], timeout=40)  # remote desktop goes with it
-                with self.lock:
-                    self.stopped["desktop"] = "closed"
-                self.log("pause: desktop closed")
-        else:
-            if "running=1" in output([REMOTE_CTL, "status"]):
-                run([REMOTE_CTL, "stop"])
-                with self.lock:
-                    self.stopped["remote"] = True
-                self.log("pause: remote desktop stopped")
-            with self.lock:
-                self.stopped["desktop"] = "hidden"  # ft-screens was told already (or isn't running)
-        self.stop_units()
-
-    def enforce(self):
-        """While paused: SteamVR restarted, and started the gaze service with it."""
-        self.stop_units()
-
-    def resume(self):
-        stopped = self.snapshot()
-        units = stopped["units"]
-        if units:
-            # They need SteamVR (Requisite=steamvr.service): without it they start with it next time.
-            if run(["systemctl", "--user", "start", *units], timeout=30) == 0:
-                self.log(f"resume: started {', '.join(units)}")
-            else:
-                self.log(f"resume: couldn't start {', '.join(units)} (SteamVR not running?)")
-        if stopped["remote"]:
-            # In a scope of its own, not this service's, so a relay restart doesn't end it.
-            run(["systemd-run", "--user", "--scope", "--collect", "--quiet", REMOTE_CTL, "start"])
-            self.log("resume: remote desktop started")
-        if stopped["desktop"] == "closed":
-            if run([DESKTOPS, "start"], timeout=60) == 0:
-                self.log("resume: desktop started")
-            else:
-                self.log("resume: the desktop didn't start (see /tmp/frametop-session.log)")
-        with self.lock:
-            self.stopped = {"units": [], "remote": False, "desktop": None}
-
-
-class GamePause:
-    """The pause state, its automatic side, and the relay's way in. `on_change(paused)` is
-    called (from the relay's thread) when the state flips, for the relay's own part."""
-
-    def __init__(self, log, on_change, buttons):
-        self.log, self.on_change, self.buttons = log, on_change, buttons
-        self.lock = threading.Lock()  # the state file, written from both threads
-        self.paused = False
-        self.reason = ""  # what paused it: gesture, button, key, controller, command, game
-        self.since = 0.0  # wall clock
-        self.ends_with_game = False  # resume when the VR game ends
-        self.game = False
-        self.game_seen = 0.0  # last "vrgame" from the helper
-        self.resume_at = None
-        self.settings = settings_of({}, buttons)
-        self.screens = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_NONBLOCK)
-        self.worker = Worker(log, self.save)
-        self.watch = ControllerWatch(log)
-        self.worker.start()
-        self.watch.start()
-        self.load()
-
-    # --- state file ---
-    def save(self):
-        with self.lock:
-            state = {"paused": self.paused, "reason": self.reason, "since": self.since,
-                     "ends_with_game": self.ends_with_game, "stopped": self.worker.snapshot()}
-            try:
-                tmp = STATE_PATH + ".tmp"
-                with open(tmp, "w") as f:
-                    json.dump(state, f)
-                os.replace(tmp, STATE_PATH)
-            except OSError as e:
-                self.log(f"pause: can't save {STATE_PATH}: {e}")
-
-    def load(self):
-        """A relay restarted while paused stays paused (the relay's part is applied by its first
-        configure)."""
-        try:
-            with open(STATE_PATH) as f:
-                state = json.load(f)
-        except (OSError, ValueError):
-            return
-        if not isinstance(state, dict) or not state.get("paused"):
-            return
-        self.paused = True
-        self.reason = str(state.get("reason", ""))
-        self.since = float(state.get("since", 0) or 0)
-        self.ends_with_game = bool(state.get("ends_with_game"))
-        self.worker.restore(state.get("stopped") or {})
-        self.log(f"paused since before this relay started ({self.reason})")
-        self.worker.submit(self.worker.enforce)
-
-    # --- settings ---
-    def configure(self, rules):
-        self.settings = settings_of(rules, self.buttons)
-        self.watch.set_gesture(self.settings["gesture"])
-        if not self.settings["auto"]:
-            self.ends_with_game, self.resume_at = False, None
-
-    # --- pausing and resuming ---
-    def sound(self, paused):
-        if self.settings["sound"] and os.path.exists(SOUNDS[paused]):
-            try:
-                subprocess.Popen(["pw-play", SOUNDS[paused]], stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
-                                 stderr=subprocess.DEVNULL, start_new_session=True)
-            except OSError:
-                pass
-
-    def tell_screens(self, paused):
-        try:
-            self.screens.sendto(b"pause on" if paused else b"pause off", SCREENS)
-        except OSError:
-            pass  # the desktop isn't running
-
-    def pause(self, reason, now):
-        if self.paused:
-            return
-        self.paused, self.reason, self.since = True, reason, time.time()
-        self.ends_with_game = self.game and self.settings["auto"]
-        self.resume_at = None
-        self.log(f"Frametop paused ({reason}{', until the game ends' if self.ends_with_game else ''})")
-        self.sound(True)
-        self.tell_screens(True)  # at once, even when the desktop is closing: that takes a while
-        self.on_change(True)
-        self.worker.submit(self.worker.pause, self.settings["desktop"])
-        self.save()
-
-    def resume(self, reason, now):
-        if not self.paused:
-            return
-        self.paused, self.resume_at, self.ends_with_game = False, None, False
-        self.log(f"Frametop resumed ({reason})")
-        self.sound(False)
-        self.tell_screens(False)
-        self.on_change(False)
-        self.worker.submit(self.worker.resume)
-        self.save()
-
-    def toggle(self, reason, now):
-        """Pausing only happens by itself as a game starts, so a game resumed during stays on."""
-        (self.resume if self.paused else self.pause)(reason, now)
-
-    def set(self, on, reason, now):
-        if on != self.paused:
-            self.toggle(reason, now)
-
-    # --- VR games ---
-    def game_state(self, running, now):
-        """From the pointer helper: a VR game runs or not (on a change and every 5 s)."""
-        self.game_seen = now
-        if running == self.game:
-            return
-        self.game = running
-        self.log(f"a VR game {'started' if running else 'ended'}")
-        if running:
-            self.resume_at = None  # the next game came before the resume
-            if self.settings["auto"]:
-                self.pause("game", now)
-        elif self.paused and self.ends_with_game:
-            self.resume_at = now + RESUME_DELAY
-
-    def tick(self, now):
-        if self.game and now - self.game_seen > GAME_STALE:
-            self.game_state(False, now)  # the helper went quiet: SteamVR (or the helper) stopped
-        if self.resume_at is not None and now >= self.resume_at:
-            self.resume("the game ended", now)
-
-    def timeout(self, now):
-        """How long the relay may wait before tick() has something to do."""
-        if self.resume_at is not None:
-            return max(0.05, self.resume_at - now)
-        return 1.0 if self.game else 3600.0
-
-    def controllers_changed(self):
-        """The 3D mouse took or gave back its hand role, which changes a controller's root path."""
-        self.watch.recheck()
-
-    def helper_started(self):
-        """The pointer helper (re)started, so SteamVR did too, maybe with the gaze service."""
-        if self.paused:
-            self.worker.submit(self.worker.enforce)
-
-    def status(self):
-        return {"t": "pause", "paused": self.paused, "reason": self.reason, "since": self.since,
-                "ends_with_game": self.ends_with_game, "game": self.game,
-                "busy": self.worker.busy, "stopped": self.worker.snapshot(),
-                "gesture_reader": self.watch.connected, "auto": self.settings["auto"],
-                "desktop": self.settings["desktop"]}

input/input-relay.py

@@ -32,8 +32,7 @@ keyboard_toggle = open or close Frametop's keyboard, float_toggle = float the de
 pointer (else the active one) in VR, or put it back if it floats, dock_all = put every floating
 window back (both to ft-floatd, @frametop_float), profile:NAME = switch to that profile (ft-layout
 use NAME: its screens and apps; docs/profiles.md), steam_menu = open the SteamVR dashboard on
-Steam's menu, or close the dashboard (steam/ft-steam menu, through Steam's UI), pause_toggle =
+Steam's menu, or close the dashboard (steam/ft-steam menu, through Steam's UI), command:CMD = run
-pause Frametop for a VR game, or resume it (game_pause.py), command:CMD = run
 CMD with sh -c (on the host, as this service: its environment, output to its log, and
 COMMAND_PATH, so ft-layout, ft-float and ft-steam need no path), key = pass through as a key, none).
 
@@ -46,8 +45,8 @@ A modifier on its own ("125": Meta) is a tap: pressed and released with no other
 button, or scroll in between; the desktop gets an F24 press before its release, so Plasma's
 launcher doesn't open on a Meta tap that's bound. A rules file without "key_bindings" gets
 DEFAULT_KEY_BINDINGS (Meta tap: steam_menu, Meta+J: gaze_left, Meta+K: gaze_right, Meta+Shift+F:
-float_toggle); one with its own, even an empty one, doesn't. The float, profile, Steam menu,
+float_toggle); one with its own, even an empty one, doesn't. The float, profile, Steam menu and
-pause and command actions work without pointer mode too. A combination with Meta also sends the desktop an
+command actions work without pointer mode too. A combination with Meta also sends the desktop an
 F24 press and Meta's release right away: so letting go of Meta doesn't open Plasma's launcher, and
 a gaze click isn't Meta+click (KWin's window move and resize). Another key while Meta is still
 held gives the desktop Meta back. While typing goes to Steam, keyboards aren't grabbed, so Steam
@@ -93,12 +92,6 @@ keyboards) get their volume entries remapped to unused stand-in codes, so their
 other keys keep working for SteamVR; a device without a keymap that has only volume
 keys (the headset's pmic_resin) is grabbed. The keymaps go back when the relay exits.
 
-Frametop can pause for VR games (game_pause.py: by hand, with a controller gesture, or by itself
-while a game runs). Paused, it stops the services that cost the game CPU and GPU, and the relay
-plays a plain one: the pointer devices feed the virtual mouse and keyboard as without pointer
-mode, typing goes to Steam, and mapped buttons and key combinations do only pause_toggle,
-steam_menu and command:CMD (game_pause.PAUSED_ACTIONS).
-
 Pointer mode (POINTER=1 in ~/.config/frametop.conf) sends pointer devices to
 the ft-pointer helper (pointer/helper), which drives the ft_pointer
 SteamVR driver. With POINTER=0, pointer devices go to the virtual mouse and
@@ -111,10 +104,7 @@ Control socket (abstract datagram @frametop_relay, JSON replies to the sender):
   vrcapture <s>     take every controller button for s seconds (0: stop), so the settings
                     app can capture one; watchers see them as events with id frame_controller
   vrbtn, vrhello, gazeawake   from the pointer helper (above)
-  vrgame 1|0        from the pointer helper: a VR game runs (on a change and every 5 s), for pausing
   textfield 1|0     from the desktop's input method (above)
-  pause on|off|toggle [reason]   pause Frametop or resume it (input/ft-pause; the gesture reader)
-  pause ?           the pause state, as {"t": "pause", ...}
 
 Runs on the Frame host as a user service (frametop-input-relay.service). The
 virtual devices are parked in systemd's file descriptor store, so a relay
@@ -128,7 +118,6 @@ kernel's evdev and uinput interfaces.
 """
 import array
 import atexit
-import collections
 import errno
 import fcntl
 import json
@@ -141,8 +130,6 @@ import subprocess
 import sys
 import time
 
-import game_pause
-
 # Linux input constants (include/uapi/linux/input-event-codes.h, input.h, uinput.h).
 EV_SYN, EV_KEY, EV_REL, EV_MSC = 0x00, 0x01, 0x02, 0x04
 SYN_REPORT = 0
@@ -209,7 +196,7 @@ RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
 ACTIONS = ("left", "right", "middle", "back", "scroll_up", "scroll_down", "dashboard", "recenter",
            "pointer_toggle", "follow_toggle", "gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right",
            "gaze_quickcal", "sens_up", "sens_down", "layout_reset", "screens_toggle", "keyboard_toggle", "float_toggle",
-           "dock_all", "steam_menu", "pause_toggle", "key", "none")
+           "dock_all", "steam_menu", "key", "none")
 # Gaze mode is a mouse feature: these never come from a controller button (docs/gaze-controllers.md).
 GAZE_ACTIONS = ("gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right", "gaze_quickcal")
 # Key combinations a rules file without "key_bindings" gets: a Meta tap opens Steam's menu, Meta+J
@@ -242,17 +229,7 @@ def known_action(a):
 
 
 def needs_pointer(a):
-    return a not in FLOAT_ACTIONS and a not in ("steam_menu", "pause_toggle") and not a.startswith((PROFILE, COMMAND))
+    return a not in FLOAT_ACTIONS and a != "steam_menu" and not a.startswith((PROFILE, COMMAND))
-
-
-def works_paused(a):
-    """An action that still does something while Frametop is paused (game_pause.py)."""
-    return a in game_pause.PAUSED_ACTIONS or a.startswith(COMMAND)
-
-
-# What pressed pause_toggle, for the log (do_action's source).
-PAUSE_SOURCES = {"mouse": "mouse button", "keyboard": "key combination", "left": "controller button",
-                 "right": "controller button"}
 KEYS = "\0frametop_keys"  # keys of keyboards grabbed for the desktop, for other readers
 REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 FT_LAYOUT = os.path.join(REPO, "layout", "ft-layout")
@@ -460,22 +437,14 @@ class Pointer:
     CLAIM_PULSE = 0.06  # seconds the claim button (switchlaserhand, no click) is held
     RESUME_PAUSE = 1.5  # mouse idle this long, then moving again, re-claims the laser
     WAKE_WINDOW = 1.0  # seconds in which WAKE_COUNTS of motion must add up
-    QUEUE_MAX = 512  # commands kept while the helper is behind (see send)
-    MOVE_EVERY = 0.004  # mouse motion goes to the helper at most this often (see flush)
 
     def __init__(self, sensitivity, idle, wake_counts=40):
-        # Never blocks (see send): a stalled helper must not stall the keyboard, volume keys and pausing.
+        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
-        self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_NONBLOCK)
-        # Commands the helper's full socket didn't take yet, in order: text, or [dyaw, dpitch] for
-        # mouse moves, which add up into one while they wait.
-        self.queue = collections.deque()
-        self.behind_logged = -60.0
         self.sensitivity = sensitivity  # degrees per mouse count
         self.idle = idle
         self.active = False
         self.last_used = 0.0
         self.dx = self.dy = 0
-        self.move_at = 0.0  # motion last went to the helper then
         self.scroll_until = None
         self.claim_at = None  # when to press the claim button
         self.claim_release = None
@@ -488,59 +457,11 @@ class Pointer:
         self.pending_since = 0.0
         self.gaze_awake_until = 0.0  # the helper's gaze mode keeps the pointer until then
 
-    def send(self, command, droppable=False):
+    def send(self, command):
-        """To the helper, in order, without blocking. While the helper doesn't keep up (place and
+        try:
-        grabprobe hold it for seconds, and ft-gazed's 90 Hz gaze fills its socket meanwhile), commands
+            self.sock.sendto(command.encode(), HELPER)
-        wait in the queue and go out from tick(). A droppable one (a scroll notch: scrolling seconds
+        except OSError:
-        late is no use) is dropped instead; presses, releases and the rest are kept, so no button
+            pass  # helper not running (SteamVR not running)
-        stays down. Moves add up (_move)."""
-        if not self.queue:
-            try:
-                self.sock.sendto(command.encode(), HELPER)
-                return
-            except BlockingIOError:
-                pass
-            except OSError:
-                return  # helper not running (SteamVR not running)
-        if not droppable:
-            self._queue(command)
-
-    def _move(self, dyaw, dpitch):
-        if self.queue and isinstance(self.queue[-1], list):
-            self.queue[-1][0] += dyaw
-            self.queue[-1][1] += dpitch
-            return
-        if not self.queue:
-            try:
-                self.sock.sendto(f"move {dyaw:.4f} {dpitch:.4f}".encode(), HELPER)
-                return
-            except BlockingIOError:
-                pass
-            except OSError:
-                return
-        self._queue([dyaw, dpitch])
-
-    def _queue(self, item):
-        if not self.queue and time.monotonic() - self.behind_logged > 60:
-            self.behind_logged = time.monotonic()
-            log("pointer helper is behind: holding its commands (logged once a minute)")
-        self.queue.append(item)
-        if len(self.queue) > self.QUEUE_MAX:
-            self.queue.popleft()  # stalled for long: the oldest goes
-
-    def drain(self):
-        """Send what waits, in order, as far as the helper takes it."""
-        while self.queue:
-            item = self.queue[0]
-            text = f"move {item[0]:.4f} {item[1]:.4f}" if isinstance(item, list) else item
-            try:
-                self.sock.sendto(text.encode(), HELPER)
-            except BlockingIOError:
-                return
-            except OSError:
-                self.queue.clear()  # the helper went away: nothing to deliver to
-                return
-            self.queue.popleft()
 
     def wake(self, now):
         if not self.active:
@@ -568,7 +489,7 @@ class Pointer:
         elif code == REL_Y:
             self.dy += value
         elif code == REL_WHEEL and value:
-            self.send(f"scroll 0 {1 if value > 0 else -1}", droppable=True)
+            self.send(f"scroll 0 {1 if value > 0 else -1}")
             self.scroll_until = now + self.SCROLL_PULSE
 
     def action(self, name, value, now, source="mouse"):
@@ -600,7 +521,7 @@ class Pointer:
             return  # the rest act on press
         elif name in ("scroll_up", "scroll_down"):
             self.wake(now)
-            self.send(f"scroll 0 {1 if name == 'scroll_up' else -1}", droppable=True)
+            self.send(f"scroll 0 {1 if name == 'scroll_up' else -1}")
             self.scroll_until = now + self.SCROLL_PULSE
         elif name == "dashboard":
             self.dashboard(now)
@@ -639,16 +560,11 @@ class Pointer:
             self.sensitivity *= 1.25 if name == "sens_up" else 0.8
             log(f"sensitivity {self.sensitivity:.4f} deg/count")
 
-    def flush(self, now=None):
+    def flush(self):
-        """Send the motion so far. With now (a mouse's SYN_REPORT), only once MOVE_EVERY has passed
+        if self.dx or self.dy:
-        since the last: a 1000 Hz mouse sent the helper, which runs every 8 ms, 1000 datagrams a
+            # Mouse right turns the ray right (negative yaw); mouse down tilts it down.
-        second. tick() sends the rest when it's due; a button sends it first, so it lands there."""
+            self.send(f"move {-self.dx * self.sensitivity:.4f} {-self.dy * self.sensitivity:.4f}")
-        if not (self.dx or self.dy) or (now is not None and now - self.move_at < self.MOVE_EVERY):
+            self.dx = self.dy = 0
-            return
-        self.move_at = time.monotonic() if now is None else now
-        # Mouse right turns the ray right (negative yaw); mouse down tilts it down.
-        self._move(-self.dx * self.sensitivity, -self.dy * self.sensitivity)
-        self.dx = self.dy = 0
 
     def dashboard(self, now=None):
         """Toggle the SteamVR dashboard with the virtual controller's system button.
@@ -663,8 +579,6 @@ class Pointer:
         self.system_at = now + (0.4 if woke else 0.0)
 
     def tick(self, now):
-        self.drain()
-        self.flush(now)
         if self.system_at is not None and now >= self.system_at:
             self.send("btn system 1")
             self.system_at = None
@@ -689,26 +603,7 @@ class Pointer:
 
     def timeout(self):
         pending = (self.scroll_until, self.claim_at, self.claim_release, self.system_at, self.system_release)
-        wait = 0.02 if self.queue or any(t is not None for t in pending) else 0.5
+        return 0.02 if any(t is not None for t in pending) else 0.5
-        if self.dx or self.dy:  # motion held back by flush
-            wait = max(0.0, min(wait, self.move_at + self.MOVE_EVERY - time.monotonic()))
-        return wait
-
-    def stand_down(self):
-        """Frametop is pausing: a pulse under way ends now, and the pointer lets go."""
-        if self.system_release is not None:
-            self.send("btn system 0")
-        if self.claim_release is not None:
-            self.send("btn a 0")
-        if self.scroll_until is not None:
-            self.send("scroll 0 0")
-        self.system_at = self.system_release = self.claim_at = self.claim_release = self.scroll_until = None
-        self.dx = self.dy = self.pending = 0
-        self.gaze_awake_until = 0.0
-        if self.active:
-            self.send("hide")
-            self.active = False
-            log("pointer off (paused)")
 
 
 class Node:
@@ -793,43 +688,23 @@ def main():
     # desktop_until: typing goes to the Frametop desktop until then (ft-screens says so
     # every second); typing_applied: the grabs match that as of the last apply_roles().
     # vr_capture_until: every controller button is taken until then (the settings app capturing one).
-    # pointer: the 3D mouse while it's in use, pointer_conf: the one the config asks for (they
+    state = {"pointer": None, "rules": {}, "share_keys": False,
-    # differ while Frametop is paused).
+             "desktop_until": 0.0, "typing_applied": None, "vr_capture_until": 0.0}
-    state = {"pointer": None, "pointer_conf": None, "rules": {}, "share_keys": False,
-             "desktop_until": 0.0, "typing_applied": None, "vr_capture_until": 0.0, "vr_bind_retry": False,
-             "pointer_holding": False}
-
-    def pause_changed(paused):
-        """Frametop paused or resumed (game_pause.py): the relay's own part."""
-        p = state["pointer_conf"]
-        if paused:
-            if p:
-                p.stand_down()
-            state["pointer"] = None
-            state["desktop_until"] = 0.0  # typing goes to Steam
-        else:
-            state["pointer"] = p
-        apply_roles()
-        vr_bind(time.monotonic())
-
-    pause = game_pause.GamePause(log, pause_changed, VR_BUTTONS)
 
     def load_config():
         conf = read_config()
         state["rules"] = read_rules()
         state["share_keys"] = conf.get("SHARE_KEYS", "0") == "1"
         if conf.get("POINTER", "0") == "1":
-            p = state["pointer_conf"] or Pointer(0.03, 30)
+            p = state["pointer"] or Pointer(0.03, 30)
             p.sensitivity = float(conf.get("POINTER_SENSITIVITY", "0.03"))
             p.idle = float(conf.get("POINTER_IDLE", "30"))
             p.wake_counts = int(conf.get("POINTER_WAKE_COUNTS", "40"))
-            state["pointer_conf"] = p
+            state["pointer"] = p
             log(f"pointer mode: {p.sensitivity} deg/count, idle {p.idle} s, wake {p.wake_counts} counts")
         else:
-            state["pointer_conf"] = None
+            state["pointer"] = None
             log("pointer mode off: pointer devices feed the virtual mouse and keyboard")
-        state["pointer"] = None if pause.paused else state["pointer_conf"]
-        pause.configure(state["rules"])
 
     load_config()
     tap = None  # the modifier (folded, MODIFIERS) pressed alone, with nothing since: its release is a tap
@@ -844,14 +719,11 @@ def main():
             state["vr_capture_until"] = 0.0
             buttons = " ".join(b for b, a in state["rules"]["controller_buttons"].items()
                                if b in VR_BUTTONS and known_action(a) and a not in ("key", "none")
-                               and a not in GAZE_ACTIONS and (not pause.paused or works_paused(a))) or "-"
+                               and a not in GAZE_ACTIONS) or "-"
             if state["rules"].get("controller_in_games"):
                 buttons = "+games " + buttons
-        state["vr_bind_retry"] = False
         try:
             screens_sock.sendto(f"vrbind {buttons}".encode(), HELPER)
-        except BlockingIOError:
-            state["vr_bind_retry"] = True  # the helper is behind: again on the next loop
         except OSError:
             pass  # helper not running; it says vrhello when it starts
 
@@ -892,12 +764,6 @@ def main():
     def do_action(action, value, now, source="mouse"):
         """A mapped mouse or controller button, or key combination (pointer mode only, but
         for the actions needs_pointer() says don't)."""
-        if action == "pause_toggle":
-            if value == 1:
-                pause.toggle(PAUSE_SOURCES.get(source, source), now)
-            return
-        if pause.paused and not works_paused(action):
-            return
         if action == "keyboard_toggle":
             if value == 1 and vr_keyboard_mode() != "never":
                 vr_keyboard("toggle")
@@ -966,7 +832,7 @@ def main():
             return value == 2 and code in combos_down
         combo = "+".join(str(c) for c in sorted(held_modifiers) + [code])
         action = state["rules"]["key_bindings"].get(combo)
-        if not known_action(action) or action in ("key", "none") or (pause.paused and not works_paused(action)):
+        if not known_action(action) or action in ("key", "none"):
             return False
         combos_down[code] = action
         if held_meta - meta_hidden:
@@ -987,7 +853,7 @@ def main():
     def modifier_tap(mod, now):
         """A modifier pressed and released alone: its binding, if it has one ("125": Meta tap)."""
         action = state["rules"]["key_bindings"].get(str(mod))
-        if not known_action(action) or action in ("key", "none") or (pause.paused and not works_paused(action)):
+        if not known_action(action) or action in ("key", "none"):
             return
         # The desktop gets a key in between before the release goes there, so the tap isn't one
         # there too: a Meta tap would open Plasma's launcher.
@@ -1188,7 +1054,7 @@ def main():
             try:
                 if cmd == "keyboard":
                     # From ft-screens (unbound, no reply): where typing goes, repeated every second.
-                    desktop = len(words) > 1 and words[1] == "desktop" and not pause.paused
+                    desktop = len(words) > 1 and words[1] == "desktop"
                     state["desktop_until"] = now + 3.0 if desktop else 0.0
                     continue
                 if cmd == "vrbtn" and len(words) == 3 and words[1] in VR_BUTTONS and words[2] in ("0", "1"):
@@ -1196,20 +1062,6 @@ def main():
                     continue
                 if cmd == "vrhello":
                     vr_bind(now)
-                    pause.helper_started()
-                    continue
-                if cmd == "vrgame" and len(words) == 2:
-                    pause.game_state(words[1] == "1", now)
-                    continue
-                if cmd == "pause" and len(words) >= 2 and words[1] in ("on", "off", "toggle", "?"):
-                    # From input/ft-pause, the settings app, or the gesture reader (unbound, no reply).
-                    reason = words[2] if len(words) > 2 else "command"
-                    if words[1] == "toggle":
-                        pause.toggle(reason, now)
-                    elif words[1] != "?":
-                        pause.set(words[1] == "on", reason, now)
-                    if addr:
-                        reply(addr, pause.status())
                     continue
                 if cmd == "textfield" and len(words) == 2:
                     text_field(words[1] == "1")
@@ -1221,8 +1073,7 @@ def main():
                 if not addr:
                     continue  # unbound sender, nowhere to reply
                 if cmd == "devices":
-                    reply(addr, {"t": "devices", "pointer_mode": state["pointer_conf"] is not None,
+                    reply(addr, {"t": "devices", "pointer_mode": state["pointer"] is not None,
-                                 "paused": pause.paused,
                                  "actions": ACTIONS,
                                  "nodes": [n.describe() for n in nodes.values() if n.candidate]})
                 elif cmd == "watch":
@@ -1304,29 +1155,17 @@ def main():
             if added:
                 apply_roles()
 
-        # The 3D mouse connecting or letting go moves a hand role, so the pause gesture's reader
-        # looks the controllers up again (game_pause.py).
-        holding = bool(state["pointer_conf"] and state["pointer_conf"].active)
-        if holding != state["pointer_holding"]:
-            state["pointer_holding"] = holding
-            pause.controllers_changed()
         ready, _, _ = select.select(list(nodes) + [control], [], [],
-                                    min(volume.timeout(now, pointer.timeout() if pointer else 0.5), pause.timeout(now)))
+                                    volume.timeout(now, pointer.timeout() if pointer else 0.5))
         now = time.monotonic()
         if pointer:
             pointer.tick(now)
-        elif state["pointer_conf"]:
-            state["pointer_conf"].drain()  # paused: what waited still goes, in order
-        if state["vr_bind_retry"]:
-            vr_bind(now)
         volume.tick(now)
-        pause.tick(now)
         if state["vr_capture_until"] and now >= state["vr_capture_until"]:
             vr_bind(now)  # capture over: back to the mapped buttons
         if (now < state["desktop_until"]) != state["typing_applied"] or waiting:
             waiting = apply_roles()
         for fd in ready:
-            pointer = state["pointer"]  # (a pause or resume in this batch changes it)
             if fd is control:
                 handle_control(now)
                 continue
@@ -1381,7 +1220,7 @@ def main():
                     continue
                 if etype == EV_KEY:
                     action = buttons.get(str(code), DEFAULT_BUTTONS.get(code, "key"))
-                    if (pointer or action == "pause_toggle") and action not in ("key", "none"):
+                    if pointer and action not in ("key", "none"):
                         do_action(action, value, now)
                         continue
                     if action == "none":
@@ -1400,7 +1239,7 @@ def main():
                         mouse.emit(etype, code, value)
                 elif etype == EV_SYN and code == SYN_REPORT:
                     if pointer:
-                        pointer.flush(now)
+                        pointer.flush()
                     mouse.sync()
                     keyboard.sync()
 

input/test/keys-test.py

@@ -3,9 +3,8 @@
 
 Runs the relay's main() against a fake pass-through keyboard and a fake mouse (pipes), with
 every socket it sends to renamed, no uinput devices, no grabs, and ft-steam swapped for a
-logger. Pausing (game_pause.py) is a stub: no threads, state file, or services. Nothing reaches
+logger. Nothing reaches the live desktop, SteamVR, or the running relay, so it's safe next
-the live desktop, SteamVR, or the running relay, so it's safe next to them. Rules come from the
+to them. Rules come from the test, not ~/.config/frametop-input.json.
-test, not ~/.config/frametop-input.json.
 
   input/test/keys-test.py [RELAY]   (default: input/input-relay.py next to this folder)
 """
@@ -27,41 +26,8 @@ with open(relay_path) as f:
     src = f.read().replace('"\\0frametop_relay"', f'"\\0{tag}_relay"')
 relay = importlib.util.module_from_spec(importlib.util.spec_from_loader("relay", loader=None))
 relay.__file__ = os.path.abspath(relay_path)
-sys.path.insert(0, os.path.dirname(os.path.abspath(relay_path)))  # its game_pause and vrws
 exec(compile(src, relay_path, "exec"), relay.__dict__)
 
-
-class StubPause:
-    """game_pause.GamePause without its threads, state file, or services."""
-    paused = False
-
-    def __init__(self, log, on_change, buttons):
-        self.on_change = on_change
-        stub["pause"] = self
-
-    def toggle(self, reason, now):
-        self.paused = not self.paused
-        self.on_change(self.paused)
-
-    def set(self, on, reason, now):
-        if on != self.paused:
-            self.toggle(reason, now)
-
-    def status(self):
-        return {"t": "pause", "paused": self.paused}
-
-    def timeout(self, now):
-        return 3600.0
-
-    def configure(self, *args):
-        pass
-
-    game_state = tick = helper_started = controllers_changed = configure
-
-
-stub = {}
-relay.game_pause.GamePause = StubPause
-
 for name in ("SCREENS", "HELPER", "FLOAT", "GAZED", "KEYS"):
     setattr(relay, name, f"\0{tag}_{name.lower()}")
 OUT = tempfile.mkdtemp(prefix="ft-keys-test-")
@@ -114,7 +80,6 @@ fake_os = type(os)("os")
 fake_os.__dict__.update(os.__dict__)
 fake_os.listdir = lambda path: ["event900", "event901"] if path == "/dev/input" else os.listdir(path)
 fake_os.stat = lambda path, *a, **k: Inode() if path in FAKE else os.stat(path, *a, **k)
-fake_os.access = lambda path, mode, *a, **k: path in FAKE or os.access(path, mode, *a, **k)
 relay.os = fake_os
 
 
@@ -178,15 +143,6 @@ def use(key_bindings):
     typed()
 
 
-def pause(cmd):
-    c = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
-    c.bind("")
-    c.settimeout(2)
-    c.sendto(f"pause {cmd} test".encode(), f"\0{tag}_relay")
-    c.recv(4096)
-    time.sleep(0.05)
-
-
 def lines(path, wait=0.3):
     time.sleep(wait)
     try:
@@ -206,7 +162,7 @@ def check(label, got, want):
         failures.append(label)
 
 
-META, RMETA, SHIFT, CTRL, RCTRL, A, F, J, P = 125, 126, 42, 29, 97, 30, 33, 36, 25
+META, RMETA, SHIFT, CTRL, RCTRL, A, F, J = 125, 126, 42, 29, 97, 30, 33, 36
 F24 = ["key 194 1", "key 194 0"]
 
 
@@ -250,25 +206,9 @@ def tests():
     check("right Ctrl tap runs its command, which finds Frametop's tools", lines(cmd_log), ["combo", "3"])
     check("Ctrl tap: F24 before Ctrl's release", typed(), ["key 97 1"] + F24 + ["key 97 0"])
 
-    use({"125+36": "gaze_left", "29": f"command:echo paused >> {cmd_log}", "42+125+25": "pause_toggle"})
-    pause("on")
-    check("pause on: paused", stub["pause"].paused, True)
-    key(META, 1); key(J, 1); key(J, 0); key(META, 0)
-    check("paused: Meta+J (gaze click) does nothing and is typed as is", typed(),
-          ["key 125 1", "key 36 1", "key 36 0", "key 125 0"])
-    key(RCTRL, 1); key(RCTRL, 0)
-    check("paused: a command still runs", lines(cmd_log), ["combo", "3", "paused"])
-    typed()
-    key(SHIFT, 1); key(META, 1); key(P, 1); key(P, 0); key(META, 0); key(SHIFT, 0)
-    check("paused: Meta+Shift+P (pause_toggle) resumes", stub["pause"].paused, False)
-    check("Meta+Shift+P isn't typed", [k for k in typed() if k.startswith("key 25 ")], [])
-    key(META, 1); key(J, 1); key(J, 0); key(META, 0)
-    check("resumed: Meta+J is a combination again", typed(), ["key 125 1"] + F24 + ["key 125 0"])
-
     check("an empty command isn't an action", relay.known_action("command:  "), False)
-    check("steam_menu, pause_toggle and commands work without pointer mode",
+    check("steam_menu and commands work without pointer mode",
-          (relay.needs_pointer("steam_menu"), relay.needs_pointer("pause_toggle"), relay.needs_pointer("command:ls")),
+          (relay.needs_pointer("steam_menu"), relay.needs_pointer("command:ls")), (False, False))
-          (False, False, False))
     print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
     shutil.rmtree(OUT, ignore_errors=True)
     os._exit(1 if failures else 0)

input/test/pause-test.py

@@ -1,214 +0,0 @@
-#!/usr/bin/env python3
-"""Offline test of pausing for VR games (input/game_pause.py): the controller gesture, and when
-Frametop pauses and resumes by itself. The worker (services, remote desktop, the desktop) and
-the controller reader are stubs, the state file is a temporary one, and ft-screens' socket is
-renamed, so nothing reaches the live Frametop, SteamVR, or the running relay.
-
-  input/test/pause-test.py
-"""
-import os
-import sys
-import tempfile
-
-sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), ".."))
-import game_pause as gp  # noqa: E402
-
-failures = []
-
-
-def check(label, got, want):
-    ok = got == want
-    print(("ok    " if ok else "FAIL  ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
-    if not ok:
-        failures.append(label)
-
-
-# ---------------------------------------------------------------- the gesture
-L, R = "left/thumbstick", "right/thumbstick"
-
-
-def presses(g, events):
-    """events: (time, button, down); how many times the gesture went off."""
-    return sum(g.feed(b, d, t) for t, b, d in events)
-
-
-def both(t, hold=0.1, gap=0.02):
-    """Both sticks clicked at t (the right one gap later), held for hold."""
-    return [(t, L, True), (t + gap, R, True), (t + hold, L, False), (t + hold + gap, R, False)]
-
-
-g = gp.Gesture((L, R), 2)
-check("both sticks twice: goes off", presses(g, both(0) + both(0.3)), 1)
-g = gp.Gesture((L, R), 2)
-check("both sticks once: nothing", presses(g, both(0)), 0)
-g = gp.Gesture((L, R), 2)
-check("both sticks twice, 0.9 s apart: nothing", presses(g, both(0) + both(0.9)), 0)
-g = gp.Gesture((L, R), 2)
-check("left held (sprint), right clicked twice: nothing",
-      presses(g, [(0, L, True), (1.0, R, True), (1.1, R, False), (1.3, R, True), (1.4, R, False)]), 0)
-g = gp.Gesture((L, R), 2)
-check("sticks 0.5 s apart, twice: nothing", presses(g, both(0, gap=0.5, hold=0.6) + both(1.2, gap=0.5, hold=0.6)), 0)
-g = gp.Gesture((L, R), 2)
-check("three double presses in a row: once", presses(g, both(0) + both(0.3) + both(0.6)), 1)
-g = gp.Gesture((L, R), 2)
-check("again after a second: twice in all", presses(g, both(0) + both(0.3) + both(2.0) + both(2.3)), 2)
-g = gp.Gesture((L, R), 2)
-check("repeated down states count once",
-      presses(g, [(0, L, True), (0.01, L, True), (0.02, R, True), (0.03, R, True), (0.1, L, False), (0.1, R, False)]
-              + both(0.3)), 1)
-g = gp.Gesture((L, R), 1)
-check("both sticks once (presses 1): goes off", presses(g, both(0)), 1)
-g = gp.Gesture(("right/b",), 2)
-check("one button double-clicked: goes off",
-      presses(g, [(0, "right/b", True), (0.1, "right/b", False), (0.3, "right/b", True)]), 1)
-g = gp.Gesture(("right/b",), 2)
-check("other buttons don't count", presses(g, [(0, "right/a", True), (0.2, "right/a", True)]), 0)
-
-VR = ("left/thumbstick", "right/thumbstick", "right/b", "right/a")
-check("no pause_gesture: both sticks twice", gp.gesture_of({}, VR), ((L, R), 2))
-check("pause_gesture null: none", gp.gesture_of({"pause_gesture": None}, VR), None)
-check("one button, presses 1: still twice", gp.gesture_of({"pause_gesture": {"buttons": ["right/b"], "presses": 1}}, VR),
-      (("right/b",), 2))
-check("unknown buttons dropped", gp.gesture_of({"pause_gesture": {"buttons": ["right/zz", "right/a"]}}, VR),
-      (("right/a",), 2))
-check("settings: defaults", gp.settings_of({}, VR)["auto"], True)
-check("settings: unknown desktop mode is hide", gp.settings_of({"pause_desktop": "x"}, VR)["desktop"], "hide")
-
-
-# ---------------------------------------------------------------- pausing by itself
-class StubWorker:
-    def __init__(self, log, save):
-        self.jobs = []
-        self.busy = False
-
-    def start(self):
-        pass
-
-    def submit(self, job, *args):
-        self.jobs.append(job.__name__)
-
-    def snapshot(self):
-        return {"units": [], "remote": False, "desktop": None}
-
-    def restore(self, stopped):
-        self.restored = stopped
-
-    def pause(self, desktop):
-        pass
-
-    def resume(self):
-        pass
-
-    def enforce(self):
-        pass
-
-
-class StubWatch:
-    connected = False
-
-    def __init__(self, log):
-        pass
-
-    def start(self):
-        pass
-
-    def set_gesture(self, spec):
-        self.spec = spec
-
-
-tmp = tempfile.mkdtemp(prefix="ft-pause-test-")
-gp.STATE_PATH = os.path.join(tmp, "pause.json")
-gp.SCREENS = f"\0ft_pause_test_{os.getpid()}_screens"
-gp.Worker, gp.ControllerWatch = StubWorker, StubWatch
-changes = []
-
-
-def new(rules=None):
-    """A relay starting afresh: not paused before."""
-    if os.path.exists(gp.STATE_PATH):
-        os.remove(gp.STATE_PATH)
-    changes.clear()
-    p = gp.GamePause(lambda *a: None, changes.append, VR)
-    p.configure(dict({"pause_sound": False}, **(rules or {})))
-    return p
-
-
-p = new()
-p.game_state(True, 0)
-check("a game starts: paused", (p.paused, p.reason, p.ends_with_game), (True, "game", True))
-check("the relay heard", changes, [True])
-p.game_state(True, 5)
-p.tick(5)
-check("the helper repeats it: still paused, nothing new", (p.paused, changes), (True, [True]))
-p.game_state(False, 10)
-p.tick(12)
-check("the game ends: not yet resumed", p.paused, True)
-p.tick(15.1)
-check("5 s later: resumed", (p.paused, changes), (False, [True, False]))
-check("the worker paused and resumed", p.worker.jobs, ["pause", "resume"])
-
-p = new()
-p.game_state(True, 0)
-p.game_state(False, 10)
-p.game_state(True, 12)
-p.tick(16)
-check("the next game starts within 5 s: stays paused", p.paused, True)
-p.game_state(False, 20)
-p.tick(25.1)
-check("and resumes 5 s after that one ends", p.paused, False)
-
-p = new()
-p.game_state(True, 0)
-p.toggle("gesture", 1)
-check("resumed during a game: running", p.paused, False)
-p.game_state(True, 6)
-p.tick(30)
-check("and stays running while that game runs", p.paused, False)
-p.game_state(False, 40)
-p.game_state(True, 50)
-check("the next game pauses again", p.paused, True)
-
-p = new()
-p.toggle("command", 0)
-check("paused outside a game: not tied to one", (p.paused, p.ends_with_game), (True, False))
-p.game_state(True, 1)
-p.game_state(False, 10)
-p.tick(20)
-check("a game comes and goes: still paused", p.paused, True)
-p.toggle("command", 21)
-check("toggled: running", p.paused, False)
-
-p = new({"pause_auto": False})
-p.game_state(True, 0)
-check("auto off: a game doesn't pause", p.paused, False)
-p.toggle("gesture", 1)
-p.game_state(False, 10)
-p.tick(20)
-check("auto off: a pause during a game outlasts it", p.paused, True)
-
-p = new()
-p.game_state(True, 0)
-p.tick(0 + gp.GAME_STALE + 0.1)
-check("the helper goes quiet: the game counts as ended", p.game, False)
-p.tick(0 + gp.GAME_STALE + gp.RESUME_DELAY + 0.2)
-check("and the pause ends", p.paused, False)
-
-p = new()
-p.toggle("gesture", 0)
-q = gp.GamePause(lambda *a: None, changes.append, VR)
-check("a relay restarted while paused stays paused", (q.paused, q.reason), (True, "gesture"))
-check("and stops the services again", q.worker.jobs, ["enforce"])
-q.toggle("gesture", 1)
-r = gp.GamePause(lambda *a: None, changes.append, VR)
-check("resumed: the next relay starts running", r.paused, False)
-
-p = new({"pause_gesture": None})
-check("pause_gesture null: the reader gets none", p.watch.spec, None)
-check("status", sorted(p.status()), sorted(["t", "paused", "reason", "since", "ends_with_game", "game", "busy",
-                                            "stopped", "gesture_reader", "auto", "desktop"]))
-
-print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
-for f in os.listdir(tmp):
-    os.remove(os.path.join(tmp, f))
-os.rmdir(tmp)
-sys.exit(1 if failures else 0)

input/vrws.py

@@ -1,198 +0,0 @@
-#!/usr/bin/python3
-"""vrws: vrserver's local web socket, with nothing but the standard library.
-
-vrserver (SteamVR) serves its controller binding page on 127.0.0.1:27062, and that page
-follows the controllers' raw input through a web socket there (mailbox input_server,
-request_input_state_updates). Reading it takes nothing from anyone: whatever the buttons are
-bound to still happens, and it works whatever app has input focus, VR games included. SteamVR
-input only reaches the focused app, or an overlay with global input, which then takes the button
-from the game (pointer/helper/vrbuttons.h). frame-voice reads its push-to-talk button the same
-way. It's undocumented, so a SteamVR update could change it (scripts/update-check.py checks it).
-The host's Python has no `websockets` module, so this is a small RFC 6455 client of its own.
-
-  controllers()   the Frame controllers SteamVR has now: {root path: "left" | "right"}. A
-                  controller's root path is /user/hand/<side>, or /devices/cv/<serial> while the
-                  3D mouse's virtual controller holds that hand role.
-  VrSocket()      a connection: open(mailbox), subscribe(path), messages(timeout) -> raw texts,
-                  each a JSON object like {"type": "update_component_states", "device": path,
-                  "components": {"/input/thumbstick/click": true, ...}} with the components
-                  that changed (mostly capacitive sensing, about 160 a second)
-
-Run as a program, it prints the controllers' button changes for a while (default 10 s):
-
-  input/vrws.py [seconds]
-"""
-import base64
-import json
-import os
-import select
-import socket
-import struct
-import sys
-import time
-import urllib.request
-
-HOST, PORT = "127.0.0.1", 27062
-ORIGIN = f"http://{HOST}:{PORT}"
-# vrserver answers the binding page's own requests; these headers make ours look like them.
-HEADERS = {"Referer": f"{ORIGIN}/dashboard/controllerbinding.html"}
-
-
-def getstate(timeout=3):
-    """vrserver's /input/getstate.json: the devices and their input components."""
-    req = urllib.request.Request(f"{ORIGIN}/input/getstate.json", headers=HEADERS)
-    with urllib.request.urlopen(req, timeout=timeout) as resp:
-        return json.load(resp)
-
-
-def controllers(timeout=3):
-    """{root path: side} for the Frame controllers SteamVR has now."""
-    return {d["root_path"]: d["side"] for d in getstate(timeout).get("devices", [])
-            if d.get("controller_type") == "frame_controller" and d.get("root_path")
-            and d.get("side") in ("left", "right")}
-
-
-class VrSocket:
-    def __init__(self, timeout=3):
-        self.sock = socket.create_connection((HOST, PORT), timeout=timeout)
-        try:
-            key = base64.b64encode(os.urandom(16)).decode()
-            self.sock.sendall((f"GET / HTTP/1.1\r\nHost: {HOST}:{PORT}\r\nUpgrade: websocket\r\n"
-                               f"Connection: Upgrade\r\nSec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n"
-                               f"Origin: {ORIGIN}\r\nReferer: {HEADERS['Referer']}\r\n\r\n").encode())
-            head = b""
-            while b"\r\n\r\n" not in head:
-                chunk = self.sock.recv(4096)
-                if not chunk:
-                    raise OSError("vrserver closed the connection during the handshake")
-                head += chunk
-            head, rest = head.split(b"\r\n\r\n", 1)
-            status = head.split(b"\r\n", 1)[0]
-            if b" 101 " not in status + b" ":
-                raise OSError(f"vrserver refused the web socket: {status.decode(errors='replace')}")
-        except BaseException:
-            self.sock.close()
-            raise
-        self.sock.settimeout(None)
-        self.buf = bytearray(rest)
-        self.parts = bytearray()  # a fragmented message so far
-        self.mailbox = ""
-
-    def fileno(self):
-        return self.sock.fileno()
-
-    def close(self):
-        try:
-            self._frame(0x8, b"")
-        except OSError:
-            pass
-        self.sock.close()
-
-    def _frame(self, opcode, payload):
-        """One frame to vrserver: final, masked (as a client's must be)."""
-        n = len(payload)
-        head = bytes([0x80 | opcode])
-        if n < 126:
-            head += bytes([0x80 | n])
-        elif n < 65536:
-            head += bytes([0x80 | 126]) + struct.pack(">H", n)
-        else:
-            head += bytes([0x80 | 127]) + struct.pack(">Q", n)
-        mask = os.urandom(4)
-        self.sock.sendall(head + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(payload)))
-
-    def send(self, text):
-        self._frame(0x1, text.encode())
-
-    def open(self, mailbox):
-        self.send(f"mailbox_open {mailbox}")
-        self.mailbox = mailbox
-
-    def subscribe(self, path, on=True):
-        kind = "request_input_state_updates" if on else "cancel_input_state_updates"
-        self.send("mailbox_send input_server " +
-                  json.dumps({"type": kind, "device_path": path, "returnAddress": self.mailbox}))
-
-    def messages(self, timeout=None):
-        """The text messages that came in, as strings: waits up to `timeout` seconds (None:
-        forever) for some, [] at the timeout. Raises OSError when the connection ends."""
-        out = self._parse()
-        if out:
-            return out
-        if not select.select([self.sock], [], [], timeout)[0]:
-            return []
-        chunk = self.sock.recv(65536)
-        if not chunk:
-            raise OSError("vrserver closed the web socket")
-        self.buf += chunk
-        return self._parse()
-
-    def _parse(self):
-        """Every whole frame in the buffer; a partial one waits for the rest."""
-        out, buf, pos = [], self.buf, 0
-        while len(buf) - pos >= 2:
-            b0, b1 = buf[pos], buf[pos + 1]
-            n, head = b1 & 0x7F, 2
-            if n == 126:
-                if len(buf) - pos < 4:
-                    break
-                n, head = struct.unpack_from(">H", buf, pos + 2)[0], 4
-            elif n == 127:
-                if len(buf) - pos < 10:
-                    break
-                n, head = struct.unpack_from(">Q", buf, pos + 2)[0], 10
-            masked = b1 & 0x80
-            if masked:
-                head += 4
-            if len(buf) - pos < head + n:
-                break
-            data = bytes(buf[pos + head:pos + head + n])
-            if masked:  # servers don't mask, but nothing says they can't
-                mask = buf[pos + head - 4:pos + head]
-                data = bytes(b ^ mask[i % 4] for i, b in enumerate(data))
-            pos += head + n
-            opcode = b0 & 0x0F
-            if opcode == 0x8:
-                raise OSError("vrserver closed the web socket")
-            if opcode == 0x9:
-                self._frame(0xA, data)  # ping: pong
-            elif opcode in (0x0, 0x1, 0x2):
-                self.parts += data
-                if b0 & 0x80:
-                    out.append(self.parts.decode(errors="replace"))
-                    self.parts = bytearray()
-        del buf[:pos]
-        return out
-
-
-def main():
-    seconds = float(sys.argv[1]) if len(sys.argv) > 1 else 10.0
-    ws = VrSocket()
-    ws.open(f"frametop_vrws_{os.getpid()}")
-    sides = controllers()
-    for path, side in sides.items():
-        print(f"{side} controller: {path}", flush=True)
-        ws.subscribe(path)
-    if not sides:
-        print("no Frame controllers", flush=True)
-    start = time.monotonic()
-    count = 0
-    try:
-        while time.monotonic() - start < seconds:
-            for text in ws.messages(timeout=0.5):
-                count += 1
-                if "/click" not in text:
-                    continue
-                msg = json.loads(text)
-                side = sides.get(msg.get("device"), msg.get("device"))
-                for name, value in sorted(msg.get("components", {}).items()):
-                    if name.endswith("/click"):
-                        print(f"{time.monotonic() - start:7.3f}  {side} {name[len('/input/'):-len('/click')]} "
-                              f"{'down' if value else 'up'}", flush=True)
-    finally:
-        ws.close()
-    print(f"{count} messages in {seconds:.0f} s")
-
-
-if __name__ == "__main__":
-    main()

pointer/driver/driver_ft_pointer.cpp

@@ -12,7 +12,7 @@
 //   aim <yaw_deg> <pitch_deg>   absolute direction (yaw 0 = -Z, the SteamVR forward)
 //   gaze                         follow the head (origin and direction) again
 //   distance <metres>            cursor distance from the anchor (default 1.5)
-//   pose <x> <y> <z> <yaw> <pitch>  exact pose, from the ft-pointer helper when it changes (it holds)
+//   pose <x> <y> <z> <yaw> <pitch>  exact pose, sent every frame by the ft-pointer helper
 //   posq <x> <y> <z> <qw> <qx> <qy> <qz>  exact pose with a full rotation (tilting a panel while moving it)
 //   btn <name> <0|1>             name: trigger, b, x, system, joystick, a (a = claim the laser, no click)
 //   scroll <x> <y>               joystick deflection -1..1
@@ -224,23 +224,12 @@ public:
         pose.result = ok ? TrackingResult_Running_OK : TrackingResult_Uninitialized;
         pose.deviceIsConnected = snapshot.visible;
         pose_ = pose;
-        // Connected, the pose goes out every frame, as SteamVR expects of a tracked device.
+        VRServerDriverHost()->TrackedDevicePoseUpdated(objectId_, pose_, sizeof(DriverPose_t));
-        // Disconnected, once: it says the same thing every frame after.
-        if (snapshot.visible || !reportedOff_) {
-            VRServerDriverHost()->TrackedDevicePoseUpdated(objectId_, pose_, sizeof(DriverPose_t));
-            reportedOff_ = !snapshot.visible;
-        }
 
-        // Inputs only when they change (and all of them the first time).
         auto input = VRDriverInput();
-        for (int i = 0; i < kButtons; ++i)
+        for (int i = 0; i < kButtons; ++i) input->UpdateBooleanComponent(buttons_[i], snapshot.buttons[i], 0);
-            if (!inputsSent_ || snapshot.buttons[i] != sentButtons_[i])
+        input->UpdateScalarComponent(scrollX_, snapshot.scrollX, 0);
-                input->UpdateBooleanComponent(buttons_[i], sentButtons_[i] = snapshot.buttons[i], 0);
+        input->UpdateScalarComponent(scrollY_, snapshot.scrollY, 0);
-        if (!inputsSent_ || snapshot.scrollX != sentScrollX_)
-            input->UpdateScalarComponent(scrollX_, sentScrollX_ = snapshot.scrollX, 0);
-        if (!inputsSent_ || snapshot.scrollY != sentScrollY_)
-            input->UpdateScalarComponent(scrollY_, sentScrollY_ = snapshot.scrollY, 0);
-        inputsSent_ = true;
     }
 
 private:
@@ -250,10 +239,6 @@ private:
     int32_t role_ = TrackedControllerRole_RightHand;
     bool hinted_ = false;  // whether the role hint currently claims role_
     DriverPose_t pose_{};
-    bool reportedOff_ = false;  // the disconnected pose has gone out (it isn't repeated)
-    // What the input components were last set to.
-    bool inputsSent_ = false, sentButtons_[kButtons] = {};
-    float sentScrollX_ = 0.f, sentScrollY_ = 0.f;
     VRInputComponentHandle_t buttons_[kButtons] = {};
     VRInputComponentHandle_t scrollX_ = 0, scrollY_ = 0;
 };
@@ -308,102 +293,65 @@ private:
         }
     }
 
-    // Parsed first, then applied under the lock, which RunFrame (vrserver's frame) takes too.
     void Handle(const char *cmd) {
-        enum { kNone, kPosq, kPose, kMove, kRecenter, kAim, kGaze, kHide, kShow, kRole, kBtn, kDistance, kScroll };
-        int kind = kNone;
-        char name[32];
-        float f[7];
-        int v = 0, role = 0, button = -1;
-        if (std::sscanf(cmd, "posq %f %f %f %f %f %f %f", &f[0], &f[1], &f[2], &f[3], &f[4], &f[5], &f[6]) == 7) {
-            kind = kPosq;
-        } else if (std::sscanf(cmd, "pose %f %f %f %f %f", &f[0], &f[1], &f[2], &f[3], &f[4]) == 5) {
-            kind = kPose;
-        } else if (std::sscanf(cmd, "move %f %f", &f[0], &f[1]) == 2) {
-            kind = kMove;
-        } else if (std::strncmp(cmd, "recenter", 8) == 0) {
-            kind = kRecenter;
-        } else if (std::sscanf(cmd, "aim %f %f", &f[0], &f[1]) == 2) {
-            kind = kAim;
-        } else if (std::strncmp(cmd, "gaze", 4) == 0) {
-            kind = kGaze;
-        } else if (std::strncmp(cmd, "hide", 4) == 0) {
-            kind = kHide;
-        } else if (std::strncmp(cmd, "show", 4) == 0) {
-            kind = kShow;
-        } else if (std::sscanf(cmd, "role %31s", name) == 1) {
-            role = !std::strcmp(name, "left")     ? TrackedControllerRole_LeftHand
-                   : !std::strcmp(name, "right")  ? TrackedControllerRole_RightHand
-                   : !std::strcmp(name, "stylus") ? TrackedControllerRole_Stylus
-                                                  : 0;
-            if (role) kind = kRole;
-        } else if (std::sscanf(cmd, "btn %31s %d", name, &v) == 2) {
-            for (int i = 0; i < kButtons; ++i)
-                if (std::strcmp(name, kButtonNames[i]) == 0) button = i;
-            if (button >= 0) kind = kBtn;
-        } else if (std::sscanf(cmd, "distance %f", &f[0]) == 1) {
-            kind = kDistance;
-        } else if (std::sscanf(cmd, "scroll %f %f", &f[0], &f[1]) == 2) {
-            kind = kScroll;
-        }
-        if (kind == kNone) return;
-
         std::lock_guard<std::mutex> guard(state_.lock);
-        switch (kind) {
+        char name[32];
-        case kPosq:
+        float a, b;
+        int v;
+        float x, y, z, qw, qx, qy, qz;
+        if (std::sscanf(cmd, "posq %f %f %f %f %f %f %f", &x, &y, &z, &qw, &qx, &qy, &qz) == 7) {
             state_.explicitPose = true;
             state_.hasQuat = true;
             state_.gaze = false;
-            std::memcpy(state_.pos, f, sizeof state_.pos);
+            state_.pos[0] = x;
-            std::memcpy(state_.quat, f + 3, sizeof state_.quat);
+            state_.pos[1] = y;
-            break;
+            state_.pos[2] = z;
-        case kPose:
+            state_.quat[0] = qw;
+            state_.quat[1] = qx;
+            state_.quat[2] = qy;
+            state_.quat[3] = qz;
+        } else if (std::sscanf(cmd, "pose %f %f %f %f %f", &x, &y, &z, &a, &b) == 5) {
             state_.explicitPose = true;
             state_.hasQuat = false;
             state_.gaze = false;
-            std::memcpy(state_.pos, f, sizeof state_.pos);
+            state_.pos[0] = x;
-            state_.yaw = f[3];
+            state_.pos[1] = y;
-            state_.pitch = f[4];
+            state_.pos[2] = z;
-            break;
+            state_.yaw = a;
-        case kMove:
+            state_.pitch = b;
+        } else if (std::sscanf(cmd, "move %f %f", &a, &b) == 2) {
             state_.explicitPose = false;
             if (state_.gaze) state_.recenter = true;  // first move starts from the gaze
-            state_.yaw += f[0];  // wrap by hand: libm remainder() is GLIBC_2.43 in the build container
+            state_.yaw += a;  // wrap by hand: libm remainder() is GLIBC_2.43 in the build container
             while (state_.yaw > 180.f) state_.yaw -= 360.f;
             while (state_.yaw < -180.f) state_.yaw += 360.f;
-            state_.pitch = std::fmax(-85.f, std::fmin(85.f, state_.pitch + f[1]));
+            state_.pitch = std::fmax(-85.f, std::fmin(85.f, state_.pitch + b));
-            break;
+        } else if (std::strncmp(cmd, "recenter", 8) == 0) {
-        case kRecenter:
             state_.explicitPose = false;
             state_.recenter = true;
-            break;
+        } else if (std::sscanf(cmd, "aim %f %f", &a, &b) == 2) {
-        case kAim:
             state_.gaze = false;
-            state_.yaw = f[0];
+            state_.yaw = a;
-            state_.pitch = f[1];
+            state_.pitch = b;
-            break;
+        } else if (std::strncmp(cmd, "gaze", 4) == 0) {
-        case kGaze:
             state_.gaze = true;
-            break;
+        } else if (std::strncmp(cmd, "hide", 4) == 0) {
-        case kHide:
             state_.visible = false;
-            break;
+        } else if (std::strncmp(cmd, "show", 4) == 0) {
-        case kShow:
             state_.visible = true;
-            break;
+        } else if (std::sscanf(cmd, "role %31s", name) == 1) {
-        case kRole:
+            state_.role = !std::strcmp(name, "left")     ? TrackedControllerRole_LeftHand
-            state_.role = role;
+                          : !std::strcmp(name, "right")  ? TrackedControllerRole_RightHand
-            break;
+                          : !std::strcmp(name, "stylus") ? TrackedControllerRole_Stylus
-        case kBtn:
+                                                         : state_.role;
-            state_.buttons[button] = v != 0;
+        } else if (std::sscanf(cmd, "btn %31s %d", name, &v) == 2) {
-            break;
+            for (int i = 0; i < kButtons; ++i)
-        case kDistance:
+                if (std::strcmp(name, kButtonNames[i]) == 0) state_.buttons[i] = v != 0;
-            state_.distance = std::fmax(0.3f, std::fmin(10.f, f[0]));
+        } else if (std::sscanf(cmd, "distance %f", &a) == 1) {
-            break;
+            state_.distance = std::fmax(0.3f, std::fmin(10.f, a));
-        case kScroll:
+        } else if (std::sscanf(cmd, "scroll %f %f", &a, &b) == 2) {
-            state_.scrollX = f[0];
+            state_.scrollX = a;
-            state_.scrollY = f[1];
+            state_.scrollY = b;
-            break;
         }
     }
 

pointer/helper/build.sh

@@ -5,5 +5,5 @@ root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
 "$root/scripts/sync.sh" >/dev/null
 exec "$root/scripts/frame.sh" -C pointer/helper 'set -e; mkdir -p build
 g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -I/opt/steamvr/tools/hellovr_vulkan_linux/src/openvr/headers -I../common \
-  -o build/ft-pointer ft-pointer.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64 -lpthread
+  -o build/ft-pointer ft-pointer.cpp pointer_*.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64 -lpthread
 echo "built build/ft-pointer"'

pointer/helper/pointer.h

@@ -0,0 +1,478 @@
+// ft-pointer's shared code: helpers, the Frame, and the Pointer struct, whose member functions are
+// in ft-pointer.cpp and pointer_*.cpp.
+// "The top" in comments here is the header comment of ft-pointer.cpp.
+#pragma once
+
+#include <openvr.h>
+
+#include "pointer_config.h"
+#include "vrbuttons.h"
+#include "vrmath.h"
+
+extern "C" {
+#include "../../hands/include/fh_gestures.h"
+}
+
+#include <algorithm>
+#include <atomic>
+#include <chrono>
+#include <cmath>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <deque>
+#include <fstream>
+#include <map>
+#include <mutex>
+#include <string>
+#include <thread>
+#include <tuple>
+#include <vector>
+
+#include <climits>
+
+#include <fcntl.h>
+#include <fnmatch.h>
+
+#include <sys/mman.h>
+#include <sys/socket.h>
+#include <sys/stat.h>
+#include <sys/un.h>
+#include <unistd.h>
+
+using namespace md;
+
+// Hand gestures from ft-hands (see "Hands" at the top): /run/user/UID/frametop-hands/gestures,
+// mapped read-only. Version 1 files have pinches only; their grips read as zero.
+class HandGestures {
+public:
+    ~HandGestures() { Close(); }
+    // A consistent copy of the file (its sequence lock), if it's there. Opens it, and
+    // checks it's still the same file, at most once a second.
+    bool Read(fh_gestures_t &out) {
+        const auto now = std::chrono::steady_clock::now();
+        if (now - checked_ > std::chrono::seconds(1)) {
+            checked_ = now;
+            const std::string path = "/run/user/" + std::to_string(getuid()) + "/frametop-hands/gestures";
+            struct stat st;
+            if (stat(path.c_str(), &st) != 0 || size_t(st.st_size) != len_ || st.st_ino != ino_) {
+                Close();
+                const int fd = open(path.c_str(), O_RDONLY | O_CLOEXEC | O_NOFOLLOW);
+                if (fd >= 0 && fstat(fd, &st) == 0 && size_t(st.st_size) >= offsetof(fh_gestures_t, grip)) {
+                    void *m = mmap(nullptr, size_t(st.st_size), PROT_READ, MAP_SHARED, fd, 0);
+                    if (m != MAP_FAILED) map_ = m, len_ = size_t(st.st_size), ino_ = st.st_ino, ++opens;
+                }
+                if (fd >= 0) close(fd);
+            }
+        }
+        if (!map_) return false;
+        const auto *g = static_cast<const fh_gestures_t *>(map_);
+        if (std::memcmp(g->magic, FH_GESTURES_MAGIC, 8) != 0) return false;
+        const size_t n = std::min(len_, sizeof out);
+        for (int tries = 0; tries < 3; ++tries) {
+            const uint64_t seq = __atomic_load_n(&g->seq, __ATOMIC_ACQUIRE);
+            if (seq & 1) continue;
+            std::memset(&out, 0, sizeof out);
+            std::memcpy(&out, map_, n);
+            __atomic_thread_fence(__ATOMIC_ACQUIRE);
+            if (__atomic_load_n(&g->seq, __ATOMIC_RELAXED) != seq) continue;
+            if (out.version < 2) std::memset(out.grip, 0, sizeof out.grip);
+            return true;
+        }
+        return false;
+    }
+    int opens = 0;   // a new file: the counters start over
+
+private:
+    void Close() {
+        if (map_) munmap(map_, len_);
+        map_ = nullptr, len_ = 0, ino_ = 0;
+    }
+    void *map_ = nullptr;
+    size_t len_ = 0;
+    ino_t ino_ = 0;
+    std::chrono::steady_clock::time_point checked_{};
+};
+
+// The HMD's recent poses (standing universe), to turn the gestures' head-frame points into
+// the room as the head was when the cameras took them.
+class PoseHistory {
+public:
+    void Add(std::chrono::steady_clock::time_point t, const vr::HmdMatrix34_t &m) {
+        poses_.push_back({t, m});
+        while (poses_.size() > 128) poses_.pop_front();   // about a second
+    }
+    // The pose at CLOCK_MONOTONIC time t_ns (steady_clock's), or the nearest kept.
+    bool At(uint64_t t_ns, vr::HmdMatrix34_t &out) const {
+        if (poses_.empty()) return false;
+        const std::chrono::steady_clock::time_point t{std::chrono::nanoseconds(t_ns)};
+        const auto *best = &poses_.front();
+        for (const auto &p : poses_)
+            if (std::chrono::abs(p.first - t) < std::chrono::abs(best->first - t)) best = &p;
+        out = best->second;
+        return true;
+    }
+
+private:
+    std::deque<std::pair<std::chrono::steady_clock::time_point, vr::HmdMatrix34_t>> poses_;
+};
+
+// One of ft-screens' panels showing a desktop: a screen (frametop.screen.N), a floating
+// window (frametop.float.N), or a floating window's popup (frametop.float.N.sub.K), not a
+// control of theirs.
+inline bool FramePanel(const std::string &key) {
+    for (const char *prefix : {"frametop.screen.", "frametop.float."}) {
+        if (key.rfind(prefix, 0) != 0) continue;
+        const std::string rest = key.substr(std::strlen(prefix));
+        const size_t dot = rest.find('.');
+        return dot == std::string::npos || rest.compare(dot, 5, ".sub.") == 0;
+    }
+    return false;
+}
+
+inline void SendTo(int fd, const char *name, const std::string &msg) {
+    sockaddr_un addr{};
+    addr.sun_family = AF_UNIX;
+    std::memcpy(addr.sun_path + 1, name, std::strlen(name));
+    const socklen_t len = offsetof(sockaddr_un, sun_path) + 1 + std::strlen(name);
+    sendto(fd, msg.data(), msg.size(), 0, reinterpret_cast<sockaddr *>(&addr), len);
+}
+
+// JSON string literal (names come from other apps).
+inline std::string JsonQuote(const std::string &s) {
+    std::string out = "\"";
+    for (const unsigned char c : s) {
+        if (c == '"' || c == '\\') out += '\\', out += char(c);
+        else if (c < 0x20) {
+            char esc[8];
+            std::snprintf(esc, sizeof esc, "\\u%04x", c);
+            out += esc;
+        } else out += char(c);
+    }
+    return out + "\"";
+}
+
+// Overlay keys, refreshed in the background from `vrcmd --overlays` (OpenVR has no
+// public call to enumerate other apps' overlays). Hidden ones are listed too: the
+// window controls under a floating panel only appear while something hovers the
+// panel, and the cursor has to find them the moment they do, not a second later.
+// Paused while the pointer is off: each vrcmd run connects to SteamVR as a new app, and a new
+// app every second kept SteamVR (and the headset's displays) from going to standby.
+// "overlays" requests (Frametop Input Settings' Ignored panels page) refresh the list even
+// while paused, and are answered from this thread once it's fresh.
+class OverlayList {
+public:
+    void Start() {
+        out_ = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC, 0);
+        thread_ = std::thread([this] {
+            while (running_) {
+                if (!paused_ || requested_) Refresh();
+                // Wait a second, or less when the pointer wakes (refresh right away then).
+                for (int i = 0; i < 10 && running_; ++i) {
+                    const bool wasPaused = paused_;
+                    std::this_thread::sleep_for(std::chrono::milliseconds(100));
+                    if ((wasPaused && !paused_) || requested_) break;
+                }
+            }
+        });
+    }
+    void SetPaused(bool paused) { paused_ = paused; }
+    void Stop() {
+        running_ = false;
+        if (thread_.joinable()) thread_.join();
+    }
+    std::vector<std::string> Keys() {
+        std::lock_guard<std::mutex> guard(lock_);
+        std::vector<std::string> keys;
+        for (const auto &e : entries_) keys.push_back(e.key);
+        return keys;
+    }
+    // Answer `to` with {"t":"overlays","list":[{"key","name","visible"}...]} after the next refresh.
+    void Request(const sockaddr_un &to, socklen_t len) {
+        if (len <= offsetof(sockaddr_un, sun_path)) return;
+        std::lock_guard<std::mutex> guard(lock_);
+        if (waiting_.size() < 8) waiting_.push_back({to, len});
+        requested_ = true;
+    }
+
+private:
+    struct Entry {
+        std::string key, name;
+        bool visible;
+    };
+    void Refresh() {
+        requested_ = false;
+        FILE *p = popen("LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrcmd --overlays 2>/dev/null", "r");
+        if (!p) return;
+        std::vector<Entry> entries;
+        char line[1024];
+        while (std::fgets(line, sizeof line, p)) {
+            // 'key' -- 'name', WxH visible VROverlayType_...
+            if (line[0] != '\'') continue;
+            const char *end = std::strchr(line + 1, '\'');
+            if (!end) continue;
+            const std::string key(line + 1, size_t(end - (line + 1)));
+            const std::string rest(end);
+            if (rest.find("Thumbnail") != std::string::npos || rest.find("Subview") != std::string::npos) continue;
+            if (key.rfind("system.pointer", 0) == 0 || key.rfind("system.cursor", 0) == 0 ||
+                key.rfind("frametop.pointer", 0) == 0 || key.rfind("frametop.guide", 0) == 0 ||
+                key == "frametop.gazepanel" ||  // the gaze calibration panel, fixed to the headset
+                key == "frametop.catcher" ||  // ft-screens' release catcher: only on a laser mid-drag
+                key == "system.HeadsetView" || key == "system.toast")
+                continue;
+            // The name can hold quotes; it ends at the last "', " (the size and state follow).
+            const auto nameAt = rest.find("-- '"), nameEnd = rest.rfind("', ");
+            const std::string name =
+                nameAt != std::string::npos && nameEnd > nameAt + 3 ? rest.substr(nameAt + 4, nameEnd - nameAt - 4) : key;
+            entries.push_back({key, name, rest.find(" not_visible ") == std::string::npos});
+        }
+        pclose(p);
+        std::vector<std::pair<sockaddr_un, socklen_t>> waiting;
+        {
+            std::lock_guard<std::mutex> guard(lock_);
+            entries_ = std::move(entries);
+            waiting.swap(waiting_);
+        }
+        if (waiting.empty()) return;
+        std::string msg = "{\"t\":\"overlays\",\"list\":[";
+        for (size_t i = 0; i < entries_.size(); ++i)
+            msg += std::string(i ? "," : "") + "{\"key\":" + JsonQuote(entries_[i].key) + ",\"name\":" +
+                   JsonQuote(entries_[i].name) + ",\"visible\":" + (entries_[i].visible ? "true" : "false") + "}";
+        msg += "]}";
+        for (const auto &[to, len] : waiting)
+            sendto(out_, msg.data(), msg.size(), MSG_DONTWAIT, reinterpret_cast<const sockaddr *>(&to), len);
+    }
+
+    std::thread thread_;
+    int out_ = -1;
+    std::atomic<bool> paused_{false};
+    std::atomic<bool> running_{true};
+    std::atomic<bool> requested_{false};
+    std::mutex lock_;
+    std::vector<Entry> entries_;  // written only by the thread; the lock guards readers
+    std::vector<std::pair<sockaddr_un, socklen_t>> waiting_;
+};
+
+using Clock = std::chrono::steady_clock;
+
+// One frame's poses and time: ReadFrame reads them after the relay's commands, and the sections
+// after it share them. handOk: PinchesAndGrips read the hand gestures (for HandsStopped).
+struct Frame {
+    vr::TrackedDevicePose_t all[vr::k_unMaxTrackedDeviceCount];
+    vr::TrackedDevicePose_t hmdRaw;
+    Clock::time_point tnow;
+    Vec3 eye;
+    bool handOk = false;
+    const vr::TrackedDevicePose_t &Hmd() const { return all[0]; }
+};
+
+// Everything the pointer keeps: its settings, what Init sets up, and the state the main loop
+// carries from one frame to the next.
+struct Pointer {
+    PointerConfig cfg;
+    // Head follow (see the top). followConf is POINTER_FOLLOW as last read: a reload only
+    // overrides a "follow" command when the setting itself changed.
+    bool follow = false, followConf = false, followReset = true;
+    // Gaze mode (see the top); gazeConf is POINTER_GAZE as last read, like followConf.
+    bool gazeOn = false, gazeConf = false;
+    // Set in Init, after VR_Init.
+    vr::IVRSystem *sys = nullptr;
+    vr::IVROverlay *overlay = nullptr;
+    // The dot, the marker, and laser mode's overlay, created in Init.
+    vr::VROverlayHandle_t cursor = vr::k_ulOverlayHandleInvalid;
+    vr::VROverlayHandle_t marker = vr::k_ulOverlayHandleInvalid;
+    vr::VROverlayHandle_t laserMode = vr::k_ulOverlayHandleInvalid;
+    bool laserModeShown = false;
+    int in = -1, out = -1;  // @ft_pointer_helper, and the socket we send from (opened in Init)
+    ControllerButtons controllerButtons;  // Frame controller buttons (vrbuttons.h)
+    OverlayList overlays;
+    std::map<std::string, vr::VROverlayHandle_t> handles;
+    std::map<std::string, bool> sceneGraph;  // no texture: plane test instead of ComputeOverlayIntersection
+    std::map<std::string, bool> visible;     // refreshed every 50 ms
+    Clock::time_point lastVisible{};  // set in Init
+    // The plane of the last panel the cursor was on, and the last point on it (panel edges).
+    Vec3 edgePoint, edgeNormal, edgeLast;
+    std::string edgeKey;
+
+    bool active = false, recenter = false, anchored = false;
+    Clock::time_point lastMouse{}, claimAt{}, claimRelease{}, wokeAt{}, noWakeUntil{};
+    bool claimPending = false, claimHeld = false;
+    // Last used wins: since when each controller has been moving (zero: it isn't).
+    Clock::time_point movingSince[vr::k_unMaxTrackedDeviceCount] = {};
+    // Tilt mode (see top of file).
+    bool leftHeld = false, tilting = false, tiltStart = false, swallowedRight = false;
+    // The mouse's buttons in gaze mode (see the top): leftDown, rightDown as the relay last said;
+    // aimRight, the held-back press is the right button's; chordDrag, a drag the right button
+    // began during the left's held-back press (the first release drops it, the other's is nothing).
+    bool leftDown = false, rightDown = false, aimRight = false, chordDrag = false;
+    bool ignoreLeftUp = false;
+    // The keyboard clicks' keys as the relay last said, and a tilt gaze_right holds during a
+    // keyboard drag (see the top): the head turns the panel, from where it was (keyTiltYaw, keyTiltPitch).
+    bool keyLeftDown = false, keyRightDown = false, keyTilting = false;
+    double keyTiltYaw = 0, keyTiltPitch = 0;
+    double tiltYaw = 0, tiltPitch = 0;
+    double dragDistance = 0, lastDistance = 1.5;  // drag lock: distance from the anchor at the press
+    bool onVrSettings = false;       // the cursor is on the SteamVR Settings page (kept while dragging)
+    bool catcherHidesHit = false;    // the laser-catching dot hides SteamVR's hit dot (Settings page)
+    Clock::time_point dropHoldUntil{};  // after a left release: keep the drag pose this long
+    bool debug = false;
+    std::string lastHit;
+    // The ft-screens panel the left button was pressed on, and where it was then (see the top).
+    std::string pressKey;
+    vr::HmdMatrix34_t pressPose{};
+    Clock::time_point lastDebug{};  // set in Init
+    vr::VROverlayHandle_t systemPointer = vr::k_ulOverlayHandleInvalid;  // found in Init
+    Vec3 pivot, tiltOrigin, lastPoint, lastOrigin, lastAim{0, 0, -1};
+    Basis tiltBasis{};
+    bool headsetOff = false;  // nobody is wearing the headset (see the main loop)
+    Vec3 anchor;
+    double yaw = 0, pitch = 0;
+    Vec3 followRef{0, 0, -1};  // head follow's reference direction (see the top)
+    Clock::time_point followAt{};  // its last update, for the easing (set in Init)
+    bool following = false;                           // past the leash: easing toward the head
+    double followLag = 0;                             // radians the reference trails the head
+    std::chrono::steady_clock::time_point leashOutSince{};  // head past the leash since (delay)
+    // Gaze mode (see the top).
+    struct Gaze {
+        double hy = 0, hp = 0, rhy = 0, rhp = 0;  // corrected, and raw
+        Clock::time_point at{};
+    } gz;
+    bool gazeOwns = true;  // the pointer follows the gaze; false: the mouse has it
+    bool nudging = false;  // the mouse took it from the gaze: the next click may be a lesson
+    double nudgeRawHy = 0, nudgeRawHp = 0, nudgeMoved = 0;
+    vr::HmdMatrix34_t nudgeHead{}, lastHead{};
+    bool haveHead = false, havePoint = false;
+    Clock::time_point nudgeAt{}, retakeSince{};
+    // A held-back press (see the top): aimHeld while the button is down; then the click
+    // (clickPress at the end of the next frame, clickRelease 40 ms later). gazeBack: give
+    // the gaze the pointer again when the press or click is over.
+    vr::TrackedDeviceIndex_t ours = vr::k_unTrackedDeviceIndexInvalid;
+    bool aimHeld = false, clickPress = false, clickRelease = false, gazeBack = false;
+    bool aimHand = false;  // the held-back press is a hold's (below): it never turns into a real press
+    Clock::time_point aimSince{}, clickReleaseAt{};
+    // Holds (see "Gaze precision" and "Hands" at the top): a pinch or grip, or a gaze
+    // precision or gaze drag button, held now. What steers the pointer meanwhile (a hand, or
+    // the mouse through its own moves), from where it pointed when the hold began (for a
+    // hand: seen from the eye then, in the room), and the pointer then.
+    HandGestures handFile;
+    PoseHistory poses;
+    enum class Src { None, Hand, Mouse, Head };
+    struct Hold {
+        Src src = Src::None;
+        int side = -1;          // a hand's side (0 left, 1 right)
+        bool grip = false;      // pressed at once and dragging (a grip, gaze drag), not a click on release
+        bool engaged = false;   // past the dead zone
+        bool pressed = false;   // a real press went out (a grip or gaze drag, or a pinch without gaze mode)
+        bool promote = false;   // held still for POINTER_GAZE_HOLD, it becomes a real press (keyboard clicks)
+        bool right = false;     // the right button (gaze_right)
+        bool keyDrag = false;   // gaze_right pressed while gaze_left aimed: a left press, either key ends it
+        double pressYaw = 0, pressPitch = 0;  // the pointer at the press (a quick tap clicks there)
+        Vec3 origin;
+        double refYaw = 0, refPitch = 0, startYaw = 0, startPitch = 0, lastYaw = 0, lastPitch = 0;
+    } hold;
+    // "precision|gazedrag <source> 1|0" from the relay, done in the frame (see Holds).
+    struct DevicePress {
+        std::string source;
+        bool drag, down;
+    };
+    std::vector<DevicePress> devicePresses;
+    // "gazekey left|right 1|0" from the relay (keyboard clicks), done in the frame.
+    struct KeyPress {
+        bool right, down;
+    };
+    std::vector<KeyPress> keyPresses;
+    uint32_t seenBegins[2][2] = {}, seenEnds[2][2] = {};  // [pinch, grip][side], as last read
+    bool handBaseline = false;
+    int handOpens = 0;
+    uint64_t handSeq = 0, handPublished = 0;
+    Clock::time_point handUsed{};  // a gesture began then (keeps the pointer, like gaze mode)
+    Clock::time_point lastTyping{};  // the relay's last "typing": a key on a keyboard
+    // Gaze mode outside games, and its dot (see the top): lastMove/lastHeld/pulseAt.
+    bool inGame = false, gazeAwake = false;
+    Clock::time_point inGameAt{}, gazeAwakeAt{};
+    Clock::time_point lastMove{}, lastHeld{}, pulseAt{};
+    // The left button, as sent to the driver; pressRight: the next press is the right button
+    // instead (gaze_right), heldButton: the one pressed.
+    bool pressRight = false;
+    std::string heldButton = "trigger";
+    bool confirmLesson = false;  // a keyboard click's quick tap: a lesson with no correction (see the top)
+    // The gaze calibration panel is up until then ("calpanel 1"; see the top); calOpened: it just
+    // came up, so a press in progress ends without a click.
+    Clock::time_point calPanelUntil{};
+    bool calOpened = false;
+    Clock::time_point lastSlow{};  // set in Init
+
+    void ApplyConfig();
+    void Init();
+    void Wake(Clock::time_point t);
+    void PressLeft();
+    void ReleaseLeft();
+    bool CanAim();
+    void AimStart(bool right);
+    void ChordDrop();
+    void LeftButton(bool down);
+    bool RightButton(bool down);
+    bool MouseMoveHeld();
+    void SendPose(Vec3 originStanding, const Basis &b);
+    std::pair<bool, Vec3> HeadPos();
+    void Borrow();
+    void GiveBack();
+    std::string FindPanel(const char *key, Panel &p, Vec3 &eye);
+    void GrabProbe(const char *key);
+    std::string Place(const char *key, Vec3 target, const Basis &bt, double grabBelow);
+    // Nearest's answer: the nearest overlay along a ray.
+        struct Hit {
+            double along = 1e9;
+            std::string key;
+            bool scene = false;
+            Vec3 point, normal;
+        };
+    // Where the cursor is this frame (FindCursor), for DrawDot and SendRay.
+    struct Spot {
+        bool dragging = false;
+        Vec3 dir, point;
+        double best = 1e9, distance = 0;
+        bool onEdge = false, occluded = false, onScene = false, onPanel = false;
+    };
+
+    // The main loop's sections, in the order main calls them, and what they share.
+    void HeadsetOff();
+    void GazeAwake();
+    void RelayCommands();
+    void Command(const char *buf, const sockaddr_un &sender, socklen_t senderLen);
+    void ReadFrame(Frame &frame);
+    void ClaimPulse(const Frame &frame);
+    void PauseOverlayList();
+    void LaserMode();
+    void HandRole(const Frame &frame);
+    void LastUsedWins(const Frame &frame);
+    void Recenter(const Frame &frame);
+    void GazeMode(const Frame &frame);
+    void Hands(const Frame &frame);
+    bool HandAngles(const float p[3], uint64_t t_ns, const Vec3 &from, double &hy, double &hp);
+    void EndHold(bool lost);
+    void StartHold(bool press, Clock::time_point tnow);
+    void Steer(double hy, double hp, double deadzone, double gain, Clock::time_point tnow);
+    void BeginHold(int side, bool grip, const fh_pinch_t &g, Clock::time_point tnow);
+    void GazePrecisionButtons(const Frame &frame);
+    bool HeadAngles(const vr::TrackedDevicePose_t &hmd, double &hy, double &hp);
+    void KeyboardClicks(const Frame &frame);
+    void CalPanelOpened();
+    void HeadSteer(const Frame &frame);
+    void PinchesAndGrips(Frame &frame);
+    void HandsStopped(const Frame &frame);
+    void MarkHeld(const Frame &frame);
+    void HeadFollow(const Frame &frame);
+    void SlowWork();
+    void Cursor(const Frame &frame);
+    void Tilt(const Frame &frame);
+    Hit Nearest(Vec3 from, Vec3 d);
+    Spot FindCursor(const Frame &frame);
+    void DrawDot(const Frame &frame, const Spot &spot);
+    void SendRay(const Frame &frame, const Spot &spot);
+    void HeldBackPress(const Frame &frame);
+    void KeyboardClickHold(const Frame &frame);
+    void Click(const Frame &frame);
+    void PollControllerButtons();
+    bool SteamVRQuit();
+};

pointer/helper/pointer_config.h

@@ -0,0 +1,114 @@
+// ft-pointer's settings, from ~/.config/frametop.conf. "The top" in comments here is the
+// header comment of ft-pointer.cpp, which describes every setting.
+#pragma once
+
+#include <algorithm>
+#include <cstdlib>
+#include <fstream>
+#include <map>
+#include <string>
+#include <vector>
+
+inline std::map<std::string, std::string> ReadConfig() {
+    std::map<std::string, std::string> conf;
+    const char *home = std::getenv("HOME");
+    std::ifstream in(std::string(home ? home : "") + "/.config/frametop.conf");
+    std::string line;
+    while (std::getline(in, line)) {
+        line = line.substr(0, line.find('#'));
+        const auto eq = line.find('=');
+        if (eq == std::string::npos) continue;
+        auto trim = [](std::string s) {
+            s.erase(0, s.find_first_not_of(" \t"));
+            s.erase(s.find_last_not_of(" \t") + 1);
+            return s;
+        };
+        conf[trim(line.substr(0, eq))] = trim(line.substr(eq + 1));
+    }
+    return conf;
+}
+
+inline double ConfDouble(const std::map<std::string, std::string> &c, const char *key, double fallback) {
+    auto it = c.find(key);
+    return it == c.end() ? fallback : std::atof(it->second.c_str());
+}
+
+// POINTER_IGNORE: overlay keys the pointer passes through, as if they weren't there
+// (display-only panels such as a performance overlay). Comma-separated shell patterns
+// (fnmatch, no escapes), so "vendor.app*" covers an app's overlays.
+inline std::vector<std::string> ParseIgnore(const std::string &list) {
+    std::vector<std::string> out;
+    size_t at = 0;
+    while (at <= list.size()) {
+        const size_t comma = std::min(list.find(',', at), list.size());
+        std::string item = list.substr(at, comma - at);
+        item.erase(0, item.find_first_not_of(" \t"));
+        item.erase(item.find_last_not_of(" \t") + 1);
+        if (!item.empty()) out.push_back(item);
+        at = comma + 1;
+    }
+    return out;
+}
+
+struct PointerConfig {
+    double freeDistance = 1.5, cursorDeg = 0.4, originFraction = 0.95, originMargin = 0.15, sceneRadius = 0.5,
+           edgeReach = 0.3, grabOffset = 0.075,
+           slideSpeed = 0.5, leashDeg = 10, leashReturn = 0.2, leashDelay = 0.2, followReach = 70;
+    // POINTER_FOLLOW and POINTER_GAZE as read. They turn head follow and gaze mode on or off
+    // only when they change, so a "follow" or "gaze" command wins until then (ApplyConfig in
+    // ft-pointer.cpp).
+    bool follow = false, gaze = false;
+    double gazeRetake = 5, gazeNudgeMax = 55, gazeHold = 0.5, gazeShow = 1;
+    bool gazeDotAlways = true;  // POINTER_GAZE_DOT (see the top)
+    double headDeadzone = 0.5, keyTap = 0.25;  // POINTER_HEAD_DEADZONE, POINTER_KEY_TAP (keyboard clicks, see the top)
+    // Hands (see the top): POINTER_HANDS, POINTER_PINCH_GAIN, POINTER_PINCH_DEADZONE, POINTER_GRIP_GAIN.
+    bool handsOn = false;
+    double pinchGain = 0.5, pinchDeadzone = 1.5, gripGain = 1.0, handBelow = 0.35, typingHold = 1.0;
+    // Gaze precision (see the top): POINTER_GAZE_MOUSE (precision: the mouse's left button
+    // holds back its press in gaze mode; direct: it clicks at once), POINTER_ROLE.
+    bool gazeMousePrecision = true;
+    bool gazeMouseHeld = true;  // POINTER_GAZE_MOUSE_MOVE (see the top)
+    std::string role = "right";
+    double pickupScale = 1;  // POINTER_CONTROLLER_PICKUP: scales the controller-moved limits
+    std::vector<std::string> ignore;  // POINTER_IGNORE (see ParseIgnore)
+
+    void Load(const std::map<std::string, std::string> &conf) {
+        freeDistance = std::clamp(ConfDouble(conf, "POINTER_DISTANCE", 1.5), 0.3, 10.0);
+        cursorDeg = std::clamp(ConfDouble(conf, "POINTER_CURSOR_DEG", 0.4), 0.05, 5.0);
+        originFraction = std::clamp(ConfDouble(conf, "POINTER_ORIGIN_FRACTION", 0.95), 0.0, 0.98);
+        originMargin = std::clamp(ConfDouble(conf, "POINTER_ORIGIN_MARGIN", 0.15), 0.0, 1.0);
+        sceneRadius = std::clamp(ConfDouble(conf, "POINTER_SCENE_RADIUS", 0.5), 0.05, 2.0);
+        edgeReach = std::clamp(ConfDouble(conf, "POINTER_EDGE_REACH", 0.3), 0.0, 2.0);
+        grabOffset = std::clamp(ConfDouble(conf, "LAYOUT_GRAB_OFFSET", 0.075), 0.0, 1.0);
+        slideSpeed = std::clamp(ConfDouble(conf, "LAYOUT_SLIDE_SPEED", 0.5), 0.02, 2.0);
+        leashDeg = std::clamp(ConfDouble(conf, "POINTER_LEASH_DEG", 10), 0.0, 90.0);
+        leashReturn = std::clamp(ConfDouble(conf, "POINTER_LEASH_RETURN", 0.2), 0.0, 5.0);
+        leashDelay = std::clamp(ConfDouble(conf, "POINTER_LEASH_DELAY", 0.2), 0.0, 5.0);
+        followReach = std::clamp(ConfDouble(conf, "POINTER_FOLLOW_REACH", 70), 10.0, 89.0);
+        follow = ConfDouble(conf, "POINTER_FOLLOW", 0) != 0;
+        gazeRetake = std::clamp(ConfDouble(conf, "POINTER_GAZE_RETAKE", 5), 1.0, 45.0);
+        gazeNudgeMax = std::clamp(ConfDouble(conf, "POINTER_GAZE_NUDGE_MAX", 55), 1.0, 110.0);
+        gazeHold = std::clamp(ConfDouble(conf, "POINTER_GAZE_HOLD", 0.5), 0.1, 5.0);
+        gazeShow = std::clamp(ConfDouble(conf, "POINTER_GAZE_SHOW", 1), 0.0, 30.0);
+        headDeadzone = std::clamp(ConfDouble(conf, "POINTER_HEAD_DEADZONE", 0.5), 0.0, 5.0);
+        keyTap = std::clamp(ConfDouble(conf, "POINTER_KEY_TAP", 0.25), 0.0, 1.0);
+        const auto gd = conf.find("POINTER_GAZE_DOT");
+        gazeDotAlways = gd == conf.end() || gd->second != "moving";
+        gaze = ConfDouble(conf, "POINTER_GAZE", 0) != 0;
+        handsOn = ConfDouble(conf, "POINTER_HANDS", 0) != 0;
+        pinchGain = std::clamp(ConfDouble(conf, "POINTER_PINCH_GAIN", 0.5), 0.05, 3.0);
+        pinchDeadzone = std::clamp(ConfDouble(conf, "POINTER_PINCH_DEADZONE", 1.5), 0.0, 10.0);
+        gripGain = std::clamp(ConfDouble(conf, "POINTER_GRIP_GAIN", 1.0), 0.05, 3.0);
+        handBelow = std::clamp(ConfDouble(conf, "POINTER_GRIP_BELOW", 0.35), 0.05, 1.0);
+        typingHold = std::clamp(ConfDouble(conf, "POINTER_PINCH_TYPING", 1.0), 0.0, 5.0);
+        const auto gm = conf.find("POINTER_GAZE_MOUSE");
+        gazeMousePrecision = gm == conf.end() || gm->second != "direct";
+        const auto mm = conf.find("POINTER_GAZE_MOUSE_MOVE");
+        gazeMouseHeld = mm == conf.end() || mm->second != "free";
+        const auto ro = conf.find("POINTER_ROLE");
+        role = ro != conf.end() && (ro->second == "left" || ro->second == "stylus") ? ro->second : "right";
+        pickupScale = std::clamp(ConfDouble(conf, "POINTER_CONTROLLER_PICKUP", 1), 0.5, 5.0);
+        const auto ig = conf.find("POINTER_IGNORE");
+        ignore = ParseIgnore(ig == conf.end() ? "" : ig->second);
+    }
+};

pointer/helper/pointer_cursor.cpp

@@ -0,0 +1,474 @@
+// ft-pointer: recenter, head follow, the overlay handles, and the cursor (see "Cursor model" at
+// the top).
+// "The top" in comments here is the header comment of ft-pointer.cpp.
+#include "pointer.h"
+
+namespace {
+
+bool Ignored(const std::vector<std::string> &patterns, const std::string &key) {
+    for (const auto &p : patterns)
+        if (fnmatch(p.c_str(), key.c_str(), FNM_NOESCAPE) == 0) return true;
+    return false;
+}
+
+vr::HmdMatrix34_t Billboard(Vec3 at, Vec3 eye) {
+    // Overlay faces +Z; point +Z at the eye, keep +Y roughly up.
+    const Vec3 z = Normalize(eye - at);
+    const Vec3 x = Normalize(Cross({0, 1, 0}, z));
+    const Vec3 y = Cross(z, x);
+    vr::HmdMatrix34_t m{};
+    const Vec3 cols[3] = {x, y, z};
+    for (int c = 0; c < 3; ++c) {
+        m.m[0][c] = float(cols[c].x);
+        m.m[1][c] = float(cols[c].y);
+        m.m[2][c] = float(cols[c].z);
+    }
+    m.m[0][3] = float(at.x);
+    m.m[1][3] = float(at.y);
+    m.m[2][3] = float(at.z);
+    return m;
+}
+
+// Unit vector v, turned toward unit vector `center` until it's at most maxRad from it.
+Vec3 PullWithin(Vec3 v, Vec3 center, double maxRad) {
+    if (std::acos(std::clamp(Dot(v, center), -1.0, 1.0)) <= maxRad) return v;
+    Vec3 axis = Cross(center, v);
+    if (Length(axis) < 1e-9) axis = Cross(center, {0, 1, 0});  // opposite: any perpendicular
+    return RotateAbout(center, Normalize(axis), maxRad);
+}
+
+// Unit direction with its pitch limited to +-maxDeg (AimBasis needs it off vertical).
+Vec3 LimitPitch(Vec3 d, double maxDeg) {
+    const double pitch = std::clamp(std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI, -maxDeg, maxDeg);
+    return Direction(std::atan2(-d.x, -d.z) * 180 / M_PI, pitch);
+}
+
+// SteamVR Settings page (see the top): the laser starts this far from the eye, the dot is
+// drawn this far out, and the laser-catching dot sits this far out (metres).
+constexpr double SETTINGS_ORIGIN = 0.25, SETTINGS_DOT = 0.6, SETTINGS_CATCHER = 8.0;
+
+// Whether the line of sight from `eye` along `d` crosses the SteamVR Settings page, drawn by
+// the dashboard's scene-graph panel (valve.steam.gamepadui.frame.menu.N, transform t); see
+// "SteamVR Settings" at the top. The page has no size in OpenVR, so this is its area as
+// measured on the Frame, generously: in metres from the panel's origin, which is near the
+// page's left edge, it ran from about -0.35 (the sidebar) to 1.15 across and +-0.4 up and
+// down, with the transform scaled 0.369. Kept in the transform's units so it scales with it.
+bool OnSettingsPage(const vr::HmdMatrix34_t &t, Vec3 eye, Vec3 d) {
+    const Vec3 c = Position(t), x{t.m[0][0], t.m[1][0], t.m[2][0]}, y{t.m[0][1], t.m[1][1], t.m[2][1]},
+               z{t.m[0][2], t.m[1][2], t.m[2][2]};
+    const double sx = Dot(x, x), sy = Dot(y, y), denom = Dot(d, z);
+    if (sx < 1e-9 || sy < 1e-9 || std::fabs(denom) < 1e-6) return false;
+    const double along = Dot(c - eye, z) / denom;
+    if (along <= 0) return false;
+    const Vec3 off = eye + d * along - c;
+    const double u = Dot(off, x) / sx, v = Dot(off, y) / sy;  // in the transform's units
+    return u >= -1.35 && u <= 3.4 && std::fabs(v) <= 1.25;
+}
+
+}  // namespace
+
+// A pending recenter (the command, a wake, a first move): the anchor goes to the eye, aimed where the
+// head faces.
+void Pointer::Recenter(const Frame &frame) {
+    const vr::TrackedDevicePose_t &hmd = frame.Hmd();
+    const auto &hm = hmd.mDeviceToAbsoluteTracking.m;
+    const Vec3 &eye = frame.eye;
+    if (recenter && hmd.bPoseIsValid) {
+        anchor = eye;
+        const Vec3 f{-hm[0][2], -hm[1][2], -hm[2][2]};
+        yaw = std::atan2(-f.x, -f.z) * 180 / M_PI;
+        pitch = std::asin(std::clamp(f.y, -1.0, 1.0)) * 180 / M_PI;
+        anchored = true;
+        recenter = false;
+        followReset = true;
+    }
+}
+
+// Head follow (see the top): past the leash (for the delay), ease the reference to the
+// head's facing, and turn the cursor with it. Not in gaze mode: the gaze places it.
+void Pointer::HeadFollow(const Frame &frame) {
+    const vr::TrackedDevicePose_t &hmd = frame.Hmd();
+    const auto &hm = hmd.mDeviceToAbsoluteTracking.m;
+    const auto tnow = frame.tnow;
+    const Vec3 &eye = frame.eye;
+    if (follow && !gazeOn && active && anchored && hmd.bPoseIsValid) {
+        const Vec3 head = LimitPitch(Vec3{-hm[0][2], -hm[1][2], -hm[2][2]}, 85);
+        if (followReset) {
+            followRef = head, followLag = 0, leashOutSince = {};
+            followReset = following = false;
+        }
+        const double dt = std::min(0.1, std::chrono::duration<double>(tnow - followAt).count());
+        const double leash = cfg.leashDeg * M_PI / 180;
+        const double lag = std::acos(std::clamp(Dot(followRef, head), -1.0, 1.0));
+        double keep = lag;  // how far the reference stays behind the head after this frame
+        if (leash <= 0) {
+            keep = 0;  // head-locked
+        } else if (following) {
+            keep = lag * (cfg.leashReturn > 0 ? std::exp(-dt / cfg.leashReturn) : 0.0);
+            // A fast turn drags it at the leash's end; past it already (after the delay), it
+            // can't fall further behind, and it closes in from there without a jump.
+            keep = std::min(keep, std::max(leash, followLag));
+            if (keep < 0.05 * M_PI / 180) keep = 0, following = false;  // landed on the facing
+        } else if (lag > leash) {
+            if (leashOutSince == decltype(leashOutSince){}) leashOutSince = tnow;
+            if (std::chrono::duration<double>(tnow - leashOutSince).count() >= cfg.leashDelay)
+                following = true, leashOutSince = {};
+        } else {
+            leashOutSince = {};  // back inside before the delay: a glance
+        }
+        followLag = keep;
+        const Vec3 ref = LimitPitch(PullWithin(followRef, head, keep), 85);
+        if (!leftHeld && tnow >= dropHoldUntil) {
+            // The cursor keeps its offset from the reference (frames without roll).
+            Vec3 d = FromBasis(AimBasis(ref), ToBasis(AimBasis(followRef), Direction(yaw, pitch)));
+            d = PullWithin(Normalize(d), ref, cfg.followReach * M_PI / 180);
+            yaw = std::atan2(-d.x, -d.z) * 180 / M_PI;
+            pitch = std::clamp(std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI, -85.0, 85.0);
+            // The ray origin closes in on the eye as the reference does on the facing.
+            anchor = eye + (anchor - eye) * (leash <= 0 ? 0.0 : lag > 1e-6 ? keep / lag : following ? 0.0 : 1.0);
+        }
+        followRef = ref;
+    }
+    followAt = tnow;
+}
+
+// Slow work, once a second: overlay handles, our device index, laser width.
+void Pointer::SlowWork() {
+    const auto now = std::chrono::steady_clock::now();
+    if (now - lastSlow > std::chrono::seconds(1)) {
+        lastSlow = now;
+        handles.clear();
+        for (const auto &key : overlays.Keys()) {
+            if (Ignored(cfg.ignore, key)) continue;
+            vr::VROverlayHandle_t h;
+            if (overlay->FindOverlay(key.c_str(), &h) != vr::VROverlayError_None) continue;
+            handles[key] = h;
+            // Scene-graph overlays are placed absolutely. Other overlays can report no
+            // texture too (gamescope's app panels, placed as dashboard tabs, share theirs
+            // from another process), and ComputeOverlayIntersection handles those.
+            uint32_t tw = 0, th = 0;
+            overlay->GetOverlayTextureSize(h, &tw, &th);
+            vr::VROverlayTransformType tt = vr::VROverlayTransform_Invalid;
+            overlay->GetOverlayTransformType(h, &tt);
+            // ft-screens' panels (frametop.screen.N, floating windows with their popups,
+            // frametop.float.N..., the keyboard) are 0x0 and absolute too (a shared
+            // texture), but they're real panels of any size.
+            sceneGraph[key] = (tw == 0 || th == 0) && tt == vr::VROverlayTransform_Absolute &&
+                              key.rfind("frametop.", 0) != 0;
+        }
+        ours = vr::k_unTrackedDeviceIndexInvalid;
+        for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) {
+            char type[64] = "";
+            sys->GetStringTrackedDeviceProperty(i, vr::Prop_ControllerType_String, type, sizeof type);
+            if (std::strcmp(type, "ft_pointer") == 0) ours = i;
+        }
+
+    }
+
+    if (now - lastVisible > std::chrono::milliseconds(50) || visible.size() != handles.size()) {
+        lastVisible = now;
+        visible.clear();
+        for (const auto &[key, h] : handles) visible[key] = overlay->IsOverlayVisible(h);
+    }
+}
+
+// The cursor: tilting a grabbed panel, or pointing (find the spot, draw the dot, send the ray).
+void Pointer::Cursor(const Frame &frame) {
+    const vr::TrackedDevicePose_t &hmd = frame.Hmd();
+    if (active && anchored && hmd.bPoseIsValid && tilting) {
+        Tilt(frame);
+    } else if (active && anchored && hmd.bPoseIsValid) {
+        const Spot spot = FindCursor(frame);
+        DrawDot(frame, spot);
+        SendRay(frame, spot);
+    }
+}
+
+void Pointer::Tilt(const Frame &frame) {
+    const vr::TrackedDevicePose_t &hmd = frame.Hmd();
+    const Vec3 &eye = frame.eye;
+    const vr::TrackedDevicePose_t &hmdRaw = frame.hmdRaw;
+    // Rotate the device around the grab point; the grabbed panel turns with it.
+    if (tiltStart) {
+        pivot = lastPoint;
+        tiltOrigin = lastOrigin;
+        tiltBasis = AimBasis(lastAim);
+        tiltStart = false;  // angles carry on from any earlier tilt in this drag
+        overlay->HideOverlay(cursor);
+        overlay->HideOverlay(marker);
+    }
+    const double yr = tiltYaw * M_PI / 180, pr = tiltPitch * M_PI / 180;
+    const Vec3 up{0, 1, 0}, side = tiltBasis.x;
+    auto turn = [&](Vec3 v) { return RotateAbout(RotateAbout(v, side, pr), up, yr); };
+    const Vec3 originStanding = pivot + turn(tiltOrigin - pivot);
+    const Basis b{turn(tiltBasis.x), turn(tiltBasis.y), turn(tiltBasis.z)};
+    const auto &S = hmd.mDeviceToAbsoluteTracking, &R = hmdRaw.mDeviceToAbsoluteTracking;
+    auto toRaw = [&](Vec3 v) { return Rotate(R, RotateInverse(S, v)); };  // directions: raw <- standing
+    const Vec3 originRaw = Position(R) + toRaw(originStanding - eye);
+    double q[4];
+    BasisQuat({toRaw(b.x), toRaw(b.y), toRaw(b.z)}, q);
+    char msg[200];
+    std::snprintf(msg, sizeof msg, "posq %.5f %.5f %.5f %.6f %.6f %.6f %.6f", originRaw.x, originRaw.y,
+                  originRaw.z, q[0], q[1], q[2], q[3]);
+    SendTo(out, "ft_pointer", msg);
+}
+
+// Nearest visible overlay along a ray (frozen while dragging).
+Pointer::Hit Pointer::Nearest(Vec3 from, Vec3 d) {
+        Hit h;
+        for (const auto &[key, handle] : handles) {
+            if (!visible[key]) continue;
+            if (sceneGraph[key]) {
+                // Plane test: overlay origin and its +Z normal, within sceneRadius of the origin.
+                vr::ETrackingUniverseOrigin uo;
+                vr::HmdMatrix34_t t{};
+                if (overlay->GetOverlayTransformAbsolute(handle, &uo, &t) != vr::VROverlayError_None) continue;
+                const Vec3 center = Position(t), normal{t.m[0][2], t.m[1][2], t.m[2][2]};
+                const double denom = Dot(d, normal);
+                if (std::fabs(denom) < 1e-4) continue;
+                const double along = Dot(center - from, normal) / denom;
+                const Vec3 at = from + d * along;
+                if (along > 0.05 && along < h.along && std::sqrt(Dot(at - center, at - center)) <= cfg.sceneRadius)
+                    h.along = along, h.key = key, h.scene = true, h.point = at, h.normal = normal;
+                continue;
+            }
+            vr::VROverlayIntersectionParams_t params{};
+            params.vSource = {float(from.x), float(from.y), float(from.z)};
+            params.vDirection = {float(d.x), float(d.y), float(d.z)};
+            params.eOrigin = vr::TrackingUniverseStanding;
+            vr::VROverlayIntersectionResults_t r{};
+            if (overlay->ComputeOverlayIntersection(handle, &params, &r) && r.fDistance > 0.05f &&
+                r.fDistance < h.along) {
+                h.along = r.fDistance, h.key = key, h.scene = false;
+                h.point = {r.vPoint.v[0], r.vPoint.v[1], r.vPoint.v[2]};
+                h.normal = {r.vNormal.v[0], r.vNormal.v[1], r.vNormal.v[2]};
+            }
+        }
+        return h;
+}
+
+// Where the cursor is this frame: on a panel, a scene-graph plane, or a panel's edge, or out in free
+// space.
+Pointer::Spot Pointer::FindCursor(const Frame &frame) {
+    const auto tnow = frame.tnow;
+    const Vec3 &eye = frame.eye;
+    const bool dragging = leftHeld || tnow < dropHoldUntil;
+    if (!dragging) tiltYaw = tiltPitch = 0;  // drop finished: back to plain pointing
+    const Vec3 dir = Direction(yaw, pitch);
+    Hit first;
+    if (!dragging) first = Nearest(anchor, dir);
+    double best = first.along;
+    std::string bestKey = first.key;
+    bool bestScene = first.scene;
+    Vec3 bestPoint = first.point, bestNormal = first.normal;
+    bool onEdge = false;
+    if (!dragging && best < 1e8 && !bestScene) {
+        edgeKey = bestKey, edgePoint = bestPoint, edgeNormal = Normalize(bestNormal), edgeLast = bestPoint;
+    } else if (!dragging && best >= 1e8 && !edgeKey.empty() && visible[edgeKey]) {
+        // Just off a panel: stay on its plane (see "Panel edges" at the top).
+        const double denom = Dot(dir, edgeNormal);
+        if (std::fabs(denom) > 1e-4) {
+            const double along = Dot(edgePoint - anchor, edgeNormal) / denom;
+            const Vec3 at = anchor + dir * along;
+            if (along > 0.05 && std::sqrt(Dot(at - edgeLast, at - edgeLast)) <= cfg.edgeReach) {
+                best = along;
+                bestKey = edgeKey;
+                onEdge = true;
+            }
+        }
+    }
+    // Held on one of ft-screens' panels that stays where it is: onto whichever of them
+    // the ray meets (see the top).
+    if (leftHeld && !pressKey.empty()) {
+        vr::ETrackingUniverseOrigin uo;
+        vr::HmdMatrix34_t now{};
+        auto it = handles.find(pressKey);
+        bool still = it != handles.end() &&
+                     overlay->GetOverlayTransformAbsolute(it->second, &uo, &now) == vr::VROverlayError_None;
+        for (int i = 0; still && i < 3; ++i)
+            for (int j = 0; j < 4; ++j)
+                if (std::fabs(now.m[i][j] - pressPose.m[i][j]) > 0.001f) still = false;
+        if (still) {
+            Hit h;
+            for (const auto &[key, handle] : handles) {
+                if (!visible[key] || !FramePanel(key)) continue;
+                vr::VROverlayIntersectionParams_t params{};
+                params.vSource = {float(anchor.x), float(anchor.y), float(anchor.z)};
+                params.vDirection = {float(dir.x), float(dir.y), float(dir.z)};
+                params.eOrigin = vr::TrackingUniverseStanding;
+                vr::VROverlayIntersectionResults_t r{};
+                if (overlay->ComputeOverlayIntersection(handle, &params, &r) && r.fDistance > 0.05f &&
+                    r.fDistance < h.along)
+                    h.along = r.fDistance, h.key = key;
+            }
+            if (h.along < 1e8) dragDistance = h.along, lastHit = h.key;
+        } else {
+            pressKey.clear();  // carried: the lock holds for the rest of this press
+        }
+    }
+    // While dragging: keep the press-time distance and show the non-interactive marker.
+    // On a scene-graph plane or a panel's edge: the laser-catching dot goes 5 cm behind it.
+    double distance = dragging ? dragDistance : (best < 1e8 ? best : cfg.freeDistance);
+    Vec3 point = anchor + dir * distance;
+    bool occluded = false;
+    if (!dragging) {
+        // The cursor lands on what you see under it: the ray above starts at the anchor,
+        // not the eye, so after leaning it can pick a panel that something nearer
+        // covers from where you are now (panels close together in view, at different
+        // depths). Anything in front of the point on the eye's line of sight wins.
+        const double toPoint = std::sqrt(Dot(point - eye, point - eye));
+        const Hit front = Nearest(eye, Normalize(point - eye));
+        if (front.along < toPoint - 0.02) {
+            occluded = true;
+            bestKey = front.key, bestScene = front.scene;
+            point = front.point;
+            if (!front.scene) edgeKey = front.key, edgePoint = front.point, edgeNormal = Normalize(front.normal),
+                              edgeLast = front.point;
+            distance = std::sqrt(Dot(point - anchor, point - anchor));
+            best = distance;
+            onEdge = false;
+        }
+        lastDistance = distance, lastHit = bestKey;
+    }
+    const bool onScene = !dragging && ((bestScene && best < 1e8) || onEdge);
+    const bool onPanel = dragging || (best < 1e8 && !onScene);
+    Spot s;
+    s.dragging = dragging, s.dir = dir, s.point = point, s.best = best, s.distance = distance;
+    s.onEdge = onEdge, s.occluded = occluded, s.onScene = onScene, s.onPanel = onPanel;
+    return s;
+}
+
+// Our dot where the cursor is (see "Looks" and "SteamVR Settings" at the top).
+void Pointer::DrawDot(const Frame &frame, const Spot &spot) {
+    const auto tnow = frame.tnow;
+    const Vec3 &eye = frame.eye;
+    const bool dragging = spot.dragging, onScene = spot.onScene, onPanel = spot.onPanel;
+    const Vec3 dir = spot.dir, point = spot.point;
+    const Vec3 sight = Normalize(point - eye);
+    if (!dragging) {
+        // SteamVR Settings (see the top): the dashboard's main panel is hidden and its
+        // scene-graph panel shows the page.
+        onVrSettings = false;
+        const auto mainIt = visible.find("valve.steam.gamepadui.main");
+        if (overlay->IsDashboardVisible() && !(mainIt != visible.end() && mainIt->second)) {
+            for (const auto &[key, handle] : handles) {
+                if (!visible[key] || key.rfind("valve.steam.gamepadui.frame.menu.", 0) != 0) continue;
+                vr::ETrackingUniverseOrigin uo;
+                vr::HmdMatrix34_t t{};
+                if (overlay->GetOverlayTransformAbsolute(handle, &uo, &t) == vr::VROverlayError_None &&
+                    OnSettingsPage(t, eye, sight))
+                    onVrSettings = true;
+            }
+        }
+    }
+    if (onVrSettings != catcherHidesHit) {
+        overlay->SetOverlayFlag(cursor, vr::VROverlayFlags_HideLaserIntersection, onVrSettings);
+        catcherHidesHit = onVrSettings;
+    }
+
+    if (onVrSettings) {
+        // The dot close in front of the page, which is nearer than any guess of ours;
+        // the laser-catching dot far behind everything, invisible, so it never covers
+        // the page, with SteamVR's hit dot hidden on it.
+        const Vec3 near = eye + sight * SETTINGS_DOT, far = eye + sight * SETTINGS_CATCHER;
+        double alpha = 1;
+        if (gazeOn && !cfg.gazeDotAlways) {
+            auto secs = [&](Clock::time_point t) { return std::chrono::duration<double>(tnow - t).count(); };
+            alpha = std::clamp(1 - std::min(secs(lastMove) - cfg.gazeShow, secs(lastHeld)) / 0.25, 0.0, 1.0);
+        }
+        if (tnow < calPanelUntil) alpha = 0;
+        overlay->SetOverlayAlpha(marker, float(alpha));
+        overlay->SetOverlayWidthInMeters(marker, float(2 * SETTINGS_DOT * std::tan(cfg.cursorDeg * M_PI / 360)));
+        auto mm = Billboard(near, eye);
+        overlay->SetOverlayTransformAbsolute(marker, vr::TrackingUniverseStanding, &mm);
+        overlay->SetOverlayAlpha(cursor, 0);
+        overlay->SetOverlayWidthInMeters(cursor, float(2 * SETTINGS_CATCHER * std::tan(cfg.cursorDeg * M_PI / 360)));
+        auto mc = Billboard(far, eye);
+        overlay->SetOverlayTransformAbsolute(cursor, vr::TrackingUniverseStanding, &mc);
+        overlay->ShowOverlay(marker);
+        overlay->ShowOverlay(cursor);
+    } else {
+        // On a panel: the non-interactive marker, pulled 5 mm toward the eye so it
+        // draws on top. In free space: the interactive dot the laser lands on.
+        const vr::VROverlayHandle_t show = onPanel ? marker : cursor, hide = onPanel ? cursor : marker;
+        const Vec3 at = onPanel ? point + Normalize(eye - point) * 0.005 : onScene ? point + dir * 0.05 : point;
+        const double dist = std::sqrt(Dot(at - eye, at - eye));
+        // Gaze mode: a pulse for each click, and with POINTER_GAZE_DOT=moving, shown only
+        // while something moves it or a press holds it (see the top); transparent
+        // otherwise, the laser still lands on it.
+        double scale = 1, alpha = 1;
+        if (gazeOn) {
+            auto secs = [&](Clock::time_point t) { return std::chrono::duration<double>(tnow - t).count(); };
+            alpha = cfg.gazeDotAlways ? 1.0 : std::clamp(1 - std::min(secs(lastMove) - cfg.gazeShow, secs(lastHeld)) / 0.25, 0.0, 1.0);
+            const double pulse = secs(pulseAt);
+            if (pulse < 0.6) {
+                scale = 1 + 1.5 * std::max(0.0, 1 - pulse / 0.3);
+                alpha = std::max(alpha, std::clamp((0.6 - pulse) / 0.3, 0.0, 1.0));
+            }
+        }
+        if (tnow < calPanelUntil) alpha = 0;  // the calibration panel is up (see the top)
+        overlay->SetOverlayAlpha(show, float(alpha));
+        overlay->SetOverlayWidthInMeters(show, float(2 * dist * std::tan(scale * cfg.cursorDeg * M_PI / 360)));
+        auto m = Billboard(at, eye);
+        overlay->SetOverlayTransformAbsolute(show, vr::TrackingUniverseStanding, &m);
+        overlay->ShowOverlay(show);
+        overlay->HideOverlay(hide);
+    }
+}
+
+void Pointer::SendRay(const Frame &frame, const Spot &spot) {
+    const vr::TrackedDevicePose_t &hmd = frame.Hmd();
+    const auto tnow = frame.tnow;
+    const Vec3 &eye = frame.eye;
+    const vr::TrackedDevicePose_t &hmdRaw = frame.hmdRaw;
+    const bool dragging = spot.dragging, occluded = spot.occluded, onEdge = spot.onEdge, onScene = spot.onScene;
+    const double best = spot.best, distance = spot.distance;
+    const Vec3 point = spot.point;
+    // Controller ray: from the eye, aimed at the cursor point, converted from the
+    // standing universe to raw tracking space via the HMD's pose in both.
+    const Vec3 aimStanding = Normalize(point - eye);
+    const auto &S = hmd.mDeviceToAbsoluteTracking, &R = hmdRaw.mDeviceToAbsoluteTracking;
+    const Vec3 aim = Rotate(R, RotateInverse(S, aimStanding));  // raw <- head <- standing
+    // Origin partway along the line of sight to the cursor (smaller hit dot).
+    const double toPoint = std::sqrt(Dot(point - eye, point - eye));
+    double originDist = std::max(0.0, std::min(toPoint * cfg.originFraction, toPoint - cfg.originMargin));
+    if (onVrSettings) originDist = std::min(originDist, SETTINGS_ORIGIN);  // SteamVR finds the page
+    const Vec3 originStanding = eye + Normalize(point - eye) * originDist;
+    const Vec3 eyeRaw = Position(R) + Rotate(R, RotateInverse(S, originStanding - eye));
+    lastPoint = point, lastOrigin = originStanding, lastAim = aimStanding;  // tilt starts from here
+    havePoint = true;
+    if (debug && tnow - lastDebug > std::chrono::milliseconds(500)) {
+        lastDebug = tnow;
+        if (systemPointer == vr::k_ulOverlayHandleInvalid) overlay->FindOverlay("system.pointer", &systemPointer);
+        std::printf("dbg %s hit=%s dist=%.2f eye->point=%.2f origin=%.2f vrsettings=%d yaw=%.1f pitch=%.1f gaze=%s steamvr_dot=%d primary=%u\n",
+                    dragging ? "DRAG" : occluded ? "INFRONT" : onEdge ? "EDGE" : onScene ? "SCENE" : (best < 1e8 ? "PANEL" : "FREE"), lastHit.empty() ? "-" : lastHit.c_str(),
+                    distance, toPoint, originDist, onVrSettings, yaw, pitch,
+                    !gazeOn ? "off" : tnow - gz.at > std::chrono::milliseconds(150) ? "stale" : gazeOwns ? "owns" : "mouse",
+                    systemPointer != vr::k_ulOverlayHandleInvalid && overlay->IsOverlayVisible(systemPointer),
+                    overlay->GetPrimaryDashboardDevice());
+        std::fflush(stdout);
+    }
+    const double ayaw = std::atan2(-aim.x, -aim.z) * 180 / M_PI;
+    const double apitch = std::asin(std::clamp(aim.y, -1.0, 1.0)) * 180 / M_PI;
+    if (dragging && (tiltYaw != 0 || tiltPitch != 0)) {
+        // Keep this drag's tilt applied, about the current cursor point.
+        const double yr = tiltYaw * M_PI / 180, pr = tiltPitch * M_PI / 180;
+        const Basis base = AimBasis(aimStanding);
+        const Vec3 up{0, 1, 0}, side = base.x;
+        auto turn = [&](Vec3 v) { return RotateAbout(RotateAbout(v, side, pr), up, yr); };
+        const Vec3 o = point + turn(originStanding - point);
+        const Basis b{turn(base.x), turn(base.y), turn(base.z)};
+        auto toRaw = [&](Vec3 v) { return Rotate(R, RotateInverse(S, v)); };
+        const Vec3 oRaw = Position(R) + toRaw(o - eye);
+        double q[4];
+        BasisQuat({toRaw(b.x), toRaw(b.y), toRaw(b.z)}, q);
+        char msg[200];
+        std::snprintf(msg, sizeof msg, "posq %.5f %.5f %.5f %.6f %.6f %.6f %.6f", oRaw.x, oRaw.y, oRaw.z, q[0],
+                      q[1], q[2], q[3]);
+        SendTo(out, "ft_pointer", msg);
+    } else {
+        char msg[160];
+        std::snprintf(msg, sizeof msg, "pose %.5f %.5f %.5f %.4f %.4f", eyeRaw.x, eyeRaw.y, eyeRaw.z, ayaw, apitch);
+        SendTo(out, "ft_pointer", msg);
+    }
+}

pointer/helper/pointer_gaze.cpp

@@ -0,0 +1,91 @@
+// ft-pointer: gaze mode and its held-back press (see "Gaze mode" at the top).
+// "The top" in comments here is the header comment of ft-pointer.cpp.
+#include "pointer.h"
+
+// Outside games, gaze mode keeps the pointer (see the top); the relay needs to know.
+void Pointer::GazeAwake() {
+    const auto t = Clock::now();
+    if (t - inGameAt > std::chrono::milliseconds(500)) {
+        inGameAt = t;
+        inGame = vr::VRApplications()->GetCurrentSceneProcessId() != 0;
+    }
+    // Hand gestures in the last two minutes keep it the same way.
+    const bool handsRecent = cfg.handsOn && handUsed != Clock::time_point{} && t - handUsed < std::chrono::minutes(2);
+    const bool awake = (gazeOn || handsRecent) && !inGame && !headsetOff;
+    if (awake != gazeAwake || t - gazeAwakeAt > std::chrono::seconds(5)) {
+        if (awake != gazeAwake)
+            std::printf("%s keeps the pointer: %s\n", gazeOn ? "gaze" : "hand use",
+                        awake ? "yes" : "no (off, in a game, or headset off)");
+        if (awake != gazeAwake) std::fflush(stdout);
+        gazeAwake = awake;
+        gazeAwakeAt = t;
+        SendTo(out, "frametop_relay", awake ? "gazeawake 1" : "gazeawake 0");
+    }
+}
+
+// Gaze mode (see the top): the gaze has the pointer, or takes it back when you look
+// well away from it. Not while a press holds the pointer, and only on fresh gaze.
+void Pointer::GazeMode(const Frame &frame) {
+    const vr::TrackedDevicePose_t &hmd = frame.Hmd();
+    const auto tnow = frame.tnow;
+    const Vec3 &eye = frame.eye;
+    if (hmd.bPoseIsValid) lastHead = hmd.mDeviceToAbsoluteTracking, haveHead = true;
+    if (gazeOn && active && hmd.bPoseIsValid && !tilting && !leftHeld && !aimHeld && !clickPress && !clickRelease &&
+        tnow >= dropHoldUntil &&
+        tnow - gz.at < std::chrono::milliseconds(150)) {
+        const Vec3 g = Rotate(hmd.mDeviceToAbsoluteTracking, Direction(gz.hy, gz.hp));
+        if (!gazeOwns && havePoint) {
+            const double off = std::acos(std::clamp(Dot(g, Normalize(lastPoint - eye)), -1.0, 1.0)) * 180 / M_PI;
+            if (off > cfg.gazeRetake && tnow - lastMouse > std::chrono::milliseconds(300)) {
+                if (retakeSince == Clock::time_point{}) retakeSince = tnow;
+                if (tnow - retakeSince >= std::chrono::milliseconds(120)) gazeOwns = true, nudging = false;
+            } else {
+                retakeSince = {};
+            }
+        }
+        if (gazeOwns) {
+            retakeSince = {};
+            anchor = eye;
+            anchored = true;
+            yaw = std::atan2(-g.x, -g.z) * 180 / M_PI;
+            pitch = std::clamp(std::asin(std::clamp(g.y, -1.0, 1.0)) * 180 / M_PI, -85.0, 85.0);
+        }
+    }
+}
+
+void Pointer::CalPanelOpened() {
+    if (calOpened) {  // the calibration panel came up: a press in progress ends, no click
+        calOpened = false;
+        if (hold.src != Src::None) EndHold(true);
+        if (leftHeld) ReleaseLeft();
+        aimHeld = aimHand = clickPress = false;
+    }
+}
+
+// A held-back press (see the top): held still long enough, it's a real press (a drag);
+// released, it's a click where the pointer is now (this frame's pose has gone out).
+void Pointer::HeldBackPress(const Frame &frame) {
+    const auto tnow = frame.tnow;
+    if (!active) aimHeld = aimRight = clickPress = aimHand = confirmLesson = false;  // released meanwhile: nothing to click
+    if (aimHeld && !aimHand && nudgeMoved < 0.2 && tnow - aimSince >= std::chrono::duration<double>(cfg.gazeHold)) {
+        aimHeld = false;
+        gazeBack = true;
+        pressRight = aimRight;  // the right button's: a real right press (see the top)
+        aimRight = false;
+        PressLeft();
+    }
+}
+
+// The click a held-back press ends in: the press now, the release 40 ms later.
+void Pointer::Click(const Frame &frame) {
+    const auto tnow = frame.tnow;
+    if (clickPress) {
+        clickPress = false;
+        PressLeft();
+        clickRelease = true;
+        clickReleaseAt = tnow + std::chrono::milliseconds(40);
+    } else if (clickRelease && tnow >= clickReleaseAt) {
+        clickRelease = false;
+        ReleaseLeft();
+    }
+}

pointer/helper/pointer_holds.cpp

@@ -0,0 +1,202 @@
+// ft-pointer: holds, hands, and the gaze precision buttons (see "Gaze precision" and "Hands" at
+// the top).
+// "The top" in comments here is the header comment of ft-pointer.cpp.
+#include "pointer.h"
+
+// Hands (see the top): pinches and grips from ft-hands.
+void Pointer::Hands(const Frame &frame) {
+    const auto tnow = frame.tnow;
+    vr::TrackedDevicePose_t h0;
+    sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &h0, 1);
+    if (h0.bPoseIsValid) poses.Add(tnow, h0.mDeviceToAbsoluteTracking);
+}
+
+// Where a gesture's point is, seen from the hold's origin: yaw and pitch, degrees.
+bool Pointer::HandAngles(const float p[3], uint64_t t_ns, const Vec3 &from, double &hy, double &hp) {
+    vr::HmdMatrix34_t head;
+    if (!poses.At(t_ns, head)) return false;
+    const Vec3 d = Normalize(Position(head) + Rotate(head, Vec3{p[0], p[1], p[2]}) - from);
+    hy = std::atan2(-d.x, -d.z) * 180 / M_PI;
+    hp = std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI;
+    return true;
+}
+
+void Pointer::EndHold(bool lost) {
+    if (hold.pressed) {
+        if (leftHeld) ReleaseLeft();
+    } else if (aimHeld) {
+        aimHeld = aimHand = false;
+        if (lost) {
+            if (gazeOn) gazeOwns = true, nudging = false;  // no click: the pointer goes back to the gaze
+        } else {
+            clickPress = true;  // the click is on the release, where the pointer is now
+            pressRight = hold.right;
+            gazeBack = nudgeMoved < 0.2;
+        }
+    }
+    if (debug)
+        std::printf("hold %s %s%s\n", hold.src == Src::Hand ? (hold.grip ? "grip" : "pinch")
+                                      : hold.src == Src::Head ? (hold.right ? "key right" : "key left")
+                                      : hold.grip ? "gaze drag" : "gaze precision",
+                    lost ? "lost" : "released", hold.pressed ? "" : lost ? " (no click)" : " (click)");
+    if (debug) std::fflush(stdout);
+    hold = {};
+}
+
+// The press, once a hold's source is set: a real press (dragging until the release), or
+// held back like gaze mode's mouse press (see the top): the click comes on the release.
+void Pointer::StartHold(bool press, Clock::time_point tnow) {
+    hold.startYaw = hold.lastYaw = yaw, hold.startPitch = hold.lastPitch = pitch;
+    hold.pressed = press;
+    if (press) {
+        gazeBack = gazeOn && gazeOwns;
+        PressLeft();
+    } else {
+        gazeOwns = false;
+        nudging = gazeOn && haveHead && tnow - gz.at < std::chrono::milliseconds(200);
+        nudgeRawHy = gz.rhy, nudgeRawHp = gz.rhp, nudgeHead = lastHead;
+        nudgeAt = aimSince = tnow, nudgeMoved = 0;
+        aimHeld = aimHand = true;
+        pulseAt = tnow;
+    }
+}
+
+// The hold's source moved to (hy, hp): past the dead zone, the pointer follows at the gain,
+// from where it was when it got past.
+void Pointer::Steer(double hy, double hp, double deadzone, double gain, Clock::time_point tnow) {
+    const double dy = std::remainder(hy - hold.refYaw, 360.0), dp = hp - hold.refPitch;
+    if (!hold.engaged) {
+        if (std::hypot(dy, dp) <= deadzone) return;
+        hold.engaged = true;
+        hold.refYaw = hy, hold.refPitch = hp;
+        hold.startYaw = hold.lastYaw = yaw, hold.startPitch = hold.lastPitch = pitch;
+        return;
+    }
+    yaw = std::remainder(hold.startYaw + gain * dy, 360.0);
+    pitch = std::clamp(hold.startPitch + gain * dp, -85.0, 85.0);
+    // How far the nudge went: for a keyboard click, from where the dot was at the press
+    // (a head wobbles on the way); otherwise the path, as with the mouse.
+    if (!hold.pressed && hold.src == Src::Head)
+        nudgeMoved = std::hypot(std::remainder(yaw - hold.pressYaw, 360.0), pitch - hold.pressPitch);
+    else if (!hold.pressed)
+        nudgeMoved += std::hypot(std::remainder(yaw - hold.lastYaw, 360.0), pitch - hold.lastPitch);
+    hold.lastYaw = yaw, hold.lastPitch = pitch;
+    lastMove = tnow;  // the dot shows while it moves (gaze mode)
+}
+
+void Pointer::BeginHold(int side, bool grip, const fh_pinch_t &g, Clock::time_point tnow) {
+    handUsed = tnow;
+    if (hold.src != Src::None) {
+        // A grip takes over a pinch on the way to it (closing the hand passes through
+        // one); otherwise one hold at a time.
+        if (hold.src != Src::Hand || !grip || hold.grip) return;
+        aimHeld = aimHand = false;
+        hold = {};
+    }
+    if (leftHeld || aimHeld || clickPress || clickRelease) return;  // the mouse's button is busy
+    if (!active) {
+        Wake(tnow);  // the first gesture only wakes the pointer, like the first mouse move
+        return;
+    }
+    vr::HmdMatrix34_t head;
+    if (!poses.At(g.begin_ns, head)) return;
+    // No pinch just after a key (typing touches thumb to index), and a grip only with
+    // the hand held up (hands on a desk curl like a loose fist; looking down at them
+    // puts them straight ahead in the head's frame, where ft-hands can't tell).
+    const Vec3 at = Position(head) + Rotate(head, Vec3{g.begin_point[0], g.begin_point[1], g.begin_point[2]});
+    if (!grip && tnow - lastTyping < std::chrono::duration<double>(cfg.typingHold)) {
+        if (debug) std::printf("hand pinch ignored: typing\n");
+        return;
+    }
+    if (grip && Position(head).y - at.y > cfg.handBelow) {
+        if (debug) std::printf("hand grip ignored: %.2f m below the eyes\n", Position(head).y - at.y);
+        return;
+    }
+    Hold h;
+    h.src = Src::Hand, h.side = side, h.grip = grip, h.origin = Position(head);
+    if (!HandAngles(g.begin_point, g.begin_ns, h.origin, h.refYaw, h.refPitch)) return;
+    hold = h;
+    // A grip, or a pinch without gaze mode: a real press where the pointer is (where
+    // you look, in gaze mode), dragging with the hand until it opens.
+    StartHold(grip || !gazeOn, tnow);
+    if (debug) std::printf("hand %s %s began\n", side ? "right" : "left", grip ? "grip" : "pinch");
+    if (debug) std::fflush(stdout);
+}
+
+// Gaze precision and gaze drag buttons (see the top): the mouse steers.
+void Pointer::GazePrecisionButtons(const Frame &frame) {
+    const auto tnow = frame.tnow;
+    for (const DevicePress &p : devicePresses) {
+        if (p.source == "left" || p.source == "right") continue;  // no controllers in gaze mode
+        if (!p.down) {
+            if (hold.src == Src::Mouse) EndHold(false);
+            continue;
+        }
+        if (hold.src != Src::None || leftHeld || aimHeld || clickPress || clickRelease) continue;
+        Hold h;
+        h.src = Src::Mouse, h.grip = p.drag;
+        hold = h;
+        StartHold(p.drag, tnow);
+        if (debug) std::printf("hold gaze %s began (%s)\n", p.drag ? "drag" : "precision", p.source.c_str());
+        if (debug) std::fflush(stdout);
+    }
+    devicePresses.clear();
+}
+
+// Pinches and grips from ft-hands (see "Hands" at the top), read every frame.
+void Pointer::PinchesAndGrips(Frame &frame) {
+    const auto tnow = frame.tnow;
+    fh_gestures_t hg;
+    const bool handOk = cfg.handsOn && handFile.Read(hg);
+    frame.handOk = handOk;
+    if (handOk && (hg.seq != handSeq || handFile.opens != handOpens)) {
+        handSeq = hg.seq;
+        handPublished = hg.publish_ns;
+        const fh_pinch_t *slots[2] = {hg.pinch, hg.grip};
+        // A new file, or a tracker whose counters went back: start counting from here.
+        bool rebase = !handBaseline || handFile.opens != handOpens;
+        for (int k = 0; k < 2; ++k)
+            for (int s = 0; s < 2; ++s)
+                rebase = rebase || slots[k][s].begins < seenBegins[k][s] || slots[k][s].ends < seenEnds[k][s];
+        if (rebase) {
+            if (hold.src == Src::Hand) EndHold(true);
+            for (int k = 0; k < 2; ++k)
+                for (int s = 0; s < 2; ++s) seenBegins[k][s] = slots[k][s].begins, seenEnds[k][s] = slots[k][s].ends;
+            handBaseline = true, handOpens = handFile.opens;
+        }
+        // Not over a VR game (unless the dashboard is up), and not with the headset off.
+        const bool allowed = !headsetOff && (!inGame || overlay->IsDashboardVisible());
+        for (int k = 1; k >= 0; --k)  // grips first: one takes over a pinch
+            for (int s = 0; s < 2; ++s) {
+                const fh_pinch_t &g = slots[k][s];
+                const bool began = g.begins != seenBegins[k][s], ended = g.ends != seenEnds[k][s];
+                const bool lost = g.flags & FH_PINCH_LOST;
+                seenBegins[k][s] = g.begins, seenEnds[k][s] = g.ends;
+                auto holding = [&] { return hold.src == Src::Hand && hold.side == s && hold.grip == (k == 1); };
+                if (holding() && ended) EndHold(lost);  // the one held ended (another may have begun)
+                if (began && allowed) {
+                    BeginHold(s, k == 1, g, tnow);
+                    // begun and ended since the last read (a quick tap): its release too
+                    if (!(g.flags & FH_PINCH_DOWN) && holding()) EndHold(lost);
+                }
+            }
+        // The held gesture's hand moves the pointer: past the dead zone (a tap's jitter,
+        // the pinch point shifting as the fingers close), then at the gain, from there.
+        if (hold.src == Src::Hand) {
+            const fh_pinch_t &g = hold.grip ? hg.grip[hold.side] : hg.pinch[hold.side];
+            double hy, hp;
+            if ((g.flags & FH_PINCH_TRACKED) && HandAngles(g.point, hg.capture_ns, hold.origin, hy, hp))
+                Steer(hy, hp, hold.grip ? 0.5 : cfg.pinchDeadzone, hold.grip ? cfg.gripGain : cfg.pinchGain, tnow);
+        }
+    }
+}
+
+// ft-hands stopped (or hung) with a gesture held: it's over, as lost.
+void Pointer::HandsStopped(const Frame &frame) {
+    const auto tnow = frame.tnow;
+    const bool handOk = frame.handOk;
+    const uint64_t nowNs =
+        uint64_t(std::chrono::duration_cast<std::chrono::nanoseconds>(tnow.time_since_epoch()).count());
+    if (hold.src == Src::Hand && (!handOk || !active || nowNs - handPublished > 1'500'000'000ull)) EndHold(true);
+    if (hold.src != Src::None && hold.src != Src::Hand && !active) EndHold(true);
+}

pointer/helper/pointer_input.cpp

@@ -0,0 +1,390 @@
+// ft-pointer: waking the pointer, its buttons, and the commands from the relay.
+// "The top" in comments here is the header comment of ft-pointer.cpp.
+#include "pointer.h"
+
+void Pointer::Wake(Clock::time_point t) {
+    if (t < noWakeUntil || headsetOff) return;
+    wokeAt = t;
+    active = true;
+    recenter = true;
+    SendTo(out, "ft_pointer", "role " + cfg.role);  // POINTER_ROLE, before it takes it
+    SendTo(out, "ft_pointer", "show");
+    claimPending = true;  // take the laser without clicking, once SteamVR has bound the device
+    claimAt = t + std::chrono::milliseconds(300);
+}
+
+void Pointer::PressLeft() {
+    // ft-screens sends the keyboard to the panel clicked last; it sees clicks on
+    // its own screens, but only we know when one lands on another panel.
+    SendTo(out, "ft_screens", "click " + (lastHit.empty() ? std::string("-") : lastHit));
+    // A click after nudging the gaze-placed pointer: the nudge is a lesson, or past
+    // POINTER_GAZE_NUDGE_MAX, a quick check (see the top).
+    if (gazeOn && nudging && !gazeOwns && havePoint && Clock::now() - nudgeAt < std::chrono::seconds(10) &&
+        (confirmLesson || nudgeMoved >= 0.2)) {
+        const Vec3 d = RotateInverse(nudgeHead, Normalize(lastPoint - Position(nudgeHead)));
+        const double ty = std::atan2(-d.x, -d.z) * 180 / M_PI, tp = std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI;
+        const double off = std::hypot(std::remainder(ty - nudgeRawHy, 360.0), tp - nudgeRawHp);
+        char msg[160];
+        if (off <= cfg.gazeNudgeMax)
+            std::snprintf(msg, sizeof msg, "lesson %.3f %.3f %.3f %.3f", nudgeRawHy, nudgeRawHp, ty, tp);
+        else
+            std::snprintf(msg, sizeof msg, "recheck %.0f", off);
+        SendTo(out, "ft_gazed", msg);
+        if (debug) std::printf("gaze %s (nudged %.2f deg, off %.2f)\n", msg, nudgeMoved, off);
+        if (debug) std::fflush(stdout);
+    }
+    nudging = confirmLesson = false;
+    leftHeld = true;
+    dragDistance = lastDistance;
+    pressKey.clear();
+    if (FramePanel(lastHit)) {
+        vr::ETrackingUniverseOrigin uo;
+        auto it = handles.find(lastHit);
+        if (it != handles.end() &&
+            overlay->GetOverlayTransformAbsolute(it->second, &uo, &pressPose) == vr::VROverlayError_None)
+            pressKey = lastHit;
+    }
+    tiltYaw = tiltPitch = 0;  // a new drag starts untilted
+    dropHoldUntil = {};
+    pulseAt = Clock::now();
+    heldButton = pressRight ? "b" : "trigger";
+    pressRight = false;
+    SendTo(out, "ft_pointer", "btn " + heldButton + " 1");
+}
+
+void Pointer::ReleaseLeft() {
+    leftHeld = false;
+    tilting = false;
+    // Hold the drag pose (tilt, frozen distance) while SteamVR finishes the drop.
+    dropHoldUntil = Clock::now() + std::chrono::milliseconds(500);
+    SendTo(out, "ft_pointer", "btn " + heldButton + " 0");
+    // ft-screens releases a button held on its screens in KWin even when SteamVR hands
+    // the release to some other overlay (its catcher usually gets it; this is the backstop).
+    SendTo(out, "ft_screens", "up");
+    if (gazeBack) gazeOwns = true, gazeBack = false;
+}
+
+// The left button, from the relay's "btn trigger", with gaze mode's held-back press (see
+// the top).
+bool Pointer::CanAim() {
+    return gazeOn && cfg.gazeMousePrecision && gazeOwns && !aimHeld && !clickPress && !clickRelease &&
+           hold.src == Src::None;
+}
+
+// Hold the press back: the pointer stops where the gaze put it.
+void Pointer::AimStart(bool right) {
+    gazeOwns = false;
+    nudging = haveHead && Clock::now() - gz.at < std::chrono::milliseconds(200);
+    nudgeRawHy = gz.rhy, nudgeRawHp = gz.rhp, nudgeHead = lastHead;
+    nudgeAt = aimSince = Clock::now(), nudgeMoved = 0;
+    aimHeld = true, aimRight = right;
+}
+
+// A drag the right button began (chordDrag): it lasts while either button is held.
+void Pointer::ChordDrop() {
+    chordDrag = false;
+    if (leftHeld) ReleaseLeft();
+    if (debug) std::printf("mouse drag dropped\n");
+    if (debug) std::fflush(stdout);
+}
+
+void Pointer::LeftButton(bool down) {
+    leftDown = down;
+    if (down) {
+        if (aimHeld && aimRight && !aimHand) {
+            ignoreLeftUp = true;  // the right's press is held back: the left does nothing
+            return;
+        }
+        if (CanAim()) {
+            AimStart(false);
+            return;
+        }
+        PressLeft();
+        return;
+    }
+    if (ignoreLeftUp) {
+        ignoreLeftUp = false;
+        return;
+    }
+    if (chordDrag) {
+        if (!rightDown) ChordDrop();  // otherwise the right holds it (tilting, maybe)
+        return;
+    }
+    if (aimHeld && !aimHand && !aimRight) {
+        aimHeld = false;
+        clickPress = true;  // after this frame's pose, so it lands where the pointer was moved to
+        gazeBack = nudgeMoved < 0.2;
+        return;
+    }
+    if (leftHeld) ReleaseLeft();
+}
+
+// The right button (see the top); false: not taken here (a tilt, or passed on as it is).
+bool Pointer::RightButton(bool down) {
+    rightDown = down;
+    if (down) {
+        if (aimHeld && !aimHand && !aimRight) {
+            // During the left's held-back press: press the left where the pointer is now (the
+            // correction is a lesson), and the mouse drags.
+            aimHeld = false;
+            chordDrag = gazeBack = true;
+            PressLeft();
+            if (debug) std::printf("mouse drag began (right during left)\n");
+            if (debug) std::fflush(stdout);
+            return true;
+        }
+        if (CanAim() && !leftHeld) {
+            AimStart(true);
+            return true;
+        }
+        return false;
+    }
+    if (chordDrag) {
+        if (leftDown) return !swallowedRight;  // the left holds it; a tilt's release ends the tilt
+        tilting = swallowedRight = false;
+        ChordDrop();
+        return true;
+    }
+    if (aimHeld && !aimHand && aimRight) {
+        aimHeld = aimRight = false;
+        pressRight = clickPress = true;  // the right click, where the pointer was moved to
+        gazeBack = nudgeMoved < 0.2;
+        return true;
+    }
+    if (leftHeld && heldButton == "b") {  // its press, held still into a real one
+        ReleaseLeft();
+        return true;
+    }
+    return false;
+}
+
+// POINTER_GAZE_MOUSE_MOVE=held (see the top): the gaze is fresh and nothing is pressed, so a
+// mouse move doesn't move the pointer.
+bool Pointer::MouseMoveHeld() {
+    return gazeOn && cfg.gazeMouseHeld && !inGame && !headsetOff && Clock::now() - gz.at < std::chrono::seconds(1) &&
+           !aimHeld && !leftHeld && !tilting && hold.src == Src::None && !clickPress && !clickRelease;
+}
+
+// Commands from the relay.
+void Pointer::RelayCommands() {
+    char buf[256];
+    ssize_t n;
+    sockaddr_un from{};
+    socklen_t fromLen = sizeof from;
+    while ((n = recvfrom(in, buf, sizeof buf - 1, 0, reinterpret_cast<sockaddr *>(&from), &fromLen)) > 0) {
+        buf[n] = 0;
+        // Reply to the sender (the layout tool binds an abstract address to get answers).
+        const sockaddr_un sender = from;
+        const socklen_t senderLen = fromLen;
+        fromLen = sizeof from;
+        Command(buf, sender, senderLen);
+    }
+}
+
+// One command from the relay (or another sender: reply answers it).
+void Pointer::Command(const char *buf, const sockaddr_un &sender, socklen_t senderLen) {
+    auto reply = [&](const std::string &msg) {
+        if (senderLen > offsetof(sockaddr_un, sun_path))
+            sendto(out, msg.data(), msg.size(), 0, reinterpret_cast<const sockaddr *>(&sender), senderLen);
+    };
+    // The gaze, from ft-gazed: not mouse input, it never wakes the pointer.
+    double g[4];
+    if (std::sscanf(buf, "gz %lf %lf %lf %lf", &g[0], &g[1], &g[2], &g[3]) == 4) {
+        gz = {g[0], g[1], g[2], g[3], Clock::now()};
+        return;
+    }
+    // The gaze calibration panel (see the top): presses answer it instead of clicking.
+    {
+        int on;
+        if (std::sscanf(buf, "calpanel %d", &on) == 1) {
+            const bool was = Clock::now() < calPanelUntil;
+            calPanelUntil = on ? Clock::now() + std::chrono::seconds(3) : Clock::time_point{};
+            if (on && !was) calOpened = true;
+            return;
+        }
+    }
+    if (Clock::now() < calPanelUntil) {
+        const bool accept = !std::strncmp(buf, "btn trigger 1", 13) || !std::strncmp(buf, "gazekey left 1", 14);
+        const bool quit = !std::strncmp(buf, "btn b 1", 7) || !std::strncmp(buf, "gazekey right 1", 15);
+        if (accept || quit) {
+            SendTo(out, "ft_gazed", accept ? "calaccept" : "calquit");
+            return;
+        }
+        if (!std::strncmp(buf, "btn ", 4) || !std::strncmp(buf, "gazekey ", 8) ||
+            !std::strncmp(buf, "precision ", 10) || !std::strncmp(buf, "gazedrag ", 9))
+            return;  // their releases, and the other buttons: nothing to click now
+    }
+    {
+        char kind[16], source[16];
+        int v;
+        if (std::sscanf(buf, "%15s %15s %d", kind, source, &v) == 3 &&
+            (!std::strcmp(kind, "precision") || !std::strcmp(kind, "gazedrag"))) {
+            lastMouse = Clock::now();
+            if (!active) Wake(Clock::now());
+            devicePresses.push_back({source, !std::strcmp(kind, "gazedrag"), v != 0});
+            return;
+        }
+        if (std::sscanf(buf, "gazekey %15s %d", source, &v) == 2 &&
+            (!std::strcmp(source, "left") || !std::strcmp(source, "right"))) {
+            lastMouse = Clock::now();
+            if (!active) Wake(Clock::now());
+            keyPresses.push_back({!std::strcmp(source, "right"), v != 0});
+            return;
+        }
+    }
+    if (std::strcmp(buf, "typing") == 0) {  // not mouse input: it never wakes the pointer
+        lastTyping = Clock::now();
+        return;
+    }
+    if (std::strncmp(buf, "vrbind", 6) == 0) {
+        std::printf("controller buttons: %s\n", controllerButtons.Bind(buf + 6).c_str());
+        std::fflush(stdout);
+        return;
+    }
+    if (std::strncmp(buf, "vrglobal", 8) == 0) {
+        const char *arg = buf + 8;
+        while (*arg == ' ') ++arg;
+        ControllerButtons::SetGlobal(std::strncmp(arg, "off", 3) != 0);
+        reply(controllerButtons.Status());
+        return;
+    }
+    if (std::strncmp(buf, "vrstatus", 8) == 0) {
+        reply(controllerButtons.Status());
+        return;
+    }
+    if (std::strncmp(buf, "overlays", 8) == 0) {
+        overlays.Request(sender, senderLen);  // answered from the list's thread
+        return;
+    }
+    if (std::strncmp(buf, "gaze", 4) == 0) {
+        const char *arg = buf + 4;
+        while (*arg == ' ') ++arg;
+        if (*arg != '?') {  // "gaze ?" only asks
+            gazeOn = std::strncmp(arg, "on", 2) == 0    ? true
+                     : std::strncmp(arg, "off", 3) == 0 ? false
+                                                        : !gazeOn;
+            gazeOwns = true, nudging = false;
+            std::printf("gaze mode %s\n", gazeOn ? "on" : "off");
+            std::fflush(stdout);
+        }
+        reply(gazeOn ? "ok on" : "ok off");
+        return;
+    }
+    const bool mouseInput = std::strncmp(buf, "move", 4) == 0 || std::strncmp(buf, "btn", 3) == 0 ||
+                            std::strncmp(buf, "scroll", 6) == 0;
+    // A move held back (POINTER_GAZE_MOUSE_MOVE=held) only wakes the pointer: it isn't using
+    // the mouse, so a drifting mouse doesn't keep the gaze from taking the pointer back.
+    const bool moveHeldBack = std::strncmp(buf, "move", 4) == 0 && MouseMoveHeld();
+    if (mouseInput && !moveHeldBack) lastMouse = Clock::now();
+    // Any mouse input wakes the pointer (after a controller took over, or a helper restart).
+    if (!active && mouseInput) Wake(Clock::now());
+    if (moveHeldBack) return;
+    double a, b;
+    char key[128];
+    double px, py, pz, pyaw, ppitch, proll = 0, pgrab = -1;
+    if (std::sscanf(buf, "grabprobe %127s", key) == 1) {
+        GrabProbe(key);
+        return;
+    }
+    if (std::sscanf(buf, "place %127s %lf %lf %lf %lf %lf %lf %lf", key, &px, &py, &pz, &pyaw, &ppitch, &proll,
+                    &pgrab) >= 6) {
+        reply(Place(key, {px, py, pz}, PanelBasis(pyaw, ppitch, proll), pgrab >= 0 ? pgrab : cfg.grabOffset));
+        return;
+    }
+    if (std::sscanf(buf, "measure %127s", key) == 1) {
+        Panel p;
+        Vec3 eye;
+        const std::string err = FindPanel(key, p, eye);
+        char msg[400] = "";
+        if (err.empty())
+            std::snprintf(msg, sizeof msg, "ok %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f",
+                          p.center.x, p.center.y, p.center.z, p.width, p.height, p.basis.x.x, p.basis.x.y,
+                          p.basis.x.z, p.basis.y.x, p.basis.y.y, p.basis.y.z, p.basis.z.x, p.basis.z.y,
+                          p.basis.z.z);
+        reply(err.empty() ? msg : "error " + err);
+        return;
+    }
+    if (std::strncmp(buf, "head", 4) == 0) {
+        vr::TrackedDevicePose_t h;
+        sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &h, 1);
+        const Vec3 e = Position(h.mDeviceToAbsoluteTracking);
+        const Vec3 f = Rotate(h.mDeviceToAbsoluteTracking, {0, 0, -1});
+        char msg[200];
+        std::snprintf(msg, sizeof msg, "ok %.4f %.4f %.4f %.2f %.2f", e.x, e.y, e.z,
+                      std::atan2(-f.x, -f.z) * 180 / M_PI, std::asin(std::clamp(f.y, -1.0, 1.0)) * 180 / M_PI);
+        reply(h.bPoseIsValid ? msg : "error no head pose (headset off?)");
+        return;
+    }
+    if (std::strncmp(buf, "debug", 5) == 0) {
+        debug = !debug;
+        std::printf("debug %s\n", debug ? "on" : "off");
+        std::fflush(stdout);
+        return;
+    }
+    if (std::strncmp(buf, "btn trigger 1", 13) == 0) {
+        LeftButton(true);
+        return;
+    } else if (std::strncmp(buf, "btn trigger 0", 13) == 0) {
+        LeftButton(false);
+        return;
+    } else if (std::strncmp(buf, "btn b 1", 7) == 0 && RightButton(true)) {
+        return;
+    } else if (std::strncmp(buf, "btn b 0", 7) == 0 && RightButton(false)) {
+        return;
+    } else if (std::strncmp(buf, "btn b 1", 7) == 0 && leftHeld) {
+        tilting = tiltStart = swallowedRight = true;  // right press while dragging: tilt, no right-click
+        return;
+    } else if (std::strncmp(buf, "btn b 0", 7) == 0 && swallowedRight) {
+        tilting = swallowedRight = false;
+        return;
+    }
+    if (tilting && std::sscanf(buf, "move %lf %lf", &a, &b) == 2) {
+        tiltYaw += a;
+        tiltPitch = std::clamp(tiltPitch + b, -80.0, 80.0);
+        return;
+    }
+    if (std::sscanf(buf, "move %lf %lf", &a, &b) == 2) {
+        lastMove = Clock::now();  // the dot shows while the mouse moves it (gaze mode)
+        if (gazeOn && gazeOwns) {
+            // The mouse takes the pointer from the gaze, from where the gaze left it.
+            gazeOwns = false;
+            nudging = haveHead && Clock::now() - gz.at < std::chrono::milliseconds(200);
+            nudgeRawHy = gz.rhy, nudgeRawHp = gz.rhp, nudgeHead = lastHead;
+            nudgeAt = Clock::now(), nudgeMoved = 0;
+        }
+        if (nudging || aimHeld) nudgeMoved += std::hypot(a, b);
+        if (!anchored) recenter = true;
+        yaw += a;
+        while (yaw > 180) yaw -= 360;
+        while (yaw < -180) yaw += 360;
+        pitch = std::clamp(pitch + b, -85.0, 85.0);
+    } else if (std::strncmp(buf, "recenter", 8) == 0) {
+        recenter = true;
+    } else if (std::strncmp(buf, "reload", 6) == 0) {
+        cfg.Load(ReadConfig());
+        ApplyConfig();
+        lastSlow = Clock::now() - std::chrono::seconds(10);  // apply POINTER_IGNORE now
+        std::printf("reloaded: free distance %.2f m, dot %.2f deg, origin %.2f, head follow %s, leash %.0f deg, "
+                    "controller pickup %.1fx, %zu ignored\n",
+                    cfg.freeDistance, cfg.cursorDeg, cfg.originFraction, follow ? "on" : "off", cfg.leashDeg, cfg.pickupScale,
+                    cfg.ignore.size());
+        std::fflush(stdout);
+    } else if (std::strncmp(buf, "follow", 6) == 0) {
+        const char *arg = buf + 6;
+        while (*arg == ' ') ++arg;
+        const bool was = follow;
+        follow = std::strncmp(arg, "on", 2) == 0 ? true : std::strncmp(arg, "off", 3) == 0 ? false : !follow;
+        if (follow && !was) followReset = true;
+        std::printf("head follow %s (leash %.0f deg)\n", follow ? "on" : "off", cfg.leashDeg);
+        std::fflush(stdout);
+    } else if (std::strncmp(buf, "show", 4) == 0) {
+        if (!active) Wake(Clock::now());
+    } else if (std::strncmp(buf, "hide", 4) == 0) {
+        active = false;
+        overlay->HideOverlay(cursor);
+        overlay->HideOverlay(marker);
+        SendTo(out, "ft_pointer", "hide");
+    } else {
+        SendTo(out, "ft_pointer", buf);  // btn, scroll
+    }
+}

pointer/helper/pointer_keys.cpp

@@ -0,0 +1,113 @@
+// ft-pointer: keyboard clicks (see "Keyboard clicks" under "Gaze precision" at the top).
+// "The top" in comments here is the header comment of ft-pointer.cpp.
+#include "pointer.h"
+
+// Where the head faces: yaw and pitch, degrees.
+bool Pointer::HeadAngles(const vr::TrackedDevicePose_t &hmd, double &hy, double &hp) {
+    const auto &hm = hmd.mDeviceToAbsoluteTracking.m;
+    if (!hmd.bPoseIsValid) return false;
+    const Vec3 f = Normalize({-hm[0][2], -hm[1][2], -hm[2][2]});
+    hy = std::atan2(-f.x, -f.z) * 180 / M_PI;
+    hp = std::asin(std::clamp(f.y, -1.0, 1.0)) * 180 / M_PI;
+    return true;
+}
+
+// Keyboard clicks (see the top): held back at the gaze, steered by the head.
+void Pointer::KeyboardClicks(const Frame &frame) {
+    const vr::TrackedDevicePose_t &hmd = frame.Hmd();
+    const auto tnow = frame.tnow;
+    for (const KeyPress &k : keyPresses) {
+        (k.right ? keyRightDown : keyLeftDown) = k.down;
+        if (!k.down) {
+            if (keyTilting && k.right) {  // the tilt ends; the head drags again from here
+                keyTilting = tilting = false;
+                hold.engaged = false;
+                if (debug) std::printf("hold key left: tilt ended\n");
+                if (debug) std::fflush(stdout);
+            }
+            // The keys holding it: a gaze_left then gaze_right drag, either; otherwise its own.
+            const bool held = hold.keyDrag ? keyLeftDown || keyRightDown : hold.right ? keyRightDown : keyLeftDown;
+            if (hold.src == Src::Head && !held) {
+                if (!hold.pressed && aimHeld && tnow - aimSince < std::chrono::duration<double>(cfg.keyTap)) {
+                    // A quick tap: a click where the dot was at the press, and the gaze was right.
+                    yaw = hold.pressYaw, pitch = hold.pressPitch;
+                    nudgeMoved = 0;
+                    confirmLesson = true;
+                }
+                EndHold(false);
+                keyTilting = false;
+            }
+            continue;
+        }
+        if (k.right && hold.src == Src::Head && !hold.right && hold.pressed && leftHeld) {
+            // gaze_right during a gaze_left drag: tilt while it's held (see the top).
+            tilting = tiltStart = keyTilting = true;
+            HeadAngles(hmd, keyTiltYaw, keyTiltPitch);
+            if (debug) std::printf("hold key left: tilt began\n");
+            if (debug) std::fflush(stdout);
+            continue;
+        }
+        if (k.right && hold.src == Src::Head && !hold.right && !hold.pressed && aimHeld) {
+            // gaze_right while gaze_left aims: press the left button where the dot is now
+            // (the correction is a lesson), then the head drags.
+            aimHeld = aimHand = false;
+            hold.pressed = hold.grip = hold.keyDrag = true, hold.promote = false, hold.engaged = false;
+            HeadAngles(hmd, hold.refYaw, hold.refPitch);
+            gazeBack = gazeOn;
+            pressRight = false;
+            PressLeft();
+            if (debug) std::printf("hold key left: pressed (right key)\n");
+            if (debug) std::fflush(stdout);
+            continue;
+        }
+        if (hold.src != Src::None || leftHeld || aimHeld || clickPress || clickRelease) continue;
+        Hold h;
+        h.src = Src::Head, h.promote = true, h.right = k.right, h.pressYaw = yaw, h.pressPitch = pitch;
+        if (!HeadAngles(hmd, h.refYaw, h.refPitch)) continue;
+        hold = h;
+        StartHold(false, tnow);
+        if (debug) std::printf("hold key %s began\n", k.right ? "right" : "left");
+        if (debug) std::fflush(stdout);
+    }
+    keyPresses.clear();
+}
+
+// A keyboard click's hold: the head steers the pointer, or turns the panel in a tilt.
+void Pointer::HeadSteer(const Frame &frame) {
+    const vr::TrackedDevicePose_t &hmd = frame.Hmd();
+    const auto tnow = frame.tnow;
+    if (hold.src == Src::Head) {
+        double hy, hp;
+        if (HeadAngles(hmd, hy, hp) && keyTilting) {
+            // The head turns the panel, as the mouse does in a tilt; the pointer stays put.
+            tiltYaw += std::remainder(hy - keyTiltYaw, 360.0);
+            tiltPitch = std::clamp(tiltPitch + hp - keyTiltPitch, -80.0, 80.0);
+            keyTiltYaw = hy, keyTiltPitch = hp;
+        } else if (HeadAngles(hmd, hy, hp)) {
+            Steer(hy, hp, hold.pressed ? 0.3 : cfg.headDeadzone, 1.0, tnow);
+        }
+    }
+}
+
+// A keyboard click held still for POINTER_GAZE_HOLD: a real press, then the head drags
+// (from where it is now).
+void Pointer::KeyboardClickHold(const Frame &frame) {
+    const vr::TrackedDevicePose_t &hmd = frame.Hmd();
+    const auto &hm = hmd.mDeviceToAbsoluteTracking.m;
+    const auto tnow = frame.tnow;
+    if (aimHeld && aimHand && hold.promote && !hold.engaged &&
+        tnow - aimSince >= std::chrono::duration<double>(cfg.gazeHold)) {
+        aimHeld = aimHand = false;
+        hold.pressed = hold.grip = true;
+        if (hmd.bPoseIsValid) {
+            const Vec3 f = Normalize({-hm[0][2], -hm[1][2], -hm[2][2]});
+            hold.refYaw = std::atan2(-f.x, -f.z) * 180 / M_PI;
+            hold.refPitch = std::asin(std::clamp(f.y, -1.0, 1.0)) * 180 / M_PI;
+        }
+        gazeBack = gazeOn;
+        pressRight = hold.right;
+        PressLeft();
+        if (debug) std::printf("hold key %s: pressed (held still)\n", hold.right ? "right" : "left");
+        if (debug) std::fflush(stdout);
+    }
+}

pointer/helper/pointer_place.cpp

@@ -0,0 +1,168 @@
+// ft-pointer: panel placement for layouts (see "Placement" at the top).
+// "The top" in comments here is the header comment of ft-pointer.cpp.
+#include "pointer.h"
+
+namespace {
+
+void SleepMs(int ms) { std::this_thread::sleep_for(std::chrono::milliseconds(ms)); }
+
+}  // namespace
+
+// Placement speeds (see "Placement" at the top).
+constexpr double kPlaceDegPerSec = 60;      // tested: 40 deg/s is applied exactly
+
+// --- Panel placement (see "Placement" at the top of the file) ---
+// Device pose, given in the standing universe, sent to the driver in raw space.
+void Pointer::SendPose(Vec3 originStanding, const Basis &b) {
+    vr::TrackedDevicePose_t s, r;
+    sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &s, 1);
+    sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseRawAndUncalibrated, 0, &r, 1);
+    const auto &S = s.mDeviceToAbsoluteTracking, &R = r.mDeviceToAbsoluteTracking;
+    auto toRaw = [&](Vec3 v) { return Rotate(R, RotateInverse(S, v)); };
+    const Vec3 o = Position(R) + toRaw(originStanding - Position(S));
+    double q[4];
+    BasisQuat({toRaw(b.x), toRaw(b.y), toRaw(b.z)}, q);
+    char msg[200];
+    std::snprintf(msg, sizeof msg, "posq %.5f %.5f %.5f %.6f %.6f %.6f %.6f", o.x, o.y, o.z, q[0], q[1], q[2], q[3]);
+    SendTo(out, "ft_pointer", msg);
+}
+
+std::pair<bool, Vec3> Pointer::HeadPos() {
+    vr::TrackedDevicePose_t s;
+    sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &s, 1);
+    return std::make_pair(s.bPoseIsValid, Position(s.mDeviceToAbsoluteTracking));
+}
+
+// Borrow the device and the laser for a placement: connect, claim, laser mode on.
+void Pointer::Borrow() {
+    SendTo(out, "ft_pointer", "show");
+    overlay->ShowOverlay(laserMode);
+    overlay->HideOverlay(cursor);
+    overlay->HideOverlay(marker);
+    SleepMs(active ? 50 : 400);  // a fresh connect needs SteamVR to bind the device
+    SendTo(out, "ft_pointer", "btn a 1");
+    SleepMs(60);
+    SendTo(out, "ft_pointer", "btn a 0");
+}
+
+void Pointer::GiveBack() {
+    if (!active) {
+        SendTo(out, "ft_pointer", "hide");
+        overlay->HideOverlay(laserMode);
+        laserModeShown = false;
+    }
+}
+
+std::string Pointer::FindPanel(const char *key, Panel &p, Vec3 &eye) {
+    vr::VROverlayHandle_t h;
+    if (overlay->FindOverlay(key, &h) != vr::VROverlayError_None) return std::string("no overlay ") + key;
+    bool valid;
+    std::tie(valid, eye) = HeadPos();
+    if (!valid) return "no head pose (headset off?)";
+    p = ScanPanel(h, eye, 0.5);
+    if (!p.found) return std::string("panel not visible: ") + key;
+    return "";
+}
+
+// grabprobe (maintenance): aim down from the panel's bottom edge, 1 cm a step, and log where
+// SteamVR's laser hits something (the hit dot shows) and whether the window controls are up.
+void Pointer::GrabProbe(const char *key) {
+    Panel p;
+    Vec3 eye;
+    const std::string err = FindPanel(key, p, eye);
+    if (!err.empty()) {
+        std::printf("grabprobe: %s\n", err.c_str());
+        std::fflush(stdout);
+        return;
+    }
+    Borrow();
+    vr::VROverlayHandle_t footer = vr::k_ulOverlayHandleInvalid;
+    overlay->FindOverlay("valve.steam.gamepadui.floatingfooter", &footer);
+    const Vec3 bottom = p.center - p.basis.y * (p.height / 2);
+    std::printf("grabprobe %s: center (%.3f %.3f %.3f) %.3f x %.3f m\n", key, p.center.x, p.center.y, p.center.z,
+                p.width, p.height);
+    for (int cm = -5; cm <= 45; ++cm) {
+        const Vec3 target = bottom - p.basis.y * (cm / 100.0);
+        SendPose(eye, AimBasis(target - eye));
+        SleepMs(90);
+        const bool dot = systemPointer != vr::k_ulOverlayHandleInvalid && overlay->IsOverlayVisible(systemPointer);
+        const bool foot = footer != vr::k_ulOverlayHandleInvalid && overlay->IsOverlayVisible(footer);
+        std::printf("  %+3d cm below the bottom edge: steamvr_dot=%d footer=%d", cm, dot, foot);
+        if (foot) {
+            vr::ETrackingUniverseOrigin uo;
+            vr::HmdMatrix34_t t{};
+            if (overlay->GetOverlayTransformAbsolute(footer, &uo, &t) == vr::VROverlayError_None) {
+                const Vec3 f = Position(t) - p.center;
+                std::printf(" footer at panel (%.3f %.3f %.3f)", Dot(f, p.basis.x), Dot(f, p.basis.y),
+                            Dot(f, p.basis.z));
+            }
+        }
+        std::printf("\n");
+    }
+    std::fflush(stdout);
+    GiveBack();
+}
+
+// Carry a floating panel so its centre lands on `target` with frame `bt` (see
+// "Placement" at the top). Returns "ok ..." or "error ...".
+std::string Pointer::Place(const char *key, Vec3 target, const Basis &bt, double grabBelow) {
+    Panel p;
+    Vec3 eye;
+    std::string err = FindPanel(key, p, eye);
+    if (!err.empty()) return "error " + err;
+    auto offBy = [&](const Panel &q, double &cm, double &deg) {
+        cm = Length(q.center - target) * 100;
+        const double c = (Dot(q.basis.x, bt.x) + Dot(q.basis.y, bt.y) + Dot(q.basis.z, bt.z) - 1) / 2;
+        deg = std::acos(std::clamp(c, -1.0, 1.0)) * 180 / M_PI;
+    };
+    double cm, deg;
+    int moves = 0;
+    Borrow();
+    for (int attempt = 0; attempt < 3; ++attempt) {
+        offBy(p, cm, deg);
+        if (cm < 1.5 && deg < 1.0) break;
+        // The rigid motion that takes the panel to the target: rotate by R, then move.
+        auto turn = [&](Vec3 v) { return FromBasis(bt, ToBasis(p.basis, v)); };
+        double q[4];
+        BasisQuat({turn({1, 0, 0}), turn({0, 1, 0}), turn({0, 0, 1})}, q);
+        const double angle = 2 * std::acos(std::clamp(q[0], -1.0, 1.0));
+        const Vec3 axis = std::sin(angle / 2) > 1e-6 ? Normalize({q[1], q[2], q[3]}) : Vec3{0, 1, 0};
+        const Vec3 grab = p.center - p.basis.y * (p.height / 2 + grabBelow);
+        const Basis d0 = AimBasis(grab - eye);
+        const Vec3 o1 = target + turn(eye - p.center);  // device origin at the end
+        ++moves;
+        SendPose(eye, d0);
+        SleepMs(150);  // hover: the window controls come up
+        SendTo(out, "ft_pointer", "btn trigger 1");
+        SleepMs(150);
+        // 1. Rotate about the device origin (the eye): the dashboard applies it exactly.
+        const int rsteps = std::max(4, int(angle * 180 / M_PI / kPlaceDegPerSec * 60));
+        for (int i = 1; i <= rsteps; ++i) {
+            const double a = angle * i / rsteps;
+            auto r = [&](Vec3 v) { return RotateAbout(v, axis, a); };
+            SendPose(eye, {r(d0.x), r(d0.y), r(d0.z)});
+            SleepMs(16);
+        }
+        // 2. Slide the device slowly: fast moves are accelerated by the dashboard.
+        const Basis d1{turn(d0.x), turn(d0.y), turn(d0.z)};
+        const int tsteps = std::max(4, int(Length(o1 - eye) / cfg.slideSpeed * 60));
+        for (int i = 1; i <= tsteps; ++i) {
+            SendPose(eye + (o1 - eye) * (double(i) / tsteps), d1);
+            SleepMs(16);
+        }
+        SleepMs(100);
+        SendTo(out, "ft_pointer", "btn trigger 0");
+        SleepMs(600);  // the dashboard re-reads the pose up to 150 ms after the release
+        err = FindPanel(key, p, eye);
+        if (!err.empty()) break;
+    }
+    GiveBack();
+    if (!err.empty()) return "error after the move: " + err;
+    offBy(p, cm, deg);
+    char msg[160];
+    std::snprintf(msg, sizeof msg, "ok %s off by %.1f cm, %.1f deg after %d move%s", key, cm, deg, moves,
+                  moves == 1 ? "" : "s");
+    std::printf("place: %s\n", msg);
+    std::fflush(stdout);
+    return msg;
+}

pointer/helper/vrbuttons.h

@@ -111,14 +111,6 @@ class ControllerButtons {
         }
     }
 
-    // Whether Poll reads any button now (the main loop can't sleep long while it does).
-    bool Watching(bool inGame) const {
-        if (!ok_ || (inGame && !games_ && !capture_)) return false;
-        for (int i = 0; i < kCount; ++i)
-            if (bound_[i]) return true;
-        return false;
-    }
-
     // {"manifest":true,"global":false,"bound":[...],"active":[...]}: active = bound and
     // delivered (a controller that has the button is on, and SteamVR lets us have it).
     std::string Status() const {

power/ft-powerd.cpp

@@ -306,19 +306,14 @@ int main() {
         std::printf("displays on: %s\n", why.c_str());
     };
 
-    // SteamVR is asked every 100 ms (events, the activity level, poses: IPC calls to
-    // vrserver). Input wakes the loop too, to wake the displays at once, but a moving mouse
-    // sends hundreds of events a second, so those wakes only drain the devices.
-    constexpr auto kTick = std::chrono::milliseconds(100);
-    auto lastVr = now - kTick;
     std::vector<pollfd> fds;
     while (!stopSignal) {
+        // Sleep until input or 100 ms: input wakes the displays at once, poses every tick.
         fds.clear();
         fds.push_back({ctl, POLLIN, 0});
         for (auto &[path, d] : inputs)
             if (d.fd >= 0) fds.push_back({d.fd, POLLIN, 0});
-        const auto wait = std::chrono::ceil<std::chrono::milliseconds>(kTick - (Clock::now() - lastVr));
+        poll(fds.data(), fds.size(), 100);
-        poll(fds.data(), fds.size(), std::clamp<int>(wait.count(), 0, kTick.count()));
         now = Clock::now();
         const bool grace = now < graceUntil;
         std::string use;  // why this tick counts as use
@@ -334,74 +329,71 @@ int main() {
             lastConf = now;
         }
 
-        if (now - lastVr >= kTick) {
+        vr::VREvent_t ev;
-            lastVr = now;
+        while (sys->PollNextEvent(&ev, sizeof ev)) {
-            vr::VREvent_t ev;
+            if (ev.eventType == vr::VREvent_Quit) {
-            while (sys->PollNextEvent(&ev, sizeof ev)) {
+                stopSignal = SIGTERM;
-                if (ev.eventType == vr::VREvent_Quit) {
+                sys->AcknowledgeQuit_Exiting();
-                    stopSignal = SIGTERM;
+            } else if (ev.eventType == vr::VREvent_TrackedDeviceActivated ||
-                    sys->AcknowledgeQuit_Exiting();
+                       ev.eventType == vr::VREvent_TrackedDeviceDeactivated) {
-                } else if (ev.eventType == vr::VREvent_TrackedDeviceActivated ||
+                lastClasses = now - std::chrono::hours(1);
-                           ev.eventType == vr::VREvent_TrackedDeviceDeactivated) {
-                    lastClasses = now - std::chrono::hours(1);
-                }
-            }
-            if (now - lastClasses > std::chrono::seconds(5)) {
-                for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i)
-                    classes[i] = sys->GetTrackedDeviceClass(i);
-                lastClasses = now;
             }
+        }
+        if (now - lastClasses > std::chrono::seconds(5)) {
+            for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i)
+                classes[i] = sys->GetTrackedDeviceClass(i);
+            lastClasses = now;
+        }
 
-            // SteamVR's standby (headset taken off) turns the displays off itself; putting the
+        // SteamVR's standby (headset taken off) turns the displays off itself; putting the
-            // headset on again (or the mount covering the sensor again) counts as use.
+        // headset on again (or the mount covering the sensor again) counts as use.
-            // A change counts once it has held for a second (the first reading after connecting
+        // A change counts once it has held for a second (the first reading after connecting
-            // is Idle).
+        // is Idle).
-            const auto level = sys->GetTrackedDeviceActivityLevel(vr::k_unTrackedDeviceIndex_Hmd);
+        const auto level = sys->GetTrackedDeviceActivityLevel(vr::k_unTrackedDeviceIndex_Hmd);
-            const bool awayNow = level == vr::k_EDeviceActivityLevel_Idle || level == vr::k_EDeviceActivityLevel_Standby ||
+        const bool awayNow = level == vr::k_EDeviceActivityLevel_Idle || level == vr::k_EDeviceActivityLevel_Standby ||
-                                 level == vr::k_EDeviceActivityLevel_Idle_Timeout;
+                             level == vr::k_EDeviceActivityLevel_Idle_Timeout;
-            if (awayNow == away) awaySince = now;
+        if (awayNow == away) awaySince = now;
-            if (awayNow != away && now - awaySince >= std::chrono::seconds(1)) {
+        if (awayNow != away && now - awaySince >= std::chrono::seconds(1)) {
-                away = awayNow;
+            away = awayNow;
-                std::printf("SteamVR: headset %s\n", away ? "off (SteamVR's standby has the displays)" : "on");
+            std::printf("SteamVR: headset %s\n", away ? "off (SteamVR's standby has the displays)" : "on");
-                if (!away) use = "headset on again";
+            if (!away) use = "headset on again";
-            }
+        }
 
-            // Raw poses: recentering or a new play area doesn't move anything.
+        // Raw poses: recentering or a new play area doesn't move anything.
-            sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseRawAndUncalibrated, 0, poses, vr::k_unMaxTrackedDeviceCount);
+        sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseRawAndUncalibrated, 0, poses, vr::k_unMaxTrackedDeviceCount);
-            for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) {
+        for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) {
-                const auto c = classes[i];
+            const auto c = classes[i];
-                if (c != vr::TrackedDeviceClass_HMD && c != vr::TrackedDeviceClass_Controller &&
+            if (c != vr::TrackedDeviceClass_HMD && c != vr::TrackedDeviceClass_Controller &&
-                    c != vr::TrackedDeviceClass_GenericTracker)
+                c != vr::TrackedDeviceClass_GenericTracker)
-                    continue;
+                continue;
-                const auto &p = poses[i];
+            const auto &p = poses[i];
-                if (!p.bPoseIsValid || p.eTrackingResult != vr::TrackingResult_Running_OK) {
+            if (!p.bPoseIsValid || p.eTrackingResult != vr::TrackingResult_Running_OK) {
-                    anchors[i].set = false;  // tracking back or a controller turned on: start over
+                anchors[i].set = false;  // tracking back or a controller turned on: start over
-                    continue;
+                continue;
-                }
+            }
-                if (!anchors[i].set || grace || now - anchors[i].at > kMoveWindow) {
+            if (!anchors[i].set || grace || now - anchors[i].at > kMoveWindow) {
-                    SetAnchor(anchors[i], p.mDeviceToAbsoluteTracking, now);
+                SetAnchor(anchors[i], p.mDeviceToAbsoluteTracking, now);
-                    continue;
+                continue;
-                }
+            }
-                double metres, degrees;
+            double metres, degrees;
-                Distance(anchors[i], p.mDeviceToAbsoluteTracking, metres, degrees);
+            Distance(anchors[i], p.mDeviceToAbsoluteTracking, metres, degrees);
-                if (metres > moveMetres || degrees > moveDegrees) {
+            if (metres > moveMetres || degrees > moveDegrees) {
-                    SetAnchor(anchors[i], p.mDeviceToAbsoluteTracking, now);
+                SetAnchor(anchors[i], p.mDeviceToAbsoluteTracking, now);
-                    if (use.empty()) {
+                if (use.empty()) {
-                        char why[96];
+                    char why[96];
-                        std::snprintf(why, sizeof why, "%s %u moved %.1f mm, %.2f deg",
+                    std::snprintf(why, sizeof why, "%s %u moved %.1f mm, %.2f deg",
-                                      c == vr::TrackedDeviceClass_HMD ? "headset" : "device", i, metres * 1000, degrees);
+                                  c == vr::TrackedDeviceClass_HMD ? "headset" : "device", i, metres * 1000, degrees);
-                        use = why;
+                    use = why;
-                    }
                 }
             }
+        }
 
-            if (off && ReadInt(kBacklight, -1) > 0) {
+        if (off && ReadInt(kBacklight, -1) > 0) {
-                // Something else lit them (SteamVR leaving standby, the brightness setting).
+            // Something else lit them (SteamVR leaving standby, the brightness setting).
-                off = false;
+            off = false;
-                std::remove(SavedPath().c_str());
+            std::remove(SavedPath().c_str());
-                use = "turned on elsewhere";
+            use = "turned on elsewhere";
-                std::printf("displays on: turned on elsewhere\n");
+            std::printf("displays on: turned on elsewhere\n");
-            }
         }
         if (!use.empty()) {
             lastUse = now;

remote/ft-remote-settings

@@ -83,7 +83,6 @@ class Window(Adw.ApplicationWindow):
         self.status = {}
         self.revealed = False
         self.setting = False  # the switch is being set from the status, not by the user
-        self.asking = False  # a status check is running
 
         self.toasts = Adw.ToastOverlay()
         view = Adw.ToolbarView()
@@ -140,27 +139,16 @@ class Window(Adw.ApplicationWindow):
                         "/tmp/frametop-remote.log and /tmp/frametop-vnc.log.")
         page.add(about)
 
-        # Each check runs remote-ctl.sh (bash, curl, and python3 for the tailnet name), so
-        # it runs every 5 seconds, when the window comes to the front, and after a change.
         self.refresh()
-        GLib.timeout_add_seconds(5, self.refresh)
+        GLib.timeout_add_seconds(2, self.refresh)
-        self.connect("notify::is-active", lambda win, _param: win.is_active() and win.refresh())
 
     # --- status ---
     def refresh(self):
-        if not self.asking:
+        proc = Gio.Subprocess.new([CTL, "status"], Gio.SubprocessFlags.STDOUT_PIPE | Gio.SubprocessFlags.STDERR_SILENCE)
-            self.asking = True
+        proc.communicate_utf8_async(None, None, self.on_status)
-            proc = Gio.Subprocess.new([CTL, "status"], Gio.SubprocessFlags.STDOUT_PIPE | Gio.SubprocessFlags.STDERR_SILENCE)
-            proc.communicate_utf8_async(None, None, self.on_status)
         return True
 
-    def refresh_soon(self):
-        """Again in a moment: remote access takes a second or two to start or stop."""
-        self.refresh()
-        GLib.timeout_add_seconds(2, lambda: self.refresh() and False)
-
     def on_status(self, proc, result):
-        self.asking = False
         try:
             _, out, _ = proc.communicate_utf8_finish(result)
         except GLib.Error:
@@ -209,7 +197,7 @@ class Window(Adw.ApplicationWindow):
             self.toast("Remote access turns on at the next desktop restart")
         else:
             self.toast("Remote access on")
-        self.refresh_soon()
+        self.refresh()
 
     def on_reveal(self, button):
         self.revealed = button.get_active()
@@ -233,7 +221,7 @@ class Window(Adw.ApplicationWindow):
         if self.status.get("running") == "1":
             subprocess.run([CTL, "restart"], stdin=subprocess.DEVNULL, capture_output=True)
         self.toast("New password set" + (": remote access restarted" if self.status.get("running") == "1" else ""))
-        self.refresh_soon()
+        self.refresh()
 
     def copy(self, text, done):
         if text:

screens/build.sh

@@ -23,6 +23,6 @@ $cxx -c -o build/handcut.o handcut.cpp
 $cxx -c -o build/handtest.o handtest.cpp
 vrlibs="$(pkg-config --libs egl glesv2 gbm) -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64"
 g++ -o build/ft-screens build/compositor.o build/vr.o build/keyboard.o build/handcut.o \
-  $(pkg-config --libs wlroots-0.20 wayland-server xkbcommon pixman-1) $vrlibs
+  $(pkg-config --libs wlroots-0.20 wayland-server xkbcommon) $vrlibs
 g++ -o build/ft-handtest build/handtest.o build/handcut.o $vrlibs
 echo "built build/ft-screens build/ft-handtest"'

screens/compositor.c

@@ -13,7 +13,7 @@
 //     scale first ("scale <screen> <s>", from ft-layout).
 //
 // Usage: ft-screens [--socket NAME] [--control NAME] [--no-vr] [--screen WxH@METRES]...
-//                   [--spares N] [--rates F,V,H] [-- COMMAND ARGS...]
+//                   [--spares N] [-- COMMAND ARGS...]
 //   --socket    Wayland socket name in $XDG_RUNTIME_DIR (default ft-screens-0)
 //   --control   the control socket's abstract name (default ft_screens)
 //   --no-vr     run without SteamVR, for tests next to the running desktop: no panels, no
@@ -22,8 +22,6 @@
 //   --screen    one per screen, in KWin's order (default: 3440x1440@2.4)
 //   --spares    KWin's outputs after the screens: spares for floating windows (ft-floatd
 //               turns them on and sizes them; see docs/floating-windows.md)
-//   --rates     frame rates in Hz for screens you look at, the rest you see, and hidden ones
-//               (0: every display frame; default 0,15,1; see frame_interval)
 //   COMMAND     run with WAYLAND_DISPLAY set to our socket (e.g. the Frametop session)
 // Runs in the dev container (wlroots 0.20); KWin connects from the host.
 #define _GNU_SOURCE
@@ -35,9 +33,7 @@
 #include <string.h>
 #include <stddef.h>
 #include <sys/socket.h>
-#include <sys/resource.h>
 #include <sys/stat.h>
-#include <sys/timerfd.h>
 #include <sys/un.h>
 #include <sys/wait.h>
 #include <time.h>
@@ -65,12 +61,6 @@
 #include "controller-click.h"
 
 #define MAX_SCREENS 24  // screens and spare outputs
-// A screen counts as playing a video while its last VIDEO_COMMITS commits each redrew at
-// least VIDEO_AREA percent of it, at VIDEO_HZ or more; for VIDEO_HOLD ms after that stops.
-#define VIDEO_COMMITS 8
-#define VIDEO_AREA 6
-#define VIDEO_HZ 10
-#define VIDEO_HOLD 1500
 
 struct config {
     int width, height;
@@ -90,13 +80,6 @@ struct screen {
     int buffer_width, buffer_height;
     struct wlr_xdg_toplevel_decoration_v1 *decoration;  // answered on the first commit
     struct wl_listener commit, destroy, decoration_destroy, set_title;
-    // Its frame rate (see tick): when its last frame callback went (ms), and what its recent
-    // commits looked like, to tell a video playing on it.
-    uint32_t frame_sent;
-    uint32_t commit_ms[VIDEO_COMMITS];
-    unsigned commit_at;
-    unsigned big;          // a bit per recent commit: it redrew at least VIDEO_AREA percent
-    uint32_t video_until;  // counts as a video until then (ms)
 };
 
 // Per client buffer: forget its import when it goes away.
@@ -121,14 +104,6 @@ struct server {
     int spares;
     struct wl_list buffers;  // tracked_buffer
     struct wl_event_source *tick;
-    int tick_fd;  // timerfd, at absolute times in step with the display's vsync (see schedule)
-    int64_t period_ns, base_ns, synced_ns;  // the display's frame time, a tick time, last sync
-    double phase_ms;                        // ticks come this long after a vsync
-    // Frame rates in Hz for each attention level (0: every display frame), and a remote
-    // viewer's lease: until then (ms) every screen gets full rate ("watch").
-    int rate[3];
-    uint32_t watch_until;
-    uint32_t typed_ms;  // the last key sent to the desktop: its screen counts as focused
     struct screen *pointer_focus;
     struct ft_controller_click controller_click;
     pid_t child;
@@ -174,25 +149,6 @@ static void track_buffer(struct server *s, struct wlr_buffer *buffer) {
 
 // ---------------------------------------------------------------- screens
 
-// A video (or anything moving over a large area) on a screen you don't look at keeps the
-// full frame rate (see frame_interval): its commits keep redrawing much of it, and keep
-// coming as fast as its rate lets them. A cursor blinking or a spinner turning redraws a
-// small part, and a page changing now and then doesn't keep coming.
-static void note_damage(struct screen *sc, struct wlr_surface *surface, struct wlr_buffer *buffer) {
-    int n = 0;
-    const pixman_box32_t *r = pixman_region32_rectangles(&surface->buffer_damage, &n);
-    int64_t area = 0;
-    for (int i = 0; i < n; ++i) area += (int64_t)(r[i].x2 - r[i].x1) * (r[i].y2 - r[i].y1);
-    const bool big = area * 100 >= (int64_t)buffer->width * buffer->height * VIDEO_AREA;
-    const unsigned all = (1u << VIDEO_COMMITS) - 1;
-    sc->big = ((sc->big << 1) | big) & all;
-    const uint32_t t = now_ms();
-    const uint32_t oldest = sc->commit_ms[sc->commit_at];  // the commit VIDEO_COMMITS ago
-    sc->commit_ms[sc->commit_at] = t;
-    sc->commit_at = (sc->commit_at + 1) % VIDEO_COMMITS;
-    if (sc->big == all && t - oldest <= VIDEO_COMMITS * 1000 / VIDEO_HZ) sc->video_until = t + VIDEO_HOLD;
-}
-
 static void screen_commit(struct wl_listener *l, void *data) {
     struct screen *sc = wl_container_of(l, sc, commit);
     struct wlr_xdg_surface *xdg = sc->toplevel->base;
@@ -216,7 +172,6 @@ static void screen_commit(struct wl_listener *l, void *data) {
     if (!buffer) return;
     sc->frame_pending = true;
     ++sc->commits;
-    note_damage(sc, xdg->surface, buffer);
     sc->buffer_width = buffer->width, sc->buffer_height = buffer->height;
     if (buffer == sc->held) return;
     struct wlr_dmabuf_attributes a;
@@ -426,64 +381,9 @@ static void keys_update(struct server *s) {
            offsetof(struct sockaddr_un, sun_path) + 1 + sizeof name - 1);
 }
 
-// How often a screen gets its frame callback (ms; 0 every tick). KWin draws a screen only
+// Every ~11 ms (90 Hz): SteamVR events, and frame callbacks for screens that committed.
-// after its callback, and its apps wait for theirs, so this is the screen's frame rate:
+static int tick(void *data) {
-// full where you look (ft_vr_screen_attention) and for a video, lower for the rest of what
-// you see, and about once a second for what you don't (hidden, behind you, paused for a VR
-// game). A screen nothing changes on costs nothing at any rate: KWin commits only when
-// something on it moved. A remote viewer ("watch") sees every screen at full rate.
-static uint32_t frame_interval(struct server *s, struct screen *sc, uint32_t t) {
-    if ((int32_t)(s->watch_until - t) > 0) return 0;
-    enum ft_attention a = ft_vr_screen_attention(sc->index);
-    if (a == FT_IN_VIEW && (int32_t)(sc->video_until - t) > 0) a = FT_FOCUSED;
-    if (a != FT_FOCUSED && t - s->typed_ms < 1500 &&
-        s->seat->keyboard_state.focused_surface == sc->toplevel->base->surface)
-        a = FT_FOCUSED;  // typing on it while looking elsewhere
-    const int hz = s->rate[a];
-    return hz > 0 ? 1000 / hz : 0;
-}
-
-static int64_t mono_ns(void) {
-    struct timespec ts;
-    clock_gettime(CLOCK_MONOTONIC, &ts);
-    return ts.tv_sec * 1000000000LL + ts.tv_nsec;
-}
-
-// Ticks come once per display frame, phase_ms after its vsync: KWin gets its frame callbacks
-// early in the frame and has the rest of it to draw before vrcompositor takes the panels.
-// The vsync is read from SteamVR once a second, and the ticks run at absolute times between
-// reads, so they keep their place (a timer set again after each tick, as before, slid
-// through the frame and came about 85 times a second at 90 Hz). Without SteamVR's timing
-// they still come at the display's rate (90 Hz until known). While paused, every 100 ms.
-static void schedule(struct server *s) {
-    const int64_t now = mono_ns();
-    int64_t next;
-    if (ft_vr_paused()) {
-        next = now + 100000000LL;
-    } else {
-        double since, hz;
-        if (now - s->synced_ns >= 1000000000LL) {
-            s->synced_ns = now;
-            if (ft_vr_vsync(&since, &hz)) {
-                const int64_t period = (int64_t)(1e9 / hz);
-                if (llabs(period - s->period_ns) > 100000) wlr_log(WLR_INFO, "display at %.1f Hz", hz);
-                s->period_ns = period;
-                s->base_ns = now - (int64_t)(since * 1e9) + (int64_t)(s->phase_ms * 1e6);
-                while (s->base_ns > now) s->base_ns -= s->period_ns;
-            }
-        }
-        next = s->base_ns + ((now - s->base_ns) / s->period_ns + 1) * s->period_ns;
-    }
-    struct itimerspec its = {.it_value = {.tv_sec = next / 1000000000LL, .tv_nsec = next % 1000000000LL}};
-    timerfd_settime(s->tick_fd, TFD_TIMER_ABSTIME, &its, NULL);
-}
-
-// Each tick: SteamVR events, and frame callbacks for screens that committed and are due.
-static int tick(int fd, uint32_t mask, void *data) {
     struct server *s = data;
-    uint64_t expirations;
-    const ssize_t got = read(fd, &expirations, sizeof expirations);  // clears it; how many doesn't matter
-    (void)got;
     ft_vr_poll(handle_vr_event, s);
     if (++s->ticks % 9 == 0) keys_update(s);
     if (s->kb_close_at && s->ticks >= s->kb_close_at) {
@@ -495,17 +395,14 @@ static int tick(int fd, uint32_t mask, void *data) {
     }
     struct timespec now;
     clock_gettime(CLOCK_MONOTONIC, &now);
-    const uint32_t t = now_ms();
-    // Due within half a tick counts as due, so 15 Hz is every 6th tick at 90 Hz, not 7th.
-    const uint32_t slack = ft_vr_paused() ? 50 : (uint32_t)(s->period_ns / 2000000);
     for (int i = 0; i < MAX_SCREENS; ++i) {
         struct screen *sc = s->screens[i];
-        if (!sc || !sc->frame_pending || t - sc->frame_sent + slack < frame_interval(s, sc, t)) continue;
+        if (sc && sc->frame_pending) {
-        sc->frame_pending = false;
+            sc->frame_pending = false;
-        sc->frame_sent = t;
+            wlr_surface_send_frame_done(sc->toplevel->base->surface, &now);
-        wlr_surface_send_frame_done(sc->toplevel->base->surface, &now);
+        }
     }
-    schedule(s);
+    wl_event_source_timer_update(s->tick, 11);
     return 0;
 }
 
@@ -530,7 +427,6 @@ static bool key_held(const struct wlr_keyboard *kb, uint32_t code) {
 // One key to the focused screen, through the seat's keyboard so its xkb state and
 // modifiers stay right.
 static void send_key(struct server *s, uint32_t code, int pressed) {
-    s->typed_ms = now_ms();
     struct wlr_keyboard_key_event ev = {
         .time_msec = now_ms(), .keycode = code, .update_state = true,
         .state = pressed ? WL_KEYBOARD_KEY_STATE_PRESSED : WL_KEYBOARD_KEY_STATE_RELEASED};
@@ -684,38 +580,6 @@ static int control_readable(int fd, uint32_t mask, void *data) {
                 handle_vr_event(&e, s);
                 snprintf(reply, sizeof reply, "ok");
             }
-        } else if (sscanf(buf, "watch %d", &value) == 1) {
-            // A remote viewer is watching for that many seconds (vnc-bridge.sh renews it).
-            s->watch_until = now_ms() + (uint32_t)(value < 0 ? 0 : value > 60 ? 60 : value) * 1000;
-            snprintf(reply, sizeof reply, "ok");
-        } else if (sscanf(buf, "rates %d %d %d", &index, &w, &h) == 3) {
-            // Frame rates in Hz: focused, in view, hidden (0: every display frame).
-            if (index < 0 || w < 0 || h < 0 || index > 240 || w > 240 || h > 240) {
-                snprintf(reply, sizeof reply, "error rates <focused> <in view> <hidden> (Hz, 0 full)");
-            } else {
-                s->rate[FT_FOCUSED] = index, s->rate[FT_IN_VIEW] = w, s->rate[FT_HIDDEN] = h;
-                wlr_log(WLR_INFO, "frame rates: focused %d, in view %d, hidden %d (0 full)", index, w, h);
-                snprintf(reply, sizeof reply, "ok");
-            }
-        } else if (sscanf(buf, "phase %lf", &scale) == 1) {
-            // Ticks this long after the vsync (ms), for tuning.
-            if (!(scale >= 0 && scale < 20)) snprintf(reply, sizeof reply, "error phase <ms after vsync>");
-            else s->phase_ms = scale, s->synced_ns = 0, snprintf(reply, sizeof reply, "ok");
-        } else if (strcmp(buf, "rates?") == 0) {
-            // "ok <focused> <in view> <hidden> <display Hz> <phase ms> <watched>" and a line per
-            // screen: "<screen> hidden|view|focused <ms between frames> <video 0|1>".
-            static const char *names[] = {"hidden", "view", "focused"};
-            const uint32_t t = now_ms();
-            int at = snprintf(reply, sizeof reply, "ok %d %d %d %.1f %.1f %d", s->rate[FT_FOCUSED],
-                              s->rate[FT_IN_VIEW], s->rate[FT_HIDDEN], 1e9 / s->period_ns, s->phase_ms,
-                              (int32_t)(s->watch_until - t) > 0);
-            for (int i = 0; i < MAX_SCREENS && at < (int)sizeof reply; ++i) {
-                struct screen *sc = s->screens[i];
-                if (!sc) continue;
-                at += snprintf(reply + at, sizeof reply - at, "\n%d %s %u %d", i + 1,
-                               names[ft_vr_screen_attention(i)], frame_interval(s, sc, t),
-                               (int32_t)(sc->video_until - t) > 0);
-            }
         } else if (strcmp(buf, "toplevels") == 0) {
             int n = 0, at = 0;
             for (int i = 0; i < MAX_SCREENS; ++i) n += s->screens[i] != NULL;
@@ -798,9 +662,6 @@ int main(int argc, char **argv) {
     s.controller_click.threshold = 8;
     for (int i = 0; i < MAX_SCREENS; ++i) s.scale[i] = 1;
     s.kb_screen = -1;
-    s.rate[FT_FOCUSED] = 0, s.rate[FT_IN_VIEW] = 15, s.rate[FT_HIDDEN] = 1;
-    s.period_ns = 1000000000LL / 90;
-    s.phase_ms = 1;
     const char *socket_name = "ft-screens-0", *control_name = "ft_screens";
     char **command = NULL;
     s.vr = true;
@@ -813,11 +674,6 @@ int main(int argc, char **argv) {
             s.spares = atoi(argv[++i]);
         } else if (strcmp(argv[i], "--no-vr") == 0) {
             s.vr = false;
-        } else if (strcmp(argv[i], "--rates") == 0 && i + 1 < argc) {
-            if (sscanf(argv[++i], "%d,%d,%d", &s.rate[FT_FOCUSED], &s.rate[FT_IN_VIEW], &s.rate[FT_HIDDEN]) != 3) {
-                fprintf(stderr, "bad --rates %s (want FOCUSED,IN_VIEW,HIDDEN in Hz, 0 full)\n", argv[i]);
-                return 2;
-            }
         } else if (strcmp(argv[i], "--screen") == 0 && i + 1 < argc && s.n_config < MAX_SCREENS) {
             struct config *c = &s.config[s.n_config];
             c->metres = 0;
@@ -833,7 +689,7 @@ int main(int argc, char **argv) {
         } else {
             fprintf(stderr,
                     "usage: %s [--socket NAME] [--control NAME] [--no-vr] [--screen WxH@METRES]... "
-                    "[--spares N] [--rates F,V,H] [-- COMMAND ARGS...]\n",
+                    "[--spares N] [-- COMMAND ARGS...]\n",
                     argv[0]);
             return 2;
         }
@@ -885,10 +741,8 @@ int main(int argc, char **argv) {
     s.relay_fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
     wl_event_loop_add_fd(s.loop, control, WL_EVENT_READABLE, control_readable, &s);
 
-    s.tick_fd = timerfd_create(CLOCK_MONOTONIC, TFD_NONBLOCK | TFD_CLOEXEC);
+    s.tick = wl_event_loop_add_timer(s.loop, tick, &s);
-    s.base_ns = mono_ns();
+    wl_event_source_timer_update(s.tick, 11);
-    s.tick = wl_event_loop_add_fd(s.loop, s.tick_fd, WL_EVENT_READABLE, tick, &s);
-    schedule(&s);
     wl_event_loop_add_signal(s.loop, SIGINT, stop, &s);
     wl_event_loop_add_signal(s.loop, SIGTERM, stop, &s);
     wl_event_loop_add_signal(s.loop, SIGCHLD, child_exited, &s);
@@ -903,18 +757,6 @@ int main(int argc, char **argv) {
         }
     }
 
-    // Ahead of the desktop's programs for the CPU, after the session started (it would
-    // inherit it): KWin waits for our ticks to draw. SteamOS lets users go to nice -8
-    // (RLIMIT_NICE 28) once they raise their soft limit; without that, as before.
-    if (s.vr) {
-        struct rlimit rl;
-        if (getrlimit(RLIMIT_NICE, &rl) == 0 && rl.rlim_cur < rl.rlim_max) {
-            rl.rlim_cur = rl.rlim_max;
-            setrlimit(RLIMIT_NICE, &rl);
-        }
-        if (setpriority(PRIO_PROCESS, 0, -5) != 0) wlr_log(WLR_INFO, "can't raise our priority (nice -5); running at 0");
-    }
-
     wl_display_run(s.display);
 
     wlr_log(WLR_INFO, "stopping");

screens/vr.cpp

@@ -43,11 +43,6 @@
 //     own; also for flatscreen games, which aren't scene apps).
 //   - during a VR game the screens hide unless the dashboard is open (g_inGames, default),
 //     or stay visible over it; the hotkey still shows them.
-//   - paused ("pause on", from the input relay when Frametop pauses for a VR game,
-//     input/game_pause.py): every screen and floating window hides whatever the mode, the
-//     hotkey, or the dashboard says, and compositor.c gives KWin a frame callback once a
-//     second, as for any hidden screen, so KWin and its apps hardly draw. "pause off" undoes
-//     it.
 //   - hand cutouts (handcut.cpp): where ft-hands (hands/) tracks a hand between an eye and a
 //     screen, that eye sees through the screen (to Room View). Only then is the screen
 //     drawn by us, into a side-by-side buffer (one half per eye); otherwise its client
@@ -292,11 +287,6 @@ struct Screen {
     long resizeSent = 0;          // g_tick of the last resize request (they're throttled)
     int resizeW = 0, resizeH = 0; // ...and its size
     std::map<int, Sub> subs;
-    // Attention (UpdateAttention): what ft_vr_screen_attention answers, and until when (ms)
-    // it stays focused or in view after the last reason for it.
-    ft_attention attention = FT_FOCUSED;
-    int64_t inputMs = INT64_MIN / 2;  // the last pointer event on it or its controls
-    int64_t focusUntil = 0, viewUntil = 0;
     double heightMetres() const {
         if (floating && cropW > 0) return metres * cropH / cropW;
         return width > 0 ? metres * height / width : metres * 9 / 16;
@@ -330,7 +320,6 @@ double g_gestureAngle = 20;  // gesture: look within this of the controller
 std::string g_gestureHand = "left";
 Lasers g_lasers = Lasers::OutsideGames;  // when controllers' lasers work the screens (see the top)
 bool g_gameRunning = false;              // a scene app (VR game) is running
-bool g_paused = false;                   // Frametop paused for a VR game: everything hidden (see the top)
 InGames g_inGames = InGames::Hide;       // during a VR game, the always mode acts like the dashboard mode
 
 // ---------------------------------------------------------------- chrome (bar, button, handle)
@@ -745,7 +734,6 @@ void UpdateGame() {
 }
 
 bool ModeVisible() {
-    if (g_paused) return false;
     switch (EffectiveMode()) {
         case Mode::Always: return !g_manual;
         case Mode::Toggle: return g_manual;
@@ -804,7 +792,7 @@ void UpdateVisibility() {
     Mat head;
     const bool haveHead = DevicePose(vr::k_unTrackedDeviceIndex_Hmd, &head);
     for (auto &[i, s] : g_screens) {
-        bool visible = !g_paused && s.shown && (shared || s.drag != Drag::None) && !s.alone;
+        bool visible = s.shown && (shared || s.drag != Drag::None) && !s.alone;
         // A floating window's panel: while a window floats on it, its output is on, and the
         // window isn't minimized (and once it has a crop).
         if (s.floating) visible = visible && s.floatOn && s.outputOn && !s.minimized && s.cropW > 0;
@@ -822,66 +810,6 @@ void UpdateVisibility() {
     }
 }
 
-// Attention, for each screen's frame rate (compositor.c gives KWin frame callbacks at a
-// rate for each level): focused while you look at the screen (within kFocusAngle of where
-// your head points), a laser or the mouse is on it, or it's being carried; in view while
-// any of it is within kViewAngle; hidden otherwise, or while it isn't shown. A level stays
-// for a moment after its reason goes, so a glance away doesn't make it stutter, and goes up
-// at once.
-constexpr double kFocusAngle = 12, kViewAngle = 60;  // degrees
-constexpr int64_t kFocusLinger = 1500, kViewLinger = 500, kInputFocus = 1500;  // ms
-
-bool RayOnPlane(const Mat &p, const Mat &d, double *x, double *y);
-
-int64_t NowMs() {
-    return std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now().time_since_epoch()).count();
-}
-
-// The smallest angle between where the head points and the screen: to the point of its
-// rectangle nearest where the head's ray meets its plane, its centre, and its corners. A
-// curved screen counts as flat; the angles hardly differ.
-double AngleToScreen(const Screen &s, const Mat &p, const Mat &head) {
-    const double hw = s.metres / 2, hh = s.heightMetres() / 2;
-    double f[3];
-    Column(head, 2, f);  // the head's +Z points backward
-    auto angleTo = [&](double u, double v) {
-        double to[3];
-        for (int i = 0; i < 3; ++i) to[i] = p.m[i][3] + u * p.m[i][0] + v * p.m[i][1] - head.m[i][3];
-        const double len = std::sqrt(Dot3(to, to)) + 1e-9;
-        return std::acos(std::clamp(-Dot3(f, to) / len, -1.0, 1.0)) * 180 / M_PI;
-    };
-    double best = 180, x, y;
-    if (RayOnPlane(p, head, &x, &y)) best = angleTo(std::clamp(x, -hw, hw), std::clamp(y, -hh, hh));
-    for (double u : {-hw, 0.0, hw})
-        for (double v : {-hh, 0.0, hh}) best = std::min(best, angleTo(u, v));
-    return best;
-}
-
-void UpdateAttention() {
-    const int64_t now = NowMs();
-    Mat head;
-    const bool haveHead = DevicePose(vr::k_unTrackedDeviceIndex_Hmd, &head);
-    for (auto &[i, s] : g_screens) {
-        if (!s.visible) {
-            s.attention = FT_HIDDEN;
-            s.focusUntil = s.viewUntil = 0;
-            continue;
-        }
-        bool focus = s.drag != Drag::None || now - s.inputMs < kInputFocus, view = focus;
-        Mat p;
-        if (!haveHead) {
-            view = true;  // nothing to go by
-        } else if (ScreenPose(s, &p)) {
-            const double a = AngleToScreen(s, p, head);
-            focus = focus || a <= kFocusAngle;
-            view = view || a <= kViewAngle;
-        }
-        if (focus) s.focusUntil = now + kFocusLinger;
-        if (view) s.viewUntil = now + kViewLinger;
-        s.attention = now < s.focusUntil ? FT_FOCUSED : now < s.viewUntil ? FT_IN_VIEW : FT_HIDDEN;
-    }
-}
-
 
 // Controllers' lasers on the screens (see the top): the flag follows the mode and whether a
 // VR game runs.
@@ -966,19 +894,6 @@ void SendFloat(const std::string &msg) {
         std::printf("to ft-floatd: %s\n", msg.c_str());
 }
 
-// A new panel: the pointer helper reads SteamVR's list of panels only every 20 s, so it's told
-// at once ("overlay <key>"), or the mouse couldn't click a menu until then.
-void AnnounceOverlay(const char *key) {
-    static const int fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
-    sockaddr_un addr{};
-    addr.sun_family = AF_UNIX;
-    const char name[] = "ft_pointer_helper";
-    std::memcpy(addr.sun_path + 1, name, sizeof name - 1);
-    const std::string msg = std::string("overlay ") + key;
-    sendto(fd, msg.data(), msg.size(), MSG_DONTWAIT, reinterpret_cast<sockaddr *>(&addr),
-           socklen_t(offsetof(sockaddr_un, sun_path) + 1 + sizeof name - 1));
-}
-
 // Where a device's ray meets the screen's plane, in the screen's x (right) and y (up),
 // metres from its centre.
 bool RayOnPlane(const Mat &p, const Mat &d, double *x, double *y) {
@@ -1326,7 +1241,6 @@ void SetSub(Screen &s, int index, int k, int x, int y, int w, int h) {
         vr::VROverlay()->SetOverlayFlag(sub.overlay, vr::VROverlayFlags_SendVRDiscreteScrollEvents, true);
         vr::VROverlay()->SetOverlayFlag(sub.overlay, vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible, s.lasers);
         vr::VROverlay()->SetOverlaySortOrder(sub.overlay, 5);
-        AnnounceOverlay(key);
         it = s.subs.emplace(k, sub).first;
         if (s.shown) {
             auto imp = g_imports.find(s.shown);
@@ -1587,7 +1501,7 @@ void UpdateSteamInFront() {
         g_keyboardAside = true;
     } else if (!front && g_keyboardAside) {
         g_keyboardAside = false;
-        if (keyboard::Show(g_asidePose)) AnnounceOverlay("frametop.keyboard");
+        keyboard::Show(g_asidePose);
     }
 }
 
@@ -1655,25 +1569,6 @@ int ft_vr_modifiers(uint32_t format, uint64_t *out, int max) {
 }
 
 bool ft_vr_screens_shown(void) { return g_vr && ModeVisible(); }
-bool ft_vr_paused(void) { return g_paused; }
-
-enum ft_attention ft_vr_screen_attention(int index) {
-    const auto it = g_screens.find(index);
-    return g_vr && it != g_screens.end() ? it->second.attention : FT_FOCUSED;
-}
-
-bool ft_vr_vsync(double *since, double *hz) {
-    if (!g_vr) return false;
-    float s = 0;
-    uint64_t frame = 0;
-    if (!vr::VRSystem()->GetTimeSinceLastVsync(&s, &frame)) return false;
-    vr::ETrackedPropertyError err = vr::TrackedProp_Success;
-    const float f = vr::VRSystem()->GetFloatTrackedDeviceProperty(vr::k_unTrackedDeviceIndex_Hmd,
-                                                                  vr::Prop_DisplayFrequency_Float, &err);
-    if (err != vr::TrackedProp_Success || !(f >= 30 && f <= 240) || !(s >= 0 && s < 1)) return false;
-    *since = s, *hz = f;
-    return true;
-}
 
 }  // extern "C"
 
@@ -1833,7 +1728,6 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) {
         auto panelEvent = [&](const vr::VREvent_t &ev, bool sub) {
             ft_event e{};
             e.screen = index;
-            if (ev.eventType != vr::VREvent_FocusLeave) s.inputMs = NowMs();
             auto at = [&] { s.ToBuffer(ev.data.mouse.x, ev.data.mouse.y, &e.x, &e.y); };
             switch (ev.eventType) {
                 case vr::VREvent_MouseMove:
@@ -1886,7 +1780,6 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) {
         // The controls light up under a laser.
         auto hover = [&](int k) {
             const bool on = ev.eventType == vr::VREvent_MouseMove || ev.eventType == vr::VREvent_FocusEnter;
-            if (on) s.inputMs = NowMs();
             if (!on && ev.eventType != vr::VREvent_FocusLeave) return;
             if (s.hover[k] != on) s.hover[k] = on, ApplyAlpha(s);
         };
@@ -1999,7 +1892,6 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) {
     UpdateGame();
     UpdateArrange();
     UpdateVisibility();
-    UpdateAttention();
     UpdateLasers();
     UpdateControls();
     UpdateGuides();
@@ -2034,9 +1926,7 @@ bool ft_vr_keyboard_show(int index) {
         std::printf("keyboard: waiting for Steam to close\n");
         return true;
     }
-    if (!keyboard::Show(FacingPose(at, head))) return false;
+    return keyboard::Show(FacingPose(at, head));
-    AnnounceOverlay("frametop.keyboard");
-    return true;
 }
 
 void ft_vr_keyboard_hide(void) {
@@ -2214,17 +2104,6 @@ void ft_vr_command(const char *cmd, char *reply, int size) {
         g_manual = always ? !want : want;
         UpdateVisibility();
         std::snprintf(reply, size, "ok %s", want ? "shown" : "hidden");
-    } else if (std::sscanf(cmd, "pause %15s", word) == 1) {
-        if (!std::strcmp(word, "on") || !std::strcmp(word, "off")) {
-            const bool on = !std::strcmp(word, "on");
-            if (on != g_paused)
-                std::printf("%s\n", on ? "paused for a VR game: everything hidden, KWin slowed down" : "resumed");
-            g_paused = on;
-            UpdateVisibility();
-        } else if (std::strcmp(word, "state") != 0) {
-            return (void)std::snprintf(reply, size, "error pause on|off|state");
-        }
-        std::snprintf(reply, size, "ok %s", g_paused ? "paused" : "running");
     } else if (std::sscanf(cmd, "ingames %15s", word) == 1) {
         if (!std::strcmp(word, "hide")) g_inGames = InGames::Hide;
         else if (!std::strcmp(word, "visible")) g_inGames = InGames::Visible;

screens/vr.h

@@ -36,16 +36,6 @@ void ft_vr_shutdown(void);
 int ft_vr_modifiers(uint32_t format, uint64_t *out, int max);
 // The screens are showing (by the visibility mode; not counting a wrist-pinned screen).
 bool ft_vr_screens_shown(void);
-// Frametop is paused for a VR game ("pause on"): everything is hidden, and KWin slows down.
-bool ft_vr_paused(void);
-// How much of a screen you see, for its frame rate (compositor.c): hidden (or out of view),
-// in view, or focused (you look at it, or a laser or the mouse is on it). Focused without
-// SteamVR (--no-vr) or for an unknown screen.
-enum ft_attention { FT_HIDDEN, FT_IN_VIEW, FT_FOCUSED };
-enum ft_attention ft_vr_screen_attention(int index);
-// The display's refresh rate and the time since its last vsync, in seconds. False without
-// them (no SteamVR, or the headset isn't reporting).
-bool ft_vr_vsync(double *since, double *hz);
 // A panel for screen `index`, width in metres, placed in a row in front of the head.
 void ft_vr_screen_create(int index, double metres, int count);
 void ft_vr_screen_destroy(int index);

scripts/report.sh

@@ -25,7 +25,6 @@ for u in frametop-input-relay frametop-pointer frametop-power; do
   echo "$u: $(systemctl --user is-enabled $u 2>/dev/null) / $(systemctl --user is-active $u 2>/dev/null)"
 done
 echo "desktop: $(pgrep -x ft-screens >/dev/null && echo running || echo 'not running'), plasmashell: $(pgrep -c plasmashell || true)"
-echo "paused for VR games: $(cat /run/user/$(id -u)/frametop-pause.json 2>/dev/null || echo 'no (never paused since boot)')"
 LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrpathreg show 2>/dev/null | sed -n '/xternal/,$p'
 
 section "What Frametop needs from SteamOS (scripts/update-check.py)"

scripts/update-check.py

@@ -12,18 +12,15 @@ This checks each one, and compares the versions with the last ones recorded as w
 From a PC: ssh frame python3 - [--mark-good] < scripts/update-check.py
 
 It only reads. It connects to SteamVR as a background app (which never starts SteamVR),
-runs `vrcmd --overlays` as the pointer helper does, maps the eye tracker's shared memory
+runs `vrcmd --overlays` as the pointer helper does, and maps the eye tracker's shared
-read-only, and follows a controller on vrserver's web socket for a moment. It never starts,
+memory read-only. It never starts, stops, or restarts anything.
-stops, or restarts anything.
 """
-import base64
 import json
 import math
 import mmap
 import os
 import platform
 import re
-import socket
 import struct
 import subprocess
 import sys
@@ -37,8 +34,6 @@ LAUNCHER = os.path.join(HOME, ".local/share/applications/deckard-nested-desktop.
 VRPATHS = os.path.join(HOME, ".config/openvr/openvrpaths.vrpath")
 BACKLIGHT = "/sys/class/backlight/ae94000.dsi.0/brightness"  # as in power/ft-powerd.cpp
 EYE_MMAP = "/dev/shm/eye-server.mmap"
-VRSERVER = "http://127.0.0.1:27062"  # its web socket: input/vrws.py
-PAUSE_STATE = f"/run/user/{os.getuid()}/frametop-pause.json"  # input/game_pause.py
 HOST_GLIBC = (2, 39)  # the newest the pointer driver may need (pointer/driver/build.sh)
 
 # Packages in the OS image that Frametop depends on, and what to try by hand when one changes.
@@ -46,9 +41,8 @@ PACKAGES = {
     "deckard-steamvr-rel": "the 3D mouse on the dashboard, on SteamVR Settings, and on a SteamVR "
                            "window's grab bar; picking up a controller; mapped controller buttons; gaze",
     "kwin": "clicks near the far edge of a screen whose scale isn't 1; every screen comes back "
-            "after a desktop restart; floating a window; no blur behind the taskbar's menus",
+            "after a desktop restart; floating a window",
-    "plasma-workspace": "the taskbar and panels after a desktop restart; no DiscoverNotifier or "
+    "plasma-workspace": "the taskbar and panels after a desktop restart",
-                        "ibus-daemon inside the desktop",
     "gamescope": "the headset's volume buttons with nothing focused; typing goes where you last clicked",
     "bluez": "a Bluetooth mouse reconnecting after it sleeps",
 }
@@ -177,21 +171,14 @@ def check_host():
 
     try:
         with open("/usr/bin/kwin_wayland", "rb") as f:
-            kwin = f.read()
+            # Qt keeps the option name as a UTF-16 string literal.
+            output_count = "output-count".encode("utf-16-le") in f.read()
     except OSError:
-        kwin = b""
+        output_count = False
-    # Qt keeps the option name as a UTF-16 string literal.
+    if output_count:
-    if "output-count".encode("utf-16-le") in kwin:
         report("ok", "KWin", "has --output-count (one output per screen)")
     else:
         report("FAIL", "KWin", "no --output-count option: the desktop gets one screen at most")
-    # The session turns these built-in effects off by id (blurEnabled, contrastEnabled in kwinrc).
-    gone = [e for e in ("blur", "contrast") if b"KWin::%s_factory" % e.encode() not in kwin]
-    if not gone:
-        report("ok", "KWin effects", "blur and contrast, which the desktop turns off, keep their ids")
-    else:
-        report("warn", "KWin effects", f"no built-in {', '.join(gone)} effect: renamed? The desktop's "
-               "kwinrc may no longer turn it off (session/frametop-session.sh)")
 
     if systemctl("cat", "steamvr.service"):
         report("ok", "steamvr.service", "Frametop's services start and stop with it")
@@ -298,11 +285,11 @@ def check_openvr():
 def check_overlays(desktop_up):
     code, out = run(f"{STEAMVR_BIN}/vrcmd", "--overlays",
                     env=dict(os.environ, LD_LIBRARY_PATH=STEAMVR_BIN))
-    # The format ft-pointer.cpp and ft_layout.py parse: 'key' -- 'name', WxH visible VROverlayType_...
+    # The format the pointer helper (pointer.h) and ft_layout.py parse: 'key' -- 'name', WxH visible VROverlayType_...
     keys = re.findall(r"^'([^']+)' -- '.*VROverlayType_", out, re.M)
     if not keys:
         report("FAIL", "vrcmd --overlays", "lists no overlays in the expected format, so the 3D mouse can't "
-               "find panels (pointer/helper/ft-pointer.cpp and layout/ft_layout.py parse it)")
+               "find panels (pointer/helper/pointer.h and layout/ft_layout.py parse it)")
     elif desktop_up and not any(re.fullmatch(r"frametop\.screen\.\d+", k) for k in keys):
         report("FAIL", "vrcmd --overlays", f"lists {len(keys)} overlays, but none of the desktop's screens")
     else:
@@ -310,19 +297,11 @@ def check_overlays(desktop_up):
 
 
 def check_services(sockets, desktop_up):
-    try:
-        with open(PAUSE_STATE) as f:
-            pause = json.load(f)
-        paused = pause["stopped"]["units"] if pause.get("paused") else []
-    except (OSError, ValueError, KeyError, TypeError):
-        paused = []
     for unit, socket in UNITS.items():
         if systemctl("is-enabled", unit) != "enabled":
             continue
         state = systemctl("is-active", unit)
-        if state != "active" and unit + ".service" in paused:
+        if state != "active":
-            report("skip", unit, "stopped while Frametop is paused for a VR game (input/ft-pause off resumes it)")
-        elif state != "active":
             report("FAIL", unit, f"{state}; see journalctl --user -u {unit}")
         elif socket and socket not in sockets:
             report("FAIL", unit, f"runs, but hasn't opened {socket}")
@@ -366,69 +345,6 @@ def check_steam_ui():
         report("warn", "Steam menu shortcut", f"{detail}; a Meta tap won't open the Steam menu (steam/ft-steam)")
 
 
-def check_vr_socket():
-    """vrserver's web socket, which the pause gesture reads the controllers from (input/vrws.py,
-    input/game_pause.py). It's undocumented: SteamVR's controller binding page uses it."""
-    label, headers = "vrserver web socket", {"Referer": f"{VRSERVER}/dashboard/controllerbinding.html"}
-    try:
-        import urllib.request
-        with urllib.request.urlopen(urllib.request.Request(f"{VRSERVER}/input/getstate.json", headers=headers),
-                                    timeout=3) as r:
-            devices = json.load(r).get("devices", [])
-    except (OSError, ValueError) as e:
-        report("FAIL", label, f"/input/getstate.json didn't answer ({e}); the pause gesture can't read the controllers")
-        return
-    found = [d for d in devices if d.get("controller_type") == "frame_controller" and d.get("root_path")
-             and d.get("side") in ("left", "right")]
-    if not found:
-        report("skip", label, "no Frame controller is on, so it wasn't checked")
-        return
-    if not all(any(c.get("path") == "/input/thumbstick/click" for c in d.get("components", [])) for d in found):
-        report("warn", label, "a controller lists no /input/thumbstick/click; the default pause gesture needs it")
-    data = b""
-    try:
-        with socket.create_connection(("127.0.0.1", 27062), timeout=3) as s:
-            key = base64.b64encode(os.urandom(16)).decode()
-            s.sendall((f"GET / HTTP/1.1\r\nHost: 127.0.0.1:27062\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n"
-                       f"Sec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\nOrigin: {VRSERVER}\r\n"
-                       f"Referer: {headers['Referer']}\r\n\r\n").encode())
-            while b"\r\n\r\n" not in data:
-                chunk = s.recv(4096)
-                if not chunk:
-                    break
-                data += chunk
-            if b" 101 " not in data.split(b"\r\n", 1)[0] + b" ":
-                report("FAIL", label, "vrserver refused the web socket; the pause gesture can't read the controllers")
-                return
-            mailbox = f"frametop_check_{os.getpid()}"
-            # Every controller: one lying still (or asleep) may send nothing at all.
-            for text in [f"mailbox_open {mailbox}"] + ["mailbox_send input_server " + json.dumps(
-                    {"type": "request_input_state_updates", "device_path": d["root_path"], "returnAddress": mailbox})
-                    for d in found]:
-                payload, mask = text.encode(), os.urandom(4)
-                size = bytes([0x80 | len(payload)]) if len(payload) < 126 else bytes([0x80 | 126]) + struct.pack(">H", len(payload))
-                s.sendall(b"\x81" + size + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(payload)))
-            deadline = time.monotonic() + 3
-            try:
-                while b"update_component_states" not in data and time.monotonic() < deadline:
-                    chunk = s.recv(65536)
-                    if not chunk:
-                        break
-                    data += chunk
-            except socket.timeout:
-                report("skip", label, "it took the subscription, but no controller sent anything in 3 s "
-                       "(lying still or asleep?): pick one up and check again")
-                return
-    except OSError as e:
-        report("FAIL", label, f"{e}; the pause gesture can't read the controllers")
-        return
-    if b"update_component_states" in data and b"/click" in data:
-        report("ok", label, "it still streams the controllers' buttons, which the pause gesture reads")
-    else:
-        report("FAIL", label, "no button states came, so the pause gesture can't read the controllers "
-               "(input/vrws.py: the message format changed?)")
-
-
 def check_eye_tracker():
     gaze = systemctl("is-enabled", "frametop-gaze") == "enabled"
     try:
@@ -478,7 +394,6 @@ def main():
         check_overlays(desktop_up)
         check_services(sockets, desktop_up)
         check_steam_ui()
-        check_vr_socket()
         check_eye_tracker()
 
     if args == ["--mark-good"]:

session/frametop-session.sh

@@ -234,40 +234,6 @@ elif [ "$(kreadconfig6 --file "$kwinrc" --group org.kde.kdecoration2 --key theme
   kwriteconfig6 --file "$kwinrc" --group org.kde.kdecoration2 --key theme --delete
 fi
 
-# KWin draws through zink on the headset's GPU, which vrcompositor needs, so the blur and
-# background contrast behind panels and menus and the window animations cost frames for
-# little. They're off by default in this desktop. Each is written once, before KWin first
-# reads it, and only if this desktop's config doesn't have it yet; the marker keeps it
-# from coming back, since System Settings deletes a setting put back to KDE's default.
-# docs/reference.md says how to turn them on again.
-default() {  # file group key value
-  [ -n "$(kreadconfig6 --file "$1" --group "$2" --key "$3")" ] || kwriteconfig6 --file "$1" --group "$2" --key "$3" "$4"
-}
-frametoprc=$XDG_CONFIG_HOME/frametoprc
-if [ "$(kreadconfig6 --file "$frametoprc" --group Defaults --key effects)" != 1 ]; then
-  default "$kwinrc" Plugins blurEnabled false
-  default "$kwinrc" Plugins contrastEnabled false
-  default "$XDG_CONFIG_HOME/kdeglobals" KDE AnimationDurationFactor 0
-  kwriteconfig6 --file "$frametoprc" --group Defaults --key effects 1
-fi
-
-# System autostart entries this desktop doesn't need, hidden for it alone by a copy with
-# Hidden=true in its own autostart folder, once and unless there's a file of that name
-# already. Discover's update notifier starts Discover itself to check for updates (about
-# 600 MB and a share of a core, with Flatpak's helper and AppStream behind it); updates
-# come with SteamOS and from Discover in the stock desktop. IBus can't reach the
-# desktop's apps: KWin's input method is ft-textinput, and the session drops the
-# variables that point apps at IBus or XIM. Deleting the copy brings an entry back.
-if [ "$(kreadconfig6 --file "$frametoprc" --group Defaults --key autostart)" != 1 ]; then
-  for entry in org.kde.discover.notifier ibus; do
-    src=/etc/xdg/autostart/$entry.desktop dst=$XDG_CONFIG_HOME/autostart/$entry.desktop
-    [ -r "$src" ] && [ ! -e "$dst" ] || continue
-    mkdir -p "$(dirname "$dst")"
-    sed '/^\[Desktop Entry\]$/a Hidden=true' "$src" > "$dst"
-  done
-  kwriteconfig6 --file "$frametoprc" --group Defaults --key autostart 1
-fi
-
 # Profiles reopen apps (docs/profiles.md), so Plasma's own session restore stays off here;
 # with both, apps would open twice.
 kwriteconfig6 --file "$XDG_CONFIG_HOME/ksmserverrc" --group General --key loginMode emptySession

session/vnc-bridge.sh

@@ -11,16 +11,6 @@
 # screen placed lower or further right. Instead the VNC screen is the primary's size, and
 # the workspace-sized RDP window inside it is shifted so the primary fills it. The pointer
 # maps 1:1. When the layout changes, the VNC screen resizes and the RDP client reconnects.
-#
-# The RDP client runs only while someone watches. While connected, krdp captures and
-# H.264-encodes every redraw in software, about 60% of a core, with FreeRDP and Xvnc adding
-# about 30% more, even with no VNC viewer. krdp starts its capture per RDP connection and
-# stops it when the connection closes, so it idles without one. So FreeRDP starts when a
-# VNC client connects (the screen is black for the few seconds that takes) and stops
-# VNC_IDLE_SEC (45) seconds after the last one leaves. Xvnc has no hook for clients, so ss
-# counts them: whenever Xvnc writes to its log (it logs each connection), every second
-# while FreeRDP runs, and every 5 seconds otherwise. The log only wakes this script up;
-# what it says doesn't matter.
 set -eu
 
 here=$(dirname "$(readlink -f "$0")")
@@ -67,51 +57,16 @@ stop_rdp() { pkill -f "[x]freerdp /v:127.0.0.1:$rdp_port " 2>/dev/null || true;
 trap 'stop_rdp; pkill -f "[X]vnc $display " 2>/dev/null || true' EXIT
 
 box bash -c 'vncpasswd -f < "$1/vnc-password" > "$1/vnc-passwd.bin" && chmod 600 "$1/vnc-passwd.bin"' - "$creds"
-exec {xlog}< <(box Xvnc "$display" -geometry "${w}x${h}" -depth 24 \
+box Xvnc "$display" -geometry "${w}x${h}" -depth 24 \
   -interface "$addr" -rfbport "$vnc_port" \
   -SecurityTypes VncAuth -PasswordFile "$creds/vnc-passwd.bin" \
-  -AlwaysShared -desktop "Steam Frame (Frametop)" 2>&1)
+  -AlwaysShared -desktop "Steam Frame (Frametop)" &
 xvnc=$!
+sleep 2
 
-# Wait up to $1 seconds, less if Xvnc logs something; its lines go on to this log.
+# Keep an RDP connection open inside the VNC screen. Reconnect if it drops or the layout changes.
-nap() {
-  local line rc=0
-  IFS= read -rt "$1" -u "$xlog" line || rc=$?
-  if [ $rc -eq 0 ]; then
-    printf '%s\n' "$line"
-    while IFS= read -rt 0.1 -u "$xlog" line; do printf '%s\n' "$line"; done
-  elif [ $rc -le 128 ]; then
-    sleep 1  # Xvnc's output closed: it's exiting
-  fi
-  return 0
-}
-nap 2
-
-# A connected VNC client: an established TCP connection to Xvnc's port. Any connection
-# counts, authenticated or not; it's on the tailnet only.
-clients() { [ -n "$(ss -Htn state established "( sport = :$vnc_port )" 2>/dev/null)" ]; }
-
-# ft-screens drops screens you aren't looking at to a low frame rate, and krdp would
-# stream that. "watch SECONDS" asks it for full rate on every screen for that long: sent
-# when a client connects and renewed every few seconds while one stays, so it lapses by
-# itself if this script dies. An older ft-screens just answers that it doesn't know it.
-watch() {
-  if command -v socat >/dev/null; then
-    printf 'watch 15' | socat -u - ABSTRACT-SENDTO:ft_screens 2>/dev/null
-  else
-    python3 -c 'import socket; socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM).sendto(b"watch 15", "\0ft_screens")' 2>/dev/null
-  fi || true
-}
-
-# What remote-view depends on: KWin's saved outputs (positions, scales, primary) and
-# Frametop's layout (screen sizes). It's read again only after one of these changes, and
-# once a minute in case a change didn't touch them, and only while FreeRDP runs.
-layout_files=("$HOME/.config/frametop/kwinoutputconfig.json" "$HOME/.config/frametop-layout.json")
-stamp() { stat -c %y "${layout_files[@]}" 2>/dev/null || true; }
-
-# Start FreeRDP inside the VNC screen, sized and shifted for $v.
 # /cert:ignore is fine here: the connection never leaves this host.
-start_rdp() {
+while kill -0 $xvnc 2>/dev/null; do
   read -r x y w h ww wh <<< "$v"
   box env DISPLAY=$display bash -c '
     size=$1 x=$2 y=$3 ww=$4 wh=$5 creds=$6 rdp_port=$7
@@ -125,7 +80,6 @@ start_rdp() {
     rdp=$!
     # FreeRDP takes no negative position, so move its window once it is up.
     for _ in $(seq 60); do
-      kill -0 $rdp 2>/dev/null || break
       win=$(xdotool search --class xfreerdp 2>/dev/null | tail -1)
       [ -n "$win" ] && break
       sleep 0.5
@@ -134,58 +88,14 @@ start_rdp() {
     wait $rdp
   ' vnc-rdp "${w}x$h" "$x" "$y" "$ww" "$wh" "$creds" "$rdp_port" || true &
   rdp=$!
-}
+  while kill -0 $rdp 2>/dev/null; do
-
+    sleep 5
-idle_sec=${VNC_IDLE_SEC:-45}
+    now=$(view) || continue
-rdp=           # FreeRDP's job while it runs
+    [ -n "$now" ] && [ "$now" != "$v" ] || continue
-seen=0         # when a client was last seen ($SECONDS)
+    echo "layout changed: $v -> $now"
-watched=-99    # when "watch" was last sent
+    v=$now
-stamp_v=       # stamp() when $v was read
+    stop_rdp
-recheck=0      # read the layout every 2 seconds until then
+  done
-next_view=0    # next time to read it
+  wait $rdp 2>/dev/null || true
-while kill -0 $xvnc 2>/dev/null; do
+  sleep 2
-  if clients; then
-    if [ $((SECONDS - watched)) -ge 5 ]; then watch; watched=$SECONDS; fi
-    seen=$SECONDS
-    if [ -z "$rdp" ]; then
-      echo "VNC client connected, starting the RDP client"
-      now=$(view) && [ -n "$now" ] && v=$now
-      stamp_v=$(stamp) recheck=0 next_view=$((SECONDS + 60))
-      start_rdp
-    fi
-  elif [ "$seen" -ne 0 ]; then
-    watched=-99
-    if [ $((SECONDS - seen)) -ge "$idle_sec" ]; then
-      seen=0
-      if [ -n "$rdp" ]; then
-        echo "no VNC client for ${idle_sec}s, stopping the RDP client"
-        stop_rdp
-        wait "$rdp" 2>/dev/null || true
-        rdp=
-      fi
-    fi
-  fi
-  if [ -n "$rdp" ] && ! kill -0 "$rdp" 2>/dev/null; then
-    # It dropped, or the layout changed: reconnect next round if a client is still there.
-    wait "$rdp" 2>/dev/null || true
-    rdp=
-    nap 2
-    continue
-  fi
-  if [ -n "$rdp" ]; then
-    # A layout change shows up in KWin's outputs a few seconds after the files change.
-    s=$(stamp)
-    [ "$s" = "$stamp_v" ] || { stamp_v=$s; recheck=$((SECONDS + 10)) next_view=$SECONDS; }
-    if [ "$SECONDS" -ge "$next_view" ]; then
-      next_view=$((SECONDS + 60))
-      [ "$SECONDS" -ge "$recheck" ] || next_view=$((SECONDS + 2))
-      now=$(view) || now=
-      if [ -n "$now" ] && [ "$now" != "$v" ]; then
-        echo "layout changed: $v -> $now"
-        v=$now
-        stop_rdp
-      fi
-    fi
-  fi
-  if [ -n "$rdp" ] || [ "$seen" -ne 0 ]; then nap 1; else nap 5; fi
 done