Add head follow: the pointer comes along when you turn your head

Off by default. With POINTER_FOLLOW=1 (a switch on the Pointer page of
Frametop Input Settings, or a mouse button mapped to Head follow on/off),
the cursor rides on a reference direction leashed POINTER_LEASH_DEG (10)
from where you face. Within the leash it stays put in the room; past it,
it turns with your head and keeps its offset. A leash of 0 locks it to
your view. Head roll is ignored, the cursor stays within 40 degrees of
the reference, and it holds still in the room while the left button is
down so the head can't nudge a click or a drag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

README.md

@@ -53,7 +53,7 @@ If you work in the desktop for long stretches, stop Steam from putting the heads
 | Meta+Shift+H in the desktop | Hides or shows all screens (also in the menu as Hide/Show Screens, and mappable). The Visibility & wrist tab of Frametop Display Settings can instead show them only with the dashboard open, or while you look at your wrist |
 | Play a VR game | The screens hide and your controllers stay in the game. Open the SteamVR dashboard, or press Meta+Shift+H, to see and use them. To keep them visible over games, change During VR games on the Visibility & wrist tab; the controllers still stay in the game, and you use the screens with the mouse or the dashboard |
 
-You can map the mouse's extra buttons to actions such as Toggle SteamVR dashboard or Recenter pointer on the Buttons page of Frametop Input Settings. Pointer speed, dot size, and the rest are on its Pointer page and take effect immediately.
+You can map the mouse's extra buttons to actions such as Toggle SteamVR dashboard, Recenter pointer, or Head follow on/off on the Buttons page of Frametop Input Settings. Pointer speed, dot size, and the rest are on its Pointer page and take effect immediately. Head follow, off by default, makes the pointer come along when you turn your head: it stays put until your head has turned past the leash angle, and a leash of 0 keeps it fixed in your view.
 
 Restarting the desktop (Restart desktop in Frametop Display Settings) closes its windows, but background work you started in it, such as servers, tmux sessions, or builds, keeps running.
 

docs/design.md

@@ -80,6 +80,8 @@ A few overlays need special handling:
 - Just off a panel, the cursor stays on that panel's plane within `POINTER_EDGE_REACH`, so resize margins and window controls just outside the panel are reachable.
 - While the left button is held, the cursor keeps the distance it had at the press and stops re-testing collisions, so dragging past a panel's edge doesn't make it jump.
 
+Head follow is off by default. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction that's on a leash `POINTER_LEASH_DEG` from where you're facing. Turn your head within the leash and the cursor stays where it is in the room. Turn further and the reference is dragged along, and the cursor turns with it, keeping its offset. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around, and the cursor can't be pushed more than 40 degrees from the reference, so it stays in view. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping.
+
 Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place.
 
 `dashboard.laserRayWidthScale` controls the beam's width, but SteamVR only applies a change from its own settings screen or at restart, so it can't be switched per device while running.

docs/reference.md

@@ -98,15 +98,15 @@ pointer/helper/run.sh status | log | restart
 pointer/driver/install.sh probe     # devices, hand roles, who owns the dashboard pointer
 ```
 
-The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, and `POINTER_LASER_WIDTH`. The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper.
+The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, `POINTER_LASER_WIDTH`, and the head follow settings `POINTER_FOLLOW` and `POINTER_LEASH_DEG`. The example config explains each. Frametop Input Settings changes them live; after editing the file by hand, restart the relay or the helper.
 
 ## Frametop Input Settings
 
 A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has four pages:
 
 - Devices lists every USB and Bluetooth mouse and keyboard, with a light that flashes when the device is used. Each device gets a role: 3D pointer (grabbed, drives the pointer; the default for anything with a mouse), Pass through (grabbed only while typing goes to the desktop; the default for keyboards, where a Meta tap toggles the dashboard if `META_DASHBOARD=1` is in `~/.config/frametop.conf`), or Ignore. A device is identified by its Bluetooth address, or its USB ids and name, so all of its input nodes share one role. Forget drops everything saved for a device.
-- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, faster or slower, pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
+- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, faster or slower, pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
-- Pointer has sliders for the pointer settings, which apply immediately, and a Recenter button.
+- Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button.
 - Bluetooth lists paired devices and has Apply Bluetooth fixes, which runs `/etc/steamframe/bt-fixups.sh` through `pkexec`. Pair new devices in Steam.
 
 Device rules are saved in `~/.config/frametop-input.json`. `input-settings/install.sh` installs the menu entry. Its launcher hands podman the real `XDG_RUNTIME_DIR` and user bus and gives the app the session's Wayland socket, because the desktop session runs on a private D-Bus and podman fails on it.

input-settings/ft_input_settings.py

@@ -37,7 +37,8 @@ DEFAULT_BUTTONS = {0x110: "left", 0x111: "right", 0x112: "middle", 0x113: "back"
 ACTION_LABELS = {
     "left": "Left click", "right": "Right click", "middle": "Middle click", "back": "Back",
     "scroll_up": "Scroll up", "scroll_down": "Scroll down", "dashboard": "Toggle SteamVR dashboard",
-    "recenter": "Recenter pointer", "pointer_toggle": "Pointer on/off", "sens_up": "Faster pointer",
+    "recenter": "Recenter pointer", "pointer_toggle": "Pointer on/off",
+    "follow_toggle": "Head follow on/off", "sens_up": "Faster pointer",
     "sens_down": "Slower pointer", "layout_reset": "Reset desktop screen layout",
     "screens_toggle": "Hide/show desktop screens", "key": "Pass through as key",
     "none": "Do nothing",
@@ -52,6 +53,7 @@ POINTER_SETTINGS = [
     ("POINTER_ORIGIN_MARGIN", "Room for small controls", 0.15, 0.03, 0.5, 0.01, "m"),
     ("POINTER_SCENE_RADIUS", "Dock / window-control reach", 0.5, 0.1, 1.5, 0.05, "m"),
     ("POINTER_EDGE_REACH", "Panel edge reach", 0.3, 0.0, 1.0, 0.05, "m"),
+    ("POINTER_LEASH_DEG", "Head follow leash", 10, 0, 60, 1, "°"),
     ("POINTER_WAKE_COUNTS", "Movement to wake", 40, 5, 200, 5, "counts"),
     ("POINTER_IDLE", "Release after idle", 30, 5, 120, 5, "s"),
 ]
@@ -356,11 +358,21 @@ class Backend(QObject):
 
     @Slot(str, float)
     def setPointerSetting(self, key, value):
-        integer = key in ("POINTER_WAKE_COUNTS", "POINTER_IDLE")
+        integer = key in ("POINTER_WAKE_COUNTS", "POINTER_IDLE", "POINTER_LEASH_DEG")
         write_conf_value(key, str(int(round(value))) if integer else f"{value:.3f}".rstrip("0").rstrip("."))
         self.reload_timer.start()  # debounce slider drags
         self.pointerChanged.emit()
 
+    @Property(bool, notify=pointerChanged)
+    def pointerFollow(self):
+        return read_conf().get("POINTER_FOLLOW", "0") not in ("", "0")
+
+    @Slot(bool)
+    def setPointerFollow(self, on):
+        write_conf_value("POINTER_FOLLOW", "1" if on else "0")
+        self.reload_timer.start()
+        self.pointerChanged.emit()
+
     @Slot(result=bool)
     def recenter(self):
         return self._send("recenter", HELPER)

input-settings/main.qml

@@ -314,6 +314,12 @@ Kirigami.ApplicationWindow {
             ]
 
             Kirigami.FormLayout {
+                Controls.Switch {
+                    Kirigami.FormData.label: "Head follow:"
+                    text: "Pointer follows your head (leash below; 0° locks it to your view)"
+                    checked: backend.pointerFollow
+                    onToggled: backend.setPointerFollow(checked)
+                }
                 Repeater {
                     model: backend.pointerSettings
                     delegate: RowLayout {

input/input-relay.py

@@ -19,8 +19,8 @@ keyboard node for its extra buttons). Roles, from ~/.config/frametop-input.json
   ignore       not grabbed, only observed for identification in the settings app
 Buttons and keys of pointer devices go through a per-device map to actions
 (left, right, middle, back, scroll_up, scroll_down, dashboard, recenter,
-pointer_toggle, sens_up, sens_down, layout_reset = put the desktop screens back in
+pointer_toggle, follow_toggle = head follow on or off, sens_up, sens_down,
-their saved layout, screens_toggle = hide or show the desktop screens, key = pass
+layout_reset = put the desktop screens back in their saved layout, screens_toggle = hide or show the desktop screens, key = pass
 through as a key, none).
 
 Keys also go to ft-screens (@ft_screens, the Frametop desktop's compositor), which
@@ -140,7 +140,7 @@ def eviocguniq(length):
 VIRTUAL_PREFIX = "frametop virtual"
 RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
 ACTIONS = ("left", "right", "middle", "back", "scroll_up", "scroll_down", "dashboard", "recenter",
-           "pointer_toggle", "sens_up", "sens_down", "layout_reset", "screens_toggle", "key", "none")
+           "pointer_toggle", "follow_toggle", "sens_up", "sens_down", "layout_reset", "screens_toggle", "key", "none")
 SCREENS = "\0ft_screens"
 KEYS = "\0frametop_keys"  # keys of keyboards grabbed for the desktop, for other readers
 FT_LAYOUT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "layout", "ft-layout")
@@ -406,6 +406,9 @@ class Pointer:
                 log("pointer off (toggle)")
             else:
                 self.wake(now)
+        elif name == "follow_toggle":
+            self.send("follow toggle")  # until the next restart; the setting is POINTER_FOLLOW
+            log("head follow toggled")
         elif name == "screens_toggle":
             try:
                 self.sock.sendto(b"toggle", SCREENS)

pointer/helper/ft-pointer.cpp

@@ -86,6 +86,18 @@
 // doesn't jump the cursor to free space or swap in the laser-catching dot, which made
 // SteamVR's resize snap back.
 //
+// Head follow (off by default; POINTER_FOLLOW=1, or the relay's "follow toggle"): the cursor
+// is carried by a reference direction on a leash, POINTER_LEASH_DEG (10) from where the head
+// faces. While the head turns within the leash, the reference and the cursor stay put in the
+// room; past it, the reference is dragged along at the leash's end, and the cursor turns with
+// it, keeping its offset (mouse movement changes the offset). At 0 the reference is the head's
+// facing, so the cursor is head-locked. Head roll is ignored (the frames have no roll), so
+// tilting the head doesn't swing the cursor. The cursor stays within kFollowReachDeg of the
+// reference, so it can't be pushed out of view. The ray starts at the eye. While the left
+// button is held (and the drop hold after it), the leash still moves the reference but the
+// cursor stays put in the room, so a click or a drag can't be nudged by the head; the offset
+// is taken up from where the cursor is when the hold ends, so it doesn't jump.
+//
 // Placement (for layout): SteamVR keeps a floating panel's position inside the
 // dashboard, where nothing outside can set it, so the helper carries panels like a user
 // would. It measures the panel (md::ScanPanel), aims the device at its grab bar
@@ -100,6 +112,7 @@
 // twice while it's more than 1.5 cm or 1 deg off.
 //
 // Commands (datagrams on @ft_pointer_helper): show, hide, recenter, move <dyaw> <dpitch>,
+// follow on|off|toggle (head follow, until the next restart or a change to POINTER_FOLLOW),
 // reload (re-read the settings below), debug (toggle a twice-a-second state log),
 // and btn/scroll lines, which are forwarded to the driver unchanged. For layouts, with a
 // reply datagram to the sender's (abstract) address:
@@ -117,7 +130,8 @@
 // POINTER_ORIGIN_FRACTION (0.95): the laser starts this far along the eye-to-cursor line,
 // but never closer than POINTER_ORIGIN_MARGIN (0.15 m) to the cursor point: SteamVR's
 // small controls (undock, frame buttons) float a few centimetres in front of their
-// panel, and a laser that starts behind them can't hit them.
+// panel, and a laser that starts behind them can't hit them. POINTER_FOLLOW (0) and
+// POINTER_LEASH_DEG (10): head follow, above.
 #include <openvr.h>
 
 #include "vrmath.h"
@@ -290,15 +304,34 @@ std::vector<uint8_t> DotTexture(int size) {
     return px;
 }
 
+// Unit vector v, turned toward unit vector `center` until it's at most maxRad from it.
+Vec3 PullWithin(Vec3 v, Vec3 center, double maxRad) {
+    if (std::acos(std::clamp(Dot(v, center), -1.0, 1.0)) <= maxRad) return v;
+    Vec3 axis = Cross(center, v);
+    if (Length(axis) < 1e-9) axis = Cross(center, {0, 1, 0});  // opposite: any perpendicular
+    return RotateAbout(center, Normalize(axis), maxRad);
+}
+
+// Unit direction with its pitch limited to +-maxDeg (AimBasis needs it off vertical).
+Vec3 LimitPitch(Vec3 d, double maxDeg) {
+    const double pitch = std::clamp(std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI, -maxDeg, maxDeg);
+    return Direction(std::atan2(-d.x, -d.z) * 180 / M_PI, pitch);
+}
+
 }  // namespace
 
 // Placement speeds (see "Placement" at the top).
 constexpr double kPlaceDegPerSec = 60;      // tested: 40 deg/s is applied exactly
+// Head follow (see the top): the farthest the cursor can be from the reference direction.
+constexpr double kFollowReachDeg = 40;
 
 int main() {
     double freeDistance = 1.5, cursorDeg = 0.4, originFraction = 0.95, originMargin = 0.15, sceneRadius = 0.5,
            edgeReach = 0.3, grabOffset = 0.075,
-           slideSpeed = 0.5;
+           slideSpeed = 0.5, leashDeg = 10;
+    // Head follow (see the top). followConf is POINTER_FOLLOW as last read: a reload only
+    // overrides a "follow" command when the setting itself changed.
+    bool follow = false, followConf = false, followReset = true;
     auto loadConfig = [&] {
         const auto conf = ReadConfig();
         freeDistance = std::clamp(ConfDouble(conf, "POINTER_DISTANCE", 1.5), 0.3, 10.0);
@@ -309,6 +342,9 @@ int main() {
         edgeReach = std::clamp(ConfDouble(conf, "POINTER_EDGE_REACH", 0.3), 0.0, 2.0);
         grabOffset = std::clamp(ConfDouble(conf, "LAYOUT_GRAB_OFFSET", 0.075), 0.0, 1.0);
         slideSpeed = std::clamp(ConfDouble(conf, "LAYOUT_SLIDE_SPEED", 0.5), 0.02, 2.0);
+        leashDeg = std::clamp(ConfDouble(conf, "POINTER_LEASH_DEG", 10), 0.0, 90.0);
+        const bool wantFollow = ConfDouble(conf, "POINTER_FOLLOW", 0) != 0;
+        if (wantFollow != followConf) follow = followConf = wantFollow, followReset = true;
     };
     loadConfig();
     const float laserWidth = float(ConfDouble(ReadConfig(), "POINTER_LASER_WIDTH", 0.8));
@@ -392,6 +428,7 @@ int main() {
     };
     Vec3 anchor;
     double yaw = 0, pitch = 0;
+    Vec3 followRef{0, 0, -1};  // head follow's reference direction (see the top)
     vr::TrackedDeviceIndex_t ours = vr::k_unTrackedDeviceIndexInvalid;
     auto lastSlow = std::chrono::steady_clock::now() - std::chrono::seconds(10);
 
@@ -667,8 +704,16 @@ int main() {
                 recenter = true;
             } else if (std::strncmp(buf, "reload", 6) == 0) {
                 loadConfig();
-                std::printf("reloaded: free distance %.2f m, dot %.2f deg, origin %.2f\n", freeDistance, cursorDeg,
+                std::printf("reloaded: free distance %.2f m, dot %.2f deg, origin %.2f, head follow %s, leash %.0f deg\n",
-                            originFraction);
+                            freeDistance, cursorDeg, originFraction, follow ? "on" : "off", leashDeg);
+                std::fflush(stdout);
+            } else if (std::strncmp(buf, "follow", 6) == 0) {
+                const char *arg = buf + 6;
+                while (*arg == ' ') ++arg;
+                const bool was = follow;
+                follow = std::strncmp(arg, "on", 2) == 0 ? true : std::strncmp(arg, "off", 3) == 0 ? false : !follow;
+                if (follow && !was) followReset = true;
+                std::printf("head follow %s (leash %.0f deg)\n", follow ? "on" : "off", leashDeg);
                 std::fflush(stdout);
             } else if (std::strncmp(buf, "show", 4) == 0) {
                 if (!active) wake(Clock::now());
@@ -757,6 +802,23 @@ int main() {
             pitch = std::asin(std::clamp(f.y, -1.0, 1.0)) * 180 / M_PI;
             anchored = true;
             recenter = false;
+            followReset = true;
+        }
+
+        // Head follow (see the top): drag the reference along on its leash, and the cursor with it.
+        if (follow && active && anchored && hmd.bPoseIsValid) {
+            const Vec3 head = LimitPitch(Vec3{-hm[0][2], -hm[1][2], -hm[2][2]}, 85);
+            if (followReset) followRef = head, followReset = false;
+            const Vec3 ref = LimitPitch(PullWithin(followRef, head, leashDeg * M_PI / 180), 85);
+            if (!leftHeld && tnow >= dropHoldUntil) {
+                // The cursor keeps its offset from the reference (frames without roll).
+                Vec3 d = FromBasis(AimBasis(ref), ToBasis(AimBasis(followRef), Direction(yaw, pitch)));
+                d = PullWithin(Normalize(d), ref, kFollowReachDeg * M_PI / 180);
+                yaw = std::atan2(-d.x, -d.z) * 180 / M_PI;
+                pitch = std::clamp(std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI, -85.0, 85.0);
+                anchor = eye;
+            }
+            followRef = ref;
         }
 
         // Slow work, once a second: overlay handles, our device index, laser width.

session/frametop.conf.example

@@ -18,6 +18,8 @@ POINTER_LASER_WIDTH=0.8    # controller beam width (dashboard.laserRayWidthScale
 POINTER_ORIGIN_MARGIN=0.15   # the laser starts at least this far (m) in front of its target, so small floating controls stay hittable
 POINTER_SCENE_RADIUS=0.5   # texture-less dashboard overlays (dock, window controls): hit radius (m) around their origin
 POINTER_EDGE_REACH=0.3     # just off a panel, the cursor stays on its plane this far (m), to reach its resize edges and window controls
+POINTER_FOLLOW=0           # 1 = the cursor follows your head on a leash (a mouse button mapped to Head follow on/off toggles it)
+POINTER_LEASH_DEG=10       # head follow: how far (degrees) your head can turn before the cursor comes along; 0 locks it to your view
 META_DASHBOARD=0   # 1 = a Meta tap on a pass-through keyboard toggles the SteamVR dashboard (pointer mode only)
 SHARE_KEYS=0       # 1 = keys of keyboards grabbed for the desktop also go to @frametop_keys, for hotkey tools (any local process can listen)
 LAYOUT_GRAB_OFFSET=0.075   # arranging screens: where the floating window's grab bar is, in metres below the screen