Compare commits

..
5 Commits
Author SHA1 Message Date
DeeJanuz c377133859 Merge experimental into shortcuts
# Conflicts:
#	input/input-relay.py
2026-10-01 17:08:15 -06:00
DeeJanuzandClaude Opus 5.5 675ef3f0d5 Relay: share a key combination's Meta release with frame-voice
A Meta+key combination (Meta+J for gaze_left, say) hides Meta's release
from the desktop, and the relay skipped share_key for it too. frame-voice
saw Meta go down on @frametop_keys and never come up, so it held all
dictated text back, waiting for that release. Keys of grabbed keyboards
are now shared as pressed, before key_binding() decides what the desktop
gets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 17:07:38 -06:00
DeeJanuzandClaude Opus 5.5 fadc0f467c Shortcuts: Steam menu, commands, and modifier taps
Key combinations (and mouse and controller buttons) get two new actions:
- steam_menu: Open Steam menu / close dashboard (steam/ft-steam menu).
- command:CMD: run CMD with sh -c, as the relay's service, with layout/,
  float/ and steam/ on its PATH. Input Settings offers it for key
  combinations as Run a command...
Both work without pointer mode.

A modifier on its own is now a key combination too: a tap, pressed and
released with no other key, mouse button, or scroll in between. A bound
tap sends the desktop F24 before the release, so Plasma's launcher stays
shut. The defaults gain a Meta tap for the Steam menu; this replaces
META_DASHBOARD, which only worked in pointer mode.

input/test/keys-test.py runs the relay against fake devices with every
outgoing socket renamed, so it's safe next to the live relay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 17:06:30 -06:00
DeeJanuzandClaude Opus 5.5 2487351556 ft-steam: open Steam's menu through Steam's own UI
steam/ft-steam menu opens the SteamVR dashboard on Steam's menu, or closes
the dashboard if it's up, without pointer mode: it asks Steam's UI over its
debugging port to show its dashboard overlay (ShowVROverlay, what Steam
calls itself) and focus the Steam frame's left menu (MenuStore.OpenMainMenu).
ft-steam check says whether those calls still exist, and update-check.py
runs it, since a Steam client update can rename them.

The CDP client moves from display-settings/steam_settings.py to
steam/steamui.py, so both use it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 17:06:30 -06:00
DeeJanuzandClaude Opus 5.5 4479fbfcc1 Input relay: typing with the pointer helper down no longer ends the relay
Typing on a pass-through keyboard tells the helper "typing". With the
helper not running (SteamVR off), that send raised ConnectionRefusedError
and the relay exited, dropping every grab until systemd restarted it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 17:01:31 -06:00
33 changed files with 611 additions and 1807 deletions

No files matched your search

+2 -3
View File
@@ -13,8 +13,6 @@ Two settings apps come with it: Frametop Display Settings for the screens, profi
Frametop is an independent project, not made by or affiliated with Valve.
Join the [Frametop Discord](https://discord.gg/W3X9f7z3Bc) for questions, ideas, and help with your setup.
## Install on the headset
You need a Steam Frame with an internet connection, a keyboard (Bluetooth, or the on-screen one), and about 3 GB of free space.
@@ -56,6 +54,7 @@ If you work in the desktop for long stretches, or leave the headset on a stand,
| Set a screen to On your head (Frametop Display Settings, Visibility & pins) | Pins it to your head where it is, like a HUD. Grab its bar to move it; it stays on your head |
| Meta+Shift+R in the desktop | Puts the screens back in their layout (also in the menu as Reset Screen Layout, and mappable to a mouse button) |
| Meta+Shift+H in the desktop | Hides or shows all screens (also in the menu as Hide/Show Screens, and mappable). The Visibility & pins tab of Frametop Display Settings can instead show them only with the dashboard open, or while you look at your wrist |
| Tap Meta, on any keyboard | Opens the Steam menu in the SteamVR dashboard, or closes the dashboard, wherever you are. The desktop's launcher is still on the taskbar and Alt+F1. Change it in Frametop Input Settings (Keyboard page), where any key combination or modifier tap can do a Frametop or Steam action, open a profile, or run a command of your own |
| Switch a screen to Hidden (Frametop Display Settings, Visibility & pins → Screens shown) | Hides just that screen until you switch it back, whatever the other visibility settings say; new windows that would open on it float instead |
| Play a VR game | The screens hide and your controllers stay in the game. Open the SteamVR dashboard, or press Meta+Shift+H, to see and use them. To keep them visible over games, change During VR games on the Visibility & pins tab; the controllers still stay in the game, and you use the screens with the mouse or the dashboard |
@@ -140,7 +139,7 @@ In a terminal on the headset, run:
cd ~/frametop && scripts/report.sh
```
This writes `frametop-report-<date>.txt` with version numbers, service states, settings, and recent logs. Bluetooth addresses and the headset's serial number are masked. Then [open an issue](https://github.com/DeeJanuz/frametop/issues), describe what you did, what you expected, and what happened, and attach the file. Quick questions can go to [Discord](https://discord.gg/W3X9f7z3Bc) instead.
This writes `frametop-report-<date>.txt` with version numbers, service states, settings, and recent logs. Bluetooth addresses and the headset's serial number are masked. Then [open an issue](https://github.com/DeeJanuz/frametop/issues), describe what you did, what you expected, and what happened, and attach the file.
## Update
+11 -102
View File
@@ -4,117 +4,26 @@ Steam, not systemd, puts the Frame to sleep: after "When Plugged In and Idle ->
(an hour by default) without input, even while it charges. That's a Steam client setting,
`system_idle_suspend_ac_sec` (0 = never), with no file or command line to change it. Steam
on the Frame runs with -cef-enable-debugging, so its UI's JavaScript context
(SharedJSContext) is reachable over the Chrome DevTools Protocol on 127.0.0.1:8080. There
`settingsStore.clientSettings` has the current values, and `SteamClient.Settings.SetSetting`
takes a change as a serialized CMsgClientSettings protobuf, which is what Steam's own
Settings -> Power page sends. Standard library only (a minimal WebSocket client).
(SharedJSContext) is reachable over the Chrome DevTools Protocol on 127.0.0.1:8080
(steam/steamui.py). There `settingsStore.clientSettings` has the current values, and
`SteamClient.Settings.SetSetting` takes a change as a serialized CMsgClientSettings protobuf,
which is what Steam's own Settings -> Power page sends.
"""
import base64
import json
import os
import socket
import struct
import urllib.request
import sys
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "steam"))
from steamui import SteamUnreachable, evaluate # noqa: E402,F401 (callers catch SteamUnreachable here)
CDP_PORT = 8080
# CMsgClientSettings field numbers (Steam's UI bundle maps the names to these).
FIELDS = {"system_idle_suspend_ac_sec": 24004, "system_idle_suspend_battery_sec": 24003}
class SteamUnreachable(Exception):
pass
class _WebSocket:
def __init__(self, url, timeout=5):
host_port, path = url[len("ws://"):].split("/", 1)
host, port = host_port.rsplit(":", 1)
self.sock = socket.create_connection((host, int(port)), timeout=timeout)
key = base64.b64encode(os.urandom(16)).decode()
self.sock.sendall((f"GET /{path} HTTP/1.1\r\nHost: {host_port}\r\nUpgrade: websocket\r\n"
f"Connection: Upgrade\r\nSec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n\r\n").encode())
head = b""
while b"\r\n\r\n" not in head:
chunk = self.sock.recv(4096)
if not chunk:
raise SteamUnreachable("Steam closed the connection")
head += chunk
if b" 101 " not in head.split(b"\r\n", 1)[0]:
raise SteamUnreachable(head.split(b"\r\n", 1)[0].decode(errors="replace"))
self.buf = head.split(b"\r\n\r\n", 1)[1]
def send(self, text):
data, mask = text.encode(), os.urandom(4)
n = len(data)
if n < 126:
head = struct.pack(">BB", 0x81, 0x80 | n)
elif n < 65536:
head = struct.pack(">BBH", 0x81, 0x80 | 126, n)
else:
head = struct.pack(">BBQ", 0x81, 0x80 | 127, n)
self.sock.sendall(head + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(data)))
def _take(self, n):
while len(self.buf) < n:
chunk = self.sock.recv(65536)
if not chunk:
raise SteamUnreachable("Steam closed the connection")
self.buf += chunk
out, self.buf = self.buf[:n], self.buf[n:]
return out
def recv(self):
message = b""
while True:
b0, b1 = self._take(2)
n = b1 & 0x7F
if n == 126:
n = struct.unpack(">H", self._take(2))[0]
elif n == 127:
n = struct.unpack(">Q", self._take(8))[0]
message += self._take(n)
if b0 & 0x80:
return message.decode(errors="replace")
def close(self):
self.sock.close()
def _evaluate(expression):
"""Runs JavaScript in Steam's SharedJSContext and returns its (awaited) value."""
try:
with urllib.request.urlopen(f"http://127.0.0.1:{CDP_PORT}/json", timeout=3) as r:
targets = json.load(r)
except OSError as e:
raise SteamUnreachable(f"Steam isn't reachable on port {CDP_PORT} ({e})") from e
url = next((t["webSocketDebuggerUrl"] for t in targets if t.get("title") == "SharedJSContext"), None)
if not url:
raise SteamUnreachable("Steam's UI isn't running")
try:
ws = _WebSocket(url)
try:
ws.send(json.dumps({"id": 1, "method": "Runtime.evaluate",
"params": {"expression": expression, "awaitPromise": True, "returnByValue": True}}))
while True:
reply = json.loads(ws.recv())
if reply.get("id") == 1:
break
finally:
ws.close()
except OSError as e:
raise SteamUnreachable(str(e)) from e
result = reply.get("result", {})
if "exceptionDetails" in result:
details = result["exceptionDetails"]
raise SteamUnreachable(details.get("exception", {}).get("description") or details.get("text", "error"))
return result.get("result", {}).get("value")
def sleep_settings():
"""{"ac": seconds, "battery": seconds}: when Steam puts the Frame to sleep without input,
plugged in and on battery (0 = never)."""
value = _evaluate("(() => { const c = settingsStore.clientSettings; "
"return {ac: c.system_idle_suspend_ac_sec, battery: c.system_idle_suspend_battery_sec}; })()")
value = evaluate("(() => { const c = settingsStore.clientSettings; "
"return {ac: c.system_idle_suspend_ac_sec, battery: c.system_idle_suspend_battery_sec}; })()")
if not isinstance(value, dict) or not all(isinstance(value.get(k), int) for k in ("ac", "battery")):
raise SteamUnreachable("Steam's settings don't have the sleep timeouts")
return value
@@ -125,7 +34,7 @@ def set_sleep_setting(name, seconds):
field, seconds = FIELDS[name], int(seconds)
if seconds < 0:
raise ValueError("seconds must be 0 (never) or more")
ok = _evaluate(f"""(async () => {{
ok = evaluate(f"""(async () => {{
const bytes = [];
const varint = n => {{ while (n > 127) {{ bytes.push((n & 127) | 128); n = Math.floor(n / 128); }} bytes.push(n); }};
varint({field} * 8); varint({seconds});
-121
View File
@@ -1,121 +0,0 @@
# Independent desktop mouse
`input/ft-mousectl desktop --save` selects normal desktop input: the physical
mouse moves across the nested KDE monitors, clicks and scrolls through KDE's
Wayland seat, and displays KDE's own cursor. No mouse event becomes a virtual
controller event. Physical controllers remain available to SteamVR and games.
While a routed mouse is connected, controller events on desktop app panels are ignored;
floating app panels, the separate grab bars and placement controls still work under the existing
controller policy. SteamVR dashboard controls are unaffected.
Native CLI commands (no additional project tooling required):
```sh
input/ft-mousectl status
input/ft-mousectl desktop --save
input/ft-mousectl center 2
input/ft-mousectl position 2 960 540
input/ft-mousectl layout
input/ft-mousectl spatial --save
```
Numbers follow KDE's WL-0, WL-1 output order, starting at 1. Positions are local
logical pixels; `layout` reads actual output geometry and scale from the running
private KDE session. Geometry follows FrameTop layout changes too. A drag keeps
the Wayland implicit grab on its initial output until all buttons are released,
while the cursor can cross onto another monitor.
The relay gathers motion and wheels until the mouse's SYN_REPORT. High resolution
wheel reports use 1/120-notch units and replace accompanying ordinary reports;
they are delivered immediately without the spatial pointer's pulse timer. The
main and thumb wheels are independent. Standard mouse button codes are passed
through, including back/forward; spatial mouse button mappings do not apply.
`DESKTOP_MOUSE_SPEED` in `~/.config/frametop.conf` sets logical pixels per motion
count, default 1, range .05–10. This initial implementation uses constant speed,
not KDE's libinput acceleration settings, because FrameTop's relay owns the
physical device outside the nested session.
The cursor overlay only displays KDE's cursor; it has no input method, laser
claim, or interactive overlay flag. Cursor shape, hotspot and buffer scale come
from KDE. GPU cursor buffers use the same zero-copy DMA-BUF import as the monitor
textures. Common 32-bit shared-memory cursor formats are converted to RGBA.
Controller fallback: `DESKTOP_CONTROLLER_FALLBACK=1` (default) allows controller
laser input into desktop apps when no grabbed physical mouse is routed by the
relay. Idle mice do not trigger fallback. The relay reports presence every
second; absence must persist for one second, so disconnect detection normally
takes about one to two seconds. Keyboard routing still follows desktop clicks.
The virtual spatial mouse/gaze pointer stays off; controller policy stays
`dashboard`, preserving SteamVR's existing pointer-mode behavior.
Reconnect or native mouse motion returns ownership to the native seat and
releases controller-held buttons before clearing its focus. A held native mouse
button prevents fallback until released. An unknown presence or a heartbeat
older than three seconds blocks automatic controller desktop input. Set
`DESKTOP_CONTROLLER_FALLBACK=0` to retain strict separation even without a mouse;
reload the relay at an appropriate pause after changing this preference.
`input/ft-mousectl status` includes `controllerFallback` with presence, freshness,
configured enablement and actual desktop controller ownership. Older running
compositors report this field unavailable until explicitly reloaded.
Both paths support images up to 512×512 and buffer scale 8. FrameTop retains the
image from the first surface commit, because KDE can commit it before selecting
the active cursor and wlroots releases current.buffer after commit callbacks.
An unsupported buffer is reported by cursorReady=false with cursorError
and hides the old cursor image. Rendering in VR still needs a wearer check for
visibility, stereo depth, monitor edges and cursor shape changes.
The private desktop and apps launched from it use KDE's `kcminputrc` cursor
preferences (24px Breeze by default). Steam's VR environment exports
`XCURSOR_SIZE=256` and `XCURSOR_THEME=steam`; inheriting these makes app cursors
huge and can make their appearance change across monitors. A live KDE
CursorChanged notification updates KDE, but already running apps may retain
the old environment until reopened. Do not resize the VR overlay to mask an
app's incorrectly configured cursor size.
The updated compositor, pointer helper and relay must all be loaded before
activation. `status` is read-only and shows readiness, actual/desired mode,
held buttons and delivery counters. Saving happens only after an acknowledged
mode change. Switching refuses while buttons/keys on a pointer device are held.
An absent compositor does not prevent explicit spatial recovery. Restarting the
desktop requires the user's approval; use `desktops.sh restart` so background
work is preserved, and restore the current app windows afterwards.
If the compositor is lost, native mouse messages are not redirected into VR.
The input helpers need reloading when upgrading this feature. A standalone
helper/compositor restart may require `input/ft-mousectl desktop` again; inspect
all three components in `status` rather than inferring success from motion alone.
Offline verification (or run `scripts/test-desktop-input.sh`):
```sh
python3 -m unittest discover -s input -p 'test_*.py'
scripts/frame.sh -C screens 'gcc -std=c11 -Wall -Wextra -Werror tests/desktop-mouse-test.c -lm -o build/desktop-mouse-test && build/desktop-mouse-test'
scripts/frame.sh -C screens 'gcc -std=c11 -Wall -Wextra -Werror $(pkg-config --cflags libdrm) tests/desktop-cursor-test.c -o build/desktop-cursor-test && build/desktop-cursor-test'
scripts/frame.sh -C screens 'gcc -std=c11 -Wall -Wextra -Werror tests/cursor-cache-test.c $(pkg-config --cflags --libs wlroots-0.20 wayland-server wayland-client libdrm pixman-1) -o build/cursor-cache-test && build/cursor-cache-test'
```
The direct seat and cursor path follow the [Wayland pointer protocol](https://wayland.freedesktop.org/docs/html/apa.html#protocol-spec-wl_pointer).
The default remains `MOUSE_MODE=spatial`. Native mode is explicitly selected
with `input/ft-mousectl desktop --save`; no driver bindings or SteamVR defaults
are changed by installation. The new native mode suppresses virtual gaze, hand
gesture and gaze-click input while selected. Switch back to spatial mode to
use those features. Ordinary VR keyboard events remain available.
## Experimental desktop handoff
`input/ft-mousectl policy last-active` enables shared desktop input. Controller
clicks and nonzero scrolls inside desktop monitors take ownership; aiming alone
does not. Native mouse motion, clicks and scrolling reclaim ownership. Same
millisecond claims favor the mouse. Held buttons keep ownership until released;
other-device events are discarded rather than replayed after the drag. SteamVR
controls and separate monitor grab/placement bars do not participate.
`input/ft-mousectl policy mouse` restores the original mouse-first behavior
(including configured mouse-absent fallback). `policy pointer` reserves desktop
input for the controller. Changes refuse while either device holds a desktop
button. `status` reports policy, owner, held buttons and ignored mouse events.
These choices are runtime-only and reset to `mouse` when the desktop restarts.
The original PR checkpoint is retained on `checkpoint/desktop-input-before-handoff`;
this experiment is developed separately on `feat/desktop-input-handoff`.
+5 -5
View File
@@ -2,8 +2,6 @@
How each part of Frametop works, where its settings live, and the commands for running parts of it by hand. For why it's built this way, see [design.md](design.md).
For independent mouse input, KDE cursor rendering and controller fallback, see [desktop-mouse.md](desktop-mouse.md).
## The desktop
From the headset, open Launch a program → Desktop. The installer replaces that launcher entry with Frametop's (`~/.local/share/applications/deckard-nested-desktop.desktop`), and `desktops.sh uninstall` gives the stock single-screen desktop back.
@@ -87,6 +85,8 @@ The relay also owns the volume keys, on every device that has them, the headset'
desktops.sh relay install # enable it (starts with the next reboot or SteamVR start)
desktops.sh relay status | log | uninstall
input/input-relay.py --no-grab # try it without taking devices from SteamVR
input/test/keys-test.py # key combinations and modifier taps, against fake devices (safe next to the live relay)
steam/ft-steam menu # what Open Steam menu does; ft-steam check: Steam's UI still has the calls
```
The first time, the relay has to start before SteamVR, so reboot or restart SteamVR after installing it. After that it's safe to restart on its own: systemd keeps the virtual devices open in its file descriptor store (`FileDescriptorStorePreserve=yes`), so SteamVR keeps the same devices.
@@ -101,7 +101,7 @@ A mouse drives SteamVR the way a controller's laser does, but shows up as a smal
Whichever device you used last wins. Picking up a controller hands the laser back at once, and moving the mouse takes it again. When the headset comes off, the pointer lets go, so the displays can sleep, and it stays off until you're wearing the headset again.
To move a floating panel, left-drag its grab bar. The scroll wheel pushes and pulls it while you drag. Hold the right button while dragging and move the mouse to tilt the panel around the grab point; the right press isn't sent as a click. The tilt stays for the rest of the drag, and releasing the left button drops the panel as it is. A mapped Toggle dashboard button (or a Meta tap, with `META_DASHBOARD=1`) wakes the pointer if needed and holds the virtual system button for 0.12 s, because SteamVR ignores a press and release in the same instant.
To move a floating panel, left-drag its grab bar. The scroll wheel pushes and pulls it while you drag. Hold the right button while dragging and move the mouse to tilt the panel around the grab point; the right press isn't sent as a click. The tilt stays for the rest of the drag, and releasing the left button drops the panel as it is. A mapped Toggle dashboard button wakes the pointer if needed and holds the virtual system button for 0.12 s, because SteamVR ignores a press and release in the same instant. Open Steam menu / close dashboard (`steam_menu`) needs no pointer: `steam/ft-steam menu` asks Steam's UI, over its debugging port (`steam/steamui.py`), to show its dashboard overlay and focus the Steam frame's menu, or to hide the dashboard if it's up.
```
pointer/driver/build.sh && pointer/driver/install.sh install # then restart SteamVR
@@ -116,10 +116,10 @@ The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `P
A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has eight pages:
- Devices lists every USB and Bluetooth mouse and keyboard, with a light that flashes when the device is used. Each device gets a role: 3D pointer (grabbed, drives the pointer; the default for anything with a mouse), Pass through (grabbed only while typing goes to the desktop; the default for keyboards, where a Meta tap toggles the dashboard if `META_DASHBOARD=1` is in `~/.config/frametop.conf`), or Ignore. A device is identified by its Bluetooth address, or its USB ids and name, so all of its input nodes share one role. Forget drops everything saved for a device.
- Devices lists every USB and Bluetooth mouse and keyboard, with a light that flashes when the device is used. Each device gets a role: 3D pointer (grabbed, drives the pointer; the default for anything with a mouse), Pass through (grabbed only while typing goes to the desktop; the default for keyboards, whose key combinations work everywhere), or Ignore. A device is identified by its Bluetooth address, or its USB ids and name, so all of its input nodes share one role. Forget drops everything saved for a device.
- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, gaze pointer on or off, gaze precision, gaze drag, gaze quick check, faster or slower, reset the screen layout, hide or show the screens, open or close the keyboard, float a window in VR or put it back, put all floating windows back, Open profile NAME (one per profile, [profiles.md](profiles.md)), pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
- Controllers maps the Frame controllers' buttons (every button but the system button) to the same actions, except passing a key through and the gaze actions: gaze mode is a mouse and keyboard feature ([gaze-controllers.md](gaze-controllers.md)). Capture a button and press it on a controller, or pick it from the list. The controllers aren't input devices on the host; only SteamVR sees them. So the pointer helper reads them with SteamVR input (`pointer/helper/vrbuttons.h`, `pointer/helper/actions/`) and sends presses to the relay (`vrbtn right/a 1`), which does the mapped action. The helper only takes the buttons that are mapped (the relay tells it with `vrbind`), at an overlay-global priority, and only while no game (scene application) runs, so games keep every button; with In games on (`controller_in_games`), a mapped button is taken from games too. That needs SteamVR's "Enable global input from overlays (Experimental)" setting (`steamvr/globalActionSetPriority`), which the page's Global input switch turns on and off. Mappings are saved as `controller_buttons` in `~/.config/frametop-input.json`.
- Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput, like frame-voice's, don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected. Its Key combinations section maps modifiers plus a key, on any keyboard, to any action but passing a key through or nothing. The gaze clicks (Gaze left click and Gaze right click) only go on key combinations. The defaults are Meta+J (gaze left click), Meta+K (gaze right click), and Meta+Shift+F (float window in VR or put it back); remove them or add others there. The combination's last key isn't typed, and the modifiers still reach the app. They're saved as `key_bindings` in the same file.
- Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput, like frame-voice's, don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected. Its Key combinations section maps modifiers plus a key, or one modifier tapped on its own, on any keyboard, to any action but passing a key through or nothing, or to Run a command…: a command line the input relay runs with `sh -c` when you press the keys (`command:CMD`). The command runs as the relay's user service, outside the desktop's session, with `layout/`, `float/` and `steam/` on its `PATH` (so `ft-layout use Work` or `ft-float launch org.kde.dolphin` work as they are), and its output goes to the relay's journal. The gaze clicks (Gaze left click and Gaze right click) only go on key combinations. The defaults are a Meta tap (open the Steam menu, or close the dashboard), Meta+J (gaze left click), Meta+K (gaze right click), and Meta+Shift+F (float window in VR or put it back); remove them or add others there. A tap is a press and release with no other key, mouse button, or scroll in between; a bound one sends the desktop F24 before the release, so Plasma's launcher doesn't open on it. The combination's last key isn't typed, and the modifiers still reach the app; while typing goes to Steam rather than the desktop, keyboards aren't grabbed, so Steam or the game sees the keys too. They're saved as `key_bindings` in the same file; a file with its own list, even an empty one, gets no defaults.
- Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button.
- Ignored panels lists the SteamVR overlays that are showing, grouped by app (the first two parts of the overlay key, such as `sasaken.frame-perf-overlay`), from the pointer helper (`overlays`). Tick a panel, or Ignore the whole app, and the pointer passes through it to what's behind. It's for panels you only look at, like a performance overlay that follows your view. The list is saved as `POINTER_IGNORE` in `~/.config/frametop.conf`: comma-separated overlay keys, where a shell pattern like `vendor.app*` covers a whole app, including panels it opens later. The helper reloads at once. Frametop's own screens aren't listed, and entries for apps that aren't open are listed below, to remove.
- Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), what the mouse's left button and movement do, the gaze dot, the eye tracker and eye bias, the gaze mode sliders, the gaze service's state (headset, samples per second, how often the tracker is losing each eye, the calibration, the nudges learned), and Quick check, Calibrate, and Check headset fit (each in a panel in the headset), Reload calibration, and Forget nudges. The gaze probe, a development tool, is in the page's overflow menu.
+40 -14
View File
@@ -9,8 +9,9 @@ to the input relay over its control socket (@frametop_relay):
- Controllers: the same for the Frame controllers' buttons, minus the gaze actions (gaze
mode is a mouse feature). They're read by the pointer helper through SteamVR input
(@ft_pointer_helper: vrstatus, vrglobal), and a mapped button is taken from games.
- Keyboard: when Frametop's keyboard opens, and key combinations for any action (Meta+Shift+F
floats a window unless the rules have their own list).
- Keyboard: when Frametop's keyboard opens, and key combinations for any action, a command
of your own too ("command:CMD"), or a modifier tapped alone (a Meta tap opens Steam's menu
and Meta+Shift+F floats a window, unless the rules have their own list).
- Pointer: speed, dot size, distance and the rest, applied live.
- Ignored panels: SteamVR overlays the pointer passes through (POINTER_IGNORE), by app or
one by one. The helper lists them (@ft_pointer_helper "overlays").
@@ -60,6 +61,7 @@ DEFAULT_BUTTONS = {0x110: "left", 0x111: "right", 0x112: "middle", 0x113: "back"
ACTION_LABELS = {
"left": "Left click", "right": "Right click", "middle": "Middle click", "back": "Back",
"scroll_up": "Scroll up", "scroll_down": "Scroll down", "dashboard": "Toggle SteamVR dashboard",
"steam_menu": "Open Steam menu / close dashboard",
"recenter": "Recenter pointer", "pointer_toggle": "Pointer on/off",
"follow_toggle": "Head follow on/off (experimental)", "gaze_toggle": "Gaze pointer on/off (experimental)",
"gaze_precision": "Gaze precision: hold to steer, release to click",
@@ -102,6 +104,8 @@ KEYBOARD_ONLY = ("gaze_left", "gaze_right")
# Profiles (docs/profiles.md): "profile:NAME" opens one (the relay runs ft-layout use NAME).
LAYOUT_PATH = os.path.expanduser("~/.config/frametop-layout.json")
PROFILE = "profile:"
# "command:CMD": the relay runs CMD with sh -c (key combinations only, here).
COMMAND = "command:"
def profile_actions():
@@ -113,6 +117,8 @@ def profile_actions():
def action_label(a):
if a.startswith(PROFILE):
return f"Open profile {a[len(PROFILE):]}"
if a.startswith(COMMAND):
return f"Run: {a[len(COMMAND):]}"
return ACTION_LABELS.get(a, a)
@@ -120,14 +126,18 @@ def is_profile(a):
return a.startswith(PROFILE) and len(a) > len(PROFILE)
def is_command(a):
return a.startswith(COMMAND) and bool(a[len(COMMAND):].strip())
def mappable(a):
"""An action a controller button can have."""
return a in CONTROLLER_ACTIONS or is_profile(a)
def shortcut_mappable(a):
"""An action a key combination can have: the gaze ones too."""
return a in SHORTCUT_ACTIONS or is_profile(a)
"""An action a key combination can have: the gaze ones and commands too."""
return a in SHORTCUT_ACTIONS or is_profile(a) or is_command(a)
# Gaze mode settings (pointer helper), like POINTER_SETTINGS.
GAZE_SETTINGS = [
("POINTER_GAZE_RETAKE", "Look away to hand back", 5, 1, 45, 0.5, "°"),
@@ -141,9 +151,10 @@ GAZE_MOUSE = {"precision": "Gaze precision: hold to steer with the mouse, releas
# Key combinations ("key_bindings" in the rules): modifiers, either side folded into the left code.
MODIFIER_CODES = {29: 29, 97: 29, 42: 42, 54: 42, 56: 56, 100: 56, 125: 125, 126: 125}
MODIFIER_NAMES = {29: "Ctrl", 42: "Shift", 56: "Alt", 125: "Meta"}
# What a rules file without "key_bindings" gets (the relay's DEFAULT_KEY_BINDINGS): Meta+J and
# Meta+K click at the gaze, Meta+Shift+F floats a window.
DEFAULT_KEY_BINDINGS = {"125+36": "gaze_left", "125+37": "gaze_right", "42+125+33": "float_toggle"}
# What a rules file without "key_bindings" gets (the relay's DEFAULT_KEY_BINDINGS): a Meta tap
# opens Steam's menu, Meta+J and Meta+K click at the gaze, Meta+Shift+F floats a window.
DEFAULT_KEY_BINDINGS = {"125": "steam_menu", "125+36": "gaze_left", "125+37": "gaze_right",
"42+125+33": "float_toggle"}
def key_bindings(rules):
@@ -295,6 +306,7 @@ class Backend(QObject):
self._capture_vr = False
self._capture_combo = "" # the action a key combination is being captured for
self._combo_mods = set()
self._combo_tap = None # a modifier pressed alone, nothing since: released, it's a tap
self._vr = {} # the helper's vrstatus, {} when it doesn't answer
self._vr_at = 0.0
self._gaze = {} # ft-gazed's status, {} when it isn't running
@@ -408,13 +420,16 @@ class Backend(QObject):
self.activity.emit(msg["id"])
code, value = int(msg["code"]), msg["value"]
if code in MODIFIER_CODES:
(self._combo_mods.add if value else self._combo_mods.discard)(MODIFIER_CODES[code])
mod = MODIFIER_CODES[code]
if value == 1:
self._combo_tap = None if self._combo_mods else mod
(self._combo_mods.add if value else self._combo_mods.discard)(mod)
if value == 0 and self._combo_tap == mod:
self._end_shortcut_capture(str(mod)) # the relay's modifier tap
elif value == 1 and code < BTN_MISC:
self._save_shortcut("+".join(str(c) for c in sorted(self._combo_mods) + [code]),
self._capture_combo)
self._capture_combo = ""
self._combo_mods = set()
self.shortcutCaptureChanged.emit()
self._end_shortcut_capture("+".join(str(c) for c in sorted(self._combo_mods) + [code]))
elif value == 1:
self._combo_tap = None # a mouse button: no tap
elif t == "event":
self.activity.emit(msg["id"])
if (self._capture_id and msg["id"] == self._capture_id and msg["type"] == "key"
@@ -890,6 +905,8 @@ class Backend(QObject):
# --- key combinations ("key_bindings") ---
def comboName(self, combo):
parts = [int(c) for c in combo.split("+") if c.isdigit()]
if len(parts) == 1 and parts[0] in MODIFIER_NAMES:
return MODIFIER_NAMES[parts[0]] + " tap"
return "+".join(MODIFIER_NAMES.get(c) or self.codeName(c).removeprefix("KEY_").title() for c in parts)
@Property("QVariantList", notify=mappingsChanged)
@@ -900,7 +917,8 @@ class Backend(QObject):
@Property("QVariantList", constant=True)
def shortcutActions(self):
return [{"value": a, "text": action_label(a)} for a in SHORTCUT_ACTIONS + profile_actions()]
return [{"value": a, "text": action_label(a)} for a in SHORTCUT_ACTIONS + profile_actions()] + \
[{"value": COMMAND, "text": "Run a command…"}]
@Property(bool, notify=shortcutCaptureChanged)
def capturingShortcut(self):
@@ -911,6 +929,7 @@ class Backend(QObject):
if shortcut_mappable(action):
self._capture_combo = action
self._combo_mods = set()
self._combo_tap = None
self._send("watch 60")
self.shortcutCaptureChanged.emit()
@@ -919,6 +938,13 @@ class Backend(QObject):
self._capture_combo = ""
self.shortcutCaptureChanged.emit()
def _end_shortcut_capture(self, combo):
self._save_shortcut(combo, self._capture_combo)
self._capture_combo = ""
self._combo_mods = set()
self._combo_tap = None
self.shortcutCaptureChanged.emit()
def _save_shortcut(self, combo, action):
rules = read_json(RULES_PATH)
rules["key_bindings"] = dict(key_bindings(rules), **{combo: action})
+27 -6
View File
@@ -192,7 +192,7 @@ Kirigami.ApplicationWindow {
wrapMode: Text.Wrap
opacity: 0.7
text: "Move or press a device to see which row it is. 3D pointer: grabbed, drives the SteamVR pointer. "
+ "Pass through: left alone (a Meta tap toggles the dashboard if META_DASHBOARD=1). Ignore: left alone."
+ "Pass through: left alone, but its key combinations (Keyboard page) work. Ignore: left alone."
}
}
}
@@ -545,7 +545,11 @@ Kirigami.ApplicationWindow {
delegate: RowLayout {
required property var modelData
Kirigami.FormData.label: modelData.label + ":"
Controls.Label { text: modelData.actionLabel }
Controls.Label {
text: modelData.actionLabel
elide: Text.ElideRight
Layout.maximumWidth: Kirigami.Units.gridUnit * 24
}
Controls.ToolButton {
icon.name: "edit-delete"
display: Controls.AbstractButton.IconOnly
@@ -568,16 +572,33 @@ Kirigami.ApplicationWindow {
}
Controls.Button {
text: backend.capturingShortcut ? "Press the keys… (Cancel)" : "Set keys…"
enabled: backend.capturingShortcut || shortcutAction.currentValue !== "command:"
|| shortcutCommand.text.trim() !== ""
onClicked: backend.capturingShortcut ? backend.cancelShortcutCapture()
: backend.startShortcutCapture(shortcutAction.currentValue)
: backend.startShortcutCapture(
shortcutAction.currentValue === "command:"
? "command:" + shortcutCommand.text.trim()
: shortcutAction.currentValue)
}
}
Controls.TextField {
id: shortcutCommand
Kirigami.FormData.label: "Command:"
visible: shortcutAction.currentValue === "command:"
placeholderText: "e.g. ft-layout use Work"
Layout.preferredWidth: Kirigami.Units.gridUnit * 24
}
Controls.Label {
Layout.maximumWidth: Kirigami.Units.gridUnit * 30
wrapMode: Text.WordWrap
text: "Hold the modifiers (Ctrl, Alt, Shift, Meta), then press the key, on any keyboard. The "
+ "combination's last key isn't typed; the modifiers still reach the app. Meta+Shift+F floats "
+ "the desktop window under the pointer in VR, or puts it back, until you remove or change it."
text: "Hold the modifiers (Ctrl, Alt, Shift, Meta), then press the key, on any keyboard. Or tap "
+ "one modifier on its own. The combination's last key isn't typed; the modifiers still reach "
+ "the app. While you're in Steam or a game, they see the keys too. Until you remove or change "
+ "them, a Meta tap opens the Steam menu (or closes the dashboard) instead of the desktop's "
+ "launcher, and Meta+Shift+F floats the desktop window under the pointer in VR, or puts it "
+ "back. A command runs with sh as the input relay's service, outside the desktop's session, "
+ "with Frametop's ft-layout, ft-float and ft-steam on its path; its output goes to the "
+ "relay's log (journalctl --user -u frametop-input-relay)."
opacity: 0.7
font: Kirigami.Theme.smallFont
}
-168
View File
@@ -1,168 +0,0 @@
#!/usr/bin/env python3
"""Native KDE mouse control, entirely over local IPC; no SteamVR client needed."""
import argparse
import importlib.util
import json
import math
import os
from pathlib import Path
import re
import socket
import tempfile
SCREENS = '\0ft_screens'
RELAY = '\0frametop_relay'
HELPER = '\0ft_pointer_helper'
CONF = Path.home() / '.config/frametop.conf'
def request(address, command):
with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) as sock:
sock.bind('')
sock.settimeout(1)
sock.sendto(command.encode(), address)
result = sock.recv(8192).decode()
if result.startswith('error'):
raise RuntimeError(result)
return result
def query(address, command):
try:
result = json.loads(request(address, command))
if not isinstance(result, dict): raise ValueError('Expected an object')
return result
except (OSError, ValueError, RuntimeError) as error:
return {'available': False, 'reason': str(error)}
def status():
result = {'relay': query(RELAY, 'mouse-mode?'), 'desktop': query(SCREENS, 'mouse?'),
'pointer': query(HELPER, 'pointerstatus'),
'controllerFallback': query(SCREENS, 'mouse-presence?'),
'desktopPolicy': query(SCREENS, 'desktop-policy?')}
result['ready'] = bool(result['relay'].get('supported') and result['desktop'].get('supported')
and 'desktopInput' in result['pointer'])
result['active'] = bool(result['ready'] and result['relay'].get('mode') == 'desktop'
and result['desktop'].get('mode') == 'desktop'
and result['pointer'].get('desktopInput'))
return result
def require_ready():
result = status()
if not result['ready']:
raise RuntimeError('Native mouse support is not loaded in all three components. '
'Inspect status; the updated desktop and input helpers must be running.')
return result
def layout_outputs():
spec = importlib.util.spec_from_file_location('mouse_layout', Path(__file__).resolve().parents[1] / 'layout/ft_layout.py')
ft = importlib.util.module_from_spec(spec)
spec.loader.exec_module(ft)
env = ft.nested_env()
if env is None: raise RuntimeError('No running FrameTop KDE desktop')
outs = ft.outputs(env)
if not outs or not any(o.get('enabled') for o in outs):
raise RuntimeError('KDE output geometry is unavailable')
return outs
def configure(outs):
commands = []
for index, output in enumerate(outs, 1):
if not output.get('enabled'):
commands.append(f'mouse-output-off {index}')
continue
scale = float(output['scale'])
if not math.isfinite(scale) or not .25 <= scale <= 8:
raise ValueError('Invalid KDE output scale')
w, h = (math.ceil(output['size'][k]/scale) for k in ('width', 'height'))
x, y = (float(output['pos'][k]) for k in ('x', 'y'))
if not all(math.isfinite(v) for v in (x, y)) or not (1 <= w <= 16384 and 1 <= h <= 16384):
raise ValueError('Invalid KDE output geometry')
commands.append(f'mouse-layout {index} {x:g} {y:g} {w} {h} {scale:g}')
# Validate the entire geometry before sending any updates.
for command in commands:
if request(SCREENS, command) != 'ok': raise RuntimeError('Mouse layout rejected')
def save_mode(mode, path=None):
path = Path(path or CONF)
text = path.read_text() if path.exists() else ''
pattern = re.compile(r'^\s*MOUSE_MODE\s*=.*$', re.M)
text = pattern.sub('MOUSE_MODE='+mode, text) if pattern.search(text) else text.rstrip('\n')+'\nMOUSE_MODE='+mode+'\n'
path.parent.mkdir(parents=True, exist_ok=True)
fd, temporary = tempfile.mkstemp(prefix=path.name+'.', dir=path.parent)
try:
os.fchmod(fd, (path.stat().st_mode & 0o777) if path.exists() else 0o600)
with os.fdopen(fd, 'w') as file:
file.write(text)
file.flush()
os.fsync(file.fileno())
os.replace(temporary, path)
finally:
if os.path.exists(temporary): os.unlink(temporary)
def switch(mode, save=False):
if mode == 'desktop':
require_ready()
configure(layout_outputs())
# Spatial recovery intentionally works with an absent desktop.
result = query(RELAY, 'mouse-mode '+mode)
if result.get('ok') is not True: raise RuntimeError(result.get('error', result.get('reason', 'Relay rejected mode')))
if save:
try: save_mode(mode)
except OSError as error:
raise RuntimeError(f'Live mode is {mode}, but saving the preference failed: {error}') from error
return dict(mode=mode, saved=save)
def main():
parser = argparse.ArgumentParser(description=__doc__)
sub = parser.add_subparsers(dest='command', required=True)
sub.add_parser('status')
sub.add_parser('layout')
policy = sub.add_parser('policy', help='Choose desktop mouse/pointer ownership (runtime only)')
policy.add_argument('mode', choices=('mouse', 'pointer', 'last-active'))
for name in ('desktop', 'spatial'):
sub.add_parser(name).add_argument('--save', action='store_true')
pos = sub.add_parser('position')
pos.add_argument('screen', type=int)
pos.add_argument('x', type=float)
pos.add_argument('y', type=float)
center = sub.add_parser('center')
center.add_argument('screen', type=int)
args = parser.parse_args()
try:
if args.command == 'status': result = status()
elif args.command == 'policy':
require_ready()
if request(SCREENS, 'desktop-policy '+args.mode) != 'ok':
raise RuntimeError('Desktop policy rejected')
result = query(SCREENS, 'desktop-policy?')
elif args.command in ('desktop', 'spatial'): result = switch(args.command, args.save)
else:
require_ready()
outs = layout_outputs()
if args.command == 'layout':
configure(outs)
result = {'outputs': outs}
else:
if not 1 <= args.screen <= len(outs): raise ValueError('Invalid screen number')
if args.command == 'center':
output = outs[args.screen-1]
x, y = (math.ceil(output['size'][k]/output['scale'])/2 for k in ('width', 'height'))
else: x, y = args.x, args.y
if not all(math.isfinite(v) for v in (x, y)): raise ValueError('Position must be finite')
if request(SCREENS, f'mouse-position {args.screen} {x:g} {y:g}') != 'ok':
raise RuntimeError('Mouse position rejected')
result = dict(screen=args.screen, x=x, y=y)
print(json.dumps(result, indent=2))
except (OSError, RuntimeError, ValueError, KeyError) as error:
parser.exit(1, str(error)+'\n')
if __name__ == '__main__': main()
+96 -209
View File
@@ -14,8 +14,7 @@ keyboard node for its extra buttons). Roles, from ~/.config/frametop-input.json
(written by the Frametop Input Settings app):
pointer grabbed; drives the universal 3D mouse (default for devices with a mouse node)
passthrough keys go to the desktop; grabbed only while typing goes there, otherwise only observed,
e.g. for the Meta dashboard shortcut (default for keyboards; the shortcut is off
unless META_DASHBOARD=1 is in ~/.config/frametop.conf)
for the key combinations below (default for keyboards)
ignore not grabbed, only observed for identification in the settings app
Buttons and keys of pointer devices go through a per-device map to actions
(left, right, middle, back, scroll_up, scroll_down, dashboard, recenter,
@@ -32,19 +31,26 @@ layout_reset = put the desktop screens back in their saved layout, screens_toggl
keyboard_toggle = open or close Frametop's keyboard, float_toggle = float the desktop window under the
pointer (else the active one) in VR, or put it back if it floats, dock_all = put every floating
window back (both to ft-floatd, @frametop_float), profile:NAME = switch to that profile (ft-layout
use NAME: its screens and apps; docs/profiles.md), key = pass through as a key, none).
use NAME: its screens and apps; docs/profiles.md), steam_menu = open the SteamVR dashboard on
Steam's menu, or close the dashboard (steam/ft-steam menu, through Steam's UI), command:CMD = run
CMD with sh -c (on the host, as this service: its environment, output to its log, and
COMMAND_PATH, so ft-layout, ft-float and ft-steam need no path), key = pass through as a key, none).
Frame controller buttons can be mapped too ("controller_buttons": {"right/a": action} in the
rules file; any action but key and the gaze ones, GAZE_ACTIONS: gaze mode is a mouse feature,
docs/gaze-controllers.md). So can key combinations on any keyboard ("key_bindings":
{"29+56+34": action}, evdev codes joined by "+", modifiers first and left-hand codes for
either side, here Ctrl+Alt+G): the combination does the action, and its last key isn't typed.
A rules file without "key_bindings" gets DEFAULT_KEY_BINDINGS (Meta+J: gaze_left, Meta+K:
gaze_right, Meta+Shift+F: float_toggle); one with its own, even an empty one, doesn't. The float
actions work without pointer mode too. A combination with Meta also sends the desktop an F24 press
and Meta's release right away: so letting go of Meta doesn't open Plasma's launcher, and a gaze
click isn't Meta+click (KWin's window move and resize). Another key while Meta is still held gives
the desktop Meta back. The controllers aren't input devices here, only SteamVR sees
A modifier on its own ("125": Meta) is a tap: pressed and released with no other key, mouse
button, or scroll in between; the desktop gets an F24 press before its release, so Plasma's
launcher doesn't open on a Meta tap that's bound. A rules file without "key_bindings" gets
DEFAULT_KEY_BINDINGS (Meta tap: steam_menu, Meta+J: gaze_left, Meta+K: gaze_right, Meta+Shift+F:
float_toggle); one with its own, even an empty one, doesn't. The float, profile, Steam menu and
command actions work without pointer mode too. A combination with Meta also sends the desktop an
F24 press and Meta's release right away: so letting go of Meta doesn't open Plasma's launcher, and
a gaze click isn't Meta+click (KWin's window move and resize). Another key while Meta is still
held gives the desktop Meta back. While typing goes to Steam, keyboards aren't grabbed, so Steam
or the game sees a combination's keys too. The controllers aren't input devices here, only SteamVR sees
them, so the pointer helper reads them with SteamVR input and sends "vrbtn <button> 1|0".
It only takes the buttons the relay tells it to ("vrbind <button>..." to @ft_pointer_helper,
sent on start, reload, and when the helper says "vrhello"), and only while no game runs,
@@ -115,7 +121,6 @@ import atexit
import errno
import fcntl
import json
import math
import os
import select
import signal
@@ -130,7 +135,8 @@ EV_SYN, EV_KEY, EV_REL, EV_MSC = 0x00, 0x01, 0x02, 0x04
SYN_REPORT = 0
BTN_MISC, KEY_MAX = 0x100, 0x2FF
KEY_A = 30
REL_X, REL_Y, REL_HWHEEL, REL_WHEEL, REL_MAX = 0x00, 0x01, 0x06, 0x08, 0x0F
REL_X, REL_Y, REL_WHEEL, REL_MAX = 0x00, 0x01, 0x08, 0x0F
SCROLLS = {0x06, REL_WHEEL, 0x0B, 0x0C} # REL_HWHEEL, REL_WHEEL and their _HI_RES
BTN_LEFT, BTN_RIGHT, BTN_MIDDLE, BTN_SIDE, BTN_EXTRA = 0x110, 0x111, 0x112, 0x113, 0x114
KEY_LEFTMETA, KEY_RIGHTMETA = 125, 126
KEY_VOLUMEDOWN, KEY_VOLUMEUP = 114, 115
@@ -190,12 +196,14 @@ RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
ACTIONS = ("left", "right", "middle", "back", "scroll_up", "scroll_down", "dashboard", "recenter",
"pointer_toggle", "follow_toggle", "gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right",
"gaze_quickcal", "sens_up", "sens_down", "layout_reset", "screens_toggle", "keyboard_toggle", "float_toggle",
"dock_all", "key", "none")
"dock_all", "steam_menu", "key", "none")
# Gaze mode is a mouse feature: these never come from a controller button (docs/gaze-controllers.md).
GAZE_ACTIONS = ("gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right", "gaze_quickcal")
# Key combinations a rules file without "key_bindings" gets: Meta+J and Meta+K click at the gaze
# (free on the Frametop desktop, and apps don't use Meta), Meta+Shift+F floats a window.
DEFAULT_KEY_BINDINGS = {"125+36": "gaze_left", "125+37": "gaze_right", "42+125+33": "float_toggle"}
# Key combinations a rules file without "key_bindings" gets: a Meta tap opens Steam's menu, Meta+J
# and Meta+K click at the gaze (free on the Frametop desktop, and apps don't use Meta),
# Meta+Shift+F floats a window.
DEFAULT_KEY_BINDINGS = {"125": "steam_menu", "125+36": "gaze_left", "125+37": "gaze_right",
"42+125+33": "float_toggle"}
KEY_F24 = 194 # sent to the desktop with a Meta combination (see the top)
# Key combinations ("key_bindings"): modifiers, each side's code folded into the left one's.
MODIFIERS = {29: 29, 97: 29, 42: 42, 54: 42, 56: 56, 100: 56, 125: 125, 126: 125}
@@ -212,16 +220,23 @@ FLOAT = "\0frametop_float" # ft-floatd, floating windows in the Frametop deskto
# Actions for ft-floatd ("float_toggle", "dock_all"): they don't need pointer mode.
FLOAT_ACTIONS = {"float_toggle": b"float pointer", "dock_all": b"dock all"}
PROFILE = "profile:" # "profile:NAME": switch to that profile (doesn't need pointer mode either)
COMMAND = "command:" # "command:CMD": run CMD (nor does this)
def known_action(a):
return a in ACTIONS or (isinstance(a, str) and a.startswith(PROFILE) and len(a) > len(PROFILE))
return a in ACTIONS or (isinstance(a, str) and any(a.startswith(p) and a[len(p):].strip()
for p in (PROFILE, COMMAND)))
def needs_pointer(a):
return a not in FLOAT_ACTIONS and not a.startswith(PROFILE)
return a not in FLOAT_ACTIONS and a != "steam_menu" and not a.startswith((PROFILE, COMMAND))
KEYS = "\0frametop_keys" # keys of keyboards grabbed for the desktop, for other readers
FT_LAYOUT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "layout", "ft-layout")
REPO = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
FT_LAYOUT = os.path.join(REPO, "layout", "ft-layout")
FT_STEAM = os.path.join(REPO, "steam", "ft-steam")
# Commands ("command:CMD") find Frametop's own tools (ft-layout, ft-float, ft-steam) on their PATH.
COMMAND_PATH = ":".join([os.path.join(REPO, d) for d in ("layout", "float", "steam")]
+ [os.environ.get("PATH", "/usr/local/bin:/usr/bin")])
DEFAULT_BUTTONS = {BTN_LEFT: "left", BTN_RIGHT: "right", BTN_MIDDLE: "middle",
BTN_SIDE: "back", BTN_EXTRA: "back"}
@@ -591,75 +606,6 @@ class Pointer:
return 0.02 if any(t is not None for t in pending) else 0.5
def desktop_mouse_present(nodes):
"""Only a physical mouse actually routed by this relay owns the desktop."""
return any(n.is_mouse and n.role == "pointer" and n.grabbed for n in nodes)
def ipc_request(address, command, timeout=.2):
"""Bounded local request/reply, with no OpenVR client or desktop wake."""
with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) as sock:
sock.bind("")
sock.settimeout(timeout)
sock.sendto(command.encode(), address)
reply = sock.recv(8192).decode()
if reply.startswith("error"):
raise RuntimeError(reply)
return reply
class DesktopMouse:
"""Physical mouse -> nested KDE seat. Never sends to the VR pointer helper."""
def __init__(self, speed=1):
if not math.isfinite(speed) or not .05 <= speed <= 10:
raise ValueError("Desktop mouse speed must be between .05 and 10")
self.speed = speed
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
self.dx = self.dy = 0
self.wheel = [0, 0]
self.hires = [0, 0]
self.has_hires = [False, False]
self.held = set()
self.sent = self.failed = 0
def send(self, command):
try:
self.sock.sendto(command.encode(), SCREENS)
self.sent += 1
except OSError:
self.failed += 1
def motion(self, code, value):
if code == REL_X: self.dx += value
elif code == REL_Y: self.dy += value
elif code in (REL_HWHEEL, REL_WHEEL): self.wheel[code == REL_WHEEL] += value
elif code in (12, 11):
axis = int(code == 11)
self.hires[axis] += value
self.has_hires[axis] = True
def flush(self):
if self.dx or self.dy:
self.send(f"mouse-move {self.dx*self.speed:.6f} {self.dy*self.speed:.6f}")
self.dx = self.dy = 0
wheel = [self.hires[i]/120 if self.has_hires[i] else self.wheel[i] for i in (0, 1)]
if any(wheel): self.send(f"mouse-wheel {wheel[0]:.6f} {wheel[1]:.6f}")
self.wheel = [0, 0]; self.hires = [0, 0]; self.has_hires = [False, False]
def button(self, code, value):
if not BTN_LEFT <= code <= 279 or value not in (0, 1): return
self.flush()
self.send(f"mouse-button {code} {value}")
if value: self.held.add(code)
else: self.held.discard(code)
def release_all(self):
self.dx = self.dy = 0
self.wheel = [0, 0]; self.hires = [0, 0]; self.has_hires = [False, False]
for code in sorted(self.held): self.send(f"mouse-button {code} 0")
self.held.clear()
class Node:
"""One input event node: a candidate device (mouse or keyboard, USB or Bluetooth),
or any other device with volume keys (candidate False, role "volume")."""
@@ -742,23 +688,13 @@ def main():
# desktop_until: typing goes to the Frametop desktop until then (ft-screens says so
# every second); typing_applied: the grabs match that as of the last apply_roles().
# vr_capture_until: every controller button is taken until then (the settings app capturing one).
state = {"pointer": None, "rules": {}, "meta_dashboard": False, "share_keys": False,
"desktop_until": 0.0, "typing_applied": None, "vr_capture_until": 0.0,
"desktop_mouse": None, "mouse_mode": "spatial", "desktop_speed": 1.0,
"controller_fallback": True}
state = {"pointer": None, "rules": {}, "share_keys": False,
"desktop_until": 0.0, "typing_applied": None, "vr_capture_until": 0.0}
def load_config():
conf = read_config()
state["rules"] = read_rules()
state["meta_dashboard"] = conf.get("META_DASHBOARD", "0") == "1"
state["share_keys"] = conf.get("SHARE_KEYS", "0") == "1"
state["mouse_mode"] = "desktop" if conf.get("MOUSE_MODE") == "desktop" else "spatial"
state["controller_fallback"] = conf.get("DESKTOP_CONTROLLER_FALLBACK", "1") == "1"
try:
speed = float(conf.get("DESKTOP_MOUSE_SPEED", "1"))
state["desktop_speed"] = speed if math.isfinite(speed) and .05 <= speed <= 10 else 1
except ValueError:
state["desktop_speed"] = 1
if conf.get("POINTER", "0") == "1":
p = state["pointer"] or Pointer(0.03, 30)
p.sensitivity = float(conf.get("POINTER_SENSITIVITY", "0.03"))
@@ -771,15 +707,13 @@ def main():
log("pointer mode off: pointer devices feed the virtual mouse and keyboard")
load_config()
meta_down = False # Meta pressed with no other key yet: a tap toggles the dashboard
tap = None # the modifier (folded, MODIFIERS) pressed alone, with nothing since: its release is a tap
screens_sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_NONBLOCK)
last_typing = 0.0 # the helper was last told of a key then (see "typing" at the top)
def vr_bind(now):
"""Tell the pointer helper which controller buttons to take from games."""
if state["desktop_mouse"]:
buttons = "-" # leave every physical controller button to SteamVR/games
elif now < state["vr_capture_until"]:
if now < state["vr_capture_until"]:
buttons = "*"
else:
state["vr_capture_until"] = 0.0
@@ -802,7 +736,7 @@ def main():
reply(addr, {"t": "event", "id": VR_DEVICE, "path": "", "name": "Steam Frame controllers",
"type": "vr", "code": button, "value": value})
action = state["rules"]["controller_buttons"].get(button)
if not state["desktop_mouse"] and (state["pointer"] or (action and not needs_pointer(action))) and known_action(action) \
if (state["pointer"] or (action and not needs_pointer(action))) and known_action(action) \
and action not in ("key", "none") and action not in GAZE_ACTIONS:
do_action(action, value, now, button.split("/")[0])
@@ -829,7 +763,7 @@ def main():
def do_action(action, value, now, source="mouse"):
"""A mapped mouse or controller button, or key combination (pointer mode only, but
for FLOAT_ACTIONS and profiles)."""
for the actions needs_pointer() says don't)."""
if action == "keyboard_toggle":
if value == 1 and vr_keyboard_mode() != "never":
vr_keyboard("toggle")
@@ -839,6 +773,17 @@ def main():
subprocess.Popen([FT_LAYOUT, "use", action[len(PROFILE):]], stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=True)
log(action)
elif action == "steam_menu":
if value == 1:
# Talks to Steam's UI for a moment; its errors go to this service's log.
subprocess.Popen([FT_STEAM, "menu"], stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
start_new_session=True)
log(action)
elif action.startswith(COMMAND):
if value == 1:
subprocess.Popen(["sh", "-c", action[len(COMMAND):]], stdin=subprocess.DEVNULL,
env=dict(os.environ, PATH=COMMAND_PATH), start_new_session=True)
log(action)
elif action in FLOAT_ACTIONS:
if value == 1:
try:
@@ -857,14 +802,20 @@ def main():
def key_binding(code, value, now):
"""A key from a keyboard: does it complete a key combination ("key_bindings")? True if
it was taken for one (then it isn't typed)."""
nonlocal meta_down
nonlocal tap
if code in MODIFIERS:
(held_modifiers.add if value else held_modifiers.discard)(MODIFIERS[code])
mod = MODIFIERS[code]
if value == 1:
tap = mod if not held_modifiers and not combos_down else None
(held_modifiers.add if value else held_modifiers.discard)(mod)
if code in (KEY_LEFTMETA, KEY_RIGHTMETA):
(held_meta.add if value else held_meta.discard)(code)
if value == 0 and code in meta_hidden:
meta_hidden.discard(code)
return True # the desktop already had it come up (below)
if value == 0 and tap == mod:
tap = None
modifier_tap(mod, now)
return False
if value == 1 and code not in combos_down and meta_hidden:
# Another key while Meta is still held after a combination: the desktop gets Meta
@@ -886,10 +837,9 @@ def main():
combos_down[code] = action
if held_meta - meta_hidden:
# The desktop saw Meta go down. Another key in between keeps its release from
# opening Plasma's launcher (and Meta from toggling the dashboard here), and Meta
# comes up there now: KWin takes Meta with a mouse button for moving or resizing
# windows, which would swallow a gaze click. Its real release is dropped (above).
meta_down = False
# opening Plasma's launcher, and Meta comes up there now: KWin takes Meta with a
# mouse button for moving or resizing windows, which would swallow a gaze click.
# Its real release is dropped (above).
to_screens(KEY_F24, 1)
to_screens(KEY_F24, 0)
for c in sorted(held_meta - meta_hidden):
@@ -900,6 +850,20 @@ def main():
log(f"key combination {combo}: {action}")
return True
def modifier_tap(mod, now):
"""A modifier pressed and released alone: its binding, if it has one ("125": Meta tap)."""
action = state["rules"]["key_bindings"].get(str(mod))
if not known_action(action) or action in ("key", "none"):
return
# The desktop gets a key in between before the release goes there, so the tap isn't one
# there too: a Meta tap would open Plasma's launcher.
to_screens(KEY_F24, 1)
to_screens(KEY_F24, 0)
if state["pointer"] or not needs_pointer(action):
do_action(action, 1, now, "keyboard")
do_action(action, 0, now, "keyboard")
log(f"key combination {mod}: {action}")
screens_down = set() # keys the desktop was told went down and not yet up (see reconcile_desktop_keys)
@@ -983,12 +947,6 @@ def main():
return "pointer" if has_mouse else "passthrough"
def release_held(node):
if node.role == "pointer" and state["desktop_mouse"]:
state["desktop_mouse"].release_all()
for code in node.held:
if code < BTN_MISC: to_screens(code, 0)
node.held.clear()
return
for code in node.held:
if node.role == "passthrough":
share_key(node, code, 0)
@@ -1082,40 +1040,6 @@ def main():
except OSError:
watchers.pop(addr, None)
def set_mouse_mode(mode, now):
if any(keys_down(n) for n in nodes.values() if n.role == "pointer"):
raise RuntimeError("Release mouse buttons before switching modes")
native = state["desktop_mouse"]
if native and native.held:
raise RuntimeError("Release mouse buttons before switching modes")
if mode == "desktop":
ipc_request(SCREENS, "mouse-mode desktop")
try:
ipc_request(HELPER, "desktopmouse on")
except (OSError, RuntimeError):
ipc_request(SCREENS, "mouse-mode spatial")
raise
if native:
state["mouse_mode"] = mode
return
state["desktop_mouse"] = DesktopMouse(state["desktop_speed"])
if state["pointer"]:
state["pointer"].active = False
state["pointer"].dx = state["pointer"].dy = 0
state["pointer"].claim_at = state["pointer"].claim_release = None
state["pointer"].scroll_until = state["pointer"].system_at = state["pointer"].system_release = None
else:
# An absent desktop must not prevent explicit spatial recovery.
try: ipc_request(SCREENS, "mouse-mode spatial")
except OSError: pass
ipc_request(HELPER, "desktopmouse off")
if native:
native.release_all(); native.sock.close()
state["desktop_mouse"] = None
state["mouse_mode"] = mode
vr_bind(now)
log(f"mouse mode: {mode}")
def handle_control(now):
while True:
try:
@@ -1152,18 +1076,6 @@ def main():
reply(addr, {"t": "devices", "pointer_mode": state["pointer"] is not None,
"actions": ACTIONS,
"nodes": [n.describe() for n in nodes.values() if n.candidate]})
elif cmd == "mouse-mode?":
native = state["desktop_mouse"]
reply(addr, {"supported": True, "mode": "desktop" if native else "spatial",
"requestedMode": state["mouse_mode"], "speed": state["desktop_speed"],
"heldButtons": sorted(native.held) if native else [],
"sent": native.sent if native else 0, "sendFailures": native.failed if native else 0})
elif cmd == "mouse-mode" and len(words) == 2 and words[1] in ("desktop", "spatial"):
try:
set_mouse_mode(words[1], now)
reply(addr, {"mode": words[1], "ok": True})
except (OSError, RuntimeError, ValueError) as error:
reply(addr, {"error": str(error), "ok": False})
elif cmd == "watch":
seconds = float(words[1]) if len(words) > 1 else 30
watchers[addr] = now + min(seconds, 600)
@@ -1211,9 +1123,6 @@ def main():
if now >= next_scan:
next_scan = now + 1.0
reconcile_desktop_keys()
if (state["mouse_mode"] == "desktop") != bool(state["desktop_mouse"]):
try: set_mouse_mode(state["mouse_mode"], now)
except (OSError, RuntimeError, ValueError): pass # desktop may be starting; retry on the next scan
current = {}
for name in os.listdir("/dev/input"):
if name.startswith("event"):
@@ -1240,18 +1149,11 @@ def main():
added = True
if added:
apply_roles()
# Presence, not activity: an idle connected mouse keeps its own seat.
present = desktop_mouse_present(nodes.values())
try:
ipc_request(SCREENS, "mouse-presence %d %d" %
(present, state["controller_fallback"]))
except (OSError, RuntimeError, ValueError):
pass # Older/missing compositor: preserve existing native mode.
ready, _, _ = select.select(list(nodes) + [control], [], [],
volume.timeout(now, pointer.timeout() if pointer else 0.5))
now = time.monotonic()
if pointer and not state["desktop_mouse"]:
if pointer:
pointer.tick(now)
volume.tick(now)
if state["vr_capture_until"] and now >= state["vr_capture_until"]:
@@ -1279,50 +1181,39 @@ def main():
if etype in (EV_KEY, EV_REL):
broadcast(node, etype, VOLUME_ORIGINAL.get(code, code) if node.remapped else code,
value, now)
if (etype == EV_KEY and value == 1 and code not in MODIFIERS) or (etype == EV_REL and code in SCROLLS):
tap = None # a key, button or scroll in between: a modifier's release isn't a tap
if etype == EV_KEY and ((node.remapped and code in VOLUME_ORIGINAL)
or (node.grabbed and code in VOLUME_STANDIN)):
volume.key(fd, code, value, now)
if value == 1:
meta_down = False # Meta used as a modifier, not a tap
continue
if node.role == "volume":
continue
if node.role != "pointer":
# Observed only, unless typing goes to the desktop. With META_DASHBOARD=1,
# a Meta tap on any keyboard toggles the dashboard.
# Observed only, unless typing goes to the desktop. Key combinations work on
# any pass-through keyboard.
if (node.role == "passthrough" and etype == EV_KEY and node.grabbed
and code < BTN_MISC and value in (0, 1)):
# Shared as pressed, key combinations included: the Meta release a
# combination keeps from the desktop must still reach frame-voice, or
# it waits for that release before typing anything.
share_key(node, code, value)
if value:
node.held.add(code)
else:
node.held.discard(code)
if node.role == "passthrough" and etype == EV_KEY and code < BTN_MISC and key_binding(code, value, now):
continue
if node.role == "passthrough" and etype == EV_KEY:
if value == 1 and code < BTN_MISC and now - last_typing >= 0.25:
last_typing = now
screens_sock.sendto(b"typing", HELPER)
try:
screens_sock.sendto(b"typing", HELPER)
except OSError:
pass # helper not running (SteamVR not running)
to_screens(code, value)
if node.grabbed and code < BTN_MISC and value in (0, 1):
share_key(node, code, value)
if value:
node.held.add(code)
else:
node.held.discard(code)
if (pointer and not state["desktop_mouse"] and state["meta_dashboard"] and node.role == "passthrough"
and etype == EV_KEY):
if code in (KEY_LEFTMETA, KEY_RIGHTMETA):
if value == 1:
meta_down = True
elif value == 0 and meta_down:
meta_down = False
pointer.dashboard()
elif value == 1:
meta_down = False # Meta used as a modifier, not a tap
continue
if etype == EV_KEY:
if state["desktop_mouse"]:
if code >= BTN_LEFT:
state["desktop_mouse"].button(code, value)
else:
to_screens(code, value)
if value: node.held.add(code)
else: node.held.discard(code)
continue
action = buttons.get(str(code), DEFAULT_BUTTONS.get(code, "key"))
if pointer and action not in ("key", "none"):
do_action(action, value, now)
@@ -1337,16 +1228,12 @@ def main():
else:
node.held.discard(code)
elif etype == EV_REL:
if state["desktop_mouse"]:
state["desktop_mouse"].motion(code, value)
elif pointer:
if pointer:
pointer.motion(code, value, now)
else:
mouse.emit(etype, code, value)
elif etype == EV_SYN and code == SYN_REPORT:
if state["desktop_mouse"]:
state["desktop_mouse"].flush()
elif pointer:
if pointer:
pointer.flush()
mouse.sync()
keyboard.sync()
+219
View File
@@ -0,0 +1,219 @@
#!/usr/bin/env python3
"""Offline test of the input relay's key combinations and modifier taps.
Runs the relay's main() against a fake pass-through keyboard and a fake mouse (pipes), with
every socket it sends to renamed, no uinput devices, no grabs, and ft-steam swapped for a
logger. Nothing reaches the live desktop, SteamVR, or the running relay, so it's safe next
to them. Rules come from the test, not ~/.config/frametop-input.json.
input/test/keys-test.py [RELAY] (default: input/input-relay.py next to this folder)
"""
import fcntl
import importlib.util
import os
import shutil
import socket
import struct
import sys
import tempfile
import threading
import time
HERE = os.path.dirname(os.path.abspath(__file__))
relay_path = sys.argv[1] if len(sys.argv) > 1 else os.path.join(HERE, "..", "input-relay.py")
tag = f"ft_keys_test_{os.getpid()}"
with open(relay_path) as f:
src = f.read().replace('"\\0frametop_relay"', f'"\\0{tag}_relay"')
relay = importlib.util.module_from_spec(importlib.util.spec_from_loader("relay", loader=None))
relay.__file__ = os.path.abspath(relay_path)
exec(compile(src, relay_path, "exec"), relay.__dict__)
for name in ("SCREENS", "HELPER", "FLOAT", "GAZED", "KEYS"):
setattr(relay, name, f"\0{tag}_{name.lower()}")
OUT = tempfile.mkdtemp(prefix="ft-keys-test-")
steam_log, cmd_log = f"{OUT}/steam.log", f"{OUT}/cmd.log"
relay.FT_STEAM = f"{OUT}/ft-steam"
with open(relay.FT_STEAM, "w") as f:
f.write(f'#!/bin/sh\necho "$@" >> {steam_log}\n')
os.chmod(relay.FT_STEAM, 0o755)
class NoDevice:
def __init__(self, *args, **kwargs):
pass
def emit(self, *args):
pass
def sync(self):
pass
relay.Virtual = NoDevice
relay.log = lambda *args: None # the relay's own log
bindings = {"now": None} # the rules' key_bindings; None: the relay's defaults
def read_rules(path=None):
rules = {"devices": {}, "buttons": {}, "controller_buttons": {}}
rules["key_bindings"] = dict(relay.DEFAULT_KEY_BINDINGS if bindings["now"] is None else bindings["now"])
return rules
relay.read_rules = read_rules
relay.read_config = lambda path=None: {"POINTER": "0"}
# The fake devices: /dev/input/event900 (keyboard) and event901 (mouse), each a pipe.
kb_r, kb_w = os.pipe()
ms_r, ms_w = os.pipe()
for fd in (kb_r, ms_r):
os.set_blocking(fd, False)
FAKE = {"/dev/input/event900": kb_r, "/dev/input/event901": ms_r}
HELD = {kb_r: set(), ms_r: set()} # what EVIOCGKEY says each holds
class Inode:
st_ino = 1
fake_os = type(os)("os")
fake_os.__dict__.update(os.__dict__)
fake_os.listdir = lambda path: ["event900", "event901"] if path == "/dev/input" else os.listdir(path)
fake_os.stat = lambda path, *a, **k: Inode() if path in FAKE else os.stat(path, *a, **k)
relay.os = fake_os
def ioctl(fd, request, arg=0, *rest):
if fd in HELD and request == relay.EVIOCGKEY:
for c in HELD[fd]:
arg[c // 8] |= 1 << (c % 8)
return 0
return fcntl.ioctl(fd, request, arg, *rest)
fake_fcntl = type(fcntl)("fcntl")
fake_fcntl.__dict__.update(fcntl.__dict__)
fake_fcntl.ioctl = ioctl
relay.fcntl = fake_fcntl
def probe(path):
if FAKE[path] == kb_r:
return relay.Node(path, kb_r, "test keyboard", relay.BUS_USB, 1, 2, "", False, True)
return relay.Node(path, ms_r, "test mouse", relay.BUS_USB, 3, 4, "", True, False)
relay.probe = probe
screens = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
screens.bind(relay.SCREENS)
screens.settimeout(0.05)
EVENT = struct.Struct("llHHi")
def send(fd, etype, code, value):
os.write(fd, EVENT.pack(0, 0, etype, code, value) + EVENT.pack(0, 0, 0, 0, 0))
time.sleep(0.03)
def key(code, value, fd=kb_w):
held = HELD[ms_r if fd == ms_w else kb_r]
(held.add if value else held.discard)(code)
send(fd, relay.EV_KEY, code, value)
def typed():
"""What the desktop was sent since the last call."""
got = []
while True:
try:
got.append(screens.recv(256).decode())
except socket.timeout:
return got
def use(key_bindings):
bindings["now"] = key_bindings
c = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
c.bind("")
c.settimeout(2)
c.sendto(b"reload", f"\0{tag}_relay")
c.recv(4096)
time.sleep(0.05)
typed()
def lines(path, wait=0.3):
time.sleep(wait)
try:
with open(path) as f:
return f.read().split()
except OSError:
return []
failures = []
def check(label, got, want):
ok = got == want
print(("ok " if ok else "FAIL ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
if not ok:
failures.append(label)
META, RMETA, SHIFT, CTRL, RCTRL, A, F, J = 125, 126, 42, 29, 97, 30, 33, 36
F24 = ["key 194 1", "key 194 0"]
def tests():
time.sleep(1.5) # the relay's first device scan
typed()
key(META, 1); key(META, 0)
check("Meta tap (default): Steam menu", lines(steam_log), ["menu"])
check("Meta tap: the desktop gets F24 before Meta's release", typed(), ["key 125 1"] + F24 + ["key 125 0"])
key(RMETA, 1); key(RMETA, 2); key(RMETA, 2); key(RMETA, 0)
check("right Meta held (autorepeat), then released: a tap", lines(steam_log), ["menu", "menu"])
typed()
key(META, 1); key(A, 1); key(A, 0); key(META, 0)
check("Meta+A: no tap", lines(steam_log), ["menu", "menu"])
check("Meta+A (unbound) is typed as is", typed(), ["key 125 1", "key 30 1", "key 30 0", "key 125 0"])
key(META, 1); key(relay.BTN_LEFT, 1, ms_w); key(relay.BTN_LEFT, 0, ms_w); key(META, 0)
check("Meta+click: no tap", lines(steam_log), ["menu", "menu"])
key(META, 1); send(ms_w, relay.EV_REL, relay.REL_WHEEL, 1); key(META, 0)
check("Meta+scroll: no tap", lines(steam_log), ["menu", "menu"])
key(META, 1); key(SHIFT, 1); key(SHIFT, 0); key(META, 0)
check("Meta+Shift: no tap", lines(steam_log), ["menu", "menu"])
typed()
key(META, 1); key(J, 1); key(J, 0); key(META, 0)
check("Meta+J (default gaze click): no tap", lines(steam_log), ["menu", "menu"])
check("Meta+J: F24, Meta up early, its real release dropped", typed(), ["key 125 1"] + F24 + ["key 125 0"])
use({"29+42+33": f"command:echo combo >> {cmd_log}", "125": "none"})
key(CTRL, 1); key(SHIFT, 1); key(F, 1); key(F, 0); key(SHIFT, 0); key(CTRL, 0)
check("Ctrl+Shift+F runs its command", lines(cmd_log), ["combo"])
check("the combination's last key isn't typed", [k for k in typed() if k.startswith("key 33 ")], [])
key(META, 1); key(META, 0)
check("Meta tap bound to none: nothing", lines(steam_log), ["menu", "menu"])
check("Meta tap bound to none: no F24", typed(), ["key 125 1", "key 125 0"])
use({"29": f"command:command -v ft-steam ft-layout ft-float | wc -l >> {cmd_log}"})
key(RCTRL, 1); key(RCTRL, 0)
check("right Ctrl tap runs its command, which finds Frametop's tools", lines(cmd_log), ["combo", "3"])
check("Ctrl tap: F24 before Ctrl's release", typed(), ["key 97 1"] + F24 + ["key 97 0"])
check("an empty command isn't an action", relay.known_action("command: "), False)
check("steam_menu and commands work without pointer mode",
(relay.needs_pointer("steam_menu"), relay.needs_pointer("command:ls")), (False, False))
print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
shutil.rmtree(OUT, ignore_errors=True)
os._exit(1 if failures else 0)
threading.Thread(target=tests, daemon=True).start()
sys.argv = [relay_path, "--no-grab"]
relay.main()
-147
View File
@@ -1,147 +0,0 @@
"""Offline physical mouse tests: no input devices, desktop or SteamVR accessed."""
import importlib.machinery
import importlib.util
import math
from pathlib import Path
import tempfile
import unittest
from types import SimpleNamespace
from unittest.mock import patch
def module(name, file):
loader = importlib.machinery.SourceFileLoader(name, str(Path(__file__).with_name(file)))
spec = importlib.util.spec_from_loader(name, loader)
result = importlib.util.module_from_spec(spec)
spec.loader.exec_module(result)
return result
relay = module('desktop_relay', 'input-relay.py')
ctl = module('mouse_control', 'ft-mousectl')
class DesktopMouseTests(unittest.TestCase):
def test_presence_requires_a_routed_grabbed_physical_mouse(self):
def node(mouse=True, role='pointer', grabbed=True):
return SimpleNamespace(is_mouse=mouse, role=role, grabbed=grabbed)
self.assertFalse(relay.desktop_mouse_present([]))
for n in (node(mouse=False), node(role='ignore'), node(role='passthrough'), node(grabbed=False)):
self.assertFalse(relay.desktop_mouse_present([n]))
self.assertTrue(relay.desktop_mouse_present([node(mouse=False), node()]))
def mouse(self, speed=1):
with patch.object(relay.socket, 'socket'):
m = relay.DesktopMouse(speed)
m.messages = []
m.send = m.messages.append
return m
def test_motion_precedes_click_at_new_location(self):
m = self.mouse(.5)
m.motion(relay.REL_X, 12)
m.motion(relay.REL_Y, -8)
m.button(272, 1)
m.button(272, 0)
m.flush()
self.assertEqual(m.messages, ['mouse-move 6.000000 -4.000000', 'mouse-button 272 1', 'mouse-button 272 0'])
def test_hires_replaces_companion_legacy_in_both_orders(self):
for events in ([(8, 1), (11, 120)], [(11, 120), (8, 1)]):
m = self.mouse()
for code, value in events: m.motion(code, value)
m.flush()
self.assertEqual(m.messages, ['mouse-wheel 0.000000 1.000000'])
def test_fractional_scroll_has_no_pulse_timer(self):
m = self.mouse()
for value in (15, 15, -30):
m.motion(11, value); m.flush()
self.assertEqual(m.messages, ['mouse-wheel 0.000000 0.125000', 'mouse-wheel 0.000000 0.125000', 'mouse-wheel 0.000000 -0.250000'])
def test_thumb_and_main_wheels_are_independent(self):
m = self.mouse()
m.motion(6, -2); m.motion(8, 3); m.motion(11, 60); m.flush()
self.assertEqual(m.messages, ['mouse-wheel -2.000000 0.500000'])
m.motion(8, -1); m.flush()
self.assertEqual(m.messages[-1], 'mouse-wheel 0.000000 -1.000000')
def test_hotplug_releases_all_buttons_without_stale_motion(self):
m = self.mouse()
m.button(272, 1); m.button(275, 1)
m.motion(0, 20); m.motion(8, 1)
m.release_all(); m.flush()
self.assertEqual(m.messages, ['mouse-button 272 1', 'mouse-button 275 1', 'mouse-button 272 0', 'mouse-button 275 0'])
self.assertFalse(m.held)
def test_button_repeat_is_ignored(self):
m = self.mouse()
m.button(272, 2); m.button(280, 1)
self.assertEqual(m.messages, [])
def test_native_messages_never_target_virtual_controller(self):
with patch.object(relay.socket, 'socket') as sock:
m = relay.DesktopMouse()
m.motion(0, 3); m.motion(8, 1); m.button(272, 1); m.release_all()
self.assertTrue(sock.return_value.sendto.call_args_list)
for call in sock.return_value.sendto.call_args_list:
self.assertEqual(call.args[1], relay.SCREENS)
def test_bad_speed_rejected_before_socket_created(self):
with patch.object(relay.socket, 'socket') as sock:
for speed in (math.nan, math.inf, 0, 11):
with self.assertRaises(ValueError): relay.DesktopMouse(speed)
sock.assert_not_called()
class ControlTests(unittest.TestCase):
def test_old_components_cannot_trigger_partial_activation(self):
with patch.object(ctl, 'status', return_value={'ready': False}), patch.object(ctl, 'configure') as configure, patch.object(ctl, 'save_mode') as save:
with self.assertRaisesRegex(RuntimeError, 'not loaded'): ctl.switch('desktop', True)
configure.assert_not_called(); save.assert_not_called()
def test_rejected_switch_does_not_persist(self):
with patch.object(ctl, 'require_ready'), patch.object(ctl, 'layout_outputs'), patch.object(ctl, 'configure'), patch.object(ctl, 'query', return_value={'ok': False, 'error': 'held buttons'}), patch.object(ctl, 'save_mode') as save:
with self.assertRaisesRegex(RuntimeError, 'held buttons'): ctl.switch('desktop', True)
save.assert_not_called()
def test_spatial_recovery_does_not_need_kde(self):
with patch.object(ctl, 'layout_outputs') as layout, patch.object(ctl, 'query', return_value={'ok': True}) as query:
self.assertEqual(ctl.switch('spatial'), {'mode': 'spatial', 'saved': False})
layout.assert_not_called(); query.assert_called_once_with(ctl.RELAY, 'mouse-mode spatial')
def test_failed_persistence_reports_actual_live_mode(self):
with patch.object(ctl, 'query', return_value={'ok': True}), patch.object(ctl, 'save_mode', side_effect=OSError('read only')):
with self.assertRaisesRegex(RuntimeError, 'Live mode is spatial'): ctl.switch('spatial', True)
def test_scaled_outputs_use_actual_logical_geometry(self):
outs = [{'enabled': True, 'pos': {'x': -1280, 'y': 20}, 'size': {'width': 1920, 'height': 1080}, 'scale': 1.5}, {'enabled': False}]
with patch.object(ctl, 'request', return_value='ok') as request:
ctl.configure(outs)
self.assertEqual([c.args for c in request.call_args_list], [(ctl.SCREENS, 'mouse-layout 1 -1280 20 1280 720 1.5'), (ctl.SCREENS, 'mouse-output-off 2')])
def test_invalid_later_output_does_not_partially_change_geometry(self):
outs = [{'enabled': True, 'pos': {'x': 0, 'y': 0}, 'size': {'width': 1920, 'height': 1080}, 'scale': 1}, {'enabled': True, 'scale': 0}]
with patch.object(ctl, 'request') as request:
with self.assertRaises(ValueError): ctl.configure(outs)
request.assert_not_called()
def test_save_preserves_other_preferences_and_permissions(self):
with tempfile.TemporaryDirectory() as root:
path = Path(root) / 'frametop.conf'
path.write_text('# user settings\nPOINTER=1\nMOUSE_MODE=spatial\nMETA_DASHBOARD=0\n')
path.chmod(0o600)
ctl.save_mode('desktop', path)
self.assertEqual(path.read_text(), '# user settings\nPOINTER=1\nMOUSE_MODE=desktop\nMETA_DASHBOARD=0\n')
self.assertEqual(path.stat().st_mode & 0o777, 0o600)
def test_failed_atomic_save_preserves_old_file(self):
with tempfile.TemporaryDirectory() as root:
path = Path(root) / 'frametop.conf'
path.write_text('MOUSE_MODE=spatial\n')
with patch.object(ctl.os, 'replace', side_effect=OSError('failed')):
with self.assertRaises(OSError): ctl.save_mode('desktop', path)
self.assertEqual(path.read_text(), 'MOUSE_MODE=spatial\n')
self.assertEqual(list(Path(root).iterdir()), [path])
if __name__ == '__main__': unittest.main()
-24
View File
@@ -941,29 +941,6 @@ def send_scales(outs):
log(f"screen {i + 1}: scale: {reply}")
except RuntimeError as e:
log(f"scale: {e}")
finally:
if 'sock' in locals(): sock.sock.close()
# Native mouse coordinates follow the actual logical KWin output layout.
# Capability probing keeps this compatible with older ft-screens builds.
try:
with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) as probe:
probe.bind(''); probe.settimeout(1)
probe.sendto(b'mouse?', SCREENS)
if not json.loads(probe.recv(8192)).get('supported'): return
sock = screens_socket()
try:
for i, o in enumerate(outs):
if not o.get('enabled', True):
sock.ask(f'mouse-output-off {i+1}')
continue
scale = float(o.get('scale', 1))
size, pos = o.get('size', {}), o.get('pos', {})
w = math.ceil(size['width']/scale - 1e-6)
h = math.ceil(size['height']/scale - 1e-6)
sock.ask(f"mouse-layout {i+1} {pos.get('x',0)} {pos.get('y',0)} {w} {h} {scale:g}")
finally: sock.sock.close()
except (OSError, ValueError, RuntimeError, KeyError):
pass # old compositor, closing desktop, or a drag in progress; next layout refresh retries
def logical_size(o):
@@ -1021,7 +998,6 @@ def apply_scales():
if moves:
subprocess.run(["kscreen-doctor", *moves], capture_output=True, env=env, timeout=20)
args += moves
send_scales(outputs(env))
return args
+3 -33
View File
@@ -857,10 +857,9 @@ int main() {
overlay->FindOverlay("system.pointer", &systemPointer);
Vec3 pivot, tiltOrigin, lastPoint, lastOrigin, lastAim{0, 0, -1};
Basis tiltBasis{};
bool desktopInput = false; // native desktop input never claims a SteamVR hand
bool headsetOff = false; // nobody is wearing the headset (see the main loop)
auto wake = [&](Clock::time_point t) {
if (t < noWakeUntil || headsetOff || desktopInput) return;
if (t < noWakeUntil || headsetOff) return;
wokeAt = t;
active = true;
recenter = true;
@@ -1274,7 +1273,7 @@ int main() {
}
// Hand gestures in the last two minutes keep it the same way.
const bool handsRecent = handsOn && handUsed != Clock::time_point{} && t - handUsed < std::chrono::minutes(2);
const bool awake = (gazeOn || handsRecent) && !inGame && !headsetOff && !desktopInput;
const bool awake = (gazeOn || handsRecent) && !inGame && !headsetOff;
if (awake != gazeAwake || t - gazeAwakeAt > std::chrono::seconds(5)) {
if (awake != gazeAwake)
std::printf("%s keeps the pointer: %s\n", gazeOn ? "gaze" : "hand use",
@@ -1366,34 +1365,6 @@ int main() {
reply(controllerButtons.Status());
continue;
}
if (std::strcmp(buf, "pointerstatus") == 0) {
char msg[256];
const int role = ours == vr::k_unTrackedDeviceIndexInvalid ? 0 :
int(vr::VRSystem()->GetControllerRoleForTrackedDeviceIndex(ours));
std::snprintf(msg, sizeof msg,
"{\"active\":%s,\"headsetOff\":%s,\"gazeMode\":%s,\"handRole\":%d,\"desktopInput\":%s}",
active ? "true" : "false", headsetOff ? "true" : "false",
gazeOn ? "true" : "false", role, desktopInput ? "true" : "false");
reply(msg);
continue;
}
if (std::strncmp(buf, "desktopmouse ", 13) == 0) {
if (std::strcmp(buf+13, "on") && std::strcmp(buf+13, "off")) {
reply("error desktopmouse on|off");
continue;
}
desktopInput = std::strcmp(buf+13, "on") == 0;
if (desktopInput) {
if (leftHeld) releaseLeft();
active = claimPending = claimHeld = false;
overlay->HideOverlay(cursor); overlay->HideOverlay(marker);
SendTo(out, "ft_pointer", "scroll 0 0");
SendTo(out, "ft_pointer", "btn a 0");
SendTo(out, "ft_pointer", "hide");
}
reply("ok");
continue;
}
if (std::strncmp(buf, "overlays", 8) == 0) {
overlays.Request(sender, senderLen); // answered from the list's thread
continue;
@@ -1781,7 +1752,6 @@ int main() {
hp = std::asin(std::clamp(f.y, -1.0, 1.0)) * 180 / M_PI;
return true;
};
if (desktopInput) keyPresses.clear();
for (const KeyPress &k : keyPresses) {
(k.right ? keyRightDown : keyLeftDown) = k.down;
if (!k.down) {
@@ -1871,7 +1841,7 @@ int main() {
handBaseline = true, handOpens = handFile.opens;
}
// Not over a VR game (unless the dashboard is up), and not with the headset off.
const bool allowed = !desktopInput && !headsetOff && (!inGame || overlay->IsDashboardVisible());
const bool allowed = !headsetOff && (!inGame || overlay->IsDashboardVisible());
for (int k = 1; k >= 0; --k) // grips first: one takes over a pinch
for (int s = 0; s < 2; ++s) {
const fh_pinch_t &g = slots[k][s];
+8 -332
View File
@@ -25,11 +25,8 @@
// COMMAND run with WAYLAND_DISPLAY set to our socket (e.g. the Frametop session)
// Runs in the dev container (wlroots 0.20); KWin connects from the host.
#define _GNU_SOURCE
#include "controller-fallback.h"
#include "desktop-handoff.h"
#include <drm_fourcc.h>
#include <linux/input-event-codes.h>
#include <math.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
@@ -61,11 +58,8 @@
#include <wlr/util/log.h>
#include "vr.h"
#include "desktop-mouse.h"
#include "desktop-cursor.h"
#include "cursor-cache.h"
#define MAX_SCREENS FT_MOUSE_OUTPUTS // screens and spare outputs
#define MAX_SCREENS 24 // screens and spare outputs
struct config {
int width, height;
@@ -110,23 +104,6 @@ struct server {
struct wl_list buffers; // tracked_buffer
struct wl_event_source *tick;
struct screen *pointer_focus;
bool desktop_mouse, cursor_ready, cursor_frame_pending;
struct ft_controller_fallback controller_fallback;
bool controller_desktop;
struct ft_handoff handoff;
struct ft_event controller_position[MAX_SCREENS];
bool controller_position_known[MAX_SCREENS];
unsigned long mouse_events_ignored;
uint32_t controller_buttons;
struct ft_cursor_cache cursor_cache;
const char *cursor_error;
const char *cursor_buffer;
struct wlr_buffer *cursor_held;
struct ft_mouse mouse;
struct wlr_surface *cursor_surface;
int cursor_hx, cursor_hy;
struct wl_listener request_cursor, cursor_commit, cursor_destroy;
unsigned long mouse_motions, mouse_clicks, mouse_wheels, vr_mouse_ignored;
pid_t child;
// Where typing goes: the screens after a click on one, Steam after a click on another
// panel. The input relay grabs the keyboards while it's the screens (see keys_update).
@@ -224,8 +201,6 @@ static void screen_destroy(struct wl_listener *l, void *data) {
if (sc->held) wlr_buffer_unlock(sc->held);
ft_vr_screen_destroy(sc->index);
if (sc->server->pointer_focus == sc) sc->server->pointer_focus = NULL;
sc->server->mouse.outputs[sc->index].enabled = false;
if (sc->server->mouse.grab == sc->index) sc->server->mouse.buttons = 0, sc->server->mouse.grab = -1;
if (sc->decoration) wl_list_remove(&sc->decoration_destroy.link);
sc->server->screens[sc->index] = NULL;
wl_list_remove(&sc->commit.link);
@@ -300,7 +275,7 @@ static void new_decoration(struct wl_listener *l, void *data) {
static void panel_key(struct server *s, uint32_t code, bool pressed);
static void deliver_pointer_event(const struct ft_event *e, void *data) {
static void handle_vr_event(const struct ft_event *e, void *data) {
struct server *s = data;
if (e->type == FT_QUIT) {
wl_display_terminate(s->display);
@@ -360,11 +335,11 @@ static void deliver_pointer_event(const struct ft_event *e, void *data) {
}
if (e->dy != 0)
wlr_seat_pointer_notify_axis(s->seat, t, WL_POINTER_AXIS_VERTICAL_SCROLL, e->dy * 15,
(int32_t)llround(e->dy * 120), WL_POINTER_AXIS_SOURCE_WHEEL,
(int32_t)(e->dy * 120), WL_POINTER_AXIS_SOURCE_WHEEL,
WL_POINTER_AXIS_RELATIVE_DIRECTION_IDENTICAL);
if (e->dx != 0)
wlr_seat_pointer_notify_axis(s->seat, t, WL_POINTER_AXIS_HORIZONTAL_SCROLL, e->dx * 15,
(int32_t)llround(e->dx * 120), WL_POINTER_AXIS_SOURCE_WHEEL,
(int32_t)(e->dx * 120), WL_POINTER_AXIS_SOURCE_WHEEL,
WL_POINTER_AXIS_RELATIVE_DIRECTION_IDENTICAL);
break;
case FT_LEAVE:
@@ -379,185 +354,11 @@ static void deliver_pointer_event(const struct ft_event *e, void *data) {
wlr_seat_pointer_notify_frame(s->seat);
}
static void set_desktop_owner(struct server *s, bool pointer) {
if (pointer == s->controller_desktop) return;
// Claims are refused during a held drag. Legacy disconnect recovery may
// still revoke a stale controller, releasing its buttons safely.
for (unsigned bit = 0; bit < 8; ++bit)
if (s->controller_buttons & (1u << bit))
wlr_seat_pointer_notify_button(s->seat, now_ms(), BTN_LEFT+bit,
WL_POINTER_BUTTON_STATE_RELEASED);
s->controller_buttons = 0;
wlr_seat_pointer_notify_clear_focus(s->seat);
wlr_seat_pointer_notify_frame(s->seat);
s->pointer_focus = NULL;
s->controller_desktop = pointer;
wlr_log(WLR_INFO, "desktop owner %s (policy %s)", pointer ? "pointer" : "mouse",
ft_handoff_name(s->handoff.policy));
ft_vr_cursor_move(-1, 0, 0, 1, false);
}
static void update_controller_fallback(struct server *s) {
bool active = s->desktop_mouse && !s->mouse.buttons &&
ft_fallback_active(&s->controller_fallback, now_ms());
if (s->desktop_mouse && s->handoff.policy != FT_PREFER_MOUSE) {
active = s->controller_desktop;
if (!s->mouse.buttons && !s->controller_buttons &&
(s->handoff.policy == FT_PREFER_POINTER ||
ft_fallback_active(&s->controller_fallback, now_ms()))) active = true;
}
set_desktop_owner(s, active);
}
static bool claim_desktop_mouse(struct server *s) {
if (!ft_handoff_claim(&s->handoff, false, now_ms(), s->mouse.buttons, s->controller_buttons)) {
++s->mouse_events_ignored;
return false;
}
ft_fallback_presence(&s->controller_fallback, true, s->controller_fallback.enabled, now_ms());
set_desktop_owner(s, false);
return true;
}
static void handle_vr_event(const struct ft_event *e, void *data) {
struct server *s = data;
// Events here are panel content, not SteamVR or separate screen grab bars.
bool desktop = e->screen >= 0 && e->screen < s->n_config;
if (desktop && e->type == FT_MOTION) {
s->controller_position[e->screen] = *e;
s->controller_position_known[e->screen] = true;
}
update_controller_fallback(s);
if (s->desktop_mouse && desktop) {
bool pointer_event = e->type == FT_MOTION || e->type == FT_BUTTON ||
e->type == FT_SCROLL || e->type == FT_LEAVE;
uint32_t bit = e->type == FT_BUTTON && e->button >= BTN_LEFT && e->button < BTN_LEFT+8
? 1u << (e->button-BTN_LEFT) : 0;
bool action = (e->type == FT_BUTTON && e->pressed && bit) ||
(e->type == FT_SCROLL && (e->dx != 0 || e->dy != 0));
if (action && s->handoff.policy != FT_PREFER_MOUSE &&
ft_handoff_claim(&s->handoff, true, now_ms(), s->mouse.buttons, s->controller_buttons)) {
set_desktop_owner(s, true);
if (e->type == FT_SCROLL && s->controller_position_known[e->screen])
deliver_pointer_event(&s->controller_position[e->screen], s);
}
if (pointer_event && (!s->controller_desktop || s->mouse.buttons ||
(e->type == FT_BUTTON && !e->pressed && !(s->controller_buttons & bit)) ||
(e->type == FT_LEAVE && s->controller_buttons))) {
++s->vr_mouse_ignored; return;
}
if (bit) {
if (e->pressed) s->controller_buttons |= bit;
else s->controller_buttons &= ~bit;
}
}
deliver_pointer_event(e, data);
}
static void desktop_motion(struct server *s) {
int index = ft_mouse_target(&s->mouse);
if (index < 0 || !s->screens[index]) return;
const struct ft_mouse_output *o = &s->mouse.outputs[index];
struct ft_event event = {.type = FT_MOTION, .screen = index,
.x = (s->mouse.x-o->x)*o->scale, .y = (s->mouse.y-o->y)*o->scale};
deliver_pointer_event(&event, s);
const struct ft_mouse_output *display = &s->mouse.outputs[s->mouse.screen];
ft_vr_cursor_move(s->mouse.screen, (s->mouse.x-display->x)*display->scale,
(s->mouse.y-display->y)*display->scale, display->scale, s->desktop_mouse);
}
static void cursor_hide(struct server *s) {
ft_vr_cursor_image(NULL, 0, 0, 0, 0, 1);
if (s->cursor_held) wlr_buffer_unlock(s->cursor_held);
s->cursor_held = NULL;
}
static void cursor_update(struct server *s) {
struct wlr_surface *surface = s->cursor_surface;
struct wlr_buffer *buffer = surface ? ft_cursor_cached(&s->cursor_cache, surface) : NULL;
s->cursor_ready = false;
s->cursor_error = "no cached KDE cursor buffer";
if (surface) s->cursor_frame_pending = true;
s->cursor_buffer = "none";
if (!buffer) { cursor_hide(s); return; }
if (buffer->width < 1 || buffer->height < 1 || buffer->width > 512 || buffer->height > 512) {
cursor_hide(s); return;
}
int scale = surface->current.scale > 0 ? surface->current.scale : 1;
if (scale > 8 || s->cursor_hx < -512 || s->cursor_hy < -512 || s->cursor_hx > 512 || s->cursor_hy > 512) {
s->cursor_error = "invalid cursor scale or hotspot";
cursor_hide(s); return;
}
struct wlr_dmabuf_attributes dma;
if (wlr_buffer_get_dmabuf(buffer, &dma)) {
s->cursor_buffer = "dmabuf";
struct ft_dmabuf b = {.width = dma.width, .height = dma.height, .format = dma.format,
.modifier = dma.modifier, .n_planes = dma.n_planes};
for (int i = 0; i < dma.n_planes && i < 4; ++i) {
b.offset[i] = dma.offset[i]; b.stride[i] = dma.stride[i]; b.fd[i] = dma.fd[i];
}
track_buffer(s, buffer);
s->cursor_ready = ft_vr_cursor_dmabuf(buffer, &b, s->cursor_hx*scale, s->cursor_hy*scale, scale);
s->cursor_error = s->cursor_ready ? "" : "SteamVR rejected GPU cursor";
if (s->cursor_ready) {
wlr_buffer_lock(buffer);
if (s->cursor_held) wlr_buffer_unlock(s->cursor_held);
s->cursor_held = buffer;
} else cursor_hide(s);
return;
}
s->cursor_buffer = "shm";
void *data; uint32_t format; size_t stride;
if (!wlr_buffer_begin_data_ptr_access(buffer, WLR_BUFFER_DATA_PTR_ACCESS_READ, &data, &format, &stride)) {
s->cursor_error = "KDE cursor buffer cannot be read";
cursor_hide(s); return;
}
uint8_t *rgba = malloc((size_t)buffer->width*buffer->height*4);
s->cursor_error = "unsupported cursor format or allocation failed";
if (rgba && ft_cursor_rgba(rgba, data, buffer->width, buffer->height, stride, format)) {
s->cursor_error = "SteamVR rejected cursor texture";
s->cursor_ready = ft_vr_cursor_image(rgba, buffer->width, buffer->height,
s->cursor_hx*scale, s->cursor_hy*scale, scale);
if (s->cursor_ready) s->cursor_error = "";
}
free(rgba);
wlr_buffer_end_data_ptr_access(buffer);
if (!s->cursor_ready) cursor_hide(s);
else if (s->cursor_held) { wlr_buffer_unlock(s->cursor_held); s->cursor_held = NULL; }
}
static void cursor_committed(struct wl_listener *l, void *data) {
struct server *s = wl_container_of(l, s, cursor_commit);
cursor_update(s);
}
static void cursor_destroyed(struct wl_listener *l, void *data) {
struct server *s = wl_container_of(l, s, cursor_destroy);
wl_list_remove(&s->cursor_commit.link); wl_list_remove(&s->cursor_destroy.link);
s->cursor_surface = NULL; s->cursor_ready = false;
cursor_hide(s);
}
static void cursor_requested(struct wl_listener *l, void *data) {
struct server *s = wl_container_of(l, s, request_cursor);
struct wlr_seat_pointer_request_set_cursor_event *e = data;
if (e->seat_client != s->seat->pointer_state.focused_client) return;
if (s->cursor_surface) {
wl_list_remove(&s->cursor_commit.link); wl_list_remove(&s->cursor_destroy.link);
}
s->cursor_surface = e->surface; s->cursor_hx = e->hotspot_x; s->cursor_hy = e->hotspot_y;
if (e->surface) {
s->cursor_commit.notify = cursor_committed;
s->cursor_destroy.notify = cursor_destroyed;
wl_signal_add(&e->surface->events.commit, &s->cursor_commit);
wl_signal_add(&e->surface->events.destroy, &s->cursor_destroy);
}
cursor_update(s);
}
// Tell the input relay where typing goes, on a change and every second: while it's the
// desktop, the relay grabs pass-through keyboards so SteamVR (and the Steam app with
// gamescope's focus) doesn't get the keys too. Without word from us for a few seconds,
// the relay gives the keyboards back, so a closed desktop doesn't keep them.
static void keys_update(struct server *s) {
if (!s->vr) return; // the isolated test desktop must never route live keyboards
if (!s->vr) return; // a test instance leaves the running desktop's keyboards alone
const bool desktop = s->keys_clicked && ft_vr_screens_shown();
const uint32_t t = now_ms();
@@ -576,7 +377,6 @@ static void keys_update(struct server *s) {
// Every ~11 ms (90 Hz): SteamVR events, and frame callbacks for screens that committed.
static int tick(void *data) {
struct server *s = data;
update_controller_fallback(s);
ft_vr_poll(handle_vr_event, s);
if (++s->ticks % 9 == 0) keys_update(s);
if (s->kb_close_at && s->ticks >= s->kb_close_at) {
@@ -588,10 +388,6 @@ static int tick(void *data) {
}
struct timespec now;
clock_gettime(CLOCK_MONOTONIC, &now);
if (s->cursor_surface && s->cursor_frame_pending) {
s->cursor_frame_pending = false;
wlr_surface_send_frame_done(s->cursor_surface, &now);
}
for (int i = 0; i < MAX_SCREENS; ++i) {
struct screen *sc = s->screens[i];
if (sc && sc->frame_pending) {
@@ -717,121 +513,8 @@ static int control_readable(int fd, uint32_t mask, void *data) {
buf[n] = 0;
unsigned code;
int value, index, w, h;
double scale, wheel_x, wheel_y;
double mx, my, mw, mh;
char mode[16], tail;
if (strcmp(buf, "desktop-policy?") == 0) {
snprintf(reply, sizeof reply,
"{\"supported\":true,\"policy\":\"%s\",\"owner\":\"%s\",\"mouseButtons\":%u,\"pointerButtons\":%u,\"mouseEventsIgnored\":%lu}",
ft_handoff_name(s->handoff.policy), s->controller_desktop ? "pointer" : "mouse",
s->mouse.buttons, s->controller_buttons, s->mouse_events_ignored);
} else if (sscanf(buf, "desktop-policy %15s %c", mode, &tail) == 1) {
if (s->mouse.buttons || s->controller_buttons ||
(strcmp(mode, "mouse") && strcmp(mode, "pointer") && strcmp(mode, "last-active")))
snprintf(reply, sizeof reply, "error policy or held button");
else {
s->handoff = (struct ft_handoff){.policy = !strcmp(mode, "last-active") ? FT_LAST_ACTIVE :
!strcmp(mode, "pointer") ? FT_PREFER_POINTER : FT_PREFER_MOUSE};
set_desktop_owner(s, s->handoff.policy == FT_PREFER_POINTER);
update_controller_fallback(s);
snprintf(reply, sizeof reply, "ok");
}
} else if (strcmp(buf, "mouse-presence?") == 0) {
snprintf(reply, sizeof reply,
"{\"supported\":true,\"known\":%s,\"present\":%s,\"enabled\":%s,\"controllerDesktop\":%s,\"fresh\":%s}",
s->controller_fallback.known ? "true" : "false",
s->controller_fallback.present ? "true" : "false",
s->controller_fallback.enabled ? "true" : "false",
s->controller_desktop ? "true" : "false",
s->controller_fallback.known && (uint32_t)(now_ms()-s->controller_fallback.received)<3000 ? "true" : "false");
} else if (sscanf(buf, "mouse-presence %d %d %c", &index, &value, &tail) == 2) {
if ((index != 0 && index != 1) || (value != 0 && value != 1))
snprintf(reply, sizeof reply, "error invalid presence");
else {
ft_fallback_presence(&s->controller_fallback, index, value, now_ms());
update_controller_fallback(s);
snprintf(reply, sizeof reply, "ok");
}
} else if (strcmp(buf, "mouse?") == 0) {
snprintf(reply, sizeof reply,
"{\"supported\":true,\"mode\":\"%s\",\"screen\":%d,\"x\":%.3f,\"y\":%.3f,"
"\"buttons\":%u,\"cursorReady\":%s,\"cursorSource\":\"kde\",\"cursorCache\":true,\"cursorBuffer\":\"%s\",\"cursorError\":\"%s\","
"\"motionEvents\":%lu,\"buttonEvents\":%lu,\"wheelEvents\":%lu,\"controllerEventsIgnored\":%lu}",
s->desktop_mouse ? "desktop" : "spatial", s->mouse.screen+1, s->mouse.x, s->mouse.y,
s->mouse.buttons, s->cursor_ready ? "true" : "false", s->cursor_buffer ? s->cursor_buffer : "none", s->cursor_error ? s->cursor_error : "no cursor request", s->mouse_motions,
s->mouse_clicks, s->mouse_wheels, s->vr_mouse_ignored);
} else if (sscanf(buf, "mouse-output-off %d %c", &index, &tail) == 1) {
if (index < 1 || index > MAX_SCREENS || s->mouse.buttons)
snprintf(reply, sizeof reply, "error invalid mouse output or held button");
else { s->mouse.outputs[index-1].enabled = false; snprintf(reply, sizeof reply, "ok"); }
} else if (sscanf(buf, "mouse-layout %d %lf %lf %lf %lf %lf %c", &index, &mx, &my, &mw, &mh, &scale, &tail) == 6) {
if (index < 1 || index > MAX_SCREENS || !s->screens[index-1] ||
!ft_mouse_output_set(&s->mouse, index-1, mx, my, mw, mh, scale))
snprintf(reply, sizeof reply, "error invalid mouse output or held button");
else { s->scale[index-1] = scale; snprintf(reply, sizeof reply, "ok"); }
} else if (sscanf(buf, "mouse-mode %15s %c", mode, &tail) == 1) {
if (s->mouse.buttons || s->controller_buttons || (strcmp(mode, "desktop") && strcmp(mode, "spatial")))
snprintf(reply, sizeof reply, "error mouse mode or held button");
else if (!strcmp(mode, "desktop") && !ft_mouse_move(&s->mouse, 0, 0))
snprintf(reply, sizeof reply, "error configure mouse layout first");
else {
s->desktop_mouse = !strcmp(mode, "desktop");
if (s->desktop_mouse) desktop_motion(s);
else {
wlr_seat_pointer_notify_clear_focus(s->seat); s->pointer_focus = NULL;
wlr_seat_pointer_notify_frame(s->seat);
ft_vr_cursor_move(-1, 0, 0, 1, false);
}
snprintf(reply, sizeof reply, "ok");
}
} else if (sscanf(buf, "mouse-move %lf %lf %c", &mx, &my, &tail) == 2) {
if (!s->desktop_mouse || (s->vr && !ft_vr_screens_shown()) || !isfinite(mx) || !isfinite(my) || fabs(mx)>16384 || fabs(my)>16384)
snprintf(reply, sizeof reply, "error desktop mouse unavailable or invalid motion");
else { if ((mx != 0 || my != 0) && claim_desktop_mouse(s) && ft_mouse_move(&s->mouse, mx, my)) { desktop_motion(s); ++s->mouse_motions; } snprintf(reply, sizeof reply, "ok"); }
} else if (sscanf(buf, "mouse-position %d %lf %lf %c", &index, &mx, &my, &tail) == 3) {
if (!s->desktop_mouse || s->mouse.buttons || s->controller_buttons || index < 1 || index > MAX_SCREENS ||
!s->mouse.outputs[index-1].enabled || !isfinite(mx) || !isfinite(my) ||
mx < 0 || my < 0 || mx >= s->mouse.outputs[index-1].width || my >= s->mouse.outputs[index-1].height)
snprintf(reply, sizeof reply, "error mouse position or held button");
else {
ft_mouse_position(&s->mouse, s->mouse.outputs[index-1].x+mx, s->mouse.outputs[index-1].y+my);
if (claim_desktop_mouse(s)) desktop_motion(s);
snprintf(reply, sizeof reply, "ok");
}
} else if (sscanf(buf, "mouse-button %u %d %c", &code, &value, &tail) == 2) {
if (!s->desktop_mouse || (value != 0 && value != 1) ||
(s->vr && !ft_vr_screens_shown() && value) || code < BTN_LEFT || code > BTN_TASK)
snprintf(reply, sizeof reply, "error mouse button or hidden desktop");
else if ((!value && !(s->mouse.buttons & (1u << (code-BTN_LEFT)))) || !claim_desktop_mouse(s))
snprintf(reply, sizeof reply, "ok");
else {
desktop_motion(s);
int target = ft_mouse_target(&s->mouse);
const struct ft_mouse_output *o = target >= 0 ? &s->mouse.outputs[target] : NULL;
if (!o || !s->screens[target]) snprintf(reply, sizeof reply, "error no mouse surface");
else {
struct ft_event event = {.type = FT_BUTTON, .screen = target, .button = code, .pressed = value,
.x = (s->mouse.x-o->x)*o->scale, .y = (s->mouse.y-o->y)*o->scale};
deliver_pointer_event(&event, s);
ft_mouse_button(&s->mouse, code, value); ++s->mouse_clicks;
if (!s->mouse.buttons) desktop_motion(s);
snprintf(reply, sizeof reply, "ok");
}
}
} else if (sscanf(buf, "mouse-wheel %lf %lf %c", &wheel_x, &wheel_y, &tail) == 2) {
int target = ft_mouse_target(&s->mouse);
if (!s->desktop_mouse || (s->vr && !ft_vr_screens_shown()) || target < 0 || !s->screens[target] ||
!s->mouse.outputs[target].enabled ||
!isfinite(wheel_x) || !isfinite(wheel_y) || fabs(wheel_x)>120 || fabs(wheel_y)>120)
snprintf(reply, sizeof reply, "error mouse wheel or hidden desktop");
else if ((wheel_x == 0 && wheel_y == 0) || !claim_desktop_mouse(s))
snprintf(reply, sizeof reply, "ok");
else {
desktop_motion(s);
struct ft_event event = {.type = FT_SCROLL, .screen = target, .dx = wheel_x, .dy = -wheel_y};
deliver_pointer_event(&event, s); ++s->mouse_wheels; snprintf(reply, sizeof reply, "ok");
}
} else if (sscanf(buf, "size %d %d %d", &index, &w, &h) == 3) {
double scale;
if (sscanf(buf, "size %d %d %d", &index, &w, &h) == 3) {
// A new resolution for a screen, live: KWin resizes the screen to match. (KWin makes
// it this size times its scale; ft-floatd sends spares' sizes divided by theirs.)
const int min_w = index - 1 < s->n_config ? 320 : 64, min_h = index - 1 < s->n_config ? 200 : 64;
@@ -870,13 +553,11 @@ static int control_readable(int fd, uint32_t mask, void *data) {
else if (strcmp(button, "middle") == 0) e.button = BTN_MIDDLE;
if (got >= 2 && strcmp(what, "leave") == 0) e.type = FT_LEAVE;
else if (got >= 4 && strcmp(what, "move") == 0) e.type = FT_MOTION;
else if (got >= 4 && strcmp(what, "scroll") == 0 && isfinite(x) && isfinite(y) && fabs(x)<=120 && fabs(y)<=120)
e.type = FT_SCROLL, e.dx = x, e.dy = y;
else if (got >= 4 && (strcmp(what, "down") == 0 || strcmp(what, "up") == 0))
e.type = FT_BUTTON, e.pressed = what[0] == 'd';
else index = 0;
if (index < 1 || index > MAX_SCREENS || !s->screens[index - 1]) {
snprintf(reply, sizeof reply, "error input <screen> move|down|up|scroll|leave [x y [button]]");
snprintf(reply, sizeof reply, "error input <screen> move|down|up|leave [x y [button]]");
} else {
handle_vr_event(&e, s);
snprintf(reply, sizeof reply, "ok");
@@ -960,7 +641,6 @@ static bool setup_dmabuf(struct server *s) {
int main(int argc, char **argv) {
struct server s = {0};
ft_mouse_init(&s.mouse);
for (int i = 0; i < MAX_SCREENS; ++i) s.scale[i] = 1;
s.kb_screen = -1;
const char *socket_name = "ft-screens-0", *control_name = "ft_screens";
@@ -1005,8 +685,7 @@ int main(int argc, char **argv) {
s.display = wl_display_create();
s.loop = wl_display_get_event_loop(s.display);
wl_list_init(&s.buffers);
struct wlr_compositor *compositor = wlr_compositor_create(s.display, 6, NULL);
ft_cursor_cache_init(&s.cursor_cache, compositor);
wlr_compositor_create(s.display, 6, NULL);
wlr_subcompositor_create(s.display);
const uint32_t shm_formats[] = {DRM_FORMAT_ARGB8888, DRM_FORMAT_XRGB8888}; // wlroots wants DRM codes
wlr_shm_create(s.display, 1, shm_formats, 2);
@@ -1021,8 +700,6 @@ int main(int argc, char **argv) {
wl_signal_add(&s.decoration->events.new_toplevel_decoration, &s.new_decoration);
s.seat = wlr_seat_create(s.display, "seat0");
s.request_cursor.notify = cursor_requested;
wl_signal_add(&s.seat->events.request_set_cursor, &s.request_cursor);
wlr_keyboard_init(&s.keyboard, &keyboard_impl, "ft-screens-keyboard");
struct xkb_context *xkb = xkb_context_new(XKB_CONTEXT_NO_FLAGS);
struct xkb_keymap *keymap = xkb_keymap_new_from_names(xkb, NULL, XKB_KEYMAP_COMPILE_NO_FLAGS);
@@ -1069,7 +746,6 @@ int main(int argc, char **argv) {
// wlroots asserts that nothing still listens to its globals when they go.
wl_list_remove(&s.new_toplevel.link);
wl_list_remove(&s.new_decoration.link);
ft_cursor_cache_finish(&s.cursor_cache);
ft_vr_shutdown();
wl_display_destroy(s.display);
return 0;
-19
View File
@@ -1,19 +0,0 @@
#ifndef FT_CONTROLLER_FALLBACK_H
#define FT_CONTROLLER_FALLBACK_H
#include <stdbool.h>
#include <stdint.h>
struct ft_controller_fallback {
bool known, present, enabled;
uint32_t received, absent_since;
};
static inline void ft_fallback_presence(struct ft_controller_fallback *f,
bool present, bool enabled, uint32_t now) {
if (!f->known || (uint32_t)(now-f->received) >= 3000 || f->present || present)
f->absent_since = now;
f->known = true; f->present = present; f->enabled = enabled; f->received = now;
}
static inline bool ft_fallback_active(const struct ft_controller_fallback *f, uint32_t now) {
return f->known && f->enabled && !f->present &&
(uint32_t)(now-f->received) < 3000 && (uint32_t)(now-f->absent_since) >= 1000;
}
#endif
-67
View File
@@ -1,67 +0,0 @@
#pragma once
#include <stdlib.h>
#include <wayland-server-core.h>
#include <wlr/types/wlr_buffer.h>
#include <wlr/types/wlr_compositor.h>
// A rendererless compositor has no persistent surface texture. wlroots releases
// current.buffer after each commit callback. KDE commits a cursor before issuing
// set_cursor, so retain small buffers from the first commit, not from set_cursor.
// This also keeps animation/callback-only commits from erasing the last image.
struct ft_cursor_cache;
struct ft_cursor_frame {
struct ft_cursor_cache *cache;
struct wlr_surface *surface;
struct wlr_buffer *buffer;
struct wl_listener commit, destroy;
struct wl_list link;
};
struct ft_cursor_cache {
struct wl_listener new_surface;
struct wl_list frames;
};
static inline struct wlr_buffer *ft_cursor_cached(struct ft_cursor_cache *cache, struct wlr_surface *surface) {
struct ft_cursor_frame *f;
wl_list_for_each(f, &cache->frames, link) if (f->surface == surface) return f->buffer;
return NULL;
}
static inline void ft_cursor_frame_committed(struct wl_listener *listener, void *data) {
(void)data;
struct ft_cursor_frame *f = wl_container_of(listener, f, commit);
if (!(f->surface->current.committed & WLR_SURFACE_STATE_BUFFER)) return;
struct wlr_buffer *buffer = f->surface->current.buffer;
// Desktop DMA-BUFs go straight to SteamVR; do not retain them here.
struct wlr_buffer *next = buffer && buffer->width > 0 && buffer->height > 0 &&
buffer->width <= 512 && buffer->height <= 512 ? wlr_buffer_lock(buffer) : NULL;
if (f->buffer) wlr_buffer_unlock(f->buffer);
f->buffer = next;
}
static inline void ft_cursor_frame_destroyed(struct wl_listener *listener, void *data) {
(void)data;
struct ft_cursor_frame *f = wl_container_of(listener, f, destroy);
wl_list_remove(&f->commit.link); wl_list_remove(&f->destroy.link); wl_list_remove(&f->link);
if (f->buffer) wlr_buffer_unlock(f->buffer);
free(f);
}
static inline void ft_cursor_surface_created(struct wl_listener *listener, void *data) {
struct ft_cursor_cache *cache = wl_container_of(listener, cache, new_surface);
struct wlr_surface *surface = data;
struct ft_cursor_frame *f = calloc(1, sizeof *f);
if (!f) { wl_client_post_no_memory(wl_resource_get_client(surface->resource)); return; }
f->cache = cache; f->surface = surface;
f->commit.notify = ft_cursor_frame_committed;
f->destroy.notify = ft_cursor_frame_destroyed;
wl_signal_add(&surface->events.commit, &f->commit);
wl_signal_add(&surface->events.destroy, &f->destroy);
wl_list_insert(&cache->frames, &f->link);
}
static inline void ft_cursor_cache_init(struct ft_cursor_cache *cache, struct wlr_compositor *compositor) {
wl_list_init(&cache->frames);
cache->new_surface.notify = ft_cursor_surface_created;
wl_signal_add(&compositor->events.new_surface, &cache->new_surface);
}
static inline void ft_cursor_cache_finish(struct ft_cursor_cache *cache) {
wl_list_remove(&cache->new_surface.link);
struct ft_cursor_frame *f, *tmp;
wl_list_for_each_safe(f, tmp, &cache->frames, link) ft_cursor_frame_destroyed(&f->destroy, NULL);
}
-26
View File
@@ -1,26 +0,0 @@
#pragma once
#include <drm_fourcc.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <string.h>
// Shared-memory Wayland cursor images use ARGB with premultiplied alpha.
// SteamVR's raw overlay texture is RGBA. Respect padding and recover RGB at edges.
static inline bool ft_cursor_rgba(uint8_t *out, const void *data, int w, int h,
size_t stride, uint32_t format) {
if (!data || w < 1 || h < 1 || w > 512 || h > 512 || stride < (size_t)w*4 ||
(format != DRM_FORMAT_ARGB8888 && format != DRM_FORMAT_ABGR8888 &&
format != DRM_FORMAT_XRGB8888 && format != DRM_FORMAT_XBGR8888)) return false;
const bool swap = format == DRM_FORMAT_ABGR8888 || format == DRM_FORMAT_XBGR8888;
const bool opaque = format == DRM_FORMAT_XRGB8888 || format == DRM_FORMAT_XBGR8888;
for (int y = 0; y < h; ++y) for (int x = 0; x < w; ++x) {
uint32_t v; memcpy(&v, (const uint8_t *)data + (size_t)y*stride + x*4, 4);
unsigned a = opaque ? 255 : v >> 24;
unsigned rgb[3] = {swap ? v & 255 : (v >> 16) & 255, (v >> 8) & 255,
swap ? (v >> 16) & 255 : v & 255};
uint8_t *p = out + ((size_t)y*w+x)*4;
for (int c = 0; c < 3; ++c) { unsigned n = a ? (rgb[c]*255 + a/2)/a : 0; p[c] = n > 255 ? 255 : n; }
p[3] = a;
}
return true;
}
-24
View File
@@ -1,24 +0,0 @@
#ifndef FT_DESKTOP_HANDOFF_H
#define FT_DESKTOP_HANDOFF_H
#include <stdbool.h>
#include <stdint.h>
enum ft_handoff_policy { FT_PREFER_MOUSE, FT_LAST_ACTIVE, FT_PREFER_POINTER };
struct ft_handoff { enum ft_handoff_policy policy; bool mouse_seen; uint32_t mouse_at; };
// Only deliberate desktop actions call this. Tracking motion never claims a seat.
static inline bool ft_handoff_claim(struct ft_handoff *h, bool pointer,
uint32_t now, uint32_t mouse_buttons,
uint32_t pointer_buttons) {
if (pointer) {
if (mouse_buttons || h->policy == FT_PREFER_MOUSE) return false;
if (h->policy == FT_LAST_ACTIVE && !pointer_buttons && h->mouse_seen && now == h->mouse_at)
return false; // Same millisecond: native mouse wins.
} else {
if (pointer_buttons || h->policy == FT_PREFER_POINTER) return false;
h->mouse_seen = true; h->mouse_at = now;
}
return true;
}
static inline const char *ft_handoff_name(enum ft_handoff_policy p) {
return p == FT_LAST_ACTIVE ? "last-active" : p == FT_PREFER_POINTER ? "pointer" : "mouse";
}
#endif
-61
View File
@@ -1,61 +0,0 @@
#pragma once
#include <math.h>
#include <stdbool.h>
#include <stdint.h>
#define FT_MOUSE_OUTPUTS 24
struct ft_mouse_output { bool enabled; double x, y, width, height, scale; };
struct ft_mouse {
struct ft_mouse_output outputs[FT_MOUSE_OUTPUTS];
double x, y;
uint32_t buttons;
int screen, grab;
bool positioned;
};
static inline void ft_mouse_init(struct ft_mouse *m) { *m = (struct ft_mouse){.screen = -1, .grab = -1}; }
static inline bool ft_mouse_output_set(struct ft_mouse *m, int i, double x, double y,
double w, double h, double scale) {
if (i < 0 || i >= FT_MOUSE_OUTPUTS || m->buttons || !isfinite(x) || !isfinite(y) ||
!isfinite(w) || !isfinite(h) || !isfinite(scale) || fabs(x) > 65536 || fabs(y) > 65536 ||
w < 1 || h < 1 || w > 16384 || h > 16384 || scale < .25 || scale > 8) return false;
m->outputs[i] = (struct ft_mouse_output){true, x, y, w, h, scale};
return true;
}
static inline bool ft_mouse_position(struct ft_mouse *m, double x, double y) {
if (!isfinite(x) || !isfinite(y)) return false;
double best = INFINITY, bx = 0, by = 0;
int screen = -1;
for (int i = 0; i < FT_MOUSE_OUTPUTS; ++i) {
const struct ft_mouse_output *o = &m->outputs[i];
if (!o->enabled) continue;
double px = fmin(fmax(x, o->x), o->x + o->width - 1);
double py = fmin(fmax(y, o->y), o->y + o->height - 1);
double d = (px-x)*(px-x) + (py-y)*(py-y);
if (d < best) best = d, bx = px, by = py, screen = i;
}
if (screen < 0) return false;
m->x = bx; m->y = by; m->screen = screen; m->positioned = true;
return true;
}
static inline bool ft_mouse_move(struct ft_mouse *m, double dx, double dy) {
if (!isfinite(dx) || !isfinite(dy) || fabs(dx) > 16384 || fabs(dy) > 16384) return false;
if (!m->positioned) {
for (int i = 0; i < FT_MOUSE_OUTPUTS; ++i) if (m->outputs[i].enabled) {
const struct ft_mouse_output *o = &m->outputs[i];
if (!ft_mouse_position(m, o->x + o->width/2, o->y + o->height/2)) return false;
break;
}
}
return ft_mouse_position(m, m->x + dx, m->y + dy);
}
// The host seat's implicit grab stays on one KWin output during a drag. Motion
// remains in that output's coordinates, including positions beyond its edges;
// nested KWin adds the output origin and can drag windows across its monitors.
static inline int ft_mouse_target(const struct ft_mouse *m) { return m->buttons ? m->grab : m->screen; }
static inline bool ft_mouse_button(struct ft_mouse *m, unsigned button, bool down) {
if (!m->positioned || button < 272 || button > 279) return false;
uint32_t bit = 1u << (button - 272);
if (down) { if (!m->buttons) m->grab = m->screen; m->buttons |= bit; }
else { m->buttons &= ~bit; if (!m->buttons) m->grab = -1; }
return true;
}
-29
View File
@@ -1,29 +0,0 @@
#include <assert.h>
#include <stdio.h>
#include "../controller-fallback.h"
int main(void) {
struct ft_controller_fallback f = {0};
assert(!ft_fallback_active(&f, 100));
ft_fallback_presence(&f, false, true, 100);
assert(!ft_fallback_active(&f, 1099));
assert(ft_fallback_active(&f, 1100));
ft_fallback_presence(&f, false, true, 1100);
assert(ft_fallback_active(&f, 1100)); // heartbeat doesn't reset debounce
assert(!ft_fallback_active(&f, 4100)); // lost relay fails closed
ft_fallback_presence(&f, false, true, 4100);
assert(!ft_fallback_active(&f, 4100)); // old absence cannot bypass new debounce
assert(ft_fallback_active(&f, 5100));
ft_fallback_presence(&f, true, true, 4200);
assert(!ft_fallback_active(&f, 9000)); // connected but idle is still a mouse
ft_fallback_presence(&f, false, false, 9100);
assert(!ft_fallback_active(&f, 10100)); // disabled preference
ft_fallback_presence(&f, true, true, 10200);
ft_fallback_presence(&f, false, true, 10300);
assert(!ft_fallback_active(&f, 11299));
assert(ft_fallback_active(&f, 11300));
ft_fallback_presence(&f, true, true, 11301);
assert(!ft_fallback_active(&f, 11301)); // reconnect wins immediately
ft_fallback_presence(&f, false, true, UINT32_MAX-500);
assert(ft_fallback_active(&f, 500)); // monotonic counter wrap
puts("disconnect debounce, heartbeat expiry, reconnect and disabled fallback passed");
}
-106
View File
@@ -1,106 +0,0 @@
// Actual rendererless Wayland/wlroots lifecycle. No OpenVR or real desktop.
#define _GNU_SOURCE
#define WLR_USE_UNSTABLE
#include <assert.h>
#include <drm_fourcc.h>
#include <fcntl.h>
#include <signal.h>
#include <stdio.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/socket.h>
#include <sys/wait.h>
#include <unistd.h>
#include <wayland-client.h>
#include <wlr/types/wlr_shm.h>
#include "../cursor-cache.h"
static struct wl_compositor *client_compositor;
static struct wl_shm *client_shm;
static void global(void *data, struct wl_registry *registry, uint32_t name, const char *interface, uint32_t version) {
(void)data; (void)version;
if (!strcmp(interface, "wl_compositor")) client_compositor = wl_registry_bind(registry, name, &wl_compositor_interface, 4);
if (!strcmp(interface, "wl_shm")) client_shm = wl_registry_bind(registry, name, &wl_shm_interface, 1);
}
static void removed(void *data, struct wl_registry *registry, uint32_t name) { (void)data; (void)registry; (void)name; }
static const struct wl_registry_listener registry_listener = {global, removed};
static void stage(struct wl_display *display, int report, int ack, char value) {
assert(wl_display_roundtrip(display) >= 0);
assert(write(report, &value, 1) == 1);
char answer;
assert(read(ack, &answer, 1) == 1 && answer == value);
}
static void client(int fd, int report, int ack) {
alarm(10);
struct wl_display *d = wl_display_connect_to_fd(fd);
assert(d);
struct wl_registry *registry = wl_display_get_registry(d);
wl_registry_add_listener(registry, &registry_listener, NULL);
assert(wl_display_roundtrip(d) >= 0 && client_compositor && client_shm);
int memory = memfd_create("offline-cursor", MFD_CLOEXEC);
assert(memory >= 0 && ftruncate(memory, 600*4) == 0);
uint32_t *pixels = mmap(NULL, 600*4, PROT_READ|PROT_WRITE, MAP_SHARED, memory, 0);
assert(pixels != MAP_FAILED); pixels[0] = 0xff123456;
struct wl_shm_pool *pool = wl_shm_create_pool(client_shm, memory, 600*4);
struct wl_buffer *small = wl_shm_pool_create_buffer(pool, 0, 8, 1, 32, WL_SHM_FORMAT_ARGB8888);
struct wl_buffer *large = wl_shm_pool_create_buffer(pool, 0, 600, 1, 2400, WL_SHM_FORMAT_ARGB8888);
struct wl_surface *surface = wl_compositor_create_surface(client_compositor);
wl_surface_attach(surface, small, 0, 0); wl_surface_commit(surface);
stage(d, report, ack, 1); // image committed before any set_cursor/consumer
wl_surface_commit(surface); stage(d, report, ack, 2); // callback-only commit preserves image
wl_surface_attach(surface, NULL, 0, 0); wl_surface_commit(surface); stage(d, report, ack, 3);
wl_surface_attach(surface, large, 0, 0); wl_surface_commit(surface); stage(d, report, ack, 4);
wl_surface_attach(surface, small, 0, 0); wl_surface_commit(surface); stage(d, report, ack, 5);
wl_surface_destroy(surface); stage(d, report, ack, 6);
wl_buffer_destroy(small); wl_buffer_destroy(large); wl_shm_pool_destroy(pool);
wl_registry_destroy(registry); wl_display_disconnect(d);
munmap(pixels, 600*4); close(memory);
_exit(0);
}
int main(void) {
alarm(10);
struct wl_display *display = wl_display_create();
struct wlr_compositor *compositor = wlr_compositor_create(display, 6, NULL);
uint32_t formats[] = {DRM_FORMAT_ARGB8888, DRM_FORMAT_XRGB8888};
assert(wlr_shm_create(display, 1, formats, 2));
struct ft_cursor_cache cache;
ft_cursor_cache_init(&cache, compositor);
int connection[2], report[2], ack[2];
assert(socketpair(AF_UNIX, SOCK_STREAM|SOCK_CLOEXEC, 0, connection) == 0);
assert(pipe(report) == 0 && pipe(ack) == 0);
pid_t pid = fork(); assert(pid >= 0);
if (!pid) { close(connection[0]); close(report[0]); close(ack[1]); client(connection[1], report[1], ack[0]); }
close(connection[1]); close(report[1]); close(ack[0]);
fcntl(report[0], F_SETFL, O_NONBLOCK);
assert(wl_client_create(display, connection[0]));
struct wl_event_loop *loop = wl_display_get_event_loop(display);
struct wlr_buffer *first = NULL;
for (char expected = 1; expected <= 6;) {
assert(wl_event_loop_dispatch(loop, 10) >= 0); wl_display_flush_clients(display);
char observed;
if (read(report[0], &observed, 1) != 1) continue;
assert(observed == expected);
if (expected == 6) assert(wl_list_empty(&cache.frames));
else {
assert(!wl_list_empty(&cache.frames));
struct ft_cursor_frame *f = wl_container_of(cache.frames.next, f, link);
assert(f->surface->current.buffer == NULL); // wlroots already released it
struct wlr_buffer *buffer = ft_cursor_cached(&cache, f->surface);
if (expected == 3 || expected == 4) assert(buffer == NULL);
else {
assert(buffer && buffer->width == 8 && buffer->height == 1);
if (expected == 1) first = buffer;
if (expected == 2) assert(buffer == first);
void *data; uint32_t format; size_t stride;
assert(wlr_buffer_begin_data_ptr_access(buffer, WLR_BUFFER_DATA_PTR_ACCESS_READ, &data, &format, &stride));
assert(*(uint32_t *)data == 0xff123456 && format == DRM_FORMAT_ARGB8888 && stride == 32);
wlr_buffer_end_data_ptr_access(buffer);
}
}
assert(write(ack[1], &observed, 1) == 1); ++expected;
}
int status; assert(waitpid(pid, &status, 0) == pid && WIFEXITED(status) && WEXITSTATUS(status) == 0);
close(report[0]); close(ack[1]); wl_display_destroy_clients(display);
ft_cursor_cache_finish(&cache); wl_display_destroy(display);
puts("cursor before selection, buffer-only lifetime, null/oversize commits and destruction passed");
}
-23
View File
@@ -1,23 +0,0 @@
#include "../desktop-cursor.h"
#include <assert.h>
#include <stdio.h>
int main(void) {
uint32_t input[] = {0x80402010, 0, 0xdeadbeef, 0xff102030, 0x00ffffff, 0};
uint8_t out[16];
assert(ft_cursor_rgba(out, input, 2, 2, 12, DRM_FORMAT_ARGB8888));
assert(out[0] == 128 && out[1] == 64 && out[2] == 32 && out[3] == 128);
assert(out[4] == 0 && out[7] == 0);
assert(out[8] == 16 && out[9] == 32 && out[10] == 48 && out[11] == 255);
assert(out[12] == 0 && out[13] == 0 && out[14] == 0 && out[15] == 0);
assert(ft_cursor_rgba(out, input, 1, 1, 4, DRM_FORMAT_ABGR8888));
assert(out[0] == 32 && out[1] == 64 && out[2] == 128);
assert(ft_cursor_rgba(out, input, 1, 1, 4, DRM_FORMAT_XRGB8888));
assert(out[0] == 64 && out[1] == 32 && out[2] == 16 && out[3] == 255);
assert(ft_cursor_rgba(out, input, 1, 1, 4, DRM_FORMAT_XBGR8888));
assert(out[0] == 16 && out[2] == 64 && out[3] == 255);
assert(!ft_cursor_rgba(out, input, 1, 1, 3, DRM_FORMAT_ARGB8888));
assert(!ft_cursor_rgba(out, input, 513, 1, 2052, DRM_FORMAT_ARGB8888));
assert(!ft_cursor_rgba(out, NULL, 1, 1, 4, DRM_FORMAT_ARGB8888));
assert(!ft_cursor_rgba(out, input, 1, 1, 4, DRM_FORMAT_RGB565));
puts("KDE cursor channels, alpha, padding and bounds passed");
}
-22
View File
@@ -1,22 +0,0 @@
#include <assert.h>
#include "../desktop-handoff.h"
int main(void) {
struct ft_handoff h = {.policy=FT_LAST_ACTIVE};
assert(ft_handoff_claim(&h, false, 100, 0, 0));
assert(!ft_handoff_claim(&h, true, 100, 0, 0));
assert(ft_handoff_claim(&h, true, 101, 0, 0));
assert(!ft_handoff_claim(&h, true, 102, 1, 0));
assert(!ft_handoff_claim(&h, false, 102, 0, 1));
assert(ft_handoff_claim(&h, true, 102, 0, 1));
assert(ft_handoff_claim(&h, false, 102, 1, 0));
assert(ft_handoff_claim(&h, false, 103, 0, 0));
h.policy=FT_PREFER_MOUSE;
assert(!ft_handoff_claim(&h, true, 104, 0, 0));
assert(ft_handoff_claim(&h, false, 104, 0, 0));
h.policy=FT_PREFER_POINTER;
assert(!ft_handoff_claim(&h, false, 105, 0, 0));
assert(ft_handoff_claim(&h, true, 105, 0, 0));
h=(struct ft_handoff){.policy=FT_LAST_ACTIVE};
assert(ft_handoff_claim(&h, false, UINT32_MAX, 0, 0));
assert(ft_handoff_claim(&h, true, 0, 0, 0));
}
@@ -1,89 +0,0 @@
"""Only addresses upstream's isolated test compositor, never live input sockets."""
import json
import socket
import time
with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) as client:
client.bind('')
client.settimeout(2)
def ask(command):
client.sendto(command.encode(), '\0ft_screens_test')
return client.recv(8192).decode()
for command in ('mouse-layout 1 0 0 960 540 2',
'mouse-layout 2 960 0 540 960 2', 'mouse-output-off 24',
'mouse-mode desktop', 'mouse-position 1 100 100', 'mouse-move 10 5',
'mouse-button 272 1', 'mouse-move 900 0', 'mouse-button 272 0',
'mouse-wheel 0 0.125'):
assert ask(command) == 'ok', command
assert ask('mouse-output-off 25').startswith('error')
assert ask('mouse-layout 25 0 0 960 540 1').startswith('error')
ask('mouse-presence 1 1')
before = json.loads(ask('mouse?'))
ask('input 1 down 200 100 left'); ask('input 1 up 200 100 left')
after = json.loads(ask('mouse?'))
assert after['controllerEventsIgnored'] >= before['controllerEventsIgnored'] + 2
ask('mouse-presence 0 1')
time.sleep(1.1)
assert json.loads(ask('mouse-presence?'))['controllerDesktop']
before = json.loads(ask('mouse?'))
ask('input 1 down 200 100 left')
after = json.loads(ask('mouse?'))
assert after['controllerEventsIgnored'] == before['controllerEventsIgnored']
ask('mouse-presence 1 1')
assert not json.loads(ask('mouse-presence?'))['controllerDesktop']
ask('mouse-button 272 1'); ask('mouse-button 272 0')
print('Native motion/click/wheel, cross-output drag, output bounds and controller fallback passed')
with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) as client:
client.bind(''); client.settimeout(2)
def ask(command):
client.sendto(command.encode(), '\0ft_screens_test')
return client.recv(8192).decode()
def policy(): return json.loads(ask('desktop-policy?'))
assert ask('desktop-policy last-active') == 'ok'
assert ask('mouse-presence 1 1') == 'ok'
assert ask('mouse-move 3 0') == 'ok'
assert policy()['owner'] == 'mouse'
for _ in range(3): ask('input 1 move 200 100')
assert policy()['owner'] == 'mouse', 'aiming must not claim ownership'
time.sleep(.01)
ask('input 1 down 200 100 left')
assert policy()['owner'] == 'pointer'
assert policy()['pointerButtons'] == 1
before = json.loads(ask('mouse?'))
ask('mouse-move 40 0'); ask('mouse-button 272 1'); ask('mouse-button 272 0'); ask('mouse-wheel 0 1')
after = json.loads(ask('mouse?'))
assert before['x'] == after['x'], 'mouse motion must not accumulate during pointer drag'
assert before['buttonEvents'] == after['buttonEvents']
assert before['wheelEvents'] == after['wheelEvents']
assert policy()['owner'] == 'pointer'
assert ask('desktop-policy mouse').startswith('error'), 'cannot change policy during drag'
ask('mouse-presence 1 1')
assert policy()['owner'] == 'pointer', 'presence heartbeat must not steal ownership'
ask('input 1 up 200 100 left')
assert policy()['pointerButtons'] == 0
ask('mouse-wheel 0 1')
assert policy()['owner'] == 'mouse'
ask('mouse-button 272 1'); time.sleep(.01)
ask('input 1 down 200 100 left'); ask('input 1 up 200 100 left')
assert policy()['owner'] == 'mouse'
assert policy()['mouseButtons'] == 1
ask('mouse-button 272 0')
time.sleep(.01); ask('input 1 down 200 100 left'); ask('input 1 up 200 100 left')
assert policy()['owner'] == 'pointer'
ask('mouse-move 0 0')
assert policy()['owner'] == 'pointer', 'zero movement must not claim'
ask('mouse-move 1 0')
assert policy()['owner'] == 'mouse'
time.sleep(.01)
ask('input 1 move 200 100'); ask('input 1 scroll 0 1')
assert policy()['owner'] == 'pointer', 'controller scroll must claim desktop'
ask('mouse-wheel 0 1')
assert policy()['owner'] == 'mouse'
assert ask('desktop-policy pointer') == 'ok'
ask('mouse-move 1 0')
assert policy()['owner'] == 'pointer'
assert ask('desktop-policy mouse') == 'ok'
ask('input 1 down 200 100 left'); ask('input 1 up 200 100 left')
assert policy()['owner'] == 'mouse'
print('Desktop handoff, aiming exclusion, drag locks, ignored releases, presence stability and policy controls passed')
-34
View File
@@ -1,34 +0,0 @@
#include "../desktop-mouse.h"
#include <assert.h>
#include <stdio.h>
int main(void) {
struct ft_mouse m;
ft_mouse_init(&m);
assert(!ft_mouse_move(&m, 0, 0));
assert(ft_mouse_output_set(&m, 0, -1280, 0, 1280, 720, 1.5));
assert(ft_mouse_output_set(&m, 1, 0, 0, 1920, 1080, 1));
assert(ft_mouse_move(&m, 0, 0));
assert(m.screen == 0 && m.x == -640 && m.y == 360);
assert(ft_mouse_position(&m, -1, 100));
assert(ft_mouse_button(&m, 272, true));
assert(!ft_mouse_output_set(&m, 0, 0, 0, 100, 100, 1));
assert(ft_mouse_move(&m, 20, 0));
assert(m.screen == 1 && m.x == 19 && ft_mouse_target(&m) == 0);
// During the grab, host coordinates exceed the old output's width.
assert((m.x-m.outputs[0].x)*m.outputs[0].scale == 1948.5);
assert(ft_mouse_button(&m, 275, true));
assert(ft_mouse_button(&m, 272, false));
assert(ft_mouse_target(&m) == 0);
assert(ft_mouse_button(&m, 275, false));
assert(ft_mouse_target(&m) == 1 && m.grab == -1);
assert(ft_mouse_move(&m, 10000, 10000));
assert(m.x == 1919 && m.y == 1079);
assert(!ft_mouse_move(&m, NAN, 0));
assert(!ft_mouse_position(&m, INFINITY, 1));
assert(!ft_mouse_move(&m, 16385, 0));
assert(!ft_mouse_output_set(&m, FT_MOUSE_OUTPUTS, 0, 0, 100, 100, 1));
assert(!ft_mouse_output_set(&m, 0, 0, 0, 0, 100, 1));
assert(!ft_mouse_output_set(&m, 0, 0, 0, 100, 100, NAN));
assert(!ft_mouse_button(&m, 280, true));
puts("desktop mouse geometry, drag and validation passed");
}
-99
View File
@@ -302,12 +302,6 @@ struct Screen {
}
};
std::map<int, Screen> g_screens;
struct DesktopCursor {
vr::VROverlayHandle_t overlay = vr::k_ulOverlayHandleInvalid;
int width = 0, height = 0, hx = 0, hy = 0, scale = 1, screen = -1;
double x = 0, y = 0, outputScale = 1;
bool enabled = false, image = false;
} g_cursor;
std::map<const void *, vr::SharedTextureHandle_t> g_imports;
// Hand cutouts (see the top and handcut.h).
@@ -590,32 +584,6 @@ Mat OnSurface(const Screen &s, double u, double v, double dz) {
return m;
}
void UpdateDesktopCursor() {
if (g_cursor.overlay == vr::k_ulOverlayHandleInvalid) return;
const auto it = g_screens.find(g_cursor.screen);
const auto activity = vr::VRSystem()->GetTrackedDeviceActivityLevel(vr::k_unTrackedDeviceIndex_Hmd);
const bool off = activity == vr::k_EDeviceActivityLevel_Idle || activity == vr::k_EDeviceActivityLevel_Standby ||
activity == vr::k_EDeviceActivityLevel_Idle_Timeout;
Mat pose;
if (!g_cursor.enabled || !g_cursor.image || off || it == g_screens.end() ||
!it->second.visible || it->second.width < 1 || !ScreenPose(it->second, &pose)) {
vr::VROverlay()->HideOverlay(g_cursor.overlay);
return;
}
const Screen &s = it->second;
const double perPixel = s.metres / s.width;
const double cursorPixel = perPixel * g_cursor.outputScale / g_cursor.scale;
const double u = (g_cursor.x - s.width/2.0)*perPixel + (g_cursor.width/2.0-g_cursor.hx)*cursorPixel;
const double v = (s.height/2.0-g_cursor.y)*perPixel + (g_cursor.hy-g_cursor.height/2.0)*cursorPixel;
const double width = g_cursor.width*cursorPixel;
const Mat m = Mul(pose, OnSurface(s, u, v, 0.004));
vr::VROverlay()->SetOverlayTransformAbsolute(g_cursor.overlay, vr::TrackingUniverseStanding, &m);
vr::VROverlay()->SetOverlayWidthInMeters(g_cursor.overlay, float(width));
vr::VROverlay()->SetOverlayCurvature(g_cursor.overlay, s.curve > 0 ? float(std::min(1.0, width/(2*M_PI*s.curve))) : 0);
vr::VROverlay()->SetOverlayAlpha(g_cursor.overlay, s.alpha);
vr::VROverlay()->ShowOverlay(g_cursor.overlay);
}
double BarY(const Screen &s) { return -(s.heightMetres() / 2 + s.chrome * 0.06 + s.chrome * 12 / 256); }
Mat BarOffset(const Screen &s) { return OnSurface(s, 0, BarY(s), 0.003); }
@@ -1572,7 +1540,6 @@ bool ft_vr_init(void) {
void ft_vr_shutdown(void) {
if (!g_vr) return;
if (g_cursor.overlay != vr::k_ulOverlayHandleInvalid) vr::VROverlay()->DestroyOverlay(g_cursor.overlay);
if (g_cutterState == 1)
for (auto &[i, s] : g_screens) g_cutter.DropPanel(i); // drops their imports while SteamVR is up
if (g_catcher != vr::k_ulOverlayHandleInvalid) vr::VROverlay()->DestroyOverlay(g_catcher);
@@ -1602,71 +1569,6 @@ int ft_vr_modifiers(uint32_t format, uint64_t *out, int max) {
bool ft_vr_screens_shown(void) { return g_vr && ModeVisible(); }
static bool EnsureDesktopCursor() {
if (!g_vr) return false;
if (g_cursor.overlay == vr::k_ulOverlayHandleInvalid) {
if (vr::VROverlay()->CreateOverlay("frametop.desktop.cursor", "KDE desktop cursor", &g_cursor.overlay) != vr::VROverlayError_None) return false;
vr::VROverlay()->SetOverlayInputMethod(g_cursor.overlay, vr::VROverlayInputMethod_None);
vr::VROverlay()->SetOverlaySortOrder(g_cursor.overlay, 200);
}
return true;
}
bool ft_vr_cursor_image(const uint8_t *rgba, int width, int height, int hx, int hy, int scale) {
if (!rgba) { g_cursor.image = false; UpdateDesktopCursor(); return true; }
if (width < 1 || height < 1 || width > 512 || height > 512 || scale < 1 || scale > 8 || !EnsureDesktopCursor()) return false;
if (vr::VROverlay()->SetOverlayRaw(g_cursor.overlay, const_cast<uint8_t *>(rgba), width, height, 4) != vr::VROverlayError_None) return false;
g_cursor.width = width; g_cursor.height = height; g_cursor.hx = hx; g_cursor.hy = hy; g_cursor.scale = scale;
g_cursor.image = true;
UpdateDesktopCursor();
return true;
}
void ft_vr_cursor_move(int screen, double x, double y, double scale, bool enabled) {
g_cursor.screen = screen; g_cursor.x = x; g_cursor.y = y; g_cursor.outputScale = scale; g_cursor.enabled = enabled;
UpdateDesktopCursor();
}
static vr::SharedTextureHandle_t ImportBuffer(const void *key, const struct ft_dmabuf *b) {
auto it = g_imports.find(key);
if (it == g_imports.end()) {
vr::DmabufAttributes_t a{};
a.unWidth = uint32_t(b->width);
a.unHeight = uint32_t(b->height);
a.unDepth = a.unMipLevels = a.unArrayLayers = a.unSampleCount = 1;
a.unFormat = b->format;
a.ulModifier = b->modifier;
a.unPlaneCount = uint32_t(b->n_planes);
for (int i = 0; i < b->n_planes && i < int(vr::MaxDmabufPlaneCount); ++i) {
a.plane[i].unOffset = b->offset[i];
a.plane[i].unStride = b->stride[i];
a.plane[i].nFd = b->fd[i];
}
vr::SharedTextureHandle_t h = 0;
if (!vr::VRIPCResourceManager()->ImportDmabuf(vr::VRApplication_Overlay, &a, &h)) {
std::fprintf(stderr, "openvr: ImportDmabuf failed: %dx%d format 0x%x modifier 0x%llx\n", b->width,
b->height, b->format, (unsigned long long)b->modifier);
return 0;
}
it = g_imports.emplace(key, h).first;
}
return it->second;
}
bool ft_vr_cursor_dmabuf(const void *key, const struct ft_dmabuf *b, int hx, int hy, int scale) {
if (!b || b->width < 1 || b->height < 1 || b->width > 512 || b->height > 512 ||
scale < 1 || scale > 8 || !EnsureDesktopCursor()) return false;
vr::SharedTextureHandle_t handle = ImportBuffer(key, b);
if (!handle) return false;
vr::Texture_t tex = {&handle, vr::TextureType_SharedTextureHandle, vr::ColorSpace_Gamma};
if (vr::VROverlay()->SetOverlayTexture(g_cursor.overlay, &tex) != vr::VROverlayError_None) return false;
g_cursor.width = b->width; g_cursor.height = b->height; g_cursor.hx = hx; g_cursor.hy = hy; g_cursor.scale = scale;
g_cursor.image = true;
UpdateDesktopCursor();
return true;
}
} // extern "C"
namespace {
@@ -1991,7 +1893,6 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) {
UpdateLasers();
UpdateControls();
UpdateGuides();
UpdateDesktopCursor();
UpdateCutouts();
UpdateCatcher();
}
-5
View File
@@ -35,11 +35,6 @@ void ft_vr_shutdown(void);
int ft_vr_modifiers(uint32_t format, uint64_t *out, int max);
// The screens are showing (by the visibility mode; not counting a wrist-pinned screen).
bool ft_vr_screens_shown(void);
// Desktop cursor: KDE supplies the image; this is a non-interactive display
// overlay only. Mouse input goes straight to the Wayland seat, never SteamVR.
bool ft_vr_cursor_image(const uint8_t *rgba, int width, int height, int hotspot_x, int hotspot_y, int buffer_scale);
bool ft_vr_cursor_dmabuf(const void *key, const struct ft_dmabuf *buf, int hotspot_x, int hotspot_y, int buffer_scale);
void ft_vr_cursor_move(int screen, double pixel_x, double pixel_y, double output_scale, bool enabled);
// A panel for screen `index`, width in metres, placed in a row in front of the head.
void ft_vr_screen_create(int index, double metres, int count);
void ft_vr_screen_destroy(int index);
+1 -1
View File
@@ -112,7 +112,7 @@ def main():
try:
with open(os.path.join(HOME, ".config/frametop.conf")) as f:
conf = [ln.split("#")[0].strip() for ln in f]
out("Settings: " + " ".join(c for c in conf if re.match(r"(POINTER|META_DASHBOARD|SHARE_KEYS|BACKEND)=", c)))
out("Settings: " + " ".join(c for c in conf if re.match(r"(POINTER|SHARE_KEYS|BACKEND)=", c)))
except OSError:
out("Settings: no ~/.config/frametop.conf")
out("ft-screens state (visibility, manual, wrist, gesture, lasers, game running, in games): "
-29
View File
@@ -1,29 +0,0 @@
#!/usr/bin/env bash
# Offline native input tests; --headless also starts an isolated test desktop.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
cd "$root"
[[ $# == 0 || ($# == 1 && $1 == --headless) ]] || { echo 'Usage: scripts/test-desktop-input.sh [--headless]' >&2; exit 2; }
python3 -m unittest discover -s input -p 'test_desktop_mouse.py'
"$root/scripts/frame.sh" -C screens 'set -e; mkdir -p build
gcc -std=c11 -Wall -Wextra -Werror tests/desktop-mouse-test.c -lm -o build/desktop-mouse-test
build/desktop-mouse-test
gcc -std=c11 -Wall -Wextra -Werror tests/controller-fallback-test.c -o build/controller-fallback-test
build/controller-fallback-test
gcc -std=c11 -Wall -Wextra -Werror tests/desktop-handoff-test.c -o build/desktop-handoff-test
build/desktop-handoff-test
gcc -std=c11 -Wall -Wextra -Werror $(pkg-config --cflags libdrm) tests/desktop-cursor-test.c -o build/desktop-cursor-test
build/desktop-cursor-test
gcc -std=c11 -Wall -Wextra -Werror tests/cursor-cache-test.c $(pkg-config --cflags --libs wlroots-0.20 wayland-server wayland-client libdrm pixman-1) -o build/cursor-cache-test
build/cursor-cache-test'
if [[ ${1:-} == --headless ]]; then
# Reuses upstream's throwaway @ft_screens_test fixture, never the live desktop.
if pgrep -f '[f]t-screens.*--control ft_screens_test' >/dev/null; then
echo 'Test desktop already running; stop it yourself before this test.' >&2
exit 1
fi
"$root/screens/build.sh"
trap '"$root/screens/test/headless.sh" stop' EXIT
"$root/screens/test/headless.sh" start 2 3
python3 "$root/screens/tests/desktop-input-integration.py"
fi
+25
View File
@@ -321,6 +321,30 @@ def check_services(sockets, desktop_up):
report("FAIL", "XRService", "not running, so hand tracking has no cameras")
def check_steam_ui():
"""Steam's UI calls the Steam menu shortcut uses (steam/ft-steam). They're Steam client
internals, so a Steam update can change them too, not just a SteamOS one."""
# Run from stdin (over SSH) there's no __file__: try the usual checkouts.
repos = [os.path.dirname(os.path.dirname(os.path.abspath(__file__)))] if "__file__" in globals() else []
repos += [os.path.join(HOME, "frametop"), os.path.join(HOME, "dev/frametop")]
tool = next((t for t in (os.path.join(r, "steam/ft-steam") for r in repos) if os.path.exists(t)), None)
if not tool:
report("skip", "Steam menu shortcut", "steam/ft-steam isn't in a checkout here")
return
try:
p = subprocess.run([tool, "check"], capture_output=True, text=True, timeout=15)
except (OSError, subprocess.TimeoutExpired) as e:
report("warn", "Steam menu shortcut", f"ft-steam check didn't run ({e})")
return
detail = (p.stdout + p.stderr).strip()
if p.returncode == 0:
report("ok", "Steam menu shortcut", "Steam's UI still has the calls steam/ft-steam makes")
elif detail.startswith("ft-steam:"):
report("skip", "Steam menu shortcut", detail[len("ft-steam:"):].strip())
else:
report("warn", "Steam menu shortcut", f"{detail}; a Meta tap won't open the Steam menu (steam/ft-steam)")
def check_eye_tracker():
gaze = systemctl("is-enabled", "frametop-gaze") == "enabled"
try:
@@ -369,6 +393,7 @@ def main():
check_openvr()
check_overlays(desktop_up)
check_services(sockets, desktop_up)
check_steam_ui()
check_eye_tracker()
if args == ["--mark-good"]:
-5
View File
@@ -177,11 +177,6 @@ export XDG_CONFIG_HOME=$HOME/.config/frametop
export XDG_STATE_HOME=$HOME/.local/state/frametop
mkdir -p "$XDG_CONFIG_HOME" "$XDG_STATE_HOME"
# Steam's VR session exports a 256px cursor. Use this desktop's own KDE
# preferences instead of inheriting the dashboard cursor size and theme.
export XCURSOR_SIZE=$(kreadconfig6 --file kcminputrc --group Mouse --key cursorSize --default 24)
export XCURSOR_THEME=$(kreadconfig6 --file kcminputrc --group Mouse --key cursorTheme --default breeze_cursors)
# Remote desktop over VNC: session/remote-desktop.sh captures the desktop with
# krdp on 127.0.0.1, and session/vnc-bridge.sh re-serves its primary screen over VNC. krdpserver runs from the container, so KWin can't
# match it to an installed app. KWin's permission check for screencast and fake
-4
View File
@@ -10,9 +10,6 @@ REMOTE=0 # 1 = serve the desktop's primary screen over VNC on the tail
FLOAT_SLOTS=8 # screens backend: how many windows can float in VR at once (spare outputs; 0 turns it off; restart the desktop after a change)
FLOAT_MARGIN=300 # floating windows: pixels around each window on its output, so menus have room past its edges
POINTER=0 # 1 = the mouse drives the universal 3D pointer (ft_pointer driver) instead of a plain mouse
MOUSE_MODE=spatial # desktop: physical mouse controls KDE directly; controllers remain separate. CLI: input/ft-mousectl desktop --save
DESKTOP_MOUSE_SPEED=1 # logical desktop pixels per physical mouse count (0.05 to 10)
DESKTOP_CONTROLLER_FALLBACK=1 # desktop controller pointers when no routed physical mouse is connected
POINTER_SENSITIVITY=0.03 # degrees per mouse count
POINTER_IDLE=30 # seconds without mouse use before the controllers get their laser back
POINTER_WAKE_COUNTS=40 # mouse counts within 1 s needed to wake or re-claim the laser (ignores desk jitter)
@@ -57,7 +54,6 @@ POINTER_PINCH_DEADZONE=1.5 # hands: degrees the pinching hand moves before the p
POINTER_GRIP_GAIN=1 # hands: the pointer moves this times the gripping hand's angle
POINTER_GRIP_BELOW=0.35 # hands: grips beginning more than this (m) below the eyes are ignored (hands on a desk)
POINTER_PINCH_TYPING=1 # hands: no pinch within this many seconds of a key (typing touches thumb to index)
META_DASHBOARD=0 # 1 = a Meta tap on a pass-through keyboard toggles the SteamVR dashboard (pointer mode only)
SHARE_KEYS=0 # 1 = keys of keyboards grabbed for the desktop also go to @frametop_keys, for hotkey tools (any local process can listen)
DISPLAY_OFF_MIN=0 # minutes without use before frametop-power turns the displays off, even if the headset seems worn (0 = never)
DISPLAY_MOVE_MM=5 # displays off: the headset or a controller moving this far (mm) within 10 s counts as use
Executable
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env python3
"""ft-steam: Steam's menu and checks, through Steam's own UI (steamui.py).
ft-steam menu open the SteamVR dashboard on Steam's menu, or close the dashboard if it's
open (the Steam button's job, as a shortcut: the input relay's steam_menu)
ft-steam check do Steam's UI calls that the above needs still exist (update-check.py)
No pointer mode needed: it asks Steam to show its dashboard overlay (window.ShowVROverlay, what
Steam itself calls) and to focus the Steam frame's left menu (MenuStore.OpenMainMenu, which in
VR invokes the frame's focus_left_frame_menu_action). SteamClient.OpenVR.VROverlay says
whether the dashboard is up and hides it. These are Steam's UI internals, not an API, so a
Steam client update can rename them: `ft-steam check` says so.
"""
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
from steamui import SteamUnreachable, evaluate # noqa: E402
WINDOW = "SteamUIStore.WindowStore.VRGamepadUIMainWindowInstance"
MENU = f"""(async () => {{
const vr = SteamClient.OpenVR.VROverlay;
if (await vr.IsDashboardVisible()) {{ vr.HideDashboard(); return "closed"; }}
const w = {WINDOW};
if (!w) throw new Error("Steam's VR dashboard window isn't open");
ShowVROverlay(w.GetVROverlayKey());
// The frame's menu takes focus only once the dashboard is up.
for (let i = 0; i < 40 && !(await vr.IsDashboardVisible()); i++) await new Promise(r => setTimeout(r, 25));
w.MenuStore.OpenMainMenu();
return "opened";
}})()"""
CHECK = f"""(() => {{
const missing = [];
const vr = SteamClient.OpenVR && SteamClient.OpenVR.VROverlay;
for (const f of ["IsDashboardVisible", "HideDashboard"]) if (!vr || typeof vr[f] !== "function") missing.push("SteamClient.OpenVR.VROverlay." + f);
if (typeof window.ShowVROverlay !== "function") missing.push("ShowVROverlay");
const w = {WINDOW};
if (!w) missing.push("{WINDOW}");
else {{
if (typeof w.GetVROverlayKey !== "function") missing.push("GetVROverlayKey");
if (!w.MenuStore || typeof w.MenuStore.OpenMainMenu !== "function") missing.push("MenuStore.OpenMainMenu");
}}
return missing;
}})()"""
def main():
cmd = sys.argv[1] if len(sys.argv) == 2 else ""
if cmd not in ("menu", "check"):
sys.exit(__doc__)
try:
result = evaluate(MENU if cmd == "menu" else CHECK)
except SteamUnreachable as e:
sys.exit(f"ft-steam: {e}")
if cmd == "check":
if result:
sys.exit("Steam's UI lacks " + ", ".join(result))
print("ok")
else:
print(result)
if __name__ == "__main__":
main()
+108
View File
@@ -0,0 +1,108 @@
"""Run JavaScript in Steam's UI, standard library only.
Steam on the Frame runs with -cef-enable-debugging, so its UI's JavaScript context
(SharedJSContext) is reachable over the Chrome DevTools Protocol on 127.0.0.1:8080. Steam's
own calls live there: SteamClient.* (settings, SteamVR's dashboard) and the UI's stores
(SteamUIStore). Used by Frametop Display Settings (display-settings/steam_settings.py) and
ft-steam (the Steam menu shortcut).
evaluate(expression) the expression's value (awaited, JSON-able); SteamUnreachable when
Steam or its UI isn't there, or the script threw
"""
import base64
import json
import os
import socket
import struct
import urllib.request
CDP_PORT = 8080
class SteamUnreachable(Exception):
pass
class _WebSocket:
def __init__(self, url, timeout=5):
host_port, path = url[len("ws://"):].split("/", 1)
host, port = host_port.rsplit(":", 1)
self.sock = socket.create_connection((host, int(port)), timeout=timeout)
key = base64.b64encode(os.urandom(16)).decode()
self.sock.sendall((f"GET /{path} HTTP/1.1\r\nHost: {host_port}\r\nUpgrade: websocket\r\n"
f"Connection: Upgrade\r\nSec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n\r\n").encode())
head = b""
while b"\r\n\r\n" not in head:
chunk = self.sock.recv(4096)
if not chunk:
raise SteamUnreachable("Steam closed the connection")
head += chunk
if b" 101 " not in head.split(b"\r\n", 1)[0]:
raise SteamUnreachable(head.split(b"\r\n", 1)[0].decode(errors="replace"))
self.buf = head.split(b"\r\n\r\n", 1)[1]
def send(self, text):
data, mask = text.encode(), os.urandom(4)
n = len(data)
if n < 126:
head = struct.pack(">BB", 0x81, 0x80 | n)
elif n < 65536:
head = struct.pack(">BBH", 0x81, 0x80 | 126, n)
else:
head = struct.pack(">BBQ", 0x81, 0x80 | 127, n)
self.sock.sendall(head + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(data)))
def _take(self, n):
while len(self.buf) < n:
chunk = self.sock.recv(65536)
if not chunk:
raise SteamUnreachable("Steam closed the connection")
self.buf += chunk
out, self.buf = self.buf[:n], self.buf[n:]
return out
def recv(self):
message = b""
while True:
b0, b1 = self._take(2)
n = b1 & 0x7F
if n == 126:
n = struct.unpack(">H", self._take(2))[0]
elif n == 127:
n = struct.unpack(">Q", self._take(8))[0]
message += self._take(n)
if b0 & 0x80:
return message.decode(errors="replace")
def close(self):
self.sock.close()
def evaluate(expression):
"""Runs JavaScript in Steam's SharedJSContext and returns its (awaited) value."""
try:
with urllib.request.urlopen(f"http://127.0.0.1:{CDP_PORT}/json", timeout=3) as r:
targets = json.load(r)
except OSError as e:
raise SteamUnreachable(f"Steam isn't reachable on port {CDP_PORT} ({e})") from e
url = next((t["webSocketDebuggerUrl"] for t in targets if t.get("title") == "SharedJSContext"), None)
if not url:
raise SteamUnreachable("Steam's UI isn't running")
try:
ws = _WebSocket(url)
try:
ws.send(json.dumps({"id": 1, "method": "Runtime.evaluate",
"params": {"expression": expression, "awaitPromise": True, "returnByValue": True}}))
while True:
reply = json.loads(ws.recv())
if reply.get("id") == 1:
break
finally:
ws.close()
except OSError as e:
raise SteamUnreachable(str(e)) from e
result = reply.get("result", {})
if "exceptionDetails" in result:
details = result["exceptionDetails"]
raise SteamUnreachable(details.get("exception", {}).get("description") or details.get("text", "error"))
return result.get("result", {}).get("value")