steam-frame/DeeJanuz--frametop
1
0 Fork 0
mirror of https://github.com/DeeJanuz/frametop.git synced 2026-10-04 22:00:05 +02:00
Code Issues Packages Projects Releases 3 Wiki Activity

Frametop: a multi-screen desktop and universal 3D mouse for the Steam Frame

Browse Source 
Several KDE Plasma screens floating in SteamVR, each a real monitor of any
resolution and shape, shown by our own compositor (ft-screens), with a
layout, wrist pinning, and visibility modes; a Bluetooth mouse that drives
all of SteamVR as a room-anchored 3D pointer (input relay, ft-pointer
helper, ft_pointer SteamVR driver); two settings apps; and Bluetooth LE
fixes. Installs on the headset with ./install.sh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DeeJanuzandClaude Opus 5.5 committed 2026-09-26 16:14:13 -06:00
commit d439bc3f25
61 files changed
+8785
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.gitignore
+6
View File
@@ -0,0 +1,6 @@
build/
target/
.env
.env.*
captures/
__pycache__/
AGENTS.md
+34
View File
@@ -0,0 +1,34 @@
# Working on Frametop
Rules for people and coding agents changing this repo. The README covers what Frametop is and how to install it; `docs/reference.md` covers each component, and `docs/design.md` records how SteamVR on the Frame behaves and why things are built the way they are. Read the findings there before changing how the screens, the pointer, or the input relay talk to SteamVR.
## Two ways to run the scripts
Every script works in both modes, and must keep working in both:
- **On the Frame** (SteamOS, VR variant): commands run locally, in this checkout.
- **From a PC over SSH:** the repo is synced to `~/dev/frametop` on the Frame (`scripts/sync.sh`), and commands run there. `scripts/_env.sh` works out which mode applies (`FRAME_LOCAL`, `FRAME_HOST`, `FRAME_REPO`).
```
scripts/sync.sh # PC -> ~/dev/frametop on the Frame
scripts/frame.sh -C <dir> '<build cmd>' # runs in the "dev" Fedora distrobox
scripts/frame.sh --host '<cmd>' # runs on the SteamOS host
```
- From a PC, edit only on the PC. The Frame's copy is a mirror that `sync.sh` overwrites.
- Build inside the `dev` container. The SteamOS host has a read-only root and no compilers. Container builds link against the container's libraries, so they run in the container (`distrobox enter dev -- ...`); the SteamVR driver is built to run on the host.
- Container packages the build needs go in the list in `setup/dev-container.sh`, so the container can be rebuilt.
- Build output goes in `build/` next to the sources. It's gitignored and never synced.
## The headset may be in use
A Steam Frame is someone's personal headset, and they may be wearing it while you work.
- Don't kill or restart `gamescope`, `steam`, `vrserver`, `vrcompositor`, the gamescope session, or the Frametop desktop without asking. Each one ends or disrupts whatever is happening in VR.
- Don't run host `sudo`, `steamos-readonly disable`, `steamos-devmode` changes, pacman installs, or reboots without explicit approval. Only the Bluetooth fixes need host `sudo`, and they ask.
- Write only inside the repo, `/tmp`, and the container unless told otherwise. The installers are the exception: they write the user services, launchers, and the SteamVR driver into the home folder.
- Never copy `.netrc`, SSH keys, or Steam config off the Frame or into this repo.
## Names
User-facing names are "Frametop", "Frametop Display Settings", and "Frametop Input Settings". Programs and files use the `ft-` / `ft_` prefix (`ft-screens`, `ft-pointer`, `ft-layout`, the `ft_pointer` driver); config, units, and overlay keys use `frametop`. Program names must stay within 15 characters: Linux truncates process names there, and the scripts find programs with `pgrep -x` / `pkill -x`.
LICENSE
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 DeeJanuz
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
README.md
+134
View File
@@ -0,0 +1,134 @@
# Frametop
A multi-monitor desktop and a universal 3D mouse for the Valve Steam Frame, installed and run on the headset itself.
- **Several desktop screens floating in SteamVR.** A full KDE Plasma desktop where every screen is a real monitor of its own: any resolution and shape (ultrawide, portrait, 4K) and any size in the room. They appear in your saved layout when the desktop starts. You move, resize, curve, and roll them by hand, or pin one to your wrist, and one shortcut puts them all back.
- **A universal 3D mouse.** A Bluetooth mouse runs all of SteamVR (the dashboard, Steam, overlays, the desktop) as a small dot anchored in the room. It snaps onto panels, drags and tilts them, and hands the laser back to your controllers when you pick one up.
- **Frametop Display Settings**, an app for the screens: how many, each one's resolution, size, scale, and curve, which has the taskbar, the layout they float in, and when they show.
- **Frametop Input Settings**, an app to choose devices, map mouse buttons (for example to open the SteamVR dashboard), and tune the pointer.
- **Bluetooth fixes** so LE mice and keyboards, like the Swiftpoint Z3, reconnect after they sleep or the headset reboots.
Frametop is an independent project. It isn't made by or affiliated with Valve.
## Install on the headset
You need a Steam Frame with an internet connection, a keyboard (Bluetooth, or the on-screen one), and about 3 GB of free space.
1. **Open a desktop.** In the launcher, choose **Launch a program → Desktop**.
2. **Open a terminal.** In the application menu, open **System → Konsole**.
3. **Clone and install:**
```
git clone https://github.com/DeeJanuz/frametop.git ~/frametop
cd ~/frametop
./install.sh
```
The installer sets up distrobox (in your home folder; the system files stay untouched), a Fedora build container, and everything below. The first run downloads about 1–2 GB. It asks before the two steps that affect you:
- **Bluetooth fixes.** These need your password for `sudo`. If you've never set one, run `passwd` first. You can also skip them and install later.
- **Restarting SteamVR.** This is needed once, and it closes everything open in VR, including the terminal. Rebooting the headset works too.
After the restart:
- **Launch a program → Desktop** opens the multi-screen desktop. Its screens arrange themselves around where you're facing.
- **Frametop Display Settings** and **Frametop Input Settings** are in the desktop's application menu, under Settings.
### Add a Bluetooth mouse or keyboard
1. Pair it in Steam: **Settings → Bluetooth**.
2. If you installed the Bluetooth fixes, apply them once for the new device: **Frametop Input Settings → Bluetooth → Apply Bluetooth fixes**. Or, in the repo: `setup/bluetooth/install.sh run`. After that it reconnects on its own.
3. Move the mouse. The dot appears where you're looking.
## Use
| Do this | To get this |
| --- | --- |
| Move the mouse | The dot moves around you and snaps onto whatever panel it's over |
| Click, right-click, scroll | Acts on the panel under the dot |
| Pick up a controller | The controller gets its laser back; move the mouse to take over again |
| Point near the bottom of a screen | Its controls fade in: the bar, the curve and roll buttons, and the resize tab on the corner |
| Drag the bar under a screen | Moves the screen; scroll while dragging to push it away or pull it closer. With the mouse, hold right while dragging to tilt it |
| Drag the tab on a screen's bottom right corner | Resizes the screen |
| Click the curve button (next to the bar) | Curves the screen around you, or flattens it |
| Drag the roll button sideways, or scroll on it | Rolls the screen; it snaps level near straight |
| While carrying a screen, sweep its laser across your other controller's ring, then let go | Pins it to that wrist, at its size and distance, as you hold it when you let go; it shows while you see its front. Grab its bar to adjust it (it stays pinned); sweep across the ring again to take it off |
| Meta+Shift+R in the desktop | Puts the screens back in their layout (also the **Reset Screen Layout** menu entry, and a button you can map) |
| Meta+Shift+H in the desktop | Hides or shows all screens, for a VR game (also **Hide/Show Screens** and a mappable button). **Frametop Display Settings → Visibility & wrist** can instead show them only with the dashboard open, or while you look at your wrist |
Map the mouse's extra buttons to actions such as **Toggle SteamVR dashboard** or **Recenter pointer** in **Frametop Input Settings → Buttons**. Speed, dot size, and the rest are on its **Pointer** page and apply immediately.
Restarting the desktop (**Frametop Display Settings → Restart desktop**) closes its windows, but background work started in it, like servers, tmux, and builds, keeps running.
## Update
```
cd ~/frametop && git pull && ./install.sh
```
## Uninstall
```
./desktops.sh uninstall # the launcher's Desktop entry goes back to the stock desktop
./desktops.sh relay uninstall
pointer/helper/run.sh uninstall
pointer/driver/install.sh uninstall # then restart SteamVR
input-settings/install.sh uninstall
display-settings/install.sh uninstall
setup/bluetooth/install.sh uninstall # if you installed the Bluetooth fixes
```
## How it works
A nested Plasma session runs inside ft-screens (`screens/`), a small Wayland compositor. KWin opens a window per screen, ft-screens gives each its own size, and hands every frame to SteamVR as its own overlay without copying it. An input relay (`input/`) keeps Bluetooth mice working in SteamVR and turns the mouse into the 3D pointer, which drives a virtual SteamVR controller (`pointer/`). The details, and everything we learned about SteamVR on the Frame, are in [docs/reference.md](docs/reference.md) and [docs/design.md](docs/design.md).
| Folder | What it is |
| --- | --- |
| `install.sh` | The one-step installer. Safe to re-run. |
| `desktops.sh` | Start, stop, and configure the desktop, and install the input relay. |
| `screens/` | ft-screens, the compositor (wlroots and OpenVR). |
| `session/` | The desktop session script and its config example. |
| `layout/` | ft-layout: where the screens float, and their sizes. |
| `input/` | The input relay (Bluetooth mice and keyboards, button maps). |
| `pointer/` | The 3D mouse: SteamVR driver, helper service, and a probe tool. |
| `display-settings/`, `input-settings/` | The two settings apps (Kirigami, Python). |
| `setup/` | The build container and the Bluetooth fixes. See [setup/README.md](setup/README.md). |
| `scripts/` | Helpers the installers use. They run commands locally on the Frame, or over SSH from a PC. |
## Developing from a PC
Everything also works from a Linux (or WSL) PC over SSH, which is handier for editing code. The scripts detect where they're running: on the Frame they work on the local checkout, and on a PC they sync the repo to `~/dev/frametop` on the Frame and run there.
1. **On the Frame:** turn on developer mode, set a password (`passwd`), and enable SSH (`sudo systemctl enable --now sshd`). Add your public key to `~/.ssh/authorized_keys`. [deck-tailscale](https://github.com/tailscale-dev/deck-tailscale) gives access from anywhere.
2. **On the PC:** add the Frame to `~/.ssh/config` as host `frame` (or set `FRAME_HOST`):
```
Host frame
HostName <the Frame's address>
User steamos
IdentityFile ~/.ssh/<your-key>
```
3. For the Bluetooth fixes, which need `sudo` without a terminal on the Frame, put the password in `.env` at the repo root. It's gitignored and never synced:
```
steamos_root_pwd="<password>"
```
Then run `./install.sh` from the PC. Daily use:
```
scripts/doctor.sh # is the Frame reachable and ready?
scripts/sync.sh # copy the repo to ~/dev/frametop on the Frame
scripts/frame.sh '<cmd>' # run in the dev container, in the Frame's copy
scripts/frame.sh -C <dir> '<cmd>' # same, in a folder of the repo
scripts/frame.sh --host '<cmd>' # run on the SteamOS host
```
The sync is one-way: it makes the Frame's copy match this repo and deletes files there that no longer exist here. It skips `.git`, `build/`, `.env`, and anything gitignored. **Edit on the PC only.** Changes made in `~/dev/frametop` on the Frame are overwritten by the next sync.
Programs built in the `dev` container use its glibc, which is newer than the host's, so they run in the container. The SteamVR driver is the exception: it's built to run on the host (see `pointer/driver/build.sh`). See [AGENTS.md](AGENTS.md) for the working rules, including what not to restart on a headset someone is using.
## License
MIT. See [LICENSE](LICENSE).
desktops.sh
Executable
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env bash
# Start, stop, or inspect the multi-screen Plasma desktop in VR on the Frame.
# Usage: desktops.sh start [screens] | stop | restart | status | log [lines]
# desktops.sh install # make the VR launcher's "Desktop" entry start Frametop
# desktops.sh uninstall # give the launcher back the stock SteamOS desktop
# desktops.sh screens N # set the default screen count in ~/.config/frametop.conf
# desktops.sh remote on|off|info # VNC access over the tailnet (applies on next start)
# desktops.sh relay install|uninstall|status|log # input relay service (see input/input-relay.py)
# start without a count uses the Frame's config. FT_WIDTH, FT_HEIGHT, FT_PHYS_WIDTH pass through.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
. "$root/scripts/_env.sh"
frame="$root/scripts/frame.sh"
action=${1:-start}
screens=${2:-${FT_SCREENS:-}}
session=$FRAME_REPO/session
override=.local/share/applications/deckard-nested-desktop.desktop
log=/tmp/frametop-session.log
# Bracketed first letter so pgrep/pkill never match the ssh shell running them.
match='[v]r-overlay-key frametop '
# Is either backend's desktop running? (gamescope, or ft-screens)
running="{ pgrep -f '$match' >/dev/null || pgrep -x ft-screens >/dev/null; }"
case $action in
start)
# In its own systemd unit, so it outlives this shell (SSH, the settings app's restart).
"$root/scripts/sync.sh" >/dev/null
"$frame" --host "$running && { echo 'already running'; exit 0; }
systemctl --user reset-failed frametop-desktop 2>/dev/null
systemd-run --user --collect --quiet --unit frametop-desktop \
${screens:+--setenv=FT_SCREENS=$screens} ${FT_WIDTH:+--setenv=FT_WIDTH=$FT_WIDTH} ${FT_HEIGHT:+--setenv=FT_HEIGHT=$FT_HEIGHT} \
${FT_PHYS_WIDTH:+--setenv=FT_PHYS_WIDTH=$FT_PHYS_WIDTH} ${FT_BACKEND:+--setenv=FT_BACKEND=$FT_BACKEND} \
bash -c 'exec $session/frametop-session.sh > $log 2>&1'
sleep 12; echo \"plasmashell processes: \$(pgrep -c plasmashell)\"
$running && echo 'started' || { echo 'failed:'; tail -20 $log; exit 1; }" ;;
install)
"$root/scripts/sync.sh" >/dev/null
"$frame" --host "set -e; mkdir -p ~/.local/share/applications
sed 's|@SESSION@|$session/frametop-session.sh|' $session/deckard-nested-desktop.desktop > ~/$override
[ -f ~/.config/frametop.conf ] || cp $session/frametop.conf.example ~/.config/frametop.conf
echo \"installed ~/$override\"; grep ^Exec= ~/$override; echo; cat ~/.config/frametop.conf" ;;
uninstall) "$frame" --host "rm -f ~/$override && echo 'removed; the launcher uses the stock desktop again'" ;;
screens)
[[ ${2:-} =~ ^[1-9]$ ]] || { echo "usage: $0 screens N (1-9)" >&2; exit 2; }
"$frame" --host "set -e; f=~/.config/frametop.conf
[ -f \$f ] || cp $session/frametop.conf.example \$f
sed -i 's/^SCREENS=[0-9]*/SCREENS=$2/' \$f; grep ^SCREENS \$f" ;;
remote)
case ${2:-info} in
on|off)
v=$([ "$2" = on ] && echo 1 || echo 0)
"$frame" --host "set -e; f=~/.config/frametop.conf
[ -f \$f ] || cp $session/frametop.conf.example \$f
grep -q '^REMOTE=' \$f || echo 'REMOTE=0 # 1 = serve the desktop over VNC on the tailnet (port 5900)' >> \$f
sed -i 's/^REMOTE=[01]/REMOTE=$v/' \$f; grep ^REMOTE \$f; echo 'applies the next time the desktop starts'" ;;
info)
"$frame" --host "grep ^REMOTE ~/.config/frametop.conf 2>/dev/null || echo 'REMOTE not set'
ip=\$(ip -4 -o addr show tailscale0 | awk '{print \$4}' | cut -d/ -f1)
echo \"VNC: \$(hostname):5900 on the tailnet (\$ip) password \$(cat ~/.config/frametop-remote/vnc-password 2>/dev/null || echo '(created on first start)')\"
if pgrep -f '[X]vnc :20 ' >/dev/null; then echo 'vnc: running'; else echo 'vnc: not running'; fi
if pgrep -f '[k]rdpserver --plasma' >/dev/null; then echo 'capture (krdp, 127.0.0.1 only): running'; else echo 'capture (krdp): not running'; fi" ;;
*) echo "usage: $0 remote on|off|info" >&2; exit 2 ;;
esac ;;
relay)
unit=frametop-input-relay.service
case ${2:-status} in
install)
"$root/scripts/sync.sh" >/dev/null
# Enabled, not started: started under a running SteamVR it would grab the
# mouse away from it. It comes up before SteamVR on the next start.
fill_template "$root/input/$unit" | on_frame "mkdir -p ~/.config/systemd/user && cat > ~/.config/systemd/user/$unit"
"$frame" --host "set -e
systemctl --user daemon-reload; systemctl --user enable $unit
echo 'enabled; starts before SteamVR on the next reboot or SteamVR restart'" ;;
uninstall) "$frame" --host "systemctl --user disable --now $unit 2>/dev/null; systemctl --user clean --what=fdstore $unit 2>/dev/null; rm -f ~/.config/systemd/user/$unit; systemctl --user daemon-reload; echo removed" ;;
status) "$frame" --host "systemctl --user is-enabled $unit 2>/dev/null; systemctl --user is-active $unit 2>/dev/null
echo \"fd store: \$(systemctl --user show -p NFileDescriptorStore --value $unit)\"
p=\$(pgrep -x vrserver | head -1); [ -n \"\$p\" ] && for e in \$(ls -l /proc/\$p/fd 2>/dev/null | grep -oE 'event[0-9]+( \\(deleted\\))?' | sort -u | tr ' ' '_'); do n=\${e%%_*}; echo \"vrserver has \$e: \$(cat /sys/class/input/\$n/device/name 2>/dev/null)\"; done; true" ;;
log) "$frame" --host "journalctl --user -u $unit --no-pager -n ${3:-30}" ;;
*) echo "usage: $0 relay install|uninstall|status|log" >&2; exit 2 ;;
esac ;;
stop)
# ft-screens: ending it ends KWin and the session. gamescope can take a while to exit
# on SIGTERM. Wait, then force it. First, programs started in the desktop move out of
# its unit (session/keep-apps.sh), so background work in them outlives the restart.
"$frame" --host "$running || { echo 'not running'; exit 0; }
$session/keep-apps.sh
systemctl --user stop frametop-desktop 2>/dev/null; pkill -x ft-screens; pkill -f '$match'
for i in \$(seq 20); do $running || { echo stopped; exit 0; }; sleep 0.5; done
pkill -KILL -x ft-screens
pkill -KILL -f '$match'; sleep 1
pkill -f '[m]ultidesk-session.sh --inner' 2>/dev/null; pkill -f '[k]rdpserver --plasma' 2>/dev/null
pkill -f '[X]vnc :20 ' 2>/dev/null; pkill -f '[x]freerdp /v:.*:3390' 2>/dev/null
$running && { echo 'still running'; exit 1; } || echo 'stopped (forced)'" ;;
restart) "$0" stop; sleep 3; exec "$0" start ${screens:+"$screens"} ;;
status) "$frame" --host "if $running; then pgrep -af '$match|[m]d-screens --socket' | cut -c1-120; else echo 'not running'; fi" ;;
log) "$frame" --host "grep -vE '^\s*$' $log | tail -n ${2:-40}" ;;
*) echo "usage: $0 start [screens] | stop | restart | status | log [lines] | install | uninstall | screens N | remote on|off|info | relay install|uninstall|status|log" >&2; exit 2 ;;
esac
display-settings/ft-display-settings
Executable
+16
View File
@@ -0,0 +1,16 @@
#!/bin/bash
# Launch Frametop Display Settings from a Plasma session on the Frame host.
# The app runs in the dev container (PySide6 and Kirigami come from Fedora there).
# podman needs the real XDG_RUNTIME_DIR and the real user bus (to reach systemd for
# the container's cgroup; the Frametop session runs on a private bus from
# dbus-run-session). The session's Wayland socket and bus go to the app itself.
here=$(cd "$(dirname "$(readlink -f "$0")")" && pwd)
wl=${WAYLAND_DISPLAY:-wayland-0}
case $wl in /*) ;; *) wl="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/$wl" ;; esac
session_bus=${DBUS_SESSION_BUS_ADDRESS:-}
export XDG_RUNTIME_DIR=/run/user/$(id -u)
export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
exec "$HOME/.local/bin/distrobox" enter dev -- env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \
QT_QPA_PLATFORM="wayland;xcb" \
python3 "$here/ft_display_settings.py" "$@"
display-settings/ft-display-settings.desktop
+9
View File
@@ -0,0 +1,9 @@
[Desktop Entry]
Type=Application
Name=Frametop Display Settings
GenericName=Screens and layout for the VR desktop
Comment=Number of screens, resolution, scale, and where they float around you
Exec=@REPO@/frametop/display-settings/ft-display-settings
Icon=video-display
Categories=Settings;HardwareSettings;
Keywords=display;screen;monitor;resolution;layout;arrange;steamvr;frametop;
display-settings/ft-layout-reset.desktop
+10
View File
@@ -0,0 +1,10 @@
[Desktop Entry]
Type=Application
Name=Reset Screen Layout
GenericName=Put the VR desktop's screens back in their layout
Comment=Float the screens and arrange them in the layout from Frametop Display Settings
Exec=@REPO@/frametop/layout/ft-layout apply
Icon=view-restore
Categories=Settings;
Keywords=display;screen;layout;arrange;reset;steamvr;frametop;
X-KDE-Shortcuts=Meta+Shift+R
display-settings/ft-screens-toggle.desktop
+10
View File
@@ -0,0 +1,10 @@
[Desktop Entry]
Type=Application
Name=Hide/Show Screens
GenericName=Hide or show the VR desktop's screens
Comment=Hide the screens (and SteamVR's laser) for a VR game; press again to bring them back
Exec=@REPO@/frametop/layout/ft-layout toggle
Icon=view-visible
Categories=Settings;
Keywords=display;screen;hide;show;steamvr;frametop;
X-KDE-Shortcuts=Meta+Shift+H
display-settings/ft_display_settings.py
+467
View File
@@ -0,0 +1,467 @@
#!/usr/bin/env python3
"""Frametop Display Settings: the Frametop screens' count, resolution, size, scale, and layout.
A Kirigami (QML) app with a Python backend, like Frametop Input Settings. It runs in
the dev container:
- Screens (ft-screens backend): each screen's resolution (any, portrait too), its width
in VR in metres, its scale, and which one has the taskbar. Resolution and width
apply at once (ft-screens' control socket, @ft_screens); adding or removing a screen
when the desktop starts again. (gamescope backend: one shared resolution, at most
1920x1080 worth of pixels, rotation for portrait.)
- Visibility (ft-screens): when the screens show (always, only with the SteamVR
dashboard open, while you look at a controller, or only when toggled), the wrist
angle within which a pinned screen shows, and pin or unpin all screens.
- Layout: a preset (curved or flat, rows, distance, gap, height) or the arrangement
captured from where the screens are now, with a preview; arrange now; save the
current arrangement; arrange automatically when the desktop starts.
Settings go to ~/.config/frametop.conf and ~/.config/frametop-layout.json. Anything
that touches SteamVR runs layout/ft-layout on the host.
Launch with display-settings/ft-display-settings (host wrapper).
"""
import os
import shutil
import socket
import sys
from PySide6.QtCore import Property, QObject, QProcess, QTimer, QUrl, Signal, Slot
from PySide6.QtGui import QGuiApplication, QIcon
from PySide6.QtQml import QQmlApplicationEngine
from PySide6.QtQuickControls2 import QQuickStyle
HERE = os.path.dirname(os.path.abspath(__file__))
LAYOUT_DIR = os.path.join(HERE, "..", "layout")
sys.path.insert(0, LAYOUT_DIR)
import ft_layout # noqa: E402 (pure Python: the same geometry ft-layout uses)
FT_LAYOUT = os.path.join(LAYOUT_DIR, "ft-layout")
DESKTOPS = os.path.join(HERE, "..", "desktops.sh")
CONF_PATH = ft_layout.CONF_PATH
# gamescope's VR backend uploads a texture the size of a screen at start, through a
# 1920x1080x4-byte buffer: more pixels than 1920x1080 abort it (see the design notes).
MAX_PIXELS = 1920 * 1080
RESOLUTIONS = [(1280, 720), (1600, 900), (1920, 1080), (1728, 1080), (1920, 800), (2224, 928), (2560, 800)]
# ft-screens has no pixel limit; portrait screens are just tall.
SCREEN_RESOLUTIONS = [(1920, 1080, ""), (2560, 1440, ""), (3840, 2160, "4K"), (2560, 1080, "ultrawide"),
(3440, 1440, "ultrawide"), (5120, 1440, "super ultrawide"), (1920, 1200, "16:10"),
(2560, 1600, "16:10"), (1080, 1920, "portrait"), (1440, 2560, "portrait"),
(2160, 3840, "portrait 4K")]
FT_SCREENS = "\0ft_screens"
SCALES = [0.75, 1.0, 1.25, 4 / 3, 1.5, 1.75, 2.0]
ROTATIONS = [("normal", "Landscape"), ("left", "Portrait"), ("right", "Portrait (flipped)")]
def write_conf_value(key, value):
"""Set KEY=value in frametop.conf, keeping comments and the rest of the file."""
try:
with open(CONF_PATH) as f:
lines = f.read().splitlines()
except OSError:
lines = []
for i, line in enumerate(lines):
if line.split("#", 1)[0].strip().startswith(f"{key}="):
comment = line[line.index("#"):] if "#" in line else ""
lines[i] = f"{key}={value}" + (f" {comment}" if comment else "")
break
else:
lines.append(f"{key}={value}")
with open(CONF_PATH, "w") as f:
f.write("\n".join(lines) + "\n")
def host_command(*cmd):
"""argv to run a command on the SteamOS host (we live in the dev container).
distrobox-host-exec reaches the host through the user's real session bus; inside the
desktop our DBUS_SESSION_BUS_ADDRESS is the nested session's private one, where it
fails (exit 127, silently)."""
if not shutil.which("distrobox-host-exec"):
return list(cmd)
bus = f"unix:path=/run/user/{os.getuid()}/bus"
return ["env", f"DBUS_SESSION_BUS_ADDRESS={bus}", "distrobox-host-exec"] + list(cmd)
class Backend(QObject):
changed = Signal()
busyChanged = Signal()
message = Signal(str, bool) # text, is error
def __init__(self):
super().__init__()
self._busy = ""
self._proc = None
self._running = False
self._running_count = 0
self._sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
self._sock.bind("") # an abstract address ft-screens can reply to
self._sock.settimeout(1.0)
self._started = {} # conf values the running desktop started with
self.poll = QTimer(interval=3000, timeout=self._check_running)
self.poll.start()
self._check_running()
# --- state ---
def _conf(self):
conf = ft_layout.read_conf()
def num(key, default, cast=float):
try:
return cast(conf.get(key, default))
except ValueError:
return default
return {"screens": num("SCREENS", 2, int), "width": num("WIDTH", 1920, int),
"height": num("HEIGHT", 1080, int), "physWidth": num("PHYS_WIDTH", 1.6)}
def _check_running(self):
# The session's own Wayland socket (host processes' environments aren't readable
# from the container, so ft_layout.nested_env() doesn't work here).
running = os.path.exists(f"/run/user/{os.getuid()}/frametop/wayland-0")
count = self._screens_running() if running and ft_layout.backend() == "screens" else 0
if running != self._running or count != self._running_count:
self._running = running
self._running_count = count
self._started = self._conf() if running else {}
self.changed.emit()
def _ask_screens(self, text):
"""Request/reply to ft-screens; None if it isn't running."""
try:
self._sock.sendto(text.encode(), FT_SCREENS)
return self._sock.recv(4096).decode()
except OSError:
return None
def _screens_running(self):
reply = self._ask_screens("screens")
return int(reply.split()[1]) if reply and reply.startswith("ok") else 0
@Property(bool, notify=changed)
def desktopRunning(self):
return self._running
@Property(str, notify=changed)
def backend(self):
return ft_layout.backend()
@Property(bool, notify=changed)
def restartNeeded(self):
"""ft-screens: screens added or removed since the desktop started; gamescope: count,
resolution, or width changed."""
if not self._running:
return False
if ft_layout.backend() == "screens":
return self._running_count != ft_layout.screen_count(ft_layout.load_layout())
return bool(self._started) and self._started != self._conf()
@Property("QVariantList", constant=True)
def screenResolutions(self):
return [{"text": f"{w} × {h}" + (f" ({t})" if t else ""), "width": w, "height": h}
for w, h, t in SCREEN_RESOLUTIONS]
@Property(int, notify=changed)
def screens(self):
return self._conf()["screens"]
@Property(int, notify=changed)
def width(self):
return self._conf()["width"]
@Property(int, notify=changed)
def height(self):
return self._conf()["height"]
@Property(float, notify=changed)
def physWidth(self):
return self._conf()["physWidth"]
@Property("QVariantList", constant=True)
def resolutions(self):
return [{"text": f"{w} × {h}", "width": w, "height": h} for w, h in RESOLUTIONS]
@Property("QVariantList", constant=True)
def scales(self):
return [{"text": f"{round(s * 100)}%", "value": round(s, 4)} for s in SCALES]
@Property("QVariantList", constant=True)
def rotations(self):
return [{"text": t, "value": v} for v, t in ROTATIONS]
@Property("QVariantList", notify=changed)
def screenList(self):
c, layout = self._conf(), ft_layout.load_layout()
out = []
if ft_layout.backend() == "screens":
primary = ft_layout.primary_screen(layout)
for i in range(ft_layout.screen_count(layout)):
w, h = ft_layout.screen_pixels(layout, i)
s = ft_layout.screen_scale(layout, i)
out.append({"index": i, "width": w, "height": h, "metres": ft_layout.screen_metres(layout, i),
"scale": round(s, 4), "primary": i == primary,
"curved": float(ft_layout.screen_entry(layout, i).get("curve", 0)) > 0,
"effective": f"{round(w / s)} × {round(h / s)}"})
return out
for i in range(c["screens"]):
s = ft_layout.screen_scale(layout, i)
rot = ft_layout.screen_rotation(layout, i)
w, h = (c["height"], c["width"]) if rot != "normal" else (c["width"], c["height"])
out.append({"index": i, "scale": round(s, 4), "rotation": rot,
"effective": f"{round(w / s)} × {round(h / s)}"})
return out
@Property("QVariantMap", notify=changed)
def layout(self):
return ft_layout.load_layout()
@Property("QVariantList", notify=changed)
def plan(self):
"""The arrangement in the head frame, for the preview."""
layout = ft_layout.load_layout()
c = self._conf()
# Before any panel was measured: the width SteamVR floats a panel at, and the aspect.
if "panel_size" not in layout or layout["panel_size"] == list(ft_layout.DEFAULT_PANEL):
layout["panel_size"] = [ft_layout.DEFAULT_PANEL[0], ft_layout.DEFAULT_PANEL[0] * c["height"] / c["width"]]
out = []
for i, t in enumerate(ft_layout.plan(layout, c["screens"])):
w, h = ft_layout.screen_size(layout, i)
out.append({"index": i, "x": t["pos"][0], "y": t["pos"][1], "z": t["pos"][2],
"faceYaw": t["face"][0], "facePitch": t["face"][1], "width": w, "height": h})
return out
@Property(str, notify=busyChanged)
def busy(self):
return self._busy
# --- screens ---
@Slot(int)
def setScreens(self, n):
write_conf_value("SCREENS", str(max(1, min(6, n))))
self.changed.emit()
@Slot(int, int)
def setResolution(self, w, h):
if w * h > MAX_PIXELS:
self.message.emit(f"{w} × {h} is more than gamescope's VR mode can draw (1920 × 1080 worth of pixels, "
f"about {MAX_PIXELS // 1000} thousand); try {round((MAX_PIXELS * w / h) ** 0.5) // 8 * 8} × "
f"{round((MAX_PIXELS * h / w) ** 0.5) // 8 * 8}", True)
return
if w >= 640 and h >= 360:
write_conf_value("WIDTH", str(w))
write_conf_value("HEIGHT", str(h))
self.changed.emit()
@Slot(float)
def setPhysWidth(self, w):
write_conf_value("PHYS_WIDTH", f"{w:.2f}")
self.changed.emit()
@Slot(int, float)
def setScale(self, i, s):
layout = ft_layout.load_layout()
screens = layout.setdefault("screens", [])
while len(screens) <= i:
screens.append({})
screens[i]["scale"] = s
ft_layout.save_layout(layout)
self.changed.emit()
if self._running:
self._run("Applying scale", "scale")
@Slot(int, str)
def setRotation(self, i, rotation):
layout = ft_layout.load_layout()
screens = layout.setdefault("screens", [])
while len(screens) <= i:
screens.append({})
screens[i]["rotation"] = rotation
screens[i].pop("roll", None) # a saved arrangement follows the new rotation
ft_layout.save_layout(layout)
self.changed.emit()
if self._running:
self._run("Rotating", "scale")
# --- ft-screens: per-screen resolution and size ---
def _edit_screen(self, i, fn):
layout = ft_layout.load_layout()
screens = layout.setdefault("screens", [])
while len(screens) <= i:
screens.append({"size": [1920, 1080], "metres": 1920 / ft_layout.PIXELS_PER_METRE})
fn(screens[i])
ft_layout.save_layout(layout)
self.changed.emit()
@Slot(int, int, int)
def setScreenSize(self, i, w, h):
if not (320 <= w <= 16384 and 200 <= h <= 16384):
self.message.emit("Width and height: 320 to 16384 pixels", True)
return
self._edit_screen(i, lambda s: s.__setitem__("size", [w, h]))
if self._running and i < self._running_count:
self._ask_screens(f"size {i + 1} {w} {h}")
@Slot(int, float)
def setScreenMetres(self, i, m):
self._edit_screen(i, lambda s: s.__setitem__("metres", round(m, 3)))
if self._running and i < self._running_count:
self._ask_screens(f"width {i + 1} {m:.3f}")
@Slot(int, bool)
def setCurved(self, i, on):
"""Curve a screen into a cylinder around you (radius: your distance to it now, from
ft-screens; the layout's distance if the desktop isn't running)."""
radius = float(ft_layout.load_layout()["preset"].get("distance", 2.0)) if on else 0.0
if self._running and i < self._running_count:
reply = self._ask_screens(f"curve {i + 1} {'on' if on else 'off'}")
if reply and reply.startswith("ok"):
radius = float(reply.split()[1])
self._edit_screen(i, lambda s: s.__setitem__("curve", round(radius, 3)))
@Slot(int)
def setPrimary(self, i):
layout = ft_layout.load_layout()
layout["primary"] = i + 1
ft_layout.save_layout(layout)
self.changed.emit()
if self._running:
self._run("Moving the taskbar", "scale")
@Slot()
def addScreen(self):
layout = ft_layout.load_layout()
layout.setdefault("screens", []).append({"size": [1920, 1080], "metres": 1920 / ft_layout.PIXELS_PER_METRE})
ft_layout.save_layout(layout)
self.changed.emit()
@Slot(int)
def removeScreen(self, i):
layout = ft_layout.load_layout()
screens = layout.get("screens", [])
if len(screens) <= 1 or i >= len(screens):
return
screens.pop(i)
if layout.get("primary") == i + 1:
layout.pop("primary")
ft_layout.save_layout(layout)
self.changed.emit()
@Slot()
def toggleScreens(self):
self._ask_screens("toggle")
# --- ft-screens: visibility and pinning ---
@Property("QVariantMap", notify=changed)
def visibility(self):
return ft_layout.visibility(ft_layout.load_layout())
@Slot(str, "QVariant")
def setVisibility(self, key, value):
layout = ft_layout.load_layout()
v = ft_layout.visibility(layout)
v[key] = value
layout["visibility"] = v
ft_layout.save_layout(layout)
self.changed.emit()
if self._running:
if key == "mode":
self._ask_screens(f"visibility {value}")
elif key == "wrist_angle":
self._ask_screens(f"wrist {float(value):.1f}")
else:
self._ask_screens(f"gesture {v['gesture_hand']} {float(v['gesture_angle']):.1f}")
@Slot(str)
def pinAll(self, hand):
reply = self._ask_screens(f"pin all {hand}") if self._running else None
if reply and reply.startswith("ok"):
self.message.emit(f"All screens ride on your {hand} wrist now; grab a screen's bar to take it off. "
"Save current arrangement keeps it.", False)
else:
self.message.emit(f"Couldn't pin: {reply or 'the desktop is not running'}", True)
@Slot()
def unpinAll(self):
reply = self._ask_screens("unpin all") if self._running else None
if not (reply and reply.startswith("ok")):
self.message.emit(f"Couldn't unpin: {reply or 'the desktop is not running'}", True)
@Slot()
def restartDesktop(self):
"""Restart the Frametop desktop (this app closes with it) to apply count and resolution."""
argv = host_command("systemd-run", "--user", "--collect", "--quiet", os.path.abspath(DESKTOPS), "restart")
QProcess.startDetached(argv[0], argv[1:])
self.message.emit("Restarting the desktop…", False)
# --- layout ---
def _edit_layout(self, fn):
layout = ft_layout.load_layout()
fn(layout)
ft_layout.save_layout(layout)
self.changed.emit()
@Slot(str)
def setMode(self, mode):
self._edit_layout(lambda l: l.__setitem__("mode", mode))
@Slot(str, "QVariant")
def setPreset(self, key, value):
def edit(layout):
layout["mode"] = "preset"
layout["preset"][key] = value
self._edit_layout(edit)
@Slot(bool)
def setAuto(self, on):
self._edit_layout(lambda l: l.__setitem__("auto", bool(on)))
@Slot()
def arrange(self):
self._run("Arranging the screens", "apply")
@Slot()
def capture(self):
self._run("Saving the current arrangement", "capture")
# --- ft-layout on the host ---
def _run(self, label, *args):
if self._proc is not None:
self.message.emit(f"Still busy: {self._busy}", True)
return
self._busy = label
self.busyChanged.emit()
proc = QProcess(self)
proc.setProcessChannelMode(QProcess.MergedChannels)
argv = host_command(os.path.abspath(FT_LAYOUT), *args)
proc.finished.connect(lambda code, _status: self._done(proc, label, code))
self._proc = proc
proc.start(argv[0], argv[1:])
def _done(self, proc, label, code):
out = bytes(proc.readAllStandardOutput()).decode(errors="replace").strip()
self._proc = None
self._busy = ""
self.busyChanged.emit()
self.changed.emit()
last = out.splitlines()[-1] if out else ""
if code == 0:
self.message.emit(f"{label}: done" + (f" ({last})" if last and not last.startswith("ok") else ""), False)
else:
self.message.emit(f"{label} failed: {last or 'exit code ' + str(code)}", True)
def main():
app = QGuiApplication(sys.argv)
app.setApplicationName("ft-display-settings")
app.setApplicationDisplayName("Frametop Display Settings")
app.setDesktopFileName("ft-display-settings")
if not QIcon.themeName():
QIcon.setThemeName("breeze")
QQuickStyle.setStyle("org.kde.desktop")
engine = QQmlApplicationEngine()
backend = Backend()
engine.rootContext().setContextProperty("backend", backend)
engine.rootContext().setContextProperty("startPage", os.environ.get("FT_DISPLAY_PAGE", "screens"))
engine.load(QUrl.fromLocalFile(os.path.join(HERE, "main.qml")))
if not engine.rootObjects():
sys.exit(1)
sys.exit(app.exec())
if __name__ == "__main__":
main()
display-settings/install.sh
Executable
+27
View File
@@ -0,0 +1,27 @@
#!/usr/bin/env bash
# Install (or remove) Frametop Display Settings and the Reset Screen Layout and Hide/Show Screens
# entries on the Frame, with global shortcuts in the Frametop desktop: Meta+Shift+R and
# Meta+Shift+H (they take effect the next time the desktop starts).
# Usage: display-settings/install.sh [install|uninstall]
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
. "$root/scripts/_env.sh"
"$root/scripts/sync.sh" >/dev/null
apps=.local/share/applications
shortcuts=.config/frametop/kglobalshortcutsrc
case ${1:-install} in
install)
for f in ft-display-settings ft-layout-reset ft-screens-toggle; do
fill_template "$root/display-settings/$f.desktop" | on_frame "mkdir -p ~/$apps && cat > ~/$apps/$f.desktop"
done
on_frame "chmod +x display-settings/ft-display-settings layout/ft-layout layout/ft_layout.py
mkdir -p ~/.config/frametop
kwriteconfig6 --file ~/$shortcuts --group services --group ft-layout-reset.desktop --key _launch 'Meta+Shift+R'
kwriteconfig6 --file ~/$shortcuts --group services --group ft-screens-toggle.desktop --key _launch 'Meta+Shift+H'"
echo "installed: Frametop Display Settings, Reset Screen Layout (Meta+Shift+R), Hide/Show Screens (Meta+Shift+H)" ;;
uninstall)
on_frame "rm -f ~/$apps/ft-display-settings.desktop ~/$apps/ft-layout-reset.desktop ~/$apps/ft-screens-toggle.desktop
[ -f ~/$shortcuts ] && for f in ft-layout-reset ft-screens-toggle; do kwriteconfig6 --file ~/$shortcuts --group services --group \$f.desktop --key _launch --delete; done
echo removed" ;;
*) echo "usage: $0 [install|uninstall]" >&2; exit 2 ;;
esac
display-settings/main.qml
+666
View File
@@ -0,0 +1,666 @@
// Frametop Display Settings (Kirigami). Backend: ft_display_settings.py ("backend").
import QtQuick
import QtQuick.Controls as Controls
import QtQuick.Layouts
import org.kde.kirigami as Kirigami
Kirigami.ApplicationWindow {
id: root
title: "Frametop Display Settings"
width: Kirigami.Units.gridUnit * 46
height: Kirigami.Units.gridUnit * 36
// Pages as tabs across the top (a side drawer was easy to miss).
readonly property var pages: backend.backend === "screens"
? [{ text: "Screens", icon: "video-display", page: screensPage },
{ text: "Layout", icon: "view-grid", page: layoutPage },
{ text: "Visibility & wrist", icon: "view-visible", page: visibilityPage }]
: [{ text: "Screens", icon: "video-display", page: screensPage },
{ text: "Layout", icon: "view-grid", page: layoutPage }]
header: Controls.TabBar {
id: tabs
Repeater {
model: root.pages
Controls.TabButton {
required property var modelData
text: modelData.text
icon.name: modelData.icon
onClicked: root.show(modelData.page)
}
}
Component.onCompleted: currentIndex = ({ layout: 1, visibility: 2 })[startPage] || 0
}
function show(page) {
pageStack.clear()
pageStack.push(page)
}
// FT_DISPLAY_PAGE=layout|visibility opens the app on that page.
pageStack.initialPage: ({ layout: layoutPage, visibility: visibilityPage })[startPage] || screensPage
Connections {
target: backend
function onMessage(text, isError) {
root.showPassiveNotification(text, isError ? "long" : "short")
}
}
Kirigami.PromptDialog {
id: restartDialog
title: "Restart the desktop?"
subtitle: "Every window on the desktop closes, this app too. It starts again with the current settings."
standardButtons: Kirigami.Dialog.NoButton
customFooterActions: [
Kirigami.Action {
text: "Restart"
icon.name: "view-refresh"
onTriggered: { restartDialog.close(); backend.restartDesktop() }
},
Kirigami.Action {
text: "Cancel"
icon.name: "dialog-cancel"
onTriggered: restartDialog.close()
}
]
}
// ---------------------------------------------------------------- Screens
Component {
id: screensPage
Kirigami.ScrollablePage {
id: spage
title: "Screens"
property bool md: backend.backend === "screens"
actions: [
Kirigami.Action {
visible: spage.md
text: "Add screen"
icon.name: "list-add"
onTriggered: backend.addScreen()
},
Kirigami.Action {
visible: spage.md && backend.desktopRunning
text: "Hide/show screens"
icon.name: "view-visible"
onTriggered: backend.toggleScreens()
},
Kirigami.Action {
visible: backend.desktopRunning
text: "Restart desktop"
icon.name: "view-refresh"
tooltip: "Closes the desktop's windows (and this app) and starts it again"
onTriggered: restartDialog.open()
}
]
header: Kirigami.InlineMessage {
position: Kirigami.InlineMessage.Position.Header
visible: backend.restartNeeded || !backend.desktopRunning
type: Kirigami.MessageType.Information
text: backend.restartNeeded
? (spage.md ? "Screens were added or removed. That applies when the desktop starts again; restarting closes its windows (and this app)."
: "The number of screens, resolution, or panel width changed. They apply when the desktop starts again; restarting closes its windows (and this app).")
: "The desktop isn't running. These settings apply the next time it starts."
actions: [
Kirigami.Action {
visible: backend.restartNeeded
text: "Restart desktop"
icon.name: "system-reboot"
onTriggered: backend.restartDesktop()
}
]
}
ColumnLayout {
spacing: Kirigami.Units.largeSpacing
// ft-screens: every screen its own resolution and size.
Repeater {
model: spage.md ? backend.screenList : []
delegate: Kirigami.AbstractCard {
id: card
required property var modelData
Layout.fillWidth: true
property int customIndex: backend.screenResolutions.length
contentItem: ColumnLayout {
RowLayout {
Kirigami.Heading { level: 3; text: "Screen " + (card.modelData.index + 1) }
Controls.Label {
text: card.modelData.width + " × " + card.modelData.height
+ (card.modelData.scale !== 1 ? ", works like " + card.modelData.effective : "")
opacity: 0.7
}
Item { Layout.fillWidth: true }
Controls.RadioButton {
text: "Taskbar here"
checked: card.modelData.primary
onToggled: if (checked) backend.setPrimary(card.modelData.index)
}
Controls.ToolButton {
icon.name: "edit-delete-remove"
enabled: backend.screenList.length > 1
display: Controls.AbstractButton.IconOnly
text: "Remove this screen"
Controls.ToolTip.text: text
Controls.ToolTip.visible: hovered
onClicked: backend.removeScreen(card.modelData.index)
}
}
Kirigami.FormLayout {
Layout.fillWidth: true
RowLayout {
Kirigami.FormData.label: "Resolution:"
Controls.ComboBox {
id: res
model: backend.screenResolutions.concat([{ text: "Custom…", width: 0, height: 0 }])
textRole: "text"
Component.onCompleted: {
const i = backend.screenResolutions.findIndex(r => r.width === card.modelData.width && r.height === card.modelData.height)
currentIndex = i >= 0 ? i : card.customIndex
}
onActivated: {
const r = model[currentIndex]
if (r.width > 0) backend.setScreenSize(card.modelData.index, r.width, r.height)
}
}
Controls.SpinBox {
id: cw
visible: res.currentIndex === card.customIndex
from: 320; to: 16384; stepSize: 8; editable: true
value: card.modelData.width
}
Controls.Label { visible: cw.visible; text: "×" }
Controls.SpinBox {
id: ch
visible: cw.visible
from: 200; to: 16384; stepSize: 8; editable: true
value: card.modelData.height
}
Controls.Button {
visible: cw.visible
text: "Set"
onClicked: backend.setScreenSize(card.modelData.index, cw.value, ch.value)
}
}
RowLayout {
Kirigami.FormData.label: "Width in VR:"
Controls.Slider {
id: metres
from: 0.5; to: 6.0; stepSize: 0.05
value: card.modelData.metres
Layout.preferredWidth: Kirigami.Units.gridUnit * 12
onMoved: backend.setScreenMetres(card.modelData.index, value)
}
Controls.Label {
text: metres.value.toFixed(2) + " m wide, "
+ (metres.value * card.modelData.height / card.modelData.width).toFixed(2) + " m tall"
}
}
Controls.ComboBox {
Kirigami.FormData.label: "Scale:"
model: backend.scales
textRole: "text"
valueRole: "value"
Component.onCompleted: currentIndex = Math.max(0, indexOfValue(card.modelData.scale))
onActivated: backend.setScale(card.modelData.index, currentValue)
}
Controls.Switch {
Kirigami.FormData.label: "Curved:"
text: "Bend around you"
checked: card.modelData.curved
onToggled: backend.setCurved(card.modelData.index, checked)
}
}
}
}
}
// gamescope: one shared resolution.
Kirigami.FormLayout {
visible: !spage.md
Layout.fillWidth: true
Controls.SpinBox {
Kirigami.FormData.label: "Screens:"
from: 1
to: 6
value: backend.screens
onValueModified: backend.setScreens(value)
}
RowLayout {
Kirigami.FormData.label: "Resolution:"
Controls.ComboBox {
id: resBox
property bool custom: currentIndex === count - 1
model: backend.resolutions.concat([{ text: "Custom…", width: 0, height: 0 }])
textRole: "text"
Component.onCompleted: {
const i = backend.resolutions.findIndex(r => r.width === backend.width && r.height === backend.height)
currentIndex = i >= 0 ? i : count - 1
}
onActivated: {
const r = model[currentIndex]
if (r.width > 0) backend.setResolution(r.width, r.height)
}
}
Controls.SpinBox {
id: customW
visible: resBox.custom
from: 640; to: 7680; stepSize: 8
editable: true
value: backend.width
}
Controls.Label { visible: resBox.custom; text: "×" }
Controls.SpinBox {
id: customH
visible: resBox.custom
from: 360; to: 4320; stepSize: 8
editable: true
value: backend.height
}
Controls.Button {
visible: resBox.custom
text: "Set"
onClicked: backend.setResolution(customW.value, customH.value)
}
}
RowLayout {
Kirigami.FormData.label: "Panel width:"
Controls.Slider {
id: physSlider
from: 0.8; to: 3.0; stepSize: 0.05
value: backend.physWidth
Layout.preferredWidth: Kirigami.Units.gridUnit * 12
onMoved: backend.setPhysWidth(value)
}
Controls.Label { text: physSlider.value.toFixed(2) + " m" }
}
Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Each screen" }
Repeater {
model: spage.md ? [] : backend.screenList
delegate: RowLayout {
required property var modelData
Kirigami.FormData.label: "Screen " + (modelData.index + 1) + ":"
Controls.ComboBox {
model: backend.rotations
textRole: "text"
valueRole: "value"
Component.onCompleted: currentIndex = Math.max(0, indexOfValue(modelData.rotation))
onActivated: backend.setRotation(modelData.index, currentValue)
}
Controls.ComboBox {
model: backend.scales
textRole: "text"
valueRole: "value"
Component.onCompleted: currentIndex = Math.max(0, indexOfValue(modelData.scale))
onActivated: backend.setScale(modelData.index, currentValue)
}
Controls.Label {
text: "looks like " + modelData.effective
opacity: 0.7
}
}
}
}
}
footer: Controls.Label {
padding: Kirigami.Units.largeSpacing
wrapMode: Text.Wrap
opacity: 0.7
text: spage.md
? "Each screen is a real monitor of its own: any resolution, portrait by choosing a tall one. Resolution, "
+ "width, and curve apply at once. In VR: move a screen by the bar underneath, curve it with the round "
+ "button next to the bar, resize it by the tab on its bottom right corner; Save current arrangement on the "
+ "Layout page keeps all of it."
: "gamescope draws every screen at the same resolution, at most 1920 × 1080 worth of pixels. Portrait turns "
+ "a screen on its side. Rotation and scale apply at once; the rest when the desktop starts."
}
}
}
// ---------------------------------------------------------------- Layout
Component {
id: layoutPage
Kirigami.ScrollablePage {
id: lpage
title: "Layout"
property var layout: backend.layout
property var preset: layout.preset || {}
property bool hasCustom: (layout.screens || []).some(s => s.pos !== undefined)
actions: [
Kirigami.Action {
text: "Arrange now"
icon.name: "view-restore"
tooltip: "Float the screens out of the dashboard and put them in this layout, around where you're facing"
enabled: backend.desktopRunning && backend.busy === ""
onTriggered: backend.arrange()
},
Kirigami.Action {
text: "Save current arrangement"
icon.name: "document-save"
tooltip: "Use where the screens are now (placed by hand) as the layout"
enabled: backend.desktopRunning && backend.busy === ""
onTriggered: backend.capture()
}
]
header: Kirigami.InlineMessage {
position: Kirigami.InlineMessage.Position.Header
visible: backend.busy !== ""
type: Kirigami.MessageType.Information
text: backend.busy + "… (the pointer is borrowed for a few seconds)"
}
ColumnLayout {
spacing: Kirigami.Units.largeSpacing
Kirigami.FormLayout {
Layout.fillWidth: true
Controls.ComboBox {
Kirigami.FormData.label: "Arrangement:"
model: [
{ text: "Curved around you", value: "arc" },
{ text: "Flat wall", value: "flat" },
{ text: "Saved arrangement", value: "custom" }
]
textRole: "text"
valueRole: "value"
currentIndex: lpage.layout.mode === "custom" ? 2 : (lpage.preset.kind === "flat" ? 1 : 0)
onActivated: {
if (currentValue === "custom") backend.setMode("custom")
else backend.setPreset("kind", currentValue)
}
}
Controls.Label {
visible: lpage.layout.mode === "custom"
Kirigami.FormData.label: ""
text: lpage.hasCustom ? "Where the screens were when you saved. Pick a preset to edit."
: "Nothing saved yet: place the screens by hand, then Save current arrangement."
opacity: 0.7
wrapMode: Text.Wrap
Layout.maximumWidth: Kirigami.Units.gridUnit * 20
}
Controls.SpinBox {
Kirigami.FormData.label: "Rows:"
visible: lpage.layout.mode !== "custom"
from: 1
to: Math.max(1, backend.screens)
value: lpage.preset.rows || 1
onValueModified: backend.setPreset("rows", value)
}
Repeater {
model: [
{ key: "distance", label: "Distance", from: 0.6, to: 3.0, step: 0.05, unit: "m", def: 1.2 },
{ key: "gap", label: "Gap", from: 0.0, to: 0.3, step: 0.01, unit: "m", def: 0.04 },
{ key: "height", label: "Height", from: -0.8, to: 0.8, step: 0.05, unit: "m", def: 0.0 }
]
delegate: RowLayout {
required property var modelData
visible: lpage.layout.mode !== "custom"
Kirigami.FormData.label: modelData.label + ":"
Controls.Slider {
id: s
from: modelData.from; to: modelData.to; stepSize: modelData.step
value: lpage.preset[modelData.key] !== undefined ? lpage.preset[modelData.key] : modelData.def
Layout.preferredWidth: Kirigami.Units.gridUnit * 12
onMoved: backend.setPreset(modelData.key, value)
}
Controls.Label {
text: (modelData.key === "height" && s.value > 0 ? "+" : "") + s.value.toFixed(2) + " " + modelData.unit
+ (modelData.key === "height" ? " (from eye level)" : "")
}
}
}
Controls.Switch {
Kirigami.FormData.label: "When the desktop starts:"
text: "Float the screens and arrange them"
checked: lpage.layout.auto !== false
onToggled: backend.setAuto(checked)
}
}
// Preview: from above (you at the bottom) and from the front.
Kirigami.Heading { level: 3; text: "Preview" }
Canvas {
id: preview
Layout.fillWidth: true
Layout.preferredHeight: Kirigami.Units.gridUnit * 13
property var plan: backend.plan
onPlanChanged: requestPaint()
onWidthChanged: requestPaint()
onPaint: {
const ctx = getContext("2d")
ctx.reset()
const text = Kirigami.Theme.textColor
const accent = Kirigami.Theme.highlightColor
const half = width / 2
ctx.font = Kirigami.Theme.smallFont.pixelSize + "px sans-serif"
ctx.fillStyle = text
ctx.fillText("From above", 4, 12)
ctx.fillText("From the front", half + 8, 12)
if (!plan || plan.length === 0) return
// From above: x right, forward (-z) up; you are the dot at the bottom.
let ends = [[0, 0]]
for (const p of plan) {
const f = p.faceYaw * Math.PI / 180
const rx = Math.cos(f) * p.width / 2, rz = -Math.sin(f) * p.width / 2
ends.push([p.x - rx, p.z - rz], [p.x + rx, p.z + rz])
}
const xs = ends.map(e => e[0]), zs = ends.map(e => e[1])
const span = Math.max(Math.max(...xs) - Math.min(...xs), Math.max(...zs) - Math.min(...zs), 0.5)
const k = Math.min(half - 20, height - 44) / span
const cx = half / 2 - (Math.max(...xs) + Math.min(...xs)) / 2 * k
const cz = height - 12 - Math.max(...zs) * k // your dot 12 px above the bottom
const P = (x, z) => [cx + x * k, cz + z * k]
ctx.fillStyle = text
ctx.beginPath(); const me = P(0, 0); ctx.arc(me[0], me[1], 4, 0, 2 * Math.PI); ctx.fill()
ctx.lineWidth = 4
ctx.lineCap = "round"
// Screens stacked in rows share a spot from above, so they share a label ("1·3"):
// the same direction on a curve, the same x on a flat wall.
const flat = plan.every(p => Math.abs(p.faceYaw - plan[0].faceYaw) < 0.1
&& Math.abs(p.facePitch - plan[0].facePitch) < 0.1)
const labels = {}
for (const p of plan) {
const f = p.faceYaw * Math.PI / 180
const rx = Math.cos(f) * p.width / 2, rz = -Math.sin(f) * p.width / 2
const a = P(p.x - rx, p.z - rz), b = P(p.x + rx, p.z + rz)
ctx.strokeStyle = accent
ctx.beginPath(); ctx.moveTo(a[0], a[1]); ctx.lineTo(b[0], b[1]); ctx.stroke()
const c = P(p.x, p.z), spot = flat ? Math.round(p.x * 50) : Math.round(p.faceYaw * 2)
labels[spot] = labels[spot] || { at: c, names: [] }
labels[spot].names.push(p.index + 1)
}
ctx.fillStyle = text
for (const spot in labels) {
const l = labels[spot], t = l.names.join("·")
ctx.fillText(t, l.at[0] - ctx.measureText(t).width / 2, l.at[1] - 7)
}
// From the front: x right, y up. A flat wall as it is; a curved layout
// unrolled (arc length by yaw and pitch), so the gaps show true.
const front = plan.map(p => {
if (flat) return { x: p.x, y: p.y, w: p.width, h: p.height }
const r = Math.sqrt(p.x * p.x + p.y * p.y + p.z * p.z)
const arc = m => 2 * r * Math.atan(m / 2 / r) // what the screen spans on the curve
return { x: -p.faceYaw * Math.PI / 180 * r, y: p.facePitch * Math.PI / 180 * r,
w: arc(p.width), h: arc(p.height) }
})
const fx = [], fy = []
front.forEach(f => { fx.push(f.x - f.w / 2, f.x + f.w / 2); fy.push(f.y - f.h / 2, f.y + f.h / 2) })
fy.push(0)
const fspan = Math.max(Math.max(...fx) - Math.min(...fx), Math.max(...fy) - Math.min(...fy), 0.5)
const fk = Math.min(half - 20, height - 44) / fspan
const ox = half + half / 2 - (Math.max(...fx) + Math.min(...fx)) / 2 * fk
const oy = 30 + (height - 42) / 2 + (Math.max(...fy) + Math.min(...fy)) / 2 * fk
ctx.strokeStyle = Kirigami.Theme.disabledTextColor
ctx.lineWidth = 1
ctx.setLineDash([4, 4])
ctx.beginPath(); ctx.moveTo(half + 8, oy); ctx.lineTo(width - 4, oy); ctx.stroke() // eye level
ctx.setLineDash([])
plan.forEach((p, i) => {
const f = front[i], x = ox + (f.x - f.w / 2) * fk, y = oy - (f.y + f.h / 2) * fk
ctx.fillStyle = Qt.rgba(accent.r, accent.g, accent.b, 0.35)
ctx.fillRect(x, y, f.w * fk, f.h * fk)
ctx.strokeStyle = accent
ctx.lineWidth = 2
ctx.strokeRect(x, y, f.w * fk, f.h * fk)
ctx.fillStyle = text
ctx.fillText(String(p.index + 1), x + f.w * fk / 2 - 3, y + f.h * fk / 2 + 4)
})
}
}
Controls.Label {
Layout.fillWidth: true
wrapMode: Text.Wrap
opacity: 0.7
text: "The layout goes around where you're facing when it's applied. Move screens by hand any time "
+ "(grab bar under each screen); to put them back: Meta+Shift+R in the desktop, the Reset "
+ "Screen Layout menu entry, Arrange now here, or a mouse button mapped to \"Reset desktop "
+ "screen layout\" in Frametop Input Settings → Buttons."
}
}
}
}
// ---------------------------------------------------------------- Visibility
Component {
id: visibilityPage
Kirigami.ScrollablePage {
id: vpage
title: "Visibility"
property var v: backend.visibility
actions: [
Kirigami.Action {
text: "Hide/show now"
icon.name: "view-visible"
enabled: backend.desktopRunning
onTriggered: backend.toggleScreens()
}
]
ColumnLayout {
spacing: Kirigami.Units.largeSpacing
Kirigami.FormLayout {
Layout.fillWidth: true
Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "When the screens show" }
Repeater {
model: [
{ value: "always", text: "Always", help: "Meta+Shift+H (or a mapped button) hides them, for a VR game." },
{ value: "dashboard", text: "Only with the SteamVR dashboard open", help: "They come and go with the dashboard. Meta+Shift+H shows them anyway." },
{ value: "gesture", text: "While I look at my wrist", help: "They show while you look toward the controller below. Meta+Shift+H shows them anyway." },
{ value: "toggle", text: "Only when I show them", help: "Hidden until Meta+Shift+H (or a mapped button) shows them." }
]
delegate: ColumnLayout {
required property var modelData
spacing: 0
Controls.RadioButton {
text: modelData.text
checked: vpage.v.mode === modelData.value
onToggled: if (checked) backend.setVisibility("mode", modelData.value)
}
Controls.Label {
text: modelData.help
opacity: 0.7
font: Kirigami.Theme.smallFont
leftPadding: Kirigami.Units.gridUnit * 1.6
wrapMode: Text.Wrap
Layout.maximumWidth: Kirigami.Units.gridUnit * 26
}
}
}
RowLayout {
Kirigami.FormData.label: "Wrist:"
visible: vpage.v.mode === "gesture"
Controls.ComboBox {
model: [{ text: "Left controller", value: "left" }, { text: "Right controller", value: "right" }]
textRole: "text"
valueRole: "value"
Component.onCompleted: currentIndex = indexOfValue(vpage.v.gesture_hand)
onActivated: backend.setVisibility("gesture_hand", currentValue)
}
}
RowLayout {
Kirigami.FormData.label: "Look within:"
visible: vpage.v.mode === "gesture"
Controls.Slider {
id: gesture
from: 5; to: 60; stepSize: 1
value: vpage.v.gesture_angle
Layout.preferredWidth: Kirigami.Units.gridUnit * 12
onMoved: backend.setVisibility("gesture_angle", value)
}
Controls.Label { text: Math.round(gesture.value) + "° of it" }
}
Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Screens on a wrist" }
RowLayout {
Kirigami.FormData.label: "Show while facing you within:"
Controls.Slider {
id: wrist
from: 20; to: 120; stepSize: 1
value: vpage.v.wrist_angle
Layout.preferredWidth: Kirigami.Units.gridUnit * 12
onMoved: backend.setVisibility("wrist_angle", value)
}
Controls.Label { text: Math.round(wrist.value) + "°" }
}
RowLayout {
Kirigami.FormData.label: "All screens:"
Controls.Button {
text: "Pin to left wrist"
enabled: backend.desktopRunning
onClicked: backend.pinAll("left")
}
Controls.Button {
text: "Pin to right wrist"
enabled: backend.desktopRunning
onClicked: backend.pinAll("right")
}
Controls.Button {
text: "Unpin"
enabled: backend.desktopRunning
onClicked: backend.unpinAll()
}
}
}
Controls.Label {
Layout.fillWidth: true
wrapMode: Text.Wrap
opacity: 0.7
text: "Pin one screen: carry it by its bar and sweep its laser across your other controller. A "
+ "ring shows the target and a dot shows where the laser is; crossing the ring arms the pin (ring "
+ "and bar turn blue), crossing it again disarms it. Turn and place the screen the way you want, "
+ "then let go: it rides on that wrist at that size and distance, however far away. To adjust a "
+ "pinned screen, grab its bar, move it, and let go (it stays pinned); sweep across the ring to "
+ "take it off. It shows while you see its front within the angle above, and fades out beyond "
+ "it. Save current arrangement (Layout) keeps pins."
}
}
}
}
}
docs/design.md
+201
View File
@@ -0,0 +1,201 @@
# Frametop design
Last updated: 2026-09-25.
## Goal
- Two or more desktop screens shown as panels in SteamVR, placed anywhere around the user (arc, stacked, tilted, behind).
- One physical mouse drives a cursor through that 3D layout. Moving the mouse moves the cursor across panels in the direction you expect from where they are in space.
- Works with a Bluetooth mouse. The keyboard should follow the focused panel.
- Desktops come from Linux on the Frame itself. PC streaming is out of scope for now.
## Approach: nested Plasma in ft-screens (2026-09-26; before: a PerWindow gamescope)
Since 2026-09-26 the nested KWin runs inside ft-screens (`screens/`), our own wlroots compositor, instead of gamescope: gamescope draws every window into one canvas of at most 1920x1080 pixels and leaves the panels to the dashboard, which caps their size. ft-screens sizes each KWin screen on its own and shows it as its own SteamVR overlay that we place and size (grab bar, resize handle, pin to a hand, hide/show). The findings log (2026-09-26) has the details. The gamescope notes below still describe `BACKEND=gamescope`.
Everything it uses ships with SteamOS.
- `session/frametop-session.sh` starts its own `gamescope --backend openvr --virtual-connector-strategy PerWindow`. Inside it runs a full Plasma session whose KWin has `--output-count N`. The nested KWin opens one window per output, and PerWindow makes each window its own SteamVR overlay.
- It's modeled on `/usr/bin/steamos-nested-desktop`, SteamOS's single-screen desktop in VR, and runs alongside it. It has its own `XDG_RUNTIME_DIR` (`/run/user/1000/frametop`), config (`~/.config/frametop`), and state (`~/.local/state/frametop`).
- It's controlled from the PC with `desktops.sh`.
Verified in the headset on 2026-09-25:
- Two 1920x1080 panels, each with wallpaper, sharp.
- A taskbar, and apps can be launched.
- Panels can be moved in VR.
Still open:
- Do windows drag between screens? This needs a mouse or keyboard connected.
- What overlay keys does gamescope give each window? Read gamescope's OpenVR backend source (ValveSoftware/gamescope).
- Does the `Gamescope WSI Layer Error: Creating swapchain for non-Gamescope swapchain` in the session log affect any apps? Apps inside the nested desktop inherit `ENABLE_GAMESCOPE_WSI=1`.
## Launcher, config, and panel placement
- Launcher: `desktops.sh install` writes `~/.local/share/applications/deckard-nested-desktop.desktop`, which overrides the stock `/usr/share/applications/deckard-nested-desktop.desktop` (`Exec=steamos-nested-desktop`) by filename. It keeps `X-Steam-Special=Desktop`, which the Frame's non-Steam app launcher uses to pick out the Desktop entry. `uninstall` removes it.
- Config: `~/.config/frametop.conf` (`SCREENS`, `WIDTH`, `HEIGHT`, `PHYS_WIDTH`).
- Placement (researched 2026-09-25): gamescope's OpenVR backend (upstream `src/Backends/OpenVRBackend.cpp`) creates every panel with `CreateDashboardOverlay`, keyed `gamescope.<wl_display>.window.<n>` for non-Steam windows, and never sets a transform. The SteamVR dashboard (web UI, `resources/webinterface/dashboard`) owns placement. Each panel is a "frame" with a dock location: `Dashboard`, `Theater`, `World` (the "Float In World" menu action), `LeftHand`, `RightHand`, or `Boot`. No `/settings/dashboard/*` key sets the initial dock location, and dock state and transforms aren't persisted, only `lastAccessedExternalOverlayKey`.
- Default floating and a layout (decided 2026-09-26: automating the dashboard won over a patched gamescope, which would have lost SteamVR's window controls): `layout/ft-layout` floats each screen with `vrcmd --dock-overlay` and the pointer helper carries it into place with the invisible controller. See the 2026-09-26 findings and `README.md`. With `--virtual-connector-strategy PerWindow` and `--vr-overlay-key frametop`, the keys are `frametop.app.<window seq>`; `.app.0` is gamescope's default connector and never has a window.
- Resolution per screen isn't possible with stock gamescope: it never sets its windows' sizes (no `xdg_toplevel` configure sizes), and draws each window into the one `-W`×`-H` composition, letterboxed. KWin 6.2 would resize a nested output on a configure, and `kscreen-doctor` offers only the one mode. Per-screen scale (KWin output scale, kept in the session's `kwinoutputconfig.json`) stands in.
## The universal 3D mouse
Goal (decided 2026-09-25): run all of SteamVR from inside the headset with a Bluetooth mouse and keyboard. That covers the dashboard, Steam, overlays, the Frametop desktop, and flatscreen apps. It works like the Apple Vision Pro's mouse: a small cursor floats in the room and "collides" with any panel, then acts like a controller laser on it.
Decisions:
- Universal, not only Frametop panels.
- The cursor is anchored in the room (world space), with recenter on demand.
- Controllers stay fully usable. Last used wins.
- Hands off VR games. A scene app with the dashboard closed gets nothing from the pointer. Flatscreen mouse games in Theater get the raw mouse passed through.
### Cursor model
1. The relay grabs the mouse (done: `input/input-relay.py`).
2. Relative motion becomes yaw and pitch of a direction anchored in the room. Recenter puts it straight ahead of the current head pose. Sensitivity is in degrees per count, with optional acceleration.
3. The pointer ray starts at the head (or a point just below the eyes) and runs along that direction. Anything it hits is the target.
4. The cursor sits on the hit surface when there is one. Otherwise it sits on a sphere at a set distance (1.5 m default).
5. Settings: distance, cursor size, snap margin (with hysteresis so it doesn't flicker at panel edges), sensitivity and acceleration, recenter key.
### Delivery: a virtual SteamVR controller
SteamVR's dashboard, and every overlay it hosts, is driven by the vrcompositor "lasermouse" action set:
- `Pointer`: the `/pose/tip` pose.
- `leftclick`, `rightclick`, `middleclick`, `back`, and `home`.
- `scroll_discrete` and `scroll_smooth`: `scroll`.
- `system`: `ToggleDashboard`.
- `quickrecenter`: `Recenter`.
The Frame controller's defaults are in `/opt/steamvr/drivers/frame_controller/resources/input/vrcompositor_bindings_frame_controller.json`.
A small OpenVR driver (`ft_pointer`) adds a virtual controller with no render model or laser of its own:
- Pose: the pointer ray (origin at the head, aimed at the cursor). A laser that starts at the eye and runs along your line of sight shows up as a dot, so SteamVR's own hit cursor becomes the floating mouse on panels.
- Inputs: mouse left to `trigger` (leftclick), right to rightclick, middle to middleclick, wheel to scroll, side buttons to back. A keyboard shortcut maps to system (ToggleDashboard) and to recenter.
- Its own `controller_type` (`ft_pointer`) with default bindings for `openvr.component.vrcompositor` and `steam.client`.
Open questions (spike 1):
- Role. The right hand collides with the real controller. The stylus role (`TrackedControllerRole_Stylus`, path `/user/stylus`) might be bindable without taking a hand.
- Does the Frame's dashboard follow a third pointer device? Is last used wins automatic (`lasermouse_secondary/switchlaserhand` exists)?
- Hiding the device from VR games: report the pose as invalid, or deactivate, whenever a scene app has focus and the dashboard is closed.
### Processes
- `input-relay.py` (host, user service): in pointer mode it sends mouse deltas and buttons to the helper over a Unix socket. In passthrough mode (pointer off, or a flatscreen game in Theater) it forwards to the virtual uinput devices as now.
- `ft_pointer` driver (inside vrserver, host): the virtual controller. It takes its pose and buttons from the helper over a Unix socket. It's built for the host ABI (glibc 2.39; the `dev` container has 2.43) in a Fedora 40 build container.
- `ft-pointer` helper (OpenVR client, in the container): cursor state, recenter, mode switching (IVROverlay `IsDashboardVisible` and the scene app's focus), the free-space cursor overlay, and settings.
Keyboard: still open. SteamVR opens keyboards (vrserver held the Z3 Keyboard), but routing physical keys into dashboard text fields and gamescope panels needs its own spike.
### Spikes
1. Driver: a minimal `ft_pointer` virtual controller with a fixed pose in front of the HMD and a scripted trigger. Does the dashboard laser follow it, and do clicks work? Try the right-hand and stylus roles.
2. Mouse-driven pose: the relay feeds the helper, the helper feeds the driver. Room-anchored cursor, recenter.
3. Free-space cursor overlay, and hiding it when SteamVR's hit dot is on a panel.
4. Modes: VR game focus (off), Theater flatscreen (passthrough), dashboard and overlays (pointer).
5. Keyboard routing.
## Phases
1. MVP: two panels. The mouse cursor crosses between them in 3D, with click and scroll working.
2. Layout: N panels, save and restore panel placement, recenter.
3. Daily use: keyboard focus follows the cursor, launch apps onto a chosen panel, cursor visuals, sensitivity settings, autostart.
## Rejected approaches
- WayVR (wayvr-org/wayvr), tried 2026-09-25 and removed. It built for aarch64 in the `dev` container and connected to SteamVR, but we couldn't see or open it in the headset. It has no bindings for `frame_controller`, so it relies on SteamVR's Oculus remap, and its KDE screen capture needs `xdg-desktop-portal-kde`, which the Frame lacks. A GitHub fork, `DeeJanuz/wayvr`, was created for it.
- A custom capture app (headless KWin, then `zkde_screencast`, then DMA-BUF, then OpenVR overlays). It's workable, but gamescope PerWindow does the same job with no code.
## Findings log
- 2026-09-26: Programs that stop their own helpers when their window closes still lose them on a desktop restart, even after keep-apps.sh moved them out of the unit (22 processes moved, all gone 7 s later when the compositor went): nothing outside the app can keep them. Run such work somewhere independent of the desktop, like a systemd user service. Controls: shown only while some controller ray passes within max(1.5 x button, 12% of the bar) of a control (5 points along the bar, each button, the tab), 0.4 s linger; a zone covering the lower quarter of the screen showed them whenever a screen was in use.
- 2026-09-26: Restarting the desktop killed everything started in it, including background servers and the jobs they ran: `systemctl --user stop frametop-desktop` kills the unit's whole cgroup, and apps launched in the nested session never get scopes of their own (KIO uses systemd scopes only when systemd is on the session bus, and the session runs on a private dbus-run-session bus). Now `desktops.sh stop` first runs session/keep-apps.sh, which moves every process in the unit except the session's own (session script, dbus, KWin, Plasma, the session services it starts; matched by comm name) into a new transient scope (StartTransientUnit with PIDs, via busctl). GUI apps still exit when the compositor goes; whether an app's children outlive it is up to the app. Controls now fade in only while some controller-class device's ray (the 3D mouse's virtual controller included) meets the screen's plane in its lower quarter or just below it, and linger ~0.8 s.
- 2026-09-26: Roll button and quieter controls (feedback: tilting screens sideways was hard; the new corner tab was too big; controls should be smaller and translucent like SteamVR's). Roll is a knob: at the press the laser's angle around the screen's centre (in the plane of the pose at the press) is recorded, and the screen turns by how far that angle has moved (RollZ about its own front axis; pinned screens roll their controller->screen transform). Within 2.5° of level (the right edge's slope) it snaps level; scrolling on the button steps 5°. Controls: bar sqrt(0.012 x distance x width), buttons and tab max(13% of the bar, 1.8% of the distance); textures are a translucent light pill and dark translucent discs with white glyphs (3x supersampled), and every control sits at 55% overlay alpha until a laser is on it (VREvent_MouseMove / FocusEnter / FocusLeave on the control) or it's being dragged.
- 2026-09-26: Screen controls and pointer depth (feedback: the wrist ring was too big; the bar and corner handle only followed distance, looked detached from flat screens; the pointer was unreliable between things close together in view at different depths). Wrist ring 6 cm (leave at 9 cm). Controls: bar width = sqrt(0.02 x distance x width), at least 4% of the distance, at most 60% of the width; the resize control is a quarter-disc tab whose corner sits on the screen's corner (the old L floated 5 cm off it); on curved screens the bar gets the screen's radius and the button and tab are placed on the cylinder (angle u/r, facing the axis). They're re-sized every half second when the distance changes by more than 8%. OpenVR's intersection mask (SetOverlayIntersectionMask) doesn't affect ComputeOverlayIntersection, so it can't be checked from code; the tab stays entirely outside the screen instead. ft-pointer: the cursor's ray starts at the recenter anchor, not the eye, so after leaning it could land on a panel that a nearer one covers from the eye; a second test along the eye's line of sight to the cursor point now takes the nearer thing.
- 2026-09-26: Wrist pinning, third version (in testing, the target wasn't visible, and a screen that pinned mid-carry was stuck at the angle the carrying hand had while pointing at the wrist). Now the laser entering a controller's 10 cm ring (leaving it past 14 cm, so it doesn't flicker) toggles an armed state, and the pin happens on release with the pose at that moment. Grabbing a pinned screen starts armed for its wrist, so moving it re-pins it. While a screen is carried, every other hand controller gets a ring overlay (its zone, facing the head, blue when armed) and a dot at the laser's closest point to it (within 35 cm, blue inside the ring); both are non-interactive overlays above the screens (sort order 20/21). Frametop Display Settings' pages moved from a collapsed side drawer, which was easy to miss, to tabs.
- 2026-09-26: Wrist pinning, second version (feedback: pinning should work by aiming, not touching, so big and far screens can ride on a wrist, and a pinned screen should only show from the front). While a screen is carried, the segment from the carrying device to its bar is tested against the other hand controllers (not ft_pointer); within 10 cm for 0.3 s, the screen pins as it is (controller->screen transform kept). A pinned screen's alpha follows the angle between its front and the direction to the head: 1 inside the wrist angle minus 10°, 0 beyond it (then hidden). Visibility modes (always / dashboard / gesture: gaze within N° of a controller / toggle) in ft-screens, with g_manual as "hidden" in always and "shown anyway" in the others. Poses are read once per tick. `get` reports the pin (hand and transform) so `ft-layout capture` keeps pins and `apply` restores them.
- 2026-09-26: ft-screens fixes after the first session in the headset (moving windows between screens worked; text is as sharp as the headset allows, so screens need to be bigger and nearer).
- The curved preset spaced screens by angle (2 atan(w/2d)), which assumes every screen sits on the circle. A flat 3.6 m screen's edges are 2.7 m away when its centre is 2 m, so its neighbours landed in front of its edges. Now each row is chained edge to edge like monitors on a desk: the middle screen (or seam) straight ahead at the distance, each neighbour hinged at the previous one's outer edge plus the gap and turned until it faces the eye (dot(centre, right) = 0, solved by scan and bisection; a fixed-point iteration diverges for screens nearly as wide as twice their distance).
- Resize took the larger of the ray's reach in x and y (y scaled by the aspect). The handle sits below the bottom edge, so moving inward without moving up never shrank a screen. Now the corner follows the ray along the diagonal, keeping the grab offset; minimum 15 cm.
- Push/pull scaled the device-to-screen offset. The 3D mouse's device sits just in front of the bar (the laser origin is near the cursor), below the screen's centre, so that offset points mostly up. Now it moves along the head-to-screen line.
- Pinning checked the screen's centre within 35 cm of a controller, but a big screen's centre is far from the edge you bring to your wrist. Now it's the nearest point of the screen's rectangle within 20 cm; the pinned screen shrinks to 32 cm, 12 cm from the controller, and gets its width back when grabbed.
- Curvature: `SetOverlayCurvature` (the fraction of a full cylinder the width covers, width / 2πr). The curve button uses the head's distance as the radius, so the screen wraps around you; the width can change and the radius stays. ComputeOverlayIntersection and the mouse coordinates follow the curve.
- A button release is delivered to the panel under the laser, maybe another screen's, so any release on any of our panels ends that device's drags.
- 2026-09-26: ft-screens, a gamescope replacement (decided: our own panels, always visible with a hide hotkey, switch as soon as it works; acceptance test: an ultrawide between two portrait screens). Spike works: three native panels, 1080×1920 / 3440×1440 / 1080×1920.
- OpenVR's public `IVRIPCResourceManagerClient_003` (`VRIPCResourceManager()`: `GetDmabufModifiers`, `ImportDmabuf`, `UnrefResource`) is how gamescope hands SteamVR its frames; a texture of type `TextureType_SharedTextureHandle` then shows it. No size limit. The Frame's runtime supports it (and `IVROverlay_028`), but SteamVR's bundled `hellovr` header predates it, so builds fetch Valve's public header (v2.15.6). SteamVR imports XRGB8888/ARGB8888 with `LINEAR` and `0x0500000000000001` (Qualcomm compressed).
- First try, KWin screencast virtual outputs (`zkde_screencast_unstable_v1.stream_virtual_output`, v3 in KWin 6.2.5): KWin crashed in `WorkspaceSceneOpenGL::textureForOutput` (std::out_of_range) streaming one while nested in gamescope, three times (the wrapper restarted it, plasmashell didn't come back until restarted by hand). In 6.2.5 only the DRM and nested Wayland backends implement `createVirtualOutput`, not `--virtual`. The nested backend's version is just another host window: `createOutput(name, size * scale, scale)`.
- So ft-screens is a minimal wlroots 0.20 compositor (`screens/compositor.c`, C) that hosts the nested KWin, plus an OpenVR side (`screens/vr.cpp`). KWin's nested backend needs `wl_compositor` v4+, `wl_shm`, `wl_seat`, `xdg_wm_base`, and `zwp_linux_dmabuf_v1` v4 (feedback: main device, format table); pointer constraints/gestures, relative pointer, and xdg-decoration are optional. We configure each KWin window's size on its first commit (`wlr_xdg_toplevel_set_size`) and KWin sizes that screen to match; each committed DMA-BUF goes straight to SteamVR, held until the next one. Frame callbacks at 90 Hz. Idle cost: ft-screens 1% CPU, KWin 1%.
- wlroots details: `wlr_shm_create` wants DRM format codes; KWin asks for server-side decorations before its first commit, and setting the mode then asserts (`surface->initialized`), so it's answered on the first commit.
- 2026-09-26: Three screens (3440×1440 centre, portrait sides) meet two gamescope limits.
- One size and shape for every screen: a 540×960 test window opened straight in Frametop's gamescope (`WAYLAND_DISPLAY=/run/user/1000/gamescope-1`) became a landscape 16:9 panel (0.59 × 0.33 m): gamescope composes each window into the shared `-W`×`-H` canvas, letterboxed. Portrait screens are rotated KWin outputs (`kscreen-doctor output.N.rotation.left`) with their panels rolled 90° in VR (`place ... roll`); a rolled placement lands exactly (it took two moves).
- At most 1920×1080 worth of pixels: `upload_buffer_size = 1920 * 1080 * 4` (rendervulkan.hpp), and the OpenVR backend uploads a flat texture of the whole output at start (`vulkan_create_flat_texture(g_nOutputWidth, g_nOutputHeight, ...)`). 3440×1440 aborted gamescope at start (`uploadBufferData: Assertion 'size <= upload_buffer_size' failed`). The largest 21:9-ish size is about 2224×928. Beyond that needs a patched gamescope.
- A new window starts docked in the dashboard, where `--dock-overlay dashboard` is ignored as redundant and doesn't open the dashboard, so `world` then fails. `float_screen` docks to theater first.
- For scale: a 1.18 m panel at 1.2 m spans about 52°, and the Frame resolves roughly 20-25 pixels per degree, so about 1,200 pixels across; more pixels only help on bigger or nearer panels.
- 2026-09-26: Screens float and arrange themselves (tested with one screen; two screens pending).
- `vrcmd --dock-overlay <dashboard|world|theater|lefthand|righthand> <key>` sends the dashboard `vrcmd_dock_overlay`. An unknown key moves the dashboard's active frame instead (`GetFramesWithAssociatedSummonKeys(key)[0] ?? activeFrame`), and a key already at that location is ignored ("redundant"). `world` takes its first transform from the dashboard's position (`setInitialTransformForLocation` → `requestSGTransform(GetDockLocationTransformID(Dashboard))`), which fails while the dashboard is closed ("Invalid transform ID"): the panel floats with no position and shows only with the dashboard. Working order: dock `dashboard` (opens the dashboard), dock `world`, `--hidedashboard`.
- Floating panels' transforms can't be read (DashboardTab, type 5), but `ComputeOverlayIntersection` works on them and returns UVs (v bottom to top). Casting rays from the head over the sphere and fitting point = O + u·U + v·V gives centre, size, and frame exactly; 230,000 rays take 0.1 s (`md::ScanPanel` in `pointer/common/vrmath.h`, `vrprobe --scan`). A floating 16:9 panel measured 1.181 × 0.664 m with `PHYS_WIDTH=1.6`.
- The grab bar: below a floating panel SteamVR's laser hits three bands, 2-4, 6-9, and 14-26 cm below the bottom edge; 7.5 cm is the grab bar (`UndockedOverlay`: `onMouseDown: startFloatingWindowMove` on a 350 px bar at `{y: -0.26}` from the frame controls). `valve.steam.gamepadui.floatingfooter` never became visible there.
- The move (`startFloatingWindowMove`): the panel is parented to the device that clicked (head, left, or right hand by the mouse event's input path) with the relative transform at the press, and at the release it's `device × relative × pushTransform` (push = scroll, along the panel normal, whole notches of ~7 cm; fractional scroll does nothing). Drops within 0.3 m of the open dashboard or 0.4 m of the other hand snap there, so the dashboard is closed first. Rotations about the device origin carry the panel exactly (±20° yaw, ±8° pitch: 0.0° error). Slow translations do too (0.1 m in 1.5 s: 9.9 cm, no rotation). The first try, 0.1 m in 12 jerky 25 ms steps right after the press, moved it 0.19 m and turned it 8.5°. With a hover before the press, rotation first, then a smooth 60 Hz slide, placement was exact (0.0 cm, 0.0°) at every slide speed tried, 0.07 to 1 m/s, including a 66° swing and a panel facing away from the eye. `place` takes 2.5-3.5 s for a floating screen and about 8 s from docked.
- The helper answers `place`/`measure`/`head` by datagram to the sender's abstract address (`recvfrom`). The first version reset the sender length before replying, so replies went nowhere.
- 2026-09-26: The pointer with the dashboard closed (user-tested: "exactly how I want the pointer to work").
- gamescope's app panels (`frametop.app.N`, the desktops) report a 0x0 texture like scene-graph overlays, but their transform type is DashboardTab (5), so `GetOverlayTransformAbsolute` fails and the plane test skipped them. `ComputeOverlayIntersection` hits them normally. Only absolutely placed 0x0 overlays are scene-graph now. They stay visible when the dashboard closes.
- Off a panel, the cursor jumped to `POINTER_DISTANCE` (1.5 m), behind the panel (~1 m), and the laser started behind the panel's resize margins and window controls. It now stays on the last panel's plane within `POINTER_EDGE_REACH` (0.3 m). The floating-window controls only show while the panel is hovered, so the overlay list includes hidden overlays, and visibility is re-read every 50 ms.
- With the dashboard closed, SteamVR's laser mouse is off until a click, even while our device is the primary dashboard device (`GetPrimaryDashboardDevice` stayed ours; `system.pointer` stayed hidden until a trigger press). So the first click on a panel only turned the laser on, and leaving every panel turned it off again. A held Frame controller keeps it on by itself. Fix: `VROverlayFlags_MakeOverlaysInteractiveIfVisible` ("the system-wide laser mouse mode will be activated whenever this overlay is visible") on a transparent 1 mm overlay, `frametop.pointer.lasermode`, 50 m below the head, shown only while the pointer is awake. vrcompositor's strings (`overlaysForcingLaserMouseOn`, `force_activate_laser_mouse`) led to it. Re-sending the claim pulse when the primary device went invalid didn't help and was removed.
- 2026-09-25: The tilt was also lost on the left release. SteamVR's dashboard finishes a floating move up to 150 ms after mouseup (`endFloatingWindowMove` races `updatePushDistance` against `s_flFinalPushMeasurementMS` = 150) and re-reads the controller pose, which had already gone back to plain pointing. The helper now holds the drag pose (tilt and frozen distance) for 0.5 s after the left release.
- 2026-09-25: Tilt works (user-tested), but it reverted when the right button was released: the device pose went back to plain pointing and the still-grabbed panel followed. Now the tilt angles accumulate per drag and stay applied (about the current cursor point) until left is released. They reset on each left press and release.
- 2026-09-25: Small controls explained by the helper debug log (`debug` command). Over the undock-type controls the helper saw FREE space, put the catcher at 1.5 m and the laser origin at 1.39 m, while the controls were about 1.1-1.2 m away. `valve.steam.gamepadui.bar` (dock) and `valve.steam.gamepadui.floatingfooter` (floating-window controls) are visible absolute overlays with texture 0x0 and a placeholder width of 1.0 m. The dashboard draws them through its scene graph, so `ComputeOverlayIntersection` never hits them. The helper now plane-tests texture-less overlays within `POINTER_SCENE_RADIUS` (0.5 m) and puts the catcher 5 cm behind that plane. The drag lock (freeze the cursor distance while left is held) fixed resize snap-back; the user confirmed it.
- 2026-09-25: User feedback round.
- The Buttons page hid bindings whenever the Z3 slept, because it listed only connected devices. The app now also lists devices with saved rules or bindings.
- "Toggle dashboard" did nothing: the relay sent the system button's press and release together. It now wakes the pointer and holds the button for 0.12 s.
- Small floating controls (undock, frame buttons) sit a few cm in front of their panel. With the laser origin at 0.95-0.98 of the way, the laser started behind them. `POINTER_ORIGIN_MARGIN` (0.15 m) keeps the origin in front.
- Tilt: the driver takes `posq` (full quaternion), and the helper rotates the device pose around the grab point while left and right are held. SteamVR's floating move (`UndockedOverlay.startFloatingWindowMove`) keeps the panel rigid with the controller (`m_sMoveDevicePath`), so this should turn the panel. Not yet tested; it needs a SteamVR restart to load the driver.
- 2026-09-25: Input relay v2 and the settings app. The relay has per-device roles (pointer, passthrough, ignore) keyed by Bluetooth address (EVIOCGUNIQ) or USB ids. It no longer grabs keyboards by default: they used to be swallowed into the virtual keyboard, which nothing types from. It has per-device button maps to named actions and a control socket `@frametop_relay` (devices, watch, reload; reload also reaches the helper, which now re-reads `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, and `POINTER_ORIGIN_FRACTION` live). `input-settings` (Kirigami and PySide6, in the container) was tested headless against the live relay with fake uinput devices: listing, roles, capture, mapping, role change, settings, and Bluetooth all work. The UI was checked through screenshots of the VNC display. Clicking through VNC, then RDP, then KWin fake input is too lossy for scripted UI tests.
- 2026-09-25: Controller handoff works for all three devices (left, right, mouse). The driver switches its role hint (Right while connected, OptOut while not), because SteamVR keeps a hand role reserved for a disconnected device that still hints it. The helper releases the pointer when a real controller moves, or when ours hasn't got the hand role within 1 s: SteamVR gives a contested role to the most recently used device, and a held Frame controller counts as used through its touch sensors.
- 2026-09-25: The beam width can't be switched live. `dashboard.laserRayWidthScale` set by any client (IVRSettings, `vrcmd`, even followed by a `laserLength` nudge or `VREvent_DashboardSectionSettingChanged`) is saved but not applied. Only the dashboard's own Settings screen, or a SteamVR restart, applies it. So the width stays at whatever it was set to. The helper starts the laser at 0.95 of the eye-to-cursor line (a few cm of beam along the line of sight, and SteamVR's hit dot is tiny), and draws its own white dot everywhere: `frametop.pointer.marker` (not interactive) on panels, `frametop.pointer.cursor` (interactive, catches the laser) in free space.
- 2026-09-25, spike 3 (helper) works. It looks right: only a dot, anchored to panel surfaces, with a floating white dot in free space.
- Bug fixed: driver poses are in raw tracking space, and client math is in the standing universe (on the Frame, standing is ~1.6 m above raw). Sending standing coordinates put the laser origin 1.6 m above the head, which also inflated SteamVR's hit dot. The helper now converts via the HMD pose in both universes each frame (`TrackingUniverseRawAndUncalibrated`).
- SteamVR's hit dot (`system.pointer`, transform type 4, not readable as absolute) is sized by distance from the laser origin. The origin sits `POINTER_ORIGIN_FRACTION` (0.5) along the eye-to-cursor line, which stays invisible and halves the dot.
- Controller laser not returning: the dashboard pointer flipped 2 → 1. Suspected cause: the relay re-claimed the laser on tiny mouse movement after a pause (sensor jitter). Fix: `POINTER_WAKE_COUNTS` (40 counts in 1 s) before waking or re-claiming. Clicks and scroll wake immediately.
- 2026-09-25, spike 2 (the mouse drives the pointer): it works end to end. The Z3's motion aims SteamVR's laser, and left click, right click, and scroll act on the dashboard. The invisible render model works: `{ft_pointer}/rendermodels/ft_pointer_invisible` is one tiny triangle with a transparent texture. The claim button (`/input/a` bound to `lasermouse_secondary/switchlaserhand`) takes the laser without clicking.
- Laser looks: `dashboard.laserRayWidthScale` (not in the Settings UI; default 1.0) = 0 hides the beam while it hits a panel. `dashboard.laserLength` is the Settings UI's "Laser Pointer Length" (0.5 = 50%, default). Neither hides the laser when it hits nothing. With an eye-origin ray the beam is still visible, because of stereo (each eye is ~31 mm off the ray).
- `vrcmd --overlays` lists every overlay (key, visibility, type, flags, handle): `system.systemui` (Steam UI), `valve.steam.desktopgame.*`, `gamescope.*`, `system.pointer` (the laser hit dot), and so on. `vrcmd --compositorcmd dump_laser_overlays` isn't handled by this build.
- Next (spike 3): the pointer helper. Relay → helper → driver. The helper does collision (enumerate visible overlays, `ComputeOverlayIntersection` from the anchor, cursor snaps onto the surface). It adds a laser-catching dot overlay at the cursor point in empty space, so the laser always hits something. It toggles the laser width by who owns the dashboard pointer.
- 2026-09-25, spike 1 results:
- Holding the right-hand role while SteamVR starts leaves the Steam UI stuck on its loading icon. So the driver starts disconnected (`deviceIsConnected=false`) and only connects on `show`.
- Once connected it becomes SteamVR's right hand (`GetTrackedDeviceIndexForControllerRole(Right)` = our device, even with the real controllers on). The real controllers still had their lasers.
- The dashboard's pointer device (`IVROverlay::GetPrimaryDashboardDevice`) goes to whichever device summoned the dashboard or last pressed its trigger. The headset's side button selects the HMD head pointer (device 0).
- With `/pose/raw` bound as `lasermouse/Pointer` and `lasermouse_secondary/switchlaserhand` on the trigger: a virtual trigger press moved the pointer to our device, and our system button closed and reopened the dashboard with ours as the pointer. `/pose/tip` didn't work, because tip comes from a render model and ours has none.
- Still to see in the headset: our dot anchored in the room (`install.sh aimhere`), and a click landing on a target.
- `pointer/probe/vrprobe` (OpenVR background client) prints devices, roles, the dashboard pointer, and head yaw and pitch. `vrcmd --info` also prints "Dashboard pointer device".
- 2026-09-25: Spike 1 prep. `pointer/driver/` holds the `ft_pointer` driver: a virtual controller with room-anchored `aim`/`gaze` pose from the head, buttons over the abstract datagram socket `@ft_pointer`, and default vrcompositor bindings to the lasermouse actions for both hands. It builds in `dev` for the host with `-static-libstdc++ -static-libgcc -Wl,--exclude-libs,ALL -fno-math-errno`: libm's `sqrtf` is versioned `GLIBC_2.43` in the container, which is newer than the host's 2.39. Its only export is `HmdDriverFactory`. Installed to `~/.local/share/frametop/ft_pointer` and registered with `vrpathreg adddriver`. Not loaded yet: that needs a SteamVR restart.
- 2026-09-25: The Z3 stopped reaching the desktop after it reconnected at 18:00:20 (sleep), seven minutes after SteamVR started. `vrserver` and `vrcompositor` held `/dev/input/event5`-`event8` as `(deleted)`. Restarting SteamVR fixed it once. The durable fix is `input/input-relay.py`: uinput virtual mouse and keyboard, created before SteamVR, plus EVIOCGRAB relay of USB and Bluetooth mice and keyboards with hotplug. Tested on the Frame with `--no-grab` and a fake USB uinput mouse: motion and BTN_LEFT relayed, unplug released cleanly, Z3 Mouse and Z3 Keyboard picked up, and the Z3 joystick nodes ignored. Enabled as a user service, not yet started.
- 2026-09-25: Chromium (Flatpak `org.chromium.Chromium`, system install) didn't launch from the taskbar. Plasma ran `kde-open appstream://org.chromium.Chromium`, which opens Discover, because the nested session had no `XDG_DATA_DIRS` and so no Flatpak exports. Fix: the session script sources `/etc/profile.d/flatpak.sh` (with a default `XDG_DATA_DIRS` first, since the script runs `set -u`). With the right env, Chromium runs fine in the nested session over Xwayland. Harmless log noise: `vaInitialize failed` (no VA-API) and a GCM `DEPRECATED_ENDPOINT`.
- 2026-09-25: VNC only. RealVNC Viewer can't speak RDP. Bridge: krdp on `127.0.0.1:3390`, then `xfreerdp` full screen inside `Xvnc :20`, then VNC on `<tailnet ip>:5900`. Verified with a screenshot of `:20`.
- 2026-09-25: Changing `SCREENS` can orphan Plasma panels. With 2 screens the taskbar panels were saved with `lastScreen=1`. After switching to 1 screen, no taskbar showed. Fixed by moving `~/.config/frametop/plasma-org.kde.plasma.desktop-appletsrc` and `plasmashellrc` aside (`*.bak-2screens`). TODO: handle this in the session script when the screen count shrinks.
- 2026-09-25: Remote desktop. `krfb` needs `xdg-desktop-portal-kde` (plugins `pw` and `xdp` only on Wayland), and `wayvnc` is wlroots-only. `krdpserver --plasma` (krdp 6.7.5, Fedora) uses KWin's screencast and fake-input protocols directly. It works against the nested KWin 6.2.5 once `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1` is set, which is needed because KWin can't match a container binary to a desktop file. Tested with `xvfb-run xfreerdp`: the right password connects and streams H.264 (OpenH264, no VA-API), and a wrong password is rejected at PostConnect. Reachable from the Mac over the tailnet.
- 2026-09-25: gamescope doesn't always exit on SIGTERM when started from the Steam launcher. `desktops.sh stop` waits 10 s, then sends SIGKILL.
- 2026-09-25: A test of gamescope PerWindow with a bare nested `kwin_wayland --output-count 2` produced two separate overlays. The full Plasma version then showed two sharp 1080p desktops with wallpaper, a taskbar, and movable panels.
- 2026-09-25: A headless `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1 kwin_wayland --virtual --output-count 2` runs next to the VR session. With the env var it exposes `zkde_screencast_unstable_v1` and `org_kde_kwin_fake_input`. Without it, KWin hides them. That's useful if the helper injects input through fake_input.
- 2026-09-25: SteamOS runs its own nested Plasma session inside gamescope (`/run/user/1000/nested_plasma`, socket `wayland-0`, X display `:2`). This is the built-in single-screen desktop in VR. Don't touch it.
- 2026-09-25: On the Frame, `gamescope --virtual-connector-strategy` accepts `SingleApplication`, `SteamControlled`, `PerAppId`, and `PerWindow`. The main VR session uses `PerAppId`.
docs/reference.md
+132
View File
@@ -0,0 +1,132 @@
# Frametop
Several desktop screens floating in SteamVR on the Steam Frame, driven by a physical mouse (Bluetooth or USB) whose cursor moves through 3D space. The cursor crosses from one screen to the next based on where the screens actually sit around you, not on a flat monitor layout.
How it works: a nested Plasma desktop runs inside ft-screens (`screens/`), our own small Wayland compositor. KWin opens a window per screen; ft-screens gives each one its own size, so every screen is a real monitor of any resolution and shape (ultrawide, portrait, 4K), and shows each as its own SteamVR panel, with KWin's frames passed to SteamVR as they are (no copy). The panels are ours: any size in metres, placed exactly by the layout, with a grab bar to move them, a handle to resize them, and pinning to a hand. See `docs/design.md`. (The older gamescope backend is still there: `BACKEND=gamescope`.)
Status: the multi-screen desktop works on ft-screens (acceptance test: a 3440 × 1440 ultrawide between two 1080 × 1920 portrait screens, wallpaper on each, the taskbar on the ultrawide). The universal 3D mouse works with the SteamVR dashboard, Steam, overlays, and the screens.
## Run
From the headset: open "Launch a program", then "Desktop". After `install`, that entry starts Frametop instead of the stock single-screen desktop.
From a terminal, on the Frame or from a PC over SSH:
```
desktops.sh install # launcher "Desktop" starts Frametop (writes ~/.local/share/applications/deckard-nested-desktop.desktop)
desktops.sh uninstall # launcher gets the stock SteamOS desktop back
desktops.sh screens 3 # default screen count
desktops.sh start [screens] | stop | restart | status | log [lines]
```
Settings: the screens (resolution, width in metres, scale, taskbar screen) and the layout are in `~/.config/frametop-layout.json`; `BACKEND`, `REMOTE`, and the pointer settings in `~/.config/frametop.conf` (see `session/frametop.conf.example`). Frametop Display Settings (below) edits both.
The session script is `session/frametop-session.sh`. It runs on the Frame host and starts ft-screens in the `dev` container (`/tmp/frametop-screens.log`), then KWin and Plasma on the host inside it. Only one instance runs at a time, and `desktops.sh start` runs it in its own systemd unit (`frametop-desktop`). It keeps its own Plasma config in `~/.config/frametop`, separate from the stock desktop.
Restarting the desktop (`desktops.sh restart`, or Restart desktop in Frametop Display Settings) closes its windows, but programs started in it keep running when they can: `session/keep-apps.sh` moves them out of the desktop's systemd unit first. Background work like servers, tmux, and builds survives. An app that stops its own helpers when its window closes still loses them; for work that must survive, run it outside the desktop, for example as a systemd user service.
## ft-screens (the compositor)
`screens/compositor.c` (wlroots 0.20) hosts the nested KWin; `screens/vr.cpp` is the SteamVR side. Build: `screens/build.sh` (the installer does it).
- Screens: each KWin window is a screen. ft-screens sizes it (`xdg_toplevel` configure) and KWin resizes that screen to match, live. Frames arrive as DMA-BUFs and go to SteamVR with `ImportDmabuf` (OpenVR's `IVRIPCResourceManagerClient`), no copy, no size limit.
- Panels: `frametop.screen.N`, with `.bar` (move), `.curve` (the round button next to it), `.roll` (the next one: drag it sideways like a knob to roll the screen, snapping level within 2.5°, or scroll on it for 5° steps), and `.resize` (the tab on the bottom right corner; screens go down to 15 cm wide). The controls are sized from both the screen's width and its distance from you, sit on its surface when it's curved, and are translucent like SteamVR's own until a laser is on them. They're invisible until a laser or the 3D mouse's cursor comes very close to one of them (about 1.5 times a button's size), and fade out a moment after it leaves. Drag the bar with any laser (a controller, or the 3D mouse, whose right-drag tilt works too); scroll while dragging to push it away or pull it closer (along the line from your head). The curve button bends the screen into a cylinder around you (radius: your distance to it), or flat again. Pin to a wrist: while carrying a screen, sweep the laser (the line from whatever carries it to its bar) across your other controller. A ring around that controller shows the target and a dot shows where the laser passes; entering the ring arms the pin (ring and bar turn blue), entering it again disarms it. Let go while armed and the screen rides on that controller as it is then, at its size and distance (a 3.6 m screen 5 m away works; so does pinning all screens), so you can arm it first and then turn it the way you want. Grab a pinned screen's bar to adjust it: it comes back to the same wrist when you let go, unless you sweep across the ring to disarm. A pinned screen shows only while you see its front within the wrist angle, fading over the last 10°.
- Visibility (Frametop Display Settings → Visibility & wrist tab): always (Meta+Shift+H, the Hide/Show Screens menu entry, or a mapped mouse button hides them), only with the SteamVR dashboard open, while you look at a chosen controller (the wrist gesture), or only when shown with the hotkey. In the last three, the hotkey shows them anyway. Visible screens keep SteamVR's laser on; hidden, VR games get their triggers back. (A controller button to show them isn't there yet: in a game the game owns the buttons.)
- Input: pointer from the panels to KWin through our seat; keys from the input relay (every keyboard it doesn't grab, and keys a pointer device passes through) to the screen that was clicked last, but not while the SteamVR dashboard is open.
- Control socket `@ft_screens` (datagrams, replies to the sender): `place N x y z yaw pitch roll`, `width N metres`, `curve N radius|on|off`, `pin N|all left|right [12 numbers]`, `unpin N|all`, `size N w h` (live resolution), `get N`, `screens`, `head`, `visibility always|dashboard|gesture|toggle`, `wrist degrees`, `gesture left|right degrees`, `hide | show | toggle`, `state`, `key code value`.
## Input relay (Bluetooth mice and keyboards)
SteamVR opens input devices only when it starts. A Bluetooth mouse that sleeps and reconnects gets new device nodes, and SteamVR keeps reading the dead ones, so the mouse stops working until SteamVR restarts. `input/input-relay.py` fixes that:
- It creates `frametop virtual mouse` and `frametop virtual keyboard` through `/dev/uinput` before SteamVR starts.
- It grabs every USB or Bluetooth mouse and keyboard as they come and go, and forwards their events. SteamVR only sees the virtual devices, which never go away.
- Service: `frametop-input-relay.service` (user unit, `Before=steamvr.service`, wanted by `steamvr.service` and `default.target`).
```
desktops.sh relay install # enable (starts on the next reboot or SteamVR start)
desktops.sh relay status | log | uninstall
```
- The first time, start it before SteamVR (reboot, or restart SteamVR after `relay install`), so SteamVR opens its virtual devices. After that, restarting the relay is safe: systemd keeps the virtual devices in its file descriptor store (`FileDescriptorStorePreserve=yes`), so SteamVR keeps the same devices.
- Test without disturbing SteamVR: `input-relay.py --no-grab`.
- This is where the 3D mouse will hook in.
## Universal 3D mouse
A Bluetooth mouse drives SteamVR like a controller laser, but it looks like a small dot floating in the room. It snaps onto panels and works on the dashboard, Steam, overlays, and this desktop. Details and findings are in `docs/design.md`, "The universal 3D mouse".
- `input/input-relay.py` (pointer mode, `POINTER=1`) sends mouse motion, clicks, and scroll to the helper. Deliberate movement or a click wakes it; 30 s idle releases it.
- `pointer/helper/ft-pointer` (`frametop-pointer.service`, runs in the `dev` container, starts with SteamVR) holds the room-anchored cursor. It does collision against every visible overlay, draws the white dot, and sends the driver an exact pose.
- `pointer/driver/` (`ft_pointer`, loaded by SteamVR) is an invisible virtual right-hand controller whose laser follows the cursor.
- Last used wins: picking up a controller hands the laser back at once, and moving the mouse takes it again. While you hold a controller the mouse steps aside.
- Moving panels: left-drag a floating panel's grab bar, and it follows the pointer around you (SteamVR's own move). The scroll wheel during a drag pushes and pulls it. **Tilt**: while left-dragging, hold the right button and move the mouse to rotate the panel around the grab point. The right press isn't sent as a right-click. The tilt stays for the rest of the drag: release right and keep moving the tilted panel, press right again to tilt further. Releasing left drops the panel as it is.
- Toggle dashboard (a mapped button or a Meta tap) wakes the pointer if needed and holds the virtual system button for 0.12 s. SteamVR ignores a press and release in the same instant.
```
pointer/driver/build.sh && pointer/driver/install.sh install # then restart SteamVR
pointer/helper/build.sh && pointer/helper/run.sh install # user service
pointer/helper/run.sh status | log | restart
pointer/driver/install.sh probe # devices, hand roles, who owns the dashboard pointer
```
Settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `POINTER_IDLE`, `POINTER_WAKE_COUNTS`, `POINTER_DISTANCE`, `POINTER_CURSOR_DEG`, `POINTER_ORIGIN_FRACTION`, `POINTER_ORIGIN_MARGIN`, `POINTER_SCENE_RADIUS`, `POINTER_EDGE_REACH`, and `POINTER_LASER_WIDTH`. See `session/frametop.conf.example`. Restart the relay or the helper after changing them.
## Frametop Input Settings (app)
A Plasma app (Kirigami, Python backend) to choose and map input devices. It's in the Plasma menu under Settings on the Frametop desktop (and the stock desktop). It runs in the `dev` container and talks to the relay's control socket `@frametop_relay`.
- **Devices**: every USB or Bluetooth mouse and keyboard, with a live activity light (move or press a device to find its row). Roles: **3D pointer** (grabbed, drives the pointer; default for anything with a mouse node), **Pass through** (not grabbed; default for keyboards; a Meta tap still toggles the dashboard), **Ignore**. A physical device is identified by its Bluetooth address (or USB ids and name), so all its nodes share a role.
- **Buttons**: pick a pointer device (devices with saved bindings are listed even while asleep or disconnected, marked "not connected"; **Forget** on the Devices page drops all of a device's saved settings), press **Capture a button**, press the button or key, then choose an action: left, right, or middle click, back, scroll up or down, toggle dashboard, recenter, pointer on or off, faster or slower, pass through as key, or do nothing. The Z3's extra buttons arrive as keys from its keyboard node. Each row has **Remove** (your own binding; the button passes through again), **Reset** (a changed built-in button goes back to its default), or **Unbind** (a built-in button does nothing). **Remove all** clears the device's custom bindings. `FT_INPUT_PAGE=buttons` opens the app on that page.
- **Pointer**: sliders for the `POINTER_*` settings, applied live (the relay and helper reload), plus Recenter.
- **Bluetooth**: paired devices, and **Apply Bluetooth fixes** (runs `/etc/steamframe/bt-fixups.sh` through `pkexec`) after pairing an LE device. Pair new devices in Steam.
Rules are saved to `~/.config/frametop-input.json` and pointer settings to `~/.config/frametop.conf`.
```
input-settings/install.sh # menu entry (ft-input-settings.desktop) -> host launcher ft-input-settings
```
The launcher gives podman the real `XDG_RUNTIME_DIR` and user bus, and gives the app the session's Wayland socket (absolute path) and bus. Without the real user bus, podman fails with `crun: ... cgroup.procs: Permission denied`, because the Frametop session runs on a private bus from `dbus-run-session`.
## Screens and layout (Frametop Display Settings, ft-layout)
When the desktop starts, its screens arrange themselves around where you're facing. Move them by hand any time; put them back with **Meta+Shift+R** in the desktop, the **Reset Screen Layout** menu entry, **Arrange now** in the app, or a mouse button mapped to **Reset desktop screen layout** (Frametop Input Settings → Buttons).
**Frametop Display Settings** (Plasma menu, Settings; Kirigami app in the `dev` container, like Frametop Input Settings):
- **Screens**: add and remove screens; each has a resolution (presets from 1080p to 4K, ultrawide, super ultrawide, portrait, or custom), a width in VR in metres (0.5 to 6), a scale, **Curved**, and **Taskbar here**. Resolution, width, and curve apply at once; adding or removing a screen when the desktop starts again (the app offers the restart).
- **Layout**: curved around you (screens hinged edge to edge like monitors on a desk, each turned to face you) or a flat wall, with rows, distance, gap, and height; or **Save current arrangement** to keep where you put the screens by hand (and their sizes). A preview shows it from above and from the front. **When the desktop starts** turns auto-arrange on or off.
`layout/ft-layout` does the work (Python standard library, on the host):
```
layout/ft-layout apply # arrange every screen (instant with ft-screens)
layout/ft-layout capture # save the current arrangement (and sizes) as the layout
layout/ft-layout plan # the arrangement as JSON (no VR needed)
layout/ft-layout scale # per-screen scale, side-by-side positions, taskbar screen to KWin
layout/ft-layout toggle # hide or show all screens
display-settings/install.sh # menu entries and the Meta+Shift+R / Meta+Shift+H shortcuts
```
The layout is saved in `~/.config/frametop-layout.json`, relative to your head when it's applied. `/tmp/frametop-layout.log` has the startup run. With `BACKEND=gamescope`, `ft-layout` floats each dashboard panel with `vrcmd --dock-overlay` and the pointer helper carries it into place (`place`), since SteamVR's dashboard owns those panels.
## Remote desktop (VNC)
With `REMOTE=1` in the config (`desktops.sh remote on`), the session also serves the VR desktop over VNC. Use RealVNC Viewer or macOS Screen Sharing.
- Address: the Frame's tailnet name or address, port 5900 (`desktops.sh remote info` prints it). It listens on the tailnet address only, not the LAN. It needs Tailscale on the Frame ([deck-tailscale](https://github.com/tailscale-dev/deck-tailscale)).
- Password: in `~/.config/frametop-remote/vnc-password` on the Frame. VNC limits it to 8 characters. `desktops.sh remote info` prints it.
- Encryption: VNC auth has none of its own, so viewers warn about an unencrypted connection. The traffic is still encrypted by the tailnet (WireGuard), which is why it listens only there.
- How it works: no VNC server can capture KWin on SteamOS. `krfb` needs `xdg-desktop-portal-kde`, which SteamOS lacks, and `wayvnc` is wlroots-only. So `session/remote-desktop.sh` captures the desktop with KDE's `krdpserver --plasma` on `127.0.0.1:3390` (never reachable from outside). `session/vnc-bridge.sh` runs TigerVNC's `Xvnc` on display `:20` with a full-screen FreeRDP client connected to it, and serves that over VNC. Everything runs in the `dev` container. The extra hop adds some latency.
- Security trade-off: with `REMOTE=1` the nested KWin runs with `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1`, so any app inside the Frametop desktop can capture its screen or inject input. This applies to that nested session only, not the stock desktop.
- To rotate the password, delete `~/.config/frametop-remote/` on the Frame and restart the desktop.
- Port 3389 is SteamOS's own `xrdp`, which starts a separate X11 session, not the VR desktop.
- Check what a viewer sees: `import -window root -display :20 /tmp/vnc.png` in the container.
## Limits
- Keyboard typing into the screens is wired (relay → ft-screens) but not yet tested with a real keyboard.
- There's no pin to the head (HUD) yet, and no controller button to show hidden screens (a mapped mouse or keyboard button works).
- The KWin cursor isn't drawn on the screens (KWin draws it as a host cursor, which ft-screens ignores); the 3D mouse's dot and SteamVR's laser dot show where you point.
- With `BACKEND=gamescope`: one resolution for all screens, at most 1920 × 1080 worth of pixels; arranging borrows the pointer for a few seconds; a SteamVR update that moves the floating window's grab bar would break arranging (`LAYOUT_GRAB_OFFSET`, the helper's `grabprobe`).
input-settings/ft-input-settings
Executable
+16
View File
@@ -0,0 +1,16 @@
#!/bin/bash
# Launch Frametop Input Settings from a Plasma session on the Frame host.
# The app runs in the dev container (PySide6 and Kirigami come from Fedora there).
# podman needs the real XDG_RUNTIME_DIR and the real user bus (to reach systemd for
# the container's cgroup; the Frametop session runs on a private bus from
# dbus-run-session). The session's Wayland socket and bus go to the app itself.
here=$(cd "$(dirname "$(readlink -f "$0")")" && pwd)
wl=${WAYLAND_DISPLAY:-wayland-0}
case $wl in /*) ;; *) wl="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/$wl" ;; esac
session_bus=${DBUS_SESSION_BUS_ADDRESS:-}
export XDG_RUNTIME_DIR=/run/user/$(id -u)
export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
exec "$HOME/.local/bin/distrobox" enter dev -- env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \
QT_QPA_PLATFORM="wayland;xcb" \
python3 "$here/ft_input_settings.py" "$@"
input-settings/ft-input-settings.desktop
+9
View File
@@ -0,0 +1,9 @@
[Desktop Entry]
Type=Application
Name=Frametop Input Settings
GenericName=Mouse and keyboard settings for the 3D pointer
Comment=Choose and map Bluetooth mice and keyboards for the SteamVR 3D pointer
Exec=@REPO@/frametop/input-settings/ft-input-settings
Icon=input-mouse
Categories=Settings;HardwareSettings;
Keywords=mouse;pointer;bluetooth;keyboard;steamvr;frametop;
input-settings/ft_input_settings.py
+421
View File
@@ -0,0 +1,421 @@
#!/usr/bin/env python3
"""Frametop Input Settings: choose and map input devices for the universal 3D mouse.
A Kirigami (QML) app with a Python backend. It runs in the dev container and talks
to the input relay over its control socket (@frametop_relay):
- Devices: every USB/Bluetooth mouse and keyboard, a live activity light to
identify them, and a role for each (3D pointer, pass through, ignore).
- Buttons: press a button or key on a pointer device, then pick an action.
- Pointer: speed, dot size, distance and the rest, applied live.
- Bluetooth: paired devices, and re-applying the Bluetooth LE fixes after pairing.
Rules go to ~/.config/frametop-input.json and pointer settings to
~/.config/frametop.conf; then the relay (and through it the helper) reloads.
Launch with input-settings/ft-input-settings (host wrapper).
"""
import json
import os
import re
import shutil
import socket
import subprocess
import sys
from PySide6.QtCore import Property, QObject, QSocketNotifier, QTimer, QUrl, Signal, Slot
from PySide6.QtGui import QGuiApplication, QIcon
from PySide6.QtQml import QQmlApplicationEngine
from PySide6.QtQuickControls2 import QQuickStyle
RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
CONF_PATH = os.path.expanduser("~/.config/frametop.conf")
RELAY = "\0frametop_relay"
HELPER = "\0ft_pointer_helper"
BTN_MISC = 0x100
# Header names that mark the start of a range, not a real key (BTN_MOUSE == BTN_LEFT).
RANGE_ALIASES = {"BTN_MISC", "BTN_MOUSE", "BTN_JOYSTICK", "BTN_GAMEPAD", "BTN_DIGI", "BTN_WHEEL",
"BTN_TRIGGER_HAPPY", "KEY_MIN_INTERESTING", "KEY_MAX", "KEY_CNT", "BTN_A", "BTN_B", "BTN_X", "BTN_Y"}
DEFAULT_BUTTONS = {0x110: "left", 0x111: "right", 0x112: "middle", 0x113: "back", 0x114: "back"}
ACTION_LABELS = {
"left": "Left click", "right": "Right click", "middle": "Middle click", "back": "Back",
"scroll_up": "Scroll up", "scroll_down": "Scroll down", "dashboard": "Toggle SteamVR dashboard",
"recenter": "Recenter pointer", "pointer_toggle": "Pointer on/off", "sens_up": "Faster pointer",
"sens_down": "Slower pointer", "layout_reset": "Reset desktop screen layout",
"screens_toggle": "Hide/show desktop screens", "key": "Pass through as key",
"none": "Do nothing",
}
ROLE_LABELS = {"pointer": "3D pointer", "passthrough": "Pass through", "ignore": "Ignore"}
# Pointer settings: key, label, default, min, max, step, unit.
POINTER_SETTINGS = [
("POINTER_SENSITIVITY", "Speed", 0.03, 0.005, 0.12, 0.001, "°/count"),
("POINTER_CURSOR_DEG", "Dot size", 0.4, 0.1, 2.0, 0.05, "°"),
("POINTER_DISTANCE", "Distance in open space", 1.5, 0.5, 4.0, 0.1, "m"),
("POINTER_ORIGIN_FRACTION", "SteamVR dot shrink", 0.95, 0.5, 0.98, 0.01, ""),
("POINTER_ORIGIN_MARGIN", "Room for small controls", 0.15, 0.03, 0.5, 0.01, "m"),
("POINTER_SCENE_RADIUS", "Dock / window-control reach", 0.5, 0.1, 1.5, 0.05, "m"),
("POINTER_EDGE_REACH", "Panel edge reach", 0.3, 0.0, 1.0, 0.05, "m"),
("POINTER_WAKE_COUNTS", "Movement to wake", 40, 5, 200, 5, "counts"),
("POINTER_IDLE", "Release after idle", 30, 5, 120, 5, "s"),
]
def code_names():
"""evdev key/button code -> name, from the kernel header (first name wins, BTN_ for buttons)."""
names = {}
try:
with open("/usr/include/linux/input-event-codes.h") as f:
for m in re.finditer(r"#define\s+((?:KEY|BTN)_\w+)\s+(0x[0-9a-fA-F]+|\d+)", f.read()):
code = int(m.group(2), 0)
name = m.group(1)
if name in RANGE_ALIASES:
continue
if code not in names or (code >= BTN_MISC and name.startswith("BTN_") and not names[code].startswith("BTN_")):
names[code] = name
except OSError:
pass
return names
def read_json(path):
try:
with open(path) as f:
return json.load(f)
except (OSError, ValueError):
return {}
def read_conf():
conf = {}
try:
with open(CONF_PATH) as f:
for line in f:
line = line.split("#", 1)[0].strip()
if "=" in line:
k, v = line.split("=", 1)
conf[k.strip()] = v.strip()
except OSError:
pass
return conf
def write_conf_value(key, value):
"""Set KEY=value in frametop.conf, keeping comments and the rest of the file."""
try:
with open(CONF_PATH) as f:
lines = f.read().splitlines()
except OSError:
lines = []
for i, line in enumerate(lines):
if line.split("#", 1)[0].strip().startswith(f"{key}="):
comment = line[line.index("#"):] if "#" in line else ""
lines[i] = f"{key}={value}" + (f" {comment}" if comment else "")
break
else:
lines.append(f"{key}={value}")
with open(CONF_PATH, "w") as f:
f.write("\n".join(lines) + "\n")
def host(*cmd):
"""Run a command on the SteamOS host (we live in the dev container)."""
runner = ["distrobox-host-exec"] if shutil.which("distrobox-host-exec") else []
try:
return subprocess.run(runner + list(cmd), capture_output=True, text=True, timeout=20)
except (OSError, subprocess.TimeoutExpired) as e:
return subprocess.CompletedProcess(cmd, 1, "", str(e))
class Backend(QObject):
devicesChanged = Signal()
mappingsChanged = Signal()
pointerChanged = Signal()
bluetoothChanged = Signal()
activity = Signal(str) # device id
captured = Signal(int, str) # code, name
message = Signal(str, bool) # text, is error
def __init__(self):
super().__init__()
self._names = code_names()
self._nodes = []
self._pointer_mode = False
self._capture_id = ""
self._bluetooth = []
self._relay_ok = False
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
self.sock.bind("") # autobind an abstract address the relay can reply to
self.sock.setblocking(False)
self.notifier = QSocketNotifier(self.sock.fileno(), QSocketNotifier.Read)
self.notifier.activated.connect(self._read)
self.poll = QTimer(interval=2000, timeout=self._refresh)
self.poll.start()
self.rewatch = QTimer(interval=50000, timeout=lambda: self._send("watch 60"))
self.rewatch.start()
self.reload_timer = QTimer(singleShot=True, interval=400, timeout=lambda: self._send("reload"))
self._refresh()
self._send("watch 60")
self.refreshBluetooth()
# --- relay socket ---
def _send(self, text, to=RELAY):
try:
self.sock.sendto(text.encode(), to)
return True
except OSError:
if to == RELAY and self._relay_ok:
self._relay_ok = False
self.devicesChanged.emit()
return False
def _refresh(self):
self._send("devices")
def _read(self):
while True:
try:
data = self.sock.recv(65536)
except BlockingIOError:
return
try:
msg = json.loads(data)
except ValueError:
continue
t = msg.get("t")
if t == "devices":
self._nodes = msg.get("nodes", [])
self._pointer_mode = bool(msg.get("pointer_mode"))
self._relay_ok = True
self.devicesChanged.emit()
elif t == "event":
self.activity.emit(msg["id"])
if (self._capture_id and msg["id"] == self._capture_id and msg["type"] == "key"
and msg["value"] == 1):
code = int(msg["code"])
self._capture_id = ""
self.captured.emit(code, self.codeName(code))
# --- devices ---
@Property(bool, notify=devicesChanged)
def relayRunning(self):
return self._relay_ok
@Property(bool, notify=devicesChanged)
def pointerMode(self):
return self._pointer_mode
@Property("QVariantList", notify=devicesChanged)
def devices(self):
"""Connected devices, plus devices with saved rules or bindings that aren't connected
(a sleeping Bluetooth mouse), so their bindings can still be edited and removed."""
all_rules = read_json(RULES_PATH)
rules = all_rules.get("devices", {})
bindings = all_rules.get("buttons", {})
grouped = {}
for n in self._nodes:
d = grouped.setdefault(n["id"], {"id": n["id"], "name": n["name"], "bus": n["bus"], "kinds": [],
"nodes": [], "role": n["role"], "grabbed": False})
d["nodes"].append(n["path"])
d["kinds"] = sorted(set(d["kinds"]) | set(n["kinds"]))
d["grabbed"] = d["grabbed"] or n["grabbed"]
if "mouse" in n["kinds"]:
d["name"] = n["name"].replace(" Mouse", "") # the Z3's nodes are "Z3 Mouse"/"Z3 Keyboard"
for d in grouped.values():
d["connected"] = True
for device_id in set(rules) | {i for i, b in bindings.items() if b}:
if device_id in grouped:
continue
rule = rules.get(device_id, {})
# Not connected: the relay's default for a device we have bindings for is "pointer".
grouped[device_id] = {"id": device_id, "name": rule.get("name", device_id), "bus": "",
"kinds": [], "nodes": [], "role": rule.get("role", "pointer"),
"grabbed": False, "connected": False}
for d in grouped.values():
d["explicit"] = d["id"] in rules and "role" in rules[d["id"]]
d["roleLabel"] = ROLE_LABELS.get(d["role"], d["role"])
return sorted(grouped.values(), key=lambda d: (not d["connected"], d["role"] != "pointer", d["name"].lower()))
@Property("QVariantList", constant=True)
def roles(self):
return [{"value": k, "text": v} for k, v in ROLE_LABELS.items()]
@Slot(str, str, str)
def setRole(self, device_id, role, name):
rules = read_json(RULES_PATH)
rules.setdefault("devices", {})[device_id] = {"role": role, "name": name}
self._save_rules(rules)
self.message.emit(f"{name}: {ROLE_LABELS.get(role, role)}", False)
@Slot(str)
def resetRole(self, device_id):
rules = read_json(RULES_PATH)
rules.get("devices", {}).get(device_id, {}).pop("role", None)
self._save_rules(rules)
@Slot(str)
def forgetDevice(self, device_id):
"""Drop everything saved for a device: role, name, and bindings."""
rules = read_json(RULES_PATH)
name = rules.get("devices", {}).pop(device_id, {}).get("name", device_id)
rules.get("buttons", {}).pop(device_id, None)
self._save_rules(rules)
self.message.emit(f"Forgot {name}", False)
def _remember_name(self, rules, device_id):
"""Keep the device's name with its rules, so it can be listed while disconnected."""
entry = rules.setdefault("devices", {}).setdefault(device_id, {})
if "name" not in entry:
for d in self.devices:
if d["id"] == device_id:
entry["name"] = d["name"]
def _save_rules(self, rules):
os.makedirs(os.path.dirname(RULES_PATH), exist_ok=True)
with open(RULES_PATH, "w") as f:
json.dump(rules, f, indent=2)
self._send("reload")
QTimer.singleShot(300, self._refresh)
self.mappingsChanged.emit()
self.devicesChanged.emit()
# --- buttons ---
@Slot(int, result=str)
def codeName(self, code):
return self._names.get(code, f"code {code}")
@Property("QVariantList", constant=True)
def actions(self):
return [{"value": k, "text": v} for k, v in ACTION_LABELS.items()]
@Slot(str, result="QVariantList")
def mappings(self, device_id):
custom = read_json(RULES_PATH).get("buttons", {}).get(device_id, {})
rows = {}
for code, action in DEFAULT_BUTTONS.items():
rows[code] = {"code": code, "action": action, "custom": False}
for code, action in custom.items():
rows[int(code)] = {"code": int(code), "action": action, "custom": True}
out = []
for code in sorted(rows):
r = rows[code]
r["isDefault"] = code in DEFAULT_BUTTONS # a built-in binding (left/right/middle/side/extra)
r["name"] = self.codeName(code)
r["actionLabel"] = ACTION_LABELS.get(r["action"], r["action"])
out.append(r)
return out
@Slot(str)
def startCapture(self, device_id):
self._capture_id = device_id
self._send("watch 60")
@Slot()
def cancelCapture(self):
self._capture_id = ""
@Slot(str, int, str)
def setMapping(self, device_id, code, action):
rules = read_json(RULES_PATH)
rules.setdefault("buttons", {}).setdefault(device_id, {})[str(code)] = action
self._remember_name(rules, device_id)
self._save_rules(rules)
self.message.emit(f"{self.codeName(code)} → {ACTION_LABELS.get(action, action)}", False)
@Slot(str, int)
def removeMapping(self, device_id, code):
"""Drop a custom binding: built-in buttons go back to their default, others pass through."""
rules = read_json(RULES_PATH)
rules.get("buttons", {}).get(device_id, {}).pop(str(code), None)
self._save_rules(rules)
default = DEFAULT_BUTTONS.get(code)
self.message.emit(f"{self.codeName(code)}: " + (f"back to {ACTION_LABELS[default]}" if default
else "binding removed"), False)
@Slot(str, int)
def unbind(self, device_id, code):
"""Make a button do nothing (also works for the built-in bindings)."""
self.setMapping(device_id, code, "none")
@Slot(str)
def clearMappings(self, device_id):
rules = read_json(RULES_PATH)
removed = len(rules.get("buttons", {}).pop(device_id, {}) or {})
self._save_rules(rules)
self.message.emit(f"Removed {removed} binding{'s' if removed != 1 else ''}; defaults restored", False)
# --- pointer settings ---
@Property("QVariantList", notify=pointerChanged)
def pointerSettings(self):
conf = read_conf()
out = []
for key, label, default, lo, hi, step, unit in POINTER_SETTINGS:
try:
value = float(conf.get(key, default))
except ValueError:
value = default
out.append({"key": key, "label": label, "value": value, "min": lo, "max": hi, "step": step,
"unit": unit, "default": default})
return out
@Slot(str, float)
def setPointerSetting(self, key, value):
integer = key in ("POINTER_WAKE_COUNTS", "POINTER_IDLE")
write_conf_value(key, str(int(round(value))) if integer else f"{value:.3f}".rstrip("0").rstrip("."))
self.reload_timer.start() # debounce slider drags
self.pointerChanged.emit()
@Slot(result=bool)
def recenter(self):
return self._send("recenter", HELPER)
# --- bluetooth ---
@Property("QVariantList", notify=bluetoothChanged)
def bluetooth(self):
return self._bluetooth
@Slot()
def refreshBluetooth(self):
devices = []
listing = host("bluetoothctl", "devices", "Paired")
for line in listing.stdout.splitlines():
parts = line.split(" ", 2)
if len(parts) < 3 or parts[0] != "Device":
continue
info = host("bluetoothctl", "info", parts[1]).stdout
battery = re.search(r"Battery Percentage: \S+ \((\d+)\)", info)
devices.append({"address": parts[1], "name": parts[2],
"connected": "Connected: yes" in info,
"battery": int(battery.group(1)) if battery else -1})
self._bluetooth = devices
self.bluetoothChanged.emit()
@Slot()
def applyBluetoothFixes(self):
if not os.path.exists("/run/host/etc/steamframe/bt-fixups.sh") and not os.path.exists("/etc/steamframe/bt-fixups.sh"):
self.message.emit("Bluetooth fixes aren't installed (setup/bluetooth/install.sh)", True)
return
result = host("pkexec", "/etc/steamframe/bt-fixups.sh")
if result.returncode == 0:
self.message.emit("Bluetooth fixes applied: " + " ".join(result.stdout.split())[:120], False)
else:
self.message.emit("Couldn't apply the Bluetooth fixes: " + (result.stderr.strip() or "cancelled")[:160], True)
self.refreshBluetooth()
def main():
app = QGuiApplication(sys.argv)
app.setApplicationName("ft-input-settings")
app.setApplicationDisplayName("Frametop Input Settings")
app.setDesktopFileName("ft-input-settings")
if not QIcon.themeName():
QIcon.setThemeName("breeze")
QQuickStyle.setStyle("org.kde.desktop")
engine = QQmlApplicationEngine()
backend = Backend()
engine.rootContext().setContextProperty("backend", backend)
engine.rootContext().setContextProperty("startPage", os.environ.get("FT_INPUT_PAGE", "devices"))
engine.load(QUrl.fromLocalFile(os.path.join(os.path.dirname(os.path.abspath(__file__)), "main.qml")))
if not engine.rootObjects():
sys.exit(1)
sys.exit(app.exec())
if __name__ == "__main__":
main()
input-settings/install.sh
Executable
+15
View File
@@ -0,0 +1,15 @@
#!/usr/bin/env bash
# Install (or remove) the Frametop Input Settings menu entry on the Frame.
# Usage: input-settings/install.sh [install|uninstall]
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
. "$root/scripts/_env.sh"
"$root/scripts/sync.sh" >/dev/null
case ${1:-install} in
install)
fill_template "$root/input-settings/ft-input-settings.desktop" |
on_frame "chmod +x input-settings/ft-input-settings && mkdir -p ~/.local/share/applications && cat > ~/.local/share/applications/ft-input-settings.desktop"
echo "installed: Frametop Input Settings (Plasma menu, Settings)" ;;
uninstall) on_frame 'rm -f ~/.local/share/applications/ft-input-settings.desktop; echo removed' ;;
*) echo "usage: $0 [install|uninstall]" >&2; exit 2 ;;
esac
input-settings/main.qml
+416
View File
@@ -0,0 +1,416 @@
// Frametop Input Settings (Kirigami). Backend: ft_input_settings.py ("backend").
import QtQuick
import QtQuick.Controls as Controls
import QtQuick.Layouts
import org.kde.kirigami as Kirigami
Kirigami.ApplicationWindow {
id: root
title: "Frametop Input Settings"
width: Kirigami.Units.gridUnit * 44
height: Kirigami.Units.gridUnit * 34
globalDrawer: Kirigami.GlobalDrawer {
isMenu: false
modal: false
collapsible: true
collapsed: root.width < Kirigami.Units.gridUnit * 30
actions: [
Kirigami.Action { text: "Devices"; icon.name: "input-mouse"; onTriggered: root.show(devicesPage) },
Kirigami.Action { text: "Buttons"; icon.name: "input-keyboard"; onTriggered: root.show(buttonsPage) },
Kirigami.Action { text: "Pointer"; icon.name: "transform-move"; onTriggered: root.show(pointerPage) },
Kirigami.Action { text: "Bluetooth"; icon.name: "preferences-system-bluetooth"; onTriggered: root.show(bluetoothPage) }
]
}
function show(page) {
pageStack.clear()
pageStack.push(page)
}
// FT_INPUT_PAGE=buttons|pointer|bluetooth opens the app on that page.
pageStack.initialPage: ({ buttons: buttonsPage, pointer: pointerPage, bluetooth: bluetoothPage })[startPage] || devicesPage
Connections {
target: backend
function onMessage(text, isError) {
root.showPassiveNotification(text, isError ? "long" : "short")
}
}
// ---------------------------------------------------------------- Devices
Component {
id: devicesPage
Kirigami.ScrollablePage {
title: "Devices"
header: Kirigami.InlineMessage {
visible: !backend.relayRunning || !backend.pointerMode
position: Kirigami.InlineMessage.Position.Header
type: backend.relayRunning ? Kirigami.MessageType.Information : Kirigami.MessageType.Error
text: !backend.relayRunning
? "The input relay isn't running (frametop-input-relay.service)."
: "Pointer mode is off (POINTER=0): pointer devices act as a plain mouse."
}
ListView {
model: backend.devices
spacing: Kirigami.Units.smallSpacing
Kirigami.PlaceholderMessage {
anchors.centerIn: parent
visible: parent.count === 0
text: "No USB or Bluetooth mice or keyboards connected"
explanation: "Connect or wake one; it appears here within a second."
}
delegate: Controls.ItemDelegate {
id: row
required property var modelData
width: ListView.view.width
hoverEnabled: false
down: false
contentItem: RowLayout {
spacing: Kirigami.Units.largeSpacing
// Activity light: flashes when the device sends input.
Rectangle {
id: light
implicitWidth: Kirigami.Units.gridUnit * 0.8
implicitHeight: implicitWidth
radius: width / 2
color: Kirigami.Theme.disabledTextColor
opacity: 0.35
Connections {
target: backend
function onActivity(id) {
if (id === row.modelData.id) {
light.color = Kirigami.Theme.positiveTextColor
light.opacity = 1
fade.restart()
}
}
}
Timer {
id: fade
interval: 350
onTriggered: { light.color = Kirigami.Theme.disabledTextColor; light.opacity = 0.35 }
}
}
Kirigami.Icon {
source: row.modelData.kinds.indexOf("mouse") >= 0 ? "input-mouse" : "input-keyboard"
implicitWidth: Kirigami.Units.iconSizes.medium
implicitHeight: implicitWidth
}
ColumnLayout {
Layout.fillWidth: true
spacing: 0
Controls.Label {
text: row.modelData.name
font.bold: true
Layout.fillWidth: true
elide: Text.ElideRight
}
Controls.Label {
text: row.modelData.connected
? row.modelData.bus + " · " + row.modelData.kinds.join(" + ") + " · " + row.modelData.id
+ (row.modelData.grabbed ? " · grabbed" : "")
: "not connected · saved settings · " + row.modelData.id
opacity: 0.7
font: Kirigami.Theme.smallFont
Layout.fillWidth: true
elide: Text.ElideRight
}
}
Controls.ComboBox {
model: backend.roles
textRole: "text"
valueRole: "value"
Component.onCompleted: currentIndex = indexOfValue(row.modelData.role)
onActivated: backend.setRole(row.modelData.id, currentValue, row.modelData.name)
}
Controls.Button {
visible: !row.modelData.connected
text: "Forget"
icon.name: "edit-delete-remove"
onClicked: backend.forgetDevice(row.modelData.id)
}
Controls.ToolButton {
icon.name: "edit-undo"
visible: row.modelData.explicit
display: Controls.AbstractButton.IconOnly
text: "Use the default role"
Controls.ToolTip.text: text
Controls.ToolTip.visible: hovered
onClicked: backend.resetRole(row.modelData.id)
}
}
}
}
footer: Controls.Label {
padding: Kirigami.Units.largeSpacing
wrapMode: Text.Wrap
opacity: 0.7
text: "Move or press a device to see which row it is. 3D pointer: grabbed, drives the SteamVR pointer. "
+ "Pass through: left alone (a Meta tap still toggles the dashboard). Ignore: left alone."
}
}
}
// ---------------------------------------------------------------- Buttons
Component {
id: buttonsPage
Kirigami.ScrollablePage {
id: bpage
title: "Buttons"
actions: [
Kirigami.Action {
text: "Remove all"
icon.name: "edit-clear-all"
tooltip: "Remove every binding you added for this device; built-in buttons go back to their defaults"
enabled: bpage.rows.some(r => r.custom)
onTriggered: backend.clearMappings(bpage.deviceId)
}
]
property string deviceId: pointerDevices.length > 0 ? pointerDevices[Math.max(0, deviceBox.currentIndex)].id : ""
property var pointerDevices: backend.devices.filter(d => d.role === "pointer")
property int capturedCode: -1
property string capturedName: ""
property bool capturing: false
property var rows: deviceId ? backend.mappings(deviceId) : []
Connections {
target: backend
function onCaptured(code, name) { bpage.capturedCode = code; bpage.capturedName = name; bpage.capturing = false }
function onMappingsChanged() { bpage.rows = bpage.deviceId ? backend.mappings(bpage.deviceId) : [] }
}
ColumnLayout {
spacing: Kirigami.Units.largeSpacing
Kirigami.PlaceholderMessage {
Layout.fillWidth: true
visible: bpage.pointerDevices.length === 0
text: "No 3D pointer devices"
explanation: "Set a device's role to 3D pointer on the Devices page."
}
Kirigami.FormLayout {
Layout.fillWidth: true
visible: bpage.pointerDevices.length > 0
Controls.ComboBox {
id: deviceBox
Kirigami.FormData.label: "Device:"
model: bpage.pointerDevices.map(d => ({ name: d.name + (d.connected ? "" : " (not connected)"), id: d.id }))
textRole: "name"
onActivated: { bpage.capturedCode = -1; bpage.rows = backend.mappings(bpage.deviceId) }
}
RowLayout {
Kirigami.FormData.label: "New mapping:"
enabled: bpage.pointerDevices.length > 0 && bpage.pointerDevices[Math.max(0, deviceBox.currentIndex)].connected
Controls.Button {
text: bpage.capturing ? "Press a button or key on the device…" : "Capture a button"
icon.name: "input-mouse-click-left"
highlighted: bpage.capturing
onClicked: {
if (bpage.capturing) { backend.cancelCapture(); bpage.capturing = false }
else { bpage.capturedCode = -1; bpage.capturing = true; backend.startCapture(bpage.deviceId) }
}
}
Controls.Label {
visible: bpage.capturedCode >= 0
text: bpage.capturedName
font.bold: true
}
Controls.ComboBox {
id: newAction
visible: bpage.capturedCode >= 0
model: backend.actions
textRole: "text"
valueRole: "value"
}
Controls.Button {
visible: bpage.capturedCode >= 0
text: "Map"
icon.name: "dialog-ok-apply"
onClicked: { backend.setMapping(bpage.deviceId, bpage.capturedCode, newAction.currentValue); bpage.capturedCode = -1 }
}
}
}
Kirigami.Heading {
visible: bpage.rows.length > 0
level: 3
text: "Current mappings"
}
Repeater {
model: bpage.rows
delegate: RowLayout {
required property var modelData
Layout.fillWidth: true
spacing: Kirigami.Units.largeSpacing
Controls.Label {
text: modelData.name
font.family: "monospace"
Layout.preferredWidth: Kirigami.Units.gridUnit * 10
}
Controls.ComboBox {
model: backend.actions
textRole: "text"
valueRole: "value"
Layout.preferredWidth: Kirigami.Units.gridUnit * 13
Component.onCompleted: currentIndex = indexOfValue(modelData.action)
onActivated: backend.setMapping(bpage.deviceId, modelData.code, currentValue)
}
Controls.Label {
text: modelData.custom ? (modelData.isDefault ? "changed" : "custom") : "default"
opacity: 0.6
Layout.preferredWidth: Kirigami.Units.gridUnit * 4
}
// Remove: your own binding goes away (the button passes through again).
// Reset: a changed built-in binding goes back to its default.
// Unbind: an untouched built-in binding is set to "Do nothing".
Controls.Button {
text: modelData.custom ? (modelData.isDefault ? "Reset" : "Remove") : "Unbind"
icon.name: modelData.custom ? (modelData.isDefault ? "edit-undo" : "edit-delete-remove")
: "list-remove"
onClicked: modelData.custom ? backend.removeMapping(bpage.deviceId, modelData.code)
: backend.unbind(bpage.deviceId, modelData.code)
}
Item { Layout.fillWidth: true }
}
}
Controls.Label {
Layout.fillWidth: true
wrapMode: Text.Wrap
opacity: 0.7
text: "Buttons without a mapping pass through (mouse buttons as clicks, keys as keys). "
+ "The Z3's extra buttons show up as keys from its keyboard node."
}
}
}
}
// ---------------------------------------------------------------- Pointer
Component {
id: pointerPage
Kirigami.ScrollablePage {
title: "Pointer"
actions: [
Kirigami.Action {
text: "Recenter"
icon.name: "zoom-fit-best"
onTriggered: backend.recenter()
}
]
Kirigami.FormLayout {
Repeater {
model: backend.pointerSettings
delegate: RowLayout {
required property var modelData
Kirigami.FormData.label: modelData.label + ":"
Controls.Slider {
id: slider
from: modelData.min
to: modelData.max
stepSize: modelData.step
value: modelData.value
Layout.preferredWidth: Kirigami.Units.gridUnit * 14
onMoved: backend.setPointerSetting(modelData.key, value)
}
Controls.Label {
text: (modelData.step < 1 ? slider.value.toFixed(modelData.step < 0.01 ? 3 : 2) : Math.round(slider.value))
+ (modelData.unit ? " " + modelData.unit : "")
Layout.preferredWidth: Kirigami.Units.gridUnit * 5
}
Controls.ToolButton {
icon.name: "edit-undo"
display: Controls.AbstractButton.IconOnly
text: "Default (" + modelData.default + ")"
Controls.ToolTip.text: text
Controls.ToolTip.visible: hovered
onClicked: { slider.value = modelData.default; backend.setPointerSetting(modelData.key, modelData.default) }
}
}
}
}
footer: Controls.Label {
padding: Kirigami.Units.largeSpacing
wrapMode: Text.Wrap
opacity: 0.7
text: "Changes apply live. SteamVR dot shrink: how close to the target the pointer's laser starts; "
+ "higher makes SteamVR's own blue dot smaller (max 0.98)."
}
}
}
// ---------------------------------------------------------------- Bluetooth
Component {
id: bluetoothPage
Kirigami.ScrollablePage {
title: "Bluetooth"
actions: [
Kirigami.Action { text: "Refresh"; icon.name: "view-refresh"; onTriggered: backend.refreshBluetooth() },
Kirigami.Action {
text: "Apply Bluetooth fixes"
icon.name: "tools-wizard"
tooltip: "Privacy off, and address resolution on bonded LE devices (asks for your password)"
onTriggered: backend.applyBluetoothFixes()
}
]
ListView {
model: backend.bluetooth
Kirigami.PlaceholderMessage {
anchors.centerIn: parent
visible: parent.count === 0
text: "No paired Bluetooth devices"
explanation: "Pair in Steam: Settings → Bluetooth."
}
delegate: Controls.ItemDelegate {
required property var modelData
width: ListView.view.width
contentItem: RowLayout {
spacing: Kirigami.Units.largeSpacing
Kirigami.Icon {
source: "preferences-system-bluetooth"
implicitWidth: Kirigami.Units.iconSizes.medium
implicitHeight: implicitWidth
opacity: modelData.connected ? 1 : 0.4
}
ColumnLayout {
Layout.fillWidth: true
spacing: 0
Controls.Label { text: modelData.name; font.bold: true }
Controls.Label {
text: modelData.address + " · " + (modelData.connected ? "connected" : "not connected")
+ (modelData.battery >= 0 ? " · battery " + modelData.battery + "%" : "")
opacity: 0.7
font: Kirigami.Theme.smallFont
}
}
}
}
}
footer: Controls.Label {
padding: Kirigami.Units.largeSpacing
wrapMode: Text.Wrap
opacity: 0.7
text: "Pair new devices in Steam (Settings → Bluetooth). After pairing an LE mouse or keyboard, "
+ "use Apply Bluetooth fixes once so it reconnects on its own."
}
}
}
}
input/frametop-input-relay.service
+21
View File
@@ -0,0 +1,21 @@
# Template: the installer replaces @REPO@ with the repo path on the Frame.
[Unit]
Description=Frametop input relay: stable virtual mouse and keyboard for SteamVR
Documentation=file://@REPO@/frametop/README.md
# SteamVR opens input devices only at startup, so the virtual devices must exist first.
Before=steamvr.service
[Service]
# READY=1 is sent only after the virtual devices exist.
Type=notify
NotifyAccess=main
ExecStart=/usr/bin/python3 @REPO@/frametop/input/input-relay.py
Restart=on-failure
RestartSec=1
# systemd holds the virtual devices' file descriptors across relay restarts,
# so SteamVR keeps the same devices. Clear with: systemctl --user clean --what=fdstore <unit>
FileDescriptorStoreMax=4
FileDescriptorStorePreserve=yes
[Install]
WantedBy=default.target steamvr.service
input/input-relay.py
Executable
+641
View File
@@ -0,0 +1,641 @@
#!/usr/bin/env python3
"""Input relay for the Steam Frame: stable virtual devices, the 3D pointer, device rules.
SteamVR opens /dev/input/event* only when it starts and never hotplugs, so a
Bluetooth mouse that sleeps and reconnects (new event nodes) stops working
until SteamVR restarts. This relay creates a virtual mouse and a virtual
keyboard through /dev/uinput once, before SteamVR starts, and feeds them from
the physical devices as they come and go.
Every USB or Bluetooth mouse and keyboard is a candidate. A physical device is
identified by its Bluetooth address (EVIOCGUNIQ) or USB bus:vendor:product:name,
so all its event nodes share one role (the Swiftpoint Z3 has a mouse node and a
keyboard node for its extra buttons). Roles, from ~/.config/frametop-input.json
(written by the Frametop Input Settings app):
pointer grabbed; drives the universal 3D mouse (default for devices with a mouse node)
passthrough not grabbed, only observed, e.g. for the Meta dashboard shortcut (default for keyboards)
ignore not grabbed, only observed for identification in the settings app
Buttons and keys of pointer devices go through a per-device map to actions
(left, right, middle, back, scroll_up, scroll_down, dashboard, recenter,
pointer_toggle, sens_up, sens_down, layout_reset = put the desktop screens back in
their saved layout, screens_toggle = hide or show the desktop screens, key = pass
through as a key, none).
Keys also go to ft-screens (@ft_screens, the Frametop desktop's compositor), which
types them into the desktop screen that has focus (not while the SteamVR dashboard is
open): from keyboards that aren't grabbed, and keys a pointer device passes through.
Pointer mode (POINTER=1 in ~/.config/frametop.conf) sends pointer devices to
the ft-pointer helper (pointer/helper), which drives the ft_pointer
SteamVR driver. With POINTER=0, pointer devices go to the virtual mouse and
keyboard instead.
Control socket (abstract datagram @frametop_relay, JSON replies to the sender):
devices list event nodes with id, name, kinds, role, grabbed
watch <seconds> stream input events from every candidate node (identification)
reload re-read both config files, re-apply roles, tell the helper
Runs on the Frame host as a user service (frametop-input-relay.service). The
virtual devices are parked in systemd's file descriptor store, so a relay
restart gets the same devices back and SteamVR never loses them. The service is
Type=notify: READY=1 goes out only after the devices exist, so SteamVR (ordered
after it) always finds them. Dependency-free: Python standard library plus the
kernel's evdev and uinput interfaces.
input-relay.py the service
input-relay.py --no-grab never grab, for testing next to a running SteamVR
"""
import array
import errno
import fcntl
import json
import os
import select
import socket
import struct
import subprocess
import sys
import time
# Linux input constants (include/uapi/linux/input-event-codes.h, input.h, uinput.h).
EV_SYN, EV_KEY, EV_REL, EV_MSC = 0x00, 0x01, 0x02, 0x04
SYN_REPORT = 0
BTN_MISC, KEY_MAX = 0x100, 0x2FF
KEY_A = 30
REL_X, REL_Y, REL_WHEEL, REL_MAX = 0x00, 0x01, 0x08, 0x0F
BTN_LEFT, BTN_RIGHT, BTN_MIDDLE, BTN_SIDE, BTN_EXTRA = 0x110, 0x111, 0x112, 0x113, 0x114
KEY_LEFTMETA, KEY_RIGHTMETA = 125, 126
BUS_USB, BUS_BLUETOOTH, BUS_VIRTUAL = 0x03, 0x05, 0x06
# struct input_event on 64-bit: struct timeval (2 x long), u16 type, u16 code, s32 value.
EVENT = struct.Struct("llHHi")
def _ioc(direction, nr, size, kind):
return (direction << 30) | (size << 16) | (ord(kind) << 8) | nr
def _iow(kind, nr, size):
return _ioc(1, nr, size, kind)
def _ior(kind, nr, size):
return _ioc(2, nr, size, kind)
UI_DEV_CREATE = _ioc(0, 1, 0, "U")
UI_DEV_DESTROY = _ioc(0, 2, 0, "U")
UI_DEV_SETUP = _iow("U", 3, 92) # struct uinput_setup: input_id (4 x u16), name[80], u32
UI_SET_EVBIT = _iow("U", 100, 4)
UI_SET_KEYBIT = _iow("U", 101, 4)
UI_SET_RELBIT = _iow("U", 102, 4)
EVIOCGRAB = _iow("E", 0x90, 4)
EVIOCGID = _ior("E", 0x02, 8)
EV_NAMES = {EV_KEY: "key", EV_REL: "rel"}
def eviocgbit(ev, length):
return _ior("E", 0x20 + ev, length)
def eviocgname(length):
return _ior("E", 0x06, length)
def eviocguniq(length):
return _ior("E", 0x08, length)
VIRTUAL_PREFIX = "frametop virtual"
RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
ACTIONS = ("left", "right", "middle", "back", "scroll_up", "scroll_down", "dashboard", "recenter",
"pointer_toggle", "sens_up", "sens_down", "layout_reset", "screens_toggle", "key", "none")
SCREENS = "\0ft_screens"
FT_LAYOUT = os.path.join(os.path.dirname(os.path.abspath(__file__)), "..", "layout", "ft-layout")
DEFAULT_BUTTONS = {BTN_LEFT: "left", BTN_RIGHT: "right", BTN_MIDDLE: "middle",
BTN_SIDE: "back", BTN_EXTRA: "back"}
def log(*args):
print(*args, flush=True)
def notify(state, fds=()):
"""sd_notify, with optional file descriptors for the fd store. No-op outside systemd."""
addr = os.environ.get("NOTIFY_SOCKET")
if not addr:
return
if addr.startswith("@"):
addr = "\0" + addr[1:]
# socket.send_fds() ignores its address argument (Python 3.12), so use sendmsg.
ancillary = [(socket.SOL_SOCKET, socket.SCM_RIGHTS, array.array("i", fds))] if fds else []
try:
with socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) as sock:
sock.sendmsg([state.encode()], ancillary, 0, addr)
except OSError as e:
log(f"sd_notify failed ({state.splitlines()[0]}): {e}")
def stored_fds():
"""File descriptors handed back by systemd's fd store, by name."""
if os.environ.get("LISTEN_PID") != str(os.getpid()):
return {}
names = os.environ.get("LISTEN_FDNAMES", "").split(":")
count = int(os.environ.get("LISTEN_FDS", "0"))
return {names[i]: 3 + i for i in range(count) if i < len(names)}
class Virtual:
"""One uinput device, reused from systemd's fd store when possible."""
def __init__(self, name, product, keys, rels, stored):
self.dirty = False
store_name = f"vdev{product}"
if store_name in stored:
self.fd = stored[store_name]
os.set_blocking(self.fd, False)
log(f"reusing {name} from the fd store")
return
self.fd = os.open("/dev/uinput", os.O_WRONLY | os.O_NONBLOCK)
fcntl.ioctl(self.fd, UI_SET_EVBIT, EV_KEY)
for code in keys:
fcntl.ioctl(self.fd, UI_SET_KEYBIT, code)
if rels:
fcntl.ioctl(self.fd, UI_SET_EVBIT, EV_REL)
for code in rels:
fcntl.ioctl(self.fd, UI_SET_RELBIT, code)
setup = struct.pack("HHHH80sI", BUS_VIRTUAL, 0x4D44, product, 1, name.encode(), 0)
fcntl.ioctl(self.fd, UI_DEV_SETUP, setup)
fcntl.ioctl(self.fd, UI_DEV_CREATE)
notify(f"FDSTORE=1\nFDNAME={store_name}", [self.fd])
log(f"created {name}")
def emit(self, etype, code, value):
os.write(self.fd, EVENT.pack(0, 0, etype, code, value))
self.dirty = True
def sync(self):
if self.dirty:
os.write(self.fd, EVENT.pack(0, 0, EV_SYN, SYN_REPORT, 0))
self.dirty = False
def bits(fd, ev, count):
buf = bytearray((count + 7) // 8)
try:
fcntl.ioctl(fd, eviocgbit(ev, len(buf)), buf)
except OSError:
return set()
return {i for i in range(count) if buf[i // 8] >> (i % 8) & 1}
def read_config(path=os.path.expanduser("~/.config/frametop.conf")):
"""KEY=VALUE lines, # comments allowed. Missing file means defaults."""
conf = {}
try:
with open(path) as f:
for line in f:
line = line.split("#", 1)[0].strip()
if "=" in line:
key, value = line.split("=", 1)
conf[key.strip()] = value.strip()
except OSError:
pass
return conf
def read_rules(path=RULES_PATH):
"""{"devices": {id: {"role", "name"}}, "buttons": {id: {"<code>": action}}}."""
try:
with open(path) as f:
rules = json.load(f)
except (OSError, ValueError):
rules = {}
rules.setdefault("devices", {})
rules.setdefault("buttons", {})
return rules
class Pointer:
"""Drives the ft_pointer SteamVR driver from a mouse (pointer mode).
The virtual controller connects when the mouse is used (taking the right
hand role and recentering on the gaze) and disconnects after `idle` seconds
without mouse activity, so the real controllers get their role back: the
last used device wins.
"""
DRIVER_BUTTONS = {"left": "trigger", "right": "b", "middle": "x", "back": "joystick"}
SCROLL_PULSE = 0.08 # seconds of joystick deflection per wheel notch
CLAIM_PULSE = 0.06 # seconds the claim button (switchlaserhand, no click) is held
RESUME_PAUSE = 1.5 # mouse idle this long, then moving again, re-claims the laser
WAKE_WINDOW = 1.0 # seconds in which WAKE_COUNTS of motion must add up
def __init__(self, sensitivity, idle, wake_counts=40):
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
self.sensitivity = sensitivity # degrees per mouse count
self.idle = idle
self.active = False
self.last_used = 0.0
self.dx = self.dy = 0
self.scroll_until = None
self.claim_at = None # when to press the claim button
self.claim_release = None
self.system_at = None # dashboard toggle: when to press the virtual system button
self.system_release = None
# Waking (or re-claiming after a pause) needs deliberate movement, so sensor
# jitter from a mouse lying on a desk can't steal the laser from a controller.
self.wake_counts = wake_counts
self.pending = 0
self.pending_since = 0.0
def send(self, command):
try:
self.sock.sendto(command.encode(), "\0ft_pointer_helper")
except OSError:
pass # helper not running (SteamVR not running)
def wake(self, now):
if not self.active:
self.send("show")
self.send("recenter")
self.active = True
self.claim_at = now + 0.3 # let SteamVR bind the freshly connected device first
log("pointer on")
elif now - self.last_used > self.RESUME_PAUSE and self.claim_at is None:
self.claim_at = now # another device may have taken the laser meanwhile
self.last_used = now
def motion(self, code, value, now):
dormant = not self.active or now - self.last_used > self.RESUME_PAUSE
if dormant and code in (REL_X, REL_Y):
if now - self.pending_since > self.WAKE_WINDOW:
self.pending, self.pending_since = 0, now
self.pending += abs(value)
if self.pending < self.wake_counts:
return # not yet deliberate movement
self.pending = 0
self.wake(now)
if code == REL_X:
self.dx += value
elif code == REL_Y:
self.dy += value
elif code == REL_WHEEL and value:
self.send(f"scroll 0 {1 if value > 0 else -1}")
self.scroll_until = now + self.SCROLL_PULSE
def action(self, name, value, now):
"""A mapped button: value 1 press, 0 release, 2 autorepeat (ignored)."""
if value == 2:
return
driver = self.DRIVER_BUTTONS.get(name)
if driver:
self.wake(now)
self.flush()
self.send(f"btn {driver} {value}")
elif value != 1:
return # the rest act on press
elif name in ("scroll_up", "scroll_down"):
self.wake(now)
self.send(f"scroll 0 {1 if name == 'scroll_up' else -1}")
self.scroll_until = now + self.SCROLL_PULSE
elif name == "dashboard":
self.dashboard(now)
elif name == "recenter":
self.wake(now)
self.send("recenter")
elif name == "pointer_toggle":
if self.active:
self.send("hide")
self.active = False
log("pointer off (toggle)")
else:
self.wake(now)
elif name == "screens_toggle":
try:
self.sock.sendto(b"toggle", SCREENS)
except OSError:
pass # ft-screens not running
elif name == "layout_reset":
# Runs a few seconds and borrows the pointer; ft-layout refuses a second copy.
subprocess.Popen([FT_LAYOUT, "apply"], stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, start_new_session=True)
log("layout reset")
elif name in ("sens_up", "sens_down"):
self.sensitivity *= 1.25 if name == "sens_up" else 0.8
log(f"sensitivity {self.sensitivity:.4f} deg/count")
def flush(self):
if self.dx or self.dy:
# Mouse right turns the ray right (negative yaw); mouse down tilts it down.
self.send(f"move {-self.dx * self.sensitivity:.4f} {-self.dy * self.sensitivity:.4f}")
self.dx = self.dy = 0
def dashboard(self, now=None):
"""Toggle the SteamVR dashboard with the virtual controller's system button.
SteamVR needs the button held for a frame or two (press and release in the
same instant is ignored), and the virtual controller must be connected and
bound, so wake it first when needed.
"""
now = time.monotonic() if now is None else now
woke = not self.active
self.wake(now)
self.system_at = now + (0.4 if woke else 0.0)
def tick(self, now):
if self.system_at is not None and now >= self.system_at:
self.send("btn system 1")
self.system_at = None
self.system_release = now + 0.12
elif self.system_release is not None and now >= self.system_release:
self.send("btn system 0")
self.system_release = None
if self.claim_at is not None and now >= self.claim_at:
self.send("btn a 1")
self.claim_at = None
self.claim_release = now + self.CLAIM_PULSE
elif self.claim_release is not None and now >= self.claim_release:
self.send("btn a 0")
self.claim_release = None
if self.scroll_until is not None and now >= self.scroll_until:
self.send("scroll 0 0")
self.scroll_until = None
if self.active and now - self.last_used > self.idle:
self.send("hide")
self.active = False
log("pointer off (idle)")
def timeout(self):
pending = (self.scroll_until, self.claim_at, self.claim_release, self.system_at, self.system_release)
return 0.02 if any(t is not None for t in pending) else 0.5
class Node:
"""One input event node of a candidate device (mouse or keyboard, USB or Bluetooth)."""
def __init__(self, path, fd, name, bus, vendor, product, uniq, is_mouse, is_keyboard):
self.path, self.fd, self.name = path, fd, name
self.bus, self.vendor, self.product, self.uniq = bus, vendor, product, uniq
self.is_mouse, self.is_keyboard = is_mouse, is_keyboard
# One physical device, whatever its node: Bluetooth address, else USB ids plus name.
base = self.name.split(" Mouse")[0].split(" Keyboard")[0]
self.id = uniq.lower() if uniq else f"usb:{vendor:04x}:{product:04x}:{base}"
self.role = None
self.grabbed = False
self.held = set() # keys and buttons currently down, released if the device vanishes
self.last_watch = 0.0
def describe(self):
kinds = [k for k, on in (("mouse", self.is_mouse), ("keyboard", self.is_keyboard)) if on]
return {"path": self.path, "name": self.name, "id": self.id, "uniq": self.uniq,
"bus": {BUS_USB: "usb", BUS_BLUETOOTH: "bluetooth"}.get(self.bus, str(self.bus)),
"kinds": kinds, "role": self.role, "grabbed": self.grabbed}
def probe(path):
"""Open a node if it is a USB or Bluetooth mouse or keyboard, else return None."""
try:
fd = os.open(path, os.O_RDONLY | os.O_NONBLOCK)
except OSError:
return None
try:
buf = bytearray(256)
fcntl.ioctl(fd, eviocgname(len(buf)), buf)
name = buf.split(b"\0", 1)[0].decode(errors="replace")
ident = bytearray(8)
fcntl.ioctl(fd, EVIOCGID, ident)
bus, vendor, product, _ = struct.unpack("HHHH", ident)
if name.startswith(VIRTUAL_PREFIX) or bus not in (BUS_USB, BUS_BLUETOOTH):
raise ValueError
uniq_buf = bytearray(64)
try:
fcntl.ioctl(fd, eviocguniq(len(uniq_buf)), uniq_buf)
uniq = uniq_buf.split(b"\0", 1)[0].decode(errors="replace")
except OSError:
uniq = ""
is_mouse = REL_X in bits(fd, EV_REL, REL_MAX + 1)
is_keyboard = KEY_A in bits(fd, EV_KEY, KEY_MAX + 1)
if not (is_mouse or is_keyboard):
raise ValueError
return Node(path, fd, name, bus, vendor, product, uniq, is_mouse, is_keyboard)
except (OSError, ValueError):
os.close(fd)
return None
def main():
can_grab = "--no-grab" not in sys.argv
stored = stored_fds()
mouse = Virtual(f"{VIRTUAL_PREFIX} mouse", 1,
keys=range(BTN_MISC, 0x118), rels=range(REL_MAX + 1), stored=stored)
keyboard = Virtual(f"{VIRTUAL_PREFIX} keyboard", 2,
keys=range(1, BTN_MISC), rels=(), stored=stored)
notify("READY=1")
control = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
control.bind("\0frametop_relay")
control.setblocking(False)
watchers = {} # address -> watch end time
state = {"pointer": None, "rules": {}}
def load_config():
conf = read_config()
state["rules"] = read_rules()
if conf.get("POINTER", "0") == "1":
p = state["pointer"] or Pointer(0.03, 30)
p.sensitivity = float(conf.get("POINTER_SENSITIVITY", "0.03"))
p.idle = float(conf.get("POINTER_IDLE", "30"))
p.wake_counts = int(conf.get("POINTER_WAKE_COUNTS", "40"))
state["pointer"] = p
log(f"pointer mode: {p.sensitivity} deg/count, idle {p.idle} s, wake {p.wake_counts} counts")
else:
state["pointer"] = None
log("pointer mode off: pointer devices feed the virtual mouse and keyboard")
load_config()
meta_down = False # Meta pressed with no other key yet: a tap toggles the dashboard
screens_sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_NONBLOCK)
def to_screens(code, value):
"""A key for the desktop screens (ft-screens decides whether it types)."""
if value in (0, 1) and code < BTN_MISC:
try:
screens_sock.sendto(f"key {code} {value}".encode(), SCREENS)
except OSError:
pass # ft-screens not running
nodes = {} # fd -> Node
seen = set() # paths already probed (rejected or open)
next_scan = 0.0
def role_of(node):
rule = state["rules"]["devices"].get(node.id, {})
if rule.get("role") in ("pointer", "passthrough", "ignore"):
return rule["role"]
has_mouse = any(n.is_mouse for n in nodes.values() if n.id == node.id) or node.is_mouse
return "pointer" if has_mouse else "passthrough"
def release_held(node):
for code in node.held:
(mouse if code >= BTN_MISC else keyboard).emit(EV_KEY, code, 0)
node.held.clear()
mouse.sync()
keyboard.sync()
def apply_roles():
for node in nodes.values():
role = role_of(node)
want_grab = can_grab and role == "pointer"
if want_grab != node.grabbed:
try:
fcntl.ioctl(node.fd, EVIOCGRAB, 1 if want_grab else 0)
node.grabbed = want_grab
except OSError as e:
log(f"{'grab' if want_grab else 'ungrab'} failed for {node.name}: {e}")
if not node.grabbed:
release_held(node)
if role != node.role:
log(f"{node.name} ({node.path}, {node.id}): {role}{', grabbed' if node.grabbed else ''}")
node.role = role
def drop(node, reason):
release_held(node)
os.close(node.fd)
del nodes[node.fd]
seen.discard(node.path)
log(f"released {node.name} ({node.path}): {reason}")
def reply(addr, obj):
try:
control.sendto(json.dumps(obj).encode(), addr)
except OSError:
watchers.pop(addr, None)
def handle_control(now):
while True:
try:
data, addr = control.recvfrom(4096)
except BlockingIOError:
return
if not addr:
continue # unbound sender, nowhere to reply
words = data.decode(errors="replace").split()
cmd = words[0] if words else ""
if cmd == "devices":
reply(addr, {"t": "devices", "pointer_mode": state["pointer"] is not None,
"actions": ACTIONS, "nodes": [n.describe() for n in nodes.values()]})
elif cmd == "watch":
seconds = float(words[1]) if len(words) > 1 else 30
watchers[addr] = now + min(seconds, 600)
reply(addr, {"t": "watching", "seconds": seconds})
elif cmd == "reload":
load_config()
apply_roles()
if state["pointer"]:
state["pointer"].send("reload")
reply(addr, {"t": "reloaded"})
else:
reply(addr, {"t": "error", "error": f"unknown command {cmd!r}"})
def broadcast(node, etype, code, value, now):
if not watchers:
return
if etype == EV_REL and now - node.last_watch < 0.05:
return # motion: enough for an activity light
node.last_watch = now
msg = {"t": "event", "id": node.id, "path": node.path, "name": node.name,
"type": EV_NAMES.get(etype, str(etype)), "code": code, "value": value}
for addr, until in list(watchers.items()):
if now > until:
del watchers[addr]
else:
reply(addr, msg)
while True:
now = time.monotonic()
pointer = state["pointer"]
if now >= next_scan:
next_scan = now + 1.0
paths = {f"/dev/input/{n}" for n in os.listdir("/dev/input") if n.startswith("event")}
seen &= paths | {n.path for n in nodes.values()}
added = False
for path in sorted(paths - seen):
seen.add(path)
node = probe(path)
if node:
nodes[node.fd] = node
added = True
if added:
apply_roles()
ready, _, _ = select.select(list(nodes) + [control], [], [],
pointer.timeout() if pointer else 0.5)
now = time.monotonic()
if pointer:
pointer.tick(now)
for fd in ready:
if fd is control:
handle_control(now)
continue
node = nodes[fd]
try:
data = os.read(fd, EVENT.size * 64)
except OSError as e:
if e.errno == errno.EAGAIN:
continue
drop(node, os.strerror(e.errno))
continue
if not data:
drop(node, "closed")
continue
buttons = state["rules"]["buttons"].get(node.id, {})
for off in range(0, len(data) - EVENT.size + 1, EVENT.size):
_, _, etype, code, value = EVENT.unpack_from(data, off)
if etype in (EV_KEY, EV_REL):
broadcast(node, etype, code, value, now)
if node.role != "pointer":
# Observed only. A Meta tap on any keyboard toggles the dashboard.
if node.role == "passthrough" and etype == EV_KEY:
to_screens(code, value)
if pointer and node.role == "passthrough" and etype == EV_KEY:
if code in (KEY_LEFTMETA, KEY_RIGHTMETA):
if value == 1:
meta_down = True
elif value == 0 and meta_down:
meta_down = False
pointer.dashboard()
elif value == 1:
meta_down = False # Meta used as a modifier, not a tap
continue
if etype == EV_KEY:
action = buttons.get(str(code), DEFAULT_BUTTONS.get(code, "key"))
if pointer and action not in ("key", "none"):
pointer.action(action, value, now)
continue
if action == "none":
continue
target = mouse if code >= BTN_MISC else keyboard
target.emit(etype, code, value)
to_screens(code, value)
if value:
node.held.add(code)
else:
node.held.discard(code)
elif etype == EV_REL:
if pointer:
pointer.motion(code, value, now)
else:
mouse.emit(etype, code, value)
elif etype == EV_SYN and code == SYN_REPORT:
if pointer:
pointer.flush()
mouse.sync()
keyboard.sync()
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
pass
install.sh
Executable
+90
View File
@@ -0,0 +1,90 @@
#!/usr/bin/env bash
# Install everything on the Steam Frame: the build container, Frametop (multi-screen
# desktop, input relay, universal 3D mouse, settings app), and optionally the Bluetooth
# fixes. Run it on the headset in a terminal, from this repo. It's safe to re-run, for
# example after `git pull`.
# (It also works from a PC over SSH; see "Developing from a PC" in the README.)
#
# Usage: ./install.sh [--yes] [--no-bluetooth]
# --yes don't ask; skips the Bluetooth fixes and the SteamVR restart
# --no-bluetooth don't offer the Bluetooth fixes
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
. "$root/scripts/_env.sh"
assume_yes=0 bluetooth=1
for arg in "$@"; do
case $arg in
--yes) assume_yes=1 ;;
--no-bluetooth) bluetooth=0 ;;
-h|--help) sed -n '2,11p' "$0"; exit 0 ;;
*) echo "unknown option: $arg" >&2; exit 2 ;;
esac
done
step() { printf '\n\033[1m== %s\033[0m\n' "$*"; }
ask() { # ask "question" default(y|n)
[ "$assume_yes" = 1 ] && [ "$2" = y ] && return 0
[ "$assume_yes" = 1 ] && return 1
local hint answer
hint=$([ "$2" = y ] && echo "Y/n" || echo "y/N")
read -r -p "$1 [$hint] " answer
answer=${answer:-$2}
[[ $answer =~ ^[Yy] ]]
}
if [ "$FRAME_LOCAL" = 1 ]; then
echo "Installing on this Steam Frame from $FRAME_REPO"
else
echo "Installing on $FRAME_HOST over SSH (repo copy at $FRAME_REPO)"
"$root/scripts/sync.sh" >/dev/null
fi
step "1/7 distrobox (container tool, installed in your home folder)"
if on_frame 'test -x ~/.local/bin/distrobox'; then
echo "already installed: $(on_frame '~/.local/bin/distrobox version | head -1')"
else
on_frame 'set -e; mkdir -p ~/dev/src
[ -d ~/dev/src/distrobox ] || git clone --depth 1 https://github.com/89luca89/distrobox.git ~/dev/src/distrobox
cd ~/dev/src/distrobox && ./install --prefix ~/.local'
fi
step "2/7 build container (Fedora 44 'dev', about 1-2 GB the first time)"
"$root/setup/dev-container.sh"
step "3/7 input relay (keeps Bluetooth mice working in SteamVR, device roles, button maps)"
"$root/desktops.sh" relay install
step "4/7 3D mouse: SteamVR driver"
"$root/pointer/driver/build.sh"
"$root/pointer/driver/install.sh" install 2>&1 | grep -v xdg-open
step "5/7 3D mouse: pointer helper service"
"$root/pointer/helper/build.sh"
"$root/pointer/helper/run.sh" install
step "6/7 multi-screen desktop (ft-screens), Frametop Input Settings, and Frametop Display Settings"
"$root/screens/build.sh"
"$root/desktops.sh" install >/dev/null
"$root/input-settings/install.sh"
"$root/display-settings/install.sh"
on_frame "sed -i 's/^POINTER=0/POINTER=1/' ~/.config/frametop.conf; grep -q '^POINTER=' ~/.config/frametop.conf || echo 'POINTER=1' >> ~/.config/frametop.conf"
echo "the launcher's Desktop entry now opens the multi-screen desktop; 3D mouse on (POINTER=1 in ~/.config/frametop.conf)"
step "7/7 Bluetooth fixes (optional; they let LE mice and keyboards like the Swiftpoint Z3 reconnect)"
if [ "$bluetooth" = 1 ] && ask "Install the Bluetooth fixes? They need your password (sudo)." n; then
"$root/setup/bluetooth/install.sh" install
else
echo "skipped. Install later with: setup/bluetooth/install.sh install"
fi
step "Done"
cat <<'EOF'
SteamVR has to restart once, to load the 3D mouse driver and to start the input relay
before it. Restarting SteamVR closes everything open in VR, including this terminal if
it's in a VR desktop. Rebooting the headset works too.
EOF
if ask "Restart SteamVR now?" n; then
on_frame 'systemctl --user restart steamvr.service'
fi
layout/ft-layout
Executable
+5
View File
@@ -0,0 +1,5 @@
#!/bin/bash
# ft-layout on the Frame host (see ft_layout.py). Works from a terminal or a menu entry
# inside the Frametop desktop too: its session has a private XDG_RUNTIME_DIR.
export XDG_RUNTIME_DIR=/run/user/$(id -u)
exec python3 "$(dirname "$(readlink -f "$0")")/ft_layout.py" "$@"
layout/ft_layout.py
Executable
+715
View File
@@ -0,0 +1,715 @@
#!/usr/bin/env python3
"""ft-layout: the Frametop screens' sizes and where they float around you.
Two desktop backends (BACKEND in ~/.config/frametop.conf):
screens (default) ft-screens, our own compositor (screens): each screen
is its own SteamVR panel with its own resolution and size in metres. ft-layout
places them directly through ft-screens' control socket (@ft_screens).
gamescope the old path: every screen one size (at most 1920x1080 worth of pixels),
panels owned by the SteamVR dashboard, so ft-layout floats each one with
`vrcmd --dock-overlay` and the pointer helper carries it into place.
The layout is relative to your head when it's applied: its position and the direction
you face (yaw only), like a recenter. It lives in ~/.config/frametop-layout.json:
{"auto": true, arrange when the desktop starts
"mode": "preset" | "custom",
"preset": {"kind": "arc" | "flat", "rows": 1, "distance": 2.0, "gap": 0.05, "height": 0},
"primary": 2, the screen with the taskbar (1-based; default: the biggest)
"visibility": {"mode": "always", ft-screens: always | dashboard (only with the SteamVR
"wrist_angle": 60, dashboard open) | gesture (while you look at a controller)
"gesture_hand": "left", "gesture_angle": 20}, | toggle (hidden until shown);
wrist_angle: a pinned screen shows while you see its front
within this many degrees
"screens": [{"size": [w, h], "metres": 3.6, ft-screens: pixels, and width in VR
"curve": 0, ft-screens: cylinder radius in metres, 0 = flat
"pin": {"hand": "left", "rel": [12]}, ft-screens: riding on that controller
"scale": 1.0, KWin output scale (1.0 = 100%)
"pos": [x, y, z], "face": [yaw, pitch], "roll": 0, custom layout
"rotation": "normal" | "left" | "right"}, ...], gamescope only
"panel_size": [w, h]} gamescope: last measured panel size
Custom positions: x right, y up, -z forward from the head, in metres; face = the
direction you look to see the screen's front straight on, in degrees, relative to your
heading; roll = the panel turned about its front, counterclockwise as you see it.
Presets: "arc" hinges the screens edge to edge around you, each turned to face you
(like monitors on a desk); "flat" puts them on one flat wall facing forward. Screen 1
is top left, then left to right.
Usage (on the Frame host; Frametop Display Settings calls it too):
ft-layout apply [--wait SECONDS] arrange every screen; --wait is for desktop start:
wait for the screens, skip if "auto" is off
ft-layout capture save the current arrangement as the custom layout
ft-layout plan print the arrangement as JSON (no VR needed)
ft-layout scale per-screen scale, positions, and primary to KWin
ft-layout screen-args ft-screens' --screen arguments for the session script
ft-layout toggle hide or show all screens (ft-screens)
ft-layout pin all|N left|right pin screens to a wrist as they are; unpin all|N
"""
import fcntl
import json
import math
import os
import re
import socket
import subprocess
import sys
import time
LAYOUT_PATH = os.path.expanduser("~/.config/frametop-layout.json")
CONF_PATH = os.path.expanduser("~/.config/frametop.conf")
VRCMD = "/opt/steamvr/bin/linuxarm64/vrcmd"
HELPER = "\0ft_pointer_helper"
SCREENS = "\0ft_screens"
LOCK_PATH = "/tmp/ft-layout.lock"
DEFAULT_PANEL = (1.18, 0.664) # gamescope: a floating 16:9 dashboard panel, measured on the Frame
PIXELS_PER_METRE = 800 # ft-screens: a new screen's default size in VR (1920 px: 2.4 m)
VISIBILITY = {"mode": "always", "wrist_angle": 60, "gesture_hand": "left", "gesture_angle": 20}
DEFAULTS = {"auto": True, "mode": "preset",
"preset": {"kind": "arc", "rows": 1, "distance": 2.0, "gap": 0.05, "height": 0.0},
"screens": [], "panel_size": list(DEFAULT_PANEL)}
def log(*args):
print(*args, flush=True)
# ---------------------------------------------------------------- config
def read_conf():
conf = {}
try:
with open(CONF_PATH) as f:
for line in f:
line = line.split("#", 1)[0].strip()
if "=" in line:
k, v = line.split("=", 1)
conf[k.strip()] = v.strip()
except OSError:
pass
return conf
def backend():
return "gamescope" if read_conf().get("BACKEND", "screens") == "gamescope" else "screens"
def screen_count(layout=None):
"""ft-screens: the configured screens; gamescope: SCREENS."""
if backend() == "screens":
return max(1, len((layout or load_layout()).get("screens", [])))
try:
return max(1, int(read_conf().get("SCREENS", "2")))
except ValueError:
return 2
def load_layout():
layout = json.loads(json.dumps(DEFAULTS))
try:
with open(LAYOUT_PATH) as f:
saved = json.load(f)
layout.update({k: v for k, v in saved.items() if k != "preset"})
layout["preset"].update(saved.get("preset", {}))
except (OSError, ValueError):
pass
if backend() == "screens" and not layout.get("screens"):
layout["screens"] = [{"size": [1920, 1080], "metres": 1920 / PIXELS_PER_METRE}]
return layout
def save_layout(layout):
os.makedirs(os.path.dirname(LAYOUT_PATH), exist_ok=True)
tmp = LAYOUT_PATH + ".tmp"
with open(tmp, "w") as f:
json.dump(layout, f, indent=2)
os.replace(tmp, LAYOUT_PATH)
def screen_entry(layout, i):
screens = layout.get("screens", [])
return screens[i] if i < len(screens) else {}
def screen_scale(layout, i):
return float(screen_entry(layout, i).get("scale", 1.0))
def screen_pixels(layout, i):
w, h = screen_entry(layout, i).get("size", [1920, 1080])
return int(w), int(h)
def screen_metres(layout, i):
w, _ = screen_pixels(layout, i)
return float(screen_entry(layout, i).get("metres", w / PIXELS_PER_METRE))
ROLL = {"normal": 0.0, "left": 90.0, "right": -90.0}
def screen_rotation(layout, i):
if backend() == "screens":
return "normal" # portrait screens are simply tall
r = screen_entry(layout, i).get("rotation", "normal")
return r if r in ROLL else "normal"
def screen_size(layout, i, panel_size=None):
"""A screen's size in VR (width, height) in metres."""
if backend() == "screens":
w, h = screen_pixels(layout, i)
m = screen_metres(layout, i)
return m, m * h / w
w, h = panel_size or layout.get("panel_size") or DEFAULT_PANEL
return (h, w) if screen_rotation(layout, i) != "normal" else (w, h)
def primary_screen(layout):
"""0-based index of the screen with the taskbar: the chosen one, or the biggest."""
n = screen_count(layout)
p = layout.get("primary")
if isinstance(p, int) and 1 <= p <= n:
return p - 1
return max(range(n), key=lambda i: screen_pixels(layout, i)[0] * screen_pixels(layout, i)[1])
# ---------------------------------------------------------------- geometry
# Head frame: x right, y up, -z forward, at the eye, turned to the heading (yaw).
# yaw 0 = -Z, positive yaw turns left, positive pitch looks up (as in SteamVR's helper).
def direction(yaw, pitch):
y, p = math.radians(yaw), math.radians(pitch)
return (-math.sin(y) * math.cos(p), math.sin(p), -math.cos(y) * math.cos(p))
def yaw_pitch(v):
n = math.sqrt(sum(c * c for c in v)) or 1.0
return math.degrees(math.atan2(-v[0], -v[2])), math.degrees(math.asin(max(-1.0, min(1.0, v[1] / n))))
def dot(a, b):
return sum(x * y for x, y in zip(a, b))
def cross(a, b):
return (a[1] * b[2] - a[2] * b[1], a[2] * b[0] - a[0] * b[2], a[0] * b[1] - a[1] * b[0])
def normalize(v):
n = math.sqrt(dot(v, v)) or 1.0
return tuple(c / n for c in v)
def turn_yaw(v, yaw):
"""Rotate v about +Y by yaw degrees (head frame -> world for the heading yaw)."""
s, c = math.sin(math.radians(yaw)), math.cos(math.radians(yaw))
return (v[0] * c + v[2] * s, v[1], -v[0] * s + v[2] * c)
def _chain(widths, d, gap):
"""One row of flat screens hinged edge to edge around the eye, each turned to face it,
like monitors on a desk: the middle screen (or the seam between the middle two) is
straight ahead at distance d; each neighbour starts at the previous screen's outer
edge (plus the gap) and is turned until it faces the eye. Returns [(x, z, yaw)], in
the head frame from above (x right, -z forward)."""
n = len(widths)
out = [None] * n
def right(yaw): # a screen's right vector for its yaw
return math.cos(math.radians(yaw)), -math.sin(math.radians(yaw))
def yaw_of(x, z):
return math.degrees(math.atan2(-x, -z))
def hang(hinge, w, side):
# Turn the screen about its hinge until it faces the eye: its centre
# C = hinge + side * w/2 * right(yaw) must have no sideways part, dot(C, right) = 0,
# that is dot(hinge, right(yaw)) = -side * w/2. Take the root nearest the hinge's
# own direction, turning outward (right: yaw falls; left: yaw rises).
def g(yaw):
rx, rz = right(yaw)
return hinge[0] * rx + hinge[1] * rz + side * w / 2
start = yaw_of(*hinge)
a, ga = start, g(start)
for k in range(1, 721): # 0.25-degree steps, up to half a turn
b = start - side * k * 0.25
gb = g(b)
if ga == 0 or ga * gb < 0:
for _ in range(50):
mid = (a + b) / 2
if g(a) * g(mid) <= 0:
b = mid
else:
a = mid
break
a, ga = b, gb
yaw = a
rx, rz = right(yaw)
return hinge[0] + side * rx * w / 2, hinge[1] + side * rz * w / 2, yaw
mid = n // 2
if n % 2:
out[mid] = (0.0, -d, 0.0)
right_edge, left_edge, first_right, first_left = (widths[mid] / 2, -d), (-widths[mid] / 2, -d), mid + 1, mid - 1
yaw_r = yaw_l = 0.0
else: # a seam straight ahead
right_edge, left_edge, first_right, first_left = (gap / 2, -d), (-gap / 2, -d), mid, mid - 1
yaw_r = yaw_l = 0.0
for i in range(first_right, n): # to the right: screen i hangs from the previous right edge
rx, rz = right(yaw_r)
g = gap if i != first_right or n % 2 else 0
hinge = (right_edge[0] + rx * g, right_edge[1] + rz * g)
cx, cz, yaw_r = hang(hinge, widths[i], +1)
out[i] = (cx, cz, yaw_r)
rx, rz = right(yaw_r)
right_edge = (cx + rx * widths[i] / 2, cz + rz * widths[i] / 2)
for i in range(first_left, -1, -1): # to the left, mirrored
rx, rz = right(yaw_l)
g = gap if i != first_left or n % 2 else 0
hinge = (left_edge[0] - rx * g, left_edge[1] - rz * g)
cx, cz, yaw_l = hang(hinge, widths[i], -1)
out[i] = (cx, cz, yaw_l)
rx, rz = right(yaw_l)
left_edge = (cx - rx * widths[i] / 2, cz - rz * widths[i] / 2)
return out
def plan(layout, count, panel_size=None):
"""Screen poses in the head frame: [{"pos": (x, y, z), "face": (yaw, pitch), "roll": deg}]."""
sizes = [screen_size(layout, i, panel_size) for i in range(count)]
rolls = [ROLL[screen_rotation(layout, i)] for i in range(count)]
if layout.get("mode") == "custom" and len(layout.get("screens", [])) >= count and all(
"pos" in s for s in layout["screens"][:count]):
return [{"pos": tuple(s["pos"]), "face": tuple(s.get("face", yaw_pitch(s["pos"]))),
"roll": float(s.get("roll", rolls[i]))} for i, s in enumerate(layout["screens"][:count])]
p = layout["preset"]
rows = max(1, min(int(p.get("rows", 1)), count))
cols = math.ceil(count / rows)
d = max(0.3, float(p.get("distance", 2.0)))
gap = max(0.0, float(p.get("gap", 0.05)))
height = float(p.get("height", 0.0))
flat = p.get("kind") == "flat"
grid = [list(range(r * cols, min(count, (r + 1) * cols))) for r in range(rows)]
out = [None] * count
if flat:
# One flat wall: rows of screens side by side, centred, facing forward.
row_h = [max(sizes[i][1] for i in row) for row in grid]
top = height + (sum(row_h) + gap * (rows - 1)) / 2
for r, row in enumerate(grid):
y = top - sum(row_h[:r]) - gap * r - row_h[r] / 2
x = -(sum(sizes[i][0] for i in row) + gap * (len(row) - 1)) / 2
for i in row:
out[i] = {"pos": (x + sizes[i][0] / 2, y, -d), "face": (0.0, 0.0), "roll": rolls[i]}
x += sizes[i][0] + gap
return out
# Curved: each row hinged edge to edge around you (see _chain); rows stacked by angle
# (a row of height h at distance d spans 2 atan(h/2d)), each tilted to face you.
span = lambda m: 2 * math.degrees(math.atan(m / 2 / d))
row_h = [max(span(sizes[i][1]) for i in row) for row in grid]
g = span(gap)
top = math.degrees(math.atan(height / d)) + (sum(row_h) + g * (rows - 1)) / 2
for r, row in enumerate(grid):
pitch = top - sum(row_h[:r]) - g * r - row_h[r] / 2
cp, sp = math.cos(math.radians(pitch)), math.sin(math.radians(pitch))
for i, (x, z, yaw) in zip(row, _chain([sizes[i][0] for i in row], d, gap)):
# Tilt the row about the eye's left-right axis: forward distance shrinks by cos,
# height grows by sin.
r_h = math.hypot(x, z)
out[i] = {"pos": (x * cp, r_h * sp, z * cp), "face": (yaw, pitch), "roll": rolls[i]}
return out
def relative_pose(center, x_axis, z_axis, eye, heading):
"""A screen's pose in the world -> custom layout entry (pos, face, roll) in the head frame."""
rel = turn_yaw(tuple(c - e for c, e in zip(center, eye)), -heading)
fyaw, fpitch = yaw_pitch(tuple(-c for c in z_axis))
# Roll: the panel's right vector against an upright panel's right and up.
right = normalize(cross((0.0, 1.0, 0.0), z_axis))
up = cross(z_axis, right)
roll = math.degrees(math.atan2(dot(x_axis, up), dot(x_axis, right)))
return {"pos": [round(v, 4) for v in rel], "face": [round(fyaw - heading, 2), round(fpitch, 2)],
"roll": round(roll, 2)}
class Socket:
"""Request/reply over an abstract datagram socket (ft-screens or the pointer helper)."""
def __init__(self, address, what):
self.address, self.what = address, what
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
self.sock.bind("") # autobind: an abstract address the other side can reply to
def ask(self, text, timeout=10.0):
self.sock.settimeout(timeout)
try:
self.sock.sendto(text.encode(), self.address)
reply = self.sock.recv(8192).decode()
except (OSError, socket.timeout) as e:
raise RuntimeError(f"{self.what} didn't answer ({e})")
if not reply.startswith("ok"):
raise RuntimeError(reply)
return reply
# ---------------------------------------------------------------- ft-screens
def screens_socket():
return Socket(SCREENS, "ft-screens (the desktop's compositor) isn't running or")
def screen_args(layout=None):
layout = layout or load_layout()
return " ".join(f"--screen {w}x{h}@{screen_metres(layout, i):.3f}"
for i, (w, h) in ((i, screen_pixels(layout, i)) for i in range(screen_count(layout))))
def visibility(layout):
v = dict(VISIBILITY)
v.update(layout.get("visibility", {}))
return v
def send_visibility(sock, layout):
v = visibility(layout)
sock.ask(f"visibility {v['mode']}")
sock.ask(f"wrist {float(v['wrist_angle']):.1f}")
sock.ask(f"gesture {v['gesture_hand']} {float(v['gesture_angle']):.1f}")
def parse_get(reply):
"""ft-screens' "get": pose, size, curve, and the pin (hand and controller->screen)."""
f = reply.split()[1:]
g = list(map(float, f[:15]))
out = {"center": tuple(g[0:3]), "x": tuple(g[3:6]), "y": tuple(g[6:9]), "z": tuple(g[9:12]),
"metres": g[12], "height": g[13], "curve": g[14], "hand": f[15] if len(f) > 15 else "none"}
if out["hand"] != "none" and len(f) >= 28:
out["rel"] = [round(float(v), 5) for v in f[16:28]]
return out
def screens_up(sock):
"""How many screens ft-screens has shown so far."""
f = sock.ask("screens").split()
return sum(1 for s in f[2:] if not s.split(":")[1].startswith("0x"))
def apply_screens(wait=0):
layout = load_layout()
count = screen_count(layout)
deadline = time.time() + wait
while True:
try:
sock = screens_socket()
if screens_up(sock) >= count or time.time() >= deadline:
break
except RuntimeError:
if time.time() >= deadline:
raise
time.sleep(1)
f = sock.ask("head").split()
eye, heading = tuple(map(float, f[1:4])), float(f[4])
send_visibility(sock, layout)
results = []
for i, t in enumerate(plan(layout, count)):
world = turn_yaw(t["pos"], heading)
center = tuple(e + v for e, v in zip(eye, world))
entry = screen_entry(layout, i)
sock.ask(f"width {i + 1} {screen_metres(layout, i):.4f}")
sock.ask(f"curve {i + 1} {float(entry.get('curve', 0)):.3f}")
results.append(sock.ask("place %d %.4f %.4f %.4f %.3f %.3f %.3f" % (i + 1, *center, t["face"][0] + heading,
t["face"][1], t["roll"])))
pin = entry.get("pin") if layout.get("mode") == "custom" else None
if pin and len(pin.get("rel", [])) == 12:
try:
sock.ask(f"pin {i + 1} {pin['hand']} " + " ".join(f"{v:.5f}" for v in pin["rel"]))
except RuntimeError as e:
log(f"screen {i + 1}: {e}") # that controller isn't on
log(f"arranged {count} screen(s)")
return results
def capture_screens():
layout = load_layout()
sock = screens_socket()
f = sock.ask("head").split()
eye, heading = tuple(map(float, f[1:4])), float(f[4])
screens = []
for i in range(screen_count(layout)):
g = parse_get(sock.ask(f"get {i + 1}"))
entry = dict(screen_entry(layout, i))
entry.update(relative_pose(g["center"], g["x"], g["z"], eye, heading))
entry["metres"] = round(g["metres"], 4) # resized by hand
entry["curve"] = round(g["curve"], 3)
entry.pop("pin", None)
if "rel" in g:
entry["pin"] = {"hand": g["hand"], "rel": g["rel"]}
screens.append(entry)
layout["screens"] = screens + layout.get("screens", [])[len(screens):]
layout["mode"] = "custom"
save_layout(layout)
return screens
# ---------------------------------------------------------------- gamescope (dashboard panels)
def vrcmd(*args, timeout=10):
env = dict(os.environ, LD_LIBRARY_PATH=os.path.dirname(VRCMD))
try:
return subprocess.run([VRCMD, *args], capture_output=True, text=True, timeout=timeout, env=env).stdout
except (OSError, subprocess.TimeoutExpired):
return ""
def screen_keys():
"""The screens' overlay keys, in window order. gamescope (PerWindow) names each
window's overlay frametop.app.<window seq>; .app.0 is its default connector, which
never gets a window. It must be skipped: docking an unknown key moves whatever
panel the dashboard shows instead."""
keys = []
for m in re.finditer(r"^'(frametop\.app\.(\d+))' .*VROverlayType_Dashboard_Main", vrcmd("--overlays"), re.M):
if int(m.group(2)) > 0:
keys.append((int(m.group(2)), m.group(1)))
return [k for _, k in sorted(keys)]
def visible_keys():
return set(re.findall(r"^'([^']+)' .* visible VROverlayType", vrcmd("--overlays"), re.M))
def helper_measure(helper, key):
f = list(map(float, helper.ask(f"measure {key}").split()[1:]))
return {"center": tuple(f[0:3]), "size": (f[3], f[4]), "x": tuple(f[5:8]), "y": tuple(f[8:11]),
"z": tuple(f[11:14])}
def float_screen(key):
"""Float a docked screen: dock it into the dashboard (which opens the dashboard; a
floated panel takes its first position from it, and none exists while it's closed),
float it, then close the dashboard (so the carry can't snap it back in). A new
window starts docked in the dashboard, where a dashboard request is ignored as
redundant (and doesn't open the dashboard), so it goes to theater first."""
vrcmd("--dock-overlay", "theater", key)
time.sleep(0.5)
vrcmd("--dock-overlay", "dashboard", key)
time.sleep(0.8)
vrcmd("--dock-overlay", "world", key)
time.sleep(0.8)
vrcmd("--hidedashboard")
time.sleep(0.8)
def apply_gamescope(wait=0):
layout = load_layout()
count = screen_count(layout)
deadline = time.time() + wait
keys = screen_keys()
while len(keys) < count and time.time() < deadline:
time.sleep(1)
keys = screen_keys()
if wait:
time.sleep(3) # let Plasma draw before the screens start moving
if not keys:
raise RuntimeError("no Frametop screens in SteamVR; is the desktop running?")
keys = keys[:count]
helper = Socket(HELPER, "the pointer helper (frametop-pointer.service)")
f = helper.ask("head").split()
eye, heading = tuple(map(float, f[1:4])), float(f[4])
vrcmd("--hidedashboard")
time.sleep(0.5)
shown = visible_keys()
size = None
for key in keys:
if key not in shown:
float_screen(key)
try:
m = helper_measure(helper, key)
except RuntimeError:
float_screen(key) # floating but without a position: float it again
m = helper_measure(helper, key)
# The landscape size: a screen's panel is always landscape before it's rolled.
size = size or (tuple(sorted(m["size"], reverse=True)))
results = []
for key, t in zip(keys, plan(layout, len(keys), size)):
center = tuple(e + v for e, v in zip(eye, turn_yaw(t["pos"], heading)))
reply = helper.ask("place %s %.4f %.4f %.4f %.3f %.3f %.3f" % (key, *center, t["face"][0] + heading,
t["face"][1], t["roll"]), timeout=30)
log(reply)
results.append(reply)
if size and list(size) != layout.get("panel_size"):
layout["panel_size"] = [round(size[0], 4), round(size[1], 4)]
save_layout(layout)
return results
def capture_gamescope():
layout = load_layout()
helper = Socket(HELPER, "the pointer helper (frametop-pointer.service)")
f = helper.ask("head").split()
eye, heading = tuple(map(float, f[1:4])), float(f[4])
shown = visible_keys()
keys = [k for k in screen_keys() if k in shown]
if not keys:
raise RuntimeError("no floating screens to capture (screens docked in the dashboard don't count)")
screens = []
for i, key in enumerate(keys):
m = helper_measure(helper, key)
entry = dict(screen_entry(layout, i))
entry.update(relative_pose(m["center"], m["x"], m["z"], eye, heading))
screens.append(entry)
layout["panel_size"] = [round(v, 4) for v in sorted(m["size"], reverse=True)]
layout["screens"] = screens + layout.get("screens", [])[len(screens):]
layout["mode"] = "custom"
save_layout(layout)
return screens
def apply(wait=0):
return apply_screens(wait) if backend() == "screens" else apply_gamescope(wait)
def capture():
return capture_screens() if backend() == "screens" else capture_gamescope()
# ---------------------------------------------------------------- KWin (scale, positions, primary)
def nested_env():
"""Environment of the running Frametop Plasma session (its private bus and runtime dir)."""
for pid in os.listdir("/proc"):
if not pid.isdigit():
continue
try:
with open(f"/proc/{pid}/comm") as f:
if f.read().strip() != "plasmashell":
continue
with open(f"/proc/{pid}/environ", "rb") as f:
env = dict(e.split("=", 1) for e in f.read().decode(errors="replace").split("\0") if "=" in e)
except OSError:
continue
if env.get("XDG_RUNTIME_DIR", "").endswith("/frametop"):
keep = ("DBUS_SESSION_BUS_ADDRESS", "WAYLAND_DISPLAY", "XDG_RUNTIME_DIR", "XDG_CONFIG_HOME")
return dict(os.environ, **{k: env[k] for k in keep if k in env})
return None
def outputs(env):
"""KWin's outputs, in screen order (WL-0, WL-1, ...)."""
try:
data = json.loads(subprocess.run(["kscreen-doctor", "-j"], capture_output=True, text=True, env=env,
timeout=10).stdout)
except (OSError, ValueError, subprocess.TimeoutExpired):
return []
outs = [o for o in data.get("outputs", []) if o.get("connected")]
return sorted(outs, key=lambda o: [int(t) if t.isdigit() else t for t in re.split(r"(\d+)", o.get("name", ""))])
KSCREEN_ROTATION = {1: "normal", 2: "left", 4: "inverted", 8: "right"} # kscreen-doctor -j "rotation"
def apply_scales():
"""Per-screen scale and rotation, positions side by side, and the primary screen (the
taskbar goes there) to KWin, which keeps them in the session's config."""
env = nested_env()
if not env:
raise RuntimeError("the Frametop desktop isn't running")
layout = load_layout()
args = []
outs = outputs(env)
for i, o in enumerate(outs):
s = screen_scale(layout, i)
if abs(float(o.get("scale", 1)) - s) > 1e-3:
args.append(f"output.{o['id']}.scale.{s:g}")
rot = screen_rotation(layout, i)
if KSCREEN_ROTATION.get(o.get("rotation"), "normal") != rot:
args.append(f"output.{o['id']}.rotation.{rot}")
if outs:
p = outs[min(primary_screen(layout), len(outs) - 1)]
if p.get("priority") != 1:
args.append(f"output.{p['id']}.priority.1")
if args:
subprocess.run(["kscreen-doctor", *args], capture_output=True, env=env, timeout=20)
# Side by side in screen order, centred vertically, so the pointer and dragged windows
# cross between neighbours.
outs = outputs(env)
# kscreen's "size" is in pixels (already turned for a rotation); positions are in
# logical units, the pixels divided by the scale (KWin rounds up).
sizes = [(math.ceil(o["size"]["width"] / float(o.get("scale", 1)) - 1e-6),
math.ceil(o["size"]["height"] / float(o.get("scale", 1)) - 1e-6))
for o in outs if o.get("size")]
if len(sizes) == len(outs) and outs:
tallest, x, moves = max(h for _, h in sizes), 0, []
for o, (w, h) in zip(outs, sizes):
want = (x, (tallest - h) // 2)
if (o.get("pos", {}).get("x"), o.get("pos", {}).get("y")) != want:
moves.append(f"output.{o['id']}.position.{want[0]},{want[1]}")
x += w
if moves:
subprocess.run(["kscreen-doctor", *moves], capture_output=True, env=env, timeout=20)
args += moves
return args
def main(argv):
if len(argv) < 2 or argv[1] in ("-h", "--help"):
print(__doc__.split("Usage")[1].split("\n", 1)[1])
return 0 if len(argv) >= 2 else 2
cmd = argv[1]
try:
if cmd == "plan":
layout = load_layout()
print(json.dumps(plan(layout, screen_count(layout))))
elif cmd == "screen-args":
print(screen_args())
elif cmd == "toggle":
log(screens_socket().ask("toggle"))
elif cmd in ("pin", "unpin") and len(argv) >= 3:
log(screens_socket().ask(" ".join(argv[1:])))
elif cmd in ("apply", "capture", "scale"):
with open(LOCK_PATH, "w") as lock:
try:
fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
log("another ft-layout is already running")
return 1
if cmd == "apply":
wait = float(argv[argv.index("--wait") + 1]) if "--wait" in argv else 0
if wait and not load_layout().get("auto", True):
log("auto-arrange is off")
if backend() == "screens": # the visibility settings apply anyway
try:
sock = screens_socket()
deadline = time.time() + wait
while screens_up(sock) < screen_count() and time.time() < deadline:
time.sleep(1)
send_visibility(sock, load_layout())
except RuntimeError as e:
log(f"visibility: {e}")
else:
apply(wait)
if wait:
# KWin keeps these, but new screens or a changed layout need them once.
for _ in range(30): # Plasma may still be starting
try:
log("kwin: " + (" ".join(apply_scales()) or "unchanged"))
break
except RuntimeError as e:
last = e
time.sleep(1)
else:
log(f"kwin: {last}")
elif cmd == "capture":
for i, s in enumerate(capture()):
log(f"screen {i + 1}: {s}")
else:
changes = apply_scales()
log("kwin: " + (" ".join(changes) if changes else "unchanged"))
else:
print(f"unknown command: {cmd}", file=sys.stderr)
return 2
except RuntimeError as e:
log(f"error: {e}")
return 1
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv))
pointer/common/vrmath.h
+152
View File
@@ -0,0 +1,152 @@
// Small vector math shared by the pointer helper, the layout tool, and the probe, plus
// ScanPanel: measuring a floating dashboard panel, whose transform OpenVR won't give out.
// Header-only. Standing-universe coordinates unless a name says otherwise.
#pragma once
#include <openvr.h>
#include <cmath>
namespace md {
struct Vec3 {
double x = 0, y = 0, z = 0;
};
inline Vec3 operator+(Vec3 a, Vec3 b) { return {a.x + b.x, a.y + b.y, a.z + b.z}; }
inline Vec3 operator-(Vec3 a, Vec3 b) { return {a.x - b.x, a.y - b.y, a.z - b.z}; }
inline Vec3 operator*(Vec3 a, double s) { return {a.x * s, a.y * s, a.z * s}; }
inline double Dot(Vec3 a, Vec3 b) { return a.x * b.x + a.y * b.y + a.z * b.z; }
inline Vec3 Cross(Vec3 a, Vec3 b) { return {a.y * b.z - a.z * b.y, a.z * b.x - a.x * b.z, a.x * b.y - a.y * b.x}; }
inline double Length(Vec3 a) { return std::sqrt(Dot(a, a)); }
inline Vec3 Normalize(Vec3 a) {
const double n = Length(a);
return n > 1e-9 ? a * (1.0 / n) : Vec3{0, 0, -1};
}
// yaw 0 = -Z (SteamVR forward), positive yaw turns left (about +Y), positive pitch looks up.
inline Vec3 Direction(double yawDeg, double pitchDeg) {
const double y = yawDeg * M_PI / 180, p = pitchDeg * M_PI / 180;
return {-std::sin(y) * std::cos(p), std::sin(p), -std::cos(y) * std::cos(p)};
}
// Rotation part of a pose matrix applied to a vector, and its transpose.
inline Vec3 Rotate(const vr::HmdMatrix34_t &m, Vec3 v) {
return {m.m[0][0] * v.x + m.m[0][1] * v.y + m.m[0][2] * v.z, m.m[1][0] * v.x + m.m[1][1] * v.y + m.m[1][2] * v.z,
m.m[2][0] * v.x + m.m[2][1] * v.y + m.m[2][2] * v.z};
}
inline Vec3 RotateInverse(const vr::HmdMatrix34_t &m, Vec3 v) {
return {m.m[0][0] * v.x + m.m[1][0] * v.y + m.m[2][0] * v.z, m.m[0][1] * v.x + m.m[1][1] * v.y + m.m[2][1] * v.z,
m.m[0][2] * v.x + m.m[1][2] * v.y + m.m[2][2] * v.z};
}
inline Vec3 Position(const vr::HmdMatrix34_t &m) { return {m.m[0][3], m.m[1][3], m.m[2][3]}; }
// Rodrigues: v rotated by angle (radians) about a unit axis.
inline Vec3 RotateAbout(Vec3 v, Vec3 axis, double angle) {
const double c = std::cos(angle), s = std::sin(angle);
return v * c + Cross(axis, v) * s + axis * (Dot(axis, v) * (1 - c));
}
// An orthonormal frame: device poses (-Z forward) and panels (+X right, +Y up, +Z out of the front).
struct Basis {
Vec3 x, y, z;
};
// Coordinates of v in the basis, and back.
inline Vec3 ToBasis(const Basis &b, Vec3 v) { return {Dot(v, b.x), Dot(v, b.y), Dot(v, b.z)}; }
inline Vec3 FromBasis(const Basis &b, Vec3 v) { return b.x * v.x + b.y * v.y + b.z * v.z; }
// Device basis for a pointing direction with no roll: -Z along aim, +X horizontal.
inline Basis AimBasis(Vec3 aim) {
const Vec3 z = Normalize(aim * -1.0);
const Vec3 x = Normalize(Cross({0, 1, 0}, z));
return {x, Cross(z, x), z};
}
// Panel basis for a panel facing the direction (yaw, pitch) points to. The front (+Z)
// faces back along that direction, toward whoever looks along it. roll turns the panel
// about its front normal, counterclockwise as you see it (90: a rotated-left monitor).
inline Basis PanelBasis(double yawDeg, double pitchDeg, double rollDeg = 0) {
const Vec3 z = Direction(yawDeg, pitchDeg) * -1.0;
const Vec3 x = Normalize(Cross({0, 1, 0}, z)), y = Cross(z, x);
const double r = rollDeg * M_PI / 180, c = std::cos(r), s = std::sin(r);
return {x * c + y * s, y * c - x * s, z};
}
// Quaternion (w, x, y, z) of a rotation whose matrix columns are the basis vectors.
inline void BasisQuat(const Basis &b, double q[4]) {
const double m[3][3] = {{b.x.x, b.y.x, b.z.x}, {b.x.y, b.y.y, b.z.y}, {b.x.z, b.y.z, b.z.z}};
const double trace = m[0][0] + m[1][1] + m[2][2];
if (trace > 0) {
const double s = 0.5 / std::sqrt(trace + 1);
q[0] = 0.25 / s, q[1] = (m[2][1] - m[1][2]) * s, q[2] = (m[0][2] - m[2][0]) * s, q[3] = (m[1][0] - m[0][1]) * s;
} else if (m[0][0] > m[1][1] && m[0][0] > m[2][2]) {
const double s = 2 * std::sqrt(1 + m[0][0] - m[1][1] - m[2][2]);
q[0] = (m[2][1] - m[1][2]) / s, q[1] = 0.25 * s, q[2] = (m[0][1] + m[1][0]) / s, q[3] = (m[0][2] + m[2][0]) / s;
} else if (m[1][1] > m[2][2]) {
const double s = 2 * std::sqrt(1 + m[1][1] - m[0][0] - m[2][2]);
q[0] = (m[0][2] - m[2][0]) / s, q[1] = (m[0][1] + m[1][0]) / s, q[2] = 0.25 * s, q[3] = (m[1][2] + m[2][1]) / s;
} else {
const double s = 2 * std::sqrt(1 + m[2][2] - m[0][0] - m[1][1]);
q[0] = (m[1][0] - m[0][1]) / s, q[1] = (m[0][2] + m[2][0]) / s, q[2] = (m[1][2] + m[2][1]) / s, q[3] = 0.25 * s;
}
}
// A panel measured by ScanPanel: centre, size, and frame (x right, y up, z out of the front).
struct Panel {
bool found = false;
Vec3 center;
double width = 0, height = 0;
Basis basis;
int hits = 0;
};
// Cast rays from `from` over the whole sphere (step in degrees) at one overlay, and fit
// point = origin + u*U + v*V to the hits (least squares). ComputeOverlayIntersection
// works on floating dashboard panels, whose transforms aren't readable, and returns the
// texture coordinates of each hit; v runs bottom to top. It's local and fast: a 0.5-degree
// scan (about 230,000 rays) takes 0.1 s.
inline Panel ScanPanel(vr::VROverlayHandle_t h, Vec3 from, double step = 1.0) {
Panel p;
double ata[3][3] = {}, atb[3][3] = {}; // normal equations for [1 u v] -> (x, y, z)
for (double pitch = -80; pitch <= 80; pitch += step)
for (double yaw = -180; yaw < 180; yaw += step) {
const Vec3 d = Direction(yaw, pitch);
vr::VROverlayIntersectionParams_t params{};
params.vSource = {float(from.x), float(from.y), float(from.z)};
params.vDirection = {float(d.x), float(d.y), float(d.z)};
params.eOrigin = vr::TrackingUniverseStanding;
vr::VROverlayIntersectionResults_t hit{};
if (!vr::VROverlay()->ComputeOverlayIntersection(h, &params, &hit)) continue;
++p.hits;
const double row[3] = {1, hit.vUVs.v[0], hit.vUVs.v[1]};
for (int i = 0; i < 3; ++i)
for (int j = 0; j < 3; ++j) {
ata[i][j] += row[i] * row[j];
atb[i][j] += row[i] * hit.vPoint.v[j];
}
}
if (p.hits < 6) return p;
auto det3 = [](const double a[3][3]) {
return a[0][0] * (a[1][1] * a[2][2] - a[1][2] * a[2][1]) - a[0][1] * (a[1][0] * a[2][2] - a[1][2] * a[2][0]) +
a[0][2] * (a[1][0] * a[2][1] - a[1][1] * a[2][0]);
};
const double d = det3(ata);
if (std::fabs(d) < 1e-12) return p;
double x[3][3]; // x[k][j]: coefficient k (1, u, v) of coordinate j, by Cramer's rule
for (int j = 0; j < 3; ++j)
for (int k = 0; k < 3; ++k) {
double t[3][3];
for (int r = 0; r < 3; ++r)
for (int c = 0; c < 3; ++c) t[r][c] = c == k ? atb[r][j] : ata[r][c];
x[k][j] = det3(t) / d;
}
const Vec3 O{x[0][0], x[0][1], x[0][2]}, U{x[1][0], x[1][1], x[1][2]}, V{x[2][0], x[2][1], x[2][2]};
p.found = true;
p.center = O + U * 0.5 + V * 0.5;
p.width = Length(U);
p.height = Length(V);
const Vec3 bx = Normalize(U), bz = Normalize(Cross(U, V));
p.basis = {bx, Cross(bz, bx), bz};
return p;
}
} // namespace md
pointer/driver/build.sh
Executable
+18
View File
@@ -0,0 +1,18 @@
#!/usr/bin/env bash
# Build the ft_pointer SteamVR driver on the Frame (dev container) and check it
# only needs glibc symbols the SteamOS host has (2.39; the container has 2.43).
# -fno-math-errno keeps sqrtf inline. libm's float functions (sqrtf, atan2f, asinf,
# remainderf) are versioned GLIBC_2.43 here, so the driver uses the double versions.
# Usage: pointer/driver/build.sh
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
"$root/scripts/sync.sh" >/dev/null
exec "$root/scripts/frame.sh" -C pointer/driver 'set -e
mkdir -p build
g++ -std=c++17 -O2 -fPIC -shared -fvisibility=hidden -fno-math-errno -Wall -Wno-unused-parameter \
-static-libstdc++ -static-libgcc -Wl,--exclude-libs,ALL \
-I/opt/steamvr/tools/hellovr_vulkan_linux/src/openvr/headers \
-o build/driver_ft_pointer.so driver_ft_pointer.cpp -lpthread
max=$(objdump -T build/driver_ft_pointer.so | grep -oE "GLIBC_[0-9.]+" | sort -uV | tail -1)
echo "built build/driver_ft_pointer.so, newest glibc symbol: $max"
[ "$(printf "%s\n" "$max" GLIBC_2.39 | sort -V | tail -1)" = GLIBC_2.39 ] || { echo "needs newer glibc than the host has" >&2; exit 1; }'
pointer/driver/driver_ft_pointer.cpp
+357
View File
@@ -0,0 +1,357 @@
// ft_pointer: a virtual SteamVR controller that the universal 3D mouse drives.
// The cursor is a point anchored in the room: `distance` metres from where the
// head was at the last recenter, in the yaw/pitch direction the mouse steers.
// The ray starts at the eye (the HMD origin) and aims at that point, so
// SteamVR's laser is seen end-on and only its hit dot shows. The device uses an
// invisible render model. The input relay (input/input-relay.py)
// drives it over a datagram socket.
//
// Control socket: abstract unix datagram "@ft_pointer", text commands:
// recenter anchor the origin at the head and aim along the gaze
// move <dyaw> <dpitch> rotate the ray (degrees; +yaw turns left, +pitch up)
// aim <yaw_deg> <pitch_deg> absolute direction (yaw 0 = -Z, the SteamVR forward)
// gaze follow the head (origin and direction) again
// distance <metres> cursor distance from the anchor (default 1.5)
// pose <x> <y> <z> <yaw> <pitch> exact pose, sent every frame by the ft-pointer helper
// posq <x> <y> <z> <qw> <qx> <qy> <qz> exact pose with a full rotation (tilting a panel while moving it)
// btn <name> <0|1> name: trigger, b, x, system, joystick, a (a = claim the laser, no click)
// scroll <x> <y> joystick deflection -1..1
// show | hide connect (take the hand role) or disconnect (give it back)
//
// The device starts disconnected, so it never holds a hand role at boot (holding
// the right hand while SteamVR started left the Steam UI stuck loading). It
// connects only while the mouse is in use, so the last used device wins. Its
// role hint follows: the configured hand while connected, OptOut while not.
// SteamVR keeps a hand role reserved for a disconnected device that still hints
// that hand, so the real controller would never get it back otherwise.
//
// Settings (steamvr.vrsettings section "driver_ft_pointer"): role (int, 2 = right hand, 5 = stylus).
#include <openvr_driver.h>
#include <atomic>
#include <cmath>
#include <cstdio>
#include <cstring>
#include <mutex>
#include <string>
#include <thread>
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
using namespace vr;
namespace {
struct State {
std::mutex lock;
bool gaze = true;
bool visible = false; // disconnected until "show"
bool recenter = false; // applied on the next frame, which has the head pose
bool anchored = false;
float anchor[3] = {};
bool explicitPose = false; // set by "pose": the helper drives position and direction
float pos[3] = {};
bool hasQuat = false; // set by "posq": use quat instead of yaw/pitch
float quat[4] = {1, 0, 0, 0};
float yaw = 0.f, pitch = 0.f; // degrees
bool buttons[6] = {};
float distance = 1.5f;
float scrollX = 0.f, scrollY = 0.f;
};
const int kButtons = 6;
const char *kButtonNames[kButtons] = {"trigger", "b", "x", "system", "joystick", "a"};
HmdQuaternion_t QuatFromYawPitch(float yawDeg, float pitchDeg) {
// Yaw about +Y, then pitch about +X. SteamVR forward is -Z.
const float y = yawDeg * float(M_PI) / 360.f, p = pitchDeg * float(M_PI) / 360.f;
const float cy = std::cos(y), sy = std::sin(y), cp = std::cos(p), sp = std::sin(p);
return {cy * cp, cy * sp, sy * cp, -sy * sp};
}
// Rotation part of a 3x4 pose matrix as a quaternion (all four branches).
HmdQuaternion_t QuatFromMatrix(const float (&m)[3][4]) {
const float trace = m[0][0] + m[1][1] + m[2][2];
if (trace > 0) {
const float s = 0.5f / std::sqrt(trace + 1.f);
return {0.25f / s, (m[2][1] - m[1][2]) * s, (m[0][2] - m[2][0]) * s, (m[1][0] - m[0][1]) * s};
}
if (m[0][0] > m[1][1] && m[0][0] > m[2][2]) {
const float s = 2.f * std::sqrt(1.f + m[0][0] - m[1][1] - m[2][2]);
return {(m[2][1] - m[1][2]) / s, 0.25f * s, (m[0][1] + m[1][0]) / s, (m[0][2] + m[2][0]) / s};
}
if (m[1][1] > m[2][2]) {
const float s = 2.f * std::sqrt(1.f + m[1][1] - m[0][0] - m[2][2]);
return {(m[0][2] - m[2][0]) / s, (m[0][1] + m[1][0]) / s, 0.25f * s, (m[1][2] + m[2][1]) / s};
}
const float s = 2.f * std::sqrt(1.f + m[2][2] - m[0][0] - m[1][1]);
return {(m[1][0] - m[0][1]) / s, (m[0][2] + m[2][0]) / s, (m[1][2] + m[2][1]) / s, 0.25f * s};
}
class PointerDevice : public ITrackedDeviceServerDriver {
public:
explicit PointerDevice(State *state) : state_(state) {}
EVRInitError Activate(uint32_t objectId) override {
objectId_ = objectId;
auto props = VRProperties();
const PropertyContainerHandle_t c = props->TrackedDeviceToPropertyContainer(objectId);
container_ = c;
EVRSettingsError err;
const int32_t configured = VRSettings()->GetInt32("driver_ft_pointer", "role", &err);
if (err == VRSettingsError_None && configured > 0) role_ = configured;
props->SetStringProperty(c, Prop_ModelNumber_String, "ft_pointer");
props->SetStringProperty(c, Prop_ManufacturerName_String, "Frametop");
props->SetStringProperty(c, Prop_ControllerType_String, "ft_pointer");
props->SetStringProperty(c, Prop_InputProfilePath_String, "{ft_pointer}/input/ft_pointer_profile.json");
props->SetStringProperty(c, Prop_RenderModelName_String, "{ft_pointer}/rendermodels/ft_pointer_invisible");
props->SetInt32Property(c, Prop_ControllerRoleHint_Int32, TrackedControllerRole_OptOut); // until "show"
props->SetInt32Property(c, Prop_DeviceClass_Int32, TrackedDeviceClass_Controller);
props->SetBoolProperty(c, Prop_NeverTracked_Bool, false);
auto input = VRDriverInput();
input->CreateBooleanComponent(c, "/input/trigger/click", &buttons_[0]);
input->CreateBooleanComponent(c, "/input/b/click", &buttons_[1]);
input->CreateBooleanComponent(c, "/input/x/click", &buttons_[2]);
input->CreateBooleanComponent(c, "/input/system/click", &buttons_[3]);
input->CreateBooleanComponent(c, "/input/joystick/click", &buttons_[4]);
input->CreateBooleanComponent(c, "/input/a/click", &buttons_[5]);
input->CreateScalarComponent(c, "/input/joystick/x", &scrollX_, VRScalarType_Absolute, VRScalarUnits_NormalizedTwoSided);
input->CreateScalarComponent(c, "/input/joystick/y", &scrollY_, VRScalarType_Absolute, VRScalarUnits_NormalizedTwoSided);
VRDriverLog()->Log("ft_pointer: activated");
return VRInitError_None;
}
void Deactivate() override { objectId_ = k_unTrackedDeviceIndexInvalid; }
void EnterStandby() override {}
void *GetComponent(const char *) override { return nullptr; }
void DebugRequest(const char *, char *response, uint32_t size) override {
if (size) response[0] = 0;
}
DriverPose_t GetPose() override { return pose_; }
void RunFrame() {
if (objectId_ == k_unTrackedDeviceIndexInvalid) return;
TrackedDevicePose_t hmd{};
VRServerDriverHost()->GetRawTrackedDevicePoses(0.f, &hmd, 1);
State snapshot;
{
std::lock_guard<std::mutex> guard(state_->lock);
if (state_->recenter || (!state_->gaze && !state_->anchored)) {
const auto &h = hmd.mDeviceToAbsoluteTracking.m;
if (hmd.bPoseIsValid) {
state_->anchor[0] = h[0][3];
state_->anchor[1] = h[1][3];
state_->anchor[2] = h[2][3];
state_->anchored = true;
if (state_->recenter) {
const float fx = -h[0][2], fy = -h[1][2], fz = -h[2][2];
// double-precision libm: the float versions are GLIBC_2.43 in the build container.
state_->yaw = float(std::atan2(double(-fx), double(-fz)) * 180.0 / M_PI);
state_->pitch = float(std::asin(double(fy)) * 180.0 / M_PI);
state_->gaze = false;
state_->recenter = false;
}
}
}
snapshot.gaze = state_->gaze;
std::memcpy(snapshot.anchor, state_->anchor, sizeof snapshot.anchor);
snapshot.distance = state_->distance;
snapshot.explicitPose = state_->explicitPose;
std::memcpy(snapshot.pos, state_->pos, sizeof snapshot.pos);
snapshot.hasQuat = state_->hasQuat;
std::memcpy(snapshot.quat, state_->quat, sizeof snapshot.quat);
snapshot.visible = state_->visible;
snapshot.yaw = state_->yaw;
snapshot.pitch = state_->pitch;
std::memcpy(snapshot.buttons, state_->buttons, sizeof snapshot.buttons);
snapshot.scrollX = state_->scrollX;
snapshot.scrollY = state_->scrollY;
}
DriverPose_t pose{};
pose.qWorldFromDriverRotation.w = 1.f;
pose.qDriverFromHeadRotation.w = 1.f;
const auto &m = hmd.mDeviceToAbsoluteTracking.m;
if (snapshot.explicitPose) {
for (int i = 0; i < 3; ++i) pose.vecPosition[i] = snapshot.pos[i];
pose.qRotation = snapshot.hasQuat
? HmdQuaternion_t{snapshot.quat[0], snapshot.quat[1], snapshot.quat[2], snapshot.quat[3]}
: QuatFromYawPitch(snapshot.yaw, snapshot.pitch);
} else if (snapshot.gaze) {
pose.vecPosition[0] = m[0][3];
pose.vecPosition[1] = m[1][3] - 0.05f; // just below the eyes
pose.vecPosition[2] = m[2][3];
pose.qRotation = QuatFromMatrix(m);
} else {
// Cursor point P = anchor + distance * dir(yaw, pitch). Aim from the eye at P.
const double yr = snapshot.yaw * M_PI / 180.0, pr = snapshot.pitch * M_PI / 180.0;
const double p[3] = {snapshot.anchor[0] - snapshot.distance * std::sin(yr) * std::cos(pr),
snapshot.anchor[1] + snapshot.distance * std::sin(pr),
snapshot.anchor[2] - snapshot.distance * std::cos(yr) * std::cos(pr)};
const double eye[3] = {m[0][3], m[1][3], m[2][3]};
const double d[3] = {p[0] - eye[0], p[1] - eye[1], p[2] - eye[2]};
const double horizontal = std::sqrt(d[0] * d[0] + d[2] * d[2]);
for (int i = 0; i < 3; ++i) pose.vecPosition[i] = eye[i];
pose.qRotation = QuatFromYawPitch(float(std::atan2(-d[0], -d[2]) * 180.0 / M_PI),
float(std::atan2(d[1], horizontal) * 180.0 / M_PI));
}
if (snapshot.visible != hinted_) {
// Claim the hand before connecting; give it up when disconnecting.
VRProperties()->SetInt32Property(container_, Prop_ControllerRoleHint_Int32,
snapshot.visible ? role_ : int32_t(TrackedControllerRole_OptOut));
hinted_ = snapshot.visible;
}
const bool ok = hmd.bPoseIsValid && snapshot.visible;
pose.poseIsValid = ok;
pose.result = ok ? TrackingResult_Running_OK : TrackingResult_Uninitialized;
pose.deviceIsConnected = snapshot.visible;
pose_ = pose;
VRServerDriverHost()->TrackedDevicePoseUpdated(objectId_, pose_, sizeof(DriverPose_t));
auto input = VRDriverInput();
for (int i = 0; i < kButtons; ++i) input->UpdateBooleanComponent(buttons_[i], snapshot.buttons[i], 0);
input->UpdateScalarComponent(scrollX_, snapshot.scrollX, 0);
input->UpdateScalarComponent(scrollY_, snapshot.scrollY, 0);
}
private:
State *state_;
uint32_t objectId_ = k_unTrackedDeviceIndexInvalid;
PropertyContainerHandle_t container_ = k_ulInvalidPropertyContainer;
int32_t role_ = TrackedControllerRole_RightHand;
bool hinted_ = false; // whether the role hint currently claims role_
DriverPose_t pose_{};
VRInputComponentHandle_t buttons_[kButtons] = {};
VRInputComponentHandle_t scrollX_ = 0, scrollY_ = 0;
};
class Provider : public IServerTrackedDeviceProvider {
public:
EVRInitError Init(IVRDriverContext *context) override {
VR_INIT_SERVER_DRIVER_CONTEXT(context);
device_ = new PointerDevice(&state_);
VRServerDriverHost()->TrackedDeviceAdded("ft_pointer_0", TrackedDeviceClass_Controller, device_);
running_ = true;
listener_ = std::thread([this] { Listen(); });
return VRInitError_None;
}
void Cleanup() override {
running_ = false;
if (sock_ >= 0) shutdown(sock_, SHUT_RDWR);
if (listener_.joinable()) listener_.join();
if (sock_ >= 0) close(sock_);
VR_CLEANUP_SERVER_DRIVER_CONTEXT();
}
const char *const *GetInterfaceVersions() override { return k_InterfaceVersions; }
void RunFrame() override {
if (device_) device_->RunFrame();
}
bool ShouldBlockStandbyMode() override { return false; }
void EnterStandby() override {}
void LeaveStandby() override {}
private:
void Listen() {
sock_ = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC, 0);
sockaddr_un addr{};
addr.sun_family = AF_UNIX;
const char name[] = "ft_pointer";
std::memcpy(addr.sun_path + 1, name, sizeof name - 1); // abstract namespace
const socklen_t len = offsetof(sockaddr_un, sun_path) + 1 + sizeof name - 1;
if (bind(sock_, reinterpret_cast<sockaddr *>(&addr), len) != 0) {
VRDriverLog()->Log("ft_pointer: cannot bind control socket");
return;
}
timeval tv{0, 200000};
setsockopt(sock_, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof tv);
char buf[256];
while (running_) {
const ssize_t n = recv(sock_, buf, sizeof buf - 1, 0);
if (n <= 0) continue;
buf[n] = 0;
Handle(buf);
}
}
void Handle(const char *cmd) {
std::lock_guard<std::mutex> guard(state_.lock);
char name[32];
float a, b;
int v;
float x, y, z, qw, qx, qy, qz;
if (std::sscanf(cmd, "posq %f %f %f %f %f %f %f", &x, &y, &z, &qw, &qx, &qy, &qz) == 7) {
state_.explicitPose = true;
state_.hasQuat = true;
state_.gaze = false;
state_.pos[0] = x;
state_.pos[1] = y;
state_.pos[2] = z;
state_.quat[0] = qw;
state_.quat[1] = qx;
state_.quat[2] = qy;
state_.quat[3] = qz;
} else if (std::sscanf(cmd, "pose %f %f %f %f %f", &x, &y, &z, &a, &b) == 5) {
state_.explicitPose = true;
state_.hasQuat = false;
state_.gaze = false;
state_.pos[0] = x;
state_.pos[1] = y;
state_.pos[2] = z;
state_.yaw = a;
state_.pitch = b;
} else if (std::sscanf(cmd, "move %f %f", &a, &b) == 2) {
state_.explicitPose = false;
if (state_.gaze) state_.recenter = true; // first move starts from the gaze
state_.yaw += a; // wrap by hand: libm remainder() is GLIBC_2.43 in the build container
while (state_.yaw > 180.f) state_.yaw -= 360.f;
while (state_.yaw < -180.f) state_.yaw += 360.f;
state_.pitch = std::fmax(-85.f, std::fmin(85.f, state_.pitch + b));
} else if (std::strncmp(cmd, "recenter", 8) == 0) {
state_.explicitPose = false;
state_.recenter = true;
} else if (std::sscanf(cmd, "aim %f %f", &a, &b) == 2) {
state_.gaze = false;
state_.yaw = a;
state_.pitch = b;
} else if (std::strncmp(cmd, "gaze", 4) == 0) {
state_.gaze = true;
} else if (std::strncmp(cmd, "hide", 4) == 0) {
state_.visible = false;
} else if (std::strncmp(cmd, "show", 4) == 0) {
state_.visible = true;
} else if (std::sscanf(cmd, "btn %31s %d", name, &v) == 2) {
for (int i = 0; i < kButtons; ++i)
if (std::strcmp(name, kButtonNames[i]) == 0) state_.buttons[i] = v != 0;
} else if (std::sscanf(cmd, "distance %f", &a) == 1) {
state_.distance = std::fmax(0.3f, std::fmin(10.f, a));
} else if (std::sscanf(cmd, "scroll %f %f", &a, &b) == 2) {
state_.scrollX = a;
state_.scrollY = b;
}
}
State state_;
PointerDevice *device_ = nullptr;
std::thread listener_;
std::atomic<bool> running_{false};
int sock_ = -1;
};
Provider g_provider;
} // namespace
extern "C" __attribute__((visibility("default"))) void *HmdDriverFactory(const char *interfaceName, int *returnCode) {
if (std::strcmp(interfaceName, IServerTrackedDeviceProvider_Version) == 0) return &g_provider;
if (returnCode) *returnCode = VRInitError_Init_InterfaceNotFound;
return nullptr;
}
pointer/driver/ft_pointer/driver.vrdrivermanifest
+7
View File
@@ -0,0 +1,7 @@
{
"alwaysActivate": true,
"name": "ft_pointer",
"directory": "",
"resourceOnly": false,
"hmd_presence": []
}
pointer/driver/ft_pointer/resources/input/ft_pointer_profile.json
+51
View File
@@ -0,0 +1,51 @@
{
"jsonid": "input_profile",
"controller_type": "ft_pointer",
"device_class": "TrackedDeviceClass_Controller",
"resource_root": "ft_pointer",
"driver_name": "ft_pointer",
"input_bindingui_mode": "controller_handed",
"should_show_binding_errors": true,
"input_source": {
"/input/trigger": {
"type": "button",
"click": true
},
"/input/b": {
"type": "button",
"click": true
},
"/input/x": {
"type": "button",
"click": true
},
"/input/system": {
"type": "button",
"click": true
},
"/input/joystick": {
"type": "joystick",
"click": true
},
"/pose/raw": {
"type": "pose"
},
"/pose/tip": {
"type": "pose"
},
"/input/a": {
"type": "button",
"click": true
}
},
"default_bindings": [
{
"app_key": "openvr.component.vrcompositor",
"binding_url": "ft_pointer_vrcompositor.json"
},
{
"app_key": "steam.client",
"binding_url": "ft_pointer_steam.json"
}
]
}
pointer/driver/ft_pointer/resources/input/ft_pointer_steam.json
+16
View File
@@ -0,0 +1,16 @@
{
"action_manifest_version": 0,
"alias_info": {},
"app_key": "steam.client",
"category": "steamvr_input",
"controller_type": "ft_pointer",
"description": "",
"name": "frametop pointer: Steam client (input comes through the dashboard laser mouse)",
"options": {},
"simulated_actions": [],
"bindings": {
"/actions/haptics": {
"sources": []
}
}
}
pointer/driver/ft_pointer/resources/input/ft_pointer_vrcompositor.json
+205
View File
@@ -0,0 +1,205 @@
{
"action_manifest_version": 0,
"alias_info": {},
"app_key": "openvr.component.vrcompositor",
"category": "steamvr_input",
"controller_type": "ft_pointer",
"description": "",
"name": "frametop pointer: dashboard laser mouse",
"options": {},
"simulated_actions": [],
"bindings": {
"/actions/lasermouse": {
"sources": [
{
"path": "/user/hand/left/input/trigger",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse/in/leftclick"
}
}
},
{
"path": "/user/hand/left/input/b",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse/in/rightclick"
}
}
},
{
"path": "/user/hand/left/input/x",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse/in/middleclick"
}
}
},
{
"path": "/user/hand/left/input/joystick",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse/in/back"
}
}
},
{
"path": "/user/hand/right/input/trigger",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse/in/leftclick"
}
}
},
{
"path": "/user/hand/right/input/b",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse/in/rightclick"
}
}
},
{
"path": "/user/hand/right/input/x",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse/in/middleclick"
}
}
},
{
"path": "/user/hand/right/input/joystick",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse/in/back"
}
}
}
],
"poses": [
{
"output": "/actions/lasermouse/in/Pointer",
"path": "/user/hand/left/pose/raw"
},
{
"output": "/actions/lasermouse/in/Pointer",
"path": "/user/hand/right/pose/raw"
}
]
},
"/actions/scroll_discrete": {
"sources": [
{
"path": "/user/hand/left/input/joystick",
"mode": "scroll",
"inputs": {
"scroll": {
"output": "/actions/scroll_discrete/in/scroll"
}
}
},
{
"path": "/user/hand/right/input/joystick",
"mode": "scroll",
"inputs": {
"scroll": {
"output": "/actions/scroll_discrete/in/scroll"
}
}
}
]
},
"/actions/scroll_smooth": {
"sources": [
{
"path": "/user/hand/left/input/joystick",
"mode": "scroll",
"inputs": {
"scroll": {
"output": "/actions/scroll_smooth/in/scroll"
}
}
},
{
"path": "/user/hand/right/input/joystick",
"mode": "scroll",
"inputs": {
"scroll": {
"output": "/actions/scroll_smooth/in/scroll"
}
}
}
]
},
"/actions/system": {
"sources": [
{
"path": "/user/hand/left/input/system",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/system/in/ToggleDashboard"
}
}
},
{
"path": "/user/hand/right/input/system",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/system/in/ToggleDashboard"
}
}
}
]
},
"/actions/lasermouse_secondary": {
"sources": [
{
"path": "/user/hand/left/input/trigger",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse_secondary/in/switchlaserhand"
}
}
},
{
"path": "/user/hand/right/input/trigger",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse_secondary/in/switchlaserhand"
}
}
},
{
"path": "/user/hand/left/input/a",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse_secondary/in/switchlaserhand"
}
}
},
{
"path": "/user/hand/right/input/a",
"mode": "button",
"inputs": {
"click": {
"output": "/actions/lasermouse_secondary/in/switchlaserhand"
}
}
}
]
}
}
}
pointer/driver/ft_pointer/resources/rendermodels/ft_pointer_invisible/ft_pointer_invisible.mtl
+3
View File
@@ -0,0 +1,3 @@
newmtl invisible
d 0
map_Kd ft_pointer_invisible.png
pointer/driver/ft_pointer/resources/rendermodels/ft_pointer_invisible/ft_pointer_invisible.obj
+9
View File
@@ -0,0 +1,9 @@
# Invisible render model for the ft_pointer virtual controller.
mtllib ft_pointer_invisible.mtl
v 0 0 0
v 0.0001 0 0
v 0 0.0001 0
vt 0 0
vn 0 0 1
usemtl invisible
f 1/1/1 2/1/1 3/1/1
pointer/driver/ft_pointer/resources/rendermodels/ft_pointer_invisible/ft_pointer_invisible.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 68 B

pointer/driver/ft_pointer/resources/settings/default.vrsettings
+6
View File
@@ -0,0 +1,6 @@
{
"driver_ft_pointer": {
"enable": true,
"role": 2
}
}
pointer/driver/install.sh
Executable
+39
View File
@@ -0,0 +1,39 @@
#!/usr/bin/env bash
# Install, remove, or poke the ft_pointer SteamVR driver on the Frame.
# Usage: pointer/driver/install.sh install # copy to ~/.local/share/frametop/ft_pointer and register
# pointer/driver/install.sh uninstall # unregister and delete
# pointer/driver/install.sh send '<cmd>' # e.g. 'btn trigger 1', 'aim 20 -5', 'gaze'
# pointer/driver/install.sh aimhere # pin the ray (room-anchored) where the head points now
# pointer/driver/install.sh probe # devices, roles, dashboard pointer (pointer/probe)
# pointer/driver/install.sh log # ft_pointer lines from vrserver.txt
# SteamVR loads drivers only at startup: restart it after install or uninstall.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
. "$root/scripts/_env.sh"
frame="$root/scripts/frame.sh"
src=$FRAME_REPO/pointer/driver
dest=/home/steamos/.local/share/frametop/ft_pointer
reg='/opt/steamvr/bin/linuxarm64/vrpathreg'
case ${1:-install} in
install)
"$root/scripts/sync.sh" >/dev/null
"$frame" --host "set -e; test -f $src/build/driver_ft_pointer.so
rm -rf $dest; mkdir -p $dest/bin/linuxarm64
cp -r $src/ft_pointer/. $dest/
cp $src/build/driver_ft_pointer.so $dest/bin/linuxarm64/
LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg adddriver $dest
LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg show | grep -A3 -i 'external'
echo 'installed; restart SteamVR to load it'" ;;
uninstall)
"$frame" --host "LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg removedriver $dest; rm -rf $dest; echo 'removed; restart SteamVR to unload it'" ;;
send)
"$frame" --host "python3 -c 'import socket,sys; s=socket.socket(socket.AF_UNIX,socket.SOCK_DGRAM); s.sendto(sys.argv[1].encode(), \"\\0ft_pointer\")' $(printf %q "${2:?command}")" ;;
probe) "$frame" -C pointer/probe 'LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 ./build/vrprobe' ;;
aimhere)
read -r yaw pitch < <("$frame" -C pointer/probe 'LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 ./build/vrprobe' | sed -n 's/^head yaw = \([-0-9.]*\) pitch = \([-0-9.]*\)$/\1 \2/p')
[ -n "${yaw:-}" ] || { echo "head pose not valid (headset off?)" >&2; exit 1; }
"$0" send "aim $yaw $pitch" && echo "aimed at yaw $yaw pitch $pitch" ;;
log) "$frame" --host "grep -iE 'ft_pointer' ~/.local/share/Steam/logs/vrserver.txt | tail -n ${2:-20}" ;;
*) echo "usage: $0 install|uninstall|send '<cmd>'|aimhere|probe|log" >&2; exit 2 ;;
esac
pointer/helper/build.sh
Executable
+9
View File
@@ -0,0 +1,9 @@
#!/usr/bin/env bash
# Build the ft-pointer helper on the Frame, in the dev container (it also runs there).
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
"$root/scripts/sync.sh" >/dev/null
exec "$root/scripts/frame.sh" -C pointer/helper 'set -e; mkdir -p build
g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -I/opt/steamvr/tools/hellovr_vulkan_linux/src/openvr/headers -I../common \
-o build/ft-pointer ft-pointer.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64 -lpthread
echo "built build/ft-pointer"'
pointer/helper/frametop-pointer.service
+19
View File
@@ -0,0 +1,19 @@
# Template: the installer replaces @REPO@ with the repo path on the Frame.
[Unit]
Description=Frametop pointer helper: the universal 3D mouse (cursor, collision, ft_pointer driver)
Documentation=file://@REPO@/frametop/README.md
# Needs SteamVR's IPC; it starts and stops with SteamVR.
After=steamvr.service frametop-input-relay.service
PartOf=steamvr.service
[Service]
# Runs in the dev container (built there against its libraries). The helper process
# lives in the container, so clean it up explicitly around distrobox enter.
ExecStartPre=-/usr/bin/pkill -x ft-pointer
ExecStart=%h/.local/bin/distrobox enter dev -- @REPO@/frametop/pointer/helper/build/ft-pointer
ExecStopPost=-/usr/bin/pkill -x ft-pointer
Restart=on-failure
RestartSec=3
[Install]
WantedBy=steamvr.service
pointer/helper/ft-pointer.cpp
+946
View File
@@ -0,0 +1,946 @@
// ft-pointer: the universal 3D mouse's brain (OpenVR overlay client, runs in the dev container).
//
// input-relay.py (pointer mode) sends mouse commands here; this program keeps the
// cursor, does collision against SteamVR's overlays, draws the free-space dot, and
// sends the ft_pointer driver the exact pose of its virtual controller.
//
// relay -> @ft_pointer_helper -> ft-pointer -> @ft_pointer -> ft_pointer driver (inside vrserver)
//
// Cursor model:
// - anchor: head position at the last recenter; yaw/pitch: direction from it (mouse-driven).
// - Every frame a ray from the anchor is tested against every visible overlay
// (ComputeOverlayIntersection). On a hit the cursor sits on that surface; otherwise
// it floats `distance` metres out and a small dot overlay is shown there, which the
// laser can hit, so SteamVR never draws a free-flying laser.
// - Then the line of sight from the eye (not the anchor) to that point is tested too:
// after the head moves, something nearer can cover the point, and the cursor goes on
// whatever you see under it (panels close together in view, at different depths).
// - Looks: the compositor ignores live changes to dashboard.laserRayWidthScale (only
// the dashboard's own Settings screen reloads it), so the beam can't be switched
// off per device. Instead the laser starts POINTER_ORIGIN_FRACTION (0.95) of the way
// from the eye to the cursor, along the line of sight: what's left of the beam is a
// few centimetres long and effectively invisible, and SteamVR's hit dot (sized by
// distance from the origin) becomes tiny. Our own white dot is the visible cursor
// everywhere: a non-interactive dot on panels (the laser passes through it), and an
// interactive one in free space (the laser lands on it instead of flying off).
// - The controller ray starts at the eye and aims at the cursor point. Everything here
// is computed in the standing universe; the pose sent to the driver is converted to
// SteamVR's raw tracking space (drivers report raw poses; on the Frame the standing
// origin is ~1.6 m above the raw one, so sending standing coordinates put the laser
// origin 1.6 m above the head). While our device
// owns the dashboard pointer, dashboard.laserRayWidthScale is 0 so only the dot shows.
// It's restored when a controller takes the pointer back.
//
// Last used wins: when a real controller moves (picked up), the pointer is released
// at once (driver "hide", which also drops its hand role hint), so the controller gets
// its role and laser back. The next mouse input reconnects and claims the laser again.
// SteamVR gives a contested hand role to the most recently used device, and a held
// Frame controller counts as used (touch sensors). If our device hasn't got the hand
// role within a second of waking, the pointer is released (no orphan white dot) and
// mouse input can't wake it again for 2 s.
//
// Laser mode: with the dashboard closed, SteamVR keeps its laser mouse off until a
// click (the first click on a panel only turned it on, the second one clicked), and a
// laser that leaves every panel turns it off again. While the pointer is awake, the
// helper shows frametop.pointer.lasermode: a transparent 1 mm overlay 50 m below the
// head with VROverlayFlags_MakeOverlaysInteractiveIfVisible, which keeps SteamVR's
// laser mouse mode on as long as it's visible. It's hidden whenever the pointer is
// released, so controllers and VR games get the normal behaviour back.
//
// Tilt: while the left button is held (dragging a panel by its grab bar, which SteamVR
// moves rigidly with the controller), pressing the right button enters tilt mode. The
// right press is not forwarded; mouse motion then rotates the virtual controller around
// the grab point (horizontal: about the vertical axis, vertical: about the view's
// horizontal axis), so the panel turns around that pivot. The tilt accumulates for the
// whole drag: after the right button is released, the rotation stays applied (about the
// moving cursor point) so the grabbed panel keeps its new orientation, and pressing right
// again continues from it. Releasing the left button drops the panel; the tilted pose (and
// the drag lock) are held 0.5 s longer, because SteamVR's dashboard finishes a floating
// move up to 150 ms after the release (UndockedOverlay.endFloatingWindowMove measures the
// push distance first) and reads the controller pose again then.
//
// Scene-graph overlays: the dashboard's dock (valve.steam.gamepadui.bar) and the controls
// under floating windows (valve.steam.gamepadui.floatingfooter, undock and friends) have
// no texture (0x0) and a placeholder width, so ComputeOverlayIntersection never hits
// them. For those the ray is tested against the overlay's plane, within
// POINTER_SCENE_RADIUS (0.5 m) of its origin; the laser-catching dot sits 5 cm behind the
// plane, so the laser reaches the buttons and still lands on the dot between them.
// Only absolutely placed 0x0 overlays count as scene-graph: gamescope's app panels (the
// desktops) also report 0x0, but they're placed as dashboard tabs, stay up when the
// dashboard closes, and ComputeOverlayIntersection hits them normally.
//
// Panel edges: off a panel, the cursor stays on that panel's plane while it's within
// POINTER_EDGE_REACH (0.3 m) of the last point it touched, instead of jumping to
// POINTER_DISTANCE. A floating panel's resize margins and the window controls under it
// sit just outside the panel, and the laser has to start in front of that plane to reach
// them (a controller's laser always does: it starts at the hand). Like on scene-graph
// planes, the laser-catching dot sits 5 cm behind the plane.
//
// Drag lock: while the left button is held, the cursor keeps the distance it had at the
// press and collision is frozen, so dragging past a panel's edge (resizing, moving)
// doesn't jump the cursor to free space or swap in the laser-catching dot, which made
// SteamVR's resize snap back.
//
// Placement (for layout): SteamVR keeps a floating panel's position inside the
// dashboard, where nothing outside can set it, so the helper carries panels like a user
// would. It measures the panel (md::ScanPanel), aims the device at its grab bar
// (LAYOUT_GRAB_OFFSET, 7.5 cm below the bottom edge; the bands at 2-4 and 14-26 cm are
// other controls), presses, moves, and releases. While grabbed, the panel follows the
// device rigidly, except that the dashboard accelerates fast translations (0.1 m in 0.3 s
// moved it 0.19 m and turned it 8.5 deg, in jerky 25 ms steps right after the press). So
// the device hovers first, and the move is split into a rotation about the device origin
// (the eye) at 60 deg/s and a smooth 60 Hz slide at LAYOUT_SLIDE_SPEED (0.5 m/s; tested
// exact from 0.07 to 1 m/s). Scroll pushes along the panel normal, but only in whole notches of
// about 7 cm, so it isn't used. The result is measured again, and the move repeated up to
// twice while it's more than 1.5 cm or 1 deg off.
//
// Commands (datagrams on @ft_pointer_helper): show, hide, recenter, move <dyaw> <dpitch>,
// reload (re-read the settings below), debug (toggle a twice-a-second state log),
// and btn/scroll lines, which are forwarded to the driver unchanged. For layouts, with a
// reply datagram to the sender's (abstract) address:
// place <overlay> <x> <y> <z> <yaw> <pitch> [roll [grab]]: centre in the standing
// universe; the front faces back along the direction (yaw, pitch), turned by roll
// (counterclockwise as seen, degrees) -> "ok ..." | "error ..."
// measure <overlay> -> "ok cx cy cz width height xx xy xz yx yy yz zx zy zz" (centre,
// size, and the panel's right, up, and front vectors)
// head -> "ok x y z yaw pitch"
// grabprobe <overlay>: log where below the panel SteamVR's laser hits something (to
// find the grab bar again if a SteamVR update moves it)
//
// Settings (~/.config/frametop.conf): POINTER_DISTANCE (m, 1.5), POINTER_CURSOR_DEG
// (angular size of the dot, 0.4), POINTER_LASER_WIDTH (controller beam width to restore, 0.8),
// POINTER_ORIGIN_FRACTION (0.95): the laser starts this far along the eye-to-cursor line,
// but never closer than POINTER_ORIGIN_MARGIN (0.15 m) to the cursor point: SteamVR's
// small controls (undock, frame buttons) float a few centimetres in front of their
// panel, and a laser that starts behind them can't hit them.
#include <openvr.h>
#include "vrmath.h"
#include <algorithm>
#include <atomic>
#include <chrono>
#include <cmath>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <fstream>
#include <map>
#include <mutex>
#include <string>
#include <thread>
#include <tuple>
#include <vector>
#include <sys/socket.h>
#include <sys/un.h>
#include <unistd.h>
namespace {
using namespace md;
std::map<std::string, std::string> ReadConfig() {
std::map<std::string, std::string> conf;
const char *home = std::getenv("HOME");
std::ifstream in(std::string(home ? home : "") + "/.config/frametop.conf");
std::string line;
while (std::getline(in, line)) {
line = line.substr(0, line.find('#'));
const auto eq = line.find('=');
if (eq == std::string::npos) continue;
auto trim = [](std::string s) {
s.erase(0, s.find_first_not_of(" \t"));
s.erase(s.find_last_not_of(" \t") + 1);
return s;
};
conf[trim(line.substr(0, eq))] = trim(line.substr(eq + 1));
}
return conf;
}
double ConfDouble(const std::map<std::string, std::string> &c, const char *key, double fallback) {
auto it = c.find(key);
return it == c.end() ? fallback : std::atof(it->second.c_str());
}
int AbstractSocket(const char *name, bool bindIt) {
const int fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
if (bindIt) {
sockaddr_un addr{};
addr.sun_family = AF_UNIX;
std::memcpy(addr.sun_path + 1, name, std::strlen(name));
const socklen_t len = offsetof(sockaddr_un, sun_path) + 1 + std::strlen(name);
if (bind(fd, reinterpret_cast<sockaddr *>(&addr), len) != 0) {
std::perror("bind @ft_pointer_helper (already running?)");
std::exit(1);
}
}
return fd;
}
void SendTo(int fd, const char *name, const std::string &msg) {
sockaddr_un addr{};
addr.sun_family = AF_UNIX;
std::memcpy(addr.sun_path + 1, name, std::strlen(name));
const socklen_t len = offsetof(sockaddr_un, sun_path) + 1 + std::strlen(name);
sendto(fd, msg.data(), msg.size(), 0, reinterpret_cast<sockaddr *>(&addr), len);
}
// Overlay keys, refreshed in the background from `vrcmd --overlays` (OpenVR has no
// public call to enumerate other apps' overlays). Hidden ones are listed too: the
// window controls under a floating panel only appear while something hovers the
// panel, and the cursor has to find them the moment they do, not a second later.
class OverlayList {
public:
void Start() {
thread_ = std::thread([this] {
while (running_) {
Refresh();
std::this_thread::sleep_for(std::chrono::seconds(1));
}
});
}
void Stop() {
running_ = false;
if (thread_.joinable()) thread_.join();
}
std::vector<std::string> Keys() {
std::lock_guard<std::mutex> guard(lock_);
return keys_;
}
private:
void Refresh() {
FILE *p = popen("LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrcmd --overlays 2>/dev/null", "r");
if (!p) return;
std::vector<std::string> keys;
char line[1024];
while (std::fgets(line, sizeof line, p)) {
// 'key' -- 'name', WxH visible VROverlayType_...
if (line[0] != '\'') continue;
const char *end = std::strchr(line + 1, '\'');
if (!end) continue;
const std::string key(line + 1, size_t(end - (line + 1)));
const std::string rest(end);
if (rest.find("Thumbnail") != std::string::npos || rest.find("Subview") != std::string::npos) continue;
if (key.rfind("system.pointer", 0) == 0 || key.rfind("system.cursor", 0) == 0 ||
key.rfind("frametop.pointer", 0) == 0 || key.rfind("frametop.guide", 0) == 0 ||
key == "system.HeadsetView" || key == "system.toast")
continue;
keys.push_back(key);
}
pclose(p);
std::lock_guard<std::mutex> guard(lock_);
keys_ = std::move(keys);
}
std::thread thread_;
std::atomic<bool> running_{true};
std::mutex lock_;
std::vector<std::string> keys_;
};
vr::HmdMatrix34_t Billboard(Vec3 at, Vec3 eye) {
// Overlay faces +Z; point +Z at the eye, keep +Y roughly up.
const Vec3 z = Normalize(eye - at);
const Vec3 x = Normalize(Cross({0, 1, 0}, z));
const Vec3 y = Cross(z, x);
vr::HmdMatrix34_t m{};
const Vec3 cols[3] = {x, y, z};
for (int c = 0; c < 3; ++c) {
m.m[0][c] = float(cols[c].x);
m.m[1][c] = float(cols[c].y);
m.m[2][c] = float(cols[c].z);
}
m.m[0][3] = float(at.x);
m.m[1][3] = float(at.y);
m.m[2][3] = float(at.z);
return m;
}
std::vector<uint8_t> DotTexture(int size) {
// White dot with a dark rim, soft edge, transparent outside.
std::vector<uint8_t> px(size * size * 4, 0);
const double c = (size - 1) / 2.0, r = size * 0.42, rim = size * 0.10;
for (int y = 0; y < size; ++y)
for (int x = 0; x < size; ++x) {
const double d = std::hypot(x - c, y - c);
const double a = std::clamp(r - d + 0.5, 0.0, 1.0);
const bool inner = d < r - rim;
uint8_t *p = &px[(y * size + x) * 4];
const uint8_t v = inner ? 255 : 40;
p[0] = p[1] = p[2] = v;
p[3] = uint8_t(a * 235);
}
return px;
}
} // namespace
// Placement speeds (see "Placement" at the top).
constexpr double kPlaceDegPerSec = 60; // tested: 40 deg/s is applied exactly
int main() {
double freeDistance = 1.5, cursorDeg = 0.4, originFraction = 0.95, originMargin = 0.15, sceneRadius = 0.5,
edgeReach = 0.3, grabOffset = 0.075,
slideSpeed = 0.5;
auto loadConfig = [&] {
const auto conf = ReadConfig();
freeDistance = std::clamp(ConfDouble(conf, "POINTER_DISTANCE", 1.5), 0.3, 10.0);
cursorDeg = std::clamp(ConfDouble(conf, "POINTER_CURSOR_DEG", 0.4), 0.05, 5.0);
originFraction = std::clamp(ConfDouble(conf, "POINTER_ORIGIN_FRACTION", 0.95), 0.0, 0.98);
originMargin = std::clamp(ConfDouble(conf, "POINTER_ORIGIN_MARGIN", 0.15), 0.0, 1.0);
sceneRadius = std::clamp(ConfDouble(conf, "POINTER_SCENE_RADIUS", 0.5), 0.05, 2.0);
edgeReach = std::clamp(ConfDouble(conf, "POINTER_EDGE_REACH", 0.3), 0.0, 2.0);
grabOffset = std::clamp(ConfDouble(conf, "LAYOUT_GRAB_OFFSET", 0.075), 0.0, 1.0);
slideSpeed = std::clamp(ConfDouble(conf, "LAYOUT_SLIDE_SPEED", 0.5), 0.02, 2.0);
};
loadConfig();
const float laserWidth = float(ConfDouble(ReadConfig(), "POINTER_LASER_WIDTH", 0.8));
vr::EVRInitError err = vr::VRInitError_None;
while (true) {
vr::VR_Init(&err, vr::VRApplication_Overlay);
if (err == vr::VRInitError_None) break;
std::fprintf(stderr, "waiting for SteamVR: %s\n", vr::VR_GetVRInitErrorAsEnglishDescription(err));
std::this_thread::sleep_for(std::chrono::seconds(2));
}
auto *sys = vr::VRSystem();
auto *overlay = vr::VROverlay();
vr::VROverlayHandle_t cursor = vr::k_ulOverlayHandleInvalid;
overlay->CreateOverlay("frametop.pointer.cursor", "Frametop pointer", &cursor);
const int texSize = 64;
auto tex = DotTexture(texSize);
overlay->SetOverlayRaw(cursor, tex.data(), texSize, texSize, 4);
overlay->SetOverlayInputMethod(cursor, vr::VROverlayInputMethod_Mouse); // the laser can land on it
overlay->SetOverlaySortOrder(cursor, 200);
// Same dot, not interactive, drawn on panels at the hit point; the laser passes through.
vr::VROverlayHandle_t marker = vr::k_ulOverlayHandleInvalid;
overlay->CreateOverlay("frametop.pointer.marker", "Frametop pointer marker", &marker);
overlay->SetOverlayRaw(marker, tex.data(), texSize, texSize, 4);
overlay->SetOverlayInputMethod(marker, vr::VROverlayInputMethod_None);
overlay->SetOverlaySortOrder(marker, 201);
// Laser mode (see the top of the file).
vr::VROverlayHandle_t laserMode = vr::k_ulOverlayHandleInvalid;
overlay->CreateOverlay("frametop.pointer.lasermode", "Frametop pointer laser mode", &laserMode);
std::vector<uint8_t> clear(4 * 4 * 4, 0);
overlay->SetOverlayRaw(laserMode, clear.data(), 4, 4, 4);
overlay->SetOverlayWidthInMeters(laserMode, 0.001f);
overlay->SetOverlayInputMethod(laserMode, vr::VROverlayInputMethod_Mouse); // the flag needs an input method
overlay->SetOverlayFlag(laserMode, vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible, true);
vr::HmdMatrix34_t below{};
below.m[0][0] = below.m[1][1] = below.m[2][2] = 1;
below.m[1][3] = -50;
overlay->SetOverlayTransformTrackedDeviceRelative(laserMode, vr::k_unTrackedDeviceIndex_Hmd, &below);
bool laserModeShown = false;
// Controller beams keep the user's width; nothing here changes it any more.
vr::VRSettings()->SetFloat("dashboard", "laserRayWidthScale", laserWidth);
const int in = AbstractSocket("ft_pointer_helper", true);
const int out = AbstractSocket(nullptr, false);
OverlayList overlays;
overlays.Start();
std::map<std::string, vr::VROverlayHandle_t> handles;
std::map<std::string, bool> sceneGraph; // no texture: plane test instead of ComputeOverlayIntersection
std::map<std::string, bool> visible; // refreshed every 50 ms
auto lastVisible = std::chrono::steady_clock::now();
// The plane of the last panel the cursor was on, and the last point on it (panel edges).
Vec3 edgePoint, edgeNormal, edgeLast;
std::string edgeKey;
bool active = false, recenter = false, anchored = false;
using Clock = std::chrono::steady_clock;
Clock::time_point lastMouse{}, claimAt{}, claimRelease{}, wokeAt{}, noWakeUntil{};
bool claimPending = false, claimHeld = false;
// Tilt mode (see top of file).
bool leftHeld = false, tilting = false, tiltStart = false, swallowedRight = false;
double tiltYaw = 0, tiltPitch = 0;
double dragDistance = 0, lastDistance = 1.5; // drag lock: distance from the anchor at the press
Clock::time_point dropHoldUntil{}; // after a left release: keep the drag pose this long
bool debug = false;
std::string lastHit;
auto lastDebug = Clock::now();
vr::VROverlayHandle_t systemPointer = vr::k_ulOverlayHandleInvalid;
overlay->FindOverlay("system.pointer", &systemPointer);
Vec3 pivot, tiltOrigin, lastPoint, lastOrigin, lastAim{0, 0, -1};
Basis tiltBasis{};
auto wake = [&](Clock::time_point t) {
if (t < noWakeUntil) return;
wokeAt = t;
active = true;
recenter = true;
SendTo(out, "ft_pointer", "show");
claimPending = true; // take the laser without clicking, once SteamVR has bound the device
claimAt = t + std::chrono::milliseconds(300);
};
Vec3 anchor;
double yaw = 0, pitch = 0;
vr::TrackedDeviceIndex_t ours = vr::k_unTrackedDeviceIndexInvalid;
auto lastSlow = std::chrono::steady_clock::now() - std::chrono::seconds(10);
// --- Panel placement (see "Placement" at the top of the file) ---
// Device pose, given in the standing universe, sent to the driver in raw space.
auto sendPose = [&](Vec3 originStanding, const Basis &b) {
vr::TrackedDevicePose_t s, r;
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &s, 1);
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseRawAndUncalibrated, 0, &r, 1);
const auto &S = s.mDeviceToAbsoluteTracking, &R = r.mDeviceToAbsoluteTracking;
auto toRaw = [&](Vec3 v) { return Rotate(R, RotateInverse(S, v)); };
const Vec3 o = Position(R) + toRaw(originStanding - Position(S));
double q[4];
BasisQuat({toRaw(b.x), toRaw(b.y), toRaw(b.z)}, q);
char msg[200];
std::snprintf(msg, sizeof msg, "posq %.5f %.5f %.5f %.6f %.6f %.6f %.6f", o.x, o.y, o.z, q[0], q[1], q[2], q[3]);
SendTo(out, "ft_pointer", msg);
};
auto sleepMs = [](int ms) { std::this_thread::sleep_for(std::chrono::milliseconds(ms)); };
auto headPos = [&] {
vr::TrackedDevicePose_t s;
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &s, 1);
return std::make_pair(s.bPoseIsValid, Position(s.mDeviceToAbsoluteTracking));
};
// Borrow the device and the laser for a placement: connect, claim, laser mode on.
auto borrow = [&] {
SendTo(out, "ft_pointer", "show");
overlay->ShowOverlay(laserMode);
overlay->HideOverlay(cursor);
overlay->HideOverlay(marker);
sleepMs(active ? 50 : 400); // a fresh connect needs SteamVR to bind the device
SendTo(out, "ft_pointer", "btn a 1");
sleepMs(60);
SendTo(out, "ft_pointer", "btn a 0");
};
auto giveBack = [&] {
if (!active) {
SendTo(out, "ft_pointer", "hide");
overlay->HideOverlay(laserMode);
laserModeShown = false;
}
};
auto findPanel = [&](const char *key, Panel &p, Vec3 &eye) -> std::string {
vr::VROverlayHandle_t h;
if (overlay->FindOverlay(key, &h) != vr::VROverlayError_None) return std::string("no overlay ") + key;
bool valid;
std::tie(valid, eye) = headPos();
if (!valid) return "no head pose (headset off?)";
p = ScanPanel(h, eye, 0.5);
if (!p.found) return std::string("panel not visible: ") + key;
return "";
};
// Spike: aim down from the panel's bottom edge, 1 cm a step, and log where SteamVR's
// laser hits something (the hit dot shows) and whether the window controls are up.
auto grabProbe = [&](const char *key) {
Panel p;
Vec3 eye;
const std::string err = findPanel(key, p, eye);
if (!err.empty()) {
std::printf("grabprobe: %s\n", err.c_str());
std::fflush(stdout);
return;
}
borrow();
vr::VROverlayHandle_t footer = vr::k_ulOverlayHandleInvalid;
overlay->FindOverlay("valve.steam.gamepadui.floatingfooter", &footer);
const Vec3 bottom = p.center - p.basis.y * (p.height / 2);
std::printf("grabprobe %s: center (%.3f %.3f %.3f) %.3f x %.3f m\n", key, p.center.x, p.center.y, p.center.z,
p.width, p.height);
for (int cm = -5; cm <= 45; ++cm) {
const Vec3 target = bottom - p.basis.y * (cm / 100.0);
sendPose(eye, AimBasis(target - eye));
sleepMs(90);
const bool dot = systemPointer != vr::k_ulOverlayHandleInvalid && overlay->IsOverlayVisible(systemPointer);
const bool foot = footer != vr::k_ulOverlayHandleInvalid && overlay->IsOverlayVisible(footer);
std::printf(" %+3d cm below the bottom edge: steamvr_dot=%d footer=%d", cm, dot, foot);
if (foot) {
vr::ETrackingUniverseOrigin uo;
vr::HmdMatrix34_t t{};
if (overlay->GetOverlayTransformAbsolute(footer, &uo, &t) == vr::VROverlayError_None) {
const Vec3 f = Position(t) - p.center;
std::printf(" footer at panel (%.3f %.3f %.3f)", Dot(f, p.basis.x), Dot(f, p.basis.y),
Dot(f, p.basis.z));
}
}
std::printf("\n");
}
std::fflush(stdout);
giveBack();
};
// Carry a floating panel so its centre lands on `target` with frame `bt` (see
// "Placement" at the top). Returns "ok ..." or "error ...".
auto place = [&](const char *key, Vec3 target, const Basis &bt, double grabBelow) -> std::string {
Panel p;
Vec3 eye;
std::string err = findPanel(key, p, eye);
if (!err.empty()) return "error " + err;
auto offBy = [&](const Panel &q, double &cm, double &deg) {
cm = Length(q.center - target) * 100;
const double c = (Dot(q.basis.x, bt.x) + Dot(q.basis.y, bt.y) + Dot(q.basis.z, bt.z) - 1) / 2;
deg = std::acos(std::clamp(c, -1.0, 1.0)) * 180 / M_PI;
};
double cm, deg;
int moves = 0;
borrow();
for (int attempt = 0; attempt < 3; ++attempt) {
offBy(p, cm, deg);
if (cm < 1.5 && deg < 1.0) break;
// The rigid motion that takes the panel to the target: rotate by R, then move.
auto turn = [&](Vec3 v) { return FromBasis(bt, ToBasis(p.basis, v)); };
double q[4];
BasisQuat({turn({1, 0, 0}), turn({0, 1, 0}), turn({0, 0, 1})}, q);
const double angle = 2 * std::acos(std::clamp(q[0], -1.0, 1.0));
const Vec3 axis = std::sin(angle / 2) > 1e-6 ? Normalize({q[1], q[2], q[3]}) : Vec3{0, 1, 0};
const Vec3 grab = p.center - p.basis.y * (p.height / 2 + grabBelow);
const Basis d0 = AimBasis(grab - eye);
const Vec3 o1 = target + turn(eye - p.center); // device origin at the end
++moves;
sendPose(eye, d0);
sleepMs(150); // hover: the window controls come up
SendTo(out, "ft_pointer", "btn trigger 1");
sleepMs(150);
// 1. Rotate about the device origin (the eye): the dashboard applies it exactly.
const int rsteps = std::max(4, int(angle * 180 / M_PI / kPlaceDegPerSec * 60));
for (int i = 1; i <= rsteps; ++i) {
const double a = angle * i / rsteps;
auto r = [&](Vec3 v) { return RotateAbout(v, axis, a); };
sendPose(eye, {r(d0.x), r(d0.y), r(d0.z)});
sleepMs(16);
}
// 2. Slide the device slowly: fast moves are accelerated by the dashboard.
const Basis d1{turn(d0.x), turn(d0.y), turn(d0.z)};
const int tsteps = std::max(4, int(Length(o1 - eye) / slideSpeed * 60));
for (int i = 1; i <= tsteps; ++i) {
sendPose(eye + (o1 - eye) * (double(i) / tsteps), d1);
sleepMs(16);
}
sleepMs(100);
SendTo(out, "ft_pointer", "btn trigger 0");
sleepMs(600); // the dashboard re-reads the pose up to 150 ms after the release
err = findPanel(key, p, eye);
if (!err.empty()) break;
}
giveBack();
if (!err.empty()) return "error after the move: " + err;
offBy(p, cm, deg);
char msg[160];
std::snprintf(msg, sizeof msg, "ok %s off by %.1f cm, %.1f deg after %d move%s", key, cm, deg, moves,
moves == 1 ? "" : "s");
std::printf("place: %s\n", msg);
std::fflush(stdout);
return msg;
};
std::printf("ft-pointer running: free distance %.2f m, dot %.2f deg\n", freeDistance, cursorDeg);
std::fflush(stdout);
while (true) {
// Commands from the relay.
char buf[256];
ssize_t n;
sockaddr_un from{};
socklen_t fromLen = sizeof from;
while ((n = recvfrom(in, buf, sizeof buf - 1, 0, reinterpret_cast<sockaddr *>(&from), &fromLen)) > 0) {
buf[n] = 0;
// Reply to the sender (the layout tool binds an abstract address to get answers).
const sockaddr_un sender = from;
const socklen_t senderLen = fromLen;
fromLen = sizeof from;
auto reply = [&](const std::string &msg) {
if (senderLen > offsetof(sockaddr_un, sun_path))
sendto(out, msg.data(), msg.size(), 0, reinterpret_cast<const sockaddr *>(&sender), senderLen);
};
const bool mouseInput = std::strncmp(buf, "move", 4) == 0 || std::strncmp(buf, "btn", 3) == 0 ||
std::strncmp(buf, "scroll", 6) == 0;
if (mouseInput) lastMouse = Clock::now();
// Any mouse input wakes the pointer (after a controller took over, or a helper restart).
if (!active && mouseInput) wake(Clock::now());
double a, b;
char key[128];
double px, py, pz, pyaw, ppitch, proll = 0, pgrab = -1;
if (std::sscanf(buf, "grabprobe %127s", key) == 1) {
grabProbe(key);
continue;
}
if (std::sscanf(buf, "place %127s %lf %lf %lf %lf %lf %lf %lf", key, &px, &py, &pz, &pyaw, &ppitch, &proll,
&pgrab) >= 6) {
reply(place(key, {px, py, pz}, PanelBasis(pyaw, ppitch, proll), pgrab >= 0 ? pgrab : grabOffset));
continue;
}
if (std::sscanf(buf, "measure %127s", key) == 1) {
Panel p;
Vec3 eye;
const std::string err = findPanel(key, p, eye);
char msg[400] = "";
if (err.empty())
std::snprintf(msg, sizeof msg, "ok %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f %.4f",
p.center.x, p.center.y, p.center.z, p.width, p.height, p.basis.x.x, p.basis.x.y,
p.basis.x.z, p.basis.y.x, p.basis.y.y, p.basis.y.z, p.basis.z.x, p.basis.z.y,
p.basis.z.z);
reply(err.empty() ? msg : "error " + err);
continue;
}
if (std::strncmp(buf, "head", 4) == 0) {
vr::TrackedDevicePose_t h;
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &h, 1);
const Vec3 e = Position(h.mDeviceToAbsoluteTracking);
const Vec3 f = Rotate(h.mDeviceToAbsoluteTracking, {0, 0, -1});
char msg[200];
std::snprintf(msg, sizeof msg, "ok %.4f %.4f %.4f %.2f %.2f", e.x, e.y, e.z,
std::atan2(-f.x, -f.z) * 180 / M_PI, std::asin(std::clamp(f.y, -1.0, 1.0)) * 180 / M_PI);
reply(h.bPoseIsValid ? msg : "error no head pose (headset off?)");
continue;
}
if (std::strncmp(buf, "debug", 5) == 0) {
debug = !debug;
std::printf("debug %s\n", debug ? "on" : "off");
std::fflush(stdout);
continue;
}
if (std::strncmp(buf, "btn trigger 1", 13) == 0) {
leftHeld = true;
dragDistance = lastDistance;
tiltYaw = tiltPitch = 0; // a new drag starts untilted
dropHoldUntil = {};
} else if (std::strncmp(buf, "btn trigger 0", 13) == 0) {
leftHeld = false;
tilting = false;
// Hold the drag pose (tilt, frozen distance) while SteamVR finishes the drop.
dropHoldUntil = Clock::now() + std::chrono::milliseconds(500);
} else if (std::strncmp(buf, "btn b 1", 7) == 0 && leftHeld) {
tilting = tiltStart = swallowedRight = true; // right press while dragging: tilt, no right-click
continue;
} else if (std::strncmp(buf, "btn b 0", 7) == 0 && swallowedRight) {
tilting = swallowedRight = false;
continue;
}
if (tilting && std::sscanf(buf, "move %lf %lf", &a, &b) == 2) {
tiltYaw += a;
tiltPitch = std::clamp(tiltPitch + b, -80.0, 80.0);
continue;
}
if (std::sscanf(buf, "move %lf %lf", &a, &b) == 2) {
if (!anchored) recenter = true;
yaw += a;
while (yaw > 180) yaw -= 360;
while (yaw < -180) yaw += 360;
pitch = std::clamp(pitch + b, -85.0, 85.0);
} else if (std::strncmp(buf, "recenter", 8) == 0) {
recenter = true;
} else if (std::strncmp(buf, "reload", 6) == 0) {
loadConfig();
std::printf("reloaded: free distance %.2f m, dot %.2f deg, origin %.2f\n", freeDistance, cursorDeg,
originFraction);
std::fflush(stdout);
} else if (std::strncmp(buf, "show", 4) == 0) {
if (!active) wake(Clock::now());
} else if (std::strncmp(buf, "hide", 4) == 0) {
active = false;
overlay->HideOverlay(cursor);
overlay->HideOverlay(marker);
SendTo(out, "ft_pointer", "hide");
} else {
SendTo(out, "ft_pointer", buf); // btn, scroll
}
}
vr::TrackedDevicePose_t all[vr::k_unMaxTrackedDeviceCount];
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0.011f, all, vr::k_unMaxTrackedDeviceCount);
const vr::TrackedDevicePose_t &hmd = all[0];
vr::TrackedDevicePose_t hmdRaw;
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseRawAndUncalibrated, 0.011f, &hmdRaw, 1);
const auto tnow = Clock::now();
// Claim pulse (switchlaserhand on the driver's "a" button, no click).
if (claimPending && tnow >= claimAt) {
SendTo(out, "ft_pointer", "btn a 1");
claimPending = false;
claimHeld = true;
claimRelease = tnow + std::chrono::milliseconds(60);
} else if (claimHeld && tnow >= claimRelease) {
SendTo(out, "ft_pointer", "btn a 0");
claimHeld = false;
}
// Laser mode on while the pointer is awake.
if (active != laserModeShown) {
laserModeShown = active;
if (active) overlay->ShowOverlay(laserMode);
else overlay->HideOverlay(laserMode);
if (debug) std::printf("laser mode %s\n", active ? "forced on" : "released");
if (debug) std::fflush(stdout);
}
// Didn't get the hand role (a held controller keeps it): release, back off.
if (active && tnow - wokeAt > std::chrono::seconds(1) && ours != vr::k_unTrackedDeviceIndexInvalid &&
sys->GetControllerRoleForTrackedDeviceIndex(ours) == vr::TrackedControllerRole_Invalid) {
active = false;
claimPending = claimHeld = false;
overlay->HideOverlay(cursor);
overlay->HideOverlay(marker);
SendTo(out, "ft_pointer", "btn a 0");
SendTo(out, "ft_pointer", "hide");
noWakeUntil = tnow + std::chrono::seconds(2);
std::printf("no hand role (a controller is in use): pointer released\n");
std::fflush(stdout);
}
// Last used wins: a real controller being moved releases the pointer.
if (active && tnow - lastMouse > std::chrono::milliseconds(500)) {
for (vr::TrackedDeviceIndex_t i = 1; i < vr::k_unMaxTrackedDeviceCount; ++i) {
if (i == ours || !all[i].bPoseIsValid) continue;
if (sys->GetTrackedDeviceClass(i) != vr::TrackedDeviceClass_Controller) continue;
const auto &v = all[i].vVelocity.v, &w = all[i].vAngularVelocity.v;
const double speed = std::sqrt(v[0] * v[0] + v[1] * v[1] + v[2] * v[2]);
const double spin = std::sqrt(w[0] * w[0] + w[1] * w[1] + w[2] * w[2]);
if (speed > 0.35 || spin > 2.0) {
active = false;
claimPending = claimHeld = false;
overlay->HideOverlay(cursor);
overlay->HideOverlay(marker);
SendTo(out, "ft_pointer", "btn a 0");
SendTo(out, "ft_pointer", "hide");
std::printf("controller %u moved: pointer released\n", i);
std::fflush(stdout);
break;
}
}
}
const auto &hm = hmd.mDeviceToAbsoluteTracking.m;
const Vec3 eye{hm[0][3], hm[1][3], hm[2][3]};
if (recenter && hmd.bPoseIsValid) {
anchor = eye;
const Vec3 f{-hm[0][2], -hm[1][2], -hm[2][2]};
yaw = std::atan2(-f.x, -f.z) * 180 / M_PI;
pitch = std::asin(std::clamp(f.y, -1.0, 1.0)) * 180 / M_PI;
anchored = true;
recenter = false;
}
// Slow work, once a second: overlay handles, our device index, laser width.
const auto now = std::chrono::steady_clock::now();
if (now - lastSlow > std::chrono::seconds(1)) {
lastSlow = now;
handles.clear();
for (const auto &key : overlays.Keys()) {
vr::VROverlayHandle_t h;
if (overlay->FindOverlay(key.c_str(), &h) != vr::VROverlayError_None) continue;
handles[key] = h;
// Scene-graph overlays are placed absolutely. Other overlays can report no
// texture too (gamescope's app panels, placed as dashboard tabs, share theirs
// from another process), and ComputeOverlayIntersection handles those.
uint32_t tw = 0, th = 0;
overlay->GetOverlayTextureSize(h, &tw, &th);
vr::VROverlayTransformType tt = vr::VROverlayTransform_Invalid;
overlay->GetOverlayTransformType(h, &tt);
// ft-screens' panels (frametop.screen.N) are 0x0 and absolute too (a shared
// texture), but they're real panels of any size.
sceneGraph[key] = (tw == 0 || th == 0) && tt == vr::VROverlayTransform_Absolute &&
key.rfind("frametop.screen.", 0) != 0;
}
ours = vr::k_unTrackedDeviceIndexInvalid;
for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) {
char type[64] = "";
sys->GetStringTrackedDeviceProperty(i, vr::Prop_ControllerType_String, type, sizeof type);
if (std::strcmp(type, "ft_pointer") == 0) ours = i;
}
}
if (now - lastVisible > std::chrono::milliseconds(50) || visible.size() != handles.size()) {
lastVisible = now;
visible.clear();
for (const auto &[key, h] : handles) visible[key] = overlay->IsOverlayVisible(h);
}
if (active && anchored && hmd.bPoseIsValid && tilting) {
// Rotate the device around the grab point; the grabbed panel turns with it.
if (tiltStart) {
pivot = lastPoint;
tiltOrigin = lastOrigin;
tiltBasis = AimBasis(lastAim);
tiltStart = false; // angles carry on from any earlier tilt in this drag
overlay->HideOverlay(cursor);
overlay->HideOverlay(marker);
}
const double yr = tiltYaw * M_PI / 180, pr = tiltPitch * M_PI / 180;
const Vec3 up{0, 1, 0}, side = tiltBasis.x;
auto turn = [&](Vec3 v) { return RotateAbout(RotateAbout(v, side, pr), up, yr); };
const Vec3 originStanding = pivot + turn(tiltOrigin - pivot);
const Basis b{turn(tiltBasis.x), turn(tiltBasis.y), turn(tiltBasis.z)};
const auto &S = hmd.mDeviceToAbsoluteTracking, &R = hmdRaw.mDeviceToAbsoluteTracking;
auto toRaw = [&](Vec3 v) { return Rotate(R, RotateInverse(S, v)); }; // directions: raw <- standing
const Vec3 originRaw = Position(R) + toRaw(originStanding - eye);
double q[4];
BasisQuat({toRaw(b.x), toRaw(b.y), toRaw(b.z)}, q);
char msg[200];
std::snprintf(msg, sizeof msg, "posq %.5f %.5f %.5f %.6f %.6f %.6f %.6f", originRaw.x, originRaw.y,
originRaw.z, q[0], q[1], q[2], q[3]);
SendTo(out, "ft_pointer", msg);
} else if (active && anchored && hmd.bPoseIsValid) {
const bool dragging = leftHeld || tnow < dropHoldUntil;
if (!dragging) tiltYaw = tiltPitch = 0; // drop finished: back to plain pointing
const Vec3 dir = Direction(yaw, pitch);
// Nearest visible overlay along a ray (frozen while dragging).
struct Hit {
double along = 1e9;
std::string key;
bool scene = false;
Vec3 point, normal;
};
auto nearest = [&](Vec3 from, Vec3 d) {
Hit h;
for (const auto &[key, handle] : handles) {
if (!visible[key]) continue;
if (sceneGraph[key]) {
// Plane test: overlay origin and its +Z normal, within sceneRadius of the origin.
vr::ETrackingUniverseOrigin uo;
vr::HmdMatrix34_t t{};
if (overlay->GetOverlayTransformAbsolute(handle, &uo, &t) != vr::VROverlayError_None) continue;
const Vec3 center = Position(t), normal{t.m[0][2], t.m[1][2], t.m[2][2]};
const double denom = Dot(d, normal);
if (std::fabs(denom) < 1e-4) continue;
const double along = Dot(center - from, normal) / denom;
const Vec3 at = from + d * along;
if (along > 0.05 && along < h.along && std::sqrt(Dot(at - center, at - center)) <= sceneRadius)
h.along = along, h.key = key, h.scene = true, h.point = at, h.normal = normal;
continue;
}
vr::VROverlayIntersectionParams_t params{};
params.vSource = {float(from.x), float(from.y), float(from.z)};
params.vDirection = {float(d.x), float(d.y), float(d.z)};
params.eOrigin = vr::TrackingUniverseStanding;
vr::VROverlayIntersectionResults_t r{};
if (overlay->ComputeOverlayIntersection(handle, &params, &r) && r.fDistance > 0.05f &&
r.fDistance < h.along) {
h.along = r.fDistance, h.key = key, h.scene = false;
h.point = {r.vPoint.v[0], r.vPoint.v[1], r.vPoint.v[2]};
h.normal = {r.vNormal.v[0], r.vNormal.v[1], r.vNormal.v[2]};
}
}
return h;
};
Hit first;
if (!dragging) first = nearest(anchor, dir);
double best = first.along;
std::string bestKey = first.key;
bool bestScene = first.scene;
Vec3 bestPoint = first.point, bestNormal = first.normal;
bool onEdge = false;
if (!dragging && best < 1e8 && !bestScene) {
edgeKey = bestKey, edgePoint = bestPoint, edgeNormal = Normalize(bestNormal), edgeLast = bestPoint;
} else if (!dragging && best >= 1e8 && !edgeKey.empty() && visible[edgeKey]) {
// Just off a panel: stay on its plane (see "Panel edges" at the top).
const double denom = Dot(dir, edgeNormal);
if (std::fabs(denom) > 1e-4) {
const double along = Dot(edgePoint - anchor, edgeNormal) / denom;
const Vec3 at = anchor + dir * along;
if (along > 0.05 && std::sqrt(Dot(at - edgeLast, at - edgeLast)) <= edgeReach) {
best = along;
bestKey = edgeKey;
onEdge = true;
}
}
}
// While dragging: keep the press-time distance and show the non-interactive marker.
// On a scene-graph plane or a panel's edge: the laser-catching dot goes 5 cm behind it.
double distance = dragging ? dragDistance : (best < 1e8 ? best : freeDistance);
Vec3 point = anchor + dir * distance;
bool occluded = false;
if (!dragging) {
// The cursor lands on what you see under it: the ray above starts at the anchor,
// not the eye, so after leaning it can pick a panel that something nearer
// covers from where you are now (panels close together in view, at different
// depths). Anything in front of the point on the eye's line of sight wins.
const double toPoint = std::sqrt(Dot(point - eye, point - eye));
const Hit front = nearest(eye, Normalize(point - eye));
if (front.along < toPoint - 0.02) {
occluded = true;
bestKey = front.key, bestScene = front.scene;
point = front.point;
if (!front.scene) edgeKey = front.key, edgePoint = front.point, edgeNormal = Normalize(front.normal),
edgeLast = front.point;
distance = std::sqrt(Dot(point - anchor, point - anchor));
best = distance;
onEdge = false;
}
lastDistance = distance, lastHit = bestKey;
}
const bool onScene = !dragging && ((bestScene && best < 1e8) || onEdge);
const bool onPanel = dragging || (best < 1e8 && !onScene);
{
// On a panel: the non-interactive marker, pulled 5 mm toward the eye so it
// draws on top. In free space: the interactive dot the laser lands on.
const vr::VROverlayHandle_t show = onPanel ? marker : cursor, hide = onPanel ? cursor : marker;
const Vec3 at = onPanel ? point + Normalize(eye - point) * 0.005 : onScene ? point + dir * 0.05 : point;
const double dist = std::sqrt(Dot(at - eye, at - eye));
overlay->SetOverlayWidthInMeters(show, float(2 * dist * std::tan(cursorDeg * M_PI / 360)));
auto m = Billboard(at, eye);
overlay->SetOverlayTransformAbsolute(show, vr::TrackingUniverseStanding, &m);
overlay->ShowOverlay(show);
overlay->HideOverlay(hide);
}
// Controller ray: from the eye, aimed at the cursor point, converted from the
// standing universe to raw tracking space via the HMD's pose in both.
const Vec3 aimStanding = Normalize(point - eye);
const auto &S = hmd.mDeviceToAbsoluteTracking, &R = hmdRaw.mDeviceToAbsoluteTracking;
const Vec3 aim = Rotate(R, RotateInverse(S, aimStanding)); // raw <- head <- standing
// Origin partway along the line of sight to the cursor (smaller hit dot).
const double toPoint = std::sqrt(Dot(point - eye, point - eye));
const double originDist = std::max(0.0, std::min(toPoint * originFraction, toPoint - originMargin));
const Vec3 originStanding = eye + Normalize(point - eye) * originDist;
const Vec3 eyeRaw = Position(R) + Rotate(R, RotateInverse(S, originStanding - eye));
lastPoint = point, lastOrigin = originStanding, lastAim = aimStanding; // tilt starts from here
if (debug && tnow - lastDebug > std::chrono::milliseconds(500)) {
lastDebug = tnow;
if (systemPointer == vr::k_ulOverlayHandleInvalid) overlay->FindOverlay("system.pointer", &systemPointer);
std::printf("dbg %s hit=%s dist=%.2f eye->point=%.2f origin=%.2f yaw=%.1f pitch=%.1f steamvr_dot=%d primary=%u\n",
dragging ? "DRAG" : occluded ? "INFRONT" : onEdge ? "EDGE" : onScene ? "SCENE" : (best < 1e8 ? "PANEL" : "FREE"), lastHit.empty() ? "-" : lastHit.c_str(),
distance, toPoint, originDist, yaw, pitch,
systemPointer != vr::k_ulOverlayHandleInvalid && overlay->IsOverlayVisible(systemPointer),
overlay->GetPrimaryDashboardDevice());
std::fflush(stdout);
}
const double ayaw = std::atan2(-aim.x, -aim.z) * 180 / M_PI;
const double apitch = std::asin(std::clamp(aim.y, -1.0, 1.0)) * 180 / M_PI;
if (dragging && (tiltYaw != 0 || tiltPitch != 0)) {
// Keep this drag's tilt applied, about the current cursor point.
const double yr = tiltYaw * M_PI / 180, pr = tiltPitch * M_PI / 180;
const Basis base = AimBasis(aimStanding);
const Vec3 up{0, 1, 0}, side = base.x;
auto turn = [&](Vec3 v) { return RotateAbout(RotateAbout(v, side, pr), up, yr); };
const Vec3 o = point + turn(originStanding - point);
const Basis b{turn(base.x), turn(base.y), turn(base.z)};
auto toRaw = [&](Vec3 v) { return Rotate(R, RotateInverse(S, v)); };
const Vec3 oRaw = Position(R) + toRaw(o - eye);
double q[4];
BasisQuat({toRaw(b.x), toRaw(b.y), toRaw(b.z)}, q);
char msg[200];
std::snprintf(msg, sizeof msg, "posq %.5f %.5f %.5f %.6f %.6f %.6f %.6f", oRaw.x, oRaw.y, oRaw.z, q[0],
q[1], q[2], q[3]);
SendTo(out, "ft_pointer", msg);
} else {
char msg[160];
std::snprintf(msg, sizeof msg, "pose %.5f %.5f %.5f %.4f %.4f", eyeRaw.x, eyeRaw.y, eyeRaw.z, ayaw, apitch);
SendTo(out, "ft_pointer", msg);
}
}
vr::VREvent_t ev;
while (sys->PollNextEvent(&ev, sizeof ev)) {
if (ev.eventType == vr::VREvent_Quit) {
sys->AcknowledgeQuit_Exiting();
overlays.Stop();
vr::VR_Shutdown();
return 0;
}
}
std::this_thread::sleep_for(std::chrono::milliseconds(8));
}
}
pointer/helper/run.sh
Executable
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# Start, stop, or inspect the ft-pointer helper on the Frame (runs in the dev container).
# Usage: pointer/helper/run.sh install|uninstall # user service, starts with SteamVR
# pointer/helper/run.sh start|stop|restart|status|log [lines]
# With the service installed, start/stop/restart/log go through systemd.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
. "$root/scripts/_env.sh"
frame="$root/scripts/frame.sh"
unit=frametop-pointer.service
installed() { "$frame" --host "test -f ~/.config/systemd/user/$unit" 2>/dev/null; }
case ${1:-status} in
install)
"$root/scripts/sync.sh" >/dev/null
fill_template "$root/pointer/helper/$unit" | on_frame "mkdir -p ~/.config/systemd/user && cat > ~/.config/systemd/user/$unit"
"$frame" --host "set -e; pkill -x ft-pointer || true
systemctl --user daemon-reload; systemctl --user enable --now $unit; sleep 3
systemctl --user is-active $unit; tail -n 3 /dev/null; journalctl --user -u $unit --no-pager -o cat -n 3" ;;
uninstall) "$frame" --host "systemctl --user disable --now $unit 2>/dev/null; rm -f ~/.config/systemd/user/$unit; systemctl --user daemon-reload; echo removed" ;;
start|stop|restart) if installed; then "$frame" --host "systemctl --user $1 $unit; systemctl --user is-active $unit"; exit; fi ;;&
log) if installed; then "$frame" --host "journalctl --user -u $unit --no-pager -o cat -n ${2:-30}"; exit; fi ;;&
start) "$frame" -C pointer/helper 'pgrep -x ft-pointer >/dev/null && { echo "already running"; exit 0; }
nohup ./build/ft-pointer > /tmp/ft-pointer.log 2>&1 &
sleep 2; pgrep -ax ft-pointer; cat /tmp/ft-pointer.log' ;;
stop) "$frame" --host 'pkill -x ft-pointer && echo stopped || echo "not running"' ;;
restart) "$0" stop; sleep 1; exec "$0" start ;;
status) "$frame" --host 'pgrep -ax ft-pointer || echo "not running"' ;;
log) "$frame" --host "tail -n ${2:-30} /tmp/ft-pointer.log" ;;
*) echo "usage: $0 start|stop|restart|status|log" >&2; exit 2 ;;
esac
pointer/probe/build.sh
Executable
+9
View File
@@ -0,0 +1,9 @@
#!/usr/bin/env bash
# Build vrprobe in the dev container on the Frame (pointer/probe/build/vrprobe).
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
"$root/scripts/sync.sh" >/dev/null
exec "$root/scripts/frame.sh" -C pointer/probe 'set -e; mkdir -p build
g++ -std=c++17 -O2 -Wall -I/opt/steamvr/tools/hellovr_vulkan_linux/src/openvr/headers -I../common \
-o build/vrprobe vrprobe.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64
echo "built build/vrprobe"'
pointer/probe/vrprobe.cpp
+166
View File
@@ -0,0 +1,166 @@
// Print SteamVR's view of tracked devices: class, hand role, controller type,
// connection, pose validity, and the dashboard's primary pointer device.
// Runs as a background OpenVR client (in the dev container).
#include <openvr.h>
#include "vrmath.h"
#include <cmath>
#include <cstdio>
#include <cstdlib>
#include <cstring>
#include <string>
#include <vector>
// Measure a floating panel (see md::ScanPanel).
static int Scan(vr::IVRSystem *sys, const char *key, double step) {
vr::VROverlayHandle_t h;
if (vr::VROverlay()->FindOverlay(key, &h) != vr::VROverlayError_None) {
std::printf("no overlay %s\n", key);
return 1;
}
vr::TrackedDevicePose_t head;
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &head, 1);
const md::Vec3 eye = md::Position(head.mDeviceToAbsoluteTracking);
const md::Panel p = md::ScanPanel(h, eye, step);
std::printf("scan %s: %d hits, head (%.3f %.3f %.3f)\n", key, p.hits, eye.x, eye.y, eye.z);
if (!p.found) return 1;
const md::Vec3 c = p.center, x = p.basis.x, y = p.basis.y, z = p.basis.z, to = c - eye;
std::printf("center (%.3f %.3f %.3f) width %.3f height %.3f distance %.3f\n", c.x, c.y, c.z, p.width, p.height,
md::Length(to));
std::printf("x (%.3f %.3f %.3f) y (%.3f %.3f %.3f) front (%.3f %.3f %.3f)\n", x.x, x.y, x.z, y.x, y.y, y.z, z.x,
z.y, z.z);
return 0;
}
// Usage: vrprobe [overlay-key...] (extra overlays to check besides the dashboard's)
// vrprobe --scan <overlay-key> [step-degrees]
int main(int argc, char **argv) {
vr::EVRInitError err = vr::VRInitError_None;
vr::IVRSystem *sys = vr::VR_Init(&err, vr::VRApplication_Background);
if (err != vr::VRInitError_None) {
std::printf("VR_Init failed: %s\n", vr::VR_GetVRInitErrorAsEnglishDescription(err));
return 1;
}
if (argc >= 3 && std::strcmp(argv[1], "--scan") == 0) {
const int r = Scan(sys, argv[2], argc >= 4 ? std::atof(argv[3]) : 1.0);
vr::VR_Shutdown();
return r;
}
static const char *classes[] = {"invalid", "HMD", "controller", "tracker", "reference", "display"};
static const char *roles[] = {"none", "left", "right", "optout", "treadmill", "stylus"};
vr::TrackedDevicePose_t poses[vr::k_unMaxTrackedDeviceCount];
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, poses, vr::k_unMaxTrackedDeviceCount);
for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) {
const auto cls = sys->GetTrackedDeviceClass(i);
if (cls == vr::TrackedDeviceClass_Invalid) continue;
char type[64] = "", serial[64] = "";
sys->GetStringTrackedDeviceProperty(i, vr::Prop_ControllerType_String, type, sizeof type);
sys->GetStringTrackedDeviceProperty(i, vr::Prop_SerialNumber_String, serial, sizeof serial);
const auto role = sys->GetControllerRoleForTrackedDeviceIndex(i);
const int hint = sys->GetInt32TrackedDeviceProperty(i, vr::Prop_ControllerRoleHint_Int32);
std::printf("%u %-10s role=%-6s hint=%d type=%-16s connected=%d pose=%d %s\n", i,
cls < 6 ? classes[cls] : "?", role < 6 ? roles[role] : "?", hint, type,
sys->IsTrackedDeviceConnected(i), poses[i].bPoseIsValid, serial);
}
std::printf("left hand = %u, right hand = %u\n",
sys->GetTrackedDeviceIndexForControllerRole(vr::TrackedControllerRole_LeftHand),
sys->GetTrackedDeviceIndexForControllerRole(vr::TrackedControllerRole_RightHand));
if (poses[0].bPoseIsValid) {
// Head forward is -Z of the HMD pose. yaw 0 = -Z, positive yaw turns left (about +Y).
const auto &m = poses[0].mDeviceToAbsoluteTracking.m;
const float fx = -m[0][2], fy = -m[1][2], fz = -m[2][2];
std::printf("head yaw = %.1f pitch = %.1f\n", std::atan2(-fx, -fz) * 180.0 / M_PI,
std::asin(fy) * 180.0 / M_PI);
}
// Pointer calibration: our device's forward ray vs where SteamVR's hit dot actually is.
for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) {
char type[64] = "";
sys->GetStringTrackedDeviceProperty(i, vr::Prop_ControllerType_String, type, sizeof type);
if (std::strcmp(type, "ft_pointer") != 0 || !poses[i].bPoseIsValid) continue;
const auto &m = poses[i].mDeviceToAbsoluteTracking.m;
const double o[3] = {m[0][3], m[1][3], m[2][3]}, f[3] = {-m[0][2], -m[1][2], -m[2][2]};
std::printf("ft_pointer origin (%.3f %.3f %.3f) forward (%.3f %.3f %.3f) yaw %.1f pitch %.1f\n", o[0], o[1], o[2],
f[0], f[1], f[2], std::atan2(-f[0], -f[2]) * 180 / M_PI, std::asin(f[1]) * 180 / M_PI);
for (const char *key : {"system.pointer", "system.pointer.secondary", "frametop.pointer.cursor"}) {
vr::VROverlayHandle_t h;
if (vr::VROverlay()->FindOverlay(key, &h) != vr::VROverlayError_None) continue;
vr::ETrackingUniverseOrigin origin;
vr::HmdMatrix34_t t{};
const bool visible = vr::VROverlay()->IsOverlayVisible(h);
vr::VROverlayTransformType type2;
vr::VROverlay()->GetOverlayTransformType(h, &type2);
if (vr::VROverlay()->GetOverlayTransformAbsolute(h, &origin, &t) != vr::VROverlayError_None) {
std::printf(" %-28s visible=%d transform type %d (not absolute)\n", key, visible, int(type2));
continue;
}
const double p2[3] = {t.m[0][3] - o[0], t.m[1][3] - o[1], t.m[2][3] - o[2]};
const double d = std::sqrt(p2[0] * p2[0] + p2[1] * p2[1] + p2[2] * p2[2]);
const double dir[3] = {p2[0] / d, p2[1] / d, p2[2] / d};
float w = 0;
vr::VROverlay()->GetOverlayWidthInMeters(h, &w);
std::printf(" %-28s visible=%d at (%.3f %.3f %.3f) dist %.2f yaw %.1f pitch %.1f width %.3f\n", key, visible,
t.m[0][3], t.m[1][3], t.m[2][3], d, std::atan2(-dir[0], -dir[2]) * 180 / M_PI,
std::asin(dir[1]) * 180 / M_PI, w);
}
}
// Overlay check: transform type, size, and a ray test along ft_pointer's laser.
std::vector<std::string> keys = {"valve.steam.gamepadui.floatingfooter", "valve.steam.gamepadui.bar", "system.systemui"};
keys.insert(keys.end(), argv + 1, argv + argc);
for (const auto &k : keys) {
const char *key = k.c_str();
vr::VROverlayHandle_t h;
if (vr::VROverlay()->FindOverlay(key, &h) != vr::VROverlayError_None) continue;
vr::VROverlayTransformType tt;
vr::VROverlay()->GetOverlayTransformType(h, &tt);
float w = 0;
vr::VROverlay()->GetOverlayWidthInMeters(h, &w);
uint32_t tw = 0, th = 0;
vr::VROverlay()->GetOverlayTextureSize(h, &tw, &th);
std::printf("overlay %-44s visible=%d type=%d width=%.3fm tex=%ux%u", key, vr::VROverlay()->IsOverlayVisible(h),
int(tt), w, tw, th);
vr::ETrackingUniverseOrigin uo;
vr::HmdMatrix34_t t{};
if (tt == vr::VROverlayTransform_Absolute && vr::VROverlay()->GetOverlayTransformAbsolute(h, &uo, &t) == vr::VROverlayError_None)
std::printf(" at (%.2f %.2f %.2f) normal (%.2f %.2f %.2f)", t.m[0][3], t.m[1][3], t.m[2][3], t.m[0][2],
t.m[1][2], t.m[2][2]);
else
std::printf(" transform type %d", int(tt));
vr::HmdVector2_t mouse{};
vr::VROverlay()->GetOverlayMouseScale(h, &mouse);
vr::VROverlayInputMethod im = vr::VROverlayInputMethod_None;
vr::VROverlay()->GetOverlayInputMethod(h, &im);
uint32_t flags = 0;
vr::VROverlay()->GetOverlayFlags(h, &flags);
std::printf(" mouse=%.0fx%.0f input=%d flags=0x%x", mouse.v[0], mouse.v[1], int(im), flags);
if (poses[0].bPoseIsValid) { // gaze ray: from the head, straight ahead
const auto &m = poses[0].mDeviceToAbsoluteTracking.m;
vr::VROverlayIntersectionParams_t params{};
params.vSource = {m[0][3], m[1][3], m[2][3]};
params.vDirection = {-m[0][2], -m[1][2], -m[2][2]};
params.eOrigin = vr::TrackingUniverseStanding;
vr::VROverlayIntersectionResults_t hit{};
const bool ok = vr::VROverlay()->ComputeOverlayIntersection(h, &params, &hit);
std::printf(" | gaze hit=%d dist=%.2f uv=(%.2f %.2f) at (%.2f %.2f %.2f)", ok, hit.fDistance, hit.vUVs.v[0],
hit.vUVs.v[1], hit.vPoint.v[0], hit.vPoint.v[1], hit.vPoint.v[2]);
}
for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) {
char type[64] = "";
sys->GetStringTrackedDeviceProperty(i, vr::Prop_ControllerType_String, type, sizeof type);
if (std::strcmp(type, "ft_pointer") != 0 || !poses[i].bPoseIsValid) continue;
const auto &m = poses[i].mDeviceToAbsoluteTracking.m;
vr::VROverlayIntersectionParams_t params{};
params.vSource = {m[0][3], m[1][3], m[2][3]};
params.vDirection = {-m[0][2], -m[1][2], -m[2][2]};
params.eOrigin = vr::TrackingUniverseStanding;
vr::VROverlayIntersectionResults_t hit{};
const bool ok = vr::VROverlay()->ComputeOverlayIntersection(h, &params, &hit);
std::printf(" | laser hit=%d dist=%.2f uv=(%.2f %.2f)", ok, hit.fDistance, hit.vUVs.v[0], hit.vUVs.v[1]);
}
std::printf("\n");
}
std::printf("primary dashboard device = %u, dashboard visible = %d\n",
vr::VROverlay()->GetPrimaryDashboardDevice(), vr::VROverlay()->IsDashboardVisible());
vr::VR_Shutdown();
return 0;
}
screens/build.sh
Executable
+19
View File
@@ -0,0 +1,19 @@
#!/usr/bin/env bash
# Build ft-screens in the dev container on the Frame (screens/build/ft-screens).
# compositor.c is the wlroots side (C; wlroots headers aren't C++), vr.cpp the OpenVR side.
# vr.cpp needs OpenVR's IVRIPCResourceManagerClient (ImportDmabuf), which the header
# shipped with SteamVR on the Frame predates, so the build uses the public header from
# Valve's openvr repo (pinned; the Frame's runtime supports its interface versions).
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
"$root/scripts/sync.sh" >/dev/null
exec "$root/scripts/frame.sh" -C screens 'set -e; mkdir -p build/include
openvr=v2.15.6
[ -f build/include/openvr-$openvr ] || { curl -fsSL "https://raw.githubusercontent.com/ValveSoftware/openvr/$openvr/headers/openvr.h" -o build/include/openvr.h && touch build/include/openvr-$openvr; }
gcc -std=c11 -O2 -Wall -Wno-unused-parameter -c -o build/compositor.o compositor.c \
$(pkg-config --cflags wlroots-0.20 wayland-server xkbcommon libdrm pixman-1)
g++ -std=c++17 -O2 -Wall -Wno-missing-field-initializers -Ibuild/include -c -o build/vr.o vr.cpp
g++ -o build/ft-screens build/compositor.o build/vr.o \
$(pkg-config --libs wlroots-0.20 wayland-server xkbcommon) \
-L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64
echo "built build/ft-screens"'
screens/compositor.c
+497
View File
@@ -0,0 +1,497 @@
// ft-screens: a minimal Wayland compositor that hosts the nested KWin and shows each of
// its screens as its own SteamVR panel. It replaces gamescope for Frametop:
// - KWin (nested Wayland backend) opens one window per screen. We send each window its
// size (xdg_toplevel configure), and KWin resizes that screen to match, so every
// screen has its own resolution and shape (ultrawide, portrait, 4K...). gamescope
// never sized its windows and drew them all into one canvas of at most 1920x1080.
// - KWin renders into DMA-BUFs and hands them to us (linux-dmabuf). We draw nothing:
// each buffer goes to SteamVR as the panel's texture (vr.cpp, ImportDmabuf).
// - Pointer input from the panels (controller lasers, the 3D mouse) goes to KWin through
// our seat, as if we were a normal desktop.
//
// Usage: ft-screens [--socket NAME] [--screen WxH@METRES]... [-- COMMAND ARGS...]
// --socket Wayland socket name in $XDG_RUNTIME_DIR (default ft-screens-0)
// --screen one per screen, in KWin's order (default: 3440x1440@2.4)
// COMMAND run with WAYLAND_DISPLAY set to our socket (e.g. the Frametop session)
// Runs in the dev container (wlroots 0.20); KWin connects from the host.
#define _GNU_SOURCE
#include <drm_fourcc.h>
#include <linux/input-event-codes.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stddef.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/un.h>
#include <sys/wait.h>
#include <time.h>
#include <unistd.h>
#include <wayland-server-core.h>
#include <xkbcommon/xkbcommon.h>
#define WLR_USE_UNSTABLE
#include <wlr/interfaces/wlr_keyboard.h>
#include <wlr/render/dmabuf.h>
#include <wlr/render/drm_format_set.h>
#include <wlr/types/wlr_buffer.h>
#include <wlr/types/wlr_compositor.h>
#include <wlr/types/wlr_data_device.h>
#include <wlr/types/wlr_keyboard.h>
#include <wlr/types/wlr_linux_dmabuf_v1.h>
#include <wlr/types/wlr_seat.h>
#include <wlr/types/wlr_shm.h>
#include <wlr/types/wlr_subcompositor.h>
#include <wlr/types/wlr_xdg_decoration_v1.h>
#include <wlr/types/wlr_xdg_shell.h>
#include <wlr/util/log.h>
#include "vr.h"
#define MAX_SCREENS 8
struct config {
int width, height;
double metres;
};
struct server;
// One KWin window = one screen.
struct screen {
struct server *server;
int index;
struct wlr_xdg_toplevel *toplevel;
struct wlr_buffer *held; // on the panel now; unlocked when the next one arrives
bool frame_pending; // a commit waits for its frame callback
struct wlr_xdg_toplevel_decoration_v1 *decoration; // answered on the first commit
struct wl_listener commit, destroy, decoration_destroy;
};
// Per client buffer: forget its import when it goes away.
struct tracked_buffer {
struct wlr_buffer *buffer;
struct wl_listener destroy;
struct wl_list link;
};
struct server {
struct wl_display *display;
struct wl_event_loop *loop;
struct wlr_seat *seat;
struct wlr_keyboard keyboard;
struct wlr_xdg_shell *xdg_shell;
struct wlr_xdg_decoration_manager_v1 *decoration;
struct wl_listener new_toplevel, new_decoration;
struct screen *screens[MAX_SCREENS];
struct config config[MAX_SCREENS];
int n_config, n_screens;
struct wl_list buffers; // tracked_buffer
struct wl_event_source *tick;
struct screen *pointer_focus;
pid_t child;
};
static uint32_t now_ms(void) {
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
return (uint32_t)(ts.tv_sec * 1000 + ts.tv_nsec / 1000000);
}
// ---------------------------------------------------------------- buffers
static void buffer_destroyed(struct wl_listener *l, void *data) {
struct tracked_buffer *t = wl_container_of(l, t, destroy);
ft_vr_forget(t->buffer);
wl_list_remove(&t->destroy.link);
wl_list_remove(&t->link);
free(t);
}
static void track_buffer(struct server *s, struct wlr_buffer *buffer) {
struct tracked_buffer *t;
wl_list_for_each(t, &s->buffers, link) if (t->buffer == buffer) return;
t = calloc(1, sizeof *t);
t->buffer = buffer;
t->destroy.notify = buffer_destroyed;
wl_signal_add(&buffer->events.destroy, &t->destroy);
wl_list_insert(&s->buffers, &t->link);
}
// ---------------------------------------------------------------- screens
static void screen_commit(struct wl_listener *l, void *data) {
struct screen *sc = wl_container_of(l, sc, commit);
struct wlr_xdg_surface *xdg = sc->toplevel->base;
if (xdg->initial_commit) {
// First commit: tell KWin the size of this screen.
const struct config *c = &sc->server->config[sc->index < sc->server->n_config ? sc->index : 0];
wlr_xdg_toplevel_set_size(sc->toplevel, c->width, c->height);
wlr_xdg_toplevel_set_activated(sc->toplevel, true);
if (sc->decoration)
wlr_xdg_toplevel_decoration_v1_set_mode(sc->decoration, WLR_XDG_TOPLEVEL_DECORATION_V1_MODE_SERVER_SIDE);
return;
}
struct wlr_buffer *buffer = xdg->surface->current.buffer;
static int logged;
if (logged < 6) {
++logged;
wlr_log(WLR_INFO, "screen %d: commit, buffer %p (%dx%d), mapped %d", sc->index + 1, (void *)buffer,
buffer ? buffer->width : 0, buffer ? buffer->height : 0, xdg->surface->mapped);
}
if (!buffer) return;
sc->frame_pending = true;
if (buffer == sc->held) return;
struct wlr_dmabuf_attributes a;
if (!wlr_buffer_get_dmabuf(buffer, &a)) {
static bool warned;
if (!warned) wlr_log(WLR_ERROR, "screen %d: not a DMA-BUF (shm?); skipped", sc->index + 1);
warned = true;
return;
}
struct ft_dmabuf b = {.width = a.width, .height = a.height, .format = a.format, .modifier = a.modifier,
.n_planes = a.n_planes};
for (int i = 0; i < a.n_planes && i < 4; ++i) {
b.offset[i] = a.offset[i];
b.stride[i] = a.stride[i];
b.fd[i] = a.fd[i];
}
track_buffer(sc->server, buffer);
if (!ft_vr_screen_present(sc->index, buffer, &b)) return;
// Keep this buffer until the next frame replaces it, so SteamVR never samples a
// buffer KWin is drawing into; then let KWin have the previous one back.
wlr_buffer_lock(buffer);
if (sc->held) wlr_buffer_unlock(sc->held);
sc->held = buffer;
}
static void screen_destroy(struct wl_listener *l, void *data) {
struct screen *sc = wl_container_of(l, sc, destroy);
wlr_log(WLR_INFO, "screen %d closed", sc->index + 1);
if (sc->held) wlr_buffer_unlock(sc->held);
ft_vr_screen_destroy(sc->index);
if (sc->server->pointer_focus == sc) sc->server->pointer_focus = NULL;
if (sc->decoration) wl_list_remove(&sc->decoration_destroy.link);
sc->server->screens[sc->index] = NULL;
wl_list_remove(&sc->commit.link);
wl_list_remove(&sc->destroy.link);
free(sc);
}
static void new_toplevel(struct wl_listener *l, void *data) {
struct server *s = wl_container_of(l, s, new_toplevel);
struct wlr_xdg_toplevel *toplevel = data;
int index = 0;
while (index < MAX_SCREENS && s->screens[index]) ++index;
if (index == MAX_SCREENS) {
wlr_log(WLR_ERROR, "more than %d screens; ignoring one", MAX_SCREENS);
return;
}
struct screen *sc = calloc(1, sizeof *sc);
sc->server = s;
sc->index = index;
sc->toplevel = toplevel;
s->screens[index] = sc;
const struct config *c = &s->config[index < s->n_config ? index : 0];
wlr_log(WLR_INFO, "screen %d: KWin window, %dx%d, %.2f m wide", index + 1, c->width, c->height, c->metres);
ft_vr_screen_create(index, c->metres, s->n_config > index + 1 ? s->n_config : index + 1);
sc->commit.notify = screen_commit;
wl_signal_add(&toplevel->base->surface->events.commit, &sc->commit);
sc->destroy.notify = screen_destroy;
wl_signal_add(&toplevel->events.destroy, &sc->destroy);
}
// KWin asks for server-side decorations for its screens; we draw none. It asks before its
// first commit, when a configure isn't allowed yet, so the answer waits for that commit.
static void decoration_destroyed(struct wl_listener *l, void *data) {
struct screen *sc = wl_container_of(l, sc, decoration_destroy);
wl_list_remove(&sc->decoration_destroy.link);
sc->decoration = NULL;
}
static void new_decoration(struct wl_listener *l, void *data) {
struct server *s = wl_container_of(l, s, new_decoration);
struct wlr_xdg_toplevel_decoration_v1 *d = data;
for (int i = 0; i < MAX_SCREENS; ++i) {
struct screen *sc = s->screens[i];
if (!sc || sc->toplevel != d->toplevel) continue;
sc->decoration = d;
sc->decoration_destroy.notify = decoration_destroyed;
wl_signal_add(&d->events.destroy, &sc->decoration_destroy);
if (d->toplevel->base->initialized)
wlr_xdg_toplevel_decoration_v1_set_mode(d, WLR_XDG_TOPLEVEL_DECORATION_V1_MODE_SERVER_SIDE);
}
}
// ---------------------------------------------------------------- input from the panels
static void handle_vr_event(const struct ft_event *e, void *data) {
struct server *s = data;
if (e->type == FT_QUIT) {
wl_display_terminate(s->display);
return;
}
if (e->screen < 0 || e->screen >= MAX_SCREENS || !s->screens[e->screen]) return;
struct screen *sc = s->screens[e->screen];
struct wlr_surface *surface = sc->toplevel->base->surface;
const uint32_t t = now_ms();
switch (e->type) {
case FT_MOTION:
case FT_BUTTON:
if (s->pointer_focus != sc) {
wlr_seat_pointer_notify_enter(s->seat, surface, e->x, e->y);
s->pointer_focus = sc;
}
wlr_seat_pointer_notify_motion(s->seat, t, e->x, e->y);
if (e->type == FT_BUTTON) {
wlr_seat_pointer_notify_button(s->seat, t, e->button,
e->pressed ? WL_POINTER_BUTTON_STATE_PRESSED
: WL_POINTER_BUTTON_STATE_RELEASED);
if (e->pressed) wlr_seat_keyboard_notify_enter(s->seat, surface, NULL, 0, NULL);
}
break;
case FT_SCROLL:
if (s->pointer_focus != sc) break;
if (e->dy != 0)
wlr_seat_pointer_notify_axis(s->seat, t, WL_POINTER_AXIS_VERTICAL_SCROLL, e->dy * 15,
(int32_t)(e->dy * 120), WL_POINTER_AXIS_SOURCE_WHEEL,
WL_POINTER_AXIS_RELATIVE_DIRECTION_IDENTICAL);
if (e->dx != 0)
wlr_seat_pointer_notify_axis(s->seat, t, WL_POINTER_AXIS_HORIZONTAL_SCROLL, e->dx * 15,
(int32_t)(e->dx * 120), WL_POINTER_AXIS_SOURCE_WHEEL,
WL_POINTER_AXIS_RELATIVE_DIRECTION_IDENTICAL);
break;
case FT_LEAVE:
if (s->pointer_focus == sc) {
wlr_seat_pointer_notify_clear_focus(s->seat);
s->pointer_focus = NULL;
}
break;
default:
break;
}
wlr_seat_pointer_notify_frame(s->seat);
}
// Every ~11 ms (90 Hz): SteamVR events, and frame callbacks for screens that committed.
static int tick(void *data) {
struct server *s = data;
ft_vr_poll(handle_vr_event, s);
struct timespec now;
clock_gettime(CLOCK_MONOTONIC, &now);
for (int i = 0; i < MAX_SCREENS; ++i) {
struct screen *sc = s->screens[i];
if (sc && sc->frame_pending) {
sc->frame_pending = false;
wlr_surface_send_frame_done(sc->toplevel->base->surface, &now);
}
}
wl_event_source_timer_update(s->tick, 11);
return 0;
}
static int child_exited(int sig, void *data) {
struct server *s = data;
int status;
pid_t pid;
while ((pid = waitpid(-1, &status, WNOHANG)) > 0)
if (pid == s->child) {
wlr_log(WLR_INFO, "session exited");
wl_display_terminate(s->display);
}
return 0;
}
// Keys from the input relay (physical keyboards): "key <evdev code> <1 press|0 release>".
// They go to the screen KWin has keyboard focus on (the last one clicked), but not while
// the SteamVR dashboard is open: typing belongs to Steam then.
static void handle_key(struct server *s, uint32_t code, int value, char *reply, int size) {
if (value == 2) return (void)snprintf(reply, size, "ok repeat ignored"); // KWin repeats itself
if (!s->seat->keyboard_state.focused_surface) return (void)snprintf(reply, size, "ok no focus");
if (ft_vr_dashboard_visible()) return (void)snprintf(reply, size, "ok dashboard open");
struct wlr_keyboard_key_event ev = {
.time_msec = now_ms(), .keycode = code, .update_state = true,
.state = value ? WL_KEYBOARD_KEY_STATE_PRESSED : WL_KEYBOARD_KEY_STATE_RELEASED};
wlr_keyboard_notify_key(&s->keyboard, &ev); // keeps the xkb state and modifiers
wlr_seat_keyboard_notify_modifiers(s->seat, &s->keyboard.modifiers);
wlr_seat_keyboard_notify_key(s->seat, ev.time_msec, code, ev.state);
snprintf(reply, size, "ok");
}
// Control socket: abstract datagram @ft_screens. Here: "size <screen> <w> <h>" (a new
// resolution, live) and "key <code> <value>"; the rest is in vr.cpp (ft_vr_command).
static int control_readable(int fd, uint32_t mask, void *data) {
struct server *s = data;
char buf[512], reply[2048];
struct sockaddr_un from;
socklen_t len = sizeof from;
ssize_t n;
while ((n = recvfrom(fd, buf, sizeof buf - 1, MSG_DONTWAIT, (struct sockaddr *)&from, &len)) > 0) {
buf[n] = 0;
unsigned code;
int value, index, w, h;
if (sscanf(buf, "size %d %d %d", &index, &w, &h) == 3) {
// A new resolution for a screen, live: KWin resizes the screen to match.
if (index < 1 || index > MAX_SCREENS || !s->screens[index - 1] || w < 320 || h < 200 || w > 16384 ||
h > 16384) {
snprintf(reply, sizeof reply, "error bad screen or size");
} else {
if (index - 1 < s->n_config) s->config[index - 1].width = w, s->config[index - 1].height = h;
wlr_xdg_toplevel_set_size(s->screens[index - 1]->toplevel, w, h);
snprintf(reply, sizeof reply, "ok");
}
} else if (sscanf(buf, "key %u %d", &code, &value) == 2) {
handle_key(s, code, value, reply, sizeof reply);
len = sizeof from;
continue; // no reply: keys are fire-and-forget
} else {
ft_vr_command(buf, reply, sizeof reply);
}
if (len > offsetof(struct sockaddr_un, sun_path))
sendto(fd, reply, strlen(reply), MSG_DONTWAIT, (struct sockaddr *)&from, len);
len = sizeof from;
}
return 0;
}
static int open_control_socket(void) {
int fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
struct sockaddr_un addr = {.sun_family = AF_UNIX};
const char name[] = "ft_screens";
memcpy(addr.sun_path + 1, name, sizeof name - 1);
if (bind(fd, (struct sockaddr *)&addr, offsetof(struct sockaddr_un, sun_path) + 1 + sizeof name - 1) != 0) {
wlr_log(WLR_ERROR, "can't bind @ft_screens (another ft-screens running?)");
close(fd);
return -1;
}
return fd;
}
static int stop(int sig, void *data) {
wl_display_terminate(((struct server *)data)->display);
return 0;
}
// ---------------------------------------------------------------- setup
static void keyboard_led(struct wlr_keyboard *kb, uint32_t leds) {}
static const struct wlr_keyboard_impl keyboard_impl = {.name = "ft-screens-keyboard", .led_update = keyboard_led};
static bool setup_dmabuf(struct server *s) {
struct stat st;
const char *node = "/dev/dri/renderD128";
if (stat(node, &st) != 0) {
wlr_log(WLR_ERROR, "no %s", node);
return false;
}
struct wlr_linux_dmabuf_feedback_v1 fb = {.main_device = st.st_rdev};
wl_array_init(&fb.tranches);
struct wlr_linux_dmabuf_feedback_v1_tranche *tr = wlr_linux_dmabuf_feedback_add_tranche(&fb);
tr->target_device = st.st_rdev;
const uint32_t formats[] = {DRM_FORMAT_XRGB8888, DRM_FORMAT_ARGB8888};
for (size_t f = 0; f < 2; ++f) {
uint64_t mods[64];
const int n = ft_vr_modifiers(formats[f], mods, 64);
for (int i = 0; i < n; ++i) wlr_drm_format_set_add(&tr->formats, formats[f], mods[i]);
wlr_log(WLR_INFO, "dmabuf: format 0x%x, %d modifiers SteamVR can import", formats[f], n);
}
struct wlr_linux_dmabuf_v1 *dmabuf = wlr_linux_dmabuf_v1_create(s->display, 4, &fb);
wlr_linux_dmabuf_feedback_v1_finish(&fb);
return dmabuf != NULL;
}
int main(int argc, char **argv) {
struct server s = {0};
const char *socket_name = "ft-screens-0";
char **command = NULL;
for (int i = 1; i < argc; ++i) {
if (strcmp(argv[i], "--socket") == 0 && i + 1 < argc) {
socket_name = argv[++i];
} else if (strcmp(argv[i], "--screen") == 0 && i + 1 < argc && s.n_config < MAX_SCREENS) {
struct config *c = &s.config[s.n_config];
c->metres = 0;
if (sscanf(argv[++i], "%dx%d@%lf", &c->width, &c->height, &c->metres) < 2) {
fprintf(stderr, "bad --screen %s (want WxH@METRES)\n", argv[i]);
return 2;
}
if (c->metres <= 0) c->metres = 1.5 * c->width / 1920.0;
++s.n_config;
} else if (strcmp(argv[i], "--") == 0) {
command = &argv[i + 1];
break;
} else {
fprintf(stderr, "usage: %s [--socket NAME] [--screen WxH@METRES]... [-- COMMAND ARGS...]\n", argv[0]);
return 2;
}
}
if (s.n_config == 0) s.config[s.n_config++] = (struct config){3440, 1440, 2.4};
setvbuf(stdout, NULL, _IOLBF, 0); // vr.cpp prints to stdout; keep it in order with the log
wlr_log_init(WLR_INFO, NULL);
if (!ft_vr_init()) return 1;
s.display = wl_display_create();
s.loop = wl_display_get_event_loop(s.display);
wl_list_init(&s.buffers);
wlr_compositor_create(s.display, 6, NULL);
wlr_subcompositor_create(s.display);
const uint32_t shm_formats[] = {DRM_FORMAT_ARGB8888, DRM_FORMAT_XRGB8888}; // wlroots wants DRM codes
wlr_shm_create(s.display, 1, shm_formats, 2);
if (!setup_dmabuf(&s)) return 1;
wlr_data_device_manager_create(s.display);
s.xdg_shell = wlr_xdg_shell_create(s.display, 3);
s.new_toplevel.notify = new_toplevel;
wl_signal_add(&s.xdg_shell->events.new_toplevel, &s.new_toplevel);
s.decoration = wlr_xdg_decoration_manager_v1_create(s.display);
s.new_decoration.notify = new_decoration;
wl_signal_add(&s.decoration->events.new_toplevel_decoration, &s.new_decoration);
s.seat = wlr_seat_create(s.display, "seat0");
wlr_keyboard_init(&s.keyboard, &keyboard_impl, "ft-screens-keyboard");
struct xkb_context *xkb = xkb_context_new(XKB_CONTEXT_NO_FLAGS);
struct xkb_keymap *keymap = xkb_keymap_new_from_names(xkb, NULL, XKB_KEYMAP_COMPILE_NO_FLAGS);
wlr_keyboard_set_keymap(&s.keyboard, keymap);
xkb_keymap_unref(keymap);
xkb_context_unref(xkb);
wlr_seat_set_keyboard(s.seat, &s.keyboard);
wlr_seat_set_capabilities(s.seat, WL_SEAT_CAPABILITY_POINTER | WL_SEAT_CAPABILITY_KEYBOARD);
if (wl_display_add_socket(s.display, socket_name) != 0) {
wlr_log(WLR_ERROR, "can't create socket %s (another ft-screens?)", socket_name);
return 1;
}
char path[256];
snprintf(path, sizeof path, "%s/%s", getenv("XDG_RUNTIME_DIR") ? getenv("XDG_RUNTIME_DIR") : "/tmp", socket_name);
wlr_log(WLR_INFO, "listening on %s; %d screen(s) configured", path, s.n_config);
const int control = open_control_socket();
if (control < 0) return 1;
wl_event_loop_add_fd(s.loop, control, WL_EVENT_READABLE, control_readable, &s);
s.tick = wl_event_loop_add_timer(s.loop, tick, &s);
wl_event_source_timer_update(s.tick, 11);
wl_event_loop_add_signal(s.loop, SIGINT, stop, &s);
wl_event_loop_add_signal(s.loop, SIGTERM, stop, &s);
wl_event_loop_add_signal(s.loop, SIGCHLD, child_exited, &s);
if (command && command[0]) {
s.child = fork();
if (s.child == 0) {
setenv("WAYLAND_DISPLAY", path, 1);
execvp(command[0], command);
perror(command[0]);
_exit(127);
}
}
wl_display_run(s.display);
wlr_log(WLR_INFO, "stopping");
if (s.child > 0) kill(s.child, SIGTERM);
wl_display_destroy_clients(s.display);
ft_vr_shutdown();
wl_display_destroy(s.display);
return 0;
}
screens/vr.cpp
+1275
View File
File diff suppressed because it is too large. Load diff
screens/vr.h
+52
View File
@@ -0,0 +1,52 @@
// The OpenVR side of ft-screens (vr.cpp), called from the wlroots compositor (compositor.c).
#pragma once
#include <stdbool.h>
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
struct ft_dmabuf {
int width, height;
uint32_t format; // DRM_FORMAT_*
uint64_t modifier; // DRM_FORMAT_MOD_*
int n_planes;
uint32_t offset[4], stride[4];
int fd[4];
};
enum ft_event_type { FT_MOTION, FT_BUTTON, FT_SCROLL, FT_LEAVE, FT_QUIT };
struct ft_event {
enum ft_event_type type;
int screen;
double x, y; // FT_MOTION: buffer pixels from the top left
uint32_t button; // FT_BUTTON: linux BTN_*
bool pressed;
double dx, dy; // FT_SCROLL: notches (positive dy: scroll down)
};
bool ft_vr_init(void);
void ft_vr_shutdown(void);
// Modifiers SteamVR can import for a DRM format. Returns the count (at most max).
int ft_vr_modifiers(uint32_t format, uint64_t *out, int max);
// The SteamVR dashboard is open (typing belongs to it then, not to the screens).
bool ft_vr_dashboard_visible(void);
// A panel for screen `index`, width in metres, placed in a row in front of the head.
void ft_vr_screen_create(int index, double metres, int count);
void ft_vr_screen_destroy(int index);
// Show a client buffer (identified by `key`) on the screen's panel. False if SteamVR
// can't import it.
bool ft_vr_screen_present(int index, const void *key, const struct ft_dmabuf *buf);
// A buffer is going away: drop its import.
void ft_vr_forget(const void *key);
// Poll panel input and SteamVR events.
void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data);
// A command from the control socket (@ft_screens); writes the reply (see vr.cpp).
void ft_vr_command(const char *command, char *reply, int reply_size);
#ifdef __cplusplus
}
#endif
scripts/_env.sh
Executable
+60
View File
@@ -0,0 +1,60 @@
# Sourced by the other scripts. Works out where the Steam Frame is and how to reach it.
#
# On the Frame itself (SteamOS, VR variant), FRAME_LOCAL=1: commands run locally, and the
# Frame's copy of the repo (FRAME_REPO) is this checkout, wherever it was cloned.
# On a PC, FRAME_LOCAL=0: commands run over SSH on $FRAME_HOST (default "frame"), and the
# Frame's copy is the one scripts/sync.sh keeps at ~/dev/frametop.
# FRAME_LOCAL, FRAME_HOST, FRAME_REPO, and FRAME_BOX (container, default "dev") can be
# set in the environment to override.
REPO_ROOT=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
if [ -z "${FRAME_LOCAL:-}" ]; then
FRAME_LOCAL=0
if grep -qx 'ID=steamos' /etc/os-release 2>/dev/null && grep -qE '^VARIANT_ID="?vr"?$' /etc/os-release; then
FRAME_LOCAL=1
fi
fi
if [ "$FRAME_LOCAL" = 1 ]; then
FRAME_REPO=$REPO_ROOT
# A terminal inside a Plasma session in VR (Frametop or the stock desktop) has that
# session's private XDG_RUNTIME_DIR and D-Bus. systemctl --user and podman need the
# real ones.
export XDG_RUNTIME_DIR=/run/user/$(id -u)
export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
else
FRAME_REPO=${FRAME_REPO:-/home/steamos/dev/frametop}
# Agent and IDE shells often inherit gpg-agent's socket while the keys are loaded
# into keychain's ssh-agent at login. Use keychain's when the current one has no keys.
if ! ssh-add -l >/dev/null 2>&1; then
kc="$HOME/.keychain/$(hostname)-sh"
# shellcheck disable=SC1090
[ -f "$kc" ] && . "$kc" >/dev/null
fi
fi
FRAME_HOST=${FRAME_HOST:-frame}
FRAME_BOX=${FRAME_BOX:-dev}
# on_frame '<command>': run a shell command on the Frame host, in FRAME_REPO.
on_frame() {
if [ "$FRAME_LOCAL" = 1 ]; then
(cd "$FRAME_REPO" && bash -c "$1")
else
ssh -o BatchMode=yes "$FRAME_HOST" "cd $(printf %q "$FRAME_REPO") && $1"
fi
}
# on_frame_script [args...] < script: run a bash script from stdin on the Frame host.
on_frame_script() {
if [ "$FRAME_LOCAL" = 1 ]; then
bash -s -- "$@"
else
ssh -o BatchMode=yes "$FRAME_HOST" "bash -s -- $(printf '%q ' "$@")"
fi
}
# fill_template <file>: print a file with @REPO@ replaced by the Frame's repo path.
fill_template() {
sed "s|@REPO@|$FRAME_REPO|g" "$1"
}
scripts/doctor.sh
Executable
+34
View File
@@ -0,0 +1,34 @@
#!/usr/bin/env bash
# Check that the Steam Frame is ready for these projects: reachable (from a PC), the
# distrobox tool and the dev container present, and disk space. Works on the Frame too.
set -uo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/_env.sh"
fail=0
check() {
local label=$1; shift
if out=$("$@" 2>&1); then
printf 'ok %s%s\n' "$label" "${out:+: $out}"
else
printf 'FAIL %s%s\n' "$label" "${out:+: $out}"
fail=1
fi
}
if [ "$FRAME_LOCAL" = 1 ]; then
echo "running on the Frame: $FRAME_REPO"
else
check "ssh key loaded" bash -c 'ssh-add -l | grep -c . | sed "s/$/ key(s)/"'
check "ssh to $FRAME_HOST" ssh -o BatchMode=yes -o ConnectTimeout=8 "$FRAME_HOST" true
if [ "$fail" = 1 ]; then
echo "Skipping device checks. Is the headset on and awake, and on the network?"
exit 1
fi
fi
check "SteamOS" on_frame '. /etc/os-release; echo "$PRETTY_NAME $VERSION_ID build $BUILD_ID, $(uname -m)"'
check "distrobox" on_frame 'test -x ~/.local/bin/distrobox && ~/.local/bin/distrobox version | head -1'
check "container $FRAME_BOX" on_frame "podman ps -a --filter name=^$FRAME_BOX\$ --format '{{.Image}} {{.Status}}' | grep ."
check "repo on the Frame" on_frame 'pwd'
check "free space in ~" on_frame "df -h ~ | awk 'NR==2{print \$4\" free\"}'"
exit "$fail"
scripts/frame.sh
Executable
+43
View File
@@ -0,0 +1,43 @@
#!/usr/bin/env bash
# Run a command on the Steam Frame, inside the "dev" distrobox by default.
# On the Frame itself it runs locally in this checkout; from a PC it runs over SSH
# in the synced copy (see scripts/_env.sh).
# Usage: scripts/frame.sh [--host] [-C subdir] command [args...]
# --host run on the SteamOS host instead of the container
# -C subdir start in <repo>/<subdir> (default: the repo root)
# The command is passed to a shell as one string, like ssh does.
set -euo pipefail
. "$(dirname "${BASH_SOURCE[0]}")/_env.sh"
on_host=0
sub=
while [ $# -gt 0 ]; do
case $1 in
--host) on_host=1; shift ;;
-C) sub=$2; shift 2 ;;
--) shift; break ;;
*) break ;;
esac
done
[ $# -gt 0 ] || { echo "usage: $0 [--host] [-C subdir] command [args...]" >&2; exit 2; }
dir=$FRAME_REPO${sub:+/$sub}
# distrobox is called by its full path: ~/.bashrc on the Frame returns early for
# non-interactive shells, so ~/.local/bin isn't on PATH. distrobox enter keeps the cwd.
if [ "$FRAME_LOCAL" = 1 ]; then
cd "$dir"
if [ "$on_host" = 1 ]; then
exec bash -c "$*"
fi
exec "$HOME/.local/bin/distrobox" enter "$FRAME_BOX" -- bash -lc "$*"
fi
tty=()
[ -t 0 ] && [ -t 1 ] && tty=(-t)
if [ "$on_host" = 1 ]; then
exec ssh "${tty[@]}" "$FRAME_HOST" "cd $(printf %q "$dir") && $*"
else
exec ssh "${tty[@]}" "$FRAME_HOST" \
"cd $(printf %q "$dir") && ~/.local/bin/distrobox enter $(printf %q "$FRAME_BOX") -- bash -lc $(printf %q "$*")"
fi
scripts/sync.sh
Executable
+18
View File
@@ -0,0 +1,18 @@
#!/usr/bin/env bash
# One-way sync of this repo from a PC to ~/dev/frametop on the Steam Frame.
# Usage: scripts/sync.sh [extra rsync args, e.g. --dry-run]
# Honors .gitignore files and skips .git (dirs, and the files submodules use), build outputs, and .env files.
# --delete only removes files inside ~/dev/frametop on the headset.
# On the Frame itself there's nothing to do: the checkout is used directly.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
. "$root/scripts/_env.sh"
if [ "$FRAME_LOCAL" = 1 ]; then
exit 0
fi
exec rsync -az --delete --info=stats1 \
--filter=':- .gitignore' \
--exclude='.git' --exclude='target/' --exclude='build/' --exclude='.env' --exclude='.env.*' \
"$@" "$root/" "$FRAME_HOST:${FRAME_REPO#/home/steamos/}/"
session/deckard-nested-desktop.desktop
+17
View File
@@ -0,0 +1,17 @@
[Desktop Entry]
Type=Application
Name=Desktop
Comment=Plasma desktop across several VR screens (Frametop)
Exec=@SESSION@
Categories=Utility;
# Installed by desktops.sh install into ~/.local/share/applications,
# where it overrides /usr/share/applications/deckard-nested-desktop.desktop.
# Remove it with desktops.sh uninstall to get the stock desktop back.
Icon=desktop
X-Steam-Library-Capsule=/usr/share/applications/steam/steamos-nested-desktop/capsule.png
X-Steam-Library-Hero=/usr/share/applications/steam/steamos-nested-desktop/hero.png
X-Steam-Library-Logo=/usr/share/applications/steam/steamos-nested-desktop/logo.png
X-Steam-Library-StoreCapsule=/usr/share/applications/steam/steamos-nested-desktop/store-logo.png
X-Steam-Controller-Template=Desktop
X-Steam-Special=Desktop
session/frametop-session.sh
Executable
+135
View File
@@ -0,0 +1,135 @@
#!/bin/bash
# Runs on the Frame host (not in the container). Starts a Plasma desktop with several
# screens, each its own SteamVR panel. Modeled on /usr/bin/steamos-nested-desktop, which
# does the same with one screen in gamescope.
#
# Backends (BACKEND in ~/.config/frametop.conf):
# screens (default) ft-screens (screens), our own compositor: KWin opens a
# window per screen, ft-screens gives each its size (from the layout, see
# layout) and shows it as its own panel. Any resolution and shape.
# gamescope gamescope in PerWindow mode: every screen one size, at most 1920x1080
# worth of pixels, panels owned by the SteamVR dashboard.
#
# Settings come from ~/.config/frametop.conf (see frametop.conf.example) and, for the
# screens backend, ~/.config/frametop-layout.json (Frametop Display Settings writes both).
# FT_BACKEND, FT_SCREENS, FT_WIDTH, FT_HEIGHT, and FT_PHYS_WIDTH override them.
set -eu
here=$(dirname "$(readlink -f "$0")")
conf=$HOME/.config/frametop.conf
BACKEND=screens SCREENS=2 WIDTH=1920 HEIGHT=1080 PHYS_WIDTH=1.6 REMOTE=0
# shellcheck disable=SC1090
[ -f "$conf" ] && . "$conf"
backend=${FT_BACKEND:-$BACKEND}
screens=${FT_SCREENS:-$SCREENS}
width=${FT_WIDTH:-$WIDTH}
height=${FT_HEIGHT:-$HEIGHT}
phys_width=${FT_PHYS_WIDTH:-$PHYS_WIDTH}
remote=${FT_REMOTE:-$REMOTE}
if [ "$backend" = gamescope ] && [ $((width * height)) -gt $((1920 * 1080)) ]; then
# gamescope's VR backend aborts above 1920x1080 worth of pixels (its upload buffer;
# see docs/design.md). Shrink a bigger size to fit, keeping its shape.
read -r width height < <(awk -v w="$width" -v h="$height" 'BEGIN { k = sqrt(1920 * 1080 / (w * h));
printf "%d %d\n", int(w * k / 8) * 8, int(h * k / 8) * 8 }')
echo "frametop: resolution too big for gamescope's VR mode; using ${width}x${height}" >&2
fi
if [ "${1:-}" != --inner ]; then
# One instance at a time. The launcher can be clicked twice.
if pgrep -f '[v]r-overlay-key frametop ' >/dev/null || pgrep -x ft-screens >/dev/null; then
echo "Frametop is already running" >&2
exit 0
fi
set -a; . /usr/share/deckard/mesavars.sh; set +a
# Flatpak apps (Chromium) publish their launcher entries under the Flatpak
# exports dirs. SSH and launcher environments may lack XDG_DATA_DIRS, and then
# Plasma can't find them and opens Discover instead.
export XDG_DATA_DIRS=${XDG_DATA_DIRS:-/usr/local/share:/usr/share}
set +u; . /etc/profile.d/flatpak.sh; set -u
unset LD_PRELOAD
# Arrange the screens in the saved layout once they're up (layout; skipped
# when auto-arrange is off).
setsid "$here/../layout/ft-layout" apply --wait 90 > /tmp/frametop-layout.log 2>&1 < /dev/null &
if [ "$backend" = gamescope ]; then
export ENABLE_GAMESCOPE_WSI=1 GAMESCOPE_MANGOAPP_SOCKET_DISABLE=1
exec gamescope --backend openvr --virtual-connector-strategy PerWindow \
-W "$width" -H "$height" -w "$width" -h "$height" \
--vr-overlay-key frametop --vr-overlay-default-name frametop \
--vr-overlay-physical-width "$phys_width" \
--vr-overlay-show-immediately --vr-overlay-enable-click-stabilization \
--vr-overlay-enable-control-bar --vr-overlay-enable-control-bar-keyboard \
--expose-wayland \
--cursor-hotspot 5,3 --cursor /usr/share/steamos/steamos-cursor.png \
-- "$0" --inner
fi
# ft-screens runs in the dev container (it's built against Fedora's wlroots); KWin and
# Plasma stay on the host and connect to its socket.
socket=ft-screens-0
read -ra screen_args <<< "$("$here/../layout/ft-layout" screen-args)"
export FT_SCREEN_COUNT=$(( ${#screen_args[@]} / 2 ))
"$HOME/.local/bin/distrobox" enter dev -- "$here/../screens/build/ft-screens" --socket "$socket" \
"${screen_args[@]}" > /tmp/frametop-screens.log 2>&1 < /dev/null &
stop_screens() { pkill -x ft-screens 2>/dev/null || true; }
trap stop_screens EXIT
for _ in $(seq 100); do [ -S "$XDG_RUNTIME_DIR/$socket" ] && break; sleep 0.2; done
[ -S "$XDG_RUNTIME_DIR/$socket" ] || { echo "ft-screens didn't start (see /tmp/frametop-screens.log)" >&2; exit 1; }
WAYLAND_DISPLAY=$XDG_RUNTIME_DIR/$socket "$0" --inner
exit
fi
# Inside the host compositor (ft-screens or gamescope) from here on.
unset LD_PRELOAD XDG_DESKTOP_PORTAL_DIR
[ "$backend" = gamescope ] || screens=${FT_SCREEN_COUNT:-$screens}
host_runtime=$XDG_RUNTIME_DIR
runtime=$host_runtime/frametop
cleanup() {
pkill -f '[k]rdpserver --plasma' 2>/dev/null || true
pkill -f '[X]vnc :20 ' 2>/dev/null || true
pkill -f '[x]freerdp /v:.*:3390' 2>/dev/null || true
fusermount3 -u -z "$runtime/doc" 2>/dev/null || true
umount --recursive "$runtime" 2>/dev/null || true
rm -rf "$runtime"
}
trap cleanup EXIT
cleanup
mkdir -m 0700 "$runtime" "$runtime/pulse" "$runtime/bin"
ln -s "$host_runtime/pulse/native" "$runtime/pulse/native"
ln -s "$host_runtime"/pipewire* "$runtime/"
# plasma-session starts KWin through kwin_wayland_wrapper. Shadow it to add our outputs.
# With ft-screens the size is only the starting one: ft-screens sets each screen's own.
cat > "$runtime/bin/kwin_wayland_wrapper" <<EOF
#!/bin/sh
exec /usr/bin/kwin_wayland_wrapper --width $width --height $height --output-count $screens --no-lockscreen "\$@"
EOF
chmod +x "$runtime/bin/kwin_wayland_wrapper"
export PATH=$runtime/bin:$PATH
# Keep the host compositor's Wayland socket reachable after moving XDG_RUNTIME_DIR.
case ${WAYLAND_DISPLAY:-gamescope-0} in
/*) ;;
*) export WAYLAND_DISPLAY=$host_runtime/${WAYLAND_DISPLAY:-gamescope-0} ;;
esac
export XDG_RUNTIME_DIR=$runtime
# Separate Plasma/KWin config and state, so this session and the built-in
# desktop never overwrite each other's screen layout or panels.
export XDG_CONFIG_HOME=$HOME/.config/frametop
export XDG_STATE_HOME=$HOME/.local/state/frametop
mkdir -p "$XDG_CONFIG_HOME" "$XDG_STATE_HOME"
# Remote desktop over VNC: session/remote-desktop.sh captures the desktop with
# krdp on 127.0.0.1, and session/vnc-bridge.sh re-serves it over VNC. krdpserver runs from the container, so KWin can't
# match it to an installed app. KWin's permission check for screencast and fake
# input is turned off for this nested session only.
if [ "$remote" = 1 ]; then
export KWIN_WAYLAND_NO_PERMISSION_CHECKS=1
"$here/remote-desktop.sh" "$runtime" > /tmp/frametop-remote.log 2>&1 &
"$here/vnc-bridge.sh" "$width" "$height" > /tmp/frametop-vnc.log 2>&1 &
fi
dbus-run-session startplasma-wayland
session/frametop.conf.example
+22
View File
@@ -0,0 +1,22 @@
# Frametop settings, read by session/frametop-session.sh on the Frame.
# Installed to ~/.config/frametop.conf. Environment variables with an FT_ prefix override these.
BACKEND=screens # screens: ft-screens, each screen its own resolution and size (set in Frametop Display Settings,
# saved in ~/.config/frametop-layout.json) | gamescope: the old path, settings below
SCREENS=2 # gamescope: number of desktop screens (VR panels)
WIDTH=1920 # gamescope: pixels per screen (at most 1920x1080 worth)
HEIGHT=1080
PHYS_WIDTH=1.6 # gamescope: panel width in metres, docked
REMOTE=0 # 1 = serve the desktop over VNC on the tailnet (port 5900; see README)
POINTER=0 # 1 = the mouse drives the universal 3D pointer (ft_pointer driver) instead of a plain mouse
POINTER_SENSITIVITY=0.03 # degrees per mouse count
POINTER_IDLE=30 # seconds without mouse use before the controllers get their laser back
POINTER_WAKE_COUNTS=40 # mouse counts within 1 s needed to wake or re-claim the laser (ignores desk jitter)
POINTER_DISTANCE=1.5 # metres to the cursor when it isn't on a panel
POINTER_CURSOR_DEG=0.4 # size of the free-space dot, in degrees
POINTER_ORIGIN_FRACTION=0.95 # laser starts this far along eye->cursor: its beam is a few cm, SteamVR's hit dot tiny
POINTER_LASER_WIDTH=0.8 # controller beam width (dashboard.laserRayWidthScale), set once at helper start
POINTER_ORIGIN_MARGIN=0.15 # the laser starts at least this far (m) in front of its target, so small floating controls stay hittable
POINTER_SCENE_RADIUS=0.5 # texture-less dashboard overlays (dock, window controls): hit radius (m) around their origin
POINTER_EDGE_REACH=0.3 # just off a panel, the cursor stays on its plane this far (m), to reach its resize edges and window controls
LAYOUT_GRAB_OFFSET=0.075 # arranging screens: where the floating window's grab bar is, in metres below the screen
LAYOUT_SLIDE_SPEED=0.5 # arranging screens: how fast the invisible controller slides a grabbed screen (m/s)
session/keep-apps.sh
Executable
+35
View File
@@ -0,0 +1,35 @@
#!/bin/bash
# Runs on the Frame host just before the desktop stops (desktops.sh stop). Moves the
# programs started inside the desktop out of its systemd unit (frametop-desktop) into a
# scope of their own, so stopping the unit (which kills everything left in it) only ends
# the desktop itself. Windows still close: a GUI app exits when its compositor goes.
# Background work started from the desktop keeps running: servers, agents, tmux, builds.
unit=frametop-desktop.service
cg=$(systemctl --user show -p ControlGroup --value "$unit" 2>/dev/null)
[ -n "$cg" ] && [ -r "/sys/fs/cgroup$cg/cgroup.procs" ] || exit 0
# The desktop's own processes stay in the unit and stop with it: the session, KWin,
# Plasma, and the session services it started (portals, input methods, kded, wallet...).
own='^(frametop-sessi|dbus-|startplasma|plasma|kwin|Xwayland|ksmserver|krdpserver|Xvnc|xfreerdp|ft-layout|'
own+='kded|kactivitymanage|kaccess|kglobalaccel|kscreen|kwalletd|ksecretd|polkit-kde|org_kde_|baloo|'
own+='xembedsniproxy|gmenudbusmenu|DiscoverNotifie|kimpanel|ibus|xdg-|at-spi|dconf-service|fusermount|agent)'
keep=()
while read -r pid; do
comm=$(cat "/proc/$pid/comm" 2>/dev/null) || continue
[[ $comm =~ $own ]] && continue
# ft-screens' launcher (distrobox enter, podman exec) stays too.
tr '\0' ' ' < "/proc/$pid/cmdline" 2>/dev/null | grep -q -e 'ft-screens' -e 'ft-layout' -e 'frametop-session' && continue
keep+=("$pid")
done < "/sys/fs/cgroup$cg/cgroup.procs"
[ ${#keep[@]} -gt 0 ] || exit 0
scope=frametop-apps-$(date +%s).scope
if busctl --user call org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager \
StartTransientUnit 'ssa(sv)a(sa(sv))' "$scope" fail 3 \
PIDs au "${#keep[@]}" "${keep[@]}" \
Description s "Programs started in the Frametop desktop, kept past its restart" \
CollectMode s inactive-or-failed 0 >/dev/null; then
echo "kept ${#keep[@]} process(es) in $scope"
else
echo "couldn't move the desktop's programs out of $unit; they stop with it" >&2
fi
session/remote-desktop.sh
Executable
+34
View File
@@ -0,0 +1,34 @@
#!/bin/bash
# Runs on the Frame host. Internal capture server for remote access: KRdp's
# krdpserver (from the dev container) talks to the nested KWin directly
# (--plasma, no desktop portal) and serves it over RDP on 127.0.0.1 only.
# Nothing outside the Frame can reach it. vnc-bridge.sh connects to it and
# re-serves the desktop over VNC. Started by frametop-session.sh when REMOTE=1.
set -eu
runtime=${1:?usage: remote-desktop.sh <nested XDG_RUNTIME_DIR>}
# 3389 is taken by SteamOS's own xrdp (a separate X11 session, not the VR desktop).
port=${RDP_PORT:-3390}
creds=$HOME/.config/frametop-remote
mkdir -p -m 0700 "$creds"
if [ ! -s "$creds/password" ]; then
(umask 077; head -c 24 /dev/urandom | base64 | tr -d '/+=' | cut -c1-20 > "$creds/password")
fi
if [ ! -s "$creds/cert.pem" ]; then
(umask 077; openssl req -x509 -newkey rsa:2048 -nodes -days 3650 -subj /CN=steam-frame \
-keyout "$creds/key.pem" -out "$creds/cert.pem" 2>/dev/null)
fi
# Wait for the nested KWin to come up.
for _ in $(seq 60); do
[ -S "$runtime/wayland-0" ] && break
sleep 1
done
# podman needs the real runtime dir. The nested one is passed only to krdpserver.
export XDG_RUNTIME_DIR=/run/user/$(id -u)
exec ~/.local/bin/distrobox enter dev -- env XDG_RUNTIME_DIR="$runtime" WAYLAND_DISPLAY=wayland-0 QT_QPA_PLATFORM=wayland \
krdpserver --plasma --address 127.0.0.1 --port "$port" \
-u steamos -p "$(cat "$creds/password")" \
--certificate "$creds/cert.pem" --certificate-key "$creds/key.pem"
session/vnc-bridge.sh
Executable
+55
View File
@@ -0,0 +1,55 @@
#!/bin/bash
# Runs on the Frame host. Serves the Frametop desktop over VNC for clients
# like RealVNC Viewer or macOS Screen Sharing. No VNC server here can capture
# KWin directly, so this bridges through krdp: Xvnc (a virtual X screen served
# over VNC) runs a full-screen FreeRDP client connected to krdpserver on
# 127.0.0.1. Both run in the dev container. VNC listens on the tailnet address only.
# Started by frametop-session.sh when REMOTE=1, after remote-desktop.sh.
set -eu
width=${1:-1920}
height=${2:-1080}
vnc_port=${VNC_PORT:-5900}
rdp_port=${RDP_PORT:-3390}
display=:20
creds=$HOME/.config/frametop-remote
addr=$(ip -4 -o addr show tailscale0 2>/dev/null | awk '{print $4}' | cut -d/ -f1)
if [ -z "$addr" ]; then
echo "tailscale0 has no address, not starting VNC" >&2
exit 1
fi
# The VNC password is limited to 8 characters by the protocol. The traffic is
# still encrypted by the tailnet (WireGuard).
if [ ! -s "$creds/vnc-password" ]; then
(umask 077; head -c 12 /dev/urandom | base64 | tr -d '/+=' | cut -c1-8 > "$creds/vnc-password")
fi
# Wait for krdpserver (started by remote-desktop.sh).
for _ in $(seq 60); do
ss -ltn | grep -q "127.0.0.1:$rdp_port " && break
sleep 1
done
export XDG_RUNTIME_DIR=/run/user/$(id -u)
exec ~/.local/bin/distrobox enter dev -- bash -c '
set -eu
creds=$1 addr=$2 vnc_port=$3 rdp_port=$4 display=$5 width=$6 height=$7
vncpasswd -f < "$creds/vnc-password" > "$creds/vnc-passwd.bin"
chmod 600 "$creds/vnc-passwd.bin"
Xvnc "$display" -geometry "${width}x${height}" -depth 24 \
-interface "$addr" -rfbport "$vnc_port" \
-SecurityTypes VncAuth -PasswordFile "$creds/vnc-passwd.bin" \
-AlwaysShared -desktop "Steam Frame (Frametop)" &
xvnc=$!
trap "kill $xvnc 2>/dev/null" EXIT
sleep 2
# Keep an RDP connection open inside the VNC screen. Reconnect if it drops.
# /cert:ignore is fine here: the connection never leaves this host.
while kill -0 $xvnc 2>/dev/null; do
DISPLAY=$display xfreerdp /v:"127.0.0.1:$rdp_port" /u:steamos /p:"$(cat "$creds/password")" \
/cert:ignore /size:"${width}x${height}" -decorations /f +clipboard >/dev/null 2>&1 || true
sleep 2
done
' vnc-bridge "$creds" "$addr" "$vnc_port" "$rdp_port" "$display" "$width" "$height"
setup/README.md
+116
View File
@@ -0,0 +1,116 @@
# setup
One-time setup for the Steam Frame itself: the `dev` build container and the Bluetooth fixes. `install.sh` runs both, so you only need this page for details, doing a step on its own, or troubleshooting. Run the scripts from the repo root, either in a terminal on the Frame (the usual way) or from a PC over SSH (see "Developing from a PC" in the README).
| Script | What it sets up | Needs host `sudo` |
| --- | --- | --- |
| `dev-container.sh` | The `dev` build container (Fedora 44 toolbox) with every package the projects need | No |
| `bluetooth/install.sh` | Persistent fixes so Bluetooth LE mice and keyboards reconnect | Yes |
## Build container
```
setup/dev-container.sh
```
It creates the `dev` distrobox if it's missing and installs the packages listed in the script, which is the source of truth for the container. It also links `/opt/steamvr` to the host's SteamVR, so OpenVR programs built there can find the runtime. It's safe to re-run, for example after adding a package to the list.
## Bluetooth LE mice and keyboards
### Why this is needed
On stock SteamOS 0.3.0, Bluetooth LE mice and keyboards that use private addresses, such as the Swiftpoint Z3, pair but never reconnect. After the device sleeps or the Frame reboots, it stays disconnected. Two separate problems cause this:
1. **BlueZ 5.79 never sets the `ADDRESS_RESOLUTION` device flag.** Without it, kernel 6.18 doesn't load the device's identity key into the Bluetooth controller, so the controller can't recognize the device's rotating private address and ignores it. This is fixed upstream in BlueZ commit `f1fb4f95f4` ("core: Fix not resolving addresses"), but SteamOS doesn't ship that fix yet.
2. **Some devices only know the Frame's public address.** The Z3 doesn't accept the Frame's identity key during pairing, so it can only reconnect to the Frame's fixed public address. SteamOS sets `Privacy = device` in `/etc/bluetooth/main.conf`, and bluetoothd turns privacy back on at every start.
Both settings reset whenever bluetoothd restarts or the Frame reboots, so they have to be reapplied every time. That's what this setup installs.
### What gets installed
| File on the Frame | Purpose |
| --- | --- |
| `/etc/steamframe/bt-fixups.sh` | Turns controller privacy off, then sets the `ADDRESS_RESOLUTION` flags (`0x6`) on every bonded LE device that has an identity key |
| `/etc/systemd/system/steamframe-bt-fixups.service` | Runs the script after every Bluetooth start |
The service runs **after** Bluetooth has started and never makes Bluetooth wait for it. That matters: SteamOS's `set-bluetooth-mac-address.service` gives the Bluetooth chip its address, and it needs `bluetooth.service` to finish starting first. SteamOS's own files, including `main.conf`, are left untouched, so system updates won't conflict.
### Step 1: have a password for sudo
The install writes to `/etc`, so it needs `sudo` and the `steamos` user's password.
- **On the headset:** `sudo` asks for the password in the terminal. If you've never set one, run `passwd` first.
- **From a PC over SSH:** there's no terminal on the Frame to ask in, so put the password in a `.env` file at the repo root:
```
steamos_root_pwd="your-password"
```
`.env` is gitignored and never synced to the Frame. The scripts send the password to `sudo` on stdin, never on a command line.
### Step 2: install the fixes
```
setup/bluetooth/install.sh install
```
It copies the files above into place (`/etc` survives SteamOS updates) and enables the service. `./install.sh` offers this same step.
### Step 3: pair your mouse or keyboard
Pair it the normal way: in Steam, **Settings → Bluetooth**. Then apply the fixes to the new device, either way:
```
setup/bluetooth/install.sh run
```
or, on the Frame, **Frametop Input Settings → Bluetooth → Apply Bluetooth fixes** (it asks for the password). Do this once per newly paired device. From then on, the service applies the fixes automatically at every boot.
If a device won't pair at all, turn the fixes on first (`run`), then pair again. With privacy on, some devices, the Z3 included, fail to finish connecting.
### Step 4: check it worked
In a terminal on the Frame (or over `ssh frame`):
```
systemctl status steamframe-bt-fixups.service # active (exited)
journalctl -b -u steamframe-bt-fixups --no-pager # "privacy turned off" (if it was on), then
# "Set device flag of <address> (LE Random)" per device
sleep 2 | btmgmt info | grep "current settings" # the list must NOT contain "privacy"
bluetoothctl info <address> | grep Connected # "yes" once the device is awake
```
The real test is to reboot the Frame, then move or click the device. It should reconnect within a few seconds, with no re-pairing.
### Troubleshooting
**The device paired but won't reconnect.** Run `setup/bluetooth/install.sh run` again, then wake the device. Check that its address appears in the service log. The script only flags devices that have an identity key: look for `[IdentityResolvingKey]` in `/var/lib/bluetooth/<controller>/<device>/info` (readable as root).
**No Bluetooth at all after a boot.** Check the controller:
```
hciconfig hci0 | head -3 # healthy: "BD Address: 90:82:C3:..." and "UP RUNNING"
```
If it shows `DOWN RAW` with address `00:00:00:00:5A:AD`, the address service failed. Start it by hand, then restart the fixes:
```
sudo systemctl start set-bluetooth-mac-address.service
sudo systemctl restart steamframe-bt-fixups.service
```
This happened once, while an earlier version of the fix made Bluetooth wait on it. Never add an `ExecStartPost` or anything else that holds up `bluetooth.service`.
**The mouse connects but does nothing in VR.** That's not Bluetooth. SteamVR only reads input devices that existed when it started. Frametop's input relay (`README.md`) handles this with permanent virtual devices.
### Uninstall
```
setup/bluetooth/install.sh uninstall
```
It removes both files and disables the service. Privacy returns to SteamOS's default at the next Bluetooth restart or reboot.
### Verified on
SteamOS 0.3.0 (build `20260922.6101926`), BlueZ 5.79, kernel 6.18, Qualcomm WCN7850 (`hci0`), with a Swiftpoint Z3, including cold-boot reconnects (2026-09-25).
setup/bluetooth/bt-fixups.sh
Executable
+44
View File
@@ -0,0 +1,44 @@
#!/bin/bash
# Installed to /etc/steamframe/bt-fixups.sh by setup/bluetooth/install.sh.
# Runs as root from steamframe-bt-fixups.service after every bluetoothd start,
# and by hand after pairing a new LE device: sudo /etc/steamframe/bt-fixups.sh
#
# Works around two things that stop Bluetooth LE mice like the Swiftpoint Z3
# from reconnecting:
# 1. BlueZ 5.79 never sets the ADDRESS_RESOLUTION device flag, so kernel 6.18
# never programs a bonded device's IRK into the controller's resolving list,
# and devices that advertise with private addresses never reconnect.
# Fixed upstream in BlueZ f1fb4f95f4 ("core: Fix not resolving addresses").
# Workaround: set flags 0x6 on every bonded LE device that has an IRK.
# 2. The Z3 doesn't accept the host's IRK when pairing, so it only knows the
# Frame's public address. Controller privacy has to be off.
set -u
# btmgmt quits before the reply unless stdin stays open for a moment.
mgmt() { sleep 2 | btmgmt --index 0 "$@" 2>&1; }
# Wait for the controller to come up and be powered on (can take a while after boot).
for _ in $(seq 60); do
mgmt info | grep -q "current settings:.* powered" && break
sleep 1
done
if mgmt info | grep -q "current settings:.* privacy"; then
mgmt power off >/dev/null
mgmt privacy off >/dev/null
mgmt power on >/dev/null
echo "privacy turned off"
fi
ctrl=$(mgmt info | awk '/^\s*addr /{print $2; exit}')
for info in /var/lib/bluetooth/"$ctrl"/*/info; do
[ -f "$info" ] || continue
grep -q '^\[IdentityResolvingKey\]' "$info" || continue
addr=$(basename "$(dirname "$info")")
case $(sed -n 's/^AddressType=//p' "$info") in
public) type=1 ;;
*) type=2 ;;
esac
echo "$(mgmt set-flags -f 0x6 -t "$type" "$addr" | tail -n 1)"
done
exit 0
setup/bluetooth/install.sh
Executable
+43
View File
@@ -0,0 +1,43 @@
#!/usr/bin/env bash
# Install (or remove) the persistent Bluetooth LE workarounds on the Frame.
# Needs host sudo. On the Frame, sudo asks for the password in the terminal.
# From a PC (or with no terminal), the password comes from steamos_root_pwd in the
# repo's .env and is sent to sudo -S on stdin, never on a command line.
# Usage: setup/bluetooth/install.sh [install|uninstall|run]
# run re-apply now without restarting bluetooth (after pairing a new device)
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
. "$root/scripts/_env.sh"
src=$FRAME_REPO/setup/bluetooth
sudo_run() {
if [ "$FRAME_LOCAL" = 1 ] && [ -t 0 ]; then
sudo bash -c "$1" # asks for the password here
return
fi
local pw
pw=$(sed -n 's/^steamos_root_pwd=//p' "$root/.env" 2>/dev/null)
pw=${pw#[\"\']}; pw=${pw%[\"\']} # .env values may be quoted
[ -n "$pw" ] || { echo "no terminal for sudo, and steamos_root_pwd is missing from $root/.env" >&2; exit 1; }
printf '%s\n' "$pw" | on_frame "sudo -S -p '' bash -c $(printf %q "$1")"
}
case ${1:-install} in
install)
"$root/scripts/sync.sh" >/dev/null
sudo_run "set -e
install -D -m 0755 -o root -g root $src/bt-fixups.sh /etc/steamframe/bt-fixups.sh
install -D -m 0644 -o root -g root $src/steamframe-bt-fixups.service /etc/systemd/system/steamframe-bt-fixups.service
rm -f /etc/systemd/system/bluetooth.service.d/steamframe.conf
rmdir /etc/systemd/system/bluetooth.service.d 2>/dev/null || true
systemctl daemon-reload
systemctl enable steamframe-bt-fixups.service
echo installed" ;;
uninstall)
sudo_run "systemctl disable steamframe-bt-fixups.service 2>/dev/null
rm -f /etc/systemd/system/steamframe-bt-fixups.service /etc/systemd/system/bluetooth.service.d/steamframe.conf /etc/steamframe/bt-fixups.sh
rmdir /etc/systemd/system/bluetooth.service.d /etc/steamframe 2>/dev/null; systemctl daemon-reload; echo removed" ;;
run) sudo_run "/etc/steamframe/bt-fixups.sh" ;;
*) echo "usage: $0 [install|uninstall|run]" >&2; exit 2 ;;
esac
setup/bluetooth/steamframe-bt-fixups.service
+17
View File
@@ -0,0 +1,17 @@
# Installed to /etc/systemd/system/ by setup/bluetooth/install.sh.
# Re-applies the Bluetooth LE workarounds after every bluetoothd start.
# A separate unit rather than ExecStartPost: the controller only powers up
# after bluetooth.service is active, so waiting inside it deadlocks.
[Unit]
Description=SteamFrame Bluetooth LE workarounds (address resolution, privacy off)
After=bluetooth.service set-bluetooth-mac-address.service
PartOf=bluetooth.service
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/etc/steamframe/bt-fixups.sh
TimeoutStartSec=5min
[Install]
WantedBy=bluetooth.service
setup/dev-container.sh
Executable
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# Create or update the "dev" build container on the Steam Frame (Fedora 44 toolbox,
# aarch64, via distrobox). Safe to re-run: it only creates what's missing and dnf
# skips installed packages. This package list is the source of truth for rebuilding
# the container.
# Usage: setup/dev-container.sh (on the Frame, or from a PC over SSH)
# Needs distrobox in ~/.local/bin on the Frame (see the top-level README).
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
. "$root/scripts/_env.sh"
packages=(
# toolchains
gcc gcc-c++ clang clang-devel cmake meson ninja-build make pkgconf-pkg-config cargo rust git
# libraries for Frametop's native pieces
pipewire-devel libxkbcommon-devel libinput-devel systemd-devel dbus-devel libdrm-devel
mesa-libgbm-devel wayland-devel vulkan-loader-devel vulkan-headers plasma-wayland-protocols wlroots-devel
# ft_pointer SteamVR driver: static C++ runtime (the host has an older glibc)
libstdc++-static
# Frametop Input Settings app (Kirigami, PySide6)
python3-pyside6 kf6-kirigami kf6-qqc2-desktop-style qt6-qtwayland breeze-icon-theme plasma-breeze
# Frametop remote desktop (VNC bridge through krdp)
krdp freerdp tigervnc-x11-server
# diagnostics and remote UI testing
wayland-utils xorg-x11-server-Xvfb ImageMagick xdotool
)
on_frame_script "${packages[@]}" <<'EOF'
set -euo pipefail
distrobox=$HOME/.local/bin/distrobox
[ -x "$distrobox" ] || { echo "distrobox not found at $distrobox (see the top-level README)" >&2; exit 1; }
if ! podman container exists dev; then
echo "creating the dev container (Fedora 44 toolbox)"
"$distrobox" create --yes --name dev --image registry.fedoraproject.org/fedora-toolbox:44
fi
"$distrobox" enter dev -- bash -c '
set -euo pipefail
echo "installing ${#@} packages (already-installed ones are skipped)"
sudo dnf install -y -q "$@" 2>&1 | { grep -vE "is already installed|^Nothing to do|^$" || true; }
# OpenVR programs built here (the pointer helper and probe) look for the runtime at /opt/steamvr.
[ -e /opt/steamvr ] || sudo ln -s /run/host/opt/steamvr /opt/steamvr
echo "dev container ready: $(. /etc/os-release; echo $PRETTY_NAME), glibc $(ldd --version | head -1 | grep -oE "[0-9.]+$")"
' dev "$@"
EOF