mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-06 18:00:11 +02:00
Compare commits
111
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4ba49af487 | ||
|
|
82fbc6fc1b | ||
|
|
8bd2ac4c92 | ||
|
|
ac692c38ea | ||
|
|
d6556ca813 | ||
|
|
9d66e36439 | ||
|
|
dac4188392 | ||
|
|
a9f1bcfdde | ||
|
|
effcfd0e48 | ||
|
|
3be79efade | ||
|
|
030ef2faf1 | ||
|
|
44c0b2d796 | ||
|
|
56aa3e4b4b | ||
|
|
977e9a0950 | ||
|
|
82979e4423 | ||
|
|
c119442c70 | ||
|
|
fb1c587d69 | ||
|
|
102e859cf8 | ||
|
|
b21dfd1bb0 | ||
|
|
bcf8516ebd | ||
|
|
db0d526f81 | ||
|
|
81f24a5d04 | ||
|
|
55f00f300d | ||
|
|
b2d32e8934 | ||
|
|
f6b4323669 | ||
|
|
903abd75dd | ||
|
|
f65afce59a | ||
|
|
1bb8e73cb9 | ||
|
|
0821b1013e | ||
|
|
3cccad3525 | ||
|
|
845bff0037 | ||
|
|
2ede132b62 | ||
|
|
a70463d95b | ||
|
|
a84889b4a5 | ||
|
|
8b9edcfbe2 | ||
|
|
0f1132ed27 | ||
|
|
4828dd46b5 | ||
|
|
d4e5e2b442 | ||
|
|
82da3525b7 | ||
|
|
2a2bca336c | ||
|
|
cd47dde566 | ||
|
|
a5d001d4b5 | ||
|
|
fa65954834 | ||
|
|
c747bea791 | ||
|
|
e918359c8f | ||
|
|
c53a4f07e9 | ||
|
|
cb34ba45a1 | ||
|
|
740aededff | ||
|
|
8655395030 | ||
|
|
6f65145553 | ||
|
|
cce37b5835 | ||
|
|
ff3724cb3c | ||
|
|
48bceaef64 | ||
|
|
c1ba71aba1 | ||
|
|
9c6647538b | ||
|
|
8a77364767 | ||
|
|
b61b4d6c6a | ||
|
|
edf605f896 | ||
|
|
b5fe574ae6 | ||
|
|
76d6542071 | ||
|
|
a94a0fb432 | ||
|
|
b1b1a16f5b | ||
|
|
276c1409e8 | ||
|
|
a0fe3bb55e | ||
|
|
13198bc203 | ||
|
|
42a26753d3 | ||
|
|
0f0674ff36 | ||
|
|
3a796b5013 | ||
|
|
67a3c02e31 | ||
|
|
662919babf | ||
|
|
98bd6ce21c | ||
|
|
608ad6034a | ||
|
|
336a60ce32 | ||
|
|
e1f27958ca | ||
|
|
c969963b83 | ||
|
|
9f2ee5ad68 | ||
|
|
dfc8e6eb5f | ||
|
|
1037d4e125 | ||
|
|
307e2b7308 | ||
|
|
d39fbb9146 | ||
|
|
7fe5d8c0dd | ||
|
|
6df6f97937 | ||
|
|
ca9492be00 | ||
|
|
41b5b68a41 | ||
|
|
9d7c8a0b91 | ||
|
|
5e3323c188 | ||
|
|
25504e0ed4 | ||
|
|
66160aed55 | ||
|
|
ec870d2d7a | ||
|
|
80797c98b9 | ||
|
|
2fadbbe148 | ||
|
|
5ee07f99a8 | ||
|
|
f0b93b79d3 | ||
|
|
13a530d829 | ||
|
|
f260a6a9e4 | ||
|
|
2e09cf9efe | ||
|
|
9a4b66ff50 | ||
|
|
17269134ce | ||
|
|
d995b815f8 | ||
|
|
f18918f781 | ||
|
|
e0208687b6 | ||
|
|
c699e13104 | ||
|
|
9b12b7d74e | ||
|
|
c7c7c1f772 | ||
|
|
2486a601e3 | ||
|
|
de25633f87 | ||
|
|
53a1836dbb | ||
|
|
6fb168a8b8 | ||
|
|
1ebf3692fb | ||
|
|
072a294941 | ||
|
|
7816633353 |
No files matched your search
@@ -25,8 +25,8 @@ scripts/frame.sh --host '<cmd>' # runs on the SteamOS host
|
|||||||
A Steam Frame is someone's personal headset, and they may be wearing it while you work.
|
A Steam Frame is someone's personal headset, and they may be wearing it while you work.
|
||||||
|
|
||||||
- Don't kill or restart `gamescope`, `steam`, `vrserver`, `vrcompositor`, the gamescope session, or the Frametop desktop without asking. Each one ends or disrupts whatever is happening in VR.
|
- Don't kill or restart `gamescope`, `steam`, `vrserver`, `vrcompositor`, the gamescope session, or the Frametop desktop without asking. Each one ends or disrupts whatever is happening in VR.
|
||||||
- Don't run host `sudo`, `steamos-readonly disable`, `steamos-devmode` changes, pacman installs, or reboots without explicit approval. Three installers need host `sudo`, and they ask for it: the Bluetooth fixes (`setup/bluetooth/install.sh`), hand tracking (`hands/run.sh install` and `caps`, which set ft-camd's file capabilities with `setcap`), and our own eye tracker's frame grabber (`gaze/tracker/install.sh`).
|
- Don't run host `sudo`, `steamos-readonly disable`, `steamos-devmode` changes, pacman installs, or reboots without explicit approval. Three installers need host `sudo`, and they ask for it: the Bluetooth fixes (`setup/bluetooth/install.sh`), hand tracking (`hands/run.sh install` and `caps`, which set ft-camd's file capabilities with `setcap`, and `uninstall` and `uncaps`, which take them back), and our own eye tracker's frame grabber (`gaze/tracker/install.sh`).
|
||||||
- Write only inside the repo, `/tmp`, and the container unless told otherwise. The installers are the exception: they write the user services, launchers, and the SteamVR driver into the home folder. The Bluetooth fixes and the eye tracker's frame grabber also install root-owned files and system services under `/etc` (`/etc/steamframe`, `/etc/frametop`, `/etc/systemd/system`).
|
- Write only inside the repo, `/tmp`, and the container unless told otherwise. The installers are the exception: they write the user services, launchers, and the SteamVR driver into the home folder. The Bluetooth fixes and the eye tracker's frame grabber also install root-owned files and system services under `/etc` (`/etc/steamframe`, `/etc/frametop`, `/etc/systemd/system`). When an installer starts writing something new outside the repo, or sets file capabilities, add it to `uninstall.sh` too: users uninstall with that script, not with each installer's `uninstall`.
|
||||||
- Never copy `.netrc`, SSH keys, or Steam config off the Frame or into this repo.
|
- Never copy `.netrc`, SSH keys, or Steam config off the Frame or into this repo.
|
||||||
|
|
||||||
## SteamOS updates
|
## SteamOS updates
|
||||||
|
|||||||
@@ -33,7 +33,7 @@ You need a Steam Frame with an internet connection, a keyboard (Bluetooth, or th
|
|||||||
|
|
||||||
The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks you four things along the way: whether to install gaze mode (experimental, yes by default), our own eye tracker for it (yes by default), and the Bluetooth fixes, then whether to restart SteamVR. The eye tracker and the Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
|
The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks you four things along the way: whether to install gaze mode (experimental, yes by default), our own eye tracker for it (yes by default), and the Bluetooth fixes, then whether to restart SteamVR. The eye tracker and the Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
|
||||||
|
|
||||||
After the restart, Launch a program → Desktop opens the multi-screen desktop, with its screens arranged around where you're facing. Frametop Display Settings and Frametop Input Settings are in the desktop's application menu, under Settings.
|
After the restart, Launch a program → Desktop opens the multi-screen desktop, with its screens arranged around where you're facing. Frametop Display Settings and Frametop Input Settings are in the desktop's application menu, under Settings. SteamOS's own single-screen desktop is still there, as Native Desktop in the same list.
|
||||||
|
|
||||||
If you work in the desktop for long stretches, or leave the headset on a stand, open Frametop Display Settings → Power. Turn on Stay awake while plugged in: by default Steam puts the Frame to sleep after an hour without input, even while it charges. And choose when the displays turn off while the headset isn't used. SteamVR turns them off a few seconds after you take the headset off, but a stand or mount that covers the proximity sensor inside it makes the headset seem worn, and its displays stay on all night.
|
If you work in the desktop for long stretches, or leave the headset on a stand, open Frametop Display Settings → Power. Turn on Stay awake while plugged in: by default Steam puts the Frame to sleep after an hour without input, even while it charges. And choose when the displays turn off while the headset isn't used. SteamVR turns them off a few seconds after you take the headset off, but a stand or mount that covers the proximity sensor inside it makes the headset seem worn, and its displays stay on all night.
|
||||||
|
|
||||||
@@ -57,6 +57,7 @@ If you work in the desktop for long stretches, or leave the headset on a stand,
|
|||||||
| While carrying a screen, sweep its laser across your other controller's ring, then let go | Pins it to that wrist, at its size and distance, as you hold it when you let go; it shows while you see its front. Grab its bar to adjust it (it stays pinned); sweep across the ring again to take it off |
|
| While carrying a screen, sweep its laser across your other controller's ring, then let go | Pins it to that wrist, at its size and distance, as you hold it when you let go; it shows while you see its front. Grab its bar to adjust it (it stays pinned); sweep across the ring again to take it off |
|
||||||
| Set a screen to On your head (Frametop Display Settings, Visibility & pins) | Pins it to your head where it is, like a HUD. Grab its bar to move it; it stays on your head |
|
| Set a screen to On your head (Frametop Display Settings, Visibility & pins) | Pins it to your head where it is, like a HUD. Grab its bar to move it; it stays on your head |
|
||||||
| Meta+Shift+R in the desktop | Puts the screens back in their layout (also in the menu as Reset Screen Layout, and mappable to a mouse button) |
|
| Meta+Shift+R in the desktop | Puts the screens back in their layout (also in the menu as Reset Screen Layout, and mappable to a mouse button) |
|
||||||
|
| Meta+Alt+Tab, or Meta+Alt+Shift+Tab | Spins every screen and floating window around you together, like a lazy susan, so the next one on your right (or left) glides to straight ahead, with the pointer and typing going to it. Their arrangement stays the same: the room turns instead of you. Tap again to keep going; Meta+Shift+R puts the screens back. Pinned screens stay where they are |
|
||||||
| Meta+Shift+H in the desktop | Hides or shows all screens (also in the menu as Hide/Show Screens, and mappable). The Visibility & pins tab of Frametop Display Settings can instead show them only with the dashboard open, or while you look at your wrist |
|
| Meta+Shift+H in the desktop | Hides or shows all screens (also in the menu as Hide/Show Screens, and mappable). The Visibility & pins tab of Frametop Display Settings can instead show them only with the dashboard open, or while you look at your wrist |
|
||||||
| Tap Meta, on any keyboard | Opens the Steam menu in the SteamVR dashboard, or closes the dashboard, wherever you are. The desktop's launcher is still on the taskbar and Alt+F1. Change it in Frametop Input Settings (Keyboard page), where any key combination or modifier tap can do a Frametop or Steam action, open a profile, or run a command of your own |
|
| Tap Meta, on any keyboard | Opens the Steam menu in the SteamVR dashboard, or closes the dashboard, wherever you are. The desktop's launcher is still on the taskbar and Alt+F1. Change it in Frametop Input Settings (Keyboard page), where any key combination or modifier tap can do a Frametop or Steam action, open a profile, or run a command of your own |
|
||||||
| Switch a screen to Hidden (Frametop Display Settings, Visibility & pins → Screens shown) | Hides just that screen until you switch it back, whatever the other visibility settings say; new windows that would open on it float instead |
|
| Switch a screen to Hidden (Frametop Display Settings, Visibility & pins → Screens shown) | Hides just that screen until you switch it back, whatever the other visibility settings say; new windows that would open on it float instead |
|
||||||
@@ -170,23 +171,22 @@ Or by hand: `cd ~/frametop && git pull && ./install.sh`.
|
|||||||
|
|
||||||
## Uninstall
|
## Uninstall
|
||||||
|
|
||||||
|
In a terminal on the headset, run:
|
||||||
|
|
||||||
```
|
```
|
||||||
./desktops.sh uninstall # the launcher's Desktop entry goes back to the stock desktop
|
curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash
|
||||||
./desktops.sh relay uninstall
|
|
||||||
pointer/helper/run.sh uninstall
|
|
||||||
power/run.sh uninstall
|
|
||||||
pointer/driver/install.sh uninstall # then restart SteamVR
|
|
||||||
input-settings/install.sh uninstall
|
|
||||||
display-settings/install.sh uninstall
|
|
||||||
remote/install.sh uninstall
|
|
||||||
setup/bluetooth/install.sh uninstall # if you installed the Bluetooth fixes
|
|
||||||
hands/run.sh uninstall # if you installed hand tracking by hand
|
|
||||||
gaze/run.sh uninstall # if you installed the gaze service
|
|
||||||
gaze/tracker/install.sh uninstall # if you installed our own eye tracker's frame grabber
|
|
||||||
gaze/probe/install.sh uninstall # if you installed the gaze probe
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Your settings stay: `~/.config/frametop.conf`, `frametop-input.json` (button maps and key combinations), `frametop-layout.json` (the layout and profiles), `frametop-float.json`, and `frametop-remote/` in `~/.config`, and the gaze calibration in `~/.local/state/frametop/gaze`. So does the desktop's own Plasma setup, in `~/.config/frametop`. Delete them too for a clean slate.
|
It works in two steps, so it never takes away the keyboard, mouse, or desktop you're using while it runs:
|
||||||
|
|
||||||
|
1. It stops Frametop from starting. Launch a program → Desktop opens the stock desktop again, and the Native Desktop entry, Frametop's services, its SteamVR driver, and its menu entries are removed, along with the system files of our eye tracker and the Bluetooth fixes and the file capabilities of hand tracking's camera broker (those need your `sudo` password). Everything running now keeps running until you restart the headset, and it offers to restart it for you.
|
||||||
|
2. After the restart, run the same command again. It deletes the code in `~/frametop`, and asks whether to delete your settings, any eye or hand recordings, and the build container (1–2 GB) too.
|
||||||
|
|
||||||
|
To see what it would do without changing anything, add `-s -- --dry-run` after `bash`. If the code isn't in `~/frametop`, add `-s -- --dir <folder>`. From the repo, the same script is `./uninstall.sh`.
|
||||||
|
|
||||||
|
Don't delete `~/frametop` by hand before you uninstall and restart: the desktop and the input relay run from it, and without it Launch a program → Desktop no longer opens anything. If you've already deleted it, the command above still works, since it doesn't need the repo.
|
||||||
|
|
||||||
|
Unless you ask for them to go, your settings stay: `~/.config/frametop.conf`, `frametop-input.json` (button maps and key combinations), `frametop-layout.json` (the layout and profiles), `frametop-float.json`, and `frametop-remote/` in `~/.config`, the gaze calibration in `~/.local/state/frametop`, and the desktop's own Plasma setup in `~/.config/frametop`. A later install picks them up again.
|
||||||
|
|
||||||
## How it works
|
## How it works
|
||||||
|
|
||||||
@@ -196,6 +196,7 @@ A Plasma session runs nested inside ft-screens (`screens/`), a small Wayland com
|
|||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `get.sh` | The one-line installer: picks stable or experimental, clones or updates the repo, and runs `install.sh`. |
|
| `get.sh` | The one-line installer: picks stable or experimental, clones or updates the repo, and runs `install.sh`. |
|
||||||
| `install.sh` | The one-step installer. Safe to re-run. |
|
| `install.sh` | The one-step installer. Safe to re-run. |
|
||||||
|
| `uninstall.sh` | The uninstaller: run it, restart the headset, and run it again. It doesn't need the rest of the repo. |
|
||||||
| `desktops.sh` | Start, stop, and configure the desktop, and install the input relay. |
|
| `desktops.sh` | Start, stop, and configure the desktop, and install the input relay. |
|
||||||
| `screens/` | ft-screens, the compositor (wlroots and OpenVR). |
|
| `screens/` | ft-screens, the compositor (wlroots and OpenVR). |
|
||||||
| `session/` | The desktop session script and its config example. |
|
| `session/` | The desktop session script and its config example. |
|
||||||
|
|||||||
+13
-2
@@ -1,7 +1,8 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# Start, stop, or inspect the multi-screen Plasma desktop in VR on the Frame.
|
# Start, stop, or inspect the multi-screen Plasma desktop in VR on the Frame.
|
||||||
# Usage: desktops.sh start [screens] | stop | restart | status | log [lines]
|
# Usage: desktops.sh start [screens] | stop | restart | status | log [lines]
|
||||||
# desktops.sh install # make the VR launcher's "Desktop" entry start Frametop
|
# desktops.sh install # make the VR launcher's "Desktop" entry start Frametop, and add
|
||||||
|
# # "Native Desktop" for the stock SteamOS desktop
|
||||||
# desktops.sh uninstall # give the launcher back the stock SteamOS desktop
|
# desktops.sh uninstall # give the launcher back the stock SteamOS desktop
|
||||||
# desktops.sh screens N # set the default screen count in ~/.config/frametop.conf
|
# desktops.sh screens N # set the default screen count in ~/.config/frametop.conf
|
||||||
# desktops.sh remote on|off|info # VNC access over the tailnet (applies on next start)
|
# desktops.sh remote on|off|info # VNC access over the tailnet (applies on next start)
|
||||||
@@ -16,6 +17,8 @@ action=${1:-start}
|
|||||||
screens=${2:-${FT_SCREENS:-}}
|
screens=${2:-${FT_SCREENS:-}}
|
||||||
session=$FRAME_REPO/session
|
session=$FRAME_REPO/session
|
||||||
override=.local/share/applications/deckard-nested-desktop.desktop
|
override=.local/share/applications/deckard-nested-desktop.desktop
|
||||||
|
native_copy=.local/share/applications/native-deckard-nested-desktop.desktop
|
||||||
|
stock=/usr/share/applications/deckard-nested-desktop.desktop
|
||||||
log=/tmp/frametop-session.log
|
log=/tmp/frametop-session.log
|
||||||
# Bracketed first letter so pgrep/pkill never match the ssh shell running them.
|
# Bracketed first letter so pgrep/pkill never match the ssh shell running them.
|
||||||
match='[v]r-overlay-key frametop '
|
match='[v]r-overlay-key frametop '
|
||||||
@@ -35,15 +38,23 @@ systemd-run --user --collect --quiet --unit frametop-desktop \
|
|||||||
sleep 12; echo \"plasmashell processes: \$(pgrep -c plasmashell)\"
|
sleep 12; echo \"plasmashell processes: \$(pgrep -c plasmashell)\"
|
||||||
$running && echo 'started' || { echo 'failed:'; tail -20 $log; exit 1; }" ;;
|
$running && echo 'started' || { echo 'failed:'; tail -20 $log; exit 1; }" ;;
|
||||||
install)
|
install)
|
||||||
|
# Also "Native Desktop", a copy of SteamOS's entry for its own desktop. Optional, so a SteamOS
|
||||||
|
# without the stock entry still installs. No X-Steam-Special, so Steam can only single out ours.
|
||||||
"$root/scripts/sync.sh" >/dev/null
|
"$root/scripts/sync.sh" >/dev/null
|
||||||
"$frame" --host "set -e; mkdir -p ~/.local/share/applications
|
"$frame" --host "set -e; mkdir -p ~/.local/share/applications
|
||||||
sed 's|@SESSION@|$session/frametop-session.sh|' $session/deckard-nested-desktop.desktop > ~/$override
|
sed 's|@SESSION@|$session/frametop-session.sh|' $session/deckard-nested-desktop.desktop > ~/$override
|
||||||
|
if [ -r $stock ]; then
|
||||||
|
sed -e 's/^Name=.*/Name=Native Desktop/' -e '/^Name\\[/d' -e '/^X-Steam-Special=/d' -e '/pick this out/d' \\
|
||||||
|
$stock > ~/$native_copy.new && mv ~/$native_copy.new ~/$native_copy
|
||||||
|
else
|
||||||
|
rm -f ~/$native_copy; echo 'no $stock: no Native Desktop entry' >&2
|
||||||
|
fi
|
||||||
[ -f ~/.config/frametop.conf ] || cp $session/frametop.conf.example ~/.config/frametop.conf
|
[ -f ~/.config/frametop.conf ] || cp $session/frametop.conf.example ~/.config/frametop.conf
|
||||||
echo \"installed ~/$override\"; grep ^Exec= ~/$override; echo; cat ~/.config/frametop.conf" ;;
|
echo \"installed ~/$override\"; grep ^Exec= ~/$override; echo; cat ~/.config/frametop.conf" ;;
|
||||||
uninstall)
|
uninstall)
|
||||||
# Also what the session puts in place at each start: Launch as Standalone's app copies and
|
# Also what the session puts in place at each start: Launch as Standalone's app copies and
|
||||||
# the title bar decoration (float/ft_apps.py, decoration/).
|
# the title bar decoration (float/ft_apps.py, decoration/).
|
||||||
"$frame" --host "rm -f ~/$override
|
"$frame" --host "rm -f ~/$override ~/$native_copy
|
||||||
rm -rf ~/.local/share/frametop/apps ~/.local/share/kwin/decorations/kwin4_decoration_qml_frametop
|
rm -rf ~/.local/share/frametop/apps ~/.local/share/kwin/decorations/kwin4_decoration_qml_frametop
|
||||||
rmdir ~/.local/share/frametop 2>/dev/null; echo 'removed; the launcher uses the stock desktop again'" ;;
|
rmdir ~/.local/share/frametop 2>/dev/null; echo 'removed; the launcher uses the stock desktop again'" ;;
|
||||||
screens)
|
screens)
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ Type=Application
|
|||||||
Name=Reset Screen Layout
|
Name=Reset Screen Layout
|
||||||
GenericName=Put the VR desktop's screens back in their layout
|
GenericName=Put the VR desktop's screens back in their layout
|
||||||
Comment=Float the screens and arrange them in the layout from Frametop Display Settings
|
Comment=Float the screens and arrange them in the layout from Frametop Display Settings
|
||||||
Exec=@REPO@/layout/ft-layout apply
|
Exec=@REPO@/layout/ft-layout-reset
|
||||||
Icon=view-restore
|
Icon=view-restore
|
||||||
Categories=Settings;
|
Categories=Settings;
|
||||||
Keywords=display;screen;layout;arrange;reset;steamvr;frametop;
|
Keywords=display;screen;layout;arrange;reset;steamvr;frametop;
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ Type=Application
|
|||||||
Name=Hide/Show Screens
|
Name=Hide/Show Screens
|
||||||
GenericName=Hide or show the VR desktop's screens
|
GenericName=Hide or show the VR desktop's screens
|
||||||
Comment=Hide the screens (and SteamVR's laser) for a VR game; press again to bring them back
|
Comment=Hide the screens (and SteamVR's laser) for a VR game; press again to bring them back
|
||||||
Exec=@REPO@/layout/ft-layout toggle
|
Exec=@REPO@/layout/ft-hide-show
|
||||||
Icon=view-visible
|
Icon=view-visible
|
||||||
Categories=Settings;
|
Categories=Settings;
|
||||||
Keywords=display;screen;hide;show;steamvr;frametop;
|
Keywords=display;screen;hide;show;steamvr;frametop;
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Controller desktop click stability
|
# Controller desktop click stability
|
||||||
|
|
||||||
Trigger presses reach KDE immediately, but controller motion within 8 logical
|
Trigger presses reach KDE immediately, but controller motion within 32 logical
|
||||||
pixels of the press stays at that position until release. Releasing without a motion outside this
|
pixels of the press stays at that position until release. Releasing without a motion outside this
|
||||||
zone delivers the click at the original position, even if the hand moved during
|
zone delivers the click at the original position, even if the hand moved during
|
||||||
release. Moving outside the zone begins a normal drag immediately; returning to
|
release. Moving outside the zone begins a normal drag immediately; returning to
|
||||||
@@ -13,11 +13,11 @@ floating-app title-bar carrying are unaffected. Multi-button gestures keep their
|
|||||||
existing behavior. A motion onto another desktop monitor starts a drag;
|
existing behavior. A motion onto another desktop monitor starts a drag;
|
||||||
cross-monitor motion is not stabilized.
|
cross-monitor motion is not stabilized.
|
||||||
|
|
||||||
CLI (runtime preferences, reset to 8 on desktop restart):
|
CLI (runtime preferences, reset to 32 on desktop restart):
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
input/ft-clickctl status
|
input/ft-clickctl status
|
||||||
input/ft-clickctl threshold 8
|
input/ft-clickctl threshold 32
|
||||||
input/ft-clickctl threshold 0 # disable without a restart
|
input/ft-clickctl threshold 0 # disable without a restart
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -29,6 +29,6 @@ This is a separate contribution from desktop mouse/controller ownership. Its
|
|||||||
hardware validation must check small controls, intentional text selection,
|
hardware validation must check small controls, intentional text selection,
|
||||||
long presses, cross-monitor dragging and simultaneous mouse use. The existing
|
long presses, cross-monitor dragging and simultaneous mouse use. The existing
|
||||||
renderer laser remains tracked; this change stabilizes desktop input rather
|
renderer laser remains tracked; this change stabilizes desktop input rather
|
||||||
than smoothing the visual laser. Default threshold is a starting point to test.
|
than smoothing the visual laser. The default was 8 at first. That's about 0.2° on a 3.4 m wide 3440-pixel screen 2 m away, and clicking took a very still hand, so it's 32 (about 0.9°) since 2026-10-03.
|
||||||
|
|
||||||
Run `scripts/test-controller-click.sh` for the isolated gesture-state tests.
|
Run `scripts/test-controller-click.sh` for the isolated gesture-state tests.
|
||||||
+23
-7
@@ -38,7 +38,7 @@ The curved layout chains screens edge to edge, like monitors on a desk: the midd
|
|||||||
|
|
||||||
A resize handle has to be able to shrink a screen from any direction, so the dragged corner follows the laser along the screen's diagonal rather than taking the larger of its horizontal and vertical reach. Pushing and pulling a carried screen moves it along the line from your head, because the 3D mouse's virtual controller sits just in front of the bar, below the screen's centre, so the line from the device points mostly upward.
|
A resize handle has to be able to shrink a screen from any direction, so the dragged corner follows the laser along the screen's diagonal rather than taking the larger of its horizontal and vertical reach. Pushing and pulling a carried screen moves it along the line from your head, because the 3D mouse's virtual controller sits just in front of the bar, below the screen's centre, so the line from the device points mostly upward.
|
||||||
|
|
||||||
Wherever ft-screens needs to know where a laser points (showing the controls, the resize tab, the roll knob), it uses the laser's own pose, the render model's `tip` component, rather than the controller's pose. On the Frame's controllers the tip points 40° below the pose's forward axis, so rays from the pose missed what the laser was actually on. The 3D mouse's virtual controller has no tip, and its laser runs along its pose.
|
Wherever ft-screens needs to know where a laser points (showing the controls, the resize tab, the roll knob), it uses the laser's own pose, the render model's `tip` component, rather than the controller's pose. On the Frame's controllers the tip points 40° below the pose's forward axis, so rays from the pose missed what the laser was actually on. The 3D mouse's virtual controller has no tip, and its laser runs along its pose. ft-screens reads the tip with `GetComponentState`: `GetComponentStateForDevicePath` without an input source handle fails for every component while a VR game runs, so in games the rays came from the pose, 40° too high.
|
||||||
|
|
||||||
`ComputeOverlayIntersection` ignores `SetOverlayIntersectionMask`, and a control can't be allowed to cover part of its screen, so the resize tab sits entirely outside the corner.
|
`ComputeOverlayIntersection` ignores `SetOverlayIntersectionMask`, and a control can't be allowed to cover part of its screen, so the resize tab sits entirely outside the corner.
|
||||||
|
|
||||||
@@ -62,7 +62,7 @@ A profile's screen part is the custom arrangement under a name: each screen's po
|
|||||||
|
|
||||||
`IVRApplications::GetCurrentSceneProcessId()` is 0 when no game is running (the Frame's home environment isn't a scene app) and the game's process ID while one is. ft-screens checks it twice a second, turns the flag off while a game runs, and by default hides the screens unless the dashboard is open. Flatscreen games run inside Steam's gamescope overlay and aren't scene apps, which is why "only with the dashboard open" is offered as a controller setting.
|
`IVRApplications::GetCurrentSceneProcessId()` is 0 when no game is running (the Frame's home environment isn't a scene app) and the game's process ID while one is. ft-screens checks it twice a second, turns the flag off while a game runs, and by default hides the screens unless the dashboard is open. Flatscreen games run inside Steam's gamescope overlay and aren't scene apps, which is why "only with the dashboard open" is offered as a controller setting.
|
||||||
|
|
||||||
The reset button needs to work in a game, where the screens have the flag off. So ft-screens turns the flag on for that button's overlay alone while a hand controller aims within about one button's width of it, and off half a second after the aim leaves a zone twice as wide. ft-screens finds the aim from the controllers' laser poses, which it reads anyway to show the controls, so it doesn't need SteamVR's laser to be on first. The game loses the controllers only while you aim at the button.
|
In a game, Frametop's panels work like SteamVR's own floating windows: point a controller at one and its laser comes on, point away and the game has the controllers again. ft-screens turns the flag on for a panel while a hand controller's laser pose meets it, its controls, or a floating window's popups. It finds that from the poses it already reads to show the controls, so SteamVR's laser doesn't have to be on first. Leaving takes a margin two control-sizes wide and 0.3 s, a drag or a held button keeps the flag on, and the keyboard, a single overlay, uses SteamVR's `ComputeOverlayIntersection`. The 3D mouse doesn't need any of this: it has its own laser mode.
|
||||||
|
|
||||||
## Floating windows
|
## Floating windows
|
||||||
|
|
||||||
@@ -85,12 +85,13 @@ The reset button needs to work in a game, where the screens have the flag off. S
|
|||||||
|
|
||||||
### The KWin script
|
### The KWin script
|
||||||
|
|
||||||
The KWin side is a script (`float/frametop-float.js`), not a C++ effect, because a script keeps working across KWin updates and an effect would have to match the host's exact KWin build. KWin scripts can call D-Bus but can't serve it, so ft-floatd's commands come back through a long poll: the script calls `NextCommand`, which answers when a command is ready, or empty after 20 seconds, under KWin's 25-second D-Bus timeout. A few things about KWin's script engine:
|
The KWin side is a script (`float/frametop-float.js`), not a C++ effect, because a script keeps working across KWin updates and an effect would have to match the host's exact KWin build. KWin scripts can call D-Bus but can't serve it, so ft-floatd's commands come back through a long poll: the script calls `NextCommand`, which answers when a command is ready, or empty after 20 seconds, under KWin's 25-second D-Bus timeout. If no reply comes within 30 seconds, a watchdog calls again, and waits twice as long each time until a reply comes (up to 5 minutes). The 30 seconds have to stay above KWin's timeout, or a late reply would start a second poll. A few things about KWin's script engine:
|
||||||
|
|
||||||
- `windowAdded` reports popups as windows of their own (`popupWindow` true, `transient` true) with their geometry.
|
- `windowAdded` reports popups as windows of their own (`popupWindow` true, `transient` true) with their geometry.
|
||||||
- Setting `frameGeometry` applies asynchronously: the app has to answer the new size first.
|
- Setting `frameGeometry` applies asynchronously: the app has to answer the new size first.
|
||||||
- A script can't read a window's maximize mode, so the script counts a window as maximized when it fills its output's maximize area.
|
- A script can't read a window's maximize mode, so the script counts a window as maximized when it fills its output's maximize area.
|
||||||
- `globalThis` isn't defined. `print` goes to the journal unless `QT_FORCE_STDERR_LOGGING=1`.
|
- `globalThis` isn't defined. `print` goes to the journal unless `QT_FORCE_STDERR_LOGGING=1`.
|
||||||
|
- `callDBus` never calls back when a call fails (an error reply, the name gone from the bus, or the 25-second timeout). KWin only logs `Received D-Bus message is error`, so a script that waits for the callback waits forever.
|
||||||
|
|
||||||
The title bar's float button is Frametop's own window decoration (`decoration/`), written in QML for KWin's Aurorae engine, which loads it without compiling. A C++ fork of Breeze would have to match SteamOS's exact KDecoration build. A decoration can only make the window requests KWin offers it, so the button toggles keep-below, which has no visible effect on a window alone on its own output, and the script treats keep-below as the floating flag.
|
The title bar's float button is Frametop's own window decoration (`decoration/`), written in QML for KWin's Aurorae engine, which loads it without compiling. A C++ fork of Breeze would have to match SteamOS's exact KDecoration build. A decoration can only make the window requests KWin offers it, so the button toggles keep-below, which has no visible effect on a window alone on its own output, and the script treats keep-below as the floating flag.
|
||||||
|
|
||||||
@@ -133,7 +134,7 @@ A few overlays need special handling:
|
|||||||
|
|
||||||
Head follow is experimental and off by default. It works, but it's only lightly tested, and the feel is mostly a matter of its settings; polishing it is left open. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction, where you were facing when your head last settled, and keeps its offset from it. The mouse can put the cursor anywhere up to `POINTER_FOLLOW_REACH` (70 degrees) from the reference, a corner of your view included. While your head stays within `POINTER_LEASH_DEG` of the reference, nothing moves on its own. Once your head has been past the leash for `POINTER_LEASH_DELAY` (0.2 s, so a glance out and back doesn't count), the reference eases to where you're facing (time constant `POINTER_LEASH_RETURN`, 0.2 s), never falling further behind than the leash, and the cursor ends up back where it was in your view. Then it waits for the leash again. Two earlier versions didn't work out. Moving the reference only while your head pulled at the end of the leash left it up to the leash off after you turned back, and getting it centred again meant overshooting with your head. Easing it toward your facing all the time moved the cursor on every small head movement. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping.
|
Head follow is experimental and off by default. It works, but it's only lightly tested, and the feel is mostly a matter of its settings; polishing it is left open. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction, where you were facing when your head last settled, and keeps its offset from it. The mouse can put the cursor anywhere up to `POINTER_FOLLOW_REACH` (70 degrees) from the reference, a corner of your view included. While your head stays within `POINTER_LEASH_DEG` of the reference, nothing moves on its own. Once your head has been past the leash for `POINTER_LEASH_DELAY` (0.2 s, so a glance out and back doesn't count), the reference eases to where you're facing (time constant `POINTER_LEASH_RETURN`, 0.2 s), never falling further behind than the leash, and the cursor ends up back where it was in your view. Then it waits for the leash again. Two earlier versions didn't work out. Moving the reference only while your head pulled at the end of the leash left it up to the leash off after you turned back, and getting it centred again meant overshooting with your head. Easing it toward your facing all the time moved the cursor on every small head movement. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping.
|
||||||
|
|
||||||
Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. By default (`POINTER_GAZE_MOUSE_MOVE=held`, the Gaze page's Mouse movement switch) moving the mouse does nothing while the gaze has the pointer: it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so the lessons aren't polluted by mouse moves to somewhere else (they used to be kept out by an 8 degree limit, which also dropped real corrections when the tracker was further off). With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual; with `free`, moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: it needs accessibility (AT-SPI) on in the Frametop session, where it's off (no registry runs), plus app restarts, and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. The right button works the same way, with the right click on the release, and pressing it while the left press is held back starts a drag where the pointer is, like Meta+J then Meta+K. That drag lasts while either button (or key) is held, so a second right press, or a second Meta+K, is free to pan and tilt the panel being dragged; with the keyboard, the head turns it. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge before a click whose correction is within `POINTER_GAZE_NUDGE_MAX` (55 degrees, half of what the headset shows across) is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on, when our tracker thinks it moved, and when a correction is past that limit (the tracker is far off, so a click there isn't trusted as a lesson), and the full calibration and the headset fit check run in the same panel, so everything a user does to calibrate happens in one place in the headset; the gaze probe, a fullscreen GTK app, is the development tool. The limit was 8 degrees, which dropped every correction while our tracker was 12 off. Its dots sit at known directions from the headset, so the panel needs no screen geometry. The quick check's dot takes the gaze once it has held still, so what the tracker says doesn't have to be close for the capture to work. The full calibration's and the five-dot check's dots wait for a click while you look at the dot (a left click or Meta+J), because a steady gaze isn't always on the dot, and take the gaze held still up to the click; a right click or MetLine truncated
|
Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. By default (`POINTER_GAZE_MOUSE_MOVE=held`, the Gaze page's Mouse movement switch) moving the mouse does nothing while the gaze has the pointer: it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so the lessons aren't polluted by mouse moves to somewhere else (they used to be kept out by an 8 degree limit, which also dropped real corrections when the tracker was further off). With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual; with `free`, moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: the session now starts an AT-SPI registry, but apps still need to expose useful accessibility trees (and may need restarting), and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. The right button works the same way, with the right click on the release, and pressing it while the left press is held back starts a drag where the pointer is, like Meta+J then Meta+K. That drag lasts while either button (or key) is held, so a second right press, or a second Meta+K, is free to pan and tilt the panel being dragged; with the keyboard, the head turns it. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge before a click whose correction is within `POINTER_GAZE_NUDGE_MAX` (55 degrees, half of what the headset shows across) is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on, when our tracker thinks it moved, and when a correction is past that limit (the tracker is far off, so a click there isn't trusted as a lesson), and the full calibration and the headset fit check run in the same panel, so everything a user does to calibrate happens in one place in the headset; the gaze probe, a fullscreen GTK app, is the development tool. The limit was 8 degrees, which dropped every correction while our tracker was 12 off. Its dots sit at known directions from the headset, so the panel needs no screen geometry. The quick check's dot takes the gaze once it has held still, so what the tracker says doesn't have to be close for the capture to work. The full calibration's and the five-dot check's dots wait for a click while you look at the dot (a left click or Meta+J), because a steady gaze isn't always on the dot, and take the gaze held still up to the click; a rightLine truncated
|
||||||
|
|
||||||
Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place.
|
Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place.
|
||||||
|
|
||||||
@@ -159,6 +160,8 @@ The relay never waits on the pointer helper. Its socket to the helper used to bl
|
|||||||
|
|
||||||
An ungrabbed keyboard reaches both sides at once. In VR, gamescope reads every input device itself (the SteamOS build's `InputStealer`, libinput with udev hotplug, so new devices too) and types into its focused app, and ft-screens types the same keys into the desktop. So Space in the desktop also paused Spotify on the dashboard. Typing now follows the last click. ft-screens sees clicks on its own screens, from the mouse or a controller. A click anywhere else is only visible for the mouse: overlay apps get SteamVR's `OverlayFocusChanged` (which panel the laser is on) but no controller button events, so the pointer helper reports the panel under the dot on each left press. ft-screens tells the relay where typing goes every second, from an unbound socket so the relay's replies can't loop back into its control socket, and the relay grabs pass-through keyboards while it's the desktop. A grab waits until the keyboard has no key down, so no key stays held on either side, and the relay lets go if ft-screens stops reporting. A program that reads every keyboard for a hotkey (a dictation tool, say) loses a grabbed keyboard. Repeating the keys on another input device doesn't work: gamescope reads that device too, whether it's the relay's virtual keyboard or one created later, and every Space, typed or dictated, paused Spotify again. So with `SHARE_KEYS=1` the relay sends a grabbed keyboard's keys to `@frametop_keys` as datagrams (`key <code> <value> <device name>`). It's off by default, because the relay can't tell who is listening: abstract sockets have no permissions, and any local process that binds the name first gets every key typed into the desktop, passwords included. A listener should accept only its own user (`SO_PASSCRED`) and skip any keyboard of its own that the relay grabs too.
|
An ungrabbed keyboard reaches both sides at once. In VR, gamescope reads every input device itself (the SteamOS build's `InputStealer`, libinput with udev hotplug, so new devices too) and types into its focused app, and ft-screens types the same keys into the desktop. So Space in the desktop also paused Spotify on the dashboard. Typing now follows the last click. ft-screens sees clicks on its own screens, from the mouse or a controller. A click anywhere else is only visible for the mouse: overlay apps get SteamVR's `OverlayFocusChanged` (which panel the laser is on) but no controller button events, so the pointer helper reports the panel under the dot on each left press. ft-screens tells the relay where typing goes every second, from an unbound socket so the relay's replies can't loop back into its control socket, and the relay grabs pass-through keyboards while it's the desktop. A grab waits until the keyboard has no key down, so no key stays held on either side, and the relay lets go if ft-screens stops reporting. A program that reads every keyboard for a hotkey (a dictation tool, say) loses a grabbed keyboard. Repeating the keys on another input device doesn't work: gamescope reads that device too, whether it's the relay's virtual keyboard or one created later, and every Space, typed or dictated, paused Spotify again. So with `SHARE_KEYS=1` the relay sends a grabbed keyboard's keys to `@frametop_keys` as datagrams (`key <code> <value> <device name>`). It's off by default, because the relay can't tell who is listening: abstract sockets have no permissions, and any local process that binds the name first gets every key typed into the desktop, passwords included. A listener should accept only its own user (`SO_PASSCRED`) and skip any keyboard of its own that the relay grabs too.
|
||||||
|
|
||||||
|
A mouse button that pointer mode passes through as a key (a side button for Back) is a pointer button to ft-screens, not a key, so it goes to the screen the pointer is on rather than where typing goes. As a key, its release was lost whenever typing moved while it was held (the dashboard closing, a click on another panel, a pause), and wlroots counts presses per button: from then on that button, the lasers' left click included, did nothing in the desktop, and KWin kept it held. So ft-screens tracks the relay's buttons itself, lets a release through whenever it took the press, and releases them when the pointer leaves the screens, its screen hides, or Frametop pauses.
|
||||||
|
|
||||||
Volume keys must never reach gamescope. With the openvr backend, gamescope sends volume up and down to Steam by moving keyboard focus to Steam for the key and then back to the previously focused surface. When nothing had focus, the one it moves back to is null, and wlroots aborts on a null focus surface (`wlr_seat_keyboard_notify_enter: Assertion 'surface' failed`), which ends the whole VR session. Keyboard focus is often empty while you work in VR, so one press of the headset's volume button could take everything down. gamescope reads the headset's buttons and every keyboard itself (`InputStealer`), as do SteamVR's processes, so the relay has to stop volume keys at the device. Grabbing `gpio-keys` would also take the headset's click button, so the relay remaps the volume entries in each device's keymap (`EVIOCSKEYCODE`) and handles the stand-in codes itself. That fix covers every device at once, including keyboards that aren't grabbed.
|
Volume keys must never reach gamescope. With the openvr backend, gamescope sends volume up and down to Steam by moving keyboard focus to Steam for the key and then back to the previously focused surface. When nothing had focus, the one it moves back to is null, and wlroots aborts on a null focus surface (`wlr_seat_keyboard_notify_enter: Assertion 'surface' failed`), which ends the whole VR session. Keyboard focus is often empty while you work in VR, so one press of the headset's volume button could take everything down. gamescope reads the headset's buttons and every keyboard itself (`InputStealer`), as do SteamVR's processes, so the relay has to stop volume keys at the device. Grabbing `gpio-keys` would also take the headset's click button, so the relay remaps the volume entries in each device's keymap (`EVIOCSKEYCODE`) and handles the stand-in codes itself. That fix covers every device at once, including keyboards that aren't grabbed.
|
||||||
|
|
||||||
Frametop's keyboard opens by itself for a text field on the desktop. The apps run inside the nested KWin, so only KWin knows when a text field has focus, and the way it tells anyone is its input method protocol (`zwp_input_method_v1`): KWin starts one input method program and activates it whenever the focused app turns on text input. `input/ft-textinput` is that program, speaking the Wayland wire protocol directly so it needs nothing but Python on the host. It only reports focus. The gamescope session puts `QT_IM_MODULE=xim` and `GTK_IM_MODULE=xim` in the systemd user environment; with those, Qt and GTK apps use X input methods and never turn on Wayland text input, so the session script drops them.
|
Frametop's keyboard opens by itself for a text field on the desktop. The apps run inside the nested KWin, so only KWin knows when a text field has focus, and the way it tells anyone is its input method protocol (`zwp_input_method_v1`): KWin starts one input method program and activates it whenever the focused app turns on text input. `input/ft-textinput` is that program, speaking the Wayland wire protocol directly so it needs nothing but Python on the host. It only reports focus. The gamescope session puts `QT_IM_MODULE=xim` and `GTK_IM_MODULE=xim` in the systemd user environment; with those, Qt and GTK apps use X input methods and never turn on Wayland text input, so the session script drops them.
|
||||||
@@ -169,10 +172,22 @@ The Frame controllers can be mapped like mouse buttons, but they aren't input de
|
|||||||
|
|
||||||
## The desktop session
|
## The desktop session
|
||||||
|
|
||||||
The session is modeled on SteamOS's `steamos-nested-desktop` and runs beside it. It has its own runtime directory, config (`~/.config/frametop`), and state, so it never disturbs the stock desktop's layout or panels. It runs on a private D-Bus from `dbus-run-session`, which has two consequences. KDE only launches apps in systemd scopes when systemd is on the session bus, so everything started in the desktop lands in its systemd unit, and stopping the unit would kill all of it; `session/keep-apps.sh` moves those programs out first. And tools that need the real user bus, like podman and `distrobox-host-exec`, have to be pointed at it explicitly.
|
The session is modeled on SteamOS's `steamos-nested-desktop` and runs beside it. It has its own runtime directory, config (`~/.config/frametop`), and state, so it never disturbs the stock desktop's layout or panels. It runs on a private D-Bus from `dbus-run-session`, which has two consequences. KDE only launches apps in systemd scopes when systemd is on the session bus, so everything started in the desktop lands in its systemd unit, and stopping the unit would kill all of it; `session/keep-apps.sh` moves those programs out first. And tools that need the real user bus, like podman and `distrobox-host-exec`, have to be pointed at it explicitly. Its own config folder also hides SteamVR's path registry (`~/.config/openvr/openvrpaths.vrpath`) from everything started in it: OpenVR programs there fail with `VRInitError_Init_PathRegistryNotFound`, and `vrpathreg adddriver` writes a new registry under `~/.config/frametop/openvr` that has no SteamVR in it and that SteamVR never reads. So Frametop's scripts run SteamVR's tools with `XDG_CONFIG_HOME=~/.config`.
|
||||||
|
|
||||||
The VR launcher starts the session from the Steam client, and the client's environment came along: `LD_LIBRARY_PATH` pointing at Steam's own runtime, whose `libavcodec` has no H.264 decoder, so VLC in the desktop couldn't play most videos, plus the client's overlay and launch settings. The session script drops the client's variables before it starts anything. SteamOS's global Mesa settings (`/usr/share/deckard/mesavars.sh`) stay, and the gamescope session's Vulkan layer (`ENABLE_GAMESCOPE_WSI`) is only kept for the gamescope backend.
|
The VR launcher starts the session from the Steam client, and the client's environment came along: `LD_LIBRARY_PATH` pointing at Steam's own runtime, whose `libavcodec` has no H.264 decoder, so VLC in the desktop couldn't play most videos, plus the client's overlay and launch settings. The session script drops the client's variables before it starts anything. SteamOS's global Mesa settings (`/usr/share/deckard/mesavars.sh`) stay, and the gamescope session's Vulkan layer (`ENABLE_GAMESCOPE_WSI`) is only kept for the gamescope backend.
|
||||||
|
|
||||||
|
### Nested accessibility
|
||||||
|
|
||||||
|
The session drops an inherited `AT_SPI_BUS_ADDRESS`, so apps cannot accidentally use the host desktop's registry. It autostarts `session/ft-atspi` in Plasma phase 2, after KWin has set the nested display environment. The helper gets the live accessibility address from `org.a11y.Bus` on the private session bus, preserves any existing registry owner, updates the accessibility bus's activation environment, and tries `StartServiceByName` first.
|
||||||
|
|
||||||
|
On SteamOS 0.3.0 with at-spi2-core 2.52.0, the native launcher can choose dbus-broker because its process belongs to a systemd user unit. Registry activation then fails: this desktop's private session bus does not have a systemd activation manager. In that case the helper starts only `at-spi2-registryd` on the already-existing accessibility bus. The registry refuses duplicate ownership. Unlike native activation's `--use-gnome-session`, the fallback does not try to register with GNOME's session manager; that flag did not explain the observed native activation failure.
|
||||||
|
|
||||||
|
The fallback registry does not exit merely when its bus disconnects in the isolated SteamOS test. Its small watcher checks both private buses every 5 seconds, and terminates and reaps only the child it started when either bus disappears or the watcher is stopped. Each check runs `gdbus` twice; once a second, that cost about 1% of a core. `keep-apps.sh` keeps the watcher in the desktop unit when `desktops.sh start` runs the desktop as `frametop-desktop`. Started from the VR launcher, the desktop runs in steam.service, which doesn't stop with it, so there the watcher is the only thing that stops the registry. There is no second accessibility bus, global systemd environment update, process-name kill, or host registry replacement. Missing accessibility files or bus errors are nonfatal; the desktop still starts. Toolkit-specific accessibility opt-ins and pointer snapping are separate work.
|
||||||
|
|
||||||
|
Run the isolated checks on the host with `/usr/bin/python3 session/test/test_accessibility.py`. They use private D-Bus buses, Xvfb and a GTK3 app, never the production display or input. Native activation uses a small `org.a11y.Bus` test provider pointing to a real private dbus-daemon with the installed registry service; the SteamOS fallback uses the installed bus launcher and broker. The tests check real app-tree discovery, existing owners, concurrent starts, session stop/restart, and teardown. They require test-only PyGObject (Gio and GTK3), Xvfb, and at-spi2-core; the runtime helper uses Python's standard library and the existing host `gdbus`. Actual Plasma autostart and VR desktop restart still require an approved hardware test.
|
||||||
|
|
||||||
|
### Other session behavior
|
||||||
|
|
||||||
Steam, not systemd, suspends the Frame: after `system_idle_suspend_ac_sec` (an hour by default) without input on AC power, it logs `Switching to power state: k_ESystemPowerState_Sleep` and suspends, even while charging. It's a Steam setting (Settings → Power → When Plugged In and Idle → Sleep after), which the Stay awake while plugged in switch in Frametop Display Settings sets to Never. SteamVR's standby, which turns the displays off when the headset comes off, is separate; see below.
|
Steam, not systemd, suspends the Frame: after `system_idle_suspend_ac_sec` (an hour by default) without input on AC power, it logs `Switching to power state: k_ESystemPowerState_Sleep` and suspends, even while charging. It's a Steam setting (Settings → Power → When Plugged In and Idle → Sleep after), which the Stay awake while plugged in switch in Frametop Display Settings sets to Never. SteamVR's standby, which turns the displays off when the headset comes off, is separate; see below.
|
||||||
|
|
||||||
Flatpak apps need `XDG_DATA_DIRS` to include Flatpak's exports, or Plasma opens Discover instead of launching them, so the session sources `/etc/profile.d/flatpak.sh`.
|
Flatpak apps need `XDG_DATA_DIRS` to include Flatpak's exports, or Plasma opens Discover instead of launching them, so the session sources `/etc/profile.d/flatpak.sh`.
|
||||||
@@ -185,6 +200,8 @@ KWin renders with OpenGL through zink on Turnip, Vulkan on the same GPU vrcompos
|
|||||||
|
|
||||||
The nested session also runs the system's XDG autostart entries, being a KDE session. Discover's update notifier started `plasma-discover --mode update` in it (520 to 620 MB resident and about 9% of a core, plus `flatpak-system-helper` and AppStream downloads), and IBus started a daemon, the kimpanel panel and its GTK extension that nothing can use: KWin hands text input to the one input method it starts (`ft-textinput`), and the session drops `QT_IM_MODULE`, `GTK_IM_MODULE` and `XMODIFIERS`. The session hides both for this desktop only, with `Hidden=true` copies in its own autostart folder. The geoclue demo agent stays: it's what answers apps' location requests to Geoclue outside GNOME, and it costs nothing while idle. Orca's entry only starts in GNOME-family desktops.
|
The nested session also runs the system's XDG autostart entries, being a KDE session. Discover's update notifier started `plasma-discover --mode update` in it (520 to 620 MB resident and about 9% of a core, plus `flatpak-system-helper` and AppStream downloads), and IBus started a daemon, the kimpanel panel and its GTK extension that nothing can use: KWin hands text input to the one input method it starts (`ft-textinput`), and the session drops `QT_IM_MODULE`, `GTK_IM_MODULE` and `XMODIFIERS`. The session hides both for this desktop only, with `Hidden=true` copies in its own autostart folder. The geoclue demo agent stays: it's what answers apps' location requests to Geoclue outside GNOME, and it costs nothing while idle. Orca's entry only starts in GNOME-family desktops.
|
||||||
|
|
||||||
|
Plasma 6.2.5 keeps each panel on a screen number (`lastScreen` in `plasma-org.kde.plasma.desktop-appletsrc`), and the numbers rank the enabled outputs by priority, so 0 is the primary screen. A panel whose number is past the screen count gets no view, and Plasma never moves it: the remap it runs at every start only moves a panel whose number has no desktop, and this desktop keeps a desktop for every output it has seen, spares included. So the taskbar was lost when the number of screens went down, and once it was found saved on a spare output, number 8 of a desktop with three screens ([#18](https://github.com/DeeJanuz/frametop/issues/18)). Before Plasma starts, the session runs `session/fix-panels.py`, which moves any panel numbered past the screen count, with its system tray's containment, to screen 0, keeping its widgets and settings. A panel stays put when screen 0 already has one on that edge, and comes back by itself if the screens do. Before each repair the file is backed up to `<file>.ft-bak.last`. `<file>.ft-bak` keeps it as it was before the first repair and is never overwritten. The repair writes over a moved panel's old screen number, so the backups are the only record of it, and `.ft-bak.last` also keeps everything changed since the first repair. Plasma's scripting can't do this while it runs (`panel.screen` is read-only in 6.2.5), so a lost taskbar comes back at the desktop's next start. `scripts/doctor.sh` and `scripts/report.sh` list the panels and their screens.
|
||||||
|
|
||||||
Remote desktop is a chain (krdp, then FreeRDP inside Xvnc) because nothing on SteamOS serves KWin over VNC directly. Kept connected all the time, it cost about a core with nobody watching: krdpserver 55 to 78% (it encodes H.264 in software with openh264: VA-API finds no driver for the Frame's GPU in the container), FreeRDP 16 to 27%, Xvnc 6 to 11%, and the bridge's layout check every 5 seconds another 4%. krdp creates its screencast session per RDP connection and drops it when the connection closes (`SessionController::onNewConnection` in krdp 6.7), so an idle krdpserver costs nothing and can stay up; only the RDP connection has to go. The bridge connects FreeRDP when a VNC client appears and disconnects 45 seconds after the last one leaves. Xvnc has no hook for its clients, so the bridge counts established connections to its port with `ss`, woken early by Xvnc's log output; looking with `ss` once a second cost about 0.9% of a core in bash, against about 0.1% this way. `Xvnc -inetd` from a systemd socket would start a server per connection and lose sharing between viewers. The layout check (`ft-layout remote-view`, which scans `/proc` for plasmashell and runs `kscreen-doctor -j`) now runs only while FreeRDP runs, and then only after `kwinoutputconfig.json` or `frametop-layout.json` changes, with one check a minute in case a change touched neither.
|
Remote desktop is a chain (krdp, then FreeRDP inside Xvnc) because nothing on SteamOS serves KWin over VNC directly. Kept connected all the time, it cost about a core with nobody watching: krdpserver 55 to 78% (it encodes H.264 in software with openh264: VA-API finds no driver for the Frame's GPU in the container), FreeRDP 16 to 27%, Xvnc 6 to 11%, and the bridge's layout check every 5 seconds another 4%. krdp creates its screencast session per RDP connection and drops it when the connection closes (`SessionController::onNewConnection` in krdp 6.7), so an idle krdpserver costs nothing and can stay up; only the RDP connection has to go. The bridge connects FreeRDP when a VNC client appears and disconnects 45 seconds after the last one leaves. Xvnc has no hook for its clients, so the bridge counts established connections to its port with `ss`, woken early by Xvnc's log output; looking with `ss` once a second cost about 0.9% of a core in bash, against about 0.1% this way. `Xvnc -inetd` from a systemd socket would start a server per connection and lose sharing between viewers. The layout check (`ft-layout remote-view`, which scans `/proc` for plasmashell and runs `kscreen-doctor -j`) now runs only while FreeRDP runs, and then only after `kwinoutputconfig.json` or `frametop-layout.json` changes, with one check a minute in case a change touched neither.
|
||||||
|
|
||||||
Program names stay within 15 characters, because Linux truncates process names there and the scripts find programs with `pgrep -x` and `pkill -x`. That's why the prefix is `ft-`.
|
Program names stay within 15 characters, because Linux truncates process names there and the scripts find programs with `pgrep -x` and `pkill -x`. That's why the prefix is `ft-`.
|
||||||
@@ -212,7 +229,7 @@ Hiding the screens during a game kept them out of view, but Frametop kept using
|
|||||||
|
|
||||||
On the Frame, SteamVR is part of the OS image (`/opt/steamvr`, the `deckard-steamvr-rel` package), next to KWin, gamescope, and the kernel, so every SteamOS update can bring a new SteamVR too. Frametop survives updates: it lives in the home folder and the `dev` container, the Bluetooth fixes are in `/etc`, which SteamOS keeps across updates, and nothing goes into `/usr`. What an update can break is what Frametop uses from the image. The public OpenVR API is versioned and stays put. The rest is less certain: `IVRIPCResourceManagerClient`, which is newer than the header SteamVR ships; the text `vrcmd --overlays` prints; the eye tracker's shared memory layout; XRService's camera buffers; KWin's nested backend; and behavior Frametop works around, such as the SteamVR Settings page that `ComputeOverlayIntersection` can't find or the scale KWin's nested backend doesn't undo.
|
On the Frame, SteamVR is part of the OS image (`/opt/steamvr`, the `deckard-steamvr-rel` package), next to KWin, gamescope, and the kernel, so every SteamOS update can bring a new SteamVR too. Frametop survives updates: it lives in the home folder and the `dev` container, the Bluetooth fixes are in `/etc`, which SteamOS keeps across updates, and nothing goes into `/usr`. What an update can break is what Frametop uses from the image. The public OpenVR API is versioned and stays put. The rest is less certain: `IVRIPCResourceManagerClient`, which is newer than the header SteamVR ships; the text `vrcmd --overlays` prints; the eye tracker's shared memory layout; XRService's camera buffers; KWin's nested backend; and behavior Frametop works around, such as the SteamVR Settings page that `ComputeOverlayIntersection` can't find or the scale KWin's nested backend doesn't undo.
|
||||||
|
|
||||||
`scripts/update-check.py`, which `scripts/doctor.sh` runs, checks what it can directly: that SteamVR still serves every OpenVR interface version the installed programs were built against (read from the binaries), that `vrcmd`'s format still parses, that the eye tracker's sample timestamp is still at the offset ft-gaze reads, and the host files, services, sockets, and driver registration. Behavior can't be checked without someone in the headset, so it records the versions of the packages that matter once things work (`--mark-good`), and after an update names what changed and what to try by hand.
|
`scripts/update-check.py`, which `scripts/doctor.sh` runs, checks what it can directly: that SteamVR still serves every OpenVR interface version the installed programs were built against (read from the binaries), that `vrcmd`'s format still parses, that the eye tracker's shared memory still has a layout ft-gaze knows (stable's, or the 0.4.x beta's, with every field from the timestamp on 5 bytes later), by the same test ft-gaze uses to pick one, and the host files, services, sockets, and driver registration. Behavior can't be checked without someone in the headset, so it records the versions of the packages that matter once things work (`--mark-good`), and after an update names what changed and what to try by hand.
|
||||||
|
|
||||||
## Approaches we dropped
|
## Approaches we dropped
|
||||||
|
|
||||||
@@ -224,6 +241,5 @@ On the Frame, SteamVR is part of the OS image (`/opt/steamvr`, the `deckard-stea
|
|||||||
|
|
||||||
- A controller button that shows the screens during a game. Games own the controllers, so this needs SteamVR input actions for ft-screens.
|
- A controller button that shows the screens during a game. Games own the controllers, so this needs SteamVR input actions for ft-screens.
|
||||||
- Drawing KWin's cursor on the screens.
|
- Drawing KWin's cursor on the screens.
|
||||||
- Plasma can lose its panels when the number of screens goes down, because they're saved against a screen that no longer exists. Removing `plasma-org.kde.plasma.desktop-appletsrc` and `plasmashellrc` from `~/.config/frametop` brings the default panels back.
|
|
||||||
- Frame pacing and GPU cost with several busy screens haven't been measured.
|
- Frame pacing and GPU cost with several busy screens haven't been measured.
|
||||||
- Real standby on a stand, with rendering and tracking paused, not just the backlight off. SteamVR has no call for it, and its activity level follows the proximity sensor.
|
- Real standby on a stand, with rendering and tracking paused, not just the backlight off. SteamVR has no call for it, and its activity level follows the proximity sensor.
|
||||||
@@ -85,7 +85,7 @@ ft-screens creates a `screen` for each toplevel in the order they appear, and in
|
|||||||
runs commands ◀─ long poll ─ spare outputs (kscreen-doctor) ◀─ @frametop_float ─ lasers, catcher
|
runs commands ◀─ long poll ─ spare outputs (kscreen-doctor) ◀─ @frametop_float ─ lasers, catcher
|
||||||
```
|
```
|
||||||
|
|
||||||
- **KWin script `frametop-float`** (`float/frametop-float.js`). ft-floatd loads it into the desktop's KWin over D-Bus (`org.kde.kwin.Scripting`). A script keeps working across KWin updates. A C++ effect would have to match the host's exact KWin build, and the build container is Fedora, not SteamOS. The script watches windows (`windowAdded`/`windowRemoved`, `frameGeometryChanged`, `outputChanged`, `interactiveMoveResizeStarted`/`Finished`, `fullScreenChanged`, `maximizedChanged`, `minimizedChanged`, `keepBelowChanged`, `windowActivated`) and the outputs (`screensChanged`). It runs commands: move a window to an output, set its geometry, put it on all virtual desktops, and restore it. It adds "Float in VR" ("Back to Desktop" on a floating window) to the window menu (`registerUserActionsMenu`). It registers no shortcut: the float key belongs to the input relay. KWin scripts can call D-Bus but can't serve it, so commands come back through a long poll. The script calls ft-floatd's `NextCommand`, which answers when a command is ready, and then the script calls it again. It also keeps KWin's placement memory from moving windows (see design.md).
|
- **KWin script `frametop-float`** (`float/frametop-float.js`). ft-floatd loads it into the desktop's KWin over D-Bus (`org.kde.kwin.Scripting`). A script keeps working across KWin updates. A C++ effect would have to match the host's exact KWin build, and the build container is Fedora, not SteamOS. The script watches windows (`windowAdded`/`windowRemoved`, `frameGeometryChanged`, `outputChanged`, `interactiveMoveResizeStarted`/`Finished`, `fullScreenChanged`, `maximizedChanged`, `minimizedChanged`, `keepBelowChanged`, `windowActivated`) and the outputs (`screensChanged`). It runs commands: move a window to an output, set its geometry, put it on all virtual desktops, and restore it. It adds "Float in VR" ("Back to Desktop" on a floating window) to the window menu (`registerUserActionsMenu`). It registers no shortcut: the float key belongs to the input relay. KWin scripts can call D-Bus but can't serve it, so commands come back through a long poll. The script calls ft-floatd's `NextCommand`, which answers when a command is ready, and then the script calls it again, or after 30 seconds without an answer. It also keeps KWin's placement memory from moving windows (see design.md).
|
||||||
- **ft-floatd** (`float/ft-floatd`, Python). The host has dbus-python and PyGObject. It runs inside the desktop's Plasma session, started from its autostart. It owns `org.frametop.Float` on the session's private bus, and it keeps the table of which window is on which output and panel. It enables and disables spare outputs and sets their scale and position with `kscreen-doctor`, and their size through ft-screens. It tells ft-screens where each floating window goes and tells the script which window goes where. It launches apps floating, opens profiles' apps, and remembers each app's placement and scale, keyed by desktop file name. Commands come in on `@frametop_float`, from `ft-float`, the input relay, and ft-screens.
|
- **ft-floatd** (`float/ft-floatd`, Python). The host has dbus-python and PyGObject. It runs inside the desktop's Plasma session, started from its autostart. It owns `org.frametop.Float` on the session's private bus, and it keeps the table of which window is on which output and panel. It enables and disables spare outputs and sets their scale and position with `kscreen-doctor`, and their size through ft-screens. It tells ft-screens where each floating window goes and tells the script which window goes where. It launches apps floating, opens profiles' apps, and remembers each app's placement and scale, keyed by desktop file name. Commands come in on `@frametop_float`, from `ft-float`, the input relay, and ft-screens.
|
||||||
- **ft-screens.** A spare output's panel is a floating window's. Floating panels get the same bar, curve, roll, resize tab, and wrist and head pins as screens, plus dock and close buttons left of the bar. Other parts: the catcher, popup and dialog overlays, and carrying a panel during a KWin move. `MAX_SCREENS` (screens and spares together) is 24. Commands arrive on `@ft_screens`. Events go out to `@frametop_float` from an unbound socket, the same way ft-screens talks to the input relay.
|
- **ft-screens.** A spare output's panel is a floating window's. Floating panels get the same bar, curve, roll, resize tab, and wrist and head pins as screens, plus dock and close buttons left of the bar. Other parts: the catcher, popup and dialog overlays, and carrying a panel during a KWin move. `MAX_SCREENS` (screens and spares together) is 24. Commands arrive on `@ft_screens`. Events go out to `@frametop_float` from an unbound socket, the same way ft-screens talks to the input relay.
|
||||||
- **Session script.** Adds `FLOAT_SLOTS` to KWin's output count, starts ft-floatd from the desktop's autostart, installs Frametop's window decoration and chooses it in the session's `kwinrc`, and writes the Launch as Standalone copies of the apps' desktop files.
|
- **Session script.** Adds `FLOAT_SLOTS` to KWin's output count, starts ft-floatd from the desktop's autostart, installs Frametop's window decoration and chooses it in the session's `kwinrc`, and writes the Launch as Standalone copies of the apps' desktop files.
|
||||||
|
|||||||
+3
-1
@@ -17,9 +17,11 @@ The input relay takes the volume keys from every device that has them, so gamesc
|
|||||||
ft-screens drops keys while no screen has focus or the SteamVR dashboard is open, but always lets through the release of a key the desktop saw pressed, so a modifier held as the dashboard opens doesn't stay down.
|
ft-screens drops keys while no screen has focus or the SteamVR dashboard is open, but always lets through the release of a key the desktop saw pressed, so a modifier held as the dashboard opens doesn't stay down.
|
||||||
|
|
||||||
- **A key whose release never arrives stays held in the desktop until the relay clears it, within about a second.** KWin repeats held keys itself, so a stuck letter repeats and a stuck modifier changes every later key (Ctrl+Alt held turns T into Konsole). The relay remembers which keys it told the desktop went down, and once a second it releases any that no keyboard holds (`reconcile_desktop_keys`, which asks the kernel with `EVIOCGKEY`). Pressing and releasing the key again also clears it.
|
- **A key whose release never arrives stays held in the desktop until the relay clears it, within about a second.** KWin repeats held keys itself, so a stuck letter repeats and a stuck modifier changes every later key (Ctrl+Alt held turns T into Konsole). The relay remembers which keys it told the desktop went down, and once a second it releases any that no keyboard holds (`reconcile_desktop_keys`, which asks the kernel with `EVIOCGKEY`). Pressing and releasing the key again also clears it.
|
||||||
- **A keyboard that disconnects mid-press, or a relay restart with a key down, is how it happens.** The once-a-second check catches the first. A relay that starts doesn't know what an earlier one left down, so it releases the modifiers on the desktop; another key left down that way stays until it's pressed and released again.
|
- **A keyboard that disconnects mid-press, or a relay restart with a key down, is how it happens.** The once-a-second check catches the first. A relay that starts doesn't know what an earlier one left down, so it releases the modifiers and mouse buttons on the desktop; another key left down that way stays until it's pressed and released again.
|
||||||
- **To see where a key went,** run `scripts/keys-report.py` and reproduce the problem while it records. It logs the modifiers, Tab, and Esc (no other keys) as the relay reads them and as its virtual keyboard sends them on, with the device roles and grabs, which programs have each keyboard open, and the relay's and desktop's logs.
|
- **To see where a key went,** run `scripts/keys-report.py` and reproduce the problem while it records. It logs the modifiers, Tab, and Esc (no other keys) as the relay reads them and as its virtual keyboard sends them on, with the device roles and grabs, which programs have each keyboard open, and the relay's and desktop's logs.
|
||||||
- **Switching where typing goes waits for keys to come up.** The relay changes a keyboard's grab only while none of its keys are down, so a press and its release go to the same side. A key held for a long time delays the switch until it's let go.
|
- **Switching where typing goes waits for keys to come up.** The relay changes a keyboard's grab only while none of its keys are down, so a press and its release go to the same side. A key held for a long time delays the switch until it's let go.
|
||||||
|
- **Mouse buttons passed through as keys follow the pointer, not typing.** In pointer mode, a mouse button set to Pass through as key (a side button for Back) goes to the screen under the pointer, only while there is one and Frametop isn't paused. Its release always goes through, and ft-screens releases it itself when the pointer leaves the screens, its screen hides or closes, or Frametop pauses; the real release that comes later is dropped. So a drag held with it ends as the laser leaves a screen, unlike a laser's own click, which KWin keeps until it comes up. wlroots counts presses per button, so a release lost on the way would leave that button dead in the desktop, the lasers' clicks included, until ft-screens restarts. The button also goes to the virtual mouse, which gamescope reads, so the app Steam has focused may see it too.
|
||||||
|
- **Media keys reach both sides.** A keyboard's media keys come from its Consumer Control node, which has volume keys too, so the relay remaps that node rather than grabbing it. Its other keys reach gamescope as well as the desktop, even while typing goes to the desktop and the keyboard itself is grabbed: Play/Pause on the desktop can pause something in Steam too. The headset's own buttons don't go to the desktop.
|
||||||
|
|
||||||
## Typing and grabbed keyboards
|
## Typing and grabbed keyboards
|
||||||
|
|
||||||
|
|||||||
+17
-12
@@ -6,18 +6,22 @@ How each part of Frametop works, where its settings live, and the commands for r
|
|||||||
|
|
||||||
From the headset, open Launch a program → Desktop. The installer replaces that launcher entry with Frametop's (`~/.local/share/applications/deckard-nested-desktop.desktop`), and `desktops.sh uninstall` gives the stock single-screen desktop back.
|
From the headset, open Launch a program → Desktop. The installer replaces that launcher entry with Frametop's (`~/.local/share/applications/deckard-nested-desktop.desktop`), and `desktops.sh uninstall` gives the stock single-screen desktop back.
|
||||||
|
|
||||||
|
The installer also adds Native Desktop to the same list: a copy of SteamOS's entry for its own desktop (`~/.local/share/applications/native-deckard-nested-desktop.desktop`), skipped when SteamOS has no such entry. The copy is made at install time, so `scripts/update-check.py` warns when SteamOS's entry changes, and `desktops.sh install` refreshes it. In Native Desktop, typing goes to Steam's side, so the input relay doesn't grab keyboards there, and a Meta tap runs Frametop's Meta action (by default, the Steam menu) as well as opening Plasma's launcher.
|
||||||
|
|
||||||
From a terminal, on the Frame or from a PC over SSH:
|
From a terminal, on the Frame or from a PC over SSH:
|
||||||
|
|
||||||
```
|
```
|
||||||
desktops.sh install # the launcher's Desktop entry starts Frametop
|
desktops.sh install # the launcher's Desktop entry starts Frametop; Native Desktop is the stock one
|
||||||
desktops.sh uninstall # back to the stock SteamOS desktop
|
desktops.sh uninstall # back to the stock SteamOS desktop, as Desktop
|
||||||
desktops.sh start | stop | restart | status | log [lines]
|
desktops.sh start | stop | restart | status | log [lines]
|
||||||
```
|
```
|
||||||
|
|
||||||
`session/frametop-session.sh` runs the desktop. It starts ft-screens in the `dev` container (log: `/tmp/frametop-screens.log`), then KWin and Plasma on the host inside it. Only one desktop runs at a time. `desktops.sh start` runs it in its own systemd unit, `frametop-desktop`. It keeps its Plasma config in `~/.config/frametop`, separate from the stock desktop's.
|
`session/frametop-session.sh` runs the desktop. It starts ft-screens in the `dev` container (log: `/tmp/frametop-screens.log`), then KWin and Plasma on the host inside it. Only one Frametop desktop runs at a time. Its check doesn't look for Native Desktop, and running both at once is untested. `desktops.sh start` runs it in its own systemd unit, `frametop-desktop`. It keeps its Plasma config in `~/.config/frametop`, separate from the stock desktop's.
|
||||||
|
|
||||||
When the VR launcher starts the desktop, it inherits the Steam client's environment. The session script drops the client's runtime from it (`LD_LIBRARY_PATH`, the `STEAM_*` settings, and the Steam overlay's Vulkan layer), so apps in the desktop use the system's libraries, including its video codecs, just as they would after a normal login.
|
When the VR launcher starts the desktop, it inherits the Steam client's environment. The session script drops the client's runtime from it (`LD_LIBRARY_PATH`, the `STEAM_*` settings, and the Steam overlay's Vulkan layer), so apps in the desktop use the system's libraries, including its video codecs, just as they would after a normal login.
|
||||||
|
|
||||||
|
The nested session also starts an AT-SPI accessibility registry through `session/ft-atspi` in Plasma's autostart. It discovers the bus from this session, ignores an inherited host accessibility address, and leaves an existing registry alone. Accessibility errors do not stop the desktop. This supplies the registry infrastructure for apps that expose AT-SPI trees; it does not enable gaze snapping or force Chromium/Electron accessibility. After an approved desktop restart, an AT-SPI-aware app should be visible on the nested bus. See [design.md](design.md#nested-accessibility) for native activation, fallback lifecycle, and the isolated test command.
|
||||||
|
|
||||||
KWin's blur and background contrast effects and its animations are off in this desktop, because KWin draws on the headset's GPU, which SteamVR needs. The session script turns them off once, the first time it starts (it leaves a setting you already have alone, and marks it done in `~/.config/frametop/frametoprc`), so turning them back on sticks. In the Frametop desktop, System Settings → Window Management → Desktop Effects has Blur and Background Contrast, and General Behavior has Animation speed. Or from a terminal, then restart the desktop:
|
KWin's blur and background contrast effects and its animations are off in this desktop, because KWin draws on the headset's GPU, which SteamVR needs. The session script turns them off once, the first time it starts (it leaves a setting you already have alone, and marks it done in `~/.config/frametop/frametoprc`), so turning them back on sticks. In the Frametop desktop, System Settings → Window Management → Desktop Effects has Blur and Background Contrast, and General Behavior has Animation speed. Or from a terminal, then restart the desktop:
|
||||||
|
|
||||||
```
|
```
|
||||||
@@ -44,7 +48,7 @@ Every screen is an overlay named `frametop.screen.N` with five controls:
|
|||||||
- `.curve` bends the screen into a cylinder around you, using your current distance as the radius, or makes it flat again.
|
- `.curve` bends the screen into a cylinder around you, using your current distance as the radius, or makes it flat again.
|
||||||
- `.roll` rolls the screen when you drag it sideways, like a knob. It snaps level within 2.5°, and scrolling on it turns 5° per notch.
|
- `.roll` rolls the screen when you drag it sideways, like a knob. It snaps level within 2.5°, and scrolling on it turns 5° per notch.
|
||||||
- `.resize`, the tab on the bottom right corner, sets the width. Screens go down to 15 cm wide.
|
- `.resize`, the tab on the bottom right corner, sets the width. Screens go down to 15 cm wide.
|
||||||
- `.reset`, left of the bar, puts every screen back in its layout around where you are now, like Meta+Shift+R (`ft-layout apply`). In a VR game, where the screens leave the controllers to the game, aiming a controller at it turns SteamVR's laser on for that button alone, so the trigger clicks it; the game gets the controllers back half a second after you aim away.
|
- `.reset`, left of the bar, puts every screen back in its layout around where you are now, like Meta+Shift+R (`ft-layout apply`).
|
||||||
|
|
||||||
The controls are sized from both the screen's width and its distance from you, follow the surface of a curved screen, and stay invisible until a laser or the 3D mouse's cursor lands on one or comes within about 1.5 times a button's size of it. While invisible they're still there, fully transparent, so SteamVR's laser can find them. They're translucent until a laser is on them, like SteamVR's own window controls.
|
The controls are sized from both the screen's width and its distance from you, follow the surface of a curved screen, and stay invisible until a laser or the 3D mouse's cursor lands on one or comes within about 1.5 times a button's size of it. While invisible they're still there, fully transparent, so SteamVR's laser can find them. They're translucent until a laser is on them, like SteamVR's own window controls.
|
||||||
|
|
||||||
@@ -62,9 +66,9 @@ The Visibility & pins tab of Frametop Display Settings decides when the screens
|
|||||||
In the last three modes the hotkey shows the screens anyway. A screen can also be hidden on its own (Screens shown on the same tab, or `ft-layout hide N`): it stays hidden whatever the mode or the hotkey says, until it's shown again there. Windows on it stay put, and a new window that would open on it floats instead (ft-floatd). Profiles use this to show only some screens. Two more settings on the same tab cover VR games, which ft-screens detects as SteamVR scene apps:
|
In the last three modes the hotkey shows the screens anyway. A screen can also be hidden on its own (Screens shown on the same tab, or `ft-layout hide N`): it stays hidden whatever the mode or the hotkey says, until it's shown again there. Windows on it stay put, and a new window that would open on it floats instead (ft-floatd). Profiles use this to show only some screens. Two more settings on the same tab cover VR games, which ft-screens detects as SteamVR scene apps:
|
||||||
|
|
||||||
- During VR games, the Always mode hides the screens unless the dashboard is open (the default), or leaves them up.
|
- During VR games, the Always mode hides the screens unless the dashboard is open (the default), or leaves them up.
|
||||||
- Controllers on the screens. Visible screens can keep SteamVR's laser mouse on, so controllers work them with the dashboard closed, but that also takes the controllers away from a game. By default this is off while a VR game runs, and the 3D mouse or the dashboard works the screens. The other choices are always on, or only with the dashboard open, which also suits flatscreen games since they aren't scene apps.
|
- Controllers on the screens. Visible screens can keep SteamVR's laser mouse on, so controllers work them with the dashboard closed, but that also takes the controllers away from a game. By default this is off while a VR game runs, and the 3D mouse or the dashboard works the screens. Pointing a controller at a screen, a floating window, or the keyboard still turns its laser on, like SteamVR's own floating windows, and pointing away gives the game the controllers back. The other choices are always on, or only with the dashboard open, which also suits flatscreen games since they aren't scene apps.
|
||||||
|
|
||||||
Input from the lasers reaches KWin through ft-screens' own seat. Keys come from the input relay, from pass-through keyboards and any key a pointer device passes through. Typing follows your last click: after a click on a screen it goes to the desktop, even with the SteamVR dashboard open, and after a mouse click on any other panel (the dashboard, Steam, an app like Spotify) it goes there instead. While it goes to the desktop, the relay grabs pass-through keyboards so gamescope, which reads every keyboard itself, doesn't type them into the Steam app too. A program that watches every keyboard for a hotkey loses a grabbed one; with `SHARE_KEYS=1` in `~/.config/frametop.conf`, their keys also go to `@frametop_keys` for it. That's off by default, since any local process that binds the name first would get everything typed into the desktop. Hidden screens don't take typing.
|
Input from the lasers reaches KWin through ft-screens' own seat. Keys come from the input relay: from pass-through keyboards, a keyboard's media keys (its Consumer Control node), and any key a pointer device passes through. A mouse button passed through as a key in pointer mode (a side button for Back) goes to the screen the pointer is on instead, wherever typing goes, and only while the pointer is on a screen that shows and Frametop isn't paused. Its release always goes through, and ft-screens releases it itself when the pointer leaves the screens, its screen hides, or Frametop pauses (`scripts/test-relay-buttons.sh` checks this offline). Typing follows your last click: after a click on a screen it goes to the desktop, even with the SteamVR dashboard open, and after a mouse click on any other panel (the dashboard, Steam, an app like Spotify) it goes there instead. While it goes to the desktop, the relay grabs pass-through keyboards so gamescope, which reads every keyboard itself, doesn't type them into the Steam app too. A program that watches every keyboard for a hotkey loses a grabbed one; with `SHARE_KEYS=1` in `~/.config/frametop.conf`, their keys also go to `@frametop_keys` for it. That's off by default, since any local process that binds the name first would get everything typed into the desktop. Hidden screens don't take typing.
|
||||||
|
|
||||||
Frametop's keyboard opens by itself when a text field on the desktop gets focus, and stays open until its Close key, a layout reset, or a mapped button closes it (or, with Keep it open off in Frametop Input Settings, until the text field loses focus). While the Steam menu (the dashboard) or Steam's own keyboard is up, it steps aside, and it comes back where it was when they're gone; one asked for meanwhile appears then. In the "only with the dashboard" visibility mode, the dashboard doesn't count. It doesn't open without a head pose (the headset in standby). It's a panel of keys (a US laptop layout, with Esc where Caps Lock would be, arrows, and a Close key) that ft-screens shows 0.7 m in front of you and below your eyes, facing you. It stays where it opened, and its grab bar (the pill along the top) moves it like a screen's. Type on it with a controller's laser or the 3D mouse. Shift, Ctrl and Alt latch for the next key, and a held key repeats. KWin starts `input/ft-textinput` as the desktop's input method, and KWin activates it whenever the focused app turns on text input for a field. It tells the relay (`textfield 1` or `0`), the relay decides by the Keyboard setting in Frametop Input Settings, and ft-screens opens the keyboard for the screen that has keyboard focus (`vrkeyboard show`, `hide`, or `toggle` from a mapped button). Its keys reach the focused screen as key presses, so it works in every app, but only apps that use Wayland text input (Qt, GTK, Firefox) open it by themselves; Chromium, Electron and X11 apps need the button. The session drops the `QT_IM_MODULE=xim` and `GTK_IM_MODULE=xim` that the gamescope session sets, or Qt and GTK apps wouldn't use Wayland text input either.
|
Frametop's keyboard opens by itself when a text field on the desktop gets focus, and stays open until its Close key, a layout reset, or a mapped button closes it (or, with Keep it open off in Frametop Input Settings, until the text field loses focus). While the Steam menu (the dashboard) or Steam's own keyboard is up, it steps aside, and it comes back where it was when they're gone; one asked for meanwhile appears then. In the "only with the dashboard" visibility mode, the dashboard doesn't count. It doesn't open without a head pose (the headset in standby). It's a panel of keys (a US laptop layout, with Esc where Caps Lock would be, arrows, and a Close key) that ft-screens shows 0.7 m in front of you and below your eyes, facing you. It stays where it opened, and its grab bar (the pill along the top) moves it like a screen's. Type on it with a controller's laser or the 3D mouse. Shift, Ctrl and Alt latch for the next key, and a held key repeats. KWin starts `input/ft-textinput` as the desktop's input method, and KWin activates it whenever the focused app turns on text input for a field. It tells the relay (`textfield 1` or `0`), the relay decides by the Keyboard setting in Frametop Input Settings, and ft-screens opens the keyboard for the screen that has keyboard focus (`vrkeyboard show`, `hide`, or `toggle` from a mapped button). Its keys reach the focused screen as key presses, so it works in every app, but only apps that use Wayland text input (Qt, GTK, Firefox) open it by themselves; Chromium, Electron and X11 apps need the button. The session drops the `QT_IM_MODULE=xim` and `GTK_IM_MODULE=xim` that the gamescope session sets, or Qt and GTK apps wouldn't use Wayland text input either.
|
||||||
|
|
||||||
@@ -99,7 +103,7 @@ The relay also owns the volume keys, on every device that has them, the headset'
|
|||||||
desktops.sh relay install # enable it (starts with the next reboot or SteamVR start)
|
desktops.sh relay install # enable it (starts with the next reboot or SteamVR start)
|
||||||
desktops.sh relay status | log | uninstall
|
desktops.sh relay status | log | uninstall
|
||||||
input/input-relay.py --no-grab # try it without taking devices from SteamVR
|
input/input-relay.py --no-grab # try it without taking devices from SteamVR
|
||||||
input/test/keys-test.py # key combinations and modifier taps, against fake devices (safe next to the live relay)
|
input/test/keys-test.py # key combinations, modifier taps, and what reaches the desktop, against fake devices (safe next to the live relay)
|
||||||
steam/ft-steam menu # what Open Steam menu does; ft-steam check: Steam's UI still has the calls
|
steam/ft-steam menu # what Open Steam menu does; ft-steam check: Steam's UI still has the calls
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -134,7 +138,7 @@ A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs
|
|||||||
- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, gaze pointer on or off, gaze precision, gaze drag, gaze quick check, faster or slower, reset the screen layout, hide or show the screens, open or close the keyboard, float a window in VR or put it back, put all floating windows back, pause or resume Frametop ([Pausing for VR games](#pausing-for-vr-games)), Open profile NAME (one per profile, [profiles.md](profiles.md)), pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
|
- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, gaze pointer on or off, gaze precision, gaze drag, gaze quick check, faster or slower, reset the screen layout, hide or show the screens, open or close the keyboard, float a window in VR or put it back, put all floating windows back, pause or resume Frametop ([Pausing for VR games](#pausing-for-vr-games)), Open profile NAME (one per profile, [profiles.md](profiles.md)), pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
|
||||||
- Controllers maps the Frame controllers' buttons (every button but the system button) to the same actions, except passing a key through and the gaze actions: gaze mode is a mouse and keyboard feature ([gaze-controllers.md](gaze-controllers.md)). Capture a button and press it on a controller, or pick it from the list. The controllers aren't input devices on the host; only SteamVR sees them. So the pointer helper reads them with SteamVR input (`pointer/helper/vrbuttons.h`, `pointer/helper/actions/`) and sends presses to the relay (`vrbtn right/a 1`), which does the mapped action. The helper only takes the buttons that are mapped (the relay tells it with `vrbind`), at an overlay-global priority, and only while no game (scene application) runs, so games keep every button; with In games on (`controller_in_games`), a mapped button is taken from games too. That needs SteamVR's "Enable global input from overlays (Experimental)" setting (`steamvr/globalActionSetPriority`), which the page's Global input switch turns on and off. Mappings are saved as `controller_buttons` in `~/.config/frametop-input.json`.
|
- Controllers maps the Frame controllers' buttons (every button but the system button) to the same actions, except passing a key through and the gaze actions: gaze mode is a mouse and keyboard feature ([gaze-controllers.md](gaze-controllers.md)). Capture a button and press it on a controller, or pick it from the list. The controllers aren't input devices on the host; only SteamVR sees them. So the pointer helper reads them with SteamVR input (`pointer/helper/vrbuttons.h`, `pointer/helper/actions/`) and sends presses to the relay (`vrbtn right/a 1`), which does the mapped action. The helper only takes the buttons that are mapped (the relay tells it with `vrbind`), at an overlay-global priority, and only while no game (scene application) runs, so games keep every button; with In games on (`controller_in_games`), a mapped button is taken from games too. That needs SteamVR's "Enable global input from overlays (Experimental)" setting (`steamvr/globalActionSetPriority`), which the page's Global input switch turns on and off. Mappings are saved as `controller_buttons` in `~/.config/frametop-input.json`.
|
||||||
- Game optimization has the pause for VR games: its state with Pause now or Resume, whether VR games pause Frametop by themselves, the controller gesture (one or two buttons, pressed once or twice; one button always takes two presses), what happens to the desktop, and the sound. They're saved as `pause_auto`, `pause_gesture`, `pause_desktop`, and `pause_sound` in `~/.config/frametop-input.json`. See [Pausing for VR games](#pausing-for-vr-games).
|
- Game optimization has the pause for VR games: its state with Pause now or Resume, whether VR games pause Frametop by themselves, the controller gesture (one or two buttons, pressed once or twice; one button always takes two presses), what happens to the desktop, and the sound. They're saved as `pause_auto`, `pause_gesture`, `pause_desktop`, and `pause_sound` in `~/.config/frametop-input.json`. See [Pausing for VR games](#pausing-for-vr-games).
|
||||||
- Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput, like frame-voice's, don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected. Its Key combinations section maps modifiers plus a key, or one modifier tapped on its own, on any keyboard, to any action but passing a key through or nothing, or to Run a command…: a command line the input relay runs with `sh -c` when you press the keys (`command:CMD`). The command runs as the relay's user service, outside the desktop's session, with `layout/`, `float/` and `steam/` on its `PATH` (so `ft-layout use Work` or `ft-float launch org.kde.dolphin` work as they are), and its output goes to the relay's journal. The gaze clicks (Gaze left click and Gaze right click) only go on key combinations. The defaults are a Meta tap (open the Steam menu, or close the dashboard), Meta+J (gaze left click), Meta+K (gaze right click), and Meta+Shift+F (float window in VR or put it back); remove them or add others there. A tap is a press and release with no other key, mouse button, or scroll in between; a bound one sends the desktop F24 before the release, so Plasma's launcher doesn't open on it. The combination's last key isn't typed, and the modifiers still reach the app; while typing goes to Steam rather than the desktop, keyboards aren't grabbed, so Steam or the game sees the keys too. They're saved as `key_bindings` in the same file; a file with its own list, even an empty one, gets no defaults.
|
- Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput, like frame-voice's, don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected. Its Key combinations section maps modifiers plus a key, or one modifier tapped on its own, on any keyboard, to any action but passing a key through or nothing, or to Run a command…: a command line the input relay runs with `sh -c` when you press the keys (`command:CMD`). The command runs as the relay's user service, outside the desktop's session, with `layout/`, `float/` and `steam/` on its `PATH` (so `ft-layout use Work` or `ft-float launch org.kde.dolphin` work as they are), and its output goes to the relay's journal. The gaze clicks (Gaze left click and Gaze right click) only go on key combinations. The defaults are a Meta tap (open the Steam menu, or close the dashboard), Meta+J (gaze left click), Meta+K (gaze right click), Meta+Shift+F (float window in VR or put it back), and Meta+Alt+Tab and Meta+Alt+Shift+Tab (spin the panels: every screen and floating window turns about your head, so the next one on the right or left comes to the front; ft-screens' `spin next|prev|<degrees>`); remove them or add others there. A tap is a press and release with no other key, mouse button, or scroll in between; a bound one sends the desktop F24 before the release, so Plasma's launcher doesn't open on it. The combination's last key isn't typed, and the modifiers still reach the app; while typing goes to Steam rather than the desktop, keyboards aren't grabbed, so Steam or the game sees the keys too. They're saved as `key_bindings` in the same file; a file with its own list, even an empty one, gets no defaults.
|
||||||
- Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button.
|
- Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button.
|
||||||
- Ignored panels lists the SteamVR overlays that are showing, grouped by app (the first two parts of the overlay key, such as `sasaken.frame-perf-overlay`), from the pointer helper (`overlays`). Tick a panel, or Ignore the whole app, and the pointer passes through it to what's behind. It's for panels you only look at, like a performance overlay that follows your view. The list is saved as `POINTER_IGNORE` in `~/.config/frametop.conf`: comma-separated overlay keys, where a shell pattern like `vendor.app*` covers a whole app, including panels it opens later. The helper reloads at once. Frametop's own screens aren't listed, and entries for apps that aren't open are listed below, to remove.
|
- Ignored panels lists the SteamVR overlays that are showing, grouped by app (the first two parts of the overlay key, such as `sasaken.frame-perf-overlay`), from the pointer helper (`overlays`). Tick a panel, or Ignore the whole app, and the pointer passes through it to what's behind. It's for panels you only look at, like a performance overlay that follows your view. The list is saved as `POINTER_IGNORE` in `~/.config/frametop.conf`: comma-separated overlay keys, where a shell pattern like `vendor.app*` covers a whole app, including panels it opens later. The helper reloads at once. Frametop's own screens aren't listed, and entries for apps that aren't open are listed below, to remove.
|
||||||
- Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), what the mouse's left button and movement do, the gaze dot, the eye tracker and eye bias, the gaze mode sliders, the gaze service's state (headset, samples per second, how often the tracker is losing each eye, the calibration, the nudges learned), and Quick check, Calibrate, and Check headset fit (each in a panel in the headset), Reload calibration, and Forget nudges. The gaze probe, a development tool, is in the page's overflow menu.
|
- Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), what the mouse's left button and movement do, the gaze dot, the eye tracker and eye bias, the gaze mode sliders, the gaze service's state (headset, samples per second, how often the tracker is losing each eye, the calibration, the nudges learned), and Quick check, Calibrate, and Check headset fit (each in a panel in the headset), Reload calibration, and Forget nudges. The gaze probe, a development tool, is in the page's overflow menu.
|
||||||
@@ -173,7 +177,7 @@ layout/ft-layout hide N|all # hide a screen on its own, whatever the visibility
|
|||||||
display-settings/install.sh # menu entries and the Meta+Shift+R and Meta+Shift+H shortcuts
|
display-settings/install.sh # menu entries and the Meta+Shift+R and Meta+Shift+H shortcuts
|
||||||
```
|
```
|
||||||
|
|
||||||
The layout is stored relative to your head when it's applied. `/tmp/frametop-layout.log` has the run from the last desktop start.
|
The layout is stored relative to your head when it's applied. `/run/user/<uid>/frametop-layout.log`, in the host's runtime directory (not the nested desktop's `/run/user/<uid>/frametop`), has the run from the last desktop start and ft-screens' layout runs after it.
|
||||||
|
|
||||||
## Floating windows
|
## Floating windows
|
||||||
|
|
||||||
@@ -221,7 +225,7 @@ Paused, Frametop leaves the headset's CPU and GPU to a VR game. The input relay
|
|||||||
- The gaze service stops (`frametop-gaze`: ft-gazed, ft-gaze, our own eye tracker, the gaze panel), so nothing reads SteamVR's eye tracking. Our frame grabber, the root service `ft-eyegrab`, goes idle by itself 3 seconds after our eye tracker stops asking it for frames.
|
- The gaze service stops (`frametop-gaze`: ft-gazed, ft-gaze, our own eye tracker, the gaze panel), so nothing reads SteamVR's eye tracking. Our frame grabber, the root service `ft-eyegrab`, goes idle by itself 3 seconds after our eye tracker stops asking it for frames.
|
||||||
- Hand tracking stops if it runs (`frametop-camd`, `frametop-hands`).
|
- Hand tracking stops if it runs (`frametop-camd`, `frametop-hands`).
|
||||||
- The desktop, as the Game optimization page of Frametop Input Settings says (`pause_desktop`): hidden (the default) or closed. Hidden, ft-screens hides every screen and floating window whatever the visibility mode, the hotkey, or the dashboard says, and gives KWin a frame callback once a second instead of every display frame. KWin draws a screen only after its frame callback, and its apps wait for theirs, so the desktop hardly draws, but its windows stay open. Remote desktop stops if it runs (`session/remote-ctl.sh`). Closed, `desktops.sh stop` closes the desktop and its windows, and resuming starts it again (about 12 seconds), in its start profile if it has one.
|
- The desktop, as the Game optimization page of Frametop Input Settings says (`pause_desktop`): hidden (the default) or closed. Hidden, ft-screens hides every screen and floating window whatever the visibility mode, the hotkey, or the dashboard says, and gives KWin a frame callback once a second instead of every display frame. KWin draws a screen only after its frame callback, and its apps wait for theirs, so the desktop hardly draws, but its windows stay open. Remote desktop stops if it runs (`session/remote-ctl.sh`). Closed, `desktops.sh stop` closes the desktop and its windows, and resuming starts it again (about 12 seconds), in its start profile if it has one.
|
||||||
- The relay lets go of the 3D mouse and feeds pointer devices to its virtual mouse and keyboard, as with `POINTER=0`. Typing goes to Steam. Mapped buttons and key combinations do nothing but pausing, the Steam menu, and commands; a key combination that does nothing is typed as usual.
|
- The relay lets go of the 3D mouse, releasing any click still held on it (a mouse button, a mapped controller button, or a key combination), and feeds pointer devices to its virtual mouse and keyboard, as with `POINTER=0`. Typing goes to Steam. Mapped buttons and key combinations do nothing but pausing, the Steam menu, and commands; a key combination that does nothing is typed as usual.
|
||||||
|
|
||||||
Resuming starts again only what pausing stopped, and plays a second sound. The pointer helper and ft-powerd keep running: they cost little, the helper is what says a game started, and stopping it would leave its virtual controller connected with its last pose.
|
Resuming starts again only what pausing stopped, and plays a second sound. The pointer helper and ft-powerd keep running: they cost little, the helper is what says a game started, and stopping it would leave its virtual controller connected with its last pose.
|
||||||
|
|
||||||
@@ -237,6 +241,7 @@ input/ft-pause on | off | toggle # pause or resume
|
|||||||
input/ft-pause status # the state as JSON (the relay's "pause ?")
|
input/ft-pause status # the state as JSON (the relay's "pause ?")
|
||||||
input/vrws.py 10 # the controllers' buttons from vrserver's web socket, for 10 s
|
input/vrws.py 10 # the controllers' buttons from vrserver's web socket, for 10 s
|
||||||
input/test/pause-test.py # the gesture and the automatic pause, offline
|
input/test/pause-test.py # the gesture and the automatic pause, offline
|
||||||
|
input/test/pause-buttons-test.py # a click held into a pause comes up, offline
|
||||||
```
|
```
|
||||||
|
|
||||||
The state outlives a relay restart, in `/run/user/UID/frametop-pause.json`. A SteamVR restart while paused starts the gaze service with it, and the relay stops it again when the pointer helper comes back.
|
The state outlives a relay restart, in `/run/user/UID/frametop-pause.json`. A SteamVR restart while paused starts the gaze service with it, and the relay stops it again when the pointer helper comes back.
|
||||||
@@ -258,10 +263,10 @@ Deferred: it costs a lot of the headset's CPU and needs more work, so `install.s
|
|||||||
|
|
||||||
Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen. The same tracker detects pinches and grips, and with `POINTER_HANDS=1` in `~/.config/frametop.conf` they work the pointer. In gaze mode a pinch clicks where you look when it opens; hold it and move the hand to correct the pointer first. Without gaze mode a pinch is a press like the mouse's button, so a held pinch drags. A grip (closing the hand) presses and drags. To install it: `hands/run.sh install`.
|
Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen. The same tracker detects pinches and grips, and with `POINTER_HANDS=1` in `~/.config/frametop.conf` they work the pointer. In gaze mode a pinch clicks where you look when it opens; hold it and move the hand to correct the pointer first. Without gaze mode a pinch is a press like the mouse's button, so a held pinch drags. A grip (closing the hand) presses and drags. To install it: `hands/run.sh install`.
|
||||||
|
|
||||||
- `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`.
|
- `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`. The Hand Recorder's installer (`hands/rec/install.sh`) sets them the same way. `hands/run.sh uncaps` takes them back while neither the services nor the Hand Recorder is installed. `hands/run.sh uninstall` and `hands/rec/install.sh uninstall` run it after removing their own part, and `uninstall.sh` always takes them back.
|
||||||
- `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches and grips, read by the pointer helper) next to the ring.
|
- `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches and grips, read by the pointer helper) next to the ring.
|
||||||
- The install leaves both off, and they don't start with SteamVR. `ft-handsctl on` starts them while SteamVR runs, and `ft-handsctl off` stops them; they also stop with SteamVR. The install links `ft-handsctl` into `~/.local/bin`. `ft-handsctl status` and `ft-handsctl log` (or `hands/run.sh status` and `log`) show how they're doing, `ft-handsctl cutouts on|off` turns just the cutouts off, and `ft-handsctl gestures` shows pinches and grips live.
|
- The install leaves both off, and they don't start with SteamVR. `ft-handsctl on` starts them while SteamVR runs, and `ft-handsctl off` stops them; they also stop with SteamVR. The install links `ft-handsctl` into `~/.local/bin`. `ft-handsctl status` and `ft-handsctl log` (or `hands/run.sh status` and `log`) show how they're doing, `ft-handsctl cutouts on|off` turns just the cutouts off, and `ft-handsctl gestures` shows pinches and grips live.
|
||||||
- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (after some SteamVR restarts the side cameras' names come out swapped, and hands land beside the holes; `hands/tools/check_sides.py --ring` tells), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, and `POINTER_PINCH_TYPING`. The example config explains each.
|
- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (`auto`, the default: ft-hands tells from the hands when some SteamVR restart has swapped the side cameras' names, and fixes them; `0` or `1` force them, and `hands/tools/check_sides.py --ring` tells which is right), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, and `POINTER_PINCH_TYPING`. The example config explains each.
|
||||||
|
|
||||||
Details, options, and the recording and replay tools are in [hands/README.md](../hands/README.md).
|
Details, options, and the recording and replay tools are in [hands/README.md](../hands/README.md).
|
||||||
|
|
||||||
|
|||||||
+48
-2
@@ -378,6 +378,17 @@ function run(c) {
|
|||||||
case "activate":
|
case "activate":
|
||||||
if (w) workspace.activeWindow = w;
|
if (w) workspace.activeWindow = w;
|
||||||
break;
|
break;
|
||||||
|
case "activate-output": { // the top window on that output (a spin brought it to the front)
|
||||||
|
const order = workspace.stackingOrder;
|
||||||
|
for (let i = order.length - 1; i >= 0; --i) {
|
||||||
|
const o = order[i];
|
||||||
|
if (o.deleted || o.minimized || o.hidden || !o.managed || !o.output) continue;
|
||||||
|
if (o.output.name !== c.output || !o.normalWindow || o.popupWindow) continue;
|
||||||
|
workspace.activeWindow = o;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
case "minimize":
|
case "minimize":
|
||||||
if (w) w.minimized = c.on;
|
if (w) w.minimized = c.on;
|
||||||
break;
|
break;
|
||||||
@@ -400,11 +411,46 @@ function run(c) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The long poll has one failure mode: a reply that never comes. KWin's callDBus never calls
|
||||||
|
// back when a call fails (an error reply, ft-floatd gone, or its 25 s D-Bus timeout, which
|
||||||
|
// ft-floatd blocked that long hits); it only logs "Received D-Bus message is error". Then
|
||||||
|
// polling stays true and the script stops hearing commands until KWin reloads it. A watchdog
|
||||||
|
// re-arms the poll. Its period must stay above that 25 s timeout, whatever ft-floatd's
|
||||||
|
// POLL_SECONDS is: by then the call it gives up on has ended. The serial is for a reply that
|
||||||
|
// still comes later (KWin stalled with the reply queued): it runs its commands (ft-floatd
|
||||||
|
// sends each one once) but doesn't poll again. Two polls would answer each other for good,
|
||||||
|
// since ft-floatd answers a waiting poll empty when the next one comes.
|
||||||
|
// When the watchdog fires again and again, ft-floatd is gone, and only starting it again
|
||||||
|
// helps (it reloads the script then). Each failed call is a line in KWin's log, so the
|
||||||
|
// script says so once and waits twice as long each time, up to 5 minutes. A reply starts
|
||||||
|
// over at 30 s, so a lost reply is still polled again after 30 s.
|
||||||
|
const POLL_WAIT = 30000, POLL_WAIT_MAX = 300000;
|
||||||
|
const pollWatchdog = new QTimer();
|
||||||
|
pollWatchdog.singleShot = true;
|
||||||
|
pollWatchdog.interval = POLL_WAIT;
|
||||||
|
let pollLost = false; // the watchdog fired, and no reply since
|
||||||
|
pollWatchdog.timeout.connect(() => {
|
||||||
|
if (!pollLost) print("frametop-float: NextCommand didn't answer, polling again");
|
||||||
|
pollLost = true;
|
||||||
|
pollWatchdog.interval = Math.min(pollWatchdog.interval * 2, POLL_WAIT_MAX);
|
||||||
|
polling = false;
|
||||||
|
poll();
|
||||||
|
});
|
||||||
|
let pollSerial = 0;
|
||||||
|
|
||||||
function poll() {
|
function poll() {
|
||||||
if (polling) return;
|
if (polling) return;
|
||||||
polling = true;
|
polling = true;
|
||||||
|
const serial = ++pollSerial;
|
||||||
|
pollWatchdog.start();
|
||||||
callDBus(SERVICE, PATH, IFACE, "NextCommand", reply => {
|
callDBus(SERVICE, PATH, IFACE, "NextCommand", reply => {
|
||||||
polling = false;
|
const current = serial === pollSerial; // not a call the watchdog gave up on
|
||||||
|
if (current) {
|
||||||
|
pollWatchdog.stop();
|
||||||
|
pollWatchdog.interval = POLL_WAIT;
|
||||||
|
pollLost = false;
|
||||||
|
polling = false;
|
||||||
|
}
|
||||||
if (reply) {
|
if (reply) {
|
||||||
try {
|
try {
|
||||||
JSON.parse(reply).forEach(run);
|
JSON.parse(reply).forEach(run);
|
||||||
@@ -412,7 +458,7 @@ function poll() {
|
|||||||
print("frametop-float: bad command " + reply + ": " + e);
|
print("frametop-float: bad command " + reply + ": " + e);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
poll();
|
if (current) poll();
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -18,6 +18,8 @@ command-line side). Replies go to the sender:
|
|||||||
("float pointer" is the float key: the window under the pointer, else the active one,
|
("float pointer" is the float key: the window under the pointer, else the active one,
|
||||||
floated or docked; the input relay sends it for float_toggle, and "dock all" for dock_all)
|
floated or docked; the input relay sends it for float_toggle, and "dock all" for dock_all)
|
||||||
dock N | close N | resize N W H | scale N STEPS (ft-screens, N = its screen)
|
dock N | close N | resize N W H | scale N STEPS (ft-screens, N = its screen)
|
||||||
|
front N (ft-screens: a spin brought panel N to the front: make its window, or the top one
|
||||||
|
on a screen, KWin's active window)
|
||||||
|
|
||||||
Spare outputs are WL-<screens> .. WL-<screens + slots - 1>. A floating window's output is its
|
Spare outputs are WL-<screens> .. WL-<screens + slots - 1>. A floating window's output is its
|
||||||
frame plus a margin on each side (FLOAT_MARGIN pixels), so menus have room; the panel shows
|
frame plus a margin on each side (FLOAT_MARGIN pixels), so menus have room; the panel shows
|
||||||
@@ -1063,6 +1065,13 @@ class Daemon:
|
|||||||
for f in list(self.floats.values()):
|
for f in list(self.floats.values()):
|
||||||
self.dock(f)
|
self.dock(f)
|
||||||
return "ok"
|
return "ok"
|
||||||
|
if cmd == "front" and len(rest) == 1 and rest[0].isdigit():
|
||||||
|
f = self.by_panel(int(rest[0]))
|
||||||
|
if f:
|
||||||
|
self.command(cmd="activate", id=f.id)
|
||||||
|
else: # a screen: its output is WL-<N - 1>
|
||||||
|
self.command(cmd="activate-output", output=f"WL-{int(rest[0]) - 1}")
|
||||||
|
return "ok"
|
||||||
if cmd in ("dock", "close", "resize", "scale") and rest and rest[0].isdigit():
|
if cmd in ("dock", "close", "resize", "scale") and rest and rest[0].isdigit():
|
||||||
f = self.by_panel(int(rest[0]))
|
f = self.by_panel(int(rest[0]))
|
||||||
if not f:
|
if not f:
|
||||||
|
|||||||
+1
-1
@@ -33,7 +33,7 @@ Gaze as an input method for the whole desktop, without replacing anything of Ste
|
|||||||
- **The mouse only corrects** (the default; the Gaze page's Mouse movement switch, `POINTER_GAZE_MOUSE_MOVE=held`): while the gaze has the pointer, moving the mouse does nothing. The buttons work like Meta+J and Meta+K: press and hold one and the pointer stops where you look; move the mouse onto what you meant and let go to click there (a left or a right click). Held still for half a second, a press is a real one (to drag). Once you've moved, the left button alone only clicks: press the right one while still holding the left to start a drag there; it lasts while either button is held. Press the right one again (a double right click, the left still held) to pan and tilt what you're dragging, as a right press does during any drag. A bumped or drifting mouse can't pull the pointer away, and every mouse move is a correction, so the tracker only learns from real ones. With the gaze stale for a second (the tracker stopped, eyes lost), in a game, or with the headset off, the mouse moves the pointer as usual. `free` (the switch off) lets the mouse take the pointer any time.
|
- **The mouse only corrects** (the default; the Gaze page's Mouse movement switch, `POINTER_GAZE_MOUSE_MOVE=held`): while the gaze has the pointer, moving the mouse does nothing. The buttons work like Meta+J and Meta+K: press and hold one and the pointer stops where you look; move the mouse onto what you meant and let go to click there (a left or a right click). Held still for half a second, a press is a real one (to drag). Once you've moved, the left button alone only clicks: press the right one while still holding the left to start a drag there; it lasts while either button is held. Press the right one again (a double right click, the left still held) to pan and tilt what you're dragging, as a right press does during any drag. A bumped or drifting mouse can't pull the pointer away, and every mouse move is a correction, so the tracker only learns from real ones. With the gaze stale for a second (the tracker stopped, eyes lost), in a game, or with the headset off, the mouse moves the pointer as usual. `free` (the switch off) lets the mouse take the pointer any time.
|
||||||
- **Keyboard clicks** (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, `POINTER_KEY_TAP`) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse, under the same limit: past `POINTER_GAZE_NUDGE_MAX` it opens the quick check instead). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; the drag lasts while either key is held. Meta+K during a Meta+J drag (again, after starting it with Meta+K: a double Meta+K) pans and tilts what you're dragging while it's held: turn your head to turn it.
|
- **Keyboard clicks** (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, `POINTER_KEY_TAP`) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse, under the same limit: past `POINTER_GAZE_NUDGE_MAX` it opens the quick check instead). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; the drag lasts while either key is held. Meta+K during a Meta+J drag (again, after starting it with Meta+K: a double Meta+K) pans and tilts what you're dragging while it's held: turn your head to turn it.
|
||||||
- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it (0.2 degrees or more, and the correction within `POINTER_GAZE_NUDGE_MAX`: 55 degrees by default, half of the 109 the headset shows across, and 1 to 110; the same limit for mouse, keyboard, and pinch clicks) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the quick check's dot does the same). A correction past `POINTER_GAZE_NUDGE_MAX` isn't learned: the helper asks ft-gazed for the quick check instead ("recheck", after its 2-minute cooldown). Tested on our tracker's 409 clicks since its Sep 29 calibration: a one-dot check set from any one of them put the next 2 minutes' clicks within 15 degrees (99% within 4.2) and the next 10 minutes' within 25 (the far ones after the headset moved), so the check gets back well under it. The limit used to be 8 degrees, and live on 2026-10-01 our tracker was 12 off after the headset went on, so every correction was dropped. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
|
- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it (0.2 degrees or more, and the correction within `POINTER_GAZE_NUDGE_MAX`: 55 degrees by default, half of the 109 the headset shows across, and 1 to 110; the same limit for mouse, keyboard, and pinch clicks) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the quick check's dot does the same). A correction past `POINTER_GAZE_NUDGE_MAX` isn't learned: the helper asks ft-gazed for the quick check instead ("recheck", after its 2-minute cooldown). Tested on our tracker's 409 clicks since its Sep 29 calibration: a one-dot check set from any one of them put the next 2 minutes' clicks within 15 degrees (99% within 4.2) and the next 10 minutes' within 25 (the far ones after the headset moved), so the check gets back well under it. The limit used to be 8 degrees, and live on 2026-10-01 our tracker was 12 off after the headset went on, so every correction was dropped. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
|
||||||
- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
|
- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. Our tracker's first calibration has no gaze to go on, since ft-eyes maps pupils to a gaze only once it has a calibration: it opens once SteamVR's tracker sees an eye and ft-eyes answers, and a click takes the 0.6 s up to it, as long as ft-eyes saw each pupil held still then (before 2026-10-05 it waited for a gaze, so a fresh install could never calibrate ours). A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
|
||||||
- Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
|
- Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
|
||||||
- **Idle while the gaze isn't used:** ft-gaze and our own tracker run only while gaze mode is on and someone wears the headset (the pointer helper says both: SteamVR drops the headset's activity level as soon as it comes off), while a check or the calibration is open or asked for, or while the Gaze page of Frametop Input Settings is open (it renews a `wake` lease). 30 seconds after the last use they stop, and our frame grabber goes idle with our tracker. With our tracker, that saves over half a core: on 2026-10-02, with gaze mode off, ft-eyes took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. A check asked for while it idles starts the tracker and opens once it sends. When the gaze is used again, it takes a few seconds to come back, and our tracker's first click re-seats it, as after the headset was off: so the quick check opens when gaze mode comes on after the service idled, as it does when you put the headset on. `ft-gazectl status` says `"awake"`, and `"idle"` says why it isn't. A stand that covers the proximity sensor makes the headset seem worn, so with gaze mode on it doesn't idle there.
|
- **Idle while the gaze isn't used:** ft-gaze and our own tracker run only while gaze mode is on and someone wears the headset (the pointer helper says both: SteamVR drops the headset's activity level as soon as it comes off), while a check or the calibration is open or asked for, or while the Gaze page of Frametop Input Settings is open (it renews a `wake` lease). 30 seconds after the last use they stop, and our frame grabber goes idle with our tracker. With our tracker, that saves over half a core: on 2026-10-02, with gaze mode off, ft-eyes took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. A check asked for while it idles starts the tracker and opens once it sends. When the gaze is used again, it takes a few seconds to come back, and our tracker's first click re-seats it, as after the headset was off: so the quick check opens when gaze mode comes on after the service idled, as it does when you put the headset on. `ft-gazectl status` says `"awake"`, and `"idle"` says why it isn't. A stand that covers the proximity sensor makes the headset seem worn, so with gaze mode on it doesn't idle there.
|
||||||
|
|
||||||
|
|||||||
+126
-16
@@ -98,6 +98,10 @@ double NowRaw() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// --- eye-server.mmap (packed, unaligned: read with memcpy) ---
|
// --- eye-server.mmap (packed, unaligned: read with memcpy) ---
|
||||||
|
// The SteamOS 0.4.x beta (SteamVR 2.18.2) moved every field from the timestamp on by 5
|
||||||
|
// bytes (measured 2026-10-04 with the ftdiag scan: timestamp 0x157 -> 0x15c, the vectors
|
||||||
|
// moved with it; the counter at 0x38 kept its place). Which layout is live is detected at
|
||||||
|
// runtime (EyeFile::Detect), so one binary serves both generations.
|
||||||
constexpr size_t kCounter = 0x38; // u32, one per sample
|
constexpr size_t kCounter = 0x38; // u32, one per sample
|
||||||
constexpr size_t kTime = 0x157; // f64, CLOCK_MONOTONIC_RAW seconds
|
constexpr size_t kTime = 0x157; // f64, CLOCK_MONOTONIC_RAW seconds
|
||||||
constexpr size_t kLeft1 = 0x15f, kRight1 = 0x16b; // set 1: unit vectors, head space
|
constexpr size_t kLeft1 = 0x15f, kRight1 = 0x16b; // set 1: unit vectors, head space
|
||||||
@@ -110,11 +114,19 @@ constexpr size_t kVar1 = 0x177, kVar2 = 0x1b3;
|
|||||||
// The measurements the filter is fed: left x, y, right x, y, then the variance of each (left
|
// The measurements the filter is fed: left x, y, right x, y, then the variance of each (left
|
||||||
// x, y, right x, y). An eye's pair stops changing while the tracker can't see it.
|
// x, y, right x, y). An eye's pair stops changing while the tracker can't see it.
|
||||||
constexpr size_t kMeas = 0x1d3;
|
constexpr size_t kMeas = 0x1d3;
|
||||||
constexpr size_t kNeed = 0x1f3;
|
constexpr size_t kNeed = 0x1f3 + 5; // enough for either layout
|
||||||
|
constexpr size_t kShifts[] = {0, 5}; // the layouts EyeFile::Detect knows: stable, 0.4.x beta
|
||||||
|
|
||||||
struct EyeFile {
|
struct EyeFile {
|
||||||
|
// Everything from the timestamp on is read at base + shift: 0 on stable, 5 on the
|
||||||
|
// 0.4.x beta (see the constants above). known once Detect has seen that layout's
|
||||||
|
// timestamp tick; before that, reading would yield garbage that still passes
|
||||||
|
// ReadSample's check.
|
||||||
|
size_t shift = 0;
|
||||||
|
bool known = false;
|
||||||
const uint8_t *p = nullptr;
|
const uint8_t *p = nullptr;
|
||||||
size_t size = 0;
|
size_t size = 0;
|
||||||
|
size_t At(size_t base) const { return base + shift; }
|
||||||
bool Open() {
|
bool Open() {
|
||||||
const int fd = open("/dev/shm/eye-server.mmap", O_RDONLY | O_CLOEXEC);
|
const int fd = open("/dev/shm/eye-server.mmap", O_RDONLY | O_CLOEXEC);
|
||||||
if (fd < 0) return false;
|
if (fd < 0) return false;
|
||||||
@@ -130,6 +142,9 @@ struct EyeFile {
|
|||||||
size = st.st_size;
|
size = st.st_size;
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
// The sample counter, or 0 without the mapping: the one read the loop makes whether or
|
||||||
|
// not the file was there (Get, V and ReadSample need it).
|
||||||
|
uint32_t Counter() const { return p ? Get<uint32_t>(kCounter) : 0; }
|
||||||
template <class T> T Get(size_t off) const {
|
template <class T> T Get(size_t off) const {
|
||||||
T v;
|
T v;
|
||||||
std::memcpy(&v, p + off, sizeof v);
|
std::memcpy(&v, p + off, sizeof v);
|
||||||
@@ -140,6 +155,53 @@ struct EyeFile {
|
|||||||
std::memcpy(f, p + off, sizeof f);
|
std::memcpy(f, p + off, sizeof f);
|
||||||
return {f[0], f[1], f[2]};
|
return {f[0], f[1], f[2]};
|
||||||
}
|
}
|
||||||
|
// A live timestamp is near the clock it comes from (its samples are 17 ms or so old).
|
||||||
|
static bool TimePlausible(double t, double now) { return t > now - 2.0 && t <= now + 2.0; }
|
||||||
|
// The eye directions are unit vectors, so their length is a second, independent check
|
||||||
|
// next to the timestamp: a mere coincidence in one field does not confirm a layout.
|
||||||
|
static bool UnitVec(Vec3 v) {
|
||||||
|
const float n = v.x * v.x + v.y * v.y + v.z * v.z;
|
||||||
|
return n > 0.81f && n < 1.21f; // |v| within 0.9 .. 1.1
|
||||||
|
}
|
||||||
|
bool Fits(size_t layout, double now) const {
|
||||||
|
return TimePlausible(Get<double>(kTime + layout), now) && UnitVec(V(kLeft1 + layout)) &&
|
||||||
|
UnitVec(V(kRight1 + layout));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Which layout is live, a step per pass of the loop, so it never stalls it. A layout
|
||||||
|
// fits when its timestamp is near the clock and its two set-1 directions are unit
|
||||||
|
// vectors; it's taken once its timestamp has moved on too, within kConfirm. That allows
|
||||||
|
// for 2 samples a second: the eye server writes 72 or 90 (15 were seen on the beta).
|
||||||
|
// kWaiting: call again on the next pass. kNone: no layout fits (a server that stopped,
|
||||||
|
// SteamVR's tracker warming up, or a layout we don't know), so the mmap stays unused;
|
||||||
|
// try again later. One detection per run is enough: the layout can't change under a
|
||||||
|
// running ft-gaze, since SteamVR starts the eye server that writes the file, and the
|
||||||
|
// gaze service stops and starts with SteamVR.
|
||||||
|
enum Detection { kWaiting, kFound, kNone };
|
||||||
|
static constexpr double kConfirm = 0.5;
|
||||||
|
static constexpr size_t kLayouts = sizeof kShifts / sizeof kShifts[0];
|
||||||
|
Detection Detect(double now) {
|
||||||
|
if (!fit_) {
|
||||||
|
for (size_t i = 0; i < kLayouts; ++i)
|
||||||
|
if (Fits(kShifts[i], now)) fit_ |= 1u << i, t0_[i] = Get<double>(kTime + kShifts[i]);
|
||||||
|
if (!fit_) return kNone;
|
||||||
|
since_ = now;
|
||||||
|
return kWaiting;
|
||||||
|
}
|
||||||
|
for (size_t i = 0; i < kLayouts; ++i)
|
||||||
|
if ((fit_ >> i & 1) && Get<double>(kTime + kShifts[i]) > t0_[i] && Fits(kShifts[i], now)) {
|
||||||
|
fit_ = 0, shift = kShifts[i], known = true;
|
||||||
|
return kFound;
|
||||||
|
}
|
||||||
|
if (now - since_ <= kConfirm) return kWaiting;
|
||||||
|
fit_ = 0;
|
||||||
|
return kNone;
|
||||||
|
}
|
||||||
|
bool Detecting() const { return fit_ != 0; }
|
||||||
|
|
||||||
|
// Detect's state: the layouts that fit at since_ (a bit each), and their timestamps then.
|
||||||
|
unsigned fit_ = 0;
|
||||||
|
double t0_[kLayouts] = {}, since_ = 0;
|
||||||
};
|
};
|
||||||
|
|
||||||
struct EyeSample {
|
struct EyeSample {
|
||||||
@@ -151,20 +213,22 @@ struct EyeSample {
|
|||||||
};
|
};
|
||||||
|
|
||||||
// A consistent copy: the writer has no seqlock we can use, so read until the counter and
|
// A consistent copy: the writer has no seqlock we can use, so read until the counter and
|
||||||
// timestamp are the same before and after.
|
// timestamp are the same before and after. Only call this once the layout is known
|
||||||
|
// (EyeFile::known): on an unknown layout these reads still pass the consistency check,
|
||||||
|
// but yield garbage.
|
||||||
bool ReadSample(const EyeFile &f, EyeSample &s) {
|
bool ReadSample(const EyeFile &f, EyeSample &s) {
|
||||||
for (int attempt = 0; attempt < 4; ++attempt) {
|
for (int attempt = 0; attempt < 4; ++attempt) {
|
||||||
const uint32_t n0 = f.Get<uint32_t>(kCounter);
|
const uint32_t n0 = f.Get<uint32_t>(kCounter);
|
||||||
const double t0 = f.Get<double>(kTime);
|
const double t0 = f.Get<double>(f.At(kTime));
|
||||||
std::atomic_thread_fence(std::memory_order_acquire);
|
std::atomic_thread_fence(std::memory_order_acquire);
|
||||||
s.left1 = f.V(kLeft1), s.right1 = f.V(kRight1), s.fix1 = f.V(kFix1);
|
s.left1 = f.V(f.At(kLeft1)), s.right1 = f.V(f.At(kRight1)), s.fix1 = f.V(f.At(kFix1));
|
||||||
s.left2 = f.V(kLeft2), s.right2 = f.V(kRight2);
|
s.left2 = f.V(f.At(kLeft2)), s.right2 = f.V(f.At(kRight2));
|
||||||
std::memcpy(s.open, f.p + kOpen, sizeof s.open);
|
std::memcpy(s.open, f.p + f.At(kOpen), sizeof s.open);
|
||||||
std::memcpy(s.var1, f.p + kVar1, sizeof s.var1);
|
std::memcpy(s.var1, f.p + f.At(kVar1), sizeof s.var1);
|
||||||
std::memcpy(s.var2, f.p + kVar2, sizeof s.var2);
|
std::memcpy(s.var2, f.p + f.At(kVar2), sizeof s.var2);
|
||||||
std::memcpy(s.meas, f.p + kMeas, sizeof s.meas);
|
std::memcpy(s.meas, f.p + f.At(kMeas), sizeof s.meas);
|
||||||
std::atomic_thread_fence(std::memory_order_acquire);
|
std::atomic_thread_fence(std::memory_order_acquire);
|
||||||
if (f.Get<uint32_t>(kCounter) == n0 && f.Get<double>(kTime) == t0) {
|
if (f.Get<uint32_t>(kCounter) == n0 && f.Get<double>(f.At(kTime)) == t0) {
|
||||||
s.n = n0, s.t = t0;
|
s.n = n0, s.t = t0;
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
@@ -525,7 +589,7 @@ int main(int argc, char **argv) {
|
|||||||
std::fprintf(stderr, "ft-gaze: action manifest %s: error %d\n", manifest.c_str(), int(me));
|
std::fprintf(stderr, "ft-gaze: action manifest %s: error %d\n", manifest.c_str(), int(me));
|
||||||
|
|
||||||
EyeFile eyes;
|
EyeFile eyes;
|
||||||
const bool haveMmap = eyes.Open();
|
bool haveMmap = eyes.Open();
|
||||||
std::fprintf(stderr, "ft-gaze: eye-server.mmap %s\n", haveMmap ? "open" : "not available");
|
std::fprintf(stderr, "ft-gaze: eye-server.mmap %s\n", haveMmap ? "open" : "not available");
|
||||||
|
|
||||||
OwnFile ownFile;
|
OwnFile ownFile;
|
||||||
@@ -544,6 +608,19 @@ int main(int argc, char **argv) {
|
|||||||
// over the dashboard. Games are told apart the way ft-screens does it, by the scene app.
|
// over the dashboard. Games are told apart the way ft-screens does it, by the scene app.
|
||||||
bool inGame = false;
|
bool inGame = false;
|
||||||
double nextGameCheck = 0;
|
double nextGameCheck = 0;
|
||||||
|
// The mmap's layout (EyeFile::Detect) is looked for while it isn't known and the eye
|
||||||
|
// server writes: the counter (0x38 in both layouts) ticks once per sample, so a silent
|
||||||
|
// server (headset off) costs nothing and logs nothing. Until a layout is known, SteamVR's
|
||||||
|
// tracker counts as unavailable. "Not recognized" waits until no layout has fitted for
|
||||||
|
// kUnknownAfter of writing, longer than SteamVR's tracker takes to warm up after the
|
||||||
|
// headset goes on (about 20 s), then goes to the journal at most once a minute; ft-gazed
|
||||||
|
// shows it on the Gaze page.
|
||||||
|
constexpr double kUnknownAfter = 30;
|
||||||
|
double nextLayoutCheck = 0, nextLayoutLog = 0, lastWrite = 0, missSince = 0;
|
||||||
|
uint32_t lastCounterSeen = eyes.Counter();
|
||||||
|
// SteamVR's eye tracker creates the mmap a second or two after SteamVR starts, so it can
|
||||||
|
// be missing when we start: look for it again every 2 s until it's there.
|
||||||
|
double nextOpen = NowRaw() + 2.0;
|
||||||
|
|
||||||
while (true) {
|
while (true) {
|
||||||
const double now = NowRaw();
|
const double now = NowRaw();
|
||||||
@@ -559,11 +636,43 @@ int main(int argc, char **argv) {
|
|||||||
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &hp, 1);
|
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &hp, 1);
|
||||||
if (hp.bPoseIsValid) history.Add(now, hp.mDeviceToAbsoluteTracking);
|
if (hp.bPoseIsValid) history.Add(now, hp.mDeviceToAbsoluteTracking);
|
||||||
|
|
||||||
// One line per new eye sample, or at 90 Hz without the mmap.
|
if (!haveMmap && now >= nextOpen) {
|
||||||
|
nextOpen = now + 2.0;
|
||||||
|
if ((haveMmap = eyes.Open())) {
|
||||||
|
std::fprintf(stderr, "ft-gaze: eye-server.mmap open\n");
|
||||||
|
lastCounterSeen = eyes.Counter();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// One line per new eye sample, or at 90 Hz without a usable mmap: none, or one whose
|
||||||
|
// layout isn't known (yet). Our own tracker needs nothing from the mmap, so it keeps
|
||||||
|
// going then; the mmap's sources print as {"ok":0} and "eye" as null.
|
||||||
EyeSample s;
|
EyeSample s;
|
||||||
bool fresh = false;
|
bool fresh = false;
|
||||||
if (haveMmap && ReadSample(eyes, s) && s.n != lastN) fresh = true, lastN = s.n;
|
const uint32_t counter = eyes.Counter();
|
||||||
if (!haveMmap && now - lastEmit >= 1.0 / 90) fresh = true, s.t = now;
|
const bool writing = haveMmap && counter != lastCounterSeen;
|
||||||
|
lastCounterSeen = counter;
|
||||||
|
if (writing) {
|
||||||
|
if (now - lastWrite > 2.0) missSince = 0; // it was silent: start counting again
|
||||||
|
lastWrite = now;
|
||||||
|
}
|
||||||
|
if (haveMmap && !eyes.known && (eyes.Detecting() || (writing && now >= nextLayoutCheck))) {
|
||||||
|
const EyeFile::Detection d = eyes.Detect(now);
|
||||||
|
if (d == EyeFile::kFound) {
|
||||||
|
std::fprintf(stderr, "ft-gaze: eye-server.mmap layout: %s\n", eyes.shift ? "beta (+5)" : "stable");
|
||||||
|
} else if (d == EyeFile::kNone) {
|
||||||
|
nextLayoutCheck = now + 1.0;
|
||||||
|
if (!missSince) missSince = now;
|
||||||
|
if (now - missSince >= kUnknownAfter && now >= nextLayoutLog) {
|
||||||
|
nextLayoutLog = now + 60.0;
|
||||||
|
std::fprintf(stderr, "ft-gaze: eye-server.mmap has eye data, but its layout is not recognized: "
|
||||||
|
"SteamVR's eye tracking is unavailable\n");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const bool mmapOk = haveMmap && eyes.known;
|
||||||
|
if (mmapOk && ReadSample(eyes, s) && s.n != lastN) fresh = true, lastN = s.n;
|
||||||
|
if (!mmapOk && now - lastEmit >= 1.0 / 90) fresh = true, s.t = now;
|
||||||
|
|
||||||
if (fresh && hp.bPoseIsValid) {
|
if (fresh && hp.bPoseIsValid) {
|
||||||
lastEmit = now;
|
lastEmit = now;
|
||||||
@@ -571,7 +680,7 @@ int main(int argc, char **argv) {
|
|||||||
const auto list = screens.Get();
|
const auto list = screens.Get();
|
||||||
const vr::HmdMatrix34_t &headNow = hp.mDeviceToAbsoluteTracking;
|
const vr::HmdMatrix34_t &headNow = hp.mDeviceToAbsoluteTracking;
|
||||||
vr::HmdMatrix34_t headThen = headNow;
|
vr::HmdMatrix34_t headThen = headNow;
|
||||||
if (haveMmap) history.At(s.t, headThen);
|
if (mmapOk) history.At(s.t, headThen);
|
||||||
|
|
||||||
// SteamVR's action: a room-space origin and fixation point, turned into the head
|
// SteamVR's action: a room-space origin and fixation point, turned into the head
|
||||||
// frame so every source reports the same kind of angles.
|
// frame so every source reports the same kind of angles.
|
||||||
@@ -599,7 +708,8 @@ int main(int argc, char **argv) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
std::string m1 = "{\"ok\":0}", m2 = m1, left = m1, right = m1, eye = "null";
|
std::string m1 = "{\"ok\":0}", m2 = m1, left = m1, right = m1, eye = "null";
|
||||||
if (haveMmap) {
|
// Only from a known layout: the unread sample's zeros would print an "unc" of 0 (eyes seen).
|
||||||
|
if (mmapOk) {
|
||||||
// lr: the angle between the two eyes' directions. It's a fraction of a degree
|
// lr: the angle between the two eyes' directions. It's a fraction of a degree
|
||||||
// normally; when the tracker loses one eye (or during a blink) it jumps.
|
// normally; when the tracker loses one eye (or during a blink) it jumps.
|
||||||
auto lr = [](Vec3 l, Vec3 r) {
|
auto lr = [](Vec3 l, Vec3 r) {
|
||||||
|
|||||||
@@ -267,6 +267,7 @@ class Service:
|
|||||||
self.checks = Checks(self, self.sel)
|
self.checks = Checks(self, self.sel)
|
||||||
self.proc = None
|
self.proc = None
|
||||||
self.proc_sources = None # what ft-gaze was last told to print
|
self.proc_sources = None # what ft-gaze was last told to print
|
||||||
|
self.mmap_unknown = False # ft-gaze said SteamVR's eye-server.mmap has a layout it doesn't know
|
||||||
self.buf = b""
|
self.buf = b""
|
||||||
self.restart_at = 0.0
|
self.restart_at = 0.0
|
||||||
self.running = True
|
self.running = True
|
||||||
@@ -511,6 +512,7 @@ class Service:
|
|||||||
self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
|
self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
|
||||||
self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
|
self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
|
||||||
self.buf = b""
|
self.buf = b""
|
||||||
|
self.mmap_unknown = False
|
||||||
log("ft-gaze started")
|
log("ft-gaze started")
|
||||||
|
|
||||||
def stop_helper(self):
|
def stop_helper(self):
|
||||||
@@ -531,6 +533,7 @@ class Service:
|
|||||||
except ProcessLookupError:
|
except ProcessLookupError:
|
||||||
pass
|
pass
|
||||||
self.proc = None
|
self.proc = None
|
||||||
|
self.mmap_unknown = False
|
||||||
|
|
||||||
def eyes_wanted(self):
|
def eyes_wanted(self):
|
||||||
return (self.tracker == "own" and not self.override and self.awake) or time.monotonic() < self.eyes_until
|
return (self.tracker == "own" and not self.override and self.awake) or time.monotonic() < self.eyes_until
|
||||||
@@ -613,6 +616,11 @@ class Service:
|
|||||||
for line in data.decode("utf-8", "replace").splitlines():
|
for line in data.decode("utf-8", "replace").splitlines():
|
||||||
if line.strip():
|
if line.strip():
|
||||||
log(line)
|
log(line)
|
||||||
|
# For the Gaze page (gazecheck's problem): ft-gaze can't read SteamVR's tracker.
|
||||||
|
if "layout is not recognized" in line:
|
||||||
|
self.mmap_unknown = True
|
||||||
|
elif "eye-server.mmap layout:" in line:
|
||||||
|
self.mmap_unknown = False
|
||||||
|
|
||||||
def judge_eyes(self, m1, down):
|
def judge_eyes(self, m1, down):
|
||||||
"""Which eyes (left, right) are closed, from SteamVR's openness (set 1); updates
|
"""Which eyes (left, right) are closed, from SteamVR's openness (set 1); updates
|
||||||
|
|||||||
+45
-6
@@ -48,6 +48,13 @@ click with nothing taken after ACCEPT_WAIT, and a failed calibration names its m
|
|||||||
reason there and in the status. A right click or Meta+K ("calquit") closes the panel. The pointer hides meanwhile ("calpanel 1",
|
reason there and in the status. A right click or Meta+K ("calquit") closes the panel. The pointer hides meanwhile ("calpanel 1",
|
||||||
renewed every second; the helper shows it again by itself when that stops).
|
renewed every second; the helper shows it again by itself when that stops).
|
||||||
|
|
||||||
|
Our tracker's first calibration: before it has one, ft-eyes publishes no gaze (it maps pupils
|
||||||
|
to a gaze only with a calibration), so there's no gaze to hold still. Its calibration runs
|
||||||
|
anyway ("blind"): someone in the headset (SteamVR's tracker sees an eye) and ft-eyes answering
|
||||||
|
are enough to start it, each dot stands in for the gaze, and a click takes the CHECK_WINDOW up
|
||||||
|
to it. ft-eyes then checks that each pupil was seen and held still in that window (calib-point)
|
||||||
|
and says why not. Without this, a fresh install could never calibrate our tracker.
|
||||||
|
|
||||||
What a capture teaches:
|
What a capture teaches:
|
||||||
our tracker quick and five: a click ("click T YAW PITCH", like a pointer lesson); full:
|
our tracker quick and five: a click ("click T YAW PITCH", like a pointer lesson); full:
|
||||||
calib-start, a calib-point for each dot, calib-fit (its calibration)
|
calib-start, a calib-point for each dot, calib-fit (its calibration)
|
||||||
@@ -336,9 +343,17 @@ class Checks:
|
|||||||
return time.monotonic() - self.seen_at < within
|
return time.monotonic() - self.seen_at < within
|
||||||
|
|
||||||
def can_run(self):
|
def can_run(self):
|
||||||
"""Someone's in the headset and the tracker is sending."""
|
"""Someone's in the headset and the tracker is sending. Our tracker sends no gaze before
|
||||||
|
its first calibration: then ft-eyes answering (calibrated() is False only once it has)
|
||||||
|
is enough, since the calibration is what it needs (see "first calibration" at the top)."""
|
||||||
|
if self.blind():
|
||||||
|
return self.eyes_seen()
|
||||||
return self.eyes_seen() and time.monotonic() - self.svc.last_sample < 2
|
return self.eyes_seen() and time.monotonic() - self.svc.last_sample < 2
|
||||||
|
|
||||||
|
def blind(self):
|
||||||
|
"""Our tracker is in use, running, and not calibrated yet: it has no gaze to send."""
|
||||||
|
return self.svc.kind == "own" and self.calibrated() is False
|
||||||
|
|
||||||
def on_gaze_on(self):
|
def on_gaze_on(self):
|
||||||
self.full_armed = True
|
self.full_armed = True
|
||||||
self.need_full("gaze mode came on without a calibration")
|
self.need_full("gaze mode came on without a calibration")
|
||||||
@@ -356,7 +371,7 @@ class Checks:
|
|||||||
if not self.can_run() and self.svc.waking():
|
if not self.can_run() and self.svc.waking():
|
||||||
return # the tracker is still starting (the service idled)
|
return # the tracker is still starting (the service idled)
|
||||||
if not self.can_run():
|
if not self.can_run():
|
||||||
why = ("the eye tracker isn't sending" if now - self.svc.last_sample >= 2
|
why = ("the eye tracker isn't sending" if now - self.svc.last_sample >= 2 and not self.blind()
|
||||||
else "no eyes seen (is the headset on?)")
|
else "no eyes seen (is the headset on?)")
|
||||||
elif now < self.full_retry_at:
|
elif now < self.full_retry_at:
|
||||||
return
|
return
|
||||||
@@ -374,7 +389,16 @@ class Checks:
|
|||||||
def problem(self):
|
def problem(self):
|
||||||
"""Why gaze mode, on, can't follow your eyes yet, or None. Our tracker not having said
|
"""Why gaze mode, on, can't follow your eyes yet, or None. Our tracker not having said
|
||||||
yet is None: Input Settings has its own line for our tracker."""
|
yet is None: Input Settings has its own line for our tracker."""
|
||||||
if not self.gaze_on or self.calibrated() is not False:
|
if not self.gaze_on:
|
||||||
|
return None
|
||||||
|
own = self.svc.kind == "own"
|
||||||
|
if self.svc.mmap_unknown and (not own or self.calibrated() is False):
|
||||||
|
# ft-gaze can't read SteamVR's tracker (EyeFile::Detect in ft-gaze.cpp): not its gaze,
|
||||||
|
# and not the eyes it sees, which our tracker's calibration waits for.
|
||||||
|
return ("SteamVR's eye data has a layout Frametop doesn't know (after a SteamOS update?), so "
|
||||||
|
+ ("the calibration can't open" if own else "SteamVR's eye tracker can't be used")
|
||||||
|
+ ". A newer Frametop may know it")
|
||||||
|
if self.calibrated() is not False:
|
||||||
return None
|
return None
|
||||||
if self.check and self.check["kind"] == "full":
|
if self.check and self.check["kind"] == "full":
|
||||||
return "Not calibrated yet: the calibration is open in the headset"
|
return "Not calibrated yet: the calibration is open in the headset"
|
||||||
@@ -408,6 +432,9 @@ class Checks:
|
|||||||
if not self.can_run():
|
if not self.can_run():
|
||||||
return "error the headset is off or the tracker isn't sending"
|
return "error the headset is off or the tracker isn't sending"
|
||||||
own = svc.kind == "own"
|
own = svc.kind == "own"
|
||||||
|
blind = self.blind()
|
||||||
|
if blind and kind != "full":
|
||||||
|
return "error our tracker isn't calibrated yet: use Calibrate"
|
||||||
if kind == "full" and own:
|
if kind == "full" and own:
|
||||||
reply = ask(EYES, "calib-start", 3.0)
|
reply = ask(EYES, "calib-start", 3.0)
|
||||||
if not reply.startswith("ok"):
|
if not reply.startswith("ok"):
|
||||||
@@ -416,8 +443,10 @@ class Checks:
|
|||||||
now = time.monotonic()
|
now = time.monotonic()
|
||||||
self.check = {"kind": kind, "reason": reason, "own": own, "dots": check_dots(kind, own), "i": 0,
|
self.check = {"kind": kind, "reason": reason, "own": own, "dots": check_dots(kind, own), "i": 0,
|
||||||
"started": now, "shown": now, "run": [], "accept": False, "done_at": None, "tries": 0,
|
"started": now, "shown": now, "run": [], "accept": False, "done_at": None, "tries": 0,
|
||||||
"skipped": 0, "captured": 0, "points": {}, "reasons": {}, "fit_reasons": set(), "note": ""}
|
"skipped": 0, "captured": 0, "points": {}, "reasons": {}, "fit_reasons": set(), "note": "",
|
||||||
log(f"{kind} check: {reason}")
|
"blind": blind}
|
||||||
|
log(f"{kind} check: {reason}" + (" (our tracker's first: no gaze yet, so each click takes the look "
|
||||||
|
"up to it)" if blind else ""))
|
||||||
if kind == "full":
|
if kind == "full":
|
||||||
st = ask(SCREENS, "state", 0.5).split()
|
st = ask(SCREENS, "state", 0.5).split()
|
||||||
if len(st) >= 3 and st[0] == "ok":
|
if len(st) >= 3 and st[0] == "ok":
|
||||||
@@ -519,6 +548,12 @@ class Checks:
|
|||||||
return
|
return
|
||||||
if c["own"]:
|
if c["own"]:
|
||||||
src = s["src"].get("own") or {}
|
src = s["src"].get("own") or {}
|
||||||
|
if c["blind"]:
|
||||||
|
# Our tracker's first calibration: no gaze yet, so the dot stands in for it (the
|
||||||
|
# gaze can't seem to move) and a click takes the look up to it. ft-eyes checks
|
||||||
|
# the pupils held still (see the top).
|
||||||
|
yaw, pitch, _ = c["dots"][c["i"]]
|
||||||
|
src = {"hy": yaw, "hp": pitch}
|
||||||
else:
|
else:
|
||||||
src = s["src"].get("mmap1") or {}
|
src = s["src"].get("mmap1") or {}
|
||||||
if "hy" not in src:
|
if "hy" not in src:
|
||||||
@@ -868,7 +903,11 @@ class Checks:
|
|||||||
self.back_since = None # gone again before DON_DELAY
|
self.back_since = None # gone again before DON_DELAY
|
||||||
elif self.back_since is not None and now - self.back_since >= DON_DELAY:
|
elif self.back_since is not None and now - self.back_since >= DON_DELAY:
|
||||||
self.away, self.back_since = False, None
|
self.away, self.back_since = False, None
|
||||||
self.full_armed = True # a calibration that closed unfinished opens again
|
if not self.check:
|
||||||
|
# A calibration that closed unfinished opens again. Not one still open: gaze mode
|
||||||
|
# coming on wakes our tracker, so its eyes come back just as the calibration
|
||||||
|
# opens, and re-arming then opened a second one when the first ended.
|
||||||
|
self.full_armed = True
|
||||||
self.auto_quick("the headset went on")
|
self.auto_quick("the headset went on")
|
||||||
if svc.kind == "own" and now - svc.own_at < 5:
|
if svc.kind == "own" and now - svc.own_at < 5:
|
||||||
reseat = any(e.get("reseat") for e in (svc.own.get("eyes") or {}).values())
|
reseat = any(e.get("reseat") for e in (svc.own.get("eyes") or {}).values())
|
||||||
|
|||||||
@@ -203,7 +203,7 @@ class GazeReader:
|
|||||||
return
|
return
|
||||||
print(line, file=sys.stderr)
|
print(line, file=sys.stderr)
|
||||||
text = line.removeprefix("ft-gaze: ")
|
text = line.removeprefix("ft-gaze: ")
|
||||||
if not text.startswith(("action manifest", "eye-server.mmap open")):
|
if not text.startswith(("action manifest", "eye-server.mmap open", "eye-server.mmap layout:")):
|
||||||
self.last_err = text # worth repeating if ft-gaze stops (not the startup lines)
|
self.last_err = text # worth repeating if ft-gaze stops (not the startup lines)
|
||||||
self.on_status(text)
|
self.on_status(text)
|
||||||
stream.read_line_async(GLib.PRIORITY_DEFAULT, self.cancel, got)
|
stream.read_line_async(GLib.PRIORITY_DEFAULT, self.cancel, got)
|
||||||
|
|||||||
Executable
+238
@@ -0,0 +1,238 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Offline test of our own eye tracker's first calibration (gaze/gazecheck.py, "blind"): before
|
||||||
|
it has a calibration, ft-eyes publishes no gaze, and the calibration must still open and take
|
||||||
|
its dots. Until 2026-10-05 it couldn't: a fresh install that chose our tracker never calibrated.
|
||||||
|
|
||||||
|
Runs ft-gazed's Service with its sockets renamed and HOME in a temp folder (state and settings
|
||||||
|
go there), a fake pointer helper (gaze mode on, headset worn), a fake ft-gaze (SteamVR sees both
|
||||||
|
eyes; "own" is {"ok":0}, as with an uncalibrated ft-eyes), a fake ft-eyes control socket, and no
|
||||||
|
panel (a stand-in process). Nothing reaches the live gaze service, the pointer helper, ft-eyes,
|
||||||
|
or SteamVR, so it's safe next to them.
|
||||||
|
|
||||||
|
gaze/test/first-calibration-test.py
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
HOME = tempfile.mkdtemp(prefix="ft-gaze-first-cal-test-")
|
||||||
|
os.environ["HOME"] = HOME # before gazecal: its STATE, and frametop.conf, follow HOME
|
||||||
|
|
||||||
|
import importlib.machinery # noqa: E402
|
||||||
|
import importlib.util # noqa: E402
|
||||||
|
import json # noqa: E402
|
||||||
|
import selectors # noqa: E402
|
||||||
|
import shutil # noqa: E402
|
||||||
|
import socket # noqa: E402
|
||||||
|
import subprocess # noqa: E402
|
||||||
|
import sys # noqa: E402
|
||||||
|
import threading # noqa: E402
|
||||||
|
import time # noqa: E402
|
||||||
|
|
||||||
|
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
GAZE = os.path.join(HERE, "..")
|
||||||
|
sys.path.insert(0, GAZE)
|
||||||
|
loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(GAZE, "ft-gazed"))
|
||||||
|
gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
|
||||||
|
loader.exec_module(gazed)
|
||||||
|
import gazecheck # noqa: E402 (the module ft-gazed imported)
|
||||||
|
|
||||||
|
tag = f"ft_gaze_first_cal_test_{os.getpid()}"
|
||||||
|
gazed.ME = f"\0{tag}_gazed"
|
||||||
|
gazed.POINTER = gazecheck.POINTER = f"\0{tag}_helper"
|
||||||
|
gazecheck.SCREENS = f"\0{tag}_screens"
|
||||||
|
gazecheck.PANEL = f"\0{tag}_panel"
|
||||||
|
gazed.EYES_SOCKET = gazecheck.EYES = f"\0{tag}_eyes"
|
||||||
|
gazed.read_settings = lambda: ("own", "auto", "auto", 55.0)
|
||||||
|
DOTS = 3
|
||||||
|
real_dots = gazecheck.check_dots
|
||||||
|
gazecheck.check_dots = lambda kind, own: real_dots(kind, own)[:DOTS] # a short calibration
|
||||||
|
logs = []
|
||||||
|
gazed.log = gazecheck.log = lambda msg: logs.append(msg)
|
||||||
|
|
||||||
|
# The fake ft-gaze: 90 samples a second, SteamVR sees both eyes, no gaze from ours.
|
||||||
|
FAKE = os.path.join(HOME, "ft-gaze")
|
||||||
|
with open(FAKE, "w") as f:
|
||||||
|
f.write('''import json, os, select, sys, time
|
||||||
|
while True:
|
||||||
|
if select.select([sys.stdin], [], [], 1 / 90)[0]:
|
||||||
|
if not os.read(0, 4096):
|
||||||
|
break
|
||||||
|
print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
|
||||||
|
"open": [0.8, 0.8]}, "own": {"ok": 0}}}), flush=True)
|
||||||
|
''')
|
||||||
|
SLEEPER = [sys.executable, "-c", "import sys; sys.stdin.read()"] # quits when its stdin closes
|
||||||
|
|
||||||
|
|
||||||
|
def start_helper(self):
|
||||||
|
"""ft-gaze, straight from here instead of the dev container."""
|
||||||
|
self.proc = subprocess.Popen([sys.executable, FAKE], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
||||||
|
stderr=subprocess.PIPE)
|
||||||
|
self.proc_sources = self.wanted_sources()
|
||||||
|
os.set_blocking(self.proc.stdout.fileno(), False)
|
||||||
|
os.set_blocking(self.proc.stderr.fileno(), False)
|
||||||
|
self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
|
||||||
|
self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
|
||||||
|
self.buf = b""
|
||||||
|
|
||||||
|
|
||||||
|
def start_eyes(self):
|
||||||
|
"""ft-eyes' process: a stand-in. Its control socket is the fake below."""
|
||||||
|
self.eyes_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||||
|
os.set_blocking(self.eyes_proc.stderr.fileno(), False)
|
||||||
|
self.sel.register(self.eyes_proc.stderr, selectors.EVENT_READ, "eyes")
|
||||||
|
|
||||||
|
|
||||||
|
def start_panel(self):
|
||||||
|
self.panel_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||||
|
os.set_blocking(self.panel_proc.stderr.fileno(), False)
|
||||||
|
self.sel.register(self.panel_proc.stderr, selectors.EVENT_READ, "panel")
|
||||||
|
|
||||||
|
|
||||||
|
gazed.Service.start_helper = start_helper
|
||||||
|
gazed.Service.start_eyes = start_eyes
|
||||||
|
gazecheck.Checks.start_panel = start_panel
|
||||||
|
|
||||||
|
# The fake ft-eyes: uncalibrated until calib-fit. "fail" answers the next calib-point with that.
|
||||||
|
eyes_state = {"cal": None, "points": [], "fail": None}
|
||||||
|
eyes = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
|
||||||
|
eyes.bind(gazed.EYES_SOCKET)
|
||||||
|
eyes.settimeout(0.2)
|
||||||
|
|
||||||
|
|
||||||
|
def eyes_answer():
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
data, addr = eyes.recvfrom(512)
|
||||||
|
except socket.timeout:
|
||||||
|
continue
|
||||||
|
except OSError:
|
||||||
|
return
|
||||||
|
w = data.decode().split()
|
||||||
|
if w[0] == "status":
|
||||||
|
reply = json.dumps({"calibration": eyes_state["cal"], "calibrating": False, "dots": 0,
|
||||||
|
"eyes": {"right": {"reseat": False}, "left": {"reseat": False}}})
|
||||||
|
elif w[0] == "calib-start":
|
||||||
|
eyes_state["points"] = []
|
||||||
|
reply = "ok"
|
||||||
|
elif w[0] == "calib-point":
|
||||||
|
eyes_state["points"].append(tuple(map(float, w[1:5])))
|
||||||
|
reply, eyes_state["fail"] = eyes_state["fail"] or "ok 50 50 1.00 1.00", None
|
||||||
|
elif w[0] == "calib-fit":
|
||||||
|
eyes_state["cal"] = {"made": "test", "dots": len(eyes_state["points"])}
|
||||||
|
reply = f"ok {len(eyes_state['points'])} dots"
|
||||||
|
else:
|
||||||
|
reply = f"fail unknown command {w[0]}"
|
||||||
|
if addr:
|
||||||
|
eyes.sendto(reply.encode(), addr)
|
||||||
|
|
||||||
|
|
||||||
|
threading.Thread(target=eyes_answer, daemon=True).start()
|
||||||
|
|
||||||
|
helper_state = {"reply": "ok off worn", "heard": []}
|
||||||
|
helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
|
||||||
|
helper.bind(gazed.POINTER)
|
||||||
|
helper.settimeout(0.2)
|
||||||
|
|
||||||
|
|
||||||
|
def helper_answer():
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
data, addr = helper.recvfrom(512)
|
||||||
|
except socket.timeout:
|
||||||
|
continue
|
||||||
|
except OSError:
|
||||||
|
return
|
||||||
|
if data.startswith(b"gaze ?") and addr:
|
||||||
|
helper.sendto(helper_state["reply"].encode(), addr)
|
||||||
|
else:
|
||||||
|
helper_state["heard"].append(data.decode())
|
||||||
|
|
||||||
|
|
||||||
|
threading.Thread(target=helper_answer, daemon=True).start()
|
||||||
|
svc = gazed.Service(None, False, gazed.POINTER)
|
||||||
|
threading.Thread(target=svc.run, daemon=True).start()
|
||||||
|
ctl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
|
||||||
|
ctl.bind("")
|
||||||
|
ctl.settimeout(3)
|
||||||
|
|
||||||
|
|
||||||
|
def ask(cmd):
|
||||||
|
ctl.sendto(cmd.encode(), gazed.ME)
|
||||||
|
return ctl.recv(65536).decode()
|
||||||
|
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(label, got, want):
|
||||||
|
ok = got == want
|
||||||
|
print(("ok " if ok else "FAIL ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
|
||||||
|
if not ok:
|
||||||
|
failures.append(label)
|
||||||
|
|
||||||
|
|
||||||
|
def wait(cond, seconds):
|
||||||
|
end = time.monotonic() + seconds
|
||||||
|
while time.monotonic() < end:
|
||||||
|
if cond():
|
||||||
|
return True
|
||||||
|
time.sleep(0.05)
|
||||||
|
return cond()
|
||||||
|
|
||||||
|
|
||||||
|
def check_state():
|
||||||
|
return svc.checks.check or {}
|
||||||
|
|
||||||
|
|
||||||
|
def take_dot(i):
|
||||||
|
"""Wait for dot i to settle, then click: is it taken?"""
|
||||||
|
wait(lambda: check_state().get("i") == i and not check_state().get("done_at"), 3)
|
||||||
|
time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1)
|
||||||
|
before = check_state().get("captured", 0)
|
||||||
|
ask("calaccept")
|
||||||
|
return wait(lambda: check_state().get("captured", 0) > before, 2)
|
||||||
|
|
||||||
|
|
||||||
|
check("gaze mode off, Gaze page open (wake): ours runs, uncalibrated", ask("wake 60"), "ok")
|
||||||
|
check("the service knows ours has no calibration", wait(lambda: svc.checks.calibrated() is False, 6), True)
|
||||||
|
check("a quick check is refused while ours has no calibration", svc.checks.start("quick", "test"),
|
||||||
|
"error our tracker isn't calibrated yet: use Calibrate")
|
||||||
|
helper_state["reply"] = "ok on worn"
|
||||||
|
check("gaze mode on: the calibration opens by itself", wait(lambda: check_state().get("kind") == "full", 6), True)
|
||||||
|
check("it runs blind (no gaze from ours yet)", check_state().get("blind"), True)
|
||||||
|
check("nothing says it can't open", any("can't open" in m for m in logs), False)
|
||||||
|
# Live 2026-10-05: turning gaze mode on woke our tracker, and the calibration opened before
|
||||||
|
# DON_DELAY of eyes had passed, so "the headset went on" came while it was open. That re-armed
|
||||||
|
# it, and a second calibration opened as soon as the first ended.
|
||||||
|
svc.checks.away, svc.checks.back_since = True, None
|
||||||
|
|
||||||
|
check("dot 1: a click takes it", take_dot(0), True)
|
||||||
|
check("the headset went on while it was open",
|
||||||
|
wait(lambda: not svc.checks.away, gazecheck.DON_DELAY + 2) and bool(svc.checks.check), True)
|
||||||
|
t0, t1, yaw, pitch = eyes_state["points"][0]
|
||||||
|
dot = gazecheck.check_dots("full", True)[0]
|
||||||
|
check("its window is the look up to the click (about CHECK_WINDOW)",
|
||||||
|
abs((t1 - t0) - gazecheck.CHECK_WINDOW) < 0.15, True)
|
||||||
|
check("ft-eyes got that dot's direction", (round(yaw, 3), round(pitch, 3)), (round(dot[0], 3), round(dot[1], 3)))
|
||||||
|
|
||||||
|
eyes_state["fail"] = "fail the left eye was seen in only 3 frames"
|
||||||
|
wait(lambda: check_state().get("i") == 1 and not check_state().get("done_at"), 3)
|
||||||
|
time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1)
|
||||||
|
ask("calaccept")
|
||||||
|
check("ft-eyes refusing a dot: its reason reaches the panel's note",
|
||||||
|
wait(lambda: "left eye in only 3 frames" in check_state().get("note", ""), 2), True)
|
||||||
|
check("dot 2, second try: taken", take_dot(1), True)
|
||||||
|
check("dot 3: taken", take_dot(2), True)
|
||||||
|
|
||||||
|
check("all dots: ours fits its calibration (calib-fit)",
|
||||||
|
wait(lambda: eyes_state["cal"] is not None and not svc.checks.check, 4), True)
|
||||||
|
check("the service sees it calibrated", wait(lambda: svc.checks.calibrated() is True, 4), True)
|
||||||
|
check("and gaze mode stays on", "gaze off" in helper_state["heard"], False)
|
||||||
|
check("and no second calibration opens", wait(lambda: svc.checks.check is not None, 3), False)
|
||||||
|
check("one calibration in the log", sum(m.startswith("full check:") for m in logs), 1)
|
||||||
|
|
||||||
|
print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
|
||||||
|
svc.running = False
|
||||||
|
time.sleep(0.7)
|
||||||
|
shutil.rmtree(HOME, ignore_errors=True)
|
||||||
|
os._exit(1 if failures else 0)
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
// Offline test of ft-gaze's eye-server.mmap layout detection (EyeFile::Detect in
|
||||||
|
// gaze/ft-gaze.cpp) and of reading a sample at the layout it found, on a made-up file in
|
||||||
|
// memory: no SteamVR, no eye tracker. Detect takes the time as an argument, so the passes
|
||||||
|
// of ft-gaze's loop are played here with a made-up clock. mmap-layout-test.sh builds and
|
||||||
|
// runs it in the dev container.
|
||||||
|
#define main ft_gaze_main
|
||||||
|
#include "../ft-gaze.cpp"
|
||||||
|
#undef main
|
||||||
|
|
||||||
|
#include <cstdio>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
|
||||||
|
int failures = 0;
|
||||||
|
#define CHECK(c) \
|
||||||
|
do { \
|
||||||
|
if (!(c)) std::printf("FAIL %s:%d: %s\n", __FILE__, __LINE__, #c), ++failures; \
|
||||||
|
} while (0)
|
||||||
|
|
||||||
|
// The file as the eye server writes it, one sample at a time, with every field from the
|
||||||
|
// timestamp on moved by `shift` (0 stable, 5 the 0.4.x beta).
|
||||||
|
struct File {
|
||||||
|
std::vector<uint8_t> bytes = std::vector<uint8_t>(324122); // eye-server.mmap's size
|
||||||
|
EyeFile eyes;
|
||||||
|
uint32_t n = 0;
|
||||||
|
File() { eyes.p = bytes.data(), eyes.size = bytes.size(); }
|
||||||
|
template <class T> void Put(size_t off, const T &v) { std::memcpy(bytes.data() + off, &v, sizeof v); }
|
||||||
|
void Sample(size_t shift, double t, bool leftLost = false) {
|
||||||
|
const float left[3] = {0.05f, 0.02f, -0.9985f}, right[3] = {-0.05f, 0.02f, -0.9985f}, none[3] = {};
|
||||||
|
Put(kCounter, ++n);
|
||||||
|
Put(kTime + shift, t);
|
||||||
|
Put(kLeft1 + shift, leftLost ? none : left);
|
||||||
|
Put(kRight1 + shift, right);
|
||||||
|
Put(kLeft2 + shift, left);
|
||||||
|
Put(kRight2 + shift, right);
|
||||||
|
const float open[2] = {0.8f, 0.7f}, var[6] = {1e-3f, 2e-3f, 3e-3f, 4e-3f, 5e-3f, 6e-3f};
|
||||||
|
const float meas[8] = {0.1f, 0.2f, 0.3f, 0.4f, 2e-5f, 3e-5f, 4e-5f, 5e-5f}, fix[3] = {0, 0.02f, -0.6f};
|
||||||
|
Put(kFix1 + shift, fix);
|
||||||
|
Put(kVar1 + shift, var);
|
||||||
|
Put(kVar2 + shift, var);
|
||||||
|
Put(kOpen + shift, open);
|
||||||
|
Put(kMeas + shift, meas);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
constexpr double kStart = 5000; // the made-up CLOCK_MONOTONIC_RAW at the first pass
|
||||||
|
constexpr double kPass = 0.004; // ft-gaze's loop
|
||||||
|
constexpr double kAge = 0.017; // how old a sample is when it appears
|
||||||
|
|
||||||
|
// Samples every 1/hz s in `shift`'s layout, Detect called on every loop pass from the first
|
||||||
|
// sample on (as ft-gaze does once the counter moved), until it decides or `until` s pass.
|
||||||
|
// Returns the outcome and when (s after the first sample) in `at`.
|
||||||
|
EyeFile::Detection Run(File &f, size_t shift, double hz, double until, double &at) {
|
||||||
|
double next = kStart;
|
||||||
|
for (double now = kStart; now < kStart + until; now += kPass) {
|
||||||
|
if (now >= next) f.Sample(shift, now - kAge), next += 1 / hz;
|
||||||
|
const EyeFile::Detection d = f.eyes.Detect(now);
|
||||||
|
if (d != EyeFile::kWaiting) {
|
||||||
|
at = now - kStart;
|
||||||
|
return d;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
at = until;
|
||||||
|
return EyeFile::kWaiting;
|
||||||
|
}
|
||||||
|
|
||||||
|
void Layouts() {
|
||||||
|
for (const size_t shift : {size_t(0), size_t(5)}) {
|
||||||
|
File f;
|
||||||
|
double at;
|
||||||
|
CHECK(Run(f, shift, 90, 2, at) == EyeFile::kFound);
|
||||||
|
CHECK(f.eyes.known && f.eyes.shift == shift);
|
||||||
|
CHECK(at <= 1 / 90.0 + kPass); // the next sample confirms it
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void SlowWriters() {
|
||||||
|
// 15 a second, as seen on the beta: the next sample, 67 ms on, confirms it (PR #26's
|
||||||
|
// fixed 60 ms wait missed about 1 try in 10 there).
|
||||||
|
File beta;
|
||||||
|
double at;
|
||||||
|
CHECK(Run(beta, 5, 15, 2, at) == EyeFile::kFound && beta.eyes.shift == 5);
|
||||||
|
CHECK(at > 1 / 15.0 - kPass && at <= 1 / 15.0 + kPass);
|
||||||
|
// 3 a second still works (within kConfirm); 1 a second can't confirm in time.
|
||||||
|
File slow;
|
||||||
|
CHECK(Run(slow, 0, 3, 2, at) == EyeFile::kFound && slow.eyes.shift == 0);
|
||||||
|
File slower;
|
||||||
|
CHECK(Run(slower, 0, 1, 2, at) == EyeFile::kNone && !slower.eyes.known);
|
||||||
|
CHECK(at > EyeFile::kConfirm && at < EyeFile::kConfirm + 2 * kPass);
|
||||||
|
CHECK(!slower.eyes.Detecting()); // and the next try starts over
|
||||||
|
}
|
||||||
|
|
||||||
|
void Refused() {
|
||||||
|
// A layout we don't know: the same fields, moved by some other amount.
|
||||||
|
for (size_t shift = 1; shift <= 16; ++shift) {
|
||||||
|
if (shift == 5) continue;
|
||||||
|
File f;
|
||||||
|
double at;
|
||||||
|
const EyeFile::Detection d = Run(f, shift, 90, 1, at);
|
||||||
|
CHECK(d == EyeFile::kNone && !f.eyes.known);
|
||||||
|
if (d != EyeFile::kNone) std::printf(" (moved by %zu)\n", shift);
|
||||||
|
}
|
||||||
|
// A server that stopped: its last sample is 10 s old. Refused at once, no waiting.
|
||||||
|
File stale;
|
||||||
|
stale.Sample(0, kStart - 10);
|
||||||
|
CHECK(stale.eyes.Detect(kStart) == EyeFile::kNone && !stale.eyes.Detecting());
|
||||||
|
// One that stopped just now: its timestamp fits but never moves on.
|
||||||
|
File stopped;
|
||||||
|
stopped.Sample(0, kStart - kAge);
|
||||||
|
CHECK(stopped.eyes.Detect(kStart) == EyeFile::kWaiting);
|
||||||
|
EyeFile::Detection d = EyeFile::kWaiting;
|
||||||
|
double now = kStart;
|
||||||
|
while (d == EyeFile::kWaiting && now < kStart + 2) d = stopped.eyes.Detect(now += kPass);
|
||||||
|
CHECK(d == EyeFile::kNone && !stopped.eyes.known);
|
||||||
|
// A set-1 direction that isn't a unit vector (here zeros).
|
||||||
|
File warm;
|
||||||
|
warm.Sample(0, kStart - kAge, true);
|
||||||
|
CHECK(warm.eyes.Detect(kStart) == EyeFile::kNone);
|
||||||
|
// An empty file (the server never wrote).
|
||||||
|
File empty;
|
||||||
|
CHECK(empty.eyes.Detect(kStart) == EyeFile::kNone);
|
||||||
|
}
|
||||||
|
|
||||||
|
void TornWrite() {
|
||||||
|
// A pass that reads the timestamp mid-write (the counter already moved on, the top half
|
||||||
|
// of the new timestamp not written yet, so it's nowhere near the clock) keeps waiting:
|
||||||
|
// the next pass confirms it.
|
||||||
|
File f;
|
||||||
|
f.Sample(0, kStart - kAge);
|
||||||
|
CHECK(f.eyes.Detect(kStart) == EyeFile::kWaiting);
|
||||||
|
const double next = kStart + 0.011 - kAge;
|
||||||
|
std::memcpy(f.bytes.data() + kTime, &next, 4);
|
||||||
|
std::memset(f.bytes.data() + kTime + 4, 0, 4);
|
||||||
|
f.Put(kCounter, ++f.n);
|
||||||
|
CHECK(f.eyes.Detect(kStart + 0.012) == EyeFile::kWaiting);
|
||||||
|
f.Put(kTime, next);
|
||||||
|
CHECK(f.eyes.Detect(kStart + 0.016) == EyeFile::kFound && f.eyes.shift == 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
void ReadsTheLayoutFound() {
|
||||||
|
// After detection on the beta, a sample comes from the moved fields.
|
||||||
|
File f;
|
||||||
|
double at;
|
||||||
|
CHECK(Run(f, 5, 90, 1, at) == EyeFile::kFound && f.eyes.shift == 5);
|
||||||
|
f.Sample(5, kStart + 1);
|
||||||
|
EyeSample s;
|
||||||
|
CHECK(ReadSample(f.eyes, s));
|
||||||
|
CHECK(s.n == f.n && s.t == kStart + 1);
|
||||||
|
CHECK(std::fabs(s.left1.x - 0.05) < 1e-6 && std::fabs(s.right2.x + 0.05) < 1e-6);
|
||||||
|
CHECK(std::fabs(s.fix1.z + 0.6) < 1e-6);
|
||||||
|
CHECK(s.open[0] == 0.8f && s.open[1] == 0.7f);
|
||||||
|
CHECK(s.var1[5] == 6e-3f && s.var2[0] == 1e-3f && s.meas[3] == 0.4f && s.meas[7] == 5e-5f);
|
||||||
|
}
|
||||||
|
|
||||||
|
void NoMapping() {
|
||||||
|
// ft-gaze without the file: the loop's one read of it gives 0, and touches no memory.
|
||||||
|
EyeFile none;
|
||||||
|
CHECK(none.Counter() == 0);
|
||||||
|
File f;
|
||||||
|
f.Sample(0, kStart);
|
||||||
|
CHECK(f.eyes.Counter() == f.n);
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
int main() {
|
||||||
|
Layouts();
|
||||||
|
SlowWriters();
|
||||||
|
Refused();
|
||||||
|
TornWrite();
|
||||||
|
ReadsTheLayoutFound();
|
||||||
|
NoMapping();
|
||||||
|
if (failures) {
|
||||||
|
std::printf("%d failed\n", failures);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
std::printf("all passed\n");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
Executable
+16
@@ -0,0 +1,16 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Offline test of ft-gaze's eye-server.mmap layout detection (gaze/test/mmap-layout-test.cpp):
|
||||||
|
# builds it against gaze/ft-gaze.cpp in the dev container and runs it there. Nothing reaches
|
||||||
|
# SteamVR or the eye tracker, so it's safe next to them. gaze/build.sh fetches the OpenVR
|
||||||
|
# header it needs, so run that once first.
|
||||||
|
#
|
||||||
|
# gaze/test/mmap-layout-test.sh
|
||||||
|
set -euo pipefail
|
||||||
|
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
|
||||||
|
"$root/scripts/sync.sh" >/dev/null
|
||||||
|
exec "$root/scripts/frame.sh" -C gaze 'set -e
|
||||||
|
[ -f build/include/openvr.h ] || { echo "no build/include/openvr.h: run gaze/build.sh first" >&2; exit 1; }
|
||||||
|
g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -Wno-missing-field-initializers -Ibuild/include -I../pointer/common \
|
||||||
|
-o build/mmap-layout-test test/mmap-layout-test.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api \
|
||||||
|
-Wl,-rpath,/opt/steamvr/bin/linuxarm64 -lpthread
|
||||||
|
build/mmap-layout-test'
|
||||||
@@ -111,6 +111,9 @@ reading only.
|
|||||||
|
|
||||||
The file is 324,122 bytes. Only bytes 0x0-0x1f3 are used; the rest is zero. It's packed and
|
The file is 324,122 bytes. Only bytes 0x0-0x1f3 are used; the rest is zero. It's packed and
|
||||||
unaligned, so read it with memcpy. Offsets are also in `~/frametop/gaze/ft-gaze.cpp`.
|
unaligned, so read it with memcpy. Offsets are also in `~/frametop/gaze/ft-gaze.cpp`.
|
||||||
|
On the SteamOS 0.4.x beta (SteamVR 2.18.2) every field from 0x157 on sits 5 bytes later, and the
|
||||||
|
counter stays at 0x38 (measured 2026-10-04, PR #26). The offsets below are stable's; ft-gaze detects
|
||||||
|
which layout is live.
|
||||||
|
|
||||||
| Offset | What |
|
| Offset | What |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
|
|||||||
@@ -9,6 +9,7 @@
|
|||||||
# Options (piped, they go after "bash -s --"):
|
# Options (piped, they go after "bash -s --"):
|
||||||
# --stable the main branch: tested releases (the default for a new install)
|
# --stable the main branch: tested releases (the default for a new install)
|
||||||
# --experimental the experimental branch: the newest features, less tested
|
# --experimental the experimental branch: the newest features, less tested
|
||||||
|
# --branch NAME another branch, such as a fix to test before it's released
|
||||||
# --dir DIR where the repo goes (default ~/frametop)
|
# --dir DIR where the repo goes (default ~/frametop)
|
||||||
# --clone-only get or update the repo, but don't run install.sh
|
# --clone-only get or update the repo, but don't run install.sh
|
||||||
# --yes, --no-bluetooth passed to install.sh (--yes also answers this script's question:
|
# --yes, --no-bluetooth passed to install.sh (--yes also answers this script's question:
|
||||||
@@ -17,7 +18,7 @@ set -euo pipefail
|
|||||||
|
|
||||||
usage() {
|
usage() {
|
||||||
cat <<'EOF'
|
cat <<'EOF'
|
||||||
usage: get.sh [--stable | --experimental] [--dir DIR] [--clone-only] [--yes] [--no-bluetooth]
|
usage: get.sh [--stable | --experimental | --branch NAME] [--dir DIR] [--clone-only] [--yes] [--no-bluetooth]
|
||||||
piped: curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash -s -- [options]
|
piped: curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash -s -- [options]
|
||||||
EOF
|
EOF
|
||||||
}
|
}
|
||||||
@@ -31,6 +32,7 @@ main() {
|
|||||||
case $1 in
|
case $1 in
|
||||||
--stable) branch=main ;;
|
--stable) branch=main ;;
|
||||||
--experimental) branch=experimental ;;
|
--experimental) branch=experimental ;;
|
||||||
|
--branch) branch=${2:?--branch needs a branch name}; shift ;;
|
||||||
--dir) dir=${2:?--dir needs a folder}; shift ;;
|
--dir) dir=${2:?--dir needs a folder}; shift ;;
|
||||||
--clone-only) clone_only=1 ;;
|
--clone-only) clone_only=1 ;;
|
||||||
--yes) yes=1; pass+=("$1") ;;
|
--yes) yes=1; pass+=("$1") ;;
|
||||||
@@ -80,6 +82,11 @@ main() {
|
|||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
if ! git ls-remote --exit-code --heads "$repo" "$branch" >/dev/null; then
|
||||||
|
echo "Frametop has no branch called $branch (or GitHub can't be reached)." >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
if [ ! -e "$dir" ]; then
|
if [ ! -e "$dir" ]; then
|
||||||
echo "Cloning Frametop ($branch) into $dir"
|
echo "Cloning Frametop ($branch) into $dir"
|
||||||
git clone --branch "$branch" "$repo" "$dir"
|
git clone --branch "$branch" "$repo" "$dir"
|
||||||
|
|||||||
@@ -0,0 +1,68 @@
|
|||||||
|
---
|
||||||
|
title: Install the Frametop Hand Recorder
|
||||||
|
---
|
||||||
|
|
||||||
|
# Install the Frametop Hand Recorder
|
||||||
|
|
||||||
|
The Hand Recorder records your hands with the Steam Frame's tracking cameras for Frametop's open hand dataset ([DeeJanuz/frametop-hands](https://huggingface.co/datasets/DeeJanuz/frametop-hands) on Hugging Face). These are the Konsole commands to install it.
|
||||||
|
|
||||||
|
> **Fixed in Frametop 0.2.1 (October 5, 2026):** the recorder now works on headsets without the Arcturus color passthrough module too. If you installed it earlier and the camera check stopped with "Not all of the headset's tracking cameras are running (ft-camd publishes only 2 of 4 mono cameras ...)", run the commands under [Update](#update).
|
||||||
|
|
||||||
|
Join the [Frametop Discord](https://discord.gg/W3X9f7z3Bc) for questions and help with recording.
|
||||||
|
|
||||||
|
For now the recorder runs inside Frametop's desktop, so these steps install Frametop first. A standalone recorder that runs from the SteamVR dashboard without Frametop is planned.
|
||||||
|
|
||||||
|
## Before you start
|
||||||
|
|
||||||
|
- You must be 18 or older, and for now you can't take part if you live in Illinois, Texas or Washington (USA). The [consent text](https://github.com/DeeJanuz/frametop/blob/main/hands/rec/CONSENT.md) explains what's recorded and what you agree to. The recorder shows it again before your first session.
|
||||||
|
- You need a Steam Frame on the stable SteamOS release (not the beta), an internet connection, and a keyboard (Bluetooth, or the on-screen one).
|
||||||
|
- You need a `sudo` password. If you've never set one, run `passwd` in Konsole first.
|
||||||
|
- Recordings are several gigabytes per round, and uploading one needs about the same again free while it runs. `df -h ~` shows your free space.
|
||||||
|
|
||||||
|
## Open Konsole
|
||||||
|
|
||||||
|
1. In the launcher, choose Launch a program → Desktop.
|
||||||
|
2. In the application menu, open System → Konsole.
|
||||||
|
|
||||||
|
## 1. Install Frametop
|
||||||
|
|
||||||
|
```
|
||||||
|
curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash -s -- --stable
|
||||||
|
```
|
||||||
|
|
||||||
|
This clones Frametop into `~/frametop` and runs its installer. The first run downloads 1–2 GB. The installer asks a few questions (gaze mode, the eye tracker, the Bluetooth fixes); the defaults are fine. At the end SteamVR restarts, which closes Konsole. If Frametop is already installed, this updates it.
|
||||||
|
|
||||||
|
## 2. Install the Hand Recorder
|
||||||
|
|
||||||
|
After SteamVR restarts, choose Launch a program → Desktop again, open Konsole, and run:
|
||||||
|
|
||||||
|
```
|
||||||
|
~/frametop/hands/rec/install.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
This builds the camera broker, the hand tracker and the headset panel (the first build takes a few minutes), asks for your `sudo` password once to let the camera broker read the cameras, and adds Frametop Hand Recorder to the application menu.
|
||||||
|
|
||||||
|
## 3. Record
|
||||||
|
|
||||||
|
Open Frametop Hand Recorder from the desktop's application menu. It walks you through the consent text, a short checklist, the recording, a review of what you recorded, and the upload to Hugging Face. Nothing leaves the headset until you press Upload.
|
||||||
|
|
||||||
|
## Update
|
||||||
|
|
||||||
|
```
|
||||||
|
curl -fsSL https://deejanuz.github.io/frametop/get.sh | bash -s -- --stable
|
||||||
|
~/frametop/hands/rec/install.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Run the second command after SteamVR has restarted, as in the install.
|
||||||
|
|
||||||
|
## Uninstall
|
||||||
|
|
||||||
|
```
|
||||||
|
~/frametop/hands/rec/install.sh uninstall
|
||||||
|
```
|
||||||
|
|
||||||
|
This removes the menu entry. With your `sudo` password, it also takes back the camera broker's permission to read the cameras. If you also installed Frametop's live hand tracking, the camera broker keeps that permission, because live hand tracking still uses it. Your recordings stay in `~/.local/share/frametop/hands/contrib`; delete that folder to remove them. To remove Frametop as well, follow [Uninstall](https://github.com/DeeJanuz/frametop#uninstall) in the README.
|
||||||
|
|
||||||
|
## Help
|
||||||
|
|
||||||
|
Ask in the [Frametop Discord](https://discord.gg/W3X9f7z3Bc), the [Frametop issues](https://github.com/DeeJanuz/frametop/issues), or the dataset's [discussion page](https://huggingface.co/datasets/DeeJanuz/frametop-hands/discussions). All three are public.
|
||||||
+3
-2
@@ -31,12 +31,13 @@ hands/run.sh restart # after changing a setting
|
|||||||
hands/run.sh status
|
hands/run.sh status
|
||||||
hands/run.sh log [lines]
|
hands/run.sh log [lines]
|
||||||
hands/run.sh caps # after rebuilding ft-camd (a rebuild clears its capabilities)
|
hands/run.sh caps # after rebuilding ft-camd (a rebuild clears its capabilities)
|
||||||
hands/run.sh uninstall
|
hands/run.sh uncaps # take them back, unless the Hand Recorder or the services use them
|
||||||
|
hands/run.sh uninstall # the services, then uncaps
|
||||||
```
|
```
|
||||||
|
|
||||||
Settings in `~/.config/frametop.conf` (`FT_<name>` in the environment overrides them), read when ft-camd and ft-hands start:
|
Settings in `~/.config/frametop.conf` (`FT_<name>` in the environment overrides them), read when ft-camd and ft-hands start:
|
||||||
|
|
||||||
- `HANDS_SWAP_SIDES=auto` (the default): ft-hands tells from the hands which side camera is which, and corrects ft-camd's names when they're backwards (see "Which camera is which" below). `1` forces them exchanged and `0` forces ft-camd's names; ft-hands still checks and logs a warning if the hands disagree.
|
- `HANDS_SWAP_SIDES=auto` (the default): ft-hands tells from the hands which side camera is which, and corrects ft-camd's names when they're backwards (see "Which camera is which" below). `1` forces them exchanged and `0` forces ft-camd's names; ft-hands still checks, and if the hands disagree it logs a warning and publishes the hands' answer as the truth (`sides.json`), so recordings are labelled right. The example config said `0` until 2026-10-05; `scripts/conf-migrate.sh` (run by `install.sh` and `hands/rec/install.sh`) turns that untouched line into `auto`.
|
||||||
- `HANDS_CPUS=5,6,7`: the CPUs the model threads run on (below).
|
- `HANDS_CPUS=5,6,7`: the CPUs the model threads run on (below).
|
||||||
- `HANDS_CAMERAS` (`auto`), `HANDS_BRIGHT` (`all`), `HANDS_BRIGHT_ON` (40), `HANDS_BRIGHT_OFF` (25): which cameras ft-hands tracks with, as `--cams`, `--bright`, `--bright-on` and `--bright-off` (see ft-hands). `HANDS_CAMERAS=mono` also keeps ft-camd off the colour cameras.
|
- `HANDS_CAMERAS` (`auto`), `HANDS_BRIGHT` (`all`), `HANDS_BRIGHT_ON` (40), `HANDS_BRIGHT_OFF` (25): which cameras ft-hands tracks with, as `--cams`, `--bright`, `--bright-on` and `--bright-off` (see ft-hands). `HANDS_CAMERAS=mono` also keeps ft-camd off the colour cameras.
|
||||||
- `HANDS_COLOR_LEFT` (`color_video0`), `HANDS_COLOR_CROP` (`subtract`): how the colour module's calibration maps onto its images, as `--color-left` and `--color-crop`.
|
- `HANDS_COLOR_LEFT` (`color_video0`), `HANDS_COLOR_CROP` (`subtract`): how the colour module's calibration maps onto its images, as `--color-left` and `--color-crop`.
|
||||||
|
|||||||
+42
-5
@@ -56,13 +56,19 @@ ANSI = re.compile(r"\x1b\[[0-9;]*m")
|
|||||||
STAMP = re.compile(r"^\w{3} \w{3} \d{2} \d{4} (\d{2}:\d{2}:\d{2})\.\d+ (\w+): ?(.*)$")
|
STAMP = re.compile(r"^\w{3} \w{3} \d{2} \d{4} (\d{2}:\d{2}:\d{2})\.\d+ (\w+): ?(.*)$")
|
||||||
# Lines worth reading; anything else is skipped before the regexes (the log grows by MBs a day).
|
# Lines worth reading; anything else is skipped before the regexes (the log grows by MBs a day).
|
||||||
KEYS = ("FPGA", "VCINT", "Created", "TrackingCameraInit", "Closing tracking camera", "Streaming",
|
KEYS = ("FPGA", "VCINT", "Created", "TrackingCameraInit", "Closing tracking camera", "Streaming",
|
||||||
"systemd suspend", "systemd resume", "XRService logging to", "Exiting XRService")
|
"systemd suspend", "systemd resume", "XRService logging to", "Exiting XRService", "ISP ")
|
||||||
|
# XRService's numbering of its tracking cameras (the TrackingCameraInit index): ft-hands names
|
||||||
|
# them this way too (track/main.cpp, cameras_from_xrservice_log).
|
||||||
|
NAMES = ("slam_left", "slam_right", "upper_left", "upper_right")
|
||||||
RE_PASSTHRU = re.compile(r"Passthrough connected but FPGA is (\S+) - loading VCINT")
|
RE_PASSTHRU = re.compile(r"Passthrough connected but FPGA is (\S+) - loading VCINT")
|
||||||
RE_INTERLEAVE = re.compile(r"Upper cameras FPGA interleaving support: (\d)")
|
RE_INTERLEAVE = re.compile(r"Upper cameras FPGA interleaving support: (\d)")
|
||||||
RE_TASKS = re.compile(r"Created (\d+) tasks \((\d+) tracking, (\d+) passthrough\)")
|
RE_TASKS = re.compile(r"Created (\d+) tasks \((\d+) tracking, (\d+) passthrough\)")
|
||||||
RE_INIT = re.compile(r"TrackingCameraInit: index: (\d+)\. video device: /dev/video(\d+)")
|
RE_INIT = re.compile(r"TrackingCameraInit: index: (\d+)\. video device: /dev/video(\d+)")
|
||||||
RE_STREAM = re.compile(r"Streaming resumed \(FPGA: (\S+), VC interleaving: (\w+)\)")
|
RE_STREAM = re.compile(r"Streaming resumed \(FPGA: (\S+), VC interleaving: (\w+)\)")
|
||||||
RE_STATE = re.compile(r"FPGA state check: (\S+)")
|
RE_STATE = re.compile(r"FPGA state check: (\S+)")
|
||||||
|
# Without the colour module XRService runs the side cameras through the ISP, as NV12 on other
|
||||||
|
# capture pipes (vfe0 and vfe1), and the upper pair on vfe3 and vfe4.
|
||||||
|
RE_ISP = re.compile(r"ISP (enabled|disabled) for tracking cameras")
|
||||||
|
|
||||||
|
|
||||||
class LogState:
|
class LogState:
|
||||||
@@ -85,7 +91,7 @@ class LogState:
|
|||||||
def _new_episode(self, t):
|
def _new_episode(self, t):
|
||||||
self.closed = False
|
self.closed = False
|
||||||
self.episode = {"start": t, "fpga_before": "", "vcint": "", "interleave": None, "tasks": None,
|
self.episode = {"start": t, "fpga_before": "", "vcint": "", "interleave": None, "tasks": None,
|
||||||
"inits": {}, "stream": "", "failure": "", "evidence": []}
|
"inits": {}, "stream": "", "isp": None, "failure": "", "evidence": []}
|
||||||
if self.closed_at:
|
if self.closed_at:
|
||||||
self.episode["evidence"].append(self.closed_at)
|
self.episode["evidence"].append(self.closed_at)
|
||||||
|
|
||||||
@@ -146,6 +152,12 @@ class LogState:
|
|||||||
ep["interleave"] = int(m.group(1))
|
ep["interleave"] = int(m.group(1))
|
||||||
ep["evidence"].append(short)
|
ep["evidence"].append(short)
|
||||||
return
|
return
|
||||||
|
m = RE_ISP.search(text)
|
||||||
|
if m:
|
||||||
|
ep = self._ep(t)
|
||||||
|
ep["isp"] = m.group(1) == "enabled"
|
||||||
|
ep["evidence"].append(short)
|
||||||
|
return
|
||||||
m = RE_TASKS.search(text)
|
m = RE_TASKS.search(text)
|
||||||
if m:
|
if m:
|
||||||
ep = self._ep(t)
|
ep = self._ep(t)
|
||||||
@@ -184,6 +196,12 @@ class LogState:
|
|||||||
got = tuple(self.nodes[i] for i in (2, 3) if i in self.nodes)
|
got = tuple(self.nodes[i] for i in (2, 3) if i in self.nodes)
|
||||||
return got if len(got) == 2 else UPPER_NODES
|
return got if len(got) == 2 else UPPER_NODES
|
||||||
|
|
||||||
|
def camera_map(self):
|
||||||
|
"""{calibration name: /dev/videoN's N} from the latest camera start's TrackingCameraInit
|
||||||
|
lines (the whole log's when that start has none yet)."""
|
||||||
|
inits = (self.episode or {}).get("inits") or self.nodes
|
||||||
|
return {NAMES[i]: node for i, node in sorted(inits.items()) if 0 <= i < len(NAMES)}
|
||||||
|
|
||||||
def tracking_nodes(self):
|
def tracking_nodes(self):
|
||||||
got = tuple(self.nodes[i] for i in range(TRACKING) if i in self.nodes)
|
got = tuple(self.nodes[i] for i in range(TRACKING) if i in self.nodes)
|
||||||
return got if len(got) == TRACKING else SIDE_NODES + UPPER_NODES
|
return got if len(got) == TRACKING else SIDE_NODES + UPPER_NODES
|
||||||
@@ -353,7 +371,9 @@ def check(log=None, proc=True, ring=True, ring_path=None):
|
|||||||
status, reason = "unknown", "no XRService log in %s" % LOG_DIR
|
status, reason = "unknown", "no XRService log in %s" % LOG_DIR
|
||||||
out = {"log": path, "xrservice": None, "ring": None,
|
out = {"log": path, "xrservice": None, "ring": None,
|
||||||
"episode": state.snapshot()["episode"] if state else {},
|
"episode": state.snapshot()["episode"] if state else {},
|
||||||
"failure": state.snapshot()["failure"] if state else ""}
|
"failure": state.snapshot()["failure"] if state else "",
|
||||||
|
"map": {name: {"node": n, "pipe": pipe_name(n)} for name, n in state.camera_map().items()} if state else {},
|
||||||
|
"ring_missing": []}
|
||||||
|
|
||||||
if proc:
|
if proc:
|
||||||
if pid is None:
|
if pid is None:
|
||||||
@@ -389,13 +409,30 @@ def check(log=None, proc=True, ring=True, ring_path=None):
|
|||||||
evidence.append("ft-camd (pid %d, %s): %d mono cameras: %s" % (
|
evidence.append("ft-camd (pid %d, %s): %d mono cameras: %s" % (
|
||||||
r["writer_pid"], "running" if r["alive"] else "stale ring", len(r["mono"]), names))
|
r["writer_pid"], "running" if r["alive"] else "stale ring", len(r["mono"]), names))
|
||||||
if r["alive"] and len(r["mono"]) < TRACKING and status == "ok":
|
if r["alive"] and len(r["mono"]) < TRACKING and status == "ok":
|
||||||
status, reason = "degraded", ("ft-camd publishes only %d of %d mono cameras (it started while "
|
have = {c["node"] for c in r["mono"]}
|
||||||
"they were missing: restart it)" % (len(r["mono"]), TRACKING))
|
want = state.tracking_nodes() if state else SIDE_NODES + UPPER_NODES
|
||||||
|
out["ring_missing"] = [n for n in want if n not in have]
|
||||||
|
status, reason = "degraded", ("ft-camd publishes only %d of %d mono cameras, missing %s" % (
|
||||||
|
len(r["mono"]), TRACKING, " ".join("video%d" % n for n in out["ring_missing"]) or "?"))
|
||||||
out.update(status=status, reason=reason, evidence=evidence,
|
out.update(status=status, reason=reason, evidence=evidence,
|
||||||
summary="ok" if status == "ok" else "%s: %s" % (status, reason))
|
summary="ok" if status == "ok" else "%s: %s" % (status, reason))
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def pipe_name(node):
|
||||||
|
"""The capture pipe behind /dev/videoN (msm_vfe3_video0 and so on), or ""."""
|
||||||
|
try:
|
||||||
|
with open("/sys/class/video4linux/video%d/name" % node) as f:
|
||||||
|
return f.read().strip()
|
||||||
|
except OSError:
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def is_ring_short(result):
|
||||||
|
"""XRService runs all the tracking cameras, but ft-camd doesn't publish them all."""
|
||||||
|
return bool(result) and result.get("status") == "degraded" and bool(result.get("ring_missing"))
|
||||||
|
|
||||||
|
|
||||||
def is_vcint_failure(result):
|
def is_vcint_failure(result):
|
||||||
return bool(result) and result.get("status") == "degraded" and result.get("reason") == VCINT_REASON
|
return bool(result) and result.get("status") == "degraded" and result.get("reason") == VCINT_REASON
|
||||||
|
|
||||||
|
|||||||
+25
-2
@@ -379,9 +379,15 @@ static bool resolve_block(cam_t *c)
|
|||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Through the ISP (no colour module), the near-black exposures come out all zeros,
|
||||||
|
* the same every time, so their dequeues change no buffer and get no votes. Such
|
||||||
|
* an index is left unmapped (slot -1): on_frame counts its frames as dark without
|
||||||
|
* reading them. Most of a camera's indices silent means it isn't streaming yet.
|
||||||
|
*/
|
||||||
static bool resolve_each(cam_t *c)
|
static bool resolve_each(cam_t *c)
|
||||||
{
|
{
|
||||||
int depth = c->maxindex + 1;
|
int depth = c->maxindex + 1, silent = 0;
|
||||||
bool taken[MAX_SLOTS] = { false };
|
bool taken[MAX_SLOTS] = { false };
|
||||||
|
|
||||||
for (int i = 0; i < depth; i++) {
|
for (int i = 0; i < depth; i++) {
|
||||||
@@ -396,6 +402,12 @@ static bool resolve_each(cam_t *c)
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (c->nobs[i] >= 3 && v1 <= 0.2 * c->nobs[i]) {
|
||||||
|
c->slot_of[i] = -1;
|
||||||
|
silent++;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
if (c->nobs[i] < 3 || best < 0 || taken[best] || v1 < 0.8 * c->nobs[i] || v2 > 0.3 * c->nobs[i])
|
if (c->nobs[i] < 3 || best < 0 || taken[best] || v1 < 0.8 * c->nobs[i] || v2 > 0.3 * c->nobs[i])
|
||||||
return false;
|
return false;
|
||||||
|
|
||||||
@@ -403,9 +415,14 @@ static bool resolve_each(cam_t *c)
|
|||||||
c->slot_of[i] = best;
|
c->slot_of[i] = best;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (silent * 2 > depth)
|
||||||
|
return false;
|
||||||
|
|
||||||
printf("%s: queue depth %d, buffers mapped one by one:", c->slug, depth);
|
printf("%s: queue depth %d, buffers mapped one by one:", c->slug, depth);
|
||||||
for (int i = 0; i < depth; i++)
|
for (int i = 0; i < depth; i++)
|
||||||
printf(" %d", c->slot_of[i]);
|
c->slot_of[i] < 0 ? printf(" -") : printf(" %d", c->slot_of[i]);
|
||||||
|
if (silent)
|
||||||
|
printf(" (- : %d indices whose frames are all zeros, skipped as dark)", silent);
|
||||||
printf("\n");
|
printf("\n");
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
@@ -677,6 +694,12 @@ static void on_frame(cam_t *c, int64_t index, uint32_t seq, uint64_t ts, uint64_
|
|||||||
|
|
||||||
int slot = c->slot_of[index];
|
int slot = c->slot_of[index];
|
||||||
|
|
||||||
|
if (slot < 0) { /* an index whose frames are all zeros (resolve_each): a dark one */
|
||||||
|
c->dark++;
|
||||||
|
c->rc->dropped++;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
/* color runs at 60 fps: skip frames early enough that the asked rate holds, before any sync */
|
/* color runs at 60 fps: skip frames early enough that the asked rate holds, before any sync */
|
||||||
if (c->color && color_fps > 0 && evtime - c->last_pub_ns < (uint64_t)(1e9 / color_fps) - 3000000) {
|
if (c->color && color_fps > 0 && evtime - c->last_pub_ns < (uint64_t)(1e9 / color_fps) - 3000000) {
|
||||||
c->paced++;
|
c->paced++;
|
||||||
|
|||||||
+24
-7
@@ -544,22 +544,26 @@ static void probe_cameras(xr_state_t *st)
|
|||||||
/*
|
/*
|
||||||
* qcom-camss can report bytesperline as the visible width while the VFE
|
* qcom-camss can report bytesperline as the visible width while the VFE
|
||||||
* writes a larger aligned pitch. sizeimage is right, so derive the pitch.
|
* writes a larger aligned pitch. sizeimage is right, so derive the pitch.
|
||||||
|
* For NV12, plane 0 normally holds the chroma rows after the luma (the side
|
||||||
|
* cameras through the ISP: 1056 wide, 1152 bytes a row); if it's too small for
|
||||||
|
* that, it holds the luma alone.
|
||||||
*/
|
*/
|
||||||
unsigned xr_camera_stride(const xr_camera_t *c)
|
unsigned xr_camera_stride(const xr_camera_t *c)
|
||||||
{
|
{
|
||||||
if (!c->height || !c->planesize[0])
|
if (!c->height || !c->planesize[0])
|
||||||
return c->bytesperline ? c->bytesperline : c->width;
|
return c->bytesperline ? c->bytesperline : c->width;
|
||||||
|
|
||||||
double bpp = 1.0;
|
bool yuv = c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21;
|
||||||
|
unsigned s = (unsigned)((double)c->planesize[0] / ((double)c->height * (yuv ? 1.5 : 1.0)));
|
||||||
if (c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21)
|
|
||||||
bpp = 1.5;
|
|
||||||
|
|
||||||
unsigned s = (unsigned)((double)c->planesize[0] / ((double)c->height * bpp));
|
|
||||||
|
|
||||||
if (s >= c->width && s <= c->width * 4)
|
if (s >= c->width && s <= c->width * 4)
|
||||||
return s;
|
return s;
|
||||||
|
|
||||||
|
s = (unsigned)(c->planesize[0] / c->height);
|
||||||
|
|
||||||
|
if (yuv && s >= c->width && s <= c->width * 4)
|
||||||
|
return s;
|
||||||
|
|
||||||
return c->bytesperline ? c->bytesperline : c->width;
|
return c->bytesperline ? c->bytesperline : c->width;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -591,7 +595,20 @@ void xr_camera_layout(const xr_camera_t *c, xr_layout_t *l)
|
|||||||
l->pitch = xr_camera_stride(c);
|
l->pitch = xr_camera_stride(c);
|
||||||
l->width = c->width < l->pitch ? c->width : l->pitch;
|
l->width = c->width < l->pitch ? c->width : l->pitch;
|
||||||
|
|
||||||
if (c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21) {
|
/*
|
||||||
|
* Without the colour module, XRService runs the side cameras through the
|
||||||
|
* ISP ("ISP enabled for tracking cameras (main VFE available)" in its log),
|
||||||
|
* and they come out NV12. They're mono sensors, so the luma is the image.
|
||||||
|
*/
|
||||||
|
bool yuv = c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21;
|
||||||
|
|
||||||
|
if (yuv && c->role && !strcmp(c->role, "tracking")) {
|
||||||
|
l->fmt = XR_FMT_GREY8;
|
||||||
|
l->rows = c->height;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (yuv) {
|
||||||
l->fmt = XR_FMT_NV12;
|
l->fmt = XR_FMT_NV12;
|
||||||
l->rows = c->height + c->height / 2;
|
l->rows = c->height + c->height / 2;
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
+1
-1
@@ -29,7 +29,7 @@ Every part runs in the dev container, as ft-hands and Input Settings do. The hos
|
|||||||
- **A tracking ft-hands** gives feedback through the hands file: which hands are seen, the palm's distance, the index tip. If none is running, the session starts `ft-hands --no-gestures --status 0` (unit `frametop-handrec-hands.service`). An ft-hands already running is used as it is.
|
- **A tracking ft-hands** gives feedback through the hands file: which hands are seen, the palm's distance, the index tip. If none is running, the session starts `ft-hands --no-gestures --status 0` (unit `frametop-handrec-hands.service`). An ft-hands already running is used as it is.
|
||||||
- **A recording ft-hands** runs once per recording part: `ft-hands --record-only --record DIR --record-for SECONDS --record-hz 10 --status 0 --sides auto|0|1` (below, "Side cameras"). It runs as a plain child process of the session, ended with SIGTERM when the part ends. SIGTERM ends ft-hands' loop, and `Recorder` writes out its queue when it's destroyed. In step mode (below) a part is one step's countdown and hold, so a take has one part per step (about 40 in the hand poses); in auto mode a take is one part, plus one more after each pause. `--record-for` is only a safety net.
|
- **A recording ft-hands** runs once per recording part: `ft-hands --record-only --record DIR --record-for SECONDS --record-hz 10 --status 0 --sides auto|0|1` (below, "Side cameras"). It runs as a plain child process of the session, ended with SIGTERM when the part ends. SIGTERM ends ft-hands' loop, and `Recorder` writes out its queue when it's destroyed. In step mode (below) a part is one step's countdown and hold, so a take has one part per step (about 40 in the hand poses); in auto mode a take is one part, plus one more after each pause. `--record-for` is only a safety net.
|
||||||
- Why a process per part rather than one kept alive and paused: measured in the dev container with `ft-ringplay`'s ring (2026-10-02), `ft-hands --record-only` writes its first set 16-27 ms after it starts and ends 4-6 ms after SIGTERM, so a new part costs nothing the 3 s countdown doesn't cover. Every reader already takes parts in order (`takes.py`, `validate.py` through the export's single stream, the labeller's `fhl_io.py`, numbering `sets-10.bin` after `sets-9.bin`), ft-hands needs no new control, and nothing is written while a step waits. Before the hold starts the session checks that the part has written a set (`Recorder.has_data`, up to 3 s more), so the hold is recorded from its first frame.
|
- Why a process per part rather than one kept alive and paused: measured in the dev container with `ft-ringplay`'s ring (2026-10-02), `ft-hands --record-only` writes its first set 16-27 ms after it starts and ends 4-6 ms after SIGTERM, so a new part costs nothing the 3 s countdown doesn't cover. Every reader already takes parts in order (`takes.py`, `validate.py` through the export's single stream, the labeller's `fhl_io.py`, numbering `sets-10.bin` after `sets-9.bin`), ft-hands needs no new control, and nothing is written while a step waits. Before the hold starts the session checks that the part has written a set (`Recorder.has_data`, up to 3 s more), so the hold is recorded from its first frame.
|
||||||
- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand.
|
- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. With `HANDS_SWAP_SIDES` forced to `0` or `1`, the hands still decide what's published once they disagree (state `"forced, disagrees"`; `read_live` also corrects an ft-hands built before that). The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand.
|
||||||
- **ft-handpanel** runs as a child process with `--watch-stdin`. It shows the panel and logs poses during each take.
|
- **ft-handpanel** runs as a child process with `--watch-stdin`. It shows the panel and logs poses during each take.
|
||||||
- **The headset button's reader** is a thread of the session (`ButtonReader`, below), not a process.
|
- **The headset button's reader** is a thread of the session (`ButtonReader`, below), not a process.
|
||||||
|
|
||||||
|
|||||||
@@ -457,11 +457,12 @@ class Backend(QObject):
|
|||||||
"""Start stays off: the check found the cameras degraded (unless --ignore-cameras)."""
|
"""Start stays off: the check found the cameras degraded (unless --ignore-cameras)."""
|
||||||
return not self._ignore_cameras and self._camera.get("status") == "degraded"
|
return not self._ignore_cameras and self._camera.get("status") == "degraded"
|
||||||
|
|
||||||
def _check_now(self):
|
def _check_now(self, repair=False):
|
||||||
|
"""repair (off this thread only: it may restart ft-camd, up to 15 s): see session.camera_check."""
|
||||||
mod = self._runner()
|
mod = self._runner()
|
||||||
if not mod or self._session_options.get("dry_run"):
|
if not mod or self._session_options.get("dry_run"):
|
||||||
return {"status": "unknown", "summary": "not checked (dry run)", "reason": "dry run", "evidence": []}
|
return {"status": "unknown", "summary": "not checked (dry run)", "reason": "dry run", "evidence": []}
|
||||||
return mod.camera_check()
|
return mod.camera_check(repair=repair)
|
||||||
|
|
||||||
@Slot()
|
@Slot()
|
||||||
def checkCameras(self):
|
def checkCameras(self):
|
||||||
@@ -470,7 +471,7 @@ class Backend(QObject):
|
|||||||
return
|
return
|
||||||
self._camera_busy = True
|
self._camera_busy = True
|
||||||
self.cameraChanged.emit()
|
self.cameraChanged.emit()
|
||||||
self._thread(lambda: self._cameraArrived.emit(self._check_now()))
|
self._thread(lambda: self._cameraArrived.emit(self._check_now(repair=True)))
|
||||||
|
|
||||||
def _on_camera(self, result):
|
def _on_camera(self, result):
|
||||||
self._camera = result
|
self._camera = result
|
||||||
@@ -499,7 +500,7 @@ class Backend(QObject):
|
|||||||
end = time.monotonic() + RECHECK_FOR_S
|
end = time.monotonic() + RECHECK_FOR_S
|
||||||
time.sleep(RECHECK_S * 2)
|
time.sleep(RECHECK_S * 2)
|
||||||
while time.monotonic() < end:
|
while time.monotonic() < end:
|
||||||
result = self._check_now()
|
result = self._check_now(repair=True)
|
||||||
# Done once a new XRService (a new log) has opened its cameras, ok or not.
|
# Done once a new XRService (a new log) has opened its cameras, ok or not.
|
||||||
if result.get("status") in ("ok", "degraded") and result.get("log") != before:
|
if result.get("status") in ("ok", "degraded") and result.get("log") != before:
|
||||||
break
|
break
|
||||||
|
|||||||
@@ -7,7 +7,9 @@
|
|||||||
# "Launch a program", which runs apps outside it; the standalone recorder is for that).
|
# "Launch a program", which runs apps outside it; the standalone recorder is for that).
|
||||||
# It doesn't turn on Frametop's live hand tracking (that's hands/run.sh install, still deferred).
|
# It doesn't turn on Frametop's live hand tracking (that's hands/run.sh install, still deferred).
|
||||||
# Usage: hands/rec/install.sh install or update
|
# Usage: hands/rec/install.sh install or update
|
||||||
# hands/rec/install.sh uninstall remove the menu entry (recordings stay where they are)
|
# hands/rec/install.sh uninstall remove the menu entry, and ft-camd's capabilities unless
|
||||||
|
# hand tracking's services use them (hands/run.sh uncaps).
|
||||||
|
# Recordings stay where they are.
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
|
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
|
||||||
. "$root/scripts/_env.sh"
|
. "$root/scripts/_env.sh"
|
||||||
@@ -23,6 +25,7 @@ case ${1:-install} in
|
|||||||
"$root/hands/rec/build.sh"
|
"$root/hands/rec/build.sh"
|
||||||
echo "== 4/4 ft-camd's capabilities (asks for your password) and the menu entry"
|
echo "== 4/4 ft-camd's capabilities (asks for your password) and the menu entry"
|
||||||
"$root/hands/run.sh" caps
|
"$root/hands/run.sh" caps
|
||||||
|
"$root/scripts/conf-migrate.sh" # HANDS_SWAP_SIDES=0, the old default, becomes auto
|
||||||
fill_template "$root/hands/rec/ft-handrec.desktop" |
|
fill_template "$root/hands/rec/ft-handrec.desktop" |
|
||||||
on_frame "mkdir -p ~/.local/share/applications && cat > $entry"
|
on_frame "mkdir -p ~/.local/share/applications && cat > $entry"
|
||||||
echo
|
echo
|
||||||
@@ -31,7 +34,9 @@ case ${1:-install} in
|
|||||||
;;
|
;;
|
||||||
uninstall)
|
uninstall)
|
||||||
on_frame "rm -f $entry"
|
on_frame "rm -f $entry"
|
||||||
echo "Removed the menu entry. Your recordings are still in ~/.local/share/frametop/hands/contrib:"
|
echo "Removed the menu entry."
|
||||||
|
"$root/hands/run.sh" uncaps # after the entry, which counts as a user of the capabilities
|
||||||
|
echo "Your recordings are still in ~/.local/share/frametop/hands/contrib:"
|
||||||
echo "delete that folder to remove them."
|
echo "delete that folder to remove them."
|
||||||
;;
|
;;
|
||||||
*) echo "usage: $0 [install|uninstall]" >&2; exit 2 ;;
|
*) echo "usage: $0 [install|uninstall]" >&2; exit 2 ;;
|
||||||
|
|||||||
+37
-4
@@ -1153,11 +1153,33 @@ def _steamvr_version():
|
|||||||
return ""
|
return ""
|
||||||
|
|
||||||
|
|
||||||
def camera_check():
|
_camd_restarted = set() # ft-camd pids camera_check(repair=True) has restarted: once each
|
||||||
"""camcheck.check() (the XRService log, its open cameras where readable, ft-camd's ring);
|
|
||||||
never raises: a failure is "unknown"."""
|
|
||||||
|
def _unit_pid(unit):
|
||||||
|
r = subprocess.run(host_command("systemctl", "--user", "show", "-p", "MainPID", "--value", unit),
|
||||||
|
capture_output=True, text=True, timeout=30)
|
||||||
try:
|
try:
|
||||||
return camcheck.check()
|
return int(r.stdout.strip() or 0)
|
||||||
|
except ValueError:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def camera_check(repair=False):
|
||||||
|
"""camcheck.check() (the XRService log, its open cameras where readable, ft-camd's ring);
|
||||||
|
never raises: a failure is "unknown". repair (the window, never during a session): when
|
||||||
|
XRService runs every tracking camera but the recorder's own ft-camd doesn't publish them all
|
||||||
|
(it started while some were missing), restart it, once per ft-camd, and check again."""
|
||||||
|
try:
|
||||||
|
r = camcheck.check()
|
||||||
|
pid = (r.get("ring") or {}).get("writer_pid")
|
||||||
|
if repair and camcheck.is_ring_short(r) and pid not in _camd_restarted and _unit_pid(CAMD_UNIT) == pid:
|
||||||
|
_camd_restarted.add(pid)
|
||||||
|
stop_unit(CAMD_UNIT)
|
||||||
|
start_camd()
|
||||||
|
r = camcheck.check()
|
||||||
|
r["evidence"].append("restarted ft-camd (pid %d) because it published only some of the cameras" % pid)
|
||||||
|
return r
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
return {"status": "unknown", "summary": "unknown: the camera check failed (%s)" % e,
|
return {"status": "unknown", "summary": "unknown: the camera check failed (%s)" % e,
|
||||||
"reason": str(e), "evidence": []}
|
"reason": str(e), "evidence": []}
|
||||||
@@ -1169,6 +1191,10 @@ def camera_text(result):
|
|||||||
return ""
|
return ""
|
||||||
if camcheck.is_vcint_failure(result):
|
if camcheck.is_vcint_failure(result):
|
||||||
return camcheck.USER_TEXT
|
return camcheck.USER_TEXT
|
||||||
|
if camcheck.is_ring_short(result):
|
||||||
|
return ("The headset's tracking cameras are all running, but the recorder can't read some of them (%s). "
|
||||||
|
"Close the Hand Recorder, run ~/frametop/hands/rec/install.sh again, and open it again. If that "
|
||||||
|
"doesn't help, ask in the Frametop Discord." % result.get("reason", ""))
|
||||||
return ("Not all of the headset's tracking cameras are running (%s). Restart SteamVR, or restart the "
|
return ("Not all of the headset's tracking cameras are running (%s). Restart SteamVR, or restart the "
|
||||||
"headset if that doesn't fix it." % result.get("reason", ""))
|
"headset if that doesn't fix it." % result.get("reason", ""))
|
||||||
|
|
||||||
@@ -1451,6 +1477,10 @@ class Session:
|
|||||||
cam = self._status.get("camera")
|
cam = self._status.get("camera")
|
||||||
if cam:
|
if cam:
|
||||||
self._session_json["camera"] = {"status": cam.get("status"), "reason": cam.get("reason", "")}
|
self._session_json["camera"] = {"status": cam.get("status"), "reason": cam.get("reason", "")}
|
||||||
|
# which device each calibrated camera was (XRService's numbering) and whether XRService
|
||||||
|
# ran the side cameras through the ISP (no colour module): to check the names later
|
||||||
|
self._session_json["device"]["camera_map"] = cam.get("map") or {}
|
||||||
|
self._session_json["device"]["isp"] = (cam.get("episode") or {}).get("isp")
|
||||||
if self.dry_run:
|
if self.dry_run:
|
||||||
self._session_json["dry_run"] = True
|
self._session_json["dry_run"] = True
|
||||||
if self.speed != 1:
|
if self.speed != 1:
|
||||||
@@ -1545,6 +1575,9 @@ class Session:
|
|||||||
self._save_session()
|
self._save_session()
|
||||||
self._log("side cameras: %s (%s, %s)" % ("SWAPPED" if swapped else "as named", new["decided_by"],
|
self._log("side cameras: %s (%s, %s)" % ("SWAPPED" if swapped else "as named", new["decided_by"],
|
||||||
json.dumps(new["evidence"])))
|
json.dumps(new["evidence"])))
|
||||||
|
if new["state"] == "forced, disagrees":
|
||||||
|
self._log("side cameras: HANDS_SWAP_SIDES in ~/.config/frametop.conf forces names the hands say are "
|
||||||
|
"backwards; the recording goes by the hands. Set HANDS_SWAP_SIDES=auto.")
|
||||||
|
|
||||||
def _sides_swapped(self):
|
def _sides_swapped(self):
|
||||||
"""session.json's decision: True, False, or None (not known yet)."""
|
"""session.json's decision: True, False, or None (not known yet)."""
|
||||||
|
|||||||
@@ -125,4 +125,9 @@ def read_live(path=None, ring_path=None, now_ns=None):
|
|||||||
return None
|
return None
|
||||||
except OSError:
|
except OSError:
|
||||||
return None
|
return None
|
||||||
|
if (s.get("state") == "forced, disagrees" and s.get("swapped") is not None
|
||||||
|
and bool(s["swapped"]) == bool(s.get("names_swapped"))):
|
||||||
|
# An ft-hands built before 2026-10-06 kept a forced HANDS_SWAP_SIDES as the truth even
|
||||||
|
# when the hands disagreed. The hands are right: the truth is the other way round.
|
||||||
|
s = dict(s, swapped=not s["swapped"], decided_by="auto")
|
||||||
return s
|
return s
|
||||||
@@ -91,6 +91,29 @@ class RulesTest(unittest.TestCase):
|
|||||||
finally:
|
finally:
|
||||||
shutil.rmtree(d)
|
shutil.rmtree(d)
|
||||||
|
|
||||||
|
def test_read_live_forced(self):
|
||||||
|
"""HANDS_SWAP_SIDES=0 forced and the hands disagree: the hands are the truth, whether
|
||||||
|
ft-hands published them (built after 2026-10-06) or kept the forced value (before)."""
|
||||||
|
d = tempfile.mkdtemp()
|
||||||
|
try:
|
||||||
|
path = os.path.join(d, "sides.json")
|
||||||
|
|
||||||
|
def live(**kw):
|
||||||
|
s = dict(pid=1, ring_ino=0, mode="0", names_swapped=False,
|
||||||
|
updated_ns=time.clock_gettime_ns(time.CLOCK_MONOTONIC), **kw)
|
||||||
|
with open(path, "w") as f:
|
||||||
|
json.dump(s, f)
|
||||||
|
return sides.read_live(path)
|
||||||
|
|
||||||
|
self.assertEqual(live(state="forced", swapped=False, decided_by="config")["swapped"], False)
|
||||||
|
self.assertEqual(live(state="forced, agrees", swapped=False, decided_by="config")["swapped"], False)
|
||||||
|
old = live(state="forced, disagrees", swapped=False, decided_by="config")
|
||||||
|
self.assertEqual((old["swapped"], old["decided_by"]), (True, "auto"))
|
||||||
|
new = live(state="forced, disagrees", swapped=True, decided_by="auto")
|
||||||
|
self.assertEqual((new["swapped"], new["decided_by"]), (True, "auto"))
|
||||||
|
finally:
|
||||||
|
shutil.rmtree(d)
|
||||||
|
|
||||||
|
|
||||||
class TakesTest(unittest.TestCase):
|
class TakesTest(unittest.TestCase):
|
||||||
"""A swapped session: one take's parts recorded before the decision (ft-camd's names) and
|
"""A swapped session: one take's parts recorded before the decision (ft-camd's names) and
|
||||||
@@ -226,6 +249,22 @@ class SessionSidesTest(unittest.TestCase):
|
|||||||
self.assertTrue(s._session_json["sides"]["reversed_from"]["swapped"])
|
self.assertTrue(s._session_json["sides"]["reversed_from"]["swapped"])
|
||||||
self.assertFalse(takes.read_json(os.path.join(self.tmp, "session.json"))["sides"]["swapped"])
|
self.assertFalse(takes.read_json(os.path.join(self.tmp, "session.json"))["sides"]["swapped"])
|
||||||
|
|
||||||
|
def test_read_sides_forced(self):
|
||||||
|
"""PR #4 on the dataset: HANDS_SWAP_SIDES=0 from the old example config, and the hands
|
||||||
|
disagree. The session takes the hands' answer, not the forced one."""
|
||||||
|
s = session.Session(os.path.join(self.tmp, "base"), {}, {}, "room", self.script, dry_run=True,
|
||||||
|
hands_dir=self.tmp)
|
||||||
|
s.session_dir = self.tmp
|
||||||
|
s._session_json = {"sides": {"swapped": None}}
|
||||||
|
self.live(mode="0", state="forced", swapped=False, decided_by="config", names_swapped=False, evidence=None)
|
||||||
|
s._read_sides(force=True)
|
||||||
|
self.assertIs(s._sides_swapped(), False)
|
||||||
|
self.live(mode="0", state="forced, disagrees", swapped=False, decided_by="config", names_swapped=False)
|
||||||
|
s._read_sides(force=True)
|
||||||
|
self.assertIs(s._sides_swapped(), True)
|
||||||
|
self.assertEqual(s._session_json["sides"]["decided_by"], "auto")
|
||||||
|
self.assertEqual(s._session_json["sides"]["reversed_from"]["decided_by"], "config")
|
||||||
|
|
||||||
def test_recorder_parts(self):
|
def test_recorder_parts(self):
|
||||||
calls = []
|
calls = []
|
||||||
|
|
||||||
|
|||||||
+30
-6
@@ -4,10 +4,11 @@
|
|||||||
# ft-handsctl on|off (on the Frame) or hands/run.sh start|stop.
|
# ft-handsctl on|off (on the Frame) or hands/run.sh start|stop.
|
||||||
# Usage: hands/run.sh install|uninstall
|
# Usage: hands/run.sh install|uninstall
|
||||||
# hands/run.sh caps # give ft-camd its capabilities again (a rebuild clears them)
|
# hands/run.sh caps # give ft-camd its capabilities again (a rebuild clears them)
|
||||||
|
# hands/run.sh uncaps # take them back, unless the Hand Recorder or the services use them
|
||||||
# hands/run.sh start|stop|restart|status|log [lines]
|
# hands/run.sh start|stop|restart|status|log [lines]
|
||||||
# install and caps need the password (sudo setcap, once per build of ft-camd): it's asked in
|
# install and caps need the password (sudo setcap, once per build of ft-camd), and so do
|
||||||
# the terminal, on the Frame or from a PC (frame_sudo in scripts/_env.sh, which also takes it
|
# uninstall and uncaps when they take the capabilities back. It's asked in the terminal, on the
|
||||||
# from the repo's .env).
|
# Frame or from a PC (frame_sudo in scripts/_env.sh, which also takes it from the repo's .env).
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
|
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
|
||||||
. "$root/scripts/_env.sh"
|
. "$root/scripts/_env.sh"
|
||||||
@@ -16,6 +17,10 @@ units="frametop-camd.service frametop-hands.service"
|
|||||||
# pidfd_getfd on XRService (ptrace_scope=1), system-wide tracepoints, and their root-only
|
# pidfd_getfd on XRService (ptrace_scope=1), system-wide tracepoints, and their root-only
|
||||||
# format files. ft-camd drops them all once it has set up.
|
# format files. ft-camd drops them all once it has set up.
|
||||||
caps=cap_sys_ptrace,cap_perfmon,cap_dac_read_search+ep
|
caps=cap_sys_ptrace,cap_perfmon,cap_dac_read_search+ep
|
||||||
|
# Installed, two things use ft-camd's capabilities: the Hand Recorder (hands/rec/install.sh: its
|
||||||
|
# menu entry) and the services. ft-cutouts uses them too, but installs nothing.
|
||||||
|
handrec_entry='~/.local/share/applications/frametop-handrec.desktop'
|
||||||
|
camd_unit='~/.config/systemd/user/frametop-camd.service'
|
||||||
|
|
||||||
sudo_run() { frame_sudo "$1"; }
|
sudo_run() { frame_sudo "$1"; }
|
||||||
|
|
||||||
@@ -29,6 +34,22 @@ set_caps() { # only when missing: a rebuild clears them, a reinstall doesn't
|
|||||||
sudo_run "setcap $caps $bin && getcap $bin"
|
sudo_run "setcap $caps $bin && getcap $bin"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# A left-over binary shouldn't keep its read-any-file powers, but while the Hand Recorder or
|
||||||
|
# the services are installed, they still need them. Best effort: without the password it warns.
|
||||||
|
drop_caps() {
|
||||||
|
local bin who
|
||||||
|
bin=$(printf %q "$FRAME_REPO/hands/build/ft-camd")
|
||||||
|
who=$(on_frame "if ! { [ -x $bin ] && getcap $bin | grep -q cap_sys_ptrace; }; then echo none
|
||||||
|
elif [ -e $handrec_entry ]; then echo recorder
|
||||||
|
elif [ -e $camd_unit ]; then echo services
|
||||||
|
else echo nobody; fi") || who=none
|
||||||
|
case $who in
|
||||||
|
recorder) echo "kept ft-camd's capabilities: the Hand Recorder uses them" ;;
|
||||||
|
services) echo "kept ft-camd's capabilities: hand tracking's services use them (hands/run.sh uninstall)" ;;
|
||||||
|
nobody) sudo_run "setcap -r $bin" || echo "warning: couldn't drop ft-camd's capabilities (no password?)" >&2 ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
states="for u in $units; do echo \"\$u: \$(systemctl --user is-active \$u)\"; done"
|
states="for u in $units; do echo \"\$u: \$(systemctl --user is-active \$u)\"; done"
|
||||||
|
|
||||||
case ${1:-status} in
|
case ${1:-status} in
|
||||||
@@ -43,11 +64,14 @@ case ${1:-status} in
|
|||||||
mkdir -p ~/.local/bin && ln -sfn $(printf %q "$FRAME_REPO/hands/ft-handsctl") ~/.local/bin/ft-handsctl
|
mkdir -p ~/.local/bin && ln -sfn $(printf %q "$FRAME_REPO/hands/ft-handsctl") ~/.local/bin/ft-handsctl
|
||||||
$states; echo 'start it with: ft-handsctl on'" ;;
|
$states; echo 'start it with: ft-handsctl on'" ;;
|
||||||
caps) set_caps ;;
|
caps) set_caps ;;
|
||||||
uninstall) "$frame" --host "systemctl --user disable --now $units 2>/dev/null
|
uncaps) drop_caps ;;
|
||||||
|
uninstall)
|
||||||
|
"$frame" --host "systemctl --user disable --now $units 2>/dev/null
|
||||||
for u in $units; do rm -f ~/.config/systemd/user/\$u; done; systemctl --user daemon-reload
|
for u in $units; do rm -f ~/.config/systemd/user/\$u; done; systemctl --user daemon-reload
|
||||||
[ -L ~/.local/bin/ft-handsctl ] && rm -f ~/.local/bin/ft-handsctl; echo removed" ;;
|
[ -L ~/.local/bin/ft-handsctl ] && rm -f ~/.local/bin/ft-handsctl; echo removed"
|
||||||
|
drop_caps ;; # once the units are gone, so they don't count as a user
|
||||||
start|stop|restart) "$frame" --host "systemctl --user $1 $units; $states" ;;
|
start|stop|restart) "$frame" --host "systemctl --user $1 $units; $states" ;;
|
||||||
status) "$frame" --host "$states; journalctl --user -u frametop-hands.service --no-pager -o cat -n 4" || true ;;
|
status) "$frame" --host "$states; journalctl --user -u frametop-hands.service --no-pager -o cat -n 4" || true ;;
|
||||||
log) "$frame" --host "journalctl --user -u frametop-camd.service -u frametop-hands.service --no-pager -o short -n ${2:-30}" ;;
|
log) "$frame" --host "journalctl --user -u frametop-camd.service -u frametop-hands.service --no-pager -o short -n ${2:-30}" ;;
|
||||||
*) echo "usage: $0 install|uninstall|caps|start|stop|restart|status|log [lines]" >&2; exit 2 ;;
|
*) echo "usage: $0 install|uninstall|caps|uncaps|start|stop|restart|status|log [lines]" >&2; exit 2 ;;
|
||||||
esac
|
esac
|
||||||
@@ -66,6 +66,22 @@ RESUME_OK = [L("12:30:00", "[SystemdInhibitor] Received systemd resume notificat
|
|||||||
for i, n in enumerate((9, 13, 6, 7))] + \
|
for i, n in enumerate((9, 13, 6, 7))] + \
|
||||||
[L("12:30:02", "[DeckardCaptureSource] Streaming resumed (FPGA: VCINT, VC interleaving: enabled)")]
|
[L("12:30:02", "[DeckardCaptureSource] Streaming resumed (FPGA: VCINT, VC interleaving: enabled)")]
|
||||||
EXIT = [L("13:00:00", "XRService - main thread exiting"), L("13:00:00", "Exiting XRService")]
|
EXIT = [L("13:00:00", "XRService - main thread exiting"), L("13:00:00", "Exiting XRService")]
|
||||||
|
# The colour module unplugged while SteamVR runs (2026-10-05 12:42): XRService reopens the
|
||||||
|
# cameras with the side pair through the ISP on vfe0 and vfe1 (NV12); VCINT stays loaded, so the
|
||||||
|
# upper pair stays on vfe2.
|
||||||
|
UNPLUG = [L("12:42:25", "Received passthrough camera connection event (connected=0)"),
|
||||||
|
L("12:42:25", "[DeckardCaptureSource] Closing tracking camera interfaces camerasToUse: 1111"),
|
||||||
|
L("12:42:26", "FPGA state check: VCINT (register value: 0x00021211)"),
|
||||||
|
L("12:42:26", "Upper cameras FPGA interleaving support: 1 (Driver features available = 1 | VCINT loaded = 1)"),
|
||||||
|
L("12:42:26", "[buildMediaCtlSetupTasks] ISP enabled for tracking cameras (main VFE available)"),
|
||||||
|
L("12:42:26", "[buildMediaCtlSetupTasks] Created 4 tasks (4 tracking, 0 passthrough)")] + \
|
||||||
|
[L("12:42:26", "TrackingCameraInit: index: %d. video device: /dev/video%d. v4l subdevice: x" % (i, n))
|
||||||
|
for i, n in enumerate((0, 3, 6, 7))]
|
||||||
|
# Started without the module (FrameEyeCameraFeed's layout): the upper pair on vfe3 and vfe4.
|
||||||
|
NO_MODULE = [L("10:00:02", "[buildMediaCtlSetupTasks] ISP enabled for tracking cameras (main VFE available)"),
|
||||||
|
L("10:00:02", "[buildMediaCtlSetupTasks] Created 4 tasks (4 tracking, 0 passthrough)")] + \
|
||||||
|
[L("10:00:03", "TrackingCameraInit: index: %d. video device: /dev/video%d. v4l subdevice: x" % (i, n))
|
||||||
|
for i, n in enumerate((0, 3, 9, 13))]
|
||||||
|
|
||||||
|
|
||||||
def state(lines):
|
def state(lines):
|
||||||
@@ -128,6 +144,23 @@ class SyntheticLogs(unittest.TestCase):
|
|||||||
self.assertEqual(status, "degraded")
|
self.assertEqual(status, "degraded")
|
||||||
self.assertEqual(reason, "only 2 of 4 tracking cameras running")
|
self.assertEqual(reason, "only 2 of 4 tracking cameras running")
|
||||||
|
|
||||||
|
def test_camera_map_with_module(self):
|
||||||
|
st = state(START + GOOD_OPEN)
|
||||||
|
self.assertEqual(st.camera_map(), {"slam_left": 9, "slam_right": 13, "upper_left": 6, "upper_right": 7})
|
||||||
|
|
||||||
|
def test_module_unplugged(self):
|
||||||
|
st = state(START + GOOD_OPEN + UNPLUG)
|
||||||
|
self.assertEqual(st.verdict()[0], "ok")
|
||||||
|
self.assertIs(st.episode["isp"], True)
|
||||||
|
self.assertEqual(st.camera_map(), {"slam_left": 0, "slam_right": 3, "upper_left": 6, "upper_right": 7})
|
||||||
|
self.assertEqual(st.tracking_nodes(), (0, 3, 6, 7))
|
||||||
|
|
||||||
|
def test_started_without_module(self):
|
||||||
|
st = state(START + NO_MODULE)
|
||||||
|
self.assertEqual(st.verdict()[0], "ok")
|
||||||
|
self.assertEqual(st.camera_map(), {"slam_left": 0, "slam_right": 3, "upper_left": 9, "upper_right": 13})
|
||||||
|
self.assertEqual(st.upper_nodes(), (9, 13))
|
||||||
|
|
||||||
def test_exited_and_empty(self):
|
def test_exited_and_empty(self):
|
||||||
self.assertEqual(state(START + GOOD_OPEN + EXIT).verdict()[0], "unknown")
|
self.assertEqual(state(START + GOOD_OPEN + EXIT).verdict()[0], "unknown")
|
||||||
self.assertEqual(state([]).verdict()[0], "unknown")
|
self.assertEqual(state([]).verdict()[0], "unknown")
|
||||||
@@ -223,9 +256,9 @@ class RealLog(unittest.TestCase):
|
|||||||
self.assertEqual(out.getvalue().split("\n")[0], "ok")
|
self.assertEqual(out.getvalue().split("\n")[0], "ok")
|
||||||
|
|
||||||
|
|
||||||
def make_ring(path, mono_names, alive=True):
|
def make_ring(path, mono_names, alive=True, nodes=None):
|
||||||
"""A ring header as ft-camd writes it (camd/fhring.h), no frames."""
|
"""A ring header as ft-camd writes it (camd/fhring.h), no frames."""
|
||||||
cams = [(b"og01a1b", n.encode(), 9 + i) for i, n in enumerate(mono_names)]
|
cams = [(b"og01a1b", n.encode(), nodes[i] if nodes else 9 + i) for i, n in enumerate(mono_names)]
|
||||||
hb = time.clock_gettime_ns(time.CLOCK_MONOTONIC) if alive else 1
|
hb = time.clock_gettime_ns(time.CLOCK_MONOTONIC) if alive else 1
|
||||||
data = bytearray(camcheck.RING_HDR.pack(b"FHRING01", 1, 0, len(cams), 0, 0, 4242, 0))
|
data = bytearray(camcheck.RING_HDR.pack(b"FHRING01", 1, 0, len(cams), 0, 0, 4242, 0))
|
||||||
struct.pack_into("<Q", data, 40, hb)
|
struct.pack_into("<Q", data, 40, hb)
|
||||||
@@ -252,6 +285,19 @@ class Ring(unittest.TestCase):
|
|||||||
self.assertEqual(r["status"], "degraded")
|
self.assertEqual(r["status"], "degraded")
|
||||||
self.assertIn("ft-camd publishes only 2 of 4", r["reason"])
|
self.assertIn("ft-camd publishes only 2 of 4", r["reason"])
|
||||||
|
|
||||||
|
def test_ring_missing_the_side_cameras(self):
|
||||||
|
# an ft-camd from before NV12 support, without the colour module: only the upper pair
|
||||||
|
with open(self.log, "w") as f:
|
||||||
|
f.write("\n".join(START + NO_MODULE) + "\n")
|
||||||
|
make_ring(self.ring, ["og0ve10_5-003e_video9", "og0ve10_5-0060_video13"], nodes=[9, 13])
|
||||||
|
r = camcheck.check(log=self.log, proc=False, ring_path=self.ring)
|
||||||
|
self.assertEqual(r["status"], "degraded")
|
||||||
|
self.assertEqual(r["ring_missing"], [0, 3])
|
||||||
|
self.assertIn("missing video0 video3", r["reason"])
|
||||||
|
self.assertTrue(camcheck.is_ring_short(r))
|
||||||
|
self.assertFalse(camcheck.is_vcint_failure(r))
|
||||||
|
self.assertEqual(r["map"]["slam_left"]["node"], 0)
|
||||||
|
|
||||||
def test_four_cameras_in_ring(self):
|
def test_four_cameras_in_ring(self):
|
||||||
make_ring(self.ring, ["a_video9", "b_video13", "c_video6", "d_video7", "a_video9_dk"])
|
make_ring(self.ring, ["a_video9", "b_video13", "c_video6", "d_video7", "a_video9_dk"])
|
||||||
r = camcheck.check(log=self.log, proc=False, ring_path=self.ring)
|
r = camcheck.check(log=self.log, proc=False, ring_path=self.ring)
|
||||||
|
|||||||
@@ -34,8 +34,28 @@ from tools.show_set import index, read_set # noqa: E402
|
|||||||
from tools import calib # noqa: E402
|
from tools import calib # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
PIPES = {'msm_vfe3_video0': 'slam_left', 'msm_vfe4_video0': 'slam_right', # as ft-hands maps them
|
PIPES = {'msm_vfe3_video0': 'slam_left', 'msm_vfe4_video0': 'slam_right', # with the colour module
|
||||||
'msm_vfe2_video0': 'upper_left', 'msm_vfe2_video1': 'upper_right'}
|
'msm_vfe2_video0': 'upper_left', 'msm_vfe2_video1': 'upper_right'}
|
||||||
|
|
||||||
|
|
||||||
|
def ring_names():
|
||||||
|
"""{/dev/videoN's N: calibration name} as ft-hands names them: by XRService's log
|
||||||
|
(camcheck.py), else by capture pipe (PIPES)."""
|
||||||
|
import camcheck
|
||||||
|
try:
|
||||||
|
by_log = {node: name for name, node in camcheck.read_log(camcheck.newest_log()).camera_map().items()}
|
||||||
|
except (OSError, ValueError):
|
||||||
|
by_log = {}
|
||||||
|
if by_log:
|
||||||
|
return by_log
|
||||||
|
out = {}
|
||||||
|
for path in os.listdir('/sys/class/video4linux'):
|
||||||
|
if path.startswith('video'):
|
||||||
|
with open('/sys/class/video4linux/%s/name' % path) as f:
|
||||||
|
name = PIPES.get(f.read().strip())
|
||||||
|
if name:
|
||||||
|
out[int(path[5:])] = name
|
||||||
|
return out
|
||||||
PAIRS = {'side': ('slam_left', 'slam_right'), 'upper': ('upper_left', 'upper_right')}
|
PAIRS = {'side': ('slam_left', 'slam_right'), 'upper': ('upper_left', 'upper_right')}
|
||||||
|
|
||||||
|
|
||||||
@@ -100,8 +120,9 @@ def live_pairs(count, names=PAIRS['side']):
|
|||||||
if not ring.alive():
|
if not ring.alive():
|
||||||
sys.exit('ft-camd isn\'t running (no heartbeat)')
|
sys.exit('ft-camd isn\'t running (no heartbeat)')
|
||||||
cams = {}
|
cams = {}
|
||||||
|
names_of = ring_names()
|
||||||
for c in ring.cams:
|
for c in ring.cams:
|
||||||
name = PIPES.get(open('/sys/class/video4linux/video%d/name' % c.node).read().strip())
|
name = names_of.get(c.node)
|
||||||
if name and not c.name.endswith('-dark'):
|
if name and not c.name.endswith('-dark'):
|
||||||
cams[name] = c
|
cams[name] = c
|
||||||
for k in range(count):
|
for k in range(count):
|
||||||
|
|||||||
+70
-14
@@ -27,7 +27,8 @@
|
|||||||
// default) tells from the hands it tracks (track/sides.h): once it's sure, it exchanges the two
|
// default) tells from the hands it tracks (track/sides.h): once it's sure, it exchanges the two
|
||||||
// cameras if they're backwards (the tracked views move with their images), and checks once
|
// cameras if they're backwards (the tracked views move with their images), and checks once
|
||||||
// more. 0 and 1 force the naming (1: exchanged; --swap-sides is --sides 1); it still checks,
|
// more. 0 and 1 force the naming (1: exchanged; --swap-sides is --sides 1); it still checks,
|
||||||
// and warns if the hands disagree. The decision is published in /run/user/UID/frametop-hands/sides.json (see
|
// and if the hands disagree it warns and publishes what the hands say as the truth ("swapped"),
|
||||||
|
// so recordings are labelled right while tracking keeps the forced names. The decision is published in /run/user/UID/frametop-hands/sides.json (see
|
||||||
// write_sides below) and, for recordings, in DIR/sides.json. --record-only can't tell (it tracks
|
// write_sides below) and, for recordings, in DIR/sides.json. --record-only can't tell (it tracks
|
||||||
// nothing): under auto it records the ring's names as they are.
|
// nothing): under auto it records the ring's names as they are.
|
||||||
//
|
//
|
||||||
@@ -63,7 +64,34 @@ namespace {
|
|||||||
|
|
||||||
volatile std::sig_atomic_t g_stop = 0, g_record = 0;
|
volatile std::sig_atomic_t g_stop = 0, g_record = 0;
|
||||||
|
|
||||||
// which calibrated camera each capture pipe carries (XRService's fixed routing)
|
// Which calibrated camera each video device carries. XRService numbers its tracking cameras
|
||||||
|
// (index 0 to 3: slam_left, slam_right, upper_left, upper_right) and logs the device each one
|
||||||
|
// opened ("TrackingCameraInit: index: 0. video device: /dev/video9"). The devices depend on
|
||||||
|
// the colour module: with it, the side cameras are on vfe3 and vfe4 and the upper pair on
|
||||||
|
// vfe2; without it, XRService runs the side cameras through the ISP on vfe0 and vfe1, and the
|
||||||
|
// upper pair on vfe3 and vfe4. So the running XRService's log decides; when it can't be read,
|
||||||
|
// the capture pipes as they are with the module. {} if the log has no cameras.
|
||||||
|
std::map<int, std::string> cameras_from_xrservice_log() {
|
||||||
|
static const char *const names[] = {"slam_left", "slam_right", "upper_left", "upper_right"};
|
||||||
|
const char *home = std::getenv("HOME");
|
||||||
|
std::ifstream in(std::string(home ? home : "") + "/.local/share/Steam/logs/xrservice.txt");
|
||||||
|
const std::string key = "TrackingCameraInit: index: ";
|
||||||
|
std::map<int, int> node_of; // index -> N of /dev/videoN, from the latest camera start
|
||||||
|
std::string line;
|
||||||
|
while (std::getline(in, line)) {
|
||||||
|
if (line.find("XRService logging to") != std::string::npos) node_of.clear();
|
||||||
|
const auto at = line.find(key);
|
||||||
|
int index = -1, node = -1;
|
||||||
|
if (at != std::string::npos &&
|
||||||
|
std::sscanf(line.c_str() + at + key.size(), "%d. video device: /dev/video%d", &index, &node) == 2 &&
|
||||||
|
index >= 0 && index < 4)
|
||||||
|
node_of[index] = node;
|
||||||
|
}
|
||||||
|
std::map<int, std::string> out;
|
||||||
|
for (auto &[index, node] : node_of) out[node] = names[index];
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
const char *camera_for_pipe(int node) {
|
const char *camera_for_pipe(int node) {
|
||||||
char path[64], name[64] = "";
|
char path[64], name[64] = "";
|
||||||
std::snprintf(path, sizeof path, "/sys/class/video4linux/video%d/name", node);
|
std::snprintf(path, sizeof path, "/sys/class/video4linux/video%d/name", node);
|
||||||
@@ -302,6 +330,7 @@ int main(int argc, char **argv) {
|
|||||||
double decided_after_s = -1;
|
double decided_after_s = -1;
|
||||||
if (sides_mode != "auto") truth = names_swapped, decided_by = sides_from, sides_state = "forced";
|
if (sides_mode != "auto") truth = names_swapped, decided_by = sides_from, sides_state = "forced";
|
||||||
std::string rec_dir;
|
std::string rec_dir;
|
||||||
|
std::string cams_json; // which device each calibrated camera is (set below), for the sides files
|
||||||
std::vector<std::pair<size_t, bool>> rec_names; // from which recorded set on, names_swapped was what
|
std::vector<std::pair<size_t, bool>> rec_names; // from which recorded set on, names_swapped was what
|
||||||
// DIR/sides.json beside a recording's sets.bin: how its side cameras are named. A set's
|
// DIR/sides.json beside a recording's sets.bin: how its side cameras are named. A set's
|
||||||
// names are right when its names_swapped equals swapped (null: not known when recorded).
|
// names are right when its names_swapped equals swapped (null: not known when recorded).
|
||||||
@@ -312,7 +341,8 @@ int main(int argc, char **argv) {
|
|||||||
write_file(rec_dir + "/sides.json",
|
write_file(rec_dir + "/sides.json",
|
||||||
"{\"swapped\": " + json_bool(truth) + ", \"decided_by\": " + json_str(truth ? decided_by : "") +
|
"{\"swapped\": " + json_bool(truth) + ", \"decided_by\": " + json_str(truth ? decided_by : "") +
|
||||||
", \"names_swapped\": [" + runs + "]" +
|
", \"names_swapped\": [" + runs + "]" +
|
||||||
(decision_evidence.empty() ? "" : ", \"evidence\": " + decision_evidence) + "}\n");
|
(decision_evidence.empty() ? "" : ", \"evidence\": " + decision_evidence) +
|
||||||
|
(cams_json.empty() ? "" : ", \"cameras\": " + cams_json) + "}\n");
|
||||||
};
|
};
|
||||||
auto start_recording = [&](const std::string &dir, std::string &e) {
|
auto start_recording = [&](const std::string &dir, std::string &e) {
|
||||||
rec = std::make_unique<Recorder>();
|
rec = std::make_unique<Recorder>();
|
||||||
@@ -340,19 +370,42 @@ int main(int argc, char **argv) {
|
|||||||
// (--with-color). Recorded names hold 15 characters, so "upper_right_dark" wouldn't fit.
|
// (--with-color). Recorded names hold 15 characters, so "upper_right_dark" wouldn't fit.
|
||||||
std::map<std::string, int> dark;
|
std::map<std::string, int> dark;
|
||||||
std::map<std::string, Camera> used;
|
std::map<std::string, Camera> used;
|
||||||
|
// ft-camd's cameras by XRService's numbering, else by capture pipe (see camera_for_pipe);
|
||||||
|
// ft-ringplay's (no device) by the name it gives
|
||||||
|
const std::map<int, std::string> by_log = cameras_from_xrservice_log();
|
||||||
|
const char *named_by = by_log.empty() ? "capture pipe" : "XRService's log";
|
||||||
for (int i = 0; i < ring.cameras(); ++i) {
|
for (int i = 0; i < ring.cameras(); ++i) {
|
||||||
if (ring.camera(i).flags & FH_CAM_COLOR) {
|
const fh_ring_cam_t &rc = ring.camera(i);
|
||||||
const std::string name = "color_video" + std::to_string(ring.camera(i).node);
|
if (rc.flags & FH_CAM_COLOR) {
|
||||||
|
const std::string name = "color_video" + std::to_string(rc.node);
|
||||||
dark[name] = i, color[name] = i;
|
dark[name] = i, color[name] = i;
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
// ft-camd's cameras by capture pipe; ft-ringplay's (no device) by the name it gives
|
const auto it = by_log.find(rc.node);
|
||||||
const char *name = camera_for_pipe(ring.camera(i).node);
|
const char *name = rc.node < 0 ? rc.name
|
||||||
if (!name && ring.camera(i).node < 0) name = ring.camera(i).name;
|
: !by_log.empty() ? (it == by_log.end() ? nullptr : it->second.c_str())
|
||||||
if (!name || !calib.count(name)) continue;
|
: camera_for_pipe(rc.node);
|
||||||
if (ring.camera(i).flags & FH_CAM_DARK) dark[std::string(name) + "_dk"] = i;
|
if (!name || !calib.count(name)) {
|
||||||
else index[name] = i, used[name] = calib[name];
|
if (!(rc.flags & FH_CAM_DARK)) std::fprintf(stderr, "video%d (%s): not one of the calibrated cameras, left out\n", rc.node, rc.name);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (int(rc.width) != calib[name].width || int(rc.height) != calib[name].height) {
|
||||||
|
std::fprintf(stderr, "video%d (%s) is %ux%u, but %s is calibrated at %dx%d: left out\n", rc.node, rc.name,
|
||||||
|
rc.width, rc.height, name, calib[name].width, calib[name].height);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (rc.flags & FH_CAM_DARK) {
|
||||||
|
dark[std::string(name) + "_dk"] = i;
|
||||||
|
} else if (index.count(name)) {
|
||||||
|
std::fprintf(stderr, "video%d (%s) would be %s too (video%d is): left out\n", rc.node, rc.name, name,
|
||||||
|
ring.camera(index[name]).node);
|
||||||
|
} else {
|
||||||
|
index[name] = i, used[name] = calib[name];
|
||||||
|
cams_json += std::string(cams_json.empty() ? "" : ", ") + json_str(name) + ": {\"node\": " +
|
||||||
|
std::to_string(rc.node) + ", \"ring\": " + json_str(rc.name) + "}";
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
cams_json = "{\"named_by\": " + json_str(named_by) + ", \"cameras\": {" + cams_json + "}}";
|
||||||
// ft-camd tells the side cameras' buffers apart by XRService's allocation order, which
|
// ft-camd tells the side cameras' buffers apart by XRService's allocation order, which
|
||||||
// some XRService restarts reverse (see the top).
|
// some XRService restarts reverse (see the top).
|
||||||
const bool have_sides = index.count("slam_left") && index.count("slam_right");
|
const bool have_sides = index.count("slam_left") && index.count("slam_right");
|
||||||
@@ -397,7 +450,7 @@ int main(int argc, char **argv) {
|
|||||||
}
|
}
|
||||||
const bool switching = automatic && !color.empty();
|
const bool switching = automatic && !color.empty();
|
||||||
Cams mode = switching || color.empty() ? Cams::Mono : fixed;
|
Cams mode = switching || color.empty() ? Cams::Mono : fixed;
|
||||||
std::printf("cameras:");
|
std::printf("cameras (by %s):", named_by);
|
||||||
for (auto &[name, i] : index) std::printf(" %s=video%d", name.c_str(), ring.camera(i).node);
|
for (auto &[name, i] : index) std::printf(" %s=video%d", name.c_str(), ring.camera(i).node);
|
||||||
for (auto &[name, i] : color) std::printf(" %s", name.c_str());
|
for (auto &[name, i] : color) std::printf(" %s", name.c_str());
|
||||||
std::printf(" tracking with %s%s models: %s%s, %d threads on CPUs", cams_name(mode),
|
std::printf(" tracking with %s%s models: %s%s, %d threads on CPUs", cams_name(mode),
|
||||||
@@ -446,6 +499,7 @@ int main(int argc, char **argv) {
|
|||||||
", \"decided_after_s\": " + std::to_string(decided_after_s) +
|
", \"decided_after_s\": " + std::to_string(decided_after_s) +
|
||||||
", \"evidence\": " + (decision_evidence.empty() ? "null" : decision_evidence) +
|
", \"evidence\": " + (decision_evidence.empty() ? "null" : decision_evidence) +
|
||||||
", \"checking\": " + (checking ? side_check.json() : "null") +
|
", \"checking\": " + (checking ? side_check.json() : "null") +
|
||||||
|
", \"cameras\": " + (cams_json.empty() ? "null" : cams_json) +
|
||||||
", \"updated_ns\": " + std::to_string(mono_ns()) + "}\n");
|
", \"updated_ns\": " + std::to_string(mono_ns()) + "}\n");
|
||||||
};
|
};
|
||||||
write_sides();
|
write_sides();
|
||||||
@@ -454,14 +508,16 @@ int main(int argc, char **argv) {
|
|||||||
const bool backwards = v == SideCheck::Swapped; // relative to the names as they are now
|
const bool backwards = v == SideCheck::Swapped; // relative to the names as they are now
|
||||||
const double after = (now - start) / 1e9;
|
const double after = (now - start) / 1e9;
|
||||||
const std::string ev = side_check.json(), text = side_check.summary();
|
const std::string ev = side_check.json(), text = side_check.summary();
|
||||||
if (sides_mode != "auto") { // forced: only say so
|
if (sides_mode != "auto") { // forced: the names stay; if the hands disagree, they're the truth
|
||||||
const std::string what = (sides_from == "option" ? "--sides " : "HANDS_SWAP_SIDES=") + sides_mode;
|
const std::string what = (sides_from == "option" ? "--sides " : "HANDS_SWAP_SIDES=") + sides_mode;
|
||||||
if (backwards)
|
if (backwards)
|
||||||
std::printf("side cameras: %s looks WRONG: the hands say the side cameras are the other way round (%s). "
|
std::printf("side cameras: %s looks WRONG: the hands say the side cameras are the other way round (%s). "
|
||||||
"Use auto.\n", what.c_str(), text.c_str());
|
"Tracking keeps the forced names; recordings are labelled by the hands. Use auto.\n",
|
||||||
|
what.c_str(), text.c_str());
|
||||||
else
|
else
|
||||||
std::printf("side cameras: %s agrees with the hands (%s)\n", what.c_str(), text.c_str());
|
std::printf("side cameras: %s agrees with the hands (%s)\n", what.c_str(), text.c_str());
|
||||||
sides_state = backwards ? "forced, disagrees" : "forced, agrees";
|
sides_state = backwards ? "forced, disagrees" : "forced, agrees";
|
||||||
|
if (backwards) truth = !names_swapped, decided_by = "auto", decided_after_s = after;
|
||||||
decision_evidence = ev;
|
decision_evidence = ev;
|
||||||
checking = false;
|
checking = false;
|
||||||
} else if (side_round == 0 || backwards) {
|
} else if (side_round == 0 || backwards) {
|
||||||
|
|||||||
@@ -77,6 +77,8 @@ ACTION_LABELS = {
|
|||||||
"screens_toggle": "Hide/show desktop screens", "keyboard_toggle": "Open/close keyboard",
|
"screens_toggle": "Hide/show desktop screens", "keyboard_toggle": "Open/close keyboard",
|
||||||
"float_toggle": "Float window in VR / put it back", "dock_all": "Put all floating windows back",
|
"float_toggle": "Float window in VR / put it back", "dock_all": "Put all floating windows back",
|
||||||
"pause_toggle": "Pause/resume Frametop (for VR games)",
|
"pause_toggle": "Pause/resume Frametop (for VR games)",
|
||||||
|
"spin_next": "Spin the panels: next one on the right to the front",
|
||||||
|
"spin_prev": "Spin the panels: next one on the left to the front",
|
||||||
"key": "Pass through as key",
|
"key": "Pass through as key",
|
||||||
"none": "Do nothing",
|
"none": "Do nothing",
|
||||||
}
|
}
|
||||||
@@ -163,9 +165,10 @@ GAZE_MOUSE = {"precision": "Gaze precision: hold to steer with the mouse, releas
|
|||||||
MODIFIER_CODES = {29: 29, 97: 29, 42: 42, 54: 42, 56: 56, 100: 56, 125: 125, 126: 125}
|
MODIFIER_CODES = {29: 29, 97: 29, 42: 42, 54: 42, 56: 56, 100: 56, 125: 125, 126: 125}
|
||||||
MODIFIER_NAMES = {29: "Ctrl", 42: "Shift", 56: "Alt", 125: "Meta"}
|
MODIFIER_NAMES = {29: "Ctrl", 42: "Shift", 56: "Alt", 125: "Meta"}
|
||||||
# What a rules file without "key_bindings" gets (the relay's DEFAULT_KEY_BINDINGS): a Meta tap
|
# What a rules file without "key_bindings" gets (the relay's DEFAULT_KEY_BINDINGS): a Meta tap
|
||||||
# opens Steam's menu, Meta+J and Meta+K click at the gaze, Meta+Shift+F floats a window.
|
# opens Steam's menu, Meta+J and Meta+K click at the gaze, Meta+Shift+F floats a window,
|
||||||
|
# Meta+Alt+Tab and Meta+Alt+Shift+Tab spin the panels.
|
||||||
DEFAULT_KEY_BINDINGS = {"125": "steam_menu", "125+36": "gaze_left", "125+37": "gaze_right",
|
DEFAULT_KEY_BINDINGS = {"125": "steam_menu", "125+36": "gaze_left", "125+37": "gaze_right",
|
||||||
"42+125+33": "float_toggle"}
|
"42+125+33": "float_toggle", "56+125+15": "spin_next", "42+56+125+15": "spin_prev"}
|
||||||
|
|
||||||
|
|
||||||
def key_bindings(rules):
|
def key_bindings(rules):
|
||||||
|
|||||||
+100
-28
@@ -30,7 +30,9 @@ gaze_quickcal = the gaze service's one-dot check ("quickcal" to @ft_gazed), sens
|
|||||||
layout_reset = put the desktop screens back in their saved layout, screens_toggle = hide or show the desktop screens,
|
layout_reset = put the desktop screens back in their saved layout, screens_toggle = hide or show the desktop screens,
|
||||||
keyboard_toggle = open or close Frametop's keyboard, float_toggle = float the desktop window under the
|
keyboard_toggle = open or close Frametop's keyboard, float_toggle = float the desktop window under the
|
||||||
pointer (else the active one) in VR, or put it back if it floats, dock_all = put every floating
|
pointer (else the active one) in VR, or put it back if it floats, dock_all = put every floating
|
||||||
window back (both to ft-floatd, @frametop_float), profile:NAME = switch to that profile (ft-layout
|
window back (both to ft-floatd, @frametop_float), spin_next and spin_prev = turn every panel in the
|
||||||
|
room about your head so the next one to the right or left comes to the front (ft-screens' "spin",
|
||||||
|
Meta+Alt+Tab and Meta+Alt+Shift+Tab by default), profile:NAME = switch to that profile (ft-layout
|
||||||
use NAME: its screens and apps; docs/profiles.md), steam_menu = open the SteamVR dashboard on
|
use NAME: its screens and apps; docs/profiles.md), steam_menu = open the SteamVR dashboard on
|
||||||
Steam's menu, or close the dashboard (steam/ft-steam menu, through Steam's UI), pause_toggle =
|
Steam's menu, or close the dashboard (steam/ft-steam menu, through Steam's UI), pause_toggle =
|
||||||
pause Frametop for a VR game, or resume it (game_pause.py), command:CMD = run
|
pause Frametop for a VR game, or resume it (game_pause.py), command:CMD = run
|
||||||
@@ -64,12 +66,18 @@ Typing on a keyboard sends the helper "typing" (at most 4 times a second): it ta
|
|||||||
pinches right after a key, since typing touches thumb to index like a pinch.
|
pinches right after a key, since typing touches thumb to index like a pinch.
|
||||||
|
|
||||||
Keys also go to ft-screens (@ft_screens, the Frametop desktop's compositor), which
|
Keys also go to ft-screens (@ft_screens, the Frametop desktop's compositor), which
|
||||||
types them into the desktop screen that has focus: from pass-through keyboards, and
|
types them into the desktop screen that has focus: from pass-through keyboards, a USB or
|
||||||
keys a pointer device passes through. Typing goes to the panel clicked last, and
|
Bluetooth keyboard's media keys (its Consumer Control node, which has volume keys, so it's
|
||||||
ft-screens says which ("keyboard desktop|steam" on the control socket, every second).
|
never grabbed and gamescope has them too), and keys a pointer device passes through. In
|
||||||
While it's the desktop, pass-through keyboards are grabbed, so gamescope, which reads
|
pointer mode a mouse button passed through as a key (BTN_MOUSE..BTN_TASK, a side button for
|
||||||
every keyboard itself, doesn't type them into its focused app too. Without word from
|
Back) goes there too, and ft-screens gives it the screen the pointer is on, not the one
|
||||||
ft-screens for 3 seconds they're released. With SHARE_KEYS=1 in ~/.config/frametop.conf,
|
typing goes to (it releases the button itself when the pointer leaves the screens, they
|
||||||
|
hide, or Frametop pauses). Other buttons and keys from KEY_OK up don't go there.
|
||||||
|
Typing goes to the panel clicked last, and ft-screens says which ("keyboard
|
||||||
|
desktop|steam" on the control socket, every second). While it's the desktop,
|
||||||
|
pass-through keyboards are grabbed, so gamescope, which reads every keyboard itself,
|
||||||
|
doesn't type them into its focused app too. Without word from ft-screens for 3 seconds
|
||||||
|
they're released. With SHARE_KEYS=1 in ~/.config/frametop.conf,
|
||||||
a grabbed keyboard's keys also go out as "key <code> <value> <device name>" datagrams on
|
a grabbed keyboard's keys also go out as "key <code> <value> <device name>" datagrams on
|
||||||
@frametop_keys, for programs that watch every keyboard for a hotkey and lose it to the grab.
|
@frametop_keys, for programs that watch every keyboard for a hotkey and lose it to the grab.
|
||||||
It's off by default: any local process that binds that name first gets every key typed
|
It's off by default: any local process that binds that name first gets every key typed
|
||||||
@@ -151,11 +159,12 @@ KEY_A = 30
|
|||||||
REL_X, REL_Y, REL_WHEEL, REL_MAX = 0x00, 0x01, 0x08, 0x0F
|
REL_X, REL_Y, REL_WHEEL, REL_MAX = 0x00, 0x01, 0x08, 0x0F
|
||||||
SCROLLS = {0x06, REL_WHEEL, 0x0B, 0x0C} # REL_HWHEEL, REL_WHEEL and their _HI_RES
|
SCROLLS = {0x06, REL_WHEEL, 0x0B, 0x0C} # REL_HWHEEL, REL_WHEEL and their _HI_RES
|
||||||
BTN_LEFT, BTN_RIGHT, BTN_MIDDLE, BTN_SIDE, BTN_EXTRA = 0x110, 0x111, 0x112, 0x113, 0x114
|
BTN_LEFT, BTN_RIGHT, BTN_MIDDLE, BTN_SIDE, BTN_EXTRA = 0x110, 0x111, 0x112, 0x113, 0x114
|
||||||
|
BTN_MOUSE, BTN_TASK = 0x110, 0x117 # mouse buttons, the first and the last
|
||||||
KEY_LEFTMETA, KEY_RIGHTMETA = 125, 126
|
KEY_LEFTMETA, KEY_RIGHTMETA = 125, 126
|
||||||
KEY_VOLUMEDOWN, KEY_VOLUMEUP = 114, 115
|
KEY_MUTE, KEY_VOLUMEDOWN, KEY_VOLUMEUP = 113, 114, 115
|
||||||
# Volume keys are remapped to KEY_MACRO29 and KEY_MACRO30: above 255, so X11 can't
|
# Volume keys are remapped to KEY_MACRO28, KEY_MACRO29 and KEY_MACRO30: above 255, so X11
|
||||||
# carry them, and bound to nothing in the default keymap.
|
# can't carry them, and bound to nothing in the default keymap.
|
||||||
VOLUME_STANDIN = {KEY_VOLUMEUP: 0x2AC, KEY_VOLUMEDOWN: 0x2AD}
|
VOLUME_STANDIN = {KEY_MUTE: 0x2AB, KEY_VOLUMEUP: 0x2AC, KEY_VOLUMEDOWN: 0x2AD}
|
||||||
VOLUME_ORIGINAL = {v: k for k, v in VOLUME_STANDIN.items()}
|
VOLUME_ORIGINAL = {v: k for k, v in VOLUME_STANDIN.items()}
|
||||||
VOLUME_CODES = set(VOLUME_STANDIN) | set(VOLUME_ORIGINAL)
|
VOLUME_CODES = set(VOLUME_STANDIN) | set(VOLUME_ORIGINAL)
|
||||||
BUS_USB, BUS_BLUETOOTH, BUS_VIRTUAL = 0x03, 0x05, 0x06
|
BUS_USB, BUS_BLUETOOTH, BUS_VIRTUAL = 0x03, 0x05, 0x06
|
||||||
@@ -209,14 +218,16 @@ RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
|
|||||||
ACTIONS = ("left", "right", "middle", "back", "scroll_up", "scroll_down", "dashboard", "recenter",
|
ACTIONS = ("left", "right", "middle", "back", "scroll_up", "scroll_down", "dashboard", "recenter",
|
||||||
"pointer_toggle", "follow_toggle", "gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right",
|
"pointer_toggle", "follow_toggle", "gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right",
|
||||||
"gaze_quickcal", "sens_up", "sens_down", "layout_reset", "screens_toggle", "keyboard_toggle", "float_toggle",
|
"gaze_quickcal", "sens_up", "sens_down", "layout_reset", "screens_toggle", "keyboard_toggle", "float_toggle",
|
||||||
"dock_all", "steam_menu", "pause_toggle", "key", "none")
|
"dock_all", "spin_next", "spin_prev", "steam_menu", "pause_toggle", "key", "none")
|
||||||
# Gaze mode is a mouse feature: these never come from a controller button (docs/gaze-controllers.md).
|
# Gaze mode is a mouse feature: these never come from a controller button (docs/gaze-controllers.md).
|
||||||
GAZE_ACTIONS = ("gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right", "gaze_quickcal")
|
GAZE_ACTIONS = ("gaze_toggle", "gaze_precision", "gaze_drag", "gaze_left", "gaze_right", "gaze_quickcal")
|
||||||
# Key combinations a rules file without "key_bindings" gets: a Meta tap opens Steam's menu, Meta+J
|
# Key combinations a rules file without "key_bindings" gets: a Meta tap opens Steam's menu, Meta+J
|
||||||
# and Meta+K click at the gaze (free on the Frametop desktop, and apps don't use Meta),
|
# and Meta+K click at the gaze (free on the Frametop desktop, and apps don't use Meta),
|
||||||
# Meta+Shift+F floats a window.
|
# Meta+Shift+F floats a window, and Meta+Alt+Tab and Meta+Alt+Shift+Tab spin the panels around
|
||||||
|
# you (ft-screens' lazy susan); not Meta+Tab, which is Cmd+Tab on a Mac reached through a remote
|
||||||
|
# desktop like RustDesk.
|
||||||
DEFAULT_KEY_BINDINGS = {"125": "steam_menu", "125+36": "gaze_left", "125+37": "gaze_right",
|
DEFAULT_KEY_BINDINGS = {"125": "steam_menu", "125+36": "gaze_left", "125+37": "gaze_right",
|
||||||
"42+125+33": "float_toggle"}
|
"42+125+33": "float_toggle", "56+125+15": "spin_next", "42+56+125+15": "spin_prev"}
|
||||||
KEY_F24 = 194 # sent to the desktop with a Meta combination (see the top)
|
KEY_F24 = 194 # sent to the desktop with a Meta combination (see the top)
|
||||||
# Key combinations ("key_bindings"): modifiers, each side's code folded into the left one's.
|
# Key combinations ("key_bindings"): modifiers, each side's code folded into the left one's.
|
||||||
MODIFIERS = {29: 29, 97: 29, 42: 42, 54: 42, 56: 56, 100: 56, 125: 125, 126: 125}
|
MODIFIERS = {29: 29, 97: 29, 42: 42, 54: 42, 56: 56, 100: 56, 125: 125, 126: 125}
|
||||||
@@ -232,6 +243,9 @@ GAZED = "\0ft_gazed"
|
|||||||
FLOAT = "\0frametop_float" # ft-floatd, floating windows in the Frametop desktop
|
FLOAT = "\0frametop_float" # ft-floatd, floating windows in the Frametop desktop
|
||||||
# Actions for ft-floatd ("float_toggle", "dock_all"): they don't need pointer mode.
|
# Actions for ft-floatd ("float_toggle", "dock_all"): they don't need pointer mode.
|
||||||
FLOAT_ACTIONS = {"float_toggle": b"float pointer", "dock_all": b"dock all"}
|
FLOAT_ACTIONS = {"float_toggle": b"float pointer", "dock_all": b"dock all"}
|
||||||
|
# Actions for ft-screens: spin_next and spin_prev turn every panel in the room about your head,
|
||||||
|
# so the next one to the right or left comes to the front. They don't need pointer mode either.
|
||||||
|
SCREENS_ACTIONS = {"spin_next": b"spin next", "spin_prev": b"spin prev"}
|
||||||
PROFILE = "profile:" # "profile:NAME": switch to that profile (doesn't need pointer mode either)
|
PROFILE = "profile:" # "profile:NAME": switch to that profile (doesn't need pointer mode either)
|
||||||
COMMAND = "command:" # "command:CMD": run CMD (nor does this)
|
COMMAND = "command:" # "command:CMD": run CMD (nor does this)
|
||||||
|
|
||||||
@@ -242,7 +256,7 @@ def known_action(a):
|
|||||||
|
|
||||||
|
|
||||||
def needs_pointer(a):
|
def needs_pointer(a):
|
||||||
return a not in FLOAT_ACTIONS and a not in ("steam_menu", "pause_toggle") and not a.startswith((PROFILE, COMMAND))
|
return a not in FLOAT_ACTIONS and a not in SCREENS_ACTIONS and a not in ("steam_menu", "pause_toggle") and not a.startswith((PROFILE, COMMAND))
|
||||||
|
|
||||||
|
|
||||||
def works_paused(a):
|
def works_paused(a):
|
||||||
@@ -422,9 +436,13 @@ class Volume:
|
|||||||
|
|
||||||
def key(self, fd, code, value, now):
|
def key(self, fd, code, value, now):
|
||||||
if value == 1:
|
if value == 1:
|
||||||
self.held = (fd, code)
|
if VOLUME_ORIGINAL.get(code, code) == KEY_MUTE:
|
||||||
self.step(code)
|
subprocess.Popen(["wpctl", "set-mute", "@DEFAULT_AUDIO_SINK@", "toggle"],
|
||||||
self.next_at = now + self.DELAY
|
stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||||
|
else:
|
||||||
|
self.held = (fd, code)
|
||||||
|
self.step(code)
|
||||||
|
self.next_at = now + self.DELAY
|
||||||
elif value == 0 and self.held == (fd, code):
|
elif value == 0 and self.held == (fd, code):
|
||||||
self.release()
|
self.release()
|
||||||
|
|
||||||
@@ -487,6 +505,10 @@ class Pointer:
|
|||||||
self.pending = 0
|
self.pending = 0
|
||||||
self.pending_since = 0.0
|
self.pending_since = 0.0
|
||||||
self.gaze_awake_until = 0.0 # the helper's gaze mode keeps the pointer until then
|
self.gaze_awake_until = 0.0 # the helper's gaze mode keeps the pointer until then
|
||||||
|
# Driver buttons this pointer pressed and hasn't released (trigger, b, x, joystick: the
|
||||||
|
# left, right, middle and back actions, from a mouse button, a mapped controller button
|
||||||
|
# or a key combination): pausing drops releases, so stand_down has to send them itself.
|
||||||
|
self.driver_down = set()
|
||||||
|
|
||||||
def send(self, command, droppable=False):
|
def send(self, command, droppable=False):
|
||||||
"""To the helper, in order, without blocking. While the helper doesn't keep up (place and
|
"""To the helper, in order, without blocking. While the helper doesn't keep up (place and
|
||||||
@@ -596,6 +618,10 @@ class Pointer:
|
|||||||
self.wake(now)
|
self.wake(now)
|
||||||
self.flush()
|
self.flush()
|
||||||
self.send(f"btn {driver} {value}")
|
self.send(f"btn {driver} {value}")
|
||||||
|
if value == 1:
|
||||||
|
self.driver_down.add(driver)
|
||||||
|
else:
|
||||||
|
self.driver_down.discard(driver)
|
||||||
elif value != 1:
|
elif value != 1:
|
||||||
return # the rest act on press
|
return # the rest act on press
|
||||||
elif name in ("scroll_up", "scroll_down"):
|
elif name in ("scroll_up", "scroll_down"):
|
||||||
@@ -695,18 +721,39 @@ class Pointer:
|
|||||||
return wait
|
return wait
|
||||||
|
|
||||||
def stand_down(self):
|
def stand_down(self):
|
||||||
"""Frametop is pausing: a pulse under way ends now, and the pointer lets go."""
|
"""Frametop is pausing: a pulse under way ends now, and the pointer lets go.
|
||||||
|
|
||||||
|
A click held into the pause (driver_down) comes up here, since pausing drops its
|
||||||
|
release; otherwise the driver keeps the button down and the virtual controller
|
||||||
|
reconnects with it pressed on resume. Gaze holds (gazekey, gazedrag, precision) aren't
|
||||||
|
tracked here: the helper ends them itself when the pointer hides.
|
||||||
|
|
||||||
|
The releases go before "hide", and "hide" follows them even when the pointer was off
|
||||||
|
already (the idle timeout or pointer_toggle with a button held): a helper built before
|
||||||
|
releases stopped waking it would wake on one and connect the virtual controller
|
||||||
|
during the game.
|
||||||
|
|
||||||
|
Not covered: gaze mode's held-back press (the helper's aim, before it becomes a real
|
||||||
|
press) turns into a click on its release. The helper drops it without a click when it
|
||||||
|
reads "hide" in the same loop as the release, as it normally does. If its socket was
|
||||||
|
full, the rest of these wait in the queue (send), "hide" can land a loop later, and
|
||||||
|
that press clicks once as the pause starts."""
|
||||||
|
releases = [f"btn {driver} 0" for driver in sorted(self.driver_down)]
|
||||||
|
self.driver_down.clear()
|
||||||
if self.system_release is not None:
|
if self.system_release is not None:
|
||||||
self.send("btn system 0")
|
releases.append("btn system 0")
|
||||||
if self.claim_release is not None:
|
if self.claim_release is not None:
|
||||||
self.send("btn a 0")
|
releases.append("btn a 0")
|
||||||
if self.scroll_until is not None:
|
if self.scroll_until is not None:
|
||||||
self.send("scroll 0 0")
|
releases.append("scroll 0 0")
|
||||||
|
for command in releases:
|
||||||
|
self.send(command)
|
||||||
self.system_at = self.system_release = self.claim_at = self.claim_release = self.scroll_until = None
|
self.system_at = self.system_release = self.claim_at = self.claim_release = self.scroll_until = None
|
||||||
self.dx = self.dy = self.pending = 0
|
self.dx = self.dy = self.pending = 0
|
||||||
self.gaze_awake_until = 0.0
|
self.gaze_awake_until = 0.0
|
||||||
if self.active:
|
if self.active or releases:
|
||||||
self.send("hide")
|
self.send("hide")
|
||||||
|
if self.active:
|
||||||
self.active = False
|
self.active = False
|
||||||
log("pointer off (paused)")
|
log("pointer off (paused)")
|
||||||
|
|
||||||
@@ -925,6 +972,13 @@ def main():
|
|||||||
except OSError:
|
except OSError:
|
||||||
pass # the Frametop desktop isn't running
|
pass # the Frametop desktop isn't running
|
||||||
log(action)
|
log(action)
|
||||||
|
elif action in SCREENS_ACTIONS:
|
||||||
|
if value == 1:
|
||||||
|
try:
|
||||||
|
screens_sock.sendto(SCREENS_ACTIONS[action], SCREENS)
|
||||||
|
except OSError:
|
||||||
|
pass # the Frametop desktop isn't running
|
||||||
|
log(action)
|
||||||
elif state["pointer"]:
|
elif state["pointer"]:
|
||||||
state["pointer"].action(action, value, now, source)
|
state["pointer"].action(action, value, now, source)
|
||||||
|
|
||||||
@@ -1001,9 +1055,14 @@ def main():
|
|||||||
|
|
||||||
screens_down = set() # keys the desktop was told went down and not yet up (see reconcile_desktop_keys)
|
screens_down = set() # keys the desktop was told went down and not yet up (see reconcile_desktop_keys)
|
||||||
|
|
||||||
def to_screens(code, value):
|
def to_screens(code, value, button=False):
|
||||||
"""A key for the desktop screens (ft-screens decides whether it types)."""
|
"""A key for the desktop screens (ft-screens decides whether it types): one below
|
||||||
if value in (0, 1) and code < BTN_MISC:
|
BTN_MISC, or with button, a mouse button (BTN_MOUSE..BTN_TASK), which ft-screens gives
|
||||||
|
the screen the pointer is on. Nothing else (gamepad, joystick, digitizer buttons, keys
|
||||||
|
from KEY_OK up), but the release of anything the desktop has down."""
|
||||||
|
if value not in (0, 1):
|
||||||
|
return
|
||||||
|
if code < BTN_MISC or (button and BTN_MOUSE <= code <= BTN_TASK) or (not value and code in screens_down):
|
||||||
try:
|
try:
|
||||||
screens_sock.sendto(f"key {code} {value}".encode(), SCREENS)
|
screens_sock.sendto(f"key {code} {value}".encode(), SCREENS)
|
||||||
except OSError:
|
except OSError:
|
||||||
@@ -1263,9 +1322,12 @@ def main():
|
|||||||
vr_bind(time.monotonic()) # a helper that's already running keeps its buttons in step
|
vr_bind(time.monotonic()) # a helper that's already running keeps its buttons in step
|
||||||
waiting = False # a keyboard's grab waits for its keys to come up
|
waiting = False # a keyboard's grab waits for its keys to come up
|
||||||
# A relay that went away with a key down left it down on the desktop, where this one
|
# A relay that went away with a key down left it down on the desktop, where this one
|
||||||
# never sent it: modifiers come up there now (a release of a key that isn't down is nothing).
|
# never sent it: modifiers and mouse buttons come up there now (a release of a key that
|
||||||
|
# isn't down is nothing).
|
||||||
for code in sorted(MODIFIERS):
|
for code in sorted(MODIFIERS):
|
||||||
to_screens(code, 0)
|
to_screens(code, 0)
|
||||||
|
for code in range(BTN_MOUSE, BTN_TASK + 1):
|
||||||
|
to_screens(code, 0, button=True)
|
||||||
while True:
|
while True:
|
||||||
now = time.monotonic()
|
now = time.monotonic()
|
||||||
pointer = state["pointer"]
|
pointer = state["pointer"]
|
||||||
@@ -1354,6 +1416,12 @@ def main():
|
|||||||
volume.key(fd, code, value, now)
|
volume.key(fd, code, value, now)
|
||||||
continue
|
continue
|
||||||
if node.role == "volume":
|
if node.role == "volume":
|
||||||
|
# A keyboard's media keys (its Consumer Control node) go to the desktop like a
|
||||||
|
# pass-through keyboard's, and nowhere else: the node isn't grabbed, so gamescope
|
||||||
|
# and SteamVR have them already. Not platform buttons: the headset's click button
|
||||||
|
# is KEY_SELECT on gpio-keys (BUS_HOST). Nor a volume key a remap missed.
|
||||||
|
if etype == EV_KEY and node.bus in (BUS_USB, BUS_BLUETOOTH) and code not in VOLUME_CODES:
|
||||||
|
to_screens(code, value)
|
||||||
continue
|
continue
|
||||||
if node.role != "pointer":
|
if node.role != "pointer":
|
||||||
# Observed only, unless typing goes to the desktop. Key combinations work on
|
# Observed only, unless typing goes to the desktop. Key combinations work on
|
||||||
@@ -1388,7 +1456,11 @@ def main():
|
|||||||
continue
|
continue
|
||||||
target = mouse if code >= BTN_MISC else keyboard
|
target = mouse if code >= BTN_MISC else keyboard
|
||||||
target.emit(etype, code, value)
|
target.emit(etype, code, value)
|
||||||
to_screens(code, value)
|
# A mouse button goes to the desktop only when pointer mode passes it through
|
||||||
|
# as a key (a side button for Back there). The rest, like every click with
|
||||||
|
# POINTER=0 or paused, reach the desktop through a laser, if at all. (A
|
||||||
|
# key-mapped button still goes to the virtual mouse too.)
|
||||||
|
to_screens(code, value, button=bool(pointer))
|
||||||
if value:
|
if value:
|
||||||
node.held.add(code)
|
node.held.add(code)
|
||||||
else:
|
else:
|
||||||
|
|||||||
+166
-22
@@ -1,11 +1,13 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""Offline test of the input relay's key combinations and modifier taps.
|
"""Offline test of the input relay's key combinations and modifier taps, and of which keys and
|
||||||
|
mouse buttons reach the desktop (ft-screens).
|
||||||
|
|
||||||
Runs the relay's main() against a fake pass-through keyboard and a fake mouse (pipes), with
|
Runs the relay's main() against a fake pass-through keyboard, a fake mouse, and two fake nodes
|
||||||
every socket it sends to renamed, no uinput devices, no grabs, and ft-steam swapped for a
|
with volume keys, a keyboard's Consumer Control node and the headset's gpio-keys (pipes), with
|
||||||
logger. Pausing (game_pause.py) is a stub: no threads, state file, or services. Nothing reaches
|
every socket it sends to renamed, no uinput devices (they record what they're sent), no grabs,
|
||||||
the live desktop, SteamVR, or the running relay, so it's safe next to them. Rules come from the
|
no volume changes, and ft-steam swapped for a logger. Pausing (game_pause.py) is a stub: no
|
||||||
test, not ~/.config/frametop-input.json.
|
threads, state file, or services. Nothing reaches the live desktop, SteamVR, or the running
|
||||||
|
relay, so it's safe next to them. Rules come from the test, not ~/.config/frametop-input.json.
|
||||||
|
|
||||||
input/test/keys-test.py [RELAY] (default: input/input-relay.py next to this folder)
|
input/test/keys-test.py [RELAY] (default: input/input-relay.py next to this folder)
|
||||||
"""
|
"""
|
||||||
@@ -72,38 +74,55 @@ with open(relay.FT_STEAM, "w") as f:
|
|||||||
os.chmod(relay.FT_STEAM, 0o755)
|
os.chmod(relay.FT_STEAM, 0o755)
|
||||||
|
|
||||||
|
|
||||||
class NoDevice:
|
EMITTED = [] # (virtual device, code, value): what the virtual mouse and keyboard were sent
|
||||||
def __init__(self, *args, **kwargs):
|
VOLUME = [] # (code, value): volume keys the relay handled itself (no wpctl here)
|
||||||
pass
|
|
||||||
|
|
||||||
def emit(self, *args):
|
|
||||||
pass
|
class NoDevice:
|
||||||
|
"""A virtual device that only records what it's sent."""
|
||||||
|
|
||||||
|
def __init__(self, name, *args, **kwargs):
|
||||||
|
self.name = name.split()[-1] # mouse or keyboard
|
||||||
|
|
||||||
|
def emit(self, etype, code, value):
|
||||||
|
EMITTED.append((self.name, code, value))
|
||||||
|
|
||||||
def sync(self):
|
def sync(self):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
relay.Virtual = NoDevice
|
relay.Virtual = NoDevice
|
||||||
|
relay.Volume.key = lambda self, fd, code, value, now: VOLUME.append((code, value))
|
||||||
relay.log = lambda *args: None # the relay's own log
|
relay.log = lambda *args: None # the relay's own log
|
||||||
bindings = {"now": None} # the rules' key_bindings; None: the relay's defaults
|
bindings = {"now": None} # the rules' key_bindings; None: the relay's defaults
|
||||||
|
devices = {"roles": {}, "buttons": {}} # the rules' "devices" roles and per-device "buttons"
|
||||||
|
MOUSE_ID = "usb:0003:0004:test mouse" # the fake mouse's id (Node.id)
|
||||||
|
|
||||||
|
|
||||||
def read_rules(path=None):
|
def read_rules(path=None):
|
||||||
rules = {"devices": {}, "buttons": {}, "controller_buttons": {}}
|
rules = {"devices": {i: {"role": r} for i, r in devices["roles"].items()},
|
||||||
|
"buttons": {i: dict(b) for i, b in devices["buttons"].items()}, "controller_buttons": {}}
|
||||||
rules["key_bindings"] = dict(relay.DEFAULT_KEY_BINDINGS if bindings["now"] is None else bindings["now"])
|
rules["key_bindings"] = dict(relay.DEFAULT_KEY_BINDINGS if bindings["now"] is None else bindings["now"])
|
||||||
return rules
|
return rules
|
||||||
|
|
||||||
|
|
||||||
relay.read_rules = read_rules
|
relay.read_rules = read_rules
|
||||||
relay.read_config = lambda path=None: {"POINTER": "0"}
|
conf = {"POINTER": "0"} # ~/.config/frametop.conf, as the relay reads it
|
||||||
|
relay.read_config = lambda path=None: dict(conf)
|
||||||
|
|
||||||
# The fake devices: /dev/input/event900 (keyboard) and event901 (mouse), each a pipe.
|
# The fake devices, each a pipe: /dev/input/event900 (keyboard), event901 (mouse), event902 (the
|
||||||
|
# keyboard's Consumer Control node: media and volume keys), event903 (the headset's gpio-keys).
|
||||||
kb_r, kb_w = os.pipe()
|
kb_r, kb_w = os.pipe()
|
||||||
ms_r, ms_w = os.pipe()
|
ms_r, ms_w = os.pipe()
|
||||||
for fd in (kb_r, ms_r):
|
cc_r, cc_w = os.pipe()
|
||||||
|
gp_r, gp_w = os.pipe()
|
||||||
|
for fd in (kb_r, ms_r, cc_r, gp_r):
|
||||||
os.set_blocking(fd, False)
|
os.set_blocking(fd, False)
|
||||||
FAKE = {"/dev/input/event900": kb_r, "/dev/input/event901": ms_r}
|
FAKE = {"/dev/input/event900": kb_r, "/dev/input/event901": ms_r, "/dev/input/event902": cc_r,
|
||||||
HELD = {kb_r: set(), ms_r: set()} # what EVIOCGKEY says each holds
|
"/dev/input/event903": gp_r}
|
||||||
|
READ_END = {kb_w: kb_r, ms_w: ms_r, cc_w: cc_r, gp_w: gp_r}
|
||||||
|
HELD = {kb_r: set(), ms_r: set(), cc_r: set(), gp_r: set()} # what EVIOCGKEY says each holds
|
||||||
|
BUS_HOST = 0x19
|
||||||
|
|
||||||
|
|
||||||
class Inode:
|
class Inode:
|
||||||
@@ -112,7 +131,7 @@ class Inode:
|
|||||||
|
|
||||||
fake_os = type(os)("os")
|
fake_os = type(os)("os")
|
||||||
fake_os.__dict__.update(os.__dict__)
|
fake_os.__dict__.update(os.__dict__)
|
||||||
fake_os.listdir = lambda path: ["event900", "event901"] if path == "/dev/input" else os.listdir(path)
|
fake_os.listdir = lambda path: [p.split("/")[-1] for p in FAKE] if path == "/dev/input" else os.listdir(path)
|
||||||
fake_os.stat = lambda path, *a, **k: Inode() if path in FAKE else os.stat(path, *a, **k)
|
fake_os.stat = lambda path, *a, **k: Inode() if path in FAKE else os.stat(path, *a, **k)
|
||||||
fake_os.access = lambda path, mode, *a, **k: path in FAKE or os.access(path, mode, *a, **k)
|
fake_os.access = lambda path, mode, *a, **k: path in FAKE or os.access(path, mode, *a, **k)
|
||||||
relay.os = fake_os
|
relay.os = fake_os
|
||||||
@@ -135,7 +154,17 @@ relay.fcntl = fake_fcntl
|
|||||||
def probe(path):
|
def probe(path):
|
||||||
if FAKE[path] == kb_r:
|
if FAKE[path] == kb_r:
|
||||||
return relay.Node(path, kb_r, "test keyboard", relay.BUS_USB, 1, 2, "", False, True)
|
return relay.Node(path, kb_r, "test keyboard", relay.BUS_USB, 1, 2, "", False, True)
|
||||||
return relay.Node(path, ms_r, "test mouse", relay.BUS_USB, 3, 4, "", True, False)
|
if FAKE[path] == ms_r:
|
||||||
|
return relay.Node(path, ms_r, "test mouse", relay.BUS_USB, 3, 4, "", True, False)
|
||||||
|
# Nodes with volume keys (role "volume"). take_volume does nothing with --no-grab, so they
|
||||||
|
# come as if it had remapped their volume keys to the stand-ins.
|
||||||
|
if FAKE[path] == cc_r:
|
||||||
|
node = relay.Node(path, cc_r, "test keyboard Consumer Control", relay.BUS_USB, 1, 2, "", False, False,
|
||||||
|
candidate=False)
|
||||||
|
else:
|
||||||
|
node = relay.Node(path, gp_r, "gpio-keys", BUS_HOST, 0, 0, "", False, False, candidate=False)
|
||||||
|
node.remapped = True
|
||||||
|
return node
|
||||||
|
|
||||||
|
|
||||||
relay.probe = probe
|
relay.probe = probe
|
||||||
@@ -152,7 +181,7 @@ def send(fd, etype, code, value):
|
|||||||
|
|
||||||
|
|
||||||
def key(code, value, fd=kb_w):
|
def key(code, value, fd=kb_w):
|
||||||
held = HELD[ms_r if fd == ms_w else kb_r]
|
held = HELD[READ_END[fd]]
|
||||||
(held.add if value else held.discard)(code)
|
(held.add if value else held.discard)(code)
|
||||||
send(fd, relay.EV_KEY, code, value)
|
send(fd, relay.EV_KEY, code, value)
|
||||||
|
|
||||||
@@ -167,6 +196,14 @@ def typed():
|
|||||||
return got
|
return got
|
||||||
|
|
||||||
|
|
||||||
|
def emitted():
|
||||||
|
"""What the virtual mouse and keyboard were sent since the last call."""
|
||||||
|
got = []
|
||||||
|
while EMITTED:
|
||||||
|
got.append(EMITTED.pop(0))
|
||||||
|
return got
|
||||||
|
|
||||||
|
|
||||||
def use(key_bindings):
|
def use(key_bindings):
|
||||||
bindings["now"] = key_bindings
|
bindings["now"] = key_bindings
|
||||||
c = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
|
c = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
|
||||||
@@ -206,13 +243,14 @@ def check(label, got, want):
|
|||||||
failures.append(label)
|
failures.append(label)
|
||||||
|
|
||||||
|
|
||||||
META, RMETA, SHIFT, CTRL, RCTRL, A, F, J, P = 125, 126, 42, 29, 97, 30, 33, 36, 25
|
META, RMETA, SHIFT, CTRL, RCTRL, ALT, A, F, J, P, TAB = 125, 126, 42, 29, 97, 56, 30, 33, 36, 25, 15
|
||||||
F24 = ["key 194 1", "key 194 0"]
|
F24 = ["key 194 1", "key 194 0"]
|
||||||
|
|
||||||
|
|
||||||
def tests():
|
def tests():
|
||||||
time.sleep(1.5) # the relay's first device scan
|
time.sleep(1.5) # the relay's first device scan
|
||||||
typed()
|
check("a relay that starts releases the modifiers and mouse buttons on the desktop", typed(),
|
||||||
|
[f"key {c} 0" for c in sorted(relay.MODIFIERS)] + [f"key {c} 0" for c in range(0x110, 0x118)])
|
||||||
key(META, 1); key(META, 0)
|
key(META, 1); key(META, 0)
|
||||||
check("Meta tap (default): Steam menu", lines(steam_log), ["menu"])
|
check("Meta tap (default): Steam menu", lines(steam_log), ["menu"])
|
||||||
check("Meta tap: the desktop gets F24 before Meta's release", typed(), ["key 125 1"] + F24 + ["key 125 0"])
|
check("Meta tap: the desktop gets F24 before Meta's release", typed(), ["key 125 1"] + F24 + ["key 125 0"])
|
||||||
@@ -237,6 +275,37 @@ def tests():
|
|||||||
check("Meta+J (default gaze click): no tap", lines(steam_log), ["menu", "menu"])
|
check("Meta+J (default gaze click): no tap", lines(steam_log), ["menu", "menu"])
|
||||||
check("Meta+J: F24, Meta up early, its real release dropped", typed(), ["key 125 1"] + F24 + ["key 125 0"])
|
check("Meta+J: F24, Meta up early, its real release dropped", typed(), ["key 125 1"] + F24 + ["key 125 0"])
|
||||||
|
|
||||||
|
# Mouse buttons reach the desktop only when pointer mode passes them through as keys.
|
||||||
|
emitted()
|
||||||
|
key(relay.BTN_LEFT, 1, ms_w); key(relay.BTN_LEFT, 0, ms_w); key(relay.BTN_SIDE, 1, ms_w); key(relay.BTN_SIDE, 0, ms_w)
|
||||||
|
check("POINTER=0: clicks don't reach the desktop", typed(), [])
|
||||||
|
check("POINTER=0: clicks go to the virtual mouse", emitted(),
|
||||||
|
[("mouse", relay.BTN_LEFT, 1), ("mouse", relay.BTN_LEFT, 0), ("mouse", relay.BTN_SIDE, 1),
|
||||||
|
("mouse", relay.BTN_SIDE, 0)])
|
||||||
|
|
||||||
|
# A keyboard's Consumer Control node isn't grabbed, so gamescope has its keys already: its
|
||||||
|
# media keys go to the desktop and nowhere else, its volume keys to the relay's volume
|
||||||
|
# handling only. The headset's gpio-keys (BUS_HOST) send nothing on.
|
||||||
|
PLAYPAUSE, SELECT, VOLUP = 164, 353, relay.KEY_VOLUMEUP
|
||||||
|
key(PLAYPAUSE, 1, cc_w); key(PLAYPAUSE, 0, cc_w)
|
||||||
|
check("media key: reaches the desktop", typed(), ["key 164 1", "key 164 0"])
|
||||||
|
check("media key: no virtual device", emitted(), [])
|
||||||
|
standin = relay.VOLUME_STANDIN[VOLUP]
|
||||||
|
key(standin, 1, cc_w); key(standin, 0, cc_w)
|
||||||
|
check("volume key (its stand-in): the relay's volume handling only", (typed(), emitted(), VOLUME[:]),
|
||||||
|
([], [], [(standin, 1), (standin, 0)]))
|
||||||
|
VOLUME.clear()
|
||||||
|
key(VOLUP, 1, cc_w); key(VOLUP, 0, cc_w)
|
||||||
|
check("a volume key a remap missed reaches nothing", (typed(), emitted(), VOLUME[:]), ([], [], []))
|
||||||
|
key(SELECT, 1, gp_w); key(SELECT, 0, gp_w)
|
||||||
|
check("the headset's click button (gpio-keys, KEY_SELECT) reaches nothing", (typed(), emitted()), ([], []))
|
||||||
|
key(PLAYPAUSE, 1, cc_w)
|
||||||
|
time.sleep(1.2) # the relay's once-a-second check (reconcile_desktop_keys)
|
||||||
|
check("a media key held for over a second stays down on the desktop", typed(), ["key 164 1"])
|
||||||
|
HELD[cc_r].discard(PLAYPAUSE) # its node lets go, and the release never comes
|
||||||
|
time.sleep(1.2)
|
||||||
|
check("...and comes up there once its node doesn't hold it", typed(), ["key 164 0"])
|
||||||
|
|
||||||
use({"29+42+33": f"command:echo combo >> {cmd_log}", "125": "none"})
|
use({"29+42+33": f"command:echo combo >> {cmd_log}", "125": "none"})
|
||||||
key(CTRL, 1); key(SHIFT, 1); key(F, 1); key(F, 0); key(SHIFT, 0); key(CTRL, 0)
|
key(CTRL, 1); key(SHIFT, 1); key(F, 1); key(F, 0); key(SHIFT, 0); key(CTRL, 0)
|
||||||
check("Ctrl+Shift+F runs its command", lines(cmd_log), ["combo"])
|
check("Ctrl+Shift+F runs its command", lines(cmd_log), ["combo"])
|
||||||
@@ -265,6 +334,81 @@ def tests():
|
|||||||
key(META, 1); key(J, 1); key(J, 0); key(META, 0)
|
key(META, 1); key(J, 1); key(J, 0); key(META, 0)
|
||||||
check("resumed: Meta+J is a combination again", typed(), ["key 125 1"] + F24 + ["key 125 0"])
|
check("resumed: Meta+J is a combination again", typed(), ["key 125 1"] + F24 + ["key 125 0"])
|
||||||
|
|
||||||
|
# Pointer mode: a click held into a pause comes up as it starts, then the pointer hides
|
||||||
|
# (stand_down); the release that comes during the pause reaches no one.
|
||||||
|
helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
|
||||||
|
helper.bind(relay.HELPER)
|
||||||
|
helper.settimeout(0.05)
|
||||||
|
|
||||||
|
def clicks():
|
||||||
|
"""The helper's button commands and hides since the last call (not the claim pulse)."""
|
||||||
|
got = []
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
m = helper.recv(256).decode()
|
||||||
|
except socket.timeout:
|
||||||
|
return got
|
||||||
|
if m == "hide" or (m.startswith("btn ") and not m.startswith("btn a ")):
|
||||||
|
got.append(m)
|
||||||
|
|
||||||
|
conf["POINTER"] = "1"
|
||||||
|
use({})
|
||||||
|
key(relay.BTN_LEFT, 1, ms_w)
|
||||||
|
time.sleep(0.5) # the claim pulse, 0.3 s after the pointer wakes
|
||||||
|
check("pointer mode: a held left button reaches the helper", clicks(), ["btn trigger 1"])
|
||||||
|
pause("on")
|
||||||
|
check("pause with the left button held: it comes up, then the pointer hides", clicks(),
|
||||||
|
["btn trigger 0", "hide"])
|
||||||
|
key(relay.BTN_LEFT, 0, ms_w)
|
||||||
|
check("paused: its release reaches no one", clicks(), [])
|
||||||
|
pause("off")
|
||||||
|
key(relay.BTN_LEFT, 1, ms_w); key(relay.BTN_LEFT, 0, ms_w)
|
||||||
|
check("resumed: a click goes to the helper again", clicks(), ["btn trigger 1", "btn trigger 0"])
|
||||||
|
|
||||||
|
# Pointer mode: a click is the helper's alone. A side button mapped to "key" (Back on the
|
||||||
|
# desktop) goes to the desktop too, and its release still does after a pause starts; paused,
|
||||||
|
# clicks go to the virtual mouse only.
|
||||||
|
check("pointer mode: a click doesn't reach the desktop", typed(), [])
|
||||||
|
emitted()
|
||||||
|
devices["buttons"] = {MOUSE_ID: {str(relay.BTN_SIDE): "key"}}
|
||||||
|
use({})
|
||||||
|
key(relay.BTN_SIDE, 1, ms_w)
|
||||||
|
check("pointer mode: a side button mapped to key reaches the desktop", typed(), [f"key {relay.BTN_SIDE} 1"])
|
||||||
|
check("...and the virtual mouse, not the helper", (emitted(), clicks()), ([("mouse", relay.BTN_SIDE, 1)], []))
|
||||||
|
pause("on")
|
||||||
|
clicks()
|
||||||
|
key(relay.BTN_SIDE, 0, ms_w)
|
||||||
|
check("paused: its release still reaches the desktop", typed(), [f"key {relay.BTN_SIDE} 0"])
|
||||||
|
emitted()
|
||||||
|
key(relay.BTN_SIDE, 1, ms_w); key(relay.BTN_SIDE, 0, ms_w); key(relay.BTN_LEFT, 1, ms_w); key(relay.BTN_LEFT, 0, ms_w)
|
||||||
|
check("paused: clicks don't reach the desktop", typed(), [])
|
||||||
|
check("paused: clicks go to the virtual mouse", emitted(),
|
||||||
|
[("mouse", relay.BTN_SIDE, 1), ("mouse", relay.BTN_SIDE, 0), ("mouse", relay.BTN_LEFT, 1),
|
||||||
|
("mouse", relay.BTN_LEFT, 0)])
|
||||||
|
pause("off")
|
||||||
|
clicks()
|
||||||
|
devices["roles"] = {MOUSE_ID: "passthrough"}
|
||||||
|
use({})
|
||||||
|
key(relay.BTN_LEFT, 1, ms_w); key(relay.BTN_LEFT, 0, ms_w); key(relay.BTN_SIDE, 1, ms_w); key(relay.BTN_SIDE, 0, ms_w)
|
||||||
|
check("a pass-through mouse's clicks don't reach the desktop", typed(), [])
|
||||||
|
check("...nor a virtual device or the helper", (emitted(), clicks()), ([], []))
|
||||||
|
devices["roles"], devices["buttons"] = {}, {}
|
||||||
|
conf["POINTER"] = "0"
|
||||||
|
|
||||||
|
use(None) # the defaults: Meta+Alt+Tab and Meta+Alt+Shift+Tab spin the panels (ft-screens)
|
||||||
|
key(META, 1); key(ALT, 1); key(TAB, 1); key(TAB, 0); key(ALT, 0); key(META, 0)
|
||||||
|
got = typed()
|
||||||
|
check("Meta+Alt+Tab (default): spin next", [m for m in got if m.startswith("spin")], ["spin next"])
|
||||||
|
check("Meta+Alt+Tab: Tab isn't typed", [m for m in got if m.startswith("key 15 ")], [])
|
||||||
|
key(META, 1); key(ALT, 1); key(SHIFT, 1); key(TAB, 1); key(TAB, 0); key(SHIFT, 0); key(ALT, 0); key(META, 0)
|
||||||
|
check("Meta+Alt+Shift+Tab (default): spin prev", [m for m in typed() if m.startswith("spin")], ["spin prev"])
|
||||||
|
pause("on")
|
||||||
|
key(META, 1); key(ALT, 1); key(TAB, 1); key(TAB, 0); key(ALT, 0); key(META, 0)
|
||||||
|
check("paused: Meta+Alt+Tab doesn't spin", [m for m in typed() if m.startswith("spin")], [])
|
||||||
|
pause("off")
|
||||||
|
check("spinning works without pointer mode", (relay.needs_pointer("spin_next"), relay.needs_pointer("spin_prev")),
|
||||||
|
(False, False))
|
||||||
|
|
||||||
check("an empty command isn't an action", relay.known_action("command: "), False)
|
check("an empty command isn't an action", relay.known_action("command: "), False)
|
||||||
check("steam_menu, pause_toggle and commands work without pointer mode",
|
check("steam_menu, pause_toggle and commands work without pointer mode",
|
||||||
(relay.needs_pointer("steam_menu"), relay.needs_pointer("pause_toggle"), relay.needs_pointer("command:ls")),
|
(relay.needs_pointer("steam_menu"), relay.needs_pointer("pause_toggle"), relay.needs_pointer("command:ls")),
|
||||||
|
|||||||
Executable
+139
@@ -0,0 +1,139 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Offline test of the relay's pointer standing down when Frametop pauses: a click still held
|
||||||
|
when the pause starts (a mouse button, a mapped controller button, or a key combination mapped
|
||||||
|
to left, right, middle or back) is released by stand_down, and "hide" follows the release, even
|
||||||
|
when the pointer was off already. Gaze holds (gazekey, gazedrag, precision) are the helper's
|
||||||
|
to end, so stand_down sends nothing for them. The pointer's socket is a recorder, so nothing
|
||||||
|
reaches the helper, SteamVR, or the running relay.
|
||||||
|
|
||||||
|
input/test/pause-buttons-test.py [RELAY]
|
||||||
|
"""
|
||||||
|
import importlib.util
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
|
||||||
|
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||||
|
relay_path = sys.argv[1] if len(sys.argv) > 1 else os.path.join(HERE, "..", "input-relay.py")
|
||||||
|
tag = f"ft_pause_buttons_test_{os.getpid()}"
|
||||||
|
src = open(relay_path).read().replace('"\\0frametop_relay"', f'"\\0{tag}_relay"')
|
||||||
|
# macOS has no SOCK_NONBLOCK; the pointer's socket is replaced below anyway.
|
||||||
|
src = src.replace("socket.SOCK_DGRAM | socket.SOCK_NONBLOCK", "socket.SOCK_DGRAM")
|
||||||
|
relay = importlib.util.module_from_spec(importlib.util.spec_from_loader("relay", loader=None))
|
||||||
|
relay.__file__ = os.path.abspath(relay_path)
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.abspath(relay_path)))
|
||||||
|
exec(compile(src, relay_path, "exec"), relay.__dict__)
|
||||||
|
|
||||||
|
failures = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(label, got, want):
|
||||||
|
ok = got == want
|
||||||
|
print(("ok " if ok else "FAIL ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
|
||||||
|
if not ok:
|
||||||
|
failures.append(label)
|
||||||
|
|
||||||
|
|
||||||
|
class Recorder:
|
||||||
|
"""Stands in for the helper's socket: remembers every command, in order."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.sent = []
|
||||||
|
|
||||||
|
def sendto(self, data, addr):
|
||||||
|
self.sent.append(data.decode())
|
||||||
|
|
||||||
|
|
||||||
|
def pointer():
|
||||||
|
p = relay.Pointer(0.02, 3600.0)
|
||||||
|
p.sock = Recorder()
|
||||||
|
return p
|
||||||
|
|
||||||
|
|
||||||
|
def since(p, mark):
|
||||||
|
"""What the pointer sent the helper after mark (a length of p.sock.sent)."""
|
||||||
|
return p.sock.sent[mark:]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- the fix
|
||||||
|
p = pointer()
|
||||||
|
p.action("left", 1, 10.0) # a mouse click (or drag) held: btn trigger 1
|
||||||
|
check("press reaches the helper", p.sock.sent, ["show", "recenter", "btn trigger 1"])
|
||||||
|
mark = len(p.sock.sent)
|
||||||
|
p.stand_down() # Frametop pauses while it's held
|
||||||
|
check("stand_down releases the held button, then hides", since(p, mark), ["btn trigger 0", "hide"])
|
||||||
|
|
||||||
|
# The release that pausing drops later is already covered: the button is no longer down,
|
||||||
|
# so a stray second stand_down sends nothing.
|
||||||
|
mark = len(p.sock.sent)
|
||||||
|
p.stand_down()
|
||||||
|
check("a second stand_down sends nothing more", since(p, mark), [])
|
||||||
|
|
||||||
|
# Two buttons down at once: a left drag tilted with the right button
|
||||||
|
p = pointer()
|
||||||
|
p.action("left", 1, 10.0)
|
||||||
|
p.action("right", 1, 10.5)
|
||||||
|
mark = len(p.sock.sent)
|
||||||
|
p.stand_down()
|
||||||
|
check("both held buttons are released, then it hides", since(p, mark), ["btn b 0", "btn trigger 0", "hide"])
|
||||||
|
|
||||||
|
# A mapped controller button and a key combination take the same path as the mouse.
|
||||||
|
p = pointer()
|
||||||
|
p.action("middle", 1, 10.0, "right") # a controller button mapped to middle
|
||||||
|
p.action("back", 1, 10.1, "keyboard") # a key combination mapped to back
|
||||||
|
mark = len(p.sock.sent)
|
||||||
|
p.stand_down()
|
||||||
|
check("controller and key combination clicks are released too", since(p, mark),
|
||||||
|
["btn joystick 0", "btn x 0", "hide"])
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- the pointer already off
|
||||||
|
# A release with no "hide" after it would wake a helper that wakes on any btn, connecting the
|
||||||
|
# virtual controller during the game.
|
||||||
|
p = pointer()
|
||||||
|
p.idle = 30.0
|
||||||
|
p.action("left", 1, 10.0)
|
||||||
|
for t in (10.3, 10.4, 41.0): # the claim pulse, then 30 s with no mouse input
|
||||||
|
p.tick(t)
|
||||||
|
check("held 30 s with no mouse input: the pointer goes off", (p.active, p.sock.sent[-1]), (False, "hide"))
|
||||||
|
mark = len(p.sock.sent)
|
||||||
|
p.stand_down()
|
||||||
|
check("idle with a button held: the release, then hide", since(p, mark), ["btn trigger 0", "hide"])
|
||||||
|
|
||||||
|
p = pointer()
|
||||||
|
p.action("left", 1, 10.0)
|
||||||
|
p.action("pointer_toggle", 1, 10.5)
|
||||||
|
mark = len(p.sock.sent)
|
||||||
|
p.stand_down()
|
||||||
|
check("pointer toggled off with a button held: the release, then hide", since(p, mark),
|
||||||
|
["btn trigger 0", "hide"])
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------- the ordinary path
|
||||||
|
p = pointer()
|
||||||
|
p.action("left", 1, 10.0)
|
||||||
|
p.action("left", 0, 11.0) # released before the pause
|
||||||
|
mark = len(p.sock.sent)
|
||||||
|
p.stand_down()
|
||||||
|
check("a button released before the pause: stand_down only hides", since(p, mark), ["hide"])
|
||||||
|
|
||||||
|
p = pointer()
|
||||||
|
p.idle = 30.0
|
||||||
|
p.action("left", 1, 10.0)
|
||||||
|
p.action("left", 0, 11.0)
|
||||||
|
for t in (10.3, 10.4, 42.0): # off by itself
|
||||||
|
p.tick(t)
|
||||||
|
mark = len(p.sock.sent)
|
||||||
|
p.stand_down()
|
||||||
|
check("nothing held and the pointer off: stand_down sends nothing", since(p, mark), [])
|
||||||
|
|
||||||
|
# Gaze holds go to the helper as their own commands, and it ends them when the pointer hides.
|
||||||
|
p = pointer()
|
||||||
|
p.action("gaze_left", 1, 10.0, "keyboard") # Meta+J held
|
||||||
|
p.action("gaze_drag", 1, 10.1, "mouse")
|
||||||
|
mark = len(p.sock.sent)
|
||||||
|
p.stand_down()
|
||||||
|
check("gaze holds: stand_down sends only hide (the helper ends them)", since(p, mark), ["hide"])
|
||||||
|
|
||||||
|
print()
|
||||||
|
if failures:
|
||||||
|
print(f"{len(failures)} failed: {', '.join(failures)}")
|
||||||
|
sys.exit(1)
|
||||||
|
print("all ok")
|
||||||
+2
-1
@@ -75,7 +75,7 @@ step "3/10 input relay (keeps Bluetooth mice working in SteamVR, device roles, b
|
|||||||
|
|
||||||
step "4/10 3D mouse: SteamVR driver"
|
step "4/10 3D mouse: SteamVR driver"
|
||||||
"$root/pointer/driver/build.sh"
|
"$root/pointer/driver/build.sh"
|
||||||
"$root/pointer/driver/install.sh" install 2>&1 | grep -v xdg-open
|
"$root/pointer/driver/install.sh" install
|
||||||
|
|
||||||
step "5/10 3D mouse: pointer helper service"
|
step "5/10 3D mouse: pointer helper service"
|
||||||
"$root/pointer/helper/build.sh"
|
"$root/pointer/helper/build.sh"
|
||||||
@@ -93,6 +93,7 @@ step "7/10 multi-screen desktop (ft-screens), Frametop Input Settings, and Frame
|
|||||||
"$root/remote/install.sh"
|
"$root/remote/install.sh"
|
||||||
on_frame "sed -i 's/^POINTER=0/POINTER=1/' ~/.config/frametop.conf; grep -q '^POINTER=' ~/.config/frametop.conf || echo 'POINTER=1' >> ~/.config/frametop.conf"
|
on_frame "sed -i 's/^POINTER=0/POINTER=1/' ~/.config/frametop.conf; grep -q '^POINTER=' ~/.config/frametop.conf || echo 'POINTER=1' >> ~/.config/frametop.conf"
|
||||||
echo "the launcher's Desktop entry now opens the multi-screen desktop; 3D mouse on (POINTER=1 in ~/.config/frametop.conf)"
|
echo "the launcher's Desktop entry now opens the multi-screen desktop; 3D mouse on (POINTER=1 in ~/.config/frametop.conf)"
|
||||||
|
"$root/scripts/conf-migrate.sh"
|
||||||
|
|
||||||
step "8/10 gaze mode (optional, experimental: the pointer goes where you look)"
|
step "8/10 gaze mode (optional, experimental: the pointer goes where you look)"
|
||||||
gaze=0
|
gaze=0
|
||||||
|
|||||||
Executable
+3
@@ -0,0 +1,3 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Hide/Show Screens' own program (see ft-layout-reset).
|
||||||
|
exec "$(dirname "$(readlink -f "$0")")/ft-layout" toggle "$@"
|
||||||
Executable
+4
@@ -0,0 +1,4 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Reset Screen Layout's own program. Steam lists menu entries by program, so entries that
|
||||||
|
# shared ft-layout (with different arguments) launched as one.
|
||||||
|
exec "$(dirname "$(readlink -f "$0")")/ft-layout" apply "$@"
|
||||||
+2
-1
@@ -534,7 +534,8 @@ def capture_screens():
|
|||||||
# ---------------------------------------------------------------- gamescope (dashboard panels)
|
# ---------------------------------------------------------------- gamescope (dashboard panels)
|
||||||
|
|
||||||
def vrcmd(*args, timeout=10):
|
def vrcmd(*args, timeout=10):
|
||||||
env = dict(os.environ, LD_LIBRARY_PATH=os.path.dirname(VRCMD))
|
# SteamVR's config folder: in the desktop, XDG_CONFIG_HOME is its own (docs/design.md).
|
||||||
|
env = dict(os.environ, LD_LIBRARY_PATH=os.path.dirname(VRCMD), XDG_CONFIG_HOME=os.path.expanduser("~/.config"))
|
||||||
try:
|
try:
|
||||||
return subprocess.run([VRCMD, *args], capture_output=True, text=True, timeout=timeout, env=env).stdout
|
return subprocess.run([VRCMD, *args], capture_output=True, text=True, timeout=timeout, env=env).stdout
|
||||||
except (OSError, subprocess.TimeoutExpired):
|
except (OSError, subprocess.TimeoutExpired):
|
||||||
|
|||||||
@@ -14,6 +14,10 @@ frame="$root/scripts/frame.sh"
|
|||||||
src=$FRAME_REPO/pointer/driver
|
src=$FRAME_REPO/pointer/driver
|
||||||
dest='$HOME/.local/share/frametop/ft_pointer' # expanded on the Frame, in the commands below
|
dest='$HOME/.local/share/frametop/ft_pointer' # expanded on the Frame, in the commands below
|
||||||
reg='/opt/steamvr/bin/linuxarm64/vrpathreg'
|
reg='/opt/steamvr/bin/linuxarm64/vrpathreg'
|
||||||
|
# SteamVR's tools with SteamVR's config folder and libraries. A terminal in the Frametop desktop
|
||||||
|
# has XDG_CONFIG_HOME=~/.config/frametop (session/frametop-session.sh), where vrpathreg finds no
|
||||||
|
# registry, so it made one there that SteamVR never reads.
|
||||||
|
steamvr='env XDG_CONFIG_HOME=$HOME/.config LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64' # expanded on the Frame
|
||||||
|
|
||||||
case ${1:-install} in
|
case ${1:-install} in
|
||||||
install)
|
install)
|
||||||
@@ -21,6 +25,11 @@ case ${1:-install} in
|
|||||||
# Replacing the files of a driver SteamVR has loaded leaves its input bindings in a bad
|
# Replacing the files of a driver SteamVR has loaded leaves its input bindings in a bad
|
||||||
# state (the 3D mouse no longer gets the laser) until SteamVR restarts, so an unchanged
|
# state (the 3D mouse no longer gets the laser) until SteamVR restarts, so an unchanged
|
||||||
# driver is left alone.
|
# driver is left alone.
|
||||||
|
# A registry an earlier install made in the desktop's config folder (see steamvr above) has
|
||||||
|
# no SteamVR in it, and hid SteamVR from OpenVR programs started in the desktop
|
||||||
|
# (VRInitError_Init_InstallationNotFound): it goes.
|
||||||
|
# vrpathreg runs 'xdg-open vrmonitor://driverinstalled' for SteamVR's desktop monitor, which
|
||||||
|
# the Frame doesn't have, so the desktop said "could not read file": a stand-in takes it.
|
||||||
"$frame" --host "set -e; test -f $src/build/driver_ft_pointer.so
|
"$frame" --host "set -e; test -f $src/build/driver_ft_pointer.so
|
||||||
rm -rf $dest.new; mkdir -p $dest.new/bin/linuxarm64
|
rm -rf $dest.new; mkdir -p $dest.new/bin/linuxarm64
|
||||||
cp -r $src/ft_pointer/. $dest.new/
|
cp -r $src/ft_pointer/. $dest.new/
|
||||||
@@ -31,15 +40,22 @@ else
|
|||||||
rm -rf $dest; mv $dest.new $dest
|
rm -rf $dest; mv $dest.new $dest
|
||||||
echo 'installed; restart SteamVR to load it'
|
echo 'installed; restart SteamVR to load it'
|
||||||
fi
|
fi
|
||||||
LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg adddriver $dest
|
stray=\$HOME/.config/frametop/openvr/openvrpaths.vrpath
|
||||||
LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg show | grep -A3 -i 'external'" ;;
|
if grep -qs '\"jsonid\" : \"vrpathreg\"' \$stray && grep -qsE '\"runtime\"[[:space:]]*:[[:space:]]*null' \$stray; then
|
||||||
|
rm -f \$stray; rmdir \$HOME/.config/frametop/openvr 2>/dev/null || true
|
||||||
|
echo 'removed a SteamVR path registry an earlier install left in ~/.config/frametop/openvr'
|
||||||
|
fi
|
||||||
|
noopen=\$(mktemp -d); trap 'rm -rf \$noopen' EXIT
|
||||||
|
printf '#!/bin/sh\nexit 0\n' > \$noopen/xdg-open; chmod +x \$noopen/xdg-open
|
||||||
|
PATH=\$noopen:\$PATH $steamvr $reg adddriver $dest
|
||||||
|
$steamvr $reg show | grep -A3 -i 'external'" ;;
|
||||||
uninstall)
|
uninstall)
|
||||||
"$frame" --host "LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg removedriver $dest; rm -rf $dest; echo 'removed; restart SteamVR to unload it'" ;;
|
"$frame" --host "$steamvr $reg removedriver $dest; rm -rf $dest; echo 'removed; restart SteamVR to unload it'" ;;
|
||||||
send)
|
send)
|
||||||
"$frame" --host "python3 -c 'import socket,sys; s=socket.socket(socket.AF_UNIX,socket.SOCK_DGRAM); s.sendto(sys.argv[1].encode(), \"\\0ft_pointer\")' $(printf %q "${2:?command}")" ;;
|
"$frame" --host "python3 -c 'import socket,sys; s=socket.socket(socket.AF_UNIX,socket.SOCK_DGRAM); s.sendto(sys.argv[1].encode(), \"\\0ft_pointer\")' $(printf %q "${2:?command}")" ;;
|
||||||
probe) "$frame" -C pointer/probe 'LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 ./build/vrprobe' ;;
|
probe) "$frame" -C pointer/probe "$steamvr ./build/vrprobe" ;;
|
||||||
aimhere)
|
aimhere)
|
||||||
read -r yaw pitch < <("$frame" -C pointer/probe 'LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 ./build/vrprobe' | sed -n 's/^head yaw = \([-0-9.]*\) pitch = \([-0-9.]*\)$/\1 \2/p')
|
read -r yaw pitch < <("$frame" -C pointer/probe "$steamvr ./build/vrprobe" | sed -n 's/^head yaw = \([-0-9.]*\) pitch = \([-0-9.]*\)$/\1 \2/p')
|
||||||
[ -n "${yaw:-}" ] || { echo "head pose not valid (headset off?)" >&2; exit 1; }
|
[ -n "${yaw:-}" ] || { echo "head pose not valid (headset off?)" >&2; exit 1; }
|
||||||
"$0" send "aim $yaw $pitch" && echo "aimed at yaw $yaw pitch $pitch" ;;
|
"$0" send "aim $yaw $pitch" && echo "aimed at yaw $yaw pitch $pitch" ;;
|
||||||
log) "$frame" --host "grep -iE 'ft_pointer' ~/.local/share/Steam/logs/vrserver.txt | tail -n ${2:-20}" ;;
|
log) "$frame" --host "grep -iE 'ft_pointer' ~/.local/share/Steam/logs/vrserver.txt | tail -n ${2:-20}" ;;
|
||||||
|
|||||||
@@ -1563,8 +1563,19 @@ int main() {
|
|||||||
// (Moves were taken first, above.)
|
// (Moves were taken first, above.)
|
||||||
const bool mouseInput = std::strncmp(buf, "btn", 3) == 0 || std::strncmp(buf, "scroll", 6) == 0;
|
const bool mouseInput = std::strncmp(buf, "btn", 3) == 0 || std::strncmp(buf, "scroll", 6) == 0;
|
||||||
if (mouseInput) lastMouse = Clock::now();
|
if (mouseInput) lastMouse = Clock::now();
|
||||||
// Any mouse input wakes the pointer (after a controller took over, or a helper restart).
|
// Mouse input wakes the pointer (after a controller took over, or a helper restart),
|
||||||
if (!active && mouseInput) wake(Clock::now());
|
// but a release doesn't (a button up, a scroll back to 0 0): the relay sends those with
|
||||||
|
// the pointer off when Frametop pauses for a game (input-relay.py stand_down), and
|
||||||
|
// waking would connect the virtual controller during the game. The release is still
|
||||||
|
// handled below (it ends its press, or goes to the driver), so no button stays down.
|
||||||
|
{
|
||||||
|
char name[16];
|
||||||
|
int v = 1;
|
||||||
|
double sx = 1, sy = 1;
|
||||||
|
const bool release = (std::sscanf(buf, "btn %15s %d", name, &v) == 2 && v == 0) ||
|
||||||
|
(std::sscanf(buf, "scroll %lf %lf", &sx, &sy) == 2 && sx == 0 && sy == 0);
|
||||||
|
if (!active && mouseInput && !release) wake(Clock::now());
|
||||||
|
}
|
||||||
char key[128];
|
char key[128];
|
||||||
double px, py, pz, pyaw, ppitch, proll = 0, pgrab = -1;
|
double px, py, pz, pyaw, ppitch, proll = 0, pgrab = -1;
|
||||||
if (std::sscanf(buf, "grabprobe %127s", key) == 1) {
|
if (std::sscanf(buf, "grabprobe %127s", key) == 1) {
|
||||||
@@ -2394,7 +2405,11 @@ int main() {
|
|||||||
|
|
||||||
// A held-back press (see the top): held still long enough, it's a real press (a drag);
|
// A held-back press (see the top): held still long enough, it's a real press (a drag);
|
||||||
// released, it's a click where the pointer is now (this frame's pose has gone out).
|
// released, it's a click where the pointer is now (this frame's pose has gone out).
|
||||||
if (!active) aimHeld = aimRight = clickPress = aimHand = confirmLesson = false; // released meanwhile: nothing to click
|
// Released meanwhile: nothing to click, and the click it was due (pressRight: a right one) is
|
||||||
|
// forgotten too, or the next press after the pointer wakes would go out as a right click.
|
||||||
|
// A "hide" read a loop after the release is too late: the click has gone out by then
|
||||||
|
// (input-relay.py stand_down).
|
||||||
|
if (!active) aimHeld = aimRight = clickPress = pressRight = aimHand = confirmLesson = false;
|
||||||
if (aimHeld && !aimHand && nudgeMoved < 0.2 && tnow - aimSince >= std::chrono::duration<double>(gazeHold)) {
|
if (aimHeld && !aimHand && nudgeMoved < 0.2 && tnow - aimSince >= std::chrono::duration<double>(gazeHold)) {
|
||||||
aimHeld = false;
|
aimHeld = false;
|
||||||
gazeBack = true;
|
gazeBack = true;
|
||||||
|
|||||||
+66
-3
@@ -63,6 +63,7 @@
|
|||||||
|
|
||||||
#include "vr.h"
|
#include "vr.h"
|
||||||
#include "controller-click.h"
|
#include "controller-click.h"
|
||||||
|
#include "relay-buttons.h"
|
||||||
|
|
||||||
#define MAX_SCREENS 24 // screens and spare outputs
|
#define MAX_SCREENS 24 // screens and spare outputs
|
||||||
// A screen counts as playing a video while its last VIDEO_COMMITS commits each redrew at
|
// A screen counts as playing a video while its last VIDEO_COMMITS commits each redrew at
|
||||||
@@ -130,6 +131,7 @@ struct server {
|
|||||||
uint32_t watch_until;
|
uint32_t watch_until;
|
||||||
uint32_t typed_ms; // the last key sent to the desktop: its screen counts as focused
|
uint32_t typed_ms; // the last key sent to the desktop: its screen counts as focused
|
||||||
struct screen *pointer_focus;
|
struct screen *pointer_focus;
|
||||||
|
struct ft_relay_buttons relay_buttons; // the input relay's mouse buttons held on the seat
|
||||||
struct ft_controller_click controller_click;
|
struct ft_controller_click controller_click;
|
||||||
pid_t child;
|
pid_t child;
|
||||||
// Where typing goes: the screens after a click on one, Steam after a click on another
|
// Where typing goes: the screens after a click on one, Steam after a click on another
|
||||||
@@ -174,6 +176,9 @@ static void track_buffer(struct server *s, struct wlr_buffer *buffer) {
|
|||||||
|
|
||||||
// ---------------------------------------------------------------- screens
|
// ---------------------------------------------------------------- screens
|
||||||
|
|
||||||
|
static bool relay_can_press(struct server *s);
|
||||||
|
static void release_relay_buttons(struct server *s, const char *why);
|
||||||
|
|
||||||
// A video (or anything moving over a large area) on a screen you don't look at keeps the
|
// A video (or anything moving over a large area) on a screen you don't look at keeps the
|
||||||
// full frame rate (see frame_interval): its commits keep redrawing much of it, and keep
|
// full frame rate (see frame_interval): its commits keep redrawing much of it, and keep
|
||||||
// coming as fast as its rate lets them. A cursor blinking or a spinner turning redraws a
|
// coming as fast as its rate lets them. A cursor blinking or a spinner turning redraws a
|
||||||
@@ -247,7 +252,10 @@ static void screen_destroy(struct wl_listener *l, void *data) {
|
|||||||
wlr_log(WLR_INFO, "screen %d closed", sc->index + 1);
|
wlr_log(WLR_INFO, "screen %d closed", sc->index + 1);
|
||||||
if (sc->held) wlr_buffer_unlock(sc->held);
|
if (sc->held) wlr_buffer_unlock(sc->held);
|
||||||
ft_vr_screen_destroy(sc->index);
|
ft_vr_screen_destroy(sc->index);
|
||||||
if (sc->server->pointer_focus == sc) sc->server->pointer_focus = NULL;
|
if (sc->server->pointer_focus == sc) {
|
||||||
|
release_relay_buttons(sc->server, "its screen closed");
|
||||||
|
sc->server->pointer_focus = NULL;
|
||||||
|
}
|
||||||
if (sc->decoration) wl_list_remove(&sc->decoration_destroy.link);
|
if (sc->decoration) wl_list_remove(&sc->decoration_destroy.link);
|
||||||
sc->server->screens[sc->index] = NULL;
|
sc->server->screens[sc->index] = NULL;
|
||||||
wl_list_remove(&sc->commit.link);
|
wl_list_remove(&sc->commit.link);
|
||||||
@@ -346,6 +354,20 @@ static void handle_vr_event(const struct ft_event *e, void *data) {
|
|||||||
if (e->screen < 0 || e->screen >= MAX_SCREENS || !s->screens[e->screen]) return;
|
if (e->screen < 0 || e->screen >= MAX_SCREENS || !s->screens[e->screen]) return;
|
||||||
struct screen *sc = s->screens[e->screen];
|
struct screen *sc = s->screens[e->screen];
|
||||||
struct wlr_surface *surface = sc->toplevel->base->surface;
|
struct wlr_surface *surface = sc->toplevel->base->surface;
|
||||||
|
if (e->type == FT_FRONT) {
|
||||||
|
// A spin brought this panel to the front: typing goes to it, as after a click there,
|
||||||
|
// and ft-floatd makes its window (or the top one on a screen) KWin's active window.
|
||||||
|
wlr_seat_keyboard_notify_enter(s->seat, surface, NULL, 0, NULL);
|
||||||
|
s->keys_clicked = true;
|
||||||
|
char msg[32];
|
||||||
|
snprintf(msg, sizeof msg, "front %d", e->screen + 1);
|
||||||
|
struct sockaddr_un addr = {.sun_family = AF_UNIX};
|
||||||
|
const char name[] = "frametop_float";
|
||||||
|
memcpy(addr.sun_path + 1, name, sizeof name - 1);
|
||||||
|
sendto(s->relay_fd, msg, strlen(msg), MSG_DONTWAIT, (struct sockaddr *)&addr,
|
||||||
|
offsetof(struct sockaddr_un, sun_path) + 1 + sizeof name - 1);
|
||||||
|
return;
|
||||||
|
}
|
||||||
const uint32_t t = now_ms();
|
const uint32_t t = now_ms();
|
||||||
const double x = e->x / s->scale[e->screen], y = e->y / s->scale[e->screen]; // KWin's units
|
const double x = e->x / s->scale[e->screen], y = e->y / s->scale[e->screen]; // KWin's units
|
||||||
switch (e->type) {
|
switch (e->type) {
|
||||||
@@ -396,6 +418,7 @@ static void handle_vr_event(const struct ft_event *e, void *data) {
|
|||||||
break;
|
break;
|
||||||
case FT_LEAVE:
|
case FT_LEAVE:
|
||||||
if (s->pointer_focus == sc) {
|
if (s->pointer_focus == sc) {
|
||||||
|
release_relay_buttons(s, "the pointer left the screens");
|
||||||
wlr_seat_pointer_notify_clear_focus(s->seat);
|
wlr_seat_pointer_notify_clear_focus(s->seat);
|
||||||
s->pointer_focus = NULL;
|
s->pointer_focus = NULL;
|
||||||
}
|
}
|
||||||
@@ -485,6 +508,8 @@ static int tick(int fd, uint32_t mask, void *data) {
|
|||||||
const ssize_t got = read(fd, &expirations, sizeof expirations); // clears it; how many doesn't matter
|
const ssize_t got = read(fd, &expirations, sizeof expirations); // clears it; how many doesn't matter
|
||||||
(void)got;
|
(void)got;
|
||||||
ft_vr_poll(handle_vr_event, s);
|
ft_vr_poll(handle_vr_event, s);
|
||||||
|
if (s->relay_buttons.held && !relay_can_press(s))
|
||||||
|
release_relay_buttons(s, ft_vr_paused() ? "paused" : "its screen hid");
|
||||||
if (++s->ticks % 9 == 0) keys_update(s);
|
if (++s->ticks % 9 == 0) keys_update(s);
|
||||||
if (s->kb_close_at && s->ticks >= s->kb_close_at) {
|
if (s->kb_close_at && s->ticks >= s->kb_close_at) {
|
||||||
s->kb_close_at = 0;
|
s->kb_close_at = 0;
|
||||||
@@ -539,13 +564,49 @@ static void send_key(struct server *s, uint32_t code, int pressed) {
|
|||||||
wlr_seat_keyboard_notify_key(s->seat, ev.time_msec, code, ev.state);
|
wlr_seat_keyboard_notify_key(s->seat, ev.time_msec, code, ev.state);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The input relay's mouse buttons (relay-buttons.h) can be pressed: the pointer is on a
|
||||||
|
// screen that shows, and nothing's paused.
|
||||||
|
static bool relay_can_press(struct server *s) {
|
||||||
|
return s->pointer_focus && !ft_vr_paused() && ft_vr_screen_visible(s->pointer_focus->index);
|
||||||
|
}
|
||||||
|
|
||||||
|
// A mouse button from the input relay (pointer mode passes a mouse's side button through as
|
||||||
|
// a key, for Back): to the screen the pointer is on, like a laser's click, wherever typing goes.
|
||||||
|
static void relay_button(struct server *s, uint32_t code, bool pressed, char *reply, int size) {
|
||||||
|
if (!ft_relay_button(&s->relay_buttons, code, pressed, relay_can_press(s)))
|
||||||
|
return (void)snprintf(reply, size, pressed ? "ok no pointer, or held" : "ok not held");
|
||||||
|
wlr_seat_pointer_notify_button(s->seat, now_ms(), code,
|
||||||
|
pressed ? WL_POINTER_BUTTON_STATE_PRESSED : WL_POINTER_BUTTON_STATE_RELEASED);
|
||||||
|
wlr_seat_pointer_notify_frame(s->seat);
|
||||||
|
snprintf(reply, size, "ok");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ...and released by us when the pointer leaves the screens, its screen hides or closes, or
|
||||||
|
// everything pauses: before the pointer leaves that screen, so KWin gets the releases.
|
||||||
|
static void release_relay_buttons(struct server *s, const char *why) {
|
||||||
|
if (!s->relay_buttons.held) return;
|
||||||
|
uint32_t code;
|
||||||
|
while ((code = ft_relay_buttons_take(&s->relay_buttons))) {
|
||||||
|
wlr_seat_pointer_notify_button(s->seat, now_ms(), code, WL_POINTER_BUTTON_STATE_RELEASED);
|
||||||
|
wlr_log(WLR_INFO, "relay button %u released (%s)", code, why);
|
||||||
|
}
|
||||||
|
wlr_seat_pointer_notify_frame(s->seat);
|
||||||
|
}
|
||||||
|
|
||||||
// Keys from the input relay (physical keyboards): "key <evdev code> <1 press|0 release>".
|
// Keys from the input relay (physical keyboards): "key <evdev code> <1 press|0 release>".
|
||||||
// They go to the screen KWin has keyboard focus on (the last one clicked), while typing
|
// They go to the screen KWin has keyboard focus on (the last one clicked), while typing
|
||||||
// goes to the desktop (keys_update). The release of a key the desktop got the press for
|
// goes to the desktop (keys_update). The release of a key the desktop got the press for
|
||||||
// always goes through, or the key stays held there (a modifier held as typing moves to
|
// always goes through, or the key stays held there (a modifier held as typing moves to
|
||||||
// Steam would otherwise modify every key typed after it).
|
// Steam would otherwise modify every key typed after it). Mouse buttons go where the
|
||||||
|
// pointer is instead (relay_button), and codes that are neither go nowhere.
|
||||||
static void handle_key(struct server *s, uint32_t code, int value, char *reply, int size) {
|
static void handle_key(struct server *s, uint32_t code, int value, char *reply, int size) {
|
||||||
if (value == 2) return (void)snprintf(reply, size, "ok repeat ignored"); // KWin repeats itself
|
if (value == 2) return (void)snprintf(reply, size, "ok repeat ignored"); // KWin repeats itself
|
||||||
|
const enum ft_relay_key kind = ft_relay_key_kind(code);
|
||||||
|
if (kind == FT_RELAY_BUTTON) {
|
||||||
|
relay_button(s, code, value != 0, reply, size);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (kind == FT_RELAY_DROP) return (void)snprintf(reply, size, "ok not a key or mouse button");
|
||||||
if (value || !key_held(&s->keyboard, code)) {
|
if (value || !key_held(&s->keyboard, code)) {
|
||||||
if (!s->seat->keyboard_state.focused_surface) return (void)snprintf(reply, size, "ok no focus");
|
if (!s->seat->keyboard_state.focused_surface) return (void)snprintf(reply, size, "ok no focus");
|
||||||
if (!s->keys_desktop) return (void)snprintf(reply, size, "ok typing goes to Steam");
|
if (!s->keys_desktop) return (void)snprintf(reply, size, "ok typing goes to Steam");
|
||||||
@@ -795,7 +856,9 @@ static bool setup_dmabuf(struct server *s) {
|
|||||||
|
|
||||||
int main(int argc, char **argv) {
|
int main(int argc, char **argv) {
|
||||||
struct server s = {0};
|
struct server s = {0};
|
||||||
s.controller_click.threshold = 8;
|
// About 0.9 degrees on a 3.4 m wide 3440-pixel screen 2 m away; 8 (the first default,
|
||||||
|
// about 0.2 degrees) needed a very still hand to click (headset test 2026-10-03).
|
||||||
|
s.controller_click.threshold = 32;
|
||||||
for (int i = 0; i < MAX_SCREENS; ++i) s.scale[i] = 1;
|
for (int i = 0; i < MAX_SCREENS; ++i) s.scale[i] = 1;
|
||||||
s.kb_screen = -1;
|
s.kb_screen = -1;
|
||||||
s.rate[FT_FOCUSED] = 0, s.rate[FT_IN_VIEW] = 15, s.rate[FT_HIDDEN] = 1;
|
s.rate[FT_FOCUSED] = 0, s.rate[FT_IN_VIEW] = 15, s.rate[FT_HIDDEN] = 1;
|
||||||
|
|||||||
+15
-1
@@ -459,7 +459,21 @@ void Hide() {
|
|||||||
bool Shown() { return g_shown; }
|
bool Shown() { return g_shown; }
|
||||||
|
|
||||||
void SetLasers(bool on) {
|
void SetLasers(bool on) {
|
||||||
if (Create()) vr::VROverlay()->SetOverlayFlag(g_overlay, vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible, on);
|
static vr::VROverlayHandle_t set = vr::k_ulOverlayHandleInvalid; // the overlay `was` is for
|
||||||
|
static bool was = false;
|
||||||
|
if (!Create() || (set == g_overlay && was == on)) return;
|
||||||
|
set = g_overlay, was = on;
|
||||||
|
vr::VROverlay()->SetOverlayFlag(g_overlay, vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible, on);
|
||||||
|
}
|
||||||
|
|
||||||
|
bool Aimed(const vr::HmdMatrix34_t &laser) {
|
||||||
|
if (!g_shown) return false;
|
||||||
|
vr::VROverlayIntersectionParams_t in{};
|
||||||
|
in.eOrigin = vr::TrackingUniverseStanding;
|
||||||
|
in.vSource = {laser.m[0][3], laser.m[1][3], laser.m[2][3]};
|
||||||
|
in.vDirection = {-laser.m[0][2], -laser.m[1][2], -laser.m[2][2]};
|
||||||
|
vr::VROverlayIntersectionResults_t out{};
|
||||||
|
return vr::VROverlay()->ComputeOverlayIntersection(g_overlay, &in, &out);
|
||||||
}
|
}
|
||||||
|
|
||||||
void Poll(void (*handle)(const Event &, void *), void *data) {
|
void Poll(void (*handle)(const Event &, void *), void *data) {
|
||||||
|
|||||||
@@ -26,6 +26,8 @@ const vr::HmdMatrix34_t &Pose();
|
|||||||
void EndDragBy(uint32_t device);
|
void EndDragBy(uint32_t device);
|
||||||
// Controllers' lasers work the panel with the dashboard closed, like the screens'.
|
// Controllers' lasers work the panel with the dashboard closed, like the screens'.
|
||||||
void SetLasers(bool on);
|
void SetLasers(bool on);
|
||||||
|
// A laser (its pose, aiming along -z) points at the panel.
|
||||||
|
bool Aimed(const vr::HmdMatrix34_t &laser);
|
||||||
// The panel's input: key presses and releases, and Closed for its Close key.
|
// The panel's input: key presses and releases, and Closed for its Close key.
|
||||||
void Poll(void (*handle)(const Event &, void *), void *data);
|
void Poll(void (*handle)(const Event &, void *), void *data);
|
||||||
void Destroy();
|
void Destroy();
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
#ifndef FT_RELAY_BUTTONS_H
|
||||||
|
#define FT_RELAY_BUTTONS_H
|
||||||
|
#include <stdbool.h>
|
||||||
|
#include <stdint.h>
|
||||||
|
#include <linux/input-event-codes.h>
|
||||||
|
// Keys from the input relay ("key <code> <value>" on the control socket). Keys go where
|
||||||
|
// typing goes (compositor.c, handle_key). Mouse buttons, a mouse's side buttons that pointer
|
||||||
|
// mode passes through as keys, go where the pointer is, like a laser's click: a press needs
|
||||||
|
// the pointer on a visible screen, nothing paused, and that button not held already, and a
|
||||||
|
// release goes whenever its press went, whatever changed since. When the pointer leaves the
|
||||||
|
// screens, or its screen hides or closes, or everything pauses, ft-screens releases the held
|
||||||
|
// ones itself (ft_relay_buttons_take), and the relay's releases later find nothing held.
|
||||||
|
// wlroots counts presses per button, so one release lost would swallow that button for good
|
||||||
|
// (a laser's left click too), and KWin would keep it held.
|
||||||
|
enum ft_relay_key { FT_RELAY_DROP, FT_RELAY_KEY, FT_RELAY_BUTTON };
|
||||||
|
|
||||||
|
// What a code from the relay is: a key for the keyboard (below BTN_MISC, and KEY_OK up to
|
||||||
|
// BTN_TRIGGER_HAPPY: xkeyboard-config names keycodes above 255 too), a mouse button for the
|
||||||
|
// pointer (BTN_MOUSE..BTN_TASK), or nothing to send (joystick, gamepad and digitizer
|
||||||
|
// buttons, BTN_TRIGGER_HAPPY and up).
|
||||||
|
static inline enum ft_relay_key ft_relay_key_kind(uint32_t code) {
|
||||||
|
if (code < BTN_MISC || (code >= KEY_OK && code < BTN_TRIGGER_HAPPY)) return FT_RELAY_KEY;
|
||||||
|
if (code >= BTN_MOUSE && code <= BTN_TASK) return FT_RELAY_BUTTON;
|
||||||
|
return FT_RELAY_DROP;
|
||||||
|
}
|
||||||
|
|
||||||
|
struct ft_relay_buttons {
|
||||||
|
uint8_t held; // pressed on the seat and not released, a bit each (1 << (code - BTN_MOUSE))
|
||||||
|
};
|
||||||
|
|
||||||
|
// A relay button pressed or released: true if it goes to the seat. can_press: the pointer
|
||||||
|
// is on a visible screen, and nothing's paused.
|
||||||
|
static inline bool ft_relay_button(struct ft_relay_buttons *b, uint32_t code, bool pressed, bool can_press) {
|
||||||
|
if (ft_relay_key_kind(code) != FT_RELAY_BUTTON) return false;
|
||||||
|
const uint8_t bit = (uint8_t)(1u << (code - BTN_MOUSE));
|
||||||
|
if (pressed) {
|
||||||
|
if (!can_press || (b->held & bit)) return false;
|
||||||
|
b->held |= bit;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
if (!(b->held & bit)) return false;
|
||||||
|
b->held &= (uint8_t)~bit;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
// The next held button, no longer held here (0: none), for ft-screens to release itself.
|
||||||
|
static inline uint32_t ft_relay_buttons_take(struct ft_relay_buttons *b) {
|
||||||
|
for (uint32_t i = 0; i <= BTN_TASK - BTN_MOUSE; ++i)
|
||||||
|
if (b->held & (1u << i)) {
|
||||||
|
b->held &= (uint8_t)~(1u << i);
|
||||||
|
return BTN_MOUSE + i;
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
#endif
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
#include <assert.h>
|
||||||
|
#include "../relay-buttons.h"
|
||||||
|
int main(void) {
|
||||||
|
// What goes where: keys to the keyboard, mouse buttons to the pointer, the rest nowhere.
|
||||||
|
assert(ft_relay_key_kind(KEY_A) == FT_RELAY_KEY && ft_relay_key_kind(KEY_PLAYPAUSE) == FT_RELAY_KEY);
|
||||||
|
assert(ft_relay_key_kind(BTN_LEFT) == FT_RELAY_BUTTON && ft_relay_key_kind(BTN_SIDE) == FT_RELAY_BUTTON);
|
||||||
|
assert(ft_relay_key_kind(BTN_EXTRA) == FT_RELAY_BUTTON && ft_relay_key_kind(BTN_TASK) == FT_RELAY_BUTTON);
|
||||||
|
for (uint32_t c = BTN_MISC; c < BTN_MOUSE; ++c) assert(ft_relay_key_kind(c) == FT_RELAY_DROP);
|
||||||
|
for (uint32_t c = BTN_TASK + 1; c < KEY_OK; ++c) assert(ft_relay_key_kind(c) == FT_RELAY_DROP); // joystick, gamepad, digitizer
|
||||||
|
assert(ft_relay_key_kind(KEY_OK) == FT_RELAY_KEY && ft_relay_key_kind(BTN_TRIGGER_HAPPY - 1) == FT_RELAY_KEY);
|
||||||
|
assert(ft_relay_key_kind(BTN_TRIGGER_HAPPY) == FT_RELAY_DROP && ft_relay_key_kind(KEY_MAX) == FT_RELAY_DROP);
|
||||||
|
assert(ft_relay_key_kind(KEY_MAX + 1) == FT_RELAY_DROP);
|
||||||
|
|
||||||
|
struct ft_relay_buttons b = {0};
|
||||||
|
// No pointer on a visible screen (or paused): the press is dropped, and so is its release.
|
||||||
|
assert(!ft_relay_button(&b, BTN_SIDE, true, false) && !b.held);
|
||||||
|
assert(!ft_relay_button(&b, BTN_SIDE, false, true));
|
||||||
|
// A release without a press is nothing, whatever the pointer does.
|
||||||
|
assert(!ft_relay_button(&b, BTN_EXTRA, false, false) && !ft_relay_button(&b, BTN_EXTRA, false, true));
|
||||||
|
// A press on a screen, then typing moves to Steam or the pointer leaves: the release still goes.
|
||||||
|
assert(ft_relay_button(&b, BTN_SIDE, true, true) && b.held == 1u << (BTN_SIDE - BTN_MOUSE));
|
||||||
|
assert(!ft_relay_button(&b, BTN_SIDE, true, true)); // a second press of a held button
|
||||||
|
assert(ft_relay_button(&b, BTN_SIDE, false, false) && !b.held);
|
||||||
|
assert(!ft_relay_button(&b, BTN_SIDE, false, false)); // once
|
||||||
|
// Not a mouse button: never the pointer's.
|
||||||
|
assert(!ft_relay_button(&b, KEY_A, true, true) && !ft_relay_button(&b, BTN_JOYSTICK, true, true));
|
||||||
|
assert(!ft_relay_button(&b, BTN_TRIGGER_HAPPY, true, true) && !b.held);
|
||||||
|
|
||||||
|
// ft-screens lets go itself (the pointer left, its screen hid, a pause): each held button
|
||||||
|
// once, then the relay's own releases find nothing held.
|
||||||
|
assert(ft_relay_button(&b, BTN_LEFT, true, true) && ft_relay_button(&b, BTN_TASK, true, true));
|
||||||
|
assert(ft_relay_buttons_take(&b) == BTN_LEFT && ft_relay_buttons_take(&b) == BTN_TASK);
|
||||||
|
assert(ft_relay_buttons_take(&b) == 0 && !b.held);
|
||||||
|
assert(!ft_relay_button(&b, BTN_LEFT, false, true) && !ft_relay_button(&b, BTN_TASK, false, true));
|
||||||
|
// ...and the next press goes through again.
|
||||||
|
assert(ft_relay_button(&b, BTN_LEFT, true, true) && ft_relay_button(&b, BTN_LEFT, false, true));
|
||||||
|
}
|
||||||
+308
-43
@@ -12,9 +12,7 @@
|
|||||||
// - a resize tab on the bottom right corner: drag it to set the width (the height
|
// - a resize tab on the bottom right corner: drag it to set the width (the height
|
||||||
// follows the screen's resolution).
|
// follows the screen's resolution).
|
||||||
// - a reset button left of the bar: every screen back in its layout, around where you
|
// - a reset button left of the bar: every screen back in its layout, around where you
|
||||||
// are now (`ft-layout apply`, like Meta+Shift+R). Where the screens leave the
|
// are now (`ft-layout apply`, like Meta+Shift+R).
|
||||||
// controllers to a VR game, aiming a controller at it turns SteamVR's laser mouse on for
|
|
||||||
// that button alone (UpdateResetLaser), so it can be clicked in a game.
|
|
||||||
// The controls are translucent, like SteamVR's own, and brighten under a laser. They
|
// The controls are translucent, like SteamVR's own, and brighten under a laser. They
|
||||||
// are invisible until a laser (a controller's, or the 3D mouse's) lands on or passes very close to
|
// are invisible until a laser (a controller's, or the 3D mouse's) lands on or passes very close to
|
||||||
// one of them (UpdateControls).
|
// one of them (UpdateControls).
|
||||||
@@ -44,7 +42,10 @@
|
|||||||
// default it's off while a game (a scene app) runs: the screens stay up over the game,
|
// default it's off while a game (a scene app) runs: the screens stay up over the game,
|
||||||
// the controllers stay in it, and the 3D mouse (its own laser mode) or the dashboard
|
// the controllers stay in it, and the 3D mouse (its own laser mode) or the dashboard
|
||||||
// works the screens. Modes: always, outside_games (default), dashboard (never on its
|
// works the screens. Modes: always, outside_games (default), dashboard (never on its
|
||||||
// own; also for flatscreen games, which aren't scene apps).
|
// own; also for flatscreen games, which aren't scene apps). Where the mode leaves the
|
||||||
|
// controllers to the game, pointing a controller at a panel (a screen, a floating window
|
||||||
|
// and its popups, their controls, the keyboard) turns the laser on for it until you
|
||||||
|
// point away, like SteamVR's own floating windows (UpdateAim).
|
||||||
// - during a VR game the screens hide unless the dashboard is open (g_inGames, default),
|
// - during a VR game the screens hide unless the dashboard is open (g_inGames, default),
|
||||||
// or stay visible over it; the hotkey still shows them.
|
// or stay visible over it; the hotkey still shows them.
|
||||||
// - paused ("pause on", from the input relay when Frametop pauses for a VR game,
|
// - paused ("pause on", from the input relay when Frametop pauses for a VR game,
|
||||||
@@ -189,8 +190,12 @@ Mat TipOffset(vr::TrackedDeviceIndex_t dev) {
|
|||||||
Tip tip{model, Identity(), false, now};
|
Tip tip{model, Identity(), false, now};
|
||||||
vr::RenderModel_ControllerMode_State_t mode{};
|
vr::RenderModel_ControllerMode_State_t mode{};
|
||||||
vr::RenderModel_ComponentState_t state{};
|
vr::RenderModel_ComponentState_t state{};
|
||||||
if (model[0] && vr::VRRenderModels()->GetComponentStateForDevicePath(model, vr::k_pch_Controller_Component_Tip,
|
// GetComponentState, not GetComponentStateForDevicePath: without an input source handle
|
||||||
vr::k_ulInvalidInputValueHandle, &mode, &state))
|
// the latter fails for every component while a VR game runs, and the rays came from the
|
||||||
|
// pose, 40 degrees above the laser. The tip doesn't move with the buttons.
|
||||||
|
vr::VRControllerState_t buttons{};
|
||||||
|
if (model[0] && vr::VRRenderModels()->GetComponentState(model, vr::k_pch_Controller_Component_Tip, &buttons, &mode,
|
||||||
|
&state))
|
||||||
tip.offset = state.mTrackingToComponentLocal, tip.found = true;
|
tip.offset = state.mTrackingToComponentLocal, tip.found = true;
|
||||||
cache[dev] = tip;
|
cache[dev] = tip;
|
||||||
return tip.offset;
|
return tip.offset;
|
||||||
@@ -237,7 +242,7 @@ constexpr double kRollSnap = 2.5; // degrees from level where rolling snaps l
|
|||||||
constexpr double kRollStep = 5; // degrees per scroll notch on the roll button
|
constexpr double kRollStep = 5; // degrees per scroll notch on the roll button
|
||||||
constexpr float kChromeIdle = 0.55f; // the controls' opacity without a laser on them
|
constexpr float kChromeIdle = 0.55f; // the controls' opacity without a laser on them
|
||||||
constexpr long kControlsLinger = 35; // ticks (~0.4 s) the controls stay after a laser leaves
|
constexpr long kControlsLinger = 35; // ticks (~0.4 s) the controls stay after a laser leaves
|
||||||
constexpr long kResetLinger = 45; // ticks (~0.5 s) the reset button keeps the laser mouse on
|
constexpr long kAimLinger = 25; // ticks (~0.3 s) a panel keeps the laser on after the aim leaves it
|
||||||
long g_tick = 0; // ft_vr_poll calls
|
long g_tick = 0; // ft_vr_poll calls
|
||||||
bool g_vr = false; // connected to SteamVR (ft-screens --no-vr runs without it)
|
bool g_vr = false; // connected to SteamVR (ft-screens --no-vr runs without it)
|
||||||
constexpr vr::TrackedDeviceIndex_t kNone = vr::k_unTrackedDeviceIndexInvalid;
|
constexpr vr::TrackedDeviceIndex_t kNone = vr::k_unTrackedDeviceIndexInvalid;
|
||||||
@@ -275,8 +280,7 @@ struct Screen {
|
|||||||
float controls = 0; // the controls' fade, 0 (hidden) .. 1
|
float controls = 0; // the controls' fade, 0 (hidden) .. 1
|
||||||
bool controlsUp = false; // the controls' overlays are shown
|
bool controlsUp = false; // the controls' overlays are shown
|
||||||
long nearUntil = 0; // a laser was near the controls until this tick
|
long nearUntil = 0; // a laser was near the controls until this tick
|
||||||
long resetNearUntil = 0; // a hand controller aimed at the reset button until this tick
|
long aimUntil = 0; // a hand controller pointed at it until this tick (UpdateAim)
|
||||||
bool resetLaser = false; // the reset button has MakeOverlaysInteractiveIfVisible
|
|
||||||
vr::TrackedDeviceIndex_t pinTarget = kNone; // moving: rides on this controller when let go
|
vr::TrackedDeviceIndex_t pinTarget = kNone; // moving: rides on this controller when let go
|
||||||
vr::TrackedDeviceIndex_t onWrist = kNone; // moving: the laser is in this controller's ring
|
vr::TrackedDeviceIndex_t onWrist = kNone; // moving: the laser is in this controller's ring
|
||||||
bool barLit = false;
|
bool barLit = false;
|
||||||
@@ -910,26 +914,20 @@ void UpdateAttention() {
|
|||||||
|
|
||||||
|
|
||||||
// Controllers' lasers on the screens (see the top): the flag follows the mode and whether a
|
// Controllers' lasers on the screens (see the top): the flag follows the mode and whether a
|
||||||
// VR game runs.
|
// VR game runs, and where the mode leaves the controllers to the game, whether one points at
|
||||||
|
// the panel (UpdateAim).
|
||||||
|
bool LasersByMode() { return g_lasers == Lasers::Always || (g_lasers == Lasers::OutsideGames && !g_gameRunning); }
|
||||||
|
long g_keyboardAimUntil = 0;
|
||||||
|
|
||||||
void UpdateLasers() {
|
void UpdateLasers() {
|
||||||
const bool want = g_lasers == Lasers::Always || (g_lasers == Lasers::OutsideGames && !g_gameRunning);
|
const bool byMode = LasersByMode();
|
||||||
for (auto &[i, s] : g_screens) {
|
for (auto &[i, s] : g_screens) {
|
||||||
|
const bool want = byMode || (s.visible && g_tick < s.aimUntil);
|
||||||
if (s.lasers == want) continue;
|
if (s.lasers == want) continue;
|
||||||
s.lasers = want;
|
s.lasers = want;
|
||||||
vr::VROverlay()->SetOverlayFlag(s.overlay, vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible, want);
|
vr::VROverlay()->SetOverlayFlag(s.overlay, vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible, want);
|
||||||
}
|
}
|
||||||
}
|
if (keyboard::Shown()) keyboard::SetLasers(byMode || g_tick < g_keyboardAimUntil);
|
||||||
|
|
||||||
// The reset button in a VR game (or the dashboard mode), where the screens don't keep
|
|
||||||
// SteamVR's laser mouse on: aiming a hand controller at it turns the laser mouse on for that
|
|
||||||
// button alone, so the trigger clicks it, and the game gets the controllers back
|
|
||||||
// kResetLinger ticks after the aim leaves it (a wider zone than the one that turns it on).
|
|
||||||
void UpdateResetLaser(Screen &s) {
|
|
||||||
const bool want = s.resetButton != vr::k_ulOverlayHandleInvalid && s.visible && !s.lasers &&
|
|
||||||
g_tick < s.resetNearUntil;
|
|
||||||
if (want == s.resetLaser) return;
|
|
||||||
s.resetLaser = want;
|
|
||||||
vr::VROverlay()->SetOverlayFlag(s.resetButton, vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible, want);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// The distance from a laser's line to a point ahead of it, or -1 when it's behind.
|
// The distance from a laser's line to a point ahead of it, or -1 when it's behind.
|
||||||
@@ -947,11 +945,10 @@ double RayDistance(const Mat &d, const Mat &c) {
|
|||||||
// kControlsLinger ticks after it leaves, and while in use.
|
// kControlsLinger ticks after it leaves, and while in use.
|
||||||
void UpdateControls() {
|
void UpdateControls() {
|
||||||
std::vector<Mat> lasers;
|
std::vector<Mat> lasers;
|
||||||
std::vector<bool> hands; // the laser is a hand controller's (not the 3D mouse's)
|
|
||||||
for (vr::TrackedDeviceIndex_t i = 1; i < vr::k_unMaxTrackedDeviceCount; ++i) {
|
for (vr::TrackedDeviceIndex_t i = 1; i < vr::k_unMaxTrackedDeviceCount; ++i) {
|
||||||
Mat d;
|
Mat d;
|
||||||
if (vr::VRSystem()->GetTrackedDeviceClass(i) == vr::TrackedDeviceClass_Controller && LaserPose(i, &d))
|
if (vr::VRSystem()->GetTrackedDeviceClass(i) == vr::TrackedDeviceClass_Controller && LaserPose(i, &d))
|
||||||
lasers.push_back(d), hands.push_back(IsHandController(i));
|
lasers.push_back(d);
|
||||||
}
|
}
|
||||||
for (auto &[i, s] : g_screens) {
|
for (auto &[i, s] : g_screens) {
|
||||||
Mat p;
|
Mat p;
|
||||||
@@ -977,16 +974,7 @@ void UpdateControls() {
|
|||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (s.resetButton != vr::k_ulOverlayHandleInvalid && !s.lasers) {
|
|
||||||
const Mat c = Mul(p, offsets[6]);
|
|
||||||
const double aim = s.grip * (s.resetLaser ? 2.0 : 0.9);
|
|
||||||
for (size_t k = 0; k < lasers.size(); ++k) {
|
|
||||||
const double r = RayDistance(lasers[k], c);
|
|
||||||
if (hands[k] && r >= 0 && r <= aim) s.resetNearUntil = g_tick + kResetLinger;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
UpdateResetLaser(s);
|
|
||||||
const bool inUse = s.drag != Drag::None || std::any_of(std::begin(s.hover), std::end(s.hover), [](bool h) { return h; });
|
const bool inUse = s.drag != Drag::None || std::any_of(std::begin(s.hover), std::end(s.hover), [](bool h) { return h; });
|
||||||
const bool want = s.visible && (inUse || g_tick < s.nearUntil);
|
const bool want = s.visible && (inUse || g_tick < s.nearUntil);
|
||||||
// The controls stay shown while their screen is, just fully transparent when not
|
// The controls stay shown while their screen is, just fully transparent when not
|
||||||
@@ -1020,19 +1008,21 @@ void SendFloat(const std::string &msg) {
|
|||||||
std::printf("to ft-floatd: %s\n", msg.c_str());
|
std::printf("to ft-floatd: %s\n", msg.c_str());
|
||||||
}
|
}
|
||||||
|
|
||||||
// A new panel: the pointer helper reads SteamVR's list of panels only every 20 s, so it's told
|
// To the pointer helper (@ft_pointer_helper), from an unbound socket.
|
||||||
// at once ("overlay <key>"), or the mouse couldn't click a menu until then.
|
void SendPointer(const std::string &msg) {
|
||||||
void AnnounceOverlay(const char *key) {
|
|
||||||
static const int fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
|
static const int fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0);
|
||||||
sockaddr_un addr{};
|
sockaddr_un addr{};
|
||||||
addr.sun_family = AF_UNIX;
|
addr.sun_family = AF_UNIX;
|
||||||
const char name[] = "ft_pointer_helper";
|
const char name[] = "ft_pointer_helper";
|
||||||
std::memcpy(addr.sun_path + 1, name, sizeof name - 1);
|
std::memcpy(addr.sun_path + 1, name, sizeof name - 1);
|
||||||
const std::string msg = std::string("overlay ") + key;
|
|
||||||
sendto(fd, msg.data(), msg.size(), MSG_DONTWAIT, reinterpret_cast<sockaddr *>(&addr),
|
sendto(fd, msg.data(), msg.size(), MSG_DONTWAIT, reinterpret_cast<sockaddr *>(&addr),
|
||||||
socklen_t(offsetof(sockaddr_un, sun_path) + 1 + sizeof name - 1));
|
socklen_t(offsetof(sockaddr_un, sun_path) + 1 + sizeof name - 1));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A new panel: the pointer helper reads SteamVR's list of panels only every 20 s, so it's told
|
||||||
|
// at once ("overlay <key>"), or the mouse couldn't click a menu until then.
|
||||||
|
void AnnounceOverlay(const char *key) { SendPointer(std::string("overlay ") + key); }
|
||||||
|
|
||||||
// Where a device's ray meets the screen's plane, in the screen's x (right) and y (up),
|
// Where a device's ray meets the screen's plane, in the screen's x (right) and y (up),
|
||||||
// metres from its centre.
|
// metres from its centre.
|
||||||
bool RayOnPlane(const Mat &p, const Mat &d, double *x, double *y) {
|
bool RayOnPlane(const Mat &p, const Mat &d, double *x, double *y) {
|
||||||
@@ -1156,23 +1146,29 @@ void EndDrag(Screen &s) {
|
|||||||
ApplyAlpha(s);
|
ApplyAlpha(s);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Run `ft-layout <cmd>` in the background, logging to /tmp/frametop-layout.log.
|
// Run `ft-layout <cmd>` in the background, logging to the host's runtime directory
|
||||||
|
// (XDG_RUNTIME_DIR, not the nested desktop's; else /tmp with O_NOFOLLOW: a symlink there
|
||||||
|
// must not be followed). The desktop start truncates the same log.
|
||||||
void RunLayout(const char *cmd) {
|
void RunLayout(const char *cmd) {
|
||||||
char exe[PATH_MAX];
|
char exe[PATH_MAX];
|
||||||
if (!realpath("/proc/self/exe", exe)) return;
|
if (!realpath("/proc/self/exe", exe)) return;
|
||||||
|
const char *runtime = std::getenv("XDG_RUNTIME_DIR");
|
||||||
|
std::string logname = std::string(runtime && *runtime ? runtime : "/tmp") + "/frametop-layout.log";
|
||||||
std::string layout(exe); // <repo>/screens/build/ft-screens -> <repo>/layout/ft-layout
|
std::string layout(exe); // <repo>/screens/build/ft-screens -> <repo>/layout/ft-layout
|
||||||
for (int up = 0; up < 3 && layout.rfind('/') != std::string::npos; ++up) layout.resize(layout.rfind('/'));
|
for (int up = 0; up < 3 && layout.rfind('/') != std::string::npos; ++up) layout.resize(layout.rfind('/'));
|
||||||
layout += "/layout/ft-layout";
|
layout += "/layout/ft-layout";
|
||||||
posix_spawn_file_actions_t io;
|
posix_spawn_file_actions_t io;
|
||||||
posix_spawn_file_actions_init(&io);
|
posix_spawn_file_actions_init(&io);
|
||||||
posix_spawn_file_actions_addopen(&io, 0, "/dev/null", O_RDONLY, 0);
|
posix_spawn_file_actions_addopen(&io, 0, "/dev/null", O_RDONLY, 0);
|
||||||
posix_spawn_file_actions_addopen(&io, 1, "/tmp/frametop-layout.log", O_WRONLY | O_CREAT | O_APPEND, 0644);
|
posix_spawn_file_actions_addopen(&io, 1, logname.c_str(), O_WRONLY | O_CREAT | O_APPEND | O_NOFOLLOW, 0644);
|
||||||
posix_spawn_file_actions_adddup2(&io, 1, 2);
|
posix_spawn_file_actions_adddup2(&io, 1, 2);
|
||||||
std::string arg(cmd);
|
std::string arg(cmd);
|
||||||
char *argv[] = {layout.data(), arg.data(), nullptr};
|
char *argv[] = {layout.data(), arg.data(), nullptr};
|
||||||
pid_t pid; // reaped by the compositor's SIGCHLD handler
|
pid_t pid; // reaped by the compositor's SIGCHLD handler
|
||||||
if (posix_spawn(&pid, layout.c_str(), &io, nullptr, argv, environ) != 0)
|
// A log that can't be opened (a symlink or a directory in its place) fails the spawn too.
|
||||||
std::printf("can't run %s\n", layout.c_str());
|
const int rc = posix_spawn(&pid, layout.c_str(), &io, nullptr, argv, environ);
|
||||||
|
if (rc != 0)
|
||||||
|
std::printf("can't run %s (log %s): %s\n", layout.c_str(), logname.c_str(), std::strerror(rc));
|
||||||
posix_spawn_file_actions_destroy(&io);
|
posix_spawn_file_actions_destroy(&io);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1419,6 +1415,7 @@ struct Press {
|
|||||||
double x = 0, y = 0; // ...and where on it, in buffer pixels
|
double x = 0, y = 0; // ...and where on it, in buffer pixels
|
||||||
double distance = 1; // from the laser's start to the last panel it met
|
double distance = 1; // from the laser's start to the last panel it met
|
||||||
long upAt = -1; // the pointer helper saw left come up: release it at this tick
|
long upAt = -1; // the pointer helper saw left come up: release it at this tick
|
||||||
|
int64_t idleSince = -1; // the pressing controller has held nothing since (ms, ReleaseStuck)
|
||||||
};
|
};
|
||||||
Press g_press;
|
Press g_press;
|
||||||
vr::VROverlayHandle_t g_catcher = vr::k_ulOverlayHandleInvalid;
|
vr::VROverlayHandle_t g_catcher = vr::k_ulOverlayHandleInvalid;
|
||||||
@@ -1455,6 +1452,47 @@ void ReleaseAway(uint32_t button, void (*handle)(const struct ft_event *, void *
|
|||||||
handle(&e, data);
|
handle(&e, data);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Whether a hand controller holds anything (a button down): 1 yes, 0 no, -1 unknown. SteamVR
|
||||||
|
// answers overlay apps only while a VR game runs (checked 2026-10-04); outside games it can't
|
||||||
|
// say. The Frame controller's axes have no types, so the buttons are all there is.
|
||||||
|
int ControllerHolds(vr::TrackedDeviceIndex_t dev) {
|
||||||
|
vr::VRControllerState_t st{};
|
||||||
|
if (!vr::VRSystem()->GetControllerState(dev, &st, sizeof st)) return -1;
|
||||||
|
return st.ulButtonPressed ? 1 : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Releases SteamVR never sends. Pausing, or hiding the screen a button went down on, takes the
|
||||||
|
// laser off it mid-click (the pause gesture's second thumbstick click does that), and SteamVR's
|
||||||
|
// laser mouse then forgets the button: no release comes, the catcher stayed up, and the game
|
||||||
|
// never got its controllers back (2026-10-04). So a held button is released here when it has
|
||||||
|
// no visible screen to come up on, or when its hand controller has held nothing for kStuckMs
|
||||||
|
// (only known during VR games, where the lost release took the game's controllers).
|
||||||
|
// The pointer helper's virtual controller has its own word for that ("up").
|
||||||
|
constexpr int64_t kStuckMs = 1000;
|
||||||
|
void ReleaseStuck(void (*handle)(const struct ft_event *, void *), void *data) {
|
||||||
|
if (!g_press.buttons) {
|
||||||
|
g_press.idleSince = -1;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const char *why = nullptr;
|
||||||
|
const auto it = g_screens.find(g_press.screen);
|
||||||
|
if (g_paused) why = "paused";
|
||||||
|
else if (g_press.screen >= 0 && (it == g_screens.end() || !it->second.visible)) why = "its screen hid";
|
||||||
|
else if (IsHandController(g_press.device)) {
|
||||||
|
const int64_t now = NowMs();
|
||||||
|
if (ControllerHolds(g_press.device) != 0) g_press.idleSince = -1;
|
||||||
|
else if (g_press.idleSince < 0) g_press.idleSince = now;
|
||||||
|
else if (now - g_press.idleSince >= kStuckMs) why = "the controller holds nothing";
|
||||||
|
}
|
||||||
|
if (!why) return;
|
||||||
|
std::printf("a held button can't come up on a screen (%s): released\n", why);
|
||||||
|
const vr::TrackedDeviceIndex_t dev = g_press.device;
|
||||||
|
for (uint32_t b = 0; b < 32; ++b)
|
||||||
|
if (g_press.buttons & (1u << b)) ReleaseAway(BTN_LEFT + b, handle, data);
|
||||||
|
g_press.buttons = 0, g_press.upAt = -1, g_press.idleSince = -1;
|
||||||
|
EndDragsBy(dev);
|
||||||
|
}
|
||||||
|
|
||||||
void ShowCatcher(bool on) {
|
void ShowCatcher(bool on) {
|
||||||
if (on == g_catcherShown || g_catcher == vr::k_ulOverlayHandleInvalid) return;
|
if (on == g_catcherShown || g_catcher == vr::k_ulOverlayHandleInvalid) return;
|
||||||
g_catcherShown = on;
|
g_catcherShown = on;
|
||||||
@@ -1500,6 +1538,137 @@ Screen *Find(int one_based) {
|
|||||||
return it == g_screens.end() ? nullptr : &it->second;
|
return it == g_screens.end() ? nullptr : &it->second;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- the lazy susan
|
||||||
|
// "spin next|prev|<degrees>": the panels in the room (screens and floating windows, not
|
||||||
|
// pinned ones) turn together about a vertical axis through your head, so the next panel to
|
||||||
|
// your right (next) or left (prev) glides to straight ahead, or the ring turns by that many
|
||||||
|
// degrees (positive turns it left, like next). Their arrangement stays as it is: the room
|
||||||
|
// turns instead of you. A spin that arrives during one adds to it, from where the panels are
|
||||||
|
// headed, so quick taps carry on smoothly. Grabbing a panel, or a command that places it,
|
||||||
|
// takes it out of the spin where it is. The 3D mouse's pointer goes to straight ahead.
|
||||||
|
constexpr double kSpinSeconds = 0.3; // how long a spin takes
|
||||||
|
constexpr double kSpinAhead = 8; // degrees: a panel this near straight ahead is the current one
|
||||||
|
constexpr double kSpinFocus = 30; // degrees: when a spin settles, the panel this near ahead gets typing
|
||||||
|
|
||||||
|
struct Spin {
|
||||||
|
bool on = false;
|
||||||
|
double cx = 0, cz = 0; // the axis
|
||||||
|
double from = 0, to = 0; // radians, turned from the poses in base (positive: to the left)
|
||||||
|
Clock::time_point start;
|
||||||
|
std::map<int, Mat> base; // index -> its pose before the spin
|
||||||
|
int front = -1; // settled: this panel came to the front (ft_vr_poll reports it)
|
||||||
|
} g_spin;
|
||||||
|
|
||||||
|
// p turned a radians about the vertical axis through (cx, cz); positive turns it to the left.
|
||||||
|
Mat Turned(const Mat &p, double a, double cx, double cz) {
|
||||||
|
const double c = std::cos(a), s = std::sin(a);
|
||||||
|
Mat r = Identity();
|
||||||
|
r.m[0][0] = float(c), r.m[0][2] = float(s);
|
||||||
|
r.m[2][0] = float(-s), r.m[2][2] = float(c);
|
||||||
|
r.m[0][3] = float(cx - c * cx - s * cz);
|
||||||
|
r.m[2][3] = float(cz + s * cx - c * cz);
|
||||||
|
return Mul(r, p);
|
||||||
|
}
|
||||||
|
|
||||||
|
bool Spinnable(const Screen &s) { return s.pinned == kNone && (!s.floating || s.floatOn) && s.drag == Drag::None; }
|
||||||
|
|
||||||
|
double SpinNow() {
|
||||||
|
if (!g_spin.on) return g_spin.to;
|
||||||
|
const double t = std::min(1.0, std::chrono::duration<double>(Clock::now() - g_spin.start).count() / kSpinSeconds);
|
||||||
|
return g_spin.from + (g_spin.to - g_spin.from) * t * t * (3 - 2 * t);
|
||||||
|
}
|
||||||
|
|
||||||
|
void UpdateSpin() {
|
||||||
|
if (!g_spin.on) return;
|
||||||
|
const double a = SpinNow();
|
||||||
|
const bool done = Clock::now() - g_spin.start >= std::chrono::duration<double>(kSpinSeconds);
|
||||||
|
for (auto it = g_spin.base.begin(); it != g_spin.base.end();) {
|
||||||
|
auto s = g_screens.find(it->first);
|
||||||
|
if (s == g_screens.end() || !Spinnable(s->second)) {
|
||||||
|
it = g_spin.base.erase(it); // grabbed, pinned, or gone: it stays where it is now
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
SetAbsolute(s->second, Turned(it->second, a, g_spin.cx, g_spin.cz));
|
||||||
|
++it;
|
||||||
|
}
|
||||||
|
if (done) {
|
||||||
|
// The panel now nearest straight ahead (of where you face) gets typing and the active window.
|
||||||
|
Mat head;
|
||||||
|
double nearest = kSpinFocus;
|
||||||
|
if (DevicePose(vr::k_unTrackedDeviceIndex_Hmd, &head)) {
|
||||||
|
for (const auto &[i, p] : g_spin.base) {
|
||||||
|
const auto it = g_screens.find(i);
|
||||||
|
Mat q;
|
||||||
|
if (it == g_screens.end() || !it->second.visible || !ScreenPose(it->second, &q)) continue;
|
||||||
|
double f[3] = {-head.m[0][2], 0, -head.m[2][2]};
|
||||||
|
double d[3] = {q.m[0][3] - head.m[0][3], 0, q.m[2][3] - head.m[2][3]};
|
||||||
|
const double fl = std::sqrt(Dot3(f, f)), dl = std::sqrt(Dot3(d, d));
|
||||||
|
if (fl < 1e-6 || dl < 1e-6) continue;
|
||||||
|
const double a = std::acos(std::clamp(Dot3(f, d) / (fl * dl), -1.0, 1.0)) * 180 / M_PI;
|
||||||
|
if (a < nearest) nearest = a, g_spin.front = i;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
g_spin.on = false;
|
||||||
|
g_spin.base.clear();
|
||||||
|
ArrangeDesktopSoon(); // KWin's outputs follow where the screens are now
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void SpinCommand(const char *arg, char *reply, int size) {
|
||||||
|
Mat head;
|
||||||
|
if (!DevicePose(vr::k_unTrackedDeviceIndex_Hmd, &head))
|
||||||
|
return (void)std::snprintf(reply, size, "error no head pose (headset off?)");
|
||||||
|
if (g_spin.on) {
|
||||||
|
g_spin.from = SpinNow(); // carry on from where the panels are now
|
||||||
|
} else {
|
||||||
|
g_spin.base.clear();
|
||||||
|
for (auto &[i, s] : g_screens) {
|
||||||
|
Mat p;
|
||||||
|
if (Spinnable(s) && ScreenPose(s, &p)) g_spin.base[i] = p;
|
||||||
|
}
|
||||||
|
g_spin.cx = head.m[0][3], g_spin.cz = head.m[2][3];
|
||||||
|
g_spin.from = g_spin.to = 0;
|
||||||
|
}
|
||||||
|
if (g_spin.base.empty()) return (void)std::snprintf(reply, size, "error nothing to spin");
|
||||||
|
double turn; // degrees, positive to the left
|
||||||
|
const bool next = !std::strcmp(arg, "next");
|
||||||
|
if (next || !std::strcmp(arg, "prev")) {
|
||||||
|
// Each visible panel's bearing from where you face, to the right positive, as it will
|
||||||
|
// be when the spin so far ends; the nearest one past straight ahead comes to the front.
|
||||||
|
double fx = -head.m[0][2], fz = -head.m[2][2];
|
||||||
|
const double n = std::sqrt(fx * fx + fz * fz) + 1e-9;
|
||||||
|
fx /= n, fz /= n;
|
||||||
|
const double rx = -fz, rz = fx;
|
||||||
|
double best = 0;
|
||||||
|
bool found = false;
|
||||||
|
for (const auto &[i, p] : g_spin.base) {
|
||||||
|
const auto s = g_screens.find(i);
|
||||||
|
if (s == g_screens.end() || !s->second.visible) continue;
|
||||||
|
const Mat q = Turned(p, g_spin.to, g_spin.cx, g_spin.cz);
|
||||||
|
const double dx = q.m[0][3] - head.m[0][3], dz = q.m[2][3] - head.m[2][3];
|
||||||
|
double a = std::atan2(dx * rx + dz * rz, dx * fx + dz * fz) * 180 / M_PI;
|
||||||
|
if (!next) a = -a;
|
||||||
|
if (a <= kSpinAhead) a += 360;
|
||||||
|
if (!found || a < best) best = a, found = true;
|
||||||
|
}
|
||||||
|
if (!found || best >= 360 - kSpinAhead) {
|
||||||
|
if (!g_spin.on) g_spin.base.clear();
|
||||||
|
return (void)std::snprintf(reply, size, "ok 0 (no other panel)");
|
||||||
|
}
|
||||||
|
if (best > 180) best -= 360; // the short way round
|
||||||
|
turn = next ? best : -best;
|
||||||
|
} else {
|
||||||
|
char *end;
|
||||||
|
turn = std::strtod(arg, &end);
|
||||||
|
if (end == arg || *end) return (void)std::snprintf(reply, size, "error spin next|prev|<degrees>");
|
||||||
|
}
|
||||||
|
g_spin.to += turn * M_PI / 180;
|
||||||
|
g_spin.start = Clock::now();
|
||||||
|
g_spin.on = true;
|
||||||
|
SendPointer("recenter"); // the 3D mouse's pointer stays in front of you, on what comes there
|
||||||
|
std::snprintf(reply, size, "ok %.1f", turn);
|
||||||
|
}
|
||||||
|
|
||||||
uint32_t LinuxButton(uint32_t vrButton) {
|
uint32_t LinuxButton(uint32_t vrButton) {
|
||||||
switch (vrButton) {
|
switch (vrButton) {
|
||||||
case vr::VRMouseButton_Right: return BTN_RIGHT;
|
case vr::VRMouseButton_Right: return BTN_RIGHT;
|
||||||
@@ -1514,6 +1683,81 @@ void ReleaseAwayBy(vr::TrackedDeviceIndex_t dev, uint32_t vrButton, void (*handl
|
|||||||
if (g_press.buttons && dev == g_press.device) ReleaseAway(LinuxButton(vrButton), handle, data);
|
if (g_press.buttons && dev == g_press.device) ReleaseAway(LinuxButton(vrButton), handle, data);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Where a laser meets a panel's surface, in the panel's u (metres along it from the centre,
|
||||||
|
// along the arc when curved) and v (up). OpenVR curves a screen into a cylinder toward its
|
||||||
|
// front, centred `curve` metres in front of it (see OnSurface).
|
||||||
|
bool RayOnSurface(const Screen &s, const Mat &p, const Mat &laser, double *u, double *v) {
|
||||||
|
const Mat inv = Inverse(p);
|
||||||
|
const double o[3] = {inv.m[0][0] * laser.m[0][3] + inv.m[0][1] * laser.m[1][3] + inv.m[0][2] * laser.m[2][3] + inv.m[0][3],
|
||||||
|
inv.m[1][0] * laser.m[0][3] + inv.m[1][1] * laser.m[1][3] + inv.m[1][2] * laser.m[2][3] + inv.m[1][3],
|
||||||
|
inv.m[2][0] * laser.m[0][3] + inv.m[2][1] * laser.m[1][3] + inv.m[2][2] * laser.m[2][3] + inv.m[2][3]};
|
||||||
|
double d[3];
|
||||||
|
for (int i = 0; i < 3; ++i) d[i] = -(inv.m[i][0] * laser.m[0][2] + inv.m[i][1] * laser.m[1][2] + inv.m[i][2] * laser.m[2][2]);
|
||||||
|
if (s.curve <= 0) {
|
||||||
|
if (std::fabs(d[2]) < 1e-6) return false;
|
||||||
|
const double t = -o[2] / d[2];
|
||||||
|
if (t <= 0) return false;
|
||||||
|
*u = o[0] + d[0] * t, *v = o[1] + d[1] * t;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
// x^2 + (z - r)^2 = r^2, on the screen's side of the axis (z < r).
|
||||||
|
const double r = s.curve, oz = o[2] - r;
|
||||||
|
const double a = d[0] * d[0] + d[2] * d[2], b = 2 * (o[0] * d[0] + oz * d[2]), c = o[0] * o[0] + oz * oz - r * r;
|
||||||
|
const double disc = b * b - 4 * a * c;
|
||||||
|
if (a < 1e-9 || disc < 0) return false;
|
||||||
|
for (double t : {(-b - std::sqrt(disc)) / (2 * a), (-b + std::sqrt(disc)) / (2 * a)}) {
|
||||||
|
const double x = o[0] + d[0] * t, z = oz + d[2] * t;
|
||||||
|
if (t <= 0 || z >= 0) continue;
|
||||||
|
*u = r * std::atan2(x, -z), *v = o[1] + d[1] * t;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Where the mode leaves the controllers to a VR game (see the top): a hand controller
|
||||||
|
// pointing at a panel, its controls, or a floating window's popups keeps that panel's laser
|
||||||
|
// on (UpdateLasers) until kAimLinger ticks after it points away, like SteamVR's own floating
|
||||||
|
// windows. Leaving takes a wider margin than arriving, and a drag or a held button keeps it
|
||||||
|
// on. The keyboard is one overlay, so SteamVR's own intersection test does there.
|
||||||
|
void UpdateAim() {
|
||||||
|
if (LasersByMode()) return;
|
||||||
|
std::vector<Mat> lasers;
|
||||||
|
for (vr::TrackedDeviceIndex_t i = 1; i < vr::k_unMaxTrackedDeviceCount; ++i) {
|
||||||
|
Mat d;
|
||||||
|
if (IsHandController(i) && LaserPose(i, &d)) lasers.push_back(d);
|
||||||
|
}
|
||||||
|
for (auto &[index, s] : g_screens) {
|
||||||
|
Mat p;
|
||||||
|
if (!s.visible || !ScreenPose(s, &p)) continue;
|
||||||
|
if (s.drag != Drag::None || (g_press.buttons && g_press.screen == index)) {
|
||||||
|
s.aimUntil = g_tick + kAimLinger;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const double m = s.grip * (g_tick < s.aimUntil ? 2.0 : 0.25), h = s.heightMetres();
|
||||||
|
// The panel and its controls: the bar row under it, the resize tab off its corner.
|
||||||
|
const double halfW = std::max(s.metres / 2 + s.grip, s.chrome / 2 + s.chrome * 0.12 + s.grip * 2) + m;
|
||||||
|
const double top = h / 2 + m, bottom = std::min(BarY(s) - s.grip, -(h / 2 + s.grip)) - m;
|
||||||
|
for (const Mat &l : lasers) {
|
||||||
|
double u, v;
|
||||||
|
if (!RayOnSurface(s, p, l, &u, &v)) continue;
|
||||||
|
bool on = std::fabs(u) <= halfW && v <= top && v >= bottom;
|
||||||
|
for (const auto &[k, sub] : s.subs) {
|
||||||
|
if (on || s.cropW <= 0) break;
|
||||||
|
const double su = (sub.x + sub.w / 2.0 - (s.cropX + s.cropW / 2.0)) * s.mpp;
|
||||||
|
const double sv = -(sub.y + sub.h / 2.0 - (s.cropY + s.cropH / 2.0)) * s.mpp;
|
||||||
|
on = std::fabs(u - su) <= sub.w * s.mpp / 2 + m && std::fabs(v - sv) <= sub.h * s.mpp / 2 + m;
|
||||||
|
}
|
||||||
|
if (on) {
|
||||||
|
s.aimUntil = g_tick + kAimLinger;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (keyboard::Shown())
|
||||||
|
for (const Mat &l : lasers)
|
||||||
|
if (keyboard::Aimed(l)) g_keyboardAimUntil = g_tick + kAimLinger;
|
||||||
|
}
|
||||||
|
|
||||||
const char *LasersName() {
|
const char *LasersName() {
|
||||||
switch (g_lasers) {
|
switch (g_lasers) {
|
||||||
case Lasers::Always: return "always";
|
case Lasers::Always: return "always";
|
||||||
@@ -1716,6 +1960,11 @@ int ft_vr_modifiers(uint32_t format, uint64_t *out, int max) {
|
|||||||
bool ft_vr_screens_shown(void) { return g_vr && ModeVisible(); }
|
bool ft_vr_screens_shown(void) { return g_vr && ModeVisible(); }
|
||||||
bool ft_vr_paused(void) { return g_paused; }
|
bool ft_vr_paused(void) { return g_paused; }
|
||||||
|
|
||||||
|
bool ft_vr_screen_visible(int index) {
|
||||||
|
const auto it = g_screens.find(index);
|
||||||
|
return !g_vr || (it != g_screens.end() && it->second.visible);
|
||||||
|
}
|
||||||
|
|
||||||
enum ft_attention ft_vr_screen_attention(int index) {
|
enum ft_attention ft_vr_screen_attention(int index) {
|
||||||
const auto it = g_screens.find(index);
|
const auto it = g_screens.find(index);
|
||||||
return g_vr && it != g_screens.end() ? it->second.attention : FT_FOCUSED;
|
return g_vr && it != g_screens.end() ? it->second.attention : FT_FOCUSED;
|
||||||
@@ -2036,6 +2285,7 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) {
|
|||||||
if (ev.eventType == vr::VREvent_MouseButtonUp)
|
if (ev.eventType == vr::VREvent_MouseButtonUp)
|
||||||
ReleaseAwayBy(ev.trackedDeviceIndex, ev.data.mouse.button, handle, data);
|
ReleaseAwayBy(ev.trackedDeviceIndex, ev.data.mouse.button, handle, data);
|
||||||
if (g_press.upAt >= 0 && g_tick >= g_press.upAt) ReleaseAway(BTN_LEFT, handle, data);
|
if (g_press.upAt >= 0 && g_tick >= g_press.upAt) ReleaseAway(BTN_LEFT, handle, data);
|
||||||
|
ReleaseStuck(handle, data);
|
||||||
RefreshChrome();
|
RefreshChrome();
|
||||||
while (vr::VRSystem()->PollNextEvent(&ev, sizeof ev)) {
|
while (vr::VRSystem()->PollNextEvent(&ev, sizeof ev)) {
|
||||||
if (ev.eventType == vr::VREvent_Quit) {
|
if (ev.eventType == vr::VREvent_Quit) {
|
||||||
@@ -2071,10 +2321,19 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) {
|
|||||||
handle(&e, data);
|
handle(&e, data);
|
||||||
}
|
}
|
||||||
++g_tick;
|
++g_tick;
|
||||||
|
UpdateSpin();
|
||||||
|
if (g_spin.front >= 0) {
|
||||||
|
ft_event e{};
|
||||||
|
e.type = FT_FRONT;
|
||||||
|
e.screen = g_spin.front;
|
||||||
|
g_spin.front = -1;
|
||||||
|
handle(&e, data);
|
||||||
|
}
|
||||||
UpdateGame();
|
UpdateGame();
|
||||||
UpdateArrange();
|
UpdateArrange();
|
||||||
UpdateVisibility();
|
UpdateVisibility();
|
||||||
UpdateAttention();
|
UpdateAttention();
|
||||||
|
UpdateAim();
|
||||||
UpdateLasers();
|
UpdateLasers();
|
||||||
UpdateControls();
|
UpdateControls();
|
||||||
UpdateGuides();
|
UpdateGuides();
|
||||||
@@ -2101,7 +2360,7 @@ bool ft_vr_keyboard_show(int index) {
|
|||||||
const double fx = -head.m[0][2], fz = -head.m[2][2], n = std::sqrt(fx * fx + fz * fz) + 1e-9;
|
const double fx = -head.m[0][2], fz = -head.m[2][2], n = std::sqrt(fx * fx + fz * fz) + 1e-9;
|
||||||
const double at[3] = {head.m[0][3] + fx / n * kKeyboardAhead, head.m[1][3] - kKeyboardBelow,
|
const double at[3] = {head.m[0][3] + fx / n * kKeyboardAhead, head.m[1][3] - kKeyboardBelow,
|
||||||
head.m[2][3] + fz / n * kKeyboardAhead};
|
head.m[2][3] + fz / n * kKeyboardAhead};
|
||||||
keyboard::SetLasers(g_lasers == Lasers::Always || (g_lasers == Lasers::OutsideGames && !g_gameRunning));
|
keyboard::SetLasers(LasersByMode());
|
||||||
g_steamInFront = SteamInFront();
|
g_steamInFront = SteamInFront();
|
||||||
if (g_steamInFront) {
|
if (g_steamInFront) {
|
||||||
g_asidePose = FacingPose(at, head);
|
g_asidePose = FacingPose(at, head);
|
||||||
@@ -2146,6 +2405,8 @@ void ft_vr_keyboard_hide(void) {
|
|||||||
// ~100 ms (it landed on something else), KWin gets it anyway
|
// ~100 ms (it landed on something else), KWin gets it anyway
|
||||||
// state -> "ok <mode> <manual 0|1> <wrist deg> <gesture hand> <gesture deg>
|
// state -> "ok <mode> <manual 0|1> <wrist deg> <gesture hand> <gesture deg>
|
||||||
// <controllers> <game running 0|1> <ingames>"
|
// <controllers> <game running 0|1> <ingames>"
|
||||||
|
// spin next|prev|<degrees> turn every panel in the room about your head (see the lazy
|
||||||
|
// susan) -> "ok <degrees turned>"
|
||||||
// cutouts on|off|state hand cutouts (see handcut.h) -> "ok <on|off> <ready|idle|unavailable>
|
// cutouts on|off|state hand cutouts (see handcut.h) -> "ok <on|off> <ready|idle|unavailable>
|
||||||
// <last composite ms> ms, predict <on|off> lead <ms> ms"
|
// <last composite ms> ms, predict <on|off> lead <ms> ms"
|
||||||
// cutouts predict on|off move the hands ahead along their velocity (on by default)
|
// cutouts predict on|off move the hands ahead along their velocity (on by default)
|
||||||
@@ -2183,6 +2444,7 @@ void ft_vr_command(const char *cmd, char *reply, int size) {
|
|||||||
Screen *s = Find(n);
|
Screen *s = Find(n);
|
||||||
if (!s) return (void)std::snprintf(reply, size, "error no screen %d", n);
|
if (!s) return (void)std::snprintf(reply, size, "error no screen %d", n);
|
||||||
EndDrag(*s);
|
EndDrag(*s);
|
||||||
|
g_spin.base.erase(n - 1);
|
||||||
SetAbsolute(*s, PanelPose(x, y, z, yaw, pitch, roll));
|
SetAbsolute(*s, PanelPose(x, y, z, yaw, pitch, roll));
|
||||||
std::snprintf(reply, size, "ok");
|
std::snprintf(reply, size, "ok");
|
||||||
} else if (std::sscanf(cmd, "width %d %lf", &n, &w) == 2) {
|
} else if (std::sscanf(cmd, "width %d %lf", &n, &w) == 2) {
|
||||||
@@ -2349,6 +2611,7 @@ void ft_vr_command(const char *cmd, char *reply, int size) {
|
|||||||
Mat m{};
|
Mat m{};
|
||||||
for (int k = 0; k < 12; ++k) m.m[k / 4][k % 4] = r[k];
|
for (int k = 0; k < 12; ++k) m.m[k / 4][k % 4] = r[k];
|
||||||
EndDrag(*s);
|
EndDrag(*s);
|
||||||
|
g_spin.base.erase(n - 1);
|
||||||
SetAbsolute(*s, m);
|
SetAbsolute(*s, m);
|
||||||
std::snprintf(reply, size, "ok");
|
std::snprintf(reply, size, "ok");
|
||||||
} else if (int k0; std::sscanf(cmd, "sub %d %d %d %d %d %d", &n, &k0, &x0, &y0, &w0, &h0) == 6 ||
|
} else if (int k0; std::sscanf(cmd, "sub %d %d %d %d %d %d", &n, &k0, &x0, &y0, &w0, &h0) == 6 ||
|
||||||
@@ -2376,6 +2639,8 @@ void ft_vr_command(const char *cmd, char *reply, int size) {
|
|||||||
if ((g_press.buttons & ButtonBit(BTN_LEFT)) && g_press.device != kNone && !IsHandController(g_press.device))
|
if ((g_press.buttons & ButtonBit(BTN_LEFT)) && g_press.device != kNone && !IsHandController(g_press.device))
|
||||||
g_press.upAt = g_tick + 9;
|
g_press.upAt = g_tick + 9;
|
||||||
std::snprintf(reply, size, "ok");
|
std::snprintf(reply, size, "ok");
|
||||||
|
} else if (std::sscanf(cmd, "spin %15s", word) == 1) {
|
||||||
|
SpinCommand(word, reply, size);
|
||||||
} else if (std::strncmp(cmd, "state", 5) == 0) {
|
} else if (std::strncmp(cmd, "state", 5) == 0) {
|
||||||
std::snprintf(reply, size, "ok %s %d %.0f %s %.0f %s %d %s", ModeName(), g_manual ? 1 : 0, g_wristAngle,
|
std::snprintf(reply, size, "ok %s %d %.0f %s %.0f %s %d %s", ModeName(), g_manual ? 1 : 0, g_wristAngle,
|
||||||
g_gestureHand.c_str(), g_gestureAngle, LasersName(), g_gameRunning ? 1 : 0,
|
g_gestureHand.c_str(), g_gestureAngle, LasersName(), g_gameRunning ? 1 : 0,
|
||||||
|
|||||||
+4
-1
@@ -17,7 +17,8 @@ struct ft_dmabuf {
|
|||||||
int fd[4];
|
int fd[4];
|
||||||
};
|
};
|
||||||
|
|
||||||
enum ft_event_type { FT_MOTION, FT_BUTTON, FT_SCROLL, FT_LEAVE, FT_QUIT, FT_KEY, FT_KEYBOARD_CLOSED };
|
// FT_FRONT: a spin (the lazy susan) brought panel `screen` to straight ahead: typing goes there.
|
||||||
|
enum ft_event_type { FT_MOTION, FT_BUTTON, FT_SCROLL, FT_LEAVE, FT_QUIT, FT_KEY, FT_KEYBOARD_CLOSED, FT_FRONT };
|
||||||
|
|
||||||
struct ft_event {
|
struct ft_event {
|
||||||
enum ft_event_type type;
|
enum ft_event_type type;
|
||||||
@@ -38,6 +39,8 @@ int ft_vr_modifiers(uint32_t format, uint64_t *out, int max);
|
|||||||
bool ft_vr_screens_shown(void);
|
bool ft_vr_screens_shown(void);
|
||||||
// Frametop is paused for a VR game ("pause on"): everything is hidden, and KWin slows down.
|
// Frametop is paused for a VR game ("pause on"): everything is hidden, and KWin slows down.
|
||||||
bool ft_vr_paused(void);
|
bool ft_vr_paused(void);
|
||||||
|
// Screen (or floating window) `index` shows now. True without SteamVR (--no-vr).
|
||||||
|
bool ft_vr_screen_visible(int index);
|
||||||
// How much of a screen you see, for its frame rate (compositor.c): hidden (or out of view),
|
// How much of a screen you see, for its frame rate (compositor.c): hidden (or out of view),
|
||||||
// in view, or focused (you look at it, or a laser or the mouse is on it). Focused without
|
// in view, or focused (you look at it, or a laser or the mouse is on it). Focused without
|
||||||
// SteamVR (--no-vr) or for an unknown screen.
|
// SteamVR (--no-vr) or for an unknown screen.
|
||||||
|
|||||||
Executable
+25
@@ -0,0 +1,25 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Update the lines of ~/.config/frametop.conf that still read exactly as an older
|
||||||
|
# frametop.conf.example wrote them. A line you changed stays as it is. install.sh and
|
||||||
|
# hands/rec/install.sh run this on every install and update.
|
||||||
|
# HANDS_SWAP_SIDES=0: the example's value until 2026-10-05. It made ft-hands keep ft-camd's side
|
||||||
|
# camera names even when they're backwards (some SteamVR restarts swap them), so hands landed
|
||||||
|
# beside their cutouts and the hand recorder labelled the side cameras wrong. auto tells from
|
||||||
|
# the hands.
|
||||||
|
set -euo pipefail
|
||||||
|
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
|
||||||
|
. "$root/scripts/_env.sh"
|
||||||
|
|
||||||
|
on_frame_script <<'EOF'
|
||||||
|
f=~/.config/frametop.conf
|
||||||
|
[ -f "$f" ] || exit 0
|
||||||
|
old="HANDS_SWAP_SIDES=0 # hand tracking (hands/run.sh install): 1 = the side cameras' names are swapped, which some SteamVR restarts cause (hands/tools/check_sides.py --ring tells)"
|
||||||
|
new="HANDS_SWAP_SIDES=auto # hand tracking: which side camera is which. auto = ft-hands tells from the hands and fixes ft-camd's names, which some SteamVR restarts swap | 0 = keep ft-camd's | 1 = exchange them"
|
||||||
|
if grep -qxF "$old" "$f"; then
|
||||||
|
tmp=$(mktemp)
|
||||||
|
awk -v old="$old" -v new="$new" '$0 == old { print new; next } { print }' "$f" > "$tmp"
|
||||||
|
cat "$tmp" > "$f" # in place: the file keeps its owner and mode
|
||||||
|
rm -f "$tmp"
|
||||||
|
echo "settings: HANDS_SWAP_SIDES=0 (the old default) is now auto: hand tracking tells the side cameras apart itself"
|
||||||
|
fi
|
||||||
|
EOF
|
||||||
@@ -36,6 +36,10 @@ check "distrobox" on_frame 'test -x ~/.local/bin/distrobox && ~/.local/bin/distr
|
|||||||
check "container $FRAME_BOX" on_frame "podman ps -a --filter name=^$FRAME_BOX\$ --format '{{.Image}} {{.Status}}' | grep ."
|
check "container $FRAME_BOX" on_frame "podman ps -a --filter name=^$FRAME_BOX\$ --format '{{.Image}} {{.Status}}' | grep ."
|
||||||
check "repo on the Frame" on_frame 'pwd'
|
check "repo on the Frame" on_frame 'pwd'
|
||||||
check "free space in ~" on_frame "df -h ~ | awk 'NR==2{print \$4\" free\"}'"
|
check "free space in ~" on_frame "df -h ~ | awk 'NR==2{print \$4\" free\"}'"
|
||||||
|
# A taskbar saved on a screen the desktop doesn't have is hidden; the desktop's next start
|
||||||
|
# moves it to the first screen (session/fix-panels.py).
|
||||||
|
check "taskbar on a screen" on_frame 'set -o pipefail; [ -f session/fix-panels.py ] || { echo "not checked (older checkout)"; exit 0; }
|
||||||
|
python3 session/fix-panels.py --check | paste -sd ";" | sed "s/;/; /g"'
|
||||||
|
|
||||||
echo "what Frametop needs from SteamOS:"
|
echo "what Frametop needs from SteamOS:"
|
||||||
if [ "$FRAME_LOCAL" = 1 ]; then
|
if [ "$FRAME_LOCAL" = 1 ]; then
|
||||||
|
|||||||
+44
-2
@@ -26,7 +26,8 @@ for u in frametop-input-relay frametop-pointer frametop-power; do
|
|||||||
done
|
done
|
||||||
echo "desktop: $(pgrep -x ft-screens >/dev/null && echo running || echo 'not running'), plasmashell: $(pgrep -c plasmashell || true)"
|
echo "desktop: $(pgrep -x ft-screens >/dev/null && echo running || echo 'not running'), plasmashell: $(pgrep -c plasmashell || true)"
|
||||||
echo "paused for VR games: $(cat /run/user/$(id -u)/frametop-pause.json 2>/dev/null || echo 'no (never paused since boot)')"
|
echo "paused for VR games: $(cat /run/user/$(id -u)/frametop-pause.json 2>/dev/null || echo 'no (never paused since boot)')"
|
||||||
LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrpathreg show 2>/dev/null | sed -n '/xternal/,$p'
|
# SteamVR's config folder: a terminal in the desktop has the session's own (docs/design.md).
|
||||||
|
XDG_CONFIG_HOME=$HOME/.config LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrpathreg show 2>/dev/null | sed -n '/xternal/,$p'
|
||||||
|
|
||||||
section "What Frametop needs from SteamOS (scripts/update-check.py)"
|
section "What Frametop needs from SteamOS (scripts/update-check.py)"
|
||||||
python3 "$repo/scripts/update-check.py" 2>&1 || true
|
python3 "$repo/scripts/update-check.py" 2>&1 || true
|
||||||
@@ -42,13 +43,54 @@ for i, s in enumerate(d.get("screens", []), 1):
|
|||||||
print("visibility:", d.get("visibility"))
|
print("visibility:", d.get("visibility"))
|
||||||
PY
|
PY
|
||||||
|
|
||||||
|
section "Plasma panels and outputs"
|
||||||
|
python3 "$repo/session/fix-panels.py" --check 2>&1 || true
|
||||||
|
python3 - "$repo" <<'PY' 2>&1 || true
|
||||||
|
import json, subprocess, sys
|
||||||
|
sys.path.insert(0, sys.argv[1] + "/layout")
|
||||||
|
import ft_layout
|
||||||
|
env = ft_layout.nested_env()
|
||||||
|
if not env:
|
||||||
|
sys.exit(print("desktop not running: no live outputs or panels"))
|
||||||
|
outs = json.loads(subprocess.run(["kscreen-doctor", "-j"], capture_output=True, text=True, env=env, timeout=10).stdout or "{}")
|
||||||
|
for o in outs.get("outputs", []):
|
||||||
|
size = o.get("size") or {}
|
||||||
|
print(f"output {o.get('name')}: enabled={o.get('enabled')} priority={o.get('priority')} {size.get('width')}x{size.get('height')}")
|
||||||
|
js = "print(JSON.stringify(panels().map(p => ({id: p.id, screen: p.screen, location: p.location}))))"
|
||||||
|
r = subprocess.run(["qdbus6", "org.kde.plasmashell", "/PlasmaShell", "org.kde.PlasmaShell.evaluateScript", js],
|
||||||
|
capture_output=True, text=True, env=env, timeout=10)
|
||||||
|
print("live panels:", (r.stdout or r.stderr).strip())
|
||||||
|
PY
|
||||||
|
|
||||||
section "Input relay (last 60 lines)"
|
section "Input relay (last 60 lines)"
|
||||||
journalctl --user -u frametop-input-relay -n 60 --no-pager -o short 2>/dev/null
|
journalctl --user -u frametop-input-relay -n 60 --no-pager -o short 2>/dev/null
|
||||||
section "Pointer helper (last 60 lines)"
|
section "Pointer helper (last 60 lines)"
|
||||||
journalctl --user -u frametop-pointer -n 60 --no-pager -o short 2>/dev/null
|
journalctl --user -u frametop-pointer -n 60 --no-pager -o short 2>/dev/null
|
||||||
section "Power service (last 30 lines)"
|
section "Power service (last 30 lines)"
|
||||||
journalctl --user -u frametop-power -n 30 --no-pager -o short 2>/dev/null
|
journalctl --user -u frametop-power -n 30 --no-pager -o short 2>/dev/null
|
||||||
for f in /tmp/frametop-session.log /tmp/frametop-screens.log /tmp/frametop-layout.log; do
|
section "Gaze"
|
||||||
|
echo "frametop-gaze: $(systemctl --user is-enabled frametop-gaze 2>/dev/null) / $(systemctl --user is-active frametop-gaze 2>/dev/null)"
|
||||||
|
echo "frametop-eyegrab (our eye tracker's frame grabber): $(systemctl is-enabled frametop-eyegrab 2>/dev/null) / $(systemctl is-active frametop-eyegrab 2>/dev/null)"
|
||||||
|
for f in calibration.json eyes/calibration.json; do # SteamVR's correction, our tracker's calibration
|
||||||
|
p=~/.local/state/frametop/gaze/$f
|
||||||
|
echo "$f: $([ -f "$p" ] && date -r "$p" '+%F %T' || echo none)"
|
||||||
|
done
|
||||||
|
python3 - "$repo" <<'PY' 2>&1 || true
|
||||||
|
import json, subprocess, sys
|
||||||
|
out = subprocess.run([sys.executable, sys.argv[1] + "/gaze/ft-gazectl", "status"], capture_output=True, text=True, timeout=10)
|
||||||
|
text = (out.stdout or out.stderr).strip()
|
||||||
|
try:
|
||||||
|
print("status:", json.dumps(json.loads(text), separators=(",", ":")))
|
||||||
|
except ValueError:
|
||||||
|
print("status:", text or "no answer from the gaze service")
|
||||||
|
PY
|
||||||
|
section "Gaze checks and calibration dots (last 10)"
|
||||||
|
tail -n 10 ~/.local/state/frametop/gaze/checks.jsonl 2>/dev/null | cut -c1-400 || true
|
||||||
|
section "Gaze service (last 60 lines, podman's left out)"
|
||||||
|
journalctl --user -u frametop-gaze -n 300 --no-pager -o short 2>/dev/null | grep -v ' podman\[' | tail -n 60
|
||||||
|
section "Our eye tracker's frame grabber (last 20 lines)"
|
||||||
|
journalctl -u frametop-eyegrab -n 20 --no-pager -o short 2>/dev/null
|
||||||
|
for f in /tmp/frametop-session.log /tmp/frametop-screens.log "$XDG_RUNTIME_DIR/frametop-layout.log"; do
|
||||||
section "$f (last 60 lines)"
|
section "$f (last 60 lines)"
|
||||||
tail -n 60 "$f" 2>/dev/null || echo "missing"
|
tail -n 60 "$f" 2>/dev/null || echo "missing"
|
||||||
done
|
done
|
||||||
|
|||||||
Executable
+4
@@ -0,0 +1,4 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
|
||||||
|
"$root/scripts/frame.sh" -C screens 'mkdir -p build; gcc -std=c11 -Wall -Wextra -Werror tests/relay-buttons-test.c -o build/relay-buttons-test && build/relay-buttons-test'
|
||||||
+74
-20
@@ -18,7 +18,6 @@ stops, or restarts anything.
|
|||||||
"""
|
"""
|
||||||
import base64
|
import base64
|
||||||
import json
|
import json
|
||||||
import math
|
|
||||||
import mmap
|
import mmap
|
||||||
import os
|
import os
|
||||||
import platform
|
import platform
|
||||||
@@ -34,7 +33,12 @@ UID = os.getuid()
|
|||||||
KNOWN_GOOD = os.path.join(HOME, ".local/state/frametop/known-good.json")
|
KNOWN_GOOD = os.path.join(HOME, ".local/state/frametop/known-good.json")
|
||||||
STEAMVR_BIN = "/opt/steamvr/bin/linuxarm64"
|
STEAMVR_BIN = "/opt/steamvr/bin/linuxarm64"
|
||||||
LAUNCHER = os.path.join(HOME, ".local/share/applications/deckard-nested-desktop.desktop")
|
LAUNCHER = os.path.join(HOME, ".local/share/applications/deckard-nested-desktop.desktop")
|
||||||
|
STOCK_LAUNCHER = "/usr/share/applications/deckard-nested-desktop.desktop"
|
||||||
|
NATIVE_LAUNCHER = os.path.join(HOME, ".local/share/applications/native-deckard-nested-desktop.desktop")
|
||||||
VRPATHS = os.path.join(HOME, ".config/openvr/openvrpaths.vrpath")
|
VRPATHS = os.path.join(HOME, ".config/openvr/openvrpaths.vrpath")
|
||||||
|
# The Frametop desktop has its own XDG_CONFIG_HOME (session/frametop-session.sh). An install from a
|
||||||
|
# terminal there, before pointer/driver/install.sh set SteamVR's, left vrpathreg's registry here.
|
||||||
|
STRAY_VRPATHS = os.path.join(HOME, ".config/frametop/openvr/openvrpaths.vrpath")
|
||||||
BACKLIGHT = "/sys/class/backlight/ae94000.dsi.0/brightness" # as in power/ft-powerd.cpp
|
BACKLIGHT = "/sys/class/backlight/ae94000.dsi.0/brightness" # as in power/ft-powerd.cpp
|
||||||
EYE_MMAP = "/dev/shm/eye-server.mmap"
|
EYE_MMAP = "/dev/shm/eye-server.mmap"
|
||||||
VRSERVER = "http://127.0.0.1:27062" # its web socket: input/vrws.py
|
VRSERVER = "http://127.0.0.1:27062" # its web socket: input/vrws.py
|
||||||
@@ -49,8 +53,11 @@ PACKAGES = {
|
|||||||
"after a desktop restart; floating a window; no blur behind the taskbar's menus",
|
"after a desktop restart; floating a window; no blur behind the taskbar's menus",
|
||||||
"plasma-workspace": "the taskbar and panels after a desktop restart; no DiscoverNotifier or "
|
"plasma-workspace": "the taskbar and panels after a desktop restart; no DiscoverNotifier or "
|
||||||
"ibus-daemon inside the desktop",
|
"ibus-daemon inside the desktop",
|
||||||
|
"at-spi2-core": "an AT-SPI-aware app appears on the nested desktop's accessibility bus; "
|
||||||
|
"the registry stops and comes back after a desktop restart",
|
||||||
"gamescope": "the headset's volume buttons with nothing focused; typing goes where you last clicked",
|
"gamescope": "the headset's volume buttons with nothing focused; typing goes where you last clicked",
|
||||||
"bluez": "a Bluetooth mouse reconnecting after it sleeps",
|
"bluez": "a Bluetooth mouse reconnecting after it sleeps",
|
||||||
|
"steamdeck-kde-presets": "Launch a program -> Native Desktop opens SteamOS's own desktop",
|
||||||
}
|
}
|
||||||
KERNEL_HINT = "display power (ft-powerd) and hand tracking"
|
KERNEL_HINT = "display power (ft-powerd) and hand tracking"
|
||||||
|
|
||||||
@@ -64,9 +71,11 @@ UNITS = {
|
|||||||
"frametop-hands": None,
|
"frametop-hands": None,
|
||||||
}
|
}
|
||||||
|
|
||||||
# eye-server.mmap offsets, the same as gaze/ft-gaze.cpp's (packed, little-endian).
|
# eye-server.mmap offsets, the same as gaze/ft-gaze.cpp's (packed, little-endian), and the
|
||||||
EYE_COUNTER, EYE_TIME, EYE_OPEN, EYE_NEED = 0x38, 0x157, 0x1CB, 0x1D3
|
# layouts its EyeFile::Detect knows: everything from the timestamp on moved by a shift.
|
||||||
EYE_VECTORS = (0x15F, 0x16B, 0x19B, 0x1A7) # set 1 left, right; set 2 left, right
|
EYE_COUNTER, EYE_TIME, EYE_NEED = 0x38, 0x157, 0x1F3 + 5
|
||||||
|
EYE_SET1 = (0x15F, 0x16B) # set 1 left, right
|
||||||
|
EYE_LAYOUTS = {0: "stable", 5: "the 0.4.x beta's (+5)"}
|
||||||
|
|
||||||
# Runs in a child process, so a SteamVR that hangs can't hang the check.
|
# Runs in a child process, so a SteamVR that hangs can't hang the check.
|
||||||
OPENVR_PROBE = r"""
|
OPENVR_PROBE = r"""
|
||||||
@@ -162,12 +171,20 @@ def check_host():
|
|||||||
("/usr/bin/kwin_wayland_wrapper", "FAIL", "the desktop can't start KWin"),
|
("/usr/bin/kwin_wayland_wrapper", "FAIL", "the desktop can't start KWin"),
|
||||||
("/usr/bin/startplasma-wayland", "FAIL", "the desktop can't start Plasma"),
|
("/usr/bin/startplasma-wayland", "FAIL", "the desktop can't start Plasma"),
|
||||||
("/usr/bin/dbus-run-session", "FAIL", "the desktop can't start its session bus"),
|
("/usr/bin/dbus-run-session", "FAIL", "the desktop can't start its session bus"),
|
||||||
|
("/usr/bin/python3", "warn", "nested accessibility can't run its startup helper"),
|
||||||
|
("/usr/bin/gdbus", "warn", "nested accessibility can't discover or check its bus"),
|
||||||
|
("/usr/lib/at-spi-bus-launcher", "warn", "nested accessibility can't start its bus"),
|
||||||
|
("/usr/lib/at-spi2-registryd", "warn", "nested accessibility has no fallback registry"),
|
||||||
|
("/usr/share/dbus-1/services/org.a11y.Bus.service", "warn",
|
||||||
|
"nested accessibility can't activate its bus"),
|
||||||
|
("/usr/share/dbus-1/accessibility-services/org.a11y.atspi.Registry.service", "warn",
|
||||||
|
"nested accessibility can't activate its registry natively"),
|
||||||
(f"{STEAMVR_BIN}/vrcmd", "FAIL", "the 3D mouse can't find panels"),
|
(f"{STEAMVR_BIN}/vrcmd", "FAIL", "the 3D mouse can't find panels"),
|
||||||
(f"{STEAMVR_BIN}/vrpathreg", "warn", "the pointer driver can't be installed or removed"),
|
(f"{STEAMVR_BIN}/vrpathreg", "warn", "the pointer driver can't be installed or removed"),
|
||||||
("/usr/share/deckard/mesavars.sh", "warn", "the desktop starts without SteamOS's Mesa settings"),
|
("/usr/share/deckard/mesavars.sh", "warn", "the desktop starts without SteamOS's Mesa settings"),
|
||||||
("/etc/profile.d/flatpak.sh", "warn", "Flatpak apps may open Discover instead of starting"),
|
("/etc/profile.d/flatpak.sh", "warn", "Flatpak apps may open Discover instead of starting"),
|
||||||
("/usr/share/applications/deckard-nested-desktop.desktop", "warn",
|
(STOCK_LAUNCHER, "warn", "SteamOS's Desktop launcher entry is gone or renamed, so Frametop's "
|
||||||
"SteamOS's Desktop launcher entry is gone or renamed, so Frametop's copy may not replace it"),
|
"copy may not replace it, and there's no Native Desktop"),
|
||||||
]
|
]
|
||||||
missing = [n for n in needed if not os.path.exists(n[0])]
|
missing = [n for n in needed if not os.path.exists(n[0])]
|
||||||
for path, state, effect in missing:
|
for path, state, effect in missing:
|
||||||
@@ -217,6 +234,14 @@ def check_host():
|
|||||||
else:
|
else:
|
||||||
report("FAIL", "pointer driver", "not registered with SteamVR; run pointer/driver/install.sh install, "
|
report("FAIL", "pointer driver", "not registered with SteamVR; run pointer/driver/install.sh install, "
|
||||||
"then restart SteamVR")
|
"then restart SteamVR")
|
||||||
|
try:
|
||||||
|
with open(STRAY_VRPATHS) as f:
|
||||||
|
stray = json.load(f).get("runtime") is None
|
||||||
|
except (OSError, ValueError, AttributeError):
|
||||||
|
stray = False
|
||||||
|
if stray:
|
||||||
|
report("warn", "OpenVR path registry", f"{STRAY_VRPATHS} has no SteamVR in it, and hides SteamVR from "
|
||||||
|
"OpenVR programs started in the Frametop desktop; pointer/driver/install.sh install removes it")
|
||||||
|
|
||||||
session = launcher_session()
|
session = launcher_session()
|
||||||
if session is None:
|
if session is None:
|
||||||
@@ -226,6 +251,16 @@ def check_host():
|
|||||||
report("ok", "launcher", f"Desktop starts {session}")
|
report("ok", "launcher", f"Desktop starts {session}")
|
||||||
else:
|
else:
|
||||||
report("FAIL", "launcher", f"Desktop starts {session}, which doesn't exist")
|
report("FAIL", "launcher", f"Desktop starts {session}, which doesn't exist")
|
||||||
|
if os.path.exists(NATIVE_LAUNCHER):
|
||||||
|
try:
|
||||||
|
same = launcher_keys(NATIVE_LAUNCHER) == launcher_keys(STOCK_LAUNCHER)
|
||||||
|
except OSError:
|
||||||
|
same = False
|
||||||
|
if same:
|
||||||
|
report("ok", "Native Desktop", "the launcher's copy matches SteamOS's Desktop entry")
|
||||||
|
else:
|
||||||
|
report("warn", "Native Desktop", "the launcher's copy no longer matches SteamOS's Desktop entry; "
|
||||||
|
"./desktops.sh install refreshes it")
|
||||||
|
|
||||||
if systemctl("is-enabled", "frametop-power") == "enabled":
|
if systemctl("is-enabled", "frametop-power") == "enabled":
|
||||||
if os.access(BACKLIGHT, os.W_OK):
|
if os.access(BACKLIGHT, os.W_OK):
|
||||||
@@ -234,6 +269,13 @@ def check_host():
|
|||||||
report("FAIL", "backlight", f"{BACKLIGHT} isn't writable, so ft-powerd can't turn the displays off")
|
report("FAIL", "backlight", f"{BACKLIGHT} isn't writable, so ft-powerd can't turn the displays off")
|
||||||
|
|
||||||
|
|
||||||
|
def launcher_keys(path):
|
||||||
|
"""A launcher entry's keys, less the ones desktops.sh changes in its Native Desktop copy."""
|
||||||
|
with open(path) as f:
|
||||||
|
pairs = [line.rstrip("\n").split("=", 1) for line in f if "=" in line and not line.startswith("#")]
|
||||||
|
return {k: v for k, v in pairs if k != "X-Steam-Special" and k.split("[")[0] != "Name"}
|
||||||
|
|
||||||
|
|
||||||
def launcher_session():
|
def launcher_session():
|
||||||
try:
|
try:
|
||||||
with open(LAUNCHER) as f:
|
with open(LAUNCHER) as f:
|
||||||
@@ -440,21 +482,31 @@ def check_eye_tracker():
|
|||||||
if len(m) < EYE_NEED:
|
if len(m) < EYE_NEED:
|
||||||
report("FAIL", "eye tracker", f"{EYE_MMAP} is {len(m)} bytes, smaller than gaze/ft-gaze.cpp reads")
|
report("FAIL", "eye tracker", f"{EYE_MMAP} is {len(m)} bytes, smaller than gaze/ft-gaze.cpp reads")
|
||||||
return
|
return
|
||||||
|
|
||||||
|
def fits(shift, before, now):
|
||||||
|
"""ft-gaze's test (EyeFile::Detect): the timestamp near the clock and moving on, and
|
||||||
|
both set-1 directions unit vectors."""
|
||||||
|
t = struct.unpack_from("<d", m, EYE_TIME + shift)[0]
|
||||||
|
return now - 2 < t <= now + 2 and t > before[shift] and all(
|
||||||
|
0.81 < sum(x * x for x in struct.unpack_from("<3f", m, o + shift)) < 1.21 for o in EYE_SET1)
|
||||||
|
|
||||||
first = struct.unpack_from("<I", m, EYE_COUNTER)[0]
|
first = struct.unpack_from("<I", m, EYE_COUNTER)[0]
|
||||||
time.sleep(0.3)
|
for attempt in range(5): # 1.5 s: a blink or a moment with an eye lost doesn't fail it
|
||||||
if struct.unpack_from("<I", m, EYE_COUNTER)[0] == first:
|
before = {shift: struct.unpack_from("<d", m, EYE_TIME + shift)[0] for shift in EYE_LAYOUTS}
|
||||||
report("skip", "eye tracker", "idle, so its layout wasn't checked")
|
time.sleep(0.3)
|
||||||
return
|
if attempt == 0 and struct.unpack_from("<I", m, EYE_COUNTER)[0] == first:
|
||||||
age = time.clock_gettime(time.CLOCK_MONOTONIC_RAW) - struct.unpack_from("<d", m, EYE_TIME)[0]
|
report("skip", "eye tracker", "idle, so its layout wasn't checked")
|
||||||
units = sum(abs(math.sqrt(sum(x * x for x in struct.unpack_from("<3f", m, o))) - 1) < 0.02
|
return
|
||||||
for o in EYE_VECTORS)
|
now = time.clock_gettime(time.CLOCK_MONOTONIC_RAW)
|
||||||
# A lost eye can zero its vector, so two of the four are enough. The timestamp is the
|
for shift, name in EYE_LAYOUTS.items():
|
||||||
# strong check: a fresh CLOCK_MONOTONIC_RAW double doesn't land on that offset by chance.
|
if fits(shift, before, now):
|
||||||
if abs(age) < 1 and units >= 2:
|
report("ok", "eye tracker", f"eye-server.mmap has a layout gaze/ft-gaze.cpp knows: {name}")
|
||||||
report("ok", "eye tracker", "eye-server.mmap still has the layout gaze/ft-gaze.cpp reads")
|
return
|
||||||
else:
|
age = now - struct.unpack_from("<d", m, EYE_TIME)[0]
|
||||||
report("FAIL" if gaze else "warn", "eye tracker", f"eye-server.mmap layout changed (sample age "
|
report("FAIL" if gaze else "warn", "eye tracker", f"eye-server.mmap has none of the layouts gaze/ft-gaze.cpp "
|
||||||
f"{age:.3g} s, {units} of 4 gaze vectors unit length); update the offsets in gaze/ft-gaze.cpp")
|
f"knows (sample age {age:.3g} s at the stable offset), so SteamVR's eye tracking can't be used: add the "
|
||||||
|
"new one to EyeFile::Detect. Right after the headset goes on its tracker warms up for about 20 s: "
|
||||||
|
"check again after that")
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
@@ -464,6 +516,8 @@ def main():
|
|||||||
# A terminal in the desktop has its session's runtime dir and bus; systemctl needs the real ones.
|
# A terminal in the desktop has its session's runtime dir and bus; systemctl needs the real ones.
|
||||||
os.environ["XDG_RUNTIME_DIR"] = f"/run/user/{UID}"
|
os.environ["XDG_RUNTIME_DIR"] = f"/run/user/{UID}"
|
||||||
os.environ["DBUS_SESSION_BUS_ADDRESS"] = f"unix:path=/run/user/{UID}/bus"
|
os.environ["DBUS_SESSION_BUS_ADDRESS"] = f"unix:path=/run/user/{UID}/bus"
|
||||||
|
# And its own config folder, where SteamVR's path registry isn't: OpenVR and vrcmd need ours.
|
||||||
|
os.environ["XDG_CONFIG_HOME"] = os.path.join(HOME, ".config")
|
||||||
|
|
||||||
versions = current_versions()
|
versions = current_versions()
|
||||||
check_versions(versions)
|
check_versions(versions)
|
||||||
|
|||||||
Executable
+176
@@ -0,0 +1,176 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Bring back a taskbar that Plasma saved against a screen this desktop doesn't have.
|
||||||
|
|
||||||
|
Plasma 6.2.5 ties each panel to a screen number (lastScreen in its containment), and the
|
||||||
|
numbers rank the enabled outputs by priority: 0 is the primary screen. A panel whose
|
||||||
|
number is past the screen count gets no view and stays hidden; Plasma never moves it,
|
||||||
|
even on a later start (sanitizeScreenLayout() only remaps a panel whose number has no
|
||||||
|
desktop, and the Frametop desktop keeps a desktop for every spare output it has seen).
|
||||||
|
So a taskbar saved on a spare output (issue #18), or on a screen that a smaller layout
|
||||||
|
dropped, was lost until the config was deleted.
|
||||||
|
|
||||||
|
The session runs this before Plasma starts, so nothing races Plasma for the file. Each
|
||||||
|
panel numbered at or past the screen count moves to screen 0, with its system tray's
|
||||||
|
containment, and keeps its widgets and settings. A panel is left where it is when screen
|
||||||
|
0 already has a panel on that edge: it comes back by itself if the screens do. Before
|
||||||
|
each repair the file is backed up to <file>.ft-bak.last, and <file>.ft-bak keeps it as it
|
||||||
|
was before the first repair (a later repair never overwrites that one). The changes go
|
||||||
|
through kwriteconfig6.
|
||||||
|
|
||||||
|
fix-panels.py [--screens N] [--file APPLETSRC] [--check]
|
||||||
|
--screens the desktop's screen count (default: the configured layout's)
|
||||||
|
--file default: $XDG_CONFIG_HOME/plasma-org.kde.plasma.desktop-appletsrc, with
|
||||||
|
XDG_CONFIG_HOME defaulting to the Frametop desktop's ~/.config/frametop
|
||||||
|
--check change nothing; list the panels, and exit 1 if one is lost
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
|
||||||
|
EDGES = {3: "top", 4: "bottom", 5: "left", 6: "right"} # Plasma::Types::Location
|
||||||
|
SYSTRAY = "org.kde.plasma.private.systemtray"
|
||||||
|
GROUP = re.compile(r"\[([^\]]*)\]")
|
||||||
|
|
||||||
|
|
||||||
|
def parse(text):
|
||||||
|
"""KConfig text -> {(group, subgroup, ...): {key: value}}."""
|
||||||
|
groups, cur = {}, None
|
||||||
|
for line in text.splitlines():
|
||||||
|
line = line.strip()
|
||||||
|
if line.startswith("["):
|
||||||
|
cur = tuple(GROUP.findall(line))
|
||||||
|
groups.setdefault(cur, {})
|
||||||
|
elif cur is not None and "=" in line and not line.startswith("#"):
|
||||||
|
k, v = line.split("=", 1)
|
||||||
|
groups[cur][re.sub(r"\[\$.*\]$", "", k.strip())] = v.strip()
|
||||||
|
return groups
|
||||||
|
|
||||||
|
|
||||||
|
def number(v, default=-1):
|
||||||
|
try:
|
||||||
|
return int(v)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return default
|
||||||
|
|
||||||
|
|
||||||
|
def panels(groups):
|
||||||
|
"""The panels, by containment id: location, lastScreen, and the ids of their system
|
||||||
|
trays' own containments (which sit on the same edge and screen as the panel)."""
|
||||||
|
trays, found = {}, {}
|
||||||
|
for g, keys in groups.items():
|
||||||
|
if len(g) == 5 and g[0] == "Containments" and g[2] == "Applets" and g[4] == "Configuration":
|
||||||
|
tray = keys.get("SystrayContainmentId")
|
||||||
|
if tray:
|
||||||
|
trays.setdefault(g[1], []).append(tray)
|
||||||
|
owned = {t for ts in trays.values() for t in ts}
|
||||||
|
for g, keys in groups.items():
|
||||||
|
if len(g) != 2 or g[0] != "Containments" or number(g[1]) <= 0 or g[1] in owned:
|
||||||
|
continue
|
||||||
|
loc = number(keys.get("location"), 0)
|
||||||
|
if loc in EDGES and keys.get("plugin") != SYSTRAY:
|
||||||
|
found[g[1]] = {"location": loc, "screen": number(keys.get("lastScreen")),
|
||||||
|
"plugin": keys.get("plugin", "?"), "trays": trays.get(g[1], [])}
|
||||||
|
return dict(sorted(found.items(), key=lambda kv: number(kv[0])))
|
||||||
|
|
||||||
|
|
||||||
|
def plan(groups, screens):
|
||||||
|
"""(moves, kept): moves are (panel id, from screen, containment ids to put on screen 0);
|
||||||
|
kept are (panel id, from screen, why) for lost panels left alone."""
|
||||||
|
found = panels(groups)
|
||||||
|
taken = {(p["screen"], p["location"]) for p in found.values() if 0 <= p["screen"] < screens}
|
||||||
|
moves, kept = [], []
|
||||||
|
for pid, p in found.items():
|
||||||
|
if p["screen"] < screens:
|
||||||
|
continue # on a screen this desktop has (Plasma puts -1 on the first one itself)
|
||||||
|
if (0, p["location"]) in taken:
|
||||||
|
kept.append((pid, p["screen"], f"the first screen already has a {EDGES[p['location']]} panel"))
|
||||||
|
continue
|
||||||
|
taken.add((0, p["location"]))
|
||||||
|
moves.append((pid, p["screen"], [pid, *p["trays"]]))
|
||||||
|
return moves, kept
|
||||||
|
|
||||||
|
|
||||||
|
def default_screens():
|
||||||
|
sys.path.insert(0, os.path.join(os.path.dirname(os.path.realpath(__file__)), "..", "layout"))
|
||||||
|
import ft_layout
|
||||||
|
return ft_layout.screen_count()
|
||||||
|
|
||||||
|
|
||||||
|
def backup(path, dst):
|
||||||
|
"""Copy path to dst whole or not at all. The copy goes to dst.tmp, reaches the disk, and
|
||||||
|
only then takes dst's name, so a copy cut off partway (a full disk, a crash, the battery)
|
||||||
|
never leaves a short dst behind, and a dst that's there already stays as it was."""
|
||||||
|
tmp = dst + ".tmp"
|
||||||
|
try:
|
||||||
|
shutil.copy2(path, tmp)
|
||||||
|
with open(tmp, "rb") as f:
|
||||||
|
os.fsync(f.fileno())
|
||||||
|
os.replace(tmp, dst)
|
||||||
|
except BaseException:
|
||||||
|
try:
|
||||||
|
os.remove(tmp)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv):
|
||||||
|
ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
|
||||||
|
ap.add_argument("--screens", type=int)
|
||||||
|
ap.add_argument("--file")
|
||||||
|
ap.add_argument("--check", action="store_true")
|
||||||
|
a = ap.parse_args(argv)
|
||||||
|
config = os.environ.get("XDG_CONFIG_HOME") or os.path.expanduser("~/.config/frametop")
|
||||||
|
path = a.file or os.path.join(config, "plasma-org.kde.plasma.desktop-appletsrc")
|
||||||
|
screens = a.screens if a.screens else default_screens()
|
||||||
|
try:
|
||||||
|
with open(path) as f:
|
||||||
|
groups = parse(f.read())
|
||||||
|
except FileNotFoundError:
|
||||||
|
if a.check:
|
||||||
|
print("no Plasma config yet (Plasma makes the default taskbar)")
|
||||||
|
return 0
|
||||||
|
moves, kept = plan(groups, screens)
|
||||||
|
|
||||||
|
if a.check:
|
||||||
|
found = panels(groups)
|
||||||
|
for pid, p in found.items():
|
||||||
|
lost = " (lost: no such screen)" if p["screen"] >= screens else ""
|
||||||
|
print(f"panel {pid}: screen {p['screen']}, {EDGES[p['location']]}{lost}")
|
||||||
|
if not found:
|
||||||
|
print("no panels")
|
||||||
|
print(f"{screens} screen(s)")
|
||||||
|
return 1 if moves or kept else 0
|
||||||
|
|
||||||
|
bak = path + ".ft-bak"
|
||||||
|
if moves:
|
||||||
|
try:
|
||||||
|
if not os.path.exists(bak):
|
||||||
|
# The config as it was before the first repair: a later run never overwrites
|
||||||
|
# it with an already-repaired state.
|
||||||
|
backup(path, bak)
|
||||||
|
# The config as it was just before this repair, so this one can be undone too,
|
||||||
|
# keeping what changed since the first (a moved panel's old screen is only here).
|
||||||
|
backup(path, bak + ".last")
|
||||||
|
except OSError as e:
|
||||||
|
# No repair without a backup; the next start tries again.
|
||||||
|
print(f"frametop: couldn't back up {path} ({e}); left the panels as they are", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
for pid, was, ids in moves:
|
||||||
|
for cid in ids:
|
||||||
|
subprocess.run(["kwriteconfig6", "--file", os.path.abspath(path), "--group", "Containments",
|
||||||
|
"--group", cid, "--key", "lastScreen", "0"], check=True)
|
||||||
|
print(f"frametop: panel {pid} was saved on screen {was}, which this desktop doesn't have "
|
||||||
|
f"({screens} screen(s)); moved it to the first screen (backups: {bak}.last from before this "
|
||||||
|
f"repair, {bak} from before the first)", file=sys.stderr)
|
||||||
|
for pid, was, why in kept:
|
||||||
|
print(f"frametop: panel {pid} is saved on screen {was}, which this desktop doesn't have; "
|
||||||
|
f"left there: {why}", file=sys.stderr)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main(sys.argv[1:]))
|
||||||
@@ -29,7 +29,7 @@ for var in $(compgen -e); do
|
|||||||
case $var in
|
case $var in
|
||||||
LD_LIBRARY_PATH | LD_PRELOAD | STEAM_* | Steam* | SRT_* | PRESSURE_VESSEL_* | MANGOHUD_* | \
|
LD_LIBRARY_PATH | LD_PRELOAD | STEAM_* | Steam* | SRT_* | PRESSURE_VESSEL_* | MANGOHUD_* | \
|
||||||
ENABLE_VK_LAYER_VALVE_steam_overlay_* | STEAMVIDEOTOKEN | QT_IM_MODULE | GTK_IM_MODULE | \
|
ENABLE_VK_LAYER_VALVE_steam_overlay_* | STEAMVIDEOTOKEN | QT_IM_MODULE | GTK_IM_MODULE | \
|
||||||
XMODIFIERS) unset "$var" ;;
|
XMODIFIERS | AT_SPI_BUS_ADDRESS) unset "$var" ;;
|
||||||
esac
|
esac
|
||||||
done
|
done
|
||||||
|
|
||||||
@@ -83,7 +83,8 @@ if [ "${1:-}" != --inner ]; then
|
|||||||
# Arrange the screens once they're up: in the profile this desktop starts with (FT_PROFILE,
|
# Arrange the screens once they're up: in the profile this desktop starts with (FT_PROFILE,
|
||||||
# from a profile's launcher entry, or the default profile), which also opens its apps, or
|
# from a profile's launcher entry, or the default profile), which also opens its apps, or
|
||||||
# else in the saved layout (skipped when auto-arrange is off). docs/profiles.md.
|
# else in the saved layout (skipped when auto-arrange is off). docs/profiles.md.
|
||||||
setsid "$here/../layout/ft-layout" start --wait 90 > /tmp/frametop-layout.log 2>&1 < /dev/null &
|
# The log starts fresh here; ft-screens appends its later ft-layout runs to it.
|
||||||
|
setsid "$here/../layout/ft-layout" start --wait 90 > "$XDG_RUNTIME_DIR/frametop-layout.log" 2>&1 < /dev/null &
|
||||||
|
|
||||||
if [ "$backend" = gamescope ]; then
|
if [ "$backend" = gamescope ]; then
|
||||||
export ENABLE_GAMESCOPE_WSI=1 GAMESCOPE_MANGOAPP_SOCKET_DISABLE=1
|
export ENABLE_GAMESCOPE_WSI=1 GAMESCOPE_MANGOAPP_SOCKET_DISABLE=1
|
||||||
@@ -190,6 +191,21 @@ if [ "$remote" = 1 ]; then
|
|||||||
"$here/remote-ctl.sh" start
|
"$here/remote-ctl.sh" start
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# AT-SPI: start the registry inside Plasma's autostart, after KWin has published the
|
||||||
|
# nested displays. Starting it before startplasma could bind to the host's X display.
|
||||||
|
# This config belongs only to Frametop; the host desktop's autostart is unchanged.
|
||||||
|
autostart=$XDG_CONFIG_HOME/autostart/frametop-atspi.desktop
|
||||||
|
mkdir -p "$(dirname "$autostart")"
|
||||||
|
cat > "$autostart" <<EOF
|
||||||
|
[Desktop Entry]
|
||||||
|
Type=Application
|
||||||
|
Name=Frametop accessibility
|
||||||
|
Exec="$here/ft-atspi"
|
||||||
|
X-KDE-autostart-phase=2
|
||||||
|
OnlyShowIn=KDE;
|
||||||
|
NoDisplay=true
|
||||||
|
EOF
|
||||||
|
|
||||||
# ft-floatd (floating windows) runs inside the Plasma session, on its D-Bus: started from
|
# ft-floatd (floating windows) runs inside the Plasma session, on its D-Bus: started from
|
||||||
# the session's autostart, which only this desktop reads (XDG_CONFIG_HOME above).
|
# the session's autostart, which only this desktop reads (XDG_CONFIG_HOME above).
|
||||||
autostart=$XDG_CONFIG_HOME/autostart/frametop-floatd.desktop
|
autostart=$XDG_CONFIG_HOME/autostart/frametop-floatd.desktop
|
||||||
@@ -272,4 +288,9 @@ fi
|
|||||||
# with both, apps would open twice.
|
# with both, apps would open twice.
|
||||||
kwriteconfig6 --file "$XDG_CONFIG_HOME/ksmserverrc" --group General --key loginMode emptySession
|
kwriteconfig6 --file "$XDG_CONFIG_HOME/ksmserverrc" --group General --key loginMode emptySession
|
||||||
|
|
||||||
|
# Plasma keeps a panel on a screen number, and never moves one whose screen this desktop
|
||||||
|
# doesn't have, like a spare output or a screen a smaller layout dropped. Put such a panel
|
||||||
|
# back on the first (primary) screen before Plasma reads the file (session/fix-panels.py).
|
||||||
|
python3 "$here/fix-panels.py" --screens "$screens" || true
|
||||||
|
|
||||||
dbus-run-session startplasma-wayland
|
dbus-run-session startplasma-wayland
|
||||||
@@ -40,7 +40,7 @@ POINTER_HEAD_DEADZONE=0.5 # keyboard clicks at the gaze (Meta+J, Meta+K): degre
|
|||||||
POINTER_KEY_TAP=0.25 # keyboard clicks: let go within this (s) and it clicks where the dot was at the press, and tells the gaze tracker it was right
|
POINTER_KEY_TAP=0.25 # keyboard clicks: let go within this (s) and it clicks where the dot was at the press, and tells the gaze tracker it was right
|
||||||
GAZE_TRACKER=auto # gaze service: auto = our own eye tracker when it's installed (gaze/tracker/install.sh, which install.sh offers), else SteamVR's | own = ours | steam = SteamVR's. Each keeps its own calibration (Calibrate on Frametop Input Settings' Gaze page)
|
GAZE_TRACKER=auto # gaze service: auto = our own eye tracker when it's installed (gaze/tracker/install.sh, which install.sh offers), else SteamVR's | own = ours | steam = SteamVR's. Each keeps its own calibration (Calibrate on Frametop Input Settings' Gaze page)
|
||||||
GAZE_EYE=auto # gaze service: eye bias. auto = each eye weighted by how far off it was at your recent nudges | left | right = that eye counts twice
|
GAZE_EYE=auto # gaze service: eye bias. auto = each eye weighted by how far off it was at your recent nudges | left | right = that eye counts twice
|
||||||
HANDS_SWAP_SIDES=0 # hand tracking (hands/run.sh install): 1 = the side cameras' names are swapped, which some SteamVR restarts cause (hands/tools/check_sides.py --ring tells)
|
HANDS_SWAP_SIDES=auto # hand tracking: which side camera is which. auto = ft-hands tells from the hands and fixes ft-camd's names, which some SteamVR restarts swap | 0 = keep ft-camd's | 1 = exchange them
|
||||||
HANDS_CPUS=5,6,7 # hand tracking: the CPUs its model threads run on
|
HANDS_CPUS=5,6,7 # hand tracking: the CPUs its model threads run on
|
||||||
HANDS_CAMERAS=auto # hand tracking: auto (by the light) | mono (the IR cameras; also keeps ft-camd off the colour ones) | color | all
|
HANDS_CAMERAS=auto # hand tracking: auto (by the light) | mono (the IR cameras; also keeps ft-camd off the colour ones) | color | all
|
||||||
HANDS_BRIGHT=all # hand tracking, auto: the cameras in bright light, all | color
|
HANDS_BRIGHT=all # hand tracking, auto: the cameras in bright light, all | color
|
||||||
|
|||||||
Executable
+87
@@ -0,0 +1,87 @@
|
|||||||
|
#!/usr/bin/python3
|
||||||
|
"""Start AT-SPI on this nested desktop's live bus, after KWin has set its displays."""
|
||||||
|
import ast
|
||||||
|
import os
|
||||||
|
import signal
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
|
||||||
|
DBUS = "org.freedesktop.DBus"
|
||||||
|
DBUS_PATH = "/org/freedesktop/DBus"
|
||||||
|
REGISTRY = "org.a11y.atspi.Registry"
|
||||||
|
|
||||||
|
|
||||||
|
def call(address, destination, path, method, *args):
|
||||||
|
return subprocess.run(
|
||||||
|
["gdbus", "call", "--address", address, "--dest", destination,
|
||||||
|
"--object-path", path, "--method", method, "--timeout", "5", *args],
|
||||||
|
check=True, capture_output=True, text=True, timeout=6).stdout.strip()
|
||||||
|
|
||||||
|
|
||||||
|
def stop(child):
|
||||||
|
if child.poll() is None:
|
||||||
|
child.terminate()
|
||||||
|
try:
|
||||||
|
child.wait(timeout=3)
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
child.kill()
|
||||||
|
child.wait()
|
||||||
|
|
||||||
|
|
||||||
|
def run_registry(address, session):
|
||||||
|
# SteamOS's registryd stays alive after a bus disconnect. Keep only our own
|
||||||
|
# child tied to both private buses; no host PID lookup or broad process kill.
|
||||||
|
registry = subprocess.Popen(["/usr/lib/at-spi2-registryd"])
|
||||||
|
try:
|
||||||
|
while registry.poll() is None:
|
||||||
|
try:
|
||||||
|
call(session, DBUS, DBUS_PATH, DBUS + ".GetId")
|
||||||
|
call(address, DBUS, DBUS_PATH, DBUS + ".GetId")
|
||||||
|
except subprocess.SubprocessError:
|
||||||
|
break # Normal session shutdown or loss of its accessibility bus.
|
||||||
|
# Each check starts two gdbus processes, and SteamVR needs the CPU.
|
||||||
|
time.sleep(5)
|
||||||
|
finally:
|
||||||
|
stop(registry)
|
||||||
|
|
||||||
|
|
||||||
|
def start():
|
||||||
|
session = os.environ.get("DBUS_SESSION_BUS_ADDRESS")
|
||||||
|
runtime = os.environ.get("XDG_RUNTIME_DIR", "")
|
||||||
|
if not session or os.path.basename(runtime) != "frametop":
|
||||||
|
return # Do not autolaunch a bus or touch the host desktop.
|
||||||
|
os.environ.pop("AT_SPI_BUS_ADDRESS", None)
|
||||||
|
reply = call(session, "org.a11y.Bus", "/org/a11y/bus", "org.a11y.Bus.GetAddress")
|
||||||
|
address, = ast.literal_eval(reply)
|
||||||
|
if not isinstance(address, str) or not address.startswith("unix:"):
|
||||||
|
raise ValueError("org.a11y.Bus returned no local accessibility bus")
|
||||||
|
if call(address, DBUS, DBUS_PATH, DBUS + ".NameHasOwner", REGISTRY) == "(true,)":
|
||||||
|
return
|
||||||
|
# Activation otherwise inherits the accessibility daemon's old environment (and
|
||||||
|
# DBUS_SESSION_BUS_ADDRESS is the accessibility bus itself). Pin the live bus.
|
||||||
|
env = {key: os.environ.get(key, "") for key in (
|
||||||
|
"DISPLAY", "WAYLAND_DISPLAY", "XAUTHORITY", "XDG_RUNTIME_DIR",
|
||||||
|
"XDG_CONFIG_HOME", "DBUS_SESSION_BUS_ADDRESS")}
|
||||||
|
env["AT_SPI_BUS_ADDRESS"] = address
|
||||||
|
call(address, DBUS, DBUS_PATH, DBUS + ".UpdateActivationEnvironment", repr(env))
|
||||||
|
try:
|
||||||
|
call(address, DBUS, DBUS_PATH, DBUS + ".StartServiceByName", REGISTRY, "0")
|
||||||
|
except subprocess.CalledProcessError:
|
||||||
|
# SteamOS's launcher can select dbus-broker because we are in a user unit,
|
||||||
|
# but the private session bus has no systemd activation manager. Reuse its
|
||||||
|
# bus rather than starting another launcher/bus. Never replace an owner.
|
||||||
|
if call(address, DBUS, DBUS_PATH, DBUS + ".NameHasOwner", REGISTRY) == "(true,)":
|
||||||
|
return
|
||||||
|
os.environ["AT_SPI_BUS_ADDRESS"] = address
|
||||||
|
# registryd itself refuses to queue behind an existing registry, including
|
||||||
|
# races with native activation. The watcher cleans up only our own child.
|
||||||
|
run_registry(address, session)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
signal.signal(signal.SIGTERM, lambda signum, frame: sys.exit(0))
|
||||||
|
try:
|
||||||
|
start()
|
||||||
|
except (OSError, ValueError, SyntaxError, subprocess.SubprocessError) as error:
|
||||||
|
print(f"frametop: accessibility unavailable: {error}", file=sys.stderr)
|
||||||
@@ -10,7 +10,7 @@ cg=$(systemctl --user show -p ControlGroup --value "$unit" 2>/dev/null)
|
|||||||
|
|
||||||
# The desktop's own processes stay in the unit and stop with it: the session, KWin,
|
# The desktop's own processes stay in the unit and stop with it: the session, KWin,
|
||||||
# Plasma, and the session services it started (portals, input methods, kded, wallet...).
|
# Plasma, and the session services it started (portals, input methods, kded, wallet...).
|
||||||
own='^(frametop-sessi|dbus-|startplasma|plasma|kwin|Xwayland|ksmserver|krdpserver|Xvnc|xfreerdp|ft-layout|'
|
own='^(frametop-sessi|dbus-|startplasma|plasma|kwin|Xwayland|ksmserver|krdpserver|Xvnc|xfreerdp|ft-layout|ft-atspi|'
|
||||||
own+='kded|kactivitymanage|kaccess|kglobalaccel|kscreen|kwalletd|ksecretd|polkit-kde|org_kde_|baloo|'
|
own+='kded|kactivitymanage|kaccess|kglobalaccel|kscreen|kwalletd|ksecretd|polkit-kde|org_kde_|baloo|'
|
||||||
own+='xembedsniproxy|gmenudbusmenu|DiscoverNotifie|kimpanel|ibus|xdg-|at-spi|dconf-service|fusermount|agent)'
|
own+='xembedsniproxy|gmenudbusmenu|DiscoverNotifie|kimpanel|ibus|xdg-|at-spi|dconf-service|fusermount|agent)'
|
||||||
keep=()
|
keep=()
|
||||||
|
|||||||
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,223 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Tests for session/fix-panels.py: issue #18's config (the taskbar saved on spare output 8
|
||||||
|
of a 3-screen desktop), panels that are fine, an edge that's taken, and a real run of
|
||||||
|
kwriteconfig6 on a copy. Nothing here touches the running desktop or its config.
|
||||||
|
|
||||||
|
python3 session/tests/test_fix_panels.py
|
||||||
|
"""
|
||||||
|
import importlib.util
|
||||||
|
import os
|
||||||
|
import resource
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import unittest
|
||||||
|
|
||||||
|
HERE = os.path.dirname(os.path.realpath(__file__))
|
||||||
|
SCRIPT = os.path.join(HERE, "..", "fix-panels.py")
|
||||||
|
spec = importlib.util.spec_from_file_location("fix_panels", SCRIPT)
|
||||||
|
fp = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(fp)
|
||||||
|
|
||||||
|
|
||||||
|
def desktop(cid, screen):
|
||||||
|
return f"""[Containments][{cid}]
|
||||||
|
activityId=7e1d1a3c-0000-4000-8000-000000000000
|
||||||
|
formfactor=0
|
||||||
|
immutability=1
|
||||||
|
lastScreen={screen}
|
||||||
|
location=0
|
||||||
|
plugin=org.kde.plasma.folder
|
||||||
|
wallpaperplugin=org.kde.image
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def panel(cid, screen, location=4, tray=None):
|
||||||
|
text = f"""[Containments][{cid}]
|
||||||
|
activityId=
|
||||||
|
formfactor=2
|
||||||
|
immutability=1
|
||||||
|
lastScreen={screen}
|
||||||
|
location={location}
|
||||||
|
plugin=org.kde.panel
|
||||||
|
wallpaperplugin=org.kde.image
|
||||||
|
|
||||||
|
[Containments][{cid}][Applets][{cid}1]
|
||||||
|
immutability=1
|
||||||
|
plugin=org.kde.plasma.kickoff
|
||||||
|
"""
|
||||||
|
if tray:
|
||||||
|
text += f"""
|
||||||
|
[Containments][{cid}][Applets][{cid}2]
|
||||||
|
immutability=1
|
||||||
|
plugin=org.kde.plasma.systemtray
|
||||||
|
|
||||||
|
[Containments][{cid}][Applets][{cid}2][Configuration]
|
||||||
|
PreloadWeight=90
|
||||||
|
SystrayContainmentId={tray}
|
||||||
|
|
||||||
|
[Containments][{tray}]
|
||||||
|
activityId=
|
||||||
|
formfactor=2
|
||||||
|
immutability=1
|
||||||
|
lastScreen={screen}
|
||||||
|
location={location}
|
||||||
|
plugin=org.kde.plasma.private.systemtray
|
||||||
|
popupHeight=432
|
||||||
|
"""
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
ISSUE_18 = "\n".join([desktop(1, 0), desktop(18, 1), desktop(19, 2)] + [desktop(20 + i, 3 + i) for i in range(8)]
|
||||||
|
+ [panel(10, 8, tray=11), "[ScreenMapping]\nitemsOnDisabledScreens=\n"])
|
||||||
|
|
||||||
|
|
||||||
|
class Plan(unittest.TestCase):
|
||||||
|
def test_issue_18(self):
|
||||||
|
moves, kept = fp.plan(fp.parse(ISSUE_18), 3)
|
||||||
|
self.assertEqual(moves, [("10", 8, ["10", "11"])])
|
||||||
|
self.assertEqual(kept, [])
|
||||||
|
|
||||||
|
def test_panel_on_a_screen_stays(self):
|
||||||
|
for screen in (0, 2, -1):
|
||||||
|
moves, kept = fp.plan(fp.parse(desktop(1, 0) + panel(2, screen, tray=8)), 3)
|
||||||
|
self.assertEqual((moves, kept), ([], []), screen)
|
||||||
|
|
||||||
|
def test_tray_is_not_a_panel(self):
|
||||||
|
found = fp.panels(fp.parse(panel(2, 0, tray=8)))
|
||||||
|
self.assertEqual(list(found), ["2"])
|
||||||
|
self.assertEqual(found["2"]["trays"], ["8"])
|
||||||
|
|
||||||
|
def test_desktops_never_move(self):
|
||||||
|
moves, _ = fp.plan(fp.parse(ISSUE_18), 1)
|
||||||
|
self.assertEqual([m[0] for m in moves], ["10"])
|
||||||
|
|
||||||
|
def test_edge_taken_on_first_screen(self):
|
||||||
|
text = panel(2, 0, tray=8) + panel(10, 8, tray=11)
|
||||||
|
moves, kept = fp.plan(fp.parse(text), 3)
|
||||||
|
self.assertEqual(moves, [])
|
||||||
|
self.assertEqual([k[:2] for k in kept], [("10", 8)])
|
||||||
|
|
||||||
|
def test_other_edge_is_free(self):
|
||||||
|
moves, kept = fp.plan(fp.parse(panel(2, 0) + panel(10, 4, location=3)), 3)
|
||||||
|
self.assertEqual(moves, [("10", 4, ["10"])])
|
||||||
|
self.assertEqual(kept, [])
|
||||||
|
|
||||||
|
def test_two_lost_on_one_edge(self):
|
||||||
|
moves, kept = fp.plan(fp.parse(panel(10, 5) + panel(12, 8)), 3)
|
||||||
|
self.assertEqual([m[0] for m in moves], ["10"])
|
||||||
|
self.assertEqual([k[0] for k in kept], ["12"])
|
||||||
|
|
||||||
|
def test_screen_count_went_down(self):
|
||||||
|
moves, _ = fp.plan(fp.parse(panel(2, 1, location=3)), 1)
|
||||||
|
self.assertEqual(moves, [("2", 1, ["2"])])
|
||||||
|
|
||||||
|
|
||||||
|
class Run(unittest.TestCase):
|
||||||
|
def setUp(self):
|
||||||
|
self.dir = tempfile.TemporaryDirectory()
|
||||||
|
self.path = os.path.join(self.dir.name, "plasma-org.kde.plasma.desktop-appletsrc")
|
||||||
|
with open(self.path, "w") as f:
|
||||||
|
f.write(ISSUE_18)
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
self.dir.cleanup()
|
||||||
|
|
||||||
|
def run_script(self, *args):
|
||||||
|
return subprocess.run([sys.executable, SCRIPT, "--file", self.path, "--screens", "3", *args],
|
||||||
|
capture_output=True, text=True)
|
||||||
|
|
||||||
|
def test_check_reports_and_changes_nothing(self):
|
||||||
|
r = self.run_script("--check")
|
||||||
|
self.assertEqual(r.returncode, 1)
|
||||||
|
self.assertIn("panel 10: screen 8, bottom (lost", r.stdout)
|
||||||
|
with open(self.path) as f:
|
||||||
|
self.assertEqual(f.read(), ISSUE_18)
|
||||||
|
|
||||||
|
def test_repair(self):
|
||||||
|
r = self.run_script()
|
||||||
|
self.assertEqual(r.returncode, 0, r.stderr)
|
||||||
|
self.assertIn("moved it to the first screen", r.stderr)
|
||||||
|
with open(self.path) as f:
|
||||||
|
groups = fp.parse(f.read())
|
||||||
|
self.assertEqual(groups[("Containments", "10")]["lastScreen"], "0")
|
||||||
|
self.assertEqual(groups[("Containments", "11")]["lastScreen"], "0")
|
||||||
|
self.assertEqual(groups[("Containments", "10", "Applets", "101")]["plugin"], "org.kde.plasma.kickoff")
|
||||||
|
self.assertEqual(groups[("Containments", "25")]["lastScreen"], "8") # a spare's desktop
|
||||||
|
for bak in (".ft-bak", ".ft-bak.last"):
|
||||||
|
with open(self.path + bak) as f:
|
||||||
|
self.assertEqual(f.read(), ISSUE_18, bak)
|
||||||
|
# A second run finds nothing to do and makes no backup.
|
||||||
|
os.remove(self.path + ".ft-bak")
|
||||||
|
os.remove(self.path + ".ft-bak.last")
|
||||||
|
r = self.run_script()
|
||||||
|
self.assertEqual((r.returncode, r.stderr), (0, ""))
|
||||||
|
self.assertEqual(os.listdir(self.dir.name), [os.path.basename(self.path)])
|
||||||
|
self.assertEqual(self.run_script("--check").returncode, 0)
|
||||||
|
|
||||||
|
def test_no_config_yet(self):
|
||||||
|
os.remove(self.path)
|
||||||
|
self.assertEqual(self.run_script().returncode, 0)
|
||||||
|
self.assertEqual(self.run_script("--check").returncode, 0)
|
||||||
|
|
||||||
|
def test_backup_is_written_once(self):
|
||||||
|
"""A backup that's already there, the config before an earlier repair, stays as it
|
||||||
|
was, and the repair still goes ahead."""
|
||||||
|
with open(self.path + ".ft-bak", "w") as f:
|
||||||
|
f.write("the config before an earlier repair\n")
|
||||||
|
r = self.run_script()
|
||||||
|
self.assertEqual(r.returncode, 0, r.stderr)
|
||||||
|
with open(self.path + ".ft-bak") as f:
|
||||||
|
self.assertEqual(f.read(), "the config before an earlier repair\n")
|
||||||
|
with open(self.path) as f:
|
||||||
|
groups = fp.parse(f.read())
|
||||||
|
self.assertEqual(groups[("Containments", "10")]["lastScreen"], "0")
|
||||||
|
self.assertEqual(groups[("Containments", "11")]["lastScreen"], "0")
|
||||||
|
|
||||||
|
def test_last_backup_is_from_before_each_repair(self):
|
||||||
|
"""<file>.ft-bak.last is the config as it was just before the latest repair, so that
|
||||||
|
repair can be undone without losing what changed since the first; <file>.ft-bak stays
|
||||||
|
the config before the first repair. The log names both."""
|
||||||
|
bak = self.path + ".ft-bak"
|
||||||
|
r = self.run_script()
|
||||||
|
self.assertEqual(r.returncode, 0, r.stderr)
|
||||||
|
# The panel is lost again later, after a change of the user's own (another widget).
|
||||||
|
changed = ISSUE_18.replace("org.kde.plasma.kickoff", "org.kde.plasma.trash")
|
||||||
|
with open(self.path, "w") as f:
|
||||||
|
f.write(changed)
|
||||||
|
r = self.run_script()
|
||||||
|
self.assertEqual(r.returncode, 0, r.stderr)
|
||||||
|
with open(bak + ".last") as f:
|
||||||
|
self.assertEqual(f.read(), changed)
|
||||||
|
with open(bak) as f:
|
||||||
|
self.assertEqual(f.read(), ISSUE_18)
|
||||||
|
with open(self.path) as f:
|
||||||
|
self.assertEqual(fp.parse(f.read())[("Containments", "10")]["lastScreen"], "0")
|
||||||
|
self.assertIn(bak + ".last ", r.stderr)
|
||||||
|
self.assertIn(bak + " ", r.stderr)
|
||||||
|
|
||||||
|
def test_failed_backup_leaves_nothing(self):
|
||||||
|
"""A backup cut off partway (a full disk; here a 1 KiB limit on file size) leaves no
|
||||||
|
backup and no temporary file behind, and the panels aren't touched without one. The
|
||||||
|
next start, with room, makes the backup and repairs."""
|
||||||
|
self.assertGreater(len(ISSUE_18), 1024)
|
||||||
|
|
||||||
|
def small_files():
|
||||||
|
resource.setrlimit(resource.RLIMIT_FSIZE, (1024, 1024))
|
||||||
|
r = subprocess.run([sys.executable, SCRIPT, "--file", self.path, "--screens", "3"],
|
||||||
|
capture_output=True, text=True, preexec_fn=small_files)
|
||||||
|
self.assertEqual(os.listdir(self.dir.name), [os.path.basename(self.path)])
|
||||||
|
self.assertEqual(r.returncode, 1, r.stderr)
|
||||||
|
self.assertIn("couldn't back up", r.stderr)
|
||||||
|
with open(self.path) as f:
|
||||||
|
self.assertEqual(f.read(), ISSUE_18)
|
||||||
|
r = self.run_script()
|
||||||
|
self.assertEqual(r.returncode, 0, r.stderr)
|
||||||
|
with open(self.path + ".ft-bak") as f:
|
||||||
|
self.assertEqual(f.read(), ISSUE_18)
|
||||||
|
with open(self.path) as f:
|
||||||
|
self.assertEqual(fp.parse(f.read())[("Containments", "10")]["lastScreen"], "0")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -37,6 +37,7 @@ fi
|
|||||||
|
|
||||||
# The VNC password is limited to 8 characters by the protocol. The traffic is
|
# The VNC password is limited to 8 characters by the protocol. The traffic is
|
||||||
# still encrypted by the tailnet (WireGuard).
|
# still encrypted by the tailnet (WireGuard).
|
||||||
|
mkdir -m 0700 -p "$creds" # remote-desktop.sh normally creates it; we may get here first
|
||||||
if [ ! -s "$creds/vnc-password" ]; then
|
if [ ! -s "$creds/vnc-password" ]; then
|
||||||
(umask 077; head -c 12 /dev/urandom | base64 | tr -d '/+=' | cut -c1-8 > "$creds/vnc-password")
|
(umask 077; head -c 12 /dev/urandom | base64 | tr -d '/+=' | cut -c1-8 > "$creds/vnc-password")
|
||||||
fi
|
fi
|
||||||
|
|||||||
Executable
+256
@@ -0,0 +1,256 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Uninstall Frametop from the Steam Frame. In a terminal on the headset (Konsole in the desktop,
|
||||||
|
# or over SSH):
|
||||||
|
#
|
||||||
|
# curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash
|
||||||
|
#
|
||||||
|
# or ~/frametop/uninstall.sh. It doesn't use the rest of the repo, so it also works when
|
||||||
|
# ~/frametop is gone or broken.
|
||||||
|
#
|
||||||
|
# It never stops what you're using now: the input relay carries the keyboard and mouse, and the
|
||||||
|
# desktop runs from the repo. So it goes in two steps:
|
||||||
|
# 1. Frametop stops starting. The launcher's Desktop entry goes back to the stock desktop, and
|
||||||
|
# Frametop's Native Desktop entry, services, SteamVR driver, menu entries, and system files
|
||||||
|
# (our eye tracker's frame grabber and the Bluetooth fixes, with sudo) are removed, and so
|
||||||
|
# are the file capabilities of hand tracking's camera broker (ft-camd, with sudo). What runs
|
||||||
|
# now keeps running until you restart the headset.
|
||||||
|
# 2. After the restart, run it again. It deletes the code (~/frametop) and, if you want, your
|
||||||
|
# settings and the build container.
|
||||||
|
# When nothing of Frametop is running, one run does both.
|
||||||
|
#
|
||||||
|
# Options (piped, they go after "bash -s --"):
|
||||||
|
# --dir DIR where the repo is, if not ~/frametop
|
||||||
|
# --dry-run show what it would do, and change nothing
|
||||||
|
set -euo pipefail
|
||||||
|
shopt -s nullglob
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
usage: uninstall.sh [--dir DIR] [--dry-run]
|
||||||
|
piped: curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash -s -- [options]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
dry=0
|
||||||
|
apps=$HOME/.local/share/applications
|
||||||
|
override=$apps/deckard-nested-desktop.desktop
|
||||||
|
native_copy=$apps/native-deckard-nested-desktop.desktop
|
||||||
|
relay_unit=$HOME/.config/systemd/user/frametop-input-relay.service
|
||||||
|
driver=$HOME/.local/share/frametop/ft_pointer
|
||||||
|
vrpathreg=/opt/steamvr/bin/linuxarm64/vrpathreg
|
||||||
|
handsctl=$HOME/.local/bin/ft-handsctl
|
||||||
|
eyegrab_files=(/etc/systemd/system/frametop-eyegrab.service /etc/frametop/ft-eyegrab)
|
||||||
|
bt_files=(/etc/systemd/system/steamframe-bt-fixups.service /etc/systemd/system/bluetooth.service.d/steamframe.conf
|
||||||
|
/etc/steamframe/bt-fixups.sh)
|
||||||
|
|
||||||
|
step() { printf '\n\033[1m== %s\033[0m\n' "$*"; }
|
||||||
|
run() { # run a command, or with --dry-run, show it
|
||||||
|
if [ "$dry" = 1 ]; then printf ' would run: %s\n' "$*"; else "$@"; fi
|
||||||
|
}
|
||||||
|
ask() { # ask "question" default(y|n)
|
||||||
|
local hint answer
|
||||||
|
if [ "$dry" = 1 ]; then echo "$1 [dry run: yes]"; return 0; fi
|
||||||
|
hint=$([ "$2" = y ] && echo "Y/n" || echo "y/N")
|
||||||
|
read -r -p "$1 [$hint] " answer </dev/tty || answer=
|
||||||
|
answer=${answer:-$2}
|
||||||
|
[[ $answer =~ ^[Yy] ]]
|
||||||
|
}
|
||||||
|
exists() { local f; for f in "$@"; do [ -e "$f" ] && return 0; done; return 1; }
|
||||||
|
size() { du -shc "$@" 2>/dev/null | tail -1 | cut -f1; }
|
||||||
|
|
||||||
|
# Is this folder Frametop's code? Only then is it deleted.
|
||||||
|
is_repo() { [ "$1" != "$HOME" ] && [ -f "$1/desktops.sh" ] && [ -f "$1/session/frametop-session.sh" ]; }
|
||||||
|
find_repo() {
|
||||||
|
local p
|
||||||
|
[ -n "$dir" ] && { echo "$dir"; return; }
|
||||||
|
# The launcher entry and the relay's unit point into the repo, until step 1 removes them.
|
||||||
|
p=$(sed -n 's#^Exec=\(.*\)/session/frametop-session\.sh.*#\1#p' "$override" 2>/dev/null | head -1)
|
||||||
|
[ -z "$p" ] && p=$(sed -n 's#^ExecStart=/usr/bin/python3 \(.*\)/input/input-relay\.py.*#\1#p' "$relay_unit" 2>/dev/null | head -1)
|
||||||
|
[ -z "$p" ] && [ -f "${BASH_SOURCE[0]:-}" ] && p=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||||
|
echo "${p:-$HOME/frametop}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Frametop's programs that are running now (step 1 leaves them running until the restart).
|
||||||
|
running() {
|
||||||
|
local n out=()
|
||||||
|
for n in ft-screens ft-pointer ft-powerd ft-eyegrab ft-camd ft-hands; do
|
||||||
|
pgrep -x "$n" >/dev/null && out+=("$n")
|
||||||
|
done
|
||||||
|
pgrep -f '[i]nput/input-relay\.py' >/dev/null && out+=("input relay")
|
||||||
|
pgrep -f '[g]aze/ft-gazed' >/dev/null && out+=("gaze service")
|
||||||
|
pgrep -f '[f]rametop-session\.sh' >/dev/null && out+=("desktop session")
|
||||||
|
echo "${out[*]}"
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
local units=() entries=() names=() sys=0 repo now f
|
||||||
|
while [ $# -gt 0 ]; do
|
||||||
|
case $1 in
|
||||||
|
--dir) dir=${2:?--dir needs a folder}; shift ;;
|
||||||
|
--dry-run) dry=1 ;;
|
||||||
|
-h|--help) usage; return 0 ;;
|
||||||
|
*) echo "unknown option: $1" >&2; usage >&2; return 2 ;;
|
||||||
|
esac
|
||||||
|
shift
|
||||||
|
done
|
||||||
|
|
||||||
|
if ! { grep -qx 'ID=steamos' /etc/os-release && grep -qE '^VARIANT_ID="?vr"?$' /etc/os-release; } 2>/dev/null; then
|
||||||
|
echo "This uninstalls Frametop from a Steam Frame (SteamOS, VR variant). Run it in a terminal on the headset." >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
if [ "$dry" = 0 ] && ! { : </dev/tty; } 2>/dev/null; then
|
||||||
|
echo "This asks questions, and there's no terminal to ask in. Run it in one (over SSH: ssh -t)." >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
[ "$dry" = 1 ] && echo "Dry run: nothing changes."
|
||||||
|
repo=$(find_repo)
|
||||||
|
camd=$repo/hands/build/ft-camd
|
||||||
|
camd_caps=$(getcap "$camd" 2>/dev/null || true)
|
||||||
|
|
||||||
|
# Step 1: what makes Frametop start. Nothing here stops a running program.
|
||||||
|
[ -f "$override" ] && grep -q 'Frametop' "$override" || override=
|
||||||
|
[ -f "$native_copy" ] || native_copy=
|
||||||
|
units=("$HOME"/.config/systemd/user/frametop-*.service)
|
||||||
|
for f in ft-input-settings ft-display-settings ft-layout-reset ft-screens-toggle ft-remote-settings ft-gazeprobe; do
|
||||||
|
[ -e "$apps/$f.desktop" ] && entries+=("$apps/$f.desktop")
|
||||||
|
done
|
||||||
|
[ -e "$apps/frametop-handrec.desktop" ] && entries+=("$apps/frametop-handrec.desktop")
|
||||||
|
entries+=("$apps"/frametop-profile-*.desktop) # one per layout profile (layout/ft_layout.py)
|
||||||
|
[ -L "$handsctl" ] || handsctl=
|
||||||
|
exists "${eyegrab_files[@]}" "${bt_files[@]}" && sys=1
|
||||||
|
|
||||||
|
if [ -n "$override" ] || [ -n "$native_copy" ] || [ ${#units[@]} -gt 0 ] || [ ${#entries[@]} -gt 0 ] ||
|
||||||
|
[ -d "$driver" ] || [ -n "$handsctl" ] || [ "$sys" = 1 ] || [ -n "$camd_caps" ]; then
|
||||||
|
step "Step 1 of 2: stop Frametop from starting"
|
||||||
|
echo "This removes:"
|
||||||
|
[ -n "$override" ] && echo " - the launcher's Desktop entry (Launch a program -> Desktop opens the stock desktop again)"
|
||||||
|
[ -n "$native_copy" ] && echo " - the launcher's Native Desktop entry (Desktop opens the same stock desktop then)"
|
||||||
|
for f in "${units[@]}"; do echo " - the service $(basename "$f")"; done
|
||||||
|
[ -d "$driver" ] && echo " - the 3D mouse's SteamVR driver (ft_pointer)"
|
||||||
|
[ ${#entries[@]} -gt 0 ] && echo " - ${#entries[@]} menu entries (Frametop Display Settings, Input Settings, ...)"
|
||||||
|
[ -n "$handsctl" ] && echo " - $handsctl"
|
||||||
|
[ -n "$camd_caps" ] && echo " - the hand camera broker's file capabilities (needs your password)"
|
||||||
|
exists "${eyegrab_files[@]}" && echo " - our eye tracker's frame grabber (a system service: needs your password)"
|
||||||
|
exists "${bt_files[@]}" && echo " - the Bluetooth fixes (system files: needs your password)"
|
||||||
|
echo "What runs now keeps running until you restart the headset, so your keyboard, mouse, and"
|
||||||
|
echo "this terminal keep working. Your settings and the code stay for now."
|
||||||
|
ask "Uninstall Frametop?" n || { echo "Nothing changed."; return 0; }
|
||||||
|
|
||||||
|
[ -n "$override" ] && run rm -f "$override"
|
||||||
|
[ -n "$native_copy" ] && run rm -f "$native_copy"
|
||||||
|
if [ ${#units[@]} -gt 0 ]; then
|
||||||
|
for f in "${units[@]}"; do names+=("$(basename "$f")"); done
|
||||||
|
run systemctl --user disable "${names[@]}" 2>/dev/null || true
|
||||||
|
run rm -f "${units[@]}"
|
||||||
|
run systemctl --user daemon-reload
|
||||||
|
fi
|
||||||
|
if [ -d "$driver" ]; then
|
||||||
|
if [ -x "$vrpathreg" ]; then
|
||||||
|
LD_LIBRARY_PATH=$(dirname "$vrpathreg") run "$vrpathreg" removedriver "$driver" ||
|
||||||
|
echo "warning: SteamVR's vrpathreg couldn't unregister the driver; SteamVR may log that it's missing" >&2
|
||||||
|
fi
|
||||||
|
run rm -rf "$driver"
|
||||||
|
fi
|
||||||
|
[ ${#entries[@]} -gt 0 ] && run rm -f "${entries[@]}"
|
||||||
|
[ -n "$handsctl" ] && run rm -f "$handsctl"
|
||||||
|
if [ "$sys" = 1 ]; then
|
||||||
|
echo "The system files need your password (sudo)."
|
||||||
|
# $1 is ft-camd when it has capabilities to remove, else empty; the system files follow.
|
||||||
|
if ! run sudo bash -c '
|
||||||
|
camd=$1; shift
|
||||||
|
for u in frametop-eyegrab steamframe-bt-fixups; do systemctl disable $u.service 2>/dev/null; done
|
||||||
|
rm -f "$@"
|
||||||
|
rmdir /etc/frametop /etc/steamframe /etc/systemd/system/bluetooth.service.d 2>/dev/null
|
||||||
|
[ -n "$camd" ] && [ -x "$camd" ] && setcap -r "$camd" 2>/dev/null
|
||||||
|
systemctl daemon-reload; true' sys "${camd_caps:+$camd}" "${eyegrab_files[@]}" "${bt_files[@]}"; then
|
||||||
|
echo "warning: the system files weren't removed (no password?). Run this again to retry." >&2
|
||||||
|
fi
|
||||||
|
elif [ -n "$camd_caps" ]; then
|
||||||
|
echo "ft-camd's file capabilities need your password (sudo) to remove."
|
||||||
|
run sudo setcap -r "$camd" ||
|
||||||
|
echo "warning: ft-camd keeps its capabilities (no password?). Run this again to retry." >&2
|
||||||
|
fi
|
||||||
|
[ "$dry" = 1 ] || echo "Frametop no longer starts."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Step 2: delete what's left, once nothing of Frametop runs.
|
||||||
|
now=$(running)
|
||||||
|
if [ -n "$now" ]; then
|
||||||
|
step "Restart the headset to finish"
|
||||||
|
echo "Still running from before: $now."
|
||||||
|
echo "They stop when the headset restarts. After that, run this again to delete the code"
|
||||||
|
echo "($repo) and, if you want, your settings and the build container:"
|
||||||
|
echo
|
||||||
|
if [ "$repo" = "$HOME/frametop" ]; then
|
||||||
|
echo " curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash"
|
||||||
|
else
|
||||||
|
echo " curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash -s -- --dir $(printf %q "$repo")"
|
||||||
|
fi
|
||||||
|
echo
|
||||||
|
if ask "Restart the headset now? This closes everything open, in VR and on the desktop." n; then
|
||||||
|
run systemctl reboot || echo "Couldn't restart it from here: restart the headset from Steam's power menu." >&2
|
||||||
|
fi
|
||||||
|
[ "$dry" = 1 ] || return 0
|
||||||
|
echo "Dry run: after the restart, step 2 would go like this."
|
||||||
|
fi
|
||||||
|
|
||||||
|
step "Step 2 of 2: delete what's left"
|
||||||
|
if [ -d "$repo" ] && is_repo "$repo"; then
|
||||||
|
if [ -f "$repo/.git" ]; then
|
||||||
|
echo "Leaving $repo: it's a git worktree. Remove it with git worktree remove."
|
||||||
|
else
|
||||||
|
local def=y
|
||||||
|
if [ -n "$(git -C "$repo" status --porcelain --untracked-files=no 2>/dev/null)" ] ||
|
||||||
|
[ -n "$(git -C "$repo" log --branches --not --remotes --oneline 2>/dev/null | head -1)" ]; then
|
||||||
|
echo "$repo has changes of its own (git -C $repo status)."
|
||||||
|
def=n
|
||||||
|
fi
|
||||||
|
if ask "Delete the Frametop code in $repo ($(size "$repo"))?" "$def"; then
|
||||||
|
cd "$HOME"
|
||||||
|
run rm -rf "$repo"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
elif [ -e "$repo" ]; then
|
||||||
|
echo "Leaving $repo: it doesn't look like Frametop's code."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Rebuilt by the desktop at each start, so nothing to keep.
|
||||||
|
run rm -rf "$HOME/.local/share/frametop/apps" "$HOME/.local/share/kwin/decorations/kwin4_decoration_qml_frametop" \
|
||||||
|
"$HOME/.cache/frametop"
|
||||||
|
|
||||||
|
local settings=("$HOME"/.config/frametop.conf* "$HOME"/.config/frametop-*.json* "$HOME/.config/frametop-remote"
|
||||||
|
"$HOME/.config/frametop" "$HOME/.local/state/frametop")
|
||||||
|
local kept=()
|
||||||
|
for f in "${settings[@]}"; do [ -e "$f" ] && kept+=("$f"); done
|
||||||
|
if [ ${#kept[@]} -gt 0 ]; then
|
||||||
|
echo "Your settings: the screen layout and profiles, button maps, gaze calibration, the remote"
|
||||||
|
echo "desktop password, and the Frametop desktop's own Plasma setup (${kept[*]/#$HOME/\~})."
|
||||||
|
ask "Delete your settings too? Keep them to pick up where you left off if you reinstall." n &&
|
||||||
|
run rm -rf "${kept[@]}"
|
||||||
|
fi
|
||||||
|
|
||||||
|
local recs=()
|
||||||
|
for f in "$HOME/.local/share/frametop/eyes" "$HOME/.local/share/frametop/hands"; do [ -e "$f" ] && recs+=("$f"); done
|
||||||
|
if [ ${#recs[@]} -gt 0 ]; then
|
||||||
|
ask "Delete your eye and hand recordings in ~/.local/share/frametop ($(size "${recs[@]}"))?" n &&
|
||||||
|
run rm -rf "${recs[@]}"
|
||||||
|
fi
|
||||||
|
[ "$dry" = 1 ] || rmdir "$HOME/.local/share/frametop" 2>/dev/null || true
|
||||||
|
|
||||||
|
if command -v podman >/dev/null && podman container exists dev 2>/dev/null; then
|
||||||
|
echo "The build container (dev) holds Frametop's compilers and libraries, 1-2 GB. If you put"
|
||||||
|
echo "anything else in it, that goes too."
|
||||||
|
if ask "Delete the build container?" n; then
|
||||||
|
run "$HOME/.local/bin/distrobox" rm --force dev </dev/null
|
||||||
|
run podman image rm registry.fedoraproject.org/fedora-toolbox:44 >/dev/null 2>&1 || true
|
||||||
|
echo "distrobox stays in ~/.local/bin, for any other containers. To remove it too:"
|
||||||
|
echo " ~/dev/src/distrobox/uninstall --prefix ~/.local"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
step "Done"
|
||||||
|
echo "Frametop is uninstalled."
|
||||||
|
}
|
||||||
|
|
||||||
|
dir=
|
||||||
|
main "$@"
|
||||||
Reference in new issue
Block a user