Compare commits

..
Author SHA1 Message Date
DeeJanuzandClaude Opus 5.5 3a796b5013 Gaze: the first calibration no longer runs twice
Turning gaze mode on wakes our tracker, so its eyes come back just as the
first calibration opens. DON_DELAY later, "the headset went on" re-armed the
automatic calibration while it was still open, and when it ended, ft-eyes'
status was still a moment old (not calibrated), so a second one opened at
once. Seen live on 2026-10-05: 26 of 27 dots, then 27 more.

The headset going on re-arms it only when no check is open.
first-calibration-test.py now has the headset go on mid-calibration and
checks that only one opens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 14:28:15 -06:00
DeeJanuz 67a3c02e31 ft-camd: map a camera whose dark frames are all zeros
Through the ISP (no colour module), the near-black exposures come out
all zeros, identical every time, so their dequeues change no buffer and
ft-camd never finished learning the side cameras' buffers ("can't tell
which buffers are this camera's yet"): the side pair published nothing.
Such an index is now left unmapped and its frames counted as dark.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 89ec1608d9)
2026-10-05 13:55:18 -06:00
DeeJanuz 662919babf camcheck: no nested parentheses in the missing cameras' message
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 917ef9d29c)
2026-10-05 13:55:17 -06:00
DeeJanuz 98bd6ce21c Hand recorder: headsets without the colour module
Without the Arcturus module, XRService runs the side cameras through the
ISP ("ISP enabled for tracking cameras (main VFE available)"): NV12 on
vfe0 and vfe1, pitch 1152. ft-camd published only grey cameras, so it
dropped them, and the recorder stopped at "ft-camd publishes only 2 of 4
mono cameras" whatever was restarted.

- ft-camd reads a tracking camera's NV12 luma as its grey image.
- ft-hands and check_sides name the cameras by XRService's numbering in
  its log (TrackingCameraInit index 0-3), not by capture pipe, which
  moves with the module; a camera whose size isn't its calibration's is
  left out. sides.json says which device each camera was.
- camcheck reports the camera map, the ISP routing and which cameras
  ft-camd is missing; the recorder says so instead of "restart it",
  restarts its own ft-camd once when that's the only problem, and keeps
  the map in session.json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 4db5266cf7)
2026-10-05 13:55:17 -06:00
DeeJanuz e1f27958ca Merge remote-tracking branch 'origin/experimental' into experimental 2026-10-05 11:12:07 -06:00
DeeJanuzandClaude Opus 5.5 c969963b83 A one-command uninstaller that keeps the headset usable
uninstall.sh replaces the README's thirteen uninstall commands. Run from a
Frametop desktop, those commands stopped the input relay second, which took
the keyboard and mouse away before the rest could be typed. And deleting
~/frametop first left Launch a program -> Desktop pointing at a missing
script.

The script works in two steps. Step 1 stops Frametop from starting: the
launcher entry, the user services (disabled, not stopped), the SteamVR
driver, the menu entries, and the eye grabber and Bluetooth fixes under
/etc (one sudo). Everything running keeps running until the headset
restarts, and the script offers to restart it. Step 2 runs once none of
Frametop's programs run. It deletes the code, and asks before deleting the
settings, recordings, and the dev container.

It doesn't use the rest of the repo, so the Pages one-liner works even when
~/frametop is gone. --dry-run shows each command without running it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 11:11:57 -06:00
DeeJanuzandClaude Opus 5.5 dfc8e6eb5f Merge branch fresh-install (our eye tracker's first calibration; SteamVR's tools from a terminal in the desktop; gaze in the report) into experimental
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:55:22 -06:00
DeeJanuzandClaude Opus 5.5 1037d4e125 Report: the gaze service, our eye tracker, and their logs
A gaze problem couldn't be told from a report: it had no gaze section. A user's report of
2026-10-05 showed only that the gaze service ran and our tracker was picked.

The report now has a Gaze section: whether the gaze service and our tracker's frame grabber
are enabled and running, when SteamVR's correction and our tracker's calibration were made
(or "none"), and the service's status (ft-gazectl status, on one line: the tracker in use,
whether it's awake and why not, checks.problem). Then the last 10 checks and calibration dots
(checks.jsonl: each dot's reply and whether it was taken), the gaze service's last 60 lines
without podman's exec lines, and the frame grabber's last 20. Nothing in them is an eye image.

Tested on this Frame: each section fills in, and the status is one line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:54:57 -06:00
DeeJanuzandClaude Opus 5.5 307e2b7308 SteamVR's tools find SteamVR from a terminal in the Frametop desktop
The desktop has its own XDG_CONFIG_HOME (~/.config/frametop), and SteamVR's tools and OpenVR
read their path registry from there. So from a terminal in the desktop:

- pointer/driver/install.sh (install.sh step 4, every reinstall or update from Konsole) wrote
  a new registry, ~/.config/frametop/openvr/openvrpaths.vrpath, with only our driver and
  "runtime": null, and never registered the driver with SteamVR. That stray file then hid
  SteamVR from every OpenVR program started in the desktop.
- scripts/update-check.py (and so doctor.sh and report.sh) reported OpenVR "can't connect to
  SteamVR as a background app" (VRInitError_Init_PathRegistryNotFound, or
  InstallationNotFound with the stray file) and "vrcmd --overlays lists no overlays". A user's
  report of 2026-10-05 showed both, with SteamVR and Frametop's services fine.
- ft-layout's vrcmd calls (the gamescope backend) found no overlays.

They now run with XDG_CONFIG_HOME=~/.config: the driver installer (install, uninstall, probe,
aimhere), update-check.py (for all its checks: nothing there reads the desktop's own config),
report.sh's vrpathreg show, and ft-layout's vrcmd. report.sh doesn't set it for the whole
report, since session/fix-panels.py --check reads the desktop's Plasma config through it.

The driver installer also removes the stray registry (only vrpathreg's, with no runtime), and
update-check.py warns about one. And vrpathreg runs `xdg-open vrmonitor://driverinstalled` to
tell SteamVR's desktop monitor, which the Frame doesn't have: in the desktop that showed "could
not read file vrmonitor://driverinstalled" on every install. A stand-in xdg-open on its PATH
takes it, so install.sh no longer filters the step's output.

Tested in a fake HOME with a copy of SteamVR's registry, a stray one, and the desktop's
XDG_CONFIG_HOME: the new installer removes the stray file, registers the driver in the copy,
and never reaches xdg-open; the old one wrote the stray file, left the copy alone, and ran
xdg-open. update-check.py from that environment: OpenVR and vrcmd ok (the old one fails both);
its stray-registry warning fires on a seeded file. ft-layout's vrcmd: 119 overlays (was 0).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:53:59 -06:00
DeeJanuzandClaude Opus 5.5 d39fbb9146 Gaze: our eye tracker can take its first calibration
A fresh install that chose our tracker (the installer's default since 12ad93d) could never
calibrate it, so gaze mode never worked. ft-eyes maps pupils to a gaze only once it has a
calibration, so before its first one ft-gaze's "own" source is empty. ft-gazed then never
counted a sample, Checks.can_run() stayed false, and the calibration refused to open: "Not
calibrated, and the calibration can't open: the eye tracker isn't sending", or "the headset is
off or the tracker isn't sending" from Calibrate. This Frame never hit it, because its
calibration came from the gaze probe. Reported 2026-10-05 on SteamOS stable.

- With our tracker in use, running, and uncalibrated ("blind"), SteamVR's tracker seeing an
  eye is enough to open the full calibration (ft-eyes answering is what makes calibrated()
  False rather than None).
- Each dot stands in for the gaze, so a click takes the CHECK_WINDOW up to it. ft-eyes already
  checks, in calib-point, that each pupil was seen in enough frames and held still then, and
  its reason for a refused dot shows in the panel's note as before.
- A quick or five-dot check is refused until then ("use Calibrate"): ours can't take a click
  without a calibration.
- With no eyes seen, the reason given is that, not "the eye tracker isn't sending".

gaze/test/first-calibration-test.py runs the service against a fake ft-gaze, ft-eyes and
pointer helper, in a temp HOME: the calibration opens by itself when gaze mode comes on, each
click sends ft-eyes the 0.6 s before it, a refused dot's reason reaches the panel, and
calib-fit leaves the tracker calibrated with gaze mode still on. It passes here and fails on
the previous code. gaze/test/idle-test.py still passes. Not yet tested in the headset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 10:51:07 -06:00
DeeJanuzandClaude Opus 5.5 7fe5d8c0dd Merge branch nested-atspi (PR #28: the desktop starts an accessibility (AT-SPI) registry, #27) into experimental
JakeGreen2145's PR plus our fix 6df6f97: the watcher checks its buses
every 5 seconds instead of every second (about 1% of a core), and
design.md says the watcher is the only cleanup when the VR launcher
runs the desktop in steam.service.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 09:27:26 -06:00
20 changed files with 857 additions and 67 deletions

No files matched your search

+1 -1
View File
@@ -26,7 +26,7 @@ A Steam Frame is someone's personal headset, and they may be wearing it while yo
- Don't kill or restart `gamescope`, `steam`, `vrserver`, `vrcompositor`, the gamescope session, or the Frametop desktop without asking. Each one ends or disrupts whatever is happening in VR.
- Don't run host `sudo`, `steamos-readonly disable`, `steamos-devmode` changes, pacman installs, or reboots without explicit approval. Three installers need host `sudo`, and they ask for it: the Bluetooth fixes (`setup/bluetooth/install.sh`), hand tracking (`hands/run.sh install` and `caps`, which set ft-camd's file capabilities with `setcap`), and our own eye tracker's frame grabber (`gaze/tracker/install.sh`).
- Write only inside the repo, `/tmp`, and the container unless told otherwise. The installers are the exception: they write the user services, launchers, and the SteamVR driver into the home folder. The Bluetooth fixes and the eye tracker's frame grabber also install root-owned files and system services under `/etc` (`/etc/steamframe`, `/etc/frametop`, `/etc/systemd/system`).
- Write only inside the repo, `/tmp`, and the container unless told otherwise. The installers are the exception: they write the user services, launchers, and the SteamVR driver into the home folder. The Bluetooth fixes and the eye tracker's frame grabber also install root-owned files and system services under `/etc` (`/etc/steamframe`, `/etc/frametop`, `/etc/systemd/system`). When an installer starts writing something new outside the repo, add it to `uninstall.sh` too: users uninstall with that script, not with each installer's `uninstall`.
- Never copy `.netrc`, SSH keys, or Steam config off the Frame or into this repo.
## SteamOS updates
+14 -14
View File
@@ -171,23 +171,22 @@ Or by hand: `cd ~/frametop && git pull && ./install.sh`.
## Uninstall
In a terminal on the headset, run:
```
./desktops.sh uninstall # the launcher's Desktop entry goes back to the stock desktop
./desktops.sh relay uninstall
pointer/helper/run.sh uninstall
power/run.sh uninstall
pointer/driver/install.sh uninstall # then restart SteamVR
input-settings/install.sh uninstall
display-settings/install.sh uninstall
remote/install.sh uninstall
setup/bluetooth/install.sh uninstall # if you installed the Bluetooth fixes
hands/run.sh uninstall # if you installed hand tracking by hand
gaze/run.sh uninstall # if you installed the gaze service
gaze/tracker/install.sh uninstall # if you installed our own eye tracker's frame grabber
gaze/probe/install.sh uninstall # if you installed the gaze probe
curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash
```
Your settings stay: `~/.config/frametop.conf`, `frametop-input.json` (button maps and key combinations), `frametop-layout.json` (the layout and profiles), `frametop-float.json`, and `frametop-remote/` in `~/.config`, and the gaze calibration in `~/.local/state/frametop/gaze`. So does the desktop's own Plasma setup, in `~/.config/frametop`. Delete them too for a clean slate.
It works in two steps, so it never takes away the keyboard, mouse, or desktop you're using while it runs:
1. It stops Frametop from starting. Launch a program → Desktop opens the stock desktop again, and Frametop's services, its SteamVR driver, its menu entries, and the system files of our eye tracker and the Bluetooth fixes are removed (those need your `sudo` password). Everything running now keeps running until you restart the headset, and it offers to restart it for you.
2. After the restart, run the same command again. It deletes the code in `~/frametop`, and asks whether to delete your settings, any eye or hand recordings, and the build container (1–2 GB) too.
To see what it would do without changing anything, add `-s -- --dry-run` after `bash`. If the code isn't in `~/frametop`, add `-s -- --dir <folder>`. From the repo, the same script is `./uninstall.sh`.
Don't delete `~/frametop` by hand before you uninstall and restart: the desktop and the input relay run from it, and without it Launch a program → Desktop no longer opens anything. If you've already deleted it, the command above still works, since it doesn't need the repo.
Unless you ask for them to go, your settings stay: `~/.config/frametop.conf`, `frametop-input.json` (button maps and key combinations), `frametop-layout.json` (the layout and profiles), `frametop-float.json`, and `frametop-remote/` in `~/.config`, the gaze calibration in `~/.local/state/frametop`, and the desktop's own Plasma setup in `~/.config/frametop`. A later install picks them up again.
## How it works
@@ -197,6 +196,7 @@ A Plasma session runs nested inside ft-screens (`screens/`), a small Wayland com
| --- | --- |
| `get.sh` | The one-line installer: picks stable or experimental, clones or updates the repo, and runs `install.sh`. |
| `install.sh` | The one-step installer. Safe to re-run. |
| `uninstall.sh` | The uninstaller: run it, restart the headset, and run it again. It doesn't need the rest of the repo. |
| `desktops.sh` | Start, stop, and configure the desktop, and install the input relay. |
| `screens/` | ft-screens, the compositor (wlroots and OpenVR). |
| `session/` | The desktop session script and its config example. |
+1 -1
View File
@@ -169,7 +169,7 @@ The Frame controllers can be mapped like mouse buttons, but they aren't input de
## The desktop session
The session is modeled on SteamOS's `steamos-nested-desktop` and runs beside it. It has its own runtime directory, config (`~/.config/frametop`), and state, so it never disturbs the stock desktop's layout or panels. It runs on a private D-Bus from `dbus-run-session`, which has two consequences. KDE only launches apps in systemd scopes when systemd is on the session bus, so everything started in the desktop lands in its systemd unit, and stopping the unit would kill all of it; `session/keep-apps.sh` moves those programs out first. And tools that need the real user bus, like podman and `distrobox-host-exec`, have to be pointed at it explicitly.
The session is modeled on SteamOS's `steamos-nested-desktop` and runs beside it. It has its own runtime directory, config (`~/.config/frametop`), and state, so it never disturbs the stock desktop's layout or panels. It runs on a private D-Bus from `dbus-run-session`, which has two consequences. KDE only launches apps in systemd scopes when systemd is on the session bus, so everything started in the desktop lands in its systemd unit, and stopping the unit would kill all of it; `session/keep-apps.sh` moves those programs out first. And tools that need the real user bus, like podman and `distrobox-host-exec`, have to be pointed at it explicitly. Its own config folder also hides SteamVR's path registry (`~/.config/openvr/openvrpaths.vrpath`) from everything started in it: OpenVR programs there fail with `VRInitError_Init_PathRegistryNotFound`, and `vrpathreg adddriver` writes a new registry under `~/.config/frametop/openvr` that has no SteamVR in it and that SteamVR never reads. So Frametop's scripts run SteamVR's tools with `XDG_CONFIG_HOME=~/.config`.
The VR launcher starts the session from the Steam client, and the client's environment came along: `LD_LIBRARY_PATH` pointing at Steam's own runtime, whose `libavcodec` has no H.264 decoder, so VLC in the desktop couldn't play most videos, plus the client's overlay and launch settings. The session script drops the client's variables before it starts anything. SteamOS's global Mesa settings (`/usr/share/deckard/mesavars.sh`) stay, and the gamescope session's Vulkan layer (`ENABLE_GAMESCOPE_WSI`) is only kept for the gamescope backend.
+1 -1
View File
@@ -33,7 +33,7 @@ Gaze as an input method for the whole desktop, without replacing anything of Ste
- **The mouse only corrects** (the default; the Gaze page's Mouse movement switch, `POINTER_GAZE_MOUSE_MOVE=held`): while the gaze has the pointer, moving the mouse does nothing. The buttons work like Meta+J and Meta+K: press and hold one and the pointer stops where you look; move the mouse onto what you meant and let go to click there (a left or a right click). Held still for half a second, a press is a real one (to drag). Once you've moved, the left button alone only clicks: press the right one while still holding the left to start a drag there; it lasts while either button is held. Press the right one again (a double right click, the left still held) to pan and tilt what you're dragging, as a right press does during any drag. A bumped or drifting mouse can't pull the pointer away, and every mouse move is a correction, so the tracker only learns from real ones. With the gaze stale for a second (the tracker stopped, eyes lost), in a game, or with the headset off, the mouse moves the pointer as usual. `free` (the switch off) lets the mouse take the pointer any time.
- **Keyboard clicks** (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, `POINTER_KEY_TAP`) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse, under the same limit: past `POINTER_GAZE_NUDGE_MAX` it opens the quick check instead). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; the drag lasts while either key is held. Meta+K during a Meta+J drag (again, after starting it with Meta+K: a double Meta+K) pans and tilts what you're dragging while it's held: turn your head to turn it.
- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it (0.2 degrees or more, and the correction within `POINTER_GAZE_NUDGE_MAX`: 55 degrees by default, half of the 109 the headset shows across, and 1 to 110; the same limit for mouse, keyboard, and pinch clicks) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the quick check's dot does the same). A correction past `POINTER_GAZE_NUDGE_MAX` isn't learned: the helper asks ft-gazed for the quick check instead ("recheck", after its 2-minute cooldown). Tested on our tracker's 409 clicks since its Sep 29 calibration: a one-dot check set from any one of them put the next 2 minutes' clicks within 15 degrees (99% within 4.2) and the next 10 minutes' within 25 (the far ones after the headset moved), so the check gets back well under it. The limit used to be 8 degrees, and live on 2026-10-01 our tracker was 12 off after the headset went on, so every correction was dropped. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. Our tracker's first calibration has no gaze to go on, since ft-eyes maps pupils to a gaze only once it has a calibration: it opens once SteamVR's tracker sees an eye and ft-eyes answers, and a click takes the 0.6 s up to it, as long as ft-eyes saw each pupil held still then (before 2026-10-05 it waited for a gaze, so a fresh install could never calibrate ours). A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
- Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
- **Idle while the gaze isn't used:** ft-gaze and our own tracker run only while gaze mode is on and someone wears the headset (the pointer helper says both: SteamVR drops the headset's activity level as soon as it comes off), while a check or the calibration is open or asked for, or while the Gaze page of Frametop Input Settings is open (it renews a `wake` lease). 30 seconds after the last use they stop, and our frame grabber goes idle with our tracker. With our tracker, that saves over half a core: on 2026-10-02, with gaze mode off, ft-eyes took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. A check asked for while it idles starts the tracker and opens once it sends. When the gaze is used again, it takes a few seconds to come back, and our tracker's first click re-seats it, as after the headset was off: so the quick check opens when gaze mode comes on after the service idled, as it does when you put the headset on. `ft-gazectl status` says `"awake"`, and `"idle"` says why it isn't. A stand that covers the proximity sensor makes the headset seem worn, so with gaze mode on it doesn't idle there.
+35 -5
View File
@@ -48,6 +48,13 @@ click with nothing taken after ACCEPT_WAIT, and a failed calibration names its m
reason there and in the status. A right click or Meta+K ("calquit") closes the panel. The pointer hides meanwhile ("calpanel 1",
renewed every second; the helper shows it again by itself when that stops).
Our tracker's first calibration: before it has one, ft-eyes publishes no gaze (it maps pupils
to a gaze only with a calibration), so there's no gaze to hold still. Its calibration runs
anyway ("blind"): someone in the headset (SteamVR's tracker sees an eye) and ft-eyes answering
are enough to start it, each dot stands in for the gaze, and a click takes the CHECK_WINDOW up
to it. ft-eyes then checks that each pupil was seen and held still in that window (calib-point)
and says why not. Without this, a fresh install could never calibrate our tracker.
What a capture teaches:
our tracker quick and five: a click ("click T YAW PITCH", like a pointer lesson); full:
calib-start, a calib-point for each dot, calib-fit (its calibration)
@@ -336,9 +343,17 @@ class Checks:
return time.monotonic() - self.seen_at < within
def can_run(self):
"""Someone's in the headset and the tracker is sending."""
"""Someone's in the headset and the tracker is sending. Our tracker sends no gaze before
its first calibration: then ft-eyes answering (calibrated() is False only once it has)
is enough, since the calibration is what it needs (see "first calibration" at the top)."""
if self.blind():
return self.eyes_seen()
return self.eyes_seen() and time.monotonic() - self.svc.last_sample < 2
def blind(self):
"""Our tracker is in use, running, and not calibrated yet: it has no gaze to send."""
return self.svc.kind == "own" and self.calibrated() is False
def on_gaze_on(self):
self.full_armed = True
self.need_full("gaze mode came on without a calibration")
@@ -356,7 +371,7 @@ class Checks:
if not self.can_run() and self.svc.waking():
return # the tracker is still starting (the service idled)
if not self.can_run():
why = ("the eye tracker isn't sending" if now - self.svc.last_sample >= 2
why = ("the eye tracker isn't sending" if now - self.svc.last_sample >= 2 and not self.blind()
else "no eyes seen (is the headset on?)")
elif now < self.full_retry_at:
return
@@ -408,6 +423,9 @@ class Checks:
if not self.can_run():
return "error the headset is off or the tracker isn't sending"
own = svc.kind == "own"
blind = self.blind()
if blind and kind != "full":
return "error our tracker isn't calibrated yet: use Calibrate"
if kind == "full" and own:
reply = ask(EYES, "calib-start", 3.0)
if not reply.startswith("ok"):
@@ -416,8 +434,10 @@ class Checks:
now = time.monotonic()
self.check = {"kind": kind, "reason": reason, "own": own, "dots": check_dots(kind, own), "i": 0,
"started": now, "shown": now, "run": [], "accept": False, "done_at": None, "tries": 0,
"skipped": 0, "captured": 0, "points": {}, "reasons": {}, "fit_reasons": set(), "note": ""}
log(f"{kind} check: {reason}")
"skipped": 0, "captured": 0, "points": {}, "reasons": {}, "fit_reasons": set(), "note": "",
"blind": blind}
log(f"{kind} check: {reason}" + (" (our tracker's first: no gaze yet, so each click takes the look "
"up to it)" if blind else ""))
if kind == "full":
st = ask(SCREENS, "state", 0.5).split()
if len(st) >= 3 and st[0] == "ok":
@@ -519,6 +539,12 @@ class Checks:
return
if c["own"]:
src = s["src"].get("own") or {}
if c["blind"]:
# Our tracker's first calibration: no gaze yet, so the dot stands in for it (the
# gaze can't seem to move) and a click takes the look up to it. ft-eyes checks
# the pupils held still (see the top).
yaw, pitch, _ = c["dots"][c["i"]]
src = {"hy": yaw, "hp": pitch}
else:
src = s["src"].get("mmap1") or {}
if "hy" not in src:
@@ -868,7 +894,11 @@ class Checks:
self.back_since = None # gone again before DON_DELAY
elif self.back_since is not None and now - self.back_since >= DON_DELAY:
self.away, self.back_since = False, None
self.full_armed = True # a calibration that closed unfinished opens again
if not self.check:
# A calibration that closed unfinished opens again. Not one still open: gaze mode
# coming on wakes our tracker, so its eyes come back just as the calibration
# opens, and re-arming then opened a second one when the first ended.
self.full_armed = True
self.auto_quick("the headset went on")
if svc.kind == "own" and now - svc.own_at < 5:
reseat = any(e.get("reseat") for e in (svc.own.get("eyes") or {}).values())
+238
View File
@@ -0,0 +1,238 @@
#!/usr/bin/env python3
"""Offline test of our own eye tracker's first calibration (gaze/gazecheck.py, "blind"): before
it has a calibration, ft-eyes publishes no gaze, and the calibration must still open and take
its dots. Until 2026-10-05 it couldn't: a fresh install that chose our tracker never calibrated.
Runs ft-gazed's Service with its sockets renamed and HOME in a temp folder (state and settings
go there), a fake pointer helper (gaze mode on, headset worn), a fake ft-gaze (SteamVR sees both
eyes; "own" is {"ok":0}, as with an uncalibrated ft-eyes), a fake ft-eyes control socket, and no
panel (a stand-in process). Nothing reaches the live gaze service, the pointer helper, ft-eyes,
or SteamVR, so it's safe next to them.
gaze/test/first-calibration-test.py
"""
import os
import tempfile
HOME = tempfile.mkdtemp(prefix="ft-gaze-first-cal-test-")
os.environ["HOME"] = HOME # before gazecal: its STATE, and frametop.conf, follow HOME
import importlib.machinery # noqa: E402
import importlib.util # noqa: E402
import json # noqa: E402
import selectors # noqa: E402
import shutil # noqa: E402
import socket # noqa: E402
import subprocess # noqa: E402
import sys # noqa: E402
import threading # noqa: E402
import time # noqa: E402
HERE = os.path.dirname(os.path.abspath(__file__))
GAZE = os.path.join(HERE, "..")
sys.path.insert(0, GAZE)
loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(GAZE, "ft-gazed"))
gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
loader.exec_module(gazed)
import gazecheck # noqa: E402 (the module ft-gazed imported)
tag = f"ft_gaze_first_cal_test_{os.getpid()}"
gazed.ME = f"\0{tag}_gazed"
gazed.POINTER = gazecheck.POINTER = f"\0{tag}_helper"
gazecheck.SCREENS = f"\0{tag}_screens"
gazecheck.PANEL = f"\0{tag}_panel"
gazed.EYES_SOCKET = gazecheck.EYES = f"\0{tag}_eyes"
gazed.read_settings = lambda: ("own", "auto", "auto", 55.0)
DOTS = 3
real_dots = gazecheck.check_dots
gazecheck.check_dots = lambda kind, own: real_dots(kind, own)[:DOTS] # a short calibration
logs = []
gazed.log = gazecheck.log = lambda msg: logs.append(msg)
# The fake ft-gaze: 90 samples a second, SteamVR sees both eyes, no gaze from ours.
FAKE = os.path.join(HOME, "ft-gaze")
with open(FAKE, "w") as f:
f.write('''import json, os, select, sys, time
while True:
if select.select([sys.stdin], [], [], 1 / 90)[0]:
if not os.read(0, 4096):
break
print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
"open": [0.8, 0.8]}, "own": {"ok": 0}}}), flush=True)
''')
SLEEPER = [sys.executable, "-c", "import sys; sys.stdin.read()"] # quits when its stdin closes
def start_helper(self):
"""ft-gaze, straight from here instead of the dev container."""
self.proc = subprocess.Popen([sys.executable, FAKE], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
self.proc_sources = self.wanted_sources()
os.set_blocking(self.proc.stdout.fileno(), False)
os.set_blocking(self.proc.stderr.fileno(), False)
self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
self.buf = b""
def start_eyes(self):
"""ft-eyes' process: a stand-in. Its control socket is the fake below."""
self.eyes_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
os.set_blocking(self.eyes_proc.stderr.fileno(), False)
self.sel.register(self.eyes_proc.stderr, selectors.EVENT_READ, "eyes")
def start_panel(self):
self.panel_proc = subprocess.Popen(SLEEPER, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
os.set_blocking(self.panel_proc.stderr.fileno(), False)
self.sel.register(self.panel_proc.stderr, selectors.EVENT_READ, "panel")
gazed.Service.start_helper = start_helper
gazed.Service.start_eyes = start_eyes
gazecheck.Checks.start_panel = start_panel
# The fake ft-eyes: uncalibrated until calib-fit. "fail" answers the next calib-point with that.
eyes_state = {"cal": None, "points": [], "fail": None}
eyes = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
eyes.bind(gazed.EYES_SOCKET)
eyes.settimeout(0.2)
def eyes_answer():
while True:
try:
data, addr = eyes.recvfrom(512)
except socket.timeout:
continue
except OSError:
return
w = data.decode().split()
if w[0] == "status":
reply = json.dumps({"calibration": eyes_state["cal"], "calibrating": False, "dots": 0,
"eyes": {"right": {"reseat": False}, "left": {"reseat": False}}})
elif w[0] == "calib-start":
eyes_state["points"] = []
reply = "ok"
elif w[0] == "calib-point":
eyes_state["points"].append(tuple(map(float, w[1:5])))
reply, eyes_state["fail"] = eyes_state["fail"] or "ok 50 50 1.00 1.00", None
elif w[0] == "calib-fit":
eyes_state["cal"] = {"made": "test", "dots": len(eyes_state["points"])}
reply = f"ok {len(eyes_state['points'])} dots"
else:
reply = f"fail unknown command {w[0]}"
if addr:
eyes.sendto(reply.encode(), addr)
threading.Thread(target=eyes_answer, daemon=True).start()
helper_state = {"reply": "ok off worn", "heard": []}
helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
helper.bind(gazed.POINTER)
helper.settimeout(0.2)
def helper_answer():
while True:
try:
data, addr = helper.recvfrom(512)
except socket.timeout:
continue
except OSError:
return
if data.startswith(b"gaze ?") and addr:
helper.sendto(helper_state["reply"].encode(), addr)
else:
helper_state["heard"].append(data.decode())
threading.Thread(target=helper_answer, daemon=True).start()
svc = gazed.Service(None, False, gazed.POINTER)
threading.Thread(target=svc.run, daemon=True).start()
ctl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
ctl.bind("")
ctl.settimeout(3)
def ask(cmd):
ctl.sendto(cmd.encode(), gazed.ME)
return ctl.recv(65536).decode()
failures = []
def check(label, got, want):
ok = got == want
print(("ok " if ok else "FAIL ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
if not ok:
failures.append(label)
def wait(cond, seconds):
end = time.monotonic() + seconds
while time.monotonic() < end:
if cond():
return True
time.sleep(0.05)
return cond()
def check_state():
return svc.checks.check or {}
def take_dot(i):
"""Wait for dot i to settle, then click: is it taken?"""
wait(lambda: check_state().get("i") == i and not check_state().get("done_at"), 3)
time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1)
before = check_state().get("captured", 0)
ask("calaccept")
return wait(lambda: check_state().get("captured", 0) > before, 2)
check("gaze mode off, Gaze page open (wake): ours runs, uncalibrated", ask("wake 60"), "ok")
check("the service knows ours has no calibration", wait(lambda: svc.checks.calibrated() is False, 6), True)
check("a quick check is refused while ours has no calibration", svc.checks.start("quick", "test"),
"error our tracker isn't calibrated yet: use Calibrate")
helper_state["reply"] = "ok on worn"
check("gaze mode on: the calibration opens by itself", wait(lambda: check_state().get("kind") == "full", 6), True)
check("it runs blind (no gaze from ours yet)", check_state().get("blind"), True)
check("nothing says it can't open", any("can't open" in m for m in logs), False)
# Live 2026-10-05: turning gaze mode on woke our tracker, and the calibration opened before
# DON_DELAY of eyes had passed, so "the headset went on" came while it was open. That re-armed
# it, and a second calibration opened as soon as the first ended.
svc.checks.away, svc.checks.back_since = True, None
check("dot 1: a click takes it", take_dot(0), True)
check("the headset went on while it was open",
wait(lambda: not svc.checks.away, gazecheck.DON_DELAY + 2) and bool(svc.checks.check), True)
t0, t1, yaw, pitch = eyes_state["points"][0]
dot = gazecheck.check_dots("full", True)[0]
check("its window is the look up to the click (about CHECK_WINDOW)",
abs((t1 - t0) - gazecheck.CHECK_WINDOW) < 0.15, True)
check("ft-eyes got that dot's direction", (round(yaw, 3), round(pitch, 3)), (round(dot[0], 3), round(dot[1], 3)))
eyes_state["fail"] = "fail the left eye was seen in only 3 frames"
wait(lambda: check_state().get("i") == 1 and not check_state().get("done_at"), 3)
time.sleep(gazecheck.CHECK_SETTLE + gazecheck.CHECK_WINDOW + 0.1)
ask("calaccept")
check("ft-eyes refusing a dot: its reason reaches the panel's note",
wait(lambda: "left eye in only 3 frames" in check_state().get("note", ""), 2), True)
check("dot 2, second try: taken", take_dot(1), True)
check("dot 3: taken", take_dot(2), True)
check("all dots: ours fits its calibration (calib-fit)",
wait(lambda: eyes_state["cal"] is not None and not svc.checks.check, 4), True)
check("the service sees it calibrated", wait(lambda: svc.checks.calibrated() is True, 4), True)
check("and gaze mode stays on", "gaze off" in helper_state["heard"], False)
check("and no second calibration opens", wait(lambda: svc.checks.check is not None, 3), False)
check("one calibration in the log", sum(m.startswith("full check:") for m in logs), 1)
print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
svc.running = False
time.sleep(0.7)
shutil.rmtree(HOME, ignore_errors=True)
os._exit(1 if failures else 0)
+42 -5
View File
@@ -56,13 +56,19 @@ ANSI = re.compile(r"\x1b\[[0-9;]*m")
STAMP = re.compile(r"^\w{3} \w{3} \d{2} \d{4} (\d{2}:\d{2}:\d{2})\.\d+ (\w+): ?(.*)$")
# Lines worth reading; anything else is skipped before the regexes (the log grows by MBs a day).
KEYS = ("FPGA", "VCINT", "Created", "TrackingCameraInit", "Closing tracking camera", "Streaming",
"systemd suspend", "systemd resume", "XRService logging to", "Exiting XRService")
"systemd suspend", "systemd resume", "XRService logging to", "Exiting XRService", "ISP ")
# XRService's numbering of its tracking cameras (the TrackingCameraInit index): ft-hands names
# them this way too (track/main.cpp, cameras_from_xrservice_log).
NAMES = ("slam_left", "slam_right", "upper_left", "upper_right")
RE_PASSTHRU = re.compile(r"Passthrough connected but FPGA is (\S+) - loading VCINT")
RE_INTERLEAVE = re.compile(r"Upper cameras FPGA interleaving support: (\d)")
RE_TASKS = re.compile(r"Created (\d+) tasks \((\d+) tracking, (\d+) passthrough\)")
RE_INIT = re.compile(r"TrackingCameraInit: index: (\d+)\. video device: /dev/video(\d+)")
RE_STREAM = re.compile(r"Streaming resumed \(FPGA: (\S+), VC interleaving: (\w+)\)")
RE_STATE = re.compile(r"FPGA state check: (\S+)")
# Without the colour module XRService runs the side cameras through the ISP, as NV12 on other
# capture pipes (vfe0 and vfe1), and the upper pair on vfe3 and vfe4.
RE_ISP = re.compile(r"ISP (enabled|disabled) for tracking cameras")
class LogState:
@@ -85,7 +91,7 @@ class LogState:
def _new_episode(self, t):
self.closed = False
self.episode = {"start": t, "fpga_before": "", "vcint": "", "interleave": None, "tasks": None,
"inits": {}, "stream": "", "failure": "", "evidence": []}
"inits": {}, "stream": "", "isp": None, "failure": "", "evidence": []}
if self.closed_at:
self.episode["evidence"].append(self.closed_at)
@@ -146,6 +152,12 @@ class LogState:
ep["interleave"] = int(m.group(1))
ep["evidence"].append(short)
return
m = RE_ISP.search(text)
if m:
ep = self._ep(t)
ep["isp"] = m.group(1) == "enabled"
ep["evidence"].append(short)
return
m = RE_TASKS.search(text)
if m:
ep = self._ep(t)
@@ -184,6 +196,12 @@ class LogState:
got = tuple(self.nodes[i] for i in (2, 3) if i in self.nodes)
return got if len(got) == 2 else UPPER_NODES
def camera_map(self):
"""{calibration name: /dev/videoN's N} from the latest camera start's TrackingCameraInit
lines (the whole log's when that start has none yet)."""
inits = (self.episode or {}).get("inits") or self.nodes
return {NAMES[i]: node for i, node in sorted(inits.items()) if 0 <= i < len(NAMES)}
def tracking_nodes(self):
got = tuple(self.nodes[i] for i in range(TRACKING) if i in self.nodes)
return got if len(got) == TRACKING else SIDE_NODES + UPPER_NODES
@@ -353,7 +371,9 @@ def check(log=None, proc=True, ring=True, ring_path=None):
status, reason = "unknown", "no XRService log in %s" % LOG_DIR
out = {"log": path, "xrservice": None, "ring": None,
"episode": state.snapshot()["episode"] if state else {},
"failure": state.snapshot()["failure"] if state else ""}
"failure": state.snapshot()["failure"] if state else "",
"map": {name: {"node": n, "pipe": pipe_name(n)} for name, n in state.camera_map().items()} if state else {},
"ring_missing": []}
if proc:
if pid is None:
@@ -389,13 +409,30 @@ def check(log=None, proc=True, ring=True, ring_path=None):
evidence.append("ft-camd (pid %d, %s): %d mono cameras: %s" % (
r["writer_pid"], "running" if r["alive"] else "stale ring", len(r["mono"]), names))
if r["alive"] and len(r["mono"]) < TRACKING and status == "ok":
status, reason = "degraded", ("ft-camd publishes only %d of %d mono cameras (it started while "
"they were missing: restart it)" % (len(r["mono"]), TRACKING))
have = {c["node"] for c in r["mono"]}
want = state.tracking_nodes() if state else SIDE_NODES + UPPER_NODES
out["ring_missing"] = [n for n in want if n not in have]
status, reason = "degraded", ("ft-camd publishes only %d of %d mono cameras, missing %s" % (
len(r["mono"]), TRACKING, " ".join("video%d" % n for n in out["ring_missing"]) or "?"))
out.update(status=status, reason=reason, evidence=evidence,
summary="ok" if status == "ok" else "%s: %s" % (status, reason))
return out
def pipe_name(node):
"""The capture pipe behind /dev/videoN (msm_vfe3_video0 and so on), or ""."""
try:
with open("/sys/class/video4linux/video%d/name" % node) as f:
return f.read().strip()
except OSError:
return ""
def is_ring_short(result):
"""XRService runs all the tracking cameras, but ft-camd doesn't publish them all."""
return bool(result) and result.get("status") == "degraded" and bool(result.get("ring_missing"))
def is_vcint_failure(result):
return bool(result) and result.get("status") == "degraded" and result.get("reason") == VCINT_REASON
+25 -2
View File
@@ -379,9 +379,15 @@ static bool resolve_block(cam_t *c)
return true;
}
/*
* Through the ISP (no colour module), the near-black exposures come out all zeros,
* the same every time, so their dequeues change no buffer and get no votes. Such
* an index is left unmapped (slot -1): on_frame counts its frames as dark without
* reading them. Most of a camera's indices silent means it isn't streaming yet.
*/
static bool resolve_each(cam_t *c)
{
int depth = c->maxindex + 1;
int depth = c->maxindex + 1, silent = 0;
bool taken[MAX_SLOTS] = { false };
for (int i = 0; i < depth; i++) {
@@ -396,6 +402,12 @@ static bool resolve_each(cam_t *c)
}
}
if (c->nobs[i] >= 3 && v1 <= 0.2 * c->nobs[i]) {
c->slot_of[i] = -1;
silent++;
continue;
}
if (c->nobs[i] < 3 || best < 0 || taken[best] || v1 < 0.8 * c->nobs[i] || v2 > 0.3 * c->nobs[i])
return false;
@@ -403,9 +415,14 @@ static bool resolve_each(cam_t *c)
c->slot_of[i] = best;
}
if (silent * 2 > depth)
return false;
printf("%s: queue depth %d, buffers mapped one by one:", c->slug, depth);
for (int i = 0; i < depth; i++)
printf(" %d", c->slot_of[i]);
c->slot_of[i] < 0 ? printf(" -") : printf(" %d", c->slot_of[i]);
if (silent)
printf(" (- : %d indices whose frames are all zeros, skipped as dark)", silent);
printf("\n");
return true;
}
@@ -677,6 +694,12 @@ static void on_frame(cam_t *c, int64_t index, uint32_t seq, uint64_t ts, uint64_
int slot = c->slot_of[index];
if (slot < 0) { /* an index whose frames are all zeros (resolve_each): a dark one */
c->dark++;
c->rc->dropped++;
return;
}
/* color runs at 60 fps: skip frames early enough that the asked rate holds, before any sync */
if (c->color && color_fps > 0 && evtime - c->last_pub_ns < (uint64_t)(1e9 / color_fps) - 3000000) {
c->paced++;
+24 -7
View File
@@ -544,22 +544,26 @@ static void probe_cameras(xr_state_t *st)
/*
* qcom-camss can report bytesperline as the visible width while the VFE
* writes a larger aligned pitch. sizeimage is right, so derive the pitch.
* For NV12, plane 0 normally holds the chroma rows after the luma (the side
* cameras through the ISP: 1056 wide, 1152 bytes a row); if it's too small for
* that, it holds the luma alone.
*/
unsigned xr_camera_stride(const xr_camera_t *c)
{
if (!c->height || !c->planesize[0])
return c->bytesperline ? c->bytesperline : c->width;
double bpp = 1.0;
if (c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21)
bpp = 1.5;
unsigned s = (unsigned)((double)c->planesize[0] / ((double)c->height * bpp));
bool yuv = c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21;
unsigned s = (unsigned)((double)c->planesize[0] / ((double)c->height * (yuv ? 1.5 : 1.0)));
if (s >= c->width && s <= c->width * 4)
return s;
s = (unsigned)(c->planesize[0] / c->height);
if (yuv && s >= c->width && s <= c->width * 4)
return s;
return c->bytesperline ? c->bytesperline : c->width;
}
@@ -591,7 +595,20 @@ void xr_camera_layout(const xr_camera_t *c, xr_layout_t *l)
l->pitch = xr_camera_stride(c);
l->width = c->width < l->pitch ? c->width : l->pitch;
if (c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21) {
/*
* Without the colour module, XRService runs the side cameras through the
* ISP ("ISP enabled for tracking cameras (main VFE available)" in its log),
* and they come out NV12. They're mono sensors, so the luma is the image.
*/
bool yuv = c->pixfmt == V4L2_PIX_FMT_NV12 || c->pixfmt == V4L2_PIX_FMT_NV21;
if (yuv && c->role && !strcmp(c->role, "tracking")) {
l->fmt = XR_FMT_GREY8;
l->rows = c->height;
return;
}
if (yuv) {
l->fmt = XR_FMT_NV12;
l->rows = c->height + c->height / 2;
} else {
+5 -4
View File
@@ -457,11 +457,12 @@ class Backend(QObject):
"""Start stays off: the check found the cameras degraded (unless --ignore-cameras)."""
return not self._ignore_cameras and self._camera.get("status") == "degraded"
def _check_now(self):
def _check_now(self, repair=False):
"""repair (off this thread only: it may restart ft-camd, up to 15 s): see session.camera_check."""
mod = self._runner()
if not mod or self._session_options.get("dry_run"):
return {"status": "unknown", "summary": "not checked (dry run)", "reason": "dry run", "evidence": []}
return mod.camera_check()
return mod.camera_check(repair=repair)
@Slot()
def checkCameras(self):
@@ -470,7 +471,7 @@ class Backend(QObject):
return
self._camera_busy = True
self.cameraChanged.emit()
self._thread(lambda: self._cameraArrived.emit(self._check_now()))
self._thread(lambda: self._cameraArrived.emit(self._check_now(repair=True)))
def _on_camera(self, result):
self._camera = result
@@ -499,7 +500,7 @@ class Backend(QObject):
end = time.monotonic() + RECHECK_FOR_S
time.sleep(RECHECK_S * 2)
while time.monotonic() < end:
result = self._check_now()
result = self._check_now(repair=True)
# Done once a new XRService (a new log) has opened its cameras, ok or not.
if result.get("status") in ("ok", "degraded") and result.get("log") != before:
break
+34 -4
View File
@@ -1153,11 +1153,33 @@ def _steamvr_version():
return ""
def camera_check():
"""camcheck.check() (the XRService log, its open cameras where readable, ft-camd's ring);
never raises: a failure is "unknown"."""
_camd_restarted = set() # ft-camd pids camera_check(repair=True) has restarted: once each
def _unit_pid(unit):
r = subprocess.run(host_command("systemctl", "--user", "show", "-p", "MainPID", "--value", unit),
capture_output=True, text=True, timeout=30)
try:
return camcheck.check()
return int(r.stdout.strip() or 0)
except ValueError:
return 0
def camera_check(repair=False):
"""camcheck.check() (the XRService log, its open cameras where readable, ft-camd's ring);
never raises: a failure is "unknown". repair (the window, never during a session): when
XRService runs every tracking camera but the recorder's own ft-camd doesn't publish them all
(it started while some were missing), restart it, once per ft-camd, and check again."""
try:
r = camcheck.check()
pid = (r.get("ring") or {}).get("writer_pid")
if repair and camcheck.is_ring_short(r) and pid not in _camd_restarted and _unit_pid(CAMD_UNIT) == pid:
_camd_restarted.add(pid)
stop_unit(CAMD_UNIT)
start_camd()
r = camcheck.check()
r["evidence"].append("restarted ft-camd (pid %d) because it published only some of the cameras" % pid)
return r
except Exception as e:
return {"status": "unknown", "summary": "unknown: the camera check failed (%s)" % e,
"reason": str(e), "evidence": []}
@@ -1169,6 +1191,10 @@ def camera_text(result):
return ""
if camcheck.is_vcint_failure(result):
return camcheck.USER_TEXT
if camcheck.is_ring_short(result):
return ("The headset's tracking cameras are all running, but the recorder can't read some of them (%s). "
"Close the Hand Recorder, run ~/frametop/hands/rec/install.sh again, and open it again. If that "
"doesn't help, ask in the Frametop Discord." % result.get("reason", ""))
return ("Not all of the headset's tracking cameras are running (%s). Restart SteamVR, or restart the "
"headset if that doesn't fix it." % result.get("reason", ""))
@@ -1451,6 +1477,10 @@ class Session:
cam = self._status.get("camera")
if cam:
self._session_json["camera"] = {"status": cam.get("status"), "reason": cam.get("reason", "")}
# which device each calibrated camera was (XRService's numbering) and whether XRService
# ran the side cameras through the ISP (no colour module): to check the names later
self._session_json["device"]["camera_map"] = cam.get("map") or {}
self._session_json["device"]["isp"] = (cam.get("episode") or {}).get("isp")
if self.dry_run:
self._session_json["dry_run"] = True
if self.speed != 1:
+48 -2
View File
@@ -66,6 +66,22 @@ RESUME_OK = [L("12:30:00", "[SystemdInhibitor] Received systemd resume notificat
for i, n in enumerate((9, 13, 6, 7))] + \
[L("12:30:02", "[DeckardCaptureSource] Streaming resumed (FPGA: VCINT, VC interleaving: enabled)")]
EXIT = [L("13:00:00", "XRService - main thread exiting"), L("13:00:00", "Exiting XRService")]
# The colour module unplugged while SteamVR runs (2026-10-05 12:42): XRService reopens the
# cameras with the side pair through the ISP on vfe0 and vfe1 (NV12); VCINT stays loaded, so the
# upper pair stays on vfe2.
UNPLUG = [L("12:42:25", "Received passthrough camera connection event (connected=0)"),
L("12:42:25", "[DeckardCaptureSource] Closing tracking camera interfaces camerasToUse: 1111"),
L("12:42:26", "FPGA state check: VCINT (register value: 0x00021211)"),
L("12:42:26", "Upper cameras FPGA interleaving support: 1 (Driver features available = 1 | VCINT loaded = 1)"),
L("12:42:26", "[buildMediaCtlSetupTasks] ISP enabled for tracking cameras (main VFE available)"),
L("12:42:26", "[buildMediaCtlSetupTasks] Created 4 tasks (4 tracking, 0 passthrough)")] + \
[L("12:42:26", "TrackingCameraInit: index: %d. video device: /dev/video%d. v4l subdevice: x" % (i, n))
for i, n in enumerate((0, 3, 6, 7))]
# Started without the module (FrameEyeCameraFeed's layout): the upper pair on vfe3 and vfe4.
NO_MODULE = [L("10:00:02", "[buildMediaCtlSetupTasks] ISP enabled for tracking cameras (main VFE available)"),
L("10:00:02", "[buildMediaCtlSetupTasks] Created 4 tasks (4 tracking, 0 passthrough)")] + \
[L("10:00:03", "TrackingCameraInit: index: %d. video device: /dev/video%d. v4l subdevice: x" % (i, n))
for i, n in enumerate((0, 3, 9, 13))]
def state(lines):
@@ -128,6 +144,23 @@ class SyntheticLogs(unittest.TestCase):
self.assertEqual(status, "degraded")
self.assertEqual(reason, "only 2 of 4 tracking cameras running")
def test_camera_map_with_module(self):
st = state(START + GOOD_OPEN)
self.assertEqual(st.camera_map(), {"slam_left": 9, "slam_right": 13, "upper_left": 6, "upper_right": 7})
def test_module_unplugged(self):
st = state(START + GOOD_OPEN + UNPLUG)
self.assertEqual(st.verdict()[0], "ok")
self.assertIs(st.episode["isp"], True)
self.assertEqual(st.camera_map(), {"slam_left": 0, "slam_right": 3, "upper_left": 6, "upper_right": 7})
self.assertEqual(st.tracking_nodes(), (0, 3, 6, 7))
def test_started_without_module(self):
st = state(START + NO_MODULE)
self.assertEqual(st.verdict()[0], "ok")
self.assertEqual(st.camera_map(), {"slam_left": 0, "slam_right": 3, "upper_left": 9, "upper_right": 13})
self.assertEqual(st.upper_nodes(), (9, 13))
def test_exited_and_empty(self):
self.assertEqual(state(START + GOOD_OPEN + EXIT).verdict()[0], "unknown")
self.assertEqual(state([]).verdict()[0], "unknown")
@@ -223,9 +256,9 @@ class RealLog(unittest.TestCase):
self.assertEqual(out.getvalue().split("\n")[0], "ok")
def make_ring(path, mono_names, alive=True):
def make_ring(path, mono_names, alive=True, nodes=None):
"""A ring header as ft-camd writes it (camd/fhring.h), no frames."""
cams = [(b"og01a1b", n.encode(), 9 + i) for i, n in enumerate(mono_names)]
cams = [(b"og01a1b", n.encode(), nodes[i] if nodes else 9 + i) for i, n in enumerate(mono_names)]
hb = time.clock_gettime_ns(time.CLOCK_MONOTONIC) if alive else 1
data = bytearray(camcheck.RING_HDR.pack(b"FHRING01", 1, 0, len(cams), 0, 0, 4242, 0))
struct.pack_into("<Q", data, 40, hb)
@@ -252,6 +285,19 @@ class Ring(unittest.TestCase):
self.assertEqual(r["status"], "degraded")
self.assertIn("ft-camd publishes only 2 of 4", r["reason"])
def test_ring_missing_the_side_cameras(self):
# an ft-camd from before NV12 support, without the colour module: only the upper pair
with open(self.log, "w") as f:
f.write("\n".join(START + NO_MODULE) + "\n")
make_ring(self.ring, ["og0ve10_5-003e_video9", "og0ve10_5-0060_video13"], nodes=[9, 13])
r = camcheck.check(log=self.log, proc=False, ring_path=self.ring)
self.assertEqual(r["status"], "degraded")
self.assertEqual(r["ring_missing"], [0, 3])
self.assertIn("missing video0 video3", r["reason"])
self.assertTrue(camcheck.is_ring_short(r))
self.assertFalse(camcheck.is_vcint_failure(r))
self.assertEqual(r["map"]["slam_left"]["node"], 0)
def test_four_cameras_in_ring(self):
make_ring(self.ring, ["a_video9", "b_video13", "c_video6", "d_video7", "a_video9_dk"])
r = camcheck.check(log=self.log, proc=False, ring_path=self.ring)
+23 -2
View File
@@ -34,8 +34,28 @@ from tools.show_set import index, read_set # noqa: E402
from tools import calib # noqa: E402
PIPES = {'msm_vfe3_video0': 'slam_left', 'msm_vfe4_video0': 'slam_right', # as ft-hands maps them
PIPES = {'msm_vfe3_video0': 'slam_left', 'msm_vfe4_video0': 'slam_right', # with the colour module
'msm_vfe2_video0': 'upper_left', 'msm_vfe2_video1': 'upper_right'}
def ring_names():
"""{/dev/videoN's N: calibration name} as ft-hands names them: by XRService's log
(camcheck.py), else by capture pipe (PIPES)."""
import camcheck
try:
by_log = {node: name for name, node in camcheck.read_log(camcheck.newest_log()).camera_map().items()}
except (OSError, ValueError):
by_log = {}
if by_log:
return by_log
out = {}
for path in os.listdir('/sys/class/video4linux'):
if path.startswith('video'):
with open('/sys/class/video4linux/%s/name' % path) as f:
name = PIPES.get(f.read().strip())
if name:
out[int(path[5:])] = name
return out
PAIRS = {'side': ('slam_left', 'slam_right'), 'upper': ('upper_left', 'upper_right')}
@@ -100,8 +120,9 @@ def live_pairs(count, names=PAIRS['side']):
if not ring.alive():
sys.exit('ft-camd isn\'t running (no heartbeat)')
cams = {}
names_of = ring_names()
for c in ring.cams:
name = PIPES.get(open('/sys/class/video4linux/video%d/name' % c.node).read().strip())
name = names_of.get(c.node)
if name and not c.name.endswith('-dark'):
cams[name] = c
for k in range(count):
+64 -11
View File
@@ -63,7 +63,34 @@ namespace {
volatile std::sig_atomic_t g_stop = 0, g_record = 0;
// which calibrated camera each capture pipe carries (XRService's fixed routing)
// Which calibrated camera each video device carries. XRService numbers its tracking cameras
// (index 0 to 3: slam_left, slam_right, upper_left, upper_right) and logs the device each one
// opened ("TrackingCameraInit: index: 0. video device: /dev/video9"). The devices depend on
// the colour module: with it, the side cameras are on vfe3 and vfe4 and the upper pair on
// vfe2; without it, XRService runs the side cameras through the ISP on vfe0 and vfe1, and the
// upper pair on vfe3 and vfe4. So the running XRService's log decides; when it can't be read,
// the capture pipes as they are with the module. {} if the log has no cameras.
std::map<int, std::string> cameras_from_xrservice_log() {
static const char *const names[] = {"slam_left", "slam_right", "upper_left", "upper_right"};
const char *home = std::getenv("HOME");
std::ifstream in(std::string(home ? home : "") + "/.local/share/Steam/logs/xrservice.txt");
const std::string key = "TrackingCameraInit: index: ";
std::map<int, int> node_of; // index -> N of /dev/videoN, from the latest camera start
std::string line;
while (std::getline(in, line)) {
if (line.find("XRService logging to") != std::string::npos) node_of.clear();
const auto at = line.find(key);
int index = -1, node = -1;
if (at != std::string::npos &&
std::sscanf(line.c_str() + at + key.size(), "%d. video device: /dev/video%d", &index, &node) == 2 &&
index >= 0 && index < 4)
node_of[index] = node;
}
std::map<int, std::string> out;
for (auto &[index, node] : node_of) out[node] = names[index];
return out;
}
const char *camera_for_pipe(int node) {
char path[64], name[64] = "";
std::snprintf(path, sizeof path, "/sys/class/video4linux/video%d/name", node);
@@ -302,6 +329,7 @@ int main(int argc, char **argv) {
double decided_after_s = -1;
if (sides_mode != "auto") truth = names_swapped, decided_by = sides_from, sides_state = "forced";
std::string rec_dir;
std::string cams_json; // which device each calibrated camera is (set below), for the sides files
std::vector<std::pair<size_t, bool>> rec_names; // from which recorded set on, names_swapped was what
// DIR/sides.json beside a recording's sets.bin: how its side cameras are named. A set's
// names are right when its names_swapped equals swapped (null: not known when recorded).
@@ -312,7 +340,8 @@ int main(int argc, char **argv) {
write_file(rec_dir + "/sides.json",
"{\"swapped\": " + json_bool(truth) + ", \"decided_by\": " + json_str(truth ? decided_by : "") +
", \"names_swapped\": [" + runs + "]" +
(decision_evidence.empty() ? "" : ", \"evidence\": " + decision_evidence) + "}\n");
(decision_evidence.empty() ? "" : ", \"evidence\": " + decision_evidence) +
(cams_json.empty() ? "" : ", \"cameras\": " + cams_json) + "}\n");
};
auto start_recording = [&](const std::string &dir, std::string &e) {
rec = std::make_unique<Recorder>();
@@ -340,19 +369,42 @@ int main(int argc, char **argv) {
// (--with-color). Recorded names hold 15 characters, so "upper_right_dark" wouldn't fit.
std::map<std::string, int> dark;
std::map<std::string, Camera> used;
// ft-camd's cameras by XRService's numbering, else by capture pipe (see camera_for_pipe);
// ft-ringplay's (no device) by the name it gives
const std::map<int, std::string> by_log = cameras_from_xrservice_log();
const char *named_by = by_log.empty() ? "capture pipe" : "XRService's log";
for (int i = 0; i < ring.cameras(); ++i) {
if (ring.camera(i).flags & FH_CAM_COLOR) {
const std::string name = "color_video" + std::to_string(ring.camera(i).node);
const fh_ring_cam_t &rc = ring.camera(i);
if (rc.flags & FH_CAM_COLOR) {
const std::string name = "color_video" + std::to_string(rc.node);
dark[name] = i, color[name] = i;
continue;
}
// ft-camd's cameras by capture pipe; ft-ringplay's (no device) by the name it gives
const char *name = camera_for_pipe(ring.camera(i).node);
if (!name && ring.camera(i).node < 0) name = ring.camera(i).name;
if (!name || !calib.count(name)) continue;
if (ring.camera(i).flags & FH_CAM_DARK) dark[std::string(name) + "_dk"] = i;
else index[name] = i, used[name] = calib[name];
const auto it = by_log.find(rc.node);
const char *name = rc.node < 0 ? rc.name
: !by_log.empty() ? (it == by_log.end() ? nullptr : it->second.c_str())
: camera_for_pipe(rc.node);
if (!name || !calib.count(name)) {
if (!(rc.flags & FH_CAM_DARK)) std::fprintf(stderr, "video%d (%s): not one of the calibrated cameras, left out\n", rc.node, rc.name);
continue;
}
if (int(rc.width) != calib[name].width || int(rc.height) != calib[name].height) {
std::fprintf(stderr, "video%d (%s) is %ux%u, but %s is calibrated at %dx%d: left out\n", rc.node, rc.name,
rc.width, rc.height, name, calib[name].width, calib[name].height);
continue;
}
if (rc.flags & FH_CAM_DARK) {
dark[std::string(name) + "_dk"] = i;
} else if (index.count(name)) {
std::fprintf(stderr, "video%d (%s) would be %s too (video%d is): left out\n", rc.node, rc.name, name,
ring.camera(index[name]).node);
} else {
index[name] = i, used[name] = calib[name];
cams_json += std::string(cams_json.empty() ? "" : ", ") + json_str(name) + ": {\"node\": " +
std::to_string(rc.node) + ", \"ring\": " + json_str(rc.name) + "}";
}
}
cams_json = "{\"named_by\": " + json_str(named_by) + ", \"cameras\": {" + cams_json + "}}";
// ft-camd tells the side cameras' buffers apart by XRService's allocation order, which
// some XRService restarts reverse (see the top).
const bool have_sides = index.count("slam_left") && index.count("slam_right");
@@ -397,7 +449,7 @@ int main(int argc, char **argv) {
}
const bool switching = automatic && !color.empty();
Cams mode = switching || color.empty() ? Cams::Mono : fixed;
std::printf("cameras:");
std::printf("cameras (by %s):", named_by);
for (auto &[name, i] : index) std::printf(" %s=video%d", name.c_str(), ring.camera(i).node);
for (auto &[name, i] : color) std::printf(" %s", name.c_str());
std::printf(" tracking with %s%s models: %s%s, %d threads on CPUs", cams_name(mode),
@@ -446,6 +498,7 @@ int main(int argc, char **argv) {
", \"decided_after_s\": " + std::to_string(decided_after_s) +
", \"evidence\": " + (decision_evidence.empty() ? "null" : decision_evidence) +
", \"checking\": " + (checking ? side_check.json() : "null") +
", \"cameras\": " + (cams_json.empty() ? "null" : cams_json) +
", \"updated_ns\": " + std::to_string(mono_ns()) + "}\n");
};
write_sides();
+1 -1
View File
@@ -75,7 +75,7 @@ step "3/10 input relay (keeps Bluetooth mice working in SteamVR, device roles, b
step "4/10 3D mouse: SteamVR driver"
"$root/pointer/driver/build.sh"
"$root/pointer/driver/install.sh" install 2>&1 | grep -v xdg-open
"$root/pointer/driver/install.sh" install
step "5/10 3D mouse: pointer helper service"
"$root/pointer/helper/build.sh"
+2 -1
View File
@@ -534,7 +534,8 @@ def capture_screens():
# ---------------------------------------------------------------- gamescope (dashboard panels)
def vrcmd(*args, timeout=10):
env = dict(os.environ, LD_LIBRARY_PATH=os.path.dirname(VRCMD))
# SteamVR's config folder: in the desktop, XDG_CONFIG_HOME is its own (docs/design.md).
env = dict(os.environ, LD_LIBRARY_PATH=os.path.dirname(VRCMD), XDG_CONFIG_HOME=os.path.expanduser("~/.config"))
try:
return subprocess.run([VRCMD, *args], capture_output=True, text=True, timeout=timeout, env=env).stdout
except (OSError, subprocess.TimeoutExpired):
+21 -5
View File
@@ -14,6 +14,10 @@ frame="$root/scripts/frame.sh"
src=$FRAME_REPO/pointer/driver
dest='$HOME/.local/share/frametop/ft_pointer' # expanded on the Frame, in the commands below
reg='/opt/steamvr/bin/linuxarm64/vrpathreg'
# SteamVR's tools with SteamVR's config folder and libraries. A terminal in the Frametop desktop
# has XDG_CONFIG_HOME=~/.config/frametop (session/frametop-session.sh), where vrpathreg finds no
# registry, so it made one there that SteamVR never reads.
steamvr='env XDG_CONFIG_HOME=$HOME/.config LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64' # expanded on the Frame
case ${1:-install} in
install)
@@ -21,6 +25,11 @@ case ${1:-install} in
# Replacing the files of a driver SteamVR has loaded leaves its input bindings in a bad
# state (the 3D mouse no longer gets the laser) until SteamVR restarts, so an unchanged
# driver is left alone.
# A registry an earlier install made in the desktop's config folder (see steamvr above) has
# no SteamVR in it, and hid SteamVR from OpenVR programs started in the desktop
# (VRInitError_Init_InstallationNotFound): it goes.
# vrpathreg runs 'xdg-open vrmonitor://driverinstalled' for SteamVR's desktop monitor, which
# the Frame doesn't have, so the desktop said "could not read file": a stand-in takes it.
"$frame" --host "set -e; test -f $src/build/driver_ft_pointer.so
rm -rf $dest.new; mkdir -p $dest.new/bin/linuxarm64
cp -r $src/ft_pointer/. $dest.new/
@@ -31,15 +40,22 @@ else
rm -rf $dest; mv $dest.new $dest
echo 'installed; restart SteamVR to load it'
fi
LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg adddriver $dest
LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg show | grep -A3 -i 'external'" ;;
stray=\$HOME/.config/frametop/openvr/openvrpaths.vrpath
if grep -qs '\"jsonid\" : \"vrpathreg\"' \$stray && grep -qsE '\"runtime\"[[:space:]]*:[[:space:]]*null' \$stray; then
rm -f \$stray; rmdir \$HOME/.config/frametop/openvr 2>/dev/null || true
echo 'removed a SteamVR path registry an earlier install left in ~/.config/frametop/openvr'
fi
noopen=\$(mktemp -d); trap 'rm -rf \$noopen' EXIT
printf '#!/bin/sh\nexit 0\n' > \$noopen/xdg-open; chmod +x \$noopen/xdg-open
PATH=\$noopen:\$PATH $steamvr $reg adddriver $dest
$steamvr $reg show | grep -A3 -i 'external'" ;;
uninstall)
"$frame" --host "LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 $reg removedriver $dest; rm -rf $dest; echo 'removed; restart SteamVR to unload it'" ;;
"$frame" --host "$steamvr $reg removedriver $dest; rm -rf $dest; echo 'removed; restart SteamVR to unload it'" ;;
send)
"$frame" --host "python3 -c 'import socket,sys; s=socket.socket(socket.AF_UNIX,socket.SOCK_DGRAM); s.sendto(sys.argv[1].encode(), \"\\0ft_pointer\")' $(printf %q "${2:?command}")" ;;
probe) "$frame" -C pointer/probe 'LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 ./build/vrprobe' ;;
probe) "$frame" -C pointer/probe "$steamvr ./build/vrprobe" ;;
aimhere)
read -r yaw pitch < <("$frame" -C pointer/probe 'LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 ./build/vrprobe' | sed -n 's/^head yaw = \([-0-9.]*\) pitch = \([-0-9.]*\)$/\1 \2/p')
read -r yaw pitch < <("$frame" -C pointer/probe "$steamvr ./build/vrprobe" | sed -n 's/^head yaw = \([-0-9.]*\) pitch = \([-0-9.]*\)$/\1 \2/p')
[ -n "${yaw:-}" ] || { echo "head pose not valid (headset off?)" >&2; exit 1; }
"$0" send "aim $yaw $pitch" && echo "aimed at yaw $yaw pitch $pitch" ;;
log) "$frame" --host "grep -iE 'ft_pointer' ~/.local/share/Steam/logs/vrserver.txt | tail -n ${2:-20}" ;;
+24 -1
View File
@@ -26,7 +26,8 @@ for u in frametop-input-relay frametop-pointer frametop-power; do
done
echo "desktop: $(pgrep -x ft-screens >/dev/null && echo running || echo 'not running'), plasmashell: $(pgrep -c plasmashell || true)"
echo "paused for VR games: $(cat /run/user/$(id -u)/frametop-pause.json 2>/dev/null || echo 'no (never paused since boot)')"
LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrpathreg show 2>/dev/null | sed -n '/xternal/,$p'
# SteamVR's config folder: a terminal in the desktop has the session's own (docs/design.md).
XDG_CONFIG_HOME=$HOME/.config LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrpathreg show 2>/dev/null | sed -n '/xternal/,$p'
section "What Frametop needs from SteamOS (scripts/update-check.py)"
python3 "$repo/scripts/update-check.py" 2>&1 || true
@@ -67,6 +68,28 @@ section "Pointer helper (last 60 lines)"
journalctl --user -u frametop-pointer -n 60 --no-pager -o short 2>/dev/null
section "Power service (last 30 lines)"
journalctl --user -u frametop-power -n 30 --no-pager -o short 2>/dev/null
section "Gaze"
echo "frametop-gaze: $(systemctl --user is-enabled frametop-gaze 2>/dev/null) / $(systemctl --user is-active frametop-gaze 2>/dev/null)"
echo "frametop-eyegrab (our eye tracker's frame grabber): $(systemctl is-enabled frametop-eyegrab 2>/dev/null) / $(systemctl is-active frametop-eyegrab 2>/dev/null)"
for f in calibration.json eyes/calibration.json; do # SteamVR's correction, our tracker's calibration
p=~/.local/state/frametop/gaze/$f
echo "$f: $([ -f "$p" ] && date -r "$p" '+%F %T' || echo none)"
done
python3 - "$repo" <<'PY' 2>&1 || true
import json, subprocess, sys
out = subprocess.run([sys.executable, sys.argv[1] + "/gaze/ft-gazectl", "status"], capture_output=True, text=True, timeout=10)
text = (out.stdout or out.stderr).strip()
try:
print("status:", json.dumps(json.loads(text), separators=(",", ":")))
except ValueError:
print("status:", text or "no answer from the gaze service")
PY
section "Gaze checks and calibration dots (last 10)"
tail -n 10 ~/.local/state/frametop/gaze/checks.jsonl 2>/dev/null | cut -c1-400 || true
section "Gaze service (last 60 lines, podman's left out)"
journalctl --user -u frametop-gaze -n 300 --no-pager -o short 2>/dev/null | grep -v ' podman\[' | tail -n 60
section "Our eye tracker's frame grabber (last 20 lines)"
journalctl -u frametop-eyegrab -n 20 --no-pager -o short 2>/dev/null
for f in /tmp/frametop-session.log /tmp/frametop-screens.log /tmp/frametop-layout.log; do
section "$f (last 60 lines)"
tail -n 60 "$f" 2>/dev/null || echo "missing"
+13
View File
@@ -35,6 +35,9 @@ KNOWN_GOOD = os.path.join(HOME, ".local/state/frametop/known-good.json")
STEAMVR_BIN = "/opt/steamvr/bin/linuxarm64"
LAUNCHER = os.path.join(HOME, ".local/share/applications/deckard-nested-desktop.desktop")
VRPATHS = os.path.join(HOME, ".config/openvr/openvrpaths.vrpath")
# The Frametop desktop has its own XDG_CONFIG_HOME (session/frametop-session.sh). An install from a
# terminal there, before pointer/driver/install.sh set SteamVR's, left vrpathreg's registry here.
STRAY_VRPATHS = os.path.join(HOME, ".config/frametop/openvr/openvrpaths.vrpath")
BACKLIGHT = "/sys/class/backlight/ae94000.dsi.0/brightness" # as in power/ft-powerd.cpp
EYE_MMAP = "/dev/shm/eye-server.mmap"
VRSERVER = "http://127.0.0.1:27062" # its web socket: input/vrws.py
@@ -227,6 +230,14 @@ def check_host():
else:
report("FAIL", "pointer driver", "not registered with SteamVR; run pointer/driver/install.sh install, "
"then restart SteamVR")
try:
with open(STRAY_VRPATHS) as f:
stray = json.load(f).get("runtime") is None
except (OSError, ValueError, AttributeError):
stray = False
if stray:
report("warn", "OpenVR path registry", f"{STRAY_VRPATHS} has no SteamVR in it, and hides SteamVR from "
"OpenVR programs started in the Frametop desktop; pointer/driver/install.sh install removes it")
session = launcher_session()
if session is None:
@@ -474,6 +485,8 @@ def main():
# A terminal in the desktop has its session's runtime dir and bus; systemctl needs the real ones.
os.environ["XDG_RUNTIME_DIR"] = f"/run/user/{UID}"
os.environ["DBUS_SESSION_BUS_ADDRESS"] = f"unix:path=/run/user/{UID}/bus"
# And its own config folder, where SteamVR's path registry isn't: OpenVR and vrcmd need ours.
os.environ["XDG_CONFIG_HOME"] = os.path.join(HOME, ".config")
versions = current_versions()
check_versions(versions)
Executable
+241
View File
@@ -0,0 +1,241 @@
#!/usr/bin/env bash
# Uninstall Frametop from the Steam Frame. In a terminal on the headset (Konsole in the desktop,
# or over SSH):
#
# curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash
#
# or ~/frametop/uninstall.sh. It doesn't use the rest of the repo, so it also works when
# ~/frametop is gone or broken.
#
# It never stops what you're using now: the input relay carries the keyboard and mouse, and the
# desktop runs from the repo. So it goes in two steps:
# 1. Frametop stops starting. The launcher's Desktop entry goes back to the stock desktop, and
# Frametop's services, SteamVR driver, menu entries, and system files (our eye tracker's
# frame grabber and the Bluetooth fixes, with sudo) are removed. What runs now keeps running
# until you restart the headset.
# 2. After the restart, run it again. It deletes the code (~/frametop) and, if you want, your
# settings and the build container.
# When nothing of Frametop is running, one run does both.
#
# Options (piped, they go after "bash -s --"):
# --dir DIR where the repo is, if not ~/frametop
# --dry-run show what it would do, and change nothing
set -euo pipefail
shopt -s nullglob
usage() {
cat <<'EOF'
usage: uninstall.sh [--dir DIR] [--dry-run]
piped: curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash -s -- [options]
EOF
}
dry=0
apps=$HOME/.local/share/applications
override=$apps/deckard-nested-desktop.desktop
relay_unit=$HOME/.config/systemd/user/frametop-input-relay.service
driver=$HOME/.local/share/frametop/ft_pointer
vrpathreg=/opt/steamvr/bin/linuxarm64/vrpathreg
handsctl=$HOME/.local/bin/ft-handsctl
eyegrab_files=(/etc/systemd/system/frametop-eyegrab.service /etc/frametop/ft-eyegrab)
bt_files=(/etc/systemd/system/steamframe-bt-fixups.service /etc/systemd/system/bluetooth.service.d/steamframe.conf
/etc/steamframe/bt-fixups.sh)
step() { printf '\n\033[1m== %s\033[0m\n' "$*"; }
run() { # run a command, or with --dry-run, show it
if [ "$dry" = 1 ]; then printf ' would run: %s\n' "$*"; else "$@"; fi
}
ask() { # ask "question" default(y|n)
local hint answer
if [ "$dry" = 1 ]; then echo "$1 [dry run: yes]"; return 0; fi
hint=$([ "$2" = y ] && echo "Y/n" || echo "y/N")
read -r -p "$1 [$hint] " answer </dev/tty || answer=
answer=${answer:-$2}
[[ $answer =~ ^[Yy] ]]
}
exists() { local f; for f in "$@"; do [ -e "$f" ] && return 0; done; return 1; }
size() { du -shc "$@" 2>/dev/null | tail -1 | cut -f1; }
# Is this folder Frametop's code? Only then is it deleted.
is_repo() { [ "$1" != "$HOME" ] && [ -f "$1/desktops.sh" ] && [ -f "$1/session/frametop-session.sh" ]; }
find_repo() {
local p
[ -n "$dir" ] && { echo "$dir"; return; }
# The launcher entry and the relay's unit point into the repo, until step 1 removes them.
p=$(sed -n 's#^Exec=\(.*\)/session/frametop-session\.sh.*#\1#p' "$override" 2>/dev/null | head -1)
[ -z "$p" ] && p=$(sed -n 's#^ExecStart=/usr/bin/python3 \(.*\)/input/input-relay\.py.*#\1#p' "$relay_unit" 2>/dev/null | head -1)
[ -z "$p" ] && [ -f "${BASH_SOURCE[0]:-}" ] && p=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
echo "${p:-$HOME/frametop}"
}
# Frametop's programs that are running now (step 1 leaves them running until the restart).
running() {
local n out=()
for n in ft-screens ft-pointer ft-powerd ft-eyegrab ft-camd ft-hands; do
pgrep -x "$n" >/dev/null && out+=("$n")
done
pgrep -f '[i]nput/input-relay\.py' >/dev/null && out+=("input relay")
pgrep -f '[g]aze/ft-gazed' >/dev/null && out+=("gaze service")
pgrep -f '[f]rametop-session\.sh' >/dev/null && out+=("desktop session")
echo "${out[*]}"
}
main() {
local units=() entries=() names=() sys=0 repo now f
while [ $# -gt 0 ]; do
case $1 in
--dir) dir=${2:?--dir needs a folder}; shift ;;
--dry-run) dry=1 ;;
-h|--help) usage; return 0 ;;
*) echo "unknown option: $1" >&2; usage >&2; return 2 ;;
esac
shift
done
if ! { grep -qx 'ID=steamos' /etc/os-release && grep -qE '^VARIANT_ID="?vr"?$' /etc/os-release; } 2>/dev/null; then
echo "This uninstalls Frametop from a Steam Frame (SteamOS, VR variant). Run it in a terminal on the headset." >&2
return 1
fi
if [ "$dry" = 0 ] && ! { : </dev/tty; } 2>/dev/null; then
echo "This asks questions, and there's no terminal to ask in. Run it in one (over SSH: ssh -t)." >&2
return 1
fi
[ "$dry" = 1 ] && echo "Dry run: nothing changes."
repo=$(find_repo)
# Step 1: what makes Frametop start. Nothing here stops a running program.
[ -f "$override" ] && grep -q 'Frametop' "$override" || override=
units=("$HOME"/.config/systemd/user/frametop-*.service)
for f in ft-input-settings ft-display-settings ft-layout-reset ft-screens-toggle ft-remote-settings ft-gazeprobe; do
[ -e "$apps/$f.desktop" ] && entries+=("$apps/$f.desktop")
done
[ -e "$apps/frametop-handrec.desktop" ] && entries+=("$apps/frametop-handrec.desktop")
entries+=("$apps"/frametop-profile-*.desktop) # one per layout profile (layout/ft_layout.py)
[ -L "$handsctl" ] || handsctl=
exists "${eyegrab_files[@]}" "${bt_files[@]}" && sys=1
if [ -n "$override" ] || [ ${#units[@]} -gt 0 ] || [ ${#entries[@]} -gt 0 ] || [ -d "$driver" ] ||
[ -n "$handsctl" ] || [ "$sys" = 1 ]; then
step "Step 1 of 2: stop Frametop from starting"
echo "This removes:"
[ -n "$override" ] && echo " - the launcher's Desktop entry (Launch a program -> Desktop opens the stock desktop again)"
for f in "${units[@]}"; do echo " - the service $(basename "$f")"; done
[ -d "$driver" ] && echo " - the 3D mouse's SteamVR driver (ft_pointer)"
[ ${#entries[@]} -gt 0 ] && echo " - ${#entries[@]} menu entries (Frametop Display Settings, Input Settings, ...)"
[ -n "$handsctl" ] && echo " - $handsctl"
exists "${eyegrab_files[@]}" && echo " - our eye tracker's frame grabber (a system service: needs your password)"
exists "${bt_files[@]}" && echo " - the Bluetooth fixes (system files: needs your password)"
echo "What runs now keeps running until you restart the headset, so your keyboard, mouse, and"
echo "this terminal keep working. Your settings and the code stay for now."
ask "Uninstall Frametop?" n || { echo "Nothing changed."; return 0; }
[ -n "$override" ] && run rm -f "$override"
if [ ${#units[@]} -gt 0 ]; then
for f in "${units[@]}"; do names+=("$(basename "$f")"); done
run systemctl --user disable "${names[@]}" 2>/dev/null || true
run rm -f "${units[@]}"
run systemctl --user daemon-reload
fi
if [ -d "$driver" ]; then
if [ -x "$vrpathreg" ]; then
LD_LIBRARY_PATH=$(dirname "$vrpathreg") run "$vrpathreg" removedriver "$driver" ||
echo "warning: SteamVR's vrpathreg couldn't unregister the driver; SteamVR may log that it's missing" >&2
fi
run rm -rf "$driver"
fi
[ ${#entries[@]} -gt 0 ] && run rm -f "${entries[@]}"
[ -n "$handsctl" ] && run rm -f "$handsctl"
if [ "$sys" = 1 ]; then
echo "The system files need your password (sudo)."
if ! run sudo bash -c '
for u in frametop-eyegrab steamframe-bt-fixups; do systemctl disable $u.service 2>/dev/null; done
rm -f "$@"
rmdir /etc/frametop /etc/steamframe /etc/systemd/system/bluetooth.service.d 2>/dev/null
systemctl daemon-reload; true' sys "${eyegrab_files[@]}" "${bt_files[@]}"; then
echo "warning: the system files weren't removed (no password?). Run this again to retry." >&2
fi
fi
[ "$dry" = 1 ] || echo "Frametop no longer starts."
fi
# Step 2: delete what's left, once nothing of Frametop runs.
now=$(running)
if [ -n "$now" ]; then
step "Restart the headset to finish"
echo "Still running from before: $now."
echo "They stop when the headset restarts. After that, run this again to delete the code"
echo "($repo) and, if you want, your settings and the build container:"
echo
if [ "$repo" = "$HOME/frametop" ]; then
echo " curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash"
else
echo " curl -fsSL https://deejanuz.github.io/frametop/uninstall.sh | bash -s -- --dir $(printf %q "$repo")"
fi
echo
if ask "Restart the headset now? This closes everything open, in VR and on the desktop." n; then
run systemctl reboot || echo "Couldn't restart it from here: restart the headset from Steam's power menu." >&2
fi
[ "$dry" = 1 ] || return 0
echo "Dry run: after the restart, step 2 would go like this."
fi
step "Step 2 of 2: delete what's left"
if [ -d "$repo" ] && is_repo "$repo"; then
if [ -f "$repo/.git" ]; then
echo "Leaving $repo: it's a git worktree. Remove it with git worktree remove."
else
local def=y
if [ -n "$(git -C "$repo" status --porcelain --untracked-files=no 2>/dev/null)" ] ||
[ -n "$(git -C "$repo" log --branches --not --remotes --oneline 2>/dev/null | head -1)" ]; then
echo "$repo has changes of its own (git -C $repo status)."
def=n
fi
if ask "Delete the Frametop code in $repo ($(size "$repo"))?" "$def"; then
cd "$HOME"
run rm -rf "$repo"
fi
fi
elif [ -e "$repo" ]; then
echo "Leaving $repo: it doesn't look like Frametop's code."
fi
# Rebuilt by the desktop at each start, so nothing to keep.
run rm -rf "$HOME/.local/share/frametop/apps" "$HOME/.local/share/kwin/decorations/kwin4_decoration_qml_frametop" \
"$HOME/.cache/frametop"
local settings=("$HOME"/.config/frametop.conf* "$HOME"/.config/frametop-*.json* "$HOME/.config/frametop-remote"
"$HOME/.config/frametop" "$HOME/.local/state/frametop")
local kept=()
for f in "${settings[@]}"; do [ -e "$f" ] && kept+=("$f"); done
if [ ${#kept[@]} -gt 0 ]; then
echo "Your settings: the screen layout and profiles, button maps, gaze calibration, the remote"
echo "desktop password, and the Frametop desktop's own Plasma setup (${kept[*]/#$HOME/\~})."
ask "Delete your settings too? Keep them to pick up where you left off if you reinstall." n &&
run rm -rf "${kept[@]}"
fi
local recs=()
for f in "$HOME/.local/share/frametop/eyes" "$HOME/.local/share/frametop/hands"; do [ -e "$f" ] && recs+=("$f"); done
if [ ${#recs[@]} -gt 0 ]; then
ask "Delete your eye and hand recordings in ~/.local/share/frametop ($(size "${recs[@]}"))?" n &&
run rm -rf "${recs[@]}"
fi
[ "$dry" = 1 ] || rmdir "$HOME/.local/share/frametop" 2>/dev/null || true
if command -v podman >/dev/null && podman container exists dev 2>/dev/null; then
echo "The build container (dev) holds Frametop's compilers and libraries, 1-2 GB. If you put"
echo "anything else in it, that goes too."
if ask "Delete the build container?" n; then
run "$HOME/.local/bin/distrobox" rm --force dev </dev/null
run podman image rm registry.fedoraproject.org/fedora-toolbox:44 >/dev/null 2>&1 || true
echo "distrobox stays in ~/.local/bin, for any other containers. To remove it too:"
echo " ~/dev/src/distrobox/uninstall --prefix ~/.local"
fi
fi
step "Done"
echo "Frametop is uninstalled."
}
dir=
main "$@"