ft-gaze computed and printed all six sources for every sample: about 1.3 KB of JSON a line
with our tracker (practice2), 120 KB a second through podman's stdio relay for ft-gazed to
json.loads 90 times a second. It also read SteamVR's gaze action for every sample outside
games (UpdateActionState and GetEyeTrackingDataRelativeToNow, two calls into vrserver, 180 a
second), though with our tracker ft-gazed only uses own and mmap1.
- ft-gaze takes --sources LIST (action, mmap1, mmap2, left, right, own, and eye for the EYE
object; all by default, so the probe and ft-eyes-session are unchanged), and with
--watch-stdin a line "sources LIST" on stdin switches them. A source left out isn't read
and prints as {"ok":0} ("eye" as null), so every line keeps the same keys. An older
ft-gaze ignores both, and prints everything as before.
- ft-gazed asks for what it reads: own,mmap1 with our tracker; left,right,mmap1 with
SteamVR's eyes; the source plus mmap1 and mmap2 on the older one-source path. While a
check or the calibration runs or waits to open, all of them, since checks record every
source (the calibration fits the action's correction too) and the fit check reads "eye".
It switches as soon as that changes, well inside the check's 0.45 s settle.
- So the action is read only during checks, or with --source action.
On recorded samples, a line with own and mmap1 is about 700 bytes instead of 1,300
(practice2), and one with left, right and mmap1 about 550 instead of 940 (test1).
gaze/test/idle-test.py now checks that ft-gaze starts with every source for a check and is
then switched to those in use, without the action or own; all its checks pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-eyes and ft-gaze run in the dev container through distrobox, so they live in podman's
libpod scope: frametop-gaze.service's limits never reach them, and they ran at nice 0
next to vrcompositor and vrserver, also at nice 0.
- ft-eyes sets itself to nice 10 and SCHED_BATCH at start, before its threads. Batch
turns off wakeup preemption, so a frame ft-eyes wakes up for can wait out a running
compositor's turn; a few ms late costs the gaze little.
- ft-gaze sets nice 5 before its threads start, but stays SCHED_OTHER: each sample goes
on to the pointer, and batch would add the same wait to every one of them.
- Both only ever lower their priority (a higher nice already set wins), and a failure
is logged and ignored. Checked in the dev container: nice 0 -> 10, policy 0 -> 3
(SCHED_BATCH) without any capability.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From curiousjtuber's PR #13: with the gaze service running, SteamVR
restarted its eye tracker every 10 to 13 s of Beat Saber, as if the
headset came off, and each restart took input focus from the game.
The PR stopped every read in a game. Only the action path reaches
SteamVR (UpdateActionState on the gaze set at overlay-global priority,
then GetEyeTrackingDataRelativeToNow); the mmap and our tracker are
read-only files. So only the action is skipped while a scene app runs,
and gaze keeps moving the pointer over the dashboard in a game. The
action source is only used with --source action.
Co-Authored-By: CuriousJ <curious.j.tuber@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- desktops.sh uninstall also removes Launch as Standalone's app copies
and the title bar decoration, and Display Settings' uninstall the
profiles' launcher entries; its install writes them back from the
saved profiles (new: ft-layout launchers)
- the README says which settings an uninstall leaves
- the install command is now curl -fsSL https://deejanuz.github.io/
frametop/get.sh | bash (GitHub Pages, from main)
- ft-gaze logs "action manifest ...: ok" instead of "error 0"
Found in a clean install test on the Frame (2026-10-01): uninstall,
then get.sh from the headset into a fresh clone; everything installed
and doctor.sh passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- gaze/tracker/lab/eyes_track.py: nothing used it
- Input Settings: the pointer role backend, whose page was removed
- ft-screens: no log line for every floating-window resize
- hands: ft-hands --help gives the palm-down default (1: off), the
uninstall removes ft-handsctl's link, .frame-job is ignored
- Display Settings: "Save as profile…", not "Save current arrangement"
- two stale comments (ft-pointer's grabprobe, ft-gaze)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pointer and gaze services need steamvr.service to be running (Requisite=), and ft-pointer, ft-screens, and ft-gaze connect as a background app before switching to overlay, so they never start a vrserver of their own. One started from the dev container never finds the headset, which left a reboot stuck in a loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 51b4e79318)
frame-eyes, a separate project until now, becomes gaze/tracker:
- ft-eyegrab (was fe-bufprobe) copies the eye-camera frames, read-only, out of
SteamVR's eyetracking process. It runs as the system service
frametop-eyegrab.service, which gaze/tracker/install.sh installs to
/etc/frametop with sudo. It keeps only CAP_SYS_PTRACE, CAP_DAC_READ_SEARCH, and
CAP_CHOWN, and copies frames only while /dev/shm/frametop-eyes-want is fresh,
holding none of the tracker's buffers otherwise.
- ft-eyes (was fe-trackd) runs under ft-gazed in the dev container, with
build/venv's pinned numpy and OpenCV: while Eye tracker is Own tracker, or on
the probe's lease ("eyes SECONDS"). No sudo password or fe-live script at
run time any more.
- lab/ holds the research tools (ft-eyes-score, -e2e, -record, -replay,
-session) and findings.md. Recordings live outside the repo, in
~/.local/share/frametop/eyes/captures; .gitignore catches stray frame dumps.
Its socket is now @ft_eyes, its output /dev/shm/frametop-eyes-gaze, and its state
~/.local/state/frametop/gaze/eyes. On practice1 -> practice2 the whole live path
(ft-eyes-e2e) gives 1.30 deg median and 3.18 for the worst tenth, as before the
move (1.30, 3.21).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-gaze reads frame-eyes' /dev/shm/frame-eyes-gaze and reports it as the source "own",
with each eye's gaze, where each lands on the screen, and its slip. The probe gets a
SteamVR / Own tracker toggle. With Own tracker on, it hides SteamVR's gaze and draws a
red dot per eye, its calibration fits fe-trackd's own calibration, and practice clicks
teach fe-trackd instead of the probe's correction.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SteamVR's combined gaze keeps going on one eye, but it holds the lost
eye's yaw, so the gaze moves half as far sideways as the eyes do.
ft-gaze now reads each eye's tracking uncertainty and raw measurement
from eye-server.mmap. When the tracker loses an eye, ft-gazed takes the
gaze from the other one, plus the offset that eye usually shows
against both, learned while both are seen. On a recording, one eye
alone came out a median 0.8 degrees from both eyes' gaze.
Glances down at the keyboard, past every screen, aren't sent. The
pointer stays put, and eyes lost there don't count as lost.
The gaze probe gets a Headset fit mode. It shows per-eye tracking,
openness and confidence, maps where each eye gets lost, gives hints,
and has a guided check. The settings app opens it from the Gaze page
and shows how often each eye is lost. The probe can also test each eye
alone, and its side panel now collapses to a title bar so the dot
isn't hidden behind it.
Snapping to UI elements is deferred; the mouse drag is the correction.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both mmap sets now add "eyes": the left and right eye's head-relative
direction, so the eyes can be calibrated separately (each has its own offset
between where the tracker says it points and where it looks). Nothing uses
it yet; it gets logged with every sample.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-gaze reads the Frame's eye tracker (SteamVR's eyetracking action and the
two gaze sets in eye-server.mmap, read only) and prints where the gaze lands
on the Frametop screens. ft-gazeprobe is a GTK playground on top of it: a
Vision Pro style calibration run, an accuracy test, refining the correction
model from logged points, and click, drag, and snap practice that learn the
tracker's error on the fly. It also follows SteamVR's eyetracking log for
restarts and the clicks SteamVR calibrates itself from, which it keeps only
in memory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
