Add our own eye tracker as a gaze source in ft-gaze and the probe

ft-gaze reads frame-eyes' /dev/shm/frame-eyes-gaze and reports it as the source "own",
with each eye's gaze, where each lands on the screen, and its slip. The probe gets a
SteamVR / Own tracker toggle. With Own tracker on, it hides SteamVR's gaze and draws a
red dot per eye, its calibration fits fe-trackd's own calibration, and practice clicks
teach fe-trackd instead of the probe's correction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DeeJanuzandClaude Opus 5.5 committed 2026-09-29 21:38:17 -06:00
1 parent 89522e1894
commit 30a9d15072
3 files changed
+263 -18

No files matched your search

+1
View File
@@ -32,6 +32,7 @@ Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction,
| SteamVR action | An `eyetracking` action bound to `/user/head/eyetracking` (`actions/`), read with `IVRInput::GetEyeTrackingDataRelativeToNow`. This is the supported way. |
| mmap set 1, set 2 | `/dev/shm/eye-server.mmap`, which SteamVR's eyetracking process writes for the HMD driver (`driver_cv.so`). It has two sets of per-eye directions in head space: set 1 is filtered, and its two eyes always share one pitch; set 2 is each eye's own reading. After each set come the tracker's variances for each eye, and at the end each eye's raw measurement and its variance (the tracker's confidence in that frame), which ft-gaze passes on for the fit check. |
| Left eye, right eye | Each eye alone, from set 2: calibrate and test them to see what one eye is worth against both. The layout is undocumented (offsets are in `ft-gaze.cpp`) and may change with a SteamVR update. ft-gaze maps it read-only; the file also carries calibration clicks to the tracker and must never be written. |
| Own tracker | Our own tracker, from `~/Desktop/Projects/frame-eyes` (experimental). `tools/fe-live` runs it: a root helper copies the eye-camera frames out of SteamVR's buffers (read-only) and `fe-trackd` finds the pupils and glints and publishes the gaze in `/dev/shm/frame-eyes-gaze`. It keeps its own calibration, not SteamVR's: the probe's calibration with the tracker toggle on Own tracker fits it, and practice clicks teach it how far the headset has moved on your face since. With Own tracker on, the probe hides SteamVR's gaze and draws a red dot where each eye alone puts it. ft-gaze reports it as `own` while it's running, and as `{"ok":0}` otherwise. |
The tracker stops when the headset is off your head. SteamVR also calibrates gaze on its own from laser-mouse clicks, treating each click as a spot you were looking at. That includes mouse clicks through the Frametop pointer, so a click where the pointer's dot isn't what you're looking at teaches SteamVR a wrong sample (it only takes clicks within 5 degrees of your gaze). In the probe, use Enter or Space as the trigger: keys don't go through SteamVR's laser. See `Accept usercal` in `~/.local/share/Steam/logs/eyetracking.txt`. When the tracker loses an eye, the same log says `CEyePoseUKF L: Large dt` (or `R`) as it starts that eye over.