Keep the Own tracker's calibration in reach, and let it learn after a re-seat

The calibration dots for the Own tracker go out to the Calibration ring angle each way on
an oval, not to the window's corners, which were too far to look at while facing the
centre. The Own tracker learns from drags up to 25 degrees (after taking the headset off
and on, its first clicks were 11-17 degrees off, and the 6-degree limit blocked them). The
probe starts on the Own tracker when it's running, and the calibration header names the
tracker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DeeJanuzandClaude Opus 5.5 committed 2026-09-29 22:51:37 -06:00
1 parent b76a8c50e0
commit c37bee27c9
2 files changed
+28 -14

No files matched your search

+1 -1
View File
@@ -32,7 +32,7 @@ Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction,
| SteamVR action | An `eyetracking` action bound to `/user/head/eyetracking` (`actions/`), read with `IVRInput::GetEyeTrackingDataRelativeToNow`. This is the supported way. | | SteamVR action | An `eyetracking` action bound to `/user/head/eyetracking` (`actions/`), read with `IVRInput::GetEyeTrackingDataRelativeToNow`. This is the supported way. |
| mmap set 1, set 2 | `/dev/shm/eye-server.mmap`, which SteamVR's eyetracking process writes for the HMD driver (`driver_cv.so`). It has two sets of per-eye directions in head space: set 1 is filtered, and its two eyes always share one pitch; set 2 is each eye's own reading. After each set come the tracker's variances for each eye, and at the end each eye's raw measurement and its variance (the tracker's confidence in that frame), which ft-gaze passes on for the fit check. | | mmap set 1, set 2 | `/dev/shm/eye-server.mmap`, which SteamVR's eyetracking process writes for the HMD driver (`driver_cv.so`). It has two sets of per-eye directions in head space: set 1 is filtered, and its two eyes always share one pitch; set 2 is each eye's own reading. After each set come the tracker's variances for each eye, and at the end each eye's raw measurement and its variance (the tracker's confidence in that frame), which ft-gaze passes on for the fit check. |
| Left eye, right eye | Each eye alone, from set 2: calibrate and test them to see what one eye is worth against both. The layout is undocumented (offsets are in `ft-gaze.cpp`) and may change with a SteamVR update. ft-gaze maps it read-only; the file also carries calibration clicks to the tracker and must never be written. | | Left eye, right eye | Each eye alone, from set 2: calibrate and test them to see what one eye is worth against both. The layout is undocumented (offsets are in `ft-gaze.cpp`) and may change with a SteamVR update. ft-gaze maps it read-only; the file also carries calibration clicks to the tracker and must never be written. |
| Own tracker | Our own tracker, from `~/Desktop/Projects/frame-eyes` (experimental). `tools/fe-live` runs it: a root helper copies the eye-camera frames out of SteamVR's buffers (read-only) and `fe-trackd` finds the pupils and glints and publishes the gaze in `/dev/shm/frame-eyes-gaze`. It keeps its own calibration, not SteamVR's: the probe's calibration with the tracker toggle on Own tracker fits it (its dots cover the corners and sides of the practice area, since the fit goes wrong past its dots), and practice clicks teach it how far the headset has moved on your face since. With Own tracker on, the probe hides SteamVR's gaze and draws a red dot where each eye alone puts it. ft-gaze reports it as `own` while it's running, and as `{"ok":0}` otherwise. | | Own tracker | Our own tracker, from `~/Desktop/Projects/frame-eyes` (experimental). `tools/fe-live` runs it: a root helper copies the eye-camera frames out of SteamVR's buffers (read-only) and `fe-trackd` finds the pupils and glints and publishes the gaze in `/dev/shm/frame-eyes-gaze`. It keeps its own calibration, not SteamVR's: the probe's calibration with the tracker toggle on Own tracker fits it (its dots go out to the Calibration ring angle each way, on an oval, since the fit goes wrong past its dots), and practice clicks teach it how far the headset has moved on your face since. With Own tracker on, the probe hides SteamVR's gaze and draws a red dot where each eye alone puts it. ft-gaze reports it as `own` while it's running, and as `{"ok":0}` otherwise. |
The tracker stops when the headset is off your head. SteamVR also calibrates gaze on its own from laser-mouse clicks, treating each click as a spot you were looking at. That includes mouse clicks through the Frametop pointer, so a click where the pointer's dot isn't what you're looking at teaches SteamVR a wrong sample (it only takes clicks within 5 degrees of your gaze). In the probe, use Enter or Space as the trigger: keys don't go through SteamVR's laser. See `Accept usercal` in `~/.local/share/Steam/logs/eyetracking.txt`. When the tracker loses an eye, the same log says `CEyePoseUKF L: Large dt` (or `R`) as it starts that eye over. The tracker stops when the headset is off your head. SteamVR also calibrates gaze on its own from laser-mouse clicks, treating each click as a spot you were looking at. That includes mouse clicks through the Frametop pointer, so a click where the pointer's dot isn't what you're looking at teaches SteamVR a wrong sample (it only takes clicks within 5 degrees of your gaze). In the probe, use Enter or Space as the trigger: keys don't go through SteamVR's laser. See `Accept usercal` in `~/.local/share/Steam/logs/eyetracking.txt`. When the tracker loses an eye, the same log says `CEyePoseUKF L: Large dt` (or `R`) as it starts that eye over.
+27 -13
View File
@@ -1409,7 +1409,7 @@ class Probe(Adw.ApplicationWindow):
the drag was too far to be the tracker's error.""" the drag was too far to be the tracker's error."""
if not self.w_snaps.get_active(): if not self.w_snaps.get_active():
return "clicked (learning is off)" return "clicked (learning is off)"
if off > self.SNAP_LEARN_MAX: if off > self.OWN_LEARN_MAX:
return f"dragged {off:.1f} deg: too far to be the tracker's error, not learned" return f"dragged {off:.1f} deg: too far to be the tracker's error, not learned"
if p.get("t_raw") is None: if p.get("t_raw") is None:
return "no sample time at the press" return "no sample time at the press"
@@ -1473,6 +1473,11 @@ class Probe(Adw.ApplicationWindow):
FLICK = 1.5 # degrees the gaze has to move from where it was at the press to step FLICK = 1.5 # degrees the gaze has to move from where it was at the press to step
REARM = 0.9 # ... and come back within, before the next glance steps again REARM = 0.9 # ... and come back within, before the next glance steps again
SNAP_LEARN_MAX = 6.0 # degrees: a lesson bigger than this (after the correction) is a wrong element SNAP_LEARN_MAX = 6.0 # degrees: a lesson bigger than this (after the correction) is a wrong element
# The Own tracker takes bigger ones: after the headset is taken off and put back on, its
# first clicks were 11-17 degrees off, and a 6-degree limit kept them from teaching it
# (frame-eyes, session 5). It keeps the median of an eye's last 5 clicks, so one click
# where you changed your mind does little harm.
OWN_LEARN_MAX = 25.0
def snap_layout(self): def snap_layout(self):
w, h = self.canvas_size() w, h = self.canvas_size()
@@ -1828,21 +1833,26 @@ class Probe(Adw.ApplicationWindow):
in the window), turned 20 degrees per round, and half the size in the middle round, in the window), turned 20 degrees per round, and half the size in the middle round,
so the fit sees the middle, halfway out, and the edge of your view. so the fit sees the middle, halfway out, and the edge of your view.
For the Own tracker: the centre, then the corners and the middle of each side of the For the Own tracker: the centre, then eight directions on an oval out to `Calibration
area practice targets use, half the size in the middle round. Its fit is quadratic and ring` degrees each way (as much as fits across and up the window), turned 20 degrees
goes wrong past its dots, and the ring (limited by the window's height) left the sides per round, and half the size in the middle round. Its fit is quadratic and goes
wrong past its dots, and the ring (limited by the window's height) left the sides
out: in frame-eyes' practice2 the worst clicks were all past the ring.""" out: in frame-eyes' practice2 the worst clicks were all past the ring."""
w, h = self.canvas_size() w, h = self.canvas_size()
cx, cy = w / 2, h / 2 cx, cy = w / 2, h / 2
if self.source == "own":
s = self.RING_SCALE[rnd]
left, right, bottom = cx - (cx - 80) * s, cx + (cx - 80) * s, cy + (cy - 80) * s
top = cy - (cy - 150) * s # clear of the round and dot text at the top
return [(cx, cy), (left, top), (cx, top), (right, top), (right, cy), (right, bottom),
(cx, bottom), (left, bottom), (left, cy)]
r = self.raw.get(self.source) r = self.raw.get(self.source)
px_per_deg = 1 / r[5] if r and r[5] else 32.0 px_per_deg = 1 / r[5] if r and r[5] else 32.0
radius = min(self.w_ring.get_value() * px_per_deg, w / 2 - 60, h / 2 - 60) * self.RING_SCALE[rnd] reach = self.w_ring.get_value() * px_per_deg
if self.source == "own":
k = self.RING_SCALE[rnd]
rx = min(reach, cx - 80) * k
up, down = min(reach, cy - 150) * k, min(reach, cy - 80) * k # clear of the text at the top
pts = [(cx, cy)]
for i in range(8):
a = math.radians(-90 + rnd * 20 + i * 45)
pts.append((cx + rx * math.cos(a), cy + (up if math.sin(a) < 0 else down) * math.sin(a)))
return pts
radius = min(reach, w / 2 - 60, h / 2 - 60) * self.RING_SCALE[rnd]
pts = [(cx, cy)] pts = [(cx, cy)]
for k in range(6): for k in range(6):
a = math.radians(-90 + rnd * 20 + k * 60) a = math.radians(-90 + rnd * 20 + k * 60)
@@ -2097,7 +2107,7 @@ class Probe(Adw.ApplicationWindow):
GLib.idle_add(self.area.queue_draw) # the pulse GLib.idle_add(self.area.queue_draw) # the pulse
total = len(self.ROUND_BG) * len(c["points"]) total = len(self.ROUND_BG) * len(c["points"])
n = c["round"] * len(c["points"]) + c["index"] + 1 n = c["round"] * len(c["points"]) + c["index"] + 1
self.text(cr, 60, 70, f"Calibration: round {c['round'] + 1} of 3 ({self.ROUND_NAMES[c['round']]}), dot {n} of {total}", self.text(cr, 60, 70, f"Calibration ({'Own tracker' if self.source == 'own' else 'SteamVR'}): round {c['round'] + 1} of 3 ({self.ROUND_NAMES[c['round']]}), dot {n} of {total}",
ink, 26) ink, 26)
hint = "Face the centre dot, look at the highlighted dot, and press. Esc cancels, S skips a dot." hint = "Face the centre dot, look at the highlighted dot, and press. Esc cancels, S skips a dot."
self.text(cr, 60, 108, c.get("retry") or hint, (0.8, 0.2, 0.1) if c.get("retry") and bright else self.text(cr, 60, 108, c.get("retry") or hint, (0.8, 0.2, 0.1) if c.get("retry") and bright else
@@ -2757,7 +2767,8 @@ def main():
ap = argparse.ArgumentParser(description="Eye tracking playground for the Frametop desktop") ap = argparse.ArgumentParser(description="Eye tracking playground for the Frametop desktop")
ap.add_argument("--screen", type=int, default=0, help="Frametop screen to open on (default: where it opens)") ap.add_argument("--screen", type=int, default=0, help="Frametop screen to open on (default: where it opens)")
ap.add_argument("--mode", choices=Probe.MODE_KEYS, help="start in this mode (fit: Headset fit)") ap.add_argument("--mode", choices=Probe.MODE_KEYS, help="start in this mode (fit: Headset fit)")
ap.add_argument("--source", choices=SOURCES, help="gaze source to start with (default: mmap2)") ap.add_argument("--source", choices=SOURCES,
help="gaze source to start with (default: own if frame-eyes' tracker is running, else mmap2)")
args, rest = ap.parse_known_args() args, rest = ap.parse_known_args()
app = Adw.Application(application_id="dev.frametop.GazeProbe", flags=Gio.ApplicationFlags.NON_UNIQUE) app = Adw.Application(application_id="dev.frametop.GazeProbe", flags=Gio.ApplicationFlags.NON_UNIQUE)
windows = [] windows = []
@@ -2766,6 +2777,9 @@ def main():
win = Probe(a, args.screen) win = Probe(a, args.screen)
if args.source: if args.source:
win.w_source.set_selected(SOURCES.index(args.source)) win.w_source.set_selected(SOURCES.index(args.source))
elif not win.own.ask("status").startswith("fail"):
# Our own tracker is running: that's what you're here to test.
win.w_source.set_selected(SOURCES.index("own"))
if args.mode: if args.mode:
win.set_mode(args.mode) win.set_mode(args.mode)
windows.append(win) windows.append(win)