mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-06 04:04:09 +02:00
Keep the Own tracker's calibration in reach, and let it learn after a re-seat
The calibration dots for the Own tracker go out to the Calibration ring angle each way on an oval, not to the window's corners, which were too far to look at while facing the centre. The Own tracker learns from drags up to 25 degrees (after taking the headset off and on, its first clicks were 11-17 degrees off, and the 6-degree limit blocked them). The probe starts on the Own tracker when it's running, and the calibration header names the tracker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b76a8c50e0
commit
c37bee27c9
2 files changed
+28
-14
No files matched your search
+1
-1
@@ -32,7 +32,7 @@ Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction,
|
||||
| SteamVR action | An `eyetracking` action bound to `/user/head/eyetracking` (`actions/`), read with `IVRInput::GetEyeTrackingDataRelativeToNow`. This is the supported way. |
|
||||
| mmap set 1, set 2 | `/dev/shm/eye-server.mmap`, which SteamVR's eyetracking process writes for the HMD driver (`driver_cv.so`). It has two sets of per-eye directions in head space: set 1 is filtered, and its two eyes always share one pitch; set 2 is each eye's own reading. After each set come the tracker's variances for each eye, and at the end each eye's raw measurement and its variance (the tracker's confidence in that frame), which ft-gaze passes on for the fit check. |
|
||||
| Left eye, right eye | Each eye alone, from set 2: calibrate and test them to see what one eye is worth against both. The layout is undocumented (offsets are in `ft-gaze.cpp`) and may change with a SteamVR update. ft-gaze maps it read-only; the file also carries calibration clicks to the tracker and must never be written. |
|
||||
| Own tracker | Our own tracker, from `~/Desktop/Projects/frame-eyes` (experimental). `tools/fe-live` runs it: a root helper copies the eye-camera frames out of SteamVR's buffers (read-only) and `fe-trackd` finds the pupils and glints and publishes the gaze in `/dev/shm/frame-eyes-gaze`. It keeps its own calibration, not SteamVR's: the probe's calibration with the tracker toggle on Own tracker fits it (its dots cover the corners and sides of the practice area, since the fit goes wrong past its dots), and practice clicks teach it how far the headset has moved on your face since. With Own tracker on, the probe hides SteamVR's gaze and draws a red dot where each eye alone puts it. ft-gaze reports it as `own` while it's running, and as `{"ok":0}` otherwise. |
|
||||
| Own tracker | Our own tracker, from `~/Desktop/Projects/frame-eyes` (experimental). `tools/fe-live` runs it: a root helper copies the eye-camera frames out of SteamVR's buffers (read-only) and `fe-trackd` finds the pupils and glints and publishes the gaze in `/dev/shm/frame-eyes-gaze`. It keeps its own calibration, not SteamVR's: the probe's calibration with the tracker toggle on Own tracker fits it (its dots go out to the Calibration ring angle each way, on an oval, since the fit goes wrong past its dots), and practice clicks teach it how far the headset has moved on your face since. With Own tracker on, the probe hides SteamVR's gaze and draws a red dot where each eye alone puts it. ft-gaze reports it as `own` while it's running, and as `{"ok":0}` otherwise. |
|
||||
|
||||
The tracker stops when the headset is off your head. SteamVR also calibrates gaze on its own from laser-mouse clicks, treating each click as a spot you were looking at. That includes mouse clicks through the Frametop pointer, so a click where the pointer's dot isn't what you're looking at teaches SteamVR a wrong sample (it only takes clicks within 5 degrees of your gaze). In the probe, use Enter or Space as the trigger: keys don't go through SteamVR's laser. See `Accept usercal` in `~/.local/share/Steam/logs/eyetracking.txt`. When the tracker loses an eye, the same log says `CEyePoseUKF L: Large dt` (or `R`) as it starts that eye over.
|
||||
|
||||
|
||||
+27
-13
@@ -1409,7 +1409,7 @@ class Probe(Adw.ApplicationWindow):
|
||||
the drag was too far to be the tracker's error."""
|
||||
if not self.w_snaps.get_active():
|
||||
return "clicked (learning is off)"
|
||||
if off > self.SNAP_LEARN_MAX:
|
||||
if off > self.OWN_LEARN_MAX:
|
||||
return f"dragged {off:.1f} deg: too far to be the tracker's error, not learned"
|
||||
if p.get("t_raw") is None:
|
||||
return "no sample time at the press"
|
||||
@@ -1473,6 +1473,11 @@ class Probe(Adw.ApplicationWindow):
|
||||
FLICK = 1.5 # degrees the gaze has to move from where it was at the press to step
|
||||
REARM = 0.9 # ... and come back within, before the next glance steps again
|
||||
SNAP_LEARN_MAX = 6.0 # degrees: a lesson bigger than this (after the correction) is a wrong element
|
||||
# The Own tracker takes bigger ones: after the headset is taken off and put back on, its
|
||||
# first clicks were 11-17 degrees off, and a 6-degree limit kept them from teaching it
|
||||
# (frame-eyes, session 5). It keeps the median of an eye's last 5 clicks, so one click
|
||||
# where you changed your mind does little harm.
|
||||
OWN_LEARN_MAX = 25.0
|
||||
|
||||
def snap_layout(self):
|
||||
w, h = self.canvas_size()
|
||||
@@ -1828,21 +1833,26 @@ class Probe(Adw.ApplicationWindow):
|
||||
in the window), turned 20 degrees per round, and half the size in the middle round,
|
||||
so the fit sees the middle, halfway out, and the edge of your view.
|
||||
|
||||
For the Own tracker: the centre, then the corners and the middle of each side of the
|
||||
area practice targets use, half the size in the middle round. Its fit is quadratic and
|
||||
goes wrong past its dots, and the ring (limited by the window's height) left the sides
|
||||
For the Own tracker: the centre, then eight directions on an oval out to `Calibration
|
||||
ring` degrees each way (as much as fits across and up the window), turned 20 degrees
|
||||
per round, and half the size in the middle round. Its fit is quadratic and goes
|
||||
wrong past its dots, and the ring (limited by the window's height) left the sides
|
||||
out: in frame-eyes' practice2 the worst clicks were all past the ring."""
|
||||
w, h = self.canvas_size()
|
||||
cx, cy = w / 2, h / 2
|
||||
if self.source == "own":
|
||||
s = self.RING_SCALE[rnd]
|
||||
left, right, bottom = cx - (cx - 80) * s, cx + (cx - 80) * s, cy + (cy - 80) * s
|
||||
top = cy - (cy - 150) * s # clear of the round and dot text at the top
|
||||
return [(cx, cy), (left, top), (cx, top), (right, top), (right, cy), (right, bottom),
|
||||
(cx, bottom), (left, bottom), (left, cy)]
|
||||
r = self.raw.get(self.source)
|
||||
px_per_deg = 1 / r[5] if r and r[5] else 32.0
|
||||
radius = min(self.w_ring.get_value() * px_per_deg, w / 2 - 60, h / 2 - 60) * self.RING_SCALE[rnd]
|
||||
reach = self.w_ring.get_value() * px_per_deg
|
||||
if self.source == "own":
|
||||
k = self.RING_SCALE[rnd]
|
||||
rx = min(reach, cx - 80) * k
|
||||
up, down = min(reach, cy - 150) * k, min(reach, cy - 80) * k # clear of the text at the top
|
||||
pts = [(cx, cy)]
|
||||
for i in range(8):
|
||||
a = math.radians(-90 + rnd * 20 + i * 45)
|
||||
pts.append((cx + rx * math.cos(a), cy + (up if math.sin(a) < 0 else down) * math.sin(a)))
|
||||
return pts
|
||||
radius = min(reach, w / 2 - 60, h / 2 - 60) * self.RING_SCALE[rnd]
|
||||
pts = [(cx, cy)]
|
||||
for k in range(6):
|
||||
a = math.radians(-90 + rnd * 20 + k * 60)
|
||||
@@ -2097,7 +2107,7 @@ class Probe(Adw.ApplicationWindow):
|
||||
GLib.idle_add(self.area.queue_draw) # the pulse
|
||||
total = len(self.ROUND_BG) * len(c["points"])
|
||||
n = c["round"] * len(c["points"]) + c["index"] + 1
|
||||
self.text(cr, 60, 70, f"Calibration: round {c['round'] + 1} of 3 ({self.ROUND_NAMES[c['round']]}), dot {n} of {total}",
|
||||
self.text(cr, 60, 70, f"Calibration ({'Own tracker' if self.source == 'own' else 'SteamVR'}): round {c['round'] + 1} of 3 ({self.ROUND_NAMES[c['round']]}), dot {n} of {total}",
|
||||
ink, 26)
|
||||
hint = "Face the centre dot, look at the highlighted dot, and press. Esc cancels, S skips a dot."
|
||||
self.text(cr, 60, 108, c.get("retry") or hint, (0.8, 0.2, 0.1) if c.get("retry") and bright else
|
||||
@@ -2757,7 +2767,8 @@ def main():
|
||||
ap = argparse.ArgumentParser(description="Eye tracking playground for the Frametop desktop")
|
||||
ap.add_argument("--screen", type=int, default=0, help="Frametop screen to open on (default: where it opens)")
|
||||
ap.add_argument("--mode", choices=Probe.MODE_KEYS, help="start in this mode (fit: Headset fit)")
|
||||
ap.add_argument("--source", choices=SOURCES, help="gaze source to start with (default: mmap2)")
|
||||
ap.add_argument("--source", choices=SOURCES,
|
||||
help="gaze source to start with (default: own if frame-eyes' tracker is running, else mmap2)")
|
||||
args, rest = ap.parse_known_args()
|
||||
app = Adw.Application(application_id="dev.frametop.GazeProbe", flags=Gio.ApplicationFlags.NON_UNIQUE)
|
||||
windows = []
|
||||
@@ -2766,6 +2777,9 @@ def main():
|
||||
win = Probe(a, args.screen)
|
||||
if args.source:
|
||||
win.w_source.set_selected(SOURCES.index(args.source))
|
||||
elif not win.own.ask("status").startswith("fail"):
|
||||
# Our own tracker is running: that's what you're here to test.
|
||||
win.w_source.set_selected(SOURCES.index("own"))
|
||||
if args.mode:
|
||||
win.set_mode(args.mode)
|
||||
windows.append(win)
|
||||
|
||||
Reference in new issue
Block a user