DeeJanuz--frametop/gaze

Gaze (experimental)

The Steam Frame's eye tracking as pointer input: a gaze mode for the 3D mouse (the pointer goes where you look, and the mouse does the last bit), and the tools to calibrate and measure it.

• ft-gaze (C++, OpenVR, runs in the dev container) reads the eye tracker and prints one JSON line per sample (90 Hz). For each source, it gives the gaze direction relative to the head and the Frametop screen pixel it lands on. The gaze service asks it for only the sources it uses (--sources, and sources LIST on its stdin): our tracker and mmap set 1 with Own tracker, about 700 bytes a line instead of 1.3 KB, and every source while a check or the calibration runs. So SteamVR's gaze action, the only source that calls into vrserver (twice a sample), is read only then. The probe gets them all.
• gazecal.py has what the probe and the gaze service share: the correction models, filters, and the reader for SteamVR's eye tracking log.
• tracker/ is our own eye tracker, an alternative to SteamVR's: ft-eyes finds the pupils and glints in the eye-camera frames that ft-eyegrab (a small root service) copies out of SteamVR's tracker. See "Our own eye tracker" below.
• probe/ft-gazeprobe (GTK 4, host Python) is a fullscreen playground, for developing the gaze tracking: day to day, the calibration and the checks run in the headset panel (Quick check, Calibrate, and Check headset fit on the Gaze page). It runs ft-gaze, draws where you're looking, measures accuracy, and tries out hold-to-adjust clicking with a calibration that learns from your adjustments.

Day to day, install the gaze service, then turn gaze mode on and calibrate on the Gaze page of Frametop Input Settings (Calibrate). The installer offers the gaze service (gaze/run.sh install) and then our own tracker (gaze/tracker/install.sh), which gaze mode uses once it's installed; the probe is installed by hand.

gaze/run.sh install           # the gaze service: builds ft-gaze and the panel, starts with SteamVR
gaze/ft-gazectl on            # the pointer follows your gaze (off: the mouse alone)
gaze/tracker/install.sh       # our own eye tracker's frame grabber (asks for sudo)
gaze/build.sh                 # build ft-gaze and the panel by hand
gaze/probe/install.sh         # development: build, and add Frametop Gaze Probe to the app menu
gaze/probe/ft-gazeprobe --screen 1

Gaze pointer

Gaze as an input method for the whole desktop, without replacing anything of SteamVR's:

• ft-gazed (host Python, a user service: gaze/run.sh install) runs ft-gaze and corrects its gaze. Two settings on the Gaze page of Frametop Input Settings (GAZE_TRACKER and GAZE_EYE in ~/.config/frametop.conf, read again when the file changes) pick whose eye tracking it uses and how it weights the eyes:

  • Eye tracker: our own (Own tracker: see "Our own eye tracker" below) or SteamVR's. The default, GAZE_TRACKER=auto, is ours when it's installed (its frame grabber, and ft-eyes' Python in the gaze service's checkout), else SteamVR's, and it switches when ours is installed or removed; picking one on the Gaze page sets it for good. The gaze service runs ours while it's the one in use. It keeps its own calibration: with Own tracker chosen, Calibrate on the Gaze page calibrates it. The gaze pointer's settings (hand back, nudges, hold to drag, the dot) are the pointer helper's, so they're the same with either.
  • Eye bias: Auto, Left, or Right. The gaze combines both eyes, each calibrated on its own, because their errors partly cancel: on 306 clicks with our tracker, the eyes' sideways errors were correlated -0.37, and both together were 0.65 degrees off (median) against 0.96 for the left eye alone and 1.11 for the right. So Left or Right leans instead of choosing: that eye counts twice as much as the other (0.03 degrees worse there toward the better eye, 0.13 toward the worse). Auto weights each eye by the inverse square of how far off it was at your last 20 nudges, once each eye has 5, and evenly before that. Each eye's miss is measured before that nudge teaches anything, so each is a fresh test. The calibration's own fit isn't used for this: on SteamVR's test of 2026-09-29, the calibration dots said the left eye was the better one, and new spots said the right. Either eye carries the gaze alone while the other is closed or lost.

  With SteamVR, each eye is its own reading (set 2), corrected by its calibration from the probe (the Left eye and Right eye sources) plus what the pointer has taught that eye since. On that test, the two eyes each calibrated and averaged were 1.70 degrees off (median; mean 1.62) against 1.72 (mean 1.84) for SteamVR's combined gaze with its calibration. A calibration from before the probe had the eyes as sources, or --source, uses the older path. That path runs on SteamVR's combined gaze (mmap set 1), corrected as a whole. When the tracker loses one eye (its variance for that eye jumps from about 0.001 to 0.02), the gaze comes from the other eye instead: that eye's own reading (set 2) plus what it usually reads against the combined gaze, learned while both eyes are seen, in 10 degree cells of where it looks. Set 1 keeps going on one eye too, but it holds the lost eye's yaw where it was, so the gaze moves half as far sideways as your eyes do. On a recording, one eye alone came out a median 0.8 degrees from both eyes' gaze over a steady look, a little more jittery.

  Looks down past the screens (under 20 degrees down, on no Frametop screen: a glance at the keyboard) aren't sent, so the pointer stays where it was instead of following you down, and eyes lost there aren't counted. It drops blinks (both eyes closing or lost), smooths with a fixation lock, and sends the result to the pointer helper 90 times a second. It follows SteamVR's eye tracking log, and when the headset goes back on (SteamVR starts its eye model over, and the error moves), older lessons count less, so the first few after relearn the offset.

• The pointer helper's gaze mode (off by default: the Gaze page of Frametop Input Settings, gaze/ft-gazectl on, POINTER_GAZE=1 in ~/.config/frametop.conf, or a mouse button or key combination mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look, and the mouse does the last bit. By default the mouse moves it only while a button is held (see "The mouse only corrects" below). With POINTER_GAZE_MOUSE_MOVE=free, moving the mouse takes the pointer, from where the gaze put it, and looking well away (5 degrees) gives it back to the gaze. A press isn't sent at once: the pointer stops where the gaze put it, and if that's wrong, drag it onto what you meant with the button still held; the click happens where you let go. To drag something, hold the press still for half a second first (POINTER_GAZE_HOLD), then move. Outside games the pointer stays on while gaze mode is on, until a controller is picked up. The dot shows all the time (POINTER_GAZE_DOT=moving: only while the mouse moves it, while a press is held, and as a pulse when you click). Gaze mode works with the mouse and the keyboard, not the controllers (docs/gaze-controllers.md explains why).

• The mouse only corrects (the default; the Gaze page's Mouse movement switch, POINTER_GAZE_MOUSE_MOVE=held): while the gaze has the pointer, moving the mouse does nothing. The buttons work like Meta+J and Meta+K: press and hold one and the pointer stops where you look; move the mouse onto what you meant and let go to click there (a left or a right click). Held still for half a second, a press is a real one (to drag). Once you've moved, the left button alone only clicks: press the right one while still holding the left to start a drag there; it lasts while either button is held. Press the right one again (a double right click, the left still held) to pan and tilt what you're dragging, as a right press does during any drag. A bumped or drifting mouse can't pull the pointer away, and every mouse move is a correction, so the tracker only learns from real ones. With the gaze stale for a second (the tracker stopped, eyes lost), in a game, or with the headset off, the mouse moves the pointer as usual. free (the switch off) lets the mouse take the pointer any time.

• Keyboard clicks (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, POINTER_KEY_TAP) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse, under the same limit: past POINTER_GAZE_NUDGE_MAX it opens the quick check instead). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; the drag lasts while either key is held. Meta+K during a Meta+J drag (again, after starting it with Meta+K: a double Meta+K) pans and tilts what you're dragging while it's held: turn your head to turn it.

• Learning from nudges: if the mouse took the pointer from the gaze and moved it (0.2 degrees or more, and the correction within POINTER_GAZE_NUDGE_MAX: 55 degrees by default, half of the 109 the headset shows across, and 1 to 110; the same limit for mouse, keyboard, and pinch clicks) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the quick check's dot does the same). A correction past POINTER_GAZE_NUDGE_MAX isn't learned: the helper asks ft-gazed for the quick check instead ("recheck", after its 2-minute cooldown). Tested on our tracker's 409 clicks since its Sep 29 calibration: a one-dot check set from any one of them put the next 2 minutes' clicks within 15 degrees (99% within 4.2) and the next 10 minutes' within 25 (the far ones after the headset moved), so the check gets back well under it. The limit used to be 8 degrees, and live on 2026-10-01 our tracker was 12 off after the headset went on, so every correction was dropped. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. ft-gazectl status shows the lessons, and ft-gazectl forget drops them.

• Checks and calibration in the headset (gaze/gazecheck.py, shown by gaze/panel/ft-gazepanel, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past POINTER_GAZE_NUDGE_MAX. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as checks.problem, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to checks.jsonl.

• Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.

• Idle while the gaze isn't used: ft-gaze and our own tracker run only while gaze mode is on and someone wears the headset (the pointer helper says both: SteamVR drops the headset's activity level as soon as it comes off), while a check or the calibration is open or asked for, or while the Gaze page of Frametop Input Settings is open (it renews a wake lease). 30 seconds after the last use they stop, and our frame grabber goes idle with our tracker. With our tracker, that saves over half a core: on 2026-10-02, with gaze mode off, ft-eyes took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. A check asked for while it idles starts the tracker and opens once it sends. When the gaze is used again, it takes a few seconds to come back, and our tracker's first click re-seats it, as after the headset was off: so the quick check opens when gaze mode comes on after the service idled, as it does when you put the headset on. ft-gazectl status says "awake", and "idle" says why it isn't. A stand that covers the proximity sensor makes the headset seem worn, so with gaze mode on it doesn't idle there.

Lessons are logged to pointer-lessons.jsonl: the raw gaze, the true direction, the correction at the time, and how far off it was.

Gaze sources

Source	Where it comes from
SteamVR action	An eyetracking action bound to /user/head/eyetracking (actions/), read with IVRInput::GetEyeTrackingDataRelativeToNow. This is the supported way.
mmap set 1, set 2	/dev/shm/eye-server.mmap, which SteamVR's eyetracking process writes for the HMD driver (driver_cv.so). It has two sets of per-eye directions in head space: set 1 is filtered, and its two eyes always share one pitch; set 2 is each eye's own reading. After each set come the tracker's variances for each eye, and at the end each eye's raw measurement and its variance (the tracker's confidence in that frame), which ft-gaze passes on for the fit check.
Left eye, right eye	Each eye alone, from set 2: calibrate and test them to see what one eye is worth against both. The layout is undocumented (offsets are in ft-gaze.cpp) and may change with a SteamVR update. ft-gaze maps it read-only; the file also carries calibration clicks to the tracker and must never be written.
Own tracker	Our own tracker (tracker/, experimental; see "Our own eye tracker"). It keeps its own calibration, not SteamVR's: Calibrate on the Gaze page fits it while Eye tracker is Own tracker (eight dots to a ring instead of six, on a slight oval, since the fit goes wrong past its dots; the probe's calibration with its tracker toggle on Own tracker does the same), and clicks teach it how far the headset has moved on your face since. After the headset was off, one look at a centre dot (the quick check, or the probe's first dot) resets that. With Own tracker on, the probe hides SteamVR's gaze and draws a red dot where each eye alone puts it, and asks the gaze service to keep the tracker running. The gaze pointer can use it too (Eye tracker: Own tracker, on the Gaze page of Frametop Input Settings). ft-gaze reports it as own while it's running, and as {"ok":0} otherwise.

The tracker stops when the headset is off your head. SteamVR also calibrates gaze on its own from laser-mouse clicks, treating each click as a spot you were looking at. That includes mouse clicks through the Frametop pointer, so a click where the pointer's dot isn't what you're looking at teaches SteamVR a wrong sample (it only takes clicks within 5 degrees of your gaze). In the probe, use Enter or Space as the trigger: keys don't go through SteamVR's laser. See Accept usercal in ~/.local/share/Steam/logs/eyetracking.txt. When the tracker loses an eye, the same log says CEyePoseUKF L: Large dt (or R) as it starts that eye over.

Our own eye tracker

gaze/tracker/ is an eye tracker of our own, because SteamVR's is about 1.5 degrees off after the best correction the gaze service can learn, and what's left is mostly look-to-look noise that no correction on top of its output can remove. Ours processes the eye cameras itself: 0.59 degrees (median) in its best live session against 0.83 for SteamVR's with the probe's correction, and after the headset was taken off and put back without recalibrating, 0.58 once your first clicks had taught it where the headset sat (tracker/findings.md has the measurements).

• ft-eyegrab (C, root, the system service frametop-eyegrab.service) copies the eye-camera frames (512x400, 90 fps per eye) out of the DMA-BUFs SteamVR's eyetracking process holds into /dev/shm/frametop-eyes-cams, owned by you. It maps them read-only, and it only copies while someone touches /dev/shm/frametop-eyes-want (ft-eyes and the recorder do, every second). Otherwise it holds none of the tracker's buffers. Its unit keeps only the capabilities that needs (CAP_SYS_PTRACE, CAP_DAC_READ_SEARCH, CAP_CHOWN). gaze/tracker/install.sh builds it and installs it to /etc/frametop with sudo, which it asks for (uninstall, status, and log too).
• ft-eyes (Python with numpy and OpenCV, in the dev container: gaze/tracker/build.sh puts the pinned requirements.txt in gaze/tracker/build/venv) finds each eye's pupil (dark threshold, closing, ellipse fit) and glint pair (eyes_pupil.py), and maps them to a gaze with a quadratic fit per eye (eyes_model.py). It follows the headset moving on your face with a per-eye shift, which your clicks teach, and uses the glints only to notice a sudden jump. It publishes the gaze in /dev/shm/frametop-eyes-gaze (ft-gaze's source own) and takes calibration dots and clicks on @ft_eyes. The gaze service runs it while Eye tracker is Own tracker, or while the probe uses it. State (the calibration, each eye's shift, the clicks) is in ~/.local/state/frametop/gaze/eyes/.
• lab/ has the tools for improving it on recordings. ft-eyes-record NAME (or ft-eyes-session, with SteamVR's gaze alongside) records the cameras. ft-eyes-score fits and scores on recordings against the probe's practice clicks. ft-eyes-e2e runs the whole live path on two recordings (calibrate on one, click through the other). ft-eyes-replay plays a recording into a scratch share. Heavy ones are meant for a PC: if you have frame-job (a personal tool, not in this repo), gaze/tracker/.frame-job sends them there. lab/py runs them with that Python (in the dev container on the Frame; on a PC, the same venv from requirements.txt, which frame-job's setup makes).

Ground rules, for anyone changing it:

• Clean room. Nothing of Valve's goes in: we don't decompile, disassemble, or patch the eyetracking binary or its network weights, and we don't copy their code or weights. Its public output (eye-server.mmap, read-only) is fair game as a baseline and as labels, and so are published papers and openly licensed pupil detectors (check each one's license: PuRe, PuReST, ElSe, and ExCuSe are non-commercial only).
• Root only reads. ft-eyegrab never writes to, stops, or signals the eyetracking process, vrserver, or vrcompositor, never opens /dev/adsp, /dev/cdsp, or /dev/spidev0.1, and never writes to /dev/shm/eye-server.mmap (it also carries calibration clicks into SteamVR's tracker), /opt, or /persist.
• Eye images are biometric data. Recordings live outside the repo, in ~/.local/share/frametop/eyes/captures (0700), and .gitignore catches stray frame dumps. They go nowhere but the machine that runs your offline jobs.
• Mind the headset's budget. Finding a pupil takes about 0.4 ms a frame while ft-eyes follows it, and 1.4-2.1 ms when it searches the whole frame. ft-eyes keeps OpenCV and numpy to one thread: their pools of one per core spun idle workers at about a quarter of a core, for frames this small. It also runs at nice 10 with SCHED_BATCH, and ft-gaze at nice 5 (not batch, since each sample goes on to the pointer): both run in the dev container's podman scope, out of reach of the gaze service's unit, on the cores vrcompositor and vrserver use at nice 0. Replays, scoring, and training go to a PC.

Headset fit

Check headset fit on the Gaze page opens it in the headset panel: a card per eye (tracked or lost, the tracker's signal, how much of the last 10 s it was seen) and the hints, live while you adjust the headset. A left click or Meta+J runs the guided check (dots, then looks down, up, left and right), and a right click or Meta+K closes it. The probe's Headset fit mode (ft-gazeprobe --mode fit, for development) has the same check with maps: it shows, for each eye, whether the tracker has it, how open it is, and the tracker's confidence in it, and a map of where you looked coloured by how often it lost that eye there. Hints under the maps say which eye gets lost where, and what to try. Enter runs a guided check: dots around the screen, then looking down at the keyboard, up, left and right. R starts over. Adjust the headset while you watch it.

Losing an eye is usually about where you look, not the tracker. On this Frame the left eye was lost 57 to 64 % of the time looking 30 to 50 degrees down (at the keyboard) and the right eye never; at screen height both were seen over 98 % of the time. Looking down, the lids come down over the eyes. That's harmless, since the gaze service ignores looks down past the screens: they show on the maps, but not in the counts or as a problem.

Probe

The probe is a development tool (in the Gaze page's overflow menu): calibration experiments, accuracy tests, and practice modes. Users calibrate and check in the headset panel instead.

The trigger is Enter, Space, or a mouse button. Right-click anywhere in the window (or press the Menu key or Shift+F10) for a menu with Run calibration, Start accuracy test, Calibrate from last test, Reset calibration, the modes, the panel, fullscreen, and Quit. The buttons at the top right show and hide the panel, leave fullscreen, and quit. The arrow in the panel's title bar collapses it to just that bar, so the dot and targets behind it stay visible; the collapsed bar stays through tests. The keys do the same (Tab, C, F11, Esc), but only after you click the window once, since Frametop sends typing to the panel you clicked last. If ft-gaze stops, the probe starts it again after 3 s and shows why it stopped. Windowed mode stays on the screen it was on, and a small KWin script tells the probe where the window is, so the dot and targets are still in the right place.

• Run calibration (start here): the initial calibration, modeled on Apple Vision Pro's eye setup. Face the centre and keep your head still. Look at one dot and press the trigger, then at each of six dots in a circle. That happens in three rounds, and the screen goes dark, then medium, then bright, because pupil size changes with brightness and the tracker's error with it. Each round turns the ring 20 degrees, and the middle round's ring is half the size, so the 21 dots cover the middle, halfway out, and the edge of your view. The ring's size is Calibration ring (degrees, 20 by default, less if the window is too small). Error grows toward the edge, and the calibration can only correct as far out as it has seen dots. The current dot is a bright pulsing dot with a point in the middle; finished dots fade to specks, so your eyes don't go back to them. Samples from blinks and from moments when the tracker lost an eye are dropped: openness under half of what it was during that look (not a fixed level, because your lids come down when you look down, and you squint in the bright round), or the angle between the eyes jumping more than 1.5 degrees from its median (that angle depends on how far away you're looking, so only a jump counts). Each dot is measured with medians, so one bad sample can't fail it. A look that lands where the gaze was for another dot of the round is refused as a look at the wrong dot. Mouse clicks don't count during a calibration run or a test: use Enter or Space. If a dot still fails, the message says why and the next try listens longer. After two failures, S (or the menu) skips the dot. Every attempt is logged to calibration-attempts.jsonl. At the end it fits every source's calibration from all the dots, replacing what it had learned (quadratic if the model was none). Esc cancels. The run is saved as calibration-*.json. With "Test after calibration" on (the default), the accuracy test starts right after, on new spots.
• Free look: the gaze dot. The trigger calibrates wherever you're looking (see below).
• Accuracy test: look at each target and press Enter or Space. "Test spots" picks where the targets go. Calibrated area (the default) puts 15 new spots inside the calibration ring (the centre, 7 halfway out, 7 near the ring), turned so none sits on a calibration dot. It checks the calibration where it was made, with your head facing the centre. The window grids reach past that area. On a wide screen that's far more than your eyes turn without your head, so they show how the calibration holds up beyond where it was made. For each source the test records the error before and after correction (degrees and pixels), the share of targets within 1 degree, jitter, and the corrected error by region of your view (centre, up, down-left, and so on), worst first. On screen, each target gets a faint line to the raw gaze and a solid line to where the corrected dot was, green under 1 degree, yellow under 2, red above. Tests since the last calibration are listed as a trend.
• Refine calibration: refits from the latest calibration run's dots plus every calibrated-area test since. It tries offset, affine, quadratic, and quadratic+grid, scoring each on points it wasn't fitted on (leave-one-out), and uses the best. Then test again: each test adds its targets, so test, refine, test is the loop. The scores are in the panel and in refinements.jsonl.
• Snap practice: a field of desktop-like elements (toolbar icons, list rows, buttons, tiles, small links), some close together, inside the calibrated area. The gaze snaps to the nearest element and highlights it, so the pointer lands on a whole element instead of a spot. Look at the orange one and tap Enter (or click) to click it. If the wrong one is highlighted, hold the press instead: the highlight locks and stops following your gaze. Glance toward the right one (look off to that side and back), and each glance steps the highlight to the next element that way. Or move the mouse, and the highlight follows it from where it was. Let go on the right one. A glance works however far off the tracker is, because only the eye movement counts, and the tracker gets that right: its error barely changes over a couple of degrees. Whichever element you let go on is taken as the one you were looking at when you pressed, and the gap from the gaze at the press is learned as the tracker's error there (not if it's over 6 degrees after the correction, which means a wrong element). That's what it would learn in real use, where nothing knows which element you meant. The probe does know (the orange one), so each click is also scored: right at the press, right in the end, and whether the snap would have been right with the calibration alone. Backspace takes back the last click's lesson. Clicks go to snaps.jsonl.
• Click practice: the white dot is a gaze pointer, the way it would be in real use. Look at the target and press (click, or Enter), and keep looking at it. The dot stops following your gaze. If it isn't on the target, keep holding and move the mouse: the dot moves with it. Let go on the target. You were looking at where you let go when you pressed, so the drag is the tracker's error there, and the click corrections learn it (not if it's over 6 degrees after the correction). A click without a drag teaches nothing: it only says the dot was close enough. The target is only for scoring: would a plain gaze click have hit at the press, and did the drag end on it. The drag is drawn for a moment. Presses go to practice.jsonl. The Frametop pointer (the mouse's own white dot) stays where the mouse puts it. The probe only reads its movement. An earlier version steered the Frametop pointer onto the gaze with the pointer helper's move commands. It lost the user's pointer: the helper's pointer goes idle, or a controller takes the laser, and the moves piled up. Taking over the real pointer belongs in the pointer helper itself, which knows its own state and can aim straight at the gaze.

The default trigger is freeze and look, the on-demand calibration. The press freezes the dot where the tracker says you're looking. Then look at the frozen dot: it's a target right where you're looking, and it stays put. After settle ms it averages the unsmoothed gaze for capture ms, or until you let go if you hold longer. The gap between the frozen dot and that average is the tracker's error at that spot, and the calibration learns it. The live dot is hidden while frozen so it can't pull your eye (the "Live dot while frozen" option shows it anyway). Freeze and look drops blink and dropout samples and uses medians, like the calibration run. A capture is thrown out if the gaze spread more than max spread (1 degree) or the error is over max error (12 degrees; the real error reaches 8-9 degrees looking well up or down). Each capture is logged to captures.jsonl.

The older triggers, nudge with head and nudge with eyes, are still there. Hold, then move the frozen dot onto what you meant with your head or eyes (nudge gain scales the movement), and release. When nudging with your eyes, don't look at the dot: it follows your gaze, error included, so it runs away.

Smoothing defaults to fixation lock. It holds the dot on the running mean of the current fixation and jumps when your gaze leaves the fixation radius. One Euro follows more smoothly, and its beta is per degree a second. Sitting still, raw gaze jitters by about 0.25-0.3 degrees, and mmap set 2 was the quietest source, so it's the default.

Click corrections ("Learn from clicks", on by default) are learned on the fly from snap and practice clicks, on top of the calibration. Right-click, Clear click corrections forgets them and keeps the calibration. The first click shifts the whole correction. More clicks bend it (the same quadratic terms, held near zero except the offset), and what's left near each click is added within about 3 degrees of it: on your data, errors less than 3 degrees apart are alike, and ones further apart aren't. Recent clicks count more, so it follows SteamVR's gaze as that moves. A big element only weakly says where on it you looked, so a wide list row barely counts sideways. Replayed on logged points, a calibration from an earlier session was 4.95 degrees off; one click brought that to 2.3, five to 1.6, and twenty to 1.2. They're saved with the calibration and start over with a new calibration run.

SteamVR's eye tracker also calibrates itself, from clicks (Accept usercal in ~/.local/share/Steam/logs/eyetracking.txt). It takes a quick mouse-button down and up as "you were looking there", if the gaze was held within 5 degrees of the click. In the logs, the accepted clicks were all under 0.14 s, one of 0.38 s was "too slow", and a click that moved between down and up was refused. It keeps that inside the running eyetracking process and saves nothing, so when SteamVR starts again, its calibration starts over and the raw gaze moves: your 13:39 and 21:23 sessions had a restart between them, and the error's shape changed, not just its offset. The panel shows when the eye tracker started, whether that was after your calibration, and how many clicks it has learned from since. Snap and practice clicks are what keep up with it: a drag is too slow for SteamVR to take, and a quick click on the snapped element teaches both calibrations the same spot.

Pick the Correction model in the panel or the right-click menu. Choosing one fits it right away from the latest calibration's dots and the calibrated-area tests since (points.jsonl), and the choice is saved with the calibration. The models (per source) are none, offset, affine (offset plus a straight-line change across your view), quadratic (the default), and affine+grid or quadratic+grid (plus a 10-degree grid for what's left). Quadratic is the second-order polynomial video eye trackers usually calibrate with. The Frame's error grows as your eyes turn away from the centre: it overstates vertical movement, more the further up or down you look, and looking up adds a sideways error. A straight line only follows part of that. A polynomial runs away outside the spots it was fitted on, so the model only follows it to 3 degrees past the range of view it has seen. They're keyed by where you're looking relative to your head, and saved in ~/.local/state/frametop/gaze/calibration.json. Freeze captures go to captures.jsonl, nudges to practice.jsonl, and test results to test-*.json in the same folder. Every calibration dot and test target is also added to points.jsonl: where in your view it was, the raw error, the error with the calibration of the time, and the spread. That's the data for refining, and for finding where the calibration is off.