mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-06 01:00:06 +02:00
Keyboard corrections are learned up to 30 degrees
A Meta+J hold's correction is always meant, but it went through the mouse's 8 degree nudge limit. Live, our tracker was 12 degrees off and every correction was dropped without a word. Debug output now says when one is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
580004bb00
commit
cbad7bc470
2 files changed
+15
-5
No files matched your search
+1
-1
@@ -28,7 +28,7 @@ Gaze as an input method for the whole desktop, without replacing anything of Ste
|
||||
|
||||
Looks down past the screens (under 20 degrees down, on no Frametop screen: a glance at the keyboard) aren't sent, so the pointer stays where it was instead of following you down, and eyes lost there aren't counted. It drops blinks (both eyes closing or lost), smooths with a fixation lock, and sends the result to the pointer helper 90 times a second. It follows SteamVR's eye tracking log, and when the headset goes back on (SteamVR starts its eye model over, and the error moves), older lessons count less, so the first few after relearn the offset.
|
||||
- The pointer helper's **gaze mode** (off by default: the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a mouse button or key combination mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look. Move the mouse and it's the mouse's, from where the gaze put it, for the last bit. Look well away (5 degrees) and the gaze takes it back. A press isn't sent at once: the pointer stops where the gaze put it, and if that's wrong, drag it onto what you meant with the button still held; the click happens where you let go. To drag something, hold the press still for half a second first (`POINTER_GAZE_HOLD`), then move. Outside games the pointer stays on while gaze mode is on, until a controller is picked up. The dot shows all the time (`POINTER_GAZE_DOT=moving`: only while the mouse moves it, while a press is held, and as a pulse when you click). Gaze mode works with the mouse and the keyboard, not the controllers ([docs/gaze-controllers.md](../docs/gaze-controllers.md) explains why).
|
||||
- **Keyboard clicks** (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, `POINTER_KEY_TAP`) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; let go of either to drop.
|
||||
- **Keyboard clicks** (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, `POINTER_KEY_TAP`) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse, but up to 30 degrees whatever `POINTER_GAZE_NUDGE_MAX` says: a keyboard correction is always meant). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; let go of either to drop.
|
||||
- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it a little (0.2 to 8 degrees) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the probe's one-dot check does the same). The helper only sends nudges up to `POINTER_GAZE_NUDGE_MAX` (8 degrees), and right after putting the headset back on our tracker can be further off than that. If so, raise it for a moment, or do a quick check. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
|
||||
- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself when gaze mode comes on without one) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
|
||||
- Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
|
||||
|
||||
Reference in new issue
Block a user