Gaze learning limit: 55 degrees, past it a quick check; a solid panel

Drops the quick check's second step (3629681): the user didn't want it, and it
re-entered itself every tick, re-placing and redrawing the panel, which
flickered in the headset.

POINTER_GAZE_NUDGE_MAX is the one learning limit, now 55 degrees by default:
half of the 109 the Frame shows across. The helper measures the correction
itself (raw gaze to click), not the mouse's path; past the limit it isn't
learned and the helper sends "recheck" for the quick check, for the mouse and
keyboard alike. ft-gazed's own limits (8 and 25) follow the setting.

Test, our tracker's 409 clicks since its Sep 29 calibration: a one-dot check
set from any one of them puts the next 2 minutes' clicks within 15 degrees (99%
within 4.2) and the next 10 minutes' within 25, so it gets well under 55.

The panel's dot is still and its capture ring fills in quarters, so it's drawn
again only a few times per dot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DeeJanuzandClaude Opus 5.5 committed 2026-10-01 10:48:57 -06:00
1 parent 362968198b
commit 8998fc1ddd
9 files changed
+76 -195

No files matched your search

+1 -1
View File
@@ -91,7 +91,7 @@ A few overlays need special handling:
Head follow is experimental and off by default. It works, but it's only lightly tested, and the feel is mostly a matter of its settings; polishing it is left open. With it on (`POINTER_FOLLOW=1`, or a mouse button mapped to Head follow on/off), the cursor rides on a reference direction, where you were facing when your head last settled, and keeps its offset from it. The mouse can put the cursor anywhere up to `POINTER_FOLLOW_REACH` (70 degrees) from the reference, a corner of your view included. While your head stays within `POINTER_LEASH_DEG` of the reference, nothing moves on its own. Once your head has been past the leash for `POINTER_LEASH_DELAY` (0.2 s, so a glance out and back doesn't count), the reference eases to where you're facing (time constant `POINTER_LEASH_RETURN`, 0.2 s), never falling further behind than the leash, and the cursor ends up back where it was in your view. Then it waits for the leash again. Two earlier versions didn't work out. Moving the reference only while your head pulled at the end of the leash left it up to the leash off after you turned back, and getting it centred again meant overshooting with your head. Easing it toward your facing all the time moved the cursor on every small head movement. A leash of 0 makes the reference your facing direction, so the cursor is locked to your view, and mouse movement shifts it within the view. Head roll is ignored, so tilting your head doesn't swing the cursor around. While the left button is held the cursor stays put in the room, so your head can't nudge a click or a drag. When you let go, it carries on from where it is instead of jumping.
Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. Moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: it needs accessibility (AT-SPI) on in the Frametop session, where it's off (no registry runs), plus app restarts, and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge of up to `POINTER_GAZE_NUDGE_MAX` (15 degrees) before a click is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on or our tracker thinks it moved, and the full calibration runs in the same panel. After the one-dot capture, the dot stays put in the room and you put the gaze-placed pointer on it, as you'd correct a click: that correction is learned whatever its size, and one past `POINTER_GAZE_NUDGE_MAX` runs the capture again. A keyboard click's correction past it opens the check too, since a keyboard correction is always meant. Its dots sit at known directions from the headset, so the panel needs no screen geometry, and each dot takes the gaze when it has held still rather than at a press: what the tracker says doesn't have to be close for the capture to work. See `gaze/README.md` for the service, the calibration, and what was measured.
Gaze mode is experimental and off by default (`POINTER_GAZE=1`, the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, or a mouse button or key combination mapped to Gaze pointer on/off). It's MAGIC pointing (Zhai, Morimoto and Ihde, 1999): the pointer goes where you look, and the mouse does the last bit. The gaze service (`gaze/ft-gazed`) sends the helper the corrected gaze at 90 Hz (from one eye while the tracker has lost the other), and while the gaze has the pointer, the cursor ray is that gaze from the eye. The pointer is aimed at the gaze each frame, not steered toward it, so nothing can pile up. An earlier try in the gaze probe steered the pointer with relative moves, and lost it when the pointer went idle or a controller had the laser. Moving the mouse takes the pointer from the gaze. A left press while the gaze has the pointer isn't sent at once: the pointer stops where the gaze put it, you drag it onto what you meant with the button still down (panels only see it hover), and the release clicks there. Clicking at once clicked wherever the gaze was, often the wrong thing, before you could correct it. The drag is the correction. Snapping the pointer onto buttons and links is deferred: it needs accessibility (AT-SPI) on in the Frametop session, where it's off (no registry runs), plus app restarts, and it makes Chromium and Electron apps use more CPU. A press held still for `POINTER_GAZE_HOLD` (0.5 s) becomes a real press, so drags still work: hold, then move. Outside games the pointer then stays: the mouse going idle doesn't release it. A moving controller still releases it, as without gaze. Gaze mode is a mouse and keyboard feature: Steam reads the Frame controllers itself, outside SteamVR's bindings, so controller clicks at the gaze kept knocking SteamVR out of laser mode (see `docs/gaze-controllers.md`). Keyboard clicks (Meta+J, Meta+K) hold the dot still in your view while the keys are down, so the head, not the mouse, does the last bit; a quick tap clicks where the dot was at the press, since the head moves as you hit the keys. The relay hides Meta from the desktop as soon as such a combination fires, because KWin takes Meta with a mouse button as a window move or resize, which swallowed the clicks. The dot shows all the time by default. With `POINTER_GAZE_DOT=moving` it shows only while the mouse moves it (`POINTER_GAZE_SHOW`), while a press is held, and as a pulse for each click; otherwise it's transparent, so the laser still lands on it. Looking more than `POINTER_GAZE_RETAKE` (5 degrees) away from it, with the mouse still, gives it back, so small eye movements around the pointer don't pull it off what you're doing. A mouse nudge before a click whose correction is within `POINTER_GAZE_NUDGE_MAX` (55 degrees, half of what the headset shows across) is sent to the gaze service as a lesson: you were looking at where you clicked when the mouse took over, so the nudge is the eye tracker's error there. Using it is what calibrates it. A one-dot check in a panel fixed to the headset tops that up when the headset goes on, when our tracker thinks it moved, and when a correction is past that limit (the tracker is far off, so a click there isn't trusted as a lesson), and the full calibration runs in the same panel. The limit was 8 degrees, which dropped every correction while our tracker was 12 off. Its dots sit at known directions from the headset, so the panel needs no screen geometry, and each dot takes the gaze when it has held still rather than at a press: what the tracker says doesn't have to be close for the capture to work. See `gaze/README.md` for the service, the calibration, and what was measured.
Replacing a loaded driver's files, as re-running the installer used to do, leaves SteamVR honoring the virtual controller's hand role but not its laser claim: the dashboard pointer stays unassigned until SteamVR restarts. The driver installer now leaves an unchanged driver in place.
+2 -2
View File
@@ -29,8 +29,8 @@ Gaze as an input method for the whole desktop, without replacing anything of Ste
Looks down past the screens (under 20 degrees down, on no Frametop screen: a glance at the keyboard) aren't sent, so the pointer stays where it was instead of following you down, and eyes lost there aren't counted. It drops blinks (both eyes closing or lost), smooths with a fixation lock, and sends the result to the pointer helper 90 times a second. It follows SteamVR's eye tracking log, and when the headset goes back on (SteamVR starts its eye model over, and the error moves), older lessons count less, so the first few after relearn the offset.
- The pointer helper's **gaze mode** (off by default: the Gaze page of Frametop Input Settings, `gaze/ft-gazectl on`, `POINTER_GAZE=1` in `~/.config/frametop.conf`, or a mouse button or key combination mapped to "Gaze pointer on/off") works like MAGIC pointing (Zhai et al., 1999). The pointer goes where you look. Move the mouse and it's the mouse's, from where the gaze put it, for the last bit. Look well away (5 degrees) and the gaze takes it back. A press isn't sent at once: the pointer stops where the gaze put it, and if that's wrong, drag it onto what you meant with the button still held; the click happens where you let go. To drag something, hold the press still for half a second first (`POINTER_GAZE_HOLD`), then move. Outside games the pointer stays on while gaze mode is on, until a controller is picked up. The dot shows all the time (`POINTER_GAZE_DOT=moving`: only while the mouse moves it, while a press is held, and as a pulse when you click). Gaze mode works with the mouse and the keyboard, not the controllers ([docs/gaze-controllers.md](../docs/gaze-controllers.md) explains why).
- **Keyboard clicks** (Meta+J left, Meta+K right; other key combinations on the Keyboard page of Input Settings): tap to click where you look. A quick tap (let go within 0.25 s, `POINTER_KEY_TAP`) clicks where the dot was when you pressed, whatever your head did, and tells the gaze service it was right there. Hold instead, and the dot stays put in your view: turn your head until it sits on what you meant, and let go to click there (the correction is a lesson, as with the mouse, but up to 30 degrees whatever `POINTER_GAZE_NUDGE_MAX` says: a keyboard correction is always meant). Hold still for half a second to press for real, then turn your head to drag. With Meta+J held, Meta+K presses where the dot is now, so you can correct first and then drag; let go of either to drop.
- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it a little (0.2 to 15 degrees, `POINTER_GAZE_NUDGE_MAX`) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the probe's one-dot check does the same). Nudges past `POINTER_GAZE_NUDGE_MAX` aren't learned, by the helper or by ft-gazed (they look like using the mouse, not correcting the gaze), but a keyboard click's correction is always meant, so one past it opens a quick check instead. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. In gaze mode, a second step follows: the dot stays where it is in the room, the gaze has the pointer again, and you put the pointer on the dot as you'd correct a click (the mouse, or Meta+J and your head). That click clicks nothing; it's learned whatever its size, and if the pointer was more than `POINTER_GAZE_NUDGE_MAX` (15 degrees) off, the dot comes back fixed to your view for another capture, up to 3 times. A right click or Meta+K skips the step; ignored, it closes after 20 s. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself when gaze mode comes on without one) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it (0.2 degrees or more, and the correction within `POINTER_GAZE_NUDGE_MAX`, 55 degrees: half of the 109 the headset shows across) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the probe's one-dot check does the same). A correction past `POINTER_GAZE_NUDGE_MAX` isn't learned: the helper asks ft-gazed for the quick check instead ("recheck", after its 2-minute cooldown). Tested on our tracker's 409 clicks since its Sep 29 calibration: a one-dot check set from any one of them put the next 2 minutes' clicks within 15 degrees (99% within 4.2) and the next 10 minutes' within 25 (the far ones after the headset moved), so the check gets back well under it. The limit used to be 8 degrees, and live on 2026-10-01 our tracker was 12 off after the headset went on, so every correction was dropped. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself when gaze mode comes on without one) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
- Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction, the correction at the time, and how far off it was.
+11 -13
View File
@@ -45,7 +45,7 @@ was when the mouse took over, and where the pointer was when you clicked (you we
there). The gap is the tracker's error there. The raw gaze is the one sent, so it also says
when that look was (the history of what was sent), and so what each eye read then:
- SteamVR: each eye learns its own error, unless the gaze was more than
POINTER_GAZE_NUDGE_MAX degrees (frametop.conf, 15 by default; the helper's limit too)
POINTER_GAZE_NUDGE_MAX degrees (frametop.conf, 55 by default: the helper's limit too)
past the correction (then it wasn't a nudge onto what you looked at);
- our tracker: unless it was more than POINTER_GAZE_NUDGE_MAX off, the look goes to it as
a click ("click T YAW PITCH" on @ft_eyes), as the
@@ -72,8 +72,8 @@ tracker's shared memory read-only.
Control socket: abstract unix datagram "@ft_gazed":
lesson <rhy> <rhp> <thy> <thp> from the pointer helper (see above)
calverify <rhy> <rhp> <thy> <thp> the same, the quick check's second step (gazecheck.py)
recheck <deg> a keyboard correction past POINTER_GAZE_NUDGE_MAX: a quick check
recheck <deg> from the pointer helper: a click's correction was past
POINTER_GAZE_NUDGE_MAX, so the quick check (gazecheck.py)
status reply: one JSON object
forget drop what the lessons taught (the calibration stays)
reload read calibration.json and the settings again
@@ -126,7 +126,7 @@ SOURCES = ("action", "mmap1", "mmap2", "left", "right") # the ones with a calib
SIDES = ("left", "right") # ft-gaze's order, and the sources for each eye alone
TRACKERS = ("steam", "own")
BIASES = ("auto", "left", "right")
NUDGE_MAX = 15.0 # degrees: POINTER_GAZE_NUDGE_MAX's default, the largest lesson taken
NUDGE_MAX = 55.0 # degrees: POINTER_GAZE_NUDGE_MAX's default, the largest lesson taken
HISTORY = 12.0 # seconds of the gaze sent, to find a lesson's look (the helper sends it up to 10 s later)
LOOK = 0.3 # seconds of samples before that moment make the look (the probe's fixation)
RETRY = 3.0 # seconds before starting ft-gaze again
@@ -166,7 +166,7 @@ def read_settings():
tracker = conf.get("GAZE_TRACKER", "steam")
bias = conf.get("GAZE_EYE", "auto")
try:
nudge = min(max(float(conf.get("POINTER_GAZE_NUDGE_MAX", NUDGE_MAX)), 1.0), 30.0) # the helper's range
nudge = min(max(float(conf.get("POINTER_GAZE_NUDGE_MAX", NUDGE_MAX)), 1.0), 110.0) # the helper's range
except ValueError:
nudge = NUDGE_MAX
return tracker if tracker in TRACKERS else "steam", bias if bias in BIASES else "auto", nudge
@@ -320,18 +320,16 @@ class Service:
eyes.append((statistics.median(e[0] for e in seen), statistics.median(e[1] for e in seen)) if seen else None)
return t, eyes
def lesson(self, rhy, rhp, thy, thp, limit=True):
"""Learn a lesson; limit=False takes it whatever its size (the quick check's own)."""
def lesson(self, rhy, rhp, thy, thp):
kind = self.kind
limit = self.nudge_max if limit else math.inf
rec = {"time": time.time(), "kind": kind, "raw": [rhy, rhp], "true": [thy, thp], "wear": self.steam.worn()}
if kind == "source":
dy, dp = thy - rhy, thp - rhp # the whole error there
cy, cp = self.correction(self.source, rhy, rhp)
left = math.hypot(dy - cy, dp - cp)
rec.update(source=self.source, model=self.mode, correction=[cy, cp], lesson_deg=left)
if left > limit:
rec["refused"] = f"more than {limit} deg past the correction"
if left > self.nudge_max:
rec["refused"] = f"more than {self.nudge_max:g} deg past the correction"
else:
self.lives[self.source].add({"time": rec["time"], "hy": rhy, "hp": rhp, "dy": dy, "dp": dp,
"wy": 1.0, "wp": 1.0, "how": "pointer"}, self.models[self.source], self.mode)
@@ -344,8 +342,8 @@ class Service:
eyes=eyes, weights=[round(w, 3) for w in weights.weights()])
if t is None:
rec["refused"] = "that gaze isn't in the last few seconds sent"
elif left > limit:
rec["refused"] = f"more than {limit} deg off"
elif left > self.nudge_max:
rec["refused"] = f"more than {self.nudge_max:g} deg off"
if "refused" in rec:
return self.taken(rec)
if kind == "own":
@@ -675,7 +673,7 @@ class Service:
elif words[:1] == ["forget"]:
self.forget_lessons()
reply = "ok"
elif words[:1] and words[0] in ("quickcal", "calibrate", "calaccept", "calquit", "calverify", "recheck"):
elif words[:1] and words[0] in ("quickcal", "calibrate", "calaccept", "calquit", "recheck"):
reply = self.checks.command(words)
elif words[:1] == ["eyes"] and len(words) == 2:
try:
+15 -76
View File
@@ -8,17 +8,9 @@ directions: look at each one.
stay on for hours with nobody in the headset. It also opens when our own tracker asks
for a click (its "reseat": the headset may sit differently on your face now), at most
once every QUICK_COOLDOWN, and on "quickcal" (Frametop Input Settings, or a mouse
button or key combination mapped to Gaze quick check). Ignored, it closes after
QUICK_TIMEOUT and changes nothing.
Then, in gaze mode, its second step: the dot stays where it is in the room (the
panel's "lock"), the gaze has the pointer again, and you bring the pointer onto the
dot as you'd correct a click, with the mouse or a keyboard click and the head (the
helper's "calpanel 2": that click clicks nothing). The helper sends it as
"calverify", and it's learned whatever its size. If the pointer was more than
POINTER_GAZE_NUDGE_MAX (15 degrees) off, the first step runs again, up to RECHECKS
times. A right click or Meta+K skips the step; ignored, it closes after
VERIFY_TIMEOUT. A keyboard click's correction past POINTER_GAZE_NUDGE_MAX outside a
check ("recheck", from the helper) opens a quick check too.
button or key combination mapped to Gaze quick check), and when a click's correction
was past POINTER_GAZE_NUDGE_MAX (55 degrees; the helper's "recheck"): the tracker is
far off. Ignored, it closes after QUICK_TIMEOUT and changes nothing.
five the middle and four around it, when the first FIVE_COUNT lessons after a quick check
were all over FIVE_LIMIT degrees off: the quick check didn't fix it.
full the calibration, as the gaze probe's: three rounds, dark, medium and bright (pupil
@@ -76,8 +68,6 @@ CHECK_SPREAD = 1.0 # degrees
ACCEPT_SPREAD = 2.5 # degrees: a capture asked for (calaccept) takes this much
DOT_TIMEOUT = 8.0 # seconds a dot of five or full waits; then it's skipped
QUICK_TIMEOUT = 6.0
VERIFY_TIMEOUT = 20.0 # seconds the quick check's second step waits for its click
RECHECKS = 3
QUICK_COOLDOWN = 120.0
DON_DELAY = 3.0 # seconds of eyes after AWAY_MIN without: the headset went on
AWAY_MIN = 3.0
@@ -312,7 +302,7 @@ class Checks:
now = time.monotonic()
self.check = {"kind": kind, "reason": reason, "own": own, "dots": check_dots(kind, own), "i": 0,
"started": now, "shown": now, "run": [], "accept": False, "done_at": None, "tries": 0,
"skipped": 0, "captured": 0, "points": {}, "verify_at": None, "rechecks": 0}
"skipped": 0, "captured": 0, "points": {}}
log(f"{kind} check: {reason}")
if kind == "full":
st = ask(SCREENS, "state", 0.5).split()
@@ -338,6 +328,7 @@ class Checks:
else:
self.to_panel("text Look at the dot")
self.to_panel(f"dot {yaw:.3f} {pitch:.3f} look")
self.last_progress = None
c["shown"] = time.monotonic()
c["run"], c["accept"], c["done_at"] = [], False, None
@@ -348,7 +339,7 @@ class Checks:
if self.away and self.back_since is None:
self.back_since = self.seen_at
c = self.check
if not c or c["done_at"] or c["verify_at"]:
if not c or c["done_at"]:
return
if c["own"]:
src = s["src"].get("own") or {}
@@ -373,10 +364,13 @@ class Checks:
run.append((now, s, g[0], g[1]))
window = [p for p in run if p[0] >= now - CHECK_WINDOW]
held = now - run[0][0]
if now - self.last_progress > 0.07:
# The ring fills in quarters: each step is a new picture for the panel to upload, so
# few of them keep it solid.
progress = math.floor(min(1.0, held / CHECK_WINDOW) * 4) / 4
if progress != self.last_progress:
yaw, pitch, _ = c["dots"][c["i"]]
self.to_panel(f"dot {yaw:.3f} {pitch:.3f} capture {min(1.0, held / CHECK_WINDOW):.2f}")
self.last_progress = now
self.to_panel(f"dot {yaw:.3f} {pitch:.3f} capture {progress:.2f}")
self.last_progress = progress
if c["accept"] and held >= 0.3 and len(window) >= 10:
sd, _ = spread([(p[2], p[3]) for p in window])
if sd <= ACCEPT_SPREAD:
@@ -493,9 +487,6 @@ class Checks:
if c["captured"]:
log(f"{kind} check done ({c['captured']} of {len(c['dots'])} dots)")
self.after_quick = [] if kind == "quick" else None
if kind == "quick" and self.gaze_on:
self.verify()
return
self.close()
return
n = len(c["dots"])
@@ -526,45 +517,6 @@ class Checks:
log(f"calibration ({c['captured']} of {n} dots): {mode}, saved")
self.close()
def verify(self):
"""The quick check's second step (see the top): bring the pointer onto the dot."""
c = self.check
yaw, pitch, _ = c["dots"][0]
c["verify_at"] = time.monotonic()
self.to_panel("lock")
self.to_panel(f"dot {yaw:.3f} {pitch:.3f} look")
self.to_panel("title Now put the pointer on the dot")
self.to_panel("text Right click or Meta+K: skip")
self.to_helper("calpanel 2")
def on_verify(self, rhy, rhp, thy, thp):
svc = self.svc
c = self.check
rec = svc.lesson(rhy, rhp, thy, thp, limit=False)
off = rec.get("lesson_deg", math.hypot(thy - rhy, thp - rhp))
self.log_check({"time": time.time(), "check": "verify", "raw": [rhy, rhp], "true": [thy, thp], "off": off,
"own": c["own"], "accepted": "refused" not in rec, "reply": rec.get("refused")})
log(f"quick check: the pointer was {off:.1f} deg off the dot"
+ (f" (not learned: {rec['refused']})" if "refused" in rec else ""))
if off <= svc.nudge_max:
self.close()
elif c["rechecks"] < RECHECKS:
c["rechecks"] += 1
log(f"more than {svc.nudge_max:.0f} deg: the quick check again ({c['rechecks']} of {RECHECKS})")
self.recapture()
else:
self.close(f"still more than {svc.nudge_max:.0f} deg off after {RECHECKS} more tries: "
"the full calibration should help (Calibrate on the Gaze page)")
def recapture(self):
"""The quick check's first step again, in the same check."""
c = self.check
now = time.monotonic()
c.update(i=0, started=now, verify_at=None, tries=0, done_at=None, captured=0, skipped=0)
self.to_panel("show quick") # fixed to the headset again
self.to_helper("calpanel 1")
self.show_dot()
def close(self, why=None):
if not self.check:
return
@@ -615,16 +567,8 @@ class Checks:
if cmd == "calquit":
self.quit()
return "ok"
if cmd == "calverify" and len(words) == 5:
if not self.check or not self.check["verify_at"]:
return "error no quick check is waiting for that"
try:
self.on_verify(*map(float, words[1:]))
except ValueError:
return "error bad numbers"
return "ok"
if cmd == "recheck" and len(words) == 2:
self.auto_quick(f"a keyboard correction was {words[1]} deg")
self.auto_quick(f"a click was corrected {words[1]} deg")
return "ok"
return "error unknown command"
@@ -655,10 +599,6 @@ class Checks:
if not self.eyes_seen(EYES_GONE):
self.close("the headset came off")
return
if c["verify_at"]:
if now - c["verify_at"] > VERIFY_TIMEOUT:
self.close("the pointer wasn't put on the dot")
return
if c["done_at"]:
return
if c["kind"] == "quick" and now - c["started"] > QUICK_TIMEOUT:
@@ -676,7 +616,7 @@ class Checks:
if now - self.gaze_heard > 5:
self.gaze_on = None # the helper isn't answering
if self.check:
self.to_helper("calpanel 2" if self.check["verify_at"] else "calpanel 1")
self.to_helper("calpanel 1")
if self.want_full_until:
cal = self.calibrated()
if cal is not None or now > self.want_full_until:
@@ -705,8 +645,7 @@ class Checks:
"last_quick_s": round(time.monotonic() - self.last_quick) if self.last_quick else None}
if c:
st["check"] = {"kind": c["kind"], "dot": c["i"] + 1, "dots": len(c["dots"]), "captured": c["captured"],
"skipped": c["skipped"], "reason": c["reason"], "verifying": bool(c["verify_at"]),
"rechecks": c["rechecks"]}
"skipped": c["skipped"], "reason": c["reason"]}
return st
def stop(self):
+6 -28
View File
@@ -12,15 +12,12 @@
//
// Control socket: abstract unix datagram "@ft_gazepanel" (--socket NAME); a sender with an
// address gets "ok" or "error ...":
// show quick|full the panel, empty, in front of you (fixed to the headset)
// lock stay where it is in the room now, until the next show (the
// quick check's second step: the head carries the pointer onto
// the dot, so the dot mustn't move with it)
// show quick|full the panel, empty, in front of you
// hide
// bg <0..1> the background's brightness (full)
// dot <yaw> <pitch> <state> [<progress 0..1>]
// the dot, head-relative degrees (yaw +left, pitch +up); state:
// look (pulsing), capture (a ring filling to progress), done,
// look, capture (a ring filling to progress), done,
// fail, off
// title <text> / text <text> a line at the top / at the bottom (empty to clear)
// ping
@@ -125,7 +122,6 @@ struct Panel {
bool dotOn = false;
double dotYaw = 0, dotPitch = 0, progress = 0;
std::string state = "off";
Clock::time_point stateAt = Clock::now();
};
void Blend(Panel &p, int x, int y, double r, double g, double b, double a) {
@@ -211,12 +207,10 @@ void Draw(Panel &p) {
double x, y;
ToPixel(p, p.dotYaw, p.dotPitch, x, y);
const double core = 0.22 * pxPerDeg;
const double t = std::chrono::duration<double>(Clock::now() - p.stateAt).count();
if (p.state == "look") {
// A ring that draws in toward the dot, again and again, and a still centre to look at.
const double phase = std::fmod(t * 1.4, 1.0);
const double rr = (1.6 - 1.1 * phase) * pxPerDeg;
Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, ink, ink, ink, 0.55 * (1 - phase) + 0.2);
// Still, so the panel looks solid and is drawn again only when something changes.
const double rr = 0.75 * pxPerDeg;
Disc(p, x, y, rr, rr - 0.12 * pxPerDeg, ink, ink, ink, 0.6);
Disc(p, x, y, core, 0, ink, ink, ink, 1);
} else if (p.state == "capture") {
const double rr = 0.75 * pxPerDeg;
@@ -232,7 +226,6 @@ void Draw(Panel &p) {
}
}
bool Animating(const Panel &p) { return p.dotOn && (p.state == "look"); }
} // namespace
@@ -300,7 +293,6 @@ int main(int argc, char **argv) {
};
std::fprintf(stderr, "ft-gazepanel running: @%s, %.2f m\n", sockName.c_str(), distance);
Clock::time_point lastDraw{};
while (!g_stop) {
char buf[512];
sockaddr_un from{};
@@ -320,24 +312,12 @@ int main(int argc, char **argv) {
place();
ov->ShowOverlay(h);
visible = dirty = true;
} else if (!std::strcmp(buf, "lock")) {
vr::TrackedDevicePose_t pose{};
vr::VRSystem()->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, &pose, 1);
if (pose.bPoseIsValid) {
// The headset's pose, moved out to the panel: where place() puts it now.
vr::HmdMatrix34_t m = pose.mDeviceToAbsoluteTracking;
for (int r = 0; r < 3; ++r) m.m[r][3] -= float(distance) * m.m[r][2];
ov->SetOverlayTransformAbsolute(h, vr::TrackingUniverseStanding, &m);
} else {
reply = "error no headset pose";
}
} else if (!std::strcmp(buf, "hide")) {
ov->HideOverlay(h);
visible = false;
} else if (std::sscanf(buf, "bg %lf", &a) == 1) {
p.bg = std::clamp(a, 0.0, 1.0), dirty = true;
} else if (std::sscanf(buf, "dot %lf %lf %15s %lf", &a, &b, state, &c) >= 3) {
if (p.state != state || a != p.dotYaw || b != p.dotPitch) p.stateAt = Clock::now();
p.dotYaw = a, p.dotPitch = b, p.state = state, p.progress = c;
p.dotOn = std::strcmp(state, "off") != 0, dirty = true;
} else if (!std::strncmp(buf, "title", 5)) {
@@ -359,11 +339,9 @@ int main(int argc, char **argv) {
vr::VRSystem()->AcknowledgeQuit_Exiting();
g_stop = true;
}
const auto now = Clock::now();
if (visible && (dirty || (Animating(p) && now - lastDraw > std::chrono::milliseconds(40)))) {
if (visible && dirty) {
Draw(p);
ov->SetOverlayRaw(h, p.px.data(), uint32_t(p.w), uint32_t(p.h), 4);
lastDraw = now;
dirty = false;
}
std::this_thread::sleep_for(std::chrono::milliseconds(visible ? 10 : 50));
+1 -1
View File
@@ -102,7 +102,7 @@ KEYBOARD_ONLY = ("gaze_left", "gaze_right")
# Gaze mode settings (pointer helper), like POINTER_SETTINGS.
GAZE_SETTINGS = [
("POINTER_GAZE_RETAKE", "Look away to hand back", 5, 1, 45, 0.5, "°"),
("POINTER_GAZE_NUDGE_MAX", "Largest correction to learn", 15, 1, 30, 0.5, "°"),
("POINTER_GAZE_NUDGE_MAX", "Largest correction to learn", 55, 1, 110, 1, "°"),
("POINTER_GAZE_HOLD", "Hold still to drag", 0.5, 0.1, 2.0, 0.05, "s"),
("POINTER_GAZE_SHOW", "Dot shows after moving", 1.0, 0.0, 5.0, 0.1, "s"),
]
+2 -3
View File
@@ -1057,9 +1057,8 @@ Kirigami.ApplicationWindow {
+ "to switch it on the fly. A click waits for the release: if the gaze is off, drag onto the target "
+ "with the button held and let go there. Hold still to drag: hold a press this long without moving "
+ "to drag something instead. Look away to hand back: how far from the pointer you look before the "
+ "gaze takes it back from the mouse. Largest correction to learn: bigger mouse moves before a click "
+ "are treated as using the mouse, not correcting the gaze; a bigger correction with a keyboard "
+ "click, or at the quick check, runs the quick check again. Eye tracker: SteamVR's, or our own "
+ "gaze takes it back from the mouse. Largest correction to learn: a click corrected further than "
+ "this isn't learned; the tracker is that far off, so the quick check runs instead. Eye tracker: SteamVR's, or our own "
+ "(gaze/tracker), which keeps its own calibration (Calibrate… with Own tracker). Eye bias: the gaze "
+ "combines both eyes, since their errors partly cancel; Left or Right counts that eye twice as "
+ "much, and Auto weights each by how far off it was at your recent nudges."
+37 -70
View File
@@ -160,9 +160,14 @@
// you were looking at: from the raw gaze when the mouse took over to where you clicked is
// the tracker's error there. The helper sends it to ft-gazed as a lesson ("lesson <raw yaw>
// <raw pitch> <true yaw> <true pitch>", the true direction relative to the head as it was
// when the mouse took over) if the mouse moved between 0.2 deg and POINTER_GAZE_NUDGE_MAX
// (8 deg) and the click came within 10 s; more is using the mouse, not a nudge. A held
// press dragged onto the target is the same: from the raw gaze at the press to the release. With no
// when the mouse took over) if the mouse moved at least 0.2 deg, the click came within 10 s,
// and the correction (raw gaze to click) is within POINTER_GAZE_NUDGE_MAX (55 deg, half what
// the headset shows across: the Frame's eyes see 109 deg each). Past that, the tracker is far
// off (or you went somewhere else with the mouse): it isn't learned, and ft-gazed is asked
// for its quick check instead ("recheck <deg>"; it waits out its cooldown). Our tracker's
// clicks since its calibration put a one-dot check within 15 deg for the next 2 minutes and
// 25 for the next 10, so the check gets back under it (2026-10-01). A held press dragged onto
// the target is the same: from the raw gaze at the press to the release. With no
// fresh gaze (a blink, the service stopped, the headset off), the pointer stays put.
//
// Gaze precision (the relay's gaze_precision and gaze_drag actions, bound to a mouse button or
@@ -181,8 +186,7 @@
// your view, so turning your head carries it onto what you meant (past
// POINTER_HEAD_DEADZONE, 0.5 deg, so a still head doesn't wobble it). The release clicks there
// (left, or right for gaze_right), and a correction is a lesson for the gaze tracker, as with
// the mouse. One past POINTER_GAZE_NUDGE_MAX isn't learned, but since a keyboard correction is
// always meant, the gaze service is told ("recheck <deg>") and runs its quick check. Held still for POINTER_GAZE_HOLD instead, it's a real press, and the head drags.
// the mouse. Held still for POINTER_GAZE_HOLD instead, it's a real press, and the head drags.
// A quick tap (let go within POINTER_KEY_TAP, 0.25 s) clicks where the dot was at the press,
// whatever the head did meanwhile, and tells the gaze tracker it was right there (a lesson
// with no correction). Pressing gaze_right while gaze_left aims (Meta+K with Meta+J held)
@@ -192,11 +196,6 @@
// ft-gazed says it's up ("calpanel 1", renewed every second; it lapses 3 s after the last),
// the dot hides and a press answers the panel instead of clicking: a left click or gaze_left
// sends "calaccept" to @ft_gazed (take this dot now), a right click or gaze_right "calquit".
// "calpanel 2" is the quick check's second step: the dot it captured stays put in the room, the
// gaze has the pointer again, and you bring the pointer onto the dot as you'd correct a click
// (mouse, or a keyboard click and the head). That click clicks nothing: it goes to @ft_gazed
// as "calverify <raw yaw> <raw pitch> <true yaw> <true pitch>", a lesson with no size limit. A
// press doesn't become a real one when held still; a right click or gaze_right is "calquit".
// POINTER_ROLE (right, left, or stylus): the hand role our device takes while connected. A
// Frame controller in your hand counts as used through its touch sensors and takes its hand's
// role back, and then no click lands (see "no hand role" in the main loop): with a controller
@@ -274,7 +273,7 @@
// panel, and a laser that starts behind them can't hit them. POINTER_FOLLOW (0) and
// POINTER_LEASH_DEG (10), POINTER_LEASH_DELAY (0.2 s), POINTER_LEASH_RETURN (0.2 s),
// POINTER_FOLLOW_REACH (70 deg): head follow, above. POINTER_GAZE (0), POINTER_GAZE_RETAKE
// (5 deg), POINTER_GAZE_NUDGE_MAX (15 deg), POINTER_GAZE_HOLD (0.5 s), POINTER_GAZE_DOT
// (5 deg), POINTER_GAZE_NUDGE_MAX (55 deg), POINTER_GAZE_HOLD (0.5 s), POINTER_GAZE_DOT
// (always), POINTER_GAZE_SHOW (1 s): gaze mode, above. POINTER_CONTROLLER_PICKUP (1, 0.5 to 5): how hard a controller must
// move to take the laser back, above. POINTER_IGNORE (empty): ignored panels, above.
// POINTER_HANDS (0), POINTER_PINCH_GAIN (0.5), POINTER_PINCH_DEADZONE (1.5 deg),
@@ -688,7 +687,7 @@ int main() {
bool follow = false, followConf = false, followReset = true;
// Gaze mode (see the top); gazeConf is POINTER_GAZE as last read, like followConf.
bool gazeOn = false, gazeConf = false;
double gazeRetake = 5, gazeNudgeMax = 15, gazeHold = 0.5, gazeShow = 1;
double gazeRetake = 5, gazeNudgeMax = 55, gazeHold = 0.5, gazeShow = 1;
bool gazeDotAlways = true; // POINTER_GAZE_DOT (see the top)
double headDeadzone = 0.5, keyTap = 0.25; // POINTER_HEAD_DEADZONE, POINTER_KEY_TAP (keyboard clicks, see the top)
// Hands (see the top): POINTER_HANDS, POINTER_PINCH_GAIN, POINTER_PINCH_DEADZONE, POINTER_GRIP_GAIN.
@@ -717,7 +716,7 @@ int main() {
const bool wantFollow = ConfDouble(conf, "POINTER_FOLLOW", 0) != 0;
if (wantFollow != followConf) follow = followConf = wantFollow, followReset = true;
gazeRetake = std::clamp(ConfDouble(conf, "POINTER_GAZE_RETAKE", 5), 1.0, 45.0);
gazeNudgeMax = std::clamp(ConfDouble(conf, "POINTER_GAZE_NUDGE_MAX", 15), 1.0, 30.0);
gazeNudgeMax = std::clamp(ConfDouble(conf, "POINTER_GAZE_NUDGE_MAX", 55), 1.0, 110.0);
gazeHold = std::clamp(ConfDouble(conf, "POINTER_GAZE_HOLD", 0.5), 0.1, 5.0);
gazeShow = std::clamp(ConfDouble(conf, "POINTER_GAZE_SHOW", 1), 0.0, 30.0);
headDeadzone = std::clamp(ConfDouble(conf, "POINTER_HEAD_DEADZONE", 0.5), 0.0, 5.0);
@@ -911,48 +910,31 @@ int main() {
bool pressRight = false;
std::string heldButton = "trigger";
bool confirmLesson = false; // a keyboard click's quick tap: a lesson with no correction (see the top)
bool keyLesson = false; // the click is a keyboard click's correction (see the top)
// The gaze calibration panel is up until then ("calpanel 1|2"; see the top); calOpened: it just
// The gaze calibration panel is up until then ("calpanel 1"; see the top); calOpened: it just
// came up, so a press in progress ends without a click.
Clock::time_point calPanelUntil{};
bool calOpened = false;
int calMode = 0; // 1: presses answer the panel; 2: the quick check's second step (see the top)
auto calVerifying = [&] { return calMode == 2 && Clock::now() < calPanelUntil; };
// The nudge as "<word> <raw yaw> <raw pitch> <true yaw> <true pitch>" (see the top).
auto nudgeMessage = [&](const char *word) {
const Vec3 d = RotateInverse(nudgeHead, Normalize(lastPoint - Position(nudgeHead)));
char msg[160];
std::snprintf(msg, sizeof msg, "%s %.3f %.3f %.3f %.3f", word, nudgeRawHy, nudgeRawHp,
std::atan2(-d.x, -d.z) * 180 / M_PI, std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI);
if (debug) std::printf("gaze %s (nudged %.2f deg)\n", msg, nudgeMoved);
if (debug) std::fflush(stdout);
return std::string(msg);
};
auto pressLeft = [&] {
const bool haveNudge = gazeOn && nudging && !gazeOwns && havePoint &&
Clock::now() - nudgeAt < std::chrono::seconds(10);
if (calVerifying()) {
// The quick check's second step (see the top): the click is the correction, whatever
// its size, and goes to the gaze service; nothing is clicked.
if (haveNudge) SendTo(out, "ft_gazed", nudgeMessage("calverify"));
else if (debug) std::printf("calverify: no gaze at the press\n");
nudging = confirmLesson = keyLesson = pressRight = gazeBack = false;
if (gazeOn) gazeOwns = true;
pulseAt = Clock::now();
return;
}
// ft-screens sends the keyboard to the panel clicked last; it sees clicks on
// its own screens, but only we know when one lands on another panel.
SendTo(out, "ft_screens", "click " + (lastHit.empty() ? std::string("-") : lastHit));
// A click after nudging the gaze-placed pointer: the nudge is a lesson (see the top).
if (haveNudge && (confirmLesson || (nudgeMoved >= 0.2 && nudgeMoved <= gazeNudgeMax)))
SendTo(out, "ft_gazed", nudgeMessage("lesson"));
else if (haveNudge && nudgeMoved > gazeNudgeMax) {
if (debug) std::printf("gaze nudge %.2f deg: over %.0f, not learned\n", nudgeMoved, gazeNudgeMax);
// A click after nudging the gaze-placed pointer: the nudge is a lesson, or past
// POINTER_GAZE_NUDGE_MAX, a quick check (see the top).
if (gazeOn && nudging && !gazeOwns && havePoint && Clock::now() - nudgeAt < std::chrono::seconds(10) &&
(confirmLesson || nudgeMoved >= 0.2)) {
const Vec3 d = RotateInverse(nudgeHead, Normalize(lastPoint - Position(nudgeHead)));
const double ty = std::atan2(-d.x, -d.z) * 180 / M_PI, tp = std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI;
const double off = std::hypot(std::remainder(ty - nudgeRawHy, 360.0), tp - nudgeRawHp);
char msg[160];
if (off <= gazeNudgeMax)
std::snprintf(msg, sizeof msg, "lesson %.3f %.3f %.3f %.3f", nudgeRawHy, nudgeRawHp, ty, tp);
else
std::snprintf(msg, sizeof msg, "recheck %.0f", off);
SendTo(out, "ft_gazed", msg);
if (debug) std::printf("gaze %s (nudged %.2f deg, off %.2f)\n", msg, nudgeMoved, off);
if (debug) std::fflush(stdout);
if (keyLesson) SendTo(out, "ft_gazed", "recheck " + std::to_string(int(std::lround(nudgeMoved))));
}
nudging = confirmLesson = keyLesson = false;
nudging = confirmLesson = false;
leftHeld = true;
dragDistance = lastDistance;
pressKey.clear();
@@ -985,8 +967,8 @@ int main() {
// the top).
auto leftButton = [&](bool down) {
if (down) {
if (gazeOn && (gazeMousePrecision || calVerifying()) && gazeOwns && !aimHeld && !clickPress &&
!clickRelease && hold.src == Src::None) {
if (gazeOn && gazeMousePrecision && gazeOwns && !aimHeld && !clickPress && !clickRelease &&
hold.src == Src::None) {
// Hold the press back: the pointer stops where the gaze put it.
gazeOwns = false;
nudging = haveHead && Clock::now() - gz.at < std::chrono::milliseconds(200);
@@ -1231,23 +1213,11 @@ int main() {
if (std::sscanf(buf, "calpanel %d", &on) == 1) {
const bool was = Clock::now() < calPanelUntil;
calPanelUntil = on ? Clock::now() + std::chrono::seconds(3) : Clock::time_point{};
calMode = std::clamp(on, 0, 2);
if (on && !was) calOpened = true;
continue;
}
}
if (calVerifying()) {
// The quick check's second step: left presses are the correction (pressLeft); a
// right press skips it.
if (!std::strncmp(buf, "btn b 1", 7) || !std::strncmp(buf, "gazekey right 1", 15)) {
SendTo(out, "ft_gazed", "calquit");
continue;
}
if (!std::strncmp(buf, "btn b 0", 7) || !std::strncmp(buf, "gazekey right 0", 15) ||
!std::strncmp(buf, "precision ", 10) || !std::strncmp(buf, "gazedrag ", 9))
continue;
}
if (calMode == 1 && Clock::now() < calPanelUntil) {
if (Clock::now() < calPanelUntil) {
const bool accept = !std::strncmp(buf, "btn trigger 1", 13) || !std::strncmp(buf, "gazekey left 1", 14);
const bool quit = !std::strncmp(buf, "btn b 1", 7) || !std::strncmp(buf, "gazekey right 1", 15);
if (accept || quit) {
@@ -1563,7 +1533,6 @@ int main() {
} else {
clickPress = true; // the click is on the release, where the pointer is now
pressRight = hold.right;
keyLesson = hold.src == Src::Head;
gazeBack = nudgeMoved < 0.2;
}
}
@@ -1697,7 +1666,6 @@ int main() {
headAngles(hold.refYaw, hold.refPitch);
gazeBack = gazeOn;
pressRight = false;
keyLesson = true;
pressLeft();
if (debug) std::printf("hold key left: pressed (right key)\n");
if (debug) std::fflush(stdout);
@@ -2029,7 +1997,7 @@ int main() {
auto secs = [&](Clock::time_point t) { return std::chrono::duration<double>(tnow - t).count(); };
alpha = std::clamp(1 - std::min(secs(lastMove) - gazeShow, secs(lastHeld)) / 0.25, 0.0, 1.0);
}
if (tnow < calPanelUntil && calMode == 1) alpha = 0;
if (tnow < calPanelUntil) alpha = 0;
overlay->SetOverlayAlpha(marker, float(alpha));
overlay->SetOverlayWidthInMeters(marker, float(2 * SETTINGS_DOT * std::tan(cursorDeg * M_PI / 360)));
auto mm = Billboard(near, eye);
@@ -2059,7 +2027,7 @@ int main() {
alpha = std::max(alpha, std::clamp((0.6 - pulse) / 0.3, 0.0, 1.0));
}
}
if (tnow < calPanelUntil && calMode == 1) alpha = 0; // the calibration panel is up (see the top)
if (tnow < calPanelUntil) alpha = 0; // the calibration panel is up (see the top)
overlay->SetOverlayAlpha(show, float(alpha));
overlay->SetOverlayWidthInMeters(show, float(2 * dist * std::tan(scale * cursorDeg * M_PI / 360)));
auto m = Billboard(at, eye);
@@ -2119,16 +2087,15 @@ int main() {
// A held-back press (see the top): held still long enough, it's a real press (a drag);
// released, it's a click where the pointer is now (this frame's pose has gone out).
if (!active) aimHeld = clickPress = aimHand = confirmLesson = keyLesson = false; // released meanwhile: nothing to click
if (aimHeld && !aimHand && nudgeMoved < 0.2 && tnow - aimSince >= std::chrono::duration<double>(gazeHold) &&
!calVerifying()) {
if (!active) aimHeld = clickPress = aimHand = confirmLesson = false; // released meanwhile: nothing to click
if (aimHeld && !aimHand && nudgeMoved < 0.2 && tnow - aimSince >= std::chrono::duration<double>(gazeHold)) {
aimHeld = false;
gazeBack = true;
pressLeft();
}
// A keyboard click held still for POINTER_GAZE_HOLD: a real press, then the head drags
// (from where it is now).
if (aimHeld && aimHand && hold.promote && !hold.engaged && !calVerifying() &&
if (aimHeld && aimHand && hold.promote && !hold.engaged &&
tnow - aimSince >= std::chrono::duration<double>(gazeHold)) {
aimHeld = aimHand = false;
hold.pressed = hold.grip = true;
@@ -2150,7 +2117,7 @@ int main() {
clickReleaseAt = tnow + std::chrono::milliseconds(40);
} else if (clickRelease && tnow >= clickReleaseAt) {
clickRelease = false;
if (leftHeld) releaseLeft(); // not after the quick check's correction: nothing was pressed
releaseLeft();
}
controllerButtons.Poll(
+1 -1
View File
@@ -29,7 +29,7 @@ POINTER_LEASH_RETURN=0.2 # head follow: once it comes along, how quickly (s) t
POINTER_FOLLOW_REACH=70 # head follow: how far (degrees) from the middle of your view the mouse can move the cursor
POINTER_GAZE=0 # 1 = the pointer goes where you look, the mouse does the last bit (needs the gaze service: gaze/run.sh install)
POINTER_GAZE_RETAKE=5 # gaze mode: how far (degrees) you look away from the pointer before the gaze takes it back from the mouse
POINTER_GAZE_NUDGE_MAX=15 # gaze mode: a correction up to this far (degrees) before a click is learned as the eye tracker's error; a bigger one with a keyboard click, or at the quick check, runs the quick check again
POINTER_GAZE_NUDGE_MAX=55 # gaze mode: a correction up to this far (degrees) before a click is learned as the eye tracker's error; a bigger one runs the quick check (55: half the headset's view)
POINTER_GAZE_HOLD=0.5 # gaze mode: a press held this long (s) without moving becomes a real press (to drag); moved or released sooner, it clicks where you let go
POINTER_GAZE_DOT=always # gaze mode: the dot shows all the time (always) | only while the mouse moves it (moving)
POINTER_GAZE_SHOW=1 # gaze mode, with POINTER_GAZE_DOT=moving: the dot shows this long (s) after the mouse moves it; also while a press is held, and a pulse per click