mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-06 03:00:06 +02:00
Gaze: idle while the gaze isn't used
The gaze service ran ft-gaze and our own eye tracker all the time: with gaze mode off, ft-eyes still took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. Now ft-gaze and our tracker run only while gaze mode is on and someone wears the headset, while a check or the calibration is open or asked for, or under a "wake" lease, which the Gaze page of Frametop Input Settings renews while it's open. 30 s after the last use they stop, and the frame grabber idles with our tracker. - The pointer helper answers "gaze ? headset" with worn|away (SteamVR's activity level for the headset); an older helper answers it as before, and the service then goes by gaze mode alone. - A quick check, calibration, or fit check asked for while idle wakes the tracker and opens once it sends; the automatic calibration waits quietly while it starts. - Status has "awake" and "idle" (why), and the Gaze page shows it. - gaze/test/idle-test.py runs the service with a fake helper and ft-gaze, offline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
8aaf7db05a
commit
5b08e43a5d
9 files changed
+307
-21
No files matched your search
@@ -35,6 +35,7 @@ Gaze as an input method for the whole desktop, without replacing anything of Ste
|
||||
- **Learning from nudges:** if the mouse took the pointer from the gaze and moved it (0.2 degrees or more, and the correction within `POINTER_GAZE_NUDGE_MAX`: 55 degrees by default, half of the 109 the headset shows across, and 1 to 110; the same limit for mouse, keyboard, and pinch clicks) before you clicked, or you dragged a held press that far, you were nudging it onto what you looked at. The helper sends that as a lesson, from the raw gaze when the mouse took over to where you clicked, and ft-gazed learns it. So using it is what calibrates it. The raw gaze is one ft-gazed sent, so it also finds when that look was, and what each eye read then. With SteamVR, each eye learns its own error. With our tracker, the look goes to it as a click, like the probe's, and it relearns how the headset sits on your face. After the headset was off, your first nudge and click there resets that (the quick check's dot does the same). A correction past `POINTER_GAZE_NUDGE_MAX` isn't learned: the helper asks ft-gazed for the quick check instead ("recheck", after its 2-minute cooldown). Tested on our tracker's 409 clicks since its Sep 29 calibration: a one-dot check set from any one of them put the next 2 minutes' clicks within 15 degrees (99% within 4.2) and the next 10 minutes' within 25 (the far ones after the headset moved), so the check gets back well under it. The limit used to be 8 degrees, and live on 2026-10-01 our tracker was 12 off after the headset went on, so every correction was dropped. One lesson moves the whole correction by only a third of what it measured (more near where it was taken), since in the first live test one 6 degree lesson moved everything and put the next target 7 degrees off. `ft-gazectl status` shows the lessons, and `ft-gazectl forget` drops them.
|
||||
- **Checks and calibration in the headset** (`gaze/gazecheck.py`, shown by `gaze/panel/ft-gazepanel`, a panel fixed to the headset that ft-gazed runs): a one-dot quick check opens when you put the headset on (SteamVR's tracker sees your eyes for 3 s after none for 3 s; its "HMD on" log line can't say, since it repeats every minute or so and can stay on for hours with nobody in the headset), when our tracker asks for a click (its "reseat", when the headset may sit differently), at most once every 2 minutes, and from Quick check on the Gaze page. Look at the dot: it takes your gaze once it has held still for 0.6 s (the steadiness counts, not where the tracker puts it, so it works however far off it is), or at once with a left click or Meta+J; a right click or Meta+K closes it, and ignoring it changes nothing. It also runs when a click's correction was past `POINTER_GAZE_NUDGE_MAX`. The dot is still and the ring fills in quarters, so the panel is drawn again only a few times per dot. If the first 3 lessons after it are still over 2 degrees off, five dots follow. The full calibration (Calibrate on the Gaze page, or by itself whenever gaze mode is on without one and your eyes are seen) is the probe's: three rounds, dark, medium and bright, of the middle and a ring around it, in a panel 64 degrees wide, with Frametop's screens hidden. Its dots (and the five-dot check's) wait for a click: look at the dot and left click or press Meta+J, and the gaze held still up to then is taken. A dot that isn't taken says why, on an orange line over the instructions: with SteamVR's tracker, what dropped most of that look's samples (an eye lost, a blink, the two eyes disagreeing); with ours, its reply (an eye seen in too few frames, or moving). A dot gets two tries, then it's skipped. A calibration left with under two thirds of its dots fails and names the most common reason, as the Gaze page does after it. A click that has taken nothing after 1.5 s says what it waits for: the gaze to hold still, or an eye tracker that isn't sending. Capturing whenever the gaze held still sometimes took a look that wasn't on the dot. The panel draws into three shared buffers SteamVR imported once, as Frametop's keyboard does: uploading each picture anew (SetOverlayRaw) flickered, and in one live test left the headset showing an old picture. Quitting it while there's still no calibration turns gaze mode off; turning it on again reopens it. One that closes otherwise unfinished (ignored for 2 minutes, too few dots) opens again after the headset comes off and on. Why gaze mode, on, can't work yet goes in the service's status as `checks.problem`, which the Gaze page shows under the Gaze pointer switch. For our tracker a check is a click and the calibration is its own (calib-point per dot); for SteamVR's, a check is a lesson for each eye and the calibration replaces calibration.json, and the lessons start over. Checks go to `checks.jsonl`.
|
||||
- Nothing writes to SteamVR, its eye tracker, or its files: ft-gaze maps the eye tracker's shared memory read-only. With no fresh gaze (a blink, the service stopped, the headset off), the pointer stays where it is, and the mouse works as always.
|
||||
- **Idle while the gaze isn't used:** ft-gaze and our own tracker run only while gaze mode is on and someone wears the headset (the pointer helper says both: SteamVR drops the headset's activity level as soon as it comes off), while a check or the calibration is open or asked for, or while the Gaze page of Frametop Input Settings is open (it renews a `wake` lease). 30 seconds after the last use they stop, and our frame grabber goes idle with our tracker. With our tracker, that saves over half a core: on 2026-10-02, with gaze mode off, ft-eyes took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. A check asked for while it idles starts the tracker and opens once it sends. When the gaze is used again, it takes a few seconds to come back, and our tracker's first click re-seats it, as after the headset was off: so the quick check opens when gaze mode comes on after the service idled, as it does when you put the headset on. `ft-gazectl status` says `"awake"`, and `"idle"` says why it isn't. A stand that covers the proximity sensor makes the headset seem worn, so with gaze mode on it doesn't idle there.
|
||||
|
||||
Lessons are logged to `pointer-lessons.jsonl`: the raw gaze, the true direction, the correction at the time, and how far off it was.
|
||||
|
||||
|
||||
+73
-5
@@ -63,9 +63,18 @@ and the error moves): lessons from before count less then, so the first few afte
|
||||
relearn the offset.
|
||||
|
||||
Our own tracker (gaze/tracker/ft-eyes) runs here too, in the dev container, while
|
||||
GAZE_TRACKER=own or the gaze probe asks for it ("eyes SECONDS", a lease the probe renews). It
|
||||
reads the eye-camera frames the root service frametop-eyegrab copies (gaze/tracker/install.sh),
|
||||
which copies them only while ft-eyes runs.
|
||||
GAZE_TRACKER=own and the gaze is in use (below), or the gaze probe asks for it ("eyes SECONDS",
|
||||
a lease the probe renews). It reads the eye-camera frames the root service frametop-eyegrab
|
||||
copies (gaze/tracker/install.sh), which copies them only while ft-eyes runs.
|
||||
|
||||
Idle: ft-gaze and our own tracker run only while the gaze is in use: gaze mode on with someone
|
||||
wearing the headset (the pointer helper says both: "gaze ? headset" -> "ok on|off worn|away"), a
|
||||
check or calibration open or asked for, or a "wake" lease (the Gaze page of Frametop Input
|
||||
Settings renews one while it's open). IDLE_AFTER after the last use they stop, and the frame
|
||||
grabber goes idle with our tracker; with our tracker, that was over half a core with gaze mode
|
||||
off. A check asked for while idle waits for the tracker to start (at most WAKE_SETTLE). Our
|
||||
tracker's next click after it starts again re-seats it, as after the headset was off, so turning
|
||||
gaze mode on after a while opens the quick check.
|
||||
|
||||
Checks and calibration (gaze/gazecheck.py): a one-dot quick check when the headset goes on or
|
||||
our tracker asks for a click, and the full calibration when gaze mode comes on without one,
|
||||
@@ -83,6 +92,8 @@ Control socket: abstract unix datagram "@ft_gazed":
|
||||
reload read calibration.json and the settings again
|
||||
eyes <seconds> keep our own tracker running that much longer (at most 120),
|
||||
whatever GAZE_TRACKER says: the probe's lease. Reply: "ok"
|
||||
wake <seconds> keep the gaze running (not idle) that much longer (at most 120),
|
||||
whatever gaze mode says: Input Settings' Gaze page. Reply: "ok"
|
||||
quickcal the one-dot check now (the calibration if there's none)
|
||||
calibrate the full calibration in the panel
|
||||
fitcheck the headset fit check in the panel
|
||||
@@ -140,6 +151,9 @@ RETRY = 3.0 # seconds before starting ft-gaze again
|
||||
EYES_RETRY = 10.0 # seconds before starting ft-eyes again after it stopped on its own
|
||||
EYES_LEASE_MAX = 120.0
|
||||
SETTLE = 0.3 # seconds after an eye is found again before the fallback learns from it
|
||||
IDLE_AFTER = 30.0 # seconds after the gaze was last in use before ft-gaze and our tracker stop
|
||||
WAKE_SETTLE = 15.0 # seconds after waking that the tracker may take to start sending
|
||||
WAKE_MAX = 120.0
|
||||
KEYBOARD_PITCH = -20.0 # degrees: gaze under this, on no screen, is a look at the keyboard
|
||||
|
||||
|
||||
@@ -230,6 +244,10 @@ class Service:
|
||||
self.eyes_proc = None # ft-eyes, while it runs
|
||||
self.eyes_until = 0.0 # the probe's lease (monotonic time)
|
||||
self.eyes_restart_at = 0.0
|
||||
self.awake = False # ft-gaze (and our tracker) run: the gaze is in use (see the top)
|
||||
self.idle_at = 0.0 # idle from then, unless it's in use again first
|
||||
self.woke_at = 0.0
|
||||
self.wake_until = 0.0 # a "wake" lease
|
||||
|
||||
self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_CLOEXEC | socket.SOCK_NONBLOCK)
|
||||
self.sock.bind(ME)
|
||||
@@ -399,6 +417,46 @@ class Service:
|
||||
log(f"lesson log: {e}")
|
||||
return rec
|
||||
|
||||
# --- Idle (see the top) ---
|
||||
|
||||
def use_reason(self):
|
||||
"""Why the gaze is in use now, or None."""
|
||||
c = self.checks
|
||||
if c.gaze_on and c.headset is not False:
|
||||
return "gaze mode is on"
|
||||
if c.active or c.pending:
|
||||
return "a check"
|
||||
if time.monotonic() < self.wake_until:
|
||||
return "asked to stay awake"
|
||||
return None
|
||||
|
||||
def idle_reason(self):
|
||||
c = self.checks
|
||||
if c.gaze_on is None:
|
||||
return "the pointer helper isn't answering (SteamVR not running?)"
|
||||
if c.gaze_on and c.headset is False:
|
||||
return "nobody is wearing the headset"
|
||||
return "gaze mode is off"
|
||||
|
||||
def waking(self):
|
||||
"""Idle, or awake too briefly for the tracker to be sending yet."""
|
||||
return not self.awake or time.monotonic() - self.woke_at < WAKE_SETTLE
|
||||
|
||||
def update_awake(self):
|
||||
now = time.monotonic()
|
||||
why = self.use_reason()
|
||||
if why:
|
||||
self.idle_at = now + IDLE_AFTER
|
||||
if why and not self.awake:
|
||||
self.awake, self.woke_at, self.restart_at = True, now, 0.0
|
||||
log(f"awake: {why}")
|
||||
elif not why and self.awake and now >= self.idle_at:
|
||||
self.awake = False
|
||||
log(f"idle: {self.idle_reason()}; ft-gaze and our tracker stop until the gaze is used again")
|
||||
self.stop_helper()
|
||||
if self.eyes_proc and not self.eyes_wanted():
|
||||
self.stop_eyes()
|
||||
|
||||
# --- ft-gaze ---
|
||||
|
||||
def start_helper(self):
|
||||
@@ -441,7 +499,7 @@ class Service:
|
||||
self.proc = None
|
||||
|
||||
def eyes_wanted(self):
|
||||
return (self.tracker == "own" and not self.override) or time.monotonic() < self.eyes_until
|
||||
return (self.tracker == "own" and not self.override and self.awake) or time.monotonic() < self.eyes_until
|
||||
|
||||
def start_eyes(self):
|
||||
"""Our own tracker, in the dev container, with build/venv's numpy and OpenCV. Like
|
||||
@@ -700,6 +758,14 @@ class Service:
|
||||
reply = "ok"
|
||||
except ValueError:
|
||||
reply = "error bad seconds"
|
||||
elif words[:1] == ["wake"] and len(words) == 2:
|
||||
try:
|
||||
secs = min(max(float(words[1]), 0.0), WAKE_MAX)
|
||||
self.wake_until = max(self.wake_until, time.monotonic() + secs)
|
||||
self.update_awake()
|
||||
reply = "ok"
|
||||
except ValueError:
|
||||
reply = "error bad seconds"
|
||||
elif words[:1] == ["reload"]:
|
||||
self.load_settings()
|
||||
self.load_calibration()
|
||||
@@ -750,6 +816,7 @@ class Service:
|
||||
st.update(calibration_samples=cal.get("dots", 0), calibration_made=cal.get("made"),
|
||||
lessons=max(len(m) for m in w.misses), own_running=bool(own),
|
||||
own_reseat=any(e.get("reseat") for e in own.get("eyes", {}).values()))
|
||||
st.update(awake=self.awake, idle=None if self.awake else self.idle_reason())
|
||||
st.update(eyes_process=self.eyes_proc is not None, eyegrab=EYES_CAMS.exists(),
|
||||
tracker_setting=self.tracker_setting, own_installed=own_installed())
|
||||
st.update({"ft_gaze": self.proc is not None, "sample_age_s": round(time.monotonic() - self.last_sample, 2)
|
||||
@@ -770,6 +837,7 @@ class Service:
|
||||
if mtime(CONF) != self.conf_mtime or (self.tracker_setting == "auto"
|
||||
and own_installed() != (self.tracker == "own")):
|
||||
self.load_settings()
|
||||
self.update_awake()
|
||||
want = self.eyes_wanted()
|
||||
if want and not self.eyes_proc and time.monotonic() >= self.eyes_restart_at:
|
||||
self.start_eyes()
|
||||
@@ -790,7 +858,7 @@ class Service:
|
||||
next_verbose = time.monotonic() + 5
|
||||
while self.running:
|
||||
now = time.monotonic()
|
||||
if not self.proc and now >= self.restart_at:
|
||||
if self.awake and not self.proc and now >= self.restart_at:
|
||||
self.start_helper()
|
||||
for key, _ in self.sel.select(timeout=0.05 if self.checks.active else 0.5):
|
||||
if key.data == "control":
|
||||
|
||||
+49
-11
@@ -10,7 +10,9 @@ directions: look at each one.
|
||||
once every QUICK_COOLDOWN, and on "quickcal" (Frametop Input Settings, or a mouse
|
||||
button or key combination mapped to Gaze quick check), and when a click's correction
|
||||
was past POINTER_GAZE_NUDGE_MAX (55 degrees; the helper's "recheck"): the tracker is
|
||||
far off. Ignored, it closes after QUICK_TIMEOUT and changes nothing.
|
||||
far off. Ignored, it closes after QUICK_TIMEOUT and changes nothing. The headset going
|
||||
on is seen only while the gaze service is awake (gaze mode on, someone wearing it: see
|
||||
ft-gazed), so it also opens when gaze mode comes on after the service idled.
|
||||
five the middle and four around it, when the first FIVE_COUNT lessons after a quick check
|
||||
were all over FIVE_LIMIT degrees off: the quick check didn't fix it.
|
||||
full the calibration, as the gaze probe's: three rounds, dark, medium and bright (pupil
|
||||
@@ -30,6 +32,9 @@ directions: look at each one.
|
||||
adjust the headset. A left click or Meta+J runs its guided check (dots, then looks
|
||||
down, up, left and right); a right click or Meta+K closes it, as does FIT_TIMEOUT.
|
||||
|
||||
A check asked for while the gaze service idles (quickcal, calibrate, fitcheck) wakes it and
|
||||
waits until the tracker sends, at most ft-gazed's WAKE_SETTLE; then it opens, or logs why not.
|
||||
|
||||
The quick check's dot captures itself: from CHECK_SETTLE after it shows (the eyes getting
|
||||
there), once the gaze has held within CHECK_SPREAD for CHECK_WINDOW (the probe's max spread and
|
||||
capture time). It's the gaze holding still that counts, not where the tracker puts it, so it
|
||||
@@ -209,7 +214,9 @@ class Checks:
|
||||
sel.register(self.out, selectors.EVENT_READ, "checks")
|
||||
self.check = None
|
||||
self.gaze_on = None
|
||||
self.headset = None # someone wears it (the helper's "worn"), False "away", None unknown
|
||||
self.gaze_heard = 0.0
|
||||
self.pending = None # (command words, when): asked for while the gaze service idled
|
||||
self.full_armed = True # gaze mode on without a calibration opens the full one (need_full)
|
||||
self.full_blocked = None # why it can't open now
|
||||
self.full_retry_at = 0.0
|
||||
@@ -294,17 +301,22 @@ class Checks:
|
||||
pass
|
||||
|
||||
def on_readable(self):
|
||||
"""Replies on our socket: the helper's "ok on|off" to "gaze ?"; the panel's are dropped."""
|
||||
"""Replies on our socket: the helper's "ok on|off [worn|away]" to "gaze ? headset" (an
|
||||
older helper leaves the headset out); the panel's are dropped."""
|
||||
while True:
|
||||
try:
|
||||
data = self.out.recv(4096).decode("utf-8", "replace")
|
||||
words = self.out.recv(4096).decode("utf-8", "replace").split()
|
||||
except (BlockingIOError, OSError):
|
||||
return
|
||||
if data in ("ok on", "ok off"):
|
||||
on = data == "ok on"
|
||||
was, self.gaze_on, self.gaze_heard = self.gaze_on, on, time.monotonic()
|
||||
if on and was is False:
|
||||
if words[:1] == ["ok"] and len(words) in (2, 3) and words[1] in ("on", "off"):
|
||||
on = words[1] == "on"
|
||||
headset = None if len(words) == 2 else words[2] == "worn"
|
||||
was = (self.gaze_on, self.headset)
|
||||
self.gaze_on, self.headset, self.gaze_heard = on, headset, time.monotonic()
|
||||
if on and was[0] is False:
|
||||
self.on_gaze_on()
|
||||
if was != (on, headset):
|
||||
self.svc.update_awake()
|
||||
|
||||
# --- State ---
|
||||
|
||||
@@ -341,6 +353,8 @@ class Checks:
|
||||
self.full_blocked = None
|
||||
return
|
||||
now = time.monotonic()
|
||||
if not self.can_run() and self.svc.waking():
|
||||
return # the tracker is still starting (the service idled)
|
||||
if not self.can_run():
|
||||
why = ("the eye tracker isn't sending" if now - self.svc.last_sample >= 2
|
||||
else "no eyes seen (is the headset on?)")
|
||||
@@ -490,6 +504,8 @@ class Checks:
|
||||
|
||||
def on_sample(self, s):
|
||||
self.sample_at = time.monotonic()
|
||||
if self.pending:
|
||||
self.run_pending()
|
||||
unc = (s["src"].get("mmap1") or {}).get("unc")
|
||||
if unc and min(unc) <= EYE_LOST:
|
||||
self.seen_at = time.monotonic()
|
||||
@@ -742,9 +758,28 @@ class Checks:
|
||||
|
||||
# --- From the service ---
|
||||
|
||||
def command(self, words):
|
||||
def run_pending(self):
|
||||
"""A check asked for while the service idled: once the tracker sends (and our tracker has
|
||||
said whether it's calibrated), or WAKE_SETTLE after waking, when its error is the real one."""
|
||||
svc = self.svc
|
||||
words, _ = self.pending
|
||||
ready = (time.monotonic() - svc.last_sample < 2 if words[0] == "fitcheck" else self.can_run()) \
|
||||
and (svc.kind != "own" or self.calibrated() is not None)
|
||||
if not ready and svc.waking():
|
||||
return
|
||||
self.pending = None
|
||||
reply = self.command(words, queue=False)
|
||||
if reply != "ok":
|
||||
log(f"{words[0]}, asked for while idle: {reply.removeprefix('error ')}")
|
||||
|
||||
def command(self, words, queue=True):
|
||||
"""quickcal, calibrate, calaccept, calquit -> a reply."""
|
||||
cmd = words[0]
|
||||
if cmd in ("quickcal", "calibrate", "fitcheck") and queue and self.svc.waking() and not self.check:
|
||||
# The tracker isn't running (or only just started): wake it, and do this once it sends.
|
||||
self.pending = (words, time.monotonic())
|
||||
self.svc.update_awake()
|
||||
return "ok waking the eye tracker first"
|
||||
if cmd == "quickcal":
|
||||
return self.start("full" if self.calibrated() is False else "quick", "asked for")
|
||||
if cmd == "calibrate":
|
||||
@@ -820,9 +855,11 @@ class Checks:
|
||||
now = time.monotonic()
|
||||
if not self.panel_proc and now >= self.panel_restart_at:
|
||||
self.start_panel()
|
||||
self.to_helper("gaze ?")
|
||||
self.to_helper("gaze ? headset")
|
||||
if now - self.gaze_heard > 5:
|
||||
self.gaze_on = None # the helper isn't answering
|
||||
self.gaze_on = self.headset = None # the helper isn't answering
|
||||
if self.pending:
|
||||
self.run_pending()
|
||||
if self.check:
|
||||
self.to_helper("calpanel 1")
|
||||
if not self.eyes_seen(AWAY_MIN):
|
||||
@@ -844,7 +881,8 @@ class Checks:
|
||||
|
||||
def status(self):
|
||||
c = self.check
|
||||
st = {"check": None, "gaze_mode": self.gaze_on, "calibrated": self.calibrated(), "eyes": self.eyes_seen(),
|
||||
st = {"check": None, "gaze_mode": self.gaze_on, "headset_worn": self.headset, "pending": self.pending[0][0]
|
||||
if self.pending else None, "calibrated": self.calibrated(), "eyes": self.eyes_seen(),
|
||||
"problem": self.problem(),
|
||||
"panel": self.panel_proc is not None,
|
||||
"last_quick_s": round(time.monotonic() - self.last_quick) if self.last_quick else None}
|
||||
|
||||
Executable
+158
@@ -0,0 +1,158 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Offline test of the gaze service idling (gaze/ft-gazed, gaze/gazecheck.py): ft-gaze runs only
|
||||
while the gaze is in use, and a check asked for while it idles waits for the tracker.
|
||||
|
||||
Runs ft-gazed's Service with its sockets renamed, a fake pointer helper (answers "gaze ?
|
||||
headset" as the test says), and a fake ft-gaze (prints samples, quits when its stdin closes).
|
||||
SteamVR's tracker is the one in use, so our own isn't started; the panel isn't either. Nothing
|
||||
reaches the live gaze service, the pointer helper, or SteamVR, so it's safe next to them.
|
||||
|
||||
gaze/test/idle-test.py
|
||||
"""
|
||||
import importlib.machinery
|
||||
import importlib.util
|
||||
import os
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
|
||||
HERE = os.path.dirname(os.path.abspath(__file__))
|
||||
GAZE = os.path.join(HERE, "..")
|
||||
sys.path.insert(0, GAZE)
|
||||
loader = importlib.machinery.SourceFileLoader("ftgazed", os.path.join(GAZE, "ft-gazed"))
|
||||
gazed = importlib.util.module_from_spec(importlib.util.spec_from_loader("ftgazed", loader))
|
||||
loader.exec_module(gazed)
|
||||
import gazecheck # noqa: E402 (the module ft-gazed imported)
|
||||
|
||||
tag = f"ft_gaze_idle_test_{os.getpid()}"
|
||||
gazed.ME = f"\0{tag}_gazed"
|
||||
gazed.POINTER = gazecheck.POINTER = f"\0{tag}_helper"
|
||||
gazecheck.SCREENS = f"\0{tag}_screens"
|
||||
gazecheck.PANEL_PROG = gazed.REPO / "nonexistent-panel" # "isn't built": no panel
|
||||
gazed.IDLE_AFTER, gazed.WAKE_SETTLE = 1.0, 3.0
|
||||
gazed.read_settings = lambda: ("steam", "steam", "auto", 55.0)
|
||||
logs = []
|
||||
gazed.log = gazecheck.log = lambda msg: logs.append(msg)
|
||||
|
||||
# The fake ft-gaze: 90 samples a second, both eyes seen, until its stdin closes.
|
||||
tmp = tempfile.mkdtemp(prefix="ft-gaze-idle-test-")
|
||||
FAKE = os.path.join(tmp, "ft-gaze")
|
||||
with open(FAKE, "w") as f:
|
||||
f.write('''import json, select, sys, time
|
||||
while True:
|
||||
if select.select([sys.stdin], [], [], 1 / 90)[0] and not sys.stdin.read(1):
|
||||
break
|
||||
eye = {"hy": 1.0, "hp": 2.0}
|
||||
print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
|
||||
"open": [0.8, 0.8]}, "left": eye, "right": eye}}), flush=True)
|
||||
''')
|
||||
started = []
|
||||
|
||||
|
||||
def start_helper(self):
|
||||
"""ft-gaze, straight from here instead of the dev container."""
|
||||
import selectors
|
||||
self.proc = subprocess.Popen([sys.executable, FAKE], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE)
|
||||
os.set_blocking(self.proc.stdout.fileno(), False)
|
||||
os.set_blocking(self.proc.stderr.fileno(), False)
|
||||
self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
|
||||
self.sel.register(self.proc.stderr, selectors.EVENT_READ, "stderr")
|
||||
self.buf = b""
|
||||
started.append(time.monotonic())
|
||||
|
||||
|
||||
gazed.Service.start_helper = start_helper
|
||||
|
||||
# The fake pointer helper.
|
||||
helper_state = {"reply": "ok off worn"}
|
||||
helper = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
|
||||
helper.bind(gazed.POINTER)
|
||||
helper.settimeout(0.2)
|
||||
|
||||
|
||||
def answer():
|
||||
while True:
|
||||
try:
|
||||
data, addr = helper.recvfrom(512)
|
||||
except socket.timeout:
|
||||
continue
|
||||
except OSError:
|
||||
return
|
||||
if data.startswith(b"gaze ?") and addr:
|
||||
helper.sendto(helper_state["reply"].encode(), addr)
|
||||
|
||||
|
||||
threading.Thread(target=answer, daemon=True).start()
|
||||
svc = gazed.Service(None, False, gazed.POINTER)
|
||||
threading.Thread(target=svc.run, daemon=True).start()
|
||||
ctl = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
|
||||
ctl.bind("")
|
||||
ctl.settimeout(2)
|
||||
|
||||
|
||||
def ask(cmd):
|
||||
ctl.sendto(cmd.encode(), gazed.ME)
|
||||
return ctl.recv(4096).decode()
|
||||
|
||||
|
||||
failures = []
|
||||
|
||||
|
||||
def check(label, got, want):
|
||||
ok = got == want
|
||||
print(("ok " if ok else "FAIL ") + label + ("" if ok else f": got {got!r}, want {want!r}"), flush=True)
|
||||
if not ok:
|
||||
failures.append(label)
|
||||
|
||||
|
||||
def wait(cond, seconds):
|
||||
end = time.monotonic() + seconds
|
||||
while time.monotonic() < end:
|
||||
if cond():
|
||||
return True
|
||||
time.sleep(0.05)
|
||||
return cond()
|
||||
|
||||
|
||||
time.sleep(2.5)
|
||||
check("gaze mode off: idle, no ft-gaze", (svc.awake, svc.proc is None), (False, True))
|
||||
check("status says why", ask("status").count('"idle": "gaze mode is off"'), 1)
|
||||
|
||||
helper_state["reply"] = "ok on worn"
|
||||
check("gaze mode on, headset worn: awake within 2 s", wait(lambda: svc.awake and svc.proc is not None, 2.5), True)
|
||||
check("samples come in", wait(lambda: svc.counts["samples"] > 20, 2), True)
|
||||
|
||||
helper_state["reply"] = "ok on away"
|
||||
check("headset off: still awake for IDLE_AFTER", wait(lambda: not svc.awake, 0.5), False)
|
||||
check("then idle, ft-gaze stopped", wait(lambda: not svc.awake and svc.proc is None, 3.5), True)
|
||||
check("why: nobody wears it", ask("status").count('"idle": "nobody is wearing the headset"'), 1)
|
||||
|
||||
helper_state["reply"] = "ok on"
|
||||
check("an older helper (no headset word): awake with gaze mode on", wait(lambda: svc.awake, 2.5), True)
|
||||
helper_state["reply"] = "ok off worn"
|
||||
check("gaze mode off again: idle", wait(lambda: not svc.awake and svc.proc is None, 4.5), True)
|
||||
|
||||
check("wake lease", ask("wake 2"), "ok")
|
||||
check("wake: awake at once", (svc.awake, wait(lambda: svc.proc is not None, 1)), (True, True))
|
||||
check("lease over (2 s + IDLE_AFTER): idle", wait(lambda: not svc.awake, 4.5), True)
|
||||
|
||||
time.sleep(0.5)
|
||||
logs.clear()
|
||||
count = len(started)
|
||||
check("quick check while idle: queued, waking", ask("quickcal"), "ok waking the eye tracker first")
|
||||
check("it woke", wait(lambda: svc.awake and len(started) > count, 1.5), True)
|
||||
check("once the tracker sends, it ran (and says why it couldn't open)",
|
||||
wait(lambda: any("quickcal, asked for while idle: the panel isn't running" in m for m in logs), 3), True)
|
||||
check("nothing left pending", svc.checks.pending, None)
|
||||
check("idle again after", wait(lambda: not svc.awake, 3), True)
|
||||
|
||||
print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
|
||||
svc.running = False
|
||||
time.sleep(0.7)
|
||||
os.remove(FAKE)
|
||||
os.rmdir(tmp)
|
||||
os._exit(1 if failures else 0)
|
||||
Reference in new issue
Block a user