When the checklist says no controllers, exported poses.jsonl has left and right
null and feedback lines carry no controller state: controllers left switched on
still get tracked (one wandered 2 m in a real session) and would read as the
hands' ground truth. Poses and live-tracker feedback inside deleted ranges are
left out too, as the images there are.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sets.bin's capture_ns is CLOCK_MONOTONIC_RAW; poses.jsonl and prompts.jsonl are
CLOCK_MONOTONIC. On 2026-10-03 the two were 0.80 s apart during a session and
1.11 s apart five hours later, so images can't be paired with poses by
capture_ns. take.json now samples RAW minus MONOTONIC as each recording part
starts and stops (as ft-hands' raw_minus_mono_ns), so readers can put each
exposure on the poses' clock.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Upload page is now three numbered steps: choose the export, log in to
Hugging Face, upload. Log in runs hub.py login, huggingface_hub's browser
login (OAuth device code, as hf auth login does): the link opens in the
browser and the page shows the code to enter, with Copy code and Cancel.
hub.py saves the token; the window never sees one, and nobody pastes one.
The terminal upload and the login command are gone from the page, and
UPLOAD.md is now a short 'About uploading' under the steps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frametop's Konsole sets XDG_RUNTIME_DIR=/run/user/UID/frametop, where podman finds
no container state, so 'distrobox enter dev -- hf auth login' failed with a crun
error. The command the Upload page shows now sets the real runtime folder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ft-screens "spin next|prev|<degrees>": every unpinned screen and
floating window turns together about a vertical axis through your
head (0.3 s, eased), so the next panel on the right or left comes to
straight ahead; the arrangement stays as it is. Taps during a spin
add to it, from where the panels are headed; grabbing a panel or
placing it (ft-layout, ft-floatd) takes it out of the spin
- when a spin settles, the panel in front gets the pointer (recenter),
typing (as after a click), and KWin's active window: its floating
window, or the top window on a screen (ft-floatd "front N", the KWin
script's activate-output). KWin's outputs follow the screens'
new places (ft-layout scale), as after a move
- the input relay: spin_next and spin_prev actions, Meta+Alt+Tab and
Meta+Alt+Shift+Tab by default; Frametop Input Settings lists them.
Not Meta+Tab: that's Cmd+Tab on a Mac reached through a remote
desktop like RustDesk, and the relay would take the Mac's app
switcher. Meta+Alt+Tab (Cmd+Option+Tab) is unused on macOS,
Windows, and KDE
Used on the Frame (SteamOS 0.3.0 build 20260922) with one screen and
three or four floating windows, through RustDesk to a Mac.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
af2ea7c put _stay_awake between exportSession and its @Slot, so the
window's Export button called a method QML couldn't see. A new test checks
every backend call in main.qml against Backend's slots and properties.
Export and Upload now say Exporting…/Uploading… with a spinner the moment
they're pressed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The helper now reads SteamVR's list of panels every 20 s instead of every second, so a panel
made in between (a floating window's menu, frametop.float.N.sub.K, or the Frametop keyboard
the first time it opens) couldn't be clicked with the mouse until the next read. ft-screens
now sends "overlay <key>" to @ft_pointer_helper right after it makes one, and the helper adds
it to its list at once (only frametop.* keys). An older helper ignores it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The calibration panel runs for as long as the gaze service does, hidden nearly all the time,
and it woke 20 to 30 times a second to look at its socket and SteamVR's events: about 0.9% of
a core, the main cost left with gaze idle.
It now waits in poll() on its command socket: up to a second while hidden, and up to 10 ms
while shown, as before (it still drains SteamVR's events each pass, so a quit is acknowledged
within a second while hidden). A command wakes it at once, so "show" draws sooner than
before. With --watch-stdin, its stdin closing wakes it as well, so stopping it doesn't wait.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-gaze's loop slept 2 ms, so 500 times a second it read the head pose
(GetDeviceToAbsoluteTrackingPose), checked the eye tracker's counter, and drained SteamVR's
events, for samples that come 90 times a second.
It now sleeps 4 ms. A new sample is printed within 4 ms of appearing, 2 on average (was 1),
and the pose history still has a pose within 2 ms of any sample's time, which keeps the
head-pose error under 0.2 degrees for a head turning 100 degrees a second. Sleeping until
the next sample is due would have thinned the pose history to 11 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The panel-edge test read visible[edgeKey], and when the last panel the cursor touched was
gone from the overlay list, that added it back as hidden. The map then had more entries than
there are handles, which made the 50 ms visibility poll run every frame, and since the last
commit it also counted as a visibility change each time, so unchanged frames were never
reused. The edge test now looks the key up without adding it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-gaze computed and printed all six sources for every sample: about 1.3 KB of JSON a line
with our tracker (practice2), 120 KB a second through podman's stdio relay for ft-gazed to
json.loads 90 times a second. It also read SteamVR's gaze action for every sample outside
games (UpdateActionState and GetEyeTrackingDataRelativeToNow, two calls into vrserver, 180 a
second), though with our tracker ft-gazed only uses own and mmap1.
- ft-gaze takes --sources LIST (action, mmap1, mmap2, left, right, own, and eye for the EYE
object; all by default, so the probe and ft-eyes-session are unchanged), and with
--watch-stdin a line "sources LIST" on stdin switches them. A source left out isn't read
and prints as {"ok":0} ("eye" as null), so every line keeps the same keys. An older
ft-gaze ignores both, and prints everything as before.
- ft-gazed asks for what it reads: own,mmap1 with our tracker; left,right,mmap1 with
SteamVR's eyes; the source plus mmap1 and mmap2 on the older one-source path. While a
check or the calibration runs or waits to open, all of them, since checks record every
source (the calibration fits the action's correction too) and the fit check reads "eye".
It switches as soon as that changes, well inside the check's 0.45 s settle.
- So the action is read only during checks, or with --source action.
On recorded samples, a line with own and mmap1 is about 700 bytes instead of 1,300
(practice2), and one with left, right and mmap1 about 550 instead of 940 (test1).
gaze/test/idle-test.py now checks that ft-gaze starts with every source for a check and is
then switched to those in use, without the action or own; all its checks pass.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The relay sent the helper one "move" per SYN_REPORT, so a 1000 Hz mouse sent 1000 datagrams
a second to a helper whose loop runs every 8 ms, and each one went through a dozen sscanf
and strncmp tests in the helper before reaching the move handler. In a 200 ms test at
1000 Hz, 149 reports now make 45 moves with the same total.
flush() on a report now sends only once 4 ms have passed since the last move; tick() sends
the rest when due, and the select timeout shrinks to match. Buttons and the gaze
keys still flush first, unconditionally, so a click lands where the pointer was. In the
helper, "move" is now tested first in the command dispatch, and its handling is one lambda.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gesture reader fetched vrserver's /input/getstate.json over HTTP every 3 seconds, the
whole time the relay runs, to notice a controller's root path changing when the 3D mouse
takes or gives back its hand role.
It now looks them up when it connects, when a message comes from a device path it doesn't
know (at most every 3 s; the device is read from the message with two string searches, not
a JSON parse of all 160 a second), 1.5 s after the relay's 3D mouse connects or lets go (the
relay tells it through GamePause.controllers_changed), and otherwise every 30 s. The keys
test's pause stub gets the new method.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The session now turns KWin's blur and contrast effects off by id (blurEnabled and
contrastEnabled in the desktop's kwinrc), and a KWin that renamed them would quietly
leave them on. The check looks for their built-in factories (KWin::blur_factory,
KWin::contrast_factory) in kwin_wayland, which it already reads for --output-count,
and warns if one is gone. The kwin and plasma-workspace retest hints gain the blur and
the hidden autostart entries.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While its window was open, Frametop Remote Access ran remote-ctl.sh status every 2 s,
and each run spawns bash, curl (the tailnet name from tailscaled) and python3 to parse
it. It now checks every 5 s, plus when the window comes to the front and once more 2 s
after turning remote access on or off or changing the password, so a change still
shows within a couple of seconds. A check doesn't start while one is still running.
Doing the check in-process would duplicate remote-ctl.sh's idea of "running", which
the session and the pause code share.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Export and upload are done in the headset now: the export page only notes
that VR may stutter a little. Upload opens the pull request first (a
draft) and shows its link, telling the person to plug in the headset and
leave it until it says Uploaded; the files then go to refs/pr/N, and the
pull request is marked open at the end. A retry of the same export goes on
in the same pull request. While an export or upload runs, a host unit holds
a logind sleep inhibitor so the Frame stays awake with the headset off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While copying, ft-eyegrab woke every 300 us (about 1,500 to 3,000 times a second) and
fingerprinted all eight slots each time: 8 x 256 strided reads from DMA-BUF memory.
- Each look now checks only the slot each camera writes next. The order is known (camera 0
3,0,1,2; camera 1 7,5,4,6,5,7,6,4), and the next slot follows from the last two; the table
starts from those orders and learns from every frame, so a SteamVR update that changes
them costs a few seconds of full scans, not frames.
- A camera with nothing in its expected slot 1.5 frames after its last one, or with no
order yet, gets all four slots checked, as before. A frame that turns up in an unexpected
slot means full scans for that camera for 2 s.
- A slot's fingerprint is taken again when it stops being one of the two in use, so a later
check sees only a new frame. A frame is still passed on when its camera starts the frame
after next.
- Between frames it sleeps until 2.5 ms before the next is due, then looks every 1 ms, with
0.5 ms of timer slack (PR_SET_TIMERSLACK, --share only). With no frames from either
camera for 0.5 s (headset off) it looks every 4 ms.
- --rec keeps its 0.3 ms polls (and the expected-slot checks), for its timestamps.
Tested offline by building poll_frames against simulated cameras that write each frame in
four bursts, the last after the next frame starts (6 s, both cameras): 1,076 frames passed
on, none torn or skipped, with the known orders and with camera 1 in a different order.
Wakeups 1,486/s -> 207/s, the poller's CPU 3.6% -> 0.8% of a core (in plain memory; the real
DMA-BUF reads cost more), and a frame's start is seen 1.35 ms after it begins on average
instead of 0.76. With a camera stalling 15 ms every 2 s, the old poller passed on 22 torn
frames and the new one 8 or fewer.
Built (glibc 2.38 symbols at most, the host has 2.39), not installed: it runs as root from
/etc/frametop, so it takes effect only after gaze/tracker/install.sh (sudo).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The loop polled the input devices with a 100 ms timeout and then, on every wake, did
SteamVR's part: PollNextEvent, the headset's activity level and every device's pose, all
IPC calls to vrserver. Input wakes it at once so the displays come on with the first
key or motion, but a moving mouse sends hundreds of events a second, so moving the mouse
meant hundreds of rounds of IPC a second instead of 10.
Every wake still drains the input devices and the control socket and counts input as
use straight away; SteamVR's part, and the backlight read that goes with it, now run
only when 100 ms have passed since the last time, and poll sleeps until then. Built in
the dev container (power/build.sh, no warnings); not run, since the live ft-powerd holds
@ft_powerd.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Handle() held the state lock through a chain of up to a dozen sscanf calls per command, and
RunFrame, which vrserver calls every frame, takes the same lock, so a burst of commands
(about 116 poses a second, plus moves and buttons) could hold up vrserver's frame. Commands
are now parsed into locals first, and the lock is held only to store the result.
RunFrame also called UpdateBooleanComponent six times and UpdateScalarComponent twice every
frame, and TrackedDevicePoseUpdated every frame even while disconnected. Components now go to
SteamVR only when they change (all of them on the first frame). The pose still goes out every
frame while the device is connected, as a tracked device's should; the disconnected pose goes
out once. The helper now sends a pose only when it changes, so the comment says the driver
keeps the last one.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The relay sent to @ft_pointer_helper on a blocking socket. When the helper stalled, a
layout placement or grabprobe holds it for seconds while ft-gazed keeps filling its socket at
90 Hz, the relay's one loop blocked with it: keyboards, the volume keys (which must never
reach gamescope), and pausing all stopped until the helper read again.
The socket is non-blocking now. A command the helper doesn't take (EAGAIN) waits in a queue,
and everything after it queues behind it so the order holds; tick() sends what it can on each
loop, and the select timeout drops to 20 ms while anything waits. Mouse moves add up into one
queued move. A scroll notch is dropped rather than queued, since scrolling seconds late is no
use; its release still goes. Presses, releases, show, hide, and the rest are kept, so no
button stays down. The queue holds at most 512 commands. While paused, the configured
pointer's queue still drains, so the releases and "hide" from standing down arrive. A
"vrbind" that hits a full socket is sent again on the next loop instead of being lost.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The nested Plasma session runs the system's XDG autostart entries. Discover's update
notifier (/etc/xdg/autostart/org.kde.discover.notifier.desktop) started
plasma-discover --mode update inside it, 520-620 MB resident and about 9% of a core,
with flatpak-system-helper and AppStream downloads behind it. IBus started a nested
ibus-daemon with kimpanel and ibus-extension-gtk3, which no app in the desktop can use:
KWin's input method is ft-textinput (zwp_input_method_v1, focus reports only; the VR
keyboard types through ft-screens' seat), and the session already drops QT_IM_MODULE,
GTK_IM_MODULE and XMODIFIERS. Nothing in Frametop talks to IBus.
Before Plasma starts, the session script copies both entries into
$XDG_CONFIG_HOME/autostart with Hidden=true, which plasma-session honours for that
desktop only. It does this once ([Defaults] autostart=1 in frametoprc) and skips a name
the user already has a file for, so deleting the copy brings the program back. The
geoclue demo agent stays (it answers apps' location requests outside GNOME and idles at
0%), and orca's entry is OnlyShowIn GNOME-family desktops, so it never ran. Tested
against a temporary XDG_CONFIG_HOME, including an existing user ibus.desktop left alone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The nested kwinrc had no [Plugins] group, so KWin ran its default blur and background
contrast effects, and kdeglobals had no AnimationDurationFactor, so animations ran at
full length. KWin renders through zink on Turnip, on the GPU vrcompositor needs, and
blur re-renders what's behind every translucent panel and menu; each animation frame is
another frame for KWin and ft-screens.
Before KWin starts, the session script now writes [Plugins] blurEnabled=false and
contrastEnabled=false to $XDG_CONFIG_HOME/kwinrc and [KDE] AnimationDurationFactor=0 to
its kdeglobals, each only if the desktop's own file has no value for it. It does this
once and records that in $XDG_CONFIG_HOME/frametoprc ([Defaults] effects=1), because
System Settings deletes a key put back to its default: without the marker, turning blur
back on wouldn't survive a restart. The ids blur and contrast are the built-in effects
of KWin 6.2.5 on SteamOS (both enabled by default in its plugin metadata). Tested
against a temporary XDG_CONFIG_HOME: fresh config, an existing blurEnabled=true kept,
and a deleted key not rewritten.
README and docs/reference.md say how to turn them back on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every frame (about 116 a second) the helper tested the cursor ray against every visible
overlay twice with ComputeOverlayIntersection, set the dot's alpha, width, transform and
visibility (five calls into SteamVR), and sent the driver a pose datagram, even with the
mouse and the head still.
Now a frame reuses the last collision result when the mouse, the anchor (1 mm) and the
eye (5 mm) haven't moved and no overlay showed, hid, or changed handle. The passes still run
at least every 100 ms, since overlays move on their own (a floating window's controls follow
it), and always while dragging. The dots' setters go to SteamVR only when their value changes:
the placement when the dot moved 0.2 mm or the eye 5 mm, which turns or resizes it by well
under 1%, and the width on a 0.5% change. The plain pose goes to the driver only when the
laser's origin moved 0.2 mm or its direction 0.04 deg (0.1 mm where it lands, 15 cm on), and
at least every 100 ms; the driver keeps the last pose and reports it every frame. A tilt's
pose, a placement, or waking sends the next one regardless.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-screens gave KWin a frame callback for every committed screen on each tick, and the tick
was an 11 ms timer set again after each run, so it slid through the display's frame and came
about 85 times a second at 90 Hz: the desktop repeated a frame several times a second (judder
in scrolling and video), and KWin drew every screen in one burst at a random point of
vrcompositor's frame. Hidden screens got the same 90 Hz unless Frametop was paused for a game.
- Ticks run on a timerfd at absolute times, once per display frame, 1 ms after the vsync
(IVRSystem::GetTimeSinceLastVsync and the HMD's display frequency, read once a second), so
KWin gets its callbacks early in the frame. Measured with --no-vr: 91 wakeups a second
instead of about 85. On the Frame the vsync times SteamVR reports lie on a 90 Hz grid.
- Each screen's callbacks come at a rate for how much of it you see (vr.cpp,
UpdateAttention): every frame while focused (within 12 degrees of where your head points,
a laser or the mouse on it in the last 1.5 s, carried, or typed on), 15 a second for the
rest of what you see (within 60 degrees), and 1 a second when hidden, behind you, or
paused. Levels rise at once and fall after 1.5 s (focused) or 0.5 s (in view). KWin draws
a screen only after its callback and its apps wait for theirs, so this throttles the apps
too. A screen where nothing changes costs nothing at any rate, as before.
- A video in view keeps every frame: 8 commits in a row that each redraw 6% or more of the
screen, at 10 a second or more, count as one (from the surface's buffer damage).
- "rates F V H" / --rates set the three rates (default 0 15 1, 0 = every frame), "rates?"
shows them and each screen's level, "watch S" gives everything full rate for S seconds for
a remote viewer (vnc-bridge.sh renews it), and "phase MS" moves the ticks for tuning.
- ft-screens' main thread runs at nice -5 after the session starts: SteamOS allows down to
-8 once the soft RLIMIT_NICE is raised, and KWin waits on these ticks. It had spent nearly
3 times as long waiting to run as running.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-eyes looked for new frames about 1,000 times a second: each pass of its loop asked the
control socket with a non-blocking recvfrom (a BlockingIOError nearly every time), read
both cameras' counters, and slept 1 ms. The frames come every 11.1 ms per camera, and only
as counters in ft-eyegrab's shared memory, so there's no fd to wait on.
Now each pass ends in select() on the control socket, with a timeout until 2 ms before the
next frame of either camera is due (from when its last one was seen), then every 1 ms until
it comes. A command wakes it at once. A camera with no frame for 0.1 s (headset off,
grabber idle) isn't waited for, and with both stopped it looks every 20 ms. Waiting for the
frame grabber's file uses the same select, 0.2 s at a time, instead of sleeping through
commands.
A new frame is still seen within about 1 ms of when it lands. On a synthetic share at 90 Hz
per camera, on a heavily loaded headset (load average 23, so ft-eyes rarely sat idle), its
waits went from 178 to 81 a second; unloaded, the old loop's 1 ms sleeps add up to about
1,000.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While FreeRDP ran, vnc-bridge.sh called ft-layout remote-view every 5 s, which scans
all of /proc for plasmashell and runs kscreen-doctor -j: about 4.4% of a core for a
layout that rarely changes.
It now stats the two files the answer depends on, the nested KWin's
~/.config/frametop/kwinoutputconfig.json (positions, scales, primary) and
~/.config/frametop-layout.json (screen sizes), once a second while FreeRDP runs. After
either changes it reads the layout every 2 s for 10 s, since KWin's outputs follow the
file a few seconds later; otherwise once a minute, in case a change touched neither.
With no VNC viewer connected nothing runs (previous commit).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vnc-bridge.sh kept FreeRDP connected to krdpserver from the moment remote desktop
started, so krdp captured and H.264-encoded every KWin redraw in software (openh264)
with nobody watching: krdpserver 55-78% of a core, xfreerdp 16-27%, Xvnc 6-11%, with 0
clients on :5900. krdp 6.7 creates its screencast session per RDP connection and drops
it when the connection closes, so krdpserver itself idles without one and stays up.
The bridge now counts established connections to Xvnc's port with ss, starts FreeRDP
when a viewer appears (the desktop shows about 3 s later; the VNC screen is black until
then) and stops it 45 s after the last one leaves (VNC_IDLE_SEC). Xvnc has no client
hook, so its log output, which it writes for every connection, wakes the bridge early;
otherwise it looks every 5 s while idle (0.1% of a core measured, against 0.9% for ss
once a second) and every second while FreeRDP runs. The layout check runs only while
FreeRDP runs.
While a viewer is connected the bridge sends "watch 15" to ft-screens (@ft_screens) at
once and every 5 s, so screens at a reduced frame rate (out of view, headset on a
stand) stream at full rate; it lapses by itself if the bridge dies, and an ft-screens
without the command just answers an error. The window search after starting FreeRDP
now ends when FreeRDP exits instead of polling for 30 s.
krdp on 127.0.0.1 with a fresh password, VNC on the tailnet address with VncAuth, and
remote-ctl.sh start/stop (pause and resume) are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The main loop slept a fixed 8 ms, about 116 wakeups a second, whether the pointer was awake
or not, and every second it looked up every overlay's handle and read a string property from
all 64 device slots to find its own device.
With the pointer off and hand gestures off, the loop now waits in poll() on its command
socket for up to 250 ms, or 20 ms while mapped Frame controller buttons are being read
(SteamVR input has no event to wait for). A mouse command ends the wait at once. The
headset's activity level, the game check, and the "vrgame" and "gazeawake" repeats keep
going at that pace. The 50 ms visibility poll and the 1 s handle lookups run only while the
pointer is awake, and waking forces both. The device index is looked for only while it's
unknown, and again after SteamVR activates or deactivates a device. The HMD pose history is
kept only with hand gestures on, its one user.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-eyes and ft-gaze run in the dev container through distrobox, so they live in podman's
libpod scope: frametop-gaze.service's limits never reach them, and they ran at nice 0
next to vrcompositor and vrserver, also at nice 0.
- ft-eyes sets itself to nice 10 and SCHED_BATCH at start, before its threads. Batch
turns off wakeup preemption, so a frame ft-eyes wakes up for can wait out a running
compositor's turn; a few ms late costs the gaze little.
- ft-gaze sets nice 5 before its threads start, but stays SCHED_OTHER: each sample goes
on to the pointer, and batch would add the same wait to every one of them.
- Both only ever lower their priority (a higher nice already set wins), and a failure
is logged and ignored. Checked in the dev container: nice 0 -> 10, policy 0 -> 3
(SCHED_BATCH) without any capability.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The helper ran `vrcmd --overlays` once a second while the pointer was awake, and in gaze
mode the pointer never sleeps. Each run is a shell plus vrcmd, a new SteamVR client, about
26 to 30 ms of CPU, so about 3% of a core all the time.
The list is now read every 20 seconds, and at once (at most once a second) when it may have
changed: the pointer waking, the dashboard opening or closing or creating an overlay, the
scene app changing, an "overlays" request, and a left click that hit nothing, which may be
on a panel that came up since. The thread waits on a condition variable instead of waking
every 100 ms, so it sleeps while paused. The main loop looks the keys up again as soon as a
new list is in, rather than at its next 1 s tick. Overlays already on the list still show
and hide within 50 ms, from the IsOverlayVisible poll.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing called cv2.setNumThreads, so OpenCV kept a pool of one worker per core for
pupil windows of 140 to 240 px. Live, its idle workers spun and yielded about 14,000
times a second each, about a quarter of a core, next to SteamVR's compositor. numpy's
OpenBLAS also started 8 threads that never had work.
eyes_pupil.py now sets OpenCV to one thread, and ft-eyes sets OPENBLAS_NUM_THREADS and
OMP_NUM_THREADS to 1 before numpy loads (a value already in the environment wins).
Replaying fit1 into a scratch share (ft-eyes-replay, 14 s measured, capped at one core
with the replay): threads 13 -> 1, involuntary context switches 3,812/s -> 430/s,
system time 6.9% -> 1.6% of a core. Under that cap the frames it kept up with went from
21-36 to 57-69 a second per eye.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dot is the current tracker's distance guess, often wrong; the ring is
where the hands should be.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
host-spawn starts a host command in the caller's folder. Started from /tmp,
the app's folder in the container is /run/host/tmp, which the host doesn't
have, so starting ft-camd (and every other host command) exited 127.
host_command now runs from home (env -C), and the launcher cds there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The checklist page measures the light when it opens, starting ft-camd if
nothing runs it (and stopping it on quit), instead of saying the cameras
aren't running. The round's lighting defaults to what the cameras measure:
daylight or indoor, from the mono cameras' ambient infrared. Lamps give off
little infrared, so dim and normal rooms read alike; picking dim, room or
daylight still overrides it. session.json gets source, measured and
ambient_ir.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vrserver sends a controller's state only when something on it changes, and one lying still
or asleep may not even send its first one. The check subscribed to one controller and failed
after 3 s of silence. It now subscribes to all, and silence after a good handshake is a skip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gaze service ran ft-gaze and our own eye tracker all the time: with gaze mode off, ft-eyes
still took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. Now ft-gaze
and our tracker run only while gaze mode is on and someone wears the headset, while a check or
the calibration is open or asked for, or under a "wake" lease, which the Gaze page of Frametop
Input Settings renews while it's open. 30 s after the last use they stop, and the frame grabber
idles with our tracker.
- The pointer helper answers "gaze ? headset" with worn|away (SteamVR's activity level for the
headset); an older helper answers it as before, and the service then goes by gaze mode alone.
- A quick check, calibration, or fit check asked for while idle wakes the tracker and opens once
it sends; the automatic calibration waits quietly while it starts.
- Status has "awake" and "idle" (why), and the Gaze page shows it.
- gaze/test/idle-test.py runs the service with a fake helper and ft-gaze, offline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-camd tells the side cameras apart by XRService's buffer allocation order,
which some XRService starts reverse; both of 2026-10-02's starts did, so the
cutouts missed the hands. HANDS_SWAP_SIDES=auto (the default) has ft-hands
vote from hands seen in both side cameras: the landmark rays meet in front
of both cameras only under the right naming. While undecided it probes the
exchanged naming with the landmark model. It decides in about 2 s of hands
(right on all 7 recordings replayed), swaps the views in place, and publishes
sides.json. 0 and 1 still force it, with a warning when the hands disagree.
Recordings carry each part's naming and the session's decision; review,
export, validate and ft-handreplay put the names right, and takes.py sides
records a decision by hand.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frametop kept using the headset during games: with gaze mode off, our eye tracker still took
about 60% of a core, remote desktop about 2 cores while on, and KWin kept drawing hidden
screens because ft-screens sent their frame callbacks at 90 Hz. Pausing gives that back, and
resuming brings back only what pausing stopped. It's also a way to keep the gaze service and
our eye tracker off during games, which PR #13 asked for.
Paused (input/game_pause.py, run by the input relay):
- frametop-gaze stops (ft-eyegrab then idles by itself), and hand tracking and remote desktop
stop if they run
- the desktop hides and slows down: ft-screens "pause on" hides every panel and sends KWin a
frame callback once a second; or, with pause_desktop "close", the desktop closes and starts
again on resume
- the relay lets go of the 3D mouse, typing goes to Steam, and mapped buttons and key
combinations do only pause_toggle, steam_menu and commands
Toggled by both thumbsticks clicked together twice (configurable), read passively from
vrserver's web socket (input/vrws.py) so it works in games and takes nothing from them; by the
new pause_toggle action; by input/ft-pause; and, with pause_auto (default on), by a VR game
starting and ending, which the pointer helper now reports ("vrgame 1|0"). Frametop Input
Settings has a Games page for it. update-check.py checks the web socket, and doesn't count a
paused gaze service as failed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since the relay leaves a node it can't read yet for the next scan (12f2e84, PR #12), it
checks os.access first, and the test's fake /dev/input paths don't exist, so the relay never
opened them and every key check failed. The fake os now says they're readable.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The second real session took 16 minutes, half of it 36 still poses. Labels
come from the auto-labeller, so what matters is variety, not clean holds.
A sweep step shows a strip of pose pictures and lights one every 4 s while
the hands move slowly near and far; each cue is a prompt event with
"cue": true. The core session is now 15 steps, about 5 minutes recorded.
The pose groups, the one-hand sweeps' groups and the cue order are shuffled
per session, seeded from its id and saved in session.json. A quick round
(--quick, or the checklist's choice) is about 2 minutes for extra lighting.
Touch the dot has 6 dots, the push sections two heights.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Gio.DesktopAppInfo.new() returns NULL for a desktop file that doesn't
exist, and PyGObject raises TypeError ("constructor returned NULL")
rather than returning None, so launch()'s `if info is None` never ran.
The exception escaped the control socket's GLib callback, GLib dropped
the watch, and ft-floatd stopped answering everything: the float key,
dock, Launch as Standalone, and profiles, until the desktop restarted.
Found on the Frame (2026-10-02): a profile saved with RustDesk's
Flatpak open records its window's app id, com.carriez.flutter_hbb,
which has no desktop file (the Flatpak's is com.rustdesk.RustDesk).
`ft-layout use` on that profile asked ft-floatd to launch it, and
ft-floatd went silent. With this, that launch replies "error no app
com.carriez.flutter_hbb" and the profile's other apps open.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
After the headset wakes, XRService sometimes fails to load the colour module's
VCINT FPGA image; then only the two side cameras run, without the IR light,
and the tracker finds no hands. hands/camcheck.py reads XRService's log, the
video nodes it holds and ft-camd's ring, and says ok, degraded or unknown.
The recorder won't start while degraded (--ignore-cameras overrides it), offers
a confirmed SteamVR restart, and stops the first hand-size step when the
tracker sees no hand at all. ft-camwatch (unit file only, not enabled) follows
the log, notifies, and with CAMWATCH_AUTO_RESTART=1 restarts SteamVR when the
headset isn't worn and nothing else uses VR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
