The nested kwinrc had no [Plugins] group, so KWin ran its default blur and background
contrast effects, and kdeglobals had no AnimationDurationFactor, so animations ran at
full length. KWin renders through zink on Turnip, on the GPU vrcompositor needs, and
blur re-renders what's behind every translucent panel and menu; each animation frame is
another frame for KWin and ft-screens.
Before KWin starts, the session script now writes [Plugins] blurEnabled=false and
contrastEnabled=false to $XDG_CONFIG_HOME/kwinrc and [KDE] AnimationDurationFactor=0 to
its kdeglobals, each only if the desktop's own file has no value for it. It does this
once and records that in $XDG_CONFIG_HOME/frametoprc ([Defaults] effects=1), because
System Settings deletes a key put back to its default: without the marker, turning blur
back on wouldn't survive a restart. The ids blur and contrast are the built-in effects
of KWin 6.2.5 on SteamOS (both enabled by default in its plugin metadata). Tested
against a temporary XDG_CONFIG_HOME: fresh config, an existing blurEnabled=true kept,
and a deleted key not rewritten.
README and docs/reference.md say how to turn them back on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While FreeRDP ran, vnc-bridge.sh called ft-layout remote-view every 5 s, which scans
all of /proc for plasmashell and runs kscreen-doctor -j: about 4.4% of a core for a
layout that rarely changes.
It now stats the two files the answer depends on, the nested KWin's
~/.config/frametop/kwinoutputconfig.json (positions, scales, primary) and
~/.config/frametop-layout.json (screen sizes), once a second while FreeRDP runs. After
either changes it reads the layout every 2 s for 10 s, since KWin's outputs follow the
file a few seconds later; otherwise once a minute, in case a change touched neither.
With no VNC viewer connected nothing runs (previous commit).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vnc-bridge.sh kept FreeRDP connected to krdpserver from the moment remote desktop
started, so krdp captured and H.264-encoded every KWin redraw in software (openh264)
with nobody watching: krdpserver 55-78% of a core, xfreerdp 16-27%, Xvnc 6-11%, with 0
clients on :5900. krdp 6.7 creates its screencast session per RDP connection and drops
it when the connection closes, so krdpserver itself idles without one and stays up.
The bridge now counts established connections to Xvnc's port with ss, starts FreeRDP
when a viewer appears (the desktop shows about 3 s later; the VNC screen is black until
then) and stops it 45 s after the last one leaves (VNC_IDLE_SEC). Xvnc has no client
hook, so its log output, which it writes for every connection, wakes the bridge early;
otherwise it looks every 5 s while idle (0.1% of a core measured, against 0.9% for ss
once a second) and every second while FreeRDP runs. The layout check runs only while
FreeRDP runs.
While a viewer is connected the bridge sends "watch 15" to ft-screens (@ft_screens) at
once and every 5 s, so screens at a reduced frame rate (out of view, headset on a
stand) stream at full rate; it lapses by itself if the bridge dies, and an ft-screens
without the command just answers an error. The window search after starting FreeRDP
now ends when FreeRDP exits instead of polling for 30 s.
krdp on 127.0.0.1 with a fresh password, VNC on the tailnet address with VncAuth, and
remote-ctl.sh start/stop (pause and resume) are unchanged.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vrserver sends a controller's state only when something on it changes, and one lying still
or asleep may not even send its first one. The check subscribed to one controller and failed
after 3 s of silence. It now subscribes to all, and silence after a good handshake is a skip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gaze service ran ft-gaze and our own eye tracker all the time: with gaze mode off, ft-eyes
still took about 60% of a core, and ft-eyegrab, ft-gaze and ft-gazed 3 to 4% each. Now ft-gaze
and our tracker run only while gaze mode is on and someone wears the headset, while a check or
the calibration is open or asked for, or under a "wake" lease, which the Gaze page of Frametop
Input Settings renews while it's open. 30 s after the last use they stop, and the frame grabber
idles with our tracker.
- The pointer helper answers "gaze ? headset" with worn|away (SteamVR's activity level for the
headset); an older helper answers it as before, and the service then goes by gaze mode alone.
- A quick check, calibration, or fit check asked for while idle wakes the tracker and opens once
it sends; the automatic calibration waits quietly while it starts.
- Status has "awake" and "idle" (why), and the Gaze page shows it.
- gaze/test/idle-test.py runs the service with a fake helper and ft-gaze, offline.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frametop kept using the headset during games: with gaze mode off, our eye tracker still took
about 60% of a core, remote desktop about 2 cores while on, and KWin kept drawing hidden
screens because ft-screens sent their frame callbacks at 90 Hz. Pausing gives that back, and
resuming brings back only what pausing stopped. It's also a way to keep the gaze service and
our eye tracker off during games, which PR #13 asked for.
Paused (input/game_pause.py, run by the input relay):
- frametop-gaze stops (ft-eyegrab then idles by itself), and hand tracking and remote desktop
stop if they run
- the desktop hides and slows down: ft-screens "pause on" hides every panel and sends KWin a
frame callback once a second; or, with pause_desktop "close", the desktop closes and starts
again on resume
- the relay lets go of the 3D mouse, typing goes to Steam, and mapped buttons and key
combinations do only pause_toggle, steam_menu and commands
Toggled by both thumbsticks clicked together twice (configurable), read passively from
vrserver's web socket (input/vrws.py) so it works in games and takes nothing from them; by the
new pause_toggle action; by input/ft-pause; and, with pause_auto (default on), by a VR game
starting and ending, which the pointer helper now reports ("vrgame 1|0"). Frametop Input
Settings has a Games page for it. update-check.py checks the web socket, and doesn't count a
paused gaze service as failed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since the relay leaves a node it can't read yet for the next scan (12f2e84, PR #12), it
checks os.access first, and the test's fake /dev/input paths don't exist, so the relay never
opened them and every key check failed. The fake os now says they're readable.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An X11 window in a Flatpak can give KWin an app id with no desktop file:
RustDesk's says com.carriez.flutter_hbb (its GTK application id), and the
Flatpak's desktop file is com.rustdesk.RustDesk. A profile recorded that
id, so it couldn't relaunch the app (PR #16 keeps that from killing
ft-floatd's socket). And the window's pid is the sandbox's own, so a
launched window matched neither by process nor by app id, and didn't
float.
desktop_name() finds the desktop file whose StartupWMClass names the
window's class (or app id) when the app id has none. Capture records
that name, a profile claims open windows by it, and a launch's window
matches by it.
Profiles saved before this keep the old id; save them again.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From curiousjtuber's PR #13: with the gaze service running, SteamVR
restarted its eye tracker every 10 to 13 s of Beat Saber, as if the
headset came off, and each restart took input focus from the game.
The PR stopped every read in a game. Only the action path reaches
SteamVR (UpdateActionState on the gaze set at overlay-global priority,
then GetEyeTrackingDataRelativeToNow); the mmap and our tracker are
read-only files. So only the action is skipped while a scene app runs,
and gaze keeps moving the pointer over the dashboard in a game. The
action source is only used with --source action.
Co-Authored-By: CuriousJ <curious.j.tuber@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From curiousjtuber's PRs: the relay leaves a new input node for the next
scan until udev gives it to the input group, rather than marking it seen
after a failed open; and ft-screens' catcher takes a screen's overlays
from one copy of All() instead of begin() and end() of two temporaries,
which crashed libc++ builds on the first click.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From jlneal's PR: a trigger press on a desktop screen stays put until the
laser moves more than 8 logical pixels, so controller jitter doesn't turn
a click into a drag. On top of it, only a hand controller's press starts
the filter: the 3D mouse's laser reaches the screens the same way, and the
PR as sent turned the mouse's short drags into clicks.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The 3D mouse drives SteamVR's laser through the ft_pointer virtual
controller, so its events reach the screens the same way a controller's
do. The filter held every press, which turned the mouse's short drags
(selecting a character or two, nudging a slider) into clicks. Mark
button events from hand controllers and start the filter only on those.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
container-up.sh starts the dev container in a scope of its own, so
distrobox enter finds it running and skips its wait for distrobox-init.
On a fresh install, init was still setting up passwordless sudo when
dev-container.sh ran sudo dnf install, and sudo asked for a password
with no terminal to read it from. container-up.sh now waits for
container_setup_done itself, and the container's sudo calls use -n,
so a password prompt fails at once with a clear message.
Fixes#9
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A user on a fresh install got "Calibration failed: only 0 of 21 dots" with
no reason. The installer never installed our tracker, so gaze used
SteamVR's, and the only way SteamVR's tracker rejects a dot is losing an
eye for most of the look. The panel just showed a red ring.
- install.sh: step 9/10 installs our tracker (gaze/tracker/install.sh)
after gaze mode, yes by default; it needs sudo, so --yes runs it only
when sudo won't prompt. If it fails, gaze keeps SteamVR's tracker.
Configs that still say GAZE_TRACKER=steam (the old template) are asked
whether to switch.
- GAZE_TRACKER=auto, the new default: ours when it's installed (the
frame grabber, its unit, and ft-eyes' Python), else SteamVR's.
ft-gazed rechecks every second, so installing it switches over. Input
Settings lists Own tracker first as recommended, and says how to
install it when it's missing (checking the host's /etc through
/run/host from the dev container).
- The calibration panel has a note line, orange over the instructions:
why a dot wasn't taken (an eye lost, a blink, the eyes disagreeing for
SteamVR's tracker, from steady_samples' new drop counts; ft-eyes' reply
for ours), what a click is still waiting for after 1.5 s, and a failed
calibration's most common reason, which the Gaze page shows too.
steady_samples keeps the same samples as before (checked on 2037
windows of recordings); a lost eye is named before a blink, since its
openness reads 0.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Turning gaze mode on without a calibration opened Calibrate only on the
off-to-on change, and only if it could open right then. With the headset
off, the eye tracker silent, or the panel not built, or with gaze mode
already on when the gaze service started, nothing opened and nothing said
why: the pointer just stayed a mouse.
- The gaze service now checks every second: gaze mode on, no calibration
for the tracker in use, eyes seen -> the full calibration opens. One
that closes unfinished opens again only after the headset comes off and
on, gaze mode off and on, or Calibrate, so it doesn't loop. A start that
fails retries every 10 s.
- Its status says why gaze mode can't work yet (checks.problem): not
calibrated and opening, open, closed unfinished, or can't open and why.
- Input Settings shows that under the Gaze pointer switch, along with the
gaze service not installed or not running and our tracker missing its
frame grabber.
- ft-gazectl on notes a missing calibration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hands/ft-cutouts on|off|status starts ft-camd and ft-hands as transient
user units with ft-hands' new --no-gestures: hands are published for
ft-screens' cutouts, but no pinch or grip is detected, so nothing clicks
or drags and a closing hand doesn't raise the tracking rate. It needs a
build and ft-camd's capabilities, not hands/run.sh install. Its units
conflict with ft-handsctl's, so each stops the other, and they stop with
SteamVR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Meta+key combination (Meta+J for gaze_left, say) hides Meta's release
from the desktop, and the relay skipped share_key for it too. frame-voice
saw Meta go down on @frametop_keys and never come up, so it held all
dictated text back, waiting for that release. Keys of grabbed keyboards
are now shared as pressed, before key_binding() decides what the desktop
gets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Key combinations (and mouse and controller buttons) get two new actions:
- steam_menu: Open Steam menu / close dashboard (steam/ft-steam menu).
- command:CMD: run CMD with sh -c, as the relay's service, with layout/,
float/ and steam/ on its PATH. Input Settings offers it for key
combinations as Run a command...
Both work without pointer mode.
A modifier on its own is now a key combination too: a tap, pressed and
released with no other key, mouse button, or scroll in between. A bound
tap sends the desktop F24 before the release, so Plasma's launcher stays
shut. The defaults gain a Meta tap for the Steam menu; this replaces
META_DASHBOARD, which only worked in pointer mode.
input/test/keys-test.py runs the relay against fake devices with every
outgoing socket renamed, so it's safe next to the live relay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
steam/ft-steam menu opens the SteamVR dashboard on Steam's menu, or closes
the dashboard if it's up, without pointer mode: it asks Steam's UI over its
debugging port to show its dashboard overlay (ShowVROverlay, what Steam
calls itself) and focus the Steam frame's left menu (MenuStore.OpenMainMenu).
ft-steam check says whether those calls still exist, and update-check.py
runs it, since a Steam client update can rename them.
The CDP client moves from display-settings/steam_settings.py to
steam/steamui.py, so both use it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Typing on a pass-through keyboard tells the helper "typing". With the
helper not running (SteamVR off), that send raised ConnectionRefusedError
and the relay exited, dropping every grab until systemd restarted it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- desktops.sh uninstall also removes Launch as Standalone's app copies
and the title bar decoration, and Display Settings' uninstall the
profiles' launcher entries; its install writes them back from the
saved profiles (new: ft-layout launchers)
- the README says which settings an uninstall leaves
- the install command is now curl -fsSL https://deejanuz.github.io/
frametop/get.sh | bash (GitHub Pages, from main)
- ft-gaze logs "action manifest ...: ok" instead of "error 0"
Found in a clean install test on the Frame (2026-10-01): uninstall,
then get.sh from the headset into a fresh clone; everything installed
and doctor.sh passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- install.sh no longer offers hand tracking (it's heavy on the CPU and
needs more work); hands/ still builds and installs by hand
- the build container drops python3-opencv and python3-numpy, which
only hand tracking's tools used: Fedora's OpenCV pulls in over a GB
- README: a new opening and feature list, the Use section by topic
(screens, mouse, floating windows, profiles, gaze), and new known
limitations
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- get.sh: curl ... | bash asks for stable (main) or experimental,
clones or updates ~/frametop, and runs install.sh; run it again to
update or switch
- install.sh offers gaze mode (step 8, yes by default) and notes what
to do if an SSH connection drops
- Input Settings' Gaze page says when the gaze service isn't installed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- frame_sudo (scripts/_env.sh) replaces three copies of sudo_run. On
the Frame it asks in the terminal even when stdin isn't one, which
install.sh's steps never had: saying yes to the Bluetooth fixes or
hand tracking stopped the install with "no terminal for sudo".
From a PC it asks through ssh -t when .env has no password.
- start_with_steamvr: the pointer, power, and gaze services restart
when SteamVR runs (a re-install runs the new code) and are left to
start with it when it doesn't, instead of failing the install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- floating-windows.md and profiles.md describe what's built, with what
isn't listed as such; the plans, phases, and branch notes are gone
- hands-migration.md is gone: the move is done; its open items are in
hands/README.md's Known issues
- reference.md: Layout & profiles, every action, key combinations,
floating windows, the gaze pointer, and hand tracking as they are
- design.md gets the KWin findings from floating-windows.md
- README, gaze/README, AGENTS, hazards, gaze-controllers, and the
example config catch up with gaze, hands, and the stuck-key fix
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- gaze/tracker/lab/eyes_track.py: nothing used it
- Input Settings: the pointer role backend, whose page was removed
- ft-screens: no log line for every floating-window resize
- hands: ft-hands --help gives the palm-down default (1: off), the
uninstall removes ft-handsctl's link, .frame-job is ignored
- Display Settings: "Save as profile…", not "Save current arrangement"
- two stale comments (ft-pointer's grabprobe, ft-gaze)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With no head pose (the headset off), ft-layout use stopped before
ft-floatd opened the apps. The screens now stay where they are, the
profile's hidden screens still hide, and the apps open.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gaze checks and calibration in a headset panel (quick, five, full, headset
fit; shared-buffer drawing, click-to-capture), keyboard and mouse gaze clicks
(Meta+J/K, the mouse's buttons alike, mouse moves only while a button is held,
double right/Meta+K tilts a drag), one 55 degree learning limit with a quick
check past it, eye presence for "the headset went on", the gaze probe as a
development tool, and the relay releasing keys the desktop has down that no
keyboard holds.
Conflicts with the profiles: the relay keeps known_action/needs_pointer and
the gaze defaults (Meta+J/K and the float key); Input Settings keeps the
profile actions everywhere and the gaze actions in key combinations only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After a gaze calibration, Meta stayed down in KWin: typing opened the overview,
and clicks on the desktop did other things. A key can be left down there when
its device vanishes with it held (release_held only let go of it on the relay's
own devices; the Z3's keyboard dropped out twice right then) or a release goes
astray. The relay now remembers which keys it told ft-screens went down, and
once a second releases any no device holds (EVIOCGKEY), and the key
combinations forget a Meta or modifier no device holds. A relay that starts
releases the modifiers on the desktop, for keys an earlier one left down.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The panel drew each picture with SetOverlayRaw, a new texture upload every
time: the full calibration's 1024x768 picture flickered on every change, and
in a live test the headset kept showing the last calibration picture after the
panel had drawn the fit check (2026-10-01). Now, as screens/keyboard.cpp does,
it writes into three linear DMA-BUFs SteamVR imported once (the size of the
biggest panel; texture bounds show the part in use) and switches between them.
A show makes the overlay visible once its first picture is in.
The full calibration's and the five-dot check's dots wait for a left click or
Meta+J while you look at the dot, in place of capturing any steady gaze (which
can be a look somewhere else); no capture ring, no 8 s skip, and the check
closes after 2 minutes without a click. The quick check still captures itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Check headset fit now runs in the headset panel too ("fitcheck"): a card per
eye (tracked or lost, the tracker's signal, how much of the last 10 s it was
seen) and the hints, from the probe's fitcheck.py, updated at most twice a
second; left click or Meta+J runs its guided check, right click or Meta+K closes
it. So Quick check, Calibrate, and Check headset fit all happen in one place.
The gaze probe moves to the Gaze page's overflow menu as "Gaze probe
(development)", and its app menu entry says it's a development tool. Texts that
sent users to it ("use Calibrate… with Own tracker") point at Calibrate.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A drag begun by right during the left's held-back press (or Meta+K during
Meta+J's) now lasts while either button or key is held, so pressing the right
one again is free: it tilts, as a right press does during any mouse drag. Meta+K
during a keyboard drag (held still into one, or the second Meta+K) tilts while
held: the head turns the panel, and the mouse can too; let go and the head drags
again from there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The right button's press is held back like the left's: hold it, the pointer
stops where you look, move onto the target, and the right click comes on the
release (held still, it's a real right press). Right during the left's held-back
press now starts a drag where the pointer is, as Meta+J then Meta+K does, in
place of a right click there; letting go of either drops it. So once you've
moved, the left alone only clicks, and the right starts a drag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
POINTER_GAZE_MOUSE_MOVE=held, the default (the Gaze page's Mouse movement
switch, with why it's on): while the gaze has the pointer, moving the mouse
does nothing; it moves the pointer only while a button is held, as a
correction. A bumped or drifting mouse can't pull the pointer off what you're
looking at, and every mouse move is a correction, so lessons aren't polluted by
mouse moves to somewhere else. With the gaze stale for a second, in a game, or
with the headset off, the mouse moves the pointer as usual. "free" is the old
behaviour.
Pressing the right button while the left one's press is held back right-clicks
where the pointer is instead (correct with the left, then right-click); both
releases are then nothing. Meta+J then Meta+K stays a drag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drops the quick check's second step (3629681): the user didn't want it, and it
re-entered itself every tick, re-placing and redrawing the panel, which
flickered in the headset.
POINTER_GAZE_NUDGE_MAX is the one learning limit, now 55 degrees by default:
half of the 109 the Frame shows across. The helper measures the correction
itself (raw gaze to click), not the mouse's path; past the limit it isn't
learned and the helper sends "recheck" for the quick check, for the mouse and
keyboard alike. ft-gazed's own limits (8 and 25) follow the setting.
Test, our tracker's 409 clicks since its Sep 29 calibration: a one-dot check
set from any one of them puts the next 2 minutes' clicks within 15 degrees (99%
within 4.2) and the next 10 minutes' within 25, so it gets well under 55.
The panel's dot is still and its capture ring fills in quarters, so it's drawn
again only a few times per dot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
After the quick check's capture, in gaze mode, the dot stays where it is in the
room (the panel's new "lock") and the gaze has the pointer again. You put the
pointer on the dot as you'd correct a click, with the mouse or Meta+J and the
head. That click clicks nothing: the helper sends it as "calverify", learned
whatever its size. Over POINTER_GAZE_NUDGE_MAX, the capture runs again, up to
3 times. A right click or Meta+K skips it.
POINTER_GAZE_NUDGE_MAX is now the one limit: 15 degrees by default (was 8),
and ft-gazed's own limits (8 for SteamVR's tracker, 25 for ours) follow it. A
keyboard correction past it isn't learned but opens a quick check ("recheck"),
in place of the 30 degree keyboard limit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Meta+J hold's correction is always meant, but it went through the mouse's
8 degree nudge limit. Live, our tracker was 12 degrees off and every correction
was dropped without a word. Debug output now says when one is.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Steam's eyetracking.txt writes "HMD on" every minute or so with nobody in the
headset, and SteamVR said it was worn for 12 hours straight, so the quick check
opened about 40 times an hour at an empty headset. It now opens when SteamVR's
tracker sees eyes for 3 s after none for 3 s, and closes when they're gone 2 s.
The log reader also drops repeated "HMD on" lines and sub-second offs, so
lessons stop counting as from an older wear every minute.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
remap_volume stopped at the first EVIOCSKEYCODE_V2 that failed, so the
volume entries after it were never tried. On a keyboard where swapping
volume up failed, volume down stayed a real key that gamescope reads,
and one press with nothing focused aborts gamescope and the VR session.
Restoring had the same hole: a failed entry left the later stand-ins in
place after the relay exited.
Now every entry is tried and the first failure is raised at the end.
take_volume already marks the device remapped on that error, so the
relay routes the stand-ins that did swap and restores them on exit.
Checked against a fake keymap with a stubbed fcntl.ioctl: with volume
up's swap failing, volume down now swaps (it stayed real before), the
same for restore, and full swaps, restores and the no-keymap case come
out as before. Found by 0x1f6 in PR #8.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From 0x1f6's robustness PR: survive a malformed control datagram (adapted
to wrap experimental's textfield command too) and plan a layout when the
rows setting exceeds what the screens fill. The PR's third commit, the
volume-key swap rollback, is left out: rolling back a partial remap leaves
every real volume key exposed to gamescope instead of some, and on restore
it turns already-restored keys back into stand-ins.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the Frame, a SteamOS update replaces SteamVR, KWin, and gamescope with the
rest of the OS image. scripts/update-check.py, run by doctor.sh and report.sh,
checks what Frametop uses from it: the OpenVR interface versions the installed
programs were built against, the vrcmd --overlays format, the eye tracker's
shared memory layout, host files, services, sockets, and the driver
registration. doctor.sh --mark-good records the package versions once things
work, and later runs say what changed and what to try by hand.
The session no longer stops when mesavars.sh or flatpak.sh is missing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
KWin's PlacementTracker keeps each window's geometry, full screen and
maximized state per layout of the outputs, and puts windows back when a
layout it has seen comes back. A spare output resizes after its window,
so resizing a floating window back to an earlier size (or changing its
scale, or full screen) made the window and its output flip forever, and
floating or docking one window could move others onto or off a spare.
The KWin script now keeps where each window belongs, reports nothing
while KWin changes the outputs, and on screensChanged puts floating
windows back (and the screens' windows when only spares changed),
cancelling KWin's requests before the app sees them. A size asked for is
held for a second against late answers.
With that: a launched app and a profile get their remembered scale back,
scale steps keep the size in pixels, ft-floatd waits for the end of an
edge resize before resizing the output (KWin cancels the resize on any
output change), a window taken over after a restart keeps its app and
panel density, and `ft-float float ID` asks the script for the window's
current place.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- KWin 6.2's scripts have no maximize mode: a window counts as maximized
when it fills its output's maximize area.
- A late event for a window that just closed brought it back into
ft-floatd's table, so a profile "found" it open, floated a window that no
longer existed, and held a slot. The script doesn't report deleted
windows, and ft-floatd ignores events for ids it has seen close.
- Docking a window that came from a screen that's hidden now puts it on the
first screen that shows instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A profile is a named layout plus the screens it hides and its apps' windows
(docs/profiles.md). ft-layout save captures them (ft-floatd's "windows",
after the KWin script reports every window as it is now); use opens them:
the screens move, open windows of each app go to their places (on a screen,
maximized or not, or floating), and missing apps start, once and then again
for each window still missing 3 s after the first. Nothing closes.
The desktop starts in FT_PROFILE or default_profile (ft-layout start, from
the session script). Each profile gets a launcher entry (Frametop: NAME, in
SteamVR's Launch a program list) that switches to it or starts the desktop
in it. The relay's profile:NAME action and Input Settings' "Open profile"
entries put one on a key, mouse button, or controller button. Display
Settings' Layout page becomes Layout & profiles: Save as profile, Open
profile, the profile's apps, and Start in profile. Plasma's own session
restore is off in the Frametop session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-screens: "conceal <screen|all>" hides a screen on its own, whatever the
visibility mode or the hotkey says, until "reveal"; "concealed" lists them.
(Not "hide N": older builds read anything starting with "hide" as the hotkey.)
ft-layout keeps it per screen ("hidden" in the layout), applies it when it
arranges the screens, and has hide/show N|all and hidden. Display Settings
gets a Shown switch per screen on the Visibility tab. ft-floatd floats a new
window that opens on a hidden screen, and puts a stray window on a screen
that shows. Profiles (next) use it to show only some screens.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-float launch APP / run COMMAND: ft-floatd starts the app and floats its
first window (matched by process or desktop file name for 30 s) where that
app last floated, or in front of you at the primary screen's density. Each
app's place (pose relative to the primary screen, size in pixels, scale) is
kept in ~/.config/frametop-float.json whenever one of its windows stops
floating; the scale isn't applied yet, since rescaling can loop (written up
in docs/floating-windows.md, Known problems).
Launch as Standalone (float/ft_apps.py): the session writes copies of the
apps' desktop files with that action and puts them first in XDG_DATA_DIRS,
so it shows in the Application Launcher's and the taskbar's right-click menus
in the Frametop desktop only. ft-floatd rewrites them when apps change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-gazepanel (gaze/panel) is a SteamVR overlay that stays put in your view and
shows dots at known head-relative directions; the gaze service runs it and
drives it (gaze/gazecheck.py):
- a one-dot quick check 3 s after the headset goes on, when our tracker asks
for a click (reseat), at most every 2 minutes, and from Quick check on the
Gaze page; five dots follow if the next 3 lessons are still over 2 degrees off
- the full calibration (the probe's three rounds, dark to bright) when gaze mode
comes on without one, or from Calibrate; quitting it with still no calibration
turns gaze mode off
Each dot takes the gaze once it has held still for 0.6 s; a left click or Meta+J
takes it at once, a right click or Meta+K closes the panel (the pointer helper
hides its dot and passes those presses on while "calpanel" lasts). Our tracker
gets clicks and its own calibration; SteamVR's gets lessons per eye, or a new
calibration.json.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The decoration's "bottom" property clashed with Item's (KWin refused it).
- No On All Desktops button with one virtual desktop, like Breeze.
- decoration/apply.sh finds the session's D-Bus through plasmashell (KWin's
environment isn't readable), and installs each try under a new name: KWin
keeps a decoration's QML by name until it restarts.
- ft-floatd starts its script with Scripting.start: after a reload, the new
script gets the old one's id while the old one is still being deleted, so
run() on /Scripting/Script<id> went to the old script and the new one never
ran (a second ft-floatd start left floating windows without their script).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gaze_left and gaze_right (Meta+J and Meta+K by default) click where you look.
A tap clicks where the dot was at the press and tells the gaze tracker it was
right there. Held, the pointer stays put in your view, so turning your head
carries it onto the target; the release clicks there and the correction is a
lesson (judged by the net correction, not the head's path). Held still for
POINTER_GAZE_HOLD it's a real press that the head drags, and Meta+K while
Meta+J aims presses where the dot is now, to correct and then drag.
A Meta combination now also sends the desktop an F24 press and Meta's release
at once: Meta's release no longer opens Plasma's launcher, and the click isn't
Meta+click (KWin's window move and resize, which swallowed right clicks). The
desktop gets Meta back for the next key if it's still held.
Also fixes e16b758, which had taken the gaze actions off the key combination
list along with the controllers'; and gaze_quickcal (the gaze service's
one-dot check) joins the actions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frametop's own window decoration (decoration/), a QML decoration for KWin's
Aurorae engine drawn like Breeze, puts a float button left of Close. It's the
Keep Below button with its own glyph: the KWin script floats a window when
keep-below is set and docks it when it's cleared, and keeps the flag set on
every floating window, so the button shows "back to the desktop" there. The
session script installs the decoration for the Frametop desktop only;
decoration/apply.sh switches a running desktop to it or back to Breeze.
Not yet tried in a running KWin.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The input relay owns the float key now: float_toggle (Meta+Shift+F unless the
rules have their own key combinations) floats the window under the pointer, or
the active one over the wallpaper, and docks it if it floats. dock_all puts
every floating window back. Both are mappable to mouse and controller buttons,
and work without pointer mode. The KWin script no longer registers a shortcut,
and ft-floatd drops the old one. Key combinations move to Input Settings'
Keyboard page.
docs/floating-windows.md records the decisions from 2026-09-30 (the title bar
button, Launch as Standalone, profiles), and docs/profiles.md plans profiles.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Steam reads the Frame controllers itself, outside SteamVR's bindings, and every
press and release it sees takes SteamVR out of laser mode, so controller clicks
at the gaze can't be done cleanly (docs/gaze-controllers.md, from the gaze-first
branch's tests).
- Gaze precision and Gaze drag take a mouse button or a key combination only;
the controller source, its aim steering, and POINTER_PRECISION_GAIN,
POINTER_PRECISION_DEADZONE and POINTER_GAZE_DRAG_GAIN are gone.
- The gaze actions (including Gaze pointer on/off) can't be mapped to controller
buttons: the relay ignores them there and doesn't ask the helper for those
buttons, and Input Settings no longer offers them.
- Last used wins in gaze mode too: picking up a controller hands it the laser,
as docs/design.md already said.
- The gaze dot shows all the time (POINTER_GAZE_DOT=always, the default;
moving brings back the old behaviour), with a switch on the Gaze page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The local RDP login between krdp and the VNC bridge uses the account's
own name instead of "steamos", its certificate the hostname instead of
"steam-frame", and the ft_pointer driver installs under the user's home
instead of /home/steamos. The remote-access address already came from
the tailnet (tailscale0 and tailscaled's local API).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A GTK 4 / libadwaita app (remote/ft-remote-settings, on the host's own
Python like the gaze probe): remote access on and off (REMOTE, applied at
once when the desktop allows it), the tailnet name and address to connect
to, and the VNC password shown, copied, or replaced. The password stays
random and made on the Frame, in ~/.config/frametop-remote; none is in
the code.
session/remote-ctl.sh starts, stops, and reports remote access; the
session uses it and leaves a remote-capable marker, since KWin allows the
capture only in a desktop that started with REMOTE=1. The password
between krdp and the VNC bridge (local only, but on krdp's command line)
is now new at every start. The installer adds the app to the menu.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
krdp streams every screen, so the VNC screen is now the primary's size and the
FreeRDP window is shifted so the primary fills it (ft-layout remote-view gives
the offset). It resizes and reconnects when the layout changes. With remote
access on, KWin's D-Bus screenshot interface is open too, for scripts that look
at the screens without the headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 6f5a23c80f2ecd26e6a96c86b102be36d518ee25)
Our own eye tracker joins the gaze folder: ft-eyegrab (the root frame
grabber, frametop-eyegrab.service) and ft-eyes, run by the gaze service
when GAZE_TRACKER=own, with its lab tools.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hand role the pointer takes isn't something to choose by hand. The
helper still reads POINTER_ROLE (right by default) for experiments.
Tried and reverted (not committed): in gaze mode, taking the stylus role
by reconnecting. SteamVR gave the device no role at all (hint 5, role
none) and kept the dashboard laser on the held controllers, so the mouse
couldn't click either. The dashboard laser needs a hand role, and a held
Frame controller takes its hand's.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two new actions for mouse buttons, controller buttons, and key
combinations: gaze_precision (hold: the pointer stops where you look and
the button's device steers it, a controller by where it points at
POINTER_PRECISION_GAIN, the mouse by its moves; release: click there) and
gaze_drag (the same with a real press at once, dragging until the
release). The relay sends "precision|gazedrag <source> 1|0" to the
helper, which runs them through the same holds as pinches and grips.
- Key combinations on any keyboard ("key_bindings" in the rules, e.g.
Ctrl+Alt+G for gaze on/off): the last key isn't typed.
- POINTER_GAZE_MOUSE: in gaze mode the left button is a precision button
(the default, as before) or clicks right away (direct).
- In gaze mode a moving controller no longer takes the pointer away.
- POINTER_ROLE (right, left, stylus): the driver takes a "role" command,
so the pointer can stay off the hand holding the precision controller,
which would otherwise take the role back. Needs the rebuilt driver.
- Input Settings: the actions on the Controllers and Buttons pages; on the
Gaze page the mouse choice, the role, the precision sliders, and key
combinations (captured from any keyboard).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hand tracking no longer starts with SteamVR: hands/run.sh install leaves
the units disabled and links hands/ft-handsctl into ~/.local/bin, which
turns it on and off (on | off | status | log | cutouts on|off | gestures).
With it on, hands show through the screens; pinches and grips move the
pointer only with POINTER_HANDS=1, now off by default. ft-camd's service
runs the mono cameras only: while the headset is worn, the colour module
writes just a half-size image into the top-left quarter of its buffers,
which ft-camd can't use yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The palm-down filter is off by default: the user's deliberate pinches,
hand raised, read 0.90-0.99, like typing.
- Typing is told apart by the keyboard instead: the input relay sends the
pointer helper "typing" on key presses, and it takes no pinch within
POINTER_PINCH_TYPING (1 s) of one.
- Grips are still held to hands raised (POINTER_GRIP_BELOW, 0.35 m below
the eyes); pinches aren't, since the user's own sat 0.35-0.45 m below,
elbow resting.
- A grip doesn't begin with the thumb on the index tip: that's a pinch
with the other fingers curled, which was taken for a grip.
- A pinch's point is the index and middle knuckles: the tips' midpoint
moved 1-2 cm as the pinch opened, dragging every release off its press.
- ft-hands --gesture-log prints what the detectors measure, 10 times a
second.
Pinches still aren't reliable enough to use; hand tracking is parked for
now in favour of the controllers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pressed when the pinch closes, released when it opens, its hand dragging
the pointer in between (at the pinch gain, past the dead zone), like the
mouse's button. That's what the gaze probe's Click practice needs: it
freezes its own gaze dot at the press and drags it by the pointer's
movement until the release. With gaze mode on, the pinch still holds the
press back and clicks on the release.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pointer helper reads ft-hands' gestures. A pinch holds the pointer
where the gaze put it and clicks on the release; held, the hand nudges
the pointer at half its angle past a 1.5 degree dead zone, which also
teaches the gaze tracker. A grip presses where the pointer is, drags with
the hand, and releases when the hand opens, unless it began more than
30 cm below the eyes (hands on a desk). Hand movement is taken in the
room with the head pose at capture time. Hand use keeps the pointer from
the relay's idle release, as gaze mode does. POINTER_HANDS and friends in
frametop.conf.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-hands tracks with the mono IR cameras in dim light and with every
camera (or the colour pair, HANDS_BRIGHT) in bright light, going by the
colour frames' mean brightness with hysteresis and a 2 s hold
(HANDS_CAMERAS=auto, the default; mono, color and all fix it). Colour
frames are placed on the mono cameras' clock by their dequeue time, and a
view in a camera a step lacks waits for that camera's next frame.
A grip (a closed hand) is a second gesture next to the pinch, in version 2
of the gestures file: every fingertip curled toward the wrist, beginning
only on a hand seen open within a second and held up in front. On the
2026-09-30 lit recording that leaves 6 false grips of 14, all with the
hands on the desk; pinch counts are unchanged. ft-handreplay logs grips
and finger curl, watch_gestures.py shows them, and tools/cut_sets.py
copies a few sets out of a recording.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A colour camera probes at most 4 buffers a frame (each probe syncs a
~9 MB buffer's cache, which made the mono cameras miss frames), and one
that goes stale twice in a row is paused (10 s, doubling to 160 s) and
learned again instead of ft-camd exiting. The colour cameras run at 2 fps
until a reader asks for more in frametop-hands/color-fps, which saves
most of their decoding while only their brightness is needed. Each mono
camera's latest near-black frame's mean goes in the ring (dark_mean), a
measure of the room's IR light. The service now starts with --with-color;
HANDS_CAMERAS=mono leaves the colour cameras out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hand tracking (ft-camd, ft-hands, and their tools) joins the desktop. The
hands file and ring move to /run/user/UID/frametop-hands/, which ft-screens'
hand cutouts now read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The runtime files move to /run/user/UID/frametop-hands/: the desktop
session deletes /run/user/UID/frametop at every start.
- The cutout copy shader runs at highp: mediump (16-bit on Adreno)
stepped 1.7 texels across a 3440-pixel screen.
- One hand no longer pinches both sides after its left/right call flips
mid-pinch, and --pinch-palm-down (0.6) holds back pinches with the palm
facing down (typing on a lap keyboard).
- ft-camd judges a colour frame fresh by its luma rows only, and logs
per-buffer changes at stale colour frames with FT_CAMD_DEBUG=1.
- hands/run.sh caps skips the setcap when ft-camd already has them.
- The replay tool dumps poses (--poses), and its pinch events carry the
hand id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Floating windows join the keyboard and the hand cutouts. Floating panels
don't get cutouts yet: their panel and popups show crops of the client
buffer (texture bounds), which the side-by-side cutout buffer doesn't
match. KWin gets both the spare outputs and our input method, and the
pointer helper's frametop. prefix already covers the float panels.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes#7: SteamVR's keyboard never came up for the desktop's apps, and
opening it for our panels doesn't work well on the Frame (it's Steam's own
panel, mounted in the dashboard's scene, it follows the laser between
panels, and it takes the controllers over to SteamVR's laser).
- KWin starts input/ft-textinput as the desktop's input method. It tells
the input relay when a text field gains or loses focus, and the relay
asks ft-screens to open or close the keyboard. The session drops the
QT_IM_MODULE=xim and GTK_IM_MODULE=xim that the gamescope session sets,
or Qt and GTK apps never report text fields.
- The keyboard is ft-screens' own panel (screens/keyboard.cpp): a US laptop
layout, typed with a controller's laser or the 3D mouse. It opens 0.7 m
in front of you, below your eyes and facing you. It has a grab bar to
move it, a Close key, latching Shift, Ctrl and Alt, and repeat on a held
key. It's drawn into shared DMA-BUFs, so it doesn't flicker. Its keys
reach the focused screen as key presses, so every app takes them.
- It steps aside while the Steam menu or Steam's own keyboard is up and
comes back after. A layout reset closes it, and it doesn't open without
a head pose.
- Frametop Input Settings has a Keyboard page: open it for every text
field, only while no keyboard is connected (the default), only from a
mapped button (the new Open/close keyboard action, for mice and
controllers), or never. A switch keeps it open until you close it.
- The pointer helper treats every frametop.* overlay as a real panel. The
keyboard's shared texture reports 0x0 like SteamVR's scene-graph
controls, and the helper had given it their wide catch radius.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- KWin's nested backend makes an output the size it's configured to times its
scale, so after Meta+scroll every size ft-floatd sent was multiplied again,
and an odd result disconnected KWin (buffer not divisible by its scale).
ft-floatd now asks for sizes in the output's scaled terms (kwin_size), and
asks again after each scale change.
- SteamVR reports mouse positions on a panel with texture bounds in the whole
texture, not the crop, so clicks on a floating window landed up to ~200 px
off. The mouse scale is now the buffer's size, as on a screen.
- A floated window starts 30 cm in front of its screen (was 5 cm), so it's
easy to point at apart from the screen behind it.
- No 1 s wait before a spare turns on (a disabled output never commits), and
the login splash on the spares isn't taken for floating windows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frame-eyes, a separate project until now, becomes gaze/tracker:
- ft-eyegrab (was fe-bufprobe) copies the eye-camera frames, read-only, out of
SteamVR's eyetracking process. It runs as the system service
frametop-eyegrab.service, which gaze/tracker/install.sh installs to
/etc/frametop with sudo. It keeps only CAP_SYS_PTRACE, CAP_DAC_READ_SEARCH, and
CAP_CHOWN, and copies frames only while /dev/shm/frametop-eyes-want is fresh,
holding none of the tracker's buffers otherwise.
- ft-eyes (was fe-trackd) runs under ft-gazed in the dev container, with
build/venv's pinned numpy and OpenCV: while Eye tracker is Own tracker, or on
the probe's lease ("eyes SECONDS"). No sudo password or fe-live script at
run time any more.
- lab/ holds the research tools (ft-eyes-score, -e2e, -record, -replay,
-session) and findings.md. Recordings live outside the repo, in
~/.local/share/frametop/eyes/captures; .gitignore catches stray frame dumps.
Its socket is now @ft_eyes, its output /dev/shm/frametop-eyes-gaze, and its state
~/.local/state/frametop/gaze/eyes. On practice1 -> practice2 the whole live path
(ft-eyes-e2e) gives 1.30 deg median and 3.18 for the worst tenth, as before the
move (1.30, 3.21).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-screens' hand cutouts read /run/user/UID/frametop/hands through
hands/include/fh_hands.h instead of their own copy of its offsets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Programs: ft-camd (the camera broker), ft-hands (the tracker), and
ft-handreplay and ft-ringplay for recordings, built by hands/build.sh
into hands/build/ with one Makefile. The first build fetches ncnn at
frame-hands' pinned tag and builds it with the same options.
- ft-camd gets its privileges from file capabilities (CAP_SYS_PTRACE,
CAP_PERFMON, CAP_DAC_READ_SEARCH) that hands/run.sh install sets with
sudo, and drops them once set up. It still works under sudo. It runs
on the host, linked statically, as frametop-camd.service. ft-hands
runs in the dev container as frametop-hands.service. Both start and
stop with SteamVR.
- Files move to /run/user/UID/frametop/ (cam-ring, hands, gestures),
not $XDG_RUNTIME_DIR, which a terminal in the Frametop desktop has
its own of. SIGUSR1 recordings go to ~/.local/share/frametop/hands.
- The calibration is read through /run/host in the container.
- Settings: HANDS_SWAP_SIDES and HANDS_CPUS in frametop.conf.
- install.sh offers hand tracking as an optional last step.
- The container gets jsoncpp-devel, glibc-static, and NumPy and OpenCV
for the Python tools.
- tools/ring.py reads the ring, and models/NOTICE credits the
Apache-2.0 models.
Checked: ft-handreplay gives identical summaries and byte-identical
depth dumps to frame-hands' fh-replay on both 2026-09-29 recordings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frametop Input Settings' Gaze page gets two settings, saved in frametop.conf and
read again by ft-gazed when the file changes: Eye tracker (GAZE_TRACKER: SteamVR's
or our own, frame-eyes' fe-trackd) and Eye bias (GAZE_EYE: auto, left, right).
ft-gazed now combines the eyes, each calibrated on its own: SteamVR's set 2 eyes
with the probe's Left eye and Right eye calibrations, or our tracker's eyes as
they come. Without per-eye calibrations, or with --source, it keeps the older
one-source path. A pointer nudge finds its look from the raw gaze the helper
echoes back; with our tracker it goes to fe-trackd as a click.
The bias leans instead of choosing (gazecal.EyeWeights): on 306 live clicks the
eyes' errors partly cancelled, both together 0.65 deg off against 0.96 and 1.11
for either alone. Left or Right counts that eye twice; auto weights each eye by
its RMS miss at its last 20 nudges, since the calibration's fit picked the wrong
eye on SteamVR's test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trackd/ becomes track/, and the calibration helper the Python tools
import moves from the prototype's folder into tools/. The model
development tools (nettest and the scripts that compare it with the
Python models or cut int8 calibration crops) stay in frame-hands with
the prototype they need.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The history of frame-hands (~/Desktop/Projects/frame-hands on the
Frame), filtered to what moves: the camera broker (camd/), the tracker
and its offline tools (trackd/), the shared file layouts (include/), the
ncnn models, the analysis tools, and the calibration and model helpers
they import from the Python prototype. The reverse-engineering notes,
probes, camprobe, and the rest of the prototype stay in frame-hands.
Unchanged here: the renames to ft- names and Frametop paths follow.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
frame-hands (the headset cameras' hand tracking, a separate project so
far) becomes a native component under hands/. docs/hands-migration.md
has what it is, where each part goes, names, build, how ft-camd gets
its privileges (file capabilities), interfaces, open items, and steps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- fh-camd --with-color publishes the Arcturus colour pair (luma, half
size, 30 fps) to the ring, flagged FH_CAM_COLOR. fh-tracker records
them, and fh-replay --cams mono|color|all tracks with them, using the
module's EEPROM calibration (load_color_calibration).
tools/check_color.py checks which node is left and how the crop maps.
- Pinch detection per hand (trackd/pinch.h), published to
$XDG_RUNTIME_DIR/frame-hands/gestures (include/fh_gestures.h). It has
begin and end counters, times, the pinch point, and the begin point
for drags. tools/watch_gestures.py shows it live, and fh-replay
reports it.
- fh-replay --depth and tools/depth_report.py measure the depth without
ground truth: noise along the line of sight against across it, the
one-camera guess, and a simulated camera loss.
- fh-tracker --swap-sides, --ring, --keep-presence, --record-only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wlroots asserts that nothing still listens to its xdg-shell and decoration
globals when the display goes, so every desktop stop ended with ft-screens
aborting and leaving a core dump.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Full screen fills the window's own panel (the margin drops to zero).
Meta+scroll over a floating window changes its scale at the same size in
pixels. Spare outputs are sized to a multiple of KWin's buffer scale, and
screens to even sizes: an odd buffer at a fractional scale is a protocol
error that disconnected KWin. The 3D mouse's drag lock now crosses onto
other Frametop panels unless the pressed one is being carried.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
