hands/rec/ft_handrec.py + main.qml (Kirigami, dev container; host launcher
hands/rec/ft-handrec): consent (CONSENT.md, asked again when its version
changes; profile.json with a random contributor id), the before-you-start
checklist with the lighting and free-space checks, the session controls
(Space pauses, Esc stops), review with a frame-set viewer that deletes
ranges, takes and sessions, export with progress and cancel (warns while
the headset is worn), and the upload page (UPLOAD.md, the
huggingface-cli command; HF_DATASET is a placeholder). --dry-run runs
sessions without processes, for testing.
hands/rec/takes.py (standard library): indexes sets.bin and sets-N.bin
without reading pixels, reads one set's cameras, keeps deleted ranges in
take.json, and exports: deleted sets left out, zstd -10 -T2 at nice 19,
manifest.json and SHA256SUMS, nothing left behind on cancel.
CONSENT.md and UPLOAD.md are drafts pending a legal review; the window
says contributions aren't open yet. The dev container gains zstd.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hands/rec/session.py runs a recording session from script.json: it starts
ft-camd and a tracking ft-hands as transient units only if they aren't
running, records each section as one take (ft-hands --record-only at
10 sets/s, a new sets-N.bin after each pause), drives ft-handpanel, and
writes session.json, calibration.json (identifying fields removed),
prompts.jsonl and take.json. Feedback comes from the live hands file and,
in the controller sections, from the panel's device poll. It also runs
from the command line (--dry-run, --speed, --ring, --no-start).
hands/rec/script.json: 11 sections, about 9 minutes without the object
and controller sections.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A head-locked SteamVR overlay for the hand recorder (hands/rec/DESIGN.md):
1.2 m ahead, 12 degrees up, 36 degrees wide, drawn with stb_truetype
into three shared DMA-BUFs as ft-gazepanel does. It shows the title,
step, wrapped instruction, note, countdown, hand chips, near/far bar
and a "Paused" cover, driven over @ft_handpanel.
It also places the touch target, a 2 cm dot in its own overlay fixed
in the room where the head was at the first command for that point,
and logs head and controller poses to poses.jsonl at 250 Hz from a
thread of its own. Both threads take one lock around OpenVR calls.
--no-vr prints each picture's state to stdout (and --dump writes the
pictures), for testing without a headset. hands/rec/build.sh builds it
in the dev container.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-hands --record-hz N records at most N frame sets a second, for the hand recorder (10).
DESIGN.md lays out the recorder: the headset panel, the session runner and its script,
the files, review and export, consent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hands/ft-cutouts on|off|status starts ft-camd and ft-hands as transient
user units with ft-hands' new --no-gestures: hands are published for
ft-screens' cutouts, but no pinch or grip is detected, so nothing clicks
or drags and a closing hand doesn't raise the tracking rate. It needs a
build and ft-camd's capabilities, not hands/run.sh install. Its units
conflict with ft-handsctl's, so each stops the other, and they stop with
SteamVR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- install.sh no longer offers hand tracking (it's heavy on the CPU and
needs more work); hands/ still builds and installs by hand
- the build container drops python3-opencv and python3-numpy, which
only hand tracking's tools used: Fedora's OpenCV pulls in over a GB
- README: a new opening and feature list, the Use section by topic
(screens, mouse, floating windows, profiles, gaze), and new known
limitations
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- frame_sudo (scripts/_env.sh) replaces three copies of sudo_run. On
the Frame it asks in the terminal even when stdin isn't one, which
install.sh's steps never had: saying yes to the Bluetooth fixes or
hand tracking stopped the install with "no terminal for sudo".
From a PC it asks through ssh -t when .env has no password.
- start_with_steamvr: the pointer, power, and gaze services restart
when SteamVR runs (a re-install runs the new code) and are left to
start with it when it doesn't, instead of failing the install.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- floating-windows.md and profiles.md describe what's built, with what
isn't listed as such; the plans, phases, and branch notes are gone
- hands-migration.md is gone: the move is done; its open items are in
hands/README.md's Known issues
- reference.md: Layout & profiles, every action, key combinations,
floating windows, the gaze pointer, and hand tracking as they are
- design.md gets the KWin findings from floating-windows.md
- README, gaze/README, AGENTS, hazards, gaze-controllers, and the
example config catch up with gaze, hands, and the stuck-key fix
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- gaze/tracker/lab/eyes_track.py: nothing used it
- Input Settings: the pointer role backend, whose page was removed
- ft-screens: no log line for every floating-window resize
- hands: ft-hands --help gives the palm-down default (1: off), the
uninstall removes ft-handsctl's link, .frame-job is ignored
- Display Settings: "Save as profile…", not "Save current arrangement"
- two stale comments (ft-pointer's grabprobe, ft-gaze)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hand tracking no longer starts with SteamVR: hands/run.sh install leaves
the units disabled and links hands/ft-handsctl into ~/.local/bin, which
turns it on and off (on | off | status | log | cutouts on|off | gestures).
With it on, hands show through the screens; pinches and grips move the
pointer only with POINTER_HANDS=1, now off by default. ft-camd's service
runs the mono cameras only: while the headset is worn, the colour module
writes just a half-size image into the top-left quarter of its buffers,
which ft-camd can't use yet.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The palm-down filter is off by default: the user's deliberate pinches,
hand raised, read 0.90-0.99, like typing.
- Typing is told apart by the keyboard instead: the input relay sends the
pointer helper "typing" on key presses, and it takes no pinch within
POINTER_PINCH_TYPING (1 s) of one.
- Grips are still held to hands raised (POINTER_GRIP_BELOW, 0.35 m below
the eyes); pinches aren't, since the user's own sat 0.35-0.45 m below,
elbow resting.
- A grip doesn't begin with the thumb on the index tip: that's a pinch
with the other fingers curled, which was taken for a grip.
- A pinch's point is the index and middle knuckles: the tips' midpoint
moved 1-2 cm as the pinch opened, dragging every release off its press.
- ft-hands --gesture-log prints what the detectors measure, 10 times a
second.
Pinches still aren't reliable enough to use; hand tracking is parked for
now in favour of the controllers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-hands tracks with the mono IR cameras in dim light and with every
camera (or the colour pair, HANDS_BRIGHT) in bright light, going by the
colour frames' mean brightness with hysteresis and a 2 s hold
(HANDS_CAMERAS=auto, the default; mono, color and all fix it). Colour
frames are placed on the mono cameras' clock by their dequeue time, and a
view in a camera a step lacks waits for that camera's next frame.
A grip (a closed hand) is a second gesture next to the pinch, in version 2
of the gestures file: every fingertip curled toward the wrist, beginning
only on a hand seen open within a second and held up in front. On the
2026-09-30 lit recording that leaves 6 false grips of 14, all with the
hands on the desk; pinch counts are unchanged. ft-handreplay logs grips
and finger curl, watch_gestures.py shows them, and tools/cut_sets.py
copies a few sets out of a recording.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A colour camera probes at most 4 buffers a frame (each probe syncs a
~9 MB buffer's cache, which made the mono cameras miss frames), and one
that goes stale twice in a row is paused (10 s, doubling to 160 s) and
learned again instead of ft-camd exiting. The colour cameras run at 2 fps
until a reader asks for more in frametop-hands/color-fps, which saves
most of their decoding while only their brightness is needed. Each mono
camera's latest near-black frame's mean goes in the ring (dark_mean), a
measure of the room's IR light. The service now starts with --with-color;
HANDS_CAMERAS=mono leaves the colour cameras out.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The runtime files move to /run/user/UID/frametop-hands/: the desktop
session deletes /run/user/UID/frametop at every start.
- The cutout copy shader runs at highp: mediump (16-bit on Adreno)
stepped 1.7 texels across a 3440-pixel screen.
- One hand no longer pinches both sides after its left/right call flips
mid-pinch, and --pinch-palm-down (0.6) holds back pinches with the palm
facing down (typing on a lap keyboard).
- ft-camd judges a colour frame fresh by its luma rows only, and logs
per-buffer changes at stale colour frames with FT_CAMD_DEBUG=1.
- hands/run.sh caps skips the setcap when ft-camd already has them.
- The replay tool dumps poses (--poses), and its pinch events carry the
hand id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Programs: ft-camd (the camera broker), ft-hands (the tracker), and
ft-handreplay and ft-ringplay for recordings, built by hands/build.sh
into hands/build/ with one Makefile. The first build fetches ncnn at
frame-hands' pinned tag and builds it with the same options.
- ft-camd gets its privileges from file capabilities (CAP_SYS_PTRACE,
CAP_PERFMON, CAP_DAC_READ_SEARCH) that hands/run.sh install sets with
sudo, and drops them once set up. It still works under sudo. It runs
on the host, linked statically, as frametop-camd.service. ft-hands
runs in the dev container as frametop-hands.service. Both start and
stop with SteamVR.
- Files move to /run/user/UID/frametop/ (cam-ring, hands, gestures),
not $XDG_RUNTIME_DIR, which a terminal in the Frametop desktop has
its own of. SIGUSR1 recordings go to ~/.local/share/frametop/hands.
- The calibration is read through /run/host in the container.
- Settings: HANDS_SWAP_SIDES and HANDS_CPUS in frametop.conf.
- install.sh offers hand tracking as an optional last step.
- The container gets jsoncpp-devel, glibc-static, and NumPy and OpenCV
for the Python tools.
- tools/ring.py reads the ring, and models/NOTICE credits the
Apache-2.0 models.
Checked: ft-handreplay gives identical summaries and byte-identical
depth dumps to frame-hands' fh-replay on both 2026-09-29 recordings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trackd/ becomes track/, and the calibration helper the Python tools
import moves from the prototype's folder into tools/. The model
development tools (nettest and the scripts that compare it with the
Python models or cut int8 calibration crops) stay in frame-hands with
the prototype they need.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The history of frame-hands (~/Desktop/Projects/frame-hands on the
Frame), filtered to what moves: the camera broker (camd/), the tracker
and its offline tools (trackd/), the shared file layouts (include/), the
ncnn models, the analysis tools, and the calibration and model helpers
they import from the Python prototype. The reverse-engineering notes,
probes, camprobe, and the rest of the prototype stay in frame-hands.
Unchanged here: the renames to ft- names and Frametop paths follow.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
