With HANDS_SWAP_SIDES set to 0 or 1, ft-hands kept the forced naming as the
published truth even after the hands showed it was backwards. The hand recorder
took that as the session's decision, so the export labelled slam_left and
slam_right the wrong way round (dataset PR #4: every take swapped).
ft-hands now publishes the hands' answer once they disagree (state "forced,
disagrees"); tracking keeps the forced names. sides.read_live corrects the same
case from an ft-hands built before this, and the recorder's log says to use auto.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without the Arcturus module, XRService runs the side cameras through the
ISP ("ISP enabled for tracking cameras (main VFE available)"): NV12 on
vfe0 and vfe1, pitch 1152. ft-camd published only grey cameras, so it
dropped them, and the recorder stopped at "ft-camd publishes only 2 of 4
mono cameras" whatever was restarted.
- ft-camd reads a tracking camera's NV12 luma as its grey image.
- ft-hands and check_sides name the cameras by XRService's numbering in
its log (TrackingCameraInit index 0-3), not by capture pipe, which
moves with the module; a camera whose size isn't its calibration's is
left out. sides.json says which device each camera was.
- camcheck reports the camera map, the ISP routing and which cameras
ft-camd is missing; the recorder says so instead of "restart it",
restarts its own ft-camd once when that's the only problem, and keeps
the map in session.json.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 4db5266cf7)
ft-camd tells the side cameras apart by XRService's buffer allocation order,
which some XRService starts reverse; both of 2026-10-02's starts did, so the
cutouts missed the hands. HANDS_SWAP_SIDES=auto (the default) has ft-hands
vote from hands seen in both side cameras: the landmark rays meet in front
of both cameras only under the right naming. While undecided it probes the
exchanged naming with the landmark model. It decides in about 2 s of hands
(right on all 7 recordings replayed), swaps the views in place, and publishes
sides.json. 0 and 1 still force it, with a warning when the hands disagree.
Recordings carry each part's naming and the session's decision; review,
export, validate and ft-handreplay put the names right, and takes.py sides
records a decision by hand.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-hands --record-hz N records at most N frame sets a second, for the hand recorder (10).
DESIGN.md lays out the recorder: the headset panel, the session runner and its script,
the files, review and export, consent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hands/ft-cutouts on|off|status starts ft-camd and ft-hands as transient
user units with ft-hands' new --no-gestures: hands are published for
ft-screens' cutouts, but no pinch or grip is detected, so nothing clicks
or drags and a closing hand doesn't raise the tracking rate. It needs a
build and ft-camd's capabilities, not hands/run.sh install. Its units
conflict with ft-handsctl's, so each stops the other, and they stop with
SteamVR.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- gaze/tracker/lab/eyes_track.py: nothing used it
- Input Settings: the pointer role backend, whose page was removed
- ft-screens: no log line for every floating-window resize
- hands: ft-hands --help gives the palm-down default (1: off), the
uninstall removes ft-handsctl's link, .frame-job is ignored
- Display Settings: "Save as profile…", not "Save current arrangement"
- two stale comments (ft-pointer's grabprobe, ft-gaze)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The palm-down filter is off by default: the user's deliberate pinches,
hand raised, read 0.90-0.99, like typing.
- Typing is told apart by the keyboard instead: the input relay sends the
pointer helper "typing" on key presses, and it takes no pinch within
POINTER_PINCH_TYPING (1 s) of one.
- Grips are still held to hands raised (POINTER_GRIP_BELOW, 0.35 m below
the eyes); pinches aren't, since the user's own sat 0.35-0.45 m below,
elbow resting.
- A grip doesn't begin with the thumb on the index tip: that's a pinch
with the other fingers curled, which was taken for a grip.
- A pinch's point is the index and middle knuckles: the tips' midpoint
moved 1-2 cm as the pinch opened, dragging every release off its press.
- ft-hands --gesture-log prints what the detectors measure, 10 times a
second.
Pinches still aren't reliable enough to use; hand tracking is parked for
now in favour of the controllers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-hands tracks with the mono IR cameras in dim light and with every
camera (or the colour pair, HANDS_BRIGHT) in bright light, going by the
colour frames' mean brightness with hysteresis and a 2 s hold
(HANDS_CAMERAS=auto, the default; mono, color and all fix it). Colour
frames are placed on the mono cameras' clock by their dequeue time, and a
view in a camera a step lacks waits for that camera's next frame.
A grip (a closed hand) is a second gesture next to the pinch, in version 2
of the gestures file: every fingertip curled toward the wrist, beginning
only on a hand seen open within a second and held up in front. On the
2026-09-30 lit recording that leaves 6 false grips of 14, all with the
hands on the desk; pinch counts are unchanged. ft-handreplay logs grips
and finger curl, watch_gestures.py shows them, and tools/cut_sets.py
copies a few sets out of a recording.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The runtime files move to /run/user/UID/frametop-hands/: the desktop
session deletes /run/user/UID/frametop at every start.
- The cutout copy shader runs at highp: mediump (16-bit on Adreno)
stepped 1.7 texels across a 3440-pixel screen.
- One hand no longer pinches both sides after its left/right call flips
mid-pinch, and --pinch-palm-down (0.6) holds back pinches with the palm
facing down (typing on a lap keyboard).
- ft-camd judges a colour frame fresh by its luma rows only, and logs
per-buffer changes at stale colour frames with FT_CAMD_DEBUG=1.
- hands/run.sh caps skips the setcap when ft-camd already has them.
- The replay tool dumps poses (--poses), and its pinch events carry the
hand id.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Programs: ft-camd (the camera broker), ft-hands (the tracker), and
ft-handreplay and ft-ringplay for recordings, built by hands/build.sh
into hands/build/ with one Makefile. The first build fetches ncnn at
frame-hands' pinned tag and builds it with the same options.
- ft-camd gets its privileges from file capabilities (CAP_SYS_PTRACE,
CAP_PERFMON, CAP_DAC_READ_SEARCH) that hands/run.sh install sets with
sudo, and drops them once set up. It still works under sudo. It runs
on the host, linked statically, as frametop-camd.service. ft-hands
runs in the dev container as frametop-hands.service. Both start and
stop with SteamVR.
- Files move to /run/user/UID/frametop/ (cam-ring, hands, gestures),
not $XDG_RUNTIME_DIR, which a terminal in the Frametop desktop has
its own of. SIGUSR1 recordings go to ~/.local/share/frametop/hands.
- The calibration is read through /run/host in the container.
- Settings: HANDS_SWAP_SIDES and HANDS_CPUS in frametop.conf.
- install.sh offers hand tracking as an optional last step.
- The container gets jsoncpp-devel, glibc-static, and NumPy and OpenCV
for the Python tools.
- tools/ring.py reads the ring, and models/NOTICE credits the
Apache-2.0 models.
Checked: ft-handreplay gives identical summaries and byte-identical
depth dumps to frame-hands' fh-replay on both 2026-09-29 recordings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
trackd/ becomes track/, and the calibration helper the Python tools
import moves from the prototype's folder into tools/. The model
development tools (nettest and the scripts that compare it with the
Python models or cut int8 calibration crops) stay in frame-hands with
the prototype they need.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
