Hands: pick the cameras by the light, and detect grips

ft-hands tracks with the mono IR cameras in dim light and with every
camera (or the colour pair, HANDS_BRIGHT) in bright light, going by the
colour frames' mean brightness with hysteresis and a 2 s hold
(HANDS_CAMERAS=auto, the default; mono, color and all fix it). Colour
frames are placed on the mono cameras' clock by their dequeue time, and a
view in a camera a step lacks waits for that camera's next frame.

A grip (a closed hand) is a second gesture next to the pinch, in version 2
of the gestures file: every fingertip curled toward the wrist, beginning
only on a hand seen open within a second and held up in front. On the
2026-09-30 lit recording that leaves 6 false grips of 14, all with the
hands on the desk; pinch counts are unchanged. ft-handreplay logs grips
and finger curl, watch_gestures.py shows them, and tools/cut_sets.py
copies a few sets out of a recording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DeeJanuzandClaude Opus 5.5 committed 2026-09-30 14:31:17 -06:00
1 parent 40a2f39b2a
commit 50c14545ed
11 files changed
+746 -185

No files matched your search

+11
View File
@@ -36,6 +36,17 @@ GAZE_TRACKER=steam # gaze service: steam = SteamVR's eye tracker | own =
GAZE_EYE=auto # gaze service: eye bias. auto = each eye weighted by how far off it was at your recent nudges | left | right = that eye counts twice
HANDS_SWAP_SIDES=0 # hand tracking (hands/run.sh install): 1 = the side cameras' names are swapped, which some SteamVR restarts cause (hands/tools/check_sides.py --ring tells)
HANDS_CPUS=5,6,7 # hand tracking: the CPUs its model threads run on
HANDS_CAMERAS=auto # hand tracking: auto (by the light) | mono (the IR cameras; also keeps ft-camd off the colour ones) | color | all
HANDS_BRIGHT=all # hand tracking, auto: the cameras in bright light, all | color
HANDS_BRIGHT_ON=40 # hand tracking, auto: the colour frames' mean brightness (0-255) that counts as bright (a dim room reads about 9)
HANDS_BRIGHT_OFF=25 # ... and back to the mono cameras under this
HANDS_COLOR_LEFT=color_video0 # hand tracking: which colour camera is passthrough_left (hands/tools/check_color.py tells)
HANDS_COLOR_CROP=subtract # hand tracking: how the colour calibration's crop applies, subtract | none (check_color.py tells)
POINTER_HANDS=1 # 1 = pinches click (on the release; hold and move to nudge the pointer first) and grips press and drag, with hand tracking
POINTER_PINCH_GAIN=0.5 # hands: the pointer moves this times the pinching hand's angle (precision)
POINTER_PINCH_DEADZONE=1.5 # hands: degrees the pinching hand moves before the pointer does (a tap's jitter)
POINTER_GRIP_GAIN=1 # hands: the pointer moves this times the gripping hand's angle
POINTER_GRIP_BELOW=0.3 # hands: grips beginning more than this (m) below the eyes are ignored (hands on a desk)
META_DASHBOARD=0 # 1 = a Meta tap on a pass-through keyboard toggles the SteamVR dashboard (pointer mode only)
SHARE_KEYS=0 # 1 = keys of keyboards grabbed for the desktop also go to @frametop_keys, for hotkey tools (any local process can listen)
DISPLAY_OFF_MIN=0 # minutes without use before frametop-power turns the displays off, even if the headset seems worn (0 = never)