Releases: ship the Hand Recorder

The image builds ft-handpanel (hands/rec/build.sh, against scripts/openvr.sh like gaze), and a release's installer offers the recorder: ft-camd's capabilities and the menu entry, nothing to build. The FrameDrop window has a Hand Recorder box, and --no-hand-recorder leaves it out. An entry from the release before is renewed without asking, since it points into that release's folder. The recorder runs ft-hands through scripts/in-box instead of the "dev" container, so a release uses its own container, and its repair hints say to reinstall the release rather than run a build script. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Report: an area per flag, and --watch records each area live
2026-10-15 00:00:42 +02:00 · 2026-10-14 14:32:07 -06:00 · 2026-10-14 14:16:16 -06:00 · 2026-10-14 13:17:00 -06:00 · 2026-10-14 13:17:00 -06:00 · 2026-10-14 09:46:01 -06:00
36 changed files with 1054 additions and 217 deletions
@@ -16,3 +16,6 @@
 **/.env.*
 frametop-report-*.txt
 .frame-job.d/
+# the fine-tuned hand models: the image fetches its own pinned copy (hands/models.sh)
+hands/models/student/
+hands/models/.student-*
@@ -57,10 +57,15 @@ jobs:
              test -x /opt/frametop/bin/$p
            done
            test -f /src/frametop/pointer/driver/build/driver_ft_pointer.so
+            test -x /src/frametop/hands/build/ft-camd
+            test -x /src/frametop/hands/build/ft-hands
+            test -x /src/frametop/hands/rec/build/ft-handpanel
+            /src/frametop/hands/models.sh check
            test -x /src/frametop/pack/build/distrobox/install
            /src/frametop/gaze/tracker/build/venv/bin/python -c "import numpy, cv2"
            /src/frametop/voice/build/venv/bin/python -c "import numpy, sentencepiece, sherpa_onnx"
-            python3 -c "import PySide6"'
+            python3 -c "import PySide6, huggingface_hub"
+            command -v zstd >/dev/null'

      # The FrameDrop manifest names the zip's URL in this repo's release (a fork's, in a fork).
      - name: Frametop.zip
@@ -7,6 +7,7 @@ Frametop is a desktop for the Steam Frame that runs on the headset itself, with
 - **Your other computers' monitors.** Stream a Windows PC's or a Mac's monitors, or a virtual monitor at any size, as screens of their own, with the same controls. See [Remote displays](#remote-displays).
 - **Profiles.** Save where your screens are, which ones show, and which apps are open where. Switch to a profile from Display Settings, a key, or its launcher entry, or start the desktop in one.
 - **One mouse for all of SteamVR.** A Bluetooth mouse drives a small dot anchored in the room. It works the screens, the dashboard, Steam, and overlays, and hands the laser back when you pick up a controller.
+- **Your hands (experimental).** Hand tracking from the headset's own cameras: poke a screen to click, pinch and grab screens to move, resize, and throw them, and see your hands over the screens.
 - **Look and click (experimental).** In gaze mode the pointer goes where you look. Meta+J and Meta+K, or the mouse buttons, click and fine-tune, and each correction teaches the tracker. Calibration runs in the headset.
 - **Made for long sessions.** The displays turn off when the headset isn't used, even on a stand that makes it seem worn. It stays awake on the charger, and you can reach the desktop remotely over VNC.

@@ -36,7 +37,7 @@ You need a Steam Frame with an internet connection, a keyboard (Bluetooth, or th

   The third and fourth choices, stable release and experimental release, download Frametop already built (`Frametop.zip`, about 1.1 GB, from the [releases](https://github.com/Frametop/frametop/releases)) and install it without compiling anything. `-s -- --release` picks the stable release without the question, and `-s -- --release --experimental` the experimental one.

-   The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks you five things along the way: whether to install voice dictation (yes by default; it downloads about 320 MB of speech models), gaze mode (experimental, yes by default), our own eye tracker for it (yes by default), and the Bluetooth fixes, then whether to restart SteamVR. The eye tracker and the Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
+   The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks you six things along the way: whether to install voice dictation (yes by default; it downloads about 320 MB of speech models), hand tracking (experimental, yes by default), gaze mode (experimental, yes by default), our own eye tracker for it (yes by default), and the Bluetooth fixes, then whether to restart SteamVR. Hand tracking, the eye tracker, and the Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.

 After the restart, Launch a program → Desktop opens the multi-screen desktop, with its screens arranged around where you're facing. Frametop Display Settings and Frametop Input Settings are in the desktop's application menu, under Settings. SteamOS's own single-screen desktop is still there, as Native Desktop in the same list.

@@ -144,6 +145,21 @@ In gaze mode the pointer goes where you look, and the mouse or the keyboard does

 By default the mouse only corrects: while the gaze has the pointer, moving the mouse does nothing until you hold a button. Each correction before a click teaches the tracker where it was off. A correction past the learning limit (55 degrees by default, about half of what you can see) starts a quick check instead. Calibrate, on the Gaze page, runs a full calibration in a panel in front of you: look at each dot and click. Check headset fit shows how well the eye tracker sees your eyes. [gaze/README.md](gaze/README.md) has the details.

+### Hand tracking (experimental)
+
+Hand tracking uses the headset's own cameras, with no controllers. The installer offers it (or run `hands/run.sh install` later; it needs `sudo` once, to let its camera broker read the cameras). Installed, it runs while the Frametop desktop does, and parks while Frametop is paused for a VR game. It uses a lot of the headset's CPU.
+
+| Do this | To get this |
+| --- | --- |
+| Poke a screen with your index finger | A click there |
+| Pinch a screen's bar, edge, or corner, and move | Moves or resizes the screen |
+| Hold a screen like a card, or in a fist | Carries it; turn your wrist to turn it. Flick it away, or let go fast to throw it |
+| Hold both hands on a screen and pull them apart | Stretches it |
+| Hold an open hand out to a far screen, then make a fist and pull it back ("grab that") | Brings it to arm's reach |
+| Turn a hand palm up toward your face, look at it, and pinch the ring that shows | Opens the Steam menu |
+
+The Hands page of Frametop Input Settings sets when it runs (with Frametop, always with SteamVR, or off) and switches each hand interaction on or off, live. `ft-handsctl status` shows how it's doing. [hands/README.md](hands/README.md) has the details.
+
 ### Leave the headset on a stand and reach it remotely

 To keep the Frame on and connected while you're not wearing it, for SSH, remote desktop, or anything else running on it, open the Power tab in Frametop Display Settings:
@@ -181,9 +197,19 @@ cd ~/frametop && scripts/report.sh

 From a release, start in `~/.local/share/frametop/releases/current` instead of `~/frametop`.

-This writes `frametop-report-<date>.txt` with version numbers, service states, settings, Frametop's keyboard and the Steam menu, and recent logs. Bluetooth addresses and the headset's serial number are masked.
+This writes `frametop-report-<date>.txt` with version numbers, settings, service states, CPU, memory and heat, and the state and recent logs of every part: the desktop (Frametop's keyboard, the Steam menu, the screens), gaze, hand tracking (its models, cameras, and how often it sees each hand), and voice (its models and the microphone). It takes up to half a minute, because it starts the gaze service for a moment to see whether the eye tracker sends. Bluetooth addresses and the headset's serial number are masked, and what you said to Frametop Voice is left out.

-If the problem is something you can make happen, like a window that won't drag or a keyboard that doesn't open, run `scripts/report.sh --watch` instead. After the usual report it records for 60 seconds (`--watch 120` for longer) while you make it happen in the headset. It notes when the Steam menu opens and closes, which laser drags what, where typing goes, and when Frametop's keyboard opens or why it doesn't. It takes up to half a minute, because it also checks gaze mode: it starts the gaze service for a moment to see whether the eye tracker sends. If gaze or its calibration doesn't work, run it while you wear the headset. `scripts/gaze-report.py` prints only the gaze part, with what looks wrong first. Then [open an issue](https://github.com/Frametop/frametop/issues), describe what you did, what you expected, and what happened, and attach the file. Quick questions can go to [Discord](https://discord.gg/W3X9f7z3Bc) instead.
+To report on one part only, name it: `--system`, `--desktop`, `--gaze`, `--hands` or `--voice` (several are fine, for example `scripts/report.sh --hands --voice`).
+
+If the problem is something you can make happen, like a window that won't drag, a pinch that doesn't register, or dictation that types nothing, add `--watch`. After the report it records for 60 seconds (`--watch 120` for longer) while you make it happen in the headset, and keeps the logs from that time:
+
+- desktop: when the Steam menu opens and closes, which laser drags what, where typing goes, and when Frametop's keyboard opens or why it doesn't
+- gaze: which eye tracker the gaze service uses, and whether its samples come in
+- hands: each second, how many frames hand tracking made, how often it saw each hand, and how close the fingertips came
+- voice: what Frametop Voice did (the number of words, not the words) and the microphone's level each second (no sound is kept)
+- system: every 5 seconds, the CPU's load, clocks and heat, and the busiest programs
+
+If gaze or its calibration doesn't work, run the report while you wear the headset. `scripts/gaze-report.py` prints only the gaze part, with what looks wrong first. Then [open an issue](https://github.com/Frametop/frametop/issues), describe what you did, what you expected, and what happened, and attach the file. Quick questions can go to [Discord](https://discord.gg/W3X9f7z3Bc) instead.

 ## Update

@@ -234,7 +260,7 @@ A Plasma session runs nested inside ft-screens (`screens/`), a small Wayland com
 | `power/` | ft-powerd: turns the displays off while the headset isn't used. |
 | `voice/` | Frametop Voice: dictation and wake words, recognised on the headset; the input relay types the text. See [voice/README.md](voice/README.md). |
 | `gaze/` | Gaze mode (experimental): the gaze service, its calibration panel, our own eye tracker, and the gaze probe. See [gaze/README.md](gaze/README.md). |
-| `hands/` | Hand tracking (experimental, deferred: the installer doesn't offer it). See [hands/README.md](hands/README.md). |
+| `hands/` | Hand tracking (experimental): the camera broker and the tracker. See [hands/README.md](hands/README.md). |
 | `display-settings/`, `input-settings/` | The two settings apps (Kirigami, Python). |
 | `remote/` | Frametop Remote Access, the app that turns remote desktop over VNC on and off. |
 | `remote-displays/` | Frametop Remote Displays, the app that adds other computers' monitors as screens. |
@@ -312,15 +312,15 @@ In gaze mode the 3D mouse's pointer goes where you look, and the mouse or the ke

 [gaze/README.md](../gaze/README.md) has the details, our own eye tracker, and the gaze probe, a development tool.

-## Hand tracking (experimental, deferred)
+## Hand tracking (experimental)

-Deferred: it costs a lot of the headset's CPU and needs more work, so `install.sh` doesn't offer it. It still builds and runs, installed by hand, for working on it.
+It costs a lot of the headset's CPU. `install.sh` offers it (yes by default, `--no-hands` skips it, and it needs `sudo` for ft-camd's file capabilities), and a release brings `ft-camd` and `ft-hands` built in its image. Installed, it runs while the Frametop desktop does (`HANDS_MODE=frametop`).

 Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen (a round hole per hand, `HANDS_MATTE=circle`; `mesh` and `capsules` are the older shapes). ft-handview (`frametop-handview.service`) draws your hands as a 3D hand model posed by the tracking: over games and the void with Room View off, and over the hole with it on (`HANDS_VIEW_ROOMVIEW=1`, the default). `HANDS_VIEW_MODEL` picks the model: `asset` (the default), `babylon`, `mrtk3`, `godot`, `mrtk2`, `robot` (a robot hand built in code from the tracked skeleton: light shells, dark joints, cyan accents), or `capsules` for the older tubes; `HANDS_VIEW_STYLE=ghost|solid` the look. The models, in `screens/assets/hands` with their licences: the WebXR Input Profiles' generic hand (`asset`; MIT, Copyright (c) 2019 Amazon), Babylon.js's hand meshes by Patrick Ryan (`babylon`; CC BY 4.0, https://github.com/BabylonJS/Assets), MRTK3's OpenXR hand (`mrtk3`; BSD-3-Clause, Copyright (c) 2025, Mixed Reality Toolkit Contributors), Godot XR Tools' hands by DigitalN8m4r3 (`godot`; CC0 1.0) and MRTK2's rigged hand (`mrtk2`; MIT, Copyright (c) Microsoft Corporation). The same tracker detects pinches and grips, and with `POINTER_HANDS=1` in `~/.config/frametop.conf` they work the pointer. In gaze mode your eyes aim and your hands click, Vision Pro style: a pinch clicks where you were looking as it began, and the hand ray stays off (`POINTER_GAZE_HAND_RAY=1` brings it back beside the gaze). Pinch, hold and move the hand a few centimetres to nudge the pointer slowly onto the target, then let go to click there; the correction teaches the gaze tracker, as the mouse's do. Pinch and hold still for half a second to press and drag with the hand. A middle pinch right-clicks the same way. Hold a pinch with one hand and pinch with the other to hold both buttons, left and right, each until its pinch opens. Without gaze mode a pinch is a press like the mouse's button, so a held pinch drags. A grip (closing the hand) presses and drags. To install it: `hands/run.sh install`. With `POINTER_HANDS=1` and gaze mode off, a bare hand raised and pointing also shows a ray that drives the pointer like a controller's laser (a pinch clicks, a middle pinch right-clicks or scrolls, reaching out and in pushes and pulls a carried panel), a controller used recently takes over its own hand only, and during a paused VR game hand tracking is parked rather than stopped, waking when the dashboard opens. Details: [hands/README.md](../hands/README.md), "Pinches and grips in the pointer". When tracking goes wrong, `ft-handsctl flag [note]` (or a voice command through frame-voice, "flag that") saves the last 12 seconds of hand crops and what the tracker made of them for the offline tools: ft-hands keeps them in RAM (`HANDS_FLAG_BUFFER=1`, about 50 MB), the drawn hands pulse cyan when a flag is taken (`HANDS_FLAG_FLASH`, `HANDS_FLAG_TONE`), and flags go to `HANDS_FLAG_DIR`. See [hands/README.md](../hands/README.md), "Flags".

 - `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`. The Hand Recorder's installer (`hands/rec/install.sh`) sets them the same way. `hands/run.sh uncaps` takes them back while neither the services nor the Hand Recorder is installed. `hands/run.sh uninstall` and `hands/rec/install.sh uninstall` run it after removing their own part, and `uninstall.sh` always takes them back.
 - `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches and grips, read by the pointer helper) next to the ring.
- `HANDS_MODE` says when they run: `off` (the default) only after `ft-handsctl on`, until `ft-handsctl off` or SteamVR stops; `frametop` while the Frametop desktop runs; `always` whenever SteamVR runs, in games too. Frametop Input Settings' Hands page sets it and applies it at once (`hands/handsmode.py`), next to the hand cutouts' margin (`HANDS_MATTE_MARGIN`, live). The install links `ft-handsctl` into `~/.local/bin`. `ft-handsctl status` and `ft-handsctl log` (or `hands/run.sh status` and `log`) show how they're doing, `ft-handsctl cutouts on|off` turns just the cutouts off, and `ft-handsctl gestures` shows pinches and grips live.
+- `HANDS_MODE` says when they run: `off` (the default until install.sh installs hand tracking, which sets `frametop`) only after `ft-handsctl on`, until `ft-handsctl off` or SteamVR stops; `frametop` while the Frametop desktop runs; `always` whenever SteamVR runs, in games too. Frametop Input Settings' Hands page sets it and applies it at once (`hands/handsmode.py`), next to the hand cutouts' margin (`HANDS_MATTE_MARGIN`, live). The install links `ft-handsctl` into `~/.local/bin`. `ft-handsctl status` and `ft-handsctl log` (or `hands/run.sh status` and `log`) show how they're doing, `ft-handsctl cutouts on|off` turns just the cutouts off, and `ft-handsctl gestures` shows pinches and grips live.
 - Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (`auto`, the default: ft-hands tells from the hands when some SteamVR restart has swapped the side cameras' names, and fixes them; `0` or `1` force them, and `hands/tools/check_sides.py --ring` tells which is right), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, typing's `POINTER_PINCH_TYPING`, `POINTER_TYPING_RAISE`, `POINTER_TYPING_BELOW`, `POINTER_TYPING_POSTURE`, `POINTER_TYPING_RECENT`, the gaze pinch's `POINTER_GAZE_PINCH`, `POINTER_GAZE_HAND_RAY`, `POINTER_GAZE_PINCH_DEADZONE`, and `POINTER_GAZE_PINCH_GAIN`, the hand ray's `POINTER_HAND_RAY`, `POINTER_RAY_DEADZONE`, `POINTER_RAY_PUSH`, `POINTER_RAY_SCROLL`, `POINTER_RAY_RELEASE`, `POINTER_RAY_GRACE`, its smoothing and taps (`POINTER_RAY_SMOOTH_MINCUTOFF`, `POINTER_RAY_SMOOTH_BETA`, `POINTER_RAY_SMOOTH_HOLD`, `POINTER_RAY_STEADY`, `POINTER_RAY_TAP`, `POINTER_RAY_TAP_DEADZONE`), its body and hand models (`POINTER_RAY_TORSO_WRISTS`, `POINTER_RAY_HAND_WEIGHT`, `POINTER_RAY_HAND_PITCH`, `POINTER_RAY_HAND_YAW`, `POINTER_RAY_ANCHOR`), and ft-screens' near reach (`HANDS_NEAR_REACH`, 0.075 m), pokes (`HANDS_POKE_ANGLE`, `HANDS_POKE_TRAVEL`, `HANDS_POKE_REST_BELOW`, `HANDS_POKE_REST_NEAR`) and near grabs (the border band `HANDS_GRAB_BAND`, a pinch on the content clicking `HANDS_PINCH_CLICK`, the card pinch `HANDS_CARD_PINCH`, the fist grab `HANDS_FIST_GRAB` and its reach `HANDS_FIST_REACH`, a second hand joining a held panel `HANDS_JOIN_REACH`, two hands turning a panel on every axis `HANDS_STRETCH_TURN`, the round buttons' hit area for hands `HANDS_BUTTON_REACH` and pinches pressing them `HANDS_PINCH_BUTTONS`, `HANDS_GRAB_SMOOTH_MINCUTOFF`, `HANDS_GRAB_SMOOTH_BETA`, `HANDS_GRAB_DEADZONE`, the wrist turning a carried panel: `HANDS_GRAB_ROTATE`, `HANDS_GRAB_ROTATE_DEADZONE`, holding through tracking loss `HANDS_HOLD_LOST_MS`, ignoring tracking no hand makes `HANDS_TRACK_GATE`), throws (the flick `HANDS_FLICK`, `HANDS_FLICK_SPEED`, `HANDS_FLICK_OPENING`, a carry let go fast `HANDS_THROW`, `HANDS_THROW_SPEED`, the push `HANDS_PUSH`, `HANDS_PUSH_SPEED`, where they go, `HANDS_THROW_DISTANCE`, and at what width, `HANDS_SEND_AWAY_WIDTH`), its "grab that" gesture (`HANDS_GRAB_THAT`, `HANDS_GRAB_THAT_DISTANCE`, `HANDS_GRAB_THAT_WIDTH`, which panels: `HANDS_GRAB_THAT_FROM`, `HANDS_GRAB_THAT_MARGIN`, `HANDS_GRAB_THAT_MARGIN_DEG`, `HANDS_GRAB_THAT_FACING`), the soft white halo round a gesture's target panel (`HANDS_TARGET_GLOW`), and the soft glow round the corner, edge or button a hand is at (`HANDS_EDGE_GLOW`). The example config explains each.

 Details, options, and the recording and replay tools are in [hands/README.md](../hands/README.md).
@@ -1,6 +1,6 @@
 # Install with FrameDrop (proof of concept)

-[FrameDrop](https://framedropvr.com) is a Windows app that sideloads onto a Steam Frame: it copies a build to the headset and adds it to the Steam library. Issue #25 asks for an "Install with FrameDrop" button. Frametop isn't an app FrameDrop can copy over as is: it installs user services, a SteamVR driver, and a container, and two optional parts need sudo. So FrameDrop installs a small installer instead. Playing "Frametop" from the library opens a window that asks what to install, and your password for the parts that need it, then installs and shows its progress. A release's Frametop.zip (about 1.1 GB, built by CI on a tag, see [pack/README.md](../pack/README.md), Releases) carries Frametop built, as an image, and installs it with `install-release.sh`: nothing compiles on the headset and nothing else downloads. The same zip works unpacked on the headset. A test zip (a few KB) clones Frametop with `get.sh` instead.
+[FrameDrop](https://framedropvr.com) is a Windows app that sideloads onto a Steam Frame: it copies a build to the headset and adds it to the Steam library. Issue #25 asks for an "Install with FrameDrop" button. Frametop isn't an app FrameDrop can copy over as is: it installs user services, a SteamVR driver, and a container, and four optional parts need sudo. So FrameDrop installs a small installer instead. Playing "Frametop" from the library opens a window that asks what to install, and your password for the parts that need it, then installs and shows its progress. A release's Frametop.zip (about 1.1 GB, built by CI on a tag, see [pack/README.md](../pack/README.md), Releases) carries Frametop built, as an image, and installs it with `install-release.sh`: nothing compiles on the headset and nothing else downloads. The same zip works unpacked on the headset. A test zip (a few KB) clones Frametop with `get.sh` instead.

 Nothing here is published yet: no release, no button.

@@ -26,7 +26,7 @@ It uses Valve's SteamOS Devkit path: pair once with the headset's devkit service

 1. In the container, it starts itself again on the host with `flatpak-spawn --host`.
 2. It clears Steam's preload and library paths, and opens `installer/progress.py` (GTK 4 and libadwaita).
-3. The window asks which optional parts to install: our own eye tracker (on by default) and the Bluetooth fixes. Both need sudo, so it asks for your SteamOS password and checks it with `sudo -v`. If your user has no password (SteamOS starts without one), it says how to set one and leaves both out.
+3. The window asks which optional parts to install: voice dictation (on by default), hand tracking (on by default), the Hand Recorder (on by default; a release's zip only, since a clone builds it with `hands/rec/install.sh`), our own eye tracker (on by default), and the Bluetooth fixes. All but voice dictation need sudo, so it asks for your SteamOS password and checks it with `sudo -v`. If your user has no password (SteamOS starts without one), it says how to set one and leaves those out.
 4. It runs the zip's `install-release.sh --yes` (a test zip: `get.sh --yes`) in a transient user service, `frametop-framedrop-install`. The service is used because Steam ends the title's whole process tree when it's quit, and starts it with an OOM score of 900. Opened from a VR desktop (the zip unpacked by hand), it still uses the user's real bus and runtime folder for the service.
 5. It follows the service's log, shows the steps as a progress bar, and reports the result. Closing it leaves the install running. Playing the title again reattaches.

@@ -13,7 +13,7 @@ The install runs in a user service of its own (UNIT): Steam ends the title's who
 tree when it's quit, and starts it with a high OOM score. Closing the window doesn't stop the
 install; playing the title again reattaches to it.

-Our eye tracker and the Bluetooth fixes need sudo. The window asks for your password on the
+Hand tracking, the Hand Recorder, our eye tracker and the Bluetooth fixes need sudo. The window asks for your password on the
 headset, checks it with sudo, and keeps it in this process's memory until the install ends.
 sudo in the install gets it through askpass (SUDO_ASKPASS), from a socket in a folder only you
 can open, and the window answers only programs in the install's service. It's never written to
@@ -101,7 +101,7 @@ def in_unit(pid):
        return False


-def install_command(eye_tracker, bluetooth, voice=True):
+def install_command(eye_tracker, bluetooth, voice=True, hands=True, hand_recorder=True):
    if RELEASE:
        args = [str(HERE / "install-release.sh"), "--yes"]
        if DRY_RUN:
@@ -112,6 +112,10 @@ def install_command(eye_tracker, bluetooth, voice=True):
            args += ["--clone-only", "--dir", str(STATE / "dry-run")]
    if not voice:
        args.append("--no-voice")
+    if not hands:
+        args.append("--no-hands")
+    if RELEASE and not hand_recorder:   # a clone (get.sh) doesn't offer it: hands/rec/install.sh builds it
+        args.append("--no-hand-recorder")
    if not eye_tracker:
        args.append("--no-eye-tracker")
    if bluetooth:
@@ -304,10 +308,16 @@ class Window(Adw.ApplicationWindow):
    def show_choices(self):
        what = f"Frametop {release_version()}" if RELEASE else "Frametop from GitHub"
        self.detail.set_label(f"This installs {what}: the multi-screen desktop, the 3D mouse, gaze "
-                              "mode, and Frametop's settings apps. Optional: voice dictation, and two "
+                              "mode, and Frametop's settings apps. Optional: voice dictation, and "
                              "parts that need your SteamOS password (sudo):")
        self.voice = Gtk.CheckButton(label="Voice dictation: a Right Ctrl tap or holding the left d-pad down, and "
                                           "what you say is typed (downloads about 320 MB)", active=True)
+        self.hands = Gtk.CheckButton(label="Hand tracking (experimental): poke, pinch and grab the screens with "
+                                           "your hands. It uses a lot of the headset's CPU", active=True)
+        # A release brings it built; a clone builds it with hands/rec/install.sh, so isn't offered it
+        self.recorder = Gtk.CheckButton(label="Hand Recorder: record your hands for Frametop's open hand "
+                                              "dataset, which trains hand tracking (nothing is uploaded "
+                                              "until you press Upload)", active=True, visible=RELEASE)
        self.eye = Gtk.CheckButton(label="Our own eye tracker for gaze mode (more accurate than SteamVR's)",
                                   active=True)
        self.bt = Gtk.CheckButton(label="Bluetooth fixes (LE mice and keyboards, like the Swiftpoint Z3, "
@@ -323,27 +333,28 @@ class Window(Adw.ApplicationWindow):
        self.go.connect("clicked", self.install)
        self.pw.connect("activate", self.install)
        pw_row, keypad = keypad_row(self.pw)
-        self.choices = [self.voice, self.eye, self.bt, pw_row, keypad, pw_note, self.error, self.go]
+        self.choices = [self.voice, self.hands, self.recorder, self.eye, self.bt, pw_row, keypad, pw_note,
+                        self.error, self.go]
        if not password_set():
-            for c in (self.eye, self.bt):
+            for c in (self.hands, self.recorder, self.eye, self.bt):
                c.set_active(False)
                c.set_sensitive(False)
            pw_row.set_visible(False)
            keypad.set_visible(False)
-            pw_note.set_label("Your user has no password, so sudo can't run and these two can't be "
+            pw_note.set_label("Your user has no password, so sudo can't run and these can't be "
                              "installed from here. To set one, run passwd in Konsole; then play "
                              "Frametop again, or install them later from a terminal "
-                              "(gaze/tracker/install.sh, setup/bluetooth/install.sh).")
-        for c in (self.eye, self.bt):
-            c.connect("toggled", lambda *_: pw_row.set_sensitive(self.eye.get_active() or self.bt.get_active()))
+                              "(hands/run.sh install, hands/rec/install.sh, gaze/tracker/install.sh, "
+                              "setup/bluetooth/install.sh).")
+        for c in (self.hands, self.recorder, self.eye, self.bt):
+            c.connect("toggled", lambda *_: pw_row.set_sensitive(self.needs_password()))
        for w in self.choices:
            self.box.append(w)

    def install(self, *_):
        if self.checking:
            return
-        needs = self.eye.get_active() or self.bt.get_active()
-        if not needs:
+        if not self.needs_password():
            return self.begin(None)
        text = self.pw.get_text()
        if not text:
@@ -370,12 +381,17 @@ class Window(Adw.ApplicationWindow):

        threading.Thread(target=check, daemon=True).start()

+    def needs_password(self):
+        return (self.hands.get_active() or (RELEASE and self.recorder.get_active()) or self.eye.get_active()
+                or self.bt.get_active())
+
    def say(self, text):
        self.error.set_label(text)
        self.error.set_visible(True)

    def begin(self, pw):
-        args = install_command(self.eye.get_active(), self.bt.get_active(), self.voice.get_active())
+        args = install_command(self.eye.get_active(), self.bt.get_active(), self.voice.get_active(),
+                               self.hands.get_active(), self.recorder.get_active())
        if pw is not None:
            try:
                self.askpass = Askpass(self.ask_again)
@@ -23,17 +23,17 @@
 #   --version V     with --release: that release (tag vV), not the newest
 #   --zip FILE|URL  with --release: this Frametop.zip instead of GitHub's newest
 #   --any-steamos   with --release: install even on a SteamOS build the release breaks on
-#   --yes, --no-voice, --no-eye-tracker, --no-bluetooth, --bluetooth   passed to install.sh (--yes also
+#   --yes, --no-voice, --no-hands, --no-eye-tracker, --no-bluetooth, --bluetooth   passed to install.sh (--yes also
 #                   answers this script's questions: the version already there, or stable)
 set -euo pipefail

 usage() {
  cat <<'EOF'
 usage: get.sh [--stable | --experimental | --branch NAME] [--dir DIR] [--clone-only] [--yes]
-              [--no-voice] [--no-eye-tracker] [--no-bluetooth | --bluetooth]
+              [--no-voice] [--no-hands] [--no-eye-tracker] [--no-bluetooth | --bluetooth]
       get.sh --release [--stable | --experimental | --version V | --zip FILE|URL]
-              [--any-steamos] [--dir DIR] [--clone-only] [--yes] [--no-voice] [--no-eye-tracker]
-              [--no-bluetooth | --bluetooth]
+              [--any-steamos] [--dir DIR] [--clone-only] [--yes] [--no-voice] [--no-hands]
+              [--no-eye-tracker] [--no-bluetooth | --bluetooth]
 piped: curl -fsSL https://frametop.github.io/frametop/get.sh | bash -s -- [options]
 EOF
 }
@@ -129,7 +129,7 @@ main() {
      --version) want=${2:?--version needs a version}; shift ;;
      --any-steamos) any=1 ;;
      --yes) yes=1; pass+=("$1") ;;
-      --no-bluetooth|--bluetooth|--no-eye-tracker|--no-voice) pass+=("$1") ;;
+      --no-bluetooth|--bluetooth|--no-eye-tracker|--no-voice|--no-hands) pass+=("$1") ;;
      -h|--help) usage; return 0 ;;
      *) echo "unknown option: $1" >&2; usage >&2; return 2 ;;
    esac
@@ -12,6 +12,8 @@ Join the [Frametop Discord](https://discord.gg/W3X9f7z3Bc) for questions and hel

 For now the recorder runs inside Frametop's desktop, so these steps install Frametop first. A standalone recorder that runs from the SteamVR dashboard without Frametop is planned.

+Installing a Frametop release (Frametop.zip, FrameDrop, or `get.sh --release`) instead? Its installer offers the Hand Recorder, built, so there's nothing to compile: leave Hand Recorder ticked (or answer yes) and skip step 2. To add it to a release later, run `~/.local/share/frametop/releases/current/hands/rec/install.sh`.
+
 ## Before you start

 - You must be 18 or older, and for now you can't take part if you live in Illinois, Texas or Washington (USA). The [consent text](https://github.com/Frametop/frametop/blob/main/hands/rec/CONSENT.md) explains what's recorded and what you agree to. The recorder shows it again before your first session.
@@ -1,6 +1,10 @@
 # Model sources that tools/convert_models.py downloads; the converted ncnn models are kept
 models/onnx/
 models/*.task
+# the fine-tuned models hands/models.sh fetches (their own licence, CC BY-NC 4.0: not in the repo)
+models/student/
+# hands/models.sh update's download folder, lock and last check
+models/.student-*
 # frame-job settings (where replays run, and the lab's capture folder): personal, not shipped
 .frame-job
 dumps/
@@ -1,6 +1,6 @@
-# Hands (experimental, deferred)
+# Hands (experimental)

-Hand tracking from the headset's own cameras. It's deferred: it costs a lot of the headset's CPU and needs more work, so `install.sh` doesn't offer it and the README doesn't list it. It still builds and runs, installed by hand (below), for working on it.
+Hand tracking from the headset's own cameras. It's experimental and costs a lot of the headset's CPU. `install.sh` offers it (yes by default; `--no-hands` skips it), a release brings it built, and installed, it runs while the Frametop desktop does (`HANDS_MODE=frametop`). `hands/run.sh install` (below) installs it later.

 It serves two things in Frametop:

@@ -14,9 +14,9 @@ Two programs, each a user service that stops when SteamVR does:
 - `ft-camd` (`camd/`, C) borrows XRService's camera buffers and publishes the four IR tracking cameras' frames to a shared-memory ring. It runs on the host.
 - `ft-hands` (`track/`, C++) finds hands in those frames with MediaPipe's palm and landmark models on ncnn, triangulates them, and publishes them. It runs in the dev container.

-`hands/run.sh install` builds them, gives ft-camd its capabilities, installs both services, and links `hands/ft-handsctl` into `~/.local/bin`. `install.sh` doesn't install it. When they run is `HANDS_MODE` in `~/.config/frametop.conf`, set on the Hands page of Frametop Input Settings or with `ft-handsctl mode`; `hands/handsmode.py` does the starting and stopping, for these two and the hand view (`frametop-handview.service`, when it's installed):
+`hands/run.sh install` builds them (a release brings them built), gives ft-camd its capabilities, installs both services, and links `hands/ft-handsctl` into `~/.local/bin`. `install.sh` runs it when you choose hand tracking, then sets `HANDS_MODE=frametop` unless it's `always`. When they run is `HANDS_MODE` in `~/.config/frametop.conf`, set on the Hands page of Frametop Input Settings or with `ft-handsctl mode`; `hands/handsmode.py` does the starting and stopping, for these two and the hand view (`frametop-handview.service`, when it's installed):

- `off` (the default): nothing starts them. `ft-handsctl on` starts hand tracking and `ft-handsctl off` stops it, as before; it stops with SteamVR.
+- `off` (the default before hand tracking is installed): nothing starts them. `ft-handsctl on` starts hand tracking and `ft-handsctl off` stops it, as before; it stops with SteamVR.
 - `frametop`: they run while the Frametop desktop does. The session (`session/frametop-session.sh`, screens backend) starts them once ft-screens is up and stops them as it ends.
 - `always`: they're wanted by `steamvr.service` (`systemctl --user add-wants`), so they run whenever SteamVR does, in games and the SteamVR dashboard too, with or without the desktop. Pausing Frametop for a VR game leaves them running and unparked.

@@ -50,7 +50,8 @@ Settings in `~/.config/frametop.conf` (`FT_<name>` in the environment overrides
 - `HANDS_SWAP_SIDES=auto` (the default): ft-hands tells from the hands which side camera is which, and corrects ft-camd's names when they're backwards (see "Which camera is which" below). `1` forces them exchanged and `0` forces ft-camd's names; ft-hands still checks, and if the hands disagree it logs a warning and publishes the hands' answer as the truth (`sides.json`), so recordings are labelled right. The example config said `0` until 2026-10-05; `scripts/conf-migrate.sh` (run by `install.sh` and `hands/rec/install.sh`) turns that untouched line into `auto`.
 - `HANDS_CPUS=5,6,7`: the CPUs the model threads run on (below).
 - `HANDS_CAMERAS` (`auto`), `HANDS_BRIGHT` (`all`), `HANDS_BRIGHT_ON` (40), `HANDS_BRIGHT_OFF` (25): which cameras ft-hands tracks with, as `--cams`, `--bright`, `--bright-on` and `--bright-off` (see ft-hands). `HANDS_CAMERAS=mono` also keeps ft-camd off the colour cameras.
- `HANDS_MODELS` (unset: `hands/models/ncnn`, the stock MediaPipe models): a folder holding `palm.ncnn.*` and `hand.ncnn.*`, as `--models`. Use it to run fine-tuned models, such as the ones trained on the hand dataset, without passing options to every launcher. Those folders usually lack the `*-int8` files, so `--int8` won't load them.
+- `HANDS_MODELS` (unset: `hands/models/student`, the fine-tuned models, when `hands/models.sh` fetched them, else `hands/models/ncnn`, the stock MediaPipe models): a folder holding `palm.ncnn.*` and `hand.ncnn.*`, as `--models`. Use it to try other models without passing options to every launcher. The fine-tuned folders lack the `*-int8` files, so `--int8` without `--models` uses the stock ones.
+- `HANDS_MODELS_UPDATE=1`: when hand tracking starts, fetch the fine-tuned models GitHub's branch pins if they changed, without a new release (`models.sh` below). `0` keeps the ones it came with.
 - `HANDS_COLOR_LEFT` (`color_video0`), `HANDS_COLOR_CROP` (`subtract`): how the colour module's calibration maps onto its images, as `--color-left` and `--color-crop`.
 - `HANDS_SKELETON=1`: fit a skeleton with your bone lengths and publish it (see "Skeleton" below); `0` publishes the filtered landmarks as before. As `--skeleton`.
 - `HANDS_PINCH_DIP=0`: `1` also starts a pinch on a quick dip of the thumb-index distance that never reaches 2 cm (see the pinch under "Gestures" below). As `--pinch-dip`.
@@ -214,7 +215,7 @@ What it measured, `ft-handreplay` on three recordings and three takes of the 202

 ft-hands detects a pinch per hand (`track/pinch.h`) and publishes it to the gestures file. The layout, and how to read it without missing quick taps, is in `include/fh_gestures.h`.

- A pinch begins when the thumb and index tips come within `--pinch-begin` (default 0.020 m). It ends when they open past `--pinch-end` (0.035 m) for 2 processed frames in a row, or when the hand stays lost for 0.25 s (flagged lost).
+- A pinch begins when the thumb and index tips come within `--pinch-begin` (default 0.020 m). It ends when they open past `--pinch-end` (0.045 m) for 2 processed frames in a row, or when the hand stays lost for 0.25 s (flagged lost).
 - A pinch held 0.3 s or longer is a drag, and holds on harder: it ends only once the tips are 5 cm apart (for the same 2 frames), or 0.12 s after they first went past 3.5 cm unless they closed back under 3 cm meanwhile. In the 2026-10-09 headset test drags ended with the tips 3.5-4.0 cm apart, a thumb hidden for a frame or fingers loosened a little mid-drag. On the lit recording's replayed distances, pinches held 0.3 s or more (typing among them) ended 33-134 ms later than before, 200 at most, and one of 25 ran into the next pinch's begin. A grip ends a pinch (as lost) only within 0.3 s of its begin, when closing the hand passes through a pinch; after that no grip begins on that hand, so curling the other fingers around a held pinch doesn't drop it (3 of 28 pinches in the same test). `tests/pinch_test.cpp` (`make check`).
 - The distance comes from MediaPipe's world landmarks: the model's own 3D hand pose, averaged over the hand's views, at the user's hand size. `--pinch-triangulated` uses the triangulated tips instead. On two recordings without deliberate pinches, the world landmarks came under 2 cm in 0.2-1% of frames, against 3.3-4.5% for the triangulated tips. In the dim recording, typing still gave 2 pinches a minute (see the next point).
 - `--pinch-palm-down MAX` holds back pinches begun with the palm facing down (MAX is the palm normal's share of the head's up axis). The default, 1, turns it off. A close held back that way has to open again before a pinch can begin. Typing curls the thumb onto the index: in the lit recording of 2026-09-30, typing on a keyboard in the lap began 23 pinches in about 2 minutes, all with the palm facing down (0.69-1.00), while the 26 deliberate ones read 0.00-0.50. But in the headset, deliberate pinches with the hand raised in front read 0.90-0.99 too, so the limit is off. Typing is caught by the pointer helper instead: the input relay tells it when you press a key, and the hands on the keys are held back (`POINTER_PINCH_TYPING` and the settings after it, below).
@@ -298,6 +299,7 @@ Python, with NumPy and OpenCV. `setup/dev-container.sh` doesn't install them, be
 - `tools/depth_report.py DEPTH`: the depth measures above.
 - `tools/cut_sets.py REC OUT [--sets N | --at I,J,...]`: copies a few frame sets (by default 8, spread evenly) out of a recording into a small one, to look at or check elsewhere without moving gigabytes. Plain Python, so it also runs on the Frame's host.
 - `tools/convert_models.py`: how `models/ncnn` was made from the OpenCV Zoo ONNX ports of MediaPipe's models (see `models/NOTICE`).
+- `models.sh`: fetches the fine-tuned models into `models/student` (`hands/build.sh` runs it, so source installs and the release image both get them): MediaPipe's two models retrained on the Frame's infrared cameras with the [Frametop Hands](https://huggingface.co/datasets/DeeJanuz/frametop-hands) dataset, from a fixed commit of [DeeJanuz/frametop-hands-models](https://huggingface.co/DeeJanuz/frametop-hands-models), each file checked against its SHA-256. They're licensed CC BY-NC 4.0 (non-commercial), not MIT, so they aren't in this repository; `models/student/LICENSE` and `README.md` come with them. `hands/models.pin` pins them: the commit, the version folder and every file's sum (frame-hands' `model-repo/pin.py` writes it); `hands/models.sh check` checks them against it. Installs don't need a release for a new version: when hand tracking starts, `frametop-hands.service` runs `hands/models.sh auto`, which reads `hands/models.pin` from GitHub (main for a stable release, experimental for an experimental one or a clone on that branch; at most every 15 minutes, within a minute) and, if it pins other models, fetches them into `models/.student-new`, checks every sum, and swaps that folder in whole. Offline, or anything not matching, it keeps the models it has. A pin only names the files above, and its `format` says which ft-hands it needs: raise it when a model needs a newer one, and installs without it keep theirs. `HANDS_MODELS_UPDATE=0` turns it off; `hands/models.sh update` checks now.

 To try the hand cutouts without restarting the desktop, `screens/build/ft-handtest [--distance m] [--width m] [--seconds s] [--matte circle|mesh|capsules] [--margin deg]` (built by `screens/build.sh`, run in the dev container, with hand tracking on) shows a test panel of its own, a light grid 1 m wide and 0.8 m ahead by default, and cuts your hands out of it the way ft-screens cuts them out of the screens. Every 2 s it prints the composite's cost.

@@ -1,11 +1,13 @@
 #!/usr/bin/env bash
 # Build hand tracking in the dev container on the Frame, into hands/build/: ft-camd and ft-hands,
 # and with --tools also ft-handreplay and ft-ringplay. The first build fetches ncnn and builds
-# it (a few minutes); NCNN=DIR, an ncnn install already on the Frame, skips that.
+# it (a few minutes); NCNN=DIR, an ncnn install already on the Frame, skips that. It also fetches
+# the fine-tuned hand models (hands/models.sh) into hands/models/student, which ft-hands prefers.
 # A rebuilt ft-camd has lost its capabilities: hands/run.sh install sets them again.
 set -euo pipefail
 root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
 targets=all
 [ "${1:-}" = --tools ] && targets="all tools"
 "$root/scripts/sync.sh" >/dev/null
+"$root/hands/models.sh" fetch
 exec "$root/scripts/frame.sh" -C hands "make -s ${NCNN:+NCNN=$NCNN} $targets && echo built \$(ls build/ft-* | tr '\n' ' ')"
@@ -10,7 +10,10 @@ PartOf=steamvr.service
 # In the container it was built in (scripts/in-box: dev for a clone, the release's own for a
 # release; it's built against Fedora's libraries). It reads ft-camd's ring and
 # writes /run/user/UID/frametop-hands/hands and gestures. Settings: HANDS_* in ~/.config/frametop.conf.
+# Before it starts, newer fine-tuned models if its branch pins them (hands/models.sh auto: at most
+# every 15 minutes, never more than a minute, offline or failing it starts with the ones it has).
 ExecStartPre=-/usr/bin/pkill -x ft-hands
+ExecStartPre=-/usr/bin/timeout 60 @REPO@/hands/models.sh auto
 ExecStart=@REPO@/scripts/in-box @REPO@/hands/build/ft-hands --status 60
 ExecStopPost=-/usr/bin/pkill -x ft-hands
 Restart=always
@@ -0,0 +1,16 @@
+# The fine-tuned hand models ft-hands uses by default (hands/models.sh): a version folder of the
+# Hugging Face repo DeeJanuz/frametop-hands-models at a fixed commit, and every file's sha256.
+# Installs read this file from GitHub (main or experimental, by their channel) when hand tracking
+# starts, so a new pin pushed there reaches them without a release. "format" is what ft-hands
+# needs to run them: raise it when a model needs a newer ft-hands, and older installs keep theirs.
+# Made by frame-hands' model-repo/pin.py.
+format 1
+rev 41afd78fbd4527456fc4cbd840ffc9f39fdc939a
+model v13b-p1
+file NOTICE 0e6b4fa8ea37665cb699abc36fcf1ee0ffaa63a78dfee788be6613dea6607f84
+file hand.ncnn.bin d6a6f00d845fb9ee490276d84b1e978da884361c372614546ed2b1b4d2f5c449
+file hand.ncnn.param 54e08f284ff6f121ff00f4a9c65afc38c0dbb22a32f4af44fc4e0fde6c4a1bbd
+file palm.ncnn.bin 9865f5fa3c6bc885cb0b50f201c4ac1bd83b2c1ad64af4e122efa3fc56d61909
+file palm.ncnn.param c61df2dde5b221a55c35c9693e530845a95ce2732949a9b3d3c583e3b2bae92e
+file LICENSE 41003d4a74749c0220e33dd415042164b5a1093ed401f36277234f772d22d3d0
+file README.md 29c6c5460e47412a9a37b22d6a7c8a9ed266a872bbcce93d3867f9e56cfe0ca1
@@ -0,0 +1,137 @@
+#!/usr/bin/env bash
+# Fetch the fine-tuned hand models into hands/models/student/ on the Frame (in the image build,
+# here: pack/Containerfile runs hands/build.sh, which runs this). hands/models.pin names them: a
+# fixed commit of the Hugging Face repo below, and every file's sha256. ft-hands uses them when
+# they're there (HANDS_MODELS overrides; hands/models/ncnn, the stock MediaPipe models, otherwise).
+#   palm.ncnn.*, hand.ncnn.*   MediaPipe's palm detector and hand landmark model, fine-tuned on the
+#                              Frametop hand dataset (DeeJanuz/frametop-hands)
+#   LICENSE, README.md         their licence (CC BY-NC 4.0, not Frametop's MIT) and model card
+# Usage: hands/models.sh [fetch]   fetch what's missing or doesn't match models.pin (the default)
+#        hands/models.sh check     check every file against the pin it was fetched by (exit 1 if not)
+#        hands/models.sh update    fetch the models the install's branch pins on GitHub now (main for
+#                                  a stable release, experimental for an experimental one or a
+#                                  clone on that branch), if they changed. Into a folder of its own,
+#                                  swapped in once every file matches. Offline or nothing new: no-op.
+#        hands/models.sh auto      update, as frametop-hands.service runs it when hand tracking
+#                                  starts: not with HANDS_MODELS_UPDATE=0 in ~/.config/frametop.conf,
+#                                  and at most every 15 minutes.
+set -euo pipefail
+root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
+. "$root/scripts/_env.sh"
+case ${1:-fetch} in fetch | check | update | auto) ;; *) sed -n '9,18p' "$0" >&2; exit 2 ;; esac
+
+on_frame_script "${1:-fetch}" "$FRAME_REPO" <<'SCRIPT'
+set -euo pipefail
+mode=$1 tree=$2
+models=$tree/hands/models dir=$tree/hands/models/student
+hf=DeeJanuz/frametop-hands-models gh=Frametop/frametop
+names=" NOTICE hand.ncnn.bin hand.ncnn.param palm.ncnn.bin palm.ncnn.param LICENSE README.md "
+
+# read_pin FILE: check every line of a pin, then set rev, model and files ("name sha256" each).
+# Anything unexpected fails it: a pin from GitHub only names files this list allows.
+read_pin() {
+  local key a b n format=
+  rev= model= files=()
+  while read -r key a b; do
+    case $key in '' | '#'*) continue ;; esac
+    case $key in
+      format) [ "$a" = 1 ] && [ -z "$b" ] || return 1; format=$a ;;
+      rev) [[ $a =~ ^[0-9a-f]{40}$ ]] && [ -z "$b" ] || return 1; rev=$a ;;
+      model) [[ $a =~ ^[A-Za-z0-9][A-Za-z0-9._-]{0,63}$ ]] && [ -z "$b" ] || return 1; model=$a ;;
+      file) [[ $names == *" $a "* && $b =~ ^[0-9a-f]{64}$ ]] || return 1; files+=("$a $b") ;;
+      *) return 1 ;;
+    esac
+  done <"$1"
+  [ -n "$format" ] && [ -n "$rev" ] && [ -n "$model" ] || return 1
+  for n in hand.ncnn.bin hand.ncnn.param palm.ncnn.bin palm.ncnn.param; do
+    [[ " ${files[*]} " == *" $n "* ]] || return 1
+  done
+}
+
+matches() { [ -f "$1" ] && [ "$(sha256sum <"$1" | cut -d' ' -f1)" = "$2" ]; }
+
+# fetch_into DIR: fetch the pin's files that are missing there or don't match.
+fetch_into() {
+  local d=$1 entry name sum src
+  mkdir -p "$d"
+  for entry in "${files[@]}"; do
+    read -r name sum <<<"$entry"
+    matches "$d/$name" "$sum" && continue
+    src=https://huggingface.co/$hf/resolve/$rev/$model/$name
+    case $name in LICENSE | README.md) src=https://huggingface.co/$hf/resolve/$rev/$name ;; esac
+    echo "fetching hand model $name ($model)"
+    if ! curl -fsSL --retry 3 --connect-timeout 10 --max-filesize 100000000 -o "$d/$name.part" "$src"; then
+      rm -f "$d/$name.part"; return 1
+    fi
+    if ! matches "$d/$name.part" "$sum"; then
+      echo "hand models: $name's sha256 doesn't match the pin; not kept" >&2
+      rm -f "$d/$name.part"; return 1
+    fi
+    mv "$d/$name.part" "$d/$name"
+  done
+}
+
+stamp() { echo "$hf@${rev::7} $model" >"$1/VERSION"; }
+
+case $mode in
+  fetch)
+    read_pin "$tree/hands/models.pin" || { echo "hand models: hands/models.pin isn't a valid pin" >&2; exit 1; }
+    fetch_into "$dir"
+    cp "$tree/hands/models.pin" "$dir/PIN"; stamp "$dir"
+    echo "hand models: $model in $dir"
+    ;;
+  check)
+    # Against the pin they were fetched by: an update may have replaced the release's own.
+    pin=$dir/PIN; [ -f "$pin" ] || pin=$tree/hands/models.pin
+    [ -d "$dir" ] || { echo "hand models: $dir missing" >&2; exit 1; }
+    read_pin "$pin" || { echo "hand models: $pin isn't a valid pin" >&2; exit 1; }
+    bad=0
+    for entry in "${files[@]}"; do
+      read -r name sum <<<"$entry"
+      matches "$dir/$name" "$sum" || { echo "hand models: $name missing or changed" >&2; bad=1; }
+    done
+    [ "$bad" = 0 ] || exit 1
+    echo "hand models: $model in $dir, all files match"
+    ;;
+  update | auto)
+    if [ "$mode" = auto ]; then
+      conf=$(sed -n 's/#.*//; s/^[[:space:]]*HANDS_MODELS_UPDATE[[:space:]]*=[[:space:]]*//p' ~/.config/frametop.conf 2>/dev/null | tail -1)
+      conf=${FT_HANDS_MODELS_UPDATE:-$conf}
+      [ "$(echo $conf | tr -d "\"'")" = 0 ] && { echo "hand models: not updating (HANDS_MODELS_UPDATE=0)"; exit 0; }
+      [ -n "$(find "$models/.student-checked" -mmin -15 2>/dev/null)" ] && exit 0
+    fi
+    if [ -f "$tree/.frametop-release" ]; then
+      case $(sed -n 's/^CHANNEL=//p' "$tree/.frametop-release" | tail -1) in
+        experimental) branch=experimental ;; *) branch=main ;;
+      esac
+    else
+      branch=$(git -C "$tree" symbolic-ref --short -q HEAD 2>/dev/null || true)
+    fi
+    case $branch in main | experimental) ;; *) echo "hand models: no updates for branch '$branch'"; exit 0 ;; esac
+    mkdir -p "$models"
+    exec 9>"$models/.student-lock"
+    flock -n 9 || { echo "hand models: another update is running"; exit 0; }
+    touch "$models/.student-checked"
+    tmp=$(mktemp); trap 'rm -f "$tmp"' EXIT
+    code=$(curl -sS --connect-timeout 4 --max-time 15 --max-filesize 16384 -o "$tmp" -w '%{http_code}' \
+      "https://raw.githubusercontent.com/$gh/$branch/hands/models.pin") || code=
+    case $code in
+      200) ;;
+      '') echo "hand models: couldn't reach GitHub; keeping these"; exit 0 ;;
+      *) echo "hand models: $branch pins none ($code); keeping these"; exit 0 ;;
+    esac
+    cmp -s "$tmp" "$dir/PIN" 2>/dev/null && { echo "hand models: up to date ($(cat "$dir/VERSION" 2>/dev/null))"; exit 0; }
+    read_pin "$tmp" || { echo "hand models: $branch's pin isn't valid (or needs a newer Frametop); keeping these" >&2; exit 1; }
+    # Into .student-new, seeded with the files we have (a stopped update keeps what it fetched),
+    # then swapped with student in one rename: ft-hands never sees half a set.
+    new=$models/.student-new
+    mkdir -p "$new"
+    [ -d "$dir" ] && cp -a --update=none "$dir/." "$new/"
+    rm -f "$new"/*.part
+    fetch_into "$new" || { echo "hand models: update failed; keeping these" >&2; exit 1; }
+    cp "$tmp" "$new/PIN"; stamp "$new"
+    if [ -d "$dir" ]; then mv -T --exchange "$new" "$dir"; rm -rf "$new"; else mv -T "$new" "$dir"; fi
+    echo "hand models: updated to $model from $branch's pin"
+    ;;
+esac
+SCRIPT
@@ -10,3 +10,7 @@ Apache License, Version 2.0 (https://www.apache.org/licenses/LICENSE-2.0).
 Changes made here: converted to ncnn with pnnx, with the palm detector's channel pads
 rewritten as ncnn Padding layers, and quantized to 8 bits (the *-int8.ncnn.* files) with
 ncnn's tools.
+
+hands/models.sh fetches fine-tuned versions of the same two models into student/ (not in this
+repository): their weights are licensed CC BY-NC 4.0, not Apache-2.0 or MIT; see student/LICENSE and
+student/README.md once fetched, or https://huggingface.co/DeeJanuz/frametop-hands-models.
@@ -1,18 +1,18 @@
 #!/usr/bin/env bash
 # Build the hand recorder's headset panel ft-handpanel in the dev container on the Frame
-# (hands/rec/build/; it also runs there). Like gaze/build.sh: the pinned public OpenVR header
-# (the DMA-BUF import is newer than the header shipped with SteamVR's samples), stb_truetype
+# (hands/rec/build/; it also runs there), or in the release image (pack/Containerfile). Like
+# gaze/build.sh: the pinned public OpenVR header (scripts/openvr.sh; the DMA-BUF import is newer
+# than the header shipped with SteamVR's samples), stb_truetype
 # for the text and stb_image (PNG and GIF) for the pose pictures and the demo clips, at the same stb commit.
 set -euo pipefail
 root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
 "$root/scripts/sync.sh" >/dev/null
 exec "$root/scripts/frame.sh" -C hands/rec 'set -e; mkdir -p build/include
-openvr=v2.15.6
-[ -f build/include/openvr-$openvr ] || { curl -fsSL "https://raw.githubusercontent.com/ValveSoftware/openvr/$openvr/headers/openvr.h" -o build/include/openvr.h && touch build/include/openvr-$openvr; }
+. ../../scripts/openvr.sh
 stb=2c980bb59875b0d32144a71867fbdebb2f77cd20
 [ -f build/include/stb-$stb ] || { curl -fsSL "https://raw.githubusercontent.com/nothings/stb/$stb/stb_truetype.h" -o build/include/stb_truetype.h && touch build/include/stb-$stb; }
 [ -f build/include/stb_image-$stb ] || { curl -fsSL "https://raw.githubusercontent.com/nothings/stb/$stb/stb_image.h" -o build/include/stb_image.h && touch build/include/stb_image-$stb; }
 g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -Wno-missing-field-initializers -Ibuild/include $(pkg-config --cflags gbm libdrm) \
-  -o build/ft-handpanel panel/ft-handpanel.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64 \
+  -o build/ft-handpanel panel/ft-handpanel.cpp $OPENVR_LIBS \
  $(pkg-config --libs gbm libdrm) -lpthread
 echo "built build/ft-handpanel"'
@@ -1,6 +1,6 @@
 #!/bin/bash
 # Launch the Frametop Hand Recorder from a Plasma session on the Frame host.
-# The app runs in the dev container (PySide6, Kirigami and zstd come from Fedora there).
+# The app runs in Frametop's container (PySide6, Kirigami and zstd come from Fedora there).
 # podman needs the real XDG_RUNTIME_DIR and the real user bus (to reach systemd for
 # the container's cgroup; the Frametop session runs on a private bus from
 # dbus-run-session). The session's Wayland socket and bus go to the app itself.
@@ -14,8 +14,7 @@ export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
 # From the home folder: the app's host commands (distrobox-host-exec) run in its folder on the
 # host, and a folder only the container has, such as /run/host/tmp, makes them all fail.
 cd "$HOME" || exit 1
-"$here/../../scripts/container-up.sh"
-exec "$HOME/.local/bin/distrobox" enter dev -- env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
+exec "$here/../../scripts/in-box" env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
  XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \
  QT_QPA_PLATFORM="wayland;xcb" \
  python3 "$here/ft_handrec.py" "$@"
@@ -5,7 +5,9 @@
 # give ft-camd its capabilities (hands/run.sh caps: asks for the password, once per build), and add
 # "Frametop Hand Recorder" to the app menu (for Frametop's desktop: it isn't tested from SteamVR's
 # "Launch a program", which runs apps outside it; the standalone recorder is for that).
-# It doesn't turn on Frametop's live hand tracking (that's hands/run.sh install, still deferred).
+# It doesn't turn on Frametop's live hand tracking (that's hands/run.sh install, which install.sh offers).
+# A release (FRAME_RELEASE) brings the three built in its image (pack/Containerfile), so there it only
+# sets the capabilities and adds the menu entry; its install.sh offers this.
 # Usage: hands/rec/install.sh            install or update
 #        hands/rec/install.sh uninstall  remove the menu entry, and ft-camd's capabilities unless
 #                                        hand tracking's services use them (hands/run.sh uncaps).
@@ -17,13 +19,17 @@ entry='~/.local/share/applications/frametop-handrec.desktop'

 case ${1:-install} in
  install)
-    echo "== 1/4 dev container packages"
-    "$root/setup/dev-container.sh"
-    echo "== 2/4 hand tracking: ft-camd and ft-hands"
-    "$root/hands/build.sh"
-    echo "== 3/4 the headset panel: ft-handpanel"
-    "$root/hands/rec/build.sh"
-    echo "== 4/4 ft-camd's capabilities (asks for your password) and the menu entry"
+    if [ "$FRAME_RELEASE" = 1 ]; then
+      echo "== ft-camd's capabilities (asks for your password, unless it has them) and the menu entry"
+    else
+      echo "== 1/4 dev container packages"
+      "$root/setup/dev-container.sh"
+      echo "== 2/4 hand tracking: ft-camd and ft-hands"
+      "$root/hands/build.sh"
+      echo "== 3/4 the headset panel: ft-handpanel"
+      "$root/hands/rec/build.sh"
+      echo "== 4/4 ft-camd's capabilities (asks for your password) and the menu entry"
+    fi
    "$root/hands/run.sh" caps
    "$root/scripts/conf-migrate.sh"   # HANDS_SWAP_SIDES=0, the old default, becomes auto
    fill_template "$root/hands/rec/ft-handrec.desktop" |
@@ -55,6 +55,9 @@ import takes  # noqa: E402  (hands/rec/takes.py: the version, and the standalone
 FT_HANDS = os.path.join(HANDS, "build", "ft-hands")
 FT_CAMD = os.path.join(HANDS, "build", "ft-camd")
 PANEL_BIN = os.path.join(HERE, "build", "ft-handpanel")
+# Runs a program in the container this tree was built for: "dev" in a clone, the release's own in a
+# release (it starts the container first)
+IN_BOX = os.path.join(REPO, "scripts", "in-box")
 SCRIPT_PATH = os.path.join(HERE, "script.json")
 POSES_DIR = os.path.join(HERE, "poses")   # the pose pictures: poses.json and its PNGs
 DEMOS_DIR = os.path.join(HERE, "demos")   # the demo clips: <id>.gif (DESIGN.md "Demo clips")
@@ -83,6 +86,8 @@ HANDS_UNIT = "frametop-handrec-hands.service"
 # The standalone Hand Recorder (takes.standalone()): its binaries run on the host, so ft-hands
 # starts directly rather than in the dev container, and repairs mean its install command.
 STANDALONE = takes.standalone()
+# A Frametop release (pack/install-release.sh) rather than a clone
+RELEASE = os.path.exists(os.path.join(REPO, ".frametop-release"))

 RECORD_HZ = 10
 SIDES_READ_S = 0.5       # how often the tracking ft-hands' side camera decision is read
@@ -161,9 +166,13 @@ def host_command(*cmd):


 def fix_hint(frametop):
-    """How to repair a missing or broken part: Frametop's command, or the standalone install's."""
+    """How to repair a missing or broken part: Frametop's command, or the standalone install's.
+    A Frametop release never builds (its programs come in its image), so a build there means
+    installing the release again."""
    if STANDALONE:
        return STANDALONE.get("reinstall") or "run the Hand Recorder's install command again"
+    if RELEASE and frametop.endswith("build.sh"):
+        return "install this Frametop release again"
    return frametop


@@ -696,7 +705,7 @@ class Recorder:
        if ring:
            argv += ["--ring", ring]
        if not in_container() and not STANDALONE:
-            argv = [os.path.expanduser("~/.local/bin/distrobox"), "enter", "dev", "--"] + argv
+            argv = [IN_BOX] + argv
        self.proc = subprocess.Popen(argv, stdin=subprocess.DEVNULL, stdout=log_file, stderr=log_file)
        self.started_ns = mono_ns()

@@ -1408,7 +1417,7 @@ def camera_text(result):
    if camcheck.is_ring_short(result):
        return ("The headset's tracking cameras are all running, but the recorder can't read some of them (%s). "
                "Close the Hand Recorder, %s, and open it again. If that doesn't help, ask in the Frametop "
-                "Discord." % (result.get("reason", ""), fix_hint("run ~/frametop/hands/rec/install.sh again")))
+                "Discord." % (result.get("reason", ""), fix_hint("run %s again" % os.path.join(REPO, "hands", "rec", "install.sh"))))
    return ("Not all of the headset's tracking cameras are running (%s). Restart SteamVR, or restart the "
            "headset if that doesn't fix it." % result.get("reason", ""))

@@ -1846,11 +1855,8 @@ class Session:

    def _start_tracker(self):
        argv = [FT_HANDS, "--no-gestures", "--status", "0"]
-        if not STANDALONE:   # built in the dev container: it runs there
-            up = os.path.join(REPO, "scripts", "container-up.sh")
-            if os.access(up, os.X_OK):
-                subprocess.run(host_command(up), capture_output=True, timeout=120)
-            argv = [os.path.expanduser("~/.local/bin/distrobox"), "enter", "dev", "--"] + argv
+        if not STANDALONE:   # built in Frametop's container: it runs there
+            argv = [IN_BOX] + argv
        if self.ring:
            argv += ["--ring", self.ring]
        self._start_unit(HANDS_UNIT, "hand tracking for feedback", argv)
@@ -918,18 +918,23 @@ class StandaloneTest(unittest.TestCase):
        self.assertIn("--no-gestures", argv)
        run.assert_not_called()   # no container to bring up

-    def test_ft_hands_in_the_dev_container(self):
-        self.assertTrue(self.recorder_argv(None)[0].endswith("distrobox"))
+    def test_ft_hands_in_frametops_container(self):
+        # scripts/in-box: "dev" in a clone, the release's own container in a release
+        self.assertEqual(self.recorder_argv(None)[:2], [session.IN_BOX, session.FT_HANDS])
        argv, _ = self.tracker_argv(None)
-        self.assertEqual(argv[:4], [os.path.expanduser("~/.local/bin/distrobox"), "enter", "dev", "--"])
-        self.assertEqual(argv[4], session.FT_HANDS)
+        self.assertEqual(argv[:2], [session.IN_BOX, session.FT_HANDS])
+        self.assertTrue(os.access(session.IN_BOX, os.X_OK))

    def test_fix_hint(self):
        with mock.patch.object(session, "STANDALONE", self.INFO):
            self.assertEqual(session.fix_hint("hands/build.sh"), "run the install command again")
            text = session.camera_text({"status": "degraded", "reason": "2 of 4", "ring_missing": ["upper_left"]})
-        with mock.patch.object(session, "STANDALONE", None):
+        with mock.patch.object(session, "STANDALONE", None), mock.patch.object(session, "RELEASE", False):
            self.assertEqual(session.fix_hint("hands/build.sh"), "hands/build.sh")
+        with mock.patch.object(session, "STANDALONE", None), mock.patch.object(session, "RELEASE", True):
+            # a release never builds: its programs come in its image
+            self.assertEqual(session.fix_hint("hands/rec/build.sh"), "install this Frametop release again")
+            self.assertEqual(session.fix_hint("hands/run.sh caps (asks for sudo)"), "hands/run.sh caps (asks for sudo)")
        self.assertIn("run the install command again", text)
        self.assertNotIn("~/frametop", text)

@@ -58,7 +58,8 @@ states="for u in $units; do echo \"\$u: \$(systemctl --user is-active \$u)\"; do

 case ${1:-status} in
  install)
-    "$root/hands/build.sh"
+    # A release brings them built (pack/Containerfile), and never builds (FRAME_RELEASE).
+    [ "$FRAME_RELEASE" = 1 ] || "$root/hands/build.sh"
    set_caps
    for u in $units; do
      fill_template "$root/$(unit_dir $u)/$u" | on_frame "mkdir -p ~/.config/systemd/user && cat > ~/.config/systemd/user/$u"
@@ -75,12 +75,40 @@ struct Run {
    bool down() const { return pinch.side(1).flags & FH_PINCH_DOWN; }
 };

+// The values these tests were written for (the defaults before 2026-10-14): they test the rules,
+// TestDefaults the defaults.
 PinchParams Params() {
    PinchParams p;
    p.other_finger = 12;   // as ft-hands
+    p.end_m = 0.035, p.drag_open_m = 0.05, p.drag_open_s = 0.12, p.grip_takeover_s = 0.3;
    return p;
 }

+void TestDefaults() {
+    // since 2026-10-14 a pinch lets go less easily: a tap still reading 4 cm hasn't ended
+    PinchParams p;
+    p.other_finger = 12;
+    Run tap{Pinch(p), Grip()};
+    for (int k = 0; k < 5; ++k) tap.Step(0.06);
+    for (int k = 0; k < 3; ++k) tap.Step(0.01);
+    tap.Step(0.04), tap.Step(0.04);
+    CHECK(tap.down(), "defaults: a tap at 4 cm is still down");
+    tap.Step(0.05), tap.Step(0.05);
+    CHECK(!tap.down() && tap.lost == 0, "defaults: a tap ends on the second frame past 4.5 cm");
+    // a drag loosened to 5.5 cm ends after drag_open_s (0.2 s: six or seven frames), opened to 7 cm at once
+    Run drag{Pinch(p), Grip()};
+    for (int k = 0; k < 5; ++k) drag.Step(0.06);
+    for (int k = 0; k < 30; ++k) drag.Step(0.01);
+    int frames = 0;
+    while (drag.down() && frames < 20) drag.Step(0.055), ++frames;
+    CHECK(!drag.down() && frames >= 6 && frames <= 8, "defaults: a drag at 5.5 cm ended after %d frames", frames);
+    Run open{Pinch(p), Grip()};
+    for (int k = 0; k < 5; ++k) open.Step(0.06);
+    for (int k = 0; k < 30; ++k) open.Step(0.01);
+    open.Step(0.07), open.Step(0.07);
+    CHECK(!open.down() && open.lost == 0, "defaults: a drag opened to 7 cm ends on the second frame");
+}
+
 void TestDragSpike() {
    // a drag (held 1 s): the tips read 3.8 cm for two frames (a hidden thumb), then close again
    Run r{Pinch(Params()), Grip()};
@@ -267,6 +295,7 @@ int main() {
    TestGripTakeover();
    TestDip();
    TestFistGuard();
+    TestDefaults();
    if (failures) {
        std::printf("%d failures\n", failures);
        return 1;
@@ -250,7 +250,10 @@ int main(int argc, char **argv) {
    double seconds = 0, status = 5;
    int threads = 3, niceness = 5;
    bool int8 = false, publish = true, track = true, gestures_on = true;
-    std::string models = std::string(argv[0]).substr(0, std::string(argv[0]).rfind('/') + 1) + "../models/ncnn";
+    // The models: HANDS_MODELS or --models, else the fine-tuned ones hands/models.sh fetched
+    // (models/student, chosen below once --int8 is known: they have no 8-bit files), else the stock ones.
+    const std::string model_root = std::string(argv[0]).substr(0, std::string(argv[0]).rfind('/') + 1) + "../models/";
+    std::string models;
    std::string record, ring_path = "/run/user/" + std::to_string(getuid()) + "/" FH_RING_NAME;
    // SteamOS starts user processes on CPUs 0-4 and keeps 5-7 (two A720s and the X4) for
    // SteamVR's compositor, whose threads there run at real-time priority, so they always
@@ -368,7 +371,7 @@ int main(int argc, char **argv) {
                        "          [--flag-buffer 0|1] (1: keep the last seconds for `ft-handsctl flag`; HANDS_FLAG_BUFFER)\n"
                        "          [--cams auto|mono|color|all] (auto) [--bright all|color] (all) [--bright-on L] (40) [--bright-off L] (25)\n"
                        "          [--color-left color_video0|color_video3] [--color-crop subtract|none]\n"
-                        "          [--pinch-begin M] (0.020) [--pinch-end M] (0.035) [--pinch-triangulated] [--pinch-palm-down MAX] (1: off)\n"
+                        "          [--pinch-begin M] (0.020) [--pinch-end M] (0.045) [--pinch-triangulated] [--pinch-palm-down MAX] (1: off)\n"
                        "          [--pinch-dip 0|1] (0; 1: also on a sharp dip, see PinchParams::dip; HANDS_PINCH_DIP)\n"
                        "          [--pinch-fist-curl R] (1.2; 0: off: no pinch begins on a fist, see PinchParams::fist_curl;\n"
                        "          HANDS_PINCH_FIST_CURL)\n"
@@ -453,6 +456,10 @@ int main(int argc, char **argv) {
        std::fflush(stdout);
        return true;
    };
+    if (models.empty())
+        models = model_root + (!int8 && access((model_root + "student/hand.ncnn.param").c_str(), R_OK) == 0 &&
+                                       access((model_root + "student/palm.ncnn.param").c_str(), R_OK) == 0
+                                   ? "student" : "ncnn");
    if (!load_calibration(calib, err) || !ring.open(ring_path.c_str(), err) || !nets.load(models, int8, err) ||
        (publish && (!pub.open(err) || (gestures_on && !gestures.open(pinch, grip, err)))) ||
        (!record.empty() && !start_recording(record, err))) {
@@ -15,7 +15,10 @@ extern "C" {

 struct PinchParams {
    double begin_m = 0.020;   // thumb and index tips closer than this: the pinch begins
-    double end_m = 0.035;     // further apart than this: it ends (the gap keeps it from flickering)
+    // further apart than this: it ends (the gap keeps it from flickering). 0.045 since 2026-10-14
+    // (0.035 before): taps and pinches let go while still held (user, 2026-10-11, gaze + hands: ends
+    // came at 4-6 cm); these four were the headset's HANDS_PINCH_* settings since, now everyone's.
+    double end_m = 0.045;
    int end_frames = 2;       // processed frames in a row past end_m before it ends, so one
                              // noisy frame doesn't drop a drag
    double grace_s = 0.25;    // a pinching hand lost this long ends its pinch (FH_PINCH_LOST)
@@ -27,11 +30,12 @@ struct PinchParams {
    // replay (pinches held 0.3 s or more, typing among them) the ends came 33-134 ms later, at most
    // 200, where any dip under end_m starting the wait over kept one held 6 s longer. A tap ends as
    // before: its release is the click.
-    double drag_s = 0.3, drag_open_m = 0.05, drag_open_s = 0.12, drag_reclose_m = 0.03;
+    double drag_s = 0.3, drag_open_m = 0.065, drag_open_s = 0.2, drag_reclose_m = 0.03;   // 0.05, 0.12 before 2026-10-14
    // A grip takes this pinch's hand (ends it, as lost) only within grip_takeover_s of its begin:
    // closing the hand into a fist passes through a pinch. Later the pinch is being held (a drag),
    // and the other fingers curling around it don't end it (3 of 28 pinches ended so, the same test).
-    double grip_takeover_s = 0.3;
+    // 0.1 since 2026-10-14 (0.3 before: grips took pinches 130 ms in).
+    double grip_takeover_s = 0.1;
    // Where the distance comes from: MediaPipe's world landmarks (the model's own 3D hand
    // pose, averaged over the hand's views, at the user's hand size), or the tracker's
    // triangulated tips. The model's pose should hold up better when the fingers hide each
@@ -109,6 +109,7 @@

 #include <json/json.h>
 #include <sys/stat.h>
+#include <unistd.h>

 #include <algorithm>
 #include <chrono>
@@ -175,7 +176,12 @@ int main(int argc, char **argv) {
    std::string flag_out;
    FlagConfig flag_cfg;
    flag_cfg.max_bytes = size_t(400) << 20, flag_cfg.writer_cpus.clear();
-    std::string timeline, depth, poses, models = std::string(argv[0]).substr(0, std::string(argv[0]).rfind('/') + 1) + "../models/ncnn";
+    // as ft-hands: the fine-tuned models (models/student) when hands/models.sh fetched them, else the stock ones
+    const std::string model_root = std::string(argv[0]).substr(0, std::string(argv[0]).rfind('/') + 1) + "../models/";
+    std::string timeline, depth, poses,
+        models = model_root + (access((model_root + "student/hand.ncnn.param").c_str(), R_OK) == 0 &&
+                                       access((model_root + "student/palm.ncnn.param").c_str(), R_OK) == 0
+                                   ? "student" : "ncnn");
    for (int i = 2; i < argc; ++i) {
        const std::string a = argv[i];
        const bool more = i + 1 < argc;
@@ -1,16 +1,21 @@
 #!/usr/bin/env bash
 # Install everything on the Steam Frame: the build container, Frametop (multi-screen
 # desktop, input relay, universal 3D mouse, settings app), and optionally voice dictation,
-# gaze mode, our own eye tracker for it, and the Bluetooth fixes. Run it on the headset in a terminal, from this repo. It's safe to re-run,
-# for example after `git pull`. (Hand tracking, hands/, is deferred: it isn't offered here.)
+# hand tracking, gaze mode, our own eye tracker for it, and the Bluetooth fixes. Run it on the
+# headset in a terminal, from this repo. It's safe to re-run, for example after `git pull`.
 # (It also works from a PC over SSH; see "Developing from a PC" in the README.)
 # In a release (pack/install-release.sh), the programs come built from its image: this installs the
 # distrobox the release brings, makes the release's container from its image, and builds nothing.
 #
-# Usage: ./install.sh [--yes] [--no-voice] [--no-eye-tracker] [--no-bluetooth | --bluetooth]
-#   --yes           don't ask; installs voice dictation, gaze mode, and our eye tracker if sudo can
-#                   run without a password prompt; skips the Bluetooth fixes and the SteamVR restart
+# Usage: ./install.sh [--yes] [--no-voice] [--no-hands] [--no-hand-recorder] [--no-eye-tracker]
+#                    [--no-bluetooth | --bluetooth]
+#   --yes           don't ask; installs voice dictation and gaze mode, and hand tracking, the Hand
+#                   Recorder and our eye tracker if sudo can run without a password prompt; skips
+#                   the Bluetooth fixes and the SteamVR restart
 #   --no-voice      don't offer voice dictation (Frametop Voice: it downloads about 320 MB)
+#   --no-hands      don't offer hand tracking or the Hand Recorder
+#   --no-hand-recorder  don't offer the Hand Recorder (a release's; a clone builds it with
+#                   hands/rec/install.sh)
 #   --no-eye-tracker  don't offer our own eye tracker
 #   --no-bluetooth  don't offer the Bluetooth fixes
 #   --bluetooth     install the Bluetooth fixes without asking (with --yes: if sudo can run
@@ -20,7 +25,7 @@ set -euo pipefail
 root=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
 . "$root/scripts/_env.sh"

-assume_yes=0 bluetooth=1 eye_tracker=1 voice=1
+assume_yes=0 bluetooth=1 eye_tracker=1 hands=1 hand_recorder=1 voice=1
 for arg in "$@"; do
  case $arg in
    --yes) assume_yes=1 ;;
@@ -28,7 +33,9 @@ for arg in "$@"; do
    --bluetooth) bluetooth=2 ;;
    --no-eye-tracker) eye_tracker=0 ;;
    --no-voice) voice=0 ;;
-    -h|--help) sed -n '2,18p' "$0"; exit 0 ;;
+    --no-hands) hands=0 hand_recorder=0 ;;
+    --no-hand-recorder) hand_recorder=0 ;;
+    -h|--help) sed -n '2,22p' "$0"; exit 0 ;;
    *) echo "unknown option: $arg" >&2; exit 2 ;;
  esac
 done
@@ -70,7 +77,7 @@ if [ "$FRAME_LOCAL" = 0 ] || [ -n "${SSH_CONNECTION:-}" ]; then
  echo "that need SteamVR start with it if it isn't running now."
 fi

-step "1/11 distrobox (container tool, installed in your home folder)"
+step "1/12 distrobox (container tool, installed in your home folder)"
 if on_frame 'test -x ~/.local/bin/distrobox'; then
  echo "already installed: $(on_frame '~/.local/bin/distrobox version | head -1')"
 elif [ "$FRAME_RELEASE" = 1 ]; then
@@ -84,30 +91,30 @@ cd ~/dev/src/distrobox && ./install --prefix ~/.local'
 fi

 if [ "$FRAME_RELEASE" = 1 ]; then
-  step "2/11 Frametop's container (from this release's image)"
+  step "2/12 Frametop's container (from this release's image)"
  "$root/pack/release-box.sh"
 else
-  step "2/11 build container (Fedora 44 'dev', about 1-2 GB the first time)"
+  step "2/12 build container (Fedora 44 'dev', about 1-2 GB the first time)"
  "$root/setup/dev-container.sh"
 fi

-step "3/11 input relay (keeps Bluetooth mice working in SteamVR, device roles, button maps)"
+step "3/12 input relay (keeps Bluetooth mice working in SteamVR, device roles, button maps)"
 "$root/desktops.sh" relay install

-step "4/11 3D mouse: SteamVR driver"
+step "4/12 3D mouse: SteamVR driver"
 build "$root/pointer/driver/build.sh"
 "$root/pointer/driver/install.sh" install

-step "5/11 3D mouse: pointer helper service, and the dashboard patch that lets it move SteamVR's windows"
+step "5/12 3D mouse: pointer helper service, and the dashboard patch that lets it move SteamVR's windows"
 build "$root/pointer/helper/build.sh"
 "$root/pointer/helper/run.sh" install
 "$root/pointer/dashpatch/run.sh" install

-step "6/11 power service (turns the displays off while the headset isn't used, even on a stand)"
+step "6/12 power service (turns the displays off while the headset isn't used, even on a stand)"
 build "$root/power/build.sh"
 "$root/power/run.sh" install

-step "7/11 multi-screen desktop (ft-screens), Frametop Input Settings, Display Settings, and Remote Displays"
+step "7/12 multi-screen desktop (ft-screens), Frametop Input Settings, Display Settings, and Remote Displays"
 build "$root/screens/build.sh"
 build "$root/stream/build.sh"  # ft-stream: remote displays (Frametop Remote Displays)
 "$root/desktops.sh" install >/dev/null
@@ -119,7 +126,7 @@ on_frame "sed -i 's/^POINTER=0/POINTER=1/' ~/.config/frametop.conf; grep -q '^PO
 echo "the launcher's Desktop entry now opens the multi-screen desktop; 3D mouse on (POINTER=1 in ~/.config/frametop.conf)"
 "$root/scripts/conf-migrate.sh"

-step "8/11 voice dictation (optional: speak, and it's typed where you type; nothing leaves the headset)"
+step "8/12 voice dictation (optional: speak, and it's typed where you type; nothing leaves the headset)"
 if [ "$voice" = 0 ]; then
  echo "skipped (--no-voice). Install later with: voice/run.sh install"
 elif ask "Install voice dictation? A Right Ctrl tap or holding the left d-pad down starts it; it downloads about 320 MB of speech models." y; then
@@ -130,7 +137,55 @@ else
  echo "skipped. Install later with: voice/run.sh install"
 fi

-step "9/11 gaze mode (optional, experimental: the pointer goes where you look)"
+# Hand tracking needs sudo once per build of ft-camd, its camera broker (file capabilities to
+# borrow XRService's camera buffers). Installed, it runs with the Frametop desktop
+# (HANDS_MODE=frametop), unless the settings already pick always.
+step "9/12 hand tracking (optional, experimental: your hands work the screens; it uses a lot of the headset's CPU)"
+if [ "$hands" = 0 ]; then
+  echo "skipped (--no-hands). Install later with: hands/run.sh install"
+elif [ "$assume_yes" = 1 ] && ! sudo_quiet; then
+  echo "skipped: it needs your password (sudo), and --yes doesn't ask. Install it later with: hands/run.sh install"
+elif ask "Install hand tracking? Poke, pinch and grab the screens with your hands. It runs while the Frametop desktop does (change that on the Hands page of Frametop Input Settings), and its camera broker needs your password (sudo)." y; then
+  if "$root/hands/run.sh" install; then
+    hands_caps=1
+    mode=$(on_frame "sed -n 's/^HANDS_MODE=\([a-z]*\).*/\1/p' ~/.config/frametop.conf | tail -1") || mode=
+    if [ "$mode" = always ]; then
+      echo "hand tracking runs whenever SteamVR does (HANDS_MODE=always, kept from your settings)"
+    else
+      if on_frame "python3 $(printf %q "$FRAME_REPO/hands/handsmode.py") set frametop" >/dev/null; then
+        echo "hand tracking runs while the Frametop desktop does (HANDS_MODE=frametop)"
+      else
+        echo "hand tracking is installed, but HANDS_MODE didn't change: choose With Frametop on the Hands page"
+      fi
+    fi
+  else
+    echo "hand tracking: not installed (above). Try again later with: hands/run.sh install"
+  fi
+else
+  echo "skipped. Install later with: hands/run.sh install"
+fi
+# The Hand Recorder (hands/rec): a release brings it built, so it's only ft-camd's capabilities
+# (set already if hand tracking just installed) and the menu entry. An entry from the release
+# before points into that release's folder, which goes two releases on: it's renewed without asking.
+# A clone builds it with hands/rec/install.sh (a few minutes), so it isn't offered here.
+if [ "$FRAME_RELEASE" = 1 ]; then
+  recorder="the Hand Recorder (records your hands for Frametop's open hand dataset)"
+  if [ "$hand_recorder" = 0 ]; then
+    echo "$recorder: skipped. Add it later with: hands/rec/install.sh"
+  elif on_frame '[ -e ~/.local/share/applications/frametop-handrec.desktop ]'; then
+    "$root/hands/rec/install.sh" install >/dev/null && echo "$recorder: updated" ||
+      echo "$recorder: couldn't update it (above). Try again with: hands/rec/install.sh"
+  elif [ "$assume_yes" = 1 ] && [ "${hands_caps:-0}" = 0 ] && ! sudo_quiet; then
+    echo "$recorder: skipped: it needs your password (sudo), and --yes doesn't ask. Add it later with: hands/rec/install.sh"
+  elif ask "Add the Hand Recorder to the app menu? It records your hands for Frametop's open hand dataset, which trains hand tracking; nothing leaves the headset until you press Upload. Its camera broker needs your password (sudo), as hand tracking does." y; then
+    "$root/hands/rec/install.sh" install ||
+      echo "$recorder: not installed (above). Try again later with: hands/rec/install.sh"
+  else
+    echo "$recorder: skipped. Add it later with: hands/rec/install.sh"
+  fi
+fi
+
+step "10/12 gaze mode (optional, experimental: the pointer goes where you look)"
 gaze=0
 if ask "Install gaze mode? You turn it on and calibrate it in Frametop Input Settings, on the Gaze page." y; then
  "$root/gaze/run.sh" install
@@ -141,7 +196,7 @@ fi

 # Ours is gaze mode's default once it's installed (GAZE_TRACKER=auto). Without it, gaze mode
 # uses SteamVR's eye tracker.
-step "10/11 our own eye tracker for gaze mode (recommended: more accurate than SteamVR's)"
+step "11/12 our own eye tracker for gaze mode (recommended: more accurate than SteamVR's)"
 if [ "$gaze" = 0 ]; then
  echo "skipped: gaze mode isn't installed. Install it later with: gaze/tracker/install.sh"
 elif [ "$eye_tracker" = 0 ]; then
@@ -169,7 +224,7 @@ else
  echo "skipped: gaze mode uses SteamVR's eye tracker. Install ours later with: gaze/tracker/install.sh"
 fi

-step "11/11 Bluetooth fixes (optional; they let LE mice and keyboards like the Swiftpoint Z3 reconnect)"
+step "12/12 Bluetooth fixes (optional; they let LE mice and keyboards like the Swiftpoint Z3 reconnect)"
 if [ "$bluetooth" = 2 ] && [ "$assume_yes" = 1 ] && ! sudo_quiet; then
  echo "skipped: they need your password (sudo), and --yes doesn't ask. Install later with: setup/bluetooth/install.sh install"
 elif [ "$bluetooth" = 2 ] || { [ "$bluetooth" = 1 ] && ask "Install the Bluetooth fixes? They need your password (sudo)." n; }; then
@@ -29,7 +29,7 @@ test-python:

 # the C/C++ unit tests that need no model runtime: header-only logic with
 # made-up events (pointer/tests: the helper's hand interaction logic). sides_test.cpp is NOT here: it pulls in ncnn (hands'
-# deferred heavy build) and stays with `make check` in the hands build
+# heavy build) and stays with `make check` in the hands build
 # (hands/Makefile).
 test-c:
    #!/usr/bin/env bash
@@ -69,8 +69,7 @@ RUN uv venv --system-site-packages /opt/frametop/venv \
 # The sources, then the binaries, built by the same build scripts as on the
 # Frame (FRAME_IN_BOX=1: they run here instead of entering the dev container).
 # Each program stays in its build/ folder in /src/frametop and is copied to
-# /opt/frametop. Hand tracking is deferred exactly as in install.sh: it builds
-# ncnn from source and isn't needed until hands/run.sh install runs.
+# /opt/frametop.
 COPY . /src/frametop
 WORKDIR /src/frametop
 ENV FRAME_IN_BOX=1 OPENVR_LIB=/opt/frametop/lib
@@ -93,6 +92,16 @@ RUN gaze/tracker/build.sh \
 # above. Its speech models aren't in the image: voice/models.sh fetches them on install.
 RUN voice/build.sh \
 && install -t /opt/frametop/bin voice/build/bin/whisper-server
+# Hand tracking: ft-camd (static: it runs on the host, with the file
+# capabilities hands/run.sh install sets) and ft-hands, which builds ncnn at the
+# tag hands/Makefile pins. Both stay in hands/build/, where their units and
+# file capabilities point, so they aren't copied to /opt/frametop.
+RUN hands/build.sh \
+ && test -x hands/build/ft-camd && test -x hands/build/ft-hands
+# The Hand Recorder's headset panel: it stays in hands/rec/build/, where the recorder
+# (hands/rec/session.py) runs it from.
+RUN hands/rec/build.sh \
+ && test -x hands/rec/build/ft-handpanel

 # A release installs from this image (pack/install-release.sh): it copies
 # /src/frametop, built, to the Frame for its host side (services, the driver,
@@ -32,8 +32,9 @@ The native binaries (`ft-screens`, `ft-pointer`, the `ft_pointer` driver,
 `build.sh` scripts, run inside the image with `FRAME_IN_BOX=1`, so there is
 one build recipe for the image and the Frame. They stay in their `build/`
 folders under `/src/frametop` and are copied to `/opt/frametop`. Hand
-tracking stays deferred exactly as in `install.sh` (ncnn is a heavy,
-separately triggered build).
+tracking's `ft-camd` and `ft-hands` (with ncnn, built from the tag
+`hands/Makefile` pins) stay in `hands/build/`, where their units and
+`ft-camd`'s file capabilities point.

 What deliberately stays outside the image for now: the **host payload** — the
 SteamVR driver registration (`vrpathreg`), ft-camd's file capabilities
@@ -4,7 +4,9 @@
 # are frametop-image.tar (Frametop, built, as a container image) and frametop-release.json (its
 # version, commit, the image file's sha256 and the image's ID, and the SteamOS table). Nothing
 # builds on the headset, and nothing else is downloaded, but voice dictation's speech models
-# (about 320 MB, fixed URLs checked by sha256: voice/models.sh) if you install it.
+# (about 320 MB, fixed URLs checked by sha256: voice/models.sh) if you install it. Hand tracking
+# and the Hand Recorder come built too: their installs only set their camera broker's file
+# capabilities (sudo), and the recorder's adds its menu entry.
 #
 # 1. It checks this SteamOS build against the SteamOS table: Frametop's newest from GitHub
 #    (pack/steamos.json on main) when it can get it, else the one in the release. Tested: on.
@@ -17,7 +19,8 @@
 #    are removed, with their containers and images.
 #
 # Usage: install-release.sh [--yes] [--any-steamos] [--unpack-only] [--dir DIR]
-#                           [--no-voice] [--no-eye-tracker] [--no-bluetooth | --bluetooth]
+#                           [--no-voice] [--no-hands] [--no-hand-recorder] [--no-eye-tracker]
+#                           [--no-bluetooth | --bluetooth]
 #   --unpack-only  stop after step 2 and the copy: don't run install.sh
 #   --dir DIR      where the releases go (default ~/.local/share/frametop/releases)
 # The rest go to install.sh. FRAMETOP_STEAMOS_TABLE names another table to fetch (a URL), or
@@ -110,8 +113,8 @@ main() {
      --any-steamos) any=1 ;;
      --unpack-only) unpack_only=1 ;;
      --dir) base=${2:?--dir needs a folder}; shift ;;
-      --no-voice|--no-eye-tracker|--no-bluetooth|--bluetooth) pass+=("$1") ;;
-      -h|--help) sed -n '2,24p' "$0"; return 0 ;;
+      --no-voice|--no-hands|--no-hand-recorder|--no-eye-tracker|--no-bluetooth|--bluetooth) pass+=("$1") ;;
+      -h|--help) sed -n '2,27p' "$0"; return 0 ;;
      *) echo "unknown option: $1" >&2; return 2 ;;
    esac
    shift
@@ -167,7 +170,7 @@ main() {
    return 1
  fi

-  printf '\n\033[1m== 0/11 loading Frametop %s (a minute or two)\033[0m\n' "$version"
+  printf '\n\033[1m== 0/12 loading Frametop %s (a minute or two)\033[0m\n' "$version"
  if [ "$(podman image inspect -f '{{.Id}}' "$image" 2>/dev/null)" = "$id" ]; then
    echo "already loaded"
  else
@@ -6,7 +6,7 @@ On a Steam Frame (SteamOS, VR variant). Download **Frametop.zip** below (about 1
 - **On the headset:** unpack Frametop.zip (in the desktop's Dolphin or Konsole) and run `Frametop/frametop-install.sh`.
 - **In a terminal on the headset:** `curl -fsSL https://frametop.github.io/frametop/get.sh | bash -s -- --release`

-A window asks what to install, and your SteamOS password for the two optional parts that need it (our own eye tracker and the Bluetooth fixes); the password is only used for this install. Nothing is compiled on the headset. Restart SteamVR once afterwards.
+A window asks what to install, and your SteamOS password for the optional parts that need it (hand tracking, the Hand Recorder, our own eye tracker, and the Bluetooth fixes); the password is only used for this install. Nothing is compiled on the headset. Restart SteamVR once afterwards.

 Before installing, the installer checks your SteamOS build against the builds Frametop was tested on, and stops if this release is known not to work there.

@@ -0,0 +1,328 @@
+#!/usr/bin/env python3
+"""report-watch: what scripts/report.sh --watch records while the problem happens. On the Frame:
+  python3 scripts/report-watch.py SECONDS AREA[,AREA...]
+Each line: the time, the area, and what it saw. Areas (any of them, each in a thread of its own):
+  system   every 5 s: the load, the CPU clocks, the hottest thermal zone, free memory, and the
+           processes that used the most CPU in those 5 s
+  desktop  ft-screens' debug line (the Steam menu, our keyboard, which laser drags what, where
+           typing goes) whenever it changes
+  gaze     the gaze service's status (which tracker, awake, samples coming in, the headset worn)
+           whenever it changes
+  hands    each second ft-hands publishes or stops: its frames per second, their latency, how often
+           each hand was in them and how sure, the closest thumb-index and thumb-middle tips; and
+           sides.json whenever the side cameras' decision changes
+  voice    Frametop Voice's events, with what was said left out (its word count stays), and the
+           microphone's level each second (peak and RMS, dBFS; no sound is kept)
+Read-only: it asks the services over their sockets and reads their published files.
+"""
+import json
+import math
+import mmap
+import os
+import socket
+import struct
+import subprocess
+import sys
+import threading
+import time
+from array import array
+from pathlib import Path
+
+RUN = Path(f"/run/user/{os.getuid()}")
+lock = threading.Lock()
+
+
+def say(area, text):
+    now = time.time()
+    with lock:
+        print(time.strftime("%H:%M:%S", time.localtime(now)) + f".{int(now * 100) % 100:02d}", f"[{area}]", text, flush=True)
+
+
+def ask(name, msg, timeout=1.0):
+    """A request to one of Frametop's abstract datagram sockets; the reply, or None."""
+    s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
+    s.bind("")
+    s.settimeout(timeout)
+    try:
+        s.sendto(msg.encode(), "\0" + name)
+        return s.recv(65536).decode()
+    except OSError:
+        return None
+    finally:
+        s.close()
+
+
+def read(path):
+    try:
+        return Path(path).read_text().strip()
+    except OSError:
+        return ""
+
+
+# ---- system
+
+
+def cpu_times():
+    """pid -> (name, utime + stime in ticks)."""
+    out = {}
+    for d in os.listdir("/proc"):
+        if not d.isdigit():
+            continue
+        try:
+            s = open(f"/proc/{d}/stat").read()
+        except OSError:
+            continue
+        name, rest = s[s.index("(") + 1:s.rindex(")")], s[s.rindex(")") + 2:].split()
+        out[int(d)] = (name, int(rest[11]) + int(rest[12]))
+    return out
+
+
+def system(end):
+    tick = os.sysconf("SC_CLK_TCK")
+    zones = sorted(Path("/sys/class/thermal").glob("thermal_zone*"))
+    cpus = sorted(Path("/sys/devices/system/cpu").glob("cpu[0-9]*/cpufreq/scaling_cur_freq"),
+                  key=lambda p: int(p.parent.parent.name[3:]))
+    before, t0 = cpu_times(), time.monotonic()
+    while time.monotonic() < end:
+        time.sleep(min(5, max(0.1, end - time.monotonic())))
+        now, t1 = cpu_times(), time.monotonic()
+        used = sorted(((v[1] - before[p][1]) / tick / (t1 - t0) * 100, v[0]) for p, v in now.items()
+                      if p in before and before[p][0] == v[0])
+        top = ", ".join(f"{name} {pct:.0f}%" for pct, name in reversed(used[-5:]) if pct >= 1)
+        before, t0 = now, t1
+        temps = []
+        for z in zones:
+            t = read(z / "temp")
+            if t.lstrip("-").isdigit():
+                temps.append((int(t) / 1000, read(z / "type")))
+        hot = max(temps) if temps else (float("nan"), "?")
+        mhz = " ".join(str(int(read(c) or 0) // 1000) for c in cpus)
+        mem = {l.split(":")[0]: int(l.split()[1]) for l in open("/proc/meminfo") if l.split(":")[0] in ("MemAvailable", "MemTotal")}
+        say("system", f"load {read('/proc/loadavg').split(' ')[0]}  CPU MHz {mhz}  hottest {hot[0]:.1f} C ({hot[1]})  "
+                      f"memory free {mem.get('MemAvailable', 0) // 1024} of {mem.get('MemTotal', 0) // 1024} MB  top: {top or '-'}")
+
+
+# ---- desktop
+
+
+def desktop(end):
+    last = None
+    while time.monotonic() < end:
+        line = ask("ft_screens", "debug") or "no answer from ft-screens"
+        if line != last:
+            say("desktop", line.strip())
+            last = line
+        time.sleep(0.2)
+
+
+# ---- gaze
+
+GAZE_KEYS = ("tracker", "source", "awake", "idle", "ft_gaze", "own_running", "eyes_process", "headset_on",
+             "eye_misses", "lessons", "calibration_samples")
+
+
+def gaze(end):
+    last = None
+    while time.monotonic() < end:
+        r = ask("ft_gazed", "status", 0.5)
+        try:
+            st = json.loads(r) if r else None
+        except ValueError:
+            st = None
+        if st is None:
+            text = "no answer from the gaze service (ft-gazed)"
+        else:
+            age = st.get("sample_age_s")
+            samples = "none" if age is None else "coming in" if age < 0.5 else f"stopped ({age:.0f} s ago)" if age < 600 else "stopped"
+            text = f"samples {samples}  " + "  ".join(f"{k} {json.dumps(st[k])}" for k in GAZE_KEYS if k in st)
+        if text != last:
+            say("gaze", text)
+            last = text
+        time.sleep(1)
+
+
+# ---- hands (include/fh_hands.h)
+
+HEAD = struct.Struct("<8sIIQQQII16x")          # magic, version, size, seq, capture_ns, publish_ns, nhands, ncapsules
+HAND = struct.Struct("<IIff252xI")             # id, flags, confidence, reserved, pts[21][3], ncapsules: 272 bytes
+V1_SIZE = HEAD.size + 2 * HAND.size + 64 * 32  # fh_hands_t
+FIT_PINCH = 912                                # fh_hand_fit_t.pinch_d (thumb-index, thumb-middle), then fit_rms
+
+
+def hands(end):
+    path = RUN / "frametop-hands/hands"
+    sides_last, quiet = None, None
+    next_report = time.monotonic() + 1
+    frames, seen, conf, lat, tips = 0, {0: 0, 1: 0}, {0: [], 1: []}, [], {0: [], 1: []}
+    last_seq = None
+    m, ino = None, None
+    while time.monotonic() < end:
+        # under its sequence lock: seq, a copy, seq again; the copy counts if both are the same even number
+        data, again = b"", None
+        try:
+            st = path.stat()   # a restarted ft-hands writes a new file
+            if m is None or (st.st_ino, st.st_size) != ino:
+                with open(path, "rb") as f:
+                    m, ino = mmap.mmap(f.fileno(), 0, prot=mmap.PROT_READ), (st.st_ino, st.st_size)
+            first = struct.unpack_from("<Q", m, 16)[0]
+            data = bytes(m)
+            again = struct.unpack_from("<Q", m, 16)[0]
+        except (OSError, ValueError):
+            m = None
+        if len(data) >= V1_SIZE:
+            magic, version, _, seq, cap, pub, n, _ = HEAD.unpack_from(data)
+            if magic == b"FHHANDS1" and seq == first == again and seq % 2 == 0 and seq != last_seq and pub:
+                new, last_seq = last_seq is not None, seq
+            else:
+                new = False
+            if new:   # not the frame already there when the watch began
+                frames += 1
+                lat.append((time.monotonic_ns() - cap) / 1e6)
+                for k in range(min(n, 2)):
+                    _, flags, c, _, _ = HAND.unpack_from(data, HEAD.size + k * HAND.size)
+                    side = flags & 1
+                    seen[side] += 1
+                    conf[side].append(c)
+                    off = V1_SIZE + k * 1024
+                    if version >= 2 and len(data) >= off + 1024 and struct.unpack_from("<I", data, off + 4)[0] & 1:
+                        tips[side].append(struct.unpack_from("<fff", data, off + FIT_PINCH))
+        if time.monotonic() >= next_report:
+            next_report += 1
+            if frames:
+                lat.sort()
+                def side(s, name):
+                    if not seen[s]:
+                        return f"{name} -"
+                    t = tips[s]
+                    fit = (f" closest thumb-index {1000 * min(x[0] for x in t):.0f} mm, thumb-middle "
+                           f"{1000 * min(x[1] for x in t):.0f} mm, fit {1000 * sum(x[2] for x in t) / len(t):.1f} mm") if t else ""
+                    return f"{name} {100 * seen[s] // frames}% conf {sum(conf[s]) / len(conf[s]):.2f}{fit}"
+                say("hands", f"{frames} frames/s  age at read {lat[len(lat) // 2]:.0f} ms (median)  "
+                             f"{side(0, 'left')}  |  {side(1, 'right')}")
+                quiet = False
+            elif quiet is not True:
+                say("hands", "no new frames from ft-hands" + ("" if path.exists() else f" ({path} missing)"))
+                quiet = True
+            frames, seen, conf, lat, tips = 0, {0: 0, 1: 0}, {0: [], 1: []}, [], {0: [], 1: []}
+            sides = read(RUN / "frametop-hands/sides.json")
+            try:
+                s = json.loads(sides)
+                sides = f"side cameras {s.get('state')}, swapped {s.get('swapped')} (by {s.get('decided_by')})"
+            except ValueError:
+                sides = sides or "no sides.json"
+            if sides != sides_last:
+                say("hands", sides)
+                sides_last = sides
+        time.sleep(1 / 120)
+
+
+# ---- voice
+
+SAID = ("text", "matched", "ran")   # what was said, and the command it ran (its words)
+
+
+def voice_events(end):
+    path = RUN / "frametop-voice/control.sock"
+    while time.monotonic() < end:
+        s = socket.socket(socket.AF_UNIX)
+        try:
+            s.connect(str(path))
+        except OSError as e:
+            say("voice", f"Frametop Voice isn't running ({e.strerror})")
+            s.close()
+            while time.monotonic() < end and not path.exists():
+                time.sleep(1)
+            continue
+        s.sendall(b"events\n")
+        s.settimeout(0.5)
+        buf = b""
+        while time.monotonic() < end:
+            try:
+                chunk = s.recv(4096)
+            except socket.timeout:
+                continue
+            except OSError:
+                chunk = b""
+            if not chunk:
+                say("voice", "Frametop Voice closed its events")
+                break
+            buf += chunk
+            while b"\n" in buf:
+                line, buf = buf.split(b"\n", 1)
+                try:
+                    ev = json.loads(line)
+                except ValueError:
+                    continue
+                ev.pop("time", None)
+                for k in SAID:
+                    if isinstance(ev.get(k), str):
+                        ev[k] = f"<{len(ev[k].split())} words>"
+                    elif k in ev:
+                        ev[k] = "<left out>"
+                say("voice", json.dumps(ev))
+        s.close()
+        time.sleep(1)
+
+
+def gain():
+    """The mic's ALSA gain, which SteamVR's proxmicmute sets to 0 while nobody wears the headset."""
+    vals = []
+    for numid in (23, 24):   # VA_DEC0, VA_DEC1 on card 0
+        try:
+            out = subprocess.run(["amixer", "-c", "0", "cget", f"numid={numid}"], capture_output=True, text=True, timeout=3).stdout
+        except (OSError, subprocess.SubprocessError):
+            return "unknown"
+        vals += [l.split("=", 1)[1] for l in out.splitlines() if ": values=" in l]
+    if not vals:
+        return "unknown"
+    return " ".join(vals) + (" (muted by proxmicmute: the headset off)" if all(v == "0" for v in vals) else "")
+
+
+def mic(end):
+    rate = 16000
+    try:
+        p = subprocess.Popen(["pw-record", "--raw", "--rate", str(rate), "--channels", "1", "--format", "s16", "-"],
+                             stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
+    except OSError as e:
+        return say("voice", f"mic: can't run pw-record ({e})")
+    silent = None
+    try:
+        while time.monotonic() < end:
+            raw = p.stdout.read(rate * 2)
+            if not raw:
+                return say("voice", "mic: pw-record stopped")
+            a = array("h", raw[: len(raw) // 2 * 2])
+            peak = max(abs(x) for x in a) / 32768 if a else 0
+            rms = math.sqrt(sum(x * x for x in a) / len(a)) / 32768 if a else 0
+            if peak == 0:   # digital silence: once, then how long it lasted
+                if silent is None:
+                    say("voice", f"mic: digital silence (all zeros); its gain: {gain()}")
+                    silent = time.monotonic()
+                continue
+            if silent is not None:
+                say("voice", f"mic: silent for {time.monotonic() - silent:.0f} s")
+                silent = None
+            say("voice", f"mic peak {20 * math.log10(peak):.0f} dBFS, RMS {20 * math.log10(max(rms, 1e-9)):.0f} dBFS")
+        if silent is not None:
+            say("voice", f"mic: silent all the {time.monotonic() - silent:.0f} s since")
+    finally:
+        p.kill()
+
+
+def main():
+    if len(sys.argv) != 3 or not sys.argv[1].isdigit():
+        sys.exit(__doc__)
+    end = time.monotonic() + int(sys.argv[1])
+    jobs = {"system": [system], "desktop": [desktop], "gaze": [gaze], "hands": [hands], "voice": [voice_events, mic]}
+    areas = [a for a in sys.argv[2].split(",") if a]
+    if not areas or any(a not in jobs for a in areas):
+        sys.exit(f"areas: {', '.join(jobs)}")
+    threads = [threading.Thread(target=f, args=(end,), daemon=True) for a in areas for f in jobs[a]]
+    for t in threads:
+        t.start()
+    for t in threads:
+        t.join(max(0, end - time.monotonic()) + 3)
+
+
+if __name__ == "__main__":
+    main()
@@ -1,31 +1,53 @@
 #!/usr/bin/env bash
-# Collect what a Frametop bug report needs into one text file: versions, service states,
-# settings, Frametop's keyboard and the Steam menu, recent logs, and the gaze report
-# (scripts/gaze-report.py). Bluetooth addresses and the headset's serial number are masked.
-# Usage: scripts/report.sh [--watch [SECONDS]]   (in a terminal on the Frame, or from a PC over SSH)
-#   --watch   then record SECONDS (default 60) while you make the problem happen: the
-#             desktop's state (the Steam menu, our keyboard, which laser drags what, where
-#             typing goes) whenever it changes, and the logs from that time
+# Collect what a Frametop bug report needs into one text file. In a terminal on the Frame, or from
+# a PC over SSH:
+#   scripts/report.sh [--system] [--desktop] [--gaze] [--hands] [--voice] [--watch [SECONDS]]
+# With no area, all of them. Every report has the versions and the settings (~/.config/frametop.conf).
+#   --system   services, what Frametop needs from SteamOS, CPU, memory, heat, the busiest processes,
+#              the session's log, SteamVR's errors
+#   --desktop  the screens' layout, Plasma's panels, our keyboard and the Steam menu, the input
+#              relay's, pointer helper's and power service's logs, ft-screens' log
+#   --gaze     (or --eyes) the gaze report (scripts/gaze-report.py): SteamVR's eye tracker and ours,
+#              calibrations, the gaze service; it wakes an idle gaze service for about 20 s
+#   --hands    hand tracking: its services, models, cameras, side cameras, ft-hands' counts, ft-camd
+#   --voice    Frametop Voice: its services, state, settings, speech models, and the microphone: its
+#              gain, PipeWire's capture chain, and its level over one second (no sound is kept)
+#   --watch    then record SECONDS (default 60, 5 to 600) of the chosen areas while you make the
+#              problem happen (scripts/report-watch.py), and their logs from that time
+# Left out: what was said to Frametop Voice (its word counts stay). Bluetooth addresses and the
+# headset's serial number are masked.
 set -euo pipefail
 root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
 . "$root/scripts/_env.sh"
-watch=0
-case ${1:-} in
-  "") ;;
-  --watch) watch=${2:-60}
-           [[ $watch =~ ^[0-9]+$ ]] && [ "$watch" -ge 5 ] && [ "$watch" -le 600 ] ||
-             { echo "--watch takes 5 to 600 seconds" >&2; exit 2; } ;;
-  -h|--help) sed -n '2,8p' "$0"; exit 0 ;;
-  *) echo "usage: $0 [--watch [SECONDS]]" >&2; exit 2 ;;
-esac
-out=$root/frametop-report-$(date +%Y%m%d-%H%M%S).txt
-echo "Collecting (up to half a minute: the gaze service is woken to see whether the eye tracker sends)..."
+all=system,desktop,gaze,hands,voice
+areas= watch=0
+usage() { echo "usage: $0 [--system] [--desktop] [--gaze] [--hands] [--voice] [--watch [SECONDS]]  (-h: more)" >&2; exit 2; }
+while [ $# -gt 0 ]; do
+  case $1 in
+    --system | --desktop | --gaze | --hands | --voice) areas+=${areas:+,}${1#--} ;;
+    --eyes) areas+=${areas:+,}gaze ;;
+    --watch)
+      watch=60
+      if [[ ${2:-} =~ ^[0-9]+$ ]]; then watch=$2; shift; fi
+      [ "$watch" -ge 5 ] && [ "$watch" -le 600 ] || { echo "--watch takes 5 to 600 seconds" >&2; exit 2; } ;;
+    -h | --help) sed -n '2,20p' "$0"; exit 0 ;;
+    *) usage ;;
+  esac
+  shift
+done
+areas=${areas:-$all}
+out=$root/frametop-report-$(date +%Y%m%d-%H%M%S)$([ "$areas" = "$all" ] || echo "-${areas//,/-}").txt
+echo "Collecting: ${areas//,/, }$([[ ,$areas, == *,gaze,* ]] && echo " (up to half a minute: the gaze service is woken to see whether the eye tracker sends)")..."

-on_frame_script "$FRAME_REPO" > "$out" 2>&1 <<'EOF' || true
-repo=$1
+on_frame_script "$FRAME_REPO" "$areas" > "$out" 2>&1 <<'EOF' || true
+repo=$1 areas=$2
 export XDG_RUNTIME_DIR=/run/user/$(id -u) DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/$(id -u)/bus
 logs=~/.local/share/Steam/logs
+has() { [[ ,$areas, == *,$1,* ]]; }
 section() { printf '\n===== %s\n' "$*"; }
+units() { for u in "$@"; do echo "$u: $(systemctl --user is-enabled $u 2>/dev/null) / $(systemctl --user is-active $u 2>/dev/null)"; done; }
+# podman's lines for each command run in the container, and ft-hands' pinch and grip lines
+noise='container (exec|exec_died|init|start|died|attach|cleanup)'
 # ask SOCKET COMMAND: a request to one of Frametop's abstract datagram sockets; prints the reply.
 ask() {
  python3 -I - "$1" "$2" <<'PY' 2>&1
@@ -41,7 +63,7 @@ except OSError as e:
 PY
 }

-section Versions
+section "Versions (report: ${areas//,/, })"
 grep -E '^(PRETTY_NAME|VERSION_ID|BUILD_ID|VARIANT_ID)=' /etc/os-release
 # The log keeps earlier runs: the last start is this SteamVR.
 grep -a -o 'vrcompositor [0-9.]* startup' $logs/vrcompositor.txt 2>/dev/null | tail -n 1 | grep . || echo "SteamVR: not found"
@@ -57,25 +79,49 @@ fi
 ~/.local/bin/distrobox version 2>/dev/null || echo "distrobox: not installed"
 podman container inspect -f "container $box: running={{.State.Running}} image={{.ImageName}}" "$box" 2>/dev/null ||
  echo "container $box: missing"
-
-section Services
-for u in frametop-input-relay frametop-pointer frametop-power frametop-gaze frametop-voice frametop-desktop; do
-  echo "$u: $(systemctl --user is-enabled $u 2>/dev/null) / $(systemctl --user is-active $u 2>/dev/null)"
-done
-for u in frametop-eyegrab steamframe-bt-fixups; do
-  echo "$u (system): $(systemctl is-enabled $u 2>/dev/null) / $(systemctl is-active $u 2>/dev/null)"
-done
-echo "desktop: $(pgrep -x ft-screens >/dev/null && echo running || echo 'not running'), plasmashell: $(pgrep -c plasmashell || true)"
-echo "paused for VR games: $(cat /run/user/$(id -u)/frametop-pause.json 2>/dev/null || echo 'no (never paused since boot)')"
-# SteamVR's config folder: a terminal in the desktop has the session's own (docs/design.md).
-XDG_CONFIG_HOME=$HOME/.config LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrpathreg show 2>/dev/null | sed -n '/xternal/,$p'
-
-section "What Frametop needs from SteamOS (scripts/update-check.py)"
-python3 "$repo/scripts/update-check.py" 2>&1 || true
+echo "up $(uptime -p 2>/dev/null | sed 's/^up //'), since $(uptime -s 2>/dev/null)"

 section Settings
 grep -v '^\s*#' ~/.config/frametop.conf 2>/dev/null | sed 's/\s*#.*//' | grep . || echo "no ~/.config/frametop.conf"
-python3 - <<'PY' 2>/dev/null || echo "no ~/.config/frametop-layout.json"
+
+if has system; then
+  section Services
+  units frametop-input-relay frametop-pointer frametop-power frametop-gaze frametop-desktop frametop-voice \
+    frametop-voice-whisper frametop-camd frametop-hands frametop-handview
+  for u in frametop-eyegrab steamframe-bt-fixups; do
+    echo "$u (system): $(systemctl is-enabled $u 2>/dev/null) / $(systemctl is-active $u 2>/dev/null)"
+  done
+  echo "failed user units: $(systemctl --user --failed --no-legend --plain 2>/dev/null | awk '{print $1}' | xargs | grep . || echo none)"
+  echo "desktop: $(pgrep -x ft-screens >/dev/null && echo running || echo 'not running'), plasmashell: $(pgrep -c plasmashell || true)"
+  echo "paused for VR games: $(cat /run/user/$(id -u)/frametop-pause.json 2>/dev/null || echo 'no (never paused since boot)')"
+  # SteamVR's config folder: a terminal in the desktop has the session's own (docs/design.md).
+  XDG_CONFIG_HOME=$HOME/.config LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 /opt/steamvr/bin/linuxarm64/vrpathreg show 2>/dev/null | sed -n '/xternal/,$p'
+
+  section "What Frametop needs from SteamOS (scripts/update-check.py)"
+  python3 "$repo/scripts/update-check.py" 2>&1 || true
+
+  section "CPU, memory, heat"
+  echo "load: $(cut -d' ' -f1-3 /proc/loadavg)"
+  free -m | sed -n 1,2p
+  df -h / /home 2>/dev/null | awk 'NR == 1 || !seen[$1]++'
+  for f in /sys/devices/system/cpu/cpu[0-9]*/cpufreq/scaling_cur_freq; do
+    c=${f%/cpufreq/*}; printf '%s %s/%s MHz  ' "${c##*/}" $(( $(cat $f) / 1000 )) $(( $(cat ${f%/*}/cpuinfo_max_freq) / 1000 ))
+  done; echo
+  for z in /sys/class/thermal/thermal_zone*; do
+    echo "$(cat $z/temp 2>/dev/null) $(cat $z/type 2>/dev/null)"
+  done | sort -rn | head -n 5 | awk '{ printf "%.1f C %s\n", $1 / 1000, $2 }'
+  echo "the busiest processes now:"
+  top -b -n 1 -w 160 2>/dev/null | sed -n '7,17p'
+
+  section "/tmp/frametop-session.log (last 60 lines)"
+  tail -n 60 /tmp/frametop-session.log 2>/dev/null || echo "missing"
+  section "SteamVR server: Frametop and errors (last 60 lines)"
+  grep -a -i -E 'ft_pointer|frametop|\[error\]' $logs/vrserver.txt 2>/dev/null | tail -n 60
+fi
+
+if has desktop; then
+  section "Screens' layout"
+  python3 - <<'PY' 2>/dev/null || echo "no ~/.config/frametop-layout.json"
 import json, os
 d = json.load(open(os.path.expanduser("~/.config/frametop-layout.json")))
 print("mode:", d.get("mode"), " auto:", d.get("auto"), " primary:", d.get("primary"))
@@ -84,9 +130,9 @@ for i, s in enumerate(d.get("screens", []), 1):
 print("visibility:", d.get("visibility"))
 PY

-section "Plasma panels and outputs"
-python3 "$repo/session/fix-panels.py" --check 2>&1 || true
-python3 - "$repo" <<'PY' 2>&1 || true
+  section "Plasma panels and outputs"
+  python3 "$repo/session/fix-panels.py" --check 2>&1 || true
+  python3 - "$repo" <<'PY' 2>&1 || true
 import json, subprocess, sys
 sys.path.insert(0, sys.argv[1] + "/layout")
 import ft_layout
@@ -103,8 +149,8 @@ r = subprocess.run(["qdbus6", "org.kde.plasmashell", "/PlasmaShell", "org.kde.Pl
 print("live panels:", (r.stdout or r.stderr).strip())
 PY

-section "Frametop's keyboard and the Steam menu"
-python3 - "$(ask frametop_relay devices)" <<'PY' 2>&1
+  section "Frametop's keyboard and the Steam menu"
+  python3 - "$(ask frametop_relay devices)" <<'PY' 2>&1
 import json, os, sys
 try:
    rules = json.load(open(os.path.expanduser("~/.config/frametop-input.json")))
@@ -121,85 +167,195 @@ print(f"  pointer mode: {d.get('pointer_mode')}, paused: {d.get('paused')}")
 for n in d.get("nodes", []):
    print("  " + json.dumps(n))
 PY
-echo "ft-textinput (tells the relay about text fields): $(pgrep -f '[f]t-textinput' >/dev/null && echo running || echo 'not running')"
-echo "the desktop's KWin input method: $(pgrep -a kwin_wayland | grep -o -- '--inputmethod [^ ]*' | head -n 1 | grep . || echo 'none (the keyboard never opens for text fields)')"
-for p in $(pgrep -x plasmashell); do
-  env=$(tr '\0' '\n' < /proc/$p/environ 2>/dev/null) || continue
-  echo "$env" | grep -q '^XDG_CONFIG_HOME=.*/frametop$' || continue
-  for v in QT_IM_MODULE GTK_IM_MODULE; do  # xim here: Qt and GTK apps never ask for the keyboard
-    printf "the desktop's %s: %s\n" $v "$(echo "$env" | sed -n "s/^$v=//p" | grep . || echo unset)"
+  echo "ft-textinput (tells the relay about text fields): $(pgrep -f '[f]t-textinput' >/dev/null && echo running || echo 'not running')"
+  echo "the desktop's KWin input method: $(pgrep -a kwin_wayland | grep -o -- '--inputmethod [^ ]*' | head -n 1 | grep . || echo 'none (the keyboard never opens for text fields)')"
+  for p in $(pgrep -x plasmashell); do
+    env=$(tr '\0' '\n' < /proc/$p/environ 2>/dev/null) || continue
+    echo "$env" | grep -q '^XDG_CONFIG_HOME=.*/frametop$' || continue
+    for v in QT_IM_MODULE GTK_IM_MODULE; do  # xim here: Qt and GTK apps never ask for the keyboard
+      printf "the desktop's %s: %s\n" $v "$(echo "$env" | sed -n "s/^$v=//p" | grep . || echo unset)"
+    done
  done
-done
-echo "ft-screens debug: $(ask ft_screens debug)"
-echo "ft-screens state (mode, manual, wrist, gesture hand and angle, lasers, game, in games): $(ask ft_screens state)"
-echo "ft-screens pause: $(ask ft_screens 'pause state'), concealed: $(ask ft_screens concealed)"
-echo "SteamVR overlays shown now:"
-XDG_CONFIG_HOME=$HOME/.config LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 timeout 10 \
-  /opt/steamvr/bin/linuxarm64/vrcmd --overlays 2>/dev/null |
-  awk '/^\x27/ { show = $0 !~ /not_visible/ } show' | sed 's/^/  /' | head -n 80
-echo "keyboard, drag and Steam menu lines from ft-screens (last 40):"
-grep -a -E 'keyboard|Steam (in front|out of the way)|: (move|resize|roll)( by| ended| not started)' \
-  /tmp/frametop-screens.log 2>/dev/null | tail -n 40 | sed 's/^/  /'
-echo "text fields, from the relay (last 20):"
-journalctl --user -u frametop-input-relay --no-pager -o short 2>/dev/null | grep 'text field' | tail -n 20 | sed 's/^/  /'
+  echo "ft-screens debug: $(ask ft_screens debug)"
+  echo "ft-screens state (mode, manual, wrist, gesture hand and angle, lasers, game, in games): $(ask ft_screens state)"
+  echo "ft-screens pause: $(ask ft_screens 'pause state'), concealed: $(ask ft_screens concealed)"
+  echo "SteamVR overlays shown now:"
+  XDG_CONFIG_HOME=$HOME/.config LD_LIBRARY_PATH=/opt/steamvr/bin/linuxarm64 timeout 10 \
+    /opt/steamvr/bin/linuxarm64/vrcmd --overlays 2>/dev/null |
+    awk '/^\x27/ { show = $0 !~ /not_visible/ } show' | sed 's/^/  /' | head -n 80
+  echo "keyboard, drag and Steam menu lines from ft-screens (last 40):"
+  grep -a -E 'keyboard|Steam (in front|out of the way)|: (move|resize|roll)( by| ended| not started)' \
+    /tmp/frametop-screens.log 2>/dev/null | tail -n 40 | sed 's/^/  /'
+  echo "text fields, from the relay (last 20):"
+  journalctl --user -u frametop-input-relay --no-pager -o short 2>/dev/null | grep 'text field' | tail -n 20 | sed 's/^/  /'

-section "Input relay (last 60 lines)"
-journalctl --user -u frametop-input-relay -n 60 --no-pager -o short 2>/dev/null
-section "Pointer helper (last 60 lines)"
-journalctl --user -u frametop-pointer -n 60 --no-pager -o short 2>/dev/null
-section "Power service (last 30 lines)"
-journalctl --user -u frametop-power -n 30 --no-pager -o short 2>/dev/null
-# Frametop Voice's log has what was said (each dictation window's words, transcripts, wake word
-# commands): those lines are left out of the report.
-if systemctl --user is-enabled frametop-voice >/dev/null 2>&1; then
-  section "Voice dictation (last 30 lines, without what was said)"
-  journalctl --user -u frametop-voice -u frametop-voice-whisper -n 60 --no-pager -o short 2>/dev/null |
-    grep -v -E 'transcribed|voice command|wake word:|window [0-9.]+s|decode skipped|line up with' | tail -n 30
+  section "Input relay (last 60 lines)"
+  journalctl --user -u frametop-input-relay -n 60 --no-pager -o short 2>/dev/null
+  section "Pointer helper (last 60 lines)"
+  journalctl --user -u frametop-pointer -n 60 --no-pager -o short 2>/dev/null
+  section "Power service (last 30 lines)"
+  journalctl --user -u frametop-power -n 30 --no-pager -o short 2>/dev/null
+  for f in /tmp/frametop-screens.log "$XDG_RUNTIME_DIR/frametop-layout.log"; do
+    section "$f (last 60 lines)"
+    tail -n 60 "$f" 2>/dev/null || echo "missing"
+  done
 fi
+
 # The gaze service, SteamVR's eye tracker and ours, the checks and calibrations, with what
 # looks wrong first. It wakes an idle gaze service for about 20 s, to see the tracker send.
-python3 "$repo/scripts/gaze-report.py" </dev/null 2>&1 || echo "scripts/gaze-report.py failed"
-for f in /tmp/frametop-session.log /tmp/frametop-screens.log "$XDG_RUNTIME_DIR/frametop-layout.log"; do
-  section "$f (last 60 lines)"
-  tail -n 60 "$f" 2>/dev/null || echo "missing"
-done
-section "SteamVR server: Frametop and errors (last 60 lines)"
-grep -a -i -E 'ft_pointer|frametop|\[error\]' $logs/vrserver.txt 2>/dev/null | tail -n 60
+if has gaze; then
+  python3 "$repo/scripts/gaze-report.py" </dev/null 2>&1 || echo "scripts/gaze-report.py failed"
+fi
+
+# Hand tracking: whether it runs, which models ft-hands loaded and whether they're the pinned ones,
+# the cameras (hands/camcheck.py), and ft-hands' own counts: how often each hand was seen, what it
+# lost, how long a step took. Its pinch and grip lines are left out (the counts are in the status).
+if has hands; then
+  section "Hand tracking"
+  python3 "$repo/hands/handsmode.py" status 2>&1 || true
+  units frametop-camd frametop-hands frametop-handview
+  student=$repo/hands/models/student
+  echo "fine-tuned models: $(cat "$student/VERSION" 2>/dev/null || echo 'none in this install (then the stock MediaPipe ones)')"
+  [ -d "$student" ] && "$repo/hands/models.sh" check 2>&1 | tail -n 3
+  echo "last checked for newer ones: $(date -r "$repo/hands/models/.student-checked" '+%F %T' 2>/dev/null || echo never)"
+  grep -E '^\s*(FT_)?HANDS_MODELS(_UPDATE)?=' ~/.config/frametop.conf 2>/dev/null | sed 's/\s*#.*//; s/^/override (ft-hands loads these instead): /'
+  python3 "$repo/hands/camcheck.py" 2>&1 | head -n 30
+  echo "sides: $(cat "$XDG_RUNTIME_DIR/frametop-hands/sides.json" 2>/dev/null || echo 'no sides.json')"
+  for c in 5 6 7; do   # ft-hands' CPUs (HANDS_CPUS)
+    f=/sys/devices/system/cpu/cpu$c/cpufreq
+    [ -r $f/scaling_cur_freq ] && echo "cpu$c: $(( $(cat $f/scaling_cur_freq) / 1000 )) of $(( $(cat $f/cpuinfo_max_freq) / 1000 )) MHz"
+  done
+  echo "hottest thermal zone: $(cat /sys/class/thermal/thermal_zone*/temp 2>/dev/null | sort -n | tail -n 1 | awk '{printf "%.1f C", $1 / 1000}')"
+  gestures='pinch (left|right)|grip  (left|right)'
+  echo "ft-hands' starts, cameras and side decisions (last 20):"
+  journalctl --user -u frametop-hands -n 5000 --no-pager -o short 2>/dev/null | grep -a -v -E "$noise|$gestures" |
+    grep -a -E 'cameras|models|colour|side cameras|hand model|heartbeat|error|failed|Failed|Started|not ' | tail -n 20 | sed 's/^/  /'
+  echo "ft-hands' last status (every minute):"
+  journalctl --user -u frametop-hands -n 5000 --no-pager -o cat 2>/dev/null | grep -a -v -E "$noise|$gestures" |
+    awk '/^[0-9.]+s +[0-9.]+ sets\/s/ { block = $0; next } /^ +(sets with|flag buffer|pinches)/ { block = block "\n" $0 }
+         END { print block }' | sed 's/^/  /'
+  # ft-camd prints each camera's rate once a minute: its other lines (starts, cameras, errors), then
+  # only the last 3 of those.
+  section "Camera broker ft-camd (last 20 lines but its rates, then its last 3 rates)"
+  journalctl --user -u frametop-camd -n 3000 --no-pager -o short 2>/dev/null | grep -a -v -E "$noise" > /tmp/ft-report-camd.$$
+  grep -a -v -E '\]: \[[0-9]+ s\] ' /tmp/ft-report-camd.$$ | tail -n 20
+  grep -a -E '\]: \[[0-9]+ s\] ' /tmp/ft-report-camd.$$ | tail -n 3
+  rm -f /tmp/ft-report-camd.$$
+fi
+
+# Frametop Voice. Its log has what was said (each dictation window's words, transcripts, wake word
+# commands): those lines are left out. The microphone: SteamVR's proxmicmute sets the mic's gain
+# (VA_DEC0/1) to 0 while nobody wears the headset, and PipeWire's echo canceller can fail at boot,
+# which leaves everything after it silent; the level check reads one second and keeps only its level.
+if has voice; then
+  section "Voice"
+  units frametop-voice frametop-voice-whisper steamvr-proxmicmute
+  echo "state: $(timeout 5 python3 "$repo/voice/ft-voice" status 2>&1 | tr -d '\n')"
+  python3 - "$repo" <<'PY' 2>&1 || true
+import sys
+sys.path.insert(0, sys.argv[1] + "/voice")
+import voice_settings as v
+s = v.load()
+print(f"settings ({v.PATH}{'' if v.PATH.exists() else ', not there: the defaults'}): controller {s['controller']} "
+      f"{s['controller_mode']}, live typing {s['live_typing']}, sounds {s['sounds']}, wake word {s['wake_word']} "
+      f"({len(s['wake_phrases'])} phrase(s)), {len(s['commands'])} command(s), advanced: {sorted(s['advanced']) or 'none'}")
+PY
+  [ -x "$repo/voice/models.sh" ] && timeout 60 "$repo/voice/models.sh" check 2>&1 | tail -n 4
+  echo "the mic's gain (0 = muted by proxmicmute, the headset off):" \
+    "VA_DEC0 $(amixer -c 0 cget numid=23 2>/dev/null | sed -n 's/.*: values=//p')," \
+    "VA_DEC1 $(amixer -c 0 cget numid=24 2>/dev/null | sed -n 's/.*: values=//p')"
+  r=$(ask ft_gazed status)
+  echo "headset worn (the gaze service): $(echo "$r" | python3 -c 'import json, sys; print(json.load(sys.stdin).get("headset_on"))' 2>/dev/null || echo unknown)"
+  echo "default source: $(wpctl inspect @DEFAULT_AUDIO_SOURCE@ 2>/dev/null | sed -n 's/.*node.name = "\(.*\)"/\1/p' | head -n 1)," \
+    "$(wpctl get-volume @DEFAULT_AUDIO_SOURCE@ 2>/dev/null)"
+  echo "PipeWire's capture chain (state, error):"
+  pw-dump 2>/dev/null | python3 -c '
+import json, sys
+for o in json.load(sys.stdin):
+    if o.get("type") == "PipeWire:Interface:Node":
+        i = o.get("info") or {}
+        p = i.get("props") or {}
+        if p.get("media.class") in ("Audio/Source", "Stream/Input/Audio"):
+            name, state, error = p.get("node.name"), i.get("state"), i.get("error")
+            print(f"  {name}: {state}" + (f", {error}" if error else ""))
+' 2>&1 || echo "  pw-dump failed"
+  echo "who has the capture device open: $(fuser /dev/snd/pcmC*c 2>/dev/null | xargs -r -n 1 sh -c 'cat /proc/$0/comm' 2>/dev/null | sort | uniq -c | xargs | grep . || echo nobody)"
+  python3 -I - <<'PY' 2>&1
+import math, subprocess
+from array import array
+try:
+    raw = subprocess.run(["timeout", "1.2", "pw-record", "--raw", "--rate", "16000", "--channels", "1", "--format", "s16", "-"],
+                         capture_output=True).stdout
+except OSError as e:
+    raise SystemExit(f"mic level: can't run pw-record ({e})")
+a = array("h", raw[: len(raw) // 2 * 2])
+if not a:
+    raise SystemExit("mic level: no samples from the default source")
+peak = max(abs(x) for x in a) / 32768
+rms = math.sqrt(sum(x * x for x in a) / len(a)) / 32768
+print("mic level over 1 s: " + ("digital silence (all zeros)" if peak == 0 else
+      f"peak {20 * math.log10(peak):.0f} dBFS, RMS {20 * math.log10(max(rms, 1e-9)):.0f} dBFS"))
+PY
+  echo "PipeWire and WirePlumber errors (last 15):"
+  journalctl --user -u pipewire -u wireplumber -n 3000 --no-pager -o short 2>/dev/null | grep -a -i -E 'error|cannot|failed' | tail -n 15 | sed 's/^/  /'
+  section "Voice dictation (last 30 lines, without what was said)"
+  journalctl --user -u frametop-voice -u frametop-voice-whisper -n 200 --no-pager -o short 2>/dev/null | grep -a -v -E "$noise" |
+    grep -v -E 'transcribed|voice command|wake word:|window [0-9.]+s|decode skipped|line up with' | tail -n 30
+fi
 EOF

 if [ "$watch" -gt 0 ]; then
  echo
-  echo "Now make the problem happen within $watch seconds: for example, open the Steam menu, try to"
-  echo "drag a window by its bar, and click into a text field. Recording..."
-  on_frame_script "$watch" >> "$out" 2>&1 <<'EOF' || true
-seconds=$1
-printf '\n===== Watched for %s s from %s\n' "$seconds" "$(date '+%F %T')"
+  echo "Now make the problem happen within $watch seconds. For example:"
+  [[ ,$areas, == *,desktop,* ]] && echo "  desktop: open the Steam menu, drag a window by its bar, click into a text field"
+  [[ ,$areas, == *,gaze,* ]] && echo "  gaze: point with your eyes, at what goes wrong"
+  [[ ,$areas, == *,hands,* ]] && echo "  hands: pinch and tap, move your hands in and out of view, where tracking fails"
+  [[ ,$areas, == *,voice,* ]] && echo "  voice: dictate, or say your wake word (only the mic's level and word counts are kept)"
+  echo "Recording..."
+  on_frame_script "$FRAME_REPO" "$areas" "$watch" >> "$out" 2>&1 <<'EOF' || true
+repo=$1 areas=$2 seconds=$3
+has() { [[ ,$areas, == *,$1,* ]]; }
+noise='container (exec|exec_died|init|start|died|attach|cleanup)'
 since=$(date '+%F %T')
 start=$(wc -l < /tmp/frametop-screens.log 2>/dev/null || echo 0)
-python3 -I - "$seconds" <<'PY'
-import socket, sys, time
-s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
-s.bind("")
-s.settimeout(1)
-end, last = time.monotonic() + float(sys.argv[1]), None
-print("ft-screens' debug line whenever it changed:")
-while time.monotonic() < end:
-    try:
-        s.sendto(b"debug", "\0ft_screens")
-        line = s.recv(4096).decode()
-    except OSError as e:
-        line = f"no answer ({e})"
-    if line != last:
-        print(time.strftime("%H:%M:%S") + f"{time.time() % 1:.2f}"[1:], line, flush=True)
-        last = line
-    time.sleep(0.2)
-PY
-printf '\n===== ft-screens log while watching\n'
-tail -n +"$((start + 1))" /tmp/frametop-screens.log 2>/dev/null | tail -n 300
-for u in frametop-input-relay frametop-pointer; do
-  printf '\n===== %s while watching\n' "$u"
-  journalctl --user -u $u --since "$since" --no-pager -o short 2>/dev/null | tail -n 200
-done
+printf '\n===== Watched %s for %s s from %s\n' "${areas//,/, }" "$seconds" "$since"
+if [ -f "$repo/scripts/report-watch.py" ]; then
+  python3 -I "$repo/scripts/report-watch.py" "$seconds" "$areas"
+else
+  echo "scripts/report-watch.py isn't in this install (older than 0.3.0-exp.12): logs only"
+  sleep "$seconds"
+fi
+log() {  # log TITLE UNIT...: their journal while watching
+  local title=$1; shift
+  printf '\n===== %s while watching\n' "$title"
+  journalctl --user $(printf -- '-u %s ' "$@") --since "$since" --no-pager -o short 2>/dev/null | grep -a -v -E "$noise" | tail -n 300
+}
+if has system; then
+  printf '\n===== warnings and errors while watching\n'
+  journalctl --user -p warning --since "$since" --no-pager -o short 2>/dev/null | grep -a -v -E "$noise" | tail -n 100
+  journalctl -k -p warning --since "$since" --no-pager -o short 2>/dev/null | tail -n 50
+fi
+if has desktop; then
+  printf '\n===== ft-screens log while watching\n'
+  tail -n +"$((start + 1))" /tmp/frametop-screens.log 2>/dev/null | tail -n 300
+  log "input relay" frametop-input-relay
+  log "pointer helper" frametop-pointer
+fi
+if has gaze; then
+  log "gaze service" frametop-gaze
+  has desktop || { printf '\n===== the pointer helper'"'"'s gaze lines while watching\n'
+    journalctl --user -u frametop-pointer --since "$since" --no-pager -o short 2>/dev/null | grep -a -i gaze | tail -n 100; }
+fi
+if has hands; then
+  log "ft-hands (pinches and grips too)" frametop-hands
+  log "ft-camd" frametop-camd
+fi
+if has voice; then
+  printf '\n===== Frametop Voice while watching (without what was said)\n'
+  journalctl --user -u frametop-voice -u frametop-voice-whisper --since "$since" --no-pager -o short 2>/dev/null |
+    grep -a -v -E "$noise" | grep -v -E 'transcribed|voice command|wake word:|window [0-9.]+s|decode skipped|line up with' | tail -n 200
+  log "PipeWire, WirePlumber and proxmicmute" pipewire wireplumber steamvr-proxmicmute
+fi
 EOF
  echo "Recorded."
 fi
@@ -42,7 +42,7 @@ POINTER_HEAD_DEADZONE=0.5  # keyboard clicks at the gaze (Meta+J, Meta+K): degre
 POINTER_KEY_TAP=0.25       # keyboard clicks: let go within this (s) and it clicks where the dot was at the press, and tells the gaze tracker it was right
 GAZE_TRACKER=auto          # gaze service: auto = our own eye tracker when it's installed (gaze/tracker/install.sh, which install.sh offers), else SteamVR's | own = ours | steam = SteamVR's. Each keeps its own calibration (Calibrate on Frametop Input Settings' Gaze page)
 GAZE_EYE=auto              # gaze service: eye bias. auto = each eye weighted by how far off it was at your recent nudges | left | right = that eye counts twice
-HANDS_MODE=off             # hand tracking (hands/run.sh install), when it runs: frametop = while the Frametop desktop runs | always = whenever SteamVR runs, in games and the dashboard too | off = only by hand (ft-handsctl on). Frametop Input Settings' Hands page applies it at once
+HANDS_MODE=off             # hand tracking (hands/run.sh install; install.sh sets frametop when it installs it), when it runs: frametop = while the Frametop desktop runs | always = whenever SteamVR runs, in games and the dashboard too | off = only by hand (ft-handsctl on). Frametop Input Settings' Hands page applies it at once
 HANDS_SWAP_SIDES=auto      # hand tracking: which side camera is which. auto = ft-hands tells from the hands and fixes ft-camd's names, which some SteamVR restarts swap | 0 = keep ft-camd's | 1 = exchange them
 HANDS_CPUS=5,6,7           # hand tracking: the CPUs its model threads run on
 HANDS_CAMERAS=auto         # hand tracking: auto (by the light) | mono (the IR cameras; also keeps ft-camd off the colour ones) | color | all
@@ -51,7 +51,8 @@ HANDS_BRIGHT_ON=40         # hand tracking, auto: the colour frames' mean bright
 HANDS_BRIGHT_OFF=25        # ... and back to the mono cameras under this
 HANDS_COLOR_LEFT=color_video0  # hand tracking: which colour camera is passthrough_left (hands/tools/check_color.py tells)
 HANDS_COLOR_CROP=subtract  # hand tracking: how the colour calibration's crop applies, subtract | none (check_color.py tells)
-#HANDS_MODELS=            # hand tracking: a folder of palm.ncnn.* and hand.ncnn.* to use instead of the stock models (hands/models/ncnn)
+#HANDS_MODELS=            # hand tracking: a folder of palm.ncnn.* and hand.ncnn.* to use instead of the fine-tuned models (hands/models/student; hands/models/ncnn has the stock ones)
+HANDS_MODELS_UPDATE=1      # hand tracking: 1 = when it starts, fetch newer fine-tuned models if GitHub's main (stable) or experimental branch pins them (hands/models.pin), without a new release | 0 = keep the ones it came with
 HANDS_MATTE=circle         # hand cutouts: circle = a round hole per hand | mesh = hand-shaped, from the skeleton | capsules = ft-hands' capsules (the old look)
 HANDS_MATTE_MARGIN=1.5     # hand cutouts: degrees (seen from the eyes) the hole reaches past the hand, held still (Hand cutout margin on the Hands page, live)
 HANDS_MATTE_MOTION=25      # ... plus how far the hand moves in this many ms
@@ -167,6 +168,6 @@ DISPLAY_MOVE_DEG=0.5 # displays off: or turning this far (degrees) within 10 s
 LAYOUT_GRAB_OFFSET=0.075   # arranging screens: where the floating window's grab bar is, in metres below the screen
 LAYOUT_SLIDE_SPEED=0.5     # arranging screens: how fast the invisible controller slides a grabbed screen (m/s)

-HANDS_PINCH_END=0.035      # ft-hands: a pinch ends once thumb and index are this far apart (m)
-HANDS_PINCH_DRAG_OPEN=0.05 # ... a pinch held 0.3 s or more (a drag) ends this far apart, or HANDS_PINCH_DRAG_OPEN_S (0.12) past HANDS_PINCH_END
-HANDS_GRIP_TAKEOVER_S=0.3  # ... a grip (fist) ends a pinch only this soon after it began
+HANDS_PINCH_END=0.045      # ft-hands: a pinch ends once thumb and index are this far apart (m)
+HANDS_PINCH_DRAG_OPEN=0.065 # ... a pinch held 0.3 s or more (a drag) ends this far apart, or HANDS_PINCH_DRAG_OPEN_S (0.2) past HANDS_PINCH_END
+HANDS_GRIP_TAKEOVER_S=0.1  # ... a grip (fist) ends a pinch only this soon after it began
@@ -18,7 +18,7 @@ packages=(
  mesa-libgbm-devel wayland-devel vulkan-loader-devel vulkan-headers plasma-wayland-protocols wlroots-devel
  # ft_pointer SteamVR driver: static C++ runtime (the host has an older glibc)
  libstdc++-static
-  # hand tracking (deferred: not installed by install.sh; hands/run.sh install builds it): ft-hands
+  # hand tracking (install.sh offers it; hands/run.sh install builds it): ft-hands
  # reads the calibration with jsoncpp; ft-camd runs on the host, linked statically. Its Python
  # tools (hands/tools) need NumPy and OpenCV, which aren't here: Fedora's python3-opencv pulls
  # in over a gigabyte (hands/README.md says how to get them)
@@ -70,7 +70,8 @@ try:
    import gi
    gi.require_version("GdkPixbuf", "2.0")
    from gi.repository import Gio  # noqa: F401
-except (ImportError, ValueError):
+# AttributeError: a "gi" that isn't PyGObject (the release image's test stage has one).
+except (ImportError, ValueError, AttributeError):
    print("skip  ft-spotwatch itself: no PyGObject here")
    sys.exit(1 if failed else 0)
Author	SHA1	Message	Date
DeeJanuzandClaude Opus 5.5	01796eb299	Releases: ship the Hand Recorder The image builds ft-handpanel (hands/rec/build.sh, against scripts/openvr.sh like gaze), and a release's installer offers the recorder: ft-camd's capabilities and the menu entry, nothing to build. The FrameDrop window has a Hand Recorder box, and --no-hand-recorder leaves it out. An entry from the release before is renewed without asking, since it points into that release's folder. The recorder runs ft-hands through scripts/in-box instead of the "dev" container, so a release uses its own container, and its repair hints say to reinstall the release rather than run a build script. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 14:32:07 -06:00
DeeJanuzandClaude Opus 5.5	3e6dba8276	Report: an area per flag, and --watch records each area live scripts/report.sh takes --system, --desktop, --gaze (--eyes), --hands and --voice; with none it reports on all of them, as before. Every report keeps the versions and the settings. New in it: CPU, memory, heat and the busiest processes (--system), and Frametop Voice (--voice): its services, state, settings (counts, not phrases or commands), speech models, and the microphone: proxmicmute's gain, PipeWire's capture chain with its errors, who holds the capture device, and its level over one second (no sound is kept). --watch SECONDS now runs scripts/report-watch.py for the chosen areas, one thread each: ft-screens' debug line (desktop), the gaze service's status (gaze), ft-hands' published frames read under their sequence lock each second (rate, age, each hand's share and confidence, closest thumb-index and thumb-middle tips) and the side cameras' decision (hands), Frametop Voice's events with what was said replaced by word counts and the mic's level each second with its gain when it goes silent (voice), and load, clocks, heat, memory and the busiest processes every 5 s (system). Then each area's logs from that time. README's Reporting problems says how. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 14:16:16 -06:00
DeeJanuzandClaude Opus 5.5	098ba9e71e	Hands: pinches let go less easily by default Fresh installs ended taps and pinches at 3.5 cm, the values the headset's tester found "release way too easily" on 2026-10-11 (ends came at 4-6 cm, grips took pinches 130 ms in). The headset has run with HANDS_PINCH_END=0.045, HANDS_PINCH_DRAG_OPEN=0.065, HANDS_PINCH_DRAG_OPEN_S=0.2 and HANDS_GRIP_TAKEOVER_S=0.1 since; they are now the defaults. pinch_test keeps the old values for its rule tests and checks the new defaults on their own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 13:17:00 -06:00
DeeJanuzandClaude Opus 5.5	eb31589aed	Report: hand tracking's state (models, cameras, sides, ft-hands counts) scripts/report.sh had no hand tracking at all, so a "tracking is bad" report couldn't tell a stock model from the pinned one, a failed camera start from bad light, or a starved tracker. Its new section: HANDS_MODE and the three services, the fine-tuned models' version and models.sh check, any HANDS_MODELS override, hands/camcheck.py, sides.json, CPU 5-7 clocks and the hottest thermal zone, ft-hands' starts and last status, and ft-camd's start lines. --watch adds ft-hands' log (pinches) too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 13:17:00 -06:00
DeeJanuzandClaude Opus 5.5	1b256289b9	Hands: new hand models reach installs without a release The pin moves out of models.sh into hands/models.pin. When hand tracking starts, frametop-hands.service runs `hands/models.sh auto`: it reads models.pin from GitHub (main for stable releases, experimental for experimental ones and clones on that branch), and if it pins other models, fetches them into models/.student-new, checks every sha256 and swaps the folder in whole with one rename. At most every 15 minutes and within a minute; offline, a 404 or anything not matching keeps the models it has. A pin from GitHub is checked line by line before use: a 40-hex commit, a plain folder name, file names from a fixed list, 64-hex sums, and a format number, so a model that needs a newer ft-hands can be held back from older installs. The Hugging Face repo stays fixed in the script. HANDS_MODELS_UPDATE=0 turns it off; `hands/models.sh update` checks now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 09:46:01 -06:00
DeeJanuzandClaude Opus 5.5	9eac1e29a9	Hands: ship the fine-tuned hand models (v13b-p1) instead of stock MediaPipe hands/models.sh fetches MediaPipe's palm detector and hand landmark model, retrained on the Frame's infrared cameras with the Frametop Hands dataset, from a pinned commit of the Hugging Face repo DeeJanuz/frametop-hands-models, and checks every file's sha256. hands/build.sh runs it, so source installs and the release image both get them; the release smoke test checks them. The weights are CC BY-NC 4.0, not MIT, so they stay out of this repository (gitignored, dockerignored); their LICENSE and model card come with them. ft-hands and ft-handreplay now default to hands/models/student when it's there (HANDS_MODELS and --models still override; --int8 keeps the stock models, which are the only ones with 8-bit files). Replays through ft-hands, hands tracked, stock -> v13b-p1: bright room 442 -> 960, dim 912 -> 1224, lit 782 -> 1433. On held-out contributors the landmark error drops from 5.2 to 3.8 px (median). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-14 08:30:04 -06:00
DeeJanuz	4d4825bf9a	Install: hand tracking is an install option, in releases too install.sh offers hand tracking (step 9/12, yes by default; --no-hands skips it, and --yes installs it only when sudo runs without a prompt). Installed, it runs with the Frametop desktop (HANDS_MODE=frametop), unless the settings already say always. get.sh, pack/install-release.sh and the FrameDrop window pass --no-hands, and the window has a hand tracking box that needs the password like the eye tracker's. The release image builds ft-camd and ft-hands (with ncnn at the tag hands/Makefile pins), and hands/run.sh install doesn't build in a release, so it only sets ft-camd's file capabilities and installs the units. The release workflow's smoke test checks both binaries. Docs drop "deferred" and describe the gestures and the Hands page.	2026-10-13 22:22:37 -06:00
DeeJanuzandClaude Opus 5.5	e6dd58c406	spotwatch test: skip ft-spotwatch when "gi" isn't PyGObject The release image's test stage imports a "gi" without require_version, which failed the v0.3.0-exp.8 build instead of skipping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-13 22:00:27 -06:00