Hands: fixes from the first headset test

- The runtime files move to /run/user/UID/frametop-hands/: the desktop session deletes /run/user/UID/frametop at every start. - The cutout copy shader runs at highp: mediump (16-bit on Adreno) stepped 1.7 texels across a 3440-pixel screen. - One hand no longer pinches both sides after its left/right call flips mid-pinch, and --pinch-palm-down (0.6) holds back pinches with the palm facing down (typing on a lap keyboard). - ft-camd judges a colour frame fresh by its luma rows only, and logs per-buffer changes at stale colour frames with FT_CAMD_DEBUG=1. - hands/run.sh caps skips the setcap when ft-camd already has them. - The replay tool dumps poses (--poses), and its pinch events carry the hand id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 00:00:07 +02:00 · 2026-09-30 14:00:54 -06:00
1 parent 4e5131d8ce
commit e2abaa06b6
@@ -14,10 +14,19 @@ Steps 1-5 are done:
 - Built in the dev container on the Frame, and on the 7i.
 - Checked without the headset:
  - `ft-handreplay` against frame-hands' `fh-replay`, both x86 with `--cost`, on the whole dim recording and the first 60 s of the bright one: identical summaries and byte-identical depth dumps. The Makefile's own ncnn build is included in that.
-  - `ft-ringplay` into `ft-hands` on the 7i tracked, pinched, and wrote `/run/user/UID/frametop/{hands,gestures}`.
+  - `ft-ringplay` into `ft-hands` on the 7i tracked, pinched, and wrote `/run/user/UID/frametop-hands/{hands,gestures}`.
  - On the Frame, ft-hands in the container finds the calibration through `/run/host/persist`, and ft-camd without its capabilities refuses with a clear message.

-Next is step 6, with the user: `hands/run.sh install` (sudo setcap), then a desktop restart from this branch so ft-screens reads the new path.
+Step 6 has started (2026-09-30 10:30):
+- `hands/run.sh install` is done, and both services run from this worktree.
+- The files moved to `/run/user/UID/frametop-hands/`, because `/run/user/UID/frametop` is the desktop session's own runtime folder, deleted at every desktop start.
+- Until the desktop restarts from a build with this branch's ft-screens, the link `/run/user/UID/frame-hands -> frametop-hands` feeds the running one. It's tmpfs, so it's gone at reboot.
+
+Found in the headset:
+- The side cameras were swapped (`HANDS_SWAP_SIDES=1`).
+- The cutout copy shader lost resolution at `mediump` (now `highp`).
+- Colour capture isn't reliable (see the README).
+- Two pinch fixes: one hand no longer pinches both sides, and the palm-down limit stops typing pinches.

 ## What frame-hands is today

@@ -66,8 +75,8 @@ Programs within 15 characters, `ft-` prefix; files under `frametop`:
 | `fh-camd` | `ft-camd` |
 | `fh-tracker` | `ft-hands` |
 | `fh-replay`, `fh-ringplay` | `ft-handreplay`, `ft-ringplay` |
-| `/run/frame-hands/ir-ring` | `$XDG_RUNTIME_DIR/frametop/cam-ring` |
-| `$XDG_RUNTIME_DIR/frame-hands/hands`, `gestures` | `$XDG_RUNTIME_DIR/frametop/hands`, `gestures` |
+| `/run/frame-hands/ir-ring` | `/run/user/UID/frametop-hands/cam-ring` |
+| `$XDG_RUNTIME_DIR/frame-hands/hands`, `gestures` | `/run/user/UID/frametop-hands/hands`, `gestures` |

 The source keeps its `fh_` identifiers and header names (`fh_hands.h`, `fh_gestures.h`, `fhring.h`), and the file formats keep their magic strings, so recordings and tools from frame-hands keep working. Programs, units and runtime paths change.

@@ -173,7 +173,7 @@ power/run.sh log

 Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen. The same tracker also detects pinches, for clicking where you look with the gaze pointer (not wired to the pointer yet). It's optional: `hands/run.sh install`, or the last step of `install.sh`.

- `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`.
+- `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`.
 - `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches) next to the ring.
 - Both start and stop with SteamVR. `hands/run.sh status` and `hands/run.sh log` show how they're doing.
 - Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (after some SteamVR restarts the side cameras' names come out swapped, and hands land beside the holes; `hands/tools/check_sides.py --ring` tells) and `HANDS_CPUS`.
@@ -24,7 +24,7 @@ Settings in `~/.config/frametop.conf` (`FT_<name>` in the environment overrides
 - `HANDS_SWAP_SIDES=1`: the two side cameras' names are swapped (see ft-camd below). Check with `tools/check_sides.py --ring`.
 - `HANDS_CPUS=5,6,7`: the CPUs the model threads run on (below).

-Files, all in `/run/user/UID/frametop/` (private to the user):
+Files, all in `/run/user/UID/frametop-hands/` (private to the user; not `/run/user/UID/frametop/`, which the desktop session deletes whenever it starts):

 | File | Written by | Layout | Read by |
 | --- | --- | --- | --- |
@@ -58,6 +58,7 @@ Options:
 - `--with-dark`: also publish the near-black frames, as extra ring cameras flagged `FH_CAM_DARK`. They show only light sources, so they're no use for hands.
 - `--with-color`: also publish the two Arcturus colour cameras, flagged `FH_CAM_COLOR`. Each is the luma of the 10-bit frame's valid 1972x2464 (the top 8 bits), at half size (`--color-scale 2`: 986x1232) and at most 30 fps (`--color-fps`; the cameras run at 60). Frames that carry the module's warped half-size copy are dropped. Their `capture_ns` is on the colour module's clock (2.2 s off the mono cameras' on 2026-09-29), so line them up with the mono cameras by `dqbuf_ns`. Each frame costs about 0.65 ms of cache sync and 1.1 ms of decoding, so both cameras at 30 fps take about 11% of a core.
 - The ring holds 8 cameras: 4 mono, plus 4 dark twins or 2 colour cameras.
+- Colour isn't reliable yet. In the lit-room test of 2026-09-30, the colour cameras kept losing their buffer mapping: 30 frames in a row looked unchanged, the camera relearned, and after 5 relearns ft-camd exited. Each relearn samples all 32 colour buffers, which also made the mono cameras miss frames. Runs with the headset idle (no hands, no cutouts) had none of this, and no half-size copies either, while the failing runs had many. So the passthrough compositor may be writing into the colour buffers while Room View shows. Whether a frame is new is judged on the luma rows only: the chroma after them hardly changes in a lit room. `FT_CAMD_DEBUG=1` prints, at each colour stale frame, how many sampled words changed in every candidate buffer.
 - `--sensor S`: only the mono cameras whose sensor name contains S.
 - `--status S`: a status line every S seconds (0: never).

@@ -114,7 +115,9 @@ How good the depth is, measured from recordings (2026-09-30, `--depth` below): t
 ft-hands detects a pinch per hand (`track/pinch.h`) and publishes it to the gestures file. The layout, and how to read it without missing quick taps, is in `include/fh_gestures.h`.

 - A pinch begins when the thumb and index tips come within `--pinch-begin` (default 0.020 m). It ends when they open past `--pinch-end` (0.035 m) for 2 processed frames in a row, or when the hand stays lost for 0.25 s (flagged lost).
- The distance comes from MediaPipe's world landmarks: the model's own 3D hand pose, averaged over the hand's views, at the user's hand size. `--pinch-triangulated` uses the triangulated tips instead. On two recordings without deliberate pinches, the world landmarks came under 2 cm in 0.2-1% of frames, against 3.3-4.5% for the triangulated tips. Typing still gave 2 pinches a minute, so a consumer should only act on a pinch while the gaze is on a target.
+- The distance comes from MediaPipe's world landmarks: the model's own 3D hand pose, averaged over the hand's views, at the user's hand size. `--pinch-triangulated` uses the triangulated tips instead. On two recordings without deliberate pinches, the world landmarks came under 2 cm in 0.2-1% of frames, against 3.3-4.5% for the triangulated tips. In the dim recording, typing still gave 2 pinches a minute before the palm check below.
+- No pinch begins while the palm faces down (`--pinch-palm-down MAX`: the palm normal's share of the head's up axis, default 0.6; 1 turns it off), and a close held back that way has to open again before a pinch can begin. Typing curls the thumb onto the index. In the lit recording of 2026-09-30, typing on a keyboard in the lap began 23 pinches in about 2 minutes, all with the palm facing down (0.69-1.00), while the 26 deliberate ones read 0.00-0.50. The limit held back every typing pinch and none of the deliberate ones. Looking down tilts the head frame, which lowers the reading for a hand on a keyboard, so the consumer's gaze check stays the other guard.
+- A hand a pinch is down on stays with that side until the pinch ends. The left/right call is a running average of the model's, and when it flipped mid-pinch, the other side took the same hand and both sides pinched at once.
 - The pinch point is midway between the thumb and index tips. A drag is the pinch point now, minus where it was when the pinch began, both turned into the room with the HMD pose at their capture times.
 - `tools/watch_gestures.py` prints begins, ends and drag offsets live, and `--distance` prints each hand's distance.

@@ -6,7 +6,7 @@
 *   2. borrows those buffers read-only with pidfd_getfd,
 *   3. opens the v4l2_dqbuf tracepoint (tp.c).
 * Then it gives up its privileges for good, creates the frame ring in the
- * user's runtime folder (/run/user/UID/frametop/cam-ring, fhring.h), learns
+ * user's runtime folder (/run/user/UID/frametop-hands/cam-ring, fhring.h), learns
 * which buffer holds which V4L2 index, and copies each complete bright frame
 * into the ring. It exits when XRService exits or reallocates its buffers;
 * start it again (or let systemd) to re-attach.
@@ -230,8 +230,11 @@ static void setup_camera(cam_t *c, xr_camera_t *cam, int pidfd)
    memset(c, 0, sizeof(*c));
    c->cam  = cam;
    xr_camera_layout(cam, &c->lay);
-    c->need = (size_t)c->lay.pitch * c->lay.rows;
    c->color = c->lay.fmt == XR_FMT_YUV420_10P;
+    /* Color: just the luma rows, not the chroma after them. Whether a frame is new is judged
+     * by the last rows sampled changing, and in a lit room the chroma has too little noise to
+     * change between frames, so every frame looked stale and the buffers kept being relearned. */
+    c->need = (size_t)c->lay.pitch * (c->color ? c->lay.height : c->lay.rows);
    c->out_w = c->color ? c->lay.width / opt_color_scale : c->lay.width;
    c->out_h = c->color ? c->lay.height / opt_color_scale : c->lay.height;

@@ -678,6 +681,23 @@ static void on_frame(cam_t *c, int64_t index, uint32_t seq, uint64_t ts, uint64_

    if (found < 0 && !fresh(cur, c->samp[slot], &nchanged)) {
        buf_sync(c->fd[slot], DMA_BUF_SYNC_END);
+        if (c->color && getenv("FT_CAMD_DEBUG")) {
+            /* which candidate buffers changed since this camera last looked, whole and tail */
+            static uint64_t d[NSAMP];
+            fprintf(stderr, "stale %s t %.3f index %lld seq %u slot %d changed %d:", c->slug, evtime / 1e9,
+                    (long long)index, seq, slot, nchanged);
+            for (int s = 0; s < c->nslots; s++) {
+                int n, tail = 0;
+                buf_sync(c->fd[s], DMA_BUF_SYNC_START);
+                sample_words(c->map[s], c->need, d);
+                buf_sync(c->fd[s], DMA_BUF_SYNC_END);
+                fresh(d, c->samp[s], &n);
+                for (int i = NSAMP - NSAMP / 8; i < NSAMP; i++)
+                    tail += d[i] != c->samp[s][i];
+                fprintf(stderr, " %d/%d", n, tail);
+            }
+            fprintf(stderr, "\n");
+        }
        c->stale++;
        c->rc->dropped++;
        if (++c->stale_run >= STALE_RELEARN) {
@@ -783,7 +803,7 @@ static char ring_dir[64], ring_file[96];
 /* Created once privileges are gone, as the user, in a folder only the user can write. */
 static uint8_t *ring_create(size_t *len_out)
 {
-    snprintf(ring_dir, sizeof(ring_dir), "/run/user/%d/frametop", (int)getuid());
+    snprintf(ring_dir, sizeof(ring_dir), "/run/user/%d/frametop-hands", (int)getuid());
    snprintf(ring_file, sizeof(ring_file), "/run/user/%d/" FH_RING_NAME, (int)getuid());

    if (mkdir(ring_dir, 0700) < 0 && errno != EEXIST)
@@ -1,7 +1,7 @@
 /*
 * fhring - the shared-memory frame ring ft-camd writes and trackers read.
 *
- * One file, /run/user/UID/frametop/cam-ring (FH_RING_NAME in the user's runtime
+ * One file, /run/user/UID/frametop-hands/cam-ring (FH_RING_NAME in the user's runtime
 * folder; the folder is private to the user), holds a header, then for each camera
 * a few slots, each a slot header followed by the image rows packed tightly
 * (stride == width for 8-bit mono). Only complete, bright frames are published.
@@ -25,7 +25,7 @@
 #define FH_RING_VERSION   1
 #define FH_RING_MAX_CAMS  8
 #define FH_RING_SLOTS     4
-#define FH_RING_NAME      "frametop/cam-ring"    /* in /run/user/UID */
+#define FH_RING_NAME      "frametop-hands/cam-ring"    /* in /run/user/UID */

 enum {
    FH_FMT_GREY8 = 0,
@@ -8,7 +8,7 @@ PartOf=steamvr.service

 [Service]
 # In the dev container (it's built against Fedora's libraries). It reads ft-camd's ring and
-# writes /run/user/UID/frametop/hands and gestures. Settings: HANDS_* in ~/.config/frametop.conf.
+# writes /run/user/UID/frametop-hands/hands and gestures. Settings: HANDS_* in ~/.config/frametop.conf.
 ExecStartPre=-@REPO@/scripts/container-up.sh
 ExecStartPre=-/usr/bin/pkill -x ft-hands
 ExecStart=%h/.local/bin/distrobox enter dev -- @REPO@/hands/build/ft-hands --status 60
@@ -1,7 +1,7 @@
 /*
 * fh_gestures - pinch state ft-hands publishes for input: look at something and pinch to
 * click it, pinch and move to drag (the Vision Pro model, with the eye tracker doing the
- * looking). /run/user/UID/frametop/gestures, next to the hands
+ * looking). /run/user/UID/frametop-hands/gestures, next to the hands
 * file, with the same sequence lock (read seq, copy, read seq again; use the copy only if
 * both reads are the same even number) and the same frame: metres in the head frame at
 * capture time, OpenVR's HMD frame (+x right, +y up, -z forward).
@@ -1,6 +1,6 @@
 /*
 * fh_hands - the tracked-hands file ft-hands publishes for ft-screens' hand cutouts
- * (/run/user/UID/frametop/hands, directory mode 0700), rewritten in place
+ * (/run/user/UID/frametop-hands/hands, directory mode 0700), rewritten in place
 * under a sequence lock: read seq, copy, read seq again; use the copy only if
 * both reads are the same even number.
 *
@@ -28,9 +28,13 @@ sudo_run() {
  printf '%s\n' "$pw" | on_frame "sudo -S -p '' bash -c $(printf %q "$1")"
 }

-set_caps() {
+set_caps() {  # only when missing: a rebuild clears them, a reinstall doesn't
  local bin
  bin=$(printf %q "$FRAME_REPO/hands/build/ft-camd")
+  if on_frame "getcap $bin | grep -q cap_sys_ptrace"; then
+    echo "ft-camd has its capabilities"
+    return
+  fi
  sudo_run "setcap $caps $bin && getcap $bin"
 }

@@ -6,7 +6,7 @@ import time

 import numpy as np

-RING_FILE = '/run/user/%d/frametop/cam-ring' % os.getuid()
+RING_FILE = '/run/user/%d/frametop-hands/cam-ring' % os.getuid()
 MAGIC = b'FHRING01'
 HDR = struct.Struct('<8sIIIIQqQ16x')                     # 64 bytes
 CAM = struct.Struct('<32s32siIIIIIQQQQQ32x')             # 160 bytes
@@ -22,7 +22,7 @@ SIDES = ('left ', 'right')


 def path():
-    return '/run/user/%d/frametop/gestures' % os.getuid()
+    return '/run/user/%d/frametop-hands/gestures' % os.getuid()


 def read(m):
@@ -114,7 +114,7 @@ void put(fh_capsule_t *caps, uint32_t &n, V3 a, V3 b, double ra, double rb) {
 }  // namespace

 std::string run_dir() {
-    const std::string dir = "/run/user/" + std::to_string(getuid()) + "/frametop";
+    const std::string dir = "/run/user/" + std::to_string(getuid()) + "/frametop-hands";
    mkdir(dir.c_str(), 0700);
    return dir;
 }
@@ -45,7 +45,8 @@ private:
 uint64_t mono_ns();
 int64_t raw_minus_mono_ns();   // camera timestamps are CLOCK_MONOTONIC_RAW

-// /run/user/UID/frametop, created private to the user if it's missing: where ft-camd's ring
+// /run/user/UID/frametop-hands, created private to the user if it's missing: where ft-camd's ring
 // (FH_RING_NAME) and the hands and gestures files live. Not $XDG_RUNTIME_DIR: a terminal in
-// the Frametop desktop has a private one of its own.
+// the Frametop desktop has a private one of its own. And not /run/user/UID/frametop: that is
+// the desktop session's private runtime folder, which it deletes whenever it starts.
 std::string run_dir();
@@ -123,6 +123,7 @@ int main(int argc, char **argv) {
        else if (a == "--pinch-begin" && more) pinch_params.begin_m = std::atof(argv[++i]);
        else if (a == "--pinch-end" && more) pinch_params.end_m = std::atof(argv[++i]);
        else if (a == "--pinch-triangulated") pinch_params.triangulated = true;
+        else if (a == "--pinch-palm-down" && more) pinch_params.palm_down_max = std::atof(argv[++i]);
        else if (a == "--swap-sides") swap_sides = true;
        else if (a == "--record-only") track = publish = false;
        else if (a == "--ring" && more) ring_path = argv[++i];
@@ -140,7 +141,7 @@ int main(int argc, char **argv) {
            std::printf("usage: %s [--seconds N] [--threads N] [--int8] [--status S] [--models DIR] [--nice N] [--no-publish]\n"
                        "          [--record DIR] [--record-for S] [--record-only] [--cpus 5,6,7] [--swap-sides]\n"
                        "          [--keep-presence P] (0.5) [--ring PATH] (ft-camd's, or ft-ringplay's)\n"
-                        "          [--pinch-begin M] (0.020) [--pinch-end M] (0.035) [--pinch-triangulated]\n"
+                        "          [--pinch-begin M] (0.020) [--pinch-end M] (0.035) [--pinch-triangulated] [--pinch-palm-down MAX] (0.6)\n"
                        "          [--contrast MODE|PALM/HAND] (clahe[:CLIP], none, stretch; default clahe:2/none)\n"
                        "Recording saves every frame set for S seconds (120) to DIR/sets.bin, for ft-handreplay; SIGUSR1\n"
                        "starts one in ~/.local/share/frametop/hands/rec-<time>. --record-only records without tracking, so it\n"
@@ -151,6 +152,7 @@ int main(int argc, char **argv) {
            return a == "--help" ? 0 : 1;
        }
    }
+    std::setvbuf(stdout, nullptr, _IOLBF, 0);   // whole lines to the journal as they come
    if (nice(niceness) < 0) std::perror("nice");   // the VR stack wins contested CPUs
    std::signal(SIGINT, [](int) { g_stop = 1; });
    std::signal(SIGTERM, [](int) { g_stop = 1; });
@@ -344,7 +346,8 @@ int main(int argc, char **argv) {
                            s.handoff_miss, s.dups, s.splits, s.created, s.merged, s.forgotten,
                            !rec ? "" : ("  recorded " + std::to_string(rec->written()) + " dropped " +
                                         std::to_string(rec->dropped())).c_str());
-            std::printf("        pinches: left %u right %u", pinch.side(0).begins, pinch.side(1).begins);
+            std::printf("        pinches: left %u right %u (held back, palm down: %d %d)", pinch.side(0).begins,
+                        pinch.side(1).begins, pinch.held_back[0], pinch.held_back[1]);
            for (int k = 0; k < 2; ++k)
                if (pinch.side(k).flags & FH_PINCH_TRACKED)
                    std::printf("  %s %s d %.3f", k ? "right" : "left", pinch.side(k).flags & FH_PINCH_DOWN ? "DOWN" : "open",
@@ -16,6 +16,8 @@ namespace {

 constexpr int kThumbTip = 4, kIndexTip = 8;

+V3 cross(V3 a, V3 b) { return {a[1] * b[2] - a[2] * b[1], a[2] * b[0] - a[0] * b[2], a[0] * b[1] - a[1] * b[0]}; }
+
 void put3(float out[3], V3 v) {
    for (int k = 0; k < 3; ++k) out[k] = float(v[k]);
 }
@@ -24,16 +26,22 @@ void put3(float out[3], V3 v) {

 void Pinch::update(const std::vector<const Hand *> &hands, const std::vector<Seen> &views, int64_t t_ns) {
    events.clear();
+    // A hand a pinch is down on belongs to that side until it ends. The left/right call is a
+    // running average of the model's, and when it flips mid-pinch the other side would take
+    // the same hand and pinch too (4 times in the 2026-09-30 lit recording).
+    int taken[2] = {0, 0};
+    for (int s = 0; s < 2; ++s)
+        if (side_[s].flags & FH_PINCH_DOWN) taken[s] = follow_[s];
    for (int s = 0; s < 2; ++s) {
        fh_pinch_t &o = side_[s];
        const bool down = o.flags & FH_PINCH_DOWN;
        // the hand: while down, the one the pinch began on; else the best tracked hand of this side
        const Hand *h = nullptr;
        for (const Hand *c : hands) {
-            if (down ? c->id != follow_[s] : c->right() != (s == 1)) continue;
+            if (down ? c->id != follow_[s] : c->right() != (s == 1) || c->id == taken[1 - s]) continue;
            if (!h || c->frames > h->frames) h = c;
        }
-        world_d[s] = tri_d[s] = -1;
+        world_d[s] = tri_d[s] = palm_down[s] = -1;
        if (!h) {
            o.flags &= ~FH_PINCH_TRACKED;
            if (down && (t_ns - seen_ns_[s]) / 1e9 > p_.grace_s) end(s, t_ns, true);
@@ -41,6 +49,8 @@ void Pinch::update(const std::vector<const Hand *> &hands, const std::vector<See
        }
        seen_ns_[s] = t_ns;
        tri_d[s] = norm(h->pts[kThumbTip] - h->pts[kIndexTip]);
+        const V3 normal = cross(h->smooth[5] - h->smooth[0], h->smooth[17] - h->smooth[0]);
+        palm_down[s] = norm(normal) > 0 ? std::fabs(normal[1]) / norm(normal) : 0;
        double sum = 0;
        int n = 0;
        for (const Seen &v : views) {
@@ -58,7 +68,13 @@ void Pinch::update(const std::vector<const Hand *> &hands, const std::vector<See
        o.strength = float(std::clamp((p_.end_m - d) / (p_.end_m - p_.begin_m), 0.0, 1.0));
        put3(o.point, point);
        if (!down) {
-            if (d < p_.begin_m) {
+            // a close held back (palm down) has to open again before a pinch can begin, so
+            // turning the hand with the fingers still closed doesn't start one
+            if (d > p_.end_m) held_[s] = false;
+            if (d < p_.begin_m && !held_[s] && palm_down[s] > p_.palm_down_max) {
+                held_[s] = true;
+                ++held_back[s];
+            } else if (d < p_.begin_m && !held_[s]) {
                o.flags = (o.flags | FH_PINCH_DOWN) & ~FH_PINCH_LOST;
                ++o.begins;
                o.begin_ns = uint64_t(t_ns);
@@ -22,6 +22,11 @@ struct PinchParams {
    // triangulated tips. The model's pose should hold up better when the fingers hide each
    // other; tomorrow's recordings will tell.
    bool triangulated = false;
+    // No pinch begins while the palm faces down more than this (|palm normal . up| in the
+    // head frame; 1 turns it off). Typing curls the thumb onto the index: in the 2026-09-30
+    // lit recording, pinches that began while typing had 0.69-1.00, deliberate ones 0.00-0.50.
+    // Looking down tilts the head frame, which lowers the reading for a hand on a keyboard.
+    double palm_down_max = 0.6;
 };

 class Pinch {
@@ -48,6 +53,8 @@ public:
    std::vector<Event> events;
    // Both distance measures for the last update, per side (-1: no hand), for logs.
    double world_d[2] = {-1, -1}, tri_d[2] = {-1, -1};
+    double palm_down[2] = {-1, -1};   // |palm normal . up| of each side's hand
+    int held_back[2] = {0, 0};          // pinches that didn't begin because the palm faced down

 private:
    void end(int s, int64_t t_ns, bool lost);
@@ -56,9 +63,10 @@ private:
    int follow_[2] = {0, 0};        // the hand id a pinch follows while down
    int open_frames_[2] = {0, 0};
    int64_t seen_ns_[2] = {0, 0};
+    bool held_[2] = {false, false};  // a close held back (palm down) that hasn't opened yet
 };

-// Writes /run/user/UID/frametop/gestures.
+// Writes /run/user/UID/frametop-hands/gestures.
 class GesturePublisher {
 public:
    bool open(const Pinch &pinch, std::string &err);
@@ -13,7 +13,7 @@
 // --timeline: per processed set, a line per hand (time, id, side, views, wrist) and per view
 //             (hand, camera, presence, next crop, set index).
 // --keep-presence P: landmark presence a tracked view needs to stay (default 0.5, as new ones).
-// --pinch-begin M, --pinch-end M, --pinch-triangulated: the pinch detector (track/pinch.h);
+// --pinch-begin M, --pinch-end M, --pinch-triangulated, --pinch-palm-down MAX: the pinch detector (track/pinch.h);
 //             the timeline gets its begin/end/lost events and both distance measures per set.
 // --cams mono|color|all: which cameras to track with (default mono). color and all need a
 //             recording made with ft-camd --with-color; --color-left NODE (color_video0 or
@@ -21,6 +21,10 @@
 //             (tools/check_color.py).
 // --contrast: how the palm search's and the landmark model's crops are equalized
 //             (default clahe:2/none, as ft-hands).
+// --poses FILE: per processed set, a line per hand: time, id, the model's left/right call,
+//             views, hand scale, then its 21 world landmarks (the model's own 3D pose, averaged
+//             over its views, times the scale; metres, hand-centred) and its 21 published
+//             points (head frame). For studying gestures (pinch against typing, a fist).
 // --depth FILE: per processed set, a line per hand for tools/depth_report.py: its views'
 //             cameras, triangulation residual, hand scale, measured and published palm, and
 //             each view's one-view palm (Tracker::single_view at the hand's scale). The
@@ -70,7 +74,7 @@ int main(int argc, char **argv) {
    double keep_presence = 0.5;   // landmark presence a tracked view needs to stay
    PinchParams pinch_params;
    std::string use = "mono", color_left = "color_video0", color_crop = "subtract";
-    std::string timeline, depth, models = std::string(argv[0]).substr(0, std::string(argv[0]).rfind('/') + 1) + "../models/ncnn";
+    std::string timeline, depth, poses, models = std::string(argv[0]).substr(0, std::string(argv[0]).rfind('/') + 1) + "../models/ncnn";
    for (int i = 2; i < argc; ++i) {
        const std::string a = argv[i];
        const bool more = i + 1 < argc;
@@ -78,6 +82,7 @@ int main(int argc, char **argv) {
        else if (a == "--slow" && more) slow = std::atof(argv[++i]);
        else if (a == "--timeline" && more) timeline = argv[++i];
        else if (a == "--depth" && more) depth = argv[++i];
+        else if (a == "--poses" && more) poses = argv[++i];
        else if (a == "--threads" && more) threads = std::atoi(argv[++i]);
        else if (a == "--models" && more) models = argv[++i];
        else if (a == "--cost") cost = true;
@@ -86,6 +91,7 @@ int main(int argc, char **argv) {
        else if (a == "--pinch-begin" && more) pinch_params.begin_m = std::atof(argv[++i]);
        else if (a == "--pinch-end" && more) pinch_params.end_m = std::atof(argv[++i]);
        else if (a == "--pinch-triangulated") pinch_params.triangulated = true;
+        else if (a == "--pinch-palm-down" && more) pinch_params.palm_down_max = std::atof(argv[++i]);
        else if (a == "--color-left" && more) color_left = argv[++i];
        else if (a == "--color-crop" && more) color_crop = argv[++i];
        else if (a == "--contrast" && more) {
@@ -127,6 +133,7 @@ int main(int argc, char **argv) {
    Tracker tracker(used, nets, pool);
    tracker.set_keep_presence(keep_presence);
    FILE *dp = depth.empty() ? nullptr : std::fopen(depth.c_str(), "w");
+    FILE *pp = poses.empty() ? nullptr : std::fopen(poses.c_str(), "w");
    if (dp)
        for (auto &[name, c] : used)
            std::fprintf(dp, "# cam %s %.4f %.4f %.4f %.1f\n", name.c_str(), c.origin[0], c.origin[1], c.origin[2], c.fx);
@@ -189,12 +196,13 @@ int main(int argc, char **argv) {
            for (const Pinch::Event &e : pinch.events) {
                if (std::string(e.what) == "begin") pinch_begin_ts[e.side] = ts;
                else pinch_len[e.side].push_back(ts - pinch_begin_ts[e.side]), pinch_lost += std::string(e.what) == "lost";
-                if (tl) std::fprintf(tl, "%.3f pinch %s %s d %.3f point %+.3f %+.3f %+.3f\n", ts, e.side ? "R" : "L", e.what,
-                                     e.distance, e.point[0], e.point[1], e.point[2]);
+                if (tl) std::fprintf(tl, "%.3f pinch %s %s d %.3f point %+.3f %+.3f %+.3f hand %u\n", ts, e.side ? "R" : "L",
+                                     e.what, e.distance, e.point[0], e.point[1], e.point[2], pinch.side(e.side).hand_id);
            }
            if (tl && (pinch.world_d[0] >= 0 || pinch.world_d[1] >= 0))   // both measures, for choosing one
-                std::fprintf(tl, "%.3f pinchd L world %.3f tri %.3f R world %.3f tri %.3f\n", ts, pinch.world_d[0],
-                             pinch.tri_d[0], pinch.world_d[1], pinch.tri_d[1]);
+                std::fprintf(tl, "%.3f pinchd L world %.3f tri %.3f R world %.3f tri %.3f palm %.2f %.2f\n", ts,
+                             pinch.world_d[0], pinch.tri_d[0], pinch.world_d[1], pinch.tri_d[1], pinch.palm_down[0],
+                             pinch.palm_down[1]);
            ++processed;
            last_out = out;
            bool l = false, r = false;
@@ -218,6 +226,22 @@ int main(int argc, char **argv) {
                if (tl)
                    std::fprintf(tl, "%.3f %d %s %d %+.3f %+.3f %+.3f\n", ts, h->id, h->pts[0][0] < 0 ? "L" : "R", h->nviews,
                                 h->pts[0][0], h->pts[0][1], h->pts[0][2]);
+                if (pp) {
+                    double world[21][3] = {};
+                    int n = 0;
+                    for (const Seen &v : seen)
+                        if (v.hand == h->id) {
+                            for (int k = 0; k < 21; ++k)
+                                for (int j = 0; j < 3; ++j) world[k][j] += v.lm.world[k][j];
+                            ++n;
+                        }
+                    std::fprintf(pp, "%.4f %d %s %d %.3f", ts, h->id, h->right() ? "R" : "L", h->nviews, h->scale);
+                    for (int k = 0; k < 21; ++k)
+                        for (int j = 0; j < 3; ++j) std::fprintf(pp, " %.4f", n ? world[k][j] / n * h->scale : NAN);
+                    for (int k = 0; k < 21; ++k)
+                        for (int j = 0; j < 3; ++j) std::fprintf(pp, " %.4f", h->smooth[k][j]);
+                    std::fputc('\n', pp);
+                }
                if (dp) {
                    std::vector<const Seen *> vs;
                    for (const Seen &v : seen)
@@ -266,6 +290,7 @@ int main(int argc, char **argv) {
    } while (in.next(cams, px));
    if (tl) std::fclose(tl);
    if (dp) std::fclose(dp);
+    if (pp) std::fclose(pp);

    const double secs = nsets > 1 ? nsets / 30.0 : 0;
    const Stats &s = tracker.stats;
@@ -301,10 +326,13 @@ int main(int argc, char **argv) {
    }
    std::printf("palms within 20 cm of the eyes: %d of %d hand updates\n", near_face, hand_updates);
    for (int k = 0; k < 2; ++k) std::sort(pinch_len[k].begin(), pinch_len[k].end());
-    std::printf("pinches (%s, %.3f/%.3f m): left %zu (median %.2f s), right %zu (median %.2f s), %d ended by losing the hand\n",
+    std::printf("pinches (%s, %.3f/%.3f m, palm down under %.2f): left %zu (median %.2f s), right %zu (median %.2f s), "
+                "%d ended by losing the hand, held back (palm down) left %d right %d\n",
                pinch_params.triangulated ? "triangulated tips" : "world landmarks", pinch_params.begin_m, pinch_params.end_m,
+                pinch_params.palm_down_max,
                pinch_len[0].size(), pinch_len[0].empty() ? 0 : pinch_len[0][pinch_len[0].size() / 2], pinch_len[1].size(),
-                pinch_len[1].empty() ? 0 : pinch_len[1][pinch_len[1].size() / 2], pinch_lost);
+                pinch_len[1].empty() ? 0 : pinch_len[1][pinch_len[1].size() / 2], pinch_lost, pinch.held_back[0],
+                pinch.held_back[1]);
    std::sort(jit_raw.begin(), jit_raw.end());
    std::sort(jit_sm.begin(), jit_sm.end());
    if (!jit_raw.empty())
@@ -59,7 +59,7 @@ bool Hands::Read() {
        const int64_t now = MonoNs();
        if (now - lastOpenTry_ < 1'000'000'000) return false;
        lastOpenTry_ = now;
-        const std::string path = "/run/user/" + std::to_string(getuid()) + "/frametop/hands";
+        const std::string path = "/run/user/" + std::to_string(getuid()) + "/frametop-hands/hands";
        fd_ = open(path.c_str(), O_RDONLY | O_CLOEXEC | O_NOFOLLOW);
        if (fd_ < 0) return false;
        struct stat st;
@@ -295,7 +295,7 @@ void main() {
 // The client's pixels, opaque (its alpha is ignored, as IgnoreTextureAlpha did).
 const char *kCopy = R"(
 #extension GL_OES_EGL_image_external : require
-precision mediump float;
+precision highp float;   // mediump (16-bit on Adreno) steps 1.7 texels across a 3440-pixel screen
 uniform samplerExternalOES tex;
 varying vec2 uv;
 void main() { gl_FragColor = vec4(texture2D(tex, uv).rgb, 1.0); })";
@@ -2,7 +2,7 @@
 // room (Room View) through the screen instead of the screen drawn over the hand.
 //
 // ft-hands (hands/) publishes the hands it sees with the headset's cameras to
-// /run/user/UID/frametop/hands (hands/include/fh_hands.h):
+// /run/user/UID/frametop-hands/hands (hands/include/fh_hands.h):
 // capsules (finger bones, palm, forearm) in the head frame at capture time. Hands turns
 // them into the room with the head pose at that time. Project() finds where each eye
 // sees them on a panel, and Renderer draws the panel's client buffer into a side-by-side
@@ -4,7 +4,7 @@
 //
 //   ft-handtest [--distance m] [--width m] [--seconds s]
 //
-// Needs hand tracking running (hands/run.sh; ft-hands publishes /run/user/UID/frametop/hands).
+// Needs hand tracking running (hands/run.sh; ft-hands publishes /run/user/UID/frametop-hands/hands).
 // Build: screens/build.sh (build/ft-handtest), run in the dev container.
 #include "handcut.h"