Hands: the hand recorder's window, review and export

hands/rec/ft_handrec.py + main.qml (Kirigami, dev container; host launcher
hands/rec/ft-handrec): consent (CONSENT.md, asked again when its version
changes; profile.json with a random contributor id), the before-you-start
checklist with the lighting and free-space checks, the session controls
(Space pauses, Esc stops), review with a frame-set viewer that deletes
ranges, takes and sessions, export with progress and cancel (warns while
the headset is worn), and the upload page (UPLOAD.md, the
huggingface-cli command; HF_DATASET is a placeholder). --dry-run runs
sessions without processes, for testing.

hands/rec/takes.py (standard library): indexes sets.bin and sets-N.bin
without reading pixels, reads one set's cameras, keeps deleted ranges in
take.json, and exports: deleted sets left out, zstd -10 -T2 at nice 19,
manifest.json and SHA256SUMS, nothing left behind on cancel.

CONSENT.md and UPLOAD.md are drafts pending a legal review; the window
says contributions aren't open yet. The dev container gains zstd.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

hands/rec/CONSENT.md

@@ -0,0 +1,60 @@
+**DRAFT: this text hasn't had a legal review yet. Contributions aren't open until it has.**
+
+# Recording your hands for the Frametop hand dataset
+
+Version: 2026-10-02
+
+Frametop's hand tracking needs a small model that finds hands in the headset's camera images. To train it, we're collecting recordings of many people's hands. This page explains what the hand recorder records, what happens to it, and what you agree to if you take part. Please read all of it.
+
+## Who can take part
+
+You must be 18 or older.
+
+## What is recorded
+
+While a session runs, the hand recorder saves:
+
+- **Camera images.** Infrared images from the headset's 4 tracking cameras, about 10 sets a second. They show your hands, your arms and whatever is in front of you: your room, your desk and the things on it. They're grey, low-detail images, but people and things can be recognised in them.
+- **Motion.** The position and rotation of the headset and the controllers, many times a second.
+- **Prompts.** What you were asked to do and when, and what the live hand tracker saw at the time.
+- **Calibration.** Where the cameras sit on the headset and how their lenses bend the image. Serial numbers and other fields that identify your headset are removed first, and the export lists what was removed.
+- **Your checklist answers.** Which objects you had, the lighting you chose, whether you wore sleeves, rings or a watch, and any notes you typed in.
+- **A contributor id.** A random number made on your headset the first time you agree to this page. It isn't linked to your name, your Steam account or your headset. It lets us keep your sessions together and find them if you withdraw.
+
+The recorder doesn't record sound, your name, your email address or your account. Your eyes and face aren't recorded: the tracking cameras look outward.
+
+## Nothing leaves your headset unless you send it
+
+- Recordings stay on your headset, in `~/.local/share/frametop/hands/contrib`. Nothing is uploaded automatically.
+- Before you share anything, you can watch every recording in the Review page. You can delete any stretch of a recording, a whole take or a whole session.
+- Export makes a package from what you kept. You upload it yourself, with your own Hugging Face account, following the Upload page. Until you do, nobody else has it.
+- An upload opens a pull request. The maintainer checks it before it becomes part of the dataset, and may decline it.
+
+## Keep other people and private things out of view
+
+While recording, please:
+
+- face away from other people, mirrors, screens showing private things, papers, letters and anything else you wouldn't want in a public dataset;
+- make sure nobody else's face or hands are in view.
+
+If something private got into a recording, delete that stretch in Review before you export. If you notice it after uploading, withdraw the session (below).
+
+## The license
+
+- **The dataset is published under Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0).** Anyone may use it for non-commercial purposes, with attribution. Your contribution is credited by its contributor id, not your name.
+- **You also give the maintainer, DeeJanuz, a non-exclusive license to use your contribution for any purpose, including commercially.** That includes copying it, changing it, and training, publishing and selling models made from it, in Frametop and elsewhere. It's non-exclusive: you keep any rights you have in your recordings and can do what you like with your own copies.
+- You confirm that you have the right to give these licenses: the recordings are yours, and nothing in them belongs to someone who hasn't agreed.
+- There is no payment, and the dataset comes with no warranty.
+
+## Withdrawing
+
+You can withdraw a contribution at any time. Send your contributor id (shown in the hand recorder) and which sessions to withdraw, or "all", through the dataset's discussion page or the Frametop repository's issues. Then:
+
+- your recordings are deleted from the dataset and purged from its repository's history, so they can't be downloaded from there again;
+- they're left out of anything trained after that.
+
+What can't be undone: copies others downloaded before the withdrawal, and models already trained with them.
+
+## Agreeing
+
+By ticking "I'm 18 or older" and "I agree", you confirm the above. You can still decide not to upload anything. If this text changes, the hand recorder asks you again before your next session.

hands/rec/UPLOAD.md

@@ -0,0 +1,57 @@
+**DRAFT: contributions aren't open yet. Please don't upload until this banner is gone.**
+
+# Uploading your recordings
+
+Your export is ready:
+
+- Folder: `@EXPORT_PATH@`
+- Size: @EXPORT_SIZE@
+- Contributor id: `@CONTRIBUTOR@`
+
+Uploads go to the Hugging Face dataset [@DATASET@](https://huggingface.co/datasets/@DATASET@). You do it yourself, from your own account; the hand recorder never uploads anything. The upload opens a pull request, so nothing is published until the maintainer has looked at it.
+
+## 1. Make a Hugging Face account
+
+Sign up at <https://huggingface.co/join>, if you don't have an account. Your username shows on your pull request, but the dataset credits your contributor id, not your name.
+
+## 2. Accept the dataset's terms
+
+Open <https://huggingface.co/datasets/@DATASET@>, read the terms and accept them. They're the same as the consent you agreed to in the hand recorder.
+
+## 3. Create a write token
+
+Go to <https://huggingface.co/settings/tokens>, press "Create new token", choose "Write" and give it a name like "frametop-hands". Copy the token.
+
+The token lets anyone who has it change things in your account. Paste it only into your own terminal in the next step: never into a chat, a website, an issue or this window.
+
+## 4. Install the Hugging Face tools
+
+Open a terminal (Konsole) and enter the dev container, then install `huggingface_hub` and log in:
+
+```
+distrobox enter dev
+pip install --user huggingface_hub
+huggingface-cli login
+```
+
+`huggingface-cli login` asks for the token: paste it there. If it asks whether to add the token as a git credential, answer no.
+
+## 5. Upload
+
+In the same terminal, run:
+
+```
+@COMMAND@
+```
+
+It uploads the export folder to `contributions/@CONTRIBUTOR@/@SESSION@` in the dataset and opens a pull request. Large uploads take a while. If it fails partway, run the same command again: parts already sent usually aren't sent twice. That can open a second pull request, which is fine: the maintainer closes the incomplete one.
+
+Keep the headset on its charger or plugged in while it uploads.
+
+## 6. The maintainer reviews it
+
+The maintainer checks your pull request (that the files are complete, and that nobody else and nothing private is in view) before merging it into the dataset. You can follow it, and answer questions, on the pull request's page.
+
+When it's merged, you can delete the session and its export on your headset to free the space.
+
+To withdraw a contribution later, see "Withdrawing" in the consent text: you'll need your contributor id, `@CONTRIBUTOR@`.

hands/rec/ft-handrec

@@ -0,0 +1,18 @@
+#!/bin/bash
+# Launch the Frametop Hand Recorder from a Plasma session on the Frame host.
+# The app runs in the dev container (PySide6, Kirigami and zstd come from Fedora there).
+# podman needs the real XDG_RUNTIME_DIR and the real user bus (to reach systemd for
+# the container's cgroup; the Frametop session runs on a private bus from
+# dbus-run-session). The session's Wayland socket and bus go to the app itself.
+# Options go to ft_handrec.py: --base DIR, --page NAME, and --dry-run for testing.
+here=$(cd "$(dirname "$(readlink -f "$0")")" && pwd)
+wl=${WAYLAND_DISPLAY:-wayland-0}
+case $wl in /*) ;; *) wl="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/$wl" ;; esac
+session_bus=${DBUS_SESSION_BUS_ADDRESS:-}
+export XDG_RUNTIME_DIR=/run/user/$(id -u)
+export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
+"$here/../../scripts/container-up.sh"
+exec "$HOME/.local/bin/distrobox" enter dev -- env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
+  XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \
+  QT_QPA_PLATFORM="wayland;xcb" \
+  python3 "$here/ft_handrec.py" "$@"

hands/rec/ft-handrec.desktop

@@ -0,0 +1,9 @@
+[Desktop Entry]
+Type=Application
+Name=Frametop Hand Recorder
+GenericName=Record your hands for the hand dataset
+Comment=Record, review and export hand recordings for Frametop's open hand dataset
+Exec=@REPO@/hands/rec/ft-handrec
+Icon=camera-video
+Categories=Utility;
+Keywords=hands;hand tracking;dataset;record;frametop;

hands/rec/ft_handrec.py

@@ -0,0 +1,713 @@
+#!/usr/bin/env python3
+"""Frametop Hand Recorder: record your hands for the open hand dataset (hands/rec/DESIGN.md).
+
+A Kirigami (QML) app with a Python backend. It runs in the dev container:
+  - Welcome: the consent text (CONSENT.md), shown the first time and again when its version
+    changes. Agreeing writes profile.json with a random contributor id.
+  - Before you start: the checklist (objects, controller straps, lighting, sleeves, privacy,
+    free space) and what will happen. Start hands it to the session runner (session.py).
+  - Session: the runner's live status, Start, Pause/Resume, Skip section and Stop (Space
+    pauses and Esc stops while the window has focus). The prompts appear in the headset.
+  - Review: sessions, their takes, and a viewer for one frame set at a time, where ranges,
+    takes and sessions can be deleted (takes.py).
+  - Export: compress what's kept into exports/<session>/ at nice 19 (takes.py), with a warning
+    when the headset is worn.
+  - Upload: UPLOAD.md with the export filled in and the upload command to copy.
+Everything lives under ~/.local/share/frametop/hands/contrib (--base). Nothing is uploaded
+from here. Launch with hands/rec/ft-handrec (host wrapper).
+"""
+import argparse
+import datetime
+import os
+import re
+import shlex
+import signal
+import sys
+import threading
+import uuid
+
+from PySide6.QtCore import Property, QObject, Qt, QTimer, QUrl, Signal, Slot
+from PySide6.QtGui import QColor, QFont, QGuiApplication, QIcon, QImage, QPainter, QPalette
+from PySide6.QtQml import QQmlApplicationEngine
+from PySide6.QtQuick import QQuickImageProvider
+from PySide6.QtQuickControls2 import QQuickStyle
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+import takes  # noqa: E402  (next to this file)
+
+# The dataset contributions go to: a placeholder until the maintainer decides (DESIGN.md).
+HF_DATASET = "DeeJanuz/frametop-hands"
+CONSENT_PATH = os.path.join(HERE, "CONSENT.md")
+UPLOAD_PATH = os.path.join(HERE, "UPLOAD.md")
+SCRIPT_PATH = os.path.join(HERE, "script.json")
+# The headset counts as worn while vrcompositor runs and a display panel is lit: SteamVR turns
+# the panels off 5 s after the headset comes off (frame-job's check; the proximity sensor's
+# readings are too noisy). The dev container sees the host's processes and /sys.
+BACKLIGHTS = "/sys/class/backlight"
+ROUND_BYTES = 10 * 1000 ** 3  # about what one round of recording takes
+# The checklist's choices; the keys are what session.json stores.
+OBJECTS = [("pencil", "Pencil or pen"), ("phone", "Phone"), ("cup", "Cup or mug (empty)"),
+           ("keyboard", "Keyboard"), ("mouse", "Mouse"), ("gamepad", "Gamepad"),
+           ("small", "Something small (a coin, a key, a bottle cap)")]
+LIGHTING = [("dim", "Dim: one lamp only"), ("room", "Normal room light"), ("daylight", "Daylight near a window")]
+SLEEVES = [("short", "Short sleeves or bare arms"), ("long", "Long sleeves"), ("", "Rather not say")]
+HANDEDNESS = [("", "Rather not say"), ("right", "Right-handed"), ("left", "Left-handed"),
+              ("both", "Both (ambidextrous)")]
+ACTIVE_STATES = ("starting", "intro", "running", "paused", "between")
+# Shown side by side in the viewer at this height; thumbnails are smaller.
+SET_HEIGHT = 480
+THUMB_HEIGHT = 96
+
+
+def read_text(path):
+    try:
+        with open(path, encoding="utf-8") as f:
+            return f.read()
+    except OSError:
+        return ""
+
+
+def consent_version():
+    """The "Version: ..." line of CONSENT.md: a new one asks everyone to agree again."""
+    m = re.search(r"^Version:\s*(\S+)", read_text(CONSENT_PATH), re.M)
+    return m.group(1) if m else "unknown"
+
+
+def is_draft(path):
+    return "DRAFT" in read_text(path).split("\n", 1)[0]
+
+
+def process_running(name):
+    """Like pgrep -x NAME, from /proc."""
+    for pid in os.listdir("/proc"):
+        if pid.isdigit():
+            try:
+                with open(f"/proc/{pid}/comm") as f:
+                    if f.read().strip() == name:
+                        return True
+            except OSError:
+                pass
+    return False
+
+
+def headset_worn():
+    """True while vrcompositor runs and any panel's backlight is on; False if unreadable."""
+    lit = False
+    try:
+        for name in os.listdir(BACKLIGHTS):
+            try:
+                with open(os.path.join(BACKLIGHTS, name, "brightness")) as f:
+                    lit = lit or int(f.read().strip()) > 0
+            except (OSError, ValueError):
+                pass
+    except OSError:
+        return False
+    return lit and process_running("vrcompositor")
+
+
+def gigabytes(n):
+    return f"{n / 1000 ** 3:.1f} GB"
+
+
+def session_label(sid):
+    """20261002-101500 -> 2026-10-02 10:15."""
+    try:
+        return datetime.datetime.strptime(sid, "%Y%m%d-%H%M%S").strftime("%Y-%m-%d %H:%M")
+    except ValueError:
+        return sid
+
+
+def grey_image(cam):
+    """A camera's raw 8-bit pixels as a QImage that owns its data."""
+    img = QImage(cam["pixels"], cam["width"], cam["height"], cam["width"], QImage.Format_Grayscale8)
+    return img.copy()
+
+
+class FrameProvider(QQuickImageProvider):
+    """image://frames/set/SESSION/TAKE/INDEX: one frame set, every camera side by side.
+    image://frames/thumb/SESSION/TAKE: the take's first slam_left image, small.
+    Anything after "?" is ignored (it makes QML load the image again)."""
+
+    def __init__(self, store):
+        super().__init__(QQuickImageProvider.ImageType.Image)
+        self.store = store
+
+    def requestImage(self, ident, size, requested):
+        parts = ident.split("?", 1)[0].split("/")
+        try:
+            if parts[0] == "thumb" and len(parts) == 3:
+                return self._thumb(parts[1], parts[2])
+            if parts[0] == "set" and len(parts) == 4:
+                return self._set(parts[1], parts[2], int(parts[3]),
+                                 requested.height() if requested.height() > 0 else SET_HEIGHT)
+        except (OSError, ValueError, IndexError, KeyError) as e:
+            print(f"ft-handrec: image {ident}: {e}", file=sys.stderr)
+        img = QImage(4, 3, QImage.Format_Grayscale8)
+        img.fill(0)
+        return img
+
+    def _index(self, session, take):
+        return takes.take_index(self.store.take_dir(session, take))
+
+    def _thumb(self, session, take):
+        index = self._index(session, take)
+        if not len(index):
+            raise ValueError("no sets")
+        names = [c["name"] for c in index.cams]
+        cams = index.read_set(0, only="slam_left" if "slam_left" in names else names[0])
+        return grey_image(cams[0]).scaledToHeight(THUMB_HEIGHT, Qt.SmoothTransformation)
+
+    def _set(self, session, take, i, height):
+        cams = self._index(session, take).read_set(i)
+        scaled = [grey_image(c).scaledToHeight(height, Qt.SmoothTransformation) for c in cams]
+        gap = 8
+        out = QImage(sum(s.width() for s in scaled) + gap * (len(scaled) - 1), height, QImage.Format_RGB32)
+        out.fill(QColor(30, 30, 30))
+        p = QPainter(out)
+        font = QFont()
+        font.setPixelSize(max(12, height // 28))
+        p.setFont(font)
+        x = 0
+        for cam, img in zip(cams, scaled):
+            p.drawImage(x, 0, img)
+            p.setPen(QColor(255, 200, 80))
+            p.drawText(x + 6, 6 + font.pixelSize(), cam["name"])
+            x += img.width() + gap
+        p.end()
+        return out
+
+
+class Backend(QObject):
+    profileChanged = Signal()
+    diskChanged = Signal()
+    lightingChanged = Signal()
+    statusChanged = Signal()
+    sessionsChanged = Signal()
+    exportChanged = Signal()
+    message = Signal(str, bool)  # text, is error
+    # From other threads (the session runner, export, the lighting check): queued to this one.
+    _statusArrived = Signal(dict)
+    _lightingArrived = Signal(str)
+    _exportProgress = Signal(float, str)
+    _exportFinished = Signal(str, str)  # path, error ("" when it worked; "cancelled")
+
+    def __init__(self, store, session_options=None):
+        super().__init__()
+        self.store = store
+        self._session_options = session_options or {}  # test hooks for Session: dry_run, speed
+        os.makedirs(store.base, mode=0o700, exist_ok=True)
+        self._session_mod = None
+        self._session_error = ""
+        self._session = None
+        self._session_id = ""
+        self._status = {}
+        self._lighting_note = ""
+        self._export_cancel = None
+        self._export_thread = None
+        self._export_fraction = 0.0
+        self._export_text = ""
+        self._export_session = ""
+        self._statusArrived.connect(self._on_status)
+        self._lightingArrived.connect(self._on_lighting)
+        self._exportProgress.connect(self._on_export_progress)
+        self._exportFinished.connect(self._on_export_finished)
+        self.disk_timer = QTimer(interval=30000, timeout=self.diskChanged.emit)
+        self.disk_timer.start()
+
+    def _thread(self, fn):
+        thread = threading.Thread(target=fn, daemon=True)
+        thread.start()
+        return thread
+
+    # --- the session runner, imported when first needed (it's written separately)
+    def _runner(self):
+        if self._session_mod is None and not self._session_error:
+            try:
+                import session as mod
+                self._session_mod = mod
+            except Exception as e:  # missing, or broken: say so, the rest of the app still works
+                self._session_error = f"The session runner (hands/rec/session.py) can't be loaded: {e}"
+        return self._session_mod
+
+    @Property(str, notify=statusChanged)
+    def runnerError(self):
+        self._runner()
+        return self._session_error
+
+    # --- consent and profile
+    @Property(str, constant=True)
+    def consentText(self):
+        return read_text(CONSENT_PATH) or "CONSENT.md is missing."
+
+    @Property(str, constant=True)
+    def consentVersion(self):
+        return consent_version()
+
+    @Property(bool, constant=True)
+    def textsDraft(self):
+        return is_draft(CONSENT_PATH) or is_draft(UPLOAD_PATH)
+
+    @Property(bool, notify=profileChanged)
+    def needsConsent(self):
+        profile = self.store.profile()
+        consent = profile.get("consent") or {}
+        return not (profile.get("contributor") and consent.get("adult") is True
+                    and consent.get("version") == consent_version())
+
+    @Property(str, notify=profileChanged)
+    def contributor(self):
+        return self.store.profile().get("contributor", "")
+
+    @Property(str, notify=profileChanged)
+    def consentAccepted(self):
+        return (self.store.profile().get("consent") or {}).get("accepted", "")
+
+    @Property(str, notify=profileChanged)
+    def handedness(self):
+        return (self.store.profile().get("optional") or {}).get("handedness", "")
+
+    @Property("QVariantList", constant=True)
+    def handednessChoices(self):
+        return [{"value": k, "text": v} for k, v in HANDEDNESS]
+
+    @Slot(bool, bool, str)
+    def acceptConsent(self, adult, agree, handedness):
+        """Write profile.json. A contributor id, once made, stays (a new consent version keeps it)."""
+        if not (adult and agree):
+            self.message.emit("Both boxes need ticking to take part", True)
+            return
+        profile = self.store.profile()
+        optional = profile.get("optional") if isinstance(profile.get("optional"), dict) else {}
+        optional["handedness"] = handedness if handedness in dict(HANDEDNESS) else ""
+        optional.setdefault("notes", "")
+        profile = {"schema": 1, "contributor": profile.get("contributor") or str(uuid.uuid4()),
+                   "consent": {"version": consent_version(),
+                               "accepted": datetime.datetime.now().astimezone().isoformat(timespec="seconds"),
+                               "adult": True},
+                   "optional": optional}
+        takes.write_json(self.store.profile_path, profile)
+        self.profileChanged.emit()
+        self.message.emit("Thank you. Your contributor id: " + profile["contributor"], False)
+
+    # --- the checklist
+    @Property("QVariantList", constant=True)
+    def objects(self):
+        return [{"value": k, "text": v} for k, v in OBJECTS]
+
+    @Property("QVariantList", constant=True)
+    def lightingChoices(self):
+        return [{"value": k, "text": v} for k, v in LIGHTING]
+
+    @Property("QVariantList", constant=True)
+    def sleeveChoices(self):
+        return [{"value": k, "text": v} for k, v in SLEEVES]
+
+    @Property(str, notify=diskChanged)
+    def freeText(self):
+        return gigabytes(takes.free_bytes(self.store.base))
+
+    @Property(bool, notify=diskChanged)
+    def diskOk(self):
+        return takes.free_bytes(self.store.base) >= ROUND_BYTES
+
+    @Property(str, notify=lightingChanged)
+    def lightingNote(self):
+        return self._lighting_note
+
+    @Slot(str)
+    def checkLighting(self, chosen):
+        """Compare the cameras' brightness now with earlier rounds (session.similar_lighting):
+        a round in light like an earlier one adds less to the dataset."""
+        mod = self._runner()
+        if not mod:
+            return
+        base = self.store.base
+        labels = dict(LIGHTING)
+
+        def run():
+            try:
+                ring = mod.ring_lighting()
+                if ring is None:
+                    note = ("The cameras aren't running yet, so the light can't be compared with your "
+                            "earlier rounds now. The session checks it when it starts.")
+                else:
+                    match = mod.similar_lighting(base, {"chosen": chosen, "ring": ring})
+                    note = "" if not match else (
+                        f"The cameras see about the same light as in your round of {session_label(match[0])} "
+                        f"({labels.get(match[1], match[1] or 'no choice')}). A different light helps the "
+                        "dataset more: change the lighting if you can, or go ahead anyway.")
+            except Exception as e:
+                note = f"Couldn't check the light: {e}"
+            self._lightingArrived.emit(note)
+        self._thread(run)
+
+    def _on_lighting(self, note):
+        self._lighting_note = note
+        self.lightingChanged.emit()
+
+    # --- the session
+    @Property("QVariantMap", notify=statusChanged)
+    def status(self):
+        return self._status
+
+    @Property(bool, notify=statusChanged)
+    def sessionActive(self):
+        return self._session is not None and self._status.get("state", "starting") in ACTIVE_STATES
+
+    @Property(str, notify=statusChanged)
+    def sessionId(self):
+        return self._session_id
+
+    @Slot("QVariantMap", str, result=bool)
+    def startSession(self, checklist, lighting):
+        if self.sessionActive:
+            return False
+        mod = self._runner()
+        if not mod:
+            self.message.emit(self._session_error, True)
+            return False
+        if self.needsConsent:
+            self.message.emit("Agree to the consent text first (Welcome page)", True)
+            return False
+        checklist = dict(checklist)
+        checklist["objects"] = [o for o in checklist.get("objects", []) if o in dict(OBJECTS)]
+        checklist["own_objects"] = [o.strip() for o in checklist.get("own_objects", []) if str(o).strip()]
+        try:
+            self._session = mod.Session(self.store.base, self.store.profile(), checklist, lighting, SCRIPT_PATH,
+                                        on_status=lambda s: self._statusArrived.emit(dict(s)),
+                                        **self._session_options)
+        except Exception as e:
+            self.message.emit(f"Couldn't set up the session: {e}", True)
+            return False
+        self._session_id = ""
+        self._status = {"state": "starting"}
+        self.statusChanged.emit()
+        session = self._session
+
+        def run():
+            try:
+                session.start()
+            except Exception as e:
+                self._statusArrived.emit({"state": "error", "error": str(e)})
+        self._thread(run)
+        return True
+
+    def _on_status(self, status):
+        self._status = status
+        if self._session is not None and not self._session_id:
+            self._session_id = os.path.basename(str(getattr(self._session, "session_dir", "") or ""))
+        self.statusChanged.emit()
+        if status.get("state") in ("done", "stopped", "error"):
+            self.sessionsChanged.emit()
+
+    def _control(self, name):
+        if self._session is None:
+            return
+        try:
+            getattr(self._session, name)()
+        except Exception as e:
+            self.message.emit(f"{name}: {e}", True)
+
+    @Slot()
+    def togglePause(self):
+        if self._status.get("state") == "paused":
+            self._control("resume")
+        elif self.sessionActive:
+            self._control("pause")
+
+    @Slot()
+    def skipSection(self):
+        if self.sessionActive:
+            self._control("skip")
+
+    @Slot()
+    def stopSession(self):
+        """Stop: the take in progress is kept, as far as it got. It can block while the
+        recording is written out, so it runs off this thread."""
+        if self.sessionActive:
+            session = self._session
+            self._thread(lambda: session.stop())
+
+    def shutdown(self):
+        """The window closes: end a running session (blocking, so its files are complete)."""
+        if self.sessionActive:
+            try:
+                self._session.stop()
+            except Exception:
+                pass
+        if self._export_cancel:
+            self._export_cancel.set()  # and wait, so export can remove its half-written copy
+            self._export_thread.join(15)
+
+    # --- review
+    @Property("QVariantList", notify=sessionsChanged)
+    def sessions(self):
+        out = []
+        for s in self.store.sessions():
+            s["label"] = session_label(s["id"])
+            s["sizeText"] = takes.human_bytes(s["bytes"])
+            s["exportText"] = takes.human_bytes(s["export_bytes"]) if s["exported"] else ""
+            s["lightingText"] = dict(LIGHTING).get(s["lighting"], "")
+            s["active"] = self.sessionActive and s["id"] == self._session_id
+            s["statusText"] = {"recording": "" if s["active"] else "interrupted", "error": "ended with an error",
+                               "stopped": "stopped early"}.get(s["status"], "")
+            out.append(s)
+        return out
+
+    @Slot()
+    def refreshSessions(self):
+        self.sessionsChanged.emit()
+        self.diskChanged.emit()
+
+    @Slot(str, result="QVariantList")
+    def takeList(self, session):
+        try:
+            rows = self.store.takes(session)
+        except ValueError:
+            return []
+        for t in rows:
+            t["durationText"] = f"{int(t['duration_s'] // 60)}:{int(t['duration_s'] % 60):02d}"
+            t["sizeText"] = takes.human_bytes(t["bytes"])
+            t.pop("ranges")
+        return rows
+
+    @Slot(str, str, result="QVariantMap")
+    def takeInfo(self, session, take):
+        """For the viewer: {count, title, cams, ranges: [[first, last] set indexes]}."""
+        try:
+            index = takes.take_index(self.store.take_dir(session, take))
+            ranges = self.store.ranges(session, take)
+        except ValueError:
+            return {"count": 0, "title": take, "cams": [], "ranges": []}
+        marks = []
+        for a, b in ranges:
+            inside = [i for i in range(len(index)) if a <= index.time_ns(i) <= b]
+            marks.append([inside[0], inside[-1]] if inside else [-1, -1])
+        return {"count": len(index), "title": self.store.take_meta(session, take).get("title") or take,
+                "cams": [c["name"] for c in index.cams], "ranges": marks}
+
+    @Slot(str, str, int, result=str)
+    def setTime(self, session, take, i):
+        """Set i's time from the take's first set, m:ss.s."""
+        try:
+            index = takes.take_index(self.store.take_dir(session, take))
+            t = (index.time_ns(i) - index.time_ns(0)) / 1e9
+        except (ValueError, IndexError):
+            return ""
+        return f"{int(t // 60)}:{t % 60:04.1f}"
+
+    def _active_guard(self, session):
+        if self.sessionActive and session == self._session_id:
+            self.message.emit("This session is still recording: stop it first", True)
+            return True
+        return False
+
+    @Slot(str, str, int, int)
+    def deleteRange(self, session, take, first, last):
+        """Leave sets first..last (indexes, either order) out of the export."""
+        if self._active_guard(session):
+            return
+        index = takes.take_index(self.store.take_dir(session, take))
+        first, last = sorted((max(0, first), min(len(index) - 1, last)))
+        if first > last:
+            return
+        self.store.delete_range(session, take, index.time_ns(first), index.time_ns(last))
+        self.message.emit(f"Sets {first + 1} to {last + 1} won't be exported", False)
+        self.sessionsChanged.emit()
+
+    @Slot(str, str, int)
+    def restoreRange(self, session, take, k):
+        self.store.restore_range(session, take, k)
+        self.message.emit("Range restored", False)
+        self.sessionsChanged.emit()
+
+    @Slot(str, str)
+    def deleteTake(self, session, take):
+        if self._active_guard(session):
+            return
+        try:
+            self.store.delete_take(session, take)
+            self.message.emit(f"Deleted {take}", False)
+        except (OSError, ValueError) as e:
+            self.message.emit(f"Couldn't delete {take}: {e}", True)
+        self.refreshSessions()
+
+    @Slot(str)
+    def deleteSession(self, session):
+        if self._active_guard(session):
+            return
+        try:
+            self.store.delete_session(session)
+            self.message.emit(f"Deleted the session of {session_label(session)}", False)
+        except (OSError, ValueError) as e:
+            self.message.emit(f"Couldn't delete the session: {e}", True)
+        self.refreshSessions()
+
+    # --- export
+    @Slot(result=bool)
+    def headsetWorn(self):
+        return headset_worn()
+
+    @Property(bool, constant=True)
+    def zstdFound(self):
+        return takes.find_zstd() is not None
+
+    @Property(bool, notify=exportChanged)
+    def exporting(self):
+        return self._export_cancel is not None
+
+    @Property(float, notify=exportChanged)
+    def exportFraction(self):
+        return self._export_fraction
+
+    @Property(str, notify=exportChanged)
+    def exportText(self):
+        return self._export_text
+
+    @Property(str, notify=exportChanged)
+    def exportSessionId(self):
+        return self._export_session
+
+    @Slot(str, bool)
+    def exportSession(self, session, keep_notes):
+        if self.exporting or self._active_guard(session):
+            return
+        cancel = threading.Event()
+        self._export_cancel = cancel
+        self._export_session = session
+        self._export_fraction = 0.0
+        self._export_text = "Starting"
+        self.exportChanged.emit()
+
+        def run():
+            try:
+                path = self.store.export(session, progress=lambda f, text: self._exportProgress.emit(f, text),
+                                         cancel=cancel, keep_notes=keep_notes)
+                self._exportFinished.emit(path, "")
+            except takes.Cancelled:
+                self._exportFinished.emit("", "cancelled")
+            except Exception as e:
+                self._exportFinished.emit("", str(e) or type(e).__name__)
+        self._export_thread = self._thread(run)
+
+    @Slot()
+    def cancelExport(self):
+        if self._export_cancel:
+            self._export_cancel.set()
+            self._export_text = "Cancelling"
+            self.exportChanged.emit()
+
+    def _on_export_progress(self, fraction, text):
+        if self._export_cancel and not self._export_cancel.is_set():
+            self._export_fraction, self._export_text = fraction, text
+            self.exportChanged.emit()
+
+    def _on_export_finished(self, path, error):
+        self._export_cancel = None
+        if error == "cancelled":
+            self._export_text = "Cancelled: nothing was kept"
+        elif error:
+            self._export_text = "Failed: " + error
+            self.message.emit("Export failed: " + error, True)
+        else:
+            self._export_fraction = 1.0
+            self._export_text = f"Exported to {path} ({takes.human_bytes(takes.tree_bytes(path))})"
+            self.message.emit("Export ready", False)
+        self.exportChanged.emit()
+        self.refreshSessions()
+
+    @Slot(str)
+    def deleteExport(self, session):
+        try:
+            self.store.delete_export(session)
+            self.message.emit("Export deleted; the session stays", False)
+        except (OSError, ValueError) as e:
+            self.message.emit(f"Couldn't delete the export: {e}", True)
+        self.refreshSessions()
+
+    @Property(str, constant=True)
+    def exportsDir(self):
+        return self.store.exports_dir
+
+    # --- upload
+    @Property(str, constant=True)
+    def dataset(self):
+        return HF_DATASET
+
+    @Slot(str, result=str)
+    def uploadCommand(self, session):
+        try:
+            path = self.store.export_dir(session)
+        except ValueError:
+            return ""
+        contributor = self.contributor or "CONTRIBUTOR"
+        return " ".join(["huggingface-cli", "upload", HF_DATASET, shlex.quote(path),
+                         f"contributions/{contributor}/{session}", "--repo-type", "dataset", "--create-pr",
+                         "--commit-message", shlex.quote(f"Hands: session {session} from {contributor}")])
+
+    @Slot(str, result=str)
+    def uploadText(self, session):
+        """UPLOAD.md with this export's path, size and command filled in."""
+        try:
+            path = self.store.export_dir(session)
+        except ValueError:
+            return ""
+        values = {"EXPORT_PATH": path, "EXPORT_SIZE": takes.human_bytes(takes.tree_bytes(path)),
+                  "CONTRIBUTOR": self.contributor or "CONTRIBUTOR", "SESSION": session, "DATASET": HF_DATASET,
+                  "COMMAND": self.uploadCommand(session)}
+        text = read_text(UPLOAD_PATH) or "UPLOAD.md is missing."
+        return re.sub(r"@([A-Z_]+)@", lambda m: values.get(m.group(1), m.group(0)), text)
+
+    @Slot(QColor)
+    def setLinkColor(self, color):
+        palette = QGuiApplication.palette()
+        palette.setColor(QPalette.Link, color)
+        QGuiApplication.setPalette(palette)
+
+    @Slot(str)
+    def copy(self, text):
+        QGuiApplication.clipboard().setText(text)
+        self.message.emit("Copied", False)
+
+
+def main():
+    ap = argparse.ArgumentParser(description="Frametop Hand Recorder")
+    ap.add_argument("--base", default=takes.DEFAULT_BASE, help="where profile.json, sessions/ and exports/ go")
+    ap.add_argument("--page", default="", help="open on this page: welcome, checklist, session, review, export, upload")
+    ap.add_argument("--dry-run", action="store_true",
+                    help="test: sessions start no processes and print the panel's commands")
+    ap.add_argument("--speed", type=float, default=1.0, help="test, with --dry-run: run sessions this much faster")
+    a, qt_args = ap.parse_known_args()
+    app = QGuiApplication([sys.argv[0]] + qt_args)
+    app.setApplicationName("ft-handrec")
+    app.setApplicationDisplayName("Frametop Hand Recorder")
+    app.setDesktopFileName("ft-handrec")
+    if not QIcon.themeName():
+        QIcon.setThemeName("breeze")
+    QQuickStyle.setStyle("org.kde.desktop")
+    store = takes.Store(a.base)
+    engine = QQmlApplicationEngine()
+    engine.addImageProvider("frames", FrameProvider(store))
+    backend = Backend(store, {"dry_run": True, "speed": a.speed} if a.dry_run else {})
+    app.aboutToQuit.connect(backend.shutdown)
+    engine.rootContext().setContextProperty("backend", backend)
+    engine.rootContext().setContextProperty("startPage", a.page)
+    engine.load(QUrl.fromLocalFile(os.path.join(HERE, "main.qml")))
+    if not engine.rootObjects():
+        sys.exit(1)
+
+    # SIGTERM and Ctrl+C quit as closing does, so a running session still stops cleanly. Python
+    # runs signal handlers between bytecodes: the timer gives it some while Qt waits.
+    def on_signal(*_):
+        engine.rootObjects()[0].setProperty("quitting", True)
+        app.quit()
+    signal.signal(signal.SIGTERM, on_signal)
+    signal.signal(signal.SIGINT, on_signal)
+    tick = QTimer(interval=500, timeout=lambda: None)
+    tick.start()
+    sys.exit(app.exec())
+
+
+if __name__ == "__main__":
+    main()

hands/rec/takes.py

@@ -0,0 +1,539 @@
+#!/usr/bin/env python3
+"""The hand recorder's sessions and takes on disk: listing, review, deleting, export.
+
+Standard library only (no Qt, no NumPy), so the window and the command line share it.
+The layout is hands/rec/DESIGN.md's "Files":
+  BASE/profile.json
+  BASE/sessions/<YYYYMMDD-HHMMSS>/session.json, calibration.json
+  BASE/sessions/<id>/takes/<NN>-<section>/sets.bin (sets-2.bin, ... after pauses),
+                                          prompts.jsonl, poses.jsonl, take.json
+  BASE/exports/<session>/
+A take's recording is one or more FHSET01 files (hands/track/record.h): per set a header
+(magic, ncams, bytes), one fh_set_cam_t per camera, then each camera's 8-bit pixels.
+Deleted ranges live in take.json ("deleted": [[from_ns, to_ns], ...], CLOCK_MONOTONIC, the
+clock of dqbuf_ns); the files keep every set until export leaves them out.
+
+usage: takes.py [--base DIR] list | takes SESSION | export SESSION
+"""
+import argparse
+import datetime
+import hashlib
+import json
+import os
+import re
+import shutil
+import struct
+import subprocess
+import threading
+
+DEFAULT_BASE = os.path.expanduser("~/.local/share/frametop/hands/contrib")
+REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+MAGIC = b"FHSET01"
+HDR = struct.Struct("<8sII")       # fh_set_hdr_t: magic, ncams, bytes (the whole record)
+CAM = struct.Struct("<16sIIQQ")    # fh_set_cam_t: name, width, height, capture_ns, dqbuf_ns
+MAX_CAMS = 16                      # SetReader's limit
+PART_RE = re.compile(r"^sets(?:-(\d+))?\.bin$")
+SESSION_RE = re.compile(r"^\d{8}-\d{6}$")
+EXPORT_SCHEMA = 1
+# zstd as DESIGN.md has it: level 10, two threads, at the lowest CPU priority (CPU work while
+# someone is in VR makes the headset stutter).
+ZSTD_ARGS = ["-10", "-T2", "-q", "-c"]
+ZSTD_PATHS = ("/usr/bin/zstd", "/usr/local/bin/zstd", "/run/host/usr/bin/zstd")
+CHUNK = 1 << 20
+
+
+class Cancelled(Exception):
+    pass
+
+
+def read_json(path, default=None):
+    try:
+        with open(path) as f:
+            return json.load(f)
+    except (OSError, ValueError):
+        return {} if default is None else default
+
+
+def write_json(path, obj):
+    """Write through a temporary file, so a crash never leaves half a file."""
+    tmp = path + ".tmp"
+    with open(tmp, "w") as f:
+        json.dump(obj, f, indent=2)
+        f.write("\n")
+    os.replace(tmp, path)
+
+
+def find_zstd():
+    """The zstd binary, or None. The dev container gets it from setup/dev-container.sh."""
+    found = shutil.which("zstd")
+    if found:
+        return found
+    return next((p for p in ZSTD_PATHS if os.access(p, os.X_OK)), None)
+
+
+def tool_version():
+    """ft-handrec plus the checkout's git describe, for session.json and the manifest."""
+    try:
+        out = subprocess.run(["git", "-C", REPO, "describe", "--always", "--dirty", "--tags"],
+                             capture_output=True, text=True, timeout=5)
+        described = out.stdout.strip()
+    except (OSError, subprocess.TimeoutExpired):
+        described = ""
+    return "ft-handrec " + (described or "unknown")
+
+
+def free_bytes(path):
+    """Free space on the file system that holds path (or its nearest existing parent)."""
+    while path and not os.path.exists(path):
+        path = os.path.dirname(path)
+    try:
+        return shutil.disk_usage(path or "/").free
+    except OSError:
+        return 0
+
+
+def tree_stat(path):
+    """(bytes, newest mtime) of the files under path."""
+    total, newest = 0, 0.0
+    for root, _, files in os.walk(path):
+        for name in files:
+            try:
+                st = os.lstat(os.path.join(root, name))
+            except OSError:
+                continue
+            total += st.st_size
+            newest = max(newest, st.st_mtime)
+    return total, newest
+
+
+def tree_bytes(path):
+    return tree_stat(path)[0]
+
+
+def in_ranges(t, ranges):
+    return any(a <= t <= b for a, b in ranges)
+
+
+def merge_ranges(ranges):
+    """Sorted, with overlapping or touching ranges joined."""
+    out = []
+    for a, b in sorted((min(a, b), max(a, b)) for a, b in ranges):
+        if out and a <= out[-1][1] + 1:
+            out[-1][1] = max(out[-1][1], b)
+        else:
+            out.append([a, b])
+    return out
+
+
+# ---------------------------------------------------------------- one take's recording
+
+class TakeIndex:
+    """Where each set of a take's recording is, without reading pixels. Sets are numbered
+    across the parts (sets.bin, sets-2.bin, ...) in order."""
+
+    def __init__(self, take_dir):
+        self.dir = take_dir
+        self.parts = self._parts(take_dir)
+        self.sets = []   # (part, offset, bytes, time_ns): time_ns is the cameras' earliest dqbuf_ns
+        self.part_starts = []   # index of each part's first set
+        self.cams = []   # the first set's [{name, width, height}]
+        for p, path in enumerate(self.parts):
+            self.part_starts.append(len(self.sets))
+            self._index(p, path)
+
+    @staticmethod
+    def _parts(take_dir):
+        found = []
+        try:
+            names = os.listdir(take_dir)
+        except OSError:
+            return []
+        for name in names:
+            m = PART_RE.match(name)
+            if m:
+                found.append((int(m.group(1) or 1), os.path.join(take_dir, name)))
+        return [path for _, path in sorted(found)]
+
+    def _index(self, p, path):
+        try:
+            size = os.path.getsize(path)
+            f = open(path, "rb")
+        except OSError:
+            return
+        with f:
+            off = 0
+            while off + HDR.size <= size:
+                f.seek(off)
+                head = f.read(HDR.size)
+                if len(head) < HDR.size:
+                    break
+                magic, ncams, nbytes = HDR.unpack(head)
+                # A truncated last set (the recorder was killed mid-write) ends the part.
+                if magic[:7] != MAGIC or not 0 < ncams <= MAX_CAMS or nbytes < HDR.size or off + nbytes > size:
+                    break
+                cams = f.read(CAM.size * ncams)
+                if len(cams) < CAM.size * ncams:
+                    break
+                fields = [CAM.unpack_from(cams, i * CAM.size) for i in range(ncams)]
+                if not self.cams:
+                    self.cams = [{"name": n.split(b"\0", 1)[0].decode(errors="replace"), "width": w, "height": h}
+                                 for n, w, h, _, _ in fields]
+                self.sets.append((p, off, nbytes, min(c[4] for c in fields)))
+                off += nbytes
+
+    def __len__(self):
+        return len(self.sets)
+
+    def time_ns(self, i):
+        return self.sets[i][3]
+
+    def duration_s(self):
+        """Recorded time: each part's first to last set, so pauses don't count."""
+        total = 0
+        ends = self.part_starts[1:] + [len(self.sets)]
+        for start, end in zip(self.part_starts, ends):
+            if end > start:
+                total += self.sets[end - 1][3] - self.sets[start][3]
+        return total / 1e9
+
+    def bytes(self):
+        return sum(s[2] for s in self.sets)
+
+    def read_set(self, i, only=None):
+        """Set i's cameras: [{name, width, height, capture_ns, dqbuf_ns, pixels}], pixels as raw
+        8-bit grey bytes (width x height, packed). only: a camera name, to read just that one."""
+        p, off, nbytes, _ = self.sets[i]
+        with open(self.parts[p], "rb") as f:
+            f.seek(off)
+            ncams = HDR.unpack(f.read(HDR.size))[1]
+            heads = f.read(CAM.size * ncams)
+            out = []
+            px_off = off + HDR.size + CAM.size * ncams
+            for k in range(ncams):
+                name, w, h, capture_ns, dqbuf_ns = CAM.unpack_from(heads, k * CAM.size)
+                name = name.split(b"\0", 1)[0].decode(errors="replace")
+                if only is None or name == only:
+                    f.seek(px_off)
+                    out.append({"name": name, "width": w, "height": h, "capture_ns": capture_ns,
+                                "dqbuf_ns": dqbuf_ns, "pixels": f.read(w * h)})
+                px_off += w * h
+            return out
+
+    def read_raw(self, i):
+        """Set i's whole record, as it is in the file."""
+        p, off, nbytes, _ = self.sets[i]
+        with open(self.parts[p], "rb") as f:
+            f.seek(off)
+            return f.read(nbytes)
+
+
+_index_cache = {}
+_index_lock = threading.Lock()
+
+
+def take_index(take_dir):
+    """A TakeIndex, cached until one of the take's recording files changes size or time."""
+    key_parts = []
+    for path in TakeIndex._parts(take_dir):
+        try:
+            st = os.stat(path)
+            key_parts.append((path, st.st_size, st.st_mtime_ns))
+        except OSError:
+            pass
+    key = tuple(key_parts)
+    with _index_lock:
+        cached = _index_cache.get(take_dir)
+        if cached and cached[0] == key:
+            return cached[1]
+    index = TakeIndex(take_dir)
+    with _index_lock:
+        _index_cache[take_dir] = (key, index)
+    return index
+
+
+# ---------------------------------------------------------------- the store
+
+class Store:
+    def __init__(self, base=DEFAULT_BASE):
+        self.base = os.path.abspath(os.path.expanduser(base))
+        self.sessions_dir = os.path.join(self.base, "sessions")
+        self.exports_dir = os.path.join(self.base, "exports")
+        self.profile_path = os.path.join(self.base, "profile.json")
+
+    # --- paths, checked: ids come from the window, so nothing may climb out of the base
+    def session_dir(self, session):
+        if not SESSION_RE.match(session or ""):
+            raise ValueError(f"not a session id: {session!r}")
+        return os.path.join(self.sessions_dir, session)
+
+    def take_dir(self, session, take):
+        if not take or "/" in take or take.startswith("."):
+            raise ValueError(f"not a take id: {take!r}")
+        return os.path.join(self.session_dir(session), "takes", take)
+
+    def export_dir(self, session):
+        self.session_dir(session)  # checks the id
+        return os.path.join(self.exports_dir, session)
+
+    def profile(self):
+        return read_json(self.profile_path)
+
+    # --- listing
+    def sessions(self):
+        """Newest first: [{id, started, takes, bytes, lighting, status, dry_run, contributor, exported,
+        export_stale (changed since), export_bytes}]."""
+        out = []
+        try:
+            names = os.listdir(self.sessions_dir)
+        except OSError:
+            return out
+        for sid in names:
+            path = os.path.join(self.sessions_dir, sid)
+            if not SESSION_RE.match(sid) or not os.path.isdir(path):
+                continue
+            meta = read_json(os.path.join(path, "session.json"))
+            size, changed = tree_stat(path)
+            manifest = os.path.join(self.export_dir(sid), "manifest.json")
+            exported = os.path.getmtime(manifest) if os.path.isfile(manifest) else 0.0
+            out.append({"id": sid, "started": meta.get("started", ""), "takes": len(self.take_ids(sid)),
+                        "bytes": size, "contributor": meta.get("contributor", ""),
+                        "lighting": (meta.get("lighting") or {}).get("chosen", ""),
+                        "status": meta.get("status", ""), "dry_run": bool(meta.get("dry_run")),
+                        "exported": bool(exported), "export_stale": bool(exported) and changed > exported,
+                        "export_bytes": tree_bytes(self.export_dir(sid)) if exported else 0})
+        return sorted(out, key=lambda s: s["id"], reverse=True)
+
+    def take_ids(self, session):
+        try:
+            names = os.listdir(os.path.join(self.session_dir(session), "takes"))
+        except OSError:
+            return []
+        return sorted(n for n in names if not n.startswith(".")
+                      and os.path.isdir(os.path.join(self.session_dir(session), "takes", n)))
+
+    def takes(self, session):
+        """[{id, section, title, status, sets, deleted_sets, ranges, duration_s, bytes, cams}]."""
+        out = []
+        for tid in self.take_ids(session):
+            path = self.take_dir(session, tid)
+            meta = read_json(os.path.join(path, "take.json"))
+            index = take_index(path)
+            ranges = merge_ranges(meta.get("deleted") or [])
+            out.append({"id": tid, "section": meta.get("section", tid.split("-", 1)[-1]),
+                        "title": meta.get("title") or tid, "status": meta.get("status", "recording"),
+                        "sets": len(index), "parts": len(index.parts),
+                        "deleted_sets": sum(1 for s in index.sets if in_ranges(s[3], ranges)),
+                        "ranges": ranges, "duration_s": index.duration_s(), "bytes": index.bytes(),
+                        "cams": [c["name"] for c in index.cams]})
+        return out
+
+    def take_meta(self, session, take):
+        return read_json(os.path.join(self.take_dir(session, take), "take.json"))
+
+    # --- deleted ranges
+    def ranges(self, session, take):
+        return merge_ranges(self.take_meta(session, take).get("deleted") or [])
+
+    def _set_ranges(self, session, take, ranges):
+        path = os.path.join(self.take_dir(session, take), "take.json")
+        meta = read_json(path)
+        meta["deleted"] = merge_ranges(ranges)
+        write_json(path, meta)
+        return meta["deleted"]
+
+    def delete_range(self, session, take, from_ns, to_ns):
+        """Leave sets from from_ns to to_ns (dqbuf_ns, inclusive) out of the export."""
+        return self._set_ranges(session, take, self.ranges(session, take) + [[int(from_ns), int(to_ns)]])
+
+    def restore_range(self, session, take, i):
+        """Take deleted range i (in ranges()' order) back."""
+        ranges = self.ranges(session, take)
+        if 0 <= i < len(ranges):
+            del ranges[i]
+        return self._set_ranges(session, take, ranges)
+
+    # --- deleting files
+    def delete_take(self, session, take):
+        path = self.take_dir(session, take)
+        if os.path.isdir(path):
+            shutil.rmtree(path)
+        meta_path = os.path.join(self.session_dir(session), "session.json")
+        meta = read_json(meta_path)
+        if take in (meta.get("takes") or []):
+            meta["takes"] = [t for t in meta["takes"] if t != take]
+            meta.setdefault("deleted_takes", []).append(take)
+            write_json(meta_path, meta)
+
+    def delete_session(self, session):
+        """The session and its export, if any."""
+        for path in (self.session_dir(session), self.export_dir(session)):
+            if os.path.isdir(path):
+                shutil.rmtree(path)
+
+    def delete_export(self, session):
+        path = self.export_dir(session)
+        if os.path.isdir(path):
+            shutil.rmtree(path)
+
+    # --- export
+    def export(self, session, progress=None, cancel=None, keep_notes=False, low_priority=True):
+        """Write exports/<session>/ (DESIGN.md "Export"): manifest.json, calibration.json, per
+        take prompts.jsonl, poses.jsonl, take.json and sets.bin.zst (sets in deleted ranges
+        left out), then SHA256SUMS. progress(fraction 0..1, text) is called as it goes;
+        cancel() (or cancel.is_set()) returning true stops it, raising Cancelled. Nothing is
+        left behind on failure or cancel. Returns the export's path.
+        low_priority: run this thread at nice 19 as well as zstd (Linux nice is per thread)."""
+        zstd = find_zstd()
+        if not zstd:
+            raise RuntimeError("zstd isn't installed (in the dev container: sudo dnf install zstd)")
+        cancelled = getattr(cancel, "is_set", cancel) or (lambda: False)
+        report = progress or (lambda fraction, text: None)
+        src = self.session_dir(session)
+        if not os.path.isdir(src):
+            raise RuntimeError(f"no session {session}")
+        if low_priority:
+            try:
+                os.setpriority(os.PRIO_PROCESS, threading.get_native_id(), 19)
+            except OSError:
+                pass
+        final = self.export_dir(session)
+        work = final + ".partial"
+        if os.path.isdir(work):
+            shutil.rmtree(work)
+        os.makedirs(work)
+        try:
+            self._export(session, src, work, zstd, report, cancelled, keep_notes)
+            if os.path.isdir(final):
+                shutil.rmtree(final)
+            os.replace(work, final)
+        except BaseException:
+            shutil.rmtree(work, ignore_errors=True)
+            raise
+        report(1.0, "Done")
+        return final
+
+    def _export(self, session, src, work, zstd, report, cancelled, keep_notes):
+        profile = self.profile()
+        meta = read_json(os.path.join(src, "session.json"))
+        takes = self.takes(session)
+        total = sum(t["bytes"] for t in takes) or 1
+        done = 0
+        take_entries = []
+        for t in takes:
+            if cancelled():
+                raise Cancelled()
+            tdir = self.take_dir(session, t["id"])
+            out = os.path.join(work, "takes", t["id"])
+            os.makedirs(out)
+            for name in ("prompts.jsonl", "poses.jsonl", "take.json"):
+                if os.path.isfile(os.path.join(tdir, name)):
+                    shutil.copyfile(os.path.join(tdir, name), os.path.join(out, name))
+            index = take_index(tdir)
+            ranges = t["ranges"]
+            keep = [i for i in range(len(index)) if not in_ranges(index.time_ns(i), ranges)]
+            entry = {"id": t["id"], "section": t["section"], "title": t["title"], "status": t["status"],
+                     "sets": len(keep), "sets_deleted": len(index) - len(keep), "cameras": index.cams,
+                     "duration_s": round(t["duration_s"], 2), "file": None}
+            done += index.bytes() - sum(index.sets[i][2] for i in keep)   # deleted sets count as done
+            if keep:
+                entry["file"] = "takes/%s/sets.bin.zst" % t["id"]
+
+                def step(nbytes, title=t["title"]):
+                    nonlocal done
+                    done += nbytes
+                    report(min(done / total, 0.99), f"Compressing {title}")
+                entry["raw_bytes"] = self._compress(index, keep, zstd, os.path.join(out, "sets.bin.zst"),
+                                                    step, cancelled)
+            take_entries.append(entry)
+        if cancelled():
+            raise Cancelled()
+        if os.path.isfile(os.path.join(src, "calibration.json")):
+            shutil.copyfile(os.path.join(src, "calibration.json"), os.path.join(work, "calibration.json"))
+        shown_profile = json.loads(json.dumps(profile))
+        if not keep_notes and isinstance(shown_profile.get("optional"), dict):
+            shown_profile["optional"].pop("notes", None)
+        manifest = {"schema": EXPORT_SCHEMA, "tool": tool_version(),
+                    "exported": datetime.datetime.now().astimezone().isoformat(timespec="seconds"),
+                    "session_id": session, "contributor": profile.get("contributor", meta.get("contributor", "")),
+                    "consent_version": (profile.get("consent") or {}).get("version", ""),
+                    "profile": shown_profile, "session": meta, "takes": take_entries}
+        write_json(os.path.join(work, "manifest.json"), manifest)
+        report(0.995, "Writing checksums")
+        sums = []
+        for root, _, files in os.walk(work):
+            for name in files:
+                path = os.path.join(root, name)
+                sums.append((os.path.relpath(path, work), sha256_file(path, cancelled)))
+        with open(os.path.join(work, "SHA256SUMS"), "w") as f:
+            for rel, digest in sorted(sums):
+                f.write(f"{digest}  {rel}\n")
+
+    @staticmethod
+    def _compress(index, keep, zstd, dest, step, cancelled):
+        """Stream the kept sets through zstd into dest. Returns the uncompressed size."""
+        raw = 0
+        with open(dest, "wb") as out:
+            proc = subprocess.Popen(["nice", "-n", "19", zstd] + ZSTD_ARGS, stdin=subprocess.PIPE, stdout=out,
+                                    stderr=subprocess.PIPE)
+            try:
+                for i in keep:
+                    if cancelled():
+                        raise Cancelled()
+                    record = index.read_raw(i)
+                    proc.stdin.write(record)
+                    raw += len(record)
+                    step(len(record))
+                proc.stdin.close()
+                err = proc.stderr.read().decode(errors="replace").strip()
+                if proc.wait() != 0:
+                    raise RuntimeError(f"zstd failed: {err or proc.returncode}")
+            except BaseException:
+                proc.kill()
+                proc.wait()
+                raise
+        return raw
+
+
+def sha256_file(path, cancelled=lambda: False):
+    h = hashlib.sha256()
+    with open(path, "rb") as f:
+        while chunk := f.read(CHUNK):
+            if cancelled():
+                raise Cancelled()
+            h.update(chunk)
+    return h.hexdigest()
+
+
+def human_bytes(n):
+    for unit in ("B", "KB", "MB", "GB"):
+        if n < 1000 or unit == "GB":
+            return f"{n:.0f} {unit}" if unit == "B" else f"{n:.1f} {unit}"
+        n /= 1000
+    return f"{n:.1f} TB"
+
+
+def main():
+    ap = argparse.ArgumentParser(description="List, check and export hand recorder sessions.")
+    ap.add_argument("--base", default=DEFAULT_BASE)
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    sub.add_parser("list")
+    sub.add_parser("takes").add_argument("session")
+    sub.add_parser("export").add_argument("session")
+    a = ap.parse_args()
+    store = Store(a.base)
+    if a.cmd == "list":
+        for s in store.sessions():
+            print(f"{s['id']}  {s['takes']} takes  {human_bytes(s['bytes'])}  {s['lighting'] or '-'}"
+                  + ("  exported" if s["exported"] else ""))
+    elif a.cmd == "takes":
+        for t in store.takes(a.session):
+            print(f"{t['id']}  {t['status']}  {t['sets']} sets ({t['deleted_sets']} deleted)  "
+                  f"{t['duration_s']:.1f} s  {human_bytes(t['bytes'])}")
+    else:
+        path = store.export(a.session, progress=lambda f, text: print(f"\r{f * 100:5.1f}% {text:40.40}", end="",
+                                                                       flush=True))
+        print(f"\n{path}  {human_bytes(tree_bytes(path))}")
+
+
+if __name__ == "__main__":
+    main()

setup/dev-container.sh

@@ -23,6 +23,8 @@ packages=(
   # tools (hands/tools) need NumPy and OpenCV, which aren't here: Fedora's python3-opencv pulls
   # in over a gigabyte (hands/README.md says how to get them)
   jsoncpp-devel glibc-static
+  # hand recorder (hands/rec): its export compresses recordings with zstd
+  zstd
   # Frametop Input Settings app (Kirigami, PySide6)
   python3-pyside6 kf6-kirigami kf6-qqc2-desktop-style qt6-qtwayland breeze-icon-theme plasma-breeze
   # Frametop remote desktop (VNC bridge through krdp)