diff --git a/hands/rec/CONSENT.md b/hands/rec/CONSENT.md new file mode 100644 index 0000000..c7fe850 --- /dev/null +++ b/hands/rec/CONSENT.md @@ -0,0 +1,60 @@ +**DRAFT: this text hasn't had a legal review yet. Contributions aren't open until it has.** + +# Recording your hands for the Frametop hand dataset + +Version: 2026-10-02 + +Frametop's hand tracking needs a small model that finds hands in the headset's camera images. To train it, we're collecting recordings of many people's hands. This page explains what the hand recorder records, what happens to it, and what you agree to if you take part. Please read all of it. + +## Who can take part + +You must be 18 or older. + +## What is recorded + +While a session runs, the hand recorder saves: + +- **Camera images.** Infrared images from the headset's 4 tracking cameras, about 10 sets a second. They show your hands, your arms and whatever is in front of you: your room, your desk and the things on it. They're grey, low-detail images, but people and things can be recognised in them. +- **Motion.** The position and rotation of the headset and the controllers, many times a second. +- **Prompts.** What you were asked to do and when, and what the live hand tracker saw at the time. +- **Calibration.** Where the cameras sit on the headset and how their lenses bend the image. Serial numbers and other fields that identify your headset are removed first, and the export lists what was removed. +- **Your checklist answers.** Which objects you had, the lighting you chose, whether you wore sleeves, rings or a watch, and any notes you typed in. +- **A contributor id.** A random number made on your headset the first time you agree to this page. It isn't linked to your name, your Steam account or your headset. It lets us keep your sessions together and find them if you withdraw. + +The recorder doesn't record sound, your name, your email address or your account. Your eyes and face aren't recorded: the tracking cameras look outward. + +## Nothing leaves your headset unless you send it + +- Recordings stay on your headset, in `~/.local/share/frametop/hands/contrib`. Nothing is uploaded automatically. +- Before you share anything, you can watch every recording in the Review page. You can delete any stretch of a recording, a whole take or a whole session. +- Export makes a package from what you kept. You upload it yourself, with your own Hugging Face account, following the Upload page. Until you do, nobody else has it. +- An upload opens a pull request. The maintainer checks it before it becomes part of the dataset, and may decline it. + +## Keep other people and private things out of view + +While recording, please: + +- face away from other people, mirrors, screens showing private things, papers, letters and anything else you wouldn't want in a public dataset; +- make sure nobody else's face or hands are in view. + +If something private got into a recording, delete that stretch in Review before you export. If you notice it after uploading, withdraw the session (below). + +## The license + +- **The dataset is published under Creative Commons Attribution-NonCommercial 4.0 (CC BY-NC 4.0).** Anyone may use it for non-commercial purposes, with attribution. Your contribution is credited by its contributor id, not your name. +- **You also give the maintainer, DeeJanuz, a non-exclusive license to use your contribution for any purpose, including commercially.** That includes copying it, changing it, and training, publishing and selling models made from it, in Frametop and elsewhere. It's non-exclusive: you keep any rights you have in your recordings and can do what you like with your own copies. +- You confirm that you have the right to give these licenses: the recordings are yours, and nothing in them belongs to someone who hasn't agreed. +- There is no payment, and the dataset comes with no warranty. + +## Withdrawing + +You can withdraw a contribution at any time. Send your contributor id (shown in the hand recorder) and which sessions to withdraw, or "all", through the dataset's discussion page or the Frametop repository's issues. Then: + +- your recordings are deleted from the dataset and purged from its repository's history, so they can't be downloaded from there again; +- they're left out of anything trained after that. + +What can't be undone: copies others downloaded before the withdrawal, and models already trained with them. + +## Agreeing + +By ticking "I'm 18 or older" and "I agree", you confirm the above. You can still decide not to upload anything. If this text changes, the hand recorder asks you again before your next session. diff --git a/hands/rec/UPLOAD.md b/hands/rec/UPLOAD.md new file mode 100644 index 0000000..09d2dc5 --- /dev/null +++ b/hands/rec/UPLOAD.md @@ -0,0 +1,57 @@ +**DRAFT: contributions aren't open yet. Please don't upload until this banner is gone.** + +# Uploading your recordings + +Your export is ready: + +- Folder: `@EXPORT_PATH@` +- Size: @EXPORT_SIZE@ +- Contributor id: `@CONTRIBUTOR@` + +Uploads go to the Hugging Face dataset [@DATASET@](https://huggingface.co/datasets/@DATASET@). You do it yourself, from your own account; the hand recorder never uploads anything. The upload opens a pull request, so nothing is published until the maintainer has looked at it. + +## 1. Make a Hugging Face account + +Sign up at , if you don't have an account. Your username shows on your pull request, but the dataset credits your contributor id, not your name. + +## 2. Accept the dataset's terms + +Open , read the terms and accept them. They're the same as the consent you agreed to in the hand recorder. + +## 3. Create a write token + +Go to , press "Create new token", choose "Write" and give it a name like "frametop-hands". Copy the token. + +The token lets anyone who has it change things in your account. Paste it only into your own terminal in the next step: never into a chat, a website, an issue or this window. + +## 4. Install the Hugging Face tools + +Open a terminal (Konsole) and enter the dev container, then install `huggingface_hub` and log in: + +``` +distrobox enter dev +pip install --user huggingface_hub +huggingface-cli login +``` + +`huggingface-cli login` asks for the token: paste it there. If it asks whether to add the token as a git credential, answer no. + +## 5. Upload + +In the same terminal, run: + +``` +@COMMAND@ +``` + +It uploads the export folder to `contributions/@CONTRIBUTOR@/@SESSION@` in the dataset and opens a pull request. Large uploads take a while. If it fails partway, run the same command again: parts already sent usually aren't sent twice. That can open a second pull request, which is fine: the maintainer closes the incomplete one. + +Keep the headset on its charger or plugged in while it uploads. + +## 6. The maintainer reviews it + +The maintainer checks your pull request (that the files are complete, and that nobody else and nothing private is in view) before merging it into the dataset. You can follow it, and answer questions, on the pull request's page. + +When it's merged, you can delete the session and its export on your headset to free the space. + +To withdraw a contribution later, see "Withdrawing" in the consent text: you'll need your contributor id, `@CONTRIBUTOR@`. diff --git a/hands/rec/ft-handrec b/hands/rec/ft-handrec new file mode 100755 index 0000000..86b5a98 --- /dev/null +++ b/hands/rec/ft-handrec @@ -0,0 +1,18 @@ +#!/bin/bash +# Launch the Frametop Hand Recorder from a Plasma session on the Frame host. +# The app runs in the dev container (PySide6, Kirigami and zstd come from Fedora there). +# podman needs the real XDG_RUNTIME_DIR and the real user bus (to reach systemd for +# the container's cgroup; the Frametop session runs on a private bus from +# dbus-run-session). The session's Wayland socket and bus go to the app itself. +# Options go to ft_handrec.py: --base DIR, --page NAME, and --dry-run for testing. +here=$(cd "$(dirname "$(readlink -f "$0")")" && pwd) +wl=${WAYLAND_DISPLAY:-wayland-0} +case $wl in /*) ;; *) wl="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/$wl" ;; esac +session_bus=${DBUS_SESSION_BUS_ADDRESS:-} +export XDG_RUNTIME_DIR=/run/user/$(id -u) +export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus +"$here/../../scripts/container-up.sh" +exec "$HOME/.local/bin/distrobox" enter dev -- env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \ + XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \ + QT_QPA_PLATFORM="wayland;xcb" \ + python3 "$here/ft_handrec.py" "$@" diff --git a/hands/rec/ft-handrec.desktop b/hands/rec/ft-handrec.desktop new file mode 100644 index 0000000..e49e2d3 --- /dev/null +++ b/hands/rec/ft-handrec.desktop @@ -0,0 +1,9 @@ +[Desktop Entry] +Type=Application +Name=Frametop Hand Recorder +GenericName=Record your hands for the hand dataset +Comment=Record, review and export hand recordings for Frametop's open hand dataset +Exec=@REPO@/hands/rec/ft-handrec +Icon=camera-video +Categories=Utility; +Keywords=hands;hand tracking;dataset;record;frametop; diff --git a/hands/rec/ft_handrec.py b/hands/rec/ft_handrec.py new file mode 100755 index 0000000..4a1bc13 --- /dev/null +++ b/hands/rec/ft_handrec.py @@ -0,0 +1,713 @@ +#!/usr/bin/env python3 +"""Frametop Hand Recorder: record your hands for the open hand dataset (hands/rec/DESIGN.md). + +A Kirigami (QML) app with a Python backend. It runs in the dev container: + - Welcome: the consent text (CONSENT.md), shown the first time and again when its version + changes. Agreeing writes profile.json with a random contributor id. + - Before you start: the checklist (objects, controller straps, lighting, sleeves, privacy, + free space) and what will happen. Start hands it to the session runner (session.py). + - Session: the runner's live status, Start, Pause/Resume, Skip section and Stop (Space + pauses and Esc stops while the window has focus). The prompts appear in the headset. + - Review: sessions, their takes, and a viewer for one frame set at a time, where ranges, + takes and sessions can be deleted (takes.py). + - Export: compress what's kept into exports// at nice 19 (takes.py), with a warning + when the headset is worn. + - Upload: UPLOAD.md with the export filled in and the upload command to copy. +Everything lives under ~/.local/share/frametop/hands/contrib (--base). Nothing is uploaded +from here. Launch with hands/rec/ft-handrec (host wrapper). +""" +import argparse +import datetime +import os +import re +import shlex +import signal +import sys +import threading +import uuid + +from PySide6.QtCore import Property, QObject, Qt, QTimer, QUrl, Signal, Slot +from PySide6.QtGui import QColor, QFont, QGuiApplication, QIcon, QImage, QPainter, QPalette +from PySide6.QtQml import QQmlApplicationEngine +from PySide6.QtQuick import QQuickImageProvider +from PySide6.QtQuickControls2 import QQuickStyle + +HERE = os.path.dirname(os.path.abspath(__file__)) +sys.path.insert(0, HERE) +import takes # noqa: E402 (next to this file) + +# The dataset contributions go to: a placeholder until the maintainer decides (DESIGN.md). +HF_DATASET = "DeeJanuz/frametop-hands" +CONSENT_PATH = os.path.join(HERE, "CONSENT.md") +UPLOAD_PATH = os.path.join(HERE, "UPLOAD.md") +SCRIPT_PATH = os.path.join(HERE, "script.json") +# The headset counts as worn while vrcompositor runs and a display panel is lit: SteamVR turns +# the panels off 5 s after the headset comes off (frame-job's check; the proximity sensor's +# readings are too noisy). The dev container sees the host's processes and /sys. +BACKLIGHTS = "/sys/class/backlight" +ROUND_BYTES = 10 * 1000 ** 3 # about what one round of recording takes +# The checklist's choices; the keys are what session.json stores. +OBJECTS = [("pencil", "Pencil or pen"), ("phone", "Phone"), ("cup", "Cup or mug (empty)"), + ("keyboard", "Keyboard"), ("mouse", "Mouse"), ("gamepad", "Gamepad"), + ("small", "Something small (a coin, a key, a bottle cap)")] +LIGHTING = [("dim", "Dim: one lamp only"), ("room", "Normal room light"), ("daylight", "Daylight near a window")] +SLEEVES = [("short", "Short sleeves or bare arms"), ("long", "Long sleeves"), ("", "Rather not say")] +HANDEDNESS = [("", "Rather not say"), ("right", "Right-handed"), ("left", "Left-handed"), + ("both", "Both (ambidextrous)")] +ACTIVE_STATES = ("starting", "intro", "running", "paused", "between") +# Shown side by side in the viewer at this height; thumbnails are smaller. +SET_HEIGHT = 480 +THUMB_HEIGHT = 96 + + +def read_text(path): + try: + with open(path, encoding="utf-8") as f: + return f.read() + except OSError: + return "" + + +def consent_version(): + """The "Version: ..." line of CONSENT.md: a new one asks everyone to agree again.""" + m = re.search(r"^Version:\s*(\S+)", read_text(CONSENT_PATH), re.M) + return m.group(1) if m else "unknown" + + +def is_draft(path): + return "DRAFT" in read_text(path).split("\n", 1)[0] + + +def process_running(name): + """Like pgrep -x NAME, from /proc.""" + for pid in os.listdir("/proc"): + if pid.isdigit(): + try: + with open(f"/proc/{pid}/comm") as f: + if f.read().strip() == name: + return True + except OSError: + pass + return False + + +def headset_worn(): + """True while vrcompositor runs and any panel's backlight is on; False if unreadable.""" + lit = False + try: + for name in os.listdir(BACKLIGHTS): + try: + with open(os.path.join(BACKLIGHTS, name, "brightness")) as f: + lit = lit or int(f.read().strip()) > 0 + except (OSError, ValueError): + pass + except OSError: + return False + return lit and process_running("vrcompositor") + + +def gigabytes(n): + return f"{n / 1000 ** 3:.1f} GB" + + +def session_label(sid): + """20261002-101500 -> 2026-10-02 10:15.""" + try: + return datetime.datetime.strptime(sid, "%Y%m%d-%H%M%S").strftime("%Y-%m-%d %H:%M") + except ValueError: + return sid + + +def grey_image(cam): + """A camera's raw 8-bit pixels as a QImage that owns its data.""" + img = QImage(cam["pixels"], cam["width"], cam["height"], cam["width"], QImage.Format_Grayscale8) + return img.copy() + + +class FrameProvider(QQuickImageProvider): + """image://frames/set/SESSION/TAKE/INDEX: one frame set, every camera side by side. + image://frames/thumb/SESSION/TAKE: the take's first slam_left image, small. + Anything after "?" is ignored (it makes QML load the image again).""" + + def __init__(self, store): + super().__init__(QQuickImageProvider.ImageType.Image) + self.store = store + + def requestImage(self, ident, size, requested): + parts = ident.split("?", 1)[0].split("/") + try: + if parts[0] == "thumb" and len(parts) == 3: + return self._thumb(parts[1], parts[2]) + if parts[0] == "set" and len(parts) == 4: + return self._set(parts[1], parts[2], int(parts[3]), + requested.height() if requested.height() > 0 else SET_HEIGHT) + except (OSError, ValueError, IndexError, KeyError) as e: + print(f"ft-handrec: image {ident}: {e}", file=sys.stderr) + img = QImage(4, 3, QImage.Format_Grayscale8) + img.fill(0) + return img + + def _index(self, session, take): + return takes.take_index(self.store.take_dir(session, take)) + + def _thumb(self, session, take): + index = self._index(session, take) + if not len(index): + raise ValueError("no sets") + names = [c["name"] for c in index.cams] + cams = index.read_set(0, only="slam_left" if "slam_left" in names else names[0]) + return grey_image(cams[0]).scaledToHeight(THUMB_HEIGHT, Qt.SmoothTransformation) + + def _set(self, session, take, i, height): + cams = self._index(session, take).read_set(i) + scaled = [grey_image(c).scaledToHeight(height, Qt.SmoothTransformation) for c in cams] + gap = 8 + out = QImage(sum(s.width() for s in scaled) + gap * (len(scaled) - 1), height, QImage.Format_RGB32) + out.fill(QColor(30, 30, 30)) + p = QPainter(out) + font = QFont() + font.setPixelSize(max(12, height // 28)) + p.setFont(font) + x = 0 + for cam, img in zip(cams, scaled): + p.drawImage(x, 0, img) + p.setPen(QColor(255, 200, 80)) + p.drawText(x + 6, 6 + font.pixelSize(), cam["name"]) + x += img.width() + gap + p.end() + return out + + +class Backend(QObject): + profileChanged = Signal() + diskChanged = Signal() + lightingChanged = Signal() + statusChanged = Signal() + sessionsChanged = Signal() + exportChanged = Signal() + message = Signal(str, bool) # text, is error + # From other threads (the session runner, export, the lighting check): queued to this one. + _statusArrived = Signal(dict) + _lightingArrived = Signal(str) + _exportProgress = Signal(float, str) + _exportFinished = Signal(str, str) # path, error ("" when it worked; "cancelled") + + def __init__(self, store, session_options=None): + super().__init__() + self.store = store + self._session_options = session_options or {} # test hooks for Session: dry_run, speed + os.makedirs(store.base, mode=0o700, exist_ok=True) + self._session_mod = None + self._session_error = "" + self._session = None + self._session_id = "" + self._status = {} + self._lighting_note = "" + self._export_cancel = None + self._export_thread = None + self._export_fraction = 0.0 + self._export_text = "" + self._export_session = "" + self._statusArrived.connect(self._on_status) + self._lightingArrived.connect(self._on_lighting) + self._exportProgress.connect(self._on_export_progress) + self._exportFinished.connect(self._on_export_finished) + self.disk_timer = QTimer(interval=30000, timeout=self.diskChanged.emit) + self.disk_timer.start() + + def _thread(self, fn): + thread = threading.Thread(target=fn, daemon=True) + thread.start() + return thread + + # --- the session runner, imported when first needed (it's written separately) + def _runner(self): + if self._session_mod is None and not self._session_error: + try: + import session as mod + self._session_mod = mod + except Exception as e: # missing, or broken: say so, the rest of the app still works + self._session_error = f"The session runner (hands/rec/session.py) can't be loaded: {e}" + return self._session_mod + + @Property(str, notify=statusChanged) + def runnerError(self): + self._runner() + return self._session_error + + # --- consent and profile + @Property(str, constant=True) + def consentText(self): + return read_text(CONSENT_PATH) or "CONSENT.md is missing." + + @Property(str, constant=True) + def consentVersion(self): + return consent_version() + + @Property(bool, constant=True) + def textsDraft(self): + return is_draft(CONSENT_PATH) or is_draft(UPLOAD_PATH) + + @Property(bool, notify=profileChanged) + def needsConsent(self): + profile = self.store.profile() + consent = profile.get("consent") or {} + return not (profile.get("contributor") and consent.get("adult") is True + and consent.get("version") == consent_version()) + + @Property(str, notify=profileChanged) + def contributor(self): + return self.store.profile().get("contributor", "") + + @Property(str, notify=profileChanged) + def consentAccepted(self): + return (self.store.profile().get("consent") or {}).get("accepted", "") + + @Property(str, notify=profileChanged) + def handedness(self): + return (self.store.profile().get("optional") or {}).get("handedness", "") + + @Property("QVariantList", constant=True) + def handednessChoices(self): + return [{"value": k, "text": v} for k, v in HANDEDNESS] + + @Slot(bool, bool, str) + def acceptConsent(self, adult, agree, handedness): + """Write profile.json. A contributor id, once made, stays (a new consent version keeps it).""" + if not (adult and agree): + self.message.emit("Both boxes need ticking to take part", True) + return + profile = self.store.profile() + optional = profile.get("optional") if isinstance(profile.get("optional"), dict) else {} + optional["handedness"] = handedness if handedness in dict(HANDEDNESS) else "" + optional.setdefault("notes", "") + profile = {"schema": 1, "contributor": profile.get("contributor") or str(uuid.uuid4()), + "consent": {"version": consent_version(), + "accepted": datetime.datetime.now().astimezone().isoformat(timespec="seconds"), + "adult": True}, + "optional": optional} + takes.write_json(self.store.profile_path, profile) + self.profileChanged.emit() + self.message.emit("Thank you. Your contributor id: " + profile["contributor"], False) + + # --- the checklist + @Property("QVariantList", constant=True) + def objects(self): + return [{"value": k, "text": v} for k, v in OBJECTS] + + @Property("QVariantList", constant=True) + def lightingChoices(self): + return [{"value": k, "text": v} for k, v in LIGHTING] + + @Property("QVariantList", constant=True) + def sleeveChoices(self): + return [{"value": k, "text": v} for k, v in SLEEVES] + + @Property(str, notify=diskChanged) + def freeText(self): + return gigabytes(takes.free_bytes(self.store.base)) + + @Property(bool, notify=diskChanged) + def diskOk(self): + return takes.free_bytes(self.store.base) >= ROUND_BYTES + + @Property(str, notify=lightingChanged) + def lightingNote(self): + return self._lighting_note + + @Slot(str) + def checkLighting(self, chosen): + """Compare the cameras' brightness now with earlier rounds (session.similar_lighting): + a round in light like an earlier one adds less to the dataset.""" + mod = self._runner() + if not mod: + return + base = self.store.base + labels = dict(LIGHTING) + + def run(): + try: + ring = mod.ring_lighting() + if ring is None: + note = ("The cameras aren't running yet, so the light can't be compared with your " + "earlier rounds now. The session checks it when it starts.") + else: + match = mod.similar_lighting(base, {"chosen": chosen, "ring": ring}) + note = "" if not match else ( + f"The cameras see about the same light as in your round of {session_label(match[0])} " + f"({labels.get(match[1], match[1] or 'no choice')}). A different light helps the " + "dataset more: change the lighting if you can, or go ahead anyway.") + except Exception as e: + note = f"Couldn't check the light: {e}" + self._lightingArrived.emit(note) + self._thread(run) + + def _on_lighting(self, note): + self._lighting_note = note + self.lightingChanged.emit() + + # --- the session + @Property("QVariantMap", notify=statusChanged) + def status(self): + return self._status + + @Property(bool, notify=statusChanged) + def sessionActive(self): + return self._session is not None and self._status.get("state", "starting") in ACTIVE_STATES + + @Property(str, notify=statusChanged) + def sessionId(self): + return self._session_id + + @Slot("QVariantMap", str, result=bool) + def startSession(self, checklist, lighting): + if self.sessionActive: + return False + mod = self._runner() + if not mod: + self.message.emit(self._session_error, True) + return False + if self.needsConsent: + self.message.emit("Agree to the consent text first (Welcome page)", True) + return False + checklist = dict(checklist) + checklist["objects"] = [o for o in checklist.get("objects", []) if o in dict(OBJECTS)] + checklist["own_objects"] = [o.strip() for o in checklist.get("own_objects", []) if str(o).strip()] + try: + self._session = mod.Session(self.store.base, self.store.profile(), checklist, lighting, SCRIPT_PATH, + on_status=lambda s: self._statusArrived.emit(dict(s)), + **self._session_options) + except Exception as e: + self.message.emit(f"Couldn't set up the session: {e}", True) + return False + self._session_id = "" + self._status = {"state": "starting"} + self.statusChanged.emit() + session = self._session + + def run(): + try: + session.start() + except Exception as e: + self._statusArrived.emit({"state": "error", "error": str(e)}) + self._thread(run) + return True + + def _on_status(self, status): + self._status = status + if self._session is not None and not self._session_id: + self._session_id = os.path.basename(str(getattr(self._session, "session_dir", "") or "")) + self.statusChanged.emit() + if status.get("state") in ("done", "stopped", "error"): + self.sessionsChanged.emit() + + def _control(self, name): + if self._session is None: + return + try: + getattr(self._session, name)() + except Exception as e: + self.message.emit(f"{name}: {e}", True) + + @Slot() + def togglePause(self): + if self._status.get("state") == "paused": + self._control("resume") + elif self.sessionActive: + self._control("pause") + + @Slot() + def skipSection(self): + if self.sessionActive: + self._control("skip") + + @Slot() + def stopSession(self): + """Stop: the take in progress is kept, as far as it got. It can block while the + recording is written out, so it runs off this thread.""" + if self.sessionActive: + session = self._session + self._thread(lambda: session.stop()) + + def shutdown(self): + """The window closes: end a running session (blocking, so its files are complete).""" + if self.sessionActive: + try: + self._session.stop() + except Exception: + pass + if self._export_cancel: + self._export_cancel.set() # and wait, so export can remove its half-written copy + self._export_thread.join(15) + + # --- review + @Property("QVariantList", notify=sessionsChanged) + def sessions(self): + out = [] + for s in self.store.sessions(): + s["label"] = session_label(s["id"]) + s["sizeText"] = takes.human_bytes(s["bytes"]) + s["exportText"] = takes.human_bytes(s["export_bytes"]) if s["exported"] else "" + s["lightingText"] = dict(LIGHTING).get(s["lighting"], "") + s["active"] = self.sessionActive and s["id"] == self._session_id + s["statusText"] = {"recording": "" if s["active"] else "interrupted", "error": "ended with an error", + "stopped": "stopped early"}.get(s["status"], "") + out.append(s) + return out + + @Slot() + def refreshSessions(self): + self.sessionsChanged.emit() + self.diskChanged.emit() + + @Slot(str, result="QVariantList") + def takeList(self, session): + try: + rows = self.store.takes(session) + except ValueError: + return [] + for t in rows: + t["durationText"] = f"{int(t['duration_s'] // 60)}:{int(t['duration_s'] % 60):02d}" + t["sizeText"] = takes.human_bytes(t["bytes"]) + t.pop("ranges") + return rows + + @Slot(str, str, result="QVariantMap") + def takeInfo(self, session, take): + """For the viewer: {count, title, cams, ranges: [[first, last] set indexes]}.""" + try: + index = takes.take_index(self.store.take_dir(session, take)) + ranges = self.store.ranges(session, take) + except ValueError: + return {"count": 0, "title": take, "cams": [], "ranges": []} + marks = [] + for a, b in ranges: + inside = [i for i in range(len(index)) if a <= index.time_ns(i) <= b] + marks.append([inside[0], inside[-1]] if inside else [-1, -1]) + return {"count": len(index), "title": self.store.take_meta(session, take).get("title") or take, + "cams": [c["name"] for c in index.cams], "ranges": marks} + + @Slot(str, str, int, result=str) + def setTime(self, session, take, i): + """Set i's time from the take's first set, m:ss.s.""" + try: + index = takes.take_index(self.store.take_dir(session, take)) + t = (index.time_ns(i) - index.time_ns(0)) / 1e9 + except (ValueError, IndexError): + return "" + return f"{int(t // 60)}:{t % 60:04.1f}" + + def _active_guard(self, session): + if self.sessionActive and session == self._session_id: + self.message.emit("This session is still recording: stop it first", True) + return True + return False + + @Slot(str, str, int, int) + def deleteRange(self, session, take, first, last): + """Leave sets first..last (indexes, either order) out of the export.""" + if self._active_guard(session): + return + index = takes.take_index(self.store.take_dir(session, take)) + first, last = sorted((max(0, first), min(len(index) - 1, last))) + if first > last: + return + self.store.delete_range(session, take, index.time_ns(first), index.time_ns(last)) + self.message.emit(f"Sets {first + 1} to {last + 1} won't be exported", False) + self.sessionsChanged.emit() + + @Slot(str, str, int) + def restoreRange(self, session, take, k): + self.store.restore_range(session, take, k) + self.message.emit("Range restored", False) + self.sessionsChanged.emit() + + @Slot(str, str) + def deleteTake(self, session, take): + if self._active_guard(session): + return + try: + self.store.delete_take(session, take) + self.message.emit(f"Deleted {take}", False) + except (OSError, ValueError) as e: + self.message.emit(f"Couldn't delete {take}: {e}", True) + self.refreshSessions() + + @Slot(str) + def deleteSession(self, session): + if self._active_guard(session): + return + try: + self.store.delete_session(session) + self.message.emit(f"Deleted the session of {session_label(session)}", False) + except (OSError, ValueError) as e: + self.message.emit(f"Couldn't delete the session: {e}", True) + self.refreshSessions() + + # --- export + @Slot(result=bool) + def headsetWorn(self): + return headset_worn() + + @Property(bool, constant=True) + def zstdFound(self): + return takes.find_zstd() is not None + + @Property(bool, notify=exportChanged) + def exporting(self): + return self._export_cancel is not None + + @Property(float, notify=exportChanged) + def exportFraction(self): + return self._export_fraction + + @Property(str, notify=exportChanged) + def exportText(self): + return self._export_text + + @Property(str, notify=exportChanged) + def exportSessionId(self): + return self._export_session + + @Slot(str, bool) + def exportSession(self, session, keep_notes): + if self.exporting or self._active_guard(session): + return + cancel = threading.Event() + self._export_cancel = cancel + self._export_session = session + self._export_fraction = 0.0 + self._export_text = "Starting" + self.exportChanged.emit() + + def run(): + try: + path = self.store.export(session, progress=lambda f, text: self._exportProgress.emit(f, text), + cancel=cancel, keep_notes=keep_notes) + self._exportFinished.emit(path, "") + except takes.Cancelled: + self._exportFinished.emit("", "cancelled") + except Exception as e: + self._exportFinished.emit("", str(e) or type(e).__name__) + self._export_thread = self._thread(run) + + @Slot() + def cancelExport(self): + if self._export_cancel: + self._export_cancel.set() + self._export_text = "Cancelling" + self.exportChanged.emit() + + def _on_export_progress(self, fraction, text): + if self._export_cancel and not self._export_cancel.is_set(): + self._export_fraction, self._export_text = fraction, text + self.exportChanged.emit() + + def _on_export_finished(self, path, error): + self._export_cancel = None + if error == "cancelled": + self._export_text = "Cancelled: nothing was kept" + elif error: + self._export_text = "Failed: " + error + self.message.emit("Export failed: " + error, True) + else: + self._export_fraction = 1.0 + self._export_text = f"Exported to {path} ({takes.human_bytes(takes.tree_bytes(path))})" + self.message.emit("Export ready", False) + self.exportChanged.emit() + self.refreshSessions() + + @Slot(str) + def deleteExport(self, session): + try: + self.store.delete_export(session) + self.message.emit("Export deleted; the session stays", False) + except (OSError, ValueError) as e: + self.message.emit(f"Couldn't delete the export: {e}", True) + self.refreshSessions() + + @Property(str, constant=True) + def exportsDir(self): + return self.store.exports_dir + + # --- upload + @Property(str, constant=True) + def dataset(self): + return HF_DATASET + + @Slot(str, result=str) + def uploadCommand(self, session): + try: + path = self.store.export_dir(session) + except ValueError: + return "" + contributor = self.contributor or "CONTRIBUTOR" + return " ".join(["huggingface-cli", "upload", HF_DATASET, shlex.quote(path), + f"contributions/{contributor}/{session}", "--repo-type", "dataset", "--create-pr", + "--commit-message", shlex.quote(f"Hands: session {session} from {contributor}")]) + + @Slot(str, result=str) + def uploadText(self, session): + """UPLOAD.md with this export's path, size and command filled in.""" + try: + path = self.store.export_dir(session) + except ValueError: + return "" + values = {"EXPORT_PATH": path, "EXPORT_SIZE": takes.human_bytes(takes.tree_bytes(path)), + "CONTRIBUTOR": self.contributor or "CONTRIBUTOR", "SESSION": session, "DATASET": HF_DATASET, + "COMMAND": self.uploadCommand(session)} + text = read_text(UPLOAD_PATH) or "UPLOAD.md is missing." + return re.sub(r"@([A-Z_]+)@", lambda m: values.get(m.group(1), m.group(0)), text) + + @Slot(QColor) + def setLinkColor(self, color): + palette = QGuiApplication.palette() + palette.setColor(QPalette.Link, color) + QGuiApplication.setPalette(palette) + + @Slot(str) + def copy(self, text): + QGuiApplication.clipboard().setText(text) + self.message.emit("Copied", False) + + +def main(): + ap = argparse.ArgumentParser(description="Frametop Hand Recorder") + ap.add_argument("--base", default=takes.DEFAULT_BASE, help="where profile.json, sessions/ and exports/ go") + ap.add_argument("--page", default="", help="open on this page: welcome, checklist, session, review, export, upload") + ap.add_argument("--dry-run", action="store_true", + help="test: sessions start no processes and print the panel's commands") + ap.add_argument("--speed", type=float, default=1.0, help="test, with --dry-run: run sessions this much faster") + a, qt_args = ap.parse_known_args() + app = QGuiApplication([sys.argv[0]] + qt_args) + app.setApplicationName("ft-handrec") + app.setApplicationDisplayName("Frametop Hand Recorder") + app.setDesktopFileName("ft-handrec") + if not QIcon.themeName(): + QIcon.setThemeName("breeze") + QQuickStyle.setStyle("org.kde.desktop") + store = takes.Store(a.base) + engine = QQmlApplicationEngine() + engine.addImageProvider("frames", FrameProvider(store)) + backend = Backend(store, {"dry_run": True, "speed": a.speed} if a.dry_run else {}) + app.aboutToQuit.connect(backend.shutdown) + engine.rootContext().setContextProperty("backend", backend) + engine.rootContext().setContextProperty("startPage", a.page) + engine.load(QUrl.fromLocalFile(os.path.join(HERE, "main.qml"))) + if not engine.rootObjects(): + sys.exit(1) + + # SIGTERM and Ctrl+C quit as closing does, so a running session still stops cleanly. Python + # runs signal handlers between bytecodes: the timer gives it some while Qt waits. + def on_signal(*_): + engine.rootObjects()[0].setProperty("quitting", True) + app.quit() + signal.signal(signal.SIGTERM, on_signal) + signal.signal(signal.SIGINT, on_signal) + tick = QTimer(interval=500, timeout=lambda: None) + tick.start() + sys.exit(app.exec()) + + +if __name__ == "__main__": + main() diff --git a/hands/rec/main.qml b/hands/rec/main.qml new file mode 100644 index 0000000..09885b6 --- /dev/null +++ b/hands/rec/main.qml @@ -0,0 +1,1128 @@ +// Frametop Hand Recorder (Kirigami). Backend: ft_handrec.py ("backend"); frame sets come from +// its image provider (image://frames/...). +import QtQuick +import QtQuick.Controls as Controls +import QtQuick.Layouts +import org.kde.kirigami as Kirigami + +Kirigami.ApplicationWindow { + id: root + title: "Frametop Hand Recorder" + width: Kirigami.Units.gridUnit * 50 + height: Kirigami.Units.gridUnit * 36 + + // The session chosen on the Export page, for the Upload page. + property string chosenSession: "" + + globalDrawer: Kirigami.GlobalDrawer { + isMenu: false + modal: false + collapsible: true + collapsed: root.width < Kirigami.Units.gridUnit * 34 + actions: [ + Kirigami.Action { text: "Welcome"; icon.name: "help-about"; onTriggered: root.show(welcomePage) }, + Kirigami.Action { + text: "Before you start"; icon.name: "view-task"; enabled: !backend.needsConsent + onTriggered: root.show(checklistPage) + }, + Kirigami.Action { + text: "Session"; icon.name: "media-record"; enabled: !backend.needsConsent + onTriggered: root.show(sessionPage) + }, + Kirigami.Action { text: "Review"; icon.name: "view-preview"; onTriggered: root.show(reviewPage) }, + Kirigami.Action { + text: "Export"; icon.name: "document-export"; enabled: !backend.needsConsent + onTriggered: root.show(exportPage) + }, + Kirigami.Action { + text: "Upload"; icon.name: "cloud-upload"; enabled: !backend.needsConsent + onTriggered: root.show(uploadPage) + } + ] + } + + // The texts wait for a legal review; until then nobody should contribute. + component DraftBanner: Kirigami.InlineMessage { + visible: backend.textsDraft + position: Kirigami.InlineMessage.Position.Header + type: Kirigami.MessageType.Warning + text: "Contributions aren't open yet. These texts are drafts waiting for a legal review: you can " + + "record and review, but please don't upload anything until this message is gone." + } + + // Read-only Markdown (CONSENT.md, UPLOAD.md), selectable for copying. + component MarkdownText: Controls.TextArea { + id: markdownText + property string markdown + // Links get the application palette's colour when the text is parsed, and Kirigami's theme + // doesn't set that palette: set it, then parse again (the theme resolves after creation). + readonly property color linkColor: Kirigami.Theme.linkColor + function parse() { + backend.setLinkColor(linkColor) + text = markdown + } + onLinkColorChanged: parse() + onMarkdownChanged: parse() + Component.onCompleted: parse() + textFormat: TextEdit.MarkdownText + readOnly: true + selectByMouse: true + wrapMode: Text.Wrap + background: null + padding: 0 + onLinkActivated: link => Qt.openUrlExternally(link) + } + + function plural(n, word) { + return n + " " + word + (n === 1 ? "" : "s") + } + + function show(page) { + pageStack.clear() + pageStack.push(page) + } + + // --page welcome|checklist|session|review|export|upload opens the app on that page; the + // Welcome page comes first until the consent is agreed to. + pageStack.initialPage: backend.needsConsent && startPage !== "review" ? welcomePage + : (({ welcome: welcomePage, checklist: checklistPage, session: sessionPage, review: reviewPage, + export: exportPage, upload: uploadPage })[startPage] || checklistPage) + + Connections { + target: backend + function onMessage(text, isError) { + root.showPassiveNotification(text, isError ? "long" : "short") + } + } + + // Asks before something is deleted for good: ask(title, text, function). + Kirigami.PromptDialog { + id: confirm + property var action: null + standardButtons: Controls.Dialog.Ok | Controls.Dialog.Cancel + onAccepted: if (action) action() + function ask(title, text, fn) { + confirm.title = title + confirm.subtitle = text + confirm.action = fn + confirm.open() + } + } + + // Closing during a session asks first. Quitting closes the window again (Qt 6), so the + // answer is remembered; the backend stops the session as the app quits. + property bool quitting: false + onClosing: close => { + if (backend.sessionActive && !quitting) { + close.accepted = false + confirm.ask("Stop the session?", "A session is recording. Closing stops it; what's recorded so far is kept.", + () => { root.quitting = true; Qt.quit() }) + } + } + + // ---------------------------------------------------------------- Welcome + Component { + id: welcomePage + Kirigami.ScrollablePage { + title: "Welcome" + header: DraftBanner { } + + ColumnLayout { + spacing: Kirigami.Units.largeSpacing + + MarkdownText { + Layout.fillWidth: true + markdown: backend.consentText + } + + Kirigami.Separator { Layout.fillWidth: true } + + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: !backend.needsConsent + type: Kirigami.MessageType.Positive + text: "You agreed to this text (" + backend.consentAccepted + "). Your contributor id is " + + backend.contributor + ": keep it if you might want to withdraw later." + } + Controls.Button { + visible: !backend.needsConsent + text: "Copy contributor id" + icon.name: "edit-copy" + onClicked: backend.copy(backend.contributor) + } + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: backend.needsConsent && backend.contributor !== "" + type: Kirigami.MessageType.Information + text: "This text changed since you last agreed to it. Please read it again; your contributor id stays the same." + } + + Controls.CheckBox { + id: adult + visible: backend.needsConsent + text: "I'm 18 or older" + } + Controls.CheckBox { + id: agree + visible: backend.needsConsent + text: "I agree to the text above" + } + RowLayout { + visible: backend.needsConsent + Controls.Label { text: "Handedness (optional):" } + Controls.ComboBox { + id: handed + model: backend.handednessChoices + textRole: "text" + valueRole: "value" + Component.onCompleted: currentIndex = Math.max(0, indexOfValue(backend.handedness)) + } + } + Controls.Button { + visible: backend.needsConsent + enabled: adult.checked && agree.checked + text: "Agree and continue" + icon.name: "go-next" + onClicked: { + backend.acceptConsent(adult.checked, agree.checked, handed.currentValue) + if (!backend.needsConsent) + root.show(checklistPage) + } + } + Controls.Button { + visible: !backend.needsConsent + text: "Continue" + icon.name: "go-next" + onClicked: root.show(checklistPage) + } + } + } + } + + // ---------------------------------------------------------------- Before you start + Component { + id: checklistPage + Kirigami.ScrollablePage { + id: checklist + title: "Before you start" + readonly property bool privacyOk: privacy1.checked && privacy2.checked && privacy3.checked + readonly property bool ready: privacyOk && lighting.currentIndex >= 0 && backend.diskOk + && backend.runnerError === "" && !backend.sessionActive + + function answers() { + const objects = [] + for (let i = 0; i < objectBoxes.count; ++i) { + const box = objectBoxes.itemAt(i) + if (box.checked) + objects.push(box.value) + } + return { + objects: objects, + own_objects: ownObjects.text.split(",").map(s => s.trim()).filter(s => s !== ""), + controllers: straps.checked ? "straps" : "none", + sleeves: sleeves.currentValue, + rings: rings.checked, + watch: watch.checked, + notes: notes.text, + privacy: privacyOk + } + } + + header: ColumnLayout { + spacing: 0 + DraftBanner { Layout.fillWidth: true } + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: backend.runnerError !== "" + position: Kirigami.InlineMessage.Position.Header + type: Kirigami.MessageType.Error + text: backend.runnerError + } + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: backend.sessionActive + position: Kirigami.InlineMessage.Position.Header + type: Kirigami.MessageType.Information + text: "A session is running." + actions: [ Kirigami.Action { text: "Go to it"; onTriggered: root.show(sessionPage) } ] + } + } + + Kirigami.FormLayout { + Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Within reach" } + Controls.Label { + Kirigami.FormData.label: "" + Layout.maximumWidth: Kirigami.Units.gridUnit * 26 + wrapMode: Text.Wrap + text: "Put these on the desk or table in front of you. Untick what you don't have; " + + "the session skips those." + } + Repeater { + id: objectBoxes + model: backend.objects + Controls.CheckBox { + required property var modelData + required property int index + readonly property string value: modelData.value + Kirigami.FormData.label: index === 0 ? "Objects:" : "" + text: modelData.text + checked: true + } + } + Controls.TextField { + id: ownObjects + Kirigami.FormData.label: "Your own:" + Layout.preferredWidth: Kirigami.Units.gridUnit * 20 + placeholderText: "other things you use, separated by commas" + } + + Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Controllers" } + ColumnLayout { + Kirigami.FormData.label: "Ergonomic Kit straps:" + Controls.RadioButton { id: straps; text: "Yes, I have the controllers with the straps" } + Controls.RadioButton { text: "No"; checked: true } + } + Controls.Label { + Layout.maximumWidth: Kirigami.Units.gridUnit * 26 + wrapMode: Text.Wrap + opacity: 0.7 + text: "With the straps, the controllers stay on your hands while your fingers move freely. " + + "The controllers' tracking then says exactly where your hands are, which teaches the " + + "model how far away a hand is. Without them, the two controller sections are skipped." + } + + Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Light" } + Controls.ComboBox { + id: lighting + Kirigami.FormData.label: "Lighting this round:" + model: backend.lightingChoices + textRole: "text" + valueRole: "value" + currentIndex: -1 + displayText: currentIndex < 0 ? "Choose…" : currentText + onActivated: backend.checkLighting(currentValue) + } + Controls.Label { + Layout.maximumWidth: Kirigami.Units.gridUnit * 26 + wrapMode: Text.Wrap + opacity: 0.7 + text: "Each round in a different light helps the most: dim, a normal room, daylight." + } + Kirigami.InlineMessage { + Layout.maximumWidth: Kirigami.Units.gridUnit * 26 + Layout.fillWidth: true + visible: backend.lightingNote !== "" && lighting.currentIndex >= 0 + Layout.preferredHeight: visible ? implicitHeight : 0 + type: Kirigami.MessageType.Warning + text: backend.lightingNote + } + Controls.Button { + visible: backend.lightingNote !== "" && lighting.currentIndex >= 0 + text: "Check the light again" + icon.name: "view-refresh" + onClicked: backend.checkLighting(lighting.currentValue) + } + + Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "You" } + Controls.ComboBox { + id: sleeves + Kirigami.FormData.label: "Sleeves:" + model: backend.sleeveChoices + textRole: "text" + valueRole: "value" + } + Controls.CheckBox { id: rings; Kirigami.FormData.label: "Wearing:"; text: "Rings" } + Controls.CheckBox { id: watch; text: "A watch or bracelet" } + Controls.Label { + Layout.maximumWidth: Kirigami.Units.gridUnit * 26 + wrapMode: Text.Wrap + opacity: 0.7 + text: "Wear what you normally do: the dataset needs hands with and without them. This only notes it." + } + Controls.TextField { + id: notes + Kirigami.FormData.label: "Notes (optional):" + Layout.preferredWidth: Kirigami.Units.gridUnit * 20 + placeholderText: "anything that helps, e.g. a bandage on a finger" + } + + Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Privacy" } + Controls.CheckBox { + id: privacy1 + Kirigami.FormData.label: "I've checked:" + text: "I'm facing away from other people, mirrors and windows" + } + Controls.CheckBox { + id: privacy2 + text: "No papers, photos, screens or other private things are in view" + } + Controls.CheckBox { + id: privacy3 + text: "Nobody else's face or hands will be in view" + } + + Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "Space" } + Controls.Label { + Kirigami.FormData.label: "Free space:" + text: backend.freeText + " (a round takes about 10 GB)" + } + Kirigami.InlineMessage { + Layout.maximumWidth: Kirigami.Units.gridUnit * 26 + Layout.fillWidth: true + visible: !backend.diskOk + type: Kirigami.MessageType.Error + text: "Not enough free space for a round. Export and delete earlier sessions (Review), " + + "or free up space, first." + } + + Kirigami.Separator { Kirigami.FormData.isSection: true; Kirigami.FormData.label: "What will happen" } + Controls.Label { + Layout.maximumWidth: Kirigami.Units.gridUnit * 26 + wrapMode: Text.Wrap + text: "Press Start, then put the headset on. A panel in the headset says what to do, one " + + "section at a time: hand poses, gestures, typing and the mouse, your objects, touching a " + + "dot, and moves with the controllers on and off. Each section is recorded as one take.\n\n" + + "Space in this window pauses and Esc stops. You can also skip a section.\n\n" + + "Nothing leaves the headset. Afterwards you watch the takes in Review, delete anything " + + "you don't want to share, and only then export." + } + Controls.Button { + text: "Start" + icon.name: "media-record" + enabled: checklist.ready + onClicked: { + if (backend.startSession(checklist.answers(), lighting.currentValue)) + root.show(sessionPage) + } + } + Controls.Label { + visible: !checklist.ready && !backend.sessionActive + opacity: 0.7 + text: !checklist.privacyOk ? "Tick the three privacy checks to start." + : lighting.currentIndex < 0 ? "Choose the lighting to start." : "" + } + } + } + } + + // ---------------------------------------------------------------- Session + Component { + id: sessionPage + Kirigami.Page { + id: sessionView + title: "Session" + readonly property var st: backend.status + readonly property string state: st.state || "" + readonly property var stateText: ({ + starting: "Starting…", intro: "Get ready", running: "Recording", paused: "Paused", + between: "Between sections", done: "Done", stopped: "Stopped", error: "Error" + }) + + // Space and Esc while the window has focus (the panel in the headset says so too). + Shortcut { + sequence: "Space" + enabled: backend.sessionActive + onActivated: backend.togglePause() + } + Shortcut { + sequence: "Esc" + enabled: backend.sessionActive + onActivated: backend.stopSession() + } + + // A hand chip like the panel's: seen (green), lost (orange) or hidden. + component HandChip: Controls.Label { + property var seen + visible: seen === true || seen === false || seen === "seen" || seen === "lost" + readonly property bool ok: seen === true || seen === "seen" + padding: Kirigami.Units.smallSpacing + leftPadding: Kirigami.Units.largeSpacing + rightPadding: Kirigami.Units.largeSpacing + color: "white" + background: Rectangle { + radius: height / 2 + color: parent.ok ? Kirigami.Theme.positiveTextColor : Kirigami.Theme.neutralTextColor + } + } + + header: Kirigami.InlineMessage { + visible: backend.runnerError !== "" || sessionView.state === "error" + position: Kirigami.InlineMessage.Position.Header + type: Kirigami.MessageType.Error + text: backend.runnerError || ("The session stopped with an error: " + (sessionView.st.error || "unknown")) + } + + Kirigami.PlaceholderMessage { + anchors.centerIn: parent + width: parent.width - Kirigami.Units.gridUnit * 4 + visible: sessionView.state === "" + icon.name: "media-record" + text: "No session yet" + explanation: "Go through Before you start, then press Start there." + helpfulAction: Kirigami.Action { + text: "Before you start"; icon.name: "view-task" + onTriggered: root.show(checklistPage) + } + } + + ColumnLayout { + anchors.fill: parent + anchors.margins: Kirigami.Units.gridUnit + visible: sessionView.state !== "" + spacing: Kirigami.Units.largeSpacing + + Kirigami.Heading { + level: 1 + text: sessionView.stateText[sessionView.state] || sessionView.state + } + Controls.Label { + visible: (sessionView.st.section_index || 0) > 0 + text: "Section " + sessionView.st.section_index + " of " + sessionView.st.section_count + + (sessionView.st.title ? ": " + sessionView.st.title : "") + font.bold: true + } + Controls.Label { + Layout.fillWidth: true + visible: !!sessionView.st.prompt + wrapMode: Text.Wrap + font.pointSize: Kirigami.Theme.defaultFont.pointSize * 1.4 + text: sessionView.st.prompt || "" + } + Controls.Label { + visible: sessionView.st.seconds_left !== undefined && sessionView.st.seconds_left !== null + && sessionView.st.seconds_left >= 0 + text: Math.ceil(sessionView.st.seconds_left || 0) + " s left" + opacity: 0.7 + } + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: !!sessionView.st.note + // (A hidden InlineMessage keeps its height in a layout.) + Layout.preferredHeight: visible ? implicitHeight : 0 + type: Kirigami.MessageType.Warning + text: sessionView.st.note || "" + } + RowLayout { + spacing: Kirigami.Units.largeSpacing + HandChip { text: "Left hand"; seen: sessionView.st.hands ? sessionView.st.hands.left : undefined } + HandChip { text: "Right hand"; seen: sessionView.st.hands ? sessionView.st.hands.right : undefined } + } + Controls.Label { + visible: !!sessionView.st.take + text: "Recording to take " + (sessionView.st.take || "") + opacity: 0.7 + font: Kirigami.Theme.smallFont + } + + Item { Layout.fillHeight: true } + + Controls.Label { + Layout.fillWidth: true + wrapMode: Text.Wrap + opacity: 0.7 + text: "The instructions appear in the headset. While this window has focus, Space pauses " + + "and resumes, and Esc stops. Stopping keeps what's recorded so far." + } + RowLayout { + // No keyboard focus on the buttons, so Space always reaches the shortcut. + Controls.Button { + visible: !backend.sessionActive + focusPolicy: Qt.NoFocus + text: "New session…" + icon.name: "media-record" + onClicked: root.show(checklistPage) + } + Controls.Button { + visible: backend.sessionActive + focusPolicy: Qt.NoFocus + text: sessionView.state === "paused" ? "Resume" : "Pause" + icon.name: sessionView.state === "paused" ? "media-playback-start" : "media-playback-pause" + onClicked: backend.togglePause() + } + Controls.Button { + visible: backend.sessionActive + focusPolicy: Qt.NoFocus + text: "Skip section" + icon.name: "media-skip-forward" + onClicked: backend.skipSection() + } + Controls.Button { + visible: backend.sessionActive + focusPolicy: Qt.NoFocus + text: "Stop" + icon.name: "media-playback-stop" + onClicked: backend.stopSession() + } + Controls.Button { + visible: !backend.sessionActive && backend.sessionId !== "" + focusPolicy: Qt.NoFocus + text: "Review this session" + icon.name: "view-preview" + onClicked: { + root.show(reviewPage) + pageStack.push(takesPage, { session: backend.sessionId }) + } + } + } + } + } + } + + // ---------------------------------------------------------------- Review + Component { + id: reviewPage + Kirigami.ScrollablePage { + title: "Review" + actions: [ + Kirigami.Action { text: "Refresh"; icon.name: "view-refresh"; onTriggered: backend.refreshSessions() } + ] + + ListView { + model: backend.sessions + spacing: Kirigami.Units.smallSpacing + + Kirigami.PlaceholderMessage { + anchors.centerIn: parent + width: parent.width - Kirigami.Units.gridUnit * 4 + visible: parent.count === 0 + text: "No sessions yet" + explanation: "Recorded sessions appear here, to watch and trim before you export them." + } + + delegate: Controls.ItemDelegate { + id: sessionRow + required property var modelData + width: ListView.view.width + onClicked: pageStack.push(takesPage, { session: modelData.id }) + + contentItem: RowLayout { + spacing: Kirigami.Units.largeSpacing + Kirigami.Icon { + source: sessionRow.modelData.active ? "media-record" : "folder-videos" + implicitWidth: Kirigami.Units.iconSizes.medium + implicitHeight: implicitWidth + } + ColumnLayout { + Layout.fillWidth: true + spacing: 0 + Controls.Label { + text: sessionRow.modelData.label + (sessionRow.modelData.active ? " (recording)" : "") + + (sessionRow.modelData.dry_run ? " (dry run)" : "") + font.bold: true + } + Controls.Label { + Layout.fillWidth: true + elide: Text.ElideRight + opacity: 0.7 + font: Kirigami.Theme.smallFont + text: root.plural(sessionRow.modelData.takes, "take") + " · " + sessionRow.modelData.sizeText + + (sessionRow.modelData.statusText ? " · " + sessionRow.modelData.statusText : "") + + (sessionRow.modelData.lightingText ? " · " + sessionRow.modelData.lightingText : "") + + (sessionRow.modelData.export_stale ? " · changed since its export" + : sessionRow.modelData.exported ? " · exported (" + sessionRow.modelData.exportText + ")" : "") + } + } + Controls.ToolButton { + icon.name: "edit-delete" + text: "Delete session" + display: Controls.AbstractButton.IconOnly + enabled: !sessionRow.modelData.active + Controls.ToolTip.text: text + Controls.ToolTip.visible: hovered + onClicked: { + const sid = sessionRow.modelData.id + confirm.ask("Delete this session?", + "The session of " + sessionRow.modelData.label + " (" + root.plural(sessionRow.modelData.takes, "take") + + ", " + sessionRow.modelData.sizeText + ") and its export are deleted from the " + + "headset for good.", + () => { pageStack.pop(pageStack.items[0]); backend.deleteSession(sid) }) + } + } + } + } + } + } + } + + Component { + id: takesPage + Kirigami.ScrollablePage { + id: takesView + property string session: "" + property var rows: backend.takeList(session) + title: "Takes" + + Connections { + target: backend + function onSessionsChanged() { takesView.rows = backend.takeList(takesView.session) } + } + + ListView { + model: takesView.rows + spacing: Kirigami.Units.smallSpacing + + Kirigami.PlaceholderMessage { + anchors.centerIn: parent + width: parent.width - Kirigami.Units.gridUnit * 4 + visible: parent.count === 0 + text: "No takes in this session" + } + + delegate: Controls.ItemDelegate { + id: takeRow + required property var modelData + width: ListView.view.width + onClicked: pageStack.push(viewerPage, { session: takesView.session, take: modelData.id }) + + contentItem: RowLayout { + spacing: Kirigami.Units.largeSpacing + Image { + Layout.preferredWidth: Kirigami.Units.gridUnit * 5 + Layout.preferredHeight: Kirigami.Units.gridUnit * 4.5 + fillMode: Image.PreserveAspectFit + asynchronous: true + source: takeRow.modelData.sets > 0 + ? "image://frames/thumb/" + takesView.session + "/" + takeRow.modelData.id : "" + } + ColumnLayout { + Layout.fillWidth: true + spacing: 0 + Controls.Label { text: takeRow.modelData.title; font.bold: true } + Controls.Label { + Layout.fillWidth: true + elide: Text.ElideRight + opacity: 0.7 + font: Kirigami.Theme.smallFont + text: takeRow.modelData.status + " · " + takeRow.modelData.durationText + " · " + + takeRow.modelData.sets + " sets" + + (takeRow.modelData.deleted_sets ? " (" + takeRow.modelData.deleted_sets + " deleted)" : "") + + " · " + takeRow.modelData.sizeText + } + } + Controls.ToolButton { + icon.name: "edit-delete" + text: "Delete take" + display: Controls.AbstractButton.IconOnly + Controls.ToolTip.text: text + Controls.ToolTip.visible: hovered + onClicked: { + const take = takeRow.modelData.id + confirm.ask("Delete this take?", + "\"" + takeRow.modelData.title + "\" (" + takeRow.modelData.durationText + + ") is deleted from the headset for good.", + () => { pageStack.pop(takesView); backend.deleteTake(takesView.session, take) }) + } + } + } + } + } + } + } + + Component { + id: viewerPage + Kirigami.Page { + id: viewer + property string session: "" + property string take: "" + property var info: backend.takeInfo(session, take) + property int index: 0 + property int markStart: -1 + property int markEnd: -1 + title: info.title + readonly property bool deletedHere: isDeleted(index) + + function step(n) { + index = Math.max(0, Math.min(info.count - 1, index + n)) + } + function isDeleted(i) { + for (const r of info.ranges) + if (i >= r[0] && i <= r[1] && r[0] >= 0) + return true + return false + } + function deleteMarked() { + backend.deleteRange(session, take, markStart, markEnd) + markStart = markEnd = -1 + info = backend.takeInfo(session, take) + } + + Connections { + target: backend + function onSessionsChanged() { viewer.info = backend.takeInfo(viewer.session, viewer.take) } + } + + // Keyboard stepping: arrows one set, Page Up/Down ten, Home/End, [ and ] mark a range, + // Delete deletes it. + Item { + id: keys + focus: true + Component.onCompleted: forceActiveFocus() + Keys.onPressed: event => { + const k = event.key + if (k === Qt.Key_Left) viewer.step(-1) + else if (k === Qt.Key_Right) viewer.step(1) + else if (k === Qt.Key_PageUp) viewer.step(-10) + else if (k === Qt.Key_PageDown) viewer.step(10) + else if (k === Qt.Key_Home) viewer.index = 0 + else if (k === Qt.Key_End) viewer.index = Math.max(0, viewer.info.count - 1) + else if (k === Qt.Key_BracketLeft) viewer.markStart = viewer.index + else if (k === Qt.Key_BracketRight) viewer.markEnd = viewer.index + else if (k === Qt.Key_Delete && viewer.markStart >= 0 && viewer.markEnd >= 0) viewer.deleteMarked() + else return + event.accepted = true + } + } + + Kirigami.PlaceholderMessage { + anchors.centerIn: parent + visible: viewer.info.count === 0 + text: "This take has no frame sets" + } + + ColumnLayout { + anchors.fill: parent + visible: viewer.info.count > 0 + spacing: Kirigami.Units.smallSpacing + + Image { + id: frame + Layout.fillWidth: true + Layout.fillHeight: true + fillMode: Image.PreserveAspectFit + asynchronous: true + cache: false + retainWhileLoading: true + source: viewer.info.count > 0 + ? "image://frames/set/" + viewer.session + "/" + viewer.take + "/" + viewer.index : "" + MouseArea { anchors.fill: parent; onClicked: keys.forceActiveFocus() } + + Rectangle { + visible: viewer.deletedHere + x: (frame.width - frame.paintedWidth) / 2 + y: (frame.height - frame.paintedHeight) / 2 + width: frame.paintedWidth + height: frame.paintedHeight + color: Qt.rgba(0.8, 0, 0, 0.35) + Controls.Label { + anchors.centerIn: parent + text: "Deleted: not exported" + color: "white" + font.bold: true + } + } + } + + // The slider, with the deleted ranges (red) and the marked range (blue) under it. + Item { + Layout.fillWidth: true + implicitHeight: slider.implicitHeight + Repeater { + model: viewer.info.ranges + Rectangle { + required property var modelData + visible: modelData[0] >= 0 + readonly property real unit: (slider.availableWidth) / Math.max(1, viewer.info.count - 1) + x: slider.leftPadding + modelData[0] * unit - 2 + width: Math.max(4, (modelData[1] - modelData[0]) * unit + 4) + y: slider.topPadding + slider.availableHeight / 2 + 4 + height: 4 + color: Kirigami.Theme.negativeTextColor + } + } + Rectangle { + visible: viewer.markStart >= 0 + readonly property int last: viewer.markEnd >= 0 ? viewer.markEnd : viewer.index + readonly property real unit: (slider.availableWidth) / Math.max(1, viewer.info.count - 1) + x: slider.leftPadding + Math.min(viewer.markStart, last) * unit - 2 + width: Math.abs(last - viewer.markStart) * unit + 4 + y: slider.topPadding + slider.availableHeight / 2 - 8 + height: 4 + color: Kirigami.Theme.highlightColor + } + Controls.Slider { + id: slider + anchors.left: parent.left + anchors.right: parent.right + focusPolicy: Qt.NoFocus + from: 0 + to: Math.max(0, viewer.info.count - 1) + stepSize: 1 + snapMode: Controls.Slider.SnapAlways + value: viewer.index + onMoved: viewer.index = Math.round(value) + } + } + + RowLayout { + Layout.fillWidth: true + Controls.Label { + text: "Set " + (viewer.index + 1) + " of " + viewer.info.count + " · " + + backend.setTime(viewer.session, viewer.take, viewer.index) + } + Item { Layout.fillWidth: true } + Controls.Button { + focusPolicy: Qt.NoFocus + text: viewer.markStart >= 0 ? "Start: " + (viewer.markStart + 1) : "Mark start" + icon.name: "go-first" + onClicked: viewer.markStart = viewer.index + } + Controls.Button { + focusPolicy: Qt.NoFocus + text: viewer.markEnd >= 0 ? "End: " + (viewer.markEnd + 1) : "Mark end" + icon.name: "go-last" + onClicked: viewer.markEnd = viewer.index + } + Controls.Button { + focusPolicy: Qt.NoFocus + enabled: viewer.markStart >= 0 && viewer.markEnd >= 0 + text: "Delete range" + icon.name: "edit-cut" + onClicked: viewer.deleteMarked() + } + Controls.Button { + focusPolicy: Qt.NoFocus + visible: viewer.markStart >= 0 || viewer.markEnd >= 0 + text: "Clear marks" + icon.name: "edit-clear" + onClicked: viewer.markStart = viewer.markEnd = -1 + } + } + + Flow { + Layout.fillWidth: true + spacing: Kirigami.Units.smallSpacing + visible: viewer.info.ranges.length > 0 + Controls.Label { text: "Deleted:"; topPadding: Kirigami.Units.smallSpacing } + Repeater { + model: viewer.info.ranges + Controls.Button { + required property var modelData + required property int index + focusPolicy: Qt.NoFocus + flat: true + text: (modelData[0] < 0 ? "a range with no sets" + : "sets " + (modelData[0] + 1) + " to " + (modelData[1] + 1)) + ": restore" + icon.name: "edit-undo" + onClicked: backend.restoreRange(viewer.session, viewer.take, index) + } + } + } + + Controls.Label { + Layout.fillWidth: true + wrapMode: Text.Wrap + opacity: 0.7 + font: Kirigami.Theme.smallFont + text: "← → step one set, Page Up/Down ten, Home/End. [ and ] mark a range's start and end, " + + "Delete deletes it. Deleted sets stay on the headset until export, which leaves them out." + } + } + } + } + + // ---------------------------------------------------------------- Export + Component { + id: exportPage + Kirigami.ScrollablePage { + id: exportView + title: "Export" + property bool worn: backend.headsetWorn() + readonly property var chosen: { + const list = backend.sessions + for (const s of list) + if (s.id === sessionBox.currentValue) + return s + return null + } + + Timer { + interval: 5000 + running: true + repeat: true + onTriggered: exportView.worn = backend.headsetWorn() + } + + header: ColumnLayout { + spacing: 0 + DraftBanner { Layout.fillWidth: true } + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: exportView.worn + position: Kirigami.InlineMessage.Position.Header + type: Kirigami.MessageType.Warning + text: "Someone seems to be wearing the headset. Exporting keeps the processor busy for a " + + "while, which can make VR stutter: export when the headset is off, or go ahead anyway." + } + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: !backend.zstdFound + position: Kirigami.InlineMessage.Position.Header + type: Kirigami.MessageType.Error + text: "zstd isn't installed in the dev container, so nothing can be exported. Run setup/dev-container.sh." + } + } + + Kirigami.FormLayout { + Controls.ComboBox { + id: sessionBox + Kirigami.FormData.label: "Session:" + model: backend.sessions + textRole: "label" + valueRole: "id" + enabled: !backend.exporting + Component.onCompleted: { + const i = indexOfValue(root.chosenSession) + currentIndex = i >= 0 ? i : (count > 0 ? 0 : -1) + } + onActivated: root.chosenSession = currentValue + } + Controls.Label { + Kirigami.FormData.label: "Recorded:" + visible: exportView.chosen !== null + text: exportView.chosen ? root.plural(exportView.chosen.takes, "take") + ", " + exportView.chosen.sizeText : "" + } + Controls.Label { + Kirigami.FormData.label: "Export:" + visible: exportView.chosen !== null + text: !exportView.chosen ? "" + : exportView.chosen.export_stale ? "changed since it was exported: export it again" + : exportView.chosen.exported ? "ready, " + exportView.chosen.exportText : "not exported yet" + } + RowLayout { + Controls.Button { + text: exportView.chosen && exportView.chosen.exported ? "Export again" : "Export" + icon.name: "document-export" + enabled: exportView.chosen !== null && !backend.exporting && backend.zstdFound + && !exportView.chosen.active + onClicked: backend.exportSession(exportView.chosen.id, false) + } + Controls.Button { + visible: backend.exporting + text: "Cancel" + icon.name: "dialog-cancel" + onClicked: backend.cancelExport() + } + } + // Drawn here: the desktop style's ProgressBar paints nothing without a QApplication. + Rectangle { + Kirigami.FormData.label: "Progress:" + Layout.preferredWidth: Kirigami.Units.gridUnit * 20 + implicitHeight: Kirigami.Units.smallSpacing * 2 + visible: backend.exporting + radius: height / 2 + color: Qt.rgba(Kirigami.Theme.textColor.r, Kirigami.Theme.textColor.g, Kirigami.Theme.textColor.b, 0.15) + Rectangle { + width: parent.width * Math.max(0, Math.min(1, backend.exportFraction)) + height: parent.height + radius: parent.radius + color: Kirigami.Theme.highlightColor + } + } + Controls.Label { + Layout.maximumWidth: Kirigami.Units.gridUnit * 26 + visible: backend.exportText !== "" && backend.exportSessionId === sessionBox.currentValue + wrapMode: Text.WrapAnywhere + text: backend.exportText + } + RowLayout { + visible: exportView.chosen !== null && exportView.chosen.exported && !backend.exporting + Controls.Button { + text: "Upload instructions" + icon.name: "go-next" + onClicked: root.show(uploadPage) + } + Controls.Button { + text: "Delete export" + icon.name: "edit-delete" + onClicked: { + const sid = exportView.chosen.id + confirm.ask("Delete this export?", "The export is deleted; the session's recordings stay.", + () => backend.deleteExport(sid)) + } + } + } + } + + footer: Controls.Label { + padding: Kirigami.Units.largeSpacing + wrapMode: Text.Wrap + opacity: 0.7 + text: "Export leaves out the ranges you deleted, compresses the rest and adds a manifest and " + + "checksums, in " + backend.exportsDir + ". It runs at the lowest priority " + + "and takes a few minutes per round. Nothing is uploaded: the Upload page explains how." + } + } + } + + // ---------------------------------------------------------------- Upload + Component { + id: uploadPage + Kirigami.ScrollablePage { + id: uploadView + title: "Upload" + readonly property var exported: backend.sessions.filter(s => s.exported) + readonly property string session: exportBox.currentIndex >= 0 ? exportBox.currentValue || "" : "" + + header: DraftBanner { } + + ColumnLayout { + spacing: Kirigami.Units.largeSpacing + + Kirigami.PlaceholderMessage { + Layout.fillWidth: true + visible: uploadView.exported.length === 0 + icon.name: "document-export" + text: "Nothing exported yet" + explanation: "Review a session, then export it. Its upload instructions appear here." + helpfulAction: Kirigami.Action { + text: "Export"; icon.name: "document-export" + onTriggered: root.show(exportPage) + } + } + + RowLayout { + visible: uploadView.exported.length > 0 + Controls.Label { text: "Export:" } + Controls.ComboBox { + id: exportBox + model: uploadView.exported + textRole: "label" + valueRole: "id" + Component.onCompleted: currentIndex = Math.max(0, indexOfValue(root.chosenSession)) + } + } + Kirigami.InlineMessage { + Layout.fillWidth: true + visible: uploadView.exported.length > 0 && exportBox.currentIndex >= 0 + && uploadView.exported[exportBox.currentIndex].export_stale + type: Kirigami.MessageType.Warning + text: "This session changed since it was exported. Export it again so the upload has your latest deletions." + } + RowLayout { + Layout.fillWidth: true + visible: uploadView.session !== "" + Controls.TextArea { + id: command + Layout.fillWidth: true + readOnly: true + selectByMouse: true + wrapMode: Text.WrapAnywhere + font.family: "monospace" + text: backend.uploadCommand(uploadView.session) + } + Controls.Button { + text: "Copy command" + icon.name: "edit-copy" + onClicked: backend.copy(command.text) + } + } + MarkdownText { + Layout.fillWidth: true + visible: uploadView.session !== "" + markdown: backend.uploadText(uploadView.session) + } + } + } + } +} diff --git a/hands/rec/takes.py b/hands/rec/takes.py new file mode 100755 index 0000000..d5110ea --- /dev/null +++ b/hands/rec/takes.py @@ -0,0 +1,539 @@ +#!/usr/bin/env python3 +"""The hand recorder's sessions and takes on disk: listing, review, deleting, export. + +Standard library only (no Qt, no NumPy), so the window and the command line share it. +The layout is hands/rec/DESIGN.md's "Files": + BASE/profile.json + BASE/sessions//session.json, calibration.json + BASE/sessions//takes/-
/sets.bin (sets-2.bin, ... after pauses), + prompts.jsonl, poses.jsonl, take.json + BASE/exports// +A take's recording is one or more FHSET01 files (hands/track/record.h): per set a header +(magic, ncams, bytes), one fh_set_cam_t per camera, then each camera's 8-bit pixels. +Deleted ranges live in take.json ("deleted": [[from_ns, to_ns], ...], CLOCK_MONOTONIC, the +clock of dqbuf_ns); the files keep every set until export leaves them out. + +usage: takes.py [--base DIR] list | takes SESSION | export SESSION +""" +import argparse +import datetime +import hashlib +import json +import os +import re +import shutil +import struct +import subprocess +import threading + +DEFAULT_BASE = os.path.expanduser("~/.local/share/frametop/hands/contrib") +REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) +MAGIC = b"FHSET01" +HDR = struct.Struct("<8sII") # fh_set_hdr_t: magic, ncams, bytes (the whole record) +CAM = struct.Struct("<16sIIQQ") # fh_set_cam_t: name, width, height, capture_ns, dqbuf_ns +MAX_CAMS = 16 # SetReader's limit +PART_RE = re.compile(r"^sets(?:-(\d+))?\.bin$") +SESSION_RE = re.compile(r"^\d{8}-\d{6}$") +EXPORT_SCHEMA = 1 +# zstd as DESIGN.md has it: level 10, two threads, at the lowest CPU priority (CPU work while +# someone is in VR makes the headset stutter). +ZSTD_ARGS = ["-10", "-T2", "-q", "-c"] +ZSTD_PATHS = ("/usr/bin/zstd", "/usr/local/bin/zstd", "/run/host/usr/bin/zstd") +CHUNK = 1 << 20 + + +class Cancelled(Exception): + pass + + +def read_json(path, default=None): + try: + with open(path) as f: + return json.load(f) + except (OSError, ValueError): + return {} if default is None else default + + +def write_json(path, obj): + """Write through a temporary file, so a crash never leaves half a file.""" + tmp = path + ".tmp" + with open(tmp, "w") as f: + json.dump(obj, f, indent=2) + f.write("\n") + os.replace(tmp, path) + + +def find_zstd(): + """The zstd binary, or None. The dev container gets it from setup/dev-container.sh.""" + found = shutil.which("zstd") + if found: + return found + return next((p for p in ZSTD_PATHS if os.access(p, os.X_OK)), None) + + +def tool_version(): + """ft-handrec plus the checkout's git describe, for session.json and the manifest.""" + try: + out = subprocess.run(["git", "-C", REPO, "describe", "--always", "--dirty", "--tags"], + capture_output=True, text=True, timeout=5) + described = out.stdout.strip() + except (OSError, subprocess.TimeoutExpired): + described = "" + return "ft-handrec " + (described or "unknown") + + +def free_bytes(path): + """Free space on the file system that holds path (or its nearest existing parent).""" + while path and not os.path.exists(path): + path = os.path.dirname(path) + try: + return shutil.disk_usage(path or "/").free + except OSError: + return 0 + + +def tree_stat(path): + """(bytes, newest mtime) of the files under path.""" + total, newest = 0, 0.0 + for root, _, files in os.walk(path): + for name in files: + try: + st = os.lstat(os.path.join(root, name)) + except OSError: + continue + total += st.st_size + newest = max(newest, st.st_mtime) + return total, newest + + +def tree_bytes(path): + return tree_stat(path)[0] + + +def in_ranges(t, ranges): + return any(a <= t <= b for a, b in ranges) + + +def merge_ranges(ranges): + """Sorted, with overlapping or touching ranges joined.""" + out = [] + for a, b in sorted((min(a, b), max(a, b)) for a, b in ranges): + if out and a <= out[-1][1] + 1: + out[-1][1] = max(out[-1][1], b) + else: + out.append([a, b]) + return out + + +# ---------------------------------------------------------------- one take's recording + +class TakeIndex: + """Where each set of a take's recording is, without reading pixels. Sets are numbered + across the parts (sets.bin, sets-2.bin, ...) in order.""" + + def __init__(self, take_dir): + self.dir = take_dir + self.parts = self._parts(take_dir) + self.sets = [] # (part, offset, bytes, time_ns): time_ns is the cameras' earliest dqbuf_ns + self.part_starts = [] # index of each part's first set + self.cams = [] # the first set's [{name, width, height}] + for p, path in enumerate(self.parts): + self.part_starts.append(len(self.sets)) + self._index(p, path) + + @staticmethod + def _parts(take_dir): + found = [] + try: + names = os.listdir(take_dir) + except OSError: + return [] + for name in names: + m = PART_RE.match(name) + if m: + found.append((int(m.group(1) or 1), os.path.join(take_dir, name))) + return [path for _, path in sorted(found)] + + def _index(self, p, path): + try: + size = os.path.getsize(path) + f = open(path, "rb") + except OSError: + return + with f: + off = 0 + while off + HDR.size <= size: + f.seek(off) + head = f.read(HDR.size) + if len(head) < HDR.size: + break + magic, ncams, nbytes = HDR.unpack(head) + # A truncated last set (the recorder was killed mid-write) ends the part. + if magic[:7] != MAGIC or not 0 < ncams <= MAX_CAMS or nbytes < HDR.size or off + nbytes > size: + break + cams = f.read(CAM.size * ncams) + if len(cams) < CAM.size * ncams: + break + fields = [CAM.unpack_from(cams, i * CAM.size) for i in range(ncams)] + if not self.cams: + self.cams = [{"name": n.split(b"\0", 1)[0].decode(errors="replace"), "width": w, "height": h} + for n, w, h, _, _ in fields] + self.sets.append((p, off, nbytes, min(c[4] for c in fields))) + off += nbytes + + def __len__(self): + return len(self.sets) + + def time_ns(self, i): + return self.sets[i][3] + + def duration_s(self): + """Recorded time: each part's first to last set, so pauses don't count.""" + total = 0 + ends = self.part_starts[1:] + [len(self.sets)] + for start, end in zip(self.part_starts, ends): + if end > start: + total += self.sets[end - 1][3] - self.sets[start][3] + return total / 1e9 + + def bytes(self): + return sum(s[2] for s in self.sets) + + def read_set(self, i, only=None): + """Set i's cameras: [{name, width, height, capture_ns, dqbuf_ns, pixels}], pixels as raw + 8-bit grey bytes (width x height, packed). only: a camera name, to read just that one.""" + p, off, nbytes, _ = self.sets[i] + with open(self.parts[p], "rb") as f: + f.seek(off) + ncams = HDR.unpack(f.read(HDR.size))[1] + heads = f.read(CAM.size * ncams) + out = [] + px_off = off + HDR.size + CAM.size * ncams + for k in range(ncams): + name, w, h, capture_ns, dqbuf_ns = CAM.unpack_from(heads, k * CAM.size) + name = name.split(b"\0", 1)[0].decode(errors="replace") + if only is None or name == only: + f.seek(px_off) + out.append({"name": name, "width": w, "height": h, "capture_ns": capture_ns, + "dqbuf_ns": dqbuf_ns, "pixels": f.read(w * h)}) + px_off += w * h + return out + + def read_raw(self, i): + """Set i's whole record, as it is in the file.""" + p, off, nbytes, _ = self.sets[i] + with open(self.parts[p], "rb") as f: + f.seek(off) + return f.read(nbytes) + + +_index_cache = {} +_index_lock = threading.Lock() + + +def take_index(take_dir): + """A TakeIndex, cached until one of the take's recording files changes size or time.""" + key_parts = [] + for path in TakeIndex._parts(take_dir): + try: + st = os.stat(path) + key_parts.append((path, st.st_size, st.st_mtime_ns)) + except OSError: + pass + key = tuple(key_parts) + with _index_lock: + cached = _index_cache.get(take_dir) + if cached and cached[0] == key: + return cached[1] + index = TakeIndex(take_dir) + with _index_lock: + _index_cache[take_dir] = (key, index) + return index + + +# ---------------------------------------------------------------- the store + +class Store: + def __init__(self, base=DEFAULT_BASE): + self.base = os.path.abspath(os.path.expanduser(base)) + self.sessions_dir = os.path.join(self.base, "sessions") + self.exports_dir = os.path.join(self.base, "exports") + self.profile_path = os.path.join(self.base, "profile.json") + + # --- paths, checked: ids come from the window, so nothing may climb out of the base + def session_dir(self, session): + if not SESSION_RE.match(session or ""): + raise ValueError(f"not a session id: {session!r}") + return os.path.join(self.sessions_dir, session) + + def take_dir(self, session, take): + if not take or "/" in take or take.startswith("."): + raise ValueError(f"not a take id: {take!r}") + return os.path.join(self.session_dir(session), "takes", take) + + def export_dir(self, session): + self.session_dir(session) # checks the id + return os.path.join(self.exports_dir, session) + + def profile(self): + return read_json(self.profile_path) + + # --- listing + def sessions(self): + """Newest first: [{id, started, takes, bytes, lighting, status, dry_run, contributor, exported, + export_stale (changed since), export_bytes}].""" + out = [] + try: + names = os.listdir(self.sessions_dir) + except OSError: + return out + for sid in names: + path = os.path.join(self.sessions_dir, sid) + if not SESSION_RE.match(sid) or not os.path.isdir(path): + continue + meta = read_json(os.path.join(path, "session.json")) + size, changed = tree_stat(path) + manifest = os.path.join(self.export_dir(sid), "manifest.json") + exported = os.path.getmtime(manifest) if os.path.isfile(manifest) else 0.0 + out.append({"id": sid, "started": meta.get("started", ""), "takes": len(self.take_ids(sid)), + "bytes": size, "contributor": meta.get("contributor", ""), + "lighting": (meta.get("lighting") or {}).get("chosen", ""), + "status": meta.get("status", ""), "dry_run": bool(meta.get("dry_run")), + "exported": bool(exported), "export_stale": bool(exported) and changed > exported, + "export_bytes": tree_bytes(self.export_dir(sid)) if exported else 0}) + return sorted(out, key=lambda s: s["id"], reverse=True) + + def take_ids(self, session): + try: + names = os.listdir(os.path.join(self.session_dir(session), "takes")) + except OSError: + return [] + return sorted(n for n in names if not n.startswith(".") + and os.path.isdir(os.path.join(self.session_dir(session), "takes", n))) + + def takes(self, session): + """[{id, section, title, status, sets, deleted_sets, ranges, duration_s, bytes, cams}].""" + out = [] + for tid in self.take_ids(session): + path = self.take_dir(session, tid) + meta = read_json(os.path.join(path, "take.json")) + index = take_index(path) + ranges = merge_ranges(meta.get("deleted") or []) + out.append({"id": tid, "section": meta.get("section", tid.split("-", 1)[-1]), + "title": meta.get("title") or tid, "status": meta.get("status", "recording"), + "sets": len(index), "parts": len(index.parts), + "deleted_sets": sum(1 for s in index.sets if in_ranges(s[3], ranges)), + "ranges": ranges, "duration_s": index.duration_s(), "bytes": index.bytes(), + "cams": [c["name"] for c in index.cams]}) + return out + + def take_meta(self, session, take): + return read_json(os.path.join(self.take_dir(session, take), "take.json")) + + # --- deleted ranges + def ranges(self, session, take): + return merge_ranges(self.take_meta(session, take).get("deleted") or []) + + def _set_ranges(self, session, take, ranges): + path = os.path.join(self.take_dir(session, take), "take.json") + meta = read_json(path) + meta["deleted"] = merge_ranges(ranges) + write_json(path, meta) + return meta["deleted"] + + def delete_range(self, session, take, from_ns, to_ns): + """Leave sets from from_ns to to_ns (dqbuf_ns, inclusive) out of the export.""" + return self._set_ranges(session, take, self.ranges(session, take) + [[int(from_ns), int(to_ns)]]) + + def restore_range(self, session, take, i): + """Take deleted range i (in ranges()' order) back.""" + ranges = self.ranges(session, take) + if 0 <= i < len(ranges): + del ranges[i] + return self._set_ranges(session, take, ranges) + + # --- deleting files + def delete_take(self, session, take): + path = self.take_dir(session, take) + if os.path.isdir(path): + shutil.rmtree(path) + meta_path = os.path.join(self.session_dir(session), "session.json") + meta = read_json(meta_path) + if take in (meta.get("takes") or []): + meta["takes"] = [t for t in meta["takes"] if t != take] + meta.setdefault("deleted_takes", []).append(take) + write_json(meta_path, meta) + + def delete_session(self, session): + """The session and its export, if any.""" + for path in (self.session_dir(session), self.export_dir(session)): + if os.path.isdir(path): + shutil.rmtree(path) + + def delete_export(self, session): + path = self.export_dir(session) + if os.path.isdir(path): + shutil.rmtree(path) + + # --- export + def export(self, session, progress=None, cancel=None, keep_notes=False, low_priority=True): + """Write exports// (DESIGN.md "Export"): manifest.json, calibration.json, per + take prompts.jsonl, poses.jsonl, take.json and sets.bin.zst (sets in deleted ranges + left out), then SHA256SUMS. progress(fraction 0..1, text) is called as it goes; + cancel() (or cancel.is_set()) returning true stops it, raising Cancelled. Nothing is + left behind on failure or cancel. Returns the export's path. + low_priority: run this thread at nice 19 as well as zstd (Linux nice is per thread).""" + zstd = find_zstd() + if not zstd: + raise RuntimeError("zstd isn't installed (in the dev container: sudo dnf install zstd)") + cancelled = getattr(cancel, "is_set", cancel) or (lambda: False) + report = progress or (lambda fraction, text: None) + src = self.session_dir(session) + if not os.path.isdir(src): + raise RuntimeError(f"no session {session}") + if low_priority: + try: + os.setpriority(os.PRIO_PROCESS, threading.get_native_id(), 19) + except OSError: + pass + final = self.export_dir(session) + work = final + ".partial" + if os.path.isdir(work): + shutil.rmtree(work) + os.makedirs(work) + try: + self._export(session, src, work, zstd, report, cancelled, keep_notes) + if os.path.isdir(final): + shutil.rmtree(final) + os.replace(work, final) + except BaseException: + shutil.rmtree(work, ignore_errors=True) + raise + report(1.0, "Done") + return final + + def _export(self, session, src, work, zstd, report, cancelled, keep_notes): + profile = self.profile() + meta = read_json(os.path.join(src, "session.json")) + takes = self.takes(session) + total = sum(t["bytes"] for t in takes) or 1 + done = 0 + take_entries = [] + for t in takes: + if cancelled(): + raise Cancelled() + tdir = self.take_dir(session, t["id"]) + out = os.path.join(work, "takes", t["id"]) + os.makedirs(out) + for name in ("prompts.jsonl", "poses.jsonl", "take.json"): + if os.path.isfile(os.path.join(tdir, name)): + shutil.copyfile(os.path.join(tdir, name), os.path.join(out, name)) + index = take_index(tdir) + ranges = t["ranges"] + keep = [i for i in range(len(index)) if not in_ranges(index.time_ns(i), ranges)] + entry = {"id": t["id"], "section": t["section"], "title": t["title"], "status": t["status"], + "sets": len(keep), "sets_deleted": len(index) - len(keep), "cameras": index.cams, + "duration_s": round(t["duration_s"], 2), "file": None} + done += index.bytes() - sum(index.sets[i][2] for i in keep) # deleted sets count as done + if keep: + entry["file"] = "takes/%s/sets.bin.zst" % t["id"] + + def step(nbytes, title=t["title"]): + nonlocal done + done += nbytes + report(min(done / total, 0.99), f"Compressing {title}") + entry["raw_bytes"] = self._compress(index, keep, zstd, os.path.join(out, "sets.bin.zst"), + step, cancelled) + take_entries.append(entry) + if cancelled(): + raise Cancelled() + if os.path.isfile(os.path.join(src, "calibration.json")): + shutil.copyfile(os.path.join(src, "calibration.json"), os.path.join(work, "calibration.json")) + shown_profile = json.loads(json.dumps(profile)) + if not keep_notes and isinstance(shown_profile.get("optional"), dict): + shown_profile["optional"].pop("notes", None) + manifest = {"schema": EXPORT_SCHEMA, "tool": tool_version(), + "exported": datetime.datetime.now().astimezone().isoformat(timespec="seconds"), + "session_id": session, "contributor": profile.get("contributor", meta.get("contributor", "")), + "consent_version": (profile.get("consent") or {}).get("version", ""), + "profile": shown_profile, "session": meta, "takes": take_entries} + write_json(os.path.join(work, "manifest.json"), manifest) + report(0.995, "Writing checksums") + sums = [] + for root, _, files in os.walk(work): + for name in files: + path = os.path.join(root, name) + sums.append((os.path.relpath(path, work), sha256_file(path, cancelled))) + with open(os.path.join(work, "SHA256SUMS"), "w") as f: + for rel, digest in sorted(sums): + f.write(f"{digest} {rel}\n") + + @staticmethod + def _compress(index, keep, zstd, dest, step, cancelled): + """Stream the kept sets through zstd into dest. Returns the uncompressed size.""" + raw = 0 + with open(dest, "wb") as out: + proc = subprocess.Popen(["nice", "-n", "19", zstd] + ZSTD_ARGS, stdin=subprocess.PIPE, stdout=out, + stderr=subprocess.PIPE) + try: + for i in keep: + if cancelled(): + raise Cancelled() + record = index.read_raw(i) + proc.stdin.write(record) + raw += len(record) + step(len(record)) + proc.stdin.close() + err = proc.stderr.read().decode(errors="replace").strip() + if proc.wait() != 0: + raise RuntimeError(f"zstd failed: {err or proc.returncode}") + except BaseException: + proc.kill() + proc.wait() + raise + return raw + + +def sha256_file(path, cancelled=lambda: False): + h = hashlib.sha256() + with open(path, "rb") as f: + while chunk := f.read(CHUNK): + if cancelled(): + raise Cancelled() + h.update(chunk) + return h.hexdigest() + + +def human_bytes(n): + for unit in ("B", "KB", "MB", "GB"): + if n < 1000 or unit == "GB": + return f"{n:.0f} {unit}" if unit == "B" else f"{n:.1f} {unit}" + n /= 1000 + return f"{n:.1f} TB" + + +def main(): + ap = argparse.ArgumentParser(description="List, check and export hand recorder sessions.") + ap.add_argument("--base", default=DEFAULT_BASE) + sub = ap.add_subparsers(dest="cmd", required=True) + sub.add_parser("list") + sub.add_parser("takes").add_argument("session") + sub.add_parser("export").add_argument("session") + a = ap.parse_args() + store = Store(a.base) + if a.cmd == "list": + for s in store.sessions(): + print(f"{s['id']} {s['takes']} takes {human_bytes(s['bytes'])} {s['lighting'] or '-'}" + + (" exported" if s["exported"] else "")) + elif a.cmd == "takes": + for t in store.takes(a.session): + print(f"{t['id']} {t['status']} {t['sets']} sets ({t['deleted_sets']} deleted) " + f"{t['duration_s']:.1f} s {human_bytes(t['bytes'])}") + else: + path = store.export(a.session, progress=lambda f, text: print(f"\r{f * 100:5.1f}% {text:40.40}", end="", + flush=True)) + print(f"\n{path} {human_bytes(tree_bytes(path))}") + + +if __name__ == "__main__": + main() diff --git a/setup/dev-container.sh b/setup/dev-container.sh index 10229ba..e814282 100755 --- a/setup/dev-container.sh +++ b/setup/dev-container.sh @@ -23,6 +23,8 @@ packages=( # tools (hands/tools) need NumPy and OpenCV, which aren't here: Fedora's python3-opencv pulls # in over a gigabyte (hands/README.md says how to get them) jsoncpp-devel glibc-static + # hand recorder (hands/rec): its export compresses recordings with zstd + zstd # Frametop Input Settings app (Kirigami, PySide6) python3-pyside6 kf6-kirigami kf6-qqc2-desktop-style qt6-qtwayland breeze-icon-theme plasma-breeze # Frametop remote desktop (VNC bridge through krdp)