Releases: ship the Hand Recorder

The image builds ft-handpanel (hands/rec/build.sh, against scripts/openvr.sh
like gaze), and a release's installer offers the recorder: ft-camd's
capabilities and the menu entry, nothing to build. The FrameDrop window has a
Hand Recorder box, and --no-hand-recorder leaves it out. An entry from the
release before is renewed without asking, since it points into that
release's folder.

The recorder runs ft-hands through scripts/in-box instead of the "dev"
container, so a release uses its own container, and its repair hints say to
reinstall the release rather than run a build script.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.github/workflows/release.yml

@@ -59,11 +59,13 @@ jobs:
             test -f /src/frametop/pointer/driver/build/driver_ft_pointer.so
             test -x /src/frametop/hands/build/ft-camd
             test -x /src/frametop/hands/build/ft-hands
+            test -x /src/frametop/hands/rec/build/ft-handpanel
             /src/frametop/hands/models.sh check
             test -x /src/frametop/pack/build/distrobox/install
             /src/frametop/gaze/tracker/build/venv/bin/python -c "import numpy, cv2"
             /src/frametop/voice/build/venv/bin/python -c "import numpy, sentencepiece, sherpa_onnx"
-            python3 -c "import PySide6"'
+            python3 -c "import PySide6, huggingface_hub"
+            command -v zstd >/dev/null'
 
       # The FrameDrop manifest names the zip's URL in this repo's release (a fork's, in a fork).
       - name: Frametop.zip

framedrop/README.md

@@ -1,6 +1,6 @@
 # Install with FrameDrop (proof of concept)
 
-[FrameDrop](https://framedropvr.com) is a Windows app that sideloads onto a Steam Frame: it copies a build to the headset and adds it to the Steam library. Issue #25 asks for an "Install with FrameDrop" button. Frametop isn't an app FrameDrop can copy over as is: it installs user services, a SteamVR driver, and a container, and three optional parts need sudo. So FrameDrop installs a small installer instead. Playing "Frametop" from the library opens a window that asks what to install, and your password for the parts that need it, then installs and shows its progress. A release's Frametop.zip (about 1.1 GB, built by CI on a tag, see [pack/README.md](../pack/README.md), Releases) carries Frametop built, as an image, and installs it with `install-release.sh`: nothing compiles on the headset and nothing else downloads. The same zip works unpacked on the headset. A test zip (a few KB) clones Frametop with `get.sh` instead.
+[FrameDrop](https://framedropvr.com) is a Windows app that sideloads onto a Steam Frame: it copies a build to the headset and adds it to the Steam library. Issue #25 asks for an "Install with FrameDrop" button. Frametop isn't an app FrameDrop can copy over as is: it installs user services, a SteamVR driver, and a container, and four optional parts need sudo. So FrameDrop installs a small installer instead. Playing "Frametop" from the library opens a window that asks what to install, and your password for the parts that need it, then installs and shows its progress. A release's Frametop.zip (about 1.1 GB, built by CI on a tag, see [pack/README.md](../pack/README.md), Releases) carries Frametop built, as an image, and installs it with `install-release.sh`: nothing compiles on the headset and nothing else downloads. The same zip works unpacked on the headset. A test zip (a few KB) clones Frametop with `get.sh` instead.
 
 Nothing here is published yet: no release, no button.
 
@@ -26,7 +26,7 @@ It uses Valve's SteamOS Devkit path: pair once with the headset's devkit service
 
 1. In the container, it starts itself again on the host with `flatpak-spawn --host`.
 2. It clears Steam's preload and library paths, and opens `installer/progress.py` (GTK 4 and libadwaita).
-3. The window asks which optional parts to install: voice dictation (on by default), hand tracking (on by default), our own eye tracker (on by default), and the Bluetooth fixes. All but voice dictation need sudo, so it asks for your SteamOS password and checks it with `sudo -v`. If your user has no password (SteamOS starts without one), it says how to set one and leaves those three out.
+3. The window asks which optional parts to install: voice dictation (on by default), hand tracking (on by default), the Hand Recorder (on by default; a release's zip only, since a clone builds it with `hands/rec/install.sh`), our own eye tracker (on by default), and the Bluetooth fixes. All but voice dictation need sudo, so it asks for your SteamOS password and checks it with `sudo -v`. If your user has no password (SteamOS starts without one), it says how to set one and leaves those out.
 4. It runs the zip's `install-release.sh --yes` (a test zip: `get.sh --yes`) in a transient user service, `frametop-framedrop-install`. The service is used because Steam ends the title's whole process tree when it's quit, and starts it with an OOM score of 900. Opened from a VR desktop (the zip unpacked by hand), it still uses the user's real bus and runtime folder for the service.
 5. It follows the service's log, shows the steps as a progress bar, and reports the result. Closing it leaves the install running. Playing the title again reattaches.
 

framedrop/installer/progress.py

@@ -13,7 +13,7 @@ The install runs in a user service of its own (UNIT): Steam ends the title's who
 tree when it's quit, and starts it with a high OOM score. Closing the window doesn't stop the
 install; playing the title again reattaches to it.
 
-Our eye tracker and the Bluetooth fixes need sudo. The window asks for your password on the
+Hand tracking, the Hand Recorder, our eye tracker and the Bluetooth fixes need sudo. The window asks for your password on the
 headset, checks it with sudo, and keeps it in this process's memory until the install ends.
 sudo in the install gets it through askpass (SUDO_ASKPASS), from a socket in a folder only you
 can open, and the window answers only programs in the install's service. It's never written to
@@ -101,7 +101,7 @@ def in_unit(pid):
         return False
 
 
-def install_command(eye_tracker, bluetooth, voice=True, hands=True):
+def install_command(eye_tracker, bluetooth, voice=True, hands=True, hand_recorder=True):
     if RELEASE:
         args = [str(HERE / "install-release.sh"), "--yes"]
         if DRY_RUN:
@@ -114,6 +114,8 @@ def install_command(eye_tracker, bluetooth, voice=True, hands=True):
         args.append("--no-voice")
     if not hands:
         args.append("--no-hands")
+    if RELEASE and not hand_recorder:   # a clone (get.sh) doesn't offer it: hands/rec/install.sh builds it
+        args.append("--no-hand-recorder")
     if not eye_tracker:
         args.append("--no-eye-tracker")
     if bluetooth:
@@ -306,12 +308,16 @@ class Window(Adw.ApplicationWindow):
     def show_choices(self):
         what = f"Frametop {release_version()}" if RELEASE else "Frametop from GitHub"
         self.detail.set_label(f"This installs {what}: the multi-screen desktop, the 3D mouse, gaze "
-                              "mode, and Frametop's settings apps. Optional: voice dictation, and three "
+                              "mode, and Frametop's settings apps. Optional: voice dictation, and "
                               "parts that need your SteamOS password (sudo):")
         self.voice = Gtk.CheckButton(label="Voice dictation: a Right Ctrl tap or holding the left d-pad down, and "
                                            "what you say is typed (downloads about 320 MB)", active=True)
         self.hands = Gtk.CheckButton(label="Hand tracking (experimental): poke, pinch and grab the screens with "
                                            "your hands. It uses a lot of the headset's CPU", active=True)
+        # A release brings it built; a clone builds it with hands/rec/install.sh, so isn't offered it
+        self.recorder = Gtk.CheckButton(label="Hand Recorder: record your hands for Frametop's open hand "
+                                              "dataset, which trains hand tracking (nothing is uploaded "
+                                              "until you press Upload)", active=True, visible=RELEASE)
         self.eye = Gtk.CheckButton(label="Our own eye tracker for gaze mode (more accurate than SteamVR's)",
                                    active=True)
         self.bt = Gtk.CheckButton(label="Bluetooth fixes (LE mice and keyboards, like the Swiftpoint Z3, "
@@ -327,18 +333,20 @@ class Window(Adw.ApplicationWindow):
         self.go.connect("clicked", self.install)
         self.pw.connect("activate", self.install)
         pw_row, keypad = keypad_row(self.pw)
-        self.choices = [self.voice, self.hands, self.eye, self.bt, pw_row, keypad, pw_note, self.error, self.go]
+        self.choices = [self.voice, self.hands, self.recorder, self.eye, self.bt, pw_row, keypad, pw_note,
+                        self.error, self.go]
         if not password_set():
-            for c in (self.hands, self.eye, self.bt):
+            for c in (self.hands, self.recorder, self.eye, self.bt):
                 c.set_active(False)
                 c.set_sensitive(False)
             pw_row.set_visible(False)
             keypad.set_visible(False)
-            pw_note.set_label("Your user has no password, so sudo can't run and these three can't be "
+            pw_note.set_label("Your user has no password, so sudo can't run and these can't be "
                               "installed from here. To set one, run passwd in Konsole; then play "
                               "Frametop again, or install them later from a terminal "
-                              "(hands/run.sh install, gaze/tracker/install.sh, setup/bluetooth/install.sh).")
-        for c in (self.hands, self.eye, self.bt):
+                              "(hands/run.sh install, hands/rec/install.sh, gaze/tracker/install.sh, "
+                              "setup/bluetooth/install.sh).")
+        for c in (self.hands, self.recorder, self.eye, self.bt):
             c.connect("toggled", lambda *_: pw_row.set_sensitive(self.needs_password()))
         for w in self.choices:
             self.box.append(w)
@@ -374,7 +382,8 @@ class Window(Adw.ApplicationWindow):
         threading.Thread(target=check, daemon=True).start()
 
     def needs_password(self):
-        return self.hands.get_active() or self.eye.get_active() or self.bt.get_active()
+        return (self.hands.get_active() or (RELEASE and self.recorder.get_active()) or self.eye.get_active()
+                or self.bt.get_active())
 
     def say(self, text):
         self.error.set_label(text)
@@ -382,7 +391,7 @@ class Window(Adw.ApplicationWindow):
 
     def begin(self, pw):
         args = install_command(self.eye.get_active(), self.bt.get_active(), self.voice.get_active(),
-                               self.hands.get_active())
+                               self.hands.get_active(), self.recorder.get_active())
         if pw is not None:
             try:
                 self.askpass = Askpass(self.ask_again)

hand-recorder.md

@@ -12,6 +12,8 @@ Join the [Frametop Discord](https://discord.gg/W3X9f7z3Bc) for questions and hel
 
 For now the recorder runs inside Frametop's desktop, so these steps install Frametop first. A standalone recorder that runs from the SteamVR dashboard without Frametop is planned.
 
+Installing a Frametop release (Frametop.zip, FrameDrop, or `get.sh --release`) instead? Its installer offers the Hand Recorder, built, so there's nothing to compile: leave Hand Recorder ticked (or answer yes) and skip step 2. To add it to a release later, run `~/.local/share/frametop/releases/current/hands/rec/install.sh`.
+
 ## Before you start
 
 - You must be 18 or older, and for now you can't take part if you live in Illinois, Texas or Washington (USA). The [consent text](https://github.com/Frametop/frametop/blob/main/hands/rec/CONSENT.md) explains what's recorded and what you agree to. The recorder shows it again before your first session.

hands/rec/build.sh

@@ -1,18 +1,18 @@
 #!/usr/bin/env bash
 # Build the hand recorder's headset panel ft-handpanel in the dev container on the Frame
-# (hands/rec/build/; it also runs there). Like gaze/build.sh: the pinned public OpenVR header
-# (the DMA-BUF import is newer than the header shipped with SteamVR's samples), stb_truetype
+# (hands/rec/build/; it also runs there), or in the release image (pack/Containerfile). Like
+# gaze/build.sh: the pinned public OpenVR header (scripts/openvr.sh; the DMA-BUF import is newer
+# than the header shipped with SteamVR's samples), stb_truetype
 # for the text and stb_image (PNG and GIF) for the pose pictures and the demo clips, at the same stb commit.
 set -euo pipefail
 root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
 "$root/scripts/sync.sh" >/dev/null
 exec "$root/scripts/frame.sh" -C hands/rec 'set -e; mkdir -p build/include
-openvr=v2.15.6
-[ -f build/include/openvr-$openvr ] || { curl -fsSL "https://raw.githubusercontent.com/ValveSoftware/openvr/$openvr/headers/openvr.h" -o build/include/openvr.h && touch build/include/openvr-$openvr; }
+. ../../scripts/openvr.sh
 stb=2c980bb59875b0d32144a71867fbdebb2f77cd20
 [ -f build/include/stb-$stb ] || { curl -fsSL "https://raw.githubusercontent.com/nothings/stb/$stb/stb_truetype.h" -o build/include/stb_truetype.h && touch build/include/stb-$stb; }
 [ -f build/include/stb_image-$stb ] || { curl -fsSL "https://raw.githubusercontent.com/nothings/stb/$stb/stb_image.h" -o build/include/stb_image.h && touch build/include/stb_image-$stb; }
 g++ -std=c++17 -O2 -Wall -Wno-unused-parameter -Wno-missing-field-initializers -Ibuild/include $(pkg-config --cflags gbm libdrm) \
-  -o build/ft-handpanel panel/ft-handpanel.cpp -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64 \
+  -o build/ft-handpanel panel/ft-handpanel.cpp $OPENVR_LIBS \
   $(pkg-config --libs gbm libdrm) -lpthread
 echo "built build/ft-handpanel"'

hands/rec/ft-handrec

@@ -1,6 +1,6 @@
 #!/bin/bash
 # Launch the Frametop Hand Recorder from a Plasma session on the Frame host.
-# The app runs in the dev container (PySide6, Kirigami and zstd come from Fedora there).
+# The app runs in Frametop's container (PySide6, Kirigami and zstd come from Fedora there).
 # podman needs the real XDG_RUNTIME_DIR and the real user bus (to reach systemd for
 # the container's cgroup; the Frametop session runs on a private bus from
 # dbus-run-session). The session's Wayland socket and bus go to the app itself.
@@ -14,8 +14,7 @@ export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus
 # From the home folder: the app's host commands (distrobox-host-exec) run in its folder on the
 # host, and a folder only the container has, such as /run/host/tmp, makes them all fail.
 cd "$HOME" || exit 1
-"$here/../../scripts/container-up.sh"
-exec "$HOME/.local/bin/distrobox" enter dev -- env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
+exec "$here/../../scripts/in-box" env WAYLAND_DISPLAY="$wl" DISPLAY="${DISPLAY:-}" \
   XAUTHORITY="${XAUTHORITY:-}" DBUS_SESSION_BUS_ADDRESS="$session_bus" \
   QT_QPA_PLATFORM="wayland;xcb" \
   python3 "$here/ft_handrec.py" "$@"

hands/rec/install.sh

@@ -6,6 +6,8 @@
 # "Frametop Hand Recorder" to the app menu (for Frametop's desktop: it isn't tested from SteamVR's
 # "Launch a program", which runs apps outside it; the standalone recorder is for that).
 # It doesn't turn on Frametop's live hand tracking (that's hands/run.sh install, which install.sh offers).
+# A release (FRAME_RELEASE) brings the three built in its image (pack/Containerfile), so there it only
+# sets the capabilities and adds the menu entry; its install.sh offers this.
 # Usage: hands/rec/install.sh            install or update
 #        hands/rec/install.sh uninstall  remove the menu entry, and ft-camd's capabilities unless
 #                                        hand tracking's services use them (hands/run.sh uncaps).
@@ -17,13 +19,17 @@ entry='~/.local/share/applications/frametop-handrec.desktop'
 
 case ${1:-install} in
   install)
-    echo "== 1/4 dev container packages"
-    "$root/setup/dev-container.sh"
-    echo "== 2/4 hand tracking: ft-camd and ft-hands"
-    "$root/hands/build.sh"
-    echo "== 3/4 the headset panel: ft-handpanel"
-    "$root/hands/rec/build.sh"
-    echo "== 4/4 ft-camd's capabilities (asks for your password) and the menu entry"
+    if [ "$FRAME_RELEASE" = 1 ]; then
+      echo "== ft-camd's capabilities (asks for your password, unless it has them) and the menu entry"
+    else
+      echo "== 1/4 dev container packages"
+      "$root/setup/dev-container.sh"
+      echo "== 2/4 hand tracking: ft-camd and ft-hands"
+      "$root/hands/build.sh"
+      echo "== 3/4 the headset panel: ft-handpanel"
+      "$root/hands/rec/build.sh"
+      echo "== 4/4 ft-camd's capabilities (asks for your password) and the menu entry"
+    fi
     "$root/hands/run.sh" caps
     "$root/scripts/conf-migrate.sh"   # HANDS_SWAP_SIDES=0, the old default, becomes auto
     fill_template "$root/hands/rec/ft-handrec.desktop" |

hands/rec/session.py

@@ -55,6 +55,9 @@ import takes  # noqa: E402  (hands/rec/takes.py: the version, and the standalone
 FT_HANDS = os.path.join(HANDS, "build", "ft-hands")
 FT_CAMD = os.path.join(HANDS, "build", "ft-camd")
 PANEL_BIN = os.path.join(HERE, "build", "ft-handpanel")
+# Runs a program in the container this tree was built for: "dev" in a clone, the release's own in a
+# release (it starts the container first)
+IN_BOX = os.path.join(REPO, "scripts", "in-box")
 SCRIPT_PATH = os.path.join(HERE, "script.json")
 POSES_DIR = os.path.join(HERE, "poses")   # the pose pictures: poses.json and its PNGs
 DEMOS_DIR = os.path.join(HERE, "demos")   # the demo clips: <id>.gif (DESIGN.md "Demo clips")
@@ -83,6 +86,8 @@ HANDS_UNIT = "frametop-handrec-hands.service"
 # The standalone Hand Recorder (takes.standalone()): its binaries run on the host, so ft-hands
 # starts directly rather than in the dev container, and repairs mean its install command.
 STANDALONE = takes.standalone()
+# A Frametop release (pack/install-release.sh) rather than a clone
+RELEASE = os.path.exists(os.path.join(REPO, ".frametop-release"))
 
 RECORD_HZ = 10
 SIDES_READ_S = 0.5       # how often the tracking ft-hands' side camera decision is read
@@ -161,9 +166,13 @@ def host_command(*cmd):
 
 
 def fix_hint(frametop):
-    """How to repair a missing or broken part: Frametop's command, or the standalone install's."""
+    """How to repair a missing or broken part: Frametop's command, or the standalone install's.
+    A Frametop release never builds (its programs come in its image), so a build there means
+    installing the release again."""
     if STANDALONE:
         return STANDALONE.get("reinstall") or "run the Hand Recorder's install command again"
+    if RELEASE and frametop.endswith("build.sh"):
+        return "install this Frametop release again"
     return frametop
 
 
@@ -696,7 +705,7 @@ class Recorder:
         if ring:
             argv += ["--ring", ring]
         if not in_container() and not STANDALONE:
-            argv = [os.path.expanduser("~/.local/bin/distrobox"), "enter", "dev", "--"] + argv
+            argv = [IN_BOX] + argv
         self.proc = subprocess.Popen(argv, stdin=subprocess.DEVNULL, stdout=log_file, stderr=log_file)
         self.started_ns = mono_ns()
 
@@ -1408,7 +1417,7 @@ def camera_text(result):
     if camcheck.is_ring_short(result):
         return ("The headset's tracking cameras are all running, but the recorder can't read some of them (%s). "
                 "Close the Hand Recorder, %s, and open it again. If that doesn't help, ask in the Frametop "
-                "Discord." % (result.get("reason", ""), fix_hint("run ~/frametop/hands/rec/install.sh again")))
+                "Discord." % (result.get("reason", ""), fix_hint("run %s again" % os.path.join(REPO, "hands", "rec", "install.sh"))))
     return ("Not all of the headset's tracking cameras are running (%s). Restart SteamVR, or restart the "
             "headset if that doesn't fix it." % result.get("reason", ""))
 
@@ -1846,11 +1855,8 @@ class Session:
 
     def _start_tracker(self):
         argv = [FT_HANDS, "--no-gestures", "--status", "0"]
-        if not STANDALONE:   # built in the dev container: it runs there
-            up = os.path.join(REPO, "scripts", "container-up.sh")
-            if os.access(up, os.X_OK):
-                subprocess.run(host_command(up), capture_output=True, timeout=120)
-            argv = [os.path.expanduser("~/.local/bin/distrobox"), "enter", "dev", "--"] + argv
+        if not STANDALONE:   # built in Frametop's container: it runs there
+            argv = [IN_BOX] + argv
         if self.ring:
             argv += ["--ring", self.ring]
         self._start_unit(HANDS_UNIT, "hand tracking for feedback", argv)

hands/rec/tests/test_session.py

@@ -918,18 +918,23 @@ class StandaloneTest(unittest.TestCase):
         self.assertIn("--no-gestures", argv)
         run.assert_not_called()   # no container to bring up
 
-    def test_ft_hands_in_the_dev_container(self):
-        self.assertTrue(self.recorder_argv(None)[0].endswith("distrobox"))
+    def test_ft_hands_in_frametops_container(self):
+        # scripts/in-box: "dev" in a clone, the release's own container in a release
+        self.assertEqual(self.recorder_argv(None)[:2], [session.IN_BOX, session.FT_HANDS])
         argv, _ = self.tracker_argv(None)
-        self.assertEqual(argv[:4], [os.path.expanduser("~/.local/bin/distrobox"), "enter", "dev", "--"])
-        self.assertEqual(argv[4], session.FT_HANDS)
+        self.assertEqual(argv[:2], [session.IN_BOX, session.FT_HANDS])
+        self.assertTrue(os.access(session.IN_BOX, os.X_OK))
 
     def test_fix_hint(self):
         with mock.patch.object(session, "STANDALONE", self.INFO):
             self.assertEqual(session.fix_hint("hands/build.sh"), "run the install command again")
             text = session.camera_text({"status": "degraded", "reason": "2 of 4", "ring_missing": ["upper_left"]})
-        with mock.patch.object(session, "STANDALONE", None):
+        with mock.patch.object(session, "STANDALONE", None), mock.patch.object(session, "RELEASE", False):
             self.assertEqual(session.fix_hint("hands/build.sh"), "hands/build.sh")
+        with mock.patch.object(session, "STANDALONE", None), mock.patch.object(session, "RELEASE", True):
+            # a release never builds: its programs come in its image
+            self.assertEqual(session.fix_hint("hands/rec/build.sh"), "install this Frametop release again")
+            self.assertEqual(session.fix_hint("hands/run.sh caps (asks for sudo)"), "hands/run.sh caps (asks for sudo)")
         self.assertIn("run the install command again", text)
         self.assertNotIn("~/frametop", text)
 

install.sh

@@ -7,12 +7,15 @@
 # In a release (pack/install-release.sh), the programs come built from its image: this installs the
 # distrobox the release brings, makes the release's container from its image, and builds nothing.
 #
-# Usage: ./install.sh [--yes] [--no-voice] [--no-hands] [--no-eye-tracker] [--no-bluetooth | --bluetooth]
-#   --yes           don't ask; installs voice dictation and gaze mode, and hand tracking and our
-#                   eye tracker if sudo can run without a password prompt; skips the Bluetooth
-#                   fixes and the SteamVR restart
+# Usage: ./install.sh [--yes] [--no-voice] [--no-hands] [--no-hand-recorder] [--no-eye-tracker]
+#                    [--no-bluetooth | --bluetooth]
+#   --yes           don't ask; installs voice dictation and gaze mode, and hand tracking, the Hand
+#                   Recorder and our eye tracker if sudo can run without a password prompt; skips
+#                   the Bluetooth fixes and the SteamVR restart
 #   --no-voice      don't offer voice dictation (Frametop Voice: it downloads about 320 MB)
-#   --no-hands      don't offer hand tracking
+#   --no-hands      don't offer hand tracking or the Hand Recorder
+#   --no-hand-recorder  don't offer the Hand Recorder (a release's; a clone builds it with
+#                   hands/rec/install.sh)
 #   --no-eye-tracker  don't offer our own eye tracker
 #   --no-bluetooth  don't offer the Bluetooth fixes
 #   --bluetooth     install the Bluetooth fixes without asking (with --yes: if sudo can run
@@ -22,7 +25,7 @@ set -euo pipefail
 root=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
 . "$root/scripts/_env.sh"
 
-assume_yes=0 bluetooth=1 eye_tracker=1 hands=1 voice=1
+assume_yes=0 bluetooth=1 eye_tracker=1 hands=1 hand_recorder=1 voice=1
 for arg in "$@"; do
   case $arg in
     --yes) assume_yes=1 ;;
@@ -30,8 +33,9 @@ for arg in "$@"; do
     --bluetooth) bluetooth=2 ;;
     --no-eye-tracker) eye_tracker=0 ;;
     --no-voice) voice=0 ;;
-    --no-hands) hands=0 ;;
-    -h|--help) sed -n '2,19p' "$0"; exit 0 ;;
+    --no-hands) hands=0 hand_recorder=0 ;;
+    --no-hand-recorder) hand_recorder=0 ;;
+    -h|--help) sed -n '2,22p' "$0"; exit 0 ;;
     *) echo "unknown option: $arg" >&2; exit 2 ;;
   esac
 done
@@ -143,6 +147,7 @@ elif [ "$assume_yes" = 1 ] && ! sudo_quiet; then
   echo "skipped: it needs your password (sudo), and --yes doesn't ask. Install it later with: hands/run.sh install"
 elif ask "Install hand tracking? Poke, pinch and grab the screens with your hands. It runs while the Frametop desktop does (change that on the Hands page of Frametop Input Settings), and its camera broker needs your password (sudo)." y; then
   if "$root/hands/run.sh" install; then
+    hands_caps=1
     mode=$(on_frame "sed -n 's/^HANDS_MODE=\([a-z]*\).*/\1/p' ~/.config/frametop.conf | tail -1") || mode=
     if [ "$mode" = always ]; then
       echo "hand tracking runs whenever SteamVR does (HANDS_MODE=always, kept from your settings)"
@@ -159,6 +164,26 @@ elif ask "Install hand tracking? Poke, pinch and grab the screens with your hand
 else
   echo "skipped. Install later with: hands/run.sh install"
 fi
+# The Hand Recorder (hands/rec): a release brings it built, so it's only ft-camd's capabilities
+# (set already if hand tracking just installed) and the menu entry. An entry from the release
+# before points into that release's folder, which goes two releases on: it's renewed without asking.
+# A clone builds it with hands/rec/install.sh (a few minutes), so it isn't offered here.
+if [ "$FRAME_RELEASE" = 1 ]; then
+  recorder="the Hand Recorder (records your hands for Frametop's open hand dataset)"
+  if [ "$hand_recorder" = 0 ]; then
+    echo "$recorder: skipped. Add it later with: hands/rec/install.sh"
+  elif on_frame '[ -e ~/.local/share/applications/frametop-handrec.desktop ]'; then
+    "$root/hands/rec/install.sh" install >/dev/null && echo "$recorder: updated" ||
+      echo "$recorder: couldn't update it (above). Try again with: hands/rec/install.sh"
+  elif [ "$assume_yes" = 1 ] && [ "${hands_caps:-0}" = 0 ] && ! sudo_quiet; then
+    echo "$recorder: skipped: it needs your password (sudo), and --yes doesn't ask. Add it later with: hands/rec/install.sh"
+  elif ask "Add the Hand Recorder to the app menu? It records your hands for Frametop's open hand dataset, which trains hand tracking; nothing leaves the headset until you press Upload. Its camera broker needs your password (sudo), as hand tracking does." y; then
+    "$root/hands/rec/install.sh" install ||
+      echo "$recorder: not installed (above). Try again later with: hands/rec/install.sh"
+  else
+    echo "$recorder: skipped. Add it later with: hands/rec/install.sh"
+  fi
+fi
 
 step "10/12 gaze mode (optional, experimental: the pointer goes where you look)"
 gaze=0

pack/Containerfile

@@ -98,6 +98,10 @@ RUN voice/build.sh \
 # file capabilities point, so they aren't copied to /opt/frametop.
 RUN hands/build.sh \
  && test -x hands/build/ft-camd && test -x hands/build/ft-hands
+# The Hand Recorder's headset panel: it stays in hands/rec/build/, where the recorder
+# (hands/rec/session.py) runs it from.
+RUN hands/rec/build.sh \
+ && test -x hands/rec/build/ft-handpanel
 
 # A release installs from this image (pack/install-release.sh): it copies
 # /src/frametop, built, to the Frame for its host side (services, the driver,

pack/install-release.sh

@@ -5,7 +5,8 @@
 # version, commit, the image file's sha256 and the image's ID, and the SteamOS table). Nothing
 # builds on the headset, and nothing else is downloaded, but voice dictation's speech models
 # (about 320 MB, fixed URLs checked by sha256: voice/models.sh) if you install it. Hand tracking
-# comes built too: its install only sets its camera broker's file capabilities (sudo).
+# and the Hand Recorder come built too: their installs only set their camera broker's file
+# capabilities (sudo), and the recorder's adds its menu entry.
 #
 # 1. It checks this SteamOS build against the SteamOS table: Frametop's newest from GitHub
 #    (pack/steamos.json on main) when it can get it, else the one in the release. Tested: on.
@@ -18,7 +19,8 @@
 #    are removed, with their containers and images.
 #
 # Usage: install-release.sh [--yes] [--any-steamos] [--unpack-only] [--dir DIR]
-#                           [--no-voice] [--no-hands] [--no-eye-tracker] [--no-bluetooth | --bluetooth]
+#                           [--no-voice] [--no-hands] [--no-hand-recorder] [--no-eye-tracker]
+#                           [--no-bluetooth | --bluetooth]
 #   --unpack-only  stop after step 2 and the copy: don't run install.sh
 #   --dir DIR      where the releases go (default ~/.local/share/frametop/releases)
 # The rest go to install.sh. FRAMETOP_STEAMOS_TABLE names another table to fetch (a URL), or
@@ -111,8 +113,8 @@ main() {
       --any-steamos) any=1 ;;
       --unpack-only) unpack_only=1 ;;
       --dir) base=${2:?--dir needs a folder}; shift ;;
-      --no-voice|--no-hands|--no-eye-tracker|--no-bluetooth|--bluetooth) pass+=("$1") ;;
-      -h|--help) sed -n '2,24p' "$0"; return 0 ;;
+      --no-voice|--no-hands|--no-hand-recorder|--no-eye-tracker|--no-bluetooth|--bluetooth) pass+=("$1") ;;
+      -h|--help) sed -n '2,27p' "$0"; return 0 ;;
       *) echo "unknown option: $1" >&2; return 2 ;;
     esac
     shift

pack/release-notes.md

@@ -6,7 +6,7 @@ On a Steam Frame (SteamOS, VR variant). Download **Frametop.zip** below (about 1
 - **On the headset:** unpack Frametop.zip (in the desktop's Dolphin or Konsole) and run `Frametop/frametop-install.sh`.
 - **In a terminal on the headset:** `curl -fsSL https://frametop.github.io/frametop/get.sh | bash -s -- --release`
 
-A window asks what to install, and your SteamOS password for the three optional parts that need it (hand tracking, our own eye tracker, and the Bluetooth fixes); the password is only used for this install. Nothing is compiled on the headset. Restart SteamVR once afterwards.
+A window asks what to install, and your SteamOS password for the optional parts that need it (hand tracking, the Hand Recorder, our own eye tracker, and the Bluetooth fixes); the password is only used for this install. Nothing is compiled on the headset. Restart SteamVR once afterwards.
 
 Before installing, the installer checks your SteamOS build against the builds Frametop was tested on, and stops if this release is known not to work there.