From 3cccad352519d72e1eac617a6864cb0a2d97fedb Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Mon, 5 Oct 2026 16:54:03 -0600 Subject: [PATCH] Hands: label side cameras by the hands when a forced HANDS_SWAP_SIDES disagrees With HANDS_SWAP_SIDES set to 0 or 1, ft-hands kept the forced naming as the published truth even after the hands showed it was backwards. The hand recorder took that as the session's decision, so the export labelled slam_left and slam_right the wrong way round (dataset PR #4: every take swapped). ft-hands now publishes the hands' answer once they disagree (state "forced, disagrees"); tracking keeps the forced names. sides.read_live corrects the same case from an ft-hands built before this, and the recorder's log says to use auto. Co-Authored-By: Claude Opus 5.5 --- hands/rec/DESIGN.md | 2 +- hands/rec/session.py | 3 +++ hands/rec/sides.py | 5 +++++ hands/rec/tests/test_sides.py | 39 +++++++++++++++++++++++++++++++++++ hands/track/main.cpp | 9 +++++--- 5 files changed, 54 insertions(+), 4 deletions(-) diff --git a/hands/rec/DESIGN.md b/hands/rec/DESIGN.md index f67a7da..1e99bd4 100644 --- a/hands/rec/DESIGN.md +++ b/hands/rec/DESIGN.md @@ -29,7 +29,7 @@ Every part runs in the dev container, as ft-hands and Input Settings do. The hos - **A tracking ft-hands** gives feedback through the hands file: which hands are seen, the palm's distance, the index tip. If none is running, the session starts `ft-hands --no-gestures --status 0` (unit `frametop-handrec-hands.service`). An ft-hands already running is used as it is. - **A recording ft-hands** runs once per recording part: `ft-hands --record-only --record DIR --record-for SECONDS --record-hz 10 --status 0 --sides auto|0|1` (below, "Side cameras"). It runs as a plain child process of the session, ended with SIGTERM when the part ends. SIGTERM ends ft-hands' loop, and `Recorder` writes out its queue when it's destroyed. In step mode (below) a part is one step's countdown and hold, so a take has one part per step (about 40 in the hand poses); in auto mode a take is one part, plus one more after each pause. `--record-for` is only a safety net. - Why a process per part rather than one kept alive and paused: measured in the dev container with `ft-ringplay`'s ring (2026-10-02), `ft-hands --record-only` writes its first set 16-27 ms after it starts and ends 4-6 ms after SIGTERM, so a new part costs nothing the 3 s countdown doesn't cover. Every reader already takes parts in order (`takes.py`, `validate.py` through the export's single stream, the labeller's `fhl_io.py`, numbering `sets-10.bin` after `sets-9.bin`), ft-hands needs no new control, and nothing is written while a step waits. Before the hold starts the session checks that the part has written a set (`Recorder.has_data`, up to 3 s more), so the hold is recorded from its first frame. -- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand. +- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. With `HANDS_SWAP_SIDES` forced to `0` or `1`, the hands still decide what's published once they disagree (state `"forced, disagrees"`; `read_live` also corrects an ft-hands built before that). The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand. - **ft-handpanel** runs as a child process with `--watch-stdin`. It shows the panel and logs poses during each take. - **The headset button's reader** is a thread of the session (`ButtonReader`, below), not a process. diff --git a/hands/rec/session.py b/hands/rec/session.py index dfae52a..1bf2a59 100755 --- a/hands/rec/session.py +++ b/hands/rec/session.py @@ -1575,6 +1575,9 @@ class Session: self._save_session() self._log("side cameras: %s (%s, %s)" % ("SWAPPED" if swapped else "as named", new["decided_by"], json.dumps(new["evidence"]))) + if new["state"] == "forced, disagrees": + self._log("side cameras: HANDS_SWAP_SIDES in ~/.config/frametop.conf forces names the hands say are " + "backwards; the recording goes by the hands. Set HANDS_SWAP_SIDES=auto.") def _sides_swapped(self): """session.json's decision: True, False, or None (not known yet).""" diff --git a/hands/rec/sides.py b/hands/rec/sides.py index 5b91618..e4f4d59 100644 --- a/hands/rec/sides.py +++ b/hands/rec/sides.py @@ -125,4 +125,9 @@ def read_live(path=None, ring_path=None, now_ns=None): return None except OSError: return None + if (s.get("state") == "forced, disagrees" and s.get("swapped") is not None + and bool(s["swapped"]) == bool(s.get("names_swapped"))): + # An ft-hands built before 2026-10-06 kept a forced HANDS_SWAP_SIDES as the truth even + # when the hands disagreed. The hands are right: the truth is the other way round. + s = dict(s, swapped=not s["swapped"], decided_by="auto") return s diff --git a/hands/rec/tests/test_sides.py b/hands/rec/tests/test_sides.py index bd36e8b..39fcc2e 100644 --- a/hands/rec/tests/test_sides.py +++ b/hands/rec/tests/test_sides.py @@ -91,6 +91,29 @@ class RulesTest(unittest.TestCase): finally: shutil.rmtree(d) + def test_read_live_forced(self): + """HANDS_SWAP_SIDES=0 forced and the hands disagree: the hands are the truth, whether + ft-hands published them (built after 2026-10-06) or kept the forced value (before).""" + d = tempfile.mkdtemp() + try: + path = os.path.join(d, "sides.json") + + def live(**kw): + s = dict(pid=1, ring_ino=0, mode="0", names_swapped=False, + updated_ns=time.clock_gettime_ns(time.CLOCK_MONOTONIC), **kw) + with open(path, "w") as f: + json.dump(s, f) + return sides.read_live(path) + + self.assertEqual(live(state="forced", swapped=False, decided_by="config")["swapped"], False) + self.assertEqual(live(state="forced, agrees", swapped=False, decided_by="config")["swapped"], False) + old = live(state="forced, disagrees", swapped=False, decided_by="config") + self.assertEqual((old["swapped"], old["decided_by"]), (True, "auto")) + new = live(state="forced, disagrees", swapped=True, decided_by="auto") + self.assertEqual((new["swapped"], new["decided_by"]), (True, "auto")) + finally: + shutil.rmtree(d) + class TakesTest(unittest.TestCase): """A swapped session: one take's parts recorded before the decision (ft-camd's names) and @@ -226,6 +249,22 @@ class SessionSidesTest(unittest.TestCase): self.assertTrue(s._session_json["sides"]["reversed_from"]["swapped"]) self.assertFalse(takes.read_json(os.path.join(self.tmp, "session.json"))["sides"]["swapped"]) + def test_read_sides_forced(self): + """PR #4 on the dataset: HANDS_SWAP_SIDES=0 from the old example config, and the hands + disagree. The session takes the hands' answer, not the forced one.""" + s = session.Session(os.path.join(self.tmp, "base"), {}, {}, "room", self.script, dry_run=True, + hands_dir=self.tmp) + s.session_dir = self.tmp + s._session_json = {"sides": {"swapped": None}} + self.live(mode="0", state="forced", swapped=False, decided_by="config", names_swapped=False, evidence=None) + s._read_sides(force=True) + self.assertIs(s._sides_swapped(), False) + self.live(mode="0", state="forced, disagrees", swapped=False, decided_by="config", names_swapped=False) + s._read_sides(force=True) + self.assertIs(s._sides_swapped(), True) + self.assertEqual(s._session_json["sides"]["decided_by"], "auto") + self.assertEqual(s._session_json["sides"]["reversed_from"]["decided_by"], "config") + def test_recorder_parts(self): calls = [] diff --git a/hands/track/main.cpp b/hands/track/main.cpp index be21f21..db670be 100644 --- a/hands/track/main.cpp +++ b/hands/track/main.cpp @@ -27,7 +27,8 @@ // default) tells from the hands it tracks (track/sides.h): once it's sure, it exchanges the two // cameras if they're backwards (the tracked views move with their images), and checks once // more. 0 and 1 force the naming (1: exchanged; --swap-sides is --sides 1); it still checks, -// and warns if the hands disagree. The decision is published in /run/user/UID/frametop-hands/sides.json (see +// and if the hands disagree it warns and publishes what the hands say as the truth ("swapped"), +// so recordings are labelled right while tracking keeps the forced names. The decision is published in /run/user/UID/frametop-hands/sides.json (see // write_sides below) and, for recordings, in DIR/sides.json. --record-only can't tell (it tracks // nothing): under auto it records the ring's names as they are. // @@ -507,14 +508,16 @@ int main(int argc, char **argv) { const bool backwards = v == SideCheck::Swapped; // relative to the names as they are now const double after = (now - start) / 1e9; const std::string ev = side_check.json(), text = side_check.summary(); - if (sides_mode != "auto") { // forced: only say so + if (sides_mode != "auto") { // forced: the names stay; if the hands disagree, they're the truth const std::string what = (sides_from == "option" ? "--sides " : "HANDS_SWAP_SIDES=") + sides_mode; if (backwards) std::printf("side cameras: %s looks WRONG: the hands say the side cameras are the other way round (%s). " - "Use auto.\n", what.c_str(), text.c_str()); + "Tracking keeps the forced names; recordings are labelled by the hands. Use auto.\n", + what.c_str(), text.c_str()); else std::printf("side cameras: %s agrees with the hands (%s)\n", what.c_str(), text.c_str()); sides_state = backwards ? "forced, disagrees" : "forced, agrees"; + if (backwards) truth = !names_swapped, decided_by = "auto", decided_after_s = after; decision_evidence = ev; checking = false; } else if (side_round == 0 || backwards) {