Hands: label side cameras by the hands when a forced HANDS_SWAP_SIDES disagrees

With HANDS_SWAP_SIDES set to 0 or 1, ft-hands kept the forced naming as the
published truth even after the hands showed it was backwards. The hand recorder
took that as the session's decision, so the export labelled slam_left and
slam_right the wrong way round (dataset PR #4: every take swapped).

ft-hands now publishes the hands' answer once they disagree (state "forced,
disagrees"); tracking keeps the forced names. sides.read_live corrects the same
case from an ft-hands built before this, and the recorder's log says to use auto.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DeeJanuzandClaude Opus 5.5 committed 2026-10-05 16:54:03 -06:00
1 parent 276c1409e8
commit 3cccad3525
5 files changed
+54 -4

No files matched your search

+1 -1
View File
@@ -29,7 +29,7 @@ Every part runs in the dev container, as ft-hands and Input Settings do. The hos
- **A tracking ft-hands** gives feedback through the hands file: which hands are seen, the palm's distance, the index tip. If none is running, the session starts `ft-hands --no-gestures --status 0` (unit `frametop-handrec-hands.service`). An ft-hands already running is used as it is.
- **A recording ft-hands** runs once per recording part: `ft-hands --record-only --record DIR --record-for SECONDS --record-hz 10 --status 0 --sides auto|0|1` (below, "Side cameras"). It runs as a plain child process of the session, ended with SIGTERM when the part ends. SIGTERM ends ft-hands' loop, and `Recorder` writes out its queue when it's destroyed. In step mode (below) a part is one step's countdown and hold, so a take has one part per step (about 40 in the hand poses); in auto mode a take is one part, plus one more after each pause. `--record-for` is only a safety net.
- Why a process per part rather than one kept alive and paused: measured in the dev container with `ft-ringplay`'s ring (2026-10-02), `ft-hands --record-only` writes its first set 16-27 ms after it starts and ends 4-6 ms after SIGTERM, so a new part costs nothing the 3 s countdown doesn't cover. Every reader already takes parts in order (`takes.py`, `validate.py` through the export's single stream, the labeller's `fhl_io.py`, numbering `sets-10.bin` after `sets-9.bin`), ft-hands needs no new control, and nothing is written while a step waits. Before the hold starts the session checks that the part has written a set (`Recorder.has_data`, up to 3 s more), so the hold is recorded from its first frame.
- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand.
- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. With `HANDS_SWAP_SIDES` forced to `0` or `1`, the hands still decide what's published once they disagree (state `"forced, disagrees"`; `read_live` also corrects an ft-hands built before that). The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand.
- **ft-handpanel** runs as a child process with `--watch-stdin`. It shows the panel and logs poses during each take.
- **The headset button's reader** is a thread of the session (`ButtonReader`, below), not a process.
+3
View File
@@ -1575,6 +1575,9 @@ class Session:
self._save_session()
self._log("side cameras: %s (%s, %s)" % ("SWAPPED" if swapped else "as named", new["decided_by"],
json.dumps(new["evidence"])))
if new["state"] == "forced, disagrees":
self._log("side cameras: HANDS_SWAP_SIDES in ~/.config/frametop.conf forces names the hands say are "
"backwards; the recording goes by the hands. Set HANDS_SWAP_SIDES=auto.")
def _sides_swapped(self):
"""session.json's decision: True, False, or None (not known yet)."""
+5
View File
@@ -125,4 +125,9 @@ def read_live(path=None, ring_path=None, now_ns=None):
return None
except OSError:
return None
if (s.get("state") == "forced, disagrees" and s.get("swapped") is not None
and bool(s["swapped"]) == bool(s.get("names_swapped"))):
# An ft-hands built before 2026-10-06 kept a forced HANDS_SWAP_SIDES as the truth even
# when the hands disagreed. The hands are right: the truth is the other way round.
s = dict(s, swapped=not s["swapped"], decided_by="auto")
return s
+39
View File
@@ -91,6 +91,29 @@ class RulesTest(unittest.TestCase):
finally:
shutil.rmtree(d)
def test_read_live_forced(self):
"""HANDS_SWAP_SIDES=0 forced and the hands disagree: the hands are the truth, whether
ft-hands published them (built after 2026-10-06) or kept the forced value (before)."""
d = tempfile.mkdtemp()
try:
path = os.path.join(d, "sides.json")
def live(**kw):
s = dict(pid=1, ring_ino=0, mode="0", names_swapped=False,
updated_ns=time.clock_gettime_ns(time.CLOCK_MONOTONIC), **kw)
with open(path, "w") as f:
json.dump(s, f)
return sides.read_live(path)
self.assertEqual(live(state="forced", swapped=False, decided_by="config")["swapped"], False)
self.assertEqual(live(state="forced, agrees", swapped=False, decided_by="config")["swapped"], False)
old = live(state="forced, disagrees", swapped=False, decided_by="config")
self.assertEqual((old["swapped"], old["decided_by"]), (True, "auto"))
new = live(state="forced, disagrees", swapped=True, decided_by="auto")
self.assertEqual((new["swapped"], new["decided_by"]), (True, "auto"))
finally:
shutil.rmtree(d)
class TakesTest(unittest.TestCase):
"""A swapped session: one take's parts recorded before the decision (ft-camd's names) and
@@ -226,6 +249,22 @@ class SessionSidesTest(unittest.TestCase):
self.assertTrue(s._session_json["sides"]["reversed_from"]["swapped"])
self.assertFalse(takes.read_json(os.path.join(self.tmp, "session.json"))["sides"]["swapped"])
def test_read_sides_forced(self):
"""PR #4 on the dataset: HANDS_SWAP_SIDES=0 from the old example config, and the hands
disagree. The session takes the hands' answer, not the forced one."""
s = session.Session(os.path.join(self.tmp, "base"), {}, {}, "room", self.script, dry_run=True,
hands_dir=self.tmp)
s.session_dir = self.tmp
s._session_json = {"sides": {"swapped": None}}
self.live(mode="0", state="forced", swapped=False, decided_by="config", names_swapped=False, evidence=None)
s._read_sides(force=True)
self.assertIs(s._sides_swapped(), False)
self.live(mode="0", state="forced, disagrees", swapped=False, decided_by="config", names_swapped=False)
s._read_sides(force=True)
self.assertIs(s._sides_swapped(), True)
self.assertEqual(s._session_json["sides"]["decided_by"], "auto")
self.assertEqual(s._session_json["sides"]["reversed_from"]["decided_by"], "config")
def test_recorder_parts(self):
calls = []
+6 -3
View File
@@ -27,7 +27,8 @@
// default) tells from the hands it tracks (track/sides.h): once it's sure, it exchanges the two
// cameras if they're backwards (the tracked views move with their images), and checks once
// more. 0 and 1 force the naming (1: exchanged; --swap-sides is --sides 1); it still checks,
// and warns if the hands disagree. The decision is published in /run/user/UID/frametop-hands/sides.json (see
// and if the hands disagree it warns and publishes what the hands say as the truth ("swapped"),
// so recordings are labelled right while tracking keeps the forced names. The decision is published in /run/user/UID/frametop-hands/sides.json (see
// write_sides below) and, for recordings, in DIR/sides.json. --record-only can't tell (it tracks
// nothing): under auto it records the ring's names as they are.
//
@@ -507,14 +508,16 @@ int main(int argc, char **argv) {
const bool backwards = v == SideCheck::Swapped; // relative to the names as they are now
const double after = (now - start) / 1e9;
const std::string ev = side_check.json(), text = side_check.summary();
if (sides_mode != "auto") { // forced: only say so
if (sides_mode != "auto") { // forced: the names stay; if the hands disagree, they're the truth
const std::string what = (sides_from == "option" ? "--sides " : "HANDS_SWAP_SIDES=") + sides_mode;
if (backwards)
std::printf("side cameras: %s looks WRONG: the hands say the side cameras are the other way round (%s). "
"Use auto.\n", what.c_str(), text.c_str());
"Tracking keeps the forced names; recordings are labelled by the hands. Use auto.\n",
what.c_str(), text.c_str());
else
std::printf("side cameras: %s agrees with the hands (%s)\n", what.c_str(), text.c_str());
sides_state = backwards ? "forced, disagrees" : "forced, agrees";
if (backwards) truth = !names_swapped, decided_by = "auto", decided_after_s = after;
decision_evidence = ev;
checking = false;
} else if (side_round == 0 || backwards) {