Hands: ft-hands works out which side camera is which

ft-camd tells the side cameras apart by XRService's buffer allocation order, which some XRService starts reverse; both of 2026-10-02's starts did, so the cutouts missed the hands. HANDS_SWAP_SIDES=auto (the default) has ft-hands vote from hands seen in both side cameras: the landmark rays meet in front of both cameras only under the right naming. While undecided it probes the exchanged naming with the landmark model. It decides in about 2 s of hands (right on all 7 recordings replayed), swaps the views in place, and publishes sides.json. 0 and 1 still force it, with a warning when the hands disagree. Recordings carry each part's naming and the session's decision; review, export, validate and ft-handreplay put the names right, and takes.py sides records a decision by hand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 02:00:06 +02:00 · 2026-10-02 22:24:13 -06:00
1 parent 97b7fc837c
commit a533bb973a
@@ -1,6 +1,7 @@
 # Hand tracking, built into build/ (hands/build.sh runs this in the dev container):
 #   make          ft-camd (camd/: runs on the host, so linked statically) and ft-hands (track/)
 #   make tools    ft-handreplay and ft-ringplay, for recordings
+#   make check    builds and runs the C++ unit tests (tests/sides_test.cpp)
 # The first build fetches ncnn (NCNN_TAG) and builds it into build/ncnn, which takes a few
 # minutes. NCNN=DIR uses an ncnn install already built instead.
 NCNN_TAG  = 20260526
@@ -11,7 +12,7 @@ CXXFLAGS += -std=c++17 -fopenmp -I$(NCNN)/include/ncnn
 LDLIBS    = $(NCNN)/lib/libncnn.a -ljsoncpp -fopenmp -lpthread

 CAMD  = camd/camd.c camd/tp.c camd/xrcams.c
-TRACK = track/calib.cpp track/nets.cpp track/tracker.cpp track/io.cpp track/record.cpp track/pinch.cpp
+TRACK = track/calib.cpp track/nets.cpp track/tracker.cpp track/io.cpp track/record.cpp track/pinch.cpp track/sides.cpp
 HDR   = $(wildcard track/*.h) camd/fhring.h include/fh_hands.h include/fh_gestures.h

 all: build/ft-camd build/ft-hands
@@ -29,6 +30,13 @@ build/ft-handreplay: track/replay.cpp $(TRACK) $(HDR) $(NCNN)/lib/libncnn.a
 	@mkdir -p build
 	$(CXX) $(CXXFLAGS) -o $@ track/replay.cpp $(TRACK) $(LDLIBS)

+build/sides-test: tests/sides_test.cpp $(TRACK) $(HDR) $(NCNN)/lib/libncnn.a
+	@mkdir -p build
+	$(CXX) $(CXXFLAGS) -o $@ tests/sides_test.cpp $(TRACK) $(LDLIBS)
+
+check: build/sides-test
+	build/sides-test
+
 build/ft-ringplay: track/ringplay.cpp track/record.h camd/fhring.h
 	@mkdir -p build
 	$(CXX) $(CXXFLAGS) -o $@ track/ringplay.cpp
@@ -44,6 +52,6 @@ build/ncnn/install/lib/libncnn.a:
 	cmake --build build/ncnn/build --target install > build/ncnn/build.log

 clean:
-	rm -f build/ft-camd build/ft-hands build/ft-handreplay build/ft-ringplay
+	rm -f build/ft-camd build/ft-hands build/ft-handreplay build/ft-ringplay build/sides-test

-.PHONY: all tools clean
+.PHONY: all tools check clean
@@ -36,7 +36,7 @@ hands/run.sh uninstall

 Settings in `~/.config/frametop.conf` (`FT_<name>` in the environment overrides them), read when ft-camd and ft-hands start:

- `HANDS_SWAP_SIDES=1`: the two side cameras' names are swapped (see ft-camd below). Check with `tools/check_sides.py --ring`.
+- `HANDS_SWAP_SIDES=auto` (the default): ft-hands tells from the hands which side camera is which, and corrects ft-camd's names when they're backwards (see "Which camera is which" below). `1` forces them exchanged and `0` forces ft-camd's names; ft-hands still checks and logs a warning if the hands disagree.
 - `HANDS_CPUS=5,6,7`: the CPUs the model threads run on (below).
 - `HANDS_CAMERAS` (`auto`), `HANDS_BRIGHT` (`all`), `HANDS_BRIGHT_ON` (40), `HANDS_BRIGHT_OFF` (25): which cameras ft-hands tracks with, as `--cams`, `--bright`, `--bright-on` and `--bright-off` (see ft-hands). `HANDS_CAMERAS=mono` also keeps ft-camd off the colour cameras.
 - `HANDS_COLOR_LEFT` (`color_video0`), `HANDS_COLOR_CROP` (`subtract`): how the colour module's calibration maps onto its images, as `--color-left` and `--color-crop`.
@@ -86,7 +86,14 @@ Options:

 It exits when XRService exits, or when a camera's buffers keep going stale, which means XRService has reallocated them. The service starts it again, and it attaches to the new buffers.

-**Which camera is which:** video9 is `slam_left`, video13 is `slam_right`, video6 is `upper_left` and video7 is `upper_right`. This was checked by rendering the same view from each camera with the factory calibration. But ft-camd tells the side cameras' buffers apart only by XRService's allocation order, and after some XRService restarts it gets them backwards. Then every hand is seen by one camera only, at the wrong depth, and the hand holes land beside the hands. With the headset on, looking at a room with some texture, `tools/check_sides.py --ring` says whether the names are right (exit 0), swapped (exit 3), or it can't tell (exit 2). When they're swapped, set `HANDS_SWAP_SIDES=1`. The colour cameras are video3 (`arcimx616 0-0010`) and video0 (`0-001a`); which of them is `passthrough_left` in the module's calibration is for `tools/check_color.py` to settle, on a recording with texture in view.
+**Which camera is which:** video9 is `slam_left`, video13 is `slam_right`, video6 is `upper_left` and video7 is `upper_right`. This was checked by rendering the same view from each camera with the factory calibration. But ft-camd tells the side cameras' buffers apart only by XRService's allocation order, and after some XRService restarts it gets them backwards. Then every hand is seen by one camera only, at the wrong depth, and the hand holes land beside the hands. ft-hands now catches this by itself (`track/sides.h`, `HANDS_SWAP_SIDES=auto`):
+- Whenever a hand's landmarks are found in two cameras at once (one of them a side camera), it intersects the rays through the 21 landmarks twice: once with the calibrations as named, once with the two side cameras exchanged. The same hand seen the right way meets within a few mm, in front of both cameras and as far away as its size says. The wrong way misses by centimetres or meets behind a camera.
+- With the names wrong, the tracker never gets such pairs on its own: it hands the hand over to where the wrong calibration puts it and finds nothing there. So 5 times a second while undecided, the check places a tracked hand in 3D under the other naming and runs the landmark model where that puts it in the other side camera.
+- It decides after 10 votes one way and none the other, or 20 with at most a fifth the other way, over at least 1 s. That takes about 1-2 s of hands in view. If the names are backwards, it exchanges them; the tracked views move with their images. Then it checks once more, more strictly.
+- The log says what it found (`side cameras: SWAPPED, now exchanged after 3.2 s (votes ...)`). So does `/run/user/UID/frametop-hands/sides.json`, which the hand recorder reads. Recordings get a `DIR/sides.json` (hands/rec/sides.py has the rules).
+- `--record-only` can't tell (it tracks nothing): it records ft-camd's names unless `--sides 0|1` says otherwise.
+
+`tools/check_sides.py --ring` is the independent check from the scene (ORB matches meeting under each naming): exit 0 as named, 3 swapped, 2 can't tell. `--pair upper` checks the upper pair the same way: in every recording so far (3 XRService starts, both side namings) the upper pair was named right. The colour cameras are video3 (`arcimx616 0-0010`) and video0 (`0-001a`); which of them is `passthrough_left` in the module's calibration is for `tools/check_color.py` to settle, on a recording with texture in view.

 ## ft-hands

@@ -102,7 +109,7 @@ Options:
 - `--threads N`: model threads, pinned to the `--cpus` list. Default 3.
 - `--cpus LIST`: CPUs for the model threads and the main loop. Default `5,6,7` (`HANDS_CPUS`). SteamOS starts user processes on CPUs 0-4, and XRService's head tracking runs on 2-3. With the headset on, a step took 8.4 ms on 5-7 against 13.2 ms on 2-4, and SteamVR's frame timing didn't change (2026-09-29, three rounds of the same replayed frames).
 - `--contrast MODE` or `PALM/HAND`: how crops are equalized before the models see them: `clahe[:CLIP]`, `none`, or `stretch` (1st-99th percentile). Default `clahe:2/none`. In the dim recording, CLAHE let the palm search find about 10% more hands, but it made the landmarks jitter more (published median 6.9 mm, against 6.0 mm with plain landmark crops).
- `--swap-sides`: swap the two side cameras (`HANDS_SWAP_SIDES`, see ft-camd).
+- `--sides auto|0|1`: which side camera is which (`HANDS_SWAP_SIDES`, see "Which camera is which"); `--swap-sides` is `--sides 1`.
 - `--seconds N`: stop after N seconds.
 - `--status S`: how often to print status, in seconds.
 - `--models DIR`: where the models are.
@@ -197,7 +204,7 @@ hands/build/ft-handreplay ~/.local/share/frametop/hands/rec-20260929-120000 --co

 Python, with NumPy and OpenCV. `setup/dev-container.sh` doesn't install them, because Fedora's `python3-opencv` pulls in over a gigabyte; in the dev container, run `sudo dnf install python3-numpy python3-opencv` once. Off the Frame, `FRAME_JOB_DEVICE_ROOT` can point at a folder with copies of the headset's calibration files.

- `tools/check_sides.py --ring` (or a recording): are the side cameras named right?
+- `tools/check_sides.py --ring` (or a recording, a sets file, `--pair upper`, `--calib DIR`, `--json`): are the side cameras named right, from the scene? ft-handreplay's `--sides file|0|1|auto` replays with DIR/sides.json's names (the default), as recorded, exchanged, or as auto decides, and reports what the side check found and when.
 - `tools/check_color.py REC`: how the colour module's calibration maps onto its images.
 - `tools/show_set.py REC`: a recording's frame sets as images.
 - `tools/watch_gestures.py [--distance]`: pinches and grips, live.
@@ -256,7 +263,7 @@ Guesses, not tested:

 ## Known issues

- **The side cameras can come out swapped.** ft-camd tells the side cameras' buffers apart only by XRService's allocation order, and some XRService restarts reverse it. For now it's caught by hand: `tools/check_sides.py --ring`, then `HANDS_SWAP_SIDES=1`. It needs a fix in ft-camd, or at least an automatic check when it starts.
+- **The side cameras can come out swapped.** ft-camd tells the side cameras' buffers apart only by XRService's allocation order, and some XRService restarts reverse it. ft-hands corrects it from the hands (`HANDS_SWAP_SIDES=auto`, the default). Until it has seen about 1-2 s of hands in both namings' reach, the cutouts may sit beside the hands. ft-camd itself still can't tell.
 - **The colour cameras can't be used while the headset is worn.** The colour module then writes only a half-size image into the top-left quarter of its buffers, and ft-camd drops those frames. So the service runs the mono cameras only, and tracking in bright light, where the mono cameras see dark hands, doesn't get the colour pair's help.
 - **The colour calibration mapping isn't settled.** Which colour camera is `passthrough_left` (`HANDS_COLOR_LEFT`) and how the module's crop applies (`HANDS_COLOR_CROP`) still need `tools/check_color.py` on a recording with a lit, textured view.
 - **Depth when one camera loses the hand.** A hand seen in one camera drifts 10% per update toward the one-camera depth guess (`kMonoDepthGain`, 0.1, in `track/tracker.cpp`). In the 2026-09-30 replays that was worse than keeping the last distance (see "3D" above). A smaller gain, such as 0.02, is the next thing to try.
@@ -26,8 +26,9 @@ Every part runs in the dev container, as ft-hands and Input Settings do. The hos

 - **ft-camd** publishes the camera ring. If it isn't running, the session starts it as the transient user unit `frametop-handrec-camd.service`, the way `hands/ft-cutouts` starts `frametop-cutouts-camd.service` (needs `hands/build/ft-camd` with capabilities: `hands/run.sh caps`). An ft-camd already running from ft-cutouts or ft-handsctl is used as it is.
 - **A tracking ft-hands** gives feedback through the hands file: which hands are seen, the palm's distance, the index tip. If none is running, the session starts `ft-hands --no-gestures --status 0` (unit `frametop-handrec-hands.service`). An ft-hands already running is used as it is.
- **A recording ft-hands** runs once per recording part: `ft-hands --record-only --record DIR --record-for SECONDS --record-hz 10 --status 0`. It runs as a plain child process of the session, ended with SIGTERM when the part ends. SIGTERM ends ft-hands' loop, and `Recorder` writes out its queue when it's destroyed. In step mode (below) a part is one step's countdown and hold, so a take has one part per step (about 40 in the hand poses); in auto mode a take is one part, plus one more after each pause. `--record-for` is only a safety net.
+- **A recording ft-hands** runs once per recording part: `ft-hands --record-only --record DIR --record-for SECONDS --record-hz 10 --status 0 --sides auto|0|1` (below, "Side cameras"). It runs as a plain child process of the session, ended with SIGTERM when the part ends. SIGTERM ends ft-hands' loop, and `Recorder` writes out its queue when it's destroyed. In step mode (below) a part is one step's countdown and hold, so a take has one part per step (about 40 in the hand poses); in auto mode a take is one part, plus one more after each pause. `--record-for` is only a safety net.
  - Why a process per part rather than one kept alive and paused: measured in the dev container with `ft-ringplay`'s ring (2026-10-02), `ft-hands --record-only` writes its first set 16-27 ms after it starts and ends 4-6 ms after SIGTERM, so a new part costs nothing the 3 s countdown doesn't cover. Every reader already takes parts in order (`takes.py`, `validate.py` through the export's single stream, the labeller's `fhl_io.py`, numbering `sets-10.bin` after `sets-9.bin`), ft-hands needs no new control, and nothing is written while a step waits. Before the hold starts the session checks that the part has written a set (`Recorder.has_data`, up to 3 s more), so the hold is recorded from its first frame.
+- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand.
 - **ft-handpanel** runs as a child process with `--watch-stdin`. It shows the panel and logs poses during each take.
 - **The headset button's reader** is a thread of the session (`ButtonReader`, below), not a process.

@@ -55,7 +56,8 @@ Test hooks:
      prompts.jsonl             what the person was asked to do, when (below)
      poses.jsonl               head and controller poses from ft-handpanel (below)
      take.json                 {"section", "title", "started_ns", "ended_ns", "status": "complete"|"stopped"|"skipped",
-                                 "deleted": [[from_ns, to_ns], ...], "notes": ""}
+                                 "deleted": [[from_ns, to_ns], ...], "notes": "",
+                                 "parts": {"sets.bin": {"names_swapped": false}, "sets-2.bin": {...}}}
  exports/<session>/            what export writes (below)
 ```

@@ -80,9 +82,14 @@ No name, email, or account. The contributor id is random, so several sessions fr
 "device": {"steamos": "<VERSION_ID from /etc/os-release>", "steamvr": "<version if known>", "cameras": [{"name", "width", "height"}]},
 "mode": "step|auto", "quick": false, "shuffle": {"seed": 123, "sweeps": {"pose-sweeps": [{"id", "hands", "cues"}]}},
 "takes": ["01-hand-size", "..."],
- "camera": {"status": "ok|unknown|degraded", "reason": "..."}, "stop_reason": "..."}
+ "camera": {"status": "ok|unknown|degraded", "reason": "..."}, "stop_reason": "...",
+ "sides": {"swapped": true|false|null, "decided_by": "auto|config|option|manual", "state": "decided|confirmed|...",
+           "evidence": {"as_named": 0, "swapped": 10, "seconds": 1.8, "miss_mm": [-1, 4.2], "probes": 30, "found": 10},
+           "decided_at": "<ISO>", "decided_ns": 0}}
 ```

+`sides`: whether ft-camd's side camera names were backwards during the session (`swapped`), as the live tracker decided it (above, "Side cameras"); `null` while nobody knows. A part's names are right when its take.json `names_swapped` equals `swapped`. Parts without a `parts` entry were recorded with ft-camd's names. Sessions from before this have no `sides`.
+
 `camera` is the camera check's verdict at the start (no paths or log lines: those go to `session.log`). `stop_reason` is there when a session was stopped at the no-hands screen, with the check's result.

 A session started before step mode existed has no `mode`: it ran as `auto`. `quick` and `shuffle` (below, "Sweeps" and "Quick round") are missing from sessions before the sweeps.
@@ -300,6 +307,7 @@ On 2026-10-02 a whole session showed "I can't see your hands": after the headset
  - `manifest.json`: profile fields except `optional.notes` unless kept, session.json (without its `uploads` records), takes, schema, tool version, the consent version.
  - `calibration.json` and `device.json`, when the session has them.
  - Per take: `prompts.jsonl`, `poses.jsonl`, `take.json`, and `sets.bin.zst` (sets in deleted ranges removed, then zstd -10 with 2 threads).
+  - The side cameras are named right in every exported set when the session's `sides.swapped` is known: parts that need it get slam_left and slam_right (and their `_dk`) exchanged in the set headers as they're compressed. The exported take.json says `"parts": {"sets.bin": {"names_swapped": <swapped>}}`, and the manifest's take entry `"sides": {"names_swapped", "renamed_sets"}`. Unknown, the names stay as recorded.
  - `SHA256SUMS`.

  Compression runs at nice 19. Before starting, the window warns if the headset is worn, judged the way `frame-job` does: `vrcompositor` runs and a `/sys/class/backlight/*/brightness` reads over 0 (SteamVR turns the panel off 5 s after the headset comes off). CPU work while in VR causes stutter. The proximity sensor is no use here: it read 9-43 with the headset sitting unworn.
@@ -313,6 +321,7 @@ On 2026-10-02 a whole session showed "I can't see your hands": after the headset
    - `device.json` holds only `cv.cad_from_cal` and `head`, each `plus_x`, `plus_z` and `position` as 3 numbers (plus `cad_from_cal`'s `method`). Without it: a warning.
    - Each `sets.bin.zst` decompresses to its end, so a truncated one fails, and every set's FHSET01 header is sane: camera names, sizes, record length. Set counts and raw bytes match the manifest. Pixels aren't decoded.
    - Every jsonl line parses.
+    - `session.sides` is `{"swapped": true|false|null}`, and every take's `sides.names_swapped` equals it (else an error: export again). Without `sides`, or `null`: a warning (the maintainer's check tells).
    - The total size: a warning over 15 GB, an error over 40 GB.

    Warnings cover notes kept in the export, a home folder path in the manifest, and missing `poses.jsonl` files.
@@ -49,6 +49,8 @@ REPO = os.path.dirname(os.path.dirname(HERE))
 HANDS = os.path.join(REPO, "hands")
 sys.path.insert(0, HANDS)
 import camcheck  # noqa: E402  (hands/camcheck.py: are all four mono cameras running?)
+sys.path.insert(0, HERE)
+import sides  # noqa: E402  (hands/rec/sides.py: which side camera is which)
 FT_HANDS = os.path.join(HANDS, "build", "ft-hands")
 FT_CAMD = os.path.join(HANDS, "build", "ft-camd")
 PANEL_BIN = os.path.join(HERE, "build", "ft-handpanel")
@@ -60,6 +62,7 @@ CAMD_UNIT = "frametop-handrec-camd.service"
 HANDS_UNIT = "frametop-handrec-hands.service"

 RECORD_HZ = 10
+SIDES_READ_S = 0.5       # how often the tracking ft-hands' side camera decision is read
 TICK_S = 0.05            # the session loop's step (real time)
 FRESH_S = 0.3            # the hands file counts as live if published this recently
 LOST_S = 1.5             # an asked-for hand lost this long gets a note
@@ -492,11 +495,16 @@ class Recorder:
    writes TAKE/sets.bin; part N (after a pause) records into TAKE/.part-N and is moved to
    TAKE/sets-N.bin when it ends (ft-hands never overwrites, and always writes DIR/sets.bin)."""

-    def __init__(self, take_dir, part, seconds, ring, log_file):
+    def __init__(self, take_dir, part, seconds, ring, log_file, swap=None):
+        """swap: the side cameras' decision (sides.py), passed on as --sides 1 or 0; None (not
+        known yet) records them as ft-camd names them (--sides auto, whatever the config says)."""
        self.take_dir, self.part = take_dir, part
        self.dir = take_dir if part == 1 else os.path.join(take_dir, ".part-%d" % part)
+        self.file = "sets.bin" if part == 1 else "sets-%d.bin" % part
+        self.names_swapped = bool(swap)
        argv = [FT_HANDS, "--record-only", "--record", self.dir, "--record-for", "%.0f" % max(seconds, 5),
-                "--record-hz", str(RECORD_HZ), "--status", "0"]
+                "--record-hz", str(RECORD_HZ), "--status", "0",
+                "--sides", "auto" if swap is None else "1" if swap else "0"]
        if ring:
            argv += ["--ring", ring]
        if not in_container():
@@ -515,7 +523,9 @@ class Recorder:
            return False

    def stop(self):
-        """End it (SIGTERM: ft-hands writes out its queue) and put the part in place."""
+        """End it (SIGTERM: ft-hands writes out its queue) and put the part in place. Afterwards
+        names_swapped is what ft-hands says it applied (its DIR/sides.json, which goes into
+        take.json's "parts" and is removed here)."""
        if self.proc.poll() is None:
            self.proc.send_signal(signal.SIGTERM)
            try:
@@ -523,6 +533,15 @@ class Recorder:
            except subprocess.TimeoutExpired:
                self.proc.kill()
                self.proc.wait()
+        side_file = os.path.join(self.dir, "sides.json")
+        try:
+            with open(side_file) as f:
+                runs = json.load(f).get("names_swapped") or []
+            if runs:
+                self.names_swapped = bool(runs[0][1])   # --record-only never changes it
+            os.remove(side_file)
+        except (OSError, ValueError, TypeError, IndexError):
+            pass
        if self.part > 1:
            src = os.path.join(self.dir, "sets.bin")
            if os.path.exists(src):
@@ -1130,6 +1149,8 @@ class Session:
        self._redo_ok = False      # R does something now
        self._step_t0 = None       # the step's first event (ready or prompt), for R
        self._fb = {}              # feedback timers
+        self._ring_path = None
+        self._sides_read = 0.0     # when the side camera decision was last read

    # --- controls (any thread)
    def start(self):
@@ -1287,6 +1308,7 @@ class Session:

    def _setup(self):
        ring_path = self.ring or os.path.join(self.hands_dir, "cam-ring")
+        self._ring_path = ring_path
        if not self.dry_run:
            if not os.access(FT_HANDS, os.X_OK):
                raise _Fail("ft-hands isn't built: hands/build.sh")
@@ -1317,7 +1339,8 @@ class Session:
                       "skipped": self.skipped},
            "mode": "auto" if self.auto else "step", "quick": self.quick,
            "shuffle": {"seed": self.seed, "sweeps": plan_record(self.plan)},
-            "takes": [], "status": "recording"}
+            "takes": [], "status": "recording", "sides": {"swapped": None}}
+        self._read_sides(force=True)
        cam = self._status.get("camera")
        if cam:
            self._session_json["camera"] = {"status": cam.get("status"), "reason": cam.get("reason", "")}
@@ -1386,6 +1409,40 @@ class Session:
        if self._session_json is not None:
            write_json(os.path.join(self.session_dir, "session.json"), self._session_json)

+    def _read_sides(self, force=False):
+        """The tracking ft-hands' side camera decision (sides.py, read_live) into session.json's
+        "sides": {"swapped", "decided_by", "state", "evidence", "decided_at"}. Later recording
+        parts are named right (Recorder's swap); parts before it are renamed when read. Without
+        a tracking ft-hands it stays undecided ("swapped": null): export then leaves the names,
+        and the maintainer's check (hub_review check) tells."""
+        now = time.monotonic()
+        if self._session_json is None or (not force and now - self._sides_read < SIDES_READ_S):
+            return
+        self._sides_read = now
+        live = sides.read_live(os.path.join(self.hands_dir, "sides.json"), self._ring_path)
+        if not live or live.get("swapped") is None:
+            return
+        cur = self._session_json.get("sides") or {}
+        swapped = bool(live["swapped"])
+        if cur.get("swapped") is not None and bool(cur["swapped"]) == swapped:
+            if live.get("state") != cur.get("state"):   # e.g. decided -> confirmed
+                cur["state"] = live.get("state")
+                self._save_session()
+            return
+        new = {"swapped": swapped, "decided_by": live.get("decided_by"), "state": live.get("state"),
+               "evidence": live.get("evidence"), "decided_at": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
+               "decided_ns": mono_ns()}
+        if cur.get("swapped") is not None:
+            new["reversed_from"] = cur
+        self._session_json["sides"] = new
+        self._save_session()
+        self._log("side cameras: %s (%s, %s)" % ("SWAPPED" if swapped else "as named", new["decided_by"],
+                                                  json.dumps(new["evidence"])))
+
+    def _sides_swapped(self):
+        """session.json's decision: True, False, or None (not known yet)."""
+        return (self._session_json or {}).get("sides", {}).get("swapped")
+
    def _ensure_ring(self, path):
        def alive():
            try:
@@ -1632,6 +1689,8 @@ class Session:
            fb["disk"] = now
            if shutil.disk_usage(self.session_dir).free < MIN_FREE:
                raise _Fail("The disk is nearly full: the session stopped")
+        if not self.dry_run:
+            self._read_sides()
        live = self._hands_file.read() if self._read_hands else None
        self._live = live
        w = self._hold_watch
@@ -1890,7 +1949,8 @@ class Session:
        if not self.dry_run:
            # a safety net only: the session ends the recording itself
            remaining = section_seconds(t["section"], worst=True, auto=self.auto) / self.speed
-            self._recorder = Recorder(t["dir"], t["part"], remaining * 1.5 + 60, self.ring, self._log_file)
+            self._recorder = Recorder(t["dir"], t["part"], remaining * 1.5 + 60, self.ring, self._log_file,
+                                      swap=self._sides_swapped())
        self._recording = True
        self._panel.cmd("poses start " + os.path.join(t["dir"], "poses.jsonl"))
        self._panel.set("rec", "rec on")
@@ -1905,6 +1965,9 @@ class Session:
            rec, self._recorder = self._recorder, None
            code = rec.stop()
            self._log("recording part %d ended (%s)" % (rec.part, code))
+            if self._take:   # how this part's side cameras are named (sides.py)
+                self._take["json"].setdefault("parts", {})[rec.file] = {"names_swapped": rec.names_swapped}
+                write_json(os.path.join(self._take["dir"], "take.json"), self._take["json"])

    def _end_take(self, status):
        t = self._take
@@ -0,0 +1,128 @@
+"""Which side camera is which, in recordings: the rules every reader shares.
+
+ft-camd tells slam_left's buffers from slam_right's by the order XRService allocated them, and
+some XRService starts reverse it: then each side camera's images carry the other's name. A
+tracking ft-hands tells from the hands (hands/track/sides.h, HANDS_SWAP_SIDES=auto) and publishes
+what it found in /run/user/UID/frametop-hands/sides.json (read_live). Two things are recorded:
+
+  swapped         whether ft-camd's naming was backwards during the recording (the truth);
+                  None while nobody knows. A hand recorder session has it in session.json
+                  "sides": {"swapped", "decided_by", "evidence", ...}; an ft-hands --record
+                  recording in DIR/sides.json.
+  names_swapped   per recording file: whether the side cameras' names in it were exchanged from
+                  ft-camd's (ft-hands --sides 1). A session's take.json "parts": {"sets.bin":
+                  {"names_swapped": false}, "sets-2.bin": {...}}; DIR/sides.json "names_swapped":
+                  [[first set, bool], ...] (it can change inside a recording when ft-hands decides).
+
+A file's names are right when names_swapped == swapped. Readers rename slam_left <-> slam_right
+(and their _dk dark frames) where they differ (needs_rename). Exports always carry the right
+names, with names_swapped = swapped written next to them, so applying the rule again is harmless.
+
+Standard library only (the window, validate.py on Windows, the maintainer's tools).
+"""
+import json
+import os
+import struct
+import time
+
+HDR = struct.Struct("<8sII")       # fh_set_hdr_t (hands/track/record.h)
+CAM = struct.Struct("<16sIIQQ")    # fh_set_cam_t: name, width, height, capture_ns, dqbuf_ns
+SIDES = {"slam_left": "slam_right", "slam_right": "slam_left",
+         "slam_left_dk": "slam_right_dk", "slam_right_dk": "slam_left_dk"}
+LIVE_FRESH_S = 6.0   # ft-hands rewrites its sides.json at least every 2 s
+
+
+def other_name(name):
+    """slam_left <-> slam_right (and their dark frames); other names as they are."""
+    return SIDES.get(name, name)
+
+
+def needs_rename(swapped, names_swapped):
+    """True when a file's side cameras carry each other's names. Unknown truth: leave them."""
+    if swapped is None:
+        return False
+    return bool(names_swapped) != bool(swapped)
+
+
+def rename_record(record):
+    """One FHSET01 record (bytes) with slam_left and slam_right's names exchanged (and their
+    dark frames'). Only the 16-byte name fields change; pixels and lengths stay."""
+    out = bytearray(record)
+    ncams = HDR.unpack_from(out, 0)[1]
+    for k in range(ncams):
+        off = HDR.size + k * CAM.size
+        raw = bytes(out[off:off + 16])
+        name = raw.split(b"\0", 1)[0].decode(errors="replace")
+        new = other_name(name)
+        if new != name:
+            out[off:off + 16] = new.encode().ljust(16, b"\0")
+    return bytes(out)
+
+
+# ------------------------------------------------------------------ sessions and recordings
+
+def session_sides(session_json):
+    """A session.json's "sides" (a dict), or {"swapped": None} when it has none."""
+    s = (session_json or {}).get("sides")
+    return s if isinstance(s, dict) else {"swapped": None}
+
+
+def part_names_swapped(take_json, part):
+    """Whether a take's part file (e.g. "sets-2.bin") was recorded with exchanged names."""
+    parts = (take_json or {}).get("parts")
+    entry = parts.get(part) if isinstance(parts, dict) else None
+    return bool(entry.get("names_swapped")) if isinstance(entry, dict) else False
+
+
+def part_rename(session_json, take_json, part):
+    """Should a session take's part file have its side cameras renamed when read?"""
+    return needs_rename(session_sides(session_json).get("swapped"), part_names_swapped(take_json, part))
+
+
+def recording_runs(rec_dir):
+    """An ft-hands --record recording (DIR/sets.bin + DIR/sides.json): [(first set, rename)],
+    or [(0, False)] when it has no sides.json (recorded before there was one)."""
+    try:
+        with open(os.path.join(rec_dir, "sides.json")) as f:
+            s = json.load(f)
+    except (OSError, ValueError):
+        return [(0, False)]
+    runs = s.get("names_swapped") or [[0, False]]
+    return [(int(first), needs_rename(s.get("swapped"), sw)) for first, sw in runs]
+
+
+def rename_at(runs, i):
+    """recording_runs()'s answer for set i."""
+    out = False
+    for first, rename in runs:
+        if i >= first:
+            out = rename
+    return out
+
+
+# ------------------------------------------------------------------ the live decision
+
+def run_dir():
+    return "/run/user/%d/frametop-hands" % os.getuid()
+
+
+def read_live(path=None, ring_path=None, now_ns=None):
+    """The tracking ft-hands' sides.json, or None if there's none or it's stale (ft-hands
+    rewrites it every 2 s and removes it when it exits). With ring_path, also None when it
+    was written for another ft-camd run than the ring there now."""
+    path = path or os.path.join(run_dir(), "sides.json")
+    try:
+        with open(path) as f:
+            s = json.load(f)
+    except (OSError, ValueError):
+        return None
+    now = time.clock_gettime_ns(time.CLOCK_MONOTONIC) if now_ns is None else now_ns
+    if not isinstance(s, dict) or (now - int(s.get("updated_ns", 0))) / 1e9 > LIVE_FRESH_S:
+        return None
+    if ring_path:
+        try:
+            if os.stat(ring_path).st_ino != s.get("ring_ino"):
+                return None
+        except OSError:
+            return None
+    return s
@@ -13,7 +13,15 @@ A take's recording is one or more FHSET01 files (hands/track/record.h): per set
 Deleted ranges live in take.json ("deleted": [[from_ns, to_ns], ...], CLOCK_MONOTONIC, the
 clock of dqbuf_ns); the files keep every set until export leaves them out.

-usage: takes.py [--base DIR] list | takes SESSION | export SESSION
+Side cameras (sides.py): a part recorded before the live tracker decided which side camera is
+which may carry slam_left's and slam_right's names the wrong way round. TakeIndex renames them as
+it reads (session.json "sides" against take.json "parts"), so review and export see the right
+names, and export writes them so: an export's files are always named right, its take.json says
+"parts": {"sets.bin": {"names_swapped": <the session's swapped>}}, and its manifest has each
+take's "sides". `takes.py sides SESSION` shows the decision; --set swapped|named records one by
+hand (e.g. from tools/check_sides.py on a session no live tracker decided).
+
+usage: takes.py [--base DIR] list | takes SESSION | export SESSION | sides SESSION [--set swapped|named]
 """
 import argparse
 import datetime
@@ -24,8 +32,12 @@ import re
 import shutil
 import struct
 import subprocess
+import sys
 import threading

+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import sides  # noqa: E402  (hands/rec/sides.py, next to this file)
+
 DEFAULT_BASE = os.path.expanduser("~/.local/share/frametop/hands/contrib")
 REPO = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
 MAGIC = b"FHSET01"
@@ -136,7 +148,14 @@ class TakeIndex:
        self.parts = self._parts(take_dir)
        self.sets = []   # (part, offset, bytes, time_ns): time_ns is the cameras' earliest dqbuf_ns
        self.part_starts = []   # index of each part's first set
-        self.cams = []   # the first set's [{name, width, height}]
+        self.cams = []   # the first set's [{name, width, height}], side cameras named right
+        # per part: are its side cameras named the wrong way round (sides.py)? read_set and
+        # read_raw rename them.
+        session = read_json(os.path.join(os.path.dirname(os.path.dirname(take_dir)), "session.json"))
+        take = read_json(os.path.join(take_dir, "take.json"))
+        self.swapped = sides.session_sides(session).get("swapped")
+        self.names_swapped = [sides.part_names_swapped(take, os.path.basename(p)) for p in self.parts]
+        self.rename = [sides.needs_rename(self.swapped, n) for n in self.names_swapped]
        for p, path in enumerate(self.parts):
            self.part_starts.append(len(self.sets))
            self._index(p, path)
@@ -176,11 +195,18 @@ class TakeIndex:
                    break
                fields = [CAM.unpack_from(cams, i * CAM.size) for i in range(ncams)]
                if not self.cams:
-                    self.cams = [{"name": n.split(b"\0", 1)[0].decode(errors="replace"), "width": w, "height": h}
-                                 for n, w, h, _, _ in fields]
+                    self.cams = [{"name": self._name(p, n), "width": w, "height": h} for n, w, h, _, _ in fields]
                self.sets.append((p, off, nbytes, min(c[4] for c in fields)))
                off += nbytes

+    def _name(self, p, raw):
+        name = raw.split(b"\0", 1)[0].decode(errors="replace")
+        return sides.other_name(name) if self.rename[p] else name
+
+    def renamed_sets(self, keep=None):
+        """How many of the sets (all, or the indices in keep) read renamed."""
+        return sum(1 for i in (range(len(self.sets)) if keep is None else keep) if self.rename[self.sets[i][0]])
+
    def __len__(self):
        return len(self.sets)

@@ -211,7 +237,7 @@ class TakeIndex:
            px_off = off + HDR.size + CAM.size * ncams
            for k in range(ncams):
                name, w, h, capture_ns, dqbuf_ns = CAM.unpack_from(heads, k * CAM.size)
-                name = name.split(b"\0", 1)[0].decode(errors="replace")
+                name = self._name(p, name)
                if only is None or name == only:
                    f.seek(px_off)
                    out.append({"name": name, "width": w, "height": h, "capture_ns": capture_ns,
@@ -220,11 +246,12 @@ class TakeIndex:
            return out

    def read_raw(self, i):
-        """Set i's whole record, as it is in the file."""
+        """Set i's whole record, as it is in the file but with the side cameras named right."""
        p, off, nbytes, _ = self.sets[i]
        with open(self.parts[p], "rb") as f:
            f.seek(off)
-            return f.read(nbytes)
+            record = f.read(nbytes)
+        return sides.rename_record(record) if self.rename[p] else record


 _index_cache = {}
@@ -232,9 +259,11 @@ _index_lock = threading.Lock()


 def take_index(take_dir):
-    """A TakeIndex, cached until one of the take's recording files changes size or time."""
+    """A TakeIndex, cached until one of the take's recording files, its take.json or its
+    session.json (the side cameras' decision) changes size or time."""
    key_parts = []
-    for path in TakeIndex._parts(take_dir):
+    meta = [os.path.join(take_dir, "take.json"), os.path.join(os.path.dirname(os.path.dirname(take_dir)), "session.json")]
+    for path in TakeIndex._parts(take_dir) + meta:
        try:
            st = os.stat(path)
            key_parts.append((path, st.st_size, st.st_mtime_ns))
@@ -376,6 +405,29 @@ class Store:
            shutil.rmtree(path)

    # --- export
+    def sides(self, session, swapped=None):
+        """session.json's side camera decision (sides.py) plus each take's parts; with swapped
+        (True/False), record that by hand first ("decided_by": "manual"). No set is rewritten:
+        reading and export rename."""
+        path = os.path.join(self.session_dir(session), "session.json")
+        meta = read_json(path)
+        if not meta:
+            raise RuntimeError(f"no session {session}")
+        if swapped is not None:
+            old = sides.session_sides(meta)
+            new = {"swapped": bool(swapped), "decided_by": "manual",
+                   "decided_at": datetime.datetime.now().astimezone().isoformat(timespec="seconds")}
+            if old.get("swapped") is not None:
+                new["replaced"] = old
+            meta["sides"] = new
+            write_json(path, meta)
+        out = {"sides": sides.session_sides(meta), "takes": {}}
+        for tid in self.take_ids(session):
+            index = take_index(self.take_dir(session, tid))
+            out["takes"][tid] = {os.path.basename(p): {"names_swapped": n, "renamed_when_read": r}
+                                 for p, n, r in zip(index.parts, index.names_swapped, index.rename)}
+        return out
+
    def export(self, session, progress=None, cancel=None, keep_notes=False, low_priority=True):
        """Write exports/<session>/ (DESIGN.md "Export"): manifest.json, calibration.json, per
        take prompts.jsonl, poses.jsonl, take.json and sets.bin.zst (sets in deleted ranges
@@ -426,15 +478,26 @@ class Store:
            tdir = self.take_dir(session, t["id"])
            out = os.path.join(work, "takes", t["id"])
            os.makedirs(out)
-            for name in ("prompts.jsonl", "poses.jsonl", "take.json"):
+            for name in ("prompts.jsonl", "poses.jsonl"):
                if os.path.isfile(os.path.join(tdir, name)):
                    shutil.copyfile(os.path.join(tdir, name), os.path.join(out, name))
            index = take_index(tdir)
            ranges = t["ranges"]
            keep = [i for i in range(len(index)) if not in_ranges(index.time_ns(i), ranges)]
+            # one file, named right where the decision is known (sides.py): its names_swapped is
+            # then the session's swapped; unknown, it's the parts' own (None if they differ)
+            if index.swapped is not None:
+                names_swapped = bool(index.swapped)
+            else:
+                names_swapped = index.names_swapped[0] if len(set(index.names_swapped)) == 1 else None
+            take_meta = read_json(os.path.join(tdir, "take.json"))
+            if take_meta:
+                take_meta["parts"] = {"sets.bin": {"names_swapped": names_swapped}}
+                write_json(os.path.join(out, "take.json"), take_meta)
            entry = {"id": t["id"], "section": t["section"], "title": t["title"], "status": t["status"],
                     "sets": len(keep), "sets_deleted": len(index) - len(keep), "cameras": index.cams,
-                     "duration_s": round(t["duration_s"], 2), "file": None}
+                     "duration_s": round(t["duration_s"], 2), "file": None,
+                     "sides": {"names_swapped": names_swapped, "renamed_sets": index.renamed_sets(keep)}}
            done += index.bytes() - sum(index.sets[i][2] for i in keep)   # deleted sets count as done
            if keep:
                entry["file"] = "takes/%s/sets.bin.zst" % t["id"]
@@ -523,6 +586,10 @@ def main():
    sub.add_parser("list")
    sub.add_parser("takes").add_argument("session")
    sub.add_parser("export").add_argument("session")
+    sp = sub.add_parser("sides", help="show (or record) which way round the side cameras were")
+    sp.add_argument("session")
+    sp.add_argument("--set", choices=("swapped", "named"),
+                    help="record the decision by hand (e.g. from tools/check_sides.py on a take)")
    a = ap.parse_args()
    store = Store(a.base)
    if a.cmd == "list":
@@ -533,6 +600,8 @@ def main():
        for t in store.takes(a.session):
            print(f"{t['id']}  {t['status']}  {t['sets']} sets ({t['deleted_sets']} deleted)  "
                  f"{t['duration_s']:.1f} s  {human_bytes(t['bytes'])}")
+    elif a.cmd == "sides":
+        print(json.dumps(store.sides(a.session, None if a.set is None else a.set == "swapped"), indent=1))
    else:
        path = store.export(a.session, progress=lambda f, text: print(f"\r{f * 100:5.1f}% {text:40.40}", end="",
                                                                       flush=True))
@@ -0,0 +1,269 @@
+#!/usr/bin/env python3
+"""Tests for the side cameras' naming in recordings (sides.py) and its readers: takes.py's
+review and export, validate.py, and session.py's live decision and recording parts.
+
+  python3 hands/rec/tests/test_sides.py
+"""
+import json
+import os
+import shutil
+import subprocess
+import sys
+import tempfile
+import time
+import unittest
+from unittest import mock
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, os.path.dirname(HERE))
+import session  # noqa: E402
+import sides  # noqa: E402
+import takes  # noqa: E402
+import validate  # noqa: E402
+from test_validate import fhset, make_session, SESSION  # noqa: E402
+
+
+def names(record):
+    ncams = sides.HDR.unpack_from(record, 0)[1]
+    return [sides.CAM.unpack_from(record, sides.HDR.size + k * sides.CAM.size)[0].split(b"\0")[0].decode()
+            for k in range(ncams)]
+
+
+class RulesTest(unittest.TestCase):
+    def test_rename_record(self):
+        rec = fhset(10 ** 9, 3)
+        out = sides.rename_record(rec)
+        self.assertEqual(names(rec), ["slam_left", "slam_right"])
+        self.assertEqual(names(out), ["slam_right", "slam_left"])
+        self.assertEqual(len(out), len(rec))
+        head = sides.HDR.size + 2 * sides.CAM.size
+        self.assertEqual(out[head:], rec[head:])   # pixels untouched
+        self.assertEqual(sides.rename_record(out), rec)
+
+    def test_other_name(self):
+        self.assertEqual(sides.other_name("slam_left_dk"), "slam_right_dk")
+        self.assertEqual(sides.other_name("upper_left"), "upper_left")
+
+    def test_needs_rename(self):
+        self.assertTrue(sides.needs_rename(True, False))
+        self.assertFalse(sides.needs_rename(True, True))
+        self.assertTrue(sides.needs_rename(False, True))
+        self.assertFalse(sides.needs_rename(None, True))   # unknown: leave the names
+
+    def test_part_rename(self):
+        sess = {"sides": {"swapped": True}}
+        take = {"parts": {"sets.bin": {"names_swapped": False}, "sets-2.bin": {"names_swapped": True}}}
+        self.assertTrue(sides.part_rename(sess, take, "sets.bin"))
+        self.assertFalse(sides.part_rename(sess, take, "sets-2.bin"))
+        self.assertTrue(sides.part_rename(sess, {}, "sets-3.bin"))   # no record: as ft-camd named them
+        self.assertFalse(sides.part_rename({}, take, "sets.bin"))    # no decision
+
+    def test_recording_runs(self):
+        d = tempfile.mkdtemp()
+        try:
+            self.assertEqual(sides.recording_runs(d), [(0, False)])
+            with open(os.path.join(d, "sides.json"), "w") as f:
+                json.dump({"swapped": True, "decided_by": "auto", "names_swapped": [[0, False], [37, True]]}, f)
+            runs = sides.recording_runs(d)
+            self.assertEqual(runs, [(0, True), (37, False)])
+            self.assertTrue(sides.rename_at(runs, 36))
+            self.assertFalse(sides.rename_at(runs, 37))
+        finally:
+            shutil.rmtree(d)
+
+    def test_read_live(self):
+        d = tempfile.mkdtemp()
+        try:
+            path, ring = os.path.join(d, "sides.json"), os.path.join(d, "cam-ring")
+            open(ring, "w").close()
+            now = time.clock_gettime_ns(time.CLOCK_MONOTONIC)
+            live = {"pid": 1, "ring_ino": os.stat(ring).st_ino, "swapped": True, "decided_by": "auto",
+                    "state": "decided", "updated_ns": now}
+            with open(path, "w") as f:
+                json.dump(live, f)
+            self.assertEqual(sides.read_live(path, ring)["swapped"], True)
+            self.assertIsNone(sides.read_live(path, ring, now_ns=now + 10 * 10 ** 9))   # stale
+            os.remove(ring)
+            open(ring, "w").close()   # a new ft-camd: a new ring file
+            if os.stat(ring).st_ino != live["ring_ino"]:
+                self.assertIsNone(sides.read_live(path, ring))
+            self.assertIsNone(sides.read_live(os.path.join(d, "none.json")))
+        finally:
+            shutil.rmtree(d)
+
+
+class TakesTest(unittest.TestCase):
+    """A swapped session: one take's parts recorded before the decision (ft-camd's names) and
+    after it (named right)."""
+
+    def setUp(self):
+        if not takes.find_zstd():
+            self.skipTest("no zstd")
+        self.tmp = tempfile.mkdtemp(prefix="handrec-sides-test-")
+        make_session(self.tmp)
+        self.sdir = os.path.join(self.tmp, "sessions", SESSION)
+        meta = takes.read_json(os.path.join(self.sdir, "session.json"))
+        meta["sides"] = {"swapped": True, "decided_by": "auto", "state": "confirmed"}
+        takes.write_json(os.path.join(self.sdir, "session.json"), meta)
+        self.tdir = os.path.join(self.sdir, "takes", "01-hand-size")
+        with open(os.path.join(self.tdir, "sets-2.bin"), "wb") as f:   # after the decision: --sides 1
+            for i in range(5):
+                f.write(sides.rename_record(fhset(9 * 10 ** 12 + i * 10 ** 8, i)))
+        tj = takes.read_json(os.path.join(self.tdir, "take.json"))
+        tj["parts"] = {"sets.bin": {"names_swapped": False}, "sets-2.bin": {"names_swapped": True}}
+        takes.write_json(os.path.join(self.tdir, "take.json"), tj)
+        self.store = takes.Store(self.tmp)
+
+    def tearDown(self):
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def test_review_reads_right_names(self):
+        index = takes.take_index(self.tdir)
+        self.assertEqual(index.rename, [True, False])
+        self.assertEqual([c["name"] for c in index.cams], ["slam_right", "slam_left"])
+        self.assertEqual([c["name"] for c in index.read_set(0)], ["slam_right", "slam_left"])
+        self.assertEqual([c["name"] for c in index.read_set(len(index) - 1)], ["slam_right", "slam_left"])
+        # the same pixels under the other name
+        raw = takes.TakeIndex(self.tdir).read_set(0, only="slam_right")[0]["pixels"]
+        self.assertEqual(raw, fhset(10 ** 12, 0)[sides.HDR.size + 2 * sides.CAM.size:][:32 * 24])
+        self.assertEqual(index.renamed_sets(), 30)
+        # the other take has no parts record: recorded as ft-camd named them, renamed too
+        other = takes.take_index(os.path.join(self.sdir, "takes", "02-no-hands"))
+        self.assertEqual(other.rename, [True])
+
+    def test_export_writes_right_names(self):
+        path = self.store.export(SESSION, low_priority=False)
+        r = validate.validate(path)
+        self.assertEqual(r.errors, [])
+        self.assertEqual(r.warnings, [])
+        m = takes.read_json(os.path.join(path, "manifest.json"))
+        t = m["takes"][0]
+        self.assertEqual(t["sides"], {"names_swapped": True, "renamed_sets": 30})
+        self.assertEqual(m["session"]["sides"]["swapped"], True)
+        self.assertEqual(takes.read_json(os.path.join(path, "takes", t["id"], "take.json"))["parts"],
+                         {"sets.bin": {"names_swapped": True}})
+        # every exported set: right names (the rule applied again changes nothing)
+        data = subprocess.run([takes.find_zstd(), "-dc", os.path.join(path, t["file"])],
+                              capture_output=True, check=True).stdout
+        off, seen = 0, 0
+        while off < len(data):
+            nbytes = sides.HDR.unpack_from(data, off)[2]
+            self.assertEqual(names(data[off:off + nbytes]), ["slam_right", "slam_left"])
+            off, seen = off + nbytes, seen + 1
+        self.assertEqual(seen, 35)
+
+    def test_validate_catches_wrong_names(self):
+        path = self.store.export(SESSION, low_priority=False)
+        mpath = os.path.join(path, "manifest.json")
+        m = takes.read_json(mpath)
+        m["takes"][0]["sides"]["names_swapped"] = False
+        takes.write_json(mpath, m)
+        report = validate.Report(path)
+        validate.check_manifest(m, report, path)
+        self.assertTrue(any("aren't named right" in e for e in report.errors), report.errors)
+
+    def test_undecided_warns(self):
+        meta = takes.read_json(os.path.join(self.sdir, "session.json"))
+        meta["sides"] = {"swapped": None}
+        takes.write_json(os.path.join(self.sdir, "session.json"), meta)
+        path = self.store.export(SESSION, low_priority=False)
+        r = validate.validate(path)
+        self.assertEqual(r.errors, [])
+        self.assertTrue(any("wasn't decided" in w for w in r.warnings), r.warnings)
+        m = takes.read_json(os.path.join(path, "manifest.json"))
+        self.assertIsNone(m["takes"][0]["sides"]["names_swapped"])   # the parts differ
+        self.assertEqual(m["takes"][0]["sides"]["renamed_sets"], 0)
+
+    def test_sides_command(self):
+        out = self.store.sides(SESSION)
+        self.assertTrue(out["sides"]["swapped"])
+        self.assertEqual(out["takes"]["01-hand-size"]["sets-2.bin"], {"names_swapped": True, "renamed_when_read": False})
+        out = self.store.sides(SESSION, False)
+        self.assertEqual(out["sides"]["decided_by"], "manual")
+        self.assertTrue(out["sides"]["replaced"]["swapped"])
+        self.assertTrue(out["takes"]["01-hand-size"]["sets-2.bin"]["renamed_when_read"])
+        self.assertEqual(takes.take_index(self.tdir).rename, [False, True])   # the cache saw session.json change
+
+
+class SessionSidesTest(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.mkdtemp(prefix="handrec-sides-session-")
+        self.script = os.path.join(self.tmp, "script.json")
+        with open(self.script, "w") as f:
+            json.dump({"version": 1, "sections": [{"id": "a", "title": "A", "prompts": [{"text": "x", "seconds": 1}]}]},
+                      f)
+
+    def tearDown(self):
+        shutil.rmtree(self.tmp, ignore_errors=True)
+
+    def live(self, **kw):
+        d = dict(pid=1, ring_ino=0, mode="auto", state="decided", swapped=True, decided_by="auto",
+                 evidence={"as_named": 0, "swapped": 10}, updated_ns=time.clock_gettime_ns(time.CLOCK_MONOTONIC))
+        d.update(kw)
+        with open(os.path.join(self.tmp, "sides.json"), "w") as f:
+            json.dump(d, f)
+
+    def test_read_sides(self):
+        s = session.Session(os.path.join(self.tmp, "base"), {}, {}, "room", self.script, dry_run=True,
+                            hands_dir=self.tmp)
+        s.session_dir = self.tmp
+        s._session_json = {"sides": {"swapped": None}}
+        s._read_sides(force=True)
+        self.assertIsNone(s._sides_swapped())   # no file
+        self.live(swapped=None, state="deciding", decided_by=None)
+        s._read_sides(force=True)
+        self.assertIsNone(s._sides_swapped())
+        self.live()
+        s._read_sides(force=True)
+        self.assertTrue(s._sides_swapped())
+        self.assertEqual(s._session_json["sides"]["evidence"]["swapped"], 10)
+        self.live(state="confirmed")
+        s._read_sides(force=True)
+        self.assertEqual(s._session_json["sides"]["state"], "confirmed")
+        self.live(swapped=False, state="decided, reversed")
+        s._read_sides(force=True)
+        self.assertFalse(s._sides_swapped())
+        self.assertTrue(s._session_json["sides"]["reversed_from"]["swapped"])
+        self.assertFalse(takes.read_json(os.path.join(self.tmp, "session.json"))["sides"]["swapped"])
+
+    def test_recorder_parts(self):
+        calls = []
+
+        class FakeProc:
+            returncode = 0
+
+            def __init__(self, argv, **kw):
+                calls.append(argv)
+
+            def poll(self):
+                return 0
+
+        take = os.path.join(self.tmp, "take")
+        os.makedirs(take)
+        with mock.patch.object(session.subprocess, "Popen", FakeProc):
+            r1 = session.Recorder(take, 1, 10, None, None)
+            r2 = session.Recorder(take, 2, 10, None, None, swap=True)
+            r3 = session.Recorder(take, 3, 10, None, None, swap=False)
+        argv = [c[c.index("--sides") + 1] for c in calls]
+        self.assertEqual(argv, ["auto", "1", "0"])
+        # ft-hands' DIR/sides.json goes into names_swapped and away
+        os.makedirs(r2.dir, exist_ok=True)
+        with open(os.path.join(r2.dir, "sets.bin"), "wb") as f:
+            f.write(fhset(10 ** 9, 0))
+        with open(os.path.join(r2.dir, "sides.json"), "w") as f:
+            json.dump({"swapped": None, "names_swapped": [[0, True]]}, f)
+        r2.stop()
+        self.assertTrue(r2.names_swapped)
+        self.assertEqual(r2.file, "sets-2.bin")
+        self.assertTrue(os.path.isfile(os.path.join(take, "sets-2.bin")))
+        self.assertFalse(os.path.exists(r2.dir))
+        with open(os.path.join(take, "sides.json"), "w") as f:
+            json.dump({"swapped": None, "names_swapped": [[0, False]]}, f)
+        r1.stop()
+        self.assertFalse(r1.names_swapped)
+        self.assertFalse(os.path.exists(os.path.join(take, "sides.json")))
+        self.assertFalse(r3.names_swapped)
+
+
+if __name__ == "__main__":
+    unittest.main()
@@ -68,7 +68,8 @@ def make_session(base, contributor=None, sid=None, device=True):
                      "rings": False, "watch": False, "notes": ""},
        "device": {"steamos": "3.8", "steamvr": "r25358740+28b72a4f-1", "cameras": [{"name": n, "width": w, "height": h}
                                                                 for n, w, h in CAMS]},
-        "calibration_removed": ["serial"], "takes": ["01-hand-size", "02-no-hands"], "status": "done"})
+        "calibration_removed": ["serial"], "takes": ["01-hand-size", "02-no-hands"], "status": "done",
+        "sides": {"swapped": False, "decided_by": "auto", "state": "confirmed"}})
    takes.write_json(os.path.join(sdir, "calibration.json"),
                     {"cameras": [{"name": "slam_left", "intrinsics": [1.0, 2.0, 3.0]}]})
    if device:
@@ -15,6 +15,8 @@ checks:
    every set's header is checked (camera names, sizes, record length) and counted against the
    manifest; the pixels themselves aren't looked at;
  - every .jsonl line parses, and take.json does;
+  - the side cameras (sides.py): the session's "sides" decision, and each take's files named
+    right by it (a warning when it wasn't decided: the maintainer's check tells then);
  - the total size.
 It runs on Linux and Windows, with Python 3.12 or later. It reports errors (don't upload or accept this) and warnings (look at it).

@@ -585,9 +587,37 @@ def check_manifest(m, report, root):
            report.error(f"manifest: take {tid}'s file is {f!r}")
        elif not isinstance(t.get("raw_bytes"), int):
            report.error(f"manifest: take {tid} has a file but no raw_bytes")
+    check_sides(m, entries, report)
    return entries


+def check_sides(m, entries, report):
+    """Which side camera is which (sides.py): session.sides.swapped is true, false or null, and
+    every take's files carry names_swapped equal to it (export renames them so)."""
+    session = m.get("session") if isinstance(m.get("session"), dict) else {}
+    s = session.get("sides")
+    if s is None:
+        report.warn("manifest: no session.sides (recorded before the side cameras were checked): "
+                    "the maintainer's check tells which way round they are")
+        return
+    if not isinstance(s, dict) or s.get("swapped") not in (True, False, None):
+        report.error("manifest: session.sides isn't {\"swapped\": true|false|null, ...}")
+        return
+    swapped = s["swapped"]
+    if swapped is None:
+        report.warn("the side cameras' naming wasn't decided while recording (no live tracker): "
+                    "the maintainer's check tells which way round they are")
+    for tid, t in entries.items():
+        ts = t.get("sides")
+        if ts is None:
+            continue
+        if not isinstance(ts, dict) or ts.get("names_swapped") not in (True, False, None):
+            report.error(f"manifest: take {tid}'s sides isn't {{\"names_swapped\": true|false|null, ...}}")
+        elif swapped is not None and ts["names_swapped"] != swapped:
+            report.error(f"take {tid}: its side cameras aren't named right (names_swapped "
+                         f"{ts['names_swapped']}, the session's swapped {swapped}): export it again")
+
+
 def summarize(m):
    """What a pull request's description says about the export: takes, minutes, lighting,
    objects, controllers, the consent and tool versions."""
@@ -0,0 +1,146 @@
+// Unit tests for the side cameras' naming check (track/sides.h), on made-up cameras and a
+// made-up hand: no models, no recordings. make check (in the dev container) builds and runs it.
+#include "../track/sides.h"
+
+#include <cstdio>
+#include <cstdlib>
+
+namespace {
+
+int failures = 0;
+#define CHECK(c)                                                         \
+    do {                                                                 \
+        if (!(c)) std::printf("FAIL %s:%d: %s\n", __FILE__, __LINE__, #c), ++failures; \
+    } while (0)
+
+V3 cross(V3 a, V3 b) { return {a[1] * b[2] - a[2] * b[1], a[2] * b[0] - a[0] * b[2], a[0] * b[1] - a[1] * b[0]}; }
+
+// An equidistant fisheye (k = 0) at origin, looking along look, image "down" toward head -y.
+Camera camera(const char *name, V3 origin, V3 look, int w, int h, double f) {
+    Camera c;
+    c.name = name, c.width = w, c.height = h, c.fx = c.fy = f, c.cx = w / 2.0, c.cy = h / 2.0;
+    const V3 z = unit(look), down{0, -1, 0};
+    const V3 y = unit(down - z * dot(down, z)), x = cross(y, z);
+    for (int i = 0; i < 3; ++i) c.R[i][0] = x[i], c.R[i][1] = y[i], c.R[i][2] = z[i];
+    c.origin = origin;
+    return c;
+}
+
+// MediaPipe-like world landmarks (m, hand-centred, flat: z = 0): x across the palm, y along the fingers
+const double kHand[21][2] = {{0, 0},           {0.03, 0.03},     {0.05, 0.05},     {0.065, 0.07},   {0.075, 0.09},
+                             {0.025, 0.09},    {0.025, 0.13},    {0.025, 0.155},   {0.025, 0.175},  {0.005, 0.095},
+                             {0.005, 0.14},    {0.005, 0.165},   {0.005, 0.185},   {-0.015, 0.09},  {-0.015, 0.13},
+                             {-0.015, 0.155},  {-0.015, 0.17},   {-0.033, 0.08},   {-0.033, 0.11},  {-0.033, 0.13},
+                             {-0.033, 0.145}};
+
+// The hand with its wrist at p, the palm facing toward `facing`, as cam sees it.
+Landmarks view(const Camera &cam, V3 p, V3 facing) {
+    const V3 n = unit(facing - p), up0{0, 1, 0};
+    const V3 along = unit(up0 - n * dot(up0, n)), across = cross(along, n);
+    Landmarks lm;
+    for (int k = 0; k < 21; ++k) {
+        const V3 q = p + across * kHand[k][0] + along * kHand[k][1];
+        lm.pts[k] = cam.project(q, nullptr);
+        lm.world[k][0] = kHand[k][0], lm.world[k][1] = kHand[k][1], lm.world[k][2] = 0;
+    }
+    lm.presence = 0.9;
+    return lm;
+}
+
+Seen seen(const std::string &cam, const Landmarks &lm) { return Seen{cam, 0, Roi{}, lm, {}}; }
+
+}  // namespace
+
+int main() {
+    std::map<std::string, Camera> cams;
+    cams["slam_left"] = camera("slam_left", {-0.045, 0, -0.03}, {-0.5, -0.6, -0.6}, 1056, 1024, 300);
+    cams["slam_right"] = camera("slam_right", {0.045, 0, -0.03}, {0.5, -0.6, -0.6}, 1056, 1024, 300);
+    cams["upper_left"] = camera("upper_left", {-0.03, 0.02, -0.04}, {-0.2, 0.1, -1}, 640, 480, 180);
+    cams["upper_right"] = camera("upper_right", {0.03, 0.02, -0.04}, {0.2, 0.1, -1}, 640, 480, 180);
+    const Camera &L = cams["slam_left"], &R = cams["slam_right"], &UL = cams["upper_left"], &UR = cams["upper_right"];
+    const V3 hand{0.0, -0.22, -0.35}, eyes{0, 0, -0.03};
+    const Landmarks inL = view(L, hand, eyes), inR = view(R, hand, eyes), inUL = view(UL, hand, eyes),
+                    inUR = view(UR, hand, eyes);
+    SideCheck sc(cams);
+    CHECK(sc.usable());
+
+    // 1. the side pair, named right: meets as named, not swapped
+    SideCheck::Miss m = sc.test("slam_left", inL, "slam_right", inR);
+    std::printf("side pair as named: miss %.4f m as named, %.4f swapped\n", m.m[0], m.m[1]);
+    CHECK(m.m[0] >= 0 && m.m[0] < 0.001);
+    CHECK(SideCheck::vote(m) == 0);
+    // 2. the same images under each other's names (what a backwards ft-camd gives)
+    m = sc.test("slam_left", inR, "slam_right", inL);
+    std::printf("side pair swapped: miss %.4f m as named, %.4f swapped\n", m.m[0], m.m[1]);
+    CHECK(m.m[1] >= 0 && m.m[1] < 0.001);
+    CHECK(SideCheck::vote(m) == 1);
+    // 3. a side camera with an upper one
+    m = sc.test("slam_left", inL, "upper_left", inUL);
+    CHECK(SideCheck::vote(m) == 0);
+    m = sc.test("slam_right", inL, "upper_left", inUL);   // slam_left's image under slam_right's name
+    CHECK(SideCheck::vote(m) == 1);
+    m = sc.test("upper_right", inUR, "slam_left", inR);   // ... and the other way round, other order
+    CHECK(SideCheck::vote(m) == 1);
+    // 4. the upper pair alone says nothing
+    CHECK(sc.add({seen("upper_left", inUL), seen("upper_right", inUR)}, 1) == 0);
+    // 5. two different hands, one in each side camera, don't vote
+    const V3 other{0.25, -0.3, -0.3};
+    m = sc.test("slam_left", inL, "slam_right", view(R, other, eyes));
+    std::printf("two hands: miss %.4f m as named, %.4f swapped\n", m.m[0], m.m[1]);
+    CHECK(SideCheck::vote(m) == -1);
+    // 6. a vote needs one way clearly better: both meeting about as well is no vote
+    CHECK(SideCheck::vote({{0.004, 0.006}}) == -1);
+    CHECK(SideCheck::vote({{0.004, 0.02}}) == 0);
+    CHECK(SideCheck::vote({{-1, 0.003}}) == 1);
+    CHECK(SideCheck::vote({{0.02, -1}}) == -1);   // too far apart to be one hand
+    CHECK(SideCheck::vote({{-1, -1}}) == -1);
+
+    // 7. the decision: min_clean votes and none against, or min_votes and at most max_other
+    // against, over min_span_s
+    const std::vector<Seen> named = {seen("slam_left", inL), seen("slam_right", inR)};
+    const std::vector<Seen> swapped = {seen("slam_left", inR), seen("slam_right", inL)};
+    const int64_t s = 1'000'000'000;
+    sc.reset();
+    for (int i = 0; i < 9; ++i) CHECK(sc.add(named, i * s / 5) == 1);
+    CHECK(sc.verdict() == SideCheck::Undecided);   // 9 votes
+    sc.add(named, 9 * s / 5);
+    CHECK(sc.verdict() == SideCheck::AsNamed);     // 10 clean votes over 1.8 s
+    std::printf("%s\n", sc.summary().c_str());
+    // fast: 30 clean votes in 0.97 s aren't enough; the vote at 1.0 s is
+    sc.reset();
+    for (int i = 0; i < 30; ++i) sc.add(swapped, i * s / 30);
+    CHECK(sc.votes[1] == 30 && sc.verdict() == SideCheck::Undecided);
+    sc.add(swapped, s);
+    CHECK(sc.verdict() == SideCheck::Swapped);
+    CHECK(sc.median_miss_mm(1) >= 0 && sc.median_miss_mm(1) < 1);
+    // one vote against: it waits for min_votes
+    sc.reset();
+    sc.add(swapped, 0);
+    for (int i = 1; i < 20; ++i) sc.add(named, i * s / 10);
+    CHECK(sc.verdict() == SideCheck::Undecided);   // 19 to 1
+    sc.add(named, 2 * s);
+    CHECK(sc.verdict() == SideCheck::AsNamed);     // 20 to 1
+    // mixed evidence waits until the other way is at most a fifth of the votes
+    sc.reset();
+    for (int i = 0; i < 20; ++i) sc.add(named, i * s / 10);
+    for (int i = 0; i < 6; ++i) sc.add(swapped, (20 + i) * s / 10);
+    CHECK(sc.verdict() == SideCheck::Undecided);   // 6 of 26
+    for (int i = 0; i < 4; ++i) sc.add(named, (26 + i) * s / 10);
+    CHECK(sc.verdict() == SideCheck::AsNamed);     // 6 of 30
+    // the stricter check after a decision
+    sc.reset();
+    sc.min_clean = 20, sc.min_votes = 40;
+    for (int i = 0; i < 19; ++i) sc.add(named, i * s / 10);
+    CHECK(sc.verdict() == SideCheck::Undecided);
+    sc.add(named, 2 * s);
+    CHECK(sc.verdict() == SideCheck::AsNamed);
+    CHECK(sc.json().find("\"as_named\": 20") != std::string::npos);
+    // without both side cameras it can't check
+    std::map<std::string, Camera> one{{"slam_left", L}, {"upper_left", UL}};
+    SideCheck none(one);
+    CHECK(!none.usable());
+    CHECK(none.add(named, 0) == 0);
+
+    std::printf(failures ? "%d FAILED\n" : "sides_test: all passed\n", failures);
+    return failures ? 1 : 0;
+}
@@ -1,11 +1,18 @@
 """Check that the side cameras' images carry the right names (slam_left vs slam_right).

-usage: python tools/check_sides.py REC_DIR [--sets N]
-       python tools/check_sides.py --ring [--sets N]    (live, from ft-camd's ring)
+usage: python tools/check_sides.py REC [--sets N] [--pair side|upper] [--calib DIR] [--json]
+       python tools/check_sides.py --ring [--sets N] [--pair side|upper]   (live, from ft-camd's ring)

-With --ring it exits 0 when the names are right, 3 when they're swapped (run ft-hands
-with --swap-sides), and 2 when it can't tell (too little texture in view, or the headset
-isn't worn).
+REC is a recording folder (its sets.bin) or a sets file (a take's sets-N.bin). It checks the
+names as they are in the files: a recording's sides.json or session.json "sides" (the hand
+recorder's) isn't applied. Exits 0 when the names are right, 3 when they're swapped, and 2 when
+it can't tell (too little texture in view, or the headset isn't worn). --pair upper checks the
+upper pair (upper_left vs upper_right) the same way. --calib DIR uses DIR/calibration.json and
+DIR/device.json (a hand recorder session's, or cut.py's) instead of /persist. --json prints one
+line: {"pair", "verdict": "named"|"swapped"|"unknown", "named", "swapped", "matches", "sets"}.
+
+ft-hands decides the side cameras' naming by itself (HANDS_SWAP_SIDES=auto, track/sides.h);
+this is the independent check, from the scene rather than hands.

 ft-camd tells the two side cameras' buffers apart by the order XRService allocated them,
 and after some XRService restarts that order puts each camera's images under the other's
@@ -15,6 +22,7 @@ with the factory calibration, once as named and once swapped: true matches meet
 front of both cameras only under the right naming.
 """
 import argparse
+import json
 import os
 import sys

@@ -26,10 +34,14 @@ from tools.show_set import index, read_set  # noqa: E402
 from tools import calib  # noqa: E402


-PIPES = {'msm_vfe3_video0': 'slam_left', 'msm_vfe4_video0': 'slam_right'}   # as ft-hands maps them
+PIPES = {'msm_vfe3_video0': 'slam_left', 'msm_vfe4_video0': 'slam_right',    # as ft-hands maps them
+         'msm_vfe2_video0': 'upper_left', 'msm_vfe2_video1': 'upper_right'}
+PAIRS = {'side': ('slam_left', 'slam_right'), 'upper': ('upper_left', 'upper_right')}


-def load_cams():
+def load_cams(calib_dir=None):
+    if calib_dir:
+        return calib.load(os.path.join(calib_dir, 'calibration.json'), os.path.join(calib_dir, 'device.json'))
    return calib.load()


@@ -68,18 +80,20 @@ def score(cam_a, cam_b, ua, ub):
    return float(np.mean((d < 0.01) & front))


-def recorded_pairs(rec, count):
-    """(label, slam_left image, slam_right image) from sets spread across a recording."""
-    path = os.path.join(rec, 'sets.bin')
+def recorded_pairs(rec, count, names=PAIRS['side']):
+    """(label, left image, right image) from sets spread across a recording (or sets file)."""
+    path = rec if os.path.isfile(rec) else os.path.join(rec, 'sets.bin')
    offs = index(path)
-    for n in np.linspace(0, len(offs) - 1, count).astype(int):
+    if not offs:
+        return
+    for n in sorted(set(np.linspace(0, len(offs) - 1, count).astype(int))):
        images = read_set(path, offs[n])
-        if 'slam_left' in images and 'slam_right' in images:
-            yield 'set %5d' % n, images['slam_left'][0], images['slam_right'][0]
+        if names[0] in images and names[1] in images:
+            yield 'set %5d' % n, images[names[0]][0], images[names[1]][0]


-def live_pairs(count):
-    """(label, slam_left image, slam_right image) from ft-camd's ring, half a second apart."""
+def live_pairs(count, names=PAIRS['side']):
+    """(label, left image, right image) from ft-camd's ring, half a second apart."""
    import time
    from tools.ring import Ring
    ring = Ring()
@@ -91,37 +105,61 @@ def live_pairs(count):
        if name and not c.name.endswith('-dark'):
            cams[name] = c
    for k in range(count):
-        a, b = ring.read(cams['slam_left']), ring.read(cams['slam_right'])
+        a, b = ring.read(cams[names[0]]), ring.read(cams[names[1]])
        if a is not None and b is not None:
            yield 'frame %2d' % k, a.image, b.image
        time.sleep(0.5)


+def verdict(named, swapped, n, total_matches):
+    """'named', 'swapped' or 'unknown', from the summed per-set scores: the winner must meet
+    clearly more often (by 0.08 a set) and at least 4 times as often. The upper cameras' small
+    images meet within 1 cm less often (0.1-0.4 a set as named, 0 swapped) than the side ones (0.4-0.9)."""
+    hi, lo = max(named, swapped), min(named, swapped)
+    if n == 0 or total_matches < 100 or (hi - lo) / n < 0.08 or lo > 0.25 * hi:
+        return 'unknown'
+    return 'named' if named > swapped else 'swapped'
+
+
+def check(pairs, left, right, out=print):
+    """Score (label, left image, right image) pairs: (verdict, named, swapped, matches, sets)."""
+    named = swapped = 0.0
+    n = total_matches = 0
+    for label, img_l, img_r in pairs:
+        ua, ub = matches(img_l, img_r)
+        s_named = score(left, right, ua, ub)     # the left-named image seen by the left camera
+        s_swapped = score(right, left, ua, ub)   # ... by the right camera
+        named, swapped, n, total_matches = named + s_named, swapped + s_swapped, n + 1, total_matches + len(ua)
+        out('%s: %4d matches, meeting as named %3.0f%%, swapped %3.0f%%' %
+            (label, len(ua), 100 * s_named, 100 * s_swapped))
+    return verdict(named, swapped, n, total_matches), named / max(n, 1), swapped / max(n, 1), total_matches, n
+
+
 def main():
    ap = argparse.ArgumentParser()
    ap.add_argument('rec', nargs='?')
    ap.add_argument('--ring', action='store_true', help='check the live cameras instead of a recording')
    ap.add_argument('--sets', type=int, default=8, help='how many sets or live frames to check')
+    ap.add_argument('--pair', choices=sorted(PAIRS), default='side')
+    ap.add_argument('--calib', help='folder with calibration.json and device.json (default: /persist)')
+    ap.add_argument('--json', action='store_true', help='one JSON line instead of the report')
    a = ap.parse_args()
    if not a.ring and not a.rec:
        ap.error('give a recording or --ring')
-    cams = load_cams()
-    left, right = cams['slam_left'], cams['slam_right']
-    named = swapped = 0.0
-    n = total_matches = 0
-    for label, img_l, img_r in (live_pairs(a.sets) if a.ring else recorded_pairs(a.rec, a.sets)):
-        ua, ub = matches(img_l, img_r)
-        s_named = score(left, right, ua, ub)     # slam_left's image seen by the left camera
-        s_swapped = score(right, left, ua, ub)   # ... by the right camera
-        named, swapped, n, total_matches = named + s_named, swapped + s_swapped, n + 1, total_matches + len(ua)
-        print('%s: %4d matches, meeting as named %3.0f%%, swapped %3.0f%%' %
-              (label, len(ua), 100 * s_named, 100 * s_swapped))
-    if n == 0 or total_matches < 100 or abs(named - swapped) / n < 0.2:
-        print('side cameras: can\'t tell (%d matches)' % total_matches)
-        sys.exit(2)
-    print('side cameras: %s (named %.2f, swapped %.2f)' %
-          ('as named' if named > swapped else 'SWAPPED', named / n, swapped / n))
-    sys.exit(0 if named > swapped else 3)
+    cams = load_cams(a.calib)
+    names = PAIRS[a.pair]
+    left, right = cams[names[0]], cams[names[1]]
+    pairs = live_pairs(a.sets, names) if a.ring else recorded_pairs(a.rec, a.sets, names)
+    v, named, swapped, total, n = check(pairs, left, right, out=(lambda s: None) if a.json else print)
+    what = 'side cameras' if a.pair == 'side' else 'upper cameras'
+    if a.json:
+        print(json.dumps({'pair': a.pair, 'verdict': v, 'named': round(named, 3), 'swapped': round(swapped, 3),
+                          'matches': total, 'sets': n}))
+    elif v == 'unknown':
+        print('%s: can\'t tell (%d matches)' % (what, total))
+    else:
+        print('%s: %s (named %.2f, swapped %.2f)' % (what, 'as named' if v == 'named' else 'SWAPPED', named, swapped))
+    sys.exit({'named': 0, 'swapped': 3, 'unknown': 2}[v])


 if __name__ == '__main__':
@@ -3,7 +3,7 @@
 // Python prototype: the same scheduling, with the models on a few threads.
 //
 //   ft-hands [--seconds N] [--threads N] [--int8] [--status S] [--models DIR] [--nice N]
-//            [--no-publish] [--no-gestures] [--record DIR] [--swap-sides] [--cams auto|mono|color|all] ...
+//            [--no-publish] [--no-gestures] [--record DIR] [--sides auto|0|1] [--cams auto|mono|color|all] ...
 //            (--help lists them all)
 //
 // --no-gestures: hands for the cutouts only. No pinch or grip detection, so nothing reaches
@@ -22,14 +22,24 @@
 // own clock, so they're placed on the mono cameras' by when they were dequeued, less the
 // mono cameras' measured delay.
 //
+// Which side camera is which (--sides, HANDS_SWAP_SIDES): ft-camd tells slam_left's buffers from
+// slam_right's by XRService's allocation order, which some XRService starts reverse. auto (the
+// default) tells from the hands it tracks (track/sides.h): once it's sure, it exchanges the two
+// cameras if they're backwards (the tracked views move with their images), and checks once
+// more. 0 and 1 force the naming (1: exchanged; --swap-sides is --sides 1); it still checks,
+// and warns if the hands disagree. The decision is published in /run/user/UID/frametop-hands/sides.json (see
+// write_sides below) and, for recordings, in DIR/sides.json. --record-only can't tell (it tracks
+// nothing): under auto it records the ring's names as they are.
+//
 // Settings in ~/.config/frametop.conf (FT_<name> in the environment overrides them, and
-// options override both): HANDS_SWAP_SIDES (1: as --swap-sides), HANDS_CPUS (as --cpus),
+// options override both): HANDS_SWAP_SIDES (auto, 0 or 1), HANDS_CPUS (as --cpus),
 // HANDS_CAMERAS, HANDS_BRIGHT, HANDS_BRIGHT_ON, HANDS_BRIGHT_OFF, HANDS_COLOR_LEFT (which
 // colour camera is passthrough_left: color_video0 or color_video3), HANDS_COLOR_CROP
 // (subtract or none: tools/check_color.py tells both).
 #include "io.h"
 #include "pinch.h"
 #include "record.h"
+#include "sides.h"

 #include <sched.h>
 #include <sys/resource.h>
@@ -45,6 +55,7 @@
 #include <cstdio>
 #include <cstring>
 #include <fstream>
+#include <optional>
 #include <thread>
 #include <utility>

@@ -148,12 +159,25 @@ struct Lighting {
    }
 };

+// Writes path through a temporary file, so a reader never sees half of it.
+bool write_file(const std::string &path, const std::string &text) {
+    const std::string tmp = path + ".tmp";
+    FILE *f = std::fopen(tmp.c_str(), "w");
+    if (!f) return false;
+    const bool ok = std::fputs(text.c_str(), f) >= 0;
+    if (std::fclose(f) != 0 || !ok || std::rename(tmp.c_str(), path.c_str()) != 0) return unlink(tmp.c_str()), false;
+    return true;
+}
+
+std::string json_bool(std::optional<bool> b) { return !b ? "null" : *b ? "true" : "false"; }
+std::string json_str(const std::string &s) { return s.empty() ? "null" : "\"" + s + "\""; }
+
 }  // namespace

 int main(int argc, char **argv) {
    double seconds = 0, status = 5;
    int threads = 3, niceness = 5;
-    bool int8 = false, publish = true, track = true, swap_sides = false, gestures_on = true;
+    bool int8 = false, publish = true, track = true, gestures_on = true;
    std::string models = std::string(argv[0]).substr(0, std::string(argv[0]).rfind('/') + 1) + "../models/ncnn";
    std::string record, ring_path = "/run/user/" + std::to_string(getuid()) + "/" FH_RING_NAME;
    // SteamOS starts user processes on CPUs 0-4 and keeps 5-7 (two A720s and the X4) for
@@ -163,7 +187,9 @@ int main(int argc, char **argv) {
    // latency 9.6 against 14.1 ms, and the compositor's late frames and CPU/GPU time didn't change.
    std::vector<int> cpus = {5, 6, 7};
    if (const auto c = parse_cpus(setting("HANDS_CPUS").c_str()); !c.empty()) cpus = c;
-    swap_sides = setting("HANDS_SWAP_SIDES") == "1";
+    // Which side camera is which (see the top): auto, 0 or 1, and where that came from
+    std::string sides_mode = setting("HANDS_SWAP_SIDES"), sides_from = "config";
+    if (sides_mode.empty()) sides_mode = "auto", sides_from = "default";
    // Which cameras (see the top).
    std::string cams_arg = setting("HANDS_CAMERAS"), bright_arg = setting("HANDS_BRIGHT");
    std::string color_left = setting("HANDS_COLOR_LEFT"), color_crop = setting("HANDS_COLOR_CROP");
@@ -200,7 +226,8 @@ int main(int argc, char **argv) {
        else if (a == "--grip-begin" && more) grip_params.begin = std::atof(argv[++i]);
        else if (a == "--grip-end" && more) grip_params.end = std::atof(argv[++i]);
        else if (a == "--gesture-log") gesture_log = true;
-        else if (a == "--swap-sides") swap_sides = true;
+        else if (a == "--swap-sides") sides_mode = "1", sides_from = "option";
+        else if (a == "--sides" && more) sides_mode = argv[++i], sides_from = "option";
        else if (a == "--record-only") track = publish = false;
        else if (a == "--ring" && more) ring_path = argv[++i];
        else if (a == "--record" && more) record = argv[++i];
@@ -223,7 +250,9 @@ int main(int argc, char **argv) {
        else {
            std::printf("usage: %s [--seconds N] [--threads N] [--int8] [--status S] [--models DIR] [--nice N] [--no-publish]\n"
                        "          [--no-gestures] (hands for the cutouts only: no pinches or grips)\n"
-                        "          [--record DIR] [--record-for S] [--record-hz N] [--record-only] [--cpus 5,6,7] [--swap-sides]\n"
+                        "          [--record DIR] [--record-for S] [--record-hz N] [--record-only] [--cpus 5,6,7]\n"
+                        "          [--sides auto|0|1] (auto: tell from the hands which side camera is which; 1: exchange them,\n"
+                        "          as --swap-sides; 0: as ft-camd names them)\n"
                        "          [--keep-presence P] (0.5) [--ring PATH] (ft-camd's, or ft-ringplay's)\n"
                        "          [--cams auto|mono|color|all] (auto) [--bright all|color] (all) [--bright-on L] (40) [--bright-off L] (25)\n"
                        "          [--color-left color_video0|color_video3] [--color-crop subtract|none]\n"
@@ -236,7 +265,7 @@ int main(int argc, char **argv) {
                        "can run beside a tracking ft-hands. With ft-camd --with-dark, recordings also get each\n"
                        "camera's newest dark frame, as <name>_dk; with --with-color, the color cameras' as color_video<N>.\n"
                        "auto picks the cameras by the light (see the top of track/main.cpp).\n"
-                        "Settings in ~/.config/frametop.conf: HANDS_SWAP_SIDES=1, HANDS_CPUS=5,6,7, HANDS_CAMERAS, HANDS_BRIGHT,\n"
+                        "Settings in ~/.config/frametop.conf: HANDS_SWAP_SIDES=auto|0|1, HANDS_CPUS=5,6,7, HANDS_CAMERAS, HANDS_BRIGHT,\n"
                        "HANDS_BRIGHT_ON, HANDS_BRIGHT_OFF, HANDS_COLOR_LEFT, HANDS_COLOR_CROP (FT_<name> overrides).\n",
                        argv[0]);
            return a == "--help" ? 0 : 1;
@@ -247,6 +276,8 @@ int main(int argc, char **argv) {
    if ((!automatic && !parse_cams(cams_arg, fixed)) || !parse_cams(bright_arg, bright_cams) || bright_cams == Cams::Mono)
        return std::fprintf(stderr, "--cams auto|mono|color|all, --bright all|color\n"), 1;
    if (color_crop != "subtract" && color_crop != "none") return std::fprintf(stderr, "--color-crop subtract|none\n"), 1;
+    if (sides_mode != "auto" && sides_mode != "0" && sides_mode != "1")
+        return std::fprintf(stderr, "--sides (HANDS_SWAP_SIDES) auto, 0 or 1, not %s\n", sides_mode.c_str()), 1;
    std::setvbuf(stdout, nullptr, _IOLBF, 0);   // whole lines to the journal as they come
    if (nice(niceness) < 0) std::perror("nice");   // the VR stack wins contested CPUs
    std::signal(SIGINT, [](int) { g_stop = 1; });
@@ -263,9 +294,31 @@ int main(int argc, char **argv) {
    Grip grip(grip_params);
    std::unique_ptr<Recorder> rec;
    uint64_t rec_start = 0, rec_next_ns = 0;   // rec_next_ns: --record-hz's next set, capture clock
+    // The side cameras (see the top). names_swapped: slam_left's and slam_right's ring cameras
+    // are exchanged from ft-camd's naming. truth: whether ft-camd's naming is backwards, once known.
+    bool names_swapped = sides_mode == "1";
+    std::optional<bool> truth;
+    std::string decided_by, sides_state = "deciding", decision_evidence;
+    double decided_after_s = -1;
+    if (sides_mode != "auto") truth = names_swapped, decided_by = sides_from, sides_state = "forced";
+    std::string rec_dir;
+    std::vector<std::pair<size_t, bool>> rec_names;   // from which recorded set on, names_swapped was what
+    // DIR/sides.json beside a recording's sets.bin: how its side cameras are named. A set's
+    // names are right when its names_swapped equals swapped (null: not known when recorded).
+    auto write_rec_sides = [&] {
+        if (!rec) return;
+        std::string runs;
+        for (auto &[from, sw] : rec_names) runs += (runs.empty() ? "" : ", ") + ("[" + std::to_string(from) + ", " + json_bool(sw) + "]");
+        write_file(rec_dir + "/sides.json",
+                   "{\"swapped\": " + json_bool(truth) + ", \"decided_by\": " + json_str(truth ? decided_by : "") +
+                       ", \"names_swapped\": [" + runs + "]" +
+                       (decision_evidence.empty() ? "" : ", \"evidence\": " + decision_evidence) + "}\n");
+    };
    auto start_recording = [&](const std::string &dir, std::string &e) {
        rec = std::make_unique<Recorder>();
        if (!rec->open(dir, e)) return rec.reset(), false;
+        rec_dir = dir, rec_names = {{0, names_swapped}};
+        write_rec_sides();
        rec_start = mono_ns();
        std::printf("recording to %s for %.0f s\n", dir.c_str(), record_for);
        std::fflush(stdout);
@@ -301,12 +354,26 @@ int main(int argc, char **argv) {
        else index[name] = i, used[name] = calib[name];
    }
    // ft-camd tells the side cameras' buffers apart by XRService's allocation order, which
-    // some XRService restarts reverse; tools/check_sides.py --ring tells when.
-    if (swap_sides && index.count("slam_left") && index.count("slam_right")) {
+    // some XRService restarts reverse (see the top).
+    const bool have_sides = index.count("slam_left") && index.count("slam_right");
+    std::map<std::string, uint64_t> last;       // per camera: the frame last used
+    auto exchange_sides = [&] {
        std::swap(index["slam_left"], index["slam_right"]);
+        std::swap(last["slam_left"], last["slam_right"]);
        if (dark.count("slam_left_dk") && dark.count("slam_right_dk")) std::swap(dark["slam_left_dk"], dark["slam_right_dk"]);
-        std::printf("side cameras swapped (--swap-sides)\n");
+    };
+    if (names_swapped && have_sides) {
+        exchange_sides();
+        std::printf("side cameras exchanged (%s)\n", sides_from == "option" ? "--sides 1" : "HANDS_SWAP_SIDES=1");
    }
+    // the mono cameras the side check may pair (not the colour ones)
+    std::map<std::string, Camera> side_cams;
+    for (auto &[name, i] : index) side_cams[name] = calib[name];
+    SideCheck side_check(side_cams);
+    bool checking = track && have_sides && side_check.usable();
+    int side_round = 0;   // auto: 0 deciding, then each check after a decision
+    if (!track && sides_mode == "auto" && have_sides)
+        std::printf("side cameras: as ft-camd names them (--record-only can't tell; the recording's sides.json says so)\n");
    // The colour pair, calibrated (see the top), unless only the mono cameras are wanted.
    if (color.size() == 2 && (automatic || fixed != Cams::Mono)) {
        const std::string left = color.count(color_left) ? color_left : color.begin()->first;
@@ -347,7 +414,6 @@ int main(int argc, char **argv) {
    Tracker tracker(used, nets, pool);
    tracker.set_keep_presence(keep_presence);
    std::map<std::string, std::vector<uint8_t>> pixels;
-    std::map<std::string, uint64_t> last;       // per camera: the frame last used
    std::map<std::string, uint64_t> lit_seen;   // per colour camera: the frame last counted for the light
    const uint64_t start = mono_ns();
    uint64_t t_status = start, next_ns = 0, t_want = 0, t_glog = 0;
@@ -359,6 +425,71 @@ int main(int argc, char **argv) {
    // to place the colour frames, whose capture clock is their own (see the top).
    double mono_delay_ns = -1;
    const std::string want_file = run_dir() + "/color-fps";
+    // The side cameras' naming, for the hand recorder and other tools (hands/rec/session.py
+    // reads it): written by a tracking, publishing ft-hands at the start, on every change and
+    // every 2 s, and removed when it exits. swapped: ft-camd's naming is backwards (null: not
+    // known yet); names_swapped: whether this ft-hands exchanged them; ring_ino: the ring file's
+    // inode (a new ft-camd makes a new one, and the decision is only good for that run).
+    const std::string sides_file = run_dir() + "/sides.json";
+    const bool write_sides_file = publish && track;
+    struct stat ring_st{};
+    stat(ring_path.c_str(), &ring_st);
+    uint64_t t_sides = 0;
+    auto write_sides = [&] {
+        if (!write_sides_file) return;
+        t_sides = mono_ns();
+        write_file(sides_file,
+                   "{\"pid\": " + std::to_string(getpid()) + ", \"ring_ino\": " + std::to_string(ring_st.st_ino) +
+                       ", \"mode\": \"" + sides_mode + "\", \"state\": \"" + sides_state + "\", \"swapped\": " +
+                       json_bool(truth) + ", \"decided_by\": " + json_str(truth ? decided_by : "") +
+                       ", \"names_swapped\": " + json_bool(names_swapped) +
+                       ", \"decided_after_s\": " + std::to_string(decided_after_s) +
+                       ", \"evidence\": " + (decision_evidence.empty() ? "null" : decision_evidence) +
+                       ", \"checking\": " + (checking ? side_check.json() : "null") +
+                       ", \"updated_ns\": " + std::to_string(mono_ns()) + "}\n");
+    };
+    write_sides();
+    // A verdict from the side check (see the top and track/sides.h).
+    auto side_verdict = [&](SideCheck::Verdict v, uint64_t now) {
+        const bool backwards = v == SideCheck::Swapped;   // relative to the names as they are now
+        const double after = (now - start) / 1e9;
+        const std::string ev = side_check.json(), text = side_check.summary();
+        if (sides_mode != "auto") {   // forced: only say so
+            const std::string what = (sides_from == "option" ? "--sides " : "HANDS_SWAP_SIDES=") + sides_mode;
+            if (backwards)
+                std::printf("side cameras: %s looks WRONG: the hands say the side cameras are the other way round (%s). "
+                            "Use auto.\n", what.c_str(), text.c_str());
+            else
+                std::printf("side cameras: %s agrees with the hands (%s)\n", what.c_str(), text.c_str());
+            sides_state = backwards ? "forced, disagrees" : "forced, agrees";
+            decision_evidence = ev;
+            checking = false;
+        } else if (side_round == 0 || backwards) {
+            if (backwards) {
+                exchange_sides();
+                names_swapped = !names_swapped;
+                tracker.exchange("slam_left", "slam_right");
+                if (rec) rec_names.push_back({rec->added(), names_swapped});
+            }
+            const bool reversed = side_round > 0;
+            truth = names_swapped, decided_by = "auto", decided_after_s = after, decision_evidence = ev;
+            sides_state = reversed ? "decided, reversed" : "decided";
+            std::printf("side cameras: %s after %.1f s (%s)\n",
+                        reversed ? "decision REVERSED" : names_swapped ? "SWAPPED, now exchanged" : "as named", after,
+                        text.c_str());
+            // check once more with the new names, more strictly; at most two changes
+            side_check.reset();
+            side_check.min_clean = 20, side_check.min_votes = 40;
+            checking = ++side_round < 3;
+        } else {
+            std::printf("side cameras: confirmed after %.1f s (%s)\n", after, text.c_str());
+            sides_state = "confirmed";
+            checking = false;
+        }
+        std::fflush(stdout);
+        write_rec_sides();
+        write_sides();
+    };

    // The colour pair's newest frames if both are newer than last used and taken together
    // (their own clock): their time, on the mono cameras' capture clock, else 0.
@@ -405,8 +536,13 @@ int main(int argc, char **argv) {
        return n ? sum / n : -1;
    };

+    int exit_code = 0;
    while (!g_stop && (seconds <= 0 || (mono_ns() - start) / 1e9 < seconds)) {
-        if (!ring.alive()) return std::fprintf(stderr, "ft-camd stopped\n"), 2;
+        if (!ring.alive()) {
+            std::fprintf(stderr, "ft-camd stopped\n");
+            exit_code = 2;
+            break;
+        }
        const uint64_t now0 = mono_ns();
        const int64_t raw_off = raw_minus_mono_ns();

@@ -495,6 +631,7 @@ int main(int argc, char **argv) {
                const bool due = record_hz <= 0 || tmin >= rec_next_ns;   // --record-hz skips the sets between
                if (due && record_hz > 0) rec_next_ns = tmin + uint64_t(1e9 / record_hz) - 5'000'000;
                if ((mono_ns() - rec_start) / 1e9 >= record_for) {
+                    write_rec_sides();
                    const size_t n = rec->written(), d = rec->dropped();
                    rec.reset();   // writes out what's queued
                    std::printf("recording done: %zu sets, %zu dropped\n", n, d);
@@ -552,6 +689,16 @@ int main(int argc, char **argv) {
        const auto hands = tracker.step(images, int64_t(tmin));
        const uint64_t capture = uint64_t(int64_t(tmin) - raw_off);   // CLOCK_MONOTONIC
        const std::vector<Seen> views = tracker.views_now();
+        std::vector<Seen> side_views;   // the tracker's views, plus the side check's own looks
+        if (checking) {
+            side_views = views;
+            for (Seen &v : side_check.probe(nets, pool, images, views, int64_t(tmin))) side_views.push_back(v);
+        }
+        if (checking && side_check.add(side_views, int64_t(tmin)) > 0) {
+            const SideCheck::Verdict v = side_check.verdict();
+            if (v != SideCheck::Undecided) side_verdict(v, mono_ns());
+        }
+        if (mono_ns() - t_sides > 2'000'000'000ull) write_sides();
        if (gestures_on) {
            grip.update(hands, views, int64_t(capture));
            pinch.update(hands, views, int64_t(capture), grip.gripping());
@@ -607,6 +754,9 @@ int main(int argc, char **argv) {
                            s.handoff_miss, s.dups, s.splits, s.created, s.merged, s.forgotten,
                            !rec ? "" : ("  recorded " + std::to_string(rec->written()) + " dropped " +
                                         std::to_string(rec->dropped())).c_str());
+            if (checking)
+                std::printf("        side cameras: %s, %s\n", side_round ? "checking the decision" : "deciding",
+                            side_check.summary().c_str());
            if (gestures_on) {
                std::printf("        pinches: left %u right %u (held back, palm down: %d %d)  grips: left %u right %u",
                            pinch.side(0).begins, pinch.side(1).begins, pinch.held_back[0], pinch.held_back[1],
@@ -632,11 +782,13 @@ int main(int argc, char **argv) {
        }
    }
    if (!color.empty()) unlink(want_file.c_str());
+    if (write_sides_file) unlink(sides_file.c_str());
+    write_rec_sides();
    if (publish) pub.write({}, mono_ns());
    if (publish && gestures_on) {
        pinch.release(int64_t(mono_ns()));   // a drag in progress ends, as lost
        grip.release(int64_t(mono_ns()));
        gestures.write(pinch, grip, mono_ns());
    }
-    return 0;
+    return exit_code;
 }
@@ -57,6 +57,7 @@ void Recorder::add(const std::vector<SetFrame> &frames) {
            return;
        }
        queue_.push_back(std::move(rec));
+        ++added_;
    }
    wake_.notify_one();
 }
@@ -42,6 +42,9 @@ public:
    ~Recorder();
    bool open(const std::string &dir, std::string &err);
    void add(const std::vector<SetFrame> &frames);
+    // Sets taken (not dropped for a full queue): the next one's place in the file, unless a
+    // write fails.
+    size_t added() const { return added_; }
    size_t written() const { return written_; }
    size_t dropped() const { return dropped_; }

@@ -53,7 +56,7 @@ private:
    std::condition_variable wake_;
    std::deque<std::vector<uint8_t>> queue_;
    bool stop_ = false;
-    size_t written_ = 0, dropped_ = 0;
+    size_t written_ = 0, dropped_ = 0, added_ = 0;
 };

 // Reads a recording back one set at a time.
@@ -3,6 +3,7 @@
 //
 //   ft-handreplay DIR [--oracle N] [--slow F] [--timeline FILE] [--threads N] [--models DIR]
 //             [--from S] [--to S] [--contrast MODE|PALM/HAND] (clahe[:CLIP], none, stretch)
+//             [--sides file|0|1|auto]
 //
 // --oracle N: every N-th set, also search every tile of every camera (slow), to see
 //             which hands were there to find. Compares that with what the tracker had.
@@ -27,18 +28,29 @@
 //             views, hand scale, then its 21 world landmarks (the model's own 3D pose, averaged
 //             over its views, times the scale; metres, hand-centred) and its 21 published
 //             points (head frame). For studying gestures (pinch against typing, a fist).
+// --sides file|0|1|auto: the side cameras' names as DIR/sides.json says they should be (file, the
+//             default: ft-hands writes one with every recording; hands/rec/sides.py has the rule;
+//             without one, as recorded), as recorded (0), exchanged (1), or as ft-hands' auto
+//             decides them from the recorded names (track/sides.h): exchanged from the set it decides on,
+//             the tracker's views moving with their images (Tracker::exchange). The side check runs in every mode, and the report
+//             says what it found and when: seconds into the recording, and after how long of
+//             hands (from the first set with a hand in view).
 // --depth FILE: per processed set, a line per hand for tools/depth_report.py: its views'
 //             cameras, triangulation residual, hand scale, measured and published palm, and
 //             each view's one-view palm (Tracker::single_view at the hand's scale). The
 //             header has each camera's centre and focal length.
 #include "pinch.h"
 #include "record.h"
+#include "sides.h"
 #include "tracker.h"

+#include <json/json.h>
+
 #include <algorithm>
 #include <chrono>
 #include <cstdio>
 #include <cstring>
+#include <fstream>
 #include <map>
 #include <set>
 #include <string>
@@ -76,7 +88,7 @@ int main(int argc, char **argv) {
    double keep_presence = 0.5;   // landmark presence a tracked view needs to stay
    PinchParams pinch_params;
    GripParams grip_params;
-    std::string use = "mono", color_left = "color_video0", color_crop = "subtract";
+    std::string use = "mono", color_left = "color_video0", color_crop = "subtract", sides = "file";
    std::string timeline, depth, poses, models = std::string(argv[0]).substr(0, std::string(argv[0]).rfind('/') + 1) + "../models/ncnn";
    for (int i = 2; i < argc; ++i) {
        const std::string a = argv[i];
@@ -91,6 +103,7 @@ int main(int argc, char **argv) {
        else if (a == "--cost") cost = true;
        else if (a == "--keep-presence" && more) keep_presence = std::atof(argv[++i]);
        else if (a == "--cams" && more) use = argv[++i];
+        else if (a == "--sides" && more) sides = argv[++i];
        else if (a == "--pinch-begin" && more) pinch_params.begin_m = std::atof(argv[++i]);
        else if (a == "--pinch-end" && more) pinch_params.end_m = std::atof(argv[++i]);
        else if (a == "--pinch-triangulated") pinch_params.triangulated = true;
@@ -120,6 +133,24 @@ int main(int argc, char **argv) {
    std::vector<std::vector<uint8_t>> px;
    if (!in.next(cams, px)) return std::fprintf(stderr, "%s: no sets\n", dir.c_str()), 1;
    if (use != "mono" && use != "color" && use != "all") return std::fprintf(stderr, "--cams mono|color|all\n"), 1;
+    if (sides != "file" && sides != "0" && sides != "1" && sides != "auto")
+        return std::fprintf(stderr, "--sides file|0|1|auto\n"), 1;
+    // --sides file: from which set on the recorded names need exchanging (hands/rec/sides.py)
+    std::vector<std::pair<int, bool>> file_runs = {{0, false}};
+    if (sides == "file") {
+        std::ifstream f(dir + "/sides.json");
+        Json::Value v;
+        Json::CharReaderBuilder b;
+        std::string e;
+        if (f && Json::parseFromStream(b, f, &v, &e) && v["swapped"].isBool()) {
+            file_runs.clear();
+            for (const Json::Value &r : v["names_swapped"])
+                file_runs.push_back({r[0].asInt(), r[1].asBool() != v["swapped"].asBool()});
+            if (file_runs.empty()) file_runs = {{0, v["swapped"].asBool()}};
+            std::printf("side cameras: %s.json says swapped %s (%s)\n", (dir + "/sides").c_str(),
+                        v["swapped"].asBool() ? "true" : "false", v["decided_by"].asString().c_str());
+        }
+    }
    if (use != "mono") {
        std::vector<std::string> nodes;
        for (auto &c : cams)
@@ -136,6 +167,21 @@ int main(int argc, char **argv) {
    }
    Pool pool(threads, {2, 3, 4});
    Tracker tracker(used, nets, pool);
+    // The side cameras (see the top): names_swapped exchanges slam_left's and slam_right's names
+    // as sets are read.
+    bool names_swapped = sides == "1";
+    std::map<std::string, Camera> mono;
+    for (auto &[name, c] : used)
+        if (name.rfind("color_", 0) != 0) mono[name] = c;
+    SideCheck side_check(mono);
+    bool checking = side_check.usable();
+    int side_round = 0;
+    double first_hand_ts = -1;
+    std::string side_report;
+    auto rename = [&](const std::string &n) -> std::string {
+        if (!names_swapped) return n;
+        return n == "slam_left" ? "slam_right" : n == "slam_right" ? "slam_left" : n;
+    };
    tracker.set_keep_presence(keep_presence);
    FILE *dp = depth.empty() ? nullptr : std::fopen(depth.c_str(), "w");
    FILE *pp = poses.empty() ? nullptr : std::fopen(poses.c_str(), "w");
@@ -164,13 +210,16 @@ int main(int argc, char **argv) {
    double grip_begin_ts[2] = {0, 0};
    std::vector<double> grip_len[2];
    do {
+        if (sides == "file")
+            for (auto &[first, rename] : file_runs)
+                if (index + 1 >= first) names_swapped = rename;
        std::map<std::string, Image> images;
        // the set's time: the mono cameras' when they're used (the color ones run on another
        // clock); color frames can repeat across sets, so a set that doesn't move time on is skipped
        uint64_t t = UINT64_MAX, t_color = UINT64_MAX;
        for (size_t i = 0; i < cams.size(); ++i) {
            if (!used.count(cams[i].name)) continue;
-            images[cams[i].name] = {px[i].data(), int(cams[i].width), int(cams[i].height), int(cams[i].width)};
+            images[rename(cams[i].name)] = {px[i].data(), int(cams[i].width), int(cams[i].height), int(cams[i].width)};
            uint64_t &ti = std::string(cams[i].name).rfind("color_", 0) == 0 ? t_color : t;
            ti = std::min(ti, cams[i].capture_ns);
        }
@@ -199,6 +248,30 @@ int main(int argc, char **argv) {
            busy_ms += ms;
            busy_until = t + uint64_t(ms * slow * 1e6) + 3'000'000;   // + the ring hand-off
            const std::vector<Seen> seen = tracker.views_now();
+            if (first_hand_ts < 0 && !seen.empty()) first_hand_ts = ts;
+            std::vector<Seen> side_views = seen;
+            if (checking)
+                for (Seen &v : side_check.probe(nets, pool, images, seen, int64_t(t))) side_views.push_back(v);
+            if (checking && side_check.add(side_views, int64_t(t)) > 0 && side_check.verdict() != SideCheck::Undecided) {
+                const bool backwards = side_check.verdict() == SideCheck::Swapped;
+                char line[400];
+                std::snprintf(line, sizeof line, "side cameras: %s %s at %.1f s (%.1f s after the first hand; %s)\n",
+                              side_round ? "check" : "decision",
+                              backwards ? (sides == "auto" ? "SWAPPED, exchanged" : "SWAPPED") : "as named", ts,
+                              ts - first_hand_ts, side_check.summary().c_str());
+                side_report += line;
+                if (tl) std::fprintf(tl, "%.3f %s", ts, line);
+                if (sides != "auto") {
+                    checking = false;   // file, 0, 1: report only
+                } else if (side_round == 0 || backwards) {
+                    if (backwards) names_swapped = !names_swapped, tracker.exchange("slam_left", "slam_right");
+                    side_check.reset();
+                    side_check.min_clean = 20, side_check.min_votes = 40;
+                    checking = ++side_round < 3;
+                } else {
+                    checking = false;
+                }
+            }
            grip.update(out, seen, int64_t(t));
            pinch.update(out, seen, int64_t(t), grip.gripping());
            next_ns = t + uint64_t((std::min(tracker.interval(), pinch.engaged() || grip.engaged() ? 1 / 30.0 : 1.0) - 0.005) * 1e9);
@@ -361,6 +434,12 @@ int main(int argc, char **argv) {
        std::printf("palm jitter (off a straight line through the last two updates): measured median %.1f mm, 90%% %.1f mm; "
                    "published median %.1f mm, 90%% %.1f mm\n", jit_raw[jit_raw.size() / 2], jit_raw[jit_raw.size() * 9 / 10],
                    jit_sm[jit_sm.size() / 2], jit_sm[jit_sm.size() * 9 / 10]);
+    if (!side_check.usable())
+        std::printf("side cameras: not both in this recording\n");
+    else
+        std::printf("%sside cameras %s: %s\n", side_report.c_str(), side_report.empty() ? "undecided" : "at the end",
+                    checking || side_report.empty() ? side_check.summary().c_str()
+                                                    : names_swapped ? "exchanged from the recorded names" : "as recorded");
    if (o_sets) {
        std::printf("oracle, %d sets: a left hand findable in %d, the tracker had it in %d (%.0f%%); right %d, had %d (%.0f%%)\n",
                    o_sets, o_left, o_left_hit, 100.0 * o_left_hit / std::max(o_left, 1), o_right, o_right_hit,
@@ -0,0 +1,216 @@
+#include "sides.h"
+
+#include <algorithm>
+#include <cmath>
+#include <cstdio>
+#include <cstdlib>
+#include <functional>
+
+namespace {
+
+bool is_side(const std::string &name) { return name == "slam_left" || name == "slam_right"; }
+
+double median(std::vector<double> v) {
+    if (v.empty()) return -1;
+    std::nth_element(v.begin(), v.begin() + v.size() / 2, v.end());
+    return v[v.size() / 2];
+}
+
+constexpr int kMaxPairs = 12;          // per step
+constexpr int kMinFront = 15;          // of 21 landmarks: rays meeting in front of both cameras
+constexpr double kNear = 0.05, kFar = 1.5;      // m along each ray: where a hand can be
+constexpr double kRatioLo = 0.6, kRatioHi = 1.9;   // as Tracker's size check (tracker.cpp)
+constexpr double kProbeDepths[] = {0.8, 1.0, 1.25};   // probe(): times the one-view distance
+
+}  // namespace
+
+SideCheck::SideCheck(const std::map<std::string, Camera> &cams) {
+    for (const auto &[name, cam] : cams) cams_[name] = &cam;
+    if (cams_.count("slam_left")) left_ = cams_["slam_left"];
+    if (cams_.count("slam_right")) right_ = cams_["slam_right"];
+}
+
+const Camera *SideCheck::cam(const std::string &name, bool swapped) const {
+    if (swapped && name == "slam_left") return right_;
+    if (swapped && name == "slam_right") return left_;
+    const auto it = cams_.find(name);
+    return it == cams_.end() ? nullptr : it->second;
+}
+
+double SideCheck::miss(const Camera &ca, const Landmarks &a, const Camera &cb, const Landmarks &b) const {
+    std::vector<double> d;
+    V3 palm{};
+    bool have_palm = false;
+    for (int k = 0; k < 21; ++k) {
+        const V3 oa = ca.origin, da = ca.ray(a.pts[k]), ob = cb.origin, db = cb.ray(b.pts[k]);
+        const V3 w = oa - ob;
+        const double ab = dot(da, db), p = dot(da, w), q = dot(db, w), den = 1 - ab * ab;
+        if (den < 1e-9) continue;   // parallel rays
+        const double ta = (ab * q - p) / den, tb = (q - ab * p) / den;
+        if (ta < kNear || tb < kNear || ta > kFar || tb > kFar) continue;
+        const V3 pa = oa + da * ta, pb = ob + db * tb;
+        d.push_back(norm(pa - pb));
+        if (k == 9) palm = (pa + pb) * 0.5, have_palm = true;
+    }
+    if (int(d.size()) < kMinFront || !have_palm) return -1;
+    // as far from each camera as the hand's apparent size says (a hand of the model's size)
+    for (const auto &[c, lm] : {std::pair<const Camera *, const Landmarks *>{&ca, &a}, {&cb, &b}}) {
+        V3 mono[21];
+        if (!Tracker::single_view(*c, *lm, 1.0, mono)) return -1;
+        const double r = norm(palm - c->origin) / std::max(norm(mono[9] - c->origin), 1e-6);
+        if (r < kRatioLo || r > kRatioHi) return -1;
+    }
+    return median(d);
+}
+
+SideCheck::Miss SideCheck::test(const std::string &cam_a, const Landmarks &a, const std::string &cam_b,
+                                const Landmarks &b) const {
+    Miss out{{-1, -1}};
+    for (int way = 0; way < 2; ++way) {
+        const Camera *ca = cam(cam_a, way == 1), *cb = cam(cam_b, way == 1);
+        if (ca && cb) out.m[way] = miss(*ca, a, *cb, b);
+    }
+    return out;
+}
+
+int SideCheck::vote(const Miss &m) {
+    for (int w = 0; w < 2; ++w) {
+        const double win = m.m[w], other = m.m[1 - w];
+        if (win >= 0 && win < kGood && (other < 0 || other > kClear * win)) return w;
+    }
+    return -1;
+}
+
+int SideCheck::add(const std::vector<Seen> &views, int64_t t_ns) {
+    if (!usable()) return 0;
+    std::vector<const Seen *> vs;
+    for (const Seen &v : views)
+        if (cams_.count(v.cam)) vs.push_back(&v);
+    int pairs = 0, voted = 0;
+    for (size_t i = 0; i < vs.size(); ++i)
+        for (size_t j = i + 1; j < vs.size() && pairs < kMaxPairs; ++j) {
+            if (vs[i]->cam == vs[j]->cam || (!is_side(vs[i]->cam) && !is_side(vs[j]->cam))) continue;
+            ++pairs;
+            const Miss m = test(vs[i]->cam, vs[i]->lm, vs[j]->cam, vs[j]->lm);
+            const int v = vote(m);
+            if (v < 0) continue;
+            ++votes[v], ++voted;
+            misses_[0].push_back(m.m[0]), misses_[1].push_back(m.m[1]);
+            if (first_ns_ < 0) first_ns_ = t_ns;
+            last_ns_ = t_ns;
+        }
+    return voted;
+}
+
+std::vector<Seen> SideCheck::probe(const Nets &nets, Pool &pool, const std::map<std::string, Image> &images,
+                                   const std::vector<Seen> &views, int64_t t_ns) {
+    if (!usable() || (probe_ns_ >= 0 && (t_ns - probe_ns_) / 1e9 < probe_interval_s - 0.01)) return {};
+    struct Job {
+        int way;
+        std::string cam;
+        Roi roi;
+        Landmarks lm;
+    };
+    std::vector<Job> jobs;
+    // the views to look from: confident ones, a different hand each time (round robin)
+    std::vector<const Seen *> order;
+    for (const Seen &v : views)
+        if (cams_.count(v.cam) && v.lm.presence >= 0.5) order.push_back(&v);
+    std::sort(order.begin(), order.end(), [](const Seen *a, const Seen *b) { return a->lm.presence > b->lm.presence; });
+    if (!order.empty()) std::rotate(order.begin(), order.begin() + (probe_turn_++ % order.size()), order.end());
+    // Only the other naming: the tracker hands hands over to the other cameras under the names
+    // as they are every step, and add() tests what that finds.
+    for (int way = 1; way < 2; ++way) {
+        bool found = false;
+        for (const Seen *v : order) {
+            for (const char *target : {"slam_left", "slam_right"}) {
+                if (v->cam == target || !images.count(target)) continue;
+                bool paired = false;   // the tracker has this hand there already: add() tests that pair
+                for (const Seen &u : views) paired = paired || (u.cam == target && u.hand == v->hand && v->hand > 0);
+                if (paired) continue;
+                const Camera *cs = cam(v->cam, way == 1), *ct = cam(target, way == 1);
+                V3 pts[21];
+                if (!cs || !ct || !Tracker::single_view(*cs, v->lm, 1.0, pts)) continue;
+                // the hand at a few distances around the one-view guess, as the tracker hands
+                // a hand over to another camera (a crop around where its landmarks land)
+                const V3 axis{ct->R[0][2], ct->R[1][2], ct->R[2][2]}, o = cs->origin;
+                for (double f : kProbeDepths) {
+                    V2 uv[21];
+                    bool visible = true;
+                    for (int k = 0; k < 21; ++k) {
+                        const V3 p = o + (pts[k] - o) * f;
+                        visible = visible && dot(unit(p - ct->origin), axis) > 0.17;   // within 80 degrees
+                        uv[k] = ct->project(p, nullptr);
+                    }
+                    const V2 c = uv[9];
+                    if (!visible || c[0] < 0 || c[1] < 0 || c[0] >= ct->width || c[1] >= ct->height) continue;
+                    jobs.push_back({way, target, roi_from_points(uv), {}});
+                    found = true;
+                }
+                if (found) break;
+            }
+            if (found) break;
+        }
+    }
+    if (jobs.empty()) return {};
+    probe_ns_ = t_ns;
+    std::vector<std::function<void()>> run;
+    for (Job &j : jobs) run.push_back([&nets, &images, &j] { j.lm = nets.landmarks(images.at(j.cam), j.roi); });
+    pool.run(run);
+    static const bool debug = std::getenv("FT_SIDES_DEBUG") != nullptr;
+    std::vector<Seen> out;
+    probes += int(jobs.size());
+    for (int way = 1; way < 2; ++way) {   // the best look
+        const Job *best = nullptr;
+        for (const Job &j : jobs)
+            if (j.way == way && (!best || j.lm.presence > best->lm.presence)) best = &j;
+        if (debug && best)
+            std::fprintf(stderr, "side probe (%s) %s at %.0f %.0f size %.0f: presence %.2f\n", way ? "swapped" : "as named",
+                         best->cam.c_str(), best->roi.center[0], best->roi.center[1], best->roi.size, best->lm.presence);
+        if (!best || best->lm.presence < 0.5) continue;
+        ++probe_hits;
+        out.push_back(Seen{best->cam, 0, best->roi, best->lm, {}});
+    }
+    return out;
+}
+
+SideCheck::Verdict SideCheck::verdict() const {
+    const int total = votes[0] + votes[1], w = votes[0] >= votes[1] ? 0 : 1;
+    const bool clean = votes[1 - w] == 0 && votes[w] >= min_clean;
+    const bool clear = votes[w] >= min_votes && votes[1 - w] <= max_other * total;
+    if (!(clean || clear) || span_s() < min_span_s) return Undecided;
+    return w == 0 ? AsNamed : Swapped;
+}
+
+void SideCheck::reset() {
+    votes[0] = votes[1] = 0;
+    probes = probe_hits = 0;
+    probe_ns_ = -1;
+    misses_[0].clear(), misses_[1].clear();
+    first_ns_ = last_ns_ = -1;
+}
+
+double SideCheck::median_miss_mm(int way) const {
+    std::vector<double> v;
+    for (double m : misses_[way])
+        if (m >= 0) v.push_back(m * 1000);
+    return median(v);
+}
+
+std::string SideCheck::summary() const {
+    char s[320];
+    std::snprintf(s, sizeof s,
+                  "votes as named %d, swapped %d, over %.1f s; median ray miss as named %.1f mm, swapped %.1f mm "
+                  "(-1: never met); probes %d, found %d",
+                  votes[0], votes[1], span_s(), median_miss_mm(0), median_miss_mm(1), probes, probe_hits);
+    return s;
+}
+
+std::string SideCheck::json() const {
+    char s[320];
+    std::snprintf(s, sizeof s,
+                  "{\"as_named\": %d, \"swapped\": %d, \"seconds\": %.2f, \"miss_mm\": [%.1f, %.1f], \"probes\": %d, "
+                  "\"found\": %d}",
+                  votes[0], votes[1], span_s(), median_miss_mm(0), median_miss_mm(1), probes, probe_hits);
+    return s;
+}
@@ -0,0 +1,94 @@
+// Are the side cameras' images under the right names? ft-camd tells slam_left's buffers from
+// slam_right's only by the order XRService allocated them, and some XRService starts reverse
+// that order: then each side camera's images carry the other's name, every hand is seen by one
+// camera only, at the wrong depth, and the cutouts land beside the hands.
+//
+// SideCheck tells from the hands the tracker already finds. Whenever a hand's landmarks are
+// found in two cameras in the same frame set (one of them a side camera), the rays through its
+// 21 landmarks are intersected twice: with the calibrations as the images are named, and with
+// the two side cameras' calibrations exchanged. The same hand seen right meets within a few mm,
+// in front of both cameras and as far away as its apparent size says; under the wrong naming
+// the rays miss by centimetres or meet behind a camera. Each such pair is a vote. It decides once
+// one way has min_clean votes and the other none, or min_votes with at most max_other of all
+// votes the other way, over at least min_span_s of hands.
+//
+// The tracker's own views rarely give a pair when the names are wrong: it hands a hand over
+// to the other cameras where the wrong calibration puts it, finds nothing there, and keeps it in
+// one camera (the replays of swapped recordings: one hand, every handoff a miss, no pair). So
+// probe() looks for itself, 5 times a second while it's checking: it takes a hand the tracker
+// sees, places it in 3D from that one view (as far as its size says) with the side cameras
+// exchanged, and runs the landmark model where that puts it in the other side camera, at 0.8,
+// 1 and 1.25 times the one-view distance. That's the tracker's handoff under the other naming
+// (the tracker does it under the current one every step); the palm detector misses many of
+// these hands, which the landmark model finds from a good crop. What it finds is a view
+// like the tracker's, and the test above votes on it: finding a hand there proves nothing by
+// itself. It doesn't depend on how the tracker paired the views up.
+//
+// FT_SIDES_DEBUG=1 in the environment prints each probe's crop and what it found (stderr).
+//
+// Upper-camera pairs with a side camera count too (the upper pair's own naming was checked
+// stable: tools/check_sides.py --pair upper). Two upper views alone say nothing.
+#pragma once
+
+#include "tracker.h"
+
+#include <map>
+#include <string>
+#include <vector>
+
+class SideCheck {
+public:
+    enum Verdict { Undecided, AsNamed, Swapped };
+
+    // cams: the mono cameras' calibrations; slam_left and slam_right are needed.
+    explicit SideCheck(const std::map<std::string, Camera> &cams);
+    bool usable() const { return left_ && right_; }
+
+    // One step's fresh views (Tracker::views_now, plus probe()'s), cameras named as the
+    // images are now. Returns how many pairs voted.
+    int add(const std::vector<Seen> &views, int64_t t_ns);
+    // At most every probe_interval_s: for each naming, one view the tracker has, looked for in
+    // the other side camera where that naming puts it (see the top). Returns the views found
+    // (presence 0.5 or more), for add() with the tracker's. Runs the models on pool.
+    std::vector<Seen> probe(const Nets &nets, Pool &pool, const std::map<std::string, Image> &images,
+                            const std::vector<Seen> &views, int64_t t_ns);
+    double probe_interval_s = 0.2;
+    int probes = 0, probe_hits = 0;   // landmark model runs, and the views they found
+
+    // One pair: the median landmark ray miss (m) as named [0] and with the side cameras
+    // exchanged [1]; -1 where they can't be one hand that way (rays meet behind a camera, or
+    // at a distance the hand's apparent size rules out). For tests.
+    struct Miss {
+        double m[2];
+    };
+    Miss test(const std::string &cam_a, const Landmarks &a, const std::string &cam_b, const Landmarks &b) const;
+    // The vote for a Miss: 0 as named, 1 swapped, -1 neither.
+    static int vote(const Miss &m);
+
+    Verdict verdict() const;
+    void reset();
+
+    // Evidence so far
+    int votes[2] = {0, 0};
+    double span_s() const { return first_ns_ >= 0 ? (last_ns_ - first_ns_) / 1e9 : 0; }
+    double median_miss_mm(int way) const;   // over the voted pairs; -1 if none met that way
+    std::string summary() const;            // for the log
+    std::string json() const;               // for the sides file and recordings
+
+    // The rule (see the top)
+    int min_clean = 10, min_votes = 20;
+    double min_span_s = 1.0, max_other = 0.2;
+    static constexpr double kGood = 0.015;   // m: a pair's median miss to count as meeting
+    static constexpr double kClear = 3.0;    // both meet: one must miss this many times less
+
+private:
+    const Camera *cam(const std::string &name, bool swapped) const;
+    double miss(const Camera &ca, const Landmarks &a, const Camera &cb, const Landmarks &b) const;
+
+    std::map<std::string, const Camera *> cams_;
+    const Camera *left_ = nullptr, *right_ = nullptr;
+    std::vector<double> misses_[2];   // per voted pair, each way (-1: implausible)
+    int64_t first_ns_ = -1, last_ns_ = -1;   // the first and last vote
+    int64_t probe_ns_ = -1;
+    unsigned probe_turn_ = 0;
+};
@@ -162,7 +162,7 @@ void Tracker::run_landmarks(const std::map<std::string, Image> &images, std::vec
    ++stats.hand_batches;
 }

-bool Tracker::single_view(const Camera &cam, const Landmarks &lm, double scale, V3 out[21]) const {
+bool Tracker::single_view(const Camera &cam, const Landmarks &lm, double scale, V3 out[21]) {
    V3 rays[21];
    for (int i = 0; i < 21; ++i) rays[i] = cam.ray(lm.pts[i]);
    std::vector<std::pair<double, double>> est;   // (depth, weight)
@@ -556,6 +556,13 @@ std::vector<Seen> Tracker::views_now() const {
    return out;
 }

+void Tracker::exchange(const std::string &a, const std::string &b) {
+    if (!cams_.count(a) || !cams_.count(b)) return;
+    const Camera *ca = cams_[a], *cb = cams_[b];
+    for (View &v : views_) v.cam = v.cam == ca ? cb : v.cam == cb ? ca : v.cam;
+    for (auto &[id, h] : hands_) h.has_pts = false, h.frames = 0, h.residual = -1;
+}
+
 void Tracker::drop_camera(const std::string &name) {
    views_.erase(std::remove_if(views_.begin(), views_.end(), [&](const View &v) { return v.cam->name == name; }),
                 views_.end());
@@ -95,7 +95,11 @@ public:
    void set_keep_presence(double p) { keep_presence_ = p; }
    // One view's 3D hand: each landmark along its ray, as far as how big the palm looks says
    // for a hand `scale` times the model's (Hand::scale). False if the palm is degenerate.
-    bool single_view(const Camera &cam, const Landmarks &lm, double scale, V3 out[21]) const;
+    static bool single_view(const Camera &cam, const Landmarks &lm, double scale, V3 out[21]);
+    // Cameras a and b's images were under each other's names (the side cameras, track/sides.h):
+    // each view moves to the other camera (its crop is in the image's pixels, so it keeps
+    // following its hand), and every hand's 3D starts over from its views at the next step.
+    void exchange(const std::string &a, const std::string &b);

 private:
    struct View {