Compare commits

...
2 Commits
Author SHA1 Message Date
DeeJanuzandClaude Opus 5.5 0821b1013e Settings: HANDS_SWAP_SIDES defaults to auto, and old untouched configs move to it
frametop.conf.example said HANDS_SWAP_SIDES=0, and desktops.sh copies it on a
fresh install, so every new install forced ft-camd's side camera names and
turned the hand tracker's own side check off. Some SteamVR restarts swap those
names, and then hands land beside their cutouts and recordings are mislabelled.

The example now says auto. scripts/conf-migrate.sh, run by install.sh and
hands/rec/install.sh, replaces the old line only where it's still exactly as
the example wrote it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:54:03 -06:00
DeeJanuzandClaude Opus 5.5 3cccad3525 Hands: label side cameras by the hands when a forced HANDS_SWAP_SIDES disagrees
With HANDS_SWAP_SIDES set to 0 or 1, ft-hands kept the forced naming as the
published truth even after the hands showed it was backwards. The hand recorder
took that as the session's decision, so the export labelled slam_left and
slam_right the wrong way round (dataset PR #4: every take swapped).

ft-hands now publishes the hands' answer once they disagree (state "forced,
disagrees"); tracking keeps the forced names. sides.read_live corrects the same
case from an ft-hands built before this, and the recorder's log says to use auto.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-05 16:54:03 -06:00
11 changed files with 84 additions and 7 deletions

No files matched your search

+1 -1
View File
@@ -263,7 +263,7 @@ Your hands show over the screens: where a tracked hand is between an eye and a s
- `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`.
- `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches and grips, read by the pointer helper) next to the ring.
- The install leaves both off, and they don't start with SteamVR. `ft-handsctl on` starts them while SteamVR runs, and `ft-handsctl off` stops them; they also stop with SteamVR. The install links `ft-handsctl` into `~/.local/bin`. `ft-handsctl status` and `ft-handsctl log` (or `hands/run.sh status` and `log`) show how they're doing, `ft-handsctl cutouts on|off` turns just the cutouts off, and `ft-handsctl gestures` shows pinches and grips live.
- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (after some SteamVR restarts the side cameras' names come out swapped, and hands land beside the holes; `hands/tools/check_sides.py --ring` tells), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, and `POINTER_PINCH_TYPING`. The example config explains each.
- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (`auto`, the default: ft-hands tells from the hands when some SteamVR restart has swapped the side cameras' names, and fixes them; `0` or `1` force them, and `hands/tools/check_sides.py --ring` tells which is right), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, and `POINTER_PINCH_TYPING`. The example config explains each.
Details, options, and the recording and replay tools are in [hands/README.md](../hands/README.md).
+1 -1
View File
@@ -36,7 +36,7 @@ hands/run.sh uninstall
Settings in `~/.config/frametop.conf` (`FT_<name>` in the environment overrides them), read when ft-camd and ft-hands start:
- `HANDS_SWAP_SIDES=auto` (the default): ft-hands tells from the hands which side camera is which, and corrects ft-camd's names when they're backwards (see "Which camera is which" below). `1` forces them exchanged and `0` forces ft-camd's names; ft-hands still checks and logs a warning if the hands disagree.
- `HANDS_SWAP_SIDES=auto` (the default): ft-hands tells from the hands which side camera is which, and corrects ft-camd's names when they're backwards (see "Which camera is which" below). `1` forces them exchanged and `0` forces ft-camd's names; ft-hands still checks, and if the hands disagree it logs a warning and publishes the hands' answer as the truth (`sides.json`), so recordings are labelled right. The example config said `0` until 2026-10-05; `scripts/conf-migrate.sh` (run by `install.sh` and `hands/rec/install.sh`) turns that untouched line into `auto`.
- `HANDS_CPUS=5,6,7`: the CPUs the model threads run on (below).
- `HANDS_CAMERAS` (`auto`), `HANDS_BRIGHT` (`all`), `HANDS_BRIGHT_ON` (40), `HANDS_BRIGHT_OFF` (25): which cameras ft-hands tracks with, as `--cams`, `--bright`, `--bright-on` and `--bright-off` (see ft-hands). `HANDS_CAMERAS=mono` also keeps ft-camd off the colour cameras.
- `HANDS_COLOR_LEFT` (`color_video0`), `HANDS_COLOR_CROP` (`subtract`): how the colour module's calibration maps onto its images, as `--color-left` and `--color-crop`.
+1 -1
View File
@@ -29,7 +29,7 @@ Every part runs in the dev container, as ft-hands and Input Settings do. The hos
- **A tracking ft-hands** gives feedback through the hands file: which hands are seen, the palm's distance, the index tip. If none is running, the session starts `ft-hands --no-gestures --status 0` (unit `frametop-handrec-hands.service`). An ft-hands already running is used as it is.
- **A recording ft-hands** runs once per recording part: `ft-hands --record-only --record DIR --record-for SECONDS --record-hz 10 --status 0 --sides auto|0|1` (below, "Side cameras"). It runs as a plain child process of the session, ended with SIGTERM when the part ends. SIGTERM ends ft-hands' loop, and `Recorder` writes out its queue when it's destroyed. In step mode (below) a part is one step's countdown and hold, so a take has one part per step (about 40 in the hand poses); in auto mode a take is one part, plus one more after each pause. `--record-for` is only a safety net.
- Why a process per part rather than one kept alive and paused: measured in the dev container with `ft-ringplay`'s ring (2026-10-02), `ft-hands --record-only` writes its first set 16-27 ms after it starts and ends 4-6 ms after SIGTERM, so a new part costs nothing the 3 s countdown doesn't cover. Every reader already takes parts in order (`takes.py`, `validate.py` through the export's single stream, the labeller's `fhl_io.py`, numbering `sets-10.bin` after `sets-9.bin`), ft-hands needs no new control, and nothing is written while a step waits. Before the hold starts the session checks that the part has written a set (`Recorder.has_data`, up to 3 s more), so the hold is recorded from its first frame.
- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand.
- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. With `HANDS_SWAP_SIDES` forced to `0` or `1`, the hands still decide what's published once they disagree (state `"forced, disagrees"`; `read_live` also corrects an ft-hands built before that). The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand.
- **ft-handpanel** runs as a child process with `--watch-stdin`. It shows the panel and logs poses during each take.
- **The headset button's reader** is a thread of the session (`ButtonReader`, below), not a process.
+1
View File
@@ -23,6 +23,7 @@ case ${1:-install} in
"$root/hands/rec/build.sh"
echo "== 4/4 ft-camd's capabilities (asks for your password) and the menu entry"
"$root/hands/run.sh" caps
"$root/scripts/conf-migrate.sh" # HANDS_SWAP_SIDES=0, the old default, becomes auto
fill_template "$root/hands/rec/ft-handrec.desktop" |
on_frame "mkdir -p ~/.local/share/applications && cat > $entry"
echo
+3
View File
@@ -1575,6 +1575,9 @@ class Session:
self._save_session()
self._log("side cameras: %s (%s, %s)" % ("SWAPPED" if swapped else "as named", new["decided_by"],
json.dumps(new["evidence"])))
if new["state"] == "forced, disagrees":
self._log("side cameras: HANDS_SWAP_SIDES in ~/.config/frametop.conf forces names the hands say are "
"backwards; the recording goes by the hands. Set HANDS_SWAP_SIDES=auto.")
def _sides_swapped(self):
"""session.json's decision: True, False, or None (not known yet)."""
+5
View File
@@ -125,4 +125,9 @@ def read_live(path=None, ring_path=None, now_ns=None):
return None
except OSError:
return None
if (s.get("state") == "forced, disagrees" and s.get("swapped") is not None
and bool(s["swapped"]) == bool(s.get("names_swapped"))):
# An ft-hands built before 2026-10-06 kept a forced HANDS_SWAP_SIDES as the truth even
# when the hands disagreed. The hands are right: the truth is the other way round.
s = dict(s, swapped=not s["swapped"], decided_by="auto")
return s
+39
View File
@@ -91,6 +91,29 @@ class RulesTest(unittest.TestCase):
finally:
shutil.rmtree(d)
def test_read_live_forced(self):
"""HANDS_SWAP_SIDES=0 forced and the hands disagree: the hands are the truth, whether
ft-hands published them (built after 2026-10-06) or kept the forced value (before)."""
d = tempfile.mkdtemp()
try:
path = os.path.join(d, "sides.json")
def live(**kw):
s = dict(pid=1, ring_ino=0, mode="0", names_swapped=False,
updated_ns=time.clock_gettime_ns(time.CLOCK_MONOTONIC), **kw)
with open(path, "w") as f:
json.dump(s, f)
return sides.read_live(path)
self.assertEqual(live(state="forced", swapped=False, decided_by="config")["swapped"], False)
self.assertEqual(live(state="forced, agrees", swapped=False, decided_by="config")["swapped"], False)
old = live(state="forced, disagrees", swapped=False, decided_by="config")
self.assertEqual((old["swapped"], old["decided_by"]), (True, "auto"))
new = live(state="forced, disagrees", swapped=True, decided_by="auto")
self.assertEqual((new["swapped"], new["decided_by"]), (True, "auto"))
finally:
shutil.rmtree(d)
class TakesTest(unittest.TestCase):
"""A swapped session: one take's parts recorded before the decision (ft-camd's names) and
@@ -226,6 +249,22 @@ class SessionSidesTest(unittest.TestCase):
self.assertTrue(s._session_json["sides"]["reversed_from"]["swapped"])
self.assertFalse(takes.read_json(os.path.join(self.tmp, "session.json"))["sides"]["swapped"])
def test_read_sides_forced(self):
"""PR #4 on the dataset: HANDS_SWAP_SIDES=0 from the old example config, and the hands
disagree. The session takes the hands' answer, not the forced one."""
s = session.Session(os.path.join(self.tmp, "base"), {}, {}, "room", self.script, dry_run=True,
hands_dir=self.tmp)
s.session_dir = self.tmp
s._session_json = {"sides": {"swapped": None}}
self.live(mode="0", state="forced", swapped=False, decided_by="config", names_swapped=False, evidence=None)
s._read_sides(force=True)
self.assertIs(s._sides_swapped(), False)
self.live(mode="0", state="forced, disagrees", swapped=False, decided_by="config", names_swapped=False)
s._read_sides(force=True)
self.assertIs(s._sides_swapped(), True)
self.assertEqual(s._session_json["sides"]["decided_by"], "auto")
self.assertEqual(s._session_json["sides"]["reversed_from"]["decided_by"], "config")
def test_recorder_parts(self):
calls = []
+6 -3
View File
@@ -27,7 +27,8 @@
// default) tells from the hands it tracks (track/sides.h): once it's sure, it exchanges the two
// cameras if they're backwards (the tracked views move with their images), and checks once
// more. 0 and 1 force the naming (1: exchanged; --swap-sides is --sides 1); it still checks,
// and warns if the hands disagree. The decision is published in /run/user/UID/frametop-hands/sides.json (see
// and if the hands disagree it warns and publishes what the hands say as the truth ("swapped"),
// so recordings are labelled right while tracking keeps the forced names. The decision is published in /run/user/UID/frametop-hands/sides.json (see
// write_sides below) and, for recordings, in DIR/sides.json. --record-only can't tell (it tracks
// nothing): under auto it records the ring's names as they are.
//
@@ -507,14 +508,16 @@ int main(int argc, char **argv) {
const bool backwards = v == SideCheck::Swapped; // relative to the names as they are now
const double after = (now - start) / 1e9;
const std::string ev = side_check.json(), text = side_check.summary();
if (sides_mode != "auto") { // forced: only say so
if (sides_mode != "auto") { // forced: the names stay; if the hands disagree, they're the truth
const std::string what = (sides_from == "option" ? "--sides " : "HANDS_SWAP_SIDES=") + sides_mode;
if (backwards)
std::printf("side cameras: %s looks WRONG: the hands say the side cameras are the other way round (%s). "
"Use auto.\n", what.c_str(), text.c_str());
"Tracking keeps the forced names; recordings are labelled by the hands. Use auto.\n",
what.c_str(), text.c_str());
else
std::printf("side cameras: %s agrees with the hands (%s)\n", what.c_str(), text.c_str());
sides_state = backwards ? "forced, disagrees" : "forced, agrees";
if (backwards) truth = !names_swapped, decided_by = "auto", decided_after_s = after;
decision_evidence = ev;
checking = false;
} else if (side_round == 0 || backwards) {
+1
View File
@@ -93,6 +93,7 @@ step "7/10 multi-screen desktop (ft-screens), Frametop Input Settings, and Frame
"$root/remote/install.sh"
on_frame "sed -i 's/^POINTER=0/POINTER=1/' ~/.config/frametop.conf; grep -q '^POINTER=' ~/.config/frametop.conf || echo 'POINTER=1' >> ~/.config/frametop.conf"
echo "the launcher's Desktop entry now opens the multi-screen desktop; 3D mouse on (POINTER=1 in ~/.config/frametop.conf)"
"$root/scripts/conf-migrate.sh"
step "8/10 gaze mode (optional, experimental: the pointer goes where you look)"
gaze=0
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
# Update the lines of ~/.config/frametop.conf that still read exactly as an older
# frametop.conf.example wrote them. A line you changed stays as it is. install.sh and
# hands/rec/install.sh run this on every install and update.
# HANDS_SWAP_SIDES=0: the example's value until 2026-10-05. It made ft-hands keep ft-camd's side
# camera names even when they're backwards (some SteamVR restarts swap them), so hands landed
# beside their cutouts and the hand recorder labelled the side cameras wrong. auto tells from
# the hands.
set -euo pipefail
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
. "$root/scripts/_env.sh"
on_frame_script <<'EOF'
f=~/.config/frametop.conf
[ -f "$f" ] || exit 0
old="HANDS_SWAP_SIDES=0 # hand tracking (hands/run.sh install): 1 = the side cameras' names are swapped, which some SteamVR restarts cause (hands/tools/check_sides.py --ring tells)"
new="HANDS_SWAP_SIDES=auto # hand tracking: which side camera is which. auto = ft-hands tells from the hands and fixes ft-camd's names, which some SteamVR restarts swap | 0 = keep ft-camd's | 1 = exchange them"
if grep -qxF "$old" "$f"; then
tmp=$(mktemp)
awk -v old="$old" -v new="$new" '$0 == old { print new; next } { print }' "$f" > "$tmp"
cat "$tmp" > "$f" # in place: the file keeps its owner and mode
rm -f "$tmp"
echo "settings: HANDS_SWAP_SIDES=0 (the old default) is now auto: hand tracking tells the side cameras apart itself"
fi
EOF
+1 -1
View File
@@ -40,7 +40,7 @@ POINTER_HEAD_DEADZONE=0.5 # keyboard clicks at the gaze (Meta+J, Meta+K): degre
POINTER_KEY_TAP=0.25 # keyboard clicks: let go within this (s) and it clicks where the dot was at the press, and tells the gaze tracker it was right
GAZE_TRACKER=auto # gaze service: auto = our own eye tracker when it's installed (gaze/tracker/install.sh, which install.sh offers), else SteamVR's | own = ours | steam = SteamVR's. Each keeps its own calibration (Calibrate on Frametop Input Settings' Gaze page)
GAZE_EYE=auto # gaze service: eye bias. auto = each eye weighted by how far off it was at your recent nudges | left | right = that eye counts twice
HANDS_SWAP_SIDES=0 # hand tracking (hands/run.sh install): 1 = the side cameras' names are swapped, which some SteamVR restarts cause (hands/tools/check_sides.py --ring tells)
HANDS_SWAP_SIDES=auto # hand tracking: which side camera is which. auto = ft-hands tells from the hands and fixes ft-camd's names, which some SteamVR restarts swap | 0 = keep ft-camd's | 1 = exchange them
HANDS_CPUS=5,6,7 # hand tracking: the CPUs its model threads run on
HANDS_CAMERAS=auto # hand tracking: auto (by the light) | mono (the IR cameras; also keeps ft-camd off the colour ones) | color | all
HANDS_BRIGHT=all # hand tracking, auto: the cameras in bright light, all | color