mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-06 01:00:06 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0821b1013e | ||
|
|
3cccad3525 |
No files matched your search
+1
-1
@@ -263,7 +263,7 @@ Your hands show over the screens: where a tracked hand is between an eye and a s
|
||||
- `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`.
|
||||
- `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches and grips, read by the pointer helper) next to the ring.
|
||||
- The install leaves both off, and they don't start with SteamVR. `ft-handsctl on` starts them while SteamVR runs, and `ft-handsctl off` stops them; they also stop with SteamVR. The install links `ft-handsctl` into `~/.local/bin`. `ft-handsctl status` and `ft-handsctl log` (or `hands/run.sh status` and `log`) show how they're doing, `ft-handsctl cutouts on|off` turns just the cutouts off, and `ft-handsctl gestures` shows pinches and grips live.
|
||||
- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (after some SteamVR restarts the side cameras' names come out swapped, and hands land beside the holes; `hands/tools/check_sides.py --ring` tells), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, and `POINTER_PINCH_TYPING`. The example config explains each.
|
||||
- Settings in `~/.config/frametop.conf`: `HANDS_SWAP_SIDES` (`auto`, the default: ft-hands tells from the hands when some SteamVR restart has swapped the side cameras' names, and fixes them; `0` or `1` force them, and `hands/tools/check_sides.py --ring` tells which is right), `HANDS_CPUS`, the cameras it tracks with (`HANDS_CAMERAS`, `HANDS_BRIGHT`, `HANDS_BRIGHT_ON`, `HANDS_BRIGHT_OFF`, `HANDS_COLOR_LEFT`, `HANDS_COLOR_CROP`), and the pointer helper's `POINTER_HANDS`, `POINTER_PINCH_GAIN`, `POINTER_PINCH_DEADZONE`, `POINTER_GRIP_GAIN`, `POINTER_GRIP_BELOW`, and `POINTER_PINCH_TYPING`. The example config explains each.
|
||||
|
||||
Details, options, and the recording and replay tools are in [hands/README.md](../hands/README.md).
|
||||
|
||||
|
||||
+1
-1
@@ -36,7 +36,7 @@ hands/run.sh uninstall
|
||||
|
||||
Settings in `~/.config/frametop.conf` (`FT_<name>` in the environment overrides them), read when ft-camd and ft-hands start:
|
||||
|
||||
- `HANDS_SWAP_SIDES=auto` (the default): ft-hands tells from the hands which side camera is which, and corrects ft-camd's names when they're backwards (see "Which camera is which" below). `1` forces them exchanged and `0` forces ft-camd's names; ft-hands still checks and logs a warning if the hands disagree.
|
||||
- `HANDS_SWAP_SIDES=auto` (the default): ft-hands tells from the hands which side camera is which, and corrects ft-camd's names when they're backwards (see "Which camera is which" below). `1` forces them exchanged and `0` forces ft-camd's names; ft-hands still checks, and if the hands disagree it logs a warning and publishes the hands' answer as the truth (`sides.json`), so recordings are labelled right. The example config said `0` until 2026-10-05; `scripts/conf-migrate.sh` (run by `install.sh` and `hands/rec/install.sh`) turns that untouched line into `auto`.
|
||||
- `HANDS_CPUS=5,6,7`: the CPUs the model threads run on (below).
|
||||
- `HANDS_CAMERAS` (`auto`), `HANDS_BRIGHT` (`all`), `HANDS_BRIGHT_ON` (40), `HANDS_BRIGHT_OFF` (25): which cameras ft-hands tracks with, as `--cams`, `--bright`, `--bright-on` and `--bright-off` (see ft-hands). `HANDS_CAMERAS=mono` also keeps ft-camd off the colour cameras.
|
||||
- `HANDS_COLOR_LEFT` (`color_video0`), `HANDS_COLOR_CROP` (`subtract`): how the colour module's calibration maps onto its images, as `--color-left` and `--color-crop`.
|
||||
|
||||
+1
-1
@@ -29,7 +29,7 @@ Every part runs in the dev container, as ft-hands and Input Settings do. The hos
|
||||
- **A tracking ft-hands** gives feedback through the hands file: which hands are seen, the palm's distance, the index tip. If none is running, the session starts `ft-hands --no-gestures --status 0` (unit `frametop-handrec-hands.service`). An ft-hands already running is used as it is.
|
||||
- **A recording ft-hands** runs once per recording part: `ft-hands --record-only --record DIR --record-for SECONDS --record-hz 10 --status 0 --sides auto|0|1` (below, "Side cameras"). It runs as a plain child process of the session, ended with SIGTERM when the part ends. SIGTERM ends ft-hands' loop, and `Recorder` writes out its queue when it's destroyed. In step mode (below) a part is one step's countdown and hold, so a take has one part per step (about 40 in the hand poses); in auto mode a take is one part, plus one more after each pause. `--record-for` is only a safety net.
|
||||
- Why a process per part rather than one kept alive and paused: measured in the dev container with `ft-ringplay`'s ring (2026-10-02), `ft-hands --record-only` writes its first set 16-27 ms after it starts and ends 4-6 ms after SIGTERM, so a new part costs nothing the 3 s countdown doesn't cover. Every reader already takes parts in order (`takes.py`, `validate.py` through the export's single stream, the labeller's `fhl_io.py`, numbering `sets-10.bin` after `sets-9.bin`), ft-hands needs no new control, and nothing is written while a step waits. Before the hold starts the session checks that the part has written a set (`Recorder.has_data`, up to 3 s more), so the hold is recorded from its first frame.
|
||||
- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand.
|
||||
- **Side cameras.** ft-camd can name the two side cameras the wrong way round (hands/README.md, "Which camera is which"). The tracking ft-hands decides from the hands within about 2 s of them being in view (`HANDS_SWAP_SIDES=auto`, hands/track/sides.h) and publishes that in `/run/user/UID/frametop-hands/sides.json`. With `HANDS_SWAP_SIDES` forced to `0` or `1`, the hands still decide what's published once they disagree (state `"forced, disagrees"`; `read_live` also corrects an ft-hands built before that). The session reads it (`sides.py`, `read_live`) and stores it in session.json `"sides"`. Each later part is recorded named right (`--sides 1` or `0`). Parts recorded before the decision use ft-camd's names (`--sides auto`; a record-only ft-hands can't tell), and readers rename them (`sides.py`). Each part's `names_swapped` goes into take.json `"parts"`. Without a tracking ft-hands nothing decides: `"swapped": null`, the names stay as recorded, and the maintainer's check (`hub_review check`, check_sides on a few sets per take) tells. `takes.py sides SESSION --set swapped|named` records a decision by hand.
|
||||
- **ft-handpanel** runs as a child process with `--watch-stdin`. It shows the panel and logs poses during each take.
|
||||
- **The headset button's reader** is a thread of the session (`ButtonReader`, below), not a process.
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ case ${1:-install} in
|
||||
"$root/hands/rec/build.sh"
|
||||
echo "== 4/4 ft-camd's capabilities (asks for your password) and the menu entry"
|
||||
"$root/hands/run.sh" caps
|
||||
"$root/scripts/conf-migrate.sh" # HANDS_SWAP_SIDES=0, the old default, becomes auto
|
||||
fill_template "$root/hands/rec/ft-handrec.desktop" |
|
||||
on_frame "mkdir -p ~/.local/share/applications && cat > $entry"
|
||||
echo
|
||||
|
||||
@@ -1575,6 +1575,9 @@ class Session:
|
||||
self._save_session()
|
||||
self._log("side cameras: %s (%s, %s)" % ("SWAPPED" if swapped else "as named", new["decided_by"],
|
||||
json.dumps(new["evidence"])))
|
||||
if new["state"] == "forced, disagrees":
|
||||
self._log("side cameras: HANDS_SWAP_SIDES in ~/.config/frametop.conf forces names the hands say are "
|
||||
"backwards; the recording goes by the hands. Set HANDS_SWAP_SIDES=auto.")
|
||||
|
||||
def _sides_swapped(self):
|
||||
"""session.json's decision: True, False, or None (not known yet)."""
|
||||
|
||||
@@ -125,4 +125,9 @@ def read_live(path=None, ring_path=None, now_ns=None):
|
||||
return None
|
||||
except OSError:
|
||||
return None
|
||||
if (s.get("state") == "forced, disagrees" and s.get("swapped") is not None
|
||||
and bool(s["swapped"]) == bool(s.get("names_swapped"))):
|
||||
# An ft-hands built before 2026-10-06 kept a forced HANDS_SWAP_SIDES as the truth even
|
||||
# when the hands disagreed. The hands are right: the truth is the other way round.
|
||||
s = dict(s, swapped=not s["swapped"], decided_by="auto")
|
||||
return s
|
||||
@@ -91,6 +91,29 @@ class RulesTest(unittest.TestCase):
|
||||
finally:
|
||||
shutil.rmtree(d)
|
||||
|
||||
def test_read_live_forced(self):
|
||||
"""HANDS_SWAP_SIDES=0 forced and the hands disagree: the hands are the truth, whether
|
||||
ft-hands published them (built after 2026-10-06) or kept the forced value (before)."""
|
||||
d = tempfile.mkdtemp()
|
||||
try:
|
||||
path = os.path.join(d, "sides.json")
|
||||
|
||||
def live(**kw):
|
||||
s = dict(pid=1, ring_ino=0, mode="0", names_swapped=False,
|
||||
updated_ns=time.clock_gettime_ns(time.CLOCK_MONOTONIC), **kw)
|
||||
with open(path, "w") as f:
|
||||
json.dump(s, f)
|
||||
return sides.read_live(path)
|
||||
|
||||
self.assertEqual(live(state="forced", swapped=False, decided_by="config")["swapped"], False)
|
||||
self.assertEqual(live(state="forced, agrees", swapped=False, decided_by="config")["swapped"], False)
|
||||
old = live(state="forced, disagrees", swapped=False, decided_by="config")
|
||||
self.assertEqual((old["swapped"], old["decided_by"]), (True, "auto"))
|
||||
new = live(state="forced, disagrees", swapped=True, decided_by="auto")
|
||||
self.assertEqual((new["swapped"], new["decided_by"]), (True, "auto"))
|
||||
finally:
|
||||
shutil.rmtree(d)
|
||||
|
||||
|
||||
class TakesTest(unittest.TestCase):
|
||||
"""A swapped session: one take's parts recorded before the decision (ft-camd's names) and
|
||||
@@ -226,6 +249,22 @@ class SessionSidesTest(unittest.TestCase):
|
||||
self.assertTrue(s._session_json["sides"]["reversed_from"]["swapped"])
|
||||
self.assertFalse(takes.read_json(os.path.join(self.tmp, "session.json"))["sides"]["swapped"])
|
||||
|
||||
def test_read_sides_forced(self):
|
||||
"""PR #4 on the dataset: HANDS_SWAP_SIDES=0 from the old example config, and the hands
|
||||
disagree. The session takes the hands' answer, not the forced one."""
|
||||
s = session.Session(os.path.join(self.tmp, "base"), {}, {}, "room", self.script, dry_run=True,
|
||||
hands_dir=self.tmp)
|
||||
s.session_dir = self.tmp
|
||||
s._session_json = {"sides": {"swapped": None}}
|
||||
self.live(mode="0", state="forced", swapped=False, decided_by="config", names_swapped=False, evidence=None)
|
||||
s._read_sides(force=True)
|
||||
self.assertIs(s._sides_swapped(), False)
|
||||
self.live(mode="0", state="forced, disagrees", swapped=False, decided_by="config", names_swapped=False)
|
||||
s._read_sides(force=True)
|
||||
self.assertIs(s._sides_swapped(), True)
|
||||
self.assertEqual(s._session_json["sides"]["decided_by"], "auto")
|
||||
self.assertEqual(s._session_json["sides"]["reversed_from"]["decided_by"], "config")
|
||||
|
||||
def test_recorder_parts(self):
|
||||
calls = []
|
||||
|
||||
|
||||
@@ -27,7 +27,8 @@
|
||||
// default) tells from the hands it tracks (track/sides.h): once it's sure, it exchanges the two
|
||||
// cameras if they're backwards (the tracked views move with their images), and checks once
|
||||
// more. 0 and 1 force the naming (1: exchanged; --swap-sides is --sides 1); it still checks,
|
||||
// and warns if the hands disagree. The decision is published in /run/user/UID/frametop-hands/sides.json (see
|
||||
// and if the hands disagree it warns and publishes what the hands say as the truth ("swapped"),
|
||||
// so recordings are labelled right while tracking keeps the forced names. The decision is published in /run/user/UID/frametop-hands/sides.json (see
|
||||
// write_sides below) and, for recordings, in DIR/sides.json. --record-only can't tell (it tracks
|
||||
// nothing): under auto it records the ring's names as they are.
|
||||
//
|
||||
@@ -507,14 +508,16 @@ int main(int argc, char **argv) {
|
||||
const bool backwards = v == SideCheck::Swapped; // relative to the names as they are now
|
||||
const double after = (now - start) / 1e9;
|
||||
const std::string ev = side_check.json(), text = side_check.summary();
|
||||
if (sides_mode != "auto") { // forced: only say so
|
||||
if (sides_mode != "auto") { // forced: the names stay; if the hands disagree, they're the truth
|
||||
const std::string what = (sides_from == "option" ? "--sides " : "HANDS_SWAP_SIDES=") + sides_mode;
|
||||
if (backwards)
|
||||
std::printf("side cameras: %s looks WRONG: the hands say the side cameras are the other way round (%s). "
|
||||
"Use auto.\n", what.c_str(), text.c_str());
|
||||
"Tracking keeps the forced names; recordings are labelled by the hands. Use auto.\n",
|
||||
what.c_str(), text.c_str());
|
||||
else
|
||||
std::printf("side cameras: %s agrees with the hands (%s)\n", what.c_str(), text.c_str());
|
||||
sides_state = backwards ? "forced, disagrees" : "forced, agrees";
|
||||
if (backwards) truth = !names_swapped, decided_by = "auto", decided_after_s = after;
|
||||
decision_evidence = ev;
|
||||
checking = false;
|
||||
} else if (side_round == 0 || backwards) {
|
||||
|
||||
@@ -93,6 +93,7 @@ step "7/10 multi-screen desktop (ft-screens), Frametop Input Settings, and Frame
|
||||
"$root/remote/install.sh"
|
||||
on_frame "sed -i 's/^POINTER=0/POINTER=1/' ~/.config/frametop.conf; grep -q '^POINTER=' ~/.config/frametop.conf || echo 'POINTER=1' >> ~/.config/frametop.conf"
|
||||
echo "the launcher's Desktop entry now opens the multi-screen desktop; 3D mouse on (POINTER=1 in ~/.config/frametop.conf)"
|
||||
"$root/scripts/conf-migrate.sh"
|
||||
|
||||
step "8/10 gaze mode (optional, experimental: the pointer goes where you look)"
|
||||
gaze=0
|
||||
|
||||
Executable
+25
@@ -0,0 +1,25 @@
|
||||
#!/usr/bin/env bash
|
||||
# Update the lines of ~/.config/frametop.conf that still read exactly as an older
|
||||
# frametop.conf.example wrote them. A line you changed stays as it is. install.sh and
|
||||
# hands/rec/install.sh run this on every install and update.
|
||||
# HANDS_SWAP_SIDES=0: the example's value until 2026-10-05. It made ft-hands keep ft-camd's side
|
||||
# camera names even when they're backwards (some SteamVR restarts swap them), so hands landed
|
||||
# beside their cutouts and the hand recorder labelled the side cameras wrong. auto tells from
|
||||
# the hands.
|
||||
set -euo pipefail
|
||||
root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
|
||||
. "$root/scripts/_env.sh"
|
||||
|
||||
on_frame_script <<'EOF'
|
||||
f=~/.config/frametop.conf
|
||||
[ -f "$f" ] || exit 0
|
||||
old="HANDS_SWAP_SIDES=0 # hand tracking (hands/run.sh install): 1 = the side cameras' names are swapped, which some SteamVR restarts cause (hands/tools/check_sides.py --ring tells)"
|
||||
new="HANDS_SWAP_SIDES=auto # hand tracking: which side camera is which. auto = ft-hands tells from the hands and fixes ft-camd's names, which some SteamVR restarts swap | 0 = keep ft-camd's | 1 = exchange them"
|
||||
if grep -qxF "$old" "$f"; then
|
||||
tmp=$(mktemp)
|
||||
awk -v old="$old" -v new="$new" '$0 == old { print new; next } { print }' "$f" > "$tmp"
|
||||
cat "$tmp" > "$f" # in place: the file keeps its owner and mode
|
||||
rm -f "$tmp"
|
||||
echo "settings: HANDS_SWAP_SIDES=0 (the old default) is now auto: hand tracking tells the side cameras apart itself"
|
||||
fi
|
||||
EOF
|
||||
@@ -40,7 +40,7 @@ POINTER_HEAD_DEADZONE=0.5 # keyboard clicks at the gaze (Meta+J, Meta+K): degre
|
||||
POINTER_KEY_TAP=0.25 # keyboard clicks: let go within this (s) and it clicks where the dot was at the press, and tells the gaze tracker it was right
|
||||
GAZE_TRACKER=auto # gaze service: auto = our own eye tracker when it's installed (gaze/tracker/install.sh, which install.sh offers), else SteamVR's | own = ours | steam = SteamVR's. Each keeps its own calibration (Calibrate on Frametop Input Settings' Gaze page)
|
||||
GAZE_EYE=auto # gaze service: eye bias. auto = each eye weighted by how far off it was at your recent nudges | left | right = that eye counts twice
|
||||
HANDS_SWAP_SIDES=0 # hand tracking (hands/run.sh install): 1 = the side cameras' names are swapped, which some SteamVR restarts cause (hands/tools/check_sides.py --ring tells)
|
||||
HANDS_SWAP_SIDES=auto # hand tracking: which side camera is which. auto = ft-hands tells from the hands and fixes ft-camd's names, which some SteamVR restarts swap | 0 = keep ft-camd's | 1 = exchange them
|
||||
HANDS_CPUS=5,6,7 # hand tracking: the CPUs its model threads run on
|
||||
HANDS_CAMERAS=auto # hand tracking: auto (by the light) | mono (the IR cameras; also keeps ft-camd off the colour ones) | color | all
|
||||
HANDS_BRIGHT=all # hand tracking, auto: the cameras in bright light, all | color
|
||||
|
||||
Reference in new issue
Block a user