diff --git a/hands/rec/DESIGN.md b/hands/rec/DESIGN.md index 276ea0c..131fb3b 100644 --- a/hands/rec/DESIGN.md +++ b/hands/rec/DESIGN.md @@ -28,12 +28,14 @@ Every part runs in the dev container, as ft-hands and Input Settings do. The hos - **A recording ft-hands** runs once per recording part: `ft-hands --record-only --record DIR --record-for SECONDS --record-hz 10 --status 0`. It runs as a plain child process of the session, ended with SIGTERM when the part ends. SIGTERM ends ft-hands' loop, and `Recorder` writes out its queue when it's destroyed. In step mode (below) a part is one step's countdown and hold, so a take has one part per step (about 40 in the hand poses); in auto mode a take is one part, plus one more after each pause. `--record-for` is only a safety net. - Why a process per part rather than one kept alive and paused: measured in the dev container with `ft-ringplay`'s ring (2026-10-02), `ft-hands --record-only` writes its first set 16-27 ms after it starts and ends 4-6 ms after SIGTERM, so a new part costs nothing the 3 s countdown doesn't cover. Every reader already takes parts in order (`takes.py`, `validate.py` through the export's single stream, the labeller's `fhl_io.py`, numbering `sets-10.bin` after `sets-9.bin`), ft-hands needs no new control, and nothing is written while a step waits. Before the hold starts the session checks that the part has written a set (`Recorder.has_data`, up to 3 s more), so the hold is recorded from its first frame. - **ft-handpanel** runs as a child process with `--watch-stdin`. It shows the panel and logs poses during each take. +- **The headset button's reader** is a thread of the session (`ButtonReader`, below), not a process. Test hooks: - `--ring PATH` goes to both ft-hands (`ft-ringplay` publishes a recording there, so the whole flow runs without the headset). - `--no-start` uses only what's already running. - `--dry-run` runs no processes and only prints the panel commands, with timing sped up by `--speed X`. - `--next-after S` presses Next by itself after S seconds of waiting (real time), so a step-mode session runs unattended. Lines on stdin steer it too: `n` or an empty line is Next, `p` pause or resume, `r` redo, `s` skip the section, `q` stop. +- `--no-headset-button` leaves the headset's button alone (`ft-handrec --no-headset-button` too). `--button-device PATH` reads it from PATH instead, an event device or a FIFO of `input_event` structs, also in a dry run: a simulated button for tests. - `--auto` runs the timed flow; `--poses DIR` takes the pose pictures from DIR; `--plan` prints the sections, their steps and length. ## Files @@ -212,7 +214,7 @@ A pose with no entry, or whose file is missing, shows no picture: the text alone The first in-headset session (2026-10-02) went too fast: each prompt advanced after 4-8 s, before there was time to read it and find the hand shape. So by default every step waits: -1. **Ready.** The panel shows the step: section title, "step N of M", the instruction, the pose picture, the where-to diagram, and "Ready? Press Space or click Next". The hand chips show which hands are seen, with no warnings yet. It waits as long as it takes, and nothing records. +1. **Ready.** The panel shows the step: section title, "step N of M", the instruction, the pose picture, the where-to diagram, and the Next hint ("Ready? Press the button on the right side of the headset", or with a mouse also Space and Next: see Controls). The hand chips show which hands are seen, with no warnings yet. It waits as long as it takes, and nothing records. 2. **Countdown.** Next starts a new recording part and a "ready" event, and the panel counts 3, 2, 1 (big), recorded so the hold is captured from its first frame. In the push sections the bar sits at near meanwhile. 3. **Hold.** The `prompt` event, the section's word ("Hold" or "Go") and the time-left bar for the prompt's seconds (or the bar's sweeps, or the targets). Then a `wait` event and the part stops. @@ -230,7 +232,7 @@ The sections, in this order (see the plan): 5. desk work (typing, mouse) 6. objects (one prompt per ticked object, own objects included) 7. touch the dot -8. controller depth, straps (push out and back at 3 heights and to each side, wrist turns, open and close) +8. controller depth, straps (push out and back at 3 heights and to each side, wrist turns, open and close). "Out and back" is straight away from the headset and back toward it: the first in-headset session took the left-right bar for sideways. The texts say so, the bar's ends read "At your chest" and "Arm out" (`near_label`, `far_label`, sent as `bar ... At your chest|Arm out`), and the picture is a side view. 9. bridge (one controller on, the bare fingertip touches the marked point on it at near, mid and far, then swap; then a controller on the desk, touched from several angles) 10. bare repeat of section 8, controllers off 11. no hands (10 s) @@ -245,7 +247,12 @@ Before section 8: "Put on both controllers and tighten the straps". Before secti ### Controls -- The window has Start, a big Next (while a step waits), Pause/Resume, Redo step, Skip section and Stop. While it has focus: Space is Next, P pauses or resumes, R redoes, S skips the section, Esc stops. The panel's bottom line and the window list them. The window also shows the step's picture, diagram, countdown and "Hold". +- The window has Start, a big Next (while a step waits; its hint is the panel's), Pause/Resume, Redo step, Skip section and Stop. While it has focus: Space is Next, P pauses or resumes, R redoes, S skips the section, Esc stops. The panel's bottom line and the window list them. The window also shows the step's picture, diagram, countdown and "Hold". +- **The headset's button.** The Frame has a click button on its right side, for use without controllers: `KEY_SELECT` (353) on the evdev device `gpio-keys`. While a step waits (the welcome, a section's intro, a step's ready screen) a press is Next; during a countdown or hold (and auto mode's timed screens) it pauses; while paused it resumes. The session finds the device in `/proc/bus/input/devices` by name and its KEY bitmap (`event3` on the maintainer's Frame) and reads `input_event` structs with plain `struct` (no python-evdev), from a thread. Only key-downs count (value 1: releases and autorepeat, value 2, don't), and presses closer than 0.3 s count once. + - It's read, never grabbed (`EVIOCGRAB`). Frametop's input relay (`input/input-relay.py`), ft-powerd, SteamVR and gamescope read the same device, and the relay remaps its volume keys (the experimental branch's `docs/hazards.md`). A grab would take the volume keys from the relay. + - `steamos` is in the `input` group, so it needs no sudo. The dev container sees the host's `/dev/input` and `/proc/bus/input/devices`, and the group carries over (checked 2026-10-02), so it works from the window there and from `session.py` on the host. If the device is missing or can't be opened, the session logs it, keeps trying every 5 s, and the hints don't mention the button. + - **Not known yet (needs the headset):** what SteamVR and gamescope do with the same press. They read it too, so it may also click whatever is under the head or gaze pointer in VR, or open something. Check on the first session; if it does, the fix is on their side or a different button, not a grab. +- **The Next hint follows what's there.** No mouse connected: "Ready? Press the button on the right side of the headset" (the window's Next can't be clicked, and Space needs the window focused). A mouse: "Ready? Press Space, click Next, or press the headset button". No button: "Ready? Press Space or click Next". The panel's bottom line leads with "Headset button: next, pause". A mouse is a device in `/proc/bus/input/devices` with EV_REL, REL_X and REL_Y that isn't made in software: uinput devices (Frametop's virtual mouse, frame-voice's keyboard) sit under `/devices/virtual/input` or on the virtual bus (6), and are left out; Bluetooth mice come through uhid, under `/devices/virtual/misc/uhid`, and count. It's looked at again at each step, so a mouse plugged in mid-session counts from the next step. - **R (redo).** During a step's countdown or hold: that step starts again from its ready screen. At a step's ready screen: the step before it (in this section) goes again. Either way a `redo` event marks the range of the try being redone, so its labels are skipped; the sets stay, to delete in review if wanted. - A pause stops the take's recording and starts it again on resume as the next part of the same take (`sets-2.bin`, and so on). takes.py reads all parts in order. A pause while a step waits only shows "Paused"; a Next pressed while paused doesn't count. diff --git a/hands/rec/ft_handrec.py b/hands/rec/ft_handrec.py index c707c8b..b94edb5 100755 --- a/hands/rec/ft_handrec.py +++ b/hands/rec/ft_handrec.py @@ -8,8 +8,8 @@ A Kirigami (QML) app with a Python backend. It runs in the dev container: free space) and what will happen. Start hands it to the session runner (session.py). - Session: the runner's live status with the step's pose picture and where-to diagram, Next, Pause/Resume, Redo, Skip section and Stop (Space: Next, P, R, S and Esc while the window has - focus). The prompts appear in the headset. Each step waits for Next unless "Advance by - itself" was ticked. + focus; the button on the headset's right side is Next, pause and resume). The prompts appear + in the headset. Each step waits for Next unless "Advance by itself" was ticked. - Review: sessions, their takes, and a viewer for one frame set at a time, where ranges, takes and sessions can be deleted (takes.py). - Export: compress what's kept into exports// at nice 19 (takes.py), with a warning @@ -212,7 +212,7 @@ class Backend(QObject): def __init__(self, store, session_options=None, hub_dry_run=False): super().__init__() self.store = store - self._session_options = session_options or {} # test hooks for Session: dry_run, speed, poses_dir + self._session_options = session_options or {} # Session options: dry_run, speed, poses_dir (tests), button self._hub_dry_run = hub_dry_run self._login = {"state": "dry" if hub_dry_run else "unknown"} self._login_busy = False @@ -923,6 +923,8 @@ def main(): help="test: sessions start no processes and print the panel's commands") ap.add_argument("--speed", type=float, default=1.0, help="test, with --dry-run: run sessions this much faster") ap.add_argument("--poses", help="test: the pose pictures' folder (default hands/rec/poses)") + ap.add_argument("--no-headset-button", action="store_true", + help="sessions don't read the headset's button (Next, pause, resume)") ap.add_argument("--hub-dry-run", action="store_true", help="test: Upload checks the export and says what it would send, with no network calls") a, qt_args = ap.parse_known_args() @@ -939,6 +941,8 @@ def main(): options = {"dry_run": True, "speed": a.speed} if a.dry_run else {} if a.poses: options["poses_dir"] = a.poses + if a.no_headset_button: + options["button"] = False backend = Backend(store, options, hub_dry_run=a.hub_dry_run) app.aboutToQuit.connect(backend.shutdown) engine.rootContext().setContextProperty("backend", backend) diff --git a/hands/rec/main.qml b/hands/rec/main.qml index d0be800..ae1eb88 100644 --- a/hands/rec/main.qml +++ b/hands/rec/main.qml @@ -387,9 +387,11 @@ Kirigami.ApplicationWindow { + "of the hand pose, where to hold your hands, and what to do. The sections: hand poses, " + "gestures, typing and the mouse, your objects, touching a dot, and moves with the " + "controllers on and off. Each section is recorded as one take.\n\n" - + "Each step waits until you're ready: press Space or click Next in this window. A 3-2-1 " + + "Each step waits until you're ready: press the button on the right side of the headset, " + + "or Space or Next in this window. A 3-2-1 " + "countdown follows, then hold the pose until the bar runs out. Nothing is recorded while " - + "a step waits. P pauses, R records the last step again, S skips a section and Esc stops.\n\n" + + "a step waits. The headset button also pauses and resumes a recording. In this window P " + + "pauses, R records the last step again, S skips a section and Esc stops.\n\n" + "Nothing leaves the headset. Afterwards you watch the takes in Review, delete anything " + "you don't want to share, and only then export." } @@ -710,7 +712,8 @@ Kirigami.ApplicationWindow { Controls.Label { visible: sessionView.waiting && sessionView.state !== "paused" opacity: 0.7 - text: "Ready? Press Space or click Next. A 3-2-1 countdown starts the recording." + text: (sessionView.st.ready_text || "Ready? Press Space or click Next") + + ". A 3-2-1 countdown starts the recording." wrapMode: Text.Wrap Layout.fillWidth: true } @@ -743,7 +746,10 @@ Kirigami.ApplicationWindow { Layout.fillWidth: true wrapMode: Text.Wrap opacity: 0.7 - text: "The instructions appear in the headset. While this window has focus: " + text: "The instructions appear in the headset. " + + (sessionView.st.button ? "The button on the right side of the headset: " + + (sessionView.stepMode ? "next, " : "") + "pause or resume. " : "") + + "While this window has focus: " + (sessionView.stepMode ? "Space: next · " : "") + "P: pause or resume · R: record the last step again · S: skip section · Esc: stop. " + "Stopping keeps what's recorded so far." diff --git a/hands/rec/poses/README.md b/hands/rec/poses/README.md index 20f187f..5050773 100644 --- a/hands/rec/poses/README.md +++ b/hands/rec/poses/README.md @@ -13,16 +13,18 @@ Every image shows a right hand as the wearer sees it from their own eyes. "Palm - **Left-hand prompts:** the panel mirrors the image horizontally. - **"Both" prompts:** the panel shows two copies, one of them mirrored. -- **`two_hands: true`:** the image already shows the whole scene: both hands, or one hand with an object, as in `hold`. Show it once, without mirroring or copying. This applies to `cross`, `overlap`, `near-face`, `typing`, `lift`, `switch`, `hold`, `touch-stick` and `touch-stick-desk`. +- **`two_hands: true`:** the image already shows the whole scene: both hands, or one hand with an object, as in `hold`. Show it once, without mirroring or copying. This applies to `cross`, `overlap`, `near-face`, `typing`, `lift`, `switch`, `hold`, `push`, `push-controller`, `touch-stick` and `touch-stick-desk`. - **`touch-stick`:** shows the left hand holding the controller while the right index touches its thumbstick. For prompts where the right hand holds the controller (`controllers: ["right"]`), mirror it. -Motion poses show the start pose, a faint blue "ghost" of the end pose, and orange arrows. Objects (keyboard, mouse, bottle, bar, controller, screen, head and headset) are plain grey shapes. +`push` and `push-controller` are side views: the wearer's head with a headset on, one arm out in front with the palm out, a ghost of the hand further out, and a straight double arrow from the headset outward, labelled "near" and "arm out". The labels use Pillow's built-in font. + +Other motion poses show the start pose, a faint blue "ghost" of the end pose, and orange arrows. Objects (keyboard, mouse, bottle, bar, controller, screen, head and headset) are plain grey shapes. Besides the pose ids in `script.json`, these extra ids exist for prompts that need a different picture: | id | for | | --- | --- | -| `push` | the bar sections: palms out, push out and back | +| `push` | the bar sections: push straight out from the headset and back, palms out | | `push-controller` | the same with controllers on | | `touch-stick-desk` | touching the thumbstick of a controller lying on the desk | | `no-hands` | the no-hands section: hands down, out of view | diff --git a/hands/rec/poses/contact-sheet.png b/hands/rec/poses/contact-sheet.png index 67cca45..1085d06 100644 Binary files a/hands/rec/poses/contact-sheet.png and b/hands/rec/poses/contact-sheet.png differ diff --git a/hands/rec/poses/make_poses.py b/hands/rec/poses/make_poses.py index c0663d8..8c2b485 100644 --- a/hands/rec/poses/make_poses.py +++ b/hands/rec/poses/make_poses.py @@ -717,6 +717,18 @@ def project(cam, W, ss, p): return ((q[0] - cam["center"][0]) / s + W * ss / 2, (cam["center"][1] - q[1]) / s + W * ss / 2) +def draw_labels(img, labels, cam, W, ss): + """Short text labels at world points, light with a dark edge (Pillow's built-in font).""" + dr = ImageDraw.Draw(img) + sc = ss * W / 512 + font = ImageFont.load_default(size=int(round(42 * sc))) + for lb in labels: + x, y = project(cam, W, ss, lb["at"]) + dr.text((x, y), lb["text"], font=font, anchor=lb.get("anchor", "mm"), fill=(232, 235, 240, 255), + stroke_width=int(round(4 * sc)), stroke_fill=ACCENT_DARK + (255,)) + return img + + def draw_arrows(arrows, cam, W, ss): img = Image.new("RGBA", (W * ss, W * ss), (0, 0, 0, 0)) dr = ImageDraw.Draw(img) @@ -801,6 +813,9 @@ def fit(spec, W): for p in a["pts"]: q = cam["R"] @ V(*p) pts.append((q[0], q[1])) + for lb in spec.get("labels", []): # rough text extent: about 2 cm per side + q = cam["R"] @ V(*lb["at"]) + pts += [(q[0] - lb.get("half", 3.0), q[1] - 1.5), (q[0] + lb.get("half", 3.0), q[1] + 1.5)] pts = np.array(pts) x0, y0 = pts.min(0) x1, y1 = pts.max(0) @@ -828,8 +843,10 @@ def render_pose(spec, W=512, ss=2): else: lrgb, la = ink(buf, ss * W / 512) rgb, a = over(rgb, a, lrgb, la) - if spec.get("arrows"): - arr = np.asarray(draw_arrows(spec["arrows"], cam, W, ss), float) / 255 + if spec.get("arrows") or spec.get("labels"): + im = draw_arrows(spec.get("arrows", []), cam, W, ss) + im = draw_labels(im, spec.get("labels", []), cam, W, ss) + arr = np.asarray(im, float) / 255 aa = arr[..., 3] rgb, a = over(rgb, a, arr[..., :3] * aa[..., None], aa) # downsample (premultiplied box filter) @@ -952,6 +969,27 @@ def with_ctrl(hand): return ctrl_parts(hand.M, hand.t) +def ctrl_open_parts(hand): + """The controller on an open hand, held on by its strap (hand local, then moved with the hand): + the body lies against the palm from the heel up to the base of the thumb, the head sits + by the thumb, the strap crosses the back of the hand, and the fingers stay free.""" + a, b = V(-1.7, 1.4, 3.5), V(1.9, 7.4, 3.4) + ax = unit(b - a) + grip = Cone(a, b, 1.45, 1.6) + head_c = b + ax * 1.7 + V(0.8, 0.0, 0.5) + head = Ell(head_c, orient(ax, (0.3, 0, 1)), (2.3, 1.9, 1.9)) + up = unit((0.3, 0.3, 1.0)) + sb = head_c + up * 1.75 + stick = U([Cone(sb - up * 0.4, sb + up * 0.6, 0.36, 0.36), + Cyl(sb + up * 0.75, orient(up, (1, 0, 0)), 0.78, 0.2, 0.12)]) + band = Box((0.0, 4.4, -2.2), orient(ax, (0, 0, 1)), (0.95, 3.8, 0.1), 0.18) + ends = [Cone((-1.7, 0.9, -2.15), (-3.9, 1.3, 0.2), 0.3, 0.3), Cone((-3.9, 1.3, 0.2), (-2.0, 1.4, 2.6), 0.3, 0.3), + Cone((1.7, 7.9, -2.15), (3.9, 7.6, 0.2), 0.3, 0.3), Cone((3.9, 7.6, 0.2), (2.2, 7.4, 2.6), 0.3, 0.3)] + M, t = hand.M, hand.t + return [Obj(U([grip, head], 0.9), (0.5, 0.54, 0.6), 1).xf(M, t), Obj(stick, (0.3, 0.32, 0.37), 2).xf(M, t), + Obj(U([band] + ends, 0.25), (0.36, 0.38, 0.43), 3).xf(M, t)] + + def keyboard(c=(0, 0, 0), nx=14, nz=5): pitch = 1.9 w, dpt = nx * pitch / 2 + 0.6, nz * pitch / 2 + 0.6 @@ -1085,14 +1123,36 @@ def spec_touch_stick_desk(f=(-1, -0.8, 0), p=(0, -1, -0.3)): return dict(cam=cam(25), layers=[L(*objs, rh)]) +def head_and_headset(R=np.eye(3)): + """The wearer's head with a headset on, facing -z (the head's local frame), as two objects.""" + head = Ell((0, 0, 0), np.eye(3), (7.6, 10.2, 8.8)) + nose = Cone((0, -1.5, -8.2), (0, -3.6, -9.6), 0.7, 0.9) + visor = Box((0, 2.4, -6.5), np.eye(3), (8.2, 2.6, 2.6), 1.4) + band = Torus((0, 3.0, 0.5), Rx(8), 8.0, 0.9) + z = V(0, 0, 0) + return [Obj(U([head, nose], 0.6), (0.5, 0.53, 0.58), 8).xf(R, z), + Obj(U([visor, band], 0.4), (0.36, 0.38, 0.43), 9).xf(R, z)] + + +def push_hand(pose, z, controllers=False): + """A right hand pushed out in front of the face (forward is -z), palm out, arm reaching back.""" + fwd = unit((0, 0.42, -1)) # the forearm points forward and a little up + R = orient(fwd, unit(np.cross(fwd, (1, 0, 0)))) # palm down along the arm + h = Hand(dict(pose, wrist=(-66, 0)), R, (3.0, -17.0, z), arm=15) # wrist bent back: fingers up + return [h] + (ctrl_open_parts(h) if controllers else []) + + def push_spec(controllers=False): - R = BACK - main = H(RELAX if not controllers else CTRL_GRIP, R, (0, 0, 0)) - ghost = H(RELAX if not controllers else CTRL_GRIP, R, (0, 0, -16)) - lm = [main] + (with_ctrl(main) if controllers else []) - lg = [ghost] + (with_ctrl(ghost) if controllers else []) - return dict(cam=cam(18, 50), layers=[L(*lg, ghost=0.45), L(*lm)], - arrows=[dict(pts=seg((-4.5, 21, -1), (-4.5, 21, -15)), heads="both")]) + pose = FLAT # open hand, palm out; with controllers the strap holds the controller on + near, far = -21.0, -44.0 + eye = V(0, 2.4, -10.5) + tip_y = 0.0 + return dict(cam=cam(4, 65), scale=0.11, + layers=[L(*push_hand(pose, far, controllers), ghost=0.45), + L(*head_and_headset(), *push_hand(pose, near, controllers))], + arrows=[dict(pts=seg(eye + V(0, tip_y, -1.0), V(0, eye[1] + tip_y, far - 7.0)), heads="both")], + labels=[dict(at=(0, eye[1] + tip_y + 4.2, near - 1.0), text="near", half=4.0), + dict(at=(0, eye[1] + tip_y + 4.2, far - 2.0), text="arm out", half=7.0)]) def ghost_pair(main_pose, ghost_pose, R, cam_, arrows, ghost_alpha=0.45, **kw): @@ -1237,8 +1297,9 @@ POSES = { "mouse": ("Hand on the mouse, move and click", False, spec_mouse), "switch": ("Switch between keyboard and mouse", True, spec_switch), "hold": ("Pick it up, use it, put it down", True, spec_hold), - "push": ("Palms out, push out and back", False, lambda: push_spec(False)), - "push-controller": ("Controller on, push out and back", False, lambda: push_spec(True)), + "push": ("Push straight out, away from the headset, and back", True, lambda: push_spec(False)), + "push-controller": ("Controllers on: push straight out from the headset, and back", True, + lambda: push_spec(True)), "touch-stick": ("Touch the thumbstick with your other index", True, spec_touch_stick), "touch-stick-desk": ("Touch the thumbstick of the controller on the desk", True, spec_touch_stick_desk), "no-hands": ("Hands down, out of view", False, spec_no_hands), diff --git a/hands/rec/poses/poses.json b/hands/rec/poses/poses.json index 7ecbcb6..4d25c5d 100644 --- a/hands/rec/poses/poses.json +++ b/hands/rec/poses/poses.json @@ -156,13 +156,13 @@ }, "push": { "file": "push.png", - "two_hands": false, - "caption": "Palms out, push out and back" + "two_hands": true, + "caption": "Push straight out, away from the headset, and back" }, "push-controller": { "file": "push-controller.png", - "two_hands": false, - "caption": "Controller on, push out and back" + "two_hands": true, + "caption": "Controllers on: push straight out from the headset, and back" }, "touch-stick": { "file": "touch-stick.png", diff --git a/hands/rec/poses/push-controller.png b/hands/rec/poses/push-controller.png index 9ed1834..d1dcd70 100644 Binary files a/hands/rec/poses/push-controller.png and b/hands/rec/poses/push-controller.png differ diff --git a/hands/rec/poses/push.png b/hands/rec/poses/push.png index d65d8b5..a21dbf6 100644 Binary files a/hands/rec/poses/push.png and b/hands/rec/poses/push.png differ diff --git a/hands/rec/script.json b/hands/rec/script.json index bd1561b..f4c28a9 100644 --- a/hands/rec/script.json +++ b/hands/rec/script.json @@ -3,7 +3,7 @@ "intro_s": 4, "between_s": 3, "welcome": {"title": "Hand recorder", "seconds": 7, - "text": "Follow the steps on this panel. Each one shows what to do with your hands, and where. A bar shows the time left."}, + "text": "Follow the steps on this panel. Each one shows what to do with your hands, and where. A bar shows the time left.|The button on the right side of the headset goes on when a step waits for you, and pauses while recording."}, "done": {"title": "All done", "seconds": 6, "text": "Thank you! You can take off the headset.|Review your recordings in the Hand recorder window."}, "stopped": {"title": "Stopped", "seconds": 4, @@ -118,15 +118,15 @@ {"id": "controller-push", "title": "Depth with controllers", "kind": "bar", "requires": ["controllers"], "before": {"text": "Put on both controllers and tighten the straps. Take your time.", "seconds": 25}, - "intro": "Follow the marker on the bar with both hands: out as it moves right, back as it moves left.", + "intro": "Push both hands straight out in front of you, away from the headset, then pull them back toward it. Only nearer and farther, not sideways. The marker on the bar shows how far: right is arm out, left is at your chest.", "defaults": {"hands": "both", "pose": "open", "picture": "push-controller", "controller": true, "controllers": ["left", "right"]}, - "near_m": 0.2, "far_m": 0.6, "period_s": 6, "lead_s": 3, "reps": 5, + "near_m": 0.2, "far_m": 0.6, "near_label": "At your chest", "far_label": "Arm out", "period_s": 6, "lead_s": 3, "reps": 5, "heights": [ - {"id": "chest", "text": "Chest height: hands open, palms out like a stop sign. Push out and back with the marker."}, - {"id": "desk", "text": "Now at desk height, palms out. Push out and back with the marker."}, - {"id": "eye", "text": "Now at eye level, palms out. Push out and back with the marker."}, - {"id": "left", "text": "Now off to your left side. Push out and back with the marker.", "reps": 2}, - {"id": "right", "text": "Now off to your right side. Push out and back with the marker.", "reps": 2} + {"id": "chest", "text": "Chest height, palms out like a stop sign. Push straight out, away from the headset, then pull back to your chest. Follow the marker."}, + {"id": "desk", "text": "Now at desk height, palms out. Push straight out, away from you, then pull back toward you. Follow the marker."}, + {"id": "eye", "text": "Now at eye level, palms out. Push straight out, away from the headset, then pull back toward it. Follow the marker."}, + {"id": "left", "text": "Now hold both hands off to your left. From there, push straight out and pull back toward you, not sideways. Follow the marker.", "reps": 2}, + {"id": "right", "text": "Now hold both hands off to your right. From there, push straight out and pull back toward you, not sideways. Follow the marker.", "reps": 2} ], "prompts": [ {"text": "Slowly turn your wrists: palms out, palms toward you, and back.", "seconds": 10, "pose": "turn-in-out", "distance": "mid"}, @@ -154,15 +154,15 @@ {"id": "bare-push", "title": "Depth, bare hands", "kind": "bar", "requires": [], "before": {"when": "controllers", "text": "Take the controllers off and put them out of view.", "seconds": 15}, - "intro": "Follow the marker on the bar with both hands: out as it moves right, back as it moves left.", + "intro": "Push both hands straight out in front of you, away from the headset, then pull them back toward it. Only nearer and farther, not sideways. The marker on the bar shows how far: right is arm out, left is at your chest.", "defaults": {"hands": "both", "pose": "open", "picture": "push"}, - "near_m": 0.2, "far_m": 0.6, "period_s": 6, "lead_s": 3, "reps": 4, + "near_m": 0.2, "far_m": 0.6, "near_label": "At your chest", "far_label": "Arm out", "period_s": 6, "lead_s": 3, "reps": 4, "heights": [ - {"id": "chest", "text": "Chest height: hands open, palms out like a stop sign. Push out and back with the marker."}, - {"id": "desk", "text": "Now at desk height, palms out. Push out and back with the marker."}, - {"id": "eye", "text": "Now at eye level, palms out. Push out and back with the marker."}, - {"id": "left", "text": "Now off to your left side. Push out and back with the marker.", "reps": 2}, - {"id": "right", "text": "Now off to your right side. Push out and back with the marker.", "reps": 2} + {"id": "chest", "text": "Chest height, palms out like a stop sign. Push straight out, away from the headset, then pull back to your chest. Follow the marker."}, + {"id": "desk", "text": "Now at desk height, palms out. Push straight out, away from you, then pull back toward you. Follow the marker."}, + {"id": "eye", "text": "Now at eye level, palms out. Push straight out, away from the headset, then pull back toward it. Follow the marker."}, + {"id": "left", "text": "Now hold both hands off to your left. From there, push straight out and pull back toward you, not sideways. Follow the marker.", "reps": 2}, + {"id": "right", "text": "Now hold both hands off to your right. From there, push straight out and pull back toward you, not sideways. Follow the marker.", "reps": 2} ], "prompts": [ {"text": "Slowly turn your wrists: palms out, palms toward you, and back.", "seconds": 10, "pose": "turn-in-out", "distance": "mid"}, diff --git a/hands/rec/session.py b/hands/rec/session.py index d1e296c..dda28f1 100755 --- a/hands/rec/session.py +++ b/hands/rec/session.py @@ -27,6 +27,7 @@ import math import mmap import os import re +import select import shutil import signal import socket @@ -61,8 +62,14 @@ COUNTDOWN_S = 3 # step mode: the 3-2-1 before each step, recorded FIRST_SET_S = 3.0 # step mode: how long the hold may wait for its recording's first set RESUME_HINT = "Paused. Resume: P in the Hand recorder window" READY_TEXT = "Ready? Press Space or click Next" +# With the headset's button: it leads when no mouse is connected (the window's Next can't be clicked). +READY_BUTTON = "Ready? Press the button on the right side of the headset" +READY_BUTTON_MOUSE = "Ready? Press Space, click Next, or press the headset button" KEYS_STEP = "Hand recorder window: Space next \u00b7 P pause \u00b7 R redo \u00b7 S skip section \u00b7 Esc stop" KEYS_AUTO = "Hand recorder window: P pause \u00b7 R redo \u00b7 S skip section \u00b7 Esc stop" +KEYS_STEP_BUTTON = ("Headset button: next, pause \u00b7 Window: Space next \u00b7 P pause \u00b7 R redo " + "\u00b7 S skip \u00b7 Esc stop") +KEYS_AUTO_BUTTON = "Headset button: pause \u00b7 Window: P pause \u00b7 R redo \u00b7 S skip section \u00b7 Esc stop" def mono_ns(): @@ -507,6 +514,174 @@ class Recorder: return self.proc.returncode +# ------------------------------------------------------------------------------------------ +# The headset's button, and mice (/proc/bus/input/devices, linux/input.h) +# +# The Frame's click button on its right side is KEY_SELECT on the "gpio-keys" device. It's read +# as any other reader does, never grabbed (EVIOCGRAB): Frametop's input relay, ft-powerd, SteamVR +# and gamescope read the same device, and the relay remaps its volume keys. + +INPUT_DEVICES = "/proc/bus/input/devices" +INPUT_EVENT = struct.Struct("@llHHi") # struct input_event: a timeval, type, code, value (24 bytes on 64-bit) +EV_KEY, EV_REL = 0x01, 0x02 +REL_X, REL_Y = 0x00, 0x01 +KEY_SELECT = 353 +BUTTON_DEVICE = "gpio-keys" +BUTTON_DEBOUNCE_S = 0.3 # presses closer than this count once + + +def parse_input_devices(text): + """/proc/bus/input/devices as [{"name", "phys", "sysfs", "bus", "handlers": [...], "bits": {"EV": int, ...}}].""" + out, dev = [], None + for line in text.splitlines() + [""]: + line = line.strip() + if not line: + if dev: + out.append(dev) + dev = None + continue + if dev is None: + dev = {"name": "", "phys": "", "sysfs": "", "bus": 0, "handlers": [], "bits": {}} + tag, _, rest = line.partition(": ") + if tag == "I": + m = re.search(r"Bus=([0-9a-fA-F]+)", rest) + dev["bus"] = int(m.group(1), 16) if m else 0 + elif tag == "N": + dev["name"] = rest.partition("=")[2].strip('"') + elif tag == "P": + dev["phys"] = rest.partition("=")[2] + elif tag == "S": + dev["sysfs"] = rest.partition("=")[2] + elif tag == "H": + dev["handlers"] = rest.partition("=")[2].split() + elif tag == "B": + key, _, words = rest.partition("=") + value = 0 + for w in words.split(): # the highest long first + try: + value = (value << 64) | int(w, 16) + except ValueError: + value = 0 + break + dev["bits"][key] = value + return out + + +def _event_node(dev): + for h in dev["handlers"]: + if re.fullmatch(r"event\d+", h): + return "/dev/input/" + h + return None + + +def find_button(devices): + """The headset button's event device: "gpio-keys" with KEY_SELECT, or None.""" + for dev in devices: + if dev["name"] == BUTTON_DEVICE and dev["bits"].get("KEY", 0) >> KEY_SELECT & 1: + return _event_node(dev) + return None + + +def real_mouse(dev): + """A pointing device a person holds: relative X and Y, not made in software. uinput devices + (Frametop's virtual mouse, frame-voice's keyboard) sit under /devices/virtual/input or on the + virtual bus (6); Bluetooth mice come through uhid, also under /devices/virtual, so those count.""" + bits = dev["bits"] + if not (bits.get("EV", 0) >> EV_REL & 1) or (bits.get("REL", 0) & 3) != 3: # REL_X and REL_Y + return False + sysfs = dev["sysfs"] + if dev["bus"] == 0x06 or "virtual" in dev["name"].lower() or "uinput" in dev["phys"]: + return False + return not sysfs.startswith("/devices/virtual/") or sysfs.startswith("/devices/virtual/misc/uhid/") + + +def read_input_devices(path=INPUT_DEVICES): + try: + with open(path) as f: + return parse_input_devices(f.read()) + except OSError: + return [] + + +def mouse_connected(path=INPUT_DEVICES): + return any(real_mouse(d) for d in read_input_devices(path)) + + +def button_presses(data): + """KEY_SELECT key-downs in a run of input_event structs (value 1; releases and autorepeat + are left out). Returns (how many, the bytes left over after the last whole event).""" + n, usable = 0, len(data) - len(data) % INPUT_EVENT.size + for off in range(0, usable, INPUT_EVENT.size): + _, _, etype, code, value = INPUT_EVENT.unpack_from(data, off) + if etype == EV_KEY and code == KEY_SELECT and value == 1: + n += 1 + return n, data[usable:] + + +class ButtonReader: + """Reads the headset button on a thread and calls on_press() per press, debounced. path: + an event device or, for testing, a FIFO carrying input_event structs. It's opened read-only + and shared; if it can't be opened (no device, no permission, /dev/input not reachable in a + container) it says so in the log and tries again now and then.""" + + def __init__(self, path, on_press, log=None, debounce_s=BUTTON_DEBOUNCE_S): + self.path, self.on_press, self.log = path, on_press, log or (lambda s: None) + self.debounce_s = debounce_s + self.ok = False # opened at least once + self._stop = threading.Event() + self._last = -1e9 + self._thread = threading.Thread(target=self._run, name="handrec-button", daemon=True) + + def start(self): + self._thread.start() + return self + + def stop(self): + self._stop.set() + self._thread.join(2) + + def _press(self, n): + now = time.monotonic() + if n and now - self._last >= self.debounce_s: + self._last = now + self.on_press() + + def _run(self): + failed = False + while not self._stop.is_set(): + try: + fd = os.open(self.path, os.O_RDONLY | os.O_NONBLOCK | os.O_CLOEXEC) + except OSError as e: + if not failed: + self.log("headset button: can't open %s (%s): the window's keys still work" % (self.path, e.strerror)) + failed = True + self._stop.wait(5) + continue + if failed or not self.ok: + self.log("headset button: reading %s" % self.path) + self.ok, failed = True, False + rest = b"" + try: + while not self._stop.is_set(): + r, _, _ = select.select([fd], [], [], 0.2) + if not r: + continue + try: + data = os.read(fd, INPUT_EVENT.size * 64) + except BlockingIOError: + continue + if not data: # a FIFO's writer left: open it again + self._stop.wait(0.2) + break + n, rest = button_presses(rest + data) + self._press(n) + except OSError as e: # the device went away + self.log("headset button: %s: %s" % (self.path, e.strerror)) + self._stop.wait(2) + finally: + os.close(fd) + + # ------------------------------------------------------------------------------------------ # The pose pictures (poses/poses.json: {"": {"file": ".png", "two_hands", "caption"}}) @@ -754,11 +929,13 @@ class Session: auto=False is step mode: each step waits for next_step(), then a recorded countdown, then the hold; nothing is recorded while it waits. auto=True: the timed flow, each prompt advancing by itself, one recording per take. next_after (a test hook): press Next by itself - after that many seconds of waiting. poses_dir: the pose pictures (poses/).""" + after that many seconds of waiting. poses_dir: the pose pictures (poses/). button: read the + headset's button (not in a dry run unless button_device, a test hook, names the device or + a FIFO of input_event structs).""" def __init__(self, base_dir, profile, checklist, lighting_choice, script_path, *, ring=None, start_processes=True, dry_run=False, speed=1.0, on_status=None, hands_dir=None, panel_bin=None, - auto=False, next_after=None, poses_dir=None): + auto=False, next_after=None, poses_dir=None, button=True, button_device=None): self.base_dir = os.path.abspath(os.path.expanduser(base_dir or BASE_DIR)) self.profile = dict(profile or {}) self.checklist = dict(checklist or {}) @@ -773,6 +950,9 @@ class Session: self.panel_bin = panel_bin or PANEL_BIN self.auto = bool(auto) self.next_after = next_after + self.button, self.button_device = bool(button), button_device + self.input_devices = INPUT_DEVICES # where mice are looked for (a test hook) + self._button = None # the ButtonReader self.print = print # where dry-run panel commands go (the CLI's stdout) self.script = load_script(self.script_path) self.plan, self.skipped = build_plan(self.script, self.checklist) @@ -790,7 +970,8 @@ class Session: "section_index": 0, "section_count": len(self.plan), "step_index": 0, "step_count": 0, "prompt": "", "seconds_left": 0.0, "note": "", "hands": {"left": None, "right": None}, "take": None, "error": "", "waiting": False, "countdown": 0, "big": "", "can_redo": False, - "image": "", "image_mode": "", "caption": "", "position": "", "distance": ""} + "image": "", "image_mode": "", "caption": "", "position": "", "distance": "", + "ready_text": READY_TEXT, "button": False, "mouse": True} self._last_emit = 0.0 self._log_file = None self._panel = None @@ -822,6 +1003,22 @@ class Session: self._want[key] = value self._wake.set() + def button_press(self): + """The headset's button: Next while a step waits, pause during a countdown or hold + (and auto mode's timed screens), resume while paused.""" + with self._lock: + paused = self._want["pause"] + state = self._status["state"] + if paused: + self.resume() + elif self._waiting: + self.next_step() + elif state in ("countdown", "running", "intro", "between"): + self.pause() + else: + return + self._log("headset button (%s)" % ("resume" if paused else "next" if self._waiting else "pause")) + def next_step(self): """Step mode: start the step that's waiting (its countdown).""" self._set("next", True) @@ -960,13 +1157,28 @@ class Session: self._panel = Panel(dry=self.dry_run, log=self._log, out=self.print) if not self.dry_run: self._start_panel() + self._start_button() self._panel.cmd("show") self._panel.cmd("paused off") for key, c in (("note", "note "), ("countdown", "countdown off"), ("hands", "hands off off"), ("bar", "bar off"), ("target", "target off"), ("image", "image off"), ("where", "where off"), - ("big", "big "), ("action", "action "), ("rec", "rec off"), - ("keys", "keys " + (KEYS_AUTO if self.auto else KEYS_STEP))): + ("big", "big "), ("action", "action "), ("rec", "rec off")): self._panel.set(key, c) + self._hints(action=False) + + def _start_button(self): + """The headset button's reader, unless turned off (a dry run has none unless a test + device is given). Without the device the session goes on: the window's keys work.""" + if not self.button or (self.dry_run and not self.button_device): + return + path = self.button_device or find_button(read_input_devices()) + if not path: + self._log("headset button: no %s device with KEY_SELECT in %s" % (BUTTON_DEVICE, INPUT_DEVICES)) + return + self._button = ButtonReader(path, self.button_press, log=self._log, debounce_s=BUTTON_DEBOUNCE_S).start() + end = time.monotonic() + 0.5 # opened in a moment, or it isn't reachable + while not self._button.ok and time.monotonic() < end: + time.sleep(0.01) def _write_calibration(self): src = host_path("/persist/xrservice.json") @@ -1116,6 +1328,9 @@ class Session: self._save_session() self._log("%s%s" % (state, ": " + error if error else "")) self._stop_units() + if self._button: + self._button.stop() + self._button = None if self._panel: screen = {"done": self.script.get("done") or {}, "stopped": self.script.get("stopped") or {}, "error": {"title": "Something went wrong", "text": "See the Hand recorder window.", @@ -1340,13 +1555,25 @@ class Session: if pos or dist else "where off") self._status.update(image=path, image_mode=mode, caption=caption, position=pos, distance=dist) + def _hints(self, action=True): + """The Next hint (with action) and the key line for what's there now: the headset button + leads when no mouse is connected. Looked at again for each step, so a mouse plugged in counts.""" + button = bool(self._button and self._button.ok) + mouse = mouse_connected(self.input_devices) + text = (READY_BUTTON_MOUSE if mouse else READY_BUTTON) if button else READY_TEXT + keys = (KEYS_AUTO_BUTTON if self.auto else KEYS_STEP_BUTTON) if button else (KEYS_AUTO if self.auto else KEYS_STEP) + if action: + self._panel.set("action", "action " + text) + self._panel.set("keys", "keys " + keys) + self._status.update(ready_text=text, button=button, mouse=mouse) + def _await_next(self): """Step mode: wait for Next (or the next_after test hook), handling the controls and the hands chips. Nothing records meanwhile.""" with self._lock: self._want["next"] = False # one pressed during the hold doesn't skip this self._waiting = True - self._panel.set("action", "action " + READY_TEXT) + self._hints() self._emit(waiting=True, seconds_left=0.0) t0 = time.monotonic() try: @@ -1606,7 +1833,7 @@ class Session: self._start_recording() tick = None if st["kind"] == "bar": # the bar at near, where the sweep starts - labels = "%s %s" % (s.get("near_label", "Near"), s.get("far_label", "Far")) + labels = "%s|%s" % (s.get("near_label", "Near"), s.get("far_label", "Far")) near, far = float(s.get("near_m", 0.2)), float(s.get("far_m", 0.6)) def tick(dt): @@ -1729,7 +1956,7 @@ class Session: """One height of the push out and back: the target sweeps near to far and back.""" near, far = float(s.get("near_m", 0.2)), float(s.get("far_m", 0.6)) period, lead = float(s.get("period_s", 6)), st["lead"] - labels = "%s %s" % (s.get("near_label", "Near"), s.get("far_label", "Far")) + labels = "%s|%s" % (s.get("near_label", "Near"), s.get("far_label", "Far")) p = st["p"] self._begin_prompt(s, p, where) state = {"t": 0.0, "sent": 0.0} @@ -1771,6 +1998,10 @@ def main(): ap.add_argument("--next-after", type=float, metavar="S", help="test: press Next by itself after S seconds of waiting (real time)") ap.add_argument("--poses", help="the pose pictures' folder, with poses.json (default hands/rec/poses)") + ap.add_argument("--no-headset-button", action="store_true", + help="don't read the headset's button (gpio-keys KEY_SELECT: Next, pause, resume)") + ap.add_argument("--button-device", metavar="PATH", + help="test: read the button from this event device or FIFO of input_event structs (also in a dry run)") a = ap.parse_args() known = ("pencil", "phone", "cup", "keyboard", "mouse", "gamepad", "small") @@ -1817,7 +2048,8 @@ def main(): s = Session(base, profile, checklist, a.lighting, a.script, ring=a.ring, start_processes=not a.no_start, dry_run=a.dry_run, speed=a.speed, on_status=on_status, hands_dir=a.hands_dir, panel_bin=a.panel, - auto=a.auto, next_after=a.next_after, poses_dir=a.poses) + auto=a.auto, next_after=a.next_after, poses_dir=a.poses, button=not a.no_headset_button, + button_device=a.button_device) est = plan_seconds(s.script, s.plan, auto=a.auto) worst = plan_seconds(s.script, s.plan, worst=True, auto=a.auto) print("%d sections, %s mode: about %.1f min%s (at most %.1f)%s" % ( diff --git a/hands/rec/tests/test_session.py b/hands/rec/tests/test_session.py index c96a412..4e01e1f 100644 --- a/hands/rec/tests/test_session.py +++ b/hands/rec/tests/test_session.py @@ -10,8 +10,10 @@ import os import shutil import sys import tempfile +import struct import time import unittest +from unittest import mock HERE = os.path.dirname(os.path.abspath(__file__)) sys.path.insert(0, os.path.dirname(HERE)) @@ -32,7 +34,9 @@ SCRIPT = { {"text": "Open.", "seconds": 4, "hands": "both", "pose": "open"}]}]} -class SessionTest(unittest.TestCase): +class SessionBase(unittest.TestCase): + """Dry-run sessions of SCRIPT, steered from the test.""" + def setUp(self): self.tmp = tempfile.mkdtemp(prefix="handrec-session-test-") self.script = os.path.join(self.tmp, "script.json") @@ -67,6 +71,9 @@ class SessionTest(unittest.TestCase): with open(os.path.join(s.session_dir, "takes", "01-poses", "prompts.jsonl")) as f: return [json.loads(line) for line in f] + + +class SessionTest(SessionBase): def test_step_flow(self): s = self.session(next_after=0.0) s.start() @@ -197,6 +204,192 @@ class SessionTest(unittest.TestCase): self.assertIn("2 steps", session.plan_summary(SCRIPT, plan)) +# /proc/bus/input/devices as the Frame has it (2026-10-02), trimmed, plus a USB mouse. +DEVICES = """I: Bus=0019 Vendor=0001 Product=0001 Version=0100 +N: Name="gpio-keys" +P: Phys=gpio-keys/input0 +S: Sysfs=/devices/platform/gpio-keys/input/input3 +U: Uniq= +H: Handlers=kbd kbd event3 qcom_pon_combo_timer_listener +B: PROP=0 +B: EV=3 +B: KEY=100000000000 0 0 0 0 200000000 0 0 0 0 0 + +I: Bus=0006 Vendor=4d44 Product=0001 Version=0001 +N: Name="frametop virtual mouse" +P: Phys= +S: Sysfs=/devices/virtual/input/input4 +U: Uniq= +H: Handlers=event4 qcom_pon_combo_timer_listener +B: PROP=0 +B: EV=7 +B: KEY=ffffff 0 0 0 0 +B: REL=ffff + +I: Bus=0003 Vendor=0001 Product=0001 Version=0001 +N: Name="frame-voice keyboard" +P: Phys=py-evdev-uinput +S: Sysfs=/devices/virtual/input/input7 +U: Uniq= +H: Handlers=kbd sysrq kbd event7 qcom_pon_combo_timer_listener +B: PROP=0 +B: EV=200003 +B: KEY=1ffffffffffffff ffffffffffffffff ffffffffffffffff fffffffffffffffe + +I: Bus=0005 Vendor=214e Product=0035 Version=0001 +N: Name="Z3 Keyboard" +P: Phys=90:82:c3:5f:4a:a7 +S: Sysfs=/devices/virtual/misc/uhid/0005:214E:0035.0005/input/input27 +U: Uniq=da:ef:2f:d2:be:e2 +H: Handlers=kbd sysrq kbd event9 qcom_pon_combo_timer_listener +B: PROP=0 +B: EV=10001f +B: KEY=300000000000 33eff 0 0 483ffff17aff32d bfd4444600000000 1 130ff38b17c007 ffff7bfad9415fff feb2ffdfffefffff fffffffffffffffe +B: REL=1040 +B: MSC=10 +""" +BT_MOUSE = """ +I: Bus=0005 Vendor=214e Product=0035 Version=0001 +N: Name="Z3 Mouse" +P: Phys=90:82:c3:5f:4a:a7 +S: Sysfs=/devices/virtual/misc/uhid/0005:214E:0035.0005/input/input26 +U: Uniq=da:ef:2f:d2:be:e2 +H: Handlers=event8 qcom_pon_combo_timer_listener +B: PROP=0 +B: EV=17 +B: KEY=ffff0000 0 0 0 0 +B: REL=1943 +B: MSC=10 +""" +USB_MOUSE = """ +I: Bus=0003 Vendor=046d Product=c077 Version=0111 +N: Name="Logitech USB Optical Mouse" +P: Phys=usb-xhci-hcd.1.auto-1/input0 +S: Sysfs=/devices/platform/soc@0/a600000.usb/xhci-hcd.1.auto/usb1/1-1/1-1:1.0/0003:046D:C077.0001/input/input40 +U: Uniq= +H: Handlers=mouse0 event13 +B: PROP=0 +B: EV=17 +B: KEY=70000 0 0 0 0 +B: REL=903 +B: MSC=10 +""" + + +def event(etype, code, value): + return session.INPUT_EVENT.pack(1, 2, etype, code, value) + + +PRESS = event(session.EV_KEY, session.KEY_SELECT, 1) + event(0, 0, 0) +RELEASE = event(session.EV_KEY, session.KEY_SELECT, 0) + event(0, 0, 0) + + +class InputTest(unittest.TestCase): + def test_devices(self): + devs = session.parse_input_devices(DEVICES) + self.assertEqual([d["name"] for d in devs], ["gpio-keys", "frametop virtual mouse", "frame-voice keyboard", + "Z3 Keyboard"]) + self.assertEqual(session.find_button(devs), "/dev/input/event3") + self.assertFalse(any(session.real_mouse(d) for d in devs)) # the virtual mouse, keyboards with wheels + for extra, name in ((BT_MOUSE, "Z3 Mouse"), (USB_MOUSE, "Logitech USB Optical Mouse")): + mice = [d["name"] for d in session.parse_input_devices(DEVICES + extra) if session.real_mouse(d)] + self.assertEqual(mice, [name]) + # another gpio-keys without KEY_SELECT isn't the button + self.assertIsNone(session.find_button(session.parse_input_devices(DEVICES.replace("200000000", "0")))) + + def test_presses(self): + data = (PRESS + event(session.EV_KEY, session.KEY_SELECT, 2) * 3 # autorepeat: not presses + + RELEASE + event(session.EV_KEY, 115, 1) + PRESS) # another key + self.assertEqual(session.button_presses(data), (2, b"")) + n, rest = session.button_presses(PRESS + PRESS[:10]) + self.assertEqual((n, rest), (1, PRESS[:10])) # half an event waits for the rest + self.assertEqual(session.button_presses(rest + PRESS[10:])[0], 1) + + def test_reader_fifo(self): + tmp = tempfile.mkdtemp(prefix="handrec-button-test-") + try: + fifo = os.path.join(tmp, "button") + os.mkfifo(fifo) + presses = [] + reader = session.ButtonReader(fifo, lambda: presses.append(time.monotonic()), debounce_s=0.3).start() + with open(fifo, "wb", buffering=0) as w: + w.write(PRESS + RELEASE) + w.write(PRESS[:7]) # a press split across writes, a bounce + time.sleep(0.05) + w.write(PRESS[7:] + RELEASE) + time.sleep(0.4) + w.write(PRESS + RELEASE) + time.sleep(0.2) + with open(fifo, "wb", buffering=0) as w: # the writer comes back: read again + time.sleep(0.4) + w.write(PRESS) + time.sleep(0.3) + reader.stop() + self.assertEqual(len(presses), 3) + self.assertTrue(reader.ok) + finally: + shutil.rmtree(tmp, ignore_errors=True) + + def test_missing_device(self): + logs = [] + reader = session.ButtonReader("/nonexistent/event99", lambda: None, log=logs.append).start() + time.sleep(0.1) + reader.stop() + self.assertFalse(reader.ok) + self.assertIn("can't open", logs[0]) + + +class ButtonSessionTest(SessionBase): + """A dry run steered with a simulated headset button (a FIFO), with no mouse, then one.""" + + def test_button(self): + fifo = os.path.join(self.tmp, "button") + os.mkfifo(fifo) + procfile = os.path.join(self.tmp, "devices") + with open(procfile, "w") as f: + f.write(DEVICES) + writer = os.open(fifo, os.O_RDWR) # kept open, so the reader never sees the end + press = lambda: os.write(writer, PRESS + RELEASE) + try: + with mock.patch.object(session, "BUTTON_DEBOUNCE_S", 0.0): + s = self.session(speed=4, button_device=fifo) + s.input_devices = procfile + s.start() + st = self.waiting(s, "starting") + self.assertEqual((st["button"], st["mouse"], st["ready_text"]), (True, False, session.READY_BUTTON)) + self.assertIn("panel: action " + session.READY_BUTTON, self.panel) + self.assertIn("panel: keys " + session.KEYS_STEP_BUTTON, self.panel) + press() + self.waiting(s, "intro") + press() + self.waiting(s, "ready", "Fist.") + press() # Next + self.wait_for(s, lambda st: st["state"] == "running", "the hold") + press() # pause + self.wait_for(s, lambda st: st["state"] == "paused", "paused") + with open(procfile, "a") as f: # a mouse arrives + f.write(BT_MOUSE) + press() # resume + st = self.waiting(s, "ready", "Open.") + self.assertEqual((st["mouse"], st["ready_text"]), (True, session.READY_BUTTON_MOUSE)) + press() + s.join(20) + self.assertEqual(s.state, "done") + names = [e["event"] for e in self.events(s)] + self.assertEqual(names, ["take", "ready", "prompt", "pause", "resume", "wait", "ready", "prompt", "wait", + "end"]) + finally: + os.close(writer) + + def test_no_button(self): + s = self.session(next_after=0.0, button=False, button_device="/nonexistent") + s.start() + s.join(20) + self.assertEqual(s.state, "done") + self.assertIn("panel: action " + session.READY_TEXT, self.panel) + self.assertIn("panel: keys " + session.KEYS_STEP, self.panel) + + class ManyPartsTest(unittest.TestCase): """A take recorded in 40 parts (step mode stops the recording between steps): review reads them in order, export makes one stream, validate passes."""