mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-14 23:00:44 +02:00
experimental
73
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 DeeJanuzandClaude Opus 5.5
|
afab3cbecc |
pointer: the 3D mouse is a treadmill again, now that it can move windows
Reverts
|
||
|
DeeJanuzandClaude Opus 5.5
|
797e5c7b12 |
pointer: the 3D mouse is the right hand again, not a treadmill
As a treadmill (the default since
|
||
|
DeeJanuzandClaude Opus 5.5
|
9d6f5984a8 |
hands: near input on SteamVR's own panels through the pointer helper's laser
The dashboard, the Steam menu, Steam's keyboard and other apps' overlays now
take a poke or a pinch on the content as a click, and a pinch on the border,
like a card or at a corner moves or resizes a floating one. The edge and
corner glows show there too.
OpenVR gives no transform or size for dashboard panels, so ft-screens casts a
7x7 cone of rays from the eyes past each index tip and fits each panel to the
hits' texture coordinates (steampanels.cpp). Near input treats the fitted
panels as foreign: no fist grabs, two-hand stretch, throws, flicks, pushes,
rewinds or two-finger scroll.
ft-screens can't click those panels, so it drives the helper's laser with
"touch" lines (hover, press, move, up, grab, end). A move or resize first
hovers inside the edge and presses the grab bar or resize margin only once a
floating panel's window controls show under it; otherwise the log says why.
The helper keeps its overlay list while hands are allowed and sends it to
ft-screens ("steamvr ...").
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
09126151d8 |
Merge branch 'hands-feed' into experimental
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
4c20053101 |
Hands: far two-fist resize, hand-tap reset asks first, easier lone-hand lazy susan
- Far stretch: both palms held out at the same far panel, then two fists within 0.6 s, scales it round its centre (a floating window by pixels) until a hand opens. Aims at the panel both hands' lines meet, or the line through the palms' midpoint. Grab that holds its fire meanwhile, and both hands are claimed from the pointer helper so a fist passing through a pinch doesn't click or drag under a raised hand ray. Near misses log why. - Reset button: a hand's tap shows "Reset the layout? Tap" under it for 5 s; a tap on it (or the button again) resets. A laser click resets at once. - Lazy susan: with the other hand still or unseen, a looser shape, 18 cm and 0.5 m/s count; any stroke rides out 2 frames off its shape; misses log open/tilt/lead against their limits. - Pointer helper: the two-hand gaze drag no longer logs "with the right button" (it is the left alone). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
1faa0241e2 |
Hands: easier pinch throws; sent away, a panel keeps its distance and size; reset on a tap; glow under hand rays only
Throws: a carry's let-go takes the fastest outward stretch in the last
150 ms before the fingers opened (a hand slows as it lets go), at 0.45 m/s
within 70 deg (was 0.55 within 60, the last 100 ms only). Every let-go
moving 0.3 m/s or more is logged.
Sent away (thrown, flicked, pushed), a panel goes along that line out as
far and as big as it was before a hand or "grab that" took it, not back
to its old spot (layouts are for that). Put down past 1.2 m, that's its
distance from then on; never taken, it flies to 3 m as before. The front
slot's screen leaves the slot the same way.
The bar's buttons sit a button's width apart, and the reset button
resets the layout on a tap (HANDS_RESET_HOLD_MS=0; 500 before).
Grab that's glow: ft_pointer's laser sits on the panel with the cursor
all the time (the 3D mouse, the gaze), so "no glow under a laser" kept
the glow off that panel. The pointer helper now tells ft-screens which
hand rays are up ("ray L R", on change and every second); our laser
counts as one only then. Controllers' lasers count as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
80bce4cf66 |
Gaze + hands: the two-hand drag works like Meta+J/K; held pinches no longer drop on a clock race; easier lazy susan
User 2026-10-11, the second gaze test: - "hand tracking stopped" ends were an unsigned age: ft-hands publishing just after the loop's tnow made it negative, wrapped to 584 years, and ended held pinches, nudges, drags and scrolls. Signed now. - The two-hand drag mirrors the keys: hold one pinch (aim, nudge), pinch with the other: the left button goes down where the dot is (as Meta+K with Meta+J held); the drag lasts while either pinch holds, and the hand still pinching drags. A second pinch during a drag joins it. - The fast-hand filter refused dozens of still pinches (frame-to-frame palm speed is noisy): smoothed, 1.0 m/s. - A three-finger scroll goes on when ft-hands swaps which finger it calls pinched. - Lazy susan: counted strokes came at 25-30 cm in 230-470 ms, at both limits: 22 cm, quick somewhere along it (0.7 m/s peak) within 0.9 s; looser hand shape. The flags replayed: 2 strokes in 45 flags of other use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
0b72710474 |
Gaze + hands: two-hand drag (hold, tap the other), pinches held through tracking pauses, no clicks from fists or strokes, the pointer hides while typing; the slot keeps only on real use
The user's 2026-10-11 test of gaze + hands: - Two-hand drag as they meant it: hold one hand's pinch (nudge it into place), tap with the other: index presses the left button where the pointer is, middle the right; the first hand drags (POINTER_GAZE_TWO_HAND). The tap-first right-drag is off by default (it could catch a tap followed by a nudge). - 9 held pinches ended as "hand tracking stopped (1.5 s)" while ft-hands still held them: 4 s with nothing pressed, 2.5 s with a press, and the line now says the file's seq and ages to find the cause. - A gaze pinch held back ends without a click when the hand closes into a fist (ft-hands' grip may not take it past 0.1 s now); a pinch begun while the palm moves faster than 0.6 m/s (a stroke) doesn't click (POINTER_PINCH_MAX_SPEED). - Three-finger scroll: 3.5 cm and 0.4 s to join. - Gaze mode hides the dot and collapses its laser for 1.5 s after each key (POINTER_TYPING_HIDE): it sat big and bright on the text being typed. - The front slot's keep counts clicks, scrolls, drags and keys, not the gaze pointer passing over it (every brought screen was kept 3 s in). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
02f1575c52 |
Gaze + hands first: right-drag by two taps, three-finger scroll, fist guard on every pinch; the slot lets go to manual controls; palm-only susan; dot-only hand laser
User 2026-10-11, gaze + hands becoming the main input: - Right-drag: an index tap on one hand, then an index pinch on the other within 1 s, holds the right button at the gaze lock; that hand drags until it ends (POINTER_GAZE_RIGHT_DRAG). It replaces the chord (POINTER_GAZE_CHORD=0). - Three-finger scroll: the thumb on both the index and middle tips (3 cm) scrolls where you looked as the hand moves, like the ray's middle-pinch stick (POINTER_GAZE_SCROLL). - The fist guard (a hand closing into a fist: no click) now covers gaze and chord pinches too, from each hand's own frame (not the ray's). - A hand's laser shows only its dot: the beam starts just short of it, as the eye's does, held still through a carry (POINTER_RAY_BEAM=0; SteamVR's beam width is one global setting it reads only at start). - The front slot gives a screen up to manual controls: grab that, a push, flick or throw, or a carry by hand or laser releases it (it moves like any other and stays where you put it); the next stroke puts it home and brings the next. - The lazy susan counts only palm-led strokes (the return stroke does nothing; the other hand palms the other way; HANDS_SWEEP_BACK=1 for both). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
12b8159d4e |
Lazy susan: readable at 1 m, cycles every screen; the laser only for pinching, none in a fist
Lazy susan (the third live test): a brought screen comes 1 m ahead, as big as its
resolution needs for text to read: its logical pixels (KWin's scale undone, now passed
to vr.cpp) at 26 px/deg, at most 100x55 deg keeping its shape, at least 40 deg wide,
curved round you past 60 deg (grabthat::ReadableSize; SCREEN_SPIN_DISTANCE, _PPD,
_MAX_DEG, _MAX_TALL_DEG). Its curve comes back on the way home. The strokes cycled
104, 1, 104 with three screens up: a screen still flying home read as straight ahead;
each one is now taken where it lives (home, or where it's going).
Accidental clicks: most were pinches with no laser up pressing where the pointer was
last left ("no ray up: a press where the pointer is", a dozen in two minutes). With
gaze off they now only wake the pointer (POINTER_PINCH_NO_RAY=1: the old air mouse).
The hand's laser shows only with the thumb and index within 6 cm (gone past 10;
POINTER_RAY_READY), and a fist drops it: fh_ray's "ready" and "open" tests, the fist
read as the index and the other three both curled under 1.1 (FistCurl; measured on 47k
flagged hands: the other three alone read as a fist in a fifth of the pinches). A ray
pinch made while the hand closes into a fist doesn't click. Input Settings switches for
all three (POINTER_ ones reload the helper).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
95d2c696ab |
Hands: two-fist resize by shape, two-finger scroll, no stuck presses, ray hidden near panels, quieter glows
From the 2026-10-11 09:43 live test: - Two fists anywhere near one panel (palms 30 cm in front to 15 behind, 30 cm round it) resize it by their shape alone (HANDS_FIST_PAIR): ft-hands' grip only begins on a hand seen open just before, at the panel. - Two fingers (index and middle straight, together, level; ring and little curled) touching the content: a swipe scrolls (a notch per 1.2 cm, the content following), a lift unmoved clicks, held still they act as a poke (HANDS_TWO_FINGER_SCROLL). - Stuck presses: a press isn't held through a tracking loss (only grabs are); a poke lets go as the index curls; a pinch-click as the tips are seen 4.5 cm apart; either held still 1.5 s (HANDS_PRESS_STILL_MS). - The hand ray hides within 22 cm in front of a panel and for 0.8 s after (HANDS_RAY_HIDE_REACH); the pointer helper ignores a ray-less pinch there without the gaze (it woke the laser). - The push/flick hint glow is off (HANDS_PUSH_GLOW): it covered near panels the palm faced while clicking. - Grab that gives panels that look small up to 10 deg more aim margin (HANDS_GRAB_THAT_SMALL). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
1521420964 |
Hands: fists resize only, easier throws, natural grab that, curve that fits, no laser while touching
From the 2026-10-11 09:25 live test: - A fist only resizes (HANDS_FIST_RESIZE_ONLY, default on): one fist holds the panel still as an anchor; two hands with a fist among them scale it about its centre, no move or turn. Pinches still carry and turn. - Throws need 0.55 m/s (was 1.0): the user's releases were 0.62-0.96 m/s. - Grab that brings a panel back at the width it had when it was last sent away from arm's reach (g_nearWidth), not grab that's maxWidth. - A natural reach arms grab that: palmArm 0.2 (was 0.45), aimFacing 80 deg (was 55), aim margin 10 deg (was 6). Flag replays: 11 fires (was 8). - Curve: a curved screen's radius follows its distance from the head after every move, and it bends at most SCREEN_CURVE_MAX_DEG (100) round: one curved up close and thrown 3 m out at full width had wrapped into a cylinder. - The pointer helper releases its laser while a hand touches a panel (ft-screens' near claim): the laser from the head to the fingertip's cursor got in the way and held the button, refusing pokes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
639a761d88 |
ft-pointer: log the hands feed going quiet, and say so when a ray goes off for it
A side "off" for no hand tracking logged as "off: its controller is in
use" (State() != Hand covered both), which hid the 2026-10-10 flicker
behind controller handoff. Now each change of the feed's liveness logs one
line ("hand tracking: nothing new for 1.5 s: sides off" / "publishing"),
and a ray dropped for it says "off: no hand tracking".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
d1ceb3d744 |
ft-pointer: a hands frame newer than the loop's clock is fresh, not 584 years old
The loop reads its clock at the top and ft-hands' files later in the pass.
A frame written in between has publish_ns after tnow, and the unsigned
"now - publish_ns" in HandsInRoom::Live wrapped round: the feed read as
stale for that one pass, both sides went "off", and the "hands" word to
ft-screens flickered to "off off" and back. 2026-10-10 20:30-20:37: 24
flickers in 7 minutes, each with ft-hands publishing at full rate (pinches
logged at the same moments), each ending ft-screens' holds and the ray.
feedage.h: AgeNs/Fresh, age 0 for a frame from after "now". Live and the
gestures check that ends holds ("hand tracking stopped") use them.
pointer/tests/feedage-test.cpp replays the loop against 30, 15 and 1 Hz
publishing: the old check calls 28-2408 passes stale, the new one none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
06cdb2ed8b |
Resize from any corner, for hands, the gaze and lasers
User, 2026-10-10: Vision Pro-style resize from all four corners. Only the bottom right worked by laser (the .resize handle), and a resize kept the screen centred. - One resize handle per panel still (SteamVR's 128 overlays are tight: a third remote display once didn't fit), but it moves to the corner being aimed at (vr.cpp UpdateCorner): a laser's ray (a controller's, or the pointer helper's device, which the gaze and the hand ray drive) landing within a fifth of the screen's height (5-20 cm) of a corner, or a hand's pinch at one (handinput's corners()). Its texture turns for that corner, it brightens the closer the aim (from the controls' idle 0.55 to 1), and it's lit blue while resizing. With no aim it lingers 0.4 s, then goes back to the bottom right. - Any corner resizes and the opposite one stays put: screens and remote displays keep their aspect (the corner follows the diagonal, ResizeFromCorner, on the cylinder for curved ones: SurfacePose), floating windows take the pixels from that corner (ft-floatd's new size is placed keeping the opposite corner, keepX/keepY; every corner of a window resizes now, the title bar's ends included). - Near hands: a pinch at any corner resizes the same way (the carry now gives the pose as well as the width). - Gaze pinches (gazehand.h): with the pointer on a screen's resize handle or grab bar, the pinch is a press after 0.12 s still (POINTER_GAZE_CONTROL_HOLD), not 0.5: nothing to click there, and the hand drags the resize. The chrome overlays are announced to the helper as they're made, so its ray and the gaze find a new remote display's handle at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
99da5e8942 |
Hands: the touch zone's margin hides the ray but leaves pinches to the helper
The near report to the pointer helper ("near <ns> <L> <R>") was one flag
for two things: the hand's ray hides, and its pinches and grips are
ft-screens'. With the 0.075 m zone, its 2.5 cm leave margin and 200 ms
linger (there so the ray doesn't flicker at the edge) would still have
swallowed pinches made just outside the zone, the live test's complaint.
Now ft-screens says 1 when the hand is within the zone itself (or
pressing, grabbing, or waiting on "grab that"): its gestures are ours; 2
in the margin: the ray stays hidden, the gestures are the helper's (ray or
gaze); 0 otherwise. The helper takes 1 as the claim and 1 or 2 as near for
the ray; an older helper takes 2 as 1, as before. ft-screens' own pinch
and fist begins need the zone itself too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
8d3de8e13f |
Merge branch 'hands-gazechord' into hands-next: the gaze chord, hand input switches
Conflicts: - ft-pointer.cpp, the gaze pinch comment: typing's "the rules for all gestures hold (..., typing, ...)" kept, gazechord's chord and POINTER_GAZE_PINCH paragraphs added after it, and a line that a hand typing holds back makes no chord. - ft-pointer.cpp, the hands settings: gazechord's gazePinchOn, nearOn, handsTrack kept; typingHold dropped for typing's typing::Params. - ft-pointer.cpp, mayPoint: both conditions (handsOn, and not a side typing holds back). - ft-pointer.cpp, beginGazePinch: gazechord's busy line (a chord isn't busy), without the old sinceKey/typingHold lines. The chord's pinch goes through typing.h's Why in the gesture loop before startGesture, like any pinch, so a hand typing holds back can't make one. - docs/reference.md settings line: typing's and reach's keys, plus POINTER_GAZE_PINCH. - hands/README.md near input: typing's hold-back sentence and gazechord's HANDS_NEAR=on. - session/frametop.conf.example: reach's HANDS_GRAB_ROTATE* and HANDS_GRAB_THAT* lines, then gazechord's HANDS_NEAR line. Reconciled: gazehand's Situation loses sinceKey and typingHold, and Refused its "ignored (typing)": the helper had stopped setting them on hands-typing (typing.h holds hands back first), so that guard could never fire. gazehand-test's key checks now drive typing.h: keys hold a hand back (2 s now, was 1), POINTER_PINCH_TYPING=0 is no guard, and a chord's pinch by a hand on the keys is held back while one lifted off them goes ahead. The chord's own pinch counts as a gesture held for typing's "busy" (free if out of the keyboard zone when keys come), as the left's hold does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
24d2d12397 |
pointer: the gaze chord in the helper, and hand methods as their own switches
The helper sends the chord's right button (btn b) apart from the left's press, so both reach the panel; the chord ends with its pinch, its hand lost, the pointer going, a controller on its side, or the calibration panel. POINTER_GAZE_PINCH (1) turns clicks at the gaze off on their own. HANDS_NEAR=on runs the handoff for ft-screens' direct touch without POINTER_HANDS (HANDS_NEAR=1 keeps its meaning), and the relay feeds it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
56fb3b3d82 |
pointer: typing holds hands back: no pinches, grips or ray, and ft-screens hears it
The helper feeds typing.h the relay's keys and each hands frame. A hand it holds back makes no pinch, middle pinch or grip (ray, gaze or plain), and has no ray: one that drove the pointer drops it (to the mouse or the gaze, not the controllers) and can't claim it again until lifted off the keys. A hand pointing or holding a gesture out of the keyboard zone isn't held back by the other hand's keys. ft-screens gets "typing <L> <R>" on a change and every second. The old 1 s guards (beginHold, gazehand's) give way to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
93adab7302 |
Hand ray: the torso faces the head and the hands (StereoKit's body model)
Ideas from StereoKit (MIT, 07b9f0d), credited where a formula comes from it: - The torso faces the head's forward counted twice plus the direction to each wrist seen (POINTER_RAY_TORSO_WRISTS, on), so looking aside while pointing swings the shoulders, and the ray, less. probe-moves replay: the shoulder's share of a still hand's ray motion 20 -> 18%, p90 2.0-3.7 -> 1.7-3.6 deg/s; the tap capture: nothing worse. - The ray can turn toward the hand's own direction, from the palm's axes (wrist, middle, ring and little knuckles) tilted -30/8 deg (POINTER_RAY_HAND_WEIGHT, 0). Off: every weight measured worse (at 0.3: judder 0.07-0.14 -> 0.11-0.33 deg rms per 90 Hz tick, raw jumps over 3 deg 159 -> 330). The gate still tests the shoulder's ray. - The knuckle point can be pinned to the palm's axes during a pinch, easing back on the steady shoulder's schedule (POINTER_RAY_ANCHOR, 0). Off: our pinch point is the knuckles already, and pinned it shook as much or more and kicked no less on release, fitted or not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
ac5111c552 |
Merge branch 'experimental' into hands-full
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
9def5f1762 |
Gaze calibration panel: a gaze precision or gaze drag press takes the dot
While the panel was up, the helper took only "btn trigger 1" and "gazekey left 1" as "take this dot". A left button mapped to gaze precision or gaze drag sends "precision|gazedrag <source> 1" instead, which the panel dropped, so the click did nothing. The panel's answers move to pointer/helper/calpanel.h, with an offline test (pointer/test/calpanel-test.sh). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
e4c51ccf6b |
Pointer: the drawn hands' overlays aren't panels, so the hand ray and pinches pass through them
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
db2e633502 |
Merge branch 'hands-gaze' into hands-full
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
0d6b0c93b5 |
Hand ray: drags move smoothly and hold steady, and a quick tap stays a click
The ray's One Euro filters (new hands/include/fh_smooth.h) run in continuous time and the helper reads the ray every loop, so the pose sent to ft_pointer moves at about 100 Hz instead of jumping at the tracker's 15-30 Hz (the dashboard's dragger made those jumps jerky). The direction is filtered on the sphere. On the probe-moves replay the pose's second difference per 90 Hz tick went from 0.90-0.96 deg rms to 0.24-0.26 at the same lag. While a ray pinch is held the ray pivots on the shoulder as it was at the press, so looking around doesn't move what's dragged; the reach for push and pull is filtered. A pinch's first 0.3 s takes 6 deg of turn before it drags. One failed read of the gestures file no longer ends a drag, and the log says why a drag ended or the ray went off. Settings: POINTER_RAY_SMOOTH_MINCUTOFF, POINTER_RAY_SMOOTH_BETA, POINTER_RAY_SMOOTH_HOLD, POINTER_RAY_STEADY, POINTER_RAY_TAP, POINTER_RAY_TAP_DEADZONE. rayeval reads publish dumps and reports the smoothness at the display's rate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
d348e0a260 |
Gaze + hands: look and pinch to click, pinch and move to nudge the pointer
With gaze mode and POINTER_HANDS on, the gaze is the pointer and the hand ray stays off (POINTER_GAZE_HAND_RAY=1 brings it back). A pinch clicks where you looked 75 ms before ft-hands saw it; held and moved past 4 deg of hand movement (POINTER_GAZE_PINCH_DEADZONE) it nudges the pointer at 0.3x (POINTER_GAZE_PINCH_GAIN), and the correction is a gaze lesson. Held still for POINTER_GAZE_HOLD it's a press the hand drags. A middle pinch right-clicks at the gaze. One log line per pinch. Rules in pointer/helper/gazehand.h, tested in pointer/tests/gazehand-test.cpp. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
7ccf34c042 |
Merge branch 'hands-view' into hands-full
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
3d1eee828a |
ft-handview: your hands drawn over games, the dashboard and the void with Room View off
Quest-style: a see-through light hand from the tracked skeleton with a bright rim, fading out behind the wrist, over everything. Each hand is a placed, side-by-side overlay facing the head, so it stays put as the head turns and redraws only as the hand moves. Hidden while a hand holds a controller (the pointer helper's word, now also on @ft_handview). frametop-handview.service runs it; nothing installs it yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
07ce0d9d59 |
Hand ray: the laser stays with one hand, and a pinch clicks where it pointed
Live test 2026-10-09: taps took several tries and the laser jumped between hands. The owner's ray dropping for a moment (closing the fingers turns the hand) handed the laser to the other raised hand at once, for good; a pinch by the other hand pressed where the old owner's ray pointed; a pinch whose ray had just dropped became a press at a stale spot, or only woke the pointer. - The owner keeps the laser through POINTER_RAY_GRACE (0.5 s) of its ray being down; the other hand takes it by pinching, by raising its ray while the owner's is down, or after the grace (pointer/helper/rayowner.h). - A pinch on a ray that dropped within the grace, or is on its way up, brings it back (fh_ray.h ShowForPinch) and is a ray pinch. - Aim lock: the press goes along the ray from just before the fingers began to close, after a pose along it has gone out; a tap that ends before its press clicks there too. - Each ray follows one tracked hand through a brief flip of ft-hands' left/right call. - Log: owner changes with the reason, the owner's ray down and back, and one line per pinch saying what it clicked or why not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
400fe0ca5f |
Pointer: ray gestures only with the ray still up; presses wait for a fresh wake
A gesture that waited for ft-screens' near report is a ray gesture only if its ray is still up when it starts; a ray pinch's press waits until a pointer the ray just woke is bound (350 ms). keys-test's pause stub knows the relay's new calls. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
d16f774d50 |
Hands: park hand tracking during VR games instead of stopping it
With hand interaction on (POINTER_HANDS=1), the hands have to be there when the dashboard opens during a game (design section 5.5): - ft-hands: while /run/user/UID/frametop-hands/park exists it processes no frames (models stay loaded), publishes no hands, ends gestures as lost, and writes 0 to hands-want every second. - ft-camd: stops copying the mono frames while hands-want is fresh and 0 (a dead ft-hands' stale file doesn't starve other ring readers). - The input relay (game_pause.py) leaves the hand services running when it pauses for a game and parks them instead; it unparks while the dashboard is open (the helper's new "vrdash 1|0") and on resume. - hands/README.md: gestures v3, parking, and the hand ray. - pause-test.py: parking, and "paused" kept off the live helper. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
5d96d211ec |
Pointer: the hand ray drives ft_pointer; ray pinch, push and pull
With POINTER_HANDS=1 and POINTER_HAND_RAY=1 (design sections 2, 5): - Per side, each hands frame goes through fh_ray's gate; the first ray up drives the pointer (a pinch by the other hand's ray takes it). It wakes the pointer and claims the laser unless a controller pressed within 2 s. The mouse takes the pointer back; the ray returns after 1 s of no mouse. - The pose is the ray's origin and direction (a visible beam); our dot only where it meets a panel, no catcher in free space. A controller taking the laser hides our dot until a ray pinch claims it back. - A ray pinch is the left button, 30 ms after a claim it had to make; the direction holds until POINTER_RAY_DEADZONE (2.5 deg), then drags. A middle pinch (fh_gestures v3) is a right click, or scrolling when held and moved. Gaze pinches and grips are unchanged with the ray down. - Push and pull while carrying (POINTER_RAY_PUSH): the origin moves along the ray, distance d0 (h/h0)^3. - ft-screens' "near <capture_ns> <L> <R>": a gesture waits for the report on its frame (60 ms at most) and is left to ft-screens if near. - The ray lets go of a pointer it woke POINTER_RAY_RELEASE after it went; at once when its side's controller is in use or in a game. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
564bb4bdc5 |
Pointer: treadmill role by default, and per-hand last used wins
- The driver's default role is the treadmill (4), hinted from Activate; the helper's POINTER_ROLE defaults to treadmill. Needs the driver reinstalled and a SteamVR restart to take effect. - The "no hand role" release skips a device whose hint is the treadmill. - Hand gestures: none begins on a side whose controller is in use, or by a hand at a controller (by position); a held one ends, as lost, when its side turns controller. - The moving-controller release no longer applies while a bare hand's ray drives the pointer (that ray goes only when its own side turns controller). - update-check.py: the virtual controller's path, and the touch components. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
742bd32c32 |
Pointer: handoff arbiter for hand interaction (log only)
Per side, a controller used recently wins, else the bare hand (pointer/helper/handoff.h, design section 1): touch and presses from the input relay, motion, charging and tracking from SteamVR, and the tracked hands from ft-hands' hands file (handsin.h, which also takes the gestures reader and the head pose history out of ft-pointer.cpp). - The relay's web socket reader (game_pause.py TouchTracker) sends "ctl <side> touch 0|1", "ctl <side> press", "ctl <side> gone" and "ctl ws 0|1" with POINTER_HANDS=1, and the pause state as "paused 1|0". - The helper logs transitions and sends ft-screens "hands <L> <R> <allowed>" (hand|controller|off, 0|1) on a change and every second. Nothing acts on it yet. - fh_gestures v3 adds the middle pinch slots (written as zero for now). - Tests: pointer/tests/handoff-test.cpp (just test-c), input/test/touch-test.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
2a675b9121 |
Remote displays: other computers' monitors as Frametop screens
A remote display is a screen of ft-screens whose picture comes from another computer, streamed by Vibepollo with Moonlight's protocol. ft-screens (screens/remote.c) runs one ft-stream per remote screen (101 and up) over a socket pair: ft-stream (stream/, GPLv3: moonlight-common-c and moonlight-embedded's libgamestream, pinned) decodes on the iris decoder into a ring of three RGBA buffers that ft-screens shows as a panel with a screen's controls, layout, profiles, curve, pins and hand cutouts. Mouse, keys and scroll go back to the host; a button's release goes through the stream it was pressed on, as Vibepollo wants, and a press carried onto another remote panel moves the pointer there. A panel out of sight gets 10 fps; the host's sound plays from one stream per host. ft-stream signs in with an API token and the host's pinned public key (stream/host.cpp), and chooses its path: a Steam Link dongle on the Frame's hotspot when it answers, or the network (the layout's "route": auto, network or dongle). ft-layout gets "remote" commands (list, add, connect, disconnect, host route and direct), starts the connected displays with the desktop and keeps disconnected ones parked. ft-gaze and ft-pointer know the remote panels. ft-screens now shuts its VR side down before its streams and waits for SteamVR before exiting (a vrcompositor crash at quit, in standby). docs/remote-displays.md has the design, the spikes and the test results. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
8b9edcfbe2 |
Pointer: a click the pointer dropped as it went off isn't a right click later
In gaze mode a mouse press is held back while you aim, and its release clicks. A release of the right button sets pressRight with the click, so the click goes out as a right one. When the pointer goes off in the same loop (the relay's stand_down sends the release, then "hide", as Frametop pauses), the click is dropped, but pressRight stayed set. The next press after resume, from the mouse, Meta+J or a pinch, then went out as a right click. Dropping the click now forgets pressRight with it. A "hide" read a loop after the release still comes too late to stop the click; that needs the helper's socket to be full as the pause starts, and is noted in the relay's stand_down. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
4828dd46b5 |
Pointer: a release never wakes the pointer
Any "btn" or "scroll" from the relay woke a pointer that was off. When Frametop pauses for a game, the relay's stand_down sends the releases of the clicks held into the pause, and with its pointer already off (the idle timeout or pointer_toggle during a press) it used to send them with no "hide" after. The helper woke on them, connected the virtual controller and took the right hand role during the game. Now a button release or a scroll back to 0 0 doesn't wake it. The release is still handled as before (it ends its press, or goes to the driver), so the driver doesn't reconnect later with the button down. Outside a pause this changes one case: a mouse button let go after a moving controller released the pointer mid-press. That release used to reconnect the controller, recenter it and release there; now it updates the disconnected controller, and ft-screens still gets its "up". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9d7c8a0b91 |
Experimental (#29)
* Input relay: typing with the pointer helper down no longer ends the relay Typing on a pass-through keyboard tells the helper "typing". With the helper not running (SteamVR off), that send raised ConnectionRefusedError and the relay exited, dropping every grab until systemd restarted it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ft-steam: open Steam's menu through Steam's own UI steam/ft-steam menu opens the SteamVR dashboard on Steam's menu, or closes the dashboard if it's up, without pointer mode: it asks Steam's UI over its debugging port to show its dashboard overlay (ShowVROverlay, what Steam calls itself) and focus the Steam frame's left menu (MenuStore.OpenMainMenu). ft-steam check says whether those calls still exist, and update-check.py runs it, since a Steam client update can rename them. The CDP client moves from display-settings/steam_settings.py to steam/steamui.py, so both use it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Shortcuts: Steam menu, commands, and modifier taps Key combinations (and mouse and controller buttons) get two new actions: - steam_menu: Open Steam menu / close dashboard (steam/ft-steam menu). - command:CMD: run CMD with sh -c, as the relay's service, with layout/, float/ and steam/ on its PATH. Input Settings offers it for key combinations as Run a command... Both work without pointer mode. A modifier on its own is now a key combination too: a tap, pressed and released with no other key, mouse button, or scroll in between. A bound tap sends the desktop F24 before the release, so Plasma's launcher stays shut. The defaults gain a Meta tap for the Steam menu; this replaces META_DASHBOARD, which only worked in pointer mode. input/test/keys-test.py runs the relay against fake devices with every outgoing socket renamed, so it's safe next to the live relay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Relay: share a key combination's Meta release with frame-voice A Meta+key combination (Meta+J for gaze_left, say) hides Meta's release from the desktop, and the relay skipped share_key for it too. frame-voice saw Meta go down on @frametop_keys and never come up, so it held all dictated text back, waiting for that release. Keys of grabbed keyboards are now shared as pressed, before key_binding() decides what the desktop gets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: ft-cutouts, the hand cutouts without pinches and grips hands/ft-cutouts on|off|status starts ft-camd and ft-hands as transient user units with ft-hands' new --no-gestures: hands are published for ft-screens' cutouts, but no pinch or grip is detected, so nothing clicks or drags and a closing hand doesn't raise the tracking rate. It needs a build and ft-camd's capabilities, not hands/run.sh install. Its units conflict with ft-handsctl's, so each stops the other, and they stop with SteamVR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ft-cutouts status: only the current run's tracker lines Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Gaze: say why gaze mode can't work yet, and open the calibration whenever it's missing Turning gaze mode on without a calibration opened Calibrate only on the off-to-on change, and only if it could open right then. With the headset off, the eye tracker silent, or the panel not built, or with gaze mode already on when the gaze service started, nothing opened and nothing said why: the pointer just stayed a mouse. - The gaze service now checks every second: gaze mode on, no calibration for the tracker in use, eyes seen -> the full calibration opens. One that closes unfinished opens again only after the headset comes off and on, gaze mode off and on, or Calibrate, so it doesn't loop. A start that fails retries every 10 s. - Its status says why gaze mode can't work yet (checks.problem): not calibrated and opening, open, closed unfinished, or can't open and why. - Input Settings shows that under the Gaze pointer switch, along with the gaze service not installed or not running and our tracker missing its frame grabber. - ft-gazectl on notes a missing calibration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Gaze: install our eye tracker, prefer it, and say why a calibration dot wasn't taken A user on a fresh install got "Calibration failed: only 0 of 21 dots" with no reason. The installer never installed our tracker, so gaze used SteamVR's, and the only way SteamVR's tracker rejects a dot is losing an eye for most of the look. The panel just showed a red ring. - install.sh: step 9/10 installs our tracker (gaze/tracker/install.sh) after gaze mode, yes by default; it needs sudo, so --yes runs it only when sudo won't prompt. If it fails, gaze keeps SteamVR's tracker. Configs that still say GAZE_TRACKER=steam (the old template) are asked whether to switch. - GAZE_TRACKER=auto, the new default: ours when it's installed (the frame grabber, its unit, and ft-eyes' Python), else SteamVR's. ft-gazed rechecks every second, so installing it switches over. Input Settings lists Own tracker first as recommended, and says how to install it when it's missing (checking the host's /etc through /run/host from the dev container). - The calibration panel has a note line, orange over the instructions: why a dot wasn't taken (an eye lost, a blink, the eyes disagreeing for SteamVR's tracker, from steady_samples' new drop counts; ft-eyes' reply for ours), what a click is still waiting for after 1.5 s, and a failed calibration's most common reason, which the Gaze page shows too. steady_samples keeps the same samples as before (checked on 2037 windows of recordings); a lost eye is named before a blink, since its openness reads 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * README: link the Frametop Discord Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Wait for a new container to finish setting up before entering it container-up.sh starts the dev container in a scope of its own, so distrobox enter finds it running and skips its wait for distrobox-init. On a fresh install, init was still setting up passwordless sudo when dev-container.sh ran sudo dnf install, and sudo asked for a password with no terminal to read it from. container-up.sh now waits for container_setup_done itself, and the container's sudo calls use -n, so a password prompt fails at once with a clear message. Fixes #9 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Prevent small desktop overlay pointer movements from starting a drag * Clear reported drag state on controller release * Click stability: only a hand controller's press starts it The 3D mouse drives SteamVR's laser through the ft_pointer virtual controller, so its events reach the screens the same way a controller's do. The filter held every press, which turned the mouse's short drags (selecting a character or two, nudging a slider) into clicks. Mark button events from hand controllers and start the filter only on those. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Input relay: retry a new device until udev gives it to the input group A new /dev/input node is root:root 0600 until udev applies GROUP=input. The scan probed each new node once and marked it seen even when the open failed, so a node caught in that gap was never opened. Behind a KVM, a switch brings back a hub of devices at once: on the Frame, four nodes failed with EACCES in one switch, the keyboard was never grabbed, and its keys went to gamescope instead of the desktop screens. A node that isn't readable yet now waits for the next scan. * Screens: take a screen's overlays from one copy in the catcher While a button pressed on a screen is held, UpdateCatcher checks every tick whether the laser is still on one of the screen's overlays. It built that list from s.All().begin() and s.All().end(), but All() returns a std::array by value: iterators into two different temporaries, which is undefined behaviour. A clang build of ft-screens got a garbage length, threw std::length_error, and aborted on the first click, taking KWin and the desktop with it. * Gaze: leave SteamVR's gaze action alone during VR games From curiousjtuber's PR #13: with the gaze service running, SteamVR restarted its eye tracker every 10 to 13 s of Beat Saber, as if the headset came off, and each restart took input focus from the game. The PR stopped every read in a game. Only the action path reaches SteamVR (UpdateActionState on the gaze set at overlay-global priority, then GetEyeTrackingDataRelativeToNow); the mmap and our tracker are read-only files. So only the action is skipped while a scene app runs, and gaze keeps moving the pointer over the dashboard in a game. The action source is only used with --source action. Co-Authored-By: CuriousJ <curious.j.tuber@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: --record-hz, and the hand recorder's design (hands/rec/DESIGN.md) ft-hands --record-hz N records at most N frame sets a second, for the hand recorder (10). DESIGN.md lays out the recorder: the headset panel, the session runner and its script, the files, review and export, consent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: ft-handpanel, the hand recorder's headset panel A head-locked SteamVR overlay for the hand recorder (hands/rec/DESIGN.md): 1.2 m ahead, 12 degrees up, 36 degrees wide, drawn with stb_truetype into three shared DMA-BUFs as ft-gazepanel does. It shows the title, step, wrapped instruction, note, countdown, hand chips, near/far bar and a "Paused" cover, driven over @ft_handpanel. It also places the touch target, a 2 cm dot in its own overlay fixed in the room where the head was at the first command for that point, and logs head and controller poses to poses.jsonl at 250 Hz from a thread of its own. Both threads take one lock around OpenVR calls. --no-vr prints each picture's state to stdout (and --dump writes the pictures), for testing without a headset. hands/rec/build.sh builds it in the dev container. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: the recorder's worn check goes by the panel's backlight, as frame-job does Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: the hand recorder's session runner and guided script hands/rec/session.py runs a recording session from script.json: it starts ft-camd and a tracking ft-hands as transient units only if they aren't running, records each section as one take (ft-hands --record-only at 10 sets/s, a new sets-N.bin after each pause), drives ft-handpanel, and writes session.json, calibration.json (identifying fields removed), prompts.jsonl and take.json. Feedback comes from the live hands file and, in the controller sections, from the panel's device poll. It also runs from the command line (--dry-run, --speed, --ring, --no-start). hands/rec/script.json: 11 sections, about 9 minutes without the object and controller sections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: the hand recorder's window, review and export hands/rec/ft_handrec.py + main.qml (Kirigami, dev container; host launcher hands/rec/ft-handrec): consent (CONSENT.md, asked again when its version changes; profile.json with a random contributor id), the before-you-start checklist with the lighting and free-space checks, the session controls (Space pauses, Esc stops), review with a frame-set viewer that deletes ranges, takes and sessions, export with progress and cancel (warns while the headset is worn), and the upload page (UPLOAD.md, the huggingface-cli command; HF_DATASET is a placeholder). --dry-run runs sessions without processes, for testing. hands/rec/takes.py (standard library): indexes sets.bin and sets-N.bin without reading pixels, reads one set's cameras, keeps deleted ranges in take.json, and exports: deleted sets left out, zstd -10 -T2 at nice 19, manifest.json and SHA256SUMS, nothing left behind on cancel. CONSENT.md and UPLOAD.md are drafts pending a legal review; the window says contributions aren't open yet. The dev container gains zstd. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: upload from the hand recorder's window, export checks, a rehearsal hands/rec/validate.py (standard library; Linux and Windows, Python 3.12+) checks an export before upload and when it's received: SHA256SUMS, an allow-list of files, the manifest's schema and keys, the consent version, a uuid4 contributor, no identifying fields in calibration.json or device.json, every sets.bin.zst decompressed to its end as a stream with each FHSET01 header checked against the manifest, jsonl lines, the total size. It decompresses with compression.zstd, zstandard or the zstd program. validate.py DIR [--json]. hands/rec/hub.py uploads an export with huggingface_hub, as a pull request to contributions/<contributor>/<session>: validate first, refuse a repeat of the same export, check the login (whoami) and access (auth_check), upload_folder(create_pr=True) with the manifest summary as the description, then record the PR under "uploads" in session.json. Errors are explained (terms not accepted, not found, 401/403, network). --dry-run makes no network calls. FT_HANDREC_DATASET overrides HF_DATASET (DeeJanuz/frametop-hands); while the texts are drafts a real upload needs FT_HANDREC_ALLOW_UPLOAD=1. The Upload page shows the login with "Check again" and how to run hf auth login in a terminal (the token never enters the window), then Upload with a phase, progress and Cancel (hub.py as a child process), the PR link, and a warning for an export uploaded before. The manual command stays as the fallback. ft-handrec --hub-dry-run. session.py also saves device.json: cv.cad_from_cal and head from /persist/device_config.json, the labeller's shape, nothing identifying; export copies it. Session ids with a -N suffix are accepted everywhere. hands/rec/rehearse.sh runs it all without the headset: ft-ringplay plays 30 s of a capture into a ring, session.py records a short test script with ft-handpanel --no-vr and a tracker, then export, validate and a dry-run upload (--repo ID uploads for real). It runs in one frame-job scope, deletes its data and stops its processes, also on Ctrl+C. hands/rec/tests/test_validate.py covers good and broken exports and hub.py without the network. The dev container gains python3-huggingface-hub. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * README: Frametop doesn't work on the SteamOS beta yet Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit |
||
|
DeeJanuzandClaude Opus 5.5
|
0ac2b10cd7 |
Remove leftovers before a release
- gaze/tracker/lab/eyes_track.py: nothing used it - Input Settings: the pointer role backend, whose page was removed - ft-screens: no log line for every floating-window resize - hands: ft-hands --help gives the palm-down default (1: off), the uninstall removes ft-handsctl's link, .frame-job is ignored - Display Settings: "Save as profile…", not "Save current arrangement" - two stale comments (ft-pointer's grabprobe, ft-gaze) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
964c250858 |
Gaze mode: a double right click or double Meta+K pans and tilts a drag
A drag begun by right during the left's held-back press (or Meta+K during Meta+J's) now lasts while either button or key is held, so pressing the right one again is free: it tilts, as a right press does during any mouse drag. Meta+K during a keyboard drag (held still into one, or the second Meta+K) tilts while held: the head turns the panel, and the mouse can too; let go and the head drags again from there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
1f46616c75 |
Gaze mode: the mouse's buttons work like Meta+J and Meta+K
The right button's press is held back like the left's: hold it, the pointer stops where you look, move onto the target, and the right click comes on the release (held still, it's a real right press). Right during the left's held-back press now starts a drag where the pointer is, as Meta+J then Meta+K does, in place of a right click there; letting go of either drops it. So once you've moved, the left alone only clicks, and the right starts a drag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
a2a5a4c98e |
Gaze mode: the mouse only corrects, and left-then-right right-clicks
POINTER_GAZE_MOUSE_MOVE=held, the default (the Gaze page's Mouse movement switch, with why it's on): while the gaze has the pointer, moving the mouse does nothing; it moves the pointer only while a button is held, as a correction. A bumped or drifting mouse can't pull the pointer off what you're looking at, and every mouse move is a correction, so lessons aren't polluted by mouse moves to somewhere else. With the gaze stale for a second, in a game, or with the headset off, the mouse moves the pointer as usual. "free" is the old behaviour. Pressing the right button while the left one's press is held back right-clicks where the pointer is instead (correct with the left, then right-click); both releases are then nothing. Meta+J then Meta+K stays a drag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
8998fc1ddd |
Gaze learning limit: 55 degrees, past it a quick check; a solid panel
Drops the quick check's second step (
|
||
|
DeeJanuzandClaude Opus 5.5
|
362968198b |
Quick check: put the pointer on the dot, and one limit for learning
After the quick check's capture, in gaze mode, the dot stays where it is in the
room (the panel's new "lock") and the gaze has the pointer again. You put the
pointer on the dot as you'd correct a click, with the mouse or Meta+J and the
head. That click clicks nothing: the helper sends it as "calverify", learned
whatever its size. Over POINTER_GAZE_NUDGE_MAX, the capture runs again, up to
3 times. A right click or Meta+K skips it.
POINTER_GAZE_NUDGE_MAX is now the one limit: 15 degrees by default (was 8),
and ft-gazed's own limits (8 for SteamVR's tracker, 25 for ours) follow it. A
keyboard correction past it isn't learned but opens a quick check ("recheck"),
in place of the 30 degree keyboard limit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
cbad7bc470 |
Keyboard corrections are learned up to 30 degrees
A Meta+J hold's correction is always meant, but it went through the mouse's 8 degree nudge limit. Live, our tracker was 12 degrees off and every correction was dropped without a word. Debug output now says when one is. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
c4c10e6d92 |
Gaze checks and calibration in a panel fixed to the headset
ft-gazepanel (gaze/panel) is a SteamVR overlay that stays put in your view and shows dots at known head-relative directions; the gaze service runs it and drives it (gaze/gazecheck.py): - a one-dot quick check 3 s after the headset goes on, when our tracker asks for a click (reseat), at most every 2 minutes, and from Quick check on the Gaze page; five dots follow if the next 3 lessons are still over 2 degrees off - the full calibration (the probe's three rounds, dark to bright) when gaze mode comes on without one, or from Calibrate; quitting it with still no calibration turns gaze mode off Each dot takes the gaze once it has held still for 0.6 s; a left click or Meta+J takes it at once, a right click or Meta+K closes the panel (the pointer helper hides its dot and passes those presses on while "calpanel" lasts). Our tracker gets clicks and its own calibration; SteamVR's gets lessons per eye, or a new calibration.json. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
dc3a1b8208 |
Keyboard clicks at the gaze: Meta+J and Meta+K
gaze_left and gaze_right (Meta+J and Meta+K by default) click where you look.
A tap clicks where the dot was at the press and tells the gaze tracker it was
right there. Held, the pointer stays put in your view, so turning your head
carries it onto the target; the release clicks there and the correction is a
lesson (judged by the net correction, not the head's path). Held still for
POINTER_GAZE_HOLD it's a real press that the head drags, and Meta+K while
Meta+J aims presses where the dot is now, to correct and then drag.
A Meta combination now also sends the desktop an F24 press and Meta's release
at once: Meta's release no longer opens Plasma's launcher, and the click isn't
Meta+click (KWin's window move and resize, which swallowed right clicks). The
desktop gets Meta back for the next key if it's still held.
Also fixes
|
||
|
DeeJanuzandClaude Opus 5.5
|
e16b7588e2 |
Gaze mode is a mouse feature: drop its controller parts
Steam reads the Frame controllers itself, outside SteamVR's bindings, and every press and release it sees takes SteamVR out of laser mode, so controller clicks at the gaze can't be done cleanly (docs/gaze-controllers.md, from the gaze-first branch's tests). - Gaze precision and Gaze drag take a mouse button or a key combination only; the controller source, its aim steering, and POINTER_PRECISION_GAIN, POINTER_PRECISION_DEADZONE and POINTER_GAZE_DRAG_GAIN are gone. - The gaze actions (including Gaze pointer on/off) can't be mapped to controller buttons: the relay ignores them there and doesn't ask the helper for those buttons, and Input Settings no longer offers them. - Last used wins in gaze mode too: picking up a controller hands it the laser, as docs/design.md already said. - The gaze dot shows all the time (POINTER_GAZE_DOT=always, the default; moving brings back the old behaviour), with a switch on the Gaze page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
d666fa031f |
Gaze first: precision and drag buttons, key combinations, pointer role
Two new actions for mouse buttons, controller buttons, and key
combinations: gaze_precision (hold: the pointer stops where you look and
the button's device steers it, a controller by where it points at
POINTER_PRECISION_GAIN, the mouse by its moves; release: click there) and
gaze_drag (the same with a real press at once, dragging until the
release). The relay sends "precision|gazedrag <source> 1|0" to the
helper, which runs them through the same holds as pinches and grips.
- Key combinations on any keyboard ("key_bindings" in the rules, e.g.
Ctrl+Alt+G for gaze on/off): the last key isn't typed.
- POINTER_GAZE_MOUSE: in gaze mode the left button is a precision button
(the default, as before) or clicks right away (direct).
- In gaze mode a moving controller no longer takes the pointer away.
- POINTER_ROLE (right, left, stylus): the driver takes a "role" command,
so the pointer can stay off the hand holding the precision controller,
which would otherwise take the role back. Needs the rebuilt driver.
- Input Settings: the actions on the Controllers and Buttons pages; on the
Gaze page the mouse choice, the role, the precision sliders, and key
combinations (captured from any keyboard).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
c2739cb0df |
Hands: park hand tracking behind ft-handsctl
Hand tracking no longer starts with SteamVR: hands/run.sh install leaves the units disabled and links hands/ft-handsctl into ~/.local/bin, which turns it on and off (on | off | status | log | cutouts on|off | gestures). With it on, hands show through the screens; pinches and grips move the pointer only with POINTER_HANDS=1, now off by default. ft-camd's service runs the mono cameras only: while the headset is worn, the colour module writes just a half-size image into the top-left quarter of its buffers, which ft-camd can't use yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |