steam-frame/DeeJanuz--frametop
1
0 Fork 0
mirror of https://github.com/DeeJanuz/frametop.git synced 2026-10-06 13:00:10 +02:00
Code Issues Packages Projects Releases 4 Wiki Activity

Quick check: put the pointer on the dot, and one limit for learning

Browse Source 
After the quick check's capture, in gaze mode, the dot stays where it is in the
room (the panel's new "lock") and the gaze has the pointer again. You put the
pointer on the dot as you'd correct a click, with the mouse or Meta+J and the
head. That click clicks nothing: the helper sends it as "calverify", learned
whatever its size. Over POINTER_GAZE_NUDGE_MAX, the capture runs again, up to
3 times. A right click or Meta+K skips it.

POINTER_GAZE_NUDGE_MAX is now the one limit: 15 degrees by default (was 8),
and ft-gazed's own limits (8 for SteamVR's tracker, 25 for ours) follow it. A
keyboard correction past it isn't learned but opens a quick check ("recheck"),
in place of the 30 degree keyboard limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DeeJanuzandClaude Opus 5.5 committed 2026-10-01 10:41:52 -06:00
1 parent cbad7bc470
commit 362968198b
9 files changed
+180 -55
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

pointer/helper/ft-pointer.cpp
+62 -29
View File
@@ -181,9 +181,8 @@
// your view, so turning your head carries it onto what you meant (past
// POINTER_HEAD_DEADZONE, 0.5 deg, so a still head doesn't wobble it). The release clicks there
// (left, or right for gaze_right), and a correction is a lesson for the gaze tracker, as with
// the mouse, but up to kKeyNudgeMax (30 deg) rather than POINTER_GAZE_NUDGE_MAX: a keyboard
// correction is always meant, and our tracker can be 12 deg off after the headset goes on (live,
// 2026-10-01: every correction was dropped). Held still for POINTER_GAZE_HOLD instead, it's a real press, and the head drags.
// the mouse. One past POINTER_GAZE_NUDGE_MAX isn't learned, but since a keyboard correction is
// always meant, the gaze service is told ("recheck <deg>") and runs its quick check. Held still for POINTER_GAZE_HOLD instead, it's a real press, and the head drags.
// A quick tap (let go within POINTER_KEY_TAP, 0.25 s) clicks where the dot was at the press,
// whatever the head did meanwhile, and tells the gaze tracker it was right there (a lesson
// with no correction). Pressing gaze_right while gaze_left aims (Meta+K with Meta+J held)
@@ -193,6 +192,11 @@
// ft-gazed says it's up ("calpanel 1", renewed every second; it lapses 3 s after the last),
// the dot hides and a press answers the panel instead of clicking: a left click or gaze_left
// sends "calaccept" to @ft_gazed (take this dot now), a right click or gaze_right "calquit".
// "calpanel 2" is the quick check's second step: the dot it captured stays put in the room, the
// gaze has the pointer again, and you bring the pointer onto the dot as you'd correct a click
// (mouse, or a keyboard click and the head). That click clicks nothing: it goes to @ft_gazed
// as "calverify <raw yaw> <raw pitch> <true yaw> <true pitch>", a lesson with no size limit. A
// press doesn't become a real one when held still; a right click or gaze_right is "calquit".
// POINTER_ROLE (right, left, or stylus): the hand role our device takes while connected. A
// Frame controller in your hand counts as used through its touch sensors and takes its hand's
// role back, and then no click lands (see "no hand role" in the main loop): with a controller
@@ -270,7 +274,7 @@
// panel, and a laser that starts behind them can't hit them. POINTER_FOLLOW (0) and
// POINTER_LEASH_DEG (10), POINTER_LEASH_DELAY (0.2 s), POINTER_LEASH_RETURN (0.2 s),
// POINTER_FOLLOW_REACH (70 deg): head follow, above. POINTER_GAZE (0), POINTER_GAZE_RETAKE
// (5 deg), POINTER_GAZE_NUDGE_MAX (8 deg), POINTER_GAZE_HOLD (0.5 s), POINTER_GAZE_DOT
// (5 deg), POINTER_GAZE_NUDGE_MAX (15 deg), POINTER_GAZE_HOLD (0.5 s), POINTER_GAZE_DOT
// (always), POINTER_GAZE_SHOW (1 s): gaze mode, above. POINTER_CONTROLLER_PICKUP (1, 0.5 to 5): how hard a controller must
// move to take the laser back, above. POINTER_IGNORE (empty): ignored panels, above.
// POINTER_HANDS (0), POINTER_PINCH_GAIN (0.5), POINTER_PINCH_DEADZONE (1.5 deg),
@@ -684,7 +688,7 @@ int main() {
bool follow = false, followConf = false, followReset = true;
// Gaze mode (see the top); gazeConf is POINTER_GAZE as last read, like followConf.
bool gazeOn = false, gazeConf = false;
double gazeRetake = 5, gazeNudgeMax = 8, gazeHold = 0.5, gazeShow = 1;
double gazeRetake = 5, gazeNudgeMax = 15, gazeHold = 0.5, gazeShow = 1;
bool gazeDotAlways = true; // POINTER_GAZE_DOT (see the top)
double headDeadzone = 0.5, keyTap = 0.25; // POINTER_HEAD_DEADZONE, POINTER_KEY_TAP (keyboard clicks, see the top)
// Hands (see the top): POINTER_HANDS, POINTER_PINCH_GAIN, POINTER_PINCH_DEADZONE, POINTER_GRIP_GAIN.
@@ -713,7 +717,7 @@ int main() {
const bool wantFollow = ConfDouble(conf, "POINTER_FOLLOW", 0) != 0;
if (wantFollow != followConf) follow = followConf = wantFollow, followReset = true;
gazeRetake = std::clamp(ConfDouble(conf, "POINTER_GAZE_RETAKE", 5), 1.0, 45.0);
gazeNudgeMax = std::clamp(ConfDouble(conf, "POINTER_GAZE_NUDGE_MAX", 8), 1.0, 30.0);
gazeNudgeMax = std::clamp(ConfDouble(conf, "POINTER_GAZE_NUDGE_MAX", 15), 1.0, 30.0);
gazeHold = std::clamp(ConfDouble(conf, "POINTER_GAZE_HOLD", 0.5), 0.1, 5.0);
gazeShow = std::clamp(ConfDouble(conf, "POINTER_GAZE_SHOW", 1), 0.0, 30.0);
headDeadzone = std::clamp(ConfDouble(conf, "POINTER_HEAD_DEADZONE", 0.5), 0.0, 5.0);
@@ -907,30 +911,46 @@ int main() {
bool pressRight = false;
std::string heldButton = "trigger";
bool confirmLesson = false; // a keyboard click's quick tap: a lesson with no correction (see the top)
// A keyboard click's correction: always meant, so learned up to kKeyNudgeMax (see the top).
bool keyLesson = false;
constexpr double kKeyNudgeMax = 30;
// The gaze calibration panel is up until then ("calpanel 1"; see the top); calOpened: it just
bool keyLesson = false; // the click is a keyboard click's correction (see the top)
// The gaze calibration panel is up until then ("calpanel 1|2"; see the top); calOpened: it just
// came up, so a press in progress ends without a click.
Clock::time_point calPanelUntil{};
bool calOpened = false;
int calMode = 0; // 1: presses answer the panel; 2: the quick check's second step (see the top)
auto calVerifying = [&] { return calMode == 2 && Clock::now() < calPanelUntil; };
// The nudge as "<word> <raw yaw> <raw pitch> <true yaw> <true pitch>" (see the top).
auto nudgeMessage = [&](const char *word) {
const Vec3 d = RotateInverse(nudgeHead, Normalize(lastPoint - Position(nudgeHead)));
char msg[160];
std::snprintf(msg, sizeof msg, "%s %.3f %.3f %.3f %.3f", word, nudgeRawHy, nudgeRawHp,
std::atan2(-d.x, -d.z) * 180 / M_PI, std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI);
if (debug) std::printf("gaze %s (nudged %.2f deg)\n", msg, nudgeMoved);
if (debug) std::fflush(stdout);
return std::string(msg);
};
auto pressLeft = [&] {
const bool haveNudge = gazeOn && nudging && !gazeOwns && havePoint &&
Clock::now() - nudgeAt < std::chrono::seconds(10);
if (calVerifying()) {
// The quick check's second step (see the top): the click is the correction, whatever
// its size, and goes to the gaze service; nothing is clicked.
if (haveNudge) SendTo(out, "ft_gazed", nudgeMessage("calverify"));
else if (debug) std::printf("calverify: no gaze at the press\n");
nudging = confirmLesson = keyLesson = pressRight = gazeBack = false;
if (gazeOn) gazeOwns = true;
pulseAt = Clock::now();
return;
}
// ft-screens sends the keyboard to the panel clicked last; it sees clicks on
// its own screens, but only we know when one lands on another panel.
SendTo(out, "ft_screens", "click " + (lastHit.empty() ? std::string("-") : lastHit));
// A click after nudging the gaze-placed pointer: the nudge is a lesson (see the top).
const double nudgeMax = keyLesson ? kKeyNudgeMax : gazeNudgeMax;
if (debug && gazeOn && nudging && !gazeOwns && nudgeMoved > nudgeMax)
std::printf("gaze nudge %.2f deg: over %.0f, not learned\n", nudgeMoved, nudgeMax);
if (gazeOn && nudging && !gazeOwns && havePoint && Clock::now() - nudgeAt < std::chrono::seconds(10) &&
(confirmLesson || (nudgeMoved >= 0.2 && nudgeMoved <= nudgeMax))) {
const Vec3 d = RotateInverse(nudgeHead, Normalize(lastPoint - Position(nudgeHead)));
char msg[160];
std::snprintf(msg, sizeof msg, "lesson %.3f %.3f %.3f %.3f", nudgeRawHy, nudgeRawHp,
std::atan2(-d.x, -d.z) * 180 / M_PI, std::asin(std::clamp(d.y, -1.0, 1.0)) * 180 / M_PI);
SendTo(out, "ft_gazed", msg);
if (debug) std::printf("gaze %s (nudged %.2f deg)\n", msg, nudgeMoved);
if (haveNudge && (confirmLesson || (nudgeMoved >= 0.2 && nudgeMoved <= gazeNudgeMax)))
SendTo(out, "ft_gazed", nudgeMessage("lesson"));
else if (haveNudge && nudgeMoved > gazeNudgeMax) {
if (debug) std::printf("gaze nudge %.2f deg: over %.0f, not learned\n", nudgeMoved, gazeNudgeMax);
if (debug) std::fflush(stdout);
if (keyLesson) SendTo(out, "ft_gazed", "recheck " + std::to_string(int(std::lround(nudgeMoved))));
}
nudging = confirmLesson = keyLesson = false;
leftHeld = true;
@@ -965,8 +985,8 @@ int main() {
// the top).
auto leftButton = [&](bool down) {
if (down) {
if (gazeOn && gazeMousePrecision && gazeOwns && !aimHeld && !clickPress && !clickRelease &&
hold.src == Src::None) {
if (gazeOn && (gazeMousePrecision || calVerifying()) && gazeOwns && !aimHeld && !clickPress &&
!clickRelease && hold.src == Src::None) {
// Hold the press back: the pointer stops where the gaze put it.
gazeOwns = false;
nudging = haveHead && Clock::now() - gz.at < std::chrono::milliseconds(200);
@@ -1211,11 +1231,23 @@ int main() {
if (std::sscanf(buf, "calpanel %d", &on) == 1) {
const bool was = Clock::now() < calPanelUntil;
calPanelUntil = on ? Clock::now() + std::chrono::seconds(3) : Clock::time_point{};
calMode = std::clamp(on, 0, 2);
if (on && !was) calOpened = true;
continue;
}
}
if (Clock::now() < calPanelUntil) {
if (calVerifying()) {
// The quick check's second step: left presses are the correction (pressLeft); a
// right press skips it.
if (!std::strncmp(buf, "btn b 1", 7) || !std::strncmp(buf, "gazekey right 1", 15)) {
SendTo(out, "ft_gazed", "calquit");
continue;
}
if (!std::strncmp(buf, "btn b 0", 7) || !std::strncmp(buf, "gazekey right 0", 15) ||
!std::strncmp(buf, "precision ", 10) || !std::strncmp(buf, "gazedrag ", 9))
continue;
}
if (calMode == 1 && Clock::now() < calPanelUntil) {
const bool accept = !std::strncmp(buf, "btn trigger 1", 13) || !std::strncmp(buf, "gazekey left 1", 14);
const bool quit = !std::strncmp(buf, "btn b 1", 7) || !std::strncmp(buf, "gazekey right 1", 15);
if (accept || quit) {
@@ -1997,7 +2029,7 @@ int main() {
auto secs = [&](Clock::time_point t) { return std::chrono::duration<double>(tnow - t).count(); };
alpha = std::clamp(1 - std::min(secs(lastMove) - gazeShow, secs(lastHeld)) / 0.25, 0.0, 1.0);
}
if (tnow < calPanelUntil) alpha = 0;
if (tnow < calPanelUntil && calMode == 1) alpha = 0;
overlay->SetOverlayAlpha(marker, float(alpha));
overlay->SetOverlayWidthInMeters(marker, float(2 * SETTINGS_DOT * std::tan(cursorDeg * M_PI / 360)));
auto mm = Billboard(near, eye);
@@ -2027,7 +2059,7 @@ int main() {
alpha = std::max(alpha, std::clamp((0.6 - pulse) / 0.3, 0.0, 1.0));
}
}
if (tnow < calPanelUntil) alpha = 0; // the calibration panel is up (see the top)
if (tnow < calPanelUntil && calMode == 1) alpha = 0; // the calibration panel is up (see the top)
overlay->SetOverlayAlpha(show, float(alpha));
overlay->SetOverlayWidthInMeters(show, float(2 * dist * std::tan(scale * cursorDeg * M_PI / 360)));
auto m = Billboard(at, eye);
@@ -2088,14 +2120,15 @@ int main() {
// A held-back press (see the top): held still long enough, it's a real press (a drag);
// released, it's a click where the pointer is now (this frame's pose has gone out).
if (!active) aimHeld = clickPress = aimHand = confirmLesson = keyLesson = false; // released meanwhile: nothing to click
if (aimHeld && !aimHand && nudgeMoved < 0.2 && tnow - aimSince >= std::chrono::duration<double>(gazeHold)) {
if (aimHeld && !aimHand && nudgeMoved < 0.2 && tnow - aimSince >= std::chrono::duration<double>(gazeHold) &&
!calVerifying()) {
aimHeld = false;
gazeBack = true;
pressLeft();
}
// A keyboard click held still for POINTER_GAZE_HOLD: a real press, then the head drags
// (from where it is now).
if (aimHeld && aimHand && hold.promote && !hold.engaged &&
if (aimHeld && aimHand && hold.promote && !hold.engaged && !calVerifying() &&
tnow - aimSince >= std::chrono::duration<double>(gazeHold)) {
aimHeld = aimHand = false;
hold.pressed = hold.grip = true;
@@ -2117,7 +2150,7 @@ int main() {
clickReleaseAt = tnow + std::chrono::milliseconds(40);
} else if (clickRelease && tnow >= clickReleaseAt) {
clickRelease = false;
releaseLeft();
if (leftHeld) releaseLeft(); // not after the quick check's correction: nothing was pressed
}
controllerButtons.Poll(