steam-frame/DeeJanuz--frametop
1
0 Fork 0
mirror of https://github.com/DeeJanuz/frametop.git synced 2026-10-06 04:04:09 +02:00
Code Issues Packages Projects Releases 3 Wiki Activity

Gaze first: test results; treadmill from the start, muted compositor binding

Browse Source 
From the first headset tests (docs/gaze-first.md, "Test results"):
- SteamVR gives the treadmill path only to a device that hints it when
  it's added, so the driver hints a role that's no hand from Activate
  (a hand still only while shown). With it, our device held the
  dashboard laser with no hand role. The helper doesn't release the
  pointer for having no hand role when POINTER_ROLE is treadmill.
- The helper's global action sets can't mute the compositor (SteamVR
  reports them inactive while the laser mouse has focus), but selecting
  pointer/bindings/vrcompositor_frame_controller_gazefirst.json (the
  stock binding without its trigger and bumper laser entries) through
  vrserver's /input/selectconfig.action does.
- input/vrws.py follows devices that connect later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DeeJanuzandClaude Opus 5.5 committed 2026-09-30 17:23:30 -06:00
1 parent bd06207db9
commit 87a63932c0
5 files changed
+506 -13
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/gaze-first.md
+13
View File
@@ -60,6 +60,19 @@ Calibration, in one head-locked panel:
- A Frame controller in the hand takes its hand's role back through its touch sensors, so a device that needs a hand role loses it whenever both controllers are held. That's why the laser has to live somewhere else.
- vrserver's web socket (`/input/getstate.json` and `request_input_state_updates`, as frame-voice uses) lists each controller's `/input/trigger/click`, `/input/bumper/click`, `/input/grip/click`, `/input/thumbstick/click`, `/input/thumbstick/x` and `y`, `/input/system/click`, and each device's `side`. The headset has `/proximity`, but it flickers off for 0.3 to 0.5 s at a time while worn, so the quick check follows SteamVR's activity level (as the helper already does) rather than the raw sensor. Reading the socket takes nothing from anyone. A controller's path is `/devices/cv/<serial>` instead of `/user/hand/<side>` while our device holds that hand.
## Test results (2026-09-30)
Run with the headset on its stand and the controllers on (`pointer/probe/lasertest`, `input/vrws.py`):
1. **Laser on the treadmill role: works.** Our device held the dashboard laser with no hand role. SteamVR gives a device the `/user/treadmill` path only if it hints treadmill when it's added, so the driver hints a role that's no hand from `Activate`; a hint changed on connecting kept it at `/devices/ft_pointer/ft_pointer_0`. `GetControllerRoleForTrackedDeviceIndex` reports no role for it, so the helper's "no hand role" release skips treadmill.
2. **Muting through the helper's action sets: doesn't work.** SteamVR reported those actions inactive (`vrstatus`: `"active": []`) while its laser mouse had input focus, as frame-voice found on 2026-09-26, and a trigger pull moved the laser to its controller until the release. **Muting through the compositor binding works:** `pointer/bindings/vrcompositor_frame_controller_gazefirst.json` is the stock binding without its trigger and bumper laser entries, selected with `POST /input/selectconfig.action` on vrserver's port 27062 (a JSON body `{"app_key": "openvr.component.vrcompositor", "controller_type": "frame_controller", "url": "file:///..."}`; a form-encoded one gets "Parse failed"). `GET /input/getactions.json?app_key=openvr.component.vrcompositor` shows the choice (`current_binding_url`). With it, the triggers showed no laser. Selecting the stock file puts it back.
3. **Snap back: works** in 11 to 15 ms after a grip or a Steam button summon. A trigger held the laser until its release (0.4 to 1 s), which the muting removes. Our device also takes the laser from "none".
4. **Web socket: works.** Every click, the thumbstick axes, and both thumbsticks clicked together arrive. The headset's `/proximity` flickers off for 0.3 to 0.5 s at a time while worn, so the quick check goes by SteamVR's activity level instead.
Also found: the bumpers, the thumbsticks' movement, and their clicks switch Steam's dashboard into its controller (gamepad) mode, and the laser owner goes to "none"; both grips go back to laser mode. Steam's own Frame controller binding (`steam_vrgamepad_bindings_frame_controller.json`, app `steam.client`) has only haptics, so that input reaches Steam's UI some other way (most likely Steam Input's virtual gamepad), and a SteamVR binding can't mute it.
Decided after the tests: gaze replaces the laser pointer's controls, never the controller mode's. Controller mode takes precedence while it's on, and leaving it gives the laser back to the gaze. Right click is one grip held with a trigger (the bumpers belong to controller mode).
## Tests before building
One headset session, about 30 minutes, with the user wearing the headset. Installing the test driver needs a SteamVR restart, which closes everything in VR, so it's done at the start of the session and only when the user says so.
input/vrws.py
+17 -6
View File
@@ -157,19 +157,30 @@ def main():
return (name.endswith("/click") or name in ("/input/thumbstick/x", "/input/thumbstick/y",
"/input/trigger/value", "/proximity"))
devices = getstate()
names = {}
for d in devices:
names[d["root_path"]] = f"{d['root_path']} ({d.get('controller_type', '?')}{', ' + d['side'] if d.get('side') else ''})"
print("device", names[d["root_path"]])
ws = VrSocket()
ws.open(f"frametop_vrws_{os.getpid()}")
for d in devices:
ws.subscribe(d["root_path"])
def follow_new():
"""Subscribe to devices that appeared (vrserver only streams changes, and a device
that connects later, or changes its path with a role, is a new subscription)."""
for d in getstate():
path = d["root_path"]
name = f"{path} ({d.get('controller_type', '?')}{', ' + d['side'] if d.get('side') else ''})"
if names.get(path) != name:
names[path] = name
print(f"{time.monotonic() - start:9.3f} device {name}", flush=True)
ws.subscribe(path)
start = time.monotonic()
follow_new()
next_poll = start + 2
last, counts = {}, {}
try:
while not seconds or time.monotonic() - start < seconds:
if time.monotonic() >= next_poll:
next_poll = time.monotonic() + 2
follow_new()
msg = ws.recv(timeout=0.5)
if not isinstance(msg, dict) or msg.get("type") != "update_component_states":
continue
pointer/bindings/vrcompositor_frame_controller_gazefirst.json
+463
View File
@@ -0,0 +1,463 @@
{
"action_manifest_version": 0,
"alias_info": {},
"app_key": "openvr.component.vrcompositor",
"bindings": {
"/actions/dualanalog": {
"sources": [
{
"inputs": {
"click": {
"output": "/actions/dualanalog/in/modeswitch1"
}
},
"mode": "button",
"path": "/user/hand/left/input/grip"
},
{
"inputs": {
"click": {
"output": "/actions/dualanalog/in/modeswitch2"
}
},
"mode": "button",
"path": "/user/hand/right/input/grip"
},
{
"inputs": {
"click": {
"output": "/actions/dualanalog/in/leftclick"
},
"position": {
"output": "/actions/dualanalog/in/leftvalue"
}
},
"mode": "joystick",
"path": "/user/hand/left/input/thumbstick"
},
{
"inputs": {
"click": {
"output": "/actions/dualanalog/in/rightclick"
},
"position": {
"output": "/actions/dualanalog/in/rightvalue"
}
},
"mode": "joystick",
"path": "/user/hand/right/input/thumbstick"
},
{
"inputs": {
"click": {
"output": "/actions/dualanalog/in/leftclick"
}
},
"mode": "button",
"path": "/user/hand/left/input/trigger"
},
{
"inputs": {
"click": {
"output": "/actions/dualanalog/in/rightclick"
}
},
"mode": "button",
"path": "/user/hand/right/input/trigger"
}
]
},
"/actions/lasermouse": {
"haptics": [
{
"output": "/actions/lasermouse/out/haptic",
"path": "/user/hand/left/output/haptic"
},
{
"output": "/actions/lasermouse/out/haptic",
"path": "/user/hand/right/output/haptic"
}
],
"poses": [
{
"output": "/actions/lasermouse/in/Pointer",
"path": "/user/hand/left/pose/tip"
},
{
"output": "/actions/lasermouse/in/Pointer",
"path": "/user/hand/right/pose/tip"
}
],
"sources": [
{
"inputs": {
"click": {
"output": "/actions/lasermouse/in/rightclick"
}
},
"mode": "joystick",
"path": "/user/hand/left/input/thumbstick"
},
{
"inputs": {
"click": {
"output": "/actions/lasermouse/in/middleclick"
}
},
"mode": "button",
"path": "/user/hand/left/input/dpad_right"
},
{
"inputs": {
"click": {
"output": "/actions/lasermouse/in/back"
}
},
"mode": "button",
"path": "/user/hand/left/input/dpad_left"
},
{
"inputs": {
"click": {
"output": "/actions/lasermouse/in/home"
}
},
"mode": "button",
"path": "/user/hand/left/input/dpad_down"
},
{
"inputs": {
"click": {
"output": "/actions/lasermouse/in/rightclick"
}
},
"mode": "joystick",
"path": "/user/hand/right/input/thumbstick"
},
{
"inputs": {
"click": {
"output": "/actions/lasermouse/in/middleclick"
}
},
"mode": "button",
"path": "/user/hand/right/input/x"
},
{
"inputs": {
"click": {
"output": "/actions/lasermouse/in/back"
}
},
"mode": "button",
"path": "/user/hand/right/input/b"
},
{
"inputs": {
"click": {
"output": "/actions/lasermouse/in/home"
}
},
"mode": "button",
"path": "/user/hand/right/input/a"
}
]
},
"/actions/lasermouse_secondary": {
"sources": [
{
"inputs": {
"click": {
"output": "/actions/lasermouse_secondary/in/switchlaserhand"
}
},
"mode": "trigger",
"path": "/user/hand/left/input/grip"
},
{
"inputs": {
"click": {
"output": "/actions/lasermouse_secondary/in/switchlaserhand"
}
},
"mode": "trigger",
"path": "/user/hand/right/input/grip"
}
]
},
"/actions/scroll_discrete": {
"sources": [
{
"inputs": {
"scroll": {
"output": "/actions/scroll_discrete/in/scroll"
}
},
"mode": "scroll",
"path": "/user/hand/left/input/thumbstick"
},
{
"inputs": {
"scroll": {
"output": "/actions/scroll_discrete/in/scroll"
}
},
"mode": "scroll",
"path": "/user/hand/right/input/thumbstick"
}
]
},
"/actions/scroll_smooth": {
"sources": [
{
"inputs": {
"scroll": {
"output": "/actions/scroll_smooth/in/scroll"
}
},
"mode": "scroll",
"parameters": {
"scroll_mode": "smooth"
},
"path": "/user/hand/left/input/thumbstick"
},
{
"inputs": {
"scroll": {
"output": "/actions/scroll_smooth/in/scroll"
}
},
"mode": "scroll",
"parameters": {
"scroll_mode": "smooth"
},
"path": "/user/hand/right/input/thumbstick"
}
]
},
"/actions/system": {
"chords": [
{
"inputs": [
[
"/user/hand/left/input/system",
"held"
],
[
"/user/hand/left/input/trigger",
"click"
]
],
"output": "/actions/system/in/TakeScreenshot"
},
{
"inputs": [
[
"/user/hand/right/input/system",
"held"
],
[
"/user/hand/right/input/trigger",
"click"
]
],
"output": "/actions/system/in/TakeScreenshot"
},
{
"inputs": [
[
"/user/hand/left/input/system",
"held"
],
[
"/user/hand/right/input/trigger",
"click"
]
],
"output": "/actions/system/in/TakeScreenshot"
},
{
"inputs": [
[
"/user/hand/right/input/system",
"held"
],
[
"/user/hand/left/input/trigger",
"click"
]
],
"output": "/actions/system/in/TakeScreenshot"
}
],
"sources": [
{
"inputs": {},
"mode": "button",
"parameters": {
"haptic_amplitude": "0"
},
"path": "/user/hand/left/input/trigger"
},
{
"inputs": {},
"mode": "button",
"parameters": {
"haptic_amplitude": "0"
},
"path": "/user/hand/right/input/trigger"
},
{
"inputs": {
"click": {
"output": "/actions/system/in/ToggleDashboard"
},
"double": {
"output": "/actions/system/in/ToggleRoomView"
},
"held": {
"output": "/actions/system/in/SystemButtonChord"
}
},
"mode": "button",
"path": "/user/hand/left/input/system"
},
{
"inputs": {
"click": {
"output": "/actions/system/in/ToggleDashboard"
},
"double": {
"output": "/actions/system/in/ToggleRoomView"
},
"held": {
"output": "/actions/system/in/SystemButtonChord"
}
},
"mode": "button",
"path": "/user/hand/right/input/system"
}
]
},
"/actions/quickmouse": {
"sources": [
{
"inputs": {
"click": {
"output": "/actions/quickmouse/in/ActivateQuickMouse"
}
},
"mode": "trigger",
"path": "/user/hand/left/input/grip"
},
{
"inputs": {
"click": {
"output": "/actions/quickmouse/in/ActivateQuickMouse"
}
},
"mode": "trigger",
"path": "/user/hand/right/input/grip"
}
]
},
"/actions/locomotion": {
"sources": [
{
"inputs": {
"position": {
"output": "/actions/locomotion/in/movement"
}
},
"mode": "joystick",
"path": "/user/hand/left/input/thumbstick"
},
{
"inputs": {
"position": {
"output": "/actions/locomotion/in/rotation"
}
},
"mode": "joystick",
"path": "/user/hand/right/input/thumbstick"
}
]
},
"/actions/quickrecenter": {
"sources": [
{
"inputs": {
"long": {
"output": "/actions/quickrecenter/in/Recenter"
}
},
"parameters": {
"long_press_delay": 1.0,
"long_press_expiry": 4.0
},
"mode": "button",
"path": "/user/hand/left/input/system"
},
{
"inputs": {
"long": {
"output": "/actions/quickrecenter/in/Recenter"
}
},
"parameters": {
"long_press_delay": 1.0,
"long_press_expiry": 4.0
},
"mode": "button",
"path": "/user/hand/right/input/system"
}
]
},
"/actions/roomsetup": {
"sources": [
{
"inputs": {
"position": {
"output": "/actions/roomsetup/in/FloorFineAdjust"
}
},
"mode": "joystick",
"path": "/user/hand/left/input/thumbstick"
},
{
"inputs": {
"position": {
"output": "/actions/roomsetup/in/FloorFineAdjust"
}
},
"mode": "joystick",
"path": "/user/hand/right/input/thumbstick"
},
{
"inputs": {
"click": {
"output": "/actions/roomsetup/in/Accept"
}
},
"mode": "button",
"path": "/user/hand/right/input/a"
},
{
"inputs": {
"click": {
"output": "/actions/roomsetup/in/Back"
}
},
"mode": "button",
"path": "/user/hand/right/input/b"
}
]
}
},
"category": "steamvr_input",
"controller_type": "frame_controller",
"description": "The Frame controller stock compositor binding without its trigger and bumper laser entries (docs/gaze-first.md)",
"name": "Frametop gaze first: no trigger or bumper laser",
"options": {},
"simulated_actions": []
}
pointer/driver/driver_ft_pointer.cpp
+10 -5
View File
@@ -117,7 +117,9 @@ public:
props->SetStringProperty(c, Prop_ControllerType_String, "ft_pointer");
props->SetStringProperty(c, Prop_InputProfilePath_String, "{ft_pointer}/input/ft_pointer_profile.json");
props->SetStringProperty(c, Prop_RenderModelName_String, "{ft_pointer}/rendermodels/ft_pointer_invisible");
props->SetInt32Property(c, Prop_ControllerRoleHint_Int32, TrackedControllerRole_OptOut); // until "show"
// A hand only while shown (see the top); a role that's no hand from the start, since
// SteamVR gives a treadmill its path only when the device is added (docs/gaze-first.md).
props->SetInt32Property(c, Prop_ControllerRoleHint_Int32, HintFor(false));
props->SetInt32Property(c, Prop_DeviceClass_Int32, TrackedDeviceClass_Controller);
props->SetBoolProperty(c, Prop_NeverTracked_Bool, false);
@@ -213,12 +215,11 @@ public:
}
if (snapshot.role > 0 && snapshot.role != role_) {
role_ = snapshot.role;
if (hinted_) VRProperties()->SetInt32Property(container_, Prop_ControllerRoleHint_Int32, role_);
VRProperties()->SetInt32Property(container_, Prop_ControllerRoleHint_Int32, HintFor(hinted_));
}
if (snapshot.visible != hinted_) {
// Claim the hand before connecting; give it up when disconnecting.
VRProperties()->SetInt32Property(container_, Prop_ControllerRoleHint_Int32,
snapshot.visible ? role_ : int32_t(TrackedControllerRole_OptOut));
VRProperties()->SetInt32Property(container_, Prop_ControllerRoleHint_Int32, HintFor(snapshot.visible));
hinted_ = snapshot.visible;
}
const bool ok = hmd.bPoseIsValid && snapshot.visible;
@@ -239,7 +240,11 @@ private:
uint32_t objectId_ = k_unTrackedDeviceIndexInvalid;
PropertyContainerHandle_t container_ = k_ulInvalidPropertyContainer;
int32_t role_ = TrackedControllerRole_RightHand;
bool hinted_ = false; // whether the role hint currently claims role_
bool hinted_ = false; // whether we're shown (a hand role is hinted only then)
int32_t HintFor(bool shown) const {
const bool hand = role_ == TrackedControllerRole_LeftHand || role_ == TrackedControllerRole_RightHand;
return shown || !hand ? role_ : int32_t(TrackedControllerRole_OptOut);
}
DriverPose_t pose_{};
VRInputComponentHandle_t buttons_[kButtons] = {};
VRInputComponentHandle_t scrollX_ = 0, scrollY_ = 0;
pointer/helper/ft-pointer.cpp
+3 -2
View File
@@ -1353,8 +1353,9 @@ int main() {
if (debug) std::fflush(stdout);
}
// Didn't get the hand role (a held controller keeps it): release, back off.
if (active && tnow - wokeAt > std::chrono::seconds(1) && ours != vr::k_unTrackedDeviceIndexInvalid &&
// Didn't get the hand role (a held controller keeps it): release, back off. Treadmill is
// no hand, and SteamVR reports no role for it (it's /user/treadmill): nothing to lose.
if (active && role != "treadmill" && tnow - wokeAt > std::chrono::seconds(1) && ours != vr::k_unTrackedDeviceIndexInvalid &&
sys->GetControllerRoleForTrackedDeviceIndex(ours) == vr::TrackedControllerRole_Invalid) {
active = false;
claimPending = claimHeld = false;