From 3c03f88f17daeb41107ba90acc73022268b1f371 Mon Sep 17 00:00:00 2001 From: baketnk Date: Thu, 24 Sep 2026 15:14:38 -0400 Subject: [PATCH] Open SteamVR binding editor directly from Bindings button --- README.md | 5 ++--- docs/design.md | 2 +- docs/overlay.md | 26 +++++++++++--------------- docs/poc.md | 4 ++-- src/overlay.cpp | 33 +-------------------------------- src/panel_surface.cpp | 36 ++++++++---------------------------- src/panel_surface.hpp | 4 ---- tests/panel_test.cpp | 29 ++++++++++++----------------- 8 files changed, 37 insertions(+), 102 deletions(-) diff --git a/README.md b/README.md index 8f8c241..f150ce7 100644 --- a/README.md +++ b/README.md @@ -28,9 +28,8 @@ See [third-party notes](docs/third-party.md). No GitHub release is published yet **Left grip (when active):** double-tap for the same explicit Enter action. Nothing inserts or submits automatically after transcription. - **Overlay:** Record/Stop, Cancel, paginated preview, Insert, Enter and Quit. - Review, settings and bindings share one Inconsolata/neon-framed surface. - **Bindings → Edit in SteamVR** opens the runtime binding editor; the tab shows - runtime-provided control names. Existing SteamVR overrides may supersede defaults. World-space by + Review and settings share one Inconsolata/neon-framed surface. + **Bindings** requests SteamVR's binding editor directly. Existing SteamVR overrides may supersede defaults. World-space by default; settings offer left wrist, right wrist and head mounting, plus recenter. Dashboard lasers provide clickable controls. Settings → Lasers anytime is an opt-in, default-off system-wide laser mode while the panel is visible; it may diff --git a/docs/design.md b/docs/design.md index 814c70b..258605e 100644 --- a/docs/design.md +++ b/docs/design.md @@ -62,7 +62,7 @@ Recording… 00:04 [ Cancel ] - Default Frame bindings: hold right X to speak, release to finish; B cancels, A inserts with a trailing space, Y inserts pending text then sends Enter. The alternate right-grip tap-then-hold gesture and left-grip double-tap Enter - remain available. All are remappable through the Bindings tab's SteamVR editor. + remain available. All are remappable through the Bindings button's SteamVR editor. A click-to-start/stop overlay button provides a binding-independent alternative. Bound recording to 20 seconds; discard accidental taps (initial threshold: 200 ms). diff --git a/docs/overlay.md b/docs/overlay.md index a1b7191..93f6101 100644 --- a/docs/overlay.md +++ b/docs/overlay.md @@ -19,7 +19,7 @@ Sans face if present. Glyph coverage depends on the selected face; full CJK coverage is not claimed. `src/panel_surface.*` renders **one 1000×680 RGBA canvas** for review, settings, -bindings, status and controls. `src/overlay_texture.*` uploads this CPU canvas into one +status and controls. `src/overlay_texture.*` uploads this CPU canvas into one persistent Vulkan RGBA8 image and submits it with `SetOverlayTexture`. The image, staging allocation and command buffer are reused; tabs do not create extra overlays or render targets. The rounded mint-to-blue perimeter, @@ -65,21 +65,17 @@ doubling an existing trailing space). Enter inserts any pending review and then queues Enter; with no pending text it queues Enter only. A failed text step never proceeds to Enter. Recording never automatically inserts or submits. -### Bindings menu +### Bindings button -The **Bindings** tab shows the runtime-provided hand/control names for PTT, -Cancel, Insert, Enter and both grip gestures. **Edit in SteamVR** requests the -in-headset binding editor for the current process/action set. SteamVR owns -remapping and persistence; the tab refreshes its origin labels once per second -while visible. Unknown/unavailable origins are reported instead of showing -bundled defaults as if they were live bindings. Long labels show a truncation -marker; the editor is the full binding view. Opening it clears pending pointer -presses and rearms controller gestures from neutral. +**Bindings** requests SteamVR's in-headset binding editor directly for the +current process/action set, without changing the FrameYap tab. SteamVR owns +remapping, persistence and the full binding view. The Review tab shows a request +or error note after the call. Opening it clears pending pointer presses and +rearms controller gestures from neutral. -OpenVR 2.15.6 provides `GetActionOrigins`, `GetOriginLocalizedName` and -`OpenBindingUI`, not a generic 2D button-glyph API. Frame's installed controller -profile references left/right SVG diagrams for SteamVR's own editor. FrameYap -uses runtime text labels; it does not copy runtime artwork into its package. +OpenVR 2.15.6 provides `OpenBindingUI`. Frame's installed controller profile +references left/right SVG diagrams for SteamVR's own editor. FrameYap +does not copy runtime artwork into its package. Editor availability and artwork rendering still require headset acceptance. Normal priority with Lasers anytime off is the practical baseline: the wearer @@ -247,7 +243,7 @@ ctest --test-dir build-ui --output-on-failure ./build-ui/frameyap_panel_test assets/fonts/Inconsolata-Regular.ttf /tmp/frameyap-ui ``` -The last command writes `-review.ppm`, `-settings.ppm`, `-bindings.ppm` and `-recording.ppm` to the +The last command writes `-review.ppm`, `-settings.ppm` and `-recording.ppm` to the supplied prefix. The native build includes these tests too; tests never initialize OpenVR or touch the real mounting preference. Physical pointing, tracking loss, recentring and readability still require a separately authorized headset check. diff --git a/docs/poc.md b/docs/poc.md index dab8645..b06d45d 100644 --- a/docs/poc.md +++ b/docs/poc.md @@ -12,8 +12,8 @@ initialize OpenVR, open a microphone, run ASR, download files or inject input. - Remappable SteamVR actions. The default Steam Frame binding maps right X (hold to record, release to transcribe) to the existing PTT action using the observed `frame_controller` profile. Right B cancels, A inserts + space, and Y - inserts pending text + Enter (or Enter only with no preview). A Bindings tab - shows runtime origin labels and opens SteamVR's remapping editor. Grip bindings remain, but both grip + inserts pending text + Enter (or Enter only with no preview). The Bindings button + requests SteamVR's remapping editor directly. Grip bindings remain, but both grip actions were inactive in the observed dashboard check; do not rely on them. If left grip becomes active, two short taps request explicit Enter. For grip gestures, first squeeze <=250 ms; second squeeze begins <=350 ms diff --git a/src/overlay.cpp b/src/overlay.cpp index 0cfd62e..cb34fc3 100644 --- a/src/overlay.cpp +++ b/src/overlay.cpp @@ -92,7 +92,6 @@ struct Overlay::Impl { unsigned overlay_shown_events = 0, overlay_hidden_events = 0, image_loaded_events = 0, image_failed_events = 0; unsigned overlay_focus_events = 0, global_focus_events = 0, input_focus_captured_events = 0; std::string last_pointer_event = "none"; - std::chrono::steady_clock::time_point next_binding_refresh{}; Impl(const std::string& assets, const std::string& font, std::optional requested, bool persist) : settings_path(default_mount_settings_path()), @@ -293,34 +292,6 @@ struct Overlay::Impl { } return {true, data.bState}; } - void refresh_bindings() { - if (!surface.bindings_visible()) { next_binding_refresh = {}; return; } - const auto now = std::chrono::steady_clock::now(); - if (now < next_binding_refresh) return; - next_binding_refresh = now + std::chrono::seconds(1); - std::array labels; - constexpr std::array order{{2, 3, 4, 5, 0, 1}}; - for (size_t row = 0; row < order.size(); ++row) { - std::array origins{}; - const auto error = input->GetActionOrigins(action_set, actions[order[row]], origins.data(), uint32_t(origins.size())); - if (error != vr::VRInputError_None) { labels[row] = "Binding information unavailable"; continue; } - for (auto origin : origins) { - if (origin == vr::k_ulInvalidInputValueHandle) continue; - std::array name{}; - if (input->GetOriginLocalizedName(origin, name.data(), uint32_t(name.size()), - vr::VRInputString_Hand | vr::VRInputString_InputSource) != vr::VRInputError_None || !name[0]) { - if (!labels[row].empty()) labels[row] += " / "; - labels[row] += "Bound (name unavailable)"; - } else { - name.back() = '\0'; - if (!labels[row].empty()) labels[row] += " / "; - labels[row] += name.data(); - } - } - if (labels[row].empty()) labels[row] = "Unbound / controller unavailable"; - } - surface.set_bindings(std::move(labels)); - } std::vector poll() { std::vector result; system->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, poses.data(), uint32_t(poses.size())); @@ -390,9 +361,8 @@ struct Overlay::Impl { reset_input(result); const auto error = input->OpenBindingUI(nullptr, action_set, vr::k_ulInvalidInputValueHandle, false); surface.set_binding_note(error == vr::VRInputError_None - ? "SteamVR editor requested. Changes appear here when available." + ? "SteamVR binding editor requested." : "SteamVR could not open bindings. Try its controller settings."); - next_binding_refresh = {}; } if (event_result.lasers_anytime) { const bool enabled = *event_result.lasers_anytime; @@ -427,7 +397,6 @@ struct Overlay::Impl { // experiment compares. SteamVR's separate permission gate is never changed here. set.nPriority = action_priority(); action_update_error = input->UpdateActionState(&set, sizeof(set), 1); - refresh_bindings(); if (action_update_error != vr::VRInputError_None) { reset_input(result); return result; } diff --git a/src/panel_surface.cpp b/src/panel_surface.cpp index 11fb9ca..41bd258 100644 --- a/src/panel_surface.cpp +++ b/src/panel_surface.cpp @@ -2,6 +2,7 @@ #include #include FT_FREETYPE_H #include +#include #include #include #include @@ -25,15 +26,14 @@ struct Rect { rounded_distance(px, py, x, y, w, h, std::min(16, h / 3)) <= 0.f; } }; -enum class Control { Review, Settings, Bindings, OpenBindings, Prev, Next, Record, Cancel, Insert, Enter, Quit, +enum class Control { Review, Settings, Bindings, Prev, Next, Record, Cancel, Insert, Enter, Quit, World, Left, Right, Head, Recenter, LasersAnytime, AdvancedDebug }; -enum class Tab { Review, Settings, Bindings }; +enum class Tab { Review, Settings }; struct Button { Rect r; Control id; const char* label; }; -constexpr std::array buttons{{ +constexpr std::array buttons{{ {{32, 138, 180, 46}, Control::Review, "Review"}, {{226, 138, 180, 46}, Control::Settings, "Settings"}, {{420, 138, 180, 46}, Control::Bindings, "Bindings"}, - {{32, 410, 420, 48}, Control::OpenBindings, "Edit in SteamVR"}, {{32, 406, 154, 44}, Control::Prev, "Previous"}, {{838, 406, 130, 44}, Control::Next, "Next"}, {{32, 574, 176, 68}, Control::Record, "Record"}, @@ -108,7 +108,6 @@ struct PanelSurface::Impl { Tab tab = Tab::Review; bool dirty = true, lasers_anytime = false, advanced_debug = false; std::string placement_note, binding_note; - std::array bindings{}; std::array pressed{{-1, -1}}; std::vector lines; size_t page = 0; @@ -247,7 +246,6 @@ struct PanelSurface::Impl { if (c == Control::Prev || c == Control::Next) return tab == Tab::Review; if (mounting(c) || c == Control::Recenter || c == Control::LasersAnytime || c == Control::AdvancedDebug) return tab == Tab::Settings; - if (c == Control::OpenBindings) return tab == Tab::Bindings; return true; } bool enabled(Control c) const { @@ -302,20 +300,7 @@ struct PanelSurface::Impl { for (size_t i = 0; i < std::min(size_t(2), detail.size()); ++i) text(detail[i], 32, 486 + int(i) * 30, 24, muted, 936); if (detail.size() > 2) text("[detail truncated]", 730, 546, 22, pink, 968); - } else if (tab == Tab::Bindings) { - constexpr std::array names{{"Hold to record", "Cancel / discard", "Insert + space", - "Insert + Enter", "Double tap: Enter", "Tap then hold: record"}}; - for (size_t i = 0; i < names.size(); ++i) { - const int y = 229 + int(i) * 30; - text(names[i], 32, y, 23, muted, 335); - auto label = wrap(bindings[i].empty() ? "Waiting for SteamVR..." : bindings[i], 23, 582); - text(label.front(), 346, y, 23, ink, 928); - if (label.size() > 1) text("...", 930, y, 23, muted, 968); - } - text("Enter inserts pending text, then presses Enter; no text = Enter only.", 32, 484, 22, muted, 968); - text("Normal mode: bindings with dashboard closed; UI with it open.", 32, 512, 22, muted, 968); - text(binding_note.empty() ? "SteamVR owns remapping. Multiple controls may appear per action." : binding_note, - 32, 539, 21, muted, 968); + if (!binding_note.empty()) text(binding_note, 32, 203, 20, muted, 968); } else { text("MOUNT AND INTERACTION", 32, 224, 22, muted, 968); text("Full logs may contain speech/text/paths. No saved audio clips.", 32, 520, 20, pink, 968); @@ -329,7 +314,6 @@ struct PanelSurface::Impl { bool on = enabled(b.id); bool selected = (b.id == Control::Review && tab == Tab::Review) || (b.id == Control::Settings && tab == Tab::Settings) || - (b.id == Control::Bindings && tab == Tab::Bindings) || (mounting(b.id) && *mounting(b.id) == mount) || (b.id == Control::LasersAnytime && lasers_anytime) || (b.id == Control::AdvancedDebug && advanced_debug); @@ -380,19 +364,15 @@ SurfaceEvent PanelSurface::pointer_up(unsigned cursor, float x, float y) { else if (c == Control::Recenter) { result.recenter = true; impl_->reset(); } else if (c == Control::LasersAnytime) result.lasers_anytime = !impl_->lasers_anytime; else if (c == Control::AdvancedDebug) result.advanced_debug = !impl_->advanced_debug; - else if (c == Control::OpenBindings) { result.open_bindings = true; impl_->reset(); } - else if (c == Control::Review || c == Control::Settings || c == Control::Bindings) { - impl_->tab = c == Control::Review ? Tab::Review : c == Control::Settings ? Tab::Settings : Tab::Bindings; + else if (c == Control::Bindings) { result.open_bindings = true; impl_->reset(); } + else if (c == Control::Review || c == Control::Settings) { + impl_->tab = c == Control::Review ? Tab::Review : Tab::Settings; impl_->reset(); impl_->dirty = true; } else if (c == Control::Prev) { --impl_->page; impl_->dirty = true; } else if (c == Control::Next) { ++impl_->page; impl_->dirty = true; } return result; } -bool PanelSurface::bindings_visible() const { return impl_->tab == Tab::Bindings; } -void PanelSurface::set_bindings(std::array labels) { - if (impl_->bindings != labels) { impl_->bindings = std::move(labels); impl_->dirty = true; } -} void PanelSurface::set_binding_note(std::string note) { if (impl_->binding_note != note) { impl_->binding_note = std::move(note); impl_->dirty = true; } } diff --git a/src/panel_surface.hpp b/src/panel_surface.hpp index f3025f2..9e5a6ba 100644 --- a/src/panel_surface.hpp +++ b/src/panel_surface.hpp @@ -1,7 +1,6 @@ #pragma once #include "overlay.hpp" #include "config.hpp" -#include #include #include #include @@ -34,10 +33,7 @@ public: void set_placement_note(std::string note); void set_lasers_anytime(bool enabled); void set_advanced_debug(bool enabled); - // PTT, Cancel, Insert, Enter, left-grip gesture, right-grip gesture. - void set_bindings(std::array labels); void set_binding_note(std::string note); - bool bindings_visible() const; bool available(UiAction action) const; private: struct Impl; diff --git a/tests/panel_test.cpp b/tests/panel_test.cpp index 4b1c939..caeb066 100644 --- a/tests/panel_test.cpp +++ b/tests/panel_test.cpp @@ -139,31 +139,26 @@ int main(int argc, char** argv) { no_action(click(surface, 680, 420)); // laser toggle only exists on Settings no_action(click(surface, 680, 474)); // debug toggle only exists on Settings - // Binding navigation is not a delivery action; settings and paging are hidden. - assert(!surface.bindings_visible()); + // Bindings opens SteamVR directly, from either tab, without replacing review. surface.pointer_down(1, 480, 610); - no_action(click(surface, 500, 160)); - assert(surface.bindings_visible()); - no_action(surface.pointer_up(1, 480, 610)); // switching tabs invalidates old approval - surface.set_bindings({"Right X", "Unbound / controller unavailable", "Left Y", "Right Y", "Left grip", "Right grip"}); - assert(surface.render(p)); assert(!surface.render(p)); - surface.set_bindings({"Right X", "Unbound / controller unavailable", "Left Y", "Right Y", "Left grip", "Right grip"}); - assert(!surface.render(p)); - if (argc >= 3) snapshot(surface, std::string(argv[2]) + "-bindings.ppm"); - no_action(click(surface, 680, 260)); - no_action(click(surface, 900, 430)); - no_action(click(surface, 680, 420)); - no_action(click(surface, 680, 474)); - const auto editor = click(surface, 200, 434); + auto editor = click(surface, 500, 160); assert(editor.open_bindings && !editor.action && !editor.mount && !editor.recenter && !editor.lasers_anytime); - no_action(surface.pointer_up(0, 200, 434)); // one launch per deliberate click + no_action(surface.pointer_up(1, 480, 610)); // opening editor invalidates old approval + no_action(surface.pointer_up(0, 500, 160)); // one launch per deliberate click surface.set_binding_note("SteamVR could not open bindings. Try its controller settings."); assert(surface.render(p)); assert(!surface.render(p)); assert(click(surface, 480, 610).action == UiAction::Insert); assert(click(surface, 680, 610).action == UiAction::Enter); assert(click(surface, 280, 610).action == UiAction::Cancel); no_action(click(surface, 100, 160)); - assert(!surface.bindings_visible()); + editor = click(surface, 500, 160); + assert(editor.open_bindings && !editor.action); + no_action(click(surface, 200, 434)); // old secondary editor button is gone + no_action(click(surface, 290, 160)); // switch to Settings + editor = click(surface, 500, 160); + assert(editor.open_bindings && !editor.action && !editor.mount); + assert(click(surface, 680, 260).mount == Mount::Head); // still in Settings + no_action(click(surface, 100, 160)); // Long UTF-8, newlines and malformed bytes are bounded, paginated and navigable. p.transcript.clear();