baketnk--frame-yap/docs/overlay.md

Native OpenVR panel

src/overlay.hpp provides RAII OpenVR ownership and registration(). The panel only emits UI actions; src/runtime.cpp owns audio, transcription and insertion. Native --run wiring in src/main.cpp is behind the explicit FRAMEYAP_NATIVE build option. The Models chooser, offline verification and explicit installer handoff are implemented locally, not an accepted Frame install/voice-typing path or a published release. Neither hardware-free tests nor a successful compile establish Frame input, visibility, comfort or text delivery. Launching the runtime is explicit, never part of a normal build or test.

Rendering and controls

Explicit development dependencies: Valve OpenVR SDK v2.15.6, Vulkan headers/loader and FreeType 2. The native runtime needs a compatible system Vulkan driver. Configure/build must not fetch them. The default font is the bundled Inconsolata Regular; see third-party notes. --font FILE overrides the JSON selection. A missing selected font falls back to bundled Inconsolata, then a system DejaVu Sans face if present. Glyph coverage depends on the selected face; full CJK coverage is not claimed.

src/panel_surface.* renders one 1080×780 RGBA canvas containing a 1000×680 main panel for review, settings, status and controls, plus transparent right/bottom margins for a thin grab underline and an external L-shaped scale handle. src/overlay_texture.* uploads this CPU canvas into one persistent Vulkan RGBA8 image and submits it with SetOverlayTexture. The image, staging allocation and command buffer are reused; tabs do not create extra overlays or render targets. The rounded mint-to-blue perimeter, shallow curved accent, and dark cards form the panel's visual language. With gradient.enabled (opt-in), a low-strength animated field blends theme.frame_start and theme.frame_end into theme.background; the perimeter and grab/scale handles share its phase and canvas coordinates. Rounded preview, status and control surfaces use independently rasterized antialiased edges and restrained baked neon halos rather than GPU bloom. The recording indicator and selected controls remain distinguishable by their labels, not color alone. Rounded control hit areas exclude their clipped corners. Rendering/uploads occur for changed content, scroll position or settings, and, when the animated gradient is enabled and visible, at most every 100 ms (10 fps) on a monotonic clock. There is no animation redraw while hidden. Laser hover and button down/up are hit-tested without an upload; static frames are reused when animation is disabled. The caller may call draw(Panel) at 10 ms intervals. Tracking transforms do not require repainting the canvas.

The Vulkan instance/device enable the extensions requested by the running SteamVR runtime and use its selected physical device and a graphics queue. There is no desktop window, swapchain, SDL video dependency or raw-upload fallback. Updates wait for the dedicated queue's previous upload and OpenVR transfer before reusing staging memory. Image barriers finish in TRANSFER_SRC_OPTIMAL, as required by OpenVR's Vulkan contract. The queue is used on the overlay thread; GPU resources outlive VR_Shutdown. The device selection, texture description and persistent panel-upload patterns are FrameYap's own implementation. GPU setup/submission errors stop startup or the run with an explicit error. This replaces the raw-upload rendering path; headset flicker acceptance still requires an on-device comparison. Native installation and offscreen GPU checks are separate from headset acceptance.

The header shows local time and date (right-aligned) instead of the former on-device/review and current-mount labels. Between the title and the clock it shows battery levels for the left controller, headset (HMD) and right controller. Controllers and the headset use OpenVR's Prop_DeviceBatteryPercentage_Float (with Prop_DeviceIsCharging_Bool) when the device reports Prop_DeviceProvidesBatteryStatus_Bool; if the headset reports none, the local Linux /sys/class/power_supply system battery is read instead (peripheral scope=Device supplies are skipped). Levels refresh every five seconds; a device without a reading is hidden, never shown as zero. 20% or less is drawn in the warning color, charging in the accent color with +. The status line carries a Buttons ready / Buttons paused chip from IVROverlay::IsDashboardVisible(): while the SteamVR dashboard is open, controller bindings do not reach FrameYap, though pointer clicks still work. The chip reflects dashboard visibility only, not every system or game input override. --check-controls prints the same battery readings. Which devices Frame actually reports still needs a headset check. Settings explains Hold Quit (hold 0.9 seconds then release) and Lasers anytime (system-wide lasers may affect games). The header updates when the displayed minute, date, battery or dashboard state changes, not every frame. Settings toggles 12/24-hour time and cycles date Off → MM/DD/YYYY → DD/MM/YYYY → YYYY-MM-DD → Off. These only affect display; mount choices remain in Settings. The Settings controls form one vertically scrollable list: hover the laser over the list and use the right stick to scroll. The tabs, status and action footer remain fixed. Scrolling cancels a pending pointer press rather than activating a different control. There are no Settings pages. OpenVR's discrete/smooth laser scroll events are requested; right-stick behavior with SteamVR's keyboard open still needs a live Frame check.

Settings → About replaces the settings area with static information: the build version (plus git commit when built from an untagged or modified tree), source URL, MIT license, the selected model's name and license from its manifest, the bundled font's license, and a pointer to docs/third-party.md. It performs no checks and no network access; Settings returns. Recording controls stay live.

At the bottom of Settings, Check for updates performs an explicit, asynchronous request to GitHub's latest published release metadata. It never runs at launch, while idle or as part of offline tests. The bounded, timeout-limited helper compares the release's numeric tag with this build's version; repository commits and prereleases are not update candidates. No archive or installer is downloaded by a check. If a newer release is found, Install update... opens Konsole with the installed handoff script. The terminal waits for the wearer to close FrameYap and press Enter before the existing installer downloads a pinned version and checksum and installs it. Until that confirmation, the terminal does not fetch anything. An unavailable check or terminal reports an error in Settings; no fallback auto-install is attempted.

The transcript wraps by glyph width and scrolls in its review viewport with the right-stick laser wheel; a new transcript resets the scroll position. The former paging row holds Open Plan, Open Keyboard and Open Draw when the installed tnkplan, tnkboard or tnkdraw wrapper is found at startup in ~/.local/bin or an absolute PATH directory. Missing apps have no button; available buttons share the row. Open Keyboard short press runs the installed wrapper with fixed --show; holding it for 800 ms runs fixed --recenter once and suppresses the short action, placing the keyboard in front of the wearer. Leaving the button, losing focus/tracking or hiding the overlay cancels a pending hold. Other companion buttons launch their installed wrappers without a shell (their normal second-launch behavior may toggle the existing panel). Restart FrameYap after installing a companion so its button is discovered. Status fits on the single status line; the old bottom detail label is gone. The footer remains available on all tabs: Record (labelled Stop while recording), Cancel, Type (labelled Enter when nothing is pending review), Type + Enter, Hold Quit. Hold Quit needs a 900 ms press and release on that same button; its thin progress bar shows the hold. Record can retry after an error; it is disabled while warming/transcribing and until an existing review is typed or discarded. Cancel can stop worker startup. Type and Type + Enter are disabled during recording and transcription. A pointer action requires a press/release on the same enabled control from the same cursor; focus loss, tab changes, action-state changes and relocation clear pending presses. Type + Enter is always a separate deliberate action, not inferred from text. Type normally appends a trailing space (without doubling an existing one); a full 4096-byte transcript without room for that suffix is queued unchanged, with no extra error for the missing space. With no pending review, Type (button or A) queues Enter alone, the same as Type + Enter; a quick second A after Type therefore submits. Type + Enter types any pending review and then queues Enter; with no pending text it queues Enter only. Y opens the Quick phrases list over the review area; each further Y press cycles its highlighted choice. Cancel closes the picker without discarding an existing review. Type + Enter sends the selected phrase without a trailing space, then Enter. The choices are short single-line literals, not speech commands. A failed text step never proceeds to Enter. Recording never automatically submits. Auto insert, when explicitly enabled, can queue text (normally + space) after transcription only under the stable Xwayland focus guard described below.

Bindings button

Bindings requests SteamVR's in-headset binding editor directly for the current process/action set, without changing the FrameYap tab. SteamVR owns remapping, persistence and the full binding view. The Review tab shows a request or error note after the call. Opening it clears pending pointer presses and rearms controller gestures from neutral.

OpenVR 2.15.6 provides OpenBindingUI. Frame's installed controller profile references left/right SVG diagrams for SteamVR's own editor. FrameYap does not copy runtime artwork into its package. Editor availability and artwork rendering still require headset acceptance.

Normal priority with Lasers anytime off is the practical baseline: the wearer reports controller actions with the dashboard closed and clickable UI with it open. The menu does not enable global overrides or promise simultaneous access.

User theme and controller configuration

Optional $XDG_CONFIG_HOME/frameyap/config.json (fallback ~/.config/frameyap/config.json, only with an absolute HOME) is read at native overlay startup. The native binary does not create a config by itself; the installer creates one with defaults on first install. Copy the shipped assets/config.example.json to that path for manual installs. Example:

{
  "font": "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
  "input_priority": "normal",
  "advanced_debug": false,
  "auto_insert": false,
  "close_mic_when_idle": false,
  "backend": "redux",
  "lock_layout": false,
  "wrist_world_fallback": true,
  "clock_24h": false,
  "date_format": "mdy",
  "quick_inputs": ["/new", "/questions", "/help"],
  "wrist": {"x": 0, "y": 0.18, "z": 0.089, "width": 0.30, "roll_degrees": 0},
  "theme": {
    "background": "#0c101b", "card": "#141c2b", "ink": "#e6f0f9",
    "muted": "#97adc1", "accent": "#1ff0a4", "warning": "#ff6e87",
    "frame_start": "#1fff91", "frame_end": "#1f70ff"
  },
  "gradient": {"enabled": false, "period_seconds": 30, "strength": 0.12},
  "buttons": {
    "ptt": "/user/hand/right/input/x",
    "cancel": "/user/hand/right/input/b",
    "insert": "/user/hand/right/input/a",
    "enter": "",
    "quick_chat": "/user/hand/right/input/y",
    "left_grip": "/user/hand/left/input/grip",
    "right_grip": "/user/hand/right/input/grip"
  }
}

clock_24h is a boolean (default false); date_format is off, mdy (default), dmy, or iso. The Settings buttons update these preferences immediately and save them to the user config when persistence is enabled. A failed save warns and leaves the selection active for this run. Time uses the device's local timezone; these controls do not change system time.

Each theme color is #RRGGBB; omitted colors keep the default. The optional gradient object defaults to {"enabled": false, "period_seconds": 30, "strength": 0.12}. enabled must be a boolean; period_seconds must be a finite number from 5 to 300 (seconds per full cycle), and strength a finite number from 0 to 0.3. The animation uses a smooth periodic cosine field with one start/end/start cycle across the canvas width. Its edge and handle colors use the same global canvas coordinates and time phase; it does not add a separate handle animation. Colors always come from theme.frame_start and theme.frame_end, while strength controls how much of those colors blends into theme.background. The default static mode keeps the solid background and existing linear perimeter/handle gradients. Settings → Animated background turns animation on/off immediately and saves only gradient.enabled in the JSON config (failed saves leave a session-only choice and a warning). Manual JSON edits take effect on restart; existing enabled configs stay enabled on upgrade. This is locally implemented, not device validated; headset appearance and rendering cost remain unverified. font is a TTF/OTF file path (not a family name); a missing file uses the bundled font. advanced_debug is a boolean (default false, not a string): an opt-in request for full diagnostic logs. Full logs may contain speech/transcribed text and local paths; raw audio clips are not archived. The Settings tab shows an Advanced debugging ON/OFF toggle and warns that changing it restarts the worker and cancels current work (including pending review). Changes take effect immediately; failed saves show a warning and keep the selection for this session. Detailed logs are bounded and owner-private; see worker diagnostics. The native save_advanced_debug(path, bool) helper updates only this value in a valid config, retaining other fields and formatting; invalid/unwritable configs are left untouched and return failure. The installer backs up original bytes before repairing invalid values, while valid true and false are retained.

close_mic_when_idle is a separate boolean, default false, also available as Settings → Close mic when idle. Normally the SDL capture device stays open while Ready and idle samples are discarded. Enabling this toggle closes it between clips and opens it on PTT: this avoids an open idle capture device but can cause an audio spike (observed with per-PTT transitions on Frame), startup latency or first-syllable clipping. Settings persistently displays OFF: discard idle audio; ON: spike / start latency below the toggle, as well as an ON/OFF indicator; the status/detail line also explains a change when toggled. The native setting is saved to config.json; a failed save applies only for this session and warns. This is not a mute switch for other applications. Quit and failure still close the device.

auto_insert is a separate boolean, default false, also available as a Settings toggle. Only a new recording arms it. It observes the Xwayland display selected by DISPLAY; its root _NET_ACTIVE_WINDOW and GAMESCOPE_FOCUSED_WINDOW must agree with the exact X keyboard-focus window. Both properties and focus are rechecked after IME lease acquisition. A watched focus-out, root focus-property change (even if the same window returns), window destruction, held keyboard key, missing X display or any disagreement permanently disarms that clip. The transcript then remains for explicit review/Type. Native Wayland focus and child text-field focus cannot be safely inferred here; those cases fall back to review. Manual Type now also requires a verifiable Xwayland target; it does not bypass an unavailable guard. No automatic Enter, speech commands or retry. The compositor can still change focus in the gap between the final check and global delivery, and IME commit is not an application receipt. This path has offline synthetic focus tests and a separate owned-target IME fixture; live speech-driven Auto insert, target coverage and headset acceptance remain unverified. The setting is preserved on upgrade and a failed preference write applies only to the current session.

Delivery uses a responsive, tick-driven queue of at most 24 Unicode codepoints per commit, with at least 150 ms between commits and before explicit Enter. The full 4096-byte literal is preserved, including all-space batches. The same target is checked before every batch; input blocked before the first send keeps review, whereas partial/uncertain input is consumed and never retried. Cancel stops remaining text and Enter; it cannot undo bytes already queued. Record and new Type actions are disabled while pacing. The clipboard is never touched. This source-grounded Gamescope workaround still needs long-text live acceptance.

quick_inputs is an editable list of 1–6 nonempty, printable ASCII strings, each at most 64 characters. Edit the JSON file and restart; there is no headset text editor. Inputs are literal (not expanded or interpreted by FrameYap) and are sent to the current Gamescope focus, so check the destination before Type + Enter. buttons maps named OpenVR actions (left_grip, right_grip, ptt, cancel, insert, enter, quick_chat) to Frame physical /user/hand/{left|right}/input/NAME button paths. Omitted actions retain their bundled defaults; an empty string disables a mapping, including after an upgrade. The Frame defaults are right X = hold-to-talk, B = Cancel, A = Type + space, Y = Quick phrases. Enter has no single-button mapping by default; the left grip double-tap still requests Type + Enter. An existing config mapping enter to right Y is migrated to Quick phrases in memory when quick_chat is absent; this does not overwrite custom mappings. Existing configs with empty actions retain those disabled mappings; change them explicitly or use SteamVR's binding editor. Paths must be distinct. Only the Frame binding is customized; SteamVR user overrides may still supersede it. On customized launches a generated action manifest and adjacent bindings are placed in $XDG_CACHE_HOME/frameyap/bindings (or ~/.cache/frameyap/bindings); the bundled manifest remains unchanged. The config is read once at launch, not hot-reloaded (the in-panel backend selection is saved separately). On install/upgrade the installer fills known missing fields, including close_mic_when_idle, backend and gradient, removes retired keys and unknown gradient subkeys, and resets invalid entries. It saves the exact prior bytes under config.json.backup-* before a repair and refuses symlink/oversized config paths; valid customizations remain intact. The installed launcher no longer pins --font, so this selection takes effect. Direct native launches with bad JSON, gradient fields, colors or button mappings fail startup rather than silently changing input behavior.

Models / backends (local implementation)

Settings → Models / backends opens a paginated local chooser. The saved "backend": "redux" selects a manifest ID, not a model download; --backend ID overrides it for that --run invocation. Selecting a listed backend restarts the owned worker and closes capture, discarding pending audio, review and any prior focus/delivery authorization. A failed preference write leaves the choice active only for this session. The currently listed Redux model is the only inference implementation supplied; adding a manifest alone does not add an inference runtime. The Models tab labels local checks as checking, missing (not_installed), invalid, or installed/verified; for the selected model it also reports loading, ready or failed. An install-in-progress note reports model provisioning, followed by a new offline check; a verified status alone is not proof the CPU runtime loaded. Recording is disabled until the selected files verify offline and the worker warms; missing files never trigger a silent download. Model loading and the default open-while-Ready idle microphone policy are unchanged. The panel's state and successful helper calls do not prove microphone transcription, input delivery or headset acceptance.

For a missing/invalid selected model, Install first presents a separate confirmation showing pinned source, approximate download size, license text, attribution and the exact raw manifest SHA-256 fingerprint. Only Confirm Install launches the local installer with that fingerprint; leaving the view or changing metadata invalidates consent. The installer rechecks the installed manifest bytes under its model lock before creating a download target or using the network, downloads only on the explicit click, hashes pinned files and then the app checks them again offline before enabling recording. Existing invalid or unsafe model files are refused rather than silently overwritten. This installs model files, not Python, Torch, moondream, Kestrel or other runtime dependencies. The source-tree UI needs an installed release for installer-backed provisioning; a hand-edited manifest is not an approved artifact. The installer handoff and native UI have not yet been accepted on a clean Frame. See packaging for an inspection-first CLI path.

Experimental controller input priority

There are two independent gates:

1. SteamVR's Developer setting Enable global input from overlays (called Experimental overlay input overrides in the SDK documentation) permits global action priority. FrameYap reads it and never changes it.
2. FrameYap config "input_priority": "experimental" requests k_nActionSetOverlayGlobalPriorityMin (0x01000000) for its existing action set. "normal" or an omitted field requests priority zero. Restart FrameYap after changing the config. Invalid values fail native startup; the installer repairs them to normal and backs up the prior bytes.

This applies to all controller sources bound to FrameYap actions, including custom SteamVR bindings. It does not replace the action manifest or change the physical button mappings. Bound sources can take input away from games or the dashboard. The request stays the same with the dashboard open/closed and Lasers anytime on/off so the experiment can compare those modes. Successful API calls do not prove that controller actions arrive or that dashboard interaction works.

Native startup logs the requested priority and SteamVR permission separately. The no-audio/no-delivery --check-controls probe also logs dashboard state, the Lasers anytime flag, IsInputAvailable, panel visibility/focus, and activity, press state and pose/role acceptance for all six actions. The laser flag is our request, not a detector for every system laser. Compare the same bindings in each dashboard/laser state; check pointer clicks and press/release through mode transitions too. Return input_priority to normal and relaunch to end the FrameYap experiment.

Placement settings

First launch defaults to World space: a standing-universe absolute transform, 1.05 m ahead of the first valid headset pose, 0.16 m below eye height, upright (yaw only). It stays there as the wearer moves. Startup waits for valid tracking rather than placing a menu at the world origin. A tracking-origin reset requests a fresh placement. Settings → Recenter in front deliberately resamples the pose.

Settings offers World space, Left wrist, Right wrist and Head on that same canvas. World/head width is 0.85 m; wrist width defaults to 0.30 m. All mounts have a thin grab bar below the main panel and an L-shaped scale bracket outside its lower-right corner, visually modeled on the user's Steam terminal-window screenshot. These are original FrameYap controls, not Steam private UI components. RGBA alpha leaves their surrounding area transparent; an explicit OpenVR intersection mask excludes empty margins from laser hit testing (the slender strokes have larger hit targets). Both handles use the configured mint-to-blue frame gradient. The extra canvas area does not shrink the main panel's physical width. Intersection rectangles use top-left pixel coordinates; only mouse events need the bottom-left GL Y flip. The first deployed pass incorrectly flipped the mask, leaving the scale corner outside its input region; this version corrects that mapping.

Hold the laser's primary click on the bar to freely position and rotate the panel with the controller, including depth, pitch, yaw and roll. Grab captures inverse(controller_down) * panel_down and applies that unchanged relative pose to each controller pose, so grabbing does not snap or reset the panel orientation. During a grab, the grabbing hand's thumbstick Y axis also moves the panel along the captured panel's local -Z normal: forward pushes away, backward pulls closer. The axis is velocity with a 0.2 dead zone, 0.6 m/s at full deflection and a ±1.5 m offset limit from the pose-only grab. Returning to center holds the current depth; the other hand's stick and scale drags do not change it. A lost axis stops adding depth without resetting placement. Frame controller bindings provide left/right vector2 actions in a separate /actions/grab set, including in generated custom button manifests. Only a grab activates this set, restricted to the grabbing hand at overlay priority; that app-action route requires SteamVR Experimental overlay input overrides. If the app axis is unavailable, controller-tagged compositor smooth-scroll events provide depth at 0.04 m per scroll unit, bounded to 3 cm per event and the same total travel limit. Only the grabbing controller is accepted. Smooth events are requested only during grab, integrated once without multiplying by elapsed time, and discarded whenever the app axis is available to avoid applying both streams. It does not promote recording or text actions, and does not depend on FrameYap's ordinary input_priority preference. Axis delivery during laser drag and dashboard/game focus needs a separate headset check. Grab scroll events affect only depth; scale scroll events are ignored, and ordinary UI scrolling resumes after release. Release leaves the last pose in the chosen mount frame; head/wrist mounts continue following that anchor afterward. A completed grab or scale on Head, Left wrist or Right wrist saves the full device-relative canvas position, rotation and scale separately for that mount. Mount changes restore each saved relative placement; restarting does too. A cancelled drag does not update the saved placement. World-space placement is resampled on startup/recenter rather than restored against a possibly different tracking origin.

Drag the corner bracket to scale between half and twice the configured width. Scaling freezes its initial plane and calibrates a ray from the source controller to the initial hit. The top-left corner of the current, potentially rotated panel stays fixed, including after repeated grab/scale operations. Changing overlay coordinates never feed back into the calculation. Both controls work on either tab; for head/wrist mounts, geometry stays in the anchor-device coordinate frame.

Settings → Lock grab/scale ("lock_layout": true, default false) hides both handles and removes their hit regions, cancels an active drag and prevents new manipulation. Unlock remains accessible in Settings. The boolean is saved across restarts/upgrades; failed saves leave a visible session-only warning. It does not disable explicit mount/recenter choices or freeze normal head/wrist tracking. The event's controller is used, with the primary dashboard device as the single-cursor fallback when the event omits it. No guessed controller or desktop pointer fallback. Release, changed UI authorization, tracking loss, hidden overlay, relocation, invalid ray geometry or a 15-second safety limit cancels the drag. If legacy trigger state is observable on press, release is also checked through that state, including outside the texture. Otherwise the drag cancels as soon as this overlay stops being the hover target, rather than waiting for a potentially missing outside MouseButtonUp. Other pointer approvals and bound actions cannot fire during a drag. Source-device reporting, out-of-bounds release, mask behavior and comfort still need on-headset acceptance. The left wrist uses VR Workspace's fallback watch-face axes: panel-right points toward the fingers (controller -Z), panel-up points out of the back of the hand (controller +Y), and panel-front points toward controller +X. The right wrist reverses panel-right and panel-front (controller +Z and -X), keeping panel-up unchanged so it faces inward with upright, unmirrored text. The controller-relative center is (0, 0.18, 0.089) m, approximating the compact HUD's surface center: its 0.12 m wrist lift, 0.09 m bottom anchor and ~0.03 m panel-center correction; Z combines the fallback 0.054 m wrist calibration and 0.035 m finger-back offset. Wrist-mounted panels are fully visible while their entire orientation is within 60° of an upright, viewer-facing panel; linear opacity fade from 60° to 75°, then hidden (including laser interaction). Pitch, yaw and roll contribute together; turning the wrist away or moving the head around it changes the angle. OpenVR's overlay alpha changes without rerendering the panel. World and head mounts do not fade. Missing headset tracking hides a wrist panel; by default a lost wrist uses a world-space fallback. Headset readability, fade feel and interaction at the threshold still need live acceptance.

To tune the selected wrist, set wrist in config.json as in the example above: x, y, z are controller-local meters (each -0.3 to 0.3), width is panel width in meters (0.15 to 0.6), and roll_degrees rotates about controller -Z (-180 to 180) before applying the offset. These settings are initial defaults for both wrists; a saved per-wrist pose takes precedence. They are read at startup and do not alter world/head placement. Invalid values fail direct native startup; the installer backs up and repairs invalid entries. A missing/untracked selected wrist temporarily falls back to world space, with a visible explanation in Settings, then reattaches when tracking returns. Use Settings → Wrist world fallback to switch between the default front-of-you fallback (ON) and hiding the panel when wrist tracking is lost (OFF). It reappears on the selected wrist when tracking returns. The toggle is saved to "wrist_world_fallback" in config.json; manual edits take effect on restart. This option affects wrist mounts only, not World or Head. While hidden, pointer controls cannot be used. The saved mount preference is not replaced by the fallback. These offsets and sizes are initial choices, not headset-comfort acceptance.

A selection saves the mount token to $XDG_CONFIG_HOME/frameyap/mount (or $HOME/.config/frameyap/mount). Completed relative adjustments save to placement-left-wrist, placement-right-wrist, or placement-head beside that file. Remove one placement file while FrameYap is closed to restore that mount's default. These are bounded, versioned device-relative transforms with owner-only atomic writes; invalid files are ignored. World-space poses, audio and transcripts are not saved. Missing/invalid mount settings default to World; write failure keeps the placement for the session and displays a warning. --mount world|left-wrist|right-wrist|head overrides the saved choice for one launch without writing it; --head remains an alias for --mount head.

Settings also has Lasers anytime (default off). Open the dashboard to change it when system-wide lasers are disabled. When enabled, FrameYap sets OpenVR's VROverlayFlags_MakeOverlaysInteractiveIfVisible on its panel. OpenVR requests system-wide laser mouse mode while the panel is visible, including with Steam's dashboard closed; it may change interaction with games. Turning it off removes that request. This is not the experimental overlay action priority override and does not promise pass-through of a dashboard-owned button or PTT delivery during dashboard focus. The choice is saved as on or off in $XDG_CONFIG_HOME/frameyap/lasers-anytime (default ~/.config/frameyap/lasers-anytime) using a private atomic file replacement; missing, symlinked or invalid files mean off. A failed save keeps the new choice only for the running session and shows a warning. To turn it back on after disabling it with the dashboard closed, open the dashboard to use its laser on the Settings button. Controls-only checks can toggle it temporarily but do not save the preference. The wearer reports that pointer clicks work, while normal-priority controller actions become unavailable in system laser mode, including when this preference is enabled. Experimental-priority coexistence remains unverified.

Hardware-free UI checks

The default build tests mount parsing, laser preference persistence, pose geometry, controller-relative grab/captured-ray scale math and stick-depth integration without any native dependencies. Drag tests cover XYZ translation, pitch/yaw/roll and lever-arm rotation, re-grabbing a moved panel, stationary stability, relative mounts, out-of-bounds scale hits, invalid poses/rays and no feedback from prior updates. UI/config/installer tests cover hidden/disabled handles, top-left mask regions, lock persistence and repair. A FreeType-only opt-in build exercises the actual renderer, pointer gating, tab switches, pagination, recording state and redraw invalidation:

cmake -S . -B build-ui -DFRAMEYAP_UI_TESTS=ON
cmake --build build-ui
ctest --test-dir build-ui --output-on-failure
# Optional synthetic fixture previews; no OpenVR initialization:
./build-ui/frameyap_panel_test assets/fonts/Inconsolata-Regular.ttf /tmp/frameyap-ui

The last command writes -review.ppm, -settings.ppm and -recording.ppm to the supplied prefix. The native build includes these tests too; tests never initialize OpenVR or touch the real mounting preference. Physical pointing, tracking loss, recentring and readability still require a separately authorized headset check.

The opt-in native --check-controls probe logs pointer counters and action callbacks to the terminal rather than repainting them on the panel. Its canvas stays static for Record/Cancel/Type/Type + Enter clicks so those clicks can be checked without diagnostic texture uploads. Switching tabs or mount still updates the visible panel. Diagnostics identify renderer=Vulkan and count textureUploads; raw/file ImageLoaded events are not GPU upload completions. The native CTest suite tests persistent image reuse, queued transfer ordering, coherent/noncoherent staging memory and error cleanup against Vulkan fakes; it does not initialize the Vulkan loader, a GPU or OpenVR for that test.

An optional offscreen check exercises the real Vulkan backend with eight synthetic RGBA patterns, verifying exact readback and image reuse. It needs a GPU/driver, is excluded from normal builds and CTest, and requires --run:

cmake --build build-native --target frameyap_texture_check
./build-native/frameyap_texture_check --run

It does not initialize OpenVR or establish compositor/headset acceptance.

assets/actions.json names nine actions: left/right grip, PTT, cancel, Type, Type + Enter, Quick phrases and the two hand-specific depth axes. insert, enter, and quick_chat remain internal binding keys; visible controls read Type, Type + Enter, Quick phrases. bindings_frame_controller.json maps right X click to hold-to-talk PTT; the grip bindings remain for optional remapping/diagnosis. In one dashboard probe grips were inactive; a later controls-only probe delivered repeated right X PTT BeginRecord/EndRecord callbacks. The wearer reports controller actions are usable with Steam's dashboard closed, not with the dashboard itself open. Neither probe used a microphone or established game-scene pass-through. bindings_knuckles.json is an additional Index/knuckles example only; it includes the two thumbstick depth axes alongside grip taps. Collisions with scene actions require separate on-device validation. Left grip double tap (releases <=250 ms, second press within 350 ms) requests explicit Enter only when enabled. Right grip: first short squeeze and release (<=250 ms), then second squeeze down within 350 ms starts capture; hold as long as needed (up to runtime's clip bound), second release ends capture. The named PTT action explicitly begins on down and ends on up. On tracking-pose invalidity, action inactivity or overlay focus loss, a held capture emits Cancel, and reconnection requires a neutral observation before any new press. PTT and left Type + Enter require an enabled panel; clickable Record remains available for retry after an error and Cancel is always available. The action set defaults to normal priority; the experimental config request is described above. Neither priority guarantees delivery while a game or dashboard owns input.

The UI cannot itself guarantee a capture started when a BeginRecord action arrives: the owning runtime checks worker readiness. Cancel invalidates capture/worker work; a failed startup can be retried with Record. The UiAction::Toggle enumerator remains for caller ABI compatibility but the panel no longer emits it: Previous is navigation only. Pointer Record/Stop emits explicit BeginRecord/EndRecord, just like PTT edges, so a stale Stop cannot become a new recording when another input has already ended capture.

Registration

assets/application.vrmanifest.in is a template, not a runnable manifest. At install, substitute absolute executable and action JSON paths and store it under the user-owned install directory, keeping bindings adjacent to action JSON. Register local.frameyap.overlay using registration(path,false,false); autolaunch is opt-in. Unregister before deleting the installed manifest. The app key is not a Steam store AppID. Registration uses OpenVR Utility init only when explicitly invoked, then verifies IsApplicationInstalled. The device required binary_path_linux_arm in the manifest (a generic or Linux-only path was silently skipped despite successful AddApplicationManifest return). Registration, repeated registration and removal were tested against the running Frame runtime, with autolaunch verified off. The overlay explicitly identifies its process with the registered app key before setting its action manifest. Cold-runtime behavior and actual SteamVR-menu launch still need validation. See packaging for the published native archive and release checks.

A live headset check must be opt-in and distinguish overlay API discovery from controller delivery, actual transcription, insertion into a disposable target, and human comfort/acceptance.

Opt-in rendering measurements

FRAMEYAP_PROFILE=1 prints aggregate CPU raster time, upload time, and grab/scale tracking/transform time on exit. Counts and timings contain no speech, typed text or pointer coordinates. The motion measurements exclude sleep and other runtime work; they do not measure motion-to-photon latency. During an initialized grab or scale, neither the renderer nor uploader runs; content updates resume on release.