baketnk--frame-yap/docs/overlay.md

# Native OpenVR panel

`src/overlay.hpp` provides RAII OpenVR ownership and `registration()`. The panel
only emits UI actions; `src/runtime.cpp` owns audio, transcription and insertion.
Native `--run` wiring in `src/main.cpp` is behind the explicit
`FRAMEYAP_NATIVE` build option. The Models chooser, offline verification and
explicit installer handoff are implemented locally, **not** an accepted Frame
install/voice-typing path or a published release. Neither hardware-free tests
nor a successful compile establish Frame input, visibility, comfort or text
delivery. Launching the runtime is explicit, never part of a normal build or test.

## Rendering and controls

Explicit development dependencies: Valve OpenVR SDK v2.15.6, Vulkan headers/loader
and FreeType 2. The native runtime needs a compatible system Vulkan driver.
Configure/build must not fetch them. The default font is the bundled Inconsolata
Regular; see [third-party notes](third-party.md). `--font FILE` overrides the JSON selection.
A missing selected font falls back to bundled Inconsolata, then a system DejaVu
Sans face if present. Glyph coverage depends on the selected face; full CJK
coverage is not claimed.

`src/panel_surface.*` renders **one 1080×780 RGBA canvas** containing a 1000×680
main panel for review, settings, status and controls, plus transparent right/bottom
margins for a thin grab underline and an external L-shaped scale handle. `src/overlay_texture.*` uploads this CPU canvas into one
persistent Vulkan RGBA8 image and submits it with `SetOverlayTexture`. The image,
staging allocation and command buffer are reused; tabs do not create extra
overlays or render targets. The rounded mint-to-blue perimeter,
shallow curved accent, and dark cards form the panel's visual language.
With `gradient.enabled` (opt-in), a low-strength animated field blends
`theme.frame_start` and `theme.frame_end` into `theme.background`; the perimeter
and grab/scale handles share its phase and canvas coordinates. Rounded
preview, status and control surfaces use independently rasterized antialiased edges
and restrained baked neon halos rather than GPU bloom. The recording indicator and
selected controls remain distinguishable by their labels, not color alone. Rounded
control hit areas exclude their clipped corners.
Rendering/uploads occur for changed content, scroll position or settings, and, when the
animated gradient is enabled and visible, at most every 100 ms (10 fps) on a
monotonic clock. There is no animation redraw while hidden. Laser hover and
button down/up are hit-tested without an upload; static frames are reused when
animation is disabled. The caller may call `draw(Panel)` at 10 ms intervals.
Tracking transforms do not require repainting the canvas.

The Vulkan instance/device enable the extensions requested by the running
SteamVR runtime and use its selected physical device and a graphics queue.
There is no desktop window, swapchain, SDL video dependency or raw-upload
fallback. Updates wait for the dedicated queue's previous upload and OpenVR
transfer before reusing staging memory. Image barriers finish in
`TRANSFER_SRC_OPTIMAL`, as required by
[OpenVR's Vulkan contract](https://github.com/ValveSoftware/openvr/wiki/Vulkan).
The queue is used on the overlay thread; GPU resources outlive `VR_Shutdown`.
The device selection, texture description and persistent panel-upload patterns
are FrameYap's own implementation.
GPU setup/submission errors stop startup or the run with an explicit error.
This replaces the raw-upload rendering path; headset flicker acceptance still
requires an on-device comparison. Native installation and offscreen GPU checks are separate from headset acceptance.

The header shows local time and date (right-aligned) instead of the former
on-device/review and current-mount labels. Between the title and the clock it
shows battery levels for the left controller, headset (HMD) and right controller.
Controllers and the headset use OpenVR's `Prop_DeviceBatteryPercentage_Float`
(with `Prop_DeviceIsCharging_Bool`) when the device reports
`Prop_DeviceProvidesBatteryStatus_Bool`; if the headset reports none, the local
Linux `/sys/class/power_supply` system battery is read instead (peripheral
`scope=Device` supplies are skipped). Levels refresh every five seconds; a device
without a reading is hidden, never shown as zero. 20% or less is drawn in the
warning color, charging in the accent color with `+`. The status line carries a
**Buttons ready / Buttons paused** chip from `IVROverlay::IsDashboardVisible()`:
while the SteamVR dashboard is open, controller bindings do not reach FrameYap,
though pointer clicks still work. The chip reflects dashboard visibility only,
not every system or game input override. `--check-controls` prints the same
battery readings. Which devices Frame actually reports still needs a headset check.
Settings explains Hold Quit (hold 0.9 seconds then
release) and Lasers anytime (system-wide lasers may affect games). The header
updates when the displayed minute, date, battery or dashboard state changes,
not every frame. Settings toggles 12/24-hour time and cycles date Off →
MM/DD/YYYY → DD/MM/YYYY → YYYY-MM-DD → Off. These only affect display;
mount choices remain in Settings. The Settings controls form one vertically
scrollable list: hover the laser over the list and use the right stick to scroll.
The tabs, status and action footer remain fixed. Scrolling cancels a pending
pointer press rather than activating a different control. There are no Settings
pages. OpenVR's discrete/smooth laser scroll events are requested; right-stick
behavior with SteamVR's keyboard open still needs a live Frame check.

Settings → **About** replaces the settings area with static information: the
build version (plus git commit when built from an untagged or modified tree),
source URL, MIT license, the selected model's name and license from its manifest,
the bundled font's license, and a pointer to `docs/third-party.md`. It performs
no checks and no network access; Settings returns. Recording controls stay live.

At the bottom of Settings, **Check for updates** performs an explicit, asynchronous
request to GitHub's latest published release metadata. It never runs at launch,
while idle or as part of offline tests. The bounded, timeout-limited helper
compares the release's numeric tag with this build's version; repository commits
and prereleases are not update candidates. No archive or installer is downloaded
by a check. If a newer release is found, **Install update...** opens Konsole
with the installed handoff script. The terminal waits for the wearer to close
FrameYap and press Enter before the existing installer downloads a pinned
version and checksum and installs it. Until that confirmation, the terminal
does not fetch anything. An unavailable check or terminal reports an error in
Settings; no fallback auto-install is attempted.

The transcript wraps by glyph width and scrolls in its review viewport with
the right-stick laser wheel; a new transcript resets the scroll position.
The former paging row holds **Open Plan**, **Open Keyboard** and **Open Draw**
when the installed `tnkplan`, `tnkboard` or `tnkdraw` wrapper is found at startup
in `~/.local/bin` or an absolute PATH directory. Missing apps have no button;
available buttons share the row. Open Keyboard short press runs the installed
wrapper with fixed `--show`; holding it for 800 ms runs fixed `--recenter`
once and suppresses the short action, placing the keyboard in front of the
wearer. Leaving the button, losing focus/tracking or hiding the overlay cancels
a pending hold. Other companion buttons launch their installed wrappers without
a shell (their normal second-launch behavior may toggle the existing panel).
Restart FrameYap after installing a companion so its button is discovered.
Status fits on the single status line; the old bottom detail label is gone.
The footer remains available on all tabs: Record (labelled Stop while recording),
Cancel, Type (labelled Enter when nothing is pending review), Type + Enter, Hold Quit. Hold Quit needs a 900 ms press and release on
that same button; its thin progress bar shows the hold. Record can retry after an
error; it is disabled while warming/transcribing and until an existing review is
typed or discarded. Cancel can stop worker startup. Type and Type + Enter are
disabled during recording and transcription. A pointer action requires
a press/release on the same enabled control from the same cursor; focus loss,
tab changes, action-state changes and relocation clear pending presses. Type + Enter
is *always* a separate deliberate action, not inferred from text. Type normally
appends a trailing space (without doubling an existing one); a full 4096-byte
transcript without room for that suffix is queued unchanged, with no extra error
for the missing space. With no pending review, Type (button or A) queues Enter
alone, the same as Type + Enter; a quick second A after Type therefore submits.
Type + Enter types any pending review and then queues
Enter; with no pending text it queues Enter only. Y opens the Quick phrases
list over the review area; each further Y press cycles its highlighted choice. Cancel closes the picker without discarding an
existing review. Type + Enter sends the selected phrase *without* a trailing
space, then Enter. The choices are short
single-line literals, not speech commands. A failed text step never proceeds to
Enter. Recording never automatically submits. Auto insert, when
explicitly enabled, can queue text (normally + space) after transcription only
under the stable Xwayland focus guard described below.

### Bindings button

**Bindings** requests SteamVR's in-headset binding editor directly for the
current process/action set, without changing the FrameYap tab. SteamVR owns
remapping, persistence and the full binding view. The Review tab shows a request
or error note after the call. Opening it clears pending pointer presses and
rearms controller gestures from neutral.

OpenVR 2.15.6 provides `OpenBindingUI`. Frame's installed controller profile
references left/right SVG diagrams for SteamVR's own editor. FrameYap
does not copy runtime artwork into its package.
Editor availability and artwork rendering still require headset acceptance.

Normal priority with Lasers anytime off is the practical baseline: the wearer
reports controller actions with the dashboard closed and clickable UI with it
open. The menu does not enable global overrides or promise simultaneous access.

### User theme and controller configuration

Optional `$XDG_CONFIG_HOME/frameyap/config.json` (fallback
`~/.config/frameyap/config.json`, only with an absolute HOME) is read at native
overlay startup. The native binary does not create a config by itself; the
installer creates one with defaults on first install. Copy the shipped
`assets/config.example.json` to that path for manual installs. Example:

```json
{
  "font": "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
  "input_priority": "normal",
  "advanced_debug": false,
  "auto_insert": false,
  "close_mic_when_idle": false,
  "backend": "redux",
  "lock_layout": false,
  "wrist_world_fallback": true,
  "clock_24h": false,
  "date_format": "mdy",
  "quick_inputs": ["/new", "/questions", "/help"],
  "wrist": {"x": 0, "y": 0.18, "z": 0.089, "width": 0.30, "roll_degrees": 0},
  "theme": {
    "background": "#0c101b", "card": "#141c2b", "ink": "#e6f0f9",
    "muted": "#97adc1", "accent": "#1ff0a4", "warning": "#ff6e87",
    "frame_start": "#1fff91", "frame_end": "#1f70ff"
  },
  "gradient": {"enabled": false, "period_seconds": 30, "strength": 0.12},
  "buttons": {
    "ptt": "/user/hand/right/input/x",
    "cancel": "/user/hand/right/input/b",
    "insert": "/user/hand/right/input/a",
    "enter": "",
    "quick_chat": "/user/hand/right/input/y",
    "left_grip": "/user/hand/left/input/grip",
    "right_grip": "/user/hand/right/input/grip"
  }
}
```

`clock_24h` is a boolean (default `false`); `date_format` is `off`, `mdy`
(default), `dmy`, or `iso`. The Settings buttons update these preferences
immediately and save them to the user config when persistence is enabled.
A failed save warns and leaves the selection active for this run. Time uses
the device's local timezone; these controls do not change system time.

Each theme color is `#RRGGBB`; omitted colors keep the default. The optional
`gradient` object defaults to `{"enabled": false, "period_seconds": 30, "strength": 0.12}`.
`enabled` must be a boolean; `period_seconds` must be a
finite number from 5 to 300 (seconds per full cycle), and `strength` a finite
number from 0 to 0.3. The animation uses a smooth periodic cosine field with
one start/end/start cycle across the canvas width. Its edge and handle colors
use the same global canvas coordinates and time phase; it does not add a
separate handle animation. Colors always come from `theme.frame_start` and
`theme.frame_end`, while `strength` controls how much of those colors blends
into `theme.background`. The default static mode keeps the solid background
and existing linear perimeter/handle gradients. Settings → **Animated background**
turns animation on/off immediately and saves only `gradient.enabled` in the JSON
config (failed saves leave a session-only choice and a warning). Manual JSON edits
take effect on restart; existing enabled configs stay enabled on upgrade. This is locally implemented,
not device validated; headset appearance and rendering cost remain unverified.
`font` is a TTF/OTF file path (not a family name); a missing file uses the
bundled font.
`advanced_debug` is a boolean (default `false`, not a string): an opt-in
request for full diagnostic logs. Full logs may contain speech/transcribed text
and local paths; **raw audio clips are not archived**. The Settings tab shows
an Advanced debugging ON/OFF toggle and warns that changing it restarts the
worker and cancels current work (including pending review). Changes take effect
immediately; failed saves show a warning and keep the selection for this session.
Detailed logs are bounded and owner-private; see [worker diagnostics](worker.md#advanced-debugging). The native
`save_advanced_debug(path, bool)` helper updates only this value in a valid
config, retaining other fields and formatting; invalid/unwritable configs are
left untouched and return failure. The installer backs up original bytes before
repairing invalid values, while valid `true` and `false` are retained.

`close_mic_when_idle` is a separate boolean, default **false**, also available
as Settings → **Close mic when idle**. Normally the SDL capture device stays
open while Ready and idle samples are discarded. Enabling this toggle closes it
between clips and opens it on PTT: this avoids an open idle capture device but
can cause an audio spike (observed with per-PTT transitions on Frame), startup
latency or first-syllable clipping. Settings persistently displays **OFF: discard
idle audio; ON: spike / start latency** below the toggle, as well as an ON/OFF
indicator; the status/detail line also explains a change when toggled. The
native setting is saved to `config.json`; a failed save applies only for this
session and warns. This is not a mute switch for other applications. Quit and
failure still close the device.

`auto_insert` is a separate boolean, default `false`, also available as a
Settings toggle. Only a **new** recording arms it. It observes the Xwayland
display selected by `DISPLAY`; its root `_NET_ACTIVE_WINDOW` and
`GAMESCOPE_FOCUSED_WINDOW` must agree with the exact X keyboard-focus window.
Both properties and focus are rechecked after IME lease acquisition. A watched
focus-out, root focus-property change (even if the same window returns), window
destruction, held keyboard key, missing X display or any disagreement permanently
disarms that clip. The transcript then remains for explicit review/Type.
Native Wayland focus and child text-field focus cannot be safely inferred here;
those cases fall back to review. Manual Type now also requires a verifiable
Xwayland target; it does not bypass an unavailable guard. No automatic Enter,
speech commands or retry.
The compositor can still change focus in the gap between the final check and
global delivery, and IME commit is not an application receipt. This path has
offline synthetic focus tests and a separate owned-target IME fixture; live
speech-driven Auto insert, target coverage and headset acceptance remain
unverified. The setting is preserved on upgrade
and a failed preference write applies only to the current session.

Delivery uses a responsive, tick-driven queue of at most 24 Unicode codepoints
per commit, with at least 150 ms between commits and before explicit Enter.
The full 4096-byte literal is preserved, including all-space batches. The same
target is checked before every batch; input blocked before the first send keeps
review, whereas partial/uncertain input is consumed and never retried. Cancel
stops remaining text and Enter; it cannot undo bytes already queued. Record and
new Type actions are disabled while pacing. The clipboard is never touched.
This source-grounded Gamescope workaround still needs long-text live acceptance.

`quick_inputs` is an editable list of 1–6 nonempty, printable ASCII strings,
each at most 64 characters. Edit the JSON file and restart; there is no headset
text editor. Inputs are literal (not expanded or interpreted by FrameYap) and
are sent to the current Gamescope focus, so check the destination before Type + Enter.
`buttons` maps named OpenVR actions (`left_grip`, `right_grip`, `ptt`, `cancel`,
`insert`, `enter`, `quick_chat`) to Frame physical `/user/hand/{left|right}/input/NAME`
button paths. Omitted actions retain their bundled defaults; an empty string
disables a mapping, including after an upgrade. The Frame defaults are right
X = hold-to-talk, B = Cancel, A = Type + space, Y = Quick phrases. Enter has no
single-button mapping by default; the left grip double-tap still requests Type + Enter.
An existing config mapping `enter` to right Y is migrated to Quick phrases in memory
when `quick_chat` is absent; this does not overwrite custom mappings.
Existing configs with empty actions retain those disabled mappings; change them explicitly
or use SteamVR's binding editor. Paths must be distinct. Only the Frame binding is customized;
SteamVR user overrides may still supersede it. On customized launches a generated
action manifest and adjacent bindings are placed in `$XDG_CACHE_HOME/frameyap/bindings`
(or `~/.cache/frameyap/bindings`); the bundled manifest remains unchanged. The
config is read once at launch, not hot-reloaded (the in-panel backend selection
is saved separately). On install/upgrade the installer fills known missing fields,
including `close_mic_when_idle`, `backend` and `gradient`, removes retired keys
and unknown `gradient` subkeys, and resets invalid entries. It saves the exact
prior bytes under `config.json.backup-*` before a repair and refuses symlink/oversized config paths; valid customizations
remain intact. The installed launcher no longer pins `--font`, so this selection takes effect. Direct native
launches with bad JSON, gradient fields, colors or button mappings fail startup
rather than silently changing input behavior.

### Models / backends (local implementation)

Settings → **Models / backends** opens a paginated local chooser. The saved
`"backend": "redux"` selects a *manifest ID*, not a model download;
`--backend ID` overrides it for that `--run` invocation. Selecting a listed backend
restarts the owned worker and closes capture, discarding pending audio, review
and any prior focus/delivery authorization. A failed preference write leaves the
choice active only for this session. The currently listed Redux model is the
only inference implementation supplied; adding a manifest alone does not add
an inference runtime. The Models tab labels local checks as checking, missing
(`not_installed`), invalid, or installed/verified; for the selected model it
also reports loading, ready or failed. An install-in-progress note reports
model provisioning, followed by a new offline check; a verified status alone
is not proof the CPU runtime loaded. Recording is disabled until the selected
files verify offline and the worker warms; missing files never trigger a silent
download. Model loading and the default open-while-Ready idle microphone policy
are unchanged. The panel's state and successful helper calls do **not** prove
microphone transcription, input delivery or headset acceptance.

For a missing/invalid selected model, **Install** first presents a separate
confirmation showing pinned source, approximate download size, license text,
attribution and the exact raw manifest SHA-256 fingerprint. Only **Confirm
Install** launches the local installer with that fingerprint; leaving the view
or changing metadata invalidates consent. The installer rechecks the *installed*
manifest bytes under its model lock before creating a download target or using
the network, downloads only on the explicit click, hashes pinned files and then
the app checks them again offline before enabling recording. Existing invalid
or unsafe model files are refused rather than silently overwritten. This
installs model files, **not** Python, Torch, moondream, Kestrel or other runtime
dependencies.
The source-tree UI needs an installed release for installer-backed provisioning;
a hand-edited manifest is not an approved artifact. The installer handoff and
native UI have not yet been accepted on a clean Frame. See
[packaging](packaging.md#consumer) for an inspection-first CLI path.

### Experimental controller input priority

There are two independent gates:

1. SteamVR's Developer setting **Enable global input from overlays** (called
   **Experimental overlay input overrides** in the SDK documentation) permits
   global action priority. FrameYap reads it and never changes it.
2. FrameYap config `"input_priority": "experimental"` requests
   `k_nActionSetOverlayGlobalPriorityMin` (`0x01000000`) for its existing action
   set. `"normal"` or an omitted field requests priority zero. Restart FrameYap
   after changing the config. Invalid values fail native startup; the installer
   repairs them to normal and backs up the prior bytes.

This applies to all controller sources bound to FrameYap actions, including
custom SteamVR bindings. It does not replace the action manifest or change the
physical button mappings. Bound sources can take input away from games or the
dashboard. The request stays the same with the dashboard open/closed and Lasers
anytime on/off so the experiment can compare those modes. Successful API calls
do not prove that controller actions arrive or that dashboard interaction works.

Native startup logs the requested priority and SteamVR permission separately.
The no-audio/no-delivery `--check-controls` probe also logs dashboard state, the
Lasers anytime flag, `IsInputAvailable`, panel visibility/focus, and activity,
press state and pose/role acceptance for all six actions. The laser flag is our
request, not a detector for every system laser. Compare the same bindings in
each dashboard/laser state; check pointer clicks and press/release through mode
transitions too. Return `input_priority` to `normal` and relaunch to end the
FrameYap experiment.

### Placement settings

First launch defaults to **World space**: a standing-universe absolute transform,
1.05 m ahead of the first valid headset pose, 0.16 m below eye height, upright
(yaw only). It stays there as the wearer moves. Startup waits for valid tracking
rather than placing a menu at the world origin. A tracking-origin reset requests
a fresh placement. Settings → Recenter in front deliberately resamples the pose.

Settings offers World space, Left wrist, Right wrist and Head on that same canvas.
World/head width is 0.85 m; wrist width defaults to 0.30 m. All mounts have
a thin grab bar below the main panel and an L-shaped scale bracket outside its
lower-right corner, visually modeled on the user's Steam terminal-window screenshot.
These are original FrameYap controls, not Steam private UI components. RGBA alpha
leaves their surrounding area transparent; an explicit OpenVR intersection mask
excludes empty margins from laser hit testing (the slender strokes have larger hit
targets). Both handles use the configured mint-to-blue frame gradient. The extra
canvas area does not shrink the main panel's physical width. Intersection rectangles
use top-left pixel coordinates; only mouse events need the bottom-left GL Y flip.
The first deployed pass incorrectly flipped the mask, leaving the scale corner
outside its input region; this version corrects that mapping.

Hold the laser's primary click on the bar to **freely position and rotate** the
panel with the controller, including depth, pitch, yaw and roll. Grab captures
`inverse(controller_down) * panel_down` and applies that unchanged relative pose
to each controller pose, so grabbing does not snap or reset the panel orientation.
During a grab, the grabbing hand's thumbstick Y axis also moves the panel along
the captured panel's local -Z normal: forward pushes away, backward pulls closer.
The axis is velocity with a 0.2 dead zone, 0.6 m/s at full deflection and a
±1.5 m offset limit from the pose-only grab. Returning to center holds the
current depth; the other hand's stick and scale drags do not change it. A lost
axis stops adding depth without resetting placement. Frame controller bindings
provide left/right vector2 actions in a separate `/actions/grab` set, including
in generated custom button manifests. Only a grab activates this set, restricted
to the grabbing hand at overlay priority; that app-action route requires SteamVR Experimental
overlay input overrides. If the app axis is unavailable, controller-tagged
compositor smooth-scroll events provide depth at 0.04 m per scroll unit, bounded
to 3 cm per event and the same total travel limit. Only the grabbing controller
is accepted. Smooth events are requested only during grab, integrated once
without multiplying by elapsed time, and discarded whenever the app axis is
available to avoid applying both streams. It does not promote recording or text actions, and does
not depend on FrameYap's ordinary `input_priority` preference. Axis delivery during laser drag and dashboard/game focus needs a
separate headset check. Grab scroll events affect only depth; scale scroll
events are ignored, and ordinary UI scrolling resumes after release.
Release leaves the last pose in the chosen mount frame; head/wrist mounts continue
following that anchor afterward. A completed grab or scale on Head, Left wrist or
Right wrist saves the full device-relative canvas position, rotation and scale
separately for that mount. Mount changes restore each saved relative placement;
restarting does too. A cancelled drag does not update the saved placement.
World-space placement is resampled on startup/recenter rather than restored
against a possibly different tracking origin.

Drag the corner bracket to scale between half and twice the configured width.
Scaling freezes its initial plane and calibrates a ray from the source controller
to the initial hit. The top-left corner of the current, potentially rotated panel
stays fixed, including after repeated grab/scale operations. Changing overlay
coordinates never feed back into the calculation. Both controls work on either tab;
for head/wrist mounts, geometry stays in the anchor-device coordinate frame.

Settings → **Lock grab/scale** (`"lock_layout": true`, default false) hides both
handles and removes their hit regions, cancels an active drag and prevents new
manipulation. Unlock remains accessible in Settings. The boolean is saved across
restarts/upgrades; failed saves leave a visible session-only warning. It does not
disable explicit mount/recenter choices or freeze normal head/wrist tracking.
The event's controller is used, with the primary dashboard device as the single-cursor
fallback when the event omits it. No guessed controller or desktop pointer fallback.
Release, changed UI authorization, tracking loss, hidden overlay, relocation, invalid
ray geometry or a 15-second safety limit cancels the drag. If legacy trigger state is
observable on press, release is also checked through that state, including outside
the texture. Otherwise the drag cancels as soon as this overlay stops being the
hover target, rather than waiting for a potentially missing outside MouseButtonUp. Other pointer approvals and bound actions cannot fire during a drag.
Source-device reporting, out-of-bounds release, mask behavior and comfort still need
on-headset acceptance. The left wrist uses
VR Workspace's fallback watch-face axes: panel-right points toward the fingers
(controller -Z), panel-up points out of the back of the hand (controller +Y),
and panel-front points toward controller +X. The right wrist reverses panel-right
and panel-front (controller +Z and -X), keeping panel-up unchanged so it faces
inward with upright, unmirrored text. The controller-relative center is
(0, 0.18, 0.089) m, approximating the compact HUD's surface center: its
0.12 m wrist lift, 0.09 m bottom anchor and ~0.03 m panel-center correction;
Z combines the fallback 0.054 m wrist calibration and 0.035 m finger-back offset. Wrist-mounted panels are fully visible while their entire
orientation is within 60° of an upright, viewer-facing panel; linear opacity
fade from 60° to 75°, then hidden (including laser interaction). Pitch, yaw and
roll contribute together; turning the wrist away or moving the head around it
changes the angle. OpenVR's overlay alpha changes without rerendering the panel.
World and head mounts do not fade. Missing headset tracking hides a wrist panel;
by default a lost wrist uses a world-space fallback. Headset readability,
fade feel and interaction at the threshold still need live acceptance.

To tune the selected wrist, set `wrist` in `config.json` as in the example above:
`x`, `y`, `z` are controller-local meters (each -0.3 to 0.3), `width` is panel
width in meters (0.15 to 0.6), and `roll_degrees` rotates about controller -Z
(-180 to 180) before applying the offset. These settings are initial defaults
for both wrists; a saved per-wrist pose takes precedence. They are read at
startup and do not alter world/head placement. Invalid values fail direct native
startup; the installer backs up and repairs invalid entries.
A missing/untracked selected wrist temporarily falls back to world space,
with a visible explanation in Settings, then reattaches when tracking returns.
Use Settings → **Wrist world fallback** to switch between the default
front-of-you fallback (ON) and hiding the panel when wrist tracking is lost
(OFF). It reappears on the selected wrist when tracking returns. The toggle is
saved to `"wrist_world_fallback"` in `config.json`; manual edits take effect on
restart. This option affects wrist mounts only, not World or Head. While hidden,
pointer controls cannot be used.
The saved mount preference is not replaced by the fallback. These offsets and sizes are
initial choices, **not headset-comfort acceptance**.

A selection saves the mount token to `$XDG_CONFIG_HOME/frameyap/mount`
(or `$HOME/.config/frameyap/mount`). Completed relative adjustments save to
`placement-left-wrist`, `placement-right-wrist`, or `placement-head` beside that
file. Remove one placement file while FrameYap is closed to restore that mount's
default. These are bounded, versioned device-relative transforms with owner-only
atomic writes; invalid files are ignored. World-space poses, audio and transcripts
are not saved. Missing/invalid mount settings default to World; write failure
keeps the placement for the session and displays a warning. `--mount
world|left-wrist|right-wrist|head` overrides the saved choice for one launch without
writing it; `--head` remains an alias for `--mount head`.

Settings also has **Lasers anytime** (default off). Open the dashboard to change
it when system-wide lasers are disabled. When enabled, FrameYap sets
OpenVR's `VROverlayFlags_MakeOverlaysInteractiveIfVisible` on its panel. OpenVR
requests system-wide laser mouse mode while the panel is visible, including
with Steam's dashboard closed; it may change interaction with games. Turning
it off removes that request. This is **not** the experimental overlay action
priority override and does not promise pass-through of a dashboard-owned
button or PTT delivery during dashboard focus. The choice is saved as `on` or
`off` in `$XDG_CONFIG_HOME/frameyap/lasers-anytime` (default
`~/.config/frameyap/lasers-anytime`) using a private atomic file replacement;
missing, symlinked or invalid files mean off. A failed save keeps the new
choice only for the running session and shows a warning. To turn it back on
after disabling it with the dashboard closed, open the dashboard to use its
laser on the Settings button. Controls-only checks can toggle it temporarily
but do not save the preference. The wearer reports that pointer clicks work,
while normal-priority controller actions become unavailable in system laser
mode, including when this preference is enabled. Experimental-priority
coexistence remains unverified.

### Hardware-free UI checks

The default build tests mount parsing, laser preference persistence, pose geometry,
controller-relative grab/captured-ray scale math and stick-depth integration without
any native dependencies.
Drag tests cover XYZ translation, pitch/yaw/roll and lever-arm rotation, re-grabbing
a moved panel, stationary stability, relative mounts, out-of-bounds scale hits,
invalid poses/rays and no feedback from prior updates. UI/config/installer tests
cover hidden/disabled handles, top-left mask regions, lock persistence and repair. A FreeType-only opt-in build exercises the actual renderer,
pointer gating, tab switches, pagination, recording state and redraw invalidation:

```sh
cmake -S . -B build-ui -DFRAMEYAP_UI_TESTS=ON
cmake --build build-ui
ctest --test-dir build-ui --output-on-failure
# Optional synthetic fixture previews; no OpenVR initialization:
./build-ui/frameyap_panel_test assets/fonts/Inconsolata-Regular.ttf /tmp/frameyap-ui
```

The last command writes `-review.ppm`, `-settings.ppm` and `-recording.ppm` to the
supplied prefix. The native build includes these tests too; tests never initialize
OpenVR or touch the real mounting preference. Physical pointing, tracking loss,
recentring and readability still require a separately authorized headset check.

The opt-in native `--check-controls` probe logs pointer counters and action
callbacks to the terminal rather than repainting them on the panel. Its canvas
stays static for Record/Cancel/Type/Type + Enter clicks so those clicks can be checked
without diagnostic texture uploads. Switching tabs or mount still updates
the visible panel. Diagnostics identify `renderer=Vulkan` and count
`textureUploads`; raw/file `ImageLoaded` events are not GPU upload completions.
The native CTest suite tests persistent image reuse, queued transfer ordering,
coherent/noncoherent staging memory and error cleanup against Vulkan fakes;
it does not initialize the Vulkan loader, a GPU or OpenVR for that test.

An optional offscreen check exercises the real Vulkan backend with eight
synthetic RGBA patterns, verifying exact readback and image reuse. It needs a
GPU/driver, is excluded from normal builds and CTest, and requires `--run`:

```sh
cmake --build build-native --target frameyap_texture_check
./build-native/frameyap_texture_check --run
```

It does not initialize OpenVR or establish compositor/headset acceptance.

`assets/actions.json` names nine actions: left/right grip, PTT, cancel,
Type, Type + Enter, Quick phrases and the two hand-specific depth axes. `insert`, `enter`, and `quick_chat` remain
internal binding keys; visible controls read Type, Type + Enter, Quick phrases.
`bindings_frame_controller.json` maps right X click to hold-to-talk PTT;
the grip bindings remain for optional remapping/diagnosis. In one dashboard
probe grips were inactive; a later controls-only probe delivered repeated right
X PTT BeginRecord/EndRecord callbacks. The wearer reports controller actions
are usable with Steam's dashboard closed, not with the dashboard itself open.
Neither probe used a microphone or established game-scene pass-through.
`bindings_knuckles.json` is an additional **Index/knuckles example only**;
it includes the two thumbstick depth axes alongside grip taps.
Collisions with scene actions require
separate on-device validation. Left grip double tap
(releases <=250 ms, second press within 350 ms) requests explicit Enter only
when enabled. Right grip: first short squeeze and release (<=250 ms), then
second squeeze **down** within 350 ms starts capture; hold as long as needed
(up to runtime's clip bound), second **release** ends capture. The named PTT
action explicitly begins on down and ends on up. On tracking-pose invalidity,
action inactivity or overlay focus loss, a held capture emits Cancel, and
reconnection requires a neutral observation before any new press. PTT and left
Type + Enter require an enabled panel; clickable Record remains available for retry
after an error and Cancel is always available. The action set defaults to normal
priority; the experimental config request is described above. Neither priority
guarantees delivery while a game or dashboard owns input.

The UI cannot itself guarantee a capture started when a BeginRecord action
arrives: the owning runtime checks worker readiness. `Cancel` invalidates
capture/worker work; a failed startup can be retried with Record. The
`UiAction::Toggle` enumerator remains for caller ABI compatibility but the
panel no longer emits it: Previous is navigation only. Pointer Record/Stop emits
explicit `BeginRecord`/`EndRecord`, just like PTT edges, so a stale Stop cannot
become a new recording when another input has already ended capture.

## Registration

`assets/application.vrmanifest.in` is a **template**, not a runnable manifest.
At install, substitute absolute executable and action JSON paths and store
it under the user-owned install directory, keeping bindings adjacent to action
JSON. Register `local.frameyap.overlay` using `registration(path,false,false)`;
autolaunch is opt-in. Unregister before deleting the installed manifest.
The app key is not a Steam store AppID. Registration uses OpenVR Utility init
only when explicitly invoked, then verifies `IsApplicationInstalled`. The device
required `binary_path_linux_arm` in the manifest (a generic or Linux-only path
was silently skipped despite successful AddApplicationManifest return).
Registration, repeated registration and removal were tested against the running
Frame runtime, with autolaunch verified off. The overlay explicitly identifies
its process with the registered app key before setting its action manifest.
Cold-runtime behavior and actual SteamVR-menu launch still need validation.
See [packaging](packaging.md) for the published native archive and release checks.

A live headset check must be opt-in and distinguish overlay API discovery from
controller delivery, actual transcription, insertion into a disposable target,
and human comfort/acceptance.

### Opt-in rendering measurements

`FRAMEYAP_PROFILE=1` prints aggregate CPU raster time, upload time, and grab/scale
tracking/transform time on exit. Counts and timings contain no speech, typed text
or pointer coordinates. The motion measurements exclude sleep and other runtime
work; they do not measure motion-to-photon latency. During an initialized grab or
scale, neither the renderer nor uploader runs; content updates resume on release.