Add safe transcription errors and opt-in private debug logging

This commit is contained in:
baketnk committed 2026-09-24 15:07:53 -04:00
1 parent 29f3729df7
commit 6592e1af1c
21 files changed
+720 -65

No files matched your search

+4
View File
@@ -39,6 +39,10 @@ See [third-party notes](docs/third-party.md). No GitHub release is published yet
panel colors, a font path and Frame controller button mappings; missing fonts
fall back to bundled Inconsolata. The installer creates/checks this file and
backs it up before repairs. See [overlay configuration](docs/overlay.md#user-theme-and-controller-configuration).
- **Advanced debugging:** Settings toggle / `"advanced_debug": true` in config.
Off by default. Restarts the worker and discards current work; full exceptions,
worker output and transcripts go to private, bounded local logs. No raw audio
archive. See [diagnostics](docs/worker.md#advanced-debugging).
- **Experimental input priority:** set `"input_priority": "experimental"` in
that config and enable SteamVR's Developer option **Enable global input from
overlays**. FrameYap then requests priority for its bound controller sources.
+1
View File
@@ -1,6 +1,7 @@
{
"font": "",
"input_priority": "normal",
"advanced_debug": false,
"wrist": {"x": 0, "y": 0.18, "z": 0.089, "width": 0.30, "roll_degrees": 0},
"theme": {
"background": "#0c101b",
+14 -1
View File
@@ -98,6 +98,7 @@ installer creates one with defaults on first install. Copy the shipped
{
"font": "/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
"input_priority": "normal",
"advanced_debug": false,
"wrist": {"x": 0, "y": 0.18, "z": 0.089, "width": 0.30, "roll_degrees": 0},
"theme": {
"background": "#0c101b", "card": "#141c2b", "ink": "#e6f0f9",
@@ -117,6 +118,17 @@ installer creates one with defaults on first install. Copy the shipped
Each theme color is `#RRGGBB`; omitted colors keep the default. `font` is a
TTF/OTF file path (not a family name); a missing file uses the bundled font.
`advanced_debug` is a boolean (default `false`, not a string): an opt-in
request for full diagnostic logs. Full logs may contain speech/transcribed text
and local paths; **raw audio clips are not archived**. The Settings tab shows
an Advanced debugging ON/OFF toggle and warns that changing it restarts the
worker and cancels current work (including pending review). Changes take effect
immediately; failed saves show a warning and keep the selection for this session.
Detailed logs are bounded and owner-private; see [worker diagnostics](worker.md#advanced-debugging). The native
`save_advanced_debug(path, bool)` helper updates only this value in a valid
config, retaining other fields and formatting; invalid/unwritable configs are
left untouched and return failure. The installer backs up original bytes before
repairing invalid values, while valid `true` and `false` are retained.
`buttons` maps named OpenVR actions (`left_grip`, `right_grip`, `ptt`, `cancel`,
`insert`, `enter`) to Frame physical `/user/hand/{left|right}/input/NAME`
button paths. Omitted actions retain their bundled defaults; an empty string
@@ -202,7 +214,8 @@ keeps the selection for the session and displays a warning. `--mount
world|left-wrist|right-wrist|head` overrides the saved choice for one launch without
writing it; `--head` remains an alias for `--mount head`.
Settings also has **Lasers anytime** (default off). When enabled, FrameYap sets
Settings also has **Lasers anytime** (default off). Open the dashboard to change
it when system-wide lasers are disabled. When enabled, FrameYap sets
OpenVR's `VROverlayFlags_MakeOverlaysInteractiveIfVisible` on its panel. OpenVR
requests system-wide laser mouse mode while the panel is visible, including
with Steam's dashboard closed; it may change interaction with games. Turning
+35 -6
View File
@@ -1,8 +1,8 @@
# Offline Redux worker adapter (component, not an installed product)
`src/worker.hpp` provides `frameyap::Worker`: call `start(python, script, model,
threads=2)` explicitly, poll until `ready()`, then `submit(id, pcm)` and poll for
one `WorkerReply` (text or generic per-request error). One request at a time;
threads=2, advanced_debug=false)` explicitly, poll until `ready()`, then `submit(id, pcm)` and poll for
one `WorkerReply` (text or privacy-safe per-request error). One request at a time;
no queue, no capture and no input injection. `stop()` discards pending audio,
terminates/reaps **only its direct child** (TERM, bounded 500 ms, then KILL),
and is safe to repeat. Destruction stops it. `start()` returns without waiting
@@ -19,16 +19,19 @@ and writes only `clip.raw` with `O_EXCL|O_NOFOLLOW`, mode 0600. Clips are
float32 (0.2..20 s). Files are unlinked after replies or shutdown, and the
private directory is removed. Private clips are not encrypted against the
account owner/root; do not use an untrusted runtime directory. The caller
should pass a trusted interpreter and script. Neither audio nor transcripts
are logged; child stderr is redirected to `/dev/null`, so worker diagnostics
are deliberately generic.
should pass a trusted interpreter and script. By default audio/transcripts are
not logged and child stderr is redirected to `/dev/null`. Request errors report
only a fixed stage (`audio`, `inference`, `response`) and built-in exception
category, never the exception message, arbitrary class name, traceback or text.
The native receiver allowlists those labels before displaying/logging them;
unknown/legacy errors remain generic.
The private pipes use unsigned LE32 payload lengths (1..65536), a one-byte
message type and, for requests/replies, unsigned LE64 request ID. `T` + ID
requests reading the fixed clip; `Y` means ready; `F` means load failure
(`M` for missing/mismatched pinned model or private clip directory, `I` for a
missing Python dependency, `D` for runtime/model load failure); `R` + ID + UTF-8
text and `E` + ID + generic UTF-8 error are replies. Text is at most 4096
text and `E` + ID + privacy-safe UTF-8 error are replies. Text is at most 4096
bytes. An unexpected or duplicate reply, wrong ID, extra frame, closed pipe
or oversized frame stops the worker. Warmup deadline is 120 s, transcription
deadline 60 s; `poll()` must be called regularly to enforce deadlines. It
@@ -58,6 +61,32 @@ or bundling. See [third-party notes](third-party.md). No public runtime bundle h
been released. Limited ARM64 measurements are in the [POC record](evidence/poc-cpu-overlay-2026-09-24.md),
not a claim of complete headset acceptance.
## Advanced debugging
Explicitly set `"advanced_debug": true` in config or enable Settings → Advanced
debugging. It is off by default. A Settings change restarts the owned worker,
closes the microphone and discards current work/review; the replacement worker
warms normally. Turning it off stops detailed capture but **does not delete
previous logs**. Manual config edits take effect on application restart.
With this opt-in the worker receives `--advanced-debug`: native/model stdout and
stderr, sample counts, full exception tracebacks and recognized transcripts are
captured. **These logs may contain private speech, transcript text and local
paths. Inspect/redact before sharing; keep out of Git.** No raw audio archive is
created; normal temporary clips still expire after replies/cancellation.
Files: `$XDG_STATE_HOME/frameyap/worker-debug.log` (fallback
`~/.local/state/frameyap/worker-debug.log`) and `worker-debug.previous.log`.
Each debug worker start rotates the current log once; only these two files are
retained, each bounded to 4 MiB. At the limit a marker is written and further
output is drained/discarded for that worker session, not allowed to block it.
The directory is owner-private 0700 and logs are 0600. Unsafe paths, symlinks,
hardlinks or preexisting permissive files are refused with a visible error;
there is no fallback to public temporary files. The app's single-instance lock
is required to avoid competing rotation by multiple workers. Debug output never
shares the framed protocol stdout. Direct worker CLI use with `--advanced-debug`
writes to its caller's stderr; the native adapter supplies the private bounded sink.
Hardware-free tests run through CTest, including fake-child cancellation, short
injected warmup/request deadlines, duplicate/stale replies, malformed frames,
missing/hash-mismatched model files and symlink refusal. Default production
+3
View File
@@ -34,6 +34,7 @@ DIGEST_RE = re.compile(r"[a-fA-F0-9]{64}\Z")
CONFIG_DEFAULTS = {
"font": "",
"input_priority": "normal",
"advanced_debug": False,
"wrist": {"x": 0, "y": 0.18, "z": 0.089, "width": 0.30, "roll_degrees": 0},
"theme": {"background": "#0c101b", "card": "#141c2b", "ink": "#e6f0f9",
"muted": "#97adc1", "accent": "#1ff0a4", "warning": "#ff6e87",
@@ -92,6 +93,8 @@ def normalized_config(data):
fixed = {"font": data.get("font") if isinstance(data.get("font"), str) else ""}
priority = data.get("input_priority", "normal")
fixed["input_priority"] = priority if priority in ("normal", "experimental") else "normal"
debug = data.get("advanced_debug", False)
fixed["advanced_debug"] = debug if type(debug) is bool else False
source = data.get("wrist")
source = source if isinstance(source, dict) else {}
fixed["wrist"] = {}
+49 -12
View File
@@ -4,7 +4,7 @@ Wire protocol: unsigned LE32 payload length (max 65536), type byte, unsigned
LE64 request id for T/R/E; Y (ready) and F (load failure) have no id. T refers
to the fixed clip.raw in the private directory. R/E carry <=4096 UTF-8 bytes.
Only one T may be outstanding. No stdout other than frames; stderr is suppressed
by the native launcher. This module never captures audio or delivers input.
by default, or captured privately with explicit --advanced-debug. This module never captures audio or delivers input.
"""
import argparse
@@ -14,6 +14,7 @@ from pathlib import Path
import stat
import struct
import sys
import traceback
MAX_FRAME = 65536
MAX_TEXT = 4096
@@ -131,7 +132,19 @@ def read_clip(directory):
return pcm
def run(model, directory, input_fd=0, output_fd=1):
def safe_request_error(stage, error):
"""Only fixed labels: never stringify exceptions, paths, audio or transcripts."""
if stage not in ("audio", "inference", "response"):
stage = "unknown"
# Use built-in categories, not an arbitrary exception class name supplied by
# a model/runtime. Subclasses are reduced to their safe built-in category.
categories = (MemoryError, ImportError, OSError, UnicodeError, TypeError,
ValueError, KeyError, IndexError, RuntimeError)
category = next((kind.__name__ for kind in categories if isinstance(error, kind)), "Exception")
return f"transcription failed [{stage}: {category}]".encode("ascii")
def run(model, directory, input_fd=0, output_fd=1, advanced_debug=False):
private_dir(directory)
send_frame(output_fd, b"Y")
while True:
@@ -141,19 +154,30 @@ def run(model, directory, input_fd=0, output_fd=1):
if len(frame) != 9 or frame[0:1] != b"T":
raise ValueError("invalid request")
request_id = frame[1:9]
stage = "audio"
try:
result = model.transcribe(audio=read_clip(directory), sample_rate=16000)
audio = read_clip(directory)
if advanced_debug:
print(f"request {int.from_bytes(request_id, 'little')}: audio samples={len(audio)} rate=16000", file=sys.stderr, flush=True)
stage = "inference"
result = model.transcribe(audio=audio, sample_rate=16000)
stage = "response"
text = result["text"]
if not isinstance(text, str):
raise ValueError("invalid model response")
raise TypeError("invalid model response")
encoded = text.encode("utf-8", errors="strict")
if len(encoded) > MAX_TEXT:
raise ValueError("transcript exceeds 4096 bytes")
if advanced_debug:
print(f"request {int.from_bytes(request_id, 'little')}: transcript={text!r}", file=sys.stderr, flush=True)
send_frame(output_fd, b"R", request_id + encoded)
except Exception:
# Model exceptions may include audio or transcripts. Do not log or
# forward them to the overlay; request-local failure only.
send_frame(output_fd, b"E", request_id + b"transcription failed")
except Exception as error:
if advanced_debug:
print(f"request {int.from_bytes(request_id, 'little')}: failure stage={stage}", file=sys.stderr, flush=True)
traceback.print_exc(file=sys.stderr)
# Only fixed stage/category labels cross IPC. Exception messages and
# tracebacks may contain private audio/text and remain suppressed.
send_frame(output_fd, b"E", request_id + safe_request_error(stage, error))
def main(argv=None):
@@ -161,6 +185,8 @@ def main(argv=None):
parser.add_argument("--model", required=True)
parser.add_argument("--threads", type=int, default=2)
parser.add_argument("--clip-dir", required=True)
parser.add_argument("--advanced-debug", action="store_true",
help="log full exceptions/runtime output and transcripts to stderr; may contain private speech")
args = parser.parse_args(argv)
if not 1 <= args.threads <= 64:
parser.error("threads must be 1..64")
@@ -168,9 +194,15 @@ def main(argv=None):
# the framed channel, not merely Python's sys.stdout wrapper.
protocol_fd = os.dup(1)
os.set_inheritable(protocol_fd, False)
with open(os.devnull, "wb") as null:
os.dup2(null.fileno(), 1)
os.dup2(null.fileno(), 2)
if args.advanced_debug:
# Native parent supplies a bounded private diagnostic sink. Model/native
# stdout must still never corrupt the duplicated framed protocol fd.
os.dup2(2, 1)
print("FrameYap advanced debugging ON: private speech/text/paths may be logged; no raw clip archive.", file=sys.stderr, flush=True)
else:
with open(os.devnull, "wb") as null:
os.dup2(null.fileno(), 1)
os.dup2(null.fileno(), 2)
try:
try:
private_dir(args.clip_dir)
@@ -180,15 +212,20 @@ def main(argv=None):
try:
model = load_model(args.model, args.threads)
except LocalModelError:
if args.advanced_debug: traceback.print_exc(file=sys.stderr)
send_frame(protocol_fd, b"F", b"M")
return 1
except ImportError:
if args.advanced_debug: traceback.print_exc(file=sys.stderr)
send_frame(protocol_fd, b"F", b"I")
return 1
except Exception:
if args.advanced_debug: traceback.print_exc(file=sys.stderr)
send_frame(protocol_fd, b"F", b"D")
return 1
run(model, args.clip_dir, output_fd=protocol_fd)
if args.advanced_debug:
print("Local model ready", file=sys.stderr, flush=True)
run(model, args.clip_dir, output_fd=protocol_fd, advanced_debug=args.advanced_debug)
return 0
finally:
os.close(protocol_fd)
+3
View File
@@ -23,6 +23,7 @@ DIGEST_RE = re.compile(r"[a-fA-F0-9]{64}\Z")
CONFIG_DEFAULTS = {
"font": "",
"input_priority": "normal",
"advanced_debug": False,
"wrist": {"x": 0, "y": 0.18, "z": 0.089, "width": 0.30, "roll_degrees": 0},
"theme": {"background": "#0c101b", "card": "#141c2b", "ink": "#e6f0f9",
"muted": "#97adc1", "accent": "#1ff0a4", "warning": "#ff6e87",
@@ -81,6 +82,8 @@ def normalized_config(data):
fixed = {"font": data.get("font") if isinstance(data.get("font"), str) else ""}
priority = data.get("input_priority", "normal")
fixed["input_priority"] = priority if priority in ("normal", "experimental") else "normal"
debug = data.get("advanced_debug", False)
fixed["advanced_debug"] = debug if type(debug) is bool else False
source = data.get("wrist")
source = source if isinstance(source, dict) else {}
fixed["wrist"] = {}
+52 -9
View File
@@ -13,7 +13,8 @@ namespace {
struct Json {
std::string value;
std::map<std::string, Json> object;
bool is_string = false, is_object = false, is_number = false;
bool is_string = false, is_object = false, is_number = false, is_bool = false;
size_t start = 0, end = 0; // original value span for non-destructive config updates
};
struct Parser {
std::string_view s;
@@ -70,23 +71,25 @@ struct Parser {
ws();
if (pos == s.size()) fail();
Json result;
if (s[pos] == '"') { result.value = str(); result.is_string = true; return result; }
result.start = pos;
auto done = [&]() { result.end = pos; return result; };
if (s[pos] == '"') { result.value = str(); result.is_string = true; return done(); }
if (eat('{')) {
result.is_object = true;
if (eat('}')) return result;
if (eat('}')) return done();
do {
ws(); if (pos == s.size() || s[pos] != '"') fail();
auto key = str();
if (!eat(':')) fail();
auto [it, inserted] = result.object.emplace(std::move(key), parse(depth + 1));
if (!inserted) fail();
if (eat('}')) return result;
if (eat('}')) return done();
} while (eat(','));
fail();
}
if (eat('[')) {
if (eat(']')) return result;
do { parse(depth + 1); if (eat(']')) return result; } while (eat(','));
if (eat(']')) return done();
do { parse(depth + 1); if (eat(']')) return done(); } while (eat(','));
fail();
}
size_t start = pos;
@@ -105,11 +108,15 @@ struct Parser {
}
}
if (pos == start) fail();
if (s.substr(start, pos - start) == "true" || s.substr(start, pos - start) == "false") {
result.is_bool = true;
result.value = s.substr(start, pos - start);
}
if (s[start] == '-' || (s[start] >= '0' && s[start] <= '9')) {
result.is_number = true;
result.value = s.substr(start, pos - start);
}
return result;
return done();
}
};
Rgba color(const Json& json) {
@@ -153,14 +160,16 @@ std::string read_file(const std::filesystem::path& p) {
data.assign(buf, size_t(in.gcount()));
return data;
}
void write_file(const std::filesystem::path& path, const std::string& content) {
void write_file(const std::filesystem::path& path, const std::string& content, bool private_file = false) {
auto temporary = path.string() + ".tmp." + std::to_string(std::chrono::steady_clock::now().time_since_epoch().count());
try {
{
std::ofstream out(temporary, std::ios::binary | std::ios::trunc);
out << content;
if (!out) throw std::runtime_error("Could not write generated OpenVR binding: " + path.string());
if (!out) throw std::runtime_error("Could not write file: " + path.string());
}
if (private_file) std::filesystem::permissions(temporary, std::filesystem::perms::owner_read |
std::filesystem::perms::owner_write, std::filesystem::perm_options::replace);
std::filesystem::rename(temporary, path);
} catch (...) {
std::error_code ignored;
@@ -185,6 +194,9 @@ Config load_config(const std::filesystem::path& path) {
if (key == "font") {
if (!value.is_string) throw std::runtime_error("Config font must be a path string");
config.font = value.value;
} else if (key == "advanced_debug") {
if (!value.is_bool) throw std::runtime_error("Config advanced_debug must be a boolean");
config.advanced_debug = value.value == "true";
} else if (key == "input_priority") {
if (!value.is_string || (value.value != "normal" && value.value != "experimental"))
throw std::runtime_error("Config input_priority must be normal or experimental");
@@ -238,6 +250,37 @@ Config load_config(const std::filesystem::path& path) {
}
return config;
}
bool save_advanced_debug(const std::filesystem::path& path, bool enabled) noexcept {
try {
if (path.empty() || !path.is_absolute() || std::filesystem::is_symlink(path)) return false;
const bool existing = std::filesystem::exists(path);
if (existing && !std::filesystem::is_regular_file(path)) return false;
if (existing) load_config(path); // reject invalid/unknown settings rather than erase customizations
std::string bytes = existing ? read_file(path) : "{}\n";
Parser parser{bytes};
auto root = parser.parse(); parser.ws();
if (!root.is_object || parser.pos != bytes.size()) return false;
const std::string value = enabled ? "true" : "false";
auto it = root.object.find("advanced_debug");
if (it != root.object.end()) {
if (!it->second.is_bool) return false;
if (it->second.value == value) return true;
bytes.replace(it->second.start, it->second.end - it->second.start, value);
} else {
bytes.insert(root.end - 1, std::string(root.object.empty() ? "" : ",") +
"\"advanced_debug\":" + value);
}
// The native reader rejects files >=4097 bytes, even if the JSON is valid.
if (bytes.size() > 4096) return false;
const auto parent = path.parent_path();
if (std::filesystem::is_symlink(parent)) return false;
std::filesystem::create_directories(parent);
write_file(path, bytes, true);
return true;
} catch (...) {
return false;
}
}
std::string resolve_font(const std::string& assets, const std::string& requested) {
const auto bundled = std::filesystem::path(assets) / "fonts/Inconsolata-Regular.ttf";
const std::array<std::filesystem::path, 4> candidates{requested, bundled,
+4
View File
@@ -18,12 +18,16 @@ struct Config {
std::string font; // absolute TTF/OTF path; empty uses the bundled face
// Requests OpenVR's experimental global action priority; SteamVR must allow it too.
bool experimental_input_priority = false;
bool advanced_debug = false; // opt-in full diagnostic logging; never raw audio recording
WristPlacement wrist;
// OpenVR action name -> physical Frame controller input path; empty disables it.
std::map<std::string, std::string> buttons;
};
std::filesystem::path default_config_path();
Config load_config(const std::filesystem::path& path);
// Update only advanced_debug in an existing valid config; false on invalid/unwritable paths.
// Other user customizations and formatting are retained; creates a minimal config if absent.
bool save_advanced_debug(const std::filesystem::path& path, bool enabled) noexcept;
std::string resolve_font(const std::string& assets, const std::string& requested);
// No writes or OpenVR access when no custom button mappings are specified.
// When customized, build a generated manifest and bindings under XDG cache.
+15 -3
View File
@@ -74,7 +74,7 @@ struct Overlay::Impl {
Config config;
PanelSurface surface;
Panel panel;
bool save_failed = false;
bool save_failed = false, debug_save_failed = false;
bool world_ready = false, placed = false, has_texture = false, shown = false;
vr::HmdMatrix34_t world_transform{};
std::optional<Mount> applied_mount;
@@ -151,6 +151,7 @@ struct Overlay::Impl {
laser_change_failed = true;
}
surface.set_lasers_anytime(lasers_anytime);
surface.set_advanced_debug(config.advanced_debug);
overlay_check(overlay->SetOverlayFlag(handle, vr::VROverlayFlags_VisibleInDashboard, true), overlay, "VisibleInDashboard");
vr::HmdVector2_t mouse_scale{{float(W), float(H)}};
overlay_check(overlay->SetOverlayMouseScale(handle, &mouse_scale), overlay, "SetOverlayMouseScale");
@@ -190,7 +191,8 @@ struct Overlay::Impl {
}
if (effective == Mount::World && applied_mount && *applied_mount != Mount::World)
world_ready = false; // a fresh world fallback near the wearer, not an old room location
std::string note = laser_change_failed ? "SteamVR declined the laser mode change." :
std::string note = debug_save_failed ? "Debug preference not saved; using it only for this session." :
laser_change_failed ? "SteamVR declined the laser mode change." :
save_failed ? "Preference could not be saved; using it for this session." : "";
if (effective != mount) note = save_failed ? "Wrist untracked; world fallback. Preference not saved." :
"Wrist not tracked - using world space until it returns.";
@@ -371,8 +373,17 @@ struct Overlay::Impl {
last_pointer_event = "up button=" + std::to_string(event.data.mouse.button);
if (event.data.mouse.button == vr::VRMouseButton_Left) {
auto event_result = surface.pointer_up(event.data.mouse.cursorIndex, event.data.mouse.x, H - event.data.mouse.y);
if (event_result.action || event_result.mount || event_result.recenter || event_result.lasers_anytime || event_result.open_bindings) ++pointer_actions;
if (event_result.action || event_result.mount || event_result.recenter || event_result.lasers_anytime || event_result.open_bindings || event_result.advanced_debug) ++pointer_actions;
if (event_result.action) result.push_back(*event_result.action);
if (event_result.advanced_debug) {
config.advanced_debug = *event_result.advanced_debug;
debug_save_failed = persist_mount && !save_advanced_debug(default_config_path(), config.advanced_debug);
surface.set_advanced_debug(config.advanced_debug);
reset_input(result);
// Runtime observes the change before handling this batch,
// restarts its worker and invalidates pending work/actions.
return result;
}
if (event_result.open_bindings) {
// The editor changes input ownership. Invalidate held gestures
// and pointer presses; never turn the returning release into input.
@@ -456,6 +467,7 @@ Overlay::Overlay(const std::string& assets, const std::string& font, std::option
Overlay::~Overlay() = default;
std::vector<UiAction> Overlay::poll() { return impl_->poll(); }
void Overlay::draw(const Panel& panel) { impl_->draw(panel); }
bool Overlay::advanced_debug() const { return impl_->config.advanced_debug; }
std::string Overlay::controls_status() {
// Compare the same actions across modes, before and after our pose/role gate.
// IsInputAvailable and a successful UpdateActionState are not delivery proof.
+1
View File
@@ -24,6 +24,7 @@ public:
Overlay& operator=(const Overlay&) = delete;
std::vector<UiAction> poll();
void draw(const Panel& panel);
bool advanced_debug() const;
std::string controls_status(); // diagnostic only, no input delivery
std::string pointer_status() const; // diagnostic counters, no input delivery
private:
+24 -12
View File
@@ -26,10 +26,10 @@ struct Rect {
}
};
enum class Control { Review, Settings, Bindings, OpenBindings, Prev, Next, Record, Cancel, Insert, Enter, Quit,
World, Left, Right, Head, Recenter, LasersAnytime };
World, Left, Right, Head, Recenter, LasersAnytime, AdvancedDebug };
enum class Tab { Review, Settings, Bindings };
struct Button { Rect r; Control id; const char* label; };
constexpr std::array<Button, 17> buttons{{
constexpr std::array<Button, 18> buttons{{
{{32, 138, 180, 46}, Control::Review, "Review"},
{{226, 138, 180, 46}, Control::Settings, "Settings"},
{{420, 138, 180, 46}, Control::Bindings, "Bindings"},
@@ -47,6 +47,7 @@ constexpr std::array<Button, 17> buttons{{
{{514, 308, 454, 58}, Control::Right, "Right wrist"},
{{32, 394, 300, 50}, Control::Recenter, "Recenter in front"},
{{514, 394, 454, 50}, Control::LasersAnytime, "Lasers anytime"},
{{32, 452, 936, 48}, Control::AdvancedDebug, "Advanced debugging (full logs)"},
}};
std::optional<UiAction> action(Control c) {
switch (c) {
@@ -105,7 +106,7 @@ struct PanelSurface::Impl {
Theme theme;
Color background, card, ink, muted, cyan, pink;
Tab tab = Tab::Review;
bool dirty = true, lasers_anytime = false;
bool dirty = true, lasers_anytime = false, advanced_debug = false;
std::string placement_note, binding_note;
std::array<std::string, 6> bindings{};
std::array<int, 2> pressed{{-1, -1}};
@@ -244,7 +245,8 @@ struct PanelSurface::Impl {
}
bool visible(Control c) const {
if (c == Control::Prev || c == Control::Next) return tab == Tab::Review;
if (mounting(c) || c == Control::Recenter || c == Control::LasersAnytime) return tab == Tab::Settings;
if (mounting(c) || c == Control::Recenter || c == Control::LasersAnytime ||
c == Control::AdvancedDebug) return tab == Tab::Settings;
if (c == Control::OpenBindings) return tab == Tab::Bindings;
return true;
}
@@ -316,10 +318,9 @@ struct PanelSurface::Impl {
32, 539, 21, muted, 968);
} else {
text("MOUNT AND INTERACTION", 32, 224, 22, muted, 968);
text("Lasers anytime enables system-wide laser mode while this panel is visible.", 32, 472, 20, muted, 968);
text(placement_note.empty() ? "May affect games. Default off; changes saved on this device." : placement_note,
32, 509, 22, muted, 968);
text("If off, open the dashboard to click this setting again.", 32, 538, 20, muted, 968);
text("Full logs may contain speech/text/paths. No saved audio clips.", 32, 520, 20, pink, 968);
text(placement_note.empty() ? "Toggle restarts worker; cancels current work. Lasers may affect games." : placement_note,
32, 540, 20, muted, 968);
}
rect({32, 550, 936, 1}, mix(card, cyan, .17f));
for (size_t i = 0; i < buttons.size(); ++i) {
@@ -330,7 +331,8 @@ struct PanelSurface::Impl {
(b.id == Control::Settings && tab == Tab::Settings) ||
(b.id == Control::Bindings && tab == Tab::Bindings) ||
(mounting(b.id) && *mounting(b.id) == mount) ||
(b.id == Control::LasersAnytime && lasers_anytime);
(b.id == Control::LasersAnytime && lasers_anytime) ||
(b.id == Control::AdvancedDebug && advanced_debug);
const Color fill = !on ? mix(background, card, .40f) :
selected ? mix(card, cyan, .14f) : card;
const Color accent = b.id == Control::Record && panel.recording ? pink : cyan;
@@ -342,9 +344,11 @@ struct PanelSurface::Impl {
const auto label = b.id == Control::Record && panel.recording ? "Stop" : b.label;
text(label, b.r.x + 16, b.r.y + b.r.h / 2 + 9, 27, on ? ink : mix(background, muted, .48f), b.r.x + b.r.w - 8);
if (mounting(b.id) && selected) text("ON", b.r.x + b.r.w - 56, b.r.y + 38, 23, cyan, b.r.x + b.r.w - 12);
if (b.id == Control::LasersAnytime)
text(lasers_anytime ? "ON" : "OFF", b.r.x + b.r.w - 66, b.r.y + 34, 22,
lasers_anytime ? cyan : muted, b.r.x + b.r.w - 12);
if (b.id == Control::LasersAnytime || b.id == Control::AdvancedDebug) {
bool active = b.id == Control::LasersAnytime ? lasers_anytime : advanced_debug;
text(active ? "ON" : "OFF", b.r.x + b.r.w - 66, b.r.y + b.r.h / 2 + 9, 22,
active ? cyan : muted, b.r.x + b.r.w - 12);
}
}
dirty = false;
return true;
@@ -375,6 +379,7 @@ SurfaceEvent PanelSurface::pointer_up(unsigned cursor, float x, float y) {
else if (auto m = mounting(c)) { impl_->mount = *m; result.mount = *m; impl_->reset(); impl_->dirty = true; }
else if (c == Control::Recenter) { result.recenter = true; impl_->reset(); }
else if (c == Control::LasersAnytime) result.lasers_anytime = !impl_->lasers_anytime;
else if (c == Control::AdvancedDebug) result.advanced_debug = !impl_->advanced_debug;
else if (c == Control::OpenBindings) { result.open_bindings = true; impl_->reset(); }
else if (c == Control::Review || c == Control::Settings || c == Control::Bindings) {
impl_->tab = c == Control::Review ? Tab::Review : c == Control::Settings ? Tab::Settings : Tab::Bindings;
@@ -402,4 +407,11 @@ void PanelSurface::set_lasers_anytime(bool enabled) {
impl_->dirty = true;
}
}
void PanelSurface::set_advanced_debug(bool enabled) {
if (impl_->advanced_debug != enabled) {
impl_->advanced_debug = enabled;
impl_->reset();
impl_->dirty = true;
}
}
} // namespace frameyap
+2
View File
@@ -12,6 +12,7 @@ struct SurfaceEvent {
std::optional<UiAction> action;
std::optional<Mount> mount;
std::optional<bool> lasers_anytime;
std::optional<bool> advanced_debug;
bool recenter = false;
bool open_bindings = false;
};
@@ -32,6 +33,7 @@ public:
void reset_pointers();
void set_placement_note(std::string note);
void set_lasers_anytime(bool enabled);
void set_advanced_debug(bool enabled);
// PTT, Cancel, Insert, Enter, left-grip gesture, right-grip gesture.
void set_bindings(std::array<std::string, 6> labels);
void set_binding_note(std::string note);
+16 -2
View File
@@ -11,6 +11,7 @@
#include <cstdlib>
#include <filesystem>
#include <fcntl.h>
#include <iostream>
#include <stdexcept>
#include <sys/file.h>
#include <sys/stat.h>
@@ -54,6 +55,7 @@ int run(const Options& options) {
Session session;
std::string detail = "Review mode. Other apps may also hear your mic. Enter is explicit.";
bool quit = false;
bool advanced_debug = overlay.advanced_debug();
const DeliveryFactory acquire = [&]() -> std::unique_ptr<DeliveryLease> {
auto input = std::make_unique<NativeDeliveryLease>(options.socket);
if (interrupted) throw std::runtime_error("Input cancelled before delivery");
@@ -77,7 +79,7 @@ int run(const Options& options) {
};
auto warm = [&] {
audio.close(); worker.stop(); session = Session{};
worker.start(options.python, options.worker, options.model, options.threads);
worker.start(options.python, options.worker, options.model, options.threads, advanced_debug);
detail = "Loading local model; microphone closed. Record again when Ready.";
};
auto stop_record = [&] {
@@ -108,6 +110,9 @@ int run(const Options& options) {
// E is a request-local error. The child still owns its
// loaded model and can accept the next utterance.
session.fail(); detail = reply->error + "; model ready. Record to retry.";
// Worker::poll allows only fixed diagnostic labels here,
// never exception messages, audio, paths or recognized text.
std::cerr << "FrameYap worker: " << reply->error << '\n';
} else {
session.reply(reply->id, reply->text);
detail = session.text().empty() ? "No speech recognized; try again." : "Focus your destination, then Insert. Cancel discards.";
@@ -139,7 +144,16 @@ int run(const Options& options) {
session.state() != State::Warming && session.state() != State::Transcribing && session.state() != State::Review};
if (panel.recording) panel.status += " - " + std::to_string(audio.seconds()) + " / 20s";
overlay.draw(panel);
for (auto action : overlay.poll()) {
auto actions = overlay.poll();
if (advanced_debug != overlay.advanced_debug()) {
advanced_debug = overlay.advanced_debug();
// Consent changes take effect before any more work or delivery. The
// Settings warning makes the worker restart/cancellation explicit.
try { warm(); }
catch (const std::exception& e) { session.fail(); detail = e.what(); }
std::erase_if(actions, [](UiAction action) { return action != UiAction::Quit; });
}
for (auto action : actions) {
try {
switch (action) {
case UiAction::Quit: quit = true; break;
+162 -13
View File
@@ -1,5 +1,6 @@
#include "worker.hpp"
#include <algorithm>
#include <array>
#include <cerrno>
#include <chrono>
@@ -26,6 +27,8 @@ namespace {
using Clock = std::chrono::steady_clock;
constexpr size_t max_frame = 65536;
constexpr size_t max_text = 4096;
constexpr size_t max_debug_log = 4 * 1024 * 1024;
constexpr std::string_view truncation = "\n[worker diagnostics truncated; further output discarded]\n";
void close_fd(int& fd) { if (fd >= 0) { ::close(fd); fd = -1; } }
void put32(unsigned char* p, uint32_t n) {
@@ -62,6 +65,102 @@ bool valid_utf8(std::string_view text) {
}
return true;
}
std::string safe_request_error(std::string_view error) {
// Treat E payloads as untrusted too: older/alternate workers might include
// raw exception messages. Only fixed diagnostic labels reach UI or logs.
for (const auto* stage : {"audio", "inference", "response", "unknown"}) {
for (const auto* category : {"MemoryError", "ImportError", "OSError", "UnicodeError", "TypeError",
"ValueError", "KeyError", "IndexError", "RuntimeError", "Exception"}) {
auto allowed = std::string("transcription failed [") + stage + ": " + category + "]";
if (error == allowed) return allowed;
}
}
return "transcription failed"; // compatible legacy/unknown error, never raw text
}
// Walk from / using directory fds: no symlinks, even in intermediate components.
// Root-owned system ancestors (including sticky /tmp) are permitted, but a
// writable non-sticky ancestor could redirect private diagnostics elsewhere.
int state_directory() {
const char* base = ::getenv("XDG_STATE_HOME");
std::string path;
if (base && *base) path = base;
else {
const char* home = ::getenv("HOME");
if (!home || !*home) throw std::runtime_error("advanced debug needs HOME or XDG_STATE_HOME");
path = std::string(home) + "/.local/state";
}
if (path.empty() || path[0] != '/')
throw std::runtime_error("advanced debug state path must be absolute");
while (path.size() > 1 && path.back() == '/') path.pop_back();
path += path == "/" ? "frameyap" : "/frameyap";
int dir = ::open("/", O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
if (dir < 0) throw std::runtime_error("cannot open debug state root");
try {
size_t pos = 1;
while (pos < path.size()) {
auto end = path.find('/', pos);
if (end == std::string::npos) end = path.size();
auto part = path.substr(pos, end - pos);
if (part.empty() || part == "." || part == "..")
throw std::runtime_error("unsafe debug state path");
if (::mkdirat(dir, part.c_str(), 0700) && errno != EEXIST)
throw std::runtime_error("cannot create private debug state directory");
int next = ::openat(dir, part.c_str(), O_RDONLY | O_DIRECTORY | O_NOFOLLOW | O_CLOEXEC);
if (next < 0) throw std::runtime_error("unsafe debug state directory");
struct stat st{};
bool final = end == path.size();
if (::fstat(next, &st) || !S_ISDIR(st.st_mode) ||
(final && (st.st_uid != ::geteuid() || (st.st_mode & 07777) != 0700)) ||
(!final && ((st.st_uid != ::geteuid() && st.st_uid != 0) ||
((st.st_mode & 0022) && !(st.st_uid == 0 && (st.st_mode & S_ISVTX)))))) {
::close(next);
throw std::runtime_error("unsafe debug state directory permissions");
}
::close(dir); dir = next;
pos = end + 1;
}
return dir;
} catch (...) { ::close(dir); throw; }
}
// Open existing files without following links; never rotate someone else's file,
// a hardlink or a permissive target. Both names are checked before any mutation.
void check_log_target(int dir, const char* name) {
int fd = ::openat(dir, name, O_RDONLY | O_NOFOLLOW | O_NONBLOCK | O_CLOEXEC);
if (fd < 0) {
if (errno == ENOENT) return;
throw std::runtime_error("unsafe existing worker debug log");
}
struct stat st{};
bool valid = ::fstat(fd, &st) == 0 && S_ISREG(st.st_mode) &&
st.st_uid == ::geteuid() && (st.st_mode & 07777) == 0600 &&
st.st_nlink == 1 && st.st_size <= static_cast<off_t>(max_debug_log);
::close(fd);
if (!valid) throw std::runtime_error("unsafe existing worker debug log");
}
int create_debug_log() {
int dir = state_directory();
try {
check_log_target(dir, "worker-debug.log");
check_log_target(dir, "worker-debug.previous.log");
if (::unlinkat(dir, "worker-debug.previous.log", 0) && errno != ENOENT)
throw std::runtime_error("cannot rotate worker debug log");
if (::renameat(dir, "worker-debug.log", dir, "worker-debug.previous.log") && errno != ENOENT)
throw std::runtime_error("cannot rotate worker debug log");
int fd = ::openat(dir, "worker-debug.log", O_WRONLY | O_CREAT | O_EXCL | O_NOFOLLOW | O_CLOEXEC, 0600);
if (fd < 0) throw std::runtime_error("cannot create private worker debug log");
::close(dir);
return fd;
} catch (...) { ::close(dir); throw; }
}
void log_bytes(int fd, const unsigned char* data, size_t size) {
while (size) {
ssize_t n = ::write(fd, data, size);
if (n < 0 && errno == EINTR) continue;
if (n <= 0) throw std::runtime_error("worker debug log write failed");
data += n; size -= static_cast<size_t>(n);
}
}
void check_runtime(const char* path) {
if (!path || path[0] != '/') throw std::runtime_error("XDG_RUNTIME_DIR must be an absolute private directory");
struct stat st{};
@@ -97,13 +196,45 @@ void write_all(int fd, const unsigned char* data, size_t size) {
struct Worker::State {
pid_t pid = -1;
int to_child = -1, from_child = -1;
int to_child = -1, from_child = -1, debug_pipe = -1, debug_file = -1;
size_t debug_written = 0;
bool debug_truncated = false;
std::string dir;
bool loaded = false;
std::optional<uint64_t> pending;
Clock::time_point deadline{};
std::vector<unsigned char> input;
std::chrono::milliseconds warmup_timeout{120000}, request_timeout{60000};
void drain_debug() {
if (debug_pipe < 0) return;
// Limit work per poll: even an endlessly noisy child cannot trap the UI.
size_t budget = 256 * 1024;
unsigned char block[4096];
while (budget) {
ssize_t n = ::read(debug_pipe, block, std::min(budget, sizeof block));
if (n > 0) {
budget -= static_cast<size_t>(n);
if (!debug_truncated) {
size_t left = max_debug_log - truncation.size() - debug_written;
size_t count = std::min(left, static_cast<size_t>(n));
log_bytes(debug_file, block, count);
debug_written += count;
if (count < static_cast<size_t>(n)) {
log_bytes(debug_file, reinterpret_cast<const unsigned char*>(truncation.data()), truncation.size());
debug_written += truncation.size();
debug_truncated = true;
}
}
continue;
}
if (n == 0) { close_fd(debug_pipe); return; }
if (errno == EINTR) continue;
if (errno != EAGAIN && errno != EWOULDBLOCK)
throw std::runtime_error("worker debug pipe read failed");
return;
}
}
};
Worker::Worker(std::chrono::milliseconds warmup, std::chrono::milliseconds request)
@@ -122,14 +253,20 @@ void Worker::stop() {
::kill(s.pid, SIGTERM); // Only our direct child, never a process group.
auto until = Clock::now() + std::chrono::milliseconds(500);
int status = 0;
while (::waitpid(s.pid, &status, WNOHANG) == 0 && Clock::now() < until)
while (::waitpid(s.pid, &status, WNOHANG) == 0 && Clock::now() < until) {
try { s.drain_debug(); } catch (...) {} // stop/destructor are best effort
::usleep(10000);
}
if (::waitpid(s.pid, &status, WNOHANG) == 0) {
::kill(s.pid, SIGKILL);
while (::waitpid(s.pid, &status, 0) < 0 && errno == EINTR) {}
}
s.pid = -1;
}
try { s.drain_debug(); } catch (...) {}
close_fd(s.debug_pipe);
close_fd(s.debug_file);
s.debug_written = 0; s.debug_truncated = false;
if (!s.dir.empty()) {
::unlink((s.dir + "/clip.raw").c_str());
::rmdir(s.dir.c_str());
@@ -139,7 +276,7 @@ void Worker::stop() {
}
void Worker::start(const std::string& python, const std::string& script,
const std::string& model, int threads) {
const std::string& model, int threads, bool advanced_debug) {
if (state_->pid > 0) throw std::logic_error("worker already started");
if (python.empty() || script.empty() || model.empty() || threads < 1 || threads > 64)
throw std::invalid_argument("python, script, local model and 1..64 threads required");
@@ -151,9 +288,12 @@ void Worker::start(const std::string& python, const std::string& script,
if (!::mkdtemp(tmp.data())) throw std::runtime_error("cannot create private clip directory");
state_->dir = tmp.data();
::chmod(state_->dir.c_str(), 0700);
int in[2]{-1,-1}, out[2]{-1,-1};
if (::pipe2(in, O_CLOEXEC) || ::pipe2(out, O_CLOEXEC)) {
if (advanced_debug) state_->debug_file = create_debug_log();
int in[2]{-1,-1}, out[2]{-1,-1}, debug[2]{-1,-1};
if (::pipe2(in, O_CLOEXEC) || ::pipe2(out, O_CLOEXEC) ||
(advanced_debug && ::pipe2(debug, O_CLOEXEC))) {
close_fd(in[0]); close_fd(in[1]); close_fd(out[0]); close_fd(out[1]);
close_fd(debug[0]); close_fd(debug[1]);
throw std::runtime_error("cannot create worker pipes");
}
std::string thread_arg = std::to_string(threads);
@@ -170,17 +310,20 @@ void Worker::start(const std::string& python, const std::string& script,
for (auto& value : environment) envp.push_back(value.data());
envp.push_back(nullptr);
const char* args[] = {python.c_str(), script.c_str(), "--model", model.c_str(),
"--threads", thread_arg.c_str(), "--clip-dir", state_->dir.c_str(), nullptr};
"--threads", thread_arg.c_str(), "--clip-dir", state_->dir.c_str(),
advanced_debug ? "--advanced-debug" : nullptr, nullptr};
posix_spawn_file_actions_t actions;
int error = posix_spawn_file_actions_init(&actions);
if (error) {
close_fd(in[0]); close_fd(in[1]); close_fd(out[0]); close_fd(out[1]);
close_fd(debug[0]); close_fd(debug[1]);
throw std::runtime_error("cannot initialize worker spawn");
}
error = posix_spawn_file_actions_adddup2(&actions, in[0], STDIN_FILENO);
if (!error) error = posix_spawn_file_actions_adddup2(&actions, out[1], STDOUT_FILENO);
if (!error) error = posix_spawn_file_actions_addopen(&actions, STDERR_FILENO, "/dev/null", O_WRONLY, 0);
for (int fd : std::array<int, 4>{in[0], in[1], out[0], out[1]})
if (!error && advanced_debug) error = posix_spawn_file_actions_adddup2(&actions, debug[1], STDERR_FILENO);
if (!error && !advanced_debug) error = posix_spawn_file_actions_addopen(&actions, STDERR_FILENO, "/dev/null", O_WRONLY, 0);
for (int fd : std::array<int, 6>{in[0], in[1], out[0], out[1], debug[0], debug[1]})
if (!error && fd > STDERR_FILENO) error = posix_spawn_file_actions_addclose(&actions, fd);
pid_t pid = -1;
if (!error) error = posix_spawnp(&pid, python.c_str(), &actions, nullptr,
@@ -188,13 +331,18 @@ void Worker::start(const std::string& python, const std::string& script,
posix_spawn_file_actions_destroy(&actions);
if (error) {
close_fd(in[0]); close_fd(in[1]); close_fd(out[0]); close_fd(out[1]);
close_fd(debug[0]); close_fd(debug[1]);
throw std::runtime_error("cannot launch configured Python worker");
}
close_fd(in[0]); close_fd(out[1]);
close_fd(in[0]); close_fd(out[1]); close_fd(debug[1]);
state_->pid = pid; state_->to_child = in[1]; state_->from_child = out[0];
int flags = ::fcntl(state_->from_child, F_GETFL);
if (flags < 0 || ::fcntl(state_->from_child, F_SETFL, flags | O_NONBLOCK))
throw std::runtime_error("cannot set nonblocking worker pipe");
state_->debug_pipe = debug[0];
for (int fd : {state_->from_child, state_->debug_pipe}) {
if (fd < 0) continue;
int flags = ::fcntl(fd, F_GETFL);
if (flags < 0 || ::fcntl(fd, F_SETFL, flags | O_NONBLOCK))
throw std::runtime_error("cannot set nonblocking worker pipe");
}
state_->deadline = Clock::now() + state_->warmup_timeout;
} catch (...) { stop(); throw; }
}
@@ -240,6 +388,7 @@ std::optional<WorkerReply> Worker::poll() {
auto& s = *state_;
if (s.pid <= 0) return std::nullopt;
try {
s.drain_debug();
if ((!s.loaded || s.pending) && Clock::now() > s.deadline)
throw std::runtime_error(s.loaded ? "worker transcription timed out" : "worker warmup timed out");
unsigned char block[4096];
@@ -280,7 +429,7 @@ std::optional<WorkerReply> Worker::poll() {
std::string text(reinterpret_cast<const char*>(s.input.data() + 13), size - 9);
if (!valid_utf8(text)) throw std::runtime_error("invalid worker UTF-8 reply");
if (type == 'R') reply.text = std::move(text);
else reply.error = std::move(text);
else reply.error = safe_request_error(text);
s.pending.reset(); s.input.clear();
::unlink((s.dir + "/clip.raw").c_str());
return reply;
+1 -1
View File
@@ -23,7 +23,7 @@ public:
Worker(const Worker&) = delete;
Worker& operator=(const Worker&) = delete;
void start(const std::string& python, const std::string& script,
const std::string& model, int threads = 2);
const std::string& model, int threads = 2, bool advanced_debug = false);
bool ready() const;
void submit(uint64_t id, const std::vector<float>& pcm);
std::optional<WorkerReply> poll();
+41 -2
View File
@@ -28,16 +28,55 @@ int main(int argc, char** argv) {
::setenv("XDG_CONFIG_HOME", dir.c_str(), 1);
assert(load_config(path).buttons.empty());
assert(!load_config(path).experimental_input_priority);
assert(!load_config(path).advanced_debug);
assert(load_config(path).wrist.width == .30f);
auto example = load_config(std::filesystem::path(argv[1]) / "config.example.json");
assert(example.buttons.at("ptt") == "/user/hand/right/input/x");
assert(example.font.empty());
assert(!example.experimental_input_priority);
assert(!example.advanced_debug);
assert(example.wrist.y == .18f && example.wrist.z == .089f);
std::filesystem::create_directories(path.parent_path());
put(path, R"({"input_priority":"experimental"})");
assert(save_advanced_debug(path, true));
assert(load_config(path).advanced_debug);
assert(get(path).find("\"advanced_debug\":true") != std::string::npos);
assert(save_advanced_debug(path, false));
assert(!load_config(path).advanced_debug);
for (const auto* invalid : {R"({"advanced_debug":"true"})", R"({"advanced_debug":0})",
R"({"advanced_debug":null})", R"({"advanced_debug":[]})"}) {
put(path, invalid);
fails([&] { load_config(path); });
assert(!save_advanced_debug(path, true));
assert(get(path) == invalid);
}
put(path, R"({"font":"escaped \u0061 and \"quotes\"","input_priority":"experimental","wrist":{"y":0.21},"theme":{"ink":"#123ABC"},"buttons":{"enter":""}})");
const auto customized = get(path);
assert(save_advanced_debug(path, true));
assert(load_config(path).advanced_debug);
assert(get(path).find(customized.substr(1, customized.size() - 2)) != std::string::npos);
assert(save_advanced_debug(path, false));
assert(!load_config(path).advanced_debug);
auto before = get(path);
assert(save_advanced_debug(path, false) && get(path) == before);
put(path, R"({"advanced_debug":false, "font":"kept"})");
assert(save_advanced_debug(path, true));
assert(get(path) == R"({"advanced_debug":true, "font":"kept"})");
before = get(path);
put(path, R"({"font":"bad","font":"duplicate"})");
assert(!save_advanced_debug(path, true));
assert(get(path) == R"({"font":"bad","font":"duplicate"})");
put(path, before);
auto link = dir / "linked-config";
std::filesystem::create_symlink(path, link);
assert(!save_advanced_debug(link, false));
assert(get(path) == before);
std::filesystem::remove(link);
put(path, R"({"font":")" + std::string(4090, 'x') + R"("})");
before = get(path);
assert(!save_advanced_debug(path, true) && get(path) == before);
put(path, R"({"input_priority":"experimental","advanced_debug":true})");
auto experimental = load_config(path);
assert(experimental.experimental_input_priority);
assert(experimental.experimental_input_priority && experimental.advanced_debug);
// A priority request must preserve the user's existing action manifest/bindings.
assert(action_manifest(argv[1], experimental) == std::filesystem::absolute(std::filesystem::path(argv[1]) / "actions.json"));
put(path, R"({"input_priority":"normal"})");
+17 -1
View File
@@ -12,7 +12,8 @@ SurfaceEvent click(PanelSurface& surface, float x, float y, unsigned cursor = 0)
return surface.pointer_up(cursor, x, y);
}
void no_action(const SurfaceEvent& event) {
assert(!event.action && !event.mount && !event.lasers_anytime && !event.recenter && !event.open_bindings);
assert(!event.action && !event.mount && !event.lasers_anytime && !event.advanced_debug &&
!event.recenter && !event.open_bindings);
}
void snapshot(PanelSurface& surface, const std::string& path) {
std::ofstream out(path, std::ios::binary);
@@ -112,6 +113,19 @@ int main(int argc, char** argv) {
laser = click(surface, 680, 420);
assert(laser.lasers_anytime == false && !laser.action && !laser.mount);
surface.set_lasers_anytime(false); assert(surface.render(p));
auto debug = click(surface, 680, 474);
assert(debug.advanced_debug == true && !debug.action && !debug.mount && !debug.lasers_anytime);
assert(!surface.render(p)); // an event is only a request; caller sets the accepted value
surface.set_advanced_debug(true); assert(surface.render(p)); assert(!surface.render(p));
surface.pointer_down(1, 680, 474);
surface.set_advanced_debug(false); assert(surface.render(p));
no_action(surface.pointer_up(1, 680, 474)); // stale press cannot toggle after state change
debug = click(surface, 680, 474);
assert(debug.advanced_debug == true);
surface.set_advanced_debug(true); assert(surface.render(p));
debug = click(surface, 680, 474);
assert(debug.advanced_debug == false);
surface.set_advanced_debug(false); assert(surface.render(p));
assert(click(surface, 280, 610).action == UiAction::Cancel);
surface.set_placement_note("Wrist not tracked - using world space until it returns.");
assert(surface.render(p)); assert(!surface.render(p));
@@ -123,6 +137,7 @@ int main(int argc, char** argv) {
no_action(surface.pointer_up(1, 680, 260));
no_action(click(surface, 680, 260)); // mount controls not active on Review
no_action(click(surface, 680, 420)); // laser toggle only exists on Settings
no_action(click(surface, 680, 474)); // debug toggle only exists on Settings
// Binding navigation is not a delivery action; settings and paging are hidden.
assert(!surface.bindings_visible());
@@ -138,6 +153,7 @@ int main(int argc, char** argv) {
no_action(click(surface, 680, 260));
no_action(click(surface, 900, 430));
no_action(click(surface, 680, 420));
no_action(click(surface, 680, 474));
const auto editor = click(surface, 200, 434);
assert(editor.open_bindings && !editor.action && !editor.mount && !editor.recenter && !editor.lasers_anytime);
no_action(surface.pointer_up(0, 200, 434)); // one launch per deliberate click
+23
View File
@@ -73,6 +73,7 @@ class InstallTests(unittest.TestCase):
self.assertNotIn("--font", launcher.read_text()) # run/check modes honor config font
config = self.home / ".config/frameyap/config.json"
self.assertEqual(json.loads(config.read_text()), installer.CONFIG_DEFAULTS)
self.assertIs(json.loads(config.read_text())["advanced_debug"], False)
self.assertEqual(list(config.parent.glob("config.json.backup-*")), [])
self.assertIn("PYTHONDONTWRITEBYTECODE=1", launcher.read_text())
self.assertTrue(os.access(root / "versions/v1/runtime/bin/helper", os.X_OK))
@@ -125,17 +126,26 @@ class InstallTests(unittest.TestCase):
self.assertEqual(fixed["theme"]["card"], installer.CONFIG_DEFAULTS["theme"]["card"])
self.assertEqual(fixed["buttons"]["cancel"], "/user/hand/right/input/b")
self.assertEqual(fixed["input_priority"], "normal")
self.assertIs(fixed["advanced_debug"], False)
self.assertEqual(fixed["wrist"], installer.CONFIG_DEFAULTS["wrist"])
backups = list(config.parent.glob("config.json.backup-*"))
self.assertEqual(len(backups), 1)
self.assertEqual(backups[0].read_bytes(), original)
fixed["input_priority"] = "experimental"
fixed["advanced_debug"] = True
fixed["buttons"]["enter"] = "" # intentional disabling survives upgrades
fixed["wrist"]["y"] = 0.2
compact = json.dumps(fixed, separators=(",", ":")).encode()
config.write_bytes(compact)
self.install("v1", archive, digest)
self.assertEqual(config.read_bytes(), compact)
self.assertIs(json.loads(config.read_text())["advanced_debug"], True)
self.assertEqual(len(list(config.parent.glob("config.json.backup-*"))), 1)
fixed["advanced_debug"] = False
compact_off = json.dumps(fixed, separators=(",", ":")).encode()
config.write_bytes(compact_off)
self.install("v1", archive, digest)
self.assertEqual(config.read_bytes(), compact_off)
self.assertEqual(len(list(config.parent.glob("config.json.backup-*"))), 1)
original = b'{"font":"/system/face.ttf","input_priority":"highest","wrist":{"x":0.04,"y":true,"width":100,"obsolete":4},"theme":{"ink":"bad","retired":"#123456"},"buttons":{"ptt":"/user/hand/left/input/grip"},"old_option":4}'
config.write_bytes(original)
@@ -145,6 +155,7 @@ class InstallTests(unittest.TestCase):
self.assertEqual(fixed["theme"]["ink"], installer.CONFIG_DEFAULTS["theme"]["ink"])
self.assertEqual(fixed["buttons"], installer.CONFIG_DEFAULTS["buttons"]) # colliding paths reset
self.assertEqual(fixed["input_priority"], "normal")
self.assertIs(fixed["advanced_debug"], False)
self.assertEqual(fixed["wrist"]["x"], 0.04)
self.assertEqual(fixed["wrist"]["y"], 0.18)
self.assertEqual(fixed["wrist"]["width"], 0.30)
@@ -170,6 +181,18 @@ class InstallTests(unittest.TestCase):
self.install("v1", archive, digest)
self.assertTrue(config.is_symlink())
def test_debug_boolean_repair_backs_up_invalid_values(self):
archive, digest = self.package("v1")
config = self.home / ".config/frameyap/config.json"
config.parent.mkdir(parents=True)
for invalid in ("true", 1, None, [], {}):
original = json.dumps({"advanced_debug": invalid, "font": "/custom/font.ttf"}).encode()
config.write_bytes(original)
self.install("v1", archive, digest)
self.assertIs(json.loads(config.read_text())["advanced_debug"], False)
self.assertEqual(json.loads(config.read_text())["font"], "/custom/font.ttf")
self.assertIn(original, [p.read_bytes() for p in config.parent.glob("config.json.backup-*")])
def test_digest_and_same_version_mismatch_leave_previous(self):
a, h = self.package("v1")
self.install("v1", a, h)
+120 -3
View File
@@ -4,8 +4,11 @@ Run: python3 -m unittest discover -s tests -p test_worker.py
Native fixture: python3 tests/test_worker.py --model ok --threads 2 --clip-dir DIR
"""
import os
import io
import contextlib
from pathlib import Path
import struct
import subprocess
import sys
import tempfile
import unittest
@@ -31,6 +34,7 @@ def fake_child():
parser.add_argument("--model", required=True)
parser.add_argument("--threads", required=True)
parser.add_argument("--clip-dir", required=True)
parser.add_argument("--advanced-debug", action="store_true")
args = parser.parse_args()
if args.model in ("fail", "missing-model", "missing-import"):
worker.send_frame(1, b"F", {"fail": b"", "missing-model": b"M",
@@ -54,6 +58,11 @@ def fake_child():
continue
assert len(msg) == 9 and msg[:1] == b"T"
requests += 1
if args.advanced_debug and args.model in ("debug-output", "debug-spam"):
os.write(2, b"private debug fixture\n")
if args.model == "debug-spam":
for _ in range(1100):
os.write(2, b"x" * 4096)
clip = Path(args.clip_dir) / "clip.raw"
assert clip.stat().st_size == 3200 * 4
ident = msg[1:9]
@@ -62,8 +71,11 @@ def fake_child():
if args.model == "oversized-frame":
os.write(1, struct.pack("<I", 65537))
continue
if args.model == "request-error" and requests == 1:
worker.send_frame(1, b"E", ident + b"transcription failed")
errors = {"request-error": b"transcription failed",
"safe-error": b"transcription failed [inference: RuntimeError]",
"unsafe-error": b"transcription failed [inference: RuntimeError] PRIVATE SPEECH\n"}
if args.model in errors and requests == 1:
worker.send_frame(1, b"E", ident + errors[args.model])
continue
text = b"a" * 4097 if args.model == "long" else "héllo 世界".encode()
worker.send_frame(1, b"R", ident + text)
@@ -110,6 +122,111 @@ class WorkerTests(unittest.TestCase):
os.close(out_r)
os.close(out_w)
def test_safe_error_labels_and_retry(self):
class PrivateError(Exception):
def __str__(self):
raise AssertionError("exception messages must not be accessed")
self.assertEqual(worker.safe_request_error("PRIVATE PATH", PrivateError()),
b"transcription failed [unknown: Exception]")
cases = [
(OSError("PRIVATE AUDIO"), {"text": "ignored"}, b"audio: OSError"),
(None, RuntimeError("PRIVATE TRANSCRIPT"), b"inference: RuntimeError"),
(None, PrivateError(), b"inference: Exception"),
(None, {}, b"response: KeyError"),
(None, {"text": 4}, b"response: TypeError"),
(None, {"text": "\ud800"}, b"response: UnicodeError"),
]
for audio_error, response, label in cases:
with self.subTest(label=label), tempfile.TemporaryDirectory() as path:
class Model:
count = 0
def transcribe(self, **kwargs):
self.count += 1
if self.count > 1:
return {"text": "retry"}
if isinstance(response, Exception):
raise response
return response
model = Model()
in_r, in_w = os.pipe()
out_r, out_w = os.pipe()
try:
worker.send_frame(in_w, b"T", struct.pack("<Q", 1))
worker.send_frame(in_w, b"T", struct.pack("<Q", 2))
os.close(in_w); in_w = -1
# A clip-read failure never calls the model; retry can succeed.
reads = [audio_error, []] if audio_error else [[], []]
if audio_error:
model.count = 1
with patch.object(worker, "read_clip", side_effect=reads):
worker.run(model, path, in_r, out_w)
self.assertEqual(worker.read_frame(out_r), b"Y")
self.assertEqual(worker.read_frame(out_r), b"E" + struct.pack("<Q", 1) +
b"transcription failed [" + label + b"]")
self.assertEqual(worker.read_frame(out_r), b"R" + struct.pack("<Q", 2) + b"retry")
finally:
os.close(in_r)
if in_w >= 0: os.close(in_w)
os.close(out_r); os.close(out_w)
def test_debug_is_explicit_and_protocol_stays_safe(self):
for enabled in (False, True):
with self.subTest(enabled=enabled), tempfile.TemporaryDirectory() as path:
class Failing:
def transcribe(self, **kwargs):
raise RuntimeError("PRIVATE EXCEPTION DETAIL")
in_r, in_w = os.pipe(); out_r, out_w = os.pipe()
captured = io.StringIO()
try:
worker.send_frame(in_w, b"T", struct.pack("<Q", 8))
os.close(in_w); in_w = -1
with patch.object(worker, "read_clip", return_value=[]), contextlib.redirect_stderr(captured):
worker.run(Failing(), path, in_r, out_w, advanced_debug=enabled)
self.assertEqual(worker.read_frame(out_r), b"Y")
self.assertEqual(worker.read_frame(out_r), b"E" + struct.pack("<Q", 8) +
b"transcription failed [inference: RuntimeError]")
if enabled:
self.assertIn("Traceback", captured.getvalue())
self.assertIn("PRIVATE EXCEPTION DETAIL", captured.getvalue())
else:
self.assertEqual(captured.getvalue(), "")
finally:
os.close(in_r)
if in_w >= 0: os.close(in_w)
os.close(out_r); os.close(out_w)
def test_debug_fd_output_never_corrupts_protocol(self):
program = '''
import os, sys
from frameyap import worker
class Model:
def transcribe(self, **kwargs):
os.write(1, b"native stdout fixture\\n")
os.write(2, b"native stderr fixture\\n")
return {"text": "private fixture transcript"}
worker.load_model = lambda *args: Model()
worker.read_clip = lambda *args: []
sys.exit(worker.main(sys.argv[1:]))
'''
request = b"T" + struct.pack("<Q", 12)
response = b"R" + struct.pack("<Q", 12) + b"private fixture transcript"
expected = struct.pack("<I", 1) + b"Y" + struct.pack("<I", len(response)) + response
with tempfile.TemporaryDirectory() as path:
for enabled in (False, True):
args = [sys.executable, "-c", program, "--model", "unused", "--clip-dir", path]
if enabled: args.append("--advanced-debug")
env = dict(os.environ, PYTHONPATH=str(Path(__file__).resolve().parents[1] / "python"))
result = subprocess.run(args, input=struct.pack("<I", len(request)) + request,
capture_output=True, env=env, timeout=10)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stdout, expected)
if enabled:
self.assertIn(b"native stdout fixture", result.stderr)
self.assertIn(b"native stderr fixture", result.stderr)
self.assertIn(b"private fixture transcript", result.stderr)
else:
self.assertEqual(result.stderr, b"")
def test_missing_weights_no_dependency_import(self):
with tempfile.TemporaryDirectory() as path:
with self.assertRaisesRegex(worker.LocalModelError, "weights missing"):
@@ -157,7 +274,7 @@ class WorkerTests(unittest.TestCase):
in_w = -1
worker.run(Large(), path, in_r, out_w)
self.assertEqual(worker.read_frame(out_r), b"Y")
self.assertEqual(worker.read_frame(out_r), b"E" + struct.pack("<Q", 7) + b"transcription failed")
self.assertEqual(worker.read_frame(out_r), b"E" + struct.pack("<Q", 7) + b"transcription failed [response: ValueError]")
finally:
os.close(in_r)
if in_w >= 0:
+133
View File
@@ -5,6 +5,8 @@
#include <cstdlib>
#include <filesystem>
#include <functional>
#include <fstream>
#include <sys/stat.h>
#include <stdexcept>
#include <string>
#include <thread>
@@ -143,6 +145,137 @@ int main(int argc, char** argv) {
return failed;
});
assert(std::filesystem::is_empty(runtime));
// A separate fake worker proves stderr is not part of the framed pipe,
// even when diagnostics fill its pipe before the ready frame.
char scratch[] = "/tmp/frameyap-debug-test-XXXXXX";
if (!::mkdtemp(scratch)) throw std::runtime_error("debug fixture setup failed");
const auto root = std::filesystem::path(scratch);
try {
const auto script = root / "fake.py";
{
std::ofstream out(script);
out << R"PY(import argparse, os, struct, sys
p = argparse.ArgumentParser()
p.add_argument('--model')
p.add_argument('--threads')
p.add_argument('--clip-dir')
p.add_argument('--advanced-debug', action='store_true')
a = p.parse_args()
protocol = os.dup(1)
if a.advanced_debug:
os.dup2(2, 1) # Match the production Python protocol/stdout split.
os.write(2, b'PRIVATE TRANSCRIPT /private/model/path\n')
os.write(1, b'PRIVATE STDOUT from model\n')
if a.model == 'spam':
for _ in range(1300): os.write(2, b'x' * 4096)
else:
null = os.open(os.devnull, os.O_WRONLY)
os.dup2(null, 1)
os.close(null)
os.write(protocol, struct.pack('<I', 1) + b'Y')
while True:
request = os.read(0, 13)
if not request: break
assert len(request) == 13 and request[4:5] == b'T'
if a.model == 'unsafe':
value = b'E' + request[5:] + b'transcription failed [inference: RuntimeError] PRIVATE SPEECH'
elif a.model == 'safe':
value = b'E' + request[5:] + b'transcription failed [inference: RuntimeError]'
else:
value = b'R' + request[5:] + b'ok'
os.write(protocol, struct.pack('<I', len(value)) + value)
)PY";
}
const auto state = root / "state";
::setenv("XDG_STATE_HOME", state.c_str(), 1);
const auto logs = state / "frameyap";
const auto current = logs / "worker-debug.log";
const auto previous = logs / "worker-debug.previous.log";
worker.start(argv[1], script, "safe", 2); // OFF must not create files.
until([&] { worker.poll(); return worker.ready(); });
worker.submit(300, clip);
until([&] { reply = worker.poll(); return reply.has_value(); });
assert(reply->error == "transcription failed [inference: RuntimeError]");
worker.stop();
assert(!std::filesystem::exists(logs));
worker.start(argv[1], script, "spam", 2, true);
until([&] { worker.poll(); return worker.ready(); });
worker.submit(301, clip);
until([&] { reply = worker.poll(); return reply.has_value(); });
assert(reply->text == "ok");
worker.stop();
assert(std::filesystem::status(logs).permissions() == std::filesystem::perms::owner_all);
assert(std::filesystem::status(current).permissions() ==
(std::filesystem::perms::owner_read | std::filesystem::perms::owner_write));
assert(std::filesystem::file_size(current) <= 4 * 1024 * 1024);
std::ifstream input(current, std::ios::binary);
std::string body((std::istreambuf_iterator<char>(input)), std::istreambuf_iterator<char>());
assert(body.find("PRIVATE TRANSCRIPT /private/model/path") != std::string::npos);
assert(body.find("PRIVATE STDOUT from model") != std::string::npos);
assert(body.find("[worker diagnostics truncated; further output discarded]") != std::string::npos);
worker.start(argv[1], script, "unsafe", 2, true);
until([&] { worker.poll(); return worker.ready(); });
worker.submit(302, clip);
until([&] { reply = worker.poll(); return reply.has_value(); });
assert(reply->error == "transcription failed" && worker.ready());
worker.stop();
assert(std::filesystem::file_size(previous) <= 4 * 1024 * 1024);
assert(std::filesystem::file_size(current) < 4096);
auto size = std::filesystem::file_size(current);
worker.start(argv[1], script, "ok", 2); // OFF retains earlier logs.
until([&] { worker.poll(); return worker.ready(); });
worker.stop();
assert(std::filesystem::file_size(current) == size && std::filesystem::exists(previous));
std::filesystem::remove(current);
std::filesystem::create_symlink(script, current);
bool refused = false;
try { worker.start(argv[1], script, "ok", 2, true); }
catch (const std::runtime_error&) { refused = true; }
assert(refused && !worker.ready());
std::filesystem::remove(current);
{ std::ofstream out(current); out << "existing"; }
::chmod(current.c_str(), 0644);
refused = false;
try { worker.start(argv[1], script, "ok", 2, true); }
catch (const std::runtime_error&) { refused = true; }
assert(refused);
std::filesystem::remove(current);
{ std::ofstream out(current); out << "linked"; }
::chmod(current.c_str(), 0600);
std::filesystem::create_hard_link(current, root / "outside-link");
refused = false;
try { worker.start(argv[1], script, "ok", 2, true); }
catch (const std::runtime_error&) { refused = true; }
assert(refused);
std::filesystem::remove(root / "outside-link");
std::filesystem::remove(current);
std::filesystem::remove_all(logs);
std::filesystem::create_directory_symlink(root, logs);
refused = false;
try { worker.start(argv[1], script, "ok", 2, true); }
catch (const std::runtime_error&) { refused = true; }
assert(refused);
std::filesystem::remove(logs);
std::filesystem::create_directory(logs);
::chmod(logs.c_str(), 0755);
refused = false;
try { worker.start(argv[1], script, "ok", 2, true); }
catch (const std::runtime_error&) { refused = true; }
assert(refused);
::unsetenv("XDG_STATE_HOME");
::setenv("HOME", root.c_str(), 1);
worker.start(argv[1], script, "ok", 2, true);
until([&] { worker.poll(); return worker.ready(); });
worker.stop();
assert(std::filesystem::exists(root / ".local/state/frameyap/worker-debug.log"));
::setenv("XDG_STATE_HOME", "relative/state", 1);
refused = false;
try { worker.start(argv[1], script, "ok", 2, true); }
catch (const std::runtime_error&) { refused = true; }
assert(refused);
std::filesystem::remove_all(root);
} catch (...) { std::filesystem::remove_all(root); throw; }
std::filesystem::remove(runtime);
} catch (...) {
std::filesystem::remove(runtime);