Mac in the headset: measure every frame, adapt to the network, separate windows

- Per-frame timing on the Mac's clock (capture, encode, network, decode,
  draw), viewer clock sync and reports, input echo, /stats and a HUD.
- scripts/macview-bench.py: repeatable runs on the real Frame, a shaping
  relay (no sudo), interleaved A/B between agent settings; results in
  bench/results/.
- Adaptive controller: ack-based send gate with jitter-aware slack, AIMD
  bitrate that knows when a stream is app-limited, fps then size tiers.
  On a 50->3->50 Mbit/s step, scroll p95 went from 4.7 s to 72 ms; no cost
  on a clean link.
- Separate mode: real AppKit event loop (HiDPI and NSScreen now work),
  cropped capture for fixed-size windows, windows kept on their display,
  graceful quit restores windows; stop/start races fixed.
- Encoder timeline clamp (no oversized frame after a pause).
- Frame Control shows each live stream's fps, delay, bitrate and tier.

Reviewed by GPT-6 Astra xhigh (read-only), 7 rounds; findings fixed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
saphidandClaude Opus 5.5 committed 2026-09-28 20:56:14 +10:00
1 parent a3c6e5c984
commit 9e4dcdd147
29 files changed
+21405 -104

No files matched your search

+4 -2
View File
@@ -165,9 +165,10 @@ final class WebSocket {
func sendJSON(_ obj: Any) {
if let d = try? JSONSerialization.data(withJSONObject: obj), let s = String(data: d, encoding: .utf8) { sendText(s) }
}
func sendBinary(_ d: Data) { send(opcode: 2, d) }
/// `taken` runs once the network stack has taken the whole frame.
func sendBinary(_ d: Data, taken: (() -> Void)? = nil) { send(opcode: 2, d, taken: taken) }
func send(opcode: UInt8, _ payload: Data) {
func send(opcode: UInt8, _ payload: Data, taken: (() -> Void)? = nil) {
var frame = Data([0x80 | opcode])
let n = payload.count
if n < 126 {
@@ -185,6 +186,7 @@ final class WebSocket {
conn.send(content: frame, completion: .contentProcessed { [weak self] _ in
guard let self else { return }
self.lock.lock(); self._pending -= size; self.lock.unlock()
taken?()
self.onDrain?()
})
}