Commit Graph
10 Commits
Author SHA1 Message Date
saphid 72352c5914 Add companion and headset panel switchers with feasibility evidence 2026-09-28 22:33:23 +10:00
saphidandClaude Opus 5.5 1b90c64b73 Docs: benchmark with the headset worn (Wi-Fi much rougher; controller bounded latency)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:02:05 +10:00
saphidandClaude Opus 5.5 39afb23d97 Docs: Steam's StartDesktopStream pairs the Frame with the Mac (verified); stream test next
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 21:45:52 +10:00
saphidandClaude Opus 5.5 bda9d77d2d Mac in the headset: USB route keeps a configured host-key alias and falls back to the network
Review round 12: a failed USB-C tunnel now retries the normal path; an
existing HostKeyAlias wins; --host with --usb is rejected; the Steam
desktop-streaming claim is now 'untested' (Valve documents the desktop
showing when a game loses focus); the Show/cleanup overlap test blocks
for real (it fails without the lock). Verified live: with the cable out,
the route is the normal path.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 21:39:40 +10:00
saphidandClaude Opus 5.5 4fae62ba14 Mac in the headset: use the Frame's USB-C network when plugged in; Steam streaming findings
- Plugged into the Mac, the Frame is a USB network device ("Steam Frame",
  usb0 at ~0.9 ms). The tunnel uses it when the Frame's usb0 answers,
  with the usual host key; otherwise the normal path. Interleaved runs:
  content p50 7 vs 10 ms, click to drawn 17 vs 27 ms, scroll p95 23 vs
  31-37 ms. FRAME_MACVIEW_USB=0 turns it off; the card says "over USB-C".
- Bench: --usb, and the route is recorded per run.
- Docs: Steam's own streaming (no SteamVR host on macOS; Remote Play pairs
  but streams games, not windows; test blocked); the Frame's USB network;
  the 2026-09-28 health-check boot-loop recurrence.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 21:33:31 +10:00
saphidandClaude Opus 5.5 7f769ca2f0 Mac in the headset: end the Frame's viewer browser after Stop; relay fixes
- Chromium on the Frame outlived its last viewer window (verified: 11
  processes left after a run). Once nothing is shown, Stop ends it, unless
  Show was pressed again meanwhile; its profile is Frame Control's own.
- Relay --delay: if the agent side fails, close the viewer side too
  (review round 9).
- Docs: the final scroll run captured 57 fps; don't blame ScreenCaptureKit.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 21:07:59 +10:00
saphidandClaude Opus 5.5 71200c5179 Merge origin/main into mac-in-headset
Brings in #4, #5, #8 (fbl100's verified Remmina/VNC mirror), APK
alternatives and the website. docs/streaming.md: Mac in the headset stays
the recommendation (now verified on the Frame); Remmina keeps #8's verified
evidence as the whole-screen fallback. docs/mac-in-headset.md cites #8's
lag finding.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 21:03:34 +10:00
saphidandClaude Opus 5.5 00b45bafc5 Mac in the headset: final benchmark numbers; relay delay is a delay line
Review round 8 (GPT-6 Astra xhigh): --delay paused upstream reads, so busy
streams saw 30-60 ms instead of 30. Verified locally: 60-62 ms round trip
with 30 ms each way under load. No saved result used --delay.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 21:02:26 +10:00
saphidandClaude Opus 5.5 9e4dcdd147 Mac in the headset: measure every frame, adapt to the network, separate windows
- Per-frame timing on the Mac's clock (capture, encode, network, decode,
  draw), viewer clock sync and reports, input echo, /stats and a HUD.
- scripts/macview-bench.py: repeatable runs on the real Frame, a shaping
  relay (no sudo), interleaved A/B between agent settings; results in
  bench/results/.
- Adaptive controller: ack-based send gate with jitter-aware slack, AIMD
  bitrate that knows when a stream is app-limited, fps then size tiers.
  On a 50->3->50 Mbit/s step, scroll p95 went from 4.7 s to 72 ms; no cost
  on a clean link.
- Separate mode: real AppKit event loop (HiDPI and NSScreen now work),
  cropped capture for fixed-size windows, windows kept on their display,
  graceful quit restores windows; stop/start races fixed.
- Encoder timeline clamp (no oversized frame after a pause).
- Frame Control shows each live stream's fps, delay, bitrate and tier.

Reviewed by GPT-6 Astra xhigh (read-only), 7 rounds; findings fixed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 20:56:14 +10:00
saphidandClaude Opus 5.5 a3c6e5c984 Mac in the headset: stream Mac windows and screens into the Frame as panels
Frame Control can now show any Mac window, or a whole screen, as its own
SteamVR panel on the Steam Frame (Tools -> Mac in the headset, macOS only).
Place it with the SteamVR dashboard; the laser clicks and scrolls, and the
Mac's own keyboard types.

- mac/frame-mac-view (Swift, no dependencies): ScreenCaptureKit capture per
  window or display, VideoToolbox H.264 with low-latency rate control (JPEG
  fallback), a loopback HTTP/WebSocket server, CGEvent/AX input playback,
  and a display-awake assertion while anyone watches.
- ui/frame_macview.py: starts the agent, runs an ssh -R tunnel with a
  supervisor that reopens it on the same port, and launches a Chromium app
  window per stream on gamescope's :0, tagged with STEAM_GAME for its own panel.
- Frame Control's key never leaves the Mac: viewers get single-use,
  per-source tickets and reconnect keys that Stop revokes.
- ui/mac-view.html: WebCodecs decode, keyframe recovery, pointer/wheel/keys back.
- Bundled in the Mac app build; tests/test_macview.py builds and drives the
  agent on macOS.

Verified on the Frame (build 20260925.6191901) with the test pattern: panel
in about 1.5 s, about 60 fps, Mac-to-window about 11-17 ms, tunnel recovery
in 4 s. Laser input and real window capture still need a person in the headset.

Also commits the other thread's first-party rule (steam-frame skill) and
the first-party options table in docs/streaming.md.

Reviewed by GPT-6 Astra (xhigh, read-only) over six rounds; all findings fixed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 13:33:46 +10:00