Commit Graph
3 Commits
Author SHA1 Message Date
saphidandClaude Opus 5.5 a3c6e5c984 Mac in the headset: stream Mac windows and screens into the Frame as panels
Frame Control can now show any Mac window, or a whole screen, as its own
SteamVR panel on the Steam Frame (Tools -> Mac in the headset, macOS only).
Place it with the SteamVR dashboard; the laser clicks and scrolls, and the
Mac's own keyboard types.

- mac/frame-mac-view (Swift, no dependencies): ScreenCaptureKit capture per
  window or display, VideoToolbox H.264 with low-latency rate control (JPEG
  fallback), a loopback HTTP/WebSocket server, CGEvent/AX input playback,
  and a display-awake assertion while anyone watches.
- ui/frame_macview.py: starts the agent, runs an ssh -R tunnel with a
  supervisor that reopens it on the same port, and launches a Chromium app
  window per stream on gamescope's :0, tagged with STEAM_GAME for its own panel.
- Frame Control's key never leaves the Mac: viewers get single-use,
  per-source tickets and reconnect keys that Stop revokes.
- ui/mac-view.html: WebCodecs decode, keyframe recovery, pointer/wheel/keys back.
- Bundled in the Mac app build; tests/test_macview.py builds and drives the
  agent on macOS.

Verified on the Frame (build 20260925.6191901) with the test pattern: panel
in about 1.5 s, about 60 fps, Mac-to-window about 11-17 ms, tunnel recovery
in 4 s. Laser input and real window capture still need a person in the headset.

Also commits the other thread's first-party rule (steam-frame skill) and
the first-party options table in docs/streaming.md.

Reviewed by GPT-6 Astra (xhigh, read-only) over six rounds; all findings fixed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 13:33:46 +10:00
saphidandClaude Opus 5.5 0181071b83 Tests against a fake Frame, and a headset smoke test
tests/fakeframe: a container that stands in for the Frame (Arch Linux, or
Arch Linux ARM on arm64) with sshd, rsync, Valve's steamos-devkit-service
and hooks (vendored unmodified), a fake Steam client for the devkit pipe and
the DevTools port (the app's JavaScript runs in Node against stand-in
SteamClient/appStore objects), stubs for steam, wpctl, flatpak, podman,
Lepton and friends, battery and thermal files under /sys, and fault
switches (fakeframe-ctl): pairing mode, approve/deny/timeout, Steam not
running, headset asleep, sshd off, disk full, runtimes missing. A second
container is the computer running Frame Control.

tests/e2e: 29 tests driving the real ui/server.py, frame_connect.py and
frame_titles.py against it; skipped unless FRAME_E2E=1. scripts/e2e.sh
builds, runs and tears down; CI runs it on ubuntu-24.04-arm.

tests/smoke + scripts/frame-smoke.sh: the core cases against a real Frame,
recorded with its BUILD_ID, cleaning up after itself; --pair to pair a
throwaway key. docs/testing.md describes the layers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 17:40:09 +10:00
saphidandClaude Opus 5.5 d4486a7681 Frame Control: Mac app, web UI, Android and Steam tooling
Package the Frame Control web UI as an installable Electron Mac app and
bring in the tooling built alongside it.

- app/: Electron wrapper that starts ui/server.py on a free loopback port,
  hardened window (sandbox, no navigation, runAsNode fuse off), login-shell
  PATH so Homebrew tools work from Finder, first-run offer to run
  connect.sh, ad-hoc signed DMG/zip via electron-builder.
- ui/: headset view (OpenVR screenshots), device status, library, Steam
  "Get games" (owned games, install, store search), Android apps as
  persistent Lepton instances with a rated F-Droid catalogue and a private
  compatibility database, Android display controls over ADB, file and
  clipboard transfer, Flatpaks, remote and power actions.
- apk-catalog/, compat-db/, frame/: catalogue build pipeline, Lakebed
  capsule for compatibility reports, Frame-side launchers.
- tests/ and CI: server guard and validation tests plus Steam helper tests,
  run on Python 3.9 with script and app syntax checks.
- Docs: README leads with the Mac app; new Android, panels, Steam games and
  field-notes docs; security notes on LAN-exposed ADB ports.

Screenshot values for the headset's IP and Wi-Fi name are placeholders.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-25 22:21:20 +10:00