0 Fork 0

mirror of https://github.com/saphid/frame-control.git synced 2026-10-06 05:02:50 +02:00

Files

T

saphidandClaude Opus 5.5 a3c6e5c984 Mac in the headset: stream Mac windows and screens into the Frame as panels

Frame Control can now show any Mac window, or a whole screen, as its own
SteamVR panel on the Steam Frame (Tools -> Mac in the headset, macOS only).
Place it with the SteamVR dashboard; the laser clicks and scrolls, and the
Mac's own keyboard types.

- mac/frame-mac-view (Swift, no dependencies): ScreenCaptureKit capture per
  window or display, VideoToolbox H.264 with low-latency rate control (JPEG
  fallback), a loopback HTTP/WebSocket server, CGEvent/AX input playback,
  and a display-awake assertion while anyone watches.
- ui/frame_macview.py: starts the agent, runs an ssh -R tunnel with a
  supervisor that reopens it on the same port, and launches a Chromium app
  window per stream on gamescope's :0, tagged with STEAM_GAME for its own panel.
- Frame Control's key never leaves the Mac: viewers get single-use,
  per-source tickets and reconnect keys that Stop revokes.
- ui/mac-view.html: WebCodecs decode, keyframe recovery, pointer/wheel/keys back.
- Bundled in the Mac app build; tests/test_macview.py builds and drives the
  agent on macOS.

Verified on the Frame (build 20260925.6191901) with the test pattern: panel
in about 1.5 s, about 60 fps, Mac-to-window about 11-17 ms, tunnel recovery
in 4 s. Laser input and real window capture still need a person in the headset.

Also commits the other thread's first-party rule (steam-frame skill) and
the first-party options table in docs/streaming.md.

Reviewed by GPT-6 Astra (xhigh, read-only) over six rounds; all findings fixed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-09-28 13:33:46 +10:00

3.4 KiB

Raw Blame History

name, description

name	description
steam-frame	Operate the user's Valve Steam Frame headset from the Mac through this repo's helpers and field notes. Use for Steam Frame SSH, screen streaming, clipboard, file push, APK or Flatpak installs, launching apps on the headset, arranging floating windows or panels in VR space, or debugging SteamOS/gamescope/SteamVR on the Frame.

Steam Frame

SSH works through the frame alias (user steamos). The headset has to be awake for anything that touches its desktop or panels.

Start here

Read docs/how-the-frame-works.md. It's the map: the layer cake (SteamVR → gamescope → nested Plasma), the verified facts, and debug recipes.
Open the topic doc for the task:

Task	Doc	Script
Floating windows in the room, one panel per app	`docs/panels.md`	`scripts/panel-on-frame.sh`
First-time access, SSH keys	`docs/ssh.md`	`scripts/connect.sh`
See the Frame from the Mac, or the Mac inside the Frame	`docs/streaming.md`	`scripts/run-on-frame.sh mac-screen`
Files and clipboard	`docs/file-transfer.md`	`scripts/push.sh`, `scripts/paste-to-frame.sh`
Android apps (Lepton)	`docs/apks.md`	`scripts/install-apk.sh`
Reach the Frame off the home LAN (Tailscale)	`docs/tailscale.md`	`scripts/tailscale-on-frame.sh`
Install or buy Steam games, Frame ratings	`docs/steam-games.md`	`ui/frame_steam.py`
Flatpaks	`docs/streaming.md`	`scripts/install-apps.sh`
Launch an app inside the desktop panel	the script's header comment	`scripts/run-on-frame.sh`
Mac GUI over all of this	`README.md` → Frame Control	`scripts/frame-ui.sh`
iPhone/iPad app (server runs on the Frame, `FRAME_LOCAL=1`)	`docs/iphone.md`	`ios/`, `ui/local-bin/ssh`
Recovery images, factory reset, boot loops	`docs/recovery-and-images.md`, `docs/how-the-frame-works.md`	`~/Downloads/steam-frame-recovery/`
Test without the headset (the Frame OS image's own sshd)	`tests/frame-container/README.md`	`tests/frame-container/frame-image.sh`
What's still unverified	`docs/open-questions.md`	—

Each script's usage is in its header comment. Read the header rather than running --help: paste-to-frame.sh, serve-bootstrap.sh and bootstrap-on-frame.sh act on any argument.

Ground rules

Label every claim verified (seen on the device, with the date and SteamOS build) or inferred. The docs use this convention. Keep it, and move items out of docs/open-questions.md once they're checked.
When you learn something new about the Frame, record it in docs/how-the-frame-works.md (or the topic doc) in the same change.
The Frame's rootfs is read-only and SteamOS updates replace it. Put changes in ~ (--user Flatpaks, ~/.config) rather than steamos-readonly disable.
sudo on the Frame asks for the user's Developer Mode password. Hand those steps to the user (open Terminal) and keep automation to non-sudo commands.
The Mac uses BSD userland and zsh (no timeout, use head -n).
First-party first. For any new capability, investigate the first-party way before anything else: Valve (SteamOS, Steam, Steam Link), Apple (the Mac and iPhone), and KDE (the Frame's desktop is Plasma). It's usually the best answer. If it isn't, write down why not. If it is, find what Frame Control can do to make it easier to set up (install over SSH, pre-seed settings, pair automatically, tell the user the one setting to turn on).

3.4 KiB Raw Blame History

Steam Frame

Start here

Ground rules

3.4 KiB

Raw Blame History