Mac in the headset: stream Mac windows and screens into the Frame as panels

Frame Control can now show any Mac window, or a whole screen, as its own
SteamVR panel on the Steam Frame (Tools -> Mac in the headset, macOS only).
Place it with the SteamVR dashboard; the laser clicks and scrolls, and the
Mac's own keyboard types.

- mac/frame-mac-view (Swift, no dependencies): ScreenCaptureKit capture per
  window or display, VideoToolbox H.264 with low-latency rate control (JPEG
  fallback), a loopback HTTP/WebSocket server, CGEvent/AX input playback,
  and a display-awake assertion while anyone watches.
- ui/frame_macview.py: starts the agent, runs an ssh -R tunnel with a
  supervisor that reopens it on the same port, and launches a Chromium app
  window per stream on gamescope's :0, tagged with STEAM_GAME for its own panel.
- Frame Control's key never leaves the Mac: viewers get single-use,
  per-source tickets and reconnect keys that Stop revokes.
- ui/mac-view.html: WebCodecs decode, keyframe recovery, pointer/wheel/keys back.
- Bundled in the Mac app build; tests/test_macview.py builds and drives the
  agent on macOS.

Verified on the Frame (build 20260925.6191901) with the test pattern: panel
in about 1.5 s, about 60 fps, Mac-to-window about 11-17 ms, tunnel recovery
in 4 s. Laser input and real window capture still need a person in the headset.

Also commits the other thread's first-party rule (steam-frame skill) and
the first-party options table in docs/streaming.md.

Reviewed by GPT-6 Astra (xhigh, read-only) over six rounds; all findings fixed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

ui/frame_macview.py

@@ -0,0 +1,361 @@
+"""Mac in the headset: stream Mac windows or displays into the Steam Frame as
+panels you can place anywhere, with the laser, wheel and keys driving the Mac.
+
+Runs on the Mac, inside Frame Control's server. The pieces:
+
+- mac/bin/frame-mac-view (Swift, built from mac/frame-mac-view): captures with
+  ScreenCaptureKit, encodes with VideoToolbox, serves ui/mac-view.html and one
+  WebSocket per stream on 127.0.0.1, and plays input back with CGEvent.
+- An `ssh -R` tunnel, so the Frame reaches the agent on its own 127.0.0.1.
+  Every request carries a random token, so other programs on the Frame
+  (Android apps included) can't watch or drive the Mac.
+- A Chromium app window on the Frame per stream, on gamescope's X display
+  with its own STEAM_GAME id, which makes it its own SteamVR panel (see
+  docs/panels.md). Chromium XR (~/chromium-xr) is preferred because it's
+  built with H.264; Flathub Chromium is the fallback.
+
+Stdlib only. MacView gets a plain ssh argv for the tunnel (its own
+connection) and the server's `run(remote, stdin=, timeout=)` for commands.
+"""
+import json
+import os
+import re
+import secrets
+import shutil
+import subprocess
+import sys
+import threading
+import time
+import urllib.error
+import urllib.request
+import zlib
+from pathlib import Path
+from urllib.parse import urlencode
+
+HERE = Path(__file__).resolve().parent
+ROOT = HERE.parent
+PAGE = HERE / "mac-view.html"
+SOURCES = ROOT / "mac" / "frame-mac-view"
+AGENT = Path(os.environ.get("FRAME_MAC_VIEW") or ROOT / "mac" / "bin" / "frame-mac-view")
+REMOTE_PORTS = range(47900, 47920)
+SUPPORTED = sys.platform == "darwin"
+
+# Stream settings by name: long side in pixels, frames per second, H.264 bits
+# per pixel per frame, codec. JPEG is for a Frame browser without H.264.
+QUALITY = {
+    "sharp": {"max": 2560, "fps": 60, "bpp": 0.14, "codec": "h264"},
+    "balanced": {"max": 1920, "fps": 60, "bpp": 0.1, "codec": "h264"},
+    "light": {"max": 1280, "fps": 30, "bpp": 0.08, "codec": "h264"},
+    "compatible": {"max": 1280, "fps": 20, "bpp": 0.1, "codec": "jpeg"},
+}
+# Viewer windows are fitted inside a panel's size. gamescope made a 1280x720
+# request 1920x1080 anyway (verified 2026-09-28, build 20260925.6191901).
+PANEL_BOX = (1920, 1080)
+
+# Opens one viewer on gamescope's X display and gives its window its own panel
+# id. Args: appid url width height tag. The page puts "[tag]" in its title at
+# once, which is how its X window is found (Chromium may hand the URL to an
+# instance that's already running, so there's no process to follow).
+LAUNCH = r"""set -u
+appid=$1 url=$2 w=$3 h=$4 tag=$5
+export DISPLAY=:0 LC_ALL=C.UTF-8
+unset WAYLAND_DISPLAY
+if ! xprop -root GAMESCOPE_FOCUSABLE_WINDOWS >/dev/null 2>&1; then
+  echo "The headset isn't showing anything (gamescope's display :0 isn't up). Wake it and try again." >&2
+  exit 2
+fi
+common=(--ozone-platform=x11 --force-device-scale-factor=1 --no-first-run --no-default-browser-check
+  --password-store=basic --disable-session-crashed-bubble --noerrdialogs --disable-infobars
+  --disable-features=Translate,MediaRouter --autoplay-policy=no-user-gesture-required
+  "--window-size=$w,$h" "--app=$url")
+if [ -x "$HOME/chromium-xr/chrome" ]; then
+  cmd=("$HOME/chromium-xr/chrome" "--user-data-dir=$HOME/.local/share/frame-control/mac-view" "${common[@]}")
+elif flatpak info org.chromium.Chromium >/dev/null 2>&1; then
+  cmd=(flatpak run org.chromium.Chromium
+    "--user-data-dir=$HOME/.var/app/org.chromium.Chromium/data/frame-mac-view" "${common[@]}")
+else
+  echo "NO_BROWSER"
+  exit 3
+fi
+log=/tmp/frame-mac-view.log
+setsid nohup "${cmd[@]}" >>"$log" 2>&1 </dev/null &
+for _ in $(seq 1 60); do
+  sleep 0.5
+  # xprop, not xwininfo: in a C locale xwininfo can't print a non-ASCII title
+  # at all, while xprop escapes those bytes and leaves the ASCII tag readable.
+  for win in $(xwininfo -root -children 2>/dev/null | awk '/^ +0x/ {print $1}'); do
+    xprop -id "$win" _NET_WM_NAME WM_NAME 2>/dev/null | grep -qF "[$tag]" || continue
+    if xprop -id "$win" -f STEAM_GAME 32c -set STEAM_GAME "$appid" 2>/dev/null; then
+      echo "panel valve.steam.desktopgame.$appid window $win"
+      exit 0
+    fi
+  done
+done
+echo "The viewer started, but its window didn't appear within 30 s. Chromium's log:" >&2
+tail -n 15 "$log" >&2
+exit 1
+"""
+
+
+class MacViewError(Exception):
+    pass
+
+
+def panel_id(src):
+    """A stable panel id per source, in the range panel-on-frame.sh uses."""
+    return 2_001_000_000 + zlib.crc32(f"mac:{src}".encode()) % 1_000_000
+
+
+def fit(w, h, box=PANEL_BOX):
+    s = min(box[0] / max(w, 1), box[1] / max(h, 1))
+    return max(320, round(w * s)), max(200, round(h * s))
+
+
+class MacView:
+    def __init__(self, tunnel_ssh, run, frame, track=None):
+        self.tunnel_ssh = list(tunnel_ssh)
+        self.run = run
+        self.frame = frame
+        self.track = track or (lambda proc: None)  # the server ends these on exit
+        self.lock = threading.Lock()
+        self.token = secrets.token_urlsafe(24)
+        self.agent = None
+        self.port = None
+        self.tunnel = None
+        self.remote_port = None
+        self.supervisor = None
+        self.closing = False
+        self.shown = set()  # sources with a viewer out there, connected or retrying
+
+    # ---- the agent on this Mac ----
+
+    def unavailable(self):
+        """Why this can't work here, or None."""
+        if not SUPPORTED:
+            return "Streaming your computer into the headset needs macOS."
+        if not AGENT.exists() and not (SOURCES.exists() and shutil.which("xcrun")):
+            return "The Mac streaming helper is missing from this copy of Frame Control."
+        return None
+
+    def build(self):
+        if AGENT.exists() and not self._stale():
+            return
+        if not (SOURCES / "build.sh").exists():
+            if AGENT.exists():
+                return
+            raise MacViewError("The Mac streaming helper is missing.")
+        r = subprocess.run(["/bin/sh", str(SOURCES / "build.sh"), str(AGENT)], capture_output=True, text=True,
+                           stdin=subprocess.DEVNULL, timeout=600)
+        if r.returncode != 0:
+            raise MacViewError("Couldn't build the Mac streaming helper: " + (r.stderr or r.stdout).strip()[-400:])
+
+    def _stale(self):
+        try:
+            built = AGENT.stat().st_mtime
+            return any(p.stat().st_mtime > built for p in (SOURCES / "Sources").glob("*.swift"))
+        except OSError:
+            return False
+
+    def ensure_agent(self):
+        with self.lock:
+            if self.agent and self.agent.poll() is None:
+                return
+            reason = self.unavailable()
+            if reason:
+                raise MacViewError(reason)
+            self.build()
+            env = {**os.environ, "FRAME_MAC_VIEW_TOKEN": self.token}
+            # The same port as before when restarting, so a running tunnel still
+            # fits; otherwise (or if it's gone) whatever the system gives.
+            for port in dict.fromkeys([self.port or 0, 0]):
+                self.agent = subprocess.Popen([str(AGENT), "serve", "--port", str(port), "--page", str(PAGE),
+                                               "--exit-on-eof"], env=env, stdin=subprocess.PIPE,
+                                              stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
+                line = self.agent.stdout.readline()
+                m = re.search(r"listening on 127\.0\.0\.1:(\d+)", line)
+                if m:
+                    break
+                self.agent.kill()
+            else:
+                raise MacViewError(f"The Mac streaming helper didn't start: {line.strip() or 'no output'}")
+            if self.port != int(m.group(1)):
+                self._drop_tunnel()
+            self.port = int(m.group(1))
+            self.track(self.agent)
+            threading.Thread(target=self.agent.stdout.read, daemon=True).start()  # drain
+
+    def call(self, path, method="GET", **query):
+        """A request to the agent; starts it if needed."""
+        self.ensure_agent()
+        url = f"http://127.0.0.1:{self.port}{path}?{urlencode({**query, 'k': self.token})}"
+        req = urllib.request.Request(url, method=method, data=b"" if method == "POST" else None)
+        try:
+            with urllib.request.urlopen(req, timeout=10) as r:
+                return json.load(r)
+        except (urllib.error.URLError, OSError, ValueError) as e:
+            raise MacViewError(f"The Mac streaming helper didn't answer: {e}")
+
+    # ---- the tunnel from the Frame ----
+
+    def _drop_tunnel(self):
+        if self.tunnel and self.tunnel.poll() is None:
+            self.tunnel.terminate()
+        self.tunnel = None
+
+    def tunnel_up(self):
+        return self.tunnel is not None and self.tunnel.poll() is None
+
+    def ensure_tunnel(self, allow_new_port=False):
+        """Open the tunnel, on the port viewers already use if there is one.
+        A new port only when nobody is watching, since viewers can't move."""
+        with self.lock:
+            if self.tunnel_up():
+                return
+            last = ""
+            ports = [self.remote_port] if self.remote_port else list(REMOTE_PORTS)
+            if self.remote_port and allow_new_port:
+                ports += [p for p in REMOTE_PORTS if p != self.remote_port]
+            for port in ports:
+                proc = subprocess.Popen([*self.tunnel_ssh, "-o", "ExitOnForwardFailure=yes",
+                                         "-o", "ServerAliveInterval=5", "-o", "ServerAliveCountMax=3", "-N",
+                                         "-R", f"127.0.0.1:{port}:127.0.0.1:{self.port}", self.frame],
+                                        stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
+                                        stderr=subprocess.PIPE, text=True)
+                # A taken port makes ssh exit once it's connected; a working
+                # tunnel answers the agent's status from the Frame's side.
+                ok = False
+                for _ in range(15):
+                    time.sleep(0.4)
+                    if proc.poll() is not None:
+                        break
+                    if self._probe(port):
+                        ok = True
+                        break
+                if ok:
+                    self.tunnel, self.remote_port = proc, port
+                    self.track(proc)
+                    self._supervise()
+                    return
+                if proc.poll() is None:
+                    proc.terminate()
+                    proc.wait()
+                last = (proc.stderr.read() or "").strip()
+                if "forward" not in last.lower():
+                    break  # not a port clash: the Frame is unreachable
+            raise MacViewError(f"Couldn't open a tunnel from {self.frame} to this Mac: {last or 'no answer through it'}")
+
+    def _supervise(self):
+        """Reopen the tunnel on the same port after the headset sleeps or the
+        network drops, so viewers that are retrying find the Mac again."""
+        if self.supervisor and self.supervisor.is_alive():
+            return
+
+        def loop():
+            while not self.closing:
+                time.sleep(5)
+                if self.closing or self.tunnel_up() or not (self.agent and self.agent.poll() is None):
+                    continue
+                try:
+                    self.ensure_tunnel()
+                except MacViewError:
+                    pass  # still unreachable; try again shortly
+
+        self.supervisor = threading.Thread(target=loop, daemon=True)
+        self.supervisor.start()
+
+    def _probe(self, port):
+        """Whether the Frame reaches the agent through the tunnel on `port`."""
+        # /ping needs no key, so none appears on the Frame's command lines.
+        cmd = f"curl -s -m 2 'http://127.0.0.1:{port}/ping'"
+        try:
+            return self.run(cmd, timeout=8).strip() == "frame-mac-view"
+        except Exception:  # noqa: BLE001 - the server's Failure, timeouts: all mean "not yet"
+            return False
+
+    # ---- viewers on the Frame ----
+
+    def show(self, src, quality="balanced", width=None, height=None):
+        if src != "test" and not (src.startswith("window:") or src.startswith("display:")):
+            raise MacViewError("Pick a window or display to show.")
+        q = QUALITY.get(quality) or QUALITY["balanced"]
+        state = self.call("/status")
+        if src != "test" and not state.get("screen"):
+            raise MacViewError("Frame Control needs Screen Recording permission first (Allow… under Mac in the headset).")
+        self.shown -= set(state.get("finished", []))  # their Mac windows closed
+        if src in self.shown or any(st.get("src") == src for st in state.get("streams", [])):
+            # Showing it again replaces the old viewer, connected or not: this
+            # revokes its keys so it can't come back alongside the new one.
+            self.stop(src)
+        # Viewers that lost the tunnel keep retrying its old port, even though
+        # the agent no longer counts them, so only move when none are out there.
+        others = self.shown - {src}
+        self.ensure_tunnel(allow_new_port=not others)
+        appid = panel_id(src)
+        # Unique per launch, so a new window is never confused with an old one.
+        tag = "fc" + secrets.token_hex(4)
+        # A single-use ticket for this source, not Frame Control's key: the URL
+        # is visible in the Frame's process list.
+        ticket = self.call("/ticket", method="POST", src=src)["ticket"]
+        params = {"src": src, "t": ticket, "tag": tag, "codec": q["codec"], "max": q["max"], "fps": q["fps"],
+                  "bpp": q["bpp"]}
+        url = f"http://127.0.0.1:{self.remote_port}/view?{urlencode(params)}"
+        w, h = fit(width or 1280, height or 720)
+        args = " ".join(_quote(str(a)) for a in (appid, url, w, h, tag))
+        try:
+            out = self.run("bash -s -- " + args, stdin=LAUNCH, timeout=60)
+        except Exception as e:  # noqa: BLE001 - the server's Failure carries the Frame's words
+            if "NO_BROWSER" in (getattr(e, "stdout", "") or ""):
+                raise MacViewError("The Frame needs a browser for this: install Chromium (Tools → Linux apps, "
+                                   "org.chromium.Chromium) or Chromium XR.")
+            raise MacViewError(str(e))
+        self.shown.add(src)
+        return {"panel": f"valve.steam.desktopgame.{appid}", "src": src, "detail": out.strip()}
+
+    def stop(self, src=None):
+        if src:
+            self.shown.discard(src)
+        else:
+            self.shown.clear()
+        if not (self.agent and self.agent.poll() is None):
+            return {"closed": 0}
+        return self.call("/close", method="POST", **({"src": src} if src else {}))
+
+    def state(self):
+        reason = self.unavailable()
+        if reason:
+            return {"available": False, "reason": reason}
+        status = self.call("/status")
+        windows = self.call("/windows").get("windows", []) if status.get("screen") else []
+        displays = self.call("/displays").get("displays", [])
+        return {"available": True, "screen": status.get("screen", False),
+                "accessibility": status.get("accessibility", False), "streams": status.get("streams", []),
+                "windows": windows, "displays": displays, "tunnel": self.tunnel_up()}
+
+    def restart_agent(self):
+        """Only if it's missing a permission: a running stream would stop."""
+        with self.lock:
+            if not (self.agent and self.agent.poll() is None):
+                return
+        status = self.call("/status")
+        if status.get("screen") and status.get("accessibility"):
+            return
+        with self.lock:
+            self.agent.terminate()
+            self.agent.wait(5)
+
+    def request_permissions(self):
+        return self.call("/permissions", method="POST")
+
+    def shutdown(self):
+        self.closing = True
+        try:
+            self.stop()
+        except MacViewError:
+            pass
+        for proc in (self.tunnel, self.agent):
+            if proc and proc.poll() is None:
+                proc.terminate()
+
+
+def _quote(s):
+    return "'" + s.replace("'", "'\\''") + "'"

ui/index.html

@@ -309,6 +309,11 @@
   .game-card .bar > i { display: block; height: 100%; background: var(--action); }
   .f-0 { color: var(--muted); }
   /* ---- Android display ---- */
+  #mvQuality { background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; border-radius: 3px; padding: 4px 6px; font: inherit; font-size: 12px; }
+  .mv-windows { max-height: 420px; overflow-y: auto; }
+  .mv-perm { background: rgba(255,190,60,.08); border: 1px solid rgba(255,190,60,.25); border-radius: 4px; padding: 10px 12px; margin-bottom: 12px; font-size: 13px; }
+  .mv-perm .row { margin-top: 8px; }
+  .mv-live { color: var(--green, #7fd67f); font-size: 12px; }
   .disp select { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; border-radius: 3px;
                  padding: 8px 10px; font: inherit; font-size: 13px; }
   .disp select:focus { outline: none; border-color: var(--blue); }
@@ -642,6 +647,25 @@
         <div><a href="https://framedropvr.com" target="_blank">FrameDrop</a>: sideloader (Windows only for now)</div>
       </div>
     </section>
+
+    <section class="panel" id="macview" hidden>
+      <div class="shelf-head"><h2>Mac in the headset</h2><span class="count" id="mvCount"></span><span class="spacer"></span>
+        <select id="mvQuality" title="Picture quality and bandwidth">
+          <option value="sharp">Sharp</option><option value="balanced" selected>Balanced</option>
+          <option value="light">Light</option><option value="compatible">Compatible (JPEG)</option>
+        </select>
+        <button class="small" id="mvRefresh">Refresh</button></div>
+      <div class="mv-perm" id="mvPerm" hidden></div>
+      <div class="list" id="mvDisplays"></div>
+      <div class="shelf-head" style="margin-top:16px"><h2>Windows</h2><span class="count" id="mvWinCount"></span></div>
+      <div class="list mv-windows" id="mvWindows"><div class="sub">Loading…</div></div>
+      <div class="row" style="margin-top:12px">
+        <button class="small" data-mv="show" data-src="test" data-title="Test pattern">Test pattern</button>
+        <button class="small danger" data-mv="stopall">Stop all</button>
+      </div>
+      <div class="hint">Each window or screen becomes its own panel in the headset. Place it with the SteamVR dashboard
+        (Float in World, Move, Size). Point and click with the laser, scroll with the thumbstick, and type on the Mac's keyboard.</div>
+    </section>
   </div>
   </div>
 </main>
@@ -2107,10 +2131,64 @@ $("shotsRefresh").onclick = loadShots;
 $("shotsSaveNew").onclick = e => saveShots(shots.list.filter(s => !s.saved), e.currentTarget);
 $("shotsFolder").onclick = e => act($("shotsFolder").textContent, () => api("/api/open", { what: "shots" }), e.currentTarget);
 
+// ---- Mac in the headset: windows and displays as panels (ui/frame_macview.py) ----
+let mvSeq = 0;
+function mvRow(src, title, sub, w, h, streaming) {
+  return `<div class="item"><div class="grow"><div class="t">${esc(title)}</div>
+      <div class="s">${esc(sub)}${streaming ? ' · <span class="mv-live">in the headset</span>' : ""}</div></div>
+    ${streaming ? `<button class="small" data-mv="stop" data-src="${esc(src)}">Stop</button>`
+                : `<button class="small action" data-mv="show" data-src="${esc(src)}" data-title="${esc(title)}" data-w="${w}" data-h="${h}">Show</button>`}
+  </div>`;
+}
+async function loadMacView(restart) {
+  const seq = ++mvSeq;
+  let s, err;
+  try { s = await api("/api/macview" + (restart ? "?restart=1" : "")); } catch (e) { err = e; }
+  if (seq !== mvSeq) return;
+  const box = $("macview");
+  if (err) { box.hidden = false; return failed($("mvWindows"), err); }
+  if (!s.available) { box.hidden = true; return; }
+  box.hidden = false;
+  const live = new Set((s.streams || []).map(x => x.src));
+  $("mvCount").textContent = live.size ? `${live.size} showing` : "";
+  const perm = [];
+  if (!s.screen) perm.push(`Frame Control needs <b>Screen Recording</b> permission to show your windows.`);
+  if (!s.accessibility) perm.push(`Allow <b>Accessibility</b> too, so clicks and keys from the headset reach the Mac.`);
+  $("mvPerm").hidden = !perm.length;
+  $("mvPerm").innerHTML = perm.join(" ") + `<div class="row"><button class="small action" data-mv="perm">Allow…</button>
+    <span class="sub">Then press Refresh. macOS may ask you to reopen Frame Control.</span></div>`;
+  $("mvDisplays").innerHTML = (s.displays || []).map(d =>
+    mvRow(d.src, `Whole screen: ${d.name}`, `${d.w}×${d.h}${d.main ? " · main display" : ""}`, d.w, d.h, live.has(d.src))).join("");
+  const wins = s.windows || [];
+  $("mvWinCount").textContent = wins.length ? `${wins.length}` : "";
+  $("mvWindows").innerHTML = !s.screen ? `<div class="sub">Windows appear here once Screen Recording is allowed.</div>`
+    : wins.length ? wins.map(w => mvRow(w.src, w.title || w.app, `${w.title ? w.app + " · " : ""}${w.w}×${w.h}`, w.w, w.h, live.has(w.src))).join("")
+    : `<div class="sub">No windows open on this Mac's current desktop.</div>`;
+}
+$("mvRefresh").onclick = () => loadMacView(true);
+$("macview").onclick = async e => {
+  const b = e.target.closest("[data-mv]"); if (!b) return;
+  const src = b.dataset.src;
+  if (b.dataset.mv === "show") {
+    await act(`Show ${b.dataset.title} in the headset`, async () => {
+      const r = await api("/api/macview", { action: "show", src, quality: $("mvQuality").value,
+                                            w: +b.dataset.w || undefined, h: +b.dataset.h || undefined });
+      return { message: `${b.dataset.title} is a panel in the headset now. Float it with the SteamVR dashboard.`, ...r };
+    }, b);
+  } else if (b.dataset.mv === "stop") {
+    await act("Stop showing it", () => api("/api/macview", { action: "stop", src }), b);
+  } else if (b.dataset.mv === "stopall") {
+    await act("Stop everything in the headset", () => api("/api/macview", { action: "stop" }), b);
+  } else if (b.dataset.mv === "perm") {
+    await act("Ask macOS for permission", () => api("/api/macview", { action: "permissions" }), b);
+  }
+  setTimeout(loadMacView, 800);
+};
+
 // ---- pages: #home, #games, #android, #tools (older section links still work) ----
 const PAGES = ["home", "games", "android", "tools"];
 const SECTION_PAGE = { view: "home", device: "home", shots: "home", library: "games", sideloaded: "games", getgames: "games",
-                       display: "android", transfer: "tools", apps: "tools", power: "tools" };
+                       display: "android", transfer: "tools", apps: "tools", power: "tools", macview: "tools" };
 let page = "home";
 function showPage() {
   const id = location.hash.slice(1);
@@ -2122,6 +2200,7 @@ function showPage() {
   document.title = page === "home" ? "Frame Control" : `${page[0].toUpperCase() + page.slice(1)} · Frame Control`;
   const section = !PAGES.includes(id) && id && $(id);
   if (section) section.scrollIntoView(); else window.scrollTo(0, 0);
+  if (page === "tools") loadMacView();
 }
 window.addEventListener("hashchange", showPage);
 // While a text field has focus, phones hide the bottom tab bar (see body.typing in the CSS).

ui/mac-view.html

@@ -0,0 +1,242 @@
+<!doctype html>
+<!-- Frame Control: one Mac window (or display) inside the Steam Frame.
+     Served by the Mac's frame-mac-view agent through an SSH tunnel and opened
+     on the Frame as its own Chromium app window, which gamescope turns into a
+     SteamVR panel. Video arrives over a WebSocket (H.264 Annex B for
+     WebCodecs, or JPEG); pointer, wheel and keys go back the same way. -->
+<html lang="en">
+<head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>Mac</title>
+<style>
+  html, body { margin: 0; height: 100%; background: #000; overflow: hidden; cursor: default; }
+  canvas { position: fixed; inset: 0; width: 100%; height: 100%; object-fit: contain; image-rendering: auto; }
+  #note { position: fixed; left: 50%; top: 16px; transform: translateX(-50%); max-width: 80%;
+          font: 15px/1.4 system-ui, sans-serif; color: #eee; background: rgba(20,22,26,.88);
+          padding: 8px 14px; border-radius: 10px; pointer-events: none; transition: opacity .4s; }
+  #note[hidden] { display: block; opacity: 0; }
+</style>
+</head>
+<body>
+<canvas id="c" width="16" height="9"></canvas>
+<div id="note">Connecting to the Mac…</div>
+<script>
+"use strict";
+const q = new URLSearchParams(location.search);
+// t: a single-use ticket from Frame Control. After the first connection the
+// Mac sends a reconnect key, kept only in memory.
+const src = q.get("src") || "test", ticket = q.get("t") || "", tag = q.get("tag") || "";
+let reconnectKey = "";
+const c = document.getElementById("c"), ctx = c.getContext("2d", { alpha: false, desynchronized: true });
+const note = document.getElementById("note");
+// Counters for Frame Control's tests (read over DevTools).
+const stats = window.stats = { frames: 0, keyFrames: 0, bytes: 0, dropped: 0, codec: "", errors: [], info: null };
+// Frame Control finds this window on the Frame by the tag in its title.
+document.title = tag ? `Mac [${tag}]` : "Mac";
+let failedStarts = 0;
+let ws = null, dec = null, codecString = "", needKey = true, closing = false, retry = 0, noteTimer = 0, lastError = "";
+
+function show(text, ms) {
+  note.textContent = text;
+  note.hidden = false;
+  clearTimeout(noteTimer);
+  if (ms) noteTimer = setTimeout(() => { note.hidden = true; }, ms);
+}
+
+function send(obj) { if (ws && ws.readyState === 1) ws.send(JSON.stringify(obj)); }
+let lastKeyAsk = 0;
+function askKeyFrame() {
+  const now = performance.now();
+  if (now - lastKeyAsk < 500) return;
+  lastKeyAsk = now;
+  send({ t: "key-frame" });
+}
+
+let hideNoteOnFrame = true;  // "Connecting…"/"Reconnecting…" go once video flows again
+function drawFrame(img, w, h) {
+  if (c.width !== w || c.height !== h) { c.width = w; c.height = h; }
+  ctx.drawImage(img, 0, 0);
+  stats.frames++;
+  if (hideNoteOnFrame) { hideNoteOnFrame = false; note.hidden = true; }
+}
+
+// ---- H.264 ----
+function startCode(b, i) { return b[i] === 0 && b[i + 1] === 0 && (b[i + 2] === 1 || (b[i + 2] === 0 && b[i + 3] === 1)); }
+function spsCodec(au) {
+  for (let i = 0; i + 7 < au.length; i++) {
+    if (!startCode(au, i)) continue;
+    const n = au[i + 2] === 1 ? i + 3 : i + 4;
+    if ((au[n] & 0x1f) === 7) return "avc1." + [au[n + 1], au[n + 2], au[n + 3]].map(b => b.toString(16).padStart(2, "0")).join("");
+    i = n;
+  }
+  return "";
+}
+function makeDecoder() {
+  if (dec) try { dec.close(); } catch (e) {}
+  codecString = "";
+  dec = new VideoDecoder({
+    output: frame => { drawFrame(frame, frame.displayWidth, frame.displayHeight); frame.close(); },
+    error: e => {
+      stats.errors.push(String(e.message || e));
+      needKey = true;
+      makeDecoder();
+      askKeyFrame();
+    },
+  });
+}
+function onH264(isKey, ts, au) {
+  if (isKey) {
+    const cs = spsCodec(au);
+    if (cs && (cs !== codecString || dec.state !== "configured")) {
+      if (dec.state === "closed") makeDecoder();
+      dec.configure({ codec: cs, optimizeForLatency: true, hardwareAcceleration: "no-preference" });
+      codecString = stats.codec = cs;
+    }
+    stats.keyFrames++;
+  }
+  if (dec.state !== "configured" || (needKey && !isKey)) { stats.dropped++; askKeyFrame(); return; }
+  // Behind: skip to the next keyframe rather than show old pictures late.
+  if (!isKey && dec.decodeQueueSize > 2) { needKey = true; stats.dropped++; askKeyFrame(); return; }
+  needKey = false;
+  dec.decode(new EncodedVideoChunk({ type: isKey ? "key" : "delta", timestamp: ts, data: au }));
+}
+
+// ---- JPEG ----
+let jpegBusy = false;
+function onJPEG(data) {
+  if (jpegBusy) { stats.dropped++; return; }
+  jpegBusy = true;
+  createImageBitmap(new Blob([data], { type: "image/jpeg" })).then(bmp => {
+    drawFrame(bmp, bmp.width, bmp.height);
+    bmp.close();
+  }).catch(e => stats.errors.push(String(e))).finally(() => { jpegBusy = false; });
+}
+
+async function pickCodec() {
+  const want = q.get("codec");
+  if (want === "jpeg") return "jpeg";
+  if (!("VideoDecoder" in window)) return "jpeg";
+  try {
+    const r = await VideoDecoder.isConfigSupported({ codec: "avc1.640028", optimizeForLatency: true });
+    return r.supported ? "h264" : "jpeg";
+  } catch (e) { return "jpeg"; }
+}
+
+async function connect() {
+  const codec = await pickCodec();
+  stats.codec = codec;
+  if (codec === "h264") makeDecoder();
+  needKey = true;
+  const p = new URLSearchParams({ src, codec, max: q.get("max") || "1920", fps: q.get("fps") || "60" });
+  if (reconnectKey) p.set("r", reconnectKey); else p.set("t", ticket);
+  if (q.get("bpp")) p.set("bpp", q.get("bpp"));
+  ws = new WebSocket(`ws://${location.host}/stream?${p}`);
+  ws.binaryType = "arraybuffer";
+  ws.onopen = () => { retry = 0; lastError = ""; };
+  ws.onmessage = ev => {
+    if (typeof ev.data === "string") return control(JSON.parse(ev.data));
+    const b = new Uint8Array(ev.data);
+    stats.bytes += b.length;
+    if (b.length < 10) return;
+    const isKey = b[0] === 1;
+    const ts = Number(new DataView(ev.data).getBigUint64(1));
+    const payload = b.subarray(9);
+    if (codec === "h264") onH264(isKey, ts, payload); else onJPEG(payload);
+  };
+  ws.onclose = () => {
+    ws = null;
+    if (closing) return;
+    // Refused before ever getting a key: the ticket expired or was revoked,
+    // and retrying can't help.
+    if (!reconnectKey && ++failedStarts >= 3) {
+      show("This view has expired. Press Show in Frame Control on the Mac to open it again.");
+      return;
+    }
+    hideNoteOnFrame = true;
+    // The Mac may be asleep or the tunnel restarting: keep trying.
+    retry = Math.min(retry + 1, 6);
+    // Keep the Mac's reason (a missing permission, a closed window) on screen.
+    show(lastError ? `${lastError} Retrying…` : "Lost the Mac. Reconnecting…");
+    setTimeout(connect, 500 * 2 ** retry);
+  };
+}
+
+function control(m) {
+  if (m.t === "hello") {
+    reconnectKey = m.r;
+    send({ t: "ack" });  // the ticket is spent only now
+  } else if (m.t === "info") {
+    stats.info = m;
+    const name = m.title && m.app && m.title !== m.app ? `${m.title} — ${m.app}` : (m.title || m.app || "Mac");
+    document.title = tag ? `${name} [${tag}]` : name;
+    if (!m.input) {
+      hideNoteOnFrame = false;  // a warning, not a connection notice: let it stay its 8 s
+      show("Clicks and keys need Accessibility permission on the Mac (Frame Control asks for it).", 8000);
+    }
+  } else if (m.t === "error") {
+    stats.errors.push(m.message);
+    lastError = m.message.replace(/\.?$/, ".");
+    show(m.message);
+  } else if (m.t === "close" || m.t === "closed") {
+    closing = true;
+    if (ws) ws.close();
+    show(m.reason ? `Stopped: ${m.reason}.` : "Stopped.");
+    window.close();
+  }
+}
+
+// ---- input ----
+// Where the picture sits inside the window (object-fit: contain letterboxes it).
+function toPicture(e) {
+  const r = c.getBoundingClientRect(), s = Math.min(r.width / c.width, r.height / c.height);
+  const w = c.width * s, h = c.height * s, left = r.left + (r.width - w) / 2, top = r.top + (r.height - h) / 2;
+  const x = (e.clientX - left) / w, y = (e.clientY - top) / h;
+  return { x, y, inside: x >= 0 && x <= 1 && y >= 0 && y <= 1 };
+}
+let pendingMove = null, moveQueued = false;
+addEventListener("pointermove", e => {
+  const p = toPicture(e);
+  if (!p.inside && !e.buttons) return;
+  pendingMove = { t: "m", e: "move", x: p.x, y: p.y };
+  if (!moveQueued) {
+    moveQueued = true;
+    requestAnimationFrame(() => { moveQueued = false; if (pendingMove) send(pendingMove); pendingMove = null; });
+  }
+});
+addEventListener("pointerdown", e => {
+  const p = toPicture(e);
+  if (!p.inside) return;
+  if (e.pointerId !== undefined) try { c.setPointerCapture(e.pointerId); } catch (err) {}
+  pendingMove = null;
+  send({ t: "m", e: "down", b: e.button < 0 ? 0 : e.button, x: p.x, y: p.y });
+  e.preventDefault();
+});
+addEventListener("pointerup", e => {
+  const p = toPicture(e);
+  send({ t: "m", e: "up", b: e.button < 0 ? 0 : e.button, x: p.x, y: p.y });
+});
+// Let go of any held button where the pointer is on the Mac, not at a corner.
+addEventListener("pointercancel", () => send({ t: "release" }));
+addEventListener("contextmenu", e => e.preventDefault());
+addEventListener("wheel", e => {
+  const p = toPicture(e), unit = e.deltaMode === 1 ? 16 : e.deltaMode === 2 ? innerHeight : 1;
+  send({ t: "wheel", dx: e.deltaX * unit, dy: e.deltaY * unit, x: p.x, y: p.y });
+  e.preventDefault();
+}, { passive: false });
+function mods(e) {
+  return ["shift", "ctrl", "alt", "meta"].filter(m => e[m + "Key"]);
+}
+function onKey(e, down) {
+  send({ t: "k", e: down ? "down" : "up", code: e.code, key: e.key, mods: mods(e) });
+  e.preventDefault();
+}
+addEventListener("keydown", e => onKey(e, true));
+addEventListener("keyup", e => onKey(e, false));
+addEventListener("blur", () => send({ t: "release" }));
+
+show("Connecting to the Mac…");
+connect();
+</script>
+</body>
+</html>

ui/server.py

@@ -39,6 +39,7 @@ sys.path.insert(0, str(Path(__file__).resolve().parent))
 import frame_android  # noqa: E402
 import frame_catalog  # noqa: E402
 import frame_host  # noqa: E402
+import frame_macview  # noqa: E402
 import frame_store  # noqa: E402
 import frame_titles  # noqa: E402
 import frame_webinstall  # noqa: E402
@@ -1213,10 +1214,50 @@ def _sweep_one(prefix, d):
         pass
 
 
+# ---- Mac in the headset (frame_macview.py) ----------------------------------
+
+# The tunnel gets its own connection: the shared master's options would win
+# over anything added after them.
+macview = frame_macview.MacView(["ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8"],
+                                lambda remote, stdin=None, timeout=30: ssh(remote, stdin=stdin, timeout=timeout),
+                                FRAME, track=_live_tunnels.add)
+
+
+def macview_state(query=None):
+    if LOCAL:
+        return {"available": False, "reason": "Runs on your computer, not the headset."}
+    try:
+        # Refresh restarts the helper, which picks up a permission just granted.
+        if (query or {}).get("restart") == ["1"]:
+            macview.restart_agent()
+        return macview.state()
+    except frame_macview.MacViewError as e:
+        raise Failure(str(e), 500)
+
+
+def macview_action(body):
+    """{action: show|stop|permissions, src, quality, w, h}."""
+    if LOCAL:
+        raise Failure("Runs on your computer, not the headset.", 400)
+    action = body.get("action")
+    try:
+        if action == "show":
+            w, h = body.get("w"), body.get("h")
+            return macview.show(str(body.get("src") or ""), str(body.get("quality") or "balanced"),
+                                int(w) if w else None, int(h) if h else None)
+        if action == "stop":
+            return macview.stop(body.get("src") or None)
+        if action == "permissions":
+            return macview.request_permissions()
+    except frame_macview.MacViewError as e:
+        raise Failure(str(e), 502)
+    raise Failure("unknown action", 400)
+
+
 POST = {"/api/android/display": android_display, "/api/android": android, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard,
         "/api/flatpak": flatpak, "/api/open": open_thing, "/api/shots/save": save_shots,
         "/api/webinstall/check": webinstall_check, "/api/webinstall/start": webinstall_start,
-        "/api/webinstall/cancel": webinstall_cancel}
+        "/api/webinstall/cancel": webinstall_cancel, "/api/macview": macview_action}
 
 
 # ---- HTTP ------------------------------------------------------------------
@@ -1336,6 +1377,8 @@ class Handler(BaseHTTPRequestHandler):
                                 "shared": frame_catalog.compat_db.shared()})
             elif path == "/api/android/catalog":
                 self.send_json({"apps": frame_catalog.catalog()})
+            elif path == "/api/macview":
+                self.send_json(macview_state(parse_qs(url.query)))
             elif path == "/api/status":
                 self.send_json(status({}))
             elif path == "/api/steam/owned":
@@ -1529,6 +1572,7 @@ def main():
         if not frame_host.WINDOWS:
             signal.signal(signal.SIGTERM, signal.SIG_IGN)
         webinstall_shutdown()
+        macview.shutdown()  # close the headset's viewers before the agent goes
         # The master was started with -N, so it stays up until told to exit.
         if CONTROL:
             subprocess.run([*MUX, "-O", "exit", FRAME], capture_output=True, stdin=subprocess.DEVNULL)