Mac in the headset: USB route keeps a configured host-key alias and falls back to the network

Review round 12: a failed USB-C tunnel now retries the normal path; an
existing HostKeyAlias wins; --host with --usb is rejected; the Steam
desktop-streaming claim is now 'untested' (Valve documents the desktop
showing when a game loses focus); the Show/cleanup overlap test blocks
for real (it fails without the lock). Verified live: with the cable out,
the route is the normal path.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
saphidandClaude Opus 5.5 committed 2026-09-28 21:39:40 +10:00
1 parent 4fae62ba14
commit bda9d77d2d
4 files changed
+91 -49

No files matched your search

+13 -7
View File
@@ -18,7 +18,7 @@ Checked 2026-09-28.
| Goal | First-party option | Chosen? | Why |
|---|---|---|---|
| One Mac screen in the headset | **Apple Screen Sharing** (VNC) → Remmina (Remmina 1.4.43 is already installed on this Frame) | Kept as the fallback (`panel-on-frame.sh mac-screen`) | It's the closest to first-party and needs nothing new. But VNC sends compressed tiles rather than video, so moving content is slow: noticeable lag even on a good 5 GHz link (**verified** 2026-09-27, see [streaming.md](streaming.md)), and the Mac's pointer isn't in the picture without a helper. It shows only whole screens |
| One Mac screen | **Steam Remote Play**, Mac as host (Valve) | For Mac games only; see [Steam's own streaming](#steams-own-streaming) | The Mac's and the Frame's Steam clients already find each other (**verified**), but Remote Play streams a game, not the desktop or a window, and a Mac can't host the Frame's VR streaming |
| One Mac screen | **Steam Remote Play**, Mac as host (Valve) | For Mac games only; see [Steam's own streaming](#steams-own-streaming) | The Mac's and the Frame's Steam clients already find each other (**verified**). But Remote Play streams a game (the whole desktop only while the game is out of focus, untested from a Mac), never single windows, and a Mac can't host the Frame's VR streaming |
| One Mac screen | **AirPlay** (Apple) | No | Apple licenses AirPlay receivers only to TV and speaker makers, and nothing official runs on Linux. UxPlay is an unofficial receiver, and it mirrors a whole screen, not single windows |
| One Mac screen | **Sidecar / Mac Virtual Display** (Apple) | No | These work only with an iPad or Apple Vision Pro |
| **Each Mac window as its own panel** | None | – | No first-party way does this: Apple's per-app streaming is only for Vision Pro, and Valve's desktop streaming needs a Windows SteamVR host. So Frame Control does it itself |
@@ -68,11 +68,14 @@ The Frame is built around Steam streaming, so this was checked first
closing as the game loads
([Steam forum](https://steamcommunity.com/groups/homestream/discussions/1/574921459914429988/),
**reported**).
- **The Mac desktop or single windows through Steam: no known way.** Remote
Play streams a running game. The desktop trick people use on Windows (a
non-Steam shortcut to `explorer.exe`) has no known Mac equivalent
(**reported**; nothing found for macOS). So Frame Control's own stream
stays the way to see Mac windows.
- **The Mac desktop through Steam: untested; single windows: no.** Valve
says Remote Play shows the host's desktop when the game loses focus
([Steam Remote Play FAQ](https://help.steampowered.com/en/faqs/view/0689-74B8-92AC-10F2),
**documented**), so a whole Mac screen may be reachable by starting a
game, then switching away from it. Nobody has tried that from a Mac
host. It would still be one screen in one panel: Remote Play has
nothing like one panel per Mac window, so Frame Control's own stream
stays the way to see separate windows.
- **What Steam's work did give us: the USB-C link.** Plugged into the Mac,
the Frame appears as a network port called "Steam Frame". Steam's Remote
Play discovery uses it, and so does Frame Control's stream now (see
@@ -84,7 +87,10 @@ Next steps, once Steam on the Frame is healthy:
library, over Wi-Fi and over USB-C.
2. Record whether it starts, how it's shown, and its latency. Steam's
streaming overlay shows this; our benchmark can't measure it.
3. If it works, Frame Control's Games page could offer "Stream from the
3. While streaming, switch away from the game on the Mac, and see whether
the Mac's desktop appears in the headset, and whether its keyboard and
pointer work.
4. If it works, Frame Control's Games page could offer "Stream from the
Mac" for Mac-installed games.
## How it works
+2
View File
@@ -808,6 +808,8 @@ def main():
args = p.parse_args()
# The agent keeps the last 4096 frames (Stats.swift) and results are read
# at the end: at 60 fps that's 68 s, less warm-up and the reporting wait.
if args.cmd != "compare" and args.host and args.usb:
p.error("--host and --usb pick the path two ways; use one")
if args.cmd != "compare" and args.warmup + args.duration > 60:
p.error("--warmup plus --duration can be at most 60 s (the agent keeps 4096 frames)")
+34 -13
View File
@@ -104,21 +104,24 @@ class Helpers(unittest.TestCase):
self.assertEqual(len(calls), 1)
# A Show waits while the cleanup checks and runs pkill.
mv.launching = 0
mv._show = lambda *a: "shown"
started = threading.Event()
in_pkill, release, entered = threading.Event(), threading.Event(), threading.Event()
def slow_pkill(remote, **kw):
started.set()
time.sleep(0.3)
calls.append("after " + remote)
mv.run = slow_pkill
def blocking_pkill(remote, **kw):
in_pkill.set()
release.wait(5)
mv.run = blocking_pkill
mv._show = lambda *a: entered.set() or "shown"
with mock.patch.object(frame_macview.time, "sleep"):
t = threading.Thread(target=mv._end_viewer_browser, args=(mv.shows,))
t.start()
started.wait(2)
self.assertEqual(mv.show("window:5"), "shown") # blocked until the pkill finished
self.assertTrue(calls[-1].startswith("after "))
t.join()
cleanup = threading.Thread(target=mv._end_viewer_browser, args=(mv.shows,))
cleanup.start()
self.assertTrue(in_pkill.wait(2))
shower = threading.Thread(target=mv.show, args=("window:5",))
shower.start()
self.assertFalse(entered.wait(0.3), "Show started while the cleanup held the lock")
release.set()
self.assertTrue(entered.wait(2))
cleanup.join()
shower.join()
def test_tunnel_prefers_the_usb_c_network_when_plugged_in(self):
mv = frame_macview.MacView(["ssh"], lambda remote, **kw: "13: usb0 inet 10.86.200.233/29 scope global", "frame")
@@ -127,6 +130,9 @@ class Helpers(unittest.TestCase):
mock.patch.object(frame_macview.subprocess, "run", return_value=ssh_g):
self.assertEqual(mv._usb_route(), ["-o", "HostName=10.86.200.233", "-o", "HostKeyAlias=frame.example.ts.net"])
conn.assert_called_once_with(("10.86.200.233", 22), timeout=1)
ssh_g.stdout = "hostname frame.example.ts.net\nhostkeyalias paired-frame\n" # a configured alias wins
conn.side_effect = None
self.assertIn("HostKeyAlias=paired-frame", mv._usb_route())
conn.side_effect = OSError("unplugged")
self.assertEqual(mv._usb_route(), [])
mv.run = lambda remote, **kw: "" # no usb0
@@ -135,6 +141,21 @@ class Helpers(unittest.TestCase):
mv.run = lambda remote, **kw: self.fail("no ssh when USB is off")
self.assertEqual(mv._usb_route(), [])
def test_a_failed_usb_tunnel_falls_back_to_the_network(self):
mv = frame_macview.MacView(["ssh"], lambda *a, **k: "", "frame")
mv._usb_route = lambda: ["-o", "HostName=10.86.200.233"]
tried = []
def attempt(via, ports):
tried.append(list(via))
mv._last_tunnel_error = "Connection refused"
return not via # USB fails, the normal path works
mv._open_tunnel = attempt
mv.tunnel_up = lambda: False
mv.ensure_tunnel()
self.assertEqual(tried, [["-o", "HostName=10.86.200.233"], []])
self.assertEqual(mv.route, "network")
class WS:
"""A minimal WebSocket client (masked frames out, plain frames in)."""
+42 -29
View File
@@ -229,37 +229,49 @@ class MacView:
ports = [self.remote_port] if self.remote_port else list(REMOTE_PORTS)
if self.remote_port and allow_new_port:
ports += [p for p in REMOTE_PORTS if p != self.remote_port]
via = self._usb_route()
self.route = "usb" if via else "network"
for port in ports:
proc = subprocess.Popen([*self.tunnel_ssh, *via, "-o", "ExitOnForwardFailure=yes",
"-o", "ServerAliveInterval=5", "-o", "ServerAliveCountMax=3", "-N",
"-R", f"127.0.0.1:{port}:127.0.0.1:{self.port}", self.frame],
stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE, text=True)
# A taken port makes ssh exit once it's connected; a working
# tunnel answers the agent's status from the Frame's side.
ok = False
for _ in range(15):
time.sleep(0.4)
if proc.poll() is not None:
break
if self._probe(port):
ok = True
break
if ok:
self.tunnel, self.remote_port = proc, port
self.track(proc)
self._supervise()
usb = self._usb_route()
# USB-C first when it's there; if that fails (unplugged just now,
# something else at that address), the normal path.
for via in ([usb, []] if usb else [[]]):
self.route = "usb" if via else "network"
if self._open_tunnel(via, ports):
return
if proc.poll() is None:
proc.terminate()
proc.wait()
last = (proc.stderr.read() or "").strip()
if "forward" not in last.lower():
break # not a port clash: the Frame is unreachable
last = self._last_tunnel_error
raise MacViewError(f"Couldn't open a tunnel from {self.frame} to this Mac: {last or 'no answer through it'}")
def _open_tunnel(self, via, ports):
"""Tries the ports on one route; True once the tunnel answers. With self.lock held."""
last = ""
for port in ports:
proc = subprocess.Popen([*self.tunnel_ssh, *via, "-o", "ExitOnForwardFailure=yes",
"-o", "ServerAliveInterval=5", "-o", "ServerAliveCountMax=3", "-N",
"-R", f"127.0.0.1:{port}:127.0.0.1:{self.port}", self.frame],
stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE, text=True)
# A taken port makes ssh exit once it's connected; a working
# tunnel answers the agent's status from the Frame's side.
ok = False
for _ in range(15):
time.sleep(0.4)
if proc.poll() is not None:
break
if self._probe(port):
ok = True
break
if ok:
self.tunnel, self.remote_port = proc, port
self.track(proc)
self._supervise()
return True
if proc.poll() is None:
proc.terminate()
proc.wait()
last = (proc.stderr.read() or "").strip()
if "forward" not in last.lower():
break # not a port clash: the Frame is unreachable this way
self._last_tunnel_error = last
return False
def _usb_route(self):
"""ssh options to reach the Frame over its USB-C network, or [].
@@ -285,7 +297,8 @@ class MacView:
alias = self.frame
try:
cfg = subprocess.run(["ssh", "-G", self.frame], capture_output=True, text=True, timeout=5).stdout
alias = next((line.split()[1] for line in cfg.splitlines() if line.startswith("hostname ")), alias)
opts = dict(line.split(None, 1) for line in cfg.splitlines() if " " in line)
alias = opts.get("hostkeyalias") or opts.get("hostname") or alias # a configured alias wins
except (OSError, subprocess.SubprocessError):
pass
return ["-o", f"HostName={ip}", "-o", f"HostKeyAlias={alias}"]