diff --git a/README.md b/README.md index fb40d32..d42fb43 100644 --- a/README.md +++ b/README.md @@ -222,7 +222,7 @@ Frame's software fits together, all checked against a real headset and labelled | [Frame Control for iPhone](docs/iphone.md) | The iPhone and iPad app, how it runs the server on the Frame, pairing | | [Recovery and OS images](docs/recovery-and-images.md) | Where to download the Frame's OS, what's inside, testing without the headset | | [AI agents and assistant](docs/agents.md) | Key-free MCP tools, human approvals, and an opt-in assistant panel | -| [Testing](docs/testing.md) | Unit tests, end-to-end tests against a fake Frame in Docker, and the headset smoke test | +| [Testing](docs/testing.md) | Unit tests, end-to-end tests against a fake Frame in Docker, the headset smoke test and a Windows test VM | | [Open questions](docs/open-questions.md) | What's still unchecked |
diff --git a/docs/streaming.md b/docs/streaming.md index 773e698..b810934 100644 --- a/docs/streaming.md +++ b/docs/streaming.md @@ -21,6 +21,15 @@ The confidence labels are the same as in [ssh.md](ssh.md). | **ADB + scrcpy (Lepton only)** | A mirror of the Android container | **Guess** | `brew install scrcpy android-platform-tools`, then `adb connect frame.local:5555` while Lepton Development is running ([adb_lepton](https://partner.steamgames.com/doc/steamhardware/steamframe/adb_lepton)), then `scrcpy`. This only shows Android apps, not SteamOS. | | VNC server on the Frame (krfb / wayvnc) | A mirror of the Plasma desktop | **Inferred (SteamOS)** | Deck users run krfb in Desktop Mode ([one.vg](https://one.vg/blog/remote-control-your-steam-deck)). On the Frame, the in-headset desktop is a virtual screen, and krfb isn't known to be preinstalled. RDP and Steam Link cover this case, so it's not recommended. | +**RDP from Windows, verified 2026-09-30:** Windows 11 25H2's Remote Desktop +(`mstsc`) against BUILD_ID 20260925.6191901. xrdp picks TLS, not NLA, so +Remote Desktop never asks for a user or password. It warns that the certificate +(`www.xrdp.org`) can't be verified. After **Yes**, xrdp shows its own "Login to +frame" box with the username blank. Any user but `steamos` gets "User does not +exist, or could not be authenticated". Signing in as `steamos` with the +Developer Mode password is still unchecked: the Frame's log showed no +successful xrdp login up to that date. + **Recommendation for A:** start with Steam Link for macOS, because Valve documents it. Use Windows App (RDP) when you want a proper Linux desktop on the Mac with keyboard, mouse, and clipboard. diff --git a/docs/testing.md b/docs/testing.md index 82a1826..61c4168 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -8,6 +8,7 @@ A fourth, a SteamOS VM, may come later ([issue #6](https://github.com/saphid/ste | Unit tests (`tests/*.py`) | `python3 -m unittest discover -s tests` | Nothing | Parsing, validation, request guards; SSH and HTTP are mocked | | Fake Frame (`tests/e2e`) | `scripts/e2e.sh` | Linux with Docker | The real server and scripts against a container that behaves like a Frame | | Headset smoke test | `scripts/frame-smoke.sh` | A Frame on the `frame` alias | Install, launch and remove on the real device, recorded with its BUILD_ID | +| [Windows test VM](#windows-test-vm) | `scripts/windows-vm.sh` | A Linux machine with KVM and Docker | The Windows build on a real Windows desktop, against a real Frame when needed | ## Unit tests @@ -163,6 +164,77 @@ refuses ids with a hyphen (`missing/invalid arguments`), which the fake had accepted. The fake now refuses them the same way, and Frame Control makes ids Steam accepts. +## Windows test VM + +The unit tests run on Windows in CI, but the app itself doesn't. Anything that +depends on the Windows desktop (Remote Desktop, the installer, the bundled +Python, file dialogs) needs a real Windows machine. This is a Windows 11 VM in a +[dockur/windows](https://github.com/dockur/windows) container on a Linux machine +with KVM, driven from the Mac with `scripts/windows-vm.sh`. + +Setting it up, once, on the Linux machine: + +- **Windows comes from Microsoft.** The container downloads the Windows 11 + image from Microsoft on first start. Windows runs unactivated, which is fine + for testing. Don't use activation workarounds or third-party Windows images. +- **Publish its ports on loopback only.** Map `127.0.0.1:2222:22` (SSH), + `127.0.0.1:8006:8006` (the web console) and, if you need it, + `127.0.0.1:13389:3389`. The Mac reaches them through `ssh -J`. Use + `restart: "no"` and `stop_grace_period: 2m` so it only runs when someone is + testing, and a `docker stop` shuts Windows down cleanly. +- **Give it SSH on first sign-in.** The container runs `/oem/install.bat` + once. Have it add the OpenSSH Server capability, start `sshd`, set PowerShell + as its default shell, and put a dedicated public key (for example + `~/.ssh/id_ed25519_winvm` on the Mac) in + `C:\ProgramData\ssh\administrators_authorized_keys`. +- Give it 4 cores, 8 GB of memory and a 32 GB disk. That's enough for the app + and the tests. + +Using it, with `WINVM_HOST` set to the Linux machine's ssh alias: + +```sh +scripts/windows-vm.sh up # start it and wait for SSH (1-2 minutes) +scripts/windows-vm.sh put Frame-Control-Setup-x64.exe +scripts/windows-vm.sh ps 'Start-Process "$env:USERPROFILE\Downloads\Frame-Control-Setup-x64.exe" /S -Wait' +scripts/windows-vm.sh shot screen.png # what's on its screen +scripts/windows-vm.sh click 723 359 # screen pixels, as in the screenshot +scripts/windows-vm.sh keys s t e a m o s ret # QEMU key names +scripts/windows-vm.sh down # shut Windows down +``` + +- **Clicks go through a scheduled task.** Commands over SSH run in a + session with no desktop, so `click` and `scroll` write the position to a + file, and a scheduled task running as the signed-in user replays it. The + VM's screen must be signed in; it is after `up`. QEMU's own `mouse_move` is + relative and drifts, so the script doesn't use it. +- **Screenshots may not show the pointer.** Check the result of a click (a + menu that opens, a button that changes) rather than the pointer's position. +- **Windows' `ssh` waits for stdin.** The script closes it for every command. + Do the same if you run `ssh` in the VM by hand. +- **Starting the app.** Run Frame Control in the signed-in session, not over + SSH. Use its Start-menu shortcut via `click`, or a scheduled task like the + one `click` uses. + +**Testing against a real Frame.** The VM reaches the headset on the LAN like +any other computer. Set up Frame Control in the VM once, then: + +```sh +scripts/windows-vm.sh frame-key add # let the VM's key into the headset +# ... test ... +scripts/windows-vm.sh frame-key remove # and take it out again +``` + +`frame-key` tags the key `windows-vm-test` in the headset's +`authorized_keys`, and `remove` deletes only that line. Follow the shared-device +procedure in [Headset smoke test](#headset-smoke-test) before installing or +launching anything on the headset. + +**Verified 2026-09-30:** Windows 11 Pro 25H2 (build 26200) against a Frame on +BUILD_ID 20260925.6191901. The script was used to install Frame Control 0.4.0, +connect it to the Frame and reproduce a Remote Desktop report from +screenshots. It also took Windows-side logs, and `frame-key` left +`authorized_keys` byte-for-byte as it was. + ## Owned media player `tests/test_media.py` covers layout evidence and overrides, OU eye ordering, diff --git a/scripts/windows-vm.sh b/scripts/windows-vm.sh new file mode 100755 index 0000000..441504d --- /dev/null +++ b/scripts/windows-vm.sh @@ -0,0 +1,128 @@ +#!/usr/bin/env zsh +# Mac or Linux: drive a Windows 11 test VM for Frame Control's Windows build. +# The VM is a dockur/windows container on another machine; this reaches it over +# SSH through that machine. See docs/testing.md#windows-test-vm. +# +# Usage: scripts/windows-vm.sh [args] +# up | down | status start it (waits for SSH), shut Windows down cleanly, show state +# ps '' run PowerShell as the VM's user +# put [] copy a file in (default: the user's Downloads) +# shot save the VM's screen +# click click screen pixel x,y in the signed-in session +# scroll turn the wheel n notches at x,y (negative scrolls down) +# keys ... type QEMU key names: a, shift-a, ret, tab, esc, spc ... +# frame-key add|remove let the VM's Frame Control key into the headset (`frame` +# alias) for a test run, then take it out again +# +# Set WINVM_HOST to the ssh alias of the machine running the container. Optional: +# WINVM_CONTAINER (frame-winvm), WINVM_USER (frame), WINVM_PORT (2222: the VM's +# sshd, published on that machine's loopback), WINVM_KEY (~/.ssh/id_ed25519_winvm). +set -euo pipefail +setopt extendedglob + +host=${WINVM_HOST:?set WINVM_HOST to the ssh alias of the machine running the VM} +ctr=${WINVM_CONTAINER:-frame-winvm} +user=${WINVM_USER:-frame} +port=${WINVM_PORT:-2222} +key=${WINVM_KEY:-$HOME/.ssh/id_ed25519_winvm} +opts=(-o ConnectTimeout=20 -o StrictHostKeyChecking=accept-new + -o UserKnownHostsFile=${TMPDIR:-/tmp}/windows-vm-known_hosts -i $key -J $host) +TAG=windows-vm-test # comment on the VM's key in the headset's authorized_keys + +die() { print -u2 "windows-vm: $*"; exit 1 } +int() { [[ $1 == (-|)<-> ]] || die "not a whole number: $1" } + +# Windows' OpenSSH waits for stdin to close, so it always gets /dev/null. The +# script travels UTF-16 base64-encoded, so no quoting survives two shells. +vm_ps() { + local b64=$(print -rn -- "\$ProgressPreference = 'SilentlyContinue'"$'\n'"$1" | + iconv -f UTF-8 -t UTF-16LE | base64 | tr -d '\n') + ssh $opts -p $port $user@127.0.0.1 "powershell -NoProfile -NonInteractive -EncodedCommand $b64" /dev/null } + +# Input has to come from the signed-in desktop session, not SSH's session 0, so +# a scheduled task running as the user replays one click or wheel turn written +# to input.txt, then deletes the file to say it's done. +INPUT_PS1='$a = (Get-Content "$PSScriptRoot\input.txt").Trim() -split "\s+" +Add-Type -Namespace WinVm -Name Input -MemberDefinition @" +[DllImport("user32.dll")] public static extern bool SetProcessDPIAware(); +[DllImport("user32.dll")] public static extern bool SetCursorPos(int x, int y); +[DllImport("user32.dll")] public static extern void mouse_event(uint flags, int dx, int dy, int data, System.IntPtr extra); +"@ +[WinVm.Input]::SetProcessDPIAware() | Out-Null +[WinVm.Input]::SetCursorPos([int]$a[0], [int]$a[1]) | Out-Null +Start-Sleep -Milliseconds 150 +if ($a[2] -eq "click") { + [WinVm.Input]::mouse_event(0x2, 0, 0, 0, [IntPtr]::Zero); Start-Sleep -Milliseconds 60 + [WinVm.Input]::mouse_event(0x4, 0, 0, 0, [IntPtr]::Zero) +} else { [WinVm.Input]::mouse_event(0x800, 0, 0, 120 * [int]$a[3], [IntPtr]::Zero) } +Remove-Item "$PSScriptRoot\input.txt"' + +pointer() { # x y click|wheel [notches] + local b64=$(print -rn -- $INPUT_PS1 | base64 | tr -d '\n') + vm_ps '$d = Join-Path $env:LOCALAPPDATA "windows-vm" +New-Item -ItemType Directory -Force $d | Out-Null +[IO.File]::WriteAllText("$d\input.ps1", [Text.Encoding]::UTF8.GetString([Convert]::FromBase64String("'$b64'"))) +$act = New-ScheduledTaskAction -Execute powershell.exe -Argument "-NoProfile -WindowStyle Hidden -ExecutionPolicy Bypass -File `"$d\input.ps1`"" +$who = New-ScheduledTaskPrincipal -UserId $env:USERNAME -LogonType Interactive +Register-ScheduledTask -TaskName WindowsVmInput -Action $act -Principal $who -Force | Out-Null +Set-Content "$d\input.txt" "'"$*"'" +Start-ScheduledTask -TaskName WindowsVmInput +foreach ($i in 1..50) { if (-not (Test-Path "$d\input.txt")) { exit 0 }; Start-Sleep -Milliseconds 200 } +Remove-Item "$d\input.txt" -ErrorAction SilentlyContinue +Write-Error "no input after 10 s: is $env:USERNAME signed in on the VM screen?"; exit 1' +} + +(( $# )) || die "usage: see the top of $0" +cmd=$1; shift +case $cmd in + up) + ssh $host "docker start $ctr" >/dev/null + for i in {1..60}; do + vm_ps 'exit 0' 2>/dev/null && { print "up"; exit 0 } + sleep 5 + done + die "Windows didn't answer on SSH within 5 minutes" ;; + down) # the container turns SIGTERM into an ACPI shutdown and waits for Windows + ssh $host "docker stop -t 150 $ctr" >/dev/null && print "down" ;; + status) ssh $host "docker ps -a --filter 'name=^$ctr\$' --format '{{.Names}}: {{.Status}}'" ;; + ps) (( $# == 1 )) || die "usage: ps ''"; vm_ps "$1" ;; + put) + [[ -f ${1:-} ]] || die "usage: put []" + scp -q $opts -P $port $1 "$user@127.0.0.1:${2:-C:/Users/$user/Downloads}/${1:t}" " + print "screendump /tmp/windows-vm-shot.ppm" | monitor + sleep 1 + ssh $host "docker exec $ctr sh -c 'cat /tmp/windows-vm-shot.ppm && rm /tmp/windows-vm-shot.ppm'" | python3 -c ' +import re, struct, sys, zlib +d = sys.stdin.buffer.read() +m = re.match(rb"P6\s+(\d+)\s+(\d+)\s+255\s", d) or sys.exit("windows-vm: no screen dump") +w, h = int(m[1]), int(m[2]); px = d[m.end():] +raw = b"".join(b"\0" + px[y * w * 3:(y + 1) * w * 3] for y in range(h)) +chunk = lambda t, b: struct.pack(">I", len(b)) + t + b + struct.pack(">I", zlib.crc32(t + b)) +open(sys.argv[1], "wb").write(b"\x89PNG\r\n\x1a\n" + chunk(b"IHDR", struct.pack(">IIBBBBB", w, h, 8, 2, 0, 0, 0)) + + chunk(b"IDAT", zlib.compress(raw)) + chunk(b"IEND", b"")) +' $1 + print $1 ;; + click) (( $# == 2 )) || die "usage: click "; int $1; int $2; pointer $1 $2 click ;; + scroll) (( $# == 3 )) || die "usage: scroll "; int $1; int $2; int $3; pointer $1 $2 wheel $3 ;; + keys) + (( $# )) || die "usage: keys ..." + for k; do [[ $k == [a-z0-9_.,/=-]## ]] || die "not a QEMU key name: $k"; done + for k; do print "sendkey $k"; done | monitor ;; + frame-key) + pub=$(vm_ps 'Get-Content (Join-Path $env:USERPROFILE ".ssh\id_ed25519_frame.pub")' | tr -d '\r') + pub=${${(z)pub}[1,2]} + [[ $pub == ssh-ed25519\ * ]] || die "the VM has no Frame Control key yet: run Set Up Connection in the app first" + case ${1:-} in + add) ssh frame "grep -qxF '$pub $TAG' ~/.ssh/authorized_keys || echo '$pub $TAG' >> ~/.ssh/authorized_keys" \$f.tmp; chmod 600 \$f.tmp; mv \$f.tmp \$f"