diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml index 4f6738f..654da64 100644 --- a/.github/workflows/checks.yml +++ b/.github/workflows/checks.yml @@ -27,11 +27,14 @@ jobs: esac done - name: Python compiles - run: python -m py_compile ui/*.py apk-catalog/*.py frame/android/*.py + run: | + python -m py_compile ui/*.py apk-catalog/*.py frame/android/*.py + # Valve's devkit-utils (vendored; run by the Frame's python3). Most have no .py suffix. + python -m py_compile $(find frame/devkit-utils -type f ! -name '*.*' ! -name LICENSE) frame/devkit-utils/devkit_utils/*.py - name: Server tests run: python -m unittest discover -s tests -v - name: App syntax - run: node --check app/main.js && node --check app/build/make-icon.js && node --check app/build/fetch-deps.js && node --check app/preload.js + run: node --check app/main.js && node --check app/build/make-icon.js && node --check app/build/fetch-deps.js && node --check app/preload.js && node --check app/install-link.js # The server runs on each desktop OS the app ships for, on the Python version # the app bundles (app/build/fetch-deps.js) and, on Ubuntu, a newer one. diff --git a/README.md b/README.md index 573e1ae..8be10f2 100644 --- a/README.md +++ b/README.md @@ -58,8 +58,8 @@ About 4,500 F-Droid apps rated for the Frame. One click installs each as its own -**📁 Files and clipboard**
-Drag files onto the window to send them. Send text or your clipboard straight to the headset's desktop. +**📁 Files, games and clipboard**
+Drag files onto the window to send them. Drop a game's .zip, folder or .exe to add it to the Steam library, with Proton or the Linux runtime picked for you. Send text or your clipboard straight to the headset's desktop. @@ -86,7 +86,8 @@ SSH, SFTP, Steam Link, remote desktop, volume, sleep, restart and shut down. Nothing is installed on the Frame for any of this: the app uses what SteamOS -already ships. [How each feature works](docs/frame-control.md). +already ships (sideloading a game copies Valve's own devkit scripts to +`~/devkit-utils`, as Valve's Devkit Client does). [How each feature works](docs/frame-control.md). ## Install @@ -147,13 +148,21 @@ computer. Connection**, which finds the headset, creates an SSH key, and asks for that password once in a terminal window. If it can't find the Frame, type the IP address from the Frame's Quick Settings. + + Before asking for the password it tries Valve's SteamOS devkit pairing: in + the headset, open Steam Settings → Developer → **Pair new host** and approve + the request, and no password is needed. (The service and the pairing-mode + step are verified on a Frame; the approval itself isn't yet. See + [SSH](docs/ssh.md#password-free-pairing-steamos-devkit-service).) 3. That's it. The app now reaches the headset whenever it's awake and on the same network. For anywhere else, see [Tailscale](docs/tailscale.md). **What it changes:** only what you click. Installs go to your user account on -the Frame (`--user` Flatpaks, Lepton instances, Steam downloads), and nothing +the Frame (`--user` Flatpaks, Lepton instances, Steam downloads, sideloaded +games in `~/devkit-game`), and nothing needs `sudo` except the power buttons. On your computer it adds a `Host frame` -entry to `~/.ssh/config` and a key at `~/.ssh/id_ed25519_frame`. +entry to `~/.ssh/config` and keys at `~/.ssh/id_ed25519_frame` and +`~/.ssh/id_rsa_frame_devkit` (the pairing service only takes RSA keys). ## Feedback @@ -177,6 +186,8 @@ Frame's software fits together, all checked against a real headset and labelled | [Scripts and headset setup](docs/scripts.md) | The command-line helpers, minimum typing, streaming options, floating panels | | [How the Frame works](docs/how-the-frame-works.md) | SteamVR → gamescope → Plasma, verified facts, debugging | | [Android apps (Lepton)](docs/apks.md) | Sideloading, the rated F-Droid catalogue, per-app instances | +| [Sideloading Linux and Windows games](docs/sideloading.md) | A .zip, folder or .exe as a Steam Devkit Game, runtime detection | +| [Install links for websites](docs/web-install.md) | `frame-control://install` links and manifests, the rules, a button to paste | | [Steam games](docs/steam-games.md) · [VR video](docs/vr-video.md) · [WebXR in Chromium](docs/webxr-chromium.md) | Installing and buying, watching VR180/360, the Chromium build | | [SSH](docs/ssh.md) · [Streaming](docs/streaming.md) · [Files](docs/file-transfer.md) · [Panels](docs/panels.md) · [Tailscale](docs/tailscale.md) | Topic notes | | [Open questions](docs/open-questions.md) | What's still unchecked | diff --git a/app/install-link.js b/app/install-link.js new file mode 100644 index 0000000..e64cce1 --- /dev/null +++ b/app/install-link.js @@ -0,0 +1,33 @@ +// Parses frame-control://install?manifest=URL and frame-control://install?url=URL +// (see docs/web-install.md). Pure, so it runs under plain node for the tests. +// This is only a first filter: ui/frame_webinstall.py applies the full URL rules +// (HTTPS, no private addresses, redirects) before anything is fetched. +const SCHEME = "frame-control"; +const MAX_LINK = 4096; +const MAX_URL = 2048; + +// {kind: "manifest" | "url", target} or null if raw isn't a usable install link. +function parseInstallLink(raw) { + if (typeof raw !== "string" || raw.length > MAX_LINK || !raw.toLowerCase().startsWith(`${SCHEME}:`)) return null; + let link; + try { link = new URL(raw); } catch { return null; } + // frame-control://install?… puts "install" in the host; accept a trailing slash too. + if (link.protocol !== `${SCHEME}:` || link.hostname !== "install" || !["", "/"].includes(link.pathname)) return null; + const keys = [...new Set(link.searchParams.keys())]; + if (keys.length !== 1 || !["manifest", "url"].includes(keys[0])) return null; + const values = link.searchParams.getAll(keys[0]); + if (values.length !== 1) return null; + const target = values[0]; + if (!target || target.length > MAX_URL) return null; + let parsed; + try { parsed = new URL(target); } catch { return null; } + if (!["https:", "http:"].includes(parsed.protocol) || parsed.username || parsed.password) return null; + return { kind: keys[0], target }; +} + +// The link among command-line arguments (Windows and Linux pass it there). +function linkFromArgv(argv) { + return (argv || []).find((a) => typeof a === "string" && a.toLowerCase().startsWith(`${SCHEME}:`)) || null; +} + +module.exports = { SCHEME, parseInstallLink, linkFromArgv }; diff --git a/app/main.js b/app/main.js index 37e71f9..ad64d8f 100644 --- a/app/main.js +++ b/app/main.js @@ -9,6 +9,7 @@ const http = require("http"); const net = require("net"); const os = require("os"); const path = require("path"); +const { SCHEME, parseInstallLink, linkFromArgv } = require("./install-link"); const run = promisify(execFile); @@ -233,14 +234,57 @@ async function firstRunCheck() { if (response === 0) setUpConnection(); } -ipcMain.handle("clipboard:read", (e) => { - if (!win || e.sender !== win.webContents || !url) return ""; +// IPC only from our own page in our own window. +function fromUi(e) { + if (!win || e.sender !== win.webContents || !url || !e.senderFrame) return false; try { - if (new URL(e.senderFrame.url).origin !== new URL(url).origin) return ""; - } catch { return ""; } - return clipboard.readText(); + return new URL(e.senderFrame.url).origin === new URL(url).origin; + } catch { return false; } +} + +ipcMain.handle("clipboard:read", (e) => fromUi(e) ? clipboard.readText() : ""); + +// frame-control://install links from websites (docs/web-install.md). They can +// arrive before the window or server exists (macOS open-url on a cold launch), +// so they wait here until the page asks for them. The page checks the link with +// the server and installs nothing until the user confirms in its dialog. +const pendingLinks = []; +let linkPage = null; // the webContents whose current page is listening + +function openInstallLink(raw) { + const req = parseInstallLink(raw); + if (!req) { + app.whenReady().then(() => dialog.showErrorBox("Frame Control can't use this link", + "Install links look like frame-control://install?manifest=https://… or frame-control://install?url=https://…")); + return; + } + pendingLinks.push(req); + if (pendingLinks.length > 5) pendingLinks.shift(); // a page opening links in a loop + deliverLinks(); + if (win) { if (win.isMinimized()) win.restore(); win.focus(); } +} + +function deliverLinks() { + if (!win || !linkPage || linkPage !== win.webContents) return; + while (pendingLinks.length) win.webContents.send("install-link", pendingLinks.shift()); +} + +ipcMain.on("install-link:ready", (e) => { + if (!fromUi(e)) return; + linkPage = e.sender; + deliverLinks(); }); +function registerScheme() { + // A checkout runs as `electron .`, so the OS must be told the script too. + // (macOS takes the scheme from Info.plist, which only the built app has.) + if (process.defaultApp) { + if (process.argv.length >= 2) app.setAsDefaultProtocolClient(SCHEME, process.execPath, [path.resolve(process.argv[1])]); + } else { + app.setAsDefaultProtocolClient(SCHEME); + } +} + function createWindow() { win = new BrowserWindow({ width: 1400, height: 950, minWidth: 760, minHeight: 560, @@ -260,7 +304,9 @@ function createWindow() { win.webContents.on("will-navigate", (e, target) => { if (!url || new URL(target).origin !== new URL(url).origin) e.preventDefault(); }); - win.on("closed", () => { win = null; }); + // A reload or a new page must ask for links again before it gets any. + win.webContents.on("did-start-loading", () => { linkPage = null; }); + win.on("closed", () => { win = null; linkPage = null; }); load(); } @@ -326,10 +372,18 @@ function buildMenu() { if (!app.requestSingleInstanceLock()) { app.quit(); } else { - app.on("second-instance", () => { + // macOS delivers install links here, even before the app is ready. + app.on("open-url", (e, link) => { e.preventDefault(); openInstallLink(link); }); + // Windows and Linux start a second instance with the link as an argument. + app.on("second-instance", (_e, argv) => { if (win) { if (win.isMinimized()) win.restore(); win.focus(); } + const link = linkFromArgv(argv); + if (link) openInstallLink(link); }); + const firstLink = IS_MAC ? null : linkFromArgv(process.argv); + if (firstLink) openInstallLink(firstLink); app.whenReady().then(() => { + registerScheme(); buildMenu(); createWindow(); }); diff --git a/app/package.json b/app/package.json index 944d660..bd5701a 100644 --- a/app/package.json +++ b/app/package.json @@ -21,6 +21,14 @@ "build": { "appId": "com.saphid.frame-control", "productName": "Frame Control", + "protocols": [ + { + "name": "Frame Control install link", + "schemes": [ + "frame-control" + ] + } + ], "directories": { "output": "dist", "buildResources": "build" @@ -28,6 +36,7 @@ "files": [ "main.js", "preload.js", + "install-link.js", "package.json", "build/icon.png" ], @@ -55,6 +64,14 @@ "*.py" ] }, + { + "from": "../frame/devkit-utils", + "to": "frame/devkit-utils", + "filter": [ + "**/*", + "!**/__pycache__/**" + ] + }, { "from": "../apk-catalog", "to": "apk-catalog", diff --git a/app/preload.js b/app/preload.js index b921279..f21551d 100644 --- a/app/preload.js +++ b/app/preload.js @@ -1,7 +1,17 @@ // Lets the page read this computer's clipboard through Electron, so sending it -// to the Frame needs no pbpaste, PowerShell, xclip or wl-clipboard. -const { contextBridge, ipcRenderer } = require("electron"); +// to the Frame needs no pbpaste, PowerShell, xclip or wl-clipboard. Also tells +// the page where a dropped file or folder lives, so a folder can be sideloaded +// as a title without zipping it (the local server reads it from there). +// It also receives frame-control://install links (docs/web-install.md): only +// what the link asked for, never an install; the page asks the user first. +const { contextBridge, ipcRenderer, webUtils } = require("electron"); contextBridge.exposeInMainWorld("frameApp", { readClipboard: () => ipcRenderer.invoke("clipboard:read"), + pathForFile: (file) => { try { return webUtils.getPathForFile(file) || ""; } catch { return ""; } }, + onInstallLink: (cb) => { + ipcRenderer.removeAllListeners("install-link"); + ipcRenderer.on("install-link", (_e, req) => cb({ kind: req.kind, target: req.target })); + ipcRenderer.send("install-link:ready"); + }, }); diff --git a/docs/frame-control.md b/docs/frame-control.md index 43f37a1..0dfb4ae 100644 --- a/docs/frame-control.md +++ b/docs/frame-control.md @@ -56,7 +56,10 @@ python3 ui/server.py # anywhere: then open http://127.0.0.1:47810 (0.85–1.3×) over ADB (`wm size`, `wm density`, `font_scale`). Reset puts all three back. Whether the settings survive the app relaunching is untested. - **Transfer**: drag and drop files to `~/Downloads`; `.apk` files install as - their own Android app. Send typed text, or your computer's clipboard, to the + their own Android app. A game's `.zip`, folder or `.exe` becomes a title in + the Steam library (Valve's Devkit Game path, with Proton or the Steam Linux + Runtime picked from the program's header), listed under **Sideloaded titles** + with Launch and Remove; see [sideloading.md](sideloading.md). Send typed text, or your computer's clipboard, to the Frame clipboard. - **Flatpaks**: install and remove them (quick picks: Moonlight, Firefox, VLC, Remmina). @@ -69,7 +72,7 @@ python3 ui/server.py # anywhere: then open http://127.0.0.1:47810 `app/` is an Electron shell. It starts `ui/server.py` on a free loopback port and shows it in its own window; the server stops when you quit the app. The -app bundles `ui/`, `scripts/`, `frame/android/` and the rated catalogue from +app bundles `ui/`, `scripts/`, `frame/android/`, Valve's `frame/devkit-utils/` and the rated catalogue from `apk-catalog/`, plus a standalone Python ([python-build-standalone](https://github.com/astral-sh/python-build-standalone)) and `adb` from Google's platform-tools, so there's nothing else to install. It @@ -95,7 +98,7 @@ library capsules and green Play buttons. both capture modes (headset view while in use, and a blank frame in standby, which the UI labels), clipboard, volume, file push, and input validation. **Not yet exercised from the UI:** Launch, Flatpak install/remove, APK drop, -and the power buttons. Each of these calls a command that was verified +title sideloading (not yet run on a headset at all), and the power buttons. Each of these calls a command that was verified separately. ## Per-platform notes diff --git a/docs/how-the-frame-works.md b/docs/how-the-frame-works.md index 20fbd33..315a889 100644 --- a/docs/how-the-frame-works.md +++ b/docs/how-the-frame-works.md @@ -48,7 +48,10 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600 | **DeoVR (Steam app 837380, Windows/Unity) runs immersively** under Proton ARM64 + FEX: Unity's OpenVR XR plugin finds `OpenVR Headset(Steam Frame)` and the `frame_controller`, the GPU shows as Turnip Adreno 750, and AVPro Video decodes through `MF-MediaEngine-Hardware`. It played 7680×3840 and 8192×4096 H.265 VR180 SBS streams in dome/fisheye mode (`FirstFrameReady`). Unity's own `VideoPlayer` (used for grid thumbnails) fails with `0xc00d36bb`, so thumbnail previews stay blank. The first launch takes about 45 s (`ComputeShaders: InitAsync`). Log: `compatdata/837380/pfx/drive_c/users/steamuser/AppData/LocalLow/Deo VR/Deo VR/Player.log`. **Verified 2026-09-25**, BUILD_ID 20260922.6101926. | [vr-video.md](vr-video.md) | | **Wolvic (VR browser APK) runs in Lepton against SteamVR's OpenXR**, with limits. The stock Lynx build aborts (`Runtime doesn't support selected swapChain color format`: it wants `GL_RGBA8`), and the stock Quest build fails with `XR_ERROR_API_VERSION_UNSUPPORTED`. Patching `DeviceDelegateOpenXR::GetSwapChainCreateInfo` in the Lynx build's `libnative-lib.so` to `GL_SRGB8_ALPHA8` (0x8C43) and re-signing fixes start-up. The Gecko engine then segfaults in `libxul`. The Chromium-engine build (Lynx v1.3-chromium) browses fine as an immersive app. Its page reports `isSessionSupported("immersive-vr") == true`, and `requestSession` succeeds, running about 36 rAF/s, but the headset shows **black** for WebXR content, or Wolvic's loading spinner that never clears, until the session is ended. Video decodes on the software `OMX.google.h264.decoder`. Tapping the URL bar's selection menu crashes it (no clipboard service). Open URLs with `am start -a VIEW -n com.igalia.wolvic/.VRBrowserActivity -d ` over the instance's ADB. DevTools is at `localabstract:content_shell_devtools_remote`. **Verified 2026-09-25**, BUILD_ID 20260922.6101926. | Web VR video, [apks.md](apks.md) | | Tailscale runs without root as a userspace `tailscaled` user service (static arm64 build in `~/.local/share/tailscale`, lingering on). In userspace mode, inbound tailnet connections reach the Frame's **loopback**, so every port, including DevTools on 8080, is reachable from the tailnet. **Verified 2026-09-25.** | [tailscale.md](tailscale.md), `scripts/tailscale-on-frame.sh` | +| **T3 Code desktop runs natively.** The stock release `T3-Code-0.0.42-arm64.AppImage` in `~/Applications/T3CodeDesktop/` starts with no extra setup: glibc 2.39, `libfuse.so.2`, GTK 3, NSS and libsecret are on the image. `panel-on-frame.sh --name t3code-desktop -- '~/Applications/T3CodeDesktop/T3-Code.AppImage'` gives it its own panel (`valve.steam.desktopgame.2000281357`, `--ozone-platform=x11`). Its bundled server listens on `127.0.0.1:3773` and shows up in onboarding as the `frame` computer, with `passwordStore: gnome-libsecret`. The image has no agent CLI and no `node`. Agents run through the LAN CLIProxyAPI (`llm-proxy.lan:8317`, which resolves on the Frame). Claude Code 2.1.283 comes from `claude.ai/install.sh`, and Codex 0.157.1 from the `codex-aarch64-unknown-linux-musl` release tarball, both into `~/.local/bin`. `with-cliproxy` and a mode-600 `~/.config/cliproxyapi/secrets.env` are copied from the Mac. The wrappers `claude-cliproxy` and `codex-cliproxy` (a `-c model_provider=cliproxy`, `wire_api="responses"`, `env_key="CLIPROXY_API_KEY"`) are set as `providers.claudeAgent.binaryPath` and `providers.codex.binaryPath` in `~/.t3/userdata/settings.json`, and T3 picked that up without a restart. Through the wrappers, `claude auth status` reports `loggedIn: true` (`oauth_token`), and both CLIs answered a prompt with `kimi-k3`. `gamescopectl screenshot` captured another layer (the Lepton T3 app) rather than this panel. `DISPLAY=:0 xwd -id ` piped to `ffmpeg` captures the window itself (1920×1080). **Verified 2026-09-26**, BUILD_ID 20260922.6101926. | Running T3 Code as a host on the Frame | | Power actions need `sudo`, which asks for the Developer Mode password over SSH. | Frame Control's power buttons | +| **Boot / recovery menu.** Hold Power ~10 s until the LED goes off, then power on while holding the **AUX button on top of the Power button** (not the volume keys) until a text menu appears. Entries: `Current` (SteamOS-A/B + build), `Previous` (the other A/B slot), `Boot from USB`, `Repair Steam Installation`, `Erase User Data` (factory reset), `ADB mode`, `Battery Ship Mode`. It auto-boots `Current` after a ~15 s countdown. **Volume Up/Down (left side) move, AUX (right side) selects.** For a boot loop, Valve says pick `Previous` (keeps user data); then `Repair Steam Installation`; `Erase User Data` wipes `~` (SSH keys, Tailscale, Flatpaks, T3 setup). Last resort is a full re-image, two ways: (1) USB: write `steamframe-repair-latest.img.bz2` to an 8 GB+ USB-C stick (Balena Etcher on the Mac), pick `Boot from USB`, then use "Wipe Device & Install SteamOS" / "Repair SteamOS" (keeps games and personal content) from the recovery desktop; (2) cable/EDL: `steamframe-repair-qdl-latest.tar.gz`, run `flash.sh` (Linux) or `flash.cmd` (Windows), then with the Frame off for 10 s hold Power + Vol Up + Vol Down for 10 s and plug it in; it reflashes and reboots. Both images: `https://steamdeck-images.steamos.cloud/recovery/` (build 20260922.5153644, 0.3.0, ~4 GB each, no published checksums); local copies in `~/Downloads/steam-frame-recovery/`. Source: Valve's [SteamOS Recovery FAQ](https://help.steampowered.com/en/faqs/view/1B71-EDF2-EB6D-2BB3) and [Installation and Repair FAQ](https://help.steampowered.com/en/faqs/view/65B4-2AA3-5F37-4227), plus a menu photo in [EloiStree/HelloSteamFrame#9](https://github.com/EloiStree/HelloSteamFrame/issues/9). **Inferred** (Valve docs, 2026-09-26); not yet tried on our Frame. | Recovering from a boot loop | +| **Boot loop cause: the SteamVR health check.** `steamvr.service` runs `/usr/share/deckard/steamvr-health-check`, which appends `frog:glasses:` to `$XDG_RUNTIME_DIR/steamvr-short-session-tracker` on every failed or <10 s SteamVR run. At 3 it runs `steam-health-check --repair-now`, which **deletes all of `~/.local/share/Steam` (games, login, Developer Mode) and `~/.steam`**, keeping only `registry.vdf`. At 4 it also tries `steamos-bootconf set-mode reboot-other` (fails as the user: `bootenv: Permission denied`). SteamVR normally fails 1–2 times per boot while it waits for the Steam client (`SteamAPI_InitEx failed … Steam is probably not running`, then `fatal stalled cross-thread pipe`). Once Steam has been wiped, it has to re-download a ~210 MB client on every boot, so SteamVR keeps failing, Steam keeps getting wiped and the Frame reboots, in a loop. Also, the Steam updater can deadlock at `Installing update...` (main process blocked writing to the `-child-update-ui` process, which is stuck in `drm_syncobj_array_wait_timeout`). Killing only the `-child-update-ui` process lets the install finish (`package/*.installed` appears). **Fix without sudo:** over USB-C ADB (`adb -s frame shell` works as `steamos` while the Frame is looping; SSH is refused once Developer Mode is lost), truncate both `/run/user/1000/steam{,vr}-short-session-tracker` files and `chmod 444` them (the health check then logs `Permission denied` and does nothing; this is tmpfs, so it resets on reboot). Unstick the updater if needed, let Steam finish installing, then hold Power 10 s and start the Frame normally. `systemctl reboot` over ADB needs interactive auth. After the fix, sign in to Steam and turn Developer Mode back on. **Verified 2026-09-26**, BUILD_ID 20260922.6101926, slot B (clean boot: 0 SteamVR failures, SSH and Tailscale back). | Diagnosing a boot loop | ## Debug recipes diff --git a/docs/install.html b/docs/install.html new file mode 100644 index 0000000..4948d26 --- /dev/null +++ b/docs/install.html @@ -0,0 +1,63 @@ + + + + + + +Install with Frame Control + + + + +
+

Install with Frame Control

+

+ + + +
+ + + diff --git a/docs/scripts.md b/docs/scripts.md index a1a1e0f..464c5a6 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -36,9 +36,11 @@ ssh frame # passwordless from now on `connect.sh` does four things: - finds the headset (`frame.local`, then `frame`, or the IP/host you pass in) -- creates a dedicated key (`~/.ssh/id_ed25519_frame`) +- creates dedicated keys (`~/.ssh/id_ed25519_frame`, plus `~/.ssh/id_rsa_frame_devkit` for pairing) - adds a `Host frame` block to `~/.ssh/config` -- runs `ssh-copy-id`, which asks for the Developer Mode password once +- tries SteamOS devkit pairing (approve on the headset, no password; **inferred**, + see [SSH](ssh.md#password-free-pairing-steamos-devkit-service)), else runs + `ssh-copy-id`, which asks for the Developer Mode password once Run `./scripts/connect.sh --harden` later if you want to turn off SSH password logins. diff --git a/docs/sideloading.md b/docs/sideloading.md new file mode 100644 index 0000000..6ff60e4 --- /dev/null +++ b/docs/sideloading.md @@ -0,0 +1,171 @@ +# Sideloading Linux and Windows games + +A game you have as files (an itch.io download, your own build, a DRM-free +release) can go into the Frame's Steam library without a Steam store page. +Frame Control uses the same path as Valve's +[SteamOS Devkit Client](https://gitlab.steamos.cloud/devkit/steamos-devkit): +the title becomes a Steam **Devkit Game**, with a runtime (Proton or a Steam +Linux Runtime) chosen from the program itself. + +For Android APKs, see [apks.md](apks.md) instead. + +**Status: nothing here has run on a headset yet.** Every device-side step is +**inferred from Valve's steamos-devkit source** (release v0.20260925.1). The +local steps (reading the zip, picking the program and runtime, building the +request) are covered by `tests/test_frame_titles.py`. + +## Using it + +Drop a game's `.zip`, folder or `.exe` on **Send to Frame**. (Folders need the +desktop app, which knows where a dropped folder lives; in a plain browser, zip +it.) A dialog shows: + +- **Name**: what Steam shows. Steam uses the title id as the name, so it's + limited to letters, digits, `_` and `-`; the dialog shows the result. +- **Launches**: the program picked to start the game, with the other + candidates in the list. +- **Runtime**: picked from the program, see below. Windows programs can switch + between Proton Experimental and Proton (stable). + +Install copies it to the Frame and registers it with Steam; progress shows in +the bar and the activity log. **Sideloaded titles** lists what's installed, +with Launch and Remove. **Copy to ~/Downloads instead** keeps the old +behaviour for a zip that isn't a game. + +From a terminal: + +```sh +python3 ui/frame_titles.py inspect Game.zip # what would be installed, no headset needed +python3 ui/frame_titles.py install Game.zip [--name N] [--exe REL] [--runtime R] +python3 ui/frame_titles.py list | launch ID | remove ID +``` + +## Choosing the runtime + +The program's header decides, not its file name: + +| Program | Runtime (Steam compat tool) | `steam_play` | Confidence | +|---|---|---|---| +| Windows `.exe`, x86-64 (PE machine `0x8664`) | `proton-experimental` | 1 | Inferred: ARM64 Proton runs x86-64 code through FEX | +| Windows `.exe`, 32-bit x86 (`0x14c`) or ARM64 (`0xaa64`) | `proton-experimental` | 1 | Inferred | +| Linux ELF, aarch64 (`e_machine` `0xB7`) | `SteamLinuxRuntime_4-arm64` | 0 | Verified: starts, but natively (see below) | +| Linux ELF, x86-64 (`0x3E`) | `SteamLinuxRuntime_4` | 0 | Verified not to start: the runtime isn't installed (see below) | +| Shell script | the runtime of the Linux binary beside it, else `SteamLinuxRuntime_4-arm64` | 0 | Guess | +| Anything else (32-bit Linux, other CPUs, DLLs, data) | refused with a message | | | + +Proton Experimental is the default rather than stable because the Frame's +ARM64 Proton and FEX stack is new and Proton fixes reach Experimental first. +If a game misbehaves, reinstall it with Proton (stable). + +The aliases and settings are the ones Valve's client sends: `RUNTIME_ALIASES` +in `devkit_client/__init__.py`, and `gui2._update_game`, which sets +`steam_play=1, steam_play_debug=0, steam_play_debug_version=2019` for Proton +and `steam_play=0` otherwise, plus `compat_tool=`. Valve's client only +offers `SteamLinuxRuntime_4-arm64` and Lepton when the device reports itself +as Deckard (the Frame). + +## Picking the program + +`ui/frame_titles.py` reads every file's header: ELF executables (PIE ones are +told from shared libraries by their `PT_INTERP` segment), PE executables (not +DLLs) and scripts with `#!`. A zip with a single top-level folder is treated +as that folder. Candidates are ranked by: + +1. Not a helper: names like `UnityCrashHandler64`, `CrashReportClient`, + `*setup*`, `unins*`, `vc_redist*`, `dxsetup`, `*prereq*`, and anything under + `_CommonRedist`, `Redist`, `DirectX` or `Engine` go last. +2. Platform: native ARM64 Linux, then Windows x86-64, then x86-64 Linux, then + other Windows builds. +3. Name: a program named like the zip or folder (build words such as + `-linux-arm64` or `_v1.2` are dropped from the name). +4. Depth, then size: Unreal's top-level `Game.exe` beats + `Game/Binaries/Win64/Game-Win64-Shipping.exe`. + +A top-level shell script beats a Linux binary one folder down (`run.sh` + +`bin/game`); a binary next to a script wins. The list in the dialog lets you +pick another. + +## What happens on the Frame (inferred) + +1. **Tools.** `frame/devkit-utils/` (Valve's scripts, vendored unmodified, MIT) + is copied to `~/devkit-utils`, where Valve's client puts it, unless the + stamp file there already matches. Files are merged, not replaced, so a + newer copy from Valve's client keeps its extra files. +2. **Folder.** `python3 ~/devkit-utils/steamos-prepare-upload --gameid ID` + makes `~/devkit-game/ID` and prints `{user, directory}`. +3. **Copy.** The files go there with `rsync -a --delete` on macOS and Linux, + or `scp -r` into a fresh folder that then replaces it on Windows. Then + `chmod -R 755`, the modes Valve's client gives an upload. +4. **Register.** `python3 ~/devkit-utils/steam-client-create-shortcut --parms JSON` + with `{gameid, directory, argv: [target], env: {}, settings, clear_settings, + force_appid: "", lepton_args: ""}`. It writes `ID-argv.json`, + `ID-env.json` and `ID-settings.json` next to the folder, then sends + `create-shortcut` to the running Steam client over `~/.steam/steam.pipe` + (authenticated by `~/.steam/steam.token`) and waits up to 5 s for Steam's + answer file. Its `error`, for example "The Steam client is not running", + is shown as the install error. The files stay, so installing again with + Steam running finishes the job. +5. **Launch** is `steam-devkit-rpc run-game gameid=ID`. **Remove** is + `steamos-delete --delete-title ID`, which deletes the folder and has Steam + drop shortcuts with no folder. Frame Control then removes the `ID-*.json` + files that Valve's script leaves behind. + +Frame Control also writes `~/devkit-game/ID-framecontrol.json` (name, source +file, target, runtime, size). **Sideloaded titles** lists every folder in +`~/devkit-game`, including titles uploaded with Valve's client. + +`argv` is one string, as in Valve's client (the start command may carry +arguments), so a program path with spaces is sent in double quotes. How Steam +splits that string is **not checked**. + +## Safety + +- Zips are unpacked on your computer first. Entries with absolute paths, `..`, + drive letters or `:` anywhere in the path, or links that point outside the + zip (or at a folder they're in) are refused. So are zips over 64 GB + unpacked, over 200,000 entries, more than 200× compressed past 1 GB, or + bigger than the free space. +- No symlink is created while unpacking, so no write can be redirected + through one. A link to a file inside the zip (`libfoo.so.1 → libfoo.so.1.2`) + becomes a copy of that file, which also works on Windows. Links to folders, + loops and dangling links are left out. +- A dropped folder that contains symlinks (or Windows junctions) is copied on your computer first, + with the same rule, because `scp -r` would follow a link out of the folder + and upload whatever it points at. +- Installs run one at a time, and Remove is refused while one runs. +- The title id is limited to `[A-Za-z0-9_-]`, at most 64 characters. Valve's + scripts pass it to a shell (`steamos-delete` runs `rm -r` on it). Valve's + reserved sideload names (`steam`, `steamvr`, and their `deckard` forms, + which would replace the Steam client itself) get `-game` added. +- Nothing needs `sudo`; everything goes to your home folder on the Frame. +- In the app, a dropped folder is read from its local path by the app's own + server, which only accepts requests from its own page (see + [frame-control.md](frame-control.md#how-it-works)). + +## Checked on a headset + +Tested 2026-09-26 on a Frame (BUILD_ID 20260922.6101926) with small static test +programs and PuTTY's official 64-bit `putty.exe`, through both the command line +and the app (inspect, install job, ▶, Remove, and install links): + +- [x] `create-shortcut` registers a title; it shows in the Steam library and in + **Sideloaded titles**, and Steam maps it to the chosen compat tool. +- [x] `steam-devkit-rpc run-game` starts it (Steam logs `devkit run-game: started + devkit game ""`), and Remove (`steamos-delete`) deletes the files, the + shortcut and the Proton prefix. +- [x] A quoted path in the start command is fine: Steam runs + `proton waitforexitandrun "/home/steamos/devkit-game//"`. +- [x] An x86-64 Windows `.exe` runs under **Proton 11 (stable)** through FEX + (ARM64EC) inside the Steam Linux Runtime 4.0 ARM64 container; PuTTY stayed up. + Proton Experimental wasn't installed at the time (it was downloading), so it's + untested. A Go-built x86-64 test program crashed in `libarm64ecfex.dll` + (a FEX limitation with that program, not the sideloading). +- [ ] **An aarch64 build runs natively, not in `SteamLinuxRuntime_4-arm64`**: + Steam records the mapping (`CompatToolMapping`, `compat_log.txt`) but launches + the devkit title without the runtime's `_v2-entry-point` prefix. Fine for a + self-contained build; a build that needs the runtime's libraries may not start. +- [ ] **An x86-64 Linux build doesn't start**: Steam logs `Tool 4183110 "Steam + Linux Runtime 4.0" is found for appID …, but is not installed`, and the Frame + doesn't install that x86-64 runtime for a devkit title (a `steam://install/4183110` + request did nothing). +- [ ] Whether these titles open as flat panels or need anything VR-specific. diff --git a/docs/ssh.md b/docs/ssh.md index 0b25cce..93aac3b 100644 --- a/docs/ssh.md +++ b/docs/ssh.md @@ -33,7 +33,8 @@ unless your router's DNS registers DHCP client names. - **Verified on device (2026-09-25):** `avahi-daemon` is running on the Frame and `frame.local` resolves from the Mac over mDNS. -- `scripts/connect.sh` tries `frame.local`, then `frame`. If neither works, it tells you to re-run it with the IP. +- `scripts/connect.sh` tries `frame.local`, then `frame`, then an mDNS browse for + the devkit service (below). If none works, it tells you to re-run it with the IP. Once you have a working address, the `Host frame` alias means you just type `ssh frame`. - To check discovery yourself: `dns-sd -G v4 frame.local` (Ctrl-C to stop), or @@ -55,10 +56,49 @@ Host frame HostName frame.local User steamos IdentityFile ~/.ssh/id_ed25519_frame + IdentityFile ~/.ssh/id_rsa_frame_devkit IdentitiesOnly yes ServerAliveInterval 30 ``` +The script only asks for the password if the pairing below doesn't work. + +## Password-free pairing (SteamOS devkit service) + +From Valve's source ([steamos-devkit-service](https://gitlab.steamos.cloud/devkit/steamos-devkit-service), +[steamos-devkit](https://gitlab.steamos.cloud/devkit/steamos-devkit) client). **Verified on a +Frame 2026-09-26** (BUILD_ID 20260922.6101926): the service runs with Developer Mode +on, `properties.json` answers with `"login": "steamos"`, the headset advertises +`_steamos-devkit._tcp` as `frame`, and `/register` needs pairing mode (below). The +approve prompt and key install are not verified yet. SteamOS's devkit service is +what Valve's Devkit Client uses to pair. `scripts/connect.sh` and +`ui/frame_connect.py` try it first: + +- The headset serves HTTP on port **32000** and advertises mDNS + `_steamos-devkit._tcp`. `GET /properties.json` gives the `login` user; the + script uses it as `User` (unless you set `FRAME_USER`, or it says `root`), + for the password fallback too, and keeps it on re-runs. +- **Open Steam Settings → Developer → Pair new host in the headset first.** + Otherwise `/register` answers at once with `403` `"please put the Steam client + in pairing mode: Settings -> Developer -> Pair new host"` (verified). The + scripts say so and keep asking for 2 minutes while you open it. +- `POST /register` with `ssh-rsa 900b919520e4cf601998a71eec318fec` + (a fixed token from Valve's client) shows an approve prompt inside the + headset naming the comment (`frame-control@`). It waits 30 s, + then installs the key for the device user and turns `sshd` on. The reply is + `200 Registered`, or `403` with `{"error": ...}` (declined, timed out, Steam + not running). +- It only accepts **RSA** keys, hence the second key, + `~/.ssh/id_rsa_frame_devkit` (3072-bit). +- A host counts as found if port 22 **or** 32000 answers. With no host given, + and `frame.local`/`frame` unreachable, it browses `_steamos-devkit._tcp` with + `dns-sd` (macOS) or `avahi-browse` (Linux) for a few seconds if installed. +- Port 32000 closed, a timeout, or an error: the script says why and falls back + to copying the ed25519 key with the Developer Mode password, as before. + +Anyone on your network can send the request, so only approve a prompt you +started. `curl http://:32000/properties.json` shows whether the service is up. + `~/.ssh/authorized_keys` lives under `/home`, which SteamOS keeps across OS updates (inferred from Deck; the Frame uses the same A/B image scheme). diff --git a/docs/web-install.md b/docs/web-install.md new file mode 100644 index 0000000..d77dfd7 --- /dev/null +++ b/docs/web-install.md @@ -0,0 +1,158 @@ +# Install links for websites + +A website can put an "Install with Frame Control" button next to its download. +Clicking it opens Frame Control, which shows what the link wants to install and +asks the user. Only after they click **Install** does it download the file and +install it on the Frame. + +What's verified: the link parsing, URL rules, manifest parsing, download, +size cap and sha256 check, by `tests/test_webinstall.py` and +`tests/test_server.py` (no network: a stub server on 127.0.0.1). Installing on +the headset is the same code as dropping a file on Frame Control: `.apk` files go +to the APK installer ([apks.md](apks.md)), `.zip` and `.exe` files to the +Linux/Windows title installer. A link hasn't been clicked through to a headset +install yet. + +## The link + +``` +frame-control://install?manifest= +frame-control://install?url= +``` + +Use `manifest` when you can: it carries the title's name and a sha256, which +Frame Control checks before installing. `url` is for a file on its own; the +dialog then names the title after the file. + +The manifest is FrameDrop's format, so one manifest serves both apps. The +schema may be `framedrop.install/v1` or `frame-control.install/v1`: + +```json +{ + "schema": "framedrop.install/v1", + "name": "My Game", + "files": [ + { "url": "https://cdn.example.com/mygame-arm64.apk", "sha256": "optional-but-better" } + ] +} +``` + +| Field | | +|---|---| +| `schema` | Required, one of the two above | +| `name` | Shown in the confirm dialog (at most 120 characters). Defaults to the file name. APKs are still named in the Steam library by their own label | +| `files` | Exactly one entry for now; more is refused with a message | +| `files[0].url` | Required. The file to install | +| `files[0].sha256` | Optional, 64 hex digits. The download must match or nothing is installed | +| `files[0].size` | Optional (Frame Control extension), bytes. Shown up front; the download must match | +| `files[0].exe` | Optional (Frame Control extension), for a `.zip` title: the program inside it to run | + +What gets installed depends on the file name's extension: + +| File | Installed as | +|---|---| +| `.apk` | An Android app in its own Lepton instance with a Steam shortcut ([apks.md](apks.md)) | +| `.zip`, `.exe` | A Linux or Windows title. Versions of Frame Control without the title installer say "Linux/Windows titles need a newer Frame Control" | +| anything else | Refused | + +## Rules + +Frame Control refuses a link, and downloads nothing, unless: + +- Every URL (the manifest's, the file's and each redirect) is `https://`. + `http://` works only for `localhost` or `127.0.0.1`, for testing: only when + Frame Control runs with `FRAME_CONTROL_LOCAL_LINKS=1`, and only when the + link itself points there. It's off by default so a website's link can't make + the app fetch from services on your computer, and a public manifest can + never send it there. +- No URL has a user name or password in it (`https://user:pw@…`). +- No host is, or resolves to, a private, loopback, link-local, CGNAT + (100.64.0.0/10), multicast or otherwise non-public address. Every address + the name has must be public, it's checked again on every redirect (at most + 5), and the download connects to the address that was checked. +- The file URL ends in a file name with one of the extensions above + (`https://example.com/games/` is refused). +- The manifest is JSON of at most 256 KB, and the file at most 4 GiB + (`MAX_MANIFEST` and `MAX_FILE` in `ui/frame_webinstall.py`). +- The user confirms. The dialog shows the title's name, the site the link came + from (and the file's host if different), the file name and type, the size if + known, and whether a sha256 was given. + +A web page can't install anything itself: it can only open the link. Frame +Control's local server refuses requests from web pages, so the only way in is +the operating system handing the link to the app, then the user's click. + +## Button for your site + +Paste this where the download is, with your manifest's URL in `MANIFEST`: + +```html +Install with Frame Control + +``` + +For a single file, use `"frame-control://install?url=" + encodeURIComponent(FILE_URL)`. + +`docs/install.html` is a landing page that does the same from a plain link: +`install.html?manifest=` tries the app and shows a "Get Frame +Control" link. It isn't published anywhere yet; host a copy to use it. + +## Testing locally + +Start Frame Control with `FRAME_CONTROL_LOCAL_LINKS=1` in its environment (for +example `FRAME_CONTROL_LOCAL_LINKS=1 npm start` in `app/`), then serve the +manifest and file from your own computer: + +```sh +cd mygame && python3 -m http.server 8000 +open 'frame-control://install?manifest=http%3A%2F%2Flocalhost%3A8000%2Fmanifest.json' # xdg-open on Linux, start "" on Windows +``` + +The manifest's file URL must then be `http://localhost:8000/…` or +`http://127.0.0.1:8000/…` too. + +## How it works + +- `app/install-link.js` parses the link (only `frame-control://install` with + exactly one `manifest` or `url`); `app/main.js` registers the scheme + (`app.setAsDefaultProtocolClient`, and electron-builder's `protocols` for the + macOS Info.plist and the Linux `.desktop` file). macOS delivers links through + `open-url`, Windows and Linux as an argument to a second instance. Links + wait in the main process until the page has loaded and asked for them + (`frameApp.onInstallLink` in `app/preload.js`). `framedrop://` is left alone. +- The page posts the link to `/api/webinstall/check`, which reads the manifest, + applies the rules, asks the file's size with a HEAD request and returns a + one-time id. Nothing is downloaded. +- **Install** posts the id to `/api/webinstall/start`. The server downloads to + a temporary folder (progress at `/api/webinstall/job`, cancellable with + `/api/webinstall/cancel`), checks size and sha256, hands the file to + `frame_webinstall.dispatch()` and deletes the folder. +- The app registers the scheme each time it starts, so the last Frame Control + started (e.g. a development checkout) handles the links. + +**Quitting during a stalled download.** On macOS and Linux, quitting stops a +download at once (`shutdown()` on its socket wakes the blocked read). On +Windows that doesn't wake a read in another thread, and closing the handle +under a TLS read isn't safe, so a download that has stalled holds the quit for +the 4-second grace period until the app stops the server; the partial file is +removed on the next start. Downloads that are still moving stop at their next +read either way. diff --git a/frame/devkit-utils/LICENSE b/frame/devkit-utils/LICENSE new file mode 100644 index 0000000..bd7a5ef --- /dev/null +++ b/frame/devkit-utils/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2017-2022 Valve Software inc., Collabora Ltd + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/frame/devkit-utils/README.md b/frame/devkit-utils/README.md new file mode 100644 index 0000000..f97cbbe --- /dev/null +++ b/frame/devkit-utils/README.md @@ -0,0 +1,18 @@ +# Valve's devkit-utils (vendored) + +Unmodified copy of `client/devkit-utils/` from Valve's +[SteamOS Devkit Client](https://gitlab.steamos.cloud/devkit/steamos-devkit), +MIT licensed (see `LICENSE`; Valve's own notes are in `VALVE-README.md`). + +- Source: steamos-devkit, commit `6f0711a` ("Code drop."), + release **v0.20260925.1** (ChangeLog entry dated 2026-09-25). + +`ui/frame_titles.py` copies this folder to `~/devkit-utils` on the Frame (where +Valve's own client puts it) and uses `steamos-prepare-upload`, +`steam-client-create-shortcut`, `steam-devkit-rpc` and `steamos-delete` to +register uploaded builds as Steam "Devkit Games". See `docs/sideloading.md`. + +To update: copy the folder from a newer checkout over this one, keep this +README, and update the version line above. The stamp Frame Control compares +on the headset is a hash of these files, so a changed copy is re-synced on the +next use. diff --git a/frame/devkit-utils/VALVE-README.md b/frame/devkit-utils/VALVE-README.md new file mode 100644 index 0000000..585a691 --- /dev/null +++ b/frame/devkit-utils/VALVE-README.md @@ -0,0 +1,4 @@ +These scripts and supporting utility module are uploaded to the devkit by the devkit client: + +- steamos-* : scripts that operate (mostly) at SteamOS level for devkit functionality purposes +- steam-client-* : scripts that relay commands to the local running Steam client diff --git a/frame/devkit-utils/deckard-capture b/frame/devkit-utils/deckard-capture new file mode 100755 index 0000000..6cae14f --- /dev/null +++ b/frame/devkit-utils/deckard-capture @@ -0,0 +1,107 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +import sys +import os +import time +import subprocess +import logging +import argparse +import json +import datetime +import io + +logging.basicConfig(format='%(message)s', level=logging.DEBUG) +logger = logging.getLogger(__name__) + +def main(): + parser = argparse.ArgumentParser(description='Capture screenshot and videos on Steam Frame device') + parser.add_argument('--filename', '-f', + default='/tmp/screenshot.png', + help='Output') + parser.add_argument('--timestamp', action='store_true', + help='Add timestamp') + parser.add_argument('--json', action='store_true', + help='Output result as JSON') + + args = parser.parse_args() + + output_buffer = io.StringIO() + try: + steamvr_path = subprocess.check_output(['steamvr', 'path'], stderr=subprocess.STDOUT, universal_newlines=True).strip() + cdd = os.path.join(steamvr_path, 'bin/linuxarm64') + run_vrcmd = os.path.join(cdd, 'vrcmd') + assert os.path.exists(run_vrcmd), "vrcmd not found" + + # Enable recording + cmd = [run_vrcmd, '--mailboxcmd', 'vrcompositor_systemlayer', 'set_local_video_record?enabled=true'] + output_buffer.write(f"Command: {' '.join(cmd)}\n") + result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) + output_buffer.write(result.stdout) + if result.returncode != 0: + raise subprocess.CalledProcessError(result.returncode, cmd) + + # Wait for the video device to produce frames. + # Note that even when disabled it outputs roughly 2 blank frames per second. + cmd = ['timeout', '1', 'ffmpeg', '-f', 'v4l2', '-i', '/dev/video99', '-frames:v', '4', '-f', 'null', '-', '-v', 'error'] + output_buffer.write(f"Command: {' '.join(cmd)}\n") + retries = 2 + while True: + result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) + output_buffer.write(result.stdout) + if result.returncode == 0: + break + retries -= 1 + if retries <= 0: + raise Exception("Failed to get video frames from /dev/video99. Is VR active? Is the v4l2 configuration correct?") + + output_filename = args.filename + if args.timestamp: + timestamp = datetime.datetime.now().strftime("%Y-%m-%d-%H-%M-%S") + base, ext = os.path.splitext(output_filename) + output_filename = f"{base}-{timestamp}{ext}" + + # Capture screenshot + cmd = ['ffmpeg', '-f', 'v4l2', '-i', '/dev/video99', '-frames:v', '1', '-q:v', '1', '-y', output_filename] + output_buffer.write(f"Command: {' '.join(cmd)}\n") + result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) + output_buffer.write(result.stdout) + if result.returncode != 0: + raise subprocess.CalledProcessError(result.returncode, cmd) + + # Disable recording + cmd = [run_vrcmd, '--mailboxcmd', 'vrcompositor_systemlayer', 'set_local_video_record?enabled=false'] + output_buffer.write(f"Command: {' '.join(cmd)}\n") + result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) + output_buffer.write(result.stdout) + if result.returncode != 0: + raise subprocess.CalledProcessError(result.returncode, cmd) + + except Exception as e: + error_msg = str(e) + # Print collected output to stderr on error + print(output_buffer.getvalue(), file=sys.stderr) + logger.error(error_msg) + + if args.json: + result = { + 'success': False, + 'error': error_msg + } + print(json.dumps(result)) + else: + print(f"Error: {error_msg}", file=sys.stderr) + + return 1 + + if args.json: + result = { + 'success': True, + 'output': output_filename + } + print(json.dumps(result)) + else: + print(f"Screenshot saved to {output_filename}") + +if __name__ == '__main__': + sys.exit(main()) \ No newline at end of file diff --git a/frame/devkit-utils/devkit_utils/__init__.py b/frame/devkit-utils/devkit_utils/__init__.py new file mode 100644 index 0000000..f02d7d4 --- /dev/null +++ b/frame/devkit-utils/devkit_utils/__init__.py @@ -0,0 +1,300 @@ +#!/usr/bin/env python +# encoding: utf-8 +"""Utility functions for the Steam client hook scripts""" + +import sys +import os +import traceback +import tempfile +import json +import logging +import fcntl +import errno +import contextlib +import time +import fcntl + + +import logging as logging_module +logger = logging_module.getLogger(__name__) + + +@contextlib.contextmanager +def wrap_outputs(stderr_prefix): + # capture stderr to file to support debugging + stderr_fd = sys.stderr.fileno() + tf = tempfile.NamedTemporaryFile( + mode='w+', + prefix=stderr_prefix, + delete=True) + if sys.version_info >= (3, 4): + # this API in the os module is only available for python3 + # but it does not seem to work with subprocess anyway + os.set_inheritable(tf.file.fileno(), True) + assert os.get_inheritable(tf.file.fileno()) + sys.stderr = tf.file + + # we can only write out a json response to stdout, + # so redirect stdout to stderr, + # and keep a handle on the original stdout for the response + stdout_fd = os.dup(sys.stdout.fileno()) + os.dup2(sys.stderr.fileno(), sys.stdout.fileno()) + + ctx = {} + try: + yield ctx + except: + logger.error(traceback.format_exc()) + finally: + tf.flush() + tf.seek(0) + os.write(stderr_fd, tf.read().encode('utf-8')) + if 'ret' in ctx: + os.write(stdout_fd, json.dumps(ctx['ret']).encode('utf-8')) + + +class SteamClientNotRunningException(Exception): + def __init__(self, error_message): + self.error_message = error_message + + def __str__(self): + return self.error_message + + +def validate_steam_client(): + """Verify that the steam client is running, and permissions are adequate""" + pid_path = os.path.normpath( + os.path.realpath( + os.path.expanduser('~/.steam/steam.pid'))) + if not os.path.exists(pid_path): + raise SteamClientNotRunningException('{0} does not exist'.format(pid_path)) + try: + pid = int(open(pid_path, 'rt').read()) + except Exception: + raise SteamClientNotRunningException('{0} is invalid'.format(pid_path)) + try: + os.kill(pid, 0) + except OSError: + raise SteamClientNotRunningException('{0} does not refer to a valid process'.format(pid_path)) + logger.info('Found steam client pid %s', pid) + + +def execute_steam_client_command(cmd): + """Send a command to the steam client over the IPC pipe""" + pipe_path = os.path.normpath( + os.path.realpath( + os.path.expanduser('~/.steam/steam.pipe'))) + try: + pipe = open(pipe_path, 'wb+', 0) + except IOError: + raise Exception('cannot open steam client pipe') + session_token = open(os.path.expanduser('~/.steam/steam.token')).read() + #pipe_cmd = 'steam://{0}'.format(cmd) + # ^ hack to execute a normal command over the IPC directly - sometimes useful + pipe_cmd = 'devkit-1 steam://devkit-1/{0}/{1}'.format( + session_token, + cmd + ) + logger.debug('Sending command line:') + logger.debug(pipe_cmd) + pipe.write('{0}\n'.format(pipe_cmd).encode('utf-8')) + pipe.close() + + +def save_argv(gameid, argv): + """Save command line and arguments if provided""" + + if argv is None: + return + + argvfile = os.path.join(os.getenv("HOME"), "devkit-game", + gameid + "-argv.json") + try: + with open(argvfile, "w") as argvf: + fcntl.flock(argvf, fcntl.LOCK_EX) + json.dump(argv, argvf) + fcntl.flock(argvf, fcntl.LOCK_UN) + except IOError: + raise Exception( + "Unable to open argv file for writing: {0}".format(argvfile)) + + +def obtain_argv(gameid, argv): + """Obtain command line with arguments""" + + # If present and not None or [], just return the local arguments + if argv: + return argv + + # From here, expect arguments to have been saved previously + argvfile = os.path.join(os.getenv("HOME"), "devkit-game", + gameid + "-argv.json") + try: + with open(argvfile, "r") as argvf: + fcntl.flock(argvf, fcntl.LOCK_EX) + argv = json.load(argvf) + fcntl.flock(argvf, fcntl.LOCK_UN) + except IOError: + raise Exception( + "Unable to open argv file for reading: {0}".format(argvfile)) + return argv + + +def save_env(gameid, env): + """Save environment variables if provided""" + + if not env: + return + + envfile = os.path.join(os.getenv("HOME"), "devkit-game", + gameid + "-env.json") + try: + with open(envfile, "w") as envf: + fcntl.flock(envf, fcntl.LOCK_EX) + json.dump(env, envf) + fcntl.flock(envf, fcntl.LOCK_UN) + except IOError: + raise Exception( + "Unable to open env file for writing: {0}".format(envfile)) + + +def obtain_env(gameid): + """Obtain environment variables for a game, if any were saved""" + + envfile = os.path.join(os.getenv("HOME"), "devkit-game", + gameid + "-env.json") + try: + with open(envfile, "r") as envf: + fcntl.flock(envf, fcntl.LOCK_EX) + env = json.load(envf) + fcntl.flock(envf, fcntl.LOCK_UN) + except IOError: + return {} + return env + + +def save_settings(gameid, data): + """Save settings""" + settingsfile = os.path.join(os.getenv("HOME"), "devkit-game", + gameid + "-settings.json") + settings = dict() + + if data.get('clear_settings', False): + settings = {} + else: + try: + with open(settingsfile, "r") as f: + fcntl.flock(f, fcntl.LOCK_EX) + settings = json.load(f) + fcntl.flock(f, fcntl.LOCK_UN) + except IOError as e: + if (e.errno != errno.ENOENT): + raise + + # Merge settings from new json + if 'settings' in data: + settings.update(data['settings']) + + try: + with open(settingsfile, "w") as f: + fcntl.flock(f, fcntl.LOCK_EX) + json.dump(settings, f) + fcntl.flock(f, fcntl.LOCK_UN) + except (IOError): + raise Exception( + "Unable to open settings file for writing: {0}".format( + settingsfile + )) + + return settings + + +def load_settings(gameid): + settingsfile = os.path.join(os.getenv("HOME"), "devkit-game", gameid + '-settings.json') + + if not os.path.isfile(settingsfile): + return None + + with open(settingsfile, "r") as f: + fcntl.flock(f, fcntl.LOCK_EX) + settings = json.load(f) + fcntl.flock(f, fcntl.LOCK_UN) + + return settings + + +class SteamResponse_Timeout(Exception): + pass + + +class SteamResponse_Error(Exception): + def __init__(self, error_response): + self.error_response = error_response + + def __str__(self): + return self.error_response + + +@contextlib.contextmanager +def wait_on_file_response(path, timeout=5): + """ +The pipe to the Steam Client is one way. +Responses from the Steam Client are written to filesystem. +Protocol is as follows: +- Steam Client creates a 'path.lock' file +- Steam Client writes either 'path' or 'path.error' to indicate a problem +- Steam Client deletes 'path.lock' +- Caller (us) can then read the response + +NOTE 1: this function is used as a context manager and will block until a response comes in or timeout. + +NOTE 2: the files are created by Steam when responding to a command. If the files already exist the response protocol will break. + """ + lock_path = '{0}.lock'.format(path) + error_path = '{0}.error'.format(path) + max_count = timeout + while True: + time.sleep(1) + if os.path.exists(error_path) or os.path.exists(path) and not os.path.exists(lock_path): + if os.path.exists(error_path): + with open(error_path, 'r') as f: + fcntl.flock(f, fcntl.LOCK_EX) + error_response = f.read() + fcntl.flock(f, fcntl.LOCK_UN) + raise SteamResponse_Error(error_response) + with open(path, 'r') as f: + fcntl.flock(f, fcntl.LOCK_EX) + success_response = f.read() + yield success_response + fcntl.flock(f, fcntl.LOCK_UN) + return + max_count -= 1 + if max_count > 0: + continue + raise SteamResponse_Timeout() + + +# Setting up as a context manager so we never miss the deletion +# Creating a temporary .lock file to guard the create operation +@contextlib.contextmanager +def create_pid(pid_path): + os.makedirs(os.path.dirname(pid_path), exist_ok=True) + lock_path = '{0}.lock'.format(pid_path) + try: + lock_file = os.open(lock_path, os.O_CREAT | os.O_EXCL) + except IOError as e: + logger.error('cannot create lock file %s for pid file %s', lock_path, pid_path) + logger.error('remove the lock file manually and run again if you are confident no other instance is active') + raise + + pid_file = open(pid_path,'w') + pid_file.write(str(os.getpid())) + pid_file.flush() + os.close(lock_file) + os.unlink(lock_path) + try: + yield pid_file + finally: + pid_file.close() + # Assume that's atomic and all is well, no need for another .lock + os.unlink(pid_path) diff --git a/frame/devkit-utils/devkit_utils/resolve.py b/frame/devkit-utils/devkit_utils/resolve.py new file mode 100644 index 0000000..1329926 --- /dev/null +++ b/frame/devkit-utils/devkit_utils/resolve.py @@ -0,0 +1,87 @@ +#!/usr/bin/env python3 + +import sys +import os +import logging +from urllib.parse import quote_plus as urllib_quote_plus +import json +import tempfile + +from . import validate_steam_client +from . import execute_steam_client_command +from . import wait_on_file_response + +import logging as logging_module +logger = logging_module.getLogger(__name__) + + +def resolve_shortcuts(): + # make sure there is a steam client online that we can talk to before doing anything + validate_steam_client() + + # scan the devkit games + installed_gameids = set([]) + devkit_game_path = os.path.expanduser('~/devkit-game') + if not os.path.exists(devkit_game_path): + logger.info('%r does not exist, creating', devkit_game_path) + os.mkdir(devkit_game_path) + entries = sorted(os.scandir(devkit_game_path), key=lambda entry: entry.name) + directories = [e for e in entries if e.is_dir()] + for d in directories: + gameid = d.name + file_names = [f.name for f in entries if f.is_file() and f.name.startswith(gameid)] + has_argv = '{0}-argv.json'.format(gameid) in file_names + has_settings = '{0}-settings.json'.format(gameid) in file_names + if (not has_argv and not has_settings): + logger.info('Subfolder %r in %r is not accompanied by devkit configuration files, ignoring', d.name, devkit_game_path) + continue + logger.info('Found installed Devkit Game: %r', gameid) + installed_gameids.add(gameid) + + # ask the Steam Client which Devkit Games are registered + with tempfile.TemporaryDirectory(prefix='list-shortcuts') as tempdir: + response = os.path.join(tempdir, 'shortcuts.json') + cmd = 'list-shortcuts?response={}'.format( + urllib_quote_plus(os.path.join(response)) + ) + # send the request + execute_steam_client_command(cmd) + with wait_on_file_response(response) as response: + client_shortcuts = json.loads(response) + logger.debug(client_shortcuts) + assert client_shortcuts['version'] == 2 + registered_gameids = set([]) + logger.info('Steam Client has %d registered devkit game(s)', len(client_shortcuts['gameids'])) + for gameid in client_shortcuts['gameids']: + logger.info('Found Devkit Game registered with Steam Client: %r', gameid) + registered_gameids.add(gameid) + + # any registered game that is not found installed on disk needs to be removed + for remove_gameid in registered_gameids - installed_gameids: + with tempfile.TemporaryDirectory(prefix='delete-shortcut') as tempdir: + logger.info('Removing stale registered Devkit Game: %r', remove_gameid) + response = os.path.join(tempdir, 'shortcut-deleted') + cmd = 'delete-shortcut?response={}&gameid={}'.format( + urllib_quote_plus(response), + remove_gameid + ) + execute_steam_client_command(cmd) + with wait_on_file_response(response) as response: + logger.info('from Steam Client: %s', response.strip()) + + # any installed game that is not found registered needs to be added + for add_gameid in installed_gameids - registered_gameids: + with tempfile.TemporaryDirectory(prefix='create-shortcut') as tempdir: + logger.info('Registering installed Dekit Game: %r', add_gameid) + response = os.path.join(tempdir, 'registered') + cmd = 'create-shortcut?response={}&gameid={}&directory={}'.format( + urllib_quote_plus(response), + add_gameid, + urllib_quote_plus(devkit_game_path) + ) + execute_steam_client_command(cmd) + with wait_on_file_response(response) as response: + logger.info('from Steam Client: %s', response.strip()) + +if __name__ == '__main__': + resolve_shortcuts() diff --git a/frame/devkit-utils/steam-client-create-shortcut b/frame/devkit-utils/steam-client-create-shortcut new file mode 100644 index 0000000..802edce --- /dev/null +++ b/frame/devkit-utils/steam-client-create-shortcut @@ -0,0 +1,92 @@ +#!/usr/bin/env python3 + +import os +import logging +import argparse +import json +import platform +import tempfile +from urllib.parse import quote_plus as urllib_quote_plus + +import devkit_utils + +logging.basicConfig(format='%(message)s', level=logging.DEBUG) +logger = logging.getLogger() + +DEVKIT_TOOL_FOLDER = os.path.expanduser('~/devkit-game') + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('--verbose', required=False, action='store_true') + parser.add_argument('--parms', required=True, action='store') + conf = parser.parse_args() + + if conf.verbose: + logger.setLevel(logging.DEBUG) + else: + logger.setLevel(logging.INFO) + + parms = json.loads(conf.parms) + gameid = parms['gameid'] + directory = parms['directory'] + assert os.path.isdir(directory) + + force_appid = parms['force_appid'] + steam_appid_path = os.path.join(directory, 'steam_appid.txt') + if force_appid: + with open(steam_appid_path, 'w') as f: + f.write(force_appid + '\n') + logger.info(f'Wrote {steam_appid_path} with AppID {force_appid}') + elif os.path.exists(steam_appid_path): + # don't overwrite an existing steam_appid.txt file from the content tree + # NOTE: if the user sets an AppID through the tool, then delete it, we may leave it in place .. + # (that's ok for now, do a clean upload if you want to get rid of it) + logger.info(f'{steam_appid_path} already exists, leaving it in place') + + # Lepton (Android runtime) titles: write UECommandLine.txt next to the .apk + is_lepton = parms['settings']['compat_tool'] == 'lepton' + uecommandline = parms['lepton_args'] if is_lepton else '' + uecommandline_path = os.path.join(directory, 'UECommandLine.txt') + if uecommandline: + with open(uecommandline_path, 'w') as f: + f.write(uecommandline + '\n') + logger.info(f'Wrote {uecommandline_path}') + elif os.path.exists(uecommandline_path): + # don't overwrite an existing UECommandLine.txt file from the content tree + # NOTE: if the user sets cmdline args through the tool, then clears them, we may leave it in place .. + # (that's ok for now, do a clean upload if you want to get rid of it) + logger.info(f'{uecommandline_path} already exists, leaving it in place') + + logger.info(f'Updating command line and runtime settings for {gameid} on {platform.node()}') + devkit_utils.save_argv(gameid, parms['argv']) + devkit_utils.save_env(gameid, parms['env']) + devkit_utils.save_settings(gameid, parms) + + ret = {} + + try: + devkit_utils.validate_steam_client() + except devkit_utils.SteamClientNotRunningException as e: + skipping = 'The Steam client is not running. Registration did not complete.' + logger.warning(skipping) + ret['error'] = skipping + else: + with tempfile.TemporaryDirectory(prefix='create-shortcut') as tempdir: + logger.info(f'Registering Devkit Game {gameid} with Steam Client') + response = os.path.join(tempdir, 'registered') + cmd = 'create-shortcut?response={}&gameid={}'.format( + urllib_quote_plus(response), + gameid, + ) + devkit_utils.execute_steam_client_command(cmd) + try: + with devkit_utils.wait_on_file_response(response) as success_response: + logger.debug(success_response) + ret['success'] = success_response + except devkit_utils.SteamResponse_Timeout: + ret['error'] = 'timeout - Steam client did not respond to registration request' + except devkit_utils.SteamResponse_Error as e: + ret['error'] = e.error_response + + # response gets written out to stdout + print(json.dumps(ret)) diff --git a/frame/devkit-utils/steam-devkit-rpc b/frame/devkit-utils/steam-devkit-rpc new file mode 100755 index 0000000..409542e --- /dev/null +++ b/frame/devkit-utils/steam-devkit-rpc @@ -0,0 +1,48 @@ +#!/usr/bin/env python3 + +import sys +import os +import logging +import argparse +import tempfile +import urllib.parse +import re + +import devkit_utils + +logging.basicConfig(format='%(message)s', level=logging.DEBUG) +logger = logging.getLogger() + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('command') + parser.add_argument('args', nargs='*') + conf = parser.parse_args() + + try: + devkit_utils.validate_steam_client() + except devkit_utils.SteamClientNotRunningException as e: + logger.error(repr(e)) + sys.exit(-1) + else: + with tempfile.TemporaryDirectory(prefix='steam-devkit-rpc') as tempdir: + response = os.path.join(tempdir, 'steam-devkit-rpc') + parms = { + 'response' : response, + } + for arg in conf.args: + (k, v) = re.split('=', arg) + parms[k] = v + cmd = f'{conf.command}/?{urllib.parse.urlencode(parms)}' + devkit_utils.execute_steam_client_command(cmd) + try: + with devkit_utils.wait_on_file_response(response) as success_response: + logger.info('success') + sys.stdout.write(success_response) + sys.exit(0) + except devkit_utils.SteamResponse_Timeout: + logger.error('timeout') + except devkit_utils.SteamResponse_Error as e: + logger.error('failed') + sys.stdout.write(e.error_response) + sys.exit(-1) diff --git a/frame/devkit-utils/steamos-delete b/frame/devkit-utils/steamos-delete new file mode 100644 index 0000000..c73a1bd --- /dev/null +++ b/frame/devkit-utils/steamos-delete @@ -0,0 +1,60 @@ +#!/usr/bin/env python3 + +import sys +import os +import shutil +import logging +import argparse +import subprocess + +import devkit_utils.resolve + +logging.basicConfig(format='%(message)s', level=logging.DEBUG) +logger = logging.getLogger(__name__) + +DEVKIT_TOOL_FOLDER = os.path.expanduser('~/devkit-game') + +def session_select_command(): + if shutil.which('holo-session-select'): + return 'holo-session-select' + return 'steamos-session-select' + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('--verbose', required=False, action='store_true') + parser.add_argument('--delete-title', required=False, action='store', help='Delete a devkit title by name') + parser.add_argument('--delete-all-titles', required=False, action='store_true', default=False, help='Delete all devkit titles uploaded') + parser.add_argument('--reset-steam-client', required=False, action='store_true', default=False, help='Reset Steam client and delete all local Steam content') + conf = parser.parse_args() + + if conf.verbose: + logger.setLevel(logging.DEBUG) + else: + logger.setLevel(logging.INFO) + + if conf.delete_all_titles: + subprocess.check_call('rm -rf ~/devkit-game/*', shell=True) + elif conf.delete_title: + gamepath = os.path.expanduser( os.path.join( '~/devkit-game', conf.delete_title ) ) + if not os.path.isdir(gamepath): + print(f'Not found: {gamepath}') + else: + subprocess.check_call(f'rm -r {gamepath}', shell=True) + + # synchronize the Steam client's view of the devkit games with the on disk state + try: + devkit_utils.resolve.resolve_shortcuts() + except Exception as e: + logger.warning(f'Steam client sync of devkit games failed: {e}') + + if conf.reset_steam_client: + # first make sure any sideloaded trampoline has been deleted + devkit_steam_trampoline_path = os.path.join(DEVKIT_TOOL_FOLDER, 'devkit-steam') + if os.path.exists(devkit_steam_trampoline_path): + os.unlink(devkit_steam_trampoline_path) + + # wipe the local Steam install + subprocess.check_call(f'rm -rf ~/.local/share/Steam', shell=True) + + # restart the session, which will initiate a reinstall of Steam from the OS client + subprocess.check_call([session_select_command(), 'gamescope']) diff --git a/frame/devkit-utils/steamos-dump-controller-config b/frame/devkit-utils/steamos-dump-controller-config new file mode 100644 index 0000000..4195119 --- /dev/null +++ b/frame/devkit-utils/steamos-dump-controller-config @@ -0,0 +1,57 @@ +#!/usr/bin/env python3 + +import os +import logging +import argparse +import tempfile +import json +from urllib.parse import quote_plus as urllib_quote_plus + +import devkit_utils + +logging.basicConfig(format='%(message)s', level=logging.DEBUG) +logger = logging.getLogger(__name__) + +DEVKIT_TOOL_FOLDER = os.path.expanduser('~/devkit-game') + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('--verbose', required=False, action='store_true') + parser.add_argument('--appid', required=False, action='store') + parser.add_argument('--gameid', required=False, action='store') + conf = parser.parse_args() + + if conf.verbose: + logger.setLevel(logging.DEBUG) + else: + logger.setLevel(logging.INFO) + + ret = {} + + try: + devkit_utils.validate_steam_client() + except devkit_utils.SteamClientNotRunningException as e: + skipping = 'The Steam client is not running.' + logger.warning(skipping) + ret['error'] = skipping + else: + with tempfile.TemporaryDirectory(prefix='controller-config') as tempdir: + response = os.path.join(tempdir, 'dumpcontrollerconfig') + cmd = f'dumpcontrollerconfig?response={urllib_quote_plus(response)}' + if conf.appid: + cmd += f'&appid={conf.appid}' + if conf.gameid: + cmd += f'&gameid={conf.gameid}' + logger.debug(f'command: {cmd}') + devkit_utils.execute_steam_client_command(cmd) + try: + with devkit_utils.wait_on_file_response(response) as success_response: + logger.debug(success_response) + ret['success'] = success_response + except devkit_utils.SteamResponse_Timeout: + ret['error'] = 'timeout - Steam did not respond to the command request' + except devkit_utils.SteamResponse_Error as e: + ret['error'] = e.error_response + + # response gets written out to stdout + print(json.dumps(ret)) diff --git a/frame/devkit-utils/steamos-get-status b/frame/devkit-utils/steamos-get-status new file mode 100755 index 0000000..caaa010 --- /dev/null +++ b/frame/devkit-utils/steamos-get-status @@ -0,0 +1,443 @@ +#!/usr/bin/env python3 + +import sys +import os +import re +import shutil +import subprocess +import logging +import enum +import argparse +import json +import shlex +import datetime +import pathlib +import socket + +logging.basicConfig(format='%(message)s', level=logging.DEBUG) +logger = logging.getLogger(__name__) + +DEVKIT_TOOL_FOLDER = os.path.expanduser('~/devkit-game') +STEAM_EXTRA_ARGS_FILE = os.path.expanduser('~/.config/systemd/user/steam.service.d/extra_args.conf') + +WIRELESS_DISABLE_POWER_MANAGEMENT = '/usr/bin/steamos-polkit-helpers/steamos-disable-wireless-power-management' + +# Must match in gui2.py +class SteamStatus(enum.Enum): + NOT_RUNNING = 0 + ERROR = 1 + OS = 2 + OS_DEV = 3 + SIDELOADED = 4 + + @classmethod + def from_string(cls, status_str): + if not status_str: + return cls.ERROR + try: + if '.' in status_str: + name = status_str.split('.')[-1] + else: + name = status_str + return cls[name] + except KeyError: + return cls.ERROR + + @property + def description(self): + DESCRIPTIONS = { + SteamStatus.NOT_RUNNING: 'not running', + SteamStatus.OS: 'OS client', + SteamStatus.OS_DEV: 'OS client dev mode', + SteamStatus.SIDELOADED: 'sideloaded client', + SteamStatus.ERROR: 'error', + } + return DESCRIPTIONS[self] + + +class SteamConfig(enum.Enum): + ERROR = 1 + # Matching the SteamStatus numeric values + OS = 2 + OS_DEV = 3 + SIDELOADED = 4 + + @property + def description(self): + DESCRIPTIONS = { + SteamConfig.OS: 'OS client', + SteamConfig.OS_DEV: 'OS client dev mode', + SteamConfig.SIDELOADED: 'sideloaded client', + SteamConfig.ERROR: 'error', + } + return DESCRIPTIONS[self] + + +SESSION_NAMES = ['gamescope', 'plasma-x11', 'plasma-x11-persistent', 'plasma-wayland', 'plasma-wayland-persistent'] + +class SessionConfig(enum.IntEnum): + # note: matches SESSION_NAMES indexes + GAMESCOPE = 0 + PLASMA_X11 = 1 + PLASMA_X11_PERSISTENT = 2 + PLASMA_WAYLAND = 3 + PLASMA_WAYLAND_PERSISTENT = 4 + ERROR = 5 + +def cef_debugging(): + '''Only sane way to check is to look for the listening port.''' + ret = subprocess.run('/usr/bin/ss -l -t -n -p | grep steamwebhelper | grep 8080 > /dev/null', shell=True) + return ( ret.returncode == 0 ) + +def steam_process_get_path_and_args(): + ret = subprocess.run(['pgrep', '-a', '-x', 'steam'], capture_output=True, text=True) + if ret.returncode != 0: + return None + # Proton may run a dummy 'steam' process that confused previous implementations of this logic + # look for a process who's real filename is 'steam' + for l in ret.stdout.splitlines(): + try: + pid = int(l.split(' ')[0]) + except: + continue + rp = os.path.realpath(f'/proc/{pid}/exe') + if os.path.basename(rp) == 'steam': + try: + with open(f'/proc/{pid}/cmdline', 'rb') as f: + cmdline = f.read() + argv = [a.decode('utf-8', errors='replace') for a in cmdline.split(b'\x00') if a] + if len(argv) >= 2: + path = argv[0] + args = argv[1:] + # strip -srt-logger-opened: injected by steam.sh at runtime + args = [a for a in args if a != '-srt-logger-opened'] + return (path, args) + return (argv[0], []) + except Exception as e: + logger.warning(f'Failed to read /proc/{pid}/cmdline: {e}') + return None + +def steam_process_get_path(): + try: + (path, _) = steam_process_get_path_and_args() + except: + return None + return path + +def steam_process_get_args(): + try: + (_, args) = steam_process_get_path_and_args() + except: + return '' + return args + +def steam_status(): + '''What is the status of the Steam client on the system?''' + s = steam_process_get_path() + if s is None: + return SteamStatus.NOT_RUNNING + if s.find('.local/share/Steam/') != -1: + if os.path.exists(os.path.expanduser('~/devkit-game/devkit-steam')): + return SteamStatus.OS_DEV + return SteamStatus.OS + if s.find('devkit-game/steam/') != -1 or s.find('devkit-game/steamdeckard/') != -1: + return SteamStatus.SIDELOADED + logger.warning(f'could not interpret pgrep result to determine steam client status: {s!r}') + return SteamStatus.ERROR + +def steam_configuration(): + '''How is the Steam client configured to run?''' + devkit_steam_trampoline_path = os.path.join(DEVKIT_TOOL_FOLDER, 'devkit-steam') + if not os.path.exists(devkit_steam_trampoline_path): + return SteamConfig.OS + t = open(devkit_steam_trampoline_path, 'rt').read() + if t.find('SteamStatus.OS_DEV') != -1: + return SteamConfig.OS_DEV + if t.find('SteamStatus.SIDELOADED') != -1: + return SteamConfig.SIDELOADED + logger.warning(f'could not determine what {devkit_steam_trampoline_path} means to do') + return SteamConfig.ERROR + + +def osclient_branch(is_deckard): + '''Which branch is the default Steam 'OS client' configured to use?''' + # makes more sense to return strings here + beta_path = os.path.expanduser('~/.steam/steam/package/beta') + if not os.path.exists(beta_path): + return 'default' # not sure that's valid actually - would be the desktop client, which will only run in desktop mode .. + t = open(beta_path, 'rt').readline().strip('\n') + try: + p = 'steamdeck_(.*)' + if re.match(p, t): + branch = re.split(p, t)[1] + return branch + # internal builds + p = 'steampal_(.*)_.*' + if re.match(p, t): + branch = re.split(p, t)[1] + return branch + if is_deckard: + p = 'linux_arm64_(.*)_.*' + if re.match(p, t): + branch = re.split(p, t)[1] + return branch + raise Exception('no match') + except: # noqa: E722 + logger.warning(f'could not determine the OS client branch config: {t!r}') + return 'error' + +def osclient_version(conf): + '''Which version is the Steam 'OS client'?''' + if conf.is_deckard: + # old Steam client was using linuxarm64/, which is now reserved for the SDK binaries + for folder in ('linuxarm64', 'steamrtarm64'): + fn = os.path.expanduser(f'~/.steam/steam/{folder}/builddate.txt') + if os.path.exists(fn): + return open(fn, 'rt').read() + return 'Unknown - no builddate.txt' + beta_path = os.path.expanduser('~/.steam/steam/package/beta') + if not os.path.exists(beta_path): + logger.warning(f'not found: {beta_path}') + return None + t = open(beta_path, 'rt').readline().strip('\n') + manifest = os.path.expanduser(f'~/.steam/steam/package/steam_client_{t}_ubuntu12.manifest') + if not os.path.exists(manifest): + logger.warning(f'not found: {manifest}') + return None + try: + version = int(re.search('"version".*"(.*)"', open(manifest,'rt').read()).group(1)) + return version + except: + logger.warning(f'could not parse version out of {manifest}') + return None + +def session_config(): + '''What is the graphics session configuration?''' + # RESTART_SESSION writes this file + conf_file = '/etc/sddm.conf.d/zz-steamos-autologin.conf' + if not os.path.exists(conf_file): + # fallback to the OS default + conf_file = '/etc/sddm.conf.d/steamos.conf' + if os.path.exists(conf_file): + s = open(conf_file, 'rt').read() + if s.find('plasmawayland.desktop') != -1: + return SessionConfig.PLASMA_WAYLAND_PERSISTENT + if s.find('plasma.desktop') != -1: + return SessionConfig.PLASMA_X11_PERSISTENT + if s.find('gamescope-wayland.desktop') != -1: + return SessionConfig.GAMESCOPE + if s.find('plasma-steamos-oneshot.desktop') != -1: + return SessionConfig.PLASMA_X11 + if s.find('plasma-steamos-wayland-oneshot.desktop') != -1: + return SessionConfig.PLASMA_WAYLAND + else: + # if the conf file doesn't exist we are likely in the default config + # check for a running gamescope for sanity + if subprocess.call('pgrep -a -x gamescope', shell=True, stdout=subprocess.DEVNULL) == 0: + return SessionConfig.GAMESCOPE + # couldn't figure it out, halp + return SessionConfig.ERROR + +def session_select_command(): + if shutil.which('holo-session-select'): + return 'holo-session-select' + return 'steamos-session-select' + +def get_os_info(): + os_info = {} + try: + for k, v in [ s.split('=') for s in open('/etc/os-release').read().split('\n') if len(s) > 0 ]: + os_info[k] = v.strip('"') + except Exception as e: + logger.error(e) + logger.error('Failed to parse OS release file') + return os_info + +def steam_default_args(conf): + if conf.is_deckard: + # Frame currently uses a different setup + return [] + + try: + if os.path.exists('/usr/lib/steamos/steam-launcher'): + output = subprocess.check_output('cat /usr/lib/steamos/steam-launcher | grep ^steamargs=', + shell=True, + universal_newlines=True) + ret = [ v.strip('"') for v in re.findall('\".*?\"', output) ] + return ret + except: + logger.warning('Failed to obtain steam default arguments from /usr/lib/steamos/steam-launcher') + + # Legacy SteamOS + try: + output = subprocess.check_output('cat /usr/bin/gamescope-session | grep ^steamargs', + shell=True, + universal_newlines=True) + ret = [ v.strip('"') for v in re.findall('\".*?\"', output) ] + except: + logger.warning('Failed to obtain steam default arguments from /usr/bin/gamescope-session') + + # Hardcoded fallback + return ['-steamos3', '-steampal', '-steamdeck', '-gamepadui'] + +def frame_osclient_extra_args(conf, steam_status): + if not conf.is_deckard or steam_status != SteamStatus.OS: + return None + if os.path.exists(STEAM_EXTRA_ARGS_FILE): + try: + content = open(STEAM_EXTRA_ARGS_FILE, 'rt').read() + match = re.search(r'Environment="STEAM_EXTRA_ARGS=(.*)"', content) + if match: + return match.group(1).replace('\\"', '"') + except Exception as e: + logger.warning(f'Failed to parse steam extra args: {e}') + return None + +def user_password_is_set(): + ret = subprocess.run('passwd', stdin=subprocess.DEVNULL, shell=True, capture_output=True, universal_newlines=True) + logger.debug(repr(ret)) + return (ret.stderr.find('Current password:') != -1) + +def steam_launch_flags(): + '''Pull various steam flags that affect title execution.''' + ret = {} + if not 'XDG_RUNTIME_DIR' in os.environ: + logger.warning('XDK_RUNTIME_DIR is not set') + return ret + env_folder = os.path.join(os.environ['XDG_RUNTIME_DIR'], 'steam/env') + if not os.path.isdir(env_folder): + return ret + for fn in os.listdir(env_folder): + filepath = os.path.join(env_folder, fn) + content = open(filepath, 'rt').read() + # Check if this is a declaration file with key=value pairs + if content.count('\n') > 1 or '=' in content: + # Parse key=value format with comments + for line in content.splitlines(): + line = line.strip() + # Skip comments and empty lines + if not line or line.startswith('#'): + continue + # Parse key=value pairs + if '=' in line: + key, value = line.split('=', 1) + ret[key.strip()] = value.strip() + else: + # Legacy format: filename is the key, file content is the value + ret[fn] = content.strip('\n') + return ret + +def renderdoc_replay_server_running(): + ret = subprocess.run(['pgrep', '-x', 'renderdoccmd'], capture_output=True) + return ret.returncode == 0 + + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('--verbose', required=False, action='store_true') + parser.add_argument('--json', required=False, action='store_true') + conf = parser.parse_args() + + if conf.verbose: + logger.setLevel(logging.DEBUG) + else: + logger.setLevel(logging.INFO) + + os_info = get_os_info() + assert os_info is not None + conf.is_deckard = os_info.get('VARIANT_ID', None) == 'vr' + os_name = os_info.get('PRETTY_NAME', None) + os_version = os_info.get('BUILD_ID', None) + + _steam_launch_flags = steam_launch_flags() + + if not conf.is_deckard: + # this bit of cargo cult is Steam Deck only + try: + # disable wireless power management for devkit usage: less latency on commands + subprocess.check_call(WIRELESS_DISABLE_POWER_MANAGEMENT) + except subprocess.CalledProcessError as e: + logger.warning(e) + + session_config = session_config() + # enum -> human readable + session_status = SESSION_NAMES[session_config] if session_config != SessionConfig.ERROR else 'error' + + steam_status = steam_status() + cef_debugging_enabled = False + if steam_status != SteamStatus.NOT_RUNNING: + cef_debugging_enabled = cef_debugging() + steam_configuration = steam_configuration() + osclient_branch = osclient_branch(conf.is_deckard) + osclient_version = osclient_version(conf) + + steam_status_description = steam_status.description + if steam_status in (SteamStatus.OS, SteamStatus.OS_DEV) : + steam_status_description += f', on branch {osclient_branch!r}' + if osclient_version is not None: + if conf.is_deckard: + # we get builddate.txt + steam_status_description += f', {osclient_version}' + else: + utc_date_string = datetime.datetime.fromtimestamp(osclient_version, datetime.UTC).isoformat() + steam_status_description += f', version {osclient_version} {utc_date_string}' + + has_side_loaded_client = os.path.exists( + os.path.join( + DEVKIT_TOOL_FOLDER, + 'steam' + ) + ) + + _user_password_is_set = user_password_is_set() + + _renderdoc_replay_server_running = renderdoc_replay_server_running() + _renderdoc_layer_enabled = _steam_launch_flags.get('ENABLE_VULKAN_RENDERDOC_CAPTURE', '0') == '1' + + _hostname = socket.gethostname() + + if conf.json: + ret = { + 'is_deckard': conf.is_deckard, + 'hostname': _hostname, + 'os_name': os_name, + 'os_version': os_version, + 'os_info': os_info, + 'session_status': session_status, + 'session_options': SESSION_NAMES, + 'session_select': session_select_command(), + 'steam_status': str(steam_status), + 'cef_debugging_enabled': cef_debugging_enabled, + 'steam_status_description': steam_status_description, + 'steam_configuration': str(steam_configuration), + 'steam_osclient_branch': osclient_branch, + 'steam_osclient_version': osclient_version, + 'has_side_loaded_client': has_side_loaded_client, + 'steam_default_args': steam_default_args(conf), + 'steam_current_args': steam_process_get_args(), + 'frame_osclient_extra_args': frame_osclient_extra_args(conf, steam_status), + 'user_password_is_set': _user_password_is_set, + 'steam_launch_flags': _steam_launch_flags, + 'renderdoc_layer_enabled': _renderdoc_layer_enabled, + 'renderdoc_replay_server_running': _renderdoc_replay_server_running, + } + json.dump(ret, sys.stdout, sort_keys=True, indent=4) + else: + logger.info(f'Hostname : {_hostname}') + logger.info(f'OS : {os_name}') + logger.info(f'OS version : {os_version}') + logger.info(f'Session mode is : {session_status}') + logger.info(f'Session select command : {session_select_command()}') + logger.info(f'Steam client status : {steam_status_description}') + logger.info(f'Steam client args : {steam_process_get_args()!r}') + logger.info(f"Steam extra args (Frame) : {frame_osclient_extra_args(conf, steam_status)!r}") + logger.info(f"Steam CEF debug : {'enabled' if cef_debugging_enabled else 'disabled'}") + logger.info(f'Steam client config : {steam_configuration.description}') + logger.info(f'Steam OS client branch : {osclient_branch}') + logger.info(f'Steam OS client version : {osclient_version}') + logger.info(f"Sideloaded client : {'available' if has_side_loaded_client else 'not installed'}") + logger.info(f'OS client arguments : {steam_default_args(conf)!r}') + logger.info(f"User password is set : {'yes' if _user_password_is_set else 'no'}") + logger.info(f"Steam launch flags : {_steam_launch_flags}") + logger.info(f"RenderDoc layer enabled : {'yes' if _renderdoc_layer_enabled else 'no'}") + logger.info(f"RenderDoc replay running : {'yes' if _renderdoc_replay_server_running else 'no'}") diff --git a/frame/devkit-utils/steamos-list-games b/frame/devkit-utils/steamos-list-games new file mode 100644 index 0000000..31cc305 --- /dev/null +++ b/frame/devkit-utils/steamos-list-games @@ -0,0 +1,34 @@ +#!/usr/bin/env python3 + +import os +import logging +import argparse +import getpass +import json +from subprocess import DEVNULL + +logging.basicConfig(format='%(message)s', level=logging.DEBUG) +logger = logging.getLogger(__name__) + +DEVKIT_TOOL_FOLDER = os.path.expanduser('~/devkit-game') + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('--verbose', required=False, action='store_true') + conf = parser.parse_args() + + if conf.verbose: + logger.setLevel(logging.DEBUG) + else: + logger.setLevel(logging.INFO) + + ret = [] + if os.path.isdir(DEVKIT_TOOL_FOLDER): + for filename in os.listdir(DEVKIT_TOOL_FOLDER): + gamefolder = os.path.join(DEVKIT_TOOL_FOLDER, filename) + if os.path.isdir(gamefolder): + ret.append( { + 'gameid': filename, + } ) + + print(json.dumps(ret)) diff --git a/frame/devkit-utils/steamos-prepare-upload b/frame/devkit-utils/steamos-prepare-upload new file mode 100644 index 0000000..8d59874 --- /dev/null +++ b/frame/devkit-utils/steamos-prepare-upload @@ -0,0 +1,50 @@ +#!/usr/bin/env python3 + +import sys +import os +import logging +import argparse +import getpass +import json +import shutil +import subprocess + +logging.basicConfig(format='%(message)s', level=logging.DEBUG) +logger = logging.getLogger(__name__) + +DEVKIT_TOOL_FOLDER = os.path.expanduser('~/devkit-game') + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('--verbose', required=False, action='store_true') + parser.add_argument('--gameid', required=True, action='store') + parser.add_argument('--restart-steam', required=False, default='0', action='store') + parser.add_argument('--use-mask-unmask', required=False, default='0', action='store') + parser.add_argument('--prevent-auto-repair', required=False, default='0', action='store') + conf = parser.parse_args() + + if conf.verbose: + logger.setLevel(logging.DEBUG) + else: + logger.setLevel(logging.INFO) + + directory = os.path.join( + os.path.expanduser(DEVKIT_TOOL_FOLDER), + conf.gameid + ) + os.makedirs(directory, exist_ok=True) + + INHIBIT_SENTINEL = os.path.expanduser('~/.config/inhibit-short-session-tracker') + if int(conf.prevent_auto_repair) == 1: + open(INHIBIT_SENTINEL, 'w').close() + logger.info(f'Created sentinel file: {INHIBIT_SENTINEL}') + elif os.path.exists(INHIBIT_SENTINEL): + os.remove(INHIBIT_SENTINEL) + logger.info(f'Removed sentinel file: {INHIBIT_SENTINEL}') + + + ret = { + 'user': getpass.getuser(), + 'directory': directory, + } + print(json.dumps(ret)) diff --git a/frame/devkit-utils/steamos-set-password.sh b/frame/devkit-utils/steamos-set-password.sh new file mode 100644 index 0000000..26b4bc3 --- /dev/null +++ b/frame/devkit-utils/steamos-set-password.sh @@ -0,0 +1,7 @@ +#!/bin/bash +# meant to be executed remotely/interactively for password prompts + +# there's some annoying trash at the top of the remote ssh screen +clear +passwd +sleep 2 diff --git a/frame/devkit-utils/steamos-set-steam-client b/frame/devkit-utils/steamos-set-steam-client new file mode 100644 index 0000000..7d463af --- /dev/null +++ b/frame/devkit-utils/steamos-set-steam-client @@ -0,0 +1,157 @@ +#!/usr/bin/env python3 + +import sys +import os +import logging +import argparse +import enum +import subprocess +import shutil +import pathlib + +logging.basicConfig(format='%(message)s', level=logging.DEBUG) +logger = logging.getLogger(__name__) + +DEVKIT_TOOL_FOLDER = os.path.expanduser('~/devkit-game') +# NOTE: only relevant to Frame + OS client with extra arguments +# sideloaded client on Frame supports full command line edit instead +STEAM_EXTRA_ARGS_FILE = os.path.expanduser('~/.config/systemd/user/steam.service.d/extra_args.conf') + +class SteamStatus(enum.Enum): + # Supported values from steamos-get-status + OS = 2 + OS_DEV = 3 + SIDELOADED = 4 + +STATUS_STRINGS = [ + ( SteamStatus.OS, 'SteamStatus.OS' ), + ( SteamStatus.OS_DEV, 'SteamStatus.OS_DEV' ), + ( SteamStatus.SIDELOADED, 'SteamStatus.SIDELOADED' ), +] + +# gamescope-session passes the execution to this script if it exists rather than start steam itself +DEVKIT_STEAM_TRAMPOLINE = os.path.expanduser('~/devkit-game/devkit-steam') + +# this script executes the sideloaded Steam client (part of the steamos-devkit-service package) +SIDE_LOADED_STEAM_CLIENT = '/usr/share/steamos-devkit/bin/devkit-standalone.py' + +def write_trampoline(text): + with open(DEVKIT_STEAM_TRAMPOLINE, 'w') as devkit_steam: + devkit_steam.write(text) + devkit_steam.flush() + os.chmod(DEVKIT_STEAM_TRAMPOLINE, 0o770) + # trying really hard to avoid leaving a zero sized trampoline if the deck is about to hang on the session restart coming next + subprocess.run(['/usr/bin/sync', DEVKIT_TOOL_FOLDER]) + +def get_os_info(): + os_info = {} + try: + for k, v in [ s.split('=') for s in open('/etc/os-release').read().split('\n') if len(s) > 0 ]: + os_info[k] = v.strip('"') + except Exception as e: + logger.error(e) + logger.error('Failed to parse OS release file') + return os_info + +if __name__ == '__main__': + parser = argparse.ArgumentParser() + parser.add_argument('--verbose', required=False, action='store_true') + parser.add_argument('--client', action='store', required=True, choices=[ v[1] for v in STATUS_STRINGS ]) + parser.add_argument('--args', action='store', required=False, help='steam client command line arguments') + parser.add_argument('--gameid', required=True, action='store') + parser.add_argument('--gdbserver', action='store_true', required=False) + conf = parser.parse_args() + + if conf.verbose: + logger.setLevel(logging.DEBUG) + else: + logger.setLevel(logging.INFO) + + target = [ v for v in STATUS_STRINGS if v[1] == conf.client ][0][0] + logging.info(f'Set steam client on device to {target}') + + if os.path.exists(DEVKIT_STEAM_TRAMPOLINE): + os.unlink(DEVKIT_STEAM_TRAMPOLINE) + + os_info = get_os_info() + assert os_info is not None + is_deckard = os_info.get('VARIANT_ID', None) == 'vr' + + if target == SteamStatus.OS: + if is_deckard: + # conf.args is the extra arguments for the normal Steam 'OS client', update it now + if conf.args is None or conf.args == '': + logger.info('Clearning extra arguments for normal Steam client') + if os.path.exists(STEAM_EXTRA_ARGS_FILE): + os.unlink(STEAM_EXTRA_ARGS_FILE) + subprocess.run(['systemctl', '--user', 'daemon-reload'], check=True) + else: + logger.info(f'Setting extra arguments for normal Steam client: {conf.args}') + os.makedirs(os.path.dirname(STEAM_EXTRA_ARGS_FILE), exist_ok=True) + with open(STEAM_EXTRA_ARGS_FILE, 'wt') as extra_args_file: + escaped_args = conf.args.replace('"', '\\"') + extra_args_file.write(f'[Service]\nEnvironment="STEAM_EXTRA_ARGS={escaped_args}"') + subprocess.run(['systemctl', '--user', 'daemon-reload'], check=True) + + # When disabling a sideloaded client, also delete the ~/.steam symlinks: + # They will be re-created by the OS client when starting, + # this prevents SteamVR trying to use the sideloaded binaries that are still there for the steam API. + # (this may happen because SteamVR starts before Steam starts and has a chance to set those symlinks correctly) + for path in pathlib.Path(os.path.expanduser('~/.steam')).glob('*'): + if path.is_symlink(): + try: + lnk = path.resolve() + if 'devkit-game' in str(lnk): + path.unlink() + print(f'Deleted: {path} -> {lnk}') + except Exception as e: + print(f'Error processing {path}: {e}') + logger.info('Devkit Steam client override is disabled - default OS client execution will resume.') + sys.exit(0) + + os.makedirs(os.path.dirname(DEVKIT_STEAM_TRAMPOLINE), exist_ok=True) + + # OS client + steam_client = '$HOME/.local/share/Steam/steam.sh' + if target == SteamStatus.SIDELOADED: + steam_client = '$HOME/devkit-game/steam/steam.sh' + + if is_deckard: + # RUNSTEAM.sh checks for SIDELOADED_STEAMROOT="${HOME}/devkit-game/steam" + # this is consistent with sideload on Steam Deck, but we use a different name 'steamdeckard' + # will be addressed when reworking the sideload and debug strategy, for now just drop in a symlink + steam_symlink = os.path.expanduser('~/devkit-game/steam') + if os.path.lexists(steam_symlink): + if os.path.islink(steam_symlink): + os.unlink(steam_symlink) + else: + # this happens if an upload in Steam Deck mode was attempted against a Steam Frame for instance + # was an easy mistake to make before recent changes + logger.warning('warning: ~/devkit-game/steam exists but is not a symlink. Removing anyway.') + shutil.rmtree(steam_symlink) + os.symlink( + os.path.expanduser('~/devkit-game/steamdeckard'), + steam_symlink, + ) + + args = '"$@"' + if conf.args is not None: + args = conf.args + + gdbserver = '' + if conf.gdbserver: + logger.info('Configuring for remote debugging via gdbserver') + gdbserver = 'export DEBUGGER="gdbserver 0.0.0.0:2345"' + + write_trampoline('''#!/bin/bash +# Generated by steamos-set-steam-client, do not edit! +# configuration tag (do not delete): {} +{} +mkdir -p $HOME/.steam/steam/logs +exec {} {} +'''.format( + conf.client, + gdbserver, + steam_client, + args +)) diff --git a/scripts/connect.sh b/scripts/connect.sh index 9ce6cd4..00bcb41 100755 --- a/scripts/connect.sh +++ b/scripts/connect.sh @@ -1,8 +1,10 @@ #!/usr/bin/env zsh -# Mac-side: find the Steam Frame, create a key, add a `Host frame` alias to -# ~/.ssh/config, copy the key, and optionally disable SSH password logins. +# Mac-side: find the Steam Frame, create keys, add a `Host frame` alias to +# ~/.ssh/config, get a key onto the headset, and optionally disable SSH password +# logins. It first pairs through Valve's SteamOS devkit service (port 32000: +# approve on the headset, no password), else copies the key with the password. # -# Verified on a Frame 2026-09-25 (except --harden). Idempotent: safe to re-run. +# Verified on a Frame 2026-09-25 (except --harden and devkit pairing). Idempotent. # # Usage: # scripts/connect.sh [HOST_OR_IP] # set up key + alias @@ -11,93 +13,232 @@ # Env: FRAME_USER (default steamos), FRAME_ALIAS (default frame). set -euo pipefail +user_from_env=${+FRAME_USER} FRAME_USER=${FRAME_USER:-steamos} FRAME_ALIAS=${FRAME_ALIAS:-frame} KEY="$HOME/.ssh/id_ed25519_frame" +# The devkit service only accepts ssh-rsa keys, so pairing uses a second key. +DEVKIT_KEY="$HOME/.ssh/id_rsa_frame_devkit" CONFIG="$HOME/.ssh/config" BEGIN_MARK="# >>> steam-frame ($FRAME_ALIAS) >>>" END_MARK="# <<< steam-frame ($FRAME_ALIAS) <<<" +DEVKIT_PORT=32000 +DEVKIT_SERVICE=_steamos-devkit._tcp +MAGIC_PHRASE=900b919520e4cf601998a71eec318fec # fixed token Valve's client appends +NAME_RE='^[A-Za-z0-9][A-Za-z0-9._-]*$' +HOST_RE='^[A-Za-z0-9][A-Za-z0-9.:%-]*$' harden=0 host_arg="" for arg in "$@"; do case "$arg" in --harden) harden=1 ;; - -h|--help) sed -n '2,11p' "$0"; exit 0 ;; + -h|--help) sed -n '2,13p' "$0"; exit 0 ;; *) host_arg="$arg" ;; esac done port_open() { # nc resolves through the system resolver (including mDNS for .local). - nc -z -G 3 "$1" 22 >/dev/null 2>&1 + nc -z -G 3 "$1" "$2" >/dev/null 2>&1 +} + +# sshd, or the devkit service, which turns sshd on once a pairing is approved. +reachable() { + port_open "$1" 22 || port_open "$1" $DEVKIT_PORT +} + +# What a command printed within $1 seconds; dns-sd never exits by itself. +run_for() { + local secs=$1; shift + "$@" 2>/dev/null & + local pid=$! + sleep "$secs" + kill $pid 2>/dev/null || true + wait $pid 2>/dev/null || true +} + +# Hosts advertising the devkit service over mDNS (dns-sd -B, then -L each). +discover_devkit() { + local name target + run_for 3 dns-sd -B $DEVKIT_SERVICE local. \ + | sed -n "s/.* Add .*${DEVKIT_SERVICE//./\\.}\\.[[:space:]]*//p" | awk '!seen[$0]++' | head -n 4 \ + | while IFS= read -r name; do + target=$(run_for 2 dns-sd -L "$name" $DEVKIT_SERVICE local. \ + | sed -n 's/.* can be reached at \([^ :]*\):[0-9].*/\1/p' | head -n 1) + [[ -n "$target" ]] && print -r -- "${target%.}" + done | awk '!seen[$0]++' } pick_host() { local candidates=() [[ -n "$host_arg" ]] && candidates+=("$host_arg") - candidates+=("$FRAME_ALIAS.local" "$FRAME_ALIAS") + [[ -z "$host_arg" ]] && candidates+=("$FRAME_ALIAS.local" "$FRAME_ALIAS") local h for h in "${candidates[@]}"; do - if port_open "$h"; then + if reachable "$h"; then print -r -- "$h"; return 0 fi - print -u2 " - $h: not resolvable or port 22 closed" + print -u2 " - $h: not resolvable, or ports 22 and $DEVKIT_PORT closed" + done + [[ -n "$host_arg" ]] && return 1 + print -u2 " - asking mDNS for $DEVKIT_SERVICE" + for h in ${(f)"$(discover_devkit)"}; do + if [[ "$h" =~ $HOST_RE ]] && reachable "$h"; then + print -r -- "$h"; return 0 + fi + print -u2 " - $h: advertised, but not reachable" done return 1 } +make_key() { # path type comment [extra ssh-keygen args] + if [[ ! -f "$1" ]]; then + ssh-keygen -q -t "$2" "${@:4}" -N '' -C "$3" -f "$1" + print " created $1" + else + print " exists: $1" + fi +} + +# Checks each step itself: pair_with_devkit calls this from an `elif`, where set -e is off. +write_config() { + touch "$CONFIG" && chmod 600 "$CONFIG" || return 1 + local tmp + tmp=$(mktemp) || return 1 + # Drop any previous managed block, then PREPEND a fresh one: ssh uses the first + # value it sees per option, so this block must precede any other "Host frame" + # or "Host *". The trailing "Host *" returns the rest of the file to global scope. + awk -v b="$BEGIN_MARK" -v e="$END_MARK" ' + $0==b {skip=1; next} + $0==e {skip=0; next} + !skip {print} + ' "$CONFIG" > "$tmp" || { rm -f "$tmp"; return 1; } + { + print -r -- "$BEGIN_MARK" + print -r -- "Host $FRAME_ALIAS" + print -r -- " HostName $HOST" + print -r -- " User $FRAME_USER" + print -r -- " IdentityFile $KEY" + print -r -- " IdentityFile $DEVKIT_KEY" + print -r -- " IdentitiesOnly yes" + print -r -- " ServerAliveInterval 30" + print -r -- "Host *" + print -r -- "$END_MARK" + cat "$tmp" + } > "$CONFIG" || { print -u2 "!! Writing $CONFIG failed; its previous contents are in $tmp"; return 1; } + rm -f "$tmp" +} + +# accept-new: after pairing, this is the first contact, so trust a first-seen host +# key (as ssh-copy-id's prompt would); a changed one still fails. +key_login_works() { + ssh -o BatchMode=yes -o ConnectTimeout=5 -o StrictHostKeyChecking=accept-new "$FRAME_ALIAS" true 2>/dev/null +} + +# The User in our managed block, so a re-run keeps one the headset named earlier. +configured_user() { + [[ -f "$CONFIG" ]] || return 0 + awk -v b="$BEGIN_MARK" -v e="$END_MARK" ' + $0==b {inside=1; next} + $0==e {exit} + inside && $1=="User" {print $2; exit} + ' "$CONFIG" +} + +devkit_url() { + if [[ "$HOST" == *:* ]]; then print -r -- "http://[$HOST]:$DEVKIT_PORT$1" + else print -r -- "http://$HOST:$DEVKIT_PORT$1"; fi +} + +# Valve's steamos-devkit-service: GET /properties.json names the login user; POST +# /register with "ssh-rsa " shows an approve prompt in the +# headset (the comment is what it displays, 30 s to answer), then installs the key +# and turns sshd on. Returns non-zero with the reason in $devkit_why to fall back. +devkit_why="" +pair_with_devkit() { + local props login comment body resp code text err + print "==> Pairing through the headset's SteamOS devkit service (no password)" + if [[ ! -r "$DEVKIT_KEY.pub" ]]; then + devkit_why="can't read the pairing key $DEVKIT_KEY.pub"; return 1 + fi + if ! props=$(curl -fsS --noproxy '*' -m 5 "$(devkit_url /properties.json)" 2>&1); then + devkit_why="devkit service not reachable on port $DEVKIT_PORT: ${${props##*curl: }%%$'\n'*}"; return 1 + fi + # properties.json is Valve's json.dumps(indent=2): "login" sits on its own line. + login=$(print -r -- "$props" | sed -n 's/.*"login"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' | head -n 1) + [[ "$login" =~ $NAME_RE && "$login" != root ]] || login="" + # Before the prompt, so the password fallback uses this user too. + if [[ -n "$login" && "$login" != "$FRAME_USER" ]]; then + if (( user_from_env )); then + print " the headset logs in as '$login'; keeping FRAME_USER=$FRAME_USER" + else + FRAME_USER=$login + print " the headset logs in as '$FRAME_USER'" + write_config || { print -u2 "Could not rewrite $CONFIG."; exit 1; } + fi + fi + # One word: the headset splits the body on spaces and shows the third field. + comment="frame-control@$(hostname -s | tr -cs 'A-Za-z0-9._-' '-' | sed 's/^[-.]*//; s/[-.]*$//')" + [[ "$comment" == "frame-control@" ]] && comment="frame-control@computer" + body="ssh-rsa $(awk '{print $2}' "$DEVKIT_KEY.pub") $comment $MAGIC_PHRASE" + print " In the headset: Steam Settings > Developer > Pair new host, then approve the request" + # The headset refuses at once unless Steam is on its "Pair new host" screen + # (verified on a Frame, 2026-09-26), so keep asking for 2 minutes while it's opened. + local deadline=$(( SECONDS + 120 )) + while true; do + if ! resp=$(print -r -- "$body" | curl -sS --noproxy '*' -m 60 -H 'Content-Type: text/plain' \ + --data-binary @- -w '\n%{http_code}' "$(devkit_url /register)" 2>&1); then + devkit_why="devkit pairing failed: no answer (${${resp##*curl: }%%$'\n'*})"; return 1 + fi + code=${resp##*$'\n'} + text=${resp%$'\n'*} + [[ "$code" == 2* ]] && break + err=$(print -r -- "$text" | sed -n 's/.*"error"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' | head -n 1) + devkit_why="devkit pairing failed: ${err:-${text:-HTTP $code}}" + [[ "$devkit_why" == *"pairing mode"* ]] && (( SECONDS < deadline )) || return 1 + sleep 3 + done + # The approval is what turns sshd on, so it may take a moment to answer. + local i + for i in {1..10}; do + key_login_works && return 0 + sleep 1 + done + devkit_why="paired, but key login still fails"; return 1 +} + print "==> Looking for the Steam Frame" if ! HOST=$(pick_host); then - print -u2 "Could not reach the Frame on port 22." + print -u2 "Could not reach the Frame on port 22 or $DEVKIT_PORT." print -u2 "Check: Developer Mode on + user password set; same Wi-Fi; no client isolation." print -u2 "Then re-run with the IP from Quick Settings: scripts/connect.sh 192.168.x.y" exit 1 fi print " found: $HOST" -print "==> SSH key" +print "==> SSH keys" mkdir -p "$HOME/.ssh" && chmod 700 "$HOME/.ssh" -if [[ ! -f "$KEY" ]]; then - ssh-keygen -q -t ed25519 -N '' -C "mac->steam-frame" -f "$KEY" - print " created $KEY" -else - print " exists: $KEY" -fi +make_key "$KEY" ed25519 "mac->steam-frame" +make_key "$DEVKIT_KEY" rsa "frame-control@$(hostname -s | tr -cs 'A-Za-z0-9._-' '-' | sed 's/^[-.]*//; s/[-.]*$//')" -b 3072 +if (( ! user_from_env )); then + prev_user=$(configured_user) + if [[ "$prev_user" =~ $NAME_RE ]]; then FRAME_USER=$prev_user; fi +fi print "==> ~/.ssh/config alias '$FRAME_ALIAS' -> $HOST" -touch "$CONFIG" && chmod 600 "$CONFIG" -tmp=$(mktemp) -# Drop any previous managed block, then PREPEND a fresh one: ssh uses the first -# value it sees per option, so this block must precede any other "Host frame" -# or "Host *". The trailing "Host *" returns the rest of the file to global scope. -awk -v b="$BEGIN_MARK" -v e="$END_MARK" ' - $0==b {skip=1; next} - $0==e {skip=0; next} - !skip {print} -' "$CONFIG" > "$tmp" -{ - print -r -- "$BEGIN_MARK" - print -r -- "Host $FRAME_ALIAS" - print -r -- " HostName $HOST" - print -r -- " User $FRAME_USER" - print -r -- " IdentityFile $KEY" - print -r -- " IdentitiesOnly yes" - print -r -- " ServerAliveInterval 30" - print -r -- "Host *" - print -r -- "$END_MARK" - cat "$tmp" -} > "$CONFIG" -rm -f "$tmp" +write_config print "==> Checking key login" -if ssh -o BatchMode=yes -o ConnectTimeout=5 "$FRAME_ALIAS" true 2>/dev/null; then +if key_login_works; then print " key login already works" +elif pair_with_devkit; then + print " paired; key login OK" else + print " $devkit_why; falling back to the password" print " copying key (enter the Developer Mode password once)" ssh-copy-id -i "$KEY.pub" -o IdentitiesOnly=yes "$FRAME_USER@$HOST" - ssh -o BatchMode=yes -o ConnectTimeout=5 "$FRAME_ALIAS" true \ - || { print -u2 "Key login still failing after ssh-copy-id."; exit 1; } + key_login_works || { print -u2 "Key login still failing after ssh-copy-id."; exit 1; } print " key login OK" fi diff --git a/tests/test_connect.py b/tests/test_connect.py new file mode 100644 index 0000000..b3366bd --- /dev/null +++ b/tests/test_connect.py @@ -0,0 +1,218 @@ +"""Setup-script checks that need no headset: devkit pairing against a stub of Valve's +steamos-devkit-service, the ~/.ssh/config block, and the mDNS output parsers. + +Run: python3 -m unittest discover -s tests +""" +import json +import socket +import sys +import threading +import unittest +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT / "ui")) + +import frame_connect as fc # noqa: E402 + +PUB = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+/x= frame-control@old\n" + + +class StubDevkit(BaseHTTPRequestHandler): + """Answers like steamos-devkit-service; `reply` picks the /register outcome.""" + reply = (200, b"Registered\n") + replies = [] # if set, each /register takes the next one instead of `reply` + properties = {"txtvers": 1, "login": "steamos", "settings": "{}", "devkit1": ["devkit-1"]} + bodies = [] + + def log_message(self, *args): + pass + + def do_GET(self): + if self.path == "/properties.json": + self.send_response(200) + self.send_header("Content-type", "application/json") + self.end_headers() + self.wfile.write(json.dumps(self.properties).encode()) + else: + self.send_response(404) + self.end_headers() + + def do_POST(self): + body = self.rfile.read(int(self.headers["Content-Length"])) + StubDevkit.bodies.append((self.path, self.headers["Content-Type"], body)) + code, text = StubDevkit.replies.pop(0) if StubDevkit.replies else self.reply + self.send_response(code) + self.send_header("Content-type", "text/plain") + self.end_headers() + self.wfile.write(text) + + +class DevkitPairing(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.server = ThreadingHTTPServer(("127.0.0.1", 0), StubDevkit) + cls.port = cls.server.server_address[1] + threading.Thread(target=cls.server.serve_forever, daemon=True).start() + + @classmethod + def tearDownClass(cls): + cls.server.shutdown() + cls.server.server_close() + + def setUp(self): + StubDevkit.reply = (200, b"Registered\n") + StubDevkit.bodies = [] + StubDevkit.replies = [] + self.said = [] + self._say, fc.say = fc.say, self.said.append + + def tearDown(self): + fc.say = self._say + + def test_register_body(self): + body = fc.register_body(PUB, "frame-control@mac") + self.assertEqual(body, "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+/x= frame-control@mac " + "900b919520e4cf601998a71eec318fec\n") + # approve-ssh-key shows split(' ')[2] as the key name. + self.assertEqual(body.split(" ")[2], "frame-control@mac") + with self.assertRaises(ValueError): + fc.register_body("ssh-ed25519 AAAAC3Nz x", "c") + + def test_key_comment_is_one_word(self): + self.assertEqual(fc.key_comment("Alex's MacBook Pro.local"), "frame-control@Alex-s-MacBook-Pro") + self.assertEqual(fc.key_comment(""), "frame-control@computer") + self.assertNotIn(" ", fc.key_comment(" a b\nc ")) + + def test_parse_login(self): + self.assertEqual(fc.parse_login(b'{"login": "steamos", "txtvers": 1}'), "steamos") + for raw in (b'{"txtvers": 1}', b'{"login": "root"}', b'{"login": "x\\nHost *"}', b'{"login": 5}'): + self.assertIsNone(fc.parse_login(raw), raw) + for raw in (b"not json", b"[1]"): + with self.assertRaises(ValueError): + fc.parse_login(raw) + + def test_devkit_error(self): + self.assertEqual(fc.devkit_error(403, b'{"error": "Steam is not running"}\n'), "Steam is not running") + self.assertEqual(fc.devkit_error(500, b"install-ssh-key:\nboom"), "install-ssh-key:\nboom") + self.assertEqual(fc.devkit_error(403, b""), "HTTP 403") + + def test_pair_ok(self): + logins = [] + reason = fc.devkit_pair("127.0.0.1", PUB, "frame-control@test", self.port, logins.append) + self.assertIsNone(reason) + self.assertEqual(logins, ["steamos"]) + path, ctype, body = StubDevkit.bodies[0] + self.assertEqual((path, ctype), ("/register", "text/plain")) + self.assertEqual(body.decode(), fc.register_body(PUB, "frame-control@test")) + self.assertTrue(any("Pair new host" in s for s in self.said)) + + NOT_PAIRING = (403, b'{"error": "devkit approve-ssh-key: please put the Steam client in pairing mode: ' + b'Settings -> Developer -> Pair new host"}') + + def test_pair_waits_for_pairing_mode(self): + # The headset refuses until Steam is on "Pair new host", then prompts. + StubDevkit.replies = [self.NOT_PAIRING, self.NOT_PAIRING, (200, b"Registered\n")] + sleep, fc.time.sleep = fc.time.sleep, lambda s: None + try: + reason = fc.devkit_pair("127.0.0.1", PUB, "c", self.port) + finally: + fc.time.sleep = sleep + self.assertIsNone(reason) + self.assertEqual(len(StubDevkit.bodies), 3) + + def test_pair_gives_up_without_pairing_mode(self): + StubDevkit.reply = self.NOT_PAIRING + wait, fc.PAIRING_MODE_WAIT = fc.PAIRING_MODE_WAIT, 0 + try: + reason = fc.devkit_pair("127.0.0.1", PUB, "c", self.port) + finally: + fc.PAIRING_MODE_WAIT = wait + self.assertIn("pairing mode", reason) + self.assertEqual(len(StubDevkit.bodies), 1) + + def test_pair_refused_falls_back(self): + StubDevkit.reply = (403, b'{"error": "timeout - Steam did not respond to the pairing request"}') + logins = [] + reason = fc.devkit_pair("127.0.0.1", PUB, "c", self.port, logins.append) + self.assertIn("timeout - Steam did not respond", reason) + # The login is still reported, so the password fallback uses the right user. + self.assertEqual(logins, ["steamos"]) + + def test_pair_without_service_falls_back(self): + with socket.socket() as s: + s.bind(("127.0.0.1", 0)) + closed = s.getsockname()[1] + logins = [] + reason = fc.devkit_pair("127.0.0.1", PUB, "c", closed, logins.append) + self.assertIn("not reachable", reason) + self.assertEqual((logins, StubDevkit.bodies), ([], [])) + + def test_pair_times_out(self): + # Accepts the connection but never answers, like a prompt nobody taps. + with socket.socket() as s: + s.bind(("127.0.0.1", 0)) + s.listen() + ok, msg = fc.register("127.0.0.1", "x", s.getsockname()[1], timeout=0.5) + self.assertFalse(ok) + self.assertIn("no answer", msg) + + +class ConfigBlock(unittest.TestCase): + def test_both_keys(self): + block = fc.config_block("frame.local", 22, "steamos") + self.assertEqual(block[0], fc.BEGIN) + self.assertEqual(block[-1], fc.END) + self.assertIn(" User steamos", block) + self.assertNotIn(" Port 22", block) + files = [line for line in block if line.startswith(" IdentityFile")] + self.assertEqual(files, [" IdentityFile ~/.ssh/id_ed25519_frame", " IdentityFile ~/.ssh/id_rsa_frame_devkit"]) + self.assertIn(" IdentitiesOnly yes", block) + self.assertEqual(block[-2], "Host *") + self.assertIn(" Port 2222", fc.config_block("10.0.0.5", 2222)) + + def test_write_config_replaces_block(self): + import tempfile + with tempfile.TemporaryDirectory() as d: + saved = fc.SSH_DIR, fc.CONFIG + fc.SSH_DIR, fc.CONFIG = Path(d), Path(d) / "config" + try: + fc.CONFIG.write_text("Host other\n User me\n", encoding="utf-8") + fc.write_config("frame.local") + self.assertEqual(fc.configured_user(), "steamos") + fc.write_config("10.0.0.5", 22, "deck") + text = fc.CONFIG.read_text(encoding="utf-8") + self.assertEqual(fc.configured_user(), "deck") # not "me" from Host other + finally: + fc.SSH_DIR, fc.CONFIG = saved + self.assertEqual(text.count(fc.BEGIN), 1) + self.assertIn("HostName 10.0.0.5", text) + self.assertIn("User deck", text) + self.assertNotIn("frame.local", text) + self.assertTrue(text.endswith("Host other\n User me\n")) + + +class MdnsParsers(unittest.TestCase): + def test_dns_sd(self): + browse = ("Browsing for _steamos-devkit._tcp\n" + "Timestamp A/R Flags if Domain Service Type Instance Name\n" + "19:34:35.419 Add 3 15 local. _steamos-devkit._tcp. frame\n" + "19:34:35.611 Add 2 1 local. _steamos-devkit._tcp. frame\n" + "19:34:35.700 Add 2 15 local. _steamos-devkit._tcp. My Frame\n" + "19:34:36.000 Rmv 0 15 local. _steamos-devkit._tcp. gone\n") + self.assertEqual(fc.parse_dns_sd_browse(browse), ["frame", "My Frame"]) + resolve = ("Lookup frame._steamos-devkit._tcp.local.\n" + "19:34:44.601 frame._steamos-devkit._tcp.local. can be reached at frame.local.:32000 (interface 15)\n") + self.assertEqual(fc.parse_dns_sd_resolve(resolve), "frame.local") + self.assertIsNone(fc.parse_dns_sd_resolve("Lookup frame\n")) + + def test_avahi(self): + out = ('+;wlan0;IPv4;frame;_steamos-devkit._tcp;local\n' + '=;wlan0;IPv6;frame;_steamos-devkit._tcp;local;frame.local;fe80::1;32000;"login=steamos"\n' + '=;wlan0;IPv4;frame;_steamos-devkit._tcp;local;frame.local;192.168.1.50;32000;"login=steamos"\n') + self.assertEqual(fc.parse_avahi(out), ["frame.local", "192.168.1.50"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_frame_apk.py b/tests/test_frame_apk.py index f9c008b..40119d4 100644 --- a/tests/test_frame_apk.py +++ b/tests/test_frame_apk.py @@ -4,6 +4,7 @@ import os import struct import sys import tempfile +import tracemalloc import unittest import zipfile @@ -134,6 +135,64 @@ class ApkInfo(unittest.TestCase): with self.assertRaises(frame_apk.ApkError): self.read(data) + def test_refuses_oversized_members(self): + # An APK from a website mustn't make the server inflate gigabytes. + data = apk({'AndroidManifest.xml': manifest('com.example.big', 0x7f010000, 0x7f010001, 21)}) + limit, frame_apk.MAX_MANIFEST = frame_apk.MAX_MANIFEST, 16 + try: + with self.assertRaises(frame_apk.ApkError): + self.read(data) + finally: + frame_apk.MAX_MANIFEST = limit + + def test_forged_sizes_dont_inflate_everything(self): + # The central directory claims 1 byte; the deflated data holds 16 MB of zeros. + buf = io.BytesIO() + with zipfile.ZipFile(buf, 'w', zipfile.ZIP_DEFLATED) as z: + z.writestr('AndroidManifest.xml', bytes(16 * 1024**2)) + data = bytearray(buf.getvalue()) + for sig, field in ((b'PK\x01\x02', 24), (b'PK\x03\x04', 22)): + at = data.index(sig) + data[at + field:at + field + 4] = struct.pack(' . ; alias/alias/escape -> ../.. ; escape/victim would land outside if links were real. + p = self.link_zip([('alias', '.', True), ('alias/alias/escape', '../..', True), ('escape/victim', b'x', False)]) + out = tempfile.mkdtemp(dir=self.dir) + frame_titles.extract_zip(p, out) + self.assertTrue(os.path.isfile(os.path.join(out, 'escape', 'victim'))) # stayed inside + self.assertFalse(os.path.exists(os.path.join(self.dir, 'victim'))) + self.assertFalse(os.path.exists(os.path.join(os.path.dirname(self.dir), 'victim'))) + for root, dirs, files in os.walk(out): + self.assertFalse([n for n in dirs + files if os.path.islink(os.path.join(root, n))]) + + def test_folder_links_are_dropped_and_order_does_not_matter(self): + # b -> a/file listed before a -> dir; and a folder link that would contain itself. + p = self.link_zip([('dir/file', b'data', False), ('b', 'a/file', True), ('a', 'dir', True), + ('dir/sub/loop', '../../a', True)]) + out = tempfile.mkdtemp(dir=self.dir) + frame_titles.extract_zip(p, out) + with open(os.path.join(out, 'b'), 'rb') as f: + self.assertEqual(f.read(), b'data') + self.assertFalse(os.path.lexists(os.path.join(out, 'a'))) + self.assertFalse(os.path.lexists(os.path.join(out, 'dir', 'sub', 'loop'))) + + def test_link_components_resolve_before_parent_steps(self): + # alias -> dirlink/../game.exe, dirlink -> deep/subdir: that's deep/game.exe, not game.exe. + p = self.link_zip([('deep/subdir/x', b'', False), ('deep/game.exe', b'deep one', False), + ('game.exe', b'top one', False), ('dirlink', 'deep/subdir', True), + ('alias', 'dirlink/../game.exe', True)]) + out = tempfile.mkdtemp(dir=self.dir) + frame_titles.extract_zip(p, out) + with open(os.path.join(out, 'alias'), 'rb') as f: + self.assertEqual(f.read(), b'deep one') + + def test_many_links_to_one_file_count_against_the_limit(self): + # The zip (1 KB) and the copies (15 KB) each fit under the limit; together they don't. + members = [('big', b'x' * 1000, False)] + [(f'alias{i}', 'big', True) for i in range(15)] + old = frame_titles.MAX_UNPACKED + frame_titles.MAX_UNPACKED = 15500 + out = tempfile.mkdtemp(dir=self.dir) + try: + with self.assertRaisesRegex(FrameError, 'links would copy'): + frame_titles.extract_zip(self.link_zip(members), out) + finally: + frame_titles.MAX_UNPACKED = old + self.assertEqual(os.listdir(out), ['big']) # refused before copying any link + + def test_oversized_link_is_refused(self): + p = self.link_zip([('big', 'x' * 5000, True)]) + with self.assertRaisesRegex(FrameError, 'oversized link'): + frame_titles.extract_zip(p, tempfile.mkdtemp(dir=self.dir)) + + @unittest.skipIf(os.name == 'nt', 'needs symlinks') + def test_unwrap_never_steps_through_a_link(self): + # A folder whose only entry links elsewhere (a junction on Windows) stays the boundary. + outside, game = os.path.join(self.dir, 'outside'), os.path.join(self.dir, 'Game') + os.makedirs(outside) + os.makedirs(game) + with open(os.path.join(outside, 'Other.exe'), 'wb') as f: + f.write(pe(0x8664)) + os.symlink(outside, os.path.join(game, 'inner')) + with self.assertRaisesRegex(FrameError, 'no Linux or Windows program'): + frame_titles.inspect(game) + + @unittest.skipIf(os.name == 'nt', 'needs symlinks') + def test_folder_with_outside_link_is_staged_without_it(self): + game, secret = os.path.join(self.dir, 'Game'), os.path.join(self.dir, 'secret') + os.makedirs(game) + os.makedirs(secret) + with open(os.path.join(secret, 'key'), 'wb') as f: + f.write(b'private') + with open(os.path.join(game, 'Game.exe'), 'wb') as f: + f.write(pe(0x8664)) + os.symlink(secret, os.path.join(game, 'leak')) + os.symlink(os.path.join(secret, 'key'), os.path.join(game, 'leak-file')) + os.symlink('Game.exe', os.path.join(game, 'Alias.exe')) + plan = frame_titles.inspect(game) + try: + self.assertNotEqual(os.path.realpath(plan['root']), os.path.realpath(game)) + self.assertEqual(sorted(os.listdir(plan['root'])), ['Alias.exe', 'Game.exe']) + self.assertFalse(os.path.islink(os.path.join(plan['root'], 'Alias.exe'))) + finally: + frame_titles.discard(plan) + + def test_links_become_copies(self): + # No symlinks on disk (Windows may not allow them); the library a link names is still there. + p = self.link_zip([('game/lib/libfoo.so.1.2', b'ELF-ish', False), ('game/lib/libfoo.so.1', 'libfoo.so.1.2', True), + ('game/lib/libfoo.so', 'libfoo.so.1', True), ('game/dangling', 'nowhere', True)]) + out = tempfile.mkdtemp(dir=self.dir) + frame_titles.extract_zip(p, out) + for name in ('libfoo.so.1', 'libfoo.so'): + path = os.path.join(out, 'game', 'lib', name) + self.assertFalse(os.path.islink(path)) + with open(path, 'rb') as f: + self.assertEqual(f.read(), b'ELF-ish') + self.assertFalse(os.path.lexists(os.path.join(out, 'game', 'dangling'))) + + def test_drive_qualified_parts_are_refused(self): + for bad in ('sub/C:../C:../victim.txt', 'game/file.exe:stream'): + with self.subTest(bad=bad): + with self.assertRaisesRegex(FrameError, 'drive or stream'): + frame_titles.extract_zip(self.zip({bad: b'x'}, 'drive.zip'), tempfile.mkdtemp(dir=self.dir)) + + def test_absurd_size_is_refused(self): + p = self.zip({'game.exe': pe(0x8664)}, 'bomb.zip') + old = frame_titles.MAX_UNPACKED + frame_titles.MAX_UNPACKED = 10 + try: + with self.assertRaisesRegex(FrameError, 'looks wrong'): + frame_titles.extract_zip(p, tempfile.mkdtemp(dir=self.dir)) + finally: + frame_titles.MAX_UNPACKED = old + + def test_not_a_zip(self): + p = os.path.join(self.dir, 'x.zip') + with open(p, 'wb') as f: + f.write(b'nope') + with self.assertRaisesRegex(FrameError, 'not a readable zip'): + frame_titles.inspect(p) + + +class Names(unittest.TestCase): + def test_title_id(self): + self.assertEqual(frame_titles.title_id('Hollow Knight: Silksong!'), 'Hollow_Knight_Silksong') + self.assertEqual(frame_titles.title_id('steam'), 'steam-game') # Valve's reserved sideload names + self.assertEqual(frame_titles.title_id('Devkit Steam'), 'Devkit_Steam') + self.assertEqual(frame_titles.title_id('devkit-steam'), 'devkit-steam-game') # the trampoline file + self.assertEqual(frame_titles.title_id('--rm -rf /'), 'rm_-rf') + self.assertEqual(len(frame_titles.title_id('x' * 200)), 64) + with self.assertRaises(FrameError): + frame_titles.title_id('!!!') + + def test_display_name(self): + self.assertEqual(frame_titles.display_name('MyGame-linux-arm64.zip'), 'MyGame') + self.assertEqual(frame_titles.display_name('Portal 2.zip'), 'Portal 2') + self.assertEqual(frame_titles.display_name('Game_v1.0.3_Win64.zip'), 'Game') + + +class Parms(unittest.TestCase): + def test_proton_parms(self): + p = frame_titles.shortcut_parms('Cool_Game', '/home/steamos/devkit-game/Cool_Game', + 'Cool Game.exe', 'proton-experimental') + self.assertEqual(p, {'gameid': 'Cool_Game', 'directory': '/home/steamos/devkit-game/Cool_Game', + 'argv': ['"Cool Game.exe"'], 'env': {}, + 'settings': {'steam_play': '1', 'steam_play_debug': '0', + 'steam_play_debug_version': '2019', + 'compat_tool': 'proton-experimental'}, + 'clear_settings': True, 'force_appid': '', 'lepton_args': ''}) + json.dumps(p) + + def test_linux_parms(self): + p = frame_titles.shortcut_parms('g', '/home/steamos/devkit-game/g', 'bin/game', 'SteamLinuxRuntime_4-arm64') + self.assertEqual(p['argv'], ['bin/game']) + self.assertEqual(p['settings'], {'steam_play': '0', 'compat_tool': 'SteamLinuxRuntime_4-arm64'}) + + def test_cleanup_names_only_this_title(self): + # A glob like Game-*.json would also delete Game-Deluxe's files. + self.assertEqual(frame_titles._json_files('Game').split(), + ['devkit-game/Game-argv.json', 'devkit-game/Game-env.json', + 'devkit-game/Game-settings.json', 'devkit-game/Game-framecontrol.json']) + + def test_launch_needs_steam_to_answer(self): + # steam-devkit-rpc exits 0 after a timeout; only its 'success' line means Steam took it. + calls = [] + old = frame_titles.ssh, frame_titles._check_id, frame_titles.ensure_utils + frame_titles._check_id, frame_titles.ensure_utils = (lambda g: g), (lambda: False) + try: + frame_titles.ssh = lambda cmd, **kw: calls.append(cmd) or 'Found steam client pid 1\ntimeout\n' + with self.assertRaisesRegex(FrameError, "didn't confirm"): + frame_titles.launch('Game') + frame_titles.ssh = lambda cmd, **kw: 'Found steam client pid 1\nsuccess\n{}' + self.assertEqual(frame_titles.launch('Game'), {'id': 'Game'}) + finally: + frame_titles.ssh, frame_titles._check_id, frame_titles.ensure_utils = old + self.assertIn('steam-devkit-rpc run-game gameid=Game', calls[0]) + + def test_remove_waits_for_installs(self): + with frame_titles._install_lock: + with self.assertRaisesRegex(FrameError, 'install is running'): + frame_titles.remove('Game') + + def test_vendored_utils_are_present(self): + for name in ('steamos-prepare-upload', 'steam-client-create-shortcut', 'steam-devkit-rpc', + 'steamos-delete', 'devkit_utils/__init__.py', 'LICENSE'): + self.assertTrue(os.path.isfile(os.path.join(frame_titles.UTILS_LOCAL, name)), name) + self.assertEqual(len(frame_titles.utils_stamp()), 20) + + +if __name__ == '__main__': + unittest.main() diff --git a/tests/test_server.py b/tests/test_server.py index 5511184..a1a7898 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -6,14 +6,17 @@ request guards and input validation, which all run before any SSH call. Run: python3 -m unittest discover -s tests """ import http.client +import io import json import os import socket +import struct import subprocess import sys import tempfile import time import unittest +import zipfile from pathlib import Path from urllib.parse import quote @@ -54,7 +57,7 @@ class ServerGuards(unittest.TestCase): @classmethod def request(cls, method, path, body=None, headers=None): conn = http.client.HTTPConnection("127.0.0.1", cls.port, timeout=10) - data = json.dumps(body).encode() if body is not None else None + data = body if isinstance(body, bytes) else json.dumps(body).encode() if body is not None else None conn.request(method, path, body=data, headers=headers or {}) r = conn.getresponse() payload = r.read() @@ -130,6 +133,58 @@ class ServerGuards(unittest.TestCase): status, _ = self.post("/api/launch", ["not", "an", "object"]) self.assertEqual(status, 400) + def test_title_upload_is_inspected_then_discarded(self): + # A zip holding a Windows x86-64 program: inspected locally, no SSH until install. + buf = io.BytesIO() + with zipfile.ZipFile(buf, "w") as z: + z.writestr("Tiny Game/Tiny Game.exe", + b"MZ" + b"\0" * 0x3A + struct.pack(" {payload}") + self.assertEqual(self.request("GET", "/api/titles/job?token=nope", headers={"X-Frame-UI": "1"})[0], 404) + self.assertEqual(self.request("POST", "/api/titles", {"action": "list"})[0], 403) + + def test_web_install_needs_the_app_page(self): + # A website can only open frame-control:// links; it can't call these itself. + link = {"url": "https://cdn.example.com/game.apk"} + self.assertEqual(self.request("POST", "/api/webinstall/check", link)[0], 403) + self.assertEqual(self.request("POST", "/api/webinstall/start", {"id": "x"})[0], 403) + status, _, _ = self.request("POST", "/api/webinstall/check", link, + {"X-Frame-UI": "1", "Host": f"evil.example:{self.port}"}) + self.assertEqual(status, 403) + + def test_web_install_validation(self): + for body in ({}, {"url": 5}, {"url": "http://cdn.example.com/game.apk"}, {"url": "https://10.0.0.2/game.apk"}, + {"url": "https://u:p@example.com/game.apk"}, {"url": "https://example.com/"}, + {"url": "https://1.1.1.1/game.sh"}, {"manifest": "file:///etc/passwd"}, + {"manifest": "https://example.com/m.json", "url": "https://example.com/g.apk"}): + status, payload = self.post("/api/webinstall/check", body) + self.assertEqual(status, 400, f"{body} -> {payload}") + # Only an id from /check starts an install, and only once. + self.assertEqual(self.post("/api/webinstall/start", {"id": "made-up"})[0], 400) + self.assertEqual(self.request("GET", "/api/webinstall/job?id=x", headers={"X-Frame-UI": "1"})[0], 404) + self.assertEqual(self.post("/api/webinstall/cancel", {"job": "x"})[0], 404) + def test_unknown_routes(self): self.assertEqual(self.request("GET", "/nope")[0], 404) self.assertEqual(self.post("/api/nope", {})[0], 404) diff --git a/tests/test_webinstall.py b/tests/test_webinstall.py new file mode 100644 index 0000000..1858297 --- /dev/null +++ b/tests/test_webinstall.py @@ -0,0 +1,438 @@ +"""Install links from websites (ui/frame_webinstall.py, app/install-link.js). No network: +name lookups are stubbed and downloads come from a server on 127.0.0.1, which +the localhost-testing rule allows. + +Run: python3 -m unittest discover -s tests +""" +import hashlib +import json +import os +import shutil +import socket +import subprocess +import sys +import tempfile +import threading +import time +import unittest +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path +from unittest import mock + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT / "ui")) + +import frame_webinstall as wi # noqa: E402 + +E = wi.WebInstallError +PAYLOAD = b"not really an apk, but bytes are bytes\n" * 1000 + + +def fake_dns(*ips): + return lambda host, port, **_: [(socket.AF_INET, socket.SOCK_STREAM, 6, "", (ip, port)) for ip in ips] + + +class Urls(unittest.TestCase): + def test_https_ok(self): + self.assertEqual(wi.check_url("https://cdn.example.com/g/mygame.apk"), ("https", "cdn.example.com", 443, False)) + self.assertEqual(wi.file_name("https://cdn.example.com/g/my%20game.apk?sig=1"), "my game.apk") + + def test_http_only_for_localhost(self): + with self.assertRaises(E): + wi.check_url("http://cdn.example.com/mygame.apk") + self.assertTrue(wi.check_url("http://localhost:8000/mygame.apk", allow_local=True)[3]) + self.assertTrue(wi.check_url("http://127.0.0.1:8000/mygame.apk", allow_local=True)[3]) + + def test_localhost_only_when_the_link_starts_there(self): + for url in ("http://localhost/x.apk", "https://127.0.0.1/x.apk"): + with self.assertRaises(E): + wi.check_url(url, allow_local=False) + + def test_other_schemes_rejected(self): + for url in ("file:///etc/passwd", "ftp://example.com/x.apk", "javascript:alert(1)", "//example.com/x.apk", ""): + with self.assertRaises(E, msg=url): + wi.check_url(url) + + def test_private_and_local_addresses_rejected(self): + for host in ("10.0.0.5", "192.168.1.20", "172.16.3.4", "127.0.0.2", "169.254.169.254", "100.64.1.1", + "0.0.0.0", "[::1]", "[fe80::1]", "[fd00::1]", "[fec0::1]", "[::ffff:192.168.1.1]", + "[2002:c0a8:101::1]", "224.0.0.1"): + with self.assertRaises(E, msg=host): + wi.check_url(f"https://{host}/x.apk") + wi.check_url("https://93.184.216.34/x.apk") + + def test_names_resolving_to_private_addresses_rejected(self): + with mock.patch.object(wi, "_getaddrinfo", fake_dns("192.168.1.9")): + with self.assertRaises(E): + wi._resolve("sneaky.example.com", 443, False) + # Every address counts, not just the first. + with mock.patch.object(wi, "_getaddrinfo", fake_dns("93.184.216.34", "10.1.2.3")): + with self.assertRaises(E): + wi._resolve("mixed.example.com", 443, False) + with mock.patch.object(wi, "_getaddrinfo", fake_dns("93.184.216.34")): + self.assertEqual(wi._resolve("cdn.example.com", 443, False), "93.184.216.34") + + def test_credentials_rejected(self): + for url in ("https://user:pw@example.com/x.apk", "https://user@example.com/x.apk", "https://:pw@example.com/x.apk"): + with self.assertRaises(E, msg=url): + wi.check_url(url) + + def test_directory_urls_rejected(self): + for url in ("https://example.com/", "https://example.com", "https://example.com/games/", + "https://example.com/%2e%2e", "https://example.com/.hidden.apk", "https://example.com/a%2Fb.apk"): + with self.assertRaises(E, msg=url): + wi.file_name(url) + + def test_file_types(self): + self.assertEqual(wi.file_kind("Game.APK"), "apk") + self.assertEqual(wi.file_kind("game.zip"), "title") + self.assertEqual(wi.file_kind("setup.exe"), "title") + for name in ("game.sh", "game.tar.gz", "game"): + with self.assertRaises(E, msg=name): + wi.file_kind(name) + + +class Manifests(unittest.TestCase): + FILE = {"url": "https://cdn.example.com/mygame-arm64.apk"} + + def test_both_schemas(self): + for schema in ("framedrop.install/v1", "frame-control.install/v1"): + m = wi.parse_manifest({"schema": schema, "name": "My Game", "files": [dict(self.FILE, sha256="AB" * 32)]}) + self.assertEqual(m["name"], "My Game") + self.assertEqual(m["file"]["url"], self.FILE["url"]) + self.assertEqual(m["file"]["sha256"], "ab" * 32) + + def test_bad_schema(self): + for schema in (None, "framedrop.install/v2", "something"): + with self.assertRaises(E, msg=schema): + wi.parse_manifest({"schema": schema, "files": [self.FILE]}) + + def test_missing_or_bad_fields(self): + base = {"schema": "framedrop.install/v1"} + for obj in ([], base, dict(base, files=[]), dict(base, files="x"), dict(base, files=[{}]), + dict(base, files=[{"url": ""}]), dict(base, files=[dict(self.FILE, sha256="abc")]), + dict(base, files=[dict(self.FILE, size=-1)]), dict(base, name=5, files=[self.FILE])): + with self.assertRaises(E, msg=obj): + wi.parse_manifest(obj) + + def test_name_optional_and_cleaned(self): + self.assertIsNone(wi.parse_manifest({"schema": "framedrop.install/v1", "files": [self.FILE]})["name"]) + m = wi.parse_manifest({"schema": "framedrop.install/v1", "name": " A\x1b[31mB\n ", "files": [self.FILE]}) + self.assertEqual(m["name"], "A[31mB") + + def test_multiple_files_refused_clearly(self): + with self.assertRaisesRegex(E, "2 files"): + wi.parse_manifest({"schema": "framedrop.install/v1", "files": [self.FILE, self.FILE]}) + + +class Stub(BaseHTTPRequestHandler): + routes = {} + + def log_message(self, *_): + pass + + def do_HEAD(self): + self.do_GET(body=False) + + def do_GET(self, body=True): + route = self.routes.get(self.path) + if route is None: + self.send_response(404) + self.end_headers() + return + status, headers, data = route + self.send_response(status) + for k, v in headers.items(): + self.send_header(k, v) + if "Content-Length" not in headers: + self.send_header("Content-Length", str(len(data))) + self.end_headers() + if body: + self.wfile.write(data) + + +class Downloads(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.httpd = ThreadingHTTPServer(("127.0.0.1", 0), Stub) + cls.base = f"http://127.0.0.1:{cls.httpd.server_address[1]}" + threading.Thread(target=cls.httpd.serve_forever, daemon=True).start() + sha = hashlib.sha256(PAYLOAD).hexdigest() + Stub.routes = { + "/game.apk": (200, {}, PAYLOAD), + "/game.zip": (200, {}, PAYLOAD), + "/redirect.apk": (302, {"Location": "/game.apk"}, b""), + "/to-lan.apk": (302, {"Location": "https://192.168.1.5/game.apk"}, b""), + "/to-http.apk": (302, {"Location": "http://cdn.example.com/game.apk"}, b""), + "/loop.apk": (302, {"Location": "/loop.apk"}, b""), + "/manifest.json": (200, {}, json.dumps({"schema": "framedrop.install/v1", "name": "Stub Game", + "files": [{"url": f"{cls.base}/game.apk", "sha256": sha}]}).encode()), + "/bad-sha.json": (200, {}, json.dumps({"schema": "frame-control.install/v1", "name": "Bad", + "files": [{"url": f"{cls.base}/game.apk", "sha256": "0" * 64}]}).encode()), + "/huge.json": (200, {}, b"{" + b" " * (wi.MAX_MANIFEST + 10) + b"}"), + "/notjson.json": (200, {}, b""), + "/short.apk": (200, {"Content-Length": str(len(PAYLOAD) + 100)}, PAYLOAD), + } + + @classmethod + def tearDownClass(cls): + cls.httpd.shutdown() + cls.httpd.server_close() + + def setUp(self): + self.tmp = tempfile.mkdtemp() + env = mock.patch.dict(os.environ, {wi.LOCAL_LINKS_ENV: "1"}) + env.start() + self.addCleanup(env.stop) + + def tearDown(self): + shutil.rmtree(self.tmp, ignore_errors=True) + + def test_localhost_links_need_the_developer_switch(self): + # Without it, a website's link can't make the app fetch from local services. + with mock.patch.dict(os.environ, {wi.LOCAL_LINKS_ENV: ""}): + for kw in ({"manifest": f"{self.base}/manifest.json"}, {"url": f"{self.base}/game.apk"}): + with self.assertRaisesRegex(wi.WebInstallError, wi.LOCAL_LINKS_ENV): + wi.plan(**kw) + + def test_manifest_round_trip(self): + p = wi.plan(manifest=f"{self.base}/manifest.json") + self.assertEqual((p["name"], p["file"], p["kind"], p["host"], p["size"]), + ("Stub Game", "game.apk", "apk", "127.0.0.1", len(PAYLOAD))) + seen = [] + path = wi.download(p, self.tmp, progress=lambda done, total: seen.append((done, total))) + self.assertEqual(Path(path).read_bytes(), PAYLOAD) + self.assertEqual(seen[-1], (len(PAYLOAD), len(PAYLOAD))) + self.assertEqual(os.listdir(self.tmp), ["game.apk"]) + + def test_direct_url_and_redirect(self): + p = wi.plan(url=f"{self.base}/redirect.apk") + self.assertEqual((p["name"], p["file"]), ("redirect.apk", "redirect.apk")) + self.assertEqual(Path(wi.download(p, self.tmp)).read_bytes(), PAYLOAD) + + def test_redirects_checked_again(self): + for path in ("/to-lan.apk", "/to-http.apk", "/loop.apk"): + with self.assertRaises(E, msg=path): + wi._open(f"{self.base}{path}", allow_local=True) + + def test_sha256_mismatch_leaves_nothing(self): + p = wi.plan(manifest=f"{self.base}/bad-sha.json") + with self.assertRaisesRegex(E, "sha256"): + wi.download(p, self.tmp) + self.assertEqual(os.listdir(self.tmp), []) + + def test_size_cap(self): + with mock.patch.object(wi, "MAX_FILE", 1000): + with self.assertRaisesRegex(E, "limit"): + wi.plan(url=f"{self.base}/game.apk") + p = {"url": f"{self.base}/game.apk", "file": "game.apk", "allowLocal": True, "size": None, "sha256": None} + with self.assertRaisesRegex(E, "limit"): + wi.download(p, self.tmp) + self.assertEqual(os.listdir(self.tmp), []) + + def test_bad_manifests(self): + for path in ("/huge.json", "/notjson.json", "/missing.json"): + with self.assertRaises(E, msg=path): + wi.plan(manifest=f"{self.base}{path}") + + def test_cut_off_download(self): + p = {"url": f"{self.base}/short.apk", "file": "short.apk", "allowLocal": True, "size": None, "sha256": None} + with self.assertRaises(E): + wi.download(p, self.tmp) + self.assertEqual(os.listdir(self.tmp), []) + + def test_aborted_connection_never_connects(self): + port = self.httpd.server_address[1] + for cls in (wi._HTTPConnection, wi._HTTPSConnection): + conn = cls("127.0.0.1", "127.0.0.1", port, 5) + wi.abort(conn) # before connect, e.g. cancelled while looking up the name + with self.assertRaisesRegex(OSError, "aborted"): + conn.connect() + + def test_cancel(self): + p = wi.plan(url=f"{self.base}/game.apk") + with self.assertRaises(wi.Cancelled): + wi.download(p, self.tmp, cancelled=lambda: True) + self.assertEqual(os.listdir(self.tmp), []) + + +class Dispatch(unittest.TestCase): + def setUp(self): + self.tmp = tempfile.mkdtemp() + + def tearDown(self): + shutil.rmtree(self.tmp, ignore_errors=True) + + def file(self, name): + path = os.path.join(self.tmp, name) + Path(path).write_bytes(PAYLOAD) + return path + + def test_apk_goes_to_the_android_installer(self): + import frame_android + with mock.patch.object(frame_android, "install", return_value={"label": "Stub"}) as install: + res = wi.dispatch(self.file("game.apk"), name="Ignored", source="https://example.com/game.apk") + install.assert_called_once_with(os.path.join(self.tmp, "game.apk"), source="https://example.com/game.apk") + self.assertEqual(res["kind"], "apk") + self.assertIn("Stub", res["message"]) + + def test_titles_without_the_titles_module(self): + with mock.patch.dict(sys.modules, {"frame_titles": None}): + with self.assertRaisesRegex(E, "newer Frame Control"): + wi.dispatch(self.file("game.zip")) + + def test_titles_go_to_frame_titles(self): + fake = mock.Mock() + fake.install.return_value = {"message": "Installed Stub"} + with mock.patch.dict(sys.modules, {"frame_titles": fake}): + res = wi.dispatch(self.file("game.exe"), name="Stub", exe=None) + fake.install.assert_called_once_with(os.path.join(self.tmp, "game.exe"), name="Stub", exe=None, progress=None) + self.assertEqual(res["message"], "Installed Stub") + + def test_other_files_refused(self): + with self.assertRaises(E): + wi.dispatch(self.file("game.sh")) + + +class ServerJobs(unittest.TestCase): + """The server's install worker (ui/server.py), with download and dispatch stubbed.""" + + @classmethod + def setUpClass(cls): + with mock.patch.dict(os.environ, {"FRAME_ALIAS": "frame-control-test.invalid"}): + import server + cls.server = server + + def run_job(self, download=None, mkdtemp_error=None): + s = self.server + job = {"phase": "download", "done": 0, "total": None, "detail": "", "message": None, "error": None, "cancel": False} + plan = {"name": "Stub", "exe": None, "url": "https://example.com/stub.apk"} + with mock.patch.object(s, "ensure_master"), \ + mock.patch.object(s.frame_webinstall, "download", side_effect=lambda *a, **k: download(job, a[1])), \ + mock.patch.object(s.tempfile, "mkdtemp", side_effect=mkdtemp_error or tempfile.mkdtemp), \ + mock.patch.object(s.frame_webinstall, "dispatch", return_value={"message": "ok"}) as dispatch: + s._webinstall_run(plan, job) + return job, dispatch + + def test_cancel_after_the_last_chunk_still_stops_the_install(self): + def download(job, tmp): + job["cancel"] = True # arrives after the downloader's last check + return os.path.join(tmp, "stub.apk") + job, dispatch = self.run_job(download) + dispatch.assert_not_called() + self.assertEqual(job["phase"], "error") + + def test_finished_download_is_dispatched(self): + job, dispatch = self.run_job(lambda job, tmp: os.path.join(tmp, "stub.apk")) + dispatch.assert_called_once() + self.assertEqual((job["phase"], job["message"]), ("done", "ok")) + + def stall_then_shutdown(self, scheme, reply): + if os.name == "nt": + # shutdown() from another thread doesn't wake a blocked recv on Windows, and + # closing the handle under a TLS read isn't safe; see web-install.md. + self.skipTest("Windows: a stalled download is only dropped when the app stops the server") + """Start a download from a server that stalls after sending reply; shutdown must stop it quickly.""" + stall = socket.socket() + stall.bind(("127.0.0.1", 0)) + stall.listen(1) + port = stall.getsockname()[1] + stalled = threading.Event() + + def serve(): + c, _ = stall.accept() + if reply is not None: + c.recv(65536) + c.sendall(reply) + stalled.set() + time.sleep(20) # longer than the test may take; TIMEOUT is 30 s + c.close() + threading.Thread(target=serve, daemon=True).start() + s = self.server + pid = "shutdown-test" + s._web_plans[pid] = {"name": "Stub", "exe": None, "url": f"{scheme}://127.0.0.1:{port}/stub.apk", + "file": "stub.apk", "allowLocal": True, "size": None, "sha256": None, + "sizeFromManifest": False} + try: + s.webinstall_start({"id": pid}) + job = s._web_jobs[pid] + self.assertTrue(stalled.wait(5)) + time.sleep(0.1) # let the client block + t0 = time.time() + s.webinstall_shutdown() + self.assertLess(time.time() - t0, 3) + self.assertEqual(s._web_workers, set()) + self.assertEqual((job["phase"], job["error"]), ("error", "download cancelled")) + s._web_plans["late"] = {"size": None} + with self.assertRaises(s.Failure) as caught: # nothing new starts once quitting + s.webinstall_start({"id": "late"}) + self.assertEqual(caught.exception.status, 503) + finally: + s._web_closing = False + s._web_jobs.clear() + s._web_plans.clear() + stall.close() + + def test_shutdown_interrupts_a_stalled_body(self): + self.stall_then_shutdown("http", b"HTTP/1.0 200 OK\r\nContent-Length: 1000000\r\n\r\npartial") + + def test_shutdown_interrupts_stalled_headers(self): + self.stall_then_shutdown("http", b"HTTP/1.1 200 OK\r\n") + + def test_shutdown_interrupts_a_stalled_tls_handshake(self): + self.stall_then_shutdown("https", None) + + def test_dead_servers_leftovers_swept(self): + dead = subprocess.Popen([sys.executable, "-c", "pass"]) + dead.wait() + # Downloads and title staging (unzipped titles) are both swept. + for prefix in (self.server.WEB_TMP_PREFIX, self.server.frame_titles.TMP_PREFIX): + gone = tempfile.mkdtemp(prefix=f"{prefix}{dead.pid}-") + live = tempfile.mkdtemp(prefix=f"{prefix}{os.getpid()}-") + try: + self.server.sweep_tmp() + self.assertFalse(os.path.exists(gone), prefix) + self.assertTrue(os.path.exists(live), prefix) + finally: + shutil.rmtree(gone, ignore_errors=True) + shutil.rmtree(live, ignore_errors=True) + + def test_temp_dir_failure_ends_the_job(self): + job, dispatch = self.run_job(mkdtemp_error=OSError("disk full")) + dispatch.assert_not_called() + self.assertEqual(job["phase"], "error") + self.assertIn("disk full", job["error"]) + + +@unittest.skipUnless(shutil.which("node"), "needs node") +class LinkParsing(unittest.TestCase): + def parse(self, links): + script = ("const { parseInstallLink, linkFromArgv } = require(process.argv[1]);" + "const links = JSON.parse(process.argv[2]);" + "console.log(JSON.stringify({ parsed: links.map(parseInstallLink)," + " argv: linkFromArgv(['/x/frame-control', '--flag', links[0]]) }));") + out = subprocess.run(["node", "-e", script, str(ROOT / "app" / "install-link.js"), json.dumps(links)], + capture_output=True, text=True, timeout=30) + self.assertEqual(out.returncode, 0, out.stderr) + return json.loads(out.stdout) + + def test_links(self): + m = "https://example.com/m.json" + good = ["frame-control://install?manifest=" + "https%3A%2F%2Fexample.com%2Fm.json", + "frame-control://install/?url=https%3A%2F%2Fcdn.example.com%2Fg.apk", + "FRAME-CONTROL://install?manifest=http%3A%2F%2Flocalhost%3A8000%2Fm.json"] + bad = ["framedrop://install?manifest=" + m, "frame-control://uninstall?manifest=" + m, + "frame-control://install?manifest=" + m + "&url=" + m, "frame-control://install?manifest=a&manifest=b", + "frame-control://install?manifest=file%3A%2F%2F%2Fetc%2Fpasswd", "frame-control://install?other=" + m, + "frame-control://install?url=https%3A%2F%2Fu%3Ap%40example.com%2Fg.apk", "frame-control://install", + "frame-control://install/sub?url=" + m, "https://example.com"] + res = self.parse(good + bad) + self.assertEqual(res["parsed"][0], {"kind": "manifest", "target": m}) + self.assertEqual(res["parsed"][1], {"kind": "url", "target": "https://cdn.example.com/g.apk"}) + self.assertEqual(res["parsed"][2]["kind"], "manifest") + self.assertEqual(res["parsed"][len(good):], [None] * len(bad)) + self.assertEqual(res["argv"], good[0]) + + +if __name__ == "__main__": + unittest.main() diff --git a/ui/frame_apk.py b/ui/frame_apk.py index 53fd58a..943af2a 100644 --- a/ui/frame_apk.py +++ b/ui/frame_apk.py @@ -12,6 +12,13 @@ import zipfile ATTR = {0x01010001: 'label', 0x01010002: 'icon', 0x01010003: 'name', 0x0101021b: 'versionCode', 0x0101021c: 'versionName', 0x0101020c: 'minSdkVersion'} T_REF, T_STRING, T_INT_DEC, T_INT_HEX = 0x01, 0x03, 0x10, 0x11 +# APKs can come from websites (install links), so nothing read from one may be +# unbounded. zipfile stops at a member's declared size, so checking it is enough. +MAX_MANIFEST = 16 * 1024**2 +MAX_ARSC = 128 * 1024**2 # real ones are a few MB; the largest apps' tens of MB +MAX_ICON = 8 * 1024**2 +MAX_VALUES = 256 # resolved values per reference, across all its hops +MAX_STEPS = 4096 # entries examined per reference, dead ends and cycles included class ApkError(Exception): @@ -129,15 +136,23 @@ class Resources: resid = (pid << 24) | (tid << 16) | index self.entries.setdefault(resid, []).append((language, density, dtype, value)) - def values(self, resid, depth=0): - """[(language, density, type, data)] with references followed.""" + def values(self, resid, depth=0, seen=frozenset(), steps=None): + """[(language, density, type, data)] with references followed: never round a + cycle, at most MAX_VALUES results and MAX_STEPS entries examined in all.""" + steps = steps if steps is not None else [MAX_STEPS] out = [] + seen = seen | {resid} for lang, dens, dtype, value in self.entries.get(resid, []): + steps[0] -= 1 + if steps[0] < 0 or len(out) >= MAX_VALUES: + break if dtype == T_REF and depth < 5: - out += [(lang or l2, dens or d2, t2, v2) for l2, d2, t2, v2 in self.values(value, depth + 1)] + if value not in seen: + out += [(lang or l2, dens or d2, t2, v2) + for l2, d2, t2, v2 in self.values(value, depth + 1, seen, steps)] else: out.append((lang, dens, dtype, value)) - return out + return out[:MAX_VALUES] def string(self, dtype, value): return self.strings[value] if dtype == T_STRING and value < len(self.strings) else None @@ -171,6 +186,24 @@ def _icons(attr, res): return [s for _, s in sorted(vals, key=lambda x: -x[0]) if s] +def _read(z, name, limit): + """A member's bytes, inflating at most limit + 1 of them whatever its header claims + (ZipFile.read inflates everything first, then trims to the declared size).""" + info = z.getinfo(name) + # Android only reads stored and deflated entries, and only those bound what + # a read inflates (Python 3.9's bzip2 and lzma readers don't). + if info.compress_type not in (zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED): + raise ApkError(f'{name} in the APK uses a compression Android does not') + size = info.file_size + if size > limit: + raise ApkError(f'{name} in the APK is {size / 1024**2:.0f} MB, more than a real one ({limit // 1024**2} MB)') + with z.open(name) as f: + data = f.read(limit + 1) + if len(data) > limit: + raise ApkError(f'{name} in the APK is larger than a real one ({limit // 1024**2} MB)') + return data + + def apk_info(path): """Package, label, version, min_sdk, abis and the best PNG icon inside the APK.""" try: @@ -182,8 +215,8 @@ def apk_info(path): if 'AndroidManifest.xml' not in names: raise ApkError('not an APK: no AndroidManifest.xml') try: - elements = manifest_elements(z.read('AndroidManifest.xml')) - res = Resources(z.read('resources.arsc') if 'resources.arsc' in names else b'') + elements = manifest_elements(_read(z, 'AndroidManifest.xml', MAX_MANIFEST)) + res = Resources(_read(z, 'resources.arsc', MAX_ARSC) if 'resources.arsc' in names else b'') except (struct.error, IndexError, zipfile.BadZipFile) as e: raise ApkError(f'could not read the APK manifest: {e}') tags = {} @@ -212,11 +245,11 @@ def apk_info(path): def _icon_png(z, names, icons): for icon in icons: if icon.endswith('.png') and icon in names: - return z.read(icon) + return _read(z, icon, MAX_ICON) # Adaptive icons are XML; fall back to the largest launcher PNG. pngs = sorted((n for n in names if n.endswith('.png') and 'ic_launcher' in n and 'foreground' not in n), key=lambda n: z.getinfo(n).file_size) - return z.read(pngs[-1]) if pngs else None + return _read(z, pngs[-1], MAX_ICON) if pngs else None if __name__ == '__main__': diff --git a/ui/frame_connect.py b/ui/frame_connect.py index bdf93ec..9b75ded 100644 --- a/ui/frame_connect.py +++ b/ui/frame_connect.py @@ -1,29 +1,41 @@ -"""Connect this computer to the Steam Frame: find it, create a key, add a `Host frame` -alias to ~/.ssh/config and copy the key over, asking for the Developer Mode -password once. The Linux and Windows twin of scripts/connect.sh (which the Mac -app uses); same config block, so either can re-run over the other. Idempotent. +"""Connect this computer to the Steam Frame: find it, create keys, add a `Host frame` +alias to ~/.ssh/config and get a key onto the headset. It first asks Valve's +SteamOS devkit service (port 32000) to pair, which needs only a tap on the +headset; if that service isn't there or says no, it copies the key over SSH, +asking for the Developer Mode password once. The Linux and Windows twin of +scripts/connect.sh (which the Mac app uses); same config block, so either can +re-run over the other. Idempotent. Usage: python3 ui/frame_connect.py [HOST_OR_IP[:PORT]] Env: FRAME_USER (default steamos), FRAME_ALIAS (default frame) """ import base64 +import json import os import platform import re +import shutil import socket import subprocess import sys import time +import urllib.error +import urllib.request from pathlib import Path FRAME_USER = os.environ.get("FRAME_USER", "steamos") +USER_FROM_ENV = "FRAME_USER" in os.environ FRAME_ALIAS = os.environ.get("FRAME_ALIAS", "frame") SSH_DIR = Path.home() / ".ssh" KEY = SSH_DIR / "id_ed25519_frame" +# The devkit service only accepts ssh-rsa keys (write_key in Valve's +# steamos-devkit-service), so pairing uses a second key next to the ed25519 one. +DEVKIT_KEY = SSH_DIR / "id_rsa_frame_devkit" CONFIG = SSH_DIR / "config" +NAME_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]*") # Both go into ~/.ssh/config, so nothing that could add a line or a directive. for _name, _value in (("FRAME_ALIAS", FRAME_ALIAS), ("FRAME_USER", FRAME_USER)): - if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._-]*", _value): + if not NAME_RE.fullmatch(_value): sys.exit(f"{_name} must be a plain name, not {_value!r}") BEGIN = f"# >>> steam-frame ({FRAME_ALIAS}) >>>" END = f"# <<< steam-frame ({FRAME_ALIAS}) <<<" @@ -42,6 +54,108 @@ def say(msg): print(msg, flush=True) +# --- Valve's SteamOS devkit pairing (steamos-devkit-service on the headset). HTTP on +# port 32000: GET /properties.json names the user to log in as; POST /register with +# "ssh-rsa " shows an approve prompt in the headset (the +# comment is what it displays, 30 s to answer), then installs the key and turns sshd on. +# The prompt only appears while Steam is on Settings > Developer > Pair new host; +# otherwise /register answers 403 "please put the Steam client in pairing mode". + +DEVKIT_PORT = 32000 +DEVKIT_SERVICE = "_steamos-devkit._tcp" +MAGIC_PHRASE = "900b919520e4cf601998a71eec318fec" # fixed token Valve's client appends +REGISTER_TIMEOUT = 60 +PAIRING_MODE_WAIT = 120 # seconds to keep asking while the user opens "Pair new host" +# A LAN host: never go through an HTTP(S)_PROXY from the environment. +_opener = urllib.request.build_opener(urllib.request.ProxyHandler({})) + + +def key_comment(node): + """"frame-control@" as one word: the headset splits the body on spaces.""" + name = re.sub(r"[^A-Za-z0-9._-]+", "-", (node or "").split(".")[0]).strip("-.") or "computer" + return f"frame-control@{name}" + + +def register_body(pub, comment): + fields = pub.split() + if len(fields) < 2 or fields[0] != "ssh-rsa": + raise ValueError("the devkit service only takes ssh-rsa keys") + return f"ssh-rsa {fields[1]} {comment} {MAGIC_PHRASE}\n" + + +def parse_login(raw): + """The `login` from /properties.json if it's a plain user name, else None. "root" + means several users are configured and Valve's client switches between them; we + can't, so it counts as no answer.""" + props = json.loads(raw) + if not isinstance(props, dict): + raise ValueError("properties.json isn't a JSON object") + login = props.get("login") + if isinstance(login, str) and NAME_RE.fullmatch(login) and login != "root": + return login + return None + + +def devkit_error(status, raw): + """A readable reason from a failed /register: its {"error": ...} JSON, or the text.""" + text = raw.decode("utf-8", "replace").strip() + try: + err = json.loads(text).get("error") + except (ValueError, AttributeError): + err = None + return str(err or text or f"HTTP {status}")[:300] + + +def devkit_url(host, port, path): + return f"http://[{host}]:{port}{path}" if ":" in host else f"http://{host}:{port}{path}" + + +def why(e): + return str(getattr(e, "reason", None) or e) + + +def fetch_login(host, port=DEVKIT_PORT, timeout=5): + """GET /properties.json. Raises OSError (HTTP errors included) or ValueError.""" + with _opener.open(devkit_url(host, port, "/properties.json"), timeout=timeout) as r: + return parse_login(r.read()) + + +def register(host, body, port=DEVKIT_PORT, timeout=REGISTER_TIMEOUT): + """POST /register, which waits while someone answers the prompt. -> (ok, message)""" + req = urllib.request.Request(devkit_url(host, port, "/register"), data=body.encode("ascii"), + headers={"Content-Type": "text/plain"}, method="POST") + try: + with _opener.open(req, timeout=timeout) as r: + return True, r.read().decode("utf-8", "replace").strip() + except urllib.error.HTTPError as e: + with e: + return False, devkit_error(e.code, e.read()) + except OSError as e: + return False, f"no answer ({why(e)})" + + +def devkit_pair(host, pub, comment, port=DEVKIT_PORT, on_login=None): + """The password-free route. -> None once paired, else the reason, which means: fall + back to copying the key with the password. on_login(user) runs before the prompt + with the login properties.json names, so the fallback uses that user too.""" + try: + login = fetch_login(host, port) + except (OSError, ValueError) as e: + return f"devkit service not reachable on port {port}: {why(e)}" + if login and on_login: + on_login(login) + say(" In the headset: Steam Settings > Developer > Pair new host, then approve the request") + body = register_body(pub, comment) + ok, msg = register(host, body, port) + # The headset refuses at once unless Steam is on its "Pair new host" screen + # (verified on a Frame, 2026-09-26), so keep asking while the user opens it. + deadline = time.monotonic() + PAIRING_MODE_WAIT + while not ok and "pairing mode" in msg and time.monotonic() < deadline: + time.sleep(3) + ok, msg = register(host, body, port) + return None if ok else f"devkit pairing failed: {msg}" + + def split_port(arg): """"host:2222" -> ("host", 2222); anything else (IPv6 too) keeps port 22.""" host, sep, port = arg.rpartition(":") @@ -58,6 +172,70 @@ def port_open(host, port=22): return False +def reachable(host, port): + """sshd, or the devkit service, which turns sshd on once a pairing is approved.""" + try: + socket.getaddrinfo(host, port, type=socket.SOCK_STREAM) + except OSError: + return False + return port_open(host, port) or port_open(host, DEVKIT_PORT) + + +# --- mDNS. There's no stdlib client, so this borrows dns-sd (macOS; Bonjour for +# Windows) or avahi-browse (Linux) when present, with short timeouts. + +def run_for(args, seconds): + """What a command printed within `seconds`; dns-sd never exits by itself.""" + try: + out = subprocess.run(args, capture_output=True, timeout=seconds).stdout + except subprocess.TimeoutExpired as e: + out = e.stdout + except OSError: + out = b"" + return (out or b"").decode("utf-8", "replace") + + +def parse_dns_sd_browse(text): + """Instance names from `dns-sd -B _steamos-devkit._tcp`, deduplicated, in order.""" + pat = re.compile(r"\sAdd\s+\d+\s+\d+\s+\S+\s+" + re.escape(DEVKIT_SERVICE) + r"\.\s+(.+?)\s*$") + names = [] + for line in text.splitlines(): + m = pat.search(line) + if m and m.group(1) not in names: + names.append(m.group(1)) + return names + + +def parse_dns_sd_resolve(text): + """The target host from `dns-sd -L` ("... can be reached at frame.local.:32000").""" + m = re.search(r"can be reached at (\S+?)\.?:\d+", text) + return m.group(1) if m else None + + +def parse_avahi(text): + """Host names, then IPv4 addresses, from `avahi-browse -rpt` resolved ("=") lines.""" + names, addrs = [], [] + for line in text.splitlines(): + f = line.split(";") + if len(f) >= 9 and f[0] == "=" and f[2] == "IPv4": + names.append(f[6]) + addrs.append(f[7]) + return list(dict.fromkeys(names + addrs)) + + +def discover_devkit(): + if shutil.which("dns-sd"): + hosts = [] + for name in parse_dns_sd_browse(run_for(["dns-sd", "-B", DEVKIT_SERVICE, "local."], 3))[:4]: + host = parse_dns_sd_resolve(run_for(["dns-sd", "-L", name, DEVKIT_SERVICE, "local."], 2)) + if host and host not in hosts: + hosts.append(host) + return hosts + if shutil.which("avahi-browse"): + return parse_avahi(run_for(["avahi-browse", "-rpt", DEVKIT_SERVICE], 5)) + return [] + + HOST_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9.:%-]*") @@ -67,9 +245,16 @@ def pick_host(arg): return None for cand in [arg] if arg else [f"{FRAME_ALIAS}.local", FRAME_ALIAS]: host, port = split_port(cand) - if port_open(host, port): + if reachable(host, port): return host, port - say(f" - {cand}: not resolvable or port {port} closed") + say(f" - {cand}: not resolvable, or ports {port} and {DEVKIT_PORT} closed") + if arg: + return None + say(f" - asking mDNS for {DEVKIT_SERVICE}") + for host in discover_devkit(): + if HOST_RE.fullmatch(host) and reachable(host, 22): + return host, 22 + say(f" - {host}: advertised, but not reachable") return None @@ -82,7 +267,23 @@ def make_ssh_dir(): SSH_DIR.mkdir(mode=0o700, exist_ok=True) -def write_config(host, port=22): +def make_key(path, kind, comment): + if path.exists(): + say(f" exists: {path}") + return + bits = ["-b", "3072"] if kind == "rsa" else [] + subprocess.run(["ssh-keygen", "-q", "-t", kind, *bits, "-N", "", "-C", comment, "-f", str(path)], check=True) + say(f" created {path}") + + +def config_block(host, port=22, user=FRAME_USER): + return [BEGIN, f"Host {FRAME_ALIAS}", f" HostName {host}", *([f" Port {port}"] if port != 22 else []), + f" User {user}", + " IdentityFile ~/.ssh/id_ed25519_frame", " IdentityFile ~/.ssh/id_rsa_frame_devkit", + " IdentitiesOnly yes", " ServerAliveInterval 30", "Host *", END] + + +def write_config(host, port=22, user=FRAME_USER): """Replace our managed block and put it first: ssh uses the first value it sees per option. The trailing "Host *" returns the rest of the file to global scope.""" make_ssh_dir() @@ -95,10 +296,7 @@ def write_config(host, port=22): skip = False elif not skip: kept.append(line) - block = [BEGIN, f"Host {FRAME_ALIAS}", f" HostName {host}", *([f" Port {port}"] if port != 22 else []), - f" User {FRAME_USER}", - " IdentityFile ~/.ssh/id_ed25519_frame", " IdentitiesOnly yes", - " ServerAliveInterval 30", "Host *", END] + block = config_block(host, port, user) tmp = CONFIG.with_name("config.frame-control.tmp") tmp.write_text("\n".join(block + kept) + "\n", encoding="utf-8") if os.name != "nt": @@ -125,6 +323,50 @@ def key_login_works(): capture_output=True).returncode == 0 +def configured_user(): + """The User in our managed block, so a re-run keeps one the headset named earlier.""" + if not CONFIG.exists(): + return None + inside = False + for line in CONFIG.read_text(encoding="utf-8").splitlines(): + if line in (BEGIN, END): + inside = line == BEGIN + elif inside and line.startswith(" User "): + name = line[7:].strip() + return name if NAME_RE.fullmatch(name) else None + return None + + +def pair_with_devkit(host, port, user): + """Try devkit pairing and confirm key login. -> (user, None) or (user, reason to fall back).""" + say("==> Pairing through the headset's SteamOS devkit service (no password)") + chosen = [user] + + def use_login(login): + if login == chosen[0]: + return + if USER_FROM_ENV: + say(f" the headset logs in as '{login}'; keeping FRAME_USER={user}") + else: + chosen[0] = login + say(f" the headset logs in as '{login}'") + write_config(host, port, login) + + try: + pub = DEVKIT_KEY.with_suffix(".pub").read_text(encoding="utf-8") + except OSError as e: + return user, f"can't read the pairing key: {e}" + reason = devkit_pair(host, pub, key_comment(platform.node()), on_login=use_login) + if reason: + return chosen[0], reason + # The approval is what turns sshd on, so it may take a moment to answer. + for _ in range(10): + if key_login_works(): + return chosen[0], None + time.sleep(1) + return chosen[0], "paired, but key login still fails" + + def main(argv): if argv and argv[0] in ("-h", "--help"): sys.exit(__doc__) @@ -143,30 +385,32 @@ def main(argv): host, port = found say(f" found: {host}" + (f" port {port}" if port != 22 else "")) - say("==> SSH key") + say("==> SSH keys") make_ssh_dir() - if KEY.exists(): - say(f" exists: {KEY}") - else: - subprocess.run(["ssh-keygen", "-q", "-t", "ed25519", "-N", "", "-C", - f"{platform.node() or 'computer'}->steam-frame", "-f", str(KEY)], check=True) - say(f" created {KEY}") + make_key(KEY, "ed25519", f"{platform.node() or 'computer'}->steam-frame") + make_key(DEVKIT_KEY, "rsa", key_comment(platform.node())) + user = FRAME_USER if USER_FROM_ENV else (configured_user() or FRAME_USER) say(f"==> ~/.ssh/config alias '{FRAME_ALIAS}' -> {host}") - write_config(host, port) + write_config(host, port, user) say("==> Checking key login") if key_login_works(): say(" key login already works") else: - say(" copying the key: enter the Developer Mode password when asked") - pub = KEY.with_suffix(".pub").read_text(encoding="utf-8").strip() - r = subprocess.run(["ssh", "-o", "StrictHostKeyChecking=accept-new", "-o", "PubkeyAuthentication=no", - "-p", str(port), f"{FRAME_USER}@{host}", ADD_KEY_CMD], input=pub + "\n", text=True) - if r.returncode != 0 or not key_login_works(): - say("Key login still isn't working. Check the password and run this again.") - return 1 - say(" key login OK") + user, reason = pair_with_devkit(host, port, user) + if not reason: + say(" paired; key login OK") + else: + say(f" {reason}; falling back to the password") + say(" copying the key: enter the Developer Mode password when asked") + pub = KEY.with_suffix(".pub").read_text(encoding="utf-8").strip() + r = subprocess.run(["ssh", "-o", "StrictHostKeyChecking=accept-new", "-o", "PubkeyAuthentication=no", + "-p", str(port), f"{user}@{host}", ADD_KEY_CMD], input=pub + "\n", text=True) + if r.returncode != 0 or not key_login_works(): + say("Key login still isn't working. Check the password and run this again.") + return 1 + say(" key login OK") say(f"\nDone. Frame Control can reach the Frame now. In a terminal: ssh {FRAME_ALIAS}") return 0 diff --git a/ui/frame_titles.py b/ui/frame_titles.py new file mode 100644 index 0000000..bff6b56 --- /dev/null +++ b/ui/frame_titles.py @@ -0,0 +1,819 @@ +"""Linux and Windows builds on the Frame as Steam "Devkit Games". + +A .zip, a folder or a single executable becomes a title in the Steam library, +through the same path as Valve's SteamOS Devkit Client: its devkit-utils +(vendored in frame/devkit-utils, synced to ~/devkit-utils on the Frame) make +~/devkit-game//, the files are copied there, and steam-client-create-shortcut +asks the running Steam client to register it with a runtime: + + Windows .exe -> Proton Experimental (steam_play=1; x86-64 runs through FEX) + aarch64 ELF -> Steam Linux Runtime 4.0 ARM64 (steam_play=0) + x86-64 ELF -> Steam Linux Runtime 4.0 (steam_play=0; runs through FEX) + +Everything device-side is inferred from Valve's steamos-devkit source until +checked on a headset; see docs/sideloading.md. + +Python stdlib only. CLI: + python3 ui/frame_titles.py inspect PATH + python3 ui/frame_titles.py install PATH [--name N] [--exe REL] [--runtime R] + python3 ui/frame_titles.py list | launch ID | remove ID +""" +import hashlib, json, os, posixpath, re, shlex, shutil, stat, struct, subprocess, sys, tempfile, threading, time, zipfile + +import frame_android +import frame_host +from frame_android import FrameError + +ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +UTILS_LOCAL = os.path.join(ROOT, 'frame', 'devkit-utils') +UTILS = 'devkit-utils' # on the Frame, relative to $HOME (where Valve's client puts it) +GAMES = 'devkit-game' # ditto; steamos-prepare-upload makes / in here +STAMP = '.frame-control-stamp' +PY = 'python3 ~/' + UTILS + '/' + +# Valve's reserved sideload names: uploading one of these replaces the Steam client itself. +# devkit-steam is the trampoline file that switches SteamOS to a sideloaded client +# (select_steam.sh); a folder there breaks Valve's devkit tools. +RESERVED_IDS = ('steam', 'steamdeckard', 'steamvr', 'steamvrdeckard', 'devkit-steam') +ID_RE = re.compile(r'^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$') +DIR_RE = re.compile(r'^/[A-Za-z0-9_./-]+$') + +# Zip limits: well above any real game, well below a zip bomb. +MAX_UNPACKED = 64 * 1024**3 +MAX_ENTRIES = 200000 +TMP_PREFIX = 'frame-title-' # then the server's PID, so server.sweep_tmp can clear a killed run's +MAX_RATIO = 200 # uncompressed / compressed, once past 1 GB + +# The Steam compat tool aliases Valve's client uses (devkit_client RUNTIME_ALIASES). +RUNTIMES = { + 'proton-experimental': {'label': 'Proton Experimental', 'steam_play': True}, + 'proton-stable': {'label': 'Proton (stable)', 'steam_play': True}, + 'SteamLinuxRuntime_4-arm64': {'label': 'Steam Linux Runtime 4.0 (ARM64)', 'steam_play': False}, + 'SteamLinuxRuntime_4': {'label': 'Steam Linux Runtime 4.0 (x86-64, through FEX)', 'steam_play': False}, +} +# Experimental rather than stable: the Frame's ARM64 Proton + FEX stack is new, +# and Proton fixes reach Experimental first. --runtime proton-stable switches. +DEFAULT_PROTON = 'proton-experimental' + +ELF_MACHINES = {0xB7: 'arm64', 0x3E: 'x86_64', 0x03: 'x86', 0x28: 'arm'} +PE_MACHINES = {0x8664: 'x86_64', 0xAA64: 'arm64', 0x14C: 'x86', 0x1C4: 'arm'} +# Executables that are never the game: crash reporters, installers, redistributables. +SKIP_RE = re.compile(r'crash|unins|setup|install|redist|dxsetup|dxwebsetup|dotnet|prereq|' + r'easyanticheat|eac_|updater|uploader|report|sandbox|helper', re.I) +SKIP_DIRS = re.compile(r'^(_*commonredist|redist|redistributables?|directx|vcredist|__installer|' + r'installers?|prereqs?|support|engine|__macosx)$', re.I) +# Trailing words of an archive name that describe the build, not the game. +BUILD_WORDS = re.compile(r'([ ._-]+(win(dows)?(32|64)?|linux(32|64)?|x64|x86(_64)?|amd64|arm64|aarch64|' + r'build|release|portable|steamos|v\d+([._]\d+)*|\d+([._]\d+)+))+$', re.I) + + +def classify(path): + """{'format': 'elf'|'pe'|'script', 'arch', 'exe'} for an executable file, else None.""" + try: + with open(path, 'rb') as f: + head = f.read(4096) + if head[:4] == b'\x7fELF': + return _elf(f, head) + if head[:2] == b'MZ': + return _pe(f, head) + except (OSError, struct.error, ValueError, OverflowError): + return None # unreadable, or a header that lies about its sizes + if head[:2] == b'#!' or path.lower().endswith('.sh'): + return {'format': 'script', 'arch': None, 'exe': True} + return None + + +def _elf(f, head): + if len(head) < 64 or head[4] not in (1, 2) or head[5] not in (1, 2): + return None + wide, end = head[4] == 2, '<' if head[5] == 1 else '>' + e_type, machine = struct.unpack_from(end + 'HH', head, 16) + arch = ELF_MACHINES.get(machine, f'elf-0x{machine:x}') + if e_type == 2: # ET_EXEC + return {'format': 'elf', 'arch': arch, 'exe': True} + if e_type != 3: # not ET_DYN either: object file, core dump + return {'format': 'elf', 'arch': arch, 'exe': False} + # ET_DYN is a PIE executable or a shared library; only executables ask for an interpreter. + if wide: + phoff, = struct.unpack_from(end + 'Q', head, 32) + phentsize, phnum = struct.unpack_from(end + 'HH', head, 54) + else: + phoff, = struct.unpack_from(end + 'I', head, 28) + phentsize, phnum = struct.unpack_from(end + 'HH', head, 42) + if phentsize < (56 if wide else 32) or phnum > 256: + return {'format': 'elf', 'arch': arch, 'exe': False} + f.seek(phoff) + table = f.read(phentsize * phnum) + interp = any(struct.unpack_from(end + 'I', table, i * phentsize)[0] == 3 # PT_INTERP + for i in range(len(table) // phentsize)) + return {'format': 'elf', 'arch': arch, 'exe': interp} + + +def _pe(f, head): + if len(head) < 0x40: + return None + lfanew, = struct.unpack_from(' a local tree to upload ---------- + +def extract_zip(zpath, dest): + """Unpack a zip into dest, refusing paths that escape it and absurd sizes.""" + try: + z = zipfile.ZipFile(zpath) + except (zipfile.BadZipFile, OSError) as e: + raise FrameError(f'{os.path.basename(zpath)} is not a readable zip: {e}') + with z: + infos = z.infolist() + if len(infos) > MAX_ENTRIES: + raise FrameError(f'the zip has {len(infos)} entries; the limit is {MAX_ENTRIES}') + total = sum(i.file_size for i in infos) + packed = max(1, os.path.getsize(zpath)) + if total > MAX_UNPACKED or (total > 1024**3 and total > packed * MAX_RATIO): + raise FrameError(f'the zip would unpack to {total / 1024**3:.1f} GB, which looks wrong') + free = shutil.disk_usage(dest).free + if total + 256 * 1024**2 > free: + raise FrameError(f'not enough space on this computer to unpack the zip ' + f'({total / 1024**3:.1f} GB needed, {free / 1024**3:.1f} GB free)') + try: + _extract_members(z, infos, os.path.realpath(dest)) + except FrameError: + raise # a RuntimeError too, but already worded + except (zipfile.BadZipFile, RuntimeError, NotImplementedError, EOFError, OSError) as e: + # Encrypted or corrupt members, unsupported compression, clashing names, a full disk. + raise FrameError(f'could not unpack {os.path.basename(zpath)}: {e}') + + +def _extract_members(z, infos, root): + # No symlink is ever created here, so no write can be redirected through one + # (chained links, Windows without the privilege). Links inside the zip are + # resolved on paper and materialised as copies once the real files are out. + links = {} + for info in infos: + rel = _safe_member(info.filename) + if rel is None: + continue + target = _inside(root, rel, info.filename) + mode = info.external_attr >> 16 + if info.is_dir(): + os.makedirs(target, exist_ok=True) + continue + os.makedirs(os.path.dirname(target), exist_ok=True) + if stat.S_ISLNK(mode): + if info.file_size > 4096: # a link's content is a path, never this big + raise FrameError(f'the zip has an oversized link: {info.filename}') + link = z.read(info).decode('utf-8', 'replace').replace('\\', '/') + dest = posixpath.normpath(posixpath.join(posixpath.dirname(rel), link)) + if link.startswith('/') or ':' in link or dest == '..' or dest.startswith('../'): + raise FrameError(f'the zip has a link that points outside it: {info.filename}') + links[rel] = link # as written: resolved later, one component at a time + continue + with z.open(info) as src, open(target, 'wb') as out: + shutil.copyfileobj(src, out, 1 << 20) + if mode & 0o111: + os.chmod(target, 0o755) + _materialise_links(root, links) + + +def _inside(root, rel, name): + """rel's path under root, refusing anything that resolves outside it.""" + target = os.path.join(root, *rel.split('/')) + if not (os.path.realpath(target) + os.sep).startswith(root + os.sep): + raise FrameError(f'the zip has a path that climbs out of it: {name}') + return target + + +def _resolve_link(path, links): + """path with every link in it followed, on paper; None if it loops or leaves the zip. + + Like the kernel: each component in turn, so 'dirlink/..' is the parent of + where dirlink points, not the folder dirlink sits in. + """ + todo, done, hops = path.split('/'), [], 0 + while todo: + part = todo.pop(0) + if part in ('', '.'): + continue + if part == '..': + if not done: + return None + done.pop() + continue + done.append(part) + text = links.get('/'.join(done)) + if text is not None: + hops += 1 + if hops > 40: + return None + done.pop() # link text is relative to the link's folder + todo = text.split('/') + todo + return '/'.join(done) + + +def _materialise_links(root, links): + """Copy the file each link names into its place (lib.so.1 -> lib.so.1.2.3 and the like). + + Only links to files: a folder link could hold itself, and game builds link + libraries, not folders. Folder, looping and dangling links are dropped. + """ + copies = [] + for rel in sorted(links): + dest = _resolve_link(rel, links) + if not dest: + continue # loops, escapes, or the zip's own top folder + src, target = _inside(root, dest, rel), _inside(root, rel, rel) + if os.path.isfile(src) and not os.path.lexists(target): # a real entry may have the name + copies.append((src, target)) + # Many links to one big file could fill the disk: the same limits as the zip itself. + need = sum(os.path.getsize(src) for src, _ in copies) + if need + _tree_size(root) > MAX_UNPACKED: + raise FrameError('the zip\'s links would copy more than it holds, which looks wrong') + if need + 256 * 1024**2 > shutil.disk_usage(root).free: + raise FrameError(f'not enough space on this computer for the zip\'s linked files ({need / 1024**3:.1f} GB)') + for src, target in copies: + os.makedirs(os.path.dirname(target), exist_ok=True) + shutil.copy2(src, target) + + +def _safe_member(name): + """The member's relative path with / separators, None to skip it; raises if it escapes.""" + rel = name.replace('\\', '/') + if rel.startswith('/') or re.match(r'^[A-Za-z]:', rel): + raise FrameError(f'the zip has an absolute path: {name}') + parts = [p for p in rel.split('/') if p not in ('', '.')] + if any(p == '..' for p in parts): + raise FrameError(f'the zip has a path that climbs out of it: {name}') + if any(':' in p for p in parts): + # Drive-qualified parts ('C:..') climb out on Windows; ':' is an NTFS stream elsewhere. + raise FrameError(f'the zip has a path with a drive or stream name: {name}') + if not parts or parts[0] == '__MACOSX' or parts[-1] in ('.DS_Store', 'Thumbs.db'): + return None + return '/'.join(parts) + + +def _redirected(path, expected): + """True if path is a symlink, or resolves somewhere else (a Windows junction isn't islink).""" + return os.path.islink(path) or os.path.normcase(os.path.realpath(path)) != os.path.normcase(expected) + + +def _has_links(root): + real = os.path.realpath(root) + for dirpath, dirnames, filenames in os.walk(real): + for n in dirnames + filenames: + if _redirected(os.path.join(dirpath, n), os.path.join(dirpath, n)): + return True + return False + + +def _stage_folder(src, dest): + """Copy src to dest; links to files inside src become copies, all other links are left out.""" + real = os.path.realpath(src) + inside = lambda p: os.path.normcase(p).startswith(os.path.normcase(real) + os.sep) # noqa: E731 + folders, copies = [], [] # decide everything first, so the space check sees the same files + for dirpath, dirnames, filenames in os.walk(real): + out = os.path.join(dest, os.path.relpath(dirpath, real)) + folders.append(out) + # Only descend into real folders: not symlinked ones, not junctions (os.walk follows those). + linked = [d for d in dirnames if _redirected(os.path.join(dirpath, d), os.path.join(dirpath, d))] + dirnames[:] = [d for d in dirnames if d not in linked] + for fn in filenames + linked: + target = os.path.realpath(os.path.join(dirpath, fn)) + if inside(target) and os.path.isfile(target): + copies.append((target, os.path.join(out, fn))) + if shutil.disk_usage(os.path.dirname(dest)).free < sum(os.path.getsize(t) for t, _ in copies) + 256 * 1024**2: + raise FrameError('not enough space on this computer to stage the folder') + for out in folders: + os.makedirs(out, exist_ok=True) + for target, out in copies: + shutil.copy2(target, out) + return dest + + +def _below(top, root): + """The folders _unwrap stepped through from top to root, as 'a/b', or ''. Taken + before staging moves root elsewhere (possibly another drive on Windows).""" + rel = os.path.relpath(root, os.path.realpath(top)).replace(os.sep, '/') + return '' if rel == '.' else rel + + +def _unwrap(root): + """Step into a single top-level folder, the usual shape of a zipped build. + + Never through a link or junction: the folder you chose (or unpacked) stays the boundary. + """ + root = os.path.realpath(root) + for _ in range(4): + entries = [e for e in os.listdir(root) if e not in ('__MACOSX', '.DS_Store', 'Thumbs.db')] + if len(entries) != 1: + break + only = os.path.join(root, entries[0]) + if not os.path.isdir(only) or _redirected(only, only): + break + root = only + return root + + +def candidates(root, title=''): + """Executables under root, best launch target first.""" + found = [] + want = _norm(title) + for dirpath, dirnames, filenames in os.walk(root): + dirnames[:] = sorted(d for d in dirnames if not os.path.islink(os.path.join(dirpath, d))) + rel_dir = os.path.relpath(dirpath, root) + parts = [] if rel_dir == '.' else rel_dir.split(os.sep) + for fn in sorted(filenames): + full = os.path.join(dirpath, fn) + if os.path.islink(full) or not os.path.isfile(full): + continue + c = classify(full) + if not c or not c['exe']: + continue + stem = _norm(os.path.splitext(fn)[0]) + skip = bool(SKIP_RE.search(fn)) or any(SKIP_DIRS.match(p) for p in parts) + match = 2 if want and stem == want else 1 if want and stem and (want in stem or stem in want) else 0 + found.append({'path': '/'.join(parts + [fn]), 'format': c['format'], 'arch': c['arch'], + 'size': os.path.getsize(full), 'depth': len(parts), 'skip': skip, 'match': match}) + found.sort(key=_rank) + _prefer_launcher_script(found) + return found + + +def _platform_rank(c): + # Native ARM64 first, then Proton, then x86-64 Linux through FEX; scripts are placed separately. + order = {('elf', 'arm64'): 0, ('pe', 'x86_64'): 1, ('elf', 'x86_64'): 2, ('pe', 'x86'): 3, ('pe', 'arm64'): 3} + return order.get((c['format'], c['arch']), 5 if c['format'] == 'script' else 6) + + +def _rank(c): + return (c['skip'], _platform_rank(c), -c['match'], c['depth'], -c['size'], c['path']) + + +def _prefer_launcher_script(found): + """A top-level shell script beats a Linux binary one folder down (run.sh + bin/game).""" + if not found or found[0]['format'] != 'elf' or found[0]['depth'] == 0: + return + for i, c in enumerate(found): + if c['format'] == 'script' and c['depth'] == 0 and not c['skip']: + found.insert(0, found.pop(i)) + return + + +def runtime_for(target, found=()): + """(compat tool alias, note) for a launch target, or raise FrameError if it can't run.""" + fmt, arch = target['format'], target['arch'] + if fmt == 'script': + # A script runs in the runtime of the binaries next to it; alone, natively. + elf = next((c for c in found if c['format'] == 'elf' and not c['skip']), None) + if elf: + return runtime_for(elf)[0], 'A shell script; runtime chosen from the Linux binary next to it.' + return 'SteamLinuxRuntime_4-arm64', 'A shell script with no Linux binary beside it; run natively.' + if fmt == 'pe': + if arch not in ('x86_64', 'x86', 'arm64'): + raise FrameError(f"{target['path']} is a Windows program for {arch}, which Proton can't run") + note = 'Windows x86-64 build: Proton runs it through FEX.' if arch == 'x86_64' else \ + f'Windows {arch} build under Proton.' + return DEFAULT_PROTON, note + if fmt == 'elf' and arch == 'arm64': + return 'SteamLinuxRuntime_4-arm64', 'Native ARM64 Linux build.' + if fmt == 'elf' and arch == 'x86_64': + return 'SteamLinuxRuntime_4', ("x86-64 Linux build: probably won't start. It needs the x86-64 Steam " + "Linux Runtime 4.0, which the Frame didn't install for a sideloaded title " + "(2026-09-26). Use an ARM64 or Windows build if there is one.") + raise FrameError(f"{target['path']} is a {arch} Linux program; the Frame runs ARM64 and x86-64 (through FEX) only") + + +def allowed_runtimes(target, found=()): + alias, _ = runtime_for(target, found) + return ['proton-experimental', 'proton-stable'] if RUNTIMES[alias]['steam_play'] else [alias] + + +def inspect(path, name=None): + """Read a .zip, folder or executable into an install plan (a JSON-safe dict). + + A zip is unpacked into a temporary folder, plan['work']; pass the plan to + discard() when done with it. Raises FrameError if nothing in it can run. + """ + path = os.path.abspath(path) + if not os.path.exists(path): + raise FrameError(f'{path} does not exist') + work = None + try: + if os.path.isdir(path): + root = _unwrap(path) + unwrapped = _below(path, root) + if _has_links(root): + # scp -r follows links, so a link out of the folder could upload + # anything; copy the folder with its links made safe first. + work = tempfile.mkdtemp(prefix=f'{TMP_PREFIX}{os.getpid()}-') + root = _stage_folder(root, os.path.join(work, os.path.basename(root))) + elif path.lower().endswith('.zip'): + work = tempfile.mkdtemp(prefix=f'{TMP_PREFIX}{os.getpid()}-') + extract_zip(path, work) + root = _unwrap(work) + unwrapped = _below(work, root) + elif classify(path): + # A single executable is uploaded on its own; don't copy a whole Downloads folder. + work = tempfile.mkdtemp(prefix=f'{TMP_PREFIX}{os.getpid()}-') + shutil.copy2(path, os.path.join(work, os.path.basename(path))) + root, unwrapped = work, '' + else: + raise FrameError(f'{os.path.basename(path)} is not a .zip, a folder or a program') + title = name or display_name(os.path.basename(path.rstrip('/\\'))) + found = candidates(root, title) + if not found: + raise FrameError(f'no Linux or Windows program found in {os.path.basename(path)}') + plan = {'source': os.path.basename(path.rstrip('/\\')), 'name': title, 'id': title_id(title), + 'root': root, 'work': work, 'candidates': found, + 'unwrapped': unwrapped, + 'size': _tree_size(root), 'warnings': []} + _choose(plan, found[0]['path']) + return plan + except BaseException: + if work: + shutil.rmtree(work, ignore_errors=True) + raise + + +def _tree_size(root): + total = 0 + for dirpath, _, filenames in os.walk(root): + for fn in filenames: + full = os.path.join(dirpath, fn) + if os.path.isfile(full) and not os.path.islink(full): + total += os.path.getsize(full) + return total + + +def _choose(plan, rel, runtime=None): + """Set plan's launch target (a path relative to root) and its runtime.""" + rel = rel.replace('\\', '/') + # A manifest may name the program as it is in the archive, above the folder + # _unwrap stepped into. A path that works as it is always wins. + prefix = plan.get('unwrapped') and plan['unwrapped'] + '/' + if (prefix and rel.startswith(prefix) and not any(c['path'] == rel for c in plan['candidates']) + and not os.path.isfile(os.path.join(plan['root'], *rel.split('/')))): + rel = rel[len(prefix):] + target = next((c for c in plan['candidates'] if c['path'] == rel), None) + if target is None: + full = os.path.realpath(os.path.join(plan['root'], *rel.replace('\\', '/').split('/'))) + root = os.path.realpath(plan['root']) + if not (full + os.sep).startswith(root + os.sep) or not os.path.isfile(full): + raise FrameError(f'{rel} is not a file in the title') + c = classify(full) + if not c or not c['exe']: + raise FrameError(f"{rel} isn't a program the Frame can start") + target = {'path': os.path.relpath(full, root).replace(os.sep, '/'), 'format': c['format'], + 'arch': c['arch'], 'size': os.path.getsize(full), 'depth': rel.count('/'), + 'skip': False, 'match': 0} + alias, note = runtime_for(target, plan['candidates']) + allowed = allowed_runtimes(target, plan['candidates']) + if runtime: + if runtime not in allowed: + raise FrameError(f"{target['path']} can't use {runtime}; choose one of {', '.join(allowed)}") + alias = runtime + plan.update(target=target['path'], format=target['format'], arch=target['arch'], runtime=alias, + runtime_label=RUNTIMES[alias]['label'], runtimes=allowed, note=note) + plan['warnings'] = (['This looks like an installer or helper, not the game itself.'] if target['skip'] else []) + return plan + + +def discard(plan): + if plan and plan.get('work'): + shutil.rmtree(plan['work'], ignore_errors=True) + + +def argv_for(rel): + # Valve's client sends the start command as one string (it may carry arguments), + # so a path with spaces is quoted. How Steam splits it is inferred. + return ['"' + rel + '"' if re.search(r'\s', rel) else rel] + + +def shortcut_parms(gameid, directory, rel, runtime): + """The JSON steam-client-create-shortcut takes, as devkit_client.new_or_ensure_game builds it.""" + settings = {'steam_play': '1' if RUNTIMES[runtime]['steam_play'] else '0'} + if RUNTIMES[runtime]['steam_play']: + # gui2._update_game sends these with every Proton title; debugging stays off. + settings.update(steam_play_debug='0', steam_play_debug_version='2019') + settings['compat_tool'] = runtime + return {'gameid': gameid, 'directory': directory, 'argv': argv_for(rel), 'env': {}, + 'settings': settings, 'clear_settings': True, 'force_appid': '', 'lepton_args': ''} + + +# ---- the Frame side ---------------------------------------------------------- + +def ssh(cmd, input=None, timeout=120): + return frame_android.ssh(cmd, input=input, timeout=timeout) + + +def _json_out(out, what): + """The JSON object a devkit-utils script prints last (its logging goes to stderr).""" + for line in reversed(out.strip().splitlines()): + line = line.strip() + if line.startswith('{') or line.startswith('['): + try: + return json.loads(line) + except ValueError: + break + raise FrameError(f'{what} gave no usable answer: {out.strip()[-300:]!r}') + + +def utils_stamp(): + """Hash of the vendored devkit-utils, compared with the copy on the Frame.""" + h = hashlib.sha256() + for dirpath, dirnames, filenames in os.walk(UTILS_LOCAL): + dirnames[:] = sorted(d for d in dirnames if d != '__pycache__') + for fn in sorted(filenames): + if fn.endswith('.pyc'): + continue + full = os.path.join(dirpath, fn) + h.update(os.path.relpath(full, UTILS_LOCAL).replace(os.sep, '/').encode() + b'\0') + with open(full, 'rb') as f: + h.update(f.read()) + return h.hexdigest()[:20] + + +def _json_files(gid): + # Exact names: a glob like Game-*.json would also match another title called Game-Deluxe. + return ' '.join(f'{GAMES}/{gid}-{k}.json' for k in ('argv', 'env', 'settings', 'framecontrol')) + + +_utils_lock = threading.Lock() + + +def ensure_utils(): + """Copy frame/devkit-utils to ~/devkit-utils on the Frame unless it's already this version.""" + with _utils_lock: # one sync at a time: they share a staging folder + return _ensure_utils() + + +def _ensure_utils(): + stamp = utils_stamp() + have = ssh(f'cat {UTILS}/{STAMP} 2>/dev/null || true', timeout=30).strip() + if have == stamp: + return False + # Merge rather than replace: Valve's own client may have put newer files there. + tmp = f'.{UTILS}.frame-control' + ssh(f'rm -rf {tmp}', timeout=30) + _copy_tree(UTILS_LOCAL, tmp, timeout=300) + ssh(f'mkdir -p {UTILS} && cp -R {tmp}/. {UTILS}/ && rm -rf {tmp} {UTILS}/__pycache__ ' + f'&& echo {stamp} > {UTILS}/{STAMP}', timeout=60) + return True + + +def _copy_tree(src, dest, timeout=3 * 3600, delete=False): + """Copy a local folder's contents to dest on the Frame (dest ends up a copy of src). + + rsync where installed (not on Windows; see server.push_file), else scp -r + into a fresh dest, which is what Windows has. dest must be a plain path. + """ + name = os.path.basename(src.rstrip('/\\')) + opts = frame_android.SSH_OPTS # read now: the server swaps in its multiplexed options + if _rsync(): + cmd = ['rsync', '-a', *(['--delete'] if delete else []), '-e', shlex.join(['ssh', *opts]), + src.rstrip('/') + '/', f'{frame_android.FRAME}:{dest.rstrip("/")}/'] + else: + # scp -r copies src *into* dest when dest exists, so dest must not. + ssh(f'rm -rf {shlex.quote(dest)}', timeout=60) + cmd = ['scp', *opts, '-r', src, f'{frame_android.FRAME}:{dest}'] + try: + subprocess.run(cmd, check=True, capture_output=True, stdin=subprocess.DEVNULL, text=True, + errors='replace', timeout=timeout) + except subprocess.TimeoutExpired: + raise FrameError(f'copying {name} to the Frame timed out') + except subprocess.CalledProcessError as e: + raise FrameError(f'copying {name} to the Frame failed: {(e.stderr or "").strip()[-300:]}') + + +def _rsync(): + # Not on Windows: a Windows rsync (cwRsync, MSYS2) wouldn't take the POSIX -e quoting. + return not frame_host.WINDOWS and bool(shutil.which('rsync')) + + +_install_lock = threading.Lock() + + +def install(path, name=None, exe=None, runtime=None, progress=None): + """Sideload a .zip, folder or executable as a Devkit Game; returns the title dict. + + name: the Steam name (sanitised to the title id), default from the file name. + exe: launch target relative to the title's root, default the best candidate. + runtime: a compat tool alias from RUNTIMES that suits the target (e.g. + 'proton-stable' instead of the default Proton Experimental). + progress: optional callable(stage_text, fraction 0..1). + """ + plan = inspect(path, name) + try: + return install_plan(plan, name=name, exe=exe, runtime=runtime, progress=progress) + finally: + discard(plan) + + +def install_plan(plan, name=None, exe=None, runtime=None, progress=None): + """Install an inspect() plan, optionally with another name, target or runtime.""" + if name: + plan['name'], plan['id'] = name, title_id(name) + if exe or runtime: + _choose(plan, exe or plan['target'], runtime) + step = progress or (lambda *a: None) + with _install_lock: + return _install(plan, step) + + +def _install(plan, step): + gid = plan['id'] + if not ID_RE.match(gid) or gid.lower() in RESERVED_IDS: + raise FrameError(f'bad title id {gid!r}') + step("Syncing Valve's devkit tools to the Frame", 0.02) + ensure_utils() + existed = ssh(f'test -d {GAMES}/{gid} && echo yes || true', timeout=30).strip() == 'yes' + step('Preparing the title folder', 0.05) + ready = _json_out(ssh(f'{PY}steamos-prepare-upload --gameid {gid}', timeout=60), 'steamos-prepare-upload') + directory = str(ready.get('directory') or '') + if not DIR_RE.match(directory) or not directory.endswith(f'/{GAMES}/{gid}'): + raise FrameError(f'steamos-prepare-upload returned an unexpected folder {directory!r}') + registered = False + try: + step(f"Copying {plan['size'] / 1e6:.0f} MB to the Frame", 0.1) + if _rsync(): + _copy_tree(plan['root'], directory, delete=True) + else: + part = f"{directory.rsplit('/', 1)[0]}/.{gid}.upload" + _copy_tree(plan['root'], part) + ssh(f'rm -rf {directory} && mv {part} {directory}', timeout=120) + # Same modes Valve's client gives an upload (rsync --chmod=Du=rwx,Dgo=rx,Fu=rwx,Fog=rx). + ssh(f'chmod -R 755 {directory}', timeout=300) + step('Registering with Steam', 0.9) + parms = shortcut_parms(gid, directory, plan['target'], plan['runtime']) + reply = _json_out(ssh(f'{PY}steam-client-create-shortcut --parms {shlex.quote(json.dumps(parms))}', + timeout=90), 'steam-client-create-shortcut') + meta = {'id': gid, 'name': plan['name'], 'target': plan['target'], 'runtime': plan['runtime'], + 'source': plan['source'], 'size': plan['size'], 'installed': time.strftime('%Y-%m-%dT%H:%M:%S')} + ssh(f'cat > {GAMES}/{gid}-framecontrol.json', input=json.dumps(meta, indent=1), timeout=30) + registered = True # the files stay: Steam registers them once it's running + if 'error' in reply: + raise FrameError(f"Uploaded, but Steam didn't register it: {reply['error']}. " + "With Steam running on the Frame, install it again.") + step('Done', 1.0) + meta.update(runtime_label=RUNTIMES[plan['runtime']]['label'], steam=str(reply.get('success', '')).strip()) + return meta + finally: + if not registered and not existed: + # A first install that failed part-way: don't leave an orphan folder behind. + try: + ssh(f'rm -rf {GAMES}/{gid} {GAMES}/.{gid}.upload {_json_files(gid)}', timeout=60) + except FrameError: + pass + + +LIST_SCRIPT = r''' +import json, os +root = os.path.expanduser('~/devkit-game') +reserved = %r +out = [] +for d in sorted(os.listdir(root)) if os.path.isdir(root) else []: + if d.startswith('.') or d.lower() in reserved or not os.path.isdir(os.path.join(root, d)): + continue + t = {'id': d} + for key, suffix in (('settings', '-settings.json'), ('argv', '-argv.json'), ('meta', '-framecontrol.json')): + try: + with open(os.path.join(root, d + suffix)) as f: + t[key] = json.load(f) + except (OSError, ValueError): + t[key] = None + out.append(t) +print(json.dumps(out)) +''' % (RESERVED_IDS,) + + +def list_titles(): + """The Devkit Games on the Frame (any uploaded by Valve's client too).""" + raw = _json_out(ssh('python3 -', input=LIST_SCRIPT, timeout=30), 'the title list') + titles = [] + for t in raw if isinstance(raw, list) else []: + if not ID_RE.match(str(t.get('id', ''))): + continue + settings, meta = t.get('settings') or {}, t.get('meta') or {} + argv = t.get('argv') if isinstance(t.get('argv'), list) else [] + alias = str(settings.get('compat_tool') or '') + titles.append({'id': t['id'], 'name': str(meta.get('name') or t['id']), + 'target': str(meta.get('target') or (argv[0] if argv else '')), + 'runtime': alias, 'runtime_label': RUNTIMES.get(alias, {}).get('label', alias or 'not set'), + 'source': str(meta.get('source') or ''), 'size': meta.get('size'), + 'installed': meta.get('installed'), 'registered': t.get('settings') is not None, + 'frame_control': bool(meta)}) + return titles + + +def _check_id(gid): + gid = str(gid or '') + if not ID_RE.match(gid) or gid.lower() in RESERVED_IDS: + raise FrameError(f'bad title id {gid!r}') + if ssh(f'test -d {GAMES}/{gid} && echo yes || true', timeout=30).strip() != 'yes': + raise FrameError(f'{gid} is not installed') + return gid + + +def launch(gid): + gid = _check_id(gid) + ensure_utils() + # steam-devkit-rpc logs 'success' when Steam answers, but exits 0 after a + # 5 s timeout too, so read its log (stderr) rather than trust the exit code. + out = ssh(f'{PY}steam-devkit-rpc run-game gameid={gid} 2>&1', timeout=60) + if 'success' not in [line.strip() for line in out.splitlines()]: + raise FrameError(f"Steam didn't confirm the launch: {out.strip()[-300:] or 'no answer'}") + return {'id': gid} + + +def remove(gid): + # Not while an install runs: it could be this title, half copied or about to register. + if not _install_lock.acquire(blocking=False): + raise FrameError('an install is running; remove the title when it has finished') + try: + gid = _check_id(gid) + ensure_utils() + # steamos-delete removes the folder and syncs Steam's shortcuts; its json files stay, so clear them too. + ssh(f'{PY}steamos-delete --delete-title {gid}', timeout=120) + ssh(f'rm -f {_json_files(gid)}', timeout=30) + return {'id': gid} + finally: + _install_lock.release() + + +def public(plan): + """A plan without its local paths, for the UI, with the runtimes each candidate may use.""" + out = {k: v for k, v in plan.items() if k not in ('root', 'work', 'candidates')} + out['candidates'] = [] + for c in plan['candidates']: + c = dict(c) + try: + c['runtimes'] = allowed_runtimes(c, plan['candidates']) + except FrameError as e: + c['runtimes'], c['blocked'] = [], str(e) + out['candidates'].append(c) + out['runtime_labels'] = {k: v['label'] for k, v in RUNTIMES.items()} + return out + + +def main(): + cmd, *args = sys.argv[1:] or ['help'] + + def opt(flag): + return args[args.index(flag) + 1] if flag in args and args.index(flag) + 1 < len(args) else None + + try: + if cmd == 'inspect' and args: + plan = inspect(args[0], opt('--name')) + try: + if opt('--exe') or opt('--runtime'): + _choose(plan, opt('--exe') or plan['target'], opt('--runtime')) + r = public(plan) + finally: + discard(plan) + elif cmd == 'install' and args: + r = install(args[0], name=opt('--name'), exe=opt('--exe'), runtime=opt('--runtime'), + progress=lambda text, _: print(text + '…', file=sys.stderr)) + elif cmd == 'list': + r = list_titles() + elif cmd in ('launch', 'remove') and args: + r = globals()[cmd](args[0]) + else: + sys.exit(__doc__) + except FrameError as e: + sys.exit(f'error: {e}') + print(json.dumps(r, indent=1)) + + +if __name__ == '__main__': + main() diff --git a/ui/frame_webinstall.py b/ui/frame_webinstall.py new file mode 100644 index 0000000..2f2e006 --- /dev/null +++ b/ui/frame_webinstall.py @@ -0,0 +1,468 @@ +"""Install links from websites: frame-control://install?manifest=URL or ?url=URL. + +The app hands the link to the page, the page shows what it will install and +asks the user first, and only then does this module download the file and pass +it to the installer for its type (dispatch()). See docs/web-install.md. + +A manifest is the same JSON FrameDrop uses, so one works for both tools: + {"schema": "framedrop.install/v1", "name": "My Game", + "files": [{"url": "https://cdn.example.com/mygame-arm64.apk", "sha256": "..."}]} +"frame-control.install/v1" is accepted with the same shape. + +Rules: HTTPS only, except http(s)://localhost or 127.0.0.1 for testing, and then +only with FRAME_CONTROL_LOCAL_LINKS=1 set and when the link itself points there. +No credentials in URLs, no private, loopback, link-local or CGNAT addresses +(checked on every redirect, and the connection goes to the address that was +checked, so DNS can't change it in between). The file URL must end in a file name. + +Python stdlib only, 3.9 compatible. +""" +import hashlib +import http.client +import ipaddress +import json +import os +import errno +import re +import select +import socket +import ssl +import tempfile +import time +from urllib.parse import unquote, urljoin, urlsplit + +SCHEMAS = ("framedrop.install/v1", "frame-control.install/v1") +MAX_FILE = 4 * 1024**3 # largest download accepted +MAX_MANIFEST = 256 * 1024 # largest manifest accepted +MAX_URL = 2048 +MAX_REDIRECTS = 5 +TIMEOUT = 30 # seconds per socket operation +CHUNK = 1 << 20 +LOCAL_HOSTS = ("localhost", "127.0.0.1") +# Off by default: otherwise any website could make the app fetch from local services. +LOCAL_LINKS_ENV = "FRAME_CONTROL_LOCAL_LINKS" +USER_AGENT = "FrameControl (+https://github.com/saphid/steam-frame)" +# What dispatch() can install, by file extension. +KINDS = {".apk": "apk", ".zip": "title", ".exe": "title"} +KIND_LABEL = {"apk": "Android app (APK)", "title": "Linux/Windows title"} +SHA256 = re.compile(r"[0-9a-fA-F]{64}") +CGNAT = ipaddress.ip_network("100.64.0.0/10") +# connect_ex() results meaning "still connecting" (the last is Windows' WSAEWOULDBLOCK). +_CONNECTING = {errno.EINPROGRESS, errno.EWOULDBLOCK, errno.EALREADY, getattr(errno, "WSAEWOULDBLOCK", 10035)} + +# Swapped out by the tests, which have no network. +_getaddrinfo = socket.getaddrinfo + + +class WebInstallError(Exception): + pass + + +class Cancelled(WebInstallError): + pass + + +# ---- URLs ------------------------------------------------------------------- + +def is_public(ip): + """True for addresses on the public internet, and nothing a LAN or this computer uses.""" + ip = ipaddress.ip_address(ip) + if ip.version == 6: + if ip.ipv4_mapped: + ip = ip.ipv4_mapped + elif ip.is_site_local: # fec0::/10: deprecated, but is_global doesn't catch it + return False + elif ip.sixtofour and not is_public(ip.sixtofour): + return False + if ip.version == 4 and ip in CGNAT: + return False + return ip.is_global and not ip.is_multicast + + +def check_url(url, allow_local=False): + """Validate a URL against the rules above; returns (scheme, host, port, is_local). + + Resolving the name is left to connect time (see _resolve), so this needs no network. + """ + if not isinstance(url, str) or not url or len(url) > MAX_URL: + raise WebInstallError("the link must be a URL of at most %d characters" % MAX_URL) + if any(c.isspace() or ord(c) < 32 for c in url): + raise WebInstallError("the URL has spaces or control characters in it") + try: + u = urlsplit(url) + port = u.port + except ValueError as e: + raise WebInstallError(f"not a valid URL: {e}") + scheme = u.scheme.lower() + if scheme not in ("https", "http"): + raise WebInstallError(f"only https:// links are allowed, not {scheme or 'a relative URL'}") + if u.username is not None or u.password is not None or "@" in u.netloc: + raise WebInstallError("URLs with a user name or password in them aren't allowed") + host = (u.hostname or "").lower().rstrip(".") + if not host: + raise WebInstallError("the URL has no host") + local = host in LOCAL_HOSTS + if local and not allow_local: + raise WebInstallError(f"localhost links are for testing: set {LOCAL_LINKS_ENV}=1, and the link itself must point there") + if scheme == "http" and not local: + raise WebInstallError("only https:// is allowed (http:// only for localhost while testing)") + if not local: + try: + literal = ipaddress.ip_address(host) + except ValueError: + literal = None + if literal is not None and not is_public(literal): + raise WebInstallError(f"{host} is a private or local address") + return scheme, host, port or (443 if scheme == "https" else 80), local + + +def file_name(url): + """The file name the URL ends in, e.g. mygame-arm64.apk.""" + path = urlsplit(url).path + name = unquote(path.rsplit("/", 1)[-1]) + if not name or name in (".", "..") or "/" in name or "\\" in name or name.startswith(".") \ + or any(ord(c) < 32 for c in name) or len(name) > 200: + raise WebInstallError("the file URL must end in a file name, e.g. https://example.com/mygame.apk") + return name + + +def file_kind(name): + ext = os.path.splitext(name.lower())[1] + kind = KINDS.get(ext) + if not kind: + raise WebInstallError(f"{name}: Frame Control installs .apk, .zip and .exe files, not {ext or 'this type'}") + return kind + + +def _resolve(host, port, local): + """One address to connect to; every address the name has must be public.""" + if local: + return "127.0.0.1" + try: + infos = _getaddrinfo(host, port, type=socket.SOCK_STREAM) + except (OSError, UnicodeError) as e: + raise WebInstallError(f"couldn't look up {host}: {e}") + ips = [info[4][0].split("%", 1)[0] for info in infos] + if not ips: + raise WebInstallError(f"couldn't look up {host}") + for ip in ips: + if not is_public(ip): + raise WebInstallError(f"{host} points to a private or local address ({ip})") + return ips[0] + + +# ---- HTTP ------------------------------------------------------------------- + +class _Abortable: + """Connects to an address checked beforehand, whatever DNS says by then. + + raw_sock is the socket to shut down to stop the connection from another + thread (abort()): http.client drops conn.sock once a response will close + the connection, yet keeps reading the body from it. + """ + raw_sock = None + aborted = False + + def _tcp(self): + """Connect without blocking, so abort() can stop a connect that hangs.""" + sock = socket.socket(socket.AF_INET6 if ":" in self._ip else socket.AF_INET, socket.SOCK_STREAM) + try: + sock.setblocking(False) + err = sock.connect_ex((self._ip, self.port)) + deadline = time.monotonic() + self.timeout + while err in _CONNECTING: + if self.aborted: + raise OSError("aborted") + if time.monotonic() > deadline: + raise socket.timeout(f"timed out connecting to {self.host}") + _, writable, failed = select.select([], [sock], [sock], 0.2) + if writable or failed: + err = sock.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR) + if err: + raise OSError(err, os.strerror(err)) + sock.settimeout(self.timeout) + self.raw_sock = sock + if self.aborted: # abort() ran just now and found nothing to shut down + raise OSError("aborted") + except BaseException: + sock.close() + raise + return sock + + +class _HTTPConnection(_Abortable, http.client.HTTPConnection): + def __init__(self, host, ip, port, timeout): + super().__init__(host, port, timeout=timeout) + self._ip = ip + + def connect(self): + self.sock = self._tcp() + + +class _HTTPSConnection(_Abortable, http.client.HTTPSConnection): + """As above, still verifying the certificate for the host name.""" + + def __init__(self, host, ip, port, timeout): + # urllib's default context, so the app's bundled CA list (frame_host.trust_bundled_cas) applies too. + super().__init__(host, port, timeout=timeout, context=ssl._create_default_https_context()) + self._ip = ip + + def connect(self): + # Wrapping detaches the plain socket, so publish the TLS one before the handshake. + sock = self._context.wrap_socket(self._tcp(), server_hostname=self.host, do_handshake_on_connect=False) + self.raw_sock = sock + try: + if self.aborted: + raise OSError("aborted") + sock.do_handshake() + except (AttributeError, ValueError) as e: + # abort()'s shutdown() can tear down the TLS state mid-way. + sock.close() + if self.aborted: + raise OSError("aborted") + raise OSError(str(e)) + except BaseException: + sock.close() + raise + self.sock = sock + + +def _open(url, allow_local, method="GET", connected=None): + """(connection, response) for url after redirects, each hop checked. Caller closes the connection. + + connected(conn) gets each connection before it's used, for abort(). + """ + for _ in range(MAX_REDIRECTS + 1): + scheme, host, port, local = check_url(url, allow_local) + ip = _resolve(host, port, local) + cls = _HTTPSConnection if scheme == "https" else _HTTPConnection + conn = cls(host, ip, port, TIMEOUT) + if connected: + connected(conn) + u = urlsplit(url) + target = (u.path or "/") + ("?" + u.query if u.query else "") + try: + conn.request(method, target, headers={"User-Agent": USER_AGENT, "Accept-Encoding": "identity"}) + r = conn.getresponse() + except (OSError, http.client.HTTPException) as e: + conn.close() + raise WebInstallError(f"couldn't reach {host}: {e}") + if r.status in (301, 302, 303, 307, 308) and r.getheader("Location"): + url = urljoin(url, r.getheader("Location").strip()) + conn.close() + continue + if r.status != 200: + conn.close() + raise WebInstallError(f"{host} answered HTTP {r.status} {r.reason}".strip()) + return conn, r + raise WebInstallError(f"more than {MAX_REDIRECTS} redirects") + + +def _length(r): + try: + n = int(r.getheader("Content-Length") or "") + except ValueError: + return None + return n if n >= 0 else None + + +# ---- manifests -------------------------------------------------------------- + +def parse_manifest(obj): + """{"name": ..., "file": {"url", "sha256", "size", "exe"}} from a manifest object.""" + if not isinstance(obj, dict): + raise WebInstallError("the manifest must be a JSON object") + schema = obj.get("schema") + if schema not in SCHEMAS: + raise WebInstallError(f"unsupported manifest schema {schema!r} (expected {' or '.join(SCHEMAS)})") + files = obj.get("files") + if not isinstance(files, list) or not files: + raise WebInstallError("the manifest has no files") + if len(files) > 1: + raise WebInstallError(f"the manifest lists {len(files)} files; Frame Control installs one file per link for now") + entry = files[0] + if not isinstance(entry, dict) or not isinstance(entry.get("url"), str) or not entry["url"]: + raise WebInstallError("the manifest's file has no url") + sha = entry.get("sha256") + if sha is not None and (not isinstance(sha, str) or not SHA256.fullmatch(sha)): + raise WebInstallError("sha256 must be 64 hex digits") + size = entry.get("size") + if size is not None and (type(size) is not int or size <= 0): + raise WebInstallError("size must be a positive integer") + exe = entry.get("exe") + if exe is not None and (not isinstance(exe, str) or not exe or len(exe) > 300): + raise WebInstallError("exe must be a path inside the archive") + name = obj.get("name") + if name is not None and not isinstance(name, str): + raise WebInstallError("name must be a string") + return {"name": clean_name(name), "file": {"url": entry["url"], "sha256": sha.lower() if sha else None, + "size": size, "exe": exe}} + + +def clean_name(name): + name = re.sub(r"[\x00-\x1f\x7f]", "", name or "").strip() + return name[:120] or None + + +def fetch_manifest(url, allow_local): + conn, r = _open(url, allow_local) + try: + n = _length(r) + if n is not None and n > MAX_MANIFEST: + raise WebInstallError(f"the manifest is over {MAX_MANIFEST // 1024} KB") + data = r.read(MAX_MANIFEST + 1) + except (OSError, http.client.HTTPException) as e: + raise WebInstallError(f"couldn't read the manifest: {e}") + finally: + conn.close() + if len(data) > MAX_MANIFEST: + raise WebInstallError(f"the manifest is over {MAX_MANIFEST // 1024} KB") + try: + obj = json.loads(data.decode("utf-8")) + except (UnicodeDecodeError, ValueError): + raise WebInstallError("the manifest isn't valid JSON") + return parse_manifest(obj) + + +def _head_size(url, allow_local): + """Content-Length from a HEAD request, or None; only for showing the size up front.""" + try: + conn, r = _open(url, allow_local, method="HEAD") + except WebInstallError: + return None + try: + return _length(r) + finally: + conn.close() + + +def plan(manifest=None, url=None): + """Everything the confirm dialog shows, fetched and checked; nothing is downloaded yet. + + Exactly one of manifest (a manifest URL) or url (a direct file URL). + """ + if (manifest is None) == (url is None): + raise WebInstallError("give either manifest or url") + link = manifest if manifest is not None else url + # localhost is for testing a link on your own computer: only with the developer + # switch on, and only for a link that starts there (a public manifest can't + # point at localhost). + allow_local = check_url(link, allow_local=os.environ.get(LOCAL_LINKS_ENV) == "1")[3] + if manifest is not None: + m = fetch_manifest(manifest, allow_local) + name, f = m["name"], m["file"] + else: + name, f = None, {"url": url, "sha256": None, "size": None, "exe": None} + _, host, _, _ = check_url(f["url"], allow_local) + fname = file_name(f["url"]) + kind = file_kind(fname) + size = f["size"] or _head_size(f["url"], allow_local) + if size is not None and size > MAX_FILE: + raise WebInstallError(f"{fname} is {size / 1024**3:.1f} GB; the limit is {MAX_FILE / 1024**3:.0f} GB") + return {"name": name or fname, "url": f["url"], "file": fname, "kind": kind, "kindLabel": KIND_LABEL[kind], + "host": host, "linkHost": urlsplit(link).hostname, "size": size, "sha256": f["sha256"], + "exe": f["exe"], "source": link, "allowLocal": allow_local, "sizeFromManifest": bool(f["size"])} + + +def abort(conn): + """Stop conn from another thread (cancel, shutdown): unblocks a read, or makes the connect fail.""" + conn.aborted = True + sock = conn.raw_sock + if sock is not None: + try: + sock.shutdown(socket.SHUT_RDWR) + except OSError: + pass + + +def download(p, dest_dir, progress=None, cancelled=None, connected=None): + """Download plan p's file into dest_dir; returns its path. Checks the size cap and sha256. + + progress(done, total_or_None) is called as bytes arrive; cancelled() may return True to stop; + connected(conn) gets each connection before it's used, for abort(). + """ + dest = os.path.join(dest_dir, p["file"]) + try: + conn, r = _open(p["url"], p["allowLocal"], connected=connected) + except WebInstallError: + if cancelled and cancelled(): + raise Cancelled("download cancelled") + raise + fd, part = tempfile.mkstemp(prefix=".part-", dir=dest_dir) + out = os.fdopen(fd, "wb") + ok = False + try: + total = _length(r) + expected = p["size"] if p.get("sizeFromManifest") else None + if total is not None and total > MAX_FILE: + raise WebInstallError(f"the file is over the {MAX_FILE / 1024**3:.0f} GB limit") + if expected is not None and total is not None and total != expected: + raise WebInstallError(f"the server says {total} bytes; the manifest says {expected}") + digest = hashlib.sha256() + done = 0 + while True: + if cancelled and cancelled(): + raise Cancelled("download cancelled") + try: + chunk = r.read(CHUNK) + except (OSError, http.client.HTTPException) as e: + if cancelled and cancelled(): + raise Cancelled("download cancelled") + raise WebInstallError(f"download failed: {e}") + if not chunk: + if cancelled and cancelled(): # abort() makes the read end early + raise Cancelled("download cancelled") + break + done += len(chunk) + if done > MAX_FILE: + raise WebInstallError(f"the file is over the {MAX_FILE / 1024**3:.0f} GB limit") + digest.update(chunk) + out.write(chunk) + if progress: + progress(done, total or expected) + out.close() + if total is not None and done != total: + raise WebInstallError(f"download cut off at {done} of {total} bytes") + if expected is not None and done != expected: + raise WebInstallError(f"downloaded {done} bytes; the manifest says {expected}") + if p["sha256"] and digest.hexdigest() != p["sha256"]: + raise WebInstallError(f"{p['file']} doesn't match the manifest's sha256; not installing it") + os.replace(part, dest) + ok = True + return dest + finally: + out.close() + conn.close() + if not ok: + try: + os.remove(part) + except OSError: + pass + + +# ---- installing ------------------------------------------------------------- + +def dispatch(path, name=None, exe=None, progress=None, source=None): + """Install a downloaded file with the installer for its type; returns {"message", "kind", "result"}. + + .apk goes to frame_android (its own Lepton instance and Steam shortcut, named by + the APK's label); .zip and .exe to frame_titles. The caller has the SSH + connection ready. + """ + kind = file_kind(os.path.basename(path)) + if kind == "apk": + import frame_android + try: + m = frame_android.install(path, source=source or os.path.basename(path)) + except frame_android.FrameError as e: + raise WebInstallError(str(e)) + return {"message": f"Installed {m['label']} as its own app in the Steam library", "kind": kind, "result": m} + try: + import frame_titles + except ImportError as e: + if e.name != "frame_titles": + raise + raise WebInstallError("Linux/Windows titles need a newer Frame Control") + result = frame_titles.install(path, name=name, exe=exe, progress=progress) + msg = result.get("message") if isinstance(result, dict) else None + return {"message": msg or f"Installed {name or os.path.basename(path)}", "kind": kind, "result": result} diff --git a/ui/index.html b/ui/index.html index 320088c..475ea72 100644 --- a/ui/index.html +++ b/ui/index.html @@ -221,12 +221,23 @@ .and-grid { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 2fr); gap: 22px; align-items: start; } .and-col { display: grid; gap: 22px; align-content: start; } .rep-item .s { white-space: normal; } - #repDlg { background: #1e2329; color: var(--text); border: 1px solid rgba(255,255,255,.1); border-radius: 4px; + #repDlg, #titleDlg, #wiDlg { background: #1e2329; color: var(--text); border: 1px solid rgba(255,255,255,.1); border-radius: 4px; padding: 22px; width: min(560px, 92vw); box-shadow: 0 20px 60px rgba(0,0,0,.6); } - #repDlg::backdrop { background: rgba(0,0,0,.55); } - #repDlg h2 { margin: 0 0 14px; font-size: 15px; letter-spacing: 1.5px; text-transform: uppercase; color: var(--bright); } - #repForm label { display: block; font-size: 12.5px; color: var(--muted); margin-top: 10px; } - #repForm label input[type=text], #repForm textarea { margin-top: 5px; } + #repDlg::backdrop, #titleDlg::backdrop, #wiDlg::backdrop { background: rgba(0,0,0,.55); } + #repDlg h2, #titleDlg h2, #wiDlg h2 { margin: 0 0 14px; font-size: 15px; letter-spacing: 1.5px; text-transform: uppercase; color: var(--bright); } + #repForm label, #titleForm label { display: block; font-size: 12.5px; color: var(--muted); margin-top: 10px; } + #repForm label input[type=text], #repForm textarea, #titleForm label input, #titleForm label select { margin-top: 5px; } + #titleForm select { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; + border-radius: 3px; padding: 8px 10px; font: inherit; } + #titleForm select:focus { outline: none; border-color: var(--blue); } + #titleForm .note { font-size: 12.5px; color: var(--muted); margin-top: 10px; } + #titleForm .note.warn { color: #d9a23a; } + #titleList { margin-top: 8px; } + #wiFacts { display: grid; grid-template-columns: max-content 1fr; gap: 6px 14px; margin: 0; font-size: 13.5px; } + #wiFacts dt { color: var(--muted); } + #wiFacts dd { margin: 0; color: var(--bright); overflow-wrap: anywhere; } + #wiWarn { color: var(--muted); font-size: 12.5px; line-height: 1.45; margin: 14px 0 0; } + #wiProg:not([hidden]) { display: block; } #repForm fieldset { border: 0; padding: 0; margin: 12px 0 0; } #repForm legend { font-size: 12.5px; color: var(--muted); padding: 0; margin-bottom: 4px; } #repForm label.opt { display: inline-flex; align-items: center; gap: 6px; margin: 4px 14px 0 0; color: var(--text); font-size: 13.5px; } @@ -458,10 +469,13 @@

Send to Frame

Drop files here - Files land in ~/Downloads. .apk files install as their own Android app. + Files land in ~/Downloads. .apk files install as their own Android app; + a game's .zip, folder or .exe becomes a title in the Steam library.
+

Sideloaded titles

+
Loading…
@@ -567,6 +581,32 @@
+ +
+

Add to the Steam library

+
+ +
+ + +
+
+
+ + +
+
+
+ +

Install from a website

+
+

A website asked Frame Control to install this. Nothing is downloaded until you click Install. + Only install software from sites you trust.

+ +
+ +
+