Compare commits

..
Author SHA1 Message Date
saphidandClaude Opus 5.5 0f8aaa393b Store: show owned itch.io games under either VR filter
itch.io's game records don't say which games are VR, so owned purchases
came back with VR unknown and the "VR experiences" filter hid them (found
in a live test: an owned Quest game vanished). Unknown still counts as flat
for every other source.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:08:19 +11:00
saphidandClaude Opus 5.5 382fb0f906 itch.io purchases: find overlapping key matches so a partial match can't hide the real start
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 19:44:38 +11:00
saphidandClaude Opus 5.5 76e90af5b7 itch.io purchases: redact the key when it's percent-encoded twice
_clean (and so _carries_key) now also matches each key character as %25XX,
hex in either case and for either letter case of the character, keeping
the 8+ character starts. Tests: a fully double-encoded key, recased forms,
a start of it, a mix of literal, single and double encoding, and in a
storage URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 18:20:54 +11:00
saphidandClaude Opus 5.5 5456f2f6f6 itch.io purchases: redact the key when it's percent-encoded after a letter-case change
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:48:54 +11:00
saphidandClaude Opus 5.5 2cfb61a208 itch.io purchases: round-4 review fixes for per-hop key checks, redaction, IPv4 and the connect watchdog
- public_apk takes a check(url) callback run on every hop, before any lookup
  or connection; the itch.io download passes its key refusal, so a later
  redirect carrying the key (or ?api_key=) is refused too.
- One redaction function removes the key and its 8+ character starts in any
  letter case, with each character literal or percent-encoded (either hex
  case); used for every itch.io message, keeping error type and retry_after.
- IPv4 is an allow-list too: the whole special-purpose blocks (incl.
  192.0.0.0/24, which Python 3.9 partly calls global) are refused besides
  requiring is_global.
- No watchdog thread: connect and TLS handshake each get what is left of the
  shared deadline as their socket timeout, then the normal read timeout, so
  nothing can shut a connection after it has been handed over.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:40:54 +11:00
saphidandClaude Opus 5.5 c6e4362c66 itch.io purchases: round-3 review fixes for IPv6, keys in storage URLs, HTTP errors and connect time
- public_address treats IPv6 as an allow-list: global unicast (2000::/3)
  only, minus 6to4, Teredo, documentation, ORCHID and benchmarking blocks.
  Site-local fec0::/10 (is_global on Python 3.9) and everything else outside
  2000::/3 is refused.
- A storage URL from itch.io (Location or JSON url) that contains the API
  key, or 8+ characters of it in any case or encoding, or an api_key
  parameter, is refused before any connection. Storage-side errors lose the
  key and keep their type and retry_after.
- http.client errors (BadStatusLine, IncompleteRead, RemoteDisconnected)
  around every api.itch.io open and read become readable, key-free
  SourceErrors raised from None.
- Connecting shares one monotonic deadline (20 s, or the caller's timeout if
  shorter) across all addresses and both connect and TLS handshake, with a
  watchdog for a stalled handshake; reads keep their own timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:28:15 +11:00
saphidandClaude Opus 5.5 88d61cb3b6 itch.io purchases: round-2 review fixes for addresses, revocation races and key errors
- public_address tests each property explicitly (reserved, private,
  loopback, link-local, multicast, unspecified) as Python 3.9's is_global
  passes 4000::1 and 64:ff9b:1::/48. IPv6 forms carrying IPv4 (mapped,
  compatible, NAT64 incl. local-use, 6to4, Teredo) are refused outright.
- Every checked address is tried in turn (10 s each to connect and
  handshake), each pinned to its own address with the same TLS host name.
- _use reads the key and its connection generation together, and deletes
  only if both still match: an older refusal can't undo a reconnect, even
  with the same key.
- Key errors are judged on itch.io's own words before redaction, and any
  "invalid key" error outside /profile is confirmed with /profile before
  the key is deleted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 14:11:57 +11:00
saphidandClaude Opus 5.5 2d63dacdf9 itch.io purchases: review fixes for redirects, key redaction, races and revoked keys
- Storage downloads no longer go through a "any host name" redirect policy.
  _web.public_apk resolves each hop, refuses loopback, private, link-local,
  shared, reserved and multicast addresses (odd IPv4 spellings and IPv4
  mapped into IPv6 included), connects to the checked address, and checks
  again on every redirect. No credentials are sent there. _web.ANY and the
  extra open_url/apk parameters are gone; _web.apk's streaming is shared as
  save_apk.
- api.itch.io redirects are no longer followed with the key: the download
  step reads a 30x Location, a JSON body's url, or takes the file itself,
  since the live response is unverified.
- Error text loses the key, and any start of it from 8 characters, before
  joining and truncating.
- Saving and deleting the key are serialised; a connect whose /profile check
  finishes after a disconnect (or another connect) does not save. The key
  file is written through a unique mkstemp file.
- 401/403 from api.itch.io deletes the key only once /profile confirms the
  key itself is refused, including at download time with cached metadata.
  Storage-host 403s are download errors, not rate limits.
- Docs list which parts are verified against the public reference and which
  are assumed or unknown without a live account.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 13:59:15 +11:00
saphidandClaude Opus 5.5 adefdf4398 Store: install the Android builds of games you own on itch.io
Opt-in: paste an itch.io API key under My itch.io purchases in the store's
sources dialog. It is checked with GET /profile, saved 0600 in the app data
dir (itch-account.json), and only ever sent to api.itch.io as a Bearer
header. A new "itch.io (my purchases)" source lists owned Android games
(/profile/owned-keys, all pages), labelled "Owned on itch.io"; installs
pick the best Android upload (/games/ID/uploads) and download it through
/uploads/ID/download into the normal APK install path. A refused key is
never saved, or is deleted if itch.io later revokes it; rate limits and
network errors stay per-source and readable. The key is kept out of URLs,
errors, logs, API responses and the page.

Tests use recorded JSON and a fake HTTPS layer under urllib's real opener.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 12:50:30 +11:00
saphidandClaude Opus 5.5 2c3f8b3e0c APK sources: optional request headers and a redirect host policy for downloads
_web.open_url/apk take headers and the hosts a download may redirect to;
_web.ANY allows any HTTPS host name (no IP literals, credentials or other
ports). Authorization is still dropped on every redirect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 12:50:23 +11:00
45 changed files with 1831 additions and 7546 deletions

No files matched your search

-2
View File
@@ -213,8 +213,6 @@ Frame's software fits together, all checked against a real headset and labelled
| [Scripts and headset setup](docs/scripts.md) | The command-line helpers, minimum typing, streaming options, floating panels |
| [How the Frame works](docs/how-the-frame-works.md) | SteamVR → gamescope → Plasma, verified facts, debugging |
| [Android apps (Lepton)](docs/apks.md) | Sideloading, the rated F-Droid catalogue, per-app instances |
| [Import from my Quest](docs/quest-import.md) | Copy games you own off your Quest by USB; what runs, and why licence-checked games may not |
| [My Meta library](docs/meta-library.md) | Experimental: install and test Quest games your Meta account owns, with your own token |
| [Sideloading Linux and Windows games](docs/sideloading.md) | A .zip, folder or .exe as a Steam Devkit Game, runtime detection |
| [Install links for websites](docs/web-install.md) | `frame-control://install` links and manifests, the rules, a button to paste |
| [VR comfort and HUD](docs/vr-utilities.md) · [Steam games](docs/steam-games.md) · [VR video](docs/vr-video.md) · [WebXR in Chromium](docs/webxr-chromium.md) | Installing and buying, watching VR180/360, the Chromium build |
+1 -43
View File
@@ -1,7 +1,7 @@
// Frame Control as a desktop app (macOS, Windows, Linux): starts ui/server.py on
// a free loopback port and shows it in a native window. The server does all the
// work over the `frame` SSH alias; this file only hosts it.
const { app, BrowserWindow, Menu, Notification, clipboard, dialog, ipcMain, nativeImage, session, shell } = require("electron");
const { app, BrowserWindow, Menu, Notification, clipboard, dialog, ipcMain, nativeImage, shell } = require("electron");
const { execFile, spawn } = require("child_process");
const { promisify } = require("util");
const fs = require("fs");
@@ -11,7 +11,6 @@ const os = require("os");
const path = require("path");
const { SCHEME, parseInstallLink, linkFromArgv } = require("./install-link");
const updater = require("./updater");
const { createMetaSignIn } = require("./meta-signin");
const run = promisify(execFile);
@@ -348,47 +347,6 @@ ipcMain.handle("comfort:notify", (e, message) => {
});
});
// ---- Sign in with Meta (app/meta-signin.js, docs/meta-library.md) ----
// The token goes from the sign-in window's own session straight to the server's
// connect path; the page asks for it and only hears the server's answer.
function postApi(apiPath, body) {
return new Promise((resolve, reject) => {
if (!url) return reject(new Error("Frame Control's server isn't running."));
const data = JSON.stringify(body);
const req = http.request(new URL(apiPath, url), {
method: "POST", timeout: 60000,
headers: { "Content-Type": "application/json", "X-Frame-UI": process.env.FRAME_UI_KEY || "1",
"Content-Length": Buffer.byteLength(data) },
}, (res) => {
let text = "";
res.setEncoding("utf8");
res.on("data", (c) => { text += c; });
res.on("end", () => {
let parsed = {};
try { parsed = JSON.parse(text); } catch {}
resolve({ status: res.statusCode, data: parsed });
});
});
req.on("error", reject);
req.on("timeout", () => req.destroy(new Error("the server didn't answer")));
req.end(data);
});
}
let metaSignIn = null;
const meta = () => (metaSignIn ||= createMetaSignIn({
BrowserWindow, session, shell, parent: () => win,
// The page only offers sign-in after its "I understand" box is ticked (checked again below).
connect: (token, attempt) => postApi("/api/meta", { action: "connect", token, acknowledged: true, attempt }),
cancel: (attempt) => postApi("/api/meta", { action: "cancel-signin", attempt }),
}));
ipcMain.handle("meta:signin", (e, acknowledged) => {
if (!fromUi(e)) return null;
if (acknowledged !== true) return { ok: false, error: "My Meta library is experimental: tick that you understand the warning first." };
return meta().signIn();
});
ipcMain.handle("meta:signout", (e) => fromUi(e) ? meta().signOut() : false);
// frame-control://install links from websites (docs/web-install.md). They can
// arrive before the window or server exists (macOS open-url on a cold launch),
// so they wait here until the page asks for them. The page checks the link with
-252
View File
@@ -1,252 +0,0 @@
// "Sign in with Meta" for My Meta library (docs/meta-library.md), the way BSManager's
// "Log in with Meta" works: Meta's own login page in a Frame Control window, then the
// access token cookie (oc_ac_at) that page sets is handed to the local server's
// existing connect path (format check, "who am I" with Meta, 0600 file).
//
// The window has its own persistent session (PARTITION), so the cookie is read only
// from there: never from Frame Control's main window, never from any browser. It has
// no preload and no Node, and it only navigates within Meta's login sites; anything
// else opens in the system browser. Nothing here logs the token, and the page only
// ever gets back the server's answer (connected, display name), never the token.
// Electron is passed in, so app/test/meta-signin.test.js can run it with fakes.
const crypto = require("crypto");
const PARTITION = "persist:meta-signin";
// BSManager's loginUrl (src/main/services/bs-version-download/bs-oculus-downloader.service.ts).
const LOGIN_URL = "https://secure.oculus.com";
const COOKIE = "oc_ac_at";
// Sites the login can go through: the Oculus store web, Meta accounts, Facebook login,
// and Instagram (Meta's login passes through www.instagram.com: seen live, 2026-10-14).
const ALLOWED = ["oculus.com", "meta.com", "facebook.com", "instagram.com"];
const TIMEOUT = 10 * 60 * 1000; // as BSManager
const underDomain = (host, domains) => {
host = String(host || "").replace(/^\./, "").toLowerCase();
return domains.some((d) => host === d || host.endsWith("." + d));
};
// Plain HTTPS on the default port, to one of the login sites.
function allowedUrl(raw) {
let u;
try { u = new URL(String(raw)); } catch { return false; }
if (u.protocol !== "https:" || u.username || u.password || (u.port && u.port !== "443")) return false;
return underDomain(u.hostname, ALLOWED);
}
// Frames inside the page: any HTTPS address (Meta's login can embed captcha and CDN
// frames from other sites), or an empty frame; never data:, blob:, javascript:, http:,
// file: or any other scheme. Frames can't reach the cookie read or IPC (main process only).
function allowedSubframeUrl(raw) {
if (/^about:(?:blank|srcdoc)$/.test(String(raw))) return true;
let u;
try { u = new URL(String(raw)); } catch { return false; }
return u.protocol === "https:" && !u.username && !u.password;
}
const onOculus = (raw) => { try { return underDomain(new URL(raw).hostname, ["oculus.com"]); } catch { return false; } };
// A cheap first look, so a signed-out or app value is never sent anywhere; the server's
// check_token (ui/frame_meta.py) has the final word.
function looksLikeUserToken(v) {
return typeof v === "string" && /^(?:OC|FRL)[A-Za-z0-9+/=_-]{18,4093}$/.test(v)
&& !/^OC\d{15}$/.test(v) && /[a-z]/.test(v);
}
const fingerprint = (v) => crypto.createHash("sha256").update(v).digest("hex");
// Anything token-shaped out of a message, in case a server error ever carried one.
function scrub(text, token) {
let s = String(text == null ? "" : text);
if (token) s = s.split(token).join("<redacted>");
return s.replace(/(?:OC|FRL)[A-Za-z0-9+/=_%|:-]{8,}/g, "<redacted>").slice(0, 600);
}
// deps: { BrowserWindow, session, shell, parent() -> BrowserWindow|null, timeout,
// connect(token, attempt) -> Promise<{status, data}> (the server's connect action),
// cancel(attempt) -> Promise (the server's cancel-signin: never save that attempt) }
function createMetaSignIn({ BrowserWindow, session, shell, connect, cancel, parent = () => null, timeout = TIMEOUT }) {
let active = null; // { win, promise, finish }
const pending = new Set(); // cancels still to reach the server, from flows already finished
let signingOut = null; // the sign-out under way; no sign-in starts while it runs
const SIGNING_OUT = "Frame Control is signing out of Meta; try again in a moment.";
const openOutside = (target) => {
if (/^https?:\/\//i.test(String(target))) shell.openExternal(String(target));
};
function signIn() {
if (signingOut) return Promise.resolve({ ok: false, error: SIGNING_OUT });
if (active) { active.win.focus(); return active.promise; }
const ses = session.fromPartition(PARTITION);
ses.setPermissionRequestHandler((_wc, _permission, callback) => callback(false));
ses.setPermissionCheckHandler(() => false);
// Electron adds its own name and the app's to the user agent; send a plain Chrome one.
if (ses.getUserAgent && ses.setUserAgent) {
ses.setUserAgent(ses.getUserAgent().replace(/\s(?:Electron|frame-control|Frame Control)\/\S+/g, ""));
}
const p = parent();
const win = new BrowserWindow({
width: 520, height: 760, title: "Sign in with Meta – Frame Control", autoHideMenuBar: true,
...(p ? { parent: p } : {}),
webPreferences: { partition: PARTITION, contextIsolation: true, nodeIntegration: false, sandbox: true,
nodeIntegrationInSubFrames: false, webviewTag: false, safeDialogs: true, devTools: false },
});
const wc = win.webContents;
let resolve;
const promise = new Promise((r) => { resolve = r; });
let done = false, busy = false, again = false, lastError = null, blocked = null;
const tried = new Set(); // fingerprints of values already sent, so each goes once
// The server refuses to save a connect from an attempt this window gave up on.
const attemptId = crypto.randomUUID();
const onCookie = (_e, cookie, _cause, removed) => {
if (!removed && cookie && cookie.name === COOKIE && underDomain(cookie.domain, ["oculus.com"])) attempt();
};
const timer = setTimeout(() => finish({ ok: false, error: "Sign-in timed out after 10 minutes. Try again." }), timeout);
if (timer.unref) timer.unref();
// Stops cookie intake first, then the window (destroy: a page's beforeunload can't
// keep it open), and answers the page. Unless it connected, and if it sent a token, it
// then has the server drop this attempt: only once the connect under way has settled,
// so the server always sees the cancel after the connect (and deletes the token if
// that connect saved it). The returned promise settles when that is done.
let finished = null;
let inflight = null; // the last connect sent
function finish(result) {
if (done) return finished || Promise.resolve(); // destroy() below re-enters through "closed"
done = true;
ses.cookies.removeListener("changed", onCookie);
clearTimeout(timer);
if (active && active.win === win) active = null;
if (!win.isDestroyed()) win.destroy();
resolve(result);
if (result.ok || !inflight) return (finished = Promise.resolve()); // nothing sent: nothing to cancel
finished = Promise.resolve(inflight).catch(() => {}).then(() => cancel(attemptId)).catch(() => {});
pending.add(finished);
finished.then(() => pending.delete(finished));
return finished;
}
async function attempt() {
if (done) return;
if (busy) { again = true; return; }
busy = true;
try {
do {
again = false;
// Only this window's own session, and only Meta's Oculus cookie.
const cookies = await ses.cookies.get({ name: COOKIE });
const c = cookies.find((k) => underDomain(k.domain, ["oculus.com"]) && looksLikeUserToken(k.value));
if (!c || done) continue;
const key = fingerprint(c.value);
if (tried.has(key)) continue;
tried.add(key);
let r;
try {
inflight = connect(c.value, attemptId);
r = await inflight;
} catch {
lastError = "Frame Control's local server didn't answer.";
continue;
}
if (done) return;
const d = (r && r.data) || {};
if (r && r.status === 200 && d.connected) {
finish({ ok: true, connected: true, user: d.user && d.user.name ? { name: scrub(d.user.name, c.value).slice(0, 80) } : null,
message: scrub(d.message || "Connected your Meta account (Experimental)", c.value) });
return;
}
lastError = scrub(d.error || `HTTP ${r && r.status}`, c.value);
} while (again && !done);
} catch {
// The session went away (the window closed); nothing to do.
} finally {
busy = false;
}
}
// Navigation stays on Meta's login sites. A link elsewhere opens in the system
// browser; a redirect elsewhere is just stopped (it may carry sign-in data).
// Every frame: the window itself stays on the list; frames inside it may load any
// HTTPS page (captchas), and one going anywhere else is just stopped.
wc.on("will-navigate", (e, target) => { if (!allowedUrl(target)) { e.preventDefault(); openOutside(target); } });
// A stopped page change in the window itself says where it went (the host only), so
// closing the stuck window explains why instead of "closed before you signed in".
const stopped = (to) => {
let host = "";
try { host = new URL(String(to)).hostname; } catch { /* not an address */ }
blocked = `Meta's sign-in went to ${scrub(host || "an address", null).slice(0, 80)}, which Frame Control doesn't allow.`;
};
wc.on("will-frame-navigate", (e) => {
if (e.isMainFrame ? allowedUrl(e.url) : allowedSubframeUrl(e.url)) return;
e.preventDefault();
if (e.isMainFrame) stopped(e.url);
});
// Redirects fire for frames too: the same rule per frame (details on the event in
// current Electron, positional arguments in older ones; unknown counts as the window).
wc.on("will-redirect", (e, target, _inPlace, isMainFrame) => {
const to = e && e.url !== undefined ? e.url : target;
const main = e && e.isMainFrame !== undefined ? e.isMainFrame : isMainFrame !== false;
if (main ? allowedUrl(to) : allowedSubframeUrl(to)) return;
e.preventDefault();
if (main) stopped(to);
});
// A page's beforeunload never keeps the window open.
wc.on("will-prevent-unload", (e) => e.preventDefault());
wc.setWindowOpenHandler(({ url: target }) => {
if (allowedUrl(target)) wc.loadURL(target).catch(() => {});
else openOutside(target);
return { action: "deny" };
});
// Signed in: the cookie changes, or the window lands back on Meta's store site.
ses.cookies.on("changed", onCookie);
wc.on("did-navigate", (_e, target) => { blocked = null; if (onOculus(target)) attempt(); }); // it recovered
wc.on("did-navigate-in-page", (_e, target) => { if (onOculus(target)) attempt(); });
win.on("closed", () => finish({ ok: false, cancelled: true,
error: lastError ? `The sign-in window closed before Frame Control could connect. ${lastError}`
: blocked ? `The sign-in window was closed before you signed in. ${blocked}`
: "The sign-in window was closed before you signed in." }));
active = { win, promise, finish };
wc.loadURL(LOGIN_URL).catch(() => {});
return promise;
}
// Signs out of Meta in Frame Control: stops any sign-in under way (and has the server
// refuse to save it), destroys the window, then clears everything
// the sign-in session stored (cookies, storage, cache), so the next sign-in starts fresh.
// One at a time (a second call joins the first), and no sign-in can start meanwhile.
// It goes round until no window is open and no connect or cancel is left, so nothing
// started before it can set a token after it (or after the Disconnect that follows).
function signOut() {
if (!signingOut) {
signingOut = (async () => {
for (;;) {
if (active) {
const { win, finish } = active;
await finish({ ok: false, cancelled: true, error: "Signed out of Meta in Frame Control." });
// Clear only once nothing can write to the session again.
if (!win.isDestroyed()) throw new Error("The Sign in with Meta window didn't close; try again.");
} else if (pending.size) {
// A connect under way settles and is cancelled before the caller (Disconnect)
// deletes the account, so it can't be saved afterwards.
await Promise.all([...pending]);
} else {
break;
}
}
const ses = session.fromPartition(PARTITION);
await ses.clearStorageData();
await ses.clearCache();
if (ses.clearAuthCache) await ses.clearAuthCache();
return true;
})().finally(() => { signingOut = null; });
}
return signingOut;
}
return { signIn, signOut, isOpen: () => !!active, isSigningOut: () => !!signingOut };
}
module.exports = { createMetaSignIn, allowedUrl, allowedSubframeUrl, looksLikeUserToken, scrub, PARTITION, LOGIN_URL, COOKIE, ALLOWED };
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "frame-control",
"version": "0.4.5",
"version": "0.4.4",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "frame-control",
"version": "0.4.5",
"version": "0.4.4",
"license": "MIT",
"devDependencies": {
"electron": "^44.4.5",
+1 -2
View File
@@ -1,7 +1,7 @@
{
"name": "frame-control",
"productName": "Frame Control",
"version": "0.4.5",
"version": "0.4.4",
"description": "Desktop app for managing a Valve Steam Frame over SSH",
"private": true,
"main": "main.js",
@@ -39,7 +39,6 @@
"preload.js",
"install-link.js",
"updater.js",
"meta-signin.js",
"package.json",
"build/icon.png"
],
+1 -8
View File
@@ -6,8 +6,7 @@
// the headset can't be reached, and keeps the Frame menu's list of headsets up to date.
// It also receives frame-control://install links (docs/web-install.md): only
// what the link asked for, never an install; the page asks the user first.
// And it passes update state both ways: see app/updater.js. It opens the Sign in
// with Meta window and signs out of it (app/meta-signin.js).
// And it passes update state both ways: see app/updater.js.
const { contextBridge, ipcRenderer, webUtils } = require("electron");
contextBridge.exposeInMainWorld("frameApp", {
@@ -40,12 +39,6 @@ contextBridge.exposeInMainWorld("frameApp", {
ipcRenderer.removeAllListeners("report:open");
ipcRenderer.on("report:open", () => cb());
},
// Sign in with Meta (app/meta-signin.js): Meta's login page in its own window. The
// answer is the server's (connected, display name), never the token.
meta: {
signIn: (acknowledged) => ipcRenderer.invoke("meta:signin", acknowledged === true),
signOut: () => ipcRenderer.invoke("meta:signout"),
},
onInstallLink: (cb) => {
ipcRenderer.removeAllListeners("install-link");
ipcRenderer.on("install-link", (_e, req) => cb({ kind: req.kind, target: req.target }));
-578
View File
@@ -1,578 +0,0 @@
// Run: node --test app/test/
// "Sign in with Meta" (app/meta-signin.js) with fake Electron windows, sessions and cookies.
const nodeTest = require("node:test");
// A broken flow can leave a sign-in waiting forever; fail it instead of hanging.
const test = (name, fn) => nodeTest(name, { timeout: 5000 }, fn);
const assert = require("node:assert");
const { EventEmitter } = require("node:events");
const { createMetaSignIn, allowedUrl, looksLikeUserToken, scrub, PARTITION, LOGIN_URL, COOKIE } = require("../meta-signin");
const TOKEN = "OCAbcdefghijklmnopQRSTUVwxyz0123456789abcdEFGH";
const OTHER = "OCzyxwvutsrqponmlkJIHGFEDcba9876543210zyxwVUTS";
class FakeCookies extends EventEmitter {
constructor() { super(); this.store = []; this.reads = 0; }
async get(filter) {
this.reads++;
return this.store.filter((c) => !filter.name || c.name === filter.name);
}
set(cookie) { // as the page would: store it, then tell listeners
this.store = this.store.filter((c) => !(c.name === cookie.name && c.domain === cookie.domain)).concat(cookie);
this.emit("changed", {}, cookie, "explicit", false);
}
}
class FakeSession {
constructor(name, log) { this.name = name; this.log = log; this.cookies = new FakeCookies(); this.cleared = []; this.permission = null; }
setPermissionRequestHandler(fn) { this.permission = fn; }
setPermissionCheckHandler(fn) { this.permissionCheck = fn; }
getUserAgent() { return "Mozilla/5.0 Chrome/140.0 frame-control/0.4.4 Electron/44.4.5 Safari/537.36"; }
setUserAgent(ua) { this.ua = ua; }
async clearStorageData(opts) { this.log.push("clear"); this.cleared.push(["storage", opts]); }
async clearCache() { this.cleared.push(["cache"]); }
async clearAuthCache() { this.cleared.push(["auth"]); }
}
function fakes({ timeout } = {}) {
const log = []; // the order things happened in
const sessions = new Map();
const defaultSession = new FakeSession("default", log);
const session = {
defaultSession,
fromPartition(name) {
if (!sessions.has(name)) sessions.set(name, new FakeSession(name, log));
return sessions.get(name);
},
};
const windows = [];
class BrowserWindow extends EventEmitter {
constructor(opts) {
super();
this.opts = opts;
this.destroyed = false;
this.veto = false; // a page whose beforeunload keeps the window open on close()
this.focused = 0;
const wc = new EventEmitter();
wc.loaded = [];
wc.loadURL = async (u) => { wc.loaded.push(u); };
wc.setWindowOpenHandler = (fn) => { wc.openHandler = fn; };
// Deliberately a different session: the code must read the partition's, not guess.
wc.session = defaultSession;
this.webContents = wc;
windows.push(this);
}
isDestroyed() { return this.destroyed; }
focus() { this.focused++; }
close() { if (this.destroyed || this.veto) return; this.destroy(); }
destroy() { if (this.destroyed) return; this.destroyed = true; log.push("destroyed"); this.emit("closed"); }
}
const opened = [];
const shell = { openExternal: (u) => { opened.push(u); } };
const calls = [], attempts = [], cancels = [];
let answer = () => ({ status: 200, data: { connected: true, user: { name: "Alex" }, message: "Connected your Meta account (Experimental)" } });
const connect = async (token, attempt) => { calls.push(token); attempts.push(attempt); log.push("connect"); return answer(token); };
const cancel = async (attempt) => {
cancels.push(attempt);
log.push(`cancel (cookie listeners: ${session.fromPartition(PARTITION).cookies.listenerCount("changed")})`);
};
const flow = createMetaSignIn({ BrowserWindow, session, shell, connect, cancel, parent: () => null, ...(timeout ? { timeout } : {}) });
return { flow, session, sessions, windows, opened, calls, attempts, cancels, log, setAnswer: (fn) => { answer = fn; },
meta: () => session.fromPartition(PARTITION) };
}
const tick = () => new Promise((r) => setImmediate(r));
async function settle() { for (let i = 0; i < 10; i++) await tick(); }
function navigate(wc, event, target) {
let prevented = false;
wc.emit(event, { preventDefault: () => { prevented = true; } }, target);
return prevented;
}
test("only plain HTTPS addresses on Meta's login sites are allowed", () => {
for (const ok of ["https://secure.oculus.com", "https://secure.oculus.com/my/profile", "https://auth.meta.com/login/",
"https://www.meta.com/", "https://www.facebook.com/login.php", "https://oculus.com:443/",
"https://www.instagram.com/accounts/login/"]) {
assert.ok(allowedUrl(ok), ok);
}
for (const bad of ["http://secure.oculus.com/", "https://oculus.com.evil.example/", "https://evil-oculus.com/",
"https://notmeta.com/", "https://example.com/", "javascript:alert(1)", "file:///etc/passwd",
"https://secure.oculus.com:8443/", "https://user:pw@secure.oculus.com/", "https://evil.example/?u=https://www.meta.com",
"data:text/html,hi", "not a url"]) {
assert.ok(!allowedUrl(bad), bad);
}
});
test("the window is isolated: its own partition, no preload, no Node, sandboxed, on Meta's login page", async () => {
const f = fakes();
f.flow.signIn();
const [w] = f.windows;
const wp = w.opts.webPreferences;
assert.strictEqual(wp.partition, PARTITION);
assert.strictEqual(PARTITION, "persist:meta-signin");
assert.strictEqual(wp.contextIsolation, true);
assert.strictEqual(wp.nodeIntegration, false);
assert.strictEqual(wp.sandbox, true);
assert.strictEqual(wp.preload, undefined);
assert.strictEqual(wp.devTools, false);
assert.deepStrictEqual(w.webContents.loaded, [LOGIN_URL]);
assert.strictEqual(LOGIN_URL, "https://secure.oculus.com");
// Permission prompts (camera, notifications...) are refused.
let granted = null;
f.meta().permission(null, "media", (v) => { granted = v; });
assert.strictEqual(granted, false);
assert.strictEqual(f.meta().permissionCheck(null, "notifications", "https://secure.oculus.com"), false);
assert.ok(!/Electron|frame-control/.test(f.meta().ua));
w.close();
});
test("navigation stays on Meta's sites; other links open in the system browser", async () => {
const f = fakes();
f.flow.signIn();
const wc = f.windows[0].webContents;
assert.strictEqual(navigate(wc, "will-navigate", "https://auth.meta.com/login/"), false);
assert.strictEqual(navigate(wc, "will-navigate", "https://www.facebook.com/login.php"), false);
assert.strictEqual(navigate(wc, "will-navigate", "https://example.com/help"), true);
assert.deepStrictEqual(f.opened, ["https://example.com/help"]);
// A redirect elsewhere is stopped and not opened anywhere (it may carry sign-in data).
assert.strictEqual(navigate(wc, "will-redirect", "https://evil.example/cb?x=1"), true);
assert.strictEqual(navigate(wc, "will-redirect", "https://auth.meta.com/next"), false);
assert.deepStrictEqual(f.opened, ["https://example.com/help"]);
// New windows: never created. Meta's own load in this window; others go outside; non-web schemes nowhere.
assert.deepStrictEqual(wc.openHandler({ url: "https://www.meta.com/legal/terms/" }), { action: "deny" });
assert.deepStrictEqual(wc.openHandler({ url: "https://example.org/" }), { action: "deny" });
assert.deepStrictEqual(wc.openHandler({ url: "file:///etc/passwd" }), { action: "deny" });
await settle();
assert.deepStrictEqual(wc.loaded, [LOGIN_URL, "https://www.meta.com/legal/terms/"]);
assert.deepStrictEqual(f.opened, ["https://example.com/help", "https://example.org/"]);
f.windows[0].close();
});
test("the cookie is read only from the sign-in partition, only for Meta's Oculus domain", async () => {
const f = fakes();
// The same cookie elsewhere: the default session and another partition. Never read.
f.session.defaultSession.cookies.store.push({ name: COOKIE, domain: ".oculus.com", value: OTHER });
f.session.fromPartition("persist:other").cookies.store.push({ name: COOKIE, domain: ".oculus.com", value: OTHER });
const done = f.flow.signIn();
const w = f.windows[0];
// A cookie with the right name on another site, and a signed-out value: ignored.
f.meta().cookies.set({ name: COOKIE, domain: ".evil.example", value: OTHER });
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: "OC123456789012345" });
w.webContents.emit("did-navigate", {}, "https://secure.oculus.com/my/profile");
await settle();
assert.deepStrictEqual(f.calls, []);
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
const r = await done;
assert.deepStrictEqual(f.calls, [TOKEN]);
assert.strictEqual(r.ok, true);
assert.strictEqual(f.session.defaultSession.cookies.reads, 0);
assert.strictEqual(f.session.fromPartition("persist:other").cookies.reads, 0);
assert.ok(f.sessions.get(PARTITION).cookies.reads > 0);
});
test("landing on the logged-in page picks up a cookie set earlier", async () => {
const f = fakes();
const done = f.flow.signIn();
f.meta().cookies.store.push({ name: COOKIE, domain: "secure.oculus.com", value: TOKEN }); // no change event
f.windows[0].webContents.emit("did-navigate", {}, "https://auth.meta.com/login/"); // not Meta's store site yet
await settle();
assert.deepStrictEqual(f.calls, []);
f.windows[0].webContents.emit("did-navigate", {}, "https://secure.oculus.com/my/profile");
assert.strictEqual((await done).ok, true);
assert.deepStrictEqual(f.calls, [TOKEN]);
});
test("after a connect the window closes, and the page gets the answer, never the token", async () => {
const f = fakes();
const done = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
const r = await done;
assert.ok(f.windows[0].isDestroyed());
assert.ok(!f.flow.isOpen());
assert.deepStrictEqual(r, { ok: true, connected: true, user: { name: "Alex" }, message: "Connected your Meta account (Experimental)" });
assert.ok(!JSON.stringify(r).includes(TOKEN));
// The cookie listener is gone: a later change sends nothing.
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: OTHER });
await settle();
assert.deepStrictEqual(f.calls, [TOKEN]);
});
test("a token the server refuses keeps the window open, is sent once, and the next one connects", async () => {
const f = fakes();
f.setAnswer((t) => t === TOKEN ? { status: 400, data: { error: "Meta didn't accept this token." } }
: { status: 200, data: { connected: true, user: null } });
const done = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
await settle();
f.windows[0].webContents.emit("did-navigate", {}, "https://secure.oculus.com/my/profile"); // same value again
await settle();
assert.deepStrictEqual(f.calls, [TOKEN]);
assert.ok(!f.windows[0].isDestroyed());
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: OTHER });
const r = await done;
assert.deepStrictEqual(f.calls, [TOKEN, OTHER]);
assert.strictEqual(r.ok, true);
assert.strictEqual(r.user, null);
});
test("the token is never logged, and an error that echoed it is scrubbed", async () => {
const seen = [];
const names = ["log", "info", "warn", "error", "debug", "trace"];
const saved = Object.fromEntries(names.map((n) => [n, console[n]]));
const writes = [process.stdout.write, process.stderr.write];
for (const n of names) console[n] = (...a) => { seen.push(a.map(String).join(" ")); };
process.stdout.write = (c, ...rest) => { seen.push(String(c)); return writes[0].call(process.stdout, c, ...rest); };
process.stderr.write = (c, ...rest) => { seen.push(String(c)); return writes[1].call(process.stderr, c, ...rest); };
let r;
try {
const f = fakes();
f.setAnswer((t) => ({ status: 500, data: { error: `RuntimeError: bad ${t} and ${t.slice(0, 12)}` } }));
const done = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
await settle();
f.windows[0].close(); // the user gives up
r = await done;
} finally {
for (const n of names) console[n] = saved[n];
[process.stdout.write, process.stderr.write] = writes;
}
assert.strictEqual(r.ok, false);
assert.strictEqual(r.cancelled, true);
assert.ok(!r.error.includes(TOKEN) && !r.error.includes(TOKEN.slice(0, 12)), r.error);
assert.match(r.error, /<redacted>/);
assert.ok(!seen.some((s) => s.includes(TOKEN.slice(0, 12))), "token in output");
assert.strictEqual(scrub(`x FRL${"a".repeat(30)} y`), "x <redacted> y");
});
test("closing the window before signing in reports it, and a second click reuses the open window", async () => {
const f = fakes();
const a = f.flow.signIn();
const b = f.flow.signIn();
assert.strictEqual(a, b);
assert.strictEqual(f.windows.length, 1);
assert.strictEqual(f.windows[0].focused, 1);
f.windows[0].close();
const r = await a;
assert.deepStrictEqual(r, { ok: false, cancelled: true, error: "The sign-in window was closed before you signed in." });
assert.deepStrictEqual(f.calls, []);
});
test("disconnect's sign-out closes the window and clears only the sign-in partition", async () => {
const f = fakes();
f.session.fromPartition("persist:other");
const pending = f.flow.signIn();
await f.flow.signOut();
const r = await pending;
assert.strictEqual(r.ok, false);
assert.ok(f.windows[0].isDestroyed());
assert.deepStrictEqual(f.meta().cleared, [["storage", undefined], ["cache"], ["auth"]]);
assert.deepStrictEqual(f.session.defaultSession.cleared, []);
assert.deepStrictEqual(f.session.fromPartition("persist:other").cleared, []);
// The next sign-in starts with a new window.
f.flow.signIn();
assert.strictEqual(f.windows.length, 2);
f.windows[1].close();
});
test("the first look at a cookie value matches the server's token rules", () => {
assert.ok(looksLikeUserToken(TOKEN));
assert.ok(looksLikeUserToken("FRL" + "aB3".repeat(10)));
for (const bad of [undefined, "", "OC123456789012345", "OC|123|abc", "OCabc%2Fdef" + "x".repeat(20), "OCshort",
"XY" + "a".repeat(30), "OC" + "A".repeat(30)]) {
assert.ok(!looksLikeUserToken(bad), String(bad));
}
});
function frameNav(wc, url, isMainFrame) {
let prevented = false;
wc.emit("will-frame-navigate", { url, isMainFrame, preventDefault: () => { prevented = true; } });
return prevented;
}
test("the window stays on Meta's sites; frames inside it may load any HTTPS page, nothing else", async () => {
const f = fakes();
f.flow.signIn();
const wc = f.windows[0].webContents;
// Frames: captcha and CDN frames from other sites load; other schemes are stopped.
for (const ok of ["https://client-api.arkoselabs.com/fc/gc/", "https://newassets.hcaptcha.com/captcha/v1/x",
"https://www.google.com/recaptcha/api2/anchor", "https://static.xx.fbcdn.net/x",
"https://www.facebook.com/plugin", "about:blank", "about:srcdoc"]) {
assert.strictEqual(frameNav(wc, ok, false), false, ok);
}
for (const bad of ["http://www.facebook.com/plugin", "data:text/html,<script>x</script>", "blob:https://evil.example/1",
"javascript:alert(1)", "file:///etc/passwd", "ftp://example.com/", "https://u:p@evil.example/",
"chrome://settings", "not a url"]) {
assert.strictEqual(frameNav(wc, bad, false), true, bad);
}
// The window itself: Meta's sites only.
assert.strictEqual(frameNav(wc, "about:blank", true), true);
assert.strictEqual(frameNav(wc, "https://evil.example/", true), true);
assert.strictEqual(frameNav(wc, "https://client-api.arkoselabs.com/", true), true);
assert.strictEqual(frameNav(wc, "https://auth.meta.com/login/", true), false);
assert.deepStrictEqual(f.opened, []); // stopped, never opened outside
f.windows[0].destroy();
});
function redirect(wc, url, isMainFrame, style) {
let prevented = false;
const e = { preventDefault: () => { prevented = true; } };
if (style === "details") wc.emit("will-redirect", Object.assign(e, { url, isMainFrame }));
else wc.emit("will-redirect", e, url, false, isMainFrame);
return prevented;
}
test("redirects follow the same rule per frame", async () => {
const f = fakes();
f.flow.signIn();
const wc = f.windows[0].webContents;
for (const style of ["details", "positional"]) {
// A captcha frame redirected within HTTPS loads; a frame redirected to data: or http: doesn't.
assert.strictEqual(redirect(wc, "https://client-api.arkoselabs.com/v2/next", false, style), false, style);
assert.strictEqual(redirect(wc, "https://newassets.hcaptcha.com/x", false, style), false, style);
assert.strictEqual(redirect(wc, "data:text/html,<b>x</b>", false, style), true, style);
assert.strictEqual(redirect(wc, "http://www.facebook.com/x", false, style), true, style);
assert.strictEqual(redirect(wc, "javascript:alert(1)", false, style), true, style);
// The window: Meta's sites only.
assert.strictEqual(redirect(wc, "https://client-api.arkoselabs.com/v2/next", true, style), true, style);
assert.strictEqual(redirect(wc, "https://auth.meta.com/next", true, style), false, style);
}
// Without frame information it is treated as the window.
assert.strictEqual(redirect(wc, "https://client-api.arkoselabs.com/", undefined, "positional"), true);
assert.deepStrictEqual(f.opened, []);
f.windows[0].destroy();
});
test("a redirect the window stops is named when the window is closed", async () => {
const f = fakes();
const p = f.flow.signIn();
const wc = f.windows[0].webContents;
assert.strictEqual(redirect(wc, "https://www.instagram.com/x?next=1", true, "details"), false); // Meta's login goes here
assert.strictEqual(redirect(wc, "https://login.evil.example/cb?code=SECRET", true, "details"), true);
f.windows[0].close();
const r = await p;
assert.strictEqual(r.ok, false);
assert.match(r.error, /went to login\.evil\.example, which Frame Control doesn't allow/);
assert.doesNotMatch(r.error, /SECRET|code=/);
});
test("a stopped redirect is forgotten once the window loads a page again", async () => {
const f = fakes();
const p = f.flow.signIn();
const wc = f.windows[0].webContents;
assert.strictEqual(redirect(wc, "https://login.evil.example/cb", true, "details"), true);
wc.emit("did-navigate", {}, "https://auth.meta.com/login/"); // the user carried on
f.windows[0].close();
assert.deepStrictEqual(await p, { ok: false, cancelled: true, error: "The sign-in window was closed before you signed in." });
});
test("a page's beforeunload can't keep the window open", async () => {
const f = fakes();
const done = f.flow.signIn();
const w = f.windows[0];
let prevented = false;
w.webContents.emit("will-prevent-unload", { preventDefault: () => { prevented = true; } });
assert.ok(prevented);
// Even if close() would be vetoed, finishing destroys the window.
w.veto = true;
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
assert.strictEqual((await done).ok, true);
assert.ok(w.isDestroyed());
});
test("closing during a connect: the cancel goes to the server only after that connect settles", async () => {
for (const outcome of ["saved", "refused", "failed"]) {
const f = fakes();
let release, fail;
f.setAnswer(() => new Promise((r, j) => { release = r; fail = j; }));
const done = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
await settle();
assert.deepStrictEqual(f.calls, [TOKEN]);
f.windows[0].close(); // the user gives up while the server and Meta are answering
const r = await done; // the page hears at once
assert.strictEqual(r.ok, false);
await settle();
assert.deepStrictEqual(f.cancels, [], outcome); // not before the connect has settled
if (outcome === "saved") release({ status: 200, data: { connected: true } });
else if (outcome === "refused") release({ status: 400, data: { error: "no" } });
else fail(new Error("socket hang up"));
await settle();
// The same attempt id the connect carried, after it settled and after cookie intake stopped.
assert.deepStrictEqual(f.cancels, [f.attempts[0]], outcome);
assert.match(f.cancels[0], /^[A-Za-z0-9-]{8,64}$/);
assert.deepStrictEqual(f.log, ["connect", "destroyed", "cancel (cookie listeners: 0)"], outcome);
assert.deepStrictEqual(f.calls, [TOKEN]);
}
});
test("closing before any connect needs no cancel", async () => {
const f = fakes();
const done = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".evil.example", value: TOKEN }); // nothing usable
await settle();
f.windows[0].close();
assert.strictEqual((await done).ok, false);
await f.flow.signOut();
await settle();
assert.deepStrictEqual(f.calls, []);
assert.deepStrictEqual(f.cancels, []);
});
test("a timeout during a connect cancels after it settles", async () => {
const f = fakes({ timeout: 50 });
let release;
f.setAnswer(() => new Promise((r) => { release = r; })); // Meta is slow
const done = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
const r = await done;
assert.match(r.error, /timed out/);
assert.ok(f.windows[0].isDestroyed());
assert.deepStrictEqual(f.cancels, []);
release({ status: 400, data: { error: "no" } });
await settle();
assert.deepStrictEqual(f.cancels, [f.attempts[0]]);
});
test("disconnect during a connect waits for it, cancels it, then clears, before the page deletes the account", async () => {
const f = fakes();
let release;
f.setAnswer(() => new Promise((r) => { release = r; }));
const done = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
await settle();
f.log.length = 0;
let signedOut = false;
const out = f.flow.signOut().then(() => { signedOut = true; }); // the page's Disconnect, step one
assert.strictEqual((await done).ok, false);
await settle();
assert.strictEqual(signedOut, false); // still waiting for the connect
assert.deepStrictEqual(f.cancels, []);
release({ status: 200, data: { connected: true } }); // it saved, just now
await out;
// Only now would the page post its disconnect: the cancel (which deletes what that
// connect saved) has reached the server, and the session is cleared.
assert.deepStrictEqual(f.log, ["destroyed", "cancel (cookie listeners: 0)", "clear"]);
assert.deepStrictEqual(f.cancels, [f.attempts[0]]);
});
test("a later disconnect still waits for a closed window's connect", async () => {
const f = fakes();
let release;
f.setAnswer(() => new Promise((r) => { release = r; }));
const done = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
await settle();
f.windows[0].close();
await done;
let signedOut = false;
const out = f.flow.signOut().then(() => { signedOut = true; });
await settle();
assert.strictEqual(signedOut, false);
release({ status: 400, data: { error: "no" } });
await out;
assert.deepStrictEqual(f.cancels, [f.attempts[0]]);
});
test("nothing sent means nothing to cancel; sign-out destroys the window, cancels, then clears", async () => {
const f = fakes();
// Nothing sent: closing cancels nothing.
const a = f.flow.signIn();
f.windows[0].close();
await a;
assert.deepStrictEqual(f.cancels, []);
// Something sent, then signed out: the attempt is cancelled before the session is cleared.
f.setAnswer(() => ({ status: 400, data: { error: "Meta didn't accept this token." } }));
const b = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
await settle();
f.windows[1].veto = true; // and its page tries to stay open
f.log.length = 0;
await f.flow.signOut();
assert.strictEqual((await b).ok, false);
assert.ok(f.windows[1].isDestroyed());
assert.deepStrictEqual(f.log, ["destroyed", "cancel (cookie listeners: 0)", "clear"]);
assert.deepStrictEqual(f.cancels, [f.attempts[0]]);
// After sign-out a new cookie in the partition sends nothing.
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: OTHER });
await settle();
assert.deepStrictEqual(f.calls, [TOKEN]);
});
test("each sign-in has its own attempt id", async () => {
const f = fakes();
f.setAnswer(() => ({ status: 400, data: { error: "no" } }));
const a = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
await settle();
f.windows[0].close();
await a;
const b = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: OTHER });
await settle();
f.windows[1].close();
await b;
await settle();
assert.strictEqual(f.attempts.length, 2);
assert.notStrictEqual(f.attempts[0], f.attempts[1]);
assert.deepStrictEqual(f.cancels, f.attempts);
});
test("no sign-in can start while a sign-out waits on an earlier connect", async () => {
const f = fakes();
let release;
f.setAnswer(() => new Promise((r) => { release = r; }));
const first = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
await settle();
const out = f.flow.signOut(); // Disconnect pressed while that connect is under way
assert.strictEqual((await first).ok, false);
assert.ok(f.flow.isSigningOut());
// The page answered, so the user tries again at once: refused, no window, nothing sent.
const second = await f.flow.signIn();
assert.deepStrictEqual(second, { ok: false, error: "Frame Control is signing out of Meta; try again in a moment." });
assert.strictEqual(f.windows.length, 1);
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: OTHER });
await settle();
assert.deepStrictEqual(f.calls, [TOKEN]);
// A second sign-out joins the first.
assert.strictEqual(f.flow.signOut(), out);
release({ status: 200, data: { connected: true } });
await out;
assert.deepStrictEqual(f.cancels, [f.attempts[0]]);
assert.strictEqual(f.meta().cleared.filter(([k]) => k === "storage").length, 1);
assert.ok(!f.flow.isSigningOut());
// Once it is done, signing in works again.
f.setAnswer(() => ({ status: 200, data: { connected: true } }));
const third = f.flow.signIn();
assert.strictEqual(f.windows.length, 2);
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: OTHER });
assert.strictEqual((await third).ok, true);
});
test("sign-out goes round until no window and no connect or cancel is left", async () => {
const f = fakes();
let releaseA;
f.setAnswer(() => new Promise((r) => { releaseA = r; }));
const a = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: TOKEN });
await settle();
f.windows[0].close(); // window A closed; its cancel waits on its connect
await a;
// Window B is open, with its own connect under way, when sign-out starts.
let releaseB;
f.setAnswer(() => new Promise((r) => { releaseB = r; }));
const b = f.flow.signIn();
f.meta().cookies.set({ name: COOKIE, domain: ".oculus.com", value: OTHER });
await settle();
let finished = false;
const out = f.flow.signOut().then(() => { finished = true; });
await b;
releaseB({ status: 400, data: { error: "no" } });
await settle();
assert.strictEqual(finished, false); // A's connect is still under way
releaseA({ status: 200, data: { connected: true } });
await out;
assert.deepStrictEqual([...f.cancels].sort(), [...f.attempts].sort());
assert.strictEqual(f.attempts.length, 2);
assert.strictEqual(f.log[f.log.length - 1], "clear");
});
+82
View File
@@ -82,6 +82,88 @@ is an error rather than a reason to fall back to `index-v1.jar`. `entry.jar`
must be signed with SHA-2 (SHA-1 is still accepted for legacy `index-v1.jar`).
Removing a repository clears its state.
## My itch.io purchases
The store can also install the Android (Quest) builds of games you **own** on
itch.io: bought, claimed free, from a bundle or a gift. It is opt-in. In the
store, open the sources dialog and use **My itch.io purchases**: create an API
key on itch.io (account settings › API keys) and paste it there. Frame Control
checks it with itch.io (`GET /profile`) before saving it; a wrong or revoked key
is reported and nothing is saved. Once connected, a source called
**itch.io (my purchases)** appears, and its games are marked **Owned on
itch.io** instead of Free. **Disconnect itch.io** deletes the key file.
How it works, using the documented
[server-side API](https://itch.io/docs/api/serverside) with
`Authorization: Bearer <key>`:
- `GET https://api.itch.io/profile/owned-keys` lists your download keys, page by
page until an empty page (500 per page, at most 40 pages). Games whose
`traits` don't include `p_android` are left out; one entry per game, with the
newest key.
- Opening a game asks `GET /games/<game_id>/uploads?download_key_id=<key id>`
for its files. Android uploads are those marked `p_android` (in `traits`, or
as a `p_android`/`platforms.android` flag) or named `.apk`. `.apk` files come
first, then full games before demos, newest first. Soundtracks, pre-orders and
externally hosted files are skipped.
- Installing asks `GET /uploads/<upload_id>/download?download_key_id=<key id>`.
The answer accepted is a redirect to itch.io's storage host, a JSON body
with a `url` field naming it, or the file itself. Frame Control follows the
first two without the key, to HTTPS on port 443 only, and only to host names
whose every address is public. For IPv4 that means outside every special
block (`0/8`, `10/8`, `100.64/10`, `127/8`, `169.254/16`, `172.16/12`,
`192.0.0/24`, `192.0.2/24`, `192.88.99/24`, `192.168/16`, `198.18/15`,
`198.51.100/24`, `203.0.113/24`, `224/4`, `240/4`), however written. IPv6 is
an allow-list: global unicast (`2000::/3`) only, minus 6to4, Teredo,
documentation, ORCHID and benchmarking blocks, so mapped or NAT64 IPv4,
site-local, unique-local and the rest are all refused. It connects only to
the addresses it checked, trying each in turn within one 20-second budget
for connecting and the TLS handshake, and checks again on each further
redirect. A storage URL, first or after any redirect, that contains the API
key (or 8+ characters of it, in any case, plain or percent-encoded) or an
`api_key` parameter is refused before it is looked up, so the key never
reaches the storage server. The APK then takes
the normal install path, with the same ABI, OpenXR layer and Meta-services
checks and warnings as any other store download.
The key is stored in `frame_host.data_dir('itch-account.json')`
(`~/Library/Application Support/Frame Control/itch-account.json` on macOS),
written through a new 0600 temporary file and moved into place. It is only
sent to `https://api.itch.io`, in the `Authorization` header, never in a URL;
Frame Control does not follow api.itch.io's redirects with it. The key, or any
part of it from 8 characters, never appears in the page, API responses, errors
or logs; the page clears the field as soon as it is sent. A disconnect made
while a key is still being checked wins: that key is not saved.
Owned-game lists are kept in memory for ten minutes and never written to the
disk cache.
When api.itch.io answers 401 or 403, Frame Control asks `GET /profile` with the
same key. If that is refused too (or itch.io says "invalid key"), the key is
deleted and the store says to connect again; otherwise only that game or
download is refused, and the key stays. Rate limits (429) and network errors
only fail that source's results, with a readable message. A storage host's
refusal is reported as a download error, not as a rate limit.
### What is verified and what isn't
Not tested against a live itch.io account at all: everything below was tested
with recorded and hand-made responses (`tests/test_itch_purchases.py`) only.
| Part | Status |
|---|---|
| Bearer key, `GET /profile`, `/profile/owned-keys` paging and its fields, the `errors` array | Documented in the public reference (checked 2026-10-13) |
| `GET /games/<id>/uploads` and `download_key_id` | Not in the public reference; paths and parameter from itch.io's app client, [go-itchio](https://github.com/itchio/go-itchio) |
| Upload fields (`filename`, `display_name`, `size`, `storage`, `type`, `traits`, `updated_at`, `demo`, `preorder`) | Assumed from go-itchio and the API's snake_case convention |
| The `p_android` trait, on games and uploads | Assumed from itch.io's `p_windows`/`p_linux`/`p_osx` naming |
| What `/uploads/<id>/download` returns | Unknown: go-itchio only builds the URL and hands it to a downloader, so a redirect, a JSON `url` and the file are all accepted |
| Whether that endpoint accepts the key as a Bearer header | Unknown: go-itchio puts it in the query string, which Frame Control refuses to do |
| itch.io's storage host names and addresses | Unknown; any public HTTPS host is allowed |
Limits: a game whose page doesn't mark Android, or whose Android build is
inside a `.zip`, won't install. Android builds have no version codes on
itch.io, so only the newest upload can be installed. Owning a game doesn't make
it run on the Frame: Quest builds that need Meta's platform services won't.
## Publish your own repository
Only publish free APKs you own or have the developer's permission to distribute.
+2
View File
@@ -60,6 +60,8 @@ does not suppress the other at the aggregator level. Queries filter the current
feed window locally: this is not an exhaustive historical itch search. Only
explicit zero-price Android entries are returned. Covers are exposed in
`images`; absent screenshots, APK version, ABI and minimum SDK stay unknown.
Games the user owns on itch.io are a separate, opt-in source that downloads
through their own API key: see [My itch.io purchases](apk-repos.md#my-itchio-purchases).
Curated GitHub entries include publisher artwork and plain-language summaries.
Repository image URLs are pinned to inspected commits. Open Brush screenshots
come from its README-linked Steam listing; SuperTux uses the upstream gameplay
-6
View File
@@ -39,12 +39,6 @@ section below), instead of going into Lepton Development:
run at once alongside Lepton Development, each in its own container
(`lepton-steamlaunch-<id>`, ADB on 5556, 5557, …).
Split APKs (a base plus `split_*.apk` files) install as one app:
`python3 ui/frame_android.py install base.apk split1.apk …`. Lepton runs one
APK, so splits that add native libraries or assets are merged into the base
and re-signed; density and language splits are refused. See
[quest-import.md](quest-import.md#split-apks). Not yet tried on a real Frame.
Verified with AntennaPod and Tabletop Tools: installed in about 7 s, launched
from the shortcut, stopped, and relaunched with their data intact. ADB and
Lepton Development aren't involved.
-367
View File
@@ -1,367 +0,0 @@
# My Meta library (Experimental)
Install Quest games your Meta account owns straight from Meta, test whether
they run on the Steam Frame, and add the result to the shared compatibility
reports. It is off until you connect, and it lives in **Android → My Meta
library**.
**Read this first.** My Meta library uses Meta's undocumented internal API
with your own Meta access token, as the open-source tools QAVS, DownQuest and
BSManager do. It may stop working whenever Meta changes things, and using it
may be against Meta's terms. Your account is your responsibility. Frame
Control is unofficial and not made or endorsed by Meta or Valve.
What it will and won't do:
- It lists and downloads only games your account owns (its active
entitlements). It never downloads anything else.
- It installs games as Meta ships them. There is **no licence-check bypass**:
no patching, no stub libraries, no OVRPort or FramePort. A game that
enforces Meta's licence check may quit or hang on the Frame, because that
check needs Meta's services, which the Frame doesn't have.
- Game files go only to a cache folder on your computer and then to your
Frame. They are deleted from the cache after each install, and never
uploaded or shared. Reports carry results only (below), never files or your
token.
## Your token
Frame Control never reads your browser's cookies. There are two ways to give
it your token.
**Sign in with Meta** (desktop app only; a plain browser doesn't show it).
Tick **I understand…**, then choose **Sign in with Meta…**. Meta's own login
page (`https://secure.oculus.com`, the address BSManager's "Log in with Meta"
opens) appears in a Frame Control window. Sign in there as usual. When Meta's
site sets its access token cookie (`oc_ac_at`), Frame Control takes it from
that window, connects with it exactly as if you had pasted it (below), and
closes the window. The window:
- has its own storage (the Electron session `persist:meta-signin`), separate
from Frame Control's main window and from every browser on your computer;
the cookie is read only from there, and only for `oculus.com`;
- runs Meta's page sandboxed, with no access to Frame Control (no preload, no
Node, context isolation on, no developer tools) and every permission request
or check (camera, notifications…) refused;
- only goes to `https://` pages on `oculus.com`, `meta.com`, `facebook.com` and
`instagram.com` (and their subdomains; Meta's login passes through Instagram).
A link anywhere else opens in your system browser; a redirect anywhere else
is stopped, and closing the window then says where it went. Pop-ups on those sites open in the same
window. Frames inside the page may load any `https://` page (Meta's login
can embed captcha and CDN frames from other sites, such as Arkose Labs,
hCaptcha, reCAPTCHA or fbcdn) or an empty frame (`about:blank`,
`about:srcdoc`); a frame going to any other scheme (`http:`, `data:`,
`blob:`, `javascript:`, `file:`…) is just stopped. Redirects follow the same
rule for the frame they happen in. Frames can't reach the
token: the cookie is read by Frame Control's main process, and the window
has no IPC;
- sends a cookie value on only if it looks like a user token (`OC` or `FRL`,
mixed case, no `%`, `|` or `:`), and each value once. If the server refuses
it, the window stays open; close it to see why;
- gives up after 10 minutes, as BSManager does;
- when it is closed, times out or is signed out of, stops reading cookies
first, then is destroyed (a page's "leave this page?" can't keep it open),
and, if it had sent a token, tells the server to drop that sign-in: each
sign-in has a random attempt id. The cancel is sent only once the connect
under way has finished (saved, refused or failed), so the server always
gets it after the connect; with nothing sent, there is nothing to cancel.
The server lists an attempt only while its
connect is being checked: cancelled then, it isn't saved; cancelled just
after its token was saved (even before the connect has finished tidying
up), that token is deleted (as a disconnect). A cancel
never removes an account a later sign-in or a pasted token saved, and one
for an unknown attempt changes nothing.
The page never sees the token: the desktop app hands it straight to the local
server's connect path and tells the page only the answer (connected, and your
Meta display name). It isn't logged anywhere.
The window keeps you signed in to Meta (in `persist:meta-signin` only) until
you **Disconnect**, which also signs out of Meta in Frame Control: it clears
that window's cookies, storage and cache, so the next sign-in starts fresh.
**Sign out of Meta in Frame Control** next to the sign-in button does just
that part, for a sign-in that didn't connect.
**Paste a token.** You copy your own token and paste it in. The two ways
BSManager's guide describes
([How to obtain your Oculus Token](https://github.com/Zagrios/bs-manager/wiki/How-to-obtain-your-Oculus-Token)):
1. **From the Meta website.** Sign in at `https://secure.oculus.com`. Open the
browser's developer tools, then **Application** (Chrome, Edge) or
**Storage** (Firefox) → **Cookies** → `https://secure.oculus.com`. Copy the
value of `oc_ac_at`. It starts with `OC`.
2. **From the Meta Quest PC app** (Windows). Press **Ctrl+Shift+I**, open
**Network**, press **Ctrl+R**, type `graph` in the filter, open the first
request, and copy the access token from its **Payload**. It starts with
`FRL`.
Treat it like a password: anyone with it can act as your Meta account. Signing
out of the Meta website or app usually ends it; then connect again with a new
one.
Frame Control refuses values that can't be a user token before sending them
anywhere: anything not starting with `OC` or `FRL`, URL-encoded copies (with
`%`), app tokens (with `|` or `:`), cut-off values (under 20 characters), and
anything with characters other than letters, digits and `+ / = - _` (the
base64 and URL-safe base64 sets), or with no lower-case letter (real tokens are
mixed case). The public tools only describe tokens made
of letters and digits; if a real token needs another character, it has to be
added in one place (`TOKEN_CHARS` in `ui/frame_meta.py`), which also makes
the redaction look for it.
## Using it
1. Tick **I understand…**, then either choose **Sign in with Meta…** (desktop
app) or paste your token and choose **Connect**. Frame Control asks Meta
"who am I" with it (a read-only query); only a token Meta accepts is kept.
2. Your Quest games are listed with their version, size, game data, and the
community verdict from the compatibility database when there is one.
3. **Install** downloads the game and installs it on the Frame, as [Import
from my Quest](quest-import.md) does. A game with OBB game data then needs
a second **Install** while you have it open on the Frame: Frame Control
downloads the data, copies it in and closes the game.
4. **Install and test** does the same, then starts the game itself (twice if
it has game data to add: once to add it, once for the test), watches it for
about 30 seconds, and records what happened. **Test again** repeats only
the test.
5. **Disconnect** deletes the token from your computer and, in the desktop
app, signs out of Meta in Frame Control (clears the sign-in window's
storage). Games already on the Frame stay.
From a terminal (the token is read from standard input, never an argument):
```sh
python3 ui/frame_meta.py connect --token-stdin # paste the token, press Enter
python3 ui/frame_meta.py status
python3 ui/frame_meta.py list [--refresh] # owned Quest games, with community verdicts
python3 ui/frame_meta.py download APP_ID # into the local install cache (a later download clears it after 6 hours)
python3 ui/frame_meta.py install APP_ID [--test] [--allow-downgrade]
python3 ui/frame_meta.py test com.example.game [--wait 30] [--no-report]
python3 ui/frame_meta.py disconnect
```
The local API: `GET /api/meta` (connected or not, never the token),
`GET /api/meta?library=1[&refresh=1]` (the games), and `POST /api/meta` with
`{"action": "connect", "token": "...", "acknowledged": true}`,
`{"action": "disconnect"}`, `{"action": "cancel-signin", "attempt": "..."}`, `{"action": "install", "app": "APP_ID", "test": false,
"allowDowngrade": false}` or `{"action": "test", "package": "..."}`. Install and
test return a background job (`/api/job`). Sign in with Meta uses the same
`connect` action: the desktop app's main process (`app/meta-signin.js`) posts
it with the cookie's value and an `"attempt"` id (8 to 64 letters, digits or
`-`) once the page's acknowledgement box is ticked. A connect whose attempt
was cancelled while it was checked answers 409 and saves nothing;
`cancel-signin` for an attempt whose token is the saved account deletes it.
## What an install does
1. Checks the game is in your library (by its Meta app id), and that there's
room on this computer for the APK, a patched copy of it and the game data.
2. Downloads the APK from Meta's download server, within the size Meta
declared (and at most 2 GB), checking its SHA-256 if Meta gives one, and
that it is an APK of the package and version Meta said.
3. Runs Import from my Quest's checks. Refused: 32-bit-only builds (no
`arm64-v8a`), and apps that need an Android newer than Lepton's (API 30).
Installed with a warning: games that use Meta's licence check
(`libovrplatformloader.so`), and VrApi-only games ("may not run").
4. Installs it with `frame_android.install` (VR detection, the launcher fix,
the OpenXR compatibility layer, artwork, a Steam library entry). An older
copy never silently replaces a newer one on the Frame unless you tick
**Allow replacing a newer version**.
5. Game data: the app's OBB (saved as Meta names it when that is a standard
`main|patch.<version>.<package>.obb` name, otherwise as
`main.<version code>.<package>.obb`) and any *required* asset files with
standard OBB names, at most 8 GB each. One install downloads at most 16 GB
in all, APK included, counted as the bytes arrive (so files whose size Meta
doesn't give count too); a file that goes past it is stopped and removed.
The data is copied into the
running game by the same two-step helper Import from my Quest uses. Other
asset files (DLC packs, `.pak` files a game fetches through Meta's
Platform SDK) aren't downloaded; the result lists them.
6. Deletes everything it downloaded, whether the install worked or not.
Downloads left by a crash are removed after six hours.
## The test, and the report
The test starts the game, waits for its Lepton instance, then looks every 10
seconds for 30 seconds: is the game's process alive, and what does its Android
log (main, system and crash buffers) say. It recognises:
| In the log | Cause given |
|---|---|
| "entitlement check failed", "not entitled", `ovr_Entitlement…` or `IsUserEntitledToApplication` reporting a failure (a call that passes doesn't count) | Meta's licence check failed |
| Platform SDK initialisation errors | Meta's Platform SDK couldn't start |
| `Failed to launch SystemActivities`, "device not supported" | The game checks it is on a Meta Quest |
| `vrapi_… was called before…`, VrApi load errors | Legacy VrApi |
| `XR_ERROR_EXTENSION_NOT_PRESENT` (with the extension's name) | An OpenXR feature the Frame lacks |
| `XR_ERROR_API_VERSION_UNSUPPORTED`, runtime/initialisation failures | OpenXR version or session problems |
| `VK_ERROR_DEVICE_LOST`, `zink`, `libgallium_dri.so`, `GL_INVALID_…`, shader failures | Graphics driver errors |
| `ClipboardManager` | Lepton's missing clipboard service |
| `NoSuchMethodError`, `UnsatisfiedLinkError`, `dlopen failed` | Android or library features Lepton lacks |
| `FATAL EXCEPTION`, `Fatal signal`, `SIGSEGV` | A crash |
| `ANR in …`, "Input dispatching timed out" | Not responding |
| `XR_SESSION_STATE_SYNCHRONIZED/VISIBLE/FOCUSED`, `xrEndFrame` | The VR scene started |
The outcome is one of **runs** or **runs with issues** (its VR scene
started), **quits** (closed by itself), **crashes**, **hangs** (Android
reported it not responding; Frame Control then closes it, and says so if it
couldn't), **didn't start** (its instance never came up), or **no clear
result**: it stayed open but its log never showed its VR scene starting. That
can be a game still loading, or one stuck after a failed check; Frame Control
can't tell, so it doesn't record it. A game already open is closed first, so
the test starts clean; if it can't be closed, nothing is tested. If the Frame
fails while the test reads the game or its log, the test says only that it
couldn't read the log: a failed command's output can be the game's log, so it
is never shown. Signatures look at the first 4 KB of each log line and are
bounded, so a huge or hostile log can't make the test slow. They
are written for Frame Control from the research in [vr-apks.md](vr-apks.md);
they are not copied from other tools.
The result goes through the same path as **Test** and **Report**
(`frame_catalog.add_report`): `via` probe, source `meta-store`, the result
(`runs`, `crashes` or `instance_failed`, the values the database accepts), and
notes with the outcome, the cause, the names of the matched signals and the
version code, all fixed texts. Nothing else: no token, account name, file or
log text. The page may show a short name from the log next to the cause (an
OpenXR extension, a library, an exception class or a signal), taken from the
log line after the token is removed from it, and left out unless it looks like
a real name (64 characters at most, no long mixed letter-and-digit runs);
reports never carry it. It is always kept on this
computer; it reaches the shared database only if **Share compatibility
results** is on in Privacy settings, as for every report. The panel offers to
turn that on.
## Your token and privacy
- Kept in `meta-account.json` in Frame Control's per-user data folder
(`~/Library/Application Support/Frame Control` on macOS,
`%APPDATA%\Frame Control` on Windows, `~/.local/share/frame-control` on
Linux), written owner-only (0600) through a temporary file. Never on the
Frame.
- Sent only to `graph.oculus.com` (in the POST body) and
`securecdn.oculus.com` (in the download address, which is how that server
takes it), over HTTPS. A download that redirects elsewhere (Meta's CDN) is
followed only to a plain HTTPS address on port 443 that doesn't carry the
token in any form, anywhere (host, path, query or fragment; literal,
recased, percent-encoded, or 8+ characters of it), nor an `access_token`
parameter. That is checked before any lookup or connection, and a redirect
that fails it is refused. API answers that redirect are refused.
- Every connection goes to an address checked to be public (an allow-list:
no private, local, shared, reserved, multicast, NAT64, 6to4 or Teredo
addresses), and to exactly that address, with TLS checked for the host
name, so a second DNS answer can't swap in another. System proxy settings
aren't used. This is the same design as the itch.io purchases branch.
- Never in logs, error messages, API answers, the page or reports: every
message is cleaned first, matching the token (or any start of it of 8+
characters) in any letter case, with each character written literally or
percent-encoded once or twice (%XX, %25XX) in either case, and any other
token-shaped value, before it is shortened.
- Disconnect deletes the file and the library Frame Control holds in memory,
and stops a download under way. A connect still being checked when you
disconnect is not saved. In the desktop app, before the token is deleted, it
also stops any sign-in under way, destroys the sign-in window, waits for a
connect still under way to finish and cancels its attempt (deleting its
token if it was just saved), and only then clears the `persist:meta-signin` session
(cookies, storage, cache); no other session is touched. Sign in with Meta
stays off from the moment you press Disconnect until the token is deleted,
and the app refuses a sign-in while it is signing out.
- With Sign in with Meta, a second copy of your Meta sign-in lives in that
session's storage, in Electron's per-user data folder (`Partitions/meta-signin`),
until you disconnect or sign out of Meta in Frame Control.
## Where the protocol comes from
Frame Control's code is its own (MIT). These public sources describe the
protocol; Frame Control takes only facts from them (addresses, query ids,
field names), not code. QAVS and BSManager are GPL-3.0; OculusGraphQLApiLib,
DownQuest and Oculus-downgrader state no licence.
| Fact | Source |
|---|---|
| GraphQL at `https://graph.oculus.com/graphql`, POST, form fields `access_token`, `doc_id`, `variables`; `OAuthException` / code 190 for a bad token | [OculusGraphQLApiLib `GraphQLClient.cs`](https://github.com/ComputerElite/OculusGraphQLApiLib/blob/866081e2/OculusGraphQLApiLib/GraphQL/GraphQLClient.cs), [DownQuest `DownQuest.js`](https://github.com/ptrpaws/DownQuest/blob/50f5d558/DownQuest.js) |
| "Who am I": `doc_id` 4149322231793299, `data.viewer.user {id, alias}` | OculusGraphQLApiLib `GraphQLClient.cs` and `Results/User.cs` |
| Owned apps: `doc_id` 4850747515044496, `data.viewer.user.active_entitlements.nodes[].item` | OculusGraphQLApiLib `GraphQLClient.cs`, `Results/Entitlement.cs`, `Results/Application.cs`; used by [QAVS `GameDownloadManager.cs`](https://github.com/ComputerElite/QuestAppVersionSwitcher/blob/36bebd5a/QuestAppVersionSwitcher/GameDownloadManager.cs) |
| One binary: `doc_id` 4734929166632773 with `binaryID`; fields `package_name`, `version`, `version_code`, `size`, `obb_binary`, `asset_files` (`file_name`, `size`, `is_required`) | OculusGraphQLApiLib `GraphQLClient.cs`, `Results/OculusBinary.cs`, `Results/AssetFile.cs` |
| Downloads: `https://securecdn.oculus.com/binaries/download/?id=<id>&access_token=<token>`, the same for the APK, `obb_binary` and asset files; QAVS takes the OBB and required asset files | OculusGraphQLApiLib `Game/GameDownloader.cs`, [QAVS `DownloadManager.cs`](https://github.com/ComputerElite/QuestAppVersionSwitcher/blob/36bebd5a/QuestAppVersionSwitcher/DownloadManager.cs), DownQuest `DownQuest.js` |
| Token checks (`OC`/`FRL`, no `%`, `|`, `:`, not `OC` + 15 digits) | OculusGraphQLApiLib `GraphQL/TokenTools.cs`, [BSManager `oculus.helpers.ts`](https://github.com/Zagrios/bs-manager/blob/00380c79/src/shared/helpers/oculus.helpers.ts) |
| How users get a token (`oc_ac_at` cookie, or the PC app's `FRL` token) | [BSManager wiki](https://github.com/Zagrios/bs-manager/wiki/How-to-obtain-your-Oculus-Token) |
| Sign in with Meta: open `https://secure.oculus.com` in an app window, and once it reaches `https://secure.oculus.com/my/profile` read the `oc_ac_at` cookie from that window's session (checked on `did-navigate`, `did-navigate-in-page`, `did-stop-loading`); 10-minute timeout | [BSManager `bs-oculus-downloader.service.ts`](https://github.com/Zagrios/bs-manager/blob/00380c79/src/main/services/bs-version-download/bs-oculus-downloader.service.ts) (`getUserTokenFromMetaAuth`) |
**Assumed, not seen in those sources:** that the entitlement and binary
answers carry the fields above in the shapes the libraries' models describe
(the stored queries are opaque); that the list isn't paginated (none of the
tools page it); that `ANDROID_6DOF` marks Quest apps; where downloads redirect
to (the tools let their HTTP client follow redirects; Frame Control follows
up to five, to any public HTTPS host, and refuses one carrying the token); and
a `sha256`
field (none was seen, so in practice only sizes are checked). The log
signatures for the test are inferred from the compatibility research, not
from a real Frame.
## Evidence and limits
**Verified locally** with a fake Meta and a fake Frame (`tests/test_meta.py`):
token checks; storage owner-only through a temporary file; redaction (the
stored token recased and percent-encoded in either case, starts of it, other
token-shaped values, and redaction before cutting); HTTPS-only URLs; the
public-address allow-list, resolving once and connecting only to a checked
address within one deadline; redirects off Meta's hosts refused when they
carry the token anywhere, before any lookup; the token never sent to the
CDN; GraphQL redirects refused; disconnect beating a connect in flight and
stopping a download; library parsing (Quest apps only, nothing expired or
inactive, details from the binary query); downloads with an OBB; size, length
and checksum limits and the whole-install budget with nothing left behind;
refusing apps not in the
library; the handoff to Import from my Quest's checks, install and data step;
the licence-check warning instead of a refusal; test outcomes from log
fixtures (a passing licence check isn't a failure; no VR scene is no clear
result, not recorded); closing before a test and saying when a close failed;
reports carrying only fixed texts, no token, log text or file names. The
real TLS connection code is the same as the itch.io branch's and is exercised
here only with a fake connection.
**Sign in with Meta, verified locally** with fake Electron windows, sessions
and cookies (`app/test/meta-signin.test.js`): the window's options (its own
partition, sandbox, context isolation, no Node, no preload) and Meta's login
address, no developer tools; permission requests and checks refused; the
navigation allow-list (links elsewhere open in the system browser, redirects
elsewhere are stopped, pop-ups never become windows); frames allowed any
HTTPS page but no other scheme, the window itself only Meta's sites; a
page's beforeunload not keeping the window open; the cookie read only from
`persist:meta-signin` and only for `oculus.com`, even when the default
session and another partition hold one; signed-out values ignored; a value
sent once, a refused one keeping the window open; the window closing after a
connect, with the page getting the answer and never the token; nothing
logged, and a server error that echoed the token scrubbed; closing the window
or signing out ending the wait; closing, timing out or signing out during a
connect cancelling that attempt (with its own id) after cookie intake stops
and only after that connect settles (saved, refused or failed), and no cancel
when nothing was sent; sign-out (Disconnect's first step) waiting for a
connect under way, even from a window closed earlier, before it finishes;
sign-out destroying the window before clearing only that partition; no
sign-in starting while a sign-out runs (it is refused: "try again in a
moment"), a second sign-out joining the first, and a sign-out waiting for
every window and connect started before it. The
frame redirects held to the frame rule, the window's to Meta's sites. The
server side (`tests/test_meta.py`): an attempt cancelled while Meta is
answering not saved, even after 300 other cancels; a cancel just after the
save deleting that token; a cancel never removing a later sign-in's or a
pasted token's account; unknown attempts changing nothing; no attempt left
listed after a connect, refused or not; attempt ids checked.
**Differences from BSManager:** BSManager uses its app's default session (and
clears only `.oculus.com` cookies afterwards); Frame Control uses a session of
its own and clears all of it on disconnect. BSManager waits for
`/my/profile`; Frame Control also reacts to the cookie changing, and to any
`oculus.com` page, and lets the server decide whether the value is a
working token.
**Not verified:** Sign in with Meta against Meta's real login page (no real
Meta account was used, and nothing was sent to Meta): whether Meta's login
still goes only through `oculus.com`, `meta.com` and `facebook.com`, whether
it uses pop-ups (they open in the same window here, which would break a
pop-up that reports back to its opener), whether `oc_ac_at` is still set on
`oculus.com` after login, and whether Meta accepts the window (Frame Control
removes Electron's name from the user agent as a precaution). A real Meta token and Meta's real answers (none was
available; nothing was sent to Meta), real downloads and their redirects,
real Quest games installed or tested on a real Frame, and whether the log
signatures match what real games print in Lepton.
+9 -26
View File
@@ -33,12 +33,6 @@ computer, exactly as they were sent.
yours.
- Every event includes the app version, OS name (macOS, Windows or Linux),
CPU architecture and Python version.
- `frame_connected` carries `headset_id`: the first 16 hex digits of a
SHA-256 hash of the headset's `/etc/machine-id`, worked out on the headset,
so the machine-id itself never leaves it. It is the same for one headset on
every computer, which lets us count headsets rather than installs; it can't
be turned back into the machine-id or a serial number. `headsets_saved` is how
many headsets this copy of Frame Control has set up.
## Usage events
@@ -47,7 +41,7 @@ computer, exactly as they were sent.
| `app_installed` | First run | |
| `app_updated` | First run of a new version | `from_version` |
| `app_opened` | At most once a day | |
| `frame_connected` | The first time a SteamOS build is seen on each headset | `steamos_build`, `steamos_version`, `headset_id`, `headsets_saved` |
| `frame_connected` | The first time a SteamOS build is seen | `steamos_build`, `steamos_version` |
| `tab_viewed` | The first click on each tab in a session | `tab` |
| `install_finished` | Any install finishes, working or not | `kind` (apk, flatpak, steam, title, web), `ok`, `seconds`, `error_category` (only when `ok` is false), `installer_code`, `xr_layer_missing`, and see below |
| `update_offered`, `update_started`, `update_failed` | The update banner | `to_version`, `error_category` |
@@ -57,15 +51,12 @@ error becomes one category from this fixed list, plus Android's own
`INSTALL_FAILED_…` code (`installer_code`) when there is one:
`android_installer`, `apk_needs_newer_android`, `apk_wrong_abi`,
`layer_missing`, `apk_repack_failed`, `tool_missing`, `apk_unreadable`,
`cant_run_on_frame`, `steam_shortcut`, `steam_not_running`,
`steam_not_responding`, `steam_js_error`, `artwork_failed`, `copy_timeout`,
`frame_not_set_up`, `frame_auth`, `frame_unreachable`, `frame_disk_full`,
`copy_failed`, `download_failed`, `flatpak`, `cancelled`, `lepton`,
`internal_error` (a mistake in Frame Control's own code) or `other`. A VR
APK that installed without Frame Control's OpenXR compatibility layer,
because this copy of the app is missing it, carries `xr_layer_missing: true`;
otherwise that field is left out. It names what was installed only when
that's already public:
`cant_run_on_frame`, `steam_shortcut`, `frame_not_set_up`, `frame_auth`,
`frame_unreachable`, `frame_disk_full`, `download_failed`, `flatpak`,
`cancelled`, `lepton` or `other`. A VR APK that installed without Frame
Control's OpenXR compatibility layer, because this copy of the app is missing
it, carries `xr_layer_missing: true`; otherwise that field is left out. It
names what was installed only when that's already public:
- F-Droid catalogue apps: `package`. Never the version, since a local build can reuse a
catalogue app's package name
@@ -83,21 +74,13 @@ SteamOS and Lepton builds. Before sending:
- the notes, app name and version are scrubbed like error messages (see
below)
- the APK's source is kept only if it's `F-Droid`, `meta-store` (a test
from [My Meta library](meta-library.md)) or the public host name of a
download link (`https://example.com/…`). File names, user names,
- the APK's source is kept only if it's `F-Droid` or the public host name of
a download link (`https://example.com/…`). File names, user names,
passwords, ports, paths, IP addresses and local host names are dropped
When you turn this on, reports you made earlier on this computer are shared
too.
My Meta library (Experimental) keeps your Meta access token in
`meta-account.json` in this data folder, owner-only, and sends it only to
Meta's API and download hosts. It is never in analytics, reports, error
details or logs; its test reports carry the package, version, version code,
title, result and the reason, nothing from your account. See
[meta-library.md](meta-library.md#your-token-and-privacy).
The maintainer's `python3 ui/frame_compat_db.py sync` copies these events
into the compatibility database, marked `via=community…`. It takes at most
30 per reporter per day.
-208
View File
@@ -1,208 +0,0 @@
# Import from my Quest
Move games you own from your own Meta Quest onto the Steam Frame. Plug the
Quest into your computer by USB; Frame Control copies the game off it with
adb, checks it can run in Lepton, and installs it on the Frame like any other
APK, with its own Lepton instance and Steam library entry
([apks.md](apks.md), [vr-apks.md](vr-apks.md)).
There is no licence-check bypass, and there won't be. A game that uses Meta's
licence check installs with a warning: on the Frame that check can't reach
Meta, so the game may quit, hang, or run without online features. See
[What won't run](#what-wont-run-and-why) below. To install games you own
straight from Meta instead, without a Quest, see [My Meta library
(Experimental)](meta-library.md).
## Before you start: developer mode on the Quest
adb only reaches a Quest with developer mode on:
1. Make a free Meta developer organisation at
[developers.meta.com](https://developers.meta.com/horizon/) on a **verified**
Meta account (Meta asks for a phone number or payment card to verify).
2. In the Meta Horizon phone app, open the headset's settings and switch
**Developer Mode** on. Restart the Quest.
3. Plug the Quest into the computer that runs Frame Control with a USB cable
that carries data (some charging cables don't).
4. Put the headset on. Accept **Allow USB debugging**, and tick **Always allow
from this computer**.
Frame Control uses the adb it already uses for the Frame (an adb you
installed, else the one it bundles). On Linux, adb also needs permission to
open the Quest: add a udev rule for Meta's USB vendor id `2833`, for example
`SUBSYSTEM=="usb", ATTR{idVendor}=="2833", MODE="0660", GROUP="plugdev"`, and
be in that group. Then unplug and replug.
The Frame's own adb connections (Frame Control's SSH tunnels to Lepton,
`127.0.0.1:<port>`) are never touched. Only a device that reports itself as
Oculus or Meta, or a Quest model, counts as a Quest.
## Using it
In the app: **Android → On your Frame → Import from my Quest**. Choose **Look
for my Quest**, then **Import** next to a game. Tick **Also copy each game's
files in Android/data** to bring the game's external files too (some games
keep saves there). Progress shows in the panel; the result and any reason it
stopped appear in the message log.
A game with game data (an OBB folder, or Android/data when ticked) takes two
imports, because Android can only take that data while the game's Lepton
instance is running, and there is no way to start the instance without the
game. Frame Control never starts the game itself:
1. **Import** installs the game and says it has data to add.
2. Start the game from the Steam library and leave it on its first screen.
3. **Import** again. Frame Control sees the game open with the same version,
copies the data in, and closes the game. Start it again to play.
A game that quits straight away without its data can't take it this way.
From a terminal:
```sh
python3 ui/frame_quest.py devices # what adb sees, and which is the Quest
python3 ui/frame_quest.py list # apps on the Quest: split or not, OBB and data sizes
python3 ui/frame_quest.py import com.example.game # copy one to the Frame
python3 ui/frame_quest.py import com.example.game --data --serial 1WMHH000XXXXXX
```
The API is `GET /api/quest` (the Quest, or guidance when there isn't one, and
its apps) and `POST /api/quest` with
`{"action": "import", "package": "...", "serial": "...", "data": false}`,
which returns a background job (`/api/job`) with a stage and percentage.
## What an import does
1. Finds the Quest (`adb devices -l`, then `ro.product.manufacturer` and
`ro.product.model`). If none is ready it says why: nothing connected, not
authorised yet (the **Allow USB debugging** prompt), no USB permission
(Linux), offline, another Android device, or several Quests.
2. Lists third-party apps (`pm list packages -3`), each with `pm path` (one
line is a single APK, several a split APK) and the size of
`/sdcard/Android/obb/<package>` and `/sdcard/Android/data/<package>`.
Labels aren't shown before import: reading them means copying the APK.
3. Checks there is room on the computer (the APKs, a merged and a patched copy
of them counted uncompressed, the OBB folder and the Android/data folder
once, since that is streamed to the Frame), then copies the APK and its splits
into a per-user cache folder (`Frame Control/quest-import` under the
system's cache folder).
4. **Checks the APK before anything else is copied**, refusing:
- 32-bit-only builds (native code but no `arm64-v8a`);
- apps that need an Android newer than Lepton's (API 30);
- split APKs Frame Control can't merge (below).
Installed with a warning:
- games that use Meta's licence check (Meta's Platform SDK,
`libovrplatformloader.so`, in the base or any split): "uses Meta's
licence check; it may quit, hang, or run without online features on the
Frame. Frame Control won't remove the check." Linking the SDK doesn't
mean a game enforces the check: Meta's store requirement for it
(VRC.Quest.Security.1) has only been *Recommended* since 2021, and what
a game does when the check fails is up to the game;
- VrApi-only builds (`libvrapi.so`, no OpenXR loader), flagged as for any
APK: the Frame has no VrApi, so they may not run.
5. If the game isn't open on the Frame: installs it through the normal APK
install (`frame_android.install`: VR detection, the launcher fix, the
OpenXR compatibility layer, artwork), and says whether it has game data
to add. An older Quest copy never replaces a newer version on the Frame
(compared by version code, which installs now record; for installs made
before that, it is read from the installed APK on the Frame) unless you
tick **Allow replacing a newer version** (`--allow-downgrade`,
`"allowDowngrade": true`). If the installed version can't be read at all,
that permission is needed too.
6. If the game is open on the Frame (the second import): refuses unless
Frame Control's install record is for this package, in this package's own
instance, at the Quest's exact version code (an open game is never
replaced), otherwise copies the OBB folder (and `Android/data` if ticked)
from the Quest, waits for Android to have finished booting, copies standard
`main|patch.<version>.<package>.obb` files with the existing OBB helper
(checked by SHA-256 inside Android, [sidequest.md](sidequest.md#obb-files))
once Android's own `dumpsys package` in that instance shows the same
package at the same version code,
streams the `Android/data` files in as a tar written straight into the SSH
transfer, then closes the game so its next start reads them, and checks
its instance has stopped; if it hasn't, or the check itself fails (a
failed `podman ps` never reads as "stopped"), the result says so. The game
is closed whatever happens during the copy, including a local read error
or Ctrl-C, and a failed close is reported without hiding the original
error. Everything
from the second check of the install record (after the pulls, which can
be long) to closing the game holds the same lock as installs and removals.
Other files in the OBB folder are listed as not copied.
7. Deletes the cache folder, whether the import worked or not. A folder left
by a crash is removed by a later import after six hours.
Only one import runs at a time. If copying game data fails, the game is
closed and the message says so; start it and import again.
### Split APKs
Lepton runs one APK per instance, so the shared install path now merges a
split APK into one (`frame_android.install(base, splits=[...])`, or
`python3 ui/frame_android.py install base.apk split1.apk ...`). Merging copies
each split's native libraries and assets into the base, drops the base
manifest's `isSplitRequired` flag, and re-signs with Frame Control's key, as
the launcher fix already does. Only splits that hold nothing but
`lib/<abi>/*.so`, `assets/` and their own signature, manifest and an empty
resource table are merged. Refused, with the reason, before anything reaches
the Frame: splits with resources (screen density or language), code
(`classes*.dex`), any other file, a manifest that declares components or
libraries (`<service>`, `<uses-library>`...), feature splits, and splits of
another package. The merged files are held in memory while the APK is
repacked and signed (as for every patched APK), so a merge adds at most
512 MB per file and 1 GB in all. Each file is read from its raw bytes and
checked against both of its ZIP headers: the local header must name the same
file with the same method, sizes and checksum as the central directory (or a
data descriptor must carry them; ZIP64 sizes are understood), the data must
end before the next entry, a stored file must be exactly its declared size, a
compressed one must end exactly at the end of its data with nothing after it,
and its checksum must match. So a forged header can't truncate a file or fill
memory. A resource table counts as empty only
if it defines nothing at all (styles and other complex entries included). A merged, re-signed APK can't update a
copy signed by the publisher.
## What won't run, and why
- **Games that enforce their Meta store licence.** Many store games link
Meta's Platform SDK (`libovrplatformloader.so`), for the licence check and
for user ids, achievements, multiplayer, cloud saves or DLC. On the Frame the
SDK can't reach Meta's services, which only exist on Meta headsets. A game
that enforces the check then quits or hangs; one that doesn't may run
without its online features. Frame Control can't tell which from the APK, so
it installs these with a warning and **won't remove or work around the
check**, and doesn't bundle, call or link to any tool that does. If yours
quits, ask the developer for a build without the check, or get the game from
Steam or itch.io. Most SideQuest, itch.io and open-source games don't link
the SDK.
- **32-bit games** (Quest 1 era): the Frame runs 64-bit ARM only. Refused.
- **VrApi-only games**: the Frame has OpenXR, not Meta's older VrApi. They
install with a warning, but may not run.
- **Anything else that needs Meta**: avatars, Quest-only device checks,
passthrough and other `XR_FB_*` features SteamVR lacks
([vr-apks.md](vr-apks.md#what-steamvrs-android-runtime-supports-verified)).
- **Private app data** (`/data/data/<package>`) can't be read over adb without
root, so only saves in `Android/data` come across.
What usually works: OpenXR games without Meta's licence check, for example
open-source and itch.io builds you sideloaded onto the Quest.
Only import games you own, for your own use.
## Evidence and limits
**Verified locally** with a fake adb (`tests/fakeadb.py`, `tests/test_quest.py`):
device detection (Quest, the Frame's tunnel, the Frame over USB, other
Android devices, unauthorised, no permissions, offline, none, several),
package listing and split detection, the refusals and the licence-check warning, the order of copies
(APK checked before the OBB is copied), the two-step install then data copy
(the game is never started by Frame Control, never replaced while open, and
closed after its data is copied), split merging and signing, the split
refusals and size limits (including a forged zip bomb), the space estimate,
and cache clean-up after success and failure.
**Not verified:** a real Quest (none was available), Quest system apps in
`pm list packages -3`, real split APKs from the Meta Store, OBB and
`Android/data` writes into a running Lepton instance on a real Frame
(the OBB helper itself is also unverified there), whether games stay open
without their OBB long enough for the second import, and the Linux udev rule.
Large games are copied into memory while being merged or patched, as every
patched APK already is.
+2 -9
View File
@@ -214,16 +214,9 @@ source-search endpoint remain unverified (the public install seam is tested).
- **Meta entitlement.** Apps that call the Oculus Platform SDK
(`libovrplatformloader.so`) to check the Quest store licence need Meta's
services. Frame Control won't work around that: no patching, no stub
libraries, no OVRPort or FramePort. Linking the SDK doesn't mean an app
enforces the check (Meta's VRC.Quest.Security.1 has only been
*Recommended* since 2021, and a failed check does nothing by itself; the app
decides), so [Import from my Quest](quest-import.md) and [My Meta
library](meta-library.md) install these with a warning: they may quit,
hang, or run without online features.
services. Frame Control won't work around that.
- **VrApi-era apps** (`libvrapi.so`, before OpenXR) need an API translator,
not a patch. They install with a warning that they may not run: some
OpenXR games ship `libvrapi.so` without depending on it.
not a patch.
## Frame Control does this for you
-100
View File
@@ -1,100 +0,0 @@
#!/usr/bin/env python3
"""A stand-in for Android's adb, for tests/test_quest.py: just enough of `devices -l`,
`shell`, and `pull` for frame_quest.py, run as a real subprocess.
$FAKEADB_STATE is a JSON file:
devices [{"serial", "state", "extra" (the rest of the `devices -l` line), "props" {getprop name: value}}]
root a folder standing in for each device's file system (remote /x/y is root/x/y)
packages {package: [APK paths, as `pm path` prints them]}
fail_pull a remote path whose pull fails
Every call is appended to $FAKEADB_LOG as a JSON line. Shell commands are the simple
`; `-separated ones frame_quest sends, each answered here: no real shell runs.
"""
import json
import os
import shutil
import sys
args = sys.argv[1:]
with open(os.environ['FAKEADB_LOG'], 'a') as f:
f.write(json.dumps(args) + '\n')
with open(os.environ['FAKEADB_STATE']) as f:
state = json.load(f)
root = state.get('root', '')
serial = None
if args[:1] == ['-s']:
serial, args = args[1], args[2:]
def fail(message, code=1):
sys.stderr.write(message + '\n')
sys.exit(code)
def local(remote):
return os.path.join(root, remote.lstrip('/'))
def size_kb(path):
total = 0
for dirpath, _, files in os.walk(path):
total += sum(os.path.getsize(os.path.join(dirpath, n)) for n in files)
return max(4, total // 1024)
if args == ['devices', '-l']:
print('List of devices attached')
for d in state.get('devices', []):
print(f"{d['serial']:<22} {d['state']} {d.get('extra', '')}".rstrip())
sys.exit(0)
device = next((d for d in state.get('devices', []) if d['serial'] == serial), None)
if serial is None or device is None:
fail(f"adb: device '{serial}' not found")
if device['state'] == 'unauthorized':
fail('adb: device unauthorized.\nThis adb server\'s $ADB_VENDOR_KEYS is not set')
if device['state'] != 'device':
fail(f"adb: device {device['state']}")
if args[:1] == ['shell']:
out = []
for command in ' '.join(args[1:]).split('; '):
command = command.replace(' 2>/dev/null', '').strip()
words = command.split()
if not words or command == 'true':
continue
if words[0] == 'getprop':
out.append(device.get('props', {}).get(words[1], ''))
elif words[0] == 'echo':
out.append(' '.join(words[1:]))
elif command == 'pm list packages -3':
out += ['package:' + p for p in state.get('packages', {})]
elif words[:2] == ['pm', 'path']:
out += ['package:' + p for p in state.get('packages', {}).get(words[2], [])]
elif words[:2] == ['du', '-sk']:
if os.path.isdir(local(words[2])):
out.append(f'{size_kb(local(words[2]))}\t{words[2]}')
elif words[:3] == ['stat', '-c', '%s']:
out += [str(os.path.getsize(local(p.strip("'")))) for p in words[3:]]
else:
fail(f'/system/bin/sh: {words[0]}: inaccessible or not found', 127)
print('\n'.join(out))
sys.exit(0)
if args[:1] == ['pull'] and len(args) == 3:
remote, dest = args[1], args[2]
if remote == state.get('fail_pull'):
fail(f"adb: error: failed to copy '{remote}' to '{dest}': remote Connection reset")
src = local(remote)
if not os.path.exists(src):
fail(f"adb: error: failed to stat remote object '{remote}': No such file or directory")
if os.path.isdir(dest):
dest = os.path.join(dest, os.path.basename(remote.rstrip('/')))
if os.path.isdir(src):
shutil.copytree(src, dest)
else:
shutil.copyfile(src, dest)
print(f'{remote}: 1 file pulled.')
sys.exit(0)
fail('adb: unknown command ' + ' '.join(args))
+5
View File
@@ -0,0 +1,5 @@
Hand-made itch.io API responses for tests/test_itch_purchases.py; no real account or
network was used. owned-keys-*.json follow the documented /profile/owned-keys sample
(https://itch.io/docs/api/serverside). uploads.json follows the shape the itch.io app's
client library (github.com/itchio/go-itchio) reads from /games/ID/uploads, with the
`traits` array it handles; `p_android` there is assumed, not documented.
+22
View File
@@ -0,0 +1,22 @@
{
"page": 1,
"per_page": 500,
"owned_keys": [
{"id": 9001, "game_id": 341190, "purchase_id": 3923754, "downloads": 6,
"created_at": "2026-03-10T23:46:36Z", "updated_at": "2026-03-10T23:46:36Z",
"game": {"id": 341190, "title": "Orbit Garden VR", "url": "https://maker.itch.io/orbit-garden",
"cover_url": "https://img.itch.zone/aW1nLzE2OTA3NTIuanBn/315x250%23c/Ihj7Wa.jpg",
"short_text": "Grow plants in zero gravity.", "type": "default", "classification": "game",
"min_price": 1299, "traits": ["p_windows", "p_android", "can_be_bought"],
"created_at": "2025-12-04T16:18:04Z", "published_at": "2026-01-03T17:28:47Z",
"user": {"id": 46531, "username": "maker", "display_name": "Maker Studio",
"url": "https://maker.itch.io"}}},
{"id": 9002, "game_id": 3, "purchase_id": null, "downloads": 0,
"created_at": "2026-02-01T10:00:00Z", "updated_at": "2026-02-01T10:00:00Z",
"game": {"id": 3, "title": "Desk Only", "url": "https://leafo.itch.io/x-moon",
"cover_url": "https://img.itch.zone/aW1hZ2UvMy8xODM3LnBuZw==/315x250%23/y2uYQI.png",
"type": "default", "classification": "game", "min_price": 0,
"traits": ["p_windows", "p_linux", "p_osx"], "published_at": "2013-03-03T23:02:14Z",
"user": {"id": 1, "username": "leafo", "display_name": "leafo", "url": "https://leafo.itch.io"}}}
]
}
+19
View File
@@ -0,0 +1,19 @@
{
"page": 2,
"per_page": 500,
"owned_keys": [
{"id": 8001, "game_id": 341190, "purchase_id": null, "downloads": 1,
"created_at": "2025-06-01T10:00:00Z", "updated_at": "2025-06-01T10:00:00Z",
"game": {"id": 341190, "title": "Orbit Garden VR", "url": "https://maker.itch.io/orbit-garden",
"type": "default", "classification": "game", "min_price": 1299,
"traits": ["p_windows", "p_android", "can_be_bought"],
"user": {"id": 46531, "username": "maker", "display_name": "Maker Studio"}}},
{"id": 8002, "game_id": 77, "purchase_id": null, "downloads": 2,
"created_at": "2025-05-01T10:00:00Z", "updated_at": "2025-05-01T10:00:00Z",
"game": {"id": 77, "title": "Free Quest Toy", "url": "https://toys.itch.io/quest-toy",
"cover_url": "http://img.example/not-itch.png",
"type": "default", "classification": "game", "min_price": 0,
"traits": ["p_android"], "published_at": "2025-04-01T00:00:00Z",
"user": {"id": 5, "username": "toys"}}}
]
}
+28
View File
@@ -0,0 +1,28 @@
{
"uploads": [
{"id": 501, "game_id": 341190, "filename": "OrbitGarden-win64.zip", "display_name": "Windows",
"size": 734003200, "storage": "hosted", "type": "default", "traits": ["p_windows"],
"created_at": "2026-04-01T10:00:00Z", "updated_at": "2026-04-02T10:00:00Z"},
{"id": 502, "game_id": 341190, "filename": "OrbitGarden-Quest-old.apk", "display_name": "Quest 1.0",
"size": 104857600, "storage": "hosted", "type": "default", "traits": ["p_android"],
"created_at": "2026-01-01T10:00:00Z", "updated_at": "2026-01-05T10:00:00Z"},
{"id": 503, "game_id": 341190, "filename": "OrbitGarden-Quest.apk", "display_name": "Quest 1.2",
"size": 115343360, "storage": "hosted", "type": "default", "traits": ["p_android"],
"created_at": "2026-04-01T10:00:00Z", "updated_at": "2026-04-03T10:00:00Z"},
{"id": 504, "game_id": 341190, "filename": "OrbitGarden-demo.apk", "display_name": "Quest demo",
"size": 52428800, "storage": "hosted", "type": "default", "traits": ["p_android", "demo"],
"created_at": "2026-05-01T10:00:00Z", "updated_at": "2026-05-01T10:00:00Z"},
{"id": 505, "game_id": 341190, "filename": "OrbitGarden-android-bundle.zip", "display_name": "Android (zip)",
"size": 120000000, "storage": "hosted", "type": "default", "traits": ["p_android"],
"created_at": "2026-06-01T10:00:00Z", "updated_at": "2026-06-01T10:00:00Z"},
{"id": 506, "game_id": 341190, "filename": "Soundtrack.apk", "display_name": "Soundtrack",
"size": 1000, "storage": "hosted", "type": "soundtrack", "traits": [],
"created_at": "2026-06-01T10:00:00Z", "updated_at": "2026-06-01T10:00:00Z"},
{"id": 507, "game_id": 341190, "filename": "mirror.apk", "display_name": "Mirror",
"size": 1000, "storage": "external", "host": "example.com", "type": "default", "traits": ["p_android"],
"created_at": "2026-06-01T10:00:00Z", "updated_at": "2026-06-01T10:00:00Z"},
{"id": 508, "game_id": 341190, "filename": "OrbitGarden-mac.dmg", "display_name": "macOS",
"size": 1000, "storage": "hosted", "type": "default", "traits": ["p_osx"],
"created_at": "2026-06-01T10:00:00Z", "updated_at": "2026-06-01T10:00:00Z"}
]
}
+12 -1
View File
@@ -19,7 +19,7 @@ function grab(name) {
fail(`unbalanced ${name}`);
}
const names = ["esc", "storeName", "storeHue", "storeInitial", "storeText", "storeOpenSource", "sourcePage", "storeImage",
"storeArt", "storeIcon", "sourceMark", "currentOffer", "detailBadges", "renderAppDetail", "renderAppDetailBody",
"storeArt", "storeIcon", "sourceMark", "storePrice", "currentOffer", "detailBadges", "renderAppDetail", "renderAppDetailBody",
"updateInstall", "openStoreApp", "loadOfferDetails"];
const code = names.map(grab).join("");
@@ -54,6 +54,17 @@ const offer = { source: "fdroid", id: "org.example.brush", source_name: "F-Droid
if (html.includes('role="alert"')) fail("a good listing showed the error state");
}
// A game the user owns on itch.io says so instead of "Free".
{
const p = page();
p.sourceState.detail = { app: { name: "Orbit Garden", offers: [{ ...offer, source: "itch-owned", source_name: "itch.io (my purchases)",
owned: true, owned_label: "Owned on itch.io", free: false }] }, choice: 0 };
if (p.renderAppDetail() !== true) fail("an owned listing reported a render failure");
const html = p.els.detailBody.innerHTML;
if (!html.includes('<div class="detail-price">Owned on itch.io</div>')) fail("owned listing has no Owned on itch.io label");
if (html.includes("No purchase needed")) fail("owned listing claims no purchase is needed");
}
// A listing that can't render (no offer to show) gets a clear message, not an empty dialog.
{
const p = page();
-44
View File
@@ -256,47 +256,3 @@ class SensorTests(unittest.TestCase):
self.assertIsNone(status.activity_level())
with patch.object(status, 'run', return_value='[{"operation":"status","activity_level":3}]'):
self.assertEqual(status.activity_level(), 3)
class DesktopBundleTests(unittest.TestCase):
"""server.comfort runs the copy cached on the headset; the sources go only when it's missing."""
@unittest.skipUnless(os.name == "posix", "checks POSIX permissions")
def test_sources_are_sent_once_per_version(self):
import server
with tempfile.TemporaryDirectory() as tmp:
here, home = Path(tmp) / 'ui', Path(tmp) / 'frame'
here.mkdir()
home.mkdir()
for name in server.COMFORT_FILES:
(here / name).write_text('# ' + name + '\n')
(here / 'frame_comfort.py').write_text('import json, sys\nprint(json.dumps({"active": False, "got": sys.argv[1]}))\n')
sent = []
def ssh(remote, *, stdin, timeout): # the headset: the script run under its own home
sent.append(stdin)
return subprocess.run([sys.executable, '-'], input=stdin, capture_output=True, text=True, check=True,
env={**os.environ, 'HOME': str(home)}).stdout
with patch.object(server, 'HERE', here), patch.object(server, 'ssh', side_effect=ssh):
first = server.comfort({'action': 'status'})
self.assertEqual(json.loads(first['got']), {'action': 'status'})
self.assertEqual(len(sent), 2) # the check, then the sources
self.assertNotIn('# frame_steam.py', sent[0])
self.assertIn('# frame_steam.py', sent[1])
self.assertEqual(server.comfort({'action': 'status'}), first)
self.assertEqual(len(sent), 3)
self.assertNotIn('# frame_steam.py', sent[2])
cached, = (home / '.cache/frame-control/comfort').iterdir()
self.assertEqual(sorted(p.name for p in cached.iterdir()), sorted(server.COMFORT_FILES))
self.assertEqual(cached.stat().st_mode & 0o777, 0o700)
self.assertTrue(all(p.stat().st_mode & 0o777 == 0o600 for p in cached.iterdir()))
# A copy cut short is completed; a changed helper is a new version.
(cached / 'frame_status.py').unlink()
server.comfort({'action': 'status'})
self.assertEqual(len(sent), 5)
self.assertTrue((cached / 'frame_status.py').exists())
(here / 'frame_steam.py').write_text('# frame_steam.py, changed\n')
server.comfort({'action': 'status'})
self.assertEqual(len(sent), 7)
self.assertEqual(len(list((home / '.cache/frame-control/comfort').iterdir())), 2)
-29
View File
@@ -59,32 +59,3 @@ const log = ()=>{}, toast = ()=>{};
'''
result = subprocess.run(['node', '-e', setup + code + checks], capture_output=True, text=True)
self.assertEqual(result.returncode, 0, result.stderr)
def test_poll_waits_while_the_frame_is_unreachable(self):
page = (ROOT / 'ui/index.html').read_text(encoding='utf-8')
start = page.index('async function pollComfort()')
code = page[start:page.index('$("comfortForm").onsubmit', start)]
js = r'''
const assert = require('node:assert/strict');
const el = {textContent: 'Checking session…'};
const $ = () => el;
let online = false, comfortBusy = false, calls = 0, timers = 0;
const api = async () => { calls++; return {}; };
const renderComfort = () => {};
const setTimeout = () => { timers++; };
''' + code + r'''
(async () => {
await pollComfort();
assert.equal(calls, 0); // no ssh attempt while the status check can't reach the Frame
assert.match(el.textContent, /when the Frame is connected/);
assert.equal(timers, 1); // still scheduled, to resume when it's back
online = true;
await pollComfort();
assert.equal(calls, 1);
online = null; // a status check that failed for another reason: keep polling
await pollComfort();
assert.equal(calls, 2);
})().catch(e => { console.error(e); process.exitCode = 1; });
'''
result = subprocess.run(['node', '-e', js], capture_output=True, text=True)
self.assertEqual(result.returncode, 0, result.stderr)
-56
View File
@@ -423,19 +423,6 @@ class InstallTests(unittest.TestCase):
self.assertIn(('remove', '3346865537'), [c.args for c in api.call_args_list])
self.assertTrue(any(c.args[0] == 'rm -rf Applications/Android/org.test.vr' for c in ssh.call_args_list))
def test_failed_update_removes_partial_apk_and_keeps_the_app(self):
with patch.object(android.frame_artwork, 'prepare', return_value=(self.images, [])), \
patch.object(android, 'read_meta', return_value=dict(self.existing)), \
patch.object(android, '_copy', side_effect=android.FrameError('copying x.apk to the Frame timed out')), \
patch.object(android, 'ssh', return_value='') as ssh, \
patch.object(android, 'shortcut_tool') as api:
with self.assertRaisesRegex(android.FrameError, 'timed out'):
android._install('x.apk', self.info, 'org.test.vr', False, None, None)
commands = [c.args[0] for c in ssh.call_args_list]
self.assertIn('rm -f Applications/Android/org.test.vr/app.apk.part', commands)
self.assertFalse(any(c.startswith('rm -rf') for c in commands))
api.assert_not_called()
def test_remove_keeps_data_when_requested_and_survives_steam_failure(self):
with patch.object(android, '_meta_or_fail', side_effect=lambda pkg: dict(self.existing)), \
patch.object(android, 'stop'), \
@@ -499,49 +486,6 @@ class InstallTests(unittest.TestCase):
self.assertIn('podman stop -t 5 lepton-steamlaunch-2800000001', ssh.call_args.args[0])
class CopyAndSshTests(unittest.TestCase):
"""The commands behind an install fail as FrameErrors with readable words, and a large APK
gets the time it needs."""
def fake(self, stderr, code=1, stdout=b''):
"""frame_host.run_ssh running a stand-in that writes these bytes, with the caller's options."""
script = (f'import sys; sys.stdout.buffer.write({stdout!r}); sys.stderr.buffer.write({stderr!r}); '
f'sys.exit({code})')
return lambda argv, **kwargs: subprocess.run([sys.executable, '-c', script], **kwargs)
def test_large_apk_copy_timeout_grows_with_its_size(self):
seen = {}
with tempfile.NamedTemporaryFile(suffix='.apk') as f, \
patch.object(android.frame_host, 'run_ssh', side_effect=lambda argv, **kw: seen.update(kw)), \
patch.object(android.os.path, 'getsize', return_value=2 * 1024 ** 3):
android._copy(f.name, 'Applications/Android/x/app.apk.part')
# 2 GB at the slowest rate allowed: 4096 s, not the old fixed 600 (300 MB at 4 Mbit/s).
self.assertGreaterEqual(seen['timeout'], 2 * 1024 ** 3 / (512 * 1024))
with tempfile.NamedTemporaryFile(suffix='.apk') as f, \
patch.object(android.frame_host, 'run_ssh', side_effect=lambda argv, **kw: seen.update(kw)):
android._copy(f.name, 'x/launch.sh', timeout=120)
self.assertEqual(seen['timeout'], 120) # a small file keeps the least time given
def test_copy_failure_in_another_encoding_is_a_frame_error(self):
# A message from a Windows code page, or a file name, isn't UTF-8.
with tempfile.NamedTemporaryFile(suffix='.apk') as f, \
patch.object(android.shutil, 'which', return_value=None), \
patch.object(android.frame_host, 'run_ssh', side_effect=self.fake(b'scp: Verbindung \x81ber\xff\n')):
with self.assertRaisesRegex(android.FrameError, r'\Acopying .* to the Frame failed: scp: Verbindung'):
android._copy(f.name, 'x/app.apk.part')
def test_ssh_failure_messages_are_text(self):
with patch.object(android.frame_host, 'run_ssh', side_effect=self.fake(b'mv: \xff cannot move\n')):
with self.assertRaisesRegex(android.FrameError, r'\Amv: .* cannot move\Z'):
android.ssh('mv a b')
with patch.object(android.frame_host, 'run_ssh', side_effect=self.fake(b'cat: write error: No space left on device\n')):
with self.assertRaises(android.FrameError) as caught:
android.ssh('cat > x', input=b'\x89PNG') # bytes in, bytes out
self.assertEqual(str(caught.exception), 'cat: write error: No space left on device')
with patch.object(android.frame_host, 'run_ssh', side_effect=self.fake(b'', code=0, stdout=b'ok \xff\n')):
self.assertEqual(android.ssh('echo ok').strip(), 'ok �')
class SteamAPITests(unittest.TestCase):
def test_artwork_api_enums_and_safe_serialization(self):
with patch.object(shortcuts, 'evaluate', return_value={'warnings': []}) as evaluate:
-14
View File
@@ -93,20 +93,6 @@ class ApkInfo(unittest.TestCase):
frame_apk.MAX_MANIFEST = limit
self.assertLess(peak, 8 * 1024**2)
def test_damaged_download_is_an_apk_error(self):
# Corrupt deflate data used to escape as zlib.error, which the upload path didn't catch
# (a 500, and no install report) and frame_android.install reported as 'other'.
buf = io.BytesIO()
with zipfile.ZipFile(buf, 'w', zipfile.ZIP_DEFLATED) as z:
z.writestr('AndroidManifest.xml', manifest('com.example.bad', 0x7f010000, 0x7f010001, 21) * 20)
data = bytearray(buf.getvalue())
at = data.index(b'PK\x03\x04')
start = at + 30 + sum(struct.unpack_from('<HH', data, at + 26)) # past the name and extra field
for i in range(start, start + 40):
data[i] ^= 0x5a
with self.assertRaisesRegex(frame_apk.ApkError, 'could not read the APK manifest'):
self.read(bytes(data))
def test_refuses_compression_android_cant_read(self):
for method in (zipfile.ZIP_BZIP2, zipfile.ZIP_LZMA):
buf = io.BytesIO()
+862
View File
@@ -0,0 +1,862 @@
"""My itch.io purchases: the account key, owned games and their Android downloads.
Offline: a fake HTTPS layer under urllib's real opener (so redirects and header
handling are the real ones) serves the recorded JSON in fixtures/itch_account.
"""
import sandbox # noqa: F401 (first: keeps tests off real data and services)
import contextlib, email.message, http.client, io, ipaddress, json, os, socket, sys, tempfile, threading, time
import unittest
import urllib.error, urllib.parse, urllib.request, zipfile
from pathlib import Path
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
from apk_sources import SourceError, SourceLimited, _itch_account as account, _web, itch, search
import server
FIX = Path(__file__).parent / 'fixtures' / 'itch_account'
KEY = 'TESTKEYabcdefghijklmnopqrstuvwxyz0123456789'
OTHER = 'OTHERKEY0123456789abcdefghijklmnopqrstuv'
def apk_bytes():
out = io.BytesIO()
with zipfile.ZipFile(out, 'w') as z:
z.writestr('AndroidManifest.xml', b'\x03\x00\x08\x00')
z.writestr('classes.dex', b'dex\n035\x00')
return out.getvalue()
class Reply(io.BytesIO):
"""Just enough of http.client.HTTPResponse for urllib's handlers."""
def __init__(self, url, code=200, body=b'', headers=None):
super().__init__(body if isinstance(body, bytes) else json.dumps(body).encode())
self.url, self.code, self.status, self.msg, self.reason = url, code, code, 'fake', 'fake'
self.headers = email.message.Message()
for k, v in (headers or {}).items():
self.headers[k] = v
def info(self):
return self.headers
def geturl(self):
return self.url
def getcode(self):
return self.code
def getheader(self, name, default=None):
return self.headers.get(name, default)
class FakeConnection:
"""_web._connect's http.client connection, answered from the same routes."""
def __init__(self, web, host):
self.web, self.host, self.reply = web, host, None
def request(self, method, path, headers=None):
req = urllib.request.Request('https://' + self.host + path, headers=headers or {})
self.reply = self.web(None, req)
def getresponse(self):
return self.reply
def close(self):
pass
# What the fake resolver answers: the storage host is public; the rest are traps.
ADDRESSES = {'storage.itch-cdn.example': '151.101.1.1', 'other-cdn.example': '151.101.2.2',
'localhost.': '127.0.0.1', 'frame.local': '192.168.1.20', '127.0.0x0.1': '127.0.0.1',
'intranet.example': '10.0.0.5', 'mapped.example': '::ffff:127.0.0.1', 'cgnat.example': '100.64.0.1',
'mixed.example': ('151.101.1.1', '10.0.0.5'), 'dual.example': ('2a04:4e42::81', '151.101.1.1'),
'nat64.example': '64:ff9b:1::a00:1', 'sitelocal.example': 'fec0::1',
'many.example': tuple('151.101.%d.1' % n for n in range(1, 9))}
REAL_CONNECT = _web._connect
class FakeItch:
"""Routes https requests by host and path; records every request with its headers."""
def __init__(self):
self.calls, self.routes = [], {}
def route(self, host_path, reply):
self.routes[host_path] = reply
def __call__(self, handler, req):
url = urllib.parse.urlsplit(req.full_url)
headers = {k.lower(): v for k, v in req.header_items()}
self.calls.append((req.full_url, headers))
reply = self.routes.get(url.hostname + url.path)
if reply is None:
return Reply(req.full_url, 404, {'errors': ['not found']})
if callable(reply):
reply = reply(req, urllib.parse.parse_qs(url.query))
if isinstance(reply, Exception):
raise reply
if isinstance(reply, Reply):
return reply
return Reply(req.full_url, 200, reply)
def urls(self, host_path):
return [u for u, _ in self.calls if urllib.parse.urlsplit(u).hostname + urllib.parse.urlsplit(u).path == host_path]
def owned_pages(req, query):
page = query['page'][0]
return {'1': json.loads((FIX / 'owned-keys-1.json').read_text()),
'2': json.loads((FIX / 'owned-keys-2.json').read_text())}.get(page, {'page': int(page), 'owned_keys': []})
class ItchPurchases(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
root = Path(self.tmp.name)
for target, value in [(account.frame_host, 'data_dir'), (search.frame_host, 'data_dir')]:
p = patch.object(target, value, lambda *parts: root.joinpath('data', *parts))
p.start()
self.addCleanup(p.stop)
(root / 'cache').mkdir()
for p in (patch.object(_web, 'cache', return_value=str(root / 'cache')),
patch.object(_web.frame_host, 'cache_dir', lambda *parts: root.joinpath('cache', *parts))):
p.start()
self.addCleanup(p.stop)
self.web = FakeItch()
self.web.route('api.itch.io/profile', {'user': {'username': 'player', 'id': 1}})
self.web.route('api.itch.io/profile/owned-keys', owned_pages)
self.web.route('api.itch.io/games/341190/uploads', json.loads((FIX / 'uploads.json').read_text()))
p = patch('urllib.request.HTTPSHandler.https_open', autospec=True, side_effect=self.web)
p.start()
self.addCleanup(p.stop)
p = patch('urllib.request.HTTPHandler.http_open', side_effect=AssertionError('plain HTTP in test'))
p.start()
self.addCleanup(p.stop)
real_lookup = _web.socket.getaddrinfo
def lookup(host, port, *args, **kwargs):
if host not in ADDRESSES:
try:
ipaddress.ip_address(host) # IP literals (and the in-process test server) need no DNS
return real_lookup(host, port, *args, **kwargs)
except ValueError:
pass
if host == 'localhost':
return real_lookup(host, port, *args, **kwargs)
raise _web.socket.gaierror('unknown test host ' + host)
found = ADDRESSES[host]
return [(10 if ':' in a else 2, 1, 6, '', (a, port)) for a in ([found] if isinstance(found, str) else found)]
self.connected, self.unreachable = [], set()
def connect(host, address, timeout, deadline):
self.connected.append((host, address))
if address[0] in self.unreachable:
raise OSError('Network is unreachable')
self.assertLessEqual(deadline - time.monotonic(), _web.CONNECT_BUDGET)
return FakeConnection(self.web, host)
for p in (patch.object(_web.socket, 'getaddrinfo', lookup), patch.object(_web, '_connect', connect)):
p.start()
self.addCleanup(p.stop)
account._forget()
self.addCleanup(account._forget)
_web._limited.clear()
self.addCleanup(_web._limited.clear)
search._status.clear()
def owned_source(self):
return next(s for s in itch.sources() if s['id'] == itch.OWNED)
def connect(self):
self.assertTrue(account.connect(KEY)['itch']['connected'])
# -- the key ---------------------------------------------------------------
def test_connect_checks_the_key_then_saves_it_0600(self):
result = account.connect(' ' + KEY + '\n')
self.assertEqual(result['message'], 'Connected to itch.io as player')
self.assertEqual(result['itch'], {'connected': True, 'username': 'player'})
self.assertNotIn(KEY, json.dumps(result))
url, headers = self.web.calls[0]
self.assertEqual(url, 'https://api.itch.io/profile')
self.assertEqual(headers['authorization'], 'Bearer ' + KEY)
self.assertEqual(json.loads(account.path().read_text())['key'], KEY)
if os.name == 'posix':
self.assertEqual(os.stat(account.path()).st_mode & 0o777, 0o600)
self.assertEqual([p.name for p in account.path().parent.iterdir()], ['itch-account.json']) # no .tmp left
self.assertNotIn(KEY, json.dumps(account.status()))
def test_reconnecting_rewrites_the_file_0600_even_if_it_was_wider(self):
self.connect()
os.chmod(account.path(), 0o644)
account.connect(OTHER)
self.assertEqual(json.loads(account.path().read_text())['key'], OTHER)
if os.name == 'posix':
self.assertEqual(os.stat(account.path()).st_mode & 0o777, 0o600)
def test_bad_or_revoked_key_stays_disconnected(self):
for reply in (Reply('https://api.itch.io/profile', 401, {'errors': ['invalid key']}),
Reply('https://api.itch.io/profile', 403, b'Forbidden'),
{'errors': ['invalid key']}):
self.web.route('api.itch.io/profile', reply)
with self.assertRaisesRegex(account.BadKey, 'did not accept the API key') as caught:
account.connect(KEY)
self.assertNotIn(KEY, str(caught.exception))
self.assertFalse(account.path().exists())
self.assertEqual(account.status(), {'connected': False, 'username': None})
self.assertEqual([s['id'] for s in itch.sources()], ['itch', 'itch-quest'])
with self.assertRaisesRegex(account.BadKey, 'does not look like'):
account.connect('short')
with self.assertRaisesRegex(account.BadKey, 'does not look like'):
account.connect(KEY + '\r\nX-Injected: 1')
self.assertEqual(len(self.web.calls), 3) # malformed keys never reach itch.io
def test_a_refused_request_with_a_working_key_keeps_the_key(self):
self.connect()
self.web.route('api.itch.io/games/341190/uploads', Reply('https://api.itch.io/games/341190/uploads', 403, b''))
with self.assertRaisesRegex(SourceError, r'refused this request \(HTTP 403\)') as caught:
itch.details(self.owned_source(), '341190')
self.assertNotIsInstance(caught.exception, account.BadKey)
self.assertTrue(account.connected())
def test_a_key_revoked_later_is_removed_with_a_clear_message(self):
self.connect()
self.web.route('api.itch.io/profile/owned-keys', Reply('https://api.itch.io/profile/owned-keys', 401, b''))
self.web.route('api.itch.io/profile', Reply('https://api.itch.io/profile', 401, b''))
with self.assertRaisesRegex(account.BadKey, 'no longer accepts the saved API key'):
itch.search(self.owned_source(), '')
self.assertFalse(account.path().exists())
self.assertNotIn(itch.OWNED, [s['id'] for s in itch.sources()])
def test_disconnect_deletes_the_key_file(self):
self.connect()
self.assertIn(itch.OWNED, [s['id'] for s in itch.sources()])
result = account.disconnect()
self.assertFalse(account.path().exists())
self.assertEqual(result['itch'], {'connected': False, 'username': None})
self.assertNotIn(itch.OWNED, [s['id'] for s in itch.sources()])
with self.assertRaisesRegex(SourceError, 'Connect your itch.io account first'):
account.owned_keys()
account.disconnect() # already gone: fine
# -- owned games -------------------------------------------------------------
def test_owned_keys_follow_pages_until_an_empty_one(self):
self.connect()
keys = account.owned_keys()
self.assertEqual([k['id'] for k in keys], [9001, 9002, 8001, 8002])
pages = self.web.urls('api.itch.io/profile/owned-keys')
self.assertEqual([urllib.parse.parse_qs(urllib.parse.urlsplit(u).query) for u in pages],
[{'page': [str(n)], 'per_page': ['500']} for n in (1, 2, 3)])
self.assertTrue(all('Bearer ' + KEY == h['authorization'] for _, h in self.web.calls))
self.assertTrue(all(KEY not in u for u, _ in self.web.calls))
account.owned_keys() # cached in memory, not on disk
self.assertEqual(len(self.web.urls('api.itch.io/profile/owned-keys')), 3)
self.assertFalse(any(KEY in p.read_text(errors='ignore') or b'Orbit' in p.read_bytes()
for p in Path(_web.cache()).iterdir()))
def test_page_cap_stops_a_server_that_never_ends(self):
self.connect()
self.web.route('api.itch.io/profile/owned-keys', lambda req, q: {'owned_keys': [{'id': int(q['page'][0])}]})
with patch.object(account, 'PAGES', 3):
self.assertEqual(len(account.owned_keys()), 3)
def test_search_lists_owned_android_games_labelled(self):
self.connect()
entries = itch.search(self.owned_source(), '')
self.assertEqual([(e['id'], e['name']) for e in entries], [('341190', 'Orbit Garden VR'), ('77', 'Free Quest Toy')])
orbit, toy = entries
self.assertEqual(orbit['download_key_id'], 9001) # the newest key for a game owned twice
self.assertTrue(orbit['owned'])
self.assertEqual(orbit['owned_label'], 'Owned on itch.io')
self.assertFalse(orbit['free'])
self.assertTrue(toy['free'])
self.assertIsNone(toy['icon']) # not an itch.io image host
self.assertEqual(orbit['page'], 'https://maker.itch.io/orbit-garden')
self.assertEqual(itch.search(self.owned_source(), 'zero gravity')[0]['id'], '341190')
# Through the store's search, owned (paid) games pass the free-only filter.
result = search.search('orbit', source=itch.OWNED)
self.assertEqual(result['sources'][0]['status'], 'ok')
offer = result['apps'][0]['offers'][0]
self.assertEqual((offer['source'], offer['source_name'], offer['owned_label']),
(itch.OWNED, 'itch.io (my purchases)', 'Owned on itch.io'))
self.assertNotIn(KEY, json.dumps(result))
def test_owned_games_show_under_either_vr_choice(self):
# itch.io doesn't say which purchases are VR, so neither filter hides them; other
# games of unknown kind still count as flat.
self.connect()
both = ['Free Quest Toy', 'Orbit Garden VR']
for vr in (True, False):
names = sorted(a['name'] for a in search.search('', vr=vr, source=itch.OWNED)['apps'])
self.assertEqual(names, both)
flat = {'source': 'x', 'id': '1', 'name': 'Flat', 'summary': '', 'vr': None}
self.assertEqual(search.group([flat], vr=True), [])
self.assertEqual(len(search.group([flat], vr=False)), 1)
self.assertEqual(search.group([dict(flat, owned=True, vr=False)], vr=True), [])
def test_only_the_account_source_may_mark_paid_games_owned(self):
entry = {'free': False, 'owned': True}
self.assertFalse(search.obtainable({'id': 'fdroid'}, entry))
self.assertTrue(search.obtainable({'id': itch.OWNED, 'owned': True}, entry))
self.assertTrue(search.obtainable({'id': 'fdroid'}, {'free': True}))
def test_android_upload_selection(self):
uploads = json.loads((FIX / 'uploads.json').read_text())['uploads']
# .apk files before other Android files, full games before demos, newest first; never
# soundtracks, externally hosted files or other platforms.
self.assertEqual([u['id'] for u in account.android_uploads(uploads)], [503, 502, 504, 505])
self.assertTrue(account.is_android({'filename': 'Game.APK'}))
self.assertTrue(account.is_android({'filename': 'game.zip', 'p_android': True}))
self.assertTrue(account.is_android({'filename': 'game.zip', 'platforms': {'android': 'all'}}))
self.assertFalse(account.is_android({'filename': 'game.zip', 'traits': ['p_windows']}))
self.assertEqual(account.android_uploads({'not': 'a list'}), [])
def test_details_offer_the_android_uploads(self):
self.connect()
entry = itch.details(self.owned_source(), '341190')
self.assertTrue(entry['downloadable'])
self.assertEqual([v['upload_id'] for v in entry['versions']], [503, 502, 504, 505])
self.assertEqual((entry['version'], entry['size'], entry['updated']), ('Quest 1.2', 115343360, '2026-04-03'))
self.assertTrue(entry['versions'][2]['demo'])
url = self.web.urls('api.itch.io/games/341190/uploads')[0]
self.assertEqual(url, 'https://api.itch.io/games/341190/uploads?download_key_id=9001')
with self.assertRaisesRegex(SourceError, 'not among'):
itch.details(self.owned_source(), '3') # owned, but not for Android
self.web.route('api.itch.io/games/77/uploads', {'uploads': []})
self.assertFalse(itch.details(self.owned_source(), '77')['downloadable'])
with self.assertRaisesRegex(SourceError, 'no Android download'):
itch.download(self.owned_source(), '77')
def test_download_url_building(self):
self.assertEqual(account.download_url(503, 9001),
'https://api.itch.io/uploads/503/download?download_key_id=9001')
for bad in [(0, 1), (1, None), ('5', 1), (True, 1), (-2, 1)]:
with self.assertRaises(SourceError):
account.download_url(*bad)
# -- downloads -----------------------------------------------------------------
def storage(self, location='https://storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk?sig=abc'):
self.web.route('api.itch.io/uploads/503/download',
Reply('https://api.itch.io/uploads/503/download', 302, b'', {'Location': location}))
self.web.route('storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk', Reply(location, 200, apk_bytes()))
def test_download_sends_the_key_to_itch_only_and_strips_it_on_redirect(self):
self.connect()
self.storage()
result = itch.download(self.owned_source(), '341190')
self.assertTrue(os.path.isfile(result['apk']))
self.assertFalse(result['verified'])
api, cdn = self.web.calls[-2:]
self.assertEqual(api[0], 'https://api.itch.io/uploads/503/download?download_key_id=9001')
self.assertEqual(api[1]['authorization'], 'Bearer ' + KEY)
self.assertTrue(cdn[0].startswith('https://storage.itch-cdn.example/'))
self.assertNotIn('authorization', cdn[1])
# Connected to the address that was checked, not to a fresh lookup.
self.assertEqual(self.connected, [('storage.itch-cdn.example', ('151.101.1.1', 443))])
self.assertTrue(all(KEY not in u for u, _ in self.web.calls))
with self.assertRaisesRegex(SourceError, 'version codes'):
itch.download(self.owned_source(), '341190', version_code=3)
def test_download_also_accepts_a_json_url_or_the_file_itself(self):
self.connect()
self.storage()
location = 'https://storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk?sig=abc'
self.web.route('api.itch.io/uploads/503/download',
Reply('https://api.itch.io/uploads/503/download', 200, {'url': location},
{'Content-Type': 'application/json'}))
self.assertTrue(os.path.isfile(itch.download(self.owned_source(), '341190')['apk']))
self.assertNotIn('authorization', self.web.calls[-1][1])
self.web.route('api.itch.io/uploads/503/download',
Reply('https://api.itch.io/uploads/503/download', 200, apk_bytes(),
{'Content-Type': 'application/octet-stream'}))
self.connected.clear()
self.assertTrue(os.path.isfile(itch.download(self.owned_source(), '341190')['apk']))
self.assertEqual(self.connected, []) # api.itch.io served it; no storage host involved
self.web.route('api.itch.io/uploads/503/download',
Reply('https://api.itch.io/uploads/503/download', 200, {'nothing': 1},
{'Content-Type': 'application/json'}))
with self.assertRaisesRegex(SourceError, 'did not say where'):
itch.download(self.owned_source(), '341190')
def test_storage_redirects_are_checked_on_every_hop(self):
self.connect()
self.storage('https://other-cdn.example/hop')
self.web.route('other-cdn.example/hop', Reply('https://other-cdn.example/hop', 302, b'',
{'Location': 'https://storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk'}))
self.assertTrue(os.path.isfile(itch.download(self.owned_source(), '341190')['apk']))
self.assertEqual([h for h, _ in self.connected], ['other-cdn.example', 'storage.itch-cdn.example'])
self.web.route('other-cdn.example/hop', Reply('https://other-cdn.example/hop', 302, b'',
{'Location': 'https://intranet.example/a.apk'}))
with self.assertRaisesRegex(SourceError, 'private network address'):
itch.download(self.owned_source(), '341190')
def test_redirects_to_plain_http_or_private_addresses_are_refused(self):
self.connect()
for location in ('http://storage.itch-cdn.example/a.apk', 'https://user:pw@cdn.example/a.apk',
'https://cdn.example:8443/a.apk'):
self.storage(location)
with self.assertRaisesRegex(SourceError, 'unexpected download URL'):
itch.download(self.owned_source(), '341190')
for location in ('https://localhost./a.apk', 'https://frame.local/a.apk', 'https://127.0.0x0.1/a.apk',
'https://intranet.example/a.apk', 'https://mapped.example/a.apk',
'https://cgnat.example/a.apk', 'https://mixed.example/a.apk',
'https://127.0.0.1/a.apk', 'https://[::1]/a.apk', 'https://169.254.169.254/a.apk',
'https://nat64.example/a.apk', 'https://[4000::1]/a.apk', 'https://[64:ff9b:1::a00:1]/a.apk',
'https://[fec0::1]/a.apk', 'https://sitelocal.example/a.apk'):
self.storage(location)
with self.assertRaisesRegex(SourceError, 'private network address'):
itch.download(self.owned_source(), '341190')
self.assertEqual(self.connected, [])
self.assertEqual({urllib.parse.urlsplit(u).hostname for u, _ in self.web.calls}, {'api.itch.io'})
def test_public_address_rules(self):
for ip in ('151.101.1.1', '8.8.8.8', '2a04:4e42::81', '2001:4860:4860::8888'):
self.assertTrue(_web.public_address(ip), ip)
for ip in ('127.0.0.1', '10.1.2.3', '172.16.0.1', '192.168.0.1', '169.254.169.254', '100.64.0.1',
'0.0.0.0', '224.0.0.1', '255.255.255.255', '240.0.0.1', '::1', '::', 'fe80::1%en0', 'fc00::1',
'ff02::1', 'not an ip',
'192.0.0.8', '192.0.0.11', '192.0.0.255', '192.0.0.170', '192.0.0.9', '192.0.0.1', # 3.9: global
'192.0.2.1', '192.88.99.1', '198.18.0.1', '198.19.255.255', '198.51.100.7', '203.0.113.9',
'0.1.2.3', '10.255.255.255', '172.31.0.1', '239.1.1.1', '254.0.0.1',
'4000::1', '100::1', 'f000::1', # reserved: Python 3.9's is_global says True for 4000::1
'::ffff:10.0.0.1', '::ffff:127.0.0.1', '::ffff:8.8.8.8', # IPv4-mapped
'::a00:1', '::8.8.8.8', # IPv4-compatible
'64:ff9b::a00:1', '64:ff9b::808:808', # NAT64 well-known prefix
'64:ff9b:1::a00:1', '64:ff9b:1::808:808', # NAT64 local-use: 3.9's is_global says True
'2002:a00:1::1', '2002:808:808::1', # 6to4
'2001:0:4136:e378:8000:63bf:3fff:fdd2', # Teredo
'fec0::1', 'feff::1', # site-local: Python 3.9's is_global says True
'fc00::1', 'fd12:3456::1', 'fe80::1', 'ff00::1', 'ff0e::1', '8000::1', 'c000::1',
'2001:db8::1', '3fff::1', # documentation
'2001:10::1', '2001:20::1', '2001:2::1'): # ORCHID, ORCHIDv2, benchmarking
self.assertFalse(_web.public_address(ip), ip)
self.assertEqual(_web.resolve_public('storage.itch-cdn.example'), [('151.101.1.1', 443)])
self.assertEqual(_web.resolve_public('dual.example'), [('2a04:4e42::81', 443), ('151.101.1.1', 443)])
def test_a_storage_url_carrying_the_key_is_refused(self):
self.connect()
for location in ('https://storage.itch-cdn.example/a.apk?api_key=' + KEY,
'https://storage.itch-cdn.example/a.apk?API_KEY=someone-elses',
'https://storage.itch-cdn.example/a.apk?apikey=x',
'https://storage.itch-cdn.example/' + KEY + '/a.apk',
'https://storage.itch-cdn.example/a.apk#' + KEY[:10],
'https://storage.itch-cdn.example/a.apk?x=' + urllib.parse.quote('%' + KEY),
'https://' + KEY.lower() + '.itch-cdn.example/a.apk'):
self.storage(location)
with self.assertRaisesRegex(SourceError, 'carrying your API key; refused') as caught:
itch.download(self.owned_source(), '341190')
self.assertNotIn(KEY.lower(), str(caught.exception).lower())
self.web.route('api.itch.io/uploads/503/download',
Reply('https://api.itch.io/uploads/503/download', 200,
{'url': 'https://storage.itch-cdn.example/a.apk?api_key=' + KEY},
{'Content-Type': 'application/json'}))
with self.assertRaisesRegex(SourceError, 'carrying your API key; refused'):
itch.download(self.owned_source(), '341190')
self.assertEqual(self.connected, [])
# A storage URL's own signature or token parameters are fine.
self.storage('https://storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk?token=abc&Signature=x')
self.assertTrue(os.path.isfile(itch.download(self.owned_source(), '341190')['apk']))
def test_a_later_redirect_carrying_the_key_is_refused_before_connecting(self):
self.connect()
bad = 'https://storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk?api_key=' + KEY
for hops in (['https://other-cdn.example/hop1'], # 2nd hop carries the key
['https://other-cdn.example/hop1', 'https://other-cdn.example/hop2']): # 3rd hop
self.connected.clear()
self.storage(hops[0])
for here, there in zip(hops, hops[1:] + [bad]):
path = urllib.parse.urlsplit(here).path
self.web.route('other-cdn.example' + path, Reply(here, 302, b'', {'Location': there}))
with self.assertRaisesRegex(SourceError, 'carrying your API key; refused') as caught:
itch.download(self.owned_source(), '341190')
self.assertNotIn(KEY, str(caught.exception))
self.assertEqual([h for h, _ in self.connected], ['other-cdn.example'] * len(hops))
self.assertFalse(any(KEY in u for u, _ in self.web.calls))
# The check runs on every hop before the name is even looked up.
seen = []
with patch.object(_web, 'resolve_public', side_effect=AssertionError('looked up')):
with self.assertRaisesRegex(SourceError, 'no thanks'):
_web.public_apk('https://storage.itch-cdn.example/a.apk', 'itch.io',
check=lambda u: seen.append(u) or (_ for _ in ()).throw(SourceError('no thanks')))
self.assertEqual(seen, ['https://storage.itch-cdn.example/a.apk'])
def test_a_partial_match_before_the_key_cannot_hide_its_start(self):
key = 'abcdefghABCDEFGHi1234567890'
for text in (key[:8].upper() + key.upper(), key[:9] + key, 'xx' + key[:8] + key + 'yy'):
out = account._clean(text, key)
self.assertNotIn('1234567890', out, (text, out))
import time
t0 = time.monotonic(); account._clean(('abcdefgh' * 12500)[:100000], key)
self.assertLess(time.monotonic() - t0, 2.0)
def test_redaction_catches_every_spelling_of_the_key(self):
hexed = ''.join('%%%02X' % ord(c) for c in KEY)
for spelling in (KEY, KEY.lower(), KEY.upper(), hexed, hexed.lower(),
KEY[:5] + hexed[15:30] + KEY[10:], # partly encoded
KEY[:9], hexed[:30].lower(), # starts of it, 8+ characters
''.join('%%%02x' % ord(c) for c in KEY.lower()), # encoded after a case change
''.join('%%%02X' % ord(c) for c in KEY.swapcase()),
'%%%02x' % ord(KEY[0].swapcase()) + KEY[1:]): # one encoded, recased letter
cleaned = account._clean('before ' + spelling + ' after', KEY)
self.assertEqual(cleaned, 'before [API key] after', spelling)
double = ''.join('%%25%02X' % ord(c) for c in KEY) # percent-encoded twice
for spelling in (double, double.lower(), double.upper(), double[:5 * 9], # full, hex recased, 9-char start
''.join('%%25%02x' % ord(c) for c in KEY.lower()), # the key recased, then double-encoded
KEY[:4] + double[20:45] + hexed[27:33] + KEY[11:]): # literal, double and single mixed
self.assertEqual(account._clean('before ' + spelling + ' after', KEY), 'before [API key] after', spelling)
self.assertEqual(account._clean(double[:5 * 7], KEY), double[:5 * 7]) # 7 characters: kept
self.assertTrue(account._carries_key('https://cdn.example/a?x=' + double, KEY))
self.assertTrue(account._carries_key('https://cdn.example/' + double[:5 * 8].lower() + '/a', KEY))
self.assertEqual(account._clean('TESTKEY is short', KEY), 'TESTKEY is short') # 7 characters: kept
self.assertEqual(account._clean(KEY + KEY.lower(), KEY), '[API key][API key]')
for url in ('https://cdn.example/a?x=' + hexed, 'https://cdn.example/a?x=' + hexed.lower(),
'https://cdn.example/' + KEY.upper() + '/a', 'https://cdn.example/a?x=' + urllib.parse.quote_plus(KEY)):
self.assertTrue(account._carries_key(url, KEY), url)
self.assertFalse(account._carries_key('https://cdn.example/a?token=abc&Signature=x', KEY))
# Storage errors carrying an encoded or lowercased key lose it, keeping their kind.
self.connect()
self.storage()
with patch.object(_web, 'public_apk', side_effect=SourceLimited('slow ' + hexed.lower(), 42)):
with self.assertRaises(SourceLimited) as caught:
itch.download(self.owned_source(), '341190')
self.assertEqual((str(caught.exception), caught.exception.retry_after), ('slow [API key]', 42))
with patch.object(_web, 'public_apk', side_effect=SourceError('host ' + KEY.lower() + '.cdn')):
with self.assertRaisesRegex(SourceError, r'^host \[API key\]\.cdn$'):
itch.download(self.owned_source(), '341190')
def test_storage_errors_lose_the_key_but_keep_their_kind(self):
self.connect()
self.storage()
with patch.object(_web, 'public_apk', side_effect=SourceLimited('slow down ' + KEY, 90)):
with self.assertRaises(SourceLimited) as caught:
itch.download(self.owned_source(), '341190')
self.assertEqual((str(caught.exception), caught.exception.retry_after), ('slow down [API key]', 90))
with patch.object(_web, 'public_apk', side_effect=SourceError('Could not find the download server ' + KEY)):
with self.assertRaisesRegex(SourceError, r'^Could not find the download server \[API key\]$') as caught:
itch.download(self.owned_source(), '341190')
self.assertIsNone(caught.exception.__cause__)
self.assertTrue(caught.exception.__suppress_context__)
def test_broken_http_responses_are_readable_and_lose_the_key(self):
self.connect()
class Truncated(Reply):
def read(self, *args):
raise http.client.IncompleteRead(b'{"owned_keys": [' + KEY.encode(), 1000)
cases = [http.client.BadStatusLine('HTTP/1.1 ' + KEY + ' oops'),
lambda req, q: Truncated(req.full_url, 200, b''),
http.client.RemoteDisconnected('closed after ' + KEY)]
for case in cases:
account._forget()
self.web.route('api.itch.io/profile/owned-keys', case)
with self.assertRaisesRegex(SourceError, '^Could not reach itch.io: ') as caught:
account.owned_keys()
self.assertNotIn(KEY, str(caught.exception))
self.assertIsNone(caught.exception.__cause__)
self.assertTrue(caught.exception.__suppress_context__)
self.assertTrue(account.connected())
# The same on the download request, for each kind of answer it reads.
for case in (http.client.BadStatusLine('HTTP/1.1 ' + KEY),
lambda req, q: Truncated(req.full_url, 200, b'', {'Content-Type': 'application/json'}),
lambda req, q: Truncated(req.full_url, 200, b'', {'Content-Type': 'application/octet-stream'})):
self.web.route('api.itch.io/uploads/503/download', case)
with self.assertRaises(SourceError) as caught:
account.download(503, 9001)
self.assertNotIn(KEY, str(caught.exception))
self.assertRegex(str(caught.exception), '^Could not (reach itch.io|download APK): ')
def test_connecting_shares_one_deadline_across_addresses(self):
clock, deadlines = [1000.0], []
def slow(host, address, timeout, deadline):
deadlines.append(deadline)
clock[0] += 7 # each address hangs for 7 s before failing
raise socket.timeout('timed out')
with patch.object(_web.time, 'monotonic', lambda: clock[0]), patch.object(_web, '_connect', slow):
with self.assertRaises(OSError):
_web._public_connection('many.example', 60)
self.assertEqual(len(set(deadlines)), 1) # one deadline for every attempt
self.assertEqual(deadlines[0], 1000.0 + _web.CONNECT_BUDGET)
self.assertEqual(len(deadlines), 3) # 8 addresses, but only 21 s of trying, not 56
self.assertLessEqual(clock[0] - 1000.0, _web.CONNECT_BUDGET + 7)
deadlines.clear()
clock[0] = 1000.0
with patch.object(_web.time, 'monotonic', lambda: clock[0]), patch.object(_web, '_connect', slow):
with self.assertRaises(OSError):
_web._public_connection('many.example', 5) # the caller's shorter timeout wins
self.assertEqual(deadlines, [1005.0])
def test_a_stalled_handshake_stops_at_the_deadline(self):
# A local server that accepts and then says nothing: the TLS handshake never completes.
server = socket.socket()
server.bind(('127.0.0.1', 0))
server.listen(1)
self.addCleanup(server.close)
timeouts = []
real_settimeout = socket.socket.settimeout
def settimeout(sock, value):
timeouts.append(value)
return real_settimeout(sock, value)
start = time.monotonic()
# No thread is left behind to shut the connection later: the deadline is only socket timeouts.
with patch.object(_web.threading, 'Timer', side_effect=AssertionError('watchdog thread')), \
patch.object(socket.socket, 'settimeout', settimeout):
with self.assertRaises(OSError):
REAL_CONNECT('storage.itch-cdn.example', server.getsockname(), 60, start + 0.5)
self.assertLess(time.monotonic() - start, 3)
self.assertTrue(timeouts and all(0 < t <= 0.5 for t in timeouts), timeouts) # what's left, each step
with patch.object(_web.socket, 'create_connection', side_effect=AssertionError('connected')):
with self.assertRaises(socket.timeout): # a deadline already gone: no connection attempt at all
REAL_CONNECT('storage.itch-cdn.example', server.getsockname(), 60, time.monotonic() - 1)
def test_an_unreachable_address_falls_back_to_the_next_checked_one(self):
self.connect()
self.storage('https://dual.example/upload/503/OrbitGarden-Quest.apk')
self.web.route('dual.example/upload/503/OrbitGarden-Quest.apk', Reply('https://dual.example/', 200, apk_bytes()))
self.unreachable.add('2a04:4e42::81')
self.assertTrue(os.path.isfile(itch.download(self.owned_source(), '341190')['apk']))
self.assertEqual(self.connected, [('dual.example', ('2a04:4e42::81', 443)), ('dual.example', ('151.101.1.1', 443))])
self.unreachable.add('151.101.1.1')
with self.assertRaisesRegex(SourceError, '^Could not download APK: Network is unreachable$'):
itch.download(self.owned_source(), '341190')
def test_storage_refusals_are_download_errors_not_rate_limits(self):
self.connect()
self.storage()
self.web.route('storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk',
Reply('https://storage.itch-cdn.example/x', 403, b'denied'))
with self.assertRaisesRegex(SourceError, r"itch.io's download server refused the file \(HTTP 403\)") as caught:
itch.download(self.owned_source(), '341190')
self.assertNotIsInstance(caught.exception, SourceLimited)
self.assertTrue(account.connected())
self.assertEqual(_web.wait_time('https://api.itch.io/'), 0)
self.web.route('storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk',
Reply('https://storage.itch-cdn.example/x', 429, b'', {'Retry-After': '60'}))
with self.assertRaises(SourceLimited):
itch.download(self.owned_source(), '341190')
def test_a_key_revoked_after_details_were_cached_is_removed_at_download(self):
self.connect()
itch.details(self.owned_source(), '341190') # uploads and owned keys now cached
for code in (401, 403):
self.connect()
self.web.route('api.itch.io/uploads/503/download', Reply('https://api.itch.io/uploads/503/download', code, b''))
self.web.route('api.itch.io/profile', Reply('https://api.itch.io/profile', code, b''))
with self.assertRaisesRegex(account.BadKey, 'no longer accepts the saved API key'):
account.download(503, 9001)
self.assertFalse(account.path().exists())
self.web.route('api.itch.io/profile', {'user': {'username': 'player'}})
# The key still works, only this download is refused: keep the key, say so.
self.connect()
self.web.route('api.itch.io/uploads/503/download', Reply('https://api.itch.io/uploads/503/download', 403, b''))
with self.assertRaisesRegex(SourceError, r'refused this request \(HTTP 403\)') as caught:
account.download(503, 9001)
self.assertNotIsInstance(caught.exception, (account.BadKey, SourceLimited))
self.assertTrue(account.connected())
def test_a_disconnect_during_the_key_check_wins(self):
checking, release = threading.Event(), threading.Event()
def slow_profile(req, query):
checking.set()
release.wait(5)
return {'user': {'username': 'player'}}
self.web.route('api.itch.io/profile', slow_profile)
outcome = {}
def run():
try:
outcome['result'] = account.connect(KEY)
except SourceError as e:
outcome['error'] = e
worker = threading.Thread(target=run)
worker.start()
self.assertTrue(checking.wait(5))
account.disconnect()
release.set()
worker.join(5)
self.assertRegex(str(outcome.get('error')), 'changed while the key was being checked; nothing was saved')
self.assertFalse(account.path().exists())
self.assertFalse(account.connected())
def test_an_old_refusal_does_not_undo_a_reconnect_with_the_same_key(self):
self.connect()
listing, release, profile_ok = threading.Event(), threading.Event(), [True]
def slow_owned(req, query):
listing.set()
release.wait(5)
return Reply(req.full_url, 401, b'')
self.web.route('api.itch.io/profile/owned-keys', slow_owned)
self.web.route('api.itch.io/profile', lambda req, q: {'user': {'username': 'player'}} if profile_ok[0]
else Reply(req.full_url, 401, b''))
outcome = {}
def run():
try:
account.owned_keys()
except SourceError as e:
outcome['error'] = e
worker = threading.Thread(target=run)
worker.start()
self.assertTrue(listing.wait(5))
account.disconnect()
account.connect(KEY) # same key, connected again while the old listing waits
profile_ok[0] = False # the old listing's confirmation now sees a refusal
release.set()
worker.join(5)
self.assertIsInstance(outcome.get('error'), account.BadKey)
self.assertTrue(account.path().exists()) # the fresh connection survives
self.assertEqual(json.loads(account.path().read_text())['key'], KEY)
def test_redacted_text_is_not_mistaken_for_a_key_error(self):
self.connect()
before = len(self.web.urls('api.itch.io/profile'))
self.web.route('api.itch.io/profile/owned-keys', {'errors': ['Invalid request: ' + KEY]})
with self.assertRaisesRegex(SourceError, r'^itch.io: Invalid request: \[API key\]$') as caught:
account.owned_keys()
self.assertNotIsInstance(caught.exception, account.BadKey)
self.assertTrue(account.connected())
self.assertEqual(len(self.web.urls('api.itch.io/profile')), before) # not even a key check needed
def test_an_invalid_key_error_elsewhere_is_confirmed_before_deleting(self):
self.connect()
self.web.route('api.itch.io/profile/owned-keys', {'errors': ['invalid key']})
with self.assertRaisesRegex(SourceError, '^itch.io: invalid key$') as caught:
account.owned_keys() # /profile still accepts the key: keep it
self.assertNotIsInstance(caught.exception, account.BadKey)
self.assertTrue(account.connected())
account._forget()
self.web.route('api.itch.io/profile', {'errors': ['invalid key']})
with self.assertRaisesRegex(account.BadKey, 'no longer accepts'):
account.owned_keys()
self.assertFalse(account.connected())
def test_concurrent_connects_never_share_a_temporary_file(self):
made = []
real = account.tempfile.mkstemp
def mkstemp(*args, **kwargs):
fd, name = real(*args, **kwargs)
made.append(name)
if os.name == 'posix':
self.assertEqual(os.stat(name).st_mode & 0o777, 0o600)
return fd, name
with patch.object(account.tempfile, 'mkstemp', mkstemp):
workers = [threading.Thread(target=lambda k=k: self.swallow(account.connect, k)) for k in (KEY, OTHER) * 3]
for w in workers:
w.start()
for w in workers:
w.join(5)
self.assertEqual(len(made), len(set(made)))
self.assertIn(json.loads(account.path().read_text())['key'], (KEY, OTHER))
self.assertEqual([p.name for p in account.path().parent.iterdir()], ['itch-account.json'])
@staticmethod
def swallow(fn, *args):
try:
fn(*args)
except SourceError:
pass # a connect overtaken by another one refuses to save: fine here
def test_existing_hosts_still_strip_authorization_on_redirect(self):
req = urllib.request.Request('https://api.github.com/a', headers={'Authorization': 'Bearer x'})
nxt = _web.Redirect(('api.github.com',)).redirect_request(req, None, 302, 'Found', {}, 'https://api.github.com/b')
self.assertFalse(nxt.has_header('Authorization'))
with self.assertRaises(SourceError):
_web.Redirect(('api.github.com',)).redirect_request(req, None, 302, 'Found', {}, 'https://evil.example/b')
def test_install_uses_the_normal_apk_install_path(self):
import frame_android
self.connect()
self.storage()
calls = []
def install(apk_path, flatscreen=None, name=None, source=None, icon_png=None, xr_compat=None, artwork=None):
with zipfile.ZipFile(apk_path) as z:
self.assertIn('AndroidManifest.xml', z.namelist())
calls.append((name, source))
return {'package': 'com.maker.orbit', 'message': 'Installed'}
with patch.object(frame_android, 'install', install):
result = search.install(itch.OWNED, '341190')
self.assertEqual(calls, [('Orbit Garden VR', 'itch.io (my purchases)')])
self.assertEqual(result['package'], 'com.maker.orbit')
# -- errors never carry the key ------------------------------------------------
def test_errors_are_readable_and_never_contain_the_key(self):
self.connect()
cases = [
(Reply('https://api.itch.io/profile/owned-keys', 429, b'', {'Retry-After': '120'}),
SourceLimited, '^itch.io is limiting requests; try again in 2 minutes$'),
(Reply('https://api.itch.io/profile/owned-keys', 503, b''), SourceError, 'HTTP error 503; try again later'),
(urllib.error.URLError('connect to api.itch.io with ' + KEY + ' failed'), SourceError,
r'^Could not reach itch.io: connect to api.itch.io with \[API key\] failed$'),
(OSError('timed out'), SourceError, '^Could not reach itch.io: timed out$'),
(Reply('https://api.itch.io/profile/owned-keys', 200, b'<html>'), SourceError, 'invalid response'),
({'errors': ['echo ' + KEY]}, SourceError, r'^itch.io: echo \[API key\]$'),
# The key straddles the 200-character cut: no piece of it may survive.
({'errors': ['x' * 180, KEY]}, SourceError, r'^itch.io: x{180}; \[API key\]$'),
({'errors': ['x' * 190 + KEY]}, SourceError, r'^itch.io: x{190}\[API key\]$'),
]
self.assertEqual(account._clean('cut at ' + KEY[:9], KEY), 'cut at [API key]') # a truncated key's start
stderr = io.StringIO()
with contextlib.redirect_stderr(stderr):
for reply, kind, message in cases:
account._forget()
_web._limited.clear()
self.web.route('api.itch.io/profile/owned-keys', reply)
with self.assertRaisesRegex(kind, message) as caught:
account.owned_keys()
self.assertNotIn(KEY, str(caught.exception))
self.assertIsNone(caught.exception.__cause__)
self.assertTrue(account.connected()) # not the key's fault: stays connected
# The store's search reports the failure as a non-fatal source status.
account._forget()
result = search.search('', source=itch.OWNED)
self.assertEqual(result['sources'][0]['status'], 'error')
self.assertNotIn(KEY, json.dumps(result) + stderr.getvalue())
def test_server_api_connects_and_never_echoes_the_key(self):
httpd = server.ThreadingHTTPServer(('127.0.0.1', 0), server.Handler)
threading.Thread(target=httpd.serve_forever, daemon=True).start()
self.addCleanup(httpd.server_close)
self.addCleanup(httpd.shutdown)
def request(method, path, body=None):
c = http.client.HTTPConnection('127.0.0.1', httpd.server_port)
c.request(method, path, json.dumps(body) if body is not None else None,
{'X-Frame-UI': '1', 'Content-Type': 'application/json'})
r = c.getresponse()
out = r.status, r.read().decode()
c.close()
return out
stderr = io.StringIO()
with contextlib.redirect_stderr(stderr):
self.web.route('api.itch.io/profile', Reply('https://api.itch.io/profile', 401, b''))
bad = request('POST', '/api/sources', {'action': 'itch-connect', 'key': KEY})
self.web.route('api.itch.io/profile', {'user': {'username': 'player'}})
good = request('POST', '/api/sources', {'action': 'itch-connect', 'key': KEY})
listed = request('GET', '/api/sources')
empty = request('POST', '/api/sources', {'action': 'itch-connect', 'key': ' '})
gone = request('POST', '/api/sources', {'action': 'itch-disconnect'})
self.assertEqual(bad[0], 400)
self.assertIn('did not accept the API key', json.loads(bad[1])['error'])
self.assertEqual(good[0], 200)
self.assertEqual(json.loads(good[1])['itch'], {'connected': True, 'username': 'player'})
self.assertEqual(json.loads(listed[1])['itch'], {'connected': True, 'username': 'player'})
self.assertIn(itch.OWNED, [s['id'] for s in json.loads(listed[1])['sources']])
self.assertEqual(empty[0], 400)
self.assertEqual(json.loads(gone[1])['itch'], {'connected': False, 'username': None})
self.assertFalse(account.path().exists())
for text in (bad[1], good[1], listed[1], empty[1], gone[1], stderr.getvalue()):
self.assertNotIn(KEY, text)
if __name__ == '__main__':
unittest.main()
+1 -17
View File
@@ -46,17 +46,6 @@ def explain(msg):
return None
class FailureDetail(unittest.TestCase):
def test_an_address_that_failed_says_why_in_a_few_words(self):
# #87: the banner put each address's detail after the headline, and with the whole
# message as the detail it said everything twice ("...may be answering here..").
self.assertEqual(fl.failure_detail({"message": "A long sentence.", "short": "SSH identity changed"}),
"SSH identity changed")
self.assertEqual(fl.failure_detail({"message": "The connection to the Frame dropped. Try again."}),
"The connection to the Frame dropped. Try again.")
self.assertEqual(fl.failure_detail(None), "SSH failed")
class Probe(unittest.TestCase):
def test_answers_refusals_and_unknown_names(self):
with socket.socket() as srv:
@@ -211,8 +200,7 @@ class Connecting(unittest.TestCase):
rows = {p["host"]: p for p in s["probes"]}
self.assertEqual(rows["nothing.invalid"]["state"], "unresolved")
self.assertEqual(rows["::1"]["state"], "sshfailed")
# Its row says why in a few words: the banner's headline already has the whole sentence (#87).
self.assertEqual(rows["::1"]["detail"], "SSH identity changed")
self.assertIn("different SSH identity", rows["::1"]["detail"])
# Every ssh command was pointed at the winner, with the host key pinned per device.
alias, opts = self.routes[-1]
self.assertEqual(alias, "frame-t")
@@ -272,8 +260,6 @@ class Connecting(unittest.TestCase):
self.assertEqual((s["phase"], s["error"]["stage"]), ("failed", "login"))
self.assertIn("SSH key", s["error"]["message"])
self.assertEqual(s["error"]["auth"], "denied") # the page offers Set Up Connection
self.assertEqual(s["error"]["short"], "This computer's key refused") # the header pill's words (#87)
self.assertEqual(s["probes"][0]["detail"], "This computer's key refused")
def test_pinned_identity_is_checked_strictly(self):
d = self.device("localhost")
@@ -363,7 +349,6 @@ class Connecting(unittest.TestCase):
self.assertEqual((s["phase"], s["error"]["stage"]), ("failed", "identity"))
self.assertEqual(s["error"]["hostkey"]["state"], "missing")
self.assertIn("missing or can't be read", s["error"]["message"])
self.assertEqual(s["error"]["short"], "Saved SSH identity missing")
view = next(x for x in fl.devices_view(self.link)["devices"] if x["id"] == d["id"])
self.assertTrue(view["identity_lost"])
scan = fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)["scan"]
@@ -649,7 +634,6 @@ class Connecting(unittest.TestCase):
self.assertEqual((shown["state"], shown["type"], shown["host"]), ("changed", "ssh-ed25519", "localhost"))
self.assertTrue(shown["fingerprint"].startswith("SHA256:"))
self.assertIn("Devices tab", s["error"]["message"])
self.assertEqual(s["error"]["short"], "SSH identity changed")
self.assertEqual(fd.known_hosts(d["id"]).read_text(), saved) # never accepted on its own
r = self.command(d)
self.assertEqual(r.returncode, 255)
-136
View File
@@ -1,136 +0,0 @@
"""The connection pill, the offline banner and the Devices tab's selection: the page's own
functions from ui/index.html, run in Node against a minimal DOM (#87)."""
import sandbox # noqa: F401 (first: keeps tests off real data and services)
import pathlib
import re
import shutil
import subprocess
import unittest
ROOT = pathlib.Path(__file__).resolve().parents[1]
PAGE = (ROOT / "ui/index.html").read_text(encoding="utf-8")
def between(start, end):
return PAGE[PAGE.index(start):PAGE.index(end)]
SETUP = r'''
const assert = require('node:assert/strict');
const elements = new Map();
const $ = id => {
if (!elements.has(id)) elements.set(id, {id, textContent: '', innerHTML: '', title: '', hidden: true, value: ''});
return elements.get(id);
};
const esc = s => String(s).replace(/[&<>"']/g, c => ({'&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;'}[c]));
const serverNow = () => 1000;
const activeStage = s => (s.stages || []).find(x => x.state === 'active');
const ago = () => 'a minute ago';
const openTrust = () => {};
const window = {};
let online = false;
const link = {live: true, s: null};
const dv = {list: [], sel: null, data: null, found: 'x', editing: 'y'};
let rendered = 0;
const renderDevices = () => { rendered++; };
let refuse = false;
const act = async (label, fn) => refuse ? undefined : fn();
const api = async () => ({ok: true});
const STAGES = [{id: 'find', label: 'Finding the headset', state: 'done'},
{id: 'identity', label: "Checking the headset's identity", state: 'failed'}];
const changed = {phase: 'failed', retry_at: 1012, device: {id: 'd1', name: 'Office Frame'}, network: {name: 'Home'},
stages: STAGES, probes: [{host: 'frame.local', detail: 'SSH identity changed'}],
error: {stage: 'identity', short: 'SSH identity changed', hostkey: {state: 'changed'},
message: 'This address answered with a different SSH identity from the one saved for Office Frame. '
+ 'If not, a different device may be answering here.'}};
'''
@unittest.skipUnless(shutil.which("node"), "Node runs the page's own functions")
class LinkUI(unittest.TestCase):
def run_page(self, checks):
code = (between("function netLabel(s) {", "function renderConnDlg() {")
+ between("async function useDevice(id) {", "async function devAction(")
+ between("function deviceStatus(d) {", "function renderDevices() {"))
script = SETUP + code + "\n(async () => {\n" + checks + "\n})().catch(e => { console.error(e); process.exitCode = 1; });"
r = subprocess.run(["node", "-e", script], capture_output=True, text=True)
self.assertEqual(r.returncode, 0, r.stderr)
def test_the_pill_leads_with_the_reason(self):
# The header pill cut off "Offline: checking the headset's identity failed · retry in…":
# the reason comes first, in a few words, and only the countdown gets cut.
self.run_page(r'''
link.s = changed;
renderPill();
assert.match($('conn').innerHTML, /<b>Office Frame<\/b><span>SSH identity changed · retry in 12 s<\/span>/);
assert.equal($('conn').title, 'Office Frame is offline: SSH identity changed · retry in 12 s. Click for details.');
// Without a short reason: the stage that failed.
link.s = {...changed, retry_at: null, error: {stage: 'identity', message: 'Something.'}};
renderPill();
assert.match($('conn').innerHTML, /<span>Checking the headset&#39;s identity failed<\/span>/);
''')
def test_the_banner_never_doubles_a_full_stop(self):
# "…may be answering here.." and "…(Developer Mode must be on).." (#87).
self.run_page(r'''
link.s = changed;
renderBanner();
assert.equal($('offDetail').textContent, 'Office Frame on Home. Tried frame.local: SSH identity changed. Trying again in 12 s.');
link.s = {...changed, probes: [
{host: 'frame.local', detail: "The Frame didn't accept this computer's SSH key. Set Up Connection pairs this computer with it again (Developer Mode must be on)."},
{host: '100.64.0.2', detail: 'No answer'}]};
renderBanner();
const text = $('offDetail').textContent;
assert.ok(!/\.\./.test(text), text);
assert.ok(text.includes("frame.local: the Frame didn't accept"), text);
assert.ok(text.includes('must be on); 100.64.0.2: no answer. Trying'), text);
assert.equal(asClause('SSH identity changed.'), 'SSH identity changed'); // abbreviations keep their case
assert.equal(asClause('Answered in 3 ms'), 'answered in 3 ms');
''')
def test_the_devices_list_keeps_the_whole_message_within_reach(self):
self.run_page(r'''
link.s = changed;
const [dot, text, full] = deviceStatus({active: true, addresses: []});
assert.equal(dot, 'off');
assert.equal(text, 'SSH identity changed');
assert.equal(full, changed.error.message); // the row's tooltip
''')
def test_using_a_headset_selects_it(self):
# After "Use this headset" on Office Frame, the last headset's details stayed on show.
self.run_page(r'''
dv.list = [{id: 'd0', name: 'Living Room Frame', active: true}, {id: 'd1', name: 'Office Frame', active: false}];
dv.sel = 'd0';
await useDevice('d1');
assert.equal(dv.sel, 'd1');
assert.equal(dv.found, null);
assert.equal(dv.editing, null);
assert.equal(rendered, 1);
// Refused (an install is running): the selection stays, and the header shows the one in use.
dv.list = [{id: 'd0', name: 'Living Room Frame', active: true}, {id: 'd2', name: 'Spare', active: false}];
dv.sel = 'd0'; refuse = true;
await useDevice('d2');
assert.equal(dv.sel, 'd0');
assert.equal($('devSel').value, 'd0');
''')
class HeaderLayout(unittest.TestCase):
def test_the_devices_tab_explanation_wraps(self):
# It was cut to one line with no way to read the rest.
self.assertIn(".dev-problem .s { white-space: normal; }", PAGE)
self.assertEqual(len(re.findall(r'<div class="item dev-problem" role="alert">', PAGE)), 2)
def test_the_header_buttons_dont_give_way(self):
# Refresh overhung the right edge at 1280 px: the pill (and the headset picker) shrink, not the buttons.
header = between("<header>", "</header>")
end = header[header.index('<div class="hdr-end">'):]
for el in ('id="conn"', 'id="devSel"', 'id="battChip"', 'id="reportBtn"', 'id="refreshAll"'):
self.assertIn(el, end)
self.assertIn(".hdr-end > :not(.pill, .devsel) { flex: none; }", PAGE)
self.assertRegex(PAGE, r"\.hdr-end \{[^}]*min-width: 0;[^}]*margin-left: auto;")
if __name__ == "__main__":
unittest.main()
-1265
View File
File diff suppressed because it is too large. Load diff
-1144
View File
File diff suppressed because it is too large. Load diff
+6 -134
View File
@@ -138,7 +138,7 @@ class Gates(Base):
tm.diagnostic("POST /api/android install", RuntimeError("boom"))
tm.diagnostic("POST /api/comfort status", RuntimeError("ssh exited 255")) # the command's own exit code?
self.assertEqual([e["properties"]["error_category"] for e in self.queued()],
["frame_unreachable", "internal_error", "other", "other"])
["frame_unreachable", "other", "other", "other"])
def test_page_events_are_checked(self):
self.assertTrue(tm.page_event({"event": "tab_viewed", "properties": {"tab": "android", "extra": "x"}})["queued"])
@@ -164,38 +164,6 @@ class Lifecycle(Base):
tm.frame_seen("20260922.1", "3.8")
self.assertEqual(len(self.queued()), 1)
def test_frame_once_per_headset_with_saved_count(self):
tm.frame_seen("20260922.1", "3.8", "0123456789abcdef", 2)
tm.frame_seen("20260922.1", "3.8", "0123456789abcdef", 2)
tm.frame_seen("20260922.1", "3.8", "fedcba9876543210", 2)
props = [e["properties"] for e in self.queued()]
self.assertEqual([(p["headset_id"], p["headsets_saved"]) for p in props],
[("0123456789abcdef", 2), ("fedcba9876543210", 2)])
def test_frame_many_headsets_are_not_sent_twice(self):
ids = ["%016x" % n for n in range(25)]
for _ in range(2):
for h in ids:
tm.frame_seen("20260922.1", "3.8", h, 25)
self.assertEqual(len(self.queued()), 25)
def test_frame_headset_id_must_be_the_hash(self):
tm.frame_seen("20260922.1", "3.8", "/etc/machine-id contents", None)
p = self.queued()[0]["properties"]
self.assertNotIn("headset_id", p)
self.assertNotIn("headsets_saved", p)
class HeadsetId(unittest.TestCase):
def test_hash_of_machine_id_and_none_without_one(self):
import hashlib
import frame_status
with mock.patch.object(frame_status, "read", return_value="abc123"):
self.assertEqual(frame_status.headset_id(),
hashlib.sha256(b"frame-control-headset:abc123").hexdigest()[:16])
with mock.patch.object(frame_status, "read", return_value=None):
self.assertIsNone(frame_status.headset_id())
class Sending(Base):
def serve(self, status=200):
@@ -295,72 +263,6 @@ class Scrub(unittest.TestCase):
self.assertEqual(tm.categorize(FileNotFoundError(2, "No such file or directory", "ssh"))[0], "tool_missing")
self.assertEqual(tm.categorize("[WinError 2] The system cannot find the file specified")[0], "tool_missing")
def test_apk_install_failures_that_were_other(self):
"""What frame_android.install can raise, worded as it is, that 0.4.x sent as 'other'
(or as frame_unreachable, when it was the Steam client or a slow copy)."""
tail = 'Traceback (most recent call last):\n File "<stdin>", line 19, in target_ws\n'
cases = {
"steam_not_running": [
"SharedJSContext not found: is the Steam client running?",
tail + "urllib.error.URLError: <urlopen error [Errno 111] Connection refused>",
tail + "ConnectionRefusedError: [Errno 111] Connection refused",
],
"steam_not_responding": [
"websocket handshake failed",
tail + "TimeoutError: timed out",
tail + "socket.timeout: timed out",
tail + ' File "<stdin>", line 45, in _read\nEOFError',
],
"steam_js_error": ['JS error: {"exceptionId": 1, "text": "Uncaught", "exception": '
'{"description": "Error: Steam artwork API unavailable; installation is incomplete"}}'],
"artwork_failed": [
"Steam artwork renderer did not produce every slot",
tail + "ValueError: incomplete artwork render",
tail + "ValueError: hero render is 0 bytes of png",
"could not prepare artwork: unknown artwork slot",
tail + "ValueError: all five Steam artwork slots are required",
],
"copy_timeout": ["copying Big Game (timed out).apk to the Frame timed out",
"copying launch.sh to the Frame timed out"],
"copy_failed": [
"copying game.apk to the Frame failed: rsync: connection unexpectedly closed (0 bytes received "
"so far) [sender]\nrsync error: error in rsync protocol data stream (code 12) at io.c(231)",
"copying game.apk to the Frame failed: ",
],
"apk_unreadable": [
"x.apk: not a readable APK: [Errno 2] No such file or directory: 'x.apk'",
"x.apk: could not read the APK manifest: Error -3 while decompressing data: invalid block type",
"x.apk: the APK manifest has no package name",
"x.apk: lib/arm64-v8a/libx.so in the APK uses a compression Android does not",
"x.apk: AndroidManifest.xml in the APK is 40 MB, more than a real one (16 MB)",
],
}
for category, messages in cases.items():
for message in messages:
self.assertEqual(tm.categorize(message)[0], category, message)
# Still where they were: ssh's own failures, and a full disk during a copy.
self.assertEqual(tm.categorize("timed out talking to frame")[0], "frame_unreachable")
self.assertEqual(tm.categorize("copying a.apk to the Frame failed: ssh: connect to host 10.0.0.2 "
"port 22: Connection refused")[0], "frame_unreachable")
self.assertEqual(tm.categorize("copying a.apk to the Frame failed: rsync: write failed on "
"\"app.apk.part\": No space left on device (28)")[0], "frame_disk_full")
self.assertEqual(tm.categorize("download failed: timed out")[0], "download_failed")
def test_python_errors_are_internal_whatever_they_say(self):
import json as _json
for error in (KeyError("paths"), TypeError("'NoneType' object is not subscriptable"),
AttributeError("x"), _json.JSONDecodeError("Expecting value", "", 0),
UnicodeDecodeError("utf-8", b"\xff", 0, 1, "invalid start byte"),
KeyError("timed out")):
self.assertEqual(tm.categorize(error)[0], "internal_error", repr(error))
self.assertEqual(tm.categorize("KeyError: 'paths'")[0], "other") # only the exception itself
self.assertEqual(tm.categorize(OSError(28, "No space left on device"))[0], "frame_disk_full")
def test_privacy_doc_lists_every_category(self):
doc = (ROOT / "docs" / "privacy.md").read_text()
for name in {n for n, _ in tm.CATEGORIES} | {"internal_error", "other"}:
self.assertIn(f"`{name}`", doc)
class Compat(Base):
def test_reports_are_shared_only_after_opting_in_without_file_names(self):
@@ -474,17 +376,14 @@ class LinkFailureProvenance(Base):
c.close()
return result
def serve(self):
def test_routes_that_rewrap_a_link_failure_keep_its_mark(self):
# Review round 4: /api/android and /api/titles re-raise frame_android's FrameError as a
# Failure; after the status poll's first connection failure, theirs are held back too,
# also past the 10-minute window.
self.httpd = self.server.ThreadingHTTPServer(("127.0.0.1", 0), self.server.Handler)
threading.Thread(target=self.httpd.serve_forever, daemon=True).start()
self.addCleanup(self.httpd.server_close)
self.addCleanup(self.httpd.shutdown)
def test_routes_that_rewrap_a_link_failure_keep_its_mark(self):
# Review round 4: /api/android and /api/titles re-raise frame_android's FrameError as a
# Failure; after the first connection failure, theirs are held back too, also past the
# 10-minute window.
self.serve()
down = subprocess.CompletedProcess(["ssh"], 255, "", "ssh: connect to host 10.0.0.5 port 22: Connection timed out\r\n")
meta = {"label": "Test App", "game_id": 5, "shortcut": None}
with mock.patch.object(frame_host, "run_ssh", return_value=down), \
@@ -500,34 +399,7 @@ class LinkFailureProvenance(Base):
self.assertEqual(self.request("/api/android", {"action": "launch", "package": "org.test"})[0], 502)
sent = [(e["properties"]["where"], e["properties"]["error_category"]) for e in self.queued()
if e["event"] == "$exception"]
# The comfort poll's own failure isn't one (see the next test).
self.assertEqual(sent, [("POST /api/android launch", "frame_unreachable"), ("POST /api/android launch", "other")])
def test_the_comfort_poll_meeting_an_unreachable_frame_is_not_sent(self):
# 0.4.2: most reports were this poll (every 5 s) finding the headset asleep, away or
# refusing ssh. Like GET /api/status, it isn't reported; a session started or cancelled
# by hand still is, once, and a real failure of the poll is too.
self.serve()
for stderr in ("ssh: connect to host 10.0.0.5 port 22: Connection timed out\r\n",
"banner exchange: Connection to UNKNOWN port -1: Connection refused\r\n",
"ssh: Could not resolve hostname frame: nodename nor servname provided\r\n",
"@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @\r\nHost key verification failed.\r\n",
"No ED25519 host key is known for frame-control-f7b4e9ac and you have requested strict "
"checking.\r\nHost key verification failed.\r\n",
"steamos@192.168.1.9: Permission denied (publickey,password).\r\n"):
down = subprocess.CompletedProcess(["ssh"], 255, "", stderr)
with mock.patch.object(frame_host, "run_ssh", return_value=down):
self.assertGreaterEqual(self.request("/api/comfort", {"action": "status"})[0], 500, stderr)
self.assertEqual(self.queued(), [])
down = subprocess.CompletedProcess(["ssh"], 255, "", "ssh: connect to host 10.0.0.5 port 22: Host is down\r\n")
with mock.patch.object(frame_host, "run_ssh", return_value=down):
self.request("/api/comfort", {"action": "cancel"})
broken = subprocess.CompletedProcess(["ssh"], 1, "", "Traceback (most recent call last):\nOSError: boom\n")
with mock.patch.object(frame_host, "run_ssh", return_value=broken):
self.assertEqual(self.request("/api/comfort", {"action": "status"})[0], 502)
sent = [(e["properties"]["where"], e["properties"]["error_category"]) for e in self.queued()
if e["event"] == "$exception"]
self.assertEqual(sent, [("POST /api/comfort cancel", "frame_unreachable"), ("POST /api/comfort status", "other")])
self.assertEqual(sent, [("POST /api/comfort status", "frame_unreachable"), ("POST /api/android launch", "other")])
def test_download_failures_after_a_poll_failure_are_still_sent(self):
# Review round 3: these file names, in the real checksum message through the web-link worker,
+3 -1
View File
@@ -32,7 +32,9 @@ Entry dict (missing facts are None, never guessed)
Rules
-----
- Only sources that distribute APKs with the developer's consent: free listings,
never paid apps re-hosted, no licence or entitlement workarounds.
never paid apps re-hosted, no licence or entitlement workarounds. The one paid
exception is a store's own download of a game the user bought there: a source
marked 'owned': True whose entries carry 'owned': True (itch.io purchases).
- Honour each site's terms and robots rules; if automated download isn't allowed,
return entries with 'downloadable': False and a 'page' link instead.
- Cache indexes under frame_host.cache_dir('apk-sources'); send a clear
+394
View File
@@ -0,0 +1,394 @@
"""The user's own itch.io account: an API key, the games it owns and their Android uploads.
Opt-in (store > sources > Connect itch.io). The key is kept in
frame_host.data_dir('itch-account.json') with mode 0600 and is only ever sent to
https://api.itch.io, in an Authorization header. Downloads end on itch.io's storage
host, fetched without the key from a checked public address (_web.public_apk).
The key never goes into a URL, an error, a log line or an API response. Owned-key
listings stay in memory, never in the disk cache.
API: https://itch.io/docs/api/serverside. See docs/apk-repos.md.
"""
import hashlib, http.client, json, os, re, tempfile, threading, time, urllib.error, urllib.parse, urllib.request, zipfile
import frame_host
from . import SourceError, SourceLimited, _web
API = 'https://api.itch.io'
HOSTS = ('api.itch.io',)
NAME = 'itch.io'
TTL = 600 # owned keys and uploads are re-read after ten minutes
PER_PAGE = 500 # the documented maximum for /profile/owned-keys
PAGES = 40 # stop after 20,000 owned keys
KEY = re.compile(r'[A-Za-z0-9_.-]{16,256}')
BAD_KEY = ('itch.io did not accept the API key. Create one in itch.io settings > '
'API keys and connect again')
REVOKED = ('itch.io no longer accepts the saved API key, so it was removed. '
'Create a new one in itch.io settings > API keys and connect again')
REDIRECTS = (301, 302, 303, 307, 308)
# Everything a connection or response can raise. http.client's errors (IncompleteRead,
# BadStatusLine...) are not OSErrors, and BadStatusLine repeats what the server sent.
NETWORK = (OSError, ValueError, http.client.HTTPException)
_lock = threading.Lock() # the cache
_cache = {} # (generation, key digest, what) -> (time, value)
_account = threading.Lock() # saving and deleting the key file
_generation = [0] # bumped by every save and delete: a connect that started earlier doesn't save
class BadKey(SourceError):
"""itch.io refused the key: wrong, revoked or deleted."""
def path():
return frame_host.data_dir('itch-account.json')
def _load():
try:
data = json.loads(path().read_text())
except (OSError, ValueError):
return None
if not isinstance(data, dict) or not isinstance(data.get('key'), str) or not KEY.fullmatch(data['key']):
return None
return data
def status():
"""{'connected', 'username'}: what the page may know. Never the key; no network."""
data = _load()
return {'connected': bool(data), 'username': data.get('username') if data else None}
def connected():
return _load() is not None
def _key():
data = _load()
if not data:
raise SourceError('Connect your itch.io account first (store > sources > Connect itch.io)')
return data['key']
def _save(key, username):
"""Write the key file through a fresh 0600 temporary file (mkstemp), then move it into place."""
target = path()
target.parent.mkdir(parents=True, exist_ok=True)
fd, tmp = tempfile.mkstemp(dir=str(target.parent), prefix='.itch-account-', suffix='.tmp')
try:
with os.fdopen(fd, 'w') as f:
os.chmod(tmp, 0o600) # mkstemp already does on POSIX; say so for every platform
json.dump({'key': key, 'username': username}, f)
os.replace(tmp, str(target))
finally:
if os.path.exists(tmp):
os.remove(tmp)
def _forget():
with _lock:
_cache.clear()
def _clean(message, key):
"""message without the key, or any start of it (8+ characters, as truncation leaves), in
any letter case, with any of its characters written literally, percent-encoded (%4B or
%4b) or percent-encoded twice (%254B, %254b). The only redaction used here: every message
that might hold the key goes through it. (The key's characters are letters, digits and
_.-, so +/space encoding can't change it.)"""
if not key:
return message
# Each character: itself, or the percent-encoding (once or twice) of either of its letter cases.
tokens = ['(?:%s)' % '|'.join([re.escape(c)] + ['%%(?:25)?%02x' % ord(v) for v in sorted({c.lower(), c.upper()})])
for c in key]
start, rest = re.compile(''.join(tokens[:8]), re.I), [re.compile(t, re.I) for t in tokens[8:]]
spans, pos = [], 0
while True:
found = start.search(message, pos)
if not found:
break
end = found.end()
for token in rest: # as much more of the key as follows
more = token.match(message, end)
if not more:
break
end = more.end()
spans.append((found.start(), end))
# Look again from the next character, not the end: a partial match just before the
# real key can overlap its start. Overlapping spans merge below.
pos = found.start() + 1
out, pos = [], 0
for a, b in spans: # in order of start; only overlapping spans merge
if out and a < pos:
pos = max(pos, b)
continue
out += [message[pos:a], '[API key]']
pos = b
return ''.join(out) + message[pos:]
class _Stay(urllib.request.HTTPRedirectHandler):
"""Don't follow: the caller decides where a redirect may go, and never sends the key there."""
def redirect_request(self, req, fp, code, msg, headers, newurl):
return None
def _open(url, key, accept='application/json'):
"""GET url on api.itch.io with the key. Redirects come back as HTTPError 30x."""
_web.checked_url(url, HOSTS)
wait = _web.wait_time(url)
if wait:
raise _web.limited_error(NAME, wait)
return urllib.request.build_opener(_Stay()).open(urllib.request.Request(
url, headers={'User-Agent': _web.UA, 'Authorization': 'Bearer ' + key, 'Accept': accept}), timeout=60)
def _refused(code, key, api_path, message=None):
"""A refusal that may be about the key (401/403, or an "invalid key" error): BadKey only
once /profile confirms the key itself is refused; otherwise it's this request (a game or
upload this purchase doesn't cover), and the key stays."""
if api_path != '/profile':
_get('/profile', key) # raises BadKey if the key is the problem
raise SourceError(message or 'itch.io refused this request (HTTP %d); your purchase may not include it' % code)
raise BadKey(BAD_KEY)
def _network_error(e, key):
reason = getattr(e, 'reason', None) or e
return SourceError(_clean('Could not reach itch.io: ' + str(reason), key))
BAD_KEY_TEXT = re.compile(r'\b(invalid|revoked|expired)\b.*\bkey\b|\bkey\b.*\b(invalid|revoked|expired)\b', re.I)
def _checked(raw, key, api_path):
"""Decoded JSON object from api.itch.io, or SourceError/BadKey from its errors array."""
if len(raw) > 8 * 1024 * 1024:
raise SourceError('itch.io response is too large')
try:
data = json.loads(raw)
except ValueError:
raise SourceError('itch.io returned an invalid response') from None
if not isinstance(data, dict):
raise SourceError('itch.io returned an invalid response')
errors = data.get('errors')
if errors: # documented: errors is an array of strings
messages = [str(m) for m in errors if isinstance(m, (str, int))]
# Judged on itch.io's own words: redacted text says "[API key]", which would look like a key error.
about_key = any(BAD_KEY_TEXT.search(m) and 'download key' not in m.lower() for m in messages)
# Each message loses the key before joining and shortening, so no part of it survives the cut.
text = _clean('; '.join(_clean(m, key) for m in messages)[:200], key)
if about_key:
_refused(None, key, api_path, 'itch.io: ' + text)
raise SourceError('itch.io: ' + (text or 'request refused'))
return data
def _get(api_path, key, params=None):
"""JSON from api.itch.io; SourceError with a readable message (never the key) on failure."""
url = API + api_path + ('?' + urllib.parse.urlencode(params) if params else '')
try:
with _open(url, key) as r:
raw = r.read(8 * 1024 * 1024 + 1)
except urllib.error.HTTPError as e:
e.close()
if e.code == 429:
raise _web.limited_error(NAME, _web.throttle(url, e.headers)) from None
if e.code in (401, 403):
_refused(e.code, key, api_path)
raise SourceError('itch.io returned HTTP error %d; try again later' % e.code) from None
except SourceError:
raise
except NETWORK as e:
raise _network_error(e, key) from None
return _checked(raw, key, api_path)
def _cached(key, what, load):
slot = (_generation[0], hashlib.sha256(key.encode()).hexdigest(), what)
with _lock:
hit = _cache.get(slot)
if hit and time.time() - hit[0] < TTL:
return hit[1]
value = load()
with _lock:
_cache[slot] = (time.time(), value)
return value
def _use(call):
"""Run call(key); a key itch.io now refuses is removed, so the account shows disconnected."""
with _account: # the key and the connection it belongs to, read together
key, generation = _key(), _generation[0]
try:
return call(key)
except BadKey:
with _account:
# Only the connection this call started with: a reconnect meanwhile (even with
# the same key) has been checked since, and is not undone by this older verdict.
if _generation[0] == generation and (_load() or {}).get('key') == key:
_delete()
raise BadKey(REVOKED) from None
def connect(key):
"""Check key with itch.io (GET /profile), then save it. A refused key is never saved, and
nor is one whose check finished after a disconnect or another connect."""
key = (key or '').strip() if isinstance(key, str) else ''
if not KEY.fullmatch(key):
raise BadKey('That does not look like an itch.io API key. Copy it from itch.io settings > API keys')
with _account:
started = _generation[0]
user = _get('/profile', key).get('user') # the network, outside the lock
if not isinstance(user, dict):
raise SourceError('itch.io returned an invalid profile')
username = user.get('username') if isinstance(user.get('username'), str) else None
with _account:
if _generation[0] != started:
raise SourceError('The itch.io connection changed while the key was being checked; '
'nothing was saved. Try again')
_generation[0] += 1
_save(key, username)
_forget()
return {'message': 'Connected to itch.io' + (' as ' + username if username else ''), 'itch': status()}
def _delete():
"""Remove the key file; call with _account held."""
_generation[0] += 1
try:
os.remove(str(path()))
except FileNotFoundError:
pass
_forget()
def disconnect():
with _account:
_delete()
return {'message': 'Disconnected from itch.io; the API key was deleted from this computer', 'itch': status()}
def owned_keys():
"""Every download key the account owns, newest first (GET /profile/owned-keys, all pages)."""
def load(key):
keys = []
for page in range(1, PAGES + 1):
batch = _get('/profile/owned-keys', key, {'page': page, 'per_page': PER_PAGE}).get('owned_keys')
# Documented: a page can hold fewer than per_page even when more follow; an empty one ends it.
if not batch or not isinstance(batch, list):
break
keys.extend(k for k in batch if isinstance(k, dict))
return keys
return _use(lambda key: _cached(key, 'owned', lambda: load(key)))
def _number(value):
return value if type(value) is int and value > 0 else None
def is_android(upload):
"""An upload marked for Android, or an .apk file. The marker's field name is not in the
public reference: itch.io's traits use p_<platform> (p_windows...), so p_android is assumed."""
traits = upload.get('traits') if isinstance(upload.get('traits'), list) else []
platforms = upload.get('platforms') if isinstance(upload.get('platforms'), dict) else {}
filename = str(upload.get('filename') or '').lower()
return 'p_android' in traits or upload.get('p_android') is True or 'android' in platforms \
or filename.endswith('.apk')
def android_uploads(uploads):
"""Installable Android uploads, best first: .apk files, then full games, then newest."""
chosen = []
for u in uploads if isinstance(uploads, list) else []:
if not isinstance(u, dict) or not _number(u.get('id')) or not is_android(u):
continue
traits = u.get('traits') if isinstance(u.get('traits'), list) else []
if u.get('type') not in (None, 'default') or u.get('preorder') is True or 'preorder' in traits:
continue # soundtracks, books, placeholders
if u.get('storage') == 'external':
continue # hosted on another site: not something itch.io can hand us
chosen.append(u)
chosen.sort(key=lambda u: str(u.get('updated_at') or u.get('created_at') or ''), reverse=True)
chosen.sort(key=lambda u: (not str(u.get('filename') or '').lower().endswith('.apk'),
u.get('demo') is True or 'demo' in (u.get('traits') or [])))
return chosen
def uploads(game_id, download_key_id):
"""The game's uploads this download key may fetch (GET /games/ID/uploads)."""
game_id, download_key_id = _number(game_id), _number(download_key_id)
if not game_id or not download_key_id:
raise SourceError('Invalid itch.io game')
return _use(lambda key: _cached(key, ('uploads', game_id, download_key_id), lambda: _get(
'/games/%d/uploads' % game_id, key, {'download_key_id': download_key_id}).get('uploads') or []))
def download_url(upload_id, download_key_id):
upload_id, download_key_id = _number(upload_id), _number(download_key_id)
if not upload_id or not download_key_id:
raise SourceError('Invalid itch.io upload')
return API + '/uploads/%d/download?' % upload_id + urllib.parse.urlencode({'download_key_id': download_key_id})
def _fetch(url, key):
"""Ask api.itch.io for the file. Not checked against a live account, so this accepts each
plausible answer: a redirect to storage, JSON naming the storage URL, or the file itself."""
try:
r = _open(url, key, 'application/vnd.android.package-archive, application/json;q=0.5, */*;q=0.1')
except urllib.error.HTTPError as e:
e.close()
if e.code in REDIRECTS:
target = e.headers.get('Location')
elif e.code == 429:
raise _web.limited_error(NAME, _web.throttle(url, e.headers)) from None
elif e.code in (401, 403):
_refused(e.code, key, '/uploads')
else:
raise SourceError('itch.io returned HTTP error %d for the download; try again later' % e.code) from None
except SourceError:
raise
except NETWORK as e:
raise _network_error(e, key) from None
else:
with r:
if 'json' not in (r.headers.get('Content-Type') or '').lower():
try:
return _web.save_apk(r) # served by api.itch.io itself
except NETWORK + (zipfile.BadZipFile,) as e:
raise SourceError(_clean('Could not download APK: ' + str(e), key)) from None
try:
raw = r.read(1024 * 1024 + 1)
except NETWORK as e:
raise _network_error(e, key) from None
target = _checked(raw, key, '/uploads').get('url')
if not isinstance(target, str) or not target:
raise SourceError('itch.io did not say where to download the file')
def refuse(hop): # on the first URL and on every redirect after it, before any lookup
if _carries_key(hop, key):
raise SourceError('itch.io pointed the download at a URL carrying your API key; refused, '
'so the key is not sent to the storage server')
try:
return _web.public_apk(urllib.parse.urljoin(url, target), NAME, check=refuse) # no key sent
except SourceLimited as e:
raise SourceLimited(_clean(str(e), key), e.retry_after) from None
except SourceError as e:
raise type(e)(_clean(str(e), key)) from None
def _carries_key(url, key):
"""True if url holds the key (or 8+ characters of its start), however cased or encoded, or
an itch.io API-key parameter: go-itchio puts the key in ?api_key=, and storage must never
get it. Other parameters (signatures, tokens) are the storage URL's own and are kept."""
for form in (url, urllib.parse.unquote(url), urllib.parse.unquote_plus(url)):
if _clean(form, key) != form:
return True
names = {name.lower() for name, _ in urllib.parse.parse_qsl(urllib.parse.urlsplit(url).query,
keep_blank_values=True)}
return bool(names & {'api_key', 'apikey', 'api-key'})
def download(upload_id, download_key_id):
"""The APK: the key goes to api.itch.io only, never to the storage host."""
url = download_url(upload_id, download_key_id)
return _use(lambda key: _fetch(url, key))
+161 -11
View File
@@ -1,5 +1,6 @@
"""Small HTTPS cache and APK downloader for public publisher sources."""
import hashlib, os, shutil, tempfile, threading, time, urllib.error, urllib.parse, urllib.request, zipfile
import hashlib, http.client, ipaddress, os, shutil, socket, ssl, tempfile, threading, time
import urllib.error, urllib.parse, urllib.request, zipfile
from email.utils import parsedate_to_datetime
import frame_host
@@ -136,6 +137,8 @@ def checked_url(url, hosts):
class Redirect(urllib.request.HTTPRedirectHandler):
"""Follow redirects only to hosts; an Authorization header never follows one."""
def __init__(self, hosts):
self.hosts = hosts
@@ -196,19 +199,16 @@ def read(url, hosts, headers=None, ttl=3600, name=None, hint=''):
raise SourceError('Could not read source: ' + str(e)) from e
def apk(url, hosts, digest=None, name=None):
def save_apk(response, digest=None):
"""Stream response (anything with read(n)) into the cache as a checked APK."""
tmp = None
name = name or _host(url)
wait = wait_time(url)
if wait:
raise limited_error(name, wait)
try:
fd, tmp = tempfile.mkstemp(dir=cache(), suffix='.part')
h = hashlib.sha256()
with os.fdopen(fd, 'wb') as f, open_url(url, hosts) as r:
with os.fdopen(fd, 'wb') as f:
size = 0
while True:
chunk = r.read(1 << 20)
chunk = response.read(1 << 20)
if not chunk:
break
size += len(chunk)
@@ -225,12 +225,162 @@ def apk(url, hosts, digest=None, name=None):
path = os.path.join(cache(), actual + '.apk')
os.replace(tmp, path)
return {'apk': path, 'obb': [], 'sha256': actual, 'verified': bool(digest)}
finally:
if tmp and os.path.exists(tmp):
os.remove(tmp)
def apk(url, hosts, digest=None, name=None):
name = name or _host(url)
wait = wait_time(url)
if wait:
raise limited_error(name, wait)
try:
with open_url(url, hosts) as r:
return save_apk(r, digest)
except urllib.error.HTTPError as e:
if e.code in (403, 429):
raise limited_error(name, throttle(url, e.headers)) from e
raise SourceError('Could not download APK: HTTP error ' + str(e.code)) from e
except (OSError, ValueError, zipfile.BadZipFile) as e:
raise SourceError('Could not download APK: ' + str(e)) from e
finally:
if tmp and os.path.exists(tmp):
os.remove(tmp)
# IPv6 is an allow-list: only global unicast (2000::/3), which rules out everything else at
# once (loopback, unspecified, mapped/compatible IPv4, NAT64 64:ff9b::, site-local fec0::,
# unique-local fc00::, link-local, multicast, reserved). Inside it, these special blocks are
# refused too: 6to4 and Teredo carry IPv4 that may be private; the rest aren't the internet.
_GLOBAL6 = ipaddress.ip_network('2000::/3')
_SPECIAL6 = tuple(ipaddress.ip_network(n) for n in (
'2001::/32', # Teredo
'2001:2::/48', # benchmarking
'2001:10::/28', # ORCHID
'2001:20::/28', # ORCHIDv2
'2001:db8::/32', # documentation
'2002::/16', # 6to4
'3fff::/20')) # documentation (2024)
# IPv4 blocks that are never the public internet, listed in full rather than trusting
# is_global (Python 3.9 calls 192.0.0.8, .11 and .255 global).
_SPECIAL4 = tuple(ipaddress.ip_network(n) for n in (
'0.0.0.0/8', '10.0.0.0/8', '100.64.0.0/10', '127.0.0.0/8', '169.254.0.0/16', '172.16.0.0/12',
'192.0.0.0/24', '192.0.2.0/24', '192.88.99.0/24', '192.168.0.0/16', '198.18.0.0/15',
'198.51.100.0/24', '203.0.113.0/24', '224.0.0.0/4', '240.0.0.0/4', '255.255.255.255/32'))
def public_address(ip):
"""True for an address on the public internet: never loopback, private, link-local,
site-local, shared (CGNAT), reserved, unspecified or multicast, nor an IPv6 wrapper
around IPv4. Python 3.9's is_global alone lets several of these through."""
try:
a = ipaddress.ip_address(ip.split('%', 1)[0])
except ValueError:
return False
if a.version == 6 and (a not in _GLOBAL6 or any(a in n for n in _SPECIAL6)):
return False
if a.version == 4 and any(a in n for n in _SPECIAL4):
return False
return a.is_global and not (a.is_reserved or a.is_private or a.is_loopback or a.is_link_local
or a.is_multicast or a.is_unspecified)
def resolve_public(host):
"""host's addresses for port 443, if every address it has is public; else SourceError."""
try:
addresses = socket.getaddrinfo(host, 443, type=socket.SOCK_STREAM)
except (OSError, UnicodeError) as e:
raise SourceError('Could not find the download server ' + host) from e
if not addresses or not all(public_address(a[4][0]) for a in addresses):
raise SourceError('The download server is on a private network address; refused')
found = []
for a in addresses:
if a[4][:2] not in found:
found.append(a[4][:2])
return found
CONNECT_BUDGET = 20 # seconds for connecting, all addresses and TLS handshakes together
def _public_connection(host, timeout, budget=CONNECT_BUDGET):
"""A verified TLS connection to host:443, made to an address checked to be public.
Each checked address is tried in turn, all within one deadline (budget, or timeout if
shorter) covering both connecting and the handshake. The one connected to is always one
that was checked, so a second lookup can't swap in another. timeout is for reads after."""
deadline = time.monotonic() + min(budget, timeout)
error = None
for address in resolve_public(host):
if time.monotonic() >= deadline:
break
try:
return _connect(host, address, timeout, deadline)
except OSError as e: # unreachable, refused, or its TLS failed: try the next one
error = e
raise error or socket.timeout('Timed out connecting to ' + host)
def _connect(host, address, timeout, deadline):
"""TLS to exactly address, verified as host, connected and handshaken before deadline
(a time.monotonic() value); reads then get timeout seconds each."""
def remaining():
left = deadline - time.monotonic()
if left <= 0:
raise socket.timeout('Timed out connecting to ' + host)
return left
sock = socket.create_connection(address, timeout=remaining())
try:
# Each step gets what is left of the deadline as its socket timeout. A handshake
# read as several receives could in theory run past it by one timeout; no thread
# is kept to cut it short, as one could fire after the connection was handed over.
sock.settimeout(remaining())
sock = ssl.create_default_context().wrap_socket(sock, server_hostname=host,
do_handshake_on_connect=False)
sock.settimeout(remaining())
sock.do_handshake()
sock.settimeout(timeout)
except BaseException:
sock.close()
raise
conn = http.client.HTTPConnection(host, 443, timeout=timeout)
conn.sock = sock
return conn
def public_apk(url, name, redirects=3, timeout=60, check=None):
"""Download an APK from a public HTTPS host that no source lists in advance (itch.io's
storage). No credentials are sent. Every hop, redirects included, is re-checked, and
check(url), if given, runs on each hop before any lookup or connection (it raises)."""
for _ in range(redirects + 1):
if check:
check(url)
try:
p = urllib.parse.urlsplit(url)
port = p.port
except (ValueError, TypeError) as e:
raise SourceError(name + ' returned an invalid download URL') from e
if p.scheme != 'https' or not p.hostname or p.username or p.password or port not in (None, 443):
raise SourceError(name + ' returned an unexpected download URL')
wait = wait_time(url)
if wait:
raise limited_error(name, wait)
try:
conn = _public_connection(p.hostname, timeout)
except OSError as e:
raise SourceError('Could not download APK: ' + str(e)) from e
try:
conn.request('GET', (p.path or '/') + ('?' + p.query if p.query else ''), headers={'User-Agent': UA})
r = conn.getresponse()
if r.status in (301, 302, 303, 307, 308):
url = urllib.parse.urljoin(url, r.getheader('Location') or '')
continue
if r.status == 429:
raise limited_error(name, throttle(url, r.headers))
if r.status != 200:
raise SourceError("%s's download server refused the file (HTTP %d); try again later"
% (name, r.status))
return save_apk(r)
except (OSError, ValueError, http.client.HTTPException, zipfile.BadZipFile) as e:
raise SourceError('Could not download APK: ' + str(e)) from e
finally:
conn.close()
raise SourceError(name + ' redirected the download too many times')
+97 -15
View File
@@ -1,17 +1,35 @@
"""Free Android VR RSS listings. Download pages stay in the publisher's UI."""
"""Free Android VR RSS listings, plus the Android builds of games the user owns.
The free feeds' download pages stay in the publisher's UI. Owned games appear only
after the user connects their itch.io account with an API key (_itch_account.py);
they download through itch.io's own API, with the download key of that purchase.
"""
import html, re, urllib.parse, xml.etree.ElementTree as ET
from email.utils import parsedate_to_datetime
from . import SourceError, _web
from . import SourceError, _itch_account, _web
KIND = 'itch'
FEEDS = ('openxr', 'oculus-quest')
OWNED = 'itch-owned'
OWNED_LABEL = 'Owned on itch.io'
def sources():
return [{'id': KIND if tag == 'openxr' else 'itch-quest', 'kind': KIND,
'name': 'itch.io (' + tag + ')', 'url': 'https://itch.io', 'tag': tag,
'builtin': True, 'enabled': True, 'trust': 'community'} for tag in FEEDS]
found = [{'id': KIND if tag == 'openxr' else 'itch-quest', 'kind': KIND,
'name': 'itch.io (' + tag + ')', 'url': 'https://itch.io', 'tag': tag,
'builtin': True, 'enabled': True, 'trust': 'community'} for tag in FEEDS]
if _itch_account.connected():
found.append({'id': OWNED, 'kind': KIND, 'name': 'itch.io (my purchases)',
'url': 'https://itch.io/my-purchases', 'owned': True,
'description': 'Android builds of games you own on itch.io',
'builtin': True, 'enabled': True, 'trust': 'user'})
return found
def _itch_page(url):
p = urllib.parse.urlsplit(url if isinstance(url, str) else '')
return p.scheme == 'https' and (p.hostname or '').endswith('.itch.io') and not p.username and ':' not in p.netloc
def _parse(source, data):
@@ -22,8 +40,7 @@ def _parse(source, data):
entries = []
for item in root.findall('./channel/item'):
page = item.findtext('link') or ''
p = urllib.parse.urlsplit(page)
if p.scheme != 'https' or not (p.hostname or '').endswith('.itch.io') or p.username or ':' in p.netloc:
if not _itch_page(page):
continue
if item.findtext('price') != '$0.00' or item.findtext('platforms/android') != 'yes':
continue
@@ -45,23 +62,81 @@ def _parse(source, data):
return entries
def _owned_entry(source, key):
"""Entry for one owned download key, or None if its game isn't for Android."""
game = key.get('game') if isinstance(key.get('game'), dict) else {}
game_id = key.get('game_id') or game.get('id')
if type(game_id) is not int or type(key.get('id')) is not int:
return None
traits = game.get('traits')
if isinstance(traits, list) and 'p_android' not in traits:
return None # the game's page offers no Android download
cover = game.get('cover_url') if isinstance(game.get('cover_url'), str) else None
if cover and not cover.startswith('https://img.itch.zone/'):
cover = None
user = game.get('user') if isinstance(game.get('user'), dict) else {}
price = game.get('min_price')
return {'source': source['id'], 'id': str(game_id), 'package': None,
'name': str(game.get('title') or 'itch.io game ' + str(game_id)),
'summary': str(game.get('short_text') or OWNED_LABEL),
'developer': user.get('display_name') or user.get('username'),
'icon': cover, 'images': {'icon': cover, 'banner': cover, 'screenshots': []},
'page': game['url'] if _itch_page(game.get('url')) else None,
'version': None, 'version_code': None, 'min_sdk': None, 'abis': None,
'vr': None, 'size': None, 'free': price == 0 if type(price) is int else None,
'owned': True, 'owned_label': OWNED_LABEL, 'license': None,
'updated': str(game.get('published_at') or '')[:10] or None,
'downloadable': None, 'download_key_id': key['id']}
def _owned(source):
entries = {}
for key in _itch_account.owned_keys():
entry = _owned_entry(source, key)
if entry:
entries.setdefault(entry['id'], entry) # keys come newest first; one entry per game
return entries
def search(source, query, limit=50):
limit = max(0, min(int(limit), 100))
if not limit:
return []
entries = {}
tag = source.get('tag', 'openxr')
if tag not in FEEDS:
raise SourceError('Unsupported itch.io feed')
url = 'https://itch.io/games/free/platform-android/tag-' + tag + '.xml'
for entry in _parse(source, _web.read(url, ('itch.io',), name='itch.io')):
entries.setdefault(entry['id'], entry)
if source.get('owned'):
entries = _owned(source)
else:
entries = {}
tag = source.get('tag', 'openxr')
if tag not in FEEDS:
raise SourceError('Unsupported itch.io feed')
url = 'https://itch.io/games/free/platform-android/tag-' + tag + '.xml'
for entry in _parse(source, _web.read(url, ('itch.io',), name='itch.io')):
entries.setdefault(entry['id'], entry)
words = query.lower().split()
return [e for e in entries.values() if all(w in (e['name'] + ' ' + e['summary']).lower()
for w in words)][:limit]
def _owned_details(source, entry_id):
entry = _owned(source).get(str(entry_id))
if not entry:
raise SourceError('This game is not among the Android games you own on itch.io')
uploads = _itch_account.uploads(int(entry['id']), entry['download_key_id'])
versions = [{'version': str(u.get('display_name') or u.get('filename') or u['id']), 'version_code': None,
'upload_id': u['id'], 'name': u.get('filename'), 'min_sdk': None,
'demo': u.get('demo') is True or 'demo' in (u.get('traits') or []),
'size': u.get('size') if type(u.get('size')) is int else None,
'updated': str(u.get('updated_at') or u.get('created_at') or '')[:10] or None}
for u in _itch_account.android_uploads(uploads)]
entry = dict(entry, versions=versions, downloadable=bool(versions))
if versions:
entry.update({k: versions[0][k] for k in ('version', 'size', 'updated')})
return entry
def details(source, entry_id):
if source.get('owned'):
return _owned_details(source, entry_id)
entry = next((e for e in search(source, '', 100) if e['id'] == entry_id), None)
if not entry:
raise SourceError('Game is not in the current free Android VR feed; open its publisher page')
@@ -69,4 +144,11 @@ def details(source, entry_id):
def download(source, entry_id, version_code=None):
raise SourceError('Open the itch.io publisher page to download; automated download pages are disallowed by robots rules')
if not source.get('owned'):
raise SourceError('Open the itch.io publisher page to download; automated download pages are disallowed by robots rules')
if version_code is not None:
raise SourceError('itch.io does not publish Android version codes; download the latest upload')
entry = _owned_details(source, entry_id)
if not entry['versions']:
raise SourceError('This game has no Android download on itch.io')
return _itch_account.download(entry['versions'][0]['upload_id'], entry['download_key_id'])
+11 -3
View File
@@ -182,7 +182,10 @@ def group(entries, query='', vr=None, installable=False):
groups = {}
for entry in entries:
entry = decorate(entry)
if vr is not None and (entry.get('vr') is True) != vr: # unknown counts as flat
# Unknown counts as flat, except the user's own purchases (itch.io doesn't say which are
# VR): those show under either choice rather than vanish from one.
if vr is not None and (entry.get('vr') is True) != vr \
and not (entry.get('owned') is True and entry.get('vr') is None):
continue
if installable and entry['fit']['installable'] is not True:
continue
@@ -215,6 +218,11 @@ def group(entries, query='', vr=None, installable=False):
return result
def obtainable(source, entry):
"""Free, or a game the user owns, listed by their own account's source (itch.io purchases)."""
return entry.get('free') is True or (source.get('owned') is True and entry.get('owned') is True)
def _launch(module, source, query, limit):
"""One search per source at a time; the newest different query runs next."""
key = source['id']
@@ -243,7 +251,7 @@ def _start(key, task):
queued = None
try:
task['entries'] = [dict(e, source=key, source_name=source['name'], trust=source.get('trust'))
for e in module.search(source, query, limit=limit) if e.get('free') is True]
for e in module.search(source, query, limit=limit) if obtainable(source, e)]
task['stale'] = bool(getattr(module, 'stale', lambda s: False)(source))
except Exception as e:
task['error'] = str(e)
@@ -300,7 +308,7 @@ def install(source_id, entry_id, version_code=None, progress=None):
if not source['enabled']:
raise SourceError('This source is disabled')
entry = module.details(source, entry_id)
if entry.get('free') is not True or entry.get('downloadable') is not True:
if not obtainable(source, entry) or entry.get('downloadable') is not True:
raise SourceError('This app must be obtained from its developer page')
if progress:
progress('Downloading', None)
+22 -381
View File
@@ -7,7 +7,7 @@ in the Steam library and gets its own SteamVR panel. Nothing goes through
Lepton Development, which wipes its apps on exit. See docs/apks.md.
Python stdlib only. CLI: python3 ui/frame_android.py
install APK [SPLIT.apk ...] [--vr|--flat] [--no-xr-compat] | info APK | versions APK-or-PKG
install APK [--vr|--flat] [--no-xr-compat] | info APK | versions APK-or-PKG
install-obb PKG OBB [OBB ...] | backup-data PKG ARCHIVE | restore-data PKG ARCHIVE
refresh-art PKG|--all | patch SRC DST [--add NAME=PATH ...] | list | launch PKG | stop PKG | remove PKG | probe PKG
"""
@@ -18,7 +18,7 @@ import frame_artwork
import frame_host
import tempfile
import zipfile
from frame_apk_vr import add_launcher_category, drop_split_required
from frame_apk_vr import add_launcher_category
from frame_apk_sign import repack
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
@@ -61,20 +61,13 @@ def ssh(cmd, input=None, timeout=120):
try:
# No inherited stdin (see server.ssh): Windows' ssh.exe would wait on it.
feed = {'input': input} if input is not None else {'stdin': subprocess.DEVNULL}
# The Frame writes UTF-8, whatever this computer's code page (Windows would decode it as
# cp1252). errors='replace': stray bytes (a file name) mustn't turn a failed command, or
# one that worked, into a UnicodeDecodeError.
text = ({'text': True, 'encoding': 'utf-8', 'errors': 'replace'}
if isinstance(input, str) or input is None else {})
p = frame_host.run_ssh(['ssh', *SSH_OPTS, FRAME, cmd], capture_output=True, **feed,
timeout=timeout, **text)
timeout=timeout, text=isinstance(input, str) or input is None)
except subprocess.TimeoutExpired:
raise frame_host.link_failure(FrameError(f'timed out talking to {FRAME}'))
if p.returncode != 0:
# Bytes input gives bytes output: decode it, so the message isn't "b'...'".
stderr, stdout = (v if isinstance(v, str) else (v or b'').decode(errors='replace')
for v in (p.stderr, p.stdout))
error = FrameError((stderr or stdout or f'ssh exited {p.returncode}').strip()[-600:])
error = FrameError((p.stderr or p.stdout or f'ssh exited {p.returncode}').strip()[-600:])
stderr = p.stderr if isinstance(p.stderr, str) else (p.stderr or b'').decode(errors='replace')
raise frame_host.link_failure(error) if frame_host.ssh_link_failed(p.returncode, stderr) else error
return p.stdout
@@ -142,22 +135,12 @@ def check_installable(info):
"(or arm64, or universal) and install that instead")
# Installs are rare; one at a time avoids every race. Reentrant so a caller (a Quest import's
# final checks and data copy) can hold it around install_obb and install, which take it too.
_install_lock = threading.RLock()
# The slowest transfer a copy is allowed before it times out, in bytes a second (about 4 Mbit/s,
# a slow or busy Wi-Fi link). A fixed 10 minutes failed any APK over about 300 MB at that rate.
COPY_MIN_RATE = 512 * 1024
_install_lock = threading.Lock() # installs are rare; one at a time avoids every race
def _copy(src, dest, executable=False, timeout=600):
"""Copy a local file to the Frame: rsync where installed (not on Windows), else scp.
timeout is the least time allowed; a large file gets longer (COPY_MIN_RATE)."""
"""Copy a local file to the Frame: rsync where installed (not on Windows), else scp."""
name = os.path.basename(src)
try:
timeout = max(timeout, os.path.getsize(src) / COPY_MIN_RATE)
except OSError:
pass # rsync or scp says what's wrong with the file
rsync = None if frame_host.WINDOWS else shutil.which('rsync') # see server.push_file
if rsync:
cmd = ['rsync', '-a', *(['--chmod=u+x'] if executable else []),
@@ -165,8 +148,7 @@ def _copy(src, dest, executable=False, timeout=600):
else:
cmd = ['scp', *SSH_OPTS, src, f'{FRAME}:{dest}']
try:
frame_host.run_ssh(cmd, check=True, capture_output=True, stdin=subprocess.DEVNULL, text=True,
errors='replace', timeout=timeout)
frame_host.run_ssh(cmd, check=True, capture_output=True, stdin=subprocess.DEVNULL, text=True, timeout=timeout)
except subprocess.TimeoutExpired:
raise FrameError(f'copying {name} to the Frame timed out')
except subprocess.CalledProcessError as e:
@@ -192,303 +174,7 @@ def _write_meta(d, meta):
install_hooks = []
# Split APK members that belong to the split itself, not the app: never merged.
SPLIT_OWN = re.compile(r'^(AndroidManifest\.xml|stamp-cert-sha256|META-INF/.*)$')
# What a mergeable split may add: native libraries and assets, nothing else.
SPLIT_ADDS = re.compile(r'^(lib/[^/]+/[^/]+\.so|assets/.*[^/])$')
# Manifest elements a config or install-time asset split carries. Anything else (an
# activity, service, uses-library...) belongs to a feature split, which needs its manifest.
SPLIT_TAGS = {'manifest', 'application', 'uses-sdk', 'module', 'delivery', 'install-time', 'fusing'}
# Every merged byte is held in memory while the APK is repacked and signed (as for every
# patched APK), so splits are bounded by what's actually inflated, not only the headers.
MAX_SPLIT_MEMBER = 512 * 1024**2
MAX_SPLIT_TOTAL = 1024**3
def _split_manifest(path, name):
"""(package, split name) from a split APK's manifest, refusing splits that declare
anything the merged app would lose with that manifest."""
with zipfile.ZipFile(path) as z:
elements = frame_apk.manifest_elements(frame_apk._read(z, 'AndroidManifest.xml', frame_apk.MAX_MANIFEST))
attrs = next((a for tag, a in elements if tag == 'manifest'), {})
split = (attrs.get('split') or (None, None, None))[2]
feature = attrs.get('isFeatureSplit')
if feature and feature[1]:
raise FrameError(f'{name} ({split or "split"}) is a feature split with its own code and manifest; '
'Frame Control can only merge splits that add native libraries or assets')
extra = sorted({tag for tag, _ in elements if tag not in SPLIT_TAGS})
if extra:
raise FrameError(f'{name} ({split or "split"}) declares <{">, <".join(extra)}> in its manifest; '
'Frame Control can only merge splits that add native libraries or assets')
return (attrs.get('package') or (None, None, None))[2], split
def split_entries(base, splits):
"""What merging splits into base adds: {entry name: (split path, entry name, size)}.
Lepton runs one APK per instance, so a split app is merged into one APK. Only splits
that add native libraries (the arm64 ABI split) or install-time assets can be: a split
with code, components or resources (screen density, language) needs its own manifest
or resource table, which Frame Control doesn't merge, so those are refused."""
try:
with zipfile.ZipFile(base) as z:
taken = set(z.namelist())
package = apk_info(base)['package']
plan, total = {}, 0
for path in splits:
name = os.path.basename(path)
split_pkg, split_name = _split_manifest(path, name)
label = f'{name} ({split_name or "split"})'
if split_pkg != package:
raise FrameError(f'{name} is not part of {package} (it says {split_pkg or "nothing"})')
with zipfile.ZipFile(path) as z:
for member in z.infolist():
entry = member.filename
if entry.endswith('/') or SPLIT_OWN.match(entry):
continue
if entry == 'resources.arsc':
# A config split for native code may carry an empty table; any resource needs merging.
if not resources_empty(frame_apk._read(z, entry, frame_apk.MAX_ARSC)):
raise FrameError(f'{label} holds screen-density or language resources; Frame Control '
'can only merge splits that add native libraries or assets')
continue
if entry.startswith('res/'):
raise FrameError(f'{label} holds screen-density or language resources ({entry}); Frame '
'Control can only merge splits that add native libraries or assets')
if re.match(r'^classes\d*\.dex$', entry):
raise FrameError(f'{label} contains code ({entry}); Frame Control can only merge splits '
'that add native libraries or assets')
if not SPLIT_ADDS.match(entry):
raise FrameError(f'{label} contains {entry}; Frame Control can only merge splits that '
'add native libraries or assets')
if entry in taken:
raise FrameError(f'{name} and another part of the app both contain {entry}')
if member.compress_type not in (zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED):
raise FrameError(f'{entry} in {name} uses a compression Android does not')
if member.file_size > MAX_SPLIT_MEMBER:
raise FrameError(f'{entry} in {name} is {member.file_size / 1024**2:.0f} MB, more than '
f'Frame Control merges ({MAX_SPLIT_MEMBER // 1024**2} MB)')
total += member.file_size
if total > MAX_SPLIT_TOTAL:
raise FrameError(f'the split APKs add more than {MAX_SPLIT_TOTAL // 1024**3} GB, more than '
'Frame Control merges')
taken.add(entry)
plan[entry] = (path, entry, member.file_size)
return plan
except (OSError, zipfile.BadZipFile, struct.error, IndexError, frame_apk.ApkError) as e:
raise FrameError(f'could not read the split APKs: {e}') from e
def resources_empty(data):
"""True only for a resource table that provably defines nothing: every type chunk's
entries absent. Bags (styles, plurals) count, and anything not understood counts too."""
def covered(start, end):
chunks, at = [], start
for chunk in frame_apk._chunks(data, start, end):
if chunk[2] != at:
return None
chunks.append(chunk)
at = chunk[2] + chunk[3]
return chunks if at == end else None
try:
if len(data) < 12 or struct.unpack_from('<H', data, 0)[0] != 0x0002:
return False
top = covered(struct.unpack_from('<H', data, 2)[0], len(data))
if top is None:
return False
for ctype, _, off, size in top:
if ctype == 0x0001: # the global string pool
continue
if ctype != 0x0200:
return False
inner = covered(off + struct.unpack_from('<H', data, off + 2)[0], off + size)
if inner is None:
return False
for kind, hs, t, _ in inner:
if kind in (0x0001, 0x0202): # type and key names, type specs
continue
if kind != 0x0201:
return False
flags, count = struct.unpack_from('<xBxxI', data, t + 8)
if flags & 0x01: # sparse: only present entries are listed
if count:
return False
elif flags & 0x02:
if any(o != 0xffff for o in struct.unpack_from(f'<{count}H', data, t + hs)):
return False
elif any(o != 0xffffffff for o in struct.unpack_from(f'<{count}I', data, t + hs)):
return False
return True
except (struct.error, IndexError):
return False
def _zip64_sizes(extra, usize, csize):
"""(uncompressed, compressed) with 0xffffffff fields taken from a ZIP64 extra field."""
at = 0
while at + 4 <= len(extra):
kind, length = struct.unpack_from('<HH', extra, at)
body = extra[at + 4:at + 4 + length]
if kind == 0x0001:
values, pos = [], 0
for field in (usize, csize):
if field == 0xffffffff:
if pos + 8 > len(body):
raise FrameError('truncated ZIP64 extra field')
values.append(struct.unpack_from('<Q', body, pos)[0])
pos += 8
else:
values.append(field)
return tuple(values)
at += 4 + length
if 0xffffffff in (usize, csize):
raise FrameError('ZIP64 sizes without a ZIP64 extra field')
return usize, csize
def _has_zip64_extra(extra):
at = 0
while at + 4 <= len(extra):
kind, length = struct.unpack_from('<HH', extra, at)
if kind == 0x0001:
return True
at += 4 + length
return False
def read_member(path, member):
"""A split member's bytes, read from its raw data and checked against both headers.
The local header must name the same file, with the same method and (unless a data
descriptor carries them, which is then checked) the same sizes and checksum as the
central directory; the data must end before the next member; a stored member must be
exactly its size and a deflated stream must end exactly at the end of its data, with
nothing after it. Output is bounded by the declared size: never trusts zipfile, which
stops early at a smaller declared size."""
with zipfile.ZipFile(path) as z:
info = z.getinfo(member)
starts = sorted({i.header_offset for i in z.infolist()} | {z.start_dir})
name, size = f'{member} in {os.path.basename(path)}', info.file_size
if info.flag_bits & 1 or info.compress_type not in (zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED):
raise FrameError(f'{name} is encrypted or uses a compression Android does not')
if size > MAX_SPLIT_MEMBER or info.compress_size > MAX_SPLIT_MEMBER:
raise FrameError(f'{name} is larger than Frame Control merges ({MAX_SPLIT_MEMBER // 1024**2} MB)')
following = next((o for o in starts if o > info.header_offset), None)
with open(path, 'rb') as raw:
raw.seek(info.header_offset)
header = raw.read(30)
if len(header) != 30 or header[:4] != b'PK\3\4':
raise FrameError(f'{name}: invalid local ZIP header')
flags, method, crc, csize, usize, nl, el = struct.unpack_from('<2xHH4xIIIHH', header, 4)
local_name, extra = raw.read(nl), raw.read(el)
if local_name != info.orig_filename.encode('utf-8' if flags & 0x800 else 'cp437', 'surrogateescape'):
raise FrameError(f'{name}: the local header names {local_name.decode("utf-8", "replace")!r}')
if method != info.compress_type or (flags & 0x08) != (info.flag_bits & 0x08):
raise FrameError(f'{name} does not match its header (local and central entries differ)')
start = info.header_offset + 30 + nl + el
compressed = raw.read(info.compress_size)
if len(compressed) != info.compress_size:
raise FrameError(f'{name} is truncated')
if flags & 0x08:
# Sizes and checksum follow the data: (signature,) crc, then sizes of 4 bytes each,
# or 8 for ZIP64. Streaming writers use the ZIP64 form even for small files when
# sizes weren't known up front, so the width isn't implied by the final sizes: take
# the layout that matches the central directory and fits before the next entry,
# preferring one that ends exactly there, then the width the local header implies.
zip64 = 0xffffffff in (csize, usize) or _has_zip64_extra(extra)
tail = raw.read(24)
if len(tail) < 12:
raise FrameError(f'{name}: truncated data descriptor')
end = start + info.compress_size
fits = []
for signed in (True, False):
if signed and tail[:4] != b'PK\7\x08':
continue
for width in (20, 12):
body = tail[4:4 + width] if signed else tail[:width]
if len(body) < width:
continue
values = struct.unpack_from('<IQQ' if width == 20 else '<III', body)
length = width + (4 if signed else 0)
if values == (info.CRC, info.compress_size, size) and \
(following is None or end + length <= following):
exact = following is not None and end + length == following
fits.append((not exact, (width == 20) != zip64, length))
if not fits:
raise FrameError(f'{name} does not match its header (local and central entries differ, '
'or its data descriptor is truncated)')
crc, csize, usize = info.CRC, info.compress_size, size
descriptor = min(fits)[2]
else:
usize, csize = _zip64_sizes(extra, usize, csize)
descriptor = 0
if (crc, csize, usize) != (info.CRC, info.compress_size, size):
raise FrameError(f'{name} does not match its header (local and central entries differ)')
if following is not None and start + info.compress_size + descriptor > following:
raise FrameError(f'{name} overlaps the next ZIP entry')
if info.compress_type == zipfile.ZIP_STORED:
if info.compress_size != size:
raise FrameError(f'{name} does not match its header (stored {info.compress_size} bytes, says {size})')
data = compressed
else:
inflate = zlib.decompressobj(-15)
try:
data = inflate.decompress(compressed, size + 1)
except zlib.error as e:
raise FrameError(f'{name} is corrupt: {e}') from e
if len(data) != size or not inflate.eof or inflate.unconsumed_tail or inflate.unused_data:
raise FrameError(f'{name} does not match its header (its compressed data is not exactly '
f'{size} bytes with nothing after it)')
if zlib.crc32(data) != info.CRC:
raise FrameError(f'{name} does not match its checksum')
return data
def merge_splits(base, splits, dst):
"""One installable APK from a base APK and its splits (see split_entries), re-signed
with Frame Control's key, without the base's isSplitRequired flag."""
plan = split_entries(base, splits)
add, total = {}, 0
try:
for entry, (path, member, size) in plan.items():
data = read_member(path, member)
total += len(data)
if total > MAX_SPLIT_TOTAL:
raise FrameError(f'the split APKs add more than {MAX_SPLIT_TOTAL // 1024**3} GB')
add[entry] = data
with zipfile.ZipFile(base) as z:
original = frame_apk._read(z, 'AndroidManifest.xml', frame_apk.MAX_MANIFEST)
manifest = drop_split_required(original)
repack(base, dst, replace={'AndroidManifest.xml': manifest} if manifest != original else None, add=add)
except (OSError, ValueError, IndexError, struct.error, zipfile.BadZipFile, frame_apk.ApkError) as e:
raise FrameError(f'could not merge the split APKs: {e}') from e
return {'merged': sorted(os.path.basename(p) for p in splits), 'entries': len(plan)}
def split_size(splits):
"""Bytes a merge adds, from the split headers (merge_splits enforces the real sizes)."""
total = 0
for path in splits:
with zipfile.ZipFile(path) as z:
total += sum(m.file_size for m in z.infolist() if SPLIT_ADDS.match(m.filename))
return total
def install(apk_path, flatscreen=None, name=None, source=None, icon_png=None, xr_compat=None, artwork=None,
splits=None):
"""splits: the rest of a split APK (base first, as apk_path); merged into one APK first."""
if splits:
start = time.time()
with tempfile.TemporaryDirectory(prefix=f'{TMP_PREFIX}{os.getpid()}-') as tmp:
merged = os.path.join(tmp, 'merged.apk')
try:
merge_splits(apk_path, list(splits), merged)
except Exception as e: # reported like any other failed install
_after_install(None, None, e, start)
raise
meta = install(merged, flatscreen, name, source or os.path.basename(apk_path), icon_png, xr_compat, artwork)
return meta
def install(apk_path, flatscreen=None, name=None, source=None, icon_png=None, xr_compat=None, artwork=None):
start, info = time.time(), None
try:
info = apk_info(apk_path)
@@ -571,7 +257,6 @@ def _install(apk_path, info, pkg, flatscreen, name, source, artwork=None):
'source': source or os.path.basename(apk_path)})
presentation['warnings'] = art_warnings + presentation.get('warnings', [])
meta = {'package': pkg, 'label': name or info['label'], 'version': info['version'],
'version_code': info.get('version_code'),
'instance': iid, 'shortcut': shortcut, 'game_id': game_id(shortcut),
'vr': info.get('vr', False), 'vr_issues': info.get('vr_issues', []),
'launchable': info.get('launchable', False), 'patched': info.get('patched', []),
@@ -594,13 +279,6 @@ def _install(apk_path, info, pkg, flatscreen, name, source, artwork=None):
ssh(f'rm -rf {d}', timeout=30)
except FrameError:
pass
elif not ok:
# An update that failed: the app still works, but a partial copy of the new APK
# (scp leaves one; it can be gigabytes) mustn't stay beside it.
try:
ssh(f'rm -f {d}/app.apk.part', timeout=30)
except FrameError:
pass
@@ -658,34 +336,6 @@ except (OSError, ValueError, TypeError, AttributeError):
"""
def _apk_parser_script():
"""Frame-side Python preamble that loads this copy's stdlib APK parser (frame_apk)."""
script = 'import sys, types, json, base64\n'
for module in ('frame_apk', 'frame_apk_vr'):
with open(os.path.join(ROOT, 'ui', module + '.py')) as f:
source = f.read()
script += f'm = types.ModuleType({module!r}); sys.modules[{module!r}] = m; exec({source!r}, m.__dict__)\n'
return script
def installed_version_code(pkg):
"""The version code of pkg's installed app.apk, read on the Frame (only its manifest
crosses the link), or None if there is no readable APK. Connection failures raise."""
if not PKG_RE.match(pkg or ''):
raise FrameError(f'bad package name {pkg!r}')
script = _apk_parser_script() + f"""try:
code = sys.modules['frame_apk'].apk_info({(APPS_DIR + '/' + pkg + '/app.apk')!r}).get('version_code')
except Exception:
code = None
print(json.dumps(code if isinstance(code, int) else None))
"""
try:
code = json.loads(ssh('python3 -', input=script, timeout=120).strip().splitlines()[-1])
except (ValueError, IndexError):
return None
return code if isinstance(code, int) and not isinstance(code, bool) else None
def art_missing(m):
"""True when a Frame Control install has no complete Steam artwork on record."""
return set((m or {}).get('artwork') or {}) != set(frame_artwork.SLOTS)
@@ -707,7 +357,13 @@ def refresh_art(pkg=None, artwork=None, fill_only=False):
m = _meta_or_fail(pkg)
d = f'{APPS_DIR}/{pkg}'
# Parse the APK on the Frame, transferring only its icon/label, not the APK.
script = _apk_parser_script()
modules = {}
for module in ('frame_apk', 'frame_apk_vr'):
with open(os.path.join(ROOT, 'ui', module + '.py')) as f:
modules[module] = f.read()
script = 'import sys, types, json, base64\n'
for module, source in modules.items():
script += f'm = types.ModuleType({module!r}); sys.modules[{module!r}] = m; exec({source!r}, m.__dict__)\n'
script += f"info = sys.modules['frame_apk'].apk_info({(d + '/app.apk')!r})\n"
script += "info['icon_png'] = base64.b64encode(info.get('icon_png') or b'').decode()\n"
script += cached_art_script(d) + "info['artwork'] = cached\nprint(json.dumps(info))\n"
@@ -768,22 +424,10 @@ def _clean_meta(m):
return m
def running_instances(strict=False):
"""Lepton container name -> adb port, for the instances that are running now.
strict: a failed `podman ps` raises FrameError instead of reading as "nothing running",
and containers without an adb port label count too (port None); for callers that must
not mistake an error for a stopped instance."""
if not strict:
out = ssh('podman ps --format "{{.Names}} {{.Labels.adb_port}}" 2>/dev/null || true')
return dict(line.split()[:2] for line in out.splitlines() if len(line.split()) >= 2)
out = ssh('podman ps --format "{{.Names}} {{.Labels.adb_port}}"')
found = {}
for line in out.splitlines():
cols = line.split()
if cols:
found[cols[0]] = cols[1] if len(cols) > 1 and cols[1] != '<no' else None
return found
def running_instances():
"""Lepton container name -> adb port, for the instances that are running now."""
out = ssh('podman ps --format "{{.Names}} {{.Labels.adb_port}}" 2>/dev/null || true')
return dict(line.split()[:2] for line in out.splitlines() if len(line.split()) >= 2)
def list_apps():
@@ -932,11 +576,8 @@ def main():
r = frame_apk_versions.alternatives(
info['package'] if info else args[0], info.get('version_code') if info else None)
elif cmd == 'install':
files = [a for a in args if not a.startswith('--')]
if not files:
raise FrameError('install requires APK [SPLIT.apk ...]')
r = install(files[0], flatscreen=False if '--vr' in args else True if '--flat' in args else None,
xr_compat=False if '--no-xr-compat' in args else None, splits=files[1:])
r = install(args[0], flatscreen=False if '--vr' in args else True if '--flat' in args else None,
xr_compat=False if '--no-xr-compat' in args else None)
elif cmd == 'refresh-art':
if len(args) != 1:
raise FrameError('usage: refresh-art PACKAGE or refresh-art --all')
+1 -4
View File
@@ -7,7 +7,6 @@ the resource table in resources.arsc.
"""
import struct
import zipfile
import zlib
# android: attribute resource ids; names can be stripped by shrinkers, ids can't.
ATTR = {0x01010001: 'label', 0x01010002: 'icon', 0x01010003: 'name',
@@ -222,9 +221,7 @@ def apk_info(path):
res = Resources(_read(z, 'resources.arsc', MAX_ARSC) if 'resources.arsc' in names else b'')
from frame_apk_vr import detection
vr_info = detection(manifest_data, names)
# A damaged download: corrupt deflate data (zlib.error), a member cut short (EOFError),
# undecodable text (UnicodeDecodeError) or an encrypted entry (RuntimeError).
except (struct.error, IndexError, zipfile.BadZipFile, zlib.error, EOFError, ValueError, RuntimeError) as e:
except (struct.error, IndexError, zipfile.BadZipFile) as e:
raise ApkError(f'could not read the APK manifest: {e}')
tags = {}
for tag, attrs in elements:
+1 -44
View File
@@ -118,7 +118,7 @@ def detection(data, names):
info, _ = inspect(data)
issues = []
if 'lib/arm64-v8a/libvrapi.so' in names:
issues.append("Uses Meta's legacy VrApi, which the Frame doesn't have; it may not run.")
issues.append("Uses Meta's legacy VrApi, which the Frame doesn't have; it won't run.")
if any(n.endswith('/libovrplatformloader.so') for n in names):
issues.append("Uses Meta's platform SDK; if it checks your Quest store licence it will quit.")
if 'lib/arm64-v8a/libopenxr_loader.so' in names:
@@ -126,46 +126,3 @@ def detection(data, names):
issues.append('Uses OpenXR (good).')
info['vr_issues'] = issues
return info
IS_SPLIT_REQUIRED = 0x01010591 # android:isSplitRequired
def drop_split_required(axml_bytes):
"""The manifest without <application android:isSplitRequired>, for a base APK whose
splits were merged into it: Android refuses a lone base that still says it needs them."""
chunks = list(frame_apk._chunks(axml_bytes, 8, len(axml_bytes)))
pool = next((c for c in chunks if c[0] == 1), None)
if pool is None:
raise frame_apk.ApkError('AndroidManifest.xml has no string pool')
strings = frame_apk._string_pool(axml_bytes, pool[2])
resmap = ()
for kind, header_size, offset, chunk_size in chunks:
if kind == 0x180:
resmap = struct.unpack_from('<%dI' % ((chunk_size - header_size) // 4), axml_bytes, offset + header_size)
result, changed = bytearray(axml_bytes[:8]), False
for kind, hs, off, size in chunks:
chunk = bytearray(axml_bytes[off:off + size])
if kind == 0x0102 and not changed:
name, astart, asize, count = struct.unpack_from('<4xIHHH', chunk, hs)
if name < len(strings) and strings[name] == 'application':
for i in range(count):
a = hs + astart + i * asize
attr = struct.unpack_from('<I', chunk, a + 4)[0]
known = resmap[attr] if attr < len(resmap) else None
if known == IS_SPLIT_REQUIRED or (known is None and attr < len(strings)
and strings[attr] == 'isSplitRequired'):
del chunk[a:a + asize]
struct.pack_into('<I', chunk, 4, len(chunk))
struct.pack_into('<H', chunk, hs + 12, count - 1)
# id/class/style attribute indexes are 1-based positions in the list.
for at in (hs + 14, hs + 16, hs + 18):
index = struct.unpack_from('<H', chunk, at)[0]
struct.pack_into('<H', chunk, at, 0 if index == i + 1 else index - 1 if index > i + 1 else index)
changed = True
break
result += chunk
if not changed:
return axml_bytes
struct.pack_into('<I', result, 4, len(result))
return bytes(result)
+4 -15
View File
@@ -238,13 +238,6 @@ def scrub_failure(text, hosts=()):
return frame_telemetry.scrub(hide_operands(hide_hosts(str(text or ""), hosts)), 600)
def failure_detail(error):
"""What an address that answered but failed SSH shows: the short reason when there is one
(the whole message is the banner's headline already), else the message."""
error = error or {}
return error.get("short") or error.get("message") or "SSH failed"
def failure_category(message, raw=""):
"""The telemetry error category (frame_unreachable, frame_auth, ...) for a failure: a fixed
word, never its text."""
@@ -817,9 +810,7 @@ class Link:
return "Connecting"
def fail(self, sid, message, raw="", **more):
"""A stage failed. `more`: what the page can offer to fix it (hostkey, auth), and `short`:
the reason in a few words, for the places a whole sentence doesn't fit (the header pill,
the address that failed)."""
"""A stage failed. `more`: what the page can offer to fix it (hostkey, auth)."""
self.stage(sid, "failed", message)
with self.cond:
self.state["error"] = dict({"stage": sid, "message": message, "raw": raw}, **more)
@@ -905,7 +896,7 @@ class Link:
self.learn(device, a["host"], net, None)
return True
with self.cond:
why_not = failure_detail(self.state["error"])
why_not = (self.state["error"] or {}).get("message") or "SSH failed"
self.probe_update(i, state="sshfailed", detail=why_not)
if outcome != "next":
return False
@@ -955,7 +946,7 @@ class Link:
self.learn(device, a["host"], net, results[pick]["rtt_ms"])
return True
with self.cond:
why_not = failure_detail(self.state["error"])
why_not = (self.state["error"] or {}).get("message") or "SSH failed"
self.probe_update(pick, state="sshfailed", detail=why_not)
if outcome != "next":
return False
@@ -1278,7 +1269,6 @@ class Link:
"If SteamOS was reinstalled or reset, or this is a new headset, check its new key and trust "
"it on the Devices tab. If not, a different device may be answering here.")
self.fail("identity", message, text,
short="Saved SSH identity missing" if lost else "SSH identity changed",
hostkey={"state": "missing" if lost else "changed", "type": shown[0] if shown else None,
"fingerprint": shown[1] if shown else None, "host": host})
return "next"
@@ -1289,8 +1279,7 @@ class Link:
if re.search(r"Permission denied", text) and not forward:
# Not something a retry fixes: the headset needs this computer's key (Set Up Connection).
self.fail("login", self.explain(text) or "The headset didn't accept this computer's key. Run Set Up "
"Connection: it pairs this computer with the headset again.", text, auth="denied",
short="This computer's key refused")
"Connection: it pairs this computer with the headset again.", text, auth="denied")
return "stop"
if step == "connected":
self.fail("login", "Logged in, but the shared SSH connection didn't start.", text)
-1439
View File
File diff suppressed because it is too large. Load diff
-644
View File
@@ -1,644 +0,0 @@
"""Import from my Quest: copy games you own off your own Meta Quest onto the Frame.
The Quest is plugged into this computer by USB with developer mode on; its own
adb copies the APK (and split APKs) into a per-user cache. The APK is checked
before anything goes to the Frame, then installed through frame_android.install
like any other APK. Game data (the OBB folder and, if asked, the external
app-data folder) is copied by a second import while the person has the game
open, since only a running Lepton instance can take it; Frame Control never
starts the game itself. The cache is always removed. Games that link Meta's Platform SDK
(libovrplatformloader.so) install with a warning: linking it doesn't mean the game
enforces a licence check, but if it does the game may quit or hang on the Frame, and
Frame Control does not remove the check or work around it. See docs/quest-import.md.
Python stdlib only. CLI: python3 ui/frame_quest.py
devices | list [--serial SERIAL] | import PACKAGE [--serial SERIAL] [--data] [--allow-downgrade]
"""
import json
import os
from pathlib import Path
import re
import secrets
import shlex
import shutil
import subprocess
import sys
import tarfile
import threading
import time
import zipfile
import frame_android as android
import frame_host
OBB = '/sdcard/Android/obb'
DATA = '/sdcard/Android/data'
OBB_NAME = r'(main|patch)\.[0-9]+\.{}\.obb'
# Serials of adb connections that aren't a USB headset: Frame Control's own SSH tunnels
# to the Frame's Lepton instances (127.0.0.1:<port>, server.AdbTunnel) and emulators.
TUNNEL_RE = re.compile(r'^(127\.0\.0\.1|localhost|\[::1\]):\d+$')
QUEST_MAKERS = {'oculus', 'meta'}
META_APPS = ('com.oculus.', 'com.meta.', 'com.facebook.')
CACHE_TTL = 6 * 3600 # an import left behind by a crash is swept after this
ADB_TIMEOUT = 30
PULL_TIMEOUT = 3 * 3600
READY_TIMEOUT = 150
BATCH = 15 # packages per adb shell command while listing
# Linking the Platform SDK doesn't mean a game enforces the licence check (Meta's
# VRC.Quest.Security.1 has only been Recommended since 2021), so it's a warning, not a refusal.
ENTITLEMENT = ("Uses Meta's licence check (Platform SDK, libovrplatformloader.so); it may quit, hang, or run "
"without online features on the Frame. Frame Control won't remove the check.")
VRAPI = ("Uses Meta's legacy VrApi and no OpenXR loader. The Frame doesn't have VrApi, so it may not run.")
NO_DEVICE = ("No Quest found on this computer's USB. Connect the Quest with a cable that carries data, "
"put the headset on and allow USB debugging. Developer mode must be on first: it needs a Meta "
"developer organisation on a verified Meta account (developers.meta.com), then Developer Mode "
"switched on for the headset in the Meta Horizon phone app.")
UNAUTHORISED = ("The Quest hasn't allowed this computer yet. Put the headset on and accept the \"Allow USB "
"debugging\" prompt (tick Always allow from this computer), then try again. No prompt? Unplug "
"and replug the cable, or switch developer mode off and on.")
NO_PERMISSIONS = ("This computer sees the Quest but isn't allowed to open it over USB. On Linux, add a udev "
"rule for Meta's USB vendor id 2833 (see docs/quest-import.md), then replug it.")
OFFLINE = "adb reports the Quest offline. Unplug and replug the cable, or restart the headset."
GONE = "The Quest was disconnected. Plug it back in, put it on, and try again."
class QuestError(android.FrameError):
pass
def _adb_cmd():
try:
return [frame_host.adb()]
except frame_host.HostError as e:
raise QuestError(str(e)) from e
def _friendly(out):
low = out.lower()
if 'unauthorized' in low:
return UNAUTHORISED
if 'no permissions' in low or 'insufficient permissions' in low:
return NO_PERMISSIONS
if 'device offline' in low:
return OFFLINE
if 'no devices/emulators found' in low or re.search(r"device '[^']*' not found", low):
return GONE
return None
def adb(*args, serial=None, timeout=ADB_TIMEOUT):
"""Run this computer's adb (the one Frame Control already finds) and return stdout."""
cmd = _adb_cmd() + (['-s', serial] if serial else []) + list(args)
try:
r = subprocess.run(cmd, capture_output=True, stdin=subprocess.DEVNULL, text=True, errors='replace',
timeout=timeout)
except subprocess.TimeoutExpired:
raise QuestError(f'adb {args[0]} took more than {timeout} s; check the Quest is still connected')
except OSError as e:
raise QuestError(f'could not run adb: {e}')
if r.returncode != 0:
out = (r.stderr or '').strip() or (r.stdout or '').strip()
raise QuestError(_friendly(out) or out[-600:] or f'adb exited {r.returncode}')
return r.stdout
# ---- Finding the Quest ------------------------------------------------------
def _parse_devices(out):
found = []
for line in out.splitlines():
line = line.strip()
if not line or line.startswith('List of devices') or line.startswith('*'):
continue
parts = line.split(None, 1)
serial, rest = parts[0], (parts[1] if len(parts) > 1 else '')
state = 'no permissions' if rest.startswith('no permissions') else (rest.split() or ['unknown'])[0]
fields = dict(t.split(':', 1) for t in rest.split() if re.match(r'^[a-z_]+:', t))
found.append({'serial': serial, 'state': state, 'usb': 'usb' in fields,
'model': fields.get('model', '').replace('_', ' '), 'manufacturer': ''})
return found
def devices():
"""Every adb device on this computer, each with a kind: quest, frame, android (another
Android device), frame-tunnel (Frame Control's own SSH-tunnelled connection to a Lepton
instance, never touched here), emulator, or unknown (not authorised, offline...)."""
found = _parse_devices(adb('devices', '-l', timeout=15))
for d in found:
if TUNNEL_RE.match(d['serial']):
d['kind'] = 'frame-tunnel'
elif d['serial'].startswith('emulator-'):
d['kind'] = 'emulator'
elif d['state'] != 'device':
d['kind'] = 'unknown'
else:
try:
props = adb('shell', 'getprop ro.product.manufacturer; getprop ro.product.model',
serial=d['serial'], timeout=15).splitlines() + ['', '']
except QuestError:
d['kind'] = 'unknown'
continue
d['manufacturer'], d['model'] = props[0].strip(), props[1].strip() or d['model']
maker = d['manufacturer'].lower()
d['kind'] = ('quest' if maker in QUEST_MAKERS or 'quest' in d['model'].lower()
else 'frame' if maker == 'valve' else 'android')
return found
def find_quest(serial=None):
"""The one connected, authorised Quest, or a QuestError saying what to do."""
found = devices()
if serial:
d = next((d for d in found if d['serial'] == serial), None)
if not d:
raise QuestError(f'No adb device {serial} on this computer. ' + NO_DEVICE)
if d['kind'] == 'quest':
return d
state = {'unauthorized': UNAUTHORISED, 'no permissions': NO_PERMISSIONS, 'offline': OFFLINE}.get(d['state'])
what = ' '.join(x for x in (d['manufacturer'] or d['kind'], d['model']) if x)
raise QuestError(state or f"{serial} isn't a Meta Quest ({what}).")
quests = [d for d in found if d['kind'] == 'quest']
if len(quests) == 1:
return quests[0]
if quests:
raise QuestError('Several Quests are connected: ' + ', '.join(f"{d['model']} ({d['serial']})" for d in quests)
+ '. Choose one, or unplug the others.')
states = {d['state'] for d in found if d['kind'] == 'unknown'}
for state, message in (('unauthorized', UNAUTHORISED), ('no permissions', NO_PERMISSIONS), ('offline', OFFLINE)):
if state in states:
raise QuestError(message)
others = [d for d in found if d['kind'] in ('android', 'frame')]
if others:
raise QuestError(NO_DEVICE + ' (adb sees ' + ', '.join(
f"{d['manufacturer']} {d['model']}".strip() for d in others) + ", which isn't a Quest.)")
raise QuestError(NO_DEVICE)
# ---- What's on it -------------------------------------------------------------
def _package_script(packages):
parts = []
for p in packages:
parts += [f'echo @@{p}', f'pm path {p}', f'du -sk {OBB}/{p} 2>/dev/null', f'du -sk {DATA}/{p} 2>/dev/null']
return '; '.join(parts + ['true'])
def _parse_packages(out, packages):
apps, current = {}, None
for line in out.splitlines():
line = line.strip()
if line.startswith('@@') and line[2:] in packages:
current = apps[line[2:]] = {'package': line[2:], 'apks': [], 'obb': None, 'data': None}
elif current is None or not line:
continue
elif line.startswith('package:'):
current['apks'].append(line[len('package:'):])
else:
size, _, where = line.partition('\t') if '\t' in line else line.partition(' ')
if size.isdigit():
key = 'obb' if where.strip().startswith(OBB + '/') else 'data' if where.strip().startswith(DATA + '/') else None
if key:
current[key] = int(size) * 1024
result = []
for p in packages:
app = apps.get(p)
if not app or not app['apks']:
continue # removed while listing
app['split'] = len(app['apks']) > 1
app['meta'] = p.startswith(META_APPS)
result.append(app)
return result
def packages(serial):
"""Third-party apps on the Quest: APK paths (several means a split APK), and the size
in bytes of the OBB and external data folders (None when there is none)."""
out = adb('shell', 'pm list packages -3', serial=serial)
names = sorted({l.split(':', 1)[1].strip() for l in out.splitlines() if l.startswith('package:')})
names = [n for n in names if android.PKG_RE.match(n)]
apps = []
for i in range(0, len(names), BATCH):
batch = names[i:i + BATCH]
apps += _parse_packages(adb('shell', _package_script(batch), serial=serial, timeout=120), batch)
return apps
def package(serial, pkg):
apps = _parse_packages(adb('shell', _package_script([pkg]), serial=serial, timeout=60), [pkg])
if not apps:
raise QuestError(f"{pkg} isn't installed on the Quest")
return apps[0]
def status(serial=None, with_apps=True):
"""For the page: the Quest (or why there isn't one), what else adb sees, and its apps."""
try:
found = devices()
except QuestError as e:
return {'quest': None, 'devices': [], 'guidance': str(e), 'apps': []}
try:
quest = find_quest(serial)
except QuestError as e:
return {'quest': None, 'devices': found, 'guidance': str(e), 'apps': []}
return {'quest': quest, 'devices': found, 'guidance': None,
'apps': packages(quest['serial']) if with_apps else []}
# ---- Checking what was copied ---------------------------------------------------
def inspect(package_name, base, splits=()):
"""Refuse what can't run on the Frame (32-bit only, Android newer than Lepton's, splits
that can't be merged), before anything is copied to it; warn about what may not run."""
names = set()
for path in [base, *splits]:
try:
with zipfile.ZipFile(path) as z:
names |= set(z.namelist())
except (zipfile.BadZipFile, OSError) as e:
raise QuestError(f'{os.path.basename(path)} copied from the Quest is not a readable APK: {e}')
info = android.apk_info(base)
label = info['label']
if info['package'] != package_name:
raise QuestError(f"The APK copied for {package_name} says it is {info['package']}; not importing it")
platform_sdk = any(n.rsplit('/', 1)[-1] == 'libovrplatformloader.so' for n in names)
abis = sorted({n.split('/')[1] for n in names if n.startswith('lib/') and n.count('/') >= 2})
if abis and 'arm64-v8a' not in abis:
raise QuestError(f"{label} is a 32-bit-only build ({', '.join(abis)}). The Frame runs 64-bit ARM "
"(arm64-v8a) apps only; older Quest games are often 32-bit. Ask the developer "
"whether a 64-bit build exists.")
android.check_installable(dict(info, abis=abis)) # Android version
if splits:
android.split_entries(base, splits) # a split APK Frame Control can't merge fails now
warnings = [ENTITLEMENT] if platform_sdk else []
openxr = 'lib/arm64-v8a/libopenxr_loader.so' in names
if 'lib/arm64-v8a/libvrapi.so' in names and not openxr:
warnings.append(VRAPI)
return {'package': info['package'], 'label': label, 'version': info['version'],
'version_code': info.get('version_code'), 'abis': abis,
'vr': bool(info.get('vr') or openxr), 'platform_sdk': platform_sdk, 'warnings': warnings}
# ---- Importing ------------------------------------------------------------------
_import_lock = threading.Lock()
_sleep = time.sleep
def cache_root():
return Path(frame_host.cache_dir('quest-import'))
def sweep_cache(now=None):
"""Remove imports a crash left behind (older than CACHE_TTL)."""
root, now = cache_root(), now or time.time()
if root.is_dir():
for d in root.iterdir():
try:
if now - d.stat().st_mtime > CACHE_TTL:
shutil.rmtree(d, ignore_errors=True)
except OSError:
pass
def _safe(name):
return re.sub(r'[^A-Za-z0-9._-]', '_', name)[:120] or 'part.apk'
def _remote_sizes(serial, paths):
out = adb('shell', 'stat -c %s ' + ' '.join(shlex.quote(p) for p in paths), serial=serial)
return sum(int(x) for x in out.split() if x.isdigit())
def _pull(serial, remote, local):
adb('pull', remote, str(local), serial=serial, timeout=PULL_TIMEOUT)
def _wait_ready(meta):
"""Until the app's Lepton instance has finished booting Android."""
container = f"lepton-steamlaunch-{int(meta['instance'])}"
deadline = time.monotonic() + READY_TIMEOUT
while time.monotonic() < deadline:
out = android.ssh(f"podman exec {container} /system/bin/sh -c 'getprop sys.boot_completed' "
"2>/dev/null || true", timeout=30)
if out.strip() == '1':
return
_sleep(3)
raise QuestError(f"{meta['label']} didn't start on the Frame within {READY_TIMEOUT // 60} minutes, so its "
"game data couldn't be added. Check it opens from the Steam library, then import again.")
def push_data(meta, folder):
"""Stream a pulled /sdcard/Android/data/<package> folder into the running instance as a
tar, written straight into the SSH transfer: nothing is staged on this computer."""
import frame_android_data
pkg = meta['package']
container = f"lepton-steamlaunch-{int(meta['instance'])}"
script = f'set -eu; mkdir -p {DATA}/{pkg}; cd {DATA}/{pkg}; tar -x -o -f -'
read_end, write_end = os.pipe()
state = {'count': 0, 'error': None}
def write():
try:
with os.fdopen(write_end, 'wb') as out, \
tarfile.open(fileobj=out, mode='w|', format=tarfile.GNU_FORMAT) as tar:
for path in sorted(Path(folder).rglob('*')):
if path.is_symlink() or not (path.is_file() or path.is_dir()):
continue # regular files and folders only
info = tar.gettarinfo(str(path), arcname=path.relative_to(folder).as_posix())
info.uid = info.gid = 0
info.uname = info.gname = ''
if path.is_file():
with path.open('rb') as f:
tar.addfile(info, f)
state['count'] += 1
else:
tar.addfile(info)
except (OSError, tarfile.TarError, ValueError) as e: # ValueError: the reader closed first
state['error'] = e
writer = threading.Thread(target=write, daemon=True)
writer.start()
try:
with os.fdopen(read_end, 'rb') as src:
frame_android_data._stream(shlex.join(['podman', 'exec', '-i', container, '/system/bin/sh', '-c', script]),
src=src)
finally:
writer.join(60)
if state['error']:
raise QuestError(f'could not read the app files copied from the Quest: {state["error"]}')
return state['count']
def space_needed(apk_bytes, merged_bytes, obb_bytes, data_bytes):
"""Bytes an import needs free on this computer: the APKs as copied, a merged APK and the
install's patched copy of it (each up to merged_bytes), the OBB folder and the
Android/data folder (streamed to the Frame, never staged twice)."""
return apk_bytes + 2 * merged_bytes + obb_bytes + data_bytes
def _check_space(work, need, pkg):
free = shutil.disk_usage(str(work)).free
if need > free:
raise QuestError(f'Importing {pkg} needs about {need / 1024**3:.1f} GB free on this computer '
f'({free / 1024**3:.1f} GB available in {work.parent})')
def import_game(package_name, serial=None, data=False, progress=None, allow_downgrade=False):
"""Copy package_name from the Quest to the Frame. progress(stage, percent).
Frame Control never starts the game itself. Its game data (OBB, and Android/data if
asked) can only be written into a running Lepton instance, and there is no way to
boot one without its game, so that's a second import: the person starts the game,
imports again, and Frame Control copies the data in and closes the game.
allow_downgrade: replace a newer version on the Frame with the Quest's older one."""
if not android.PKG_RE.match(package_name or ''):
raise QuestError('invalid package name')
report = progress or (lambda stage, percent=None: None)
if not _import_lock.acquire(blocking=False):
raise QuestError('Another Quest import is running; wait for it to finish')
try:
report('Looking for your Quest', 1)
quest = find_quest(serial)
app = package(quest['serial'], package_name)
sweep_cache()
work = cache_root() / f'{package_name}-{secrets.token_hex(4)}'
work.mkdir(parents=True)
try:
return _import(quest['serial'], app, data, work, report, allow_downgrade)
finally:
shutil.rmtree(work, ignore_errors=True)
finally:
_import_lock.release()
def _running(pkg):
# strict: a failed `podman ps` raises, never reads as "not running".
return f'lepton-steamlaunch-{android.instance_id(pkg)}' in android.running_instances(strict=True)
def _installed_code(pkg, meta):
"""The installed version code: from the install record, else (installs made before
records kept it) read from the installed app.apk on the Frame; None if unknown."""
code = (meta or {}).get('version_code')
if isinstance(code, int) and not isinstance(code, bool):
return code
return android.installed_version_code(pkg) if meta else None
def _version(name, code):
return ' '.join(x for x in (f'version {name}' if name else '', f'code {code}' if code is not None else 'unknown code') if x)
def _check_downgrade(existing, checked, allow):
"""Refuse to replace a newer install on the Frame with the Quest's older copy."""
if not existing or allow:
return
have, new = _installed_code(existing['package'], existing), checked['version_code']
if have is None or new is None:
raise QuestError(f"Frame Control can't tell which version of {checked['label']} is on the Frame, so it "
"won't replace it in case that one is newer. Allow replacing a newer version (or "
"--allow-downgrade) to install your Quest's copy anyway.")
if int(new) < int(have):
raise QuestError(f"The Frame has a newer {checked['label']} ({_version(existing.get('version'), have)}) than "
f"your Quest ({_version(checked['version'], new)}). Frame Control won't replace it with "
"an older version unless you allow it (Allow replacing a newer version, or "
"--allow-downgrade).")
def _check_target(pkg, meta, checked):
"""The install record of the game open on the Frame, if it is this package, in its own
instance, at the Quest's exact version code; else a QuestError."""
label = checked['label']
if (not meta or meta.get('package') != pkg or android._int(meta.get('instance')) != android.instance_id(pkg)):
raise QuestError(f"{label} is open on the Frame, but Frame Control has no matching install record for "
'it. Close it, then import again to install it.')
have, new = _installed_code(pkg, meta), checked['version_code']
if have is None or new is None or int(have) != int(new):
raise QuestError(f"{label} is open on the Frame with a different version (Frame: "
f"{_version(meta.get('version'), have)}; Quest: {_version(checked['version'], new)}). "
"Close it, then import again to install the Quest's version.")
return meta
def _check_android_package(meta, code):
"""The package Android itself has installed in the instance is this one, at this version."""
pkg = meta['package']
container = f"lepton-steamlaunch-{int(meta['instance'])}"
out = android.ssh(f"podman exec {container} /system/bin/sh -c 'dumpsys package {pkg}' 2>/dev/null || true",
timeout=60)
# Only this package's own block (dumpsys can list other packages too).
block = re.search(r'^\s*Package \[' + re.escape(pkg) + r'\] \(.*?(?=^\s*Package \[|\Z)', out, re.M | re.S)
found = re.search(r'\bversionCode=(\d+)', block.group(0)) if block else None
if not found:
raise QuestError(f'Android in {container} does not have {pkg} installed')
if int(found.group(1)) != int(code):
raise QuestError(f'Android in {container} has {pkg} version code {found.group(1)}, not the '
f"Quest's {code}")
def _close(pkg):
"""Stop the game and confirm its instance is gone: (closed, why not)."""
try:
android.stop(pkg)
for _ in range(6):
if not _running(pkg):
return True, None
_sleep(2)
return False, 'it is still running'
except Exception as e: # never replaces the error that led here; unconfirmed counts as not closed
return False, str(e) or type(e).__name__
def _import(serial, app, data, work, report, allow_downgrade=False):
pkg = app['package']
obb_bytes = app['obb'] or 0
data_bytes = (app['data'] or 0) if data else 0
apk_bytes = _remote_sizes(serial, app['apks'])
_check_space(work, space_needed(apk_bytes, apk_bytes, obb_bytes, data_bytes), pkg)
# 1. The APK (and its splits), checked before anything else is copied.
local = []
for i, remote in enumerate(app['apks']):
report(f'Copying the app from your Quest ({i + 1} of {len(app["apks"])})', 5 + 20 * i // len(app['apks']))
name = 'base.apk' if os.path.basename(remote) == 'base.apk' else f'{i:02d}-{_safe(os.path.basename(remote))}'
_pull(serial, remote, work / name)
if not (work / name).is_file():
raise QuestError(f'adb did not copy {remote} from the Quest')
local.append(work / name)
base = next((p for p in local if p.name == 'base.apk'), local[0])
splits = [p for p in local if p != base]
report('Checking it can run on the Frame', 27)
checked = inspect(pkg, str(base), [str(p) for p in splits])
warnings = list(checked['warnings'])
label = checked['label']
# A merge stores what splits compressed: count it uncompressed, now the files are here.
merged = base.stat().st_size + android.split_size([str(p) for p in splits])
_check_space(work, space_needed(0, merged, obb_bytes, data_bytes), pkg)
wants_data = app['obb'] is not None or bool(data and app['data'] is not None)
if not _running(pkg):
# 2a. Install. The game is never started here: see import_game. Checked and installed
# under the shared install lock, so nothing installs or removes it in between.
report('Installing on the Frame', 60)
with android._install_lock:
if _running(pkg):
raise QuestError(f'{label} was started on the Frame during the import. Close it, then import again.')
_check_downgrade(android.read_meta(pkg), checked, allow_downgrade)
meta = android.install(str(base), splits=[str(p) for p in splits], source='Meta Quest (USB)')
report('Done', 100)
message = f"Imported {meta['label']} from your Quest. It's in the Steam library."
if wants_data:
what = ' and '.join(x for x in ('its OBB game data' if app['obb'] is not None else '',
'its Android/data files' if data and app['data'] is not None else '') if x)
message += (f" To add {what}, start {meta['label']} from the Steam library, leave it on its first "
"screen, then import it again here: Frame Control copies the data in and closes the game.")
note = android.layer_note(meta)
return {'message': ' '.join([message, *warnings, *([note] if note else [])]), 'app': meta, 'obb': [],
'files': 0, 'warnings': warnings, 'split': bool(splits), 'needs_game_data': wants_data,
'closed': None}
# 2b. The game is open on the Frame: add its data to exactly this install, never replacing it.
_check_target(pkg, android.read_meta(pkg), checked) # early, before copying the data off the Quest
if not wants_data:
raise QuestError(f"{label} is already installed and open on the Frame, and has no game data to add. "
'Close it first to install it again.')
obb_files = []
if app['obb'] is not None:
report('Copying game data (OBB) from your Quest', 30)
(work / 'obb').mkdir()
_pull(serial, f'{OBB}/{pkg}', work / 'obb')
folder = work / 'obb' / pkg
files = sorted(p for p in folder.iterdir() if p.is_file()) if folder.is_dir() else []
obb_files = [p for p in files if re.fullmatch(OBB_NAME.format(re.escape(pkg)), p.name)]
skipped = [p.name for p in files if p not in obb_files]
if skipped:
warnings.append('Not copied, not standard OBB file names: ' + ', '.join(skipped))
data_folder = None
if data and app['data'] is not None:
report('Copying app files from your Quest', 50)
(work / 'data').mkdir()
_pull(serial, f'{DATA}/{pkg}', work / 'data')
data_folder = work / 'data' / pkg if (work / 'data' / pkg).is_dir() else None
result = add_game_data(pkg, checked, [str(p) for p in obb_files], data_folder, report, warnings)
result['split'] = bool(splits)
return result
def add_game_data(pkg, checked, obb_files, data_folder, report, warnings=(), source='your Quest'):
"""Copy game data into the game open on the Frame, then close it: the second step of a
two-step install (Import from my Quest, and My Meta library). checked is inspect()'s
result for the APK the data belongs to; obb_files are local OBB paths, data_folder a
local Android/data/<package> folder or None. Raises QuestError, after closing the game,
if anything fails."""
warnings = list(warnings)
label = checked['label']
copied_obb, copied_files, failure = [], 0, None
# Fetching the data can take a long time: check the target again, and hold the shared
# install lock (install, remove and install_obb take it too) until the game is closed.
with android._install_lock:
if not _running(pkg):
raise QuestError(f'{label} was closed on the Frame during the import. Start it, then import again.')
meta = _check_target(pkg, android.read_meta(pkg), checked)
closed, why = None, None
try:
report('Waiting for Android on the Frame', 70)
_wait_ready(meta)
_check_android_package(meta, checked['version_code'])
if obb_files:
report('Copying game data (OBB) to the Frame', 80)
copied_obb = android.install_obb(pkg, [str(p) for p in obb_files])['obb']
if data_folder:
report('Copying app files to the Frame', 92)
copied_files = push_data(meta, data_folder)
except Exception as e: # any failure, local (OSError) or on the Frame, is reported below
failure = e
finally:
# Whatever happened, even Ctrl-C part-way through a write, the game is closed so it
# never runs on half-copied data. A failed close never hides the original error.
interrupted = sys.exc_info()[0] is not None
closed, why = _close(pkg)
if interrupted and not closed:
print(f"Frame Control couldn't close {meta['label']} on the Frame ({why}); close it from Steam "
'before playing.', file=sys.stderr)
if failure:
message = f"Copying {meta['label']}'s game data failed: {failure}."
message += (' The game was closed; start it and import again to retry.' if closed else
f" Frame Control also couldn't close the game ({why}): close it from Steam, then start it "
'and import again.')
raise QuestError(message) from failure
report('Done', 100)
message = f"Added {meta['label']}'s game data from {source}"
message += ('; the game is closed, start it again from the Steam library.' if closed else
f", but Frame Control couldn't close the game ({why}). Close it from Steam before playing, so it "
'starts with the new data.')
if copied_obb:
message += f' Added {len(copied_obb)} OBB file{"s" if len(copied_obb) != 1 else ""}.'
if copied_files:
message += f' Copied {copied_files} app file{"s" if copied_files != 1 else ""}.'
return {'message': ' '.join([message, *warnings]), 'app': meta, 'obb': copied_obb, 'files': copied_files,
'warnings': warnings, 'split': False, 'needs_game_data': False, 'closed': closed}
def main():
cmd, *args = sys.argv[1:] or ['help']
serial = args[args.index('--serial') + 1] if '--serial' in args and args.index('--serial') + 1 < len(args) else None
try:
if cmd == 'devices':
r = devices()
elif cmd == 'list':
r = packages(find_quest(serial)['serial'])
elif cmd == 'import':
names = [a for a in args if not a.startswith('--') and a != serial]
if len(names) != 1:
raise QuestError('usage: import PACKAGE [--serial SERIAL] [--data] [--allow-downgrade]')
r = import_game(names[0], serial, data='--data' in args, allow_downgrade='--allow-downgrade' in args,
progress=lambda stage, percent=None: print(f'{percent:>3}% {stage}' if percent is not None
else stage, file=sys.stderr))
else:
sys.exit(__doc__)
except (android.FrameError, OSError) as e:
sys.exit(f'error: {e}')
print(json.dumps(r, indent=1))
if __name__ == '__main__':
main()
+1 -11
View File
@@ -6,7 +6,6 @@ error. Sensor paths verified on SteamOS 0.3.0; OpenVR timing on 0.4.1
"""
import ctypes as C
import glob
import hashlib
import math
import json
import os
@@ -122,16 +121,7 @@ def os_release():
k, _, v = line.partition("=")
out[k] = v.strip('"')
return {"version": out.get("VERSION_ID"), "build": out.get("BUILD_ID"),
"variant": out.get("VARIANT_ID"), "headset": headset_id()}
def headset_id():
"""A one-way hash of the headset's /etc/machine-id: the same for this headset on every
computer, and the machine-id itself never leaves the headset. None without one."""
mid = read("/etc/machine-id")
if not mid:
return None
return hashlib.sha256(("frame-control-headset:" + mid).encode()).hexdigest()[:16]
"variant": out.get("VARIANT_ID")}
def volume():
+8 -41
View File
@@ -50,7 +50,6 @@ SETTINGS = STATE / 'settings.json'
OUTBOX = STATE / 'outbox.jsonl'
SENT = STATE / 'sent.jsonl'
SENT_KEEP = 200
FRAMES_SEEN_KEEP = 100 # build/headset pairs already reported, enough for several headsets' updates
OUTBOX_MAX = 2000 # events kept while offline; the oldest go first
FLUSH_EVERY = 60
REPEAT_WINDOW = 600 # the same diagnostic error is sent at most once in this many seconds
@@ -267,23 +266,9 @@ CATEGORIES = [
('layer_missing', re.compile(r'OpenXR compatibility layer')),
('apk_repack_failed', re.compile(r'could not prepare the APK for the Frame')),
('tool_missing', re.compile(r"\[WinError 2\]|No such file or directory: '(?:ssh|scp|rsync|adb)")),
('apk_unreadable', re.compile(r'(?i)not a zip|bad apk|AndroidManifest|ApkError|unexpected package name|'
r'not a readable APK|not an APK|APK manifest|in the APK (?:uses a compression|is )')),
('apk_unreadable', re.compile(r'(?i)not a zip|bad apk|AndroidManifest|ApkError|unexpected package name')),
('cant_run_on_frame', re.compile(r"can't run on the Frame")),
('steam_shortcut', re.compile(r'(?i)steam did not return a shortcut|shortcut list|no Steam shortcut')),
# The Frame-side scripts that talk to the Steam client's UI (frame/android/steam_shortcuts.py), which
# fail with their own words or a Python traceback, its last lines kept. Before frame_unreachable:
# "Connection refused" and "timed out" here are Steam's debug port on the Frame, not ssh.
('steam_not_running', re.compile(r'SharedJSContext not found|'
r'URLError: <urlopen error \[Errno 111\] Connection refused>|'
r'ConnectionRefusedError: \[Errno 111\]')),
('steam_not_responding', re.compile(r'websocket handshake failed|(?:socket\.timeout|TimeoutError): timed out|'
r'(?:\A|\n)EOFError\Z')),
('steam_js_error', re.compile(r'JS error: ')),
('artwork_failed', re.compile(r'incomplete artwork render|artwork renderer did not produce|'
r'render is \d+ bytes of|could not prepare artwork|all five Steam artwork slots')),
# Copying the APK (or its launcher) to the Frame: rsync or scp ran out of time.
('copy_timeout', re.compile(r'to the Frame timed out\Z')),
('frame_not_set_up', re.compile(r'(?i)Could not resolve hostname|no "?frame"? (?:SSH )?alias')),
('frame_auth', re.compile(r'(?i)Permission denied|Host key verification failed|'
r'REMOTE HOST IDENTIFICATION HAS CHANGED|host key is known for')),
@@ -295,8 +280,6 @@ CATEGORIES = [
r'Network is unreachable|Host is down|asleep|kex_exchange|banner exchange|'
r'ssh: connect to host |mux_client_|client_loop: ')),
('frame_disk_full', re.compile(r'(?i)No space left|disk full|ENOSPC')),
# rsync or scp failed for another reason ("connection unexpectedly closed", or no words at all).
('copy_failed', re.compile(r'\Acopying .+ to the Frame failed: ')),
('download_failed', re.compile(r'(?i)HTTP (?:Error )?\d{3}|URLError|download|certificate verify failed')),
('flatpak', re.compile(r'(?i)flatpak|flathub')),
('cancelled', re.compile(r'(?i)cancel')),
@@ -304,16 +287,8 @@ CATEGORIES = [
]
# Python's own errors for a mistake in Frame Control's code, whatever their message says.
BUG_ERRORS = (KeyError, IndexError, TypeError, AttributeError, NameError, AssertionError,
ZeroDivisionError, RecursionError, UnicodeError, json.JSONDecodeError)
def categorize(message):
"""(category, detail): a fixed category name, plus an Android installer code when there is one.
message may be the exception itself: one of BUG_ERRORS is internal_error."""
if isinstance(message, BUG_ERRORS):
return 'internal_error', None
"""(category, detail): a fixed category name, plus an Android installer code when there is one."""
text = str(message or '')
for name, pattern in CATEGORIES:
m = pattern.search(text)
@@ -391,24 +366,16 @@ def app_started():
_save(s)
def frame_seen(build, version, headset=None, saved=None):
"""The Frame's SteamOS build (public build numbers), once per build for each headset.
headset: the anonymous hash from frame_status.headset_id; saved: how many headsets
this copy of the app has set up."""
headset = headset if isinstance(headset, str) and re.fullmatch(r'[0-9a-f]{16}', headset) else None
key = f'{build}/{version}' + (f'/{headset}' if headset else '')
def frame_seen(build, version):
"""The Frame's SteamOS build, once per build (public build numbers)."""
key = f'{build}/{version}'
with _lock:
s = settings()
if not build or key in s['frames_seen']:
return
s['frames_seen'] = (s['frames_seen'] + [key])[-FRAMES_SEEN_KEEP:]
s['frames_seen'] = (s['frames_seen'] + [key])[-20:]
_save(s)
p = {'steamos_build': str(build)[:40], 'steamos_version': str(version or '')[:40]}
if headset:
p['headset_id'] = headset
if isinstance(saved, int) and saved >= 0:
p['headsets_saved'] = saved
capture('frame_connected', p)
capture('frame_connected', {'steamos_build': str(build)[:40], 'steamos_version': str(version or '')[:40]})
def install_finished(kind, ok, seconds=None, error=None, diagnose=True, **props):
@@ -486,7 +453,7 @@ def compat_report(report):
if k in p:
p[k] = scrub(p[k], n)
src = str(p.pop('source', '') or '')
if src in ('F-Droid', 'meta-store'): # fixed source tags, never a person's file name
if src == 'F-Droid':
p['source'] = src
elif src.startswith(('http://', 'https://')) and _scrub_url(src) != '<url>':
p['source'] = _scrub_url(src)
+43 -300
View File
@@ -51,10 +51,6 @@
nav a:hover { color: var(--bright); }
nav a.on { color: var(--blue); border-bottom-color: var(--blue); }
.spacer { flex: 1; }
/* The connection pill and the buttons after it, pushed to the right: closer together than the tabs, and the
pill (not the buttons) gives way when the window is narrow, so nothing is pushed past the right edge. */
.hdr-end { display: flex; align-items: center; gap: 10px; min-width: 0; flex: 0 1 auto; margin-left: auto; }
.hdr-end > :not(.pill, .devsel) { flex: none; }
.chip { display: inline-flex; align-items: center; gap: 8px; height: 32px; padding: 0 12px; border-radius: 3px;
background: var(--btn); color: var(--text); font-size: 13px; white-space: nowrap; }
.dot { width: 8px; height: 8px; border-radius: 50%; background: var(--dim); flex: none; }
@@ -64,7 +60,6 @@
main { max-width: 1480px; margin: 0 auto; padding: 22px calc(28px + env(safe-area-inset-right)) 30px calc(28px + env(safe-area-inset-left)); display: flex; flex-direction: column; gap: 26px; }
.grid-top { display: grid; grid-template-columns: minmax(0, 1.7fr) minmax(330px, 1fr); gap: 22px; }
.grid-3 { display: grid; grid-template-columns: repeat(auto-fit, minmax(300px, 1fr)); gap: 22px; }
@media (max-width: 1366px) { header { gap: 20px; } nav a { padding: 0 11px; } }
@media (max-width: 1150px) { .grid-top, .grid-3 { grid-template-columns: 1fr; } header { gap: 16px; padding: env(safe-area-inset-top) calc(18px + env(safe-area-inset-right)) 0 calc(18px + env(safe-area-inset-left)); } nav a { padding: 0 10px; } }
@media (max-width: 820px) { .brand b { display: none; } }
nav svg { display: none; }
@@ -79,7 +74,7 @@
.page { display: flex; flex-direction: column; gap: 26px; }
.page:not(.on) { display: none; }
nav kbd { font: inherit; font-size: 10.5px; color: var(--dim); margin-left: 7px; letter-spacing: 0; }
@media (max-width: 1366px) { nav kbd { display: none; } } /* the tabs' titles still name the key */
@media (max-width: 1150px) { nav kbd { display: none; } }
/* ---- offline banner ---- */
.banner { display: flex; align-items: center; gap: 14px; padding: 12px 16px; border-radius: 4px;
@@ -92,14 +87,14 @@
.wait::before { content: ""; width: 7px; height: 7px; border-radius: 50%; background: var(--dim); flex: none; }
/* ---- connection pill, its details dialog, and the Devices tab (frame_link.py) ---- */
.pill { height: 40px; padding: 0 12px; gap: 9px; max-width: 420px; min-width: 150px; flex: 0 1 auto; background: var(--btn); }
.pill { height: 40px; padding: 0 12px; gap: 9px; max-width: 420px; min-width: 190px; flex: 0 1 auto; background: var(--btn); }
.pill .dot { flex: none; }
.pill .dot.wait { background: var(--warn); box-shadow: 0 0 6px rgba(217,162,58,.7); animation: pulse 1s ease-in-out infinite; }
@keyframes pulse { 50% { opacity: .35; } }
.pill .pt { display: flex; flex-direction: column; align-items: flex-start; min-width: 0; line-height: 1.2; text-align: left; }
.pill .pt b { font-weight: 500; font-size: 13px; color: var(--bright); max-width: 100%; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.pill .pt span { font-size: 11.5px; color: var(--muted); max-width: 100%; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.devsel { background: var(--btn); color: var(--text); border: 0; border-radius: 3px; height: 40px; padding: 0 8px; font: inherit; font-size: 13px; max-width: 160px; min-width: 90px; }
.devsel { background: var(--btn); color: var(--text); border: 0; border-radius: 3px; height: 40px; padding: 0 8px; font: inherit; font-size: 13px; max-width: 160px; }
.dlg { background: #1e2329; color: var(--text); border: 1px solid rgba(255,255,255,.1); border-radius: 4px;
padding: 22px; width: min(640px, 94vw); box-shadow: 0 20px 60px rgba(0,0,0,.6); }
.dlg::backdrop { background: rgba(0,0,0,.55); }
@@ -137,7 +132,7 @@
.dev-panel select, .dlg select { background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; border-radius: 3px;
padding: 8px 8px; font: inherit; font-size: 13px; }
.addr .t { overflow-wrap: anywhere; }
.addr .s, .dev-problem .s { white-space: normal; }
.addr .s { white-space: normal; }
.addr-edit { display: grid; grid-template-columns: 2fr 1fr 1.4fr auto auto; gap: 8px; align-items: center; width: 100%; }
.addr-add { display: grid; grid-template-columns: 2fr 1fr 1.4fr auto; gap: 8px; margin-top: 12px; }
@media (max-width: 700px) { .dev-form, .addr-edit, .addr-add { grid-template-columns: 1fr; } }
@@ -660,7 +655,7 @@
<a href="#tools" title="Files, clipboard, Linux apps, remote and power (4)"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M14.7 6.3a4 4 0 0 0-5.2 5.2L3.5 17.5a2.1 2.1 0 0 0 3 3l6-6a4 4 0 0 0 5.2-5.2l-2.5 2.5-2.5-2.5z"/></svg>Tools<kbd>4</kbd></a>
<a href="#devices" class="desk-only" title="Headsets, their addresses and the connection (5)"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M5 12.5a10 10 0 0 1 14 0M8.5 16a5 5 0 0 1 7 0"/><circle cx="12" cy="19.5" r="1"/><path d="M2 9a15 15 0 0 1 20 0"/></svg>Devices<kbd>5</kbd></a>
</nav>
<div class="hdr-end">
<div class="spacer"></div>
<button class="pill" id="conn" role="status" aria-live="polite" title="Connection details"><span class="dot wait"></span><span class="pt"><b>Connecting…</b><span></span></span></button>
<select class="devsel desk-only" id="devSel" aria-label="Headset" title="Switch headset" hidden></select>
<span class="chip" id="battChip" title="Battery">—</span>
@@ -670,7 +665,6 @@
<button id="refreshAll" title="Refresh (R)" aria-label="Refresh">
<svg width="15" height="15" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.4" aria-hidden="true"><path d="M21 12a9 9 0 1 1-3-6.7"/><path d="M21 3v6h-6"/></svg>
</button>
</div>
</header>
<main>
@@ -683,8 +677,7 @@
</div>
<div class="notice" id="privacyNotice" hidden>
<div class="grow">Frame Control sends anonymous usage statistics: that it was installed and opened, its version,
your operating system, which tabs you use, whether installs on the Frame worked, and how many headsets you use
(a one-way hash of each headset's machine ID, the same on every computer, so they're counted once). Never file names, paths,
your operating system, which tabs you use, and whether installs on the Frame worked. Never file names, paths,
addresses or anything you've typed, and it isn't linked to you. You can also share whether Android apps
worked and the details of errors, which helps fix problems faster.</div>
<button class="small" id="noticeSettings">Privacy settings</button>
@@ -958,55 +951,6 @@
library under its own name and opens in its own panel. Test launches it and records
whether it stays up; Report records how it worked for you.</div>
</div>
<section class="panel" id="quest">
<div class="shelf-head"><h2>Import from my Quest</h2><span class="count" id="questCount"></span><span class="spacer"></span>
<button class="small" id="questScan" title="Look for a Meta Quest plugged into this computer by USB">Look for my Quest</button></div>
<div class="sub" id="questState" role="status" aria-live="polite">Plug your Quest into this computer by USB, put it on, then Look for my Quest.</div>
<label class="sub" style="display:block;margin-top:8px"><input type="checkbox" id="questData"> Also copy each game's files in Android/data (some games keep saves there)</label>
<label class="sub" style="display:block;margin-top:4px"><input type="checkbox" id="questOlder"> Allow replacing a newer version on the Frame with the Quest's older one</label>
<progress id="questProgress" max="100" hidden aria-label="Import progress" style="width:100%;margin-top:8px"></progress>
<div class="list" id="questApps" style="margin-top:8px"></div>
<div class="hint">Copies games you own from your own Quest (developer mode on) to the Frame. A game with
OBB data takes two imports: the first installs it; start it from the Steam library, then import again to
copy its data in. Frame Control never starts the game itself. Games that
use Meta's licence check install with a warning: on the Frame that check can't reach Meta, so they may quit,
hang, or run without online features. Frame Control won't remove that check. <a href="https://github.com/saphid/frame-control/blob/main/docs/quest-import.md" target="_blank" rel="noopener noreferrer">How it works</a></div>
</section>
<section class="panel" id="metaLib">
<div class="shelf-head"><h2>My Meta library</h2><span class="tag">Experimental</span><span class="count" id="metaCount"></span><span class="spacer"></span>
<button class="small" id="metaRefresh" hidden title="Read your Meta library again">Refresh</button>
<button class="small" id="metaDisconnect" hidden title="Forget your Meta token on this computer (and sign out of Meta in Frame Control)">Disconnect</button></div>
<div class="sub" id="metaState" role="status" aria-live="polite">Off. Nothing is sent to Meta until you connect.</div>
<p class="hint" style="color:var(--warn)">Experimental. This uses Meta's undocumented internal API with your own Meta
access token. It may stop working whenever Meta changes things, and using it may be against Meta's terms; your
account is your responsibility. It only lists and downloads games your account owns, installs them as they are,
and never removes a licence check.</p>
<div id="metaConnectBox">
<label class="sub" style="display:block;margin-top:6px"><input type="checkbox" id="metaAck"> I understand, and want to
use my own Meta token with Frame Control</label>
<div id="metaSignInRow" hidden style="display:flex;gap:8px;align-items:center;margin-top:6px">
<button class="small" id="metaSignIn" disabled title="Opens Meta's own login page in a Frame Control window">Sign in with Meta…</button>
<span class="hint" style="margin:0">or paste your token:</span><span class="spacer"></span>
<button class="small" id="metaSignOut" title="Clear the Meta sign-in that Frame Control's sign-in window keeps">Sign out of Meta in Frame Control</button></div>
<div style="display:flex;gap:8px;margin-top:6px">
<input type="password" id="metaToken" autocomplete="off" spellcheck="false" style="flex:1"
placeholder="Your Meta access token (starts with OC or FRL)" aria-label="Meta access token" disabled>
<button class="small" id="metaConnect" disabled>Connect</button></div>
<div class="hint"><a href="https://github.com/saphid/frame-control/blob/main/docs/meta-library.md#your-token" target="_blank" rel="noopener noreferrer">How to get your token</a>.
Sign in with Meta (desktop app) opens Meta's login page in its own Frame Control window and takes the token
from there; it never reads your browser.
Frame Control checks it with Meta, then keeps it in a private file on this computer (never on the Frame, and
never in reports). Disconnect deletes it.</div>
</div>
<div class="sub" id="metaShare" hidden style="margin-top:6px"></div>
<label class="sub" id="metaOlderBox" hidden style="display:block;margin-top:4px"><input type="checkbox" id="metaOlder"> Allow replacing a newer version on the Frame</label>
<progress id="metaProgress" max="100" hidden aria-label="Install progress" style="width:100%;margin-top:8px"></progress>
<div class="list" id="metaApps" style="margin-top:8px"></div>
<div class="hint">Install downloads the game from Meta and installs it like Import from my Quest; a game with OBB
data takes a second Install while it is open. Install and test also starts it, adds its data, watches it for
about 30 seconds and records whether it ran; Test again does just the test. Results go to the compatibility
reports, shared only if you share compatibility results. <a href="https://github.com/saphid/frame-control/blob/main/docs/meta-library.md" target="_blank" rel="noopener noreferrer">How it works</a></div>
</section>
<section class="panel disp" id="display">
<div class="shelf-head"><h2>Android display</h2><span class="count">Lepton</span><span class="spacer"></span>
<button class="small" id="dispReload" title="Read the current values again">Refresh</button></div>
@@ -1198,8 +1142,7 @@
<div class="shelf-head"><h2>Privacy &amp; updates</h2><span class="spacer"></span><span class="sub" id="appVersion"></span></div>
<label class="popt"><input type="checkbox" id="tUsage"><b>Anonymous usage statistics</b>
<span class="sub">Installs, opens, version, operating system, tabs used, and whether installs on the Frame
worked (with an error category, never the message). F-Droid package names only. Each headset's SteamOS
build, a one-way hash of its machine ID (to count headsets, not just installs), and how many you've set up.</span></label>
worked (with an error category, never the message). F-Droid package names only.</span></label>
<label class="popt"><input type="checkbox" id="tCompat"><b>Share compatibility results</b>
<span class="sub">Your APK reports and tests (package, version, result, your notes) go to the shared
compatibility database, so the verdicts get better for everyone.</span></label>
@@ -1265,6 +1208,7 @@
<div class="sources-intro"><div class="store-eyebrow">Your app shelves</div><h1 id="sourcesTitle">A world of apps.</h1><p>Choose where you discover games and apps. We bring them together for you.</p></div>
<div id="sourcesList"></div>
<form id="addSource" class="source-repo-form"><h2>Add a source</h2><p>Have a favorite F-Droid repository? Add its address to browse its apps here.</p><label>Repository address<input name="url" type="url" required placeholder="https://example.org/fdroid/repo"></label><details><summary>Verify this source (optional)</summary><label>Repository fingerprint<input name="fingerprint" type="text" autocomplete="off" placeholder="Paste the fingerprint from the repository owner"></label></details><button class="action">Add source</button></form>
<form id="itchAccount" class="source-repo-form"><h2>My itch.io purchases</h2><p id="itchAccountNote">Install the Android builds of games you own on itch.io. Create an API key in itch.io settings › API keys and paste it here. It stays on this computer and is only sent to itch.io.</p><label id="itchKeyField">itch.io API key<input name="key" type="password" autocomplete="off" spellcheck="false" maxlength="300" placeholder="Paste your API key"></label><button class="action" id="itchConnect">Connect itch.io</button><button type="button" class="small" id="itchDisconnect" hidden>Disconnect itch.io</button></form>
<p id="sourceError" class="store-notice" role="status" aria-live="polite"></p>
</dialog>
@@ -1484,8 +1428,6 @@ function applyHostWording(h) {
// The iPhone app: the server runs on the Frame, and saving goes to the phone's Photos.
document.body.classList.toggle("mobile", !!h.mobile);
$("shotsFolder").hidden = !!h.mobile;
$("quest").hidden = !!h.mobile;
$("metaLib").hidden = !!h.mobile; // the Quest plugs into a computer; the phone app's server runs on the Frame
if (!h.mobile) $("shotsFolder").textContent = h.fileManager === "your file manager" ? "Open folder" : `Show in ${h.fileManager}`;
$("shotsSaveNew").textContent = h.mobile ? `Save all to ${h.computer}` : `Save new to ${h.computer}`;
$("clipMac").textContent = `Send ${h.computer} clipboard`;
@@ -1646,7 +1588,7 @@ $("vrUtilities").onclick = async e => {
await loadUtilities();
};
// The Frame owns the clock. Poll independently of the selected tab, from the first status check on.
// The Frame owns the clock. Poll independently of status and the selected tab.
let comfortBusy = false, comfortSeen = new Set(), comfortSession = null, comfortHydrated = false;
async function localNotification(message, request = false) {
try {
@@ -1705,12 +1647,7 @@ function renderComfort(s) {
if ($("comfortEvents").textContent !== history) $("comfortEvents").textContent = history;
}
async function pollComfort() {
// Not while the Frame can't be reached: each poll would be another ssh attempt waiting to
// time out, and the banner already says why. Status checks bring it back (setOnline).
if (online === false) {
const message = "Session status appears when the Frame is connected.";
if ($("comfortStatus").textContent !== message) $("comfortStatus").textContent = message;
} else if (!comfortBusy) {
if (!comfortBusy) {
comfortBusy = true;
try { renderComfort(await api("/api/comfort", {action: "status"})); }
catch (e) {
@@ -1744,6 +1681,7 @@ $("castBtn").onclick = () => {
// Fullscreen is a direct user gesture; iPhone falls back to its inline viewer.
$("viewer").requestFullscreen?.().catch(() => {});
};
pollComfort();
// ---- battery ----
function battery(b, power) {
@@ -3204,15 +3142,28 @@ function sourceMark(s) { const name=storeName(s); return `<span class="offer-mar
function sourceDescription(s) { return s.description || ({fdroid:"Independent, open-source apps",sidequest:"Discover something new in VR",itch:"Games from independent creators",github:"Direct from the people who make it"})[s.kind] || (s.name.includes("GitHub") ? "Direct from the people who make it" : s.name.includes("Droid") ? "Independent, open-source apps" : s.name.includes("SideQuest") ? "Discover something new in VR" : s.name.includes("itch") ? "Games from independent creators" : "Apps shared by this community"); }
async function loadSources() {
try {
const {sources} = await api("/api/sources"); sourceState.sources = sources;
const {sources, itch} = await api("/api/sources"); sourceState.sources = sources; showItchAccount(itch);
$("sourceFilter").innerHTML = `<option value="">All sources</option>` + sources.filter(s=>s.enabled).map(s=>`<option value="${esc(s.id)}">${esc(storeName(s.name))}</option>`).join("");
$("sourceFilter").value = sourceState.source;
$("sourcesList").innerHTML = sources.map(s=>`<div class="source-setting">${sourceMark(s.name)}<div class="grow"><strong>${esc(storeName(s.name))}</strong><p>${esc(sourceDescription(s))}</p><small>${esc(({official:"Official store",community:"Community",user:"Added by you"})[s.trust] || "Community")}</small>${s.builtin === false ? `<button class="small" data-remove="${esc(s.id)}" aria-label="Remove ${esc(s.name)}">Remove</button>` : ""}</div><input class="source-toggle" role="switch" type="checkbox" aria-label="Enable ${esc(storeName(s.name))}" data-enable="${esc(s.id)}" ${s.enabled ? "checked" : ""}></div>`).join("") || `<p class="store-notice">Your sources will appear here when they are available.</p>`;
} catch (e) { $("sourceError").textContent = e.message; }
}
// Owned on itch.io (the user's own purchases) or free: what the store shows instead of a price.
const storePrice = o => o.owned ? (o.owned_label || "Owned") : "Free";
function showItchAccount(itch) {
const on = !!itch?.connected;
$("itchAccountNote").textContent = on ? `Connected${itch.username ? " as " + itch.username : ""}. Android builds of games you own appear in the store, marked Owned on itch.io. Disconnecting deletes the API key from this computer.` : "Install the Android builds of games you own on itch.io. Create an API key in itch.io settings › API keys and paste it here. It stays on this computer and is only sent to itch.io.";
$("itchKeyField").hidden = on; $("itchConnect").hidden = on; $("itchDisconnect").hidden = !on;
$("itchAccount").elements.key.required = !on;
}
async function itchAccount(body) {
$("sourceError").textContent = body.action === 'itch-connect' ? "Checking the key with itch.io…" : "Disconnecting…";
try { const result=await api('/api/sources',body);$("sourceError").textContent=result.message;showItchAccount(result.itch);sourceState.source='';await loadSources();await searchSources(); }
catch(e) { $("sourceError").textContent=e.message;await loadSources(); }
}
function storeCard(app) {
const i=sourceState.apps.indexOf(app), o=app.offers[0];
return `<button class="store-card" data-app="${i}" aria-label="View ${esc(app.name)}">${storeArt(o)}<span class="store-card-content"><span class="store-card-title">${storeIcon(o)}<strong>${esc(app.name)}</strong></span><span class="store-summary">${esc(storeText(app.summary) || "A new addition to your Frame")}</span><span class="store-card-foot"><span>${o.vr ? "VR experience" : o.vr === false ? "Flat-screen app" : "Android app"}</span><b>Free</b></span></span></button>`;
return `<button class="store-card" data-app="${i}" aria-label="View ${esc(app.name)}">${storeArt(o)}<span class="store-card-content"><span class="store-card-title">${storeIcon(o)}<strong>${esc(app.name)}</strong></span><span class="store-summary">${esc(storeText(app.summary) || "A new addition to your Frame")}</span><span class="store-card-foot"><span>${o.vr ? "VR experience" : o.vr === false ? "Flat-screen app" : "Android app"}</span><b>${esc(storePrice(app.offers.find(x=>x.owned) || o))}</b></span></span></button>`;
}
function storeRow(title, subtitle, apps) { return apps.length ? `<section class="store-row"><div class="store-row-head"><h2>${esc(title)}</h2><p>${esc(subtitle)}</p></div><div class="store-grid">${apps.slice(0,5).map(storeCard).join("")}</div></section>` : ""; }
function storeEmpty() {
@@ -3257,7 +3208,7 @@ async function searchSources(quiet) {
finally { if(request===sourceState.request)$("searchGrid").setAttribute("aria-busy","false"); }
}
function currentOffer() { return sourceState.detail?.app.offers[sourceState.detail.choice]; }
function detailBadges(o) { return `<span class="store-badge">${o.vr ? "VR experience" : o.vr===false ? "Flat-screen app" : "Android app"}</span>${o.verdict?.tone==='works' ? '<span class="store-badge works">✓ Works on Frame</span>' : ''}<span class="store-badge">Free</span>${storeOpenSource(o) ? '<span class="store-badge">Open source</span>' : ''}`; }
function detailBadges(o) { return `<span class="store-badge">${o.vr ? "VR experience" : o.vr===false ? "Flat-screen app" : "Android app"}</span>${o.verdict?.tone==='works' ? '<span class="store-badge works">✓ Works on Frame</span>' : ''}<span class="store-badge">${esc(storePrice(o))}</span>${storeOpenSource(o) ? '<span class="store-badge">Open source</span>' : ''}`; }
// The store's app page. (Not renderDetail: the Devices page declares that further down,
// and a later function declaration silently replaces an earlier one; issue #68.)
// True when the page rendered; false when it shows the error instead.
@@ -3273,7 +3224,7 @@ function renderAppDetail() {
function renderAppDetailBody() {
const restoreOfferFocus=document.activeElement?.name==='storeOffer';
const d=sourceState.detail;if(!d)return;const o=currentOffer(), images=o.artwork?.screenshots||[], page=sourcePage(o.page);
$("detailBody").innerHTML=storeArt(o,"detail-banner",true)+`<div class="detail-content"><div class="detail-heading">${storeIcon(o)}<div><h1 id="detailTitle">${esc(o.name)}</h1><p>${esc(o.developer || o.author || 'From '+storeName(o.source_name))}</p></div></div><div class="store-badges">${detailBadges(o)}</div><div class="detail-layout"><div><div class="detail-gallery" id="detailGallery" aria-label="App screenshots" tabindex="0">${images.map((p,i)=>storeArt(o,"",false,p,`${o.name} screenshot ${i+1}`)).join("")}</div>${images.length ? `<div class="gallery-controls"><span>${images.length} screenshots · scroll to explore</span><div><button data-gallery="-1" aria-label="Previous screenshot">←</button> <button data-gallery="1" aria-label="Next screenshot">→</button></div></div>` : ''}<h2>About this app</h2><div class="detail-description">${esc(storeText(o.description || o.summary || "Discover this app on its developer's page."))}</div>${page ? `<p><a href="${esc(page)}" target="_blank" rel="noopener noreferrer">Meet the creators ↗</a></p>` : ''}<details class="detail-technical"><summary>Technical details</summary><dl><dt>Package</dt><dd>${esc(o.package||"Unknown")}</dd><dt>Version</dt><dd>${esc(o.version||"Unknown")} (${esc(o.version_code??"unknown")})</dd><dt>Android API</dt><dd>${esc(o.min_sdk??"Unknown")}</dd><dt>Architecture</dt><dd>${esc(o.abis?.join(', ')||"Unknown / no native code")}</dd><dt>Updated</dt><dd>${esc(o.updated||"Unknown")}</dd><dt>Compatibility</dt><dd>${(o.fit?.reasons||[]).map(esc).join(' · ')||'No known installation blockers'}</dd></dl></details></div><aside class="detail-buy"><div class="detail-price">Free</div><p class="detail-small">No purchase needed.</p><p class="detail-verdict ${esc(o.verdict?.tone||'unknown')}">${esc(o.verdict?.label||'Not yet checked on the Frame')}</p><button id="detailInstall" class="action store-primary">Install</button><progress id="detailProgress" max="100" hidden aria-label="Installation progress"></progress><p id="detailInstallNote" class="detail-small" role="status" aria-live="polite"></p><h2>Get it from</h2><div id="detailOffers">${d.app.offers.map((v,j)=>`<label class="offer-choice"><input type="radio" name="storeOffer" value="${j}" ${j===d.choice?'checked':''}><span>${sourceMark(v.source_name)}</span><span><strong>${esc(storeName(v.source_name))}${j===0?' · Recommended':''}</strong><small>${v.verified===true?'✓ Verified download':'Verification not provided'}</small><small>${esc(v.version ? 'v'+v.version : 'Latest release')}${v.size!=null?' · '+(v.size/1048576).toFixed(0)+' MB':''}</small></span></label>`).join('')}</div></aside></div></div>`;
$("detailBody").innerHTML=storeArt(o,"detail-banner",true)+`<div class="detail-content"><div class="detail-heading">${storeIcon(o)}<div><h1 id="detailTitle">${esc(o.name)}</h1><p>${esc(o.developer || o.author || 'From '+storeName(o.source_name))}</p></div></div><div class="store-badges">${detailBadges(o)}</div><div class="detail-layout"><div><div class="detail-gallery" id="detailGallery" aria-label="App screenshots" tabindex="0">${images.map((p,i)=>storeArt(o,"",false,p,`${o.name} screenshot ${i+1}`)).join("")}</div>${images.length ? `<div class="gallery-controls"><span>${images.length} screenshots · scroll to explore</span><div><button data-gallery="-1" aria-label="Previous screenshot">←</button> <button data-gallery="1" aria-label="Next screenshot">→</button></div></div>` : ''}<h2>About this app</h2><div class="detail-description">${esc(storeText(o.description || o.summary || "Discover this app on its developer's page."))}</div>${page ? `<p><a href="${esc(page)}" target="_blank" rel="noopener noreferrer">Meet the creators ↗</a></p>` : ''}<details class="detail-technical"><summary>Technical details</summary><dl><dt>Package</dt><dd>${esc(o.package||"Unknown")}</dd><dt>Version</dt><dd>${esc(o.version||"Unknown")} (${esc(o.version_code??"unknown")})</dd><dt>Android API</dt><dd>${esc(o.min_sdk??"Unknown")}</dd><dt>Architecture</dt><dd>${esc(o.abis?.join(', ')||"Unknown / no native code")}</dd><dt>Updated</dt><dd>${esc(o.updated||"Unknown")}</dd><dt>Compatibility</dt><dd>${(o.fit?.reasons||[]).map(esc).join(' · ')||'No known installation blockers'}</dd></dl></details></div><aside class="detail-buy"><div class="detail-price">${esc(storePrice(o))}</div><p class="detail-small">${o.owned ? "Downloads with your itch.io account." : "No purchase needed."}</p><p class="detail-verdict ${esc(o.verdict?.tone||'unknown')}">${esc(o.verdict?.label||'Not yet checked on the Frame')}</p><button id="detailInstall" class="action store-primary">Install</button><progress id="detailProgress" max="100" hidden aria-label="Installation progress"></progress><p id="detailInstallNote" class="detail-small" role="status" aria-live="polite"></p><h2>Get it from</h2><div id="detailOffers">${d.app.offers.map((v,j)=>`<label class="offer-choice"><input type="radio" name="storeOffer" value="${j}" ${j===d.choice?'checked':''}><span>${sourceMark(v.source_name)}</span><span><strong>${esc(storeName(v.source_name))}${j===0?' · Recommended':''}</strong><small>${v.verified===true?'✓ Verified download':'Verification not provided'}</small><small>${esc(v.version ? 'v'+v.version : 'Latest release')}${v.size!=null?' · '+(v.size/1048576).toFixed(0)+' MB':''}</small></span></label>`).join('')}</div></aside></div></div>`;
updateInstall();
if(restoreOfferFocus)document.querySelector('#detailOffers input:checked')?.focus();
}
@@ -3343,6 +3294,9 @@ async function addSource(body) {
if(result){$("addSource").reset();sourceState.source='';await loadSources();await searchSources();}
}
$("addSource").onsubmit=e=>{e.preventDefault();const f=new FormData(e.target);addSource({action:'add',url:f.get('url'),fingerprint:f.get('fingerprint')});};
// The key goes to the server once and leaves the page straight away: the field is cleared before the reply.
$("itchAccount").onsubmit=e=>{e.preventDefault();const input=e.target.elements.key, key=input.value.trim();input.value='';if(key)itchAccount({action:'itch-connect',key});};
$("itchDisconnect").onclick=()=>itchAccount({action:'itch-disconnect'});
document.querySelectorAll('[data-close]').forEach(b=>b.onclick=()=>$(b.dataset.close).close());
$('appDetail').addEventListener('close',()=>{sourceState.detailRequest++;});
document.querySelectorAll('[data-store-tab]').forEach(b=>b.onclick=()=>{const browse=b.dataset.storeTab==='browse';$('appStore').hidden=!browse;$('androidLibrary').hidden=browse;$('repNew').hidden=browse;$('sourcesOpen').hidden=!browse;document.querySelectorAll('[data-store-tab]').forEach(t=>{t.classList.toggle('on',t===b);t.setAttribute('aria-pressed',t===b);});if(!browse)loadAndroid();});
@@ -3357,7 +3311,6 @@ async function loadAndroid() {
try {
androidApps = (await api("/api/android")).apps;
} catch (e) { return failed($("andApps"), e); }
if (metaLib.connected) renderMetaApps(); // "On the Frame" marks in My Meta library
$("andCount").textContent = androidApps.length ? `${androidApps.length} installed` : "";
$("andApps").innerHTML = androidApps.length ? androidApps.map(a => {
const c = cat.apps && cat.apps.find(x => x.p === a.package);
@@ -3381,199 +3334,6 @@ async function loadAndroid() {
});
}
// ---- Import from my Quest (frame_quest.py): adb on this computer, never on the Frame ----
const quest = { serial: null };
function questSize(bytes) {
if (bytes == null) return "";
if (bytes >= 1073741824) return `${(bytes / 1073741824).toFixed(1)} GB`;
if (bytes >= 1048576) return `${Math.round(bytes / 1048576)} MB`;
return `${Math.max(1, Math.round(bytes / 1024))} KB`;
}
async function loadQuest() {
$("questScan").disabled = true; $("questState").textContent = "Looking for your Quest…"; $("questApps").innerHTML = "";
let d;
try { d = await api("/api/quest"); }
catch (e) { $("questState").textContent = e.message; return; }
finally { $("questScan").disabled = false; }
quest.serial = d.quest ? d.quest.serial : null;
$("questCount").textContent = d.quest ? `${d.apps.length} app${d.apps.length === 1 ? "" : "s"}` : "";
if (!d.quest) { $("questState").textContent = d.guidance || "No Quest found."; return; }
$("questState").textContent = `${d.quest.model || "Quest"} connected (${d.quest.serial}). Games that use Meta's licence check install with a warning: they may quit or hang on the Frame.`;
const installed = new Set(androidApps.map(a => a.package));
$("questApps").innerHTML = d.apps.length ? d.apps.map(a => {
const bits = [a.split ? `split APK (${a.apks.length} files)` : "single APK"];
if (a.obb != null) bits.push(`OBB ${questSize(a.obb)}`);
if (a.data != null) bits.push(`Android/data ${questSize(a.data)}`);
if (a.meta) bits.push("Meta app: needs Meta's services");
return `<div class="item"><div class="grow"><div class="t">${esc(a.package)}</div><div class="s">${esc(bits.join(" · "))}</div></div>
${installed.has(a.package) ? `<span class="tag">On the Frame</span>` : ""}
<button class="small" data-quest-import="${esc(a.package)}"${installing.has("quest:" + a.package) ? " disabled" : ""}>${installed.has(a.package) ? "Import again" : "Import"}</button></div>`;
}).join("") : `<div class="sub">No games or apps you installed are on this Quest.</div>`;
}
$("questScan").onclick = loadQuest;
$("questApps").addEventListener("click", e => {
const b = e.target.closest("[data-quest-import]");
if (!b) return;
const pkg = b.dataset.questImport, key = "quest:" + pkg;
if (installing.has(key)) return;
b.disabled = true;
const bar = $("questProgress");
bar.hidden = false; bar.value = 0;
runJob(`Import ${pkg} from Quest`, key,
() => api("/api/quest", { action: "import", package: pkg, serial: quest.serial, data: $("questData").checked, allowDowngrade: $("questOlder").checked }),
j => { if (j.stage) $("questState").textContent = `${pkg}: ${j.stage}`; if (j.percent != null) bar.value = j.percent; })
.then(r => { bar.hidden = true; if (r && r.app) { $("questState").textContent = r.message || `Imported ${pkg}`; loadAndroid(); }
else { b.disabled = false; $("questState").textContent = `${pkg} wasn't imported; the reason is in the message log below.`; } });
});
// ---- My Meta library, Experimental (frame_meta.py): the person's own token, games they own ----
// The token is only ever in the password box until Connect sends it; the server never sends it back.
const metaLib = { apps: [], results: new Map(), connected: false };
const META_VERDICT = { works: ["v-works", "Works on Frame"], maybe: ["v-maybe", "Works with problems"], no: ["v-no", "Doesn't work"] };
function metaShareNote(s) {
const box = $("metaShare");
if (!metaLib.connected || !s) { box.hidden = true; return; }
box.hidden = false;
box.innerHTML = s.shared ? "Test results are shared with the compatibility database (Privacy settings)."
: s.blocked ? `Test results are saved on this computer only (${esc(s.blocked)}).`
: `Test results are saved on this computer only. <button class="small" id="metaShareOn">Share test results</button>`;
const on = $("metaShareOn");
if (on) on.onclick = async () => {
if (!confirm("Share compatibility results?\n\nYour test results (package, version, whether it ran, and why not) go to "
+ "Frame Control's shared compatibility database, so everyone sees better verdicts. Never your token, "
+ "account or game files. You can turn it off in Privacy settings.")) return;
await setTelemetry({ compat: true });
loadMeta();
};
}
function renderMeta(d) {
metaLib.connected = !!d.connected;
$("metaConnectBox").hidden = metaLib.connected;
$("metaDisconnect").hidden = $("metaRefresh").hidden = $("metaOlderBox").hidden = !metaLib.connected;
$("metaState").textContent = metaLib.connected
? `Connected${d.user && d.user.name ? ` as ${d.user.name}` : ""}.` : "Off. Nothing is sent to Meta until you connect.";
metaShareNote(d.sharing);
}
function renderMetaApps() {
$("metaCount").textContent = metaLib.connected ? `${metaLib.apps.length} game${metaLib.apps.length === 1 ? "" : "s"}` : "";
if (!metaLib.connected) { $("metaApps").innerHTML = ""; return; }
const installed = new Set(androidApps.map(a => a.package));
$("metaApps").innerHTML = metaLib.apps.length ? metaLib.apps.map(a => {
const bits = [a.package || "package unknown", a.version ? `v${a.version}${a.version_code != null ? ` (${a.version_code})` : ""}` : ""];
if (a.size) bits.push(questSize(a.size));
if (a.data_files) bits.push(`OBB ${questSize(a.data_size)}`);
if (a.problem) bits.push(a.problem);
const v = a.verdict && META_VERDICT[a.verdict.verdict];
const verdict = v ? `<div class="s ${v[0]}" title="${esc((a.verdict.lines || []).join("\n"))}">Community: ${v[1]}</div>` : "";
const r = metaLib.results.get(a.package);
const result = r ? `<div class="s">${esc(r)}</div>` : "";
const busy = installing.has("meta:" + a.id) ? " disabled" : "";
const on = installed.has(a.package);
const can = a.package && !a.problem;
return `<div class="item"><div class="grow"><div class="t">${esc(a.title)}</div><div class="s">${esc(bits.filter(Boolean).join(" · "))}</div>${verdict}${result}</div>
${on ? `<span class="tag">On the Frame</span>` : ""}
${can ? `<button class="small" data-meta-install="${esc(a.id)}"${busy}>${on ? "Install again" : "Install"}</button>
<button class="small" data-meta-install="${esc(a.id)}" data-test="1"${busy} title="Install, start it, watch it for about 30 s and record whether it ran">Install and test</button>` : ""}
${on ? `<button class="small" data-meta-test="${esc(a.package)}"${busy} title="Start it again, watch it for about 30 s and record whether it ran">Test again</button>` : ""}</div>`;
}).join("") : `<div class="sub">No Quest games in this Meta account.</div>`;
}
async function loadMeta(refresh = false) {
let d;
try { d = await api("/api/meta"); } catch (e) { $("metaState").textContent = e.message; return; }
renderMeta(d);
if (!d.connected) { metaLib.apps = []; renderMetaApps(); return; }
$("metaState").textContent += " Reading your library…";
try { d = await api(`/api/meta?library=1${refresh ? "&refresh=1" : ""}`); }
catch (e) { $("metaState").textContent = e.message; return; }
renderMeta(d);
metaLib.apps = d.apps || [];
renderMetaApps();
}
// Sign in with Meta: the desktop app only (app/meta-signin.js); a plain browser pastes the token.
const metaSignInOk = !!(window.frameApp && window.frameApp.meta && window.frameApp.meta.signIn);
$("metaSignInRow").hidden = !metaSignInOk;
let metaDisconnecting = false; // from pressing Disconnect until the server has deleted the token
$("metaAck").onchange = e => {
$("metaToken").disabled = $("metaConnect").disabled = !e.target.checked;
$("metaSignIn").disabled = !e.target.checked || metaDisconnecting;
};
$("metaSignIn").onclick = async () => {
if (!$("metaAck").checked || metaDisconnecting) return;
$("metaSignIn").disabled = true;
$("metaState").textContent = "Sign in to Meta in the window that opened…";
try {
const r = await window.frameApp.meta.signIn(true);
if (r && r.ok) { toast(r.message || "Connected"); loadMeta(); }
else { await loadMeta(); $("metaState").textContent = (r && r.error) || "Sign-in didn't finish."; }
} catch (e) { $("metaState").textContent = e.message; }
finally { $("metaSignIn").disabled = !$("metaAck").checked || metaDisconnecting; }
};
$("metaSignOut").onclick = async () => {
try { await window.frameApp.meta.signOut(); toast("Signed out of Meta in Frame Control"); }
catch (e) { toast(e.message, true); }
};
$("metaConnect").onclick = async () => {
const token = $("metaToken").value;
$("metaConnect").disabled = true;
try {
const d = await api("/api/meta", { action: "connect", token, acknowledged: $("metaAck").checked });
$("metaToken").value = "";
toast(d.message || "Connected");
loadMeta();
} catch (e) { $("metaState").textContent = e.message; }
finally { $("metaConnect").disabled = !$("metaAck").checked; }
};
$("metaDisconnect").onclick = async () => {
// One Disconnect at a time, guarded from before the confirm until the delete returns.
if (metaDisconnecting) return;
metaDisconnecting = true; $("metaDisconnect").disabled = $("metaSignIn").disabled = true;
if (!confirm("Disconnect your Meta account? Its token is deleted from this computer. Games already installed stay."
+ (metaSignInOk ? "\n\nThis also signs out of Meta in Frame Control (its Sign in with Meta window), so the next sign-in starts fresh." : ""))) {
metaDisconnecting = false; $("metaDisconnect").disabled = false;
$("metaSignIn").disabled = !$("metaAck").checked;
return;
}
// No sign-in from here until the token is deleted. Sign-out first: it stops the sign-in
// window (and any connect it started) before the account is forgotten.
try {
if (metaSignInOk) {
try { await window.frameApp.meta.signOut(); } catch (e) { toast(`Couldn't sign out of Meta in Frame Control: ${e.message}`, true); }
}
try { const d = await api("/api/meta", { action: "disconnect" }); toast(d.message || "Disconnected"); }
catch (e) { toast(e.message, true); }
} finally { metaDisconnecting = false; $("metaDisconnect").disabled = false; }
$("metaAck").checked = false; $("metaToken").disabled = $("metaConnect").disabled = $("metaSignIn").disabled = true;
metaLib.results.clear(); loadMeta();
};
$("metaRefresh").onclick = () => loadMeta(true);
$("metaApps").addEventListener("click", async e => {
const b = e.target.closest("[data-meta-install],[data-meta-test]");
if (!b) return;
const bar = $("metaProgress");
if (b.dataset.metaTest) {
const pkg = b.dataset.metaTest, key = "meta:" + (metaLib.apps.find(a => a.package === pkg) || {}).id;
if (!confirm("Test it now?\n\nIt opens in the headset for about 30 seconds and records whether it stays up.")) return;
b.disabled = true;
const r = await runJob(`Test ${pkg}`, key, () => api("/api/meta", { action: "test", package: pkg }));
if (r && r.message) metaLib.results.set(pkg, r.message);
renderMetaApps(); loadReports();
return;
}
const app = metaLib.apps.find(a => a.id === b.dataset.metaInstall), test = !!b.dataset.test;
if (!app || installing.has("meta:" + app.id)) return;
if (test && !confirm(`Install and test ${app.title}?\n\nFrame Control starts it in the headset (twice if it has game data to add), `
+ "watches it for about 30 seconds and records whether it ran.")) return;
b.disabled = true; bar.hidden = false; bar.value = 0;
const r = await runJob(`${test ? "Install and test" : "Install"} ${app.title} from Meta`, "meta:" + app.id,
() => api("/api/meta", { action: "install", app: app.id, test, allowDowngrade: $("metaOlder").checked }),
j => { if (j.stage) $("metaState").textContent = `${app.title}: ${j.stage}`; if (j.percent != null) bar.value = j.percent; });
bar.hidden = true;
if (r && r.message) { $("metaState").textContent = r.message; metaLib.results.set(app.package, r.test ? r.test.message : r.message); }
else $("metaState").textContent = `${app.title} wasn't installed; the reason is in the message log below.`;
loadAndroid(); renderMetaApps(); if (test) loadReports();
});
loadMeta();
async function loadCatalog() {
try { cat.apps = (await api("/api/android/catalog")).apps; }
catch (e) { $("catGrid").innerHTML = `<div class="sub">${esc(e.message)}</div>`; return; }
@@ -4752,7 +4512,7 @@ if (window.frameApp && window.frameApp.onInstallLink) {
}
setView("headset");
refresh().then(() => { loadShots(); pollComfort(); }); // after status, so app names resolve and comfort knows if the Frame answers
refresh().then(() => loadShots()); // after status, so app names resolve
document.addEventListener("visibilitychange", () => { if (!document.hidden && online === false) refresh(); });
</script>
<script>
@@ -4781,20 +4541,6 @@ function netLabel(s) {
}
function viaText(v) { return v ? v.host + (v.ip && v.ip !== v.host ? ` (${v.ip})` : "") : ""; }
// Why the last attempt failed, in a few words: the server's short reason, else the stage that failed.
function failureReason(s) {
const err = s.error || {};
if (err.short) return err.short;
const st = (s.stages || []).find(x => x.id === err.stage);
return st ? `${st.label} failed` : "Not connected";
}
// A sentence or a label as a clause inside another sentence: no closing full stop (the sentence
// adds its own) and a lower-case first letter, unless the first word is an abbreviation ("SSH").
function asClause(text) {
const t = String(text || "").trim().replace(/[.\s]+$/, "");
return /^[A-Z][a-z]/.test(t) ? t[0].toLowerCase() + t.slice(1) : t;
}
// The pill in the header: always there, and says what's happening right now.
function renderPill() {
const s = link.s;
@@ -4806,8 +4552,9 @@ function renderPill() {
line = `${netLabel(s)} → ${viaText(s.via)}` + (s.via && s.via.rtt_ms != null ? ` · ${s.via.rtt_ms} ms` : "");
} else if (s.phase === "failed") {
dot = "off";
// The reason first, in a few words, so a narrow pill cuts the countdown rather than the reason.
line = failureReason(s) + (s.retry_at ? ` · retry in ${Math.max(0, Math.ceil(s.retry_at - serverNow()))} s` : "");
const left = s.retry_at ? Math.max(0, Math.ceil(s.retry_at - serverNow())) : null;
const st = (s.stages || []).find(x => x.id === (s.error || {}).stage);
line = `Offline: ${st ? st.label.toLowerCase() : "not connected"} failed` + (left != null ? ` · retry in ${left} s` : "");
} else {
const st = activeStage(s);
const probing = (s.probes || []).find(p => p.state === "trying" || p.state === "resolving");
@@ -4815,7 +4562,7 @@ function renderPill() {
: "Connecting…";
}
$("conn").innerHTML = `<span class="dot ${dot}"></span><span class="pt"><b>${esc(dev)}</b><span>${esc(line)}</span></span>`;
$("conn").title = `${dev}${s.phase === "failed" ? " is offline" : ""}: ${line}. Click for details.`;
$("conn").title = `${dev}: ${line}. Click for details.`;
return true;
}
window.connPill = renderPill;
@@ -4832,13 +4579,13 @@ function renderBanner() {
if (s.phase === "failed" && s.error) {
$("offline").hidden = false;
$("offMsg").textContent = s.error.message;
const tried = (s.probes || []).map(p => `${p.host}: ${asClause(p.detail)}`).join("; ");
const tried = (s.probes || []).map(p => `${p.host}: ${p.detail.toLowerCase()}`).join("; ");
const left = s.retry_at ? Math.max(0, Math.ceil(s.retry_at - serverNow())) : null;
$("offDetail").textContent = `${s.device ? s.device.name : "Frame"} on ${netLabel(s)}. ${tried ? "Tried " + tried + ". " : ""}` +
(left != null ? `Trying again in ${left} s.` : "");
} else if (s.phase === "connecting" && online === false) {
const st = activeStage(s);
$("offDetail").textContent = `Trying again: ${st ? asClause(st.label) + (st.detail ? " (" + st.detail + ")" : "") : "connecting"}…`;
$("offDetail").textContent = `Trying again: ${st ? st.label.toLowerCase() + (st.detail ? " (" + st.detail + ")" : "") : "connecting"}…`;
}
}
window.connBanner = renderBanner;
@@ -5016,10 +4763,7 @@ async function useDevice(id) {
if (!res) { // refused (an install is running): show the headset still in use
const current = dv.list.find(x => x.active);
if (current) $("devSel").value = current.id;
return;
}
// The Devices tab shows the headset now in use, not the one it was showing before.
if (dv.sel !== id) { dv.sel = id; dv.found = null; dv.editing = null; renderDevices(); }
}
async function devAction(label, body, btn) {
const res = await act(label, () => api("/api/devices", body), btn);
@@ -5031,8 +4775,7 @@ function deviceStatus(d) {
const s = link.s;
if (d.active && s) {
return s.phase === "connected" ? ["on", `Connected via ${viaText(s.via)}`]
: s.phase === "failed" ? ["off", s.error ? s.error.short || s.error.message : "Offline", s.error && s.error.message]
: ["", "Connecting…"];
: s.phase === "failed" ? ["off", s.error ? s.error.message : "Offline"] : ["", "Connecting…"];
}
const last = Math.max(0, ...d.addresses.map(a => a.last_ok || 0));
return ["", d.transient ? "Not set up in Frame Control" : last ? `Last connected ${ago(last)}` : "Not connected yet"];
@@ -5042,10 +4785,10 @@ function renderDevices() {
if (!dv.data) return;
$("devCount").textContent = dv.list.length;
$("devList").innerHTML = dv.list.map(d => {
const [dot, text, full] = deviceStatus(d);
const [dot, text] = deviceStatus(d);
return `<div class="item dev-item${d.id === dv.sel ? " sel" : ""}" data-dev="${esc(d.id)}">
<span class="dot ${dot}"></span><div class="grow"><div class="t">${esc(d.name)} ${d.active ? '<span class="tag">In use</span>' : ""}</div>
<div class="s"${full ? ` title="${esc(full)}"` : ""}>ssh ${esc(d.alias)} · ${esc(text)}</div></div>
<div class="s">ssh ${esc(d.alias)} · ${esc(text)}</div></div>
${d.active ? "" : `<button class="small" data-use="${esc(d.id)}">Use this headset</button>`}</div>`;
}).join("") || `<div class="sub">No headsets yet. Add one to get started.</div>`;
$("devList").querySelectorAll("[data-dev]").forEach(el => el.onclick = e => {
@@ -5096,10 +4839,10 @@ function renderDetail() {
<button class="small" data-mv="1" title="Try later"${i < d.addresses.length - 1 ? "" : " disabled"}>↓</button>
<button class="small" data-ed>Edit</button><button class="small danger" data-rm>Remove</button></div>`).join("");
const err = d.active && link.s && link.s.phase === "failed" ? link.s.error : null;
const problem = err && err.hostkey ? `<div class="item dev-problem" role="alert"><div class="grow"><div class="t">${err.hostkey.state === "missing"
const problem = err && err.hostkey ? `<div class="item" role="alert"><div class="grow"><div class="t">${err.hostkey.state === "missing"
? "Its saved SSH identity is missing" : "Its SSH identity has changed"}</div>
<div class="s">${esc(err.message)}</div></div><button class="small action" id="dvTrustNow">Check the new key…</button></div>`
: err && err.auth === "denied" ? `<div class="item dev-problem" role="alert"><div class="grow"><div class="t">The headset didn't accept this computer's key</div>
: err && err.auth === "denied" ? `<div class="item" role="alert"><div class="grow"><div class="t">The headset didn't accept this computer's key</div>
<div class="s">Set Up Connection pairs this computer with it again. On the headset, Developer Mode must be on.</div></div>
<button class="small action" id="dvPair">Set Up Connection…</button></div>` : "";
el.innerHTML = `<div class="shelf-head"><h2>${esc(d.name)}</h2>${d.active ? '<span class="count">In use</span>' : ""}<span class="spacer"></span>
+30 -194
View File
@@ -46,6 +46,7 @@ import frame_assistant # noqa: E402
import frame_android # noqa: E402
from apk_sources import search as apk_search, SourceError # noqa: E402
from apk_sources import fdroid as apk_fdroid # noqa: E402
from apk_sources import _itch_account as itch_account # noqa: E402
import frame_apk_versions # noqa: E402
import frame_catalog # noqa: E402
import frame_devices # noqa: E402
@@ -57,8 +58,6 @@ import frame_link # noqa: E402
import frame_macview # noqa: E402
import frame_media # noqa: E402
import frame_panels # noqa: E402
import frame_quest # noqa: E402
import frame_meta # noqa: E402
import frame_report # noqa: E402
import frame_store # noqa: E402
import frame_telemetry # noqa: E402
@@ -405,38 +404,11 @@ def status(_body):
s = json.loads(ssh("python3 -", stdin=(HERE / "frame_status.py").read_text(), timeout=20))
osr = s.get("os") if isinstance(s, dict) else None
if isinstance(osr, dict):
# The headset's hash is for usage statistics only; the page and MCP clients don't get it.
frame_telemetry.frame_seen(osr.get("build"), osr.get("version"), osr.pop("headset", None),
len(LINK.reg.devices()) if LINK else None)
frame_telemetry.frame_seen(osr.get("build"), osr.get("version"))
frame_report.frame.update(build=osr.get("build"), version=osr.get("version"))
return s
COMFORT_FILES = ("frame_comfort.py", "frame_status.py", "frame_steam.py")
COMFORT_MISSING = {"comfort": "missing"} # what COMFORT_RUN prints when the cached copy isn't complete
# Runs frame_comfort.py from ~/.cache/frame-control/comfort/<version>, first writing any of
# `files` that aren't there (none: just check they all are). Private, and each file written
# whole or not at all, so a copy cut short is finished by the next call.
COMFORT_RUN = """import json, os, pathlib, subprocess, sys
os.umask(0o077)
files = %r
root = pathlib.Path.home() / '.cache/frame-control/comfort' / %r
if files:
root.mkdir(parents=True, exist_ok=True)
for name, source in files.items():
path = root / name
if not path.exists():
tmp = root / (name + '.' + str(os.getpid()))
tmp.write_text(source)
tmp.replace(path)
if not all((root / name).is_file() for name in %r):
print('{"comfort": "missing"}')
sys.exit()
r = subprocess.run([sys.executable, str(root / 'frame_comfort.py'), %r], capture_output=True, text=True)
print(r.stdout, end='')
"""
def comfort(body):
try:
frame_comfort.validate(body)
@@ -444,14 +416,25 @@ def comfort(body):
raise Failure(str(e), 400)
# Content-addressed, user-only helper bundle. Desktop and phone use the same
# on-headset state/lock; no listener, service registration or third-party app.
# The page polls this every few seconds, so run the copy already on the headset and
# send the sources only when that version isn't there yet.
files = {name: (HERE / name).read_text() for name in COMFORT_FILES}
import hashlib
files = {name: (HERE / name).read_text() for name in
("frame_comfort.py", "frame_status.py", "frame_steam.py")}
version = hashlib.sha256(json.dumps(files, sort_keys=True).encode()).hexdigest()[:16]
out = json.loads(ssh("python3 -", stdin=COMFORT_RUN % ({}, version, COMFORT_FILES, json.dumps(body)), timeout=65))
if out == COMFORT_MISSING:
out = json.loads(ssh("python3 -", stdin=COMFORT_RUN % (files, version, COMFORT_FILES, json.dumps(body)),
timeout=65))
script = """import json, os, pathlib, subprocess, sys
os.umask(0o077)
files = %r
root = pathlib.Path.home() / '.cache/frame-control/comfort' / %r
root.mkdir(parents=True, exist_ok=True)
for name, source in files.items():
path = root / name
if not path.exists():
tmp = root / (name + '.' + str(os.getpid()))
tmp.write_text(source)
tmp.replace(path)
r = subprocess.run([sys.executable, str(root / 'frame_comfort.py'), %r], capture_output=True, text=True)
print(r.stdout, end='')
""" % (files, version, json.dumps(body))
out = json.loads(ssh("python3 -", stdin=script, timeout=65))
if out.get("error") and "active" not in out:
raise Failure(out["error"], 409)
return out
@@ -1327,149 +1310,6 @@ def android(body):
raise Failure("unknown action", 400)
# ---- Import from my Quest (frame_quest.py, docs/quest-import.md) --------------
QUEST_ON_FRAME = "Import from my Quest runs on your computer, with the Quest plugged into it by USB."
QUEST_SERIAL = re.compile(r"^[A-Za-z0-9._:\[\]-]{1,128}$")
def quest_serial(value):
if value in (None, ""):
return None
if not isinstance(value, str) or not QUEST_SERIAL.match(value):
raise Failure("invalid Quest serial", 400)
return value
def quest_status(query):
"""The Quest on this computer's USB (or what to do), and the apps on it."""
if LOCAL:
return {"quest": None, "devices": [], "guidance": QUEST_ON_FRAME, "apps": []}
try:
return frame_quest.status(quest_serial((parse_qs(query).get("serial") or [None])[0]))
except frame_android.FrameError as e:
raise Failure(str(e)) from e
def quest(body):
"""{"action": "import", "package", "serial"?, "data"?}: a background job with progress."""
if LOCAL:
raise Failure(QUEST_ON_FRAME, 400)
if body.get("action") != "import":
raise Failure("unknown action", 400)
pkg = str(body.get("package", ""))
if not frame_android.PKG_RE.match(pkg):
raise Failure("invalid Android package id", 400)
serial, data, older = quest_serial(body.get("serial")), body.get("data", False), body.get("allowDowngrade", False)
if type(data) is not bool or type(older) is not bool:
raise Failure("data and allowDowngrade must be true or false", 400)
ensure_master()
return start_job(f"Import {pkg} from Quest",
lambda report: frame_quest.import_game(pkg, serial, data=data, progress=report,
allow_downgrade=older), progress=True)
# ---- My Meta library, Experimental (frame_meta.py, docs/meta-library.md) ------
#
# Never echoes the token: answers come from frame_meta.status() and _public(), and every
# error passes through frame_meta.redact().
META_APP_ID = re.compile(r"^[0-9]{1,25}$")
def _meta_failure(e):
if isinstance(e, frame_meta.SignInCancelled):
return Failure(frame_meta.redact(e, 600), 409)
status = 400 if isinstance(e, frame_meta.TokenRejected) or "token" in str(e).lower() else 502
return Failure(frame_meta.redact(e, 600), status)
def meta_sharing():
"""Whether test results reach the shared compatibility database (Privacy settings)."""
blocked = frame_telemetry.blocked()
return {"shared": bool(frame_catalog.compat_db.shared() or frame_telemetry.enabled("compat")),
"blocked": blocked or None}
def meta_status(query):
"""The connected account (never its token) and, with library=1, the Quest games it owns
with any community verdicts."""
q = parse_qs(query)
out = dict(frame_meta.status(), sharing=meta_sharing())
if out["connected"] and q.get("library") == ["1"]:
try:
apps = frame_meta.with_verdicts(frame_meta.library(refresh=q.get("refresh") == ["1"]))
out["apps"] = [frame_meta._public(a) for a in apps]
except frame_android.FrameError as e:
raise _meta_failure(e) from None
return out
def meta(body):
"""{"action": "connect", "token", "acknowledged": true, "attempt"?} | {"action": "disconnect"} |
{"action": "cancel-signin", "attempt"} |
{"action": "install", "app", "test"?, "allowDowngrade"?} | {"action": "test", "package"}."""
action = body.get("action")
try:
if action == "connect":
if body.get("acknowledged") is not True:
raise Failure("My Meta library is experimental: tick that you understand the warning first", 400)
if not isinstance(body.get("token"), str):
raise Failure("paste your Meta access token", 400)
attempt = body.get("attempt")
if attempt is not None and not (isinstance(attempt, str) and frame_meta.SIGNIN_ATTEMPT.fullmatch(attempt)):
raise Failure("invalid sign-in attempt id", 400)
return dict(frame_meta.connect(body["token"], attempt=attempt), sharing=meta_sharing(),
message="Connected your Meta account (Experimental)")
if action == "cancel-signin":
attempt = body.get("attempt")
if not (isinstance(attempt, str) and frame_meta.SIGNIN_ATTEMPT.fullmatch(attempt)):
raise Failure("invalid sign-in attempt id", 400)
return dict(frame_meta.cancel_signin(attempt), sharing=meta_sharing(),
message="Cancelled that Sign in with Meta; it won't be saved")
if action == "disconnect":
return dict(frame_meta.disconnect(), sharing=meta_sharing(),
message="Disconnected your Meta account; its token was deleted from this computer")
if action == "install":
app_id, test, older = str(body.get("app", "")), body.get("test", False), body.get("allowDowngrade", False)
if not META_APP_ID.match(app_id):
raise Failure("invalid Meta app id", 400)
if type(test) is not bool or type(older) is not bool:
raise Failure("test and allowDowngrade must be true or false", 400)
ensure_master()
def work(report):
try:
return frame_meta.install(app_id, test=test, progress=report, allow_downgrade=older)
except frame_android.FrameError as e:
raise frame_meta.MetaError(frame_meta.redact(e, 1200)) from None
except Exception as e: # never let an unexpected error's text carry the token
raise frame_meta.MetaError(frame_meta.redact(f"{type(e).__name__}: {e}", 600)) from None
return start_job(f"{'Install and test' if test else 'Install'} {app_id} from Meta", work, progress=True)
if action == "test":
pkg = str(body.get("package", ""))
if not frame_android.PKG_RE.match(pkg):
raise Failure("invalid Android package id", 400)
ensure_master()
def test():
# run_test's own errors are fixed texts; anything else could carry the game's
# log (an ssh error is its output), so it is never shown, not even redacted.
try:
return frame_meta.run_test(pkg)
except frame_meta.MetaError:
raise
except frame_android.FrameError:
raise frame_meta.MetaError(frame_meta.LOG_UNREADABLE) from None
except Exception as e:
raise frame_meta.MetaError(f"The test stopped unexpectedly ({type(e).__name__}); "
"try again.") from None
return start_job(f"Test {pkg}", test)
except frame_meta.MetaError as e:
raise _meta_failure(e) from None
raise Failure("unknown action", 400)
# Errors that are the APK's own fault, so they belong in the compatibility
# database as install_failed. Connection trouble and the like don't.
APK_FAULTS = {"android_installer", "apk_needs_newer_android", "apk_wrong_abi"}
@@ -2371,6 +2211,13 @@ def source_manage(body):
if action == 'remove':
apk_search.manage_repo('remove_repo', source_id=source_text(body, 'source'))
return {'message': 'Repository removed'}
if action == 'itch-connect': # the key is checked with itch.io, saved 0600, never echoed
key = body.get('key')
if not isinstance(key, str) or not key.strip() or len(key) > 300:
raise Failure('Paste your itch.io API key', 400)
return itch_account.connect(key)
if action == 'itch-disconnect':
return itch_account.disconnect()
raise Failure('Unknown source action', 400)
except SourceError as e:
raise Failure(str(e), 400)
@@ -2382,7 +2229,7 @@ POST = {
"/api/agent/approval": agent_approval, "/api/assistant/chat": assistant_chat,
"/api/input": remote_input, "/api/touch": remote_touch,
"/api/settings/artwork": frame_steamgriddb.save_settings,
"/api/sources": source_manage, "/api/sources/install": source_install, "/api/android/display": android_display, "/api/android": android, "/api/quest": quest, "/api/meta": meta, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard,
"/api/sources": source_manage, "/api/sources/install": source_install, "/api/android/display": android_display, "/api/android": android, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard,
"/api/flatpak": flatpak, "/api/open": open_thing, "/api/shots/save": save_shots,
"/api/webinstall/check": webinstall_check, "/api/webinstall/start": webinstall_start,
"/api/webinstall/cancel": webinstall_cancel,
@@ -2441,10 +2288,6 @@ def connection_state():
# ---- HTTP ------------------------------------------------------------------
# POST reads the page repeats on a timer, as (path, action).
POLLS = {("/api/comfort", "status")}
def action_of(body):
"""The action a request asked for, for diagnostics: a short word, never user data."""
a = body.get("action") if isinstance(body, dict) else None
@@ -2591,7 +2434,7 @@ class Handler(BaseHTTPRequestHandler):
except SourceError as e:
raise Failure(str(e), 400)
elif path == "/api/sources":
self.send_json({"sources": apk_search.sources()})
self.send_json({"sources": apk_search.sources(), "itch": itch_account.status()})
elif path == "/api/search":
self.send_json(source_search(url.query))
elif path == "/api/apk-versions":
@@ -2601,10 +2444,6 @@ class Handler(BaseHTTPRequestHandler):
apps = frame_android.list_apps()
backfill_art(apps=apps)
self.send_json({"apps": apps})
elif path == "/api/quest":
self.send_json(quest_status(url.query))
elif path == "/api/meta":
self.send_json(meta_status(url.query))
elif path == "/api/titles":
ensure_master()
titles_list = frame_titles.list_titles()
@@ -2702,10 +2541,7 @@ class Handler(BaseHTTPRequestHandler):
except ClientGone:
raise
except Failure as e:
# A poll that couldn't reach the headset is the headset asleep or away, which the page
# already shows; like GET /api/status, not an error to report.
polled = (path, action_of(body)) in POLLS and frame_telemetry.link_failed(e)
if e.status >= 500 and not polled:
if e.status >= 500:
frame_telemetry.diagnostic(f"POST {path} {action_of(body)}", e)
self.send_error_json(str(e), e.status, e.apk)
except (ValueError, TypeError) as e: