mirror of
https://github.com/saphid/frame-control.git
synced 2026-10-14 07:02:35 +02:00
Compare commits
20
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0f8aaa393b | ||
|
|
382fb0f906 | ||
|
|
76e90af5b7 | ||
|
|
5456f2f6f6 | ||
|
|
2cfb61a208 | ||
|
|
c6e4362c66 | ||
|
|
88d61cb3b6 | ||
|
|
2d63dacdf9 | ||
|
|
adefdf4398 | ||
|
|
2c3f8b3e0c | ||
|
|
5f3798b79b | ||
|
|
6e0b140efc | ||
|
|
226ec40720 | ||
|
|
f1aaed6279 | ||
|
|
b89538f506 | ||
|
|
311fb5f6c7 | ||
|
|
93ae2b1bd1 | ||
|
|
12bd10ee42 | ||
|
|
8aeeaef666 | ||
|
|
5fb02770db |
No files matched your search
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "frame-control",
|
||||
"version": "0.4.3",
|
||||
"version": "0.4.4",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "frame-control",
|
||||
"version": "0.4.3",
|
||||
"version": "0.4.4",
|
||||
"license": "MIT",
|
||||
"devDependencies": {
|
||||
"electron": "^44.4.5",
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "frame-control",
|
||||
"productName": "Frame Control",
|
||||
"version": "0.4.3",
|
||||
"version": "0.4.4",
|
||||
"description": "Desktop app for managing a Valve Steam Frame over SSH",
|
||||
"private": true,
|
||||
"main": "main.js",
|
||||
|
||||
@@ -82,6 +82,88 @@ is an error rather than a reason to fall back to `index-v1.jar`. `entry.jar`
|
||||
must be signed with SHA-2 (SHA-1 is still accepted for legacy `index-v1.jar`).
|
||||
Removing a repository clears its state.
|
||||
|
||||
## My itch.io purchases
|
||||
|
||||
The store can also install the Android (Quest) builds of games you **own** on
|
||||
itch.io: bought, claimed free, from a bundle or a gift. It is opt-in. In the
|
||||
store, open the sources dialog and use **My itch.io purchases**: create an API
|
||||
key on itch.io (account settings › API keys) and paste it there. Frame Control
|
||||
checks it with itch.io (`GET /profile`) before saving it; a wrong or revoked key
|
||||
is reported and nothing is saved. Once connected, a source called
|
||||
**itch.io (my purchases)** appears, and its games are marked **Owned on
|
||||
itch.io** instead of Free. **Disconnect itch.io** deletes the key file.
|
||||
|
||||
How it works, using the documented
|
||||
[server-side API](https://itch.io/docs/api/serverside) with
|
||||
`Authorization: Bearer <key>`:
|
||||
|
||||
- `GET https://api.itch.io/profile/owned-keys` lists your download keys, page by
|
||||
page until an empty page (500 per page, at most 40 pages). Games whose
|
||||
`traits` don't include `p_android` are left out; one entry per game, with the
|
||||
newest key.
|
||||
- Opening a game asks `GET /games/<game_id>/uploads?download_key_id=<key id>`
|
||||
for its files. Android uploads are those marked `p_android` (in `traits`, or
|
||||
as a `p_android`/`platforms.android` flag) or named `.apk`. `.apk` files come
|
||||
first, then full games before demos, newest first. Soundtracks, pre-orders and
|
||||
externally hosted files are skipped.
|
||||
- Installing asks `GET /uploads/<upload_id>/download?download_key_id=<key id>`.
|
||||
The answer accepted is a redirect to itch.io's storage host, a JSON body
|
||||
with a `url` field naming it, or the file itself. Frame Control follows the
|
||||
first two without the key, to HTTPS on port 443 only, and only to host names
|
||||
whose every address is public. For IPv4 that means outside every special
|
||||
block (`0/8`, `10/8`, `100.64/10`, `127/8`, `169.254/16`, `172.16/12`,
|
||||
`192.0.0/24`, `192.0.2/24`, `192.88.99/24`, `192.168/16`, `198.18/15`,
|
||||
`198.51.100/24`, `203.0.113/24`, `224/4`, `240/4`), however written. IPv6 is
|
||||
an allow-list: global unicast (`2000::/3`) only, minus 6to4, Teredo,
|
||||
documentation, ORCHID and benchmarking blocks, so mapped or NAT64 IPv4,
|
||||
site-local, unique-local and the rest are all refused. It connects only to
|
||||
the addresses it checked, trying each in turn within one 20-second budget
|
||||
for connecting and the TLS handshake, and checks again on each further
|
||||
redirect. A storage URL, first or after any redirect, that contains the API
|
||||
key (or 8+ characters of it, in any case, plain or percent-encoded) or an
|
||||
`api_key` parameter is refused before it is looked up, so the key never
|
||||
reaches the storage server. The APK then takes
|
||||
the normal install path, with the same ABI, OpenXR layer and Meta-services
|
||||
checks and warnings as any other store download.
|
||||
|
||||
The key is stored in `frame_host.data_dir('itch-account.json')`
|
||||
(`~/Library/Application Support/Frame Control/itch-account.json` on macOS),
|
||||
written through a new 0600 temporary file and moved into place. It is only
|
||||
sent to `https://api.itch.io`, in the `Authorization` header, never in a URL;
|
||||
Frame Control does not follow api.itch.io's redirects with it. The key, or any
|
||||
part of it from 8 characters, never appears in the page, API responses, errors
|
||||
or logs; the page clears the field as soon as it is sent. A disconnect made
|
||||
while a key is still being checked wins: that key is not saved.
|
||||
Owned-game lists are kept in memory for ten minutes and never written to the
|
||||
disk cache.
|
||||
|
||||
When api.itch.io answers 401 or 403, Frame Control asks `GET /profile` with the
|
||||
same key. If that is refused too (or itch.io says "invalid key"), the key is
|
||||
deleted and the store says to connect again; otherwise only that game or
|
||||
download is refused, and the key stays. Rate limits (429) and network errors
|
||||
only fail that source's results, with a readable message. A storage host's
|
||||
refusal is reported as a download error, not as a rate limit.
|
||||
|
||||
### What is verified and what isn't
|
||||
|
||||
Not tested against a live itch.io account at all: everything below was tested
|
||||
with recorded and hand-made responses (`tests/test_itch_purchases.py`) only.
|
||||
|
||||
| Part | Status |
|
||||
|---|---|
|
||||
| Bearer key, `GET /profile`, `/profile/owned-keys` paging and its fields, the `errors` array | Documented in the public reference (checked 2026-10-13) |
|
||||
| `GET /games/<id>/uploads` and `download_key_id` | Not in the public reference; paths and parameter from itch.io's app client, [go-itchio](https://github.com/itchio/go-itchio) |
|
||||
| Upload fields (`filename`, `display_name`, `size`, `storage`, `type`, `traits`, `updated_at`, `demo`, `preorder`) | Assumed from go-itchio and the API's snake_case convention |
|
||||
| The `p_android` trait, on games and uploads | Assumed from itch.io's `p_windows`/`p_linux`/`p_osx` naming |
|
||||
| What `/uploads/<id>/download` returns | Unknown: go-itchio only builds the URL and hands it to a downloader, so a redirect, a JSON `url` and the file are all accepted |
|
||||
| Whether that endpoint accepts the key as a Bearer header | Unknown: go-itchio puts it in the query string, which Frame Control refuses to do |
|
||||
| itch.io's storage host names and addresses | Unknown; any public HTTPS host is allowed |
|
||||
|
||||
Limits: a game whose page doesn't mark Android, or whose Android build is
|
||||
inside a `.zip`, won't install. Android builds have no version codes on
|
||||
itch.io, so only the newest upload can be installed. Owning a game doesn't make
|
||||
it run on the Frame: Quest builds that need Meta's platform services won't.
|
||||
|
||||
## Publish your own repository
|
||||
|
||||
Only publish free APKs you own or have the developer's permission to distribute.
|
||||
|
||||
@@ -60,6 +60,8 @@ does not suppress the other at the aggregator level. Queries filter the current
|
||||
feed window locally: this is not an exhaustive historical itch search. Only
|
||||
explicit zero-price Android entries are returned. Covers are exposed in
|
||||
`images`; absent screenshots, APK version, ABI and minimum SDK stay unknown.
|
||||
Games the user owns on itch.io are a separate, opt-in source that downloads
|
||||
through their own API key: see [My itch.io purchases](apk-repos.md#my-itchio-purchases).
|
||||
Curated GitHub entries include publisher artwork and plain-language summaries.
|
||||
Repository image URLs are pinned to inspected commits. Open Brush screenshots
|
||||
come from its README-linked Steam listing; SuperTux uses the upstream gameplay
|
||||
|
||||
+56
-6
@@ -104,7 +104,9 @@ connection is an SSH ControlMaster that every command shares; when it dies (the
|
||||
headset slept or left the network) the connector notices and starts again. On
|
||||
Windows, where OpenSSH can't share a connection, the same handshake runs once
|
||||
and each command then connects on its own; a command that can't reach the
|
||||
headset makes the connector start again.
|
||||
headset makes the connector start again. There ssh's stderr goes to a file that
|
||||
the connector reads as it grows, not a pipe: Windows' OpenSSH can block writing
|
||||
to a redirected pipe.
|
||||
|
||||
Once connected, every `ssh`, `scp` and `rsync` the app runs gets
|
||||
`-o HostName=<address> -o HostKeyAlias=frame-control-<id>
|
||||
@@ -121,9 +123,57 @@ lease that moved) would look like a new host there. The app keeps one known_host
|
||||
file per headset instead, so saving or forgetting one headset's key never touches
|
||||
another's: a different device answering at one of its
|
||||
addresses is refused, and the pill says so. A headset's first connection trusts
|
||||
the key it shows, as Set Up Connection does. After reinstalling SteamOS the
|
||||
headset has a new key; **Forget identity** on the Devices tab lets the next
|
||||
connection save the new one.
|
||||
the key it shows, as Set Up Connection does: while a headset has never had an
|
||||
identity here, every ssh command gets `StrictHostKeyChecking=accept-new` (the
|
||||
first contact saves it, whichever command makes it), and from then on
|
||||
`StrictHostKeyChecking=yes`. `accept-new` never replaces a saved key, so it
|
||||
can't hide a change. The saved key's fingerprint is also recorded in
|
||||
`devices.json` (`identity`), along with a flag that never clears
|
||||
(`identity_established`), as soon as ssh saves the key: even if the login is then
|
||||
refused. A headset counts as having had an identity if the flag is set, a
|
||||
fingerprint is recorded, its pin file exists at all, or a connection to it ever
|
||||
worked (older files get the flag from that when loaded, so editing or removing
|
||||
the address that worked doesn't undo it); then
|
||||
a pin file that's deleted, empty, unreadable or not parseable **fails closed**
|
||||
(*Checking the headset's identity* fails, "missing or can't be read") rather than
|
||||
trusting whatever answers, and only **Check its key…** saves a key again. It
|
||||
says when the key it reads matches the recorded fingerprint.
|
||||
|
||||
**When the key changes** (SteamOS reinstalled or reset, a new headset at the
|
||||
same address, or a different device answering there), nothing is accepted by
|
||||
itself. The connection fails at *Checking the headset's identity*; the banner
|
||||
and the headset on the Devices tab say so, with **Check the new key…** (also
|
||||
**Check its key…** on the Devices tab at any time). That reads the key the
|
||||
headset shows now, and nothing more: ssh writes it to a throwaway known_hosts
|
||||
file during the key exchange, before any login, and is stopped as soon as it
|
||||
has. Every other kind of contact is ruled out on the command line, ahead of
|
||||
anything `~/.ssh/config` says: no authentication of any kind, no agent
|
||||
(`IdentityAgent=none`, `ForwardAgent=no`), no X11 or port forwarding, no local
|
||||
or remote command (`-N`), no terminal, and `-F none`. Through a ProxyJump host
|
||||
it builds that hop itself, as a ProxyCommand whose ssh forwards nothing and runs
|
||||
no local command (ProxyJump's own ssh would take `ForwardAgent` and
|
||||
`LocalCommand` from `~/.ssh/config`); it still logs in to the jump host as
|
||||
configured. Every word of that ProxyCommand is quoted for the shell that runs it
|
||||
(and `%` doubled for ssh's own tokens), the forward is `[%h]:%p` so an IPv6
|
||||
headset works, and an IPv6 jump host is passed bare with its port. Through a ProxyCommand from `~/.ssh/config`, or several jump hosts,
|
||||
it refuses, since those can't be made safe from here: add an address it can
|
||||
reach directly, or compare the key by hand. It shows the key's SHA256 fingerprint next to
|
||||
the saved one, with the command that prints it on the headset
|
||||
(`ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub`, in Konsole on its desktop).
|
||||
**This is my headset: trust its new key** then saves exactly that key, replacing
|
||||
the old one, and reconnects. The server keeps the key it showed for 10 minutes
|
||||
and saves it only if the page sends back the same fingerprint, and only while
|
||||
the headset's port, user and that address are unchanged; it never saves whatever
|
||||
answers next. The API's old `forget-identity`, which deleted the pin so that the
|
||||
next contact saved any key unseen, is refused. Set Up Connection can't fix a changed key: it checks your
|
||||
own `known_hosts`, not the headset's file.
|
||||
|
||||
**When the headset refuses this computer's key** (`Permission denied
|
||||
(publickey,…)`), the connection stops at *Logging in*, and the banner and the
|
||||
Devices tab offer **Set Up Connection…**, which pairs this computer again.
|
||||
|
||||
Neither is reported as a fault: the diagnostics report each kind at most once
|
||||
a session ([privacy.md](privacy.md)).
|
||||
|
||||
## One server at a time
|
||||
|
||||
@@ -144,7 +194,7 @@ All under the usual `/api/` guards (loopback `Host`, `X-Frame-UI` header).
|
||||
| `GET /api/devices` | Headsets, the current network, known networks, the next free alias |
|
||||
| `GET /api/devices/tailscale?id=` | Tailscale peers, likely headsets first |
|
||||
| `GET /api/devices/mdns?id=` | Headsets found on this network |
|
||||
| `POST /api/devices` | `{"action": ...}`: `use`, `update` (name, user, port), `remove`, `address-add`, `address-update`, `address-remove`, `address-move`, `test`, `forget-identity`, `name-network`, `setup` (alias, optional host), `retry` |
|
||||
| `POST /api/devices` | `{"action": ...}`: `use`, `update` (name, user, port), `remove`, `address-add`, `address-update`, `address-remove`, `address-move`, `test`, `identity-scan` (read the key the headset shows: `scan` with `type`, `fingerprint`, `host`, `saved`, `recorded`, `matches`, `recorded_match`), `identity-trust` (`fingerprint`: save the key `identity-scan` showed), `forget-identity` (refused: use the two before), `name-network`, `setup` (alias, optional host), `retry` |
|
||||
|
||||
Every host, alias and user is checked against strict patterns before it's
|
||||
stored, because they end up in ssh arguments and `~/.ssh/config`; nothing goes
|
||||
@@ -161,7 +211,7 @@ share the format later (it still connects to one host; see
|
||||
```json
|
||||
{"version": 1, "active": "f67f8b7e",
|
||||
"devices": [{"id": "f67f8b7e", "name": "Steam Frame", "alias": "frame", "user": "steamos", "port": 22,
|
||||
"identity_files": ["~/.ssh/id_ed25519_frame"],
|
||||
"identity_files": ["~/.ssh/id_ed25519_frame"], "identity": ["SHA256:…"], "identity_established": true,
|
||||
"addresses": [{"host": "frame.local", "kind": "mdns", "label": "",
|
||||
"networks": ["n-e0998baa61"], "last_ok": 1790593550.4, "last_rtt_ms": 0.9}]}],
|
||||
"networks": {"n-e0998baa61": {"name": "Home Wi-Fi", "ssid": null, "gateway": "192.168.1.1",
|
||||
|
||||
+4
-2
@@ -101,8 +101,10 @@ scrubbed:
|
||||
- `token=`, `key=`, `password=` and similar values are replaced
|
||||
|
||||
The same error is sent at most once every 10 minutes. When ssh reports that
|
||||
it couldn't reach the Frame (asleep, away, or not set up yet), that is sent
|
||||
at most once per kind each time Frame Control runs.
|
||||
it couldn't reach the Frame (asleep, away, or not set up yet), or that ssh and
|
||||
the Frame refused each other's keys (a changed host key, or this computer's
|
||||
key not accepted), that is sent at most once per kind each time Frame Control
|
||||
runs.
|
||||
|
||||
## Report a problem
|
||||
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
"""Small APKs built in memory (binary manifest plus resource table), for test_frame_apk
|
||||
and the end-to-end tests. Deliberately doesn't import sandbox: tests/e2e runs in a
|
||||
throwaway container and keeps its real home, which sandbox would move."""
|
||||
import io
|
||||
import os
|
||||
import struct
|
||||
import sys
|
||||
import zipfile
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'ui'))
|
||||
import frame_apk # noqa: E402
|
||||
|
||||
|
||||
def pool(strings, utf8=False):
|
||||
"""A ResStringPool chunk."""
|
||||
data, offsets = b'', []
|
||||
for s in strings:
|
||||
offsets.append(len(data))
|
||||
if utf8:
|
||||
b = s.encode()
|
||||
data += bytes([len(s), len(b)]) + b + b'\0'
|
||||
else:
|
||||
data += struct.pack('<H', len(s)) + s.encode('utf-16-le') + b'\0\0'
|
||||
data += b'\0' * (-len(data) % 4)
|
||||
start = 28 + 4 * len(strings)
|
||||
body = struct.pack(f'<{len(strings)}I', *offsets) + data
|
||||
return struct.pack('<HHIIIIII', 1, 28, 28 + len(body), len(strings), 0, 0x100 if utf8 else 0, start, 0) + body
|
||||
|
||||
|
||||
def manifest(package, label_ref, version_ref, min_sdk, package_raw=True, foreign_label=False):
|
||||
"""<manifest package versionName><uses-sdk minSdkVersion/><application label icon/></manifest>.
|
||||
|
||||
package_raw=False drops the package's raw string (as some repackers do);
|
||||
foreign_label adds a non-android `label` attribute after android:label.
|
||||
"""
|
||||
strings = ['label', 'icon', 'versionName', 'minSdkVersion', 'package', 'manifest', 'uses-sdk',
|
||||
'application', package, 'junk', 'label', 'versionCode'] # the second 'label' has no android id
|
||||
resmap = struct.pack('<4I', 0x01010001, 0x01010002, 0x0101021c, 0x0101020c)
|
||||
resmap = struct.pack('<HHI', 0x0180, 8, 8 + len(resmap)) + resmap
|
||||
|
||||
def element(name, attrs):
|
||||
body = struct.pack('<IIHHHHHH', 0xffffffff, name, 20, 20, len(attrs), 0, 0, 0)
|
||||
for aname, raw, dtype, value in attrs:
|
||||
body += struct.pack('<IIIHBBI', 0xffffffff, aname, raw, 8, 0, dtype, value)
|
||||
return struct.pack('<HHIII', 0x0102, 16, 16 + len(body), 1, 0xffffffff) + body
|
||||
|
||||
none = 0xffffffff
|
||||
chunks = (pool(strings) + resmap
|
||||
+ element(5, [(4, 8 if package_raw else none, frame_apk.T_STRING, 8),
|
||||
(2, none, frame_apk.T_REF, version_ref),
|
||||
(11, none, frame_apk.T_INT_DEC, 210)])
|
||||
+ element(6, [(3, none, frame_apk.T_INT_DEC, min_sdk)])
|
||||
+ element(7, [(0, none, frame_apk.T_REF, label_ref), (1, none, frame_apk.T_REF, 0x7f020000)]
|
||||
+ ([(10, 9, frame_apk.T_STRING, 9)] if foreign_label else [])))
|
||||
return struct.pack('<HHI', 3, 8, 8 + len(chunks)) + chunks
|
||||
|
||||
|
||||
def resources(values):
|
||||
"""resources.arsc with package 0x7f; values: {(type id, entry, language, density): global string index}."""
|
||||
strings = ['French label', 'App label', '2.1', 'res/icon_lo.png', 'res/icon_hi.png', 'res/icon.xml']
|
||||
pkg_body = b''
|
||||
for (tid, lang, density), entries in values.items():
|
||||
cfg = struct.pack('<I4x2s4xH', 64, lang.encode().ljust(2, b'\0'), density).ljust(64, b'\0')
|
||||
count = max(entries) + 1
|
||||
offsets, data = [], b''
|
||||
for i in range(count):
|
||||
if i in entries:
|
||||
offsets.append(len(data))
|
||||
data += struct.pack('<HHI', 8, 0, 0) + struct.pack('<HBBI', 8, 0, frame_apk.T_STRING, entries[i])
|
||||
else:
|
||||
offsets.append(0xffffffff)
|
||||
header = 20 + 64
|
||||
estart = header + 4 * count
|
||||
body = struct.pack(f'<{count}I', *offsets) + data
|
||||
pkg_body += struct.pack('<HHIBBHII', 0x0201, header, header + len(body), tid, 0, 0, count, estart) + cfg + body
|
||||
pkg_header = struct.pack('<HHII', 0x0200, 288, 288 + len(pkg_body), 0x7f).ljust(288, b'\0')
|
||||
pkg = pkg_header + pkg_body
|
||||
table = pool(strings, utf8=True) + pkg
|
||||
return struct.pack('<HHII', 2, 12, 12 + len(table), 1) + table
|
||||
|
||||
|
||||
def apk(files):
|
||||
buf = io.BytesIO()
|
||||
with zipfile.ZipFile(buf, 'w') as z:
|
||||
for name, data in files.items():
|
||||
z.writestr(name, data)
|
||||
return buf.getvalue()
|
||||
+27
-4
@@ -10,6 +10,7 @@ import atexit
|
||||
import http.client
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
@@ -22,7 +23,22 @@ from pathlib import Path
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
sys.path[:0] = [str(ROOT / 'ui'), str(ROOT / 'tests'), str(ROOT / 'tests' / 'smoke')]
|
||||
|
||||
ENABLED = os.environ.get('FRAME_E2E') == '1'
|
||||
# Only inside the compose `host` container (FRAME_E2E_CONTAINER from compose.yaml, plus
|
||||
# the container runtime's marker file): these tests drive ui/server.py with the real
|
||||
# home, ~/.ssh and cache, which is only safe in a throwaway container.
|
||||
REQUESTED = os.environ.get('FRAME_E2E') == '1'
|
||||
IN_CONTAINER = os.environ.get('FRAME_E2E_CONTAINER') == '1' and (
|
||||
os.path.exists('/.dockerenv') or os.path.exists('/run/.containerenv'))
|
||||
ENABLED = REQUESTED and IN_CONTAINER
|
||||
if ENABLED:
|
||||
# What tests/sandbox.py does, minus moving HOME: the container's real ~/.ssh is what
|
||||
# points `ssh frame` at the fake Frame, and OpenSSH reads ~/.ssh from the password
|
||||
# database, not $HOME. (These tests used to get this through test_frame_apk.)
|
||||
_data = tempfile.mkdtemp(prefix='frame-control-e2e-')
|
||||
atexit.register(shutil.rmtree, _data, ignore_errors=True)
|
||||
os.environ['FRAME_CONTROL_DATA_DIR'] = _data
|
||||
os.environ['FRAME_CONTROL_TELEMETRY'] = '0'
|
||||
os.environ['FRAME_COMPAT_DB_URL'] = 'http://127.0.0.1:9'
|
||||
CTL = os.environ.get('FAKEFRAME_CTL', 'http://fakeframe:9999').rstrip('/')
|
||||
FAKE_HOST = os.environ.get('FAKEFRAME_HOST', 'fakeframe')
|
||||
HOME = '/home/steamos'
|
||||
@@ -30,9 +46,14 @@ SERVER_LOG = os.path.join(tempfile.gettempdir(), 'fakeframe-e2e-server.log')
|
||||
|
||||
|
||||
def require():
|
||||
"""Call at module level: skips the module unless the fake Frame is up."""
|
||||
if not ENABLED:
|
||||
raise unittest.SkipTest('needs the fake Frame: run scripts/e2e.sh (sets FRAME_E2E=1)')
|
||||
"""Skips (the module, at module level; the class, in setUpClass) unless this is
|
||||
scripts/e2e.sh's container with the fake Frame up."""
|
||||
if ENABLED:
|
||||
return
|
||||
if REQUESTED:
|
||||
raise unittest.SkipTest('FRAME_E2E=1 outside the e2e container: these tests use the real home, ~/.ssh '
|
||||
'and app cache, so they only run in scripts/e2e.sh\'s throwaway container')
|
||||
raise unittest.SkipTest('needs the fake Frame: run scripts/e2e.sh (sets FRAME_E2E=1)')
|
||||
|
||||
|
||||
# ---- the fake Frame's control port --------------------------------------------
|
||||
@@ -97,6 +118,7 @@ class Server:
|
||||
|
||||
@classmethod
|
||||
def start(cls):
|
||||
require() # never a server on this computer's real data
|
||||
if cls.proc and cls.proc.poll() is None:
|
||||
return
|
||||
with socket.socket() as s:
|
||||
@@ -193,6 +215,7 @@ class FrameTestCase(unittest.TestCase):
|
||||
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
require()
|
||||
Server.start()
|
||||
|
||||
def setUp(self):
|
||||
|
||||
@@ -6,6 +6,8 @@ import sys
|
||||
import harness
|
||||
from harness import api, ok, finished, ssh
|
||||
|
||||
harness.require()
|
||||
|
||||
sys.path.insert(0, str(harness.ROOT / 'ui'))
|
||||
import frame_mcp
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ import unittest
|
||||
|
||||
import harness
|
||||
from harness import HOME, exists, ok, state, upload, wait_for
|
||||
from test_frame_apk import apk, manifest, resources
|
||||
from apk_fixtures import apk, manifest, resources
|
||||
|
||||
harness.require()
|
||||
|
||||
|
||||
@@ -43,6 +43,7 @@ services:
|
||||
FAKEFRAME_CTL: http://fakeframe:9999
|
||||
FAKEFRAME_HOST: fakeframe
|
||||
FRAME_E2E: "1"
|
||||
FRAME_E2E_CONTAINER: "1" # tests/e2e/harness.py refuses to run anywhere else
|
||||
healthcheck:
|
||||
test: ["CMD", "test", "-f", "/home/tester/.ready"]
|
||||
interval: 1s
|
||||
|
||||
+96
-3
@@ -3,10 +3,20 @@
|
||||
each step of a connection, and plays a ControlMaster. What each HostName does comes
|
||||
from $FAKESSH_HOSTS (JSON: host -> "ok", "wrong" (a different host key), "denied" or
|
||||
"slow" (hangs after connecting);
|
||||
every call is appended to $FAKESSH_LOG as a JSON line. POSIX only."""
|
||||
every call is appended to $FAKESSH_LOG as a JSON line. POSIX only.
|
||||
|
||||
With $FAKESSH_PINS set, host keys are checked as OpenSSH does against UserKnownHostsFile under
|
||||
HostKeyAlias: "wrong" is then just a different key from the others' (a reinstalled headset),
|
||||
StrictHostKeyChecking=yes refuses a key not saved, accept-new saves a first one, and a saved key
|
||||
that differs is refused. "evil" is a device with its own key that lets anyone in without
|
||||
authentication ("none") and asks for the agent: it records in $FAKESSH_DIR what it got
|
||||
(agent-forwarded, command-ran). Like ssh, the first value of an option counts."""
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
@@ -17,12 +27,12 @@ hosts = json.loads(os.environ.get("FAKESSH_HOSTS", "{}"))
|
||||
opts = {}
|
||||
i = 0
|
||||
while i < len(args) and args[i].startswith("-"):
|
||||
if args[i] in ("-o", "-O", "-p", "-l"):
|
||||
if args[i] in ("-o", "-O", "-p", "-l", "-F", "-W"):
|
||||
key = args[i]
|
||||
val = args[i + 1]
|
||||
if key == "-o":
|
||||
k, _, v = val.partition("=")
|
||||
opts[k.lower()] = v
|
||||
opts.setdefault(k.lower(), v)
|
||||
else:
|
||||
opts[key] = val
|
||||
i += 2
|
||||
@@ -30,6 +40,12 @@ while i < len(args) and args[i].startswith("-"):
|
||||
opts[args[i]] = True
|
||||
i += 1
|
||||
alias = args[i] if i < len(args) else ""
|
||||
command = args[i + 1:]
|
||||
if opts.get("-F") != "none":
|
||||
# $FAKESSH_CONFIG: what ~/.ssh/config says (JSON, lower-case option -> value), below the
|
||||
# command line's options, as ssh has it; `-F none` reads no config.
|
||||
for k, v in json.loads(os.environ.get("FAKESSH_CONFIG", "{}")).items():
|
||||
opts.setdefault(k, v)
|
||||
host = opts.get("hostname", alias).replace("%%", "%")
|
||||
marker = os.path.join(os.environ["FAKESSH_DIR"], "master-" + host.replace("/", "_"))
|
||||
say = lambda s: (sys.stderr.write(s + "\n"), sys.stderr.flush())
|
||||
@@ -44,6 +60,23 @@ if opts.get("-O") == "exit":
|
||||
os.unlink(marker)
|
||||
sys.exit(0)
|
||||
|
||||
if "-W" in opts: # a jump host's ssh, forwarding to the headset: the call is logged, that's all
|
||||
sys.exit(0)
|
||||
if opts.get("proxycommand", "none") != "none":
|
||||
# As OpenSSH: expand %-tokens, then `$SHELL -c "exec <line>"` (here /bin/sh).
|
||||
line, out, n = opts["proxycommand"], "", 0
|
||||
while n < len(line):
|
||||
if line[n] == "%" and n + 1 < len(line):
|
||||
out += {"%": "%", "h": host, "p": opts.get("port", "22"), "r": opts.get("user", "")}.get(
|
||||
line[n + 1], line[n:n + 2])
|
||||
n += 2
|
||||
else:
|
||||
out += line[n]
|
||||
n += 1
|
||||
if subprocess.run(["/bin/sh", "-c", "exec " + out], stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.DEVNULL).returncode:
|
||||
say("kex_exchange_identification: Connection closed by remote host")
|
||||
sys.exit(255)
|
||||
what = hosts.get(host)
|
||||
if what is None:
|
||||
say(f"ssh: Could not resolve hostname {host}: nodename nor servname provided, or not known")
|
||||
@@ -53,6 +86,66 @@ say("debug1: Connection established.")
|
||||
if what == "slow":
|
||||
time.sleep(30)
|
||||
say(f"debug1: Authenticating to {host}:22 as '{opts.get('user', 'tester')}'")
|
||||
if os.environ.get("FAKESSH_PINS") and "hostkeyalias" in opts and "userknownhostsfile" in opts:
|
||||
name, path = opts["hostkeyalias"], os.path.expanduser(opts["userknownhostsfile"])
|
||||
blob = base64.b64encode(b"fake-" + {"wrong": b"other", "evil": b"evil"}.get(what, b"headset") + b"-key").decode()
|
||||
fp = base64.b64encode(hashlib.sha256(base64.b64decode(blob)).digest()).decode().rstrip("=")
|
||||
say(f"debug1: Server host key: ssh-ed25519 SHA256:{fp}")
|
||||
try:
|
||||
saved = [ln.split() for ln in open(path, errors="replace").read().splitlines() if ln.strip()]
|
||||
except OSError:
|
||||
saved = []
|
||||
saved = [f[2] for f in saved if len(f) >= 3 and name in f[0].split(",")]
|
||||
strict = opts.get("stricthostkeychecking", "ask")
|
||||
if saved and blob not in saved:
|
||||
say("@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@")
|
||||
say("@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @")
|
||||
say("Host key verification failed.")
|
||||
sys.exit(255)
|
||||
if not saved and strict != "accept-new":
|
||||
say(f"No ED25519 host key is known for {name} and you have requested strict checking.")
|
||||
say("Host key verification failed.")
|
||||
sys.exit(255)
|
||||
if not saved:
|
||||
try:
|
||||
with open(path, "a") as f:
|
||||
f.write(f"{name} ssh-ed25519 {blob}\n")
|
||||
say(f"Warning: Permanently added '{name}' (ED25519) to the list of known hosts.")
|
||||
except OSError: # ssh doesn't create the folder: it goes on, saving nothing
|
||||
say(f"Failed to add the host to the list of known hosts ({path}).")
|
||||
else:
|
||||
say(f"debug1: Host '{name}' is known and matches the ED25519 host key.")
|
||||
say("debug1: Next authentication method: publickey")
|
||||
if what == "evil":
|
||||
say(f'Authenticated to {host} ([127.0.0.1]:22) using "none".')
|
||||
got = os.path.join(os.environ["FAKESSH_DIR"], "evil-")
|
||||
if opts.get("forwardagent", "no") != "no" and opts.get("identityagent", "") != "none":
|
||||
open(got + "agent-forwarded", "w").close()
|
||||
if "-N" not in opts and (command or opts.get("remotecommand", "none") != "none"):
|
||||
open(got + "command-ran", "w").close()
|
||||
sys.exit(0)
|
||||
time.sleep(30) # -N: holds the session open
|
||||
sys.exit(0)
|
||||
if opts.get("preferredauthentications") == "none":
|
||||
say(f"tester@{host}: Permission denied (publickey,password).")
|
||||
sys.exit(255)
|
||||
if what == "denied":
|
||||
say(f"tester@{host}: Permission denied (publickey).")
|
||||
sys.exit(255)
|
||||
what = "ok"
|
||||
say(f'Authenticated to {host} ([127.0.0.1]:22) using "publickey".')
|
||||
if opts.get("controlmaster") != "yes":
|
||||
sys.exit(0)
|
||||
open(marker, "w").close()
|
||||
def bye(*_):
|
||||
if os.path.exists(marker):
|
||||
os.unlink(marker)
|
||||
sys.exit(0)
|
||||
signal.signal(signal.SIGTERM, bye)
|
||||
while True:
|
||||
time.sleep(0.2)
|
||||
if not os.path.exists(marker):
|
||||
sys.exit(0)
|
||||
say("debug1: Server host key: ssh-ed25519 SHA256:fakefakefakefakefakefakefakefakefakefakefak")
|
||||
if what == "wrong":
|
||||
say("@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@")
|
||||
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
Hand-made itch.io API responses for tests/test_itch_purchases.py; no real account or
|
||||
network was used. owned-keys-*.json follow the documented /profile/owned-keys sample
|
||||
(https://itch.io/docs/api/serverside). uploads.json follows the shape the itch.io app's
|
||||
client library (github.com/itchio/go-itchio) reads from /games/ID/uploads, with the
|
||||
`traits` array it handles; `p_android` there is assumed, not documented.
|
||||
+22
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"page": 1,
|
||||
"per_page": 500,
|
||||
"owned_keys": [
|
||||
{"id": 9001, "game_id": 341190, "purchase_id": 3923754, "downloads": 6,
|
||||
"created_at": "2026-03-10T23:46:36Z", "updated_at": "2026-03-10T23:46:36Z",
|
||||
"game": {"id": 341190, "title": "Orbit Garden VR", "url": "https://maker.itch.io/orbit-garden",
|
||||
"cover_url": "https://img.itch.zone/aW1nLzE2OTA3NTIuanBn/315x250%23c/Ihj7Wa.jpg",
|
||||
"short_text": "Grow plants in zero gravity.", "type": "default", "classification": "game",
|
||||
"min_price": 1299, "traits": ["p_windows", "p_android", "can_be_bought"],
|
||||
"created_at": "2025-12-04T16:18:04Z", "published_at": "2026-01-03T17:28:47Z",
|
||||
"user": {"id": 46531, "username": "maker", "display_name": "Maker Studio",
|
||||
"url": "https://maker.itch.io"}}},
|
||||
{"id": 9002, "game_id": 3, "purchase_id": null, "downloads": 0,
|
||||
"created_at": "2026-02-01T10:00:00Z", "updated_at": "2026-02-01T10:00:00Z",
|
||||
"game": {"id": 3, "title": "Desk Only", "url": "https://leafo.itch.io/x-moon",
|
||||
"cover_url": "https://img.itch.zone/aW1hZ2UvMy8xODM3LnBuZw==/315x250%23/y2uYQI.png",
|
||||
"type": "default", "classification": "game", "min_price": 0,
|
||||
"traits": ["p_windows", "p_linux", "p_osx"], "published_at": "2013-03-03T23:02:14Z",
|
||||
"user": {"id": 1, "username": "leafo", "display_name": "leafo", "url": "https://leafo.itch.io"}}}
|
||||
]
|
||||
}
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"page": 2,
|
||||
"per_page": 500,
|
||||
"owned_keys": [
|
||||
{"id": 8001, "game_id": 341190, "purchase_id": null, "downloads": 1,
|
||||
"created_at": "2025-06-01T10:00:00Z", "updated_at": "2025-06-01T10:00:00Z",
|
||||
"game": {"id": 341190, "title": "Orbit Garden VR", "url": "https://maker.itch.io/orbit-garden",
|
||||
"type": "default", "classification": "game", "min_price": 1299,
|
||||
"traits": ["p_windows", "p_android", "can_be_bought"],
|
||||
"user": {"id": 46531, "username": "maker", "display_name": "Maker Studio"}}},
|
||||
{"id": 8002, "game_id": 77, "purchase_id": null, "downloads": 2,
|
||||
"created_at": "2025-05-01T10:00:00Z", "updated_at": "2025-05-01T10:00:00Z",
|
||||
"game": {"id": 77, "title": "Free Quest Toy", "url": "https://toys.itch.io/quest-toy",
|
||||
"cover_url": "http://img.example/not-itch.png",
|
||||
"type": "default", "classification": "game", "min_price": 0,
|
||||
"traits": ["p_android"], "published_at": "2025-04-01T00:00:00Z",
|
||||
"user": {"id": 5, "username": "toys"}}}
|
||||
]
|
||||
}
|
||||
+28
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"uploads": [
|
||||
{"id": 501, "game_id": 341190, "filename": "OrbitGarden-win64.zip", "display_name": "Windows",
|
||||
"size": 734003200, "storage": "hosted", "type": "default", "traits": ["p_windows"],
|
||||
"created_at": "2026-04-01T10:00:00Z", "updated_at": "2026-04-02T10:00:00Z"},
|
||||
{"id": 502, "game_id": 341190, "filename": "OrbitGarden-Quest-old.apk", "display_name": "Quest 1.0",
|
||||
"size": 104857600, "storage": "hosted", "type": "default", "traits": ["p_android"],
|
||||
"created_at": "2026-01-01T10:00:00Z", "updated_at": "2026-01-05T10:00:00Z"},
|
||||
{"id": 503, "game_id": 341190, "filename": "OrbitGarden-Quest.apk", "display_name": "Quest 1.2",
|
||||
"size": 115343360, "storage": "hosted", "type": "default", "traits": ["p_android"],
|
||||
"created_at": "2026-04-01T10:00:00Z", "updated_at": "2026-04-03T10:00:00Z"},
|
||||
{"id": 504, "game_id": 341190, "filename": "OrbitGarden-demo.apk", "display_name": "Quest demo",
|
||||
"size": 52428800, "storage": "hosted", "type": "default", "traits": ["p_android", "demo"],
|
||||
"created_at": "2026-05-01T10:00:00Z", "updated_at": "2026-05-01T10:00:00Z"},
|
||||
{"id": 505, "game_id": 341190, "filename": "OrbitGarden-android-bundle.zip", "display_name": "Android (zip)",
|
||||
"size": 120000000, "storage": "hosted", "type": "default", "traits": ["p_android"],
|
||||
"created_at": "2026-06-01T10:00:00Z", "updated_at": "2026-06-01T10:00:00Z"},
|
||||
{"id": 506, "game_id": 341190, "filename": "Soundtrack.apk", "display_name": "Soundtrack",
|
||||
"size": 1000, "storage": "hosted", "type": "soundtrack", "traits": [],
|
||||
"created_at": "2026-06-01T10:00:00Z", "updated_at": "2026-06-01T10:00:00Z"},
|
||||
{"id": 507, "game_id": 341190, "filename": "mirror.apk", "display_name": "Mirror",
|
||||
"size": 1000, "storage": "external", "host": "example.com", "type": "default", "traits": ["p_android"],
|
||||
"created_at": "2026-06-01T10:00:00Z", "updated_at": "2026-06-01T10:00:00Z"},
|
||||
{"id": 508, "game_id": 341190, "filename": "OrbitGarden-mac.dmg", "display_name": "macOS",
|
||||
"size": 1000, "storage": "hosted", "type": "default", "traits": ["p_osx"],
|
||||
"created_at": "2026-06-01T10:00:00Z", "updated_at": "2026-06-01T10:00:00Z"}
|
||||
]
|
||||
}
|
||||
@@ -19,7 +19,7 @@ function grab(name) {
|
||||
fail(`unbalanced ${name}`);
|
||||
}
|
||||
const names = ["esc", "storeName", "storeHue", "storeInitial", "storeText", "storeOpenSource", "sourcePage", "storeImage",
|
||||
"storeArt", "storeIcon", "sourceMark", "currentOffer", "detailBadges", "renderAppDetail", "renderAppDetailBody",
|
||||
"storeArt", "storeIcon", "sourceMark", "storePrice", "currentOffer", "detailBadges", "renderAppDetail", "renderAppDetailBody",
|
||||
"updateInstall", "openStoreApp", "loadOfferDetails"];
|
||||
const code = names.map(grab).join("");
|
||||
|
||||
@@ -54,6 +54,17 @@ const offer = { source: "fdroid", id: "org.example.brush", source_name: "F-Droid
|
||||
if (html.includes('role="alert"')) fail("a good listing showed the error state");
|
||||
}
|
||||
|
||||
// A game the user owns on itch.io says so instead of "Free".
|
||||
{
|
||||
const p = page();
|
||||
p.sourceState.detail = { app: { name: "Orbit Garden", offers: [{ ...offer, source: "itch-owned", source_name: "itch.io (my purchases)",
|
||||
owned: true, owned_label: "Owned on itch.io", free: false }] }, choice: 0 };
|
||||
if (p.renderAppDetail() !== true) fail("an owned listing reported a render failure");
|
||||
const html = p.els.detailBody.innerHTML;
|
||||
if (!html.includes('<div class="detail-price">Owned on itch.io</div>')) fail("owned listing has no Owned on itch.io label");
|
||||
if (html.includes("No purchase needed")) fail("owned listing claims no purchase is needed");
|
||||
}
|
||||
|
||||
// A listing that can't render (no offer to show) gets a clear message, not an empty dialog.
|
||||
{
|
||||
const p = page();
|
||||
|
||||
+31
-1
@@ -1,12 +1,20 @@
|
||||
"""Imported first by every test module: nothing a test does reaches this person's
|
||||
app data, their telemetry, or the shared compatibility database.
|
||||
app data, their ~/.ssh, their telemetry, or the shared compatibility database.
|
||||
|
||||
Must run before any ui module is imported, since those read these at import time.
|
||||
The variables are set in os.environ, so servers and scripts the tests start
|
||||
inherit them too (they pass {**os.environ, ...}).
|
||||
"""
|
||||
import atexit
|
||||
import os
|
||||
import shutil
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
# The environment and real ~/.ssh from before the changes below: test_sandbox
|
||||
# reruns tests from that environment and checks ~/.ssh is left alone.
|
||||
ORIGINAL_ENV = dict(os.environ)
|
||||
REAL_SSH_DIR = Path.home() / ".ssh"
|
||||
|
||||
_dir = tempfile.mkdtemp(prefix="frame-control-tests-")
|
||||
atexit.register(shutil.rmtree, _dir, ignore_errors=True)
|
||||
@@ -14,3 +22,25 @@ os.environ["FRAME_CONTROL_DATA_DIR"] = _dir
|
||||
os.environ["FRAME_CONTROL_TELEMETRY"] = "0"
|
||||
# A maintainer's machine holds the database key; send anything that slips through nowhere.
|
||||
os.environ["FRAME_COMPAT_DB_URL"] = "http://127.0.0.1:9"
|
||||
|
||||
# A throwaway home, so Path.home() / "~" land here: ~/.ssh (config, host-key pins,
|
||||
# its lock file) and the other per-user folders some modules use. A server started
|
||||
# without FRAME_CONTROL_SSH_DIR used to import the real ~/.ssh/config and seed a pin
|
||||
# into the real ~/.ssh/frame-control-hosts on every run.
|
||||
#
|
||||
# tests/e2e must not import this (directly or through a test_* module): they run in
|
||||
# a throwaway container whose real ~/.ssh points `ssh frame` at the fake Frame, and
|
||||
# OpenSSH finds ~/.ssh from the password database, not $HOME. Shared helpers they
|
||||
# need live in sandbox-free modules such as apk_fixtures.py.
|
||||
HOME = os.path.join(_dir, "home")
|
||||
SSH_DIR = os.path.join(HOME, ".ssh")
|
||||
os.makedirs(SSH_DIR, mode=0o700)
|
||||
os.environ["HOME"] = HOME
|
||||
os.environ["USERPROFILE"] = HOME # Path.home() on Windows
|
||||
os.environ["FRAME_CONTROL_SSH_DIR"] = SSH_DIR # frame_devices.ssh_dir(), frame_connect.SSH_DIR
|
||||
# The per-user folders that don't follow HOME when these are set: frame_host.cache_dir()
|
||||
# (servers warm indexes there and sweep its staging folders) and data_dir()'s fallbacks.
|
||||
for _name, _sub in (("LOCALAPPDATA", "AppData/Local"), ("APPDATA", "AppData/Roaming"),
|
||||
("XDG_CACHE_HOME", ".cache"), ("XDG_DATA_HOME", ".local/share"),
|
||||
("XDG_STATE_HOME", ".local/state"), ("XDG_CONFIG_HOME", ".config")):
|
||||
os.environ[_name] = os.path.join(HOME, *_sub.split("/"))
|
||||
@@ -1,4 +1,5 @@
|
||||
"""MCP protocol, exact-action approvals and explicit assistant data sharing."""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import http.client
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import hashlib, io, json, os, sys, tempfile, time, unittest, urllib.error, urllib.request, zipfile
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import http.client
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
"""Fake-Frame session clock/actions plus real helper serialization and sensor probes."""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import os
|
||||
import shutil
|
||||
import json
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
"""Run the actual shared page's comfort renderer against a minimal DOM/bridge."""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import pathlib
|
||||
import shutil
|
||||
import subprocess
|
||||
|
||||
@@ -264,6 +264,30 @@ class Pins(Base):
|
||||
self.assertFalse(fd.pinned("d1"))
|
||||
self.assertFalse(fd.forget_pin("d1"))
|
||||
|
||||
def test_a_trusted_key_replaces_the_saved_one_and_fingerprints_match_ssh_keygen(self):
|
||||
key = self.dir / "hostkey"
|
||||
frame_host.run_ssh(["ssh-keygen", "-q", "-t", "ed25519", "-N", "", "-f", str(key)], capture_output=True,
|
||||
stdin=subprocess.DEVNULL, check=True, timeout=20)
|
||||
kind, blob = key.with_suffix(".pub").read_text().split()[:2]
|
||||
want = frame_host.run_ssh(["ssh-keygen", "-l", "-E", "sha256", "-f", str(key.with_suffix(".pub"))],
|
||||
capture_output=True, text=True, stdin=subprocess.DEVNULL, timeout=10).stdout.split()[1]
|
||||
self.assertEqual(fd.fingerprint(blob), want)
|
||||
self.assertIsNone(fd.fingerprint("not base64!"))
|
||||
(self.ssh / "known_hosts").write_text(f"frame.local {KEY}\n")
|
||||
fd.seed_pin("d1", ["frame.local"])
|
||||
old = fd.pinned_keys("d1")
|
||||
fd.pin_key("d1", kind, blob)
|
||||
self.assertEqual(fd.known_hosts("d1").read_text(), f"frame-control-d1 {kind} {blob}\n") # old key gone
|
||||
self.assertEqual(fd.pinned_keys("d1"), [(kind, want)])
|
||||
self.assertNotEqual(old, fd.pinned_keys("d1"))
|
||||
self.assertTrue(fd.pinned("d1"))
|
||||
for bad in (("ssh-ed25519\nHost *", blob), ("ssh-ed25519", "AAAA AAAA"), ("evil", blob)):
|
||||
with self.assertRaises(fd.DeviceError):
|
||||
fd.pin_key("d1", *bad)
|
||||
# Only plain entries for this headset's alias count.
|
||||
self.assertIsNone(fd.key_entry(f"frame-control-d2 {kind} {blob}", "frame-control-d1"))
|
||||
self.assertIsNone(fd.key_entry(f"# frame-control-d1 {kind} {blob}", "frame-control-d1"))
|
||||
|
||||
def test_each_headset_has_its_own_file(self):
|
||||
(self.ssh / "known_hosts").write_text(f"a.local {KEY}\nb.local {KEY}\n")
|
||||
fd.seed_pin("da", ["a.local"])
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
"""Offline authenticated repository fixtures; no tests contact a server."""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
"""Offline artwork, Steam API and launcher supervision regressions."""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
|
||||
+1
-76
@@ -11,82 +11,7 @@ import zipfile
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'ui'))
|
||||
import frame_apk # noqa: E402
|
||||
|
||||
|
||||
def pool(strings, utf8=False):
|
||||
"""A ResStringPool chunk."""
|
||||
data, offsets = b'', []
|
||||
for s in strings:
|
||||
offsets.append(len(data))
|
||||
if utf8:
|
||||
b = s.encode()
|
||||
data += bytes([len(s), len(b)]) + b + b'\0'
|
||||
else:
|
||||
data += struct.pack('<H', len(s)) + s.encode('utf-16-le') + b'\0\0'
|
||||
data += b'\0' * (-len(data) % 4)
|
||||
start = 28 + 4 * len(strings)
|
||||
body = struct.pack(f'<{len(strings)}I', *offsets) + data
|
||||
return struct.pack('<HHIIIIII', 1, 28, 28 + len(body), len(strings), 0, 0x100 if utf8 else 0, start, 0) + body
|
||||
|
||||
|
||||
def manifest(package, label_ref, version_ref, min_sdk, package_raw=True, foreign_label=False):
|
||||
"""<manifest package versionName><uses-sdk minSdkVersion/><application label icon/></manifest>.
|
||||
|
||||
package_raw=False drops the package's raw string (as some repackers do);
|
||||
foreign_label adds a non-android `label` attribute after android:label.
|
||||
"""
|
||||
strings = ['label', 'icon', 'versionName', 'minSdkVersion', 'package', 'manifest', 'uses-sdk',
|
||||
'application', package, 'junk', 'label', 'versionCode'] # the second 'label' has no android id
|
||||
resmap = struct.pack('<4I', 0x01010001, 0x01010002, 0x0101021c, 0x0101020c)
|
||||
resmap = struct.pack('<HHI', 0x0180, 8, 8 + len(resmap)) + resmap
|
||||
|
||||
def element(name, attrs):
|
||||
body = struct.pack('<IIHHHHHH', 0xffffffff, name, 20, 20, len(attrs), 0, 0, 0)
|
||||
for aname, raw, dtype, value in attrs:
|
||||
body += struct.pack('<IIIHBBI', 0xffffffff, aname, raw, 8, 0, dtype, value)
|
||||
return struct.pack('<HHIII', 0x0102, 16, 16 + len(body), 1, 0xffffffff) + body
|
||||
|
||||
none = 0xffffffff
|
||||
chunks = (pool(strings) + resmap
|
||||
+ element(5, [(4, 8 if package_raw else none, frame_apk.T_STRING, 8),
|
||||
(2, none, frame_apk.T_REF, version_ref),
|
||||
(11, none, frame_apk.T_INT_DEC, 210)])
|
||||
+ element(6, [(3, none, frame_apk.T_INT_DEC, min_sdk)])
|
||||
+ element(7, [(0, none, frame_apk.T_REF, label_ref), (1, none, frame_apk.T_REF, 0x7f020000)]
|
||||
+ ([(10, 9, frame_apk.T_STRING, 9)] if foreign_label else [])))
|
||||
return struct.pack('<HHI', 3, 8, 8 + len(chunks)) + chunks
|
||||
|
||||
|
||||
def resources(values):
|
||||
"""resources.arsc with package 0x7f; values: {(type id, entry, language, density): global string index}."""
|
||||
strings = ['French label', 'App label', '2.1', 'res/icon_lo.png', 'res/icon_hi.png', 'res/icon.xml']
|
||||
pkg_body = b''
|
||||
for (tid, lang, density), entries in values.items():
|
||||
cfg = struct.pack('<I4x2s4xH', 64, lang.encode().ljust(2, b'\0'), density).ljust(64, b'\0')
|
||||
count = max(entries) + 1
|
||||
offsets, data = [], b''
|
||||
for i in range(count):
|
||||
if i in entries:
|
||||
offsets.append(len(data))
|
||||
data += struct.pack('<HHI', 8, 0, 0) + struct.pack('<HBBI', 8, 0, frame_apk.T_STRING, entries[i])
|
||||
else:
|
||||
offsets.append(0xffffffff)
|
||||
header = 20 + 64
|
||||
estart = header + 4 * count
|
||||
body = struct.pack(f'<{count}I', *offsets) + data
|
||||
pkg_body += struct.pack('<HHIBBHII', 0x0201, header, header + len(body), tid, 0, 0, count, estart) + cfg + body
|
||||
pkg_header = struct.pack('<HHII', 0x0200, 288, 288 + len(pkg_body), 0x7f).ljust(288, b'\0')
|
||||
pkg = pkg_header + pkg_body
|
||||
table = pool(strings, utf8=True) + pkg
|
||||
return struct.pack('<HHII', 2, 12, 12 + len(table), 1) + table
|
||||
|
||||
|
||||
def apk(files):
|
||||
buf = io.BytesIO()
|
||||
with zipfile.ZipFile(buf, 'w') as z:
|
||||
for name, data in files.items():
|
||||
z.writestr(name, data)
|
||||
return buf.getvalue()
|
||||
from apk_fixtures import apk, manifest, pool, resources # noqa: E402,F401
|
||||
|
||||
|
||||
class ApkInfo(unittest.TestCase):
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
"""Local-only VR manifest, raw ZIP and independent signature checks."""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import io
|
||||
import os
|
||||
from pathlib import Path
|
||||
@@ -15,7 +16,7 @@ import frame_apk
|
||||
import frame_apk_vr as vr
|
||||
import frame_apk_sign as signing
|
||||
import frame_android
|
||||
from test_frame_apk import pool
|
||||
from apk_fixtures import pool
|
||||
|
||||
|
||||
DEFAULT = 'android.intent.category.DEFAULT'
|
||||
|
||||
@@ -6,6 +6,7 @@ identity line, then starts TLS as the *client*.
|
||||
|
||||
Run: python3 -m unittest discover -s tests
|
||||
"""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
|
||||
@@ -0,0 +1,862 @@
|
||||
"""My itch.io purchases: the account key, owned games and their Android downloads.
|
||||
|
||||
Offline: a fake HTTPS layer under urllib's real opener (so redirects and header
|
||||
handling are the real ones) serves the recorded JSON in fixtures/itch_account.
|
||||
"""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import contextlib, email.message, http.client, io, ipaddress, json, os, socket, sys, tempfile, threading, time
|
||||
import unittest
|
||||
import urllib.error, urllib.parse, urllib.request, zipfile
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
|
||||
from apk_sources import SourceError, SourceLimited, _itch_account as account, _web, itch, search
|
||||
import server
|
||||
|
||||
FIX = Path(__file__).parent / 'fixtures' / 'itch_account'
|
||||
KEY = 'TESTKEYabcdefghijklmnopqrstuvwxyz0123456789'
|
||||
OTHER = 'OTHERKEY0123456789abcdefghijklmnopqrstuv'
|
||||
|
||||
|
||||
def apk_bytes():
|
||||
out = io.BytesIO()
|
||||
with zipfile.ZipFile(out, 'w') as z:
|
||||
z.writestr('AndroidManifest.xml', b'\x03\x00\x08\x00')
|
||||
z.writestr('classes.dex', b'dex\n035\x00')
|
||||
return out.getvalue()
|
||||
|
||||
|
||||
class Reply(io.BytesIO):
|
||||
"""Just enough of http.client.HTTPResponse for urllib's handlers."""
|
||||
|
||||
def __init__(self, url, code=200, body=b'', headers=None):
|
||||
super().__init__(body if isinstance(body, bytes) else json.dumps(body).encode())
|
||||
self.url, self.code, self.status, self.msg, self.reason = url, code, code, 'fake', 'fake'
|
||||
self.headers = email.message.Message()
|
||||
for k, v in (headers or {}).items():
|
||||
self.headers[k] = v
|
||||
|
||||
def info(self):
|
||||
return self.headers
|
||||
|
||||
def geturl(self):
|
||||
return self.url
|
||||
|
||||
def getcode(self):
|
||||
return self.code
|
||||
|
||||
def getheader(self, name, default=None):
|
||||
return self.headers.get(name, default)
|
||||
|
||||
|
||||
class FakeConnection:
|
||||
"""_web._connect's http.client connection, answered from the same routes."""
|
||||
|
||||
def __init__(self, web, host):
|
||||
self.web, self.host, self.reply = web, host, None
|
||||
|
||||
def request(self, method, path, headers=None):
|
||||
req = urllib.request.Request('https://' + self.host + path, headers=headers or {})
|
||||
self.reply = self.web(None, req)
|
||||
|
||||
def getresponse(self):
|
||||
return self.reply
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
|
||||
# What the fake resolver answers: the storage host is public; the rest are traps.
|
||||
ADDRESSES = {'storage.itch-cdn.example': '151.101.1.1', 'other-cdn.example': '151.101.2.2',
|
||||
'localhost.': '127.0.0.1', 'frame.local': '192.168.1.20', '127.0.0x0.1': '127.0.0.1',
|
||||
'intranet.example': '10.0.0.5', 'mapped.example': '::ffff:127.0.0.1', 'cgnat.example': '100.64.0.1',
|
||||
'mixed.example': ('151.101.1.1', '10.0.0.5'), 'dual.example': ('2a04:4e42::81', '151.101.1.1'),
|
||||
'nat64.example': '64:ff9b:1::a00:1', 'sitelocal.example': 'fec0::1',
|
||||
'many.example': tuple('151.101.%d.1' % n for n in range(1, 9))}
|
||||
REAL_CONNECT = _web._connect
|
||||
|
||||
|
||||
class FakeItch:
|
||||
"""Routes https requests by host and path; records every request with its headers."""
|
||||
|
||||
def __init__(self):
|
||||
self.calls, self.routes = [], {}
|
||||
|
||||
def route(self, host_path, reply):
|
||||
self.routes[host_path] = reply
|
||||
|
||||
def __call__(self, handler, req):
|
||||
url = urllib.parse.urlsplit(req.full_url)
|
||||
headers = {k.lower(): v for k, v in req.header_items()}
|
||||
self.calls.append((req.full_url, headers))
|
||||
reply = self.routes.get(url.hostname + url.path)
|
||||
if reply is None:
|
||||
return Reply(req.full_url, 404, {'errors': ['not found']})
|
||||
if callable(reply):
|
||||
reply = reply(req, urllib.parse.parse_qs(url.query))
|
||||
if isinstance(reply, Exception):
|
||||
raise reply
|
||||
if isinstance(reply, Reply):
|
||||
return reply
|
||||
return Reply(req.full_url, 200, reply)
|
||||
|
||||
def urls(self, host_path):
|
||||
return [u for u, _ in self.calls if urllib.parse.urlsplit(u).hostname + urllib.parse.urlsplit(u).path == host_path]
|
||||
|
||||
|
||||
def owned_pages(req, query):
|
||||
page = query['page'][0]
|
||||
return {'1': json.loads((FIX / 'owned-keys-1.json').read_text()),
|
||||
'2': json.loads((FIX / 'owned-keys-2.json').read_text())}.get(page, {'page': int(page), 'owned_keys': []})
|
||||
|
||||
|
||||
class ItchPurchases(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmp = tempfile.TemporaryDirectory()
|
||||
self.addCleanup(self.tmp.cleanup)
|
||||
root = Path(self.tmp.name)
|
||||
for target, value in [(account.frame_host, 'data_dir'), (search.frame_host, 'data_dir')]:
|
||||
p = patch.object(target, value, lambda *parts: root.joinpath('data', *parts))
|
||||
p.start()
|
||||
self.addCleanup(p.stop)
|
||||
(root / 'cache').mkdir()
|
||||
for p in (patch.object(_web, 'cache', return_value=str(root / 'cache')),
|
||||
patch.object(_web.frame_host, 'cache_dir', lambda *parts: root.joinpath('cache', *parts))):
|
||||
p.start()
|
||||
self.addCleanup(p.stop)
|
||||
self.web = FakeItch()
|
||||
self.web.route('api.itch.io/profile', {'user': {'username': 'player', 'id': 1}})
|
||||
self.web.route('api.itch.io/profile/owned-keys', owned_pages)
|
||||
self.web.route('api.itch.io/games/341190/uploads', json.loads((FIX / 'uploads.json').read_text()))
|
||||
p = patch('urllib.request.HTTPSHandler.https_open', autospec=True, side_effect=self.web)
|
||||
p.start()
|
||||
self.addCleanup(p.stop)
|
||||
p = patch('urllib.request.HTTPHandler.http_open', side_effect=AssertionError('plain HTTP in test'))
|
||||
p.start()
|
||||
self.addCleanup(p.stop)
|
||||
real_lookup = _web.socket.getaddrinfo
|
||||
|
||||
def lookup(host, port, *args, **kwargs):
|
||||
if host not in ADDRESSES:
|
||||
try:
|
||||
ipaddress.ip_address(host) # IP literals (and the in-process test server) need no DNS
|
||||
return real_lookup(host, port, *args, **kwargs)
|
||||
except ValueError:
|
||||
pass
|
||||
if host == 'localhost':
|
||||
return real_lookup(host, port, *args, **kwargs)
|
||||
raise _web.socket.gaierror('unknown test host ' + host)
|
||||
found = ADDRESSES[host]
|
||||
return [(10 if ':' in a else 2, 1, 6, '', (a, port)) for a in ([found] if isinstance(found, str) else found)]
|
||||
self.connected, self.unreachable = [], set()
|
||||
|
||||
def connect(host, address, timeout, deadline):
|
||||
self.connected.append((host, address))
|
||||
if address[0] in self.unreachable:
|
||||
raise OSError('Network is unreachable')
|
||||
self.assertLessEqual(deadline - time.monotonic(), _web.CONNECT_BUDGET)
|
||||
return FakeConnection(self.web, host)
|
||||
for p in (patch.object(_web.socket, 'getaddrinfo', lookup), patch.object(_web, '_connect', connect)):
|
||||
p.start()
|
||||
self.addCleanup(p.stop)
|
||||
account._forget()
|
||||
self.addCleanup(account._forget)
|
||||
_web._limited.clear()
|
||||
self.addCleanup(_web._limited.clear)
|
||||
search._status.clear()
|
||||
|
||||
def owned_source(self):
|
||||
return next(s for s in itch.sources() if s['id'] == itch.OWNED)
|
||||
|
||||
def connect(self):
|
||||
self.assertTrue(account.connect(KEY)['itch']['connected'])
|
||||
|
||||
# -- the key ---------------------------------------------------------------
|
||||
|
||||
def test_connect_checks_the_key_then_saves_it_0600(self):
|
||||
result = account.connect(' ' + KEY + '\n')
|
||||
self.assertEqual(result['message'], 'Connected to itch.io as player')
|
||||
self.assertEqual(result['itch'], {'connected': True, 'username': 'player'})
|
||||
self.assertNotIn(KEY, json.dumps(result))
|
||||
url, headers = self.web.calls[0]
|
||||
self.assertEqual(url, 'https://api.itch.io/profile')
|
||||
self.assertEqual(headers['authorization'], 'Bearer ' + KEY)
|
||||
self.assertEqual(json.loads(account.path().read_text())['key'], KEY)
|
||||
if os.name == 'posix':
|
||||
self.assertEqual(os.stat(account.path()).st_mode & 0o777, 0o600)
|
||||
self.assertEqual([p.name for p in account.path().parent.iterdir()], ['itch-account.json']) # no .tmp left
|
||||
self.assertNotIn(KEY, json.dumps(account.status()))
|
||||
|
||||
def test_reconnecting_rewrites_the_file_0600_even_if_it_was_wider(self):
|
||||
self.connect()
|
||||
os.chmod(account.path(), 0o644)
|
||||
account.connect(OTHER)
|
||||
self.assertEqual(json.loads(account.path().read_text())['key'], OTHER)
|
||||
if os.name == 'posix':
|
||||
self.assertEqual(os.stat(account.path()).st_mode & 0o777, 0o600)
|
||||
|
||||
def test_bad_or_revoked_key_stays_disconnected(self):
|
||||
for reply in (Reply('https://api.itch.io/profile', 401, {'errors': ['invalid key']}),
|
||||
Reply('https://api.itch.io/profile', 403, b'Forbidden'),
|
||||
{'errors': ['invalid key']}):
|
||||
self.web.route('api.itch.io/profile', reply)
|
||||
with self.assertRaisesRegex(account.BadKey, 'did not accept the API key') as caught:
|
||||
account.connect(KEY)
|
||||
self.assertNotIn(KEY, str(caught.exception))
|
||||
self.assertFalse(account.path().exists())
|
||||
self.assertEqual(account.status(), {'connected': False, 'username': None})
|
||||
self.assertEqual([s['id'] for s in itch.sources()], ['itch', 'itch-quest'])
|
||||
with self.assertRaisesRegex(account.BadKey, 'does not look like'):
|
||||
account.connect('short')
|
||||
with self.assertRaisesRegex(account.BadKey, 'does not look like'):
|
||||
account.connect(KEY + '\r\nX-Injected: 1')
|
||||
self.assertEqual(len(self.web.calls), 3) # malformed keys never reach itch.io
|
||||
|
||||
def test_a_refused_request_with_a_working_key_keeps_the_key(self):
|
||||
self.connect()
|
||||
self.web.route('api.itch.io/games/341190/uploads', Reply('https://api.itch.io/games/341190/uploads', 403, b''))
|
||||
with self.assertRaisesRegex(SourceError, r'refused this request \(HTTP 403\)') as caught:
|
||||
itch.details(self.owned_source(), '341190')
|
||||
self.assertNotIsInstance(caught.exception, account.BadKey)
|
||||
self.assertTrue(account.connected())
|
||||
|
||||
def test_a_key_revoked_later_is_removed_with_a_clear_message(self):
|
||||
self.connect()
|
||||
self.web.route('api.itch.io/profile/owned-keys', Reply('https://api.itch.io/profile/owned-keys', 401, b''))
|
||||
self.web.route('api.itch.io/profile', Reply('https://api.itch.io/profile', 401, b''))
|
||||
with self.assertRaisesRegex(account.BadKey, 'no longer accepts the saved API key'):
|
||||
itch.search(self.owned_source(), '')
|
||||
self.assertFalse(account.path().exists())
|
||||
self.assertNotIn(itch.OWNED, [s['id'] for s in itch.sources()])
|
||||
|
||||
def test_disconnect_deletes_the_key_file(self):
|
||||
self.connect()
|
||||
self.assertIn(itch.OWNED, [s['id'] for s in itch.sources()])
|
||||
result = account.disconnect()
|
||||
self.assertFalse(account.path().exists())
|
||||
self.assertEqual(result['itch'], {'connected': False, 'username': None})
|
||||
self.assertNotIn(itch.OWNED, [s['id'] for s in itch.sources()])
|
||||
with self.assertRaisesRegex(SourceError, 'Connect your itch.io account first'):
|
||||
account.owned_keys()
|
||||
account.disconnect() # already gone: fine
|
||||
|
||||
# -- owned games -------------------------------------------------------------
|
||||
|
||||
def test_owned_keys_follow_pages_until_an_empty_one(self):
|
||||
self.connect()
|
||||
keys = account.owned_keys()
|
||||
self.assertEqual([k['id'] for k in keys], [9001, 9002, 8001, 8002])
|
||||
pages = self.web.urls('api.itch.io/profile/owned-keys')
|
||||
self.assertEqual([urllib.parse.parse_qs(urllib.parse.urlsplit(u).query) for u in pages],
|
||||
[{'page': [str(n)], 'per_page': ['500']} for n in (1, 2, 3)])
|
||||
self.assertTrue(all('Bearer ' + KEY == h['authorization'] for _, h in self.web.calls))
|
||||
self.assertTrue(all(KEY not in u for u, _ in self.web.calls))
|
||||
account.owned_keys() # cached in memory, not on disk
|
||||
self.assertEqual(len(self.web.urls('api.itch.io/profile/owned-keys')), 3)
|
||||
self.assertFalse(any(KEY in p.read_text(errors='ignore') or b'Orbit' in p.read_bytes()
|
||||
for p in Path(_web.cache()).iterdir()))
|
||||
|
||||
def test_page_cap_stops_a_server_that_never_ends(self):
|
||||
self.connect()
|
||||
self.web.route('api.itch.io/profile/owned-keys', lambda req, q: {'owned_keys': [{'id': int(q['page'][0])}]})
|
||||
with patch.object(account, 'PAGES', 3):
|
||||
self.assertEqual(len(account.owned_keys()), 3)
|
||||
|
||||
def test_search_lists_owned_android_games_labelled(self):
|
||||
self.connect()
|
||||
entries = itch.search(self.owned_source(), '')
|
||||
self.assertEqual([(e['id'], e['name']) for e in entries], [('341190', 'Orbit Garden VR'), ('77', 'Free Quest Toy')])
|
||||
orbit, toy = entries
|
||||
self.assertEqual(orbit['download_key_id'], 9001) # the newest key for a game owned twice
|
||||
self.assertTrue(orbit['owned'])
|
||||
self.assertEqual(orbit['owned_label'], 'Owned on itch.io')
|
||||
self.assertFalse(orbit['free'])
|
||||
self.assertTrue(toy['free'])
|
||||
self.assertIsNone(toy['icon']) # not an itch.io image host
|
||||
self.assertEqual(orbit['page'], 'https://maker.itch.io/orbit-garden')
|
||||
self.assertEqual(itch.search(self.owned_source(), 'zero gravity')[0]['id'], '341190')
|
||||
# Through the store's search, owned (paid) games pass the free-only filter.
|
||||
result = search.search('orbit', source=itch.OWNED)
|
||||
self.assertEqual(result['sources'][0]['status'], 'ok')
|
||||
offer = result['apps'][0]['offers'][0]
|
||||
self.assertEqual((offer['source'], offer['source_name'], offer['owned_label']),
|
||||
(itch.OWNED, 'itch.io (my purchases)', 'Owned on itch.io'))
|
||||
self.assertNotIn(KEY, json.dumps(result))
|
||||
|
||||
def test_owned_games_show_under_either_vr_choice(self):
|
||||
# itch.io doesn't say which purchases are VR, so neither filter hides them; other
|
||||
# games of unknown kind still count as flat.
|
||||
self.connect()
|
||||
both = ['Free Quest Toy', 'Orbit Garden VR']
|
||||
for vr in (True, False):
|
||||
names = sorted(a['name'] for a in search.search('', vr=vr, source=itch.OWNED)['apps'])
|
||||
self.assertEqual(names, both)
|
||||
flat = {'source': 'x', 'id': '1', 'name': 'Flat', 'summary': '', 'vr': None}
|
||||
self.assertEqual(search.group([flat], vr=True), [])
|
||||
self.assertEqual(len(search.group([flat], vr=False)), 1)
|
||||
self.assertEqual(search.group([dict(flat, owned=True, vr=False)], vr=True), [])
|
||||
|
||||
def test_only_the_account_source_may_mark_paid_games_owned(self):
|
||||
entry = {'free': False, 'owned': True}
|
||||
self.assertFalse(search.obtainable({'id': 'fdroid'}, entry))
|
||||
self.assertTrue(search.obtainable({'id': itch.OWNED, 'owned': True}, entry))
|
||||
self.assertTrue(search.obtainable({'id': 'fdroid'}, {'free': True}))
|
||||
|
||||
def test_android_upload_selection(self):
|
||||
uploads = json.loads((FIX / 'uploads.json').read_text())['uploads']
|
||||
# .apk files before other Android files, full games before demos, newest first; never
|
||||
# soundtracks, externally hosted files or other platforms.
|
||||
self.assertEqual([u['id'] for u in account.android_uploads(uploads)], [503, 502, 504, 505])
|
||||
self.assertTrue(account.is_android({'filename': 'Game.APK'}))
|
||||
self.assertTrue(account.is_android({'filename': 'game.zip', 'p_android': True}))
|
||||
self.assertTrue(account.is_android({'filename': 'game.zip', 'platforms': {'android': 'all'}}))
|
||||
self.assertFalse(account.is_android({'filename': 'game.zip', 'traits': ['p_windows']}))
|
||||
self.assertEqual(account.android_uploads({'not': 'a list'}), [])
|
||||
|
||||
def test_details_offer_the_android_uploads(self):
|
||||
self.connect()
|
||||
entry = itch.details(self.owned_source(), '341190')
|
||||
self.assertTrue(entry['downloadable'])
|
||||
self.assertEqual([v['upload_id'] for v in entry['versions']], [503, 502, 504, 505])
|
||||
self.assertEqual((entry['version'], entry['size'], entry['updated']), ('Quest 1.2', 115343360, '2026-04-03'))
|
||||
self.assertTrue(entry['versions'][2]['demo'])
|
||||
url = self.web.urls('api.itch.io/games/341190/uploads')[0]
|
||||
self.assertEqual(url, 'https://api.itch.io/games/341190/uploads?download_key_id=9001')
|
||||
with self.assertRaisesRegex(SourceError, 'not among'):
|
||||
itch.details(self.owned_source(), '3') # owned, but not for Android
|
||||
self.web.route('api.itch.io/games/77/uploads', {'uploads': []})
|
||||
self.assertFalse(itch.details(self.owned_source(), '77')['downloadable'])
|
||||
with self.assertRaisesRegex(SourceError, 'no Android download'):
|
||||
itch.download(self.owned_source(), '77')
|
||||
|
||||
def test_download_url_building(self):
|
||||
self.assertEqual(account.download_url(503, 9001),
|
||||
'https://api.itch.io/uploads/503/download?download_key_id=9001')
|
||||
for bad in [(0, 1), (1, None), ('5', 1), (True, 1), (-2, 1)]:
|
||||
with self.assertRaises(SourceError):
|
||||
account.download_url(*bad)
|
||||
|
||||
# -- downloads -----------------------------------------------------------------
|
||||
|
||||
def storage(self, location='https://storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk?sig=abc'):
|
||||
self.web.route('api.itch.io/uploads/503/download',
|
||||
Reply('https://api.itch.io/uploads/503/download', 302, b'', {'Location': location}))
|
||||
self.web.route('storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk', Reply(location, 200, apk_bytes()))
|
||||
|
||||
def test_download_sends_the_key_to_itch_only_and_strips_it_on_redirect(self):
|
||||
self.connect()
|
||||
self.storage()
|
||||
result = itch.download(self.owned_source(), '341190')
|
||||
self.assertTrue(os.path.isfile(result['apk']))
|
||||
self.assertFalse(result['verified'])
|
||||
api, cdn = self.web.calls[-2:]
|
||||
self.assertEqual(api[0], 'https://api.itch.io/uploads/503/download?download_key_id=9001')
|
||||
self.assertEqual(api[1]['authorization'], 'Bearer ' + KEY)
|
||||
self.assertTrue(cdn[0].startswith('https://storage.itch-cdn.example/'))
|
||||
self.assertNotIn('authorization', cdn[1])
|
||||
# Connected to the address that was checked, not to a fresh lookup.
|
||||
self.assertEqual(self.connected, [('storage.itch-cdn.example', ('151.101.1.1', 443))])
|
||||
self.assertTrue(all(KEY not in u for u, _ in self.web.calls))
|
||||
with self.assertRaisesRegex(SourceError, 'version codes'):
|
||||
itch.download(self.owned_source(), '341190', version_code=3)
|
||||
|
||||
def test_download_also_accepts_a_json_url_or_the_file_itself(self):
|
||||
self.connect()
|
||||
self.storage()
|
||||
location = 'https://storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk?sig=abc'
|
||||
self.web.route('api.itch.io/uploads/503/download',
|
||||
Reply('https://api.itch.io/uploads/503/download', 200, {'url': location},
|
||||
{'Content-Type': 'application/json'}))
|
||||
self.assertTrue(os.path.isfile(itch.download(self.owned_source(), '341190')['apk']))
|
||||
self.assertNotIn('authorization', self.web.calls[-1][1])
|
||||
self.web.route('api.itch.io/uploads/503/download',
|
||||
Reply('https://api.itch.io/uploads/503/download', 200, apk_bytes(),
|
||||
{'Content-Type': 'application/octet-stream'}))
|
||||
self.connected.clear()
|
||||
self.assertTrue(os.path.isfile(itch.download(self.owned_source(), '341190')['apk']))
|
||||
self.assertEqual(self.connected, []) # api.itch.io served it; no storage host involved
|
||||
self.web.route('api.itch.io/uploads/503/download',
|
||||
Reply('https://api.itch.io/uploads/503/download', 200, {'nothing': 1},
|
||||
{'Content-Type': 'application/json'}))
|
||||
with self.assertRaisesRegex(SourceError, 'did not say where'):
|
||||
itch.download(self.owned_source(), '341190')
|
||||
|
||||
def test_storage_redirects_are_checked_on_every_hop(self):
|
||||
self.connect()
|
||||
self.storage('https://other-cdn.example/hop')
|
||||
self.web.route('other-cdn.example/hop', Reply('https://other-cdn.example/hop', 302, b'',
|
||||
{'Location': 'https://storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk'}))
|
||||
self.assertTrue(os.path.isfile(itch.download(self.owned_source(), '341190')['apk']))
|
||||
self.assertEqual([h for h, _ in self.connected], ['other-cdn.example', 'storage.itch-cdn.example'])
|
||||
self.web.route('other-cdn.example/hop', Reply('https://other-cdn.example/hop', 302, b'',
|
||||
{'Location': 'https://intranet.example/a.apk'}))
|
||||
with self.assertRaisesRegex(SourceError, 'private network address'):
|
||||
itch.download(self.owned_source(), '341190')
|
||||
|
||||
def test_redirects_to_plain_http_or_private_addresses_are_refused(self):
|
||||
self.connect()
|
||||
for location in ('http://storage.itch-cdn.example/a.apk', 'https://user:pw@cdn.example/a.apk',
|
||||
'https://cdn.example:8443/a.apk'):
|
||||
self.storage(location)
|
||||
with self.assertRaisesRegex(SourceError, 'unexpected download URL'):
|
||||
itch.download(self.owned_source(), '341190')
|
||||
for location in ('https://localhost./a.apk', 'https://frame.local/a.apk', 'https://127.0.0x0.1/a.apk',
|
||||
'https://intranet.example/a.apk', 'https://mapped.example/a.apk',
|
||||
'https://cgnat.example/a.apk', 'https://mixed.example/a.apk',
|
||||
'https://127.0.0.1/a.apk', 'https://[::1]/a.apk', 'https://169.254.169.254/a.apk',
|
||||
'https://nat64.example/a.apk', 'https://[4000::1]/a.apk', 'https://[64:ff9b:1::a00:1]/a.apk',
|
||||
'https://[fec0::1]/a.apk', 'https://sitelocal.example/a.apk'):
|
||||
self.storage(location)
|
||||
with self.assertRaisesRegex(SourceError, 'private network address'):
|
||||
itch.download(self.owned_source(), '341190')
|
||||
self.assertEqual(self.connected, [])
|
||||
self.assertEqual({urllib.parse.urlsplit(u).hostname for u, _ in self.web.calls}, {'api.itch.io'})
|
||||
|
||||
def test_public_address_rules(self):
|
||||
for ip in ('151.101.1.1', '8.8.8.8', '2a04:4e42::81', '2001:4860:4860::8888'):
|
||||
self.assertTrue(_web.public_address(ip), ip)
|
||||
for ip in ('127.0.0.1', '10.1.2.3', '172.16.0.1', '192.168.0.1', '169.254.169.254', '100.64.0.1',
|
||||
'0.0.0.0', '224.0.0.1', '255.255.255.255', '240.0.0.1', '::1', '::', 'fe80::1%en0', 'fc00::1',
|
||||
'ff02::1', 'not an ip',
|
||||
'192.0.0.8', '192.0.0.11', '192.0.0.255', '192.0.0.170', '192.0.0.9', '192.0.0.1', # 3.9: global
|
||||
'192.0.2.1', '192.88.99.1', '198.18.0.1', '198.19.255.255', '198.51.100.7', '203.0.113.9',
|
||||
'0.1.2.3', '10.255.255.255', '172.31.0.1', '239.1.1.1', '254.0.0.1',
|
||||
'4000::1', '100::1', 'f000::1', # reserved: Python 3.9's is_global says True for 4000::1
|
||||
'::ffff:10.0.0.1', '::ffff:127.0.0.1', '::ffff:8.8.8.8', # IPv4-mapped
|
||||
'::a00:1', '::8.8.8.8', # IPv4-compatible
|
||||
'64:ff9b::a00:1', '64:ff9b::808:808', # NAT64 well-known prefix
|
||||
'64:ff9b:1::a00:1', '64:ff9b:1::808:808', # NAT64 local-use: 3.9's is_global says True
|
||||
'2002:a00:1::1', '2002:808:808::1', # 6to4
|
||||
'2001:0:4136:e378:8000:63bf:3fff:fdd2', # Teredo
|
||||
'fec0::1', 'feff::1', # site-local: Python 3.9's is_global says True
|
||||
'fc00::1', 'fd12:3456::1', 'fe80::1', 'ff00::1', 'ff0e::1', '8000::1', 'c000::1',
|
||||
'2001:db8::1', '3fff::1', # documentation
|
||||
'2001:10::1', '2001:20::1', '2001:2::1'): # ORCHID, ORCHIDv2, benchmarking
|
||||
self.assertFalse(_web.public_address(ip), ip)
|
||||
self.assertEqual(_web.resolve_public('storage.itch-cdn.example'), [('151.101.1.1', 443)])
|
||||
self.assertEqual(_web.resolve_public('dual.example'), [('2a04:4e42::81', 443), ('151.101.1.1', 443)])
|
||||
|
||||
def test_a_storage_url_carrying_the_key_is_refused(self):
|
||||
self.connect()
|
||||
for location in ('https://storage.itch-cdn.example/a.apk?api_key=' + KEY,
|
||||
'https://storage.itch-cdn.example/a.apk?API_KEY=someone-elses',
|
||||
'https://storage.itch-cdn.example/a.apk?apikey=x',
|
||||
'https://storage.itch-cdn.example/' + KEY + '/a.apk',
|
||||
'https://storage.itch-cdn.example/a.apk#' + KEY[:10],
|
||||
'https://storage.itch-cdn.example/a.apk?x=' + urllib.parse.quote('%' + KEY),
|
||||
'https://' + KEY.lower() + '.itch-cdn.example/a.apk'):
|
||||
self.storage(location)
|
||||
with self.assertRaisesRegex(SourceError, 'carrying your API key; refused') as caught:
|
||||
itch.download(self.owned_source(), '341190')
|
||||
self.assertNotIn(KEY.lower(), str(caught.exception).lower())
|
||||
self.web.route('api.itch.io/uploads/503/download',
|
||||
Reply('https://api.itch.io/uploads/503/download', 200,
|
||||
{'url': 'https://storage.itch-cdn.example/a.apk?api_key=' + KEY},
|
||||
{'Content-Type': 'application/json'}))
|
||||
with self.assertRaisesRegex(SourceError, 'carrying your API key; refused'):
|
||||
itch.download(self.owned_source(), '341190')
|
||||
self.assertEqual(self.connected, [])
|
||||
# A storage URL's own signature or token parameters are fine.
|
||||
self.storage('https://storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk?token=abc&Signature=x')
|
||||
self.assertTrue(os.path.isfile(itch.download(self.owned_source(), '341190')['apk']))
|
||||
|
||||
def test_a_later_redirect_carrying_the_key_is_refused_before_connecting(self):
|
||||
self.connect()
|
||||
bad = 'https://storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk?api_key=' + KEY
|
||||
for hops in (['https://other-cdn.example/hop1'], # 2nd hop carries the key
|
||||
['https://other-cdn.example/hop1', 'https://other-cdn.example/hop2']): # 3rd hop
|
||||
self.connected.clear()
|
||||
self.storage(hops[0])
|
||||
for here, there in zip(hops, hops[1:] + [bad]):
|
||||
path = urllib.parse.urlsplit(here).path
|
||||
self.web.route('other-cdn.example' + path, Reply(here, 302, b'', {'Location': there}))
|
||||
with self.assertRaisesRegex(SourceError, 'carrying your API key; refused') as caught:
|
||||
itch.download(self.owned_source(), '341190')
|
||||
self.assertNotIn(KEY, str(caught.exception))
|
||||
self.assertEqual([h for h, _ in self.connected], ['other-cdn.example'] * len(hops))
|
||||
self.assertFalse(any(KEY in u for u, _ in self.web.calls))
|
||||
# The check runs on every hop before the name is even looked up.
|
||||
seen = []
|
||||
with patch.object(_web, 'resolve_public', side_effect=AssertionError('looked up')):
|
||||
with self.assertRaisesRegex(SourceError, 'no thanks'):
|
||||
_web.public_apk('https://storage.itch-cdn.example/a.apk', 'itch.io',
|
||||
check=lambda u: seen.append(u) or (_ for _ in ()).throw(SourceError('no thanks')))
|
||||
self.assertEqual(seen, ['https://storage.itch-cdn.example/a.apk'])
|
||||
|
||||
def test_a_partial_match_before_the_key_cannot_hide_its_start(self):
|
||||
key = 'abcdefghABCDEFGHi1234567890'
|
||||
for text in (key[:8].upper() + key.upper(), key[:9] + key, 'xx' + key[:8] + key + 'yy'):
|
||||
out = account._clean(text, key)
|
||||
self.assertNotIn('1234567890', out, (text, out))
|
||||
import time
|
||||
t0 = time.monotonic(); account._clean(('abcdefgh' * 12500)[:100000], key)
|
||||
self.assertLess(time.monotonic() - t0, 2.0)
|
||||
|
||||
def test_redaction_catches_every_spelling_of_the_key(self):
|
||||
hexed = ''.join('%%%02X' % ord(c) for c in KEY)
|
||||
for spelling in (KEY, KEY.lower(), KEY.upper(), hexed, hexed.lower(),
|
||||
KEY[:5] + hexed[15:30] + KEY[10:], # partly encoded
|
||||
KEY[:9], hexed[:30].lower(), # starts of it, 8+ characters
|
||||
''.join('%%%02x' % ord(c) for c in KEY.lower()), # encoded after a case change
|
||||
''.join('%%%02X' % ord(c) for c in KEY.swapcase()),
|
||||
'%%%02x' % ord(KEY[0].swapcase()) + KEY[1:]): # one encoded, recased letter
|
||||
cleaned = account._clean('before ' + spelling + ' after', KEY)
|
||||
self.assertEqual(cleaned, 'before [API key] after', spelling)
|
||||
double = ''.join('%%25%02X' % ord(c) for c in KEY) # percent-encoded twice
|
||||
for spelling in (double, double.lower(), double.upper(), double[:5 * 9], # full, hex recased, 9-char start
|
||||
''.join('%%25%02x' % ord(c) for c in KEY.lower()), # the key recased, then double-encoded
|
||||
KEY[:4] + double[20:45] + hexed[27:33] + KEY[11:]): # literal, double and single mixed
|
||||
self.assertEqual(account._clean('before ' + spelling + ' after', KEY), 'before [API key] after', spelling)
|
||||
self.assertEqual(account._clean(double[:5 * 7], KEY), double[:5 * 7]) # 7 characters: kept
|
||||
self.assertTrue(account._carries_key('https://cdn.example/a?x=' + double, KEY))
|
||||
self.assertTrue(account._carries_key('https://cdn.example/' + double[:5 * 8].lower() + '/a', KEY))
|
||||
self.assertEqual(account._clean('TESTKEY is short', KEY), 'TESTKEY is short') # 7 characters: kept
|
||||
self.assertEqual(account._clean(KEY + KEY.lower(), KEY), '[API key][API key]')
|
||||
for url in ('https://cdn.example/a?x=' + hexed, 'https://cdn.example/a?x=' + hexed.lower(),
|
||||
'https://cdn.example/' + KEY.upper() + '/a', 'https://cdn.example/a?x=' + urllib.parse.quote_plus(KEY)):
|
||||
self.assertTrue(account._carries_key(url, KEY), url)
|
||||
self.assertFalse(account._carries_key('https://cdn.example/a?token=abc&Signature=x', KEY))
|
||||
# Storage errors carrying an encoded or lowercased key lose it, keeping their kind.
|
||||
self.connect()
|
||||
self.storage()
|
||||
with patch.object(_web, 'public_apk', side_effect=SourceLimited('slow ' + hexed.lower(), 42)):
|
||||
with self.assertRaises(SourceLimited) as caught:
|
||||
itch.download(self.owned_source(), '341190')
|
||||
self.assertEqual((str(caught.exception), caught.exception.retry_after), ('slow [API key]', 42))
|
||||
with patch.object(_web, 'public_apk', side_effect=SourceError('host ' + KEY.lower() + '.cdn')):
|
||||
with self.assertRaisesRegex(SourceError, r'^host \[API key\]\.cdn$'):
|
||||
itch.download(self.owned_source(), '341190')
|
||||
|
||||
def test_storage_errors_lose_the_key_but_keep_their_kind(self):
|
||||
self.connect()
|
||||
self.storage()
|
||||
with patch.object(_web, 'public_apk', side_effect=SourceLimited('slow down ' + KEY, 90)):
|
||||
with self.assertRaises(SourceLimited) as caught:
|
||||
itch.download(self.owned_source(), '341190')
|
||||
self.assertEqual((str(caught.exception), caught.exception.retry_after), ('slow down [API key]', 90))
|
||||
with patch.object(_web, 'public_apk', side_effect=SourceError('Could not find the download server ' + KEY)):
|
||||
with self.assertRaisesRegex(SourceError, r'^Could not find the download server \[API key\]$') as caught:
|
||||
itch.download(self.owned_source(), '341190')
|
||||
self.assertIsNone(caught.exception.__cause__)
|
||||
self.assertTrue(caught.exception.__suppress_context__)
|
||||
|
||||
def test_broken_http_responses_are_readable_and_lose_the_key(self):
|
||||
self.connect()
|
||||
|
||||
class Truncated(Reply):
|
||||
def read(self, *args):
|
||||
raise http.client.IncompleteRead(b'{"owned_keys": [' + KEY.encode(), 1000)
|
||||
cases = [http.client.BadStatusLine('HTTP/1.1 ' + KEY + ' oops'),
|
||||
lambda req, q: Truncated(req.full_url, 200, b''),
|
||||
http.client.RemoteDisconnected('closed after ' + KEY)]
|
||||
for case in cases:
|
||||
account._forget()
|
||||
self.web.route('api.itch.io/profile/owned-keys', case)
|
||||
with self.assertRaisesRegex(SourceError, '^Could not reach itch.io: ') as caught:
|
||||
account.owned_keys()
|
||||
self.assertNotIn(KEY, str(caught.exception))
|
||||
self.assertIsNone(caught.exception.__cause__)
|
||||
self.assertTrue(caught.exception.__suppress_context__)
|
||||
self.assertTrue(account.connected())
|
||||
# The same on the download request, for each kind of answer it reads.
|
||||
for case in (http.client.BadStatusLine('HTTP/1.1 ' + KEY),
|
||||
lambda req, q: Truncated(req.full_url, 200, b'', {'Content-Type': 'application/json'}),
|
||||
lambda req, q: Truncated(req.full_url, 200, b'', {'Content-Type': 'application/octet-stream'})):
|
||||
self.web.route('api.itch.io/uploads/503/download', case)
|
||||
with self.assertRaises(SourceError) as caught:
|
||||
account.download(503, 9001)
|
||||
self.assertNotIn(KEY, str(caught.exception))
|
||||
self.assertRegex(str(caught.exception), '^Could not (reach itch.io|download APK): ')
|
||||
|
||||
def test_connecting_shares_one_deadline_across_addresses(self):
|
||||
clock, deadlines = [1000.0], []
|
||||
|
||||
def slow(host, address, timeout, deadline):
|
||||
deadlines.append(deadline)
|
||||
clock[0] += 7 # each address hangs for 7 s before failing
|
||||
raise socket.timeout('timed out')
|
||||
with patch.object(_web.time, 'monotonic', lambda: clock[0]), patch.object(_web, '_connect', slow):
|
||||
with self.assertRaises(OSError):
|
||||
_web._public_connection('many.example', 60)
|
||||
self.assertEqual(len(set(deadlines)), 1) # one deadline for every attempt
|
||||
self.assertEqual(deadlines[0], 1000.0 + _web.CONNECT_BUDGET)
|
||||
self.assertEqual(len(deadlines), 3) # 8 addresses, but only 21 s of trying, not 56
|
||||
self.assertLessEqual(clock[0] - 1000.0, _web.CONNECT_BUDGET + 7)
|
||||
deadlines.clear()
|
||||
clock[0] = 1000.0
|
||||
with patch.object(_web.time, 'monotonic', lambda: clock[0]), patch.object(_web, '_connect', slow):
|
||||
with self.assertRaises(OSError):
|
||||
_web._public_connection('many.example', 5) # the caller's shorter timeout wins
|
||||
self.assertEqual(deadlines, [1005.0])
|
||||
|
||||
def test_a_stalled_handshake_stops_at_the_deadline(self):
|
||||
# A local server that accepts and then says nothing: the TLS handshake never completes.
|
||||
server = socket.socket()
|
||||
server.bind(('127.0.0.1', 0))
|
||||
server.listen(1)
|
||||
self.addCleanup(server.close)
|
||||
timeouts = []
|
||||
real_settimeout = socket.socket.settimeout
|
||||
|
||||
def settimeout(sock, value):
|
||||
timeouts.append(value)
|
||||
return real_settimeout(sock, value)
|
||||
start = time.monotonic()
|
||||
# No thread is left behind to shut the connection later: the deadline is only socket timeouts.
|
||||
with patch.object(_web.threading, 'Timer', side_effect=AssertionError('watchdog thread')), \
|
||||
patch.object(socket.socket, 'settimeout', settimeout):
|
||||
with self.assertRaises(OSError):
|
||||
REAL_CONNECT('storage.itch-cdn.example', server.getsockname(), 60, start + 0.5)
|
||||
self.assertLess(time.monotonic() - start, 3)
|
||||
self.assertTrue(timeouts and all(0 < t <= 0.5 for t in timeouts), timeouts) # what's left, each step
|
||||
with patch.object(_web.socket, 'create_connection', side_effect=AssertionError('connected')):
|
||||
with self.assertRaises(socket.timeout): # a deadline already gone: no connection attempt at all
|
||||
REAL_CONNECT('storage.itch-cdn.example', server.getsockname(), 60, time.monotonic() - 1)
|
||||
|
||||
def test_an_unreachable_address_falls_back_to_the_next_checked_one(self):
|
||||
self.connect()
|
||||
self.storage('https://dual.example/upload/503/OrbitGarden-Quest.apk')
|
||||
self.web.route('dual.example/upload/503/OrbitGarden-Quest.apk', Reply('https://dual.example/', 200, apk_bytes()))
|
||||
self.unreachable.add('2a04:4e42::81')
|
||||
self.assertTrue(os.path.isfile(itch.download(self.owned_source(), '341190')['apk']))
|
||||
self.assertEqual(self.connected, [('dual.example', ('2a04:4e42::81', 443)), ('dual.example', ('151.101.1.1', 443))])
|
||||
self.unreachable.add('151.101.1.1')
|
||||
with self.assertRaisesRegex(SourceError, '^Could not download APK: Network is unreachable$'):
|
||||
itch.download(self.owned_source(), '341190')
|
||||
|
||||
def test_storage_refusals_are_download_errors_not_rate_limits(self):
|
||||
self.connect()
|
||||
self.storage()
|
||||
self.web.route('storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk',
|
||||
Reply('https://storage.itch-cdn.example/x', 403, b'denied'))
|
||||
with self.assertRaisesRegex(SourceError, r"itch.io's download server refused the file \(HTTP 403\)") as caught:
|
||||
itch.download(self.owned_source(), '341190')
|
||||
self.assertNotIsInstance(caught.exception, SourceLimited)
|
||||
self.assertTrue(account.connected())
|
||||
self.assertEqual(_web.wait_time('https://api.itch.io/'), 0)
|
||||
self.web.route('storage.itch-cdn.example/upload/503/OrbitGarden-Quest.apk',
|
||||
Reply('https://storage.itch-cdn.example/x', 429, b'', {'Retry-After': '60'}))
|
||||
with self.assertRaises(SourceLimited):
|
||||
itch.download(self.owned_source(), '341190')
|
||||
|
||||
def test_a_key_revoked_after_details_were_cached_is_removed_at_download(self):
|
||||
self.connect()
|
||||
itch.details(self.owned_source(), '341190') # uploads and owned keys now cached
|
||||
for code in (401, 403):
|
||||
self.connect()
|
||||
self.web.route('api.itch.io/uploads/503/download', Reply('https://api.itch.io/uploads/503/download', code, b''))
|
||||
self.web.route('api.itch.io/profile', Reply('https://api.itch.io/profile', code, b''))
|
||||
with self.assertRaisesRegex(account.BadKey, 'no longer accepts the saved API key'):
|
||||
account.download(503, 9001)
|
||||
self.assertFalse(account.path().exists())
|
||||
self.web.route('api.itch.io/profile', {'user': {'username': 'player'}})
|
||||
# The key still works, only this download is refused: keep the key, say so.
|
||||
self.connect()
|
||||
self.web.route('api.itch.io/uploads/503/download', Reply('https://api.itch.io/uploads/503/download', 403, b''))
|
||||
with self.assertRaisesRegex(SourceError, r'refused this request \(HTTP 403\)') as caught:
|
||||
account.download(503, 9001)
|
||||
self.assertNotIsInstance(caught.exception, (account.BadKey, SourceLimited))
|
||||
self.assertTrue(account.connected())
|
||||
|
||||
def test_a_disconnect_during_the_key_check_wins(self):
|
||||
checking, release = threading.Event(), threading.Event()
|
||||
|
||||
def slow_profile(req, query):
|
||||
checking.set()
|
||||
release.wait(5)
|
||||
return {'user': {'username': 'player'}}
|
||||
self.web.route('api.itch.io/profile', slow_profile)
|
||||
outcome = {}
|
||||
|
||||
def run():
|
||||
try:
|
||||
outcome['result'] = account.connect(KEY)
|
||||
except SourceError as e:
|
||||
outcome['error'] = e
|
||||
worker = threading.Thread(target=run)
|
||||
worker.start()
|
||||
self.assertTrue(checking.wait(5))
|
||||
account.disconnect()
|
||||
release.set()
|
||||
worker.join(5)
|
||||
self.assertRegex(str(outcome.get('error')), 'changed while the key was being checked; nothing was saved')
|
||||
self.assertFalse(account.path().exists())
|
||||
self.assertFalse(account.connected())
|
||||
|
||||
def test_an_old_refusal_does_not_undo_a_reconnect_with_the_same_key(self):
|
||||
self.connect()
|
||||
listing, release, profile_ok = threading.Event(), threading.Event(), [True]
|
||||
|
||||
def slow_owned(req, query):
|
||||
listing.set()
|
||||
release.wait(5)
|
||||
return Reply(req.full_url, 401, b'')
|
||||
self.web.route('api.itch.io/profile/owned-keys', slow_owned)
|
||||
self.web.route('api.itch.io/profile', lambda req, q: {'user': {'username': 'player'}} if profile_ok[0]
|
||||
else Reply(req.full_url, 401, b''))
|
||||
outcome = {}
|
||||
|
||||
def run():
|
||||
try:
|
||||
account.owned_keys()
|
||||
except SourceError as e:
|
||||
outcome['error'] = e
|
||||
worker = threading.Thread(target=run)
|
||||
worker.start()
|
||||
self.assertTrue(listing.wait(5))
|
||||
account.disconnect()
|
||||
account.connect(KEY) # same key, connected again while the old listing waits
|
||||
profile_ok[0] = False # the old listing's confirmation now sees a refusal
|
||||
release.set()
|
||||
worker.join(5)
|
||||
self.assertIsInstance(outcome.get('error'), account.BadKey)
|
||||
self.assertTrue(account.path().exists()) # the fresh connection survives
|
||||
self.assertEqual(json.loads(account.path().read_text())['key'], KEY)
|
||||
|
||||
def test_redacted_text_is_not_mistaken_for_a_key_error(self):
|
||||
self.connect()
|
||||
before = len(self.web.urls('api.itch.io/profile'))
|
||||
self.web.route('api.itch.io/profile/owned-keys', {'errors': ['Invalid request: ' + KEY]})
|
||||
with self.assertRaisesRegex(SourceError, r'^itch.io: Invalid request: \[API key\]$') as caught:
|
||||
account.owned_keys()
|
||||
self.assertNotIsInstance(caught.exception, account.BadKey)
|
||||
self.assertTrue(account.connected())
|
||||
self.assertEqual(len(self.web.urls('api.itch.io/profile')), before) # not even a key check needed
|
||||
|
||||
def test_an_invalid_key_error_elsewhere_is_confirmed_before_deleting(self):
|
||||
self.connect()
|
||||
self.web.route('api.itch.io/profile/owned-keys', {'errors': ['invalid key']})
|
||||
with self.assertRaisesRegex(SourceError, '^itch.io: invalid key$') as caught:
|
||||
account.owned_keys() # /profile still accepts the key: keep it
|
||||
self.assertNotIsInstance(caught.exception, account.BadKey)
|
||||
self.assertTrue(account.connected())
|
||||
account._forget()
|
||||
self.web.route('api.itch.io/profile', {'errors': ['invalid key']})
|
||||
with self.assertRaisesRegex(account.BadKey, 'no longer accepts'):
|
||||
account.owned_keys()
|
||||
self.assertFalse(account.connected())
|
||||
|
||||
def test_concurrent_connects_never_share_a_temporary_file(self):
|
||||
made = []
|
||||
real = account.tempfile.mkstemp
|
||||
|
||||
def mkstemp(*args, **kwargs):
|
||||
fd, name = real(*args, **kwargs)
|
||||
made.append(name)
|
||||
if os.name == 'posix':
|
||||
self.assertEqual(os.stat(name).st_mode & 0o777, 0o600)
|
||||
return fd, name
|
||||
with patch.object(account.tempfile, 'mkstemp', mkstemp):
|
||||
workers = [threading.Thread(target=lambda k=k: self.swallow(account.connect, k)) for k in (KEY, OTHER) * 3]
|
||||
for w in workers:
|
||||
w.start()
|
||||
for w in workers:
|
||||
w.join(5)
|
||||
self.assertEqual(len(made), len(set(made)))
|
||||
self.assertIn(json.loads(account.path().read_text())['key'], (KEY, OTHER))
|
||||
self.assertEqual([p.name for p in account.path().parent.iterdir()], ['itch-account.json'])
|
||||
|
||||
@staticmethod
|
||||
def swallow(fn, *args):
|
||||
try:
|
||||
fn(*args)
|
||||
except SourceError:
|
||||
pass # a connect overtaken by another one refuses to save: fine here
|
||||
|
||||
def test_existing_hosts_still_strip_authorization_on_redirect(self):
|
||||
req = urllib.request.Request('https://api.github.com/a', headers={'Authorization': 'Bearer x'})
|
||||
nxt = _web.Redirect(('api.github.com',)).redirect_request(req, None, 302, 'Found', {}, 'https://api.github.com/b')
|
||||
self.assertFalse(nxt.has_header('Authorization'))
|
||||
with self.assertRaises(SourceError):
|
||||
_web.Redirect(('api.github.com',)).redirect_request(req, None, 302, 'Found', {}, 'https://evil.example/b')
|
||||
|
||||
def test_install_uses_the_normal_apk_install_path(self):
|
||||
import frame_android
|
||||
self.connect()
|
||||
self.storage()
|
||||
calls = []
|
||||
|
||||
def install(apk_path, flatscreen=None, name=None, source=None, icon_png=None, xr_compat=None, artwork=None):
|
||||
with zipfile.ZipFile(apk_path) as z:
|
||||
self.assertIn('AndroidManifest.xml', z.namelist())
|
||||
calls.append((name, source))
|
||||
return {'package': 'com.maker.orbit', 'message': 'Installed'}
|
||||
with patch.object(frame_android, 'install', install):
|
||||
result = search.install(itch.OWNED, '341190')
|
||||
self.assertEqual(calls, [('Orbit Garden VR', 'itch.io (my purchases)')])
|
||||
self.assertEqual(result['package'], 'com.maker.orbit')
|
||||
|
||||
# -- errors never carry the key ------------------------------------------------
|
||||
|
||||
def test_errors_are_readable_and_never_contain_the_key(self):
|
||||
self.connect()
|
||||
cases = [
|
||||
(Reply('https://api.itch.io/profile/owned-keys', 429, b'', {'Retry-After': '120'}),
|
||||
SourceLimited, '^itch.io is limiting requests; try again in 2 minutes$'),
|
||||
(Reply('https://api.itch.io/profile/owned-keys', 503, b''), SourceError, 'HTTP error 503; try again later'),
|
||||
(urllib.error.URLError('connect to api.itch.io with ' + KEY + ' failed'), SourceError,
|
||||
r'^Could not reach itch.io: connect to api.itch.io with \[API key\] failed$'),
|
||||
(OSError('timed out'), SourceError, '^Could not reach itch.io: timed out$'),
|
||||
(Reply('https://api.itch.io/profile/owned-keys', 200, b'<html>'), SourceError, 'invalid response'),
|
||||
({'errors': ['echo ' + KEY]}, SourceError, r'^itch.io: echo \[API key\]$'),
|
||||
# The key straddles the 200-character cut: no piece of it may survive.
|
||||
({'errors': ['x' * 180, KEY]}, SourceError, r'^itch.io: x{180}; \[API key\]$'),
|
||||
({'errors': ['x' * 190 + KEY]}, SourceError, r'^itch.io: x{190}\[API key\]$'),
|
||||
]
|
||||
self.assertEqual(account._clean('cut at ' + KEY[:9], KEY), 'cut at [API key]') # a truncated key's start
|
||||
stderr = io.StringIO()
|
||||
with contextlib.redirect_stderr(stderr):
|
||||
for reply, kind, message in cases:
|
||||
account._forget()
|
||||
_web._limited.clear()
|
||||
self.web.route('api.itch.io/profile/owned-keys', reply)
|
||||
with self.assertRaisesRegex(kind, message) as caught:
|
||||
account.owned_keys()
|
||||
self.assertNotIn(KEY, str(caught.exception))
|
||||
self.assertIsNone(caught.exception.__cause__)
|
||||
self.assertTrue(account.connected()) # not the key's fault: stays connected
|
||||
# The store's search reports the failure as a non-fatal source status.
|
||||
account._forget()
|
||||
result = search.search('', source=itch.OWNED)
|
||||
self.assertEqual(result['sources'][0]['status'], 'error')
|
||||
self.assertNotIn(KEY, json.dumps(result) + stderr.getvalue())
|
||||
|
||||
def test_server_api_connects_and_never_echoes_the_key(self):
|
||||
httpd = server.ThreadingHTTPServer(('127.0.0.1', 0), server.Handler)
|
||||
threading.Thread(target=httpd.serve_forever, daemon=True).start()
|
||||
self.addCleanup(httpd.server_close)
|
||||
self.addCleanup(httpd.shutdown)
|
||||
|
||||
def request(method, path, body=None):
|
||||
c = http.client.HTTPConnection('127.0.0.1', httpd.server_port)
|
||||
c.request(method, path, json.dumps(body) if body is not None else None,
|
||||
{'X-Frame-UI': '1', 'Content-Type': 'application/json'})
|
||||
r = c.getresponse()
|
||||
out = r.status, r.read().decode()
|
||||
c.close()
|
||||
return out
|
||||
stderr = io.StringIO()
|
||||
with contextlib.redirect_stderr(stderr):
|
||||
self.web.route('api.itch.io/profile', Reply('https://api.itch.io/profile', 401, b''))
|
||||
bad = request('POST', '/api/sources', {'action': 'itch-connect', 'key': KEY})
|
||||
self.web.route('api.itch.io/profile', {'user': {'username': 'player'}})
|
||||
good = request('POST', '/api/sources', {'action': 'itch-connect', 'key': KEY})
|
||||
listed = request('GET', '/api/sources')
|
||||
empty = request('POST', '/api/sources', {'action': 'itch-connect', 'key': ' '})
|
||||
gone = request('POST', '/api/sources', {'action': 'itch-disconnect'})
|
||||
self.assertEqual(bad[0], 400)
|
||||
self.assertIn('did not accept the API key', json.loads(bad[1])['error'])
|
||||
self.assertEqual(good[0], 200)
|
||||
self.assertEqual(json.loads(good[1])['itch'], {'connected': True, 'username': 'player'})
|
||||
self.assertEqual(json.loads(listed[1])['itch'], {'connected': True, 'username': 'player'})
|
||||
self.assertIn(itch.OWNED, [s['id'] for s in json.loads(listed[1])['sources']])
|
||||
self.assertEqual(empty[0], 400)
|
||||
self.assertEqual(json.loads(gone[1])['itch'], {'connected': False, 'username': None})
|
||||
self.assertFalse(account.path().exists())
|
||||
for text in (bad[1], good[1], listed[1], empty[1], gone[1], stderr.getvalue()):
|
||||
self.assertNotIn(KEY, text)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
unittest.main()
|
||||
@@ -1,4 +1,5 @@
|
||||
"""Every public sideload entry point reaches the mandatory five-slot writer."""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import contextlib
|
||||
import io
|
||||
import json
|
||||
|
||||
+428
-5
@@ -9,6 +9,7 @@ import http.client
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import shlex
|
||||
import shutil
|
||||
import socket
|
||||
import subprocess
|
||||
@@ -199,7 +200,7 @@ class Connecting(unittest.TestCase):
|
||||
rows = {p["host"]: p for p in s["probes"]}
|
||||
self.assertEqual(rows["nothing.invalid"]["state"], "unresolved")
|
||||
self.assertEqual(rows["::1"]["state"], "sshfailed")
|
||||
self.assertIn("different headset", rows["::1"]["detail"])
|
||||
self.assertIn("different SSH identity", rows["::1"]["detail"])
|
||||
# Every ssh command was pointed at the winner, with the host key pinned per device.
|
||||
alias, opts = self.routes[-1]
|
||||
self.assertEqual(alias, "frame-t")
|
||||
@@ -208,9 +209,10 @@ class Connecting(unittest.TestCase):
|
||||
self.assertIn(f"Port={self.port}", opts)
|
||||
master = [c for c in self.calls() if "ControlMaster=yes" in c][-1]
|
||||
self.assertIn("StrictHostKeyChecking=accept-new", master) # first connection: nothing pinned yet
|
||||
# ssh takes an option's first value: accept-new must come before the commands' own "yes".
|
||||
self.assertLess(master.index("StrictHostKeyChecking=accept-new"), master.index("StrictHostKeyChecking=yes"))
|
||||
self.assertIn("StrictHostKeyChecking=yes", opts) # every other command checks the pinned key
|
||||
self.assertNotIn("StrictHostKeyChecking=yes", master)
|
||||
# This stand-in ssh saved no key, so commands may still save the first one (accept-new);
|
||||
# test_a_first_connection_pins_and_then_checks_strictly has real pins.
|
||||
self.assertIn("StrictHostKeyChecking=accept-new", opts)
|
||||
self.assertTrue(fd.known_hosts(d["id"]).parent.is_dir()) # where ssh saves the key it accepts
|
||||
# It learned: 127.0.0.1 works on this network.
|
||||
learned = {a["host"]: a for a in self.reg.get(d["id"])["addresses"]}
|
||||
@@ -257,6 +259,7 @@ class Connecting(unittest.TestCase):
|
||||
s = self.link.snapshot()
|
||||
self.assertEqual((s["phase"], s["error"]["stage"]), ("failed", "login"))
|
||||
self.assertIn("SSH key", s["error"]["message"])
|
||||
self.assertEqual(s["error"]["auth"], "denied") # the page offers Set Up Connection
|
||||
|
||||
def test_pinned_identity_is_checked_strictly(self):
|
||||
d = self.device("localhost")
|
||||
@@ -266,6 +269,426 @@ class Connecting(unittest.TestCase):
|
||||
master = [c for c in self.calls() if "ControlMaster=yes" in c][-1]
|
||||
self.assertIn("StrictHostKeyChecking=yes", master)
|
||||
|
||||
def pins(self):
|
||||
"""The stand-in ssh checks host keys as OpenSSH does (tests/fakessh/ssh)."""
|
||||
p = mock.patch.dict(os.environ, {"FAKESSH_PINS": "1"})
|
||||
p.start()
|
||||
self.addCleanup(p.stop)
|
||||
|
||||
def command(self, device, host="127.0.0.1"):
|
||||
"""One ssh command as the server runs it, with the route the connector applied."""
|
||||
return subprocess.run(["ssh", *self.link.host_opts(device, host), device["alias"], "true"],
|
||||
capture_output=True, text=True, timeout=20)
|
||||
|
||||
def test_a_first_connection_pins_and_then_checks_strictly(self):
|
||||
# 0.4.2 on Windows: "No ED25519 host key is known for frame-control-<id> and you have
|
||||
# requested strict checking." Every command said "yes" before anything was saved, so one
|
||||
# that ran before the connector's handshake saved the key (or without it) failed.
|
||||
self.pins()
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "ok"})
|
||||
r = self.command(d)
|
||||
self.assertEqual(r.returncode, 0, r.stderr) # the first contact saves the key...
|
||||
self.assertIn("Permanently added", r.stderr)
|
||||
self.assertTrue(fd.pinned(d["id"]))
|
||||
self.assertIn("StrictHostKeyChecking=yes", self.link.host_opts(d, "127.0.0.1")) # ...then it's checked
|
||||
self.assertEqual(self.reg.get(d["id"])["identity"], [k[1] for k in fd.pinned_keys(d["id"])]) # and recorded
|
||||
# The connector's first handshake, for another new headset.
|
||||
self.reg.remove_device(d["id"])
|
||||
d = self.device("localhost")
|
||||
self.link.connect(["start"])
|
||||
s = self.link.snapshot()
|
||||
self.assertEqual(s["phase"], "connected", s["error"])
|
||||
self.assertTrue(fd.pinned(d["id"]))
|
||||
self.assertTrue(self.reg.get(d["id"])["identity"])
|
||||
alias, opts = self.routes[-1]
|
||||
self.assertIn("StrictHostKeyChecking=yes", opts) # every command after it checks strictly
|
||||
self.assertNotIn("StrictHostKeyChecking=accept-new", opts)
|
||||
|
||||
def lose_pin(self, device_id, how):
|
||||
f = fd.known_hosts(device_id)
|
||||
if how == "deleted":
|
||||
f.unlink()
|
||||
elif how == "empty":
|
||||
f.write_text("")
|
||||
elif how == "unreadable":
|
||||
f.chmod(0)
|
||||
self.addCleanup(lambda: f.exists() and f.chmod(0o600))
|
||||
elif how == "another alias":
|
||||
f.write_text(f.read_text().replace(f"frame-control-{device_id}", "frame-control-someone-else"))
|
||||
elif how == "not utf-8":
|
||||
f.write_bytes(b"\xff\xfe" + f.read_bytes())
|
||||
|
||||
def test_a_lost_or_unusable_pin_fails_closed(self):
|
||||
# Review pr85-review-r1: accept-new can't tell a new headset from one whose saved key
|
||||
# went missing; real OpenSSH then trusted a replacement key. Once a headset had an
|
||||
# identity, nothing is trusted again until the person checks it.
|
||||
self.pins()
|
||||
cases = ["deleted", "empty", "another alias", "not utf-8"]
|
||||
if os.name == "posix" and os.geteuid() != 0:
|
||||
cases.append("unreadable") # root reads anything
|
||||
for n, how in enumerate(cases):
|
||||
with self.subTest(how=how):
|
||||
self.hosts({"localhost": "ok"})
|
||||
d = self.reg.add_device(f"frame-l{n}", port=self.port, hosts=[])
|
||||
self.reg.add_address(d["id"], "localhost", kind="lan")
|
||||
self.link.override = None
|
||||
self.reg.set_active(d["id"])
|
||||
self.link.connect(["switch"])
|
||||
self.assertEqual(self.link.snapshot()["phase"], "connected")
|
||||
self.link.close_master()
|
||||
d = self.reg.get(d["id"])
|
||||
self.lose_pin(d["id"], how)
|
||||
self.hosts({"localhost": "wrong"}) # another device, or a reinstalled headset
|
||||
r = self.command(d)
|
||||
self.assertEqual(r.returncode, 255, r.stderr)
|
||||
self.assertIn("Host key verification failed", r.stderr)
|
||||
self.assertNotIn("Permanently added", r.stderr)
|
||||
self.link.connect(["dropped"])
|
||||
s = self.link.snapshot()
|
||||
self.assertEqual((s["phase"], s["error"]["stage"]), ("failed", "identity"))
|
||||
self.assertEqual(s["error"]["hostkey"]["state"], "missing")
|
||||
self.assertIn("missing or can't be read", s["error"]["message"])
|
||||
view = next(x for x in fl.devices_view(self.link)["devices"] if x["id"] == d["id"])
|
||||
self.assertTrue(view["identity_lost"])
|
||||
scan = fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)["scan"]
|
||||
self.assertFalse(scan["matches"] or scan["recorded_match"]) # not the key it had
|
||||
self.hosts({"localhost": "ok"})
|
||||
scan = fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)["scan"]
|
||||
self.assertTrue(scan["recorded_match"]) # its own key, as recorded
|
||||
self.assertFalse(scan["matches"]) # but not usable from the file: trusting restores it
|
||||
fl.devices_action(self.link, {"action": "identity-trust", "id": d["id"],
|
||||
"fingerprint": scan["fingerprint"]}, None)
|
||||
self.link.connect(["switch"])
|
||||
self.assertEqual(self.link.snapshot()["phase"], "connected", self.link.snapshot()["error"])
|
||||
self.link.close_master()
|
||||
|
||||
def test_a_key_check_only_reads_the_key(self):
|
||||
# Review pr85-review-r1: with ForwardAgent yes in ~/.ssh/config, a device that accepts
|
||||
# "none" got the agent and ran a command before anyone trusted it.
|
||||
self.pins()
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "evil"})
|
||||
os.environ["FAKESSH_CONFIG"] = json.dumps({"forwardagent": "yes", "remotecommand": "touch /tmp/x",
|
||||
"permitlocalcommand": "yes", "requesttty": "force"})
|
||||
self.addCleanup(os.environ.pop, "FAKESSH_CONFIG", None)
|
||||
for proxied in (False, True):
|
||||
route = ("jump", "me@jumpy:2222") if proxied else (None, None)
|
||||
with self.subTest(proxied=proxied), mock.patch.object(fl, "proxy_route", return_value=route):
|
||||
t0 = time.monotonic()
|
||||
scan = fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)["scan"]
|
||||
self.assertLess(time.monotonic() - t0, 10) # stopped once it had the key, not left open
|
||||
self.assertTrue(scan["fingerprint"].startswith("SHA256:"))
|
||||
self.assertEqual([p.name for p in self.dir.iterdir() if p.name.startswith("evil-")], [])
|
||||
call = [c for c in self.calls() if "PreferredAuthentications=none" in c][-1]
|
||||
self.assertEqual(call[call.index("-F") + 1], "none") # never ~/.ssh/config for the device
|
||||
self.assertIn("-N", call)
|
||||
hop = [c for c in call if c.startswith("ProxyCommand=")]
|
||||
if proxied:
|
||||
# Review pr85-review-r2: ProxyJump's own ssh inherits the config (ForwardAgent,
|
||||
# LocalCommand); the scan builds the jump host's ssh itself, hardened.
|
||||
self.assertEqual(len(hop), 1)
|
||||
words = shlex.split(hop[0][len("ProxyCommand="):])
|
||||
self.assertEqual(words[-7:], ["-l", "me", "-p", "2222", "-W", "[%h]:%p", "jumpy"])
|
||||
for o in fl.Link.HOP_OPTS:
|
||||
self.assertIn(o, words)
|
||||
self.assertLess(call.index(hop[0]), call.index("HostKeyAlias=frame-control-" + d["id"]))
|
||||
else:
|
||||
self.assertEqual(hop, [])
|
||||
for o in ("ForwardAgent=no", "IdentityAgent=none", "ClearAllForwardings=yes", "RemoteCommand=none",
|
||||
"PermitLocalCommand=no", "RequestTTY=no", "PubkeyAuthentication=no"):
|
||||
self.assertIn(o, call)
|
||||
self.assertLess(call.index("StrictHostKeyChecking=accept-new"), call.index("HostKeyAlias=frame-control-"
|
||||
+ d["id"]))
|
||||
self.assertFalse(fd.pinned(d["id"]))
|
||||
self.assertEqual(sorted(p.name for p in fd.known_hosts(d["id"]).parent.iterdir()), []) # no scan files
|
||||
|
||||
@unittest.skipIf(os.name != "posix", "the stand-in ssh runs ProxyCommand with /bin/sh")
|
||||
def test_the_jump_hosts_command_line_is_quoted_whatever_the_names(self):
|
||||
# Review pr85-review-r3: the hop's words went into the ProxyCommand line unquoted, so an
|
||||
# ssh at /tmp/x/bin$(:>INJECTED)/ssh ran shell code; and an IPv6 jump host broke.
|
||||
self.pins()
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "ok"})
|
||||
for n, name in enumerate(["with space", "bin$(touch INJECTED-0)", "back`touch INJECTED-1`tick",
|
||||
"quote'it\"s", "per%cent %h %%", "semi;touch INJECTED-2"]):
|
||||
with self.subTest(name=name):
|
||||
bindir = self.dir / f"b{n}" / name
|
||||
bindir.mkdir(parents=True)
|
||||
(bindir / "ssh").symlink_to(FAKESSH / "ssh")
|
||||
for spec, at in (("me@jumpy:2222", "jumpy"), ("[::1]:2200", "::1")):
|
||||
before = len(self.calls())
|
||||
with mock.patch.dict(os.environ, {"PATH": f"{bindir}{os.pathsep}{os.environ['PATH']}"}), \
|
||||
mock.patch.object(fl, "proxy_route", return_value=("jump", spec)):
|
||||
scan = fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)["scan"]
|
||||
self.assertTrue(scan["fingerprint"].startswith("SHA256:"))
|
||||
hops = [c for c in self.calls()[before:] if "-W" in c]
|
||||
self.assertEqual(len(hops), 1, self.calls()[before:])
|
||||
self.assertEqual(hops[0][-1], at) # the jump host, bare
|
||||
self.assertEqual(hops[0][hops[0].index("-W") + 1], "[localhost]:%d" % self.port)
|
||||
outer = [c for c in self.calls()[before:] if "PreferredAuthentications=none" in c][-1]
|
||||
line = next(c for c in outer if c.startswith("ProxyCommand="))
|
||||
self.assertTrue(line[len("ProxyCommand="):].startswith(shlex.quote(str(bindir / "ssh")
|
||||
.replace("%", "%%"))))
|
||||
self.assertEqual([p.name for p in self.dir.rglob("INJECTED*")], [])
|
||||
self.assertFalse(list(Path.cwd().glob("INJECTED*")))
|
||||
|
||||
def test_a_key_check_refuses_routes_it_cant_harden(self):
|
||||
self.pins()
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "ok"})
|
||||
before = len(self.calls())
|
||||
for route in (("command", "nc %h %p"), ("jump", "a,b"), ("jump", "-oProxyCommand=x"), ("jump", "ssh://a"),
|
||||
("jump", "a b"), ("jump", "a%h"), ("jump", "jumpy:99999")):
|
||||
with self.subTest(route=route), mock.patch.object(fl, "proxy_route", return_value=route):
|
||||
with self.assertRaises(fd.DeviceError) as caught:
|
||||
fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)
|
||||
self.assertIn("can't make safe", str(caught.exception))
|
||||
self.assertFalse(any("PreferredAuthentications=none" in c for c in self.calls()[before:]))
|
||||
|
||||
def test_a_key_read_before_the_address_or_port_changed_isnt_trusted(self):
|
||||
self.pins()
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "ok"})
|
||||
for change in ({"action": "update", "id": d["id"], "port": self.port + 1},
|
||||
{"action": "address-remove", "id": d["id"], "host": "localhost"}):
|
||||
with self.subTest(change=change["action"]):
|
||||
scan = fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)["scan"]
|
||||
fl.devices_action(self.link, change, None)
|
||||
with self.assertRaises(fd.DeviceError):
|
||||
fl.devices_action(self.link, {"action": "identity-trust", "id": d["id"],
|
||||
"fingerprint": scan["fingerprint"]}, None)
|
||||
self.assertFalse(fd.pinned(d["id"]))
|
||||
self.reg.update_device(d["id"], port=self.port)
|
||||
if not self.reg.get(d["id"])["addresses"]:
|
||||
self.reg.add_address(d["id"], "localhost", kind="lan")
|
||||
|
||||
def test_terminals_judge_identity_by_the_registry_not_a_frozen_route(self):
|
||||
# Review pr85-review-r2: named_route used the device as routed (before its first key was
|
||||
# recorded), so a Terminal window fell back to accept-new once the pin was deleted.
|
||||
self.pins()
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "ok"})
|
||||
self.link.connect(["start"])
|
||||
self.assertEqual(self.link.snapshot()["phase"], "connected")
|
||||
self.assertFalse(self.link.routed_device.get("identity")) # the frozen copy knows nothing of it
|
||||
fd.known_hosts(d["id"]).unlink()
|
||||
alias, opts = self.link.named_route()
|
||||
self.assertIn("StrictHostKeyChecking=yes", opts)
|
||||
self.assertNotIn("StrictHostKeyChecking=accept-new", opts)
|
||||
self.hosts({"localhost": "wrong"})
|
||||
r = subprocess.run(["ssh", *opts, alias, "true"], capture_output=True, text=True, timeout=20)
|
||||
self.assertEqual(r.returncode, 255, r.stderr)
|
||||
self.assertFalse(fd.pinned(d["id"]))
|
||||
|
||||
def test_a_key_saved_before_a_refused_login_is_kept_and_checked(self):
|
||||
# Review pr85-review-r2: ssh saves the key during the key exchange; if the login is then
|
||||
# refused, it was neither recorded nor protected, and the route stayed accept-new.
|
||||
self.pins()
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "denied"})
|
||||
self.link.connect(["start"])
|
||||
s = self.link.snapshot()
|
||||
self.assertEqual((s["phase"], s["error"]["stage"]), ("failed", "login"))
|
||||
got = self.reg.get(d["id"])
|
||||
self.assertEqual(got["identity"], [f for _, f in fd.pinned_keys(d["id"])])
|
||||
self.assertTrue(got["identity_established"])
|
||||
if os.name == "posix":
|
||||
self.assertEqual(fd.known_hosts(d["id"]).stat().st_mode & 0o777, 0o600)
|
||||
self.assertIn("StrictHostKeyChecking=yes", self.routes[-1][1])
|
||||
fd.known_hosts(d["id"]).unlink()
|
||||
self.hosts({"localhost": "wrong"})
|
||||
r = self.command(self.reg.get(d["id"]))
|
||||
self.assertEqual(r.returncode, 255)
|
||||
self.assertNotIn("Permanently added", r.stderr)
|
||||
|
||||
def test_a_key_a_command_saved_tightens_the_route(self):
|
||||
self.pins()
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "denied"})
|
||||
with self.link.route_lock:
|
||||
self.link.routed_device = d
|
||||
self.link.apply(d["alias"], self.link.host_opts(d, "127.0.0.1"))
|
||||
opts = self.routes[-1][1]
|
||||
self.assertIn("StrictHostKeyChecking=accept-new", opts)
|
||||
r = subprocess.run(["ssh", *opts, d["alias"], "true"], capture_output=True, text=True, timeout=20)
|
||||
self.assertIn("Permanently added", r.stderr)
|
||||
self.assertEqual(r.returncode, 255) # saved the key, then refused the login
|
||||
self.link.settle_pin() # what the connector's loop does every tick while accept-new is applied
|
||||
self.assertIn("StrictHostKeyChecking=yes", self.routes[-1][1])
|
||||
self.assertNotIn("StrictHostKeyChecking=accept-new", self.routes[-1][1])
|
||||
self.assertTrue(self.reg.get(d["id"])["identity"])
|
||||
|
||||
def test_editing_the_address_that_worked_keeps_the_identity_history(self):
|
||||
# Review pr85-review-r2: last_ok lives on the address, so editing or removing it made a
|
||||
# headset whose pin was gone (and had no fingerprint recorded) new again.
|
||||
path = self.dir / "legacy.json"
|
||||
path.write_text(json.dumps({"version": 1, "active": "abcd1234", "networks": {}, "devices": [
|
||||
{"id": "abcd1234", "name": "Old", "alias": "frame-old", "user": "steamos", "port": 22,
|
||||
"identity_files": [], "addresses": [{"host": "10.0.0.5", "kind": "lan", "label": "",
|
||||
"networks": [], "last_ok": 1790000000.0, "last_rtt_ms": 3}]}]}))
|
||||
reg = fd.Registry(path)
|
||||
self.assertTrue(reg.get("abcd1234")["identity_established"]) # migrated from last_ok
|
||||
reg.update_address("abcd1234", "10.0.0.5", new_host="10.0.0.6")
|
||||
reg.remove_address("abcd1234", "10.0.0.6")
|
||||
reg.add_address("abcd1234", "10.0.0.7")
|
||||
d = fd.Registry(path).get("abcd1234") # saved, too
|
||||
self.assertFalse(any(a.get("last_ok") for a in d["addresses"]))
|
||||
self.assertTrue(fd.identity_established(d))
|
||||
self.assertEqual(self.link.strictness(d), "yes")
|
||||
# A connection that works sets it as well, for a headset added since.
|
||||
n = self.reg.add_device("frame-new", hosts=["10.0.0.8"])
|
||||
self.reg.record_success(n["id"], "10.0.0.8", None, 1)
|
||||
self.reg.remove_address(n["id"], "10.0.0.8")
|
||||
self.assertTrue(self.reg.get(n["id"])["identity_established"])
|
||||
|
||||
def test_ssh_last_words_arrive_even_when_the_reader_lags(self):
|
||||
# Review pr85-review-r2: ssh's exit with nothing queued yet was taken as the end, and a
|
||||
# late reader lost "Permission denied" ("ssh stopped").
|
||||
self.pins()
|
||||
link = fl.Link(self.reg, env_alias=None, mux_base=["ssh", "-o", "BatchMode=yes"], control=None,
|
||||
apply=lambda *a: None, explain=explain)
|
||||
self.addCleanup(link.stop)
|
||||
real = threading.Thread
|
||||
|
||||
class Late(real):
|
||||
def __init__(self, *a, target=None, **kw):
|
||||
if target is not None and getattr(target, "__name__", "") == "tail":
|
||||
inner = target
|
||||
|
||||
def target():
|
||||
time.sleep(1)
|
||||
inner()
|
||||
super().__init__(*a, target=target, **kw)
|
||||
self.device("localhost")
|
||||
self.hosts({"localhost": "denied"})
|
||||
with mock.patch.object(fl.threading, "Thread", Late):
|
||||
link.connect(["start"])
|
||||
s = link.snapshot()
|
||||
self.assertEqual((s["error"]["stage"], s["error"].get("auth")), ("login", "denied"), s["error"])
|
||||
self.assertIn("SSH key", s["error"]["message"])
|
||||
|
||||
def test_forgetting_an_identity_is_refused(self):
|
||||
self.pins()
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "ok"})
|
||||
self.link.connect(["start"])
|
||||
saved = fd.known_hosts(d["id"]).read_text()
|
||||
with self.assertRaises(fd.DeviceError) as caught:
|
||||
fl.devices_action(self.link, {"action": "forget-identity", "id": d["id"]}, None)
|
||||
self.assertIn("identity-scan", str(caught.exception))
|
||||
self.assertEqual(fd.known_hosts(d["id"]).read_text(), saved)
|
||||
|
||||
def test_without_a_shared_connection_ssh_writes_to_a_file_not_a_pipe(self):
|
||||
# Windows: no ControlMaster, and its OpenSSH can block writing to a stderr pipe.
|
||||
self.pins()
|
||||
tmp = self.dir / "tmp"
|
||||
tmp.mkdir()
|
||||
link = fl.Link(self.reg, env_alias=None, mux_base=["ssh", "-o", "BatchMode=yes"], control=None,
|
||||
apply=lambda *a: None, explain=explain)
|
||||
self.addCleanup(link.stop)
|
||||
real = subprocess.Popen
|
||||
seen = []
|
||||
|
||||
def popen(argv, **kw):
|
||||
if "-v" in argv: # the handshake (run_ssh calls go through Popen too)
|
||||
seen.append(kw.get("stderr"))
|
||||
return real(argv, **kw)
|
||||
with mock.patch.object(tempfile, "tempdir", str(tmp)), mock.patch.object(fl.subprocess, "Popen", popen):
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "ok"})
|
||||
link.connect(["start"])
|
||||
s = link.snapshot()
|
||||
self.assertEqual(s["phase"], "connected", s["error"])
|
||||
self.assertEqual([st["state"] for st in s["stages"]], ["done"] * 5)
|
||||
self.assertIn("saved this headset's identity", next(st for st in s["stages"] if st["id"] == "identity")["detail"])
|
||||
self.reg.update_device(d["id"], user="nobody")
|
||||
self.hosts({"localhost": "denied"})
|
||||
link.connect(["switch"])
|
||||
s = link.snapshot()
|
||||
self.assertEqual((s["phase"], s["error"]["stage"], s["error"].get("auth")), ("failed", "login", "denied"))
|
||||
self.assertTrue(seen and all(isinstance(x, int) for x in seen)) # a file descriptor, never PIPE
|
||||
self.assertNotIn(subprocess.PIPE, seen)
|
||||
for _ in range(50): # the reader removes its file once ssh has exited
|
||||
if not list(tmp.iterdir()):
|
||||
break
|
||||
time.sleep(0.05)
|
||||
self.assertEqual(list(tmp.iterdir()), [])
|
||||
|
||||
def test_a_changed_key_is_shown_never_accepted_and_trusted_on_request(self):
|
||||
# 0.4.2 on Linux and Windows: "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!", with
|
||||
# "Run Set Up Connection again" as the advice, which can't replace the saved key.
|
||||
self.pins()
|
||||
d = self.device("localhost")
|
||||
self.hosts({"localhost": "ok"})
|
||||
self.link.connect(["start"])
|
||||
self.assertEqual(self.link.snapshot()["phase"], "connected")
|
||||
saved = fd.known_hosts(d["id"]).read_text()
|
||||
old = fd.pinned_keys(d["id"])
|
||||
self.link.close_master()
|
||||
self.hosts({"localhost": "wrong"}) # SteamOS reinstalled: a new host key
|
||||
self.link.connect(["dropped"])
|
||||
s = self.link.snapshot()
|
||||
self.assertEqual((s["phase"], s["error"]["stage"]), ("failed", "identity"))
|
||||
shown = s["error"]["hostkey"]
|
||||
self.assertEqual((shown["state"], shown["type"], shown["host"]), ("changed", "ssh-ed25519", "localhost"))
|
||||
self.assertTrue(shown["fingerprint"].startswith("SHA256:"))
|
||||
self.assertIn("Devices tab", s["error"]["message"])
|
||||
self.assertEqual(fd.known_hosts(d["id"]).read_text(), saved) # never accepted on its own
|
||||
r = self.command(d)
|
||||
self.assertEqual(r.returncode, 255)
|
||||
self.assertIn("REMOTE HOST IDENTIFICATION HAS CHANGED", r.stderr) # nor by any command
|
||||
# Trusting needs the key read first, and exactly the fingerprint shown.
|
||||
with self.assertRaises(fd.DeviceError):
|
||||
fl.devices_action(self.link, {"action": "identity-trust", "id": d["id"],
|
||||
"fingerprint": shown["fingerprint"]}, None)
|
||||
out = fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)
|
||||
scan = out["scan"]
|
||||
self.assertEqual(scan["fingerprint"], shown["fingerprint"])
|
||||
self.assertFalse(scan["matches"])
|
||||
self.assertEqual(scan["saved"], [{"type": t, "fingerprint": f} for t, f in old])
|
||||
self.assertEqual(fd.known_hosts(d["id"]).read_text(), saved) # reading it saves nothing
|
||||
self.assertEqual(sorted(p.name for p in fd.known_hosts(d["id"]).parent.iterdir()), [d["id"]]) # no scan files
|
||||
call = [c for c in self.calls() if "PreferredAuthentications=none" in c][-1] # no login offered
|
||||
self.assertLess(call.index("StrictHostKeyChecking=accept-new"), call.index("StrictHostKeyChecking=yes"))
|
||||
self.assertNotIn(f"UserKnownHostsFile={fd.known_hosts_opt(d['id'])}",
|
||||
call[:call.index("StrictHostKeyChecking=yes")]) # a throwaway file, first
|
||||
with self.assertRaises(fd.DeviceError):
|
||||
fl.devices_action(self.link, {"action": "identity-trust", "id": d["id"], "fingerprint": "SHA256:other"}, None)
|
||||
self.link.scans[d["id"]]["at"] -= fl.SCAN_TTL + 1
|
||||
with self.assertRaises(fd.DeviceError): # shown too long ago
|
||||
fl.devices_action(self.link, {"action": "identity-trust", "id": d["id"],
|
||||
"fingerprint": shown["fingerprint"]}, None)
|
||||
fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)
|
||||
with mock.patch.object(self.link, "kick") as kick:
|
||||
out = fl.devices_action(self.link, {"action": "identity-trust", "id": d["id"],
|
||||
"fingerprint": shown["fingerprint"]}, None)
|
||||
kick.assert_called_with("switch")
|
||||
self.assertEqual(fd.pinned_keys(d["id"]), [("ssh-ed25519", shown["fingerprint"])]) # the old key is gone
|
||||
self.assertEqual(next(x for x in out["devices"] if x["id"] == d["id"])["keys"],
|
||||
[{"type": "ssh-ed25519", "fingerprint": shown["fingerprint"]}])
|
||||
self.link.connect(["switch"])
|
||||
self.assertEqual(self.link.snapshot()["phase"], "connected", self.link.snapshot()["error"])
|
||||
out = fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)
|
||||
self.assertTrue(out["scan"]["matches"])
|
||||
|
||||
def test_test_now_never_saves_a_key(self):
|
||||
self.pins()
|
||||
d = self.device("127.0.0.1")
|
||||
self.hosts({"127.0.0.1": "ok"})
|
||||
self.link.test(d["id"])
|
||||
rows = self.link.snapshot()["tests"][d["id"]]["rows"]
|
||||
self.assertEqual(rows[0]["ssh"], "unpinned")
|
||||
self.assertFalse(fd.pinned(d["id"]))
|
||||
|
||||
def test_reading_a_key_from_a_headset_that_doesnt_answer_says_why(self):
|
||||
self.pins()
|
||||
d = self.device("nothing.invalid")
|
||||
with self.assertRaises(fd.DeviceError) as caught:
|
||||
fl.devices_action(self.link, {"action": "identity-scan", "id": d["id"]}, None)
|
||||
self.assertIn("Can't find the Frame", str(caught.exception))
|
||||
self.assertEqual([p.name for p in fd.known_hosts(d["id"]).parent.iterdir()], [])
|
||||
|
||||
def test_ssh_goes_to_the_ipv4_address_that_answered(self):
|
||||
self.device("localhost")
|
||||
self.hosts({"localhost": "ok"})
|
||||
@@ -501,7 +924,7 @@ class Connecting(unittest.TestCase):
|
||||
{"action": "update", "id": d["id"], "port": 2222},
|
||||
{"action": "address-remove", "id": d["id"], "host": "localhost"},
|
||||
{"action": "address-update", "id": d["id"], "host": "localhost", "newHost": "127.0.0.1"},
|
||||
{"action": "forget-identity", "id": d["id"]}):
|
||||
{"action": "identity-trust", "id": d["id"], "fingerprint": "SHA256:x"}):
|
||||
with self.assertRaises(fd.DeviceError, msg=body):
|
||||
fl.devices_action(self.link, body, open_setup=None, busy=lambda: 1)
|
||||
# Renaming, or changing another headset, is fine.
|
||||
|
||||
@@ -7,6 +7,7 @@ input reaching the source, and closing.
|
||||
|
||||
Run: python3 -m unittest discover -s tests
|
||||
"""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import base64
|
||||
import http.client
|
||||
import json
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
"""Owned media planning, eye isolation, decoder choice and fake-Frame ownership."""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import argparse
|
||||
import io
|
||||
import json
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
"""Fake-Frame panel responses and the headset page's real HTTP guards."""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import http.client
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT / "ui"))
|
||||
import frame_connect # noqa: E402
|
||||
import frame_devices # noqa: E402
|
||||
import frame_host # noqa: E402
|
||||
|
||||
|
||||
def inside(path, folder):
|
||||
path, folder = os.path.realpath(path), os.path.realpath(folder)
|
||||
return path == folder or path.startswith(folder + os.sep)
|
||||
|
||||
|
||||
def snapshot(folder):
|
||||
"""{relative path: (size, mtime)} for everything under FOLDER."""
|
||||
out = {}
|
||||
for base, dirs, files in os.walk(folder):
|
||||
for name in dirs + files:
|
||||
p = os.path.join(base, name)
|
||||
try:
|
||||
st = os.lstat(p)
|
||||
except OSError:
|
||||
continue
|
||||
out[os.path.relpath(p, folder)] = (st.st_size, st.st_mtime_ns)
|
||||
return out
|
||||
|
||||
|
||||
class Sandbox(unittest.TestCase):
|
||||
"""Tests once wrote a host-key pin into the real ~/.ssh/frame-control-hosts on every
|
||||
run (a server started without FRAME_CONTROL_SSH_DIR imported the real ~/.ssh/config)."""
|
||||
|
||||
def test_home_and_ssh_dir_are_throwaway(self):
|
||||
tmp = tempfile.gettempdir()
|
||||
self.assertTrue(inside(Path.home(), tmp), Path.home())
|
||||
self.assertTrue(inside(os.path.expanduser("~"), tmp))
|
||||
for d in (frame_devices.ssh_dir(), frame_devices.pin_dir(), frame_connect.SSH_DIR,
|
||||
os.environ["FRAME_CONTROL_SSH_DIR"], frame_host.cache_dir(), frame_host.data_dir()):
|
||||
self.assertTrue(inside(d, tmp), d)
|
||||
self.assertFalse(inside(d, sandbox.REAL_SSH_DIR), d)
|
||||
|
||||
def test_cache_folders_on_every_platform_are_throwaway(self):
|
||||
"""cache_dir() reads LOCALAPPDATA on Windows and XDG_CACHE_HOME on Linux, which a
|
||||
moved HOME alone doesn't cover: servers sweep staging folders there."""
|
||||
tmp = tempfile.gettempdir()
|
||||
for platform in ("MAC", "WINDOWS", None):
|
||||
with mock.patch.object(frame_host, "MAC", platform == "MAC"), \
|
||||
mock.patch.object(frame_host, "WINDOWS", platform == "WINDOWS"):
|
||||
self.assertTrue(inside(frame_host.cache_dir("apk-sources"), tmp), platform)
|
||||
for name in ("LOCALAPPDATA", "APPDATA", "XDG_CACHE_HOME", "XDG_DATA_HOME", "XDG_STATE_HOME"):
|
||||
self.assertTrue(inside(os.environ[name], tmp), name)
|
||||
|
||||
def test_end_to_end_tests_stay_out_of_it(self):
|
||||
"""tests/e2e keeps the container's real home (OpenSSH reads ~/.ssh from the password
|
||||
database, not $HOME), so nothing there may import sandbox, even through a test_* module."""
|
||||
here = Path(__file__).resolve().parent
|
||||
pattern = re.compile(r"^\s*(import sandbox|from sandbox |import test_|from test_)", re.M)
|
||||
for f in [*(here / "e2e").glob("*.py"), *(here / "smoke").glob("*.py"), here / "apk_fixtures.py"]:
|
||||
self.assertIsNone(pattern.search(f.read_text(encoding="utf-8")), f)
|
||||
|
||||
def test_end_to_end_tests_start_no_server_outside_their_container(self):
|
||||
"""Plain discovery over tests/e2e, with or without FRAME_E2E=1, must skip everything
|
||||
and never start ui/server.py: outside scripts/e2e.sh's container it would run on
|
||||
this computer's real app data, ~/.ssh and cache."""
|
||||
e2e = str(Path(__file__).resolve().parent / "e2e")
|
||||
code = ("import io, sys, unittest; sys.path.insert(0, sys.argv[1]); import harness; calls = []\n"
|
||||
"harness.Server.start = classmethod(lambda cls: calls.append(cls))\n"
|
||||
"suite = unittest.defaultTestLoader.discover(sys.argv[1], top_level_dir=sys.argv[1])\n"
|
||||
"r = unittest.TextTestRunner(stream=io.StringIO()).run(suite)\n"
|
||||
"print(len(calls), r.testsRun, len(r.skipped), len(r.errors) + len(r.failures))")
|
||||
for extra in ({}, {"FRAME_E2E": "1"}):
|
||||
env = {k: v for k, v in os.environ.items() if not k.startswith("FRAME_E2E")}
|
||||
env.update(extra, PYTHONDONTWRITEBYTECODE="1")
|
||||
out = subprocess.run([sys.executable, "-c", code, e2e], env=env, capture_output=True, text=True,
|
||||
timeout=120)
|
||||
self.assertEqual(out.returncode, 0, out.stderr[-2000:])
|
||||
started, ran, skipped, broken = map(int, out.stdout.split())
|
||||
self.assertEqual((started, broken), (0, 0), (extra, out.stdout))
|
||||
self.assertEqual(ran, skipped, (extra, out.stdout))
|
||||
self.assertGreater(skipped, 0, extra)
|
||||
|
||||
def test_child_processes_inherit_it(self):
|
||||
out = subprocess.run(
|
||||
[sys.executable, "-c", "import sys; sys.path.insert(0, sys.argv[1]); import frame_devices, frame_connect;"
|
||||
" from pathlib import Path; print(Path.home()); print(frame_devices.ssh_dir()); print(frame_connect.SSH_DIR)",
|
||||
str(ROOT / "ui")], env={**os.environ, "PYTHONDONTWRITEBYTECODE": "1"},
|
||||
capture_output=True, text=True, check=True).stdout.split("\n")
|
||||
for d in filter(None, out):
|
||||
self.assertFalse(inside(d, sandbox.REAL_SSH_DIR), d)
|
||||
self.assertTrue(inside(d, tempfile.gettempdir()), d)
|
||||
|
||||
def test_real_ssh_folder_untouched_by_tests_that_start_servers(self):
|
||||
"""Reruns, from the environment the suite was started in, the tests that used to
|
||||
leak (servers started without FRAME_CONTROL_SSH_DIR) plus the device and connect
|
||||
tests, and checks nothing under the real ~/.ssh changed."""
|
||||
real = sandbox.REAL_SSH_DIR
|
||||
if inside(real, tempfile.gettempdir()):
|
||||
self.skipTest("this run's HOME is already a temporary folder")
|
||||
before = snapshot(real)
|
||||
env = {k: v for k, v in sandbox.ORIGINAL_ENV.items() if k != "FRAME_CONTROL_SSH_DIR"}
|
||||
env["PYTHONDONTWRITEBYTECODE"] = "1"
|
||||
r = subprocess.run([sys.executable, "-m", "unittest", "-q", "test_server.OneServer", "test_server.LocalMode",
|
||||
"test_devices", "test_connect"],
|
||||
cwd=str(Path(__file__).resolve().parent), env=env, capture_output=True, text=True,
|
||||
timeout=300)
|
||||
self.assertEqual(r.returncode, 0, r.stderr[-3000:])
|
||||
after = snapshot(real)
|
||||
changed = sorted(k for k in set(before) | set(after) if before.get(k) != after.get(k)
|
||||
and not k.startswith("agent")) # ssh-agent sockets come and go on their own
|
||||
self.assertEqual(changed, [], f"tests changed the real {real}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -535,7 +535,11 @@ class UnreachableMessages(unittest.TestCase):
|
||||
"ssh: connect to host 192.168.1.9 port 22: No route to host": "isn't answering",
|
||||
"ssh: connect to host 192.168.1.9 port 22: Connection refused": "refused",
|
||||
"steamos@192.168.1.9: Permission denied (publickey,password).": "SSH key",
|
||||
"Host key verification failed.": "identity changed",
|
||||
"Host key verification failed.": "identity has changed",
|
||||
"@@@@\n@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @\nHost key verification failed.":
|
||||
"Check the new key on the Devices tab",
|
||||
"No ED25519 host key is known for frame-control-f7b4e9ac and you have requested strict checking.\n"
|
||||
"Host key verification failed.": "isn't saved yet",
|
||||
"kex_exchange_identification: read: Connection reset by peer": "dropped",
|
||||
"Timed out talking to frame": "too long",
|
||||
}
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
@@ -335,6 +335,26 @@ class LinkFailureProvenance(Base):
|
||||
tm.diagnostic("job steam", e)
|
||||
self.assertEqual([p["properties"]["error_category"] for p in self.queued()], ["frame_unreachable"])
|
||||
|
||||
def test_host_key_and_login_refusals_are_sent_once(self):
|
||||
# 0.4.2: each of these came in as a $exception from the status poll, again every 10 minutes.
|
||||
for _ in range(3):
|
||||
for stderr in ("@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@\r\n"
|
||||
"@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @\r\n"
|
||||
"IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!\r\nHost key verification failed.\r\n",
|
||||
"No ED25519 host key is known for frame-control-f7b4e9ac and you have requested strict "
|
||||
"checking.\r\nHost key verification failed.\r\n",
|
||||
"jane doe@frame-control-f7b4e9ac: Permission denied (publickey,password).\r\n",
|
||||
"steamos@192.168.1.9: Permission denied (publickey,password).\n"):
|
||||
e = self.ssh_fails(self.server, 255, stderr)
|
||||
self.assertTrue(getattr(e, "frame_link_failed", False), stderr)
|
||||
tm.diagnostic("POST /api/comfort status", e)
|
||||
self.assertEqual([p["properties"]["error_category"] for p in self.queued()], ["frame_auth"])
|
||||
# A command on the Frame that was refused something isn't ssh refusing the headset.
|
||||
for code, stderr in ((1, "steamos@frame: Permission denied (publickey).\n"),
|
||||
(255, "cat: /root/x: Permission denied\n"),
|
||||
(255, "scp: /home/steamos/x: Permission denied\n")):
|
||||
self.assertFalse(getattr(self.ssh_fails(self.server, code, stderr), "frame_link_failed", False), stderr)
|
||||
|
||||
def test_a_command_that_fails_on_the_frame_is_not_marked(self):
|
||||
self.poll_fails()
|
||||
for module in (self.server, self.android):
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
Run: python3 -m unittest discover -s tests
|
||||
"""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
"""Own VR telemetry, optional catalogue and entitlement guards; no headset needed."""
|
||||
import sandbox # noqa: F401 (first: keeps tests off real data and services)
|
||||
import ctypes
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
@@ -32,7 +32,9 @@ Entry dict (missing facts are None, never guessed)
|
||||
Rules
|
||||
-----
|
||||
- Only sources that distribute APKs with the developer's consent: free listings,
|
||||
never paid apps re-hosted, no licence or entitlement workarounds.
|
||||
never paid apps re-hosted, no licence or entitlement workarounds. The one paid
|
||||
exception is a store's own download of a game the user bought there: a source
|
||||
marked 'owned': True whose entries carry 'owned': True (itch.io purchases).
|
||||
- Honour each site's terms and robots rules; if automated download isn't allowed,
|
||||
return entries with 'downloadable': False and a 'page' link instead.
|
||||
- Cache indexes under frame_host.cache_dir('apk-sources'); send a clear
|
||||
|
||||
@@ -0,0 +1,394 @@
|
||||
"""The user's own itch.io account: an API key, the games it owns and their Android uploads.
|
||||
|
||||
Opt-in (store > sources > Connect itch.io). The key is kept in
|
||||
frame_host.data_dir('itch-account.json') with mode 0600 and is only ever sent to
|
||||
https://api.itch.io, in an Authorization header. Downloads end on itch.io's storage
|
||||
host, fetched without the key from a checked public address (_web.public_apk).
|
||||
The key never goes into a URL, an error, a log line or an API response. Owned-key
|
||||
listings stay in memory, never in the disk cache.
|
||||
API: https://itch.io/docs/api/serverside. See docs/apk-repos.md.
|
||||
"""
|
||||
import hashlib, http.client, json, os, re, tempfile, threading, time, urllib.error, urllib.parse, urllib.request, zipfile
|
||||
|
||||
import frame_host
|
||||
from . import SourceError, SourceLimited, _web
|
||||
|
||||
API = 'https://api.itch.io'
|
||||
HOSTS = ('api.itch.io',)
|
||||
NAME = 'itch.io'
|
||||
TTL = 600 # owned keys and uploads are re-read after ten minutes
|
||||
PER_PAGE = 500 # the documented maximum for /profile/owned-keys
|
||||
PAGES = 40 # stop after 20,000 owned keys
|
||||
KEY = re.compile(r'[A-Za-z0-9_.-]{16,256}')
|
||||
BAD_KEY = ('itch.io did not accept the API key. Create one in itch.io settings > '
|
||||
'API keys and connect again')
|
||||
REVOKED = ('itch.io no longer accepts the saved API key, so it was removed. '
|
||||
'Create a new one in itch.io settings > API keys and connect again')
|
||||
REDIRECTS = (301, 302, 303, 307, 308)
|
||||
# Everything a connection or response can raise. http.client's errors (IncompleteRead,
|
||||
# BadStatusLine...) are not OSErrors, and BadStatusLine repeats what the server sent.
|
||||
NETWORK = (OSError, ValueError, http.client.HTTPException)
|
||||
_lock = threading.Lock() # the cache
|
||||
_cache = {} # (generation, key digest, what) -> (time, value)
|
||||
_account = threading.Lock() # saving and deleting the key file
|
||||
_generation = [0] # bumped by every save and delete: a connect that started earlier doesn't save
|
||||
|
||||
|
||||
class BadKey(SourceError):
|
||||
"""itch.io refused the key: wrong, revoked or deleted."""
|
||||
|
||||
|
||||
def path():
|
||||
return frame_host.data_dir('itch-account.json')
|
||||
|
||||
|
||||
def _load():
|
||||
try:
|
||||
data = json.loads(path().read_text())
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
if not isinstance(data, dict) or not isinstance(data.get('key'), str) or not KEY.fullmatch(data['key']):
|
||||
return None
|
||||
return data
|
||||
|
||||
|
||||
def status():
|
||||
"""{'connected', 'username'}: what the page may know. Never the key; no network."""
|
||||
data = _load()
|
||||
return {'connected': bool(data), 'username': data.get('username') if data else None}
|
||||
|
||||
|
||||
def connected():
|
||||
return _load() is not None
|
||||
|
||||
|
||||
def _key():
|
||||
data = _load()
|
||||
if not data:
|
||||
raise SourceError('Connect your itch.io account first (store > sources > Connect itch.io)')
|
||||
return data['key']
|
||||
|
||||
|
||||
def _save(key, username):
|
||||
"""Write the key file through a fresh 0600 temporary file (mkstemp), then move it into place."""
|
||||
target = path()
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
fd, tmp = tempfile.mkstemp(dir=str(target.parent), prefix='.itch-account-', suffix='.tmp')
|
||||
try:
|
||||
with os.fdopen(fd, 'w') as f:
|
||||
os.chmod(tmp, 0o600) # mkstemp already does on POSIX; say so for every platform
|
||||
json.dump({'key': key, 'username': username}, f)
|
||||
os.replace(tmp, str(target))
|
||||
finally:
|
||||
if os.path.exists(tmp):
|
||||
os.remove(tmp)
|
||||
|
||||
|
||||
def _forget():
|
||||
with _lock:
|
||||
_cache.clear()
|
||||
|
||||
|
||||
def _clean(message, key):
|
||||
"""message without the key, or any start of it (8+ characters, as truncation leaves), in
|
||||
any letter case, with any of its characters written literally, percent-encoded (%4B or
|
||||
%4b) or percent-encoded twice (%254B, %254b). The only redaction used here: every message
|
||||
that might hold the key goes through it. (The key's characters are letters, digits and
|
||||
_.-, so +/space encoding can't change it.)"""
|
||||
if not key:
|
||||
return message
|
||||
# Each character: itself, or the percent-encoding (once or twice) of either of its letter cases.
|
||||
tokens = ['(?:%s)' % '|'.join([re.escape(c)] + ['%%(?:25)?%02x' % ord(v) for v in sorted({c.lower(), c.upper()})])
|
||||
for c in key]
|
||||
start, rest = re.compile(''.join(tokens[:8]), re.I), [re.compile(t, re.I) for t in tokens[8:]]
|
||||
spans, pos = [], 0
|
||||
while True:
|
||||
found = start.search(message, pos)
|
||||
if not found:
|
||||
break
|
||||
end = found.end()
|
||||
for token in rest: # as much more of the key as follows
|
||||
more = token.match(message, end)
|
||||
if not more:
|
||||
break
|
||||
end = more.end()
|
||||
spans.append((found.start(), end))
|
||||
# Look again from the next character, not the end: a partial match just before the
|
||||
# real key can overlap its start. Overlapping spans merge below.
|
||||
pos = found.start() + 1
|
||||
out, pos = [], 0
|
||||
for a, b in spans: # in order of start; only overlapping spans merge
|
||||
if out and a < pos:
|
||||
pos = max(pos, b)
|
||||
continue
|
||||
out += [message[pos:a], '[API key]']
|
||||
pos = b
|
||||
return ''.join(out) + message[pos:]
|
||||
|
||||
|
||||
class _Stay(urllib.request.HTTPRedirectHandler):
|
||||
"""Don't follow: the caller decides where a redirect may go, and never sends the key there."""
|
||||
|
||||
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
||||
return None
|
||||
|
||||
|
||||
def _open(url, key, accept='application/json'):
|
||||
"""GET url on api.itch.io with the key. Redirects come back as HTTPError 30x."""
|
||||
_web.checked_url(url, HOSTS)
|
||||
wait = _web.wait_time(url)
|
||||
if wait:
|
||||
raise _web.limited_error(NAME, wait)
|
||||
return urllib.request.build_opener(_Stay()).open(urllib.request.Request(
|
||||
url, headers={'User-Agent': _web.UA, 'Authorization': 'Bearer ' + key, 'Accept': accept}), timeout=60)
|
||||
|
||||
|
||||
def _refused(code, key, api_path, message=None):
|
||||
"""A refusal that may be about the key (401/403, or an "invalid key" error): BadKey only
|
||||
once /profile confirms the key itself is refused; otherwise it's this request (a game or
|
||||
upload this purchase doesn't cover), and the key stays."""
|
||||
if api_path != '/profile':
|
||||
_get('/profile', key) # raises BadKey if the key is the problem
|
||||
raise SourceError(message or 'itch.io refused this request (HTTP %d); your purchase may not include it' % code)
|
||||
raise BadKey(BAD_KEY)
|
||||
|
||||
|
||||
def _network_error(e, key):
|
||||
reason = getattr(e, 'reason', None) or e
|
||||
return SourceError(_clean('Could not reach itch.io: ' + str(reason), key))
|
||||
|
||||
|
||||
BAD_KEY_TEXT = re.compile(r'\b(invalid|revoked|expired)\b.*\bkey\b|\bkey\b.*\b(invalid|revoked|expired)\b', re.I)
|
||||
|
||||
|
||||
def _checked(raw, key, api_path):
|
||||
"""Decoded JSON object from api.itch.io, or SourceError/BadKey from its errors array."""
|
||||
if len(raw) > 8 * 1024 * 1024:
|
||||
raise SourceError('itch.io response is too large')
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
except ValueError:
|
||||
raise SourceError('itch.io returned an invalid response') from None
|
||||
if not isinstance(data, dict):
|
||||
raise SourceError('itch.io returned an invalid response')
|
||||
errors = data.get('errors')
|
||||
if errors: # documented: errors is an array of strings
|
||||
messages = [str(m) for m in errors if isinstance(m, (str, int))]
|
||||
# Judged on itch.io's own words: redacted text says "[API key]", which would look like a key error.
|
||||
about_key = any(BAD_KEY_TEXT.search(m) and 'download key' not in m.lower() for m in messages)
|
||||
# Each message loses the key before joining and shortening, so no part of it survives the cut.
|
||||
text = _clean('; '.join(_clean(m, key) for m in messages)[:200], key)
|
||||
if about_key:
|
||||
_refused(None, key, api_path, 'itch.io: ' + text)
|
||||
raise SourceError('itch.io: ' + (text or 'request refused'))
|
||||
return data
|
||||
|
||||
|
||||
def _get(api_path, key, params=None):
|
||||
"""JSON from api.itch.io; SourceError with a readable message (never the key) on failure."""
|
||||
url = API + api_path + ('?' + urllib.parse.urlencode(params) if params else '')
|
||||
try:
|
||||
with _open(url, key) as r:
|
||||
raw = r.read(8 * 1024 * 1024 + 1)
|
||||
except urllib.error.HTTPError as e:
|
||||
e.close()
|
||||
if e.code == 429:
|
||||
raise _web.limited_error(NAME, _web.throttle(url, e.headers)) from None
|
||||
if e.code in (401, 403):
|
||||
_refused(e.code, key, api_path)
|
||||
raise SourceError('itch.io returned HTTP error %d; try again later' % e.code) from None
|
||||
except SourceError:
|
||||
raise
|
||||
except NETWORK as e:
|
||||
raise _network_error(e, key) from None
|
||||
return _checked(raw, key, api_path)
|
||||
|
||||
|
||||
def _cached(key, what, load):
|
||||
slot = (_generation[0], hashlib.sha256(key.encode()).hexdigest(), what)
|
||||
with _lock:
|
||||
hit = _cache.get(slot)
|
||||
if hit and time.time() - hit[0] < TTL:
|
||||
return hit[1]
|
||||
value = load()
|
||||
with _lock:
|
||||
_cache[slot] = (time.time(), value)
|
||||
return value
|
||||
|
||||
|
||||
def _use(call):
|
||||
"""Run call(key); a key itch.io now refuses is removed, so the account shows disconnected."""
|
||||
with _account: # the key and the connection it belongs to, read together
|
||||
key, generation = _key(), _generation[0]
|
||||
try:
|
||||
return call(key)
|
||||
except BadKey:
|
||||
with _account:
|
||||
# Only the connection this call started with: a reconnect meanwhile (even with
|
||||
# the same key) has been checked since, and is not undone by this older verdict.
|
||||
if _generation[0] == generation and (_load() or {}).get('key') == key:
|
||||
_delete()
|
||||
raise BadKey(REVOKED) from None
|
||||
|
||||
|
||||
def connect(key):
|
||||
"""Check key with itch.io (GET /profile), then save it. A refused key is never saved, and
|
||||
nor is one whose check finished after a disconnect or another connect."""
|
||||
key = (key or '').strip() if isinstance(key, str) else ''
|
||||
if not KEY.fullmatch(key):
|
||||
raise BadKey('That does not look like an itch.io API key. Copy it from itch.io settings > API keys')
|
||||
with _account:
|
||||
started = _generation[0]
|
||||
user = _get('/profile', key).get('user') # the network, outside the lock
|
||||
if not isinstance(user, dict):
|
||||
raise SourceError('itch.io returned an invalid profile')
|
||||
username = user.get('username') if isinstance(user.get('username'), str) else None
|
||||
with _account:
|
||||
if _generation[0] != started:
|
||||
raise SourceError('The itch.io connection changed while the key was being checked; '
|
||||
'nothing was saved. Try again')
|
||||
_generation[0] += 1
|
||||
_save(key, username)
|
||||
_forget()
|
||||
return {'message': 'Connected to itch.io' + (' as ' + username if username else ''), 'itch': status()}
|
||||
|
||||
|
||||
def _delete():
|
||||
"""Remove the key file; call with _account held."""
|
||||
_generation[0] += 1
|
||||
try:
|
||||
os.remove(str(path()))
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
_forget()
|
||||
|
||||
|
||||
def disconnect():
|
||||
with _account:
|
||||
_delete()
|
||||
return {'message': 'Disconnected from itch.io; the API key was deleted from this computer', 'itch': status()}
|
||||
|
||||
|
||||
def owned_keys():
|
||||
"""Every download key the account owns, newest first (GET /profile/owned-keys, all pages)."""
|
||||
def load(key):
|
||||
keys = []
|
||||
for page in range(1, PAGES + 1):
|
||||
batch = _get('/profile/owned-keys', key, {'page': page, 'per_page': PER_PAGE}).get('owned_keys')
|
||||
# Documented: a page can hold fewer than per_page even when more follow; an empty one ends it.
|
||||
if not batch or not isinstance(batch, list):
|
||||
break
|
||||
keys.extend(k for k in batch if isinstance(k, dict))
|
||||
return keys
|
||||
return _use(lambda key: _cached(key, 'owned', lambda: load(key)))
|
||||
|
||||
|
||||
def _number(value):
|
||||
return value if type(value) is int and value > 0 else None
|
||||
|
||||
|
||||
def is_android(upload):
|
||||
"""An upload marked for Android, or an .apk file. The marker's field name is not in the
|
||||
public reference: itch.io's traits use p_<platform> (p_windows...), so p_android is assumed."""
|
||||
traits = upload.get('traits') if isinstance(upload.get('traits'), list) else []
|
||||
platforms = upload.get('platforms') if isinstance(upload.get('platforms'), dict) else {}
|
||||
filename = str(upload.get('filename') or '').lower()
|
||||
return 'p_android' in traits or upload.get('p_android') is True or 'android' in platforms \
|
||||
or filename.endswith('.apk')
|
||||
|
||||
|
||||
def android_uploads(uploads):
|
||||
"""Installable Android uploads, best first: .apk files, then full games, then newest."""
|
||||
chosen = []
|
||||
for u in uploads if isinstance(uploads, list) else []:
|
||||
if not isinstance(u, dict) or not _number(u.get('id')) or not is_android(u):
|
||||
continue
|
||||
traits = u.get('traits') if isinstance(u.get('traits'), list) else []
|
||||
if u.get('type') not in (None, 'default') or u.get('preorder') is True or 'preorder' in traits:
|
||||
continue # soundtracks, books, placeholders
|
||||
if u.get('storage') == 'external':
|
||||
continue # hosted on another site: not something itch.io can hand us
|
||||
chosen.append(u)
|
||||
chosen.sort(key=lambda u: str(u.get('updated_at') or u.get('created_at') or ''), reverse=True)
|
||||
chosen.sort(key=lambda u: (not str(u.get('filename') or '').lower().endswith('.apk'),
|
||||
u.get('demo') is True or 'demo' in (u.get('traits') or [])))
|
||||
return chosen
|
||||
|
||||
|
||||
def uploads(game_id, download_key_id):
|
||||
"""The game's uploads this download key may fetch (GET /games/ID/uploads)."""
|
||||
game_id, download_key_id = _number(game_id), _number(download_key_id)
|
||||
if not game_id or not download_key_id:
|
||||
raise SourceError('Invalid itch.io game')
|
||||
return _use(lambda key: _cached(key, ('uploads', game_id, download_key_id), lambda: _get(
|
||||
'/games/%d/uploads' % game_id, key, {'download_key_id': download_key_id}).get('uploads') or []))
|
||||
|
||||
|
||||
def download_url(upload_id, download_key_id):
|
||||
upload_id, download_key_id = _number(upload_id), _number(download_key_id)
|
||||
if not upload_id or not download_key_id:
|
||||
raise SourceError('Invalid itch.io upload')
|
||||
return API + '/uploads/%d/download?' % upload_id + urllib.parse.urlencode({'download_key_id': download_key_id})
|
||||
|
||||
|
||||
def _fetch(url, key):
|
||||
"""Ask api.itch.io for the file. Not checked against a live account, so this accepts each
|
||||
plausible answer: a redirect to storage, JSON naming the storage URL, or the file itself."""
|
||||
try:
|
||||
r = _open(url, key, 'application/vnd.android.package-archive, application/json;q=0.5, */*;q=0.1')
|
||||
except urllib.error.HTTPError as e:
|
||||
e.close()
|
||||
if e.code in REDIRECTS:
|
||||
target = e.headers.get('Location')
|
||||
elif e.code == 429:
|
||||
raise _web.limited_error(NAME, _web.throttle(url, e.headers)) from None
|
||||
elif e.code in (401, 403):
|
||||
_refused(e.code, key, '/uploads')
|
||||
else:
|
||||
raise SourceError('itch.io returned HTTP error %d for the download; try again later' % e.code) from None
|
||||
except SourceError:
|
||||
raise
|
||||
except NETWORK as e:
|
||||
raise _network_error(e, key) from None
|
||||
else:
|
||||
with r:
|
||||
if 'json' not in (r.headers.get('Content-Type') or '').lower():
|
||||
try:
|
||||
return _web.save_apk(r) # served by api.itch.io itself
|
||||
except NETWORK + (zipfile.BadZipFile,) as e:
|
||||
raise SourceError(_clean('Could not download APK: ' + str(e), key)) from None
|
||||
try:
|
||||
raw = r.read(1024 * 1024 + 1)
|
||||
except NETWORK as e:
|
||||
raise _network_error(e, key) from None
|
||||
target = _checked(raw, key, '/uploads').get('url')
|
||||
if not isinstance(target, str) or not target:
|
||||
raise SourceError('itch.io did not say where to download the file')
|
||||
def refuse(hop): # on the first URL and on every redirect after it, before any lookup
|
||||
if _carries_key(hop, key):
|
||||
raise SourceError('itch.io pointed the download at a URL carrying your API key; refused, '
|
||||
'so the key is not sent to the storage server')
|
||||
try:
|
||||
return _web.public_apk(urllib.parse.urljoin(url, target), NAME, check=refuse) # no key sent
|
||||
except SourceLimited as e:
|
||||
raise SourceLimited(_clean(str(e), key), e.retry_after) from None
|
||||
except SourceError as e:
|
||||
raise type(e)(_clean(str(e), key)) from None
|
||||
|
||||
|
||||
def _carries_key(url, key):
|
||||
"""True if url holds the key (or 8+ characters of its start), however cased or encoded, or
|
||||
an itch.io API-key parameter: go-itchio puts the key in ?api_key=, and storage must never
|
||||
get it. Other parameters (signatures, tokens) are the storage URL's own and are kept."""
|
||||
for form in (url, urllib.parse.unquote(url), urllib.parse.unquote_plus(url)):
|
||||
if _clean(form, key) != form:
|
||||
return True
|
||||
names = {name.lower() for name, _ in urllib.parse.parse_qsl(urllib.parse.urlsplit(url).query,
|
||||
keep_blank_values=True)}
|
||||
return bool(names & {'api_key', 'apikey', 'api-key'})
|
||||
|
||||
|
||||
def download(upload_id, download_key_id):
|
||||
"""The APK: the key goes to api.itch.io only, never to the storage host."""
|
||||
url = download_url(upload_id, download_key_id)
|
||||
return _use(lambda key: _fetch(url, key))
|
||||
+161
-11
@@ -1,5 +1,6 @@
|
||||
"""Small HTTPS cache and APK downloader for public publisher sources."""
|
||||
import hashlib, os, shutil, tempfile, threading, time, urllib.error, urllib.parse, urllib.request, zipfile
|
||||
import hashlib, http.client, ipaddress, os, shutil, socket, ssl, tempfile, threading, time
|
||||
import urllib.error, urllib.parse, urllib.request, zipfile
|
||||
from email.utils import parsedate_to_datetime
|
||||
|
||||
import frame_host
|
||||
@@ -136,6 +137,8 @@ def checked_url(url, hosts):
|
||||
|
||||
|
||||
class Redirect(urllib.request.HTTPRedirectHandler):
|
||||
"""Follow redirects only to hosts; an Authorization header never follows one."""
|
||||
|
||||
def __init__(self, hosts):
|
||||
self.hosts = hosts
|
||||
|
||||
@@ -196,19 +199,16 @@ def read(url, hosts, headers=None, ttl=3600, name=None, hint=''):
|
||||
raise SourceError('Could not read source: ' + str(e)) from e
|
||||
|
||||
|
||||
def apk(url, hosts, digest=None, name=None):
|
||||
def save_apk(response, digest=None):
|
||||
"""Stream response (anything with read(n)) into the cache as a checked APK."""
|
||||
tmp = None
|
||||
name = name or _host(url)
|
||||
wait = wait_time(url)
|
||||
if wait:
|
||||
raise limited_error(name, wait)
|
||||
try:
|
||||
fd, tmp = tempfile.mkstemp(dir=cache(), suffix='.part')
|
||||
h = hashlib.sha256()
|
||||
with os.fdopen(fd, 'wb') as f, open_url(url, hosts) as r:
|
||||
with os.fdopen(fd, 'wb') as f:
|
||||
size = 0
|
||||
while True:
|
||||
chunk = r.read(1 << 20)
|
||||
chunk = response.read(1 << 20)
|
||||
if not chunk:
|
||||
break
|
||||
size += len(chunk)
|
||||
@@ -225,12 +225,162 @@ def apk(url, hosts, digest=None, name=None):
|
||||
path = os.path.join(cache(), actual + '.apk')
|
||||
os.replace(tmp, path)
|
||||
return {'apk': path, 'obb': [], 'sha256': actual, 'verified': bool(digest)}
|
||||
finally:
|
||||
if tmp and os.path.exists(tmp):
|
||||
os.remove(tmp)
|
||||
|
||||
|
||||
def apk(url, hosts, digest=None, name=None):
|
||||
name = name or _host(url)
|
||||
wait = wait_time(url)
|
||||
if wait:
|
||||
raise limited_error(name, wait)
|
||||
try:
|
||||
with open_url(url, hosts) as r:
|
||||
return save_apk(r, digest)
|
||||
except urllib.error.HTTPError as e:
|
||||
if e.code in (403, 429):
|
||||
raise limited_error(name, throttle(url, e.headers)) from e
|
||||
raise SourceError('Could not download APK: HTTP error ' + str(e.code)) from e
|
||||
except (OSError, ValueError, zipfile.BadZipFile) as e:
|
||||
raise SourceError('Could not download APK: ' + str(e)) from e
|
||||
finally:
|
||||
if tmp and os.path.exists(tmp):
|
||||
os.remove(tmp)
|
||||
|
||||
|
||||
# IPv6 is an allow-list: only global unicast (2000::/3), which rules out everything else at
|
||||
# once (loopback, unspecified, mapped/compatible IPv4, NAT64 64:ff9b::, site-local fec0::,
|
||||
# unique-local fc00::, link-local, multicast, reserved). Inside it, these special blocks are
|
||||
# refused too: 6to4 and Teredo carry IPv4 that may be private; the rest aren't the internet.
|
||||
_GLOBAL6 = ipaddress.ip_network('2000::/3')
|
||||
_SPECIAL6 = tuple(ipaddress.ip_network(n) for n in (
|
||||
'2001::/32', # Teredo
|
||||
'2001:2::/48', # benchmarking
|
||||
'2001:10::/28', # ORCHID
|
||||
'2001:20::/28', # ORCHIDv2
|
||||
'2001:db8::/32', # documentation
|
||||
'2002::/16', # 6to4
|
||||
'3fff::/20')) # documentation (2024)
|
||||
# IPv4 blocks that are never the public internet, listed in full rather than trusting
|
||||
# is_global (Python 3.9 calls 192.0.0.8, .11 and .255 global).
|
||||
_SPECIAL4 = tuple(ipaddress.ip_network(n) for n in (
|
||||
'0.0.0.0/8', '10.0.0.0/8', '100.64.0.0/10', '127.0.0.0/8', '169.254.0.0/16', '172.16.0.0/12',
|
||||
'192.0.0.0/24', '192.0.2.0/24', '192.88.99.0/24', '192.168.0.0/16', '198.18.0.0/15',
|
||||
'198.51.100.0/24', '203.0.113.0/24', '224.0.0.0/4', '240.0.0.0/4', '255.255.255.255/32'))
|
||||
|
||||
|
||||
def public_address(ip):
|
||||
"""True for an address on the public internet: never loopback, private, link-local,
|
||||
site-local, shared (CGNAT), reserved, unspecified or multicast, nor an IPv6 wrapper
|
||||
around IPv4. Python 3.9's is_global alone lets several of these through."""
|
||||
try:
|
||||
a = ipaddress.ip_address(ip.split('%', 1)[0])
|
||||
except ValueError:
|
||||
return False
|
||||
if a.version == 6 and (a not in _GLOBAL6 or any(a in n for n in _SPECIAL6)):
|
||||
return False
|
||||
if a.version == 4 and any(a in n for n in _SPECIAL4):
|
||||
return False
|
||||
return a.is_global and not (a.is_reserved or a.is_private or a.is_loopback or a.is_link_local
|
||||
or a.is_multicast or a.is_unspecified)
|
||||
|
||||
|
||||
def resolve_public(host):
|
||||
"""host's addresses for port 443, if every address it has is public; else SourceError."""
|
||||
try:
|
||||
addresses = socket.getaddrinfo(host, 443, type=socket.SOCK_STREAM)
|
||||
except (OSError, UnicodeError) as e:
|
||||
raise SourceError('Could not find the download server ' + host) from e
|
||||
if not addresses or not all(public_address(a[4][0]) for a in addresses):
|
||||
raise SourceError('The download server is on a private network address; refused')
|
||||
found = []
|
||||
for a in addresses:
|
||||
if a[4][:2] not in found:
|
||||
found.append(a[4][:2])
|
||||
return found
|
||||
|
||||
|
||||
CONNECT_BUDGET = 20 # seconds for connecting, all addresses and TLS handshakes together
|
||||
|
||||
|
||||
def _public_connection(host, timeout, budget=CONNECT_BUDGET):
|
||||
"""A verified TLS connection to host:443, made to an address checked to be public.
|
||||
|
||||
Each checked address is tried in turn, all within one deadline (budget, or timeout if
|
||||
shorter) covering both connecting and the handshake. The one connected to is always one
|
||||
that was checked, so a second lookup can't swap in another. timeout is for reads after."""
|
||||
deadline = time.monotonic() + min(budget, timeout)
|
||||
error = None
|
||||
for address in resolve_public(host):
|
||||
if time.monotonic() >= deadline:
|
||||
break
|
||||
try:
|
||||
return _connect(host, address, timeout, deadline)
|
||||
except OSError as e: # unreachable, refused, or its TLS failed: try the next one
|
||||
error = e
|
||||
raise error or socket.timeout('Timed out connecting to ' + host)
|
||||
|
||||
|
||||
def _connect(host, address, timeout, deadline):
|
||||
"""TLS to exactly address, verified as host, connected and handshaken before deadline
|
||||
(a time.monotonic() value); reads then get timeout seconds each."""
|
||||
def remaining():
|
||||
left = deadline - time.monotonic()
|
||||
if left <= 0:
|
||||
raise socket.timeout('Timed out connecting to ' + host)
|
||||
return left
|
||||
sock = socket.create_connection(address, timeout=remaining())
|
||||
try:
|
||||
# Each step gets what is left of the deadline as its socket timeout. A handshake
|
||||
# read as several receives could in theory run past it by one timeout; no thread
|
||||
# is kept to cut it short, as one could fire after the connection was handed over.
|
||||
sock.settimeout(remaining())
|
||||
sock = ssl.create_default_context().wrap_socket(sock, server_hostname=host,
|
||||
do_handshake_on_connect=False)
|
||||
sock.settimeout(remaining())
|
||||
sock.do_handshake()
|
||||
sock.settimeout(timeout)
|
||||
except BaseException:
|
||||
sock.close()
|
||||
raise
|
||||
conn = http.client.HTTPConnection(host, 443, timeout=timeout)
|
||||
conn.sock = sock
|
||||
return conn
|
||||
|
||||
|
||||
def public_apk(url, name, redirects=3, timeout=60, check=None):
|
||||
"""Download an APK from a public HTTPS host that no source lists in advance (itch.io's
|
||||
storage). No credentials are sent. Every hop, redirects included, is re-checked, and
|
||||
check(url), if given, runs on each hop before any lookup or connection (it raises)."""
|
||||
for _ in range(redirects + 1):
|
||||
if check:
|
||||
check(url)
|
||||
try:
|
||||
p = urllib.parse.urlsplit(url)
|
||||
port = p.port
|
||||
except (ValueError, TypeError) as e:
|
||||
raise SourceError(name + ' returned an invalid download URL') from e
|
||||
if p.scheme != 'https' or not p.hostname or p.username or p.password or port not in (None, 443):
|
||||
raise SourceError(name + ' returned an unexpected download URL')
|
||||
wait = wait_time(url)
|
||||
if wait:
|
||||
raise limited_error(name, wait)
|
||||
try:
|
||||
conn = _public_connection(p.hostname, timeout)
|
||||
except OSError as e:
|
||||
raise SourceError('Could not download APK: ' + str(e)) from e
|
||||
try:
|
||||
conn.request('GET', (p.path or '/') + ('?' + p.query if p.query else ''), headers={'User-Agent': UA})
|
||||
r = conn.getresponse()
|
||||
if r.status in (301, 302, 303, 307, 308):
|
||||
url = urllib.parse.urljoin(url, r.getheader('Location') or '')
|
||||
continue
|
||||
if r.status == 429:
|
||||
raise limited_error(name, throttle(url, r.headers))
|
||||
if r.status != 200:
|
||||
raise SourceError("%s's download server refused the file (HTTP %d); try again later"
|
||||
% (name, r.status))
|
||||
return save_apk(r)
|
||||
except (OSError, ValueError, http.client.HTTPException, zipfile.BadZipFile) as e:
|
||||
raise SourceError('Could not download APK: ' + str(e)) from e
|
||||
finally:
|
||||
conn.close()
|
||||
raise SourceError(name + ' redirected the download too many times')
|
||||
+97
-15
@@ -1,17 +1,35 @@
|
||||
"""Free Android VR RSS listings. Download pages stay in the publisher's UI."""
|
||||
"""Free Android VR RSS listings, plus the Android builds of games the user owns.
|
||||
|
||||
The free feeds' download pages stay in the publisher's UI. Owned games appear only
|
||||
after the user connects their itch.io account with an API key (_itch_account.py);
|
||||
they download through itch.io's own API, with the download key of that purchase.
|
||||
"""
|
||||
import html, re, urllib.parse, xml.etree.ElementTree as ET
|
||||
from email.utils import parsedate_to_datetime
|
||||
|
||||
from . import SourceError, _web
|
||||
from . import SourceError, _itch_account, _web
|
||||
|
||||
KIND = 'itch'
|
||||
FEEDS = ('openxr', 'oculus-quest')
|
||||
OWNED = 'itch-owned'
|
||||
OWNED_LABEL = 'Owned on itch.io'
|
||||
|
||||
|
||||
def sources():
|
||||
return [{'id': KIND if tag == 'openxr' else 'itch-quest', 'kind': KIND,
|
||||
'name': 'itch.io (' + tag + ')', 'url': 'https://itch.io', 'tag': tag,
|
||||
'builtin': True, 'enabled': True, 'trust': 'community'} for tag in FEEDS]
|
||||
found = [{'id': KIND if tag == 'openxr' else 'itch-quest', 'kind': KIND,
|
||||
'name': 'itch.io (' + tag + ')', 'url': 'https://itch.io', 'tag': tag,
|
||||
'builtin': True, 'enabled': True, 'trust': 'community'} for tag in FEEDS]
|
||||
if _itch_account.connected():
|
||||
found.append({'id': OWNED, 'kind': KIND, 'name': 'itch.io (my purchases)',
|
||||
'url': 'https://itch.io/my-purchases', 'owned': True,
|
||||
'description': 'Android builds of games you own on itch.io',
|
||||
'builtin': True, 'enabled': True, 'trust': 'user'})
|
||||
return found
|
||||
|
||||
|
||||
def _itch_page(url):
|
||||
p = urllib.parse.urlsplit(url if isinstance(url, str) else '')
|
||||
return p.scheme == 'https' and (p.hostname or '').endswith('.itch.io') and not p.username and ':' not in p.netloc
|
||||
|
||||
|
||||
def _parse(source, data):
|
||||
@@ -22,8 +40,7 @@ def _parse(source, data):
|
||||
entries = []
|
||||
for item in root.findall('./channel/item'):
|
||||
page = item.findtext('link') or ''
|
||||
p = urllib.parse.urlsplit(page)
|
||||
if p.scheme != 'https' or not (p.hostname or '').endswith('.itch.io') or p.username or ':' in p.netloc:
|
||||
if not _itch_page(page):
|
||||
continue
|
||||
if item.findtext('price') != '$0.00' or item.findtext('platforms/android') != 'yes':
|
||||
continue
|
||||
@@ -45,23 +62,81 @@ def _parse(source, data):
|
||||
return entries
|
||||
|
||||
|
||||
def _owned_entry(source, key):
|
||||
"""Entry for one owned download key, or None if its game isn't for Android."""
|
||||
game = key.get('game') if isinstance(key.get('game'), dict) else {}
|
||||
game_id = key.get('game_id') or game.get('id')
|
||||
if type(game_id) is not int or type(key.get('id')) is not int:
|
||||
return None
|
||||
traits = game.get('traits')
|
||||
if isinstance(traits, list) and 'p_android' not in traits:
|
||||
return None # the game's page offers no Android download
|
||||
cover = game.get('cover_url') if isinstance(game.get('cover_url'), str) else None
|
||||
if cover and not cover.startswith('https://img.itch.zone/'):
|
||||
cover = None
|
||||
user = game.get('user') if isinstance(game.get('user'), dict) else {}
|
||||
price = game.get('min_price')
|
||||
return {'source': source['id'], 'id': str(game_id), 'package': None,
|
||||
'name': str(game.get('title') or 'itch.io game ' + str(game_id)),
|
||||
'summary': str(game.get('short_text') or OWNED_LABEL),
|
||||
'developer': user.get('display_name') or user.get('username'),
|
||||
'icon': cover, 'images': {'icon': cover, 'banner': cover, 'screenshots': []},
|
||||
'page': game['url'] if _itch_page(game.get('url')) else None,
|
||||
'version': None, 'version_code': None, 'min_sdk': None, 'abis': None,
|
||||
'vr': None, 'size': None, 'free': price == 0 if type(price) is int else None,
|
||||
'owned': True, 'owned_label': OWNED_LABEL, 'license': None,
|
||||
'updated': str(game.get('published_at') or '')[:10] or None,
|
||||
'downloadable': None, 'download_key_id': key['id']}
|
||||
|
||||
|
||||
def _owned(source):
|
||||
entries = {}
|
||||
for key in _itch_account.owned_keys():
|
||||
entry = _owned_entry(source, key)
|
||||
if entry:
|
||||
entries.setdefault(entry['id'], entry) # keys come newest first; one entry per game
|
||||
return entries
|
||||
|
||||
|
||||
def search(source, query, limit=50):
|
||||
limit = max(0, min(int(limit), 100))
|
||||
if not limit:
|
||||
return []
|
||||
entries = {}
|
||||
tag = source.get('tag', 'openxr')
|
||||
if tag not in FEEDS:
|
||||
raise SourceError('Unsupported itch.io feed')
|
||||
url = 'https://itch.io/games/free/platform-android/tag-' + tag + '.xml'
|
||||
for entry in _parse(source, _web.read(url, ('itch.io',), name='itch.io')):
|
||||
entries.setdefault(entry['id'], entry)
|
||||
if source.get('owned'):
|
||||
entries = _owned(source)
|
||||
else:
|
||||
entries = {}
|
||||
tag = source.get('tag', 'openxr')
|
||||
if tag not in FEEDS:
|
||||
raise SourceError('Unsupported itch.io feed')
|
||||
url = 'https://itch.io/games/free/platform-android/tag-' + tag + '.xml'
|
||||
for entry in _parse(source, _web.read(url, ('itch.io',), name='itch.io')):
|
||||
entries.setdefault(entry['id'], entry)
|
||||
words = query.lower().split()
|
||||
return [e for e in entries.values() if all(w in (e['name'] + ' ' + e['summary']).lower()
|
||||
for w in words)][:limit]
|
||||
|
||||
|
||||
def _owned_details(source, entry_id):
|
||||
entry = _owned(source).get(str(entry_id))
|
||||
if not entry:
|
||||
raise SourceError('This game is not among the Android games you own on itch.io')
|
||||
uploads = _itch_account.uploads(int(entry['id']), entry['download_key_id'])
|
||||
versions = [{'version': str(u.get('display_name') or u.get('filename') or u['id']), 'version_code': None,
|
||||
'upload_id': u['id'], 'name': u.get('filename'), 'min_sdk': None,
|
||||
'demo': u.get('demo') is True or 'demo' in (u.get('traits') or []),
|
||||
'size': u.get('size') if type(u.get('size')) is int else None,
|
||||
'updated': str(u.get('updated_at') or u.get('created_at') or '')[:10] or None}
|
||||
for u in _itch_account.android_uploads(uploads)]
|
||||
entry = dict(entry, versions=versions, downloadable=bool(versions))
|
||||
if versions:
|
||||
entry.update({k: versions[0][k] for k in ('version', 'size', 'updated')})
|
||||
return entry
|
||||
|
||||
|
||||
def details(source, entry_id):
|
||||
if source.get('owned'):
|
||||
return _owned_details(source, entry_id)
|
||||
entry = next((e for e in search(source, '', 100) if e['id'] == entry_id), None)
|
||||
if not entry:
|
||||
raise SourceError('Game is not in the current free Android VR feed; open its publisher page')
|
||||
@@ -69,4 +144,11 @@ def details(source, entry_id):
|
||||
|
||||
|
||||
def download(source, entry_id, version_code=None):
|
||||
raise SourceError('Open the itch.io publisher page to download; automated download pages are disallowed by robots rules')
|
||||
if not source.get('owned'):
|
||||
raise SourceError('Open the itch.io publisher page to download; automated download pages are disallowed by robots rules')
|
||||
if version_code is not None:
|
||||
raise SourceError('itch.io does not publish Android version codes; download the latest upload')
|
||||
entry = _owned_details(source, entry_id)
|
||||
if not entry['versions']:
|
||||
raise SourceError('This game has no Android download on itch.io')
|
||||
return _itch_account.download(entry['versions'][0]['upload_id'], entry['download_key_id'])
|
||||
@@ -182,7 +182,10 @@ def group(entries, query='', vr=None, installable=False):
|
||||
groups = {}
|
||||
for entry in entries:
|
||||
entry = decorate(entry)
|
||||
if vr is not None and (entry.get('vr') is True) != vr: # unknown counts as flat
|
||||
# Unknown counts as flat, except the user's own purchases (itch.io doesn't say which are
|
||||
# VR): those show under either choice rather than vanish from one.
|
||||
if vr is not None and (entry.get('vr') is True) != vr \
|
||||
and not (entry.get('owned') is True and entry.get('vr') is None):
|
||||
continue
|
||||
if installable and entry['fit']['installable'] is not True:
|
||||
continue
|
||||
@@ -215,6 +218,11 @@ def group(entries, query='', vr=None, installable=False):
|
||||
return result
|
||||
|
||||
|
||||
def obtainable(source, entry):
|
||||
"""Free, or a game the user owns, listed by their own account's source (itch.io purchases)."""
|
||||
return entry.get('free') is True or (source.get('owned') is True and entry.get('owned') is True)
|
||||
|
||||
|
||||
def _launch(module, source, query, limit):
|
||||
"""One search per source at a time; the newest different query runs next."""
|
||||
key = source['id']
|
||||
@@ -243,7 +251,7 @@ def _start(key, task):
|
||||
queued = None
|
||||
try:
|
||||
task['entries'] = [dict(e, source=key, source_name=source['name'], trust=source.get('trust'))
|
||||
for e in module.search(source, query, limit=limit) if e.get('free') is True]
|
||||
for e in module.search(source, query, limit=limit) if obtainable(source, e)]
|
||||
task['stale'] = bool(getattr(module, 'stale', lambda s: False)(source))
|
||||
except Exception as e:
|
||||
task['error'] = str(e)
|
||||
@@ -300,7 +308,7 @@ def install(source_id, entry_id, version_code=None, progress=None):
|
||||
if not source['enabled']:
|
||||
raise SourceError('This source is disabled')
|
||||
entry = module.details(source, entry_id)
|
||||
if entry.get('free') is not True or entry.get('downloadable') is not True:
|
||||
if not obtainable(source, entry) or entry.get('downloadable') is not True:
|
||||
raise SourceError('This app must be obtained from its developer page')
|
||||
if progress:
|
||||
progress('Downloading', None)
|
||||
|
||||
+2
-1
@@ -31,7 +31,8 @@ import frame_host # noqa: E402
|
||||
FRAME_USER = os.environ.get("FRAME_USER", "steamos")
|
||||
USER_FROM_ENV = "FRAME_USER" in os.environ
|
||||
FRAME_ALIAS = os.environ.get("FRAME_ALIAS", "frame")
|
||||
SSH_DIR = Path.home() / ".ssh"
|
||||
# $FRAME_CONTROL_SSH_DIR in tests, as frame_devices.ssh_dir(), so they never touch the real one.
|
||||
SSH_DIR = Path(os.environ.get("FRAME_CONTROL_SSH_DIR") or Path.home() / ".ssh")
|
||||
KEY = SSH_DIR / "id_ed25519_frame"
|
||||
# The devkit service only accepts ssh-rsa keys (write_key in Valve's
|
||||
# steamos-devkit-service), so pairing uses a second key next to the ed25519 one.
|
||||
|
||||
+118
-12
@@ -28,8 +28,10 @@ remembered IP is caught.
|
||||
|
||||
Python stdlib only.
|
||||
"""
|
||||
import base64
|
||||
import contextlib
|
||||
import copy
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
@@ -418,19 +420,100 @@ def seed_pin(device_id, hosts, port=22, sources=None):
|
||||
if len(f) >= 3 and not line.startswith("#") and not f[0].startswith("@"):
|
||||
keys.append(f"{name} {f[1]} {f[2]}")
|
||||
if keys:
|
||||
target = known_hosts(device_id)
|
||||
target.parent.mkdir(**({} if frame_host.WINDOWS else {"mode": 0o700}), parents=True, exist_ok=True)
|
||||
fd_, tmp = tempfile.mkstemp(prefix=".seed-", dir=str(target.parent))
|
||||
with os.fdopen(fd_, "w", encoding="utf-8") as fh:
|
||||
fh.write("\n".join(dict.fromkeys(keys)) + "\n")
|
||||
os.replace(tmp, target) # whole file at once: ssh never sees half of it
|
||||
_write_pin(device_id, keys)
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def pin_dir():
|
||||
"""Where the per-headset files go, created private: ssh won't create it itself."""
|
||||
d = ssh_dir() / PIN_DIR
|
||||
d.mkdir(**({} if frame_host.WINDOWS else {"mode": 0o700}), parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def _write_pin(device_id, lines):
|
||||
target = known_hosts(device_id)
|
||||
fd_, tmp = tempfile.mkstemp(prefix=".seed-", dir=str(pin_dir()))
|
||||
with os.fdopen(fd_, "w", encoding="utf-8") as fh:
|
||||
fh.write("\n".join(dict.fromkeys(lines)) + "\n")
|
||||
frame_host.make_private(tmp) # Windows: this user only, not the folder's inherited ACL
|
||||
os.replace(tmp, target) # whole file at once: ssh never sees half of it
|
||||
|
||||
|
||||
# The host key types OpenSSH writes to known_hosts.
|
||||
KEY_TYPE_RE = re.compile(r"ssh-ed25519|ssh-rsa|ssh-dss|ecdsa-sha2-nistp(?:256|384|521)|"
|
||||
r"sk-ssh-ed25519@openssh\.com|sk-ecdsa-sha2-nistp256@openssh\.com")
|
||||
|
||||
|
||||
def fingerprint(blob):
|
||||
"""OpenSSH's SHA256 fingerprint of a base64 key blob (what `ssh-keygen -l` and ssh -v print),
|
||||
or None if it isn't base64."""
|
||||
try:
|
||||
raw = base64.b64decode(blob, validate=True)
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
return "SHA256:" + base64.b64encode(hashlib.sha256(raw).digest()).decode().rstrip("=")
|
||||
|
||||
|
||||
def key_entry(line, name):
|
||||
"""(type, blob, fingerprint) from a known_hosts line for `name`, or None."""
|
||||
f = (line or "").split()
|
||||
if len(f) < 3 or line.startswith("#") or name not in f[0].split(",") or not KEY_TYPE_RE.fullmatch(f[1]):
|
||||
return None
|
||||
fp = fingerprint(f[2])
|
||||
return (f[1], f[2], fp) if fp else None
|
||||
|
||||
|
||||
def pinned_keys(device_id):
|
||||
"""[(type, fingerprint)] of the keys saved for a headset (plain entries only)."""
|
||||
try:
|
||||
lines = known_hosts(device_id).read_text(encoding="utf-8").splitlines()
|
||||
except (OSError, UnicodeDecodeError):
|
||||
return []
|
||||
name = host_key_alias(device_id)
|
||||
return [(e[0], e[2]) for e in (key_entry(line, name) for line in lines) if e]
|
||||
|
||||
|
||||
def pin_key(device_id, key_type, blob):
|
||||
"""Save one key as the headset's whole identity, replacing whatever was saved: what the
|
||||
person confirmed after seeing its fingerprint (see frame_link.Link.trust_identity)."""
|
||||
if not KEY_TYPE_RE.fullmatch(key_type or "") or not fingerprint(blob):
|
||||
raise DeviceError("That isn't an SSH host key")
|
||||
_write_pin(device_id, [f"{host_key_alias(device_id)} {key_type} {blob}"])
|
||||
|
||||
|
||||
def scan_file(device_id):
|
||||
"""(path, how ssh is told) for a throwaway known_hosts file that a key check writes the
|
||||
headset's key to, without touching the saved one. `~` as in known_hosts_opt."""
|
||||
name = f".scan-{device_id}-{secrets.token_hex(4)}"
|
||||
path = pin_dir() / name
|
||||
path.touch(**({} if frame_host.WINDOWS else {"mode": 0o600}))
|
||||
frame_host.make_private(path)
|
||||
return path, (str(path) if os.environ.get("FRAME_CONTROL_SSH_DIR") else f"~/.ssh/{PIN_DIR}/{name}")
|
||||
|
||||
|
||||
def identity_established(device):
|
||||
"""Whether this headset ever had an SSH identity here: the device's own flag (set once it
|
||||
had a key or a connection that worked, and never cleared, whatever happens to its
|
||||
addresses), a fingerprint recorded in devices.json, a pin file at all (even empty,
|
||||
unreadable or not parseable), or an address that worked. Then a missing or unusable pin
|
||||
fails closed, needing Check its key; only a headset with none of these may save the
|
||||
first key it meets (accept-new)."""
|
||||
if device.get("identity_established") or device.get("identity"):
|
||||
return True
|
||||
try:
|
||||
if os.path.lexists(known_hosts(device["id"])):
|
||||
return True
|
||||
except OSError:
|
||||
return True
|
||||
return any(a.get("last_ok") for a in device.get("addresses") or ())
|
||||
|
||||
|
||||
def forget_pin(device_id):
|
||||
"""Drop a headset's saved key, e.g. after SteamOS was reinstalled. The next connection
|
||||
trusts whatever key the headset shows, as a first connection does."""
|
||||
"""Drop a headset's saved key (the headset was removed, or the API's forget-identity).
|
||||
The next connection trusts whatever key the headset shows, as a first connection does;
|
||||
the page's Trust a new key flow (frame_link) shows the key first instead."""
|
||||
try:
|
||||
known_hosts(device_id).unlink()
|
||||
return True
|
||||
@@ -510,6 +593,10 @@ class Registry:
|
||||
for a in d["addresses"]:
|
||||
a.setdefault("networks", [])
|
||||
a.setdefault("label", "")
|
||||
if d.get("identity") or any(a.get("last_ok") for a in d["addresses"]):
|
||||
# Files from before the flag existed: it had an identity (or a connection that
|
||||
# worked, which saved one). Kept from now on, whatever happens to its addresses.
|
||||
d["identity_established"] = True
|
||||
return NAME_RE.fullmatch(d.get("id", "")) is not None
|
||||
except (KeyError, TypeError, DeviceError):
|
||||
return False
|
||||
@@ -712,8 +799,25 @@ class Registry:
|
||||
a["networks"] = (a["networks"] + [network_id])[-16:]
|
||||
a["last_ok"] = time.time()
|
||||
a["last_rtt_ms"] = rtt_ms
|
||||
self._find(device_id)["identity_established"] = True # survives editing the address away
|
||||
self.save()
|
||||
|
||||
def record_identity(self, device_id, fingerprints):
|
||||
"""Remember the fingerprints of the key saved for a headset, so that losing the pin
|
||||
file later can't turn into trusting whatever answers (identity_established)."""
|
||||
fingerprints = [f for f in fingerprints if isinstance(f, str) and f.startswith("SHA256:")][:8]
|
||||
if not fingerprints:
|
||||
return
|
||||
with self._changing():
|
||||
try:
|
||||
d = self._find(device_id)
|
||||
except DeviceError:
|
||||
return
|
||||
if d.get("identity") != fingerprints or not d.get("identity_established"):
|
||||
d["identity"] = fingerprints
|
||||
d["identity_established"] = True
|
||||
self.save()
|
||||
|
||||
def undismiss(self, alias):
|
||||
"""Set Up Connection is about to run for this alias: import its block again."""
|
||||
with self._changing():
|
||||
@@ -797,15 +901,17 @@ class Registry:
|
||||
d["addresses"].append(dict(new_address(host), label="From Set Up Connection"))
|
||||
d["config_host"] = host
|
||||
changed = True
|
||||
if seed and host:
|
||||
seed_pin(d["id"], [host], b["port"])
|
||||
if seed and host and seed_pin(d["id"], [host], b["port"]):
|
||||
d["identity"] = [f for _, f in pinned_keys(d["id"])][:8] or d.get("identity")
|
||||
d["identity_established"] = True
|
||||
elif host and host != d.get("config_host"):
|
||||
# Set Up Connection ran again and found the headset somewhere new.
|
||||
d["config_host"] = host
|
||||
if not any(a["host"] == host for a in d["addresses"]):
|
||||
d["addresses"].insert(0, dict(new_address(host), label="From Set Up Connection"))
|
||||
if seed:
|
||||
seed_pin(d["id"], [host], b["port"])
|
||||
if seed and seed_pin(d["id"], [host], b["port"]) and not d.get("identity"):
|
||||
d["identity"] = [f for _, f in pinned_keys(d["id"])][:8] or None
|
||||
d["identity_established"] = True
|
||||
changed = True
|
||||
if d.get("config_login") != [user, b["port"]]:
|
||||
# Set Up Connection (or an edit) changed who to log in as, or the port.
|
||||
|
||||
+5
-1
@@ -38,9 +38,13 @@ class HostError(RuntimeError):
|
||||
|
||||
|
||||
# The start of a line in which ssh itself says the link to the headset failed (not the command it ran).
|
||||
# That includes ssh refusing the headset's host key, and the headset refusing this computer's
|
||||
# key ("user@host: Permission denied (publickey,...)."): either way no command ran.
|
||||
SSH_LINK_FAILED = re.compile(r"^(?:ssh: connect to host |ssh: Could not resolve hostname |banner exchange: |"
|
||||
r"kex_exchange_identification: |mux_client_\w+: |client_loop: |"
|
||||
r"Connection (?:closed|reset) by \S+ port \d+|Connection timed out during banner exchange)",
|
||||
r"Connection (?:closed|reset) by \S+ port \d+|Connection timed out during banner exchange|"
|
||||
r"Host key verification failed\.|No \S+ host key is known for |"
|
||||
r"[^\n@:]{1,64}@\S+: Permission denied \()", # a Windows user name may have spaces
|
||||
re.M)
|
||||
|
||||
|
||||
|
||||
+411
-33
@@ -28,11 +28,14 @@ Python stdlib only. Runs on this computer, never on the Frame.
|
||||
import copy
|
||||
import hashlib
|
||||
import ipaddress
|
||||
import os
|
||||
import queue
|
||||
import re
|
||||
import shlex
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
|
||||
@@ -50,6 +53,7 @@ NETWORK_EVERY = 5 # how often the network fingerprint is read
|
||||
TAILSCALE_EVERY = 30
|
||||
RETRY = (5, 10, 20, 30) # seconds before automatic retries after a failure
|
||||
REQUEST_GAP = 5 # a request may start a new attempt this long after the last one
|
||||
SCAN_TTL = 600 # a key shown for "Trust a new key" may be trusted this long after it was read
|
||||
|
||||
STAGES = [("network", "Checking this computer's network"), ("find", "Finding the headset"),
|
||||
("ssh", "Opening SSH"), ("identity", "Checking the headset's identity"),
|
||||
@@ -90,6 +94,32 @@ def ssh_g(alias):
|
||||
return got.get("hostname") or alias, port, got.get("user"), proxied
|
||||
|
||||
|
||||
def proxy_route(alias):
|
||||
"""How ~/.ssh/config sends ALIAS: (None, None) directly, ("jump", spec) through
|
||||
ProxyJump, or ("command", text) through a ProxyCommand."""
|
||||
try:
|
||||
out = frame_host.run_ssh(["ssh", "-G", alias], capture_output=True, stdin=subprocess.DEVNULL, text=True,
|
||||
timeout=10).stdout
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
out = ""
|
||||
got = {}
|
||||
for line in out.splitlines():
|
||||
k, _, v = line.partition(" ")
|
||||
if k in ("proxyjump", "proxycommand") and k not in got:
|
||||
got[k] = v.strip()
|
||||
if got.get("proxyjump") not in (None, "", "none"):
|
||||
return "jump", got["proxyjump"]
|
||||
if got.get("proxycommand") not in (None, "", "none"):
|
||||
return "command", got["proxycommand"]
|
||||
return None, None
|
||||
|
||||
|
||||
# One jump host, as ProxyJump names it: [user@]host[:port], nothing that could start an
|
||||
# option, split a shell word or carry an ssh % token.
|
||||
JUMP_RE = re.compile(r"(?:([A-Za-z0-9][A-Za-z0-9._-]{0,63})@)?"
|
||||
r"(\[[0-9A-Fa-f:.]{2,45}\]|[A-Za-z0-9][A-Za-z0-9.-]{0,252})(?::(\d{1,5}))?")
|
||||
|
||||
|
||||
def probe(host, port, timeout=PROBE_TIMEOUT, update=None):
|
||||
"""Try a TCP connection to host:port. -> {"state", "detail", "ip", "rtt_ms"}.
|
||||
|
||||
@@ -257,6 +287,43 @@ def ssh_target(host, ip):
|
||||
return host
|
||||
|
||||
|
||||
def _unlink(path):
|
||||
try:
|
||||
os.unlink(path)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _scanned_key(path, name):
|
||||
"""The first whole, valid known_hosts line for `name` that a key check wrote, or None.
|
||||
Only lines ssh has finished (ending in a newline): a half-written key mustn't count."""
|
||||
try:
|
||||
with open(path, encoding="utf-8") as fh:
|
||||
text = fh.read()
|
||||
except (OSError, UnicodeDecodeError):
|
||||
return None
|
||||
for line in text.splitlines(True):
|
||||
if line.endswith("\n"):
|
||||
entry = frame_devices.key_entry(line.strip(), name)
|
||||
if entry:
|
||||
return entry
|
||||
return None
|
||||
|
||||
|
||||
def proxy_command(literal):
|
||||
"""A ProxyCommand that runs `literal` (argv, ending with the jump host) with `-W` to the
|
||||
headset, each word exactly as given. ssh first expands its %-tokens in the line, so every
|
||||
% of ours is doubled; then it hands the line to a shell (`$SHELL -c` on macOS and Linux)
|
||||
or, on Windows, starts it as a command line, so every word is quoted for that: a path
|
||||
with spaces, $(...), backticks or quotes stays one plain word. The forward is
|
||||
[%h]:%p, so an IPv6 address (with a zone) reads as one host."""
|
||||
words = [w.replace("%", "%%") for w in literal]
|
||||
target = "[%h]:%p"
|
||||
if frame_host.WINDOWS:
|
||||
return subprocess.list2cmdline([*words[:-1], "-W", target, words[-1]])
|
||||
return " ".join(shlex.quote(w) for w in [*words[:-1], "-W", target, words[-1]])
|
||||
|
||||
|
||||
def probe_raw(host, port, result):
|
||||
"""ssh's own wording for a failed probe, so the server's UNREACHABLE table explains it."""
|
||||
return {"unresolved": f"ssh: Could not resolve hostname {host}: not found",
|
||||
@@ -275,7 +342,8 @@ class Link:
|
||||
self.session_alias = env_alias # ...and stays selectable after switching away
|
||||
self.mux_base = list(mux_base) # ["ssh", "-o", "BatchMode=yes", ControlPath...]
|
||||
self.control = control # ControlPath, or None where ssh can't share connections
|
||||
self.apply = apply # apply(alias, host_opts): point every ssh command at the headset
|
||||
self._apply = apply # apply(alias, host_opts): point every ssh command at the headset
|
||||
self.applied = (None, []) # what it was last given
|
||||
self.explain = explain # ssh error text -> plain reason, or None
|
||||
self.cond = threading.Condition()
|
||||
self.version = 0
|
||||
@@ -301,6 +369,7 @@ class Link:
|
||||
self.routed_device = None
|
||||
self.last_failure = None # {"stage", "category", "at"}: for report diagnostics, no free text
|
||||
self.attempt_device = None # the headset the current attempt is for
|
||||
self.scans = {} # device id -> the host key it showed for "Trust a new key"
|
||||
|
||||
# ---- publishing ----
|
||||
def publish(self, **fields):
|
||||
@@ -505,13 +574,76 @@ class Link:
|
||||
return [*mux, *(device.get("frozen") or [])] # what ~/.ssh/config said when it was routed
|
||||
if not host: # a headset with no addresses: reach nothing, not whatever ~/.ssh/config says
|
||||
return ["-o", "HostName=no-address.invalid"]
|
||||
# StrictHostKeyChecking=yes: whatever ~/.ssh/config says for Host *, every command
|
||||
# checks the headset's pinned key (only the connector's first handshake may save one).
|
||||
return [*mux, "-o", "StrictHostKeyChecking=yes", "-o", f"HostName={frame_devices.ssh_host(host)}",
|
||||
# Whatever ~/.ssh/config says for Host *, every command checks the headset's pinned key:
|
||||
# StrictHostKeyChecking=yes. Only a headset that never had an identity here (just
|
||||
# added, never connected) gets accept-new: the first contact saves its key, as Set Up
|
||||
# Connection and the connector's first handshake do, rather than every command failing
|
||||
# with "No host key is known" until the connector gets there. Once it had one, a pin
|
||||
# file that's gone, empty or unreadable fails closed (identity_established): accept-new
|
||||
# can't tell that from a new headset and would trust whatever answers.
|
||||
strict = self.strictness(device)
|
||||
if strict == "accept-new":
|
||||
try:
|
||||
frame_devices.pin_dir() # ssh won't create the folder its known_hosts file goes in
|
||||
except OSError:
|
||||
pass # ssh then says it couldn't save the key, and goes on
|
||||
return [*mux, "-o", f"StrictHostKeyChecking={strict}", "-o", f"HostName={frame_devices.ssh_host(host)}",
|
||||
"-o", f"HostKeyAlias={frame_devices.host_key_alias(device['id'])}",
|
||||
"-o", f"UserKnownHostsFile={frame_devices.known_hosts_opt(device['id'])}", "-o", "HashKnownHosts=no",
|
||||
"-o", f"User={device['user']}", "-o", f"Port={device['port']}"]
|
||||
|
||||
def apply(self, alias, opts):
|
||||
self.applied = (alias, list(opts))
|
||||
self._apply(alias, opts)
|
||||
|
||||
def strictness(self, device):
|
||||
"""yes, or accept-new for a headset that never had an identity. Judged on what the
|
||||
registry says now, not only the copy handed in (a route frozen at connect, say, from
|
||||
before its first key was recorded). A key saved since is recorded on the way."""
|
||||
try:
|
||||
current = self.reg.get(device["id"])
|
||||
except frame_devices.DeviceError:
|
||||
current = {}
|
||||
if not (frame_devices.identity_established(device) or
|
||||
(current and frame_devices.identity_established(current))):
|
||||
return "accept-new"
|
||||
self.settle_pin(dict(device, **current) if current else device, tighten=False)
|
||||
return "yes"
|
||||
|
||||
def settle_pin(self, device=None, tighten=True):
|
||||
"""A key ssh saved on a first contact, whatever happened next (logged in or refused,
|
||||
by the handshake or any command): record it, make the file private, and from then on
|
||||
route every command strictly. Idempotent and cheap when there's nothing to do."""
|
||||
device = device or self.routed_device
|
||||
if not device or device.get("transient") or device.get("none") or not device.get("id"):
|
||||
return
|
||||
did = device["id"]
|
||||
if frame_devices.pinned(did):
|
||||
fps = [f for _, f in frame_devices.pinned_keys(did)]
|
||||
if fps and (device.get("identity") != fps or not device.get("identity_established")):
|
||||
try:
|
||||
current = self.reg.get(did)
|
||||
except frame_devices.DeviceError:
|
||||
return
|
||||
if current.get("identity") != fps or not current.get("identity_established"):
|
||||
frame_host.make_private(frame_devices.known_hosts(did)) # ssh wrote it
|
||||
self.reg.record_identity(did, fps)
|
||||
if not tighten:
|
||||
return
|
||||
with self.route_lock:
|
||||
alias, opts = self.applied
|
||||
if alias == device.get("alias") and "StrictHostKeyChecking=accept-new" in opts and \
|
||||
frame_devices.identity_established(dict(device, **self._current(did))):
|
||||
self.opts = [("StrictHostKeyChecking=yes" if o == "StrictHostKeyChecking=accept-new" else o)
|
||||
for o in opts]
|
||||
self.apply(alias, self.opts)
|
||||
|
||||
def _current(self, device_id):
|
||||
try:
|
||||
return self.reg.get(device_id)
|
||||
except frame_devices.DeviceError:
|
||||
return {}
|
||||
|
||||
def public_device(self, d):
|
||||
return {k: d.get(k) for k in ("id", "name", "alias", "user", "port", "transient")}
|
||||
|
||||
@@ -541,6 +673,8 @@ class Link:
|
||||
if t - last_ts >= TAILSCALE_EVERY and self.state["network"] and not reasons:
|
||||
last_ts = t
|
||||
self.refresh_network()
|
||||
if "StrictHostKeyChecking=accept-new" in self.applied[1]:
|
||||
self.settle_pin() # a command's first contact may have saved the key
|
||||
phase = self.state["phase"]
|
||||
if phase == "connected" and not self.alive():
|
||||
reasons.append("dropped")
|
||||
@@ -675,10 +809,11 @@ class Link:
|
||||
return "Starting up"
|
||||
return "Connecting"
|
||||
|
||||
def fail(self, sid, message, raw=""):
|
||||
def fail(self, sid, message, raw="", **more):
|
||||
"""A stage failed. `more`: what the page can offer to fix it (hostkey, auth)."""
|
||||
self.stage(sid, "failed", message)
|
||||
with self.cond:
|
||||
self.state["error"] = {"stage": sid, "message": message, "raw": raw}
|
||||
self.state["error"] = dict({"stage": sid, "message": message, "raw": raw}, **more)
|
||||
probes = copy.deepcopy(self.state["probes"])
|
||||
self.note_failure(sid, message, raw, probes)
|
||||
|
||||
@@ -867,6 +1002,7 @@ class Link:
|
||||
if device.get("transient"):
|
||||
return
|
||||
self.reg.record_success(device["id"], host, net.get("id"), rtt)
|
||||
self.reg.record_identity(device["id"], [f for _, f in frame_devices.pinned_keys(device["id"])])
|
||||
# Set Up Connection may have changed the block while this attempt ran: take that
|
||||
# in (and reconnect) rather than writing this attempt's older settings over it.
|
||||
self.watch_config()
|
||||
@@ -921,7 +1057,17 @@ class Link:
|
||||
|
||||
def handshake(self, device, a, found, user):
|
||||
"""SSH to one address, following ssh -v through stages 3-5.
|
||||
-> "ok", "next" (try another address) or "stop"."""
|
||||
-> "ok", "next" (try another address) or "stop". A key ssh saved is recorded and
|
||||
protected however it ends: a login refused after the key was saved, a timeout, a stop."""
|
||||
try:
|
||||
return self._handshake(device, a, found, user)
|
||||
finally:
|
||||
if not device.get("transient"):
|
||||
with self.route_lock:
|
||||
current = self.attempt_gen == self.gen
|
||||
self.settle_pin(device, tighten=current)
|
||||
|
||||
def _handshake(self, device, a, found, user):
|
||||
# The IP that answered (with a link-local IPv6 address's zone), so ssh doesn't
|
||||
# look the name up again and stall on an address that didn't answer.
|
||||
opts = self.host_opts(device, ssh_target(a["host"], found.get("ip")))
|
||||
@@ -939,15 +1085,9 @@ class Link:
|
||||
return "ok"
|
||||
extra = []
|
||||
if not device.get("transient"):
|
||||
if frame_devices.pinned(device["id"]):
|
||||
extra = ["-o", "StrictHostKeyChecking=yes"]
|
||||
else:
|
||||
# First connection since this headset was added: trust what it shows
|
||||
# (as Set Up Connection does), and pin it from now on. ssh won't create
|
||||
# the folder its known_hosts file goes in.
|
||||
extra = ["-o", "StrictHostKeyChecking=accept-new"]
|
||||
pins = frame_devices.known_hosts(device["id"]).parent
|
||||
pins.mkdir(**({} if frame_host.WINDOWS else {"mode": 0o700}), parents=True, exist_ok=True)
|
||||
# A headset's first connection trusts what it shows (as Set Up Connection does) and
|
||||
# pins it; any later one checks the pin strictly (see host_opts).
|
||||
extra = ["-o", f"StrictHostKeyChecking={self.strictness(device)}"]
|
||||
if self.control:
|
||||
# No ConnectTimeout: with it, OpenSSH's master takes ~5s to open its socket.
|
||||
# `extra` first: ssh takes the first value of an option, and it may say accept-new.
|
||||
@@ -955,27 +1095,70 @@ class Link:
|
||||
"-o", "ServerAliveCountMax=2", "-N", alias]
|
||||
else:
|
||||
argv = [*self.mux_base, *extra, *opts, "-v", "-o", "ConnectTimeout=10", alias, "true"]
|
||||
lines = queue.Queue()
|
||||
collecting = [True]
|
||||
reader_done = threading.Event()
|
||||
errfile = None
|
||||
try:
|
||||
proc = subprocess.Popen(argv, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.PIPE, **frame_host.DETACHED)
|
||||
if self.control:
|
||||
proc = subprocess.Popen(argv, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.PIPE, **frame_host.DETACHED)
|
||||
else:
|
||||
# No master (Windows): ssh runs `true` and exits. Its stderr goes to a file read
|
||||
# as it grows, not a pipe: Windows' OpenSSH can block writing to a redirected
|
||||
# pipe (see frame_host.run_ssh), and the handshake would never finish.
|
||||
fd_, errfile = tempfile.mkstemp(prefix="frame-ssh-")
|
||||
try:
|
||||
proc = subprocess.Popen(argv, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
|
||||
stderr=fd_, **frame_host.DETACHED)
|
||||
finally:
|
||||
os.close(fd_)
|
||||
except OSError as e:
|
||||
if errfile:
|
||||
_unlink(errfile)
|
||||
self.fail("ssh", f"Couldn't run ssh: {e}", str(e))
|
||||
return "stop"
|
||||
self.pending = proc # so stop() can end it mid-handshake
|
||||
lines = queue.Queue()
|
||||
collecting = [True]
|
||||
|
||||
def read():
|
||||
for raw in iter(proc.stderr.readline, b""):
|
||||
if collecting[0]:
|
||||
lines.put(raw.decode("utf-8", "replace").rstrip())
|
||||
reader_done.set()
|
||||
lines.put(None)
|
||||
proc.stderr.close()
|
||||
threading.Thread(target=read, daemon=True).start()
|
||||
|
||||
def tail():
|
||||
buf = b""
|
||||
try:
|
||||
with open(errfile, "rb") as fh:
|
||||
while True:
|
||||
chunk = fh.read()
|
||||
if not chunk:
|
||||
if proc.poll() is None:
|
||||
time.sleep(0.05)
|
||||
continue
|
||||
chunk = fh.read() # whatever it wrote just before exiting
|
||||
if not chunk:
|
||||
break
|
||||
buf += chunk
|
||||
*done, buf = buf.split(b"\n")
|
||||
for raw in done:
|
||||
lines.put(raw.decode("utf-8", "replace").rstrip())
|
||||
except OSError:
|
||||
pass
|
||||
if buf:
|
||||
lines.put(buf.decode("utf-8", "replace").rstrip())
|
||||
reader_done.set()
|
||||
lines.put(None)
|
||||
_unlink(errfile)
|
||||
threading.Thread(target=read if errfile is None else tail, daemon=True).start()
|
||||
|
||||
step, said, authed = "ssh", [], False
|
||||
deadline = time.monotonic() + HANDSHAKE_TIMEOUT
|
||||
mismatch = False
|
||||
shown = None # (type, fingerprint) of the key the headset showed
|
||||
exited = None # when ssh was first seen to have exited
|
||||
while True:
|
||||
left = deadline - time.monotonic()
|
||||
if self.stopped:
|
||||
@@ -993,7 +1176,14 @@ class Link:
|
||||
if authed and self.control and self.check(opts, alias):
|
||||
break
|
||||
if proc.poll() is not None and lines.empty():
|
||||
line = None
|
||||
# ssh exited: its last words (Permission denied…) may still be on their way.
|
||||
# The reader's end marker says when they're all in; a pipe that something
|
||||
# else holds open (a ProxyCommand's child) gets a short grace instead.
|
||||
exited = exited or time.monotonic()
|
||||
if reader_done.is_set() and lines.empty() or time.monotonic() - exited > 2:
|
||||
line = None
|
||||
else:
|
||||
continue
|
||||
else:
|
||||
continue
|
||||
if line is None: # ssh exited
|
||||
@@ -1002,7 +1192,7 @@ class Link:
|
||||
break
|
||||
if authed and self.control and self.check(opts, alias):
|
||||
break
|
||||
return self.failed(step, said, mismatch, alias)
|
||||
return self.failed(step, said, mismatch, alias, shown, a["host"], device)
|
||||
if not line.startswith("debug"):
|
||||
said.append(line)
|
||||
m = CONNECTING.search(line)
|
||||
@@ -1015,12 +1205,15 @@ class Link:
|
||||
self.stage("identity", "active", "Waiting for the headset's host key")
|
||||
elif HOSTKEY.search(line):
|
||||
m = HOSTKEY.search(line)
|
||||
shown = (m.group(1), m.group(2))
|
||||
self.stage("identity", "active", f"It shows {m.group(1)} key {m.group(2)[:20]}…")
|
||||
elif KNOWN.search(line):
|
||||
self.stage("identity", "done", "Matches the identity saved for this headset")
|
||||
step = "login"
|
||||
self.stage("login", "active", f"Logging in as {user}")
|
||||
elif ADDED.search(line):
|
||||
if not device.get("transient"):
|
||||
self.settle_pin(device, tighten=False) # now, before the login can fail
|
||||
self.stage("identity", "done", "First connection: saved this headset's identity")
|
||||
step = "login"
|
||||
self.stage("login", "active", f"Logging in as {user}")
|
||||
@@ -1059,18 +1252,34 @@ class Link:
|
||||
# OpenSSH lower-cases host names (FRAME.LOCAL logs as frame.local).
|
||||
return not m or m.group(1).lower() in (host.lower(), alias.lower()) or "Authentication succeeded" in line
|
||||
|
||||
def failed(self, step, said, mismatch, alias):
|
||||
def failed(self, step, said, mismatch, alias, shown=None, host=None, device=None):
|
||||
text = "\n".join(said).strip()
|
||||
if mismatch:
|
||||
self.fail("identity", "This address answered as a different headset (its SSH identity doesn't match). "
|
||||
"If SteamOS was reinstalled, use Forget Identity on the Devices tab.", text)
|
||||
# A key that differs from the saved one, or (strict, pinned) none of the saved key's
|
||||
# type any more: either way not the identity saved. Never accepted here: the page
|
||||
# shows the new key's fingerprint and the person decides (Trust a new key).
|
||||
name = (device or {}).get("name") or alias
|
||||
lost = bool(device) and not frame_devices.pinned(device["id"]) and \
|
||||
frame_devices.identity_established(device)
|
||||
if lost:
|
||||
message = (f"The SSH identity saved for {name} is missing or can't be read, so Frame Control won't "
|
||||
"trust whatever answers. Check its key and trust it on the Devices tab.")
|
||||
else:
|
||||
message = (f"This address answered with a different SSH identity from the one saved for {name}. "
|
||||
"If SteamOS was reinstalled or reset, or this is a new headset, check its new key and trust "
|
||||
"it on the Devices tab. If not, a different device may be answering here.")
|
||||
self.fail("identity", message, text,
|
||||
hostkey={"state": "missing" if lost else "changed", "type": shown[0] if shown else None,
|
||||
"fingerprint": shown[1] if shown else None, "host": host})
|
||||
return "next"
|
||||
# A refused key is the same at every address: stop. (Judged by ssh's own words, not
|
||||
# the step: a jump host's progress lines look like the headset's.) A forward that
|
||||
# a jump host couldn't open is about this address only: try the next.
|
||||
forward = re.search(r"open failed|forwarding failed|Connection refused|Connection closed|timed out", text)
|
||||
if re.search(r"Permission denied", text) and not forward:
|
||||
self.fail("login", self.explain(text) or "The headset didn't accept this computer's key.", text)
|
||||
# Not something a retry fixes: the headset needs this computer's key (Set Up Connection).
|
||||
self.fail("login", self.explain(text) or "The headset didn't accept this computer's key. Run Set Up "
|
||||
"Connection: it pairs this computer with the headset again.", text, auth="denied")
|
||||
return "stop"
|
||||
if step == "connected":
|
||||
self.fail("login", "Logged in, but the shared SSH connection didn't start.", text)
|
||||
@@ -1110,9 +1319,11 @@ class Link:
|
||||
lead = f"Answered in {res['rtt_ms']:g} ms" if res.get("rtt_ms") is not None else "Through the jump host"
|
||||
rows[i]["ssh"] = "checking"
|
||||
put()
|
||||
# "yes" before host_opts (ssh takes an option's first value): a test only looks, so
|
||||
# it reports a key not saved yet rather than saving it as host_opts' accept-new would.
|
||||
argv = [*self.mux_base[:3], "-o", "ControlPath=none", "-o", "ConnectTimeout=8",
|
||||
*self.host_opts(device, ssh_target(a["host"], res.get("ip"))),
|
||||
"-o", "StrictHostKeyChecking=yes", device["alias"], "true"]
|
||||
"-o", "StrictHostKeyChecking=yes",
|
||||
*self.host_opts(device, ssh_target(a["host"], res.get("ip"))), device["alias"], "true"]
|
||||
try:
|
||||
r = frame_host.run_ssh(argv, capture_output=True, stdin=subprocess.DEVNULL, text=True,
|
||||
errors="replace", timeout=20)
|
||||
@@ -1140,6 +1351,156 @@ class Link:
|
||||
put(done=True, finished=now())
|
||||
self.devices_changed()
|
||||
|
||||
# ---- Trust a new key ----
|
||||
# Every way a key check could do more than read the key: no login of any kind, no agent
|
||||
# (nor forwarding it), no X11, port or tunnel forwarding, no local or remote command, no
|
||||
# terminal, no shared connection, no learning keys. Given first, so ssh takes these over
|
||||
# anything ~/.ssh/config (or host_opts) says. -N: no command, even if a device accepts "none".
|
||||
SCAN_OPTS = ["BatchMode=yes", "ControlMaster=no", "ControlPath=none", "ConnectTimeout=8",
|
||||
"StrictHostKeyChecking=accept-new", "UpdateHostKeys=no", "HashKnownHosts=no",
|
||||
"PreferredAuthentications=none", "PubkeyAuthentication=no", "PasswordAuthentication=no",
|
||||
"KbdInteractiveAuthentication=no", "HostbasedAuthentication=no", "IdentityAgent=none",
|
||||
"ForwardAgent=no", "ForwardX11=no", "ForwardX11Trusted=no", "ClearAllForwardings=yes",
|
||||
"Tunnel=no", "PermitLocalCommand=no", "RemoteCommand=none", "RequestTTY=no"]
|
||||
SCAN_TIMEOUT = 20
|
||||
|
||||
def scan_identity(self, device_id):
|
||||
"""Read the headset's host key without trusting it or logging in. ssh writes it to a
|
||||
throwaway known_hosts file during the key exchange, before any authentication, and is
|
||||
stopped as soon as it has (or when it gives up); SCAN_OPTS and -N rule out anything else.
|
||||
`-F none` too, unless ~/.ssh/config sends the alias through a jump host (then it's
|
||||
needed for that, and SCAN_OPTS still win). The address that showed the changed key goes
|
||||
first, then the rest in order. -> {"type", "fingerprint", "host", "saved", "recorded",
|
||||
"matches", "recorded_match"}; the key is kept (SCAN_TTL) for trust_identity, which pins
|
||||
exactly it."""
|
||||
device = self.reg.get(device_id)
|
||||
if not device["addresses"]:
|
||||
raise frame_devices.DeviceError("This headset has no addresses to ask")
|
||||
hosts = [a["host"] for a in device["addresses"]]
|
||||
with self.cond:
|
||||
err = self.state.get("error") or {}
|
||||
mine = (self.state.get("device") or {}).get("id") == device_id
|
||||
first = (err.get("hostkey") or {}).get("host") if mine else None
|
||||
if first in hosts:
|
||||
hosts.remove(first)
|
||||
hosts.insert(0, first)
|
||||
route = self.scan_route(device)
|
||||
reasons = []
|
||||
for host in hosts:
|
||||
key, err_text = self._scan_one(device, host, route)
|
||||
if key:
|
||||
saved = frame_devices.pinned_keys(device_id)
|
||||
recorded = [f for f in device.get("identity") or [] if isinstance(f, str)]
|
||||
scan = {"type": key[0], "blob": key[1], "fingerprint": key[2], "host": host,
|
||||
"port": device["port"], "user": device["user"], "at": now()}
|
||||
with self.cond:
|
||||
self.scans[device_id] = scan
|
||||
return {"type": key[0], "fingerprint": key[2], "host": host,
|
||||
"saved": [{"type": t, "fingerprint": f} for t, f in saved], "recorded": recorded,
|
||||
"matches": any(f == key[2] for _, f in saved), "recorded_match": key[2] in recorded}
|
||||
text = err_text.strip()
|
||||
reasons.append(f"{host}: " + (self.explain(text) or (text.splitlines() or ["no answer"])[-1]))
|
||||
raise frame_devices.DeviceError("Couldn't read the headset's key. " + "; ".join(reasons))
|
||||
|
||||
# The jump host's own ssh (a ProxyCommand the scan builds itself): it may log in to the jump
|
||||
# host as configured (keys, the agent), but it never forwards the agent, X11 or ports, runs
|
||||
# no local or remote command, has no terminal, shares no connection and goes no further.
|
||||
HOP_OPTS = ["BatchMode=yes", "ForwardAgent=no", "ForwardX11=no", "ForwardX11Trusted=no",
|
||||
"ClearAllForwardings=yes", "Tunnel=no", "PermitLocalCommand=no", "RemoteCommand=none",
|
||||
"RequestTTY=no", "ControlMaster=no", "ControlPath=none", "ProxyCommand=none", "ProxyJump=none"]
|
||||
|
||||
def scan_route(self, device):
|
||||
"""The options that take a key check to the headset: `-F none` and nothing else when
|
||||
it's reached directly. Through a jump host (ProxyJump), `-F none` and a ProxyCommand
|
||||
built here, whose ssh gets HOP_OPTS ahead of ~/.ssh/config: config options can't be
|
||||
overridden for ProxyJump's own ssh, which inherits ForwardAgent, LocalCommand and the
|
||||
rest. Through a ProxyCommand from ~/.ssh/config, or several jump hosts, it refuses:
|
||||
those can't be made safe from here."""
|
||||
kind, spec = proxy_route(device["alias"])
|
||||
if kind is None:
|
||||
return ["-F", "none"]
|
||||
m = JUMP_RE.fullmatch(spec or "") if kind == "jump" else None
|
||||
if not m or (m.group(3) and not 1 <= int(m.group(3)) <= 65535):
|
||||
how = "a ProxyCommand" if kind == "command" else "this jump host setting"
|
||||
raise frame_devices.DeviceError(
|
||||
f"{device['name']} is reached through {how} in ~/.ssh/config, which Frame Control can't make safe "
|
||||
"for talking to a device it doesn't trust yet, so it won't read the key that way. Add an address "
|
||||
"it can reach directly (or through a single ProxyJump host) and check its key again; or compare "
|
||||
"the key by hand (on the headset, in Konsole: ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub)")
|
||||
user, host, port = m.groups()
|
||||
# The ssh this server runs, by its full path: the shell that runs a ProxyCommand may set
|
||||
# up another PATH.
|
||||
ssh = frame_host.which(self.mux_base[0])
|
||||
if not ssh:
|
||||
raise frame_devices.DeviceError(f"Couldn't find {self.mux_base[0]} to reach the jump host with")
|
||||
hop = [ssh, *(a for o in self.HOP_OPTS for a in ("-o", o)), *(["-l", user] if user else []),
|
||||
*(["-p", port] if port else []), host.strip("[]")] # ssh takes a bare IPv6 address here
|
||||
return ["-F", "none", "-o", "ProxyCommand=" + proxy_command(hop)]
|
||||
|
||||
def _scan_one(self, device, host, route):
|
||||
"""-> ((type, blob, fingerprint) or None, what ssh said)."""
|
||||
name = frame_devices.host_key_alias(device["id"])
|
||||
path, opt = frame_devices.scan_file(device["id"])
|
||||
opts = [a for o in [*self.SCAN_OPTS, f"UserKnownHostsFile={opt}"] for a in ("-o", o)]
|
||||
argv = [self.mux_base[0], *route, *opts, *self.host_opts(device, host), "-N", device["alias"]]
|
||||
fd_, errfile = tempfile.mkstemp(prefix="frame-scan-") # a file, not a pipe (see frame_host.run_ssh)
|
||||
key, proc = None, None
|
||||
try:
|
||||
try:
|
||||
proc = subprocess.Popen(argv, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=fd_,
|
||||
**frame_host.DETACHED)
|
||||
finally:
|
||||
os.close(fd_)
|
||||
deadline = time.monotonic() + self.SCAN_TIMEOUT
|
||||
while key is None:
|
||||
ended = proc.poll() is not None
|
||||
key = _scanned_key(path, name)
|
||||
if key or ended or self.stopped or time.monotonic() > deadline:
|
||||
break
|
||||
time.sleep(0.05)
|
||||
except OSError as e:
|
||||
return None, str(e)
|
||||
finally:
|
||||
if proc and proc.poll() is None:
|
||||
proc.kill() # it has the key (or gave up): nothing more to do with that device
|
||||
if proc:
|
||||
try:
|
||||
proc.wait(5)
|
||||
except subprocess.TimeoutExpired:
|
||||
pass
|
||||
try:
|
||||
with open(errfile, "rb") as fh:
|
||||
said = fh.read().decode("utf-8", "replace")
|
||||
except OSError:
|
||||
said = ""
|
||||
_unlink(errfile)
|
||||
_unlink(path)
|
||||
if key is None and not said.strip() and proc and proc.returncode is not None and proc.returncode < 0:
|
||||
said = f"Timed out talking to {device['alias']}"
|
||||
return key, said
|
||||
|
||||
def trust_identity(self, device_id, fingerprint):
|
||||
"""Pin the key scan_identity read, and only if `fingerprint` is the one it showed (what
|
||||
the person saw and confirmed), replacing whatever was saved; and only while the headset
|
||||
is still reached as it was then (same port and user, the address still one of its own).
|
||||
-> the scan."""
|
||||
with self.cond:
|
||||
scan = self.scans.get(device_id)
|
||||
if not scan or now() - scan["at"] > SCAN_TTL:
|
||||
self.scans.pop(device_id, None)
|
||||
raise frame_devices.DeviceError("Check the headset's key again first: the one shown has expired")
|
||||
if not isinstance(fingerprint, str) or fingerprint != scan["fingerprint"]:
|
||||
raise frame_devices.DeviceError("That isn't the key the headset showed. Check its key again")
|
||||
self.scans.pop(device_id)
|
||||
d = self.reg.get(device_id) # still a headset here
|
||||
if (d["port"], d["user"]) != (scan["port"], scan["user"]) or \
|
||||
scan["host"] not in [a["host"] for a in d["addresses"]]:
|
||||
raise frame_devices.DeviceError("The headset's address, port or user changed since its key was read. "
|
||||
"Check its key again")
|
||||
frame_devices.pin_key(device_id, scan["type"], scan["blob"])
|
||||
self.reg.record_identity(device_id, [scan["fingerprint"]])
|
||||
return scan
|
||||
|
||||
|
||||
# ---- the page's API: /api/devices -------------------------------------------------
|
||||
|
||||
@@ -1158,6 +1519,9 @@ def devices_view(link):
|
||||
view = {k: v for k, v in d.items() if k not in ("config_host", "addresses")}
|
||||
view["active"] = d["id"] == active["id"]
|
||||
view["pinned"] = frame_devices.pinned(d["id"])
|
||||
view["keys"] = [{"type": t, "fingerprint": f} for t, f in frame_devices.pinned_keys(d["id"])]
|
||||
# It had an identity, but the saved key is gone or unusable: nothing is trusted until checked.
|
||||
view["identity_lost"] = not view["pinned"] and frame_devices.identity_established(d)
|
||||
view["addresses"] = [dict(a, network_names=[names.get(n, "an unnamed network") for n in a["networks"]])
|
||||
for a in d["addresses"]]
|
||||
devices.append(view)
|
||||
@@ -1190,7 +1554,7 @@ def devices_action(link, body, open_setup, busy=lambda: 0):
|
||||
is_active = did == active["id"]
|
||||
moves = action == "use" or (action == "retry" and link.alive()) or (is_active and (
|
||||
# (a retry while connected would cut the install's connection)
|
||||
action in ("remove", "address-remove", "forget-identity")
|
||||
action in ("remove", "address-remove", "identity-trust")
|
||||
or (action == "update" and login_change(reg, did, body))
|
||||
or (action == "address-update" and body.get("newHost") not in (None, body.get("host")))))
|
||||
if moves and busy():
|
||||
@@ -1270,11 +1634,25 @@ def devices_action(link, body, open_setup, busy=lambda: 0):
|
||||
threading.Thread(target=link.test, args=(did,), daemon=True).start()
|
||||
msg = f"Testing {len(d['addresses'])} address{'es' * (len(d['addresses']) != 1)}"
|
||||
elif action == "forget-identity":
|
||||
# Dropping the pin used to let the next contact save whatever key answered, unseen.
|
||||
# A new key is now always shown first: identity-scan, then identity-trust.
|
||||
reg.get(did)
|
||||
raise frame_devices.DeviceError("Forgetting a headset's identity is no longer supported: use Check its key "
|
||||
"on the Devices tab (identity-scan, then identity-trust), which shows the "
|
||||
"new key before saving it")
|
||||
elif action == "identity-scan":
|
||||
d = reg.get(did)
|
||||
frame_devices.forget_pin(did)
|
||||
scan = link.scan_identity(did)
|
||||
msg = (f"{d['name']} shows the key already saved for it" if scan["matches"]
|
||||
else f"{d['name']} shows a {scan['type']} key, {scan['fingerprint']}")
|
||||
link.devices_changed()
|
||||
return dict(devices_view(link), message=msg, scan=scan)
|
||||
elif action == "identity-trust":
|
||||
d = reg.get(did)
|
||||
link.trust_identity(did, body.get("fingerprint"))
|
||||
if is_active:
|
||||
link.kick("switch")
|
||||
msg = f"Forgot {d['name']}'s SSH identity; the next connection saves the one it shows"
|
||||
link.kick("switch") # reconnect with the key now saved
|
||||
msg = f"Saved {d['name']}'s new SSH identity; reconnecting"
|
||||
elif action == "name-network":
|
||||
reg.name_network(body.get("network"), body.get("name"))
|
||||
if link.state["network"]:
|
||||
|
||||
@@ -53,12 +53,14 @@ SENT_KEEP = 200
|
||||
OUTBOX_MAX = 2000 # events kept while offline; the oldest go first
|
||||
FLUSH_EVERY = 60
|
||||
REPEAT_WINDOW = 600 # the same diagnostic error is sent at most once in this many seconds
|
||||
# Not faults in Frame Control: the headset asleep, away or not set up yet. The status poll
|
||||
# Not faults in Frame Control: the headset asleep, away or not set up yet, or ssh and the headset
|
||||
# not trusting each other (a host key that changed, a key not authorised): the page says what to do
|
||||
# about each (Devices tab, Set Up Connection). The status poll
|
||||
# meets these every few seconds, so each is sent at most once per session, whatever the wording,
|
||||
# but only for an error marked where ssh ran as ssh failing to reach the headset
|
||||
# (frame_host.link_failure). The message alone isn't evidence: a download's "Connection reset by
|
||||
# peer", or a file name with ssh's words in it, keeps the usual window.
|
||||
EXPECTED_CATEGORIES = ('frame_unreachable', 'frame_not_set_up')
|
||||
EXPECTED_CATEGORIES = ('frame_unreachable', 'frame_not_set_up', 'frame_auth')
|
||||
DEFAULT_HOST = 'https://us.i.posthog.com'
|
||||
|
||||
LEVELS = ('usage', 'compat', 'diagnostics')
|
||||
@@ -268,7 +270,8 @@ CATEGORIES = [
|
||||
('cant_run_on_frame', re.compile(r"can't run on the Frame")),
|
||||
('steam_shortcut', re.compile(r'(?i)steam did not return a shortcut|shortcut list|no Steam shortcut')),
|
||||
('frame_not_set_up', re.compile(r'(?i)Could not resolve hostname|no "?frame"? (?:SSH )?alias')),
|
||||
('frame_auth', re.compile(r'(?i)Permission denied|Host key verification failed')),
|
||||
('frame_auth', re.compile(r'(?i)Permission denied|Host key verification failed|'
|
||||
r'REMOTE HOST IDENTIFICATION HAS CHANGED|host key is known for')),
|
||||
# Anything ssh says about its own connection: "ssh: connect to host … port 22: <reason>" (Windows
|
||||
# says "Unknown error"), a dropped shared connection (mux_client_…, client_loop), and Windows'
|
||||
# "banner exchange: Connection to UNKNOWN port -1" (its ssh can't name a peer whose connect
|
||||
|
||||
+98
-8
@@ -700,6 +700,7 @@
|
||||
<div class="grow"><div class="t" id="offMsg">Can't reach the Frame</div>
|
||||
<div class="s" id="offDetail">Retrying every few seconds. Everything fills in when it answers.</div></div>
|
||||
<button class="small desk-only" id="offDetails" hidden>Details</button>
|
||||
<button class="small desk-only" id="offTrust" hidden>Check the new key…</button>
|
||||
<button class="small" id="offSetup" hidden>Set Up Connection…</button>
|
||||
<button class="action small" id="offRetry">Retry now</button>
|
||||
</div>
|
||||
@@ -1207,6 +1208,7 @@
|
||||
<div class="sources-intro"><div class="store-eyebrow">Your app shelves</div><h1 id="sourcesTitle">A world of apps.</h1><p>Choose where you discover games and apps. We bring them together for you.</p></div>
|
||||
<div id="sourcesList"></div>
|
||||
<form id="addSource" class="source-repo-form"><h2>Add a source</h2><p>Have a favorite F-Droid repository? Add its address to browse its apps here.</p><label>Repository address<input name="url" type="url" required placeholder="https://example.org/fdroid/repo"></label><details><summary>Verify this source (optional)</summary><label>Repository fingerprint<input name="fingerprint" type="text" autocomplete="off" placeholder="Paste the fingerprint from the repository owner"></label></details><button class="action">Add source</button></form>
|
||||
<form id="itchAccount" class="source-repo-form"><h2>My itch.io purchases</h2><p id="itchAccountNote">Install the Android builds of games you own on itch.io. Create an API key in itch.io settings › API keys and paste it here. It stays on this computer and is only sent to itch.io.</p><label id="itchKeyField">itch.io API key<input name="key" type="password" autocomplete="off" spellcheck="false" maxlength="300" placeholder="Paste your API key"></label><button class="action" id="itchConnect">Connect itch.io</button><button type="button" class="small" id="itchDisconnect" hidden>Disconnect itch.io</button></form>
|
||||
<p id="sourceError" class="store-notice" role="status" aria-live="polite"></p>
|
||||
</dialog>
|
||||
|
||||
@@ -1358,6 +1360,18 @@
|
||||
<button type="submit" class="small danger" id="rmGo">Remove</button></div>
|
||||
</form>
|
||||
</dialog>
|
||||
<dialog id="trustDlg" class="dlg" aria-labelledby="trustTitle">
|
||||
<form method="dialog" id="trustForm">
|
||||
<h2 id="trustTitle">Check the headset's new key</h2>
|
||||
<div class="sub" id="trustText"></div>
|
||||
<dl class="facts" id="trustKeys"></dl>
|
||||
<div class="sub" id="trustHow" hidden>To check it, on the headset open the desktop from the SteamVR dashboard, start
|
||||
Konsole and run <code id="trustCmd"></code>. The <code>SHA256:</code> value it prints must match the key above.</div>
|
||||
<div class="row rep-actions"><span class="sub" id="trustMsg" role="status"></span><span class="spacer"></span>
|
||||
<button type="button" class="small" id="trustCancel">Cancel</button>
|
||||
<button type="submit" class="small action" id="trustGo" disabled>This is my headset: trust its new key</button></div>
|
||||
</form>
|
||||
</dialog>
|
||||
<div class="toast" id="toast"></div>
|
||||
|
||||
<script>
|
||||
@@ -3128,15 +3142,28 @@ function sourceMark(s) { const name=storeName(s); return `<span class="offer-mar
|
||||
function sourceDescription(s) { return s.description || ({fdroid:"Independent, open-source apps",sidequest:"Discover something new in VR",itch:"Games from independent creators",github:"Direct from the people who make it"})[s.kind] || (s.name.includes("GitHub") ? "Direct from the people who make it" : s.name.includes("Droid") ? "Independent, open-source apps" : s.name.includes("SideQuest") ? "Discover something new in VR" : s.name.includes("itch") ? "Games from independent creators" : "Apps shared by this community"); }
|
||||
async function loadSources() {
|
||||
try {
|
||||
const {sources} = await api("/api/sources"); sourceState.sources = sources;
|
||||
const {sources, itch} = await api("/api/sources"); sourceState.sources = sources; showItchAccount(itch);
|
||||
$("sourceFilter").innerHTML = `<option value="">All sources</option>` + sources.filter(s=>s.enabled).map(s=>`<option value="${esc(s.id)}">${esc(storeName(s.name))}</option>`).join("");
|
||||
$("sourceFilter").value = sourceState.source;
|
||||
$("sourcesList").innerHTML = sources.map(s=>`<div class="source-setting">${sourceMark(s.name)}<div class="grow"><strong>${esc(storeName(s.name))}</strong><p>${esc(sourceDescription(s))}</p><small>${esc(({official:"Official store",community:"Community",user:"Added by you"})[s.trust] || "Community")}</small>${s.builtin === false ? `<button class="small" data-remove="${esc(s.id)}" aria-label="Remove ${esc(s.name)}">Remove</button>` : ""}</div><input class="source-toggle" role="switch" type="checkbox" aria-label="Enable ${esc(storeName(s.name))}" data-enable="${esc(s.id)}" ${s.enabled ? "checked" : ""}></div>`).join("") || `<p class="store-notice">Your sources will appear here when they are available.</p>`;
|
||||
} catch (e) { $("sourceError").textContent = e.message; }
|
||||
}
|
||||
// Owned on itch.io (the user's own purchases) or free: what the store shows instead of a price.
|
||||
const storePrice = o => o.owned ? (o.owned_label || "Owned") : "Free";
|
||||
function showItchAccount(itch) {
|
||||
const on = !!itch?.connected;
|
||||
$("itchAccountNote").textContent = on ? `Connected${itch.username ? " as " + itch.username : ""}. Android builds of games you own appear in the store, marked Owned on itch.io. Disconnecting deletes the API key from this computer.` : "Install the Android builds of games you own on itch.io. Create an API key in itch.io settings › API keys and paste it here. It stays on this computer and is only sent to itch.io.";
|
||||
$("itchKeyField").hidden = on; $("itchConnect").hidden = on; $("itchDisconnect").hidden = !on;
|
||||
$("itchAccount").elements.key.required = !on;
|
||||
}
|
||||
async function itchAccount(body) {
|
||||
$("sourceError").textContent = body.action === 'itch-connect' ? "Checking the key with itch.io…" : "Disconnecting…";
|
||||
try { const result=await api('/api/sources',body);$("sourceError").textContent=result.message;showItchAccount(result.itch);sourceState.source='';await loadSources();await searchSources(); }
|
||||
catch(e) { $("sourceError").textContent=e.message;await loadSources(); }
|
||||
}
|
||||
function storeCard(app) {
|
||||
const i=sourceState.apps.indexOf(app), o=app.offers[0];
|
||||
return `<button class="store-card" data-app="${i}" aria-label="View ${esc(app.name)}">${storeArt(o)}<span class="store-card-content"><span class="store-card-title">${storeIcon(o)}<strong>${esc(app.name)}</strong></span><span class="store-summary">${esc(storeText(app.summary) || "A new addition to your Frame")}</span><span class="store-card-foot"><span>${o.vr ? "VR experience" : o.vr === false ? "Flat-screen app" : "Android app"}</span><b>Free</b></span></span></button>`;
|
||||
return `<button class="store-card" data-app="${i}" aria-label="View ${esc(app.name)}">${storeArt(o)}<span class="store-card-content"><span class="store-card-title">${storeIcon(o)}<strong>${esc(app.name)}</strong></span><span class="store-summary">${esc(storeText(app.summary) || "A new addition to your Frame")}</span><span class="store-card-foot"><span>${o.vr ? "VR experience" : o.vr === false ? "Flat-screen app" : "Android app"}</span><b>${esc(storePrice(app.offers.find(x=>x.owned) || o))}</b></span></span></button>`;
|
||||
}
|
||||
function storeRow(title, subtitle, apps) { return apps.length ? `<section class="store-row"><div class="store-row-head"><h2>${esc(title)}</h2><p>${esc(subtitle)}</p></div><div class="store-grid">${apps.slice(0,5).map(storeCard).join("")}</div></section>` : ""; }
|
||||
function storeEmpty() {
|
||||
@@ -3181,7 +3208,7 @@ async function searchSources(quiet) {
|
||||
finally { if(request===sourceState.request)$("searchGrid").setAttribute("aria-busy","false"); }
|
||||
}
|
||||
function currentOffer() { return sourceState.detail?.app.offers[sourceState.detail.choice]; }
|
||||
function detailBadges(o) { return `<span class="store-badge">${o.vr ? "VR experience" : o.vr===false ? "Flat-screen app" : "Android app"}</span>${o.verdict?.tone==='works' ? '<span class="store-badge works">✓ Works on Frame</span>' : ''}<span class="store-badge">Free</span>${storeOpenSource(o) ? '<span class="store-badge">Open source</span>' : ''}`; }
|
||||
function detailBadges(o) { return `<span class="store-badge">${o.vr ? "VR experience" : o.vr===false ? "Flat-screen app" : "Android app"}</span>${o.verdict?.tone==='works' ? '<span class="store-badge works">✓ Works on Frame</span>' : ''}<span class="store-badge">${esc(storePrice(o))}</span>${storeOpenSource(o) ? '<span class="store-badge">Open source</span>' : ''}`; }
|
||||
// The store's app page. (Not renderDetail: the Devices page declares that further down,
|
||||
// and a later function declaration silently replaces an earlier one; issue #68.)
|
||||
// True when the page rendered; false when it shows the error instead.
|
||||
@@ -3197,7 +3224,7 @@ function renderAppDetail() {
|
||||
function renderAppDetailBody() {
|
||||
const restoreOfferFocus=document.activeElement?.name==='storeOffer';
|
||||
const d=sourceState.detail;if(!d)return;const o=currentOffer(), images=o.artwork?.screenshots||[], page=sourcePage(o.page);
|
||||
$("detailBody").innerHTML=storeArt(o,"detail-banner",true)+`<div class="detail-content"><div class="detail-heading">${storeIcon(o)}<div><h1 id="detailTitle">${esc(o.name)}</h1><p>${esc(o.developer || o.author || 'From '+storeName(o.source_name))}</p></div></div><div class="store-badges">${detailBadges(o)}</div><div class="detail-layout"><div><div class="detail-gallery" id="detailGallery" aria-label="App screenshots" tabindex="0">${images.map((p,i)=>storeArt(o,"",false,p,`${o.name} screenshot ${i+1}`)).join("")}</div>${images.length ? `<div class="gallery-controls"><span>${images.length} screenshots · scroll to explore</span><div><button data-gallery="-1" aria-label="Previous screenshot">←</button> <button data-gallery="1" aria-label="Next screenshot">→</button></div></div>` : ''}<h2>About this app</h2><div class="detail-description">${esc(storeText(o.description || o.summary || "Discover this app on its developer's page."))}</div>${page ? `<p><a href="${esc(page)}" target="_blank" rel="noopener noreferrer">Meet the creators ↗</a></p>` : ''}<details class="detail-technical"><summary>Technical details</summary><dl><dt>Package</dt><dd>${esc(o.package||"Unknown")}</dd><dt>Version</dt><dd>${esc(o.version||"Unknown")} (${esc(o.version_code??"unknown")})</dd><dt>Android API</dt><dd>${esc(o.min_sdk??"Unknown")}</dd><dt>Architecture</dt><dd>${esc(o.abis?.join(', ')||"Unknown / no native code")}</dd><dt>Updated</dt><dd>${esc(o.updated||"Unknown")}</dd><dt>Compatibility</dt><dd>${(o.fit?.reasons||[]).map(esc).join(' · ')||'No known installation blockers'}</dd></dl></details></div><aside class="detail-buy"><div class="detail-price">Free</div><p class="detail-small">No purchase needed.</p><p class="detail-verdict ${esc(o.verdict?.tone||'unknown')}">${esc(o.verdict?.label||'Not yet checked on the Frame')}</p><button id="detailInstall" class="action store-primary">Install</button><progress id="detailProgress" max="100" hidden aria-label="Installation progress"></progress><p id="detailInstallNote" class="detail-small" role="status" aria-live="polite"></p><h2>Get it from</h2><div id="detailOffers">${d.app.offers.map((v,j)=>`<label class="offer-choice"><input type="radio" name="storeOffer" value="${j}" ${j===d.choice?'checked':''}><span>${sourceMark(v.source_name)}</span><span><strong>${esc(storeName(v.source_name))}${j===0?' · Recommended':''}</strong><small>${v.verified===true?'✓ Verified download':'Verification not provided'}</small><small>${esc(v.version ? 'v'+v.version : 'Latest release')}${v.size!=null?' · '+(v.size/1048576).toFixed(0)+' MB':''}</small></span></label>`).join('')}</div></aside></div></div>`;
|
||||
$("detailBody").innerHTML=storeArt(o,"detail-banner",true)+`<div class="detail-content"><div class="detail-heading">${storeIcon(o)}<div><h1 id="detailTitle">${esc(o.name)}</h1><p>${esc(o.developer || o.author || 'From '+storeName(o.source_name))}</p></div></div><div class="store-badges">${detailBadges(o)}</div><div class="detail-layout"><div><div class="detail-gallery" id="detailGallery" aria-label="App screenshots" tabindex="0">${images.map((p,i)=>storeArt(o,"",false,p,`${o.name} screenshot ${i+1}`)).join("")}</div>${images.length ? `<div class="gallery-controls"><span>${images.length} screenshots · scroll to explore</span><div><button data-gallery="-1" aria-label="Previous screenshot">←</button> <button data-gallery="1" aria-label="Next screenshot">→</button></div></div>` : ''}<h2>About this app</h2><div class="detail-description">${esc(storeText(o.description || o.summary || "Discover this app on its developer's page."))}</div>${page ? `<p><a href="${esc(page)}" target="_blank" rel="noopener noreferrer">Meet the creators ↗</a></p>` : ''}<details class="detail-technical"><summary>Technical details</summary><dl><dt>Package</dt><dd>${esc(o.package||"Unknown")}</dd><dt>Version</dt><dd>${esc(o.version||"Unknown")} (${esc(o.version_code??"unknown")})</dd><dt>Android API</dt><dd>${esc(o.min_sdk??"Unknown")}</dd><dt>Architecture</dt><dd>${esc(o.abis?.join(', ')||"Unknown / no native code")}</dd><dt>Updated</dt><dd>${esc(o.updated||"Unknown")}</dd><dt>Compatibility</dt><dd>${(o.fit?.reasons||[]).map(esc).join(' · ')||'No known installation blockers'}</dd></dl></details></div><aside class="detail-buy"><div class="detail-price">${esc(storePrice(o))}</div><p class="detail-small">${o.owned ? "Downloads with your itch.io account." : "No purchase needed."}</p><p class="detail-verdict ${esc(o.verdict?.tone||'unknown')}">${esc(o.verdict?.label||'Not yet checked on the Frame')}</p><button id="detailInstall" class="action store-primary">Install</button><progress id="detailProgress" max="100" hidden aria-label="Installation progress"></progress><p id="detailInstallNote" class="detail-small" role="status" aria-live="polite"></p><h2>Get it from</h2><div id="detailOffers">${d.app.offers.map((v,j)=>`<label class="offer-choice"><input type="radio" name="storeOffer" value="${j}" ${j===d.choice?'checked':''}><span>${sourceMark(v.source_name)}</span><span><strong>${esc(storeName(v.source_name))}${j===0?' · Recommended':''}</strong><small>${v.verified===true?'✓ Verified download':'Verification not provided'}</small><small>${esc(v.version ? 'v'+v.version : 'Latest release')}${v.size!=null?' · '+(v.size/1048576).toFixed(0)+' MB':''}</small></span></label>`).join('')}</div></aside></div></div>`;
|
||||
updateInstall();
|
||||
if(restoreOfferFocus)document.querySelector('#detailOffers input:checked')?.focus();
|
||||
}
|
||||
@@ -3267,6 +3294,9 @@ async function addSource(body) {
|
||||
if(result){$("addSource").reset();sourceState.source='';await loadSources();await searchSources();}
|
||||
}
|
||||
$("addSource").onsubmit=e=>{e.preventDefault();const f=new FormData(e.target);addSource({action:'add',url:f.get('url'),fingerprint:f.get('fingerprint')});};
|
||||
// The key goes to the server once and leaves the page straight away: the field is cleared before the reply.
|
||||
$("itchAccount").onsubmit=e=>{e.preventDefault();const input=e.target.elements.key, key=input.value.trim();input.value='';if(key)itchAccount({action:'itch-connect',key});};
|
||||
$("itchDisconnect").onclick=()=>itchAccount({action:'itch-disconnect'});
|
||||
document.querySelectorAll('[data-close]').forEach(b=>b.onclick=()=>$(b.dataset.close).close());
|
||||
$('appDetail').addEventListener('close',()=>{sourceState.detailRequest++;});
|
||||
document.querySelectorAll('[data-store-tab]').forEach(b=>b.onclick=()=>{const browse=b.dataset.storeTab==='browse';$('appStore').hidden=!browse;$('androidLibrary').hidden=browse;$('repNew').hidden=browse;$('sourcesOpen').hidden=!browse;document.querySelectorAll('[data-store-tab]').forEach(t=>{t.classList.toggle('on',t===b);t.setAttribute('aria-pressed',t===b);});if(!browse)loadAndroid();});
|
||||
@@ -4542,6 +4572,10 @@ function renderBanner() {
|
||||
const s = link.s;
|
||||
if (!link.live || !s) return;
|
||||
$("offDetails").hidden = false;
|
||||
// A changed host key: never accepted by itself. The person checks the new key and decides.
|
||||
const hk = s.phase === "failed" && s.error && s.error.hostkey && s.device ? s.device.id : null;
|
||||
$("offTrust").hidden = !hk;
|
||||
if (hk) $("offTrust").onclick = () => openTrust(hk);
|
||||
if (s.phase === "failed" && s.error) {
|
||||
$("offline").hidden = false;
|
||||
$("offMsg").textContent = s.error.message;
|
||||
@@ -4642,6 +4676,9 @@ function onConnection(s) {
|
||||
loadDevices();
|
||||
}
|
||||
link.phase = s.phase; link.device = devId;
|
||||
// What the Devices tab offers for the failure (check a new key, pair again) follows it.
|
||||
const problem = s.phase === "failed" && s.error ? (s.error.hostkey ? "hostkey" : s.error.auth || "") : "";
|
||||
if (problem !== link.problem) { link.problem = problem; renderDevices(); }
|
||||
if (page === "devices") renderTests();
|
||||
}
|
||||
|
||||
@@ -4801,8 +4838,15 @@ function renderDetail() {
|
||||
<button class="small" data-mv="-1" title="Try earlier"${i ? "" : " disabled"}>↑</button>
|
||||
<button class="small" data-mv="1" title="Try later"${i < d.addresses.length - 1 ? "" : " disabled"}>↓</button>
|
||||
<button class="small" data-ed>Edit</button><button class="small danger" data-rm>Remove</button></div>`).join("");
|
||||
const err = d.active && link.s && link.s.phase === "failed" ? link.s.error : null;
|
||||
const problem = err && err.hostkey ? `<div class="item" role="alert"><div class="grow"><div class="t">${err.hostkey.state === "missing"
|
||||
? "Its saved SSH identity is missing" : "Its SSH identity has changed"}</div>
|
||||
<div class="s">${esc(err.message)}</div></div><button class="small action" id="dvTrustNow">Check the new key…</button></div>`
|
||||
: err && err.auth === "denied" ? `<div class="item" role="alert"><div class="grow"><div class="t">The headset didn't accept this computer's key</div>
|
||||
<div class="s">Set Up Connection pairs this computer with it again. On the headset, Developer Mode must be on.</div></div>
|
||||
<button class="small action" id="dvPair">Set Up Connection…</button></div>` : "";
|
||||
el.innerHTML = `<div class="shelf-head"><h2>${esc(d.name)}</h2>${d.active ? '<span class="count">In use</span>' : ""}<span class="spacer"></span>
|
||||
${d.active ? "" : `<button class="small action" id="dvUse">Use this headset</button>`}</div>
|
||||
${d.active ? "" : `<button class="small action" id="dvUse">Use this headset</button>`}</div>${problem}
|
||||
<form class="dev-form" id="dvForm">
|
||||
<label>Name<input type="text" id="dvName" value="${esc(d.name)}" maxlength="60"></label>
|
||||
<label>SSH alias<input type="text" value="${esc(d.alias)}" readonly title="Terminal: ssh ${esc(d.alias)}"></label>
|
||||
@@ -4823,10 +4867,12 @@ function renderDetail() {
|
||||
<button class="small" id="dvTs">Find on Tailscale</button>
|
||||
<button class="small" id="dvMdns">Find on this network</button>
|
||||
<span class="spacer"></span>
|
||||
<button class="small" id="dvForget" title="After reinstalling SteamOS the headset shows a new identity">Forget identity</button>
|
||||
<button class="small" id="dvTrust" title="After SteamOS is reinstalled or reset the headset shows a new identity">Check its key…</button>
|
||||
<button class="small danger" id="dvRemove">Remove…</button></div>
|
||||
<div class="found" id="dvFound"></div>
|
||||
<div class="hint">Identity: ${d.pinned ? "saved. A different device answering at one of these addresses is refused."
|
||||
<div class="hint">Identity: ${d.pinned ? `saved${(d.keys || []).length ? ` (${esc(d.keys.map(k => k.fingerprint).join(", "))})` : ""}.
|
||||
A different device answering at one of these addresses is refused.`
|
||||
: d.identity_lost ? "the saved key is missing or can't be read, so nothing is trusted until you check its key."
|
||||
: "not saved yet; the next connection saves it."} Kind "auto" is picked from the address:
|
||||
<code>.local</code> names are mDNS, <code>100.x</code> and <code>.ts.net</code> are Tailscale.</div>`;
|
||||
$("adKind").insertAdjacentHTML("afterbegin", `<option value="" selected>Kind: auto</option>`);
|
||||
@@ -4859,7 +4905,9 @@ function renderDetail() {
|
||||
$("dvTest").onclick = e => devAction(`Test ${d.name}'s addresses`, { action: "test", id: d.id }, e.currentTarget);
|
||||
$("dvTs").onclick = e => findOn("tailscale", d, e.currentTarget);
|
||||
$("dvMdns").onclick = e => findOn("mdns", d, e.currentTarget);
|
||||
$("dvForget").onclick = e => devAction(`Forget ${d.name}'s identity`, { action: "forget-identity", id: d.id }, e.currentTarget);
|
||||
$("dvTrust").onclick = () => openTrust(d.id);
|
||||
if ($("dvTrustNow")) $("dvTrustNow").onclick = () => openTrust(d.id);
|
||||
if ($("dvPair")) $("dvPair").onclick = () => setUpHeadset(d.alias);
|
||||
$("dvRemove").onclick = () => askRemove(d);
|
||||
renderFound(d);
|
||||
renderTests();
|
||||
@@ -4912,6 +4960,48 @@ function renderFound(d) {
|
||||
});
|
||||
}
|
||||
|
||||
// Trust a new key: read the key the headset shows now (nothing is saved), show it next to the
|
||||
// saved one, and save it only when the person says it's their headset. The server pins exactly
|
||||
// the key it showed here (frame_link.Link.trust_identity), never whatever answers next.
|
||||
const HOST_KEY_FILE = { "ssh-ed25519": "ed25519", "ssh-rsa": "rsa", "ssh-dss": "dsa" };
|
||||
async function openTrust(id) {
|
||||
const d = dv.list.find(x => x.id === id);
|
||||
const name = d ? d.name : "the headset";
|
||||
$("trustTitle").textContent = `Check ${name}'s key`;
|
||||
$("trustText").textContent = "Asking the headset for its key…";
|
||||
$("trustKeys").innerHTML = ""; $("trustMsg").textContent = "";
|
||||
$("trustHow").hidden = true; $("trustGo").disabled = true;
|
||||
$("trustCancel").onclick = () => $("trustDlg").close();
|
||||
$("trustForm").onsubmit = e => e.preventDefault();
|
||||
if (!$("trustDlg").open) $("trustDlg").showModal();
|
||||
let res;
|
||||
try { res = await api("/api/devices", { action: "identity-scan", id }); }
|
||||
catch (e) { $("trustText").textContent = e.message; return; }
|
||||
const scan = res.scan;
|
||||
const saved = (scan.saved || []).map(k => `${k.type} ${k.fingerprint}`).join("\n")
|
||||
|| ((scan.recorded || []).length ? `${scan.recorded.join("\n")}\n(recorded; the saved file is missing or unreadable)` : "None");
|
||||
$("trustKeys").innerHTML = [["Saved key", saved], ["It shows now", `${scan.type} ${scan.fingerprint}`], ["At", scan.host]]
|
||||
.map(([k, v]) => `<dt>${esc(k)}</dt><dd style="white-space:pre-wrap; word-break:break-all"><code>${esc(v)}</code></dd>`).join("");
|
||||
if (scan.matches) {
|
||||
$("trustText").textContent = `${name} shows the key already saved for it, so there's nothing to change. If it still won't connect, try Test now.`;
|
||||
return;
|
||||
}
|
||||
$("trustText").textContent = scan.recorded_match
|
||||
? `${name} shows the key recorded for it before, but the saved copy is missing or can't be read. Trust it to save it again.`
|
||||
: (scan.saved || []).length || (scan.recorded || []).length
|
||||
? `${name} now shows a different SSH key from the one saved. That's expected after SteamOS was reinstalled or reset, or if this is a new headset at the same address. If none of that happened, don't trust it: another device may be answering in its place.`
|
||||
: `No key is saved for ${name} yet. Trust this one if it's your headset.`;
|
||||
$("trustCmd").textContent = `ssh-keygen -lf /etc/ssh/ssh_host_${HOST_KEY_FILE[scan.type] || "ecdsa"}_key.pub`;
|
||||
$("trustHow").hidden = false;
|
||||
$("trustGo").disabled = false;
|
||||
$("trustForm").onsubmit = async e => {
|
||||
e.preventDefault();
|
||||
const ok = await devAction(`Trust ${name}'s new key`, { action: "identity-trust", id, fingerprint: scan.fingerprint }, $("trustGo"));
|
||||
if (ok) $("trustDlg").close();
|
||||
else $("trustMsg").textContent = "Not saved. Close this and check the key again.";
|
||||
};
|
||||
}
|
||||
|
||||
function askRemove(d) {
|
||||
$("rmDevTitle").textContent = `Remove ${d.name}?`;
|
||||
$("rmDevText").textContent = `Frame Control forgets this headset, its ${d.addresses.length} address${d.addresses.length === 1 ? "" : "es"} and its saved identity. Its SSH keys stay.`;
|
||||
|
||||
+17
-4
@@ -46,6 +46,7 @@ import frame_assistant # noqa: E402
|
||||
import frame_android # noqa: E402
|
||||
from apk_sources import search as apk_search, SourceError # noqa: E402
|
||||
from apk_sources import fdroid as apk_fdroid # noqa: E402
|
||||
from apk_sources import _itch_account as itch_account # noqa: E402
|
||||
import frame_apk_versions # noqa: E402
|
||||
import frame_catalog # noqa: E402
|
||||
import frame_devices # noqa: E402
|
||||
@@ -211,9 +212,14 @@ UNREACHABLE = [
|
||||
(re.compile(r"port \d+: Connection refused"),
|
||||
"The Frame refused the connection. Check Developer Mode is still on."),
|
||||
(re.compile(r"Permission denied \(publickey"),
|
||||
"The Frame didn't accept this computer's SSH key. Run Set Up Connection again."),
|
||||
(re.compile(r"Host key verification failed"),
|
||||
"The Frame's SSH identity changed (after a reinstall, or a different device). Run Set Up Connection again."),
|
||||
"The Frame didn't accept this computer's SSH key. Set Up Connection pairs this computer with it "
|
||||
"again (Developer Mode must be on)."),
|
||||
# Before "Host key verification failed", which ssh prints after each of these.
|
||||
(re.compile(r"No \S+ host key is known for "),
|
||||
"The Frame's SSH identity isn't saved yet. Reconnecting saves it; or check and trust it on the Devices tab."),
|
||||
(re.compile(r"REMOTE HOST IDENTIFICATION HAS CHANGED|Host key verification failed"),
|
||||
"The Frame's SSH identity has changed: after SteamOS was reinstalled or reset, or a different device "
|
||||
"answering at its address. Check the new key on the Devices tab before trusting it."),
|
||||
(re.compile(r"kex_exchange_identification|Connection closed by .* port \d+|Connection reset by .* port \d+"),
|
||||
"The connection to the Frame dropped. Try again."),
|
||||
]
|
||||
@@ -2205,6 +2211,13 @@ def source_manage(body):
|
||||
if action == 'remove':
|
||||
apk_search.manage_repo('remove_repo', source_id=source_text(body, 'source'))
|
||||
return {'message': 'Repository removed'}
|
||||
if action == 'itch-connect': # the key is checked with itch.io, saved 0600, never echoed
|
||||
key = body.get('key')
|
||||
if not isinstance(key, str) or not key.strip() or len(key) > 300:
|
||||
raise Failure('Paste your itch.io API key', 400)
|
||||
return itch_account.connect(key)
|
||||
if action == 'itch-disconnect':
|
||||
return itch_account.disconnect()
|
||||
raise Failure('Unknown source action', 400)
|
||||
except SourceError as e:
|
||||
raise Failure(str(e), 400)
|
||||
@@ -2421,7 +2434,7 @@ class Handler(BaseHTTPRequestHandler):
|
||||
except SourceError as e:
|
||||
raise Failure(str(e), 400)
|
||||
elif path == "/api/sources":
|
||||
self.send_json({"sources": apk_search.sources()})
|
||||
self.send_json({"sources": apk_search.sources(), "itch": itch_account.status()})
|
||||
elif path == "/api/search":
|
||||
self.send_json(source_search(url.query))
|
||||
elif path == "/api/apk-versions":
|
||||
|
||||
Reference in new issue
Block a user