Compare commits

..
Author SHA1 Message Date
saphidandClaude Sonnet 5.5 8fb00b263c fix(tracking): review 7 - rescan on cleanup, non-finite Health values
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:29:56 +10:00
saphidandClaude Sonnet 5.5 4937caf310 fix(tracking): don't call an unworn eye-camera artifact a pulse
Unworn Frame runs gave a steady 'clear' 90-96 BPM. pulse now reads the
proximity sensor and refuses to report when the headset is not worn.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:20:23 +10:00
saphidandClaude Sonnet 5.5 128e7a4c94 fix(tracking): never signal the eye-camera capture; finish cleanup exhaustively
Terminating eyetracking --calib left the DSP eye camera stuck streaming on the
Frame. Capture now lets the bounded run end by itself, deleting images
meanwhile, and only kills as a last resort. Also hardens Ctrl-C cleanup and
heart-check error handling.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:15:24 +10:00
saphidandClaude Opus 5.5 1a439f05c2 fix(tracking): make eye-image cleanup exhaustive and confirm capture ownership
Follow-up review findings: remove() now tries every capture directory and
reports any it could not delete; each cleanup step runs even if an earlier
one fails; the directory the capture tool reports (once its output is
flushed) must match the one we reduced, and is always removed. heart-check
keeps readings with an unrecognised flag and reports unreadable references
without a traceback.

Part of #27

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 13:07:33 +10:00
saphidandClaude Opus 5.5 016d2b8c2d fix(tracking): address independent review of the pulse experiment
- Watch for a second capture directory on every poll; stop and remove every
  directory that appeared, and fail loudly if an eye image can't be deleted.
- Start the worker pool inside the cleanup block.
- Flat patches no longer rank first (zero-power SNR is 0, not infinity).
- Align eyes to the truly nearest frame.
- Keep patch grids as float arrays (a 300 s run no longer needs ~0.4 GB).
- heart-check: tolerate unusual flags, close the Health archive, give a clear
  error when export.xml is missing, and build the lookup index once.

Found by a SWE-2 Max read-only review. Part of #27

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 12:59:13 +10:00
saphidandClaude Opus 5.5 42ec68ed51 fix(tracking): reduce eye images while capturing, with a backlog cap
The capture tool's stdout is block-buffered, so waiting for its directory name
left every image on disk until the capture ended (3,554 PNGs in a 20 s run on
the Frame). Detect the new capture directory instead, decode in three worker
processes, and stop the capture if more than 900 images wait. On the Frame a
30 s run now peaks at 7 images on disk.

Part of #27

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 12:44:51 +10:00
saphidandClaude Opus 5.5 fa49fab5bf feat(tracking): experimental eye-camera pulse estimate and heart-rate comparison tool
Adds 'pulse', which captures the IR eye cameras through SteamVR's own
eyetracking --calib mode, reduces each image to patch averages and deletes it
immediately, then estimates pulse from skin brightness. Adds heart-check.py to
show OSC readings live and compare recordings with an Apple Health export or
CSV, and a synthetic Mac BLE strap for relay tests.

Part of #27

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 12:15:05 +10:00
saphid ce9e476ec7 feat(tracking): add local OpenXR OSC and BlueZ heart-rate tools 2026-09-28 22:33:02 +10:00
Alex Southwell dcf9689f64 Merge pull request #11 from saphid/apk-alternatives
Offer older APK versions that fit when Lepton refuses an app
2026-09-28 17:38:35 +10:00
saphidandClaude Opus 5.5 224340edc9 APK alternatives: refresh the catalogue after an install job; pin the test's premise
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 17:29:29 +10:00
saphidandClaude Opus 5.5 c814cb95d0 Merge main into apk-alternatives: install other versions as background jobs
Main now runs Android installs as background jobs and maps SSH failures to
one offline message. Alternative-version installs go through the same job,
the alternatives dialog waits on it with runJob, and send_error_json keeps
the apk blocker that opens the dialog.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 17:21:55 +10:00
saphidandClaude Opus 5.5 a1fa4ce140 Fix the cross-provider review's findings on APK alternatives
One malformed or unreachable repo no longer hides the others; skip bad
index entries; a refreshed raw index outdates its reduced copy; style the
dialog like the others; validate package ids with PKG_RE.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 15:15:11 +10:00
saphidandClaude Opus 5.5 50405ccf88 Add IzzyOnDroid to APK alternatives; keep the catalogue's index file
Reducing an index no longer deletes apk-catalog/data/index-v2.json, which
the catalogue build reads. Drop the measurement log from docs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 14:30:15 +10:00
saphid 32196b4260 Reduce F-Droid indexes and fetch APK alternatives asynchronously 2026-09-28 14:26:40 +10:00
saphid fbe7ba9575 Find installable APK alternatives in F-Droid main and archive 2026-09-28 14:16:43 +10:00
29 changed files with 3133 additions and 789 deletions

No files matched your search

-2
View File
@@ -28,8 +28,6 @@ desktop or panels.
| Launch an app inside the desktop panel | the script's header comment | `scripts/run-on-frame.sh` | | Launch an app inside the desktop panel | the script's header comment | `scripts/run-on-frame.sh` |
| Mac GUI over all of this | `README.md` → Frame Control | `scripts/frame-ui.sh` | | Mac GUI over all of this | `README.md` → Frame Control | `scripts/frame-ui.sh` |
| iPhone/iPad app (server runs on the Frame, `FRAME_LOCAL=1`) | `docs/iphone.md` | `ios/`, `ui/local-bin/ssh` | | iPhone/iPad app (server runs on the Frame, `FRAME_LOCAL=1`) | `docs/iphone.md` | `ios/`, `ui/local-bin/ssh` |
| Frame unreachable, Wi-Fi dead, Steam won't start (doctor runbook) | `docs/frame-doctor.md` | — |
| Power draw, heat, fan, battery wear, quiet-mode plan | `docs/power-and-heat.md` | — |
| Recovery images, factory reset, boot loops | `docs/recovery-and-images.md`, `docs/how-the-frame-works.md` | `~/Downloads/steam-frame-recovery/` | | Recovery images, factory reset, boot loops | `docs/recovery-and-images.md`, `docs/how-the-frame-works.md` | `~/Downloads/steam-frame-recovery/` |
| Test without the headset (the Frame OS image's own sshd) | `tests/frame-container/README.md` | `tests/frame-container/frame-image.sh` | | Test without the headset (the Frame OS image's own sshd) | `tests/frame-container/README.md` | `tests/frame-container/frame-image.sh` |
| What's still unverified | `docs/open-questions.md` | — | | What's still unverified | `docs/open-questions.md` | — |
+1 -1
View File
@@ -1,7 +1,7 @@
.DS_Store .DS_Store
__pycache__/ __pycache__/
apk-catalog/data/cache/ apk-catalog/data/cache/
apk-catalog/data/index-v2.json* apk-catalog/data/index-v2*.json*
compat-db/.env.lakebed.server compat-db/.env.lakebed.server
compat-db/.lakebed/ compat-db/.lakebed/
tests/smoke/results/ tests/smoke/results/
+1
View File
@@ -204,6 +204,7 @@ Frame's software fits together, all checked against a real headset and labelled
| [SSH](docs/ssh.md) · [Streaming](docs/streaming.md) · [Files](docs/file-transfer.md) · [Panels](docs/panels.md) · [Tailscale](docs/tailscale.md) | Topic notes | | [SSH](docs/ssh.md) · [Streaming](docs/streaming.md) · [Files](docs/file-transfer.md) · [Panels](docs/panels.md) · [Tailscale](docs/tailscale.md) | Topic notes |
| [Frame Control for iPhone](docs/iphone.md) | The iPhone and iPad app, how it runs the server on the Frame, pairing | | [Frame Control for iPhone](docs/iphone.md) | The iPhone and iPad app, how it runs the server on the Frame, pairing |
| [Recovery and OS images](docs/recovery-and-images.md) | Where to download the Frame's OS, what's inside, testing without the headset | | [Recovery and OS images](docs/recovery-and-images.md) | Where to download the Frame's OS, what's inside, testing without the headset |
| [Eye tracking and heart rate](docs/tracking.md) | Our OpenXR → OSC bridge, BlueZ heart-rate panel and optional local session log; SlimeVR feasibility notes |
| [Testing](docs/testing.md) | Unit tests, end-to-end tests against a fake Frame in Docker, and the headset smoke test | | [Testing](docs/testing.md) | Unit tests, end-to-end tests against a fake Frame in Docker, and the headset smoke test |
| [Open questions](docs/open-questions.md) | What's still unchecked | | [Open questions](docs/open-questions.md) | What's still unchecked |
+27
View File
@@ -46,6 +46,33 @@ Lepton Development must be installed once. Over SSH,
`ssh frame 'steam steam://install/3056000'` queues it, but the install still `ssh frame 'steam steam://install/3056000'` queues it, but the install still
needs to be confirmed or started in the headset. needs to be confirmed or started in the headset.
## When an app needs a newer Android
Lepton is Android 11 (API 30), with arm64-v8a only. If Frame Control refuses
an APK, it shows compatible versions from F-Droid's main and archive repos and
IzzyOnDroid. It shows at most eight version names, newest first, preferring an
arm64-only build, and says how many compatible builds it found in total.
Each index is reduced to its compatible builds once a day and cached (about
16 MB). The first lookup takes about 30 s and 100 MB of memory; later ones are
instant.
Choose **Install** to download a listed version, verify its SHA-256 against
the index, and install it as its own app.
You can also inspect a file or look up a package from the command line:
```sh
python3 ui/frame_android.py info some-app.apk
python3 ui/frame_android.py versions some-app.apk
python3 ui/frame_android.py versions org.example.app
```
The search links open APKMirror, APKPure, Uptodown, F-Droid and GitHub. Pick a
version whose minimum is Android 11 or lower and that has an arm64-v8a build
(or no native code). Frame Control does not fetch APKs from those search sites.
Older versions may lack fixes, and being installable does not guarantee an
app will run: see the missing services below. Android may refuse a downgrade
or an update signed by a different publisher; removing the app deletes its data.
## Installed apps disappear when Lepton Development closes (verified 2026-09-25) ## Installed apps disappear when Lepton Development closes (verified 2026-09-25)
Lepton Development runs in a throwaway "dev" context. When it exits for any Lepton Development runs in a throwaway "dev" context. When it exits for any
-490
View File
@@ -1,490 +0,0 @@
# Frame doctor runbook
Checks and fixes for a Frame that's unreachable, crashing, or whose Steam,
SteamVR, Lepton or panels misbehave. A future `scripts/frame-doctor.sh` should
run the checks in section order, print OK, WARN or BROKEN for each, and apply
only the fixes marked **safe**. Anything marked **ask** needs the user's OK,
and anything marked **user** needs a hand on the headset.
Sources are the Frame's own journal and `coredumpctl` history (boots from
2026-09-25 to 2026-09-28) and this repo's docs. Each entry cites where it came
from. Dates are when a fact was seen. BUILD_IDs were 20260922.6101926 until
2026-09-26 and 20260925.6191901 after.
## Never do these
- `modprobe -r ath12k` on a wedged Wi-Fi chip. It oopsed the kernel on
2026-09-28 and caused the displays-broken, Steam-damaged boot in section 3.
- Leave WoWLAN armed. The next sleep breaks Wi-Fi until reboot (section 2).
- Suspend the Frame from a script. Nothing can wake it remotely (section 6).
- Let the SteamOS health checks count up to their repair. The SteamVR one
re-extracts Steam at 3 failures and tries to switch OS slots at 4 (section 4).
- Kill `gamescope` to stop a gamescope crash loop. Kill the orphaned SteamVR
processes instead (section 3).
- Write the sudo password to disk or logs.
- Leave `power.pauseCompositorOnStandby` or `power.turnOffScreensTimeout`
changed after testing (section 3).
- Force a power-off (holding Power) or reset while Steam is extracting or
repairing. That's how files got truncated on 2026-09-28. Use an orderly
`systemctl reboot`/`poweroff` or the power menu. Holding Power is for an
unresponsive Frame, with the user's involvement.
- Run long diagnostics while a Steam or SteamVR restart loop is live without
freezing the health-check trackers first (section 4). The repair threshold is
3 SteamVR failures, and a loop reaches it in under a minute.
## 0. Reaching the Frame
| Path | How | Works when |
|---|---|---|
| Tailscale | `ssh frame` (`frame.<tailnet>.ts.net`) | Wi-Fi up, and Tailscale on the Mac and the Frame |
| LAN | `ssh -o HostName=192.168.1.237 -o HostKeyAlias=frame.<tailnet>.ts.net frame`, or `frame.local` | Wi-Fi up. The alias avoids "Host key verification failed" |
| USB-C | Same, with `HostName=10.86.200.233` | Cable to the Mac, even with Wi-Fi dead. The Mac gets `en9` "Steam Frame" 10.86.200.234/29 (`networksetup -listallhardwareports`) |
| ADB over USB-C | `adb -s frame shell` | SSH refused, for example after Developer Mode was lost ([how-the-frame-works.md](how-the-frame-works.md), boot-loop row) |
- **Asleep means off the network (verified 2026-09-27, unreachable for about
2.5 h).** Every path times out and nothing remote wakes it
(section 6). **user**: press power. `tailscale status | grep frame` on the
Mac shows "offline, last seen N ago".
- **`frame` alias doesn't resolve.** The Mac's Tailscale is off. Use
`frame.local` ([tailscale.md](tailscale.md)). Bare `frame` doesn't resolve on
macOS. Check with `dns-sd -G v4 frame.local` ([ssh.md](ssh.md)).
- **Pairing answers `403 "please put the Steam client in pairing mode"`.**
**user**: Steam, then Settings → Developer → Pair new host. `connect.sh`
retries for 2 min ([ssh.md](ssh.md)).
- **iPhone app can't use devkit pairing.** It only installs an RSA key, and
Citadel signs RSA with SHA-1, which OpenSSH 9.7 rejects. Use ed25519 and
password pairing instead ([ssh.md](ssh.md), 2026-09-27).
- **Locked out after `connect.sh --harden`.** Undo with `sudo rm
/etc/ssh/sshd_config.d/01-frame-keys-only.conf && sudo systemctl reload sshd`
(**ask**, over USB-C or ADB) ([ssh.md](ssh.md)).
- **No SSH at all (Developer Mode off).** Run `scripts/serve-bootstrap.sh`, and
the **user** types `curl -fsS mac.local:8765|bash` in Konsole. Stop the
server afterwards, because it's plain HTTP ([ssh.md](ssh.md)).
- **Tailscale exposes every loopback port** (8080 Steam DevTools, 5555
unauthenticated ADB, 27062, 3389) to the tailnet. Check read-only with
`~/.local/bin/tailscale debug prefs | grep ShieldsUp` (the CLI isn't on `PATH`; verified 2026-09-28) and the tailnet ACLs. Report it
as a WARN. `~/.local/bin/tailscale set --shields-up` is a mitigation, not a check, and it
also blocks inbound SSH over Tailscale, so it's **ask**, and only with
another way in available ([tailscale.md](tailscale.md)).
- **sudo:** `printf '%s\n' "$PW" | ssh frame 'sudo -S -p "" …'`. It's the
password the user set on the Frame.
## 1. Boot and crash history
```sh
ssh frame 'uptime; journalctl --list-boots --no-pager | tail -n 6'
ssh frame 'journalctl -b -1 -k --no-pager -q | grep -aE "Unable to handle kernel|Internal error|Kernel panic" | tail -n 3'
ssh frame 'coredumpctl list --no-pager --since -1d'
```
- **Kernel oops in the previous boot.** Report "oops observed". An oops alone
doesn't prove a reset, because Linux can keep running after one. Classify the
reset as unclean only if the oops is among the last lines of that boot and no
shutdown lines follow
(`journalctl -b -1 -q -n 30 | grep -aE "systemd-shutdown|Reached target.*(Reboot|Power)"`
is empty). On 2026-09-28 the oops was the last thing logged at 20:57:53. After
an unclean reset, check sections 3 and 4 closely.
- **A boot ending with no shutdown lines and no errors.** On 2026-09-26 there
were five boots of 0–12 min like this (−12, −9, −8, −7, −5), with nothing
failing beforehand. They were probably hard power-offs during setup. Treat
them as unexplained, not as crashes.
- **Crash signatures seen so far** (all `coredumpctl`, UID 1000):
| When | What crashed | Cause | Section |
|---|---|---|---|
| 09-25 21:02–21:03 | vrcompositor SEGV, steamwebhelper SEGV, then Android composer, surfaceflinger and gamescope ABRT | Lepton crash cascade. Two `pasta` processes were both failing to listen on port 16385 just before | 7 |
| 09-25 22:30–22:42 | `app_process64` ×3 | Android apps during APK testing | 7 |
| 09-25 23:20 | `ffmpeg` | hardware H.264 encoder | 10 |
| 09-26 13:56–22:45, 09-27 09:51 | `chromium-xr/chrome` ×16 | Chromium XR (panels, Mac view) | 8 |
| 09-26 21:11–21:49 | XRService ABRT ×9, vrcompositor SEGV ×5, gamescope ABRT ×4 | leftover SteamVR processes from a failed start (29 Steam restarts, 31 SteamVR failures that boot) | 3 |
| 09-28 16:27–17:49 | `app_process64` ×4 | Android runtime amid `binder_user_error` floods | 7 |
| 09-28 17:01 | `kdeconnectd` | SMS plugin during device teardown | 9 |
| 09-28 20:58–21:39 | vrcompositor SEGV ×10, XRService ×15, steamwebhelper ×2 | broken displays after a kernel oops | 3 |
Per-boot counters a doctor should print:
```sh
ssh frame 'for b in 0 -1; do
k=$(journalctl -b $b -k -q) || { echo "boot $b: journal unreadable"; continue; }
u=$(journalctl -b $b --user -u steam.service -q) || { echo "boot $b: user journal unreadable"; continue; }
s=$(journalctl -b $b -q) || { echo "boot $b: journal unreadable"; continue; }
echo "boot $b dsi=$(grep -ac "wait for video done" <<<"$k") steam_restarts=$(grep -ac "Scheduled restart" <<<"$u") steamvr_fail=$(grep -ac "steamvr.service: Failed" <<<"$s")"
done'
```
Report an unreadable journal as unknown, not as zero. What matters is
whether the counts are **still rising**, so run it twice a minute apart. One
or two SteamVR start failures around boot are normal
([how-the-frame-works.md](how-the-frame-works.md), boot-loop row). Healthy
boots on 2026-09-28 were 0 / 0 / 0. The 2026-09-26 21:22 boot reached
0 / 29 / 31 (leftover processes), and the 2026-09-28 20:58 boot reached
424 / 61 / 62 (broken displays), both rising every ~15 s.
## 2. Wi-Fi
| Check | Healthy | Broken |
|---|---|---|
| `nmcli -t d \| grep ^wlan0` | `wlan0:wifi:connected:…` | `wlan0:wifi:unavailable:` |
| `journalctl -b -k \| grep -a ath12k` | none, or a few at boot | `failed to wakeup from wow: -110`, `Resuming from non M3 state (RESET)`, `WMI_PDEV_SET_PARAM_CMDID timeout`, `fail to start mac operations` |
| `iw phy phy0 wowlan show` | `WoWLAN is disabled.` | `wake up on magic packet` |
- **WoWLAN armed (safe).** Disarm it by UUID, because the user may have made
same-name duplicates. `default` means "use NetworkManager's global
`wifi.wake-on-wlan`". The Frame sets none (checked 2026-09-28), so it falls
back to `ignore`, which leaves the chip untouched and doesn't clear an armed
chip. `0` disarms it:
```sh
set -e
U=$(nmcli -t -f UUID,DEVICE c show --active | awk -F: '$2=="wlan0"{print $1}')
[ -n "$U" ] || { echo "no active connection on wlan0"; exit 1; }
nmcli -g 802-11-wireless.wake-on-wlan c show "$U" # record the old value
systemd-run --user --wait --pipe -q nmcli c modify "$U" 802-11-wireless.wake-on-wlan 0
systemd-run --user --wait --pipe -q nmcli device modify wlan0 802-11-wireless.wake-on-wlan 0
iw phy phy0 wowlan show | grep -q "WoWLAN is disabled" || { echo "still armed"; exit 1; }
```
Use the **active** connection on wlan0, because there can be same-name
duplicates. `c modify` saves the setting. `device modify` changes only
WoWLAN on the live device, unlike `device reapply`, which would also apply
any other saved changes such as IP or DNS. Tested 2026-09-28: Wi-Fi stayed
connected. NetworkManager only allows the
modify under `systemd-run --user`. From SSH it's `auth`. Leave the profile
at `0`, since that stays safe even if a global `wifi.wake-on-wlan` is added
later. Setting the recorded old value back is **ask**. On 2026-09-28 the
profile was set back to `default` by hand. If the Wi-Fi is `unavailable`,
there's no active connection, so this has to wait for the reboot, and then
arming comes from the profile.
- **`unavailable` after resume (user/ask).** Do a **clean** reboot: power
menu, or `sudo systemctl reboot` over USB-C. Never reload the module.
- **Duplicate "ThisIsTheWifi" profiles.** Ones with `TIMESTAMP-REAL` `never`
are unused. Deleting them is **ask**.
## 3. Displays, SteamVR, gamescope
| Check | Healthy | Broken |
|---|---|---|
| `journalctl -b -k \| grep -ac "wait for video done"` | `0` | hundreds (`msm_dsi ae94000.dsi / ae96000.dsi`) |
| `coredumpctl list vrcompositor --since -10min` | none | SEGV every ~15 s |
| `grep -a "failed to wait for present" ~/.local/share/Steam/logs/vrcompositor.txt` | none recent | `WaitForPendingPresent: failed to wait for present` |
| `journalctl -b --user -u steamvr.service \| grep -a "left-over process"` | none | `Found left-over process … (vrserver) … (vrcompositor) in control group` |
| journal `gamescope` | quiet | `rendervulkan.cpp:2181 … Assertion '!modifiers.empty()'` about once a second |
- **Broken displays (DSI timeouts).** The chain is: the GPU can't present,
vrcompositor SEGVs on its first frame, `steamvr.service` fails and stops the
gamescope VR session, and gamescope and Steam get SIGKILLed. The user sees
"There was an issue launching Steam". Fix (**ask/user**): a **clean**
reboot. An unclean reset after a kernel oops caused it, and the clean reboot
had 0 DSI errors (2026-09-28). Don't touch Steam while this is happening.
- **Leftover SteamVR processes (2026-09-26 21:22 boot).** A first
`steamvr.service` start failed on `dependency`, its vrserver, XRService and
vrcompositor kept running, and each restart crashed against them. The same
fix as the next item applies, with the same guard.
- **gamescope crash loop on `!modifiers.empty()`** (verified 2026-09-25,
[apks.md](apks.md)). gamescope keeps attaching to SteamVR processes orphaned
from a dead session. The broad fix is
`for p in vrdashboard vrcompositor vrserver; do pkill -TERM -x $p; done`,
and it recovers within about a minute. That kills **every** matching
process, including a working session, so it's always **ask**. A doctor may
signal automatically only **individually verified stale PIDs**, and only
when **all** of these hold:
- The loop is live: new vrcompositor/gamescope crashes in the last 2
minutes, and `NRestarts` rising between two reads.
- The process started before the current `steamvr.service` main process:
compare `ps -o pid,lstart,args -C vrserver,vrcompositor,vrdashboard`
with `systemctl --user show steamvr.service -p ExecMainStartTimestamp`.
- The journal ties it to the failed run:
`Found left-over process <pid> (…) in control group`.
Re-read `/proc/<pid>/stat` start time and `comm` just before signalling, and
signal by number, never by name. A process that's merely outside
`steamvr.service`'s cgroup could be a legitimate launch, so that's **ask**.
- **Standby test settings left on.** Check that `vrcmd --get-settings`
(or `~/.config/openvr/config/steamvr.vrsettings`) shows
`power.pauseCompositorOnStandby` = 1 and `power.turnOffScreensTimeout` = 5.
If they differ, report a WARN and leave them alone (**ask**), since the user
may want them. If a doctor run changes them for a test, it must snapshot both
values first, including whether they were set in `steamvr.vrsettings` at all.
Afterwards it restores exactly those values, removing the keys if they were
absent, rather than the defaults 1 and 5.
The bool setter needs `1`/`0`, not `true`
([how-the-frame-works.md](how-the-frame-works.md)).
- **Dashboard open over an app** (`visible-blurred` just means it's open).
Run `SteamClient.OpenVR.VROverlay.HideDashboard()` over CDP on port 8080
only when the doctor itself is driving an app test. Otherwise it's **ask**,
because the user may have opened it.
- **Steam launch stuck in standby** at `ShowInterstitials`/`CreatingProcess`
(`console_log.txt`). Run `SteamClient.Apps.ContinueGameAction(<action id>,
"<appid>", "<task>")` over CDP.
## 4. Steam client and the SteamOS health checks
| Check | Healthy | Broken |
|---|---|---|
| `systemctl --user show steam.service -p NRestarts` | `0` or stable | climbing every ~15 s |
| `tail -n 40 ~/.local/share/Steam/logs/connection_log.txt \| grep -a "Logged On"` | `[Logged On, …] [U:1:<id>]` | only `[Logged Off, 0, 0] [U:1:0]` |
| `grep -a BVerifyInstalledFiles ~/.local/share/Steam/logs/steam_output.log` | none | `<file> is N bytes, expected M`, `bad symlink …` |
| last line of `steam_output.log` | client running | `Installing update...` or `Extracting package...` for minutes |
| `pgrep -af child-update-ui` + `/proc/<pid>/wchan` | none | `drm_syncobj_array_wait_timeout` |
| `cat /run/user/1000/steam{,vr}-short-session-tracker; ls -l` those files | empty | `frog…` / `frog:glasses:…` building up |
Check section 3 first. If the displays are broken, Steam can't get past its
first frame, whatever the files look like.
**The two health checks** (read from `/usr/share/deckard/`, 2026-09-28):
- `steam-health-check` (run by `steam.service`) appends `frog` to
`steam-short-session-tracker` for each run that fails in under 120 s or lasts
under 5 s. At 5 it runs `do_repair`. On BUILD_ID 20260925.6191901 the
script deletes `~/.steam` (keeping `registry.vdf`) and then either extracts
`/usr/lib/steam/steam.tar.zst` (705 MB) over `~/.local/share/Steam` (the
"unpacked" install this Frame has) or, on an overlay install, deletes the
upper-dir files that shadow `/usr/local/steam`. Then it touches
`.install-complete`. It also repairs at **every Steam start** if
`.install-complete` is missing, whatever the counter says.
- `steamvr-health-check` (run by `steamvr.service`) appends `frog:glasses:`
for each failed or under-10-s SteamVR run. At 3 it runs `steam-health-check
--repair-now`. At 4 it also runs `steamos-bootconf set-mode reboot-other`,
which fails as non-root.
- **What a repair erases isn't consistent across notes.** On 2026-09-26
(BUILD_ID 20260922.6101926) the boot-loop row in
[how-the-frame-works.md](how-the-frame-works.md) records that all of
`~/.local/share/Steam` was deleted, including games, login and Developer
Mode. On 2026-09-28 the scripts above only extract over it, a repair ran at
21:13 (`.install-complete` mtime), and the login survived. Treat any repair
as possibly destructive. Before a restart that could trigger one, check that
`.install-complete` exists.
- **Stop them counting while you fix the cause (safe, resets at boot).** Do
this **first**, right after connecting, if `NRestarts` or either tracker is
rising, before any long checks:
```sh
rc=0
for f in /run/user/1000/steam-short-session-tracker /run/user/1000/steamvr-short-session-tracker; do
{ [ -e "$f" ] || : > "$f"; } && chmod u+w "$f" && : > "$f" && chmod 444 "$f" || rc=1
# verify: empty and not writable
[ -e "$f" ] && [ ! -s "$f" ] && [ ! -w "$f" ] && echo "frozen $f" || { echo "NOT frozen $f"; rc=1; }
done
exit $rc
```
A doctor must stop and not restart Steam or SteamVR unless this exits 0.
It's idempotent, so run it on files that are already 444. Both were
already 444 on 2026-09-28, applied by an earlier session. This only stops
the **counting**. The start-time repair when `.install-complete` is missing
still runs. Re-apply after every reboot while the loop's cause is unfixed.
Fixes:
- **Verify files yourself (safe, read-only).** The record is
`~/.local/share/Steam/package/steam_client_<branch>_linuxarm64.installed`,
with lines of `path,size;mtime;crc32` (size `-1` is a directory). Compare
sizes and `zlib.crc32` (13,518 files on 2026-09-28). `steam_output.log` is
rewritten on every launch, so copy it before the next restart. `bad symlink`
reports taken mid-extraction are transient.
- **Truncated files (ask).** Restart Steam (`systemctl --user restart
steam.service`), and it re-verifies and re-extracts from `package/`.
Preconditions:
- The displays are healthy.
- The trackers are frozen.
- `.install-complete` exists.
- The updater is idle: the `steam_output.log` tail hasn't changed for 60 s
and the steam process isn't writing (`/proc/<pid>/io` `write_bytes` is
steady).
- No game or app is running.
Re-verify afterwards.
- **Updater deadlocked on the update UI.** Kill only the `-child-update-ui`
process, and the install continues (worked 2026-09-26). On 2026-09-28 it
was followed by a truncated `steamui.so`, so re-verify afterwards. If the
deadlock came from broken displays, fix those first.
- **Stale pending install (ask, with backup).** `package/steam_client_<branch>_linuxarm64`
with no extension means an install is pending. Only act when all of these hold:
- `cmp` shows it's identical to `.manifest`.
- The verify is clean.
- The updater is idle (as above).
Then stop Steam, move it to `~/.cache/frame-control/…pending-backup`, and
start Steam. The log should
show `Nothing to do`, then `Verification complete`, then webhelpers.
- **Heavy repair (ask).** `steam-health-check --repair-now`, or the boot
menu's `Repair Steam Installation` (section 12).
- **Dead ends (don't repeat).**
- `STEAM_EXTRA_ARGS=-no-child-update-ui` still draws GLX in-process and
blocks.
- With `DISPLAY` unset, Steam exits ("XOpenDisplay failed"), with no text
fallback.
- Xvfb has no GLX visual here.
- Steam's launch path is `steam.service` → `/usr/share/deckard/select_steam.sh
RUNSTEAM.sh`. Runtime drop-ins in `/run/user/1000/systemd/user/steam.service.d/`
are cleared at reboot. Remove any you add.
- **`create-shortcut` refuses ids with hyphens** (`missing/invalid arguments`).
Ids must match `^[A-Za-z_][A-Za-z0-9_.]+$`. It reports "Steam client is not
running" when Steam is down, and re-running finishes the install without a
re-upload ([sideloading.md](sideloading.md)).
## 5. Idle sleep and keep-awake
- **Frame slept mid-task.** SSH doesn't count as activity, and Steam's idle
timer (`system_idle_suspend_ac_sec` 3600, `…_battery_sec` 900) suspends it.
The journal shows `Switching to power state: [ k_ESystemPowerState_Sleep ]`.
Fix (**safe**): `scripts/keep-awake.sh on` before long work and `off` after.
It uses one shared unit and one saved-settings file. So a doctor records
whether `fc-keep-awake` was already active, and runs `off` only if it was
the one that turned it on. Otherwise it releases another task's lock and
restores that task's saved timers.
It sets both timers to 0 and holds the `fc-keep-awake` user-unit inhibitor.
A plain SSH-session inhibitor is refused.
- **Check:** `systemctl --user is-active fc-keep-awake`,
`systemd-inhibit --list | grep "Frame Control"`. WARN if it's held with no
agent working, since that drains the battery on battery power.
- **Charging shows "Discharging" at ~0 W while full on a charger.** This is a
reporting quirk. Treat under 0.5 W on a charger as "not charging"
([how-the-frame-works.md](how-the-frame-works.md)).
## 6. Remote wake
It doesn't work on this build. WoWLAN arms, but the WCN7850 is reset in both
`deep` and `s2idle`, packets don't wake it, and Wi-Fi is dead after resume.
Tested 2026-09-28. Details are in [how-the-frame-works.md](how-the-frame-works.md)
and [open-questions.md](open-questions.md). Doctor checks: `cat
/sys/power/mem_sleep` should read `s2idle [deep]` (resets at boot), and WoWLAN
should be disabled.
## 7. Lepton (Android)
| Check | Broken sign |
|---|---|
| `podman ps --format '{{.Names}} {{.Ports}}'` | two containers with the same name or instance, or both bound to the same host port. Several instances with different ports are normal ([apks.md](apks.md)) |
| journal `pasta` | `Listen failed for HOST TCP port 0.0.0.0/16385: Address already in use` repeating |
| journal | `android.hardware.graphics.composer@2.1-service` or `surfaceflinger` aborts right after an app crash |
| `dmesg` / journal | floods of `binder_user_error: N callbacks suppressed` near `app_process64` crashes |
| journal | `Clearing baked app data due to non steamlaunch container` |
- **Duplicate Lepton containers or port clash (2026-09-25 21:01).** Two
`pasta` instances fought over 16385, and a minute later the compositor,
webhelper, Android composer, surfaceflinger and gamescope crashed together.
Fix (**ask**): find the two instances that share the port (`podman ps`,
`ss -ltnp | grep 16385`) and stop only the duplicate. Leave other instances
running.
- **Lepton's graphics HAL aborts after an app crash, taking the container down**
(3 times on 2026-09-25). It's intermittent, so retry ([apks.md](apks.md)).
Then check section 3 for a gamescope loop.
- **All ADB-installed apps gone.** Lepton Development wipes its data whenever
it exits outside a Steam launch. Use `install-apk.sh` (a per-app Steam
launch, whose data survives), or the launch option `LEPTON_NO_CLEANUP=1
%command%` (inferred) ([apks.md](apks.md)).
- **App dies on first file write with `ENOENT`.** Its `STEAM_COMPAT_DATA_PATH`
is outside `~/.local/share/Steam`. Use `steamapps/compatdata/<id>`.
- **Lepton won't start outside Steam.** Set `IS_PARENT=true` and use `setsid
--wait`. "unbound variable" means `STEAM_COMPAT_SHADER_PATH` is unset
([apks.md](apks.md)).
- **App compatibility, not a fault** ([apks.md](apks.md)):
- Compose older than 1.11, SDL2/Kivy and Godot 4.3 crash on the missing
clipboard service.
- `INSTALL_FAILED_OLDER_SDK` means minSdk is over 30.
- `INSTALL_FAILED_NO_MATCHING_ABIS` means there's no arm64 build.
- `monkey` returning `-5` means you should launch the activity directly.
`--brief` prints a metadata line first, so take the last line:
`adb -s $S shell am start -W -n "$(adb -s $S shell cmd package resolve-activity --brief -c android.intent.category.LAUNCHER <pkg> | tail -n 1)"`.
## 8. Chromium XR (panels, Mac view, WebXR)
- **16 crashes on 2026-09-26/27.** The logs showed `Failed to create a
temporary file for memory-mapping: No such process (3)`, then `Received
signal 11 SEGV_MAPERR`. The cause isn't known yet. Check with
`coredumpctl list chrome --since -1d`.
- **Zygote crash about 30 s after a Steam-launched start.** Steam's
`gameoverlayrenderer.so` is in `LD_PRELOAD`, and the launcher strips it
(verified 2026-09-27, [webxr-chromium.md](webxr-chromium.md)). Check that
the running chrome's `/proc/<pid>/environ` has no `gameoverlayrenderer`.
- **XR process seccomp crash (syscall 209) or `VRInitError_Init_Internal`.**
The launcher runs with `--disable-seccomp-filter-sandbox`. Use that profile
only for VR sites ([webxr-chromium.md](webxr-chromium.md)).
- **Mac view kept working when Steam was down** (2026-09-28). It's a separate
Chromium talking to the Mac over the LAN. It's a useful way in when Steam is
broken, but it's killed by any reboot and needs relaunching.
## 9. KDE Connect
- **`kdeconnectd` crashed on 2026-09-28 17:01** in `kdeconnect_sms.so` under
`Device::~Device` (device teardown). Check whether it's still running with
`pgrep -f frame-control/kdeconnect/root/usr/lib/kdeconnectd`. Fix
(**safe**): restart it the way this repo launches it. A doctor should
report a missing daemon rather than guess.
## 10. Streaming and capture
- **`ffmpeg` crash with `h264_v4l2m2m`** (hardware encoder, 2026-09-25/26).
Use `libx264 -preset ultrafast -tune zerolatency`
([how-the-frame-works.md](how-the-frame-works.md)).
- **`vrcmd --screenshot` writes nothing.** Use `ui/frame_vrshot.py`
(`IVRScreenshots`).
- **No Mac cursor in the VNC mirror.** Load `scripts/mac-cursor-ring.lua` in
Hammerspoon on the Mac (`dofile(".../scripts/mac-cursor-ring.lua")` in
`~/.hammerspoon/init.lua`). It isn't a standalone script. Toggle it with
ctrl+alt+cmd+M.
- **Remmina asks for the Mac login password.** macOS offers RFB type 30
first. Seed the password with `--update-profile … --set-option password`
([streaming.md](streaming.md)).
## 11. Panels
- **Window stays on the default panel.** Run one `panel-on-frame.sh` at a
time. Tag windows by hand with `DISPLAY=:0 xprop -id <win> -f STEAM_GAME 32c
-set STEAM_GAME <id>` ([panels.md](panels.md)).
- **Single-instance apps** (Remmina, KDE). Close them in Plasma first.
- **Wayland-only apps** can't be floated this way.
- **Dragging selects instead of scrolling.** That's by design for tagged
windows (laser mode).
- **`Failed to get app info`** for a made-up id is benign.
## 12. Boot loop, recovery, re-image
Work down this list, least destructive first ([recovery-and-images.md](recovery-and-images.md)).
The boot menu is inferred from Valve's docs and hasn't been tried on this Frame.
1. **If the Frame is reachable, freeze the health-check trackers first and
verify them** (section 4), then diagnose. A reboot clears the freeze and
restarts the failing services, and 3 SteamVR failures trigger a repair.
2. **Clean reboot** only when the diagnosed fault needs one (broken displays,
dead Wi-Fi). Straight after reconnecting, freeze and verify the trackers
again before anything else.
3. **Boot menu (user):** shut down cleanly if the Frame responds. Hold Power
~10 s until the LED is off only if it doesn't, and never while Steam is
extracting or repairing. Then power on holding **AUX** (top button). Choose `Previous` (the other A/B slot, keeps data),
then try `Repair Steam Installation`.
4. **`Erase User Data`** wipes `~`: SSH keys, Tailscale, Flatpaks and setup
(**ask**).
5. **Re-image** with `steamframe-oobe-repair-<build>` over USB or cable/EDL
(`qdl`). Copies are in `~/Downloads/steam-frame-recovery/` (**ask**).
## What a doctor script should do
1. Find a path (Tailscale, then LAN, then USB), and report which one worked.
2. Print uptime, the last boots, and whether the previous boot ended in an
oops.
3. Run the read-only checks in sections 2–11 and print one line each: OK,
WARN or BROKEN.
4. **Order matters.** If a restart loop is live (`NRestarts` or a tracker
rising between two reads a few seconds apart), freeze the trackers
**before** any other check. Then fix displays before Steam. After any
reboot, check the trackers again, because they reset.
5. Apply only **safe** fixes, printing each command. For reboots, deleting
profiles, heavy repairs and anything touching a user profile: print the
fix and ask.
6. Never do anything under "Never do these".
7. Re-run the checks after any fix and report the before and after.
8. The fake-Frame harness (`fakeframe-ctl sleep|disk-full|sshd|devkit-service|keys`,
[testing.md](testing.md)) can exercise the unreachable, disk-full and
no-SSH branches without a headset.
## Incident 2026-09-28
| Time | What happened |
|---|---|
| 19:36 | WoWLAN armed, `deep` suspend, magic packets sent. No wake. Power-button resume: chip in MHI RESET, Wi-Fi dead. User restarted. |
| 20:10 | WoWLAN disarmed. Clean boot, no DSI errors. |
| 20:29 | `s2idle` via sudo, WoWLAN re-armed, suspend. No wake, same chip reset, Wi-Fi `unavailable`. |
| 20:57:53 | Over USB-C: `modprobe -r ath12k`, and the **kernel oopsed** and the Frame reset itself. |
| 20:58 | Boot with `steamclient.so` truncated (18.6 of 50.3 MB) and DSI timeouts from +28 s. Steam "couldn't connect", then "There was an issue launching Steam". Mac view still worked. |
| 21:00–21:13 | Steam re-extracts and hangs on "Installing update..." (update UI stuck on the GPU). Killing the UI child let it continue, and `steamui.so` was later found truncated. The health-check repair ran at 21:13. |
| 21:28 | Own CRC check: all 13,518 files OK. |
| 21:30 | Moved aside the identical pending manifest. Steam reached login, then died every ~15 s: vrcompositor SEGV on DSI timeouts. |
| 21:39 | User did a clean reboot. 0 DSI errors, Steam logged on 21:40:30, NRestarts 0. |
+9 -2
View File
@@ -20,6 +20,15 @@ SteamVR (vrserver, vrcompositor, vrdashboard) ← renders the room + pane
Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 3056000 Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 3056000
``` ```
## Tracking additions (verified 2026-09-28)
On SteamOS 0.4.1, build `20260925.6191901`, SteamVR exposes combined gaze
through `XR_EXT_eye_gaze_interaction` in a headless OpenXR 1.0 session. Our
reader obtained valid tracked samples and sent OSC to a configured loopback
receiver. BlueZ LE discovery works; GTK4/GI can render our heart-rate panel.
No BLE strap or SlimeVR trackers were attached. See [tracking](tracking.md)
for the evidence, privacy defaults and untested integration boundaries.
## Facts worth knowing ## Facts worth knowing
| Fact | Where it matters | | Fact | Where it matters |
@@ -55,8 +64,6 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600
| **Tools on the image:** Python 3.12.3, `ffmpeg`, `openssl`, `curl`, `rsync`, `zip`/`unzip`, `flatpak`, `wpctl`, `podman`. **No `adb`.** `steamos` is uid 1000, in `wheel`, and sudoers has `%wheel ALL=(ALL) ALL`, so `sudo -S` takes the Developer Mode password on stdin. **Verified 2026-09-27.** | Running Frame Control's server on the Frame (`FRAME_LOCAL=1`, [iphone.md](iphone.md)) | | **Tools on the image:** Python 3.12.3, `ffmpeg`, `openssl`, `curl`, `rsync`, `zip`/`unzip`, `flatpak`, `wpctl`, `podman`. **No `adb`.** `steamos` is uid 1000, in `wheel`, and sudoers has `%wheel ALL=(ALL) ALL`, so `sudo -S` takes the Developer Mode password on stdin. **Verified 2026-09-27.** | Running Frame Control's server on the Frame (`FRAME_LOCAL=1`, [iphone.md](iphone.md)) |
| **Each Lepton instance is a podman container** named `lepton-steamlaunch-<instance id>`, labelled with its ADB port (`podman ps --format '{{.Names}} {{.Labels.adb_port}}'`). `podman exec <container> /system/bin/sh -c '…'` runs Android's shell inside it with no adb at all (used for `pidof` and `logcat` by the app tester). Running `wm size`/`wm density` that way is untested. **Verified 2026-09-27.** | `ui/frame_android.py`, the iPhone app's display settings | | **Each Lepton instance is a podman container** named `lepton-steamlaunch-<instance id>`, labelled with its ADB port (`podman ps --format '{{.Names}} {{.Labels.adb_port}}'`). `podman exec <container> /system/bin/sh -c '…'` runs Android's shell inside it with no adb at all (used for `pidof` and `logcat` by the app tester). Running `wm size`/`wm density` that way is untested. **Verified 2026-09-27.** | `ui/frame_android.py`, the iPhone app's display settings |
| **Asleep means off the network.** In standby the Frame stops answering on its LAN address, `frame.local` and Tailscale alike (`Host is down`, `No route to host`, timeouts), and ping fails. It was unreachable for about 2.5 hours until woken. Nothing over SSH can wake it. **Verified 2026-09-27.** | Frame Control's offline banner and retries | | **Asleep means off the network.** In standby the Frame stops answering on its LAN address, `frame.local` and Tailscale alike (`Host is down`, `No route to host`, timeouts), and ping fails. It was unreachable for about 2.5 hours until woken. Nothing over SSH can wake it. **Verified 2026-09-27.** | Frame Control's offline banner and retries |
| **What puts it to sleep is Steam's idle timer**, not logind. The journal shows `steamui_system: Switching to power state: [ k_ESystemPowerState_Sleep ] reason: 'ComputeNextPowerState: active: 3600 < 3600 (k_EACState_Connected)'`, then Steam suspends. SSH work doesn't count as activity. The timers are the client settings `system_idle_suspend_ac_sec` (3600) and `system_idle_suspend_battery_sec` (900); 0 means Never (Settings → Power → Sleep after inactivity). They can be written over DevTools the way the settings page does. logind refuses a `systemd-inhibit --mode=block` sleep lock from an SSH session (`Interactive authentication required`) but accepts one started with `systemd-run --user`. `scripts/keep-awake.sh on|off|status` does both and restores the old timers on `off`. **Verified 2026-09-28**, BUILD_ID 20260925.6191901. Whether Steam's suspend honours the inhibitor on its own is **inferred** (polkit gives `steamos` no `suspend-ignore-inhibit`), not tested. | Keeping the Frame awake for agent work |
| **Wake-on-WLAN doesn't work from `deep` or `s2idle`, and leaving it on breaks Wi-Fi after resume. Leave it off.** Sleep is `PM: suspend entry (deep)` (`/sys/power/mem_sleep` = `s2idle [deep]`). The Wi-Fi is a WCN7850 on `ath12k_pci` (PCIe, SM8650). Magic-packet WoWLAN can be armed without sudo: under `systemd-run --user --wait --pipe`, `nmcli c modify <connection> 802-11-wireless.wake-on-wlan magic` then `nmcli device reapply wlan0` makes `iw phy phy0 wowlan show` report `wake up on magic packet` (from SSH, `settings.modify.system` is only `auth`). **Tested 2026-09-28**, BUILD_ID 20260925.6191901, on the charger: after `PM: suspend entry (deep)` at 19:36:55, a unicast magic packet (UDP 9 and 7, to 192.168.1.237, with the Mac's ARP entry still present) and broadcast packets (192.168.1.255 and 255.255.255.255) got no wake in 30 s each. On a manual power-button wake 12 min later, the journal showed the chip had been reset during sleep: `mhi mhi0: Resuming from non M3 state (RESET)`, then `ath12k_pci: failed to wakeup from wow: -110`, WMI timeouts, `wiphy_resume returns -11`. Wi-Fi then disconnected and didn't come back, and the Frame needed a restart. So WoW did arm (no power-down fallback), but the WCN7850 loses power in `deep`. To turn it off, `wake-on-wlan default` alone doesn't clear the chip. `default` means NM's global `wifi.wake-on-wlan`, and the Frame sets none, so it falls back to `ignore`, which leaves the chip untouched. Set `0` and reapply (`WoWLAN is disabled.`), then set `default` again, or leave `0`. The Steam Deck with iwd fails differently: the NM setting never reached the driver there ([Switchboard](https://github.com/lfkdsk/Switchboard/blob/main/docs/steam-deck.md#wake-on-wlan)). `s2idle` failed the same way (tested 2026-09-28, set with `echo s2idle | sudo tee /sys/power/mem_sleep`, which lasts until reboot): `PM: suspend entry (s2idle)` at 20:29:57, no wake from unicast or broadcast packets, and on the power-button wake the same `Resuming from non M3 state (RESET)` and `failed to wakeup from wow`. Wi-Fi stayed `unavailable` until a restart. So the WCN7850 is reset during sleep either way. That points at ath12k WoW on this kernel/firmware rather than at the sleep depth. `systemctl suspend -i` under `systemd-run --user` asks for authentication, and plain `systemctl suspend` is refused while keep-awake's block inhibitor is held. | Waking the Frame remotely, [open-questions.md](open-questions.md) |
| **Battery at full on a charger** can read `Discharging` at about 0 W (for example 99 %, 0.0 W, USB-C PD 18 W). Treat under 0.5 W on a charger as "not charging", not "draining". **Verified 2026-09-27.** | Frame Control's battery card | | **Battery at full on a charger** can read `Discharging` at about 0 W (for example 99 %, 0.0 W, USB-C PD 18 W). Treat under 0.5 W on a charger as "not charging", not "draining". **Verified 2026-09-27.** | Frame Control's battery card |
| **The OS image is downloadable.** Valve's recovery images for the Frame are at `https://steamdeck-images.steamos.cloud/recovery/`. The root filesystem inside is btrfs, and it runs as an SSH test target on ARM64 Linux without the headset (`tests/frame-container/frame-image.sh`). **Verified 2026-09-27.** | [recovery-and-images.md](recovery-and-images.md) | | **The OS image is downloadable.** Valve's recovery images for the Frame are at `https://steamdeck-images.steamos.cloud/recovery/`. The root filesystem inside is btrfs, and it runs as an SSH test target on ARM64 Linux without the headset (`tests/frame-container/frame-image.sh`). **Verified 2026-09-27.** | [recovery-and-images.md](recovery-and-images.md) |
| **Boot / recovery menu.** Hold Power ~10 s until the LED goes off, then power on while holding the **AUX button on top of the Power button** (not the volume keys) until a text menu appears. Entries: `Current` (SteamOS-A/B + build), `Previous` (the other A/B slot), `Boot from USB`, `Repair Steam Installation`, `Erase User Data` (factory reset), `ADB mode`, `Battery Ship Mode`. It auto-boots `Current` after a ~15 s countdown. **Volume Up/Down (left side) move, AUX (right side) selects.** For a boot loop, Valve says pick `Previous` (keeps user data); then `Repair Steam Installation`; `Erase User Data` wipes `~` (SSH keys, Tailscale, Flatpaks, T3 setup). Last resort is a full re-image, two ways: (1) USB: write `steamframe-oobe-repair-<build>.img.bz2` to an 8 GB+ USB-C stick (Balena Etcher on the Mac), pick `Boot from USB`, then use "Wipe Device & Install SteamOS" / "Repair SteamOS" (keeps games and personal content) from the recovery desktop; (2) cable/EDL: `steamframe-oobe-repair-qdl-<build>.tar.gz`, run `flash.sh` (Linux) or `flash.cmd` (Windows), then with the Frame off for 10 s hold Power + Vol Up + Vol Down for 10 s and plug it in; it reflashes and reboots. Both images: `https://steamdeck-images.steamos.cloud/recovery/` (build 20260922.5153644, 0.3.0, 3.8 GiB each, no published checksums); local copies in `~/Downloads/steam-frame-recovery/`. File names, checksums and what's inside: [recovery-and-images.md](recovery-and-images.md). Source: Valve's [SteamOS Recovery FAQ](https://help.steampowered.com/en/faqs/view/1B71-EDF2-EB6D-2BB3) and [Installation and Repair FAQ](https://help.steampowered.com/en/faqs/view/65B4-2AA3-5F37-4227), plus a menu photo in [EloiStree/HelloSteamFrame#9](https://github.com/EloiStree/HelloSteamFrame/issues/9). **Inferred** (Valve docs, 2026-09-26); not yet tried on our Frame. | Recovering from a boot loop | | **Boot / recovery menu.** Hold Power ~10 s until the LED goes off, then power on while holding the **AUX button on top of the Power button** (not the volume keys) until a text menu appears. Entries: `Current` (SteamOS-A/B + build), `Previous` (the other A/B slot), `Boot from USB`, `Repair Steam Installation`, `Erase User Data` (factory reset), `ADB mode`, `Battery Ship Mode`. It auto-boots `Current` after a ~15 s countdown. **Volume Up/Down (left side) move, AUX (right side) selects.** For a boot loop, Valve says pick `Previous` (keeps user data); then `Repair Steam Installation`; `Erase User Data` wipes `~` (SSH keys, Tailscale, Flatpaks, T3 setup). Last resort is a full re-image, two ways: (1) USB: write `steamframe-oobe-repair-<build>.img.bz2` to an 8 GB+ USB-C stick (Balena Etcher on the Mac), pick `Boot from USB`, then use "Wipe Device & Install SteamOS" / "Repair SteamOS" (keeps games and personal content) from the recovery desktop; (2) cable/EDL: `steamframe-oobe-repair-qdl-<build>.tar.gz`, run `flash.sh` (Linux) or `flash.cmd` (Windows), then with the Frame off for 10 s hold Power + Vol Up + Vol Down for 10 s and plug it in; it reflashes and reboots. Both images: `https://steamdeck-images.steamos.cloud/recovery/` (build 20260922.5153644, 0.3.0, 3.8 GiB each, no published checksums); local copies in `~/Downloads/steam-frame-recovery/`. File names, checksums and what's inside: [recovery-and-images.md](recovery-and-images.md). Source: Valve's [SteamOS Recovery FAQ](https://help.steampowered.com/en/faqs/view/1B71-EDF2-EB6D-2BB3) and [Installation and Repair FAQ](https://help.steampowered.com/en/faqs/view/65B4-2AA3-5F37-4227), plus a menu photo in [EloiStree/HelloSteamFrame#9](https://github.com/EloiStree/HelloSteamFrame/issues/9). **Inferred** (Valve docs, 2026-09-26); not yet tried on our Frame. | Recovering from a boot loop |
Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

-30
View File
@@ -137,33 +137,3 @@ Still open: 4, 6, 7, 12–15, 16 (off-LAN and after a reboot), 17–21.
reports, not tested with the Frame. reports, not tested with the Frame.
- `connect.sh --harden`, `serve-bootstrap.sh` and - `connect.sh --harden`, `serve-bootstrap.sh` and
`bootstrap-on-frame.sh` haven't run against real hardware. `bootstrap-on-frame.sh` haven't run against real hardware.
## Remote wake (2026-09-28)
Goal: the Frame sleeps on the charger but Frame Control can wake it to reach
it over SSH. Findings so far are in the Wake-on-WLAN row of
[how-the-frame-works.md](how-the-frame-works.md). Independent review: GPT-6
Astra (xhigh), 2026-09-28.
- **Magic packet from `deep`: no (tested 2026-09-28).** Unicast and broadcast packets didn't wake it, the chip came back in MHI RESET, and Wi-Fi stayed broken until a restart. Details are in how-the-frame-works.md. Don't leave WoWLAN on with `deep`.
- **`s2idle`: no (tested 2026-09-28).** Same chip reset and broken Wi-Fi as `deep`. SteamOS doesn't pick `deep` itself (no `sleep.conf.d`, no `mem_sleep_default`, and no sleep hook touching ath12k), so this was a clean one-setting test. Wake over Wi-Fi is out until a SteamOS/ath12k update. Re-test after updates.
- **Recovering from the broken Wi-Fi: reboot cleanly, never `modprobe -r ath12k`.** On 2026-09-28, unloading the wedged driver oopsed the kernel (`Unable to handle kernel paging request`) and the Frame reset itself. The next boot had truncated Steam files (`steamclient.so`, then `steamui.so`), and the displays were broken for the whole boot (`msm_dsi … wait for video done timed out` from 28 s in, 240 times). `vrcompositor` segfaulted on its first present, `steamvr.service` took the gamescope session and Steam down every ~15 s, and the screen said "There was an issue launching Steam". Steam's updater also hung on the same GPU wait. The fixes: Steam re-verified its files, a leftover pending-install manifest identical to the `.manifest` was moved aside, and a clean reboot brought the displays back (0 DSI errors). The USB-C cable gives SSH at 10.86.200.233 when Wi-Fi is down. Checks and fixes are in [frame-doctor.md](frame-doctor.md).
- **Charger / smart-plug wake (hypothesis):** during confirmed sleep, test
physically attaching the charger, detaching it, and switching off its AC
supply, each separately. If one works, a Home Assistant smart plug on the
charger can wake it while it keeps deep sleep.
- **RTC dark wake:** a root `WakeSystem=yes` timer that checks for queued
work and suspends again. Needs a root unit.
- **Controller wake:** does a paired controller's button wake it? `hci0` is a
UART radio with no paired devices listed, so the controllers may use a
separate link.
- **AC-only Never:** `system_idle_suspend_ac_sec = 0`, with battery left at
15 min. This is Steam's own setting and needs no sudo, but the Frame stays
awake with its displays off rather than suspended. Measure wall power and
confirm the displays blank.
- **Off the LAN:** a sleeping Frame's Tailscale can't receive anything, so
something awake on the LAN has to send the packet (for example the
EdgeRouter's `etherwake`, already used for lxso2, or the Mac).
- **Staying on instead of sleeping:** what draws power and heat while idle, the
controls, and the step-by-step plan are in
[power-and-heat.md](power-and-heat.md).
-175
View File
@@ -1,175 +0,0 @@
# Power, heat and what you can control
What the Frame spends power on while it's on, where the heat comes from, and
which controls exist. Everything here was **read** on the Frame on
2026-09-29 (BUILD_ID 20260925.6191901). Nothing was changed. Numbers under
load haven't been measured yet.
## Sensors you can read without sudo
| What | Where |
|---|---|
| Power per rail (W ×10⁶) | hwmon `max34417_10`: `vph` (whole system), `s1c`, `s3c`, `s6c`. `max34417_12`: `apc0`/`apc1`/`apc2` (CPU clusters), `nsp1`. `max34417_1a`: `gfx` (GPU), `nsp2`, `bob`. Each `powerN_input` has a `powerN_label` |
| Board temperatures (m°C) | `/sys/bus/iio/devices/iio:device0/in_temp_*_input`: battery, left and right display, heatsink fins, fan exhaust, Wi-Fi, flash, 40-pin connector, nRF radio, PMIC and charger die |
| CPU, GPU and modem zones | `/sys/class/thermal/thermal_zone*/{type,temp}` (per-core top and bottom, `gpuss-*`, `nsp*`, `video`) |
| Charger input and charge current | `iio:device0/in_current_pm8550b_{iin,ichg}_fb_input` (µA) |
| Battery | `/sys/class/power_supply/max1720x_bat_7-36/uevent` (cycle count, health, current) |
| Fan | hwmon `slg4ax46073v`: `fan1_input` (RPM), `pwm1` (%) |
| Proximity (worn or not) | `/sys/bus/iio/devices/iio:device2/in_proximity_raw` |
| Why the fan ramped | `journalctl -u deckard-fan-control` (`C3 temperature of 95.36 greater than max 95! Setting fan to max speed.`) |
The right display thermistor reads −16 °C, so it's absent or broken. Ignore it.
## Idle baseline (2026-09-29 08:05)
Conditions:
- On the 12 V USB-C charger, battery 100%, 5 cycles.
- Headset off-head, SteamVR in standby, backlight 0.
- Steam, SteamVR, the tracking service and one Chromium (Mac view) running.
- `pauseCompositorOnStandby` = false, set by another session this morning for testing.
30 s average:
| Rail | W | Notes |
|---|---|---|
| `vph` (everything) | **4.71** | |
| CPU `apc0+1+2` | 0.61 | 91% idle overall |
| GPU `gfx` | 0.25 | GPU at 366 of 903 MHz |
| NSP `nsp1+2` | 0.08 | Neural and DSP processors |
| `s1c` + `s3c` + `s6c` + `bob` | 1.01 | SoC, memory and peripheral supplies (which rail is which isn't documented) |
| Unmetered remainder | ~2.8 | Cameras, display link, Wi-Fi, fan, sensors, conversion losses (inferred) |
Temperatures:
| Sensor | °C |
|---|---|
| CPU cores | 40–45 |
| Board (heatsink, Wi-Fi, flash) | 34–37 |
| Left display thermistor | 46 (the warmest) |
| Charger IC | 37 |
| Battery | 23 |
The fan ran at about 8,350 RPM at `pwm1` 41.
What was running while idle (`top`):
- The compositor was at about 14% of one core.
- Steam was at about 7%.
- The tracking service (XRService) was at about 4%, and still had 4 camera nodes open (`/dev/video0,3,9,13`).
- vrserver and gamescope were at about 3% each.
## Why it's warm while "doing nothing"
- **The compositor keeps rendering in standby** when
`power.pauseCompositorOnStandby` is false. The default is true. Check
`~/.config/openvr/config/steamvr.vrsettings`.
- **The tracking cameras stay open in standby.** XRService holds them so
tracking resumes instantly.
- **The fan never goes below 40% on the charger.** That's by design in
`/usr/share/deckard-fan-control/deckard-config.yaml`:
`fan_charging_min_speed: 40`, against `fan_min_speed: 30` on battery.
Charging, including topping up at 100%, heats the charger IC and battery
area.
- **The display link stays up at backlight 0.** The DSI connector reports
`dpms=On` while the backlight is 0, so the panels are dark but still driven.
- **Heavy load reaches the throttle limit.** On 2026-09-28 at 22:09–22:12,
cores C3, C5 and C7 hit 95 °C and the fan went to max. The cause wasn't
investigated. It came around a Steam restart after the reboot.
## Controls
No sudo needed:
| Control | How | Effect | Caveat |
|---|---|---|---|
| Steam idle sleep | `scripts/keep-awake.sh`, `system_idle_suspend_{ac,battery}_sec` | When it sleeps | Sleep has no remote wake |
| Compositor pause in standby | `vrcmd --set-settings-bool power.pauseCompositorOnStandby 1` | Stops rendering while off-head | Other sessions toggle it for testing, so coordinate |
| Screens-off delay | `vrcmd --set-settings-float power.turnOffScreensTimeout <s>` | Backlight off sooner or later | Default 5 s |
| Stop the whole VR stack | `systemctl --user stop steamvr.service` | Cameras, tracking and compositor off | Also stops the gamescope VR session and Steam (seen 2026-09-28), so panels and Mac view go too. The restart cost hasn't been measured |
| Background apps | Close Mac-view Chromium, stop Lepton containers (`podman`) | Less CPU and memory | Mac view needs relaunching |
| Brightness | Steam settings | Panel power while worn | — |
Needs root (reset at reboot unless made persistent):
| Control | Where | Effect |
|---|---|---|
| CPU governor and max frequency per cluster | `/sys/devices/system/cpu/cpufreq/policy{0,2,5,7}/scaling_{governor,max_freq}` (`powersave`, `conservative`, `schedutil`, …) | Caps CPU power and heat |
| Take cores offline | `/sys/devices/system/cpu/cpuN/online` | Fewer active cores |
| GPU max frequency | `/sys/class/devfreq/3d00000.gpu/max_freq` | Caps GPU power (it hurts VR smoothness when worn) |
| Fan curve | The service reads `/usr/share/…/deckard-config.yaml`, which is on the read-only rootfs. It could be overridden with a systemd drop-in pointing at a copy in `/etc` | Quieter fan while charging, but hotter parts |
| Charge current | `pm8550b-charger` `constant_charge_current` (1.0 A, max 1.2 A) | Writability unverified. There's **no** charge-limit or end-threshold file, so the battery sits at 100% on the charger |
None of these have been tried yet.
## Plan: leave it on, but cheaply
Goal: the Frame stays on the charger, reachable over SSH, drawing as little
power and making as little heat, fan wear and battery wear as possible while
nobody wears it. When someone puts it on, everything comes back quickly.
Sleeping isn't an option until remote wake works (see
[open-questions.md](open-questions.md#remote-wake-2026-09-28)).
Rules for every step:
- Change one thing at a time. Snapshot the setting first, measure 5 minutes
(`vph` plus temperatures and fan), then restore it unless it's being kept.
- Freeze the Steam and SteamVR health-check trackers first
([frame-doctor.md](frame-doctor.md) §4).
- Anything that could leave the Frame unreachable, and every root change, waits
until someone is home to recover it.
- `power.pauseCompositorOnStandby` belongs to another session's testing. Ask
before touching it.
### 1. Measure (read-only, safe remotely)
- [x] Idle baseline off-head on the charger (above).
- [ ] A sampler script (`scripts/frame-power-sample.sh`) that logs `vph`, the
CPU/GPU rails, key temperatures, fan RPM, the proximity sensor and the
charger current every few seconds to a CSV. Every later step uses it.
- [ ] The same baseline worn, idle in the home space.
- [ ] Under load: Mac view streaming, and one VR game.
- [ ] On battery (unplugged) to separate charging heat from everything else.
- [ ] Find what caused the 95 °C spike on 2026-09-28 22:09–22:12 (journal and
process history around the Steam restart).
### 2. Settings without sudo (one at a time, measured)
- [ ] Compositor pause in standby (after asking the owning session).
- [ ] Shorter screens-off delay.
- [ ] Stop `steamvr.service` while off-head. Measure the saving and how long it
takes to come back, since it also stops the gamescope session and Steam.
- [ ] Close Mac-view Chromium and stop idle Lepton containers.
- [ ] Steam's "never sleep on AC" (`system_idle_suspend_ac_sec = 0`), keeping
the battery timer. Confirm the displays go dark.
### 3. Root settings (at home, with approval)
- [ ] CPU: `powersave` or a lower max frequency while off-head.
- [ ] Fan: a copy of the fan config with a lower charging minimum, through a
systemd drop-in. Only if temperatures in steps 1–2 leave headroom.
- [ ] Battery: check whether charge current is writable. There's no charge
limit, so the fallback is a Home Assistant smart plug that lets the
battery cycle between roughly 80% and 100%.
- [ ] Decide which root changes to make persistent (drop-ins in `/etc`, which
survive SteamOS updates, unlike `/usr`).
### 4. A quiet mode
- [ ] Put the kept settings behind one switch: off-head for N minutes → quiet
mode; on-head (proximity sensor) or a Frame Control request → normal.
- [ ] Run it from Frame Control / keep-awake, not a hand-edited setting, so it
can always be undone.
- [ ] Add a doctor check that reports whether quiet mode is on and that it
restores cleanly.
### 5. Remote wake (at home)
- [ ] Charger wake: during confirmed sleep, plug in, unplug, and switch the
charger's AC off and on, one at a time.
- [ ] Controller-button wake.
- [ ] RTC dark wake (root timer that wakes, checks for queued work, sleeps).
- [ ] Re-test WoWLAN after each SteamOS or kernel update.
### 6. Doctor script
- [ ] Turn [frame-doctor.md](frame-doctor.md) into `scripts/frame-doctor.sh`:
read-only checks by default, fixes only with a flag, and **ask** fixes
never automatic. Add the sensor reads from this page.
+7
View File
@@ -148,3 +148,10 @@ For example, on 2026-09-27 the smoke test found that Steam's `create-shortcut`
refuses ids with a hyphen (`missing/invalid arguments`), which the fake had refuses ids with a hyphen (`missing/invalid arguments`), which the fake had
accepted. The fake now refuses them the same way, and Frame Control makes ids accepted. The fake now refuses them the same way, and Frame Control makes ids
Steam accepts. Steam accepts.
## Tracking protocols and fake BlueZ
`tests/test_tracking.py` exercises our gaze conversion, OSC sender, HRS parser
and BlueZ lifecycle with an in-memory fake object tree. It runs in the normal
unit suite without Bluetooth, GTK or OpenXR. Real Frame results and the absent
strap/tracker boundaries are recorded in [tracking](tracking.md).
+329
View File
@@ -0,0 +1,329 @@
# Eye tracking and heart rate
Frame Control's own tools run on the Frame, using OpenXR and BlueZ. No
VRCFaceTracking, LunaHR, Pulsoid or other tracking app is required. This is a
command-line first version; it does not add a desktop app tab.
## What was checked
**Verified 2026-09-28**, on a real aarch64 Frame running SteamOS 0.4.1,
BUILD_ID `20260925.6191901`:
| Check | Result |
|---|---|
| OpenXR gaze | SteamVR advertises `XR_EXT_eye_gaze_interaction`, `XR_MND_headless` and `XR_KHR_convert_timespec_time`. `supportsEyeGazeInteraction=1`. A headless session reached FOCUSED and produced 269 valid, tracked orientations in the first ten-second probe. |
| Our gaze → OSC bridge | A separate ten-second run produced 280 valid samples and 280 correctly padded 44-byte `/tracking/eye/CenterPitchYaw` messages at an explicitly configured loopback receiver. Only counters and packet-layout checks were retained. |
| Bluetooth stack | BlueZ active, adapter powered, central/peripheral roles available. LE discovery started and stopped successfully. No pairing or adapter power settings changed. |
| Our heart-rate panel | GTK4/GI runs on the stock image. A **synthetic 72 BPM** notification displayed in our X11 window, tagged `STEAM_GAME=2000000027`. Window capture checked; no real heart-rate measurement was taken. |
| SlimeVR, separate feasibility check | Native aarch64 server v21.1.0 ran with an isolated Temurin 21 JRE, created its driver sockets and accepted a local TCP connection on port 21110. Driver v6.0.0 loaded with all shared libraries resolved; `HmdDriverFactory("IServerTrackedDeviceProvider_004")` returned a non-null provider and error 0. |
![Our heart-rate panel on the Frame, showing synthetic 72 BPM](img/heart-rate-panel.png)
The image is a capture of our own Frame window using a fake notification,
not a real sensor reading.
**Untested:** a real BLE strap's notifications, physical fit/contact behaviour,
end-to-end heart-rate display/OSC/log with a strap, avatar response in VRChat,
gaze accuracy/calibration, coexistence with every immersive app, in-headset
panel placement, SlimeVR tracker/calibration data and the SlimeVR driver running
inside SteamVR. No trackers or strap are attached. The driver was loaded in a
separate process; it was **not registered or activated in SteamVR**. Steam and
SteamVR were not stopped or restarted.
**Verified blocker resolved:** importing `tkinter` fails because `libtk8.6.so`
is absent. The panel uses the installed GTK4/GI bindings instead. The Frame's
OpenXR headers advertise a newer API version than the runtime accepts; our
reader requests OpenXR 1.0 explicitly.
## Install our tools
From this checkout on your computer, while the Frame is awake:
```sh
python3 scripts/tracking-on-frame.py install
```
This copies our Python code and compiles our small C OpenXR reader into
`~/.local/share/frame-control/tracking/` on the Frame. It uses the Frame's
existing compiler, OpenXR headers/loader, Python, dbus-python, GI and GTK4.
Nothing is downloaded, and no sudo, driver registration, system setting,
service or autostart is added. `FRAME_ALIAS` can select another SSH alias.
The desktop app/server keeps its existing stdlib-only dependency set.
## Eye tracking → OSC
Start with a ten-second capability/data-availability check:
```sh
python3 scripts/tracking-on-frame.py gaze --seconds 10
```
This prints support, session-state numbers and sample counters. It opens no
OSC socket and prints no gaze coordinates. Exit 0 means at least one valid
sample, 3 means no valid sample was observed, and 1 means an API/runtime error.
If there are no valid samples, wake/wear the headset and check its tracking
setup; a successful capability check alone does not prove usable gaze.
To send to VRChat running **on the Frame**, explicitly enable OSC in VRChat
and choose its local UDP endpoint:
```sh
python3 scripts/tracking-on-frame.py gaze --seconds 3600 --osc 127.0.0.1 9000
```
For a receiver on another computer, replace `127.0.0.1` with that computer's
IP address and choose its listening port. Addresses are IP literals (IPv4 or
IPv6); there is no discovery or default destination. Loopback here always
means **the Frame**, not the computer running the SSH command. OSC uses
unencrypted UDP: configure only a receiver you intend to receive this data.
**Documented:** [VRChat's eye OSC interface](https://docs.vrchat.com/docs/osc-eye-tracking)
accepts `/tracking/eye/CenterPitchYaw` with two floats in degrees, positive down
and right. We locate OpenXR's combined gaze pose relative to VIEW (the head),
rotate its -Z forward vector and convert that direction to these angles.
Only active, orientation-valid **and tracked** samples are sent, at up to
30 Hz. No eyelid/blink, individual-eye or face values are invented. We do not
send neutral gaze on tracking loss; VRChat's documented timeout restores its
automatic eye behaviour after input stops.
**Privacy:** gaze is personal data. It stays in process memory and a private
pipe between our reader and bridge. There is no gaze log option, telemetry,
OSC receiver or raw gaze on stdout/stderr. Only an explicit `--osc IP PORT`
opens an output socket. Runtime diagnostics and validity counters are not
measurements. Stop with Ctrl-C or let `--seconds` expire (maximum 24 hours).
A lost headless session ends the run; it does not silently reconnect.
## BLE heart rate → our panel, OSC and optional log
First discover/pair your strap in SteamOS's Bluetooth settings. Select that
strap's Bluetooth address explicitly; our tool does not scan for or connect
to arbitrary nearby devices.
```sh
python3 scripts/tracking-on-frame.py heart \
--device AA:BB:CC:DD:EE:FF --panel --seconds 3600
```
This uses BlueZ's standard Heart Rate Service (`180d`) and Heart Rate
Measurement (`2a37`) notifications. It finds the characteristic only beneath
the selected device's HRS service. The reader handles 8- and 16-bit BPM,
contact flags and optional energy/RR fields; energy and RR intervals are
validated for length but discarded. Zero BPM, reported loss of skin contact,
malformed packets and readings older than five seconds are not shown as a
current measurement. A disconnect stops the run; reconnect and start again.
This is a social/fitness readout, not a medical monitor.
The panel is our GTK4 window on gamescope's X display. Use SteamVR's panel
controls to float/dock it (see [panels](panels.md)). **Stop**, closing the panel,
Ctrl-C, SSH hangup or the duration limit ends our subscription. A connection
that was already open when we started is preserved; a connection we opened
is disconnected on exit. No Bluetooth power or pairing state is changed.
Add either output explicitly:
```sh
python3 scripts/tracking-on-frame.py heart \
--device AA:BB:CC:DD:EE:FF --panel --seconds 3600 \
--osc 127.0.0.1 9000 --address /avatar/parameters/HeartRate \
--log /home/steamos/heart-session.csv
```
The OSC value is integer BPM. `HeartRate` is a chosen avatar parameter, **not a
built-in VRChat heart-rate feature**; your avatar/receiver must define the
matching parameter. `--address` can select another literal OSC path. The local
panel works without OSC, a log or an avatar integration.
The optional CSV contains only `unix_seconds,bpm`. It is created privately
(mode 0600), refuses existing files/symlinks, and lives **on the Frame** at the
path you specify. Nothing is logged by default, and heart-rate values are not
printed to the terminal. Delete your session file when you no longer need it.
### Checking heart rate against a reference
`scripts/heart-check.py` runs on your computer. `listen` shows our OSC
readings live as they arrive, so you can watch them next to another device:
```sh
python3 scripts/heart-check.py listen --port 9000 --out ours.csv
```
Point the Frame at it with `--osc <your computer's IP> 9000`. `compare` lines
up two recordings by time and reports the mean difference, bias, the share
within ±5 BPM and the delay between them. It passes when the mean difference
is at most 5 BPM, at least 80% of reference readings are matched and nothing
was shown while the sensor reported lost skin contact:
```sh
python3 scripts/heart-check.py compare ours.csv reference.csv
python3 scripts/heart-check.py compare ours.csv ~/Downloads/export.zip
```
The reference can be a CSV (`time,bpm[,flags]`, time in unix seconds or ISO
8601) or an Apple Health export (`export.zip` or `export.xml`). Only heart-rate
records within the recording's time range are read. Everything stays on your
computer.
`scripts/heart-test-strap.swift` turns a Mac into a synthetic strap. It
advertises the standard Heart Rate Service and sends a fixed, known sequence
(8-bit and 16-bit values and a skin-contact loss), printing each sent value, so
`compare` can check that the Frame shows exactly what was sent. It needs
Bluetooth permission for the process that runs it. **Untested on 2026-09-29:**
it compiled, but on this Mac, launched from an agent session, macOS never
delivered a Bluetooth state and no permission prompt appeared, so it never
advertised.
## Pulse from the eye cameras (experimental)
The Frame has no heart-rate sensor. **Verified 2026-09-29** (SteamOS 0.4.1,
build `20260925.6191901`): its sensors are an ambient light/proximity sensor
(`vcnl4000`), a hall sensor (`als31300`), two passthrough cameras
(`arcimx616`), two tracking cameras (`og01a1b`) and two IR eye cameras
(`og0ve10`). There is no optical heart-rate (PPG) sensor.
The experiment asks whether the eye cameras can see a pulse anyway. With each
heartbeat, the blood volume in the skin around the eye changes slightly and
its IR reflectance changes with it. This is camera-based photoplethysmography;
near-IR works, though the signal is weaker than in green light.
```sh
python3 scripts/tracking-on-frame.py pulse --seconds 60 --show
```
How it works:
- **Capture (verified).** SteamVR ships `eyetracking --calib N`, which saves
both eye cameras for N seconds as 400×400 8-bit IR PNGs with a monotonic
timestamp per frame, at about 90 fps per eye. SteamVR's live eye tracker,
part of `steamvr.service`, gets its frames from the DSP and stops its
cameras when the headset is off. Unworn captures ran alongside it: its PID
and log were unchanged and our OpenXR gaze session still started
afterwards. **Untested:** whether the capture and the live tracker coexist
while the headset is worn and tracking.
- **Privacy.** Each image is reduced to a 16×16 grid of patch averages as
soon as it is complete, then deleted. Three worker processes do this beside
the capture. If more than 900 images (about five seconds) ever wait, we stop
reading them and delete them undecoded until the capture ends, and report
an error. The capture directory is removed on exit, even after errors. No image is kept or leaves the Frame. The estimate
is printed only with `--show`, and sent or saved only with `--osc` or
`--log`, as for the strap.
- **Estimate.** Patch traces are averaged down to 15 Hz and turned into
relative change. A 2-second moving median removes drift and blinks.
Patches with frequent spikes (the eyeball and eyelid) are dropped, as are
dark or saturated ones. The 20% of patches with the clearest rhythm between
42 and 180 BPM are combined in the frequency domain. Output is an overall
estimate plus one estimate per second over 15-second windows. A result
counts as **clear** only when the top patches agree and the combined signal
stands out from the noise. Otherwise the command exits 3 and sends no OSC.
`--log` still records the per-second estimates, so a comparison shows how
far off an unclear result was.
The thresholds are provisional until checked on real wearers.
**Verified on the Frame, unworn, 2026-09-29:** captures of 3,600-5,400 eye
frames never had more than 8 images on disk, finished a few seconds after the
capture ended and left no capture directory. **The estimator alone gave a
false "clear" pulse.** With nobody wearing the headset, five runs reported a
steady, self-consistent rhythm (90, 90, 93, 94 and 96 BPM; patch agreement
100%, signal/noise 0.63-0.70), and earlier runs reported 127-129 BPM at lower
signal/noise. It is a periodic camera or illumination artifact, and its
frequency drifts between runs. A wearer-less scene cannot contain a pulse, so
the signal/noise gate cannot tell this artifact from one. Because of that,
`pulse` reads the Frame's proximity sensor (`vcnl4000`) before and after the
capture. It reads about 3 unworn (**verified**). If either reading is below 20
the result is never called clear, nothing is sent over OSC, and the command
exits 3. **Inferred, unmeasured:** that a worn reading is well above 20; the
cut-off is provisional until someone wears the headset. If the sensor can't be
read, the guard is skipped. Confirming a real pulse also needs a reference
(below).
**Do not interrupt the capture. Verified on the Frame, 2026-09-29:** sending
SIGTERM to `eyetracking --calib` left the DSP service's eye camera (OV6211)
stuck "streaming": its log had no "Stopping streaming" line, and every later
request failed with "Failed to start streaming". Head tracking kept working.
Clearing it needs the DSP service restarted or the Frame rebooted, so `pulse`
never signals the tool. After Ctrl-C or a failure it keeps deleting images
until the tool ends by itself (at most the `--seconds` plus a few seconds),
and only kills a tool that overruns by 30 s, with a warning that the eye
cameras may need a reboot. So an interrupted run can take a while to return.
**Verified on synthetic data** (unit tests): a 0.3% brightness pulse in a
third of the patches, with noise, drift, blinks and eye movement, is
recovered within 1.5 BPM at 58, 72 and 115 BPM; noise and blinks alone are
not reported as a pulse. **Not yet verified:** whether a real wearer's eye
images contain a usable pulse, and how accurate it is. That needs someone
wearing the headset and a reference, as below.
### Comparing with an Apple Watch
1. On the watch, start a workout (for example **Other**) so it measures heart
rate every few seconds rather than occasionally.
2. Put the Frame on, sit still and look ahead. Run:
```sh
python3 scripts/tracking-on-frame.py pulse --seconds 120 --show \
--log /home/steamos/pulse.csv
```
The per-second estimates print at the end. Compare them with what the
watch showed.
3. End the workout. On the iPhone, open Health → your picture → **Export All
Health Data**, and AirDrop `export.zip` to the Mac.
4. On the Mac:
```sh
scp frame:pulse.csv . && ssh frame rm pulse.csv
python3 scripts/heart-check.py compare pulse.csv ~/Downloads/export.zip
```
This first version analyses after the capture ends, because the method must
prove itself before a live panel is worth building. The Apple Watch is a
reference, not ground truth: in workouts it is typically within a few BPM of
a chest strap when you are still.
## SlimeVR: feasibility only
SlimeVR is an independent application stack. Neither of our features installs,
launches or depends on it. Users who want it can follow
[SlimeVR's setup documentation](https://docs.slimevr.dev/server/index.html).
The consented upstream releases tested were
[server v21.1.0](https://github.com/SlimeVR/SlimeVR-Server/releases/tag/v21.1.0)
and [driver v6.0.0](https://github.com/SlimeVR/SlimeVR-OpenVR-Driver/releases/tag/v6.0.0),
under SlimeVR's MIT/Apache-2.0 licensing.
**Verified layout, read-only:** the Frame's registered runtime is `/opt/steamvr`;
its native driver is `drivers/cv/bin/linuxarm64/driver_cv.so`, with a
`drivers/cv/driver.vrdrivermanifest`. Frame controller manifests/resources are
under `drivers/frame_controller/`. Configuration is under
`~/.config/openvr/config/`, not the Steam client's config directory. The
SlimeVR release also uses `slimevr/bin/linuxarm64/driver_slimevr.so` plus its
manifest. Nothing in those installed SteamVR directories was changed.
**Inferred:** the matching ABI/layout and standalone factory success make
SteamVR integration plausible. They do not prove successful driver `Init`,
server/driver IPC, tracking, or calibration. That needs a separate integration
check with hardware and an agreed SteamVR restart. No Java executable was on
PATH for this check, so an isolated JRE was used. SlimeVR's server opens LAN
listeners; our temporary server was stopped and the temporary downloads,
configuration and logs were removed. It is not left installed or running.
## Tests and remaining checks
```sh
python3 -m unittest discover -s tests
```
`tests/test_tracking.py` covers HRS packet parsing, contact/staleness, OSC
padding/types and a real loopback socket, quaternion signs, opt-in networking,
private/exclusive logging and a fake BlueZ object tree. The fake checks service
ownership, notification routing, delayed GATT discovery and connection cleanup.
It does not pretend to be a physical strap or a real OpenXR runtime.
Before calling hardware support complete, attach a strap and check BPM against
its own display/reference, loss of contact, disconnect/reconnect, Stop, OSC and
CSV together. Check avatar eyes while looking up/down/left/right in a supported
VRChat session. No third-party tracking app is needed for either test.
Independent review attempt: `devin -p --model swe-2-max` with the frozen diff,
contribution standards and read-only instructions returned no output for ten
minutes. It was terminated with exit 143. No completed review or actual model
identity was returned; hardware checks and independent review remain follow-up
work before making the draft ready.
+135
View File
@@ -0,0 +1,135 @@
/* Frame Control's OpenXR gaze source. No values on stdout/stderr or disk.
* The Python bridge supplies a private pipe with --fd; standalone probes only
* report counters. Uses a headless session, never submits frames or takes focus. */
#define XR_USE_TIMESPEC
#include <time.h>
#include <openxr/openxr.h>
#include <openxr/openxr_platform.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
static volatile sig_atomic_t stopped;
static void stop(int sig) { (void)sig; stopped = 1; }
#define CHECK(call) do { result = (call); if (XR_FAILED(result)) { \
fprintf(stderr, "%s failed (%d)\n", #call, result); goto cleanup; } } while (0)
int main(int argc, char **argv) {
int seconds = 10, fd = -1, running = 0, rc = 1;
unsigned active = 0, valid = 0, samples = 0;
for (int i = 1; i < argc; i++) {
if (!strcmp(argv[i], "--seconds") && i+1 < argc) seconds = atoi(argv[++i]);
else if (!strcmp(argv[i], "--fd") && i+1 < argc) fd = atoi(argv[++i]);
else { fprintf(stderr, "usage: gaze [--seconds 1..86400] [--fd private-pipe]\n"); return 2; }
}
if (seconds < 1 || seconds > 86400 || (fd != -1 && fd < 3)) return 2;
FILE *out = fd == -1 ? NULL : fdopen(fd, "w");
if (fd != -1 && !out) return 2;
signal(SIGINT, stop); signal(SIGTERM, stop); signal(SIGHUP, stop); signal(SIGPIPE, SIG_IGN);
XrResult result;
XrInstance instance = XR_NULL_HANDLE;
XrSession session = XR_NULL_HANDLE;
XrActionSet set = XR_NULL_HANDLE;
XrSpace gaze = XR_NULL_HANDLE, view = XR_NULL_HANDLE;
const char *extensions[] = {"XR_EXT_eye_gaze_interaction", "XR_MND_headless", "XR_KHR_convert_timespec_time"};
XrInstanceCreateInfo create = {.type = XR_TYPE_INSTANCE_CREATE_INFO};
strcpy(create.applicationInfo.applicationName, "Frame Control gaze");
create.applicationInfo.apiVersion = XR_MAKE_VERSION(1, 0, 0);
create.enabledExtensionCount = 3; create.enabledExtensionNames = extensions;
CHECK(xrCreateInstance(&create, &instance));
XrSystemGetInfo get = {.type = XR_TYPE_SYSTEM_GET_INFO, .formFactor = XR_FORM_FACTOR_HEAD_MOUNTED_DISPLAY};
XrSystemId system;
CHECK(xrGetSystem(instance, &get, &system));
XrSystemEyeGazeInteractionPropertiesEXT eye = {.type = XR_TYPE_SYSTEM_EYE_GAZE_INTERACTION_PROPERTIES_EXT};
XrSystemProperties props = {.type = XR_TYPE_SYSTEM_PROPERTIES, .next = &eye};
CHECK(xrGetSystemProperties(instance, system, &props));
printf("supportsEyeGazeInteraction=%u\n", eye.supportsEyeGazeInteraction);
if (!eye.supportsEyeGazeInteraction) goto cleanup;
XrSessionCreateInfo sc = {.type = XR_TYPE_SESSION_CREATE_INFO, .systemId = system};
CHECK(xrCreateSession(instance, &sc, &session));
XrActionSetCreateInfo asc = {.type = XR_TYPE_ACTION_SET_CREATE_INFO};
strcpy(asc.actionSetName, "gaze"); strcpy(asc.localizedActionSetName, "Gaze");
CHECK(xrCreateActionSet(instance, &asc, &set));
XrActionCreateInfo ac = {.type = XR_TYPE_ACTION_CREATE_INFO, .actionType = XR_ACTION_TYPE_POSE_INPUT};
strcpy(ac.actionName, "gaze_pose"); strcpy(ac.localizedActionName, "Gaze pose");
XrAction action;
CHECK(xrCreateAction(set, &ac, &action));
XrPath profile, input;
CHECK(xrStringToPath(instance, "/interaction_profiles/ext/eye_gaze_interaction", &profile));
CHECK(xrStringToPath(instance, "/user/eyes_ext/input/gaze_ext/pose", &input));
XrActionSuggestedBinding binding = {action, input};
XrInteractionProfileSuggestedBinding suggested = {.type = XR_TYPE_INTERACTION_PROFILE_SUGGESTED_BINDING,
.interactionProfile = profile, .countSuggestedBindings = 1, .suggestedBindings = &binding};
CHECK(xrSuggestInteractionProfileBindings(instance, &suggested));
XrSessionActionSetsAttachInfo attach = {.type = XR_TYPE_SESSION_ACTION_SETS_ATTACH_INFO, .countActionSets = 1, .actionSets = &set};
CHECK(xrAttachSessionActionSets(session, &attach));
XrActionSpaceCreateInfo space = {.type = XR_TYPE_ACTION_SPACE_CREATE_INFO, .action = action, .poseInActionSpace.orientation.w = 1};
CHECK(xrCreateActionSpace(session, &space, &gaze));
XrReferenceSpaceCreateInfo ref = {.type = XR_TYPE_REFERENCE_SPACE_CREATE_INFO, .referenceSpaceType = XR_REFERENCE_SPACE_TYPE_VIEW,
.poseInReferenceSpace.orientation.w = 1};
CHECK(xrCreateReferenceSpace(session, &ref, &view));
PFN_xrConvertTimespecTimeToTimeKHR convert;
CHECK(xrGetInstanceProcAddr(instance, "xrConvertTimespecTimeToTimeKHR", (PFN_xrVoidFunction *)&convert));
struct timespec start, now;
clock_gettime(CLOCK_MONOTONIC, &start);
while (!stopped) {
clock_gettime(CLOCK_MONOTONIC, &now);
if (now.tv_sec - start.tv_sec >= seconds) break;
XrEventDataBuffer event = {.type = XR_TYPE_EVENT_DATA_BUFFER};
while ((result = xrPollEvent(instance, &event)) == XR_SUCCESS) {
if (event.type == XR_TYPE_EVENT_DATA_SESSION_STATE_CHANGED) {
XrSessionState state = ((XrEventDataSessionStateChanged *)&event)->state;
printf("sessionState=%d\n", state); fflush(stdout);
if (state == XR_SESSION_STATE_READY && !running) {
XrSessionBeginInfo begin = {.type = XR_TYPE_SESSION_BEGIN_INFO, .primaryViewConfigurationType = XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO};
CHECK(xrBeginSession(session, &begin)); running = 1;
} else if (state == XR_SESSION_STATE_STOPPING) {
CHECK(xrEndSession(session)); running = 0; stopped = 1;
} else if (state == XR_SESSION_STATE_EXITING || state == XR_SESSION_STATE_LOSS_PENDING) stopped = 1;
} else if (event.type == XR_TYPE_EVENT_DATA_INSTANCE_LOSS_PENDING) stopped = 1;
event.type = XR_TYPE_EVENT_DATA_BUFFER;
}
if (XR_FAILED(result)) goto cleanup;
if (running && !stopped) {
XrActiveActionSet activeSet = {set, XR_NULL_PATH};
XrActionsSyncInfo sync = {.type = XR_TYPE_ACTIONS_SYNC_INFO, .countActiveActionSets = 1, .activeActionSets = &activeSet};
CHECK(xrSyncActions(session, &sync));
if (result != XR_SUCCESS) {
struct timespec delay = {.tv_nsec = 33333333};
nanosleep(&delay, NULL);
continue; /* No stale gaze when the runtime denies focus. */
}
XrActionStateGetInfo ag = {.type = XR_TYPE_ACTION_STATE_GET_INFO, .action = action};
XrActionStatePose pose = {.type = XR_TYPE_ACTION_STATE_POSE};
CHECK(xrGetActionStatePose(session, &ag, &pose));
samples++;
if (pose.isActive) {
active++;
XrTime time; CHECK(convert(instance, &now, &time));
XrSpaceLocation location = {.type = XR_TYPE_SPACE_LOCATION};
CHECK(xrLocateSpace(gaze, view, time, &location));
XrSpaceLocationFlags needed = XR_SPACE_LOCATION_ORIENTATION_VALID_BIT | XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT;
if ((location.locationFlags & needed) == needed) {
valid++;
if (out) {
XrQuaternionf q = location.pose.orientation;
if (fprintf(out, "%g %g %g %g\n", q.x, q.y, q.z, q.w) < 0 || fflush(out)) goto cleanup;
}
}
}
}
struct timespec delay = {.tv_nsec = 33333333}; nanosleep(&delay, NULL);
}
printf("samples=%u active=%u valid=%u\n", samples, active, valid);
rc = valid ? 0 : 3; /* Distinguish a working session from observed gaze. */
cleanup:
if (view) xrDestroySpace(view);
if (gaze) xrDestroySpace(gaze);
if (session) xrDestroySession(session);
if (set) xrDestroyActionSet(set);
if (instance) xrDestroyInstance(instance);
if (out) fclose(out);
return rc;
}
+481
View File
@@ -0,0 +1,481 @@
"""Experimental pulse estimate from the Frame's IR eye-tracking cameras.
Skin brightens and darkens very slightly with each heartbeat as blood volume
changes (photoplethysmography). The eye cameras film the skin around each eye
under steady IR light at about 90 frames per second, so that rhythm may be
visible in the average brightness of small patches of skin.
Capture uses SteamVR's own `eyetracking --calib` mode, which writes PNG pairs
to /tmp. Each image is reduced to a grid of patch averages the moment it is
complete, then deleted; the capture directory is removed on exit. No image
leaves the Frame or outlives the run. This is an experiment, not a medical
measurement.
"""
from array import array
import bisect
import cmath
import json
import math
import os
from pathlib import Path
import re
import shutil
import subprocess
import time
ET_DIR = Path("/opt/steamvr/tools/eyetracking")
ET_BIN = ET_DIR / "bin/linuxarm64/eyetracking"
ET_WEIGHTS = ET_DIR / "resources/et_dsp_20250610_03136.weights"
GRID = 16 # 16 × 16 patches of 25 × 25 pixels on the 400 × 400 image
RATE = 15.0 # analysis sample rate, Hz; the band of interest ends at 3 Hz
LOW, HIGH = 42.0, 180.0 # BPM search band
# The Frame's proximity sensor (vcnl4000) reads about 3 with nobody wearing it.
# A worn reading has not been measured yet, so the cut-off is provisional.
IIO = Path("/sys/bus/iio/devices")
WORN_MIN = 20.0
WINDOW = 15.0 # seconds per windowed estimate
# ---------------------------------------------------------------- capture ---
def grid_means(pixels, width, height, stride, channels, grid=GRID):
"""Average of channel 0 in each of grid × grid equal patches."""
bw, bh = width // grid, height // grid
sums = [0] * (grid * grid)
for y in range(bh * grid):
start = y * stride
row = pixels[start:start + width * channels:channels]
base = (y // bh) * grid
for bx in range(grid):
sums[base + bx] += sum(row[bx * bw:(bx + 1) * bw])
area = bw * bh
return [s / area for s in sums]
def load_grid(path):
import gi
gi.require_version("GdkPixbuf", "2.0")
from gi.repository import GdkPixbuf
image = GdkPixbuf.Pixbuf.new_from_file(str(path))
return grid_means(image.read_pixel_bytes().get_data(), image.get_width(), image.get_height(),
image.get_rowstride(), image.get_n_channels())
def reduce_file(loader, path):
"""Reduce one image to its patch grid and delete it, whatever happens."""
try:
return loader(path)
finally:
os.unlink(path)
class Capture:
"""Run SteamVR's eye-camera capture and reduce frames as they arrive."""
NAME = re.compile(r"^(left|right)_(\d+)\.png$")
# Only SteamVR's own capture directories are read and removed.
PREFIX = "/tmp/etcalib_"
# Printed by the capture tool; its output is only flushed when it exits.
WRITING = re.compile(r"Writing capture to: (\S+)")
# About five seconds of frames (~65 MB in RAM-backed /tmp). If reduction
# falls further behind than this, the capture stops rather than letting
# eye images pile up.
MAX_BACKLOG = 900
def __init__(self, seconds, runner=subprocess.Popen, loader=load_grid, workers=3):
self.seconds, self.runner, self.loader, self.workers = seconds, runner, loader, workers
self.grids = {"left": {}, "right": {}}
self.directory = None
self.pool = None
self.submitted = set()
self.futures = {}
self.new = set() # every capture directory that appeared during our run
self.before = None # capture directories that existed before the run
self.log = None
def candidates(self):
parent, stem = os.path.split(self.PREFIX)
return {Path(entry.path) for entry in os.scandir(parent)
if entry.name.startswith(stem) and entry.is_dir(follow_symlinks=False)}
def run(self):
# The capture tool's output goes to a private temporary file, read for
# failure messages. It is block-buffered, so the capture directory is
# found by watching for a new one rather than waiting for its name.
import tempfile
self.before = before = self.candidates()
process = None
with tempfile.TemporaryFile("w+") as log:
self.log = log
try:
if self.workers:
# SteamVR pins its eye tracker to cores 0-1; decoding runs beside it.
import concurrent.futures
import multiprocessing
self.pool = concurrent.futures.ProcessPoolExecutor(
self.workers, mp_context=multiprocessing.get_context("fork"))
process = self.runner([str(ET_BIN), "-b", "CDSP", "-w", str(ET_WEIGHTS), "--calib", str(self.seconds)],
cwd=str(ET_BIN.parent), stdout=log, stderr=subprocess.STDOUT)
deadline = time.monotonic() + self.seconds + 30
while True:
# Checked on every poll: a second capture directory means
# we can't tell which images are ours, so stop.
self.new |= self.candidates() - before
if len(self.new) > 1:
raise RuntimeError("another eye-camera capture is running")
if not self.directory and self.new:
self.directory = next(iter(self.new))
finished = process.poll() is not None
self.reduce(final=finished)
if finished:
break
if time.monotonic() > deadline:
raise RuntimeError("eye-camera capture did not finish")
time.sleep(0.02)
log.seek(0)
text = log.read()
if process.returncode or not self.directory:
reason = "cameras unavailable" if "Failed to" in text else f"exit {process.returncode}"
raise RuntimeError(f"eye-camera capture failed ({reason})")
named = self.written(log)
if named and named != self.directory:
raise RuntimeError("the capture wrote somewhere else; not using those images")
return self.frames()
finally:
# Each step runs even if an earlier one failed: eye images must
# be removed whatever else went wrong.
# Ctrl-C and SIGTERM (raised as KeyboardInterrupt) are held
# until the images are gone, then re-raised.
interrupted, failed = None, None
for step in (lambda: self.finish(process),
lambda: self.pool and self.pool.shutdown(wait=True, cancel_futures=True),
self.adopt_reported):
try:
step()
except BaseException as error:
if isinstance(error, Exception):
failed = failed or error
else:
interrupted = interrupted or error
self.log = None
self.remove()
if interrupted:
raise interrupted
if failed:
raise RuntimeError(f"the eye-camera capture did not stop cleanly: {failed}")
def finish(self, process):
"""Let the capture tool end by itself, deleting its images meanwhile.
Never signal it: stopping the tool early leaves the headset's eye
camera stuck streaming until the DSP service restarts (verified on the
Frame with SIGTERM). Its run is bounded by `seconds`, so waiting is
short. Only if it overruns by a wide margin is it killed."""
if not process:
return
interrupted = None
give_up = time.monotonic() + self.seconds + 30
while True:
try:
if process.poll() is not None:
break
self.discard()
if time.monotonic() > give_up:
process.kill()
process.wait()
raise RuntimeError("the eye-camera capture had to be killed; "
"the eye cameras may need a reboot to stream again")
time.sleep(0.05)
except KeyboardInterrupt as error: # keep cleaning up until it ends
interrupted = interrupted or error
if interrupted:
raise interrupted
def discard(self):
"""Delete the eye images written so far, without reading them."""
if self.before is not None:
self.new |= self.candidates() - self.before
for directory in self.new | ({self.directory} if self.directory else set()):
if str(directory).startswith(self.PREFIX) and directory.is_dir() and not directory.is_symlink():
for entry in os.scandir(directory):
if self.NAME.match(entry.name):
try:
os.unlink(entry.path)
except OSError:
pass
def adopt_reported(self):
"""The directory the tool reported is ours by its own account."""
named = self.written(self.log)
if named and str(named).startswith(self.PREFIX):
self.new.add(named)
def written(self, log):
"""The directory the capture tool reported, once its output is flushed."""
log.seek(0)
match = self.WRITING.search(log.read())
return Path(match.group(1)) if match else None
def reduce(self, final=False):
"""Reduce and delete every complete image; an image is complete once a
later one of the same eye exists, or the capture has ended."""
if not self.directory or not self.directory.is_dir():
return
pending = {"left": [], "right": []}
for entry in os.scandir(self.directory):
match = self.NAME.match(entry.name)
if match:
pending[match.group(1)].append((int(match.group(2)), entry.path))
if sum(len(files) for files in pending.values()) > self.MAX_BACKLOG:
raise RuntimeError("eye-image processing fell behind; capture abandoned")
for eye, files in pending.items():
files.sort()
ready = files if final else files[:-1]
for index, path in ready:
if path in self.submitted:
continue
self.submitted.add(path)
if self.pool:
self.futures[(eye, index)] = self.pool.submit(reduce_file, self.loader, path)
else:
self.grids[eye][index] = array("f", reduce_file(self.loader, path))
for key, future in list(self.futures.items()):
if final or future.done():
self.grids[key[0]][key[1]] = array("f", future.result())
del self.futures[key]
def frames(self):
"""[(monotonic seconds, eye, grid)] joined with the capture metadata."""
meta = json.loads((self.directory / "meta.json").read_text())
frames = []
for pair in meta["frames"]:
for eye in ("left", "right"):
info = pair.get(eye) or {}
match = self.NAME.match(Path(info.get("fname", "")).name)
grid = self.grids[eye].get(int(match.group(2))) if match else None
if info.get("valid") and grid is not None:
frames.append((float(info["tsMono"]), eye, grid))
return frames
def remove(self):
"""Remove every capture directory that appeared during the run. Eye
images must not outlive it, so a failure to delete is an error."""
left = []
if self.before is not None:
try:
self.new |= self.candidates() - self.before # even if interrupted before the first poll
except OSError:
pass
for directory in self.new | ({self.directory} if self.directory else set()):
if str(directory).startswith(self.PREFIX) and directory.is_dir() and not directory.is_symlink():
try:
shutil.rmtree(directory)
except OSError:
left.append(str(directory))
if left:
raise RuntimeError("could not delete eye images in " + ", ".join(sorted(left)))
# --------------------------------------------------------------- analysis ---
def fft(values):
"""In-place iterative radix-2 FFT of a list whose length is a power of 2."""
n = len(values)
a = list(values)
j = 0
for i in range(1, n):
bit = n >> 1
while j & bit:
j ^= bit
bit >>= 1
j |= bit
if i < j:
a[i], a[j] = a[j], a[i]
size = 2
while size <= n:
step = cmath.exp(-2j * math.pi / size)
half = size // 2
for start in range(0, n, size):
w = 1
for k in range(start, start + half):
t = w * a[k + half]
a[k + half] = a[k] - t
a[k] += t
w *= step
size *= 2
return a
def resample(times, values, rate=RATE):
"""Average samples into 1/rate bins from the first sample; empty bins are
filled from the previous bin."""
if not times:
return []
start = times[0]
count = int((times[-1] - start) * rate) + 1
sums, counts = [0.0] * count, [0] * count
for t, v in zip(times, values):
i = min(int((t - start) * rate), count - 1)
sums[i] += v
counts[i] += 1
out, last = [], None
for s, c in zip(sums, counts):
last = s / c if c else last
out.append(last)
first = next(v for v in out if v is not None)
return [first if v is None else v for v in out]
def clean(signal, rate=RATE):
"""Relative change with slow drift removed; None if the patch is mostly
blinks or eye movement. Spikes (blinks, saccades) become gaps at the
local level rather than clipped steps, which would add false rhythm."""
mean = sum(signal) / len(signal)
x = [v / mean - 1 for v in signal]
half = int(rate) # 2-second centred moving median removes drift and blinks
trend = []
for i in range(len(x)):
chunk = sorted(x[max(0, i - half):i + half + 1])
trend.append(chunk[len(chunk) // 2])
residual = [v - m for v, m in zip(x, trend)]
mad = sorted(abs(v) for v in residual)[len(residual) // 2] or 1e-9
limit = 4 * 1.4826 * mad
spikes = sum(1 for v in residual if abs(v) > limit)
if spikes > 0.05 * len(residual):
return None
return [v if abs(v) <= limit else 0.0 for v in residual]
def spectrum(signal, rate=RATE):
"""[(bpm, power)] within the search band, Hann-windowed and zero-padded."""
n = len(signal)
size = 1
while size < max(n * 4, 256):
size *= 2
window = [0.5 - 0.5 * math.cos(2 * math.pi * i / (n - 1)) for i in range(n)] if n > 1 else [1.0]
padded = [s * w for s, w in zip(signal, window)] + [0.0] * (size - n)
result = fft(padded)
out = []
for k in range(size // 2):
bpm = k * rate / size * 60
if LOW <= bpm <= HIGH:
out.append((bpm, abs(result[k]) ** 2))
return out
def peak(spec):
"""Peak BPM with parabolic interpolation between spectral bins."""
i = max(range(len(spec)), key=lambda k: spec[k][1])
if 0 < i < len(spec) - 1:
a, b, c = spec[i - 1][1], spec[i][1], spec[i + 1][1]
denominator = a - 2 * b + c
offset = 0.5 * (a - c) / denominator if denominator else 0.0
return spec[i][0] + offset * (spec[1][0] - spec[0][0])
return spec[i][0]
def snr(spec, bpm, width=4.0):
"""Power near the pulse and its first harmonic against the rest of the band."""
near = sum(p for f, p in spec if abs(f - bpm) <= width or abs(f - 2 * bpm) <= width)
rest = sum(p for f, p in spec) - near
if near <= 0:
return 0.0
return near / rest if rest > 0 else float("inf")
def normalised(spec):
total = sum(p for _, p in spec) or 1.0
return [p / total for _, p in spec]
def usable(values):
"""Patches that are neither dark nor saturated for the whole recording."""
mean = sum(values) / len(values)
return 8 <= mean <= 245
def estimate(times, patches, rate=RATE, share=0.2, window=WINDOW):
"""Estimate pulse from patch brightness traces.
times: monotonic seconds per frame. patches: {name: [brightness per frame]}.
Selects the share of usable patches with the clearest periodic signal,
combines their spectra and reports the overall and windowed estimates.
"""
cleaned = {}
for name, values in patches.items():
if usable(values):
signal = clean(resample(times, values, rate), rate)
if signal is not None and any(signal): # flat patches carry no rhythm
cleaned[name] = signal
if not cleaned or len(next(iter(cleaned.values()))) < rate * 8:
raise ValueError("need at least 8 seconds of usable eye-camera frames")
quality = []
for name, signal in cleaned.items():
spec = spectrum(signal, rate)
own = peak(spec)
quality.append((snr(spec, own), name, own, spec))
quality.sort(reverse=True)
chosen = quality[:max(4, int(len(quality) * share))]
combined = [sum(values) for values in zip(*(normalised(spec) for _, _, _, spec in chosen))]
bins = [f for f, _ in chosen[0][3]]
bpm = peak(list(zip(bins, combined)))
top = chosen[:8]
agree = sum(1 for _, _, own, _ in top if abs(own - bpm) <= 5) / len(top)
series = []
samples = int(window * rate)
length = len(next(iter(cleaned.values())))
for end in range(samples, length + 1, int(rate)):
specs = [spectrum(cleaned[name][end - samples:end], rate) for _, name, _, _ in chosen]
total = [sum(values) for values in zip(*(normalised(s) for s in specs))]
window_bins = [f for f, _ in specs[0]]
series.append((times[0] + end / rate, peak(list(zip(window_bins, total)))))
return {
"bpm": bpm,
"agreement": agree,
"patches": len(chosen),
"usable": len(cleaned),
"snr": snr(list(zip(bins, combined)), bpm),
"series": series,
}
def analyse(frames):
"""Estimate from Capture.frames(); both eyes' patches are analysed together
on a shared time base, each sample taken from that eye's nearest frame."""
times = sorted({t for t, _, _ in frames})
patches = {}
for eye in ("left", "right"):
eye_frames = sorted((t, grid) for t, e, grid in frames if e == eye)
if not eye_frames:
continue
eye_times = [t for t, _ in eye_frames]
nearest = []
for t in times:
i = bisect.bisect_left(eye_times, t)
if i == len(eye_times) or (i > 0 and t - eye_times[i - 1] <= eye_times[i] - t):
i -= 1
nearest.append(i)
for k in range(len(eye_frames[0][1])):
patches[f"{eye}{k}"] = [eye_frames[i][1][k] for i in nearest]
return estimate(times, patches)
def proximity(root=None):
"""The headset's proximity reading, or None if it can't be read."""
try:
for device in sorted(Path(root or IIO).glob("iio:device*")):
if (device / "name").read_text().strip() == "vcnl4000":
return float((device / "in_proximity_raw").read_text())
except (OSError, ValueError):
pass
return None
def worn(reading):
"""False only when the sensor says the headset is not on a face."""
return reading is None or reading >= WORN_MIN
def reliable(result):
"""Whether the estimate is clear enough to show as a reading. Thresholds
are provisional until checked against a reference on a real wearer."""
return result["agreement"] >= 0.75 and result["snr"] >= 0.5
+438
View File
@@ -0,0 +1,438 @@
#!/usr/bin/env python3
"""Frame-local tracking tools. No network destination or data log by default."""
import argparse
import math
import os
from pathlib import Path
import signal
import socket
import struct
import subprocess
import time
class TrackingError(RuntimeError):
"""A safe, actionable status message containing no sensor data."""
HRS = "0000180d-0000-1000-8000-00805f9b34fb"
MEASUREMENT = "00002a37-0000-1000-8000-00805f9b34fb"
DEVICE = "org.bluez.Device1"
SERVICE = "org.bluez.GattService1"
CHARACTERISTIC = "org.bluez.GattCharacteristic1"
def heart_rate(data):
"""Validate the Bluetooth HRS measurement, returning BPM/contact only.
Energy and RR intervals are checked for length but never retained.
None means contact is supported and the strap reports no skin contact.
"""
data = bytes(data)
if len(data) < 2 or data[0] & 0xe0:
raise ValueError("invalid HRS measurement")
flags = data[0]
size = 2 if flags & 1 else 1
end = 1 + size + (2 if flags & 8 else 0)
if len(data) < end:
raise ValueError("truncated HRS measurement")
extra = len(data) - end
if (flags & 16 and (extra < 2 or extra % 2)) or (not flags & 16 and extra):
raise ValueError("invalid HRS optional fields")
if flags & 4 and not flags & 2:
return None
bpm = int.from_bytes(data[1:1 + size], "little")
return bpm if bpm else None
def gaze_angles(quaternion):
"""OpenXR head-relative -Z forward → VRChat degrees, down/right positive."""
if len(quaternion) != 4 or not all(math.isfinite(v) for v in quaternion):
raise ValueError("invalid gaze orientation")
norm = math.sqrt(sum(v * v for v in quaternion))
if not 0.9 < norm < 1.1:
raise ValueError("invalid gaze orientation")
x, y, z, w = (v / norm for v in quaternion)
# Rotate OpenXR's forward vector (0, 0, -1) into VIEW space.
dx, dy, dz = -2 * (x*z + w*y), 2 * (w*x - y*z), 2 * (x*x + y*y) - 1
return math.degrees(math.atan2(-dy, math.hypot(dx, dz))), math.degrees(math.atan2(dx, -dz))
def osc_message(address, values):
if not address.startswith("/") or any(c.isspace() or c in '\0#*,?[]{}' for c in address):
raise ValueError("OSC address must be a literal path")
def string(value):
encoded = value.encode("utf-8") + b"\0"
return encoded + b"\0" * (-len(encoded) % 4)
tags, payload = ",", b""
for value in values:
if type(value) is int:
tags += "i"
payload += struct.pack(">i", value)
else:
if not math.isfinite(value):
raise ValueError("OSC value must be finite")
tags += "f"
payload += struct.pack(">f", value)
return string(address) + string(tags) + payload
class Osc:
def __init__(self, endpoint=None):
self.sock = None
self.target = None
if endpoint:
import ipaddress
address = ipaddress.ip_address(endpoint[0])
port = int(endpoint[1])
if address.is_unspecified or address.is_multicast or not 1 <= port <= 65535:
raise ValueError("OSC needs a unicast IP address and port 1..65535")
self.sock = socket.socket(socket.AF_INET6 if address.version == 6 else socket.AF_INET, socket.SOCK_DGRAM)
self.target = (str(address), port)
def send(self, address, values):
if self.sock:
self.sock.sendto(osc_message(address, values), self.target)
def close(self):
if self.sock:
self.sock.close()
class HeartSession:
def __init__(self, osc, address, log=None, clock=time.monotonic):
self.osc, self.address, self.clock = osc, address, clock
self.bpm, self.updated = None, None
self.log = None
if log:
# Exclusive creation refuses existing files and symlinks; mode is
# private even with a permissive process umask.
fd = os.open(log, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
self.log = os.fdopen(fd, "w")
self.log.write("unix_seconds,bpm\n")
def notification(self, data):
self.bpm = heart_rate(data)
self.updated = self.clock()
if self.bpm is not None:
self.osc.send(self.address, [self.bpm])
if self.log:
self.log.write(f"{time.time():.3f},{self.bpm}\n")
self.log.flush()
def current(self):
if self.updated is None or self.clock() - self.updated > 5:
return None
return self.bpm
def close(self):
if self.log:
self.log.close()
class BluezHeart:
"""One explicitly selected, already discovered strap; no ambient scan."""
def __init__(self, bus, interface, address, on_value):
self.bus, self.interface, self.on_value = bus, interface, on_value
self.device = self.characteristic = None
self.connected_here = False
self.notifying = False
self.match = None
objects = self.objects()
matches = [path for path, interfaces in objects.items()
if str(interfaces.get(DEVICE, {}).get("Address", "")).upper() == address.upper()]
if len(matches) != 1:
raise TrackingError("Strap not found uniquely in BlueZ; pair/discover it in SteamOS Bluetooth settings first")
self.device = matches[0]
self.match = bus.add_signal_receiver(self.changed, signal_name="PropertiesChanged",
dbus_interface="org.freedesktop.DBus.Properties",
bus_name="org.bluez", path_keyword="path")
try:
if not objects[self.device][DEVICE].get("Connected"):
self.call(self.device, DEVICE).Connect(timeout=20)
self.connected_here = True
except Exception:
self.close()
raise
def objects(self):
return self.call("/", "org.freedesktop.DBus.ObjectManager").GetManagedObjects()
def call(self, path, kind):
return self.interface(self.bus.get_object("org.bluez", path), kind)
def subscribe(self):
objects = self.objects()
if not objects.get(self.device, {}).get(DEVICE, {}).get("ServicesResolved"):
return False
services = {p for p, obj in objects.items() if str(obj.get(SERVICE, {}).get("UUID", "")).lower() == HRS
and obj[SERVICE].get("Device") == self.device}
for path, obj in objects.items():
props = obj.get(CHARACTERISTIC, {})
if props.get("Service") in services and str(props.get("UUID", "")).lower() == MEASUREMENT:
if "notify" not in props.get("Flags", []):
raise TrackingError("Heart-rate characteristic does not support notifications")
self.characteristic = path
self.call(path, CHARACTERISTIC).StartNotify()
self.notifying = True
return True
raise TrackingError("Selected device has no standard Heart Rate Service measurement")
def changed(self, kind, changes, invalidated, path=None):
if kind == CHARACTERISTIC and path == self.characteristic and "Value" in changes:
self.on_value(changes["Value"])
elif kind == DEVICE and path == self.device and "Connected" in changes and not changes["Connected"]:
self.on_value(None)
def close(self):
try:
if self.notifying:
self.call(self.characteristic, CHARACTERISTIC).StopNotify()
finally:
if self.match:
self.match.remove()
if self.connected_here:
self.call(self.device, DEVICE).Disconnect()
def run_gaze(args, osc):
binary = Path(__file__).with_name("gaze")
command = [str(binary), "--seconds", str(args.seconds)]
if not args.osc:
return subprocess.call(command)
read_fd, write_fd = os.pipe()
process = None
try:
process = subprocess.Popen(command + ["--fd", str(write_fd)], pass_fds=(write_fd,))
os.close(write_fd)
write_fd = None
with os.fdopen(read_fd) as source:
read_fd = None
for line in source:
try:
angles = gaze_angles([float(v) for v in line.split()])
except ValueError:
continue
osc.send("/tracking/eye/CenterPitchYaw", angles)
return process.wait()
finally:
if read_fd is not None:
os.close(read_fd)
if write_fd is not None:
os.close(write_fd)
if process and process.poll() is None:
process.terminate()
try:
process.wait(timeout=5)
except subprocess.TimeoutExpired:
process.kill()
process.wait()
class HeartPanel:
"""Our GTK panel, using the Frame's existing GTK4/GI platform libraries."""
def __init__(self):
os.environ["GDK_BACKEND"] = "x11"
import gi
gi.require_version("Gtk", "4.0")
gi.require_version("GdkX11", "4.0")
from gi.repository import Gtk, Gdk, GdkX11, GLib
Gtk.init()
self.running = True
self.window = Gtk.Window(title="Frame Control · Heart rate")
self.window.set_default_size(480, 320)
self.window.connect("close-request", self.stop)
Gtk.Settings.get_default().set_property("gtk-application-prefer-dark-theme", True)
box = Gtk.Box(orientation=Gtk.Orientation.VERTICAL, spacing=16)
box.set_valign(Gtk.Align.CENTER)
box.set_halign(Gtk.Align.CENTER)
box.set_size_request(440, -1)
for side in ("top", "bottom", "start", "end"):
getattr(box, "set_margin_" + side)(24)
self.window.set_child(box)
title = Gtk.Label(label="Heart rate")
title.add_css_class("title-2")
box.append(title)
self.reading = Gtk.Label(label="—")
self.reading.add_css_class("reading")
box.append(self.reading)
self.status = Gtk.Label(label="Waiting for strap")
box.append(self.status)
button = Gtk.Button(label="Stop")
button.connect("clicked", self.stop)
box.append(button)
css = Gtk.CssProvider()
css.load_from_data(b".reading { font-size: 144px; font-weight: 700; }")
Gtk.StyleContext.add_provider_for_display(Gdk.Display.get_default(), css, Gtk.STYLE_PROVIDER_PRIORITY_APPLICATION)
self.window.present()
context = GLib.MainContext.default()
while context.pending():
context.iteration(False)
try:
xid = GdkX11.X11Surface.get_xid(self.window.get_surface())
subprocess.run(["xprop", "-id", str(xid), "-f", "STEAM_GAME", "32c", "-set", "STEAM_GAME", "2000000027"],
check=True, stdout=subprocess.DEVNULL)
except Exception:
self.window.destroy()
raise
def stop(self, *args):
self.running = False
return True
def update(self, bpm):
self.reading.set_label(str(bpm) if bpm is not None else "—")
self.status.set_label("beats per minute" if bpm is not None else "Waiting for strap")
def close(self):
self.window.destroy()
def run_heart(args, osc):
import dbus
from dbus.mainloop.glib import DBusGMainLoop
from gi.repository import GLib
DBusGMainLoop(set_as_default=True)
session = HeartSession(osc, args.address, args.log)
reader, root = None, None
failure = []
def value(data):
if data is None:
session.bpm = None
failure.append("Strap disconnected; reconnect and start again")
return
try:
session.notification(data)
except ValueError:
session.bpm = None
except OSError:
failure.append("OSC or session log write failed")
try:
reader = BluezHeart(dbus.SystemBus(), dbus.Interface, args.device, value)
context = GLib.MainContext.default()
deadline = time.monotonic() + 20
while not reader.subscribe():
if time.monotonic() > deadline:
raise TrackingError("Timed out waiting for the strap's GATT services")
while context.pending():
context.iteration(False)
time.sleep(0.1)
end = time.monotonic() + args.seconds
print("Heart-rate notifications started; readings stay local unless OSC or a log was selected.")
if args.panel:
root = HeartPanel()
running = lambda: root.running if root else True
while running() and time.monotonic() < end and not failure:
while context.pending():
context.iteration(False)
if root:
root.update(session.current())
time.sleep(0.05)
if failure:
raise TrackingError(failure[0])
return 0
finally:
if root:
root.close()
try:
if reader:
reader.close()
finally:
session.close()
def run_pulse(args, osc):
"""Experimental: estimate pulse from the IR eye cameras (see pulse.py)."""
import pulse
if not pulse.ET_BIN.exists():
raise TrackingError("SteamVR's eye-tracking tool is not installed on this Frame")
print(f"Capturing {args.seconds} s from the eye cameras. Wear the headset and keep still.", flush=True)
readings = [pulse.proximity()]
try:
frames = pulse.Capture(args.seconds).run()
readings.append(pulse.proximity())
except RuntimeError as error: # capture status only; no image data
raise TrackingError(str(error))
print(f"Captured {len(frames)} eye frames; images already deleted. Analysing...", flush=True)
try:
result = pulse.analyse(frames)
except ValueError as error:
raise TrackingError(str(error))
on_face = all(pulse.worn(reading) for reading in readings)
if not on_face:
# Unworn, the cameras still show a steady periodic artifact that looks
# like a pulse (verified on the Frame), so it is never called clear.
print("The proximity sensor says the headset is not being worn; no pulse can be read.")
clear = on_face and pulse.reliable(result)
offset = time.time() - time.monotonic() # the capture's timestamps are CLOCK_MONOTONIC
if args.log:
fd = os.open(args.log, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)
with os.fdopen(fd, "w") as log:
log.write("unix_seconds,bpm\n")
for when, bpm in result["series"]:
log.write(f"{when + offset:.3f},{bpm:.1f}\n")
if clear:
osc.send(args.address, [round(result["bpm"])])
if args.show:
print(f"Estimate: {result['bpm']:.1f} BPM ({'clear' if clear else 'NOT clear'}; "
f"patch agreement {result['agreement']:.0%}, signal/noise {result['snr']:.2f}, "
f"{result['patches']} of {result['usable']} usable patches)")
print("Per-second estimates (15 s windows): "
+ " ".join(str(round(bpm)) for _, bpm in result["series"]))
else:
print("A clear pulse was found." if clear else "No clear pulse was found.")
return 0 if clear else 3
def main():
parser = argparse.ArgumentParser(description=__doc__)
commands = parser.add_subparsers(dest="command", required=True)
gaze = commands.add_parser("gaze", help="headless OpenXR; prints counters only without --osc")
heart = commands.add_parser("heart", help="standard BLE HRS from an explicitly selected strap")
pulse = commands.add_parser("pulse", help="experimental pulse estimate from the IR eye cameras")
for command in (gaze, heart, pulse):
command.add_argument("--osc", nargs=2, metavar=("IP", "PORT"), help="explicit UDP destination; no default")
for command in (gaze, heart):
command.add_argument("--seconds", type=int, default=10, help="bounded run, 1..86400 seconds (default: 10)")
pulse.add_argument("--seconds", type=int, default=60, help="capture length, 20..300 seconds (default: 60)")
heart.add_argument("--device", required=True, help="strap Bluetooth address already discovered by BlueZ")
heart.add_argument("--panel", action="store_true", help="show our panel on gamescope DISPLAY=:0")
for command in (heart, pulse):
command.add_argument("--address", default="/avatar/parameters/HeartRate", help="integer BPM OSC parameter")
command.add_argument("--log", help="new private CSV file; disabled by default")
pulse.add_argument("--show", action="store_true", help="print the estimate and per-second series")
args = parser.parse_args()
if not 1 <= args.seconds <= 86400:
parser.error("--seconds must be 1..86400")
if args.command == "pulse" and not 20 <= args.seconds <= 300:
parser.error("pulse --seconds must be 20..300")
def interrupted(signum, frame):
raise KeyboardInterrupt
signal.signal(signal.SIGTERM, interrupted)
if hasattr(signal, "SIGHUP"):
signal.signal(signal.SIGHUP, interrupted)
osc = None
try:
if args.command in ("heart", "pulse"):
osc_message(args.address, [0])
if getattr(args, "panel", False):
os.environ["DISPLAY"] = ":0"
osc = Osc(args.osc)
run = {"gaze": run_gaze, "heart": run_heart, "pulse": run_pulse}[args.command]
return run(args, osc)
except TrackingError as error:
print(str(error))
return 1
except KeyboardInterrupt:
return 130
except Exception as error:
# Never dump notifications, gaze, BLE addresses or exception payloads.
print(f"Tracking stopped ({type(error).__name__}). Check the device, runtime and selected output.")
return 1
finally:
if osc:
osc.close()
if __name__ == "__main__":
raise SystemExit(main())
+274
View File
@@ -0,0 +1,274 @@
#!/usr/bin/env python3
"""Check Frame Control's heart-rate readings against a reference, on your computer.
python3 scripts/heart-check.py listen --port 9000 --out ours.csv
python3 scripts/heart-check.py compare ours.csv reference.csv
python3 scripts/heart-check.py compare ours.csv ~/Downloads/export.zip
`listen` receives our integer-BPM OSC messages (send them here with
`tracking-on-frame.py heart --osc <this computer's IP> 9000`) and shows each
reading live, so you can watch it next to a reference such as your Apple
Watch. `--out` also records `unix_seconds,bpm` to a new private file.
`compare` lines up two recordings by time and reports how far apart they are.
The reference can be:
- a CSV whose first column is a time (unix seconds or ISO 8601) and whose
second is BPM, such as our own log, `listen --out`, or the test strap's
output (a third `flags` column marks skin-contact loss as "no reading");
- an Apple Health export (`export.zip` or `export.xml`, from Health → your
profile → Export All Health Data). Only heart-rate records within the
recording's time range are read.
Everything stays on this computer. Nothing is uploaded.
"""
import argparse
import bisect
import csv
from datetime import datetime
import io
import os
from pathlib import Path
import re
import socket
import struct
import sys
import time
import zipfile
from xml.etree import ElementTree
ADDRESS = "/avatar/parameters/HeartRate"
def parse_osc(packet):
"""Return (address, values) for a single OSC message with i/f arguments."""
def string(offset):
end = packet.index(b"\0", offset)
return packet[offset:end].decode("utf-8"), (end + 4) & ~3
address, offset = string(0)
tags, offset = string(offset)
if not tags.startswith(","):
raise ValueError("not an OSC message")
values = []
for tag in tags[1:]:
if tag not in "if" or offset + 4 > len(packet):
raise ValueError("unsupported OSC argument")
values.append(struct.unpack(">i" if tag == "i" else ">f", packet[offset:offset + 4])[0])
offset += 4
return address, values
def listen(port, address, out, seconds, clock=time.time, stream=sys.stdout):
log = None
if out:
log = os.fdopen(os.open(out, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600), "w")
log.write("unix_seconds,bpm\n")
received = 0
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
sock.bind(("0.0.0.0", port))
sock.settimeout(0.5)
print(f"Listening for {address} on UDP port {port}. Ctrl-C stops.", file=stream, flush=True)
end = clock() + seconds if seconds else None
try:
while end is None or clock() < end:
try:
packet = sock.recv(1024)
except socket.timeout:
continue
try:
path, values = parse_osc(packet)
except (ValueError, UnicodeDecodeError):
continue
if path != address or len(values) != 1:
continue
now = clock()
bpm = int(values[0])
received += 1
print(f"{time.strftime('%H:%M:%S', time.localtime(now))} {bpm:3d} bpm", file=stream, flush=True)
if log:
log.write(f"{now:.3f},{bpm}\n")
log.flush()
except KeyboardInterrupt:
pass
finally:
if log:
log.close()
return received
def parse_time(text):
text = text.strip()
if re.fullmatch(r"\d+(\.\d+)?", text):
return float(text)
# Apple Health uses "2026-09-29 09:12:03 +1000".
text = re.sub(r" ([+-]\d{2}):?(\d{2})$", r"\1\2", text).replace("Z", "+0000")
for pattern in ("%Y-%m-%d %H:%M:%S%z", "%Y-%m-%dT%H:%M:%S%z", "%Y-%m-%dT%H:%M:%S.%f%z"):
try:
return datetime.strptime(text, pattern).timestamp()
except ValueError:
pass
raise ValueError(f"unrecognised time {text!r}")
def read_csv(path):
"""[(time, bpm or None)], sorted. A header row is skipped."""
samples = []
with open(path, newline="") as source:
for row in csv.reader(source):
if len(row) < 2:
continue
try:
when, bpm = parse_time(row[0]), int(float(row[1]))
except (ValueError, OverflowError):
continue # header or unparseable line
try:
flags = int(float(row[2])) if len(row) > 2 else None
except (ValueError, OverflowError):
flags = None # an unrecognised flag leaves the reading as it is
if flags is not None and flags & 4 and not flags & 2:
bpm = None # contact supported and not detected: no reading
samples.append((when, bpm))
return sorted(samples, key=lambda s: s[0])
def read_health(path, start, end):
"""Heart-rate records from an Apple Health export between start and end."""
samples = []
def scan(source):
for _, element in ElementTree.iterparse(source):
if element.tag == "Record" and element.get("type") == "HKQuantityTypeIdentifierHeartRate":
try:
when = parse_time(element.get("startDate") or "")
value = round(float(element.get("value") or ""))
except (ValueError, OverflowError):
continue
if start <= when <= end:
samples.append((when, value))
element.clear()
if zipfile.is_zipfile(path):
with zipfile.ZipFile(path) as archive:
names = [n for n in archive.namelist() if n.endswith("/export.xml") or n == "export.xml"]
if not names:
raise ValueError("no export.xml in that archive; use Health's Export All Health Data")
with archive.open(names[0]) as source:
scan(source)
else:
with open(path, "rb") as source:
scan(source)
return sorted(samples)
def read_any(path, start=None, end=None):
path = str(path)
if path.endswith((".zip", ".xml")):
return read_health(path, start, end)
return read_csv(path)
def value_at(samples, when, hold, times=None):
"""Our reading at a moment: the latest sample no older than `hold` seconds.
`times` is the samples' time column, if the caller has already built it."""
times = times if times is not None else [s[0] for s in samples]
i = bisect.bisect_right(times, when) - 1
if i < 0 or when - times[i] > hold:
return None
return samples[i][1]
def compare(ours, reference, max_lag=10.0, hold=5.0):
"""Compare our readings with the reference at each reference moment.
`lag` is the delay added to reference times before looking up ours (our
readings arrive after the sensor's). The best lag within ±max_lag is used.
"""
real = [(t, b) for t, b in reference if b is not None]
if not real or not any(b is not None for _, b in ours):
raise ValueError("both recordings need at least one reading")
best = None
ours_times = [t for t, _ in ours]
steps = int(max_lag * 4)
for step in range(-steps, steps + 1):
lag = step / 4
pairs = [(value_at(ours, t + lag, hold, ours_times), b) for t, b in real]
pairs = [(o, r) for o, r in pairs if o is not None]
if not pairs:
continue
error = sum(abs(o - r) for o, r in pairs) / len(pairs)
key = (-len(pairs), error, abs(lag))
if best is None or key < best[0]:
best = (key, lag, pairs)
if best is None:
raise ValueError("the recordings do not overlap in time")
_, lag, pairs = best
differences = [o - r for o, r in pairs]
# While the reference says "no reading" (lost skin contact), we must not
# produce new readings. Count ours that arrive during such a stretch.
gaps = [t for t, b in reference if b is None]
reference_times = [t for t, _ in reference]
shown_in_gaps = 0
for t, bpm in ours:
i = bisect.bisect_right(reference_times, t - lag) - 1
if bpm is not None and i >= 0 and reference[i][1] is None:
shown_in_gaps += 1
return {
"reference_readings": len(real),
"matched": len(pairs),
"lag_seconds": lag,
"mean_abs_error": sum(abs(d) for d in differences) / len(differences),
"bias": sum(differences) / len(differences),
"max_abs_error": max(abs(d) for d in differences),
"within_5": sum(1 for d in differences if abs(d) <= 5) / len(differences),
"exact": sum(1 for d in differences if d == 0) / len(differences),
"no_contact_moments": len(gaps),
"shown_during_no_contact": shown_in_gaps,
}
def report(result, tolerance, stream=sys.stdout):
print(f"Reference readings: {result['reference_readings']}, matched: {result['matched']}", file=stream)
print(f"Best alignment: ours {result['lag_seconds']:+.2f} s after the reference", file=stream)
print(f"Mean absolute difference: {result['mean_abs_error']:.2f} BPM "
f"(bias {result['bias']:+.2f}, worst {result['max_abs_error']})", file=stream)
print(f"Within ±5 BPM: {result['within_5']:.0%}; identical: {result['exact']:.0%}", file=stream)
if result["no_contact_moments"]:
print(f"No-contact moments: {result['no_contact_moments']}, where we showed a stale "
f"reading: {result['shown_during_no_contact']}", file=stream)
coverage = result["matched"] / result["reference_readings"]
passed = (result["mean_abs_error"] <= tolerance and coverage >= 0.8
and not result["shown_during_no_contact"])
print(("PASS" if passed else "FAIL") + f" (mean difference ≤ {tolerance} BPM, ≥80% of reference "
f"readings matched, nothing shown without contact)", file=stream)
return passed
def main(argv=None):
parser = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
commands = parser.add_subparsers(dest="command", required=True)
heard = commands.add_parser("listen", help="show and optionally record our OSC readings live")
heard.add_argument("--port", type=int, default=9000)
heard.add_argument("--address", default=ADDRESS)
heard.add_argument("--out", help="new private CSV file")
heard.add_argument("--seconds", type=float, default=0, help="stop after this long (default: until Ctrl-C)")
check = commands.add_parser("compare", help="compare our recording with a reference")
check.add_argument("ours", type=Path)
check.add_argument("reference", type=Path)
check.add_argument("--tolerance", type=float, default=5.0, help="allowed mean difference in BPM (default 5)")
check.add_argument("--max-lag", type=float, default=10.0, help="largest time offset to search, seconds")
args = parser.parse_args(argv)
if args.command == "listen":
count = listen(args.port, args.address, args.out, args.seconds)
print(f"Received {count} readings.")
return 0 if count else 3
try:
ours = read_csv(args.ours)
if not ours:
raise ValueError("our recording has no readings")
reference = read_any(args.reference, ours[0][0] - 60, ours[-1][0] + 60)
result = compare(ours, reference, args.max_lag)
except (ValueError, OSError, csv.Error, ElementTree.ParseError, zipfile.BadZipFile) as error:
print(f"Could not compare: {error}")
return 1
return 0 if report(result, args.tolerance) else 1
if __name__ == "__main__":
raise SystemExit(main())
+100
View File
@@ -0,0 +1,100 @@
// A synthetic Bluetooth heart-rate strap for testing, run on the Mac.
//
// swiftc -O scripts/heart-test-strap.swift -o /tmp/heart-test-strap
// /tmp/heart-test-strap [seconds] # default 60
//
// Advertises the standard Heart Rate Service (180d) as "FC Test Strap" and,
// while a central is subscribed, sends one Heart Rate Measurement (2a37) per
// second from a fixed, known sequence. Each sent value is printed to stdout as
// `unix_seconds,bpm,flags` so scripts/heart-check.py can compare what the
// Frame received with what was sent. Every value is synthetic; this is not a
// sensor. macOS asks for Bluetooth permission the first time it runs.
import CoreBluetooth
import Foundation
let hrs = CBUUID(string: "180D")
let measurement = CBUUID(string: "2A37")
/// The known sequence: an 8-bit ramp, 16-bit encodings, a skin-contact loss
/// (which must show as no reading) and a recovery.
func packet(_ second: Int) -> (bpm: Int, flags: UInt8) {
switch second % 60 {
case 0..<30: return (60 + second % 60 * 4, 0x06) // 60…176, contact detected
case 30..<40: return (180 - (second % 60 - 30) * 3, 0x07) // 16-bit value
case 40..<45: return (150, 0x04) // contact lost
default: return (72 + (second % 60 - 45), 0x06)
}
}
final class Strap: NSObject, CBPeripheralManagerDelegate {
let seconds: Int
var manager: CBPeripheralManager!
var characteristic: CBMutableCharacteristic!
var subscribed = false
var sent = 0
init(seconds: Int) {
self.seconds = seconds
super.init()
manager = CBPeripheralManager(delegate: self, queue: nil)
}
func peripheralManagerDidUpdateState(_ peripheral: CBPeripheralManager) {
guard peripheral.state == .poweredOn else {
FileHandle.standardError.write("Bluetooth state \(peripheral.state.rawValue)\n".data(using: .utf8)!)
if peripheral.state == .unauthorized || peripheral.state == .unsupported || peripheral.state == .poweredOff {
FileHandle.standardError.write("Bluetooth unavailable (state \(peripheral.state.rawValue))\n".data(using: .utf8)!)
exit(1)
}
return
}
characteristic = CBMutableCharacteristic(type: measurement, properties: [.notify], value: nil, permissions: [])
let service = CBMutableService(type: hrs, primary: true)
service.characteristics = [characteristic]
peripheral.add(service)
}
func peripheralManager(_ peripheral: CBPeripheralManager, didAdd service: CBService, error: Error?) {
if let error { fail("add service: \(error.localizedDescription)") }
peripheral.startAdvertising([CBAdvertisementDataLocalNameKey: "FC Test Strap",
CBAdvertisementDataServiceUUIDsKey: [hrs]])
}
func peripheralManagerDidStartAdvertising(_ peripheral: CBPeripheralManager, error: Error?) {
if let error { fail("advertise: \(error.localizedDescription)") }
FileHandle.standardError.write("Advertising FC Test Strap\n".data(using: .utf8)!)
Timer.scheduledTimer(withTimeInterval: 1, repeats: true) { _ in self.tick() }
}
func peripheralManager(_ peripheral: CBPeripheralManager, central: CBCentral, didSubscribeTo characteristic: CBCharacteristic) {
subscribed = true
FileHandle.standardError.write("Central subscribed\n".data(using: .utf8)!)
}
func peripheralManager(_ peripheral: CBPeripheralManager, central: CBCentral, didUnsubscribeFrom characteristic: CBCharacteristic) {
subscribed = false
FileHandle.standardError.write("Central unsubscribed\n".data(using: .utf8)!)
}
func tick() {
guard subscribed else { return }
if sent >= seconds { exit(0) }
let (bpm, flags) = packet(sent)
var bytes: [UInt8] = [flags, UInt8(bpm & 0xff)]
if flags & 1 != 0 { bytes.append(UInt8(bpm >> 8)) }
if manager.updateValue(Data(bytes), for: characteristic, onSubscribedCentrals: nil) {
print(String(format: "%.3f,%d,%d", Date().timeIntervalSince1970, bpm, flags))
fflush(stdout)
sent += 1
}
}
func fail(_ message: String) -> Never {
FileHandle.standardError.write("\(message)\n".data(using: .utf8)!)
exit(1)
}
}
let seconds = CommandLine.arguments.count > 1 ? Int(CommandLine.arguments[1]) ?? 60 : 60
let strap = Strap(seconds: seconds)
RunLoop.main.run()
-74
View File
@@ -1,74 +0,0 @@
#!/usr/bin/env zsh
# Mac-side: stop the Steam Frame from going to sleep while an agent works on it.
#
# The Frame sleeps when Steam's own idle timer runs out ("Sleep after
# inactivity": 60 min on AC, 15 min on battery by default). SSH activity
# doesn't count as input, and asleep the Frame is off the network. `on` sets
# both timers to Never through Steam's UI (DevTools on 127.0.0.1:8080, via
# ui/frame_steam.py) and holds a logind sleep inhibitor as a user unit.
# `off` drops the inhibitor and restores the timers `on` saved.
#
# Usage:
# scripts/keep-awake.sh on
# scripts/keep-awake.sh off
# scripts/keep-awake.sh status
set -euo pipefail
FRAME_ALIAS=${FRAME_ALIAS:-frame}
HERE=${0:A:h}
cmd=${1:-status}
case $cmd in on|off|status) ;; *) echo "usage: keep-awake.sh on|off|status" >&2; exit 2 ;; esac
ssh -o ConnectTimeout=8 "$FRAME_ALIAS" \
'mkdir -p ~/.cache/frame-control && cat > ~/.cache/frame-control/frame_steam.py' < "$HERE/../ui/frame_steam.py"
# Runs on the Frame. Verified 2026-09-28 (BUILD_ID 20260925.6191901): the
# timers are client settings system_idle_suspend_{ac,battery}_sec (0 = Never),
# written the way Steam's settings page does (steamui module exporting the
# SetSetting wrapper). logind refuses an inhibitor from an SSH session
# ("Interactive authentication required") but allows one from a user unit.
ssh "$FRAME_ALIAS" python3 - "$cmd" <<'EOF'
import json, os, subprocess, sys
sys.path.insert(0, os.path.expanduser("~/.cache/frame-control"))
from frame_steam import Page
cmd = sys.argv[1]
saved_path = os.path.expanduser("~/.cache/frame-control/keep-awake.json")
unit = "fc-keep-awake"
keys = ("system_idle_suspend_ac_sec", "system_idle_suspend_battery_sec")
if cmd == "off": # release the lock first, even if Steam's UI is down
subprocess.run(["systemctl", "--user", "stop", unit], stderr=subprocess.DEVNULL)
page = Page()
def read():
return {k: page.eval(f"settingsStore.clientSettings.{k}") for k in keys}
def write(values):
page.eval("""(async () => { let req;
webpackChunksteamui.push([[Symbol()], {}, r => { req = r }]);
const mod = Object.keys(req.m).map(id => req.m[id].toString().includes("Settings.SetSetting") ? req(id) : null).find(Boolean);
const set = Object.values(mod).find(f => typeof f == "function" && f.toString().includes("SetSetting("));
for (const [k, v] of Object.entries(%s)) await set(k, v);
await new Promise(r => setTimeout(r, 1000)); })()""" % json.dumps(values))
def inhibitor():
return subprocess.run(["systemctl", "--user", "is-active", "-q", unit]).returncode == 0
if cmd == "on":
current = read()
if not os.path.exists(saved_path):
with open(saved_path, "w") as f:
json.dump(current, f)
write({k: 0 for k in keys})
if not inhibitor():
subprocess.run(["systemd-run", "--user", "-q", f"--unit={unit}",
"--description=Frame Control: keep the Frame awake",
"systemd-inhibit", "--what=sleep:idle:handle-suspend-key:handle-power-key",
"--who=Frame Control", "--why=Keep the Frame awake while an agent works on it",
"--mode=block", "sleep", "infinity"], check=True)
elif cmd == "off":
if os.path.exists(saved_path): # no backup: leave the timers as they are
with open(saved_path) as f:
write(json.load(f))
os.remove(saved_path)
print(json.dumps({"timers": read(), "inhibitor": inhibitor()}))
EOF
+60
View File
@@ -0,0 +1,60 @@
#!/usr/bin/env python3
"""Install or run our local-only tracking tools on the Frame.
python3 scripts/tracking-on-frame.py install
python3 scripts/tracking-on-frame.py gaze --seconds 10
python3 scripts/tracking-on-frame.py gaze --seconds 3600 --osc 127.0.0.1 9000
python3 scripts/tracking-on-frame.py heart --device AA:BB:CC:DD:EE:FF --panel
python3 scripts/tracking-on-frame.py pulse --seconds 60 --show # experimental
FRAME_ALIAS overrides the SSH alias (default: frame). Runs in the foreground;
Ctrl-C stops the reader. No service, autostart, sudo or SteamVR settings changes.
"""
import os
from pathlib import Path
import shlex
import subprocess
import sys
import tarfile
REMOTE = '"$HOME/.local/share/frame-control/tracking"'
INSTALL = '''set -eu
base="$HOME/.local/share/frame-control/tracking"
mkdir -p "$base"
stage=$(mktemp -d "$base/.install.XXXXXX")
trap 'rm -rf "$stage"' EXIT
tar -xf - -C "$stage"
cc -O2 -Wall -Wextra -Werror "$stage/gaze.c" \\
-L/opt/steamvr/bin/linuxarm64 -Wl,-rpath,/opt/steamvr/bin/linuxarm64 \\
-lopenxr_loader -o "$stage/gaze"
chmod 700 "$stage/gaze" "$stage/tracking.py"
chmod 600 "$stage/pulse.py"
mv "$stage/gaze" "$stage/tracking.py" "$stage/pulse.py" "$base/"
echo 'Installed Frame Control tracking tools (no service started).'
'''
def main():
if len(sys.argv) < 2 or sys.argv[1] not in ("install", "gaze", "heart", "pulse"):
print(__doc__)
return 2
host = os.environ.get("FRAME_ALIAS", "frame")
if not host or host.startswith("-"):
raise ValueError("invalid SSH alias")
ssh = ["ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=10", host]
if sys.argv[1] == "install":
import tempfile
source = Path(__file__).resolve().parents[1] / "frame" / "tracking"
with tempfile.TemporaryFile() as archive:
with tarfile.open(fileobj=archive, mode="w") as tar:
for name in ("gaze.c", "tracking.py", "pulse.py"):
tar.add(source / name, arcname=name)
archive.seek(0)
return subprocess.call(ssh + ["bash -c " + shlex.quote(INSTALL)], stdin=archive)
# Allocate a tty so SSH forwards Ctrl-C and hangup to the foreground process.
command = 'exec python3 ' + REMOTE + '/tracking.py ' + shlex.join(sys.argv[1:])
return subprocess.call(ssh[:-1] + ["-tt", host, command])
if __name__ == "__main__":
raise SystemExit(main())
+4 -2
View File
@@ -35,7 +35,7 @@ def manifest(package, label_ref, version_ref, min_sdk, package_raw=True, foreign
foreign_label adds a non-android `label` attribute after android:label. foreign_label adds a non-android `label` attribute after android:label.
""" """
strings = ['label', 'icon', 'versionName', 'minSdkVersion', 'package', 'manifest', 'uses-sdk', strings = ['label', 'icon', 'versionName', 'minSdkVersion', 'package', 'manifest', 'uses-sdk',
'application', package, 'junk', 'label'] # the second 'label' has no android id 'application', package, 'junk', 'label', 'versionCode'] # the second 'label' has no android id
resmap = struct.pack('<4I', 0x01010001, 0x01010002, 0x0101021c, 0x0101020c) resmap = struct.pack('<4I', 0x01010001, 0x01010002, 0x0101021c, 0x0101020c)
resmap = struct.pack('<HHI', 0x0180, 8, 8 + len(resmap)) + resmap resmap = struct.pack('<HHI', 0x0180, 8, 8 + len(resmap)) + resmap
@@ -48,7 +48,8 @@ def manifest(package, label_ref, version_ref, min_sdk, package_raw=True, foreign
none = 0xffffffff none = 0xffffffff
chunks = (pool(strings) + resmap chunks = (pool(strings) + resmap
+ element(5, [(4, 8 if package_raw else none, frame_apk.T_STRING, 8), + element(5, [(4, 8 if package_raw else none, frame_apk.T_STRING, 8),
(2, none, frame_apk.T_REF, version_ref)]) (2, none, frame_apk.T_REF, version_ref),
(11, none, frame_apk.T_INT_DEC, 210)])
+ element(6, [(3, none, frame_apk.T_INT_DEC, min_sdk)]) + element(6, [(3, none, frame_apk.T_INT_DEC, min_sdk)])
+ element(7, [(0, none, frame_apk.T_REF, label_ref), (1, none, frame_apk.T_REF, 0x7f020000)] + element(7, [(0, none, frame_apk.T_REF, label_ref), (1, none, frame_apk.T_REF, 0x7f020000)]
+ ([(10, 9, frame_apk.T_STRING, 9)] if foreign_label else []))) + ([(10, 9, frame_apk.T_STRING, 9)] if foreign_label else [])))
@@ -108,6 +109,7 @@ class ApkInfo(unittest.TestCase):
self.assertEqual(info['package'], 'com.example.demo') self.assertEqual(info['package'], 'com.example.demo')
self.assertEqual(info['label'], 'App label') # the default, not French self.assertEqual(info['label'], 'App label') # the default, not French
self.assertEqual(info['version'], '2.1') self.assertEqual(info['version'], '2.1')
self.assertEqual(info['version_code'], 210)
self.assertEqual(info['min_sdk'], 26) self.assertEqual(info['min_sdk'], 26)
self.assertEqual(info['abis'], ['arm64-v8a', 'x86_64']) self.assertEqual(info['abis'], ['arm64-v8a', 'x86_64'])
self.assertEqual(info['icon_png'], b'hi') # largest-density PNG, skipping the XML icon self.assertEqual(info['icon_png'], b'hi') # largest-density PNG, skipping the XML icon
+260
View File
@@ -0,0 +1,260 @@
"""Offline version lookup with small index-v2 fixtures."""
import io
import json
import os
import sys
import tempfile
import time
import unittest
from concurrent.futures import ThreadPoolExecutor
from unittest.mock import patch
sys.path.insert(0, os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'ui'))
import frame_apk_versions as versions
import frame_catalog
import frame_android
def build(code, sdk=30, abis=None):
return {'manifest': {'versionName': str(code), 'versionCode': code,
'usesSdk': {'minSdkVersion': sdk}, 'nativecode': abis or []},
'file': {'name': f'/example_{code}.apk', 'sha256': str(code).zfill(64)}}
class VersionsTest(unittest.TestCase):
def setUp(self):
self.tmp = tempfile.TemporaryDirectory()
self.addCleanup(self.tmp.cleanup)
data = os.path.join(self.tmp.name, 'data')
os.mkdir(data)
for name, builds in [('index-v2.json', [build(5, 33), build(4, abis=['x86_64']),
build(3, abis=['arm64-v8a', 'x86_64']), build(2)]),
('index-v2.archive.json', [build(1, 21), build(2)]),
('index-v2.izzy.json', [])]:
with open(os.path.join(data, name), 'w') as f:
json.dump({'packages': {'org.example.app': {'versions': {str(i): b for i, b in enumerate(builds)}}}}, f)
self.enter_patch(patch.object(frame_catalog, 'CATALOG', self.tmp.name))
self.enter_patch(patch.dict(os.environ, {'FRAME_CONTROL_APP': ''}))
self.network = self.enter_patch(patch.object(frame_catalog.urllib.request, 'urlopen', side_effect=AssertionError('network used')))
def enter_patch(self, p):
result = p.start()
self.addCleanup(p.stop)
return result
def test_filter_order_archive_and_dedup(self):
result = versions.alternatives('org.example.app')
self.assertEqual([v['version_code'] for v in result['versions']], [3, 2, 1])
self.assertEqual(result['versions'][-1]['source'], 'F-Droid archive')
self.assertEqual(result['versions'][-1]['url'], 'https://f-droid.org/archive/example_1.apk')
self.assertEqual(result['errors'], [])
self.network.assert_not_called()
def test_current_version(self):
result = versions.alternatives('org.example.app', 3)
self.assertEqual([v['version_code'] for v in result['versions']], [2, 1])
def test_fallback(self):
result = versions.alternatives('com.missing.app')
self.assertEqual(result['versions'], [])
self.assertEqual([v['source'] for v in result['links']], ['APKMirror', 'APKPure', 'Uptodown', 'F-Droid', 'GitHub'])
self.assertTrue(all('com.missing.app' in v['url'] for v in result['links']))
self.assertIn('Android 11', result['note'])
self.assertIn('arm64-v8a', result['note'])
def test_failed_indexes_keep_search_links(self):
with patch.object(frame_catalog, 'load_index', side_effect=OSError('offline')):
result = versions.alternatives('org.example.app')
self.assertEqual(len(result['errors']), 3)
self.assertEqual(len(result['links']), 5)
def test_no_compatible_versions(self):
with patch.object(frame_catalog, 'load_index', return_value={}):
result = versions.alternatives('org.example.app')
self.assertEqual(result['versions'], [])
self.assertEqual(len(result['links']), 5)
def test_reduction_memory_cache_and_refresh(self):
repo = versions.REPOS[0][1]
index = frame_catalog.load_index(repo)
self.assertEqual([v['version_code'] for v in index['org.example.app']], [3, 2])
self.assertEqual(set(index['org.example.app'][0]),
{'version', 'version_code', 'min_sdk', 'abis', 'name', 'sha256'})
raw = os.path.join(self.tmp.name, 'data', 'index-v2.json')
self.assertTrue(os.path.exists(raw)) # the catalogue build reads it
os.utime(raw, ns=(1, 1))
with patch.object(frame_catalog.json, 'load', side_effect=AssertionError('reparsed')):
self.assertIs(frame_catalog.load_index(repo), index)
path = raw + '.installable-v1'
with open(path, 'w') as f:
json.dump({}, f)
os.utime(path, ns=(1, 1))
self.assertEqual(frame_catalog.load_index(repo, cached_only=True), {})
payload = json.dumps({'packages': {'org.example.app': {'versions': {'x': build(9)}}}}).encode()
with patch.object(frame_catalog.urllib.request, 'urlopen', return_value=io.BytesIO(payload)) as fetch:
self.assertEqual(frame_catalog.load_index(repo)['org.example.app'][0]['version_code'], 9)
fetch.assert_called_once()
self.assertTrue(os.path.exists(raw))
def test_malformed_entries_are_skipped(self):
raw = os.path.join(self.tmp.name, 'odd.json')
with open(raw, 'w') as f:
json.dump({'packages': {'a.b': {'versions': {'x': {'manifest': {}}, 'y': None, 'z': build(4)}},
'c.d': {'versions': None}, 'e.f': []}}, f)
self.assertEqual([v['version_code'] for v in frame_catalog._reduce_index(raw)['a.b']], [4])
def test_one_failing_repo_keeps_the_others(self):
real = frame_catalog.load_index
def load(repo, cached_only=False):
if 'izzy' in repo:
raise KeyError('file')
return real(repo, cached_only=cached_only)
with patch.object(frame_catalog, 'load_index', side_effect=load):
result = versions.alternatives('org.example.app')
self.assertEqual([v['version_code'] for v in result['versions']], [3, 2, 1])
self.assertEqual(len(result['errors']), 1)
def test_newer_raw_index_outdates_reduced_copy(self):
repo = versions.REPOS[0][1]
frame_catalog.load_index(repo)
raw = os.path.join(self.tmp.name, 'data', 'index-v2.json')
with open(raw, 'w') as f:
json.dump({'packages': {'org.example.app': {'versions': {'x': build(8)}}}}, f)
os.utime(raw, ns=(time.time_ns() + 10**9,) * 2)
self.assertEqual(frame_catalog.load_index(repo)['org.example.app'][0]['version_code'], 8)
def test_concurrent_requests_share_download(self):
raw = os.path.join(self.tmp.name, 'data', 'index-v2.json')
os.remove(raw)
payload = json.dumps({'packages': {'org.example.app': {'versions': {'x': build(7)}}}}).encode()
with patch.object(frame_catalog.urllib.request, 'urlopen', side_effect=lambda *a, **k: io.BytesIO(payload)) as fetch:
with ThreadPoolExecutor(max_workers=4) as pool:
indexes = list(pool.map(frame_catalog.load_index, [versions.REPOS[0][1]] * 4))
fetch.assert_called_once()
self.assertTrue(all(index is indexes[0] for index in indexes))
def test_failed_refresh_preserves_cache(self):
repo = versions.REPOS[0][1]
index = frame_catalog.load_index(repo)
path = os.path.join(self.tmp.name, 'data', 'index-v2.json.installable-v1')
os.utime(path, ns=(1, 1))
os.utime(os.path.join(self.tmp.name, 'data', 'index-v2.json'), ns=(1, 1))
with patch.object(frame_catalog.urllib.request, 'urlopen', return_value=io.BytesIO(b'{')):
with self.assertRaises(ValueError):
frame_catalog.load_index(repo)
self.assertEqual(frame_catalog.load_index(repo, cached_only=True), index)
self.assertFalse(any(name.endswith('.part') for name in os.listdir(os.path.dirname(path))))
def test_stream_boundaries_and_invalid_index(self):
raw = os.path.join(self.tmp.name, 'stream.json')
with open(raw, 'w') as f:
json.dump({'repo': {'description': 'é' * 70000}, 'packages': {
'org.example.app': {'metadata': {'text': 'escaped " packages { }' * 6000},
'versions': {'x': build(7)}}}, 'tail': {}}, f)
self.assertEqual(frame_catalog._reduce_index(raw)['org.example.app'][0]['version_code'], 7)
for invalid in ('{}', '{"packages": []}', '{"packages": {', '{"packages": {}} trailing'):
with open(raw, 'w') as f:
f.write(invalid)
with self.assertRaises(ValueError):
frame_catalog._reduce_index(raw)
def test_cap_and_preferred_build(self):
records = [dict(version=str(i), version_code=i, min_sdk=21, abis=[],
name='/app_%s.apk' % i, sha256=str(i)) for i in range(20)]
records += [dict(records[-1], version_code=21, abis=['arm64-v8a'], name='/arm.apk'),
dict(records[-1], version_code=22, abis=['arm64-v8a', 'x86_64'], name='/all.apk')]
with patch.object(frame_catalog, 'load_index', return_value={'org.example.app': records}):
result = versions.alternatives('org.example.app')
self.assertEqual(result['total'], 22)
self.assertEqual(len(result['versions']), 8)
self.assertEqual(len({v['version'] for v in result['versions']}), 8)
self.assertEqual([v['version_code'] for v in result['versions']], [21, 18, 17, 16, 15, 14, 13, 12])
def test_android_names_and_verdict(self):
for sdk, name in [(23, 'Android 6.0'), (30, 'Android 11'), (32, 'Android 12L'), (33, 'Android 13'), (99, 'Android API 99')]:
self.assertEqual(versions.android_name(sdk), name)
info = {'package': 'org.example.app', 'label': 'Example', 'version': '5.0',
'version_code': 50, 'min_sdk': 33, 'abis': ['arm64-v8a']}
description = versions.describe(info)
for text in ['org.example.app', '5.0', 'code 50', 'Android 13', 'arm64-v8a', 'cannot install']:
self.assertIn(text, description)
info.update(min_sdk=30, abis=[])
self.assertIn('can install', versions.describe(info))
info['abis'] = ['armeabi-v7a']
self.assertIn('no arm64-v8a build', versions.describe(info))
def test_install_resolves_index_hash(self):
versions.alternatives('org.example.app')
with patch.object(versions, 'alternatives', side_effect=AssertionError('recomputed')), \
patch.object(frame_catalog, 'fetch_apk', return_value='/tmp/example.apk') as fetch, \
patch.object(frame_android, 'apk_info', return_value={'package': 'org.example.app', 'version_code': 1}), \
patch.object(frame_android, 'install', return_value={'label': 'Example'}) as install:
versions.install('org.example.app', 'https://f-droid.org/archive/example_1.apk')
self.assertEqual(fetch.call_args[0][0]['h'], str(1).zfill(64))
install.assert_called_once_with('/tmp/example.apk', source='F-Droid archive')
with self.assertRaises(frame_android.FrameError):
versions.install('org.example.app', 'https://evil.example/app.apk')
def test_install_checks_identity_without_network_refresh(self):
versions.alternatives('org.example.app')
for name in ('index-v2.json', 'index-v2.archive.json'):
os.utime(os.path.join(self.tmp.name, 'data', name + '.installable-v1'), ns=(1, 1))
for info in ({'package': 'wrong.package', 'version_code': 1},
{'package': 'org.example.app', 'version_code': 99}):
with patch.object(frame_catalog, 'fetch_apk', return_value='/tmp/example.apk'), \
patch.object(frame_android, 'apk_info', return_value=info), \
patch.object(frame_android, 'install') as install:
with self.assertRaises(frame_android.FrameError):
versions.install('org.example.app', 'https://f-droid.org/archive/example_1.apk')
install.assert_not_called()
self.network.assert_not_called()
class UploadVersionsTest(unittest.TestCase):
def test_endpoint_validation(self):
import server
for query in ('', 'package=', 'package=foo', 'package=a..b', 'package=a.1b',
'package=a.b/path', 'package=a.b&package=c.d', 'package=a.b&code=-1',
'package=a.b&code=x', 'package=a.b&code=', 'package=a.b&code=1&code=2'):
handler = object.__new__(server.Handler)
handler.path = '/api/apk-versions?' + query
with patch.object(handler, 'local_request', return_value=True), \
patch.object(handler, 'send_json') as reply, \
patch.object(versions, 'alternatives') as lookup:
handler.do_GET()
self.assertEqual(reply.call_args[0][1], 400, query)
lookup.assert_not_called()
handler.path = '/api/apk-versions?package=org.example_app.demo&code=123'
with patch.object(handler, 'local_request', return_value=True), \
patch.object(handler, 'send_json') as reply, \
patch.object(versions, 'alternatives', return_value={'total': 0}) as lookup:
handler.do_GET()
lookup.assert_called_once_with('org.example_app.demo', 123)
reply.assert_called_once_with({'total': 0})
def test_blocked_uploads_do_not_lookup_before_reply(self):
import server
info = {'package': 'org.example.app', 'label': 'Example', 'version': '5',
'version_code': 5, 'min_sdk': 33, 'abis': [], 'icon_png': None}
for mode in ('apkinfo', 'apk'):
handler = object.__new__(server.Handler)
handler.headers = {'X-Filename': 'app.apk', 'X-Mode': mode, 'Content-Length': '1'}
handler.rfile = io.BytesIO(b'x')
with patch.object(frame_android, 'apk_info', return_value=dict(info)), \
patch.object(versions, 'alternatives', side_effect=AssertionError('lookup during upload')) as lookup, \
patch.object(server, 'ensure_master') as ssh:
if mode == 'apkinfo':
reply = handler.upload()
self.assertNotIn('alternatives', reply['apk'])
self.assertIn('API 33', reply['apk']['blocker'])
else:
with self.assertRaises(server.Failure) as error:
handler.upload()
self.assertEqual(error.exception.status, 400)
self.assertEqual(error.exception.apk['package'], info['package'])
lookup.assert_not_called()
ssh.assert_not_called()
if __name__ == '__main__':
unittest.main()
+472
View File
@@ -0,0 +1,472 @@
"""Eye-camera pulse estimate and the heart-rate comparison tool; no headset needed."""
import importlib.util
import io
import json
import math
import os
from pathlib import Path
import random
import socket
import struct
import sys
import tempfile
import unittest
import zipfile
from unittest.mock import patch
ROOT = Path(__file__).resolve().parents[1]
def load(name, path):
spec = importlib.util.spec_from_file_location(name, ROOT / path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
pulse = load("pulse", "frame/tracking/pulse.py")
check = load("heart_check", "scripts/heart-check.py")
def synthetic(bpm, pulsing, seconds=40, fps=30, patches=48, noise=0.004, seed=7):
"""Patch brightness with a faint pulse in `pulsing` patches, plus sensor
noise, slow drift, blinks in some patches and eye movement in others."""
rng = random.Random(seed)
times = [i / fps for i in range(seconds * fps)]
values = {}
for k in range(patches):
base, amplitude, trace = 40 + k, 0.003 if k < pulsing else 0.0, []
for t in times:
v = base * (1 + amplitude * math.sin(2 * math.pi * bpm / 60 * t)
+ noise * rng.gauss(0, 1) + 0.02 * math.sin(0.1 * t + k))
if k % 8 == 7 and int(t * 10) % 47 == 0:
v *= 0.5 # blink
if k % 8 == 6 and int(t * 10) % 23 == 0:
v *= 1.3 # frequent eye movement
trace.append(v)
values[k] = trace
return times, values
class Estimate(unittest.TestCase):
def test_finds_pulse(self):
for bpm in (58, 72, 115):
with self.subTest(bpm=bpm):
result = pulse.estimate(*synthetic(bpm, 16))
self.assertAlmostEqual(result["bpm"], bpm, delta=1.5)
self.assertTrue(pulse.reliable(result))
self.assertTrue(all(abs(b - bpm) <= 2 for _, b in result["series"]))
def test_noise_and_blinks_are_not_a_pulse(self):
result = pulse.estimate(*synthetic(72, 0))
self.assertFalse(pulse.reliable(result))
def test_frequent_spike_patches_are_dropped(self):
times, values = synthetic(72, 16)
result = pulse.estimate(times, values)
self.assertLess(result["usable"], len(values))
def test_needs_enough_frames(self):
with self.assertRaises(ValueError):
pulse.estimate(*synthetic(72, 16, seconds=5))
times, values = synthetic(72, 16)
with self.assertRaises(ValueError):
pulse.estimate(times, {k: [0.0] * len(times) for k in values}) # all dark
def test_series_times(self):
times, values = synthetic(72, 16, seconds=20)
series = pulse.estimate(times, values)["series"]
self.assertAlmostEqual(series[0][0], pulse.WINDOW, delta=0.1)
self.assertEqual(len(series), 20 - int(pulse.WINDOW) + 1)
def test_flat_patches_do_not_rank_first(self):
times, values = synthetic(72, 16)
values["flat"] = [100.0] * len(times)
result = pulse.estimate(times, values)
self.assertAlmostEqual(result["bpm"], 72, delta=1.5)
self.assertEqual(pulse.snr([(60.0, 0.0), (61.0, 0.0)], 60), 0.0)
def test_analyse_uses_nearest_frame(self):
times, values = synthetic(72, 4, patches=4, seconds=10)
# Right-eye frames 1 ms before each left frame: nearest is that frame.
frames = [(t, "left", [values[k][i] for k in range(4)]) for i, t in enumerate(times)]
frames += [(t - 0.001, "right", [float(i)] * 4) for i, t in enumerate(times)]
seen = {}
original = pulse.estimate
with patch.object(pulse, "estimate", lambda t, p: seen.update(p) or original(t, p)):
pulse.analyse(frames)
self.assertEqual(seen["right0"][:5], [0.0, 0.0, 1.0, 1.0, 2.0])
def test_fft_matches_dft(self):
signal = [math.sin(i) + (i % 3) for i in range(16)]
fast = pulse.fft(signal)
for k in range(16):
slow = sum(signal[n] * complex(math.cos(2 * math.pi * k * n / 16), -math.sin(2 * math.pi * k * n / 16))
for n in range(16))
self.assertAlmostEqual(abs(fast[k] - slow), 0, places=9)
def test_grid_means(self):
# 4 × 4 image in RGB with padding: left half 10, right half 30 (channel 0).
width, height, channels, stride = 4, 4, 3, 16
pixels = bytearray(stride * height)
for y in range(height):
for x in range(width):
pixels[y * stride + x * channels] = 10 if x < 2 else 30
pixels[y * stride + x * channels + 1] = 255 # other channels ignored
self.assertEqual(pulse.grid_means(bytes(pixels), width, height, stride, channels, grid=2),
[10, 30, 10, 30])
def test_analyse_combines_both_eyes(self):
times, values = synthetic(80, 16, patches=16)
frames = []
for i, t in enumerate(times):
frames.append((t, "left", [values[k][i] for k in range(16)]))
frames.append((t + 0.001, "right", [values[k][i] for k in range(16)]))
result = pulse.analyse(frames)
self.assertAlmostEqual(result["bpm"], 80, delta=1.5)
self.assertEqual(result["usable"] % 2, 0)
class FakeCaptureTool:
"""Stands in for `eyetracking --calib`: writes PNG names over a few polls."""
def __init__(self, directory, frames=6, fail=False):
self.directory, self.frames, self.fail = Path(directory), frames, fail
self.polls = 0
self.returncode = None
self.stdout = None
self.announce = None # a different directory to report, if set
def __call__(self, command, cwd, stdout, stderr):
self.command = command
self.stdout = stdout
if self.fail:
stdout.write("Failed to initialize cameras\n")
stdout.flush()
self.returncode = 1
return self
# Real output is block-buffered until exit, so nothing is printed here.
stdout.flush()
self.directory.mkdir()
return self
def poll(self):
if self.returncode is not None:
return self.returncode
if self.polls < self.frames:
for eye in ("left", "right"):
(self.directory / f"{eye}_{self.polls}.png").write_bytes(b"png")
self.polls += 1
return None
meta = {"frames": [{eye: {"fname": str(self.directory / f"{eye}_{i}.png"), "frameNum": i + 10,
"tsMono": 100 + i / 90, "valid": i != 2} for eye in ("left", "right")}
for i in range(self.frames)]}
(self.directory / "meta.json").write_text(json.dumps(meta))
# The real tool's buffered output only appears once it exits.
self.stdout.write(f"Writing capture to: {self.announce or self.directory}\nCaptured images\n")
self.stdout.flush()
self.returncode = 0
return 0
def terminate(self):
raise AssertionError("the capture tool must never be signalled")
kill = terminate
def wait(self, timeout=None):
return self.returncode
class WornCheck(unittest.TestCase):
def sensor(self, name, raw):
root = Path(self.enterContext(tempfile.TemporaryDirectory()))
device = root / "iio:device2"
device.mkdir()
(device / "name").write_text(name + "\n")
(device / "in_proximity_raw").write_text(raw + "\n")
return root
def test_reads_the_proximity_sensor(self):
self.assertEqual(pulse.proximity(self.sensor("vcnl4000", "3.250000000")), 3.25)
def test_unreadable_sensor_does_not_block(self):
self.assertIsNone(pulse.proximity(self.sensor("other", "9")))
self.assertIsNone(pulse.proximity(self.sensor("vcnl4000", "junk")))
self.assertIsNone(pulse.proximity(Path("/nonexistent")))
self.assertTrue(pulse.worn(None))
def test_low_reading_means_unworn(self):
self.assertFalse(pulse.worn(3.1))
self.assertTrue(pulse.worn(pulse.WORN_MIN))
class CaptureLifecycle(unittest.TestCase):
def setUp(self):
self.root = Path(tempfile.mkdtemp())
self.directory = self.root / "etcalib_test"
(self.root / "etcalib_older").mkdir() # an earlier capture is not ours
self.loaded = []
def tearDown(self):
import shutil
shutil.rmtree(self.root, ignore_errors=True)
def loader(self, path):
self.loaded.append(Path(path).name)
self.assertTrue(Path(path).exists())
return [float(len(self.loaded))]
def capture(self, tool):
class TestCapture(pulse.Capture):
# A temporary directory stands in for /tmp/etcalib_*.
PREFIX = str(self.root / "etcalib_")
capture = TestCapture(20, runner=tool, loader=self.loader, workers=0)
with patch.object(pulse.time, "sleep", lambda s: None):
return capture, capture.run()
def test_reduces_deletes_and_joins_metadata(self):
tool = FakeCaptureTool(self.directory)
capture, frames = self.capture(tool)
self.assertEqual(sorted(self.loaded), sorted(f"{e}_{i}.png" for e in ("left", "right") for i in range(6)))
self.assertFalse(self.directory.exists()) # images and metadata removed
self.assertTrue((self.root / "etcalib_older").exists())
self.assertEqual(len(frames), 10) # frame 2 invalid in both eyes
self.assertEqual(tool.command[-2:], ["--calib", "20"])
self.assertTrue(all(isinstance(t, float) and eye in ("left", "right") for t, eye, _ in frames))
def test_camera_failure(self):
tool = FakeCaptureTool(self.directory, fail=True)
with self.assertRaises(RuntimeError) as raised:
self.capture(tool)
self.assertIn("cameras unavailable", str(raised.exception))
def test_backlog_abandons_capture_and_removes_images(self):
class Stalled(FakeCaptureTool):
def poll(self):
if self.polls > 8:
self.returncode = 0 # the bounded run ends by itself
return 0
for i in range(20):
for eye in ("left", "right"):
(self.directory / f"{eye}_{self.polls * 20 + i}.png").write_bytes(b"png")
self.polls += 1
return None
tool = Stalled(self.directory)
class TestCapture(pulse.Capture):
PREFIX = str(self.root / "etcalib_")
MAX_BACKLOG = 50
capture = TestCapture(20, runner=tool, loader=self.loader, workers=0)
capture.reduce = lambda final=False, original=capture.reduce: (
original(final) if tool.polls > 3 else None) # reduction stalls
with patch.object(pulse.time, "sleep", lambda s: None), self.assertRaises(RuntimeError) as raised:
capture.run()
self.assertIn("fell behind", str(raised.exception))
self.assertEqual(tool.returncode, 0) # left to end by itself, never signalled
self.assertFalse(self.directory.exists()) # no image left behind
def test_second_capture_directory_stops_and_removes_both(self):
foreign = self.root / "etcalib_foreign"
class Racing(FakeCaptureTool):
def poll(self):
if self.polls == 2:
foreign.mkdir()
(foreign / "left_0.png").write_bytes(b"png")
return super().poll()
tool = Racing(self.directory)
with self.assertRaises(RuntimeError) as raised:
self.capture(tool)
self.assertIn("another eye-camera capture", str(raised.exception))
self.assertFalse(self.directory.exists())
self.assertFalse(foreign.exists()) # no eye image outlives the run
self.assertTrue((self.root / "etcalib_older").exists())
def test_images_elsewhere_are_not_used_but_are_removed(self):
tool = FakeCaptureTool(self.directory)
tool.announce = self.root / "etcalib_reported"
tool.announce.mkdir()
(tool.announce / "left_0.png").write_bytes(b"png")
with self.assertRaises(RuntimeError) as raised:
self.capture(tool)
self.assertIn("somewhere else", str(raised.exception))
self.assertFalse(tool.announce.exists())
self.assertFalse(self.directory.exists())
def test_interrupt_during_cleanup_still_removes_images(self):
class Stubborn(FakeCaptureTool):
interrupts = 0
def poll(self):
if self.polls == 2 and self.interrupts < 2:
self.interrupts += 1
raise KeyboardInterrupt # Ctrl-C mid-capture, and again while waiting
return super().poll()
tool = Stubborn(self.directory)
with self.assertRaises(KeyboardInterrupt):
self.capture(tool)
self.assertEqual(tool.returncode, 0) # still let finish, never signalled
self.assertFalse(self.directory.exists())
def test_overrunning_tool_is_killed_as_a_last_resort(self):
class Hung(FakeCaptureTool):
killed = False
def poll(self):
(self.directory / "left_0.png").write_bytes(b"png")
return None if not self.killed else -9
def kill(self):
self.killed = True
tool = Hung(self.directory)
clock = iter(range(0, 10000, 20))
with patch.object(pulse.time, "monotonic", lambda: next(clock)), \
self.assertRaises(RuntimeError) as raised:
self.capture(tool)
self.assertIn("may need a reboot", str(raised.exception))
self.assertTrue(tool.killed)
self.assertFalse(self.directory.exists())
def test_cleanup_tries_every_directory(self):
capture = pulse.Capture(20)
capture.PREFIX = str(self.root / "etcalib_")
first, second = self.root / "etcalib_a", self.root / "etcalib_b"
for directory in (first, second):
directory.mkdir()
(directory / "left_0.png").write_bytes(b"png")
capture.new = {first, second}
real = pulse.shutil.rmtree
def flaky(path):
if Path(path) == first:
raise OSError("busy")
real(path)
with patch.object(pulse.shutil, "rmtree", flaky), self.assertRaises(RuntimeError) as raised:
capture.remove()
self.assertIn("etcalib_a", str(raised.exception))
self.assertFalse(second.exists())
def test_removes_a_directory_seen_only_at_the_end(self):
capture = pulse.Capture(20)
capture.PREFIX = str(self.root / "etcalib_")
capture.before = capture.candidates()
late = self.root / "etcalib_late"
late.mkdir()
(late / "left_0.png").write_bytes(b"png")
capture.remove()
self.assertFalse(late.exists())
self.assertTrue((self.root / "etcalib_older").exists())
def test_never_adopts_directories_without_a_baseline(self):
capture = pulse.Capture(20)
capture.PREFIX = str(self.root / "etcalib_")
capture.remove()
self.assertTrue((self.root / "etcalib_older").exists())
def test_only_removes_capture_directories(self):
capture = pulse.Capture(20)
capture.directory = self.root
capture.remove()
self.assertTrue(self.root.exists())
class HeartCheck(unittest.TestCase):
def write(self, name, text):
path = Path(self.enterContext(tempfile.TemporaryDirectory())) / name
path.write_text(text)
return path
def test_osc_round_trip(self):
tracking = load("tracking", "frame/tracking/tracking.py")
self.assertEqual(check.parse_osc(tracking.osc_message("/avatar/parameters/HeartRate", [72])),
("/avatar/parameters/HeartRate", [72]))
address, values = check.parse_osc(tracking.osc_message("/x", [1.5, -2.0]))
self.assertEqual((address, values), ("/x", [1.5, -2.0]))
with self.assertRaises(ValueError):
check.parse_osc(b"/x\0\0,s\0\0abc\0")
def test_listen_records_readings(self):
tracking = load("tracking", "frame/tracking/tracking.py")
with tempfile.TemporaryDirectory() as directory:
out = Path(directory) / "ours.csv"
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as probe:
probe.bind(("127.0.0.1", 0))
port = probe.getsockname()[1]
import threading
def send():
import time
time.sleep(0.3)
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sender:
sender.sendto(tracking.osc_message(check.ADDRESS, [72]), ("127.0.0.1", port))
sender.sendto(tracking.osc_message("/other", [1]), ("127.0.0.1", port))
threading.Thread(target=send).start()
shown = io.StringIO()
count = check.listen(port, check.ADDRESS, str(out), 1.5, stream=shown)
self.assertEqual(count, 1)
self.assertIn("72 bpm", shown.getvalue())
self.assertEqual(out.read_text().splitlines()[1].split(",")[1], "72")
self.assertEqual(out.stat().st_mode & 0o777, 0o600)
def test_compare_finds_lag_and_contact_loss(self):
reference = [(1000.0 + i, 60 + i, 6) for i in range(40)] + [(1040.0 + i, 150, 4) for i in range(5)]
ours = [(1002.0 + i, 60 + i) for i in range(40)] # 2 s late, nothing during contact loss
ref = self.write("ref.csv", "unix_seconds,bpm,flags\n" + "".join(f"{t},{b},{f}\n" for t, b, f in reference))
mine = self.write("ours.csv", "unix_seconds,bpm\n" + "".join(f"{t},{b}\n" for t, b in ours))
result = check.compare(check.read_csv(mine), check.read_any(ref))
self.assertEqual(result["lag_seconds"], 2.0)
self.assertEqual(result["mean_abs_error"], 0)
self.assertEqual(result["shown_during_no_contact"], 0)
self.assertTrue(check.report(result, 5, stream=io.StringIO()))
# A stale reading sent during contact loss fails the check.
stale = check.read_csv(mine) + [(1043.0, 99)]
result = check.compare(stale, check.read_any(ref))
self.assertEqual(result["shown_during_no_contact"], 1)
self.assertFalse(check.report(result, 5, stream=io.StringIO()))
def test_compare_against_apple_health_export(self):
records = "".join(
f'<Record type="HKQuantityTypeIdentifierHeartRate" unit="count/min" '
f'startDate="2026-09-29 12:00:{s:02d} +1000" endDate="2026-09-29 12:00:{s:02d} +1000" value="{70 + s % 3}"/>'
for s in range(0, 60, 5))
other = '<Record type="HKQuantityTypeIdentifierStepCount" startDate="2026-09-29 12:00:00 +1000" value="9"/>'
xml = f'<?xml version="1.0"?><HealthData>{other}{records}</HealthData>'
with tempfile.TemporaryDirectory() as directory:
archive = Path(directory) / "export.zip"
with zipfile.ZipFile(archive, "w") as z:
z.writestr("apple_health_export/export.xml", xml)
start = check.parse_time("2026-09-29 12:00:00 +1000")
samples = check.read_any(archive, start - 60, start + 120)
self.assertEqual(len(samples), 12)
self.assertEqual(samples[0], (start, 70))
ours = [(start + i + 1.0, 71) for i in range(60)]
result = check.compare(ours, samples)
self.assertLessEqual(result["mean_abs_error"], 1)
def test_unusual_flags_and_missing_export(self):
path = self.write("ref.csv", "time,bpm,flags\n1000,70,6.0\n1001,71,yes\n1002,72,4\n")
self.assertEqual(check.read_csv(path), [(1000.0, 70), (1001.0, 71), (1002.0, None)])
with tempfile.TemporaryDirectory() as directory:
archive = Path(directory) / "other.zip"
with zipfile.ZipFile(archive, "w") as z:
z.writestr("notes.txt", "x")
with self.assertRaises(ValueError):
check.read_any(archive, 0, 1)
def test_compare_reports_bad_input_without_traceback(self):
good = self.write("ref.csv", "1000,70\n")
shown = io.StringIO()
with patch("sys.stdout", shown):
self.assertEqual(check.main(["compare", str(good.parent / "missing.csv"), str(good)]), 1)
self.assertEqual(check.main(["compare", str(self.write("ours.csv", "1000,inf\n1001,70\n")),
str(self.write("bad.xml", "<not closed"))]), 1)
self.assertEqual(shown.getvalue().count("Could not compare"), 2)
def test_health_records_with_non_finite_values_are_skipped(self):
record = '<Record type="HKQuantityTypeIdentifierHeartRate" startDate="2026-09-29 12:00:00 +1000" value="%s"/>'
path = self.write("export.xml", "<HealthData>" + record % "inf" + record % "72" + "</HealthData>")
start = check.parse_time("2026-09-29 12:00:00 +1000")
self.assertEqual(check.read_health(path, start - 1, start + 1), [(start, 72)])
def test_time_formats(self):
self.assertEqual(check.parse_time("1700000000.5"), 1700000000.5)
self.assertEqual(check.parse_time("2023-11-14T22:13:20Z"), 1700000000)
self.assertEqual(check.parse_time("2023-11-15 08:13:20 +1000"), 1700000000)
with self.assertRaises(ValueError):
check.parse_time("yesterday")
if __name__ == "__main__":
unittest.main()
+17
View File
@@ -212,6 +212,23 @@ class ServerGuards(unittest.TestCase):
self.assertNotEqual(status, 200, body) self.assertNotEqual(status, 200, body)
self.assertNotIn("job", body) self.assertNotIn("job", body)
def test_android_install_of_another_version_runs_as_a_job(self):
pkg = "org.example.frame_control.not_in_any_repo"
sys.path.insert(0, str(ROOT / "ui"))
import frame_apk_versions
# The job must fail on the cached lookup, before any download: nothing is cached for this package.
self.assertEqual(frame_apk_versions._versions(pkg, cached_only=True)[0], [])
status, started = self.post("/api/android", {"action": "install", "package": pkg,
"url": f"https://f-droid.org/repo/{pkg}_1.apk"})
self.assertEqual(status, 200, started)
for _ in range(200):
job = json.loads(self.request("GET", f"/api/job?id={started['job']}", headers={"X-Frame-UI": "1"})[2])
if job["done"]:
break
time.sleep(0.05)
self.assertTrue(job["done"])
self.assertIn("no longer available", job["error"])
def test_unknown_routes(self): def test_unknown_routes(self):
self.assertEqual(self.request("GET", "/nope")[0], 404) self.assertEqual(self.request("GET", "/nope")[0], 404)
self.assertEqual(self.post("/api/nope", {})[0], 404) self.assertEqual(self.post("/api/nope", {})[0], 404)
+188
View File
@@ -0,0 +1,188 @@
"""Tracking protocols and fake-Frame BlueZ lifecycle; no headset or strap needed."""
import importlib.util
import math
import os
from pathlib import Path
import socket
import stat
import struct
import tempfile
import unittest
from unittest.mock import Mock, patch
SPEC = importlib.util.spec_from_file_location("tracking", Path(__file__).resolve().parents[1] / "frame/tracking/tracking.py")
t = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(t)
class Protocols(unittest.TestCase):
def test_hrs_formats(self):
self.assertEqual(t.heart_rate(b"\x00\x48"), 72)
self.assertEqual(t.heart_rate(b"\x01\x2c\x01"), 300)
self.assertEqual(t.heart_rate(b"\x1e\x48\x01\x00\x00\x04\x00\x04"), 72)
self.assertEqual(t.heart_rate(b"\x02\x48"), 72) # contact not supported
self.assertIsNone(t.heart_rate(b"\x04\x48")) # no contact
self.assertIsNone(t.heart_rate(b"\x00\x00"))
def test_hrs_malformed(self):
for packet in (b"", b"\x00", b"\x01\x48", b"\x08\x48\x00", b"\x10\x48",
b"\x10\x48\x00", b"\x00\x48\x01", b"\xe0\x48"):
with self.subTest(packet=packet), self.assertRaises(ValueError):
t.heart_rate(packet)
def test_gaze_coordinates(self):
self.assertEqual(t.gaze_angles([0, 0, 0, 1]), (0, 0))
angle = math.radians(15)
pitch, yaw = t.gaze_angles([math.sin(angle), 0, 0, math.cos(angle)])
self.assertAlmostEqual(pitch, -30) # OpenXR +X rotation looks up
self.assertAlmostEqual(yaw, 0)
pitch, yaw = t.gaze_angles([0, -math.sin(angle), 0, math.cos(angle)])
self.assertAlmostEqual(pitch, 0)
self.assertAlmostEqual(yaw, 30) # right
def test_invalid_gaze(self):
for pose in ([0, 0, 0, 0], [math.nan, 0, 0, 1], [0, 0, 0], [0, 0, 0, math.inf]):
with self.assertRaises(ValueError):
t.gaze_angles(pose)
def test_osc_wire(self):
self.assertEqual(t.osc_message('/x', [72]), b'/x\0\0,i\0\0' + struct.pack('>i', 72))
self.assertEqual(t.osc_message('/x', [1.0, -2.0]), b'/x\0\0,ff\0' + struct.pack('>ff', 1, -2))
for address in ('x', '/x\0y', '/x y', '/x*'):
with self.assertRaises(ValueError):
t.osc_message(address, [1])
def test_default_never_opens_socket(self):
with patch.object(t.socket, 'socket') as create:
osc = t.Osc()
osc.send('/x', [72])
osc.close()
create.assert_not_called()
def test_only_configured_endpoint(self):
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as receiver:
receiver.bind(('127.0.0.1', 0))
receiver.settimeout(1)
osc = t.Osc(receiver.getsockname())
try:
osc.send('/tracking/eye/CenterPitchYaw', [0.0, 30.0])
self.assertEqual(receiver.recv(1024), t.osc_message('/tracking/eye/CenterPitchYaw', [0.0, 30.0]))
finally:
osc.close()
def test_endpoint_validation(self):
for endpoint in [('example.org', 9000), ('0.0.0.0', 9000), ('224.0.0.1', 9000), ('127.0.0.1', 0), ('::1', 65536)]:
with self.assertRaises(ValueError):
t.Osc(endpoint)
def test_heart_staleness_contact_and_log(self):
now = [0]
osc = Mock()
with tempfile.TemporaryDirectory() as directory:
path = Path(directory) / 'session.csv'
session = t.HeartSession(osc, '/hr', path, lambda: now[0])
self.assertIsNone(session.current())
session.notification(b'\x00\x48')
self.assertEqual(session.current(), 72)
osc.send.assert_called_once_with('/hr', [72])
now[0] = 6
self.assertIsNone(session.current())
session.notification(b'\x04\x48')
self.assertIsNone(session.current())
self.assertEqual(osc.send.call_count, 1)
session.close()
self.assertEqual(path.read_text().splitlines()[0], 'unix_seconds,bpm')
self.assertEqual(len(path.read_text().splitlines()), 2)
if os.name != 'nt':
self.assertEqual(stat.S_IMODE(path.stat().st_mode), 0o600)
with self.assertRaises(FileExistsError):
t.HeartSession(osc, '/hr', path)
def test_no_log_by_default(self):
with patch.object(t.os, 'open') as create:
session = t.HeartSession(Mock(), '/hr')
session.notification(b'\x00\x48')
session.close()
create.assert_not_called()
class FakeBluez(unittest.TestCase):
def setUp(self):
self.device = '/org/bluez/hci0/dev_TEST'
self.service = self.device + '/service1'
self.char = self.service + '/char1'
self.objects = {
self.device: {t.DEVICE: {'Address': 'AA:BB:CC:DD:EE:FF', 'Connected': False, 'ServicesResolved': True}},
self.service: {t.SERVICE: {'UUID': t.HRS, 'Device': self.device}},
self.char: {t.CHARACTERISTIC: {'UUID': t.MEASUREMENT, 'Service': self.service, 'Flags': ['notify']}},
}
self.api = Mock()
self.api.GetManagedObjects.side_effect = lambda: self.objects
self.bus = Mock()
self.interface = Mock(return_value=self.api)
self.values = Mock()
def reader(self):
return t.BluezHeart(self.bus, self.interface, 'AA:BB:CC:DD:EE:FF', self.values)
def test_subscribe_receive_and_cleanup(self):
reader = self.reader()
self.api.Connect.assert_called_once()
self.assertTrue(reader.subscribe())
self.api.StartNotify.assert_called_once()
reader.changed(t.CHARACTERISTIC, {'Value': [0, 72]}, [], self.char)
self.values.assert_called_once_with([0, 72])
reader.changed(t.CHARACTERISTIC, {'Value': [0, 73]}, [], '/other/strap')
self.assertEqual(self.values.call_count, 1)
reader.close()
self.api.StopNotify.assert_called_once()
self.api.Disconnect.assert_called_once()
self.bus.add_signal_receiver.return_value.remove.assert_called_once()
def test_preserve_existing_connection(self):
self.objects[self.device][t.DEVICE]['Connected'] = True
reader = self.reader()
reader.subscribe()
reader.close()
self.api.Connect.assert_not_called()
self.api.Disconnect.assert_not_called()
def test_only_selected_device_service(self):
self.objects[self.service][t.SERVICE]['Device'] = '/other/device'
reader = self.reader()
try:
with self.assertRaises(RuntimeError):
reader.subscribe()
finally:
reader.close()
self.api.StartNotify.assert_not_called()
def test_wait_for_services(self):
self.objects[self.device][t.DEVICE]['ServicesResolved'] = False
reader = self.reader()
self.assertFalse(reader.subscribe())
reader.close()
self.api.StartNotify.assert_not_called()
self.api.StopNotify.assert_not_called()
def test_disconnect_notification(self):
reader = self.reader()
reader.changed(t.DEVICE, {'Connected': 0}, [], self.device) # dbus.Boolean behaves as int
self.values.assert_called_once_with(None)
reader.close()
def test_failed_notify_cleans_connection(self):
reader = self.reader()
self.api.StartNotify.side_effect = RuntimeError('failure')
with self.assertRaises(RuntimeError):
reader.subscribe()
reader.close()
self.api.StopNotify.assert_not_called()
self.api.Disconnect.assert_called_once()
def test_unknown_device_does_not_connect_or_scan(self):
self.objects.clear()
with self.assertRaises(RuntimeError):
self.reader()
self.api.Connect.assert_not_called()
self.api.StartDiscovery.assert_not_called()
+10 -2
View File
@@ -6,7 +6,7 @@ apps, the lepton-show-flatscreen marker; plus a non-Steam shortcut, so it shows
in the Steam library and gets its own SteamVR panel. Nothing goes through in the Steam library and gets its own SteamVR panel. Nothing goes through
Lepton Development, which wipes its apps on exit. See docs/apks.md. Lepton Development, which wipes its apps on exit. See docs/apks.md.
Python stdlib only. CLI: python3 ui/frame_android.py {install APK|list|launch PKG|stop PKG|remove PKG|probe PKG} Python stdlib only. CLI: python3 ui/frame_android.py {info APK|versions APK-or-PKG|install APK|list|launch PKG|stop PKG|remove PKG|probe PKG}
""" """
import json, os, re, shlex, shutil, subprocess, sys, threading, time, zlib import json, os, re, shlex, shutil, subprocess, sys, threading, time, zlib
@@ -276,7 +276,15 @@ def probe(pkg, wait=20):
def main(): def main():
cmd, *args = sys.argv[1:] or ['help'] cmd, *args = sys.argv[1:] or ['help']
try: try:
if cmd == 'install': if cmd in ('info', 'versions'):
import frame_apk_versions
if cmd == 'info':
print(frame_apk_versions.describe(apk_info(args[0])))
return
info = apk_info(args[0]) if os.path.isfile(args[0]) or args[0].lower().endswith('.apk') else None
r = frame_apk_versions.alternatives(
info['package'] if info else args[0], info.get('version_code') if info else None)
elif cmd == 'install':
r = install(args[0], flatscreen='--vr' not in args) r = install(args[0], flatscreen='--vr' not in args)
elif cmd == 'list': elif cmd == 'list':
r = list_apps() r = list_apps()
+1
View File
@@ -229,6 +229,7 @@ def apk_info(path):
min_sdk = sdk.get('minSdkVersion') min_sdk = sdk.get('minSdkVersion')
info = { info = {
'package': package, 'package': package,
'version_code': manifest.get('versionCode', (None, None))[1],
'version': _text(manifest.get('versionName'), res) or '', 'version': _text(manifest.get('versionName'), res) or '',
'label': _text(app.get('label'), res) or package, 'label': _text(app.get('label'), res) or package,
'abis': sorted({n.split('/')[1] for n in names if n.startswith('lib/') and n.count('/') >= 2}), 'abis': sorted({n.split('/')[1] for n in names if n.startswith('lib/') and n.count('/') >= 2}),
+90
View File
@@ -0,0 +1,90 @@
"""Explain APK requirements and find installable versions in F-Droid's indexes."""
from urllib.parse import quote, urlencode
import frame_android
import frame_catalog
ANDROID = dict(enumerate([
'1.0', '1.1', '1.5', '1.6', '2.0', '2.0.1', '2.1', '2.2', '2.3', '2.3.3',
'3.0', '3.1', '3.2', '4.0', '4.0.3', '4.1', '4.2', '4.3', '4.4', '4.4W',
'5.0', '5.1', '6.0', '7.0', '7.1', '8.0', '8.1', '9', '10', '11', '12',
'12L', '13', '14', '15', '16',
], 1))
REPOS = (('F-Droid', 'https://f-droid.org/repo/'),
('F-Droid archive', 'https://f-droid.org/archive/'),
('IzzyOnDroid', 'https://apt.izzysoft.de/fdroid/repo/'))
NOTE = ('Pick a version whose minimum is Android 11 or lower and that has an '
'arm64-v8a build (or no native code). Installable does not mean every feature works.')
def android_name(sdk):
return 'Android ' + ANDROID[sdk] if sdk in ANDROID else f'Android API {sdk}'
def describe(info):
sdk = info.get('min_sdk')
minimum = f'{android_name(sdk)} (API {sdk})' if sdk else 'not specified'
try:
frame_android.check_installable(info)
verdict = 'Lepton can install this APK. Features may still need services Lepton lacks.'
except frame_android.FrameError as e:
verdict = f'Lepton cannot install this APK: {e}'
return (f"{info['package']} · {info.get('version') or '?'} "
f"(code {info.get('version_code') if info.get('version_code') is not None else '?'})\n"
f"Minimum: {minimum}\nABIs: {', '.join(info['abis']) or 'no native code'}\n{verdict}")
def search_links(package):
q = quote(package, safe='')
return [{'source': name, 'url': url} for name, url in (
('APKMirror', 'https://www.apkmirror.com/?' + urlencode({'post_type': 'app_release', 's': package})),
('APKPure', 'https://apkpure.com/search?q=' + q),
('Uptodown', 'https://en.uptodown.com/android/search/' + q),
('F-Droid', 'https://search.f-droid.org/?q=' + q),
('GitHub', 'https://github.com/search?type=repositories&q=' + q),
)]
def _versions(package, cached_only=False):
versions, errors, seen = [], [], set()
for source, repo in REPOS:
try:
index = frame_catalog.load_index(repo, cached_only=cached_only)
except Exception as e: # one bad repo (dropped download, odd index) mustn't hide the others
errors.append(f'Could not check {source}: {e}')
continue
for v in index.get(package, []):
url = repo + v['name'].lstrip('/')
key = (v['version_code'], v.get('sha256') or url)
if key in seen:
continue
seen.add(key)
versions.append(dict(v, url=url, source=source))
return versions, errors
def alternatives(package, current_version_code=None):
"""At most eight releases, preferring arm64-only builds over universal builds."""
versions, errors = _versions(package)
versions = [v for v in versions if v['version_code'] != current_version_code]
total = len(versions)
versions.sort(key=lambda v: (v['abis'] == ['arm64-v8a'], v['version_code']), reverse=True)
releases = {}
for v in versions:
releases.setdefault(v['version'], v)
versions = sorted(releases.values(), key=lambda v: v['version_code'], reverse=True)[:8]
return {'package': package, 'versions': versions, 'total': total,
'links': search_links(package), 'note': NOTE, 'errors': errors}
def install(package, url):
# Resolve the selection again: the client cannot supply a trusted hash or arbitrary URL.
records, _ = _versions(package, cached_only=True)
version = next((v for v in records if v['url'] == url), None)
if not version:
raise frame_android.FrameError('That version is no longer available; check the APK again')
apk = frame_catalog.fetch_apk({'a': version['url'], 'h': version['sha256'], 'n': package})
info = frame_android.apk_info(apk)
if info['package'] != package or info.get('version_code') != version['version_code']:
raise frame_android.FrameError('The downloaded APK does not match the selected version')
return frame_android.install(apk, source=version['source'])
+139 -2
View File
@@ -2,7 +2,7 @@
list, verified downloads, installs into per-app Lepton instances, and list, verified downloads, installs into per-app Lepton instances, and
compatibility reports. Python stdlib only. compatibility reports. Python stdlib only.
""" """
import hashlib, os, shutil, sys, tempfile, threading, time, urllib.error, urllib.request import hashlib, json, os, shutil, sys, tempfile, threading, time, urllib.error, urllib.request
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
CATALOG = os.path.join(ROOT, 'apk-catalog') CATALOG = os.path.join(ROOT, 'apk-catalog')
@@ -17,12 +17,149 @@ import frame_compat_db as compat_db # noqa: E402
# the per-user cache (FRAME_CONTROL_APP is set by app/main.js). # the per-user cache (FRAME_CONTROL_APP is set by app/main.js).
CACHE = (str(frame_host.cache_dir('apk')) if os.environ.get('FRAME_CONTROL_APP') or '.app/Contents/Resources' in CATALOG CACHE = (str(frame_host.cache_dir('apk')) if os.environ.get('FRAME_CONTROL_APP') or '.app/Contents/Resources' in CATALOG
else os.path.join(CATALOG, 'data', 'cache')) else os.path.join(CATALOG, 'data', 'cache'))
APK_HOSTS = ('https://f-droid.org/repo/', 'https://f-droid.org/archive/') # Repo base URL -> local name of its index; every APK download must come from one of these.
INDEX_FILES = {'https://f-droid.org/repo/': 'index-v2.json',
'https://f-droid.org/archive/': 'index-v2.archive.json',
'https://apt.izzysoft.de/fdroid/repo/': 'index-v2.izzy.json'}
APK_HOSTS = tuple(INDEX_FILES)
_lock = threading.Lock() _lock = threading.Lock()
_cache = {'mtime': None, 'sig': None, 'apps': None, 'by_pkg': None} _cache = {'mtime': None, 'sig': None, 'apps': None, 'by_pkg': None}
_env = {} _env = {}
_index_lock = threading.Lock()
_indexes = {}
class _IndexReader:
"""Decode one object member at a time; never retain the whole raw index."""
def __init__(self, stream):
self.stream, self.buffer = stream, ''
self.decoder = json.JSONDecoder()
def fill(self):
chunk = self.stream.read(1 << 16)
if not chunk:
raise ValueError('incomplete F-Droid index')
self.buffer += chunk
def peek(self):
self.buffer = self.buffer.lstrip()
while not self.buffer:
self.fill()
self.buffer = self.buffer.lstrip()
return self.buffer[0]
def expect(self, char):
if self.peek() != char:
raise ValueError('invalid F-Droid index')
self.buffer = self.buffer[1:]
def value(self):
self.peek()
while True:
try:
value, end = self.decoder.raw_decode(self.buffer)
self.buffer = self.buffer[end:]
return value
except json.JSONDecodeError:
self.fill()
def members(self):
self.expect('{')
if self.peek() != '}':
while True:
key = self.value()
if not isinstance(key, str):
raise ValueError('invalid F-Droid index key')
self.expect(':')
yield key
if self.peek() == '}':
break
self.expect(',')
self.expect('}')
def _reduce_index(path):
from pick import installable
packages = {}
found = False
with open(path, encoding='utf-8') as f:
reader = _IndexReader(f)
for key in reader.members():
if key != 'packages':
reader.value()
continue
found = True
for package in reader.members():
records, entry = [], reader.value()
versions = entry.get('versions') if isinstance(entry, dict) else None
for v in (versions.values() if isinstance(versions, dict) else ()):
# Skip malformed entries rather than losing the whole repo.
if not (isinstance(v, dict) and isinstance(v.get('manifest'), dict)
and isinstance(v.get('file'), dict) and v['file'].get('name')):
continue
if not installable(v):
continue
m, file = v['manifest'], v['file']
records.append({'version': m.get('versionName', ''),
'version_code': m.get('versionCode', 0),
'min_sdk': m.get('usesSdk', {}).get('minSdkVersion', 1),
'abis': m.get('nativecode') or [],
'name': file['name'], 'sha256': file.get('sha256')})
if records:
packages[package] = records
if reader.buffer.strip() or f.read().strip():
raise ValueError('trailing data in F-Droid index')
if not found:
raise ValueError('invalid F-Droid index')
return packages
def load_index(repo, cached_only=False):
"""Compact installable records by package, cached on disk and by mtime in memory."""
if repo not in APK_HOSTS:
raise ValueError('unexpected index URL')
directory = CACHE if os.environ.get('FRAME_CONTROL_APP') or '.app/Contents/Resources' in CATALOG else os.path.join(CATALOG, 'data')
filename = INDEX_FILES[repo]
raw = os.path.join(directory, filename)
path = raw + '.installable-v1'
with _index_lock:
mtime = os.stat(path).st_mtime_ns if os.path.exists(path) else None
# A newer raw index (the catalogue script refreshed it) outdates the reduced copy.
newer_raw = mtime is not None and os.path.exists(raw) and os.stat(raw).st_mtime_ns > mtime
if mtime is not None and (cached_only or (time.time() - mtime / 1e9 < 86400 and not newer_raw)):
cached = _indexes.get(path)
if cached is None or cached[0] != mtime:
with open(path) as f:
cached = (mtime, json.load(f))
_indexes[path] = cached
return cached[1]
if cached_only:
return {}
os.makedirs(directory, exist_ok=True)
fd, tmp = tempfile.mkstemp(prefix=filename, suffix='.part', dir=directory)
os.close(fd)
try:
if os.path.exists(raw) and time.time() - os.path.getmtime(raw) < 86400:
index = _reduce_index(raw)
refreshed = os.stat(raw).st_mtime_ns
else:
with open(tmp, 'wb') as f, urllib.request.urlopen(repo + 'index-v2.json', timeout=30) as r:
shutil.copyfileobj(r, f, 1 << 20)
index = _reduce_index(tmp)
refreshed = time.time_ns()
with open(tmp, 'w') as f:
json.dump(index, f, separators=(',', ':'))
os.utime(tmp, ns=(refreshed, refreshed))
os.replace(tmp, path)
_indexes[path] = (os.stat(path).st_mtime_ns, index)
return index
finally:
if os.path.exists(tmp):
os.remove(tmp)
def catalog(): def catalog():
"""Rated apps with the database's reports applied.""" """Rated apps with the database's reports applied."""
path = os.path.join(CATALOG, 'site', 'apps.js') path = os.path.join(CATALOG, 'site', 'apps.js')
+58 -3
View File
@@ -254,10 +254,10 @@
.and-grid { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 2fr); gap: 22px; align-items: start; } .and-grid { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 2fr); gap: 22px; align-items: start; }
.and-col { display: grid; gap: 22px; align-content: start; } .and-col { display: grid; gap: 22px; align-content: start; }
.rep-item .s { white-space: normal; } .rep-item .s { white-space: normal; }
#repDlg, #titleDlg, #wiDlg, #pwDlg { background: #1e2329; color: var(--text); border: 1px solid rgba(255,255,255,.1); border-radius: 4px; #repDlg, #titleDlg, #wiDlg, #pwDlg, #apkAltDlg { background: #1e2329; color: var(--text); border: 1px solid rgba(255,255,255,.1); border-radius: 4px;
padding: 22px; width: min(560px, 92vw); box-shadow: 0 20px 60px rgba(0,0,0,.6); } padding: 22px; width: min(560px, 92vw); box-shadow: 0 20px 60px rgba(0,0,0,.6); }
#repDlg::backdrop, #titleDlg::backdrop, #wiDlg::backdrop, #pwDlg::backdrop { background: rgba(0,0,0,.55); } #repDlg::backdrop, #titleDlg::backdrop, #wiDlg::backdrop, #pwDlg::backdrop, #apkAltDlg::backdrop { background: rgba(0,0,0,.55); }
#repDlg h2, #titleDlg h2, #wiDlg h2, #pwDlg h2 { margin: 0 0 14px; font-size: 15px; letter-spacing: 1.5px; text-transform: uppercase; color: var(--bright); } #repDlg h2, #titleDlg h2, #wiDlg h2, #pwDlg h2, #apkAltDlg h2 { margin: 0 0 14px; font-size: 15px; letter-spacing: 1.5px; text-transform: uppercase; color: var(--bright); }
#repForm label, #titleForm label { display: block; font-size: 12.5px; color: var(--muted); margin-top: 10px; } #repForm label, #titleForm label { display: block; font-size: 12.5px; color: var(--muted); margin-top: 10px; }
#repForm label input[type=text], #repForm textarea, #titleForm label input, #titleForm label select { margin-top: 5px; } #repForm label input[type=text], #repForm textarea, #titleForm label input, #titleForm label select { margin-top: 5px; }
#titleForm select { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; #titleForm select { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent;
@@ -656,6 +656,16 @@
<span class="last" id="lastLog">Ready</span> <span class="last" id="lastLog">Ready</span>
<span class="sub" id="drawerHint">Show ▴</span> <span class="sub" id="drawerHint">Show ▴</span>
</div> </div>
<dialog id="apkAltDlg" aria-labelledby="apkAltTitle">
<h2 id="apkAltTitle">Try another APK version</h2>
<p id="apkAltReason"></p>
<div class="list" id="apkAltVersions"></div>
<p class="sub" id="apkAltNote"></p>
<div id="apkAltLinks"></div>
<p class="sub" id="apkAltErrors"></p>
<div class="actions"><button id="apkAltClose">Close</button></div>
</dialog>
<dialog id="repDlg" aria-labelledby="repTitle"> <dialog id="repDlg" aria-labelledby="repTitle">
<form method="dialog" id="repForm"> <form method="dialog" id="repForm">
<h2 id="repTitle">Report an APK</h2> <h2 id="repTitle">Report an APK</h2>
@@ -1410,12 +1420,56 @@ function upload(file, mode) {
xhr.onload = () => { xhr.onload = () => {
$("prog").style.display = "none"; $("prog").style.display = "none";
let data; try { data = JSON.parse(xhr.responseText); } catch { data = { error: `HTTP ${xhr.status}` }; } let data; try { data = JSON.parse(xhr.responseText); } catch { data = { error: `HTTP ${xhr.status}` }; }
if (data.apk?.blocker && xhr.status >= 300) checkApkAlternatives(data.apk);
xhr.status < 300 ? resolve(data) : reject(new Error(data.error)); xhr.status < 300 ? resolve(data) : reject(new Error(data.error));
}; };
xhr.onerror = () => { $("prog").style.display = "none"; reject(new Error("network error")); }; xhr.onerror = () => { $("prog").style.display = "none"; reject(new Error("network error")); };
xhr.send(file); xhr.send(file);
}); });
} }
let apkLookup = 0;
async function checkApkAlternatives(apk) {
const lookup = ++apkLookup;
$("apkAltReason").textContent = apk.blocker;
$("apkAltVersions").textContent = "Checking F-Droid for older versions…";
$("apkAltVersions").onclick = null;
for (const id of ["apkAltNote", "apkAltErrors", "apkAltLinks"]) $(id).textContent = "";
if (!$("apkAltDlg").open) $("apkAltDlg").showModal();
const query = new URLSearchParams({package: apk.package});
if (apk.version_code != null) query.set("code", apk.version_code);
try {
const result = await api(`/api/apk-versions?${query}`);
if (lookup === apkLookup && $("apkAltDlg").open) showApkAlternatives(apk.blocker, result);
} catch (e) {
if (lookup === apkLookup) $("apkAltVersions").textContent = e.message;
}
}
function showApkAlternatives(reason, result) {
$("apkAltReason").textContent = reason;
$("apkAltNote").textContent = `${result.versions.length} of ${result.total} compatible versions. ${result.note}`;
$("apkAltErrors").textContent = result.errors.join(" · ");
$("apkAltLinks").innerHTML = result.links.map(l => `<a href="${esc(l.url)}" target="_blank" rel="noopener noreferrer">${esc(l.source)}</a>`).join(" · ");
$("apkAltVersions").innerHTML = result.versions.length ? result.versions.map((v, i) =>
`<div class="item"><div class="grow"><div class="t">${esc(v.version || "?")} <span class="sub">code ${esc(v.version_code)}</span></div>
<div class="s">${esc(v.source)} · minimum API ${esc(v.min_sdk)} · ${esc(v.abis.join(", ") || "no native code")}</div></div>
<button class="small" data-version="${i}" ${v.sha256 ? "" : "disabled"}>Install</button></div>`).join("") :
`<p>No compatible version found in F-Droid. Try the searches below.</p>`;
$("apkAltVersions").onclick = async e => {
const b = e.target.closest("[data-version]"); if (!b) return;
const v = result.versions[+b.dataset.version];
if (installing.has(result.package)) return;
b.disabled = true; b.textContent = "Installing…";
const res = await runJob(`Install ${result.package} ${v.version}`, result.package, () => api("/api/android", {
action: "install", package: result.package, url: v.url
}));
if (res) $("apkAltDlg").close(); else { b.disabled = false; b.textContent = "Install"; }
await loadAndroid();
if (cat.apps) { const y = window.scrollY; filterCatalog(); window.scrollTo(0, y); }
};
if (!$("apkAltDlg").open) $("apkAltDlg").showModal();
}
$("apkAltClose").onclick = () => $("apkAltDlg").close();
const TITLE_EXT = /\.(zip|exe)$/i; const TITLE_EXT = /\.(zip|exe)$/i;
async function sendFiles(files, dirs = new Set()) { async function sendFiles(files, dirs = new Set()) {
for (const [i, f] of files.entries()) { for (const [i, f] of files.entries()) {
@@ -1996,6 +2050,7 @@ $("repFile").onchange = async () => {
const { apk } = await upload(file, "apkinfo"); const { apk } = await upload(file, "apkinfo");
$("repPkg").value = apk.package; $("repVer").value = apk.version; $("repLabel").value = apk.label; $("repPkg").value = apk.package; $("repVer").value = apk.version; $("repLabel").value = apk.label;
if (!$("repSrc").value) $("repSrc").value = file.name; if (!$("repSrc").value) $("repSrc").value = file.name;
if (apk.blocker) checkApkAlternatives(apk);
$("repFileNote").textContent = apk.blocker ? `Note: ${apk.blocker}` : `${apk.package} ${apk.version}`; $("repFileNote").textContent = apk.blocker ? `Note: ${apk.blocker}` : `${apk.package} ${apk.version}`;
} catch (e) { $("repFileNote").textContent = e.message; } } catch (e) { $("repFileNote").textContent = e.message; }
$("repFile").value = ""; $("repFile").value = "";
+32 -6
View File
@@ -37,6 +37,7 @@ from urllib.parse import parse_qs, unquote, urlparse
sys.path.insert(0, str(Path(__file__).resolve().parent)) sys.path.insert(0, str(Path(__file__).resolve().parent))
import frame_android # noqa: E402 import frame_android # noqa: E402
import frame_apk_versions # noqa: E402
import frame_catalog # noqa: E402 import frame_catalog # noqa: E402
import frame_host # noqa: E402 import frame_host # noqa: E402
import frame_store # noqa: E402 import frame_store # noqa: E402
@@ -91,9 +92,10 @@ exit 1
class Failure(Exception): class Failure(Exception):
def __init__(self, message, status=502): def __init__(self, message, status=502, apk=None):
super().__init__(message) super().__init__(message)
self.status = status self.status = status
self.apk = apk
# What ssh prints when it never reached the Frame, and what to tell the user # What ssh prints when it never reached the Frame, and what to tell the user
@@ -561,16 +563,28 @@ def open_thing(body):
raise Failure("unknown target", 400) raise Failure("unknown target", 400)
def apk_versions(query):
args = parse_qs(query, keep_blank_values=True)
packages, codes = args.get('package', []), args.get('code', [])
if len(packages) != 1 or not frame_android.PKG_RE.match(packages[0]):
raise Failure('invalid Android package id', 400)
if codes and (len(codes) != 1 or not re.fullmatch(r'[0-9]{1,19}', codes[0])):
raise Failure('invalid version code', 400)
return frame_apk_versions.alternatives(packages[0], int(codes[0]) if codes else None)
def android(body): def android(body):
"""Android apps, each in its own persistent Lepton instance (frame_android.py).""" """Android apps, each in its own persistent Lepton instance (frame_android.py)."""
action, pkg = body.get("action"), str(body.get("package", "")) action, pkg = body.get("action"), str(body.get("package", ""))
ensure_master() ensure_master()
try: try:
if action == "install": if action == "install":
frame_catalog.app(pkg) # an unknown package fails now, not in the background url = body.get("url")
if not url:
frame_catalog.app(pkg) # an unknown package fails now, not in the background
def work(): def work():
m = frame_catalog.install(pkg) m = frame_apk_versions.install(pkg, url) if url else frame_catalog.install(pkg)
return {"message": f"Installed {m['label']}. It's in the Steam library; launching it opens its own panel.", return {"message": f"Installed {m['label']}. It's in the Steam library; launching it opens its own panel.",
"app": m} "app": m}
return start_job(f"Install {pkg}", work) return start_job(f"Install {pkg}", work)
@@ -1303,8 +1317,10 @@ class Handler(BaseHTTPRequestHandler):
def send_json(self, obj, status=200): def send_json(self, obj, status=200):
self.send_bytes(json.dumps(obj).encode(), "application/json", status) self.send_bytes(json.dumps(obj).encode(), "application/json", status)
def send_error_json(self, message, status): def send_error_json(self, message, status, apk=None):
body, offline_status = error_body(message) body, offline_status = error_body(message)
if apk is not None:
body["apk"] = apk
self.send_json(body, offline_status or status) self.send_json(body, offline_status or status)
def do_GET(self): def do_GET(self):
@@ -1319,6 +1335,8 @@ class Handler(BaseHTTPRequestHandler):
self.send_json({"os": "SteamOS", "fileManager": None, "computer": DEVICE, "mobile": True} if LOCAL else self.send_json({"os": "SteamOS", "fileManager": None, "computer": DEVICE, "mobile": True} if LOCAL else
{"os": frame_host.NAME, "fileManager": frame_host.FILE_MANAGER, {"os": frame_host.NAME, "fileManager": frame_host.FILE_MANAGER,
"computer": "Mac" if frame_host.MAC else "PC"}) "computer": "Mac" if frame_host.MAC else "PC"})
elif path == "/api/apk-versions":
self.send_json(apk_versions(url.query))
elif path == "/api/android": elif path == "/api/android":
ensure_master() ensure_master()
self.send_json({"apps": frame_android.list_apps()}) self.send_json({"apps": frame_android.list_apps()})
@@ -1358,7 +1376,7 @@ class Handler(BaseHTTPRequestHandler):
else: else:
self.send_json({"error": "not found"}, 404) self.send_json({"error": "not found"}, 404)
except Failure as e: except Failure as e:
self.send_error_json(str(e), e.status) self.send_error_json(str(e), e.status, e.apk)
except frame_android.FrameError as e: except frame_android.FrameError as e:
self.send_error_json(str(e), 502) self.send_error_json(str(e), 502)
except Exception as e: except Exception as e:
@@ -1384,7 +1402,7 @@ class Handler(BaseHTTPRequestHandler):
raise Failure("request body must be a JSON object", 400) raise Failure("request body must be a JSON object", 400)
self.send_json(handler(body)) self.send_json(handler(body))
except Failure as e: except Failure as e:
self.send_error_json(str(e), e.status) self.send_error_json(str(e), e.status, e.apk)
except (ValueError, TypeError) as e: except (ValueError, TypeError) as e:
self.send_json({"error": f"bad request: {e}"}, 400) self.send_json({"error": f"bad request: {e}"}, 400)
except frame_android.FrameError as e: except frame_android.FrameError as e:
@@ -1489,6 +1507,14 @@ class Handler(BaseHTTPRequestHandler):
keep = True # stage_title owns tmp now, and removes it on failure keep = True # stage_title owns tmp now, and removes it on failure
return stage_title(str(dest), temp_dir=str(tmp)) return stage_title(str(dest), temp_dir=str(tmp))
if mode == "apk": if mode == "apk":
try:
info = frame_android.apk_info(str(dest))
except frame_android.FrameError as e:
raise Failure(str(e), 400)
try:
frame_android.check_installable(info)
except frame_android.FrameError as e:
raise Failure(str(e), 400, {"package": info["package"], "version_code": info.get("version_code"), "blocker": str(e)})
ensure_master() ensure_master()
try: try:
m = frame_android.install(str(dest), source=name) m = frame_android.install(str(dest), source=name)