Compare commits

...
Author SHA1 Message Date
saphid 64f46f2930 docs(comfort): record locked Frame and Simulator session verification 2026-09-29 20:15:10 +10:00
saphidandClaude Opus 5.5 be1ab129c9 Tests: read the page as UTF-8 (Windows' default codec can't read its arrows)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 11:33:34 +10:00
saphidandClaude Opus 5.5 ab1985b6b3 Comfort: a state missing 'started' can't crash status (timestamps of 0 still count)
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 11:12:41 +10:00
saphidandClaude Opus 5.5 1203ec0a9e Merge main into family-comfort; a session state without 'started' can't crash status
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 11:09:35 +10:00
Alex Southwell ef56025ad1 Merge pull request #19 from saphid/frame-input
Keyboard and trackpad for the Frame, through its own KDE Connect
2026-09-29 10:33:35 +10:00
Alex Southwell 697452e5a9 Merge pull request #42 from saphid/theatre-media
Add owned Frame-side theatre and stereo media previews
2026-09-29 10:27:40 +10:00
saphid ae7f733b90 Merge remote-tracking branch 'origin/main' into theatre-media
# Conflicts:
#	ui/server.py
2026-09-29 10:21:27 +10:00
saphidandClaude Opus 5.5 b00db2484d Keep the backslash upload-name test POSIX-only
Windows treats the backslash as a separator, so that name can't occur there.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 10:15:37 +10:00
saphid b0d6039eec Merge remote-tracking branch 'origin/main' into theatre-media
# Conflicts:
#	docs/testing.md
#	ui/server.py
2026-09-29 10:08:45 +10:00
saphidandClaude Opus 5.5 86b8ab1d82 Assert the start-failure test reaches systemd-run
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 10:01:17 +10:00
saphidandClaude Opus 5.5 020e3386e1 Make media stop race-free and cover start failures
Stop always calls systemctl and treats exit 5 (unit already collected)
as done, so there is no is-active/stop race. Cleanup never masks the
copy error, the play ssh timeout covers the remote worst case, and
tests cover stop exit codes and systemd-run stderr reporting.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:30:03 +10:00
saphidandClaude Opus 5.5 f81f70ed5d Fix media stop, upload cleanup and review findings
- Stop is a no-op when the collected player unit is already gone
  (raw systemctl stop exits 5 on the Frame; verified 2026-09-29).
- Surface systemd-run stderr when the player can't start.
- Keep the copy error if the cleanup ssh also fails; reject upload
  names that the play path can never accept.
- Allow 60 s for play (ffprobe 30 s + systemd-run 15 s remote).
- Docs: four-hour cap is unconditional; no delete action yet; fix a
  garbled timing sentence.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:22:22 +10:00
saphid 222d5eccc9 fix(comfort): harden timer recovery and notification UX after review 2026-09-29 08:31:00 +10:00
saphid 4d4e45f622 Pin fake Frame data directory modes 2026-09-28 22:46:00 +10:00
saphid 6ecd0cea39 Match fake Frame user data ownership to the headset 2026-09-28 22:44:14 +10:00
saphid c335cd5130 Fix media test portability and e2e discovery 2026-09-28 22:37:28 +10:00
saphid 0d448ab510 Add owned OpenVR media playback and stereo previews 2026-09-28 22:33:50 +10:00
saphid d3fa282377 docs(comfort): include verified desktop and iPhone notification evidence 2026-09-28 22:32:54 +10:00
saphid 0622afcae9 feat(ui): add family controls, casting and native notifications 2026-09-28 22:29:58 +10:00
saphid 522c46ed6f feat(comfort): run safe session timers and alerts on the Frame 2026-09-28 22:29:58 +10:00
37 changed files with 2127 additions and 174 deletions

No files matched your search

+5 -1
View File
@@ -93,7 +93,11 @@ SSH, SFTP, Steam Link, remote desktop, volume, sleep, restart and shut down.
</tr>
</table>
Nothing is installed on the Frame for any of this: the app uses what SteamOS
The optional [Family and comfort](docs/family-comfort.md) card adds session
limits, breaks, local alerts and one-click casting. A session copies a small
Frame Control worker into your headset user account.
For the other features, nothing is installed on the Frame: the app uses what SteamOS
already ships (sideloading a game copies Valve's own devkit scripts to
`~/devkit-utils`, as Valve's Devkit Client does). [How each feature works](docs/frame-control.md).
+17 -2
View File
@@ -1,7 +1,7 @@
// Frame Control as a desktop app (macOS, Windows, Linux): starts ui/server.py on
// a free loopback port and shows it in a native window. The server does all the
// work over the `frame` SSH alias; this file only hosts it.
const { app, BrowserWindow, Menu, clipboard, dialog, ipcMain, shell } = require("electron");
const { app, BrowserWindow, Menu, Notification, clipboard, dialog, ipcMain, shell } = require("electron");
const { execFile, spawn } = require("child_process");
const { promisify } = require("util");
const fs = require("fs");
@@ -254,6 +254,21 @@ ipcMain.handle("update:get", (e) => fromUi(e) ? publicUpdate() : null);
ipcMain.handle("update:check", (e) => fromUi(e) ? checkForUpdate({ manual: true }).then(publicUpdate) : null);
ipcMain.handle("update:install", (e) => { if (fromUi(e)) installUpdate(); });
ipcMain.handle("comfort:notify", (e, message) => {
if (!fromUi(e) || typeof message !== "string" || message.length > 500) throw new Error("Invalid notification");
if (!Notification.isSupported()) throw new Error("System notifications are unavailable");
return new Promise((resolve, reject) => {
const notification = new Notification({title: "Frame Control", body: message});
const timer = setTimeout(() => reject(new Error("Notification delivery was not confirmed. Check system notification settings.")), 5000);
notification.once("show", () => { clearTimeout(timer); resolve(true); });
notification.once("failed", (_event, error) => {
clearTimeout(timer);
reject(new Error("Notification delivery failed. Check system notification settings: " + error));
});
notification.show();
});
});
// frame-control://install links from websites (docs/web-install.md). They can
// arrive before the window or server exists (macOS open-url on a cold launch),
// so they wait here until the page asks for them. The page checks the link with
@@ -376,7 +391,7 @@ function createWindow() {
title: "Frame Control", backgroundColor: BG, show: false,
...(IS_MAC ? { titleBarStyle: "hiddenInset", trafficLightPosition: { x: 18, y: 26 } }
: { icon: path.join(__dirname, "build", "icon.png") }),
webPreferences: { contextIsolation: true, nodeIntegration: false, sandbox: true,
webPreferences: { contextIsolation: true, nodeIntegration: false, sandbox: true, backgroundThrottling: false,
preload: path.join(__dirname, "preload.js") },
});
win.once("ready-to-show", () => win.show());
+1
View File
@@ -9,6 +9,7 @@
const { contextBridge, ipcRenderer, webUtils } = require("electron");
contextBridge.exposeInMainWorld("frameApp", {
notify: (message, request) => ipcRenderer.invoke("comfort:notify", message, request),
readClipboard: () => ipcRenderer.invoke("clipboard:read"),
setUpConnection: () => ipcRenderer.invoke("connection:setup"),
// While the keyboard-and-trackpad panel holds the keyboard, ⌘W, ⌘R and the rest go to the Frame.
@@ -0,0 +1,70 @@
{
"tested_head": "be1ab129c9b8663be8d4cd9d5aea0a4c587a16ba",
"date": "2026-09-29",
"steam_os_build": "20260925.6191901",
"battery_percent_start": 100,
"battery_percent_end": 100,
"activity_level": 3,
"warning_to_home_seconds": 60.76079535484314,
"after": {
"path": "/routes/library/home",
"visible": true,
"apps": []
},
"session": {
"id": "2f03a8c2eb88482ca28b10246ace496b",
"boot": "6c5ddc81-747f-44e5-b8e0-00e0f13fd08e",
"active": false,
"options": {
"minutes": 1,
"breakMinutes": 1,
"stillMinutes": 1,
"batteryAlert": true,
"heatAlert": true
},
"started": 81160.254818568,
"deadline": 81220.254818568,
"lastSample": 81230.770869338,
"used": 0,
"nextBreak": 60,
"stillSent": false,
"warned": 81170.389722976,
"events": [
{
"id": "2f03a8c2eb88482ca28b10246ace496b:1",
"kind": "started",
"message": "Session started. Steam Home opens at the limit; games are not closed.",
"time": 1790676762.9003663
},
{
"id": "2f03a8c2eb88482ca28b10246ace496b:2",
"kind": "warning",
"message": "One minute left. Save your progress; Steam Home will open.",
"time": 1790676773.0352101
},
{
"id": "2f03a8c2eb88482ca28b10246ace496b:3",
"kind": "finished",
"message": "Session ended: Steam Home opened. Your game is still running.",
"time": 1790676833.7960055
}
],
"seq": 3,
"latched": [],
"error": null,
"time": 1790676838.9175165,
"remaining": 0,
"heartbeat": 81230.770869338,
"unavailable": [],
"activity": 3
},
"cleanup": {
"prior_page_restored": true,
"prior_dashboard_visibility_restored": true,
"worker_exited": true,
"temporary_simulator_key_removed": true,
"phone_server_exited": true,
"lock_released": true,
"owned_simulator_shutdown": true
}
}
+30
View File
@@ -0,0 +1,30 @@
# Family and comfort: locked real-Frame verification
**Verified 2026-09-29**, tested feature source at `be1ab12`, SteamOS 0.4.1 build
`20260925.6191901`, iOS 26.5 Simulator. The test acquired
`/tmp/frame-test.lock` before deploying or launching anything.
The Simulator installed the bundled server in the Frame user account, connected
over its SSH tunnel, requested notification permission, displayed a native test
banner, and started a one-minute session through the shared UI API. The real
session warning reached the Simulator as a native banner. Steam Home opened
**60.76 seconds after the successful warning**. The running-app list was empty
before and after; this run does not prove preservation of a running game.
![Real session warning delivered to the iOS Simulator](../img/comfort-live-warning-ios.png)
[Recorded session and cleanup result](comfort-session-20260929.json).
Local raw logs and headset/Simulator captures are retained at
`/tmp/frame-comfort-evidence/locked-run/` on the verification Mac.
**Verified cleanup:** previous page and hidden-dashboard state restored; comfort
worker and phone server exited; temporary Simulator SSH key removed; test lock
released; only the owned Simulator shut down. Battery stayed at 100%. No global
settings, power state, Steam or SteamVR lifecycle changes were made.
**Unverified while unworn:** SteamVR activity was 3 (standby), so active-use
break/check-in clocks stayed at zero. Headset captures returned blank standby
frames. The casting shortcut was invoked, but this run does not establish
visible output or frame rate. No low-battery or overheating condition was
induced. Earlier worn/active-device results remain separately documented in
[Family and comfort](../family-comfort.md).
+122
View File
@@ -0,0 +1,122 @@
# Family and comfort
Frame Control's Home tab has a **Family and comfort** card, on desktop and
on iPhone. No third-party notification or parental-control app is needed.
This is Frame Control code using Python, Steam and SteamVR already on the Frame.
![Family and comfort controls in the desktop app](img/comfort-desktop.png)
## Sessions
Set a limit of 1–240 minutes, optional break and check-in intervals, then
**Start session**. Break and check-in intervals of 0 turn those reminders off.
**Cancel session** cancels the timer and monitoring without changing the game.
Cancel before starting a session with different settings.
The Frame shows a one-minute warning, then opens Steam Home in its dashboard.
**Games stay running**: save and pause before the limit. Some games pause when
the dashboard opens; others do not. There is no kill, power-off, Steam restart,
account restriction or parental lock. The wearer can return to the game.
**Documented implementation:** the timer is a single, opt-in Python worker in
the Frame user's account. Desktop and iPhone share its state. It keeps going
when the companion disconnects, closes or is suspended. It exits after
completion or cancellation (normally within five seconds). Cancellation waits
for any in-flight SteamVR action to finish within its timeout; it is not a boot
service. A Frame reboot invalidates the session. Suspend counts toward the
limit, using Linux's boot-time clock. If a warning was delayed by suspend or a
SteamVR failure, Home waits until at least a full minute after a successful
warning. A failed Home transition remains active and retries, with an error
shown in the companion. A stale worker is reported as unverified enforcement.
## Alerts and breaks
During a session, battery, overheating and check-in alerts go to connected
companions. Break reminders and session warnings also appear on the headset.
- **Low battery:** 15% or below while discharging. One alert until charging or
recovery to 20%, so values around 15% do not produce repeated notifications.
- **Overheating:** a thermal zone reaches its own kernel-reported hot/critical
trip, or the battery reports `Overheat`. Missing sensors mean unknown, not
safe. These are status alerts, not medical advice or an extra thermal governor.
- **Check in:** an alert after the chosen number of active minutes.
- **Breaks:** a SteamVR reminder and companion notification at the chosen interval.
**Inferred:** SteamVR activity levels 1 and 2 are a useful proxy for use, not
proof someone is wearing the headset. Inactive readings reset continuous use;
missing readings add no time. Long gaps count at most 30 seconds. Breaks and
check-ins are distinct from the elapsed-time session limit.
Click **Enable / test notifications** on each companion. iOS asks for permission;
macOS, Windows and Linux follow their notification settings. The page also shows
recent events and errors. Keep Frame Control open and connected for companion
alerts. **Phone alerts are local, not push notifications:** iOS suspension,
force-quit or a lost SSH connection prevents live delivery. Old alerts are not
replayed as a notification burst on reconnect. Headset warnings and the session
limit continue without the phone. A physical iPhone's background delivery has
not been verified and is not guaranteed.
## Casting
**Cast headset view** starts the existing headset Live view and requests full
screen where supported. Show that screen to people in the room, or use the
computer/phone's own screen mirroring. It creates no new stream transport,
public URL or LAN server. iPhone uses the inline viewer if full screen is not
available. The image includes private content visible to the wearer.
## What is installed
The shared authenticated `/api/comfort` endpoint copies three bundled Python
files to `~/.cache/frame-control/comfort/<content-hash>/`. Session state and
locks live in `~/.local/state/frame-control/comfort/`, with a private directory
and 0600 state file. There is no network listener or system service. Cancel a
session before removing these directories. The iPhone's normal server still
exits on disconnect; the explicitly started comfort worker is the exception.
## Verification
**Verified 2026-09-28**, SteamOS 0.4.1, build `20260925.6191901`: shipped
`/opt/steamvr/bin/linuxarm64/vrcmd --notify TEXT` reported success for a custom
reminder. Steam's CDP `SteamUIStore.Navigate('/library/home')` and
`SteamClient.OpenVR.VROverlay.ShowDashboard('valve.steam.gamepadui.main')`
opened Home while the running app ID stayed unchanged. Prior page and dashboard
visibility were restored. Kernel hot/critical trips and SteamVR activity were
read from the real device. No temperature or battery fault was induced.
**Verified locally:** deterministic fake-Frame tests cover late warnings,
failed warnings/Home actions, cancellation, activity gaps, thresholds, duplicate
suppression, reboot invalidation, shared session state and the exact Home
JavaScript. `python3 -m unittest discover -s tests` runs them. The iOS Simulator
build tests notification content and bounds. Physical iPhone delivery and
wearer-perceived headset notification visibility remain unverified.
**Verified end to end on the same Frame:** a two-minute session with no companion
connection for 135 seconds emitted its warning, break and check-in, then opened
Home. The running app ID was unchanged; the test restored the previous page and
dashboard visibility and confirmed the worker exited. Casting through the Home
shortcut decoded the existing headset stream at 30 fps.
**Verified on the iOS 26.5 Simulator:** connected to the real Frame, approved the
notification prompt, and saw the native Frame Control test banner. Seven iOS
tests passed.
![Native test notification in the iOS Simulator](img/comfort-notification-ios.png)
Desktop and 390-pixel phone layouts had no horizontal overflow.
On macOS the development Electron app's real notification attempt was denied
(`UNErrorDomain` 1); the bridge now returns that failure instead of reporting
success. Successful macOS/Windows/Linux notification display remains unverified.
**Verified on the real Frame:** its naturally discharging 15% battery produced
one low-battery event during a short session; the test then cancelled the
session. Overheating alerts use fake sensor samples in tests: the shared
headset was not deliberately overheated.
**Verified 2026-09-29 on the same Frame:** a fresh one-minute session opened
Home more than 60 seconds after the successful warning. The test restored the
previous page and dashboard visibility. Local regression coverage now includes
slow notification delivery, a total Home-action timeout, failed worker startup,
unreadable saved state, malformed activity samples and notification UX: 173
Python tests passed. Desktop and 390-pixel layouts were checked again; system
notification-denial guidance stayed visible across polls. Initial event history
did not replay notifications, and only the latest new event was announced.
Binary file not shown.

After

Width:  |  Height:  |  Size: 192 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.0 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 409 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 901 KiB

+12 -1
View File
@@ -26,7 +26,10 @@ as its transport too), so the desktop and phone share one code path. Android
display settings use `podman exec` into each Lepton container instead of adb,
which the Frame doesn't have.
Nothing is left running on the Frame after the phone disconnects; the copied
The app server stops after the phone disconnects. An explicitly started
[comfort session](family-comfort.md) keeps its timer and headset reminders running
until the session ends or is cancelled; phone notifications require the app to
remain connected and running. The copied
files stay in `~/.cache/frame-control` (delete it any time).
## Pairing
@@ -108,3 +111,11 @@ running), a real sleep/restart/shut down on the Frame, and a physical iPhone.
Debug builds have Simulator test hooks (`FRAME_TEST_HOST`, `FRAME_TEST_PAGE`,
`FRAME_TEST_JS`, and the tunnel URL in the app's Caches folder); release builds
don't.
## Family and comfort
The shared Home card sets session limits, breaks and check-ins, and offers
**Cast headset view**. **Enable / test notifications** requests iOS notification
permission and sends a local test. These are local notifications, not APNs push;
iOS background suspension can interrupt phone alerts. The headset timer still
runs. See [the behavior and verification limits](family-comfort.md).
+5 -4
View File
@@ -102,10 +102,11 @@ Still open: 4, 6, 7, 12–15, 16 (off-LAN and after a reboot), 17–21.
20. **F-Droid 2.0** (Compose 1.12): does it run? If so, the catalogue can
install it instead of 1.17.2.
21. **DeoVR local files:** does DeoVR's file browser show `Videos → VR`
(the symlink from `push-vr-video.sh`) or `Z:\home\steamos\Videos\VR`, and do
the colour-coded test clips play in 3D (red left eye, cyan right) for both
H.264 and H.265? Does the DLNA browser find a server on the Mac?
21. **Owned media player:** worn-headset comfort, audio quality/lip sync, long
movies and 4K/8K decoding remain to check. Native spatial-photo container
extraction, large/immersive splats and existing-panel theatre docking need
further implementation. Remote eye isolation, short hardware decode and
owned-overlay cleanup are verified; see [vr-video.md](vr-video.md).
## Verified 2026-09-27
+9
View File
@@ -123,3 +123,12 @@ a limit on the number of floating panels.
keyboard) or virtual desktops arrange windows within the 1280×800 rectangle.
- **Windows-only overlay tools** (Desktop+, OVR Toolkit, OVRdrop) do this for a
PC's desktop in SteamVR. They don't run on the Frame's standalone Linux.
## Frame Control's media theatre
[The owned media player](vr-video.md) can show its video or stereo image on a
larger, head-relative screen with its own dark surround. **Verified remotely
2026-09-28**, SteamOS 0.4.1 / BUILD_ID 20260925.6191901: screen, eye isolation,
surround and cleanup. It does not alter panel docking or global settings.
Its `Overlay` RGBA rendering hook is available to stream producers; applying
SteamVR theatre docking to existing Mac/PC panels is still unverified here.
+1 -1
View File
@@ -99,7 +99,7 @@ controls to place each panel. See [docs/panels.md](panels.md).
| `scripts/frame-ui.sh` | Mac | Start the Frame Control web UI (`ui/server.py`) and open it (**verified**) |
| `scripts/apk-catalog.sh` | Mac | Refresh the rated F-Droid catalogue that Frame Control's Android section shows (**verified**) |
| `scripts/compat-db-backup.sh` | Mac | Maintainer-only: back up the shared compatibility database locally and to Google Drive (**verified**) |
| `scripts/push-vr-video.sh` | Mac → Frame | Upload VR180/360 videos to `~/Videos/VR`, linked into DeoVR's Proton prefix; `--launch` starts DeoVR (**verified**: upload and link; in-headset playback of local files not yet checked). See [docs/vr-video.md](vr-video.md) |
| `scripts/push-vr-video.sh` | Computer → Frame | Upload movies, stereo PNG/JPEG or small `.splat` files to `~/Videos/FrameControl`; `--launch` starts our OpenVR player, `--theatre` adds a larger screen and dark surround. No separate viewer. **Verified remotely**: decode, stereo output and cleanup; worn-headset checks remain. See [docs/vr-video.md](vr-video.md) |
| `scripts/push.sh` | Mac → Frame | `rsync` files to `~/Downloads` (or a given path) on the Frame (**verified**) |
| `scripts/serve-bootstrap.sh` | Mac | Fallback: serve `bootstrap-on-frame.sh` with your public key embedded |
| `scripts/bootstrap-on-frame.sh` | Frame | Fallback: install the key and enable `sshd` |
+4 -2
View File
@@ -45,8 +45,10 @@ flat 2D desktop streaming into a window on the Frame's Linux desktop.
| Immersed / Virtual Desktop | Vendor apps | Immersed has a Mac agent but no known Frame client. Virtual Desktop's developer said he'd "try" to port it ([NewsBreak](https://www.newsbreak.com/news/4892834783961-virtual-desktop-dev-says-he-ll-try-to-bring-the-app-to-steam-frame)). | Not available as of 2026-09-25. Check again later. |
| WiVRn / ALVR | VR streaming from a Linux or Windows PC | Irrelevant for a Mac host (no SteamVR/OpenXR runtime on macOS) | N/A |
For **VR video files** (180°/360° stereo), don't stream the Mac's screen. Play
them on the Frame in DeoVR instead: see [vr-video.md](vr-video.md).
For **local movies and stereo photos**, Frame Control's own OpenVR player
runs on the Frame; see [vr-video.md](vr-video.md). It currently renders a flat
stereo screen. VR180/360 projection is not implemented; the same page records
DeoVR only as an optional, independently installed alternative.
### Pre-seeding the Remmina profile (no typing in the headset)
+23
View File
@@ -149,6 +149,20 @@ refuses ids with a hyphen (`missing/invalid arguments`), which the fake had
accepted. The fake now refuses them the same way, and Frame Control makes ids
Steam accepts.
## Owned media player
`tests/test_media.py` covers layout evidence and overrides, OU eye ordering,
hardware-decoder command construction, malformed splats, stereo parallax and
fake-Frame library/process ownership. `tests/e2e/test_media_transfer.py` checks the real
HTTP/SSH upload and library listing without pretending the fake renders VR.
Real decode timings and captured stereo output are recorded in
[vr-video.md](vr-video.md). Generated media only; no external player required.
**Verified 2026-09-28**, real Frame BUILD_ID 20260925.6191901: `~/.local`
and `~/.local/share` are `steamos:steamos`, mode 0755. The fake supervisor
sets those parent owners too; previously its root-created Steam manifests
left the parents root-owned and incorrectly prevented user runtime installs.
## Agent interfaces
`tests/test_agent.py` exercises MCP stdio, exact-action human approvals and the
@@ -156,3 +170,12 @@ assistant against an in-process HTTP endpoint with canned responses (no keys or
external calls). `tests/e2e/test_agents.py` runs the MCP/HTTP/SSH path against the
fake Frame for approved installs, clipboard and file transfer. Headset Chromium
rendering and real screenshots still need a device; see [agent evidence](agents.md#evidence-and-limits).
## Family and comfort
`tests/test_comfort.py` uses an injected clock, fake headset sensor readings and
actions, plus a Node fake of Steam's Home API. It covers warnings before Home,
late/suspended sessions, cancellation, failed actions, duplicate alerts, reboot
invalidation, per-zone thermal trips and shared on-headset state. The server
guards reject invalid session settings before SSH. See
[real-device evidence and limits](family-comfort.md#verification).
+139 -72
View File
@@ -1,81 +1,148 @@
# Watching VR video (180°/360°) on the Frame
# Movies, stereo photos and splats
The confidence labels are the same as in [ssh.md](ssh.md). Everything here
was checked on SteamOS 0.3.0, build 20260922.6101926.
Frame Control has its own Frame-side OpenVR player. It uses SteamOS's ffmpeg
and V4L2 hardware decoder, Python and SteamVR. **No separate player or viewer
is required.** Chromium and immersive WebXR are not in this playback path.
## The short version
## Use it
1. Install **DeoVR Video Player** from Steam (free, app 837380) and start it once
in the headset. That creates its Proton prefix.
2. On the Mac: `scripts/push-vr-video.sh --launch ~/Movies/beach_180_LR.mp4`
3. In the headset, open DeoVR's local file browser → **Videos → VR** and
pick the file.
In **Tools → Media in the headset**, send a file, choose its layout and press
**Play**. **Theatre** gives it a larger screen and an 85% black surround.
**Stop** removes both. Refresh reads the library and the player's state.
The screen follows your head; it isn't a saved world-space panel.
## Why DeoVR
The Frame's Chromium can't play VR video in 3D. It has no immersive WebXR
([how-the-frame-works.md](how-the-frame-works.md)). DeoVR's Windows build
runs under Proton ARM64 + FEX as a real SteamVR app. **Verified 2026-09-25:**
it found the Steam Frame headset and controller over OpenVR, and decoded
7680×3840 and 8192×4096 H.265 VR180 side-by-side streams through AVPro's
hardware Media Foundation path, mapped onto a 180° dome or fisheye mesh.
Known quirks (verified):
- The first launch takes about 45 s while it compiles shaders.
- Grid thumbnails stay blank. Unity's own video player, which DeoVR uses for
previews, fails with `0xc00d36bb` under Proton. Full playback uses AVPro
and isn't affected.
- The in-app store and web content are separate from local files. You don't
need an account to play your own files.
## Getting files onto the headset
`scripts/push-vr-video.sh` copies files with `rsync --partial`, so an
interrupted upload resumes. They go to `~/Videos/VR` on the Frame (`/home`
has about 860 GB free). The script also links that folder into DeoVR's prefix
as `C:\users\steamuser\Videos\VR`. It's reachable at
`Z:\home\steamos\Videos\VR` as well. **Verified** that the upload and link
work. **Not yet checked** whether DeoVR's file browser lands there
(open question 21).
Speed: a test upload over Wi-Fi ran at about 3–5 MB/s (verified 2026-09-25,
one sample). At that rate an 8K file of several GB takes tens of minutes, so
start big uploads before you put the headset on.
## Naming files so they play correctly
DeoVR guesses the projection from the file name. Its binary contains the tags
`_180`, `_360`, `_fisheye`, `_fisheye190`, `_mkx200`, `_vrca220` and `_rf52`
(verified). For stereo layout, the common DeoVR convention is `_LR`/`_SBS`
(side by side) and `_TB` (top/bottom) (inferred). If a video looks wrong
(doubled, warped, or flat), change the projection and stereo mode in DeoVR's
player menu.
Examples: `trip_180_LR.mp4`, `concert_360_TB.mp4`, `hike_fisheye190_LR.mp4`.
Codecs: H.265 at 8K played (verified, streamed). Local H.264 and H.265
files haven't been played yet. The test clips below cover that.
## Test clips
The script doesn't include these. To check a setup, make two 20 s clips:
3840×1920, 180° side by side, with the left eye tinted red and the right eye
cyan. In the headset each eye should see only its own colour. A single mixed
colour means the stereo split is wrong.
The command-line route uses the same player:
```sh
ffmpeg -f lavfi -i testsrc2=size=1920x1920:rate=30:duration=20 \
-filter_complex "[0:v]split[a][b];[a]colorchannelmixer=rr=1:gg=0.3:bb=0.3[l];[b]colorchannelmixer=rr=0.3:gg=1:bb=1[r];[l][r]hstack" \
-c:v libx264 -pix_fmt yuv420p -b:v 20M frame-test_180_LR_h264.mp4
scripts/push-vr-video.sh frame-test_180_LR_h264.mp4
scripts/push-vr-video.sh --launch --theatre ~/Movies/film_SBS.mp4
scripts/push-vr-video.sh --launch --layout ou ~/Pictures/stereo.png
scripts/push-vr-video.sh --launch capture.splat
scripts/push-vr-video.sh --list
scripts/push-vr-video.sh --stop
```
## Streaming from the Mac instead of copying (untested)
Uploads live in `~/Videos/FrameControl/<id>/` on the Frame. Each upload gets
its own directory, so sending another file with the same name doesn't replace
it. The UI's existing upload limit is 8 GiB. Transfers use the app's rsync/scp
path; resumable large uploads are not implemented here yet. Existing
`~/Videos/VR` files and Proton prefixes are left alone. The script no longer
launches DeoVR, links a Proton prefix, or accepts directories.
DeoVR has a DLNA browser (the binary contains `Searching for DLNA
devices...` and a UPnP ContentDirectory client). A DLNA server on the Mac
should therefore appear in DeoVR without copying anything, for example
`brew install rclone` then `rclone serve dlna ~/Movies/VR`. This is **inferred**, not
tried. 8K VR video needs roughly 50–100 Mbit/s sustained, and the Wi-Fi
sample above (about 30–40 Mbit/s) suggests copying first is the safer default.
| Media | Supported preview |
|---|---|
| Movies | H.264 / H.265, mono or left-first SBS / top-first OU; audio through the Frame's PulseAudio-compatible server |
| Stereo photos | PNG / JPEG containing both eyes, SBS or OU |
| Gaussian splats | Common 32-byte `.splat` records, 1–20,000 Gaussians; a stationary stereo preview |
Auto layout reads delimited filename tags: `_SBS`, `_HSBS`, `_LR`, `_OU`,
`_TB`, `_HOU`, `_HTB`, `_FSBS`, `_FOU`, `_FTB`. SBS/OU without `F` means
half-resolution packing. Full packing preserves each eye's original aspect.
It also accepts FFmpeg's `stereo_mode` metadata (`left_right`, `top_bottom`,
`mono`); left/right and top/bottom metadata are treated as full packing.
Choose an explicit layout if that assumption doesn't match the file.
Unknown or conflicting tags ask for a choice, rather than silently flattening
stereo. The explicit selector always wins. Layout is not guessed from resolution.
## What was verified
**Verified remotely, 2026-09-28:** SteamOS 0.4.1, BUILD_ID
`20260925.6191901`, SteamVR 2.18.1. Nobody wore the headset for these checks.
All test media were generated by us. No paid content or DRM was involved.
- Stock `ffmpeg` `h264_v4l2m2m` decoded 150 frames of our 1920×1080 H.264
test in 0.128 s (decode only); converting all frames to RGBA took 0.242 s.
- Our ffmpeg → Python → OpenVR path submitted all 150 frames and exited 0.
The 1280×720 prototype took 4.775 s; the full 1920×1080 player took
4.618 s. These are wall times, not in-headset frame-rate measurements.
Finishing a 5 s clip early showed those probes weren't paced, so the final
player paces output at 30 fps: all 150 frames then completed in **5.025 s**.
- The H.265 hardware decoder also completed the generated 1080p clip
(60 frames, exit 0); this is a short compatibility check, not a 4K/8K benchmark.
- Our actual player, launched through Frame Control's media API, displayed
SBS and OU photos with red only in the left-eye capture and cyan only in
the right. OpenVR's `SideBySide_Parallel` flag separates the eyes; we
rearrange OU rows ourselves.
- A generated 48-Gaussian `.splat` rendered in our own CPU renderer and appeared
in both eyes with separate perspective projections.
- Theatre's owned dark surround and screen appeared in captures. Steam's
dashboard remained available over them. Stop removed the owned overlays
and ended the dedicated user service. No global SteamVR setting was changed.
- A generated H.264/AAC clip reached the Pulse audio output and completed
all 100 video frames with exit 0. This proves the output path, not audible
quality or lip sync.
![Frame Control SBS eye-isolation proof: red left, cyan right](img/media-sbs-proof.png)
![Our Gaussian-splat stereo preview on the Frame](img/media-splat-proof.png)
**Verified failed route:** GStreamer 1.24.2's `playbin` selected
`v4l2h264dec`, delivered the first RGBA sample and then segfaulted (exit 139)
in the basic appsink probe and the OpenVR probe. We do not ship that route.
The ffmpeg path above completed instead.
## Limits and blockers
- **Not verified:** worn-headset comfort, sound quality/lip sync, long movies,
4K/8K decode, HDR, controller interaction or behaviour on other OS builds.
Output is bounded to 1920×1080 packed pixels before OU rearrangement.
- **Not implemented:** pause, seeking, subtitles, playlists, right-eye-first
layouts, non-square pixel correction, VR180/360 projection and fisheye.
This preview is a flat stereo screen, not a dome player.
- **Native spatial-photo blocker:** HEIC/HEIF/AVIF/MPO stereo-container
extraction isn't implemented in our viewer. We reject these rather than
displaying one image and calling it spatial. Export both eyes to PNG/JPEG
first. This is a current implementation gap, not a claim that the Frame
cannot support these containers.
- **Large/immersive splat blocker:** the owned CPU rasterizer projects 3D
covariance into each eye and alpha-composites Gaussians, but renders a
fixed view at 320×240 per eye. It caps each Gaussian footprint at 32 pixels
and normalizes the scene to a two-metre box. Large scenes, PLY/SPZ, live
head-position parallax and navigation need a GPU scene renderer; this
version rejects files above 20,000 records. It is a stereo preview, not
an immersive walk-through.
- A single player can run at a time. It stops at EOF, on **Stop**, or after
four hours in any case, so longer movies are cut off. Still images remain
until Stop or that timeout. There is no delete action yet: remove old
uploads from `~/Videos/FrameControl/` over SSH. The log is
`~/.local/share/frame-control/media/player.log` on the Frame.
- **Panel/stream theatre remains outside this media slice:** SteamVR's
`vrcmd --dock-overlay` accepts `theater`, but docking an existing panel
and its dimming behaviour were not verified here. The shared headset had
workspace and stream tests active. We did not reposition their panels.
Integration with [#22](https://github.com/saphid/frame-control/issues/22)
and [#31](https://github.com/saphid/frame-control/issues/31) can use the
owned rendering hook below; no second stream/window manager is introduced.
## Integration hook
`POST /api/upload` with `X-Mode: media` and `X-Filename` sends a file and
returns its `id`. `POST /api/media` accepts:
```json
{"action":"play","id":"<32 hex characters>/film_SBS.mp4","layout":"auto","theatre":true}
```
Other actions are `list`, `status`, `stop`. These use the normal `X-Frame-UI`
guard. Play reports **starting**, not a claim that frames reached the headset;
read status for `playing`, `ended`, `stopped` or `error`.
The own-code files are copied to `~/.local/share/frame-control/media/` and
run in the `frame-control-media.service` systemd user unit. Stop affects only
that unit, including its decoder child.
For a Frame-side stream producer, `frame_media_player.Overlay` exposes
`create(key, width, distance, stereo=False, aspect=1, order=1)`,
`pixels(handle, rgba, width, height)` and `close()`. Use an owned unique key,
pass interleaved RGBA bytes (left/right halves for stereo) and always close in
`finally`. `create` uses a head-relative transform; it does not move another
app's panel. The player demonstrates a separate black surround overlay.
`frame_media.stereo_pixels` converts OU to SBS. This is the narrow rendering
hook for stream/workspace work; it doesn't capture or manage a Mac/PC stream.
## Optional separate app
You can independently install **DeoVR Video Player** (free Steam app 837380)
if you prefer its VR180/360 features. Earlier tests on SteamOS 0.3.0,
build `20260922.6101926` (2026-09-25), verified its OpenVR initialization and
8K H.265 streamed VR180 decoding under Proton. Frame Control's media features
neither install nor launch it, and do not depend on it. Its behaviour and
file-naming conventions are not evidence about our player.
@@ -17,6 +17,7 @@
78427FC66780623F31E7501E /* FrameControlApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = 93C8E0D7C3F4F628941B3D5A /* FrameControlApp.swift */; };
84423CB45629465420180A64 /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 8F2CB550FC81C01E6BDD5A71 /* Assets.xcassets */; };
9657F7BC23E3352E5AB30777 /* SetupView.swift in Sources */ = {isa = PBXBuildFile; fileRef = DB544223FC60A59CC3E8EF5F /* SetupView.swift */; };
A0C5B00E257230A38DBD9E54 /* ComfortNotificationTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = E81218B75FEEE47B8D8BAE20 /* ComfortNotificationTests.swift */; };
A8C7AED25A6280682FCE45DC /* Citadel in Frameworks */ = {isa = PBXBuildFile; productRef = 6BA549B6CC0A0CB847126456 /* Citadel */; };
DC043FB74BE2D23F3A5826BF /* FrameControlTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1740B691F9C25E5FB6F9EFC3 /* FrameControlTests.swift */; };
E6898C714A92D3979F73B6E1 /* FrameFinder.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2F288DF6636A417F0CA3A6CD /* FrameFinder.swift */; };
@@ -48,6 +49,7 @@
BF0FCA7117DA3ABA449B4EE0 /* InstallLink.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = InstallLink.swift; sourceTree = "<group>"; };
D6C4E6C28315CA8729FCAAEA /* WebShell.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = WebShell.swift; sourceTree = "<group>"; };
DB544223FC60A59CC3E8EF5F /* SetupView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = SetupView.swift; sourceTree = "<group>"; };
E81218B75FEEE47B8D8BAE20 /* ComfortNotificationTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ComfortNotificationTests.swift; sourceTree = "<group>"; };
EDC7BA8014DBC302D08FD397 /* HeadsetServer.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HeadsetServer.swift; sourceTree = "<group>"; };
F3E2F5607DD877272483D64E /* FrameControl.app */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.application; path = FrameControl.app; sourceTree = BUILT_PRODUCTS_DIR; };
/* End PBXFileReference section */
@@ -107,6 +109,7 @@
75A17B1C79C8C3C60FEABBA6 /* FrameControlTests */ = {
isa = PBXGroup;
children = (
E81218B75FEEE47B8D8BAE20 /* ComfortNotificationTests.swift */,
1740B691F9C25E5FB6F9EFC3 /* FrameControlTests.swift */,
);
path = FrameControlTests;
@@ -274,6 +277,7 @@
isa = PBXSourcesBuildPhase;
buildActionMask = 2147483647;
files = (
A0C5B00E257230A38DBD9E54 /* ComfortNotificationTests.swift in Sources */,
DC043FB74BE2D23F3A5826BF /* FrameControlTests.swift in Sources */,
);
runOnlyForDeploymentPostprocessing = 0;
+44 -2
View File
@@ -1,6 +1,7 @@
import SwiftUI
import UIKit
import WebKit
import UserNotifications
/// The Frame Control page, served by the server on the headset, in a web view.
/// window.frameApp (the same bridge the desktop app's preload.js provides) lets
@@ -48,6 +49,7 @@ struct WebShell: UIViewRepresentable {
let installCb = null;
window.frameApp = {
platform: "ios",
notify: (message, request) => call("notify", { message, request }),
readClipboard: () => call("readClipboard"),
setUpConnection: () => call("setUpConnection"),
open: (what) => call("open", what),
@@ -58,13 +60,31 @@ struct WebShell: UIViewRepresentable {
})();
"""
final class Coordinator: NSObject, WKScriptMessageHandlerWithReply, WKNavigationDelegate, WKUIDelegate {
final class Coordinator: NSObject, WKScriptMessageHandlerWithReply, WKNavigationDelegate, WKUIDelegate, UNUserNotificationCenterDelegate {
let model: AppModel
weak var web: WKWebView?
var loaded: URL?
private var installReady = false
init(model: AppModel) { self.model = model }
init(model: AppModel) {
self.model = model
super.init()
UNUserNotificationCenter.current().delegate = self
}
func userNotificationCenter(_ center: UNUserNotificationCenter, willPresent notification: UNNotification,
withCompletionHandler completionHandler: @escaping (UNNotificationPresentationOptions) -> Void) {
completionHandler([.banner, .sound, .list])
}
static func notificationContent(_ message: String) -> UNMutableNotificationContent? {
guard !message.isEmpty, message.count <= 500 else { return nil }
let content = UNMutableNotificationContent()
content.title = "Frame Control"
content.body = message
content.sound = .default
return content
}
// MARK: bridge
@@ -76,6 +96,28 @@ struct WebShell: UIViewRepresentable {
}
let arg = body["arg"]
switch name {
case "notify":
guard message.frameInfo.isMainFrame,
message.frameInfo.securityOrigin.host == "127.0.0.1",
let args = arg as? [String: Any], let text = args["message"] as? String,
let content = Self.notificationContent(text) else {
return replyHandler(nil, "Invalid notification")
}
let center = UNUserNotificationCenter.current()
let send: (Bool, Error?) -> Void = { allowed, error in
guard allowed else {
return replyHandler(nil, error?.localizedDescription ?? "Notifications are off. Enable them in iOS Settings.")
}
let request = UNNotificationRequest(identifier: UUID().uuidString, content: content, trigger: nil)
center.add(request) { error in replyHandler(error == nil, error?.localizedDescription) }
}
if args["request"] as? Bool == true {
center.requestAuthorization(options: [.alert, .sound], completionHandler: send)
} else {
center.getNotificationSettings { settings in
send(settings.authorizationStatus == .authorized || settings.authorizationStatus == .provisional, nil)
}
}
case "readClipboard":
replyHandler(UIPasteboard.general.string ?? "", nil)
case "setUpConnection":
@@ -0,0 +1,14 @@
import XCTest
import UserNotifications
@testable import Frame_Control
final class ComfortNotificationTests: XCTestCase {
func testNotificationContentAndBounds() {
let content = WebShell.Coordinator.notificationContent("Time for a break")
XCTAssertEqual(content?.title, "Frame Control")
XCTAssertEqual(content?.body, "Time for a break")
XCTAssertNotNil(content?.sound)
XCTAssertNil(WebShell.Coordinator.notificationContent(""))
XCTAssertNil(WebShell.Coordinator.notificationContent(String(repeating: "x", count: 501)))
}
}
+6 -63
View File
@@ -1,65 +1,8 @@
#!/usr/bin/env zsh
# Mac-side: upload VR videos to the Steam Frame so DeoVR can play them in 3D.
#
# Files go to ~/Videos/VR on the Frame. The script links that folder into
# DeoVR's Proton prefix as C:\users\steamuser\Videos\VR, so DeoVR's file
# browser finds it under Videos. (It's also reachable as Z:\home\steamos\Videos\VR.)
# Uploads resume if interrupted.
#
# Name files so DeoVR picks the projection: include _180 or _360 (or _fisheye190,
# _mkx200, _rf52 ...) plus the stereo layout (_LR / _SBS side by side, _TB over-
# under), e.g. "beach_180_LR.mp4". You can also change it in DeoVR's player.
#
# Usage:
# scripts/push-vr-video.sh FILE_OR_DIR... # upload
# scripts/push-vr-video.sh --launch FILE... # upload, then start DeoVR
# scripts/push-vr-video.sh --launch # just start DeoVR
# scripts/push-vr-video.sh --list # what's on the Frame
# Upload media for Frame Control's own OpenVR player (no external player).
# Usage: scripts/push-vr-video.sh [--launch] [--layout auto|mono|sbs|ou|full-sbs|full-ou] [--theatre] FILE...
# scripts/push-vr-video.sh --list | --stop
# Files go to ~/Videos/FrameControl/<id>/; --launch accepts exactly one file.
# Directories, automatic DeoVR launching and Proton-prefix links are no longer used.
set -euo pipefail
FRAME_ALIAS=${FRAME_ALIAS:-frame}
DEOVR_APPID=837380
REMOTE_DIR="Videos/VR"
PREFIX_VIDEOS=".local/share/Steam/steamapps/compatdata/$DEOVR_APPID/pfx/drive_c/users/steamuser/Videos"
launch=0 list=0
while (( $# )); do
case "$1" in
-h|--help) sed -n '2,17p' "$0"; exit 0 ;;
--launch) launch=1; shift ;;
--list) list=1; shift ;;
--) shift; break ;;
-*) print -u2 "push-vr-video: unknown option $1"; exit 2 ;;
*) break ;;
esac
done
(( $# || launch || list )) || { sed -n '2,17p' "$0" >&2; exit 2; }
for f in "$@"; do
[[ -e "$f" ]] || { print -u2 "push-vr-video: no such file: $f"; exit 2; }
done
# Create the folder and link it into DeoVR's prefix (the prefix exists once
# DeoVR has run). Refresh a stale link, but never replace a real directory.
if (( $# || launch )); then
ssh "$FRAME_ALIAS" "mkdir -p ~/$REMOTE_DIR
p=~/$PREFIX_VIDEOS
if [ -d \"\$p\" ] && { [ -L \"\$p/VR\" ] || [ ! -e \"\$p/VR\" ]; }; then ln -sfn ~/$REMOTE_DIR \"\$p/VR\"
elif [ -d \"\$p/VR\" ]; then echo \"warning: \$p/VR is a real folder, so uploads won't show under DeoVR's Videos; browse Z:\\\\home\\\\steamos\\\\Videos\\\\VR instead\" >&2; fi
[ -d \"\$p\" ] || echo 'note: DeoVR has not run yet; use Z:\\home\\steamos\\Videos\\VR or run this again after starting it once' >&2"
fi
if (( $# )); then
# -L: send what a symlink points at; a Mac-side link would dangle on the Frame
rsync -aL --partial --progress -- "$@" "$FRAME_ALIAS:$REMOTE_DIR/"
fi
if (( list )); then
ssh "$FRAME_ALIAS" "cd ~/$REMOTE_DIR && ls -lhR"
fi
if (( launch )); then
ssh "$FRAME_ALIAS" "command -v steam >/dev/null || { echo 'steam not found on the Frame' >&2; exit 1; }
steam steam://rungameid/$DEOVR_APPID </dev/null >/dev/null 2>&1 &"
print "DeoVR starting on the Frame. Open Local files / the file browser → Videos → VR."
fi
exec python3 "${0:A:h}/../ui/frame_media_cli.py" "$@"
+21
View File
@@ -0,0 +1,21 @@
"""Media transfer through the real HTTP/SSH path; the fake has no VR renderer."""
import harness
from harness import ok, ssh
harness.require()
class Media(harness.FrameTestCase):
def test_upload_and_list_without_launching_a_viewer(self):
# The transfer endpoint doesn't decode. Rendering belongs to the real
# headset smoke checks documented in docs/vr-video.md.
self.assertEqual(ssh('stat -c "%U:%G %a" ~/.local/share').strip(), 'steamos:steamos 755')
result = ok('POST', '/api/upload', raw=b'fake-media', headers={
'X-Mode': 'media', 'X-Filename': 'test_SBS.png'})
identity = result['id']
try:
files = ok('POST', '/api/media', {'action': 'list'})['files']
self.assertIn(identity, [f['id'] for f in files])
self.assertEqual(ssh('cat ~/Videos/FrameControl/'+identity), 'fake-media')
finally:
ssh('rm -rf ~/Videos/FrameControl/'+identity.split('/')[0])
@@ -231,6 +231,13 @@ def sysfs(state):
def runtimes(state):
"""Installed compat tools as Steam app manifests, and the Lepton launcher itself."""
# Verified 2026-09-28, BUILD_ID 20260925.6191901: both parents are
# steamos:steamos 0755. The root supervisor must not leave them root-owned
# when creating Steam's fake manifests; user-account app installs need them.
for directory in (HOME + '/.local', HOME + '/.local/share'):
os.makedirs(directory, mode=0o755, exist_ok=True)
chown(directory)
os.chmod(directory, 0o755)
apps = fs.STEAM_ROOT + '/steamapps'
for alias, installed in state['runtimes'].items():
acf = f'{apps}/appmanifest_{fs.RUNTIME_APPIDS[alias]}.acf'
+257
View File
@@ -0,0 +1,257 @@
"""Fake-Frame session clock/actions plus real helper serialization and sensor probes."""
import os
import shutil
import json
from pathlib import Path
import subprocess
import sys
import tempfile
import unittest
from unittest.mock import Mock, patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
import frame_comfort as comfort
import frame_status as status
OPTIONS = {'action': 'start', 'minutes': 3, 'breakMinutes': 1, 'stillMinutes': 1,
'batteryAlert': True, 'heatAlert': True}
class SessionTests(unittest.TestCase):
def setUp(self):
self.s = comfort.new_session(OPTIONS, 0, 'boot-one')
self.warn, self.home = Mock(), Mock()
def step(self, now, **sample):
comfort.tick(self.s, now, sample, self.warn, self.home, read_clock=lambda: now)
def test_warning_then_home_never_closes_a_game(self):
self.step(119)
self.warn.assert_not_called()
self.step(120)
self.warn.assert_called_once()
self.step(179)
self.home.assert_not_called()
self.step(180)
self.home.assert_called_once()
self.assertFalse(self.s['active'])
self.step(181)
self.home.assert_called_once()
def test_late_wakeup_always_gets_a_full_warning_minute(self):
self.step(400)
self.home.assert_not_called()
self.step(459)
self.home.assert_not_called()
self.step(460)
self.home.assert_called_once()
def test_slow_warning_still_leaves_a_full_minute(self):
comfort.tick(self.s, 120, {}, self.warn, self.home, read_clock=lambda: 140)
self.assertEqual(self.s['warned'], 140)
self.step(180)
self.home.assert_not_called()
self.step(199)
self.home.assert_not_called()
self.step(200)
self.home.assert_called_once()
def test_failed_warning_never_stops_session(self):
self.warn.side_effect = RuntimeError('offline')
with self.assertRaises(RuntimeError):
self.step(200)
self.assertIsNone(self.s['warned'])
self.home.assert_not_called()
self.warn.side_effect = None
self.step(300)
self.step(359)
self.home.assert_not_called()
self.step(360)
self.home.assert_called_once()
def test_failed_home_stays_active_and_retries(self):
self.step(120)
self.home.side_effect = RuntimeError('Steam offline')
with self.assertRaises(RuntimeError):
self.step(180)
self.assertTrue(self.s['active'])
self.home.side_effect = None
self.step(185)
self.assertFalse(self.s['active'])
def test_cancel_prevents_all_actions(self):
self.s['active'] = False
self.step(999, battery={'percent': 1, 'status': 'Discharging'})
self.warn.assert_not_called()
self.home.assert_not_called()
self.assertEqual(self.s['events'], [])
def test_breaks_and_checkin_require_measured_activity(self):
self.step(20, activity=1)
self.step(40, activity=2)
self.step(60, activity=1)
self.assertEqual([e['kind'] for e in self.s['events']], ['break', 'still'])
self.step(70, activity=1)
self.assertEqual(len(self.s['events']), 2)
self.step(75, activity=3)
self.assertEqual(self.s['used'], 0)
self.assertFalse(self.s['stillSent'])
def test_unknown_activity_and_gaps_do_not_count_as_wear(self):
self.step(25)
self.assertEqual(self.s['used'], 0)
self.assertEqual(self.s['unavailable'], ['battery', 'temperature', 'activity'])
self.step(100, activity=1)
self.assertEqual(self.s['used'], 30)
def test_alerts_latch_and_rearm_without_battery_chatter(self):
low = {'percent': 10, 'status': 'Discharging'}
self.step(1, battery=low, thermal=['cpu'])
self.step(2, battery=low, thermal=['cpu'])
self.step(3)
self.assertEqual(len(self.s['events']), 2)
self.step(4, battery={'percent': 16, 'status': 'Discharging'}, thermal=[])
self.step(5, battery=low, thermal=[])
self.assertEqual(len(self.s['events']), 2)
self.step(6, battery={'percent': 22, 'status': 'Discharging'}, thermal=[])
self.step(7, battery=low, thermal=['cpu'])
self.assertEqual([e['kind'] for e in self.s['events']], ['battery', 'heat', 'battery', 'heat'])
def test_disabled_alerts_and_charging(self):
self.s['options']['heatAlert'] = False
self.step(1, battery={'percent': 2, 'status': 'Charging'}, thermal=['cpu'])
self.assertEqual(self.s['events'], [])
def test_invalid_options(self):
for key, value in [('minutes', 0), ('minutes', 241), ('minutes', True), ('minutes', 2.5),
('breakMinutes', -1), ('stillMinutes', '1'), ('heatAlert', 1)]:
with self.subTest(key=key, value=value), self.assertRaises(ValueError):
comfort.validate({**OPTIONS, key: value})
for value in (None, [], {'action': 'shutdown'}):
with self.assertRaises(ValueError):
comfort.validate(value)
def test_restart_invalidates_session_and_stale_worker_is_explicit(self):
with patch.object(comfort, 'boot', return_value='boot-one'), patch.object(comfort.time, 'time', return_value=999):
current = comfort.current(self.s, 100)
self.assertIn('not responding', current['error'])
self.assertEqual(current['time'], 999)
with patch.object(comfort, 'boot', return_value='boot-two'):
result = comfort.current(self.s, 100)
self.assertFalse(result['active'])
self.assertIn('restarted', result['error'])
@unittest.skipUnless(os.name == "posix", "on-headset state uses POSIX flock")
def test_real_state_commands_share_one_session_and_cancel(self):
with tempfile.TemporaryDirectory() as tmp, patch.object(comfort, 'ROOT', Path(tmp)), \
patch.object(comfort, 'boot', return_value='boot-one'), \
patch.object(comfort, 'clock', return_value=0), patch.object(comfort.subprocess, 'Popen') as spawn:
started = comfort.command(OPTIONS)
self.assertEqual(comfort.command({'action': 'status'})['id'], started['id'])
with self.assertRaises(ValueError):
comfort.command(OPTIONS)
self.assertFalse(comfort.command({'action': 'cancel'})['active'])
spawn.assert_called_once()
self.assertEqual((Path(tmp) / 'session.json').stat().st_mode & 0o777, 0o600)
@unittest.skipUnless(os.name == "posix", "on-headset state uses POSIX flock")
def test_cancel_clears_stale_worker_error(self):
with tempfile.TemporaryDirectory() as tmp, patch.object(comfort, 'ROOT', Path(tmp)), \
patch.object(comfort, 'boot', return_value='boot-one'), \
patch.object(comfort, 'clock', return_value=200):
with comfort.locked():
comfort.save(self.s)
self.assertIn('not responding', comfort.command({'action': 'status'})['error'])
cancelled = comfort.command({'action': 'cancel'})
self.assertFalse(cancelled['active'])
self.assertIsNone(cancelled['error'])
self.assertIsNone(comfort.command({'action': 'status'})['error'])
@unittest.skipUnless(os.name == "posix", "on-headset state uses POSIX flock")
def test_failed_spawn_leaves_session_inactive_and_retryable(self):
with tempfile.TemporaryDirectory() as tmp, patch.object(comfort, 'ROOT', Path(tmp)), \
patch.object(comfort, 'boot', return_value='boot-one'), \
patch.object(comfort, 'clock', return_value=0), patch.object(comfort.subprocess, 'Popen') as spawn:
spawn.side_effect = OSError('process limit')
with self.assertRaises(OSError):
comfort.command(OPTIONS)
failed = comfort.command({'action': 'status'})
self.assertFalse(failed['active'])
self.assertIn('Could not start', failed['error'])
spawn.side_effect = None
self.assertTrue(comfort.command(OPTIONS)['active'])
@unittest.skipUnless(os.name == "posix", "on-headset state uses POSIX flock")
def test_unreadable_state_is_preserved_and_can_be_replaced(self):
for contents in (b'{broken', b'\xff', b'null', b'[]', b'42', b'"x"'):
with self.subTest(contents=contents):
with tempfile.TemporaryDirectory() as tmp, patch.object(comfort, 'ROOT', Path(tmp)), \
patch.object(comfort, 'boot', return_value='boot-one'), \
patch.object(comfort, 'clock', return_value=0), patch.object(comfort.subprocess, 'Popen'):
(Path(tmp) / 'session.json').write_bytes(contents)
failed = comfort.command({'action': 'status'})
self.assertFalse(failed['active'])
self.assertIn('unreadable', failed['error'])
backups = list(Path(tmp).glob('session-unreadable-*.json'))
self.assertEqual(len(backups), 1)
self.assertEqual(backups[0].read_bytes(), contents)
self.assertTrue(comfort.command(OPTIONS)['active'])
def test_home_has_total_process_deadline_and_propagates_timeout(self):
with patch.object(comfort.subprocess, 'run', side_effect=subprocess.TimeoutExpired('home', 15)) as run:
with self.assertRaises(subprocess.TimeoutExpired):
comfort.home()
self.assertEqual(run.call_args.kwargs['timeout'], 15)
self.assertEqual(run.call_args.args[0][-1], '--home')
def test_native_warning_reports_failures_and_quotes_as_one_argument(self):
with patch.object(comfort.subprocess, 'run') as run:
run.return_value = subprocess.CompletedProcess([], 0, 'Notification succeeded', '')
comfort.notify('Save "now"; $(nothing)')
args = run.call_args.args[0]
self.assertEqual(args, [comfort.VRCMD, '--notify', 'Frame Control: Save "now"; $(nothing)'])
run.return_value.stdout = 'Notification failed with error 1'
with self.assertRaises(RuntimeError):
comfort.notify('test')
@unittest.skipUnless(shutil.which("node"), "Node exercises the fake Steam JS context")
def test_home_javascript_against_fake_steam_preserves_game(self):
# Same JS runs in Steam CDP. This fake records navigation and refuses any
# unexpected API call; it offers no shutdown or terminate-game primitive.
js = '''let running = [123], path = '/routes/library/app/123', visible = false;
const location = {get pathname() {return path;}};
const SteamUIStore = {Navigate(p) {path = '/routes' + p;}};
const SteamClient = {OpenVR: {VROverlay: {
async ShowDashboard(key) {if (key !== 'valve.steam.gamepadui.main') throw Error(key); visible = true;},
async IsDashboardVisible() {return visible;}
}}};
'''
js += comfort.HOME_JS + '.then(result => console.log(JSON.stringify({result, running, visible})));'
r = subprocess.run(['node', '-e', js], capture_output=True, text=True, check=True)
result = json.loads(r.stdout)
self.assertEqual(result['running'], [123])
self.assertTrue(result['visible'])
self.assertEqual(result['result']['path'], '/routes/library/home')
class SensorTests(unittest.TestCase):
def test_hot_trip_uses_its_own_zone_not_hottest_unrelated_chip(self):
values = {'/z/a/temp': '90000', '/z/a/trip_point_0_type': 'hot', '/z/a/trip_point_0_temp': '110000',
'/z/b/temp': '45000', '/z/b/trip_point_0_type': 'hot', '/z/b/trip_point_0_temp': '44000', '/z/b/type': 'battery'}
def glob(pattern):
if pattern.endswith('thermal_zone*'):
return ['/z/a', '/z/b']
return [pattern.replace('*', '0')]
with patch.object(status.glob, 'glob', side_effect=glob), patch.object(status, 'read', side_effect=values.get):
self.assertEqual(status.thermal_alerts(), [{'zone': 'battery', 'tempC': 45, 'limitC': 44}])
def test_missing_thermal_and_activity_are_unknown(self):
with patch.object(status.glob, 'glob', return_value=[]):
self.assertIsNone(status.thermal_alerts())
with patch.object(status, 'run', return_value='unavailable'):
self.assertIsNone(status.activity_level())
for malformed in ('{}', '[null, 42, "bad"]'):
with patch.object(status, 'run', return_value=malformed):
self.assertIsNone(status.activity_level())
with patch.object(status, 'run', return_value='[{"operation":"status","activity_level":3}]'):
self.assertEqual(status.activity_level(), 3)
+60
View File
@@ -0,0 +1,60 @@
"""Run the actual shared page's comfort renderer against a minimal DOM/bridge."""
import pathlib
import shutil
import subprocess
import unittest
ROOT = pathlib.Path(__file__).resolve().parents[1]
@unittest.skipUnless(shutil.which('node'), 'Node exercises the shared page JS')
class ComfortUI(unittest.TestCase):
def test_notification_failure_survives_poll_until_success(self):
page = (ROOT / 'ui/index.html').read_text(encoding='utf-8')
code = page[page.index('let comfortBusy ='):page.index('async function pollComfort()')]
setup = r'''
const assert = require('node:assert/strict');
const elements = new Map();
const $ = id => {
if (!elements.has(id)) elements.set(id, {textContent:'', hidden:true, disabled:false, type: 'number'});
return elements.get(id);
};
let denied = 0;
const window = {frameApp:{notify:async()=>{denied++;throw Error('permission denied');}}};
const log = ()=>{}, toast = ()=>{};
'''
checks = r'''
(async()=>{
const active = {id:'session-one',active:true,time:100,remaining:120,
options:{minutes:2,breakMinutes:1,stillMinutes:1,batteryAlert:true,heatAlert:true},
events:[{id:'event-one',kind:'battery',time:99,message:'Low battery'}]};
renderComfort(active); // initial history must not replay even a fresh event
assert.equal(denied,0);
assert.equal($('comfortAnnouncement').textContent,'');
active.events.push({id:'event-two',kind:'break',time:100,message:'Take a break'});
renderComfort(active);
await new Promise(resolve=>setImmediate(resolve));
assert.equal(denied,1);
assert.equal($('comfortAnnouncement').textContent,'Take a break');
assert.equal($('comfortNotificationStatus').hidden,false);
assert.match($('comfortNotificationStatus').textContent,/notification settings/);
renderComfort({...active,time:105}); // the next normal poll must not erase failure
assert.equal($('comfortNotificationStatus').hidden,false);
assert.equal($('sessionStart').disabled,true);
assert.equal($('sessionMinutes').disabled,true);
assert.equal($('sessionCancel').disabled,false);
let requests=0;
window.frameApp.notify=async()=>{requests++;};
renderComfort({...active,time:106});
assert.equal($('comfortAnnouncement').textContent,'Take a break');
assert.equal(requests,0); // polling does not replay an already-seen event
await localNotification('test',true);
assert.equal($('comfortNotificationStatus').hidden,true);
renderComfort({...active,active:false});
assert.equal($('sessionStart').disabled,false);
assert.equal($('sessionMinutes').disabled,false);
assert.equal($('sessionCancel').disabled,true);
})().catch(e=>{console.error(e);process.exitCode=1;});
'''
result = subprocess.run(['node', '-e', setup + code + checks], capture_output=True, text=True)
self.assertEqual(result.returncode, 0, result.stderr)
+141
View File
@@ -0,0 +1,141 @@
"""Owned media planning, eye isolation, decoder choice and fake-Frame ownership."""
import json
import os
from pathlib import Path
import struct
import sys
import tempfile
import unittest
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
import frame_media as media
import frame_media_player as player
import frame_media_remote as remote
import frame_splat as splat
import server
class Media(unittest.TestCase):
def test_layout_evidence_and_override(self):
for name, layout in [('film_SBS.mp4', 'sbs'), ('film.OU.mkv', 'ou'),
('film_FSBS.mp4', 'full-sbs'), ('photo_TB.png', 'ou')]:
self.assertEqual(media.plan(name)['layout'], layout)
self.assertEqual(media.plan('film_SBS_OU.mp4', 'mono')['source'], 'explicit')
self.assertEqual(media.plan('film.mkv', metadata={'stereo_mode': 'top_bottom'})['layout'], 'full-ou')
for name in ('film.mp4', 'film_SBS_OU.mp4', 'businessbs.mp4'):
with self.assertRaises(ValueError):
media.plan(name)
with self.assertRaises(ValueError):
media.plan('film.mkv', metadata={'stereo_mode': 'right_left'})
def test_unsupported_containers_do_not_flatten_spatial_photos(self):
for name in ('spatial.HEIC', 'stereo.mpo', 'cloud.ply', 'cloud.spz', 'app.exe'):
with self.assertRaises(ValueError):
media.plan(name, 'sbs')
def test_ou_pixels_keep_each_eye_and_row(self):
a, b, c, d = [bytes([n])*8 for n in (1, 2, 3, 4)]
data, width, height = media.stereo_pixels(a+b+c+d, 2, 4, 'ou')
self.assertEqual((data, width, height), (a+c+b+d, 4, 2))
with self.assertRaises(ValueError):
media.stereo_pixels(b'bad', 2, 4, 'sbs')
self.assertEqual(media.geometry(3840, 2160, 'sbs'), (1920, 1080, 2))
self.assertEqual(media.geometry(1920, 1080, 'ou'), (1920, 1080, .5))
def test_decode_is_hardware_and_one_clock_for_audio(self):
for codec, decoder in [('h264', 'h264_v4l2m2m'), ('hevc', 'hevc_v4l2m2m')]:
cmd = player.decoder_command(Path('/tmp/a file.mp4'), {'codec_name': codec}, 1280, 720, True)
self.assertIn(decoder, cmd)
self.assertIn('-re', cmd)
self.assertIn('pulse', cmd)
self.assertIn(str(Path('/tmp/a file.mp4')), cmd)
with self.assertRaises(ValueError):
player.decoder_command(Path('x.webm'), {'codec_name': 'vp9'}, 640, 480, False)
cmd = player.decoder_command(Path('x.png'), {'codec_name': 'png'}, 640, 480, False, True)
self.assertNotIn('-re', cmd)
self.assertIn('-frames:v', cmd)
def test_fake_frame_library_and_traversal(self):
with tempfile.TemporaryDirectory() as d, patch.object(remote, 'ROOT', Path(d)):
identity = 'a'*32+'/space and quote\'.png'
path = Path(d)/identity
path.parent.mkdir()
path.write_bytes(b'test')
self.assertEqual(remote.media_path(identity), path.resolve())
for bad in ('../../etc/passwd', '/etc/passwd', 'a'*32+'/..', 'a'*32+'/x/y', None):
with self.assertRaises((ValueError, FileNotFoundError)):
remote.media_path(bad)
link = path.parent/'link.png'
link.symlink_to(path)
with self.assertRaises(ValueError):
remote.media_path('a'*32+'/link.png')
with patch.object(remote, 'status', return_value={'state': 'idle'}):
files = remote.run({'action': 'list'})['files']
self.assertEqual([f['id'] for f in files], [identity])
def test_server_rejects_bad_actions_before_ssh(self):
with patch.object(server, 'ssh') as ssh:
for body in ({'action': 'delete'}, {'action': 'play', 'id': '../x'},
{'action': 'play', 'id': 'a'*32+'/x', 'layout': 'invalid'},
{'action': 'play', 'id': 'a'*32+'/x', 'theatre': 'false'}):
with self.assertRaises(server.Failure):
server.media(body)
ssh.assert_not_called()
def test_fake_frame_stop_only_owns_our_unit(self):
for rc in (0, 5): # 5: already collected ("not loaded"), a no-op
with patch.object(remote.subprocess, 'run') as run, patch.object(remote, 'status', return_value={'state': 'ended'}):
run.return_value.returncode = rc
self.assertEqual(remote.run({'action': 'stop'})['state'], 'ended')
self.assertEqual(run.call_args.args[0], ['systemctl', '--user', 'stop', 'frame-control-media.service'])
with patch.object(remote.subprocess, 'run') as run, patch.object(remote, 'status', return_value={}):
run.return_value.returncode, run.return_value.stderr = 1, 'Access denied'
with self.assertRaisesRegex(RuntimeError, 'Access denied'):
remote.run({'action': 'stop'})
def test_start_failure_reports_systemd_error(self):
with tempfile.TemporaryDirectory() as d, patch.object(remote, 'ROOT', Path(d)), \
patch.object(remote, 'STATUS', Path(d)/'status.json'), \
patch.object(remote, 'active', return_value=False), \
patch.object(remote.subprocess, 'run') as run:
identity = 'b'*32+'/still_SBS.png'
(Path(d)/identity).parent.mkdir()
(Path(d)/identity).write_bytes(b'x')
run.return_value.returncode, run.return_value.stderr = 1, 'Unit already exists'
with patch.object(remote, 'probe', return_value=({}, False)), \
self.assertRaisesRegex(RuntimeError, 'Unit already exists'):
remote.run({'action': 'play', 'id': identity})
self.assertEqual(run.call_args.args[0][0], 'systemd-run') # reset-failed's result is ignored
def test_upload_rejects_unplayable_names_and_keeps_copy_error(self):
with patch.object(server, 'ssh') as ssh, patch.object(server, 'push_file') as push:
# On Windows a backslash is a separator, so such a name can't reach here.
for name in ('.hidden.mp4',) + (('a\\b_SBS.mp4',) if os.sep == '/' else ()):
with self.assertRaises(server.Failure):
server.push_media(Path('/tmp')/name)
ssh.assert_not_called()
push.side_effect = server.Failure('copy failed')
ssh.side_effect = [None, server.Failure('link down')]
with self.assertRaisesRegex(server.Failure, 'copy failed'):
server.push_media(Path('/tmp/film_SBS.mp4'))
def test_splat_invalid_records_and_stereo_parallax(self):
with tempfile.TemporaryDirectory() as d:
path = Path(d)/'small.splat'
path.write_bytes(struct.pack('<6f8B', 0, 0, 0, .001, .001, .001,
255, 0, 0, 255, 255, 128, 128, 128))
data, w, h = splat.render(path, 64, 48)
self.assertEqual((len(data), w, h), (128*48*4, 128, 48))
def centroid(eye):
weights = [(x, data[(y*w+x+eye*64)*4]) for y in range(h) for x in range(64)]
return sum(x*v for x,v in weights)/sum(v for x,v in weights)
self.assertGreater(centroid(0), centroid(1))
for bad in (b'', b'bad', struct.pack('<6f8B', float('nan'), 0, 0, 1, 1, 1, *([128]*8))):
path.write_bytes(bad)
with self.assertRaises(ValueError):
splat.read(path)
if __name__ == '__main__':
unittest.main()
+3
View File
@@ -84,6 +84,7 @@ class ServerGuards(unittest.TestCase):
def test_api_needs_custom_header(self):
# <img src> and plain form posts from other sites can't set it.
self.assertEqual(self.request("POST", "/api/comfort", {"action": "start"})[0], 403)
self.assertEqual(self.request("GET", "/api/status")[0], 403)
self.assertEqual(self.request("GET", "/api/screenshot?view=headset")[0], 403)
self.assertEqual(self.request("GET", "/api/shots")[0], 403)
@@ -99,6 +100,8 @@ class ServerGuards(unittest.TestCase):
def test_input_validation(self):
cases = [
("/api/comfort", {"action": "poweroff"}),
("/api/comfort", {"action": "start", "minutes": 0}),
("/api/launch", {"appid": "620; rm -rf ~"}),
("/api/launch", {"appid": ""}),
("/api/flatpak", {"id": "org.example.App;id", "action": "install"}),
+264
View File
@@ -0,0 +1,264 @@
"""Opt-in session worker ON the Frame; no root, extra apps, or power actions.
One worker per user, shared by desktop and phone. State survives companion
connections, not headset reboots. See docs/family-comfort.md for guarantees.
"""
import contextlib
import json
import os
from pathlib import Path
import subprocess
import sys
import time
import uuid
from frame_steam import Page
from frame_status import battery, thermal_alerts, activity_level
ROOT = Path.home() / '.local/state/frame-control/comfort'
VRCMD = '/opt/steamvr/bin/linuxarm64/vrcmd'
HOME_JS = """(async () => {
SteamUIStore.Navigate('/library/home');
await SteamClient.OpenVR.VROverlay.ShowDashboard('valve.steam.gamepadui.main');
if (!await SteamClient.OpenVR.VROverlay.IsDashboardVisible()) throw Error('Steam dashboard did not open');
return {path: location.pathname};
})()"""
def clock():
# CLOCK_BOOTTIME includes headset suspend; wall-clock corrections don't alter limits.
return time.clock_gettime(time.CLOCK_BOOTTIME)
def boot():
return Path('/proc/sys/kernel/random/boot_id').read_text().strip()
def validate(body):
if not isinstance(body, dict) or body.get('action') not in ('status', 'start', 'cancel'):
raise ValueError('Choose status, start or cancel')
if body['action'] == 'start':
for key, low, high in (('minutes', 1, 240), ('breakMinutes', 0, 120), ('stillMinutes', 0, 240)):
n = body.get(key)
if type(n) is not int or not low <= n <= high:
raise ValueError(f'{key} must be a whole number from {low} to {high}')
for key in ('batteryAlert', 'heatAlert'):
if type(body.get(key)) is not bool:
raise ValueError(f'{key} must be true or false')
return body
def new_session(body, now, boot_id):
return {'id': uuid.uuid4().hex, 'boot': boot_id, 'active': True,
'options': {k: body[k] for k in ('minutes', 'breakMinutes', 'stillMinutes', 'batteryAlert', 'heatAlert')},
'started': now, 'deadline': now + body['minutes'] * 60, 'lastSample': now,
'used': 0, 'nextBreak': body['breakMinutes'] * 60, 'stillSent': False,
'warned': None, 'events': [], 'seq': 0, 'latched': [], 'error': None}
def event(s, kind, message):
s['seq'] += 1
s['events'].append({'id': s['id'] + ':' + str(s['seq']), 'kind': kind,
'message': message, 'time': time.time()})
s['events'] = s['events'][-40:]
def notify(message):
r = subprocess.run([VRCMD, '--notify', 'Frame Control: ' + message],
capture_output=True, text=True, timeout=20)
if r.returncode or 'succeeded' not in r.stdout:
raise RuntimeError('SteamVR could not show the reminder: ' + (r.stderr or r.stdout)[-300:])
def home():
# A total process deadline also bounds a CDP peer that keeps sending events
# without completing the request. Keep cancellation ordered after this action.
r = subprocess.run([sys.executable, str(Path(__file__).resolve()), '--home'],
capture_output=True, text=True, timeout=15)
if r.returncode:
raise RuntimeError('Steam Home failed: ' + (r.stdout or r.stderr)[-300:])
def open_home():
page = Page()
try:
result = page.eval(HOME_JS)
if result.get('path') != '/routes/library/home':
raise RuntimeError('Steam did not navigate Home')
finally:
page.sock.close()
def tick(s, now, sample, warn=notify, go_home=home, read_clock=clock):
"""One deterministic step; injected actions/samples also exercise a fake Frame."""
if not s.get('active'):
return
o = s['options']
s['heartbeat'] = now
delta = max(0, min(30, now - s['lastSample']))
s['lastSample'] = now
level = sample.get('activity')
b = sample.get('battery') or {}
s['unavailable'] = []
if o['batteryAlert'] and b.get('percent') is None:
s['unavailable'].append('battery')
if o['heatAlert'] and sample.get('thermal') is None:
s['unavailable'].append('temperature')
if (o['breakMinutes'] or o['stillMinutes']) and level is None:
s['unavailable'].append('activity')
s['activity'] = level
if level in (1, 2):
s['used'] += delta
elif level is not None:
s['used'] = 0
s['nextBreak'] = o['breakMinutes'] * 60
s['stillSent'] = False
# Missing samples never count as time worn. No catch-up burst after a disconnect.
if now >= s['deadline'] - 60 and s['warned'] is None:
warn('One minute left. Save your progress; Steam Home will open.')
s['warned'] = max(now, read_clock())
event(s, 'warning', 'One minute left. Save your progress; Steam Home will open.')
if s['warned'] is not None and now >= max(s['deadline'], s['warned'] + 60):
go_home()
s['active'] = False
event(s, 'finished', 'Session ended: Steam Home opened. Your game is still running.')
return
if o['breakMinutes'] and s['used'] >= s['nextBreak']:
warn('Time for a break. Take off the headset and rest your eyes.')
event(s, 'break', 'Time for a break. Take off the headset and rest your eyes.')
s['nextBreak'] = s['used'] + o['breakMinutes'] * 60
if o['stillMinutes'] and not s['stillSent'] and s['used'] >= o['stillMinutes'] * 60:
event(s, 'still', f"Headset still active after {o['stillMinutes']} active minute(s). Check in with the wearer.")
s['stillSent'] = True
low = b.get('percent') is not None and b['percent'] <= 15 and b.get('status') == 'Discharging'
hot = sample.get('thermal')
for kind, enabled, value, message in (
('battery', o['batteryAlert'], low if b else None, 'Frame battery is low (15% or less).'),
('heat', o['heatAlert'], bool(hot) if hot is not None else None,
'Frame reports a hot/critical thermal trip or battery overheat. Ask the wearer to take a break.')):
if enabled and value and kind not in s['latched']:
event(s, kind, message)
s['latched'].append(kind)
elif value is False and kind in s['latched']:
# Battery hysteresis prevents repeated alerts around 15%.
if kind != 'battery' or b.get('status') == 'Charging' or (b.get('percent') or 0) >= 20:
s['latched'].remove(kind)
@contextlib.contextmanager
def locked(name='state.lock', nonblocking=False):
import fcntl # only needed ON the Linux headset, not by desktop validation/tests
ROOT.mkdir(parents=True, exist_ok=True, mode=0o700)
with (ROOT / name).open('a') as f:
fcntl.flock(f, fcntl.LOCK_EX | (fcntl.LOCK_NB if nonblocking else 0))
yield f
def read_state():
try:
state = json.loads((ROOT / 'session.json').read_text())
if not isinstance(state, dict):
raise ValueError('Saved session must be an object')
return state
except FileNotFoundError:
return {'active': False, 'events': []}
except (ValueError, UnicodeDecodeError):
# Preserve the unreadable state for diagnosis, then allow a new session.
(ROOT / 'session.json').replace(ROOT / ('session-unreadable-' + uuid.uuid4().hex + '.json'))
return {'active': False, 'events': [],
'error': 'Saved session was unreadable. Start a new session.'}
def save(s):
p = ROOT / 'session.tmp'
p.write_text(json.dumps(s))
p.chmod(0o600)
p.replace(ROOT / 'session.json')
def current(s, now):
if s.get('active') and s.get('boot') != boot():
s['active'] = False
s['error'] = 'Headset restarted. Start a new session.'
out = dict(s)
out['time'] = time.time() # event age uses the Frame's clock, not the phone's
out['remaining'] = max(0, max(s.get('deadline', now), (s.get('warned') or 0) + 60) - now) if s.get('active') else 0
beat = s.get('heartbeat', s.get('started', now)) # a hand-edited state may lack either
if s.get('active') and now - beat > 90:
out['error'] = 'Session worker is not responding. Timer enforcement is unverified; cancel and start again.'
return out
def watch():
try:
with locked('worker.lock', nonblocking=True) as worker:
while True:
with locked():
s = current(read_state(), clock())
if not s.get('active'):
save(s)
# Release ownership before state.lock: a concurrent start
# cannot miss the gap between an old worker and its exit.
import fcntl
fcntl.flock(worker, fcntl.LOCK_UN)
return
try:
b = battery()
hot = thermal_alerts()
if b and b.get('health') == 'Overheat':
hot = (hot or []) + ['battery']
tick(s, clock(), {'battery': b, 'thermal': hot, 'activity': activity_level()})
s['error'] = None
except Exception as e:
error = str(e)
if s.get('error') != error:
event(s, 'error', 'Session action failed: ' + error)
s['error'] = error
save(s)
time.sleep(5)
except BlockingIOError:
pass # another connection already started the single worker
def command(body):
validate(body)
with locked():
s = current(read_state(), clock())
if body['action'] == 'start':
if s.get('active'):
raise ValueError('A session is already running. Cancel it before starting another.')
s = new_session(body, clock(), boot())
event(s, 'started', 'Session started. Steam Home opens at the limit; games are not closed.')
elif body['action'] == 'cancel':
s['active'] = False
s['error'] = None
if s.get('id'):
event(s, 'cancelled', 'Session timer and monitoring cancelled.')
save(s)
if body['action'] == 'start':
try:
subprocess.Popen([sys.executable, str(Path(__file__).resolve()), '--watch'],
stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
start_new_session=True, close_fds=True)
except OSError as e:
s['active'] = False
s['error'] = 'Could not start session worker: ' + str(e)
event(s, 'error', s['error'])
save(s)
raise
return current(s, clock())
if __name__ == '__main__':
if sys.argv[1:] == ['--watch']:
watch()
else:
try:
if sys.argv[1:] == ['--home']:
open_home()
print(json.dumps({'home': True}))
else:
print(json.dumps(command(json.loads(sys.argv[1]))))
except Exception as e:
print(json.dumps({'error': str(e)}))
sys.exit(1)
+67
View File
@@ -0,0 +1,67 @@
"""Media planning shared by Frame Control and its own Frame-side player.
No viewer dependencies. Filename hints are suggestions, never guesses from
resolution. Explicit layout wins; conflicting hints require a choice.
"""
import re
from pathlib import Path
LAYOUTS = ('auto', 'mono', 'sbs', 'ou', 'full-sbs', 'full-ou')
VIDEO = {'.mp4', '.mkv', '.mov', '.webm', '.m4v'}
PHOTO = {'.png', '.jpg', '.jpeg'}
def plan(name, layout='auto', metadata=None):
if layout not in LAYOUTS:
raise ValueError('Choose auto, mono, sbs, ou, full-sbs or full-ou')
suffix = Path(name).suffix.lower()
if suffix in {'.heic', '.heif', '.avif', '.mpo'}:
raise ValueError('Native spatial-photo containers are not supported yet; export both eyes as SBS or OU PNG/JPEG')
if suffix == '.splat':
return {'kind': 'splat', 'layout': 'sbs', 'source': 'renderer'}
if suffix not in VIDEO | PHOTO:
raise ValueError('Use MP4/MKV/MOV/WebM video, PNG/JPEG stereo photos, or a .splat file')
source = 'explicit'
if layout == 'auto':
tokens = set(re.split(r'[^a-z0-9]+', Path(name).stem.lower()))
hints = set()
for value, tags in [('full-sbs', {'fsbs'}), ('full-ou', {'fou', 'ftb'}),
('sbs', {'sbs', 'hsbs', 'lr'}), ('ou', {'ou', 'hou', 'tb', 'htb'})]:
if tokens & tags:
hints.add(value)
if len(hints) > 1:
raise ValueError('Conflicting stereo filename tags; choose the layout explicitly')
layout = next(iter(hints), None)
source = 'filename'
if not layout:
# Matroska StereoMode/FFmpeg stereo_mode: only known left-first modes.
mode = (metadata or {}).get('stereo_mode')
layout = {'left_right': 'full-sbs', 'top_bottom': 'full-ou', 'mono': 'mono'}.get(mode)
source = 'metadata'
if mode and layout is None:
raise ValueError('Unsupported stereo metadata; choose the eye order/layout explicitly')
if not layout:
raise ValueError('No stereo layout found; choose mono, SBS or OU (left/top eye first)')
return {'kind': 'video' if suffix in VIDEO else 'photo', 'layout': layout, 'source': source}
def geometry(width, height, layout):
"""Bound transfer to 1920x1080; return packed dimensions and texel aspect."""
if not 0 < width <= 32768 or not 0 < height <= 32768:
raise ValueError('Invalid media dimensions')
if layout not in LAYOUTS[1:]:
raise ValueError('Resolve the layout before playback')
scale = min(1, 1920 / width, 1080 / height)
w, h = max(2, int(width * scale) // 2 * 2), max(2, int(height * scale) // 2 * 2)
return w, h, {'mono': 1, 'sbs': 2, 'ou': .5, 'full-sbs': 1, 'full-ou': 1}[layout]
def stereo_pixels(data, width, height, layout):
"""Normalize top/bottom to OpenVR's left/right texture; preserve eye order."""
if len(data) != width * height * 4:
raise ValueError('Incomplete RGBA frame')
if layout not in ('ou', 'full-ou'):
return data, width, height
stride, half = width * 4, height // 2
return b''.join(data[y*stride:(y+1)*stride] +
data[(y+half)*stride:(y+half+1)*stride] for y in range(half)), width*2, half
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""Send local media to Frame Control's own OpenVR player."""
import argparse
import json
from pathlib import Path
import sys
sys.path.insert(0, str(Path(__file__).resolve().parent))
import frame_media
import server
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument('files', nargs='*', type=Path)
ap.add_argument('--launch', action='store_true', help='play the one file being sent')
ap.add_argument('--layout', choices=frame_media.LAYOUTS, default='auto')
ap.add_argument('--theatre', action='store_true', help='bigger screen and dark surround')
ap.add_argument('--list', action='store_true')
ap.add_argument('--stop', action='store_true')
args = ap.parse_args()
if args.launch and len(args.files) != 1:
ap.error('--launch needs exactly one file')
if not args.files and not (args.list or args.stop):
ap.error('choose files, --list or --stop')
for path in args.files:
if not path.is_file():
ap.error('not a file: %s' % path)
frame_media.plan(path.name, 'mono')
if args.stop:
print(json.dumps(server.media({'action': 'stop'})))
for path in args.files:
result = server.push_media(path.resolve())
print(json.dumps(result))
if args.launch:
print(json.dumps(server.media({'action': 'play', 'id': result['id'],
'layout': args.layout, 'theatre': args.theatre})))
if args.list:
print(json.dumps(server.media({'action': 'list'})))
if __name__ == '__main__':
try:
main()
except (ValueError, server.Failure) as e:
sys.exit(str(e))
+213
View File
@@ -0,0 +1,213 @@
#!/usr/bin/env python3
"""Frame Control's local-media OpenVR player. Runs on the Frame, no third-party app.
SteamOS ffmpeg does hardware video decoding, scaling and audio output. OpenVR
owns only our screen and optional black surround. Exiting destroys both.
"""
import argparse
import ctypes as C
import json
import os
from pathlib import Path
import signal
import subprocess
import time
import frame_media
import frame_splat
LIB = '/opt/steamvr/bin/linuxarm64/libopenvr_api.so'
H = C.c_uint64
# Slots from Valve's openvr_capi.h, IVROverlay_028. Fail closed on another ABI.
SLOTS = {
'CreateOverlay': (1, [C.c_char_p, C.c_char_p, C.POINTER(H)]),
'DestroyOverlay': (3, [H]),
'SetOverlayFlag': (11, [H, C.c_int, C.c_bool]),
'SetOverlayAlpha': (16, [H, C.c_float]),
'SetOverlayTexelAspect': (18, [H, C.c_float]),
'SetOverlaySortOrder': (20, [H, C.c_uint32]),
'SetOverlayWidthInMeters': (22, [H, C.c_float]),
'SetOverlayTransformTrackedDeviceRelative': (35, [H, C.c_uint32, C.c_void_p]),
'ShowOverlay': (43, [H]),
'SetOverlayRaw': (62, [H, C.c_void_p, C.c_uint32, C.c_uint32, C.c_uint32]),
}
class Overlay:
def __init__(self):
self.handles = []
self.vr = C.CDLL(LIB)
self.vr.VR_InitInternal2.argtypes = [C.POINTER(C.c_int), C.c_int, C.c_char_p]
self.vr.VR_GetGenericInterface.argtypes = [C.c_char_p, C.POINTER(C.c_int)]
self.vr.VR_GetGenericInterface.restype = C.c_void_p
err = C.c_int()
self.vr.VR_InitInternal2(C.byref(err), 2, None)
if err.value:
raise RuntimeError('SteamVR init failed: %s' % err.value)
ptr = self.vr.VR_GetGenericInterface(b'FnTable:IVROverlay_028', C.byref(err))
if not ptr or err.value:
self.vr.VR_ShutdownInternal()
raise RuntimeError('SteamVR needs IVROverlay_028: %s' % err.value)
self.table = C.cast(ptr, C.POINTER(C.c_void_p))
def call(self, name, *values):
slot, args = SLOTS[name]
rc = C.CFUNCTYPE(C.c_int, *args)(self.table[slot])(*values)
if rc:
raise RuntimeError('OpenVR %s failed: %s' % (name, rc))
def create(self, key, width, distance, stereo=False, aspect=1, order=1):
handle = H()
self.call('CreateOverlay', key.encode(), b'Frame Control media', C.byref(handle))
self.handles.append(handle)
self.call('SetOverlayWidthInMeters', handle, width)
self.call('SetOverlaySortOrder', handle, order)
self.call('SetOverlayTexelAspect', handle, aspect)
if stereo:
self.call('SetOverlayFlag', handle, 1024, True) # SideBySide_Parallel
matrix = (C.c_float * 12)(1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, -distance)
self.call('SetOverlayTransformTrackedDeviceRelative', handle, 0, matrix)
return handle
def pixels(self, handle, data, width, height):
buf = C.create_string_buffer(data)
self.call('SetOverlayRaw', handle, buf, width, height, 4)
self.call('ShowOverlay', handle)
def close(self):
try:
for h in reversed(self.handles):
self.call('DestroyOverlay', h)
finally:
self.vr.VR_ShutdownInternal()
def probe(path):
result = subprocess.run(['ffprobe', '-v', 'error', '-show_streams', '-of', 'json', str(path)],
capture_output=True, text=True, timeout=30)
if result.returncode:
raise ValueError(result.stderr[-2000:] or 'Cannot read media')
streams = json.loads(result.stdout)['streams']
video = next((s for s in streams if s['codec_type'] == 'video'), None)
if not video:
raise ValueError('No image or video stream')
return video, any(s['codec_type'] == 'audio' for s in streams)
def decoder_command(path, info, width, height, audio, photo=False):
cmd = ['ffmpeg', '-nostdin', '-hide_banner', '-loglevel', 'error']
if not photo:
cmd += ['-re', '-readrate_initial_burst', '0']
codec = {'h264': 'h264_v4l2m2m', 'hevc': 'hevc_v4l2m2m'}.get(info['codec_name'])
if not codec:
raise ValueError('Hardware playback currently supports H.264 and H.265 only')
cmd += ['-c:v', codec]
cmd += ['-i', str(path), '-map', '0:v:0', '-vf', 'scale=%s:%s' % (width, height),
'-pix_fmt', 'rgba']
if photo:
cmd += ['-frames:v', '1']
else:
cmd += ['-r', '30']
cmd += ['-f', 'rawvideo', 'pipe:1']
if audio and not photo:
cmd += ['-map', '0:a:0', '-f', 'pulse', 'Frame Control Media']
return cmd
def write_status(path, **values):
tmp = path.with_suffix('.tmp')
tmp.write_text(json.dumps(values))
tmp.replace(path)
def play(args):
path = Path(args.file).resolve(strict=True)
status = Path(args.status)
splat = path.suffix.lower() == '.splat'
if splat:
data, width, height = frame_splat.render(path)
plan = frame_media.plan(path.name)
aspect, photo, command = 1, True, None
else:
info, audio = probe(path)
plan = frame_media.plan(path.name, args.layout, info.get('tags'))
width, height, aspect = frame_media.geometry(info['width'], info['height'], plan['layout'])
photo = plan['kind'] == 'photo'
command = decoder_command(path, info, width, height, audio, photo)
vr, proc, frames, started = None, None, 0, time.monotonic()
# systemd sends SIGTERM to the whole unit, including ffmpeg. Python unwinds
# ownership; no unrelated Steam/SteamVR process or setting is touched.
def stop(signum, frame):
raise InterruptedError('Stopped')
signal.signal(signal.SIGTERM, stop)
signal.signal(signal.SIGINT, stop)
try:
vr = Overlay()
if args.theatre:
surround = vr.create('framecontrol.media.surround', 40, 4, order=0)
vr.call('SetOverlayAlpha', surround, .85)
vr.pixels(surround, b'\x00\x00\x00\xff', 1, 1)
screen = vr.create('framecontrol.media.screen', 3 if args.theatre else 1.6, 2,
plan['layout'] != 'mono', aspect)
if splat:
vr.pixels(screen, data, width, height)
write_status(status, state='playing', file=path.name, frames=1, **plan)
while True:
time.sleep(1)
proc = subprocess.Popen(command, stdout=subprocess.PIPE)
video_start = time.monotonic()
while True:
data = proc.stdout.read(width * height * 4)
if not data:
break
data, outw, outh = frame_media.stereo_pixels(data, width, height, plan['layout'])
if not photo:
time.sleep(max(0, video_start + frames/30 - time.monotonic()))
vr.pixels(screen, data, outw, outh)
frames += 1
if frames == 1 or frames % 30 == 0:
write_status(status, state='playing', file=path.name, frames=frames,
seconds=time.monotonic()-started, **plan)
if not photo:
time.sleep(max(0, video_start + frames/30 - time.monotonic()))
rc = proc.wait(timeout=10)
if rc:
raise RuntimeError('ffmpeg exited %s; see media log' % rc)
if not frames:
raise RuntimeError('Decoder produced no frames')
if photo:
while True:
time.sleep(1)
write_status(status, state='ended', frames=frames, seconds=time.monotonic()-started)
except InterruptedError:
write_status(status, state='stopped', frames=frames)
finally:
if proc:
if proc.poll() is None:
proc.terminate()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait()
proc.stdout.close()
if vr:
vr.close()
def main():
ap = argparse.ArgumentParser(description=__doc__)
ap.add_argument('file')
ap.add_argument('--layout', choices=frame_media.LAYOUTS, default='auto')
ap.add_argument('--theatre', action='store_true')
ap.add_argument('--status', required=True)
args = ap.parse_args()
try:
play(args)
except Exception as e:
write_status(Path(args.status), state='error', error=str(e))
raise
if __name__ == '__main__':
main()
+109
View File
@@ -0,0 +1,109 @@
"""Frame-side library and process ownership for Frame Control media.
Only the dedicated systemd user unit is controlled. No SteamVR settings change.
"""
import argparse
import json
from pathlib import Path
import re
import subprocess
import sys
import frame_media
from frame_media_player import probe
ROOT = Path.home() / 'Videos' / 'FrameControl'
RUNTIME = Path.home() / '.local' / 'share' / 'frame-control' / 'media'
UNIT = 'frame-control-media.service'
STATUS = RUNTIME / 'status.json'
def media_path(identity):
if not isinstance(identity, str) or '\\' in identity or '\x00' in identity:
raise ValueError('Invalid media id')
parts = Path(identity).parts
if len(parts) != 2 or not re.fullmatch('[0-9a-f]{32}', parts[0]) or parts[1].startswith('.'):
raise ValueError('Invalid media id')
candidate = ROOT / identity
if candidate.is_symlink() or candidate.parent.is_symlink():
raise ValueError('Media links are not supported')
path = candidate.resolve(strict=True)
if not path.is_file() or ROOT.resolve() not in path.parents:
raise ValueError('Media file is outside the library')
return path
def active():
return subprocess.run(['systemctl', '--user', 'is-active', '--quiet', UNIT]).returncode == 0
def status():
running = active()
try:
state = json.loads(STATUS.read_text())
except (OSError, ValueError):
state = {'state': 'idle'}
if not running and state.get('state') in ('playing', 'starting', 'paused'):
state = {'state': 'stopped', 'message': 'Player exited; check the media log if this was unexpected'}
return dict(state, running=running)
def run(body):
action = body.get('action')
if action == 'list':
files = []
if ROOT.exists():
for folder in sorted(ROOT.iterdir()):
if not re.fullmatch('[0-9a-f]{32}', folder.name) or not folder.is_dir() or folder.is_symlink():
continue
for path in sorted(folder.iterdir()):
if path.is_file() and not path.is_symlink() and not path.name.startswith('.'):
files.append({'id': folder.name+'/'+path.name, 'name': path.name, 'bytes': path.stat().st_size})
return {'files': files, 'player': status()}
if action == 'status':
return status()
if action == 'stop':
# --collect unloads the unit after it exits; systemctl then exits 5
# ("not loaded", verified on the Frame). That's a finished player, not an error.
stopped = subprocess.run(['systemctl', '--user', 'stop', UNIT], capture_output=True, text=True, timeout=15)
if stopped.returncode not in (0, 5):
raise RuntimeError('Could not stop the media player: ' + (stopped.stderr.strip() or 'exit %s' % stopped.returncode))
return {'message': 'Media player stopped', **status()}
if action != 'play':
raise ValueError('Media action must be list, status, play or stop')
path = media_path(body.get('id'))
if type(body.get('theatre', False)) is not bool:
raise ValueError('theatre must be true or false')
info = {} if path.suffix.lower() == '.splat' else probe(path)[0]
plan = frame_media.plan(path.name, body.get('layout', 'auto'), info.get('tags'))
if active():
raise ValueError('Stop the current media before starting another file')
# systemd owns the process group and refuses a concurrent start of this name.
# The runtime cap also cleans up if the controlling computer disconnects.
subprocess.run(['systemctl', '--user', 'reset-failed', UNIT], stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, timeout=10)
STATUS.write_text(json.dumps({'state': 'starting', 'file': path.name}))
command = ['systemd-run', '--user', '--quiet', '--collect', '--unit='+UNIT,
'--property=RuntimeMaxSec=14400', '--property=TimeoutStopSec=8',
'--property=StandardOutput=append:'+str(RUNTIME/'player.log'),
'--property=StandardError=append:'+str(RUNTIME/'player.log'),
'python3', str(RUNTIME/'frame_media_player.py'), str(path),
'--layout', plan['layout'], '--status', str(STATUS)]
if body.get('theatre'):
command.append('--theatre')
started = subprocess.run(command, capture_output=True, text=True, timeout=15)
if started.returncode:
raise RuntimeError('Could not start the media player: ' + (started.stderr.strip() or 'systemd-run exited %s' % started.returncode))
return {'message': 'Starting Frame Control media', 'plan': plan}
def main():
try:
print(json.dumps(run(json.load(sys.stdin))))
except Exception as e:
print(json.dumps({'error': str(e)}))
sys.exit(1)
if __name__ == '__main__':
main()
+83
View File
@@ -0,0 +1,83 @@
"""Small, bounded CPU Gaussian-splat preview renderer (Frame Control-owned).
Reads the common 32-byte .splat record: position/scale float32 triplets,
RGBA bytes, then normalized quaternion bytes (wxyz). Two perspective cameras,
projected 3D covariance, back-to-front alpha compositing. This is a stationary
stereo preview, not a six-degree-of-freedom scene or a large-scene renderer.
"""
import math
from pathlib import Path
import struct
MAX_SPLATS = 20000
RECORD = struct.Struct('<6f8B')
def read(path):
size = Path(path).stat().st_size
if not size or size % RECORD.size or size > MAX_SPLATS * RECORD.size:
raise ValueError('Use a 32-byte .splat file with 1–20,000 Gaussians; PLY/SPZ and larger scenes are not supported yet')
values = []
with open(path, 'rb') as stream:
for row in RECORD.iter_unpack(stream.read(MAX_SPLATS * RECORD.size + 1)):
xyz, scales = row[:3], row[3:6]
if not all(math.isfinite(v) and abs(v) <= 1e6 for v in row[:6]) or min(scales) <= 0:
raise ValueError('Invalid splat position or scale')
q = [(v - 128) / 128 for v in row[10:14]]
length = math.sqrt(sum(v*v for v in q))
if length < .01:
raise ValueError('Invalid splat quaternion')
w, x, y, z = [v / length for v in q]
rotation = ((1-2*(y*y+z*z), 2*(x*y-z*w), 2*(x*z+y*w)),
(2*(x*y+z*w), 1-2*(x*x+z*z), 2*(y*z-x*w)),
(2*(x*z-y*w), 2*(y*z+x*w), 1-2*(x*x+y*y)))
cov = [[sum(rotation[i][k]*rotation[j][k]*scales[k]**2 for k in range(3))
for j in range(3)] for i in range(3)]
values.append((xyz, cov, row[6:10]))
return values
def render(path, width=320, height=240):
values = read(path)
lo = [min(p[0][i] for p in values) for i in range(3)]
hi = [max(p[0][i] for p in values) for i in range(3)]
center = [(a+b)/2 for a, b in zip(lo, hi)]
radius = max(max(b-a for a, b in zip(lo, hi))/2, .01)
# Normalize captures to a two-metre box. Source units are not assumed metres.
normalized = [([(xyz[i]-center[i])/radius for i in range(3)],
[[v/radius**2 for v in row] for row in cov], color)
for xyz, cov, color in values]
normalized.sort(key=lambda p: p[0][2]) # camera is at z=3; farthest first
focal = width * .8
eyes = []
for eye in (-.032, .032):
pixels = bytearray(b'\x00\x00\x00\xff' * (width*height))
for (x, y, z), cov, color in normalized:
x -= eye
depth = 3-z
px, py = width/2+focal*x/depth, height/2-focal*y/depth
jac = ((focal/depth, 0, focal*x/depth**2),
(0, -focal/depth, -focal*y/depth**2))
screen = [[sum(jac[i][a]*cov[a][b]*jac[j][b] for a in range(3) for b in range(3))
for j in range(2)] for i in range(2)]
a, b, c = screen[0][0]+.3, screen[0][1], screen[1][1]+.3
det = a*c-b*b
if det <= 0 or not math.isfinite(det):
raise ValueError('Splat covariance is not renderable')
# A footprint cap bounds work on malformed or oversized Gaussians.
rx, ry = min(32, math.ceil(3*math.sqrt(a))), min(32, math.ceil(3*math.sqrt(c)))
for sy in range(max(0, int(py)-ry), min(height, int(py)+ry+1)):
dy = sy+.5-py
for sx in range(max(0, int(px)-rx), min(width, int(px)+rx+1)):
dx = sx+.5-px
power = (c*dx*dx-2*b*dx*dy+a*dy*dy)/det
if power > 9:
continue
alpha = color[3]/255 * math.exp(-.5*power)
offset = (sy*width+sx)*4
for k in range(3):
pixels[offset+k] = round(color[k]*alpha+pixels[offset+k]*(1-alpha))
eyes.append(pixels)
stride = width*4
return b''.join(eyes[0][y*stride:(y+1)*stride]+eyes[1][y*stride:(y+1)*stride]
for y in range(height)), width*2, height
+56 -24
View File
@@ -156,27 +156,59 @@ def flatpaks():
return out
uptime = read("/proc/uptime")
procs = process_names()
print(json.dumps({
"time": time.time(),
"hostname": socket.gethostname(),
"os": os_release(),
"uptime": float(uptime.split()[0]) if uptime else None,
"battery": battery(),
"power": power_source(),
"disk": {"root": disk("/"), "home": disk("/home")},
"memory": memory(),
"temp": max_temp(),
"wifi": wifi(),
"ip": ip_addr(),
"volume": volume(),
"services": {
"steamvr": "vrserver" in procs,
"desktop": "plasmashell" in procs,
"lepton": port_listening(5555),
"rdp": "xrdp" in procs,
},
"games": games(),
"flatpaks": flatpaks(),
}))
def thermal_alerts():
"""Use the kernel's per-zone hot/critical trips, never a guessed chip limit."""
alerts, known = [], False
for z in glob.glob("/sys/class/thermal/thermal_zone*"):
t = num(z + "/temp", 0.001)
for trip in glob.glob(z + "/trip_point_*_type"):
if read(trip) not in ("hot", "critical"):
continue
limit = num(trip[:-4] + "temp", 0.001)
if t is not None and limit is not None and limit > 0:
known = True
if t >= limit:
alerts.append({"zone": read(z + "/type"), "tempC": t, "limitC": limit})
return alerts if known else None
def activity_level():
try:
rows = json.loads(run("/opt/steamvr/bin/linuxarm64/vrcmd", "--stats"))
if not isinstance(rows, list):
return None
return next((r.get("activity_level") for r in rows
if isinstance(r, dict) and r.get("operation") == "status"), None)
except (ValueError, TypeError):
return None
def main():
uptime = read("/proc/uptime")
procs = process_names()
print(json.dumps({
"time": time.time(),
"hostname": socket.gethostname(),
"os": os_release(),
"uptime": float(uptime.split()[0]) if uptime else None,
"battery": battery(),
"power": power_source(),
"disk": {"root": disk("/"), "home": disk("/home")},
"memory": memory(),
"temp": max_temp(),
"wifi": wifi(),
"ip": ip_addr(),
"volume": volume(),
"services": {
"steamvr": "vrserver" in procs,
"desktop": "plasmashell" in procs,
"lepton": port_listening(5555),
"rdp": "xrdp" in procs,
},
"games": games(),
"flatpaks": flatpaks(),
}))
if __name__ == "__main__":
main()
+189 -1
View File
@@ -87,6 +87,7 @@
.wait::before { content: ""; width: 7px; height: 7px; border-radius: 50%; background: var(--dim); flex: none; }
/* ---- drop anywhere ---- */
#media select { min-width: 0; max-width: 100%; flex: 1; }
.dropzone { position: fixed; inset: 0; z-index: 40; display: grid; place-items: center; pointer-events: none;
background: rgba(14,20,27,.82); backdrop-filter: blur(3px); }
.dropzone > div { padding: 42px 60px; border: 2px dashed var(--blue); border-radius: 8px; text-align: center;
@@ -101,6 +102,7 @@
.shelf-head .count { color: var(--muted); font-size: 13px; }
.shelf-head .spacer { flex: 1; }
.sub { color: var(--muted); font-size: 12.5px; }
.sr-only { position: absolute; width: 1px; height: 1px; overflow: hidden; clip-path: inset(50%); white-space: nowrap; }
.hint { color: var(--muted); font-size: 12.5px; margin-top: 11px; line-height: 1.5; }
/* ---- buttons ---- */
@@ -347,7 +349,7 @@
.disp .k2 { color: var(--muted); font-size: 11px; letter-spacing: 1px; text-transform: uppercase; margin: 12px 0 5px; }
.disp .seg { flex-wrap: wrap; }
.disp .seg button { padding: 0 10px; }
.disp input[type=number] { width: 72px; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent;
.disp input[type=number], #comfort input[type=number] { width: 72px; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent;
border-radius: 3px; height: 32px; padding: 0 8px; font: inherit; font-size: 13px; }
/* Touch screens can't hover: keep the library's name and Play button showing. */
@media (hover: none) { .capsule .over { opacity: 1; } .capsule:hover { transform: none; } }
@@ -536,6 +538,36 @@
</section>
</div>
<section class="panel" id="comfort" style="margin:20px 0;padding:16px">
<div class="shelf-head"><h2>Family and comfort</h2><span class="spacer"></span>
<button class="action small" id="castBtn">Cast headset view</button>
</div>
<p class="hint">Share this screen with people in the room. Casting shows everything the wearer sees, including private content.</p>
<form id="comfortForm">
<div class="row" style="flex-wrap:wrap;gap:12px">
<label>Session limit (minutes) <input id="sessionMinutes" type="number" min="1" max="240" value="30" required style="width:75px"></label>
<label>Break every <input id="breakMinutes" type="number" min="0" max="120" value="20" required style="width:75px"> minutes</label>
<label>Check in after <input id="stillMinutes" type="number" min="0" max="240" value="30" required style="width:75px"> active minutes</label>
</div>
<div class="row" style="flex-wrap:wrap;margin:14px 0;gap:14px">
<label><input id="batteryAlert" type="checkbox" checked> Alert on low battery</label>
<label><input id="heatAlert" type="checkbox" checked> Alert on overheating</label>
<button class="action small" id="sessionStart" type="submit">Start session</button>
<button class="small" id="sessionCancel" type="button" disabled>Cancel session</button>
<button class="small" id="testNotification" type="button">Enable / test notifications</button>
</div>
</form>
<p id="comfortStatus" role="status">Checking session…</p>
<p id="comfortNotificationStatus" class="hint" role="status" hidden></p>
<p class="hint">A one-minute warning, then Steam Home. Games stay running: save and pause first. Keep this app open and connected for notifications.</p>
<details class="hint"><summary>How sessions and alerts work</summary>
<p>This is a reminder, not a parental lock. The timer continues on the Frame if you disconnect; a headset restart cancels it. Cancel before changing settings. Set break/check-in to 0 to turn them off.</p>
<p>Battery, heat and check-in alerts appear on connected companions during a session. Headset warnings and break reminders continue without a companion. iOS may suspend phone notifications in the background. Breaks and check-ins count SteamVR activity, not confirmed wear time.</p>
</details>
<div id="comfortEvents" class="hint"></div>
<span id="comfortAnnouncement" class="sr-only" aria-live="polite"></span>
</section>
<section class="panel" id="pad">
<div class="shelf-head"><h2>Keyboard and trackpad</h2><span class="sub" id="padState"></span><span class="spacer"></span>
<button class="small" id="padOn">Turn on</button>
@@ -714,6 +746,31 @@
<div class="hint">Clipboard needs the desktop panel open in the headset.</div>
</section>
<section class="panel" id="media">
<div class="shelf-head"><h2>Media in the headset</h2><span class="count">Preview</span></div>
<p class="sub">Play your movies and stereo photos with Frame Control's own player.</p>
<div class="row"><button class="small" id="mediaChoose">Send media…</button>
<button class="small" id="mediaRefresh">Refresh</button></div>
<input type="file" id="mediaInput" accept=".mp4,.mkv,.mov,.m4v,.webm,.png,.jpg,.jpeg,.splat" hidden>
<div class="row" style="margin-top:12px"><label for="mediaFile">On the Frame</label>
<select id="mediaFile"><option value="">Refresh to load your media</option></select></div>
<div class="row" style="margin-top:8px"><label for="mediaLayout">Layout</label>
<select id="mediaLayout">
<option value="auto">Detect from name or metadata</option><option value="mono">2D</option>
<option value="sbs">Half SBS · left / right</option><option value="ou">Half OU · top / bottom</option>
<option value="full-sbs">Full SBS · left / right</option><option value="full-ou">Full OU · top / bottom</option>
</select></div>
<div class="row" style="margin-top:8px"><label><input type="checkbox" id="mediaTheatre" checked>
Theatre · bigger screen, dark surround</label></div>
<div class="row" style="margin-top:12px"><button class="action small" id="mediaPlay">Play</button>
<button class="small" id="mediaStop">Stop</button></div>
<p class="sub" id="mediaState" role="status" aria-live="polite">Choose a file to begin.</p>
<div class="hint">H.264/H.265 video; SBS/OU PNG or JPEG photos. Screens follow your head.
Small .splat files show a stationary stereo preview (up to 20,000 Gaussians).
Native HEIC/MPO photos, large splat scenes, seeking and subtitles aren't supported yet.
Stop works from here; playback also ends after four hours.</div>
</section>
<section class="panel" id="apps">
<div class="shelf-head"><h2>Linux apps</h2><span class="count">Flatpak</span></div>
<div class="list" id="flatpaks"><div class="sub">Loading…</div></div>
@@ -1071,6 +1128,101 @@ function refresh() {
return refreshing;
}
// The Frame owns the clock. Poll independently of status and the selected tab.
let comfortBusy = false, comfortSeen = new Set(), comfortSession = null, comfortHydrated = false;
async function localNotification(message, request = false) {
try {
if (window.frameApp?.notify) await window.frameApp.notify(message, request);
else {
if (!("Notification" in window)) throw new Error("This browser has no notifications; use the Frame Control app.");
const permission = request ? await Notification.requestPermission() : Notification.permission;
if (permission !== "granted") throw new Error("Notifications are off. Enable them in system settings.");
new Notification("Frame Control", {body: message});
}
$("comfortNotificationStatus").hidden = true;
} catch (e) {
$("comfortNotificationStatus").hidden = false;
$("comfortNotificationStatus").textContent = "Notifications aren't available on this device. Check system notification settings, then use Enable / test notifications. Headset reminders continue during an active session.";
throw e;
}
}
function renderComfort(s) {
const firstSnapshot = !comfortHydrated;
comfortHydrated = true;
$("sessionStart").disabled = !!s.active;
for (const id of ["sessionMinutes", "breakMinutes", "stillMinutes", "batteryAlert", "heatAlert"])
$(id).disabled = !!s.active;
$("sessionCancel").disabled = !s.active;
if (s.id !== comfortSession) {
comfortSession = s.id;
comfortSeen.clear();
if (s.options) for (const [key, value] of Object.entries(s.options)) {
const el = $(key === "minutes" ? "sessionMinutes" : key);
if (!el) continue;
if (el.type === "checkbox") el.checked = value; else el.value = value;
}
}
let statusText = s.error || (s.active
? `${Math.ceil(s.remaining / 60)} min until Steam Home · ${s.activity == null ? "activity unknown" : s.activity === 3 ? "headset in standby" : "monitoring"}`
: "No session running.");
if (s.active && s.unavailable?.length)
statusText += " · No readings: " + s.unavailable.join(", ");
if ($("comfortStatus").textContent !== statusText) $("comfortStatus").textContent = statusText;
let latest = null;
for (const e of s.events || []) {
if (comfortSeen.has(e.id)) continue;
comfortSeen.add(e.id);
// Historical events remain visible but never produce a burst on reconnect.
if (!firstSnapshot && s.time - e.time >= 0 && s.time - e.time < 30 && !["started", "cancelled"].includes(e.kind)) {
latest = e.message;
log(e.message); toast(e.message, e.kind === "error");
localNotification(e.message).catch(() => {}); // the notification status keeps the failure visible
}
}
// Keep only the server's bounded history; a new page seeds it without replay.
comfortSeen = new Set((s.events || []).map(e => e.id));
if (latest !== null) $("comfortAnnouncement").textContent = latest;
const history = (s.events || []).slice(-3).map(e => e.message).join(" · ");
if ($("comfortEvents").textContent !== history) $("comfortEvents").textContent = history;
}
async function pollComfort() {
if (!comfortBusy) {
comfortBusy = true;
try { renderComfort(await api("/api/comfort", {action: "status"})); }
catch (e) {
const message = "Session status unavailable: " + e.message;
if ($("comfortStatus").textContent !== message) $("comfortStatus").textContent = message;
}
finally { comfortBusy = false; }
}
setTimeout(pollComfort, 5000);
}
$("comfortForm").onsubmit = async e => {
e.preventDefault();
const result = await act("Start session", () => api("/api/comfort", {
action: "start", minutes: Number($("sessionMinutes").value), breakMinutes: Number($("breakMinutes").value),
stillMinutes: Number($("stillMinutes").value), batteryAlert: $("batteryAlert").checked, heatAlert: $("heatAlert").checked,
}), $("sessionStart"));
if (result) renderComfort(result);
};
$("sessionCancel").onclick = async () => {
const result = await act("Cancel session", () => api("/api/comfort", {action: "cancel"}), $("sessionCancel"));
if (result) renderComfort(result);
};
$("testNotification").onclick = () => act("Test notification", async () => {
await localNotification("Comfort notifications are enabled on this device.", true);
return {message: "Test notification sent. Check your device's notification settings if it didn't appear."};
});
$("castBtn").onclick = () => {
setView("headset");
if (!live) toggleLive(true);
$("viewer").scrollIntoView({behavior: "smooth", block: "center"});
// Fullscreen is a direct user gesture; iPhone falls back to its inline viewer.
$("viewer").requestFullscreen?.().catch(() => {});
};
pollComfort();
// ---- battery ----
function battery(b, power) {
const pct = b?.percent;
@@ -1602,6 +1754,42 @@ function upload(file, mode) {
xhr.send(file);
});
}
async function refreshMedia(selectId) {
const data = await api("/api/media", { action: "list" });
const selected = selectId || $("mediaFile").value;
$("mediaFile").replaceChildren();
if (!data.files.length) $("mediaFile").add(new Option("Send media to begin", ""));
for (const file of data.files) $("mediaFile").add(new Option(file.name, file.id));
if (data.files.some(f => f.id === selected)) $("mediaFile").value = selected;
const p = data.player;
$("mediaState").textContent = p.error || `${p.state}${p.file ? ": " + p.file : ""}`;
return data;
}
$("mediaChoose").onclick = () => $("mediaInput").click();
$("mediaInput").onchange = () => act("Send media", async () => {
const file = $("mediaInput").files[0];
if (!file) return;
const result = await upload(file, "media");
await refreshMedia(result.id);
$("mediaInput").value = "";
return result;
}, $("mediaChoose"));
$("mediaRefresh").onclick = () => act("Refresh media", () => refreshMedia(), $("mediaRefresh"));
$("mediaPlay").onclick = () => act("Play media", async () => {
const id = $("mediaFile").value;
if (!id) throw new Error("Send or select a media file first");
const result = await api("/api/media", { action: "play", id,
layout: $("mediaLayout").value, theatre: $("mediaTheatre").checked });
$("mediaState").textContent = "Starting…";
setTimeout(() => refreshMedia().catch(e => { $("mediaState").textContent = e.message; }), 1500);
return result;
}, $("mediaPlay"));
$("mediaStop").onclick = () => act("Stop media", async () => {
const result = await api("/api/media", { action: "stop" });
await refreshMedia();
return result;
}, $("mediaStop"));
let apkLookup = 0;
async function checkApkAlternatives(apk) {
const lookup = ++apkLookup;
+105 -1
View File
@@ -44,8 +44,10 @@ import frame_assistant # noqa: E402
import frame_android # noqa: E402
import frame_apk_versions # noqa: E402
import frame_catalog # noqa: E402
import frame_comfort # noqa: E402
import frame_host # noqa: E402
import frame_macview # noqa: E402
import frame_media # noqa: E402
import frame_report # noqa: E402
import frame_store # noqa: E402
import frame_telemetry # noqa: E402
@@ -269,6 +271,37 @@ def status(_body):
return s
def comfort(body):
try:
frame_comfort.validate(body)
except ValueError as e:
raise Failure(str(e), 400)
# Content-addressed, user-only helper bundle. Desktop and phone use the same
# on-headset state/lock; no listener, service registration or third-party app.
import hashlib
files = {name: (HERE / name).read_text() for name in
("frame_comfort.py", "frame_status.py", "frame_steam.py")}
version = hashlib.sha256(json.dumps(files, sort_keys=True).encode()).hexdigest()[:16]
script = """import json, os, pathlib, subprocess, sys
os.umask(0o077)
files = %r
root = pathlib.Path.home() / '.cache/frame-control/comfort' / %r
root.mkdir(parents=True, exist_ok=True)
for name, source in files.items():
path = root / name
if not path.exists():
tmp = root / (name + '.' + str(os.getpid()))
tmp.write_text(source)
tmp.replace(path)
r = subprocess.run([sys.executable, str(root / 'frame_comfort.py'), %r], capture_output=True, text=True)
print(r.stdout, end='')
""" % (files, version, json.dumps(body))
out = json.loads(ssh("python3 -", stdin=script, timeout=65))
if out.get("error") and "active" not in out:
raise Failure(out["error"], 409)
return out
def headset_view():
"""Both eyes as SteamVR composites them (see frame_vrshot.py); PNG bytes."""
# `timeout`: VR_Init can block if SteamVR is restarting.
@@ -1609,6 +1642,75 @@ def _sweep_one(prefix, d):
pass
# ---- Our Frame-side media player -----------------------------------------
_MEDIA_LOCK = threading.Lock()
def media(body):
action = body.get("action")
if action not in ("list", "status", "play", "stop"):
raise Failure("Media action must be list, status, play or stop", 400)
if action == "play":
identity = body.get("id")
if not isinstance(identity, str) or not re.fullmatch(r"[0-9a-f]{32}/[^/\\\x00]+", identity):
raise Failure("Invalid media id", 400)
if body.get("layout", "auto") not in frame_media.LAYOUTS:
raise Failure("Invalid media layout", 400)
if type(body.get("theatre", False)) is not bool:
raise Failure("theatre must be true or false", 400)
with _MEDIA_LOCK:
# Ship only our small stdlib modules, atomically, to the user account.
sources = {name: (HERE / name).read_text() for name in (
"frame_media.py", "frame_media_player.py", "frame_media_remote.py", "frame_splat.py")}
installer = """import json, os, pathlib, sys, tempfile
root = pathlib.Path.home()/'.local/share/frame-control/media'
root.mkdir(parents=True, exist_ok=True)
for name, source in json.load(sys.stdin).items():
path = root/name
fd, temp = tempfile.mkstemp(dir=root, prefix=name+'.')
with os.fdopen(fd, 'w') as f:
f.write(source)
os.replace(temp, path)
"""
ssh("python3 -c " + shlex.quote(installer), stdin=json.dumps(sources))
try:
out = ssh("python3 ~/.local/share/frame-control/media/frame_media_remote.py",
stdin=json.dumps(body), timeout=75) # remote worst case: ffprobe 30 + reset-failed 10 + systemd-run 15 s
except Failure as e:
for line in reversed(getattr(e, "stdout", "").splitlines()):
try:
detail = json.loads(line).get("error")
except (ValueError, AttributeError):
continue
if detail:
raise Failure(detail) from None
raise
return json.loads(out)
def push_media(path):
# Validate the format, but leave layout selection until playback (ffprobe
# can then read metadata on the Frame, where it is installed).
name = Path(path).name
frame_media.plan(name, "mono")
if name.startswith(".") or "\\" in name:
raise Failure("Rename the file: media names can't start with a dot or contain a backslash", 400)
token = secrets.token_hex(16)
dest = "Videos/FrameControl/" + token + "/"
ssh("mkdir -p ~/" + dest)
try:
push_file(path, dest)
except Exception:
try:
ssh("rm -rf ~/" + dest)
except Exception:
pass # keep the copy error; an empty folder isn't listed as media
raise
return {"message": "Media sent. Choose its layout and press Play.",
"id": token + "/" + name}
# ---- Mac in the headset (frame_macview.py) ----------------------------------
# The tunnel gets its own connection: the shared master's options would win
@@ -1675,7 +1777,7 @@ def agent_approval(body):
return frame_agent.approvals.decide(body.get("confirmation"), body.get("accept"))
POST = {"/api/agent/call": agent_call, "/api/agent/approval": agent_approval,
POST = {"/api/comfort": comfort, "/api/media": media, "/api/agent/call": agent_call, "/api/agent/approval": agent_approval,
"/api/assistant/chat": assistant_chat, "/api/android/display": android_display, "/api/android": android, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard,
"/api/input": remote_input,
"/api/flatpak": flatpak, "/api/open": open_thing, "/api/shots/save": save_shots,
@@ -1972,6 +2074,8 @@ class Handler(BaseHTTPRequestHandler):
raise Failure("upload interrupted", 400)
f.write(chunk)
remaining -= len(chunk)
if mode == "media":
return push_media(dest)
if mode == "apkinfo":
# Read an APK for a report without installing it.
try: