From a3c6e5c984354e9d13ddfeef157d38cb053781be Mon Sep 17 00:00:00 2001 From: saphid <4596216+saphid@users.noreply.github.com> Date: Mon, 28 Sep 2026 13:33:46 +1000 Subject: [PATCH 01/51] Mac in the headset: stream Mac windows and screens into the Frame as panels Frame Control can now show any Mac window, or a whole screen, as its own SteamVR panel on the Steam Frame (Tools -> Mac in the headset, macOS only). Place it with the SteamVR dashboard; the laser clicks and scrolls, and the Mac's own keyboard types. - mac/frame-mac-view (Swift, no dependencies): ScreenCaptureKit capture per window or display, VideoToolbox H.264 with low-latency rate control (JPEG fallback), a loopback HTTP/WebSocket server, CGEvent/AX input playback, and a display-awake assertion while anyone watches. - ui/frame_macview.py: starts the agent, runs an ssh -R tunnel with a supervisor that reopens it on the same port, and launches a Chromium app window per stream on gamescope's :0, tagged with STEAM_GAME for its own panel. - Frame Control's key never leaves the Mac: viewers get single-use, per-source tickets and reconnect keys that Stop revokes. - ui/mac-view.html: WebCodecs decode, keyframe recovery, pointer/wheel/keys back. - Bundled in the Mac app build; tests/test_macview.py builds and drives the agent on macOS. Verified on the Frame (build 20260925.6191901) with the test pattern: panel in about 1.5 s, about 60 fps, Mac-to-window about 11-17 ms, tunnel recovery in 4 s. Laser input and real window capture still need a person in the headset. Also commits the other thread's first-party rule (steam-frame skill) and the first-party options table in docs/streaming.md. Reviewed by GPT-6 Astra (xhigh, read-only) over six rounds; all findings fixed. Co-Authored-By: Claude Opus 5.5 (1M context) --- .claude/skills/steam-frame/SKILL.md | 6 + .gitignore | 1 + README.md | 1 + app/package.json | 16 +- docs/frame-control.md | 4 + docs/how-the-frame-works.md | 1 + docs/mac-in-headset.md | 204 ++++++++++ docs/open-questions.md | 13 + docs/streaming.md | 18 +- mac/frame-mac-view/Sources/Capture.swift | 339 ++++++++++++++++ mac/frame-mac-view/Sources/Encoder.swift | 172 ++++++++ mac/frame-mac-view/Sources/Input.swift | 168 ++++++++ mac/frame-mac-view/Sources/Server.swift | 267 ++++++++++++ mac/frame-mac-view/Sources/main.swift | 490 +++++++++++++++++++++++ mac/frame-mac-view/build.sh | 10 + tests/test_macview.py | 274 +++++++++++++ ui/frame_macview.py | 361 +++++++++++++++++ ui/index.html | 81 +++- ui/mac-view.html | 242 +++++++++++ ui/server.py | 46 ++- 20 files changed, 2708 insertions(+), 6 deletions(-) create mode 100644 docs/mac-in-headset.md create mode 100644 mac/frame-mac-view/Sources/Capture.swift create mode 100644 mac/frame-mac-view/Sources/Encoder.swift create mode 100644 mac/frame-mac-view/Sources/Input.swift create mode 100644 mac/frame-mac-view/Sources/Server.swift create mode 100644 mac/frame-mac-view/Sources/main.swift create mode 100755 mac/frame-mac-view/build.sh create mode 100644 tests/test_macview.py create mode 100644 ui/frame_macview.py create mode 100644 ui/mac-view.html diff --git a/.claude/skills/steam-frame/SKILL.md b/.claude/skills/steam-frame/SKILL.md index 0437825..48c9f1c 100644 --- a/.claude/skills/steam-frame/SKILL.md +++ b/.claude/skills/steam-frame/SKILL.md @@ -48,3 +48,9 @@ running `--help`: `paste-to-frame.sh`, `serve-bootstrap.sh` and - `sudo` on the Frame asks for the user's Developer Mode password. Hand those steps to the user (open Terminal) and keep automation to non-sudo commands. - The Mac uses BSD userland and zsh (no `timeout`, use `head -n`). +- **First-party first.** For any new capability, investigate the first-party + way before anything else: Valve (SteamOS, Steam, Steam Link), Apple (the Mac + and iPhone), and KDE (the Frame's desktop is Plasma). It's usually the best + answer. If it isn't, write down why not. If it is, find what Frame Control + can do to make it easier to set up (install over SSH, pre-seed settings, + pair automatically, tell the user the one setting to turn on). diff --git a/.gitignore b/.gitignore index d89ecc2..0c187b8 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,4 @@ apk-catalog/data/index-v2.json* compat-db/.env.lakebed.server compat-db/.lakebed/ tests/smoke/results/ +mac/bin/ diff --git a/README.md b/README.md index 5f25235..36bffb4 100644 --- a/README.md +++ b/README.md @@ -192,6 +192,7 @@ Frame's software fits together, all checked against a real headset and labelled | [Sideloading Linux and Windows games](docs/sideloading.md) | A .zip, folder or .exe as a Steam Devkit Game, runtime detection | | [Install links for websites](docs/web-install.md) | `frame-control://install` links and manifests, the rules, a button to paste | | [Steam games](docs/steam-games.md) · [VR video](docs/vr-video.md) · [WebXR in Chromium](docs/webxr-chromium.md) | Installing and buying, watching VR180/360, the Chromium build | +| [Mac in the headset](docs/mac-in-headset.md) | Mac windows and screens as panels in the Frame, with laser and keyboard input | | [SSH](docs/ssh.md) · [Streaming](docs/streaming.md) · [Files](docs/file-transfer.md) · [Panels](docs/panels.md) · [Tailscale](docs/tailscale.md) | Topic notes | | [Frame Control for iPhone](docs/iphone.md) | The iPhone and iPad app, how it runs the server on the Frame, pairing | | [Recovery and OS images](docs/recovery-and-images.md) | Where to download the Frame's OS, what's inside, testing without the headset | diff --git a/app/package.json b/app/package.json index bd5701a..a06fee8 100644 --- a/app/package.json +++ b/app/package.json @@ -9,8 +9,8 @@ "scripts": { "start": "env -u ELECTRON_RUN_AS_NODE electron .", "icon": "env -u ELECTRON_RUN_AS_NODE electron build/make-icon.js", - "dist": "node build/fetch-deps.js mac arm64 && electron-builder --mac --arm64 --publish never", - "dist:dir": "node build/fetch-deps.js mac arm64 && electron-builder --mac --arm64 --dir", + "dist": "sh ../mac/frame-mac-view/build.sh && node build/fetch-deps.js mac arm64 && electron-builder --mac --arm64 --publish never", + "dist:dir": "sh ../mac/frame-mac-view/build.sh && node build/fetch-deps.js mac arm64 && electron-builder --mac --arm64 --dir", "dist:linux": "node build/fetch-deps.js linux x64 arm64 && electron-builder --linux --x64 --arm64 --publish never", "dist:win": "node build/fetch-deps.js win x64 && electron-builder --win --x64 --publish never" }, @@ -105,9 +105,19 @@ "dmg", "zip" ], + "extraResources": [ + { + "from": "../mac/bin", + "to": "mac/bin", + "filter": [ + "frame-mac-view" + ] + } + ], "extendInfo": { "NSAppleEventsUsageDescription": "Frame Control opens Terminal for SSH sessions and for power actions that need the Developer Mode password.", - "NSLocalNetworkUsageDescription": "Frame Control connects to your Steam Frame over SSH on the local network." + "NSLocalNetworkUsageDescription": "Frame Control connects to your Steam Frame over SSH on the local network.", + "NSScreenCaptureUsageDescription": "Frame Control shows your Mac's windows and screens inside the Steam Frame when you ask it to." }, "artifactName": "Frame-Control-mac-${arch}.${ext}" }, diff --git a/docs/frame-control.md b/docs/frame-control.md index 299c57c..5befb8c 100644 --- a/docs/frame-control.md +++ b/docs/frame-control.md @@ -70,6 +70,10 @@ counts them while they run. Runtime picked from the program's header), listed under **Sideloaded titles** with Launch and Remove; see [sideloading.md](sideloading.md). Send typed text, or your computer's clipboard, to the Frame clipboard. +- **Mac in the headset** (macOS): show any Mac window, or a whole screen, as + its own panel in the headset. Place it with the SteamVR dashboard, click and + scroll with the laser, and type on the Mac. Streams hardware H.264 through + an SSH tunnel; see [mac-in-headset.md](mac-in-headset.md). - **Flatpaks**: install and remove them (quick picks: Moonlight, Firefox, VLC, Remmina). - **One-click tools**: SSH or SFTP in a terminal window, Steam Link, and remote diff --git a/docs/how-the-frame-works.md b/docs/how-the-frame-works.md index 74f81f1..9681da8 100644 --- a/docs/how-the-frame-works.md +++ b/docs/how-the-frame-works.md @@ -51,6 +51,7 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600 | **T3 Code desktop runs natively.** The stock release `T3-Code-0.0.42-arm64.AppImage` in `~/Applications/T3CodeDesktop/` starts with no extra setup: glibc 2.39, `libfuse.so.2`, GTK 3, NSS and libsecret are on the image. `panel-on-frame.sh --name t3code-desktop -- '~/Applications/T3CodeDesktop/T3-Code.AppImage'` gives it its own panel (`valve.steam.desktopgame.2000281357`, `--ozone-platform=x11`). Its bundled server listens on `127.0.0.1:3773` and shows up in onboarding as the `frame` computer, with `passwordStore: gnome-libsecret`. The image has no agent CLI and no `node`. Agents run through the LAN CLIProxyAPI (`llm-proxy.lan:8317`, which resolves on the Frame). Claude Code 2.1.283 comes from `claude.ai/install.sh`, and Codex 0.157.1 from the `codex-aarch64-unknown-linux-musl` release tarball, both into `~/.local/bin`. `with-cliproxy` and a mode-600 `~/.config/cliproxyapi/secrets.env` are copied from the Mac. The wrappers `claude-cliproxy` and `codex-cliproxy` (a `-c model_provider=cliproxy`, `wire_api="responses"`, `env_key="CLIPROXY_API_KEY"`) are set as `providers.claudeAgent.binaryPath` and `providers.codex.binaryPath` in `~/.t3/userdata/settings.json`, and T3 picked that up without a restart. Through the wrappers, `claude auth status` reports `loggedIn: true` (`oauth_token`), and both CLIs answered a prompt with `kimi-k3`. `gamescopectl screenshot` captured another layer (the Lepton T3 app) rather than this panel. `DISPLAY=:0 xwd -id ` piped to `ffmpeg` captures the window itself (1920×1080). **Verified 2026-09-26**, BUILD_ID 20260922.6101926. | Running T3 Code as a host on the Frame | | Power actions need `sudo`, which asks for the Developer Mode password over SSH. | Frame Control's power buttons | | **SSH server:** OpenSSH 9.7p1. It offers `publickey,password` (keyboard-interactive is off, PAM on) and also asks `userdbctl ssh-authorized-keys` for keys. OpenSSH ≥ 8.8 rejects SHA-1 `ssh-rsa` signatures by default, so a client whose RSA support is SHA-1 only (the Swift library Citadel, for one) can't log in with the RSA key that devkit pairing installs; use ed25519 (**inferred** from OpenSSH defaults). **Verified 2026-09-27**, BUILD_ID 20260922.6101926. | [iphone.md](iphone.md), `ui/frame_connect.py` | +| **A Chromium app window on gamescope's `:0` with its own `STEAM_GAME` becomes a panel, even when started over SSH.** `chromium-xr/chrome --ozone-platform=x11 --app=URL --window-size=1280,720` got a 1920×1080 window, and tagging it produced `[Overlays] Created: valve.steam.desktopgame.` in `~/.local/share/Steam/logs/vrwebhelper_systemui.txt`. Chromium XR decoded a 1080p H.264 WebCodecs stream from the Mac at about 60 fps. XTest events sent to `:0` (libXtst through Python ctypes) did not reach the page. The Frame has `libXtst`, `xprop`, `xwininfo`, `curl` and the `C.utf8` locale. **Verified 2026-09-28**, BUILD_ID 20260925.6191901. | [mac-in-headset.md](mac-in-headset.md) | | **Tools on the image:** Python 3.12.3, `ffmpeg`, `openssl`, `curl`, `rsync`, `zip`/`unzip`, `flatpak`, `wpctl`, `podman`. **No `adb`.** `steamos` is uid 1000, in `wheel`, and sudoers has `%wheel ALL=(ALL) ALL`, so `sudo -S` takes the Developer Mode password on stdin. **Verified 2026-09-27.** | Running Frame Control's server on the Frame (`FRAME_LOCAL=1`, [iphone.md](iphone.md)) | | **Each Lepton instance is a podman container** named `lepton-steamlaunch-`, labelled with its ADB port (`podman ps --format '{{.Names}} {{.Labels.adb_port}}'`). `podman exec /system/bin/sh -c '…'` runs Android's shell inside it with no adb at all (used for `pidof` and `logcat` by the app tester). Running `wm size`/`wm density` that way is untested. **Verified 2026-09-27.** | `ui/frame_android.py`, the iPhone app's display settings | | **Asleep means off the network.** In standby the Frame stops answering on its LAN address, `frame.local` and Tailscale alike (`Host is down`, `No route to host`, timeouts), and ping fails. It was unreachable for about 2.5 hours until woken. Nothing over SSH can wake it. **Verified 2026-09-27.** | Frame Control's offline banner and retries | diff --git a/docs/mac-in-headset.md b/docs/mac-in-headset.md new file mode 100644 index 0000000..55728bc --- /dev/null +++ b/docs/mac-in-headset.md @@ -0,0 +1,204 @@ +# Mac in the headset + +Frame Control can show any Mac window, or a whole Mac screen, as its own panel +in the Steam Frame. You place each panel anywhere in the room with the SteamVR +dashboard. The laser clicks and drags, the thumbstick scrolls, and you type on +the Mac's own keyboard. Find it under **Tools → Mac in the headset** (macOS +only). + +The confidence labels are the same as in [ssh.md](ssh.md). + +## Why this design + +First-party options come first, as the repo's rule asks, with the reason +each one was or wasn't chosen. The full list for every device is in +[streaming.md](streaming.md#first-party-options-and-why-they-do-or-dont-fit). +Checked 2026-09-28. + +| Goal | First-party option | Chosen? | Why | +|---|---|---|---| +| One Mac screen in the headset | **Apple Screen Sharing** (VNC) → Remmina (Remmina 1.4.43 is already installed on this Frame) | Kept as the fallback (`panel-on-frame.sh mac-screen`) | It's the closest to first-party and needs nothing new. But VNC sends compressed tiles rather than video, so moving content is slow. It shows only whole screens | +| One Mac screen | **Steam Remote Play**, Mac as host (Valve) | No | macOS isn't a SteamVR host, and Mac-hosted Remote Play is reported broken ([Steam forum](https://steamcommunity.com/groups/homestream/discussions/1/574921459914429988/)). It streams games, not the desktop. **Not tested here**; one real try is still worth doing | +| One Mac screen | **AirPlay** (Apple) | No | Apple licenses AirPlay receivers only to TV and speaker makers, and nothing official runs on Linux. UxPlay is an unofficial receiver, and it mirrors a whole screen, not single windows | +| One Mac screen | **Sidecar / Mac Virtual Display** (Apple) | No | These work only with an iPad or Apple Vision Pro | +| **Each Mac window as its own panel** | None | – | No first-party way does this: Apple's per-app streaming is only for Vision Pro, and Valve's desktop streaming needs a Windows SteamVR host. So Frame Control does it itself | +| Mac keyboard and trackpad driving the headset | **Bluetooth HID** | No | macOS can't act as a Bluetooth keyboard or mouse. A real Bluetooth keyboard paired with the Frame still works | +| Mac keyboard and trackpad | **KDE Connect** (KDE) | No | The Frame has no `kdeconnectd` and it isn't on Flathub. Its Mac app has no keyboard or mouse sharing (**inferred**), and on Wayland it can only reach the desktop panel | +| Mac keyboard and trackpad | **xrdp** (Valve, Developer Mode) | No | It runs a separate Linux session that you view on the Mac. It isn't the headset's view, and it doesn't carry input the other way | + +What that leaves is our own stream: nothing to install on the Mac or the +Frame, and whole screens or single windows. Here the Mac's own keyboard and +trackpad need no forwarding, because the windows are still on the Mac. The +laser is the only input that has to be sent back. + +Other routes that were compared: + +| Option | One screen | Each window | Speed | Verdict | +|---|---|---|---|---| +| Sunshine → Moonlight | ✓ | – | Good | Sunshine's macOS support is still experimental ([discussion #777](https://github.com/orgs/LizardByte/discussions/777)), and it captures whole screens only | +| Virtual Desktop, Immersed | – | – | – | No Frame client as of September 2026 | +| **Frame Control's own stream** | ✓ | ✓ | Hardware H.264, sending only changed frames | **Built** | + +To type into VR surfaces other than these panels (SteamVR's dashboard, +games), the Frame supports a uinput keyboard and mouse without sudo +(verified 2026-09-27: `steamos` is in `input`, and `/dev/uinput` is +`root:input 660`). That's a separate feature, not part of this one. + +## How it works + +``` +Mac Frame +ScreenCaptureKit (one window or display) + → VideoToolbox H.264 (hardware, low-latency, + no B-frames) + → frame-mac-view, 127.0.0.1 ──ssh -R──→ 127.0.0.1:479xx + → Chromium app window per stream + (WebCodecs decode), on gamescope's + X display, tagged STEAM_GAME + → its own SteamVR panel + ← CGEvent (clicks, drags, wheel, keys) ←──── pointer, wheel and key events +``` + +- **The agent** is `mac/bin/frame-mac-view`, built from `mac/frame-mac-view` + (Swift, no dependencies; `build.sh`). Frame Control's server starts it on + first use and stops it on quit. + - It captures with ScreenCaptureKit, which sends frames only when something + changes, so idle windows cost nothing. + - It encodes in hardware with VideoToolbox's low-latency rate control (plain + real-time mode where that's unavailable). + - While anyone is watching, it keeps the Mac's display awake. A sleeping + display isn't drawn, so there would be nothing to capture. +- **The link** is an `ssh -R` tunnel on its own connection. It's encrypted and + works anywhere `ssh frame` works, Tailscale included, with no firewall + changes on the Mac. If the headset sleeps or the network drops, Frame + Control reopens the tunnel on the same port, and open viewers reconnect by + themselves. +- **Access.** + - Frame Control's own key never leaves the Mac. + - Each viewer is opened with a **single-use ticket**. It's tied to one + window or display and expires after a minute. It's spent as soon as the + viewer confirms it has received its reconnect key. Until then, a retry + gets the same key, so a connection lost at that moment doesn't strand + the viewer. Stop revokes tickets that haven't been used yet. + - After that, the viewer holds a reconnect key for that one source, in + memory only. **Stop** revokes it. + - Remaining risk: a program running as `steamos` on the Frame could read a + ticket from Chromium's command line in the first second or so and use it + first. That gets it the one source being opened, not the Mac, and the + real viewer would then fail to connect. Android apps in Lepton run in + their own podman container, so they shouldn't see the Frame's process + list (inferred, not checked). +- **The viewer** is `ui/mac-view.html`, served by the agent. It opens on the + Frame as a Chromium app window, preferring Chromium XR (`~/chromium-xr`, + built with H.264) over Flathub Chromium. + - The page puts `[fcNNNNN]` in its title. The launcher finds the window by + that tag and sets `STEAM_GAME` to a stable id per source, which gives it + its own panel (see [panels.md](panels.md)). The same Mac window gets the + same panel id each time. + - It decodes with WebCodecs. If it falls behind, it skips to the next + keyframe instead of showing old frames late. + - It falls back to JPEG stills (**Compatible** quality) where H.264 isn't + available. +- **Flow control.** When the link backs up, the agent skips capture frames + *before* encoding, so no reference frame goes missing. Once the link drains, + it sends the newest picture. +- **Input.** + - A click on a window's panel brings that Mac window to the front + (Accessibility API), then clicks at the same point. Double clicks, right + clicks, drags and the wheel work too. + - Keys typed into the panel are sent as Mac key codes. Any keys or buttons + still held down are released if the viewer loses focus or disconnects, or + when the stream stops. Characters the key + table doesn't know, such as those from other keyboard layouts, are typed + as text. + - The Mac's own keyboard and trackpad keep working as normal. Click a panel + with the laser, then type on the Mac. + +## Permissions (Mac) + +- **Screen Recording**, to see windows. Without it, the card asks for it. +- **Accessibility**, so input from the headset reaches the Mac. Without it the + stream still works, and the viewer says clicks won't go through. + +Both are granted to Frame Control. After granting, press **Refresh**, which +restarts the helper so it picks them up. The app is ad-hoc signed, so macOS +may ask again after an update. + +## Quality settings + +| Setting | Long side | fps | Codec | Use | +|---|---|---|---|---| +| Sharp | 2560 | 60 | H.264, ~0.14 bits/pixel | Text-heavy windows on a strong link | +| Balanced (default) | 1920 | 60 | H.264, ~0.1 bits/pixel | Most things | +| Light | 1280 | 30 | H.264 | Weak Wi-Fi or Tailscale off the LAN | +| Compatible | 1280 | 20 | JPEG | A Frame browser without H.264 | + +## Checked so far (2026-09-28) + +- **Mac (checked by hand, macOS 26.5.2).** + - The agent builds, and lists windows and displays. + - In a browser, the test pattern decoded at about 60 fps (H.264) and 30 fps + (JPEG, about 22 Mbps). + - A click in the viewer arrived at the same point in the source. + - `pmset -g assertions` showed the display-awake assertion only while a + stream was being watched. +- **Automated** (`tests/test_macview.py`, on CI's macOS runner): the agent + builds (a build failure fails the job). + - `/ping` and the page are open; everything else needs the key. + - Tickets work once and only for their own source, and Stop revokes + reconnect keys. + - A WebSocket frame claiming 2^63 bytes closes that socket, and the agent + keeps running. + - A stream sends an SPS-led H.264 keyframe, and sends another when asked. + - Stop ends the stream on the Mac even if the viewer ignores it. + - The test doesn't decode video, time it, or check where clicks land. +- **Linux aarch64 (verified in a stand-in, not on the Frame).** In an Arch + Linux ARM container with sshd, Xvfb as `:0` and Chromium 153: + - The real `show` path worked in 1–2.3 s: tunnel, ticket, launcher, + window found and tagged `STEAM_GAME`. + - Frame Control's key didn't appear anywhere in the stand-in's process + list. + - After the tunnel was killed, it came back on the same port within 4 s, + and the viewer reconnected by itself. + - Chromium decoded the stream in software with the GPU off. + - Stop closed the window, and Chromium exited. + - This test found and fixed a bug: in a C locale, `xwininfo` can't print a + title with non-ASCII characters, so the launcher reads `_NET_WM_NAME` + with `xprop`. +- **On the Frame (verified 2026-09-28, SteamOS build 20260925.6191901, + from the Mac, nobody wearing the headset).** Frame Control's **Show** with + the test pattern: + - The panel was ready in 1.45 s. SteamVR logged `[Overlays] Created: + valve.steam.desktopgame.2001639889` (in `vrwebhelper_systemui.txt`). + - The window was tagged `STEAM_GAME`, and gamescope sized it to 1920×1080 + although 1280×720 was asked for. + - Chromium XR decoded it live at about 60 fps: the frame counter advanced + 62 in 1.04 s. + - Over home Wi-Fi, the frames in the viewer window had been drawn on the + Mac about 11–17 ms earlier, plus `xwd`'s own time. That's measured + against the two clocks, which were 37–39 ms apart (±4 ms, measured over + one SSH session). SteamVR's compositor and the display come on top. + - Clicks and keys injected with XTest into gamescope's Xwayland didn't + reach the page. That's inconclusive, not a failure: XTest on gamescope + isn't how real input arrives. The laser should arrive as a left mouse + button and the thumbstick as a wheel, because the window has an app id + (from gamescope's source; see [panels.md](panels.md)). +- **Not yet checked:** + - Clicking, dragging and scrolling with the laser while wearing the + headset. + - Real window capture and input on a Mac with both permissions granted. + - Keys from SteamVR's on-screen keyboard. + - Whether Flathub Chromium has H.264. Chromium XR is used when it's + installed, as it is on this Frame. + - Latency with real, busy windows at Sharp. + +## Limits + +- Only windows on the Mac's current desktop (Space) are listed, and + minimised windows can't be captured. +- A window's panel shows only that window. Its menus and sheets are separate + windows on the Mac, so open them from the Mac or use **Whole screen**. +- Keys go to whichever Mac window is in front. Clicking a panel brings its + window to the front first. +- Ctrl stays Ctrl. On the Mac, copy is ⌘C, so use Meta+C on a keyboard paired + with the Frame. diff --git a/docs/open-questions.md b/docs/open-questions.md index 44583b2..8592280 100644 --- a/docs/open-questions.md +++ b/docs/open-questions.md @@ -125,6 +125,19 @@ Still open: 4, 6, 7, 11 (in-headset connect), 12–15, 16 (off-LAN and after a r - The Mac EDL flashing script in `~/Downloads/steam-frame-recovery/` hasn't been run against a Frame. +## Mac in the headset (2026-09-28) + +The test pattern streams to the Frame as its own panel at about 60 fps +(verified, build 20260925.6191901; see +[mac-in-headset.md](mac-in-headset.md#checked-so-far-2026-09-28)). Still to +check in the headset: + +- Laser clicks, drags and thumbstick scrolling in a viewer panel. +- Real window capture and input once Screen Recording and Accessibility are + granted to Frame Control. +- Keys from SteamVR's on-screen keyboard. +- Whether Flathub Chromium decodes H.264 (otherwise use Compatible). + ## Unconfirmed claims made in these docs - `/home` and `/etc` persist across Frame OS updates. This is inferred from diff --git a/docs/streaming.md b/docs/streaming.md index 8a1b448..95313bb 100644 --- a/docs/streaming.md +++ b/docs/streaming.md @@ -34,7 +34,8 @@ flat 2D desktop streaming into a window on the Frame's Linux desktop. | Option | Setup | Confidence | Verdict | |---|---|---|---| -| **macOS Screen Sharing (VNC) → Remmina on the Frame** | **Mac:** System Settings → General → Sharing → Screen Sharing on → (i) → enable "VNC viewers may control screen with password". **Frame:** `./scripts/install-apps.sh remmina` from the Mac, then open Remmina in the headset and connect to `vnc://.local` | **Inferred.** Remmina is on Flathub for **aarch64** with VNC and RDP ([Flathub](https://flathub.org/apps/org.remmina.Remmina)). The Frame desktop runs Flatpaks ([UploadVR](https://www.uploadvr.com/flatpaks-open-source-steam-frame/)). macOS VNC is built in. | **Recommended.** Nothing to install on the Mac, and it's easy to set up. Latency is fine for productivity but not for games. You'll type the Mac's hostname once in Remmina on the headset, then save the profile. To avoid even that, the script can pre-seed a Remmina profile over SSH (see below). | +| **Frame Control → Tools → Mac in the headset** | Nothing to install. Allow Screen Recording and Accessibility for Frame Control, then press **Show** next to any window or screen | **Verified** on the Mac and in an aarch64 Linux stand-in; **not yet tried in the headset** (2026-09-28) | **Recommended.** Hardware H.264 over an SSH tunnel. Each window becomes its own panel you can place anywhere. Laser clicks and scrolls, and the Mac's keyboard types. See [mac-in-headset.md](mac-in-headset.md) | +| **macOS Screen Sharing (VNC) → Remmina on the Frame** | **Mac:** System Settings → General → Sharing → Screen Sharing on → (i) → enable "VNC viewers may control screen with password". **Frame:** `./scripts/install-apps.sh remmina` from the Mac, then open Remmina in the headset and connect to `vnc://.local` | **Inferred.** Remmina is on Flathub for **aarch64** with VNC and RDP ([Flathub](https://flathub.org/apps/org.remmina.Remmina)). The Frame desktop runs Flatpaks ([UploadVR](https://www.uploadvr.com/flatpaks-open-source-steam-frame/)). macOS VNC is built in. | **Fallback.** Nothing to install on the Mac, and it's easy to set up. Latency is fine for productivity but not for games. You'll type the Mac's hostname once in Remmina on the headset, then save the profile. To avoid even that, the script can pre-seed a Remmina profile over SSH (see below). | | Sunshine (Mac) → Moonlight (Frame Flatpak) | `brew install` Sunshine on the Mac, then `./scripts/install-apps.sh moonlight` | Moonlight Flatpak supports **aarch64** ([Flathub](https://flathub.org/apps/com.moonlight_stream.Moonlight)). **Sunshine on macOS is poorly supported**: install problems on Apple Silicon/Sequoia, and no virtual gamepads ([LizardByte discussion #777](https://github.com/orgs/LizardByte/discussions/777)). | Try it if VNC is too laggy. Expect some friction. | | Steam Remote Play with the Mac as host | Steam on the Mac, Steam Link/Remote Play on the Frame | macOS-hosted Remote Play is reported broken or flaky in 2024–2026 ([Steam discussion](https://steamcommunity.com/groups/homestream/discussions/1/574921459914429988/)) | Not recommended. It's only for games, if it works at all. | | Immersed / Virtual Desktop | Vendor apps | Immersed has a Mac agent but no known Frame client. Virtual Desktop's developer said he'd "try" to port it ([NewsBreak](https://www.newsbreak.com/news/4892834783961-virtual-desktop-dev-says-he-ll-try-to-bring-the-app-to-steam-frame)). | Not available as of 2026-09-25. Check again later. | @@ -53,6 +54,21 @@ choose to save it. Remmina stores passwords encrypted with a per-install key, so the script doesn't try to write the password. (The Remmina file format is standard; the Flatpak data path is inferred.) +## First-party options, and why they do or don't fit + +Checked 2026-09-28. The first-party way is usually the best one, so these are +listed first; the sections above and below explain the alternatives. + +| Goal | First-party option | Fits? | Why, and what would make it easier | +|---|---|---|---| +| Type and point from the **iPhone** | **KDE Connect** (KDE; official [iOS app](https://apps.apple.com/app/kde-connect/id1580245991)) remote touchpad and keyboard, plus clipboard and files | **Best candidate, untested on the Frame** | The Frame doesn't have it (verified: no `kdeconnectd`), it isn't on Flathub, and the root is read-only, so it would have to run from `~` or a container. On Wayland it types through KWin, so it can only reach the desktop panel, not SteamVR or games (**inferred**). Steam Deck users report its remote input breaking after SteamOS updates ([SteamOS #1939](https://github.com/ValveSoftware/SteamOS/issues/1939)). If it works, Frame Control could install it and pair it for you. | +| Type and point from the **Mac** | KDE Connect for macOS (KDE builds) | Same as above | Its Mac app sends clipboard and files but has no keyboard/mouse sharing (**inferred**). | +| Either | **Bluetooth keyboard and mouse** paired in SteamOS (Valve) | Yes, with real hardware | Neither device can pretend to be one: iOS refuses the HID service ([Apple forums](https://developer.apple.com/forums/thread/733916)), and macOS has no built-in way. | +| Either | **xrdp** in Developer Mode (Valve) | No | Input goes into a *separate* desktop shown on the Mac, not into what you see in the headset. | +| **Mac screen** in the Frame | **Screen Sharing** (Apple's VNC server) + Remmina (already installed on this Frame, profile pre-seeded by `install-apps.sh`) | **Yes, closest to first-party** | Only the Mac side is first-party; Remmina is the client. Turn on System Settings → General → Sharing → Screen Sharing → (i) → "VNC viewers may control screen with password". Still to test in the headset (open question 11). | +| Mac screen | **Steam Remote Play** with the Mac as host (Valve) | Probably not | macOS isn't a SteamVR host, and Mac-hosted Remote Play is reported broken ([Steam forum](https://steamcommunity.com/groups/homestream/discussions/1/574921459914429988/)). One quick test is worth doing: Steam on the Mac, then Remote Play from the Frame's Steam. | +| Mac or **iPhone screen** | **AirPlay** (Apple) | Not officially | It's Apple's own mirroring for both, but Apple only licenses receivers to TV and speaker makers; nothing official runs on Linux. UxPlay (below) is the unofficial receiver. | + ## C. Show the iPhone's screen inside the Frame iOS only shares its screen two ways: **AirPlay** (Screen Mirroring in Control diff --git a/mac/frame-mac-view/Sources/Capture.swift b/mac/frame-mac-view/Sources/Capture.swift new file mode 100644 index 0000000..4967045 --- /dev/null +++ b/mac/frame-mac-view/Sources/Capture.swift @@ -0,0 +1,339 @@ +// Where pictures come from: one Mac window or one display (ScreenCaptureKit), +// or a generated test pattern that needs no permissions and shows the input +// the viewer sends, for checking the whole path without touching the Mac. +import AppKit +import CoreMedia +import CoreVideo +import Foundation +import ScreenCaptureKit + +enum Source: Equatable { + case window(CGWindowID) + case display(CGDirectDisplayID) + case test + + init?(_ s: String) { + let parts = s.split(separator: ":", maxSplits: 1).map(String.init) + switch (parts.first, parts.count > 1 ? UInt32(parts[1]) : nil) { + case ("window", let id?): self = .window(id) + case ("display", let id?): self = .display(id) + case ("test", _): self = .test + default: return nil + } + } + + var key: String { + switch self { + case .window(let id): return "window:\(id)" + case .display(let id): return "display:\(id)" + case .test: return "test" + } + } +} + +/// Size to capture at: the source's pixel size, shrunk to fit a box whose +/// long side is `maxLong` (default 1920), even numbers for the encoder. +func fitSize(width: Double, height: Double, maxLong: Int) -> (Int, Int) { + let scale = min(1, Double(maxLong) / max(width, height, 1)) + let w = max(16, Int((width * scale / 2).rounded()) * 2), h = max(16, Int((height * scale / 2).rounded()) * 2) + return (w, h) +} + +/// What CGWindowList says about a window right now (no permission needed for +/// bounds; titles need Screen Recording). +struct WindowInfo { + let id: CGWindowID + let pid: pid_t + let app: String + let title: String + let bounds: CGRect + let layer: Int + let onScreen: Bool + + static func all(onScreenOnly: Bool = true) -> [WindowInfo] { + let opts: CGWindowListOption = onScreenOnly ? [.optionOnScreenOnly, .excludeDesktopElements] : [.excludeDesktopElements] + let list = CGWindowListCopyWindowInfo(opts, kCGNullWindowID) as? [[CFString: Any]] ?? [] + return list.compactMap(WindowInfo.init) + } + + static func find(_ id: CGWindowID) -> WindowInfo? { + let list = CGWindowListCopyWindowInfo(.optionIncludingWindow, id) as? [[CFString: Any]] ?? [] + return list.compactMap(WindowInfo.init).first { $0.id == id } + } + + init?(_ d: [CFString: Any]) { + guard let id = d[kCGWindowNumber] as? CGWindowID, let pid = d[kCGWindowOwnerPID] as? pid_t, + let b = d[kCGWindowBounds] as? [String: Any], let rect = CGRect(dictionaryRepresentation: b as CFDictionary) + else { return nil } + self.id = id + self.pid = pid + app = d[kCGWindowOwnerName] as? String ?? "" + title = d[kCGWindowName] as? String ?? "" + bounds = rect + layer = d[kCGWindowLayer] as? Int ?? 0 + onScreen = d[kCGWindowIsOnscreen] as? Bool ?? false + } +} + +protocol CaptureSource: AnyObject { + var onFrame: ((CVPixelBuffer, CMTime) -> Void)? { get set } + var onEnded: ((String) -> Void)? { get set } + /// Points on the Mac's global display space that the picture covers. + var frameRect: CGRect { get } + var title: String { get } + var app: String { get } + var pid: pid_t? { get } + func start(maxLong: Int, fps: Int, completion: @escaping (String?) -> Void) + func stop() + func pointer(x: Double, y: Double, text: String?) // for the test pattern +} + +extension CaptureSource { + func pointer(x: Double, y: Double, text: String?) {} +} + +/// ScreenCaptureKit, for a window or a display. +final class SCKSource: NSObject, CaptureSource, SCStreamOutput, SCStreamDelegate { + let source: Source + var onFrame: ((CVPixelBuffer, CMTime) -> Void)? + var onEnded: ((String) -> Void)? + private(set) var frameRect = CGRect.zero + private(set) var title = "" + private(set) var app = "" + private(set) var pid: pid_t? + private var stream: SCStream? + private var config = SCStreamConfiguration() + private var maxLong = 1920 + private var scale = 2.0 + private var poll: DispatchSourceTimer? + private let queue = DispatchQueue(label: "frame-mac-view.capture", qos: .userInteractive) + /// Set by stop(), on `queue`; a start still enumerating windows checks it + /// before it starts capturing, so a viewer that left early leaves nothing on. + private var cancelled = false + /// The window or display no longer exists, so retrying can't help. + private(set) var gone = false + var onChange: (() -> Void)? // title or size changed + + init(_ source: Source) { self.source = source } + + func start(maxLong: Int, fps: Int, completion: @escaping (String?) -> Void) { + self.maxLong = maxLong + SCShareableContent.getExcludingDesktopWindows(true, onScreenWindowsOnly: false) { [self] content, error in + queue.async { self.begin(content, error, fps: fps, completion: completion) } + } + } + + private func begin(_ content: SCShareableContent?, _ error: Error?, fps: Int, completion: @escaping (String?) -> Void) { + if cancelled { return } + guard let content else { + return completion("Screen Recording isn't allowed for Frame Control (\(error?.localizedDescription ?? "no content"))") + } + let filter: SCContentFilter + switch source { + case .window(let id): + guard let w = content.windows.first(where: { $0.windowID == id }) else { + gone = true + return completion("that window has closed") + } + filter = SCContentFilter(desktopIndependentWindow: w) + title = w.title ?? "" + app = w.owningApplication?.applicationName ?? "" + pid = w.owningApplication?.processID + frameRect = w.frame + case .display(let id): + guard let d = content.displays.first(where: { $0.displayID == id }) else { + gone = true + return completion("that display isn't connected") + } + filter = SCContentFilter(display: d, excludingWindows: []) + title = displayName(id) + app = "Mac" + frameRect = CGDisplayBounds(id) + case .test: + return completion("not a ScreenCaptureKit source") + } + scale = Double(filter.pointPixelScale) + let (w, h) = fitSize(width: frameRect.width * scale, height: frameRect.height * scale, maxLong: maxLong) + config.width = w + config.height = h + config.minimumFrameInterval = CMTime(value: 1, timescale: CMTimeScale(fps)) + config.pixelFormat = kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange + config.colorMatrix = CGDisplayStream.yCbCrMatrix_ITU_R_709_2 + config.colorSpaceName = CGColorSpace.sRGB + config.showsCursor = true + config.queueDepth = 5 + config.scalesToFit = true + config.preservesAspectRatio = true + config.capturesAudio = false + let stream = SCStream(filter: filter, configuration: config, delegate: self) + do { + try stream.addStreamOutput(self, type: .screen, sampleHandlerQueue: queue) + } catch { + return completion("couldn't capture: \(error.localizedDescription)") + } + self.stream = stream + stream.startCapture { error in + self.queue.async { + if self.cancelled { + stream.stopCapture { _ in } + return + } + if let error { return completion("couldn't capture: \(error.localizedDescription)") } + self.startPolling() + completion(nil) + } + } + } + + func stop() { + queue.async { + self.cancelled = true + self.poll?.cancel() + self.poll = nil + self.stream?.stopCapture { _ in } + self.stream = nil + } + } + + /// Windows move, resize, retitle and close; ScreenCaptureKit doesn't say. + private func startPolling() { + guard case .window(let id) = source else { return } + let t = DispatchSource.makeTimerSource(queue: queue) + t.schedule(deadline: .now() + 1, repeating: 1) + t.setEventHandler { [weak self] in + guard let self else { return } + guard let info = WindowInfo.find(id) else { + self.stop() + self.onEnded?("the window closed") + return + } + var changed = false + if !info.title.isEmpty, info.title != self.title { self.title = info.title; changed = true } + let old = self.frameRect + self.frameRect = info.bounds + if abs(old.width - info.bounds.width) > 1 || abs(old.height - info.bounds.height) > 1 { + let (w, h) = fitSize(width: info.bounds.width * self.scale, height: info.bounds.height * self.scale, maxLong: self.maxLong) + self.config.width = w + self.config.height = h + self.stream?.updateConfiguration(self.config) { _ in } + changed = true + } + if changed { self.onChange?() } + } + t.resume() + poll = t + } + + func stream(_ stream: SCStream, didOutputSampleBuffer sample: CMSampleBuffer, of type: SCStreamOutputType) { + guard type == .screen, sample.isValid, let pb = CMSampleBufferGetImageBuffer(sample) else { return } + // Only complete frames carry new pixels; idle and blank ones don't. + if let atts = CMSampleBufferGetSampleAttachmentsArray(sample, createIfNecessary: false) as? [[SCStreamFrameInfo: Any]], + let raw = atts.first?[.status] as? Int, let status = SCFrameStatus(rawValue: raw), status != .complete { + return + } + onFrame?(pb, CMSampleBufferGetPresentationTimeStamp(sample)) + } + + func stream(_ stream: SCStream, didStopWithError error: Error) { + onEnded?("capture stopped: \(error.localizedDescription)") + } +} + +func displayName(_ id: CGDirectDisplayID) -> String { + for screen in NSScreen.screens { + if (screen.deviceDescription[NSDeviceDescriptionKey("NSScreenNumber")] as? NSNumber)?.uint32Value == id { + return screen.localizedName + } + } + return "Display \(id)" +} + +/// A moving test card: bars, a clock and a frame counter, plus a dot where the +/// viewer's pointer is and the last key it sent, so input can be checked too. +final class TestSource: CaptureSource { + var onFrame: ((CVPixelBuffer, CMTime) -> Void)? + var onEnded: ((String) -> Void)? + let frameRect = CGRect(x: 0, y: 0, width: 1280, height: 720) + let title = "Test pattern" + let app = "Frame Control" + let pid: pid_t? = nil + private var timer: DispatchSourceTimer? + private var pool: CVPixelBufferPool? + private var n = 0 + private var dot: (Double, Double)? + private var lastText = "Point or type in the headset" + private let queue = DispatchQueue(label: "frame-mac-view.test") + private var size = (1280, 720) + + func start(maxLong: Int, fps: Int, completion: @escaping (String?) -> Void) { + size = fitSize(width: 1280, height: 720, maxLong: maxLong) + let attrs: [CFString: Any] = [kCVPixelBufferPixelFormatTypeKey: kCVPixelFormatType_32BGRA, + kCVPixelBufferWidthKey: size.0, kCVPixelBufferHeightKey: size.1, + kCVPixelBufferIOSurfacePropertiesKey: [:] as CFDictionary] + CVPixelBufferPoolCreate(nil, nil, attrs as CFDictionary, &pool) + let t = DispatchSource.makeTimerSource(queue: queue) + t.schedule(deadline: .now(), repeating: 1.0 / Double(fps)) + t.setEventHandler { [weak self] in self?.draw() } + t.resume() + timer = t + completion(nil) + } + + func stop() { + timer?.cancel() + timer = nil + } + + func pointer(x: Double, y: Double, text: String?) { + queue.async { + if x >= 0 { self.dot = (x, y) } + if let text { self.lastText = text } + } + } + + private func draw() { + guard let pool else { return } + var out: CVPixelBuffer? + CVPixelBufferPoolCreatePixelBuffer(nil, pool, &out) + guard let pb = out else { return } + CVPixelBufferLockBaseAddress(pb, []) + defer { CVPixelBufferUnlockBaseAddress(pb, []) } + let (w, h) = size + guard let ctx = CGContext(data: CVPixelBufferGetBaseAddress(pb), width: w, height: h, bitsPerComponent: 8, + bytesPerRow: CVPixelBufferGetBytesPerRow(pb), space: CGColorSpace(name: CGColorSpace.sRGB)!, + bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue | CGBitmapInfo.byteOrder32Little.rawValue) + else { return } + let colors: [(CGFloat, CGFloat, CGFloat)] = [(0.75, 0.75, 0.75), (0.75, 0.75, 0), (0, 0.75, 0.75), (0, 0.75, 0), + (0.75, 0, 0.75), (0.75, 0, 0), (0, 0, 0.75)] + let bw = CGFloat(w) / CGFloat(colors.count) + for (i, c) in colors.enumerated() { + ctx.setFillColor(red: c.0, green: c.1, blue: c.2, alpha: 1) + ctx.fill(CGRect(x: CGFloat(i) * bw, y: CGFloat(h) * 0.35, width: bw + 1, height: CGFloat(h) * 0.65)) + } + ctx.setFillColor(red: 0.08, green: 0.09, blue: 0.11, alpha: 1) + ctx.fill(CGRect(x: 0, y: 0, width: w, height: Int(Double(h) * 0.35))) + // A bar sweeping once a second shows motion and dropped frames. + let x = CGFloat(n % 60) / 60 * CGFloat(w) + ctx.setFillColor(red: 1, green: 1, blue: 1, alpha: 1) + ctx.fill(CGRect(x: x, y: CGFloat(h) * 0.35, width: max(4, CGFloat(w) / 120), height: CGFloat(h) * 0.65)) + let f = DateFormatter() + f.dateFormat = "HH:mm:ss.SSS" + let label = "Frame Control test pattern \(f.string(from: Date())) frame \(n)\n\(lastText)" + let text = NSAttributedString(string: label, attributes: [ + .font: NSFont.monospacedSystemFont(ofSize: CGFloat(h) / 24, weight: .medium), + .foregroundColor: NSColor.white, + ]) + let ns = NSGraphicsContext(cgContext: ctx, flipped: false) + NSGraphicsContext.saveGraphicsState() + NSGraphicsContext.current = ns + text.draw(at: CGPoint(x: CGFloat(w) * 0.03, y: CGFloat(h) * 0.08)) + NSGraphicsContext.restoreGraphicsState() + if let (px, py) = dot { + let r = CGFloat(h) / 40 + ctx.setFillColor(red: 1, green: 0.2, blue: 0.2, alpha: 1) + ctx.fillEllipse(in: CGRect(x: CGFloat(px) * CGFloat(w) - r, y: (1 - CGFloat(py)) * CGFloat(h) - r, width: 2 * r, height: 2 * r)) + } + n += 1 + onFrame?(pb, CMClockGetTime(CMClockGetHostTimeClock())) + } +} diff --git a/mac/frame-mac-view/Sources/Encoder.swift b/mac/frame-mac-view/Sources/Encoder.swift new file mode 100644 index 0000000..114f403 --- /dev/null +++ b/mac/frame-mac-view/Sources/Encoder.swift @@ -0,0 +1,172 @@ +// VideoToolbox encoding: H.264 for the normal path (hardware, low-latency rate +// control, no B-frames, Annex B output that WebCodecs takes without a +// description), or JPEG stills for viewers that can't decode H.264. +import CoreMedia +import Foundation +import VideoToolbox + +enum Codec: String { + case h264, jpeg +} + +final class Encoder { + let codec: Codec + let fps: Int + let bitsPerPixel: Double + private(set) var width = 0 + private(set) var height = 0 + private var session: VTCompressionSession? + private var forceKey = true + private var lastPts = CMTime.invalid + /// Called on VideoToolbox's thread with one access unit (or JPEG) per frame. + var onFrame: ((Data, Bool, CMTime) -> Void)? + var onError: ((String) -> Void)? + /// Called once per frame handed to VideoToolbox, however it went. + var onDone: (() -> Void)? + + init(codec: Codec, fps: Int, bitsPerPixel: Double) { + self.codec = codec + self.fps = fps + self.bitsPerPixel = bitsPerPixel + } + + deinit { invalidate() } + + func requestKeyFrame() { forceKey = true } + + func invalidate() { + if let s = session { + VTCompressionSessionCompleteFrames(s, untilPresentationTimeStamp: .invalid) + VTCompressionSessionInvalidate(s) + } + session = nil + } + + private func makeSession(width w: Int, height h: Int) -> Bool { + invalidate() + var spec: [CFString: Any] = [:] + if codec == .h264 { spec[kVTVideoEncoderSpecification_EnableLowLatencyRateControl] = true } + var s: VTCompressionSession? + let type = codec == .h264 ? kCMVideoCodecType_H264 : kCMVideoCodecType_JPEG + func create(_ spec: [CFString: Any]) -> OSStatus { + VTCompressionSessionCreate(allocator: nil, width: Int32(w), height: Int32(h), codecType: type, + encoderSpecification: spec as CFDictionary, imageBufferAttributes: nil, + compressedDataAllocator: nil, outputCallback: nil, refcon: nil, + compressionSessionOut: &s) + } + var err = create(spec) + // Low-latency rate control needs Apple's hardware encoder; without it + // (some VMs), plain real-time encoding still works. + if err != noErr, !spec.isEmpty { err = create([:]) } + guard err == noErr, let s else { + onError?("couldn't start the \(codec.rawValue) encoder (VideoToolbox \(err))") + return false + } + func set(_ key: CFString, _ value: Any) { VTSessionSetProperty(s, key: key, value: value as CFTypeRef) } + set(kVTCompressionPropertyKey_RealTime, true) + set(kVTCompressionPropertyKey_ColorPrimaries, kCVImageBufferColorPrimaries_ITU_R_709_2) + set(kVTCompressionPropertyKey_TransferFunction, kCVImageBufferTransferFunction_ITU_R_709_2) + set(kVTCompressionPropertyKey_YCbCrMatrix, kCVImageBufferYCbCrMatrix_ITU_R_709_2) + if codec == .h264 { + let bps = min(max(Double(w * h * fps) * bitsPerPixel, 2_000_000), 60_000_000) + set(kVTCompressionPropertyKey_ProfileLevel, kVTProfileLevel_H264_ConstrainedHigh_AutoLevel) + set(kVTCompressionPropertyKey_AllowFrameReordering, false) + set(kVTCompressionPropertyKey_AverageBitRate, Int(bps)) + set(kVTCompressionPropertyKey_ExpectedFrameRate, fps) + // A keyframe every 10 s at most, so a viewer that lost one recovers + // even if it never asks. Viewers ask for one when they start. + set(kVTCompressionPropertyKey_MaxKeyFrameIntervalDuration, 10) + } else { + set(kVTCompressionPropertyKey_Quality, 0.8) + } + VTCompressionSessionPrepareToEncodeFrames(s) + session = s + lastPts = .invalid + width = w + height = h + forceKey = true + return true + } + + /// False if the frame never reached VideoToolbox (then onDone won't come). + @discardableResult + func encode(_ pb: CVPixelBuffer, pts given: CMTime) -> Bool { + // VideoToolbox needs strictly increasing timestamps; a resent picture + // stamped "now" can be followed by a capture stamped a moment earlier. + var pts = given + if lastPts.isValid, CMTimeCompare(pts, lastPts) <= 0 { pts = CMTimeAdd(lastPts, CMTime(value: 1, timescale: 1_000_000)) } + let w = CVPixelBufferGetWidth(pb), h = CVPixelBufferGetHeight(pb) + if session == nil || w != width || h != height { + guard makeSession(width: w, height: h) else { return false } + } + guard let s = session else { return false } + var props: CFDictionary? + if forceKey { + props = [kVTEncodeFrameOptionKey_ForceKeyFrame: true] as CFDictionary + forceKey = false + } + let codec = self.codec + lastPts = pts + let status = VTCompressionSessionEncodeFrame(s, imageBuffer: pb, presentationTimeStamp: pts, duration: .invalid, + frameProperties: props, infoFlagsOut: nil) { [weak self] status, _, sample in + guard let self else { return } + defer { self.onDone?() } + guard status == noErr, let sample else { return } + if codec == .jpeg { + if let data = Self.bytes(sample) { self.onFrame?(data, true, pts) } + } else if let (data, key) = Self.annexB(sample) { + self.onFrame?(data, key, pts) + } + } + if status != noErr { forceKey = true } + return status == noErr + } + + private static func bytes(_ sample: CMSampleBuffer) -> Data? { + guard let block = CMSampleBufferGetDataBuffer(sample) else { return nil } + var length = 0 + var ptr: UnsafeMutablePointer? + guard CMBlockBufferGetDataPointer(block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &length, + dataPointerOut: &ptr) == noErr, let ptr else { return nil } + return Data(bytes: ptr, count: length) + } + + /// AVCC sample -> Annex B access unit, with SPS and PPS before keyframes. + static func annexB(_ sample: CMSampleBuffer) -> (Data, Bool)? { + guard let avcc = bytes(sample) else { return nil } + var key = true + if let atts = CMSampleBufferGetSampleAttachmentsArray(sample, createIfNecessary: false) as? [[CFString: Any]], + let first = atts.first, first[kCMSampleAttachmentKey_NotSync] as? Bool == true { + key = false + } + let start: [UInt8] = [0, 0, 0, 1] + var out = Data() + if key, let fmt = CMSampleBufferGetFormatDescription(sample) { + var count = 0 + CMVideoFormatDescriptionGetH264ParameterSetAtIndex(fmt, parameterSetIndex: 0, parameterSetPointerOut: nil, + parameterSetSizeOut: nil, parameterSetCountOut: &count, + nalUnitHeaderLengthOut: nil) + for i in 0..? + var n = 0 + if CMVideoFormatDescriptionGetH264ParameterSetAtIndex(fmt, parameterSetIndex: i, parameterSetPointerOut: &p, + parameterSetSizeOut: &n, parameterSetCountOut: nil, + nalUnitHeaderLengthOut: nil) == noErr, let p { + out.append(contentsOf: start) + out.append(p, count: n) + } + } + } + let bytes = [UInt8](avcc) + var i = 0 + while i + 4 <= bytes.count { + let n = Int(bytes[i]) << 24 | Int(bytes[i + 1]) << 16 | Int(bytes[i + 2]) << 8 | Int(bytes[i + 3]) + i += 4 + guard n > 0, i + n <= bytes.count else { break } + out.append(contentsOf: start) + out.append(contentsOf: bytes[i..<(i + n)]) + i += n + } + return (out, key) + } +} diff --git a/mac/frame-mac-view/Sources/Input.swift b/mac/frame-mac-view/Sources/Input.swift new file mode 100644 index 0000000..0f51efc --- /dev/null +++ b/mac/frame-mac-view/Sources/Input.swift @@ -0,0 +1,168 @@ +// Turns the viewer's pointer and key events into real Mac input. Needs the +// Accessibility permission ("control your computer"); without it macOS drops +// the events silently, so the agent reports the permission to the viewer. +import AppKit +import ApplicationServices +import Foundation + +@_silgen_name("_AXUIElementGetWindow") +private func _AXUIElementGetWindow(_ element: AXUIElement, _ id: UnsafeMutablePointer) -> AXError + +enum Input { + static let source = CGEventSource(stateID: .hidSystemState) + /// What each viewer (session id) is holding down, so one panel closing + /// lets go of its own keys and buttons and nobody else's. Main thread. + private static var buttonsHeld: [Int: Set] = [:] + private static var keysHeld: [Int: Set] = [:] + + private static var lastDown: (time: TimeInterval, point: CGPoint, button: Int, count: Int)? + + static var allowed: Bool { AXIsProcessTrusted() } + + /// Brings a window to the front so a click lands on it, not on whatever + /// covers it, and typing goes to it. + static func focus(window id: CGWindowID, pid: pid_t) { + if frontWindow() == id { return } + let app = AXUIElementCreateApplication(pid) + var value: CFTypeRef? + if AXUIElementCopyAttributeValue(app, kAXWindowsAttribute as CFString, &value) == .success, + let windows = value as? [AXUIElement] { + for w in windows { + var wid: CGWindowID = 0 + if _AXUIElementGetWindow(w, &wid) == .success, wid == id { + AXUIElementPerformAction(w, kAXRaiseAction as CFString) + AXUIElementSetAttributeValue(w, kAXMainAttribute as CFString, kCFBooleanTrue) + break + } + } + } + AXUIElementSetAttributeValue(app, kAXFrontmostAttribute as CFString, kCFBooleanTrue) + NSRunningApplication(processIdentifier: pid)?.activate() + } + + static func frontWindow() -> CGWindowID? { + WindowInfo.all().first { $0.layer == 0 }?.id + } + + /// `button` uses the browser's numbering: 0 left, 1 middle, 2 right. + static func mouse(_ kind: String, button: Int, at p: CGPoint, owner: Int) { + let b: CGMouseButton = button == 2 ? .right : button == 1 ? .center : .left + let type: CGEventType + switch kind { + case "down": + type = b == .left ? .leftMouseDown : b == .right ? .rightMouseDown : .otherMouseDown + buttonsHeld[owner, default: []].insert(button) + case "up": + type = b == .left ? .leftMouseUp : b == .right ? .rightMouseUp : .otherMouseUp + buttonsHeld[owner]?.remove(button) + // Another viewer still holding it keeps it down. + if buttonsHeld.values.contains(where: { $0.contains(button) }) { return } + default: // a drag is whatever this viewer is holding + let mine = buttonsHeld[owner] ?? [] + if mine.contains(0) { type = .leftMouseDragged } + else if mine.contains(2) { type = .rightMouseDragged } + else if mine.contains(1) { type = .otherMouseDragged } + else { type = .mouseMoved } + } + let mine = buttonsHeld[owner] ?? [] + let held: CGMouseButton = mine.contains(0) ? .left : mine.contains(2) ? .right : mine.contains(1) ? .center : .left + guard let e = CGEvent(mouseEventSource: source, mouseType: type, mouseCursorPosition: p, + mouseButton: kind == "move" ? held : b) else { return } + if kind == "down" || kind == "up" { + let now = ProcessInfo.processInfo.systemUptime + var count = 1 + if kind == "down" { + if let l = lastDown, l.button == button, now - l.time < NSEvent.doubleClickInterval, + abs(l.point.x - p.x) < 5, abs(l.point.y - p.y) < 5 { count = l.count + 1 } + lastDown = (now, p, button, count) + } else if let l = lastDown, l.button == button { + count = l.count + } + e.setIntegerValueField(.mouseEventClickState, value: Int64(count)) + } + e.post(tap: .cghidEventTap) + } + + static func scroll(dx: Double, dy: Double, at p: CGPoint, owner: Int) { + mouse("move", button: 0, at: p, owner: owner) + // Browsers report pixels with +y meaning "scroll down"; macOS's +y is up. + guard let e = CGEvent(scrollWheelEvent2Source: source, units: .pixel, wheelCount: 2, + wheel1: Int32(-dy.rounded()), wheel2: Int32(-dx.rounded()), wheel3: 0) else { return } + e.location = p + e.post(tap: .cghidEventTap) + } + + /// Lets go of every button and key this viewer is holding down, so a + /// dropped connection can't leave, say, Shift or ⌘ stuck on. + static func releaseAll(owner: Int) { + let p = CGEvent(source: nil)?.location ?? .zero + let buttons = buttonsHeld.removeValue(forKey: owner) ?? [] + let keys = keysHeld.removeValue(forKey: owner) ?? [] + // Only what no other viewer is still holding. + for b in buttons where !buttonsHeld.values.contains(where: { $0.contains(b) }) { + let type: CGEventType = b == 2 ? .rightMouseUp : b == 1 ? .otherMouseUp : .leftMouseUp + CGEvent(mouseEventSource: source, mouseType: type, mouseCursorPosition: p, + mouseButton: b == 2 ? .right : b == 1 ? .center : .left)?.post(tap: .cghidEventTap) + } + for k in keys where !keysHeld.values.contains(where: { $0.contains(k) }) { + CGEvent(keyboardEventSource: source, virtualKey: k, keyDown: false)?.post(tap: .cghidEventTap) + } + } + + static func key(code: String, key: String, down: Bool, mods: [String], owner: Int) { + var flags = CGEventFlags() + if mods.contains("shift") { flags.insert(.maskShift) } + if mods.contains("ctrl") { flags.insert(.maskControl) } + if mods.contains("alt") { flags.insert(.maskAlternate) } + if mods.contains("meta") { flags.insert(.maskCommand) } + if let vk = keyCodes[code] { + guard let e = CGEvent(keyboardEventSource: source, virtualKey: vk, keyDown: down) else { return } + if down { + keysHeld[owner, default: []].insert(vk) + } else { + keysHeld[owner]?.remove(vk) + if keysHeld.values.contains(where: { $0.contains(vk) }) { return } // still held elsewhere + } + e.flags = flags + e.post(tap: .cghidEventTap) + } else if down, !key.isEmpty, key.count <= 4, key.unicodeScalars.allSatisfy({ $0.value >= 0x20 }) { + // A key the table doesn't know (a non-US layout, the headset's + // on-screen keyboard): type its character instead. + text(key) + } + } + + static func text(_ s: String) { + let units = Array(s.utf16) + for chunk in stride(from: 0, to: units.count, by: 16) { + let part = Array(units[chunk.. macOS virtual key code (ANSI positions). + static let keyCodes: [String: CGKeyCode] = [ + "KeyA": 0x00, "KeyS": 0x01, "KeyD": 0x02, "KeyF": 0x03, "KeyH": 0x04, "KeyG": 0x05, "KeyZ": 0x06, "KeyX": 0x07, + "KeyC": 0x08, "KeyV": 0x09, "IntlBackslash": 0x0A, "KeyB": 0x0B, "KeyQ": 0x0C, "KeyW": 0x0D, "KeyE": 0x0E, + "KeyR": 0x0F, "KeyY": 0x10, "KeyT": 0x11, "Digit1": 0x12, "Digit2": 0x13, "Digit3": 0x14, "Digit4": 0x15, + "Digit6": 0x16, "Digit5": 0x17, "Equal": 0x18, "Digit9": 0x19, "Digit7": 0x1A, "Minus": 0x1B, "Digit8": 0x1C, + "Digit0": 0x1D, "BracketRight": 0x1E, "KeyO": 0x1F, "KeyU": 0x20, "BracketLeft": 0x21, "KeyI": 0x22, + "KeyP": 0x23, "Enter": 0x24, "KeyL": 0x25, "KeyJ": 0x26, "Quote": 0x27, "KeyK": 0x28, "Semicolon": 0x29, + "Backslash": 0x2A, "Comma": 0x2B, "Slash": 0x2C, "KeyN": 0x2D, "KeyM": 0x2E, "Period": 0x2F, "Tab": 0x30, + "Space": 0x31, "Backquote": 0x32, "Backspace": 0x33, "Escape": 0x35, "MetaRight": 0x36, "MetaLeft": 0x37, + "ShiftLeft": 0x38, "CapsLock": 0x39, "AltLeft": 0x3A, "ControlLeft": 0x3B, "ShiftRight": 0x3C, + "AltRight": 0x3D, "ControlRight": 0x3E, "F17": 0x40, "NumpadDecimal": 0x41, "NumpadMultiply": 0x43, + "NumpadAdd": 0x45, "NumLock": 0x47, "NumpadDivide": 0x4B, "NumpadEnter": 0x4C, "NumpadSubtract": 0x4E, + "F18": 0x4F, "F19": 0x50, "NumpadEqual": 0x51, "Numpad0": 0x52, "Numpad1": 0x53, "Numpad2": 0x54, + "Numpad3": 0x55, "Numpad4": 0x56, "Numpad5": 0x57, "Numpad6": 0x58, "Numpad7": 0x59, "F20": 0x5A, + "Numpad8": 0x5B, "Numpad9": 0x5C, "F5": 0x60, "F6": 0x61, "F7": 0x62, "F3": 0x63, "F8": 0x64, "F9": 0x65, + "F11": 0x67, "F13": 0x69, "F16": 0x6A, "F14": 0x6B, "F10": 0x6D, "ContextMenu": 0x6E, "F12": 0x6F, + "F15": 0x71, "Insert": 0x72, "Help": 0x72, "Home": 0x73, "PageUp": 0x74, "Delete": 0x75, "F4": 0x76, + "End": 0x77, "F2": 0x78, "PageDown": 0x79, "F1": 0x7A, "ArrowLeft": 0x7B, "ArrowRight": 0x7C, + "ArrowDown": 0x7D, "ArrowUp": 0x7E, "OSLeft": 0x37, "OSRight": 0x36, + ] +} diff --git a/mac/frame-mac-view/Sources/Server.swift b/mac/frame-mac-view/Sources/Server.swift new file mode 100644 index 0000000..83db9e3 --- /dev/null +++ b/mac/frame-mac-view/Sources/Server.swift @@ -0,0 +1,267 @@ +// A small HTTP/1.1 + WebSocket server on Network.framework, loopback only. +// Enough for the agent's JSON endpoints, the viewer page and one WebSocket per +// stream; not a general web server. +import CryptoKit +import Foundation +import Network + +struct Request { + let method: String + let path: String + let query: [String: String] + let headers: [String: String] // lower-cased names +} + +final class Server { + let queue = DispatchQueue(label: "frame-mac-view.server") + private let listener: NWListener + private let handle: (Request, HTTPConnection) -> Void + + init(port: UInt16, handle: @escaping (Request, HTTPConnection) -> Void) throws { + let tcp = NWProtocolTCP.Options() + tcp.noDelay = true + let params = NWParameters(tls: nil, tcp: tcp) + // Port 0 lets the system pick; `port` then says which. + params.requiredLocalEndpoint = .hostPort(host: "127.0.0.1", port: NWEndpoint.Port(rawValue: port) ?? .any) + params.allowLocalEndpointReuse = true + listener = try NWListener(using: params) + self.handle = handle + } + + var port: UInt16? { listener.port?.rawValue } + + func start(ready: @escaping (Error?) -> Void) { + listener.stateUpdateHandler = { state in + switch state { + case .ready: ready(nil) + case .failed(let e): ready(e) + default: break + } + } + listener.newConnectionHandler = { [weak self] conn in + guard let self else { return } + HTTPConnection(conn, queue: self.queue, handle: self.handle).start() + } + listener.start(queue: queue) + } +} + +final class HTTPConnection { + let conn: NWConnection + let queue: DispatchQueue + private let handle: (Request, HTTPConnection) -> Void + private var buffer = Data() + private var retained: HTTPConnection? // alive until the response is sent + + init(_ conn: NWConnection, queue: DispatchQueue, handle: @escaping (Request, HTTPConnection) -> Void) { + self.conn = conn + self.queue = queue + self.handle = handle + } + + func start() { + retained = self + conn.start(queue: queue) + readHead() + } + + private func readHead() { + conn.receive(minimumIncompleteLength: 1, maximumLength: 16384) { [self] data, _, done, error in + if let data { buffer.append(data) } + if let end = buffer.range(of: Data("\r\n\r\n".utf8)) { + guard let req = Self.parse(buffer[.. 16384 { + close() + } else { + readHead() + } + } + } + + static func parse(_ head: Data) -> Request? { + guard let text = String(data: head, encoding: .utf8) else { return nil } + let lines = text.components(separatedBy: "\r\n") + let parts = lines[0].split(separator: " ") + guard parts.count >= 2, let url = URLComponents(string: String(parts[1])) else { return nil } + var headers: [String: String] = [:] + for line in lines.dropFirst() { + guard let colon = line.firstIndex(of: ":") else { continue } + headers[line[.. WebSocket? { + guard req.headers["upgrade"]?.lowercased() == "websocket", let key = req.headers["sec-websocket-key"] else { + respond(400, text: "expected a WebSocket upgrade") + return nil + } + let accept = Data(Insecure.SHA1.hash(data: Data((key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11").utf8))).base64EncodedString() + let head = "HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: \(accept)\r\n\r\n" + conn.send(content: Data(head.utf8), completion: .contentProcessed { _ in }) + let ws = WebSocket(conn, queue: queue) + retained = nil + return ws + } +} + +/// Server side of RFC 6455. Sends are counted until the network stack has +/// taken them, so the stream can skip frames instead of queueing seconds of +/// video on a slow link. +final class WebSocket { + static let maxMessage: UInt64 = 1 << 20 + let conn: NWConnection + let queue: DispatchQueue + var onText: ((String) -> Void)? + var onClose: (() -> Void)? + private var buffer = Data() + private var fragments = Data() + private var fragmentOpcode: UInt8 = 0 + private var closed = false + private var retained: WebSocket? + private let lock = NSLock() + private var _pending = 0 + var onDrain: (() -> Void)? + + /// Bytes handed to the connection that it hasn't sent yet. Any thread. + var pendingBytes: Int { lock.lock(); defer { lock.unlock() }; return _pending } + + init(_ conn: NWConnection, queue: DispatchQueue) { + self.conn = conn + self.queue = queue + } + + func start() { + retained = self + read() + } + + func sendText(_ s: String) { send(opcode: 1, Data(s.utf8)) } + func sendJSON(_ obj: Any) { + if let d = try? JSONSerialization.data(withJSONObject: obj), let s = String(data: d, encoding: .utf8) { sendText(s) } + } + func sendBinary(_ d: Data) { send(opcode: 2, d) } + + func send(opcode: UInt8, _ payload: Data) { + var frame = Data([0x80 | opcode]) + let n = payload.count + if n < 126 { + frame.append(UInt8(n)) + } else if n < 65536 { + frame.append(126) + frame.append(contentsOf: [UInt8(n >> 8), UInt8(n & 0xff)]) + } else { + frame.append(127) + for shift in stride(from: 56, through: 0, by: -8) { frame.append(UInt8((UInt64(n) >> UInt64(shift)) & 0xff)) } + } + frame.append(payload) + let size = frame.count + lock.lock(); _pending += size; lock.unlock() + conn.send(content: frame, completion: .contentProcessed { [weak self] _ in + guard let self else { return } + self.lock.lock(); self._pending -= size; self.lock.unlock() + self.onDrain?() + }) + } + + func close() { + guard !closed else { return } + closed = true + send(opcode: 8, Data([0x03, 0xe8])) // 1000, normal closure + conn.send(content: nil, isComplete: true, completion: .contentProcessed { [weak self] _ in self?.conn.cancel() }) + finish() + } + + private func finish() { + let cb = onClose + onClose = nil + onText = nil + onDrain = nil + cb?() + retained = nil + } + + private func read() { + conn.receive(minimumIncompleteLength: 1, maximumLength: 65536) { [weak self] data, _, done, error in + guard let self, !self.closed else { return } + if let data { self.buffer.append(data) } + self.parse() + if error != nil || done { + self.closed = true + self.conn.cancel() + self.finish() + } else if !self.closed { + self.read() + } + } + } + + private func parse() { + while buffer.count >= 2 { + let b = [UInt8](buffer.prefix(14)) + let fin = b[0] & 0x80 != 0, opcode = b[0] & 0x0f, masked = b[1] & 0x80 != 0 + // Lengths are unsigned and clients only send small control + // messages, so anything big (or a 64-bit length with the top bit + // set) is refused before it's turned into an Int. + var len64 = UInt64(b[1] & 0x7f), off = 2 + if len64 == 126 { + guard b.count >= 4 else { return } + len64 = UInt64(b[2]) << 8 | UInt64(b[3]); off = 4 + } else if len64 == 127 { + guard b.count >= 10 else { return } + len64 = 0 + for i in 2..<10 { len64 = len64 << 8 | UInt64(b[i]) } + off = 10 + } + guard len64 <= WebSocket.maxMessage else { return close() } + let len = Int(len64) + let maskOff = off + if masked { off += 4 } + guard buffer.count >= off + len else { return } + let start = buffer.startIndex + var payload = Data(buffer[(start + off)..<(start + off + len)]) + if masked { + let mask = [UInt8](buffer[(start + maskOff)..<(start + maskOff + 4)]) + payload.withUnsafeMutableBytes { p in + for i in 0.., display: or test. +import AppKit +import ApplicationServices +import Foundation +import IOKit.pwr_mgt +import ScreenCaptureKit +import Security + +let version = "1" + +func windowsJSON() -> [[String: Any]] { + let me = ProcessInfo.processInfo.processIdentifier + let skip: Set = ["Window Server", "Dock", "Control Centre", "Control Center", "Notification Centre", + "Notification Center", "Spotlight", "SystemUIServer", "Wallpaper", "WindowManager"] + return WindowInfo.all().filter { + $0.layer == 0 && $0.pid != me && !skip.contains($0.app) && $0.bounds.width >= 120 && $0.bounds.height >= 80 + }.map { + ["id": $0.id, "src": "window:\($0.id)", "pid": $0.pid, "app": $0.app, "title": $0.title, + "w": Int($0.bounds.width), "h": Int($0.bounds.height)] + } +} + +func displaysJSON() -> [[String: Any]] { + var ids = [CGDirectDisplayID](repeating: 0, count: 16) + var n: UInt32 = 0 + // Online, not active: a display that's asleep is still one you can stream. + CGGetOnlineDisplayList(16, &ids, &n) + return ids.prefix(Int(n)).filter { CGDisplayMirrorsDisplay($0) == kCGNullDirectDisplay }.map { id in + let b = CGDisplayBounds(id) + return ["id": id, "src": "display:\(id)", "name": displayName(id), "w": Int(b.width), "h": Int(b.height), + "main": CGDisplayIsMain(id) != 0] + } +} + +func permissionsJSON() -> [String: Any] { + ["screen": CGPreflightScreenCaptureAccess(), "accessibility": AXIsProcessTrusted()] +} + +func printJSON(_ obj: Any) { + let d = (try? JSONSerialization.data(withJSONObject: obj, options: [.sortedKeys, .prettyPrinted])) ?? Data() + FileHandle.standardOutput.write(d + Data("\n".utf8)) +} + +/// One viewer watching one source. +final class Session { + let id: Int + let source: Source + let capture: CaptureSource + let encoder: Encoder + let ws: WebSocket + let codec: Codec + let reconnectKey: String + private let lock = NSLock() + private var last: (CVPixelBuffer, CMTime)? + private var skipped = false + private var stopped = false + private var inFlight = 0 + /// Frames already queued for the network; beyond this, or with two frames + /// already in the encoder, new frames are skipped (before encoding, so no + /// reference frame goes missing) and the newest picture is sent once + /// things catch up. Only that newest picture is kept meanwhile. + let maxPending: Int + static let maxInFlight = 2 + var onEnd: ((Session) -> Void)? + var onAck: (() -> Void)? + var onFinished: ((String) -> Void)? + private let encodeQueue = DispatchQueue(label: "frame-mac-view.encode", qos: .userInteractive) + + init(id: Int, source: Source, capture: CaptureSource, ws: WebSocket, codec: Codec, fps: Int, bitsPerPixel: Double, + reconnectKey: String) { + self.id = id + self.source = source + self.capture = capture + self.ws = ws + self.codec = codec + self.reconnectKey = reconnectKey + encoder = Encoder(codec: codec, fps: fps, bitsPerPixel: bitsPerPixel) + maxPending = codec == .jpeg ? 3 << 20 : 1 << 20 + } + + /// Encodes `pb` unless the link or the encoder is busy, in which case it + /// becomes the picture to send next. + private func offer(_ pb: CVPixelBuffer, pts: CMTime) { + lock.lock() + last = (pb, pts) + let busy = stopped || ws.pendingBytes > maxPending || inFlight >= Session.maxInFlight + if busy { skipped = true } else { inFlight += 1 } + lock.unlock() + if !busy { submit(pb, pts: pts) } + } + + private func submit(_ pb: CVPixelBuffer, pts: CMTime) { + encodeQueue.async { + if !self.encoder.encode(pb, pts: pts) { self.finished() } + } + } + + /// An encode finished (or failed): send the newest skipped picture if + /// there's room now. + private func finished() { + lock.lock() + inFlight = max(0, inFlight - 1) + lock.unlock() + resendIfRoom() + } + + private func resendIfRoom() { + lock.lock() + var next: (CVPixelBuffer, CMTime)? + if !stopped, skipped, ws.pendingBytes <= maxPending / 2, inFlight < Session.maxInFlight, let l = last { + next = l + skipped = false + inFlight += 1 + } + lock.unlock() + if let (pb, _) = next { submit(pb, pts: CMClockGetTime(CMClockGetHostTimeClock())) } + } + + func start(maxLong: Int, fps: Int) { + encoder.onFrame = { [weak self] data, key, pts in + guard let self else { return } + var msg = Data([key ? 1 : 0]) + var us = UInt64(max(0, CMTimeGetSeconds(pts)) * 1_000_000).bigEndian + msg.append(Data(bytes: &us, count: 8)) + msg.append(data) + self.ws.sendBinary(msg) + } + encoder.onDone = { [weak self] in self?.finished() } + encoder.onError = { [weak self] message in self?.ws.sendJSON(["t": "error", "message": message]) } + capture.onFrame = { [weak self] pb, pts in self?.offer(pb, pts: pts) } + capture.onEnded = { [weak self] reason in + guard let self else { return } + self.ws.sendJSON(["t": "closed", "reason": reason]) + self.onFinished?(self.source.key) + self.end() + } + ws.onDrain = { [weak self] in self?.resendIfRoom() } + ws.onText = { [weak self] text in self?.handle(text) } + ws.onClose = { [weak self] in self?.end() } + ws.start() + // Reconnect with this (kept in the page's memory, never on a command line). + ws.sendJSON(["t": "hello", "r": reconnectKey]) + if let sck = capture as? SCKSource { + sck.onChange = { [weak self] in self?.sendInfo() } + } + capture.start(maxLong: maxLong, fps: fps) { [weak self] error in + guard let self else { return } + if let error { + if (self.capture as? SCKSource)?.gone == true { + // Final, like a window closing mid-stream: the viewer closes + // and its key is revoked, rather than retrying for ever. + self.ws.sendJSON(["t": "closed", "reason": error]) + self.onFinished?(self.source.key) + } else { + self.ws.sendJSON(["t": "error", "message": error]) + } + self.end() + return + } + self.sendInfo() + } + } + + func sendInfo() { + ws.sendJSON(["t": "info", "src": source.key, "title": capture.title, "app": capture.app, "codec": codec.rawValue, + "input": source == .test || Input.allowed, + "aspect": Double(capture.frameRect.width / max(capture.frameRect.height, 1))]) + } + + var isStopped: Bool { lock.lock(); defer { lock.unlock() }; return stopped } + + func end() { + lock.lock() + let was = stopped + stopped = true + last = nil + lock.unlock() + guard !was else { return } + capture.stop() + encodeQueue.async { self.encoder.invalidate() } + let owner = id + DispatchQueue.main.async { Input.releaseAll(owner: owner) } + ws.close() + onEnd?(self) + } + + /// A point in the picture (0...1 each way) -> Mac global coordinates. + private func point(_ x: Double, _ y: Double) -> CGPoint { + let r = capture.frameRect + return CGPoint(x: r.minX + min(max(x, 0), 1) * r.width, y: r.minY + min(max(y, 0), 1) * r.height) + } + + private func handle(_ text: String) { + guard !isStopped, let d = text.data(using: .utf8), + let m = try? JSONSerialization.jsonObject(with: d) as? [String: Any], let t = m["t"] as? String else { return } + let x = m["x"] as? Double ?? -1, y = m["y"] as? Double ?? -1 + if t == "ack" { + onAck?() + onAck = nil + return + } + if t == "key-frame" { + encodeQueue.async { self.encoder.requestKeyFrame() } // before the resend, same queue + lock.lock(); skipped = last != nil; lock.unlock() + resendIfRoom() + return + } + if source == .test { + let desc: String? + switch t { + case "m": desc = (m["e"] as? String) == "move" ? nil : "mouse \(m["e"] ?? "") button \(m["b"] ?? 0)" + case "wheel": desc = "wheel \(m["dx"] ?? 0), \(m["dy"] ?? 0)" + case "k": desc = (m["e"] as? String) == "down" ? "key \(m["code"] ?? "") \"\(m["key"] ?? "")\"" : nil + case "text": desc = "text \"\(m["s"] ?? "")\"" + default: desc = nil + } + capture.pointer(x: x, y: y, text: desc) + return + } + DispatchQueue.main.async { [self] in + guard !isStopped else { return } // Stop revokes input at once + switch t { + case "m": + let kind = m["e"] as? String ?? "move", b = m["b"] as? Int ?? 0 + let p = point(x, y) + if kind == "down", case .window(let wid) = source, let pid = capture.pid { + Input.focus(window: wid, pid: pid) + } + Input.mouse(kind, button: b, at: p, owner: id) + case "wheel": + Input.scroll(dx: m["dx"] as? Double ?? 0, dy: m["dy"] as? Double ?? 0, at: point(x, y), owner: id) + case "k": + Input.key(code: m["code"] as? String ?? "", key: m["key"] as? String ?? "", + down: (m["e"] as? String) == "down", mods: m["mods"] as? [String] ?? [], owner: id) + case "text": + if let s = m["s"] as? String, s.count <= 4096 { Input.text(s) } + case "release": + Input.releaseAll(owner: id) + case "focus": + if case .window(let wid) = source, let pid = capture.pid { Input.focus(window: wid, pid: pid) } + default: break + } + } + } +} + +func randomKey() -> String { + var bytes = [UInt8](repeating: 0, count: 24) + _ = SecRandomCopyBytes(kSecRandomDefault, bytes.count, &bytes) + return Data(bytes).base64EncodedString().replacingOccurrences(of: "+", with: "-") + .replacingOccurrences(of: "/", with: "_").replacingOccurrences(of: "=", with: "") +} + +func sameSecret(_ a: String, _ b: String) -> Bool { + guard !a.isEmpty, a.utf8.count == b.utf8.count else { return false } + return zip(a.utf8, b.utf8).reduce(0, { $0 | ($1.0 ^ $1.1) }) == 0 +} + +final class Agent { + /// Frame Control's own key. It stays on the Mac: the Frame only ever sees + /// single-use tickets and per-stream reconnect keys, both tied to one source. + let token: String + let page: URL? + var sessions: [Int: Session] = [:] { didSet { keepDisplayAwake(!sessions.isEmpty) } } + var nextId = 1 + let lock = NSLock() + private var assertion: IOPMAssertionID = 0 + /// ticket -> (source, expiry, reconnect key once redeemed). A ticket opens + /// one viewer within a minute; it may be redeemed again, for the same key, + /// only until that viewer confirms it has the key ("ack"). + private var tickets: [String: (src: String, expiry: Date, key: String?)] = [:] + /// reconnect key -> source, until Stop for that source. + private var reconnectKeys: [String: String] = [:] + /// Sources that ended for good (the window closed), for Frame Control. + private var finished = Set() + + /// While anyone watches, keep the Mac's display on: a sleeping display + /// stops being drawn, so there'd be nothing to capture (and it would lock). + private func keepDisplayAwake(_ on: Bool) { + if on, assertion == 0 { + IOPMAssertionCreateWithName(kIOPMAssertionTypePreventUserIdleDisplaySleep as CFString, + IOPMAssertionLevel(kIOPMAssertionLevelOn), + "Frame Control is showing this Mac in a Steam Frame" as CFString, &assertion) + } else if !on, assertion != 0 { + IOPMAssertionRelease(assertion) + assertion = 0 + } + } + + init(token: String, page: URL?) { + self.token = token + self.page = page + } + + /// Whether this request may open a stream of `src`: Frame Control's key, + /// an unused ticket for that source, or a live reconnect key for it. + private func mayStream(_ req: Request, src: String) -> String? { + lock.lock() + defer { lock.unlock() } + let now = Date() + tickets = tickets.filter { $0.value.expiry > now } + if sameSecret(req.query["k"] ?? "", token) { return randomKey() } + if let t = req.query["t"], let entry = tickets[t], entry.src == src { + // A retry before the viewer got its key gets the same key back. + let key = entry.key ?? randomKey() + tickets[t] = (src, entry.expiry, key) + reconnectKeys[key] = src + return key + } + if let r = req.query["r"], reconnectKeys[r] == src { return r } + return nil + } + + /// The viewer has its reconnect key, so the ticket that led to it can't + /// be used again (also when the ack comes over a reconnection). + func acknowledged(key: String) { + lock.lock(); tickets = tickets.filter { $0.value.key != key }; lock.unlock() + } + + func handle(_ req: Request, _ c: HTTPConnection) { + // Open to anything that reaches the port: a liveness check for the + // tunnel, and the viewer page, which holds no secrets. + switch (req.method, req.path) { + case ("GET", "/ping"): return c.respond(200, text: "frame-mac-view") + case ("GET", "/view"): + guard let page, let body = try? Data(contentsOf: page) else { return c.respond(404, text: "no viewer page") } + return c.respond(200, body: body, type: "text/html; charset=utf-8") + case ("GET", "/stream"): + guard let src = Source(req.query["src"] ?? "") else { return c.respond(400, text: "bad src") } + guard let key = mayStream(req, src: src.key) else { return c.respond(403, text: "forbidden") } + return stream(req, c, src: src, key: key) + default: break + } + guard sameSecret(req.query["k"] ?? req.headers["x-token"] ?? "", token) else { + return c.respond(403, text: "forbidden") + } + switch (req.method, req.path) { + case ("GET", "/status"): + lock.lock() + let list = sessions.values.map { ["id": $0.id, "src": $0.source.key, "title": $0.capture.title, "app": $0.capture.app] } + lock.unlock() + var s = permissionsJSON() + s["version"] = version + s["streams"] = list + lock.lock(); s["finished"] = Array(finished); lock.unlock() + c.respond(json: s) + case ("GET", "/windows"): + c.respond(json: ["windows": windowsJSON(), "screen": CGPreflightScreenCaptureAccess()]) + case ("GET", "/displays"): + c.respond(json: ["displays": displaysJSON()]) + case ("POST", "/ticket"): + guard let src = Source(req.query["src"] ?? "") else { return c.respond(400, text: "bad src") } + let t = randomKey() + lock.lock() + tickets[t] = (src.key, Date().addingTimeInterval(60), nil) + finished.remove(src.key) + lock.unlock() + c.respond(json: ["ticket": t]) + case ("POST", "/close"): + // Tell viewers to close their windows, but don't rely on them: + // the sessions end here and can't reconnect. + let src = req.query["src"] + lock.lock() + let matching = sessions.values.filter { src == nil || $0.source.key == src } + reconnectKeys = reconnectKeys.filter { src != nil && $0.value != src } + tickets = tickets.filter { src != nil && $0.value.src != src } // not yet used ones too + lock.unlock() + for s in matching { s.ws.sendJSON(["t": "close"]) } + DispatchQueue.global().asyncAfter(deadline: .now() + 0.3) { for s in matching { s.end() } } + c.respond(json: ["closed": matching.count]) + case ("POST", "/permissions"): + DispatchQueue.main.async { requestPermissions() } + c.respond(json: permissionsJSON()) + default: + c.respond(404, text: "not found") + } + } + + private func stream(_ req: Request, _ c: HTTPConnection, src: Source, key: String) { + let codec = Codec(rawValue: req.query["codec"] ?? "h264") ?? .h264 + let maxLong = min(max(Int(req.query["max"] ?? "") ?? 1920, 320), 3840) + let fps = min(max(Int(req.query["fps"] ?? "") ?? 60, 5), 120) + let bpp = min(max(Double(req.query["bpp"] ?? "") ?? 0.1, 0.02), 0.5) + guard let ws = c.upgrade(req) else { return } + let capture: CaptureSource = src == .test ? TestSource() : SCKSource(src) + lock.lock() + // One live viewer per key: a reconnection (or a second use of an + // unacknowledged ticket) replaces the one before. + let replaced = sessions.values.filter { $0.reconnectKey == key } + let session = Session(id: nextId, source: src, capture: capture, ws: ws, codec: codec, fps: fps, + bitsPerPixel: bpp, reconnectKey: key) + nextId += 1 + sessions[session.id] = session + lock.unlock() + for old in replaced { old.end() } + session.onEnd = { [weak self] s in + guard let self else { return } + self.lock.lock(); self.sessions[s.id] = nil; self.lock.unlock() + } + // The source is gone (its window closed): its viewers can't come back. + session.onFinished = { [weak self] src in + guard let self else { return } + self.lock.lock() + self.finished.insert(src) + self.reconnectKeys = self.reconnectKeys.filter { $0.value != src } + self.lock.unlock() + } + session.onAck = { [weak self] in self?.acknowledged(key: key) } + session.start(maxLong: maxLong, fps: fps) + } +} + +func requestPermissions() { + if !CGPreflightScreenCaptureAccess() { CGRequestScreenCaptureAccess() } + if !AXIsProcessTrusted() { + AXIsProcessTrustedWithOptions([kAXTrustedCheckOptionPrompt.takeUnretainedValue() as String: true] as CFDictionary) + } +} + +func argument(_ name: String, in args: [String]) -> String? { + guard let i = args.firstIndex(of: name), i + 1 < args.count else { return nil } + return args[i + 1] +} + +let args = Array(CommandLine.arguments.dropFirst()) +switch args.first { +case "windows": + printJSON(["windows": windowsJSON(), "screen": CGPreflightScreenCaptureAccess()]) +case "displays": + printJSON(["displays": displaysJSON()]) +case "permissions": + printJSON(permissionsJSON()) +case "request-permissions": + requestPermissions() + printJSON(permissionsJSON()) +case "serve": + let port = UInt16(argument("--port", in: args) ?? "") ?? 0 + let token = ProcessInfo.processInfo.environment["FRAME_MAC_VIEW_TOKEN"] ?? "" + guard !token.isEmpty else { + FileHandle.standardError.write(Data("frame-mac-view: set FRAME_MAC_VIEW_TOKEN\n".utf8)) + exit(2) + } + let page = argument("--page", in: args).map { URL(fileURLWithPath: $0) } + let agent = Agent(token: token, page: page) + // Quit when the parent goes away (it holds our stdin open). + if args.contains("--exit-on-eof") { + DispatchQueue.global().async { + while FileHandle.standardInput.availableData.count > 0 {} + exit(0) + } + } + let server: Server + do { + server = try Server(port: port) { req, c in agent.handle(req, c) } + } catch { + FileHandle.standardError.write(Data("frame-mac-view: \(error)\n".utf8)) + exit(1) + } + server.start { [server] error in + if let error { + FileHandle.standardError.write(Data("frame-mac-view: can't listen on 127.0.0.1:\(port): \(error)\n".utf8)) + exit(1) + } + print("frame-mac-view listening on 127.0.0.1:\(server.port ?? port)") + fflush(stdout) + } + _ = NSApplication.shared // AppKit for NSScreen names and app activation + withExtendedLifetime(server) { RunLoop.main.run() } +default: + FileHandle.standardError.write(Data("usage: frame-mac-view serve|windows|displays|permissions|request-permissions\n".utf8)) + exit(2) +} diff --git a/mac/frame-mac-view/build.sh b/mac/frame-mac-view/build.sh new file mode 100755 index 0000000..4fafda2 --- /dev/null +++ b/mac/frame-mac-view/build.sh @@ -0,0 +1,10 @@ +#!/bin/sh +# Builds the Mac streaming agent into mac/bin/frame-mac-view (arm64, macOS 14+). +# Frame Control runs it for "Mac in the headset"; the Electron app bundles it. +set -eu +here=$(cd "$(dirname "$0")" && pwd) +out=${1:-$here/../bin/frame-mac-view} +mkdir -p "$(dirname "$out")" +xcrun swiftc -O -swift-version 5 -target arm64-apple-macos14.0 \ + -o "$out" "$here"/Sources/*.swift +echo "built $out" diff --git a/tests/test_macview.py b/tests/test_macview.py new file mode 100644 index 0000000..1a31ee1 --- /dev/null +++ b/tests/test_macview.py @@ -0,0 +1,274 @@ +"""Mac in the headset (ui/frame_macview.py and the frame-mac-view agent). + +The Python checks run anywhere. On macOS the agent is built and driven over +HTTP and WebSocket with its test pattern, which needs no Screen Recording +permission: status, the token, the viewer page, H.264 keyframes, pointer +input reaching the source, and closing. + +Run: python3 -m unittest discover -s tests +""" +import base64 +import http.client +import json +import os +import shutil +import socket +import struct +import subprocess +import sys +import tempfile +import time +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT / "ui")) + +import frame_macview # noqa: E402 + + +class Helpers(unittest.TestCase): + def test_panel_id_is_stable_and_in_range(self): + a = frame_macview.panel_id("window:123") + self.assertEqual(a, frame_macview.panel_id("window:123")) + self.assertNotEqual(a, frame_macview.panel_id("window:124")) + self.assertTrue(2_001_000_000 <= a < 2_002_000_000) + + def test_fit_keeps_aspect_inside_the_panel(self): + self.assertEqual(frame_macview.fit(2560, 1440), (1920, 1080)) + w, h = frame_macview.fit(800, 1600) + self.assertEqual(h, 1080) + self.assertAlmostEqual(w / h, 0.5, places=2) + + @unittest.skipUnless(shutil.which("bash"), "needs bash") + def test_launch_script_parses(self): + r = subprocess.run(["bash", "-n"], input=frame_macview.LAUNCH, text=True, capture_output=True) + self.assertEqual(r.returncode, 0, r.stderr) + + def test_show_checks_the_source_before_anything_else(self): + mv = frame_macview.MacView(["ssh"], lambda *a, **k: self.fail("no ssh"), "frame") + with self.assertRaises(frame_macview.MacViewError): + mv.show("rm -rf /") + + def test_missing_browser_is_explained(self): + calls = [] + + def run(remote, stdin=None, timeout=30): + calls.append(remote) + e = RuntimeError("exit 3") + e.stdout = "NO_BROWSER\n" + raise e + + mv = frame_macview.MacView(["ssh"], run, "frame") + mv.call = lambda path, **kw: {"screen": True, "ticket": "tk"} + mv.ensure_tunnel = lambda **kw: None + mv.remote_port = 47900 + with self.assertRaises(frame_macview.MacViewError) as cm: + mv.show("window:5") + self.assertIn("Chromium", str(cm.exception)) + self.assertTrue(calls[0].startswith("bash -s -- ")) + + def test_screen_permission_is_checked_before_the_headset(self): + mv = frame_macview.MacView(["ssh"], lambda *a, **k: self.fail("no ssh"), "frame") + mv.call = lambda path, **kw: {"screen": False} + with self.assertRaises(frame_macview.MacViewError) as cm: + mv.show("display:1") + self.assertIn("Screen Recording", str(cm.exception)) + + +class WS: + """A minimal WebSocket client (masked frames out, plain frames in).""" + + def __init__(self, port, path): + self.s = socket.create_connection(("127.0.0.1", port), timeout=10) + key = base64.b64encode(os.urandom(16)).decode() + self.s.sendall(f"GET {path} HTTP/1.1\r\nHost: x\r\nUpgrade: websocket\r\nConnection: Upgrade\r\n" + f"Sec-WebSocket-Key: {key}\r\nSec-WebSocket-Version: 13\r\n\r\n".encode()) + head = b"" + while b"\r\n\r\n" not in head: + head += self.s.recv(1) + self.status = int(head.split()[1]) + self.buf = b"" + + def _read(self, n): + while len(self.buf) < n: + chunk = self.s.recv(65536) + if not chunk: + raise EOFError + self.buf += chunk + out, self.buf = self.buf[:n], self.buf[n:] + return out + + def recv(self): + b0, b1 = self._read(2) + n = b1 & 0x7F + if n == 126: + n = struct.unpack(">H", self._read(2))[0] + elif n == 127: + n = struct.unpack(">Q", self._read(8))[0] + return b0 & 0x0F, self._read(n) + + def send_text(self, text): + data, mask = text.encode(), os.urandom(4) + head = bytes([0x81, 0x80 | len(data)]) if len(data) < 126 else bytes([0x81, 0xFE]) + struct.pack(">H", len(data)) + self.s.sendall(head + mask + bytes(c ^ mask[i % 4] for i, c in enumerate(data))) + + def close(self): + self.s.close() + + +@unittest.skipUnless(sys.platform == "darwin" and shutil.which("xcrun"), "the agent is macOS-only") +class Agent(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.tmp = tempfile.mkdtemp() + cls.bin = Path(cls.tmp) / "frame-mac-view" + r = subprocess.run(["/bin/sh", str(ROOT / "mac" / "frame-mac-view" / "build.sh"), str(cls.bin)], + capture_output=True, text=True, timeout=600) + if r.returncode: # a real failure on a Mac with Xcode: don't hide it as a skip + raise AssertionError("agent didn't build:\n" + (r.stderr or r.stdout)[-2000:]) + cls.token = "t0ken-" + os.urandom(6).hex() + cls.proc = subprocess.Popen([str(cls.bin), "serve", "--port", "0", "--page", str(ROOT / "ui" / "mac-view.html"), + "--exit-on-eof"], env={**os.environ, "FRAME_MAC_VIEW_TOKEN": cls.token}, + stdin=subprocess.PIPE, stdout=subprocess.PIPE, text=True) + line = cls.proc.stdout.readline() + cls.port = int(line.rsplit(":", 1)[1]) + + @classmethod + def tearDownClass(cls): + cls.proc.stdin.close() # --exit-on-eof + cls.proc.stdout.close() + try: + cls.proc.wait(5) + except subprocess.TimeoutExpired: + cls.proc.kill() + shutil.rmtree(cls.tmp, ignore_errors=True) + + def get(self, path, method="GET"): + c = http.client.HTTPConnection("127.0.0.1", self.port, timeout=10) + c.request(method, path) + r = c.getresponse() + return r.status, r.read() + + def test_token_is_required(self): + self.assertEqual(self.get("/status")[0], 403) + self.assertEqual(self.get("/status?k=wrong")[0], 403) + status, body = self.get(f"/status?k={self.token}") + self.assertEqual(status, 200) + self.assertIn("screen", json.loads(body)) + + def test_serves_the_viewer_page(self): + status, body = self.get(f"/view?k={self.token}&src=test") + self.assertEqual(status, 200) + self.assertIn(b"VideoDecoder", body) + + def test_lists_displays(self): + status, body = self.get(f"/displays?k={self.token}") + self.assertEqual(status, 200) + self.assertIsInstance(json.loads(body)["displays"], list) + + def ticket(self, src): + status, body = self.get(f"/ticket?k={self.token}&src={src}", method="POST") + self.assertEqual(status, 200) + return json.loads(body)["ticket"] + + def test_ping_and_page_are_open_but_streams_are_not(self): + self.assertEqual(self.get("/ping"), (200, b"frame-mac-view")) + self.assertEqual(WS(self.port, "/stream?src=test").status, 403) + self.assertEqual(self.get("/ticket?src=test", method="POST")[0], 403) + + def test_tickets_are_single_use_and_tied_to_one_source(self): + t = self.ticket("test") + self.assertEqual(WS(self.port, f"/stream?src=display:1&t={t}").status, 403) # wrong source + ws = WS(self.port, f"/stream?src=test&t={t}") + self.assertEqual(ws.status, 101) + hello = json.loads(ws.recv()[1]) + self.assertEqual(hello["t"], "hello") + # Until the viewer acknowledges, a retry (the hello got lost) gets the + # same key, and replaces the first viewer rather than adding one. + retry = WS(self.port, f"/stream?src=test&t={t}") + self.assertEqual(retry.status, 101) + self.assertEqual(json.loads(retry.recv()[1])["r"], hello["r"]) + time.sleep(0.3) + _, body = self.get(f"/status?k={self.token}") + self.assertEqual(len(json.loads(body)["streams"]), 1) + ws.close() + ws = retry + ws.send_text(json.dumps({"t": "ack"})) + time.sleep(0.3) + self.assertEqual(WS(self.port, f"/stream?src=test&t={t}").status, 403) # spent + again = WS(self.port, f"/stream?src=test&r={hello['r']}") # the viewer reconnecting + self.assertEqual(again.status, 101) + again.close() + ws.close() + # Stop revokes the reconnect key. + self.get(f"/close?k={self.token}&src=test", method="POST") + self.assertEqual(WS(self.port, f"/stream?src=test&r={hello['r']}").status, 403) + + def test_stop_revokes_tickets_not_yet_used(self): + t = self.ticket("test") + self.get(f"/close?k={self.token}&src=test", method="POST") + self.assertEqual(WS(self.port, f"/stream?src=test&t={t}").status, 403) + + def test_bad_frame_lengths_close_the_socket_not_the_agent(self): + ws = WS(self.port, f"/stream?src=test&t={self.ticket('test')}") + self.assertEqual(ws.status, 101) + # A masked frame claiming 2^63 bytes. + ws.s.sendall(bytes([0x81, 0xFF]) + struct.pack(">Q", 1 << 63) + os.urandom(4)) + with self.assertRaises((EOFError, OSError)): + for _ in range(1000): + ws.recv() + ws.close() + self.assertEqual(self.get(f"/status?k={self.token}")[0], 200) + + def test_stream_input_and_close(self): + ws = WS(self.port, f"/stream?k={self.token}&src=test&codec=h264&fps=30&max=640") + self.assertEqual(ws.status, 101) + info = None + key = None + for _ in range(200): + op, data = ws.recv() + if op == 1: + msg = json.loads(data) + if msg["t"] == "hello": + continue + if msg["t"] == "error": + self.skipTest("no H.264 encoder here: " + msg["message"]) + if msg["t"] == "info": + info = msg + elif op == 2 and data[0] == 1: + key = data + if info and key: + break + self.assertEqual(info["src"], "test") + self.assertTrue(info["input"]) + # A keyframe: flags, 8-byte timestamp, then Annex B with the SPS first. + self.assertEqual(key[9:13], b"\x00\x00\x00\x01") + self.assertEqual(key[13] & 0x1F, 7) # NAL type 7, SPS + ws.send_text(json.dumps({"t": "m", "e": "down", "b": 0, "x": 0.5, "y": 0.5})) + ws.send_text(json.dumps({"t": "key-frame"})) + got_key = False + for _ in range(200): + op, data = ws.recv() + if op == 2 and data[0] == 1: + got_key = True + break + self.assertTrue(got_key, "no keyframe after asking for one") + _, body = self.get(f"/status?k={self.token}") + self.assertEqual([s["src"] for s in json.loads(body)["streams"]], ["test"]) + status, body = self.get(f"/close?k={self.token}", method="POST") + self.assertEqual(json.loads(body)["closed"], 1) + for _ in range(400): + op, data = ws.recv() + if op == 1: + break + self.assertEqual(json.loads(data)["t"], "close") + # Without the viewer doing anything, the agent ends the stream itself. + time.sleep(1) + _, body = self.get(f"/status?k={self.token}") + self.assertEqual(json.loads(body)["streams"], []) + ws.close() + + +if __name__ == "__main__": + unittest.main() diff --git a/ui/frame_macview.py b/ui/frame_macview.py new file mode 100644 index 0000000..cf31580 --- /dev/null +++ b/ui/frame_macview.py @@ -0,0 +1,361 @@ +"""Mac in the headset: stream Mac windows or displays into the Steam Frame as +panels you can place anywhere, with the laser, wheel and keys driving the Mac. + +Runs on the Mac, inside Frame Control's server. The pieces: + +- mac/bin/frame-mac-view (Swift, built from mac/frame-mac-view): captures with + ScreenCaptureKit, encodes with VideoToolbox, serves ui/mac-view.html and one + WebSocket per stream on 127.0.0.1, and plays input back with CGEvent. +- An `ssh -R` tunnel, so the Frame reaches the agent on its own 127.0.0.1. + Every request carries a random token, so other programs on the Frame + (Android apps included) can't watch or drive the Mac. +- A Chromium app window on the Frame per stream, on gamescope's X display + with its own STEAM_GAME id, which makes it its own SteamVR panel (see + docs/panels.md). Chromium XR (~/chromium-xr) is preferred because it's + built with H.264; Flathub Chromium is the fallback. + +Stdlib only. MacView gets a plain ssh argv for the tunnel (its own +connection) and the server's `run(remote, stdin=, timeout=)` for commands. +""" +import json +import os +import re +import secrets +import shutil +import subprocess +import sys +import threading +import time +import urllib.error +import urllib.request +import zlib +from pathlib import Path +from urllib.parse import urlencode + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parent +PAGE = HERE / "mac-view.html" +SOURCES = ROOT / "mac" / "frame-mac-view" +AGENT = Path(os.environ.get("FRAME_MAC_VIEW") or ROOT / "mac" / "bin" / "frame-mac-view") +REMOTE_PORTS = range(47900, 47920) +SUPPORTED = sys.platform == "darwin" + +# Stream settings by name: long side in pixels, frames per second, H.264 bits +# per pixel per frame, codec. JPEG is for a Frame browser without H.264. +QUALITY = { + "sharp": {"max": 2560, "fps": 60, "bpp": 0.14, "codec": "h264"}, + "balanced": {"max": 1920, "fps": 60, "bpp": 0.1, "codec": "h264"}, + "light": {"max": 1280, "fps": 30, "bpp": 0.08, "codec": "h264"}, + "compatible": {"max": 1280, "fps": 20, "bpp": 0.1, "codec": "jpeg"}, +} +# Viewer windows are fitted inside a panel's size. gamescope made a 1280x720 +# request 1920x1080 anyway (verified 2026-09-28, build 20260925.6191901). +PANEL_BOX = (1920, 1080) + +# Opens one viewer on gamescope's X display and gives its window its own panel +# id. Args: appid url width height tag. The page puts "[tag]" in its title at +# once, which is how its X window is found (Chromium may hand the URL to an +# instance that's already running, so there's no process to follow). +LAUNCH = r"""set -u +appid=$1 url=$2 w=$3 h=$4 tag=$5 +export DISPLAY=:0 LC_ALL=C.UTF-8 +unset WAYLAND_DISPLAY +if ! xprop -root GAMESCOPE_FOCUSABLE_WINDOWS >/dev/null 2>&1; then + echo "The headset isn't showing anything (gamescope's display :0 isn't up). Wake it and try again." >&2 + exit 2 +fi +common=(--ozone-platform=x11 --force-device-scale-factor=1 --no-first-run --no-default-browser-check + --password-store=basic --disable-session-crashed-bubble --noerrdialogs --disable-infobars + --disable-features=Translate,MediaRouter --autoplay-policy=no-user-gesture-required + "--window-size=$w,$h" "--app=$url") +if [ -x "$HOME/chromium-xr/chrome" ]; then + cmd=("$HOME/chromium-xr/chrome" "--user-data-dir=$HOME/.local/share/frame-control/mac-view" "${common[@]}") +elif flatpak info org.chromium.Chromium >/dev/null 2>&1; then + cmd=(flatpak run org.chromium.Chromium + "--user-data-dir=$HOME/.var/app/org.chromium.Chromium/data/frame-mac-view" "${common[@]}") +else + echo "NO_BROWSER" + exit 3 +fi +log=/tmp/frame-mac-view.log +setsid nohup "${cmd[@]}" >>"$log" 2>&1 /dev/null | awk '/^ +0x/ {print $1}'); do + xprop -id "$win" _NET_WM_NAME WM_NAME 2>/dev/null | grep -qF "[$tag]" || continue + if xprop -id "$win" -f STEAM_GAME 32c -set STEAM_GAME "$appid" 2>/dev/null; then + echo "panel valve.steam.desktopgame.$appid window $win" + exit 0 + fi + done +done +echo "The viewer started, but its window didn't appear within 30 s. Chromium's log:" >&2 +tail -n 15 "$log" >&2 +exit 1 +""" + + +class MacViewError(Exception): + pass + + +def panel_id(src): + """A stable panel id per source, in the range panel-on-frame.sh uses.""" + return 2_001_000_000 + zlib.crc32(f"mac:{src}".encode()) % 1_000_000 + + +def fit(w, h, box=PANEL_BOX): + s = min(box[0] / max(w, 1), box[1] / max(h, 1)) + return max(320, round(w * s)), max(200, round(h * s)) + + +class MacView: + def __init__(self, tunnel_ssh, run, frame, track=None): + self.tunnel_ssh = list(tunnel_ssh) + self.run = run + self.frame = frame + self.track = track or (lambda proc: None) # the server ends these on exit + self.lock = threading.Lock() + self.token = secrets.token_urlsafe(24) + self.agent = None + self.port = None + self.tunnel = None + self.remote_port = None + self.supervisor = None + self.closing = False + self.shown = set() # sources with a viewer out there, connected or retrying + + # ---- the agent on this Mac ---- + + def unavailable(self): + """Why this can't work here, or None.""" + if not SUPPORTED: + return "Streaming your computer into the headset needs macOS." + if not AGENT.exists() and not (SOURCES.exists() and shutil.which("xcrun")): + return "The Mac streaming helper is missing from this copy of Frame Control." + return None + + def build(self): + if AGENT.exists() and not self._stale(): + return + if not (SOURCES / "build.sh").exists(): + if AGENT.exists(): + return + raise MacViewError("The Mac streaming helper is missing.") + r = subprocess.run(["/bin/sh", str(SOURCES / "build.sh"), str(AGENT)], capture_output=True, text=True, + stdin=subprocess.DEVNULL, timeout=600) + if r.returncode != 0: + raise MacViewError("Couldn't build the Mac streaming helper: " + (r.stderr or r.stdout).strip()[-400:]) + + def _stale(self): + try: + built = AGENT.stat().st_mtime + return any(p.stat().st_mtime > built for p in (SOURCES / "Sources").glob("*.swift")) + except OSError: + return False + + def ensure_agent(self): + with self.lock: + if self.agent and self.agent.poll() is None: + return + reason = self.unavailable() + if reason: + raise MacViewError(reason) + self.build() + env = {**os.environ, "FRAME_MAC_VIEW_TOKEN": self.token} + # The same port as before when restarting, so a running tunnel still + # fits; otherwise (or if it's gone) whatever the system gives. + for port in dict.fromkeys([self.port or 0, 0]): + self.agent = subprocess.Popen([str(AGENT), "serve", "--port", str(port), "--page", str(PAGE), + "--exit-on-eof"], env=env, stdin=subprocess.PIPE, + stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) + line = self.agent.stdout.readline() + m = re.search(r"listening on 127\.0\.0\.1:(\d+)", line) + if m: + break + self.agent.kill() + else: + raise MacViewError(f"The Mac streaming helper didn't start: {line.strip() or 'no output'}") + if self.port != int(m.group(1)): + self._drop_tunnel() + self.port = int(m.group(1)) + self.track(self.agent) + threading.Thread(target=self.agent.stdout.read, daemon=True).start() # drain + + def call(self, path, method="GET", **query): + """A request to the agent; starts it if needed.""" + self.ensure_agent() + url = f"http://127.0.0.1:{self.port}{path}?{urlencode({**query, 'k': self.token})}" + req = urllib.request.Request(url, method=method, data=b"" if method == "POST" else None) + try: + with urllib.request.urlopen(req, timeout=10) as r: + return json.load(r) + except (urllib.error.URLError, OSError, ValueError) as e: + raise MacViewError(f"The Mac streaming helper didn't answer: {e}") + + # ---- the tunnel from the Frame ---- + + def _drop_tunnel(self): + if self.tunnel and self.tunnel.poll() is None: + self.tunnel.terminate() + self.tunnel = None + + def tunnel_up(self): + return self.tunnel is not None and self.tunnel.poll() is None + + def ensure_tunnel(self, allow_new_port=False): + """Open the tunnel, on the port viewers already use if there is one. + A new port only when nobody is watching, since viewers can't move.""" + with self.lock: + if self.tunnel_up(): + return + last = "" + ports = [self.remote_port] if self.remote_port else list(REMOTE_PORTS) + if self.remote_port and allow_new_port: + ports += [p for p in REMOTE_PORTS if p != self.remote_port] + for port in ports: + proc = subprocess.Popen([*self.tunnel_ssh, "-o", "ExitOnForwardFailure=yes", + "-o", "ServerAliveInterval=5", "-o", "ServerAliveCountMax=3", "-N", + "-R", f"127.0.0.1:{port}:127.0.0.1:{self.port}", self.frame], + stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, + stderr=subprocess.PIPE, text=True) + # A taken port makes ssh exit once it's connected; a working + # tunnel answers the agent's status from the Frame's side. + ok = False + for _ in range(15): + time.sleep(0.4) + if proc.poll() is not None: + break + if self._probe(port): + ok = True + break + if ok: + self.tunnel, self.remote_port = proc, port + self.track(proc) + self._supervise() + return + if proc.poll() is None: + proc.terminate() + proc.wait() + last = (proc.stderr.read() or "").strip() + if "forward" not in last.lower(): + break # not a port clash: the Frame is unreachable + raise MacViewError(f"Couldn't open a tunnel from {self.frame} to this Mac: {last or 'no answer through it'}") + + def _supervise(self): + """Reopen the tunnel on the same port after the headset sleeps or the + network drops, so viewers that are retrying find the Mac again.""" + if self.supervisor and self.supervisor.is_alive(): + return + + def loop(): + while not self.closing: + time.sleep(5) + if self.closing or self.tunnel_up() or not (self.agent and self.agent.poll() is None): + continue + try: + self.ensure_tunnel() + except MacViewError: + pass # still unreachable; try again shortly + + self.supervisor = threading.Thread(target=loop, daemon=True) + self.supervisor.start() + + def _probe(self, port): + """Whether the Frame reaches the agent through the tunnel on `port`.""" + # /ping needs no key, so none appears on the Frame's command lines. + cmd = f"curl -s -m 2 'http://127.0.0.1:{port}/ping'" + try: + return self.run(cmd, timeout=8).strip() == "frame-mac-view" + except Exception: # noqa: BLE001 - the server's Failure, timeouts: all mean "not yet" + return False + + # ---- viewers on the Frame ---- + + def show(self, src, quality="balanced", width=None, height=None): + if src != "test" and not (src.startswith("window:") or src.startswith("display:")): + raise MacViewError("Pick a window or display to show.") + q = QUALITY.get(quality) or QUALITY["balanced"] + state = self.call("/status") + if src != "test" and not state.get("screen"): + raise MacViewError("Frame Control needs Screen Recording permission first (Allow… under Mac in the headset).") + self.shown -= set(state.get("finished", [])) # their Mac windows closed + if src in self.shown or any(st.get("src") == src for st in state.get("streams", [])): + # Showing it again replaces the old viewer, connected or not: this + # revokes its keys so it can't come back alongside the new one. + self.stop(src) + # Viewers that lost the tunnel keep retrying its old port, even though + # the agent no longer counts them, so only move when none are out there. + others = self.shown - {src} + self.ensure_tunnel(allow_new_port=not others) + appid = panel_id(src) + # Unique per launch, so a new window is never confused with an old one. + tag = "fc" + secrets.token_hex(4) + # A single-use ticket for this source, not Frame Control's key: the URL + # is visible in the Frame's process list. + ticket = self.call("/ticket", method="POST", src=src)["ticket"] + params = {"src": src, "t": ticket, "tag": tag, "codec": q["codec"], "max": q["max"], "fps": q["fps"], + "bpp": q["bpp"]} + url = f"http://127.0.0.1:{self.remote_port}/view?{urlencode(params)}" + w, h = fit(width or 1280, height or 720) + args = " ".join(_quote(str(a)) for a in (appid, url, w, h, tag)) + try: + out = self.run("bash -s -- " + args, stdin=LAUNCH, timeout=60) + except Exception as e: # noqa: BLE001 - the server's Failure carries the Frame's words + if "NO_BROWSER" in (getattr(e, "stdout", "") or ""): + raise MacViewError("The Frame needs a browser for this: install Chromium (Tools → Linux apps, " + "org.chromium.Chromium) or Chromium XR.") + raise MacViewError(str(e)) + self.shown.add(src) + return {"panel": f"valve.steam.desktopgame.{appid}", "src": src, "detail": out.strip()} + + def stop(self, src=None): + if src: + self.shown.discard(src) + else: + self.shown.clear() + if not (self.agent and self.agent.poll() is None): + return {"closed": 0} + return self.call("/close", method="POST", **({"src": src} if src else {})) + + def state(self): + reason = self.unavailable() + if reason: + return {"available": False, "reason": reason} + status = self.call("/status") + windows = self.call("/windows").get("windows", []) if status.get("screen") else [] + displays = self.call("/displays").get("displays", []) + return {"available": True, "screen": status.get("screen", False), + "accessibility": status.get("accessibility", False), "streams": status.get("streams", []), + "windows": windows, "displays": displays, "tunnel": self.tunnel_up()} + + def restart_agent(self): + """Only if it's missing a permission: a running stream would stop.""" + with self.lock: + if not (self.agent and self.agent.poll() is None): + return + status = self.call("/status") + if status.get("screen") and status.get("accessibility"): + return + with self.lock: + self.agent.terminate() + self.agent.wait(5) + + def request_permissions(self): + return self.call("/permissions", method="POST") + + def shutdown(self): + self.closing = True + try: + self.stop() + except MacViewError: + pass + for proc in (self.tunnel, self.agent): + if proc and proc.poll() is None: + proc.terminate() + + +def _quote(s): + return "'" + s.replace("'", "'\\''") + "'" diff --git a/ui/index.html b/ui/index.html index 3845ef7..73948d6 100644 --- a/ui/index.html +++ b/ui/index.html @@ -309,6 +309,11 @@ .game-card .bar > i { display: block; height: 100%; background: var(--action); } .f-0 { color: var(--muted); } /* ---- Android display ---- */ + #mvQuality { background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; border-radius: 3px; padding: 4px 6px; font: inherit; font-size: 12px; } + .mv-windows { max-height: 420px; overflow-y: auto; } + .mv-perm { background: rgba(255,190,60,.08); border: 1px solid rgba(255,190,60,.25); border-radius: 4px; padding: 10px 12px; margin-bottom: 12px; font-size: 13px; } + .mv-perm .row { margin-top: 8px; } + .mv-live { color: var(--green, #7fd67f); font-size: 12px; } .disp select { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; border-radius: 3px; padding: 8px 10px; font: inherit; font-size: 13px; } .disp select:focus { outline: none; border-color: var(--blue); } @@ -642,6 +647,25 @@
FrameDrop: sideloader (Windows only for now)
+ + @@ -2107,10 +2131,64 @@ $("shotsRefresh").onclick = loadShots; $("shotsSaveNew").onclick = e => saveShots(shots.list.filter(s => !s.saved), e.currentTarget); $("shotsFolder").onclick = e => act($("shotsFolder").textContent, () => api("/api/open", { what: "shots" }), e.currentTarget); +// ---- Mac in the headset: windows and displays as panels (ui/frame_macview.py) ---- +let mvSeq = 0; +function mvRow(src, title, sub, w, h, streaming) { + return `
${esc(title)}
+
${esc(sub)}${streaming ? ' · in the headset' : ""}
+ ${streaming ? `` + : ``} +
`; +} +async function loadMacView(restart) { + const seq = ++mvSeq; + let s, err; + try { s = await api("/api/macview" + (restart ? "?restart=1" : "")); } catch (e) { err = e; } + if (seq !== mvSeq) return; + const box = $("macview"); + if (err) { box.hidden = false; return failed($("mvWindows"), err); } + if (!s.available) { box.hidden = true; return; } + box.hidden = false; + const live = new Set((s.streams || []).map(x => x.src)); + $("mvCount").textContent = live.size ? `${live.size} showing` : ""; + const perm = []; + if (!s.screen) perm.push(`Frame Control needs Screen Recording permission to show your windows.`); + if (!s.accessibility) perm.push(`Allow Accessibility too, so clicks and keys from the headset reach the Mac.`); + $("mvPerm").hidden = !perm.length; + $("mvPerm").innerHTML = perm.join(" ") + `
+ Then press Refresh. macOS may ask you to reopen Frame Control.
`; + $("mvDisplays").innerHTML = (s.displays || []).map(d => + mvRow(d.src, `Whole screen: ${d.name}`, `${d.w}×${d.h}${d.main ? " · main display" : ""}`, d.w, d.h, live.has(d.src))).join(""); + const wins = s.windows || []; + $("mvWinCount").textContent = wins.length ? `${wins.length}` : ""; + $("mvWindows").innerHTML = !s.screen ? `
Windows appear here once Screen Recording is allowed.
` + : wins.length ? wins.map(w => mvRow(w.src, w.title || w.app, `${w.title ? w.app + " · " : ""}${w.w}×${w.h}`, w.w, w.h, live.has(w.src))).join("") + : `
No windows open on this Mac's current desktop.
`; +} +$("mvRefresh").onclick = () => loadMacView(true); +$("macview").onclick = async e => { + const b = e.target.closest("[data-mv]"); if (!b) return; + const src = b.dataset.src; + if (b.dataset.mv === "show") { + await act(`Show ${b.dataset.title} in the headset`, async () => { + const r = await api("/api/macview", { action: "show", src, quality: $("mvQuality").value, + w: +b.dataset.w || undefined, h: +b.dataset.h || undefined }); + return { message: `${b.dataset.title} is a panel in the headset now. Float it with the SteamVR dashboard.`, ...r }; + }, b); + } else if (b.dataset.mv === "stop") { + await act("Stop showing it", () => api("/api/macview", { action: "stop", src }), b); + } else if (b.dataset.mv === "stopall") { + await act("Stop everything in the headset", () => api("/api/macview", { action: "stop" }), b); + } else if (b.dataset.mv === "perm") { + await act("Ask macOS for permission", () => api("/api/macview", { action: "permissions" }), b); + } + setTimeout(loadMacView, 800); +}; + // ---- pages: #home, #games, #android, #tools (older section links still work) ---- const PAGES = ["home", "games", "android", "tools"]; const SECTION_PAGE = { view: "home", device: "home", shots: "home", library: "games", sideloaded: "games", getgames: "games", - display: "android", transfer: "tools", apps: "tools", power: "tools" }; + display: "android", transfer: "tools", apps: "tools", power: "tools", macview: "tools" }; let page = "home"; function showPage() { const id = location.hash.slice(1); @@ -2122,6 +2200,7 @@ function showPage() { document.title = page === "home" ? "Frame Control" : `${page[0].toUpperCase() + page.slice(1)} · Frame Control`; const section = !PAGES.includes(id) && id && $(id); if (section) section.scrollIntoView(); else window.scrollTo(0, 0); + if (page === "tools") loadMacView(); } window.addEventListener("hashchange", showPage); // While a text field has focus, phones hide the bottom tab bar (see body.typing in the CSS). diff --git a/ui/mac-view.html b/ui/mac-view.html new file mode 100644 index 0000000..d56ecf9 --- /dev/null +++ b/ui/mac-view.html @@ -0,0 +1,242 @@ + + + + + + +Mac + + + + +
Connecting to the Mac…
+ + + diff --git a/ui/server.py b/ui/server.py index ef22af6..bd8a3db 100755 --- a/ui/server.py +++ b/ui/server.py @@ -39,6 +39,7 @@ sys.path.insert(0, str(Path(__file__).resolve().parent)) import frame_android # noqa: E402 import frame_catalog # noqa: E402 import frame_host # noqa: E402 +import frame_macview # noqa: E402 import frame_store # noqa: E402 import frame_titles # noqa: E402 import frame_webinstall # noqa: E402 @@ -1213,10 +1214,50 @@ def _sweep_one(prefix, d): pass +# ---- Mac in the headset (frame_macview.py) ---------------------------------- + +# The tunnel gets its own connection: the shared master's options would win +# over anything added after them. +macview = frame_macview.MacView(["ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=8"], + lambda remote, stdin=None, timeout=30: ssh(remote, stdin=stdin, timeout=timeout), + FRAME, track=_live_tunnels.add) + + +def macview_state(query=None): + if LOCAL: + return {"available": False, "reason": "Runs on your computer, not the headset."} + try: + # Refresh restarts the helper, which picks up a permission just granted. + if (query or {}).get("restart") == ["1"]: + macview.restart_agent() + return macview.state() + except frame_macview.MacViewError as e: + raise Failure(str(e), 500) + + +def macview_action(body): + """{action: show|stop|permissions, src, quality, w, h}.""" + if LOCAL: + raise Failure("Runs on your computer, not the headset.", 400) + action = body.get("action") + try: + if action == "show": + w, h = body.get("w"), body.get("h") + return macview.show(str(body.get("src") or ""), str(body.get("quality") or "balanced"), + int(w) if w else None, int(h) if h else None) + if action == "stop": + return macview.stop(body.get("src") or None) + if action == "permissions": + return macview.request_permissions() + except frame_macview.MacViewError as e: + raise Failure(str(e), 502) + raise Failure("unknown action", 400) + + POST = {"/api/android/display": android_display, "/api/android": android, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard, "/api/flatpak": flatpak, "/api/open": open_thing, "/api/shots/save": save_shots, "/api/webinstall/check": webinstall_check, "/api/webinstall/start": webinstall_start, - "/api/webinstall/cancel": webinstall_cancel} + "/api/webinstall/cancel": webinstall_cancel, "/api/macview": macview_action} # ---- HTTP ------------------------------------------------------------------ @@ -1336,6 +1377,8 @@ class Handler(BaseHTTPRequestHandler): "shared": frame_catalog.compat_db.shared()}) elif path == "/api/android/catalog": self.send_json({"apps": frame_catalog.catalog()}) + elif path == "/api/macview": + self.send_json(macview_state(parse_qs(url.query))) elif path == "/api/status": self.send_json(status({})) elif path == "/api/steam/owned": @@ -1529,6 +1572,7 @@ def main(): if not frame_host.WINDOWS: signal.signal(signal.SIGTERM, signal.SIG_IGN) webinstall_shutdown() + macview.shutdown() # close the headset's viewers before the agent goes # The master was started with -N, so it stays up until told to exit. if CONTROL: subprocess.run([*MUX, "-O", "exit", FRAME], capture_output=True, stdin=subprocess.DEVNULL) From 45f720883abec30e77199fcbe1fa47d5c0ebf166 Mon Sep 17 00:00:00 2001 From: saphid <4596216+saphid@users.noreply.github.com> Date: Mon, 28 Sep 2026 17:22:55 +1000 Subject: [PATCH 02/51] Docs: house style for announcing features and fixes Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/announcements.md | 144 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 144 insertions(+) create mode 100644 docs/announcements.md diff --git a/docs/announcements.md b/docs/announcements.md new file mode 100644 index 0000000..a62c0ab --- /dev/null +++ b/docs/announcements.md @@ -0,0 +1,144 @@ +# Announcing changes + +How we tell people about Frame Control features and fixes as they merge. The +same few sentences feed the X post, the release notes and the website, so they +are written once, in the pull request, while the change is fresh. + +## What gets announced + +| Kind | Announce? | Example | +|---|---|---| +| **New** — something you can now do | Yes, its own post | Stream Mac windows into the Frame as panels | +| **Better** — something existing got noticeably easier, faster or wider | Yes, its own post or a roundup | APKs install without the Android SDK | +| **Fixed** — something broken that users hit | Yes if people reported it or it blocked a flow; otherwise the next roundup | Mac mirror showed a zoomed-in corner | +| **Release** — a tagged build | Always, one post linking the release | Frame Control 0.3.1 | +| Tests, refactors, CI, docs-only, website polish | No | Fake Frame tests, screenshot crop | + +If a change isn't worth a sentence to someone who owns a Frame, it isn't +announced. + +## The voice + +Write it the way the README and release notes already read. + +- **Lead with what the person can now do**, in their words: "Install older + versions of an app when the newest won't run on the Frame", not "Add APK + version fallback resolver". +- **Plain and specific.** Name the thing, give the number: "about 30 fps", + "4,500 apps", "up to 8 older versions". No "blazing", "game-changing", + "excited to announce", "huge", or exclamation marks. +- **Say where it works.** Platforms and what it was tested on, briefly: + "Tested on a real Frame from macOS 27." Don't claim what wasn't tested. +- **Say the catch.** If it needs a setup step, an unsigned build, or only works + on one OS, say so in the same post. +- **Sentence case**, full sentences, British spelling to match the docs. + Contractions are fine. +- **No emoji in the text.** One image, GIF or short clip carries the tone + instead. The only symbol is the kind label below. +- **Unofficial, always.** Never imply Valve made or endorses it. Say "Steam + Frame" for the headset and "Frame Control" for the app. +- **Credit people.** If a user reported the bug or suggested the feature and is + happy to be named, thank them by handle. + +## The formats + +Every announceable PR ends with an `## Announcement` section holding these. +The reviewer checks it like code. + +### 1. The post (X, and any other social account) + +``` +: + + + + +``` + +- `` is `New`, `Better` or `Fixed`. +- 280 characters maximum including the link (X counts any link as 23). +- One link: the release if it has shipped, otherwise the PR. +- One visual when the change is visible: a screenshot from the app, a GIF, or a + short clip from the headset. Alt text describes what it shows. +- No hashtags, except `#SteamFrame` on releases and on posts about something + new, because people search for it. + +### 2. The release-note line + +One bullet under **New in x.y.z**, same as the current release notes: the +first half of the post's first sentence, no kind label, no link. + +### 3. Release post + +``` +Frame Control : + +. Also: . + +Windows, macOS and Linux: +#SteamFrame +``` + +The release title on GitHub uses the same `Frame Control : ` +line, as 0.3.0 and 0.3.1 already do. + +### Roundups + +Small fixes that don't earn their own post wait for a roundup, posted with the +next release or when three or more have piled up: + +``` +Fixed in Frame Control this week: +- +- +- + + +``` + +## Examples from what has already merged + +**#11, older APK versions** + +``` +New: when an Android app is too new for the Frame, Frame Control now offers +older versions that will install. + +It checks F-Droid, its archive and IzzyOnDroid, and verifies each download +before it goes on the headset. + +https://github.com/saphid/steam-frame/pull/11 +``` + +**#8, Mac mirror fixes** + +``` +Fixed: mirroring your Mac into the Steam Frame now fits the whole desktop in +the panel, asks for the right password, and shows the cursor. + +Tested end to end on a real Frame from macOS 27. + +https://github.com/saphid/steam-frame/pull/8 +``` + +**v0.3.1** + +``` +Frame Control 0.3.1: install APKs without the Android SDK + +Frame Control now reads APK files itself, so there's nothing extra to install. +Also: Linux and Windows game sideloading, and one-click install links. + +Windows, macOS and Linux: https://github.com/saphid/steam-frame/releases/tag/v0.3.1 +#SteamFrame +``` + +## Posting + +Nothing is posted without a person approving it. The flow is: + +1. The PR carries its `## Announcement` section. +2. On merge, the post is drafted from that section (manually for now). +3. Alex approves or edits it, then it's posted from the project account. +4. Replies and questions that turn out to be bugs become GitHub issues labelled + `feedback`, same as the website form. From b393e9085419bf44f5239b253b15581085d2e88f Mon Sep 17 00:00:00 2001 From: saphid <4596216+saphid@users.noreply.github.com> Date: Mon, 28 Sep 2026 17:23:06 +1000 Subject: [PATCH 03/51] Keep the Frame awake during agent work Steam's own idle timer (60 min on AC, 15 on battery) suspends the Frame, and SSH work doesn't count as activity. scripts/keep-awake.sh on sets both timers to Never through Steam's DevTools (reusing ui/frame_steam.py) and holds a logind sleep inhibitor as a user unit; off releases the inhibitor and restores the saved timers. Findings recorded in how-the-frame-works.md. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/how-the-frame-works.md | 1 + scripts/keep-awake.sh | 74 +++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+) create mode 100755 scripts/keep-awake.sh diff --git a/docs/how-the-frame-works.md b/docs/how-the-frame-works.md index 5c392d1..cdf2498 100644 --- a/docs/how-the-frame-works.md +++ b/docs/how-the-frame-works.md @@ -55,6 +55,7 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600 | **Tools on the image:** Python 3.12.3, `ffmpeg`, `openssl`, `curl`, `rsync`, `zip`/`unzip`, `flatpak`, `wpctl`, `podman`. **No `adb`.** `steamos` is uid 1000, in `wheel`, and sudoers has `%wheel ALL=(ALL) ALL`, so `sudo -S` takes the Developer Mode password on stdin. **Verified 2026-09-27.** | Running Frame Control's server on the Frame (`FRAME_LOCAL=1`, [iphone.md](iphone.md)) | | **Each Lepton instance is a podman container** named `lepton-steamlaunch-`, labelled with its ADB port (`podman ps --format '{{.Names}} {{.Labels.adb_port}}'`). `podman exec /system/bin/sh -c '…'` runs Android's shell inside it with no adb at all (used for `pidof` and `logcat` by the app tester). Running `wm size`/`wm density` that way is untested. **Verified 2026-09-27.** | `ui/frame_android.py`, the iPhone app's display settings | | **Asleep means off the network.** In standby the Frame stops answering on its LAN address, `frame.local` and Tailscale alike (`Host is down`, `No route to host`, timeouts), and ping fails. It was unreachable for about 2.5 hours until woken. Nothing over SSH can wake it. **Verified 2026-09-27.** | Frame Control's offline banner and retries | +| **What puts it to sleep is Steam's idle timer**, not logind. The journal shows `steamui_system: Switching to power state: [ k_ESystemPowerState_Sleep ] reason: 'ComputeNextPowerState: active: 3600 < 3600 (k_EACState_Connected)'`, then Steam suspends. SSH work doesn't count as activity. The timers are the client settings `system_idle_suspend_ac_sec` (3600) and `system_idle_suspend_battery_sec` (900); 0 means Never (Settings → Power → Sleep after inactivity). They can be written over DevTools the way the settings page does. logind refuses a `systemd-inhibit --mode=block` sleep lock from an SSH session (`Interactive authentication required`) but accepts one started with `systemd-run --user`. `scripts/keep-awake.sh on|off|status` does both and restores the old timers on `off`. **Verified 2026-09-28**, BUILD_ID 20260925.6191901. Whether Steam's suspend honours the inhibitor on its own is **inferred** (polkit gives `steamos` no `suspend-ignore-inhibit`), not tested. | Keeping the Frame awake for agent work | | **Battery at full on a charger** can read `Discharging` at about 0 W (for example 99 %, 0.0 W, USB-C PD 18 W). Treat under 0.5 W on a charger as "not charging", not "draining". **Verified 2026-09-27.** | Frame Control's battery card | | **The OS image is downloadable.** Valve's recovery images for the Frame are at `https://steamdeck-images.steamos.cloud/recovery/`. The root filesystem inside is btrfs, and it runs as an SSH test target on ARM64 Linux without the headset (`tests/frame-container/frame-image.sh`). **Verified 2026-09-27.** | [recovery-and-images.md](recovery-and-images.md) | | **Boot / recovery menu.** Hold Power ~10 s until the LED goes off, then power on while holding the **AUX button on top of the Power button** (not the volume keys) until a text menu appears. Entries: `Current` (SteamOS-A/B + build), `Previous` (the other A/B slot), `Boot from USB`, `Repair Steam Installation`, `Erase User Data` (factory reset), `ADB mode`, `Battery Ship Mode`. It auto-boots `Current` after a ~15 s countdown. **Volume Up/Down (left side) move, AUX (right side) selects.** For a boot loop, Valve says pick `Previous` (keeps user data); then `Repair Steam Installation`; `Erase User Data` wipes `~` (SSH keys, Tailscale, Flatpaks, T3 setup). Last resort is a full re-image, two ways: (1) USB: write `steamframe-oobe-repair-.img.bz2` to an 8 GB+ USB-C stick (Balena Etcher on the Mac), pick `Boot from USB`, then use "Wipe Device & Install SteamOS" / "Repair SteamOS" (keeps games and personal content) from the recovery desktop; (2) cable/EDL: `steamframe-oobe-repair-qdl-.tar.gz`, run `flash.sh` (Linux) or `flash.cmd` (Windows), then with the Frame off for 10 s hold Power + Vol Up + Vol Down for 10 s and plug it in; it reflashes and reboots. Both images: `https://steamdeck-images.steamos.cloud/recovery/` (build 20260922.5153644, 0.3.0, 3.8 GiB each, no published checksums); local copies in `~/Downloads/steam-frame-recovery/`. File names, checksums and what's inside: [recovery-and-images.md](recovery-and-images.md). Source: Valve's [SteamOS Recovery FAQ](https://help.steampowered.com/en/faqs/view/1B71-EDF2-EB6D-2BB3) and [Installation and Repair FAQ](https://help.steampowered.com/en/faqs/view/65B4-2AA3-5F37-4227), plus a menu photo in [EloiStree/HelloSteamFrame#9](https://github.com/EloiStree/HelloSteamFrame/issues/9). **Inferred** (Valve docs, 2026-09-26); not yet tried on our Frame. | Recovering from a boot loop | diff --git a/scripts/keep-awake.sh b/scripts/keep-awake.sh new file mode 100755 index 0000000..c4a3301 --- /dev/null +++ b/scripts/keep-awake.sh @@ -0,0 +1,74 @@ +#!/usr/bin/env zsh +# Mac-side: stop the Steam Frame from going to sleep while an agent works on it. +# +# The Frame sleeps when Steam's own idle timer runs out ("Sleep after +# inactivity": 60 min on AC, 15 min on battery by default). SSH activity +# doesn't count as input, and asleep the Frame is off the network. `on` sets +# both timers to Never through Steam's UI (DevTools on 127.0.0.1:8080, via +# ui/frame_steam.py) and holds a logind sleep inhibitor as a user unit. +# `off` drops the inhibitor and restores the timers `on` saved. +# +# Usage: +# scripts/keep-awake.sh on +# scripts/keep-awake.sh off +# scripts/keep-awake.sh status +set -euo pipefail + +FRAME_ALIAS=${FRAME_ALIAS:-frame} +HERE=${0:A:h} +cmd=${1:-status} +case $cmd in on|off|status) ;; *) echo "usage: keep-awake.sh on|off|status" >&2; exit 2 ;; esac + +ssh -o ConnectTimeout=8 "$FRAME_ALIAS" \ + 'mkdir -p ~/.cache/frame-control && cat > ~/.cache/frame-control/frame_steam.py' < "$HERE/../ui/frame_steam.py" + +# Runs on the Frame. Verified 2026-09-28 (BUILD_ID 20260925.6191901): the +# timers are client settings system_idle_suspend_{ac,battery}_sec (0 = Never), +# written the way Steam's settings page does (steamui module exporting the +# SetSetting wrapper). logind refuses an inhibitor from an SSH session +# ("Interactive authentication required") but allows one from a user unit. +ssh "$FRAME_ALIAS" python3 - "$cmd" <<'EOF' +import json, os, subprocess, sys +sys.path.insert(0, os.path.expanduser("~/.cache/frame-control")) +from frame_steam import Page + +cmd = sys.argv[1] +saved_path = os.path.expanduser("~/.cache/frame-control/keep-awake.json") +unit = "fc-keep-awake" +keys = ("system_idle_suspend_ac_sec", "system_idle_suspend_battery_sec") + +if cmd == "off": # release the lock first, even if Steam's UI is down + subprocess.run(["systemctl", "--user", "stop", unit], stderr=subprocess.DEVNULL) +page = Page() +def read(): + return {k: page.eval(f"settingsStore.clientSettings.{k}") for k in keys} +def write(values): + page.eval("""(async () => { let req; + webpackChunksteamui.push([[Symbol()], {}, r => { req = r }]); + const mod = Object.keys(req.m).map(id => req.m[id].toString().includes("Settings.SetSetting") ? req(id) : null).find(Boolean); + const set = Object.values(mod).find(f => typeof f == "function" && f.toString().includes("SetSetting(")); + for (const [k, v] of Object.entries(%s)) await set(k, v); + await new Promise(r => setTimeout(r, 1000)); })()""" % json.dumps(values)) +def inhibitor(): + return subprocess.run(["systemctl", "--user", "is-active", "-q", unit]).returncode == 0 + +if cmd == "on": + current = read() + if not os.path.exists(saved_path): + with open(saved_path, "w") as f: + json.dump(current, f) + write({k: 0 for k in keys}) + if not inhibitor(): + subprocess.run(["systemd-run", "--user", "-q", f"--unit={unit}", + "--description=Frame Control: keep the Frame awake", + "systemd-inhibit", "--what=sleep:idle:handle-suspend-key:handle-power-key", + "--who=Frame Control", "--why=Keep the Frame awake while an agent works on it", + "--mode=block", "sleep", "infinity"], check=True) +elif cmd == "off": + if os.path.exists(saved_path): # no backup: leave the timers as they are + with open(saved_path) as f: + write(json.load(f)) + os.remove(saved_path) + +print(json.dumps({"timers": read(), "inhibitor": inhibitor()})) +EOF From 9eeca79b5da23f5184be7818c42f2cdf58dcb9cb Mon Sep 17 00:00:00 2001 From: saphid <4596216+saphid@users.noreply.github.com> Date: Mon, 28 Sep 2026 17:34:21 +1000 Subject: [PATCH 04/51] Analytics, self-update and Report a problem - Anonymous PostHog analytics (ui/frame_telemetry.py): usage on by default after a first-run notice; compatibility results and error details opt-in, offered together by the notice's "Share more to help fix problems" button. Random id, no person profiles or GeoIP, scrubbed text, an offline outbox, and "Show what's been sent" in the new Privacy panel. Inert without a project key, from a source checkout, or with DO_NOT_TRACK=1. - APK installs now record install_failed when the APK itself won't install, and offer a 20-second test after installing. Opted-in reports reach the shared database through PostHog and `frame_compat_db.py sync`. - The desktop app updates itself from published releases (app/updater.js): update.json from releases/latest/download, SHA-256 checked, no downgrades; macOS bundle swap, Windows NSIS, Linux AppImage, otherwise the release page. scripts/publish-release.sh publishes a tested draft with its manifest. - Report a problem (header button, Privacy panel, Help menu) files a GitHub issue through the website's feedback API, with a previewed, scrubbed diagnostics snapshot; activity and logs only when asked for. Reviewed by GPT-6 Astra (xhigh, read-only) three times; all findings fixed. Docs: docs/privacy.md, docs/releasing.md. Co-Authored-By: Claude Opus 5.5 (1M context) --- .github/workflows/checks.yml | 4 +- README.md | 15 +- app/main.js | 100 ++++++- app/package-lock.json | 4 +- app/package.json | 6 +- app/preload.js | 16 ++ app/test/updater.test.js | 59 ++++ app/updater.js | 250 +++++++++++++++++ compat-db/README.md | 29 +- docs/frame-control.md | 13 +- docs/privacy.md | 133 +++++++++ docs/releasing.md | 59 ++++ scripts/publish-release.sh | 45 +++ tests/test_telemetry.py | 405 +++++++++++++++++++++++++++ ui/frame_android.py | 38 ++- ui/frame_compat_db.py | 150 +++++++++- ui/frame_report.py | 147 ++++++++++ ui/frame_telemetry.py | 522 +++++++++++++++++++++++++++++++++++ ui/index.html | 270 +++++++++++++++++- ui/server.py | 103 ++++++- ui/telemetry.json | 5 + 21 files changed, 2326 insertions(+), 47 deletions(-) create mode 100644 app/test/updater.test.js create mode 100644 app/updater.js create mode 100644 docs/privacy.md create mode 100644 docs/releasing.md create mode 100755 scripts/publish-release.sh create mode 100644 tests/test_telemetry.py create mode 100644 ui/frame_report.py create mode 100644 ui/frame_telemetry.py create mode 100644 ui/telemetry.json diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml index 9239b5d..b0d46ca 100644 --- a/.github/workflows/checks.yml +++ b/.github/workflows/checks.yml @@ -34,7 +34,9 @@ jobs: - name: Server tests run: python -m unittest discover -s tests -v - name: App syntax - run: node --check app/main.js && node --check app/build/make-icon.js && node --check app/build/fetch-deps.js && node --check app/preload.js && node --check app/install-link.js + run: node --check app/main.js && node --check app/build/make-icon.js && node --check app/build/fetch-deps.js && node --check app/preload.js && node --check app/install-link.js && node --check app/updater.js + - name: Updater tests + run: node --test app/test/updater.test.js - name: Website run: node --test site/test/*.test.mjs && node --check site/public/js/site.js && node --check site/public/js/feedback.js diff --git a/README.md b/README.md index 75167b0..2acc3df 100644 --- a/README.md +++ b/README.md @@ -104,6 +104,10 @@ already ships (sideloading a game copies Valve's own devkit scripts to | **Linux** (arm64) | [AppImage](https://github.com/saphid/steam-frame/releases/latest/download/Frame-Control-linux-arm64.AppImage) · [.deb](https://github.com/saphid/steam-frame/releases/latest/download/Frame-Control-linux-arm64.deb) | `ssh`, and `adb` for Android apps (`sudo apt install adb`) | The app brings its own Python and `adb`; SSH is built into macOS and Windows. +From 0.4 it updates itself: when a new version is published, a banner offers +**Update and restart**. It sends anonymous usage statistics, which you can turn +off. Sharing compatibility results and error details is opt-in. See +[docs/privacy.md](docs/privacy.md). Google doesn't publish `adb` for arm64 Linux, so that build uses your distribution's. If you already have `adb`, the app uses yours. @@ -172,9 +176,11 @@ entry to `~/.ssh/config` and keys at `~/.ssh/id_ed25519_frame` and ## Feedback This is a first public test, so reports are really useful, especially from -Windows and Linux. The quickest way is the -[feedback form](https://frame-control.pages.dev/feedback/): no GitHub account -needed, and it opens an issue here. Please include: +Windows and Linux. The quickest way is **Report a problem** in the app (the +warning-sign button at the top, or **Help → Report a Problem…**). It adds +diagnostics with personal details removed, shows you exactly what's included, +and opens an issue here. You don't need a GitHub account. Without the app, +use the [feedback form](https://frame-control.pages.dev/feedback/). Please include: - what you tried and what happened - your computer's OS and your SteamOS build (Steam Settings → System) @@ -230,7 +236,8 @@ cd app && npm install && npm start # run the app from the checkout The server is Python stdlib only; the app is Electron. GitHub Actions runs the tests on macOS, Windows and Linux, and a `v*` tag builds all three installers -into the release. See [building](docs/frame-control.md#building). +into a draft release, which reaches users once published. See +[building](docs/frame-control.md#building) and [releasing](docs/releasing.md). ## License diff --git a/app/main.js b/app/main.js index ad64d8f..f3463b8 100644 --- a/app/main.js +++ b/app/main.js @@ -10,6 +10,7 @@ const net = require("net"); const os = require("os"); const path = require("path"); const { SCHEME, parseInstallLink, linkFromArgv } = require("./install-link"); +const updater = require("./updater"); const run = promisify(execFile); @@ -114,7 +115,11 @@ function ping(target) { } async function startServer() { + // The version and whether this is a built app go to ui/frame_telemetry.py, which + // sends nothing from a source checkout. const env = { ...process.env, PATH: await loginPath(), FRAME_CONTROL_APP: "1", + FRAME_CONTROL_VERSION: app.getVersion(), FRAME_CONTROL_LOG: LOG, + ...(app.isPackaged ? { FRAME_CONTROL_PACKAGED: "1" } : {}), ...(fs.existsSync(TOOLS) ? { FRAME_CONTROL_TOOLS: TOOLS } : {}) }; python = await findPython(env); if (!python) throw new Error(`Frame Control needs Python 3.8 or later. ${PYTHON_HELP}`); @@ -243,6 +248,9 @@ function fromUi(e) { } ipcMain.handle("clipboard:read", (e) => fromUi(e) ? clipboard.readText() : ""); +ipcMain.handle("update:get", (e) => fromUi(e) ? publicUpdate() : null); +ipcMain.handle("update:check", (e) => fromUi(e) ? checkForUpdate({ manual: true }).then(publicUpdate) : null); +ipcMain.handle("update:install", (e) => { if (fromUi(e)) installUpdate(); }); // frame-control://install links from websites (docs/web-install.md). They can // arrive before the window or server exists (macOS open-url on a cold launch), @@ -275,6 +283,81 @@ ipcMain.on("install-link:ready", (e) => { deliverLinks(); }); +// ---- updates (app/updater.js, docs/releasing.md) ---- +// Checked shortly after launch and every few hours; the page shows a banner and +// the Update button calls installUpdate. +const UPDATE_EVERY = 6 * 3600 * 1000; +const update = { status: "idle", current: app.getVersion(), latest: null, error: null, progress: 0, how: null }; + +function publicUpdate() { + const r = update.latest; + return { status: update.status, current: update.current, error: update.error, progress: update.progress, + latest: r && { version: r.version, notes: r.notes, page: r.page }, + canInstall: !!update.how && update.how.method !== "manual", why: update.how && update.how.why }; +} + +function setUpdate(fields) { + Object.assign(update, fields); + if (win && linkPage === win.webContents) win.webContents.send("update:state", publicUpdate()); +} + +async function checkForUpdate({ manual = false } = {}) { + if (["checking", "downloading", "ready"].includes(update.status)) return; + setUpdate({ status: "checking", error: null }); + try { + const latest = await updater.latestRelease(); + const how = updater.updateMethod({ platform: process.platform, isPackaged: app.isPackaged, + execPath: process.execPath, env: process.env, + exists: fs.existsSync, writable: updater.writable }); + if (updater.isNewer(latest.version, update.current)) { + setUpdate({ status: "available", latest, how }); + if (manual) offerUpdateDialog(); + } else { + setUpdate({ status: "none", latest, how }); + if (manual) dialog.showMessageBox(win, { type: "info", message: "Frame Control is up to date", + detail: `You have ${update.current}, the newest version.` }); + } + } catch (e) { + // A failed check: nothing to install, and never an older release kept from before. + setUpdate({ status: "check-failed", error: e.message, latest: null }); + if (manual) dialog.showMessageBox(win, { type: "warning", message: "Couldn't check for updates", detail: e.message }); + } +} + +async function offerUpdateDialog() { + const r = update.latest; + const { response } = await dialog.showMessageBox(win, { + type: "info", message: `Frame Control ${r.version} is available`, + detail: `You have ${update.current}.` + (update.how.method === "manual" ? ` Download it from the release page (${update.how.why}).` : ""), + buttons: [update.how.method === "manual" ? "Open Release Page" : "Update and Restart", "Later"], defaultId: 0, cancelId: 1, + }); + if (response === 0) installUpdate(); +} + +async function installUpdate() { + // "error" here only ever means an install failed, so trying again is safe. + if (update.status !== "available" && update.status !== "error") return; + if (!update.latest || !updater.isNewer(update.latest.version, update.current)) return; + if (!update.how || update.how.method === "manual") { shell.openExternal(update.latest.page); return; } + setUpdate({ status: "downloading", progress: 0, error: null }); + try { + const start = await updater.prepare(update.latest, update.how, + (done, total) => { if (total) setUpdate({ progress: done / total }); }, update.current); + setUpdate({ status: "ready", progress: 1 }); + start(); + quitting = true; + app.quit(); + } catch (e) { + setUpdate({ status: "error", error: e.message }); + } +} + +function scheduleUpdateChecks() { + if (process.env.FRAME_CONTROL_NO_UPDATE_CHECK === "1") return; + setTimeout(checkForUpdate, 8000); + setInterval(checkForUpdate, UPDATE_EVERY).unref(); +} + function registerScheme() { // A checkout runs as `electron .`, so the OS must be told the script too. // (macOS takes the scheme from Info.plist, which only the built app has.) @@ -336,7 +419,11 @@ async function setUpConnection() { function buildMenu() { const template = [ - ...(IS_MAC ? [{ role: "appMenu" }] : []), + ...(IS_MAC ? [{ label: app.name, submenu: [ + { role: "about" }, { label: "Check for Updates…", click: () => checkForUpdate({ manual: true }) }, + { type: "separator" }, { role: "services" }, { type: "separator" }, + { role: "hide" }, { role: "hideOthers" }, { role: "unhide" }, { type: "separator" }, { role: "quit" }, + ] }] : []), { role: "fileMenu" }, { role: "editMenu" }, { @@ -363,7 +450,15 @@ function buildMenu() { ...(IS_MAC ? [{ role: "windowMenu" }] : []), { role: "help", - submenu: [{ label: "Project on GitHub", click: () => shell.openExternal("https://github.com/saphid/steam-frame") }], + submenu: [ + ...(IS_MAC ? [] : [{ label: "Check for Updates…", click: () => checkForUpdate({ manual: true }) }]), + { label: "Report a Problem…", click: () => { + if (win && url && linkPage === win.webContents) win.webContents.send("report:open"); + else shell.openExternal("https://frame-control.pages.dev/feedback/"); // the page isn't up + } }, + { label: "Release Notes", click: () => shell.openExternal(updater.RELEASES) }, + { label: "Project on GitHub", click: () => shell.openExternal("https://github.com/saphid/steam-frame") }, + ], }, ]; Menu.setApplicationMenu(Menu.buildFromTemplate(template)); @@ -386,6 +481,7 @@ if (!app.requestSingleInstanceLock()) { registerScheme(); buildMenu(); createWindow(); + scheduleUpdateChecks(); }); app.on("activate", () => { if (!win) createWindow(); }); app.on("window-all-closed", () => app.quit()); diff --git a/app/package-lock.json b/app/package-lock.json index f8f2d77..264b1dd 100644 --- a/app/package-lock.json +++ b/app/package-lock.json @@ -1,12 +1,12 @@ { "name": "frame-control", - "version": "0.3.1", + "version": "0.4.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "frame-control", - "version": "0.3.1", + "version": "0.4.0", "license": "MIT", "devDependencies": { "electron": "^44.4.5", diff --git a/app/package.json b/app/package.json index bd5701a..78f6989 100644 --- a/app/package.json +++ b/app/package.json @@ -1,7 +1,7 @@ { "name": "frame-control", "productName": "Frame Control", - "version": "0.3.1", + "version": "0.4.0", "description": "Desktop app for managing a Valve Steam Frame over SSH", "private": true, "main": "main.js", @@ -37,6 +37,7 @@ "main.js", "preload.js", "install-link.js", + "updater.js", "package.json", "build/icon.png" ], @@ -46,7 +47,8 @@ "to": "ui", "filter": [ "*.py", - "*.html" + "*.html", + "telemetry.json" ] }, { diff --git a/app/preload.js b/app/preload.js index f21551d..aaf5290 100644 --- a/app/preload.js +++ b/app/preload.js @@ -4,11 +4,27 @@ // as a title without zipping it (the local server reads it from there). // It also receives frame-control://install links (docs/web-install.md): only // what the link asked for, never an install; the page asks the user first. +// And it passes update state both ways: see app/updater.js. const { contextBridge, ipcRenderer, webUtils } = require("electron"); contextBridge.exposeInMainWorld("frameApp", { readClipboard: () => ipcRenderer.invoke("clipboard:read"), pathForFile: (file) => { try { return webUtils.getPathForFile(file) || ""; } catch { return ""; } }, + // Updates (app/updater.js): the page shows a banner and an Update button. + update: { + get: () => ipcRenderer.invoke("update:get"), + check: () => ipcRenderer.invoke("update:check"), + install: () => ipcRenderer.invoke("update:install"), + onState: (cb) => { + ipcRenderer.removeAllListeners("update:state"); + ipcRenderer.on("update:state", (_e, s) => cb(s)); + }, + }, + // Help → Report a Problem… opens the page's report dialog (ui/frame_report.py). + onReportProblem: (cb) => { + ipcRenderer.removeAllListeners("report:open"); + ipcRenderer.on("report:open", () => cb()); + }, onInstallLink: (cb) => { ipcRenderer.removeAllListeners("install-link"); ipcRenderer.on("install-link", (_e, req) => cb({ kind: req.kind, target: req.target })); diff --git a/app/test/updater.test.js b/app/test/updater.test.js new file mode 100644 index 0000000..4bf9594 --- /dev/null +++ b/app/test/updater.test.js @@ -0,0 +1,59 @@ +// Run: node --test app/test/ +const test = require("node:test"); +const assert = require("node:assert"); +const { isNewer, assetName, updateMethod, macBundle } = require("../updater"); + +test("versions compare numerically, and a release beats its pre-releases", () => { + assert.ok(isNewer("0.3.10", "0.3.9")); + assert.ok(isNewer("v1.0.0", "0.9.9")); + assert.ok(!isNewer("0.3.1", "0.3.1")); + assert.ok(!isNewer("0.3.0", "0.3.1")); + assert.ok(isNewer("1.0.0", "1.0.0-beta.1")); + assert.ok(!isNewer("1.0.0-beta.1", "1.0.0")); + assert.ok(!isNewer("garbage", "0.1.0")); +}); + +test("asset names match what electron-builder publishes", () => { + assert.strictEqual(assetName("darwin", "arm64", "mac-zip"), "Frame-Control-mac-arm64.zip"); + assert.strictEqual(assetName("win32", "x64", "nsis"), "Frame-Control-Setup-x64.exe"); + assert.strictEqual(assetName("linux", "x64", "appimage"), "Frame-Control-linux-x86_64.AppImage"); + assert.strictEqual(assetName("linux", "arm64", "appimage"), "Frame-Control-linux-arm64.AppImage"); +}); + +const base = { isPackaged: true, env: {}, exists: () => false, writable: () => true }; + +test("macOS updates in place only from a writable, non-translocated location", () => { + const exe = "/Applications/Frame Control.app/Contents/MacOS/Frame Control"; + assert.strictEqual(macBundle(exe), "/Applications/Frame Control.app"); + assert.deepStrictEqual(updateMethod({ ...base, platform: "darwin", execPath: exe }), + { method: "mac-zip", bundle: "/Applications/Frame Control.app" }); + const dmg = "/Volumes/Frame Control 0.3.1/Frame Control.app/Contents/MacOS/Frame Control"; + assert.strictEqual(updateMethod({ ...base, platform: "darwin", execPath: dmg }).method, "manual"); + const trans = "/private/var/folders/x/AppTranslocation/ABC/d/Frame Control.app/Contents/MacOS/Frame Control"; + assert.strictEqual(updateMethod({ ...base, platform: "darwin", execPath: trans }).method, "manual"); + assert.strictEqual(updateMethod({ ...base, platform: "darwin", execPath: exe, writable: () => false }).method, "manual"); +}); + +test("Windows needs the installer's copy; Linux needs an AppImage", () => { + const exe = "C:\\Users\\a\\AppData\\Local\\Programs\\Frame Control\\Frame Control.exe"; + assert.strictEqual(updateMethod({ ...base, platform: "win32", execPath: exe, exists: () => true }).method, "nsis"); + assert.strictEqual(updateMethod({ ...base, platform: "win32", execPath: exe }).method, "manual"); + assert.strictEqual(updateMethod({ ...base, platform: "linux", execPath: "/opt/x", env: { APPIMAGE: "/home/a/F.AppImage" } }).method, + "appimage"); + assert.strictEqual(updateMethod({ ...base, platform: "linux", execPath: "/opt/Frame Control/frame-control" }).method, "manual"); + assert.strictEqual(updateMethod({ ...base, isPackaged: false, platform: "darwin", execPath: "x" }).method, "manual"); +}); + +test("update.json assets always download from this repository's release", () => { + const r = require("../updater").fromManifest({ version: "0.4.0", notes: "n", assets: [ + { name: "Frame-Control-mac-arm64.zip", url: "https://evil.example/x.zip", digest: "sha256:" + "a".repeat(64) }] }); + assert.strictEqual(r.assets[0].url, "https://github.com/saphid/frame-control/releases/download/v0.4.0/Frame-Control-mac-arm64.zip"); + assert.throws(() => require("../updater").fromManifest({ version: "nope", assets: [] })); +}); + +test("prepare refuses a release that isn't newer (no downgrades)", async () => { + const { prepare } = require("../updater"); + const release = { version: "0.3.1", assets: [] }; + await assert.rejects(prepare(release, { method: "appimage", appImage: "/nonexistent/x" }, null, "0.4.0"), /isn't newer/); + await assert.rejects(prepare(release, { method: "appimage", appImage: "/nonexistent/x" }, null, "0.3.1"), /isn't newer/); +}); diff --git a/app/updater.js b/app/updater.js new file mode 100644 index 0000000..85caab5 --- /dev/null +++ b/app/updater.js @@ -0,0 +1,250 @@ +// Update checks and self-update for the desktop app (docs/releasing.md). +// +// The newest version is GitHub's "latest" release of saphid/frame-control. Drafts +// and pre-releases never count, so a build reaches people only when the +// maintainer publishes it after testing (scripts/publish-release.sh). That +// script attaches update.json (version, notes, each asset's SHA-256), read +// through github.com's latest/download link: the REST API allows only 60 +// unauthenticated requests an hour per IP address, shared by everyone behind +// the same router, so it's only the fallback. +// +// Every download is checked against the SHA-256 digest GitHub records for the +// asset before anything is replaced. How the update is applied: +// macOS the .zip: unpacked next to the running app, swapped in by a small +// script once the app has quit, then reopened. +// Windows the NSIS installer, run silently over the current install; it +// reopens the app. A copy unpacked from the .zip is updated by hand. +// Linux the AppImage replaces itself; .deb installs are updated by hand. +// When the app can't update itself it opens the release page instead. +const { execFile, spawn } = require("child_process"); +const crypto = require("crypto"); +const fs = require("fs"); +const https = require("https"); +const os = require("os"); +const path = require("path"); + +const REPO = "saphid/frame-control"; // renamed from saphid/steam-frame; GitHub redirects the old name +const LATEST = `https://api.github.com/repos/${REPO}/releases/latest`; +const MANIFEST = `https://github.com/${REPO}/releases/latest/download/update.json`; +const RELEASES = `https://github.com/${REPO}/releases`; + +// "0.3.1" or "v0.3.1" -> [0, 3, 1]; pre-release suffixes sort before the release. +function parseVersion(v) { + const m = String(v || "").trim().replace(/^v/i, "").match(/^(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z.-]+))?$/); + return m ? { nums: [+m[1], +m[2], +m[3]], pre: m[4] || null } : null; +} + +function isNewer(candidate, current) { + const a = parseVersion(candidate), b = parseVersion(current); + if (!a || !b) return false; + for (let i = 0; i < 3; i++) if (a.nums[i] !== b.nums[i]) return a.nums[i] > b.nums[i]; + if (a.pre === b.pre) return false; + if (!a.pre) return true; // 1.0.0 is newer than 1.0.0-beta + if (!b.pre) return false; + return a.pre > b.pre; +} + +// The asset this copy of the app updates from, by the names electron-builder gives them. +function assetName(platform, arch, method) { + if (method === "mac-zip") return `Frame-Control-mac-${arch}.zip`; + if (method === "nsis") return `Frame-Control-Setup-${arch}.exe`; + if (method === "appimage") return `Frame-Control-linux-${arch === "x64" ? "x86_64" : arch}.AppImage`; + return null; +} + +// How this copy can update itself: mac-zip, nsis, appimage, or manual (with why). +function updateMethod({ platform, isPackaged, execPath, env, exists, writable }) { + if (!isPackaged) return { method: "manual", why: "running from a source checkout" }; + if (platform === "darwin") { + const bundle = macBundle(execPath); + if (!bundle) return { method: "manual", why: "can't find the app bundle" }; + if (bundle.includes("/AppTranslocation/") || bundle.startsWith("/Volumes/")) { + return { method: "manual", why: "move Frame Control to Applications first" }; + } + if (!writable(path.dirname(bundle))) return { method: "manual", why: `${path.dirname(bundle)} isn't writable` }; + return { method: "mac-zip", bundle }; + } + if (platform === "win32") { + // electron-builder's NSIS install puts its uninstaller next to the app. + const dir = path.dirname(execPath); + if (exists(path.join(dir, "Uninstall Frame Control.exe"))) return { method: "nsis" }; + return { method: "manual", why: "not installed with the installer" }; + } + if (platform === "linux" && env.APPIMAGE) { + if (!writable(path.dirname(env.APPIMAGE))) return { method: "manual", why: "the AppImage's folder isn't writable" }; + return { method: "appimage", appImage: env.APPIMAGE }; + } + return { method: "manual", why: "installed from a package" }; +} + +function macBundle(execPath) { + const i = execPath.indexOf(".app/Contents/MacOS/"); + return i < 0 ? null : execPath.slice(0, i + 4); +} + +function get(url, { headers = {}, timeout = 20000, redirects = 5 } = {}) { + return new Promise((resolve, reject) => { + const req = https.get(url, { headers: { "user-agent": "FrameControl-updater", ...headers }, timeout }, (res) => { + if ([301, 302, 303, 307, 308].includes(res.statusCode) && res.headers.location && redirects > 0) { + res.resume(); + const next = new URL(res.headers.location, url); + if (next.protocol !== "https:") return reject(new Error("refusing a non-HTTPS redirect")); + return resolve(get(next.href, { headers, timeout, redirects: redirects - 1 })); + } + if (res.statusCode !== 200) { res.resume(); return reject(new Error(`HTTP ${res.statusCode} from ${new URL(url).host}`)); } + resolve(res); + }); + req.on("timeout", () => req.destroy(new Error("timed out"))); + req.on("error", reject); + }); +} + +async function getJson(url, headers) { + const res = await get(url, { headers }); + let body = ""; + for await (const chunk of res) body += chunk; + return JSON.parse(body); +} + +// update.json and the API's release both become { version, notes, page, assets }. +function fromManifest(m) { + if (!parseVersion(m.version) || !Array.isArray(m.assets)) throw new Error("update.json is malformed"); + const base = `https://github.com/${REPO}/releases/download/v${String(m.version).replace(/^v/i, "")}/`; + return { version: String(m.version).replace(/^v/i, ""), notes: String(m.notes || "").slice(0, 4000), + page: m.page || RELEASES, + // Assets always come from this repository's release, whatever the manifest says. + assets: m.assets.map((a) => ({ name: String(a.name), url: base + encodeURIComponent(String(a.name)), + size: a.size, digest: a.digest || null })) }; +} + +function fromApi(r) { + if (r.draft || r.prerelease) throw new Error("GitHub returned an unpublished release"); + return { version: String(r.tag_name || "").replace(/^v/i, ""), notes: String(r.body || "").slice(0, 4000), + page: r.html_url || RELEASES, + assets: (r.assets || []).map((a) => ({ name: a.name, url: a.browser_download_url, size: a.size, + digest: a.digest || null })) }; +} + +async function latestRelease() { + try { + return fromManifest(await getJson(MANIFEST)); + } catch (e) { + if (!/HTTP 404/.test(e.message)) throw e; // releases before update.json existed + } + return fromApi(await getJson(LATEST, { accept: "application/vnd.github+json" })); +} + +async function download(asset, dest, onProgress) { + const m = /^sha256:([0-9a-f]{64})$/.exec(asset.digest || ""); + if (!m) throw new Error(`GitHub has no SHA-256 for ${asset.name}, so it can't be checked`); + const res = await get(asset.url, { timeout: 60000 }); + const total = +res.headers["content-length"] || asset.size || 0; + const hash = crypto.createHash("sha256"); + // "wx": a new file only, never through an existing file or symlink at that path. + const out = fs.createWriteStream(dest, { mode: 0o755, flags: "wx" }); + let done = 0; + await new Promise((resolve, reject) => { + res.on("data", (chunk) => { hash.update(chunk); done += chunk.length; onProgress && onProgress(done, total); }); + res.on("error", reject); + out.on("error", reject); + out.on("finish", resolve); + res.pipe(out); + }); + if (hash.digest("hex") !== m[1]) { + fs.rmSync(dest, { force: true }); + throw new Error(`${asset.name} didn't match its SHA-256; nothing was changed`); + } +} + +const run = (cmd, args) => new Promise((resolve, reject) => + execFile(cmd, args, { timeout: 120000 }, (err, stdout, stderr) => err ? reject(new Error((stderr || err.message).trim())) : resolve(stdout))); + +// Waits for this process to exit, swaps the new bundle in (putting the old one +// back if that fails), and reopens the app. +const MAC_SWAP = `set -u +pid="$1"; app="$2"; new="$3"; stage="$4" +while kill -0 "$pid" 2>/dev/null; do sleep 0.2; done +old="$stage/old.app" +if mv "$app" "$old"; then + if mv "$new" "$app"; then rm -rf "$old"; else mv "$old" "$app"; fi +fi +xattr -dr com.apple.quarantine "$app" 2>/dev/null +rm -rf "$stage" +open "$app" +`; + +async function applyMac(release, bundle, onProgress) { + const asset = release.assets.find((a) => a.name === assetName("darwin", process.arch, "mac-zip")); + if (!asset) throw new Error(`the release has no ${assetName("darwin", process.arch, "mac-zip")}`); + // Staged beside the app, so the final move stays on one volume. + const stage = fs.mkdtempSync(path.join(path.dirname(bundle), ".frame-control-update-")); + try { + const zip = path.join(stage, asset.name); + await download(asset, zip, onProgress); + await run("/usr/bin/ditto", ["-x", "-k", zip, stage]); + fs.rmSync(zip, { force: true }); + const name = fs.readdirSync(stage).find((n) => n.endsWith(".app")); + if (!name) throw new Error("the download has no app in it"); + const fresh = path.join(stage, name); + const version = (await run("/usr/bin/plutil", ["-extract", "CFBundleShortVersionString", "raw", + path.join(fresh, "Contents", "Info.plist")])).trim(); + if (version !== release.version) throw new Error(`the download is version ${version}, not ${release.version}`); + const script = path.join(stage, "swap.sh"); + fs.writeFileSync(script, MAC_SWAP); + return () => spawn("/bin/sh", [script, String(process.pid), bundle, fresh, stage], + { detached: true, stdio: "ignore" }).unref(); + } catch (e) { + fs.rmSync(stage, { recursive: true, force: true }); + throw e; + } +} + +async function applyNsis(release, onProgress) { + const asset = release.assets.find((a) => a.name === assetName("win32", process.arch, "nsis")); + if (!asset) throw new Error(`the release has no ${assetName("win32", process.arch, "nsis")}`); + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "frame-control-update-")); + const exe = path.join(dir, asset.name); + await download(asset, exe, onProgress); + // /S: silent, into the existing install. --force-run: open the app afterwards. + return () => spawn(exe, ["--updated", "/S", "--force-run"], { detached: true, stdio: "ignore" }).unref(); +} + +async function applyAppImage(release, appImage, onProgress) { + const asset = release.assets.find((a) => a.name === assetName("linux", process.arch, "appimage")); + if (!asset) throw new Error(`the release has no ${assetName("linux", process.arch, "appimage")}`); + // A private folder beside the AppImage, so the final rename stays on one filesystem. + const stage = fs.mkdtempSync(path.join(path.dirname(appImage), ".frame-control-update-")); + try { + const next = path.join(stage, asset.name); + await download(asset, next, onProgress); + fs.chmodSync(next, 0o755); + fs.renameSync(next, appImage); // the running copy keeps its open file + } finally { + fs.rmSync(stage, { recursive: true, force: true }); + } + // Without FUSE the AppImage runs extracted (--appimage-extract-and-run, which isn't passed + // on to the app); a FUSE mount lives under /tmp/.mount_*. Keep the same mode on restart. + const extracted = process.env.APPIMAGE_EXTRACT_AND_RUN === "1" || !process.execPath.includes("/.mount_"); + const env = { ...process.env, APPIMAGE: appImage, ...(extracted ? { APPIMAGE_EXTRACT_AND_RUN: "1" } : {}) }; + // Started only once this process has exited, or the new copy would lose the single-instance lock. + return () => spawn("/bin/sh", ["-c", 'while kill -0 "$1" 2>/dev/null; do sleep 0.2; done; exec "$2"', + "sh", String(process.pid), appImage], { detached: true, stdio: "ignore", env }).unref(); +} + +// Downloads and prepares the update; returns a function that starts the swap, +// to be called just before the app quits. +async function prepare(release, how, onProgress, current) { + if (!isNewer(release.version, current)) throw new Error(`${release.version} isn't newer than ${current}`); + if (how.method === "mac-zip") return applyMac(release, how.bundle, onProgress); + if (how.method === "nsis") return applyNsis(release, onProgress); + if (how.method === "appimage") return applyAppImage(release, how.appImage, onProgress); + throw new Error(how.why || "this copy can't update itself"); +} + +function writable(dir) { + try { fs.accessSync(dir, fs.constants.W_OK); return true; } catch { return false; } +} + +module.exports = { REPO, RELEASES, parseVersion, isNewer, assetName, updateMethod, macBundle, latestRelease, + fromManifest, fromApi, + download, prepare, writable }; diff --git a/compat-db/README.md b/compat-db/README.md index 5c81211..d851728 100644 --- a/compat-db/README.md +++ b/compat-db/README.md @@ -1,9 +1,32 @@ # compat-db: Frame Control's compatibility database A private [Lakebed](https://docs.lakebed.dev/) capsule holding compatibility -reports for Android apps on the Steam Frame. For now only the maintainer's -copy of Frame Control has the key to read or write it. Everyone else's reports -stay on their own Mac (see `shared()` in `ui/frame_compat_db.py`). +reports for Android apps on the Steam Frame. Only the maintainer's copy of +Frame Control has the key to read or write it (see `shared()` in +`ui/frame_compat_db.py`). Everyone else's reports stay on their computer +unless they turn on **Share compatibility results**. Then the reports also go +to PostHog as `compat_report` events, and the maintainer syncs them in (below). + +## Community reports + +```sh +python3 ui/frame_compat_db.py sync --dry-run # what would be added +python3 ui/frame_compat_db.py sync # add them +``` + +`sync` reads `compat_report` events through PostHog's query API and adds +them with `via` set to `community`, `community-probe` or `community-install`. +It skips invalid reports and anything over 30 per reporter per day. Each run +re-reads the last 30 days, because an offline copy sends its reports late, +with the time they were made. `posthog-sync.json`, next to the outbox, +remembers which reports it has handled and each reporter's daily count, so +nothing is added twice and the cap holds across runs. + +It needs: + +- the PostHog project id: `"project"` in `ui/telemetry.json` +- a personal API key with `query:read`: `POSTHOG_PERSONAL_API_KEY`, or in the + Keychain (service `frame-control-posthog`, account `personal-api-key`) - Live: `https://frame-compat.lakebed.app` (deploy `dep_dDmcsosVSiFirpW6`, claimed, so it doesn't expire). The browser page only says it's private. diff --git a/docs/frame-control.md b/docs/frame-control.md index 0dfb4ae..34796d9 100644 --- a/docs/frame-control.md +++ b/docs/frame-control.md @@ -47,9 +47,11 @@ python3 ui/server.py # anywhere: then open http://127.0.0.1:47810 Steam library), then launch, stop, test or remove it. **Report an APK** records whether any APK worked (F-Droid or not: pick a file, type a package, or use an installed app). Your reports are saved on your computer and change the verdicts - you see. They aren't uploaded anywhere: the shared database is maintainer-only - for now (see [compat-db/README.md](../compat-db/README.md)). Uses the app's bundled - `adb`, or yours if you have one. + you see. With **Share compatibility results** on (Privacy & updates), they also + go to the shared database ([privacy.md](privacy.md), + [compat-db/README.md](../compat-db/README.md)). A failed install records + itself when the APK was the problem, and after an install the app offers a + 20-second test. Uses the app's bundled `adb`, or yours if you have one. - **Android display**: pick a running Lepton instance (by the app in it) and set its resolution (Native 1920×1080, or Sharp 2560×1440 with density scaled to match), UI scale (Smaller / Default / Larger, or an exact dpi) and text size @@ -139,5 +141,6 @@ npm run dist:win # Windows: installer and .zip npm run dist:linux # Linux: AppImage and .deb, x64 and arm64 ``` -Pushing a `v*` tag builds all three in GitHub Actions and attaches them to the -release (`.github/workflows/release.yml`). +Pushing a `v*` tag builds all three in GitHub Actions and attaches them to a +draft release (`.github/workflows/release.yml`). Running copies are offered it +once you publish it: see [releasing.md](releasing.md). diff --git a/docs/privacy.md b/docs/privacy.md new file mode 100644 index 0000000..af44eae --- /dev/null +++ b/docs/privacy.md @@ -0,0 +1,133 @@ +# Privacy and analytics + +Frame Control sends anonymous analytics to [PostHog](https://posthog.com) +(EU cloud) so the maintainer can see how many people use it, which features +matter and where installs fail. You choose how much in **Privacy & updates**, +the last panel on the page. `ui/frame_telemetry.py` is the whole +implementation. + +## The three levels + +| Level | Default | What it sends | +|---|---|---| +| Anonymous usage statistics | On, after a notice on first run | The events in the table below | +| Share compatibility results | Off | Your Android compatibility reports and tests | +| Send error details | Off | Scrubbed error messages and tracebacks | + +Nothing is sent until the first-run notice has been shown. The notice's +**Share more to help fix problems** button turns on the second and third +levels together. Either can be turned off later. Turning a level off +deletes that level's events that haven't been sent yet. + +**Show what's been sent** in the panel lists the last 50 events that left your +computer, exactly as they were sent. + +## Anonymous + +- Events carry a random id, made when Frame Control first runs and kept in + its data folder (`telemetry/settings.json`). It isn't derived from your + computer, account or network. To get a new one, delete that file. +- Events are sent without person profiles (`$process_person_profile: false`) + and without location lookup (`$geoip_disable: true`). The PostHog project + is also set to discard client IP addresses. +- Every event includes the app version, OS name (macOS, Windows or Linux), + CPU architecture and Python version. + +## Usage events + +| Event | When | Properties besides the common ones | +|---|---|---| +| `app_installed` | First run | | +| `app_updated` | First run of a new version | `from_version` | +| `app_opened` | At most once a day | | +| `frame_connected` | The first time a SteamOS build is seen | `steamos_build`, `steamos_version` | +| `tab_viewed` | The first click on each tab in a session | `tab` | +| `install_finished` | Any install finishes, working or not | `kind` (apk, flatpak, steam, title, web), `ok`, `seconds`, `error_category`, `installer_code`, and see below | +| `update_offered`, `update_started`, `update_failed` | The update banner | `to_version`, `error_category` | + +`install_finished` never includes a file name, path or error message. An +error becomes one category from a fixed list (for example `apk_wrong_abi` or +`frame_unreachable`), plus Android's own `INSTALL_FAILED_…` code when there +is one. It names what was installed only when that's already public: + +- F-Droid catalogue apps: `package`. Never the version, since a local build can reuse a + catalogue app's package name +- Flathub apps: `flatpak_id` +- Steam games: `steam_appid` +- A sideloaded title: only its runtime (Proton or Linux) + +Any other APK is sent as `catalog: false`, with no name. + +## Compatibility results (opt-in) + +Each report becomes a `compat_report` event with the fields the Report dialog +shows: package, version, result or rating, your notes, how it was run, and the +SteamOS and Lepton builds. Before sending: + +- the notes, app name and version are scrubbed like error messages (see + below) +- the APK's source is kept only if it's `F-Droid` or the public host name of + a download link (`https://example.com/…`). File names, user names, + passwords, ports, paths, IP addresses and local host names are dropped + +When you turn this on, reports you made earlier on this computer are shared +too. + +The maintainer's `python3 ui/frame_compat_db.py sync` copies these events +into the compatibility database, marked `via=community…`. It takes at most +30 per reporter per day. + +## Error details (opt-in) + +`$exception` events carry an error message, the Frame Control file, line and +function it came from, and the request that failed (for example +`POST /api/android install`). Before anything is sent, the message is +scrubbed: + +- your home folder becomes `~`, and any user name becomes `` +- IP and MAC addresses, email addresses, `.local`, `.lan` and Tailscale host + names, Steam ids, SSH and PEM keys, API tokens and long hex strings are + replaced +- URLs are cut down to their scheme and a public host name, or ``. User + names, passwords, ports, paths and queries are dropped +- `token=`, `key=`, `password=` and similar values are replaced + +The same error is sent at most once every 10 minutes. + +## Report a problem + +**Report a problem** is the warning-sign button in the header, also in the +Privacy panel and under **Help → Report a Problem…**. It files a public issue +on [GitHub](https://github.com/saphid/frame-control/issues) through the +website's feedback service, so no GitHub account is needed. It works whatever +the analytics settings are, because the person sends it deliberately. + +With **Include diagnostics** ticked (the default), the report adds: + +- the app version and whether it's a built app +- the OS, its release and CPU, and the Python version +- the Frame's SteamOS build, if it has connected since the app started +- which analytics levels are on + +**Also include recent activity and the server log** is off by default, +because those lines can name files and apps. When ticked, it adds the newest +Activity lines and server log lines, without the request lines. + +Everything is scrubbed like error details and limited to what fits in the +issue. Environment details are kept first, then the newest lines. **Show +exactly what's included** shows the snapshot that will be sent, and later +activity isn't added to it. If the service can't be reached, the dialog offers +**Copy report** and **Open on GitHub instead**, a prefilled issue. + +## Turning it all off + +Untick the boxes, or set `DO_NOT_TRACK=1` or `FRAME_CONTROL_TELEMETRY=0` in +the environment that starts Frame Control. A copy run from a source checkout +never sends anything unless `FRAME_CONTROL_TELEMETRY=1` is set. + +## Update checks + +The desktop app asks GitHub for the latest release shortly after starting, +then every 6 hours (`api.github.com/repos/saphid/frame-control/releases/latest`). +That request carries no id. To stop it, set +`FRAME_CONTROL_NO_UPDATE_CHECK=1`. See [releasing.md](releasing.md). diff --git a/docs/releasing.md b/docs/releasing.md new file mode 100644 index 0000000..7db5e3a --- /dev/null +++ b/docs/releasing.md @@ -0,0 +1,59 @@ +# Releasing and updates + +Frame Control checks for updates itself. The desktop app offers a new version +only once it's GitHub's **latest release**, and drafts and pre-releases never +count. So a build reaches people only when you publish it, after testing it. + +## Steps + +1. Bump `version` in `app/package.json`, commit, and push a tag: + + ```sh + git tag v0.4.0 && git push origin v0.4.0 + ``` + + `.github/workflows/release.yml` builds macOS, Windows and Linux, and + attaches everything to a **draft** release for that tag. Nobody is + offered a draft. + +2. Download the draft's installers and test them. An installed copy of the + previous version won't offer the draft, so install it directly. + +3. Write the release notes on the draft. The update banner links to them. + +4. Publish: + + ```sh + scripts/publish-release.sh v0.4.0 + ``` + + The script checks that all eight installers are attached, each with the + SHA-256 digest GitHub records. It attaches `update.json` (the version, the + notes and each installer's digest), then publishes the release and marks it + latest. From then on, running copies see the update. They check about 8 + seconds after starting, then every 6 hours, and anyone can use **Check for + Updates…** (the app menu on macOS, the Help menu elsewhere). + +To pull a bad release, mark the previous one as latest +(`gh release edit v0.3.9 --latest`) or turn the bad one back into a draft. +Copies that already updated stay on it. Nothing downgrades them. + +## How a copy updates itself + +`app/updater.js` reads `update.json` from +`github.com/saphid/frame-control/releases/latest/download/`. It falls back to the +REST API only when a release has no manifest, because the API allows just 60 +unauthenticated requests an hour per IP address, shared by a whole household. +Then it downloads the installer for its platform and checks it +against the SHA-256 digest GitHub publishes for the asset. It refuses if the +digest is missing or doesn't match. Then: + +| Installed from | Update | +|---|---| +| macOS `.dmg`, app in a writable folder such as Applications | The `.zip` is unpacked next to the app and its version checked. After the app quits, a small script swaps the new app in, putting the old one back if that fails, and reopens it. Updates don't get the download quarantine, so there's no `xattr` step. | +| Windows installer | The new `Setup` runs silently over the install (`/S --force-run`) and reopens the app. | +| Linux AppImage | The new AppImage replaces the old file and is started. | +| macOS app still on the disk image or translocated, Windows `.zip`, Linux `.deb` | The banner opens the release page instead. | + +Version 0.3.1 and earlier have no updater, so people on them have to download +the new version once by hand. diff --git a/scripts/publish-release.sh b/scripts/publish-release.sh new file mode 100755 index 0000000..82b5c10 --- /dev/null +++ b/scripts/publish-release.sh @@ -0,0 +1,45 @@ +#!/bin/sh +# Publish a tested draft release so running copies of Frame Control offer it +# (docs/releasing.md). Checks every installer is attached with a SHA-256 +# digest first, since the app's updater refuses assets without one, then +# attaches update.json, the manifest the updater reads. +# Usage: scripts/publish-release.sh v0.4.0 +set -eu +tag="${1:?usage: $0 vX.Y.Z}" +repo=saphid/frame-control +expected="Frame-Control-mac-arm64.dmg Frame-Control-mac-arm64.zip Frame-Control-Setup-x64.exe +Frame-Control-win-x64.zip Frame-Control-linux-x86_64.AppImage Frame-Control-linux-arm64.AppImage +Frame-Control-linux-amd64.deb Frame-Control-linux-arm64.deb" + +info=$(gh release view "$tag" -R "$repo" --json isDraft,isPrerelease,assets) +version=$(sed -n 's/.*"version": *"\([^"]*\)".*/\1/p' "$(dirname "$0")/../app/package.json") +[ "v$version" = "$tag" ] || echo "note: app/package.json here says $version (the release was built from the tag)" + +missing="" +for name in $expected; do + digest=$(printf '%s' "$info" | python3 -c 'import json,sys +d=json.load(sys.stdin); n=sys.argv[1] +print(next((a.get("digest") or "" for a in d["assets"] if a["name"]==n), "absent"))' "$name") + case "$digest" in + sha256:*) echo "ok $name" ;; + absent) echo "MISSING $name"; missing=1 ;; + *) echo "NO HASH $name"; missing=1 ;; + esac +done +[ -z "$missing" ] || { echo "not publishing: fix the assets above" >&2; exit 1; } + +# update.json: what running copies read (app/updater.js), from github.com's +# latest/download link rather than the rate-limited REST API. +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT +gh release view "$tag" -R "$repo" --json tagName,url,body,assets | python3 -c 'import json,sys +d=json.load(sys.stdin) +names=set(sys.argv[1].split()) +print(json.dumps({"version": d["tagName"].lstrip("v"), "page": d["url"], "notes": d["body"][:4000], + "assets": [{"name": a["name"], "size": a["size"], "digest": a["digest"]} + for a in d["assets"] if a["name"] in names]}, indent=1))' "$expected" > "$tmp/update.json" +gh release upload "$tag" -R "$repo" "$tmp/update.json" --clobber +echo "ok update.json" + +gh release edit "$tag" -R "$repo" --draft=false --prerelease=false --latest +echo "published $tag; running copies will offer it at their next check" diff --git a/tests/test_telemetry.py b/tests/test_telemetry.py new file mode 100644 index 0000000..19043bc --- /dev/null +++ b/tests/test_telemetry.py @@ -0,0 +1,405 @@ +"""Anonymous analytics (ui/frame_telemetry.py): what's collected at each level, +what's scrubbed, and that nothing is sent without a key, the notice, or consent. + +Run: python3 -m unittest discover -s tests +""" +import json +import os +import sys +import tempfile +import threading +import time +import unittest +from http.server import BaseHTTPRequestHandler, HTTPServer +from pathlib import Path +from unittest import mock + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT / "ui")) + +import frame_compat_db as db # noqa: E402 +import frame_report as fr # noqa: E402 +import frame_telemetry as tm # noqa: E402 + + +class Base(unittest.TestCase): + """A packaged build with a key, its state in a temp folder.""" + + def setUp(self): + tmp = tempfile.TemporaryDirectory() + self.addCleanup(tmp.cleanup) + state = Path(tmp.name) + for name, value in (("STATE", state), ("SETTINGS", state / "settings.json"), + ("OUTBOX", state / "outbox.jsonl"), ("SENT", state / "sent.jsonl")): + p = mock.patch.object(tm, name, value) + p.start() + self.addCleanup(p.stop) + env = mock.patch.dict(os.environ, {"FRAME_CONTROL_POSTHOG_KEY": "phc_test", "FRAME_CONTROL_PACKAGED": "1", + "FRAME_CONTROL_POSTHOG_HOST": "http://127.0.0.1:9", + "FRAME_CONTROL_VERSION": "9.9.9"}) + env.start() + self.addCleanup(env.stop) + for k in ("DO_NOT_TRACK", "FRAME_CONTROL_TELEMETRY"): + os.environ.pop(k, None) + tm._seen_errors.clear() + + def queued(self): + return tm._read_lines(tm.OUTBOX) + + +class Gates(Base): + def test_blocked_without_key_or_in_a_checkout_or_by_do_not_track(self): + self.assertIsNone(tm.blocked()) + with mock.patch.dict(os.environ, {"FRAME_CONTROL_POSTHOG_KEY": ""}), \ + mock.patch.object(tm, "HERE", Path(tempfile.gettempdir()) / "no-config-here"): + self.assertIn("key", tm.blocked()) + with mock.patch.dict(os.environ, {"FRAME_CONTROL_PACKAGED": ""}): + self.assertIn("source checkout", tm.blocked()) + with mock.patch.dict(os.environ, {"DO_NOT_TRACK": "1"}): + self.assertIn("DO_NOT_TRACK", tm.blocked()) + self.assertFalse(tm.capture("app_opened")) + self.assertFalse(tm.OUTBOX.exists()) + + def test_usage_is_on_by_default_the_others_are_opt_in(self): + self.assertTrue(tm.capture("app_opened")) + self.assertFalse(tm.capture("compat_report", {}, level="compat")) + self.assertFalse(tm.capture("$exception", {}, level="diagnostics")) + self.assertEqual([e["event"] for e in self.queued()], ["app_opened"]) + + def test_events_are_anonymous(self): + tm.capture("app_opened") + e = self.queued()[0] + self.assertEqual(e["distinct_id"], tm.settings()["id"]) + self.assertIs(e["properties"]["$process_person_profile"], False) + self.assertIs(e["properties"]["$geoip_disable"], True) + self.assertEqual(e["properties"]["app_version"], "9.9.9") + + def test_turning_a_level_off_drops_its_unsent_events(self): + tm.update_settings({"diagnostics": True}) + tm.capture("app_opened") + tm.diagnostic("somewhere", RuntimeError("boom")) + self.assertEqual(len(self.queued()), 2) + tm.update_settings({"diagnostics": False}) + self.assertEqual([e["event"] for e in self.queued()], ["app_opened"]) + tm.update_settings({"usage": False}) + self.assertEqual(self.queued(), []) + self.assertFalse(tm.capture("app_opened")) + + def test_the_same_error_is_sent_once_in_a_while(self): + tm.update_settings({"diagnostics": True}) + for _ in range(3): + tm.diagnostic("POST /api/android install", RuntimeError("boom")) + self.assertEqual(len(self.queued()), 1) + + def test_page_events_are_checked(self): + self.assertTrue(tm.page_event({"event": "tab_viewed", "properties": {"tab": "android", "extra": "x"}})["queued"]) + self.assertEqual(self.queued()[0]["properties"].get("extra"), None) + with self.assertRaises(ValueError): + tm.page_event({"event": "anything_else"}) + with self.assertRaises(ValueError): + tm.page_event({"event": "tab_viewed", "properties": {"tab": "/Users/me/secret"}}) + + +class Lifecycle(Base): + def test_install_update_and_one_open_a_day(self): + tm.app_started() + tm.app_started() + self.assertEqual([e["event"] for e in self.queued()], ["app_installed", "app_opened"]) + with mock.patch.dict(os.environ, {"FRAME_CONTROL_VERSION": "10.0.0"}): + tm.app_started() + e = self.queued()[-1] + self.assertEqual((e["event"], e["properties"]["from_version"]), ("app_updated", "9.9.9")) + + def test_frame_build_once(self): + tm.frame_seen("20260922.1", "3.8") + tm.frame_seen("20260922.1", "3.8") + self.assertEqual(len(self.queued()), 1) + + +class Sending(Base): + def serve(self, status=200): + got = [] + + class H(BaseHTTPRequestHandler): + def do_POST(self): + got.append((self.path, json.loads(self.rfile.read(int(self.headers["Content-Length"]))))) + self.send_response(status) + self.end_headers() + self.wfile.write(b'{"status": 1}') + + def log_message(self, *a): + pass + + httpd = HTTPServer(("127.0.0.1", 0), H) + threading.Thread(target=httpd.serve_forever, daemon=True).start() + self.addCleanup(httpd.server_close) + self.addCleanup(httpd.shutdown) + os.environ["FRAME_CONTROL_POSTHOG_HOST"] = f"http://127.0.0.1:{httpd.server_port}" + return got + + def test_nothing_is_sent_before_the_notice_was_shown(self): + got = self.serve() + tm.capture("app_opened") + self.assertEqual(tm.flush(), 0) + self.assertEqual(got, []) + tm.update_settings({"noticeShown": True}) + self.assertEqual(tm.flush(), 1) + path, body = got[0] + self.assertEqual((path, body["api_key"], body["batch"][0]["event"]), ("/batch/", "phc_test", "app_opened")) + self.assertEqual(self.queued(), []) + self.assertEqual([e["event"] for e in tm.state()["sent"]], ["app_opened"]) + + def test_a_failed_send_keeps_the_events(self): + self.serve(status=500) + tm.update_settings({"noticeShown": True}) + tm.capture("app_opened") + self.assertEqual(tm.flush(), 0) + self.assertEqual(len(self.queued()), 1) + + +class Scrub(unittest.TestCase): + def test_personal_details_are_removed(self): + home = str(Path.home()) + text = (f"open {home}/Downloads/My Game.apk failed; ssh alex@192.168.1.20 (frame.local) " + "key ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM steam 76561198000000000 mac 3c:22:fb:12:34:56 " + "url https://example.com/private/path?token=abc phc_abcdefghijklmnopqrstu C:\\Users\\Bob\\x " + "/home/carol/y") + out = tm.scrub(text) + for leaked in (home, "192.168.1.20", "frame.local", "AAAAC3Nza", "76561198000000000", "3c:22:fb", + "private/path", "phc_abcdefghijklmnopqrstu", "Bob", "carol", "alex@"): + self.assertNotIn(leaked, out) + self.assertIn("https://example.com/…", out) + self.assertIn("~/Downloads", out) + + def test_categories(self): + self.assertEqual(tm.categorize("adb: failed to install: INSTALL_FAILED_NO_MATCHING_ABIS: x"), + ("android_installer", "INSTALL_FAILED_NO_MATCHING_ABIS")) + self.assertEqual(tm.categorize("X has no arm64-v8a build (armeabi-v7a)")[0], "apk_wrong_abi") + self.assertEqual(tm.categorize("ssh: connect to host 10.0.0.2 port 22: Connection refused")[0], + "frame_unreachable") + self.assertEqual(tm.categorize("something new")[0], "other") + + +class Compat(Base): + def test_reports_are_shared_only_after_opting_in_without_file_names(self): + r = {"id": "r1", "package": "org.example", "version": "1.0", "rating": "works", "via": "user", + "notes": f"from {Path.home()}/x", "source": "MyPrivateBuild.apk", "date": "2026-09-28T10:00:00"} + self.assertFalse(tm.compat_report(r)) + tm.update_settings({"compat": True}) + self.assertTrue(tm.compat_report(r)) + p = self.queued()[-1]["properties"] + self.assertEqual((p["package"], p["rating"], p["id"]), ("org.example", "works", "r1")) + self.assertNotIn("source", p) + self.assertNotIn(str(Path.home()), p["notes"]) + r2 = dict(r, id="r2", source="https://f-droid.org/repo/org.example_1.apk") + tm.compat_report(r2) + self.assertEqual(self.queued()[-1]["properties"]["source"], "https://f-droid.org/…") + + def test_opting_in_shares_earlier_local_reports(self): + with mock.patch.object(db, "shared", return_value=False), \ + mock.patch.object(db, "_outbox", return_value=[{"id": "old1", "package": "org.a", "rating": "works", + "date": "2026-09-01T00:00:00"}]): + tm.update_settings({"compat": True}) + tm.update_settings({"compat": True}) # already sent: not again + self.assertEqual([e["properties"]["id"] for e in self.queued() if e["event"] == "compat_report"], ["old1"]) + + +class ApkInstallReports(unittest.TestCase): + """server.apk_installed: an APK that won't install is reported; connection trouble isn't.""" + + def setUp(self): + import server + self.server = server + for target, name in ((server.frame_catalog, "add_report"), (server.frame_telemetry, "install_finished")): + p = mock.patch.object(target, name) + setattr(self, name, p.start()) + self.addCleanup(p.stop) + + def test_wrong_abi_is_an_install_failed_report(self): + info = {"package": "org.x", "version": "2.0", "label": "X"} + self.server.apk_installed(info, None, self.server.frame_android.FrameError( + "X has no arm64-v8a build (armeabi-v7a); Lepton is 64-bit ARM only"), 3.0) + args, kw = self.add_report.call_args + self.assertEqual((args[0], args[1], kw["result"], kw["via"]), ("org.x", "2.0", "install_failed", "install")) + self.assertIs(self.install_finished.call_args[0][1], False) + + def test_connection_trouble_is_not_reported(self): + self.server.apk_installed({"package": "org.x", "version": "2.0"}, None, + self.server.frame_android.FrameError("timed out talking to frame"), 3.0) + self.add_report.assert_not_called() + + def test_private_package_names_stay_here(self): + with mock.patch.dict(self.server.frame_catalog._cache, {"by_pkg": {"org.public": {}}}): + self.server.apk_installed({"package": "com.private.thing", "version": "1"}, {"package": "com.private.thing"}, + None, 2.0) + self.assertIsNone(self.install_finished.call_args[1]["package"]) + self.server.apk_installed({"package": "org.public", "version": "1"}, {"package": "org.public"}, None, 2.0) + self.assertEqual(self.install_finished.call_args[1]["package"], "org.public") + + +class CommunitySync(unittest.TestCase): + def ev(self, i, who="a", day="2026-09-28", **kw): + return ({"id": f"id{i}", "package": "org.x", "rating": "works", "date": f"{day}T00:00:00", + "via": "probe", **kw}, who, f"{day} 10:00:00") + + def test_rows_are_validated_marked_and_capped_per_reporter(self): + events = [self.ev(i) for i in range(5)] + [self.ev(9, who="b", rating="nonsense"), self.ev(10, who="b")] + rows, skipped = db.community_rows(events, {}, cap=3) + self.assertEqual([r["id"] for r in rows], ["id0", "id1", "id2", "id10"]) + self.assertTrue(all(r["via"] == "community-probe" for r in rows)) + self.assertEqual(len(skipped), 3) + + def test_the_cap_and_duplicates_hold_across_syncs(self): + state = {} + rows, _ = db.community_rows([self.ev(i) for i in range(3)], state, cap=3) + self.assertEqual(len(rows), 3) + rows, skipped = db.community_rows([self.ev(i) for i in range(6)], state, cap=3) # overlapping re-read + self.assertEqual(rows, []) + self.assertEqual([why for _, why in skipped], ["over the daily limit for one reporter"] * 3) + + def test_a_malformed_event_is_skipped_not_fatal(self): + rows, skipped = db.community_rows([self.ev(1, via=["probe"]), ("not json", "a", "2026-09-28"), self.ev(2)], {}) + self.assertEqual([r["id"] for r in rows], ["id2"]) + self.assertEqual(len(skipped), 2) + + +class Regressions(Base): + """Findings from the cross-provider review.""" + + def test_urls_lose_credentials_paths_and_private_hosts(self): + for text, leaked in (("https://alice:secret@example.com/private.apk?token=credential", ("alice", "secret", "private", "credential")), + ("https://alice:secret@192.168.1.4/private.apk", ("alice", "192.168", "private")), + ("fe80::1234 and 2001:db8::5", ("fe80", "2001:db8")), + ("sk-proj-abcdefghijklmnopqrstuv", ("abcdefghijk",)), + ("http://frame.local:8080/x", ("frame.local", "8080"))): + out = tm.scrub(text) + for s in leaked: + self.assertNotIn(s, out, (text, out)) + + def test_compat_labels_versions_and_sources_are_scrubbed(self): + tm.update_settings({"compat": True}) + tm.compat_report({"id": "r9", "package": "org.x", "rating": "works", "date": "2026-09-28T00:00:00", + "label": "alice@example.com build", "version": "1.0-alice@example.com", + "source": "https://alice:secret@192.168.1.4/private.apk"}) + p = self.queued()[-1]["properties"] + self.assertNotIn("alice", json.dumps(p)) + self.assertNotIn("source", p) + + def test_an_unsent_report_is_shared_again_after_opting_out_and_in(self): + with mock.patch.object(db, "shared", return_value=False), \ + mock.patch.object(db, "_outbox", return_value=[{"id": "q1", "package": "org.a", "rating": "works", + "date": "2026-09-01T00:00:00"}]): + tm.update_settings({"compat": True}) + tm.update_settings({"compat": False}) + self.assertEqual(self.queued(), []) + tm.update_settings({"compat": True}) + self.assertEqual([e["properties"]["id"] for e in self.queued() if e["event"] == "compat_report"], ["q1"]) + + def test_opting_out_waits_for_a_send_in_progress(self): + tm.update_settings({"noticeShown": True}) + tm.capture("app_opened") + order = [] + started = threading.Event() + + def slow_open(req, timeout): + started.set() + time.sleep(0.3) + order.append("sent") + return mock.MagicMock(__enter__=lambda s: s, __exit__=lambda *a: False, read=lambda: b"{}") + + with mock.patch.object(tm.urllib.request, "urlopen", side_effect=slow_open): + th = threading.Thread(target=tm.flush) + th.start() + started.wait(2) + tm.update_settings({"usage": False}) + order.append("opted out") + th.join() + self.assertEqual(order, ["sent", "opted out"]) + + def test_project_id_comes_from_the_config(self): + with mock.patch.dict(os.environ, {"FRAME_CONTROL_POSTHOG_PROJECT": "12345"}): + self.assertEqual(tm.config()["project"], "12345") + + +class ReportProblem(Base): + """Report a problem: diagnostics are scrubbed and fit the feedback API; sending goes to it.""" + + def serve(self, status=201, reply=None): + got = [] + + class H(BaseHTTPRequestHandler): + def do_POST(self): + got.append(json.loads(self.rfile.read(int(self.headers["Content-Length"])))) + self.send_response(status) + self.end_headers() + self.wfile.write(json.dumps(reply or {"ok": True, "number": 42, + "url": "https://github.com/saphid/frame-control/issues/42"}).encode()) + + def log_message(self, *a): + pass + + httpd = HTTPServer(("127.0.0.1", 0), H) + threading.Thread(target=httpd.serve_forever, daemon=True).start() + self.addCleanup(httpd.server_close) + self.addCleanup(httpd.shutdown) + p = mock.patch.object(fr, "FEEDBACK_URL", f"http://127.0.0.1:{httpd.server_port}/api/feedback") + p.start() + self.addCleanup(p.stop) + return got + + def test_diagnostics_are_scrubbed_and_include_the_log(self): + log = tm.STATE / "server.log" + log.write_text("GET /api/status 200\nTraceback: ssh alice@192.168.1.9 failed in %s/x\n" % Path.home()) + with mock.patch.dict(os.environ, {"FRAME_CONTROL_LOG": str(log)}): + text = fr.diagnostics(["16:00 Install failed: https://bob:pw@example.com/a.apk"], include_logs=True, limit=5000) + self.assertIn("Frame Control 9.9.9", text) + self.assertIn("Traceback", text) + self.assertNotIn("GET /api/status", text) + for leaked in ("alice", "192.168.1.9", str(Path.home()), "bob", "pw@"): + self.assertNotIn(leaked, text) + + def test_a_report_fits_the_api_limit_in_utf16_units(self): + body = {"title": "Live view stops", "message": "It stops 😀 " * 600, "diagnostics": "log 😀 line\n" * 2000} + title, message = fr.compose(body) + self.assertLessEqual(fr.u16(message), fr.MESSAGE_MAX) + self.assertTrue(message.startswith("It stops")) + with self.assertRaises(ValueError): + fr.compose({"title": "hi", "message": "It stops after a minute."}) + + def test_logs_only_when_asked_and_environment_is_kept_first(self): + log = tm.STATE / "server.log" + log.write_text("".join(f"old line {i}\n" for i in range(200)) + "newest line\n") + with mock.patch.dict(os.environ, {"FRAME_CONTROL_LOG": str(log)}): + plain = fr.diagnostics(["Copy Jane Doe tax return.pdf to ~/Downloads"]) + full = fr.diagnostics(["Install failed"], include_logs=True, limit=400) + self.assertNotIn("Jane Doe", plain) + self.assertNotIn("line", plain) + self.assertTrue(full.startswith("Frame Control 9.9.9")) + self.assertIn("Install failed", full) + self.assertIn("newest line", full) + self.assertLessEqual(fr.u16(full), 400) + + def test_the_previewed_diagnostics_are_what_is_sent(self): + got = self.serve() + fr.send({"title": "Live view stops", "message": "It stops after a minute.", "elapsed": 9000, + "diagnostics": "Frame Control 9.9.9\nssh janes-mac.tail12345.ts.net failed"}) + self.assertIn("Frame Control 9.9.9", got[0]["message"]) + self.assertNotIn("janes-mac", got[0]["message"]) + + def test_send_files_an_issue_and_reports_its_number(self): + got = self.serve() + res = fr.send({"kind": "bug", "title": "Live view stops", "message": "It stops after a minute.", + "elapsed": 9000, "github": "@someone"}) + self.assertEqual((res["number"], res["url"]), (42, "https://github.com/saphid/frame-control/issues/42")) + sent = got[0] + self.assertEqual((sent["kind"], sent["github"], sent["website"], sent["elapsed"]), ("bug", "someone", "", 9000)) + self.assertEqual(sent["message"], "It stops after a minute.") + + def test_the_services_error_is_passed_on(self): + self.serve(status=429, reply={"error": "That's a lot of feedback in one hour."}) + with self.assertRaisesRegex(fr.ReportError, "a lot of feedback"): + fr.send({"title": "Live view stops", "message": "It stops after a minute.", "elapsed": 9000}) + + +if __name__ == "__main__": + unittest.main() diff --git a/ui/frame_android.py b/ui/frame_android.py index 3bff853..9f6f789 100644 --- a/ui/frame_android.py +++ b/ui/frame_android.py @@ -106,16 +106,36 @@ def _write_meta(d, meta): ssh(f'cat > {d}/meta.json.tmp && mv {d}/meta.json.tmp {d}/meta.json', input=json.dumps(meta, indent=1)) +# Called after every install, worked or not, as fn(info, meta, error, seconds): +# info is None if the APK couldn't be read, meta None and error set if it failed. +install_hooks = [] + + def install(apk_path, flatscreen=True, name=None, source=None, icon_png=None): - info = apk_info(apk_path) - if icon_png: - info['icon_png'] = icon_png - check_installable(info) - pkg = info['package'] - if not PKG_RE.match(pkg): - raise FrameError(f'unexpected package name {pkg!r}') - with _install_lock: - return _install(apk_path, info, pkg, flatscreen, name, source) + start, info = time.time(), None + try: + info = apk_info(apk_path) + if icon_png: + info['icon_png'] = icon_png + check_installable(info) + pkg = info['package'] + if not PKG_RE.match(pkg): + raise FrameError(f'unexpected package name {pkg!r}') + with _install_lock: + meta = _install(apk_path, info, pkg, flatscreen, name, source) + except FrameError as e: + _after_install(info, None, e, start) + raise + _after_install(info, meta, None, start) + return meta + + +def _after_install(info, meta, error, start): + for hook in install_hooks: + try: + hook(info, meta, error, time.time() - start) + except Exception: + pass # reporting must never change an install's outcome def _install(apk_path, info, pkg, flatscreen, name, source): diff --git a/ui/frame_compat_db.py b/ui/frame_compat_db.py index 98c67c4..1576818 100644 --- a/ui/frame_compat_db.py +++ b/ui/frame_compat_db.py @@ -8,12 +8,18 @@ New reports go to a local outbox first and are sent from there, so nothing is lost offline. A mirror of every report is kept for offline reads. Both live in frame_host.data_dir('compat-db'). Python stdlib only. -CLI: python3 ui/frame_compat_db.py {count|export FILE|import FILE|flush} +Everyone else can opt in to sharing (the Privacy panel): their reports then +also go to PostHog as compat_report events (frame_telemetry.py), and the +maintainer's `sync` pulls them into the database, at most SYNC_DAILY_CAP per +reporter per day, marked via=community[-probe|-install]. + +CLI: python3 ui/frame_compat_db.py {count|export FILE|import FILE|flush|sync} (import restores a backup; reports already in the database are skipped.) """ import json, os, subprocess, sys, threading, time, urllib.error, urllib.parse, urllib.request, uuid import frame_host +import frame_telemetry URL = os.environ.get('FRAME_COMPAT_DB_URL', 'https://frame-compat.lakebed.app') KEYCHAIN = ('frame-control-compat-db', 'app-key') @@ -216,11 +222,147 @@ def add(report): if shared(): flush() _mem['at'] = 0 # refetch on next load + else: + frame_telemetry.compat_report(r) # only if this person opted in to sharing except Exception: pass # stays queued; load() shows it and a later call sends it return r +# ---- community reports: PostHog -> the database (maintainer only) --------------- + +POSTHOG_KEYCHAIN = ('frame-control-posthog', 'personal-api-key') +SYNC_STATE = os.path.join(STATE, 'posthog-sync.json') +SYNC_DAILY_CAP = 30 +COMMUNITY_VIA = {'user': 'community', 'probe': 'community-probe', 'install': 'community-install'} + + +def posthog_personal_key(): + k = os.environ.get('POSTHOG_PERSONAL_API_KEY') + if k: + return k + if frame_host.MAC: + p = subprocess.run(['security', 'find-generic-password', '-s', POSTHOG_KEYCHAIN[0], '-a', + POSTHOG_KEYCHAIN[1], '-w'], capture_output=True, text=True) + if p.returncode == 0 and p.stdout.strip(): + return p.stdout.strip() + raise DBError('No PostHog personal API key (set POSTHOG_PERSONAL_API_KEY, or on macOS the Keychain ' + f'item service {POSTHOG_KEYCHAIN[0]}, account {POSTHOG_KEYCHAIN[1]})') + + +def _posthog_query(sql): + cfg = frame_telemetry.config() + project = os.environ.get('FRAME_CONTROL_POSTHOG_PROJECT') or cfg.get('project') + if not project: + raise DBError('No PostHog project id (ui/telemetry.json "project", or FRAME_CONTROL_POSTHOG_PROJECT)') + # The query API lives on the app host (eu.posthog.com), not the ingestion host (eu.i.posthog.com). + host = cfg['host'].replace('.i.posthog.com', '.posthog.com') + req = urllib.request.Request(f'{host}/api/projects/{urllib.parse.quote(str(project))}/query/', method='POST', + data=json.dumps({'query': {'kind': 'HogQLQuery', 'query': sql}}).encode(), + headers={'authorization': 'Bearer ' + posthog_personal_key(), + 'content-type': 'application/json'}) + try: + with _opener.open(req, timeout=60) as r: + return json.loads(r.read()) + except urllib.error.HTTPError as e: + raise DBError(f'PostHog said HTTP {e.code}: {e.read()[:300]!r}') + except (urllib.error.URLError, TimeoutError, OSError, ValueError) as e: + raise DBError(f"can't reach PostHog: {e}") + + +SYNC_OVERLAP_DAYS = 30 # re-read this far back: offline copies send late, with their original time +SYNC_PAGE = 5000 + + +def community_rows(events, state, cap=SYNC_DAILY_CAP): + """(reports, skipped): compat_report events as database rows. `state` ({"seen": {id: day}, + "counts": {"reporter|day": n}}) persists between syncs, so an event read twice is handled + once and each reporter gets at most `cap` reports a day in total.""" + seen, counts = state.setdefault('seen', {}), state.setdefault('counts', {}) + out, skipped = [], [] + for props, reporter, ts in events: + if isinstance(props, str): + try: + props = json.loads(props) + except ValueError: + props = None + if not isinstance(props, dict): + skipped.append((None, 'unreadable properties')) + continue + bad = [k for k in (*FIELDS, 'id') if props.get(k) is not None and not isinstance(props[k], (str, int, float))] + if bad: + skipped.append((str(props.get('id'))[:60], f'bad field {bad[0]}')) + continue + r = {k: (str(props[k]) if props.get(k) is not None else None) for k in FIELDS} + r['id'] = str(props['id']) if props.get('id') is not None else None + if r['id'] in seen: + continue # handled in an earlier sync (or earlier in this one) + r['via'] = COMMUNITY_VIA.get(r.get('via') or 'user', 'community') + why = problem(r) + if why: + skipped.append((r.get('id'), why)) + continue + day = str(ts)[:10] + seen[r['id']] = day + key_ = f'{reporter}|{day}' + if counts.get(key_, 0) >= cap: + skipped.append((r['id'], 'over the daily limit for one reporter')) + continue + counts[key_] = counts.get(key_, 0) + 1 + out.append(r) + return out, skipped + + +def _sync_state(): + try: + with open(SYNC_STATE) as f: + s = json.load(f) + return s if isinstance(s, dict) else {} + except (OSError, ValueError): + return {} + + +def _save_sync_state(s): + """Forget ids and counts older than the overlap window (plus a margin).""" + cutoff = time.strftime('%Y-%m-%d', time.gmtime(time.time() - (SYNC_OVERLAP_DAYS + 15) * 86400)) + s['seen'] = {k: d for k, d in s.get('seen', {}).items() if d >= cutoff} + s['counts'] = {k: n for k, n in s.get('counts', {}).items() if k.rsplit('|', 1)[-1] >= cutoff} + os.makedirs(STATE, exist_ok=True) + with open(SYNC_STATE + '.tmp', 'w') as f: + json.dump(s, f) + os.replace(SYNC_STATE + '.tmp', SYNC_STATE) + + +def sync(dry_run=False): + """Pull community reports from PostHog into the database. Returns (added, skipped). + Reads the last SYNC_OVERLAP_DAYS each time, since events carry the time they were + made, not when they arrived; the saved state keeps that from adding anything twice.""" + key() # the maintainer's copy only + state = _sync_state() + since = time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(time.time() - SYNC_OVERLAP_DAYS * 86400)) + events = [] + for page in range(40): + res = _posthog_query("SELECT properties, distinct_id, timestamp FROM events " + f"WHERE event = 'compat_report' AND timestamp >= toDateTime('{since}') " + f"ORDER BY timestamp, uuid LIMIT {SYNC_PAGE} OFFSET {page * SYNC_PAGE}") + rows = res.get('results') or [] + events += rows + if len(rows) < SYNC_PAGE: + break + rows, skipped = community_rows(events, state) + if dry_run: + return rows, skipped + if rows: + os.makedirs(STATE, exist_ok=True) + with _lock, open(OUTBOX, 'a') as f: + f.writelines(json.dumps(r, ensure_ascii=False) + '\n' for r in rows) + # Saved before sending: the rows are in the outbox now, and flush retries them if sending fails. + _save_sync_state(state) + flush() # also retries rows a failed earlier sync left in the outbox + _mem['at'] = 0 + return rows, skipped + + def main(): cmd, *args = sys.argv[1:] or ['count'] try: @@ -248,6 +390,12 @@ def main(): 'reports already in the database were not duplicated') elif cmd == 'flush': print(f'{flush()} still queued') + elif cmd == 'sync': + rows, skipped = sync(dry_run='--dry-run' in args) + for rid, why in skipped: + print(f'skipped {rid!r}: {why}', file=sys.stderr) + print(f"{len(rows)} community reports {'found' if '--dry-run' in args else 'added'}, " + f'{len(skipped)} skipped') else: sys.exit(__doc__) except DBError as e: diff --git a/ui/frame_report.py b/ui/frame_report.py new file mode 100644 index 0000000..f898ea5 --- /dev/null +++ b/ui/frame_report.py @@ -0,0 +1,147 @@ +"""Report a problem from inside Frame Control. Python stdlib only. + +The page's Report a problem dialog shows the diagnostics below before anything +is sent, then this sends the report to the website's feedback API +(site/functions/api/feedback.js), which files it as a GitHub issue. The user +needs no GitHub account. Everything collected is scrubbed first +(frame_telemetry.scrub), since the issue is public. +""" +import json +import os +import platform +import sys +import time +import urllib.error +import urllib.request + +import frame_host +import frame_telemetry + +FEEDBACK_URL = os.environ.get('FRAME_CONTROL_FEEDBACK_URL', 'https://frame-control.pages.dev/api/feedback') +ISSUES_URL = 'https://github.com/saphid/frame-control/issues/new' +KINDS = ('bug', 'idea', 'question', 'other') +MESSAGE_MAX = 5000 # the feedback API's limit, in JavaScript (UTF-16) units +TEXT_MAX = 3500 # the person's own text +DIAG_MAX = 1300 # the diagnostics block, so text + diagnostics always fit MESSAGE_MAX +LOG_LINES = 40 +ACTIVITY_LINES = 25 +HEAD = '\n\n---\nDiagnostics from Frame Control (personal details removed):\n```\n' +TAIL = '\n```' + +frame = {} # the Frame's last known SteamOS build, set by server.status() + + +def u16(s): + """Length as the website's validator counts it (JavaScript strings are UTF-16).""" + return len(s.encode('utf-16-le')) // 2 + + +def cut(s, n): + """s shortened to at most n UTF-16 units, never splitting a character.""" + while u16(s) > n: + s = s[:max(0, len(s) - max(1, (u16(s) - n) // 2))] + return s + + +def _log_tail(): + """The last lines of the server log the app writes (FRAME_CONTROL_LOG), newest first.""" + path = os.environ.get('FRAME_CONTROL_LOG') + if not path: + return [] + try: + with open(path, 'rb') as f: + f.seek(0, os.SEEK_END) + f.seek(max(0, f.tell() - 64 * 1024)) + lines = f.read().decode('utf-8', 'replace').splitlines() + except OSError: + return [] + # Request lines ("GET /api/status ...") are noise; keep what went wrong. + keep = [ln for ln in lines if ln.strip() and not ln.startswith(('GET ', 'POST '))] + return list(reversed(keep[-LOG_LINES:])) + + +def diagnostics(activity=(), include_logs=False, limit=DIAG_MAX): + """What a report includes, scrubbed and at most `limit` UTF-16 units. Always the versions + and builds; recent activity and the server log only when asked for, since they can name + files. Sections are filled in order of use, newest lines first, so trimming drops the oldest.""" + t = frame_telemetry.state() + levels = ', '.join(f"{name} {'on' if on else 'off'}" for name, on in + (('usage', t['usage']), ('compat', t['compat']), ('error details', t['diagnostics']))) + env = [ + f"Frame Control {frame_telemetry.app_version()}" + f"{' (built app)' if os.environ.get('FRAME_CONTROL_PACKAGED') else ' (source checkout)'}", + f"Computer: {frame_host.NAME} {platform.release()} {platform.machine()}, Python {'%d.%d.%d' % sys.version_info[:3]}", + f"SteamOS: {frame.get('build') or 'unknown'} ({frame.get('version') or 'not connected since start'})", + f"Analytics: {levels}", + f"Report time: {time.strftime('%Y-%m-%d %H:%M %Z')}", + ] + out = frame_telemetry.scrub('\n'.join(env), limit=limit) + if not include_logs: + return cut(out, limit) + sections = [('Recent activity (newest first):', [str(a)[:300] for a in list(activity)[:ACTIVITY_LINES] if isinstance(a, str)]), + ('Server log (newest first):', _log_tail())] + for title, lines in sections: + if not lines: + continue + block = '\n\n' + title + if u16(out + block) > limit: + break + out += block + for line in lines: + line = '\n' + frame_telemetry.scrub(line, 300) + if u16(out + line) > limit: + break + out += line + return out + + +def compose(body): + """(title, message) for the feedback API: the person's text, then the diagnostics exactly as + the dialog previewed them (passed back, scrubbed again and bounded here).""" + title = ' '.join(str(body.get('title') or '').split()) + text = str(body.get('message') or '').strip() + if len(title) < 5: + raise ValueError('give it a short title (at least 5 characters)') + if len(text) < 10: + raise ValueError('say a little more about what happened (at least 10 characters)') + title, text = cut(title, 120), cut(text, TEXT_MAX) + diag = body.get('diagnostics') + if not isinstance(diag, str) or not diag.strip(): + return title, text + diag = cut(frame_telemetry.scrub(diag, 20000), DIAG_MAX) + return title, f'{text}{HEAD}{diag}{TAIL}' + + +def send(body): + """File the report. Returns {"number", "url"} of the new issue; raises ReportError.""" + kind = body.get('kind') if body.get('kind') in KINDS else 'bug' + title, message = compose(body) + payload = {'kind': kind, 'title': title, 'message': message, 'website': '', + 'github': str(body.get('github') or '').strip().lstrip('@')[:40], + 'version': frame_telemetry.app_version(), 'os': f'{frame_host.NAME} {platform.machine()}', + 'steamos': str(frame.get('build') or '')[:120], + # How long the dialog was open; the API treats anything under 3 s as a script. + 'elapsed': max(0, int(body.get('elapsed') or 0))} + req = urllib.request.Request(FEEDBACK_URL, data=json.dumps(payload).encode(), method='POST', + headers={'content-type': 'application/json', + 'user-agent': f'FrameControl/{frame_telemetry.app_version()}'}) + try: + with urllib.request.urlopen(req, timeout=30) as r: + res = json.loads(r.read() or b'{}') + except urllib.error.HTTPError as e: + try: + why = json.loads(e.read() or b'{}').get('error') + except ValueError: + why = None + e.close() + raise ReportError(why or f'the feedback service said HTTP {e.code}') + except (urllib.error.URLError, OSError, ValueError) as e: + raise ReportError(f"couldn't reach the feedback service: {e}") + if not res.get('url'): + raise ReportError("the feedback service didn't file it; try again in a moment") + frame_telemetry.capture('problem_reported', {'kind': kind, 'with_diagnostics': bool(body.get('diagnostics'))}) + return {'number': res.get('number'), 'url': res['url'], 'message': f"Sent. It's issue #{res.get('number')} on GitHub."} + + +class ReportError(RuntimeError): + pass diff --git a/ui/frame_telemetry.py b/ui/frame_telemetry.py new file mode 100644 index 0000000..6dca188 --- /dev/null +++ b/ui/frame_telemetry.py @@ -0,0 +1,522 @@ +"""Anonymous analytics for Frame Control, sent to PostHog. Python stdlib only. + +Three levels, each chosen in the page's Privacy panel (docs/privacy.md lists +every event and property): + +- usage (on by default, after the first-run notice has been shown): installs of + Frame Control, daily opens, updates, which tabs are used, and whether installs + on the Frame worked, with an error category from a fixed list. Never file + names, paths, hostnames, IP addresses, window titles or account data. +- compat (opt-in): Android compatibility reports, the same fields the Report + dialog shows, so they reach the shared database (frame_compat_db.py). The + maintainer's sync (python3 ui/frame_compat_db.py sync) moves them there. +- diagnostics (opt-in): error messages and Python tracebacks, scrubbed of + home folders, user names, addresses and keys. + +The first-run notice offers compat and diagnostics together, and the page's +Report a problem dialog (frame_report.py) files bug reports whatever is chosen +here. + +Events are identified by a random id made on first run, not by the person or +computer, and sent without person profiles or GeoIP. Nothing is sent without a +project key (ui/telemetry.json or $FRAME_CONTROL_POSTHOG_KEY), from a source +checkout unless $FRAME_CONTROL_TELEMETRY=1, or when $DO_NOT_TRACK=1 or +$FRAME_CONTROL_TELEMETRY=0. + +Events wait in an outbox file and are sent in batches from a background thread, +so going offline loses nothing. The last SENT_KEEP sent events are kept on this +computer so the page can show exactly what left it. +""" +import ipaddress +import json +import os +import platform +import re +import sys +import threading +import time +import traceback +import urllib.error +import urllib.request +import uuid +from pathlib import Path +from urllib.parse import urlsplit + +import frame_host + +HERE = Path(__file__).resolve().parent +STATE = frame_host.data_dir('telemetry') +SETTINGS = STATE / 'settings.json' +OUTBOX = STATE / 'outbox.jsonl' +SENT = STATE / 'sent.jsonl' +SENT_KEEP = 200 +OUTBOX_MAX = 2000 # events kept while offline; the oldest go first +FLUSH_EVERY = 60 +REPEAT_WINDOW = 600 # the same diagnostic error is sent at most once in this many seconds +DEFAULT_HOST = 'https://eu.i.posthog.com' + +LEVELS = ('usage', 'compat', 'diagnostics') +# Events the page may send through /api/telemetry, and the properties each may carry. +PAGE_EVENTS = {'tab_viewed': {'tab'}, 'update_offered': {'to_version'}, + 'update_started': {'to_version'}, 'update_failed': {'to_version', 'error_category'}} +TABS = {'view', 'shots', 'library', 'getgames', 'android', 'transfer', 'apps', 'display', 'power', 'privacy'} + +_lock = threading.RLock() +_send_lock = threading.Lock() # held while sending; consent changes wait for it +_seen_errors = {} +_flusher = None +_wake = threading.Event() + + +# ---- configuration and settings ------------------------------------------------- + +def config(): + """PostHog host and project key: the environment, else ui/telemetry.json.""" + try: + with open(HERE / 'telemetry.json') as f: + c = json.load(f) + except (OSError, ValueError): + c = {} + host = os.environ.get('FRAME_CONTROL_POSTHOG_HOST') or c.get('host') or DEFAULT_HOST + key = os.environ.get('FRAME_CONTROL_POSTHOG_KEY') or c.get('key') or '' + project = os.environ.get('FRAME_CONTROL_POSTHOG_PROJECT') or c.get('project') or '' + return {'host': host.rstrip('/'), 'key': key, 'project': str(project)} + + +def blocked(): + """Why nothing may be sent at all, whatever the settings say, or None.""" + if os.environ.get('DO_NOT_TRACK') == '1' or os.environ.get('FRAME_CONTROL_TELEMETRY') == '0': + return 'turned off by DO_NOT_TRACK or FRAME_CONTROL_TELEMETRY=0' + if not config()['key']: + return 'no PostHog project key in this build' + if not os.environ.get('FRAME_CONTROL_PACKAGED') and os.environ.get('FRAME_CONTROL_TELEMETRY') != '1': + return 'running from a source checkout (set FRAME_CONTROL_TELEMETRY=1 to send)' + return None + + +def _defaults(): + return {'id': str(uuid.uuid4()), 'usage': True, 'compat': False, 'diagnostics': False, + 'notice_shown': False, 'installed_sent': False, 'last_version': None, 'last_open_day': None, + 'frames_seen': [], 'compat_sent': []} + + +def settings(): + with _lock: + s = _defaults() + try: + with open(SETTINGS) as f: + saved = json.load(f) + if isinstance(saved, dict): + s.update({k: v for k, v in saved.items() if k in s}) + except (OSError, ValueError): + pass + if not SETTINGS.exists(): + _save(s) # keep the id stable from the first call + return s + + +def _save(s): + try: + STATE.mkdir(parents=True, exist_ok=True) + tmp = SETTINGS.with_suffix('.tmp') + tmp.write_text(json.dumps(s, indent=1)) + os.replace(tmp, SETTINGS) + except OSError: + pass + + +def enabled(level): + """Whether events of this level are collected: never when sending is blocked, so a + source checkout or a test run leaves nothing behind.""" + if blocked(): + return False + return bool(settings().get(level)) + + +def update_settings(changes): + """Apply the page's choices. Turning a level off drops its unsent events; a send already + under way finishes first, so nothing leaves after this returns.""" + with _send_lock, _lock: + s = settings() + if 'noticeShown' in changes: + s['notice_shown'] = bool(changes['noticeShown']) or s['notice_shown'] + for level in LEVELS: + if level in changes: + s[level] = bool(changes[level]) + s['notice_shown'] = True + _save(s) + _drop_unwanted(s) + if changes.get('compat'): + backfill_compat() + _wake.set() + return state() + + +def state(): + """What the page shows: the choices, why sending is blocked, and what was sent.""" + s = settings() + return {'usage': s['usage'], 'compat': s['compat'], 'noticeShown': s['notice_shown'], + 'diagnostics': s['diagnostics'], + 'blocked': blocked(), 'id': s['id'], 'queued': len(_read_lines(OUTBOX)), + 'sent': list(reversed(_read_lines(SENT)))[:50]} + + +# ---- scrubbing and error categories --------------------------------------------- + +def _user_names(): + names = set() + for v in (os.environ.get('USER'), os.environ.get('USERNAME'), Path.home().name): + if v and len(v) > 2: + names.add(v) + return names + + +URL_RE = re.compile(r'[A-Za-z][A-Za-z0-9+.-]*://[^\s\'"<>]+') +SCRUBS = [ + (re.compile(r'ssh-(?:rsa|ed25519|dss)\s+\S+'), ''), + (re.compile(r'-----BEGIN [^-]+-----.*?-----END [^-]+-----', re.S), ''), + (re.compile(r'\b(?:phc|phx|ghp|gho|ghu|ghs|github_pat|sk|pk|rk|xox[abpr])[_-][A-Za-z0-9_-]{12,}'), ''), + (re.compile(r'(?i)\b(token|key|secret|password|passwd|pwd|auth|signature|sig)=[^\s&]+'), r'\1='), + (re.compile(r'[\w.+-]+@[\w-]+(?:\.[\w-]+)+'), ''), + (re.compile(r'\b(?:\d{1,3}\.){3}\d{1,3}\b'), ''), + (re.compile(r'\b(?:[0-9a-fA-F]{2}[:-]){5}[0-9a-fA-F]{2}\b'), ''), + (re.compile(r'\b7656119\d{10}\b'), ''), + (re.compile(r'\b(?:[\w-]+\.)+(?:local|lan|home|internal|localdomain|ts\.net)\b'), ''), + (re.compile(r'\b[0-9a-fA-F]{32,}\b'), ''), +] +IPV6_RE = re.compile(r'(?' + except ValueError: + return m.group(0) + + +def public_host(host): + """A host name that's safe to send: not an address, not a private or single-label name.""" + host = (host or '').lower().rstrip('.') + if not host or '.' not in host: + return None + try: + ipaddress.ip_address(host.strip('[]')) + return None + except ValueError: + pass + if re.search(r'\.(?:local|lan|home|internal|localdomain|ts\.net|arpa)$', host) or not re.fullmatch(r'[a-z0-9.-]+', host): + return None + return host + + +def _scrub_url(u): + """Only the scheme and a public host name of a URL; never user names, passwords, ports, + paths or queries.""" + try: + parts = urlsplit(u) + host = public_host(parts.hostname) + except ValueError: + host = None + return f'{parts.scheme}://{host}/…' if host else '' + + +def scrub(text, limit=2000): + """Text with URLs, home folders, user names, addresses, hosts, ids and keys replaced.""" + if text is None: + return None + t = URL_RE.sub(lambda m: _scrub_url(m.group(0)), str(text)) # first, before anything splits a URL + home = str(Path.home()) + if len(home) > 3: + t = t.replace(home, '~') + t = re.sub(r'(/Users/|/home/|[A-Za-z]:\\Users\\)[^/\\\s]+', r'\1', t) + for pattern, repl in SCRUBS: + t = pattern.sub(repl, t) + t = IPV6_RE.sub(_ipv6, t) + for name in _user_names(): + t = re.sub(r'\b%s\b' % re.escape(name), '', t) + return t[:limit] + + +# From the most to the least specific; the first match wins. +CATEGORIES = [ + ('android_installer', re.compile(r'INSTALL_(?:FAILED|PARSE_FAILED)_[A-Z_]+')), + ('apk_needs_newer_android', re.compile(r'needs Android API')), + ('apk_wrong_abi', re.compile(r'no arm64-v8a build')), + ('apk_unreadable', re.compile(r'(?i)not a zip|bad apk|AndroidManifest|ApkError|unexpected package name')), + ('cant_run_on_frame', re.compile(r"can't run on the Frame")), + ('steam_shortcut', re.compile(r'(?i)steam did not return a shortcut|shortcut list|no Steam shortcut')), + ('frame_not_set_up', re.compile(r'(?i)Could not resolve hostname|no "?frame"? (?:SSH )?alias')), + ('frame_auth', re.compile(r'(?i)Permission denied|Host key verification failed')), + ('frame_unreachable', re.compile(r'(?i)timed out|Connection (?:refused|reset|closed)|No route to host|' + r'Network is unreachable|Operation timed out|asleep|kex_exchange')), + ('frame_disk_full', re.compile(r'(?i)No space left|disk full|ENOSPC')), + ('download_failed', re.compile(r'(?i)HTTP (?:Error )?\d{3}|URLError|download|certificate verify failed')), + ('flatpak', re.compile(r'(?i)flatpak|flathub')), + ('cancelled', re.compile(r'(?i)cancel')), + ('lepton', re.compile(r'(?i)lepton|podman|instance')), +] + + +def categorize(message): + """(category, detail): a fixed category name, plus an Android installer code when there is one.""" + text = str(message or '') + for name, pattern in CATEGORIES: + m = pattern.search(text) + if m: + return name, (m.group(0) if name == 'android_installer' else None) + return 'other', None + + +# ---- capturing ------------------------------------------------------------------ + +def _common(): + return {'app_version': app_version(), 'os': frame_host.NAME, 'arch': platform.machine().lower(), + 'python': '%d.%d' % sys.version_info[:2], '$lib': 'frame-control', + # Anonymous events: no person profile, no location lookup. + '$process_person_profile': False, '$geoip_disable': True} + + +def app_version(): + v = os.environ.get('FRAME_CONTROL_VERSION') + if v: + return v + try: + with open(HERE.parent / 'app' / 'package.json') as f: + return json.load(f).get('version') or 'dev' + except (OSError, ValueError): + return 'dev' + + +def capture(event, props=None, level='usage'): + """Queue an event if its level is on. Never raises.""" + try: + if level not in LEVELS or not enabled(level): + return False + s = settings() + e = {'event': event, 'distinct_id': s['id'], 'uuid': str(uuid.uuid4()), + 'timestamp': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), + 'properties': {**_common(), **(props or {}), 'level': level}} + with _lock: + lines = _read_lines(OUTBOX) + [e] + _write_lines(OUTBOX, lines[-OUTBOX_MAX:]) + return True + except Exception: + return False + + +def page_event(body): + """An event from the page, checked against PAGE_EVENTS.""" + name = body.get('event') + allowed = PAGE_EVENTS.get(name) + if allowed is None: + raise ValueError('unknown event') + props = {k: str(v)[:40] for k, v in (body.get('properties') or {}).items() if k in allowed} + if name == 'tab_viewed' and props.get('tab') not in TABS: + raise ValueError('unknown tab') + return {'queued': capture(name, props)} + + +def app_started(): + """Once per server start: first install, an update, and one open a day.""" + if blocked(): + return + with _lock: + s = settings() + version, today = app_version(), time.strftime('%Y-%m-%d') + if not s['installed_sent']: + capture('app_installed') + s['installed_sent'] = True + elif s['last_version'] and s['last_version'] != version: + capture('app_updated', {'from_version': s['last_version']}) + if s['last_open_day'] != today: + capture('app_opened') + s['last_open_day'] = today + s['last_version'] = version + _save(s) + + +def frame_seen(build, version): + """The Frame's SteamOS build, once per build (public build numbers).""" + key = f'{build}/{version}' + with _lock: + s = settings() + if not build or key in s['frames_seen']: + return + s['frames_seen'] = (s['frames_seen'] + [key])[-20:] + _save(s) + capture('frame_connected', {'steamos_build': str(build)[:40], 'steamos_version': str(version or '')[:40]}) + + +def install_finished(kind, ok, seconds=None, error=None, **props): + """kind: apk, flatpak, steam, title or web. props must already be public (no file names).""" + p = {'kind': kind, 'ok': bool(ok), **{k: v for k, v in props.items() if v is not None}} + if seconds is not None: + p['seconds'] = round(seconds, 1) + if error is not None: + p['error_category'], code = categorize(error) + if code: + p['installer_code'] = code + capture('install_finished', p) + if error is not None and not ok: + diagnostic(f'{kind} install failed', error) + + +def diagnostic(where, error, tb=None): + """An error for the opt-in diagnostics level: scrubbed text, and a traceback if there is one.""" + if not enabled('diagnostics'): + return + message = scrub(error) + fingerprint = f'{where}|{message[:120]}' + now = time.time() + with _lock: + if now - _seen_errors.get(fingerprint, 0) < REPEAT_WINDOW: + return + _seen_errors[fingerprint] = now + exc_type = type(error).__name__ if isinstance(error, BaseException) else 'Error' + frames = [] + if tb is None and isinstance(error, BaseException): + tb = error.__traceback__ + for fs in traceback.extract_tb(tb) if tb else []: + frames.append({'filename': os.path.basename(fs.filename), 'lineno': fs.lineno, 'function': fs.name, + 'in_app': True, 'platform': 'python'}) + capture('$exception', {'$exception_list': [{'type': exc_type, 'value': message, + 'mechanism': {'handled': True, 'type': 'generic'}, + 'stacktrace': {'type': 'raw', 'frames': frames[-30:]}}], + '$exception_type': exc_type, '$exception_message': message, + 'where': scrub(where, 200), 'error_category': categorize(error)[0]}, + level='diagnostics') + + +COMPAT_FIELDS = ('package', 'version', 'result', 'rating', 'notes', 'via', 'date', 'steamos', 'lepton', + 'runtime', 'label', 'source', 'id') + + +def compat_report(report): + """A compatibility report for the shared database (compat level only). Free text is + scrubbed; the source is kept only as F-Droid or a public download host.""" + if not report.get('id') or not enabled('compat'): + return False + p = {k: report.get(k) for k in COMPAT_FIELDS if report.get(k) not in (None, '')} + for k, n in (('notes', 1000), ('label', 120), ('version', 80)): + if k in p: + p[k] = scrub(p[k], n) + src = str(p.pop('source', '') or '') + if src == 'F-Droid': + p['source'] = src + elif src.startswith(('http://', 'https://')) and _scrub_url(src) != '': + p['source'] = _scrub_url(src) + return capture('compat_report', p, level='compat') + + +def backfill_compat(): + """On opting in, share the reports this computer kept before (not ones already sent or queued).""" + try: + import frame_compat_db + if frame_compat_db.shared(): + return 0 # the maintainer's copy writes to the database directly + done = set(settings()['compat_sent']) + done |= {e['properties'].get('id') for e in _read_lines(OUTBOX) if e.get('event') == 'compat_report'} + n = 0 + for r in frame_compat_db._outbox(): + if r.get('id') not in done and compat_report(r): + n += 1 + return n + except Exception: + return 0 + + +# ---- the outbox ----------------------------------------------------------------- + +def _read_lines(path): + try: + with open(path) as f: + out = [] + for line in f: + try: + out.append(json.loads(line)) + except ValueError: + pass + return out + except OSError: + return [] + + +def _write_lines(path, rows): + STATE.mkdir(parents=True, exist_ok=True) + tmp = Path(str(path) + '.tmp') + with open(tmp, 'w') as f: + f.writelines(json.dumps(r, ensure_ascii=False) + '\n' for r in rows) + os.replace(tmp, path) + + +def _drop_unwanted(s): + """Unsent events whose level is now off never leave the computer.""" + keep = {level: s[level] for level in LEVELS} + rows = _read_lines(OUTBOX) + kept = [e for e in rows if keep.get(e.get('properties', {}).get('level'), False)] + if len(kept) != len(rows): + _write_lines(OUTBOX, kept) + + +def flush(timeout=20): + """Send what's queued. Returns how many were sent; on failure they stay queued.""" + with _send_lock: + if blocked() or not settings()['notice_shown']: + return 0 + cfg = config() + with _lock: + _drop_unwanted(settings()) + batch = _read_lines(OUTBOX)[:100] + if not batch: + return 0 + body = json.dumps({'api_key': cfg['key'], 'batch': batch}).encode() + req = urllib.request.Request(cfg['host'] + '/batch/', data=body, method='POST', + headers={'content-type': 'application/json', + 'user-agent': f'FrameControl/{app_version()}'}) + try: + with urllib.request.urlopen(req, timeout=timeout) as r: + r.read() + except urllib.error.HTTPError as e: + e.close() + return 0 + except (urllib.error.URLError, OSError, ValueError): + return 0 + sent_ids = {e['uuid'] for e in batch} + with _lock: + _write_lines(OUTBOX, [e for e in _read_lines(OUTBOX) if e.get('uuid') not in sent_ids]) + _write_lines(SENT, (_read_lines(SENT) + batch)[-SENT_KEEP:]) + compat = [e['properties'].get('id') for e in batch if e.get('event') == 'compat_report'] + if compat: # remembered only once PostHog has them, so an opt-out before sending can't lose them + s = settings() + s['compat_sent'] = (s['compat_sent'] + compat)[-5000:] + _save(s) + return len(batch) + + +def start(): + """Record this start and send in the background from now on.""" + global _flusher + try: + app_started() + except Exception: + pass + if _flusher: + return + + def loop(): + while True: + try: + while flush() == 100: # a full batch: there may be more + pass + except Exception: + pass + _wake.wait(FLUSH_EVERY) + _wake.clear() + + _flusher = threading.Thread(target=loop, name='telemetry', daemon=True) + _flusher.start() + + +def wake(): + _wake.set() diff --git a/ui/index.html b/ui/index.html index 475ea72..43d58f9 100644 --- a/ui/index.html +++ b/ui/index.html @@ -44,7 +44,12 @@ .brand { display: flex; align-items: center; gap: 11px; color: var(--bright); text-decoration: none; } .brand svg { width: 30px; height: 30px; } .brand b { font-weight: 700; font-size: 15px; letter-spacing: 2.5px; } - nav { display: flex; gap: 4px; height: 100%; } + /* The tabs shrink and scroll sideways rather than push the header wider than the window. */ + nav { display: flex; gap: 4px; height: 100%; min-width: 0; flex-shrink: 1; overflow-x: auto; scrollbar-width: none; + mask-image: linear-gradient(to right, #000 calc(100% - 28px), transparent); } + nav::-webkit-scrollbar { display: none; } + nav a { flex: none; } + @media (max-width: 1600px) { header { gap: 18px; } nav a { padding: 0 9px !important; } } nav a { display: flex; align-items: center; padding: 0 13px; height: 100%; color: var(--text); text-decoration: none; font-weight: 500; font-size: 15px; letter-spacing: 1.2px; text-transform: uppercase; border-bottom: 3px solid transparent; transition: color .15s; } @@ -195,6 +200,17 @@ .actions { display: grid; grid-template-columns: repeat(2, 1fr); gap: 8px; } .actions button { justify-content: flex-start; height: 40px; } .actions svg { width: 16px; height: 16px; flex: none; opacity: .85; } + .notice { display: flex; gap: 14px; align-items: center; flex-wrap: wrap; padding: 12px 16px; border-radius: 4px; + background: rgba(26,159,255,.12); border-left: 3px solid var(--blue); font-size: 13.5px; line-height: 1.5; } + .notice .grow { flex: 1; min-width: 260px; } + .notice .progress { width: 160px; margin-top: 0; display: block; } + .popt { display: grid; grid-template-columns: auto 1fr; gap: 4px 10px; align-items: start; margin: 0 0 14px; cursor: pointer; } + .popt input { margin: 3px 0 0; width: 16px; height: 16px; accent-color: var(--blue); } + .popt b { font-weight: 600; color: var(--text); } + .popt .sub { grid-column: 2; line-height: 1.45; } + .sentlog { max-height: 260px; overflow: auto; background: rgba(0,0,0,.3); border-radius: 3px; padding: 10px; + font: 11.5px ui-monospace, SFMono-Regular, Menlo, monospace; white-space: pre-wrap; word-break: break-all; margin: 8px 0 0; } + details summary { cursor: pointer; color: var(--link); font-size: 13px; margin-top: 12px; } .links { margin-top: 16px; padding-top: 14px; border-top: 1px solid rgba(255,255,255,.06); font-size: 13px; color: var(--muted); } .links a { color: var(--link); text-decoration: none; } .links a:hover { color: #fff; } .links div { margin: 5px 0; } @@ -221,10 +237,18 @@ .and-grid { display: grid; grid-template-columns: minmax(0, 1fr) minmax(0, 2fr); gap: 22px; align-items: start; } .and-col { display: grid; gap: 22px; align-content: start; } .rep-item .s { white-space: normal; } - #repDlg, #titleDlg, #wiDlg { background: #1e2329; color: var(--text); border: 1px solid rgba(255,255,255,.1); border-radius: 4px; + #bugDlg, #repDlg, #titleDlg, #wiDlg { background: #1e2329; color: var(--text); border: 1px solid rgba(255,255,255,.1); border-radius: 4px; padding: 22px; width: min(560px, 92vw); box-shadow: 0 20px 60px rgba(0,0,0,.6); } - #repDlg::backdrop, #titleDlg::backdrop, #wiDlg::backdrop { background: rgba(0,0,0,.55); } - #repDlg h2, #titleDlg h2, #wiDlg h2 { margin: 0 0 14px; font-size: 15px; letter-spacing: 1.5px; text-transform: uppercase; color: var(--bright); } + #bugDlg::backdrop, #repDlg::backdrop, #titleDlg::backdrop, #wiDlg::backdrop { background: rgba(0,0,0,.55); } + #bugDlg { width: min(640px, calc(100vw - 40px)); } + #bugForm label.field { display: block; font-size: 12.5px; color: var(--muted); margin-top: 10px; } + #bugForm label.field input, #bugForm label.field textarea, #bugForm select { margin-top: 5px; } + #bugForm select { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; + border-radius: 3px; padding: 8px 10px; font: inherit; } + #bugForm .popt { margin: 14px 0 0; } + #bugForm .sentlog { max-height: 200px; } + #bugWarn { color: var(--muted); font-size: 12.5px; line-height: 1.45; margin: 12px 0 0; } + #bugDlg h2, #repDlg h2, #titleDlg h2, #wiDlg h2 { margin: 0 0 14px; font-size: 15px; letter-spacing: 1.5px; text-transform: uppercase; color: var(--bright); } #repForm label, #titleForm label { display: block; font-size: 12.5px; color: var(--muted); margin-top: 10px; } #repForm label input[type=text], #repForm textarea, #titleForm label input, #titleForm label select { margin-top: 5px; } #titleForm select { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; @@ -309,16 +333,36 @@ Apps Display Power + Privacy
Connecting… — +
+ +
@@ -543,6 +587,24 @@
FrameDrop: sideloader (Windows only for now)
+ +
+

Privacy & updates

+ + + +
+
Show what's been sent
+
+ + What's collected, exactly
+
@@ -581,6 +643,30 @@ + +
+

Report a problem

+ + + + + + +
Show exactly what's included
Loading…
+

This becomes a public issue on GitHub (saphid/frame-control). No GitHub account is needed.

+
+ + + +
+
+

Add to the Steam library

@@ -1162,7 +1248,8 @@ async function sendFiles(files, dirs = new Set()) { if (!f.size) { toast(`${f.name}: empty files aren't supported`, true); continue; } if (TITLE_EXT.test(f.name)) { await sideload(f, path); continue; } const apk = f.name.toLowerCase().endsWith(".apk"); - await act(apk ? `Install ${f.name}` : `Copy ${f.name} to ~/Downloads`, () => upload(f, apk ? "apk" : "push")); + const res = await act(apk ? `Install ${f.name}` : `Copy ${f.name} to ~/Downloads`, () => upload(f, apk ? "apk" : "push")); + if (apk && res && res.app) await offerTest(res.app); } $("fileInput").value = ""; refresh(); @@ -1544,7 +1631,8 @@ document.body.addEventListener("click", async e => { const pkg = b.dataset.pkg, action = b.dataset.and; if (action === "install") { toast(`Installing ${b.dataset.name}. The first time takes a minute…`); - await act(`Install ${b.dataset.name}`, () => api("/api/android", { action, package: pkg }), b); + const res = await act(`Install ${b.dataset.name}`, () => api("/api/android", { action, package: pkg }), b); + if (res && res.app) await offerTest(res.app); } else if (action === "remove") { if (!confirm(`Remove ${b.dataset.name} and its data from the Frame?`)) return; await act(`Remove ${b.dataset.name}`, () => api("/api/android", { action, package: pkg }), b); @@ -1661,13 +1749,19 @@ loadDisplays(); const RLABEL = { works: "Works", issues: "Problems", broken: "Doesn't work", runs: "Runs (test)", crashes: "Crashed (test)", install_failed: "Won't install", instance_failed: "Didn't start (test)" }; const RCLASS = { works: "works", runs: "works", issues: "maybe" }; +let repShared = false; +function setRepHint() { + $("repHint").textContent = repShared + ? "Reports go to Frame Control's shared compatibility database and change the verdicts in the catalogue. Any APK can be reported, including ones not on F-Droid." + : telemetry.compat && !telemetry.blocked + ? "Reports change the verdicts you see and are shared with Frame Control's compatibility database (Privacy settings). Any APK can be reported, including ones not on F-Droid." + : "Reports are saved on this computer and change the verdicts you see. Turn on Share compatibility results in Privacy settings to add them to the shared database. Any APK can be reported, including ones not on F-Droid."; +} async function loadReports() { let reps, shared; try { ({ reports: reps, shared } = await api("/api/android/reports")); } catch (e) { $("repList").innerHTML = `
${esc(e.message)}
`; return; } - $("repHint").textContent = shared - ? "Reports go to Frame Control's shared compatibility database and change the verdicts in the catalogue. Any APK can be reported, including ones not on F-Droid." - : "Reports are saved on this computer and change the verdicts you see. They aren't uploaded: the shared database is maintainer-only for now. Any APK can be reported, including ones not on F-Droid."; + repShared = shared; setRepHint(); $("repCount").textContent = reps.length ? `${reps.length} newest` : ""; $("repList").innerHTML = reps.length ? reps.slice(0, 40).map(r => { const k = r.rating || r.result; @@ -1820,7 +1914,163 @@ const spy = new IntersectionObserver(entries => { const top = entries.filter(e => e.isIntersecting).sort((a, b) => a.boundingClientRect.top - b.boundingClientRect.top)[0]; if (top) document.querySelectorAll("nav a").forEach(a => a.classList.toggle("on", a.getAttribute("href") === "#" + top.target.id)); }, { rootMargin: "-80px 0px -55% 0px" }); -["view", "shots", "library", "getgames", "android", "transfer", "apps", "display", "power"].forEach(id => spy.observe($(id))); +["view", "shots", "library", "getgames", "android", "transfer", "apps", "display", "power", "privacy"].forEach(id => spy.observe($(id))); + +// ---- after an APK install: test it, so the compatibility database learns whether it runs ---- +async function offerTest(m) { + if (!m.package || !confirm(`${m.label || m.package} is installed. Test it now?\n\nIt opens in the headset for about 20 seconds ` + + "and records whether it stays up.")) return; + toast(`Testing ${m.label || m.package}: launching it and watching for 20 s…`); + await act(`Test ${m.label || m.package}`, () => api("/api/android", { action: "probe", package: m.package })); + loadReports(); +} + +// ---- privacy: anonymous analytics levels (ui/frame_telemetry.py, docs/privacy.md) ---- +const telemetry = { usage: false, compat: false, blocked: "not loaded" }; +function renderTelemetry(s) { + Object.assign(telemetry, s); + setRepHint(); + $("tUsage").checked = s.usage; $("tCompat").checked = s.compat; $("tDiag").checked = s.diagnostics; + $("tStatus").textContent = s.blocked ? `Nothing is being sent: ${s.blocked}.` + : `${s.queued ? s.queued + " waiting to send. " : ""}Your anonymous id is ${s.id.slice(0, 8)}…; it isn't linked to you or this computer.`; + $("tSent").textContent = s.sent.length ? s.sent.map(e => JSON.stringify({ event: e.event, time: e.timestamp, ...e.properties })).join("\n\n") + : "Nothing sent yet."; + const showNotice = !s.blocked && !s.noticeShown && s.usage; + $("privacyNotice").hidden = !showNotice; + if (showNotice) api("/api/telemetry", { noticeShown: true }).catch(() => {}); +} +async function loadTelemetry() { + try { renderTelemetry(await api("/api/telemetry")); } catch {} +} +async function setTelemetry(change) { + try { renderTelemetry(await api("/api/telemetry", change)); } + catch (e) { toast(`Couldn't save: ${e.message}`, true); loadTelemetry(); } +} +$("tUsage").onchange = e => setTelemetry({ usage: e.target.checked }); +$("tCompat").onchange = e => setTelemetry({ compat: e.target.checked }); +$("tDiag").onchange = e => setTelemetry({ diagnostics: e.target.checked }); +$("tSentBox").ontoggle = () => { if ($("tSentBox").open) loadTelemetry(); }; +$("noticeOk").onclick = () => { $("privacyNotice").hidden = true; }; +$("noticeMore").onclick = async () => { + $("privacyNotice").hidden = true; + await setTelemetry({ compat: true, diagnostics: true }); + toast("Thanks! Compatibility results and error details will be shared too. Change it any time in Privacy."); +}; +$("noticeSettings").onclick = () => { $("privacyNotice").hidden = true; location.hash = "#privacy"; $("privacy").scrollIntoView(); }; +loadTelemetry(); +function pageEvent(event, properties) { + if (telemetry.usage && !telemetry.blocked) api("/api/telemetry/event", { event, properties }).catch(() => {}); +} +// Which tabs get used: once per tab per session. +const tabsSeen = new Set(); +document.querySelectorAll("nav a").forEach(a => a.addEventListener("click", () => { + const tab = a.getAttribute("href").slice(1); + if (!tabsSeen.has(tab)) { tabsSeen.add(tab); pageEvent("tab_viewed", { tab }); } +})); + +// ---- report a problem (ui/frame_report.py): a GitHub issue via the website, with diagnostics ---- +const bug = { opened: 0, preview: "" }; +const activityLines = () => [...$("log").children].slice(0, 25).map(el => el.textContent.trim()); +function bugReportText() { + const body = `${$("bugText").value.trim()}${bug.preview ? "\n\n---\nDiagnostics:\n```\n" + bug.preview + "\n```" : ""}`; + return { title: $("bugTitleIn").value.trim(), body }; +} +function bugGithubUrl() { + const { title, body } = bugReportText(); + return "https://github.com/saphid/frame-control/issues/new?labels=feedback&title=" + encodeURIComponent(title) + + "&body=" + encodeURIComponent(body.slice(0, 6000)); +} +// The preview is a snapshot: exactly this text is sent, even if more activity happens meanwhile. +async function loadBugPreview() { + if (!$("bugDiag").checked) { bug.preview = ""; $("bugDiagText").textContent = "Nothing: diagnostics are off."; return; } + $("bugDiagText").textContent = "Loading…"; + try { bug.preview = (await api("/api/report/preview", { activity: activityLines(), includeLogs: $("bugLogs").checked })).text; } + catch (e) { bug.preview = ""; $("bugDiagText").textContent = `Couldn't collect diagnostics: ${e.message}`; return; } + $("bugDiagText").textContent = bug.preview; +} +function openBugReport() { + $("bugForm").reset(); + $("bugMsg").textContent = ""; $("bugGithubLink").hidden = true; $("bugSend").disabled = false; + $("bugCancel").textContent = "Cancel"; + $("bugDiagBox").open = false; $("bugLogs").disabled = false; + bug.opened = performance.now(); + $("bugDlg").showModal(); + loadBugPreview(); +} +document.body.addEventListener("click", e => { if (e.target.closest("[data-report]")) openBugReport(); }); +$("bugDiag").onchange = () => { $("bugLogs").disabled = !$("bugDiag").checked; loadBugPreview(); }; +$("bugLogs").onchange = loadBugPreview; +$("bugCancel").onclick = () => $("bugDlg").close(); +$("bugCopy").onclick = async () => { + const { title, body } = bugReportText(); + try { await navigator.clipboard.writeText(`${title}\n\n${body}`); $("bugMsg").textContent = "Copied."; } + catch { $("bugMsg").textContent = "Couldn't copy; select the text under Show exactly what's included."; } +}; +$("bugForm").onsubmit = async e => { + e.preventDefault(); + if (!$("bugForm").reportValidity()) return; + $("bugSend").disabled = true; $("bugMsg").textContent = "Sending…"; + try { + const res = await api("/api/report", { + kind: $("bugKind").value, title: $("bugTitleIn").value, message: $("bugText").value, + github: $("bugGithub").value, diagnostics: $("bugDiag").checked ? bug.preview : "", + elapsed: Math.round(performance.now() - bug.opened) }); + $("bugMsg").innerHTML = `Sent. Issue #${esc(String(res.number))} on GitHub.`; + $("bugCancel").textContent = "Close"; + log(res.message, "ok"); + } catch (err) { + $("bugMsg").textContent = `Couldn't send it: ${err.message}.`; + $("bugGithubLink").href = bugGithubUrl(); $("bugGithubLink").hidden = false; + $("bugSend").disabled = false; + } +}; +if (window.frameApp && window.frameApp.onReportProblem) window.frameApp.onReportProblem(openBugReport); + +// ---- updates (the desktop app only: app/updater.js) ---- +const upd = { dismissed: false, offered: null }; +function renderUpdate(s) { + if (!s) return; + $("appVersion").textContent = `Version ${s.current}`; + const r = s.latest; + const show = r && ["available", "downloading", "ready", "error"].includes(s.status) && !upd.dismissed + && !(s.status === "error" && !r); + $("updateBar").hidden = !show; + if (!show) return; + if (s.status === "available" && upd.offered !== r.version) { upd.offered = r.version; pageEvent("update_offered", { to_version: r.version }); } + const busy = s.status === "downloading" || s.status === "ready"; + $("updateText").innerHTML = s.status === "error" + ? `Updating to ${esc(r.version)} didn't work: ${esc(s.error || "unknown error")}` + : busy ? `Downloading Frame Control ${esc(r.version)}… It restarts when it's ready.` + : `Frame Control ${esc(r.version)} is available. You have ${esc(s.current)}.` + + (s.canInstall ? "" : ` ${esc(s.why ? "It can't update itself here (" + s.why + ")," : "")} download it from the release page.`); + $("updateProg").hidden = !busy; + $("updateProg").firstElementChild.style.width = Math.round((s.progress || 0) * 100) + "%"; + $("updateGo").textContent = s.canInstall ? (s.status === "error" ? "Try again" : "Update and restart") : "Open release page"; + $("updateGo").disabled = busy; $("updateLater").hidden = busy; +} +if (window.frameApp && window.frameApp.update) { + window.frameApp.update.onState(renderUpdate); + window.frameApp.update.get().then(renderUpdate); + $("updateCheck").hidden = false; + $("updateCheck").onclick = async () => { + const btn = $("updateCheck"); + btn.disabled = true; + let s; + try { s = await window.frameApp.update.check(); } finally { btn.disabled = false; } + upd.dismissed = false; renderUpdate(s); + if (s && s.status === "none") toast(`You have the newest version (${s.current})`); + if (s && s.status === "check-failed") toast(`Couldn't check for updates: ${s.error}`, true); + }; + $("updateGo").onclick = () => { + pageEvent("update_started", { to_version: upd.offered || "" }); + window.frameApp.update.install(); + }; + $("updateLater").onclick = () => { upd.dismissed = true; $("updateBar").hidden = true; }; + $("updateNotes").onclick = async () => { + const s = await window.frameApp.update.get(); + if (s && s.latest) window.open(s.latest.page, "_blank"); + }; +} // ---- install links from websites (frame-control://install, docs/web-install.md) ---- // The app passes each link here. The server checks it and reads the manifest; diff --git a/ui/server.py b/ui/server.py index 05661e3..58e8d57 100755 --- a/ui/server.py +++ b/ui/server.py @@ -36,7 +36,9 @@ sys.path.insert(0, str(Path(__file__).resolve().parent)) import frame_android # noqa: E402 import frame_catalog # noqa: E402 import frame_host # noqa: E402 +import frame_report # noqa: E402 import frame_store # noqa: E402 +import frame_telemetry # noqa: E402 import frame_titles # noqa: E402 import frame_webinstall # noqa: E402 @@ -153,7 +155,12 @@ def terminal(argv): # ---- actions --------------------------------------------------------------- def status(_body): - return json.loads(ssh("python3 -", stdin=(HERE / "frame_status.py").read_text(), timeout=20)) + s = json.loads(ssh("python3 -", stdin=(HERE / "frame_status.py").read_text(), timeout=20)) + osr = s.get("os") if isinstance(s, dict) else None + if isinstance(osr, dict): + frame_telemetry.frame_seen(osr.get("build"), osr.get("version")) + frame_report.frame.update(build=osr.get("build"), version=osr.get("version")) + return s def headset_view(): @@ -335,7 +342,16 @@ def steam(body): raise Failure("bad appid", 400) if action not in ("install", "store"): raise Failure("action must be install or store", 400) - return steam_frame(action, appid) + if action == "store": + return steam_frame(action, appid) + # Starts Steam's download; Steam reports the rest in the headset. + try: + res = steam_frame(action, appid) + except Failure as e: + frame_telemetry.install_finished("steam", False, error=e, steam_appid=appid) + raise + frame_telemetry.install_finished("steam", True, steam_appid=appid) + return res def steam_search(query): @@ -408,10 +424,16 @@ def flatpak(body): if not FLATPAK_ID.match(app): raise Failure("bad Flatpak app ID", 400) if action == "install": - # Per-user, so it survives SteamOS updates and needs no sudo (as install-apps.sh). - ssh("flatpak remote-add --user --if-not-exists flathub " - "https://dl.flathub.org/repo/flathub.flatpakrepo && " - f"flatpak install --user -y --noninteractive flathub {shlex.quote(app)}", timeout=900) + start = time.time() + try: + # Per-user, so it survives SteamOS updates and needs no sudo (as install-apps.sh). + ssh("flatpak remote-add --user --if-not-exists flathub " + "https://dl.flathub.org/repo/flathub.flatpakrepo && " + f"flatpak install --user -y --noninteractive flathub {shlex.quote(app)}", timeout=900) + except Failure as e: + frame_telemetry.install_finished("flatpak", False, time.time() - start, e, flatpak_id=app) + raise + frame_telemetry.install_finished("flatpak", True, time.time() - start, flatpak_id=app) return {"message": f"Installed {app}"} if action == "uninstall": out = ssh(f"flatpak uninstall --user -y -- {shlex.quote(app)}", timeout=300) @@ -468,13 +490,37 @@ def android(body): runtime=body.get("runtime") or "instance", label=body.get("label"), source=body.get("source")) name = r.get("label") or pkg - where = "" if frame_catalog.compat_db.shared() else " on this computer" + where = ("" if frame_catalog.compat_db.shared() else + " and shared it" if frame_telemetry.enabled("compat") else " on this computer") return {"message": f"Saved your report for {name}{where}", "report": r} except frame_android.FrameError as e: raise Failure(str(e)) raise Failure("unknown action", 400) +# Errors that are the APK's own fault, so they belong in the compatibility +# database as install_failed. Connection trouble and the like don't. +APK_FAULTS = {"android_installer", "apk_needs_newer_android", "apk_wrong_abi"} + + +def apk_installed(info, meta, error, seconds): + """Every APK install (catalogue, dropped file, web link): usage analytics, and an + install_failed report when the APK itself wouldn't install.""" + pkg = (info or {}).get("package") + by_pkg = frame_catalog._cache.get("by_pkg") or {} + in_catalog = bool(pkg) and pkg in by_pkg + # Package names only for catalogue apps, which are public; a private APK's name stays here. + # No version: a local rebuild can share a catalogue app's package name but carry anything in its version. + frame_telemetry.install_finished("apk", error is None, seconds, error, catalog=in_catalog, + package=pkg if in_catalog else None) + if error is not None and pkg and frame_telemetry.categorize(error)[0] in APK_FAULTS: + frame_catalog.add_report(pkg, info.get("version"), result="install_failed", notes=str(error)[:300], + via="install", label=info.get("label")) + + +frame_android.install_hooks.append(apk_installed) + + # ---- Sideloaded titles (Linux/Windows builds as Steam Devkit Games) -------- # # Installing is two steps: inspect (a dropped file is uploaded and a zip @@ -528,14 +574,18 @@ def _run_title_install(token, entry, name, exe, runtime): with _titles_lock: _title_jobs[token].update(fields) + start = time.time() try: m = frame_titles.install_plan(entry["plan"], name=name, exe=exe, runtime=runtime, progress=lambda stage, fraction: update(stage=stage, fraction=fraction)) update(title=m, message=f"Installed {m['id']} in the Steam library ({m['runtime_label']})") + frame_telemetry.install_finished("title", True, time.time() - start, runtime=m.get("runtime")) except frame_android.FrameError as e: update(error=str(e)) + frame_telemetry.install_finished("title", False, time.time() - start, e) except Exception as e: update(error=f"{type(e).__name__}: {e}") + frame_telemetry.install_finished("title", False, time.time() - start, e) finally: _drop_staged(entry) update(done=True, time=time.time()) @@ -964,10 +1014,16 @@ def _webinstall_run(plan, job): ensure_master() res = frame_webinstall.dispatch(path, name=plan["name"], exe=plan["exe"], progress=detail, source=plan["url"]) job["message"], job["phase"] = res["message"], "done" + if res.get("kind") != "apk": # APKs are counted by apk_installed + frame_telemetry.install_finished("web", True, kind_detail=res.get("kind")) except Exception as e: + stage = job.get("phase") # download or install, before it becomes "error" known = (frame_webinstall.WebInstallError, Failure, frame_android.FrameError) job["error"] = str(e) if isinstance(e, known) else f"{type(e).__name__}: {e}" job["phase"] = "error" + # An APK that failed to install was counted by apk_installed. + if not isinstance(e, frame_webinstall.Cancelled) and not (stage == "install" and plan.get("kind") == "apk"): + frame_telemetry.install_finished("web", False, error=e, stage=stage, kind_detail=plan.get("kind")) finally: with _web_lock: job.pop("_conn", None) @@ -1066,14 +1122,36 @@ def _sweep_one(prefix, d): pass +# ---- Report a problem (frame_report.py) -------------------------------------- + +def report_preview(body): + """Exactly the diagnostics a report would include, for the dialog to show first.""" + return {"text": frame_report.diagnostics(body.get("activity") or (), include_logs=bool(body.get("includeLogs")))} + + +def report_send(body): + try: + return frame_report.send(body) + except frame_report.ReportError as e: + raise Failure(str(e)) + + POST = {"/api/android/display": android_display, "/api/android": android, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard, "/api/flatpak": flatpak, "/api/open": open_thing, "/api/shots/save": save_shots, "/api/webinstall/check": webinstall_check, "/api/webinstall/start": webinstall_start, - "/api/webinstall/cancel": webinstall_cancel} + "/api/webinstall/cancel": webinstall_cancel, + "/api/telemetry": frame_telemetry.update_settings, "/api/telemetry/event": frame_telemetry.page_event, + "/api/report/preview": report_preview, "/api/report": report_send} # ---- HTTP ------------------------------------------------------------------ +def action_of(body): + """The action a request asked for, for diagnostics: a short word, never user data.""" + a = body.get("action") if isinstance(body, dict) else None + return a if isinstance(a, str) and re.fullmatch(r"[a-z]{1,20}", a) else "" + + def _pipe_reader(pipe): """Chunks from a pipe via a thread; select() can't wait on pipes on Windows.""" chunks = queue.Queue() # unbounded: the pump never blocks, so it ends at EOF @@ -1182,6 +1260,8 @@ class Handler(BaseHTTPRequestHandler): "shared": frame_catalog.compat_db.shared()}) elif path == "/api/android/catalog": self.send_json({"apps": frame_catalog.catalog()}) + elif path == "/api/telemetry": + self.send_json(frame_telemetry.state()) elif path == "/api/status": self.send_json(status({})) elif path == "/api/steam/owned": @@ -1208,12 +1288,14 @@ class Handler(BaseHTTPRequestHandler): except frame_android.FrameError as e: self.send_json({"error": str(e)}, 502) except Exception as e: + frame_telemetry.diagnostic(f"GET {path}", e) self.send_json({"error": f"{type(e).__name__}: {e}"}, 500) def do_POST(self): if not self.local_request(): return path = urlparse(self.path).path + body = None try: if path == "/api/upload": self.send_json(self.upload()) @@ -1230,12 +1312,16 @@ class Handler(BaseHTTPRequestHandler): raise Failure("request body must be a JSON object", 400) self.send_json(handler(body)) except Failure as e: + if e.status >= 500: + frame_telemetry.diagnostic(f"POST {path} {action_of(body)}", e) self.send_json({"error": str(e)}, e.status) except (ValueError, TypeError) as e: self.send_json({"error": f"bad request: {e}"}, 400) except frame_android.FrameError as e: + frame_telemetry.diagnostic(f"POST {path} {action_of(body)}", e) self.send_json({"error": str(e)}, 502) except Exception as e: + frame_telemetry.diagnostic(f"POST {path} {action_of(body)}", e) self.send_json({"error": f"{type(e).__name__}: {e}"}, 500) def stream_video(self, query): @@ -1356,6 +1442,7 @@ def main(): args = ap.parse_args() httpd = ThreadingHTTPServer(("127.0.0.1", args.port), Handler) sweep_tmp() + frame_telemetry.start() if not frame_host.WINDOWS: signal.signal(signal.SIGTERM, lambda *_: (_ for _ in ()).throw(KeyboardInterrupt)) if args.exit_on_eof: diff --git a/ui/telemetry.json b/ui/telemetry.json new file mode 100644 index 0000000..3c89053 --- /dev/null +++ b/ui/telemetry.json @@ -0,0 +1,5 @@ +{ + "host": "https://eu.i.posthog.com", + "key": "", + "project": "" +} From b9e31178f3c7d81058a8da674f3bfc3fdf15ea3d Mon Sep 17 00:00:00 2001 From: saphid <4596216+saphid@users.noreply.github.com> Date: Mon, 28 Sep 2026 17:42:19 +1000 Subject: [PATCH 05/51] Keyboard and trackpad for the Frame, through its own KDE Connect MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Home → Keyboard and trackpad, in every version of Frame Control (Mac, Windows, Linux, iPhone, iPad) with nothing to install on the device in your hand. On a phone: a trackpad (drag, tap, two-finger scroll and right-click) and a field that types on the Frame. On a computer: click the pad to pass the mouse and keyboard through; Esc to stop. First-party route: ui/frame_input_agent.py runs on the Frame and talks KDE Connect's LAN protocol (v7) to kdeconnectd as if it were a phone. KDE Connect isn't on the image, but Valve's package repository has it; the agent fetches it and four libraries into ~ (no root, survives updates), starts it, pairs by itself (accepting over D-Bus), and stops it again when the last device disconnects. Each device has its own identity; a stuck KDE Connect is restarted once. Verified against the real Frame (SteamOS 0.4.1): first-time install, pairing, pointer moves from the Mac's server and the iPhone app (Simulator), Mac and iPhone at once, two installs at once, a frozen daemon replaced, and the daemon stopping when the app quits. Reviewed by GPT-6 Astra (xhigh) over seven rounds; all findings fixed except per-event delivery acknowledgement (documented known limit). Co-Authored-By: Claude Opus 5.5 (1M context) --- README.md | 3 + app/main.js | 1 + app/preload.js | 2 + docs/how-the-frame-works.md | 3 + docs/streaming.md | 67 +++- ios/FrameControl/App/AppModel.swift | 8 +- ios/FrameControl/SSH/HeadsetServer.swift | 4 +- tests/page/pad_queue.mjs | 48 +++ tests/test_input.py | 196 ++++++++++++ ui/frame_input_agent.py | 376 +++++++++++++++++++++++ ui/index.html | 260 ++++++++++++++++ ui/server.py | 161 ++++++++++ 12 files changed, 1118 insertions(+), 11 deletions(-) create mode 100644 tests/page/pad_queue.mjs create mode 100644 tests/test_input.py create mode 100644 ui/frame_input_agent.py diff --git a/README.md b/README.md index 5f25235..1fc7fc6 100644 --- a/README.md +++ b/README.md @@ -67,6 +67,9 @@ Drag files onto the window to send them. Drop a game's .zip, folder or .exe to a **📸 Screenshots**
Browse the shots you take in the headset and save them to your Pictures folder. +**⌨️ Keyboard and trackpad**
+Type and point in the Frame's apps from your computer or phone, through KDE Connect on the Frame. Nothing to install on the device in your hand. + diff --git a/app/main.js b/app/main.js index bf6ff66..06c10f1 100644 --- a/app/main.js +++ b/app/main.js @@ -244,6 +244,7 @@ function fromUi(e) { ipcMain.handle("clipboard:read", (e) => fromUi(e) ? clipboard.readText() : ""); ipcMain.handle("connection:setup", (e) => { if (fromUi(e)) setUpConnection(); }); +ipcMain.on("keys:capture", (e, on) => { if (fromUi(e)) win.webContents.setIgnoreMenuShortcuts(on === true); }); // frame-control://install links from websites (docs/web-install.md). They can // arrive before the window or server exists (macOS open-url on a cold launch), diff --git a/app/preload.js b/app/preload.js index 43bc34d..24065a4 100644 --- a/app/preload.js +++ b/app/preload.js @@ -10,6 +10,8 @@ const { contextBridge, ipcRenderer, webUtils } = require("electron"); contextBridge.exposeInMainWorld("frameApp", { readClipboard: () => ipcRenderer.invoke("clipboard:read"), setUpConnection: () => ipcRenderer.invoke("connection:setup"), + // While the keyboard-and-trackpad panel holds the keyboard, ⌘W, ⌘R and the rest go to the Frame. + captureKeys: (on) => ipcRenderer.send("keys:capture", !!on), pathForFile: (file) => { try { return webUtils.getPathForFile(file) || ""; } catch { return ""; } }, onInstallLink: (cb) => { ipcRenderer.removeAllListeners("install-link"); diff --git a/docs/how-the-frame-works.md b/docs/how-the-frame-works.md index 74f81f1..696854e 100644 --- a/docs/how-the-frame-works.md +++ b/docs/how-the-frame-works.md @@ -37,6 +37,9 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600 | SteamVR settings live in `~/.config/openvr/config/steamvr.vrsettings`, not under `~/.local/share/Steam/config/`. `dashboard.lastAccessedExternalOverlayKey` names the last panel you used. | Settings tweaks | | The Steam client's journal (`journalctl --user`) carries SteamVR system UI lines such as `[Overlays] Created: …` and `vroverlay_uid`. It's the quickest way to see panels come and go. | Debugging | | Present: `rsync`, `flatpak`, `python3`, `git`, `qdbus6`, `xrdp`, `xprop`, `xwininfo`, `xterm`, `konsole`, `dolphin`, `gamescopectl`. Missing: `wl-copy`, `xclip`, `xsel`, `kdeconnect-cli`, `tailscale` (installable in `~`, see below), `krfb`, `wayvnc`. | Script design | +| **SteamOS updates arrive on their own.** The Frame went from 0.3.0 (build 20260922.6101926) to **0.4.1, build 20260925.6191901**, between 2026-09-27 and 2026-09-28 with no action from us; `~` (keys, user Flatpaks, `~/.local/share`) survived. **Verified 2026-09-28.** | Keep changes in `~` | +| **Valve's package repository has more than the image.** `pacman -Si` / `pacman -Sp` work as `steamos` without root and list Valve's own builds, such as `kdeconnect` 24.02.2 and `python-evdev` 1.7.0 in `extra`. Downloading those packages and unpacking them into `~` runs them without touching the read-only root. The repository URLs say not to share them, so fetch them on the Frame with `pacman -Sp` rather than writing them down. **Verified 2026-09-28**, SteamOS 0.4.1. | [streaming.md](streaming.md#input-type-and-point-in-the-frame-from-the-mac-or-iphone) | +| **gamescope runs two Xwayland displays.** `:0` holds Steam's VR bar and menus (`valve.steam.gamepadui.*`) and ignores XTest pointer motion; `:1` holds apps such as Chromium and takes it. There's also a libei socket, `/run/user/1000/gamescope-0-ei`. **Verified 2026-09-28**, SteamOS 0.4.1. | Keyboard and trackpad | | Flathub is a **system** remote. `--user` installs over SSH work and show up in the desktop menu. | `install-apps.sh` | | `/` is 10 GB and read-only. `/home` is 929 GB. | Where to put things | | Clipboard: Klipper over the nested D-Bus bus (`qdbus6 org.kde.klipper …`). | `paste-to-frame.sh` | diff --git a/docs/streaming.md b/docs/streaming.md index 8a1b448..a9a4f5f 100644 --- a/docs/streaming.md +++ b/docs/streaming.md @@ -66,18 +66,67 @@ capture it. ## Input: type and point in the Frame from the Mac or iPhone -**Verified 2026-09-27** on the headset: `steamos` is in the `input` group and -`/dev/uinput` is `crw-rw-r-- root input`, so **our own code can create a -virtual keyboard and mouse without sudo**. The Frame has no `python-evdev`, -`ydotool`, `wtype` or KDE Connect; `kwin_wayland` and `plasmashell` run only -while the desktop panel is open in the headset. +**Built: Home → Keyboard and trackpad**, in every version of Frame Control +(Mac, Windows, Linux, iPhone and iPad), with nothing to install on the device +you're holding. On a phone the panel is a trackpad (drag to move, tap to click, +two fingers to scroll, two-finger tap to right-click) plus a text field that +types on the Frame. On a computer, clicking the pad passes your mouse and +keyboard through to the Frame until you press Esc (⌘ is sent as Ctrl on a Mac). -| Option | Mac | iPhone | Notes | +It goes through **KDE Connect**, the first-party route (KDE makes the Frame's +desktop): Frame Control's server runs [`ui/frame_input_agent.py`](../ui/frame_input_agent.py) +on the Frame, which talks KDE Connect's own LAN protocol to the Frame's +`kdeconnectd` as if it were a phone. KDE Connect does the typing and clicking. + +**Verified 2026-09-28** (SteamOS 0.4.1, build 20260925.6191901): + +- KDE Connect isn't installed, but **Valve's package repository for the Frame + has it** (`kdeconnect` 24.02.2 in `extra`). The Frame lacks only `kpeople`, + `libfakekey`, `modemmanager-qt` and `pulseaudio-qt`. The agent fetches them + with `pacman -Sp` + `curl` into `~/.local/share/frame-control/kdeconnect` + (8 MB download, 82 MB unpacked, about 11 s), with no root and nothing on the + read-only system, so SteamOS updates leave it alone. +- It pairs by itself: the agent asks to pair and accepts on KDE Connect's side + over D-Bus (`qdbus6 … acceptPairing`). It keeps its identity in + `…/kdeconnect/bridge`, so later connections are already paired. (A pair + request to a device that's already paired makes KDE Connect unpair it, so + the agent only asks when it isn't paired.) +- Protocol version 7: whoever opens the TCP connection sends its identity line + in plain text, then acts as the **TLS server** (KDE Connect's + `lanlinkprovider.cpp`). Remote input is `kdeconnect.mousepad.request` with + `dx`/`dy`, `singleclick`, `rightclick`, `singlehold`/`singlerelease`, + `scroll`, `key` (any text) or `specialKey` (1 Backspace … 14 Escape, + 21–32 F1–F12) and modifier flags. +- **KDE Connect runs only while something uses the keyboard and trackpad.** + Each device gets its own KDE Connect identity (KDE Connect keeps one + connection per device, so a shared one would make a phone and a computer + knock each other off). The last one to disconnect stops KDE Connect, so it + isn't left running, or discoverable on your network, afterwards. +- KDE Connect 24.02 **hangs or crashes when asked to unpair a device that's + offline** (seen twice: once spinning at 100% CPU with D-Bus unresponsive, + once exiting). Frame Control never unpairs. If its copy stops answering, the + agent restarts it once (tested by freezing it with `kill -STOP`). +- Moves from the iPhone app (Simulator) and the Mac's server moved the Frame's + X pointer by exactly the amount sent. +- gamescope runs **two Xwayland displays**. `:0` holds Steam's VR bar and menus + and ignores injected pointer motion; `:1` holds apps such as Chromium and + takes it. KDE Connect runs on `:1`, so it reaches apps, not Steam's own menus. + There's also a `gamescope-0-ei` (libei) socket. +- **Not yet tested:** typing and clicking as seen in the headset, and whether + it reaches the KDE desktop panel (Plasma is its own session). +- **Known limit:** keys and clicks typed while the link is reconnecting wait + and are sent once it's back, but anything sent in the moment the Wi-Fi + drops, before SSH notices, can be lost. Confirming every event would add a + round trip to each pointer move. + +Our own `uinput` keyboard and mouse would also work (`steamos` is in the +`input` group and `/dev/uinput` is group-writable, verified 2026-09-27), and +remains the fallback if KDE Connect ever can't be fetched. + +| Other option | Mac | iPhone | Why not | |---|---|---|---| -| **A uinput keyboard and mouse in Frame Control's server** | ✓ | ✓ | **Recommended.** The server opens `/dev/uinput` with `ctypes` (standard library only) and the page sends key and pointer events through the tunnel it already has. On the phone: a trackpad area (drag to move, tap to click, two fingers to scroll) and the iOS keyboard for typing. On the Mac: a "control the Frame" mode that captures the keyboard and pointer (Esc to release). Uinput devices look like real hardware to the kernel, so libinput, KWin and gamescope should take them; [frame-voice](https://github.com/DeeJanuz/frame-voice) already types into a Frame through a uinput keyboard. **Untested**: which surfaces in VR (desktop panel, SteamVR dashboard, games, Android apps in Lepton) accept the pointer. About a day or two of work | -| **Bluetooth keyboard and mouse** | – | – | Real hardware paired in SteamOS settings. The iPhone can't pretend to be a Bluetooth keyboard: iOS won't advertise the HID service ([Apple forums](https://developer.apple.com/forums/thread/733916)) | +| **Bluetooth keyboard and mouse** | – | – | Needs real hardware, paired in SteamOS settings. The iPhone can't pretend to be a Bluetooth keyboard: iOS won't advertise the HID service ([Apple forums](https://developer.apple.com/forums/thread/733916)) | | **Deskflow** (formerly Input Leap / Barrier) | ✓ | – | Moves the Mac's own mouse and keyboard onto the Frame's screen edge. Flathub has an aarch64 build ([Flathub](https://flathub.org/apps/org.deskflow.deskflow)); on Wayland it needs the InputCapture/libei portal, and only works while Plasma is running. No iPhone client | -| **KDE Connect** | ~ | ✓ | Its iOS app has a remote touchpad and keyboard, but the Frame would need KDE Connect installed (not on Flathub; `pacman` on a read-only root). More moving parts than the uinput route | | **Remmina / Steam Link / RDP** | ✓ | – | Input only reaches the streamed session, not the headset's own apps | Other ways to get text in: diff --git a/ios/FrameControl/App/AppModel.swift b/ios/FrameControl/App/AppModel.swift index 94e25fa..c56f144 100644 --- a/ios/FrameControl/App/AppModel.swift +++ b/ios/FrameControl/App/AppModel.swift @@ -1,4 +1,5 @@ import Citadel +import CryptoKit import Foundation import SwiftUI import UIKit @@ -33,6 +34,11 @@ final class AppModel: ObservableObject { } var deviceName: String { UIDevice.current.userInterfaceIdiom == .pad ? "iPad" : "iPhone" } + /// Stable per install: a hash of this device's SSH key, which is made once and kept in the Keychain. + var clientID: String { + let digest = SHA256.hash(data: Data(authorizedKeysLine.utf8)) + return deviceName.lowercased() + "-" + digest.prefix(6).map { String(format: "%02x", $0) }.joined() + } private var hostKey: String? { UserDefaults.standard.string(forKey: Self.hostKeyKey) } // MARK: pairing @@ -167,7 +173,7 @@ final class AppModel: ObservableObject { guard current() else { throw CancellationError() } step("Starting Frame Control on the headset") let key = Self.randomKey() - let server = try await HeadsetServer.start(in: dir, over: l, key: key, device: deviceName) + let server = try await HeadsetServer.start(in: dir, over: l, key: key, device: deviceName, client: clientID) guard current() else { throw CancellationError() } let f = try await PortForwarder.start(over: l, to: server.port) forwarder = f diff --git a/ios/FrameControl/SSH/HeadsetServer.swift b/ios/FrameControl/SSH/HeadsetServer.swift index 443e99d..d59388d 100644 --- a/ios/FrameControl/SSH/HeadsetServer.swift +++ b/ios/FrameControl/SSH/HeadsetServer.swift @@ -80,8 +80,10 @@ final class HeadsetServer: @unchecked Sendable { } /// Starts the server in dir and waits for it to say which port it took. - static func start(in dir: String, over link: FrameLink, key: String, device: String) async throws -> HeadsetServer { + /// client is a stable id for this install (keyboard-and-trackpad pairing is kept per client). + static func start(in dir: String, over link: FrameLink, key: String, device: String, client: String) async throws -> HeadsetServer { let command = "cd \(dir) && FRAME_LOCAL=1 FRAME_UI_KEY=\(key) FRAME_DEVICE=\(shellQuote(device)) " + + "FRAME_CLIENT=\(shellQuote(client)) " + "exec python3 -I -u -B \"$PWD/ui/server.py\" --port 0 --exit-on-eof 2>&1" let stream = try await link.client.executeCommandStream(command) let box = PortWaiter() diff --git a/tests/page/pad_queue.mjs b/tests/page/pad_queue.mjs new file mode 100644 index 0000000..f8e748c --- /dev/null +++ b/tests/page/pad_queue.mjs @@ -0,0 +1,48 @@ +// The page's input queue, run in node against the real functions from ui/index.html. +import { readFileSync } from "fs"; +const src = readFileSync(new URL("../../ui/index.html", import.meta.url), "utf8"); +const grab = name => src.match(new RegExp(`(?:const ${name} = [^\\n]*\\n)|((?:async )?function ${name}\\([\\s\\S]*?\\n}\\n)`))[0]; +const code = ["isMove", "padSend", "padKeep", "padFlush"].map(grab).join(""); +const fail = msg => { console.log("FAIL " + msg); process.exit(1); }; + +function page(api) { + const pad = { state: "ready", queue: [], sending: false }; + const shown = []; + const fns = new Function("pad", "api", "padShow", code + "; return { padSend, padFlush };")(pad, api, s => { shown.push(s); pad.state = s.state; }); + return { pad, shown, ...fns }; +} + +// While a request is in flight, events queue by these rules. +{ + const { pad, padSend } = page(() => new Promise(() => {})); // a request that never returns + const send = (state, e) => { pad.state = state; padSend(e); }; + send("ready", { singlehold: true }); // goes out, and stays in flight + send("error", { dx: 3 }); // dropped: a stale move while broken + send("error", { singlerelease: true }); // kept + send("off", { singlerelease: true }); // off: nothing was held + send("pairing", { key: "a" }); // kept while reconnecting + send("pairing", { dx: 1 }); // dropped + for (let i = 0; i < 250; i++) send("pairing", { key: "x" }); + send("pairing", { singlerelease: true }); // kept past the 200 cap + const q = pad.queue; + if (!(q[0].singlerelease && q[1].key === "a" && q.at(-1).singlerelease && !q.some(e => "dx" in e) + && q.filter(e => e.singlerelease).length === 2)) fail("queueing " + JSON.stringify(q.slice(0, 3)) + " len " + q.length); +} + +// A request that fails, or reaches a link that had dropped, keeps its keys and releases. +for (const [label, api] of [["request failed", async () => { throw new Error("offline"); }], + ["not sent", async () => ({ state: "starting", sent: false })]]) { + const { pad, padSend, shown } = page(api); + padSend({ dx: 5 }); // sent and lost: a move, fine to drop + await new Promise(r => setTimeout(r, 0)); + pad.state = "ready"; + padSend({ singlerelease: true }); + padSend({ dx: 7 }); // queued while that request is in flight: stale once it fails + padSend({ key: "b" }); + await new Promise(r => setTimeout(r, 0)); + const q = pad.queue; + if (!(q.some(e => e.singlerelease) && q.some(e => e.key === "b") && !q.some(e => "dx" in e))) + fail(label + ": " + JSON.stringify(q)); + if (!shown.length || shown.at(-1).state === "ready") fail(label + ": state not shown"); +} +console.log("queue rules ok"); diff --git a/tests/test_input.py b/tests/test_input.py new file mode 100644 index 0000000..bc2afbc --- /dev/null +++ b/tests/test_input.py @@ -0,0 +1,196 @@ +"""Keyboard and pointer: event checks, and the agent's KDE Connect protocol against a fake kdeconnectd. + +The fake follows what KDE Connect 24.02 does (read from its source, +core/backends/lan/lanlinkprovider.cpp): it accepts the TCP connection, reads the +identity line, then starts TLS as the *client*. + +Run: python3 -m unittest discover -s tests +""" +import json +import os +import shutil +import socket +import ssl +import subprocess +import sys +import tempfile +import threading +import time +import unittest +from pathlib import Path + +ROOT = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(ROOT / "ui")) + + +class InputEvents(unittest.TestCase): + @classmethod + def setUpClass(cls): + import server + cls.server = server + + def check(self, event): + return self.server.input_event(event) + + def test_keeps_known_fields(self): + self.assertEqual(self.check({"dx": 3, "dy": -1.234}), {"dx": 3.0, "dy": -1.23}) + self.assertEqual(self.check({"singleclick": True, "other": 1}), {"singleclick": True}) + self.assertEqual(self.check({"key": "héllo", "shift": True}), {"key": "héllo", "shift": True}) + self.assertEqual(self.check({"specialKey": 12, "ctrl": True}), {"specialKey": 12, "ctrl": True}) + self.assertEqual(self.check({"scroll": True, "dy": 1}), {"scroll": True, "dy": 1.0}) + + def test_clamps_movement(self): + self.assertEqual(self.check({"dx": 1e9})["dx"], self.server.INPUT_MOVE_LIMIT) + self.assertEqual(self.check({"dy": -1e9})["dy"], -self.server.INPUT_MOVE_LIMIT) + + def test_rejects_bad_events(self): + bad = [None, [], "a", {}, {"shift": True}, {"dx": "1"}, {"dx": True}, {"dx": float("nan")}, + {"key": ""}, {"key": 5}, {"key": "x" * 501}, {"specialKey": 0}, {"specialKey": 33}, + {"specialKey": True}, {"specialKey": 1.5}, {"singleclick": "yes"}] + for event in bad: + with self.assertRaises(self.server.Failure, msg=repr(event)): + self.check(event) + + def test_send_while_link_is_down_says_not_sent(self): + # The page keeps unsent keys and clicks and sends them once the agent is ready again. + from types import SimpleNamespace + + class Probe(self.server.InputAgent): + def start(self): + self.proc, self.status = SimpleNamespace(poll=lambda: None, stdin=None), {"state": "starting"} + + agent = Probe() + agent.status, agent.proc = {"state": "ready"}, SimpleNamespace(poll=lambda: 255) # ssh has exited + self.assertEqual(agent.send([{"key": "x"}]), {"state": "starting", "sent": False}) + + def test_command_passes_client_quoted(self): + os.environ["FRAME_CLIENT"] = "test-client-1" + try: + cmd = self.server.InputAgent().command() + finally: + del os.environ["FRAME_CLIENT"] + self.assertTrue(cmd.startswith("python3 -u -c '")) + self.assertIn(" test-client-1 ", cmd) + + def test_batch_limits(self): + with self.assertRaises(self.server.Failure): + self.server.remote_input({"events": "dx"}) + with self.assertRaises(self.server.Failure): + self.server.remote_input({"events": [{"dx": 1}] * (self.server.INPUT_BATCH_LIMIT + 1)}) + + +class FakeKdeConnect: + """Just enough of kdeconnectd: pairs when asked and records remote-input packets.""" + + def __init__(self, paired=False): + self.listener = socket.socket() + self.listener.bind(("127.0.0.1", 0)) + self.listener.listen(1) + self.port = self.listener.getsockname()[1] + self.paired, self.identity, self.received, self.pair_requests = paired, None, [], 0 + threading.Thread(target=self.serve, daemon=True).start() + + def serve(self): + conn, _ = self.listener.accept() + line = b"" + while not line.endswith(b"\n"): + line += conn.recv(1) + self.identity = json.loads(line) + ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + ctx.check_hostname, ctx.verify_mode = False, ssl.CERT_NONE + tls = ctx.wrap_socket(conn) # "Starting client ssl (but I'm the server TCP socket)" + tls.sendall(b'{"id": 1, "type": "kdeconnect.mousepad.keyboardstate", "body": {"state": true}}\n') + buf = b"" + while True: + try: + chunk = tls.recv(65536) + except OSError: + return + if not chunk: + return + buf += chunk + while b"\n" in buf: + raw, buf = buf.split(b"\n", 1) + p = json.loads(raw) + if p["type"] == "kdeconnect.pair": + self.pair_requests += 1 + elif p["type"] == "kdeconnect.mousepad.request" and self.paired: + self.received.append(p["body"]) + + +@unittest.skipIf(sys.platform == "win32", "the agent runs on the Frame (Linux)") +@unittest.skipUnless(shutil.which("openssl"), "needs openssl to make a certificate") +class AgentProtocol(unittest.TestCase): + @classmethod + def setUpClass(cls): + import frame_input_agent + cls.agent = frame_input_agent + cls.dir = tempfile.TemporaryDirectory() + cls.cert, cls.key = Path(cls.dir.name, "cert.pem"), Path(cls.dir.name, "key.pem") + subprocess.run(["openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", "-days", "1", + "-subj", "/CN=framecontrol_test", "-keyout", str(cls.key), "-out", str(cls.cert)], + check=True, capture_output=True) + + @classmethod + def tearDownClass(cls): + cls.dir.cleanup() + + def wait_for(self, check): + for _ in range(100): + if check(): + return + time.sleep(0.02) + self.fail("timed out") + + def test_pairs_then_forwards_events(self): + fake = FakeKdeConnect() + link = self.agent.Link("framecontrol_0123456789abcdef01234567", self.cert, self.key, port=fake.port) + link.pair(lambda: fake.paired, lambda: setattr(fake, "paired", fake.pair_requests > 0), timeout=5) + self.assertEqual(fake.pair_requests, 1) + ident = fake.identity["body"] + self.assertEqual(fake.identity["type"], "kdeconnect.identity") + self.assertEqual(ident["protocolVersion"], 7) + self.assertIn("kdeconnect.mousepad.request", ident["outgoingCapabilities"]) + link.read(0.5) + self.assertTrue(link.keyboard) + for body in self.agent.events(b'[{"dx": 4, "dy": -2}, {"key": "hi"}]'): + link.send(body) + self.wait_for(lambda: len(fake.received) == 2) + self.assertEqual(fake.received, [{"dx": 4, "dy": -2}, {"key": "hi"}]) + + def test_already_paired_sends_no_pair_request(self): + # A pair request to a device that's already paired makes KDE Connect unpair it. + fake = FakeKdeConnect(paired=True) + link = self.agent.Link("framecontrol_0123456789abcdef01234567", self.cert, self.key, port=fake.port) + link.pair(lambda: True, lambda: self.fail("accepted a pairing that wasn't needed")) + link.send({"singleclick": True}) + self.wait_for(lambda: fake.received == [{"singleclick": True}]) + self.assertEqual(fake.pair_requests, 0) + + def test_client_args_are_folder_safe(self): + old = sys.argv + try: + sys.argv = ["-c", "../../etc x", "Alex's Mac"] + self.assertEqual(self.agent.client_args(), ("etcx", "Frame Control (Alex's Mac)")) + sys.argv = ["-c"] + self.assertEqual(self.agent.client_args(), ("default", "Frame Control")) + finally: + sys.argv = old + + def test_events_parsing(self): + self.assertEqual(self.agent.events(b'{"dx": 1}'), [{"dx": 1}]) + self.assertEqual(self.agent.events(b'[{"dx": 1}, 5, {}]'), [{"dx": 1}]) + self.assertEqual(self.agent.events(b"not json"), []) + + +@unittest.skipUnless(shutil.which("node"), "needs node") +class PageQueue(unittest.TestCase): + """The page's input queue (tests/page/pad_queue.mjs runs the real functions from index.html).""" + + def test_queue_rules(self): + r = subprocess.run(["node", str(ROOT / "tests" / "page" / "pad_queue.mjs")], capture_output=True, text=True) + self.assertEqual(r.returncode, 0, r.stdout + r.stderr) + + +if __name__ == "__main__": + unittest.main() diff --git a/ui/frame_input_agent.py b/ui/frame_input_agent.py new file mode 100644 index 0000000..6a38861 --- /dev/null +++ b/ui/frame_input_agent.py @@ -0,0 +1,376 @@ +"""Keyboard and pointer for the Steam Frame. Frame Control's server runs this ON the Frame. + +It speaks KDE Connect's LAN protocol (version 7, as in KDE Connect 24.02) to the +Frame's own kdeconnectd, as a phone would, and forwards remote-input events read +from stdin: one JSON object (or list of them) per line, each a KDE Connect +"mousepad" request body such as {"dx": 4, "dy": -2} or {"key": "hello"}. +KDE Connect does the typing and clicking. + +KDE Connect isn't installed on the Frame, but Valve's package repository for it +has a build. The first run fetches that and the few libraries the Frame lacks +into ~/.local/share/frame-control/kdeconnect: no root, and SteamOS updates +leave it alone. + +Status goes to stdout, one JSON object per line: +{"state": "installing" | "starting" | "pairing" | "ready" | "error", ...}. + +Standard library only: this runs on the Frame's own Python. +""" +import fcntl +import json +import os +import selectors +import shutil +import signal +import socket +import ssl +import subprocess +import sys +import time +from pathlib import Path + +BASE = Path.home() / ".local/share/frame-control/kdeconnect" +ROOT = BASE / "root" +BRIDGE = BASE / "bridge" +# kdeconnect plus the dependencies the Frame's image doesn't have (checked 2026-09-28, +# SteamOS 0.4.1); `pacman -Sp` adds any others still missing. +PACKAGES = ["kdeconnect", "kpeople", "libfakekey", "modemmanager-qt", "pulseaudio-qt"] +PORT = int(os.environ.get("FRAME_INPUT_PORT", "1716")) +UID = os.getuid() +MOUSEPAD = "kdeconnect.mousepad.request" + + +def say(state, **more): + print(json.dumps({"state": state, **more}), flush=True) + + +def packet(kind, body): + return (json.dumps({"id": int(time.time() * 1000), "type": kind, "body": body}) + "\n").encode() + + +# ---- KDE Connect on the Frame ------------------------------------------------ + +def daemon_path(): + for path in (Path("/usr/lib/kdeconnectd"), ROOT / "usr/lib/kdeconnectd"): + if path.exists(): + return path + return None + + +def install(): + say("installing", message="Fetching KDE Connect from the Frame's package repository") + found = subprocess.run(["pacman", "-Sp", *PACKAGES], capture_output=True, text=True, timeout=120) + urls = [u for u in found.stdout.split() if u.startswith("https://")] + if found.returncode or not urls: + raise RuntimeError("Couldn't find KDE Connect in the Frame's package repository: " + + (found.stderr.strip() or "no packages listed")) + download, stage = BASE / "download", BASE / "root.new" + for d in (download, stage): + shutil.rmtree(d, ignore_errors=True) + d.mkdir(parents=True) + for url in urls: + name = download / url.rsplit("/", 1)[1] + subprocess.run(["curl", "-fsSL", "--retry", "2", "-o", str(name), url], check=True, timeout=600) + if subprocess.run(["tar", "--zstd", "-xf", str(name), "-C", str(stage)], capture_output=True).returncode: + subprocess.run(["bsdtar", "-xf", str(name), "-C", str(stage)], check=True, capture_output=True) + shutil.rmtree(ROOT, ignore_errors=True) + stage.rename(ROOT) + shutil.rmtree(download, ignore_errors=True) + + +def app_display(): + """The X display that apps (not Steam's own VR menus) are on. + + gamescope runs two Xwayland servers: on 2026-09-28 :0 held Steam's VR bar and + menus and ignored XTest pointer motion, while :1 held apps such as Chromium and + took it. Inferred to hold in general. + """ + return ":1" if Path("/tmp/.X11-unix/X1").exists() else ":0" + + +def daemon_env(daemon): + env = dict(os.environ, DBUS_SESSION_BUS_ADDRESS=f"unix:path=/run/user/{UID}/bus", + XDG_RUNTIME_DIR=f"/run/user/{UID}", DISPLAY=app_display(), QT_QPA_PLATFORM="xcb") + if str(daemon).startswith(str(ROOT)): + env.update(LD_LIBRARY_PATH=str(ROOT / "usr/lib"), QT_PLUGIN_PATH=str(ROOT / "usr/lib/qt6/plugins"), + QML_IMPORT_PATH=str(ROOT / "usr/lib/qt6/qml"), + XDG_DATA_DIRS=f"{ROOT / 'usr/share'}:/usr/share") + return env + + +def listening(): + try: + socket.create_connection(("127.0.0.1", PORT), 1).close() + return True + except OSError: + return False + + +def our_daemons(): + """Process ids of the kdeconnectd that Frame Control installed (never a system one).""" + pids = [] + for proc in Path("/proc").iterdir(): + if proc.name.isdigit(): + try: + if os.readlink(proc / "exe").startswith(str(ROOT) + "/"): + pids.append(int(proc.name)) + except OSError: + pass + return pids + + +def stop_daemon(): + """Stop our kdeconnectd and wait until it's gone (so its port is closed too).""" + for sig, wait in ((signal.SIGTERM, 30), (signal.SIGKILL, 30)): # tenths of a second + for pid in our_daemons(): + try: + os.kill(pid, sig) + except ProcessLookupError: + pass + for _ in range(wait): + if not our_daemons() and not listening(): + return + time.sleep(0.1) + + +def ensure_daemon(): + if listening(): + return + BASE.mkdir(parents=True, exist_ok=True) + daemon = daemon_path() + if not daemon: + install() + daemon = daemon_path() + say("starting", message="Starting KDE Connect on the Frame") + log = open(BASE / "kdeconnectd.log", "ab") + # Its own session, so it outlives this connection and serves the next one. + subprocess.Popen([str(daemon)], env=daemon_env(daemon), cwd=str(Path.home()), stdin=subprocess.DEVNULL, + stdout=log, stderr=log, start_new_session=True) + for _ in range(40): + if listening(): + return + time.sleep(0.25) + raise RuntimeError(f"KDE Connect didn't start; see {BASE / 'kdeconnectd.log'} on the Frame") + + +def qdbus(device, method): + """Call a method on KDE Connect's D-Bus object for our device; its output, or None.""" + env = dict(os.environ, DBUS_SESSION_BUS_ADDRESS=f"unix:path=/run/user/{UID}/bus") + try: + r = subprocess.run(["qdbus6", "org.kde.kdeconnect", f"/modules/kdeconnect/devices/{device}", + f"org.kde.kdeconnect.device.{method}"], capture_output=True, text=True, env=env, timeout=5) + except (OSError, subprocess.TimeoutExpired): + return None + return r.stdout.strip() if r.returncode == 0 else None + + +# ---- our identity -------------------------------------------------------------- + +def identity(client): + """A device id and certificate for this client, made once and kept (pairing is tied to them). + + Each computer or phone gets its own: KDE Connect keeps one connection per device, + so a shared identity would make them knock each other off. + """ + folder = BRIDGE / client + folder.mkdir(parents=True, exist_ok=True) + id_file, cert, key = folder / "id", folder / "cert.pem", folder / "key.pem" + if not (id_file.exists() and cert.exists() and key.exists()): + device = "framecontrol_" + os.urandom(12).hex() # KDE Connect wants 32-38 of [A-Za-z0-9_] + subprocess.run(["openssl", "req", "-x509", "-newkey", "ec", "-pkeyopt", "ec_paramgen_curve:prime256v1", + "-nodes", "-days", "3650", "-subj", f"/O=KDE/OU=Kde connect/CN={device}", + "-keyout", str(key), "-out", str(cert)], check=True, capture_output=True) + os.chmod(key, 0o600) + id_file.write_text(device) + return id_file.read_text().strip(), cert, key + + +# ---- the link ------------------------------------------------------------------ + +class Link: + """One TLS connection to kdeconnectd, as a paired device that sends remote input.""" + + def __init__(self, device, cert, key, port=PORT, name="Frame Control"): + self.device, self.buf, self.keyboard = device, b"", None + raw = socket.create_connection(("127.0.0.1", port), 5) + raw.sendall(packet("kdeconnect.identity", { + "deviceId": device, "deviceName": name, "deviceType": "phone", "protocolVersion": 7, + "incomingCapabilities": [], "outgoingCapabilities": [MOUSEPAD], "tcpPort": port})) + # KDE Connect's rule: whoever opened the TCP connection is the TLS server. + ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER) + ctx.load_cert_chain(str(cert), str(key)) + ctx.verify_mode = ssl.CERT_NONE # both sides are on this machine + self.sock = ctx.wrap_socket(raw, server_side=True) + self.sock.setblocking(False) + + def send(self, body): + # Bounded: if KDE Connect stops reading, fail (and be restarted) rather than hang. + self.sock.settimeout(5) + try: + self.sock.sendall(packet(MOUSEPAD, body)) + finally: + self.sock.setblocking(False) + + def pair(self, paired, accept, timeout=15): + """Ask to pair and accept it on KDE Connect's side (we control both ends). + + Only asks when not already paired: a pair request to a device that is + already paired makes KDE Connect unpair it. + """ + if paired(): + return + self.sock.settimeout(5) + self.sock.sendall(packet("kdeconnect.pair", {"pair": True})) + self.sock.setblocking(False) + end = time.time() + timeout + while time.time() < end: + accept() + self.read(0.5) + if paired(): + return + raise RuntimeError("KDE Connect didn't accept the pairing") + + def read(self, wait=0.0): + """Packets waiting from kdeconnectd; None once it has closed the connection.""" + if wait: + sel = selectors.DefaultSelector() + sel.register(self.sock, selectors.EVENT_READ) + sel.select(wait) + sel.close() + try: + while True: + chunk = self.sock.recv(65536) + if not chunk: + return None + self.buf += chunk + except (ssl.SSLWantReadError, BlockingIOError): + pass + out = [] + while b"\n" in self.buf: + line, self.buf = self.buf.split(b"\n", 1) + if line.strip(): + p = json.loads(line) + if p.get("type") == "kdeconnect.mousepad.keyboardstate": + self.keyboard = bool(p.get("body", {}).get("state")) + out.append(p) + return out + + +def events(line): + """The event bodies in one stdin line (an object or a list of objects).""" + try: + value = json.loads(line) + except ValueError: + return [] + return [e for e in (value if isinstance(value, list) else [value]) if isinstance(e, dict) and e] + + +def connect(device, cert, key, name): + link = Link(device, cert, key, name=name) + link.pair(lambda: qdbus(device, "isPaired") == "true", lambda: qdbus(device, "acceptPairing")) + link.read(0.5) # its hello, including whether it can type + return link + + +def client_args(): + """argv: a folder-safe id for the computer or phone, and the name KDE Connect shows for it.""" + client = sys.argv[1] if len(sys.argv) > 1 else "default" + client = "".join(c for c in client if c.isalnum() or c in "-_")[:64] or "default" + name = (sys.argv[2] if len(sys.argv) > 2 else "")[:60].strip() + return client, f"Frame Control ({name})" if name else "Frame Control" + + +def main(): + client, name = client_args() + # A dropped ssh (the Frame slept, the app quit) hangs up on us: exit through the + # clean-up below rather than dying on the spot. + for sig in (signal.SIGHUP, signal.SIGTERM): + signal.signal(sig, lambda *_: sys.exit(0)) + BASE.mkdir(parents=True, exist_ok=True) + # Every agent holds this lock shared while it runs. The last one out gets it + # exclusively and stops KDE Connect, so it runs, and shows up on the network, + # only while something is using the keyboard and trackpad. + clients = open(BASE / "clients.lock", "w") + fcntl.flock(clients, fcntl.LOCK_SH) + try: + return run(client, name) + finally: + # Finish the clean-up even if a second hang-up or TERM arrives meanwhile. + for sig in (signal.SIGHUP, signal.SIGTERM): + signal.signal(sig, signal.SIG_IGN) + fcntl.flock(clients, fcntl.LOCK_UN) + try: + fcntl.flock(clients, fcntl.LOCK_EX | fcntl.LOCK_NB) + except OSError: + pass # another device is still using it + else: + with daemon_lock(): + stop_daemon() + + +class daemon_lock: + """Installing, starting and restarting KDE Connect happen one agent at a time.""" + + def __enter__(self): + self.file = open(BASE / "daemon.lock", "w") + fcntl.flock(self.file, fcntl.LOCK_EX) + + def __exit__(self, *_): + self.file.close() + + +def run(client, name): + try: + with daemon_lock(): + ensure_daemon() + device, cert, key = identity(client) + say("pairing") + seen = our_daemons() + try: + link = connect(device, cert, key, name) + except (OSError, RuntimeError): + if not seen: + raise + # Ours, but not answering (KDE Connect 24.02 can hang, for one after + # unpairing a device that's offline): start it afresh, once. If another + # agent already replaced it, just use the new one. + say("starting", message="Restarting KDE Connect on the Frame") + with daemon_lock(): + if set(our_daemons()) & set(seen): + stop_daemon() + ensure_daemon() + link = connect(device, cert, key, name) + except (OSError, RuntimeError, subprocess.SubprocessError) as e: + say("error", message=str(e)) + return 1 + say("ready", keyboard=link.keyboard is not False) + stdin, pending = sys.stdin.fileno(), b"" + sel = selectors.DefaultSelector() + sel.register(stdin, selectors.EVENT_READ) + sel.register(link.sock, selectors.EVENT_READ) + while True: + for key_, _ in sel.select(30): + if key_.fileobj == stdin: + chunk = os.read(stdin, 65536) # raw reads: a buffered readline could strand lines select can't see + if not chunk: # the server went away + return 0 + *lines, pending = (pending + chunk).split(b"\n") + try: + for line in lines: + for body in events(line): + link.send(body) + except OSError as e: + say("error", message=f"Lost KDE Connect: {e}") + return 1 + else: + packets = link.read() + if packets is None: + say("error", message="KDE Connect closed the connection") + return 1 + if any(p.get("type") == "kdeconnect.pair" and not p.get("body", {}).get("pair") for p in packets): + say("error", message="KDE Connect unpaired Frame Control") + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/ui/index.html b/ui/index.html index 3845ef7..7cf4b42 100644 --- a/ui/index.html +++ b/ui/index.html @@ -322,6 +322,19 @@ border-radius: 3px; height: 32px; padding: 0 8px; font: inherit; font-size: 13px; } /* Touch screens can't hover: keep the library's name and Play button showing. */ @media (hover: none) { .capsule .over { opacity: 1; } .capsule:hover { transform: none; } } + /* ---- keyboard and trackpad ---- */ + .pad-area { position: relative; height: 190px; margin: 12px 0; border-radius: 4px; background: rgba(0,0,0,.28); + border: 1px dashed rgba(255,255,255,.14); display: grid; place-items: center; cursor: pointer; + touch-action: none; user-select: none; -webkit-user-select: none; -webkit-touch-callout: none; outline: none; } + .pad-area:focus-visible, .pad-area.captured { border: 1px solid var(--blue); box-shadow: 0 0 0 1px var(--blue) inset; } + .pad-area.off { cursor: default; opacity: .55; } + .pad-hint { max-width: 420px; padding: 0 16px; text-align: center; color: var(--muted); font-size: 13px; pointer-events: none; } + .pad-keys { display: flex; flex-wrap: wrap; gap: 8px; align-items: center; } + .pad-keys input { flex: 1 1 220px; min-width: 0; height: 32px; padding: 0 10px; border-radius: 3px; border: 1px solid transparent; + background: rgba(0,0,0,.28); color: var(--text); font: inherit; } + .pad-keys input:focus { outline: none; border-color: var(--blue); } + #pad .hint { margin-top: 10px; } + /* ---- phones, upright or on their side: tabs move to a bottom bar, as in iOS apps. Every edge keeps clear of the safe area (notch or Dynamic Island, rounded corners, home indicator); env() is zero on desktops. ---- */ @media (max-width: 640px), (max-height: 500px) and (hover: none) { @@ -358,6 +371,10 @@ button.small { height: 32px; } .actions button { height: 46px; } .cat-tools select { max-width: none; flex: 1 1 100%; } + .pad-area { height: 240px; } + .pad-keys .seg { flex: 1 1 100%; display: grid; grid-template-columns: repeat(4, 1fr); } + .pad-keys .pad-clicks { grid-template-columns: repeat(2, 1fr); } + .pad-keys .seg button { padding: 0 4px; justify-content: center; text-align: center; } /* The on-screen keyboard covers the bottom of the screen; the tab bar would ride on top of it. */ body.typing nav { display: none; } } @@ -471,6 +488,36 @@ +
+

Keyboard and trackpad

+ +
+
+
+
+
+ + + + + + + + + + + + + + + +
+
Types and points in apps on the Frame, such as Chromium or the desktop's Linux apps; Steam's own VR menus + don't take it. It works through KDE Connect on the Frame: the first time, Frame Control fetches Valve's build of it + (an 8 MB download, 82 MB unpacked) into your home folder there and pairs with it. Nothing to install on this device.
+
+

Screenshots

@@ -2128,6 +2175,219 @@ window.addEventListener("hashchange", showPage); const typesText = el => el?.matches?.('textarea, input:not([type=range], [type=checkbox], [type=radio], [type=file], [type=button])'); document.addEventListener("focusin", e => document.body.classList.toggle("typing", !!typesText(e.target))); document.addEventListener("focusout", () => document.body.classList.remove("typing")); + +// ---- keyboard and trackpad: events go to KDE Connect on the Frame (see ui/frame_input_agent.py) ---- +const pad = { state: "off", queue: [], sending: false, poll: null, captured: false }; +const PAD_STATES = { + off: "", starting: "Connecting…", installing: "Setting up KDE Connect on the Frame (first time only)…", + pairing: "Pairing with KDE Connect…", ready: "On", +}; +// KDE Connect's specialKey numbers (plugins/mousepad in its source). +const PAD_SPECIAL = { Backspace: 1, Tab: 2, ArrowLeft: 4, ArrowUp: 5, ArrowRight: 6, ArrowDown: 7, PageUp: 8, + PageDown: 9, Home: 10, End: 11, Enter: 12, Delete: 13, Escape: 14, + ...Object.fromEntries(Array.from({ length: 12 }, (_, i) => [`F${i + 1}`, 21 + i])) }; +const IS_MAC = /^Mac/.test(navigator.platform); +const TOUCH = matchMedia("(hover: none)").matches; + +function padShow(status) { + pad.state = status.state || "off"; + const ready = pad.state === "ready"; + const busy = !ready && pad.state !== "off" && pad.state !== "error"; + $("padState").textContent = pad.state === "error" ? "" : PAD_STATES[pad.state] || ""; + $("padOn").hidden = ready || busy; + $("padOn").textContent = pad.state === "error" ? "Try again" : "Turn on"; + $("padArea").classList.toggle("off", !ready); + $("padHint").textContent = pad.state === "error" ? status.message || "Couldn't reach KDE Connect on the Frame." + : !ready ? (busy ? PAD_STATES[pad.state] : "Turn on to use this as a trackpad for the Frame.") + : TOUCH ? "Drag to move the pointer · tap to click · two fingers to scroll · two-finger tap to right-click" + : pad.captured ? "Your mouse and keyboard now control the Frame. Press Esc to stop." + : "Click here to use your mouse and keyboard on the Frame (Esc to stop), or drag like a trackpad."; + clearTimeout(pad.poll); + if (ready && pad.queue.length) padFlush(); + if (busy) pad.poll = setTimeout(async () => padShow(await api("/api/input").catch(e => ({ state: "error", message: e.message }))), 1000); +} +async function padStart() { + localStorage.padUsed = "1"; // from now on it turns on with the page + padShow({ state: "starting" }); + padShow(await api("/api/input?start=1").catch(e => ({ state: "error", message: e.message }))); +} +$("padOn").onclick = padStart; + +// Queue events and send them one request at a time, merging pointer moves made meanwhile. +const isMove = e => Object.keys(e).every(k => k === "dx" || k === "dy"); +function padSend(event) { + // While it reconnects, keys and clicks wait in the queue; pointer moves would be stale. + // A button release is always kept, so a drag never leaves the button held on the Frame. + const room = pad.queue.length < 200 || event.singlerelease; + const reconnecting = ["starting", "pairing", "installing"].includes(pad.state) && room; + // Even after an error: a button pressed on the Frame must be let go once it's back. + const release = event.singlerelease && pad.state !== "off"; + if (pad.state !== "ready" && !(reconnecting && !isMove(event)) && !release) return; + const last = pad.queue[pad.queue.length - 1]; + if (last && isMove(last) && isMove(event)) { last.dx = (last.dx || 0) + (event.dx || 0); last.dy = (last.dy || 0) + (event.dy || 0); } + else pad.queue.push(event); + padFlush(); +} +// Events that didn't reach the Frame (the link had dropped, and is reconnecting): keep +// the keys and clicks for when it's back, but not pointer moves, which would be stale. +// Button releases are kept whatever the queue's length. +function padKeep(events) { + pad.queue = [...events, ...pad.queue].filter(e => !isMove(e)).filter((e, i) => i < 200 || e.singlerelease); +} +async function padFlush() { + if (pad.sending || !pad.queue.length || pad.state !== "ready") return; + pad.sending = true; + const events = pad.queue.splice(0, 200); + try { + const status = await api("/api/input", { events }); + if (!status.sent) padKeep(events); + if (status.state !== "ready") padShow(status); + } catch (e) { + padKeep(events); + padShow({ state: "error", message: e.message }); + } finally { + pad.sending = false; + if (pad.queue.length) padFlush(); + } +} +function padKey(e) { + const mods = { ...(e.ctrlKey || (IS_MAC && e.metaKey) ? { ctrl: true } : {}), ...(e.altKey ? { alt: true } : {}), + ...(!IS_MAC && e.metaKey ? { super: true } : {}) }; + if (PAD_SPECIAL[e.key]) return { specialKey: PAD_SPECIAL[e.key], ...mods, ...(e.shiftKey ? { shift: true } : {}) }; + if (e.key.length === 1) return { key: e.key, ...mods }; // already shifted, so no shift flag + return null; +} + +// Touch: one finger moves, a quick tap clicks; two fingers scroll, or right-click when tapped. +const PAD_SPEED = 1.6, PAD_SCROLL_STEP = 18; +const touches = new Map(); +let gesture = null; +$("padArea").addEventListener("pointerdown", e => { + if (pad.state === "off" || pad.state === "error") { padStart(); return; } + if (pad.state !== "ready") return; + if (e.pointerType === "mouse") { + if (pad.captured) return; // mousedown and mouseup below send the clicks + // Lock the pointer to the pad and pass the whole mouse and keyboard through; where + // that isn't allowed, the mouse drags like a finger on a trackpad instead. + if (e.button === 0 && !pad.noLock && $("padArea").requestPointerLock) { $("padArea").requestPointerLock(); return; } + // Only the left button drags and taps like a finger; the others are clicks of their own. + if (e.button === 1 || e.button === 2) { padSend(e.button === 2 ? { rightclick: true } : { middleclick: true }); return; } + if (e.button !== 0) return; + } + try { $("padArea").setPointerCapture(e.pointerId); } catch {} // keeps the drag if the finger leaves the pad + touches.set(e.pointerId, { x: e.clientX, y: e.clientY }); + if (touches.size === 1) gesture = { start: performance.now(), moved: 0, fingers: 1, scroll: 0 }; + else if (gesture) gesture.fingers = Math.max(gesture.fingers, touches.size); + e.preventDefault(); +}); +$("padArea").addEventListener("pointermove", e => { + if (pad.captured) { padSend({ dx: e.movementX, dy: e.movementY }); return; } + const t = touches.get(e.pointerId); + if (!t || !gesture) return; + const dx = e.clientX - t.x, dy = e.clientY - t.y; + t.x = e.clientX; t.y = e.clientY; + gesture.moved += Math.abs(dx) + Math.abs(dy); + if (touches.size >= 2) { + // Scroll with the lead finger only, a notch per step, like a laptop trackpad (content follows the fingers). + if (e.pointerId !== touches.keys().next().value) return; + gesture.scroll += dy; + while (Math.abs(gesture.scroll) >= PAD_SCROLL_STEP) { + const up = gesture.scroll > 0; + padSend({ scroll: true, dy: up ? 1 : -1 }); + gesture.scroll -= up ? PAD_SCROLL_STEP : -PAD_SCROLL_STEP; + } + } else if (gesture.fingers === 1) { + padSend({ dx: dx * PAD_SPEED, dy: dy * PAD_SPEED }); + } +}); +function padLift(e) { + if (!touches.delete(e.pointerId) || touches.size || !gesture) return; + const quick = performance.now() - gesture.start < 250 && gesture.moved < 10; + if (quick) padSend(gesture.fingers >= 2 ? { rightclick: true } : { singleclick: true }); + gesture = null; +} +$("padArea").addEventListener("pointerup", padLift); +$("padArea").addEventListener("pointercancel", e => { touches.delete(e.pointerId); if (!touches.size) gesture = null; }); + +// A computer's mouse and keyboard: captured while the pointer is locked to the pad. +document.addEventListener("pointerlockerror", () => { pad.noLock = true; }); +document.addEventListener("pointerlockchange", () => { + pad.captured = document.pointerLockElement === $("padArea"); + window.frameApp?.captureKeys?.(pad.captured); // the desktop app's menu shortcuts go to the Frame too + // Esc or switching windows mid-drag: let go of the button on the Frame as well. + if (!pad.captured && pad.held) { padSend({ singlerelease: true }); pad.held = false; } + $("padArea").classList.toggle("captured", pad.captured); + padShow({ state: pad.state }); +}); +$("padArea").addEventListener("mousedown", e => { + if (!pad.captured) return; + if (e.button === 0) { padSend({ singlehold: true }); pad.held = true; } + else if (e.button === 1) padSend({ middleclick: true }); + else if (e.button === 2) padSend({ rightclick: true }); +}); +$("padArea").addEventListener("mouseup", e => { if (pad.held && e.button === 0) { padSend({ singlerelease: true }); pad.held = false; } }); +$("padArea").addEventListener("contextmenu", e => e.preventDefault()); +$("padArea").addEventListener("wheel", e => { + if (pad.state !== "ready") return; + e.preventDefault(); + if (e.deltaY) padSend({ scroll: true, dy: e.deltaY > 0 ? -1 : 1 }); +}, { passive: false }); +// Capture phase, so the page's own shortcuts (tabs 1-4, R, zoom) don't also fire. +window.addEventListener("keydown", e => { + if (!pad.captured) return; + e.preventDefault(); e.stopImmediatePropagation(); + const event = padKey(e); + if (event) padSend(event); +}, true); + +// The text field: each keystroke goes to the Frame. While focused it holds one space, +// so Backspace always has something to delete; empty otherwise, to show its prompt. +const padType = $("padType"); +padType.addEventListener("beforeinput", e => { + if (e.inputType === "insertCompositionText") return; // IME composing; sent on compositionend + e.preventDefault(); + if (pad.state === "off" || pad.state === "error") padStart(); // what's typed meanwhile waits in the queue + const text = e.data ?? e.dataTransfer?.getData("text/plain") ?? ""; + if (/^insert(Text|ReplacementText|FromPaste|FromDrop)$/.test(e.inputType) && text) padText(text); + else if (e.inputType === "deleteContentBackward") padSend({ specialKey: 1 }); + else if (e.inputType === "deleteWordBackward") padSend({ specialKey: 1, ctrl: true }); // Ctrl+Backspace: a word, on Linux + else if (e.inputType === "deleteContentForward") padSend({ specialKey: 13 }); + else if (e.inputType === "deleteWordForward") padSend({ specialKey: 13, ctrl: true }); + else if (e.inputType === "insertLineBreak" || e.inputType === "insertParagraph") padSend({ specialKey: 12 }); +}); +padType.addEventListener("compositionend", e => { if (e.data) padText(e.data); padType.value = " "; padCaret(); }); +// Text in pieces the server takes (500 characters), with Enter for each line break. +function padText(text) { + text.replace(/\r\n?/g, "\n").split("\n").forEach((line, i) => { + if (i) padSend({ specialKey: 12 }); + const chars = Array.from(line); // whole characters, so an emoji isn't cut in two + for (let at = 0; at < chars.length; at += 500) padSend({ key: chars.slice(at, at + 500).join("") }); + }); +} +padType.addEventListener("keydown", e => { + const special = PAD_SPECIAL[e.key]; + // Text and deletions arrive as beforeinput; this takes the keys that don't edit the field. + if (!special || e.key === "Backspace" || e.key === "Delete" || e.isComposing) return; + e.preventDefault(); + padSend(padKey(e)); +}); +// The caret stays after the space, so Backspace always has something to delete. +const padCaret = () => requestAnimationFrame(() => padType.setSelectionRange(1, 1)); +padType.addEventListener("focus", () => { padType.value = " "; padCaret(); if (pad.state === "off") padStart(); }); +padType.addEventListener("blur", () => { padType.value = ""; }); +padType.addEventListener("input", e => { + if (e.isComposing) return; // an input method is still composing; compositionend sends it + if (padType.value !== " ") padType.value = " "; + padCaret(); +}); +padType.addEventListener("click", padCaret); +document.querySelector("#pad .pad-keys").addEventListener("click", e => { + const b = e.target.closest("button"); + if (!b) return; + if (b.dataset.padClick) padSend({ [b.dataset.padClick]: true }); + else if (b.dataset.padKey) padSend({ specialKey: +b.dataset.padKey }); +}); +api("/api/input").then(s => s.state === "off" && localStorage.padUsed ? padStart() : padShow(s), () => padShow({ state: "off" })); showPage(); document.addEventListener("keydown", e => { if (e.metaKey || e.ctrlKey || e.altKey || /INPUT|TEXTAREA|SELECT/.test(document.activeElement.tagName) || document.querySelector("dialog[open]")) return; diff --git a/ui/server.py b/ui/server.py index ef22af6..df1c63f 100755 --- a/ui/server.py +++ b/ui/server.py @@ -10,6 +10,7 @@ Env: FRAME_ALIAS (default frame) FRAME_LOCAL=1 run on the Frame itself (the iPhone app starts it there over SSH) FRAME_UI_KEY required X-Frame-UI value (the iPhone app passes a fresh one) FRAME_DEVICE what to call the device the page runs on (e.g. iPhone) + FRAME_CLIENT a stable id for that device (keyboard-and-trackpad pairing is kept per id) """ import argparse import base64 @@ -54,6 +55,8 @@ if LOCAL: os.environ["PATH"] = f"{HERE / 'local-bin'}{os.pathsep}{os.environ.get('PATH', '')}" UI_KEY = os.environ.get("FRAME_UI_KEY") or "1" DEVICE = os.environ.get("FRAME_DEVICE") or "phone" +# What the Frame's KDE Connect calls this device (keyboard and trackpad). +INPUT_NAME = DEVICE if LOCAL else socket.gethostname().split(".")[0] FRAME = os.environ.get("FRAME_ALIAS", "frame") if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._-]*", FRAME): sys.exit(f"FRAME_ALIAS must be a plain host alias, not {FRAME!r}") @@ -497,6 +500,161 @@ def clipboard(body): return {"message": ssh(PASTE_CMD, stdin=text, timeout=30).strip()} +# ---- keyboard and pointer (KDE Connect on the Frame, see frame_input_agent.py) ---- + +INPUT_FLAGS = ("singleclick", "doubleclick", "middleclick", "rightclick", "singlehold", "singlerelease", + "scroll", "ctrl", "alt", "shift", "super") +INPUT_MOVE_LIMIT = 2000 # pixels per event +INPUT_TEXT_LIMIT = 500 # characters per event +INPUT_BATCH_LIMIT = 200 # events per request + + +def input_event(event): + """A KDE Connect remote-input body with only the fields it knows, in range.""" + if not isinstance(event, dict): + raise Failure("each input event must be an object", 400) + out = {} + for name in ("dx", "dy"): + value = event.get(name) + if value is None: + continue + if isinstance(value, bool) or not isinstance(value, (int, float)) or value != value: + raise Failure(f"{name} must be a number", 400) + out[name] = max(-INPUT_MOVE_LIMIT, min(INPUT_MOVE_LIMIT, round(float(value), 2))) + for name in INPUT_FLAGS: + if event.get(name) is True: + out[name] = True + key = event.get("key") + if key is not None: + if not isinstance(key, str) or not 0 < len(key) <= INPUT_TEXT_LIMIT: + raise Failure(f"key must be text of 1 to {INPUT_TEXT_LIMIT} characters", 400) + out["key"] = key + special = event.get("specialKey") + if special is not None: + # KDE Connect's numbering: 1 Backspace … 14 Escape, 21-32 F1-F12. + if isinstance(special, bool) or not isinstance(special, int) or not 1 <= special <= 32: + raise Failure("specialKey must be a whole number from 1 to 32", 400) + out["specialKey"] = special + if not set(out) - {"ctrl", "alt", "shift", "super"}: + raise Failure("input event has nothing to do", 400) + return out + + +def input_client(): + """A stable id for this device, so KDE Connect on the Frame keeps its pairing apart. + + The iPhone app passes one (FRAME_CLIENT). A computer makes one the first time + and keeps it: host names alone can clash (desk.home and desk.office). + """ + if os.environ.get("FRAME_CLIENT"): + return os.environ["FRAME_CLIENT"] + if LOCAL: + return DEVICE + path = frame_host.data_dir("input-client-id") + try: + saved = path.read_text().strip() + if re.fullmatch(r"[A-Za-z0-9_-]{4,64}", saved): + return saved + except OSError: + pass + made = (re.sub(r"[^A-Za-z0-9-]", "", INPUT_NAME)[:24] or "computer") + "-" + secrets.token_hex(4) + try: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(made) + except OSError: + pass # still works this time; it pairs again next time + return made + + +class InputAgent: + """frame_input_agent.py running on the Frame, fed events over one long-lived ssh. + + It sets up KDE Connect there if needed, pairs, and reports its state + ({"state": "off" | "installing" | "starting" | "pairing" | "ready" | "error"}). + """ + + def __init__(self, source=HERE / "frame_input_agent.py"): + self.source, self.proc, self.lock = source, None, threading.Lock() + self.status = {"state": "off"} + + def command(self): + code = base64.b64encode(self.source.read_bytes()).decode() + client = input_client() + return ("python3 -u -c " + shlex.quote( + f"import base64;exec(compile(base64.b64decode('{code}'),'frame_input_agent','exec'))") + + f" {shlex.quote(client)} {shlex.quote(INPUT_NAME)}") + + def start(self): + with self.lock: + if self.proc and self.proc.poll() is None: + return + ensure_master() + self.status = {"state": "starting"} + errors = tempfile.TemporaryFile() + self.proc = proc = subprocess.Popen([*SSH, FRAME, self.command()], stdin=subprocess.PIPE, + stdout=subprocess.PIPE, stderr=errors) + _live_tunnels.add(proc) + threading.Thread(target=self._watch, args=(proc, errors), daemon=True).start() + + def _watch(self, proc, errors): + for line in proc.stdout: + try: + status = json.loads(line) + except ValueError: + continue + if isinstance(status, dict) and isinstance(status.get("state"), str): + with self.lock: + if self.proc is proc: + self.status = status + proc.wait() + _live_tunnels.discard(proc) + errors.seek(0) + detail = strip_ansi(errors.read().decode(errors="replace")).strip() + with self.lock: + if self.proc is proc and self.status.get("state") != "error": + message = detail.splitlines()[-1] if detail else "The connection to the Frame ended" + friendly = unreachable(message) + self.status = {"state": "error", "message": friendly or message, **({"offline": True} if friendly else {})} + + def send(self, events): + """Forward events if the agent is ready; start it if it isn't running. + + Returns the state, with "sent" saying whether the events went; if not, + the page keeps them and sends them again once the state is "ready". + """ + with self.lock: + proc, ready = self.proc, self.status.get("state") == "ready" + sent = False + if not (proc and proc.poll() is None): + self.start() + elif ready and events: + try: + proc.stdin.write((json.dumps(events) + "\n").encode()) + proc.stdin.flush() + sent = True + except (BrokenPipeError, OSError, ValueError): + pass # _watch reports how it ended + with self.lock: + return {**self.status, "sent": sent} + + def stop(self): + with self.lock: + proc, self.proc, self.status = self.proc, None, {"state": "off"} + if proc and proc.poll() is None: + proc.terminate() + + +_input = InputAgent() + + +def remote_input(body): + """{"events": [...]} sends keyboard and pointer events; {} (or none yet) just starts the agent.""" + events = body.get("events", []) + if not isinstance(events, list) or len(events) > INPUT_BATCH_LIMIT: + raise Failure(f"events must be a list of at most {INPUT_BATCH_LIMIT}", 400) + return _input.send([input_event(e) for e in events]) + + def flatpak(body): app, action = str(body.get("id", "")), body.get("action") if not FLATPAK_ID.match(app): @@ -1214,6 +1372,7 @@ def _sweep_one(prefix, d): POST = {"/api/android/display": android_display, "/api/android": android, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard, + "/api/input": remote_input, "/api/flatpak": flatpak, "/api/open": open_thing, "/api/shots/save": save_shots, "/api/webinstall/check": webinstall_check, "/api/webinstall/start": webinstall_start, "/api/webinstall/cancel": webinstall_cancel} @@ -1327,6 +1486,8 @@ class Handler(BaseHTTPRequestHandler): self.send_json({"titles": frame_titles.list_titles()}) elif path == "/api/titles/job": self.send_json(title_job(url.query)) + elif path == "/api/input": + self.send_json(_input.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_input.status)) elif path == "/api/job": self.send_json(job_status(url.query)) elif path == "/api/android/displays": From 73eef14ecd7928f75217621955cdd267e2c7d956 Mon Sep 17 00:00:00 2001 From: saphid <4596216+saphid@users.noreply.github.com> Date: Mon, 28 Sep 2026 19:44:44 +1000 Subject: [PATCH 06/51] Report APKs refused before install; offer a compatibility test after installing an alternative Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_frame_apk_versions.py | 23 +++++++++++++++++++++++ ui/index.html | 1 + ui/server.py | 5 +++++ 3 files changed, 29 insertions(+) diff --git a/tests/test_frame_apk_versions.py b/tests/test_frame_apk_versions.py index 8cf7701..53f1511 100644 --- a/tests/test_frame_apk_versions.py +++ b/tests/test_frame_apk_versions.py @@ -256,5 +256,28 @@ class UploadVersionsTest(unittest.TestCase): ssh.assert_not_called() + def test_blocked_uploads_are_reported_like_failed_installs(self): + import server + info = {'package': 'org.example.app', 'label': 'Example', 'version': '5', + 'version_code': 5, 'min_sdk': 33, 'abis': [], 'icon_png': None} + for apk_info, expected_info in ((dict(info), 'org.example.app'), + (frame_android.FrameError('not an APK'), None)): + handler = object.__new__(server.Handler) + handler.headers = {'X-Filename': 'app.apk', 'X-Mode': 'apk', 'Content-Length': '1'} + handler.rfile = io.BytesIO(b'x') + calls = [] + patch_info = (patch.object(frame_android, 'apk_info', side_effect=apk_info) + if isinstance(apk_info, Exception) else + patch.object(frame_android, 'apk_info', return_value=apk_info)) + with patch_info, patch.object(frame_android, 'install_hooks', [lambda *a: calls.append(a)]), \ + patch.object(server, 'ensure_master'): + with self.assertRaises(server.Failure): + handler.upload() + self.assertEqual(len(calls), 1) + got_info, meta, error, _ = calls[0] + self.assertEqual((got_info or {}).get('package'), expected_info) + self.assertIsNone(meta) + self.assertIsInstance(error, frame_android.FrameError) + if __name__ == '__main__': unittest.main() diff --git a/ui/index.html b/ui/index.html index 82e6437..92ec81a 100644 --- a/ui/index.html +++ b/ui/index.html @@ -1545,6 +1545,7 @@ function showApkAlternatives(reason, result) { if (res) $("apkAltDlg").close(); else { b.disabled = false; b.textContent = "Install"; } await loadAndroid(); if (cat.apps) { const y = window.scrollY; filterCatalog(); window.scrollTo(0, y); } + if (res && res.app) await offerTest(res.app); }; if (!$("apkAltDlg").open) $("apkAltDlg").showModal(); } diff --git a/ui/server.py b/ui/server.py index 8a04aa0..23aa600 100755 --- a/ui/server.py +++ b/ui/server.py @@ -1595,13 +1595,18 @@ class Handler(BaseHTTPRequestHandler): keep = True # stage_title owns tmp now, and removes it on failure return stage_title(str(dest), temp_dir=str(tmp)) if mode == "apk": + # Checked here, before install(), to hand the page a blocker it can offer + # alternatives for; report these failures the way install() would have. + start = time.time() try: info = frame_android.apk_info(str(dest)) except frame_android.FrameError as e: + frame_android._after_install(None, None, e, start) raise Failure(str(e), 400) try: frame_android.check_installable(info) except frame_android.FrameError as e: + frame_android._after_install(info, None, e, start) raise Failure(str(e), 400, {"package": info["package"], "version_code": info.get("version_code"), "blocker": str(e)}) ensure_master() try: From e67802f15d973940545d84fdca0e3849f4c8d4b8 Mon Sep 17 00:00:00 2001 From: saphid <4596216+saphid@users.noreply.github.com> Date: Mon, 28 Sep 2026 20:11:48 +1000 Subject: [PATCH 07/51] Tests: keep the blocked-upload test from reaching real install reporting Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/test_frame_apk_versions.py | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/test_frame_apk_versions.py b/tests/test_frame_apk_versions.py index 53f1511..508ddcf 100644 --- a/tests/test_frame_apk_versions.py +++ b/tests/test_frame_apk_versions.py @@ -242,6 +242,7 @@ class UploadVersionsTest(unittest.TestCase): handler.rfile = io.BytesIO(b'x') with patch.object(frame_android, 'apk_info', return_value=dict(info)), \ patch.object(versions, 'alternatives', side_effect=AssertionError('lookup during upload')) as lookup, \ + patch.object(frame_android, 'install_hooks', []), \ patch.object(server, 'ensure_master') as ssh: if mode == 'apkinfo': reply = handler.upload() From f0765277220219382a6302de9c73b8ac5efb4f72 Mon Sep 17 00:00:00 2001 From: saphid <4596216+saphid@users.noreply.github.com> Date: Mon, 28 Sep 2026 20:14:36 +1000 Subject: [PATCH 08/51] Send analytics to the existing PostHog project; make bug reports private - Analytics go to the maintainer's PostHog US project 343535, tagged $lib = frame-control. Every event carries $ip 0.0.0.0, since PostHog stores the sender's address otherwise (checked live), including events queued by earlier versions. - Report a problem sends a private problem_report event to PostHog instead of a public GitHub issue, with its own random id so a contact address can't be linked to analytics. The dialog asks how to reach the person and shows a reference. Maintainers read reports on the PostHog dashboard or with `python3 ui/frame_report.py inbox`. - Community sync pages by timestamp in UTC: PostHog refuses OFFSET for personal API keys. Co-Authored-By: Claude Opus 5.5 (1M context) --- README.md | 2 +- docs/privacy.md | 34 ++++++++----- tests/test_telemetry.py | 92 ++++++++++++++++++++++++--------- ui/frame_compat_db.py | 20 +++++--- ui/frame_report.py | 110 ++++++++++++++++++++++------------------ ui/frame_telemetry.py | 59 ++++++++++++++------- ui/index.html | 30 ++++------- ui/telemetry.json | 6 +-- 8 files changed, 220 insertions(+), 133 deletions(-) diff --git a/README.md b/README.md index c4574f7..a87d650 100644 --- a/README.md +++ b/README.md @@ -182,7 +182,7 @@ This is a first public test, so reports are really useful, especially from Windows and Linux. The quickest way is **Report a problem** in the app (the warning-sign button at the top, or **Help → Report a Problem…**). It adds diagnostics with personal details removed, shows you exactly what's included, -and opens an issue here. You don't need a GitHub account. Without the app, +and sends it privately to the maintainer; nothing is published. Without the app, use the [feedback form](https://frame-control.pages.dev/feedback/). Please include: - what you tried and what happened diff --git a/docs/privacy.md b/docs/privacy.md index af44eae..db7004f 100644 --- a/docs/privacy.md +++ b/docs/privacy.md @@ -1,7 +1,7 @@ # Privacy and analytics Frame Control sends anonymous analytics to [PostHog](https://posthog.com) -(EU cloud) so the maintainer can see how many people use it, which features +(US cloud) so the maintainer can see how many people use it, which features matter and where installs fail. You choose how much in **Privacy & updates**, the last panel on the page. `ui/frame_telemetry.py` is the whole implementation. @@ -28,8 +28,9 @@ computer, exactly as they were sent. its data folder (`telemetry/settings.json`). It isn't derived from your computer, account or network. To get a new one, delete that file. - Events are sent without person profiles (`$process_person_profile: false`) - and without location lookup (`$geoip_disable: true`). The PostHog project - is also set to discard client IP addresses. + and without location lookup (`$geoip_disable: true`). Each carries a + placeholder address (`$ip: 0.0.0.0`), so PostHog stores that instead of + yours. - Every event includes the app version, OS name (macOS, Windows or Linux), CPU architecture and Python version. @@ -97,10 +98,14 @@ The same error is sent at most once every 10 minutes. ## Report a problem **Report a problem** is the warning-sign button in the header, also in the -Privacy panel and under **Help → Report a Problem…**. It files a public issue -on [GitHub](https://github.com/saphid/frame-control/issues) through the -website's feedback service, so no GitHub account is needed. It works whatever -the analytics settings are, because the person sends it deliberately. +Privacy panel and under **Help → Report a Problem…**. It sends the report +privately to Frame Control's PostHog project as a `problem_report` event, the +same way as the analytics above, so only the maintainer can read it and +nothing is published. It works whatever the analytics settings are, because +the person sends it deliberately. The report has the kind, title and text you +wrote, how to reach you if you gave it, a short reference shown after sending, +and the diagnostics below. It has its own random id, so it isn't linked to +your analytics events. With **Include diagnostics** ticked (the default), the report adds: @@ -114,10 +119,14 @@ because those lines can name files and apps. When ticked, it adds the newest Activity lines and server log lines, without the request lines. Everything is scrubbed like error details and limited to what fits in the -issue. Environment details are kept first, then the newest lines. **Show +report. Environment details are kept first, then the newest lines. **Show exactly what's included** shows the snapshot that will be sent, and later -activity isn't added to it. If the service can't be reached, the dialog offers -**Copy report** and **Open on GitHub instead**, a prefilled issue. +activity isn't added to it. If PostHog can't be reached, **Copy report** puts +the whole report on the clipboard. + +The maintainer reads reports on the Frame Control dashboard in PostHog, or +with `python3 ui/frame_report.py inbox [days]`, which uses the same personal +API key as `frame_compat_db.py sync`. ## Turning it all off @@ -128,6 +137,7 @@ never sends anything unless `FRAME_CONTROL_TELEMETRY=1` is set. ## Update checks The desktop app asks GitHub for the latest release shortly after starting, -then every 6 hours (`api.github.com/repos/saphid/frame-control/releases/latest`). -That request carries no id. To stop it, set +then every 6 hours: the latest release's `update.json` on GitHub, or +`api.github.com/repos/saphid/frame-control/releases/latest` if that fails. +Those requests carry no id. To stop it, set `FRAME_CONTROL_NO_UPDATE_CHECK=1`. See [releasing.md](releasing.md). diff --git a/tests/test_telemetry.py b/tests/test_telemetry.py index 19043bc..6a033c7 100644 --- a/tests/test_telemetry.py +++ b/tests/test_telemetry.py @@ -322,18 +322,17 @@ class Regressions(Base): class ReportProblem(Base): - """Report a problem: diagnostics are scrubbed and fit the feedback API; sending goes to it.""" + """Report a problem: diagnostics are scrubbed and bounded; the report goes privately to PostHog.""" - def serve(self, status=201, reply=None): + def serve(self, status=200): got = [] class H(BaseHTTPRequestHandler): def do_POST(self): - got.append(json.loads(self.rfile.read(int(self.headers["Content-Length"])))) + got.append((self.path, json.loads(self.rfile.read(int(self.headers["Content-Length"]))))) self.send_response(status) self.end_headers() - self.wfile.write(json.dumps(reply or {"ok": True, "number": 42, - "url": "https://github.com/saphid/frame-control/issues/42"}).encode()) + self.wfile.write(b'{"status":"Ok"}') def log_message(self, *a): pass @@ -342,7 +341,7 @@ class ReportProblem(Base): threading.Thread(target=httpd.serve_forever, daemon=True).start() self.addCleanup(httpd.server_close) self.addCleanup(httpd.shutdown) - p = mock.patch.object(fr, "FEEDBACK_URL", f"http://127.0.0.1:{httpd.server_port}/api/feedback") + p = mock.patch.dict(os.environ, {"FRAME_CONTROL_POSTHOG_HOST": f"http://127.0.0.1:{httpd.server_port}"}) p.start() self.addCleanup(p.stop) return got @@ -358,11 +357,12 @@ class ReportProblem(Base): for leaked in ("alice", "192.168.1.9", str(Path.home()), "bob", "pw@"): self.assertNotIn(leaked, text) - def test_a_report_fits_the_api_limit_in_utf16_units(self): - body = {"title": "Live view stops", "message": "It stops 😀 " * 600, "diagnostics": "log 😀 line\n" * 2000} - title, message = fr.compose(body) - self.assertLessEqual(fr.u16(message), fr.MESSAGE_MAX) - self.assertTrue(message.startswith("It stops")) + def test_a_report_is_bounded_in_utf16_units(self): + body = {"title": "Live view stops", "message": "It stops 😀 " * 800, "diagnostics": "log 😀 line\n" * 2000} + title, text, diag = fr.compose(body) + self.assertLessEqual(fr.u16(text), fr.TEXT_MAX) + self.assertLessEqual(fr.u16(diag), fr.DIAG_MAX) + self.assertTrue(text.startswith("It stops")) with self.assertRaises(ValueError): fr.compose({"title": "hi", "message": "It stops after a minute."}) @@ -381,24 +381,66 @@ class ReportProblem(Base): def test_the_previewed_diagnostics_are_what_is_sent(self): got = self.serve() - fr.send({"title": "Live view stops", "message": "It stops after a minute.", "elapsed": 9000, + fr.send({"title": "Live view stops", "message": "It stops after a minute.", "diagnostics": "Frame Control 9.9.9\nssh janes-mac.tail12345.ts.net failed"}) - self.assertIn("Frame Control 9.9.9", got[0]["message"]) - self.assertNotIn("janes-mac", got[0]["message"]) + diag = got[0][1]["batch"][0]["properties"]["diagnostics"] + self.assertIn("Frame Control 9.9.9", diag) + self.assertNotIn("janes-mac", diag) - def test_send_files_an_issue_and_reports_its_number(self): + def test_send_is_a_private_posthog_event_whatever_the_settings(self): got = self.serve() - res = fr.send({"kind": "bug", "title": "Live view stops", "message": "It stops after a minute.", - "elapsed": 9000, "github": "@someone"}) - self.assertEqual((res["number"], res["url"]), (42, "https://github.com/saphid/frame-control/issues/42")) - sent = got[0] - self.assertEqual((sent["kind"], sent["github"], sent["website"], sent["elapsed"]), ("bug", "someone", "", 9000)) - self.assertEqual(sent["message"], "It stops after a minute.") + tm.update_settings({"usage": False}) # analytics off: a deliberate report still goes + res = fr.send({"kind": "idea", "title": "Live view stops", "message": "It stops after a minute.", + "contact": "me@example.com"}) + path, body = got[0] + event = body["batch"][0] + self.assertEqual((path, body["api_key"], event["event"]), ("/batch/", "phc_test", "problem_report")) + props = event["properties"] + self.assertEqual((props["kind"], props["title"], props["message"], props["contact"], props["report_id"]), + ("idea", "Live view stops", "It stops after a minute.", "me@example.com", res["id"])) + self.assertEqual((props["$process_person_profile"], props["$geoip_disable"]), (False, True)) + self.assertNotEqual(event["distinct_id"], tm.settings()["id"]) # not linked to the analytics + self.assertIn(res["id"], res["message"]) + self.assertEqual([e["event"] for e in tm._read_lines(tm.SENT)], ["problem_report"]) - def test_the_services_error_is_passed_on(self): - self.serve(status=429, reply={"error": "That's a lot of feedback in one hour."}) - with self.assertRaisesRegex(fr.ReportError, "a lot of feedback"): - fr.send({"title": "Live view stops", "message": "It stops after a minute.", "elapsed": 9000}) + def test_a_sent_report_is_not_an_error_if_the_local_log_fails(self): + self.serve() + with mock.patch.object(tm, "record_sent", side_effect=OSError("disk full")): + res = fr.send({"title": "Live view stops", "message": "It stops after a minute."}) + self.assertTrue(res["id"]) + + def test_events_queued_by_older_versions_get_the_placeholder_address(self): + got = self.serve() + tm.update_settings({"noticeShown": True}) + tm._write_lines(tm.OUTBOX, [{"event": "app_opened", "distinct_id": "x", "uuid": "u1", + "properties": {"level": "usage"}}]) + self.assertEqual(tm.flush(), 1) + self.assertEqual(got[0][1]["batch"][0]["properties"]["$ip"], "0.0.0.0") + self.assertEqual(tm._read_lines(tm.SENT)[0]["properties"]["$ip"], "0.0.0.0") + + def test_the_inbox_skips_malformed_reports(self): + good = ["2026-09-28T09:50:00Z", "AB12CD34", "bug", "Live view stops", "It stops.", None, + "0.4.0", "macOS", "", ""] + rows = [["2026-09-28T10:00:00Z", "X", "bug", "Hand-made", None, None, None, None, None, None], ["short"], good] + with mock.patch.object(db, "_posthog_query", return_value={"results": rows}), \ + mock.patch.object(sys, "argv", ["frame_report.py", "inbox"]), \ + mock.patch("builtins.print") as out: + fr.main() + printed = " ".join(str(c.args[0]) for c in out.call_args_list if c.args) + self.assertIn("AB12CD34", printed) + self.assertIn("Hand-made", printed) + + def test_a_refused_report_is_an_error(self): + self.serve(status=401) + with self.assertRaisesRegex(fr.ReportError, "HTTP 401"): + fr.send({"title": "Live view stops", "message": "It stops after a minute."}) + self.assertEqual(tm._read_lines(tm.SENT), []) + + def test_no_key_means_no_report(self): + with mock.patch.dict(os.environ, {"FRAME_CONTROL_POSTHOG_KEY": ""}), \ + mock.patch.object(tm, "HERE", tm.STATE): + with self.assertRaisesRegex(fr.ReportError, "no PostHog project key"): + fr.send({"title": "Live view stops", "message": "It stops after a minute."}) if __name__ == "__main__": diff --git a/ui/frame_compat_db.py b/ui/frame_compat_db.py index bc28dd4..6371435 100644 --- a/ui/frame_compat_db.py +++ b/ui/frame_compat_db.py @@ -267,7 +267,7 @@ def _posthog_query(sql): project = os.environ.get('FRAME_CONTROL_POSTHOG_PROJECT') or cfg.get('project') if not project: raise DBError('No PostHog project id (ui/telemetry.json "project", or FRAME_CONTROL_POSTHOG_PROJECT)') - # The query API lives on the app host (eu.posthog.com), not the ingestion host (eu.i.posthog.com). + # The query API lives on the app host (us.posthog.com), not the ingestion host (us.i.posthog.com). host = cfg['host'].replace('.i.posthog.com', '.posthog.com') req = urllib.request.Request(f'{host}/api/projects/{urllib.parse.quote(str(project))}/query/', method='POST', data=json.dumps({'query': {'kind': 'HogQLQuery', 'query': sql}}).encode(), @@ -352,15 +352,21 @@ def sync(dry_run=False): key() # the maintainer's copy only state = _sync_state() since = time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(time.time() - SYNC_OVERLAP_DAYS * 86400)) - events = [] - for page in range(40): - res = _posthog_query("SELECT properties, distinct_id, timestamp FROM events " - f"WHERE event = 'compat_report' AND timestamp >= toDateTime('{since}') " - f"ORDER BY timestamp, uuid LIMIT {SYNC_PAGE} OFFSET {page * SYNC_PAGE}") + events, after = [], f"timestamp >= toDateTime('{since}', 'UTC')" + for _ in range(40): + # Keyset paging: PostHog refuses OFFSET with a personal API key. The cursor is in UTC, + # since a local time is ambiguous in the hour clocks go back. + res = _posthog_query("SELECT properties, distinct_id, timestamp, toString(uuid), " + "formatDateTime(timestamp, '%Y-%m-%d %H:%i:%S.%f', 'UTC') FROM events " + f"WHERE event = 'compat_report' AND {after} " + f"ORDER BY timestamp, toString(uuid) LIMIT {SYNC_PAGE}") rows = res.get('results') or [] - events += rows + events += [row[:3] for row in rows] if len(rows) < SYNC_PAGE: break + last_uuid, last_ts = rows[-1][3], rows[-1][4] + after = (f"(timestamp > toDateTime64('{last_ts}', 6, 'UTC') OR " + f"(timestamp = toDateTime64('{last_ts}', 6, 'UTC') AND toString(uuid) > '{last_uuid}'))") rows, skipped = community_rows(events, state) if dry_run: return rows, skipped diff --git a/ui/frame_report.py b/ui/frame_report.py index f898ea5..ffcfa9d 100644 --- a/ui/frame_report.py +++ b/ui/frame_report.py @@ -1,32 +1,26 @@ """Report a problem from inside Frame Control. Python stdlib only. The page's Report a problem dialog shows the diagnostics below before anything -is sent, then this sends the report to the website's feedback API -(site/functions/api/feedback.js), which files it as a GitHub issue. The user -needs no GitHub account. Everything collected is scrubbed first -(frame_telemetry.scrub), since the issue is public. +is sent, then this sends the report privately to Frame Control's PostHog +project as a `problem_report` event: only the maintainer can read it, and +nothing is published. It is sent whatever the analytics settings are, because +the person sends it deliberately. Diagnostics are scrubbed first +(frame_telemetry.scrub); the person's own words are sent as written. """ -import json import os import platform import sys import time -import urllib.error -import urllib.request +import uuid import frame_host import frame_telemetry -FEEDBACK_URL = os.environ.get('FRAME_CONTROL_FEEDBACK_URL', 'https://frame-control.pages.dev/api/feedback') -ISSUES_URL = 'https://github.com/saphid/frame-control/issues/new' KINDS = ('bug', 'idea', 'question', 'other') -MESSAGE_MAX = 5000 # the feedback API's limit, in JavaScript (UTF-16) units -TEXT_MAX = 3500 # the person's own text -DIAG_MAX = 1300 # the diagnostics block, so text + diagnostics always fit MESSAGE_MAX -LOG_LINES = 40 +TEXT_MAX = 5000 # the person's own text, in JavaScript (UTF-16) units like the page's maxlength +DIAG_MAX = 8000 # the diagnostics block +LOG_LINES = 60 ACTIVITY_LINES = 25 -HEAD = '\n\n---\nDiagnostics from Frame Control (personal details removed):\n```\n' -TAIL = '\n```' frame = {} # the Frame's last known SteamOS build, set by server.status() @@ -96,52 +90,72 @@ def diagnostics(activity=(), include_logs=False, limit=DIAG_MAX): def compose(body): - """(title, message) for the feedback API: the person's text, then the diagnostics exactly as - the dialog previewed them (passed back, scrubbed again and bounded here).""" + """(title, text, diagnostics): the diagnostics exactly as the dialog previewed them (passed + back, scrubbed again and bounded here).""" title = ' '.join(str(body.get('title') or '').split()) text = str(body.get('message') or '').strip() if len(title) < 5: raise ValueError('give it a short title (at least 5 characters)') if len(text) < 10: raise ValueError('say a little more about what happened (at least 10 characters)') - title, text = cut(title, 120), cut(text, TEXT_MAX) diag = body.get('diagnostics') - if not isinstance(diag, str) or not diag.strip(): - return title, text - diag = cut(frame_telemetry.scrub(diag, 20000), DIAG_MAX) - return title, f'{text}{HEAD}{diag}{TAIL}' + diag = cut(frame_telemetry.scrub(diag, 40000), DIAG_MAX) if isinstance(diag, str) and diag.strip() else '' + return cut(title, 120), cut(text, TEXT_MAX), diag def send(body): - """File the report. Returns {"number", "url"} of the new issue; raises ReportError.""" + """Send the report to PostHog. Returns {"id", "message"}; raises ReportError.""" kind = body.get('kind') if body.get('kind') in KINDS else 'bug' - title, message = compose(body) - payload = {'kind': kind, 'title': title, 'message': message, 'website': '', - 'github': str(body.get('github') or '').strip().lstrip('@')[:40], - 'version': frame_telemetry.app_version(), 'os': f'{frame_host.NAME} {platform.machine()}', - 'steamos': str(frame.get('build') or '')[:120], - # How long the dialog was open; the API treats anything under 3 s as a script. - 'elapsed': max(0, int(body.get('elapsed') or 0))} - req = urllib.request.Request(FEEDBACK_URL, data=json.dumps(payload).encode(), method='POST', - headers={'content-type': 'application/json', - 'user-agent': f'FrameControl/{frame_telemetry.app_version()}'}) + title, text, diag = compose(body) + ref = uuid.uuid4().hex[:8].upper() + props = {**frame_telemetry.common(), 'kind': kind, 'title': title, 'message': text, + 'contact': str(body.get('contact') or '').strip()[:120], 'diagnostics': diag, + 'report_id': ref, 'steamos': str(frame.get('build') or '')[:120], 'level': 'report'} + # Its own random id: a report can carry contact details, so it isn't linked to this copy's analytics. + event = {'event': 'problem_report', 'distinct_id': str(uuid.uuid4()), 'uuid': str(uuid.uuid4()), + 'timestamp': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'properties': props} try: - with urllib.request.urlopen(req, timeout=30) as r: - res = json.loads(r.read() or b'{}') - except urllib.error.HTTPError as e: - try: - why = json.loads(e.read() or b'{}').get('error') - except ValueError: - why = None - e.close() - raise ReportError(why or f'the feedback service said HTTP {e.code}') - except (urllib.error.URLError, OSError, ValueError) as e: - raise ReportError(f"couldn't reach the feedback service: {e}") - if not res.get('url'): - raise ReportError("the feedback service didn't file it; try again in a moment") - frame_telemetry.capture('problem_reported', {'kind': kind, 'with_diagnostics': bool(body.get('diagnostics'))}) - return {'number': res.get('number'), 'url': res['url'], 'message': f"Sent. It's issue #{res.get('number')} on GitHub."} + frame_telemetry.post([event], timeout=30) + except frame_telemetry.SendError as e: + raise ReportError(str(e)) + try: + frame_telemetry.record_sent([event]) + except OSError: + pass # it was sent; failing to log it here mustn't make the person send it again + return {'id': ref, 'message': f'Sent privately to the Frame Control developer (report {ref}).'} class ReportError(RuntimeError): pass + + +def inbox(days=30): + """The maintainer's recent reports from PostHog, newest first (needs the personal API key + frame_compat_db.sync uses).""" + import frame_compat_db + res = frame_compat_db._posthog_query( + "SELECT timestamp, properties.report_id, properties.kind, properties.title, properties.message, " + "properties.contact, properties.app_version, properties.os, properties.steamos, properties.diagnostics " + f"FROM events WHERE event = 'problem_report' AND timestamp > now() - INTERVAL {int(days)} DAY " + "ORDER BY timestamp DESC LIMIT 200") + return res.get('results') or [] + + +def main(): + cmd, *args = sys.argv[1:] or ['inbox'] + if cmd != 'inbox': + sys.exit('usage: frame_report.py inbox [days]') + for row in inbox(*(args[:1] or [30])): + if not isinstance(row, list) or len(row) != 10: + continue + ts, ref, kind, title, text, contact, version, osname, steamos, diag = (str(v or '') for v in row) + print(f"== {ts[:16].replace('T', ' ')} {ref} [{kind}] {title}") + print(f" {version} on {osname}, SteamOS {steamos or 'unknown'}{', reply to ' + contact if contact else ''}") + print(' ' + text.replace('\n', '\n ')) + if diag: + print(' --- diagnostics\n ' + diag.replace('\n', '\n ')) + print() + + +if __name__ == '__main__': + main() diff --git a/ui/frame_telemetry.py b/ui/frame_telemetry.py index 818bf4b..5771ac7 100644 --- a/ui/frame_telemetry.py +++ b/ui/frame_telemetry.py @@ -14,8 +14,8 @@ every event and property): home folders, user names, addresses and keys. The first-run notice offers compat and diagnostics together, and the page's -Report a problem dialog (frame_report.py) files bug reports whatever is chosen -here. +Report a problem dialog (frame_report.py) sends bug reports privately to the +same project whatever is chosen here. Events are identified by a random id made on first run, not by the person or computer, and sent without person profiles or GeoIP. Nothing is sent without a @@ -53,7 +53,7 @@ SENT_KEEP = 200 OUTBOX_MAX = 2000 # events kept while offline; the oldest go first FLUSH_EVERY = 60 REPEAT_WINDOW = 600 # the same diagnostic error is sent at most once in this many seconds -DEFAULT_HOST = 'https://eu.i.posthog.com' +DEFAULT_HOST = 'https://us.i.posthog.com' LEVELS = ('usage', 'compat', 'diagnostics') # Events the page may send through /api/telemetry, and the properties each may carry. @@ -270,11 +270,12 @@ def categorize(message): # ---- capturing ------------------------------------------------------------------ -def _common(): +def common(): return {'app_version': app_version(), 'os': frame_host.NAME, 'arch': platform.machine().lower(), 'python': '%d.%d' % sys.version_info[:2], '$lib': 'frame-control', - # Anonymous events: no person profile, no location lookup. - '$process_person_profile': False, '$geoip_disable': True} + # Anonymous events: no person profile, no location lookup, and a placeholder address, + # since PostHog stores the sender's IP unless an event gives one. + '$process_person_profile': False, '$geoip_disable': True, '$ip': '0.0.0.0'} def app_version(): @@ -296,7 +297,7 @@ def capture(event, props=None, level='usage'): s = settings() e = {'event': event, 'distinct_id': s['id'], 'uuid': str(uuid.uuid4()), 'timestamp': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), - 'properties': {**_common(), **(props or {}), 'level': level}} + 'properties': {**common(), **(props or {}), 'level': level}} with _lock: lines = _read_lines(OUTBOX) + [e] _write_lines(OUTBOX, lines[-OUTBOX_MAX:]) @@ -459,28 +460,50 @@ def _drop_unwanted(s): _write_lines(OUTBOX, kept) +def post(batch, timeout=20): + """Send events to PostHog now. Raises SendError if they weren't accepted.""" + cfg = config() + if not cfg['key']: + raise SendError('no PostHog project key in this build') + for e in batch: # also events queued by versions that didn't add the placeholder address + e.setdefault('properties', {})['$ip'] = '0.0.0.0' + body = json.dumps({'api_key': cfg['key'], 'batch': batch}).encode() + req = urllib.request.Request(cfg['host'] + '/batch/', data=body, method='POST', + headers={'content-type': 'application/json', + 'user-agent': f'FrameControl/{app_version()}'}) + try: + with urllib.request.urlopen(req, timeout=timeout) as r: + r.read() + except urllib.error.HTTPError as e: + e.close() + raise SendError(f'PostHog said HTTP {e.code}') + except (urllib.error.URLError, OSError, ValueError) as e: + raise SendError(f"couldn't reach PostHog: {e}") + + +def record_sent(events): + """Add events sent outside the outbox to the log the page shows.""" + with _lock: + _write_lines(SENT, (_read_lines(SENT) + list(events))[-SENT_KEEP:]) + + +class SendError(RuntimeError): + pass + + def flush(timeout=20): """Send what's queued. Returns how many were sent; on failure they stay queued.""" with _send_lock: if blocked() or not settings()['notice_shown']: return 0 - cfg = config() with _lock: _drop_unwanted(settings()) batch = _read_lines(OUTBOX)[:100] if not batch: return 0 - body = json.dumps({'api_key': cfg['key'], 'batch': batch}).encode() - req = urllib.request.Request(cfg['host'] + '/batch/', data=body, method='POST', - headers={'content-type': 'application/json', - 'user-agent': f'FrameControl/{app_version()}'}) try: - with urllib.request.urlopen(req, timeout=timeout) as r: - r.read() - except urllib.error.HTTPError as e: - e.close() - return 0 - except (urllib.error.URLError, OSError, ValueError): + post(batch, timeout) + except SendError: return 0 sent_ids = {e['uuid'] for e in batch} with _lock: diff --git a/ui/index.html b/ui/index.html index 92ec81a..6006c4d 100644 --- a/ui/index.html +++ b/ui/index.html @@ -759,19 +759,18 @@ - - +
Show exactly what's included
Loading…
-

This becomes a public issue on GitHub (saphid/frame-control). No GitHub account is needed.

+

Sent privately to the Frame Control developer. Nothing is published.

-
@@ -2331,18 +2330,14 @@ document.querySelectorAll("nav a").forEach(a => a.addEventListener("click", () = if (!tabsSeen.has(tab)) { tabsSeen.add(tab); pageEvent("tab_viewed", { tab }); } })); -// ---- report a problem (ui/frame_report.py): a GitHub issue via the website, with diagnostics ---- -const bug = { opened: 0, preview: "" }; +// ---- report a problem (ui/frame_report.py): sent privately to PostHog, with diagnostics ---- +const bug = { preview: "" }; const activityLines = () => [...$("log").children].slice(0, 25).map(el => el.textContent.trim()); function bugReportText() { - const body = `${$("bugText").value.trim()}${bug.preview ? "\n\n---\nDiagnostics:\n```\n" + bug.preview + "\n```" : ""}`; + const contact = $("bugContact").value.trim(); + const body = `Kind: ${$("bugKind").value}${contact ? `\nContact: ${contact}` : ""}\n\n${$("bugText").value.trim()}${bug.preview ? "\n\n---\nDiagnostics:\n```\n" + bug.preview + "\n```" : ""}`; return { title: $("bugTitleIn").value.trim(), body }; } -function bugGithubUrl() { - const { title, body } = bugReportText(); - return "https://github.com/saphid/frame-control/issues/new?labels=feedback&title=" + encodeURIComponent(title) - + "&body=" + encodeURIComponent(body.slice(0, 6000)); -} // The preview is a snapshot: exactly this text is sent, even if more activity happens meanwhile. async function loadBugPreview() { if (!$("bugDiag").checked) { bug.preview = ""; $("bugDiagText").textContent = "Nothing: diagnostics are off."; return; } @@ -2353,10 +2348,9 @@ async function loadBugPreview() { } function openBugReport() { $("bugForm").reset(); - $("bugMsg").textContent = ""; $("bugGithubLink").hidden = true; $("bugSend").disabled = false; + $("bugMsg").textContent = ""; $("bugSend").disabled = false; $("bugCancel").textContent = "Cancel"; $("bugDiagBox").open = false; $("bugLogs").disabled = false; - bug.opened = performance.now(); $("bugDlg").showModal(); loadBugPreview(); } @@ -2376,14 +2370,12 @@ $("bugForm").onsubmit = async e => { try { const res = await api("/api/report", { kind: $("bugKind").value, title: $("bugTitleIn").value, message: $("bugText").value, - github: $("bugGithub").value, diagnostics: $("bugDiag").checked ? bug.preview : "", - elapsed: Math.round(performance.now() - bug.opened) }); - $("bugMsg").innerHTML = `Sent. Issue #${esc(String(res.number))} on GitHub.`; + contact: $("bugContact").value, diagnostics: $("bugDiag").checked ? bug.preview : "" }); + $("bugMsg").textContent = `Sent, thank you. Your reference is ${res.id}.`; $("bugCancel").textContent = "Close"; log(res.message, "ok"); } catch (err) { - $("bugMsg").textContent = `Couldn't send it: ${err.message}.`; - $("bugGithubLink").href = bugGithubUrl(); $("bugGithubLink").hidden = false; + $("bugMsg").textContent = `Couldn't send it: ${err.message}. Try again later, or use Copy report.`; $("bugSend").disabled = false; } }; diff --git a/ui/telemetry.json b/ui/telemetry.json index 3c89053..357c5ca 100644 --- a/ui/telemetry.json +++ b/ui/telemetry.json @@ -1,5 +1,5 @@ { - "host": "https://eu.i.posthog.com", - "key": "", - "project": "" + "host": "https://us.i.posthog.com", + "key": "phc_qkmbgQBvl2oBXGUVzfV6gG52EpmJdeaQyaRIxHRoQoL", + "project": "343535" } From 33a92a2e1d09d5abbc265e6be997e03206429cbd Mon Sep 17 00:00:00 2001 From: saphid <4596216+saphid@users.noreply.github.com> Date: Mon, 28 Sep 2026 20:20:27 +1000 Subject: [PATCH 09/51] Tests: run in a sandbox that can't touch real app data, telemetry or the shared database On a maintainer's Mac the compatibility-database key is in the Keychain, so a test that reached install reporting published fake reports. Every test module now imports tests/sandbox.py first, which points app data at a throwaway directory (new FRAME_CONTROL_DATA_DIR), turns telemetry off and sends the database nowhere. Co-Authored-By: Claude Opus 5.5 (1M context) --- tests/sandbox.py | 16 ++++++++++++++++ tests/test_compat_db.py | 1 + tests/test_connect.py | 1 + tests/test_frame_apk.py | 1 + tests/test_frame_apk_versions.py | 1 + tests/test_frame_titles.py | 1 + tests/test_server.py | 1 + tests/test_steam.py | 1 + tests/test_telemetry.py | 1 + tests/test_webinstall.py | 1 + ui/frame_host.py | 7 +++++-- 11 files changed, 30 insertions(+), 2 deletions(-) create mode 100644 tests/sandbox.py diff --git a/tests/sandbox.py b/tests/sandbox.py new file mode 100644 index 0000000..5d2a3f0 --- /dev/null +++ b/tests/sandbox.py @@ -0,0 +1,16 @@ +"""Imported first by every test module: nothing a test does reaches this person's +app data, their telemetry, or the shared compatibility database. + +Must run before any ui module is imported, since those read these at import time. +""" +import atexit +import os +import shutil +import tempfile + +_dir = tempfile.mkdtemp(prefix="frame-control-tests-") +atexit.register(shutil.rmtree, _dir, ignore_errors=True) +os.environ["FRAME_CONTROL_DATA_DIR"] = _dir +os.environ["FRAME_CONTROL_TELEMETRY"] = "0" +# A maintainer's machine holds the database key; send anything that slips through nowhere. +os.environ["FRAME_COMPAT_DB_URL"] = "http://127.0.0.1:9" diff --git a/tests/test_compat_db.py b/tests/test_compat_db.py index 7081d79..37208d5 100644 --- a/tests/test_compat_db.py +++ b/tests/test_compat_db.py @@ -2,6 +2,7 @@ Run: python3 -m unittest discover -s tests """ +import sandbox # noqa: F401 (first: keeps tests off real data and services) import os import sys import tempfile diff --git a/tests/test_connect.py b/tests/test_connect.py index b3366bd..0e8972a 100644 --- a/tests/test_connect.py +++ b/tests/test_connect.py @@ -3,6 +3,7 @@ steamos-devkit-service, the ~/.ssh/config block, and the mDNS output parsers. Run: python3 -m unittest discover -s tests """ +import sandbox # noqa: F401 (first: keeps tests off real data and services) import json import socket import sys diff --git a/tests/test_frame_apk.py b/tests/test_frame_apk.py index 1af14f9..ebe7b0b 100644 --- a/tests/test_frame_apk.py +++ b/tests/test_frame_apk.py @@ -1,4 +1,5 @@ """frame_apk against a small APK built here: binary manifest plus resource table.""" +import sandbox # noqa: F401 (first: keeps tests off real data and services) import io import os import struct diff --git a/tests/test_frame_apk_versions.py b/tests/test_frame_apk_versions.py index 508ddcf..ea70373 100644 --- a/tests/test_frame_apk_versions.py +++ b/tests/test_frame_apk_versions.py @@ -1,4 +1,5 @@ """Offline version lookup with small index-v2 fixtures.""" +import sandbox # noqa: F401 (first: keeps tests off real data and services) import io import json import os diff --git a/tests/test_frame_titles.py b/tests/test_frame_titles.py index 55d080d..acb9454 100644 --- a/tests/test_frame_titles.py +++ b/tests/test_frame_titles.py @@ -1,4 +1,5 @@ """frame_titles without a headset: executable headers, launch targets, zips, runtimes.""" +import sandbox # noqa: F401 (first: keeps tests off real data and services) import json import os import shutil diff --git a/tests/test_server.py b/tests/test_server.py index ec389bd..1bb565d 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -5,6 +5,7 @@ request guards and input validation, which all run before any SSH call. Run: python3 -m unittest discover -s tests """ +import sandbox # noqa: F401 (first: keeps tests off real data and services) import http.client import io import json diff --git a/tests/test_steam.py b/tests/test_steam.py index d29d975..615bf1e 100644 --- a/tests/test_steam.py +++ b/tests/test_steam.py @@ -2,6 +2,7 @@ Run: python3 -m unittest discover -s tests """ +import sandbox # noqa: F401 (first: keeps tests off real data and services) import json import subprocess import sys diff --git a/tests/test_telemetry.py b/tests/test_telemetry.py index 6a033c7..4b400c6 100644 --- a/tests/test_telemetry.py +++ b/tests/test_telemetry.py @@ -3,6 +3,7 @@ what's scrubbed, and that nothing is sent without a key, the notice, or consent. Run: python3 -m unittest discover -s tests """ +import sandbox # noqa: F401 (first: keeps tests off real data and services) import json import os import sys diff --git a/tests/test_webinstall.py b/tests/test_webinstall.py index 1858297..e197b49 100644 --- a/tests/test_webinstall.py +++ b/tests/test_webinstall.py @@ -4,6 +4,7 @@ the localhost-testing rule allows. Run: python3 -m unittest discover -s tests """ +import sandbox # noqa: F401 (first: keeps tests off real data and services) import hashlib import json import os diff --git a/ui/frame_host.py b/ui/frame_host.py index a60b82c..a3842f3 100644 --- a/ui/frame_host.py +++ b/ui/frame_host.py @@ -33,8 +33,11 @@ class HostError(RuntimeError): def data_dir(*parts): - """Per-user app data: ~/Library/Application Support, %APPDATA% or $XDG_DATA_HOME.""" - if MAC: + """Per-user app data: ~/Library/Application Support, %APPDATA% or $XDG_DATA_HOME + (or $FRAME_CONTROL_DATA_DIR, which the tests point at a throwaway directory).""" + if os.environ.get("FRAME_CONTROL_DATA_DIR"): + base = Path(os.environ["FRAME_CONTROL_DATA_DIR"]) + elif MAC: base = Path.home() / "Library" / "Application Support" / "Frame Control" elif WINDOWS: base = Path(os.environ.get("APPDATA") or Path.home() / "AppData" / "Roaming") / "Frame Control" From 9e4dcdd147edf95a02cccdf2f1dbb076ab3f1e7b Mon Sep 17 00:00:00 2001 From: saphid <4596216+saphid@users.noreply.github.com> Date: Mon, 28 Sep 2026 20:56:14 +1000 Subject: [PATCH 10/51] Mac in the headset: measure every frame, adapt to the network, separate windows - Per-frame timing on the Mac's clock (capture, encode, network, decode, draw), viewer clock sync and reports, input echo, /stats and a HUD. - scripts/macview-bench.py: repeatable runs on the real Frame, a shaping relay (no sudo), interleaved A/B between agent settings; results in bench/results/. - Adaptive controller: ack-based send gate with jitter-aware slack, AIMD bitrate that knows when a stream is app-limited, fps then size tiers. On a 50->3->50 Mbit/s step, scroll p95 went from 4.7 s to 72 ms; no cost on a clean link. - Separate mode: real AppKit event loop (HiDPI and NSScreen now work), cropped capture for fixed-size windows, windows kept on their display, graceful quit restores windows; stop/start races fixed. - Encoder timeline clamp (no oversized frame after a pause). - Frame Control shows each live stream's fps, delay, bitrate and tier. Reviewed by GPT-6 Astra xhigh (read-only), 7 rounds; findings fixed. Co-Authored-By: Claude Opus 5.5 (1M context) --- bench/pages/present.html | 24 + bench/pages/scroll.html | 32 + bench/pages/type.html | 9 + ...26-09-28-a3c6e5c-dirty-ab-adapt-clean.json | 4211 +++++++++++++++++ ...6-09-28-a3c6e5c-dirty-ab-adapt-clean2.json | 4150 ++++++++++++++++ ...26-09-28-a3c6e5c-dirty-ab-adapt-step3.json | 3743 +++++++++++++++ .../2026-09-28-a3c6e5c-dirty-ab-keepwarm.json | 2172 +++++++++ .../2026-09-28-a3c6e5c-dirty-adapt-clean.json | 984 ++++ ...2026-09-28-a3c6e5c-dirty-adapt-clean2.json | 1116 +++++ ...26-09-28-a3c6e5c-dirty-baseline-fixed.json | 1124 +++++ .../2026-09-28-a3c6e5c-dirty-baseline.json | 1124 +++++ docs/how-the-frame-works.md | 1 + docs/mac-in-headset.md | 161 +- docs/mac-window-separation.md | 200 + mac/frame-mac-view/Sources/CGVirtualDisplay.h | 28 + mac/frame-mac-view/Sources/Capture.swift | 103 +- mac/frame-mac-view/Sources/Controller.swift | 234 + mac/frame-mac-view/Sources/Encoder.swift | 73 +- mac/frame-mac-view/Sources/Separate.swift | 295 ++ mac/frame-mac-view/Sources/Server.swift | 6 +- mac/frame-mac-view/Sources/Stats.swift | 180 + mac/frame-mac-view/Sources/main.swift | 321 +- mac/frame-mac-view/build.sh | 1 + mac/frame-mac-view/lab.sh | 50 + scripts/macview-bench.py | 815 ++++ tests/test_macview.py | 80 +- ui/frame_macview.py | 25 +- ui/index.html | 39 +- ui/mac-view.html | 208 +- 29 files changed, 21405 insertions(+), 104 deletions(-) create mode 100644 bench/pages/present.html create mode 100644 bench/pages/scroll.html create mode 100644 bench/pages/type.html create mode 100644 bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-clean.json create mode 100644 bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-clean2.json create mode 100644 bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-step3.json create mode 100644 bench/results/2026-09-28-a3c6e5c-dirty-ab-keepwarm.json create mode 100644 bench/results/2026-09-28-a3c6e5c-dirty-adapt-clean.json create mode 100644 bench/results/2026-09-28-a3c6e5c-dirty-adapt-clean2.json create mode 100644 bench/results/2026-09-28-a3c6e5c-dirty-baseline-fixed.json create mode 100644 bench/results/2026-09-28-a3c6e5c-dirty-baseline.json create mode 100644 docs/mac-window-separation.md create mode 100644 mac/frame-mac-view/Sources/CGVirtualDisplay.h create mode 100644 mac/frame-mac-view/Sources/Controller.swift create mode 100644 mac/frame-mac-view/Sources/Separate.swift create mode 100644 mac/frame-mac-view/Sources/Stats.swift create mode 100755 mac/frame-mac-view/lab.sh create mode 100755 scripts/macview-bench.py diff --git a/bench/pages/present.html b/bench/pages/present.html new file mode 100644 index 0000000..2e783fd --- /dev/null +++ b/bench/pages/present.html @@ -0,0 +1,24 @@ + + +fmv-present + + diff --git a/bench/pages/scroll.html b/bench/pages/scroll.html new file mode 100644 index 0000000..602ced9 --- /dev/null +++ b/bench/pages/scroll.html @@ -0,0 +1,32 @@ + + +fmv-bench scroll +
+ diff --git a/bench/pages/type.html b/bench/pages/type.html new file mode 100644 index 0000000..288904c --- /dev/null +++ b/bench/pages/type.html @@ -0,0 +1,9 @@ + + +fmv-bench type + + diff --git a/bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-clean.json b/bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-clean.json new file mode 100644 index 0000000..0b1f835 --- /dev/null +++ b/bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-clean.json @@ -0,0 +1,4211 @@ +{ + "label": "ab-adapt-clean", + "date": "2026-09-28T17:50:22", + "commit": "a3c6e5c+dirty", + "config": { + "quality": "balanced", + "mode": "separate", + "net": "none", + "delay_ms": 0, + "buffer_ms": 250, + "host": "frame", + "ssh_opts": [], + "encoder": "", + "duration_s": 15.0, + "browser_flags": [] + }, + "frame_build": "20260925.6191901", + "mac": "26.5.2", + "headset": ":35.258683 [Info] - 0 - entering standby", + "scenarios": [], + "arms": [ + "off:FRAME_MAC_VIEW_ADAPT=0", + "on:" + ], + "runs": { + "off": [ + [ + { + "scenario": "test", + "frames_sent": 944, + "frames_drawn": 944, + "frames_shown": 648, + "duration_s": 15.7, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 944 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 944 + }, + "encode": { + "p50": 4.3, + "p95": 5.0, + "p99": 5.2, + "n": 944 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 944 + }, + "network": { + "p50": 5.6, + "p95": 9.6, + "p99": 12.9, + "n": 944 + }, + "decode": { + "p50": 1.0, + "p95": 1.7, + "p99": 4.2, + "n": 944 + }, + "draw": { + "p50": 0.3, + "p95": 0.5, + "p99": 0.9, + "n": 944 + }, + "present": { + "p50": 4.5, + "p95": 15.8, + "p99": 18.4, + "n": 648 + }, + "content": { + "p50": 11.5, + "p95": 15.8, + "p99": 19.3, + "n": 944 + }, + "content_shown": { + "p50": 17.0, + "p95": 28.4, + "p99": 31.5, + "n": 648 + } + }, + "fps": 60.1, + "fps_shown": 41.2, + "late_pct": 0.95, + "stall_max": 68.8, + "stalls_over_100ms": 0, + "mbps": 0.48, + "keyframes": 1, + "size": "1280x720", + "captured": 1034, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 28.8, + "p95": 38.1, + "p99": 45.1, + "n": 30 + }, + "input_shown_ms": { + "p50": 30.2, + "p95": 45.6, + "p99": 50.1, + "n": 19 + }, + "input_parts_ms": { + "uplink": { + "p50": 7.1, + "p95": 21.5, + "p99": 23.6, + "n": 30 + }, + "mac": { + "p50": 4.2, + "p95": 15.1, + "p99": 15.8, + "n": 30 + }, + "back": { + "p50": 12.2, + "p95": 16.3, + "p99": 16.9, + "n": 30 + } + }, + "inputs": { + "sent": 30, + "seen": 30 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.51, + "content_p50": 10.9, + "content_p95": 16.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.49, + "content_p50": 10.2, + "content_p95": 15.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.5, + "content_p50": 10.2, + "content_p95": 13.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.51, + "content_p50": 10.3, + "content_p95": 15.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 60, + "mbps": 0.5, + "content_p50": 10.4, + "content_p95": 14.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.49, + "content_p50": 10.7, + "content_p95": 14.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 60, + "mbps": 0.59, + "content_p50": 10.7, + "content_p95": 13.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.1, + "content_p95": 11.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.4, + "content_p95": 12.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 60, + "mbps": 0.45, + "content_p50": 10.6, + "content_p95": 14.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.44, + "content_p50": 13.8, + "content_p95": 16.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 60, + "mbps": 0.44, + "content_p50": 14.3, + "content_p95": 18.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 60, + "mbps": 0.45, + "content_p50": 13.7, + "content_p95": 15.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 60, + "mbps": 0.45, + "content_p50": 14.1, + "content_p95": 17.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.1, + "content_p95": 18.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 44, + "mbps": 0.32, + "content_p50": 12.1, + "content_p95": 13.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 11.5, + "content_p95": 15.8, + "input_p50": 28.8, + "input_p95": 38.1, + "grades": { + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "local", + "stall_max": "local" + }, + "controller": { + "adapt": false, + "baseRtt": 5.833, + "ceiling": 5529600, + "fps": 60, + "inFlight": 2, + "scale": 1, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 65.9, + "cpu": { + "mac_agent_pct": 16.1, + "frame_viewer_pct": 66.4, + "frame_tailscaled_pct": 5.8, + "frame_sshd_pct": 0.6, + "frame_gamescope_pct": 28.1, + "frame_total_pct": 31.6 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 142 Hz", + "show_s": 1.52, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 742, + "frames_drawn": 742, + "frames_shown": 603, + "duration_s": 15.8, + "stages_ms": { + "capture": { + "p50": -3.8, + "p95": 12.1, + "p99": 22.3, + "n": 742 + }, + "queue": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.1, + "n": 742 + }, + "encode": { + "p50": 6.7, + "p95": 9.0, + "p99": 16.4, + "n": 742 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.5, + "n": 742 + }, + "network": { + "p50": 8.1, + "p95": 24.9, + "p99": 61.3, + "n": 742 + }, + "decode": { + "p50": 6.0, + "p95": 12.4, + "p99": 26.2, + "n": 742 + }, + "draw": { + "p50": 0.6, + "p95": 1.2, + "p99": 1.9, + "n": 742 + }, + "present": { + "p50": 2.1, + "p95": 19.6, + "p99": 24.7, + "n": 603 + }, + "content": { + "p50": 19.5, + "p95": 44.4, + "p99": 118.6, + "n": 742 + }, + "content_shown": { + "p50": 25.0, + "p95": 51.3, + "p99": 125.2, + "n": 603 + } + }, + "fps": 47.1, + "fps_shown": 38.3, + "late_pct": 20.92, + "stall_max": 128.0, + "stalls_over_100ms": 2, + "mbps": 9.15, + "keyframes": 1, + "size": "1920x1290", + "captured": 815, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 40, + "mbps": 9.15, + "content_p50": 28.7, + "content_p95": 139.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 36, + "mbps": 7.35, + "content_p50": 23.6, + "content_p95": 37.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 44, + "mbps": 9.92, + "content_p50": 21.9, + "content_p95": 39.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 53, + "mbps": 9.98, + "content_p50": 20.4, + "content_p95": 37.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 45, + "mbps": 9.58, + "content_p50": 20.2, + "content_p95": 41.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 53, + "mbps": 10.27, + "content_p50": 19.2, + "content_p95": 37.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 48, + "mbps": 11.11, + "content_p50": 29.9, + "content_p95": 107.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 44, + "mbps": 8.39, + "content_p50": 24.1, + "content_p95": 34.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 49, + "mbps": 8.68, + "content_p50": 14.5, + "content_p95": 39.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 43, + "mbps": 8.08, + "content_p50": 16.6, + "content_p95": 36.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 53, + "mbps": 8.85, + "content_p50": 15.1, + "content_p95": 25.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 50, + "mbps": 8.82, + "content_p50": 20.1, + "content_p95": 35.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 54, + "mbps": 8.42, + "content_p50": 18.9, + "content_p95": 26.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 52, + "mbps": 9.27, + "content_p50": 12.8, + "content_p95": 28.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 47, + "mbps": 9.89, + "content_p50": 21.5, + "content_p95": 42.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 31, + "mbps": 6.44, + "content_p50": 21.6, + "content_p95": 39.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 19.5, + "content_p95": 44.4, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "controller": { + "adapt": false, + "baseRtt": 7.541, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 51.6, + "cpu": { + "mac_agent_pct": 7.1, + "frame_viewer_pct": 80.9, + "frame_tailscaled_pct": 7.9, + "frame_sshd_pct": 0.9, + "frame_gamescope_pct": 25.3, + "frame_total_pct": 31.1 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 142 Hz", + "show_s": 1.68, + "panel": "valve.steam.desktopgame.2001300498", + "src": "separate:13051" + } + ], + [ + { + "scenario": "test", + "frames_sent": 944, + "frames_drawn": 944, + "frames_shown": 608, + "duration_s": 15.7, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 944 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 944 + }, + "encode": { + "p50": 4.0, + "p95": 4.7, + "p99": 5.2, + "n": 944 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 944 + }, + "network": { + "p50": 4.8, + "p95": 40.4, + "p99": 74.2, + "n": 944 + }, + "decode": { + "p50": 1.1, + "p95": 5.4, + "p99": 10.8, + "n": 944 + }, + "draw": { + "p50": 0.3, + "p95": 0.5, + "p99": 0.9, + "n": 944 + }, + "present": { + "p50": 4.9, + "p95": 17.7, + "p99": 22.2, + "n": 608 + }, + "content": { + "p50": 10.5, + "p95": 52.3, + "p99": 82.1, + "n": 944 + }, + "content_shown": { + "p50": 16.0, + "p95": 33.4, + "p99": 82.8, + "n": 608 + } + }, + "fps": 60.1, + "fps_shown": 38.7, + "late_pct": 3.61, + "stall_max": 120.1, + "stalls_over_100ms": 2, + "mbps": 0.47, + "keyframes": 1, + "size": "1280x720", + "captured": 1034, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 26.7, + "p95": 89.3, + "p99": 106.5, + "n": 30 + }, + "input_shown_ms": { + "p50": 29.4, + "p95": 44.4, + "p99": 106.9, + "n": 18 + }, + "input_parts_ms": { + "uplink": { + "p50": 9.2, + "p95": 38.3, + "p99": 70.5, + "n": 30 + }, + "mac": { + "p50": 5.2, + "p95": 14.1, + "p99": 18.3, + "n": 30 + }, + "back": { + "p50": 10.4, + "p95": 16.2, + "p99": 86.2, + "n": 30 + } + }, + "inputs": { + "sent": 30, + "seen": 30 + }, + "timeline": [ + { + "t": 0, + "fps": 61, + "mbps": 0.48, + "content_p50": 10.3, + "content_p95": 14.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 59, + "mbps": 0.46, + "content_p50": 10.4, + "content_p95": 13.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.8, + "content_p95": 11.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.7, + "content_p95": 11.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 61, + "mbps": 0.48, + "content_p50": 9.6, + "content_p95": 11.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 59, + "mbps": 0.46, + "content_p50": 9.6, + "content_p95": 11.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 61, + "mbps": 0.57, + "content_p50": 9.9, + "content_p95": 34.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 59, + "mbps": 0.45, + "content_p50": 9.7, + "content_p95": 11.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 61, + "mbps": 0.47, + "content_p50": 9.8, + "content_p95": 13.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 59, + "mbps": 0.46, + "content_p50": 10.5, + "content_p95": 40.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.44, + "content_p50": 11.8, + "content_p95": 82.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.4, + "content_p95": 68.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 61, + "mbps": 0.46, + "content_p50": 12.4, + "content_p95": 71.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 59, + "mbps": 0.44, + "content_p50": 11.8, + "content_p95": 70.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 61, + "mbps": 0.47, + "content_p50": 11.8, + "content_p95": 70.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 43, + "mbps": 0.32, + "content_p50": 12.1, + "content_p95": 69.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 10.5, + "content_p95": 52.3, + "input_p50": 26.7, + "input_p95": 89.3, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "input_p50": "local", + "input_p95": "acceptable", + "fps": "local", + "late_pct": "acceptable", + "stall_max": "acceptable" + }, + "controller": { + "adapt": false, + "baseRtt": 5.458, + "ceiling": 5529600, + "fps": 60, + "inFlight": 5, + "scale": 1, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 65.9, + "cpu": { + "mac_agent_pct": 14.0, + "frame_viewer_pct": 66.0, + "frame_tailscaled_pct": 5.1, + "frame_sshd_pct": 0.6, + "frame_gamescope_pct": 28.5, + "frame_total_pct": 32.3 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 135 Hz", + "show_s": 1.73, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 823, + "frames_drawn": 823, + "frames_shown": 523, + "duration_s": 15.6, + "stages_ms": { + "capture": { + "p50": -4.4, + "p95": 5.1, + "p99": 12.6, + "n": 823 + }, + "queue": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.1, + "n": 823 + }, + "encode": { + "p50": 6.6, + "p95": 7.8, + "p99": 10.5, + "n": 823 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 823 + }, + "network": { + "p50": 16.2, + "p95": 90.9, + "p99": 145.2, + "n": 823 + }, + "decode": { + "p50": 8.7, + "p95": 25.2, + "p99": 35.5, + "n": 823 + }, + "draw": { + "p50": 0.6, + "p95": 0.9, + "p99": 1.7, + "n": 823 + }, + "present": { + "p50": 1.9, + "p95": 20.9, + "p99": 26.5, + "n": 523 + }, + "content": { + "p50": 33.3, + "p95": 108.6, + "p99": 167.3, + "n": 823 + }, + "content_shown": { + "p50": 35.0, + "p95": 106.8, + "p99": 141.4, + "n": 523 + } + }, + "fps": 52.6, + "fps_shown": 33.4, + "late_pct": 14.84, + "stall_max": 232.0, + "stalls_over_100ms": 32, + "mbps": 9.35, + "keyframes": 1, + "size": "1920x1290", + "captured": 895, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 52, + "mbps": 8.9, + "content_p50": 30.2, + "content_p95": 85.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 53, + "mbps": 9.82, + "content_p50": 32.6, + "content_p95": 106.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 53, + "mbps": 9.45, + "content_p50": 31.9, + "content_p95": 96.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 56, + "mbps": 9.42, + "content_p50": 37.6, + "content_p95": 99.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 57, + "mbps": 9.26, + "content_p50": 38.2, + "content_p95": 97.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 57, + "mbps": 11.43, + "content_p50": 56.3, + "content_p95": 144.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 50, + "mbps": 8.55, + "content_p50": 27.0, + "content_p95": 104.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 52, + "mbps": 9.63, + "content_p50": 37.1, + "content_p95": 99.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 46, + "mbps": 8.79, + "content_p50": 31.2, + "content_p95": 115.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 55, + "mbps": 9.43, + "content_p50": 29.4, + "content_p95": 97.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 50, + "mbps": 9.29, + "content_p50": 29.8, + "content_p95": 93.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 55, + "mbps": 9.78, + "content_p50": 27.0, + "content_p95": 98.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 52, + "mbps": 9.19, + "content_p50": 53.0, + "content_p95": 116.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 55, + "mbps": 9.33, + "content_p50": 45.5, + "content_p95": 102.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 45, + "mbps": 7.78, + "content_p50": 28.6, + "content_p95": 102.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 35, + "mbps": 6.17, + "content_p50": 62.5, + "content_p95": 211.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 33.3, + "content_p95": 108.6, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "acceptable", + "content_p95": "bad", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "controller": { + "adapt": false, + "baseRtt": 7.791, + "ceiling": 14860800, + "fps": 60, + "inFlight": 0, + "scale": 1, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 57.4, + "cpu": { + "mac_agent_pct": 7.6, + "frame_viewer_pct": 83.9, + "frame_tailscaled_pct": 7.8, + "frame_sshd_pct": 0.9, + "frame_gamescope_pct": 26.7, + "frame_total_pct": 32.6 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 135 Hz", + "show_s": 1.66, + "panel": "valve.steam.desktopgame.2001667691", + "src": "separate:13436" + } + ], + [ + { + "scenario": "test", + "frames_sent": 943, + "frames_drawn": 943, + "frames_shown": 631, + "duration_s": 15.7, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 943 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 943 + }, + "encode": { + "p50": 4.2, + "p95": 4.8, + "p99": 5.1, + "n": 943 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.3, + "n": 943 + }, + "network": { + "p50": 6.4, + "p95": 23.1, + "p99": 64.7, + "n": 943 + }, + "decode": { + "p50": 1.0, + "p95": 2.8, + "p99": 6.3, + "n": 943 + }, + "draw": { + "p50": 0.3, + "p95": 0.5, + "p99": 0.9, + "n": 943 + }, + "present": { + "p50": 5.2, + "p95": 15.7, + "p99": 21.4, + "n": 631 + }, + "content": { + "p50": 12.2, + "p95": 29.7, + "p99": 72.2, + "n": 943 + }, + "content_shown": { + "p50": 18.4, + "p95": 32.7, + "p99": 75.5, + "n": 631 + } + }, + "fps": 60.1, + "fps_shown": 40.2, + "late_pct": 5.73, + "stall_max": 123.4, + "stalls_over_100ms": 1, + "mbps": 0.48, + "keyframes": 1, + "size": "1280x720", + "captured": 1034, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 31.4, + "p95": 78.1, + "p99": 187.4, + "n": 30 + }, + "input_shown_ms": { + "p50": 38.8, + "p95": 56.9, + "p99": 62.9, + "n": 20 + }, + "input_parts_ms": { + "uplink": { + "p50": 11.4, + "p95": 40.8, + "p99": 170.5, + "n": 30 + }, + "mac": { + "p50": 7.0, + "p95": 15.0, + "p99": 16.1, + "n": 30 + }, + "back": { + "p50": 11.6, + "p95": 18.8, + "p99": 29.4, + "n": 30 + } + }, + "inputs": { + "sent": 30, + "seen": 30 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.51, + "content_p50": 9.3, + "content_p95": 53.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.49, + "content_p50": 9.3, + "content_p95": 51.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.51, + "content_p50": 22.3, + "content_p95": 86.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.51, + "content_p50": 16.9, + "content_p95": 65.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 60, + "mbps": 0.47, + "content_p50": 11.6, + "content_p95": 37.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.59, + "content_p50": 12.3, + "content_p95": 16.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 60, + "mbps": 0.49, + "content_p50": 12.6, + "content_p95": 16.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.9, + "content_p95": 15.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.49, + "content_p50": 11.8, + "content_p95": 17.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 60, + "mbps": 0.46, + "content_p50": 11.8, + "content_p95": 19.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.46, + "content_p50": 12.1, + "content_p95": 15.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.6, + "content_p95": 18.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 60, + "mbps": 0.46, + "content_p50": 12.5, + "content_p95": 17.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 60, + "mbps": 0.46, + "content_p50": 13.0, + "content_p95": 18.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 60, + "mbps": 0.46, + "content_p50": 14.0, + "content_p95": 22.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 43, + "mbps": 0.3, + "content_p50": 13.6, + "content_p95": 22.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 12.2, + "content_p95": 29.7, + "input_p50": 31.4, + "input_p95": 78.1, + "grades": { + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "acceptable", + "fps": "local", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "controller": { + "adapt": false, + "baseRtt": 5.125, + "ceiling": 5529600, + "fps": 60, + "inFlight": 2, + "scale": 1, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 65.9, + "cpu": { + "mac_agent_pct": 15.5, + "frame_viewer_pct": 67.2, + "frame_tailscaled_pct": 22.5, + "frame_sshd_pct": 2.6, + "frame_gamescope_pct": 28.6, + "frame_total_pct": 38.9 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 132 Hz", + "show_s": 1.52, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 624, + "frames_drawn": 622, + "frames_shown": 522, + "duration_s": 15.7, + "stages_ms": { + "capture": { + "p50": -2.0, + "p95": 13.8, + "p99": 34.2, + "n": 624 + }, + "queue": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.2, + "n": 624 + }, + "encode": { + "p50": 6.5, + "p95": 8.1, + "p99": 13.7, + "n": 624 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.5, + "n": 624 + }, + "network": { + "p50": 7.5, + "p95": 29.1, + "p99": 83.6, + "n": 624 + }, + "decode": { + "p50": 7.6, + "p95": 83.1, + "p99": 242.3, + "n": 622 + }, + "draw": { + "p50": 0.8, + "p95": 2.2, + "p99": 5.1, + "n": 622 + }, + "present": { + "p50": 1.5, + "p95": 19.9, + "p99": 25.1, + "n": 522 + }, + "content": { + "p50": 25.3, + "p95": 110.6, + "p99": 326.3, + "n": 622 + }, + "content_shown": { + "p50": 30.3, + "p95": 91.4, + "p99": 143.8, + "n": 522 + } + }, + "fps": 39.5, + "fps_shown": 33.2, + "late_pct": 33.66, + "stall_max": 362.0, + "stalls_over_100ms": 5, + "mbps": 8.59, + "keyframes": 2, + "size": "1920x1290", + "captured": 678, + "viewer_never_drawn": 2, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 42, + "mbps": 9.07, + "content_p50": 24.3, + "content_p95": 39.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 42, + "mbps": 9.11, + "content_p50": 27.3, + "content_p95": 76.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 43, + "mbps": 8.81, + "content_p50": 25.6, + "content_p95": 96.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 44, + "mbps": 8.11, + "content_p50": 46.5, + "content_p95": 89.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 37, + "mbps": 8.52, + "content_p50": 19.4, + "content_p95": 49.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 43, + "mbps": 9.51, + "content_p50": 28.1, + "content_p95": 92.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 35, + "mbps": 10.84, + "content_p50": 24.8, + "content_p95": 61.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 25, + "mbps": 6.13, + "content_p50": 21.8, + "content_p95": 62.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 52, + "mbps": 9.53, + "content_p50": 17.0, + "content_p95": 29.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 37, + "mbps": 8.18, + "content_p50": 35.9, + "content_p95": 78.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 21, + "mbps": 5.48, + "content_p50": 44.5, + "content_p95": 350.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 34, + "mbps": 7.4, + "content_p50": 27.0, + "content_p95": 54.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 54, + "mbps": 8.87, + "content_p50": 15.4, + "content_p95": 25.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 43, + "mbps": 10.59, + "content_p50": 131.1, + "content_p95": 322.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 43, + "mbps": 9.22, + "content_p50": 23.9, + "content_p95": 42.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 27, + "mbps": 5.81, + "content_p50": 27.6, + "content_p95": 53.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 25.3, + "content_p95": 110.6, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "acceptable", + "content_p95": "bad", + "fps": "bad", + "late_pct": "bad", + "stall_max": "bad" + }, + "controller": { + "adapt": false, + "baseRtt": 7.583, + "ceiling": 14860800, + "fps": 60, + "inFlight": 0, + "scale": 1, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 43.2, + "cpu": { + "mac_agent_pct": 5.8, + "frame_viewer_pct": 86.7, + "frame_tailscaled_pct": 9.9, + "frame_sshd_pct": 1.0, + "frame_gamescope_pct": 25.0, + "frame_total_pct": 77.7 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 141 Hz", + "show_s": 1.54, + "panel": "valve.steam.desktopgame.2001796574", + "src": "separate:13568" + } + ] + ], + "on": [ + [ + { + "scenario": "test", + "frames_sent": 843, + "frames_drawn": 843, + "frames_shown": 580, + "duration_s": 15.7, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 843 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 12.6, + "n": 843 + }, + "encode": { + "p50": 4.2, + "p95": 4.8, + "p99": 5.1, + "n": 843 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.3, + "n": 843 + }, + "network": { + "p50": 4.8, + "p95": 66.0, + "p99": 82.6, + "n": 843 + }, + "decode": { + "p50": 1.1, + "p95": 4.1, + "p99": 6.3, + "n": 843 + }, + "draw": { + "p50": 0.3, + "p95": 0.6, + "p99": 1.0, + "n": 843 + }, + "present": { + "p50": 4.0, + "p95": 17.6, + "p99": 23.5, + "n": 580 + }, + "content": { + "p50": 10.9, + "p95": 75.0, + "p99": 91.6, + "n": 843 + }, + "content_shown": { + "p50": 16.5, + "p95": 81.2, + "p99": 93.9, + "n": 580 + } + }, + "fps": 53.6, + "fps_shown": 36.9, + "late_pct": 6.06, + "stall_max": 127.2, + "stalls_over_100ms": 2, + "mbps": 0.43, + "keyframes": 1, + "size": "1280x720", + "captured": 1037, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 27.2, + "p95": 95.5, + "p99": 107.8, + "n": 30 + }, + "input_shown_ms": { + "p50": 32.1, + "p95": 95.9, + "p99": 108.5, + "n": 19 + }, + "input_parts_ms": { + "uplink": { + "p50": 8.0, + "p95": 44.4, + "p99": 44.8, + "n": 30 + }, + "mac": { + "p50": 5.8, + "p95": 13.7, + "p99": 17.9, + "n": 30 + }, + "back": { + "p50": 11.2, + "p95": 84.3, + "p99": 85.8, + "n": 30 + } + }, + "inputs": { + "sent": 30, + "seen": 30 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.1, + "content_p95": 13.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.1, + "content_p95": 11.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.48, + "content_p50": 9.6, + "content_p95": 12.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.0, + "content_p95": 12.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 59, + "mbps": 0.46, + "content_p50": 9.9, + "content_p95": 35.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 54, + "mbps": 0.43, + "content_p50": 9.8, + "content_p95": 51.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 52, + "mbps": 0.52, + "content_p50": 10.6, + "content_p95": 68.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 50, + "mbps": 0.39, + "content_p50": 10.6, + "content_p95": 73.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 47, + "mbps": 0.39, + "content_p50": 11.2, + "content_p95": 80.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 52, + "mbps": 0.41, + "content_p50": 11.2, + "content_p95": 79.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 52, + "mbps": 0.4, + "content_p50": 10.5, + "content_p95": 75.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 51, + "mbps": 0.4, + "content_p50": 13.4, + "content_p95": 75.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 50, + "mbps": 0.4, + "content_p50": 14.7, + "content_p95": 77.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 45, + "mbps": 0.37, + "content_p50": 15.3, + "content_p95": 102.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 52, + "mbps": 0.4, + "content_p50": 15.6, + "content_p95": 80.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 39, + "mbps": 0.3, + "content_p50": 14.7, + "content_p95": 80.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 10.9, + "content_p95": 75.0, + "input_p50": 27.2, + "input_p95": 95.5, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "input_p50": "local", + "input_p95": "acceptable", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "controller": { + "adapt": true, + "baseRtt": 5.875, + "ceiling": 5529600, + "fps": 60, + "inFlight": 0, + "scale": 1, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 66.1, + "cpu": { + "mac_agent_pct": 14.9, + "frame_viewer_pct": 58.7, + "frame_tailscaled_pct": 4.9, + "frame_sshd_pct": 0.6, + "frame_gamescope_pct": 25.8, + "frame_total_pct": 28.7 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz", + "show_s": 1.32, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 756, + "frames_drawn": 756, + "frames_shown": 606, + "duration_s": 15.6, + "stages_ms": { + "capture": { + "p50": -7.9, + "p95": 7.1, + "p99": 14.9, + "n": 756 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 7.8, + "n": 756 + }, + "encode": { + "p50": 6.8, + "p95": 9.0, + "p99": 12.8, + "n": 756 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 756 + }, + "network": { + "p50": 7.1, + "p95": 27.0, + "p99": 45.8, + "n": 756 + }, + "decode": { + "p50": 5.9, + "p95": 11.9, + "p99": 20.9, + "n": 756 + }, + "draw": { + "p50": 0.6, + "p95": 0.8, + "p99": 1.6, + "n": 756 + }, + "present": { + "p50": 3.3, + "p95": 20.5, + "p99": 25.9, + "n": 606 + }, + "content": { + "p50": 16.6, + "p95": 45.7, + "p99": 64.8, + "n": 756 + }, + "content_shown": { + "p50": 22.8, + "p95": 50.9, + "p99": 68.4, + "n": 606 + } + }, + "fps": 48.4, + "fps_shown": 38.8, + "late_pct": 21.06, + "stall_max": 88.2, + "stalls_over_100ms": 0, + "mbps": 9.16, + "keyframes": 1, + "size": "1920x1290", + "captured": 841, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 50, + "mbps": 8.74, + "content_p50": 14.0, + "content_p95": 30.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 45, + "mbps": 8.64, + "content_p50": 22.9, + "content_p95": 49.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 47, + "mbps": 8.91, + "content_p50": 20.8, + "content_p95": 45.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 46, + "mbps": 9.41, + "content_p50": 18.6, + "content_p95": 53.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 49, + "mbps": 8.61, + "content_p50": 13.0, + "content_p95": 25.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 53, + "mbps": 11.17, + "content_p50": 11.7, + "content_p95": 51.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 51, + "mbps": 9.1, + "content_p50": 12.4, + "content_p95": 27.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 48, + "mbps": 9.45, + "content_p50": 19.5, + "content_p95": 56.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 55, + "mbps": 9.21, + "content_p50": 11.9, + "content_p95": 30.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 50, + "mbps": 8.55, + "content_p50": 23.1, + "content_p95": 63.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 49, + "mbps": 8.94, + "content_p50": 19.5, + "content_p95": 36.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 54, + "mbps": 9.51, + "content_p50": 16.0, + "content_p95": 31.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 41, + "mbps": 8.36, + "content_p50": 21.5, + "content_p95": 45.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 37, + "mbps": 8.55, + "content_p50": 22.4, + "content_p95": 51.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 48, + "mbps": 10.2, + "content_p50": 19.6, + "content_p95": 34.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 33, + "mbps": 5.73, + "content_p50": 10.7, + "content_p95": 54.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 16.6, + "content_p95": 45.7, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "local" + }, + "controller": { + "adapt": true, + "baseRtt": 7.041, + "ceiling": 14860800, + "fps": 60, + "inFlight": 0, + "scale": 1, + "target": 14860800, + "tier": 0 + }, + "events": [ + { + "t": 0.19, + "e": "tier 0" + } + ], + "source_fps": 53.9, + "cpu": { + "mac_agent_pct": 7.7, + "frame_viewer_pct": 83.2, + "frame_tailscaled_pct": 9.0, + "frame_sshd_pct": 1.0, + "frame_gamescope_pct": 25.4, + "frame_total_pct": 32.3 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 142 Hz", + "show_s": 1.54, + "panel": "valve.steam.desktopgame.2001527998", + "src": "separate:13175" + } + ], + [ + { + "scenario": "test", + "frames_sent": 932, + "frames_drawn": 932, + "frames_shown": 637, + "duration_s": 15.7, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 932 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.2, + "n": 932 + }, + "encode": { + "p50": 4.2, + "p95": 4.9, + "p99": 5.2, + "n": 932 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 932 + }, + "network": { + "p50": 4.6, + "p95": 9.1, + "p99": 12.6, + "n": 932 + }, + "decode": { + "p50": 1.1, + "p95": 1.6, + "p99": 4.1, + "n": 932 + }, + "draw": { + "p50": 0.3, + "p95": 0.5, + "p99": 0.9, + "n": 932 + }, + "present": { + "p50": 5.0, + "p95": 15.8, + "p99": 19.3, + "n": 637 + }, + "content": { + "p50": 10.5, + "p95": 15.4, + "p99": 19.0, + "n": 932 + }, + "content_shown": { + "p50": 16.1, + "p95": 28.2, + "p99": 33.7, + "n": 637 + } + }, + "fps": 59.4, + "fps_shown": 40.6, + "late_pct": 0.75, + "stall_max": 126.0, + "stalls_over_100ms": 1, + "mbps": 0.48, + "keyframes": 1, + "size": "1280x720", + "captured": 1031, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 24.6, + "p95": 39.8, + "p99": 45.9, + "n": 30 + }, + "input_shown_ms": { + "p50": 34.5, + "p95": 41.8, + "p99": 58.5, + "n": 24 + }, + "input_parts_ms": { + "uplink": { + "p50": 5.1, + "p95": 19.2, + "p99": 21.0, + "n": 30 + }, + "mac": { + "p50": 5.2, + "p95": 14.8, + "p99": 16.1, + "n": 30 + }, + "back": { + "p50": 10.7, + "p95": 14.8, + "p99": 14.9, + "n": 30 + } + }, + "inputs": { + "sent": 30, + "seen": 30 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.49, + "content_p50": 9.2, + "content_p95": 10.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 59, + "mbps": 0.45, + "content_p50": 9.5, + "content_p95": 12.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 61, + "mbps": 0.47, + "content_p50": 9.8, + "content_p95": 11.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 53, + "mbps": 0.45, + "content_p50": 10.0, + "content_p95": 13.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 60, + "mbps": 0.48, + "content_p50": 9.8, + "content_p95": 12.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 57, + "mbps": 0.47, + "content_p50": 10.7, + "content_p95": 14.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 60, + "mbps": 0.58, + "content_p50": 10.3, + "content_p95": 12.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 60, + "mbps": 0.49, + "content_p50": 9.8, + "content_p95": 12.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.49, + "content_p50": 9.9, + "content_p95": 12.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 59, + "mbps": 0.45, + "content_p50": 10.1, + "content_p95": 12.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 61, + "mbps": 0.48, + "content_p50": 10.2, + "content_p95": 12.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 59, + "mbps": 0.45, + "content_p50": 10.6, + "content_p95": 14.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 61, + "mbps": 0.47, + "content_p50": 14.3, + "content_p95": 17.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 60, + "mbps": 0.47, + "content_p50": 14.1, + "content_p95": 16.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 60, + "mbps": 0.45, + "content_p50": 13.8, + "content_p95": 15.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 42, + "mbps": 0.32, + "content_p50": 13.8, + "content_p95": 18.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 10.5, + "content_p95": 15.4, + "input_p50": 24.6, + "input_p95": 39.8, + "grades": { + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "local", + "stall_max": "acceptable" + }, + "controller": { + "adapt": true, + "baseRtt": 5.459, + "ceiling": 5529600, + "fps": 60, + "inFlight": 2, + "scale": 1, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 65.7, + "cpu": { + "mac_agent_pct": 15.9, + "frame_viewer_pct": 66.0, + "frame_tailscaled_pct": 5.4, + "frame_sshd_pct": 0.8, + "frame_gamescope_pct": 28.8, + "frame_total_pct": 32.1 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 141 Hz", + "show_s": 1.49, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 604, + "frames_drawn": 604, + "frames_shown": 496, + "duration_s": 15.8, + "stages_ms": { + "capture": { + "p50": -3.6, + "p95": 11.2, + "p99": 19.8, + "n": 604 + }, + "queue": { + "p50": 0.0, + "p95": 15.4, + "p99": 23.5, + "n": 604 + }, + "encode": { + "p50": 6.8, + "p95": 9.3, + "p99": 19.2, + "n": 604 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.7, + "n": 604 + }, + "network": { + "p50": 7.9, + "p95": 76.6, + "p99": 94.2, + "n": 604 + }, + "decode": { + "p50": 6.3, + "p95": 19.1, + "p99": 25.5, + "n": 604 + }, + "draw": { + "p50": 0.6, + "p95": 1.0, + "p99": 1.6, + "n": 604 + }, + "present": { + "p50": 0.9, + "p95": 19.0, + "p99": 22.0, + "n": 496 + }, + "content": { + "p50": 23.4, + "p95": 100.3, + "p99": 145.3, + "n": 604 + }, + "content_shown": { + "p50": 28.9, + "p95": 104.7, + "p99": 158.2, + "n": 496 + } + }, + "fps": 38.2, + "fps_shown": 31.3, + "late_pct": 25.87, + "stall_max": 180.7, + "stalls_over_100ms": 22, + "mbps": 6.88, + "keyframes": 1, + "size": "1920x1290", + "captured": 843, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 28, + "mbps": 6.12, + "content_p50": 30.5, + "content_p95": 161.7, + "bitrate": 7430400, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 24, + "mbps": 4.2, + "content_p50": 36.9, + "content_p95": 120.7, + "bitrate": 8223440, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 41, + "mbps": 8.28, + "content_p50": 20.3, + "content_p95": 62.3, + "bitrate": 11110898, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 42, + "mbps": 9.01, + "content_p50": 22.2, + "content_p95": 91.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 47, + "mbps": 8.61, + "content_p50": 15.8, + "content_p95": 81.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 43, + "mbps": 8.02, + "content_p50": 17.4, + "content_p95": 56.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 40, + "mbps": 7.97, + "content_p50": 29.0, + "content_p95": 82.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 36, + "mbps": 7.83, + "content_p50": 33.3, + "content_p95": 100.5, + "bitrate": 6992996, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 40, + "mbps": 5.6, + "content_p50": 21.7, + "content_p95": 97.9, + "bitrate": 7742295, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 43, + "mbps": 7.07, + "content_p50": 19.5, + "content_p95": 90.7, + "bitrate": 10470493, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 44, + "mbps": 7.09, + "content_p50": 19.9, + "content_p95": 97.5, + "bitrate": 7246454, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 35, + "mbps": 5.19, + "content_p50": 22.4, + "content_p95": 166.6, + "bitrate": 5307220, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 39, + "mbps": 4.5, + "content_p50": 22.9, + "content_p95": 84.3, + "bitrate": 5887942, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 36, + "mbps": 5.81, + "content_p50": 28.4, + "content_p95": 94.8, + "bitrate": 8002349, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 36, + "mbps": 7.68, + "content_p50": 27.4, + "content_p95": 107.5, + "bitrate": 11948287, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 30, + "mbps": 5.87, + "content_p50": 33.0, + "content_p95": 104.9, + "bitrate": 14562426, + "tier": 1, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 23.4, + "content_p95": 100.3, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "bad", + "fps": "bad", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "controller": { + "adapt": true, + "baseRtt": 6.959, + "ceiling": 14860800, + "fps": 60, + "inFlight": 2, + "scale": 1, + "target": 7430400, + "tier": 0 + }, + "events": [ + { + "t": 0.87, + "e": "down to 7430 kbit/s: queue 191 ms, held 3, oldest 218 ms, base 7 ms, sent 4207 got 3329" + }, + { + "t": 7.54, + "e": "down to 8741 kbit/s: queue 52 ms, held 3, oldest 35 ms, base 7 ms, sent 10505 got 9712" + }, + { + "t": 7.84, + "e": "down to 6992 kbit/s: queue 17 ms, held 3, oldest 3 ms, base 7 ms, sent 8895 got 9452" + }, + { + "t": 10.54, + "e": "down to 7246 kbit/s: queue 19 ms, held 3, oldest 80 ms, base 7 ms, sent 8009 got 8051" + }, + { + "t": 11.74, + "e": "down to 5307 kbit/s: queue 26 ms, held 3, oldest 9 ms, base 6 ms, sent 5808 got 5896" + }, + { + "t": 12.34, + "e": "tier 1" + }, + { + "t": 16.14, + "e": "tier 0" + }, + { + "t": 17.34, + "e": "down to 7430 kbit/s: queue 94 ms, held 2, oldest 94 ms, base 6 ms, sent 7608 got 6892" + } + ], + "source_fps": 53.4, + "cpu": { + "mac_agent_pct": 6.5, + "frame_viewer_pct": 69.3, + "frame_tailscaled_pct": 6.4, + "frame_sshd_pct": 0.6, + "frame_gamescope_pct": 23.2, + "frame_total_pct": 27.8 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz", + "show_s": 1.35, + "panel": "valve.steam.desktopgame.2001652781", + "src": "separate:13306" + } + ], + [ + { + "scenario": "test", + "frames_sent": 482, + "frames_drawn": 482, + "frames_shown": 479, + "duration_s": 15.7, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 482 + }, + "queue": { + "p50": 8.8, + "p95": 16.5, + "p99": 18.6, + "n": 482 + }, + "encode": { + "p50": 3.7, + "p95": 4.5, + "p99": 5.0, + "n": 482 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 482 + }, + "network": { + "p50": 6.0, + "p95": 36.8, + "p99": 84.9, + "n": 482 + }, + "decode": { + "p50": 1.0, + "p95": 2.7, + "p99": 4.8, + "n": 482 + }, + "draw": { + "p50": 0.3, + "p95": 0.7, + "p99": 0.9, + "n": 482 + }, + "present": { + "p50": 0.6, + "p95": 10.8, + "p99": 13.0, + "n": 479 + }, + "content": { + "p50": 22.9, + "p95": 54.1, + "p99": 104.8, + "n": 482 + }, + "content_shown": { + "p50": 24.9, + "p95": 54.9, + "p99": 77.6, + "n": 479 + } + }, + "fps": 30.6, + "fps_shown": 30.4, + "late_pct": 68.81, + "stall_max": 218.0, + "stalls_over_100ms": 10, + "mbps": 0.19, + "keyframes": 2, + "size": "960x540", + "captured": 1037, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 53.9, + "p95": 117.8, + "p99": 222.5, + "n": 29 + }, + "input_shown_ms": { + "p50": 54.4, + "p95": 123.9, + "p99": 223.5, + "n": 29 + }, + "input_parts_ms": { + "uplink": { + "p50": 17.7, + "p95": 77.5, + "p99": 127.7, + "n": 29 + }, + "mac": { + "p50": 8.2, + "p95": 30.4, + "p99": 63.3, + "n": 29 + }, + "back": { + "p50": 19.5, + "p95": 38.3, + "p99": 38.6, + "n": 29 + } + }, + "inputs": { + "sent": 30, + "seen": 29 + }, + "timeline": [ + { + "t": 0, + "fps": 55, + "mbps": 0.46, + "content_p50": 9.9, + "content_p95": 20.9, + "bitrate": 1545640, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 33, + "mbps": 0.28, + "content_p50": 27.7, + "content_p95": 40.2, + "bitrate": 772820, + "tier": 3, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 32, + "mbps": 0.28, + "content_p50": 22.9, + "content_p95": 29.0, + "bitrate": 772820, + "tier": 3, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 25, + "mbps": 0.23, + "content_p50": 26.5, + "content_p95": 106.6, + "bitrate": 386410, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 4, + "fps": 21, + "mbps": 0.14, + "content_p50": 37.0, + "content_p95": 54.1, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 5, + "fps": 24, + "mbps": 0.1, + "content_p50": 32.6, + "content_p95": 96.2, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 6, + "fps": 31, + "mbps": 0.12, + "content_p50": 29.6, + "content_p95": 46.7, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 7, + "fps": 30, + "mbps": 0.12, + "content_p50": 25.7, + "content_p95": 80.1, + "bitrate": 380000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 8, + "fps": 32, + "mbps": 0.2, + "content_p50": 29.0, + "content_p95": 56.4, + "bitrate": 671280, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 9, + "fps": 26, + "mbps": 0.13, + "content_p50": 31.8, + "content_p95": 47.8, + "bitrate": 458624, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 10, + "fps": 30, + "mbps": 0.14, + "content_p50": 36.1, + "content_p95": 64.2, + "bitrate": 458624, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 11, + "fps": 29, + "mbps": 0.14, + "content_p50": 28.1, + "content_p95": 56.7, + "bitrate": 458624, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 12, + "fps": 26, + "mbps": 0.11, + "content_p50": 18.7, + "content_p95": 27.8, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 13, + "fps": 32, + "mbps": 0.24, + "content_p50": 20.1, + "content_p95": 24.8, + "bitrate": 468000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 14, + "fps": 32, + "mbps": 0.18, + "content_p50": 19.4, + "content_p95": 32.8, + "bitrate": 468000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 15, + "fps": 24, + "mbps": 0.12, + "content_p50": 21.9, + "content_p95": 35.2, + "bitrate": 671280, + "tier": 4, + "w": 960, + "net": null + } + ], + "adapt": [], + "content_p50": 22.9, + "content_p95": 54.1, + "input_p50": 53.9, + "input_p95": 117.8, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "input_p50": "acceptable", + "input_p95": "acceptable", + "fps": "bad", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "controller": { + "adapt": true, + "baseRtt": 6.417, + "ceiling": 5529600, + "fps": 30, + "inFlight": 2, + "scale": 0.5, + "target": 1386355, + "tier": 4 + }, + "events": [ + { + "t": 0.87, + "e": "down to 1545 kbit/s: queue 18 ms, held 3, oldest 11 ms, base 5 ms, sent 424 got 420" + }, + { + "t": 1.27, + "e": "down to 772 kbit/s: queue 40 ms, held 3, oldest 55 ms, base 5 ms, sent 374 got 368" + }, + { + "t": 1.48, + "e": "tier 3" + }, + { + "t": 3.37, + "e": "down to 386 kbit/s: queue 119 ms, held 5, oldest 20 ms, base 5 ms, sent 210 got 208" + }, + { + "t": 3.88, + "e": "tier 4" + }, + { + "t": 4.07, + "e": "down to 300 kbit/s: queue 55 ms, held 0, oldest 0 ms, base 5 ms, sent 254 got 254" + }, + { + "t": 4.97, + "e": "down to 300 kbit/s: queue 113 ms, held 1, oldest 91 ms, base 5 ms, sent 154 got 161" + }, + { + "t": 9.78, + "e": "down to 458 kbit/s: queue 206 ms, held 6, oldest 0 ms, base 6 ms, sent 109 got 127" + }, + { + "t": 12.17, + "e": "down to 300 kbit/s: queue 4 ms, held 6, oldest 215 ms, base 6 ms, sent 116 got 103" + } + ], + "source_fps": 66.1, + "cpu": { + "mac_agent_pct": 10.9, + "frame_viewer_pct": 42.3, + "frame_tailscaled_pct": 3.8, + "frame_sshd_pct": 0.4, + "frame_gamescope_pct": 15.4, + "frame_total_pct": 23.3 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 70 Hz", + "show_s": 1.46, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 523, + "frames_drawn": 523, + "frames_shown": 515, + "duration_s": 15.7, + "stages_ms": { + "capture": { + "p50": -4.7, + "p95": 10.3, + "p99": 17.5, + "n": 523 + }, + "queue": { + "p50": 0.0, + "p95": 18.2, + "p99": 33.6, + "n": 523 + }, + "encode": { + "p50": 6.8, + "p95": 8.4, + "p99": 17.2, + "n": 523 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.4, + "n": 523 + }, + "network": { + "p50": 7.7, + "p95": 34.2, + "p99": 205.8, + "n": 523 + }, + "decode": { + "p50": 6.2, + "p95": 15.1, + "p99": 23.5, + "n": 523 + }, + "draw": { + "p50": 0.6, + "p95": 0.8, + "p99": 1.7, + "n": 523 + }, + "present": { + "p50": 0.5, + "p95": 11.4, + "p99": 14.3, + "n": 515 + }, + "content": { + "p50": 23.9, + "p95": 65.6, + "p99": 214.3, + "n": 523 + }, + "content_shown": { + "p50": 26.4, + "p95": 66.0, + "p99": 215.2, + "n": 515 + } + }, + "fps": 33.3, + "fps_shown": 32.7, + "late_pct": 56.13, + "stall_max": 321.6, + "stalls_over_100ms": 10, + "mbps": 6.29, + "keyframes": 2, + "size": "1920x1290", + "captured": 789, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 29, + "mbps": 6.01, + "content_p50": 17.9, + "content_p95": 139.0, + "bitrate": 7430400, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 26, + "mbps": 2.1, + "content_p50": 33.3, + "content_p95": 86.5, + "bitrate": 2202984, + "tier": 3, + "w": 1440, + "net": null + }, + { + "t": 2, + "fps": 30, + "mbps": 2.11, + "content_p50": 23.4, + "content_p95": 40.3, + "bitrate": 2770610, + "tier": 3, + "w": 1440, + "net": null + }, + { + "t": 3, + "fps": 30, + "mbps": 3.24, + "content_p50": 19.3, + "content_p95": 47.2, + "bitrate": 3853181, + "tier": 3, + "w": 1440, + "net": null + }, + { + "t": 4, + "fps": 26, + "mbps": 3.85, + "content_p50": 20.9, + "content_p95": 120.6, + "bitrate": 5294082, + "tier": 3, + "w": 1440, + "net": null + }, + { + "t": 5, + "fps": 26, + "mbps": 2.95, + "content_p50": 23.3, + "content_p95": 46.7, + "bitrate": 3187180, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 29, + "mbps": 3.51, + "content_p50": 27.5, + "content_p95": 48.7, + "bitrate": 4898398, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 28, + "mbps": 5.29, + "content_p50": 28.2, + "content_p95": 48.8, + "bitrate": 6685266, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 29, + "mbps": 7.09, + "content_p50": 23.8, + "content_p95": 50.0, + "bitrate": 9063588, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 35, + "mbps": 9.72, + "content_p50": 31.0, + "content_p95": 59.7, + "bitrate": 13502047, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 42, + "mbps": 12.62, + "content_p50": 23.9, + "content_p95": 34.4, + "bitrate": 14860800, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 47, + "mbps": 11.31, + "content_p50": 20.5, + "content_p95": 41.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 35, + "mbps": 8.1, + "content_p50": 20.7, + "content_p95": 120.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 35, + "mbps": 6.11, + "content_p50": 21.1, + "content_p95": 36.8, + "bitrate": 7430400, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 42, + "mbps": 8.02, + "content_p50": 24.8, + "content_p95": 38.7, + "bitrate": 10055362, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 34, + "mbps": 6.95, + "content_p50": 19.2, + "content_p95": 309.3, + "bitrate": 12271987, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 23.9, + "content_p95": 65.6, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "fps": "bad", + "late_pct": "bad", + "stall_max": "bad" + }, + "controller": { + "adapt": true, + "baseRtt": 7.209, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "target": 7502051, + "tier": 0 + }, + "events": [ + { + "t": 0.78, + "e": "down to 7430 kbit/s: queue 95 ms, held 3, oldest 0 ms, base 7 ms, sent 4459 got 5191" + }, + { + "t": 1.08, + "e": "down to 3715 kbit/s: queue 130 ms, held 2, oldest 127 ms, base 7 ms, sent 2434 got 2754" + }, + { + "t": 1.48, + "e": "down to 2202 kbit/s: queue 44 ms, held 1, oldest 37 ms, base 7 ms, sent 1917 got 2447" + }, + { + "t": 1.68, + "e": "tier 3" + }, + { + "t": 4.98, + "e": "down to 3187 kbit/s: queue 102 ms, held 0, oldest 31 ms, base 7 ms, sent 3802 got 3541" + }, + { + "t": 5.18, + "e": "tier 2" + }, + { + "t": 9.38, + "e": "tier 1" + }, + { + "t": 11.48, + "e": "tier 0" + }, + { + "t": 13.28, + "e": "down to 7430 kbit/s: queue 257 ms, held 5, oldest 240 ms, base 6 ms, sent 4758 got 6534" + }, + { + "t": 15.99, + "e": "down to 6774 kbit/s: queue 1 ms, held 6, oldest 284 ms, base 7 ms, sent 5665 got 4867" + } + ], + "source_fps": 50.3, + "cpu": { + "mac_agent_pct": 5.8, + "frame_viewer_pct": 64.7, + "frame_tailscaled_pct": 5.9, + "frame_sshd_pct": 0.6, + "frame_gamescope_pct": 16.2, + "frame_total_pct": 26.5 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 71 Hz", + "show_s": 1.37, + "panel": "valve.steam.desktopgame.2001605080", + "src": "separate:13687" + } + ] + ] + }, + "table": { + "test": { + "content_p50": { + "off": [ + 11.5, + 10.5, + 12.2 + ], + "on": [ + 10.9, + 10.5, + 22.9 + ] + }, + "content_p95": { + "off": [ + 15.8, + 52.3, + 29.7 + ], + "on": [ + 75.0, + 15.4, + 54.1 + ] + }, + "input_p50": { + "off": [ + 28.8, + 26.7, + 31.4 + ], + "on": [ + 27.2, + 24.6, + 53.9 + ] + }, + "input_p95": { + "off": [ + 38.1, + 89.3, + 78.1 + ], + "on": [ + 95.5, + 39.8, + 117.8 + ] + }, + "fps": { + "off": [ + 60.1, + 60.1, + 60.1 + ], + "on": [ + 53.6, + 59.4, + 30.6 + ] + }, + "late_pct": { + "off": [ + 0.95, + 3.61, + 5.73 + ], + "on": [ + 6.06, + 0.75, + 68.81 + ] + }, + "stall_max": { + "off": [ + 68.8, + 120.1, + 123.4 + ], + "on": [ + 127.2, + 126.0, + 218.0 + ] + }, + "mbps": { + "off": [ + 0.48, + 0.47, + 0.48 + ], + "on": [ + 0.43, + 0.48, + 0.19 + ] + } + }, + "scroll": { + "content_p50": { + "off": [ + 19.5, + 33.3, + 25.3 + ], + "on": [ + 16.6, + 23.4, + 23.9 + ] + }, + "content_p95": { + "off": [ + 44.4, + 108.6, + 110.6 + ], + "on": [ + 45.7, + 100.3, + 65.6 + ] + }, + "input_p50": { + "off": [], + "on": [] + }, + "input_p95": { + "off": [], + "on": [] + }, + "fps": { + "off": [ + 47.1, + 52.6, + 39.5 + ], + "on": [ + 48.4, + 38.2, + 33.3 + ] + }, + "late_pct": { + "off": [ + 20.92, + 14.84, + 33.66 + ], + "on": [ + 21.06, + 25.87, + 56.13 + ] + }, + "stall_max": { + "off": [ + 128.0, + 232.0, + 362.0 + ], + "on": [ + 88.2, + 180.7, + 321.6 + ] + }, + "mbps": { + "off": [ + 9.15, + 9.35, + 8.59 + ], + "on": [ + 9.16, + 6.88, + 6.29 + ] + } + } + } +} \ No newline at end of file diff --git a/bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-clean2.json b/bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-clean2.json new file mode 100644 index 0000000..d88f64a --- /dev/null +++ b/bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-clean2.json @@ -0,0 +1,4150 @@ +{ + "label": "ab-adapt-clean2", + "date": "2026-09-28T20:18:23", + "commit": "a3c6e5c+dirty", + "config": { + "quality": "balanced", + "mode": "separate", + "net": "none", + "delay_ms": 0, + "buffer_ms": 250, + "host": "frame", + "ssh_opts": [], + "encoder": "", + "duration_s": 15.0, + "browser_flags": [] + }, + "frame_build": "20260925.6191901", + "mac": "26.5.2", + "headset": "8:23.135290 [Info] - 0 - leaving standby", + "scenarios": [], + "arms": [ + "off:FRAME_MAC_VIEW_ADAPT=0", + "on:" + ], + "runs": { + "off": [ + [ + { + "scenario": "test", + "frames_sent": 925, + "frames_drawn": 925, + "frames_shown": 649, + "duration_s": 15.4, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 925 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 925 + }, + "encode": { + "p50": 3.6, + "p95": 4.5, + "p99": 4.8, + "n": 925 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.4, + "n": 925 + }, + "network": { + "p50": 4.0, + "p95": 10.6, + "p99": 57.5, + "n": 925 + }, + "decode": { + "p50": 1.0, + "p95": 2.9, + "p99": 6.5, + "n": 925 + }, + "draw": { + "p50": 0.3, + "p95": 0.8, + "p99": 1.0, + "n": 925 + }, + "present": { + "p50": 4.2, + "p95": 14.9, + "p99": 19.7, + "n": 649 + }, + "content": { + "p50": 9.5, + "p95": 16.2, + "p99": 68.6, + "n": 925 + }, + "content_shown": { + "p50": 14.8, + "p95": 26.6, + "p99": 33.5, + "n": 649 + } + }, + "fps": 59.9, + "fps_shown": 42.1, + "late_pct": 2.05, + "stall_max": 150.2, + "stalls_over_100ms": 2, + "mbps": 0.47, + "keyframes": 1, + "size": "1280x720", + "captured": 1047, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 28.3, + "p95": 38.3, + "p99": 41.2, + "n": 30 + }, + "input_shown_ms": { + "p50": 34.8, + "p95": 43.5, + "p99": 47.2, + "n": 21 + }, + "input_parts_ms": { + "uplink": { + "p50": 11.3, + "p95": 19.6, + "p99": 23.2, + "n": 30 + }, + "mac": { + "p50": 5.5, + "p95": 14.1, + "p99": 14.8, + "n": 30 + }, + "back": { + "p50": 9.9, + "p95": 13.8, + "p99": 15.2, + "n": 30 + } + }, + "inputs": { + "sent": 30, + "seen": 30 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.48, + "content_p50": 9.0, + "content_p95": 12.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.47, + "content_p50": 8.8, + "content_p95": 11.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.0, + "content_p95": 15.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 59, + "mbps": 0.46, + "content_p50": 9.2, + "content_p95": 12.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 61, + "mbps": 0.46, + "content_p50": 9.1, + "content_p95": 13.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.57, + "content_p50": 9.0, + "content_p95": 13.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 59, + "mbps": 0.48, + "content_p50": 9.5, + "content_p95": 20.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 61, + "mbps": 0.47, + "content_p50": 9.2, + "content_p95": 20.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.47, + "content_p50": 9.4, + "content_p95": 12.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 60, + "mbps": 0.45, + "content_p50": 9.8, + "content_p95": 12.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.7, + "content_p95": 13.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 60, + "mbps": 0.46, + "content_p50": 12.8, + "content_p95": 14.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 59, + "mbps": 0.46, + "content_p50": 13.6, + "content_p95": 67.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 61, + "mbps": 0.48, + "content_p50": 13.1, + "content_p95": 102.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.1, + "content_p95": 11.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 25, + "mbps": 0.19, + "content_p50": 9.2, + "content_p95": 13.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 9.5, + "content_p95": 16.2, + "input_p50": 28.3, + "input_p95": 38.3, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "acceptable", + "stall_max": "acceptable" + }, + "controller": { + "adapt": false, + "baseRtt": 5.375, + "ceiling": 5529600, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 40, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 68.0, + "cpu": { + "mac_agent_pct": 11.5, + "frame_viewer_pct": 39.7, + "frame_tailscaled_pct": 4.8, + "frame_sshd_pct": 0.6, + "frame_gamescope_pct": 6.2, + "frame_total_pct": 23.9 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 69 Hz", + "show_s": 2.03, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 835, + "frames_drawn": 835, + "frames_shown": 628, + "duration_s": 15.2, + "stages_ms": { + "capture": { + "p50": -3.6, + "p95": 1.6, + "p99": 8.2, + "n": 835 + }, + "queue": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.1, + "n": 835 + }, + "encode": { + "p50": 6.4, + "p95": 8.6, + "p99": 14.2, + "n": 835 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.4, + "n": 835 + }, + "network": { + "p50": 6.1, + "p95": 12.0, + "p99": 24.3, + "n": 835 + }, + "decode": { + "p50": 5.8, + "p95": 10.9, + "p99": 14.7, + "n": 835 + }, + "draw": { + "p50": 0.6, + "p95": 1.4, + "p99": 1.8, + "n": 835 + }, + "present": { + "p50": 4.2, + "p95": 17.9, + "p99": 22.4, + "n": 628 + }, + "content": { + "p50": 20.0, + "p95": 32.3, + "p99": 45.5, + "n": 835 + }, + "content_shown": { + "p50": 25.2, + "p95": 42.2, + "p99": 53.7, + "n": 628 + } + }, + "fps": 54.9, + "fps_shown": 41.3, + "late_pct": 6.59, + "stall_max": 84.3, + "stalls_over_100ms": 0, + "mbps": 9.44, + "keyframes": 1, + "size": "1920x1290", + "captured": 946, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 57, + "mbps": 10.21, + "content_p50": 19.2, + "content_p95": 24.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 55, + "mbps": 9.18, + "content_p50": 18.6, + "content_p95": 45.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 57, + "mbps": 8.93, + "content_p50": 19.1, + "content_p95": 24.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 57, + "mbps": 9.42, + "content_p50": 19.5, + "content_p95": 25.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 56, + "mbps": 9.88, + "content_p50": 19.9, + "content_p95": 25.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 52, + "mbps": 9.25, + "content_p50": 19.7, + "content_p95": 27.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 53, + "mbps": 12.28, + "content_p50": 20.6, + "content_p95": 38.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 53, + "mbps": 9.14, + "content_p50": 19.2, + "content_p95": 24.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 55, + "mbps": 9.31, + "content_p50": 20.4, + "content_p95": 28.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 55, + "mbps": 9.22, + "content_p50": 21.8, + "content_p95": 34.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 56, + "mbps": 8.46, + "content_p50": 22.7, + "content_p95": 41.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 56, + "mbps": 9.18, + "content_p50": 20.0, + "content_p95": 23.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 54, + "mbps": 9.08, + "content_p50": 21.4, + "content_p95": 27.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 55, + "mbps": 8.78, + "content_p50": 19.6, + "content_p95": 28.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 55, + "mbps": 9.45, + "content_p50": 18.8, + "content_p95": 26.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 9, + "mbps": 1.68, + "content_p50": 25.8, + "content_p95": 44.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 20.0, + "content_p95": 32.3, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "local", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "local" + }, + "controller": { + "adapt": false, + "baseRtt": 6.875, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 40, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 62.2, + "cpu": { + "mac_agent_pct": 7.4, + "frame_viewer_pct": 70.3, + "frame_tailscaled_pct": 7.7, + "frame_sshd_pct": 0.8, + "frame_gamescope_pct": 6.7, + "frame_total_pct": 28.8 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz", + "show_s": 1.27, + "panel": "valve.steam.desktopgame.2001924380", + "src": "separate:5316" + } + ], + [ + { + "scenario": "test", + "frames_sent": 912, + "frames_drawn": 912, + "frames_shown": 618, + "duration_s": 15.2, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 912 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.2, + "n": 912 + }, + "encode": { + "p50": 4.2, + "p95": 4.9, + "p99": 5.2, + "n": 912 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 912 + }, + "network": { + "p50": 3.8, + "p95": 8.0, + "p99": 11.1, + "n": 912 + }, + "decode": { + "p50": 1.0, + "p95": 2.3, + "p99": 4.2, + "n": 912 + }, + "draw": { + "p50": 0.3, + "p95": 0.7, + "p99": 1.0, + "n": 912 + }, + "present": { + "p50": 5.1, + "p95": 16.1, + "p99": 17.8, + "n": 618 + }, + "content": { + "p50": 9.7, + "p95": 14.3, + "p99": 17.6, + "n": 912 + }, + "content_shown": { + "p50": 15.3, + "p95": 27.8, + "p99": 29.4, + "n": 618 + } + }, + "fps": 60.0, + "fps_shown": 40.7, + "late_pct": 0.33, + "stall_max": 120.5, + "stalls_over_100ms": 1, + "mbps": 0.48, + "keyframes": 1, + "size": "1280x720", + "captured": 1032, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 22.6, + "p95": 32.7, + "p99": 46.2, + "n": 30 + }, + "input_shown_ms": { + "p50": 28.6, + "p95": 39.0, + "p99": 47.0, + "n": 23 + }, + "input_parts_ms": { + "uplink": { + "p50": 7.3, + "p95": 18.0, + "p99": 22.1, + "n": 30 + }, + "mac": { + "p50": 5.2, + "p95": 14.0, + "p99": 17.2, + "n": 30 + }, + "back": { + "p50": 9.9, + "p95": 13.6, + "p99": 14.3, + "n": 30 + } + }, + "inputs": { + "sent": 30, + "seen": 30 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.5, + "content_p50": 8.9, + "content_p95": 71.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.48, + "content_p50": 8.7, + "content_p95": 10.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.48, + "content_p50": 8.7, + "content_p95": 10.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.48, + "content_p50": 9.1, + "content_p95": 10.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 60, + "mbps": 0.47, + "content_p50": 8.7, + "content_p95": 10.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.58, + "content_p50": 8.7, + "content_p95": 11.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 60, + "mbps": 0.48, + "content_p50": 8.9, + "content_p95": 11.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 60, + "mbps": 0.48, + "content_p50": 8.7, + "content_p95": 10.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.2, + "content_p95": 13.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.6, + "content_p95": 11.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.6, + "content_p95": 14.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 60, + "mbps": 0.47, + "content_p50": 12.3, + "content_p95": 15.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 60, + "mbps": 0.47, + "content_p50": 12.8, + "content_p95": 15.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 60, + "mbps": 0.48, + "content_p50": 12.7, + "content_p95": 14.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 60, + "mbps": 0.46, + "content_p50": 12.9, + "content_p95": 17.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 12, + "mbps": 0.09, + "content_p50": 12.6, + "content_p95": 14.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 9.7, + "content_p95": 14.3, + "input_p50": 22.6, + "input_p95": 32.7, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "local", + "stall_max": "acceptable" + }, + "controller": { + "adapt": false, + "baseRtt": 5.334, + "ceiling": 5529600, + "fps": 60, + "inFlight": 2, + "scale": 1, + "slack": 40, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 67.9, + "cpu": { + "mac_agent_pct": 16.1, + "frame_viewer_pct": 46.3, + "frame_tailscaled_pct": 5.1, + "frame_sshd_pct": 0.6, + "frame_gamescope_pct": 8.9, + "frame_total_pct": 28.6 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 140 Hz", + "show_s": 1.27, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 864, + "frames_drawn": 864, + "frames_shown": 626, + "duration_s": 15.2, + "stages_ms": { + "capture": { + "p50": -4.6, + "p95": 0.4, + "p99": 0.8, + "n": 864 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 864 + }, + "encode": { + "p50": 6.7, + "p95": 8.5, + "p99": 11.1, + "n": 864 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.3, + "n": 864 + }, + "network": { + "p50": 6.0, + "p95": 9.2, + "p99": 14.7, + "n": 864 + }, + "decode": { + "p50": 5.7, + "p95": 9.9, + "p99": 12.9, + "n": 864 + }, + "draw": { + "p50": 0.6, + "p95": 1.3, + "p99": 1.9, + "n": 864 + }, + "present": { + "p50": 4.3, + "p95": 16.8, + "p99": 21.2, + "n": 626 + }, + "content": { + "p50": 19.5, + "p95": 25.2, + "p99": 32.2, + "n": 864 + }, + "content_shown": { + "p50": 24.4, + "p95": 36.6, + "p99": 41.8, + "n": 626 + } + }, + "fps": 57.0, + "fps_shown": 41.3, + "late_pct": 3.36, + "stall_max": 105.4, + "stalls_over_100ms": 1, + "mbps": 9.44, + "keyframes": 1, + "size": "1920x1290", + "captured": 978, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 56, + "mbps": 9.38, + "content_p50": 19.1, + "content_p95": 24.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 56, + "mbps": 9.47, + "content_p50": 19.2, + "content_p95": 24.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 56, + "mbps": 8.94, + "content_p50": 19.3, + "content_p95": 25.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 57, + "mbps": 9.51, + "content_p50": 19.7, + "content_p95": 26.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 57, + "mbps": 9.65, + "content_p50": 18.6, + "content_p95": 22.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 57, + "mbps": 9.0, + "content_p50": 18.7, + "content_p95": 24.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 55, + "mbps": 11.82, + "content_p50": 20.2, + "content_p95": 33.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 57, + "mbps": 9.24, + "content_p50": 18.9, + "content_p95": 21.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 58, + "mbps": 9.37, + "content_p50": 19.3, + "content_p95": 25.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 57, + "mbps": 9.2, + "content_p50": 20.5, + "content_p95": 27.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 58, + "mbps": 9.12, + "content_p50": 20.1, + "content_p95": 25.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 57, + "mbps": 9.29, + "content_p50": 19.4, + "content_p95": 25.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 57, + "mbps": 9.29, + "content_p50": 19.9, + "content_p95": 23.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 58, + "mbps": 8.8, + "content_p50": 19.8, + "content_p95": 23.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 58, + "mbps": 9.34, + "content_p50": 20.3, + "content_p95": 28.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 10, + "mbps": 1.68, + "content_p50": 19.6, + "content_p95": 22.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 19.5, + "content_p95": 25.2, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "local", + "fps": "acceptable", + "late_pct": "acceptable", + "stall_max": "acceptable" + }, + "controller": { + "adapt": false, + "baseRtt": 6.75, + "ceiling": 14860800, + "fps": 60, + "inFlight": 0, + "scale": 1, + "slack": 40, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 64.3, + "cpu": { + "mac_agent_pct": 8.6, + "frame_viewer_pct": 73.0, + "frame_tailscaled_pct": 8.0, + "frame_sshd_pct": 0.9, + "frame_gamescope_pct": 7.8, + "frame_total_pct": 30.0 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 142 Hz", + "show_s": 1.26, + "panel": "valve.steam.desktopgame.2001204762", + "src": "separate:5722" + } + ], + [ + { + "scenario": "test", + "frames_sent": 912, + "frames_drawn": 912, + "frames_shown": 618, + "duration_s": 15.2, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 912 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 912 + }, + "encode": { + "p50": 4.2, + "p95": 4.8, + "p99": 5.1, + "n": 912 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 912 + }, + "network": { + "p50": 4.6, + "p95": 8.9, + "p99": 13.7, + "n": 912 + }, + "decode": { + "p50": 1.2, + "p95": 2.2, + "p99": 3.9, + "n": 912 + }, + "draw": { + "p50": 0.3, + "p95": 0.7, + "p99": 1.0, + "n": 912 + }, + "present": { + "p50": 5.4, + "p95": 17.0, + "p99": 19.3, + "n": 618 + }, + "content": { + "p50": 10.6, + "p95": 15.0, + "p99": 19.9, + "n": 912 + }, + "content_shown": { + "p50": 16.0, + "p95": 27.7, + "p99": 30.8, + "n": 618 + } + }, + "fps": 60.0, + "fps_shown": 40.6, + "late_pct": 0.99, + "stall_max": 37.7, + "stalls_over_100ms": 0, + "mbps": 0.47, + "keyframes": 1, + "size": "1280x720", + "captured": 1034, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 30.9, + "p95": 103.7, + "p99": 196.7, + "n": 30 + }, + "input_shown_ms": { + "p50": 36.3, + "p95": 106.8, + "p99": 200.8, + "n": 19 + }, + "input_parts_ms": { + "uplink": { + "p50": 13.9, + "p95": 74.2, + "p99": 183.2, + "n": 30 + }, + "mac": { + "p50": 5.5, + "p95": 16.4, + "p99": 18.7, + "n": 30 + }, + "back": { + "p50": 11.2, + "p95": 15.5, + "p99": 20.2, + "n": 30 + } + }, + "inputs": { + "sent": 30, + "seen": 30 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.49, + "content_p50": 9.4, + "content_p95": 13.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.47, + "content_p50": 9.5, + "content_p95": 11.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.47, + "content_p50": 9.4, + "content_p95": 11.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.6, + "content_p95": 11.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.1, + "content_p95": 12.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.58, + "content_p50": 10.5, + "content_p95": 12.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.3, + "content_p95": 13.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.3, + "content_p95": 12.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.1, + "content_p95": 11.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 60, + "mbps": 0.45, + "content_p50": 10.9, + "content_p95": 14.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.46, + "content_p50": 11.4, + "content_p95": 15.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 60, + "mbps": 0.48, + "content_p50": 11.5, + "content_p95": 16.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 60, + "mbps": 0.48, + "content_p50": 13.3, + "content_p95": 17.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 60, + "mbps": 0.46, + "content_p50": 12.1, + "content_p95": 16.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.5, + "content_p95": 16.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 12, + "mbps": 0.09, + "content_p50": 12.9, + "content_p95": 15.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 10.6, + "content_p95": 15.0, + "input_p50": 30.9, + "input_p95": 103.7, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "acceptable", + "fps": "local", + "late_pct": "local", + "stall_max": "local" + }, + "controller": { + "adapt": false, + "baseRtt": 5.625, + "ceiling": 5529600, + "fps": 60, + "inFlight": 2, + "scale": 1, + "slack": 40, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 68.0, + "cpu": { + "mac_agent_pct": 15.7, + "frame_viewer_pct": 52.1, + "frame_tailscaled_pct": 8.9, + "frame_sshd_pct": 1.3, + "frame_gamescope_pct": 9.1, + "frame_total_pct": 23.1 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz", + "show_s": 1.26, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 870, + "frames_drawn": 870, + "frames_shown": 627, + "duration_s": 15.2, + "stages_ms": { + "capture": { + "p50": -4.4, + "p95": 0.4, + "p99": 0.8, + "n": 870 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 870 + }, + "encode": { + "p50": 6.7, + "p95": 8.5, + "p99": 13.9, + "n": 870 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.5, + "n": 870 + }, + "network": { + "p50": 6.3, + "p95": 27.2, + "p99": 133.0, + "n": 870 + }, + "decode": { + "p50": 5.9, + "p95": 15.5, + "p99": 21.7, + "n": 870 + }, + "draw": { + "p50": 0.6, + "p95": 1.5, + "p99": 1.9, + "n": 870 + }, + "present": { + "p50": 3.6, + "p95": 16.4, + "p99": 21.1, + "n": 627 + }, + "content": { + "p50": 20.5, + "p95": 45.9, + "p99": 170.0, + "n": 870 + }, + "content_shown": { + "p50": 27.1, + "p95": 55.8, + "p99": 200.4, + "n": 627 + } + }, + "fps": 57.3, + "fps_shown": 41.3, + "late_pct": 4.6, + "stall_max": 321.6, + "stalls_over_100ms": 2, + "mbps": 9.4, + "keyframes": 1, + "size": "1920x1290", + "captured": 986, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 56, + "mbps": 9.16, + "content_p50": 27.2, + "content_p95": 49.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 55, + "mbps": 9.46, + "content_p50": 23.6, + "content_p95": 237.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 58, + "mbps": 8.98, + "content_p50": 21.5, + "content_p95": 26.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 58, + "mbps": 9.81, + "content_p50": 22.4, + "content_p95": 55.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 57, + "mbps": 9.66, + "content_p50": 21.1, + "content_p95": 31.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 58, + "mbps": 8.89, + "content_p50": 19.9, + "content_p95": 24.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 58, + "mbps": 11.75, + "content_p50": 20.5, + "content_p95": 74.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 57, + "mbps": 9.0, + "content_p50": 20.1, + "content_p95": 27.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 57, + "mbps": 9.35, + "content_p50": 20.5, + "content_p95": 24.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 57, + "mbps": 9.17, + "content_p50": 19.8, + "content_p95": 24.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 57, + "mbps": 8.89, + "content_p50": 19.3, + "content_p95": 25.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 58, + "mbps": 9.09, + "content_p50": 18.9, + "content_p95": 22.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 58, + "mbps": 9.74, + "content_p50": 19.7, + "content_p95": 31.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 57, + "mbps": 8.62, + "content_p50": 18.5, + "content_p95": 21.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 58, + "mbps": 9.36, + "content_p50": 25.9, + "content_p95": 37.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 11, + "mbps": 1.84, + "content_p50": 21.5, + "content_p95": 123.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 20.5, + "content_p95": 45.9, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "fps": "acceptable", + "late_pct": "acceptable", + "stall_max": "bad" + }, + "controller": { + "adapt": false, + "baseRtt": 7.125, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 40, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 64.9, + "cpu": { + "mac_agent_pct": 8.8, + "frame_viewer_pct": 84.1, + "frame_tailscaled_pct": 13.1, + "frame_sshd_pct": -0.9, + "frame_gamescope_pct": 9.9, + "frame_total_pct": 35.6 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 143 Hz", + "show_s": 1.44, + "panel": "valve.steam.desktopgame.2001042553", + "src": "separate:5856" + } + ] + ], + "on": [ + [ + { + "scenario": "test", + "frames_sent": 942, + "frames_drawn": 942, + "frames_shown": 641, + "duration_s": 15.7, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 942 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 942 + }, + "encode": { + "p50": 3.7, + "p95": 4.5, + "p99": 5.5, + "n": 942 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.5, + "n": 942 + }, + "network": { + "p50": 4.0, + "p95": 7.1, + "p99": 14.6, + "n": 942 + }, + "decode": { + "p50": 1.0, + "p95": 2.3, + "p99": 3.7, + "n": 942 + }, + "draw": { + "p50": 0.3, + "p95": 0.7, + "p99": 1.0, + "n": 942 + }, + "present": { + "p50": 4.9, + "p95": 17.5, + "p99": 19.9, + "n": 641 + }, + "content": { + "p50": 9.5, + "p95": 13.2, + "p99": 23.3, + "n": 942 + }, + "content_shown": { + "p50": 15.0, + "p95": 27.3, + "p99": 29.9, + "n": 641 + } + }, + "fps": 60.0, + "fps_shown": 40.9, + "late_pct": 1.17, + "stall_max": 75.6, + "stalls_over_100ms": 0, + "mbps": 0.47, + "keyframes": 1, + "size": "1280x720", + "captured": 1064, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 27.2, + "p95": 51.1, + "p99": 60.4, + "n": 30 + }, + "input_shown_ms": { + "p50": 33.7, + "p95": 53.0, + "p99": 72.9, + "n": 20 + }, + "input_parts_ms": { + "uplink": { + "p50": 11.8, + "p95": 39.5, + "p99": 41.9, + "n": 30 + }, + "mac": { + "p50": 4.6, + "p95": 14.7, + "p99": 19.7, + "n": 30 + }, + "back": { + "p50": 9.5, + "p95": 14.0, + "p99": 15.3, + "n": 30 + } + }, + "inputs": { + "sent": 30, + "seen": 30 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.48, + "content_p50": 9.2, + "content_p95": 12.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.47, + "content_p50": 8.9, + "content_p95": 24.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.48, + "content_p50": 8.7, + "content_p95": 11.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.47, + "content_p50": 8.6, + "content_p95": 11.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 60, + "mbps": 0.45, + "content_p50": 9.0, + "content_p95": 10.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.59, + "content_p50": 8.6, + "content_p95": 10.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 60, + "mbps": 0.45, + "content_p50": 8.5, + "content_p95": 11.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 60, + "mbps": 0.47, + "content_p50": 8.6, + "content_p95": 13.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.45, + "content_p50": 8.0, + "content_p95": 11.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 60, + "mbps": 0.45, + "content_p50": 9.1, + "content_p95": 14.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.2, + "content_p95": 11.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.8, + "content_p95": 14.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.6, + "content_p95": 12.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 60, + "mbps": 0.46, + "content_p50": 11.4, + "content_p95": 15.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 60, + "mbps": 0.44, + "content_p50": 11.0, + "content_p95": 12.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 42, + "mbps": 0.31, + "content_p50": 11.2, + "content_p95": 14.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 9.5, + "content_p95": 13.2, + "input_p50": 27.2, + "input_p95": 51.1, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "acceptable", + "stall_max": "local" + }, + "controller": { + "adapt": true, + "baseRtt": 5.541, + "ceiling": 5529600, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 52.292, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 67.8, + "cpu": { + "mac_agent_pct": 12.1, + "frame_viewer_pct": 45.2, + "frame_tailscaled_pct": 5.1, + "frame_sshd_pct": 0.7, + "frame_gamescope_pct": 7.9, + "frame_total_pct": 27.2 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 141 Hz", + "show_s": 1.3, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 857, + "frames_drawn": 857, + "frames_shown": 618, + "duration_s": 15.2, + "stages_ms": { + "capture": { + "p50": -2.9, + "p95": 4.5, + "p99": 6.3, + "n": 857 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 857 + }, + "encode": { + "p50": 6.6, + "p95": 9.0, + "p99": 15.0, + "n": 857 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.4, + "n": 857 + }, + "network": { + "p50": 5.2, + "p95": 12.8, + "p99": 25.4, + "n": 857 + }, + "decode": { + "p50": 5.8, + "p95": 10.4, + "p99": 13.5, + "n": 857 + }, + "draw": { + "p50": 0.6, + "p95": 1.4, + "p99": 1.8, + "n": 857 + }, + "present": { + "p50": 4.7, + "p95": 16.8, + "p99": 20.3, + "n": 618 + }, + "content": { + "p50": 19.5, + "p95": 31.7, + "p99": 45.5, + "n": 857 + }, + "content_shown": { + "p50": 25.7, + "p95": 38.5, + "p99": 47.8, + "n": 618 + } + }, + "fps": 56.4, + "fps_shown": 40.7, + "late_pct": 5.6, + "stall_max": 120.5, + "stalls_over_100ms": 1, + "mbps": 9.4, + "keyframes": 1, + "size": "1920x1290", + "captured": 971, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 53, + "mbps": 9.14, + "content_p50": 20.2, + "content_p95": 40.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 58, + "mbps": 9.36, + "content_p50": 20.2, + "content_p95": 43.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 54, + "mbps": 9.42, + "content_p50": 18.9, + "content_p95": 28.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 57, + "mbps": 9.46, + "content_p50": 18.1, + "content_p95": 25.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 56, + "mbps": 8.81, + "content_p50": 18.9, + "content_p95": 25.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 57, + "mbps": 11.11, + "content_p50": 18.6, + "content_p95": 34.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 57, + "mbps": 9.96, + "content_p50": 18.1, + "content_p95": 24.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 57, + "mbps": 8.79, + "content_p50": 18.4, + "content_p95": 27.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 58, + "mbps": 9.1, + "content_p50": 20.3, + "content_p95": 27.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 56, + "mbps": 9.65, + "content_p50": 19.3, + "content_p95": 28.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 56, + "mbps": 9.15, + "content_p50": 20.4, + "content_p95": 34.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 57, + "mbps": 9.46, + "content_p50": 20.4, + "content_p95": 31.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 57, + "mbps": 9.53, + "content_p50": 20.4, + "content_p95": 31.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 57, + "mbps": 8.94, + "content_p50": 20.2, + "content_p95": 25.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 57, + "mbps": 9.2, + "content_p50": 19.4, + "content_p95": 27.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 10, + "mbps": 1.69, + "content_p50": 18.1, + "content_p95": 23.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 19.5, + "content_p95": 31.7, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "local", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "controller": { + "adapt": true, + "baseRtt": 6.666, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 47.293, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 63.9, + "cpu": { + "mac_agent_pct": 8.5, + "frame_viewer_pct": 72.5, + "frame_tailscaled_pct": 8.3, + "frame_sshd_pct": 0.9, + "frame_gamescope_pct": 7.1, + "frame_total_pct": 30.1 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz", + "show_s": 1.27, + "panel": "valve.steam.desktopgame.2001321860", + "src": "separate:5452" + } + ], + [ + { + "scenario": "test", + "frames_sent": 740, + "frames_drawn": 740, + "frames_shown": 520, + "duration_s": 15.2, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 740 + }, + "queue": { + "p50": 0.0, + "p95": 13.2, + "p99": 19.4, + "n": 740 + }, + "encode": { + "p50": 4.1, + "p95": 4.8, + "p99": 5.2, + "n": 740 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 740 + }, + "network": { + "p50": 3.6, + "p95": 125.3, + "p99": 165.0, + "n": 740 + }, + "decode": { + "p50": 1.0, + "p95": 6.1, + "p99": 10.2, + "n": 740 + }, + "draw": { + "p50": 0.3, + "p95": 0.6, + "p99": 1.1, + "n": 740 + }, + "present": { + "p50": 0.9, + "p95": 15.9, + "p99": 20.1, + "n": 520 + }, + "content": { + "p50": 9.5, + "p95": 137.4, + "p99": 173.0, + "n": 740 + }, + "content_shown": { + "p50": 14.9, + "p95": 66.7, + "p99": 174.2, + "n": 520 + } + }, + "fps": 48.7, + "fps_shown": 34.2, + "late_pct": 16.35, + "stall_max": 267.4, + "stalls_over_100ms": 16, + "mbps": 0.36, + "keyframes": 2, + "size": "960x540", + "captured": 1032, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 32.1, + "p95": 63.7, + "p99": 188.2, + "n": 29 + }, + "input_shown_ms": { + "p50": 36.2, + "p95": 64.1, + "p99": 188.9, + "n": 20 + }, + "input_parts_ms": { + "uplink": { + "p50": 10.5, + "p95": 31.9, + "p99": 47.7, + "n": 29 + }, + "mac": { + "p50": 5.4, + "p95": 19.2, + "p99": 30.2, + "n": 29 + }, + "back": { + "p50": 9.7, + "p95": 29.3, + "p99": 168.5, + "n": 29 + } + }, + "inputs": { + "sent": 30, + "seen": 29 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.47, + "content_p50": 8.6, + "content_p95": 10.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.47, + "content_p50": 8.8, + "content_p95": 11.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.47, + "content_p50": 8.6, + "content_p95": 10.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.47, + "content_p50": 9.3, + "content_p95": 12.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 60, + "mbps": 0.47, + "content_p50": 8.6, + "content_p95": 10.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.56, + "content_p50": 9.1, + "content_p95": 11.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 59, + "mbps": 0.45, + "content_p50": 9.1, + "content_p95": 25.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 60, + "mbps": 0.46, + "content_p50": 12.2, + "content_p95": 57.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 52, + "mbps": 0.4, + "content_p50": 22.0, + "content_p95": 159.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 47, + "mbps": 0.35, + "content_p50": 90.3, + "content_p95": 164.7, + "bitrate": 2764800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 45, + "mbps": 0.34, + "content_p50": 93.8, + "content_p95": 166.0, + "bitrate": 345600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 21, + "mbps": 0.18, + "content_p50": 81.0, + "content_p95": 177.2, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 12, + "fps": 27, + "mbps": 0.13, + "content_p50": 29.0, + "content_p95": 176.3, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 13, + "fps": 31, + "mbps": 0.11, + "content_p50": 21.7, + "content_p95": 29.4, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 14, + "fps": 31, + "mbps": 0.11, + "content_p50": 19.6, + "content_p95": 29.3, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 15, + "fps": 7, + "mbps": 0.02, + "content_p50": 21.9, + "content_p95": 29.3, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + } + ], + "adapt": [], + "content_p50": 9.5, + "content_p95": 137.4, + "input_p50": 32.1, + "input_p95": 63.7, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "bad", + "input_p50": "local", + "input_p95": "local", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "bad" + }, + "controller": { + "adapt": true, + "baseRtt": 5.625, + "ceiling": 5529600, + "fps": 30, + "inFlight": 3, + "scale": 0.5, + "slack": 86.958, + "target": 917248, + "tier": 4 + }, + "events": [ + { + "t": 9.22, + "e": "down to 2764 kbit/s: queue 25 ms, held 4, oldest 0 ms, base 5 ms, sent 354 got 379" + }, + { + "t": 10.02, + "e": "down to 1382 kbit/s: queue 99 ms, held 5, oldest 0 ms, base 5 ms, sent 346 got 346" + }, + { + "t": 10.32, + "e": "down to 691 kbit/s: queue 109 ms, held 4, oldest 15 ms, base 5 ms, sent 316 got 309" + }, + { + "t": 10.82, + "e": "down to 345 kbit/s: queue 105 ms, held 5, oldest 177 ms, base 5 ms, sent 304 got 243" + }, + { + "t": 11.22, + "e": "down to 300 kbit/s: queue 118 ms, held 0, oldest 42 ms, base 5 ms, sent 302 got 332" + }, + { + "t": 11.42, + "e": "tier 4" + }, + { + "t": 11.62, + "e": "down to 300 kbit/s: queue 140 ms, held 3, oldest 0 ms, base 5 ms, sent 291 got 291" + }, + { + "t": 12.22, + "e": "down to 300 kbit/s: queue 61 ms, held 2, oldest 1 ms, base 5 ms, sent 152 got 137" + }, + { + "t": 12.52, + "e": "down to 300 kbit/s: queue 102 ms, held 0, oldest 4 ms, base 5 ms, sent 156 got 146" + } + ], + "source_fps": 67.9, + "cpu": { + "mac_agent_pct": 13.4, + "frame_viewer_pct": 35.6, + "frame_tailscaled_pct": 4.1, + "frame_sshd_pct": 0.5, + "frame_gamescope_pct": 7.0, + "frame_total_pct": 24.6 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 138 Hz", + "show_s": 1.29, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 852, + "frames_drawn": 852, + "frames_shown": 607, + "duration_s": 15.2, + "stages_ms": { + "capture": { + "p50": -4.5, + "p95": 0.5, + "p99": 6.7, + "n": 852 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.5, + "n": 852 + }, + "encode": { + "p50": 6.6, + "p95": 9.3, + "p99": 16.5, + "n": 852 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.4, + "n": 852 + }, + "network": { + "p50": 5.9, + "p95": 15.8, + "p99": 58.8, + "n": 852 + }, + "decode": { + "p50": 5.8, + "p95": 11.6, + "p99": 23.2, + "n": 852 + }, + "draw": { + "p50": 0.6, + "p95": 1.3, + "p99": 1.8, + "n": 852 + }, + "present": { + "p50": 4.2, + "p95": 16.4, + "p99": 20.1, + "n": 607 + }, + "content": { + "p50": 19.6, + "p95": 37.9, + "p99": 106.2, + "n": 852 + }, + "content_shown": { + "p50": 25.7, + "p95": 45.1, + "p99": 110.7, + "n": 607 + } + }, + "fps": 56.2, + "fps_shown": 40.0, + "late_pct": 5.52, + "stall_max": 147.0, + "stalls_over_100ms": 3, + "mbps": 9.36, + "keyframes": 1, + "size": "1920x1290", + "captured": 982, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 44, + "mbps": 8.61, + "content_p50": 29.3, + "content_p95": 126.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 55, + "mbps": 9.63, + "content_p50": 19.5, + "content_p95": 29.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 57, + "mbps": 9.2, + "content_p50": 20.4, + "content_p95": 71.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 53, + "mbps": 9.04, + "content_p50": 19.5, + "content_p95": 36.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 58, + "mbps": 9.08, + "content_p50": 18.9, + "content_p95": 22.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 57, + "mbps": 9.22, + "content_p50": 19.3, + "content_p95": 25.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 57, + "mbps": 11.78, + "content_p50": 20.5, + "content_p95": 37.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 57, + "mbps": 8.73, + "content_p50": 19.2, + "content_p95": 24.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 58, + "mbps": 9.29, + "content_p50": 19.1, + "content_p95": 23.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 58, + "mbps": 9.5, + "content_p50": 18.9, + "content_p95": 24.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 58, + "mbps": 8.58, + "content_p50": 19.2, + "content_p95": 23.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 57, + "mbps": 9.44, + "content_p50": 20.2, + "content_p95": 28.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 58, + "mbps": 9.7, + "content_p50": 20.1, + "content_p95": 37.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 58, + "mbps": 9.15, + "content_p50": 20.8, + "content_p95": 41.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 58, + "mbps": 9.29, + "content_p50": 19.7, + "content_p95": 23.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 9, + "mbps": 1.64, + "content_p50": 20.9, + "content_p95": 38.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 19.6, + "content_p95": 37.9, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "local", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "controller": { + "adapt": true, + "baseRtt": 6.625, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 48.604, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 64.6, + "cpu": { + "mac_agent_pct": 8.3, + "frame_viewer_pct": 71.7, + "frame_tailscaled_pct": 8.0, + "frame_sshd_pct": 0.9, + "frame_gamescope_pct": 7.2, + "frame_total_pct": 29.1 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz", + "show_s": 1.27, + "panel": "valve.steam.desktopgame.2001299121", + "src": "separate:5587" + } + ], + [ + { + "scenario": "test", + "frames_sent": 908, + "frames_drawn": 908, + "frames_shown": 382, + "duration_s": 15.2, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 908 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.2, + "n": 908 + }, + "encode": { + "p50": 4.2, + "p95": 4.8, + "p99": 5.1, + "n": 908 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 908 + }, + "network": { + "p50": 4.2, + "p95": 7.5, + "p99": 10.1, + "n": 908 + }, + "decode": { + "p50": 1.1, + "p95": 4.0, + "p99": 4.6, + "n": 908 + }, + "draw": { + "p50": 0.4, + "p95": 1.1, + "p99": 1.3, + "n": 908 + }, + "present": { + "p50": 6.2, + "p95": 17.0, + "p99": 18.3, + "n": 382 + }, + "content": { + "p50": 10.5, + "p95": 14.2, + "p99": 16.6, + "n": 908 + }, + "content_shown": { + "p50": 17.1, + "p95": 27.2, + "p99": 29.4, + "n": 382 + } + }, + "fps": 59.7, + "fps_shown": 25.1, + "late_pct": 0.33, + "stall_max": 122.8, + "stalls_over_100ms": 1, + "mbps": 0.47, + "keyframes": 1, + "size": "1280x720", + "captured": 1033, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 25.2, + "p95": 33.5, + "p99": 34.1, + "n": 30 + }, + "input_shown_ms": { + "p50": 32.4, + "p95": 43.4, + "p99": 43.4, + "n": 9 + }, + "input_parts_ms": { + "uplink": { + "p50": 5.9, + "p95": 17.6, + "p99": 18.3, + "n": 30 + }, + "mac": { + "p50": 6.2, + "p95": 14.5, + "p99": 20.8, + "n": 30 + }, + "back": { + "p50": 10.7, + "p95": 13.5, + "p99": 14.8, + "n": 30 + } + }, + "inputs": { + "sent": 30, + "seen": 30 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.7, + "content_p95": 15.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.47, + "content_p50": 12.1, + "content_p95": 14.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.47, + "content_p50": 11.0, + "content_p95": 14.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.47, + "content_p50": 9.9, + "content_p95": 12.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.3, + "content_p95": 13.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.56, + "content_p50": 10.3, + "content_p95": 13.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 60, + "mbps": 0.45, + "content_p50": 9.3, + "content_p95": 11.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 56, + "mbps": 0.44, + "content_p50": 8.9, + "content_p95": 10.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.47, + "content_p50": 9.4, + "content_p95": 11.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.6, + "content_p95": 11.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.44, + "content_p50": 9.1, + "content_p95": 11.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 60, + "mbps": 0.46, + "content_p50": 11.1, + "content_p95": 13.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.3, + "content_p95": 14.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 60, + "mbps": 0.44, + "content_p50": 12.3, + "content_p95": 14.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.4, + "content_p95": 14.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 12, + "mbps": 0.09, + "content_p50": 12.5, + "content_p95": 13.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 10.5, + "content_p95": 14.2, + "input_p50": 25.2, + "input_p95": 33.5, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "local", + "stall_max": "acceptable" + }, + "controller": { + "adapt": true, + "baseRtt": 5.167, + "ceiling": 5529600, + "fps": 60, + "inFlight": 0, + "scale": 1, + "slack": 49.102, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 68.0, + "cpu": { + "mac_agent_pct": 15.5, + "frame_viewer_pct": 51.6, + "frame_tailscaled_pct": 5.2, + "frame_sshd_pct": 0.8, + "frame_gamescope_pct": 9.5, + "frame_total_pct": 33.4 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 135 Hz", + "show_s": 1.37, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 843, + "frames_drawn": 843, + "frames_shown": 414, + "duration_s": 15.2, + "stages_ms": { + "capture": { + "p50": -4.7, + "p95": 0.5, + "p99": 1.3, + "n": 843 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 843 + }, + "encode": { + "p50": 6.6, + "p95": 9.2, + "p99": 17.1, + "n": 843 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.4, + "n": 843 + }, + "network": { + "p50": 6.4, + "p95": 14.7, + "p99": 68.7, + "n": 843 + }, + "decode": { + "p50": 6.0, + "p95": 11.7, + "p99": 16.1, + "n": 843 + }, + "draw": { + "p50": 0.7, + "p95": 1.4, + "p99": 1.7, + "n": 843 + }, + "present": { + "p50": 4.0, + "p95": 18.1, + "p99": 25.9, + "n": 414 + }, + "content": { + "p50": 20.5, + "p95": 33.6, + "p99": 103.0, + "n": 843 + }, + "content_shown": { + "p50": 26.1, + "p95": 41.0, + "p99": 122.7, + "n": 414 + } + }, + "fps": 55.6, + "fps_shown": 27.3, + "late_pct": 6.29, + "stall_max": 153.6, + "stalls_over_100ms": 3, + "mbps": 9.25, + "keyframes": 1, + "size": "1920x1290", + "captured": 970, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 53, + "mbps": 9.13, + "content_p50": 22.1, + "content_p95": 110.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 53, + "mbps": 9.46, + "content_p50": 20.7, + "content_p95": 27.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 56, + "mbps": 9.19, + "content_p50": 20.2, + "content_p95": 26.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 57, + "mbps": 9.36, + "content_p50": 20.1, + "content_p95": 27.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 57, + "mbps": 9.66, + "content_p50": 19.9, + "content_p95": 24.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 57, + "mbps": 8.68, + "content_p50": 20.5, + "content_p95": 25.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 46, + "mbps": 10.48, + "content_p50": 25.4, + "content_p95": 126.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 56, + "mbps": 9.16, + "content_p50": 20.1, + "content_p95": 35.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 57, + "mbps": 8.76, + "content_p50": 21.1, + "content_p95": 28.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 57, + "mbps": 9.29, + "content_p50": 20.5, + "content_p95": 24.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 57, + "mbps": 8.87, + "content_p50": 20.4, + "content_p95": 40.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 58, + "mbps": 9.17, + "content_p50": 20.3, + "content_p95": 23.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 56, + "mbps": 9.05, + "content_p50": 19.5, + "content_p95": 23.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 56, + "mbps": 8.85, + "content_p50": 20.8, + "content_p95": 31.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 57, + "mbps": 9.48, + "content_p50": 20.6, + "content_p95": 34.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 10, + "mbps": 1.65, + "content_p50": 24.8, + "content_p95": 39.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 20.5, + "content_p95": 33.6, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "local", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "controller": { + "adapt": true, + "baseRtt": 7.167, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 48.229, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 63.8, + "cpu": { + "mac_agent_pct": 8.5, + "frame_viewer_pct": 78.1, + "frame_tailscaled_pct": 8.2, + "frame_sshd_pct": 0.9, + "frame_gamescope_pct": 9.0, + "frame_total_pct": 29.8 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 71 Hz", + "show_s": 1.26, + "panel": "valve.steam.desktopgame.2001733678", + "src": "separate:5994" + } + ] + ] + }, + "table": { + "test": { + "content_p50": { + "off": [ + 9.5, + 9.7, + 10.6 + ], + "on": [ + 9.5, + 9.5, + 10.5 + ] + }, + "content_p95": { + "off": [ + 16.2, + 14.3, + 15.0 + ], + "on": [ + 13.2, + 137.4, + 14.2 + ] + }, + "input_p50": { + "off": [ + 28.3, + 22.6, + 30.9 + ], + "on": [ + 27.2, + 32.1, + 25.2 + ] + }, + "input_p95": { + "off": [ + 38.3, + 32.7, + 103.7 + ], + "on": [ + 51.1, + 63.7, + 33.5 + ] + }, + "fps": { + "off": [ + 59.9, + 60.0, + 60.0 + ], + "on": [ + 60.0, + 48.7, + 59.7 + ] + }, + "late_pct": { + "off": [ + 2.05, + 0.33, + 0.99 + ], + "on": [ + 1.17, + 16.35, + 0.33 + ] + }, + "stall_max": { + "off": [ + 150.2, + 120.5, + 37.7 + ], + "on": [ + 75.6, + 267.4, + 122.8 + ] + }, + "mbps": { + "off": [ + 0.47, + 0.48, + 0.47 + ], + "on": [ + 0.47, + 0.36, + 0.47 + ] + } + }, + "scroll": { + "content_p50": { + "off": [ + 20.0, + 19.5, + 20.5 + ], + "on": [ + 19.5, + 19.6, + 20.5 + ] + }, + "content_p95": { + "off": [ + 32.3, + 25.2, + 45.9 + ], + "on": [ + 31.7, + 37.9, + 33.6 + ] + }, + "input_p50": { + "off": [], + "on": [] + }, + "input_p95": { + "off": [], + "on": [] + }, + "fps": { + "off": [ + 54.9, + 57.0, + 57.3 + ], + "on": [ + 56.4, + 56.2, + 55.6 + ] + }, + "late_pct": { + "off": [ + 6.59, + 3.36, + 4.6 + ], + "on": [ + 5.6, + 5.52, + 6.29 + ] + }, + "stall_max": { + "off": [ + 84.3, + 105.4, + 321.6 + ], + "on": [ + 120.5, + 147.0, + 153.6 + ] + }, + "mbps": { + "off": [ + 9.44, + 9.44, + 9.4 + ], + "on": [ + 9.4, + 9.36, + 9.25 + ] + } + } + } +} \ No newline at end of file diff --git a/bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-step3.json b/bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-step3.json new file mode 100644 index 0000000..332db76 --- /dev/null +++ b/bench/results/2026-09-28-a3c6e5c-dirty-ab-adapt-step3.json @@ -0,0 +1,3743 @@ +{ + "label": "ab-adapt-step3", + "date": "2026-09-28T20:24:10", + "commit": "a3c6e5c+dirty", + "config": { + "quality": "balanced", + "mode": "separate", + "net": "50@0,3@8,50@16", + "delay_ms": 0, + "buffer_ms": 250, + "host": "frame", + "ssh_opts": [], + "encoder": "", + "duration_s": 24.0, + "browser_flags": [] + }, + "frame_build": "20260925.6191901", + "mac": "26.5.2", + "headset": "8:23.135290 [Info] - 0 - leaving standby", + "scenarios": [], + "arms": [ + "off:FRAME_MAC_VIEW_ADAPT=0", + "on:" + ], + "runs": { + "off": [ + [ + { + "scenario": "scroll", + "frames_sent": 1179, + "frames_drawn": 1110, + "frames_shown": 576, + "duration_s": 24.2, + "stages_ms": { + "capture": { + "p50": -4.7, + "p95": 0.5, + "p99": 4.3, + "n": 1179 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.3, + "n": 1179 + }, + "encode": { + "p50": 6.6, + "p95": 8.7, + "p99": 13.8, + "n": 1179 + }, + "socket": { + "p50": 0.1, + "p95": 1690.5, + "p99": 2066.6, + "n": 1179 + }, + "network": { + "p50": 13.6, + "p95": 4872.0, + "p99": 5491.7, + "n": 1179 + }, + "decode": { + "p50": 6.0, + "p95": 46.0, + "p99": 75.8, + "n": 1110 + }, + "draw": { + "p50": 0.7, + "p95": 1.3, + "p99": 2.0, + "n": 1110 + }, + "present": { + "p50": 5.1, + "p95": 34.3, + "p99": 44.3, + "n": 576 + }, + "content": { + "p50": 27.5, + "p95": 4986.4, + "p99": 5514.6, + "n": 1110 + }, + "content_shown": { + "p50": 36.0, + "p95": 4759.3, + "p99": 5454.2, + "n": 576 + } + }, + "fps": 45.8, + "fps_shown": 23.8, + "late_pct": 19.19, + "stall_max": 232.1, + "stalls_over_100ms": 5, + "mbps": 8.62, + "keyframes": 3, + "size": "1920x1290", + "captured": 1500, + "viewer_never_drawn": 69, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 55, + "mbps": 9.44, + "content_p50": 22.8, + "content_p95": 36.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 1, + "fps": 57, + "mbps": 9.75, + "content_p50": 25.3, + "content_p95": 139.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 2, + "fps": 58, + "mbps": 9.8, + "content_p50": 24.2, + "content_p95": 110.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 3, + "fps": 58, + "mbps": 9.52, + "content_p50": 22.5, + "content_p95": 37.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 4, + "fps": 56, + "mbps": 8.91, + "content_p50": 23.4, + "content_p95": 30.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 5, + "fps": 53, + "mbps": 8.91, + "content_p50": 23.4, + "content_p95": 77.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 6, + "fps": 58, + "mbps": 12.0, + "content_p50": 22.9, + "content_p95": 54.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 7, + "fps": 58, + "mbps": 8.82, + "content_p50": 21.8, + "content_p95": 28.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 8, + "fps": 58, + "mbps": 9.48, + "content_p50": 1079.9, + "content_p95": 2016.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 9, + "fps": 57, + "mbps": 9.19, + "content_p50": 3208.0, + "content_p95": 4125.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 10, + "fps": 58, + "mbps": 8.91, + "content_p50": 5139.1, + "content_p95": 5585.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 11, + "fps": 29, + "mbps": 6.33, + "content_p50": 5245.6, + "content_p95": 5430.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 12, + "fps": 10, + "mbps": 2.92, + "content_p50": 4133.7, + "content_p95": 4182.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 13, + "fps": 9, + "mbps": 4.39, + "content_p50": 3552.8, + "content_p95": 3589.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 14, + "fps": 25, + "mbps": 5.77, + "content_p50": 2820.8, + "content_p95": 2926.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 15, + "fps": 10, + "mbps": 2.52, + "content_p50": 1747.5, + "content_p95": 1791.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 16, + "fps": 13, + "mbps": 10.51, + "content_p50": 1139.0, + "content_p95": 1179.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 17, + "fps": 31, + "mbps": 13.74, + "content_p50": 164.3, + "content_p95": 295.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 18, + "fps": 57, + "mbps": 9.05, + "content_p50": 24.4, + "content_p95": 30.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 19, + "fps": 57, + "mbps": 9.45, + "content_p50": 29.5, + "content_p95": 43.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 20, + "fps": 58, + "mbps": 9.16, + "content_p50": 27.3, + "content_p95": 63.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 21, + "fps": 58, + "mbps": 9.75, + "content_p50": 26.9, + "content_p95": 40.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 22, + "fps": 57, + "mbps": 9.45, + "content_p50": 28.5, + "content_p95": 34.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 23, + "fps": 58, + "mbps": 9.21, + "content_p50": 25.9, + "content_p95": 30.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 24, + "fps": 12, + "mbps": 1.9, + "content_p50": 25.9, + "content_p95": 28.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + } + ], + "adapt": [ + { + "at": 8.0, + "to_mbit": 3.0, + "p95_bound": 63.8, + "worst_p95": 5585.2, + "settled_s": 10.0 + } + ], + "content_p50": 27.5, + "content_p95": 4986.4, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "acceptable", + "content_p95": "bad", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "acceptable", + "adapt@8s": "bad" + }, + "controller": { + "adapt": false, + "baseRtt": 10.625, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 40, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 62.0, + "cpu": { + "mac_agent_pct": 7.7, + "frame_viewer_pct": 62.6, + "frame_tailscaled_pct": 20.6, + "frame_sshd_pct": 2.4, + "frame_gamescope_pct": 9.7, + "frame_total_pct": 44.8 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 71 Hz", + "show_s": 1.28, + "panel": "valve.steam.desktopgame.2001525774", + "src": "separate:6129" + }, + { + "scenario": "test", + "frames_sent": 1464, + "frames_drawn": 1464, + "frames_shown": 1198, + "duration_s": 24.4, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 1464 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 1464 + }, + "encode": { + "p50": 4.2, + "p95": 4.9, + "p99": 5.2, + "n": 1464 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.3, + "n": 1464 + }, + "network": { + "p50": 6.8, + "p95": 102.3, + "p99": 162.7, + "n": 1464 + }, + "decode": { + "p50": 1.1, + "p95": 10.7, + "p99": 16.6, + "n": 1464 + }, + "draw": { + "p50": 0.3, + "p95": 0.8, + "p99": 1.2, + "n": 1464 + }, + "present": { + "p50": 0.6, + "p95": 12.6, + "p99": 15.1, + "n": 1198 + }, + "content": { + "p50": 12.8, + "p95": 116.8, + "p99": 168.7, + "n": 1464 + }, + "content_shown": { + "p50": 14.7, + "p95": 30.1, + "p99": 173.8, + "n": 1198 + } + }, + "fps": 60.0, + "fps_shown": 49.1, + "late_pct": 3.69, + "stall_max": 185.4, + "stalls_over_100ms": 16, + "mbps": 0.49, + "keyframes": 2, + "size": "1280x720", + "captured": 1584, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 34.4, + "p95": 86.0, + "p99": 207.9, + "n": 48 + }, + "input_shown_ms": { + "p50": 34.9, + "p95": 86.7, + "p99": 208.5, + "n": 46 + }, + "input_parts_ms": { + "uplink": { + "p50": 13.9, + "p95": 29.5, + "p99": 55.9, + "n": 48 + }, + "mac": { + "p50": 6.5, + "p95": 16.2, + "p99": 18.2, + "n": 48 + }, + "back": { + "p50": 12.8, + "p95": 48.6, + "p99": 178.9, + "n": 48 + } + }, + "inputs": { + "sent": 48, + "seen": 48 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.52, + "content_p50": 11.7, + "content_p95": 15.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 1, + "fps": 60, + "mbps": 0.48, + "content_p50": 11.8, + "content_p95": 16.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 2, + "fps": 60, + "mbps": 0.48, + "content_p50": 11.2, + "content_p95": 16.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 3, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.3, + "content_p95": 14.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 4, + "fps": 60, + "mbps": 0.48, + "content_p50": 11.1, + "content_p95": 15.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 5, + "fps": 60, + "mbps": 0.5, + "content_p50": 10.7, + "content_p95": 13.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 6, + "fps": 60, + "mbps": 0.59, + "content_p50": 11.6, + "content_p95": 15.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 7, + "fps": 60, + "mbps": 0.5, + "content_p50": 11.9, + "content_p95": 14.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 8, + "fps": 60, + "mbps": 0.49, + "content_p50": 15.7, + "content_p95": 19.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 9, + "fps": 60, + "mbps": 0.47, + "content_p50": 15.3, + "content_p95": 19.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 10, + "fps": 60, + "mbps": 0.47, + "content_p50": 16.1, + "content_p95": 55.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 11, + "fps": 60, + "mbps": 0.48, + "content_p50": 23.0, + "content_p95": 71.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 12, + "fps": 60, + "mbps": 0.49, + "content_p50": 72.3, + "content_p95": 174.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 13, + "fps": 60, + "mbps": 0.5, + "content_p50": 79.0, + "content_p95": 173.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 14, + "fps": 60, + "mbps": 0.48, + "content_p50": 48.3, + "content_p95": 163.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 15, + "fps": 60, + "mbps": 0.48, + "content_p50": 68.1, + "content_p95": 168.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 16, + "fps": 60, + "mbps": 0.57, + "content_p50": 13.8, + "content_p95": 125.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 17, + "fps": 60, + "mbps": 0.48, + "content_p50": 13.7, + "content_p95": 19.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 18, + "fps": 60, + "mbps": 0.46, + "content_p50": 11.6, + "content_p95": 14.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 19, + "fps": 60, + "mbps": 0.48, + "content_p50": 12.1, + "content_p95": 16.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 20, + "fps": 60, + "mbps": 0.47, + "content_p50": 12.1, + "content_p95": 21.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 21, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.0, + "content_p95": 14.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 22, + "fps": 60, + "mbps": 0.47, + "content_p50": 12.5, + "content_p95": 15.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 23, + "fps": 60, + "mbps": 0.47, + "content_p50": 9.8, + "content_p95": 13.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 24, + "fps": 24, + "mbps": 0.17, + "content_p50": 10.2, + "content_p95": 14.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + } + ], + "adapt": [ + { + "at": 8.0, + "to_mbit": 3.0, + "p95_bound": 40.0, + "worst_p95": 174.5, + "settled_s": 9.0 + } + ], + "content_p50": 12.8, + "content_p95": 116.8, + "input_p50": 34.4, + "input_p95": 86.0, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "bad", + "input_p50": "local", + "input_p95": "acceptable", + "fps": "local", + "late_pct": "acceptable", + "stall_max": "acceptable", + "adapt@8s": "bad" + }, + "controller": { + "adapt": false, + "baseRtt": 6.042, + "ceiling": 5529600, + "fps": 60, + "inFlight": 2, + "scale": 1, + "slack": 40, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 64.9, + "cpu": { + "mac_agent_pct": 14.9, + "frame_viewer_pct": 43.7, + "frame_tailscaled_pct": 21.3, + "frame_sshd_pct": -1.7, + "frame_gamescope_pct": 9.8, + "frame_total_pct": 42.4 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 72 Hz", + "show_s": 1.42, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + } + ], + [ + { + "scenario": "scroll", + "frames_sent": 1168, + "frames_drawn": 1081, + "frames_shown": 565, + "duration_s": 24.3, + "stages_ms": { + "capture": { + "p50": -7.8, + "p95": 0.4, + "p99": 3.8, + "n": 1168 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 1168 + }, + "encode": { + "p50": 6.7, + "p95": 9.3, + "p99": 15.8, + "n": 1168 + }, + "socket": { + "p50": 0.1, + "p95": 1496.6, + "p99": 2773.0, + "n": 1168 + }, + "network": { + "p50": 9.8, + "p95": 4895.9, + "p99": 5418.1, + "n": 1168 + }, + "decode": { + "p50": 5.9, + "p95": 17.7, + "p99": 65.2, + "n": 1081 + }, + "draw": { + "p50": 0.6, + "p95": 1.3, + "p99": 1.8, + "n": 1081 + }, + "present": { + "p50": 5.3, + "p95": 30.8, + "p99": 41.9, + "n": 565 + }, + "content": { + "p50": 23.8, + "p95": 4355.1, + "p99": 5463.9, + "n": 1081 + }, + "content_shown": { + "p50": 32.9, + "p95": 4606.2, + "p99": 5408.3, + "n": 565 + } + }, + "fps": 44.5, + "fps_shown": 23.3, + "late_pct": 17.67, + "stall_max": 398.2, + "stalls_over_100ms": 2, + "mbps": 8.39, + "keyframes": 3, + "size": "1920x1290", + "captured": 1509, + "viewer_never_drawn": 87, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 55, + "mbps": 9.32, + "content_p50": 23.8, + "content_p95": 73.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 1, + "fps": 58, + "mbps": 9.52, + "content_p50": 22.8, + "content_p95": 28.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 2, + "fps": 58, + "mbps": 9.4, + "content_p50": 22.9, + "content_p95": 26.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 3, + "fps": 57, + "mbps": 9.62, + "content_p50": 23.6, + "content_p95": 33.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 4, + "fps": 58, + "mbps": 8.86, + "content_p50": 22.4, + "content_p95": 25.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 5, + "fps": 57, + "mbps": 9.67, + "content_p50": 23.2, + "content_p95": 29.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 6, + "fps": 58, + "mbps": 12.04, + "content_p50": 23.7, + "content_p95": 46.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 7, + "fps": 58, + "mbps": 8.85, + "content_p50": 22.5, + "content_p95": 27.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 8, + "fps": 57, + "mbps": 9.27, + "content_p50": 979.8, + "content_p95": 1991.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 9, + "fps": 58, + "mbps": 8.89, + "content_p50": 3131.5, + "content_p95": 3947.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 10, + "fps": 58, + "mbps": 9.25, + "content_p50": 5057.4, + "content_p95": 5532.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 11, + "fps": 5, + "mbps": 6.6, + "content_p50": 5369.2, + "content_p95": 5392.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 12, + "fps": 0, + "mbps": 3.01, + "content_p50": null, + "content_p95": null, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 13, + "fps": 0, + "mbps": 5.04, + "content_p50": null, + "content_p95": null, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 14, + "fps": 0, + "mbps": 1.64, + "content_p50": null, + "content_p95": null, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 15, + "fps": 0, + "mbps": 2.19, + "content_p50": null, + "content_p95": null, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 16, + "fps": 27, + "mbps": 13.34, + "content_p50": 332.4, + "content_p95": 451.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 17, + "fps": 57, + "mbps": 9.49, + "content_p50": 24.1, + "content_p95": 32.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 18, + "fps": 58, + "mbps": 9.06, + "content_p50": 23.0, + "content_p95": 28.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 19, + "fps": 58, + "mbps": 9.39, + "content_p50": 23.3, + "content_p95": 26.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 20, + "fps": 57, + "mbps": 8.89, + "content_p50": 23.4, + "content_p95": 32.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 21, + "fps": 58, + "mbps": 9.87, + "content_p50": 23.7, + "content_p95": 28.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 22, + "fps": 57, + "mbps": 9.49, + "content_p50": 22.2, + "content_p95": 29.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 23, + "fps": 57, + "mbps": 8.75, + "content_p50": 22.3, + "content_p95": 27.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 24, + "fps": 15, + "mbps": 2.32, + "content_p50": 22.0, + "content_p95": 25.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + } + ], + "adapt": [ + { + "at": 8.0, + "to_mbit": 3.0, + "p95_bound": 42.7, + "worst_p95": 5532.6, + "settled_s": 9.0 + } + ], + "content_p50": 23.8, + "content_p95": 4355.1, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "bad", + "fps": "bad", + "late_pct": "bad", + "stall_max": "bad", + "adapt@8s": "bad" + }, + "controller": { + "adapt": false, + "baseRtt": 10.292, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 40, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 62.1, + "cpu": { + "mac_agent_pct": 7.4, + "frame_viewer_pct": 65.4, + "frame_tailscaled_pct": 8.1, + "frame_sshd_pct": 0.9, + "frame_gamescope_pct": 11.5, + "frame_total_pct": 38.7 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 71 Hz", + "show_s": 1.29, + "panel": "valve.steam.desktopgame.2001170343", + "src": "separate:6551" + }, + { + "scenario": "test", + "frames_sent": 1459, + "frames_drawn": 1459, + "frames_shown": 572, + "duration_s": 24.3, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 1459 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 1459 + }, + "encode": { + "p50": 4.1, + "p95": 4.8, + "p99": 5.1, + "n": 1459 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.3, + "n": 1459 + }, + "network": { + "p50": 4.6, + "p95": 9.0, + "p99": 17.9, + "n": 1459 + }, + "decode": { + "p50": 1.1, + "p95": 2.6, + "p99": 4.2, + "n": 1459 + }, + "draw": { + "p50": 0.4, + "p95": 0.8, + "p99": 1.2, + "n": 1459 + }, + "present": { + "p50": 5.1, + "p95": 15.6, + "p99": 18.8, + "n": 572 + }, + "content": { + "p50": 10.7, + "p95": 15.6, + "p99": 24.6, + "n": 1459 + }, + "content_shown": { + "p50": 16.1, + "p95": 27.1, + "p99": 31.8, + "n": 572 + } + }, + "fps": 60.0, + "fps_shown": 23.5, + "late_pct": 1.92, + "stall_max": 157.3, + "stalls_over_100ms": 1, + "mbps": 0.46, + "keyframes": 2, + "size": "1280x720", + "captured": 1580, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 29.6, + "p95": 50.1, + "p99": 163.0, + "n": 48 + }, + "input_shown_ms": { + "p50": 36.5, + "p95": 52.4, + "p99": 163.6, + "n": 21 + }, + "input_parts_ms": { + "uplink": { + "p50": 12.0, + "p95": 21.1, + "p99": 34.4, + "n": 48 + }, + "mac": { + "p50": 5.5, + "p95": 15.1, + "p99": 16.3, + "n": 48 + }, + "back": { + "p50": 11.0, + "p95": 15.7, + "p99": 150.9, + "n": 48 + } + }, + "inputs": { + "sent": 48, + "seen": 48 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.3, + "content_p95": 14.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 1, + "fps": 60, + "mbps": 0.45, + "content_p50": 9.8, + "content_p95": 11.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 2, + "fps": 60, + "mbps": 0.45, + "content_p50": 10.1, + "content_p95": 12.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 3, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.0, + "content_p95": 12.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 4, + "fps": 60, + "mbps": 0.45, + "content_p50": 10.6, + "content_p95": 13.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 5, + "fps": 60, + "mbps": 0.45, + "content_p50": 9.9, + "content_p95": 11.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 6, + "fps": 60, + "mbps": 0.56, + "content_p50": 10.1, + "content_p95": 13.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 7, + "fps": 60, + "mbps": 0.44, + "content_p50": 9.8, + "content_p95": 13.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 8, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.8, + "content_p95": 16.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 9, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.4, + "content_p95": 14.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 10, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.8, + "content_p95": 14.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 11, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.7, + "content_p95": 15.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 12, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.9, + "content_p95": 17.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 13, + "fps": 60, + "mbps": 0.44, + "content_p50": 13.3, + "content_p95": 16.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 14, + "fps": 60, + "mbps": 0.44, + "content_p50": 13.4, + "content_p95": 107.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 15, + "fps": 60, + "mbps": 0.44, + "content_p50": 13.4, + "content_p95": 18.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 16, + "fps": 60, + "mbps": 0.55, + "content_p50": 9.5, + "content_p95": 13.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 17, + "fps": 60, + "mbps": 0.43, + "content_p50": 9.3, + "content_p95": 12.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 18, + "fps": 60, + "mbps": 0.44, + "content_p50": 9.8, + "content_p95": 13.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 19, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.5, + "content_p95": 13.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 20, + "fps": 60, + "mbps": 0.44, + "content_p50": 9.7, + "content_p95": 13.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 21, + "fps": 60, + "mbps": 0.44, + "content_p50": 9.3, + "content_p95": 13.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 22, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.7, + "content_p95": 12.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 23, + "fps": 60, + "mbps": 0.45, + "content_p50": 9.5, + "content_p95": 16.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 24, + "fps": 19, + "mbps": 0.13, + "content_p50": 10.2, + "content_p95": 13.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + } + ], + "adapt": [ + { + "at": 8.0, + "to_mbit": 3.0, + "p95_bound": 40.0, + "worst_p95": 107.3, + "settled_s": 0.0 + } + ], + "content_p50": 10.7, + "content_p95": 15.6, + "input_p50": 29.6, + "input_p95": 50.1, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "acceptable", + "stall_max": "acceptable", + "adapt@8s": "local" + }, + "controller": { + "adapt": false, + "baseRtt": 5.916, + "ceiling": 5529600, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 40, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 65.0, + "cpu": { + "mac_agent_pct": 14.7, + "frame_viewer_pct": 54.2, + "frame_tailscaled_pct": 6.0, + "frame_sshd_pct": 0.8, + "frame_gamescope_pct": 14.5, + "frame_total_pct": 39.3 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 70 Hz", + "show_s": 1.37, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + } + ] + ], + "on": [ + [ + { + "scenario": "scroll", + "frames_sent": 1012, + "frames_drawn": 1012, + "frames_shown": 975, + "duration_s": 24.2, + "stages_ms": { + "capture": { + "p50": -4.2, + "p95": 3.0, + "p99": 4.5, + "n": 1012 + }, + "queue": { + "p50": 0.0, + "p95": 15.9, + "p99": 17.4, + "n": 1012 + }, + "encode": { + "p50": 6.4, + "p95": 7.5, + "p99": 8.5, + "n": 1012 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.4, + "n": 1012 + }, + "network": { + "p50": 9.9, + "p95": 51.0, + "p99": 167.2, + "n": 1012 + }, + "decode": { + "p50": 5.2, + "p95": 8.7, + "p99": 13.2, + "n": 1012 + }, + "draw": { + "p50": 0.6, + "p95": 1.4, + "p99": 1.7, + "n": 1012 + }, + "present": { + "p50": 0.6, + "p95": 11.6, + "p99": 14.3, + "n": 975 + }, + "content": { + "p50": 26.6, + "p95": 69.2, + "p99": 189.9, + "n": 1012 + }, + "content_shown": { + "p50": 31.6, + "p95": 71.1, + "p99": 190.4, + "n": 975 + } + }, + "fps": 41.8, + "fps_shown": 40.3, + "late_pct": 31.82, + "stall_max": 245.3, + "stalls_over_100ms": 5, + "mbps": 6.13, + "keyframes": 3, + "size": "1920x1290", + "captured": 1507, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 56, + "mbps": 9.54, + "content_p50": 21.9, + "content_p95": 42.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 1, + "fps": 58, + "mbps": 9.38, + "content_p50": 21.4, + "content_p95": 26.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 2, + "fps": 57, + "mbps": 9.41, + "content_p50": 22.3, + "content_p95": 27.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 3, + "fps": 58, + "mbps": 9.67, + "content_p50": 23.0, + "content_p95": 25.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 4, + "fps": 57, + "mbps": 8.99, + "content_p50": 22.9, + "content_p95": 28.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 5, + "fps": 58, + "mbps": 9.25, + "content_p50": 23.3, + "content_p95": 32.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 6, + "fps": 57, + "mbps": 11.74, + "content_p50": 22.2, + "content_p95": 46.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 7, + "fps": 58, + "mbps": 8.89, + "content_p50": 21.1, + "content_p95": 25.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 8, + "fps": 25, + "mbps": 3.02, + "content_p50": 75.9, + "content_p95": 218.9, + "bitrate": 3715200, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 9, + "fps": 50, + "mbps": 3.15, + "content_p50": 56.5, + "content_p95": 84.0, + "bitrate": 2112376, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 10, + "fps": 24, + "mbps": 0.94, + "content_p50": 37.9, + "content_p95": 54.8, + "bitrate": 1689900, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 11, + "fps": 30, + "mbps": 1.93, + "content_p50": 48.0, + "content_p95": 57.0, + "bitrate": 2414756, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 12, + "fps": 30, + "mbps": 2.74, + "content_p50": 58.5, + "content_p95": 92.4, + "bitrate": 2611641, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 13, + "fps": 30, + "mbps": 2.12, + "content_p50": 45.3, + "content_p95": 79.0, + "bitrate": 2089312, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 14, + "fps": 30, + "mbps": 2.24, + "content_p50": 52.0, + "content_p95": 71.4, + "bitrate": 2633067, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 15, + "fps": 30, + "mbps": 2.47, + "content_p50": 53.7, + "content_p95": 93.6, + "bitrate": 2357098, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 16, + "fps": 30, + "mbps": 2.61, + "content_p50": 24.7, + "content_p95": 39.8, + "bitrate": 2957087, + "tier": 3, + "w": 1440, + "net": 50.0 + }, + { + "t": 17, + "fps": 30, + "mbps": 3.74, + "content_p50": 29.5, + "content_p95": 38.0, + "bitrate": 4561519, + "tier": 3, + "w": 1440, + "net": 50.0 + }, + { + "t": 18, + "fps": 30, + "mbps": 5.32, + "content_p50": 29.2, + "content_p95": 41.6, + "bitrate": 6236880, + "tier": 3, + "w": 1440, + "net": 50.0 + }, + { + "t": 19, + "fps": 30, + "mbps": 6.62, + "content_p50": 35.0, + "content_p95": 68.5, + "bitrate": 8466786, + "tier": 2, + "w": 1920, + "net": 50.0 + }, + { + "t": 20, + "fps": 30, + "mbps": 7.48, + "content_p50": 36.1, + "content_p95": 41.1, + "bitrate": 12628270, + "tier": 2, + "w": 1920, + "net": 50.0 + }, + { + "t": 21, + "fps": 42, + "mbps": 7.71, + "content_p50": 31.7, + "content_p95": 39.0, + "bitrate": 14860800, + "tier": 1, + "w": 1920, + "net": 50.0 + }, + { + "t": 22, + "fps": 46, + "mbps": 8.79, + "content_p50": 33.6, + "content_p95": 40.6, + "bitrate": 14860800, + "tier": 1, + "w": 1920, + "net": 50.0 + }, + { + "t": 23, + "fps": 53, + "mbps": 8.78, + "content_p50": 28.4, + "content_p95": 42.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 24, + "fps": 13, + "mbps": 1.89, + "content_p50": 24.7, + "content_p95": 28.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + } + ], + "adapt": [ + { + "at": 8.0, + "to_mbit": 3.0, + "p95_bound": 45.2, + "worst_p95": 218.9, + "settled_s": 8.0 + } + ], + "content_p50": 26.6, + "content_p95": 69.2, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "acceptable", + "content_p95": "acceptable", + "fps": "bad", + "late_pct": "bad", + "stall_max": "acceptable", + "adapt@8s": "bad" + }, + "controller": { + "adapt": true, + "baseRtt": 6.333, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 51.355, + "target": 14860800, + "tier": 0 + }, + "events": [ + { + "t": 8.31, + "e": "down to 7430 kbit/s: queue 100 ms, held 6, oldest 197 ms, base 10 ms, sent 5528 got 5268 wants 8906" + }, + { + "t": 8.71, + "e": "down to 3715 kbit/s: queue 209 ms, held 4, oldest 115 ms, base 10 ms, sent 2861 got 2421 wants 10373" + }, + { + "t": 9.41, + "e": "down to 2640 kbit/s: queue 48 ms, held 0, oldest 59 ms, base 10 ms, sent 3222 got 2933 wants 3166" + }, + { + "t": 9.71, + "e": "down to 2112 kbit/s: queue 48 ms, held 0, oldest 31 ms, base 10 ms, sent 2907 got 3032 wants 2906" + }, + { + "t": 9.81, + "e": "tier 3" + }, + { + "t": 10.21, + "e": "down to 1689 kbit/s: queue 186 ms, held 4, oldest 0 ms, base 10 ms, sent 2091 got 2288 wants 3137" + }, + { + "t": 12.91, + "e": "down to 2611 kbit/s: queue 44 ms, held 0, oldest 27 ms, base 10 ms, sent 3243 got 2901 wants 3243" + }, + { + "t": 13.31, + "e": "down to 2089 kbit/s: queue 41 ms, held 0, oldest 60 ms, base 10 ms, sent 2775 got 2764 wants 2775" + }, + { + "t": 15.31, + "e": "down to 2357 kbit/s: queue 44 ms, held 0, oldest 28 ms, base 10 ms, sent 2539 got 2729 wants 2539" + }, + { + "t": 19.11, + "e": "tier 2" + }, + { + "t": 21.31, + "e": "tier 1" + }, + { + "t": 23.51, + "e": "tier 0" + } + ], + "source_fps": 62.3, + "cpu": { + "mac_agent_pct": 7.3, + "frame_viewer_pct": 57.5, + "frame_tailscaled_pct": 7.4, + "frame_sshd_pct": 0.8, + "frame_gamescope_pct": 10.6, + "frame_total_pct": 47.8 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 71 Hz", + "show_s": 1.27, + "panel": "valve.steam.desktopgame.2001853011", + "src": "separate:6269" + }, + { + "scenario": "test", + "frames_sent": 1457, + "frames_drawn": 1457, + "frames_shown": 578, + "duration_s": 24.3, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 1457 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 1457 + }, + "encode": { + "p50": 4.2, + "p95": 4.9, + "p99": 5.2, + "n": 1457 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 1457 + }, + "network": { + "p50": 5.0, + "p95": 10.1, + "p99": 13.0, + "n": 1457 + }, + "decode": { + "p50": 1.0, + "p95": 2.2, + "p99": 3.6, + "n": 1457 + }, + "draw": { + "p50": 0.3, + "p95": 0.7, + "p99": 1.1, + "n": 1457 + }, + "present": { + "p50": 5.6, + "p95": 16.1, + "p99": 18.0, + "n": 578 + }, + "content": { + "p50": 11.1, + "p95": 16.2, + "p99": 19.2, + "n": 1457 + }, + "content_shown": { + "p50": 18.9, + "p95": 27.0, + "p99": 31.0, + "n": 578 + } + }, + "fps": 60.0, + "fps_shown": 23.8, + "late_pct": 0.82, + "stall_max": 32.9, + "stalls_over_100ms": 0, + "mbps": 0.47, + "keyframes": 2, + "size": "1280x720", + "captured": 1578, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 30.0, + "p95": 47.7, + "p99": 76.0, + "n": 48 + }, + "input_shown_ms": { + "p50": 33.7, + "p95": 47.5, + "p99": 81.8, + "n": 17 + }, + "input_parts_ms": { + "uplink": { + "p50": 10.2, + "p95": 27.4, + "p99": 49.5, + "n": 48 + }, + "mac": { + "p50": 5.3, + "p95": 15.1, + "p99": 16.3, + "n": 48 + }, + "back": { + "p50": 12.5, + "p95": 18.7, + "p99": 19.2, + "n": 48 + } + }, + "inputs": { + "sent": 48, + "seen": 48 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.49, + "content_p50": 10.0, + "content_p95": 11.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 1, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.5, + "content_p95": 15.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 2, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.1, + "content_p95": 12.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 3, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.6, + "content_p95": 13.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 4, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.7, + "content_p95": 13.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 5, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.8, + "content_p95": 12.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 6, + "fps": 60, + "mbps": 0.56, + "content_p50": 10.8, + "content_p95": 13.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 7, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.9, + "content_p95": 12.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 8, + "fps": 60, + "mbps": 0.45, + "content_p50": 13.4, + "content_p95": 15.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 9, + "fps": 60, + "mbps": 0.46, + "content_p50": 13.8, + "content_p95": 16.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 10, + "fps": 60, + "mbps": 0.45, + "content_p50": 14.1, + "content_p95": 16.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 11, + "fps": 60, + "mbps": 0.45, + "content_p50": 13.8, + "content_p95": 22.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 12, + "fps": 60, + "mbps": 0.45, + "content_p50": 14.3, + "content_p95": 18.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 13, + "fps": 60, + "mbps": 0.45, + "content_p50": 14.2, + "content_p95": 18.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 14, + "fps": 60, + "mbps": 0.45, + "content_p50": 14.1, + "content_p95": 18.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 15, + "fps": 60, + "mbps": 0.47, + "content_p50": 14.2, + "content_p95": 17.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 16, + "fps": 60, + "mbps": 0.57, + "content_p50": 9.9, + "content_p95": 18.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 17, + "fps": 60, + "mbps": 0.45, + "content_p50": 9.9, + "content_p95": 11.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 18, + "fps": 60, + "mbps": 0.45, + "content_p50": 10.0, + "content_p95": 12.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 19, + "fps": 60, + "mbps": 0.45, + "content_p50": 10.0, + "content_p95": 11.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 20, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.7, + "content_p95": 11.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 21, + "fps": 60, + "mbps": 0.45, + "content_p50": 9.8, + "content_p95": 11.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 22, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.9, + "content_p95": 12.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 23, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.9, + "content_p95": 12.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 24, + "fps": 17, + "mbps": 0.11, + "content_p50": 10.9, + "content_p95": 12.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + } + ], + "adapt": [ + { + "at": 8.0, + "to_mbit": 3.0, + "p95_bound": 40.0, + "worst_p95": 22.7, + "settled_s": 0.0 + } + ], + "content_p50": 11.1, + "content_p95": 16.2, + "input_p50": 30.0, + "input_p95": 47.7, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "local", + "stall_max": "local", + "adapt@8s": "local" + }, + "controller": { + "adapt": true, + "baseRtt": 6.083, + "ceiling": 5529600, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 47.104, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 64.9, + "cpu": { + "mac_agent_pct": 15.6, + "frame_viewer_pct": 49.6, + "frame_tailscaled_pct": 5.8, + "frame_sshd_pct": 0.7, + "frame_gamescope_pct": 11.3, + "frame_total_pct": 37.4 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 70 Hz", + "show_s": 1.29, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + } + ], + [ + { + "scenario": "scroll", + "frames_sent": 1044, + "frames_drawn": 1044, + "frames_shown": 786, + "duration_s": 24.3, + "stages_ms": { + "capture": { + "p50": -4.6, + "p95": 0.4, + "p99": 1.8, + "n": 1044 + }, + "queue": { + "p50": 0.0, + "p95": 14.9, + "p99": 17.0, + "n": 1044 + }, + "encode": { + "p50": 6.5, + "p95": 8.8, + "p99": 17.0, + "n": 1044 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.5, + "n": 1044 + }, + "network": { + "p50": 10.1, + "p95": 52.0, + "p99": 225.2, + "n": 1044 + }, + "decode": { + "p50": 5.4, + "p95": 9.5, + "p99": 17.6, + "n": 1044 + }, + "draw": { + "p50": 0.6, + "p95": 1.3, + "p99": 1.7, + "n": 1044 + }, + "present": { + "p50": 0.7, + "p95": 14.4, + "p99": 22.3, + "n": 786 + }, + "content": { + "p50": 25.5, + "p95": 74.6, + "p99": 236.2, + "n": 1044 + }, + "content_shown": { + "p50": 33.0, + "p95": 74.9, + "p99": 234.6, + "n": 786 + } + }, + "fps": 43.0, + "fps_shown": 32.4, + "late_pct": 27.59, + "stall_max": 430.7, + "stalls_over_100ms": 7, + "mbps": 6.22, + "keyframes": 3, + "size": "1920x1290", + "captured": 1499, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 54, + "mbps": 9.98, + "content_p50": 23.9, + "content_p95": 33.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 1, + "fps": 57, + "mbps": 9.24, + "content_p50": 23.4, + "content_p95": 32.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 2, + "fps": 57, + "mbps": 9.09, + "content_p50": 23.0, + "content_p95": 32.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 3, + "fps": 54, + "mbps": 9.77, + "content_p50": 24.1, + "content_p95": 36.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 4, + "fps": 58, + "mbps": 9.09, + "content_p50": 23.2, + "content_p95": 27.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 5, + "fps": 52, + "mbps": 8.72, + "content_p50": 24.5, + "content_p95": 63.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 6, + "fps": 58, + "mbps": 11.69, + "content_p50": 23.1, + "content_p95": 50.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 7, + "fps": 57, + "mbps": 8.99, + "content_p50": 22.4, + "content_p95": 32.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 8, + "fps": 20, + "mbps": 3.23, + "content_p50": 205.5, + "content_p95": 270.8, + "bitrate": 7430400, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 9, + "fps": 34, + "mbps": 1.66, + "content_p50": 18.4, + "content_p95": 428.1, + "bitrate": 1765116, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 10, + "fps": 57, + "mbps": 1.27, + "content_p50": 23.3, + "content_p95": 101.0, + "bitrate": 2240789, + "tier": 0, + "w": 1920, + "net": 3.0 + }, + { + "t": 11, + "fps": 35, + "mbps": 1.73, + "content_p50": 31.8, + "content_p95": 148.1, + "bitrate": 1405577, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 12, + "fps": 30, + "mbps": 1.26, + "content_p50": 38.8, + "content_p95": 64.3, + "bitrate": 1596134, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 13, + "fps": 31, + "mbps": 1.84, + "content_p50": 43.3, + "content_p95": 59.6, + "bitrate": 2289953, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 14, + "fps": 30, + "mbps": 2.61, + "content_p50": 59.7, + "content_p95": 85.4, + "bitrate": 2373537, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 15, + "fps": 30, + "mbps": 2.39, + "content_p50": 53.7, + "content_p95": 87.6, + "bitrate": 2660890, + "tier": 3, + "w": 1440, + "net": 3.0 + }, + { + "t": 16, + "fps": 30, + "mbps": 3.04, + "content_p50": 25.4, + "content_p95": 37.6, + "bitrate": 3707143, + "tier": 3, + "w": 1440, + "net": 50.0 + }, + { + "t": 17, + "fps": 30, + "mbps": 4.45, + "content_p50": 31.8, + "content_p95": 41.7, + "bitrate": 5099706, + "tier": 3, + "w": 1440, + "net": 50.0 + }, + { + "t": 18, + "fps": 28, + "mbps": 5.76, + "content_p50": 34.2, + "content_p95": 51.9, + "bitrate": 7698527, + "tier": 2, + "w": 1920, + "net": 50.0 + }, + { + "t": 19, + "fps": 30, + "mbps": 7.38, + "content_p50": 37.7, + "content_p95": 45.2, + "bitrate": 10412237, + "tier": 2, + "w": 1920, + "net": 50.0 + }, + { + "t": 20, + "fps": 41, + "mbps": 8.63, + "content_p50": 34.5, + "content_p95": 64.1, + "bitrate": 14024186, + "tier": 1, + "w": 1920, + "net": 50.0 + }, + { + "t": 21, + "fps": 46, + "mbps": 7.87, + "content_p50": 35.1, + "content_p95": 45.4, + "bitrate": 14860800, + "tier": 1, + "w": 1920, + "net": 50.0 + }, + { + "t": 22, + "fps": 53, + "mbps": 9.21, + "content_p50": 24.4, + "content_p95": 37.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 23, + "fps": 57, + "mbps": 9.68, + "content_p50": 23.2, + "content_p95": 31.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + }, + { + "t": 24, + "fps": 15, + "mbps": 2.34, + "content_p50": 25.1, + "content_p95": 34.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": 50.0 + } + ], + "adapt": [ + { + "at": 8.0, + "to_mbit": 3.0, + "p95_bound": 62.6, + "worst_p95": 428.1, + "settled_s": 8.0 + } + ], + "content_p50": 25.5, + "content_p95": 74.6, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "acceptable", + "content_p95": "acceptable", + "fps": "bad", + "late_pct": "bad", + "stall_max": "bad", + "adapt@8s": "bad" + }, + "controller": { + "adapt": true, + "baseRtt": 7.875, + "ceiling": 14860800, + "fps": 60, + "inFlight": 4, + "scale": 1, + "slack": 48.916, + "target": 14860800, + "tier": 0 + }, + "events": [ + { + "t": 8.32, + "e": "down to 7430 kbit/s: queue 98 ms, held 6, oldest 191 ms, base 9 ms, sent 6068 got 6112 wants 9608" + }, + { + "t": 8.92, + "e": "down to 3715 kbit/s: queue 197 ms, held 5, oldest 239 ms, base 9 ms, sent 2451 got 2882 wants 8733" + }, + { + "t": 9.22, + "e": "down to 2206 kbit/s: queue 237 ms, held 4, oldest 161 ms, base 9 ms, sent 3261 got 2451 wants 10510" + }, + { + "t": 9.52, + "e": "down to 1765 kbit/s: queue 392 ms, held 6, oldest 456 ms, base 9 ms, sent 2553 got 2705 wants 10396" + }, + { + "t": 11.22, + "e": "down to 1756 kbit/s: queue 48 ms, held 0, oldest 3 ms, base 8 ms, sent 1996 got 1952 wants 2107" + }, + { + "t": 11.52, + "e": "tier 3" + }, + { + "t": 11.92, + "e": "down to 1405 kbit/s: queue 158 ms, held 2, oldest 0 ms, base 8 ms, sent 1907 got 1907 wants 2384" + }, + { + "t": 14.92, + "e": "down to 2373 kbit/s: queue 47 ms, held 0, oldest 58 ms, base 8 ms, sent 2973 got 2637 wants 2972" + }, + { + "t": 18.22, + "e": "tier 2" + }, + { + "t": 20.32, + "e": "tier 1" + }, + { + "t": 22.42, + "e": "tier 0" + } + ], + "source_fps": 61.7, + "cpu": { + "mac_agent_pct": 7.1, + "frame_viewer_pct": 59.9, + "frame_tailscaled_pct": 7.4, + "frame_sshd_pct": 0.7, + "frame_gamescope_pct": 9.8, + "frame_total_pct": 37.2 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 70 Hz", + "show_s": 1.32, + "panel": "valve.steam.desktopgame.2001220345", + "src": "separate:6403" + }, + { + "scenario": "test", + "frames_sent": 1455, + "frames_drawn": 1455, + "frames_shown": 569, + "duration_s": 24.3, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 1455 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 1455 + }, + "encode": { + "p50": 4.0, + "p95": 4.7, + "p99": 5.0, + "n": 1455 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 1455 + }, + "network": { + "p50": 5.7, + "p95": 9.8, + "p99": 13.3, + "n": 1455 + }, + "decode": { + "p50": 1.1, + "p95": 2.6, + "p99": 3.9, + "n": 1455 + }, + "draw": { + "p50": 0.4, + "p95": 0.8, + "p99": 1.2, + "n": 1455 + }, + "present": { + "p50": 5.7, + "p95": 17.2, + "p99": 19.5, + "n": 569 + }, + "content": { + "p50": 11.5, + "p95": 15.9, + "p99": 19.4, + "n": 1455 + }, + "content_shown": { + "p50": 17.6, + "p95": 29.4, + "p99": 31.2, + "n": 569 + } + }, + "fps": 59.8, + "fps_shown": 23.4, + "late_pct": 1.1, + "stall_max": 115.1, + "stalls_over_100ms": 1, + "mbps": 0.47, + "keyframes": 2, + "size": "1280x720", + "captured": 1580, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 29.1, + "p95": 50.1, + "p99": 55.5, + "n": 48 + }, + "input_shown_ms": { + "p50": 44.2, + "p95": 51.9, + "p99": 55.9, + "n": 15 + }, + "input_parts_ms": { + "uplink": { + "p50": 10.1, + "p95": 23.2, + "p99": 27.8, + "n": 48 + }, + "mac": { + "p50": 6.0, + "p95": 17.1, + "p99": 20.1, + "n": 48 + }, + "back": { + "p50": 11.6, + "p95": 17.2, + "p99": 18.3, + "n": 48 + } + }, + "inputs": { + "sent": 48, + "seen": 48 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.9, + "content_p95": 13.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 1, + "fps": 60, + "mbps": 0.47, + "content_p50": 11.0, + "content_p95": 13.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 2, + "fps": 60, + "mbps": 0.47, + "content_p50": 11.0, + "content_p95": 14.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 3, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.9, + "content_p95": 15.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 4, + "fps": 61, + "mbps": 0.48, + "content_p50": 11.4, + "content_p95": 18.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 5, + "fps": 59, + "mbps": 0.47, + "content_p50": 10.8, + "content_p95": 13.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 6, + "fps": 61, + "mbps": 0.58, + "content_p50": 11.3, + "content_p95": 25.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 7, + "fps": 59, + "mbps": 0.45, + "content_p50": 11.0, + "content_p95": 13.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 8, + "fps": 61, + "mbps": 0.48, + "content_p50": 13.9, + "content_p95": 17.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 9, + "fps": 59, + "mbps": 0.45, + "content_p50": 13.8, + "content_p95": 18.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 10, + "fps": 61, + "mbps": 0.45, + "content_p50": 13.9, + "content_p95": 16.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 11, + "fps": 59, + "mbps": 0.45, + "content_p50": 11.8, + "content_p95": 15.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 12, + "fps": 61, + "mbps": 0.46, + "content_p50": 13.4, + "content_p95": 16.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 13, + "fps": 59, + "mbps": 0.45, + "content_p50": 13.4, + "content_p95": 16.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 14, + "fps": 57, + "mbps": 0.43, + "content_p50": 13.2, + "content_p95": 18.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 15, + "fps": 59, + "mbps": 0.45, + "content_p50": 13.4, + "content_p95": 16.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 3.0 + }, + { + "t": 16, + "fps": 61, + "mbps": 0.56, + "content_p50": 10.5, + "content_p95": 15.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 17, + "fps": 59, + "mbps": 0.43, + "content_p50": 10.8, + "content_p95": 14.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 18, + "fps": 61, + "mbps": 0.46, + "content_p50": 10.7, + "content_p95": 13.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 19, + "fps": 59, + "mbps": 0.45, + "content_p50": 10.7, + "content_p95": 13.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 20, + "fps": 61, + "mbps": 0.46, + "content_p50": 10.5, + "content_p95": 15.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 21, + "fps": 59, + "mbps": 0.44, + "content_p50": 10.7, + "content_p95": 13.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 22, + "fps": 60, + "mbps": 0.45, + "content_p50": 10.8, + "content_p95": 12.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 23, + "fps": 60, + "mbps": 0.45, + "content_p50": 10.6, + "content_p95": 12.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + }, + { + "t": 24, + "fps": 19, + "mbps": 0.13, + "content_p50": 10.6, + "content_p95": 16.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": 50.0 + } + ], + "adapt": [ + { + "at": 8.0, + "to_mbit": 3.0, + "p95_bound": 40.0, + "worst_p95": 18.3, + "settled_s": 0.0 + } + ], + "content_p50": 11.5, + "content_p95": 15.9, + "input_p50": 29.1, + "input_p95": 50.1, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "acceptable", + "stall_max": "acceptable", + "adapt@8s": "local" + }, + "controller": { + "adapt": true, + "baseRtt": 6, + "ceiling": 5529600, + "fps": 60, + "inFlight": 2, + "scale": 1, + "slack": 49.792, + "target": 5529600, + "tier": 0 + }, + "events": [], + "source_fps": 65.0, + "cpu": { + "mac_agent_pct": 14.2, + "frame_viewer_pct": 52.2, + "frame_tailscaled_pct": 5.8, + "frame_sshd_pct": 0.8, + "frame_gamescope_pct": 12.2, + "frame_total_pct": 38.8 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 72 Hz", + "show_s": 1.38, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + } + ] + ] + }, + "table": { + "scroll": { + "content_p50": { + "off": [ + 27.5, + 23.8 + ], + "on": [ + 26.6, + 25.5 + ] + }, + "content_p95": { + "off": [ + 4986.4, + 4355.1 + ], + "on": [ + 69.2, + 74.6 + ] + }, + "input_p50": { + "off": [], + "on": [] + }, + "input_p95": { + "off": [], + "on": [] + }, + "fps": { + "off": [ + 45.8, + 44.5 + ], + "on": [ + 41.8, + 43.0 + ] + }, + "late_pct": { + "off": [ + 19.19, + 17.67 + ], + "on": [ + 31.82, + 27.59 + ] + }, + "stall_max": { + "off": [ + 232.1, + 398.2 + ], + "on": [ + 245.3, + 430.7 + ] + }, + "mbps": { + "off": [ + 8.62, + 8.39 + ], + "on": [ + 6.13, + 6.22 + ] + } + }, + "test": { + "content_p50": { + "off": [ + 12.8, + 10.7 + ], + "on": [ + 11.1, + 11.5 + ] + }, + "content_p95": { + "off": [ + 116.8, + 15.6 + ], + "on": [ + 16.2, + 15.9 + ] + }, + "input_p50": { + "off": [ + 34.4, + 29.6 + ], + "on": [ + 30.0, + 29.1 + ] + }, + "input_p95": { + "off": [ + 86.0, + 50.1 + ], + "on": [ + 47.7, + 50.1 + ] + }, + "fps": { + "off": [ + 60.0, + 60.0 + ], + "on": [ + 60.0, + 59.8 + ] + }, + "late_pct": { + "off": [ + 3.69, + 1.92 + ], + "on": [ + 0.82, + 1.1 + ] + }, + "stall_max": { + "off": [ + 185.4, + 157.3 + ], + "on": [ + 32.9, + 115.1 + ] + }, + "mbps": { + "off": [ + 0.49, + 0.46 + ], + "on": [ + 0.47, + 0.47 + ] + } + } + } +} \ No newline at end of file diff --git a/bench/results/2026-09-28-a3c6e5c-dirty-ab-keepwarm.json b/bench/results/2026-09-28-a3c6e5c-dirty-ab-keepwarm.json new file mode 100644 index 0000000..21c9e37 --- /dev/null +++ b/bench/results/2026-09-28-a3c6e5c-dirty-ab-keepwarm.json @@ -0,0 +1,2172 @@ +{ + "label": "ab-keepwarm", + "date": "2026-09-28T20:28:57", + "commit": "a3c6e5c+dirty", + "config": { + "quality": "balanced", + "mode": "separate", + "net": "none", + "delay_ms": 0, + "buffer_ms": 250, + "host": "frame", + "ssh_opts": [], + "encoder": "", + "duration_s": 20.0, + "browser_flags": [] + }, + "frame_build": "20260925.6191901", + "mac": "26.5.2", + "headset": "8:23.135290 [Info] - 0 - leaving standby", + "scenarios": [], + "arms": [ + "cold:FRAME_MAC_VIEW_WARM=0", + "warm:FRAME_MAC_VIEW_WARM=40" + ], + "runs": { + "cold": [ + [ + { + "scenario": "type", + "frames_sent": 145, + "frames_drawn": 145, + "frames_shown": 142, + "duration_s": 21.1, + "stages_ms": { + "capture": { + "p50": -2.5, + "p95": 2.7, + "p99": 19.4, + "n": 145 + }, + "queue": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.3, + "n": 145 + }, + "encode": { + "p50": 6.5, + "p95": 9.6, + "p99": 18.2, + "n": 145 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 1.0, + "n": 145 + }, + "network": { + "p50": 3.7, + "p95": 7.5, + "p99": 22.8, + "n": 145 + }, + "decode": { + "p50": 2.1, + "p95": 4.2, + "p99": 5.1, + "n": 145 + }, + "draw": { + "p50": 0.6, + "p95": 1.3, + "p99": 2.0, + "n": 145 + }, + "present": { + "p50": 0.5, + "p95": 9.3, + "p99": 12.6, + "n": 142 + }, + "content": { + "p50": 13.9, + "p95": 25.6, + "p99": 38.0, + "n": 145 + }, + "content_shown": { + "p50": 14.9, + "p95": 26.0, + "p99": 36.9, + "n": 142 + } + }, + "fps": 6.9, + "fps_shown": 6.7, + "late_pct": 51.37, + "stall_max": 1007.5, + "stalls_over_100ms": 66, + "mbps": 0.03, + "keyframes": 0, + "size": "1920x1290", + "captured": 145, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 44.5, + "p95": 68.2, + "p99": 72.7, + "n": 66 + }, + "input_shown_ms": { + "p50": 44.9, + "p95": 68.9, + "p99": 73.1, + "n": 66 + }, + "input_parts_ms": { + "uplink": { + "p50": 4.2, + "p95": 17.4, + "p99": 41.0, + "n": 66 + }, + "mac": { + "p50": 22.8, + "p95": 36.8, + "p99": 42.9, + "n": 66 + }, + "back": { + "p50": 14.5, + "p95": 20.7, + "p99": 25.4, + "n": 66 + } + }, + "inputs": { + "asked": 65, + "sent": 66, + "seen": 66 + }, + "timeline": [ + { + "t": 0, + "fps": 37, + "mbps": 0.26, + "content_p50": 14.9, + "content_p95": 36.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 42, + "mbps": 0.15, + "content_p50": 12.7, + "content_p95": 15.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 6, + "mbps": 0.02, + "content_p50": 15.1, + "content_p95": 20.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 5, + "mbps": 0.02, + "content_p50": 13.6, + "content_p95": 14.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 5, + "mbps": 0.02, + "content_p50": 16.5, + "content_p95": 18.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 5, + "mbps": 0.02, + "content_p50": 14.0, + "content_p95": 20.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 5, + "mbps": 0.02, + "content_p50": 14.5, + "content_p95": 15.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 5, + "mbps": 0.02, + "content_p50": 15.5, + "content_p95": 18.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 5, + "mbps": 0.02, + "content_p50": 13.4, + "content_p95": 16.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 5, + "mbps": 0.02, + "content_p50": 14.3, + "content_p95": 16.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 5, + "mbps": 0.02, + "content_p50": 14.9, + "content_p95": 16.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 5, + "mbps": 0.02, + "content_p50": 14.0, + "content_p95": 18.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 2, + "mbps": 0.01, + "content_p50": 13.2, + "content_p95": 17.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 3, + "mbps": 0.01, + "content_p50": 12.9, + "content_p95": 13.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 3, + "mbps": 0.01, + "content_p50": 14.2, + "content_p95": 25.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 1, + "mbps": 0.0, + "content_p50": 13.9, + "content_p95": 13.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 16, + "fps": 1, + "mbps": 0.0, + "content_p50": 15.8, + "content_p95": 15.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 17, + "fps": 2, + "mbps": 0.01, + "content_p50": 13.6, + "content_p95": 25.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 18, + "fps": 1, + "mbps": 0.0, + "content_p50": 15.7, + "content_p95": 15.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 19, + "fps": 1, + "mbps": 0.0, + "content_p50": 17.6, + "content_p95": 17.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 20, + "fps": 1, + "mbps": 0.0, + "content_p50": 16.0, + "content_p95": 16.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 13.9, + "content_p95": 25.6, + "input_p50": 44.5, + "input_p95": 68.2, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local" + }, + "controller": { + "adapt": true, + "baseRtt": 5.75, + "ceiling": 14860800, + "fps": 60, + "inFlight": 0, + "scale": 1, + "slack": 41.666, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 6.9, + "cpu": { + "mac_agent_pct": 1.7, + "frame_viewer_pct": 10.2, + "frame_tailscaled_pct": 1.6, + "frame_sshd_pct": 0.2, + "frame_gamescope_pct": 5.4, + "frame_total_pct": 32.0 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 71 Hz", + "show_s": 1.57, + "panel": "valve.steam.desktopgame.2001615880", + "src": "separate:6692" + } + ], + [ + { + "scenario": "type", + "frames_sent": 190, + "frames_drawn": 190, + "frames_shown": 190, + "duration_s": 21.1, + "stages_ms": { + "capture": { + "p50": -4.7, + "p95": 4.9, + "p99": 7.4, + "n": 190 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 190 + }, + "encode": { + "p50": 6.4, + "p95": 9.0, + "p99": 12.6, + "n": 190 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.3, + "n": 190 + }, + "network": { + "p50": 2.8, + "p95": 5.1, + "p99": 7.8, + "n": 190 + }, + "decode": { + "p50": 2.0, + "p95": 4.0, + "p99": 5.9, + "n": 190 + }, + "draw": { + "p50": 0.6, + "p95": 1.5, + "p99": 2.0, + "n": 190 + }, + "present": { + "p50": 0.6, + "p95": 8.5, + "p99": 13.4, + "n": 190 + }, + "content": { + "p50": 12.8, + "p95": 19.4, + "p99": 21.3, + "n": 190 + }, + "content_shown": { + "p50": 13.8, + "p95": 22.9, + "p99": 25.0, + "n": 190 + } + }, + "fps": 9.0, + "fps_shown": 9.0, + "late_pct": 41.88, + "stall_max": 2314.7, + "stalls_over_100ms": 67, + "mbps": 0.03, + "keyframes": 0, + "size": "1920x1290", + "captured": 190, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 35.9, + "p95": 91.9, + "p99": 180.6, + "n": 65 + }, + "input_shown_ms": { + "p50": 36.3, + "p95": 92.4, + "p99": 181.0, + "n": 65 + }, + "input_parts_ms": { + "uplink": { + "p50": 5.3, + "p95": 19.9, + "p99": 57.7, + "n": 65 + }, + "mac": { + "p50": 17.6, + "p95": 38.0, + "p99": 89.2, + "n": 65 + }, + "back": { + "p50": 13.2, + "p95": 20.9, + "p99": 21.3, + "n": 65 + } + }, + "inputs": { + "asked": 65, + "sent": 66, + "seen": 65 + }, + "timeline": [ + { + "t": 0, + "fps": 14, + "mbps": 0.03, + "content_p50": 12.3, + "content_p95": 17.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 6, + "mbps": 0.02, + "content_p50": 12.0, + "content_p95": 20.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 5, + "mbps": 0.02, + "content_p50": 15.5, + "content_p95": 19.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 5, + "mbps": 0.02, + "content_p50": 12.5, + "content_p95": 18.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 11, + "mbps": 0.03, + "content_p50": 13.2, + "content_p95": 19.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 12, + "mbps": 0.03, + "content_p50": 14.7, + "content_p95": 18.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 6, + "mbps": 0.02, + "content_p50": 12.6, + "content_p95": 14.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 13, + "mbps": 0.03, + "content_p50": 13.5, + "content_p95": 22.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 19, + "mbps": 0.05, + "content_p50": 12.4, + "content_p95": 16.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 5, + "mbps": 0.02, + "content_p50": 13.0, + "content_p95": 14.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 5, + "mbps": 0.02, + "content_p50": 12.9, + "content_p95": 13.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 5, + "mbps": 0.02, + "content_p50": 10.9, + "content_p95": 21.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 3, + "mbps": 0.01, + "content_p50": 14.2, + "content_p95": 14.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 1, + "mbps": 0.0, + "content_p50": 13.7, + "content_p95": 13.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 2, + "mbps": 0.01, + "content_p50": 12.7, + "content_p95": 15.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 42, + "mbps": 0.14, + "content_p50": 12.4, + "content_p95": 16.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 16, + "fps": 25, + "mbps": 0.1, + "content_p50": 12.2, + "content_p95": 14.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 17, + "fps": 7, + "mbps": 0.11, + "content_p50": 15.2, + "content_p95": 20.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 18, + "fps": 4, + "mbps": 0.01, + "content_p50": 14.2, + "content_p95": 14.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 12.8, + "content_p95": 19.4, + "input_p50": 35.9, + "input_p95": 91.9, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "acceptable" + }, + "controller": { + "adapt": true, + "baseRtt": 5.709, + "ceiling": 14860800, + "fps": 60, + "inFlight": 0, + "scale": 1, + "slack": 41.666, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 9.0, + "cpu": { + "mac_agent_pct": 2.0, + "frame_viewer_pct": 12.8, + "frame_tailscaled_pct": 2.0, + "frame_sshd_pct": 0.2, + "frame_gamescope_pct": 6.0, + "frame_total_pct": 26.5 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 71 Hz", + "show_s": 1.29, + "panel": "valve.steam.desktopgame.2001147903", + "src": "separate:7108" + } + ], + [ + { + "scenario": "type", + "frames_sent": 83, + "frames_drawn": 83, + "frames_shown": 83, + "duration_s": 21.1, + "stages_ms": { + "capture": { + "p50": 0.2, + "p95": 19.9, + "p99": 50.0, + "n": 83 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 21.0, + "n": 83 + }, + "encode": { + "p50": 6.6, + "p95": 8.9, + "p99": 19.9, + "n": 83 + }, + "socket": { + "p50": 0.1, + "p95": 0.3, + "p99": 9.8, + "n": 83 + }, + "network": { + "p50": 3.4, + "p95": 8.8, + "p99": 43.6, + "n": 83 + }, + "decode": { + "p50": 2.6, + "p95": 8.7, + "p99": 11.1, + "n": 83 + }, + "draw": { + "p50": 0.7, + "p95": 1.1, + "p99": 1.7, + "n": 83 + }, + "present": { + "p50": 0.6, + "p95": 5.7, + "p99": 11.5, + "n": 83 + }, + "content": { + "p50": 14.6, + "p95": 60.4, + "p99": 124.2, + "n": 83 + }, + "content_shown": { + "p50": 15.9, + "p95": 61.1, + "p99": 124.8, + "n": 83 + } + }, + "fps": 3.9, + "fps_shown": 3.9, + "late_pct": 61.45, + "stall_max": 9230.5, + "stalls_over_100ms": 39, + "mbps": 0.15, + "keyframes": 0, + "size": "1920x1290", + "captured": 114, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 44.3, + "p95": 236.4, + "p99": 1853.7, + "n": 39 + }, + "input_shown_ms": { + "p50": 45.2, + "p95": 236.9, + "p99": 1854.2, + "n": 39 + }, + "input_parts_ms": { + "uplink": { + "p50": 5.3, + "p95": 18.0, + "p99": 45.1, + "n": 39 + }, + "mac": { + "p50": 21.2, + "p95": 150.9, + "p99": 1809.2, + "n": 39 + }, + "back": { + "p50": 13.9, + "p95": 23.3, + "p99": 179.5, + "n": 39 + } + }, + "inputs": { + "asked": 65, + "sent": 66, + "seen": 39 + }, + "timeline": [ + { + "t": 0, + "fps": 7, + "mbps": 0.04, + "content_p50": 16.2, + "content_p95": 179.5, + "bitrate": 469728, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 10, + "mbps": 0.04, + "content_p50": 16.5, + "content_p95": 30.9, + "bitrate": 469728, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 5, + "mbps": 0.02, + "content_p50": 13.8, + "content_p95": 14.6, + "bitrate": 469728, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 6, + "mbps": 0.02, + "content_p50": 13.5, + "content_p95": 18.5, + "bitrate": 673370, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 7, + "mbps": 0.02, + "content_p50": 12.7, + "content_p95": 13.4, + "bitrate": 1217929, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 15, + "mbps": 0.06, + "content_p50": 13.3, + "content_p95": 19.4, + "bitrate": 1786562, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 6, + "mbps": 0.02, + "content_p50": 13.9, + "content_p95": 20.1, + "bitrate": 2543412, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 5, + "mbps": 0.11, + "content_p50": 14.2, + "content_p95": 23.2, + "bitrate": 3182528, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 3, + "mbps": 0.01, + "content_p50": 13.5, + "content_p95": 13.5, + "bitrate": 4891587, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 17, + "fps": 5, + "mbps": 0.97, + "content_p50": 41.2, + "content_p95": 89.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 18, + "fps": 5, + "mbps": 0.65, + "content_p50": 36.5, + "content_p95": 44.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 20, + "fps": 5, + "mbps": 0.72, + "content_p50": 26.1, + "content_p95": 40.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 21, + "fps": 4, + "mbps": 0.5, + "content_p50": 17.6, + "content_p95": 24.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 14.6, + "content_p95": 60.4, + "input_p50": 44.3, + "input_p95": 236.4, + "grades": { + "input_replies": "bad", + "content_p50": "local", + "content_p95": "acceptable", + "input_p50": "local", + "input_p95": "bad" + }, + "controller": { + "adapt": true, + "baseRtt": 6.375, + "ceiling": 14860800, + "fps": 60, + "inFlight": 0, + "scale": 1, + "slack": 44.666, + "target": 14860800, + "tier": 0 + }, + "events": [ + { + "t": 0.39, + "e": "down to 469 kbit/s: queue 165 ms, held 4, oldest 213 ms, base 5 ms, sent 44 got 28 wants 234" + } + ], + "source_fps": 5.4, + "cpu": { + "mac_agent_pct": 1.5, + "frame_viewer_pct": 8.1, + "frame_tailscaled_pct": 1.8, + "frame_sshd_pct": 0.1, + "frame_gamescope_pct": 5.5, + "frame_total_pct": 25.1 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 72 Hz", + "show_s": 1.27, + "panel": "valve.steam.desktopgame.2001547827", + "src": "separate:7247" + } + ] + ], + "warm": [ + [ + { + "scenario": "type", + "frames_sent": 114, + "frames_drawn": 114, + "frames_shown": 112, + "duration_s": 21.1, + "stages_ms": { + "capture": { + "p50": 0.2, + "p95": 5.8, + "p99": 10.8, + "n": 114 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 114 + }, + "encode": { + "p50": 6.6, + "p95": 12.2, + "p99": 14.3, + "n": 114 + }, + "socket": { + "p50": 0.1, + "p95": 0.4, + "p99": 2.9, + "n": 114 + }, + "network": { + "p50": 3.9, + "p95": 12.5, + "p99": 44.8, + "n": 114 + }, + "decode": { + "p50": 2.2, + "p95": 5.4, + "p99": 9.2, + "n": 114 + }, + "draw": { + "p50": 0.6, + "p95": 1.5, + "p99": 1.9, + "n": 114 + }, + "present": { + "p50": 0.5, + "p95": 0.9, + "p99": 1.2, + "n": 112 + }, + "content": { + "p50": 14.5, + "p95": 29.9, + "p99": 65.7, + "n": 114 + }, + "content_shown": { + "p50": 15.1, + "p95": 28.9, + "p99": 52.5, + "n": 112 + } + }, + "fps": 5.4, + "fps_shown": 5.3, + "late_pct": 72.17, + "stall_max": 1001.0, + "stalls_over_100ms": 67, + "mbps": 0.03, + "keyframes": 0, + "size": "1920x1290", + "captured": 162, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 46.6, + "p95": 71.3, + "p99": 84.0, + "n": 66 + }, + "input_shown_ms": { + "p50": 47.1, + "p95": 71.8, + "p99": 84.7, + "n": 66 + }, + "input_parts_ms": { + "uplink": { + "p50": 4.3, + "p95": 9.7, + "p99": 34.2, + "n": 66 + }, + "mac": { + "p50": 25.8, + "p95": 42.8, + "p99": 44.3, + "n": 66 + }, + "back": { + "p50": 15.2, + "p95": 29.3, + "p99": 29.9, + "n": 66 + } + }, + "inputs": { + "asked": 65, + "sent": 66, + "seen": 66 + }, + "timeline": [ + { + "t": 0, + "fps": 29, + "mbps": 0.26, + "content_p50": 14.3, + "content_p95": 21.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 7, + "mbps": 0.03, + "content_p50": 13.4, + "content_p95": 15.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 5, + "mbps": 0.02, + "content_p50": 16.9, + "content_p95": 18.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 9, + "mbps": 0.03, + "content_p50": 16.1, + "content_p95": 20.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 5, + "mbps": 0.02, + "content_p50": 14.4, + "content_p95": 19.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 10, + "mbps": 0.04, + "content_p50": 13.8, + "content_p95": 29.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 6, + "mbps": 0.02, + "content_p50": 14.2, + "content_p95": 28.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 4, + "mbps": 0.02, + "content_p50": 16.8, + "content_p95": 28.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 6, + "mbps": 0.03, + "content_p50": 13.8, + "content_p95": 16.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 5, + "mbps": 0.02, + "content_p50": 13.5, + "content_p95": 20.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 9, + "mbps": 0.05, + "content_p50": 29.9, + "content_p95": 95.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 7, + "mbps": 0.02, + "content_p50": 13.8, + "content_p95": 30.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 2, + "mbps": 0.01, + "content_p50": 13.3, + "content_p95": 17.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 1, + "mbps": 0.0, + "content_p50": 16.6, + "content_p95": 16.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 2, + "mbps": 0.01, + "content_p50": 15.2, + "content_p95": 17.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 1, + "mbps": 0.0, + "content_p50": 14.2, + "content_p95": 14.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 16, + "fps": 1, + "mbps": 0.0, + "content_p50": 19.1, + "content_p95": 19.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 17, + "fps": 2, + "mbps": 0.01, + "content_p50": 14.2, + "content_p95": 14.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 18, + "fps": 1, + "mbps": 0.0, + "content_p50": 13.9, + "content_p95": 13.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 19, + "fps": 1, + "mbps": 0.0, + "content_p50": 14.4, + "content_p95": 14.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 20, + "fps": 1, + "mbps": 0.0, + "content_p50": 15.1, + "content_p95": 15.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 14.5, + "content_p95": 29.9, + "input_p50": 46.6, + "input_p95": 71.3, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "acceptable" + }, + "controller": { + "adapt": true, + "baseRtt": 5.541, + "ceiling": 14860800, + "fps": 60, + "inFlight": 1, + "scale": 1, + "slack": 48.292, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 7.7, + "cpu": { + "mac_agent_pct": 2.0, + "frame_viewer_pct": 10.9, + "frame_tailscaled_pct": 2.8, + "frame_sshd_pct": 0.4, + "frame_gamescope_pct": 5.4, + "frame_total_pct": 36.0 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 70 Hz", + "show_s": 1.28, + "panel": "valve.steam.desktopgame.2001621121", + "src": "separate:6829" + } + ], + [ + { + "scenario": "type", + "frames_sent": 220, + "frames_drawn": 220, + "frames_shown": 217, + "duration_s": 21.1, + "stages_ms": { + "capture": { + "p50": -5.5, + "p95": 3.8, + "p99": 14.3, + "n": 220 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.3, + "n": 220 + }, + "encode": { + "p50": 6.5, + "p95": 9.0, + "p99": 18.8, + "n": 220 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 1.7, + "n": 220 + }, + "network": { + "p50": 3.1, + "p95": 8.9, + "p99": 46.7, + "n": 220 + }, + "decode": { + "p50": 2.0, + "p95": 4.1, + "p99": 7.6, + "n": 220 + }, + "draw": { + "p50": 0.7, + "p95": 1.4, + "p99": 1.8, + "n": 220 + }, + "present": { + "p50": 0.6, + "p95": 10.4, + "p99": 13.9, + "n": 217 + }, + "content": { + "p50": 13.0, + "p95": 25.6, + "p99": 122.5, + "n": 220 + }, + "content_shown": { + "p50": 14.1, + "p95": 27.2, + "p99": 122.9, + "n": 217 + } + }, + "fps": 10.4, + "fps_shown": 10.3, + "late_pct": 38.46, + "stall_max": 4500.7, + "stalls_over_100ms": 64, + "mbps": 0.05, + "keyframes": 0, + "size": "1920x1290", + "captured": 220, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 51.4, + "p95": 63.8, + "p99": 80.7, + "n": 64 + }, + "input_shown_ms": { + "p50": 51.8, + "p95": 64.3, + "p99": 81.3, + "n": 64 + }, + "input_parts_ms": { + "uplink": { + "p50": 4.1, + "p95": 17.8, + "p99": 29.2, + "n": 64 + }, + "mac": { + "p50": 31.3, + "p95": 35.3, + "p99": 39.4, + "n": 64 + }, + "back": { + "p50": 13.6, + "p95": 19.9, + "p99": 26.1, + "n": 64 + } + }, + "inputs": { + "asked": 65, + "sent": 66, + "seen": 64 + }, + "timeline": [ + { + "t": 0, + "fps": 28, + "mbps": 0.27, + "content_p50": 16.7, + "content_p95": 61.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 8, + "mbps": 0.03, + "content_p50": 13.7, + "content_p95": 19.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 5, + "mbps": 0.02, + "content_p50": 13.3, + "content_p95": 14.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 5, + "mbps": 0.02, + "content_p50": 13.2, + "content_p95": 13.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 5, + "mbps": 0.02, + "content_p50": 12.9, + "content_p95": 14.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 5, + "mbps": 0.02, + "content_p50": 16.9, + "content_p95": 24.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 5, + "mbps": 0.02, + "content_p50": 13.6, + "content_p95": 17.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 5, + "mbps": 0.02, + "content_p50": 13.1, + "content_p95": 15.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 6, + "mbps": 0.02, + "content_p50": 13.8, + "content_p95": 15.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 6, + "mbps": 0.02, + "content_p50": 14.5, + "content_p95": 16.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 6, + "mbps": 0.02, + "content_p50": 12.6, + "content_p95": 19.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 48, + "mbps": 0.15, + "content_p50": 12.8, + "content_p95": 16.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 47, + "mbps": 0.15, + "content_p50": 12.6, + "content_p95": 16.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 30, + "mbps": 0.09, + "content_p50": 12.8, + "content_p95": 15.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 2, + "mbps": 0.01, + "content_p50": 11.6, + "content_p95": 13.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 6, + "mbps": 0.11, + "content_p50": 42.9, + "content_p95": 143.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 16, + "fps": 3, + "mbps": 0.01, + "content_p50": 12.8, + "content_p95": 13.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 13.0, + "content_p95": 25.6, + "input_p50": 51.4, + "input_p95": 63.8, + "grades": { + "input_replies": "local", + "content_p50": "local", + "content_p95": "local", + "input_p50": "acceptable", + "input_p95": "local" + }, + "controller": { + "adapt": true, + "baseRtt": 5.792, + "ceiling": 14860800, + "fps": 60, + "inFlight": 0, + "scale": 1, + "slack": 41.666, + "target": 14860800, + "tier": 0 + }, + "events": [], + "source_fps": 10.4, + "cpu": { + "mac_agent_pct": 2.5, + "frame_viewer_pct": 14.7, + "frame_tailscaled_pct": 2.9, + "frame_sshd_pct": 0.4, + "frame_gamescope_pct": 6.1, + "frame_total_pct": 33.5 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 71 Hz", + "show_s": 1.27, + "panel": "valve.steam.desktopgame.2001606834", + "src": "separate:6969" + } + ], + [ + { + "scenario": "type", + "frames_sent": 128, + "frames_drawn": 128, + "frames_shown": 126, + "duration_s": 21.1, + "stages_ms": { + "capture": { + "p50": -3.5, + "p95": 1.4, + "p99": 26.9, + "n": 128 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.7, + "n": 128 + }, + "encode": { + "p50": 6.7, + "p95": 9.9, + "p99": 23.0, + "n": 128 + }, + "socket": { + "p50": 0.1, + "p95": 0.3, + "p99": 2.8, + "n": 128 + }, + "network": { + "p50": 3.3, + "p95": 5.1, + "p99": 205.9, + "n": 128 + }, + "decode": { + "p50": 2.1, + "p95": 4.2, + "p99": 6.4, + "n": 128 + }, + "draw": { + "p50": 0.7, + "p95": 1.7, + "p99": 1.9, + "n": 128 + }, + "present": { + "p50": 0.6, + "p95": 9.2, + "p99": 13.7, + "n": 126 + }, + "content": { + "p50": 13.4, + "p95": 21.6, + "p99": 264.8, + "n": 128 + }, + "content_shown": { + "p50": 14.5, + "p95": 26.1, + "p99": 265.4, + "n": 126 + } + }, + "fps": 6.1, + "fps_shown": 6.0, + "late_pct": 17.83, + "stall_max": 9246.3, + "stalls_over_100ms": 17, + "mbps": 0.02, + "keyframes": 0, + "size": "1920x1290", + "captured": 129, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 42.5, + "p95": 194.0, + "p99": 404.5, + "n": 25 + }, + "input_shown_ms": { + "p50": 43.2, + "p95": 194.6, + "p99": 405.1, + "n": 25 + }, + "input_parts_ms": { + "uplink": { + "p50": 9.4, + "p95": 26.2, + "p99": 35.0, + "n": 25 + }, + "mac": { + "p50": 23.3, + "p95": 135.9, + "p99": 176.2, + "n": 25 + }, + "back": { + "p50": 13.8, + "p95": 21.6, + "p99": 264.8, + "n": 25 + } + }, + "inputs": { + "asked": 65, + "sent": 66, + "seen": 25 + }, + "timeline": [ + { + "t": 0, + "fps": 24, + "mbps": 0.06, + "content_p50": 13.0, + "content_p95": 14.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 58, + "mbps": 0.19, + "content_p50": 13.6, + "content_p95": 18.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 26, + "mbps": 0.08, + "content_p50": 12.9, + "content_p95": 15.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 6, + "mbps": 0.08, + "content_p50": 13.5, + "content_p95": 17.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 4, + "mbps": 0.05, + "content_p50": 18.8, + "content_p95": 25.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 3, + "mbps": 0.01, + "content_p50": 13.3, + "content_p95": 13.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 4, + "mbps": 0.03, + "content_p50": 264.8, + "content_p95": 354.5, + "bitrate": 300000, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 3, + "mbps": 0.01, + "content_p50": 13.8, + "content_p95": 15.1, + "bitrate": 300000, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 13.4, + "content_p95": 21.6, + "input_p50": 42.5, + "input_p95": 194.0, + "grades": { + "input_replies": "bad", + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "bad" + }, + "controller": { + "adapt": true, + "baseRtt": 0, + "ceiling": 14860800, + "fps": 60, + "inFlight": 0, + "scale": 1, + "slack": 41.666, + "target": 14855423, + "tier": 0 + }, + "events": [ + { + "t": 10.93, + "e": "down to 300 kbit/s: queue 0 ms, held 0, oldest 201 ms, base 5 ms, sent 61 got 0 wants 51" + } + ], + "source_fps": 6.1, + "cpu": { + "mac_agent_pct": 2.1, + "frame_viewer_pct": 11.1, + "frame_tailscaled_pct": 2.6, + "frame_sshd_pct": 0.4, + "frame_gamescope_pct": 6.0, + "frame_total_pct": 25.6 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 72 Hz", + "show_s": 1.32, + "panel": "valve.steam.desktopgame.2001763888", + "src": "separate:7401" + } + ] + ] + }, + "table": { + "type": { + "content_p50": { + "cold": [ + 13.9, + 12.8, + 14.6 + ], + "warm": [ + 14.5, + 13.0, + 13.4 + ] + }, + "content_p95": { + "cold": [ + 25.6, + 19.4, + 60.4 + ], + "warm": [ + 29.9, + 25.6, + 21.6 + ] + }, + "input_p50": { + "cold": [ + 44.5, + 35.9, + 44.3 + ], + "warm": [ + 46.6, + 51.4, + 42.5 + ] + }, + "input_p95": { + "cold": [ + 68.2, + 91.9, + 236.4 + ], + "warm": [ + 71.3, + 63.8, + 194.0 + ] + }, + "fps": { + "cold": [ + 6.9, + 9.0, + 3.9 + ], + "warm": [ + 5.4, + 10.4, + 6.1 + ] + }, + "late_pct": { + "cold": [ + 51.37, + 41.88, + 61.45 + ], + "warm": [ + 72.17, + 38.46, + 17.83 + ] + }, + "stall_max": { + "cold": [ + 1007.5, + 2314.7, + 9230.5 + ], + "warm": [ + 1001.0, + 4500.7, + 9246.3 + ] + }, + "mbps": { + "cold": [ + 0.03, + 0.03, + 0.15 + ], + "warm": [ + 0.03, + 0.05, + 0.02 + ] + } + } + } +} \ No newline at end of file diff --git a/bench/results/2026-09-28-a3c6e5c-dirty-adapt-clean.json b/bench/results/2026-09-28-a3c6e5c-dirty-adapt-clean.json new file mode 100644 index 0000000..7f449e1 --- /dev/null +++ b/bench/results/2026-09-28-a3c6e5c-dirty-adapt-clean.json @@ -0,0 +1,984 @@ +{ + "label": "adapt-clean", + "date": "2026-09-28T17:13:59", + "commit": "a3c6e5c+dirty", + "config": { + "quality": "balanced", + "mode": "separate", + "net": "none", + "delay_ms": 0, + "buffer_ms": 250, + "host": "frame", + "ssh_opts": [], + "encoder": "", + "duration_s": 20.0, + "browser_flags": [] + }, + "frame_build": "20260925.6191901", + "mac": "26.5.2", + "headset": ":39.639829 [Info] - 0 - entering standby", + "scenarios": [ + { + "scenario": "test", + "frames_sent": 653, + "frames_drawn": 653, + "frames_shown": 649, + "duration_s": 20.7, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 653 + }, + "queue": { + "p50": 11.2, + "p95": 16.1, + "p99": 17.1, + "n": 653 + }, + "encode": { + "p50": 3.5, + "p95": 4.2, + "p99": 4.4, + "n": 653 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 653 + }, + "network": { + "p50": 2.9, + "p95": 7.0, + "p99": 12.0, + "n": 653 + }, + "decode": { + "p50": 1.0, + "p95": 2.4, + "p99": 2.8, + "n": 653 + }, + "draw": { + "p50": 0.3, + "p95": 0.6, + "p99": 0.9, + "n": 653 + }, + "present": { + "p50": 0.5, + "p95": 0.9, + "p99": 5.9, + "n": 649 + }, + "content": { + "p50": 19.8, + "p95": 26.2, + "p99": 30.1, + "n": 653 + }, + "content_shown": { + "p50": 20.3, + "p95": 26.9, + "p99": 31.1, + "n": 649 + } + }, + "fps": 31.5, + "fps_shown": 31.3, + "late_pct": 98.01, + "stall_max": 144.0, + "stalls_over_100ms": 2, + "mbps": 0.13, + "keyframes": 2, + "size": "960x540", + "captured": 1336, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 43.5, + "p95": 61.8, + "p99": 63.4, + "n": 40 + }, + "input_shown_ms": { + "p50": 44.3, + "p95": 62.3, + "p99": 63.9, + "n": 40 + }, + "input_parts_ms": { + "uplink": { + "p50": 11.1, + "p95": 29.2, + "p99": 36.1, + "n": 40 + }, + "mac": { + "p50": 10.4, + "p95": 28.2, + "p99": 32.4, + "n": 40 + }, + "back": { + "p50": 17.5, + "p95": 27.9, + "p99": 28.8, + "n": 40 + } + }, + "inputs": { + "sent": 40, + "seen": 40 + }, + "timeline": [ + { + "t": 0, + "fps": 31, + "mbps": 0.13, + "content_p50": 21.5, + "content_p95": 27.1, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 1, + "fps": 31, + "mbps": 0.12, + "content_p50": 22.4, + "content_p95": 26.3, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 2, + "fps": 32, + "mbps": 0.13, + "content_p50": 18.9, + "content_p95": 24.8, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 3, + "fps": 31, + "mbps": 0.12, + "content_p50": 18.2, + "content_p95": 24.6, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 4, + "fps": 32, + "mbps": 0.13, + "content_p50": 21.0, + "content_p95": 25.6, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 5, + "fps": 32, + "mbps": 0.12, + "content_p50": 19.5, + "content_p95": 24.3, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 6, + "fps": 31, + "mbps": 0.12, + "content_p50": 22.2, + "content_p95": 25.6, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 7, + "fps": 32, + "mbps": 0.13, + "content_p50": 19.9, + "content_p95": 24.0, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 8, + "fps": 31, + "mbps": 0.21, + "content_p50": 22.7, + "content_p95": 26.9, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 9, + "fps": 31, + "mbps": 0.12, + "content_p50": 20.2, + "content_p95": 26.1, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 10, + "fps": 32, + "mbps": 0.12, + "content_p50": 18.2, + "content_p95": 23.2, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 11, + "fps": 31, + "mbps": 0.12, + "content_p50": 19.8, + "content_p95": 24.2, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 12, + "fps": 32, + "mbps": 0.13, + "content_p50": 19.2, + "content_p95": 24.8, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 13, + "fps": 32, + "mbps": 0.12, + "content_p50": 21.0, + "content_p95": 24.3, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 14, + "fps": 30, + "mbps": 0.13, + "content_p50": 20.4, + "content_p95": 96.7, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 15, + "fps": 32, + "mbps": 0.13, + "content_p50": 18.2, + "content_p95": 24.4, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 16, + "fps": 31, + "mbps": 0.12, + "content_p50": 18.4, + "content_p95": 25.1, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 17, + "fps": 32, + "mbps": 0.13, + "content_p50": 17.2, + "content_p95": 23.5, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 18, + "fps": 32, + "mbps": 0.19, + "content_p50": 19.4, + "content_p95": 25.2, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 19, + "fps": 31, + "mbps": 0.14, + "content_p50": 21.1, + "content_p95": 84.3, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 20, + "fps": 24, + "mbps": 0.08, + "content_p50": 20.8, + "content_p95": 25.1, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + } + ], + "adapt": [], + "content_p50": 19.8, + "content_p95": 26.2, + "input_p50": 43.5, + "input_p95": 61.8, + "grades": { + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "bad", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "source_fps": 64.5, + "cpu": { + "mac_agent_pct": 11.3, + "frame_viewer_pct": 35.9, + "frame_tailscaled_pct": 3.5, + "frame_sshd_pct": 0.6, + "frame_gamescope_pct": 7.5, + "frame_total_pct": 16.1 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 141 Hz", + "show_s": 4.68, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 937, + "frames_drawn": 937, + "frames_shown": 720, + "duration_s": 20.7, + "stages_ms": { + "capture": { + "p50": -3.9, + "p95": 0.9, + "p99": 8.2, + "n": 937 + }, + "queue": { + "p50": 0.0, + "p95": 16.3, + "p99": 20.1, + "n": 937 + }, + "encode": { + "p50": 6.4, + "p95": 7.2, + "p99": 9.3, + "n": 937 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 937 + }, + "network": { + "p50": 7.3, + "p95": 13.0, + "p99": 22.8, + "n": 937 + }, + "decode": { + "p50": 6.2, + "p95": 12.7, + "p99": 18.6, + "n": 937 + }, + "draw": { + "p50": 0.6, + "p95": 1.1, + "p99": 1.9, + "n": 937 + }, + "present": { + "p50": 2.9, + "p95": 18.5, + "p99": 24.5, + "n": 720 + }, + "content": { + "p50": 19.9, + "p95": 37.2, + "p99": 50.2, + "n": 937 + }, + "content_shown": { + "p50": 27.0, + "p95": 45.7, + "p99": 55.0, + "n": 720 + } + }, + "fps": 45.3, + "fps_shown": 34.8, + "late_pct": 28.63, + "stall_max": 99.3, + "stalls_over_100ms": 0, + "mbps": 7.59, + "keyframes": 2, + "size": "1920x1290", + "captured": 1215, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 30, + "mbps": 4.01, + "content_p50": 31.9, + "content_p95": 52.3, + "bitrate": 4867140, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 30, + "mbps": 4.12, + "content_p50": 29.9, + "content_p95": 45.3, + "bitrate": 3807054, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 30, + "mbps": 3.26, + "content_p50": 31.8, + "content_p95": 52.5, + "bitrate": 3235995, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 30, + "mbps": 3.72, + "content_p50": 24.5, + "content_p95": 37.7, + "bitrate": 4238740, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 30, + "mbps": 5.09, + "content_p50": 25.2, + "content_p95": 39.6, + "bitrate": 5992063, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 31, + "mbps": 7.29, + "content_p50": 29.1, + "content_p95": 53.5, + "bitrate": 5526067, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 35, + "mbps": 4.58, + "content_p50": 25.3, + "content_p95": 32.4, + "bitrate": 6018152, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 46, + "mbps": 7.12, + "content_p50": 25.1, + "content_p95": 34.3, + "bitrate": 8412933, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 46, + "mbps": 8.25, + "content_p50": 27.1, + "content_p95": 40.3, + "bitrate": 10760191, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 47, + "mbps": 9.3, + "content_p50": 27.8, + "content_p95": 41.2, + "bitrate": 13717060, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 55, + "mbps": 8.98, + "content_p50": 16.5, + "content_p95": 21.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 51, + "mbps": 9.32, + "content_p50": 18.6, + "content_p95": 35.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 56, + "mbps": 9.53, + "content_p50": 15.4, + "content_p95": 26.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 55, + "mbps": 8.46, + "content_p50": 18.6, + "content_p95": 23.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 54, + "mbps": 9.82, + "content_p50": 15.9, + "content_p95": 28.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 54, + "mbps": 11.11, + "content_p50": 15.5, + "content_p95": 26.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 16, + "fps": 54, + "mbps": 9.79, + "content_p50": 17.4, + "content_p95": 25.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 17, + "fps": 57, + "mbps": 9.3, + "content_p50": 16.3, + "content_p95": 23.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 18, + "fps": 55, + "mbps": 8.3, + "content_p50": 19.5, + "content_p95": 31.0, + "bitrate": 7984791, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 19, + "fps": 51, + "mbps": 8.56, + "content_p50": 16.4, + "content_p95": 29.1, + "bitrate": 10220855, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 20, + "fps": 40, + "mbps": 7.13, + "content_p50": 18.3, + "content_p95": 23.9, + "bitrate": 12025604, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 19.9, + "content_p95": 37.2, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "local", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "local" + }, + "source_fps": 58.7, + "cpu": { + "mac_agent_pct": 6.6, + "frame_viewer_pct": 72.4, + "frame_tailscaled_pct": 8.1, + "frame_sshd_pct": 0.9, + "frame_gamescope_pct": 10.0, + "frame_total_pct": 24.2 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 141 Hz", + "show_s": 6.43, + "panel": "valve.steam.desktopgame.2001414593", + "src": "separate:9384" + }, + { + "scenario": "type", + "frames_sent": 25, + "frames_drawn": 25, + "frames_shown": 25, + "duration_s": 14.7, + "stages_ms": { + "capture": { + "p50": -0.3, + "p95": 0.9, + "p99": 1.1, + "n": 25 + }, + "queue": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 25 + }, + "encode": { + "p50": 5.0, + "p95": 33.4, + "p99": 33.6, + "n": 25 + }, + "socket": { + "p50": 0.1, + "p95": 0.1, + "p99": 0.2, + "n": 25 + }, + "network": { + "p50": 6.1, + "p95": 14.9, + "p99": 24.4, + "n": 25 + }, + "decode": { + "p50": 3.0, + "p95": 10.5, + "p99": 11.4, + "n": 25 + }, + "draw": { + "p50": 0.6, + "p95": 1.0, + "p99": 1.1, + "n": 25 + }, + "present": { + "p50": 0.8, + "p95": 1.4, + "p99": 1.7, + "n": 25 + }, + "content": { + "p50": 12.6, + "p95": 41.0, + "p99": 46.3, + "n": 25 + }, + "content_shown": { + "p50": 13.9, + "p95": 41.8, + "p99": 46.6, + "n": 25 + } + }, + "fps": 1.7, + "fps_shown": 1.7, + "late_pct": 100.0, + "stall_max": 3099.2, + "stalls_over_100ms": 21, + "mbps": 0.04, + "keyframes": 4, + "size": "960x646", + "captured": 79, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 41.3, + "p95": 60.3, + "p99": 68.6, + "n": 18 + }, + "input_shown_ms": { + "p50": 43.0, + "p95": 61.1, + "p99": 69.6, + "n": 18 + }, + "input_parts_ms": { + "uplink": { + "p50": 5.3, + "p95": 11.6, + "p99": 14.2, + "n": 18 + }, + "mac": { + "p50": 23.1, + "p95": 35.0, + "p99": 42.6, + "n": 18 + }, + "back": { + "p50": 11.8, + "p95": 19.2, + "p99": 22.7, + "n": 18 + } + }, + "inputs": { + "sent": 66, + "seen": 18 + }, + "timeline": [ + { + "t": 3, + "fps": 1, + "mbps": 0.01, + "content_p50": 11.8, + "content_p95": 11.8, + "bitrate": 1224075, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 4, + "fps": 1, + "mbps": 0.02, + "content_p50": 14.7, + "content_p95": 14.7, + "bitrate": 2091896, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 5, + "fps": 4, + "mbps": 0.1, + "content_p50": 18.6, + "content_p95": 37.4, + "bitrate": 3071304, + "tier": 3, + "w": 1440, + "net": null + }, + { + "t": 6, + "fps": 5, + "mbps": 0.13, + "content_p50": 4.2, + "content_p95": 19.2, + "bitrate": 4031277, + "tier": 3, + "w": 1440, + "net": null + }, + { + "t": 7, + "fps": 6, + "mbps": 0.04, + "content_p50": 12.0, + "content_p95": 46.3, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 10, + "fps": 1, + "mbps": 0.01, + "content_p50": 12.2, + "content_p95": 12.2, + "bitrate": 300000, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 12, + "fps": 1, + "mbps": 0.02, + "content_p50": -1.2, + "content_p95": -1.2, + "bitrate": 842857, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 14, + "fps": 1, + "mbps": 0.02, + "content_p50": 14.4, + "content_p95": 14.4, + "bitrate": 2091896, + "tier": 4, + "w": 960, + "net": null + } + ], + "adapt": [], + "content_p50": 12.6, + "content_p95": 41.0, + "input_p50": 41.3, + "input_p95": 60.3, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "input_p50": "local", + "input_p95": "local" + }, + "source_fps": 5.4, + "cpu": { + "mac_agent_pct": 1.2, + "frame_viewer_pct": 5.0, + "frame_tailscaled_pct": 1.5, + "frame_sshd_pct": 0.2, + "frame_gamescope_pct": 4.7, + "frame_total_pct": 8.5 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 140 Hz", + "show_s": 5.12, + "panel": "valve.steam.desktopgame.2001910179", + "src": "separate:9510" + } + ] +} \ No newline at end of file diff --git a/bench/results/2026-09-28-a3c6e5c-dirty-adapt-clean2.json b/bench/results/2026-09-28-a3c6e5c-dirty-adapt-clean2.json new file mode 100644 index 0000000..79fd46c --- /dev/null +++ b/bench/results/2026-09-28-a3c6e5c-dirty-adapt-clean2.json @@ -0,0 +1,1116 @@ +{ + "label": "adapt-clean2", + "date": "2026-09-28T17:16:27", + "commit": "a3c6e5c+dirty", + "config": { + "quality": "balanced", + "mode": "separate", + "net": "none", + "delay_ms": 0, + "buffer_ms": 250, + "host": "frame", + "ssh_opts": [], + "encoder": "", + "duration_s": 20.0, + "browser_flags": [] + }, + "frame_build": "20260925.6191901", + "mac": "26.5.2", + "headset": ":39.639829 [Info] - 0 - entering standby", + "scenarios": [ + { + "scenario": "test", + "frames_sent": 1229, + "frames_drawn": 1229, + "frames_shown": 814, + "duration_s": 20.7, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 1229 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 1229 + }, + "encode": { + "p50": 4.2, + "p95": 4.8, + "p99": 5.0, + "n": 1229 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 1229 + }, + "network": { + "p50": 6.1, + "p95": 10.9, + "p99": 28.8, + "n": 1229 + }, + "decode": { + "p50": 1.2, + "p95": 3.4, + "p99": 4.4, + "n": 1229 + }, + "draw": { + "p50": 0.3, + "p95": 0.7, + "p99": 1.1, + "n": 1229 + }, + "present": { + "p50": 5.5, + "p95": 16.5, + "p99": 19.2, + "n": 814 + }, + "content": { + "p50": 12.2, + "p95": 17.6, + "p99": 34.3, + "n": 1229 + }, + "content_shown": { + "p50": 18.0, + "p95": 29.6, + "p99": 38.7, + "n": 814 + } + }, + "fps": 59.3, + "fps_shown": 39.3, + "late_pct": 1.55, + "stall_max": 168.0, + "stalls_over_100ms": 2, + "mbps": 0.47, + "keyframes": 2, + "size": "1280x720", + "captured": 1337, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 27.8, + "p95": 41.1, + "p99": 46.2, + "n": 40 + }, + "input_shown_ms": { + "p50": 32.3, + "p95": 47.0, + "p99": 53.4, + "n": 25 + }, + "input_parts_ms": { + "uplink": { + "p50": 8.0, + "p95": 19.4, + "p99": 31.7, + "n": 40 + }, + "mac": { + "p50": 5.3, + "p95": 14.2, + "p99": 18.3, + "n": 40 + }, + "back": { + "p50": 12.7, + "p95": 16.6, + "p99": 17.7, + "n": 40 + } + }, + "inputs": { + "sent": 40, + "seen": 40 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.2, + "content_p95": 13.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.0, + "content_p95": 13.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.6, + "content_p95": 17.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.9, + "content_p95": 15.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 60, + "mbps": 0.49, + "content_p50": 11.0, + "content_p95": 17.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.49, + "content_p50": 10.6, + "content_p95": 13.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 60, + "mbps": 0.59, + "content_p50": 10.4, + "content_p95": 15.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 60, + "mbps": 0.49, + "content_p50": 10.3, + "content_p95": 15.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.48, + "content_p50": 11.8, + "content_p95": 15.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 60, + "mbps": 0.47, + "content_p50": 11.9, + "content_p95": 14.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.45, + "content_p50": 12.3, + "content_p95": 23.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 58, + "mbps": 0.44, + "content_p50": 12.5, + "content_p95": 17.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 60, + "mbps": 0.46, + "content_p50": 14.2, + "content_p95": 17.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 58, + "mbps": 0.44, + "content_p50": 14.2, + "content_p95": 27.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 60, + "mbps": 0.47, + "content_p50": 14.3, + "content_p95": 17.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 60, + "mbps": 0.46, + "content_p50": 13.3, + "content_p95": 17.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 16, + "fps": 60, + "mbps": 0.57, + "content_p50": 11.6, + "content_p95": 14.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 17, + "fps": 60, + "mbps": 0.46, + "content_p50": 12.3, + "content_p95": 18.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 18, + "fps": 60, + "mbps": 0.44, + "content_p50": 12.4, + "content_p95": 14.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 19, + "fps": 60, + "mbps": 0.46, + "content_p50": 13.7, + "content_p95": 16.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 20, + "fps": 33, + "mbps": 0.24, + "content_p50": 13.3, + "content_p95": 135.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 12.2, + "content_p95": 17.6, + "input_p50": 27.8, + "input_p95": 41.1, + "grades": { + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "acceptable", + "stall_max": "acceptable" + }, + "source_fps": 64.6, + "cpu": { + "mac_agent_pct": 15.4, + "frame_viewer_pct": 57.6, + "frame_tailscaled_pct": 5.8, + "frame_sshd_pct": 0.8, + "frame_gamescope_pct": 11.1, + "frame_total_pct": 22.2 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz", + "show_s": 5.83, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 734, + "frames_drawn": 734, + "frames_shown": 707, + "duration_s": 20.6, + "stages_ms": { + "capture": { + "p50": -4.8, + "p95": 6.5, + "p99": 13.1, + "n": 734 + }, + "queue": { + "p50": 8.4, + "p95": 19.8, + "p99": 28.9, + "n": 734 + }, + "encode": { + "p50": 6.3, + "p95": 7.1, + "p99": 10.1, + "n": 734 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 734 + }, + "network": { + "p50": 7.4, + "p95": 20.9, + "p99": 48.9, + "n": 734 + }, + "decode": { + "p50": 5.4, + "p95": 13.5, + "p99": 17.2, + "n": 734 + }, + "draw": { + "p50": 0.6, + "p95": 1.2, + "p99": 1.8, + "n": 734 + }, + "present": { + "p50": 0.5, + "p95": 13.6, + "p99": 21.3, + "n": 707 + }, + "content": { + "p50": 24.8, + "p95": 48.1, + "p99": 69.4, + "n": 734 + }, + "content_shown": { + "p50": 26.9, + "p95": 50.5, + "p99": 68.5, + "n": 707 + } + }, + "fps": 35.6, + "fps_shown": 34.3, + "late_pct": 58.12, + "stall_max": 168.7, + "stalls_over_100ms": 2, + "mbps": 4.09, + "keyframes": 3, + "size": "960x646", + "captured": 1122, + "viewer_never_drawn": 0, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 45, + "mbps": 7.92, + "content_p50": 23.2, + "content_p95": 38.8, + "bitrate": 8869129, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 46, + "mbps": 8.52, + "content_p50": 31.8, + "content_p95": 45.2, + "bitrate": 11334867, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 45, + "mbps": 8.17, + "content_p50": 29.2, + "content_p95": 65.7, + "bitrate": 6587726, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 45, + "mbps": 6.56, + "content_p50": 26.5, + "content_p95": 43.5, + "bitrate": 7164744, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 46, + "mbps": 6.64, + "content_p50": 21.5, + "content_p95": 30.9, + "bitrate": 6256828, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 44, + "mbps": 7.24, + "content_p50": 22.0, + "content_p95": 47.5, + "bitrate": 6291668, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 44, + "mbps": 5.19, + "content_p50": 28.2, + "content_p95": 50.0, + "bitrate": 5347917, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 44, + "mbps": 4.54, + "content_p50": 23.7, + "content_p95": 43.0, + "bitrate": 3863869, + "tier": 1, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 35, + "mbps": 3.83, + "content_p50": 28.2, + "content_p95": 41.2, + "bitrate": 4222978, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 29, + "mbps": 4.65, + "content_p50": 30.0, + "content_p95": 53.8, + "bitrate": 5482055, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 28, + "mbps": 4.05, + "content_p50": 26.4, + "content_p95": 121.7, + "bitrate": 3981384, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 30, + "mbps": 3.12, + "content_p50": 24.5, + "content_p95": 42.4, + "bitrate": 2876549, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 31, + "mbps": 2.72, + "content_p50": 22.6, + "content_p95": 36.5, + "bitrate": 2445066, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 29, + "mbps": 2.4, + "content_p50": 26.4, + "content_p95": 46.1, + "bitrate": 2690671, + "tier": 2, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 26, + "mbps": 2.21, + "content_p50": 24.8, + "content_p95": 55.1, + "bitrate": 2168791, + "tier": 3, + "w": 1440, + "net": null + }, + { + "t": 15, + "fps": 31, + "mbps": 1.65, + "content_p50": 24.1, + "content_p95": 33.2, + "bitrate": 1566951, + "tier": 3, + "w": 1440, + "net": null + }, + { + "t": 16, + "fps": 30, + "mbps": 1.43, + "content_p50": 19.8, + "content_p95": 38.0, + "bitrate": 1317132, + "tier": 3, + "w": 1440, + "net": null + }, + { + "t": 17, + "fps": 31, + "mbps": 1.33, + "content_p50": 23.2, + "content_p95": 41.1, + "bitrate": 1190325, + "tier": 3, + "w": 1440, + "net": null + }, + { + "t": 18, + "fps": 25, + "mbps": 1.11, + "content_p50": 23.6, + "content_p95": 99.7, + "bitrate": 860009, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 19, + "fps": 29, + "mbps": 0.64, + "content_p50": 22.1, + "content_p95": 41.5, + "bitrate": 731007, + "tier": 4, + "w": 960, + "net": null + }, + { + "t": 20, + "fps": 21, + "mbps": 0.47, + "content_p50": 26.1, + "content_p95": 31.1, + "bitrate": 619308, + "tier": 4, + "w": 960, + "net": null + } + ], + "adapt": [], + "content_p50": 24.8, + "content_p95": 48.1, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "fps": "bad", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "source_fps": 54.5, + "cpu": { + "mac_agent_pct": 5.6, + "frame_viewer_pct": 57.2, + "frame_tailscaled_pct": 5.3, + "frame_sshd_pct": 0.7, + "frame_gamescope_pct": 8.9, + "frame_total_pct": 21.0 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 142 Hz", + "show_s": 6.59, + "panel": "valve.steam.desktopgame.2001551415", + "src": "separate:9635" + }, + { + "scenario": "type", + "frames_sent": 75, + "frames_drawn": 75, + "frames_shown": 75, + "duration_s": 20.0, + "stages_ms": { + "capture": { + "p50": 0.3, + "p95": 4.2, + "p99": 5.1, + "n": 75 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 75 + }, + "encode": { + "p50": 6.8, + "p95": 9.4, + "p99": 11.7, + "n": 75 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 75 + }, + "network": { + "p50": 4.0, + "p95": 12.4, + "p99": 26.6, + "n": 75 + }, + "decode": { + "p50": 2.3, + "p95": 7.2, + "p99": 14.4, + "n": 75 + }, + "draw": { + "p50": 0.7, + "p95": 1.5, + "p99": 1.6, + "n": 75 + }, + "present": { + "p50": 0.5, + "p95": 1.1, + "p99": 1.2, + "n": 75 + }, + "content": { + "p50": 14.4, + "p95": 25.1, + "p99": 37.7, + "n": 75 + }, + "content_shown": { + "p50": 14.9, + "p95": 25.6, + "p99": 38.9, + "n": 75 + } + }, + "fps": 3.8, + "fps_shown": 3.8, + "late_pct": 97.3, + "stall_max": 1109.7, + "stalls_over_100ms": 70, + "mbps": 0.03, + "keyframes": 2, + "size": "1920x1290", + "captured": 75, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 45.2, + "p95": 77.2, + "p99": 83.1, + "n": 66 + }, + "input_shown_ms": { + "p50": 45.9, + "p95": 77.9, + "p99": 84.3, + "n": 66 + }, + "input_parts_ms": { + "uplink": { + "p50": 5.8, + "p95": 16.5, + "p99": 29.9, + "n": 66 + }, + "mac": { + "p50": 24.4, + "p95": 41.2, + "p99": 41.6, + "n": 66 + }, + "back": { + "p50": 14.5, + "p95": 25.1, + "p99": 31.6, + "n": 66 + } + }, + "inputs": { + "sent": 66, + "seen": 66 + }, + "timeline": [ + { + "t": 0, + "fps": 4, + "mbps": 0.01, + "content_p50": 2.0, + "content_p95": 4.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 5, + "mbps": 0.02, + "content_p50": 15.7, + "content_p95": 37.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 5, + "mbps": 0.02, + "content_p50": 17.1, + "content_p95": 18.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 5, + "mbps": 0.02, + "content_p50": 15.7, + "content_p95": 26.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 5, + "mbps": 0.02, + "content_p50": 15.9, + "content_p95": 19.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 5, + "mbps": 0.02, + "content_p50": 17.8, + "content_p95": 21.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 5, + "mbps": 0.22, + "content_p50": 18.7, + "content_p95": 31.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 5, + "mbps": 0.02, + "content_p50": 18.1, + "content_p95": 25.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 5, + "mbps": 0.02, + "content_p50": 12.5, + "content_p95": 13.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 6, + "mbps": 0.02, + "content_p50": 6.4, + "content_p95": 16.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 6, + "mbps": 0.02, + "content_p50": 14.4, + "content_p95": 18.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 5, + "mbps": 0.02, + "content_p50": 11.9, + "content_p95": 14.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 3, + "mbps": 0.01, + "content_p50": 12.9, + "content_p95": 22.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 1, + "mbps": 0.0, + "content_p50": 13.5, + "content_p95": 13.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 3, + "mbps": 0.01, + "content_p50": 2.2, + "content_p95": 2.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 1, + "mbps": 0.0, + "content_p50": 110.2, + "content_p95": 110.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 16, + "fps": 1, + "mbps": 0.0, + "content_p50": 14.4, + "content_p95": 14.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 17, + "fps": 2, + "mbps": 0.25, + "content_p50": -0.0, + "content_p95": 17.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 18, + "fps": 1, + "mbps": 0.0, + "content_p50": 14.1, + "content_p95": 14.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 19, + "fps": 1, + "mbps": 0.0, + "content_p50": 17.5, + "content_p95": 17.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 14.4, + "content_p95": 25.1, + "input_p50": 45.2, + "input_p95": 77.2, + "grades": { + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "acceptable" + }, + "source_fps": 3.8, + "cpu": { + "mac_agent_pct": 1.2, + "frame_viewer_pct": 7.1, + "frame_tailscaled_pct": 2.2, + "frame_sshd_pct": 0.3, + "frame_gamescope_pct": 4.7, + "frame_total_pct": 15.3 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 141 Hz", + "show_s": 6.64, + "panel": "valve.steam.desktopgame.2001684656", + "src": "separate:9763" + } + ] +} \ No newline at end of file diff --git a/bench/results/2026-09-28-a3c6e5c-dirty-baseline-fixed.json b/bench/results/2026-09-28-a3c6e5c-dirty-baseline-fixed.json new file mode 100644 index 0000000..d12e58c --- /dev/null +++ b/bench/results/2026-09-28-a3c6e5c-dirty-baseline-fixed.json @@ -0,0 +1,1124 @@ +{ + "label": "baseline-fixed", + "date": "2026-09-28T17:02:53", + "commit": "a3c6e5c+dirty", + "config": { + "quality": "balanced", + "mode": "separate", + "net": "none", + "delay_ms": 0, + "buffer_ms": 250, + "host": "frame", + "ssh_opts": [], + "encoder": "", + "duration_s": 20.0, + "browser_flags": [] + }, + "frame_build": "20260925.6191901", + "mac": "26.5.2", + "headset": ":51.824645 [Info] - 0 - entering standby", + "scenarios": [ + { + "scenario": "test", + "frames_sent": 1246, + "frames_drawn": 1223, + "frames_shown": 804, + "duration_s": 20.7, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 1246 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 1246 + }, + "encode": { + "p50": 4.0, + "p95": 4.7, + "p99": 4.9, + "n": 1246 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 1246 + }, + "network": { + "p50": 4.0, + "p95": 6.8, + "p99": 221.1, + "n": 1246 + }, + "decode": { + "p50": 1.3, + "p95": 2.5, + "p99": 4.2, + "n": 1223 + }, + "draw": { + "p50": 0.4, + "p95": 0.5, + "p99": 0.9, + "n": 1223 + }, + "present": { + "p50": 6.2, + "p95": 15.6, + "p99": 16.9, + "n": 804 + }, + "content": { + "p50": 9.9, + "p95": 12.9, + "p99": 16.4, + "n": 1223 + }, + "content_shown": { + "p50": 16.1, + "p95": 25.5, + "p99": 27.9, + "n": 804 + } + }, + "fps": 59.0, + "fps_shown": 38.8, + "late_pct": 0.33, + "stall_max": 433.3, + "stalls_over_100ms": 1, + "mbps": 0.48, + "keyframes": 2, + "size": "1280x720", + "captured": 1598, + "viewer_never_drawn": 23, + "input_ms": { + "p50": 28.8, + "p95": 39.0, + "p99": 45.2, + "n": 40 + }, + "input_shown_ms": { + "p50": 33.7, + "p95": 55.8, + "p99": 58.6, + "n": 29 + }, + "input_parts_ms": { + "uplink": { + "p50": 9.8, + "p95": 21.7, + "p99": 22.5, + "n": 40 + }, + "mac": { + "p50": 4.5, + "p95": 15.7, + "p99": 18.4, + "n": 40 + }, + "back": { + "p50": 10.6, + "p95": 13.2, + "p99": 15.7, + "n": 40 + } + }, + "inputs": { + "sent": 40, + "seen": 40 + }, + "timeline": [ + { + "t": 0, + "fps": 59, + "mbps": 0.49, + "content_p50": 9.3, + "content_p95": 11.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.6, + "content_p95": 12.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.48, + "content_p50": 9.2, + "content_p95": 11.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.48, + "content_p50": 9.8, + "content_p95": 11.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 38, + "mbps": 0.58, + "content_p50": 9.3, + "content_p95": 395.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.48, + "content_p50": 9.3, + "content_p95": 11.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 60, + "mbps": 0.47, + "content_p50": 9.4, + "content_p95": 12.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.5, + "content_p95": 10.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.46, + "content_p50": 8.9, + "content_p95": 11.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 60, + "mbps": 0.45, + "content_p50": 8.8, + "content_p95": 12.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.45, + "content_p50": 11.5, + "content_p95": 14.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 60, + "mbps": 0.46, + "content_p50": 11.2, + "content_p95": 14.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 61, + "mbps": 0.47, + "content_p50": 10.7, + "content_p95": 12.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 59, + "mbps": 0.44, + "content_p50": 10.3, + "content_p95": 13.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 61, + "mbps": 0.57, + "content_p50": 10.1, + "content_p95": 14.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 59, + "mbps": 0.45, + "content_p50": 10.2, + "content_p95": 12.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 16, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.1, + "content_p95": 11.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 17, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.0, + "content_p95": 11.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 18, + "fps": 60, + "mbps": 0.48, + "content_p50": 9.9, + "content_p95": 11.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 19, + "fps": 60, + "mbps": 0.47, + "content_p50": 9.5, + "content_p95": 13.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 20, + "fps": 46, + "mbps": 0.34, + "content_p50": 9.2, + "content_p95": 13.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 9.9, + "content_p95": 12.9, + "input_p50": 28.8, + "input_p95": 39.0, + "grades": { + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "local", + "stall_max": "bad" + }, + "cpu": { + "mac_agent_pct": 14.3, + "frame_viewer_pct": 49.7, + "frame_tailscaled_pct": 5.7, + "frame_sshd_pct": 0.7, + "frame_gamescope_pct": 6.7, + "frame_total_pct": 16.5 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 142 Hz", + "show_s": 3.62, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 1047, + "frames_drawn": 1042, + "frames_shown": 798, + "duration_s": 20.7, + "stages_ms": { + "capture": { + "p50": -5.5, + "p95": 5.8, + "p99": 14.2, + "n": 1047 + }, + "queue": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.1, + "n": 1047 + }, + "encode": { + "p50": 6.7, + "p95": 9.0, + "p99": 14.1, + "n": 1047 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.3, + "n": 1047 + }, + "network": { + "p50": 4.9, + "p95": 9.3, + "p99": 47.3, + "n": 1047 + }, + "decode": { + "p50": 6.4, + "p95": 15.2, + "p99": 22.6, + "n": 1042 + }, + "draw": { + "p50": 0.6, + "p95": 0.8, + "p99": 1.6, + "n": 1042 + }, + "present": { + "p50": 4.7, + "p95": 19.6, + "p99": 24.9, + "n": 798 + }, + "content": { + "p50": 14.7, + "p95": 32.2, + "p99": 73.2, + "n": 1042 + }, + "content_shown": { + "p50": 21.1, + "p95": 41.7, + "p99": 66.3, + "n": 798 + } + }, + "fps": 50.4, + "fps_shown": 38.6, + "late_pct": 16.04, + "stall_max": 152.4, + "stalls_over_100ms": 3, + "mbps": 9.37, + "keyframes": 3, + "size": "1920x1290", + "captured": 1312, + "viewer_never_drawn": 5, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 51, + "mbps": 8.99, + "content_p50": 14.9, + "content_p95": 32.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 55, + "mbps": 10.09, + "content_p50": 17.4, + "content_p95": 26.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 51, + "mbps": 8.51, + "content_p50": 15.1, + "content_p95": 25.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 44, + "mbps": 7.9, + "content_p50": 16.7, + "content_p95": 73.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 46, + "mbps": 11.34, + "content_p50": 17.4, + "content_p95": 120.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 45, + "mbps": 8.55, + "content_p50": 15.1, + "content_p95": 25.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 51, + "mbps": 9.61, + "content_p50": 17.4, + "content_p95": 24.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 54, + "mbps": 9.52, + "content_p50": 16.9, + "content_p95": 27.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 50, + "mbps": 8.51, + "content_p50": 15.8, + "content_p95": 33.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 54, + "mbps": 9.22, + "content_p50": 12.4, + "content_p95": 27.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 51, + "mbps": 9.96, + "content_p50": 9.9, + "content_p95": 28.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 51, + "mbps": 9.27, + "content_p50": 14.1, + "content_p95": 22.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 52, + "mbps": 9.58, + "content_p50": 14.4, + "content_p95": 25.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 54, + "mbps": 8.61, + "content_p50": 7.8, + "content_p95": 22.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 43, + "mbps": 10.54, + "content_p50": 19.0, + "content_p95": 37.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 49, + "mbps": 8.51, + "content_p50": 16.1, + "content_p95": 27.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 16, + "fps": 51, + "mbps": 11.23, + "content_p50": 11.1, + "content_p95": 110.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 17, + "fps": 52, + "mbps": 8.99, + "content_p50": 13.9, + "content_p95": 26.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 18, + "fps": 52, + "mbps": 9.05, + "content_p50": 11.6, + "content_p95": 26.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 19, + "fps": 49, + "mbps": 9.3, + "content_p50": 12.4, + "content_p95": 28.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 20, + "fps": 37, + "mbps": 6.58, + "content_p50": 15.7, + "content_p95": 31.0, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 14.7, + "content_p95": 32.2, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "local", + "fps": "acceptable", + "late_pct": "bad", + "stall_max": "acceptable" + }, + "cpu": { + "mac_agent_pct": 7.6, + "frame_viewer_pct": 73.9, + "frame_tailscaled_pct": 8.0, + "frame_sshd_pct": 0.9, + "frame_gamescope_pct": 6.5, + "frame_total_pct": 19.4 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 138 Hz", + "show_s": 3.38, + "panel": "valve.steam.desktopgame.2001739297", + "src": "separate:8529" + }, + { + "scenario": "type", + "frames_sent": 77, + "frames_drawn": 77, + "frames_shown": 77, + "duration_s": 20.0, + "stages_ms": { + "capture": { + "p50": 0.2, + "p95": 4.7, + "p99": 4.8, + "n": 77 + }, + "queue": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.1, + "n": 77 + }, + "encode": { + "p50": 6.8, + "p95": 11.7, + "p99": 14.7, + "n": 77 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 77 + }, + "network": { + "p50": 4.9, + "p95": 8.7, + "p99": 75.8, + "n": 77 + }, + "decode": { + "p50": 4.2, + "p95": 9.9, + "p99": 14.5, + "n": 77 + }, + "draw": { + "p50": 1.0, + "p95": 1.6, + "p99": 2.1, + "n": 77 + }, + "present": { + "p50": 0.9, + "p95": 1.3, + "p99": 1.7, + "n": 77 + }, + "content": { + "p50": 16.7, + "p95": 34.2, + "p99": 88.1, + "n": 77 + }, + "content_shown": { + "p50": 17.9, + "p95": 34.6, + "p99": 88.9, + "n": 77 + } + }, + "fps": 3.8, + "fps_shown": 3.8, + "late_pct": 94.74, + "stall_max": 1156.0, + "stalls_over_100ms": 71, + "mbps": 0.03, + "keyframes": 2, + "size": "1920x1290", + "captured": 151, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 51.2, + "p95": 73.2, + "p99": 79.9, + "n": 66 + }, + "input_shown_ms": { + "p50": 52.1, + "p95": 73.9, + "p99": 80.5, + "n": 66 + }, + "input_parts_ms": { + "uplink": { + "p50": 5.6, + "p95": 13.2, + "p99": 17.5, + "n": 66 + }, + "mac": { + "p50": 34.1, + "p95": 43.6, + "p99": 48.7, + "n": 66 + }, + "back": { + "p50": 16.4, + "p95": 26.5, + "p99": 26.8, + "n": 66 + } + }, + "inputs": { + "sent": 66, + "seen": 66 + }, + "timeline": [ + { + "t": 0, + "fps": 5, + "mbps": 0.01, + "content_p50": 7.7, + "content_p95": 15.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 1, + "fps": 5, + "mbps": 0.02, + "content_p50": 7.4, + "content_p95": 16.7, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 2, + "fps": 5, + "mbps": 0.02, + "content_p50": 16.2, + "content_p95": 21.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 3, + "fps": 6, + "mbps": 0.02, + "content_p50": 19.1, + "content_p95": 26.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 4, + "fps": 5, + "mbps": 0.02, + "content_p50": 18.9, + "content_p95": 20.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 5, + "fps": 5, + "mbps": 0.21, + "content_p50": 13.7, + "content_p95": 43.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 6, + "fps": 5, + "mbps": 0.02, + "content_p50": 20.2, + "content_p95": 24.5, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 7, + "fps": 4, + "mbps": 0.01, + "content_p50": 16.4, + "content_p95": 16.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 8, + "fps": 5, + "mbps": 0.02, + "content_p50": 15.2, + "content_p95": 24.6, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 9, + "fps": 5, + "mbps": 0.02, + "content_p50": 7.7, + "content_p95": 17.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 10, + "fps": 5, + "mbps": 0.02, + "content_p50": 6.0, + "content_p95": 26.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 11, + "fps": 7, + "mbps": 0.02, + "content_p50": 19.2, + "content_p95": 46.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 12, + "fps": 4, + "mbps": 0.01, + "content_p50": 21.7, + "content_p95": 22.3, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 13, + "fps": 1, + "mbps": 0.0, + "content_p50": 13.4, + "content_p95": 13.4, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 14, + "fps": 3, + "mbps": 0.01, + "content_p50": 19.0, + "content_p95": 20.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 15, + "fps": 1, + "mbps": 0.0, + "content_p50": 88.1, + "content_p95": 88.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 16, + "fps": 1, + "mbps": 0.25, + "content_p50": 26.8, + "content_p95": 26.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 17, + "fps": 2, + "mbps": 0.01, + "content_p50": 19.1, + "content_p95": 19.2, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 18, + "fps": 1, + "mbps": 0.0, + "content_p50": 175.1, + "content_p95": 175.1, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 19, + "fps": 1, + "mbps": 0.0, + "content_p50": 16.9, + "content_p95": 16.9, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + }, + { + "t": 20, + "fps": 1, + "mbps": 0.0, + "content_p50": 14.8, + "content_p95": 14.8, + "bitrate": 14860800, + "tier": 0, + "w": 1920, + "net": null + } + ], + "adapt": [], + "content_p50": 16.7, + "content_p95": 34.2, + "input_p50": 51.2, + "input_p95": 73.2, + "grades": { + "content_p50": "local", + "content_p95": "local", + "input_p50": "acceptable", + "input_p95": "acceptable" + }, + "cpu": { + "mac_agent_pct": 1.1, + "frame_viewer_pct": 8.0, + "frame_tailscaled_pct": 1.8, + "frame_sshd_pct": 0.5, + "frame_gamescope_pct": 3.7, + "frame_total_pct": 9.1 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 143 Hz", + "show_s": 3.79, + "panel": "valve.steam.desktopgame.2001132791", + "src": "separate:8652" + } + ] +} \ No newline at end of file diff --git a/bench/results/2026-09-28-a3c6e5c-dirty-baseline.json b/bench/results/2026-09-28-a3c6e5c-dirty-baseline.json new file mode 100644 index 0000000..25b9f92 --- /dev/null +++ b/bench/results/2026-09-28-a3c6e5c-dirty-baseline.json @@ -0,0 +1,1124 @@ +{ + "label": "baseline", + "date": "2026-09-28T16:53:43", + "commit": "a3c6e5c+dirty", + "config": { + "quality": "balanced", + "mode": "separate", + "net": "none", + "delay_ms": 0, + "buffer_ms": 250, + "host": "frame", + "ssh_opts": [], + "encoder": "", + "duration_s": 20.0, + "browser_flags": [] + }, + "frame_build": "20260925.6191901", + "mac": "26.5.2", + "headset": ":51.824645 [Info] - 0 - entering standby", + "scenarios": [ + { + "scenario": "test", + "frames_sent": 1248, + "frames_drawn": 1245, + "frames_shown": 809, + "duration_s": 20.8, + "stages_ms": { + "capture": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.0, + "n": 1248 + }, + "queue": { + "p50": 0.0, + "p95": 0.1, + "p99": 0.1, + "n": 1248 + }, + "encode": { + "p50": 4.2, + "p95": 4.8, + "p99": 5.1, + "n": 1248 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 1248 + }, + "network": { + "p50": 4.2, + "p95": 6.9, + "p99": 13.9, + "n": 1248 + }, + "decode": { + "p50": 1.3, + "p95": 2.5, + "p99": 4.6, + "n": 1245 + }, + "draw": { + "p50": 0.4, + "p95": 0.6, + "p99": 1.0, + "n": 1245 + }, + "present": { + "p50": 6.9, + "p95": 15.4, + "p99": 19.5, + "n": 809 + }, + "content": { + "p50": 10.3, + "p95": 13.5, + "p99": 19.6, + "n": 1245 + }, + "content_shown": { + "p50": 17.4, + "p95": 25.6, + "p99": 30.7, + "n": 809 + } + }, + "fps": 60.0, + "fps_shown": 39.0, + "late_pct": 0.96, + "stall_max": 110.3, + "stalls_over_100ms": 1, + "mbps": 0.47, + "keyframes": 2, + "size": "1280x720", + "captured": 1588, + "viewer_never_drawn": 3, + "input_ms": { + "p50": 28.0, + "p95": 34.4, + "p99": 37.6, + "n": 40 + }, + "input_shown_ms": { + "p50": 35.1, + "p95": 49.5, + "p99": 50.5, + "n": 27 + }, + "input_parts_ms": { + "uplink": { + "p50": 11.5, + "p95": 19.7, + "p99": 22.1, + "n": 40 + }, + "mac": { + "p50": 4.7, + "p95": 12.3, + "p99": 16.1, + "n": 40 + }, + "back": { + "p50": 11.2, + "p95": 12.9, + "p99": 16.5, + "n": 40 + } + }, + "inputs": { + "sent": 40, + "seen": 40 + }, + "timeline": [ + { + "t": 0, + "fps": 60, + "mbps": 0.49, + "content_p50": 10.2, + "content_p95": 13.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.4, + "content_p95": 12.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.5, + "content_p95": 12.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 60, + "mbps": 0.48, + "content_p50": 10.7, + "content_p95": 12.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.6, + "content_p95": 13.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 60, + "mbps": 0.58, + "content_p50": 10.6, + "content_p95": 12.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 60, + "mbps": 0.45, + "content_p50": 10.8, + "content_p95": 13.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.5, + "content_p95": 13.0, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 60, + "mbps": 0.47, + "content_p50": 10.5, + "content_p95": 12.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.3, + "content_p95": 13.6, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 60, + "mbps": 0.46, + "content_p50": 10.8, + "content_p95": 14.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 58, + "mbps": 0.56, + "content_p50": 10.4, + "content_p95": 61.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 60, + "mbps": 0.45, + "content_p50": 10.8, + "content_p95": 12.7, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 60, + "mbps": 0.46, + "content_p50": 12.4, + "content_p95": 16.1, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.8, + "content_p95": 12.2, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 60, + "mbps": 0.47, + "content_p50": 9.7, + "content_p95": 12.3, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 16, + "fps": 60, + "mbps": 0.45, + "content_p50": 9.7, + "content_p95": 12.5, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 17, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.7, + "content_p95": 10.9, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 18, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.7, + "content_p95": 11.4, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 19, + "fps": 60, + "mbps": 0.46, + "content_p50": 9.9, + "content_p95": 18.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 20, + "fps": 47, + "mbps": 0.36, + "content_p50": 9.9, + "content_p95": 11.8, + "bitrate": 5529600, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 10.3, + "content_p95": 13.5, + "input_p50": 28.0, + "input_p95": 34.4, + "grades": { + "content_p50": "local", + "content_p95": "local", + "input_p50": "local", + "input_p95": "local", + "fps": "local", + "late_pct": "local", + "stall_max": "acceptable" + }, + "cpu": { + "mac_agent_pct": 15.4, + "frame_viewer_pct": 51.1, + "frame_tailscaled_pct": 5.3, + "frame_sshd_pct": 0.8, + "frame_gamescope_pct": 5.7, + "frame_total_pct": 15.3 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 140 Hz", + "show_s": 1.87, + "panel": "valve.steam.desktopgame.2001639889", + "src": "test" + }, + { + "scenario": "scroll", + "frames_sent": 690, + "frames_drawn": 683, + "frames_shown": 645, + "duration_s": 20.7, + "stages_ms": { + "capture": { + "p50": -1.2, + "p95": 16.9, + "p99": 24.5, + "n": 690 + }, + "queue": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.1, + "n": 690 + }, + "encode": { + "p50": 4.1, + "p95": 7.0, + "p99": 13.2, + "n": 690 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.3, + "n": 690 + }, + "network": { + "p50": 6.2, + "p95": 14.6, + "p99": 119.2, + "n": 690 + }, + "decode": { + "p50": 3.7, + "p95": 9.4, + "p99": 23.3, + "n": 683 + }, + "draw": { + "p50": 0.4, + "p95": 0.8, + "p99": 1.2, + "n": 683 + }, + "present": { + "p50": 0.8, + "p95": 18.6, + "p99": 23.6, + "n": 645 + }, + "content": { + "p50": 15.0, + "p95": 40.3, + "p99": 112.7, + "n": 683 + }, + "content_shown": { + "p50": 20.2, + "p95": 42.7, + "p99": 106.9, + "n": 645 + } + }, + "fps": 33.0, + "fps_shown": 31.1, + "late_pct": 42.82, + "stall_max": 492.6, + "stalls_over_100ms": 6, + "mbps": 3.11, + "keyframes": 3, + "size": "1280x860", + "captured": 905, + "viewer_never_drawn": 7, + "input_ms": { + "n": 0 + }, + "input_shown_ms": { + "n": 0 + }, + "input_parts_ms": { + "uplink": { + "n": 0 + }, + "mac": { + "n": 0 + }, + "back": { + "n": 0 + } + }, + "inputs": { + "sent": 0, + "seen": 0 + }, + "timeline": [ + { + "t": 0, + "fps": 36, + "mbps": 3.42, + "content_p50": 17.7, + "content_p95": 34.3, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 35, + "mbps": 2.8, + "content_p50": 13.6, + "content_p95": 27.7, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 31, + "mbps": 2.15, + "content_p50": 12.1, + "content_p95": 34.1, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 37, + "mbps": 2.29, + "content_p50": 14.6, + "content_p95": 28.5, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 35, + "mbps": 2.11, + "content_p50": 16.1, + "content_p95": 29.3, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 33, + "mbps": 2.05, + "content_p50": 10.9, + "content_p95": 27.9, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 33, + "mbps": 5.71, + "content_p50": 17.1, + "content_p95": 41.7, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 31, + "mbps": 3.3, + "content_p50": 12.1, + "content_p95": 31.3, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 24, + "mbps": 4.37, + "content_p50": 26.6, + "content_p95": 236.8, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 36, + "mbps": 3.19, + "content_p50": 11.6, + "content_p95": 26.5, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 36, + "mbps": 3.53, + "content_p50": 15.2, + "content_p95": 29.6, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 29, + "mbps": 3.56, + "content_p50": 11.5, + "content_p95": 36.8, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 29, + "mbps": 1.99, + "content_p50": 13.2, + "content_p95": 41.2, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 31, + "mbps": 3.4, + "content_p50": 23.9, + "content_p95": 44.5, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 32, + "mbps": 2.85, + "content_p50": 14.3, + "content_p95": 37.6, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 31, + "mbps": 3.39, + "content_p50": 19.8, + "content_p95": 35.9, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 16, + "fps": 22, + "mbps": 2.74, + "content_p50": 22.5, + "content_p95": 112.7, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 17, + "fps": 44, + "mbps": 3.2, + "content_p50": 14.0, + "content_p95": 31.0, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 18, + "fps": 35, + "mbps": 4.12, + "content_p50": 16.8, + "content_p95": 32.9, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 19, + "fps": 36, + "mbps": 2.63, + "content_p50": 13.9, + "content_p95": 25.2, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 20, + "fps": 27, + "mbps": 1.71, + "content_p50": 17.2, + "content_p95": 40.3, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 15.0, + "content_p95": 40.3, + "input_p50": null, + "input_p95": null, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "fps": "bad", + "late_pct": "bad", + "stall_max": "bad" + }, + "cpu": { + "mac_agent_pct": 5.1, + "frame_viewer_pct": 44.2, + "frame_tailscaled_pct": 4.8, + "frame_sshd_pct": 0.6, + "frame_gamescope_pct": 4.2, + "frame_total_pct": 13.7 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz", + "show_s": 1.82, + "panel": "valve.steam.desktopgame.2001631563", + "src": "separate:6889" + }, + { + "scenario": "type", + "frames_sent": 76, + "frames_drawn": 76, + "frames_shown": 76, + "duration_s": 20.2, + "stages_ms": { + "capture": { + "p50": 0.5, + "p95": 7.2, + "p99": 10.1, + "n": 76 + }, + "queue": { + "p50": 0.0, + "p95": 0.0, + "p99": 0.1, + "n": 76 + }, + "encode": { + "p50": 4.4, + "p95": 9.7, + "p99": 16.7, + "n": 76 + }, + "socket": { + "p50": 0.1, + "p95": 0.2, + "p99": 0.2, + "n": 76 + }, + "network": { + "p50": 5.2, + "p95": 43.2, + "p99": 263.5, + "n": 76 + }, + "decode": { + "p50": 2.1, + "p95": 4.8, + "p99": 13.4, + "n": 76 + }, + "draw": { + "p50": 0.7, + "p95": 1.0, + "p99": 1.1, + "n": 76 + }, + "present": { + "p50": 0.9, + "p95": 1.5, + "p99": 1.8, + "n": 76 + }, + "content": { + "p50": 12.9, + "p95": 54.2, + "p99": 264.6, + "n": 76 + }, + "content_shown": { + "p50": 14.1, + "p95": 55.3, + "p99": 265.9, + "n": 76 + } + }, + "fps": 3.8, + "fps_shown": 3.8, + "late_pct": 96.0, + "stall_max": 1327.9, + "stalls_over_100ms": 68, + "mbps": 0.02, + "keyframes": 2, + "size": "1280x860", + "captured": 125, + "viewer_never_drawn": 0, + "input_ms": { + "p50": 50.1, + "p95": 110.6, + "p99": 165.7, + "n": 65 + }, + "input_shown_ms": { + "p50": 50.8, + "p95": 111.5, + "p99": 166.2, + "n": 65 + }, + "input_parts_ms": { + "uplink": { + "p50": 7.6, + "p95": 18.3, + "p99": 23.2, + "n": 65 + }, + "mac": { + "p50": 30.6, + "p95": 72.4, + "p99": 119.2, + "n": 65 + }, + "back": { + "p50": 12.3, + "p95": 28.1, + "p99": 33.9, + "n": 65 + } + }, + "inputs": { + "sent": 66, + "seen": 65 + }, + "timeline": [ + { + "t": 0, + "fps": 5, + "mbps": 0.01, + "content_p50": 12.3, + "content_p95": 28.1, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 1, + "fps": 4, + "mbps": 0.01, + "content_p50": 16.8, + "content_p95": 33.9, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 2, + "fps": 6, + "mbps": 0.01, + "content_p50": 9.0, + "content_p95": 22.2, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 3, + "fps": 5, + "mbps": 0.01, + "content_p50": 11.5, + "content_p95": 20.6, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 4, + "fps": 5, + "mbps": 0.01, + "content_p50": 9.2, + "content_p95": 13.1, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 5, + "fps": 5, + "mbps": 0.01, + "content_p50": 3.0, + "content_p95": 15.1, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 6, + "fps": 5, + "mbps": 0.14, + "content_p50": 14.0, + "content_p95": 21.1, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 7, + "fps": 5, + "mbps": 0.01, + "content_p50": 9.5, + "content_p95": 16.2, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 8, + "fps": 5, + "mbps": 0.01, + "content_p50": 13.8, + "content_p95": 16.2, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 9, + "fps": 5, + "mbps": 0.01, + "content_p50": 14.0, + "content_p95": 28.1, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 10, + "fps": 5, + "mbps": 0.01, + "content_p50": 12.8, + "content_p95": 31.7, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 11, + "fps": 6, + "mbps": 0.01, + "content_p50": 12.3, + "content_p95": 14.4, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 12, + "fps": 4, + "mbps": 0.01, + "content_p50": 15.2, + "content_p95": 17.2, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 13, + "fps": 1, + "mbps": 0.0, + "content_p50": 18.9, + "content_p95": 18.9, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 14, + "fps": 2, + "mbps": 0.0, + "content_p50": 203.5, + "content_p95": 215.0, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 15, + "fps": 1, + "mbps": 0.0, + "content_p50": 16.1, + "content_p95": 16.1, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 16, + "fps": 2, + "mbps": 0.16, + "content_p50": 15.5, + "content_p95": 35.1, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 17, + "fps": 1, + "mbps": 0.0, + "content_p50": 264.6, + "content_p95": 264.6, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 18, + "fps": 1, + "mbps": 0.0, + "content_p50": 12.9, + "content_p95": 12.9, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 19, + "fps": 2, + "mbps": 0.0, + "content_p50": 11.7, + "content_p95": 54.2, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + }, + { + "t": 20, + "fps": 1, + "mbps": 0.0, + "content_p50": 339.6, + "content_p95": 339.6, + "bitrate": 6604800, + "tier": 0, + "w": 1280, + "net": null + } + ], + "adapt": [], + "content_p50": 12.9, + "content_p95": 54.2, + "input_p50": 50.1, + "input_p95": 110.6, + "grades": { + "content_p50": "local", + "content_p95": "acceptable", + "input_p50": "acceptable", + "input_p95": "acceptable" + }, + "cpu": { + "mac_agent_pct": 1.2, + "frame_viewer_pct": 6.5, + "frame_tailscaled_pct": 1.6, + "frame_sshd_pct": 0.3, + "frame_gamescope_pct": 2.4, + "frame_total_pct": 8.0 + }, + "viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 141 Hz", + "show_s": 1.43, + "panel": "valve.steam.desktopgame.2001427053", + "src": "separate:7011" + } + ] +} \ No newline at end of file diff --git a/docs/how-the-frame-works.md b/docs/how-the-frame-works.md index 9681da8..bcac736 100644 --- a/docs/how-the-frame-works.md +++ b/docs/how-the-frame-works.md @@ -52,6 +52,7 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600 | Power actions need `sudo`, which asks for the Developer Mode password over SSH. | Frame Control's power buttons | | **SSH server:** OpenSSH 9.7p1. It offers `publickey,password` (keyboard-interactive is off, PAM on) and also asks `userdbctl ssh-authorized-keys` for keys. OpenSSH ≥ 8.8 rejects SHA-1 `ssh-rsa` signatures by default, so a client whose RSA support is SHA-1 only (the Swift library Citadel, for one) can't log in with the RSA key that devkit pairing installs; use ed25519 (**inferred** from OpenSSH defaults). **Verified 2026-09-27**, BUILD_ID 20260922.6101926. | [iphone.md](iphone.md), `ui/frame_connect.py` | | **A Chromium app window on gamescope's `:0` with its own `STEAM_GAME` becomes a panel, even when started over SSH.** `chromium-xr/chrome --ozone-platform=x11 --app=URL --window-size=1280,720` got a 1920×1080 window, and tagging it produced `[Overlays] Created: valve.steam.desktopgame.` in `~/.local/share/Steam/logs/vrwebhelper_systemui.txt`. Chromium XR decoded a 1080p H.264 WebCodecs stream from the Mac at about 60 fps. XTest events sent to `:0` (libXtst through Python ctypes) did not reach the page. The Frame has `libXtst`, `xprop`, `xwininfo`, `curl` and the `C.utf8` locale. **Verified 2026-09-28**, BUILD_ID 20260925.6191901. | [mac-in-headset.md](mac-in-headset.md) | +| **Streaming video into a panel: what the Frame adds.** Chromium XR decodes H.264 in **software** (1920×1290 at about 9 Mbit/s took 4–6 ms per frame); its GL is ANGLE → zink → Turnip on the Adreno 750, and it logs `GetVSyncParametersIfAvailable() failed`. An idle page's `requestAnimationFrame` runs at about 140 Hz; when the headset is worn or woken, SteamVR picks the 4320×2160 @ 144 Hz mode. **An unworn headset throttles panel apps** whatever they draw: a local canvas page ran at 58 fps for about 6 s, then 28, then about 15 once in standby (vrserver logs `entering standby`, with `power.pauseCompositorOnStandby 1`). `vrcmd --handlewakeup` gave 137–144 fps for about 2 s, then 36. So a panel's frame rate and compositor delay can only be measured while it's worn. `tailscaled` runs with `--tun=userspace-networking` and cost about 8% of a core at 9 Mbit/s (0.5% over the LAN address). Wi-Fi power saving is on (`iw … get power_save`). **Verified 2026-09-28**, BUILD_ID 20260925.6191901. | [mac-in-headset.md](mac-in-headset.md#measuring) | | **Tools on the image:** Python 3.12.3, `ffmpeg`, `openssl`, `curl`, `rsync`, `zip`/`unzip`, `flatpak`, `wpctl`, `podman`. **No `adb`.** `steamos` is uid 1000, in `wheel`, and sudoers has `%wheel ALL=(ALL) ALL`, so `sudo -S` takes the Developer Mode password on stdin. **Verified 2026-09-27.** | Running Frame Control's server on the Frame (`FRAME_LOCAL=1`, [iphone.md](iphone.md)) | | **Each Lepton instance is a podman container** named `lepton-steamlaunch-`, labelled with its ADB port (`podman ps --format '{{.Names}} {{.Labels.adb_port}}'`). `podman exec /system/bin/sh -c '…'` runs Android's shell inside it with no adb at all (used for `pidof` and `logcat` by the app tester). Running `wm size`/`wm density` that way is untested. **Verified 2026-09-27.** | `ui/frame_android.py`, the iPhone app's display settings | | **Asleep means off the network.** In standby the Frame stops answering on its LAN address, `frame.local` and Tailscale alike (`Host is down`, `No route to host`, timeouts), and ping fails. It was unreachable for about 2.5 hours until woken. Nothing over SSH can wake it. **Verified 2026-09-27.** | Frame Control's offline banner and retries | diff --git a/docs/mac-in-headset.md b/docs/mac-in-headset.md index 55728bc..142c9e3 100644 --- a/docs/mac-in-headset.md +++ b/docs/mac-in-headset.md @@ -99,9 +99,10 @@ ScreenCaptureKit (one window or display) keyframe instead of showing old frames late. - It falls back to JPEG stills (**Compatible** quality) where H.264 isn't available. -- **Flow control.** When the link backs up, the agent skips capture frames - *before* encoding, so no reference frame goes missing. Once the link drains, - it sends the newest picture. +- **Flow control.** The agent never lets frames queue up anywhere on the + way. It skips capture frames *before* encoding, so no reference frame goes + missing, and it lowers the bitrate, then the frame rate, then the size, to + fit the link (see [Adapting to the network](#adapting-to-the-network)). - **Input.** - A click on a window's panel brings that Mac window to the front (Accessibility API), then clicks at the same point. Double clicks, right @@ -133,6 +134,157 @@ may ask again after an update. | Light | 1280 | 30 | H.264 | Weak Wi-Fi or Tailscale off the LAN | | Compatible | 1280 | 20 | JPEG | A Frame browser without H.264 | +## Measuring + +Every frame carries a sequence number, and the agent records its journey on +the Mac's clock (`Sources/Stats.swift`): + +| Stage | From → to | +|---|---| +| capture | the Mac composited it (ScreenCaptureKit's display time) → the agent got it | +| queue, encode | → encoding started → the encoder finished | +| network | → the viewer received it | +| decode, draw | → WebCodecs decoded it → it was drawn on the page's canvas | +| present | → the page's next animation frame | + +- **Clock sync.** The viewer syncs its clock to the Mac's the way NTP does: + it pings over the stream's own WebSocket and keeps the sample with the + shortest round trip. It then reports, in Mac time, when each frame arrived + (right away, so the agent can pace itself) and when it was decoded and + drawn (in batches every 250 ms). +- **Input.** The first frame captured after a click or key carries that + event's id. So input latency is the viewer's event → injected on the Mac → + the first frame after it → drawn in the headset. +- **Where to see it.** + - `GET /stats` (key required) returns every frame and input record. + - `/status` includes a two-second summary, which Frame Control's card + shows next to each live stream. + - In the headset, add `?stats=1` to the viewer or press + Ctrl+Alt+Shift+S for an overlay. +- **The benchmark.** `scripts/macview-bench.py` runs fixed scenarios on the + real Frame, from the Mac, with nobody wearing the headset: + - **test** is the moving test pattern. + - **scroll** is a Chrome page on its own display scrolling at 240 pt/s, + which gives about 9 Mbit/s of real 1920×1290 video. + - **type** types into a Chrome text box, first fast and then with pauses. + + It writes `bench/results/--