Compare commits

...
30 Commits
Author SHA1 Message Date
saphidandClaude Sonnet 5.5 024ec593f9 Merge origin/main into pad-gamescope
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 21:06:06 +10:00
saphidandClaude Sonnet 5.5 0373555519 Pad: count gamescope's real typing time, with the settle margin added once
Review found the estimate used 12 ms a character where the agent takes 16 ms
(32 shifted), and added the margin on every request so it piled up.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 21:01:53 +10:00
saphidandClaude Sonnet 5.5 3539215517 Pad: drain the KDE Connect queue without losing, stranding or reordering input
Second review found the queue could be trimmed under an in-flight send (deleting
unsent events), the worker could retire as a request enqueued, and ASCII could
overtake an accent still being typed (or the reverse). The worker now takes its
batch out while sending and puts it back on failure, retires under the lock, and
each way waits out the other's estimated typing time.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 20:57:08 +10:00
saphidandClaude Sonnet 5.5 9f3fb89e30 Pad: keep typing in order and acknowledge it once, with accents through one KDE Connect queue
Review found accent-only input repeated forever (sent was false), text split
across two transports out of order, accents stuck until the next keypress,
duplicated on retry, and racy. A batch with an accent now sends its whole
keyboard through KDE Connect; later keys follow while any are queued; the
server owns one ordered, bounded queue drained by a single thread; sent means
the whole batch is taken.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 20:52:41 +10:00
saphidandClaude Sonnet 5.5 dc7854d10b Touch: a bare release no longer clears the stale-focus notice before the page has seen it
The page leaves the panel off releases, so a release says nothing about focus.
Found in review of the stale-latch fix.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 20:52:41 +10:00
Alex Southwell 28073e212a Merge pull request #65 from saphid/fix/server-stdin-abort
A stop signal no longer crashes the server, and the app restarts it by itself
2026-09-30 22:58:55 +10:00
saphidandClaude Opus 5.5 6abc765e22 App: Try Again also works when the server is up but its page failed to load
The button was accepted only while no server was known. If the server answered
and the page then failed to load, the error page showed with the server still
known, and the button did nothing. It's now accepted from the error page itself
(the window's only data: page).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 22:51:24 +10:00
saphid f73efe414c Merge main into fix/server-stdin-abort 2026-09-30 22:39:14 +10:00
Alex Southwell 704e5d7780 Merge pull request #59 from saphid/feat/contact-email-opt-in
Optional contact email with separate update and follow-up consent
2026-09-30 22:09:42 +10:00
Alex Southwell edbe8d4109 Merge pull request #63 from saphid/screenshot-copy
Screenshots: Copy, right-click menu, and new shots appear on their own
2026-09-30 20:43:22 +10:00
saphidandClaude Opus 5.5 f2c8466ba9 Screenshots: Refresh retries every failed preview, even if a background check lands meanwhile
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 20:32:28 +10:00
saphidandClaude Opus 5.5 6cf01729e2 Screenshots: fixes from review
- A right-click menu open when the headset changes closes, so it can't act on the other headset's shot.
- A preview being retried by Refresh is no longer dropped when a background check lands first.
- A late failure from a headset switched away from no longer drops the new headset's preview.
- Tab and Escape close the menu and give focus back to where it was.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 20:26:48 +10:00
saphidandClaude Opus 5.5 47a29afb4c Screenshots: recheck fixes
- In Control, a right-click on the viewer goes to the Frame only; the copy menu stays out of the way.
- Thumbnails no longer hold up the next check: a save shows as saved straight away, and a
  new shot appears while older previews are still loading.
- Refresh (or a save) during a background check reads again after it, so the answer is fresh.
- A preview that failed is retried on Refresh, not by every background check.
- Copy reports a failure if the app refuses the image, and if the browser can't copy text.
- Windows: Show in File Explorer works when the path has spaces.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 20:19:03 +10:00
saphidandClaude Opus 5.5 a0f810c018 App: restart the server by itself when it stops, and a Try Again button on the error page
A server that had been up for a minute starts again without asking. One that
stops sooner shows the error page, now headed "Frame Control stopped", with a
Try Again button (the menu item was the only way, and hard to find).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 20:17:24 +10:00
saphidandClaude Opus 5.5 2ca0e6924a Contact email tests: pin the in-gap save and same-second report timing
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 10:38:54 +10:00
saphidandClaude Opus 5.5 cf2db32721 Contact email: don't strand a change saved as a send finishes; time reports exactly
Third review follow-ups:
- A sender that found nothing waiting checks again after letting go of the
  send lock, so a change saved in that moment is sent, not left for a retrier.
- A report is compared with a removal using its full-precision start time, so
  a report sent after the address was removed is logged as sent.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 10:35:25 +10:00
saphidandClaude Opus 5.5 3f0f09b138 Contact email: don't block Save on a slow send; redact reports still in flight
Second review follow-ups:
- Saving returns once the choice is stored; a send already under way picks up
  the newest change, or the background retry is woken.
- A problem report still being sent when its address is removed is logged as
  <removed>, checked under the same lock the removal holds.
- The prompt re-checks the privacy notice after fetching its state.
- docs/privacy.md: offline contact changes are sent later by themselves; the
  prompt never follows straight after the privacy notice.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 10:29:17 +10:00
saphidandClaude Opus 5.5 b8ed53f2ff Contact email: newest choice wins by rev, removal wipes the local log
Review follow-ups:
- Each contact_consent event carries a rev that goes up with every change,
  sends are serialized, and `contacts` picks every field from the highest
  rev per copy, so a withdrawal can't lose to an earlier event sent in the
  same second or with a skewed clock.
- Removing the address also replaces it with <removed> in the local
  sent log (earlier contact events and problem reports).
- The prompt is rechecked when the Frame connects, not only at page load.
- No thanks hides the bar only once the dismissal is saved.
- docs/privacy.md: say that the analytics switches don't block a report or
  contact change the person sends deliberately.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 10:22:45 +10:00
saphidandClaude Opus 5.5 19a0d0af18 Ask for an optional contact email, with separate update and follow-up consent
Problem reports arrive with no way to reply. People can now leave an email
address with two separate opt-ins: occasional update notices, and follow-up
questions from the maintainer.

- ui/frame_contact.py keeps the address and choices locally and sends each
  change privately to PostHog as a contact_consent event under its own random
  contact id; removing the address sends a withdrawal without it. Changes made
  offline wait and are retried.
- A one-time, dismissible prompt appears after the Frame first connects; No
  thanks and showing it once are both remembered.
- Privacy & updates gains a Contact email section to add, change or remove it.
- The report form's contact field now goes with a report only when "may
  contact me with follow-up questions" is ticked (contact_followup).
- frame_report.py contacts [updates|followup] lists who agreed to what,
  using the newest event per copy.
- docs/privacy.md says what is collected, why, where and how to remove it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 10:15:53 +10:00
saphidandClaude Opus 5.5 1a5f089e57 Server: a stop signal no longer crashes it (SIGABRT) while the app holds stdin
The --exit-on-eof watcher read stdin with a buffered read, which holds stdin's
lock. When SIGTERM stopped the server first, Python aborted at exit trying to
take that lock back, and the app showed "The server stopped unexpectedly
(SIGABRT)". It now uses os.read. The startup line is printed inside the try,
so a signal that arrives while it's printed still runs the cleanup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 20:12:47 +10:00
saphidandClaude Sonnet 5.5 5a6b925448 Trackpad and key row go through gamescope's input; KDE Connect only for accents and emoji
The pad reaches whichever panel has focus (Steam's menus too), needs nothing installed, and no longer clamps to 1280x720. Characters that aren't on a US keyboard start KDE Connect the first time they're typed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:03:37 +10:00
saphidandClaude Opus 5.5 83d74548cd Screenshots: Copy button, right-click menu, and new shots appear on their own
Each screenshot card and the viewer get a Copy button that puts the image on
the clipboard (natively in the desktop app, as PNG in a browser). Right-click
a screenshot to open, copy, save, show it in Finder, or copy its path or name;
right-click the viewer to copy or save. The shelf re-lists the Frame's
screenshots every 8 s while the window is visible and connected, redraws only
when something changed, and keeps thumbnails it already has. Switching
headsets clears the list and ignores answers still on their way.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 15:08:36 +10:00
saphidandClaude Sonnet 5.5 60ade8c2e8 Merge origin/main into live-touch
Kept both route tables: main's media, agent, assistant and Mac view routes
plus /api/touch.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 11:04:47 +10:00
saphidandClaude Opus 5.5 aa1ac7368c frame_touch: any event that goes through clears the stale flag
A trackpad move names no panel, so waiting for an aimed event could leave
the page re-reading panels for the rest of the session; a release still
aimed at the old panel doesn't count.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 10:13:09 +10:00
saphidandClaude Opus 5.5 46b6966cf7 frame_touch: build the socket path on the Frame, so Windows can import it for tests
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:41:42 +10:00
saphidandClaude Opus 5.5 be2e770e66 Control: stale clears, pauses reconverge, pastes split per request
- The Frame says it has caught up as soon as an aimed event lands after a
  stale one, so the page stops re-reading the panels.
- After a device pause it lets go of everything it holds (releases that
  arrived while paused were dropped), and waits for the device once per
  batch, not once per event.
- The quick focus check no longer freshens the panel geometry's age.
- Each request carries at most about 100 characters of text.
- Turning Control off while it connects doesn't report an error.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:40:00 +10:00
saphidandClaude Opus 5.5 597f98af6d Control: fixes from the SWE-2 Max review
- The Frame side tracks keys as well as buttons and lets go of both when the
  session ends.
- A stale tap tells the page, which re-reads the panels at once.
- A paused input device waits instead of ending the session; only a
  disconnect does. An OS error on one event skips it.
- Presses check focus with two property reads and do the full lookup only
  when it changed.
- Writes to an agent's stdin are serialized, so two devices sending at once
  can't tear a line (the keyboard agent too).
- Connecting gives up with a message after 15 s instead of hanging on
  "Connecting…"; text goes in 100-character pieces so releases don't wait
  behind a long paste; a cancelled mouse gesture releases what's held.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:28:28 +10:00
saphidandClaude Opus 5.5 ac07efa8f1 Control: close the targeting gaps from the second review
- The focused panel's display comes from GAMESCOPE_FOCUS_DISPLAY (gamescope
  packs ":1" into the first value), so a window id repeated across :0 and :1
  can't be mistaken; the pid is only the fallback.
- Presses, keys, text and scrolls read focus afresh on the Frame; only moves
  use a reading up to a second old.
- A gesture remembers the panel it started on and does nothing more if that
  stops being the one in use; a press with no panel to aim at isn't sent.
- Opening a screenshot clears the panel Control would act on; switching to
  another app releases held keys and buttons.
- Trimming keeps a click with its position; the error backoff holds for new
  input too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 08:52:00 +10:00
saphidandClaude Opus 5.5 0f4f0b863f Control: fixes from review
- Keys held on the Frame are released with buttons when Control stops or
  the view loses focus; keys for the Frame no longer trigger Frame Control's
  own shortcuts.
- Taps only act when the picture on screen is the panel in use; positions,
  presses, keys, text and scrolls name their panel (display and window: ids
  repeat across :0 and :1, told apart by pid), and the Frame drops them if
  focus has moved on. Releases always go.
- While connecting, a tap keeps its position; on an error only releases wait
  and retries back off; trimming a long queue never drops a release.
- Lifting one of two scrolling fingers ends the scroll; a cancelled touch
  isn't a tap; clicks and holds on the bars around the picture do nothing.
- A capture loop from before a Live restart can't stop the new video.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 08:40:46 +10:00
saphidandClaude Opus 5.5 e7c81733b1 Live view: a Desktop view that stays still, and Control to tap on the Frame
The live view gets a second source and a way to use the Frame from it:

- Desktop: the app panel in use in the headset, streamed from its own window
  (x11grab of gamescope's redirected window), so it doesn't move as the
  wearer looks around. A picker shows any other panel, view only.
- Control: on the Desktop view a tap or click lands exactly where you put it;
  drag is a mouse drag, press and hold right-clicks, two fingers scroll, and
  on a computer the mouse, wheel and keyboard work directly. On the headset
  view the view is a trackpad. A text field and key row type from a phone.

Input goes through gamescope's own EIS socket (the way Steam feeds Remote
Play input) with the libei already on the image: ui/frame_touch.py, over
the same long-lived ssh machinery as the keyboard agent, nothing to
install. It reaches the panel that has focus on either X display, which
the KDE Connect route can't. Verified on the Frame and from the iPhone app
in the Simulator.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 08:23:55 +10:00
16 changed files with 1490 additions and 75 deletions

No files matched your search

+1 -1
View File
@@ -73,7 +73,7 @@ Drag files onto the window to send them. Drop a game's .zip, folder or .exe to a
Browse the shots you take in the headset and save them to your Pictures folder.
**⌨️ Keyboard and trackpad**<br>
Type and point in the Frame's apps from your computer or phone, through KDE Connect, which Frame Control brings along and sets up on the Frame. Nothing else to install, anywhere.
Type and point in the headset from your computer or phone, through Valve's own input path, with nothing to install. Accents and emoji use KDE Connect, which Frame Control brings along and sets up the first time you type one.
</td>
</tr>
+28 -6
View File
@@ -1,7 +1,7 @@
// Frame Control as a desktop app (macOS, Windows, Linux): starts ui/server.py on
// a free loopback port and shows it in a native window. The server does all the
// work over the `frame` SSH alias; this file only hosts it.
const { app, BrowserWindow, Menu, Notification, clipboard, dialog, ipcMain, shell } = require("electron");
const { app, BrowserWindow, Menu, Notification, clipboard, dialog, ipcMain, nativeImage, shell } = require("electron");
const { execFile, spawn } = require("child_process");
const { promisify } = require("util");
const fs = require("fs");
@@ -149,7 +149,7 @@ async function startServer() {
const target = `http://127.0.0.1:${port}/`;
for (let i = 0; i < 100; i++) {
if (exited !== null) throw new Error(`The server exited (${exited}). See ${LOG}.`);
if (await ping(target)) { url = target; return; }
if (await ping(target)) { url = target; serverStarted = Date.now(); return; }
await new Promise((r) => setTimeout(r, 100));
}
if (server === child) server = null;
@@ -175,18 +175,27 @@ function stopServer() {
if (server) endServer(server);
}
function errorPage(message) {
function errorPage(message, title = "Frame Control couldn't start") {
const esc = (s) => s.replace(/[&<>]/g, (c) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;" }[c]));
const html = `<!doctype html><meta charset="utf-8"><body style="margin:0;height:100vh;display:grid;
place-items:center;background:${BG};color:#e6edf3;font:14px -apple-system,sans-serif">
<div style="max-width:560px;padding:32px;line-height:1.5"><h2>Frame Control couldn't start</h2>
<p>${esc(message)}</p><p style="color:#8b98a8">Fix it, then choose Frame → Restart Server.</p></div>`;
<div style="max-width:560px;padding:32px;line-height:1.5"><h2>${esc(title)}</h2>
<p>${esc(message)}</p>
<p><button onclick="this.disabled = true; frameApp.restartServer()" style="font:inherit;padding:6px 16px;
border-radius:6px;border:1px solid #30363d;background:#21262d;color:inherit;cursor:pointer">Try Again</button></p>
<p style="color:#8b98a8">Frame → Restart Server does the same.</p></div>`;
return "data:text/html;charset=utf-8," + encodeURIComponent(html);
}
// A server that had been running starts again by itself (something stopped it: a
// signal, a crash). One that stops again within a minute shows the error instead,
// so a server that can't stay up doesn't restart forever.
let serverStarted = 0;
function serverDied(why) {
url = null;
if (win) win.loadURL(errorPage(`The server stopped unexpectedly (${why}). See ${LOG}.`));
if (!win) return;
if (Date.now() - serverStarted > 60000) restartServer();
else win.loadURL(errorPage(`Its server stopped unexpectedly (${why}). See ${LOG}.`, "Frame Control stopped"));
}
// Restarts that overlap share one: two could each start a server, and the one
@@ -263,7 +272,20 @@ function fromUi(e) {
} catch { return false; }
}
// The error page's Try Again button. The error page is the only data: page the window
// shows (`url` can still be set then: the server answered but the page failed to load).
ipcMain.handle("server:restart", (e) => {
if (win && e.sender === win.webContents && e.senderFrame && e.senderFrame.url.startsWith("data:")) restartServer();
});
ipcMain.handle("clipboard:read", (e) => fromUi(e) ? clipboard.readText() : "");
// A PNG or JPEG (a screenshot) onto the clipboard as an image.
ipcMain.handle("clipboard:writeImage", (e, bytes) => {
if (!fromUi(e) || !(bytes instanceof Uint8Array)) return false;
const img = nativeImage.createFromBuffer(Buffer.from(bytes));
if (img.isEmpty()) throw new Error("not an image");
clipboard.writeImage(img);
return true;
});
ipcMain.handle("connection:setup", (e) => { if (fromUi(e)) setUpConnection(); });
ipcMain.on("keys:capture", (e, on) => { if (fromUi(e)) win.webContents.setIgnoreMenuShortcuts(on === true); });
ipcMain.handle("update:get", (e) => fromUi(e) ? publicUpdate() : null);
+4 -2
View File
@@ -2,8 +2,8 @@
// to the Frame needs no pbpaste, PowerShell, xclip or wl-clipboard. Also tells
// the page where a dropped file or folder lives, so a folder can be sideloaded
// as a title without zipping it (the local server reads it from there).
// It can open Set Up Connection when the headset can't be reached, and keeps the
// Frame menu's list of headsets up to date.
// It can put a screenshot on the clipboard as an image, open Set Up Connection when
// the headset can't be reached, and keeps the Frame menu's list of headsets up to date.
// It also receives frame-control://install links (docs/web-install.md): only
// what the link asked for, never an install; the page asks the user first.
// And it passes update state both ways: see app/updater.js.
@@ -12,7 +12,9 @@ const { contextBridge, ipcRenderer, webUtils } = require("electron");
contextBridge.exposeInMainWorld("frameApp", {
notify: (message, request) => ipcRenderer.invoke("comfort:notify", message, request),
readClipboard: () => ipcRenderer.invoke("clipboard:read"),
writeImage: (bytes) => ipcRenderer.invoke("clipboard:writeImage", bytes),
setUpConnection: () => ipcRenderer.invoke("connection:setup"),
restartServer: () => ipcRenderer.invoke("server:restart"), // the "couldn't start" page's Try Again
// The Frame menu's headset switcher: the page tells it the headsets, and hears picks.
devicesChanged: (list) => ipcRenderer.send("devices:changed", list),
onUseDevice: (cb) => {
+54 -3
View File
@@ -103,8 +103,10 @@ privately to Frame Control's PostHog project as a `problem_report` event, the
same way as the analytics above, so only the maintainer can read it and
nothing is published. It works whatever the analytics settings are, because
the person sends it deliberately. The report has the kind, title and text you
wrote, how to reach you if you gave it, a short reference shown after sending,
and the diagnostics below. It has its own random id, so it isn't linked to
wrote, a short reference shown after sending, and the diagnostics below. Your
email address goes with it only if you tick **The maintainer may contact me
with follow-up questions** (the report then carries `contact_followup: true`);
it's filled in from **Contact email** below when you've agreed there. It has its own random id, so it isn't linked to
your analytics events.
With **Include diagnostics** ticked (the default), the report adds:
@@ -128,11 +130,60 @@ The maintainer reads reports on the Frame Control dashboard in PostHog, or
with `python3 ui/frame_report.py inbox [days]`, which uses the same personal
API key as `frame_compat_db.py sync`.
## Contact email (optional)
Frame Control never needs an email address. If you'd like to leave one, there
are two separate choices, both off until you tick them:
| Choice | What it's for |
|---|---|
| **Email me about Frame Control updates** | Occasional notices about new releases and updates |
| **The maintainer may contact me with follow-up questions** | Questions about problem reports you send, mostly |
You're asked once, in a bar at the top of the page, after the Frame has
connected for the first time, and never while or straight after the
first-run privacy notice is showing. **No thanks** hides it for good, and it isn't
shown again even if you ignore it. **Contact email** in **Privacy & updates**
is where you add, change or remove the address and either choice at any time.
**What's sent, and where.** The address and the two choices go privately to
Frame Control's PostHog project, the same place as problem reports, as a
`contact_consent` event with `email`, `updates`, `followup`, `action` (`set`
or `withdraw`) and the common properties above. Only the maintainer can read
that project, and nothing in it is published or shared. It's sent only when
you save, whatever the analytics settings are, because you chose to. It
carries its own random contact id, not the analytics id, so it isn't linked
to your usage events, and a `rev` number that goes up with each change, so
the newest choice always wins. Like everything else sent, it's listed under
**Show what's been sent**. On this computer the address and choices are kept in
`contact/contact.json` in Frame Control's data folder. An address is only
kept with at least one choice ticked.
**Removing it.** **Remove my email** (or clearing the address and saving)
deletes it from this computer, including from the **Show what's been sent**
log (in earlier contact events and problem reports), and sends a `withdraw`
event with no address in it. The maintainer's list only uses the newest event from each copy, so from
then on the address isn't listed for either choice. Unticking one choice
works the same way for that choice. If you're offline, the change waits on
this computer and is sent when PostHog can be reached. The earlier event
stays in PostHog until its data retention removes it; to have it deleted
sooner, ask the maintainer (for example in a problem report).
Nothing sends email yet: this only records who agreed to what. The
maintainer lists the addresses with
`python3 ui/frame_report.py contacts [updates|followup]`, which uses the same
personal API key as `inbox`.
## Turning it all off
Untick the boxes, or set `DO_NOT_TRACK=1` or `FRAME_CONTROL_TELEMETRY=0` in
the environment that starts Frame Control. A copy run from a source checkout
never sends anything unless `FRAME_CONTROL_TELEMETRY=1` is set.
never sends analytics unless `FRAME_CONTROL_TELEMETRY=1` is set.
These switches cover the analytics above. A problem report or a contact email
is sent only because you pressed its Send or Save button, so those still go
when you choose to send them (a contact change saved while offline is sent
by itself once PostHog can be reached); if you don't, nothing is sent.
## Update checks
+7 -4
View File
@@ -123,10 +123,13 @@ two fingers to scroll, two-finger tap to right-click) plus a text field that
types on the Frame. On a computer, clicking the pad passes your mouse and
keyboard through to the Frame until you press Esc (⌘ is sent as Ctrl on a Mac).
It goes through **KDE Connect**, the first-party route (KDE makes the Frame's
desktop): Frame Control's server runs [`ui/frame_input_agent.py`](../ui/frame_input_agent.py)
on the Frame, which talks KDE Connect's own LAN protocol to the Frame's
`kdeconnectd` as if it were a phone. KDE Connect does the typing and clicking.
Since Control (below), the pad goes through **gamescope's own input socket**
([`ui/frame_touch.py`](../ui/frame_touch.py)), so it reaches whichever panel has
focus, Steam's own menus included, and needs nothing installed on the Frame.
Only characters that aren't on a US keyboard (accents, emoji) go through
**KDE Connect** as below; it starts the first time you type one. (Before
Control, everything went through KDE Connect, which is why the notes below
describe it in detail.)
**Verified 2026-09-28** (SteamOS 0.4.1, build 20260925.6191901):
+280
View File
@@ -0,0 +1,280 @@
"""A contact email (ui/frame_contact.py): kept only with a matching choice, sent privately,
withdrawn when removed, never lost offline, and the one-time prompt stays dismissed.
Run: python3 -m unittest discover -s tests
"""
import sandbox # noqa: F401 (first: keeps tests off real data and services)
import sys
import threading
import time
import unittest
from pathlib import Path
from unittest import mock
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT / "ui"))
sys.path.insert(0, str(Path(__file__).resolve().parent))
import frame_compat_db as db # noqa: E402
import frame_contact as fc # noqa: E402
import frame_report as fr # noqa: E402
import frame_telemetry as tm # noqa: E402
from test_telemetry import Base, ReportProblem # noqa: E402
REPORT = {"title": "RDP not working", "message": "It never connects on Windows."}
class Contact(Base):
"""Base's temp telemetry state, ReportProblem's PostHog stand-in, and a temp contact file."""
serve = ReportProblem.serve
def setUp(self):
super().setUp()
self.addCleanup(fc._removed.clear)
for name, value in (("STATE", tm.STATE / "contact"), ("FILE", tm.STATE / "contact" / "contact.json")):
p = mock.patch.object(fc, name, value)
p.start()
self.addCleanup(p.stop)
self.got = self.serve()
def events(self):
return [body["batch"][0] for _, body in self.got]
def offline(self):
return mock.patch.object(tm, "post", side_effect=tm.SendError("couldn't reach PostHog"))
# ---- storage and consent flags
def test_nothing_is_kept_or_sent_until_chosen(self):
s = fc.state()
self.assertEqual((s["email"], s["updates"], s["followup"], s["waiting"]), ("", False, False, False))
self.assertFalse(fc.FILE.exists())
self.assertEqual(self.got, [])
def test_an_address_needs_a_choice_and_a_real_address(self):
with self.assertRaisesRegex(ValueError, "tick"):
fc.save({"email": "me@example.com"})
with self.assertRaisesRegex(ValueError, "email address"):
fc.save({"email": "not an address", "updates": True})
self.assertEqual(fc.load()["email"], "")
self.assertEqual(self.got, [])
def test_each_choice_is_sent_privately_on_its_own(self):
fc.save({"email": " me@example.com ", "updates": True})
fc.save({"email": "me@example.com", "updates": False, "followup": True})
first, second = self.events()
self.assertEqual(first["event"], "contact_consent")
self.assertEqual({k: first["properties"][k] for k in ("email", "updates", "followup", "action")},
{"email": "me@example.com", "updates": True, "followup": False, "action": "set"})
self.assertEqual((second["properties"]["updates"], second["properties"]["followup"]), (False, True))
self.assertEqual(first["distinct_id"], second["distinct_id"]) # one contact id, newest wins
self.assertNotEqual(first["distinct_id"], tm.settings()["id"]) # not the analytics id
self.assertEqual((first["properties"]["$process_person_profile"], first["properties"]["$geoip_disable"]),
(False, True))
self.assertEqual([e["event"] for e in tm._read_lines(tm.SENT)], ["contact_consent"] * 2)
def test_sent_whatever_the_analytics_settings(self):
tm.update_settings({"usage": False})
fc.save({"email": "me@example.com", "followup": True})
self.assertEqual(len(self.got), 1)
def test_saving_the_same_choice_again_sends_nothing(self):
fc.save({"email": "me@example.com", "updates": True})
fc.save({"email": "me@example.com", "updates": True})
self.assertEqual(len(self.got), 1)
# ---- withdrawal
def test_removing_the_address_sends_a_withdrawal_without_it(self):
fc.save({"email": "me@example.com", "updates": True, "followup": True})
s = fc.save({"email": "", "updates": True, "followup": True})
self.assertEqual((s["email"], s["updates"], s["followup"]), ("", False, False))
withdrawal = self.events()[-1]["properties"]
self.assertEqual((withdrawal["action"], withdrawal["email"], withdrawal["updates"], withdrawal["followup"]),
("withdraw", "", False, False))
self.assertNotIn("me@example.com", fc.FILE.read_text())
def test_an_address_still_waiting_is_withdrawn_too(self):
with self.offline():
fc.save({"email": "me@example.com", "updates": True}) # may already be on its way
with mock.patch.object(tm, "post") as post:
fc.save({"email": ""})
self.assertEqual([c.args[0][0]["properties"]["action"] for c in post.call_args_list], ["withdraw"])
self.assertFalse(fc.state()["waiting"])
def test_offline_the_newest_choice_waits_and_a_withdrawal_is_never_lost(self):
fc.save({"email": "me@example.com", "updates": True})
with self.offline():
s = fc.save({"email": ""})
self.assertTrue(s["waiting"])
self.assertFalse(fc._send_pending())
self.assertEqual(fc.load()["pending"]["properties"]["action"], "withdraw")
self.assertTrue(fc._send_pending())
self.assertFalse(fc.state()["waiting"])
self.assertEqual([e["properties"]["action"] for e in self.events()], ["set", "withdraw"])
def test_removing_the_address_wipes_it_from_the_sent_log_too(self):
fc.save({"email": "me@example.com", "followup": True})
fr.send({**REPORT, "contact": "me@example.com", "contactFollowup": True})
self.assertIn("me@example.com", tm.SENT.read_text())
fc.save({"email": ""})
self.assertNotIn("me@example.com", tm.SENT.read_text())
self.assertEqual([e["properties"].get("action") for e in tm._read_lines(tm.SENT)
if e["event"] == "contact_consent"], ["set", "withdraw"])
def test_each_change_has_a_higher_rev_so_the_newest_wins_whatever_the_clock(self):
fc.save({"email": "me@example.com", "updates": True})
fc.save({"email": "new@example.com", "updates": True})
fc.save({"email": ""})
self.assertEqual([e["properties"]["rev"] for e in self.events()], [1, 2, 3])
def test_a_withdrawal_during_a_send_goes_after_it(self):
started, release, order = threading.Event(), threading.Event(), []
real = tm.post
def slow(batch, timeout=20):
order.append(batch[0]["properties"]["action"])
if len(order) == 1:
started.set()
release.wait(5)
real(batch, timeout)
with mock.patch.object(tm, "post", side_effect=slow):
t = threading.Thread(target=fc.save, args=({"email": "me@example.com", "updates": True},))
t.start()
self.assertTrue(started.wait(5))
w = threading.Thread(target=fc.save, args=({"email": ""},))
w.start()
for _ in range(500): # the withdrawal is saved while the first send is still out
if fc.load()["rev"] == 2:
break
time.sleep(0.01)
self.assertEqual(fc.load()["pending"]["properties"]["action"], "withdraw")
release.set()
t.join(5)
w.join(5)
self.assertEqual(order, ["set", "withdraw"])
self.assertEqual([e["properties"]["action"] for e in self.events()], ["set", "withdraw"])
self.assertFalse(fc.state()["waiting"])
self.assertNotIn("me@example.com", tm.SENT.read_text())
def test_a_report_still_sending_when_its_address_is_removed_is_logged_without_it(self):
fc.save({"email": "me@example.com", "followup": True})
real = tm.post
def remove_meanwhile(batch, timeout=20):
real(batch, timeout)
fc.save({"email": ""}) # removed while the report is on its way, before it's logged
with mock.patch.object(tm, "post", side_effect=remove_meanwhile):
fr.send({**REPORT, "contact": "me@example.com", "contactFollowup": True})
self.assertNotIn("me@example.com", tm.SENT.read_text())
fr.send({**REPORT, "contact": "me@example.com", "contactFollowup": True})
self.assertIn("me@example.com", tm.SENT.read_text()) # sent again after removal: logged as sent
def test_only_reports_started_before_the_removal_are_redacted_even_within_a_second(self):
fc._removed["me@example.com"] = 1790000000.3
event = lambda: {"timestamp": "2026-09-21T12:53:20Z", "properties": {"contact": "me@example.com"}}
before, after = event(), event() # the same whole second as the removal
fc.redact_removed(before, 1790000000.1)
fc.redact_removed(after, 1790000000.6)
self.assertEqual((before["properties"]["contact"], after["properties"]["contact"]),
("<removed>", "me@example.com"))
def test_saving_during_a_slow_send_returns_at_once(self):
busy = fc._send_lock
busy.acquire()
try:
s = fc.save({"email": "me@example.com", "updates": True})
finally:
busy.release()
self.assertTrue(s["waiting"]) # left for the send under way (or the retry) to take
self.assertEqual(self.got, [])
self.assertTrue(fc._send_pending())
self.assertEqual(len(self.got), 1)
def test_a_change_saved_as_a_send_finishes_is_not_left_behind(self):
real = fc._send_lock
class Lock: # a Save lands after the sender found nothing waiting, before it lets go
saved = False
def acquire(self, blocking=True):
return real.acquire(blocking)
def release(self):
if not Lock.saved:
Lock.saved = True
s = threading.Thread(target=fc.save, args=({"email": "me@example.com", "updates": True},))
s.start()
s.join(5)
assert not s.is_alive() # the change is saved while the sender still holds the lock
real.release()
with mock.patch.object(fc, "_send_lock", Lock()):
self.assertTrue(fc._send_pending())
self.assertEqual([e["properties"]["email"] for e in self.events()], ["me@example.com"])
self.assertFalse(fc.state()["waiting"])
# ---- the one-time prompt
def test_the_prompt_waits_for_a_working_setup_then_stays_dismissed(self):
self.assertFalse(fc.state()["showPrompt"]) # a new install: the Frame hasn't connected yet
tm.frame_seen("20260901.1", "3.8")
self.assertTrue(fc.state()["showPrompt"])
fc.prompt({"prompt": "dismissed"})
fc.prompt({"prompt": "shown"}) # a later session can't bring it back
self.assertEqual(fc.load()["prompt"], "dismissed")
self.assertFalse(fc.state()["showPrompt"])
self.assertEqual(self.got, []) # No thanks sends nothing
with self.assertRaises(ValueError):
fc.prompt({"prompt": "reset"})
def test_the_prompt_is_shown_once_and_saving_answers_it(self):
tm.frame_seen("20260901.1", "3.8")
fc.prompt({"prompt": "shown"})
self.assertFalse(fc.state()["showPrompt"])
fc.save({"email": "me@example.com", "followup": True, "fromPrompt": True})
self.assertEqual(fc.load()["prompt"], "answered")
# ---- reports and the maintainer's list
def test_a_report_carries_the_address_only_with_follow_up_consent(self):
fr.send({**REPORT, "contact": "me@example.com"})
fr.send({**REPORT, "contact": "me@example.com", "contactFollowup": True})
without, with_ = (e["properties"] for e in self.events())
self.assertEqual((without["contact"], without["contact_followup"]), ("", False))
self.assertEqual((with_["contact"], with_["contact_followup"]), ("me@example.com", True))
with self.assertRaisesRegex(ValueError, "email address"):
fr.send({**REPORT, "contact": "discord:me", "contactFollowup": True})
def test_contacts_lists_the_newest_choice_per_copy_by_consent(self):
rows = [["a", "both@example.com", True, "true", "2026-09-01T10:00:00Z"],
["b", "news@example.com", "true", False, "2026-09-02T10:00:00Z"],
["c", "", False, False, "2026-09-03T10:00:00Z"], # withdrawn
["d", "not-an-address", True, True, "2026-09-03T10:00:00Z"], ["short"]]
with mock.patch.object(db, "_posthog_query", return_value={"results": rows}) as q:
found = fr.contacts()
self.assertIn("argMax(properties.email, tuple(ifNull(toInt(properties.rev), 0), timestamp))",
q.call_args.args[0])
self.assertEqual(found, {"updates": [("both@example.com", "2026-09-01"), ("news@example.com", "2026-09-02")],
"followup": [("both@example.com", "2026-09-01")]})
with mock.patch.object(fr, "contacts", return_value=found), \
mock.patch.object(sys, "argv", ["frame_report.py", "contacts", "followup"]), \
mock.patch("builtins.print") as out:
fr.main()
printed = " ".join(str(c.args[0]) for c in out.call_args_list if c.args)
self.assertIn("both@example.com", printed)
self.assertNotIn("news@example.com", printed)
def test_the_page_can_reach_it(self):
import server
self.assertIs(server.POST["/api/contact"], fc.save)
self.assertIs(server.POST["/api/contact/prompt"], fc.prompt)
# Run these once, in test_telemetry, not again through the import above.
del Base, ReportProblem
if __name__ == "__main__":
unittest.main()
+191
View File
@@ -630,3 +630,194 @@ class PageQueue(unittest.TestCase):
if __name__ == "__main__":
unittest.main()
class PadEvents(unittest.TestCase):
"""The trackpad and key row go through gamescope; accents and emoji go the KDE Connect way."""
@classmethod
def setUpClass(cls):
import server
cls.pad = staticmethod(server.pad_events)
def test_moves_clicks_and_scroll(self):
out, extra = self.pad([{"dx": 3, "dy": -2}, {"singleclick": True}, {"singlehold": True},
{"singlerelease": True}, {"scroll": True, "dy": 1}, {"rightclick": True}])
self.assertEqual(out[0], {"dx": 3, "dy": -2})
self.assertEqual(out[1:3], [{"button": "left", "down": True}, {"button": "left", "down": False}])
self.assertEqual(out[3:5], [{"button": "left", "down": True}, {"button": "left", "down": False}])
self.assertEqual(out[5], {"scroll": [0, -15]}) # up = content follows the finger
self.assertEqual(out[6:], [{"button": "right", "down": True}, {"button": "right", "down": False}])
self.assertEqual(extra, [])
def test_keys_and_modifiers(self):
out, _ = self.pad([{"specialKey": 12}, {"specialKey": 1, "ctrl": True}, {"key": "c", "ctrl": True}])
self.assertEqual(out[:2], [{"key": 28, "down": True}, {"key": 28, "down": False}])
self.assertEqual([e["key"] for e in out[2:6]], [29, 14, 14, 29]) # Ctrl+Backspace
self.assertEqual([(e["key"], e["down"]) for e in out[6:]],
[(29, True), (46, True), (46, False), (29, False)]) # Ctrl+C
def test_text_stays_ascii_through_gamescope(self):
out, slow = self.pad([{"key": "Hi "}, {"key": "there"}])
self.assertEqual(out, [{"text": "Hi there"}])
self.assertEqual(slow, [])
def test_keyboard_goes_to_kde_whole_and_in_order(self):
events = [{"key": "a"}, {"key": "é"}, {"specialKey": 12}, {"key": "c", "ctrl": True}, {"dx": 2, "dy": 1}]
out, slow = self.pad(events, kde=True)
self.assertEqual(out, [{"dx": 2, "dy": 1}]) # the pointer still goes through gamescope
self.assertEqual(slow, events[:4]) # the keyboard as sent, in order
def test_shifted_key_with_modifier(self):
out, _ = self.pad([{"key": "A", "ctrl": True}])
self.assertEqual([e["key"] for e in out], [29, 42, 30, 30, 42, 29])
class PadDelivery(unittest.TestCase):
"""remote_input's promises: "sent" means the whole batch is taken; accents go once, in order."""
class Agent:
def __init__(self, sent=True, state="ready"):
self.sent, self.state, self.got = sent, state, []
def send(self, events):
if events and self.sent:
self.got.append(events)
return {"state": self.state, "sent": self.sent and bool(events)}
def setUp(self):
import server
self.s = server
self.saved = (server._touch, server._input, server.KDE_WAIT)
server._touch, server._input = self.Agent(), self.Agent()
self.reset()
def tearDown(self):
self.s._touch, self.s._input, self.s.KDE_WAIT = self.saved
self.settle()
self.reset()
def reset(self):
self.s._kde_queue.clear()
self.s._kde_until = self.s._typed_until = 0.0
self.s._kde_worker[0] = None
def settle(self):
worker = self.s._kde_worker[0]
if worker:
worker.join(5)
def typed(self):
return [e for batch in self.s._input.got for e in batch]
def test_accent_only_batch_is_acknowledged_and_typed_once(self):
status = self.s.remote_input({"events": [{"key": "é"}]})
self.assertTrue(status["sent"]) # so the page doesn't send it again
self.settle()
self.assertEqual(self.typed(), [{"key": "é"}])
def test_mixed_text_stays_in_one_ordered_stream(self):
self.s.remote_input({"events": [{"key": "aéb"}, {"specialKey": 12}]})
self.settle()
self.assertEqual(self.typed(), [{"key": "aéb"}, {"specialKey": 12}])
self.assertEqual(self.s._touch.got, []) # none of it split off through gamescope
def test_batch_gamescope_didnt_take_is_not_queued(self):
self.s._touch = self.Agent(sent=False, state="starting")
status = self.s.remote_input({"events": [{"dx": 1, "dy": 1}, {"key": "é"}]})
self.assertFalse(status["sent"]) # the page tries the whole batch again...
self.assertEqual(self.s._kde_queue, []) # ...so the accent isn't queued twice
def test_waits_for_kde_connect_then_sends_without_more_input(self):
self.s._input = self.Agent(sent=False, state="starting")
self.s.remote_input({"events": [{"key": "é"}]})
time.sleep(0.2)
self.assertEqual(self.s._kde_queue, [{"key": "é"}])
self.s._input.sent = True
self.settle()
self.assertEqual(self.typed(), [{"key": "é"}])
self.assertEqual(self.s._kde_queue, [])
def test_later_ascii_waits_behind_a_pending_accent(self):
self.s._input = self.Agent(sent=False, state="starting")
self.s.remote_input({"events": [{"key": "é"}]})
self.s.remote_input({"events": [{"key": "x"}]})
self.assertEqual(self.s._touch.got, []) # "x" can't overtake the "é"
self.assertEqual(self.s._kde_queue, [{"key": "é"}, {"key": "x"}])
self.s._input.sent = True
self.settle()
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "x"}])
def test_queue_is_bounded_and_dropped_on_error(self):
self.s._input = self.Agent(sent=False, state="error")
self.s.remote_input({"events": [{"key": "é"}] * 150})
self.s.remote_input({"events": [{"key": "é"}] * 150})
self.settle()
self.assertEqual(self.s._kde_queue, [])
self.s._input = self.Agent(sent=False, state="starting")
self.s.KDE_WAIT = 0.3
self.s.remote_input({"events": [{"key": "é"}] * 150})
self.s.remote_input({"events": [{"key": "é"}] * 150})
self.assertLessEqual(len(self.s._kde_queue), self.s.KDE_QUEUE_LIMIT)
self.settle()
self.assertEqual(self.s._kde_queue, [])
def test_ascii_just_after_an_accent_stays_behind_it(self):
self.s.remote_input({"events": [{"key": "é"}]})
self.settle()
self.s.remote_input({"events": [{"key": "x"}]}) # KDE Connect may still be typing the é
self.settle()
self.assertEqual(self.s._touch.got, [])
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "x"}])
def test_accent_just_after_gamescope_text_waits_for_it(self):
self.s.remote_input({"events": [{"key": "x" * 50}]})
self.assertEqual(self.s._touch.got, [[{"text": "x" * 50}]])
start = time.time()
self.s.remote_input({"events": [{"key": "é"}]})
self.settle()
self.assertGreater(time.time() - start, 0.6) # 50 keys at 16 ms, plus the margin
self.assertEqual(self.typed(), [{"key": "é"}])
def test_typing_time_counts_every_key_transition(self):
t = self.s.typing_seconds
self.assertAlmostEqual(t([{"text": "ab"}]), 4 * 0.008)
self.assertAlmostEqual(t([{"text": "A"}]), 4 * 0.008) # shift down, key down, key up, shift up
self.assertAlmostEqual(t([{"text": "x" * 500}]), 8.0) # a long paste
self.assertAlmostEqual(t([{"key": 29, "down": True}, {"dx": 3, "dy": 0}]), 0.008)
def test_the_margin_is_added_once_not_per_request(self):
for _ in range(10):
self.s.remote_input({"events": [{"key": "ab"}]})
# ten requests of 4 transitions each: the work queues up, with no margin added per request
self.assertLess(self.s._typed_until - time.time(), 10 * 4 * 0.008 + 0.05)
def test_trimming_while_sending_doesnt_drop_unsent_events(self):
slow = self.Agent()
gate = threading.Event()
send = slow.send
def held(events):
gate.wait(5)
return send(events)
slow.send = held
self.s._input = slow
self.s.remote_input({"events": [{"key": "é"}] * 150})
time.sleep(0.2) # the worker has taken its 150 and is mid-send
self.s.remote_input({"events": [{"key": "ü"}] * 150})
gate.set()
self.settle()
sent = self.typed()
self.assertEqual(sent[:150], [{"key": "é"}] * 150)
self.assertEqual(sent[150:], [{"key": "ü"}] * 150) # none of the new ones went missing
def test_input_queued_as_the_worker_finishes_is_still_sent(self):
for _ in range(40):
self.reset()
self.s._input.got.clear()
self.s.remote_input({"events": [{"key": "é"}]})
time.sleep(0.002)
self.s.remote_input({"events": [{"key": "ü"}]})
self.settle()
self.assertEqual(self.s._kde_queue, [])
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "ü"}])
+18
View File
@@ -111,6 +111,8 @@ class ServerGuards(unittest.TestCase):
("/api/volume", {"level": 1.5}),
("/api/clipboard", {"text": ""}),
("/api/open", {"what": "anything-else"}),
("/api/open", {"what": "shot", "id": "1/250820/../../.ssh/id_ed25519"}),
("/api/open", {"what": "shot"}),
("/api/shots/save", {"ids": []}),
("/api/shots/save", {"ids": "1/250820/20260925225208_1.jpg"}),
("/api/shots/save", {"ids": [1]}),
@@ -121,6 +123,10 @@ class ServerGuards(unittest.TestCase):
status, payload = self.post(path, body)
self.assertEqual(status, 400, f"{path} {body} -> {payload}")
def test_showing_a_shot_needs_it_saved_here(self):
status, payload = self.post("/api/open", {"what": "shot", "id": "1/250820/19990101000000_1.jpg"})
self.assertEqual(status, 404, payload)
def test_screenshot_ids_checked_before_ssh(self):
for shot in ("../../etc/passwd", "1/250820/x.jpg", "1/2/20260925225208_1.jpg;id", "1/250820/20260925225208_1.gif"):
status, _, _ = self.request("GET", f"/api/shots/image?id={quote(shot)}", headers={"X-Frame-UI": "1"})
@@ -279,6 +285,18 @@ class OneServer(unittest.TestCase):
r = conn.getresponse()
self.assertEqual(r.status, 403, r.read())
@unittest.skipIf(os.name == "nt", "no SIGTERM on Windows")
def test_sigterm_while_the_app_holds_stdin_exits_cleanly(self):
"""The app keeps stdin open; a stop signal used to abort Python (SIGABRT) at exit."""
env = {**os.environ, "FRAME_CONTROL_DATA_DIR": tempfile.mkdtemp(prefix="frame-one-server-"),
"FRAME_ALIAS": "frame-control-test.invalid"}
proc = subprocess.Popen([sys.executable, str(ROOT / "ui" / "server.py"), "--port", "0", "--exit-on-eof"],
env=env, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
self.addCleanup(lambda: (proc.stdin.close(), proc.stdout.close()))
self.assertIn("Frame Control on", proc.stdout.readline())
proc.terminate()
self.assertEqual(proc.wait(30), 0, proc.stdout.read())
class ArtworkSettings(unittest.TestCase):
"""The settings panel's endpoints, with and without the page's X-Frame-UI key."""
+5 -3
View File
@@ -392,13 +392,14 @@ class ReportProblem(Base):
got = self.serve()
tm.update_settings({"usage": False}) # analytics off: a deliberate report still goes
res = fr.send({"kind": "idea", "title": "Live view stops", "message": "It stops after a minute.",
"contact": "me@example.com"})
"contact": "me@example.com", "contactFollowup": True})
path, body = got[0]
event = body["batch"][0]
self.assertEqual((path, body["api_key"], event["event"]), ("/batch/", "phc_test", "problem_report"))
props = event["properties"]
self.assertEqual((props["kind"], props["title"], props["message"], props["contact"], props["report_id"]),
("idea", "Live view stops", "It stops after a minute.", "me@example.com", res["id"]))
self.assertIs(props["contact_followup"], True)
self.assertEqual((props["$process_person_profile"], props["$geoip_disable"]), (False, True))
self.assertNotEqual(event["distinct_id"], tm.settings()["id"]) # not linked to the analytics
self.assertIn(res["id"], res["message"])
@@ -421,8 +422,9 @@ class ReportProblem(Base):
def test_the_inbox_skips_malformed_reports(self):
good = ["2026-09-28T09:50:00Z", "AB12CD34", "bug", "Live view stops", "It stops.", None,
"0.4.0", "macOS", "", ""]
rows = [["2026-09-28T10:00:00Z", "X", "bug", "Hand-made", None, None, None, None, None, None], ["short"], good]
"0.4.0", "macOS", "", "", None]
rows = [["2026-09-28T10:00:00Z", "X", "bug", "Hand-made", None, None, None, None, None, None, None],
["short"], good]
with mock.patch.object(db, "_posthog_query", return_value={"results": rows}), \
mock.patch.object(sys, "argv", ["frame_report.py", "inbox"]), \
mock.patch("builtins.print") as out:
+8
View File
@@ -147,6 +147,14 @@ class Apply(unittest.TestCase):
self.assertFalse(self.t.STALE[0])
self.assertEqual(self.said[-1][1].get("stale"), None)
def test_stale_survives_a_bare_release(self):
gs = FakeGamescope()
panel = self.t.apply(gs, {"button": "left", "down": True, "window": 99, "display": ":1"}, None)
self.assertTrue(self.t.STALE[0])
self.t.apply(gs, {"button": "left", "down": False}, panel) # the page's release names no panel
self.assertTrue(self.t.STALE[0])
self.assertEqual(self.said[-1][1].get("stale"), True)
def test_same_window_id_on_the_other_display_is_another_panel(self):
gs = FakeGamescope()
self.t.apply(gs, {"fx": 0.5, "fy": 0.5, "window": 7, "display": ":0"}, None)
+218
View File
@@ -0,0 +1,218 @@
"""An email address the person chooses to leave, and what it may be used for. Python stdlib only.
Two separate opt-in choices, both off until ticked:
- updates: occasional notices about Frame Control releases and updates
- followup: the maintainer may ask follow-up questions, mainly about problem reports
The address and the choices are kept on this computer (frame_host.data_dir('contact')) and
sent privately to Frame Control's PostHog project as a `contact_consent` event, the same way
as problem reports (frame_report.py), so only the maintainer can read them. Every change
sends a new event under this copy's own random contact id (not the analytics id), numbered
by `rev`, and the highest rev for an id is the one that counts, whatever the clocks say:
removing the address sends a withdrawal with no address in it, and wipes the address from
the local log of what was sent. The maintainer lists who agreed to what with
`python3 ui/frame_report.py contacts`. Nothing here sends email.
A change that can't be sent (offline) waits in the state file and is retried in the
background, so a withdrawal is never lost. The page's one-time prompt is remembered here
too: once it has been shown or dismissed it never comes back.
"""
import json
import os
import re
import threading
import time
import uuid
import frame_host
import frame_telemetry
STATE = frame_host.data_dir('contact')
FILE = STATE / 'contact.json'
EMAIL_MAX = 254
EMAIL_RE = re.compile(r'[^@\s]+@[^@\s]+\.[^@\s.]+')
PROMPTS = ('new', 'shown', 'dismissed', 'answered')
RETRY_EVERY = 600
_lock = threading.RLock()
_send_lock = threading.Lock() # one send at a time, so events reach PostHog in rev order
_removed = {} # address (lower case) -> when it was removed, for reports still being sent then
_wake = threading.Event()
_retrier = None
def _defaults():
return {'id': str(uuid.uuid4()), 'email': '', 'updates': False, 'followup': False,
'prompt': 'new', 'pending': None, 'rev': 0}
def load():
with _lock:
s = _defaults()
try:
with open(FILE) as f:
saved = json.load(f)
if isinstance(saved, dict):
s.update({k: v for k, v in saved.items() if k in s})
except (OSError, ValueError):
pass
return s
def _save(s):
STATE.mkdir(parents=True, exist_ok=True)
tmp = FILE.with_suffix('.tmp')
tmp.write_text(json.dumps(s, indent=1))
os.replace(tmp, FILE)
def valid_email(email):
return len(email) <= EMAIL_MAX and bool(EMAIL_RE.fullmatch(email))
def state():
"""What the page shows. showPrompt: the one-time prompt hasn't been shown or answered yet,
and the Frame has connected at least once (setup worked), so it never greets a new install."""
s = load()
set_up = bool(frame_telemetry.settings().get('frames_seen'))
return {'email': s['email'], 'updates': s['updates'], 'followup': s['followup'],
'waiting': s['pending'] is not None, 'showPrompt': s['prompt'] == 'new' and set_up}
def _event(s):
email = s['email'] if s['updates'] or s['followup'] else ''
return {'event': 'contact_consent', 'distinct_id': s['id'], 'uuid': str(uuid.uuid4()),
'timestamp': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()),
'properties': {**frame_telemetry.common(), 'email': email, 'updates': bool(email and s['updates']),
'followup': bool(email and s['followup']),
'action': 'set' if email else 'withdraw', 'rev': s['rev'], 'level': 'contact'}}
def _send_pending(block=True):
"""Send what's waiting, including changes made while sending. True if nothing is left
waiting. Without block, a send already under way is left to pick up the newest change."""
if not _send_lock.acquire(blocking=block):
return False
try:
while True:
with _lock:
event = load()['pending']
if event is None:
break
try:
frame_telemetry.post([event], timeout=30)
except frame_telemetry.SendError:
return False
_sent(event)
finally:
_send_lock.release()
# A change saved just as this finished found the lock still held and left it to us.
with _lock:
left = load()['pending'] is not None
return _send_pending(block=False) if left else True
def _sent(event):
with _lock:
s = load()
if s['pending'] and s['pending'].get('uuid') == event['uuid']: # not replaced meanwhile
s['pending'] = None
_save(s)
# A withdrawal, or the address still in use: not an old one removed while this was on its way.
if event['properties']['email'] in ('', s['email']):
try:
frame_telemetry.record_sent([event])
except OSError:
pass
def _forget_locally(email):
"""Take a removed address out of the log of what was sent (contact events and reports)."""
with frame_telemetry._lock:
_removed[email.lower()] = time.time()
rows = frame_telemetry._read_lines(frame_telemetry.SENT)
hit = False
for e in rows:
p = e.get('properties') or {}
for k in ('email', 'contact'):
if p.get(k) and str(p[k]).strip().lower() == email.lower():
p[k], hit = '<removed>', True
if hit:
frame_telemetry._write_lines(frame_telemetry.SENT, rows)
def redact_removed(event, started):
"""Before logging a report (started at time.time() `started`) whose address was removed
while it was being sent: take the address out. Call with frame_telemetry._lock held, so a
removal can't slip between this and the log."""
p = event.get('properties') or {}
removed_at = _removed.get(str(p.get('contact') or '').strip().lower())
if removed_at is not None and started <= removed_at:
p['contact'] = '<removed>'
def save(body):
"""Set, change or remove the address and the two choices. An address needs at least one
choice ticked; an empty address (or neither ticked) removes it and withdraws both."""
email = str(body.get('email') or '').strip()
updates, followup = bool(body.get('updates')), bool(body.get('followup'))
if email and not valid_email(email):
raise ValueError("that doesn't look like an email address")
if email and not (updates or followup):
raise ValueError('tick what the address may be used for, or remove it')
if not email:
updates = followup = False
with _lock:
s = load()
old = s['email']
changed = (email, updates, followup) != (s['email'], s['updates'], s['followup'])
s.update(email=email, updates=updates, followup=followup)
if body.get('fromPrompt') or email:
s['prompt'] = 'answered'
if changed:
# Only the newest choice matters, so it replaces anything still waiting. A withdrawal
# is sent even for an address still waiting here: its send may already be under way.
s['rev'] += 1
s['pending'] = _event(s)
_save(s)
if old and old.lower() != email.lower():
try:
_forget_locally(old)
except OSError:
pass
if changed and not _send_pending(block=False):
_wake.set() # offline, or a send under way that will take this change with it
return state()
def prompt(body):
"""The one-time prompt was shown, or dismissed with No thanks. Either way it stays gone."""
action = body.get('prompt')
if action not in ('shown', 'dismissed'):
raise ValueError('unknown prompt action')
with _lock:
s = load()
if s['prompt'] in ('new', 'shown'):
s['prompt'] = action
_save(s)
return state()
def start():
"""Retry a change that couldn't be sent, from now on in the background."""
global _retrier
if _retrier:
return
def loop():
while True:
try:
_send_pending()
except Exception:
pass
_wake.wait(RETRY_EVERY)
_wake.clear()
_retrier = threading.Thread(target=loop, name='contact', daemon=True)
_retrier.start()
+13
View File
@@ -153,6 +153,19 @@ def open_path(path):
stderr=subprocess.DEVNULL, **DETACHED)
def reveal_path(path):
"""Show a file selected in its folder (Linux file managers vary, so there the folder opens)."""
path = Path(path)
if MAC:
cmd = ["open", "-R", str(path)]
elif WINDOWS:
cmd = f'explorer /select,"{path}"' # as one string: Explorer wants the quotes after the comma
else:
return open_path(path.parent)
subprocess.Popen(cmd, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, **DETACHED)
open_url = open_path # the same openers hand URLs to the default browser
+68 -7
View File
@@ -6,6 +6,10 @@ project as a `problem_report` event: only the maintainer can read it, and
nothing is published. It is sent whatever the analytics settings are, because
the person sends it deliberately. Diagnostics are scrubbed first
(frame_telemetry.scrub); the person's own words are sent as written.
An email address goes with a report only when the person ticks "may contact me with
follow-up questions" (contact_followup). Standing choices made in Settings are
frame_contact.py's `contact_consent` events; `contacts` lists them.
"""
import os
import platform
@@ -13,6 +17,7 @@ import sys
import time
import uuid
import frame_contact
import frame_host
import frame_telemetry
@@ -107,19 +112,26 @@ def send(body):
"""Send the report to PostHog. Returns {"id", "message"}; raises ReportError."""
kind = body.get('kind') if body.get('kind') in KINDS else 'bug'
title, text, diag = compose(body)
followup = bool(body.get('contactFollowup'))
contact = str(body.get('contact') or '').strip() if followup else ''
if followup and not frame_contact.valid_email(contact):
raise ValueError('add your email address for follow-up questions, or untick that box')
ref = uuid.uuid4().hex[:8].upper()
props = {**frame_telemetry.common(), 'kind': kind, 'title': title, 'message': text,
'contact': str(body.get('contact') or '').strip()[:120], 'diagnostics': diag,
'contact': contact, 'contact_followup': followup, 'diagnostics': diag,
'report_id': ref, 'steamos': str(frame.get('build') or '')[:120], 'level': 'report'}
# Its own random id: a report can carry contact details, so it isn't linked to this copy's analytics.
event = {'event': 'problem_report', 'distinct_id': str(uuid.uuid4()), 'uuid': str(uuid.uuid4()),
'timestamp': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'properties': props}
started = time.time()
try:
frame_telemetry.post([event], timeout=30)
except frame_telemetry.SendError as e:
raise ReportError(str(e))
try:
frame_telemetry.record_sent([event])
with frame_telemetry._lock: # the lock a removal holds while wiping its address
frame_contact.redact_removed(event, started)
frame_telemetry.record_sent([event])
except OSError:
pass # it was sent; failing to log it here mustn't make the person send it again
return {'id': ref, 'message': f'Sent privately to the Frame Control developer (report {ref}).'}
@@ -135,22 +147,71 @@ def inbox(days=30):
import frame_compat_db
res = frame_compat_db._posthog_query(
"SELECT timestamp, properties.report_id, properties.kind, properties.title, properties.message, "
"properties.contact, properties.app_version, properties.os, properties.steamos, properties.diagnostics "
"properties.contact, properties.app_version, properties.os, properties.steamos, properties.diagnostics, "
"properties.contact_followup "
f"FROM events WHERE event = 'problem_report' AND timestamp > now() - INTERVAL {int(days)} DAY "
"ORDER BY timestamp DESC LIMIT 200")
return res.get('results') or []
def _yes(v):
return v is True or str(v).lower() in ('true', '1')
def contacts():
"""{'updates': [(email, since)], 'followup': [...]}: the addresses whose newest
contact_consent event agrees to each, oldest first. A withdrawal, or a change to another
address, replaces what came before, so withdrawn addresses are never listed. "Newest" is
the highest rev from that copy (then time), so every field comes from the same event
whatever order they arrived in or what the clocks said."""
import frame_compat_db
newest = "tuple(ifNull(toInt(properties.rev), 0), timestamp)"
res = frame_compat_db._posthog_query(
f"SELECT distinct_id, argMax(properties.email, {newest}), argMax(properties.updates, {newest}), "
f"argMax(properties.followup, {newest}), argMax(timestamp, {newest}) FROM events "
"WHERE event = 'contact_consent' GROUP BY distinct_id ORDER BY max(timestamp) LIMIT 100000")
out = {'updates': [], 'followup': []}
for row in res.get('results') or []:
if not isinstance(row, list) or len(row) != 5:
continue
_, email, updates, followup, ts = row
email = str(email or '').strip()
if not frame_contact.valid_email(email):
continue
for kind, agreed in (('updates', updates), ('followup', followup)):
if _yes(agreed):
out[kind].append((email, str(ts or '')[:10]))
return out
USAGE = 'usage: frame_report.py inbox [days] | contacts [updates|followup]'
def main():
cmd, *args = sys.argv[1:] or ['inbox']
if cmd == 'contacts':
kinds = args[:1] or ['updates', 'followup']
if not set(kinds) <= {'updates', 'followup'}:
sys.exit(USAGE)
found = contacts()
for kind in kinds:
print(f"== {'Release and update notices' if kind == 'updates' else 'Follow-up questions'}"
f" ({len(found[kind])})")
for email, since in found[kind]:
print(f" {email} (since {since})")
print()
return
if cmd != 'inbox':
sys.exit('usage: frame_report.py inbox [days]')
sys.exit(USAGE)
for row in inbox(*(args[:1] or [30])):
if not isinstance(row, list) or len(row) != 10:
if not isinstance(row, list) or len(row) != 11:
continue
ts, ref, kind, title, text, contact, version, osname, steamos, diag = (str(v or '') for v in row)
ts, ref, kind, title, text, contact, version, osname, steamos, diag = (str(v or '') for v in row[:10])
# Reports from before contact_followup existed only carried an address given for a reply.
reply = contact and (row[10] is None or _yes(row[10]))
print(f"== {ts[:16].replace('T', ' ')} {ref} [{kind}] {title}")
print(f" {version} on {osname}, SteamOS {steamos or 'unknown'}{', reply to ' + contact if contact else ''}")
print(f" {version} on {osname}, SteamOS {steamos or 'unknown'}"
f"{', may follow up at ' + contact if reply else ''}")
print(' ' + text.replace('\n', '\n '))
if diag:
print(' --- diagnostics\n ' + diag.replace('\n', '\n '))
+3 -1
View File
@@ -369,8 +369,10 @@ def apply(gs, event, panel):
say("ready", focus=panel.get("window"), display=panel.get("display"), stale=True) # the page re-syncs
STALE[0] = True
return panel
if STALE[0] and not stale:
release = event.get("down") is False and ("button" in event or "key" in event)
if STALE[0] and not stale and not (release and "window" not in event):
# Anything that goes through (a trackpad move names no panel) means caught up: stop re-syncing.
# A bare release doesn't: the page leaves the panel off releases, so it says nothing about focus.
STALE[0] = False
say("ready", focus=(panel or {}).get("window"), display=(panel or {}).get("display"))
if "fx" in event and panel and panel.get("window"):
+308 -42
View File
@@ -175,7 +175,7 @@
.seg { display: inline-flex; background: rgba(0,0,0,.3); border-radius: 3px; padding: 2px; }
.seg button { background: transparent; height: 28px; font-size: 12.5px; letter-spacing: .6px; text-transform: uppercase; }
.seg button.on { background: var(--btn-hi); color: var(--bright); }
input[type=text], input[type=search], input[type=url], input[type=password], textarea { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent;
input[type=text], input[type=search], input[type=url], input[type=password], input[type=email], textarea { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent;
border-radius: 3px; padding: 9px 11px; font: inherit; }
textarea { resize: vertical; min-height: 76px; }
input:focus, textarea:focus { outline: none; border-color: var(--blue); background: rgba(0,0,0,.4); }
@@ -258,7 +258,12 @@
.shot-card .row { flex-wrap: nowrap; }
.shot-card .grow { flex: 1; min-width: 0; }
.shot-card .t { color: var(--bright); font-size: 13px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.shot-card .s { color: var(--muted); font-size: 12px; }
.shot-card .s { color: var(--muted); font-size: 12px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.ctx-menu { position: fixed; z-index: 1000; min-width: 200px; padding: 4px; border-radius: 4px; background: #232c38;
box-shadow: 0 10px 28px rgba(0,0,0,.6), 0 0 0 1px rgba(255,255,255,.08); }
.ctx-menu button { display: block; width: 100%; height: 30px; padding: 0 10px; text-align: left; background: none; }
.ctx-menu button:hover, .ctx-menu button:focus-visible { background: var(--blue); color: #fff; outline: none; }
.ctx-menu hr { border: 0; border-top: 1px solid rgba(255,255,255,.1); margin: 4px 2px; }
/* ---- library shelf (portrait capsules, like Steam's library home) ---- */
.shelf { display: grid; grid-template-columns: repeat(auto-fill, minmax(150px, 1fr)); gap: 16px; }
@@ -296,6 +301,11 @@
background: rgba(26,159,255,.12); border-left: 3px solid var(--blue); font-size: 13.5px; line-height: 1.5; }
.notice .grow { flex: 1; min-width: 260px; }
.notice .progress { width: 160px; margin-top: 0; display: block; }
.contact-opts { display: flex; gap: 6px 18px; flex-wrap: wrap; margin-top: 8px; }
.contact-opts label { display: inline-flex; gap: 7px; align-items: center; cursor: pointer; }
.contact-opts input { width: 15px; height: 15px; margin: 0; accent-color: var(--blue); }
#contactNotice input[type=email] { width: min(320px, 100%); margin-top: 8px; padding: 7px 10px; }
#cEmail { max-width: 420px; }
.popt { display: grid; grid-template-columns: auto 1fr; gap: 4px 10px; align-items: start; margin: 0 0 14px; cursor: pointer; }
.popt input { margin: 3px 0 0; width: 16px; height: 16px; accent-color: var(--blue); }
.popt b { font-weight: 600; color: var(--text); }
@@ -674,6 +684,17 @@
<button class="small" id="noticeMore" title="Also share compatibility results and error details (you can turn either off later)">Share more to help fix problems</button>
<button class="action small" id="noticeOk">OK</button>
</div>
<form class="notice" id="contactNotice" hidden>
<div class="grow"><b>Leave an email address?</b> Optional: Frame Control works the same either way, and
you can change or remove it any time in Privacy &amp; updates.
<div><input type="email" id="cpEmail" maxlength="254" placeholder="you@example.com" aria-label="Email address" required></div>
<div class="contact-opts">
<label><input type="checkbox" id="cpUpdates"> Email me about Frame Control updates</label>
<label><input type="checkbox" id="cpFollowup"> The maintainer may contact me with follow-up questions</label></div></div>
<span class="sub" id="cpMsg" role="status"></span>
<button type="button" class="small" id="cpNo">No thanks</button>
<button type="submit" class="action small" id="cpSave">Save</button>
</form>
<div class="banner" id="offline" role="alert" hidden>
<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" aria-hidden="true"><path d="M2 8.5a15 15 0 0 1 20 0M5.5 12a10 10 0 0 1 13 0M9 15.5a5 5 0 0 1 6 0"/><circle cx="12" cy="19" r="1.2" fill="currentColor"/><path d="M3 3l18 18"/></svg>
<div class="grow"><div class="t" id="offMsg">Can't reach the Frame</div>
@@ -701,6 +722,7 @@
<button class="action" id="shotBtn">Capture</button>
<button id="liveBtn" title="Keep updating, as video">Live</button>
<button id="ctrlBtn" title="Control the Frame by tapping or clicking on the view (C)">Control</button>
<button id="copyBtn" disabled title="Copy the image to the clipboard (or right-click it)">Copy</button>
<button id="saveBtn" disabled>Save</button>
</span>
</div>
@@ -854,10 +876,9 @@
<button data-pad-key="12" title="Enter">Enter</button>
</span>
</div>
<div class="hint">Types and points in apps on the Frame, such as Chromium or the desktop's Linux apps; Steam's own VR menus
don't take it. It works through KDE Connect, which comes with Frame Control: the first time, it copies Valve's build
for the Frame (3.6 MB, 18 MB unpacked) into your home folder there and pairs with it. No internet needed, and nothing
else to install. <a href="#" data-about>Licences</a></div>
<div class="hint">Types and points in whichever panel has focus in the headset, through Valve's own input path: nothing to
install and no internet needed. Accents and emoji go through KDE Connect, which comes with Frame Control and starts
the first time you type one (it copies Valve's build for the Frame, 3.6 MB, into your home folder there). <a href="#" data-about>Licences</a></div>
</section>
<section id="shots">
@@ -867,7 +888,7 @@
<button class="action small" id="shotsSaveNew" disabled>Save new to this computer</button>
</div>
<div class="shot-grid" id="shotGrid"><div class="sub">Loading…</div></div>
<div class="hint">Screenshots you take in the headset with Steam's screenshot shortcut. Click one to open it in the viewer; Save copies it to <code>~/Pictures/SteamFrame</code>.</div>
<div class="hint">Screenshots you take in the headset with Steam's screenshot shortcut. New ones appear on their own. Click one to open it in the viewer, Copy puts it on the clipboard, and Save copies it to <code>~/Pictures/SteamFrame</code>. Right-click for more.</div>
</section>
</div>
@@ -1127,6 +1148,17 @@
<span class="sub">Error messages and where in Frame Control they happened, with your home
folder, user name, addresses and keys removed.</span></label>
<div class="hint" id="tStatus"></div>
<h3 style="margin-top:22px">Contact email (optional)</h3>
<form id="contactForm">
<input type="email" id="cEmail" maxlength="254" placeholder="you@example.com" aria-label="Email address">
<div class="contact-opts">
<label><input type="checkbox" id="cUpdates"> Email me about Frame Control updates</label>
<label><input type="checkbox" id="cFollowup"> The maintainer may contact me with follow-up questions</label></div>
<div class="row" style="margin-top:10px"><button type="submit" class="small action" id="cSave">Save</button>
<button type="button" class="small" id="cRemove">Remove my email</button></div>
</form>
<div class="hint" id="cStatus">Sent privately to the Frame Control maintainer, never shared or published.
Updates are occasional release notices; follow-up questions are mainly about problem reports you send.</div>
<details id="tSentBox"><summary>Show what's been sent</summary><div class="sentlog" id="tSent"></div></details>
<div class="row" style="margin-top:14px"><button class="small action" data-report>Report a problem</button>
<button class="small" id="updateCheck" hidden>Check for updates</button>
@@ -1234,7 +1266,9 @@
placeholder="e.g. Installing an APK stops at 'copying to the Frame'"></label>
<label class="field">What happened?<textarea id="bugText" maxlength="5000" required minlength="10"
placeholder="What you did, what happened, and what you expected."></textarea></label>
<label class="field">How can we reach you? (optional, for a reply)<input type="text" id="bugContact" maxlength="120" placeholder="Email, GitHub or Discord name"></label>
<label class="popt"><input type="checkbox" id="bugFollowup"><b>The maintainer may contact me with follow-up questions</b>
<span class="sub">Optional. Your email address goes with this report only when this is ticked.</span></label>
<label class="field">Your email address<input type="email" id="bugContact" maxlength="254" placeholder="you@example.com" disabled></label>
<label class="popt"><input type="checkbox" id="bugDiag" checked><b>Include diagnostics</b>
<span class="sub">Frame Control's version, your OS and the Frame's SteamOS build.</span></label>
<label class="popt"><input type="checkbox" id="bugLogs"><b>Also include recent activity and the server log</b>
@@ -1869,7 +1903,7 @@ async function capture() {
if (source === "panel") $("srcBadge").textContent = `Desktop · ${shotPanel?.name || "panel"}`;
ctrlShow();
$("stamp").hidden = false; $("stamp").textContent = new Date().toLocaleTimeString();
$("saveBtn").disabled = false;
$("saveBtn").disabled = $("copyBtn").disabled = false;
return true;
} catch (e) {
log("Capture failed: " + e.message, "e");
@@ -1951,7 +1985,7 @@ async function startVideo() {
$("viewer").classList.remove("busy");
c.hidden = false; $("viewerEmpty").hidden = true; $("zoombar").hidden = false; $("asleep").hidden = true;
$("srcBadge").hidden = false; $("srcBadge").textContent = `${video.label} · video`;
$("stamp").hidden = false; $("saveBtn").disabled = false;
$("stamp").hidden = false; $("saveBtn").disabled = $("copyBtn").disabled = false;
}
frames++;
const now = performance.now();
@@ -2020,6 +2054,35 @@ $("saveBtn").onclick = () => lastShot ? download(lastShot.blob, lastShot.file) :
if (!b) return toast("Couldn't encode the image", true);
download(b, `frame-${view}-${new Date().toISOString().replace(/[:.]/g, "-")}.png`);
}, "image/png");
const copyViewer = () => act("Copy image", () => copyImage(lastShot ? lastShot.blob
: new Promise((ok, bad) => $("canvas").toBlob(b => b ? ok(b) : bad(new Error("couldn't encode the image")), "image/png"))));
$("copyBtn").onclick = copyViewer;
// Right-click the viewer to copy or save what it shows (in Control, a right-click goes to the Frame).
$("canvas").oncontextmenu = e => {
if (ctrl.on || $("saveBtn").disabled) return;
e.preventDefault();
showMenu(e, [["Copy image", copyViewer], ["Save image", () => $("saveBtn").click()]]);
};
// An image onto the clipboard. The app does it natively (JPEG too); a browser
// takes PNG only, and the blob is handed over as a promise so the click still counts.
async function copyImage(blob) {
if (window.frameApp?.writeImage) {
if (!await window.frameApp.writeImage(new Uint8Array(await (await blob).arrayBuffer()))) throw new Error("the app refused the image");
return { message: "Copied the image" };
}
if (!navigator.clipboard?.write || !window.ClipboardItem) throw new Error("This browser can't copy images here");
await navigator.clipboard.write([new ClipboardItem({ "image/png": Promise.resolve(blob).then(pngBlob) })]);
return { message: "Copied the image" };
}
async function pngBlob(blob) {
if (blob.type === "image/png") return blob;
const bmp = await createImageBitmap(blob);
const c = document.createElement("canvas");
c.width = bmp.width; c.height = bmp.height;
c.getContext("2d").drawImage(bmp, 0, 0);
bmp.close();
return new Promise((ok, bad) => c.toBlob(b => b ? ok(b) : bad(new Error("couldn't encode the image")), "image/png"));
}
function download(blob, name) {
if (savesToDevice()) return act("Save image", () => saveToDevice([{ blob, name }]));
const a = document.createElement("a");
@@ -3497,8 +3560,11 @@ loadReports();
api("/api/host").then(applyHostWording).catch(() => {});
// ---- Steam screenshots from the headset ----
const shots = { list: [], urls: [] };
// thumbs: id -> promise of an object URL, kept across reloads so a refresh
// that finds a new shot fetches only that one's thumbnail.
const shots = { list: [], sig: null, thumbs: new Map(), loading: null, gen: 0, fill: 0 };
const STEAMVR_APPID = "250820";
const SHOTS_POLL_MS = 8000;
function shotApp(appid) {
if (appid === STEAMVR_APPID) return "SteamVR";
const g = state?.games?.find(x => x.appid === appid);
@@ -3510,35 +3576,85 @@ async function shotBlob(id, thumb) {
if (!r.ok) throw new Error((await r.json().catch(() => ({}))).error || `HTTP ${r.status}`);
return r.blob();
}
async function loadShots() {
$("shotsRefresh").disabled = true;
function shotThumb(id) {
const thumbs = shots.thumbs; // this headset's: a late failure must not touch the next one's
if (!thumbs.has(id)) {
const p = shotBlob(id, true).then(b => URL.createObjectURL(b));
p.catch(() => { if (thumbs.get(id) === p) thumbs.delete(id); }); // try again next time
thumbs.set(id, p);
}
return thumbs.get(id);
}
// quiet: a background check. It keeps what's shown if the Frame can't be read,
// and redraws only when the shots (or whether they're saved here) changed.
function loadShots(quiet) {
if (shots.loading) {
// A check already on its way will do for another check; Refresh, or a save
// that just finished, reads again after it so the answer is a fresh one.
const again = () => loadShots();
return quiet === true ? shots.loading : shots.loading.then(again, again);
}
const p = readShots(quiet === true).finally(() => { if (shots.loading === p) shots.loading = null; });
return shots.loading = p;
}
// Forget the shots of a headset we've switched away from, and ignore its answers still on their way.
function resetShots() {
closeMenu(); // its items were about the other headset's shot
shots.gen++; shots.fill++; shots.loading = null; shots.list = []; shots.sig = null;
for (const p of shots.thumbs.values()) p.then(URL.revokeObjectURL, () => {});
shots.thumbs = new Map();
}
async function readShots(quiet) {
const gen = shots.gen;
if (!quiet) $("shotsRefresh").disabled = true;
let got;
try {
shots.list = (await api("/api/shots")).shots;
got = await api("/api/shots");
} catch (e) {
return failed($("shotGrid"), e);
if (!quiet && gen === shots.gen) { shots.sig = null; failed($("shotGrid"), e); }
return;
} finally { $("shotsRefresh").disabled = false; }
shots.urls.forEach(URL.revokeObjectURL); shots.urls = [];
if (gen !== shots.gen) return;
({ shots: shots.list, folder: shots.folder } = got);
const sig = JSON.stringify(shots.list.map(s => [s.id, s.saved, shotApp(s.appid)]));
if (sig !== shots.sig) { shots.sig = sig; drawShots(); }
fillThumbs(quiet); // not waited for: the next check, or a save, needn't sit behind the thumbnails
}
// Thumbnails one at a time over the shared SSH connection. One that failed is
// tried again on Refresh, not by every background check.
async function fillThumbs(quiet) {
const run = ++shots.fill;
// Refresh puts every failed one back in line first, so a background pass that takes over carries them all on.
if (!quiet) for (const img of document.querySelectorAll("#shotGrid img[data-failed]")) delete img.dataset.failed;
for (const img of document.querySelectorAll("#shotGrid img[data-shot]:not([src])" + (quiet ? ":not([data-failed])" : ""))) {
const s = shots.list[+img.dataset.shot];
let url;
try { url = await shotThumb(s.id); } catch (e) { img.alt = "Preview failed"; img.dataset.failed = 1; }
if (run !== shots.fill) return; // a newer pass (after a check, a redraw or another headset) has taken over
if (url) img.src = url;
}
}
// "Sep 28, 10:37 PM": short enough to sit beside the card's buttons (the full date is its tooltip).
function shotTime(t) {
return new Date(t * 1000).toLocaleString([], { month: "short", day: "numeric", hour: "numeric", minute: "2-digit" });
}
function drawShots() {
const ids = new Set(shots.list.map(s => s.id));
for (const [id, p] of shots.thumbs) {
if (!ids.has(id)) { shots.thumbs.delete(id); p.then(URL.revokeObjectURL, () => {}); }
}
const unsaved = shots.list.filter(s => !s.saved).length;
$("shotCount").textContent = shots.list.length ? `${shots.list.length} on the Frame` + (unsaved && !HOST.mobile ? ` · ${unsaved} not on this ${HOST.computer}` : "") : "";
$("shotsSaveNew").disabled = HOST.mobile ? !shots.list.length : !unsaved;
$("shotGrid").innerHTML = shots.list.length ? shots.list.map((s, i) => `<div class="shot-card">
<img class="thumb" data-shot="${i}" alt="Screenshot from ${esc(shotApp(s.appid))}" title="Open in the viewer">
$("shotGrid").innerHTML = shots.list.length ? shots.list.map((s, i) => `<div class="shot-card" data-card="${i}">
<img class="thumb" data-shot="${i}" alt="Screenshot from ${esc(shotApp(s.appid))}" title="Open in the viewer (right-click for more)">
<div class="row"><div class="grow">
<div class="t">${esc(shotApp(s.appid))}</div>
<div class="s">${esc(new Date(s.time * 1000).toLocaleString())}</div></div>
<div class="s" title="${esc(new Date(s.time * 1000).toLocaleString())}">${esc(shotTime(s.time))}</div></div>
<button class="small" data-shot-copy="${i}" title="Copy the image to the clipboard">Copy</button>
${s.saved && !HOST.mobile ? `<span class="tag">On ${HOST.computer}</span>` : `<button class="small" data-shot-save="${i}">Save</button>`}
</div></div>`).join("")
: `<div class="sub">No screenshots on the Frame yet.</div>`;
// Thumbnails one at a time over the shared SSH connection.
for (const img of document.querySelectorAll("#shotGrid img[data-shot]")) {
const s = shots.list[+img.dataset.shot];
try {
const url = URL.createObjectURL(await shotBlob(s.id, true));
shots.urls.push(url);
img.src = url;
} catch (e) { img.alt = "Preview failed"; }
if (!img.isConnected) return; // the list was reloaded meanwhile
}
}
async function openShot(s) {
if (live) toggleLive(false);
@@ -3556,7 +3672,7 @@ async function openShot(s) {
draw();
$("srcBadge").textContent = `Screenshot · ${shotApp(s.appid)}`;
$("stamp").hidden = false; $("stamp").textContent = new Date(s.time * 1000).toLocaleString();
$("saveBtn").disabled = false;
$("saveBtn").disabled = $("copyBtn").disabled = false;
$("view").scrollIntoView({ behavior: "smooth" });
} catch (e) {
toast("Couldn't open the screenshot: " + e.message, true);
@@ -3575,15 +3691,95 @@ async function saveShots(list, btn) {
() => api("/api/shots/save", { ids: list.map(s => s.id) }), btn);
if (res) loadShots();
}
const copyShot = (s, btn) => act("Copy screenshot", () => copyImage(shotBlob(s.id, false)), btn);
$("shotGrid").onclick = e => {
const img = e.target.closest("img[data-shot]");
if (img) return openShot(shots.list[+img.dataset.shot]);
const c = e.target.closest("[data-shot-copy]");
if (c) return copyShot(shots.list[+c.dataset.shotCopy], c);
const b = e.target.closest("[data-shot-save]");
if (b) saveShots([shots.list[+b.dataset.shotSave]], b);
};
$("shotsRefresh").onclick = loadShots;
// A small right-click menu. items: [label, fn] pairs, null for a divider.
// back: the keyboard closed it, so focus returns to where it was.
function closeMenu(back) {
const menu = document.querySelector(".ctx-menu");
if (!menu) return;
menu.remove();
if (back === true && menu.opener?.isConnected) menu.opener.focus();
}
function showMenu(e, items) {
closeMenu();
const menu = document.createElement("div");
menu.className = "ctx-menu";
menu.setAttribute("role", "menu");
menu.opener = document.activeElement;
for (const it of items) {
if (!it) { menu.append(document.createElement("hr")); continue; }
const b = document.createElement("button");
b.textContent = it[0];
b.setAttribute("role", "menuitem");
b.onclick = () => { closeMenu(); it[1](); };
menu.append(b);
}
(document.fullscreenElement || document.body).append(menu); // in front of a fullscreen viewer
const r = menu.getBoundingClientRect();
menu.style.left = Math.max(4, Math.min(e.clientX, innerWidth - r.width - 4)) + "px";
menu.style.top = Math.max(4, Math.min(e.clientY, innerHeight - r.height - 4)) + "px";
menu.querySelector("button").focus();
}
document.addEventListener("pointerdown", e => { if (!e.target.closest(".ctx-menu")) closeMenu(); }, true);
document.addEventListener("keydown", e => {
const menu = document.querySelector(".ctx-menu");
if (!menu) return;
if (e.key === "Escape" || e.key === "Tab") { e.preventDefault(); return closeMenu(true); } // Tab doesn't wander off behind it
if (e.key !== "ArrowDown" && e.key !== "ArrowUp") return;
e.preventDefault(); // arrows move through the menu, not the page
const items = [...menu.querySelectorAll("button")];
const at = items.indexOf(document.activeElement), step = e.key === "ArrowDown" ? 1 : -1;
items[at < 0 ? (step > 0 ? 0 : items.length - 1) : (at + step + items.length) % items.length].focus();
});
addEventListener("blur", closeMenu);
document.addEventListener("fullscreenchange", closeMenu);
addEventListener("scroll", closeMenu, true);
// Right-click a screenshot for everything it can do.
$("shotGrid").oncontextmenu = e => {
const card = e.target.closest("[data-card]");
if (!card) return;
e.preventDefault();
const s = shots.list[+card.dataset.card];
const items = [
["Open in viewer", () => openShot(s)],
["Copy image", () => copyShot(s)],
];
if (!s.saved || HOST.mobile) items.push(null, [`Save to ${HOST.computer}`, () => saveShots([s])]);
else {
const sep = shots.folder.includes("\\") ? "\\" : "/";
items.push(null);
items.push([`Show in ${!HOST.fileManager || HOST.fileManager === "your file manager" ? "folder" : HOST.fileManager}`,
() => act("Show screenshot", () => api("/api/open", { what: "shot", id: s.id }))]);
items.push(["Copy file path", () => act("Copy file path", async () => {
await copyText(shots.folder + sep + s.file);
return { message: "Copied the file path" };
})]);
}
items.push(null, ["Copy file name", () => act("Copy file name", async () => {
await copyText(s.file);
return { message: "Copied " + s.file };
})]);
showMenu(e, items);
};
function copyText(text) {
if (!navigator.clipboard?.writeText) throw new Error("This browser can't copy here");
return navigator.clipboard.writeText(text);
}
$("shotsRefresh").onclick = () => loadShots();
$("shotsSaveNew").onclick = e => saveShots(shots.list.filter(s => !s.saved), e.currentTarget);
$("shotsFolder").onclick = e => act($("shotsFolder").textContent, () => api("/api/open", { what: "shots" }), e.currentTarget);
// Watch for new shots: a cheap listing over the shared SSH connection while the
// window is visible and the Frame is reachable, and again on coming back to it.
setInterval(() => { if (!document.hidden && online) loadShots(true); }, SHOTS_POLL_MS);
document.addEventListener("visibilitychange", () => { if (!document.hidden && online) loadShots(true); });
// ---- Panel switcher: our UI over SteamVR's panel API ----
let panelSeq = 0;
@@ -3704,10 +3900,10 @@ const typesText = el => el?.matches?.('textarea, input:not([type=range], [type=c
document.addEventListener("focusin", e => document.body.classList.toggle("typing", !!typesText(e.target)));
document.addEventListener("focusout", () => document.body.classList.remove("typing"));
// ---- keyboard and trackpad: events go to KDE Connect on the Frame (see ui/frame_input_agent.py) ----
// ---- keyboard and trackpad: events go to the Frame through gamescope (ui/frame_touch.py); accents and emoji through KDE Connect ----
const pad = { state: "off", queue: [], sending: false, poll: null, captured: false };
const PAD_STATES = {
off: "", starting: "Connecting…", installing: "Setting up KDE Connect on the Frame (first time only)…",
off: "", starting: "Connecting…", installing: "Setting up (first time only)…",
pairing: "Pairing with KDE Connect…", ready: "On",
};
// KDE Connect's specialKey numbers (plugins/mousepad in its source).
@@ -3725,7 +3921,7 @@ function padShow(status) {
$("padOn").hidden = ready || busy;
$("padOn").textContent = pad.state === "error" ? "Try again" : "Turn on";
$("padArea").classList.toggle("off", !ready);
$("padHint").textContent = pad.state === "error" ? status.message || "Couldn't reach KDE Connect on the Frame."
$("padHint").textContent = pad.state === "error" ? status.message || "Couldn't reach the Frame."
: !ready ? (busy ? PAD_STATES[pad.state] : "Turn on to use this as a trackpad for the Frame.")
: TOUCH ? "Drag to move the pointer · tap to click · two fingers to scroll · two-finger tap to right-click"
: pad.captured ? "Your mouse and keyboard now control the Frame. Press Esc to stop."
@@ -3964,7 +4160,68 @@ $("noticeMore").onclick = async () => {
toast("Thanks! Compatibility results and error details will be shared too. Change it any time in Privacy.");
};
$("noticeSettings").onclick = () => { $("privacyNotice").hidden = true; location.hash = "#privacy"; };
loadTelemetry();
const telemetryLoaded = loadTelemetry();
// ---- contact email: two separate opt-ins (ui/frame_contact.py, docs/privacy.md) ----
const contact = { email: "", updates: false, followup: false };
function renderContact(s) {
Object.assign(contact, s);
$("cEmail").value = s.email; $("cUpdates").checked = s.updates; $("cFollowup").checked = s.followup;
$("cRemove").hidden = !s.email;
$("cStatus").textContent = s.waiting ? "Saved here; it's sent to the maintainer when Frame Control can reach PostHog."
: s.email ? `Saved. ${s.email} may get ${[s.updates && "update notices", s.followup && "follow-up questions"].filter(Boolean).join(" and ")}. Remove it any time.`
: "Sent privately to the Frame Control maintainer, never shared or published. Updates are occasional release notices; follow-up questions are mainly about problem reports you send.";
}
async function saveContact(change) {
const s = await api("/api/contact", change);
renderContact(s);
return s;
}
async function loadContact() {
await telemetryLoaded;
try { renderContact(await api("/api/contact")); } catch { return; }
checkContactPrompt();
}
// One time only, only once the Frame has connected, and never on top of the privacy notice or
// straight after it (two asks in a row is nagging): checked at load and whenever the Frame connects.
async function checkContactPrompt() {
if (!$("contactNotice").hidden || !$("privacyNotice").hidden) return;
let s;
try { s = await api("/api/contact"); } catch { return; }
if (!s.showPrompt || !$("contactNotice").hidden || !$("privacyNotice").hidden) return;
$("contactNotice").hidden = false;
api("/api/contact/prompt", { prompt: "shown" }).catch(() => {});
}
$("contactNotice").onsubmit = async e => {
e.preventDefault();
if (!$("cpUpdates").checked && !$("cpFollowup").checked) { $("cpMsg").textContent = "Tick at least one, or choose No thanks."; return; }
$("cpSave").disabled = true;
try {
await saveContact({ email: $("cpEmail").value, updates: $("cpUpdates").checked, followup: $("cpFollowup").checked, fromPrompt: true });
$("contactNotice").hidden = true;
toast("Thanks! Change or remove it any time in Privacy & updates.");
} catch (err) { $("cpMsg").textContent = `Couldn't save: ${err.message}`; }
finally { $("cpSave").disabled = false; }
};
$("cpNo").onclick = async () => {
try { await api("/api/contact/prompt", { prompt: "dismissed" }); $("contactNotice").hidden = true; }
catch (err) { $("cpMsg").textContent = `Couldn't save that: ${err.message}. Try again.`; }
};
$("contactForm").onsubmit = async e => {
e.preventDefault();
const email = $("cEmail").value.trim();
if (email && !$("cUpdates").checked && !$("cFollowup").checked) {
$("cStatus").textContent = "Tick what your email may be used for, or use Remove my email."; return;
}
try { await saveContact({ email, updates: $("cUpdates").checked, followup: $("cFollowup").checked });
toast(email ? "Contact email saved." : "Contact email removed."); }
catch (err) { toast(`Couldn't save: ${err.message}`, true); }
};
$("cRemove").onclick = async () => {
try { await saveContact({ email: "", updates: false, followup: false }); toast("Contact email removed. Neither choice applies any more."); }
catch (err) { toast(`Couldn't remove it: ${err.message}`, true); }
};
loadContact();
function pageEvent(event, properties) {
if (telemetry.usage && !telemetry.blocked) api("/api/telemetry/event", { event, properties }).catch(() => {});
}
@@ -3979,8 +4236,8 @@ document.querySelectorAll("nav a").forEach(a => a.addEventListener("click", () =
const bug = { preview: "" };
const activityLines = () => [...$("log").children].slice(0, 25).map(el => el.textContent.trim());
function bugReportText() {
const contact = $("bugContact").value.trim();
const body = `Kind: ${$("bugKind").value}${contact ? `\nContact: ${contact}` : ""}\n\n${$("bugText").value.trim()}${bug.preview ? "\n\n---\nDiagnostics:\n```\n" + bug.preview + "\n```" : ""}`;
const email = $("bugFollowup").checked ? $("bugContact").value.trim() : "";
const body = `Kind: ${$("bugKind").value}${email ? `\nFollow-up questions welcome: ${email}` : ""}\n\n${$("bugText").value.trim()}${bug.preview ? "\n\n---\nDiagnostics:\n```\n" + bug.preview + "\n```" : ""}`;
return { title: $("bugTitleIn").value.trim(), body };
}
// The preview is a snapshot: exactly this text is sent, even if more activity happens meanwhile.
@@ -3996,12 +4253,20 @@ function openBugReport() {
$("bugMsg").textContent = ""; $("bugSend").disabled = false;
$("bugCancel").textContent = "Cancel";
$("bugDiagBox").open = false; $("bugLogs").disabled = false;
// A standing yes to follow-up questions (Privacy & updates) fills this in; it can be unticked.
$("bugFollowup").checked = contact.followup; $("bugContact").value = contact.followup ? contact.email : "";
$("bugContact").disabled = !contact.followup; $("bugContact").required = contact.followup;
$("bugDlg").showModal();
loadBugPreview();
}
document.body.addEventListener("click", e => { if (e.target.closest("[data-report]")) openBugReport(); });
$("bugDiag").onchange = () => { $("bugLogs").disabled = !$("bugDiag").checked; loadBugPreview(); };
$("bugLogs").onchange = loadBugPreview;
$("bugFollowup").onchange = () => {
const on = $("bugFollowup").checked;
$("bugContact").disabled = !on; $("bugContact").required = on;
if (on && !$("bugContact").value) { $("bugContact").value = contact.email; $("bugContact").focus(); }
};
$("bugCancel").onclick = () => $("bugDlg").close();
$("bugCopy").onclick = async () => {
const { title, body } = bugReportText();
@@ -4015,7 +4280,8 @@ $("bugForm").onsubmit = async e => {
try {
const res = await api("/api/report", {
kind: $("bugKind").value, title: $("bugTitleIn").value, message: $("bugText").value,
contact: $("bugContact").value, diagnostics: $("bugDiag").checked ? bug.preview : "" });
contactFollowup: $("bugFollowup").checked, contact: $("bugFollowup").checked ? $("bugContact").value : "",
diagnostics: $("bugDiag").checked ? bug.preview : "" });
$("bugMsg").textContent = `Sent, thank you. Your reference is ${res.id}.`;
$("bugCancel").textContent = "Close";
log(res.message, "ok");
@@ -4177,7 +4443,7 @@ if (window.frameApp && window.frameApp.onInstallLink) {
}
setView("headset");
refresh().then(loadShots); // after status, so app names resolve
refresh().then(() => loadShots()); // after status, so app names resolve
document.addEventListener("visibilitychange", () => { if (!document.hidden && online === false) refresh(); });
</script>
<script>
@@ -4298,7 +4564,7 @@ function onConnection(s) {
lastImg = null; lastShot = null;
$("canvas").hidden = true; $("viewerEmpty").hidden = false; $("zoombar").hidden = true;
["stamp", "srcBadge", "asleep"].forEach(id => $(id).hidden = true);
$("saveBtn").disabled = true;
$("saveBtn").disabled = $("copyBtn").disabled = true;
// Lists and their buttons (Remove, Launch…) were the other headset's: clear them
// before anyone clicks one, until the new headset's arrive.
["games", "titleList", "andApps", "flatpaks", "shotGrid", "gmGrid"].forEach(id => {
@@ -4307,7 +4573,7 @@ function onConnection(s) {
disp.list = []; disp.port = null;
// The lists behind those panels too, so filters can't bring the old ones back.
gm.owned = null; gm.byId = new Map(); gm.results = []; gm.store = []; gm.storeQ = null; gm.shown = 0; gm.seq++;
androidApps = []; shots.list = [];
androidApps = []; resetShots();
titlesSeq++; disp.seq++; // answers to loads already on their way are ignored
if (cat.apps) filterCatalog(); // "Installed" tags were the other headset's
// Confirmations still open were checked against the other headset.
@@ -4323,7 +4589,7 @@ function onConnection(s) {
$("offline").hidden = true;
const reload = link.reload;
link.reload = false;
refresh().then(() => { if (reload) reloadAll(); });
refresh().then(() => { if (reload) reloadAll(); checkContactPrompt(); });
} else if (s.phase === "failed" && s.error) {
setOnline(false, s.error.message);
} else if (s.phase === "connecting" && prev && prev.phase === "connected") {
+284 -6
View File
@@ -50,6 +50,7 @@ import frame_catalog # noqa: E402
import frame_devices # noqa: E402
import frame_steamgriddb
import frame_comfort # noqa: E402
import frame_contact # noqa: E402
import frame_host # noqa: E402
import frame_link # noqa: E402
import frame_macview # noqa: E402
@@ -59,6 +60,7 @@ import frame_report # noqa: E402
import frame_store # noqa: E402
import frame_telemetry # noqa: E402
import frame_titles # noqa: E402
import frame_touch # noqa: E402
import frame_webinstall # noqa: E402
import frame_vr # noqa: E402
import frame_utilities # noqa: E402
@@ -1004,12 +1006,274 @@ def licenses():
return notices
# KDE Connect's specialKey numbers -> Linux key codes (KEY_BACKSPACE, KEY_TAB, ...).
PAD_KEYS = {1: 14, 2: 15, 4: 105, 5: 103, 6: 106, 7: 108, 8: 104, 9: 109, 10: 102, 11: 107, 12: 28, 13: 111, 14: 1,
**{21 + i: code for i, code in enumerate([59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 87, 88])}}
PAD_MODS = (("ctrl", 29), ("alt", 56), ("shift", 42), ("super", 125))
PAD_CLICKS = {"singleclick": "left", "rightclick": "right", "middleclick": "middle"}
PAD_NOTCH = 15 # scroll units for one wheel notch
def typed_in_kde(events):
"""Whether these events must go the KDE Connect way: they hold an accent or emoji, or
earlier ones are queued, being sent or still being typed there (typing must stay in
order, so it can't be split between the two ways)."""
with _kde_lock:
waiting = bool(_kde_queue) or time.time() < _kde_until
return waiting or any(ch not in frame_touch.ASCII for e in events for ch in e.get("key", ""))
def pad_events(events, kde=False):
"""The trackpad's and key row's events as (gamescope events, KDE Connect events).
Gamescope's own input reaches every panel and needs nothing installed, but types only
US-keyboard characters. With kde, the keyboard goes to KDE Connect as it was sent, so
text stays in order; the pointer always goes through gamescope.
"""
ascii_keys = frame_touch.ASCII
out, slow = [], []
def tap(code, mods):
down = [{"key": c, "down": True} for c in mods]
out.extend(down + [{"key": code, "down": True}, {"key": code, "down": False}]
+ [{"key": c, "down": False} for c in reversed(mods)])
for e in events:
mods = [code for name, code in PAD_MODS if e.get(name)]
if "dx" in e or "dy" in e:
if not e.get("scroll"):
out.append({k: e[k] for k in ("dx", "dy") if k in e})
if e.get("scroll"):
out.append({"scroll": [0, -(e.get("dy") or 0) * PAD_NOTCH]})
for flag, button in PAD_CLICKS.items():
if e.get(flag):
out += [{"button": button, "down": True}, {"button": button, "down": False}]
if e.get("doubleclick"):
out += [{"button": "left", "down": d} for d in (True, False, True, False)]
if e.get("singlehold"):
out.append({"button": "left", "down": True})
if e.get("singlerelease"):
out.append({"button": "left", "down": False})
if kde and ("key" in e or "specialKey" in e):
slow.append({k: e[k] for k in ("key", "specialKey", "ctrl", "alt", "shift", "super") if k in e})
continue
if "specialKey" in e and e["specialKey"] in PAD_KEYS:
tap(PAD_KEYS[e["specialKey"]], mods)
if "key" in e:
for ch in e["key"]:
if ch not in ascii_keys:
continue # only reached when kde is wrong; the caller checks typed_in_kde first
elif mods:
code, shifted = ascii_keys[ch]
tap(code, mods + ([42] if shifted and 42 not in mods else []))
else:
if out and "text" in out[-1] and len(out[-1]["text"]) < 100:
out[-1]["text"] += ch
else:
out.append({"text": ch})
return out, slow
def remote_input(body):
"""{"events": [...]} sends keyboard and pointer events; {} (or none yet) just starts the agent."""
"""{"events": [...]} sends keyboard and pointer events; {} (or none yet) just starts the agent.
"sent" in the answer means the whole batch is taken (the page keeps it and tries again
if not), so the keyboard's share is queued only once that's so.
"""
events = body.get("events", [])
if not isinstance(events, list) or len(events) > INPUT_BATCH_LIMIT:
raise Failure(f"events must be a list of at most {INPUT_BATCH_LIMIT}", 400)
return _input.send([input_event(e) for e in events])
events = [input_event(e) for e in events]
touch, slow = pad_events(events, typed_in_kde(events))
status = _touch.send(touch)
if touch and not status.get("sent"):
return status
if touch:
_note_typed(touch)
if slow:
_queue_for_kde(slow)
return {**status, "sent": True}
# Typing KDE Connect is to do: it starts the first time it's needed, so this waits for it.
# Nothing on the Frame says when typed text has landed, so each way holds the other back
# for as long as what it was given should take (a short margin on a time estimate).
_kde_queue = []
_kde_lock = threading.Lock()
_kde_worker = [None]
_kde_until = 0.0 # keys typed through KDE Connect should have landed by then
_typed_until = 0.0 # when gamescope's agent will have typed everything it's been given
KDE_QUEUE_LIMIT = 200
KDE_WAIT = 90 # seconds to wait for KDE Connect before giving up on what's queued
TYPING_SETTLE = 0.6 # seconds past the estimate, once
def typing_seconds(touch):
"""How long gamescope's agent takes over these: it sleeps 8 ms after every key transition."""
transitions = 0
for e in touch:
if "text" in e:
transitions += sum(2 + 2 * frame_touch.ASCII[ch][1] for ch in e["text"] if ch in frame_touch.ASCII)
elif "key" in e:
transitions += 1
return transitions * 0.008
def _note_typed(touch):
global _typed_until
seconds = typing_seconds(touch)
if seconds:
with _kde_lock: # the end of the work, which queues up; the margin is added once, where it's checked
_typed_until = max(_typed_until, time.time()) + seconds
def _queue_for_kde(events):
with _kde_lock:
_kde_queue.extend(events)
del _kde_queue[:-KDE_QUEUE_LIMIT]
if _kde_worker[0] is None:
_kde_worker[0] = threading.Thread(target=_drain_kde, daemon=True)
_kde_worker[0].start()
def _drain_kde():
"""The only sender, in order. A batch leaves the queue while it's being sent (the queue
can be trimmed meanwhile) and goes back to the front if it didn't go."""
global _kde_until
deadline = time.time() + KDE_WAIT
while True:
with _kde_lock:
batch = _kde_queue[:INPUT_BATCH_LIMIT]
del _kde_queue[:len(batch)]
if not batch:
_kde_worker[0] = None # retired under the lock, so a new request starts another
return
wait = _typed_until + TYPING_SETTLE - time.time()
_kde_until = time.time() + 3600 # held while in flight
if wait > 0:
time.sleep(wait) # gamescope's typing, from just before, goes first
status = _input.send(batch)
with _kde_lock:
if status.get("sent"):
_kde_until = time.time() + len(batch) * 0.03 + TYPING_SETTLE
deadline = time.time() + KDE_WAIT
continue
_kde_until = 0.0
if status.get("state") == "error" or time.time() > deadline:
_kde_queue.clear() # not coming; don't hold it forever
_kde_worker[0] = None
return
_kde_queue[:0] = batch
del _kde_queue[:-KDE_QUEUE_LIMIT] # as when queuing: the oldest go first
time.sleep(0.4)
# ---- touch: the headset's panels, through gamescope's own input (frame_touch.py) ----
PANEL_WINDOW = re.compile(r"^\d{1,10}$")
PANEL_DISPLAY = re.compile(r"^:\d{1,2}$")
TOUCH_BUTTONS = ("left", "right", "middle")
def panels():
"""The headset's app panels (window, display, name, size) and which has focus."""
return json.loads(ssh("python3 - panels", stdin=(HERE / "frame_touch.py").read_text(), timeout=20))
def panel_target(q):
window, display = (q.get("window") or [""])[0], (q.get("display") or [""])[0]
if not PANEL_WINDOW.match(window) or not PANEL_DISPLAY.match(display):
raise Failure("window must be a window id and display an X display such as :1", 400)
return window, display
def panel_capture(query):
"""One frame of a panel's own window, as PNG (its pixels, without the room around it)."""
window, display = panel_target(parse_qs(query))
return ssh(f"DISPLAY={display} timeout 10 ffmpeg -hide_banner -loglevel error -nostdin -f x11grab "
f"-window_id {window} -i {display} -frames:v 1 -f image2pipe -c:v png -", timeout=20, text=False)
def touch_event(event):
"""A frame_touch.py event with only the fields it knows, in range."""
if not isinstance(event, dict):
raise Failure("each touch event must be an object", 400)
def num(name, limit):
value = event.get(name)
if isinstance(value, bool) or not isinstance(value, (int, float)) or value != value:
raise Failure(f"{name} must be a number", 400)
return max(-limit, min(limit, round(float(value), 4)))
out = {}
if "fx" in event or "fy" in event:
if "window" not in event:
raise Failure("a position needs the panel's window and display", 400)
out.update(fx=num("fx", 1), fy=num("fy", 1))
# Any event can name the panel it's meant for; the Frame drops it if another has focus.
if "window" in event:
window, display = event.get("window"), event.get("display")
if isinstance(window, bool) or not isinstance(window, int) or window <= 0:
raise Failure("window must be the panel's window id", 400)
if not isinstance(display, str) or not PANEL_DISPLAY.match(display):
raise Failure("display must be an X display such as :1", 400)
out.update(window=window, display=display)
for name in ("dx", "dy"):
if name in event:
out[name] = num(name, INPUT_MOVE_LIMIT)
if "button" in event:
if event["button"] not in TOUCH_BUTTONS:
raise Failure(f"button must be one of {', '.join(TOUCH_BUTTONS)}", 400)
out["button"], out["down"] = event["button"], event.get("down") is not False
if "scroll" in event:
sc = event["scroll"]
if not isinstance(sc, list) or len(sc) != 2:
raise Failure("scroll must be [dx, dy]", 400)
out["scroll"] = [num_value(v, 5000) for v in sc]
if "key" in event:
key = event["key"]
if isinstance(key, bool) or not isinstance(key, int) or not 0 < key < 768:
raise Failure("key must be a Linux key code", 400)
out["key"], out["down"] = key, event.get("down") is not False
if "text" in event:
text = event["text"]
if not isinstance(text, str) or not 0 < len(text) <= INPUT_TEXT_LIMIT:
raise Failure(f"text must be 1 to {INPUT_TEXT_LIMIT} characters", 400)
out["text"] = text
if not set(out) - {"window", "display"}:
raise Failure("touch event has nothing to do", 400)
return out
def num_value(value, limit):
if isinstance(value, bool) or not isinstance(value, (int, float)) or value != value:
raise Failure("scroll values must be numbers", 400)
return max(-limit, min(limit, round(float(value), 2)))
class TouchAgent(InputAgent):
"""frame_touch.py on the Frame, fed events over one long-lived ssh. Nothing to install:
it uses gamescope's own input socket and the libei that's on the image."""
def __init__(self):
super().__init__(source=HERE / "frame_touch.py", packages=[])
def deliver(self, report, force=False):
return ""
def command(self, folder=""):
code = base64.b64encode(self.source.read_bytes()).decode()
return "python3 -u -c " + shlex.quote(
f"import base64;exec(compile(base64.b64decode('{code}'),'frame_touch','exec'))")
_touch = TouchAgent()
def remote_touch(body):
"""{"events": [...]} points, clicks, scrolls and types into the focused panel."""
events = body.get("events", [])
if not isinstance(events, list) or len(events) > INPUT_BATCH_LIMIT:
raise Failure(f"events must be a list of at most {INPUT_BATCH_LIMIT}", 400)
return _touch.send([touch_event(e) for e in events])
# ---- touch: the headset's panels, through gamescope's own input (frame_touch.py) ----
@@ -1192,6 +1456,12 @@ def open_thing(body):
SHOTS_DIR.mkdir(parents=True, exist_ok=True)
frame_host.open_path(SHOTS_DIR)
return {"message": f"Opened {SHOTS_DIR} in {frame_host.FILE_MANAGER}"}
if what == "shot":
saved = SHOTS_DIR / shot_path(body.get("id")).rsplit("/", 1)[-1]
if not saved.exists():
raise Failure("That screenshot isn't saved on this computer yet", 404)
frame_host.reveal_path(saved)
return {"message": f"Showed {saved.name} in {frame_host.FILE_MANAGER}"}
except frame_host.HostError as e:
raise Failure(str(e), 500)
raise Failure("unknown target", 400)
@@ -2151,6 +2421,7 @@ POST = {
"/api/webinstall/check": webinstall_check, "/api/webinstall/start": webinstall_start,
"/api/webinstall/cancel": webinstall_cancel,
"/api/telemetry": frame_telemetry.update_settings, "/api/telemetry/event": frame_telemetry.page_event,
"/api/contact": frame_contact.save, "/api/contact/prompt": frame_contact.prompt,
"/api/report/preview": report_preview, "/api/report": report_send, "/api/macview": macview_action, "/api/panels": panels_action,
"/api/devices": lambda body: devices_post(body)}
@@ -2364,7 +2635,7 @@ class Handler(BaseHTTPRequestHandler):
elif path == "/api/touch":
self.send_json(_touch.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_touch.status))
elif path == "/api/input":
self.send_json(_input.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_input.status))
self.send_json(_touch.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_touch.status))
elif path == "/api/job":
self.send_json(job_status(url.query))
elif path == "/api/android/displays":
@@ -2380,6 +2651,8 @@ class Handler(BaseHTTPRequestHandler):
self.send_json(macview_state(parse_qs(url.query)))
elif path == "/api/telemetry":
self.send_json(frame_telemetry.state())
elif path == "/api/contact":
self.send_json(frame_contact.state())
elif path == "/api/computer/state":
self.send_json(json.loads(ssh("python3 -", stdin=(HERE / "frame_computer.py").read_text(), timeout=20)))
elif path == "/api/status":
@@ -2644,6 +2917,7 @@ def main():
sweep_tmp()
threading.Thread(target=apk_search.warm, daemon=True).start() # big indexes download before the first search
frame_telemetry.start()
frame_contact.start()
global LINK, _ONE_SERVER
if not LOCAL:
if not PRIVATE: # a private server only uses the headsets (see one_server)
@@ -2656,12 +2930,16 @@ def main():
signal.signal(signal.SIGTERM, lambda *_: (_ for _ in ()).throw(KeyboardInterrupt))
if args.exit_on_eof:
def watch_stdin():
sys.stdin.buffer.read()
# os.read, not sys.stdin.buffer.read: a buffered read holds stdin's lock,
# and if a signal stops the server first, Python aborts (SIGABRT) at exit
# when it can't take that lock back from this thread.
while os.read(0, 4096):
pass
threading.Thread(target=httpd.shutdown, daemon=True).start()
threading.Thread(target=watch_stdin, daemon=True).start()
# The real port, which --port 0 leaves to the system (the iPhone app reads it from here).
print(f"Frame Control on http://127.0.0.1:{httpd.server_address[1]} (alias: {FRAME}; Ctrl-C to stop)", flush=True)
try:
# The real port, which --port 0 leaves to the system (the iPhone app reads it from here).
print(f"Frame Control on http://127.0.0.1:{httpd.server_address[1]} (alias: {FRAME}; Ctrl-C to stop)", flush=True)
httpd.serve_forever()
except KeyboardInterrupt:
pass