Compare commits

..
Author SHA1 Message Date
saphidandClaude Sonnet 5.5 024ec593f9 Merge origin/main into pad-gamescope
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 21:06:06 +10:00
saphidandClaude Sonnet 5.5 0373555519 Pad: count gamescope's real typing time, with the settle margin added once
Review found the estimate used 12 ms a character where the agent takes 16 ms
(32 shifted), and added the margin on every request so it piled up.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 21:01:53 +10:00
saphidandClaude Sonnet 5.5 3539215517 Pad: drain the KDE Connect queue without losing, stranding or reordering input
Second review found the queue could be trimmed under an in-flight send (deleting
unsent events), the worker could retire as a request enqueued, and ASCII could
overtake an accent still being typed (or the reverse). The worker now takes its
batch out while sending and puts it back on failure, retires under the lock, and
each way waits out the other's estimated typing time.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 20:57:08 +10:00
saphidandClaude Sonnet 5.5 9f3fb89e30 Pad: keep typing in order and acknowledge it once, with accents through one KDE Connect queue
Review found accent-only input repeated forever (sent was false), text split
across two transports out of order, accents stuck until the next keypress,
duplicated on retry, and racy. A batch with an accent now sends its whole
keyboard through KDE Connect; later keys follow while any are queued; the
server owns one ordered, bounded queue drained by a single thread; sent means
the whole batch is taken.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 20:52:41 +10:00
saphidandClaude Sonnet 5.5 dc7854d10b Touch: a bare release no longer clears the stale-focus notice before the page has seen it
The page leaves the panel off releases, so a release says nothing about focus.
Found in review of the stale-latch fix.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-01 20:52:41 +10:00
Alex Southwell 28073e212a Merge pull request #65 from saphid/fix/server-stdin-abort
A stop signal no longer crashes the server, and the app restarts it by itself
2026-09-30 22:58:55 +10:00
saphidandClaude Opus 5.5 6abc765e22 App: Try Again also works when the server is up but its page failed to load
The button was accepted only while no server was known. If the server answered
and the page then failed to load, the error page showed with the server still
known, and the button did nothing. It's now accepted from the error page itself
(the window's only data: page).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 22:51:24 +10:00
saphid f73efe414c Merge main into fix/server-stdin-abort 2026-09-30 22:39:14 +10:00
Alex Southwell 704e5d7780 Merge pull request #59 from saphid/feat/contact-email-opt-in
Optional contact email with separate update and follow-up consent
2026-09-30 22:09:42 +10:00
Alex Southwell edbe8d4109 Merge pull request #63 from saphid/screenshot-copy
Screenshots: Copy, right-click menu, and new shots appear on their own
2026-09-30 20:43:22 +10:00
saphidandClaude Opus 5.5 f2c8466ba9 Screenshots: Refresh retries every failed preview, even if a background check lands meanwhile
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 20:32:28 +10:00
saphidandClaude Opus 5.5 6cf01729e2 Screenshots: fixes from review
- A right-click menu open when the headset changes closes, so it can't act on the other headset's shot.
- A preview being retried by Refresh is no longer dropped when a background check lands first.
- A late failure from a headset switched away from no longer drops the new headset's preview.
- Tab and Escape close the menu and give focus back to where it was.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 20:26:48 +10:00
saphidandClaude Opus 5.5 47a29afb4c Screenshots: recheck fixes
- In Control, a right-click on the viewer goes to the Frame only; the copy menu stays out of the way.
- Thumbnails no longer hold up the next check: a save shows as saved straight away, and a
  new shot appears while older previews are still loading.
- Refresh (or a save) during a background check reads again after it, so the answer is fresh.
- A preview that failed is retried on Refresh, not by every background check.
- Copy reports a failure if the app refuses the image, and if the browser can't copy text.
- Windows: Show in File Explorer works when the path has spaces.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 20:19:03 +10:00
saphidandClaude Opus 5.5 a0f810c018 App: restart the server by itself when it stops, and a Try Again button on the error page
A server that had been up for a minute starts again without asking. One that
stops sooner shows the error page, now headed "Frame Control stopped", with a
Try Again button (the menu item was the only way, and hard to find).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 20:17:24 +10:00
saphidandClaude Opus 5.5 1a5f089e57 Server: a stop signal no longer crashes it (SIGABRT) while the app holds stdin
The --exit-on-eof watcher read stdin with a buffered read, which holds stdin's
lock. When SIGTERM stopped the server first, Python aborted at exit trying to
take that lock back, and the app showed "The server stopped unexpectedly
(SIGABRT)". It now uses os.read. The startup line is printed inside the try,
so a signal that arrives while it's printed still runs the cleanup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 20:12:47 +10:00
saphidandClaude Sonnet 5.5 5a6b925448 Trackpad and key row go through gamescope's input; KDE Connect only for accents and emoji
The pad reaches whichever panel has focus (Steam's menus too), needs nothing installed, and no longer clamps to 1280x720. Characters that aren't on a US keyboard start KDE Connect the first time they're typed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:03:37 +10:00
saphidandClaude Opus 5.5 83d74548cd Screenshots: Copy button, right-click menu, and new shots appear on their own
Each screenshot card and the viewer get a Copy button that puts the image on
the clipboard (natively in the desktop app, as PNG in a browser). Right-click
a screenshot to open, copy, save, show it in Finder, or copy its path or name;
right-click the viewer to copy or save. The shelf re-lists the Frame's
screenshots every 8 s while the window is visible and connected, redraws only
when something changed, and keeps thumbnails it already has. Switching
headsets clears the list and ignores answers still on their way.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 15:08:36 +10:00
saphidandClaude Sonnet 5.5 60ade8c2e8 Merge origin/main into live-touch
Kept both route tables: main's media, agent, assistant and Mac view routes
plus /api/touch.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 11:04:47 +10:00
saphidandClaude Opus 5.5 aa1ac7368c frame_touch: any event that goes through clears the stale flag
A trackpad move names no panel, so waiting for an aimed event could leave
the page re-reading panels for the rest of the session; a release still
aimed at the old panel doesn't count.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 10:13:09 +10:00
saphidandClaude Opus 5.5 46b6966cf7 frame_touch: build the socket path on the Frame, so Windows can import it for tests
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:41:42 +10:00
saphidandClaude Opus 5.5 be2e770e66 Control: stale clears, pauses reconverge, pastes split per request
- The Frame says it has caught up as soon as an aimed event lands after a
  stale one, so the page stops re-reading the panels.
- After a device pause it lets go of everything it holds (releases that
  arrived while paused were dropped), and waits for the device once per
  batch, not once per event.
- The quick focus check no longer freshens the panel geometry's age.
- Each request carries at most about 100 characters of text.
- Turning Control off while it connects doesn't report an error.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:40:00 +10:00
saphidandClaude Opus 5.5 597f98af6d Control: fixes from the SWE-2 Max review
- The Frame side tracks keys as well as buttons and lets go of both when the
  session ends.
- A stale tap tells the page, which re-reads the panels at once.
- A paused input device waits instead of ending the session; only a
  disconnect does. An OS error on one event skips it.
- Presses check focus with two property reads and do the full lookup only
  when it changed.
- Writes to an agent's stdin are serialized, so two devices sending at once
  can't tear a line (the keyboard agent too).
- Connecting gives up with a message after 15 s instead of hanging on
  "Connecting…"; text goes in 100-character pieces so releases don't wait
  behind a long paste; a cancelled mouse gesture releases what's held.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:28:28 +10:00
saphidandClaude Opus 5.5 ac07efa8f1 Control: close the targeting gaps from the second review
- The focused panel's display comes from GAMESCOPE_FOCUS_DISPLAY (gamescope
  packs ":1" into the first value), so a window id repeated across :0 and :1
  can't be mistaken; the pid is only the fallback.
- Presses, keys, text and scrolls read focus afresh on the Frame; only moves
  use a reading up to a second old.
- A gesture remembers the panel it started on and does nothing more if that
  stops being the one in use; a press with no panel to aim at isn't sent.
- Opening a screenshot clears the panel Control would act on; switching to
  another app releases held keys and buttons.
- Trimming keeps a click with its position; the error backoff holds for new
  input too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 08:52:00 +10:00
saphidandClaude Opus 5.5 0f4f0b863f Control: fixes from review
- Keys held on the Frame are released with buttons when Control stops or
  the view loses focus; keys for the Frame no longer trigger Frame Control's
  own shortcuts.
- Taps only act when the picture on screen is the panel in use; positions,
  presses, keys, text and scrolls name their panel (display and window: ids
  repeat across :0 and :1, told apart by pid), and the Frame drops them if
  focus has moved on. Releases always go.
- While connecting, a tap keeps its position; on an error only releases wait
  and retries back off; trimming a long queue never drops a release.
- Lifting one of two scrolling fingers ends the scroll; a cancelled touch
  isn't a tap; clicks and holds on the bars around the picture do nothing.
- A capture loop from before a Live restart can't stop the new video.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 08:40:46 +10:00
saphidandClaude Opus 5.5 e7c81733b1 Live view: a Desktop view that stays still, and Control to tap on the Frame
The live view gets a second source and a way to use the Frame from it:

- Desktop: the app panel in use in the headset, streamed from its own window
  (x11grab of gamescope's redirected window), so it doesn't move as the
  wearer looks around. A picker shows any other panel, view only.
- Control: on the Desktop view a tap or click lands exactly where you put it;
  drag is a mouse drag, press and hold right-clicks, two fingers scroll, and
  on a computer the mouse, wheel and keyboard work directly. On the headset
  view the view is a trackpad. A text field and key row type from a phone.

Input goes through gamescope's own EIS socket (the way Steam feeds Remote
Play input) with the libei already on the image: ui/frame_touch.py, over
the same long-lived ssh machinery as the keyboard agent, nothing to
install. It reaches the panel that has focus on either X display, which
the KDE Connect route can't. Verified on the Frame and from the iPhone app
in the Simulator.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 08:23:55 +10:00
16 changed files with 783 additions and 321 deletions

No files matched your search

+1 -1
View File
@@ -73,7 +73,7 @@ Drag files onto the window to send them. Drop a game's .zip, folder or .exe to a
Browse the shots you take in the headset and save them to your Pictures folder.
**⌨️ Keyboard and trackpad**<br>
Type and point in the Frame's apps from your computer or phone, through KDE Connect, which Frame Control brings along and sets up on the Frame. Nothing else to install, anywhere.
Type and point in the headset from your computer or phone, through Valve's own input path, with nothing to install. Accents and emoji use KDE Connect, which Frame Control brings along and sets up the first time you type one.
</td>
</tr>
+28 -6
View File
@@ -1,7 +1,7 @@
// Frame Control as a desktop app (macOS, Windows, Linux): starts ui/server.py on
// a free loopback port and shows it in a native window. The server does all the
// work over the `frame` SSH alias; this file only hosts it.
const { app, BrowserWindow, Menu, Notification, clipboard, dialog, ipcMain, shell } = require("electron");
const { app, BrowserWindow, Menu, Notification, clipboard, dialog, ipcMain, nativeImage, shell } = require("electron");
const { execFile, spawn } = require("child_process");
const { promisify } = require("util");
const fs = require("fs");
@@ -149,7 +149,7 @@ async function startServer() {
const target = `http://127.0.0.1:${port}/`;
for (let i = 0; i < 100; i++) {
if (exited !== null) throw new Error(`The server exited (${exited}). See ${LOG}.`);
if (await ping(target)) { url = target; return; }
if (await ping(target)) { url = target; serverStarted = Date.now(); return; }
await new Promise((r) => setTimeout(r, 100));
}
if (server === child) server = null;
@@ -175,18 +175,27 @@ function stopServer() {
if (server) endServer(server);
}
function errorPage(message) {
function errorPage(message, title = "Frame Control couldn't start") {
const esc = (s) => s.replace(/[&<>]/g, (c) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;" }[c]));
const html = `<!doctype html><meta charset="utf-8"><body style="margin:0;height:100vh;display:grid;
place-items:center;background:${BG};color:#e6edf3;font:14px -apple-system,sans-serif">
<div style="max-width:560px;padding:32px;line-height:1.5"><h2>Frame Control couldn't start</h2>
<p>${esc(message)}</p><p style="color:#8b98a8">Fix it, then choose Frame → Restart Server.</p></div>`;
<div style="max-width:560px;padding:32px;line-height:1.5"><h2>${esc(title)}</h2>
<p>${esc(message)}</p>
<p><button onclick="this.disabled = true; frameApp.restartServer()" style="font:inherit;padding:6px 16px;
border-radius:6px;border:1px solid #30363d;background:#21262d;color:inherit;cursor:pointer">Try Again</button></p>
<p style="color:#8b98a8">Frame → Restart Server does the same.</p></div>`;
return "data:text/html;charset=utf-8," + encodeURIComponent(html);
}
// A server that had been running starts again by itself (something stopped it: a
// signal, a crash). One that stops again within a minute shows the error instead,
// so a server that can't stay up doesn't restart forever.
let serverStarted = 0;
function serverDied(why) {
url = null;
if (win) win.loadURL(errorPage(`The server stopped unexpectedly (${why}). See ${LOG}.`));
if (!win) return;
if (Date.now() - serverStarted > 60000) restartServer();
else win.loadURL(errorPage(`Its server stopped unexpectedly (${why}). See ${LOG}.`, "Frame Control stopped"));
}
// Restarts that overlap share one: two could each start a server, and the one
@@ -263,7 +272,20 @@ function fromUi(e) {
} catch { return false; }
}
// The error page's Try Again button. The error page is the only data: page the window
// shows (`url` can still be set then: the server answered but the page failed to load).
ipcMain.handle("server:restart", (e) => {
if (win && e.sender === win.webContents && e.senderFrame && e.senderFrame.url.startsWith("data:")) restartServer();
});
ipcMain.handle("clipboard:read", (e) => fromUi(e) ? clipboard.readText() : "");
// A PNG or JPEG (a screenshot) onto the clipboard as an image.
ipcMain.handle("clipboard:writeImage", (e, bytes) => {
if (!fromUi(e) || !(bytes instanceof Uint8Array)) return false;
const img = nativeImage.createFromBuffer(Buffer.from(bytes));
if (img.isEmpty()) throw new Error("not an image");
clipboard.writeImage(img);
return true;
});
ipcMain.handle("connection:setup", (e) => { if (fromUi(e)) setUpConnection(); });
ipcMain.on("keys:capture", (e, on) => { if (fromUi(e)) win.webContents.setIgnoreMenuShortcuts(on === true); });
ipcMain.handle("update:get", (e) => fromUi(e) ? publicUpdate() : null);
+4 -2
View File
@@ -2,8 +2,8 @@
// to the Frame needs no pbpaste, PowerShell, xclip or wl-clipboard. Also tells
// the page where a dropped file or folder lives, so a folder can be sideloaded
// as a title without zipping it (the local server reads it from there).
// It can open Set Up Connection when the headset can't be reached, and keeps the
// Frame menu's list of headsets up to date.
// It can put a screenshot on the clipboard as an image, open Set Up Connection when
// the headset can't be reached, and keeps the Frame menu's list of headsets up to date.
// It also receives frame-control://install links (docs/web-install.md): only
// what the link asked for, never an install; the page asks the user first.
// And it passes update state both ways: see app/updater.js.
@@ -12,7 +12,9 @@ const { contextBridge, ipcRenderer, webUtils } = require("electron");
contextBridge.exposeInMainWorld("frameApp", {
notify: (message, request) => ipcRenderer.invoke("comfort:notify", message, request),
readClipboard: () => ipcRenderer.invoke("clipboard:read"),
writeImage: (bytes) => ipcRenderer.invoke("clipboard:writeImage", bytes),
setUpConnection: () => ipcRenderer.invoke("connection:setup"),
restartServer: () => ipcRenderer.invoke("server:restart"), // the "couldn't start" page's Try Again
// The Frame menu's headset switcher: the page tells it the headsets, and hears picks.
devicesChanged: (list) => ipcRenderer.send("devices:changed", list),
onUseDevice: (cb) => {
+5 -19
View File
@@ -107,14 +107,7 @@ wrote, a short reference shown after sending, and the diagnostics below. Your
email address goes with it only if you tick **The maintainer may contact me
with follow-up questions** (the report then carries `contact_followup: true`);
it's filled in from **Contact email** below when you've agreed there. It has its own random id, so it isn't linked to
your analytics events. With that box ticked, the address also becomes your
**Contact email** below with follow-up questions ticked, so you remove it there
like any other. If it's a different address from the one saved there, it
replaces it, and update notices stop until you turn them on again (they were
agreed for the old address); the form says so before you send. The report then also
carries this copy's contact id and change number (`contact_id`, `contact_rev`,
see below), so removing or changing the address later takes back the
follow-up permission given with the report too.
your analytics events.
With **Include diagnostics** ticked (the default), the report adds:
@@ -148,8 +141,8 @@ are two separate choices, both off until you tick them:
| **The maintainer may contact me with follow-up questions** | Questions about problem reports you send, mostly |
You're asked once, in a bar at the top of the page, after the Frame has
connected for the first time, and never in the same visit as the first-run
privacy notice. **No thanks** hides it for good, and it isn't
connected for the first time, and never while or straight after the
first-run privacy notice is showing. **No thanks** hides it for good, and it isn't
shown again even if you ignore it. **Contact email** in **Privacy & updates**
is where you add, change or remove the address and either choice at any time.
@@ -158,10 +151,7 @@ Frame Control's PostHog project, the same place as problem reports, as a
`contact_consent` event with `email`, `updates`, `followup`, `action` (`set`
or `withdraw`) and the common properties above. Only the maintainer can read
that project, and nothing in it is published or shared. It's sent only when
you save, or when you send a problem report with follow-up questions ticked,
whatever the analytics settings are, because you chose to. With a report, the
address and choices are saved before the report is sent and stay saved if it
fails; like any change, they're sent as soon as PostHog can be reached. It
you save, whatever the analytics settings are, because you chose to. It
carries its own random contact id, not the analytics id, so it isn't linked
to your usage events, and a `rev` number that goes up with each change, so
the newest choice always wins. Like everything else sent, it's listed under
@@ -174,11 +164,7 @@ deletes it from this computer, including from the **Show what's been sent**
log (in earlier contact events and problem reports), and sends a `withdraw`
event with no address in it. The maintainer's list only uses the newest event from each copy, so from
then on the address isn't listed for either choice. Unticking one choice
works the same way for that choice. This also covers problem reports you sent
from this copy with follow-up questions ticked: if your newest choice since the
report (by change number, not the clock) no longer agrees to follow-up
questions at that address, the maintainer's inbox shows the permission as
withdrawn and leaves the address out. If you're offline, the change waits on
works the same way for that choice. If you're offline, the change waits on
this computer and is sent when PostHog can be reached. The earlier event
stays in PostHog until its data retention removes it; to have it deleted
sooner, ask the maintainer (for example in a problem report).
+7 -4
View File
@@ -123,10 +123,13 @@ two fingers to scroll, two-finger tap to right-click) plus a text field that
types on the Frame. On a computer, clicking the pad passes your mouse and
keyboard through to the Frame until you press Esc (⌘ is sent as Ctrl on a Mac).
It goes through **KDE Connect**, the first-party route (KDE makes the Frame's
desktop): Frame Control's server runs [`ui/frame_input_agent.py`](../ui/frame_input_agent.py)
on the Frame, which talks KDE Connect's own LAN protocol to the Frame's
`kdeconnectd` as if it were a phone. KDE Connect does the typing and clicking.
Since Control (below), the pad goes through **gamescope's own input socket**
([`ui/frame_touch.py`](../ui/frame_touch.py)), so it reaches whichever panel has
focus, Steam's own menus included, and needs nothing installed on the Frame.
Only characters that aren't on a US keyboard (accents, emoji) go through
**KDE Connect** as below; it starts the first time you type one. (Before
Control, everything went through KDE Connect, which is why the notes below
describe it in detail.)
**Verified 2026-09-28** (SteamOS 0.4.1, build 20260925.6191901):
+2 -134
View File
@@ -59,16 +59,6 @@ class Contact(Base):
self.assertEqual(fc.load()["email"], "")
self.assertEqual(self.got, [])
def test_only_a_real_true_counts_as_consent(self):
for wrong in ("false", "true", 1, 0, [], {}):
with self.assertRaisesRegex(ValueError, "true or false"):
fc.save({"email": "me@example.com", "updates": wrong, "followup": True})
with self.assertRaisesRegex(ValueError, "true or false"):
fr.send({**REPORT, "contact": "me@example.com", "contactFollowup": wrong})
self.assertEqual((fc.load()["email"], self.got), ("", []))
fc.save({"email": "me@example.com", "updates": True}) # left out is no
self.assertEqual((fc.load()["updates"], fc.load()["followup"]), (True, False))
def test_each_choice_is_sent_privately_on_its_own(self):
fc.save({"email": " me@example.com ", "updates": True})
fc.save({"email": "me@example.com", "updates": False, "followup": True})
@@ -249,121 +239,15 @@ class Contact(Base):
# ---- reports and the maintainer's list
def reports(self):
return [e["properties"] for e in self.events() if e["event"] == "problem_report"]
def test_a_report_carries_the_address_only_with_follow_up_consent(self):
fr.send({**REPORT, "contact": "me@example.com"})
self.assertFalse(fc.FILE.exists()) # no follow-up: nothing kept, nothing linked
fr.send({**REPORT, "contact": "me@example.com", "contactFollowup": True})
without, with_ = self.reports()
self.assertEqual((without["contact"], without["contact_followup"], without["contact_id"]), ("", False, ""))
without, with_ = (e["properties"] for e in self.events())
self.assertEqual((without["contact"], without["contact_followup"]), ("", False))
self.assertEqual((with_["contact"], with_["contact_followup"]), ("me@example.com", True))
self.assertEqual((with_["contact_id"], with_["contact_rev"]), (fc.load()["id"], fc.load()["rev"]))
self.assertNotEqual(with_["contact_id"], tm.settings()["id"]) # not the analytics id
with self.assertRaisesRegex(ValueError, "email address"):
fr.send({**REPORT, "contact": "discord:me", "contactFollowup": True})
def test_follow_up_given_with_a_report_is_kept_and_removed_in_settings(self):
fr.send({**REPORT, "contact": "me@example.com", "contactFollowup": True})
s = fc.state()
self.assertEqual((s["email"], s["updates"], s["followup"]), ("me@example.com", False, True))
consent = [e for e in self.events() if e["event"] == "contact_consent"]
self.assertEqual([(e["properties"]["action"], e["properties"]["rev"]) for e in consent], [("set", 1)])
self.assertEqual(consent[0]["distinct_id"], self.reports()[0]["contact_id"])
fr.send({**REPORT, "contact": "ME@example.com", "contactFollowup": True}) # already agreed
self.assertEqual(len([e for e in self.events() if e["event"] == "contact_consent"]), 1)
self.assertEqual(self.reports()[1]["contact_rev"], 1)
fc.save({"email": ""}) # Remove my email
last = self.events()[-1]
self.assertEqual((last["properties"]["action"], last["properties"]["email"], last["properties"]["rev"]),
("withdraw", "", 2))
logged = [e["properties"].get("contact") for e in tm._read_lines(tm.SENT) if e["event"] == "problem_report"]
self.assertEqual(logged, ["<removed>", "<removed>"])
def test_a_report_to_another_address_replaces_it_with_follow_up_only(self):
"""Update notices were agreed for the old address, not the new one (the form says so)."""
fc.save({"email": "old@example.com", "updates": True})
fr.send({**REPORT, "contact": "new@example.com", "contactFollowup": True})
s = fc.state()
self.assertEqual((s["email"], s["updates"], s["followup"]), ("new@example.com", False, True))
self.assertEqual(self.reports()[0]["contact_rev"], 2)
fc.save({"email": "new@example.com", "updates": True, "followup": False})
fr.send({**REPORT, "contact": "NEW@example.com", "contactFollowup": True}) # same address: kept
s = fc.state()
self.assertEqual((s["email"], s["updates"], s["followup"]), ("new@example.com", True, True))
def test_a_removal_while_the_report_saves_its_address_still_counts(self):
"""Removed while the report's own consent is on its way: the report keeps that consent's
rev (so the removal is newer) and is logged without the address."""
post, removed = tm.post, []
def slow_post(events, **kw):
post(events, **kw)
if not removed and events[0]["event"] == "contact_consent":
removed.append(fc.save({"email": ""})) # Remove my email, mid-send
with mock.patch.object(tm, "post", side_effect=slow_post):
fr.send({**REPORT, "contact": "me@example.com", "contactFollowup": True})
report = self.reports()[0]
self.assertEqual((report["contact_rev"], fc.load()["rev"], fc.state()["email"]), (1, 2, ""))
consents = [[e["distinct_id"], e["properties"]["email"], e["properties"]["followup"], e["properties"]["rev"]]
for e in self.events() if e["event"] == "contact_consent"]
row = self.report_row(cid=report["contact_id"], rev=report["contact_rev"])
fr.mark_withdrawn([row], consents)
self.assertEqual(row[10], "withdrawn")
logged = [e["properties"]["contact"] for e in tm._read_lines(tm.SENT) if e["event"] == "problem_report"]
self.assertEqual(logged, ["<removed>"])
def report_row(self, contact="me@example.com", followup=True, cid="copy", rev=1):
return ["2026-09-10T10:00:00Z", "AB12CD34", "bug", "RDP", "It never connects.", contact,
"0.4.0", "Windows", "", "", followup, cid, rev]
def test_a_later_change_takes_back_a_reports_follow_up_permission(self):
reports = [self.report_row(), # removed later
self.report_row(cid="other"), # another copy, still agrees
self.report_row(rev=3), # sent after the removal
self.report_row(cid="moved"), # address changed later
self.report_row(cid="news-only"), # follow-up unticked later
self.report_row(contact="Me@Example.com", cid="case"), # same address, any case
self.report_row(cid="", followup=True), # no contact id: left alone
self.report_row(cid="bad", rev="x")] # malformed rev: treated as 0
consents = [["copy", "me@example.com", True, 1], ["copy", "", False, 2],
["other", "me@example.com", True, 1], ["other", "me@example.com", True, 2],
["moved", "new@example.com", True, 2], ["news-only", "me@example.com", False, 2],
["case", "me@example.com", True, 2], ["bad", "", False, 1], ["short"], ["x", "", False, "?"]]
fr.mark_withdrawn(reports, consents)
self.assertEqual([r[10] for r in reports],
["withdrawn", True, True, "withdrawn", "withdrawn", True, True, "withdrawn"])
def test_the_change_number_decides_not_the_clock(self):
"""The clock went back between the report and the removal: the removal still counts."""
fr.send({**REPORT, "contact": "me@example.com", "contactFollowup": True})
with mock.patch.object(fc.time, "gmtime", return_value=time.gmtime(0)):
fc.save({"email": ""})
report = self.reports()[0]
row = self.report_row(cid=report["contact_id"], rev=report["contact_rev"])
consents = [[e["distinct_id"], e["properties"]["email"], e["properties"]["followup"], e["properties"]["rev"]]
for e in self.events() if e["event"] == "contact_consent"]
self.assertEqual(self.events()[-1]["timestamp"], "1970-01-01T00:00:00Z")
fr.mark_withdrawn([row], consents)
self.assertEqual(row[10], "withdrawn")
def test_the_inbox_shows_withdrawn_follow_up_without_the_address(self):
reports = [self.report_row(), ["short"]]
consents = [["copy", "", False, 2]]
with mock.patch.object(db, "_posthog_query", side_effect=[{"results": reports}, {"results": consents}]) as q, \
mock.patch.object(sys, "argv", ["frame_report.py", "inbox", "30"]), \
mock.patch("builtins.print") as out:
fr.main()
self.assertIn("properties.contact_rev", q.call_args_list[0].args[0])
self.assertIn("event = 'contact_consent'", q.call_args_list[1].args[0])
printed = " ".join(str(c.args[0]) for c in out.call_args_list if c.args)
self.assertIn("follow-up permission since withdrawn", printed)
self.assertNotIn("me@example.com", printed)
with mock.patch.object(db, "_posthog_query", return_value={"results": [self.report_row(followup=False)]}) as q:
fr.inbox()
self.assertEqual(q.call_count, 1) # nothing to reconcile, no second query
def test_contacts_lists_the_newest_choice_per_copy_by_consent(self):
rows = [["a", "both@example.com", True, "true", "2026-09-01T10:00:00Z"],
["b", "news@example.com", "true", False, "2026-09-02T10:00:00Z"],
@@ -388,22 +272,6 @@ class Contact(Base):
self.assertIs(server.POST["/api/contact"], fc.save)
self.assertIs(server.POST["/api/contact/prompt"], fc.prompt)
def test_saving_is_not_headset_work(self):
"""A slow send mustn't hold up switching headsets, nor be refused after a switch."""
import io
import server
seen = []
for path in ("/api/contact", "/api/contact/prompt"):
h = server.Handler.__new__(server.Handler)
body = b'{"prompt": "shown"}' if path.endswith("prompt") else b'{"email": "me@example.com", "updates": true}'
h.path, h.rfile = path, io.BytesIO(body)
h.headers = {"Content-Length": str(len(body)), "X-Frame-Device": "a-headset-switched-away-from"}
h.local_request = lambda: True
h.send_json = lambda obj, status=200: seen.append((status, server._work[0]))
with mock.patch.object(fc, "_send_pending", side_effect=lambda block=True: seen.append(("send", server._work[0]))):
h.do_POST()
self.assertEqual(seen, [("send", 0), (200, 0), (200, 0)])
# Run these once, in test_telemetry, not again through the import above.
del Base, ReportProblem
+191
View File
@@ -630,3 +630,194 @@ class PageQueue(unittest.TestCase):
if __name__ == "__main__":
unittest.main()
class PadEvents(unittest.TestCase):
"""The trackpad and key row go through gamescope; accents and emoji go the KDE Connect way."""
@classmethod
def setUpClass(cls):
import server
cls.pad = staticmethod(server.pad_events)
def test_moves_clicks_and_scroll(self):
out, extra = self.pad([{"dx": 3, "dy": -2}, {"singleclick": True}, {"singlehold": True},
{"singlerelease": True}, {"scroll": True, "dy": 1}, {"rightclick": True}])
self.assertEqual(out[0], {"dx": 3, "dy": -2})
self.assertEqual(out[1:3], [{"button": "left", "down": True}, {"button": "left", "down": False}])
self.assertEqual(out[3:5], [{"button": "left", "down": True}, {"button": "left", "down": False}])
self.assertEqual(out[5], {"scroll": [0, -15]}) # up = content follows the finger
self.assertEqual(out[6:], [{"button": "right", "down": True}, {"button": "right", "down": False}])
self.assertEqual(extra, [])
def test_keys_and_modifiers(self):
out, _ = self.pad([{"specialKey": 12}, {"specialKey": 1, "ctrl": True}, {"key": "c", "ctrl": True}])
self.assertEqual(out[:2], [{"key": 28, "down": True}, {"key": 28, "down": False}])
self.assertEqual([e["key"] for e in out[2:6]], [29, 14, 14, 29]) # Ctrl+Backspace
self.assertEqual([(e["key"], e["down"]) for e in out[6:]],
[(29, True), (46, True), (46, False), (29, False)]) # Ctrl+C
def test_text_stays_ascii_through_gamescope(self):
out, slow = self.pad([{"key": "Hi "}, {"key": "there"}])
self.assertEqual(out, [{"text": "Hi there"}])
self.assertEqual(slow, [])
def test_keyboard_goes_to_kde_whole_and_in_order(self):
events = [{"key": "a"}, {"key": "é"}, {"specialKey": 12}, {"key": "c", "ctrl": True}, {"dx": 2, "dy": 1}]
out, slow = self.pad(events, kde=True)
self.assertEqual(out, [{"dx": 2, "dy": 1}]) # the pointer still goes through gamescope
self.assertEqual(slow, events[:4]) # the keyboard as sent, in order
def test_shifted_key_with_modifier(self):
out, _ = self.pad([{"key": "A", "ctrl": True}])
self.assertEqual([e["key"] for e in out], [29, 42, 30, 30, 42, 29])
class PadDelivery(unittest.TestCase):
"""remote_input's promises: "sent" means the whole batch is taken; accents go once, in order."""
class Agent:
def __init__(self, sent=True, state="ready"):
self.sent, self.state, self.got = sent, state, []
def send(self, events):
if events and self.sent:
self.got.append(events)
return {"state": self.state, "sent": self.sent and bool(events)}
def setUp(self):
import server
self.s = server
self.saved = (server._touch, server._input, server.KDE_WAIT)
server._touch, server._input = self.Agent(), self.Agent()
self.reset()
def tearDown(self):
self.s._touch, self.s._input, self.s.KDE_WAIT = self.saved
self.settle()
self.reset()
def reset(self):
self.s._kde_queue.clear()
self.s._kde_until = self.s._typed_until = 0.0
self.s._kde_worker[0] = None
def settle(self):
worker = self.s._kde_worker[0]
if worker:
worker.join(5)
def typed(self):
return [e for batch in self.s._input.got for e in batch]
def test_accent_only_batch_is_acknowledged_and_typed_once(self):
status = self.s.remote_input({"events": [{"key": "é"}]})
self.assertTrue(status["sent"]) # so the page doesn't send it again
self.settle()
self.assertEqual(self.typed(), [{"key": "é"}])
def test_mixed_text_stays_in_one_ordered_stream(self):
self.s.remote_input({"events": [{"key": "aéb"}, {"specialKey": 12}]})
self.settle()
self.assertEqual(self.typed(), [{"key": "aéb"}, {"specialKey": 12}])
self.assertEqual(self.s._touch.got, []) # none of it split off through gamescope
def test_batch_gamescope_didnt_take_is_not_queued(self):
self.s._touch = self.Agent(sent=False, state="starting")
status = self.s.remote_input({"events": [{"dx": 1, "dy": 1}, {"key": "é"}]})
self.assertFalse(status["sent"]) # the page tries the whole batch again...
self.assertEqual(self.s._kde_queue, []) # ...so the accent isn't queued twice
def test_waits_for_kde_connect_then_sends_without_more_input(self):
self.s._input = self.Agent(sent=False, state="starting")
self.s.remote_input({"events": [{"key": "é"}]})
time.sleep(0.2)
self.assertEqual(self.s._kde_queue, [{"key": "é"}])
self.s._input.sent = True
self.settle()
self.assertEqual(self.typed(), [{"key": "é"}])
self.assertEqual(self.s._kde_queue, [])
def test_later_ascii_waits_behind_a_pending_accent(self):
self.s._input = self.Agent(sent=False, state="starting")
self.s.remote_input({"events": [{"key": "é"}]})
self.s.remote_input({"events": [{"key": "x"}]})
self.assertEqual(self.s._touch.got, []) # "x" can't overtake the "é"
self.assertEqual(self.s._kde_queue, [{"key": "é"}, {"key": "x"}])
self.s._input.sent = True
self.settle()
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "x"}])
def test_queue_is_bounded_and_dropped_on_error(self):
self.s._input = self.Agent(sent=False, state="error")
self.s.remote_input({"events": [{"key": "é"}] * 150})
self.s.remote_input({"events": [{"key": "é"}] * 150})
self.settle()
self.assertEqual(self.s._kde_queue, [])
self.s._input = self.Agent(sent=False, state="starting")
self.s.KDE_WAIT = 0.3
self.s.remote_input({"events": [{"key": "é"}] * 150})
self.s.remote_input({"events": [{"key": "é"}] * 150})
self.assertLessEqual(len(self.s._kde_queue), self.s.KDE_QUEUE_LIMIT)
self.settle()
self.assertEqual(self.s._kde_queue, [])
def test_ascii_just_after_an_accent_stays_behind_it(self):
self.s.remote_input({"events": [{"key": "é"}]})
self.settle()
self.s.remote_input({"events": [{"key": "x"}]}) # KDE Connect may still be typing the é
self.settle()
self.assertEqual(self.s._touch.got, [])
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "x"}])
def test_accent_just_after_gamescope_text_waits_for_it(self):
self.s.remote_input({"events": [{"key": "x" * 50}]})
self.assertEqual(self.s._touch.got, [[{"text": "x" * 50}]])
start = time.time()
self.s.remote_input({"events": [{"key": "é"}]})
self.settle()
self.assertGreater(time.time() - start, 0.6) # 50 keys at 16 ms, plus the margin
self.assertEqual(self.typed(), [{"key": "é"}])
def test_typing_time_counts_every_key_transition(self):
t = self.s.typing_seconds
self.assertAlmostEqual(t([{"text": "ab"}]), 4 * 0.008)
self.assertAlmostEqual(t([{"text": "A"}]), 4 * 0.008) # shift down, key down, key up, shift up
self.assertAlmostEqual(t([{"text": "x" * 500}]), 8.0) # a long paste
self.assertAlmostEqual(t([{"key": 29, "down": True}, {"dx": 3, "dy": 0}]), 0.008)
def test_the_margin_is_added_once_not_per_request(self):
for _ in range(10):
self.s.remote_input({"events": [{"key": "ab"}]})
# ten requests of 4 transitions each: the work queues up, with no margin added per request
self.assertLess(self.s._typed_until - time.time(), 10 * 4 * 0.008 + 0.05)
def test_trimming_while_sending_doesnt_drop_unsent_events(self):
slow = self.Agent()
gate = threading.Event()
send = slow.send
def held(events):
gate.wait(5)
return send(events)
slow.send = held
self.s._input = slow
self.s.remote_input({"events": [{"key": "é"}] * 150})
time.sleep(0.2) # the worker has taken its 150 and is mid-send
self.s.remote_input({"events": [{"key": "ü"}] * 150})
gate.set()
self.settle()
sent = self.typed()
self.assertEqual(sent[:150], [{"key": "é"}] * 150)
self.assertEqual(sent[150:], [{"key": "ü"}] * 150) # none of the new ones went missing
def test_input_queued_as_the_worker_finishes_is_still_sent(self):
for _ in range(40):
self.reset()
self.s._input.got.clear()
self.s.remote_input({"events": [{"key": "é"}]})
time.sleep(0.002)
self.s.remote_input({"events": [{"key": "ü"}]})
self.settle()
self.assertEqual(self.s._kde_queue, [])
self.assertEqual(self.typed(), [{"key": "é"}, {"key": "ü"}])
+18
View File
@@ -111,6 +111,8 @@ class ServerGuards(unittest.TestCase):
("/api/volume", {"level": 1.5}),
("/api/clipboard", {"text": ""}),
("/api/open", {"what": "anything-else"}),
("/api/open", {"what": "shot", "id": "1/250820/../../.ssh/id_ed25519"}),
("/api/open", {"what": "shot"}),
("/api/shots/save", {"ids": []}),
("/api/shots/save", {"ids": "1/250820/20260925225208_1.jpg"}),
("/api/shots/save", {"ids": [1]}),
@@ -121,6 +123,10 @@ class ServerGuards(unittest.TestCase):
status, payload = self.post(path, body)
self.assertEqual(status, 400, f"{path} {body} -> {payload}")
def test_showing_a_shot_needs_it_saved_here(self):
status, payload = self.post("/api/open", {"what": "shot", "id": "1/250820/19990101000000_1.jpg"})
self.assertEqual(status, 404, payload)
def test_screenshot_ids_checked_before_ssh(self):
for shot in ("../../etc/passwd", "1/250820/x.jpg", "1/2/20260925225208_1.jpg;id", "1/250820/20260925225208_1.gif"):
status, _, _ = self.request("GET", f"/api/shots/image?id={quote(shot)}", headers={"X-Frame-UI": "1"})
@@ -279,6 +285,18 @@ class OneServer(unittest.TestCase):
r = conn.getresponse()
self.assertEqual(r.status, 403, r.read())
@unittest.skipIf(os.name == "nt", "no SIGTERM on Windows")
def test_sigterm_while_the_app_holds_stdin_exits_cleanly(self):
"""The app keeps stdin open; a stop signal used to abort Python (SIGABRT) at exit."""
env = {**os.environ, "FRAME_CONTROL_DATA_DIR": tempfile.mkdtemp(prefix="frame-one-server-"),
"FRAME_ALIAS": "frame-control-test.invalid"}
proc = subprocess.Popen([sys.executable, str(ROOT / "ui" / "server.py"), "--port", "0", "--exit-on-eof"],
env=env, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
self.addCleanup(lambda: (proc.stdin.close(), proc.stdout.close()))
self.assertIn("Frame Control on", proc.stdout.readline())
proc.terminate()
self.assertEqual(proc.wait(30), 0, proc.stdout.read())
class ArtworkSettings(unittest.TestCase):
"""The settings panel's endpoints, with and without the page's X-Frame-UI key."""
+5 -6
View File
@@ -391,16 +391,15 @@ class ReportProblem(Base):
def test_send_is_a_private_posthog_event_whatever_the_settings(self):
got = self.serve()
tm.update_settings({"usage": False}) # analytics off: a deliberate report still goes
with mock.patch.object(fr.frame_contact, "from_report", return_value=("contact-id", 1)): # test_contact
res = fr.send({"kind": "idea", "title": "Live view stops", "message": "It stops after a minute.",
"contact": "me@example.com", "contactFollowup": True})
res = fr.send({"kind": "idea", "title": "Live view stops", "message": "It stops after a minute.",
"contact": "me@example.com", "contactFollowup": True})
path, body = got[0]
event = body["batch"][0]
self.assertEqual((path, body["api_key"], event["event"]), ("/batch/", "phc_test", "problem_report"))
props = event["properties"]
self.assertEqual((props["kind"], props["title"], props["message"], props["contact"], props["report_id"]),
("idea", "Live view stops", "It stops after a minute.", "me@example.com", res["id"]))
self.assertEqual((props["contact_followup"], props["contact_id"], props["contact_rev"]), (True, "contact-id", 1))
self.assertIs(props["contact_followup"], True)
self.assertEqual((props["$process_person_profile"], props["$geoip_disable"]), (False, True))
self.assertNotEqual(event["distinct_id"], tm.settings()["id"]) # not linked to the analytics
self.assertIn(res["id"], res["message"])
@@ -423,8 +422,8 @@ class ReportProblem(Base):
def test_the_inbox_skips_malformed_reports(self):
good = ["2026-09-28T09:50:00Z", "AB12CD34", "bug", "Live view stops", "It stops.", None,
"0.4.0", "macOS", "", "", None, None, None]
rows = [["2026-09-28T10:00:00Z", "X", "bug", "Hand-made", None, None, None, None, None, None, None, None, None],
"0.4.0", "macOS", "", "", None]
rows = [["2026-09-28T10:00:00Z", "X", "bug", "Hand-made", None, None, None, None, None, None, None],
["short"], good]
with mock.patch.object(db, "_posthog_query", return_value={"results": rows}), \
mock.patch.object(sys, "argv", ["frame_report.py", "inbox"]), \
+8
View File
@@ -147,6 +147,14 @@ class Apply(unittest.TestCase):
self.assertFalse(self.t.STALE[0])
self.assertEqual(self.said[-1][1].get("stale"), None)
def test_stale_survives_a_bare_release(self):
gs = FakeGamescope()
panel = self.t.apply(gs, {"button": "left", "down": True, "window": 99, "display": ":1"}, None)
self.assertTrue(self.t.STALE[0])
self.t.apply(gs, {"button": "left", "down": False}, panel) # the page's release names no panel
self.assertTrue(self.t.STALE[0])
self.assertEqual(self.said[-1][1].get("stale"), True)
def test_same_window_id_on_the_other_display_is_another_panel(self):
gs = FakeGamescope()
self.t.apply(gs, {"fx": 0.5, "fy": 0.5, "window": 7, "display": ":0"}, None)
+2 -36
View File
@@ -71,31 +71,6 @@ def valid_email(email):
return len(email) <= EMAIL_MAX and bool(EMAIL_RE.fullmatch(email))
def flag(body, key):
"""A consent choice: true only when it really is true (not "false" or 1), left out is no."""
v = body.get(key)
if v is not None and not isinstance(v, bool):
raise ValueError(f'{key} must be true or false')
return v is True
def from_report(email):
"""Follow-up questions agreed to with a problem report: the address becomes the contact
email with that choice ticked, so it shows in Settings and is removed the same way. Update
notices stay on only for the same address: a different one replaces the old address with
follow-up questions only (the report form says so before sending). Returns (contact id,
rev) for the report to carry, read together with the change itself: a later change from
this copy has a higher rev, and the newest such change decides whether the report's
follow-up permission still stands, whatever the clocks say."""
with _lock:
s = load()
same = s['email'].lower() == email.lower()
changed, cid, rev = _apply({'email': s['email'] if same else email,
'updates': s['updates'] and same, 'followup': True})
_deliver(changed)
return cid, rev
def state():
"""What the page shows. showPrompt: the one-time prompt hasn't been shown or answered yet,
and the Frame has connected at least once (setup worked), so it never greets a new install."""
@@ -180,14 +155,8 @@ def redact_removed(event, started):
def save(body):
"""Set, change or remove the address and the two choices. An address needs at least one
choice ticked; an empty address (or neither ticked) removes it and withdraws both."""
_deliver(_apply(body)[0])
return state()
def _apply(body):
"""save()'s change, kept here and waiting to send. Returns (changed, contact id, rev)."""
email = str(body.get('email') or '').strip()
updates, followup = flag(body, 'updates'), flag(body, 'followup')
updates, followup = bool(body.get('updates')), bool(body.get('followup'))
if email and not valid_email(email):
raise ValueError("that doesn't look like an email address")
if email and not (updates or followup):
@@ -212,12 +181,9 @@ def _apply(body):
_forget_locally(old)
except OSError:
pass
return changed, s['id'], s['rev']
def _deliver(changed):
if changed and not _send_pending(block=False):
_wake.set() # offline, or a send under way that will take this change with it
return state()
def prompt(body):
+13
View File
@@ -153,6 +153,19 @@ def open_path(path):
stderr=subprocess.DEVNULL, **DETACHED)
def reveal_path(path):
"""Show a file selected in its folder (Linux file managers vary, so there the folder opens)."""
path = Path(path)
if MAC:
cmd = ["open", "-R", str(path)]
elif WINDOWS:
cmd = f'explorer /select,"{path}"' # as one string: Explorer wants the quotes after the comma
else:
return open_path(path.parent)
subprocess.Popen(cmd, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL, **DETACHED)
open_url = open_path # the same openers hand URLs to the default browser
+9 -47
View File
@@ -112,22 +112,18 @@ def send(body):
"""Send the report to PostHog. Returns {"id", "message"}; raises ReportError."""
kind = body.get('kind') if body.get('kind') in KINDS else 'bug'
title, text, diag = compose(body)
followup = frame_contact.flag(body, 'contactFollowup')
followup = bool(body.get('contactFollowup'))
contact = str(body.get('contact') or '').strip() if followup else ''
if followup and not frame_contact.valid_email(contact):
raise ValueError('add your email address for follow-up questions, or untick that box')
started = time.time() # a removal from now on (even while saving the address) is redacted from the log
# It becomes the contact email in Settings, where it's changed or removed like any other.
contact_id, contact_rev = frame_contact.from_report(contact) if followup else ('', 0)
ref = uuid.uuid4().hex[:8].upper()
props = {**frame_telemetry.common(), 'kind': kind, 'title': title, 'message': text,
'contact': contact, 'contact_followup': followup, 'diagnostics': diag,
# Only with an address: a later change from this copy (higher rev) can take it back.
'contact_id': contact_id, 'contact_rev': contact_rev,
'report_id': ref, 'steamos': str(frame.get('build') or '')[:120], 'level': 'report'}
# Its own random id: a report can carry contact details, so it isn't linked to this copy's analytics.
event = {'event': 'problem_report', 'distinct_id': str(uuid.uuid4()), 'uuid': str(uuid.uuid4()),
'timestamp': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'properties': props}
started = time.time()
try:
frame_telemetry.post([event], timeout=30)
except frame_telemetry.SendError as e:
@@ -147,50 +143,15 @@ class ReportError(RuntimeError):
def inbox(days=30):
"""The maintainer's recent reports from PostHog, newest first (needs the personal API key
frame_compat_db.sync uses). Column 10 is whether the person may be asked follow-up
questions now: 'withdrawn' when a later choice from the same copy took it back."""
frame_compat_db.sync uses)."""
import frame_compat_db
days = int(days)
res = frame_compat_db._posthog_query(
"SELECT timestamp, properties.report_id, properties.kind, properties.title, properties.message, "
"properties.contact, properties.app_version, properties.os, properties.steamos, properties.diagnostics, "
"properties.contact_followup, properties.contact_id, properties.contact_rev "
f"FROM events WHERE event = 'problem_report' AND timestamp > now() - INTERVAL {days} DAY "
"properties.contact_followup "
f"FROM events WHERE event = 'problem_report' AND timestamp > now() - INTERVAL {int(days)} DAY "
"ORDER BY timestamp DESC LIMIT 200")
rows = [r for r in res.get('results') or [] if isinstance(r, list) and len(r) == 13]
if any(r[11] and _yes(r[10]) for r in rows):
later = frame_compat_db._posthog_query(
"SELECT distinct_id, properties.email, properties.followup, ifNull(toInt(properties.rev), 0) "
"FROM events WHERE event = 'contact_consent' LIMIT 100000")
mark_withdrawn(rows, later.get('results') or [])
return rows
def mark_withdrawn(reports, consents):
"""Mark reports whose follow-up permission was taken back: the newest contact choice from
the same copy made after the report (a higher rev than it carries, not a later clock) no
longer agrees to follow-up questions at that address."""
newest = {}
for c in consents:
if not isinstance(c, list) or len(c) != 4:
continue
cid, email, followup, rev = c
try:
rev = int(rev or 0)
except (TypeError, ValueError):
continue
if rev > newest.get(str(cid), (-1,))[0]:
newest[str(cid)] = (rev, str(email or ''), followup)
for r in reports:
if not (r[11] and _yes(r[10])):
continue
try:
sent_at = int(r[12] or 0)
except (TypeError, ValueError):
sent_at = 0
rev, email, followup = newest.get(str(r[11]), (-1, '', None))
if rev > sent_at and not (_yes(followup) and email.strip().lower() == str(r[5] or '').strip().lower()):
r[10] = 'withdrawn'
return res.get('results') or []
def _yes(v):
@@ -243,13 +204,14 @@ def main():
if cmd != 'inbox':
sys.exit(USAGE)
for row in inbox(*(args[:1] or [30])):
if not isinstance(row, list) or len(row) != 11:
continue
ts, ref, kind, title, text, contact, version, osname, steamos, diag = (str(v or '') for v in row[:10])
# Reports from before contact_followup existed only carried an address given for a reply.
reply = contact and (row[10] is None or _yes(row[10]))
print(f"== {ts[:16].replace('T', ' ')} {ref} [{kind}] {title}")
print(f" {version} on {osname}, SteamOS {steamos or 'unknown'}"
f"{', may follow up at ' + contact if reply else ''}"
f"{', follow-up permission since withdrawn' if row[10] == 'withdrawn' else ''}")
f"{', may follow up at ' + contact if reply else ''}")
print(' ' + text.replace('\n', '\n '))
if diag:
print(' --- diagnostics\n ' + diag.replace('\n', '\n '))
+3 -1
View File
@@ -369,8 +369,10 @@ def apply(gs, event, panel):
say("ready", focus=panel.get("window"), display=panel.get("display"), stale=True) # the page re-syncs
STALE[0] = True
return panel
if STALE[0] and not stale:
release = event.get("down") is False and ("button" in event or "key" in event)
if STALE[0] and not stale and not (release and "window" not in event):
# Anything that goes through (a trackpad move names no panel) means caught up: stop re-syncing.
# A bare release doesn't: the page leaves the panel off releases, so it says nothing about focus.
STALE[0] = False
say("ready", focus=(panel or {}).get("window"), display=(panel or {}).get("display"))
if "fx" in event and panel and panel.get("window"):
+207 -56
View File
@@ -258,7 +258,12 @@
.shot-card .row { flex-wrap: nowrap; }
.shot-card .grow { flex: 1; min-width: 0; }
.shot-card .t { color: var(--bright); font-size: 13px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.shot-card .s { color: var(--muted); font-size: 12px; }
.shot-card .s { color: var(--muted); font-size: 12px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.ctx-menu { position: fixed; z-index: 1000; min-width: 200px; padding: 4px; border-radius: 4px; background: #232c38;
box-shadow: 0 10px 28px rgba(0,0,0,.6), 0 0 0 1px rgba(255,255,255,.08); }
.ctx-menu button { display: block; width: 100%; height: 30px; padding: 0 10px; text-align: left; background: none; }
.ctx-menu button:hover, .ctx-menu button:focus-visible { background: var(--blue); color: #fff; outline: none; }
.ctx-menu hr { border: 0; border-top: 1px solid rgba(255,255,255,.1); margin: 4px 2px; }
/* ---- library shelf (portrait capsules, like Steam's library home) ---- */
.shelf { display: grid; grid-template-columns: repeat(auto-fill, minmax(150px, 1fr)); gap: 16px; }
@@ -717,6 +722,7 @@
<button class="action" id="shotBtn">Capture</button>
<button id="liveBtn" title="Keep updating, as video">Live</button>
<button id="ctrlBtn" title="Control the Frame by tapping or clicking on the view (C)">Control</button>
<button id="copyBtn" disabled title="Copy the image to the clipboard (or right-click it)">Copy</button>
<button id="saveBtn" disabled>Save</button>
</span>
</div>
@@ -870,10 +876,9 @@
<button data-pad-key="12" title="Enter">Enter</button>
</span>
</div>
<div class="hint">Types and points in apps on the Frame, such as Chromium or the desktop's Linux apps; Steam's own VR menus
don't take it. It works through KDE Connect, which comes with Frame Control: the first time, it copies Valve's build
for the Frame (3.6 MB, 18 MB unpacked) into your home folder there and pairs with it. No internet needed, and nothing
else to install. <a href="#" data-about>Licences</a></div>
<div class="hint">Types and points in whichever panel has focus in the headset, through Valve's own input path: nothing to
install and no internet needed. Accents and emoji go through KDE Connect, which comes with Frame Control and starts
the first time you type one (it copies Valve's build for the Frame, 3.6 MB, into your home folder there). <a href="#" data-about>Licences</a></div>
</section>
<section id="shots">
@@ -883,7 +888,7 @@
<button class="action small" id="shotsSaveNew" disabled>Save new to this computer</button>
</div>
<div class="shot-grid" id="shotGrid"><div class="sub">Loading…</div></div>
<div class="hint">Screenshots you take in the headset with Steam's screenshot shortcut. Click one to open it in the viewer; Save copies it to <code>~/Pictures/SteamFrame</code>.</div>
<div class="hint">Screenshots you take in the headset with Steam's screenshot shortcut. New ones appear on their own. Click one to open it in the viewer, Copy puts it on the clipboard, and Save copies it to <code>~/Pictures/SteamFrame</code>. Right-click for more.</div>
</section>
</div>
@@ -1262,9 +1267,8 @@
<label class="field">What happened?<textarea id="bugText" maxlength="5000" required minlength="10"
placeholder="What you did, what happened, and what you expected."></textarea></label>
<label class="popt"><input type="checkbox" id="bugFollowup"><b>The maintainer may contact me with follow-up questions</b>
<span class="sub">Optional. Your email address goes with this report only when this is ticked, and is kept as your contact email in Privacy &amp; updates, where you can remove it.</span></label>
<span class="sub">Optional. Your email address goes with this report only when this is ticked.</span></label>
<label class="field">Your email address<input type="email" id="bugContact" maxlength="254" placeholder="you@example.com" disabled></label>
<p class="hint" id="bugReplaces" role="status" hidden></p>
<label class="popt"><input type="checkbox" id="bugDiag" checked><b>Include diagnostics</b>
<span class="sub">Frame Control's version, your OS and the Frame's SteamOS build.</span></label>
<label class="popt"><input type="checkbox" id="bugLogs"><b>Also include recent activity and the server log</b>
@@ -1899,7 +1903,7 @@ async function capture() {
if (source === "panel") $("srcBadge").textContent = `Desktop · ${shotPanel?.name || "panel"}`;
ctrlShow();
$("stamp").hidden = false; $("stamp").textContent = new Date().toLocaleTimeString();
$("saveBtn").disabled = false;
$("saveBtn").disabled = $("copyBtn").disabled = false;
return true;
} catch (e) {
log("Capture failed: " + e.message, "e");
@@ -1981,7 +1985,7 @@ async function startVideo() {
$("viewer").classList.remove("busy");
c.hidden = false; $("viewerEmpty").hidden = true; $("zoombar").hidden = false; $("asleep").hidden = true;
$("srcBadge").hidden = false; $("srcBadge").textContent = `${video.label} · video`;
$("stamp").hidden = false; $("saveBtn").disabled = false;
$("stamp").hidden = false; $("saveBtn").disabled = $("copyBtn").disabled = false;
}
frames++;
const now = performance.now();
@@ -2050,6 +2054,35 @@ $("saveBtn").onclick = () => lastShot ? download(lastShot.blob, lastShot.file) :
if (!b) return toast("Couldn't encode the image", true);
download(b, `frame-${view}-${new Date().toISOString().replace(/[:.]/g, "-")}.png`);
}, "image/png");
const copyViewer = () => act("Copy image", () => copyImage(lastShot ? lastShot.blob
: new Promise((ok, bad) => $("canvas").toBlob(b => b ? ok(b) : bad(new Error("couldn't encode the image")), "image/png"))));
$("copyBtn").onclick = copyViewer;
// Right-click the viewer to copy or save what it shows (in Control, a right-click goes to the Frame).
$("canvas").oncontextmenu = e => {
if (ctrl.on || $("saveBtn").disabled) return;
e.preventDefault();
showMenu(e, [["Copy image", copyViewer], ["Save image", () => $("saveBtn").click()]]);
};
// An image onto the clipboard. The app does it natively (JPEG too); a browser
// takes PNG only, and the blob is handed over as a promise so the click still counts.
async function copyImage(blob) {
if (window.frameApp?.writeImage) {
if (!await window.frameApp.writeImage(new Uint8Array(await (await blob).arrayBuffer()))) throw new Error("the app refused the image");
return { message: "Copied the image" };
}
if (!navigator.clipboard?.write || !window.ClipboardItem) throw new Error("This browser can't copy images here");
await navigator.clipboard.write([new ClipboardItem({ "image/png": Promise.resolve(blob).then(pngBlob) })]);
return { message: "Copied the image" };
}
async function pngBlob(blob) {
if (blob.type === "image/png") return blob;
const bmp = await createImageBitmap(blob);
const c = document.createElement("canvas");
c.width = bmp.width; c.height = bmp.height;
c.getContext("2d").drawImage(bmp, 0, 0);
bmp.close();
return new Promise((ok, bad) => c.toBlob(b => b ? ok(b) : bad(new Error("couldn't encode the image")), "image/png"));
}
function download(blob, name) {
if (savesToDevice()) return act("Save image", () => saveToDevice([{ blob, name }]));
const a = document.createElement("a");
@@ -3527,8 +3560,11 @@ loadReports();
api("/api/host").then(applyHostWording).catch(() => {});
// ---- Steam screenshots from the headset ----
const shots = { list: [], urls: [] };
// thumbs: id -> promise of an object URL, kept across reloads so a refresh
// that finds a new shot fetches only that one's thumbnail.
const shots = { list: [], sig: null, thumbs: new Map(), loading: null, gen: 0, fill: 0 };
const STEAMVR_APPID = "250820";
const SHOTS_POLL_MS = 8000;
function shotApp(appid) {
if (appid === STEAMVR_APPID) return "SteamVR";
const g = state?.games?.find(x => x.appid === appid);
@@ -3540,35 +3576,85 @@ async function shotBlob(id, thumb) {
if (!r.ok) throw new Error((await r.json().catch(() => ({}))).error || `HTTP ${r.status}`);
return r.blob();
}
async function loadShots() {
$("shotsRefresh").disabled = true;
function shotThumb(id) {
const thumbs = shots.thumbs; // this headset's: a late failure must not touch the next one's
if (!thumbs.has(id)) {
const p = shotBlob(id, true).then(b => URL.createObjectURL(b));
p.catch(() => { if (thumbs.get(id) === p) thumbs.delete(id); }); // try again next time
thumbs.set(id, p);
}
return thumbs.get(id);
}
// quiet: a background check. It keeps what's shown if the Frame can't be read,
// and redraws only when the shots (or whether they're saved here) changed.
function loadShots(quiet) {
if (shots.loading) {
// A check already on its way will do for another check; Refresh, or a save
// that just finished, reads again after it so the answer is a fresh one.
const again = () => loadShots();
return quiet === true ? shots.loading : shots.loading.then(again, again);
}
const p = readShots(quiet === true).finally(() => { if (shots.loading === p) shots.loading = null; });
return shots.loading = p;
}
// Forget the shots of a headset we've switched away from, and ignore its answers still on their way.
function resetShots() {
closeMenu(); // its items were about the other headset's shot
shots.gen++; shots.fill++; shots.loading = null; shots.list = []; shots.sig = null;
for (const p of shots.thumbs.values()) p.then(URL.revokeObjectURL, () => {});
shots.thumbs = new Map();
}
async function readShots(quiet) {
const gen = shots.gen;
if (!quiet) $("shotsRefresh").disabled = true;
let got;
try {
shots.list = (await api("/api/shots")).shots;
got = await api("/api/shots");
} catch (e) {
return failed($("shotGrid"), e);
if (!quiet && gen === shots.gen) { shots.sig = null; failed($("shotGrid"), e); }
return;
} finally { $("shotsRefresh").disabled = false; }
shots.urls.forEach(URL.revokeObjectURL); shots.urls = [];
if (gen !== shots.gen) return;
({ shots: shots.list, folder: shots.folder } = got);
const sig = JSON.stringify(shots.list.map(s => [s.id, s.saved, shotApp(s.appid)]));
if (sig !== shots.sig) { shots.sig = sig; drawShots(); }
fillThumbs(quiet); // not waited for: the next check, or a save, needn't sit behind the thumbnails
}
// Thumbnails one at a time over the shared SSH connection. One that failed is
// tried again on Refresh, not by every background check.
async function fillThumbs(quiet) {
const run = ++shots.fill;
// Refresh puts every failed one back in line first, so a background pass that takes over carries them all on.
if (!quiet) for (const img of document.querySelectorAll("#shotGrid img[data-failed]")) delete img.dataset.failed;
for (const img of document.querySelectorAll("#shotGrid img[data-shot]:not([src])" + (quiet ? ":not([data-failed])" : ""))) {
const s = shots.list[+img.dataset.shot];
let url;
try { url = await shotThumb(s.id); } catch (e) { img.alt = "Preview failed"; img.dataset.failed = 1; }
if (run !== shots.fill) return; // a newer pass (after a check, a redraw or another headset) has taken over
if (url) img.src = url;
}
}
// "Sep 28, 10:37 PM": short enough to sit beside the card's buttons (the full date is its tooltip).
function shotTime(t) {
return new Date(t * 1000).toLocaleString([], { month: "short", day: "numeric", hour: "numeric", minute: "2-digit" });
}
function drawShots() {
const ids = new Set(shots.list.map(s => s.id));
for (const [id, p] of shots.thumbs) {
if (!ids.has(id)) { shots.thumbs.delete(id); p.then(URL.revokeObjectURL, () => {}); }
}
const unsaved = shots.list.filter(s => !s.saved).length;
$("shotCount").textContent = shots.list.length ? `${shots.list.length} on the Frame` + (unsaved && !HOST.mobile ? ` · ${unsaved} not on this ${HOST.computer}` : "") : "";
$("shotsSaveNew").disabled = HOST.mobile ? !shots.list.length : !unsaved;
$("shotGrid").innerHTML = shots.list.length ? shots.list.map((s, i) => `<div class="shot-card">
<img class="thumb" data-shot="${i}" alt="Screenshot from ${esc(shotApp(s.appid))}" title="Open in the viewer">
$("shotGrid").innerHTML = shots.list.length ? shots.list.map((s, i) => `<div class="shot-card" data-card="${i}">
<img class="thumb" data-shot="${i}" alt="Screenshot from ${esc(shotApp(s.appid))}" title="Open in the viewer (right-click for more)">
<div class="row"><div class="grow">
<div class="t">${esc(shotApp(s.appid))}</div>
<div class="s">${esc(new Date(s.time * 1000).toLocaleString())}</div></div>
<div class="s" title="${esc(new Date(s.time * 1000).toLocaleString())}">${esc(shotTime(s.time))}</div></div>
<button class="small" data-shot-copy="${i}" title="Copy the image to the clipboard">Copy</button>
${s.saved && !HOST.mobile ? `<span class="tag">On ${HOST.computer}</span>` : `<button class="small" data-shot-save="${i}">Save</button>`}
</div></div>`).join("")
: `<div class="sub">No screenshots on the Frame yet.</div>`;
// Thumbnails one at a time over the shared SSH connection.
for (const img of document.querySelectorAll("#shotGrid img[data-shot]")) {
const s = shots.list[+img.dataset.shot];
try {
const url = URL.createObjectURL(await shotBlob(s.id, true));
shots.urls.push(url);
img.src = url;
} catch (e) { img.alt = "Preview failed"; }
if (!img.isConnected) return; // the list was reloaded meanwhile
}
}
async function openShot(s) {
if (live) toggleLive(false);
@@ -3586,7 +3672,7 @@ async function openShot(s) {
draw();
$("srcBadge").textContent = `Screenshot · ${shotApp(s.appid)}`;
$("stamp").hidden = false; $("stamp").textContent = new Date(s.time * 1000).toLocaleString();
$("saveBtn").disabled = false;
$("saveBtn").disabled = $("copyBtn").disabled = false;
$("view").scrollIntoView({ behavior: "smooth" });
} catch (e) {
toast("Couldn't open the screenshot: " + e.message, true);
@@ -3605,15 +3691,95 @@ async function saveShots(list, btn) {
() => api("/api/shots/save", { ids: list.map(s => s.id) }), btn);
if (res) loadShots();
}
const copyShot = (s, btn) => act("Copy screenshot", () => copyImage(shotBlob(s.id, false)), btn);
$("shotGrid").onclick = e => {
const img = e.target.closest("img[data-shot]");
if (img) return openShot(shots.list[+img.dataset.shot]);
const c = e.target.closest("[data-shot-copy]");
if (c) return copyShot(shots.list[+c.dataset.shotCopy], c);
const b = e.target.closest("[data-shot-save]");
if (b) saveShots([shots.list[+b.dataset.shotSave]], b);
};
$("shotsRefresh").onclick = loadShots;
// A small right-click menu. items: [label, fn] pairs, null for a divider.
// back: the keyboard closed it, so focus returns to where it was.
function closeMenu(back) {
const menu = document.querySelector(".ctx-menu");
if (!menu) return;
menu.remove();
if (back === true && menu.opener?.isConnected) menu.opener.focus();
}
function showMenu(e, items) {
closeMenu();
const menu = document.createElement("div");
menu.className = "ctx-menu";
menu.setAttribute("role", "menu");
menu.opener = document.activeElement;
for (const it of items) {
if (!it) { menu.append(document.createElement("hr")); continue; }
const b = document.createElement("button");
b.textContent = it[0];
b.setAttribute("role", "menuitem");
b.onclick = () => { closeMenu(); it[1](); };
menu.append(b);
}
(document.fullscreenElement || document.body).append(menu); // in front of a fullscreen viewer
const r = menu.getBoundingClientRect();
menu.style.left = Math.max(4, Math.min(e.clientX, innerWidth - r.width - 4)) + "px";
menu.style.top = Math.max(4, Math.min(e.clientY, innerHeight - r.height - 4)) + "px";
menu.querySelector("button").focus();
}
document.addEventListener("pointerdown", e => { if (!e.target.closest(".ctx-menu")) closeMenu(); }, true);
document.addEventListener("keydown", e => {
const menu = document.querySelector(".ctx-menu");
if (!menu) return;
if (e.key === "Escape" || e.key === "Tab") { e.preventDefault(); return closeMenu(true); } // Tab doesn't wander off behind it
if (e.key !== "ArrowDown" && e.key !== "ArrowUp") return;
e.preventDefault(); // arrows move through the menu, not the page
const items = [...menu.querySelectorAll("button")];
const at = items.indexOf(document.activeElement), step = e.key === "ArrowDown" ? 1 : -1;
items[at < 0 ? (step > 0 ? 0 : items.length - 1) : (at + step + items.length) % items.length].focus();
});
addEventListener("blur", closeMenu);
document.addEventListener("fullscreenchange", closeMenu);
addEventListener("scroll", closeMenu, true);
// Right-click a screenshot for everything it can do.
$("shotGrid").oncontextmenu = e => {
const card = e.target.closest("[data-card]");
if (!card) return;
e.preventDefault();
const s = shots.list[+card.dataset.card];
const items = [
["Open in viewer", () => openShot(s)],
["Copy image", () => copyShot(s)],
];
if (!s.saved || HOST.mobile) items.push(null, [`Save to ${HOST.computer}`, () => saveShots([s])]);
else {
const sep = shots.folder.includes("\\") ? "\\" : "/";
items.push(null);
items.push([`Show in ${!HOST.fileManager || HOST.fileManager === "your file manager" ? "folder" : HOST.fileManager}`,
() => act("Show screenshot", () => api("/api/open", { what: "shot", id: s.id }))]);
items.push(["Copy file path", () => act("Copy file path", async () => {
await copyText(shots.folder + sep + s.file);
return { message: "Copied the file path" };
})]);
}
items.push(null, ["Copy file name", () => act("Copy file name", async () => {
await copyText(s.file);
return { message: "Copied " + s.file };
})]);
showMenu(e, items);
};
function copyText(text) {
if (!navigator.clipboard?.writeText) throw new Error("This browser can't copy here");
return navigator.clipboard.writeText(text);
}
$("shotsRefresh").onclick = () => loadShots();
$("shotsSaveNew").onclick = e => saveShots(shots.list.filter(s => !s.saved), e.currentTarget);
$("shotsFolder").onclick = e => act($("shotsFolder").textContent, () => api("/api/open", { what: "shots" }), e.currentTarget);
// Watch for new shots: a cheap listing over the shared SSH connection while the
// window is visible and the Frame is reachable, and again on coming back to it.
setInterval(() => { if (!document.hidden && online) loadShots(true); }, SHOTS_POLL_MS);
document.addEventListener("visibilitychange", () => { if (!document.hidden && online) loadShots(true); });
// ---- Panel switcher: our UI over SteamVR's panel API ----
let panelSeq = 0;
@@ -3734,10 +3900,10 @@ const typesText = el => el?.matches?.('textarea, input:not([type=range], [type=c
document.addEventListener("focusin", e => document.body.classList.toggle("typing", !!typesText(e.target)));
document.addEventListener("focusout", () => document.body.classList.remove("typing"));
// ---- keyboard and trackpad: events go to KDE Connect on the Frame (see ui/frame_input_agent.py) ----
// ---- keyboard and trackpad: events go to the Frame through gamescope (ui/frame_touch.py); accents and emoji through KDE Connect ----
const pad = { state: "off", queue: [], sending: false, poll: null, captured: false };
const PAD_STATES = {
off: "", starting: "Connecting…", installing: "Setting up KDE Connect on the Frame (first time only)…",
off: "", starting: "Connecting…", installing: "Setting up (first time only)…",
pairing: "Pairing with KDE Connect…", ready: "On",
};
// KDE Connect's specialKey numbers (plugins/mousepad in its source).
@@ -3755,7 +3921,7 @@ function padShow(status) {
$("padOn").hidden = ready || busy;
$("padOn").textContent = pad.state === "error" ? "Try again" : "Turn on";
$("padArea").classList.toggle("off", !ready);
$("padHint").textContent = pad.state === "error" ? status.message || "Couldn't reach KDE Connect on the Frame."
$("padHint").textContent = pad.state === "error" ? status.message || "Couldn't reach the Frame."
: !ready ? (busy ? PAD_STATES[pad.state] : "Turn on to use this as a trackpad for the Frame.")
: TOUCH ? "Drag to move the pointer · tap to click · two fingers to scroll · two-finger tap to right-click"
: pad.captured ? "Your mouse and keyboard now control the Frame. Press Esc to stop."
@@ -3964,7 +4130,6 @@ async function offerTest(m) {
// ---- privacy: anonymous analytics levels (ui/frame_telemetry.py, docs/privacy.md) ----
const telemetry = { usage: false, compat: false, blocked: "not loaded" };
let privacyNoticeShown = false; // this visit: then the contact prompt waits for another one
function renderTelemetry(s) {
Object.assign(telemetry, s);
setRepHint();
@@ -3975,7 +4140,6 @@ function renderTelemetry(s) {
: "Nothing sent yet.";
const showNotice = !s.blocked && !s.noticeShown && s.usage;
$("privacyNotice").hidden = !showNotice;
if (showNotice) privacyNoticeShown = true;
if (showNotice) api("/api/telemetry", { noticeShown: true }).catch(() => {});
}
async function loadTelemetry() {
@@ -4018,14 +4182,13 @@ async function loadContact() {
try { renderContact(await api("/api/contact")); } catch { return; }
checkContactPrompt();
}
// One time only, only once the Frame has connected, and never in a visit that showed the privacy
// notice (two asks in a row is nagging): checked at load and whenever the Frame connects.
// One time only, only once the Frame has connected, and never on top of the privacy notice or
// straight after it (two asks in a row is nagging): checked at load and whenever the Frame connects.
async function checkContactPrompt() {
await telemetryLoaded;
if (privacyNoticeShown || !$("contactNotice").hidden) return;
if (!$("contactNotice").hidden || !$("privacyNotice").hidden) return;
let s;
try { s = await api("/api/contact"); } catch { return; }
if (!s.showPrompt || privacyNoticeShown || !$("contactNotice").hidden) return;
if (!s.showPrompt || !$("contactNotice").hidden || !$("privacyNotice").hidden) return;
$("contactNotice").hidden = false;
api("/api/contact/prompt", { prompt: "shown" }).catch(() => {});
}
@@ -4093,7 +4256,6 @@ function openBugReport() {
// A standing yes to follow-up questions (Privacy & updates) fills this in; it can be unticked.
$("bugFollowup").checked = contact.followup; $("bugContact").value = contact.followup ? contact.email : "";
$("bugContact").disabled = !contact.followup; $("bugContact").required = contact.followup;
bugReplaces();
$("bugDlg").showModal();
loadBugPreview();
}
@@ -4104,17 +4266,7 @@ $("bugFollowup").onchange = () => {
const on = $("bugFollowup").checked;
$("bugContact").disabled = !on; $("bugContact").required = on;
if (on && !$("bugContact").value) { $("bugContact").value = contact.email; $("bugContact").focus(); }
bugReplaces();
};
// Sending with another address replaces the saved one (ui/frame_contact.py from_report): say so first.
function bugReplaces() {
const email = $("bugContact").value.trim(), old = contact.email;
const replaces = $("bugFollowup").checked && email && old && email.toLowerCase() !== old.toLowerCase();
$("bugReplaces").hidden = !replaces;
$("bugReplaces").textContent = !replaces ? "" : `Sending replaces ${old} as your contact email${contact.updates
? ", and update notices stop until you turn them on again in Privacy & updates" : ""}.`;
}
$("bugContact").oninput = bugReplaces;
$("bugCancel").onclick = () => $("bugDlg").close();
$("bugCopy").onclick = async () => {
const { title, body } = bugReportText();
@@ -4137,7 +4289,6 @@ $("bugForm").onsubmit = async e => {
$("bugMsg").textContent = `Couldn't send it: ${err.message}. Try again later, or use Copy report.`;
$("bugSend").disabled = false;
}
api("/api/contact").then(renderContact).catch(() => {}); // the report may have saved the address
};
if (window.frameApp && window.frameApp.onReportProblem) window.frameApp.onReportProblem(openBugReport);
@@ -4292,7 +4443,7 @@ if (window.frameApp && window.frameApp.onInstallLink) {
}
setView("headset");
refresh().then(loadShots); // after status, so app names resolve
refresh().then(() => loadShots()); // after status, so app names resolve
document.addEventListener("visibilitychange", () => { if (!document.hidden && online === false) refresh(); });
</script>
<script>
@@ -4413,7 +4564,7 @@ function onConnection(s) {
lastImg = null; lastShot = null;
$("canvas").hidden = true; $("viewerEmpty").hidden = false; $("zoombar").hidden = true;
["stamp", "srcBadge", "asleep"].forEach(id => $(id).hidden = true);
$("saveBtn").disabled = true;
$("saveBtn").disabled = $("copyBtn").disabled = true;
// Lists and their buttons (Remove, Launch…) were the other headset's: clear them
// before anyone clicks one, until the new headset's arrive.
["games", "titleList", "andApps", "flatpaks", "shotGrid", "gmGrid"].forEach(id => {
@@ -4422,7 +4573,7 @@ function onConnection(s) {
disp.list = []; disp.port = null;
// The lists behind those panels too, so filters can't bring the old ones back.
gm.owned = null; gm.byId = new Map(); gm.results = []; gm.store = []; gm.storeQ = null; gm.shown = 0; gm.seq++;
androidApps = []; shots.list = [];
androidApps = []; resetShots();
titlesSeq++; disp.seq++; // answers to loads already on their way are ignored
if (cat.apps) filterCatalog(); // "Installed" tags were the other headset's
// Confirmations still open were checked against the other headset.
+280 -9
View File
@@ -60,6 +60,7 @@ import frame_report # noqa: E402
import frame_store # noqa: E402
import frame_telemetry # noqa: E402
import frame_titles # noqa: E402
import frame_touch # noqa: E402
import frame_webinstall # noqa: E402
import frame_vr # noqa: E402
import frame_utilities # noqa: E402
@@ -134,7 +135,6 @@ LINK = None # the connector (frame_link.Link); None on the Frame itself
# install's clean-up) to the other headset.
_work_lock = threading.Lock()
_work = [0]
NOT_HEADSET_WORK = {"/api/devices", "/api/contact", "/api/contact/prompt"}
@contextlib.contextmanager
@@ -1006,12 +1006,274 @@ def licenses():
return notices
# KDE Connect's specialKey numbers -> Linux key codes (KEY_BACKSPACE, KEY_TAB, ...).
PAD_KEYS = {1: 14, 2: 15, 4: 105, 5: 103, 6: 106, 7: 108, 8: 104, 9: 109, 10: 102, 11: 107, 12: 28, 13: 111, 14: 1,
**{21 + i: code for i, code in enumerate([59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 87, 88])}}
PAD_MODS = (("ctrl", 29), ("alt", 56), ("shift", 42), ("super", 125))
PAD_CLICKS = {"singleclick": "left", "rightclick": "right", "middleclick": "middle"}
PAD_NOTCH = 15 # scroll units for one wheel notch
def typed_in_kde(events):
"""Whether these events must go the KDE Connect way: they hold an accent or emoji, or
earlier ones are queued, being sent or still being typed there (typing must stay in
order, so it can't be split between the two ways)."""
with _kde_lock:
waiting = bool(_kde_queue) or time.time() < _kde_until
return waiting or any(ch not in frame_touch.ASCII for e in events for ch in e.get("key", ""))
def pad_events(events, kde=False):
"""The trackpad's and key row's events as (gamescope events, KDE Connect events).
Gamescope's own input reaches every panel and needs nothing installed, but types only
US-keyboard characters. With kde, the keyboard goes to KDE Connect as it was sent, so
text stays in order; the pointer always goes through gamescope.
"""
ascii_keys = frame_touch.ASCII
out, slow = [], []
def tap(code, mods):
down = [{"key": c, "down": True} for c in mods]
out.extend(down + [{"key": code, "down": True}, {"key": code, "down": False}]
+ [{"key": c, "down": False} for c in reversed(mods)])
for e in events:
mods = [code for name, code in PAD_MODS if e.get(name)]
if "dx" in e or "dy" in e:
if not e.get("scroll"):
out.append({k: e[k] for k in ("dx", "dy") if k in e})
if e.get("scroll"):
out.append({"scroll": [0, -(e.get("dy") or 0) * PAD_NOTCH]})
for flag, button in PAD_CLICKS.items():
if e.get(flag):
out += [{"button": button, "down": True}, {"button": button, "down": False}]
if e.get("doubleclick"):
out += [{"button": "left", "down": d} for d in (True, False, True, False)]
if e.get("singlehold"):
out.append({"button": "left", "down": True})
if e.get("singlerelease"):
out.append({"button": "left", "down": False})
if kde and ("key" in e or "specialKey" in e):
slow.append({k: e[k] for k in ("key", "specialKey", "ctrl", "alt", "shift", "super") if k in e})
continue
if "specialKey" in e and e["specialKey"] in PAD_KEYS:
tap(PAD_KEYS[e["specialKey"]], mods)
if "key" in e:
for ch in e["key"]:
if ch not in ascii_keys:
continue # only reached when kde is wrong; the caller checks typed_in_kde first
elif mods:
code, shifted = ascii_keys[ch]
tap(code, mods + ([42] if shifted and 42 not in mods else []))
else:
if out and "text" in out[-1] and len(out[-1]["text"]) < 100:
out[-1]["text"] += ch
else:
out.append({"text": ch})
return out, slow
def remote_input(body):
"""{"events": [...]} sends keyboard and pointer events; {} (or none yet) just starts the agent."""
"""{"events": [...]} sends keyboard and pointer events; {} (or none yet) just starts the agent.
"sent" in the answer means the whole batch is taken (the page keeps it and tries again
if not), so the keyboard's share is queued only once that's so.
"""
events = body.get("events", [])
if not isinstance(events, list) or len(events) > INPUT_BATCH_LIMIT:
raise Failure(f"events must be a list of at most {INPUT_BATCH_LIMIT}", 400)
return _input.send([input_event(e) for e in events])
events = [input_event(e) for e in events]
touch, slow = pad_events(events, typed_in_kde(events))
status = _touch.send(touch)
if touch and not status.get("sent"):
return status
if touch:
_note_typed(touch)
if slow:
_queue_for_kde(slow)
return {**status, "sent": True}
# Typing KDE Connect is to do: it starts the first time it's needed, so this waits for it.
# Nothing on the Frame says when typed text has landed, so each way holds the other back
# for as long as what it was given should take (a short margin on a time estimate).
_kde_queue = []
_kde_lock = threading.Lock()
_kde_worker = [None]
_kde_until = 0.0 # keys typed through KDE Connect should have landed by then
_typed_until = 0.0 # when gamescope's agent will have typed everything it's been given
KDE_QUEUE_LIMIT = 200
KDE_WAIT = 90 # seconds to wait for KDE Connect before giving up on what's queued
TYPING_SETTLE = 0.6 # seconds past the estimate, once
def typing_seconds(touch):
"""How long gamescope's agent takes over these: it sleeps 8 ms after every key transition."""
transitions = 0
for e in touch:
if "text" in e:
transitions += sum(2 + 2 * frame_touch.ASCII[ch][1] for ch in e["text"] if ch in frame_touch.ASCII)
elif "key" in e:
transitions += 1
return transitions * 0.008
def _note_typed(touch):
global _typed_until
seconds = typing_seconds(touch)
if seconds:
with _kde_lock: # the end of the work, which queues up; the margin is added once, where it's checked
_typed_until = max(_typed_until, time.time()) + seconds
def _queue_for_kde(events):
with _kde_lock:
_kde_queue.extend(events)
del _kde_queue[:-KDE_QUEUE_LIMIT]
if _kde_worker[0] is None:
_kde_worker[0] = threading.Thread(target=_drain_kde, daemon=True)
_kde_worker[0].start()
def _drain_kde():
"""The only sender, in order. A batch leaves the queue while it's being sent (the queue
can be trimmed meanwhile) and goes back to the front if it didn't go."""
global _kde_until
deadline = time.time() + KDE_WAIT
while True:
with _kde_lock:
batch = _kde_queue[:INPUT_BATCH_LIMIT]
del _kde_queue[:len(batch)]
if not batch:
_kde_worker[0] = None # retired under the lock, so a new request starts another
return
wait = _typed_until + TYPING_SETTLE - time.time()
_kde_until = time.time() + 3600 # held while in flight
if wait > 0:
time.sleep(wait) # gamescope's typing, from just before, goes first
status = _input.send(batch)
with _kde_lock:
if status.get("sent"):
_kde_until = time.time() + len(batch) * 0.03 + TYPING_SETTLE
deadline = time.time() + KDE_WAIT
continue
_kde_until = 0.0
if status.get("state") == "error" or time.time() > deadline:
_kde_queue.clear() # not coming; don't hold it forever
_kde_worker[0] = None
return
_kde_queue[:0] = batch
del _kde_queue[:-KDE_QUEUE_LIMIT] # as when queuing: the oldest go first
time.sleep(0.4)
# ---- touch: the headset's panels, through gamescope's own input (frame_touch.py) ----
PANEL_WINDOW = re.compile(r"^\d{1,10}$")
PANEL_DISPLAY = re.compile(r"^:\d{1,2}$")
TOUCH_BUTTONS = ("left", "right", "middle")
def panels():
"""The headset's app panels (window, display, name, size) and which has focus."""
return json.loads(ssh("python3 - panels", stdin=(HERE / "frame_touch.py").read_text(), timeout=20))
def panel_target(q):
window, display = (q.get("window") or [""])[0], (q.get("display") or [""])[0]
if not PANEL_WINDOW.match(window) or not PANEL_DISPLAY.match(display):
raise Failure("window must be a window id and display an X display such as :1", 400)
return window, display
def panel_capture(query):
"""One frame of a panel's own window, as PNG (its pixels, without the room around it)."""
window, display = panel_target(parse_qs(query))
return ssh(f"DISPLAY={display} timeout 10 ffmpeg -hide_banner -loglevel error -nostdin -f x11grab "
f"-window_id {window} -i {display} -frames:v 1 -f image2pipe -c:v png -", timeout=20, text=False)
def touch_event(event):
"""A frame_touch.py event with only the fields it knows, in range."""
if not isinstance(event, dict):
raise Failure("each touch event must be an object", 400)
def num(name, limit):
value = event.get(name)
if isinstance(value, bool) or not isinstance(value, (int, float)) or value != value:
raise Failure(f"{name} must be a number", 400)
return max(-limit, min(limit, round(float(value), 4)))
out = {}
if "fx" in event or "fy" in event:
if "window" not in event:
raise Failure("a position needs the panel's window and display", 400)
out.update(fx=num("fx", 1), fy=num("fy", 1))
# Any event can name the panel it's meant for; the Frame drops it if another has focus.
if "window" in event:
window, display = event.get("window"), event.get("display")
if isinstance(window, bool) or not isinstance(window, int) or window <= 0:
raise Failure("window must be the panel's window id", 400)
if not isinstance(display, str) or not PANEL_DISPLAY.match(display):
raise Failure("display must be an X display such as :1", 400)
out.update(window=window, display=display)
for name in ("dx", "dy"):
if name in event:
out[name] = num(name, INPUT_MOVE_LIMIT)
if "button" in event:
if event["button"] not in TOUCH_BUTTONS:
raise Failure(f"button must be one of {', '.join(TOUCH_BUTTONS)}", 400)
out["button"], out["down"] = event["button"], event.get("down") is not False
if "scroll" in event:
sc = event["scroll"]
if not isinstance(sc, list) or len(sc) != 2:
raise Failure("scroll must be [dx, dy]", 400)
out["scroll"] = [num_value(v, 5000) for v in sc]
if "key" in event:
key = event["key"]
if isinstance(key, bool) or not isinstance(key, int) or not 0 < key < 768:
raise Failure("key must be a Linux key code", 400)
out["key"], out["down"] = key, event.get("down") is not False
if "text" in event:
text = event["text"]
if not isinstance(text, str) or not 0 < len(text) <= INPUT_TEXT_LIMIT:
raise Failure(f"text must be 1 to {INPUT_TEXT_LIMIT} characters", 400)
out["text"] = text
if not set(out) - {"window", "display"}:
raise Failure("touch event has nothing to do", 400)
return out
def num_value(value, limit):
if isinstance(value, bool) or not isinstance(value, (int, float)) or value != value:
raise Failure("scroll values must be numbers", 400)
return max(-limit, min(limit, round(float(value), 2)))
class TouchAgent(InputAgent):
"""frame_touch.py on the Frame, fed events over one long-lived ssh. Nothing to install:
it uses gamescope's own input socket and the libei that's on the image."""
def __init__(self):
super().__init__(source=HERE / "frame_touch.py", packages=[])
def deliver(self, report, force=False):
return ""
def command(self, folder=""):
code = base64.b64encode(self.source.read_bytes()).decode()
return "python3 -u -c " + shlex.quote(
f"import base64;exec(compile(base64.b64decode('{code}'),'frame_touch','exec'))")
_touch = TouchAgent()
def remote_touch(body):
"""{"events": [...]} points, clicks, scrolls and types into the focused panel."""
events = body.get("events", [])
if not isinstance(events, list) or len(events) > INPUT_BATCH_LIMIT:
raise Failure(f"events must be a list of at most {INPUT_BATCH_LIMIT}", 400)
return _touch.send([touch_event(e) for e in events])
# ---- touch: the headset's panels, through gamescope's own input (frame_touch.py) ----
@@ -1194,6 +1456,12 @@ def open_thing(body):
SHOTS_DIR.mkdir(parents=True, exist_ok=True)
frame_host.open_path(SHOTS_DIR)
return {"message": f"Opened {SHOTS_DIR} in {frame_host.FILE_MANAGER}"}
if what == "shot":
saved = SHOTS_DIR / shot_path(body.get("id")).rsplit("/", 1)[-1]
if not saved.exists():
raise Failure("That screenshot isn't saved on this computer yet", 404)
frame_host.reveal_path(saved)
return {"message": f"Showed {saved.name} in {frame_host.FILE_MANAGER}"}
except frame_host.HostError as e:
raise Failure(str(e), 500)
raise Failure("unknown target", 400)
@@ -2367,7 +2635,7 @@ class Handler(BaseHTTPRequestHandler):
elif path == "/api/touch":
self.send_json(_touch.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_touch.status))
elif path == "/api/input":
self.send_json(_input.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_input.status))
self.send_json(_touch.send([]) if parse_qs(url.query).get("start") == ["1"] else dict(_touch.status))
elif path == "/api/job":
self.send_json(job_status(url.query))
elif path == "/api/android/displays":
@@ -2441,8 +2709,7 @@ class Handler(BaseHTTPRequestHandler):
body = json.loads(self.rfile.read(length) or b"{}")
if not isinstance(body, dict):
raise Failure("request body must be a JSON object", 400)
# Not headset work: switching headsets mustn't wait for (or refuse) these.
with (contextlib.nullcontext() if path in NOT_HEADSET_WORK else working(meant)):
with (contextlib.nullcontext() if path == "/api/devices" else working(meant)):
result = handler(body)
self.send_json(result)
except Failure as e:
@@ -2663,12 +2930,16 @@ def main():
signal.signal(signal.SIGTERM, lambda *_: (_ for _ in ()).throw(KeyboardInterrupt))
if args.exit_on_eof:
def watch_stdin():
sys.stdin.buffer.read()
# os.read, not sys.stdin.buffer.read: a buffered read holds stdin's lock,
# and if a signal stops the server first, Python aborts (SIGABRT) at exit
# when it can't take that lock back from this thread.
while os.read(0, 4096):
pass
threading.Thread(target=httpd.shutdown, daemon=True).start()
threading.Thread(target=watch_stdin, daemon=True).start()
# The real port, which --port 0 leaves to the system (the iPhone app reads it from here).
print(f"Frame Control on http://127.0.0.1:{httpd.server_address[1]} (alias: {FRAME}; Ctrl-C to stop)", flush=True)
try:
# The real port, which --port 0 leaves to the system (the iPhone app reads it from here).
print(f"Frame Control on http://127.0.0.1:{httpd.server_address[1]} (alias: {FRAME}; Ctrl-C to stop)", flush=True)
httpd.serve_forever()
except KeyboardInterrupt:
pass