* Live view: a Desktop view that stays still, and Control to tap on the Frame
The live view gets a second source and a way to use the Frame from it:
- Desktop: the app panel in use in the headset, streamed from its own window
(x11grab of gamescope's redirected window), so it doesn't move as the
wearer looks around. A picker shows any other panel, view only.
- Control: on the Desktop view a tap or click lands exactly where you put it;
drag is a mouse drag, press and hold right-clicks, two fingers scroll, and
on a computer the mouse, wheel and keyboard work directly. On the headset
view the view is a trackpad. A text field and key row type from a phone.
Input goes through gamescope's own EIS socket (the way Steam feeds Remote
Play input) with the libei already on the image: ui/frame_touch.py, over
the same long-lived ssh machinery as the keyboard agent, nothing to
install. It reaches the panel that has focus on either X display, which
the KDE Connect route can't. Verified on the Frame and from the iPhone app
in the Simulator.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: fixes from review
- Keys held on the Frame are released with buttons when Control stops or
the view loses focus; keys for the Frame no longer trigger Frame Control's
own shortcuts.
- Taps only act when the picture on screen is the panel in use; positions,
presses, keys, text and scrolls name their panel (display and window: ids
repeat across :0 and :1, told apart by pid), and the Frame drops them if
focus has moved on. Releases always go.
- While connecting, a tap keeps its position; on an error only releases wait
and retries back off; trimming a long queue never drops a release.
- Lifting one of two scrolling fingers ends the scroll; a cancelled touch
isn't a tap; clicks and holds on the bars around the picture do nothing.
- A capture loop from before a Live restart can't stop the new video.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: close the targeting gaps from the second review
- The focused panel's display comes from GAMESCOPE_FOCUS_DISPLAY (gamescope
packs ":1" into the first value), so a window id repeated across :0 and :1
can't be mistaken; the pid is only the fallback.
- Presses, keys, text and scrolls read focus afresh on the Frame; only moves
use a reading up to a second old.
- A gesture remembers the panel it started on and does nothing more if that
stops being the one in use; a press with no panel to aim at isn't sent.
- Opening a screenshot clears the panel Control would act on; switching to
another app releases held keys and buttons.
- Trimming keeps a click with its position; the error backoff holds for new
input too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: fixes from the SWE-2 Max review
- The Frame side tracks keys as well as buttons and lets go of both when the
session ends.
- A stale tap tells the page, which re-reads the panels at once.
- A paused input device waits instead of ending the session; only a
disconnect does. An OS error on one event skips it.
- Presses check focus with two property reads and do the full lookup only
when it changed.
- Writes to an agent's stdin are serialized, so two devices sending at once
can't tear a line (the keyboard agent too).
- Connecting gives up with a message after 15 s instead of hanging on
"Connecting…"; text goes in 100-character pieces so releases don't wait
behind a long paste; a cancelled mouse gesture releases what's held.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: stale clears, pauses reconverge, pastes split per request
- The Frame says it has caught up as soon as an aimed event lands after a
stale one, so the page stops re-reading the panels.
- After a device pause it lets go of everything it holds (releases that
arrived while paused were dropped), and waits for the device once per
batch, not once per event.
- The quick focus check no longer freshens the panel geometry's age.
- Each request carries at most about 100 characters of text.
- Turning Control off while it connects doesn't report an error.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* frame_touch: build the socket path on the Frame, so Windows can import it for tests
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* frame_touch: any event that goes through clears the stale flag
A trackpad move names no panel, so waiting for an aimed event could leave
the page re-reading panels for the rest of the session; a release still
aimed at the old panel doesn't count.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Stop always calls systemctl and treats exit 5 (unit already collected)
as done, so there is no is-active/stop race. Cleanup never masks the
copy error, the play ssh timeout covers the remote worst case, and
tests cover stop exit codes and systemd-run stderr reporting.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Stop is a no-op when the collected player unit is already gone
(raw systemctl stop exits 5 on the Frame; verified 2026-09-29).
- Surface systemd-run stderr when the player can't start.
- Keep the copy error if the cleanup ssh also fails; reject upload
names that the play path can never accept.
- Allow 60 s for play (ffprobe 30 s + systemd-run 15 s remote).
- Docs: four-hour cap is unconditional; no delete action yet; fix a
garbled timing sentence.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
HTTPServer.server_bind calls socket.getfqdn, which stalled past the MCP
backend's 10-second startup window on GitHub's macOS runners.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Any unexpected error while launching clears the launch and reports it, so
the pad can always be started again; the temporary stderr file is made
inside the handled path and a failure reading it is tolerated.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
discard() swallows OSError as well as Failure, so a launch that fails and
can't remove its copy still reports the error and can be started again.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The agent holds its copy from its first status line on and tidies it up
however it ends. Before that (a launch error, or stopped before the agent
ran) the server removes the copy itself. discard() only ever removes
incoming copies, never the iPhone bundle's own. The retry race test waits
for both contenders' decisions instead of sleeping.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- A start turned off while copying removes the copy instead of starting an
agent that would be killed before it could tidy up.
- A local read error while copying removes the partial copy too.
- The retry race test holds the new start until the retry has decided, so
it fails every time without the fix (checked 3/3), not by luck.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Each start keeps its copy (holding a lock on it) until it ends, however
it ends, then removes it; a restart can still unpack it.
- The server removes a partial copy when copying fails.
- Other copies are removed only when unlocked and over an hour old.
- Tests: a start racing the need-packages retry (one agent, not two), a
restart that must unpack its copy, a held copy surviving the sweep.
Both regression tests fail without their fix.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The need-packages retry only runs if no start() has taken over, so two
agents can't end up running for one session.
- Each copy goes to its own incoming folder; the agent removes its own once
connected (and any a cancelled start left over an hour ago), so a stopped
start can't delete files another is copying or still needs.
- A missing package folder means need-packages, not an error.
- Tests keep fake agents running, so they check ready and which agent owns
the session, plus a bounded retry and the tidy rule.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- iPhone build fails if the bundle can't be made, instead of shipping a
stale archive with an empty version.
- A different build that another device is using right now is left running
and replaced once nobody is, rather than stopped under them.
- If what's installed changed after the server looked, the agent asks for
the packages (need-packages) and the server copies them and starts again.
- The installed-build check sends bytes, so Windows' CRLF can't break it.
- Starting again while a stopped start is still copying launches anew;
concurrent copies use their own temporary names.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The keyboard and trackpad no longer fetch KDE Connect from Valve's package
repository on the Frame. The desktop apps and the iPhone app's Frame bundle
carry Valve's arm64 build of kdeconnect 24.02.2-1 and the five libraries it
links (kcontacts, kpeople, modemmanager-qt, pulseaudio-qt, libfakekey),
pinned by SHA-256 in frame/kdeconnect/packages.json and downloaded at build
time from the kdeconnect-frame-24.02.2-1 release, which also holds Valve's
complete source package for each.
On first use the computer copies them over its SSH connection (the iPhone
bundle already has them on the Frame); the agent checks each SHA-256,
unpacks them and stamps which build it is, so later starts copy nothing.
No internet on the Frame, 3.6 MB instead of 8 MB, 18 MB unpacked instead of
82 MB (ModemManager and friends were packaging-only dependencies).
GPL/LGPL compliance: frame/kdeconnect/NOTICE.md names each exact version,
licence and source; per-project licence texts in frame/kdeconnect/LICENSES;
THIRD_PARTY_NOTICES.md; an About and licences dialog in the app.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Brings in #4, #5, #8 (fbl100's verified Remmina/VNC mirror), APK
alternatives and the website. docs/streaming.md: Mac in the headset stays
the recommendation (now verified on the Frame); Remmina keeps #8's verified
evidence as the whole-screen fallback. docs/mac-in-headset.md cites #8's
lag finding.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Home → Keyboard and trackpad, in every version of Frame Control (Mac,
Windows, Linux, iPhone, iPad) with nothing to install on the device in
your hand. On a phone: a trackpad (drag, tap, two-finger scroll and
right-click) and a field that types on the Frame. On a computer: click
the pad to pass the mouse and keyboard through; Esc to stop.
First-party route: ui/frame_input_agent.py runs on the Frame and talks
KDE Connect's LAN protocol (v7) to kdeconnectd as if it were a phone.
KDE Connect isn't on the image, but Valve's package repository has it;
the agent fetches it and four libraries into ~ (no root, survives
updates), starts it, pairs by itself (accepting over D-Bus), and stops
it again when the last device disconnects. Each device has its own
identity; a stuck KDE Connect is restarted once.
Verified against the real Frame (SteamOS 0.4.1): first-time install,
pairing, pointer moves from the Mac's server and the iPhone app
(Simulator), Mac and iPhone at once, two installs at once, a frozen
daemon replaced, and the daemon stopping when the app quits.
Reviewed by GPT-6 Astra (xhigh) over seven rounds; all findings fixed
except per-event delivery acknowledgement (documented known limit).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Flatpak installs record their outcome inside main's background job; failed
jobs are diagnostics too. The Privacy panel lives on the Tools page (#privacy
opens it), tab analytics use the four page names, and "Test it now?" reads the
install job's result.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Anonymous PostHog analytics (ui/frame_telemetry.py): usage on by default
after a first-run notice; compatibility results and error details opt-in,
offered together by the notice's "Share more to help fix problems" button.
Random id, no person profiles or GeoIP, scrubbed text, an offline outbox,
and "Show what's been sent" in the new Privacy panel. Inert without a
project key, from a source checkout, or with DO_NOT_TRACK=1.
- APK installs now record install_failed when the APK itself won't install,
and offer a 20-second test after installing. Opted-in reports reach the
shared database through PostHog and `frame_compat_db.py sync`.
- The desktop app updates itself from published releases (app/updater.js):
update.json from releases/latest/download, SHA-256 checked, no downgrades;
macOS bundle swap, Windows NSIS, Linux AppImage, otherwise the release page.
scripts/publish-release.sh publishes a tested draft with its manifest.
- Report a problem (header button, Privacy panel, Help menu) files a GitHub
issue through the website's feedback API, with a previewed, scrubbed
diagnostics snapshot; activity and logs only when asked for.
Reviewed by GPT-6 Astra (xhigh, read-only) three times; all findings fixed.
Docs: docs/privacy.md, docs/releasing.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Main now runs Android installs as background jobs and maps SSH failures to
one offline message. Alternative-version installs go through the same job,
the alternatives dialog waits on it with runJob, and send_error_json keeps
the apk blocker that opens the dialog.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
One malformed or unreachable repo no longer hides the others; skip bad
index entries; a refreshed raw index outdates its reduced copy; style the
dialog like the others; validate package ids with PKG_RE.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Frame Control can now show any Mac window, or a whole screen, as its own
SteamVR panel on the Steam Frame (Tools -> Mac in the headset, macOS only).
Place it with the SteamVR dashboard; the laser clicks and scrolls, and the
Mac's own keyboard types.
- mac/frame-mac-view (Swift, no dependencies): ScreenCaptureKit capture per
window or display, VideoToolbox H.264 with low-latency rate control (JPEG
fallback), a loopback HTTP/WebSocket server, CGEvent/AX input playback,
and a display-awake assertion while anyone watches.
- ui/frame_macview.py: starts the agent, runs an ssh -R tunnel with a
supervisor that reopens it on the same port, and launches a Chromium app
window per stream on gamescope's :0, tagged with STEAM_GAME for its own panel.
- Frame Control's key never leaves the Mac: viewers get single-use,
per-source tickets and reconnect keys that Stop revokes.
- ui/mac-view.html: WebCodecs decode, keyframe recovery, pointer/wheel/keys back.
- Bundled in the Mac app build; tests/test_macview.py builds and drives the
agent on macOS.
Verified on the Frame (build 20260925.6191901) with the test pattern: panel
in about 1.5 s, about 60 fps, Mac-to-window about 11-17 ms, tunnel recovery
in 4 s. Laser input and real window capture still need a person in the headset.
Also commits the other thread's first-party rule (steam-frame skill) and
the first-party options table in docs/streaming.md.
Reviewed by GPT-6 Astra (xhigh, read-only) over six rounds; all findings fixed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Valve publishes no Steam Frame OS image, so tests/frame-container builds the
Frame's SSH surface on Valve and Collabora's Holo Core aarch64 base: a
steamos user with a password and sudo, OpenSSH with keys and passwords,
Python, and a systemctl that only records requests.
Against it from the Simulator: password pairing, the host-key pin, the power
password check. Found and fixed: a changed host key or a refused login said
"Can't reach the Frame" and retried forever; they now say "Pair with the
Frame again" and offer that. The server's key rejection now says the header
may be wrong, not only missing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An iPhone can't run Python or ssh, but the Frame can. The app (ios/, SwiftUI)
connects with its own SSH key (Citadel), copies the server and helpers to
~/.cache/frame-control/<version> on the Frame once per version, starts
ui/server.py there with FRAME_LOCAL=1 on the Frame's 127.0.0.1, and shows the
page through an SSH tunnel. The server exits when the phone disconnects.
Server: FRAME_LOCAL=1 puts ui/local-bin on PATH, whose ssh stand-in runs each
`ssh frame COMMAND` locally (and serves as rsync's transport), so desktop and
phone share one code path. Android display goes through podman exec there, as
the Frame has no adb. FRAME_UI_KEY replaces the fixed X-Frame-UI value with a
per-session key. Power actions take the Developer Mode password via sudo -S.
--port 0 now prints the port it took.
Page: a bottom tab bar and safe areas on phones, Play buttons visible on touch
screens, saving through the share sheet, SSH/SFTP/Steam Link/remote desktop
opening in their iOS apps, and a password dialog for power.
App: pairing with the Developer Mode password once (never stored) or with a
key the user adds; host key pinned on first use; plain-language connection
errors with quiet retries; frame-control://install links; alerts and confirms.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The page was one 6,800px scroll with nine nav links (hidden below 1150px).
It is now four tabs, Home, Games, Android and Tools, switched with 1-4; old
section links still land on the right tab.
When the Frame can't be reached, the server turns ssh's connection errors into
one plain message (503, offline: true), the page shows a single banner with
Retry and Set Up Connection, retries every 8 s, and reloads every panel when
the Frame answers. Panels say "Waiting for the Frame" instead of raw ssh text.
Flatpak and Android catalogue installs run as background jobs the page polls,
so a slow install no longer holds a request for up to 15 minutes or reports a
false failure; the bottom bar counts running installs.
Files can be dropped anywhere in the window, as the README already said.
Recent reports show the newest five, with Show all. Android display explains
an empty or failed read. A topped-up headset on a charger reads as not
charging rather than "still draining, using 0.0 W".
Fixes a race where the catalogue and reports loads wrote the compat-db
mirror's .tmp file at once and one failed with a 500.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Links to localhost need FRAME_CONTROL_LOCAL_LINKS=1: otherwise any website's
link could make the app fetch from services on this computer.
- Title staging folders (unzipped titles) carry the server's PID and are swept
on the next start like download folders, so quitting mid-install doesn't leave
gigabytes behind.
- An install from a link refreshes Sideloaded titles.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Combines the three feature branches on bundle-deps. Conflicts in server.py,
index.html, preload.js, README and test_server.py keep both sides. The
web-install downloader now uses urllib's default HTTPS context, so the
bundled CA list from 770f26c applies to it on Windows too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Dropping a game's .zip, folder or .exe on Send to Frame now adds it to the
headset's Steam library through Valve's SteamOS Devkit title path, with the
runtime picked from the program's header: Windows PE -> Proton Experimental
(steam_play=1), aarch64 ELF -> SteamLinuxRuntime_4-arm64, x86-64 ELF ->
SteamLinuxRuntime_4 (through FEX). Other architectures are refused.
- frame/devkit-utils: Valve's devkit-utils vendored unmodified (MIT,
steamos-devkit v0.20260925.1), synced to ~/devkit-utils by stamp, bundled in
the app and compiled in CI.
- ui/frame_titles.py: inspect (safe unzip, ELF/PE classification, launch
target ranking), install(path, name=None, exe=None, runtime=None,
progress=None), list, launch, remove, plus a CLI.
- ui/server.py: /api/titles (inspect/install/discard/launch/remove),
/api/titles/job progress, and an upload mode 'title'.
- ui/index.html: confirm dialog (name, launch target, runtime), install
progress, and a Sideloaded titles list with Launch and Remove. The app's
preload passes a dropped folder's path.
- tests and docs/sideloading.md. Device-side behaviour is inferred from
Valve's source; the headset was offline, so none of it has been checked on
a Frame yet.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A site can link to frame-control://install?manifest=URL (or ?url=URL) to
install a title with Frame Control. Manifests use FrameDrop's format, so
framedrop.install/v1 is accepted as well as frame-control.install/v1.
- app/install-link.js parses links; main.js registers the scheme (plus
electron-builder protocols for Info.plist and the .desktop file), takes
links from open-url, second-instance argv and the first argv, and holds
them until the page asks for them through preload's onInstallLink.
- ui/frame_webinstall.py checks the URLs (HTTPS only; localhost over http
only when the link itself is local; no userinfo; every address public,
rechecked on redirects and pinned for the connection), reads the
manifest, downloads with a size cap and sha256 check, and dispatch()
sends .apk to frame_android and .zip/.exe to frame_titles when present.
- server.py adds /api/webinstall/check, start, job and cancel behind the
existing Host and X-Frame-UI guards; a start needs a one-time id from
check. Downloads stop on cancel and on shutdown, and leftovers from a
killed server are swept by PID.
- index.html asks before anything downloads (name, source host, file,
type, size, whether a sha256 was given) and shows progress.
- docs/web-install.md, docs/install.html (landing page, unpublished).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The bundled Python only trusts roots already in the Windows certificate
store, which Windows fills lazily, so on a new install Steam store
search, F-Droid downloads and the compat DB failed with
CERTIFICATE_VERIFY_FAILED. fetch-deps.js now also bundles curl's pinned
copy of Mozilla's CA list, and the server adds it to the default HTTPS
context on top of the system certificates (before any urlopen, since
urllib keeps the context it first builds).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Alex Southwell