Real-Frame testing (2026-09-29) found an unworn headset enters standby
within seconds; SetOverlayRaw then returns RequestFailed (23) and the
movie died. The player now drops frames during standby, keeps audio
and pacing, re-sends stills and the theatre surround after waking, and
only errors after five minutes without an accepted frame.
A Stop arriving while the player is already shutting down is ignored,
so a finished video stays 'ended' instead of 'error: Stopped'. The
status now reports the layout's real source (filename/metadata).
Docs record the end-to-end device matrix (API, web UI, CLI).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Live view: a Desktop view that stays still, and Control to tap on the Frame
The live view gets a second source and a way to use the Frame from it:
- Desktop: the app panel in use in the headset, streamed from its own window
(x11grab of gamescope's redirected window), so it doesn't move as the
wearer looks around. A picker shows any other panel, view only.
- Control: on the Desktop view a tap or click lands exactly where you put it;
drag is a mouse drag, press and hold right-clicks, two fingers scroll, and
on a computer the mouse, wheel and keyboard work directly. On the headset
view the view is a trackpad. A text field and key row type from a phone.
Input goes through gamescope's own EIS socket (the way Steam feeds Remote
Play input) with the libei already on the image: ui/frame_touch.py, over
the same long-lived ssh machinery as the keyboard agent, nothing to
install. It reaches the panel that has focus on either X display, which
the KDE Connect route can't. Verified on the Frame and from the iPhone app
in the Simulator.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: fixes from review
- Keys held on the Frame are released with buttons when Control stops or
the view loses focus; keys for the Frame no longer trigger Frame Control's
own shortcuts.
- Taps only act when the picture on screen is the panel in use; positions,
presses, keys, text and scrolls name their panel (display and window: ids
repeat across :0 and :1, told apart by pid), and the Frame drops them if
focus has moved on. Releases always go.
- While connecting, a tap keeps its position; on an error only releases wait
and retries back off; trimming a long queue never drops a release.
- Lifting one of two scrolling fingers ends the scroll; a cancelled touch
isn't a tap; clicks and holds on the bars around the picture do nothing.
- A capture loop from before a Live restart can't stop the new video.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: close the targeting gaps from the second review
- The focused panel's display comes from GAMESCOPE_FOCUS_DISPLAY (gamescope
packs ":1" into the first value), so a window id repeated across :0 and :1
can't be mistaken; the pid is only the fallback.
- Presses, keys, text and scrolls read focus afresh on the Frame; only moves
use a reading up to a second old.
- A gesture remembers the panel it started on and does nothing more if that
stops being the one in use; a press with no panel to aim at isn't sent.
- Opening a screenshot clears the panel Control would act on; switching to
another app releases held keys and buttons.
- Trimming keeps a click with its position; the error backoff holds for new
input too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: fixes from the SWE-2 Max review
- The Frame side tracks keys as well as buttons and lets go of both when the
session ends.
- A stale tap tells the page, which re-reads the panels at once.
- A paused input device waits instead of ending the session; only a
disconnect does. An OS error on one event skips it.
- Presses check focus with two property reads and do the full lookup only
when it changed.
- Writes to an agent's stdin are serialized, so two devices sending at once
can't tear a line (the keyboard agent too).
- Connecting gives up with a message after 15 s instead of hanging on
"Connecting…"; text goes in 100-character pieces so releases don't wait
behind a long paste; a cancelled mouse gesture releases what's held.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Control: stale clears, pauses reconverge, pastes split per request
- The Frame says it has caught up as soon as an aimed event lands after a
stale one, so the page stops re-reading the panels.
- After a device pause it lets go of everything it holds (releases that
arrived while paused were dropped), and waits for the device once per
batch, not once per event.
- The quick focus check no longer freshens the panel geometry's age.
- Each request carries at most about 100 characters of text.
- Turning Control off while it connects doesn't report an error.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* frame_touch: build the socket path on the Frame, so Windows can import it for tests
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* frame_touch: any event that goes through clears the stale flag
A trackpad move names no panel, so waiting for an aimed event could leave
the page re-reading panels for the rest of the session; a release still
aimed at the old panel doesn't count.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Stop always calls systemctl and treats exit 5 (unit already collected)
as done, so there is no is-active/stop race. Cleanup never masks the
copy error, the play ssh timeout covers the remote worst case, and
tests cover stop exit codes and systemd-run stderr reporting.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Stop is a no-op when the collected player unit is already gone
(raw systemctl stop exits 5 on the Frame; verified 2026-09-29).
- Surface systemd-run stderr when the player can't start.
- Keep the copy error if the cleanup ssh also fails; reject upload
names that the play path can never accept.
- Allow 60 s for play (ffprobe 30 s + systemd-run 15 s remote).
- Docs: four-hour cap is unconditional; no delete action yet; fix a
garbled timing sentence.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Any unexpected error while launching clears the launch and reports it, so
the pad can always be started again; the temporary stderr file is made
inside the handled path and a failure reading it is tolerated.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
discard() swallows OSError as well as Failure, so a launch that fails and
can't remove its copy still reports the error and can be started again.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The agent holds its copy from its first status line on and tidies it up
however it ends. Before that (a launch error, or stopped before the agent
ran) the server removes the copy itself. discard() only ever removes
incoming copies, never the iPhone bundle's own. The retry race test waits
for both contenders' decisions instead of sleeping.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- A start turned off while copying removes the copy instead of starting an
agent that would be killed before it could tidy up.
- A local read error while copying removes the partial copy too.
- The retry race test holds the new start until the retry has decided, so
it fails every time without the fix (checked 3/3), not by luck.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Review round 12: a failed USB-C tunnel now retries the normal path; an
existing HostKeyAlias wins; --host with --usb is rejected; the Steam
desktop-streaming claim is now 'untested' (Valve documents the desktop
showing when a game loses focus); the Show/cleanup overlap test blocks
for real (it fails without the lock). Verified live: with the cable out,
the route is the normal path.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Each start keeps its copy (holding a lock on it) until it ends, however
it ends, then removes it; a restart can still unpack it.
- The server removes a partial copy when copying fails.
- Other copies are removed only when unlocked and over an hour old.
- Tests: a start racing the need-packages retry (one agent, not two), a
restart that must unpack its copy, a held copy surviving the sweep.
Both regression tests fail without their fix.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Plugged into the Mac, the Frame is a USB network device ("Steam Frame",
usb0 at ~0.9 ms). The tunnel uses it when the Frame's usb0 answers,
with the usual host key; otherwise the normal path. Interleaved runs:
content p50 7 vs 10 ms, click to drawn 17 vs 27 ms, scroll p95 23 vs
31-37 ms. FRAME_MACVIEW_USB=0 turns it off; the card says "over USB-C".
- Bench: --usb, and the route is recorded per run.
- Docs: Steam's own streaming (no SteamVR host on macOS; Remote Play pairs
but streams games, not windows; test blocked); the Frame's USB network;
the 2026-09-28 health-check boot-loop recurrence.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The need-packages retry only runs if no start() has taken over, so two
agents can't end up running for one session.
- Each copy goes to its own incoming folder; the agent removes its own once
connected (and any a cancelled start left over an hour ago), so a stopped
start can't delete files another is copying or still needs.
- A missing package folder means need-packages, not an error.
- Tests keep fake agents running, so they check ready and which agent owns
the session, plus a bounded retry and the tidy rule.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- iPhone build fails if the bundle can't be made, instead of shipping a
stale archive with an empty version.
- A different build that another device is using right now is left running
and replaced once nobody is, rather than stopped under them.
- If what's installed changed after the server looked, the agent asks for
the packages (need-packages) and the server copies them and starts again.
- The installed-build check sends bytes, so Windows' CRLF can't break it.
- Starting again while a stopped start is still copying launches anew;
concurrent copies use their own temporary names.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Review round 11: the cleanup's check and pkill now hold a lock that Show
takes to count itself in, so a new viewer can't start between them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Review round 10: a Show replacing its own stream could have its new viewer
ended by the cleanup; a viewer reset left the delayed relay pending.
Verified: the relay delivers what's queued, then closes the agent side.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Chromium on the Frame outlived its last viewer window (verified: 11
processes left after a run). Once nothing is shown, Stop ends it, unless
Show was pressed again meanwhile; its profile is Frame Control's own.
- Relay --delay: if the agent side fails, close the viewer side too
(review round 9).
- Docs: the final scroll run captured 57 fps; don't blame ScreenCaptureKit.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The keyboard and trackpad no longer fetch KDE Connect from Valve's package
repository on the Frame. The desktop apps and the iPhone app's Frame bundle
carry Valve's arm64 build of kdeconnect 24.02.2-1 and the five libraries it
links (kcontacts, kpeople, modemmanager-qt, pulseaudio-qt, libfakekey),
pinned by SHA-256 in frame/kdeconnect/packages.json and downloaded at build
time from the kdeconnect-frame-24.02.2-1 release, which also holds Valve's
complete source package for each.
On first use the computer copies them over its SSH connection (the iPhone
bundle already has them on the Frame); the agent checks each SHA-256,
unpacks them and stamps which build it is, so later starts copy nothing.
No internet on the Frame, 3.6 MB instead of 8 MB, 18 MB unpacked instead of
82 MB (ModemManager and friends were packaging-only dependencies).
GPL/LGPL compliance: frame/kdeconnect/NOTICE.md names each exact version,
licence and source; per-project licence texts in frame/kdeconnect/LICENSES;
THIRD_PARTY_NOTICES.md; an About and licences dialog in the app.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Brings in #4, #5, #8 (fbl100's verified Remmina/VNC mirror), APK
alternatives and the website. docs/streaming.md: Mac in the headset stays
the recommendation (now verified on the Frame); Remmina keeps #8's verified
evidence as the whole-screen fallback. docs/mac-in-headset.md cites #8's
lag finding.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Per-frame timing on the Mac's clock (capture, encode, network, decode,
draw), viewer clock sync and reports, input echo, /stats and a HUD.
- scripts/macview-bench.py: repeatable runs on the real Frame, a shaping
relay (no sudo), interleaved A/B between agent settings; results in
bench/results/.
- Adaptive controller: ack-based send gate with jitter-aware slack, AIMD
bitrate that knows when a stream is app-limited, fps then size tiers.
On a 50->3->50 Mbit/s step, scroll p95 went from 4.7 s to 72 ms; no cost
on a clean link.
- Separate mode: real AppKit event loop (HiDPI and NSScreen now work),
cropped capture for fixed-size windows, windows kept on their display,
graceful quit restores windows; stop/start races fixed.
- Encoder timeline clamp (no oversized frame after a pause).
- Frame Control shows each live stream's fps, delay, bitrate and tier.
Reviewed by GPT-6 Astra xhigh (read-only), 7 rounds; findings fixed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Second review follow-up: with several activities (e.g. a splash activity ahead
of the game), the alias fallback now prefers the real activity named by the
alias's android:targetActivity. Reads targetActivity by resource id, updates
the error text and docs/vr-apks.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Review follow-up. inspect() now returns 'repairable' and the filters it can
patch: the real activity's VR MAIN filter, or, when LAUNCHER/VR sits only on an
<activity-alias>, any real MAIN filter with a category to copy. install() and
patch() repair on 'repairable' instead of 'vr_activity', so a flat Godot 4
export is fixed and a VR category only on the alias no longer aborts install.
Tests cover both shapes, an already-launchable activity with an alias, and an
end-to-end patch.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On a maintainer's Mac the compatibility-database key is in the Keychain, so a
test that reached install reporting published fake reports. Every test module
now imports tests/sandbox.py first, which points app data at a throwaway
directory (new FRAME_CONTROL_DATA_DIR), turns telemetry off and sends the
database nowhere.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Lepton's apk-info-extractor ignores <activity-alias>, and Godot 4 exports put
LAUNCHER only on an alias (com.godot.game.GodotAppLauncher). Open Saber Plus
0.7.67 installed but Lepton exited with 'APP_ACTIVITY is empty'. Count only
real activities as launchable and patch the activity's VR intent filter.
Verified on the Frame: Open Saber Plus launches and renders.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Analytics go to the maintainer's PostHog US project 343535, tagged
$lib = frame-control. Every event carries $ip 0.0.0.0, since PostHog
stores the sender's address otherwise (checked live), including events
queued by earlier versions.
- Report a problem sends a private problem_report event to PostHog instead
of a public GitHub issue, with its own random id so a contact address
can't be linked to analytics. The dialog asks how to reach the person and
shows a reference. Maintainers read reports on the PostHog dashboard or
with `python3 ui/frame_report.py inbox`.
- Community sync pages by timestamp in UTC: PostHog refuses OFFSET for
personal API keys.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
