Compare commits

...
Author SHA1 Message Date
saphidandClaude Opus 5.5 a8d5c4b44d Panels: one deadline for the whole run; read gamescope's properties once
Round-2 review of #83: the display search ran before the usual 5 s property
and geometry reads, and its 30 s memory never helped because the server
starts a fresh `python3 - panels` per request. With X tools answering in
4.6 s the run passed the server's 20 s.

- One xprop call reads all of gamescope's root properties; :0 is read first
  and, when it has them, used with no other probe. Other displays are only
  asked (2 s each) if :0 has none.
- focus() and focusable() reuse that reading instead of re-reading.
- A `panels` run has one deadline (12 s); every xprop/xwininfo is cut to
  what's left, and past it the run returns what it found with a "why"
  saying the headset was too slow.
- The display memory is documented as being for the long-lived input helper
  only.
- Tests: `frame_touch.py panels` run as a fresh process with fake xprop and
  xwininfo on PATH (lists panels; 4.6 s tools return in about 12 s, where
  the previous head ran past 60 s), plus unit tests for the single read,
  probing, the deadline and partial results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 14:36:53 +11:00
saphid 4e374de040 Merge remote-tracking branch 'origin/main' into fix/no-app-panel-73
# Conflicts:
#	ui/index.html
2026-10-12 14:20:08 +11:00
saphidandClaude Opus 5.5 62d233f9c9 Panels: bound the gamescope display search; show a failed first listing in Control
Review of #83: the display search shares one 6 s deadline (2 s per display)
and is remembered (found for good, not found for 30 s), so a stalled display
can't use up the server's 20 s. Control now shows a failed panel listing
instead of "Finding the headset's panels..." when the first one fails.
Page test (tests/page/desk_panels.mjs) covers the Desktop listing: the
headset's reason, initial failure and recovery.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 14:10:13 +11:00
saphidandClaude Opus 5.5 fcf2d12411 Desktop view: say why no panel was found; desktop-closed clipboard is a 409
Issue #73 (SteamOS 0.4.5): Capture on the Desktop view failed with "no app
panel is open in the headset" while the Plasma desktop was plainly open.
frame_touch.py's panels() came back empty and the page only knew to say
"none open".

- frame_touch.py: read gamescope's properties from :0 first, probing the
  other displays if :0 has none; list displays from /proc/net/unix's
  abstract X sockets as well as /tmp/.X11-unix; when nothing is found,
  return a "why" (X tools missing, listed windows that aren't X11 windows,
  desktop running but not listed, or nothing open).
- index.html: the Desktop view's Capture and Control show that reason, or
  the listing error, instead of a bare "no app panel".
- server.py: /api/clipboard answers 409 when the headset's desktop isn't
  open (plasmashell not running), so it no longer reaches telemetry as an
  $exception; other paste failures stay 502.

Fixes #73

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 13:59:30 +11:00
7 changed files with 573 additions and 49 deletions

No files matched your search

+10
View File
@@ -261,6 +261,16 @@ How (**verified 2026-09-29**, SteamOS 0.4.1, build 20260925.6191901):
-window_id <window> -i :<display>` works on gamescope's redirected windows,
while grabbing the root gives black. It streams as H.264 like the headset view
(about 30 fps at 720p).
- **When no panel is found**, `frame_touch.py panels` says why (`"why"`), and the
Desktop view shows that instead of a bare "no app panel": X tools missing,
gamescope listing windows that aren't X11 windows on any display, the desktop
running but not listed, or nothing open. It reads gamescope's properties from
`:0` first (one `xprop` call) and probes the other displays, 2 s each, only if `:0`
has none. A whole run stops after 12 s, inside the server's 20 s, and says the headset
was too slow. Added for
[#73](https://github.com/saphid/frame-control/issues/73) (SteamOS 0.4.5, desktop
open but not found). The likely cause there was the page itself: a second
`loadPanels` (the panel switcher's) replaced the Desktop view's, fixed in #84.
- Tested from the iPhone app (Simulator): a tap on the Desktop view focused a
text box in the panel and the text field typed into it; a trackpad move went
exactly (+40, +25).
+82
View File
@@ -0,0 +1,82 @@
// The live view's Desktop panel list (#73), run in node against the real functions from
// ui/index.html with a fake server: the headset's reason when it lists no panels, a failed
// listing shown (first time too) rather than "Finding…", and recovery after it.
import { readFileSync } from "fs";
const src = readFileSync(new URL("../../ui/index.html", import.meta.url), "utf8");
const grab = name => {
const m = src.match(new RegExp(`(?:\\nconst ${name} = [^\\n]*\\n)|((?:async )?function ${name}\\([\\s\\S]*?\\n}\\n)`));
if (!m) throw new Error("not found: " + name);
return m[0];
};
// The page's scripts share one global scope, so a second loadPanels would silently replace
// the Desktop view's (what broke #73 until #84): the one grabbed here must be the only one.
for (const name of ["loadPanels", "noPanelWhy", "deskPanel", "ctrlTarget"]) {
const count = (src.match(new RegExp(`\\n(?:async )?function ${name}\\(`, "g")) || []).length;
if (count !== 1) { console.log(`FAIL ${name} is defined ${count} times`); process.exit(1); }
}
const code = ["desk", "panelKey", "noPanelWhy", "deskPanel", "loadPanels", "ctrlTarget"].map(grab).join("");
const fail = msg => { console.log("FAIL " + msg); process.exit(1); };
function page(answer) {
const calls = [], logged = [];
const env = {
view: "flat", live: false, ctrlMode: () => "abs", renderPanelPick: () => {}, toggleLive: () => {}, ctrlShow: () => {},
log: (m) => logged.push(m),
api: async (path, body) => { calls.push([path, body]); return answer(); },
};
const fns = new Function(...Object.keys(env), `${code}
return { desk, noPanelWhy, loadPanels, ctrlTarget };`)(...Object.values(env));
return { ...fns, calls, logged };
}
// The Desktop view asks for the headset's panels with a plain GET (no switcher action).
{
const p = page(() => ({ panels: [], focus: null, why: "The headset's desktop is running, but SteamOS isn't listing it as a panel." }));
if (p.ctrlTarget().why !== "Finding the headset's panels…") fail("before loading: " + p.ctrlTarget().why);
await p.loadPanels();
if (!(p.calls.length === 1 && p.calls[0][0] === "/api/panels" && p.calls[0][1] === undefined))
fail("request " + JSON.stringify(p.calls));
if (!p.desk.loaded) fail("not marked loaded");
if (!p.noPanelWhy().startsWith("The headset's desktop is running")) fail("why for capture: " + p.noPanelWhy());
if (!p.ctrlTarget().why.startsWith("The headset's desktop is running")) fail("why for control: " + p.ctrlTarget().why);
}
// An older headset script without "why": the generic advice.
{
const p = page(() => ({ panels: [], focus: null }));
await p.loadPanels();
if (!p.noPanelWhy().startsWith("No app panel is open in the headset. Open the desktop")) fail("generic: " + p.noPanelWhy());
}
// The first listing fails: Control and Capture say so (logged once), then recover.
{
let broken = true;
const p = page(() => {
if (broken) throw new Error("Timed out talking to frame");
return { panels: [{ window: 7, display: ":0", name: "Desktop" }], focus: 7, focus_display: ":0" };
});
await p.loadPanels(); await p.loadPanels();
const why = p.ctrlTarget().why;
if (why !== "Couldn't list the headset's panels: Timed out talking to frame") fail("initial failure in Control: " + why);
if (p.noPanelWhy() !== why) fail("initial failure in Capture: " + p.noPanelWhy());
if (p.logged.length !== 1) fail("logged " + p.logged.length + " times");
broken = false;
await p.loadPanels();
if (p.desk.error) fail("error kept after recovery");
const t = p.ctrlTarget();
if (!t.why || t.why.includes("Couldn't list")) fail("after recovery: " + JSON.stringify(t));
if (p.desk.panels.length !== 1) fail("panels after recovery");
}
// Waiting for the Frame: shown in Control, not logged every poll. An answer about the
// previous headset (stale) is ignored.
{
let err;
const p = page(() => { throw err; });
err = Object.assign(new Error("Can't reach the Frame"), { offline: true });
await p.loadPanels(); await p.loadPanels();
if (p.logged.length) fail("offline logged: " + p.logged);
if (p.ctrlTarget().why !== "Couldn't list the headset's panels: Can't reach the Frame") fail("offline: " + p.ctrlTarget().why);
const q = page(() => { throw Object.assign(new Error("old headset"), { offline: true, stale: true }); });
await q.loadPanels();
if (q.logged.length || q.desk.error) fail("stale answer used: " + q.desk.error);
if (q.ctrlTarget().why !== "Finding the headset's panels…") fail("stale: " + q.ctrlTarget().why);
}
console.log("ok");
+41
View File
@@ -551,6 +551,47 @@ class UnreachableMessages(unittest.TestCase):
self.assertEqual(self.server.error_body(raw), ({"error": raw}, None), raw)
class ClipboardDesktopClosed(unittest.TestCase):
"""The headset's desktop being closed is the person's to fix: a 409, not a 5xx fault for telemetry."""
@classmethod
def setUpClass(cls):
sys.path.insert(0, str(ROOT / "ui"))
import server
cls.server = server
def paste_failing_with(self, message):
s = self.server
saved = s.ssh
def fake_ssh(remote, **kw):
raise s.Failure(message)
s.ssh = fake_ssh
try:
s.clipboard({"text": "hello"})
finally:
s.ssh = saved
def test_desktop_closed_is_a_409(self):
with self.assertRaises(self.server.Failure) as cm:
self.paste_failing_with("The headset's desktop isn't open (plasmashell is not running). "
"Open the desktop in the headset, then try again.")
self.assertEqual(cm.exception.status, 409)
self.assertIn("Open the desktop", str(cm.exception))
# Older wording (scripts/paste-to-frame.sh) is recognised too.
with self.assertRaises(self.server.Failure) as cm:
self.paste_failing_with("plasmashell is not running: open the desktop in the headset first.")
self.assertEqual(cm.exception.status, 409)
def test_other_failures_stay_faults(self):
with self.assertRaises(self.server.Failure) as cm:
self.paste_failing_with("Klipper call failed (bus: none)")
self.assertEqual(cm.exception.status, 502)
def test_paste_script_says_it(self):
self.assertIn(self.server.DESKTOP_CLOSED, self.server.PASTE)
class StatusProbe(unittest.TestCase):
# frame_status.py only ever runs on the Frame (Linux); it needs os.statvfs.
@unittest.skipIf(os.name == "nt", "Frame-side script; POSIX only")
+243 -5
View File
@@ -2,9 +2,14 @@
Run: python3 -m unittest discover -s tests
"""
import io
import json
import os
import shutil
import subprocess
import sys
import tempfile
import time
import unittest
from pathlib import Path
@@ -51,12 +56,9 @@ class Mapping(unittest.TestCase):
self.assertEqual(t.locate(5, None)["display"], ":0")
def test_focus_display_is_decoded(self):
t = self.t
saved = t.xprop_root
self.addCleanup(lambda: setattr(t, "xprop_root", saved))
for values, want in (([12602, 0, 58], ":1"), ([12346], ":0"), ([], None), ([0x41], None)):
t.xprop_root = lambda d, n, v=values: v
self.assertEqual(t.focus_display(), want)
self.assertEqual(self.t.focus_display({"GAMESCOPE_FOCUS_DISPLAY": values}), want)
self.assertIsNone(self.t.focus_display({}))
def test_ascii_table_covers_printable_characters(self):
for code in range(0x20, 0x7F):
@@ -259,6 +261,236 @@ class ServerChecks(unittest.TestCase):
self.s.remote_touch({"events": [{"dx": 1}] * (self.s.INPUT_BATCH_LIMIT + 1)})
class FindingPanels(unittest.TestCase):
"""panels() with faked headset probes: where gamescope's properties are, and why nothing was found (#73)."""
@classmethod
def setUpClass(cls):
import frame_touch
cls.t = frame_touch
def setUp(self):
t = self.t
names = ("read_props", "window_info", "window_pid", "displays", "desktop_running")
saved = {n: getattr(t, n) for n in names}
saved_which = t.shutil.which
self.real_displays = saved["displays"]
self.real_read_props = saved["read_props"]
def restore():
for n, v in saved.items():
setattr(t, n, v)
t.shutil.which = saved_which
t._ROOT.clear()
self.addCleanup(restore)
t._ROOT.clear()
self.props = {} # (display, property) -> values
self.windows = {} # (display, window) -> info
self.reads = [] # displays whose gamescope properties were read
t.read_props = lambda d, limit=5.0: self.reads.append(d) or {n: list(v) for (dd, n), v in self.props.items() if dd == d}
t.window_info = lambda d, w: self.windows.get((d, w))
t.window_pid = lambda d, w: None
t.displays = lambda: [":0", ":1", ":2"]
t.desktop_running = lambda: False
t.shutil.which = lambda tool: "/usr/bin/" + tool
def plasma_on(self, display):
# SteamOS 0.4.1: the nested Plasma desktop (kwin_wayland, app id 0) is a window on :0.
self.props[(display, "GAMESCOPE_FOCUSABLE_WINDOWS")] = [4194307, 0, 2222]
self.props[(display, "GAMESCOPE_FOCUSED_WINDOW")] = [4194307]
self.props[(display, "GAMESCOPE_FOCUS_DISPLAY")] = [0x303A] # ":0"
self.windows[(":0", 4194307)] = {"name": "KDE Wayland Compositor", "x": 0, "y": 0, "width": 1280, "height": 800}
self.windows[(":0", "root")] = {"width": 1280, "height": 720}
def test_desktop_panel_on_display_0(self):
self.plasma_on(":0")
out = self.t.panels()
self.assertEqual([(p["window"], p["display"], p["focused"]) for p in out["panels"]], [(4194307, ":0", True)])
self.assertNotIn("why", out)
def test_display_0_costs_one_read(self):
# The layout of every build seen so far: one read of :0's properties for the whole run.
self.plasma_on(":0")
self.t.panels()
self.assertEqual(self.reads, [":0"])
def test_gamescope_properties_on_another_display_are_found(self):
# A build that numbers its displays differently: :0 has no gamescope properties.
self.plasma_on(":1")
self.assertEqual(self.t.gamescope()[0], ":1")
self.reads.clear()
self.t._ROOT.clear()
self.assertEqual(len(self.t.panels()["panels"]), 1)
self.assertEqual(self.reads, [":0", ":1"]) # read once, then reused by focus() and focusable()
def test_other_displays_are_probed_briefly(self):
limits = []
self.t.read_props = lambda d, limit=5.0: limits.append((d, limit)) or {}
self.assertEqual(self.t.gamescope(), (":0", {}))
self.assertEqual(limits, [(":0", 5.0), (":1", self.t.ROOT_PROBE_TIMEOUT), (":2", self.t.ROOT_PROBE_TIMEOUT)])
def test_input_helper_remembers_the_display(self):
# The input helper lives for a session: a found display is kept, a failed search retried later.
t = self.t
self.plasma_on(":1")
t.gamescope(); t.gamescope()
self.assertEqual(self.reads, [":0", ":1", ":1"])
self.props.clear()
self.reads.clear()
t._ROOT.clear()
t.gamescope(); t.gamescope()
self.assertEqual(self.reads, [":0", ":1", ":2", ":0"]) # given up: :0 for ROOT_RETRY
t._ROOT[1] -= t.ROOT_RETRY
self.reads.clear()
t.gamescope()
self.assertEqual(self.reads, [":0", ":1", ":2"])
def test_read_props_parses_xprop(self):
t = self.t
saved = t.subprocess.run
self.addCleanup(setattr, t.subprocess, "run", saved)
out = ("GAMESCOPE_FOCUSABLE_WINDOWS(CARDINAL) = 1, 0, 2, 3, 769, 4\n"
"GAMESCOPE_FOCUSED_WINDOW(CARDINAL) = 0\nGAMESCOPE_FOCUS_DISPLAY: not found.\n")
t.subprocess.run = lambda *a, **k: subprocess.CompletedProcess(a, 0, out, "")
self.assertEqual(self.real_read_props(":0"), {"GAMESCOPE_FOCUSABLE_WINDOWS": [1, 0, 2, 3, 769, 4],
"GAMESCOPE_FOCUSED_WINDOW": [0]})
t.subprocess.run = lambda *a, **k: subprocess.CompletedProcess(a, 0, "GAMESCOPE_FOCUSED_WINDOW: not found.\n", "")
self.assertEqual(self.real_read_props(":0"), {})
def hang(*a, **k):
raise subprocess.TimeoutExpired(a[0], k.get("timeout"))
t.subprocess.run = hang
self.assertEqual(self.real_read_props(":0"), {})
def test_deadline_returns_what_was_found(self):
t, clock = self.t, [0.0]
self.plasma_on(":0")
self.props[(":0", "GAMESCOPE_FOCUSABLE_WINDOWS")] = [4194307, 0, 2222, 5, 0, 1, 6, 0, 1]
saved = t.time.monotonic
self.addCleanup(setattr, t.time, "monotonic", saved)
t.time.monotonic = lambda: clock[0]
info = self.windows.get
def slow(d, w): # every xwininfo takes 1 s: the run is out of time after a few
t.budget()
clock[0] += 1
return info((d, w))
t.window_info = slow
out = t.panels(limit=7) # focus: 2 reads; the first panel: 3 displays and its root
self.assertEqual([p["window"] for p in out["panels"]], [4194307])
self.assertIn("took too long", out["why"])
self.assertIn("Some may be missing", out["why"])
self.assertIsNone(t.DEADLINE[0]) # the next run (or the input helper) has no deadline left over
def test_listed_window_that_isnt_x11_explains_itself(self):
self.plasma_on(":0")
del self.windows[(":0", 4194307)] # say, a desktop drawn as a Wayland window
out = self.t.panels()
self.assertEqual(out["panels"], [])
self.assertIn("can't find its window", out["why"])
self.assertIn("Headset view", out["why"])
def test_desktop_running_but_not_listed(self):
self.t.desktop_running = lambda: True
out = self.t.panels()
self.assertEqual(out["panels"], [])
self.assertIn("desktop is running", out["why"])
def test_nothing_open(self):
out = self.t.panels()
self.assertEqual(out, {"focus": None, "focus_display": None, "panels": [],
"why": "No app panel is open in the headset. Open the desktop or an app in "
"the headset, then try again."})
def test_missing_x_tools_are_named(self):
self.plasma_on(":0")
self.t.shutil.which = lambda tool: None if tool == "xwininfo" else "/usr/bin/" + tool
self.windows.clear() # xwininfo can't run, so nothing resolves
self.assertIn("no xwininfo", self.t.panels()["why"])
def test_displays_include_abstract_sockets(self):
# /tmp/.X11-unix not visible to the SSH session: the abstract sockets still name the displays.
import builtins
t = self.t
real_open, real_listdir = builtins.open, t.os.listdir
def fake_open(path, *a, **k):
if path == "/proc/net/unix":
return io.StringIO("0000: 00000002 0 ... @/tmp/.X11-unix/X1\n0000: ... @/tmp/.X11-unix/X0\n"
"0000: ... /run/user/1000/wayland-0\n")
return real_open(path, *a, **k)
def fake_listdir(path):
if path == "/tmp/.X11-unix":
raise FileNotFoundError(path)
return real_listdir(path)
builtins.open, t.os.listdir = fake_open, fake_listdir
try:
self.assertEqual(self.real_displays(), [":0", ":1"])
finally:
builtins.open, t.os.listdir = real_open, real_listdir
FAKE_XPROP = """#!/bin/sh
sleep "${FAKE_X_DELAY:-0}"
if [ "$1" = "-root" ] && [ "$DISPLAY" = ":0" ]; then
echo "GAMESCOPE_FOCUSABLE_WINDOWS(CARDINAL) = 11, 0, 101, 12, 769, 102"
echo "GAMESCOPE_FOCUSED_WINDOW(CARDINAL) = 11"
echo "GAMESCOPE_FOCUS_DISPLAY(CARDINAL) = 12346"
fi
"""
FAKE_XWININFO = """#!/bin/sh
sleep "${FAKE_X_DELAY:-0}"
[ "$DISPLAY" = ":0" ] || exit 1
echo 'xwininfo: Window id: 0xb "Desktop"'
echo ' Absolute upper-left X: 0'
echo ' Absolute upper-left Y: 0'
echo ' Width: 1280'
echo ' Height: 800'
echo ' Map State: IsViewable'
"""
@unittest.skipIf(sys.platform == "win32", "fake X tools are shell scripts; frame_touch.py runs on the Frame")
class PanelsAsAFreshProcess(unittest.TestCase):
"""`frame_touch.py panels` the way the server runs it: a new process each request, with
fake xprop and xwininfo first on PATH (two displays, :0 and :1)."""
def run_panels(self, delay):
with tempfile.TemporaryDirectory() as bin_dir:
for name, body in (("xprop", FAKE_XPROP), ("xwininfo", FAKE_XWININFO)):
path = Path(bin_dir) / name
path.write_text(body)
path.chmod(0o755)
env = dict(os.environ, PATH=bin_dir + os.pathsep + os.environ.get("PATH", ""), FAKE_X_DELAY=str(delay),
PYTHONDONTWRITEBYTECODE="1")
code = ("import sys; sys.path.insert(0, sys.argv[1]); import frame_touch as t; "
"t.displays = lambda: [':0', ':1']; t.desktop_running = lambda: False; "
"sys.argv = ['frame_touch.py', 'panels']; sys.exit(t.main())")
start = time.monotonic()
r = subprocess.run([sys.executable, "-c", code, str(ROOT / "ui")], env=env, capture_output=True,
text=True, timeout=60)
took = time.monotonic() - start
self.assertEqual(r.returncode, 0, r.stderr)
return json.loads(r.stdout), took
def test_lists_panels(self):
out, _ = self.run_panels(0)
self.assertEqual(out["focus"], 11)
self.assertEqual(out["focus_display"], ":0")
self.assertEqual([(p["window"], p["display"], p["focused"]) for p in out["panels"]],
[(11, ":0", True), (12, ":0", False)])
self.assertNotIn("why", out)
def test_slow_x_tools_stay_inside_the_servers_timeout(self):
# Review of #83: X tools answering in 4.6 s each would take well over the server's
# 20 s; one deadline for the run (PANELS_DEADLINE, 12 s) returns in time, saying why.
out, took = self.run_panels(4.6)
self.assertLess(took, 15, out)
self.assertEqual(out["panels"], [])
self.assertIn("took too long", out["why"])
@unittest.skipUnless(shutil.which("node"), "needs node")
class PageGestures(unittest.TestCase):
"""Control's gestures and queue (tests/page/ctrl_gestures.mjs runs the real functions from index.html)."""
@@ -268,6 +500,12 @@ class PageGestures(unittest.TestCase):
timeout=60)
self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
def test_desktop_panel_list(self):
# #73: the headset's reason for no panels, and a failed listing, reach Capture and Control.
r = subprocess.run(["node", str(ROOT / "tests" / "page" / "desk_panels.mjs")], capture_output=True, text=True,
timeout=60)
self.assertEqual(r.returncode, 0, r.stdout + r.stderr)
class Script(unittest.TestCase):
def test_helper_compiles_on_the_frames_python(self):
+172 -35
View File
@@ -25,7 +25,9 @@ Standard library only (ctypes for libei), like the rest of what runs on the Fram
import ctypes
import json
import os
import re
import select
import shutil
import subprocess
import sys
import time
@@ -53,14 +55,41 @@ def say(state, **more):
# ---- which panel has focus -----------------------------------------------------
def xprop_root(display, name):
PROPS = ("GAMESCOPE_FOCUSABLE_WINDOWS", "GAMESCOPE_FOCUSED_WINDOW", "GAMESCOPE_FOCUS_DISPLAY")
TOOL_TIMEOUT = 5.0 # seconds for any one xprop or xwininfo
PANELS_DEADLINE = 12.0 # seconds for a whole `panels` run: well inside the server's 20 s for /api/panels
DEADLINE = [None] # when the current `panels` run must stop asking X tools (monotonic), if it has one
class OutOfTime(Exception):
"""The current run's deadline passed: stop and report what was found."""
def budget(limit=TOOL_TIMEOUT):
"""Seconds an X tool may take now: its own limit, cut to what's left of the run's deadline."""
if DEADLINE[0] is None:
return limit
left = DEADLINE[0] - time.monotonic()
if left <= 0:
raise OutOfTime()
return min(limit, left)
def read_props(display, limit=TOOL_TIMEOUT):
"""gamescope's root properties on one display, in one xprop call: {name: [values]} for
those the display has, {} if none (or xprop didn't answer)."""
try:
out = subprocess.run(["xprop", "-root", name], env=dict(os.environ, DISPLAY=display),
capture_output=True, text=True, timeout=5).stdout
out = subprocess.run(["xprop", "-root", *PROPS], env=dict(os.environ, DISPLAY=display),
capture_output=True, text=True, timeout=budget(limit)).stdout
except (OSError, subprocess.SubprocessError):
return []
values = out.split("=", 1)[1] if "=" in out else ""
return [int(v) for v in values.replace(",", " ").split() if v.isdigit()]
return {}
props = {}
for line in out.splitlines():
name, sep, values = line.partition("=")
name = name.split("(", 1)[0].strip()
if sep and name in PROPS:
props[name] = [int(v) for v in values.replace(",", " ").split() if v.isdigit()]
return props
def window_info(display, window):
@@ -68,7 +97,7 @@ def window_info(display, window):
try:
which = ["-root"] if window == "root" else ["-id", str(window)]
out = subprocess.run(["xwininfo", *which], env=dict(os.environ, DISPLAY=display),
capture_output=True, text=True, timeout=5).stdout
capture_output=True, text=True, timeout=budget()).stdout
except (OSError, subprocess.SubprocessError):
return None
if "IsViewable" not in out:
@@ -86,13 +115,58 @@ def window_info(display, window):
def displays():
return sorted(f":{n[1:]}" for n in os.listdir("/tmp/.X11-unix") if n[1:].isdigit())
"""Every X display on the Frame: the sockets in /tmp/.X11-unix, and the abstract ones
X servers on Linux usually listen on as well (in /proc/net/unix), in case a SteamOS
build stops sharing /tmp/.X11-unix with SSH sessions."""
found = set()
try:
found.update(int(n[1:]) for n in os.listdir("/tmp/.X11-unix") if n[:1] == "X" and n[1:].isdigit())
except OSError:
pass
try:
with open("/proc/net/unix") as f:
found.update(int(n) for n in re.findall(r"@/tmp/\.X11-unix/X(\d+)", f.read()))
except OSError:
pass
return [f":{n}" for n in sorted(found)]
# The input helper (`frame_touch.py` reading events) lives for a whole Control session and
# reads focus often, so it remembers which display has gamescope's properties: found, for
# good; not found, for ROOT_RETRY. A `panels` run is a fresh process each time, so this
# saves it nothing; it costs nothing either while :0 has the properties.
_ROOT = [] # [display, when it was given up on, or None if found]
ROOT_PROBE_TIMEOUT = 2.0 # seconds for each display other than :0, so a stalled one can't use the run
ROOT_RETRY = 30.0
def gamescope():
"""(display, properties): the X display whose root window carries gamescope's panel
properties (GAMESCOPE_FOCUSABLE_WINDOWS and friends), and what they say now.
That's :0 on every build seen so far (SteamOS 0.3.0 to 0.4.1), so :0 is read first and,
when it has them, used with no other probe. Only if it hasn't are the other displays
asked, briefly each; with none, :0 and no properties.
"""
if _ROOT and (_ROOT[1] is None or time.monotonic() - _ROOT[1] < ROOT_RETRY):
return _ROOT[0], read_props(_ROOT[0])
props, found = read_props(":0"), ":0"
if not props:
found = None
for display in displays():
if display != ":0":
props = read_props(display, ROOT_PROBE_TIMEOUT)
if props:
found = display
break
_ROOT[:] = [found, None] if found else [":0", time.monotonic()]
return _ROOT[0], props
def window_pid(display, window):
try:
out = subprocess.run(["xprop", "-id", str(window), "_NET_WM_PID"], env=dict(os.environ, DISPLAY=display),
capture_output=True, text=True, timeout=5).stdout
capture_output=True, text=True, timeout=budget()).stdout
except (OSError, subprocess.SubprocessError):
return None
value = out.rsplit("=", 1)[-1].strip() if "=" in out else ""
@@ -120,20 +194,20 @@ def locate(window, pid):
"root": [root.get("width", info["width"]), root.get("height", info["height"])]}
def focusable():
"""gamescope's focusable windows as (window, app id, pid)."""
t = xprop_root(":0", "GAMESCOPE_FOCUSABLE_WINDOWS")
def focusable(props=None):
"""gamescope's focusable windows as (window, app id, pid), from props if already read."""
t = (gamescope()[1] if props is None else props).get("GAMESCOPE_FOCUSABLE_WINDOWS", [])
return [tuple(t[i:i + 3]) for i in range(0, len(t) - 2, 3)]
def focus_display():
"""The display of the focused window, from GAMESCOPE_FOCUS_DISPLAY on :0's root.
def focus_display(props=None):
"""The display of the focused window, from GAMESCOPE_FOCUS_DISPLAY on gamescope's root.
gamescope writes the name (":1") as 32-bit items, so its first four bytes land,
little-endian, in the first value: 12602 is 0x313A, ":1" (steamcompmgr.cpp;
seen 2026-09-29).
"""
values = xprop_root(":0", "GAMESCOPE_FOCUS_DISPLAY")
values = (gamescope()[1] if props is None else props).get("GAMESCOPE_FOCUS_DISPLAY")
if not values:
return None
name = (values[0] & 0xFFFFFFFF).to_bytes(4, "little").split(b"\0", 1)[0].decode("ascii", "replace")
@@ -141,19 +215,23 @@ def focus_display():
def focus_now():
"""Just which window and display have focus: two property reads, for checking a press."""
window = (xprop_root(":0", "GAMESCOPE_FOCUSED_WINDOW") or [0])[0]
return (window or None, focus_display() if window else None)
"""Just which window and display have focus: one property read, for checking a press."""
props = gamescope()[1]
window = (props.get("GAMESCOPE_FOCUSED_WINDOW") or [0])[0]
return (window or None, focus_display(props) if window else None)
def focus():
def focus(props=None):
"""The panel that has focus in the headset: window, display, name and sizes (gamescope
publishes the window and its display on :0's root)."""
window = (xprop_root(":0", "GAMESCOPE_FOCUSED_WINDOW") or [0])[0]
publishes the window and its display on its root display, :0 so far). props: gamescope's
properties if already read."""
if props is None:
props = gamescope()[1]
window = (props.get("GAMESCOPE_FOCUSED_WINDOW") or [0])[0]
if not window:
return {"window": None}
app, pid = next(((a, p) for w, a, p in focusable() if w == window), (None, None))
display = focus_display()
app, pid = next(((a, p) for w, a, p in focusable(props) if w == window), (None, None))
display = focus_display(props)
info = window_info(display, window) if display else None
if info:
root = window_info(display, "root") or {}
@@ -164,18 +242,77 @@ def focus():
return {**panel, "app": app} if panel else {"window": None}
def panels():
"""Every app panel (gamescope's focusable windows), for watching one that hasn't focus."""
now = focus()
found = []
for window, app, pid in focusable():
panel = locate(window, pid)
if panel and panel["width"] > 1 and panel["height"] > 1 and \
not any(f["window"] == window and f["display"] == panel["display"] for f in found):
panel.pop("root", None)
found.append({**panel, "app": app, "focused": (window, panel["display"]) ==
(now.get("window"), now.get("display"))})
return {"focus": now.get("window"), "focus_display": now.get("display"), "panels": found}
def panels(limit=None):
"""Every app panel (gamescope's focusable windows), for watching one that hasn't focus.
One deadline (PANELS_DEADLINE) covers the whole run, inside the server's timeout: past
it, the panels found so far come back with a "why" saying the headset was too slow.
"""
DEADLINE[0] = time.monotonic() + (PANELS_DEADLINE if limit is None else limit)
now, found, unfound, late = {}, [], 0, False
try:
props = gamescope()[1]
now = focus(props)
for window, app, pid in focusable(props):
panel = locate(window, pid)
unfound += panel is None
if panel and panel["width"] > 1 and panel["height"] > 1 and \
not any(f["window"] == window and f["display"] == panel["display"] for f in found):
panel.pop("root", None)
found.append({**panel, "app": app, "focused": (window, panel["display"]) ==
(now.get("window"), now.get("display"))})
late = time.monotonic() >= DEADLINE[0] # the last read may have been cut short
except OutOfTime:
late = True
finally:
DEADLINE[0] = None
out = {"focus": now.get("window"), "focus_display": now.get("display"), "panels": found}
if late:
out["why"] = ("The headset took too long to list its panels (its X tools didn't answer in time). "
+ ("Some may be missing. " if found else "") + "Try again, or use the Headset view.")
elif not found:
out["why"] = why_none(unfound)
return out
# The headset's Linux desktop, by process name (docs/how-the-frame-works.md).
DESKTOP = ("plasmashell", "kwin_wayland", "kwin_wayland_wrapper", "steamos-nested-desktop", "startplasma-wayland")
def desktop_running():
"""Whether the headset's Linux desktop (the nested Plasma session) is running."""
try:
pids = [p for p in os.listdir("/proc") if p.isdigit()]
except OSError:
return False
for pid in pids:
try:
with open(f"/proc/{pid}/comm") as f:
if f.read().strip() in DESKTOP:
return True
except OSError:
continue
return False
def why_none(listed):
"""Why panels() found nothing, in words the person at the computer can act on.
listed: how many of the windows gamescope lists aren't X11 windows on any display."""
missing = [tool for tool in ("xprop", "xwininfo") if not shutil.which(tool)]
if missing:
return (f"This SteamOS build has no {' or '.join(missing)}, which Frame Control uses to find the "
"headset's panels. Use the Headset view instead, and please report your SteamOS version.")
if listed:
# gamescope lists windows, but none is an X11 window on any display: a Wayland
# window (a desktop drawn straight to gamescope, say) can't be found or captured this way.
return (f"The headset has {listed} panel{'s' if listed > 1 else ''} open, but Frame Control can't "
f"find {'its' if listed == 1 else 'their'} window. This SteamOS build may draw "
f"{'it' if listed == 1 else 'them'} in a way the Desktop view can't capture yet. "
"Use the Headset view instead, and please report your SteamOS version.")
if desktop_running():
return ("The headset's desktop is running, but SteamOS isn't listing it as a panel. Look at it in "
"the headset (or close and reopen it), then try again, or use the Headset view.")
return "No app panel is open in the headset. Open the desktop or an app in the headset, then try again."
def to_root(panel, fx, fy):
+16 -7
View File
@@ -1889,7 +1889,7 @@ async function capture() {
var shotPanel = null;
if (view === "flat") {
shotPanel = deskPanel() || (await loadPanels(), deskPanel());
if (!shotPanel) throw new Error("no app panel is open in the headset");
if (!shotPanel) throw new Error(noPanelWhy());
url_ = `/api/screenshot?view=panel&window=${shotPanel.window}&display=${encodeURIComponent(shotPanel.display)}`;
}
const r = await fetch(url_, { headers: {"X-Frame-UI": UI_KEY} });
@@ -2101,8 +2101,14 @@ document.querySelectorAll("[data-eye]").forEach(b => b.onclick = () => setEye(b.
// Control reaches, so the view follows it unless you pick another to watch.
// Panels are "display/window": window ids are per X display, and the Frame has two.
// desk.shown is the panel whose picture is on screen now; Control only acts on that one.
const desk = { panels: [], focus: null, pick: "", timer: null, loaded: false, shown: null };
// desk.why is the headset's own reason when it lists no panels; desk.error, why listing them failed.
const desk = { panels: [], focus: null, pick: "", timer: null, loaded: false, shown: null, why: "", error: "" };
const panelKey = p => p ? `${p.display}/${p.window}` : null;
// Why the Desktop view has no panel to show: the headset's own reason when it gave one.
function noPanelWhy() {
if (desk.error) return "Couldn't list the headset's panels: " + desk.error;
return desk.why || "No app panel is open in the headset. Open the desktop or an app in the headset, then try again.";
}
function deskPanel() {
const want = desk.pick || desk.focus;
// Nothing in use and nothing picked: show the first panel rather than nothing.
@@ -2112,7 +2118,8 @@ async function loadPanels() {
try {
const before = panelKey(deskPanel());
const r = await api("/api/panels");
desk.panels = r.panels || []; desk.loaded = true;
desk.panels = r.panels || []; desk.loaded = true; desk.error = "";
desk.why = r.why || "";
desk.focus = r.focus ? `${r.focus_display}/${r.focus}` : null;
if (desk.pick && !desk.panels.some(p => panelKey(p) === desk.pick)) desk.pick = "";
renderPanelPick();
@@ -2120,8 +2127,11 @@ async function loadPanels() {
if (view === "flat" && live && now !== before) { toggleLive(false); toggleLive(true); }
ctrlShow();
} catch (e) {
// Polled every few seconds: waiting for the Frame, or an answer about the previous headset, isn't news.
if (!e.offline && !e.stale) log("Couldn't list the headset's panels: " + e.message, "e");
if (e.stale) return; // an answer about the previous headset says nothing about this one
// Polled every few seconds: waiting for the Frame, or the same failure again, isn't news.
if (!e.offline && e.message !== desk.error) log("Couldn't list the headset's panels: " + e.message, "e");
desk.error = e.message;
ctrlShow();
}
}
function watchPanels() {
@@ -2166,8 +2176,7 @@ function ctrlMode() { return view === "flat" ? "abs" : "rel"; }
function ctrlTarget() {
if (ctrlMode() === "rel") return { ok: true };
const p = deskPanel();
if (!desk.loaded) return { why: "Finding the headset's panels…" };
if (!p) return { why: "No app panel is open in the headset." };
if (!p) return { why: desk.loaded || desk.error ? noPanelWhy() : "Finding the headset's panels…" };
if (!desk.focus) return { why: "Nobody's using a panel in the headset yet. Taps work once one is in use." };
if (panelKey(p) !== desk.focus) return { why: "Only the panel in use takes taps. Pick “Panel in use” to control it." };
if (desk.shown !== desk.focus) return { why: live ? "Waiting for the picture of the panel in use…"
+9 -2
View File
@@ -697,7 +697,7 @@ PASTE = r"""set -u
text=$(cat; printf x); text=${text%x}
pid=$(pgrep -u "$(id -u)" -x plasmashell | head -n 1)
if [ -z "$pid" ]; then
echo "plasmashell is not running: open the desktop in the headset first." >&2
echo "The headset's desktop isn't open (plasmashell is not running). Open the desktop in the headset, then try again." >&2
exit 2
fi
bus=$(tr '\0' '\n' < "/proc/$pid/environ" | sed -n 's/^DBUS_SESSION_BUS_ADDRESS=//p')
@@ -709,6 +709,7 @@ else
exit 2
fi
"""
DESKTOP_CLOSED = "plasmashell is not running" # what PASTE says when the headset's desktop isn't open
# base64 keeps the script intact through every local shell's quoting rules.
PASTE_CMD = 'bash -c "$(echo %s | base64 -d)"' % base64.b64encode(PASTE.encode()).decode()
@@ -725,7 +726,13 @@ def clipboard(body):
text = body.get("text")
if not isinstance(text, str) or not text:
raise Failure("nothing to send", 400)
return {"message": ssh(PASTE_CMD, stdin=text, timeout=30).strip()}
try:
return {"message": ssh(PASTE_CMD, stdin=text, timeout=30).strip()}
except Failure as e:
if DESKTOP_CLOSED in str(e):
# The headset's state, for the person to change, not a fault: a 409 isn't reported as one.
raise Failure(str(e), 409) from None
raise
# ---- keyboard and pointer (KDE Connect on the Frame, see frame_input_agent.py) ----