mirror of
https://github.com/saphid/frame-control.git
synced 2026-10-06 03:00:18 +02:00
saphidandClaude Opus 5.5
d4486a7681
Package the Frame Control web UI as an installable Electron Mac app and bring in the tooling built alongside it. - app/: Electron wrapper that starts ui/server.py on a free loopback port, hardened window (sandbox, no navigation, runAsNode fuse off), login-shell PATH so Homebrew tools work from Finder, first-run offer to run connect.sh, ad-hoc signed DMG/zip via electron-builder. - ui/: headset view (OpenVR screenshots), device status, library, Steam "Get games" (owned games, install, store search), Android apps as persistent Lepton instances with a rated F-Droid catalogue and a private compatibility database, Android display controls over ADB, file and clipboard transfer, Flatpaks, remote and power actions. - apk-catalog/, compat-db/, frame/: catalogue build pipeline, Lakebed capsule for compatibility reports, Frame-side launchers. - tests/ and CI: server guard and validation tests plus Steam helper tests, run on Python 3.9 with script and app syntax checks. - Docs: README leads with the Mac app; new Android, panels, Steam games and field-notes docs; security notes on LAN-exposed ADB ports. Screenshot values for the headset's IP and Wi-Fi name are placeholders. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
74 lines
7.4 KiB
Markdown
74 lines
7.4 KiB
Markdown
# How the Frame is put together (field notes)
|
||
|
||
What we learnt by poking at a real Frame over SSH. Unless a line says
|
||
otherwise, it was **verified 2026-09-25** on SteamOS 0.3.0 (`VARIANT_ID=vr`,
|
||
build 20260922.6101926, kernel 6.18, aarch64). Topic docs go deeper. This page
|
||
is the map.
|
||
|
||
## The layer cake
|
||
|
||
```
|
||
SteamVR (vrserver, vrcompositor, vrdashboard) ← renders the room + panels
|
||
└─ gamescope --backend openvr ← one SteamVR overlay per app id
|
||
├─ Xwayland :0 (Steam UI, games, tagged apps) ← STEAM_GAME property = app id
|
||
├─ Xwayland :1 (STEAM_GAME_DISPLAY_0)
|
||
├─ Wayland socket gamescope-0
|
||
└─ steamos-nested-desktop ← "the Linux desktop" panel
|
||
└─ dbus-run-session startplasma-wayland
|
||
└─ kwin_wayland 1280×800, Wayland wayland-0, Xwayland :2
|
||
└─ plasmashell, Konsole, Dolphin, Flatpaks you open there
|
||
Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 3056000
|
||
```
|
||
|
||
## Facts worth knowing
|
||
|
||
| Fact | Where it matters |
|
||
|---|---|
|
||
| The desktop is a **nested** Plasma session: runtime dir `/run/user/1000/nested_plasma`, its own D-Bus bus, `WAYLAND_DISPLAY=wayland-0`, `DISPLAY=:2`. A plain `ssh frame app` can't find it. Copy the env from `plasmashell`'s `/proc/<pid>/environ`. | `run-on-frame.sh`, `paste-to-frame.sh` |
|
||
| The desktop size is hard-coded to 1280×800 in `/usr/bin/steamos-nested-desktop` (read-only rootfs). | [panels.md](panels.md) |
|
||
| gamescope runs with `--virtual-connector-strategy PerAppId`. Each app id becomes a SteamVR overlay `valve.steam.desktopgame.<id>`, which is a panel you can float. Setting `STEAM_GAME` on an X11 window on `:0` makes a new panel. | `panel-on-frame.sh`, [panels.md](panels.md) |
|
||
| Handy gamescope root properties on `:0`: `GAMESCOPE_FOCUSABLE_APPS`, `GAMESCOPE_FOCUSABLE_WINDOWS` (triples: window, app id, pid), `GAMESCOPE_FOCUSED_APP`. Read them with `DISPLAY=:0 xprop -root`. | Debugging panels |
|
||
| `gamescopectl screenshot <file>` (with `WAYLAND_DISPLAY=gamescope-0`) captures gamescope's flat layer. | Frame Control's capture |
|
||
| The **headset view** (both eyes, fully composited: room, panels, dashboard, controllers) comes from OpenVR `IVRScreenshots::RequestScreenshot(VRScreenshotType_Stereo)`. It's callable from `python3` with `ctypes` against `/opt/steamvr/bin/linuxarm64/libopenvr_api.so` as an overlay app. The compositor appends `.png`, writing a 1920×1080 side-by-side image (960×1080 per eye) plus a left-eye preview, in about 0.3s. In standby the frame is blank. `vrcmd --screenshot` and `vrcmd --compositorcmd screenshot_request` wrote nothing, even with `steamvr/rawCapturePath` set. | `ui/frame_vrshot.py` |
|
||
| Battery: `/sys/class/power_supply/max1720x_bat_7-36` gives µV/µA (current is positive while charging), `time_to_full_now`/`time_to_empty_now` in seconds, and `temp` in tenths of °C. The charger shows up as `tcpm-source-psy-…` (`type=USB`, `usb_type=C PD [PD_PPS]`), for example 12 V × 1.67 A. | Frame Control's battery card |
|
||
| `vrcmd --stats` reports `activity_level` (3 = standby). | Telling whether the headset is being worn |
|
||
| The SteamVR dashboard has docking: Float in World, Move, Size, Curvature, controller docking, Theater, Multitasking View. **Inferred** from `/opt/steamvr/resources/webinterface/dashboard/` and not yet driven by hand. | [panels.md](panels.md) |
|
||
| SteamVR settings live in `~/.config/openvr/config/steamvr.vrsettings`, not under `~/.local/share/Steam/config/`. `dashboard.lastAccessedExternalOverlayKey` names the last panel you used. | Settings tweaks |
|
||
| The Steam client's journal (`journalctl --user`) carries SteamVR system UI lines such as `[Overlays] Created: …` and `vroverlay_uid<appid>`. It's the quickest way to see panels come and go. | Debugging |
|
||
| Present: `rsync`, `flatpak`, `python3`, `git`, `qdbus6`, `xrdp`, `xprop`, `xwininfo`, `xterm`, `konsole`, `dolphin`, `gamescopectl`. Missing: `wl-copy`, `xclip`, `xsel`, `kdeconnect-cli`, `tailscale`, `krfb`, `wayvnc`. | Script design |
|
||
| Flathub is a **system** remote. `--user` installs over SSH work and show up in the desktop menu. | `install-apps.sh` |
|
||
| `/` is 10 GB and read-only. `/home` is 929 GB. | Where to put things |
|
||
| Clipboard: Klipper over the nested D-Bus bus (`qdbus6 org.kde.klipper …`). | `paste-to-frame.sh` |
|
||
| Lepton listens for ADB on the Frame's loopback `5555`, so tunnel it over SSH. It's Android 11 (API 30), 64-bit ARM only, with no `clipboard` service: Compose < 1.11, SDL/Kivy and Godot 4.3 apps crash on launch. | [apks.md](apks.md), `apk-catalog/` |
|
||
| Lepton Development deletes every ADB-installed app when it exits (`clear_baked_app_data "non steamlaunch container"` in `…/common/Lepton/lepton`) unless `LEPTON_NO_CLEANUP` is set. | [apks.md](apks.md) |
|
||
| Any APK can run as its own Lepton instance: run `…/common/Lepton/lepton waitforexitandrun -- app.apk` with `SteamAppId` set and `STEAM_COMPAT_DATA_PATH` under `~/.local/share/Steam`. Data persists and each gets its own container and panel. `frame/android/lepton-app.sh`, `ui/frame_android.py`. | [apks.md](apks.md) |
|
||
| The Steam client runs with `-cef-enable-debugging`, so its UI answers Chrome DevTools on loopback `127.0.0.1:8080`. The `SharedJSContext` page has `appStore` (owned apps), `downloadsStore` and `SteamClient.*`. `steam steam://install/<appid>` over SSH installs an owned game; when the options dialog shows (state 7), `SteamClient.Installs.ContinueInstall()` accepts it. **Verified 2026-09-25** with Balatro and Broforce. The Frame rating is `steam_hw_compat_category_packed >> 8 & 3`. | [steam-games.md](steam-games.md), `ui/frame_steam.py` |
|
||
| Chromium Flatpak 154 has **no immersive WebXR**: `navigator.xr` exists, but `isSessionSupported("immersive-vr")` returns `false`. Web VR180 players (DL8/DeoVR embeds) still play video inline as a flat, pannable view, and their VR button opens a tab on immersiveweb.dev. Forcing it doesn't help. `--enable-features=OpenXR,WebXR --force-webxr-runtime=openxr`, with `/opt/steamvr` and `XR_RUNTIME_JSON` exposed to the Flatpak, still returns `false`. The aarch64 Linux binary has no OpenXR code at all (no `xrCreateInstance`, no loader), even though `chrome://flags` lists `#webxr-runtime` → OpenXR. To watch in 3D, use a native player. Started with `--remote-debugging-port=9222`, Chromium answers DevTools on loopback. **Verified 2026-09-25**, BUILD_ID 20260922.6101926. | Web video, [panels.md](panels.md) |
|
||
| Power actions need `sudo`, which asks for the Developer Mode password over SSH. | Frame Control's power buttons |
|
||
|
||
## Debug recipes
|
||
|
||
```sh
|
||
# Which panels (app ids) exist right now?
|
||
ssh frame 'DISPLAY=:0 xprop -root GAMESCOPE_FOCUSABLE_APPS GAMESCOPE_FOCUSED_APP'
|
||
|
||
# Watch panels being created
|
||
ssh frame 'journalctl --user -f | grep --line-buffered "\[Overlays\]"'
|
||
|
||
# gamescope's full flags (in case Valve changes them)
|
||
ssh frame 'tr "\0" " " < /proc/$(pgrep -x gamescope | head -n 1)/cmdline'
|
||
|
||
# Everything the SteamVR dashboard can say (find hidden features)
|
||
ssh frame 'cat /opt/steamvr/resources/webinterface/dashboard/localization/dashboard_english.json'
|
||
```
|
||
|
||
## Where the rest lives
|
||
|
||
- Access and SSH: [ssh.md](ssh.md)
|
||
- Seeing the Frame from the Mac, and the Mac from the Frame: [streaming.md](streaming.md)
|
||
- Files and clipboard: [file-transfer.md](file-transfer.md)
|
||
- Android apps: [apks.md](apks.md)
|
||
- Installing and buying Steam games: [steam-games.md](steam-games.md)
|
||
- Floating windows in space: [panels.md](panels.md)
|
||
- What's still unverified: [open-questions.md](open-questions.md)
|