diff --git a/CLAUDE.md b/CLAUDE.md index 80e21a3..5322550 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -401,6 +401,19 @@ scheme, explicit component or not, no property to disable): it writes `steam://o (= the host Steam client's `~/.steam/steam.pipe`) and starts nothing. So "open in external player" hand-offs (Stremio → 4XVR) can't work inside Lepton; don't retry without Valve changing it. Lepton installs exactly one `*.apk` per app folder (two break `get_apk_path`). +**Text input (2026-10-03, verified in the headset with Stremio VR):** Lepton's Android has no IME (`ime list` empty) +and runs VR apps headless (`lepton.headless=true`): no Android window has input focus (`dumpsys input` FocusedWindows +empty), so neither `input text`, a USB keyboard nor Steam's keyboard reach the app. `lepton-show-flatscreen` on a VR +app keeps VR working (FrameBridge 72 fps) and gives the window focus; Steam's on-screen keyboard then opens for text +fields. Unity's TMP_InputField on non-Quest Android waits for the system keyboard (`TouchScreenKeyboardShouldBeUsed`) +and deselects a frame later unless `isKeyboardUsingEvents` (Android: `InPlaceEditing() && m_HideSoftKeyboard`); +uGUI InputField's LateUpdate keeps the field when `InPlaceEditing()`. Patch `frame.unity_text_input` rewrites them +(`mov w0,#0|#1; ret`) at Cpp2IL's **Offset** (= file offset; RVA differs by 0x4000 in Stremio's lib), Cpp2IL +2022.1 pre-release (Unity 6 / metadata v31; Il2CppDumper can't), cached per libil2cpp sha. `/dev/uinput` has an ACL +for steamos (Steam Input) → agent v33 `_keyboard` uinput keyboard ("Type on Frame") reaches everything with focus. +Installing a second APK into a container with `pm install` re-runs Lepton's post-install hook on the **main** app +(`lepton.active_app_id`; moves its files to /data/steam_app) and corrupts it on the next start (fix: touch the APK → +re-bake); `cmd_real package install` skips the hook. **Lepton storage (2026-09-30):** each app's /sdcard (= /storage/emulated/0 → `/lepton-data/external`) has `Movies`/`Download`/`Documents` symlinked to the Frame's `~/Videos`/`~/Downloads`/`~/Documents` (liblepton/mounting.sh, only if they exist at start); agent v24 `storage_targets` reads that mapping. Android's MediaProvider canonicalises paths to /home/steamos/... and rejects every file ("doesn't appear under [/system/media...]"), `sm list-volumes` is empty: the media index never works, apps must browse folders. Lepton installs with `adb install -g` (runtime permissions granted, MANAGE_EXTERNAL_STORAGE too). Files: `install/files.py`, `frameport frame send|storage`, GUI Files tab (formerly Frame → Send files). **SteamVR per-app settings (2026-09-30):** editing steamvr.vrsettings while SteamVR runs is lost; the web API (127.0.0.1:27062 /app/setsettings) needs `x-steamvr-secret`. `native/vrsettings` = `fp_vrsettings.exe` (freestanding, OpenVR `FnTable:IVRSettings_003` as a Utility app, loads SteamVR's bin/win64/openvr_api.dll) sets them live and SteamVR persists them: section `steam.app.`, keys `preferredRefreshRate` (float) and `motionSmoothingOverride` (0 global, 1 on, 2 off, 3 always). Steam Link (vrlink) lists the Frame's rates 72/80/90/96/108/120/144 in vrserver.txt and follows the per-app preference ("host preferred N Hz"; whether the key is honoured is unverified in-headset yet). Judder metric: vrcompositor.txt session summary dropped + "Timed out. N total" (Stormland: 0 dropped but 313 timeouts in 2 min); fpsVR (`%LOCALAPPDATA%\fpsVR\*.json`, 0.1 ms histograms) gives p99 CPU/GPU ms. `pcvr.steamvr_tuning` (default on, PC only) applies on Play: highest rate whose budget ≥ p99×1.05, at least one step down, smoothing on. diff --git a/README.md b/README.md index f474b73..ce09484 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,8 @@ Install games that target the Meta Quest, Android, or general PCVR onto your **V - **Per-game recipes:** a tested catalog plus detection rules; every patch explained in plain words. - **FrameBridge:** FramePort's OpenXR adapter emulates what the Frame natively lacks (passthrough, room, controller models, curved and 360° layers); game settings as simple switches. +- **Type on Frame:** your computer's keyboard works as a keyboard on the Frame, in apps, Steam and the desktop. Unity + text fields that would close at once on the Frame are fixed per game. - **Beyond Quest:** Android apps as windows, PC VR via Proton or Revive, a Files tab with drag and drop. - **Self-updating** releases, redacted diagnostics, one-click problem reports and working-config sharing. diff --git a/docs/COMPATIBILITY.md b/docs/COMPATIBILITY.md index 00797ba..d1e5316 100644 --- a/docs/COMPATIBILITY.md +++ b/docs/COMPATIBILITY.md @@ -1,6 +1,6 @@ # Compatibility -The built-in catalog has tested settings for 38 games (27 work, 5 work with known issues, 6 can't run on the Frame). +The built-in catalog has tested settings for 39 games (27 work, 6 work with known issues, 6 can't run on the Frame). Other games get suggested patches from detection rules; each suggestion states its reason, and every patch can be switched on or off under **Customize**: described in plain words, with **Show technical details** for the exact effect of each patch. diff --git a/docs/FRAME_RUNTIME.md b/docs/FRAME_RUNTIME.md index 23c6d5f..3e1139c 100644 --- a/docs/FRAME_RUNTIME.md +++ b/docs/FRAME_RUNTIME.md @@ -105,3 +105,14 @@ - Processes started from Steam (Konsole, SSH sessions?) share steam.service's cgroup: use `systemd-run --user`. - SSH: `sshd` must be enabled (`sudo systemctl enable --now sshd`), which needs a user password (`passwd`). - mDNS: avahi-daemon runs by default; hostname `frame` → `frame.local`. + +## Text input + +- Lepton's Android has no on-screen keyboard (IME) and VR apps run headless, so no Android window has keyboard focus + and key presses (Steam's keyboard, USB/Bluetooth keyboards, `input text`) don't reach VR apps. The app folder's + `lepton-show-flatscreen` marker shows the window (VR keeps working) and gives it focus; Steam's on-screen keyboard + then opens for its text fields (FramePort patch `device.text_input_window`). +- `/dev/uinput` is writable by the steamos user (ACL for Steam Input): a uinput virtual keyboard works like a real + one everywhere (FramePort's "Type on Frame", agent `_keyboard`). +- Unity text fields close without a system keyboard; see `frame.unity_text_input` in PLAYBOOK.md. + diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 09afe0f..1359545 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -77,6 +77,18 @@ reality), showing only what matters for that game. Changes are kept with the gam ![Game settings](images/game-settings.png) +## Typing on the Frame + +- **Type on Frame** (Steam Frame page, a game's menu, or the keyboard icon on the sidebar's Frame card): while the + window is open, this computer's keyboard works as a keyboard plugged into the Frame. Select a text field in the + headset (in an app, Steam or the desktop) and type; Esc and shortcuts go to the Frame too. Paste longer text into + the box to type it in one go (US keyboard layout). Click **Done** to disconnect. +- **Steam's on-screen keyboard** opens for text fields of apps shown as a window (2D apps, and VR apps with + **Show the app's Android window**). +- **Unity apps whose text fields close at once** (a caret flashes, nothing can be typed): FramePort suggests + **Make Unity text fields work** for them. The first time, it downloads Cpp2IL (a tool that finds the right spot in the + game's code, ~17 MB). Games added before this version: open the game's menu → **Analyze again**, then reinstall. + ## Updating FramePort checks for a new release at start and every 6 hours (it only downloads the release information). When one diff --git a/docs/PLAYBOOK.md b/docs/PLAYBOOK.md index 36bbc0a..e1db41c 100644 --- a/docs/PLAYBOOK.md +++ b/docs/PLAYBOOK.md @@ -14,6 +14,7 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke | Symptom | Cause | Fix | |---|---|---| | Every game suddenly fails to start: `crun: create keyring …: Disk quota exceeded`, `is not a running context` | rootless podman leaked a kernel keyring per launch; 200-key quota exhausted | `keyring = false` in `~/.config/containers/containers.conf` (FramePort agent does it), then reboot the Frame once | +| A Unity app's text field shows a caret for a moment and loses focus; no keyboard appears (e.g. Stremio VR login) | Unity's TMP_InputField/InputField wait for Android's on-screen keyboard and close themselves without one (Lepton has none); headless VR apps also have no focused Android window, so no key press reaches them | `frame.unity_text_input` (Cpp2IL finds `TouchScreenKeyboardShouldBeUsed`/`isKeyboardUsingEvents`, rewritten to false/true) + `device.text_input_window` (lepton-show-flatscreen: Steam's keyboard and Type on Frame work) | | `APP_ACTIVITY is empty`, nothing starts | Manifest has category INFO only; Lepton needs LAUNCHER | `frame.launcher` (automatic) | | PC VR game on the Frame shows as a flat window / Revive: `Unable to load LibOVRRT DLL` / `LoaderInstance::CreateInstance chained CreateInstance call failed` | Frame SteamVR runtime rejects OpenXR apiVersion 1.1 (`XR_ERROR_API_VERSION_UNSUPPORTED`), which Proton 11's VR helper requests | `pcvr.xr_timefix` (Frame OpenXR layer, default on): retries xrCreateInstance as 1.0 | | Unreal PC VR game on the Frame runs as a flat window although OpenXR works (no `LogHMD` OVRPlugin lines; Unreal logs nothing when it skips the Oculus plugin) / launch.log: `FramePort oculushmd: could not create the OculusHMDConnected event` | UE's OculusHMD (and LibOVR's `ovr_Detect`) only start when the Windows event `OculusHMDConnected` exists and is signalled; on a PC the Oculus service creates it. Revive hooks `OpenEventW` for it, but that relies on Detours patching Wine's (ARM64EC) kernelbase | `pcvr.oculus_unreal` (default for Unreal Rift games; PC VR counterpart of overport's `patch_oculus_unreal`): launch.sh runs the injector through `fp_oculushmd.exe`, which provides the real event until the game exits | diff --git a/tests/test_unity_text_input.py b/tests/test_unity_text_input.py index b0980b0..feb4659 100644 --- a/tests/test_unity_text_input.py +++ b/tests/test_unity_text_input.py @@ -106,10 +106,11 @@ def test_catalog_device_toggles_round_trip(): def test_reanalyze_refreshes_the_suggestion_and_keeps_a_users_recipe(monkeypatch, tmp_path): + from test_patches import _analysis + from frameport import pipeline from frameport.core import library from frameport.core.models import Recipe - from test_patches import _analysis apk = tmp_path / "game.apk" apk.write_bytes(b"x")