mirror of
https://github.com/daniel-lynch/ovrplugin-openxr-shim.git
synced 2026-10-06 03:00:05 +02:00
Daniel LynchandClaude Opus 4.8
a72a79ad29
An independent reimplementation of Meta's libOVRPlugin ABI on top of OpenXR, so VrApi-era Meta Quest VR titles can run on non-Meta OpenXR runtimes (Monado, Steam Frame) instead of being locked to Meta hardware. Original code only — no Meta/Epic/Capcom binaries, headers, or assets. Includes a desktop harness that drives the shim against Monado headless. Scope/legal: interoperability; entitlement handling is out of scope. See README for the legal/scope section and docs/ for the research trail and design notes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D6sFYGXZPsq3v7xtcDES6g
124 lines
8.5 KiB
Markdown
124 lines
8.5 KiB
Markdown
# RE4 VR Shim — Handoff (2026-06-25 session)
|
||
|
||
Entry point for next session. Prior handoff: `HANDOFF-2026-06-24.md`.
|
||
Detailed per-finding log lives in the auto-memory (`MEMORY.md` index).
|
||
|
||
## TL;DR
|
||
- **Root cause of the black flashes / 3×-hand / stutter is one thing: a GPU-load-gated
|
||
render-submit race.** Under load UE 4.25's RHI thread `vkQueueSubmit`s the eye render
|
||
*after* our `xrEndFrame`, so we present a stale/empty (black) swapchain image. PROVEN by
|
||
texture dump (gameplay eye image `min=max=0`, pure black) and confirmed load-gated
|
||
(in-game **Standing** comfort setting flashes black; **Sitting** doesn't — higher camera =
|
||
more visible geometry = blows frame budget).
|
||
- **Fixes tried this session:**
|
||
- `vkQueueWaitIdle` before resolve (`debug.re4vr.qwait`) — **no effect** (UE hadn't
|
||
submitted yet; can't drain unsubmitted work).
|
||
- **Deferred-flush pipeline** (`debug.re4vr.pipeline=1`, rewrote the render-ahead path to
|
||
hold frame N and flush+present at N+1 after UE submits) — reduced *pure* black but
|
||
**crashes more + adds latency**, marginal. **SHELVED.** Pipeline path has been the
|
||
unstable one all along.
|
||
- **FFR** (`debug.re4vr.ffr`, NEW this session) — implemented via `XR_FB_foveation`
|
||
(dynamic). Validated (game uses foveation natively) and helps headroom/smoothness, but
|
||
**not enough** to bring Standing under budget alone.
|
||
- **RE of the original libs confirmed:** `ovrp_EndFrame4` does **zero GPU sync** — VrApi's
|
||
system-owned textures carry it. Nothing to copy; our explicit-barrier approach is right,
|
||
we just have a timing/threading mismatch. OpenXR gives the same free sync IF we
|
||
submit-before-release (portable; the Quest-specific path would NOT port to Steam Frame).
|
||
- **NEXT (the real lever):** re-enable the **game's native dynamic-performance subsystem**
|
||
that our stubs disable (see below). AppSW turned out to be a weaker lead — RE4 doesn't
|
||
drive it.
|
||
|
||
## PRIMARY next task — re-enable the game's dynamic-perf machinery
|
||
RE4 was built to scale its own GPU load under pressure, but we stub the whole subsystem as
|
||
unsupported (logcat, all fire ~8×/session):
|
||
```
|
||
ovrp_GetGPUFrameTime -> Unsupported (game polls GPU frame time to decide)
|
||
ovrp_GetTiledMultiResLevel -> Unsupported (TiledMultiRes = Oculus name for FFR)
|
||
ovrp_SetTiledMultiResLevel -> Unsupported
|
||
ovrp_SetTiledMultiResDynamic -> Unsupported (dynamic FFR driven by GPU time)
|
||
ovrp_IsPerfMetricsSupported -> Unsupported
|
||
ovrp_GetSystemRecommendedMSAALevel2 -> NotYetImplemented
|
||
ovrp_GetLayerTextureFoveation -> NotYetImplemented (still stubbed even though we apply FFR at OpenXR)
|
||
```
|
||
Plan:
|
||
1. **Report TiledMultiRes supported + honor `Set*` calls** by mapping the game's requested
|
||
FFR level/dynamic onto our OpenXR `apply_foveation()` (xr_runtime.c). Today FFR is a
|
||
static prop (`debug.re4vr.ffr`); instead let the GAME drive the level (it already wants
|
||
dynamic, GPU-time-based). Stub fns in `shim/src/stubs.c`; real handlers go in
|
||
xr_runtime.c next to `apply_foveation`.
|
||
2. **Implement `ovrp_GetGPUFrameTime`** so the game's dynamic loop has its input. Source:
|
||
OpenXR has no portable GPU-time query; options = `XR_FB_... ` perf ext, or feed our own
|
||
measured frame dt (we already compute it for the `FRAME`/`HITCH` trace). Even an
|
||
approximate value lets the game's scaler engage.
|
||
3. Re-check Standing after: if the game drops its own resolution/foveation under load and
|
||
the black stops, this is the fix. It's the game's *intended* GPU-bound mitigation.
|
||
|
||
## SECONDARY — Application SpaceWarp (verify first, likely not viable as-is)
|
||
- `ovrp_GetLayerTextureSpaceWarp` is **stubbed but RE4 does NOT call it** this session; UE
|
||
passes a **108-byte EyeFov desc, flags=0** (no motion-vector variant). So UE isn't
|
||
rendering motion vectors → AppSW can't be fed.
|
||
- The **system** compositor IS running SpaceWarp/ASW (logcat tag `SpaceWarp`/`SpaceWarpCore`,
|
||
PID 2438) — that's compositor-side reprojection (no app MV needed). It may already be
|
||
reprojecting; our black frames defeat it (can't extrapolate a black frame).
|
||
- IF you pursue AppSW: needs (a) game to render velocity (UE 4.25 may not support it without
|
||
the Meta fork), (b) provide MV+depth swapchains via `ovrp_GetLayerTextureSpaceWarp`,
|
||
(c) submit `XrCompositionLayerSpaceWarpInfoFB` (chained on the projection view) via
|
||
`XR_FB_space_warp` (extension + types already in bundled headers, openxr.h:6021+;
|
||
`XrSystemSpaceWarpPropertiesFB` gives recommended MV size). **Verify (a) before investing.**
|
||
|
||
## Current build / stable config
|
||
Build/deploy:
|
||
```
|
||
./shim/build_android.sh && ./packaging/repack.sh
|
||
adb -s <redacted-serial> install -r packaging/out/re4vr-shim.apk
|
||
adb -s <redacted-serial> logcat | grep -i xrr
|
||
```
|
||
Device Quest (USB serial `<redacted-serial>`, or `<redacted-ip>:5555`).
|
||
Pkg `com.Armature.VR4`, activity `com.epicgames.ue4.GameActivity`.
|
||
**STABLE config left set:** `pipeline=0` (deferred-flush OFF), `ffr=2` (medium), `sscap=0`.
|
||
Consider compiling the deferred-flush path out / default FFR medium for a clean baseline.
|
||
|
||
## Runtime debug toggles (system props; relaunch unless noted "live")
|
||
- `debug.re4vr.ffr 0|1|2|3` — NEW. Fixed Foveated Rendering off/low/med/high (XR_FB_foveation, dynamic). Default 2.
|
||
- `debug.re4vr.pipeline 1` — deferred-flush render-ahead (live). UNSTABLE/crashes; shelved.
|
||
- `debug.re4vr.qwait 1` — diag: vkQueueWaitIdle before resolve (live). No effect; keep for reference.
|
||
- `debug.re4vr.trace 1|2` — NEW. 1=per-frame FRAME line (dt+HITCH) + starvation logs; 2=per-call WAIT/BEGIN/END + per-layer dump (live).
|
||
- `debug.re4vr.sscap 1` — cap supersample 1.2x→1.0x.
|
||
- `debug.re4vr.copyring 1` — copy-ring (eye-only; gameplay black). From prior session.
|
||
- `debug.re4vr.diag 1|2|3|4` — visual A/B: 1=proj only, 2=mono, 3=zoom, 4=head-lock quad (live).
|
||
- `debug.re4vr.dump N` — eye-texture readback to /sdcard/Android/data/com.Armature.VR4/files/*.ppm. **UNRELIABLE probe — see below.**
|
||
- `debug.re4vr.depth 1`, `debug.re4vr.noflushwait 1` — prior-session probes.
|
||
|
||
## What's RULED OUT for the black (don't re-investigate)
|
||
Quad placement/presence (diag=1/4), ViewportRect (UE always submits full), empty-frame
|
||
submission (`SUBMIT-BLACK=0` over 6730 frames), head-vs-eye pose mismatch (poses match
|
||
exactly), stereo layout/IPD (array, correct), pure frame hitching (pipeline cut hitches
|
||
804→39 but black persisted). It is specifically the **render-submit timing race**, GPU-load-gated.
|
||
|
||
## Diagnostic recipes (USE THESE — earlier probes misled us)
|
||
- **Perception is ground truth.** The `dump` probe is UNRELIABLE: it reads stale swapchain
|
||
buffers AND its fence-wait *forces* the render (observer effect — "dumping forced the load").
|
||
- **Recordings: limited-range.** `YAVG=16` IS black (not grey). 30fps mono UNDERSAMPLES 72Hz
|
||
flashes (lower bound). Separate sustained black (menu/scene transitions) from brief isolated
|
||
flashes. ffmpeg: `-vf "signalstats,metadata=print:key=lavfi.signalstats.YAVG:file=y.txt"`;
|
||
blend with `tmix=frames=N` to reveal ghosting; search the WHOLE clip, not the start.
|
||
- **Frame timing (reliable, not undersampled):** `debug.re4vr.trace 1` → grep `HITCH` / `FRAME`.
|
||
- **RE pipeline:** `JAVA_HOME=tools/jdk-21.0.11+10 tools/ghidra_12.1.2_PUBLIC/support/analyzeHeadless
|
||
ghidra_proj NAME -import dump/apk_libs/lib/arm64-v8a/libOVRPlugin.so -scriptPath ghidra_scripts
|
||
-postScript DumpEndFrame.java -deleteProject`. Original libs extracted in `dump/apk_libs/`.
|
||
|
||
## Key code locations (xr_runtime.c unless noted)
|
||
- Eye-fov submit / build_composition (~560), the resolve barrier path is in `vk_session.c`
|
||
(`xrr_vk_flush_submit_ex`, barrier COLOR_ATTACHMENT_WRITE→MEMORY_READ; NO QueueWaitIdle anywhere).
|
||
- `submit_pending` (~740): where a black frame reaches the compositor (`SUBMIT-BLACK` log).
|
||
- Pipeline (deferred-flush) branch in `xrr_end_frame` (~900); sync branch (~960).
|
||
- `apply_foveation` / `ffr_level` (just before `xrr_setup_layer`, ~1148) — FFR; extend here
|
||
to let the game drive the level (TiledMultiRes handlers).
|
||
- Capability stubs to implement: `shim/src/stubs.c` (GetGPUFrameTime, TiledMultiRes*, etc.).
|
||
- Texture handoff: `xrr_get_layer_texture` (~1230) — UE renders into `colorImages[stage]`.
|
||
|
||
## Open side-bug (parked)
|
||
One-time Seated-mode height bug: spawned very high at title, height oscillated high/normal
|
||
in-game; not replicable. Tracking-origin/recenter race at init (`ovrp_SetTrackingOriginType2`,
|
||
`ovrp_GetLocalTrackingSpaceRecenterCount` stubbed). Note `ReorientHMDOnControllerRecenter` is stubbed.
|