mirror of
https://github.com/daniel-lynch/ovrplugin-openxr-shim.git
synced 2026-10-06 02:00:05 +02:00
Initial public release: OVRPlugin→OpenXR interoperability shim
An independent reimplementation of Meta's libOVRPlugin ABI on top of OpenXR, so VrApi-era Meta Quest VR titles can run on non-Meta OpenXR runtimes (Monado, Steam Frame) instead of being locked to Meta hardware. Original code only — no Meta/Epic/Capcom binaries, headers, or assets. Includes a desktop harness that drives the shim against Monado headless. Scope/legal: interoperability; entitlement handling is out of scope. See README for the legal/scope section and docs/ for the research trail and design notes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01D6sFYGXZPsq3v7xtcDES6g
This commit is contained in:
Daniel LynchandClaude Opus 4.8 committed
commit
a72a79ad29
57 files changed
+9304
No files matched your search
@@ -0,0 +1,43 @@
|
||||
# RE4 VR shim — docs
|
||||
|
||||
Research trail and session handoffs for the OVRPlugin→OpenXR shim that runs RE4 VR
|
||||
on non-Meta OpenXR runtimes (Quest 2 today; Steam Frame the goal).
|
||||
|
||||
- **`handoffs/`** — chronological session handoffs (the day-by-day journey).
|
||||
- **`research/`** — reverse-engineering notes, design docs, the ghost-fix writeup, and
|
||||
`related-work.md` (how Overport/others run Quest games elsewhere; why this shim is novel).
|
||||
- `../analysis/` — raw RE dumps (`*.txt`, mostly gitignored; `all_exports.txt` +
|
||||
`shim_surface.txt` feed `shim/gen_stubs.sh`).
|
||||
- Operational docs live at repo root: `README.md`, `TESTING.md`, `HOST.md`.
|
||||
|
||||
## The "ghost" — how it was actually solved (the short version)
|
||||
|
||||
For weeks the headline bug was a **VR "ghost"**: doubled/tripled hands and watches,
|
||||
seated-mode deform, standing-mode black flashes. It survived every theory it *looked*
|
||||
like — stereo geometry, FOV/IPD, depth reprojection, render-submit ordering, GPU load.
|
||||
|
||||
The break came from building **P4 passthru** (`research/ghost-fix-2026-06-27.md`):
|
||||
running the real Meta `libOVRPlugin` *inside* our app proved native is flawless, so the
|
||||
bug was in **our path**. Then always-on anomaly logging caught the real signature —
|
||||
**dropped frames** — and a recording + frame-blending showed it was **whole-frame
|
||||
temporal judder**, not a stereo/geometry artifact. Localizing the stall showed UE's
|
||||
render thread blocking ~85–150ms during motion at only ~14ms of GPU work.
|
||||
|
||||
Root cause: the shim left `ovrp_WaitToBeginFrame` a **no-op** and ran `xrWaitFrame` on
|
||||
the **render** thread, so the game thread was unpaced and UE's pipelined renderer
|
||||
desynced → render-thread stalls → dropped frames → the compositor's timewarp filled the
|
||||
gaps → judder. **Fix: pace the game thread like vrapi** (`xrWaitFrame` in
|
||||
`xrr_wait_frame`, frameState handed to the render thread via a 1:1 FIFO ring). Result:
|
||||
render stall 150→3ms, drops 0.79% (~native 0.34%), steady 72Hz — ghost gone.
|
||||
Commits `79476c3` (fix) + `c0ecff3` (XR_FRAME_DISCARDED guard).
|
||||
|
||||
Secondary wins kept as defaults: game-driven FFR (apply the game's foveation), and the
|
||||
per-frame luma readback off.
|
||||
|
||||
## Reading order if you're new
|
||||
|
||||
1. `research/ghost-fix-2026-06-27.md` — the full diagnosis chain (start here).
|
||||
2. `handoffs/` newest → oldest — the journey, including the dead ends (reproject,
|
||||
submit-ordering, depth) that were ruled out.
|
||||
3. `research/RECON.md`, `RE-NOTES.md`, `SHIM-SCOPE.md` — how the shim was reverse-engineered.
|
||||
4. `research/render-submit-sync-design.md` — the render/submit sync design.
|
||||
@@ -0,0 +1,74 @@
|
||||
# RE4 VR Shim — Handoff (2026-06-24 session)
|
||||
|
||||
Entry point for the next session. Detailed investigation log: `NEXT-SESSION-LATENCY.md`.
|
||||
|
||||
## TL;DR
|
||||
- The head-motion **ghosting is frame drops** caused by the synchronous tile-memory
|
||||
**flush-wait serializing CPU↔GPU** (proven: removing it on the title dropped frame
|
||||
time 13–36ms → ~3ms and killed the ghosting).
|
||||
- **Shipped fix (always on): perf levels.** We were no-op'ing the game's CPU/GPU level
|
||||
requests; now forwarded via `XR_EXT_performance_settings`. Confirmed on device: GPU
|
||||
clock 305→587MHz, CPU steady 2419MHz, framerate up. The default build is playable
|
||||
(synchronous path + perf fix).
|
||||
- **Copy-ring (the real serialization fix) is built but blocked** on an MSAA-resolve
|
||||
interaction in gameplay — see below. Default OFF (`debug.re4vr.copyring`).
|
||||
- **Caveat — the title still ghosts INITIALLY in every build we tried.** Default build:
|
||||
title ghosting reduced by the perf fix but not gone. Copy-ring on: title ghosts at
|
||||
first, then "looks good after a while" (settles, not clean from frame 1). Likely
|
||||
cause of the *initial* ghosting in both: the title's **asset streaming / load hitches**
|
||||
(the ~49ms `CPU&GPU` spikes in VrApi, Stale frames), which NONE of the fixes address.
|
||||
So: perf fix + copy-ring attack the steady-state flush-wait drops; the initial
|
||||
load-driven hitching is a separate, still-open cause.
|
||||
|
||||
## Build / deploy
|
||||
```
|
||||
./shim/build_android.sh && ./packaging/repack.sh && adb install -r packaging/out/re4vr-shim.apk
|
||||
adb logcat -d -s xrr # shim logs
|
||||
```
|
||||
Device = Quest 2 (USB serial <redacted-serial>, or wireless <redacted-ip>:5555).
|
||||
Package `com.Armature.VR4`, activity `com.epicgames.ue4.GameActivity`.
|
||||
|
||||
## Runtime debug toggles (system props; relaunch unless noted)
|
||||
- `debug.re4vr.copyring 1` — shim copy-ring (eye-only; gameplay black, see below)
|
||||
- `debug.re4vr.sscap 1` — cap supersample 1.2x→1.0x (free GPU/bandwidth)
|
||||
- `debug.re4vr.diag 1|2|3` — 1=projection only(drop quads), 2=force mono, 3=2x zoom (live)
|
||||
- `debug.re4vr.dump N` — one-shot eye-texture readback to /sdcard/Android/data/com.Armature.VR4/files/
|
||||
- `debug.re4vr.depth 1` — DEAD END (UE passes None; Meta ignores plain KHR depth)
|
||||
- `debug.re4vr.pipeline 1` — DEAD END (render-ahead by holding OpenXR images; breaks UE stage lockstep)
|
||||
- `debug.re4vr.noflushwait 1` — perf probe: skip flush-wait (renders black; FPS recovers → confirms serialization)
|
||||
|
||||
## The open problem: copy-ring gameplay = MSAA resolve into our shim
|
||||
RenderDoc (offline replay over USB) showed:
|
||||
- UE renders the eye with **2× MSAA** into its own target, render pass **resolve
|
||||
attachment = our copy-ring shim** (the image we copy to OpenXR). No stage mismatch.
|
||||
- On the **title** that resolve lands → copy-ring works. In **gameplay** the resolved
|
||||
shim isn't the scene at our copy (white/garbage) → black/flashing.
|
||||
- So: UE's MSAA resolve into our **externally-created** shim VkImage works on the title
|
||||
but not in gameplay. Next suspects: the shim's `MUTABLE_FORMAT`/sRGB-vs-UNORM resolve
|
||||
view, create flags vs a valid resolve-dst, or Adreno tile-MSAA specifics. Single-shim
|
||||
made it worse (UE needs distinct per-stage images for TAA/buffering).
|
||||
- The fundamental tension: removing the flush-wait needs pipelining, which breaks UE's
|
||||
1:1 stage↔acquire lockstep (the synchronous path relies on it). Copy-ring decouples
|
||||
via shim images but inherits the MSAA-resolve issue.
|
||||
|
||||
## RenderDoc offline-replay pipeline (set up this session — no headset needed to analyze)
|
||||
- App must be **debuggable**: `tools/apktool.jar` decode → add `android:debuggable="true"`
|
||||
→ rebuild → sign (debug.keystore) → `packaging/out/re4vr-dbg.apk`. (RE-uses the shim.)
|
||||
- RenderDoc server installed (`org.renderdoc.renderdoccmd.arm64`). Capture over **USB**
|
||||
(wireless adb drops the replay; local desktop replay of an Android capture fails).
|
||||
- Headless analysis: `qrenderdoc --python script.py` with
|
||||
`adb://<usb-serial>` → CreateRemoteServerConnection → CopyCaptureToRemote →
|
||||
remote.OpenCapture. Scripts + captures in `~/renderdoc-captures/` (rd_*.py,
|
||||
RE4/black.rdc = gameplay, RE4/title.rdc). Use `action.outputs` for RT attribution and
|
||||
SaveTexture (NOT GetMinMax/Typeless — it lies) for content.
|
||||
|
||||
## Recommended next steps
|
||||
1. Chase the MSAA-resolve-into-shim failure (RenderDoc pipeline is ready): compare the
|
||||
title resolve (works) vs gameplay (fails) render-pass setup; try shim WITHOUT
|
||||
MUTABLE_FORMAT, or matching UE's exact resolve-target image create info.
|
||||
2. If copy-ring stays blocked, the perf fix is the shipped win; consider lighter levers
|
||||
(sscap) and accept residual menu-screen ghosting.
|
||||
|
||||
## Notes
|
||||
- Diagnostic logging is still in the shim (rate-limited; strip before public release).
|
||||
- Depth format mapping was corrected (None=10; D16=6/D24_S8=7/D32_FP=8/D32_S824=9).
|
||||
@@ -0,0 +1,123 @@
|
||||
# RE4 VR Shim — Handoff (2026-06-25 session)
|
||||
|
||||
Entry point for next session. Prior handoff: `HANDOFF-2026-06-24.md`.
|
||||
Detailed per-finding log lives in the auto-memory (`MEMORY.md` index).
|
||||
|
||||
## TL;DR
|
||||
- **Root cause of the black flashes / 3×-hand / stutter is one thing: a GPU-load-gated
|
||||
render-submit race.** Under load UE 4.25's RHI thread `vkQueueSubmit`s the eye render
|
||||
*after* our `xrEndFrame`, so we present a stale/empty (black) swapchain image. PROVEN by
|
||||
texture dump (gameplay eye image `min=max=0`, pure black) and confirmed load-gated
|
||||
(in-game **Standing** comfort setting flashes black; **Sitting** doesn't — higher camera =
|
||||
more visible geometry = blows frame budget).
|
||||
- **Fixes tried this session:**
|
||||
- `vkQueueWaitIdle` before resolve (`debug.re4vr.qwait`) — **no effect** (UE hadn't
|
||||
submitted yet; can't drain unsubmitted work).
|
||||
- **Deferred-flush pipeline** (`debug.re4vr.pipeline=1`, rewrote the render-ahead path to
|
||||
hold frame N and flush+present at N+1 after UE submits) — reduced *pure* black but
|
||||
**crashes more + adds latency**, marginal. **SHELVED.** Pipeline path has been the
|
||||
unstable one all along.
|
||||
- **FFR** (`debug.re4vr.ffr`, NEW this session) — implemented via `XR_FB_foveation`
|
||||
(dynamic). Validated (game uses foveation natively) and helps headroom/smoothness, but
|
||||
**not enough** to bring Standing under budget alone.
|
||||
- **RE of the original libs confirmed:** `ovrp_EndFrame4` does **zero GPU sync** — VrApi's
|
||||
system-owned textures carry it. Nothing to copy; our explicit-barrier approach is right,
|
||||
we just have a timing/threading mismatch. OpenXR gives the same free sync IF we
|
||||
submit-before-release (portable; the Quest-specific path would NOT port to Steam Frame).
|
||||
- **NEXT (the real lever):** re-enable the **game's native dynamic-performance subsystem**
|
||||
that our stubs disable (see below). AppSW turned out to be a weaker lead — RE4 doesn't
|
||||
drive it.
|
||||
|
||||
## PRIMARY next task — re-enable the game's dynamic-perf machinery
|
||||
RE4 was built to scale its own GPU load under pressure, but we stub the whole subsystem as
|
||||
unsupported (logcat, all fire ~8×/session):
|
||||
```
|
||||
ovrp_GetGPUFrameTime -> Unsupported (game polls GPU frame time to decide)
|
||||
ovrp_GetTiledMultiResLevel -> Unsupported (TiledMultiRes = Oculus name for FFR)
|
||||
ovrp_SetTiledMultiResLevel -> Unsupported
|
||||
ovrp_SetTiledMultiResDynamic -> Unsupported (dynamic FFR driven by GPU time)
|
||||
ovrp_IsPerfMetricsSupported -> Unsupported
|
||||
ovrp_GetSystemRecommendedMSAALevel2 -> NotYetImplemented
|
||||
ovrp_GetLayerTextureFoveation -> NotYetImplemented (still stubbed even though we apply FFR at OpenXR)
|
||||
```
|
||||
Plan:
|
||||
1. **Report TiledMultiRes supported + honor `Set*` calls** by mapping the game's requested
|
||||
FFR level/dynamic onto our OpenXR `apply_foveation()` (xr_runtime.c). Today FFR is a
|
||||
static prop (`debug.re4vr.ffr`); instead let the GAME drive the level (it already wants
|
||||
dynamic, GPU-time-based). Stub fns in `shim/src/stubs.c`; real handlers go in
|
||||
xr_runtime.c next to `apply_foveation`.
|
||||
2. **Implement `ovrp_GetGPUFrameTime`** so the game's dynamic loop has its input. Source:
|
||||
OpenXR has no portable GPU-time query; options = `XR_FB_... ` perf ext, or feed our own
|
||||
measured frame dt (we already compute it for the `FRAME`/`HITCH` trace). Even an
|
||||
approximate value lets the game's scaler engage.
|
||||
3. Re-check Standing after: if the game drops its own resolution/foveation under load and
|
||||
the black stops, this is the fix. It's the game's *intended* GPU-bound mitigation.
|
||||
|
||||
## SECONDARY — Application SpaceWarp (verify first, likely not viable as-is)
|
||||
- `ovrp_GetLayerTextureSpaceWarp` is **stubbed but RE4 does NOT call it** this session; UE
|
||||
passes a **108-byte EyeFov desc, flags=0** (no motion-vector variant). So UE isn't
|
||||
rendering motion vectors → AppSW can't be fed.
|
||||
- The **system** compositor IS running SpaceWarp/ASW (logcat tag `SpaceWarp`/`SpaceWarpCore`,
|
||||
PID 2438) — that's compositor-side reprojection (no app MV needed). It may already be
|
||||
reprojecting; our black frames defeat it (can't extrapolate a black frame).
|
||||
- IF you pursue AppSW: needs (a) game to render velocity (UE 4.25 may not support it without
|
||||
the Meta fork), (b) provide MV+depth swapchains via `ovrp_GetLayerTextureSpaceWarp`,
|
||||
(c) submit `XrCompositionLayerSpaceWarpInfoFB` (chained on the projection view) via
|
||||
`XR_FB_space_warp` (extension + types already in bundled headers, openxr.h:6021+;
|
||||
`XrSystemSpaceWarpPropertiesFB` gives recommended MV size). **Verify (a) before investing.**
|
||||
|
||||
## Current build / stable config
|
||||
Build/deploy:
|
||||
```
|
||||
./shim/build_android.sh && ./packaging/repack.sh
|
||||
adb -s <redacted-serial> install -r packaging/out/re4vr-shim.apk
|
||||
adb -s <redacted-serial> logcat | grep -i xrr
|
||||
```
|
||||
Device Quest (USB serial `<redacted-serial>`, or `<redacted-ip>:5555`).
|
||||
Pkg `com.Armature.VR4`, activity `com.epicgames.ue4.GameActivity`.
|
||||
**STABLE config left set:** `pipeline=0` (deferred-flush OFF), `ffr=2` (medium), `sscap=0`.
|
||||
Consider compiling the deferred-flush path out / default FFR medium for a clean baseline.
|
||||
|
||||
## Runtime debug toggles (system props; relaunch unless noted "live")
|
||||
- `debug.re4vr.ffr 0|1|2|3` — NEW. Fixed Foveated Rendering off/low/med/high (XR_FB_foveation, dynamic). Default 2.
|
||||
- `debug.re4vr.pipeline 1` — deferred-flush render-ahead (live). UNSTABLE/crashes; shelved.
|
||||
- `debug.re4vr.qwait 1` — diag: vkQueueWaitIdle before resolve (live). No effect; keep for reference.
|
||||
- `debug.re4vr.trace 1|2` — NEW. 1=per-frame FRAME line (dt+HITCH) + starvation logs; 2=per-call WAIT/BEGIN/END + per-layer dump (live).
|
||||
- `debug.re4vr.sscap 1` — cap supersample 1.2x→1.0x.
|
||||
- `debug.re4vr.copyring 1` — copy-ring (eye-only; gameplay black). From prior session.
|
||||
- `debug.re4vr.diag 1|2|3|4` — visual A/B: 1=proj only, 2=mono, 3=zoom, 4=head-lock quad (live).
|
||||
- `debug.re4vr.dump N` — eye-texture readback to /sdcard/Android/data/com.Armature.VR4/files/*.ppm. **UNRELIABLE probe — see below.**
|
||||
- `debug.re4vr.depth 1`, `debug.re4vr.noflushwait 1` — prior-session probes.
|
||||
|
||||
## What's RULED OUT for the black (don't re-investigate)
|
||||
Quad placement/presence (diag=1/4), ViewportRect (UE always submits full), empty-frame
|
||||
submission (`SUBMIT-BLACK=0` over 6730 frames), head-vs-eye pose mismatch (poses match
|
||||
exactly), stereo layout/IPD (array, correct), pure frame hitching (pipeline cut hitches
|
||||
804→39 but black persisted). It is specifically the **render-submit timing race**, GPU-load-gated.
|
||||
|
||||
## Diagnostic recipes (USE THESE — earlier probes misled us)
|
||||
- **Perception is ground truth.** The `dump` probe is UNRELIABLE: it reads stale swapchain
|
||||
buffers AND its fence-wait *forces* the render (observer effect — "dumping forced the load").
|
||||
- **Recordings: limited-range.** `YAVG=16` IS black (not grey). 30fps mono UNDERSAMPLES 72Hz
|
||||
flashes (lower bound). Separate sustained black (menu/scene transitions) from brief isolated
|
||||
flashes. ffmpeg: `-vf "signalstats,metadata=print:key=lavfi.signalstats.YAVG:file=y.txt"`;
|
||||
blend with `tmix=frames=N` to reveal ghosting; search the WHOLE clip, not the start.
|
||||
- **Frame timing (reliable, not undersampled):** `debug.re4vr.trace 1` → grep `HITCH` / `FRAME`.
|
||||
- **RE pipeline:** `JAVA_HOME=tools/jdk-21.0.11+10 tools/ghidra_12.1.2_PUBLIC/support/analyzeHeadless
|
||||
ghidra_proj NAME -import dump/apk_libs/lib/arm64-v8a/libOVRPlugin.so -scriptPath ghidra_scripts
|
||||
-postScript DumpEndFrame.java -deleteProject`. Original libs extracted in `dump/apk_libs/`.
|
||||
|
||||
## Key code locations (xr_runtime.c unless noted)
|
||||
- Eye-fov submit / build_composition (~560), the resolve barrier path is in `vk_session.c`
|
||||
(`xrr_vk_flush_submit_ex`, barrier COLOR_ATTACHMENT_WRITE→MEMORY_READ; NO QueueWaitIdle anywhere).
|
||||
- `submit_pending` (~740): where a black frame reaches the compositor (`SUBMIT-BLACK` log).
|
||||
- Pipeline (deferred-flush) branch in `xrr_end_frame` (~900); sync branch (~960).
|
||||
- `apply_foveation` / `ffr_level` (just before `xrr_setup_layer`, ~1148) — FFR; extend here
|
||||
to let the game drive the level (TiledMultiRes handlers).
|
||||
- Capability stubs to implement: `shim/src/stubs.c` (GetGPUFrameTime, TiledMultiRes*, etc.).
|
||||
- Texture handoff: `xrr_get_layer_texture` (~1230) — UE renders into `colorImages[stage]`.
|
||||
|
||||
## Open side-bug (parked)
|
||||
One-time Seated-mode height bug: spawned very high at title, height oscillated high/normal
|
||||
in-game; not replicable. Tracking-origin/recenter race at init (`ovrp_SetTrackingOriginType2`,
|
||||
`ovrp_GetLocalTrackingSpaceRecenterCount` stubbed). Note `ReorientHMDOnControllerRecenter` is stubbed.
|
||||
@@ -0,0 +1,104 @@
|
||||
# RE4 VR Shim — Handoff (2026-06-26, Lever 2 reproject session)
|
||||
|
||||
Continuation of `HANDOFF-2026-06-26-skipblack.md`. Branch `latency-perffix-copyring`.
|
||||
This session **built and shipped the working in-game black fix (Lever 2 reproject)** and hit
|
||||
the tuning wall that points to a render-ahead pivot. Detail in auto-memory (MEMORY.md index):
|
||||
`luma-gate-works-bad-vs-benign-black`, `reproject-works-playable`,
|
||||
`reproject-tuning-and-renderahead-pivot`, `submit-count-cannot-detect-black`,
|
||||
`skipblack-blacks-are-sustained-onset-only`.
|
||||
|
||||
## TL;DR — the in-game black is FIXED (reproject), "1000x better than black, playable"
|
||||
The whole project's black-hunt is functionally solved. On a truncation-black frame we blit the
|
||||
held last-good eye image back over it and present with the SAVED pose; the compositor timewarps
|
||||
it to the current head pose instead of flashing black. Committed through `3522e5c`.
|
||||
|
||||
## What was built this session (commits a45362c..3522e5c)
|
||||
1. **Post-hitch detector + barcode validation tool** (a45362c, c2a6c29): instrument-only black
|
||||
flag; on-screen frameIndex barcode that turns red on a flagged frame for frame-exact video
|
||||
correlation (`debug.re4vr.barcode`).
|
||||
2. **Submit-count ruled out** (508f55a): UE issues a flat 3 vkQueueSubmits/frame on black AND
|
||||
normal frames — truncation is fewer DRAWS, not fewer submits. Dead end, don't retry.
|
||||
3. **Per-frame luma black gate** (571aa00): `xrr_vk_frame_luma` reads max luma of a few rows of
|
||||
the resolved eye image; covers the whole sustained-black TAIL (post-hitch only caught the
|
||||
onset). `debug.re4vr.lumagate`, `lumathr` (default 12). **Bad black = luma<thr AND
|
||||
appLayers==1** (appLayers>=2 = a menu's black eye-fov behind a UI quad — benign).
|
||||
4. **Reproject** (7c0882b): `debug.re4vr.reproject=1`. Copy-ring hold (`L->holdImage`, no
|
||||
cross-frame swapchain hold so no pipeline=1 instability). Save acquired->hold on good frames
|
||||
(with pose), restore hold->acquired on black frames, submit with saved pose
|
||||
(`g_lastGoodViews`, used in `build_composition`).
|
||||
5. **Tuning** (e3bbe85, 3522e5c): `savethr` (default 40, only hold clearly-lit frames — fixes
|
||||
dim reprojected frames); `abruptdrop` (default 0 = reproject all; raise to skip gradual
|
||||
fades); `holdevery` (default 4, throttle the save copy for perf).
|
||||
|
||||
## State of play (device-tested)
|
||||
- **Black flashes GONE, playable.** Stable, no crashes over multiple soaks.
|
||||
- **Perf ~17ms (~59fps).** Breakdown measured: resolve-wait ~7ms + luma ~3.5ms + save ~0.6ms.
|
||||
Folding luma into the resolve cmd buffer = WASH (luma's cost is a forced Adreno tile-resolve
|
||||
from the TRANSFER_SRC transition, not shareable wait overhead — tried + reverted). 72Hz needs
|
||||
attacking the 7ms resolve-wait = render-ahead.
|
||||
- **Remaining artifacts (all inherent to single-frame reprojection):** (a) reproject OVERRIDES
|
||||
the game's intentional fades (comfort/teleport/scene/load) — they're black too; abruptdrop
|
||||
helps but can't cleanly separate fade from shallow truncation. (b) long ~1s holds during
|
||||
loads are jarring. (c) head-rotation during a hold shows timewarp edge-black; depth-hold (not
|
||||
done) would add positional reprojection.
|
||||
|
||||
## NEXT — revisit RENDER-AHEAD (user's strategic call), but verify the premise first
|
||||
Why now: render-ahead adds 1 frame latency -> UE gets more GPU budget/frame -> FEWER/shorter
|
||||
truncations -> reproject (and its artifacts/long-holds) needed far less; and reproject+luma are
|
||||
now a SAFETY NET for render-ahead's old instability (detect+hide any black instead of flashing).
|
||||
- **GATE (do this first):** resolve the contradiction — memory `black-is-not-submit-timing`
|
||||
says UE renders empty regardless of timing, but the user observed black DROPPED when the luma
|
||||
readback (a per-frame GPU sync) turned on. Experiment: use the luma gate to COUNT
|
||||
appLayers==1 truncation-black frames per minute at different added-latency/sync levels (e.g.
|
||||
baseline vs lumagate-on vs an added sync). If latency reduces truncation frequency ->
|
||||
render-ahead is justified; if not -> stay with reproject + tuning + maybe depth-hold.
|
||||
- If justified: build render-ahead informed by the luma signal (present held frame only when
|
||||
needed, not blindly like the old `pipeline=1`), with reproject as the fallback. See
|
||||
`deferred-flush-unstable-abandoned` for the old failure modes (acquire/release imbalance).
|
||||
|
||||
## Game perf RE (done this session) — one new lever, several myths busted
|
||||
Static RE of the GAME binary `dump/apk_libs/lib/arm64-v8a/libUE4.so` via capstone. Full report:
|
||||
`analysis/game-perf-RE.md`; memory `game-perf-RE-findings`. This REPLACES the behavioral
|
||||
inferences in `game-doesnt-drive-dynamic-perf-from-feed` with code proof:
|
||||
- **No game-side dynamic-FFR loop exists** — `SetTiledMultiRes{Level,Dynamic}` args are config
|
||||
bytes (`SetTiledMultiResDynamic(Settings.byte[0x11d])`), never compared to GPU time/metrics. So
|
||||
shim-side dynamic FFR is the ONLY path (nothing we return makes the game loop). Boot `Dynamic(0)`
|
||||
is just that config value.
|
||||
- **Perf-metrics gate hypothesis REFUTED** — game never gates FFR on `IsPerfMetricsSupported`;
|
||||
perf-metrics feed only a stats/HUD collector. Commit 68a7093 unlocked no behavior.
|
||||
- **AppSW dead** (`GetLayerTextureSpaceWarp` not even a string in the binary).
|
||||
- **`GetSystemRecommendedMSAALevel2` = hazard** (over-report -> more MSAA -> more load). Keep at 1.
|
||||
|
||||
### NEW game-driven lever — implement `ovrp_GetAdaptiveGpuPerformanceScale2` (do next session)
|
||||
At its call site the game does `resolution = baseDensity * sqrt(scale)` where `scale` comes from
|
||||
this API (table slot g+0x130). We return Unsupported -> scale=1.0 -> "viewport always full under
|
||||
load". **If the shim returns `scale < 1.0` under GPU pressure (derive from `g_gpuFrameMs` vs the
|
||||
13.9ms budget), the game self-downscales resolution — NO game patch.** Gated on `Settings.byte[0x19]
|
||||
bit4` (adaptive-res enabled) — VERIFY on device. CAVEAT: this is a GPU-LOAD (pixel) lever; the
|
||||
truncation is DRAW-bound (quarter-res still blacks), so it's a throughput/comfort/headroom win that
|
||||
*might* indirectly slacken the render thread — NOT a guaranteed truncation fix. Easy + game-native;
|
||||
worth stacking, measure truncation effect with the luma counter. (Find the API's stub in
|
||||
`shim/src/stubs.c`; we already own `g_gpuFrameMs` + apply-foveation path.)
|
||||
|
||||
## Secondary follow-ups (the user "likes them all")
|
||||
- **Depth-hold** (artifact reduction): enable depth (works on device, gameplay renders), hold
|
||||
+restore depth alongside color, chain it (`build_composition` already chains depth) -> proper
|
||||
positional reprojection. Helps the head-rotation swim/edge-black.
|
||||
- **Verify the ghosting-fixed-by-readback** observation independently.
|
||||
|
||||
## Build / deploy / config
|
||||
```
|
||||
./shim/build_android.sh && ./packaging/repack.sh # repack bundles LAST build — check timestamp
|
||||
adb -s <redacted-serial> install -r packaging/out/re4vr-shim.apk
|
||||
adb -s <redacted-serial> logcat | grep -iE "xrr|REPROJECT"
|
||||
```
|
||||
Clean device state set this session: `reproject=1 holdevery=4 savethr=40 abruptdrop=0 depth=0
|
||||
trace=0 barcode=0`. Launch blocked by controllers-asleep dialog — wake controllers first.
|
||||
|
||||
## Key reproject toggles (system props, re-read each frame unless noted)
|
||||
- `debug.re4vr.reproject 1` — the fix (implies luma readback).
|
||||
- `debug.re4vr.lumathr N` (12) — black threshold; `savethr N` (40) — min luma to HOLD a frame.
|
||||
- `debug.re4vr.abruptdrop N` (0=reproject all; ~25 skips fades but misses shallow truncations).
|
||||
- `debug.re4vr.holdevery N` (4) — save last-good every Nth frame (perf throttle).
|
||||
- `debug.re4vr.barcode 1` — on-screen frameIndex barcode (red = reprojected frame), validation.
|
||||
- `debug.re4vr.depth 1` — experimental depth swapchain (renders OK; not yet held for reproject).
|
||||
@@ -0,0 +1,81 @@
|
||||
# RE4 VR Shim — Handoff (2026-06-26, skipblack instrument session)
|
||||
|
||||
Continuation of `HANDOFF-2026-06-26.md` (morning). Branch `latency-perffix-copyring`.
|
||||
This session built + ran **Lever 2 step 1 (instrument-only black-frame detector)** and
|
||||
validated the signal is real but coarse. Detailed finding in auto-memory
|
||||
`skipblack-posthitch-signal-validated`. Design: `analysis/lever2-detect-skip-black-handoff.md`.
|
||||
|
||||
## TL;DR — the post-hitch black signal is REAL but needs a tighter gate + frame-accurate proof
|
||||
Added `debug.re4vr.skipblack 1` (instrument-only): after a HITCH (dt>20ms), flag the next
|
||||
`skipblackn` frames as LIKELY-TRUNCATED and log them. NO behavior change at level 1. Ran on
|
||||
device (standing, walk off bridge into house). 47 HITCHes / 89 flagged frames. Result:
|
||||
|
||||
- **The truncated/black frame has a signature: a SHORT (<12ms) frame right after a big hitch**
|
||||
— it finishes fast because UE rendered almost nothing into it (matches RenderDoc 28-vs-198
|
||||
draws → pure black).
|
||||
- **Big hitches (≥40ms): 25, of which 52% have that <12ms truncated recovery frame.**
|
||||
- **Mild hitches (20–40ms): 22, only 5%** — these recover to a full ~14ms frame = a 1-frame
|
||||
judder, NOT black. The current heuristic OVER-flags these.
|
||||
- HITCH magnitudes span 20ms → 3.8s. The multi-hundred-ms / multi-second ones are
|
||||
**load/streaming stalls** (level transitions) — a different event from per-frame gameplay
|
||||
black drops. Must be separated from the gameplay black.
|
||||
- Ground truth is still COARSE: user confirmed "black" happened, but not frame-accurate; the
|
||||
short-dt=black link rests on the single RenderDoc capture, not per-frame verified.
|
||||
|
||||
Raw frame/flag sequence persisted: `analysis/skipblack-2026-06-26/frame-sequence.log`.
|
||||
|
||||
## NEXT TASK — do BOTH (user decision this session):
|
||||
### (3) Tighten the detector, then re-instrument
|
||||
- **Gate the arm on hitch ≥40ms** (drops the 22 harmless mild hitches; ~halves false flags).
|
||||
- **Add short-dt confirmation:** only treat a post-hitch frame as truncated if its own
|
||||
dt < ~12ms. Combines the two corroborating signals; cuts the ~48% of big hitches whose
|
||||
recovery wasn't short (those may be a different failure mode — investigate separately).
|
||||
- **Handle sustained-overload clusters** (e.g. f=3162–3173: ~10 consecutive hitches). The
|
||||
fixed "next 2 frames" model is wrong there — the whole burst is bad. Consider: stay armed
|
||||
while consecutive frames keep hitching, not a fixed count.
|
||||
- Re-run instrument-only and re-inspect (reuse the awk recipes from this session's analysis).
|
||||
- Code is all in `shim/src/xr_runtime.c`: `skipblack_level()`/`skipblack_count()` (~205) and
|
||||
the flag block inside the FRAME/HITCH trace (~1186, look for `SKIPBLACK: LIKELY-TRUNCATED`).
|
||||
|
||||
### (2) Frame-accurate ground truth via Meta Cast (do alongside / before reproject)
|
||||
- In-game recording is broken (empty mp4); display 0 is FLAG_SECURE so scrcpy/screenrecord
|
||||
can't see VR. Use **Meta Cast → desktop screen-record** while walking off the bridge.
|
||||
- Run with `skipblack=1 skipblackn=2 trace=1`, capture logcat with timestamps in parallel.
|
||||
- Align the video's visible black flashes to the `SKIPBLACK: LIKELY-TRUNCATED` log
|
||||
timestamps to PROVE (or refute) flag == perceived black, per frame. This is the
|
||||
`verify-before-theory` discipline — don't build the reproject on the coarse signal alone.
|
||||
|
||||
### THEN — Lever 2 step 2 (reproject, `debug.re4vr.skipblack 2`)
|
||||
Only after (2)+(3). Re-present the **last-good** eye image with the current pose so the
|
||||
OpenXR compositor timewarps it instead of showing black.
|
||||
- **Use the COPY-ring, NOT a cross-frame swapchain hold** — holding across frames is what
|
||||
sank `pipeline=1` (see memory `deferred-flush-unstable-abandoned`). Copy last-good eye →
|
||||
shim VkImage (`xrr_vk_alloc_images`/`shimImages[]`, copy path in end_frame ~922), blit it
|
||||
into the acquired image on a flagged frame, submit the composition with the CURRENT pose.
|
||||
- Low harm on false positives: reprojecting a stale frame ≈ what the compositor already does
|
||||
during the hitch, so over-flagging mild hitches is tolerable but wasteful — the ≥40ms gate
|
||||
keeps it clean.
|
||||
- Success = black flashes replaced by (at worst) brief reprojection judder, not black.
|
||||
|
||||
## Build / deploy / test (unchanged)
|
||||
```
|
||||
./shim/build_android.sh && ./packaging/repack.sh # repack bundles LAST build — check
|
||||
# shim/build/arm64/libOVRPlugin.so timestamp
|
||||
adb -s <redacted-serial> install -r packaging/out/re4vr-shim.apk
|
||||
adb -s <redacted-serial> logcat | grep -iE "xrr|SKIPBLACK"
|
||||
```
|
||||
Launch is BLOCKED by the controllers-asleep system dialog
|
||||
(`app_launch_blocked_controller_required`) — wake controllers + headset on before
|
||||
`am start -n com.Armature.VR4/com.epicgames.ue4.GameActivity`.
|
||||
Logcat is dominated by PerfMetrics polling — filter to `FRAME f=|HITCH|SKIPBLACK` for pacing.
|
||||
|
||||
## Props
|
||||
- `debug.re4vr.skipblack 0|1|2` — 0 off / 1 instrument-only (built) / 2 reproject (TODO).
|
||||
- `debug.re4vr.skipblackn N` — post-hitch frames to flag (default 2). Will change with the
|
||||
cluster-aware rewrite in (3).
|
||||
- Reset to baseline now: skipblack=0, skipblackn=0, trace=0, ffr=2, resscale=100.
|
||||
|
||||
## State
|
||||
- Code committed? **NO — `skipblack` detector is UNCOMMITTED** in the working tree
|
||||
(`shim/src/xr_runtime.c`). Commit it (instrument-only, gated off, safe) before further work.
|
||||
- Branch `latency-perffix-copyring`, clean except the skipblack edit + the new analysis dir.
|
||||
@@ -0,0 +1,74 @@
|
||||
# RE4 VR Shim — Handoff (2026-06-26 session)
|
||||
|
||||
Entry point for next session. Prior: `HANDOFF-2026-06-25.md`. Detailed findings in the
|
||||
auto-memory (`MEMORY.md` index). Commits this session: `68a7093`, `5fa59f4` (branch
|
||||
`latency-perffix-copyring`).
|
||||
|
||||
## TL;DR — the in-game black is a UE frame-drop, and we proved it
|
||||
After a long, thorough investigation, the in-game black is **conclusively a UE-internal
|
||||
frame-drop under load**, NOT a shim bug:
|
||||
- RenderDoc (`~/renderdoc-captures/RE4/work_frame.rdc`): under load UE renders only
|
||||
**~28 draws into its eye target vs ~198 in a normal frame** (a truncated frame), and the
|
||||
**resolved/presented eye = pure black** (MEAN [0,0,0]).
|
||||
- **It's draw / render-thread-time bound, NOT pixel-bound:** even quarter-res
|
||||
(`resscale=50`) + max FFR (`ffr=3`) still blacks. Resolution doesn't change draw count.
|
||||
- **Ruled out, decisively:** submit-timing (built a vkQueueSubmit hook + present-on-submit;
|
||||
even a full one-frame defer still blacks), image-targeting (LAYER MISMATCH = 0,
|
||||
stage==acquiredIndex), empty-frame submission (SUBMIT-BLACK/COMPOSE-EMPTY = 0),
|
||||
load-reduction (FFR + quarter-res).
|
||||
- Load-gated: bridge (sparse) never blacks; house/dense geometry blacks; worse while casting.
|
||||
|
||||
## NEXT TASK — Lever 2: detect the bad frame + reproject
|
||||
Full design in `analysis/lever2-detect-skip-black-handoff.md`. Idea: when UE hands us a
|
||||
truncated/black frame, DON'T present it — re-present the last-good eye image with the
|
||||
current pose so the OpenXR compositor reprojects (timewarp) instead of showing black.
|
||||
- **Bad-frame signal:** start with the **post-hitch heuristic** (the truncated frame is the
|
||||
recovery frame after a `dt>20ms` hitch we already detect). The vkQueueSubmit hook's
|
||||
submit-count probably WON'T distinguish bad frames (it's draw-bound; UE may batch).
|
||||
- **Re-present:** reuse the copy-ring infra (`xrr_vk_alloc_images`, `shimImages[]`, the
|
||||
copy path in end_frame) — hold a copy of the last-good eye image, blit it into the
|
||||
acquired image on a bad frame. Avoids the cross-frame-hold instability that sank
|
||||
`pipeline=1`.
|
||||
- **Discipline:** implement the detector as **instrument-only first** (log "likely
|
||||
truncated frame" after each hitch), validate the signal correlates with perceived black,
|
||||
THEN build the reproject. Gate behind `debug.re4vr.skipblack` (default 0).
|
||||
|
||||
## What was built this session (all committed, gated off by default)
|
||||
- **Dynamic-perf bridge** (`68a7093`): `ovrp_*TiledMultiRes*` (FFR) + `GetGPUFrameTime` +
|
||||
`IsPerfMetricsSupported`/`GetPerfMetrics{Float,Int}` forwarded to OpenXR. Re-enables RE4's
|
||||
perf APIs (were Unsupported). The game accepts them but does NOT self-scale from the feed.
|
||||
Fixed a NULL-foveation-profile runtime crash (use `XR_FOVEATION_LEVEL_NONE_FB`).
|
||||
- **UE vkQueueSubmit hook** (`5fa59f4`): dlsym-patch UE's exported global
|
||||
`VulkanDynamicAPI::vkQueueSubmit` -> trampoline -> `xrr_on_ue_submit`. Robust + reusable
|
||||
(e.g. for Lever 2 instrumentation). Gated: `debug.re4vr.submithook` 0/1/2.
|
||||
- **resscale** (`5fa59f4`): `debug.re4vr.resscale` = % of eye size. Doesn't fix the black;
|
||||
keep as a load knob.
|
||||
|
||||
## Settled side-findings (don't re-investigate)
|
||||
- **Title ghost = cold-load reprojection judder** (cold-launch only; self-resolves warm;
|
||||
single eye-fov layer, no quad). See memory `title-ghost-is-coldload-reprojection-judder`.
|
||||
- **Bridge "vignette" = the GAME's comfort vignette** (user toggled it off; not a shim bug).
|
||||
- **An in-game "crash" was an OOM SIGKILL** (lmkd), worsened by in-game recording (which is
|
||||
broken — empty mp4). Capture via Meta Cast -> desktop record instead (scrcpy/screenrecord
|
||||
can't see VR: display 0 is FLAG_SECURE). RenderDoc recipe: memory `renderdoc-remote-replay`.
|
||||
|
||||
## Build / deploy / config
|
||||
```
|
||||
./shim/build_android.sh && ./packaging/repack.sh
|
||||
adb -s <redacted-serial> install -r packaging/out/re4vr-shim.apk
|
||||
adb -s <redacted-serial> logcat | grep -i xrr
|
||||
```
|
||||
WARNING: `repack.sh` silently bundles the LAST successful build — confirm the build had no
|
||||
errors + check `shim/build/arm64/libOVRPlugin.so` timestamp before repack.
|
||||
Device Quest 2, USB serial `<redacted-serial>`. Pkg `com.Armature.VR4`, activity
|
||||
`com.epicgames.ue4.GameActivity` (launching needs controllers awake, else a system dialog
|
||||
blocks `am start`).
|
||||
Props reset to baseline: ffr=2, resscale=100, sscap=0, submithook=0, defern=3, trace=0
|
||||
(none fix the black — all cosmetic/diagnostic now).
|
||||
|
||||
## Key runtime toggles (system props; relaunch unless noted)
|
||||
- `debug.re4vr.submithook 0|1|2` — UE submit hook: 0 off, 1 instrument, 2 present-on-submit.
|
||||
- `debug.re4vr.defern N` — present-on-submit: present on Nth render submit (default 3).
|
||||
- `debug.re4vr.resscale N` — eye buffer = N% of native (default 100).
|
||||
- `debug.re4vr.ffr 0..3`, `debug.re4vr.sscap 1`, `debug.re4vr.trace 1|2` — FFR / supersample
|
||||
cap / frame+hitch trace. `debug.re4vr.pipeline 1` (deferred-flush, unstable, shelved).
|
||||
@@ -0,0 +1,97 @@
|
||||
# RE4 VR Shim — Handoff (2026-06-27, native-parity / P4 passthru)
|
||||
|
||||
Branch `latency-perffix-copyring`. Goal this arc: **get the shim to native (stock VrApi) parity.**
|
||||
Stock RE4 VR is flawless on this Quest 2; our OpenXR shim had black logos + eye-layer ghosting.
|
||||
Detailed findings in auto-memory (MEMORY.md): `native-parity-diagnosis`, `logo-black-is-layer-zorder`,
|
||||
`eye-ghost-submission-path-clean-need-p4`, `three-perf-levers-built`.
|
||||
|
||||
## SHIPPED + VERIFIED this session
|
||||
**Logo black FIXED — it was a layer z-order bug.** RE'd it: the 3 intro logos are Oculus splash
|
||||
layers (`OculusHMD::FSplash`); the game submits `[logo-quad, eye-fov]` in that order, and OpenXR
|
||||
composites strictly in array order (last = on top), so our opaque eye-fov drew ON TOP of the logo →
|
||||
black. VrApi treats eye-fov as the base regardless of order. Fix: `build_composition`
|
||||
(`shim/src/xr_runtime.c`) stable-partitions so projection layers go bottom, quads on top. Gated
|
||||
`debug.re4vr.eyebottom` (default ON). **Device-verified: logos now render.** A/B: `eyebottom 0`.
|
||||
|
||||
## STILL OPEN: eye-layer ghosting (hand deforms / "three hands", close objects, SEATED mode)
|
||||
Standing mode = black+reproject instead (load-gated, see `standing-vs-sitting-blackflash`); seated =
|
||||
clean ghost repro (no reproject confound). **Every shim submission metric is CLEAN** (all
|
||||
device-verified, see `eye-ghost-submission-path-clean-need-p4`): depth on didn't fix it; submit is
|
||||
18ms EARLY not late; VrApi `Stale=0`; pose render==submit (`g_xr.views`); FOV render==submit; ffr
|
||||
ruled out; mono cast shows a normal hand in sharp frames → it's a per-eye/stereo artifact we can't
|
||||
see from our side. **Hence P4.**
|
||||
|
||||
## NEXT TASK — build the full P4 passthru forwarding (DE-RISKED, viable)
|
||||
Run the REAL OVRPlugin inside our app and LOG native's actual per-eye poses/FOV/layer submission, to
|
||||
diff against our clean-but-ghosting values. Feasibility PROVEN: `PASSTHRU-PROBE: dlopen OK` — real
|
||||
lib + libvrapi load in our unofficial app, entry points resolve (probe in `core.c` `passthru_probe()`
|
||||
at PreInitialize3, gated `debug.re4vr.passthru_probe`).
|
||||
|
||||
**Already in place:** SONAME-patched real lib at `packaging/libs/arm64/libOVRPlugin_real.so`
|
||||
(`patchelf --set-soname libOVRPlugin_real.so`); `repack.sh` auto-bundles it when present (verified in
|
||||
the APK). `libvrapi.so` already rides along from the original APK.
|
||||
|
||||
**The build (all-or-nothing — real lib must own the whole session to run EndFrame4):**
|
||||
1. New `shim/src/passthru.c`: `dlopen("libOVRPlugin_real.so", RTLD_NOW|RTLD_LOCAL)` once; build a
|
||||
table of real fn pointers (dlsym each `ovrp_*` the game uses); expose `pt_active()` +
|
||||
`pt_<fn>()` accessors (or a single dispatch). Gate: `debug.re4vr.passthru` (default 0).
|
||||
2. In EVERY game-called export (the ~30 in `core.c`/`layers.c` + the ~12 stubs the game hits — see
|
||||
below), add at the top: `if (pt_active()) { ...optional log...; return real_ovrp_X(args); }`.
|
||||
Partial forwarding CRASHES mid-frame, so forward the COMPLETE called set. The game-called set =
|
||||
everything currently implemented in core.c/layers.c PLUS these stubs it invokes (from device log):
|
||||
`DestroyDistortionWindow2, SetupDisplayObjects2, SetReorientHMDOnControllerRecenter,
|
||||
SetClientColorDesc, SetAppEngineInfo2, SetAppCPUPriority2, InitializeMixedReality,
|
||||
GetViewportStencil, GetSystemRecommendedMSAALevel2, GetLocalTrackingSpaceRecenterCount,
|
||||
GetLayerTextureFoveation, GetControllerHapticsDesc2`. (Confirm the live set by grepping a passthru
|
||||
boot for any of OUR `stub ovrp`/impl logs that still fire — those are unforwarded calls.)
|
||||
3. LOG the ghost-relevant ones: `EndFrame4` (per-layer pose+fov+swapchain+flags),
|
||||
`GetNodePoseState3`(Eye L/R return), `GetNodeFrustum2`(return), `CalculateEyeLayerDesc2`,
|
||||
`WaitToBeginFrame`/`EndFrame4` timing. Forward these to real, log args/returns.
|
||||
4. Boot test: with `passthru 1`, does the game run NATIVE through our shim (looks like stock, no
|
||||
ghost)? If yes → capture native EndFrame4 per-eye poses/fov, **diff against our values** (ours are
|
||||
in the STEREO/HEADvsEYE logs). The delta is the ghost cause. If the game won't init native (Init5
|
||||
entitlement/session), fall back to static RE of the projection path.
|
||||
CAVEAT: in passthru our shim must NOT also init OpenXR — make `xrr_init`/frame-loop no-op when
|
||||
pt_active (real lib owns the session). Watch for double-init of VrApi/OpenXR.
|
||||
|
||||
## The 3 perf levers (built earlier this session — all DEAD ENDS, gated off)
|
||||
- **Adaptive-res** (`ovrp_GetAdaptiveGpuPerformanceScale2`, core.c): INERT — game's `Settings.byte
|
||||
[0x19] bit4` clear, never downscales. `debug.re4vr.adaptivescale`.
|
||||
- **Render-ahead** (`debug.re4vr.renderahead`): stable now (old pipeline=1 instability fixed) but
|
||||
does NOT reduce black (still 66-76% under load — black is draw/streaming-bound, not latency).
|
||||
BLACKCOUNT counter (`debug.re4vr.blackcount`) is the measurement tool.
|
||||
- **Depth-hold** (`debug.re4vr.depthhold`, needs `depth 1`): built, untested, ADDS memory pressure.
|
||||
- **Reproject remains the actual in-game black fix.** Corridor black = engine streaming/memory stall
|
||||
(lmkd thrash, Graphics 1.69GB), not GPU — confirmed; GPU knobs don't help. Streaming-throttle CVar
|
||||
is a documented-but-not-built future lever (config-injection DEAD — Shipping ignores external
|
||||
UE4CommandLine.txt; would need CVar memory-patch via `IConsoleManager`).
|
||||
|
||||
## Code state (UNCOMMITTED on branch) — RECOMMEND COMMIT before further work
|
||||
`git status`: M `core.c stubs.c vk_session.c xr_runtime.c xr_runtime.h packaging/repack.sh`; new
|
||||
`HANDOFF-2026-06-27*.md`, `save_backup/`, `packaging/libs/arm64/libOVRPlugin_real.so` (986KB binary —
|
||||
needed for passthru; confirm it's committed or staged). +383/-88 in shim. Contains: eyebottom fix
|
||||
(keep — verified), 3 perf levers (gated off), QUADLUMA + per-layer luma diag, passthru probe,
|
||||
depth-aware vk helpers (`xrr_vk_alloc_images_ex`/`xrr_vk_copy_submit_ex`).
|
||||
|
||||
## Build / deploy / device workflow
|
||||
```
|
||||
./shim/build_android.sh && ./packaging/repack.sh # repack auto-bundles libOVRPlugin_real.so
|
||||
adb -s <redacted-serial> install -r packaging/out/re4vr-shim.apk # -r over SAME-signed shim PRESERVES data (no OBB/save dance)
|
||||
adb -s <redacted-serial> logcat -G 16M # big buffer so brief windows don't rotate
|
||||
adb -s <redacted-serial> logcat | grep -iE "xrr|PASSTHRU"
|
||||
```
|
||||
- **Every cold relaunch needs a manual dismiss** of the UnOfficialApp dialog (debug-signed) and the
|
||||
ControllerRequired dialog if controllers asleep → autonomous cold-launch capture is blocked; ask
|
||||
the user to dismiss + boot. Logos play on cold start only.
|
||||
- **SAVE DISCIPLINE:** `adb pull /sdcard/Android/data/com.Armature.VR4/files/savegame00.sav` BEFORE
|
||||
any uninstall (uninstall wipes app data; cloud doesn't restore unofficial-app saves — lost one this
|
||||
session). Only stock<->shim swaps need uninstall; shim->shim is `install -r` (preserves data). OBB
|
||||
preserve trick: `adb shell mv /sdcard/Android/obb/com.Armature.VR4 /sdcard/obb_bak` before uninstall.
|
||||
- Stock original APK (pristine, real 907KB OVRPlugin) for A/B: `dump/obb/VR4-Android-Shipping-arm64.apk`.
|
||||
- Current device props: `eyebottom 1 reproject 1 depth 1 depthhold 0 renderahead 0 blackcount 1
|
||||
adaptivescale 1 passthru_probe 1` (set `passthru 1` for the new path once built).
|
||||
|
||||
## RE tooling (works)
|
||||
capstone 5.0.7 in python (NOT pyelftools — absent; parse ELF phdrs manually, see scratchpad
|
||||
disasm.py pattern). `nm -DC` for libUE4 dynsym. PluginWrapper offset→name mapping was UNRELIABLE;
|
||||
trust call-site offsets + the dynamic `stub ovrp` log signal instead.
|
||||
@@ -0,0 +1,105 @@
|
||||
# RE4 VR Shim — Handoff (2026-06-27, 3-levers + streaming-bound reframe)
|
||||
|
||||
Branch `latency-perffix-copyring`. Prior: `HANDOFF-2026-06-26-reproject.md`. Auto-memory:
|
||||
`three-perf-levers-built`, plus the prior reproject/black chain. This session built the 3
|
||||
forward perf levers, soak-tested them, and **reframed the in-game black as a level-streaming /
|
||||
memory-bandwidth stall (NOT GPU-bound)** — which retires the GPU levers and points at a UE
|
||||
async-streaming CVar as the only remaining cause-fix.
|
||||
|
||||
## What was built (uncommitted on the branch; compiles clean, gated, device-tested)
|
||||
1. **Lever A — dynamic resolution** `ovrp_GetAdaptiveGpuPerformanceScale2` (real impl in
|
||||
`core.c`, helper `xrr_adaptive_gpu_scale()` in `xr_runtime.c`, stub removed from `stubs.c`).
|
||||
Props `debug.re4vr.adaptivescale` (1), `adaptivefloor` (60). **DEVICE RESULT: inert** — zero
|
||||
`VIEWPORT SUB` under extreme load ⇒ game's `Settings.byte[0x19] bit4` is clear, game ignores
|
||||
our scale (the handoff caveat held).
|
||||
2. **Lever B — render-ahead + truncation counter.** Shared `reproject_resolved_eye()` now feeds
|
||||
both present paths; render-ahead via `debug.re4vr.renderahead 1` (pose-patches `g_pending` to
|
||||
last-good on a black held frame; clean engage/disengage drain + presentIndex guard).
|
||||
`BLACKCOUNT` counter (`debug.re4vr.blackcount`, default 1). **DEVICE RESULT: render-ahead is
|
||||
STABLE (old pipeline=1 instability fixed) but NOT a black cure** — still spiked to 66–76%
|
||||
black with `pipelined=1`. Confirms black is draw/CPU-bound, not latency-bound. Counter works
|
||||
perfectly and was the instrument for everything below.
|
||||
3. **Lever C — depth-hold** (`debug.re4vr.depthhold 1`, default 0; needs `depth 1`; sync path
|
||||
only). Depth-aware `xrr_vk_alloc_images_ex` / `xrr_vk_copy_submit_ex` (isDepth). **No crash
|
||||
from the depth barriers, but it ADDS graphics memory in the exact pressured zone — net
|
||||
negative for the streaming-stall black; left OFF.**
|
||||
|
||||
## THE REFRAME (this session's real finding) — black is a STREAMING/MEMORY stall
|
||||
User observation: the long black/reproject episode was **in a corridor next to a load-zone
|
||||
door** — which has *fewer* draws than the open areas that render fine. That contradicts
|
||||
"draw-bound." The trace agrees:
|
||||
- `dumpsys meminfo`: **Graphics 1.69 GB**, TOTAL RSS 2.87 GB (Quest 2). During the black window
|
||||
(02:06:41–45) the system logged `RescueParty: lmkd_native` + **`ActivityManager: Lost
|
||||
connection to lmkd`** — the low-memory killer daemon fell over. Severe memory thrash.
|
||||
- Black frames were at **near-normal 72Hz cadence** (dt ~21–24ms), only ~8 true hitches in 25s
|
||||
but 240–275/360 frames black ⇒ GPU keeping cadence while UE hands us **empty** frames ⇒ the
|
||||
**game thread is stalled (streaming the next zone in), not the GPU**.
|
||||
Conclusion: near a load door UE pre-streams the adjacent zone's textures → memory spike → page
|
||||
reclaim/lmkd thrash → game-thread stall → empty (black) eye frames. **GPU knobs (resolution,
|
||||
FFR, render-ahead) cannot help; they target a bottleneck that isn't the limiter.**
|
||||
|
||||
### Mitigation that DID help (memory, not pixels)
|
||||
Config `renderahead=0 reproject=1 depth=0 depthhold=0 sscap=1`: peak black **76% → ~16%**, most
|
||||
windows 0%. Note **`sscap` did nothing** (eye buffer stayed 1728×1900 — already the recommended
|
||||
size, no supersample to cap); the win came from **`depth=0` freeing graphics memory**. Cleanest
|
||||
possible proof it's memory-bound. Reproject hides the residual ⇒ playable.
|
||||
|
||||
**Recommended shipping config:** `renderahead=0 reproject=1 holdevery=4 savethr=40 abruptdrop=0
|
||||
depth=0 depthhold=0 blackcount=1 adaptivescale=1 adaptivefloor=60` (adaptivescale harmless even
|
||||
if inert).
|
||||
|
||||
## NEXT TASK — throttle UE async streaming via a CVar (the only remaining cause-fix)
|
||||
User's instinct: deprioritize the background stream-ahead, accept a longer load screen. Maps to
|
||||
UE4 CVars: **`s.AsyncLoadingTimeLimit`**, `s.LevelStreamingActorsUpdateTimeLimit`,
|
||||
`s.PriorityAsyncLoadingExtraTime`, `s.AsyncLoadingUseFullTimeLimit`, and memory-side
|
||||
`r.Streaming.PoolSize` / `r.Streaming.LimitPoolSizeToVRAM` / prefetch distance. Lowering the
|
||||
async time limits spreads streaming over more frames (less per-frame game-thread theft → smoother
|
||||
active scene, slower stream). Lowering the streaming pool caps the pre-spike.
|
||||
|
||||
### CHEAP path PROBED & DEAD (don't retry the same way)
|
||||
External `UE4CommandLine.txt` override is NOT read by this Shipping build. Tested
|
||||
`-execcmds="t.MaxFPS 20"` at both `/sdcard/UE4Game/VR4/UE4CommandLine.txt` and the app-scoped
|
||||
`/sdcard/Android/data/com.Armature.VR4/files/UE4Game/VR4/UE4CommandLine.txt`; **end_frame
|
||||
heartbeat stayed at 71 fps** (decisive — heartbeat = game render rate; VrApi FPS=72 is just the
|
||||
reprojecting compositor, ignore it). The engine only uses the baked `assets/UE4CommandLine.txt`
|
||||
(`../../../VR4/VR4.uproject`). Minor caveat: `t.MaxFPS` *could* be VR-overridden, but two paths
|
||||
+ heartbeat make "file not read" the strong read.
|
||||
- **Secondary cheap probe worth ONE try before memory-patching:** `GameUserSettings.ini`
|
||||
scalability — `GGameUserSettingsIni` is read from a *writable* app-scoped Saved/Config path.
|
||||
Write `[ScalabilityGroups] sg.TextureQuality=0 / sg.ViewDistanceQuality=1` and see if the
|
||||
texture-streaming pool shrinks (watch `dumpsys meminfo` Graphics + BLACKCOUNT in the corridor).
|
||||
Uncertain (game may overwrite settings via its own UI), but declarative if it sticks.
|
||||
|
||||
### ROBUST path — set the CVar from the shim (the actual handoff task)
|
||||
The console-variable machinery is present and several helpers are **exported (`T`)** in
|
||||
`dump/apk_libs/lib/arm64-v8a/libUE4.so`:
|
||||
- `_Z22CreateConsoleVariablesv`, `_Z27ForEachCVarInSectionFromIni...`, and (seen via strings)
|
||||
`LoadConsoleVariablesFromINI`, plus `IConsoleVariable` / `IConsoleManager` infra and
|
||||
`OnCVarChange`. Kismet `GetConsoleVariable{Int,Float,Bool}Value` exist too.
|
||||
- **Plan:** after UE init (we already dlsym-patch UE's exported globals for the submit hook — same
|
||||
technique), resolve `IConsoleManager::Get()` then `FindConsoleVariable("s.AsyncLoadingTimeLimit")`
|
||||
and call `->Set("2", ECVF_SetByCode)`. RE steps: (a) find `IConsoleManager::Get` (likely a local
|
||||
symbol — use the full symbol table / capstone like the game-perf RE; `CreateConsoleVariablesv` and
|
||||
the `RegisterConsoleVariable` callsites reference the manager singleton). (b) confirm the
|
||||
`IConsoleVariable::Set(const TCHAR*, EConsoleVariableFlags)` vtable slot ABI. (c) call it for the
|
||||
3–4 streaming CVars at a safe point (first frame). Gate behind `debug.re4vr.streamthrottle`.
|
||||
- **Verify:** corridor soak, `BLACKCOUNT` delta + `dumpsys meminfo` Graphics, and confirm load
|
||||
transitions still complete (just slower). The counter is already the measurement tool.
|
||||
|
||||
## IS THIS NECESSARY OR OVER-ENGINEERING? (honest call)
|
||||
**Leaning over-engineering for the value.** The streaming-stall black is already well-mitigated
|
||||
(reproject + depth-off: 76%→16% peak, mostly 0%, residual hidden as a brief hold). The CVar
|
||||
memory-patch is invasive RE (console-manager ABI, fragile across game updates) for a marginal win
|
||||
on a transient that's already hidden. **Recommend: stop at the memory-light + reproject config,
|
||||
document the CVar route as a known lever, and only build it if the corridor reproject-holds are
|
||||
perceptually annoying enough to the user to justify it.** Try the GameUserSettings probe first
|
||||
(cheap) if pursuing further.
|
||||
|
||||
## Build / deploy / config
|
||||
```
|
||||
./shim/build_android.sh && ./packaging/repack.sh # check libOVRPlugin.so timestamp before repack
|
||||
adb -s <redacted-serial> install -r packaging/out/re4vr-shim.apk
|
||||
adb -s <redacted-serial> logcat | grep -iE "xrr|REPROJECT|BLACKCOUNT|VIEWPORT SUB"
|
||||
```
|
||||
Files changed this session: `core.c`, `stubs.c`, `vk_session.c`, `xr_runtime.c/.h` (+316/−89).
|
||||
Not yet committed.
|
||||
@@ -0,0 +1,381 @@
|
||||
# RE4 VR Shim — Next Session Handoff: Motion Reprojection / Latency
|
||||
|
||||
**Date locked:** 2026-06-23. **State:** RE4 VR is **playable** on Quest 2 via the
|
||||
OVRPlugin→OpenXR shim. One known issue remains: **ghosting/duplication during head
|
||||
movement** (and the title/menu "duping" — same root cause). This doc is the plan to
|
||||
fix it.
|
||||
|
||||
Deploy any change with: `./shim/build_android.sh && ./packaging/repack.sh && adb install -r packaging/out/re4vr-shim.apk`
|
||||
Logs: `adb logcat -d -s xrr`. Device = Quest 2 over wireless adb (`adb connect <redacted-ip>:5555`).
|
||||
|
||||
---
|
||||
|
||||
## What works (do not regress)
|
||||
- Rendering is correct & stable when the head is still; stereo/IPD/FOV/poses all verified.
|
||||
- **Tile-memory flush barrier** (`src/vk_session.c xrr_vk_flush_image`) — THE fix for the
|
||||
black/tearing. Quest's Adreno is a tiler; UE's eye render must be flushed to main
|
||||
memory + made visible before the compositor reads. We submit a `VkImageMemoryBarrier`
|
||||
on UE's queue in `xrr_end_frame` before `xrReleaseSwapchainImage`.
|
||||
- **Floor tracking** = `XR_REFERENCE_SPACE_TYPE_LOCAL_FLOOR` when the game requests
|
||||
FloorLevel (gun sits on hip). Eye-level sinks you into the floor; STAGE reverses movement.
|
||||
- Menus placed at the game's real submitted pose/size (respect `ovrpLayerSubmitFlag_HeadLocked`).
|
||||
- `DestroyLayer` implemented (+ slot reuse); finite swapchain-wait timeout (no hangs).
|
||||
|
||||
## The remaining problem (precisely)
|
||||
- Symptom: scene/menu **renders correctly but duplicates in two places during head
|
||||
movement**; offset tracks head-motion direction; absent when still. Also shows in-game
|
||||
as **black flashes on movement**. Same root cause.
|
||||
- **Proven NOT the cause** (ruled out by logging + a pulled headset video whose mono
|
||||
recorded frames are CLEAN): content rendering, stereo/IPD (65-68mm), FOV, swapchain
|
||||
stage-vs-acquired index (always matched), session cycling (gone, ~72fps steady),
|
||||
eye poses (flags=0xf), pixel count (capping supersample 1.2x→1.0 did NOT help).
|
||||
- ~~Root cause = submit latency from the mandatory flush-wait.~~ **REFUTED 2026-06-24
|
||||
by on-device measurement — see below.**
|
||||
|
||||
## UPDATE 2026-06-24: latency theory DISPROVEN; render-ahead abandoned
|
||||
Measured on device (synchronous build, instrumented `xrr_end_frame`):
|
||||
- **`submit-vs-predictedDisplayTime = −17 to −18 ms`** every frame (in menu AND gameplay).
|
||||
We hand `xrEndFrame` to the compositor ~1.5 frames (at 90 Hz) **BEFORE** the intended
|
||||
display moment. We are NOT late — there is healthy headroom. The whole "xrEndFrame
|
||||
lands after predictedDisplayTime" premise is false.
|
||||
- flush-wait blocks the render thread ~1 ms in menus, up to ~8–9 ms in gameplay, but
|
||||
never enough to miss the deadline (still −17 ms).
|
||||
- **Title screen dupes too, where flush-wait is only ~1 ms** → a bug that persists with
|
||||
near-zero latency cannot be a latency bug.
|
||||
- App runs **90/90 fps → ASW/motion-smoothing is NOT engaging** (would show 45/90).
|
||||
- The pulled-video mono frames are clean → the doubling is introduced **at display time,
|
||||
in stereo** (a mono capture can't show stereo divergence).
|
||||
|
||||
**Conclusion: the dupe is a stereo/compositor presentation issue, NOT latency.** The
|
||||
render-ahead pipeline (below) was therefore the wrong fix AND broke rendering (see
|
||||
"Why render-ahead failed"). It is now OFF by default and gated behind
|
||||
`debug.re4vr.pipeline` purely for reference. Do not pursue it.
|
||||
|
||||
### Ruled out on 2026-06-24 (with the exact data)
|
||||
- Layer layout: `layout=3` = `ovrpLayout_Array`, `arraySize=2`, we submit
|
||||
`imageArrayIndex=eye` — correct, not a side-by-side/double-wide mismatch.
|
||||
- Per-eye FOV: properly asymmetric & mirrored (L eye `right=+0.785`, R eye
|
||||
`left=−0.785`, toed toward the nose), `up/down` symmetric. Correct.
|
||||
- IPD ~0.068 m. Eye positions sane (`L≈(-0.037,1.088,0.005) R≈(0.031,1.090,0.007)`).
|
||||
|
||||
### Head-vs-eye camera mismatch — ALSO RULED OUT (2026-06-24)
|
||||
`HEADvsEYE:` capture: `head.pos == eyeMid` exactly and `head.q == e0.q == e1.q`
|
||||
exactly on every frame. UE's Head-derived cameras and our submitted `xrLocateViews`
|
||||
eye poses are geometrically identical (parallel rig, midpoint = head). Not it.
|
||||
|
||||
### Visual A/B probes (debug.re4vr.diag) — localized the dupe INTO the per-eye image
|
||||
Added a no-rebuild toggle: `adb shell setprop debug.re4vr.diag N; relaunch`.
|
||||
- `diag=1` projection only (drop quads), `diag=2` force mono (both eyes sample
|
||||
array layer 0). Applied in `build_composition`.
|
||||
- **diag=2 (force mono): dupe STILL present** — a fixed offset down-and-right,
|
||||
visible with the head still, small gap. → NOT stereo divergence (both eyes get
|
||||
the identical image yet still doubled) and NOT a motion/reprojection ghost (those
|
||||
vanish when still).
|
||||
- **diag=1 (projection only): dupe STILL present** → NOT the quad/menu overlapping.
|
||||
- `SUBMITLIST:` capture confirms only ONE eye-fov layer (LayerId=1) is submitted
|
||||
(most frames `nLayers=1`); occasionally + one quad (LayerId=0). Eye layer flags
|
||||
`0x4` (InverseAlpha-ish) / `0x14`; we don't act on them (compositing hints, not
|
||||
layout — unlikely to cause a positional dupe).
|
||||
|
||||
### Where it stands: the duplicate is INSIDE one eye's image
|
||||
With both eyes fed the identical array-layer-0 image and still doubled, each per-eye
|
||||
image itself contains two offset copies. So the doubling is introduced either (a) in
|
||||
UE's render INTO the eye texture, or (b) in the compositor's per-eye display path —
|
||||
NOT in our stereo/layer pairing, poses, FOV, layout, or timing (all verified correct).
|
||||
|
||||
### ROOT CAUSE FOUND (2026-06-24): the dupe is a QUAD overlapping the projection
|
||||
Texture readback + visual A/B settled it:
|
||||
- Dumped both eye array layers + the quad to PPM (`xrr_vk_dump_image`, gated by
|
||||
`debug.re4vr.dump`). **Both eye textures are CLEAN single images** (the eye
|
||||
projection is a clean RE4 castle scene; pre-title eye is pure black; the quad is a
|
||||
clean "armature" studio logo). So nothing we render is doubled.
|
||||
- **Closing one eye: still doubled** → the second copy is within each eye's image
|
||||
(NOT stereo divergence — earlier "converges at a head pose" was the world-locked
|
||||
quad parallaxing against the projection).
|
||||
- **`diag=1` (drop quad layers): the dupe DISAPPEARS** (logo becomes single).
|
||||
→ **The duplicate IS the quad.** The game's UI/logo is present in the eye projection
|
||||
AND re-submitted as a separate world-locked quad layer; the compositor shows both,
|
||||
offset, so they parallax with head motion. Worst on title/menus (heavy UI quads),
|
||||
mild in-game (few quads), flashes (quad submitted only on some frames per SUBMITLIST),
|
||||
persists one-eyed (two real copies). NOT latency, stereo, FOV, layout, or timing.
|
||||
|
||||
### REFINED 2026-06-24 (via Quest recordings + frame blends): TWO artifacts
|
||||
Pulled Quest spectator recordings (`/sdcard/Oculus/VideoShots/`) and blended head-
|
||||
motion frame pairs (`ffmpeg fps=30` + ImageMagick `-average`). Single spectator frames
|
||||
are always clean; blending two frames ~0.1-0.2s apart during a head turn exposes
|
||||
differential motion. Findings:
|
||||
- **Artifact A — logo duplication (diag=0):** the blend shows TWO "resident evil 4"
|
||||
logos offset vertically while the background is nearly aligned → a second logo copy
|
||||
that OVER-moves vs the scene. With `diag=1` (quads dropped) the over-moving copy is
|
||||
gone. BUT the logo is STILL present in `diag=1` → the logo also lives in the eye
|
||||
projection (the game renders it into the eye buffer) AND is re-submitted as a quad.
|
||||
The quad copy is the visible dupe. Fix = handle/suppress the duplicate quad.
|
||||
- **Artifact B — scene ghosting on head motion (persists with diag=1):** user still
|
||||
sees heavy ghosting of the whole scene during head turns with quads dropped, yet
|
||||
single spectator frames are clean → it's PER-EYE reprojection ghosting at display
|
||||
(mono spectator can't show it). Textbook symptom of a **projection layer submitted
|
||||
WITHOUT depth** → compositor can't do positional reprojection → head translation
|
||||
uncompensated → ghosting. (`xrr_setup_layer_depth` currently returns Unsupported.)
|
||||
|
||||
### Depth layer (XR_KHR_composition_layer_depth) — IMPLEMENTED + TESTED → NEGATIVE
|
||||
Implemented behind `debug.re4vr.depth` (default off): enable the ext in pre_init,
|
||||
create a D32_FLOAT depth swapchain per eye layer (`setup_layer`), hand UE the depth
|
||||
images via GetLayerTexture2, acquire/wait/flush(depth-aspect barrier)/release in
|
||||
lockstep with color, chain `XrCompositionLayerDepthInfoKHR` on each projection view.
|
||||
Tunable: `debug.re4vr.depth_nearz_mm`, `debug.re4vr.depth_revz`. Confined to the
|
||||
synchronous path (`!g_pipelineActive`).
|
||||
On device: depth swapchain created cleanly (fmt=126, 3 images, matches color), NO
|
||||
xrEndFrame/validation errors, both reverse-Z and standard-Z tried. **Result: zero
|
||||
improvement to the ghosting**, and the Quest spectator recording went **fully black**
|
||||
with depth on (headset still rendered). → Meta's runtime accepts but does NOT use
|
||||
plain KHR depth for reprojection (it uses Application SpaceWarp / motion vectors).
|
||||
**Depth is not the fix here.** Left gated behind the prop (off); do not enable.
|
||||
|
||||
Recommendation (original, now amended): depth did NOT fix B.
|
||||
|
||||
### ROOT CAUSE OF ARTIFACT B FOUND (2026-06-24): frame drops from flush-wait serialization
|
||||
Meta `VrApi` perf log during head motion (depth OFF, normal build):
|
||||
- Calm: `FPS=72/72` steady. During motion: `FPS=30-64/72` (and `30-55/90`) with
|
||||
`Stale=40-70` → the app misses display rate, compositor shows stale/reprojected
|
||||
frames → the ghosting. Settles when still (app catches up).
|
||||
- `App` GPU time is only 2-6ms (rendering is cheap!) but `CPU&GPU` total is 13-36ms
|
||||
→ CPU↔GPU are SERIALIZED, inflating frame time. The synchronous flush-wait
|
||||
(`xrr_vk_flush_wait` in end_frame, ~8ms/frame measured) blocks the render thread on
|
||||
UE's GPU work each frame, killing CPU/GPU pipelining. Depth=1 doubled it (color +
|
||||
depth flush-wait) → `30/90, 31ms`, which is why depth made ghosting WORSE.
|
||||
**Artifact B = frame drops caused by the flush-wait serializing CPU and GPU.** Not
|
||||
latency-submit, not depth, not stereo. The fix is to get the flush-wait off the
|
||||
critical path so CPU/GPU pipeline again.
|
||||
|
||||
### Perf-level fix — IMPLEMENTED + CONFIRMED WIN (2026-06-24)
|
||||
We were no-op'ing `ovrp_SetSystemCpuLevel2/GpuLevel2`. The game requests CPU=2
|
||||
(SUSTAINED_HIGH) and GPU=3 (BOOST); now forwarded via XR_EXT_performance_settings
|
||||
(`xrr_set_perf_level`, always on). On device: `perf level: CPU ovrp=2->xr=50 rc=0`,
|
||||
`GPU ovrp=3->xr=75 rc=0`. Clocks ramped GPU 305-490MHz -> **525-587MHz**, CPU steady
|
||||
2419MHz. FPS improved to mostly 72/72 + some 90/90 (user confirmed "fps higher, logo
|
||||
better"). Remaining: motion drops to 45-68/72 (flush-wait, see copy ring) + occasional
|
||||
severe 49ms hitches (likely title asset streaming, not our code). Keep this fix.
|
||||
|
||||
### UE-source facts (ue_src/, real OVR_Plugin_Types.h) that pin the copy-ring design
|
||||
- UE wraps OUR VkImage as its RHI render target (`RHICreateTexture2D[Array]FromResource`
|
||||
in CustomPresent_Vulkan) — so handing UE shim images via GetLayerTexture2 works.
|
||||
- Present flow (`FinishRHIFrame_RHIThread`): for each layer UpdateLayer_RHIThread builds
|
||||
the ovrpLayerSubmit (TextureStage = the stage UE rendered into THIS frame) -> EndFrame4
|
||||
(our xrr_end_frame) -> on success, `IncrementSwapChainIndex_RHIThread` advances UE's
|
||||
stage for next frame. So `submit->TextureStage` reliably names the image UE just drew.
|
||||
- depthFormat 10 = ovrpTextureFormat_None (D16=6,D24_S8=7,D32_FP=8,D32_S824=9) -> UE does
|
||||
NOT render depth. Depth path is moot (mapping corrected).
|
||||
- ovrpLayerSubmit_EyeFov tail carries ViewportRect[2], DepthNear/Far, Fov[2] (in the
|
||||
un-reversed union pad) if ever needed.
|
||||
|
||||
THE FIX for the motion drops (validated target): shim-owned copy ring.
|
||||
Design (eye layers only, behind debug.re4vr.copyring): allocate shim VkImages per eye
|
||||
layer (color, COLOR_ATTACHMENT|SAMPLED|TRANSFER_SRC); GetLayerTexture2 returns shim
|
||||
images so UE renders into them on its own stage cadence. OpenXR swapchain gets
|
||||
+TRANSFER_DST usage. Each end_frame: copy shimImages[submit->TextureStage] ->
|
||||
openxr[acquiredIndex] (4 barriers: shim COLOR->TRANSFER_SRC, openxr UNDEFINED->
|
||||
TRANSFER_DST, vkCmdCopyImage all array layers, openxr TRANSFER_DST->COLOR_ATTACHMENT,
|
||||
shim TRANSFER_SRC->COLOR_ATTACHMENT) submitted NO-wait; present the PREVIOUS frame's
|
||||
openxr image (wait its copy token = already done -> no CPU stall) with the stored
|
||||
composition; hold this frame's openxr. The copy both resolves tile memory AND is
|
||||
pipelined, so the CPU never blocks on the current frame's GPU = breaks the serialization.
|
||||
UE's stage is decoupled (copy uses TextureStage explicitly), so no stage-coupling break.
|
||||
|
||||
### COPY RING — IMPLEMENTED (untested), behind debug.re4vr.copyring (default off)
|
||||
Code: `xrr_vk_alloc_images`/`xrr_vk_free_images`/`xrr_vk_copy_submit` (vk_session.c),
|
||||
shim fields on XrLayer, setup_layer allocates shim images + adds TRANSFER_DST to the
|
||||
eye swapchain, GetLayerTexture2 hands UE the shim images, end_frame has a copy-ring
|
||||
branch (copy shim[TextureStage]->openxr[acquiredIndex] no-wait, present previous frame,
|
||||
hold current). Builds clean. **VALIDATION CUT: eye layers only — quads are DROPPED in
|
||||
copy-ring mode, so MENUS/UI ARE HIDDEN.** It's for measuring whether removing the eye
|
||||
flush-wait serialization restores framerate; if confirmed, next step is to handle quads
|
||||
(composite current-frame quads on top of the pipelined eye, or give quads shim+copy too).
|
||||
|
||||
### TEST PLAN (run together when ready; perf-level fix is already always-on)
|
||||
All live except where noted. Relaunch after setting copyring/sscap (decided at setup).
|
||||
- Baseline (perf fix only): props all 0. Move head, `adb logcat -d | grep VrApi` — note
|
||||
the FPS/Stale distribution (expect 72/72 mostly + drops to 45-68/72 on motion).
|
||||
- Copy ring: `adb shell setprop debug.re4vr.copyring 1` + relaunch. Expect: scene visible
|
||||
but NO menus; check VrApi FPS holds 72/72 (or 90/90) on motion with fewer Stale, and
|
||||
ghosting reduced. Log: `copy-ring: ENABLED`, `copyring: allocated N shim images`,
|
||||
`end_frame copy-ring`. If black/garbage -> a barrier/layout bug in xrr_vk_copy_submit.
|
||||
- Supersample cap (stackable): `adb shell setprop debug.re4vr.sscap 1` + relaunch
|
||||
(eye 1728x1900 -> 1440x1584; frees GPU + shrinks the copy). Log: `supersample cap: 1.0x`.
|
||||
- Revert: set all back to 0, relaunch.
|
||||
The 49ms hitches (title asset streaming) are NOT addressed by any of these.
|
||||
|
||||
### COPY-RING TEST RESULT (2026-06-24): works on title, BLACK in-game
|
||||
On-device with copyring=1:
|
||||
- Engaged cleanly (`copy-ring: ENABLED`, `allocated 3 shim images` for 1440 & 1728 eye
|
||||
layers), frames flowed (`end_frame copy-ring #20161+`), one transient `xrEndFrame
|
||||
FAILED rc=-23` (XR_ERROR_LAYER_INVALID) at the 1440->1728 layer swap.
|
||||
- **TITLE: after a moment, ghosting RESOLVED — "everything looks good."** Frame pacing
|
||||
hugely improved: long stretches of `FPS=72/72 Stale=0 CPU&GPU=2.9-3.2ms` (vs baseline
|
||||
13-36ms) = the CPU/GPU serialization is broken, exactly as intended. **This proves the
|
||||
flush-wait serialization is the ghosting root cause.**
|
||||
- **IN-GAME: mostly BLACK.** From the one in-game snapshot: copy-ring still running,
|
||||
app stuck at stage=2 (note: title also runs stage=2 and works, so stage isn't it),
|
||||
a `WaitSwapchainImage TIMEOUT layer=1` near startup, no error flood.
|
||||
- Leading hypothesis: the 1-frame pipeline holds 2 of the 3 OpenXR swapchain images;
|
||||
under heavier in-game load the compositor can't return the 3rd in time -> begin_frame
|
||||
xrWaitSwapchainImage TIMES OUT -> the present-pending chain breaks and does NOT
|
||||
self-recover -> persistent black. (Title is light enough to never time out.)
|
||||
- NEXT (tractable, not fundamental): make the copy-ring chain robust to a timeout —
|
||||
on a missed acquire, present empty (valid) that frame and cleanly re-establish the
|
||||
chain next good frame (never submit an invalid/stale layer; keep swapchain
|
||||
acquire/release perfectly balanced across the timeout path). Then re-test in-game.
|
||||
Also worth: confirm the timeout frequency in-game (capture was flaky — headset must
|
||||
be worn for the whole window). Copy-ring left OFF by default; perf fix is the
|
||||
shippable win so far.
|
||||
|
||||
### COPY-RING RETEST + PIXEL DUMP (2026-06-24): shim redirection breaks gameplay render
|
||||
Robustness fix (present-empty on broken chain) added; retested in-game = still black.
|
||||
Live buffer showed copy-ring running fine in-game: ~4000 frames (`copy-ring #2161..6241`),
|
||||
**0 TIMEOUT, ~2 chain-broken, no FAILED** — so NOT timeouts/chain-break/errors.
|
||||
Added a pixel dump to the copy-ring path (`debug.re4vr.dump` -> cr_shim_a0.ppm /
|
||||
cr_xr_a0.ppm). Dumped the shim (UE's render target) and the post-copy openxr image:
|
||||
- **Both are pure black (mean=0, std=0 — every pixel 0) in gameplay** (`shim stage=0`).
|
||||
The copy is faithful (openxr == shim); the SHIM ITSELF is black = UE rendered nothing
|
||||
into the shim image in gameplay.
|
||||
- On the TITLE the copy-ring showed content (castle) -> shim had content there.
|
||||
Conclusion: **UE renders into our shim image on the title but NOT in gameplay** — its
|
||||
heavier gameplay render path apparently doesn't write to the resource-wrapped shim
|
||||
(RHICreateTexture2DArrayFromResource) image we hand it via GetLayerTexture2. Likely a
|
||||
UE render-target/MSAA/resolve detail specific to the 3D scene path. Needs UE-internals
|
||||
+ RenderDoc to chase; not resolvable via remote logcat/dump loop.
|
||||
|
||||
## RENDERDOC DEEP-DIVE (2026-06-24) — copy-ring gameplay = MSAA resolve interaction
|
||||
Set up offline RenderDoc replay (huge: no headset needed for analysis):
|
||||
- App made debuggable via apktool (packaging/work/dbg flow -> re4vr-dbg.apk); RenderDoc
|
||||
Android server installed; capture over USB; replay headless with
|
||||
`qrenderdoc --python` over `adb://<usb-serial>` + CreateRemoteServerConnection +
|
||||
CopyCaptureToRemote + remote.OpenCapture (local replay of an Android capture fails;
|
||||
wireless adb drops the replay connection — USB is required). Scripts in
|
||||
~/renderdoc-captures/rd_*.py ; captures RE4/black.rdc (gameplay), RE4/title.rdc.
|
||||
- FINDING: UE renders the eye with **2x MSAA** into its OWN target (RenderDoc res 11965,
|
||||
ms=2), and the render pass **resolve attachment is our copy-ring shim** (ms=1) — i.e.
|
||||
UE resolves the MSAA scene INTO the shim we copy. So there is NO stage mismatch; the
|
||||
shim IS the resolve target. BUT in the gameplay capture the resolved shim is not the
|
||||
scene at our copy point (reads white/garbage), while on the title it lands fine.
|
||||
- So the copy-ring gameplay black is an **MSAA-resolve-into-our-shim interaction**: our
|
||||
externally-created VkImage (MUTABLE_FORMAT + COLOR|SAMPLED|TRANSFER_SRC|DST|INPUT_ATT,
|
||||
TILING_OPTIMAL) isn't receiving UE's MSAA resolve correctly in the gameplay path.
|
||||
Suspects to chase next: the MUTABLE_FORMAT/sRGB-vs-UNORM view used as resolve target,
|
||||
the image's create flags vs what a valid resolve dst needs, or tile-memory MSAA
|
||||
specifics on Adreno. (Single-shim made it worse — UE needs distinct per-stage images.)
|
||||
- Tooling note: GetMinMax(...,CompType.Typeless) gives misleading values; trust
|
||||
SaveTexture/visual instead.
|
||||
|
||||
## NET STATE (end of 2026-06-24 session)
|
||||
- SHIPPABLE WIN: perf-level fix (always on) — clocks boost, framerate up, confirmed.
|
||||
- ROOT CAUSE PROVEN: motion ghosting = frame drops from the flush-wait serializing
|
||||
CPU/GPU (copy-ring eliminated it on the title -> CPU&GPU 13-36ms dropped to ~3ms).
|
||||
- COPY-RING: built + behind debug.re4vr.copyring (default off). Works on title, but the
|
||||
shim redirection leaves gameplay black (UE not rendering into the shim in-game).
|
||||
Parked pending UE-render-path investigation.
|
||||
- DEAD ENDS (with evidence): latency-submit theory (we submit early), depth layer (UE
|
||||
passes None; Meta ignores plain KHR depth anyway), render-ahead-by-holding-OpenXR-
|
||||
images (breaks UE's texture-stage coupling), stereo/FOV/layout (all correct).
|
||||
- Levers available: debug.re4vr.sscap (supersample cap), and the perf fix is permanent.
|
||||
- Diagnostic toolkit retained: debug.re4vr.{copyring,depth,pipeline,noflushwait,diag,
|
||||
dump,sscap} + the perf/flush-wait probes. UE renders into shim-allocated
|
||||
VkImages on its own stage cadence (decoupled from the OpenXR swapchain, so no
|
||||
stage-coupling break like the render-ahead attempt). Each frame: copy the shim image
|
||||
into a freshly-acquired OpenXR image and pipeline the flush (wait the PREVIOUS frame's
|
||||
copy, which is already done) → CPU never blocks on the current frame's GPU. Releases
|
||||
the OpenXR image normally each frame (no holding → stage cadence intact). Cost: one
|
||||
full-res image copy/frame (~10% bandwidth) + extra VRAM; big but well-scoped. This is
|
||||
the original "option 2" and is now backed by the VrApi frame-drop data.
|
||||
Tooling that worked: `debug.re4vr.diag` visual A/B + `debug.re4vr.dump` texture readback
|
||||
+ Quest recording → frame-blend (the only way to make the artifact objectively visible).
|
||||
|
||||
### (Artifact A) figure out the correct quad handling
|
||||
Open question — why does the same content appear in BOTH the projection and a quad
|
||||
(real OVRPlugin presumably shows it once)? Avenues:
|
||||
1. Confirm the overlap: dump eye + quad on the SAME title frame and check the logo is
|
||||
in both (double-render) vs the quad being the only intended copy.
|
||||
2. UI routing: the game may render UI into the eye buffer only because some ovrp_* call
|
||||
we stub makes it think it's NOT in a layer-composited VR mode. Audit stubs that gate
|
||||
UE's "render UI to a separate layer vs into the eye buffer" decision.
|
||||
3. Quad placement: we world-lock the quad at the app's submitted pose in appSpace
|
||||
(LOCAL_FLOOR). If the app's pose assumes a different space/convention, our quad is
|
||||
offset from where the projection shows the same content; correct placement (or
|
||||
head-locking) could make them coincide.
|
||||
Workaround that proves the cause (not a fix): `diag=1` drops quads → dupe gone but
|
||||
menus/overlays vanish and title head-tracking feels less smooth.
|
||||
|
||||
### (superseded) read back the eye texture — DONE, textures are clean
|
||||
Add a one-shot GPU readback of eye-swapchain array layer 0 (copy VkImage→host buffer,
|
||||
dump PPM, `adb pull`) gated behind a prop. If the dumped texture is doubled → it's
|
||||
UE's render (game/UE-side; investigate multiview / the RE4 VR mod's render setup). If
|
||||
the dumped texture is CLEAN/single → the compositor introduces it at display (per-eye
|
||||
distortion/reprojection path; pursue Meta-specific settings / frame capture).
|
||||
Everything cheaper than this has been exhausted. Device left on the clean synchronous
|
||||
(playable) path; `debug.re4vr.diag`/`debug.re4vr.pipeline` both 0.
|
||||
|
||||
## (Dead end, kept for reference) The render-ahead pipeline
|
||||
|
||||
## The fix: take the flush-wait OUT of the critical path (pipeline it) — IMPLEMENTED 2026-06-24
|
||||
Render-ahead by one frame so we never block the submit. **Status: built, compiles
|
||||
clean, NOT yet tested on device.** Deploy + test per the commands at top.
|
||||
|
||||
How it works now (`xr_runtime.c xrr_end_frame`, gated by `g_pipelineActive`):
|
||||
1. `xrr_begin_frame`: acquires image `A_N` for each layer as before (unchanged).
|
||||
2. `xrr_end_frame` **(A)**: `xrr_vk_flush_submit(A_N)` — submits the barrier, returns a
|
||||
ring token, does **NOT** wait.
|
||||
3. **(B)**: waits the *previous* frame's flush token (already done → ~free), releases
|
||||
`A_{N-1}` (FIFO → releases the older, present-pending image), and `xrEndFrame`s the
|
||||
**stored** composition `g_pending` (frame N-1's views + predictedDisplayTime).
|
||||
4. **(C)** builds frame N's composition into `g_pending`; **(D)** promotes `A_N` to
|
||||
`presentPending` (held one more frame; `begin_frame` re-acquires fresh).
|
||||
- **Why the stage↔acquire invariant survives:** still exactly one acquire + one release
|
||||
per frame, just offset by one → the OpenXR FIFO and UE's `TextureStage` stay in
|
||||
lockstep (the `LAYER MISMATCH` log will fire if this ever breaks — watch it).
|
||||
- Net: +1 frame latency (~13ms, absorbed by normal reprojection); CPU never blocks on
|
||||
the flush → `xrEndFrame` lands on schedule → ghosting should clear.
|
||||
- **Engage gate:** pipeline turns on once `xrr_vk_flush_ready()` AND every active layer
|
||||
has `imageCount >= 2` (Quest gives 3). Until then it runs the **synchronous fallback**
|
||||
(old flush+wait+release path, retained) — so worst case = today's behaviour, not a
|
||||
regression. Look for `render-ahead pipeline engaged` in logcat to confirm it switched.
|
||||
- New split flush API in `vk_session.c`: `xrr_vk_flush_submit`/`_wait`/`_ready`; ring
|
||||
grown to `XRR_MAX_LAYERS*2` so a token stays valid a full frame.
|
||||
|
||||
### On-device validation checklist
|
||||
- Confirm `render-ahead pipeline engaged` appears once, early.
|
||||
- Watch for `LAYER MISMATCH` (should NOT appear) and `xrEndFrame FAILED` (should NOT).
|
||||
- Heartbeat now prints `pipelined=1`. Framerate should stay ~72fps.
|
||||
- **First tuning knob if ghosting persists:** `submit_pending()` uses the STORED
|
||||
`g_pending.displayTime` (frame N-1's predicted time). If motion judders/over-reprojects,
|
||||
try using the *current* `g_xr.frameState.predictedDisplayTime` instead while keeping
|
||||
the stored views — one-line change, documented inline. (Views must stay stored.)
|
||||
- Session teardown: `pipeline_reset()` releases held images + clears `g_pending` on
|
||||
STOPPING so a restart doesn't present a stale composition over destroyed swapchains.
|
||||
- Risk: medium-high (acquire/release pairing, holding an image across frames). If it
|
||||
misbehaves, set `g_pipelineActive` permanently 0 to fall back to the synchronous path.
|
||||
|
||||
## Other levers to try (cheaper, possibly complementary)
|
||||
- **Cap supersample**: `src/layers.c ovrp_CalculateEyeLayerDesc2` — `if (textureScale>1) textureScale=1;`
|
||||
Tried, didn't fix ghosting alone, but frees GPU headroom; may help combined with pipelining.
|
||||
- **Submit a depth layer** (`XR_KHR_composition_layer_depth`): better positional reprojection.
|
||||
`xrr_setup_layer_depth` currently returns Unsupported; would need a depth swapchain +
|
||||
`GetLayerTexture2` depth handles + `XrCompositionLayerDepthInfoKHR` on the projection.
|
||||
- **Adjust predicted display time**: account for the flush-wait latency when filling
|
||||
`xrEndFrame.displayTime` so the compositor reprojects less. Hacky; secondary.
|
||||
|
||||
## Key files / functions
|
||||
- `src/vk_session.c` — `xrr_vk_flush_image` (the flush+wait; pipelining changes the wait),
|
||||
`detect_ue_queue` (queue is family0/idx0), `xrr_vk_set_handles`.
|
||||
- `src/xr_runtime.c` — `xrr_begin_frame`, `xrr_end_frame` (acquire/flush/release/compose),
|
||||
`make_app_space` (LOCAL_FLOOR), heartbeat/diag logs.
|
||||
- `src/layers.c` — `ovrp_CalculateEyeLayerDesc2` (resolution/FOV), quad placement is in
|
||||
`xr_runtime.c` end_frame.
|
||||
- Diagnostic logging still in (rate-limited, harmless): begin/end heartbeats, view poses,
|
||||
layer stage-vs-acquired mismatch, quad pose/size/flags, GetNodePose nodes. Strip before
|
||||
any public release.
|
||||
|
||||
## Verified facts to trust
|
||||
- App's rendered frames are CLEAN (pulled headset video confirms) — the bug is display-time.
|
||||
- Frame loop runs ~72fps with the flush-wait; the issue is per-frame reprojection from
|
||||
pose/time latency, not dropped framerate.
|
||||
- Removing the flush-wait → black (runtime won't sync for us). The wait is mandatory in
|
||||
its current synchronous form; pipelining is how to keep it without the latency.
|
||||
@@ -0,0 +1,60 @@
|
||||
# Reverse-engineering notes — libOVRPlugin.so
|
||||
|
||||
Pinned version: OVRPlugin **1.51** / pkg `ovrplugin-android-universal:19.0.0.449.531`.
|
||||
Cross-reference header: public **OVRPlugin.cs @ v1.51** (Unity Oculus Integration,
|
||||
many GitHub mirrors) — gives ovrp_ signatures + [StructLayout] struct layouts.
|
||||
|
||||
## RE pipeline (working)
|
||||
Headless Ghidra 12.1.2, driven by `ghidra_scripts/DumpOvrp.java`:
|
||||
```
|
||||
export JAVA_HOME=~/dev/re4vr-port/tools/jdk-21.0.11+10 # portable Temurin JDK
|
||||
tools/ghidra_12.1.2_PUBLIC/support/analyzeHeadless ghidra_proj re4vr \
|
||||
-import dump/apk_libs/lib/arm64-v8a/libOVRPlugin.so \
|
||||
-scriptPath ghidra_scripts -postScript DumpOvrp.java -deleteProject
|
||||
```
|
||||
Gotchas solved: Ghidra needs a JDK (JREs rejected) -> portable Temurin in tools/.
|
||||
Ghidra 12 dropped bundled Jython -> scripts must be Java, not .py.
|
||||
Output: `analysis/ovrp_decomp.txt` (537 ovrp_ sigs, 27 core bodies).
|
||||
|
||||
## Key structural finding
|
||||
Exported ovrp_* are THIN THUNKS: `(*PTR_ovrp_X)()` jumping to the real impl,
|
||||
which calls internal C++ `OVR::Util::Compositor::*` (partially symbolized).
|
||||
=> The game (Java System.loadLibrary + dlsym) calls the EXPORTED symbols. So the
|
||||
SHIM just re-exports the same ovrp_ symbol names backed by OpenXR and replaces
|
||||
libOVRPlugin.so wholesale. We do NOT reverse the internal Compositor C++.
|
||||
Decompilation = reference for semantics + struct sizes only.
|
||||
|
||||
## ABI confirmed (triangulation: Ghidra shape + v1.51 header types = MATCH)
|
||||
ovrp_GetNodePoseState3(ovrpStep, int frameIndex, ovrpNode, ovrpPoseStatef* out):
|
||||
- null out -> returns 0xfffffc17 = -1001 (ovrpFailure_InvalidParameter)
|
||||
- not init -> returns 0xfffffc16 = -1002 (ovrpFailure_NotInitialized)
|
||||
- success -> memcpy(out, ..., 0x58) then return 0
|
||||
- **ovrpPoseStatef = 0x58 = 88 bytes** == public layout:
|
||||
ovrpPosef(28) + 4x ovrpVector3f(48) + double Time(8) -> pad 88. EXACT MATCH.
|
||||
=> Public OVRPlugin.cs v1.51 struct layouts are TRUSTWORTHY for this binary.
|
||||
ovrpResult error-code convention confirmed (-1001 invalid, -1002 not-init).
|
||||
|
||||
## Layer structs reversed (2026-06-23, analysis/struct_layouts.txt)
|
||||
Ghidra recovered C++ type NAMES from demangled symbols but empty layouts (no DWARF);
|
||||
real offsets came from decompiling OVR::Util::Compositor methods.
|
||||
- Two compositor backends: CompositorVRAPI_OpenGL + CompositorVRAPI_Vulkan (RE4=Vulkan).
|
||||
- **ovrpLayerDesc = 0x7c (124B)** [VERIFIED]: ImportLayerDesc memset's 0x7c; switch on
|
||||
+0x00 = ovrpShape (cases 0,1,2,4,5 overlays; case 3 = EyeFov projection). Three
|
||||
historical copy sizes 0x68/0x6c/0x7c = base / +DepthFormat / +MotionVector. Full
|
||||
field layout written to header, offsets confirmed (Fov@0x20, VisibleRect@0x40,
|
||||
DepthFormat@0x68).
|
||||
- **ovrpLayerSubmit = 0x130 (304B)** [VERIFIED]: EndFrame4 allocs count*0x130. Header
|
||||
fields (LayerId, TextureStage, ViewportRect[2], Pose@0x28, Flags) = public layout;
|
||||
per-shape union tail still [TODO] (reserved bytes for now). Stored internally as
|
||||
ovrpLayerSubmitUnion in a std::map<int,pair<union,int>>.
|
||||
Header now has _Static_asserts sizeof(ovrpLayerDesc)==124 && ovrpLayerSubmit==304;
|
||||
both pass, shim rebuilds (239 symbols).
|
||||
|
||||
## Next RE passes (when continuing)
|
||||
- Dedup the export-thunk vs impl entries in the dump (script polish).
|
||||
- For each CORE fn: pair Ghidra arg-shape with v1.51 C# sig -> finalized C header
|
||||
for the shim (ovrp types + struct layouts).
|
||||
- Reverse outliers NOT in the public header (e.g. SpaceWarp/ASW property paths the
|
||||
binary references, any custom Armature behavior).
|
||||
- Entitlement (libovrplatformloader's ovr_* surface) is out of scope for this repo:
|
||||
no replacement ships and nothing is circumvented (see README "Legal / scope").
|
||||
@@ -0,0 +1,204 @@
|
||||
# RE4 VR (Quest 2) — Dump & Recon Checklist
|
||||
|
||||
Goal of this phase: **non-destructively dump your own legally-owned copy of RE4 VR
|
||||
off the Quest 2 and answer the one question that decides the whole project** —
|
||||
is the VR runtime OpenXR (shimmable) or VrApi (proprietary, must reimplement)?
|
||||
|
||||
Target app: `com.Armature.VR4` (Armature Studio / Capcom / Oculus, UE 4.25.3)
|
||||
|
||||
Legal posture: dump-your-own only. We extract from hardware *you own* running a
|
||||
copy *you own*. **Nothing here gets redistributed** — only patches/shims you
|
||||
author, applied by people who dump their own copy. Same model as ReXGlue.
|
||||
|
||||
---
|
||||
|
||||
## 0. Prereqs (do while the Quest charges)
|
||||
|
||||
- [ ] Install Android platform-tools (adb): `sudo apt install android-tools-adb`
|
||||
or grab Google's platform-tools zip.
|
||||
- [ ] Verify: `adb version`
|
||||
- [ ] Install analysis tooling:
|
||||
- [ ] Ghidra (native .so disassembly/patching)
|
||||
- [ ] `patchelf`, `binutils` (`readelf`, `nm`, `objdump`), `file`, `unzip`
|
||||
- [ ] Python 3 + `pip install lief` (scripted ELF inspection/patching)
|
||||
- [ ] FModel **or** umodel/UModel (UE 4.25 .pak browsing) — optional this phase
|
||||
- [ ] FluffyQuack's UnrealPak tools — optional this phase
|
||||
- [ ] Enable **Developer Mode** on the Quest (Meta Quest mobile app →
|
||||
Devices → Developer Mode → on; requires a registered dev org — free).
|
||||
- [ ] Plug Quest into PC via USB-C, put on headset, **Allow USB debugging**
|
||||
when prompted (and check "always allow from this computer").
|
||||
|
||||
---
|
||||
|
||||
## 1. Confirm the device + app are visible
|
||||
|
||||
```bash
|
||||
adb devices # should list one device, state "device" not "unauthorized"
|
||||
adb shell pm list packages | grep -i armature # expect: com.Armature.VR4
|
||||
adb shell dumpsys package com.Armature.VR4 | grep -i versionName
|
||||
```
|
||||
|
||||
- [ ] Device shows as `device`
|
||||
- [ ] `com.Armature.VR4` present
|
||||
- [ ] Note the versionName here: `____________`
|
||||
|
||||
---
|
||||
|
||||
## 2. Locate and pull the APK(s)
|
||||
|
||||
Split APKs are common, so grab every path.
|
||||
|
||||
```bash
|
||||
adb shell pm path com.Armature.VR4 # prints one or more base/split apk paths
|
||||
mkdir -p ~/dev/re4vr-port/dump && cd ~/dev/re4vr-port/dump
|
||||
# pull each path the command above printed, e.g.:
|
||||
adb pull /data/app/~~xxxx/com.Armature.VR4-yyyy/base.apk .
|
||||
# repeat for any split_*.apk lines
|
||||
```
|
||||
|
||||
- [ ] `base.apk` pulled
|
||||
- [ ] Any `split_*.apk` pulled
|
||||
- [ ] Record sizes: `ls -lh *.apk`
|
||||
|
||||
---
|
||||
|
||||
## 3. Pull the OBB (game data / .pak files)
|
||||
|
||||
```bash
|
||||
adb shell ls -la /sdcard/Android/obb/com.Armature.VR4/
|
||||
adb pull /sdcard/Android/obb/com.Armature.VR4/ .
|
||||
```
|
||||
|
||||
- [ ] OBB pulled (e.g. `main.NNN.com.Armature.VR4.obb`)
|
||||
- [ ] Note the version number NNN in the OBB filename: `______`
|
||||
(you'll need it if you ever repack)
|
||||
|
||||
---
|
||||
|
||||
## 4. ⭐ THE DECISIVE CHECK — OpenXR vs VrApi
|
||||
|
||||
This is the whole reason we're here. Inspect the native libs in the APK.
|
||||
|
||||
```bash
|
||||
cd ~/dev/re4vr-port/dump
|
||||
unzip -l base.apk | grep -iE 'lib/arm64-v8a/.*\.so' # list native libs
|
||||
# the money grep:
|
||||
unzip -l base.apk | grep -iE 'arm64.*(vrapi|openxr|ovrplatform|oculus|UE4)'
|
||||
```
|
||||
|
||||
Interpret the result:
|
||||
|
||||
| Lib found in `lib/arm64-v8a/` | Meaning | Difficulty |
|
||||
|-----------------------------------|-----------------------------------------------------|------------|
|
||||
| `libopenxr_loader.so` | ✅ Standard OpenXR — Steam Frame provides a runtime; you translate vendor extensions. | Tractable |
|
||||
| `libvrapi.so` | ⚠️ Proprietary Meta VrApi — must reimplement/shim the runtime. | Hard |
|
||||
| `libovrplatformloader.so` | Present either way — this is the **entitlement check** to NOP/stub. | Required patch |
|
||||
| `libUE4.so` (or split into modules)| The Unreal runtime itself — the host you'll be hooking. | n/a |
|
||||
|
||||
- [x] **RESULT — runtime is:** ☑ **VrApi** (legacy OVRPlugin path). Confirmed
|
||||
2026-06-23 on app v2.3 (versionCode 203). `libvrapi.so` present,
|
||||
`libopenxr_loader.so` ABSENT. `libOVRPlugin.so` NEEDs `libvrapi.so` and
|
||||
imports 114 `vrapi_*` symbols; 0 OpenXR symbols anywhere.
|
||||
- [x] `libovrplatformloader.so` present? ☑ yes (hard-NEEDED by libUE4.so)
|
||||
|
||||
**REFINED WIRING (the seam that matters):**
|
||||
```
|
||||
libUE4.so --(ovrp_* C API, 239 refs)--> libOVRPlugin.so --(114 vrapi_)--> libvrapi.so --> Horizon OS
|
||||
libUE4.so --(NEEDED + ovr_* Platform SDK, 132 refs)--> libovrplatformloader.so (entitlement)
|
||||
```
|
||||
- libUE4.so has **0 vrapi_ refs** — game speaks **OVRPlugin's ovrp_* C API**, not
|
||||
VrApi directly. libvrapi is just OVRPlugin's backend.
|
||||
- => **PORT SEAM = reimplement libOVRPlugin.so (ovrp_* on OpenXR), drop libvrapi.**
|
||||
ovrp_* is the documented Unity-shared API (OVR_Plugin.h); modern Meta OVRPlugin
|
||||
has an OpenXR backend = reference impl / prior art.
|
||||
- => **Stub the ovr_* Platform SDK** (libovrplatformloader.so) — entitlement/account,
|
||||
just return OK; don't reimplement.
|
||||
|
||||
> *Editor's note (2026-06): the entitlement-stub approach described here and elsewhere in this
|
||||
> doc (the §4 table, the wiring summary above, the decision tree below) was **not** carried
|
||||
> into the project. Entitlement handling is out of scope and the shim ships no circumvention
|
||||
> code — see the README's scope section. These passages are kept as a record of the original
|
||||
> recon, not as instructions.*
|
||||
|
||||
> If you want to double-check beyond filename presence, extract and inspect
|
||||
> imports of `libUE4.so` (it may dynamically link the runtime):
|
||||
> ```bash
|
||||
> unzip base.apk 'lib/arm64-v8a/*' -d apk_libs
|
||||
> readelf -d apk_libs/lib/arm64-v8a/libUE4.so | grep -i NEEDED
|
||||
> nm -D --defined-only apk_libs/lib/arm64-v8a/libopenxr_loader.so 2>/dev/null | grep -i xr | head
|
||||
> nm -D apk_libs/lib/arm64-v8a/libUE4.so | grep -iE 'xrCreate|vrapi_' | head
|
||||
> ```
|
||||
> `xrCreate*`/`xr*` symbols → OpenXR codepath. `vrapi_*` symbols → VrApi codepath.
|
||||
> A binary can ship both libs but only *call* one — the symbol check tells you
|
||||
> which is actually wired up.
|
||||
|
||||
---
|
||||
|
||||
## 5. Manifest & build recon
|
||||
|
||||
```bash
|
||||
# Needs apktool (sudo apt install apktool) OR aapt from build-tools
|
||||
aapt dump badging base.apk | grep -iE 'sdkVersion|native-code|package'
|
||||
apktool d -s base.apk -o apk_decoded # -s = don't decode .dex, faster
|
||||
grep -iE 'oculus|vr|xr|entitlement|permission' apk_decoded/AndroidManifest.xml
|
||||
```
|
||||
|
||||
- [ ] minSdk / targetSdk noted: `______`
|
||||
- [ ] `native-code` ABI (expect `arm64-v8a`): `______`
|
||||
- [ ] Any Oculus/entitlement metadata flags in manifest noted below.
|
||||
|
||||
---
|
||||
|
||||
## 6. Confirm .pak accessibility (asset layer)
|
||||
|
||||
The community reports these are unencrypted — verify so you know the asset
|
||||
layer is open if you ever need it.
|
||||
|
||||
```bash
|
||||
# unzip the OBB (it's a zip), find Content/Paks/*.pak, then:
|
||||
# try opening in FModel/umodel as UE 4.25.3, no AES key
|
||||
```
|
||||
|
||||
- [x] .pak opens with no AES key (confirms community finding) ☑ yes
|
||||
Verified 2026-06-23: pakchunk9 footer bEncryptedIndex=0, EncryptionKeyGuid
|
||||
all-zero, pak version 9 (FrozenIndex / UE4.25-26). Plaintext index — 53
|
||||
readable asset paths incl. /Game/Levels/BIO4/... Asset layer fully open.
|
||||
- OBB layout: store (uncompressed) zip. main.203 (4.0GB) + patch.203 (4.0GB,
|
||||
the v2.3 update layer) + a stashed VR4-Android-Shipping-arm64.apk (== base.apk).
|
||||
Paks at VR4/Content/Paks/pakchunk{0..9}[optional]-Android_ETC2.pak (ETC2 =
|
||||
Android texture compression; 'optional' = hi-res texture chunks).
|
||||
Bink cutscenes at VR4/Content/Movies/*.bk2.
|
||||
|
||||
---
|
||||
|
||||
## 7. Record findings → decide path
|
||||
|
||||
Fill this in, then we branch:
|
||||
|
||||
```
|
||||
versionName: 2.3 (versionCode 203, minSdk 25, targetSdk 29)
|
||||
runtime: VrApi (libvrapi via libOVRPlugin; NO openxr) [CONFIRMED 2026-06-23]
|
||||
ovrplatform loader: present (hard-NEEDED by libUE4.so)
|
||||
UE version: 4.25.3 (engine = libUE4.so, stripped, arm64)
|
||||
ABIs: arm64-v8a (single base.apk, no splits)
|
||||
paks encrypted: NO — unencrypted, no AES key (CONFIRMED 2026-06-23, pak v9)
|
||||
device: Quest 2 serial <redacted-serial> (codename hollywood)
|
||||
```
|
||||
|
||||
**Decision tree:**
|
||||
- **OpenXR** → next phase: map which Meta OpenXR vendor extensions the binary
|
||||
requests, plan the OpenXR→Steam-Frame-runtime shim + entitlement stub.
|
||||
This is the "weekend-of-shimming" branch.
|
||||
> *Editor's note (2026-06): the "entitlement stub" floated here was later removed from the
|
||||
> project. Entitlement handling is out of scope and the shim ships no circumvention code —
|
||||
> see the README's scope section. This line is left as a record of the original plan.*
|
||||
- **VrApi** → next phase: scope a VrApi reimplementation/translation shim
|
||||
(much larger). Reassess whether the project is worth it vs. waiting/UEVR.
|
||||
- **Either way** → `libovrplatformloader.so` entitlement bypass is a required
|
||||
Ghidra patch (legal for your own copy).
|
||||
|
||||
---
|
||||
|
||||
## Notes / scratch
|
||||
|
||||
(paste command output, symbol dumps, and decisions here as you go)
|
||||
@@ -0,0 +1,60 @@
|
||||
# OVRPlugin -> OpenXR shim — scope
|
||||
|
||||
Derived 2026-06-23 from RE4 VR v2.3 (`com.Armature.VR4`). Method: extracted the
|
||||
distinct `ovrp_*` names `libUE4.so` references (dlsym targets) and intersected
|
||||
with `libOVRPlugin.so`'s exports. Raw lists in `analysis/`.
|
||||
|
||||
## Headline numbers
|
||||
- OVRPlugin exports **438** `ovrp_*` entry points.
|
||||
- The game actually references **239** of them. That's the shim surface.
|
||||
- But **~150 of the 239 are stub-to-constant / no-op** for a basic port.
|
||||
Realistically **~85-90 functions need real implementation**, of which the
|
||||
genuinely hard core is **~40** (frame loop + layer/swapchain + the display
|
||||
interop).
|
||||
|
||||
## Render API: VULKAN (confirmed)
|
||||
`libUE4.so` has VulkanRHI compiled in and calls `ovrp_Get{Instance,Device}ExtensionsVk`.
|
||||
libGLESv2/EGL are NEEDED but that's the standard Android baseline; the active
|
||||
renderer is Vulkan. => swapchain path is the standard **OpenXR `XR_KHR_vulkan_enable2`**
|
||||
(import the app's VkImages into `XrSwapchain`). Well-trodden, not exotic.
|
||||
|
||||
## STUB to no-op/constant (~131 counted, +misc ≈ 150)
|
||||
| Group | count | how to stub |
|
||||
|---|---|---|
|
||||
| Mixed Reality Capture (`ovrp_Media_*`) | 37 | return not-initialized / no-op |
|
||||
| Camera device (passthrough/depth) | 21 | report unavailable |
|
||||
| External camera (MRC) | 13 | report 0 cameras |
|
||||
| Perf/GPU/CPU/ASW/foveation tuning | 31 | accept+ignore, return safe defaults |
|
||||
| Boundary / Guardian | 7 | "not configured" (or map to XR play bounds later) |
|
||||
| Hand tracking | 7 | disabled (RE4 VR is controller-only) |
|
||||
| System-info getters | 15 | return plausible constants (headset type, region…) |
|
||||
|
||||
Stubbing these = ~150 functions for near-free. None affect core gameplay.
|
||||
|
||||
## MUST implement (~88 counted; the project's real work)
|
||||
| Group | count | notes |
|
||||
|---|---|---|
|
||||
| Tracking / poses | 32 | mostly mechanical: map OpenXR `xrLocateSpace`/views to `ovrp` pose structs |
|
||||
| Display / layer / swapchain | 20 | THE HARD PART — projection layers, eye FOV, foveation, swapchain stages |
|
||||
| Input / controllers | 11 | map Touch controller -> OpenXR action set; mechanical but fiddly |
|
||||
| Eye / user params | 10 | IPD, eye height, pixels-per-tan-angle -> from XR view config |
|
||||
| Init / shutdown | 8 | session create/begin/end, instance+system setup |
|
||||
| Frame loop | 6 | `xrWaitFrame`/`xrBeginFrame`/`xrEndFrame` <-> `ovrp_*Frame4`, predicted display time |
|
||||
|
||||
## Verdict
|
||||
Single-developer-feasible, meaty. The "239 functions" headline collapses to
|
||||
**~40 hard + ~45 mechanical + ~150 stubs.** The hard 40 are the standard guts of
|
||||
any OpenXR app (frame loop, projection layers, Vulkan swapchain, pose/input
|
||||
mapping). **Modern Meta OVRPlugin already ships an OpenXR backend** — that's the
|
||||
reference for exact `ovrp_*` semantics and struct layouts, which removes most of
|
||||
the guesswork.
|
||||
|
||||
Biggest unknowns / next probes:
|
||||
1. Exact `ovrp_*` struct layouts/ABI (need OVR_Plugin.h matching this OVRPlugin
|
||||
version, or RE them in Ghidra). ABI mismatch = crashes.
|
||||
2. How the Java side loads libOVRPlugin (System.loadLibrary) — confirms the
|
||||
replacement mechanism (drop-in shim .so with same SONAME).
|
||||
3. Entitlement: out of scope for this project — no `ovr_*` replacement ships here;
|
||||
the platform's real check runs unchanged (see README "Legal / scope").
|
||||
4. Whether Steam Frame exposes an Android-app OpenXR runtime at all (the OTHER
|
||||
big external unknown — the shim is moot if the APK can't load there).
|
||||
@@ -0,0 +1,195 @@
|
||||
# P4 passthru — NATIVE ground-truth (2026-06-27, title screen, seated)
|
||||
|
||||
Captured via `debug.re4vr.passthru=1` (real libOVRPlugin owns the session through our shim).
|
||||
Boots clean to the title screen, no ghost — so these are the CORRECT values to match.
|
||||
|
||||
## Eye layer desc (CalculateEyeLayerDesc2)
|
||||
- size: **1440 x 1584** per eye
|
||||
- FovL: U 1.111 D 1.192 L 0.933 R 1.000 (tangents)
|
||||
- FovR: U 1.111 D 1.192 L 1.000 R 0.933
|
||||
- => asymmetric (U≠D) AND canted stereo (per-eye L/R mirrored: inner edge = 1.000, outer = 0.933)
|
||||
|
||||
## EndFrame4 submit
|
||||
- 1 layer, id=1 (first 3 frames) then id=2, **flags=0x4**, pose = all-zero (0,0,0)/(0,0,0,0)
|
||||
- flags 0x4 = bit2 (NOT HeadLocked=0x1)
|
||||
|
||||
## Eye poses (GetNodePoseState3, head tilted on table)
|
||||
- eye0 (L): pos=(-0.0934, 1.2170, 0.2354) quat=(0.0314, 0.1109, -0.0207, -0.9931)
|
||||
- eye1 (R): pos=(-0.0271, 1.2203, 0.2503) quat=(same as L)
|
||||
- both eyes share orientation; offset L->R ≈ (+0.066, +0.003, +0.015) m in world (IPD ~66mm)
|
||||
|
||||
## OUR shim values (captured passthru=0, title screen) — DIFF RESULT
|
||||
- eye size: **1440x1584** — MATCHES native (the 1728x1900 was an old supersample config).
|
||||
- FOV: ours via OpenXR xrLocateViews, logged as angles (rad). Steady-state L.fov(l,r,u,d)=
|
||||
(-0.750,0.785,0.838,-0.873) -> tangents U1.110 D1.190 L0.932 R0.991 — MATCHES native
|
||||
(U1.111 D1.192 L0.933 R1.000). The game does NOT call GetNodeFrustum2; FOV comes from
|
||||
CalculateEyeLayerDesc2 in BOTH paths.
|
||||
- IPD: 0.065-0.068 — MATCHES native ~0.066.
|
||||
- eye orientation: e0.q==e1.q==head.q in both — MATCHES.
|
||||
- submit flags: eye-fov LayerId flags=0x4 — MATCHES native (0x14 later = menu-quad layer
|
||||
present, same game logic).
|
||||
|
||||
## CONCLUSION (title screen)
|
||||
Static eye geometry (size/fov/ipd/orientation/flags) is IDENTICAL native vs shim at the title
|
||||
screen. The seated in-game hand-deform ghost is therefore NOT a static-geometry mismatch.
|
||||
ONE anomaly: our shim's FIRST ~5s report a WIDER outer FOV (L outer angle -0.855, tan 1.149)
|
||||
that settles to native's -0.750/0.933 — a cold-start transient (matches title-ghost-is-coldload
|
||||
-reprojection-judder; self-resolves warm). Not the persistent in-game ghost.
|
||||
|
||||
## CONFIRMED: passthru (native) has ZERO ghost — hands + title (user, 2026-06-27)
|
||||
=> the ghost is in OUR shim's path, not the game/headset.
|
||||
|
||||
## *** ACTUAL ROOT CAUSE: DROPPED FRAMES -> compositor reprojection multiples (2026-06-27 PM) ***
|
||||
The submit-ordering theory below was WRONG (submithook=2 present-on-submit AND submithook=3
|
||||
+full device-wait-idle completion BOTH still ghost). Added always-on ANOMALY logging of the
|
||||
OpenXR runtime RESPONSES (not our inputs, which all check out): locateViews validity,
|
||||
WaitSwapchainImage timeout, xrEndFrame errors, frame-pacing. In-game ghost capture result:
|
||||
- frame-pacing: 42 MISS / 3986 frames (~1%), gaps 21-137ms vs 13.9ms period
|
||||
- locateViews invalid: 0 WaitSwapchain timeout: 0 xrEndFrame err: 0
|
||||
=> the ONLY fault is DROPPED FRAMES. We miss the display deadline; the compositor reprojects the
|
||||
held frame to fill the gap; during motion that = the flashing multiples. Scales exactly with user's
|
||||
report: 21ms gap (1 drop)=mild, 137ms (~10 drops)=violent; load/position-dependent; non-deterministic;
|
||||
clean dumped eye textures (pixels fine, just late). User CONFIRMED dump "no ghost" was the dump's
|
||||
crawling fps freezing reprojection, not a fix — consistent.
|
||||
KEY OPEN Q: are the drops OURS (shim latency, e.g. per-frame xrr_vk_flush_wait) or the game's load
|
||||
(native hits them too but its compositor rides them out cleanly)? Added ENDFRAME-PACE log at
|
||||
ovrp_EndFrame4 ENTRY (core.c) that runs in BOTH modes -> compare NATIVE vs shim gap rate.
|
||||
- if NATIVE also hits 137ms gaps clean -> fix = match native compositor frame-timing/handling
|
||||
(ExtraLatencyMode / phase sync / how late frames are submitted), NOT eliminate hitches.
|
||||
- if NATIVE smooth -> our shim adds the latency -> reduce it (flush-wait is prime suspect).
|
||||
NOTE for Steam Frame: passthru is Meta-only (vrapi); the OpenXR shim is the only cross-platform
|
||||
path, so this drop/repro fix is what matters for portability.
|
||||
|
||||
## *** SUCCESS: game-thread pacing FIXED the ghost (2026-06-27, user-confirmed) ***
|
||||
User: title ghost GONE, seated ghost GONE, standing black-flash GONE, "way better... feels like a
|
||||
great success." Logs: BC (render-thread stall) 148ms->~3ms (FIXED); frame-pacing drops 0.79%
|
||||
(180/22862, was ~1%+10%-in-motion, native 0.34%); WAITPACE steady 13.92ms. The ROOT CAUSE was: our
|
||||
shim left the game thread UNPACED (WaitToBeginFrame no-op) and paced the render thread instead, which
|
||||
desynced UE's internal pipeline and stalled the render thread during motion -> dropped frames ->
|
||||
compositor judder = the "ghost". Fix = pace the game thread (xrWaitFrame in xrr_wait_frame) + FIFO
|
||||
frameState handoff to render thread. Keep ffr=-1 (game FFR) + blackcount=0. RESIDUAL (tunable):
|
||||
occasional "BeginFrame too many times" -> XR_FRAME_DISCARDED = brief hiccup; from the ring DROPPING
|
||||
oldest frameState when render falls behind (desyncs wait/begin 1:1). FIX: block the game thread for
|
||||
ring space (back-pressure) instead of dropping. THEN: strip diagnostic scaffolding (FLOOP/STALL/
|
||||
JUDDER/WAITPACE/burst-dump/submithook leftovers), make ffr=-1+blackcount=0 defaults, commit.
|
||||
|
||||
## (attempt that became the fix) game-thread pacing rework
|
||||
FLOOP trace proved the loop: WAIT(N) on GAME thread (tid A) runs 1 frame ahead of BEGIN/END(N-1)
|
||||
on RENDER thread (tid B); steady BEGIN->END ~1ms. Old shim: WaitToBeginFrame=no-op, xrWaitFrame on
|
||||
RENDER thread inside ovrp_BeginFrame4 (a workaround for "BeginFrame too many times"). REWORK (matches
|
||||
vrapi + OpenXR's recommended pipelined model): xrWaitFrame now runs in xrr_wait_frame on the GAME
|
||||
thread (blocks=paces it), NOT under g_xrlock; the frameState is handed to the render thread via a 1:1
|
||||
FIFO ring (g_fsRing/g_fsHead/g_fsTail + g_fsCond). xrr_begin_frame pops the frameState (cond_timedwait
|
||||
20ms) instead of calling xrWaitFrame; xrBeginFrame/xrEndFrame stay on the render thread. Theory: the
|
||||
game thread was unpaced (no-op wait) so UE's internal pipeline desynced and the render thread stalled
|
||||
~85ms during motion -> dropped frames -> judder. Pacing the game thread should fix it. RISK: wait/
|
||||
begin must stay 1:1 to the runtime; if a begin is rejected (inFrame) after a wait was pushed, could
|
||||
desync -> watch for xrWaitFrame/xrBeginFrame xrfail. Keep ffr=-1 + blackcount=0 (both help). Testing.
|
||||
|
||||
## *** REFINED: BC (UE render thread) BLOCKS ~85ms at only ~14ms GPU = architectural (2026-06-27 latest) ***
|
||||
After FFR=-1 (applied ffr=1, GPU fed~13.7ms) AND blackcount/lumagate off (luma readback gone):
|
||||
ghost PERSISTS. STALL split now: BC (begin_frame->end_frame = UE render) = 47-101ms while GPU does
|
||||
only ~14ms of work => UE's RENDER THREAD is BLOCKING ~85ms, not computing. A(end->begin)~0.3-1.8ms,
|
||||
our end-frame phases <25ms. FFR helped (BC was 5ms one run) but BC hitch is non-deterministic and
|
||||
returns. Native at the SAME ~14ms GPU + same FFR = smooth (0.34% 1-frame drops). So the cause is
|
||||
NOT GPU load, NOT our per-frame overhead, NOT submit-timing — it's UE's render thread intermittently
|
||||
stalling under our OpenXR frame-loop. RULED OUT for the stall: xrWaitFrame (<25ms, 1 hit),
|
||||
lock-wait, flush, xrEndFrame, WaitSwapchainImage (all <25ms). The block is in UE's own
|
||||
render-recording window (begin_frame return -> EndFrame4 call).
|
||||
HYPOTHESIS (user's "vrapi lazy / openxr eager"): our frame loop paces the RENDER thread (xrWaitFrame
|
||||
in ovrp_BeginFrame4) and makes the GAME thread's ovrp_WaitToBeginFrame a NO-OP — opposite of vrapi,
|
||||
which paces the GAME thread. So the game thread runs unpaced/eager and the render-thread back-pressure
|
||||
(xrWaitFrame + frames-in-flight + compositor holding our 3 swapchain images during reproj) makes UE's
|
||||
render thread block intermittently => drops => judder. Comment at xrr_wait_frame says game-thread
|
||||
pacing was tried and caused "BeginFrame too many times" (XR_ERROR_CALL_ORDER_INVALID) -> they
|
||||
worked around by render-thread pacing. The REAL fix is likely an architectural frame-loop rework:
|
||||
pace the GAME thread (like vrapi) with correct xrWaitFrame/Begin/End 1:1:1 ordering across the two
|
||||
threads. Non-trivial, real risk. Other cheap-ish probes: more swapchain images (UE frames-in-flight
|
||||
stall if compositor holds our 3); check UE's own RHI frame-pacing/dynamic-res CVars.
|
||||
|
||||
## (helped, secondary) full-res render (FFR forced OFF) raised GPU load -> drop judder
|
||||
Stall localization (per-phase + gap-split probes in xr_runtime.c) showed the 105-250ms stalls are
|
||||
NOT in any of our blocking calls (xrWaitFrame/lock/flush/xrEndFrame/WaitSwapchainImage all <25ms,
|
||||
A=end->begin ~0.3-1.4ms) — they're in BC = begin_frame->end_frame = UE's own render. Matched-motion:
|
||||
native = 0.34% drops (all 1-frame); shim = more drops + 137-264ms stalls. So OUR shim makes UE's
|
||||
render hitch. WHY: GPU-TIME log shows `fed=14ms gameLevel=1 dynamic=0 -> applied ffr=0` — the GAME
|
||||
requests foveation (TiledMultiRes level 1, which native honors) but our shim had debug.re4vr.ffr=0
|
||||
FORCING foveation OFF -> UE renders FULL RES -> GPU pinned at ~14ms (right at the 13.9ms/72Hz budget,
|
||||
zero headroom) -> any head-motion load spike pushes GPU over budget -> render thread stalls on GPU ->
|
||||
dropped frame -> compositor timewarp judder = the ghost. Native applies the game's FFR -> headroom ->
|
||||
smooth. FIX (quality-neutral, matches native): debug.re4vr.ffr=-1 (game-driven) so we apply the
|
||||
game's requested foveation level. Testing now. If confirmed, make ffr=-1 (game-driven) the default
|
||||
in code (not 0). FFR maps game TiledMultiRes -> XR_FB_foveation (xr_runtime.c apply_foveation /
|
||||
foveation_entrypoints, ~L1766+).
|
||||
|
||||
## *** VISUALLY CONFIRMED: whole-frame TEMPORAL JUDDER (2026-06-27 late) ***
|
||||
Pulled the user's on-device recordings (/sdcard/Oculus/VideoShots/*.mp4, 30fps mono). Blending 3
|
||||
consecutive frames (ImageMagick -evaluate-sequence mean) of the title screen shows the "Resident
|
||||
Evil" banner + candle flames DOUBLED — two sharp offset copies (diagonal shift), WHOLE frame, not
|
||||
just close objects. = real temporal judder (two distinct positions), not motion blur, not stereo.
|
||||
Matches user: "blend frames shows it / not just close objects / mild-violent / random."
|
||||
Tooling that works: adb pull the VideoShots mp4 (adb screenrecord gives 0 bytes — Quest blocks the
|
||||
VR surface); ffmpeg extract frames; `compare -metric MAE` to find motion spikes; `convert
|
||||
-evaluate-sequence mean` to blend & reveal judder; Read the PNG to view it.
|
||||
Key: frame-pacing shows we DO present ~72fps (not half-rate), yet consecutive frames land at TWO
|
||||
positions => the predicted-display-time or submitted pose ALTERNATES/jitters frame-to-frame and the
|
||||
compositor timewarp snaps between spots. Added JUDDER probe: per-frame predictedDisplayTime delta in
|
||||
xrr_begin_frame (after xrWaitFrame) — steady ~13.9ms = pose source; alternating/jittery = timing.
|
||||
Mechanism candidate: our frame loop splits ovrp_WaitToBeginFrame(game thread, no-op) from
|
||||
xrWaitFrame+LocateViews+Begin (render thread, in xrr_begin_frame) — this nonstandard pacing can give
|
||||
the compositor jittery predicted times => judder. Native (vrapi) is phase-locked => smooth.
|
||||
|
||||
## (SUPERSEDED) render/composite SUBMIT-ORDERING race theory
|
||||
Chain of elimination, all by in-MOTION data (static title was a red herring — geometry matches
|
||||
statically; ghost only shows in motion):
|
||||
- FOV: render (CalculateEyeLayerDesc2) == composite (g_xr.views) == native, per-frame, stable.
|
||||
(Earlier "inner-fov mismatch" was MY arithmetic error: tan(0.785 rad)=1.000, not 0.991.)
|
||||
- IPD ~0.065, eye orientation == head, head == eye-mid, no LAYER MISMATCH (stage==acquired),
|
||||
full viewport, 3-image swapchain. ALL geometry/composition intrinsics correct.
|
||||
- DECISIVE: a 60-frame eye-texture burst dump (debug.re4vr.dump=N) showed EVERY frame CLEAN
|
||||
(single hand) AND the user saw NO ghost while the dump ran. The dump adds a per-frame GPU
|
||||
fence-wait (synchronous readback) that stalls the game thread enough that UE's eye render
|
||||
(on its own RHI-thread queue, NOT our s_queue — qwait was a no-op, confirming separate queue)
|
||||
lands before we release+xrEndFrame. => the ghost is: WE RELEASE THE SWAPCHAIN / xrEndFrame
|
||||
BEFORE UE SUBMITS ITS EYE RENDER. OpenXR then syncs the compositor against incomplete/previous
|
||||
content => flashing per-eye double on fast/close motion. NOT geometry, NOT depth, NOT reproject.
|
||||
- NEXT: test the built submit-hook (debug.re4vr.submithook=2 = present/release-on-submit; patches
|
||||
UE's global vkQueueSubmit PFN) — it orders our release AFTER UE's eye submit. Was refuted for
|
||||
BLACK but the ghost is a different artifact. If it fixes the ghost, refine to minimize latency.
|
||||
If not, try a completion fence injected at the hooked submit, or device-wait before release.
|
||||
|
||||
## Full call census (PTC, native returns) — 41 PT_FWD'd fns, all match our shim's returns
|
||||
All getters return success/same values in native and shim. Only diff: GetMixedRealityInitialized
|
||||
native=1 vs ours=0 (MR irrelevant to eye render). GetSystemDisplayFrequency2/PerfMetrics have
|
||||
pre-init transient failures (-1002/-1008) then succeed — same as ours. CONCLUSION: the game makes
|
||||
the same calls and gets the same answers in both modes => the ghost is NOT a getter-return diff;
|
||||
it's in COMPOSITION (our OpenXR layer submit vs native vrapi compositor), which is not a game call.
|
||||
|
||||
## *** KEY DIFFERENCE: native submits eye-fov with ReverseZ depth reprojection ***
|
||||
Native EndFrame4 eye-fov layer flags=**0x4 = ovrpLayerSubmitFlag_ReverseZ** (1<<2). NOT NoDepth(0x8).
|
||||
=> native composites with DEPTH-BASED POSITIONAL TIMEWARP, reverse-Z convention. And the game DOES
|
||||
request a depth buffer: our CalculateEyeLayerDesc2 logged depthFormat=10. Depth-aware reprojection
|
||||
is exactly what corrects close-object parallax under head translation — its absence = "deform/swim
|
||||
on close objects" = the seated hand ghost. PRIME SUSPECT.
|
||||
|
||||
Our shim CAN chain XrCompositionLayerDepthInfoKHR (xr_runtime.c build_composition L896, reverse-z via
|
||||
g_depthRevZ; depth swapchain created L2012 gated on `depth_wanted()` + game depthFormat). But memory
|
||||
says "depth on didn't fix it" — so VERIFY whether the game actually RENDERS valid depth into OUR
|
||||
depth swapchain (UE only renders depth if GetLayerTexture2 returns a depth handle AND its RHI targets
|
||||
it). If our depth image is empty/garbage, positional timewarp is a no-op (or worse) => ghost persists
|
||||
even with depth=1. That's the next probe.
|
||||
|
||||
## NEXT probe: verify our depth reprojection is functionally live (passthru=0, depth=1)
|
||||
1. "setup_layer: DEPTH swapchain ..." present? (swapchain created)
|
||||
2. does GetLayerTexture2 return a depth handle to the game (outDepthTex non-null path)?
|
||||
3. is depth chained each frame in build_composition (diag==0 && !pipeline && depthSwapchain)?
|
||||
4. is the depth IMAGE actually written by UE (dump min/max; all-1.0 or all-0 = not rendered)?
|
||||
If depth never reaches our swapchain -> that's the fix (wire UE's depth -> our depth image), and it
|
||||
would explain native(ReverseZ)=clean vs ours=ghost.
|
||||
|
||||
## (superseded) NEXT: in-game passthru (the actual repro)
|
||||
Title doesn't exercise the seated hand-deform. Run passthru=1, load save, play SEATED:
|
||||
(a) does native eliminate the hand-deform? If yes -> ghost is in our submission/render path,
|
||||
not geometry (since geometry matches). Capture in-game native EndFrame4/pose/fov, diff vs
|
||||
our in-game STEREO/views/HEADvsEYE logs at the same moment.
|
||||
(b) if native ALSO deforms -> not our shim's fault (game/headset).
|
||||
@@ -0,0 +1,86 @@
|
||||
# Lever 2 handoff — detect dropped/black frames and reproject instead of presenting black
|
||||
|
||||
Entry point for implementing the elegant fix to the in-game black. Read alongside the
|
||||
auto-memory: `black-is-not-submit-timing-ue-renders-empty`, `ingame-black-render-content`,
|
||||
`gameplay-eye-image-is-pure-black-confirmed`, `standing-vs-sitting-blackflash`.
|
||||
|
||||
## The settled root cause (do not re-litigate)
|
||||
The in-game black is a **UE-internal frame-drop under GPU load**, NOT a shim bug:
|
||||
- RenderDoc (`~/renderdoc-captures/RE4/work_frame.rdc`): under load UE renders only **~28
|
||||
draws into its eye target vs ~198 in a normal frame** (a truncated frame), and the
|
||||
**resolved eye = pure black** (MEAN [0,0,0], reliable ms=1 read).
|
||||
- Ruled out conclusively: submit-timing (present-on-submit incl. full one-frame defer
|
||||
still blacks — `debug.re4vr.submithook 2`/`defern 99`), image targeting (LAYER MISMATCH
|
||||
count = 0, stage==acquiredIndex), empty-frame submission (SUBMIT-BLACK/COMPOSE-EMPTY = 0).
|
||||
- Load-gated: bridge (sparse) never blacks; house/dense geometry blacks; "especially while
|
||||
casting" (extra load). The game does NOT self-scale from our GPU-time feed.
|
||||
|
||||
## The idea
|
||||
When UE hands us a truncated/black frame, **don't present it** — re-present the **last
|
||||
good** eye image with the current frame's pose, so the OpenXR compositor **timewarps/
|
||||
reprojects** the last good content to the new head pose. A reprojected (slightly stale)
|
||||
frame is far better than a black flash. This is exactly the "app didn't produce a new
|
||||
frame" path that compositors are built for; our black frames currently defeat it.
|
||||
|
||||
## Two hard parts
|
||||
|
||||
### A. A cheap "this frame is bad" signal (the crux)
|
||||
Per-frame GPU readback to detect black is too costly + unreliable (observer effect; see
|
||||
`texture-dump-is-unreliable-probe`). Candidate signals, cheapest first:
|
||||
1. **Post-hitch heuristic:** the truncated frame is the RECOVERY frame after a stall
|
||||
(RenderDoc: "recovery frame was 26 draws"). We already detect HITCH (dt>20ms in the
|
||||
FRAME trace). Try: skip+repeat the 1–2 frames following a detected hitch. Coarse but
|
||||
zero new cost; test first.
|
||||
2. **Submit/command-buffer count via the vkQueueSubmit hook (already built,
|
||||
`debug.re4vr.submithook 1`):** a truncated frame issues fewer submits / command buffers.
|
||||
Instrument submits-per-frame (between END markers) and correlate with perceived black.
|
||||
Under load we saw ~6 submits/frame normal — a dropped frame may show fewer. Needs a
|
||||
black ground-truth to calibrate (hard without readback; use the post-hitch frames as a
|
||||
proxy, or a one-off RenderDoc cross-check).
|
||||
3. **GPU frame time over budget:** `g_gpuFrameMs` already tracked; but it's the PREVIOUS
|
||||
frame's dt (lagging), so use it to predict the next frame is at-risk, not to gate the
|
||||
current one.
|
||||
Recommendation: start with (1) post-hitch repeat — simplest, no new signal — and measure.
|
||||
If it helps but is too coarse, add (2) via the hook.
|
||||
|
||||
### B. Re-presenting the last good frame
|
||||
OpenXR requires xrEndFrame every frame after xrBeginFrame; you can't simply skip present.
|
||||
To reproject, present the LAST GOOD eye image again with the current `predictedDisplayTime`
|
||||
+ located views (the runtime timewarps it). Mechanics:
|
||||
- Keep a reference to the last-good eye image. Two options: (a) DON'T release frame N-1's
|
||||
swapchain image and re-submit it (risky — holding across frames is what made
|
||||
`pipeline=1` unstable; see `deferred-flush-unstable-abandoned`), or (b) **copy** the last
|
||||
good eye image into a shim-owned VkImage (the copy-ring infra already exists:
|
||||
`xrr_vk_alloc_images`, `shimImages[]`, the copy path in end_frame) and present a normal
|
||||
fresh swapchain image blitted from the held copy. (b) avoids the cross-frame-hold
|
||||
instability.
|
||||
- On a bad frame: skip UE's (black) image, blit last-good copy → the acquired swapchain
|
||||
image, submit the previous composition's layer with the CURRENT pose/displayTime.
|
||||
- Gate the whole thing behind a new `debug.re4vr.skipblack` prop (default 0), like the
|
||||
other levers, so the known-good path is untouched.
|
||||
|
||||
## Key code locations (shim/src/xr_runtime.c unless noted)
|
||||
- `xrr_end_frame` sync branch (~1053): where present happens; add the skip+repeat here.
|
||||
- FRAME/HITCH trace (~1123): the hitch signal (dt>20ms) for heuristic (1).
|
||||
- `build_composition` (~620): the composition we'd re-submit with updated pose.
|
||||
- Copy-ring infra: `xrr_vk_alloc_images` / `shimImages[]` (vk_session.c) + the copy-ring
|
||||
path in end_frame (~922) — reuse for holding/blitting the last-good image.
|
||||
- vkQueueSubmit hook + `xrr_on_ue_submit` (~820): submits-per-frame instrumentation for
|
||||
signal (2). `debug.re4vr.submithook 1` = instrument.
|
||||
- Pose update for reprojection: `xrr_eye_fov_tangents` + the located views in `g_xr.views`.
|
||||
|
||||
## Validation
|
||||
Device: Quest 2 `<redacted-serial>` (USB). Build/deploy:
|
||||
`./shim/build_android.sh && ./packaging/repack.sh && adb -s <redacted-serial> install -r packaging/out/re4vr-shim.apk`
|
||||
(NOTE: repack silently bundles the LAST successful build — always confirm the build had no
|
||||
errors and check `shim/build/arm64/libOVRPlugin.so` timestamp before repack.)
|
||||
Test: Standing, walk off the bridge toward the house (reliable black trigger), `trace=1`.
|
||||
Success = black flashes replaced by (at worst) brief reprojection judder, not black.
|
||||
|
||||
## Prerequisite vs Lever 1
|
||||
If Lever 1 (force lower GPU load so UE completes frames: `ffr=3` + `debug.re4vr.resscale`
|
||||
< 100 + `sscap=1`) sufficiently stops the drops at acceptable quality, Lever 2 may be
|
||||
unnecessary or only needed for the worst spikes. Decide after the Lever 1 result.
|
||||
```
|
||||
debug.re4vr.resscale = percent of eye size (default 100; e.g. 75, 50). New this session.
|
||||
```
|
||||
@@ -0,0 +1,104 @@
|
||||
# Related work — how others run Quest games elsewhere, and why this shim is different
|
||||
|
||||
A survey of the projects in the same space, why none of them is what this repo is, and
|
||||
what (little) we'd adopt from them. Based on public repos + black-box surface inspection
|
||||
of publicly-distributed binaries — no decompilation of anyone's proprietary internals.
|
||||
|
||||
## TL;DR
|
||||
|
||||
There is **no public, open-source reimplementation of `libOVRPlugin.so` on OpenXR.** A
|
||||
whole-of-GitHub search for `ovrplugin` returns three repos, none a shim. The one tool that
|
||||
*runs* VrApi/OVRPlugin Quest games on other headsets — **Overport** — does it by
|
||||
**redistributing Meta's own newer OVRPlugin binary** plus a vendor OpenXR loader, not by
|
||||
reimplementing anything. So this project (a clean-room, from-scratch `ovrp_*` engine on
|
||||
OpenXR) appears to be the only open implementation of that translation layer.
|
||||
|
||||
## Overport (`ovrport/app`)
|
||||
|
||||
GPLv3, Kotlin/Compose, ~349★. **Two halves, only one of which is open:**
|
||||
|
||||
1. **The patcher (open, in the repo).** A Compose Multiplatform + ARSCLib app/CLI that
|
||||
rewrites a Quest APK: strips entitlements (via injected **Frida** scripts + a SKU/asset
|
||||
config), fixes the manifest, swaps icons/labels, and applies engine-specific smali
|
||||
patches. Its VR-library handling is three tiny patches:
|
||||
- `CopyOVRPluginVrApiPatch` — drop a bundled `libOVRPlugin.so` into any game that has
|
||||
`libvrapi.so`
|
||||
- `RemoveVrApiPatch` — delete the game's `libvrapi.so`
|
||||
- `CopyLibrariesPatch` — copy in the rest of the bundle
|
||||
|
||||
2. **The translation libraries (closed, NOT in the repo).** The libraries it injects are
|
||||
downloaded at patch time from the author's server
|
||||
(`ovrp.crx.moe/api/v1/releases/index` -> `files.crx.moe/.../libraries.zip`), version-
|
||||
managed separately. Source unpublished.
|
||||
|
||||
### What's actually in `libraries.zip` (black-box surface inspection)
|
||||
|
||||
The bundle is **Meta's / vendors' proprietary binaries**, not original code:
|
||||
|
||||
| File | What it is |
|
||||
|---|---|
|
||||
| `libOVRPlugin.so` (4.1 MB, 611 `ovrp_` exports) | **Meta's own OVRPlugin** — internal build paths intact in `.rodata` (`arvr/projects/integrations/OVRPlugin/Src/Util/CompositorOpenXR.cpp`). A *newer* build than RE4VR's bundled one (986 KB, VrApi-era), specifically one with the **OpenXR compositor backend**. |
|
||||
| `libopenxr_loader_{meta,pico,yvr,generic}.so` | Per-vendor OpenXR loaders; the patcher picks one for the target headset. |
|
||||
| `libovrplatformloader*.so`, `libpxrplatformloader.so` | Entitlement/platform loaders (the Frida-mocked entitlement piece). |
|
||||
|
||||
### Overport's actual strategy (now unambiguous)
|
||||
|
||||
For a VrApi-era title it: **replaces the game's old VrApi-routed Meta OVRPlugin with a newer
|
||||
Meta OVRPlugin that has an OpenXR backend**, deletes `libvrapi.so`, drops in the target
|
||||
vendor's OpenXR loader, and bypasses entitlement with Frida. The newer Meta OVRPlugin's
|
||||
`CompositorOpenXR` path then talks to e.g. Pico's OpenXR runtime.
|
||||
|
||||
**Consequences:**
|
||||
|
||||
- It ships **Meta's (and Pico's/YVR's) proprietary binaries** verbatim. That is a very
|
||||
different — and far more exposed — legal posture than a clean-room reimplementation.
|
||||
- It depends on the **newer OVRPlugin's C ABI still matching what the old game's UE/Unity
|
||||
build calls.** For an old VrApi-era UE4 title like RE4VR this is not guaranteed; the
|
||||
surface has drifted across OVRPlugin versions.
|
||||
- There is **no original translation code** to learn from — the hard part is Meta's, and
|
||||
we already have RE4VR's real `libOVRPlugin_real.so` for reference.
|
||||
|
||||
### Why this matters for us
|
||||
|
||||
Our shim is the open, clean-room alternative to the one piece Overport keeps closed (and
|
||||
which is, in fact, Meta's). It can't be replaced by their blob in a clean open product
|
||||
because their blob *is* Meta's binary. Their **patcher**, however, is genuinely useful prior
|
||||
art for the *packaging* layer (entitlement strip, manifest/engine smali patches) — see the
|
||||
off-Quest patches we adopt below.
|
||||
|
||||
## Quake III Arena VR Edition (`GUNNM-VR/...`)
|
||||
|
||||
A **source port**, not a shim. Built on the open-source Quake3e engine + baseq3a, recompiled
|
||||
for Android/Quest with Vulkan, using `#ifdef` to select OpenXR (PCVR) or VRAPI (Quest) at
|
||||
compile time. This is the easy case and the exact opposite of ours: with engine source you
|
||||
just compile against whichever runtime. We **can't** recompile RE4VR (closed UE4 binary), so
|
||||
we must be a binary-compatible `libOVRPlugin.so` instead. Useful only as a contrast.
|
||||
|
||||
## Off-Quest patches worth adopting (packaging layer)
|
||||
|
||||
For **RE4VR on a real Quest** we need none of these — the shim alone is sufficient (good
|
||||
confirmation). They become necessary only when porting the APK to a **non-Quest** Android VR
|
||||
device (Pico / Steam Frame / Monado-on-Android). Reimplemented in our own packaging in
|
||||
`packaging/steamframe_patches.sh` (see the `steamframe-port` branch):
|
||||
|
||||
| Patch (Overport name) | Why off-Quest | Status |
|
||||
|---|---|---|
|
||||
| `OculusUnrealPatch` | UE gates the Oculus HMD path on `Build.MANUFACTURER`/`MODEL`; off-Quest it's false so our shim is never called. Spoof them in `GameActivity` smali. | **adopt** |
|
||||
| `RemoveUsesLibraryPatch` | Strip `<uses-(native-)library>` entries (except `libopenxr.google.so`) that name Meta-only libs and would block install/launch elsewhere. | **adopt** |
|
||||
| `DisableControllerOffsetPatch` | Touch->other-controller pose offset; our `xr_input` has no offset, so non-Touch controllers may be misplaced. | **shim TODO** (runtime, not packaging) |
|
||||
| `RemoveUnrealForceQuitPatch` | Strip `System.exit` from `AndroidThunkJava_ForceQuit` so a failed off-Quest check can't hard-kill the app. | adopt (defensive) |
|
||||
| `FixUnrealCrashPatch` | Creates a stub `UnityPlayer.currentActivity` so an engine-agnostic injected blob can find the Activity. | **skip** — our shim gets the Activity natively from `Initialize5` + JNI. |
|
||||
| `MetaXRAudioPatch` | Hex-NOPs `libMetaXRAudioWwise/Unity.so`. | **N/A** — RE4VR uses `libovraudio64.so`, not Meta XR Audio. |
|
||||
| `DisableSpaceWarp` / `ForcePassthrough` | Config toggles. | N/A — RE4VR doesn't use AppSpaceWarp; equivalent to our `debug.re4vr.*` props. |
|
||||
|
||||
## Sources
|
||||
|
||||
- Overport patcher: <https://github.com/ovrport/app> (GPLv3)
|
||||
- Overport library index: `https://ovrp.crx.moe/api/v1/releases/index`
|
||||
- Quake III VR Edition: <https://github.com/GUNNM-VR/Quake-III-Arena-VR-Edition>
|
||||
- Meta deprecates VrApi / "all-in on OpenXR":
|
||||
<https://developers.meta.com/horizon/blog/oculus-all-in-on-openxr-deprecates-proprietary-apis/>
|
||||
- OVRPlugin vs VRAPI vs LibOVR:
|
||||
<https://developers.meta.com/horizon/documentation/unity/os-openxr-vrapi/>
|
||||
- Allegations Meta's OVRPlugin blocks non-Meta runtimes (Voices of VR #1526):
|
||||
<https://voicesofvr.com/1526-allegations-that-metas-ovrplugin-is-undermining-the-spirit-of-openxr-by-blocking-non-meta-headsets-on-pcvr/>
|
||||
@@ -0,0 +1,90 @@
|
||||
# Render-submit race — fix design (#2)
|
||||
|
||||
Pairs with `render-submit-sync-RE.md` (subagent RE of UE/OVRPlugin submit timing).
|
||||
Status: design draft; final approach (A vs B) gated on the RE findings.
|
||||
|
||||
## Confirmed mechanism
|
||||
- Our `ovrp_EndFrame4` → `xrEndFrame` presents **synchronously** (composites immediately).
|
||||
- UE 4.25's RHI thread `vkQueueSubmit`s the eye-render command buffer **after**
|
||||
`ovrp_EndFrame4` returns. **Proof:** `debug.re4vr.qwait` (a `vkQueueWaitIdle` on UE's
|
||||
queue *before* we release/present) is a **no-op** — if UE's render were already on the
|
||||
queue, draining it would turn the black image correct; it doesn't, so the submit isn't
|
||||
on the queue yet when `end_frame` runs.
|
||||
- The original VrApi path tolerates this because VrApi's EndFrame **also defers** the
|
||||
present onto the RHI submit (render flush + present ride the same queue submission / RHI
|
||||
flush), so render is naturally ordered before present. We broke that by presenting
|
||||
eagerly inside `xrEndFrame`.
|
||||
|
||||
## Why the current sync can't fix it (`vk_session.c`)
|
||||
`xrr_vk_flush_submit_ex` records a `COLOR_ATTACHMENT_WRITE→MEMORY_READ` barrier and
|
||||
submits it on UE's queue (`s_queue`), then `xrr_vk_flush_wait` waits its fence. Vulkan
|
||||
queue execution is in-order, so this is correct **iff UE already submitted the eye
|
||||
render**. Under load UE hasn't, so the barrier resolves an un-rendered image and the fence
|
||||
signals against empty content → we release+present black. Load-gated exactly as observed
|
||||
(sparse bridge = render makes the deadline = no black; dense geo = late = black).
|
||||
|
||||
## Prior attempts and why they failed
|
||||
- **Barrier-only (current default):** ordered before UE's later submit → black under load.
|
||||
- **qwait (`vkQueueWaitIdle`):** no-op (UE hasn't submitted). Diagnostic only.
|
||||
- **Deferred-flush pipeline (`pipeline=1`, present N-1):** conceptually right (gives UE a
|
||||
full frame to land its submit) but holds the **OpenXR swapchain image acquired across
|
||||
`xrEndFrame`** → Meta runtime mis-composites + crashes. Unstable; shelved.
|
||||
|
||||
## Fix options
|
||||
**A. Observe UE's submit (hook/interpose `vkQueueSubmit`) — most robust.**
|
||||
Wrap `vkQueueSubmit` so the shim sees exactly when UE flushes the eye render; record that
|
||||
submit's fence (or a timeline value). In `end_frame`, wait that fence before releasing +
|
||||
presenting. No swapchain-lifecycle hacks, minimal added latency (only the necessary wait).
|
||||
Open question (→ RE): is UE's `vkQueueSubmit` interceptable from our in-process `.so`
|
||||
(symbol interposition / a thin Vulkan layer), and which submit carries the eye render?
|
||||
|
||||
**B. Deferred present via a shim-owned copy (robust fallback, +1 frame latency).**
|
||||
Decouple the deferral from the OpenXR swapchain lifecycle (the cause of pipeline=1's
|
||||
instability): keep acquire→release **within a single frame**, but present one frame late.
|
||||
At frame N+1, UE's render-N submit has landed; copy UE's frame-N eye image into a
|
||||
shim-owned `VkImage` (barrier-ordered after UE's submit, on the same queue), then present
|
||||
the shim copy via a normal same-frame acquire/release. Never holds an OpenXR image across
|
||||
`xrEndFrame`. Costs 1 frame of latency + one image copy.
|
||||
|
||||
**C. Bounded spin-wait for the submit in `end_frame`** — fragile (no clean way to detect
|
||||
the submit without a hook), adds latency/stalls. Not recommended.
|
||||
|
||||
**D. Use an ovrp call UE makes around submit as the sync point** — only viable if the RE
|
||||
finds UE invokes an OVRPlugin entry point right after the render submit. Unlikely; → RE.
|
||||
|
||||
## RE outcome (`render-submit-sync-RE.md`) → Option A is feasible
|
||||
- **Original = zero Vulkan sync, definitively:** libOVRPlugin imports *no* `vk*` (only
|
||||
`vrapi_*`); libvrapi imports no `vk*` either. The compositor is a separate system
|
||||
process; swapchains are cross-process system-owned (`vrapi_CreateTextureSwapChainCrossProcess`).
|
||||
EndFrame4 only hands over swapchain handle + image index + pose; ordering is implicit via
|
||||
system swapchain ownership — the exact analogue of OpenXR acquire/wait/release.
|
||||
- **EndFrame4 runs on the RHI thread:** `FCustomPresent::FinishRendering_RHIThread` →
|
||||
`FOculusHMD::FinishRHIFrame_RHIThread` → `ovrp_EndFrame4` (via PluginWrapper dispatch).
|
||||
- **UE's eye-render submit is interceptable:** OculusHMD never calls `vkQueueSubmit`
|
||||
directly; it goes through the FVulkan RHI's **global dispatch pointer**
|
||||
`VulkanDynamicAPI::vkQueueSubmit` (a global PFN resolvable by symbol at runtime), batched/deferred on the RHI
|
||||
thread. A global PFN we can patch → trampoline. **This makes Option A viable and portable**
|
||||
(pure Vulkan + a UE symbol; no Quest/VrApi dependency, so it carries to Steam Frame).
|
||||
|
||||
## DECISION: Option A (hook `VulkanDynamicAPI::vkQueueSubmit`), but resolve the
|
||||
## threading interleave FIRST (instrument before we sync)
|
||||
Critical open question the RE could not pin from statics: **on the RHI thread, does the
|
||||
eye-render `vkQueueSubmit` happen BEFORE or AFTER the `EndFrame4` call?**
|
||||
- If **submit-before-EndFrame**: by EndFrame the render is on the queue; we just
|
||||
`vkWaitForFences` on the recorded submit fence before release/present. No deadlock, no
|
||||
added latency. (But the `qwait` no-op argues against this — nothing was on the queue.)
|
||||
- If **EndFrame-before-submit** (what `qwait` implies, same thread): we must NOT block in
|
||||
EndFrame (that thread does the later submit → self-deadlock). Instead **present-on-submit**:
|
||||
EndFrame stores the pending composition; the `vkQueueSubmit` trampoline, on seeing the
|
||||
eye-render submit, triggers the barrier+release+present. This mimics the original (present
|
||||
rides the RHI submit) with **no fixed frame of latency** and no cross-frame swapchain hold.
|
||||
|
||||
The interleave decides wait-in-EndFrame vs present-on-submit, so **step 1 is the hook as
|
||||
pure instrumentation** (no behavior change): patch the global PFN, log every submit with
|
||||
tid + timestamp + a monotonic seq, and log EndFrame with the same clock. One device run off
|
||||
the bridge confirms the order (and validates the patch offset + that `s_queue` is the queue
|
||||
UE submits eyes on — the RE's two stated uncertainties). Then implement the matching
|
||||
variant behind a `debug.re4vr.*` toggle.
|
||||
|
||||
Device test throughout: Standing, walk off the bridge toward the house (reliable black
|
||||
trigger), `trace=1`.
|
||||
Reference in new issue
Block a user