daniel-lynch--ovrplugin-openxr-shim/docs/handoffs/HANDOFF-2026-06-25.md

RE4 VR Shim — Handoff (2026-06-25 session)

Entry point for next session. Prior handoff: HANDOFF-2026-06-24.md. Detailed per-finding log lives in the auto-memory (MEMORY.md index).

TL;DR

• Root cause of the black flashes / 3×-hand / stutter is one thing: a GPU-load-gated render-submit race. Under load UE 4.25's RHI thread vkQueueSubmits the eye render after our xrEndFrame, so we present a stale/empty (black) swapchain image. PROVEN by texture dump (gameplay eye image min=max=0, pure black) and confirmed load-gated (in-game Standing comfort setting flashes black; Sitting doesn't — higher camera = more visible geometry = blows frame budget).
• Fixes tried this session:
  • vkQueueWaitIdle before resolve (debug.re4vr.qwait) — no effect (UE hadn't submitted yet; can't drain unsubmitted work).
  • Deferred-flush pipeline (debug.re4vr.pipeline=1, rewrote the render-ahead path to hold frame N and flush+present at N+1 after UE submits) — reduced pure black but crashes more + adds latency, marginal. SHELVED. Pipeline path has been the unstable one all along.
  • FFR (debug.re4vr.ffr, NEW this session) — implemented via XR_FB_foveation (dynamic). Validated (game uses foveation natively) and helps headroom/smoothness, but not enough to bring Standing under budget alone.
• RE of the original libs confirmed: ovrp_EndFrame4 does zero GPU sync — VrApi's system-owned textures carry it. Nothing to copy; our explicit-barrier approach is right, we just have a timing/threading mismatch. OpenXR gives the same free sync IF we submit-before-release (portable; the Quest-specific path would NOT port to Steam Frame).
• NEXT (the real lever): re-enable the game's native dynamic-performance subsystem that our stubs disable (see below). AppSW turned out to be a weaker lead — RE4 doesn't drive it.

PRIMARY next task — re-enable the game's dynamic-perf machinery

RE4 was built to scale its own GPU load under pressure, but we stub the whole subsystem as unsupported (logcat, all fire ~8×/session):

ovrp_GetGPUFrameTime              -> Unsupported       (game polls GPU frame time to decide)
ovrp_GetTiledMultiResLevel        -> Unsupported       (TiledMultiRes = Oculus name for FFR)
ovrp_SetTiledMultiResLevel        -> Unsupported
ovrp_SetTiledMultiResDynamic      -> Unsupported       (dynamic FFR driven by GPU time)
ovrp_IsPerfMetricsSupported       -> Unsupported
ovrp_GetSystemRecommendedMSAALevel2 -> NotYetImplemented
ovrp_GetLayerTextureFoveation     -> NotYetImplemented  (still stubbed even though we apply FFR at OpenXR)

Plan:

1. Report TiledMultiRes supported + honor Set* calls by mapping the game's requested FFR level/dynamic onto our OpenXR apply_foveation() (xr_runtime.c). Today FFR is a static prop (debug.re4vr.ffr); instead let the GAME drive the level (it already wants dynamic, GPU-time-based). Stub fns in shim/src/stubs.c; real handlers go in xr_runtime.c next to apply_foveation.
2. Implement ovrp_GetGPUFrameTime so the game's dynamic loop has its input. Source: OpenXR has no portable GPU-time query; options = XR_FB_... perf ext, or feed our own measured frame dt (we already compute it for the FRAME/HITCH trace). Even an approximate value lets the game's scaler engage.
3. Re-check Standing after: if the game drops its own resolution/foveation under load and the black stops, this is the fix. It's the game's intended GPU-bound mitigation.

SECONDARY — Application SpaceWarp (verify first, likely not viable as-is)

• ovrp_GetLayerTextureSpaceWarp is stubbed but RE4 does NOT call it this session; UE passes a 108-byte EyeFov desc, flags=0 (no motion-vector variant). So UE isn't rendering motion vectors → AppSW can't be fed.
• The system compositor IS running SpaceWarp/ASW (logcat tag SpaceWarp/SpaceWarpCore, PID 2438) — that's compositor-side reprojection (no app MV needed). It may already be reprojecting; our black frames defeat it (can't extrapolate a black frame).
• IF you pursue AppSW: needs (a) game to render velocity (UE 4.25 may not support it without the Meta fork), (b) provide MV+depth swapchains via ovrp_GetLayerTextureSpaceWarp, (c) submit XrCompositionLayerSpaceWarpInfoFB (chained on the projection view) via XR_FB_space_warp (extension + types already in bundled headers, openxr.h:6021+; XrSystemSpaceWarpPropertiesFB gives recommended MV size). Verify (a) before investing.

Current build / stable config

Build/deploy:

./shim/build_android.sh && ./packaging/repack.sh
adb -s <redacted-serial> install -r packaging/out/re4vr-shim.apk
adb -s <redacted-serial> logcat | grep -i xrr

Device Quest (USB serial <redacted-serial>, or <redacted-ip>:5555). Pkg com.Armature.VR4, activity com.epicgames.ue4.GameActivity. STABLE config left set: pipeline=0 (deferred-flush OFF), ffr=2 (medium), sscap=0. Consider compiling the deferred-flush path out / default FFR medium for a clean baseline.

Runtime debug toggles (system props; relaunch unless noted "live")

• debug.re4vr.ffr 0|1|2|3 — NEW. Fixed Foveated Rendering off/low/med/high (XR_FB_foveation, dynamic). Default 2.
• debug.re4vr.pipeline 1 — deferred-flush render-ahead (live). UNSTABLE/crashes; shelved.
• debug.re4vr.qwait 1 — diag: vkQueueWaitIdle before resolve (live). No effect; keep for reference.
• debug.re4vr.trace 1|2 — NEW. 1=per-frame FRAME line (dt+HITCH) + starvation logs; 2=per-call WAIT/BEGIN/END + per-layer dump (live).
• debug.re4vr.sscap 1 — cap supersample 1.2x→1.0x.
• debug.re4vr.copyring 1 — copy-ring (eye-only; gameplay black). From prior session.
• debug.re4vr.diag 1|2|3|4 — visual A/B: 1=proj only, 2=mono, 3=zoom, 4=head-lock quad (live).
• debug.re4vr.dump N — eye-texture readback to /sdcard/Android/data/com.Armature.VR4/files/*.ppm. UNRELIABLE probe — see below.
• debug.re4vr.depth 1, debug.re4vr.noflushwait 1 — prior-session probes.

What's RULED OUT for the black (don't re-investigate)

Quad placement/presence (diag=1/4), ViewportRect (UE always submits full), empty-frame submission (SUBMIT-BLACK=0 over 6730 frames), head-vs-eye pose mismatch (poses match exactly), stereo layout/IPD (array, correct), pure frame hitching (pipeline cut hitches 804→39 but black persisted). It is specifically the render-submit timing race, GPU-load-gated.

Diagnostic recipes (USE THESE — earlier probes misled us)

• Perception is ground truth. The dump probe is UNRELIABLE: it reads stale swapchain buffers AND its fence-wait forces the render (observer effect — "dumping forced the load").
• Recordings: limited-range. YAVG=16 IS black (not grey). 30fps mono UNDERSAMPLES 72Hz flashes (lower bound). Separate sustained black (menu/scene transitions) from brief isolated flashes. ffmpeg: -vf "signalstats,metadata=print:key=lavfi.signalstats.YAVG:file=y.txt"; blend with tmix=frames=N to reveal ghosting; search the WHOLE clip, not the start.
• Frame timing (reliable, not undersampled): debug.re4vr.trace 1 → grep HITCH / FRAME.
• RE pipeline: JAVA_HOME=tools/jdk-21.0.11+10 tools/ghidra_12.1.2_PUBLIC/support/analyzeHeadless ghidra_proj NAME -import dump/apk_libs/lib/arm64-v8a/libOVRPlugin.so -scriptPath ghidra_scripts -postScript DumpEndFrame.java -deleteProject. Original libs extracted in dump/apk_libs/.

Key code locations (xr_runtime.c unless noted)

• Eye-fov submit / build_composition (~560), the resolve barrier path is in vk_session.c (xrr_vk_flush_submit_ex, barrier COLOR_ATTACHMENT_WRITE→MEMORY_READ; NO QueueWaitIdle anywhere).
• submit_pending (~740): where a black frame reaches the compositor (SUBMIT-BLACK log).
• Pipeline (deferred-flush) branch in xrr_end_frame (~900); sync branch (~960).
• apply_foveation / ffr_level (just before xrr_setup_layer, ~1148) — FFR; extend here to let the game drive the level (TiledMultiRes handlers).
• Capability stubs to implement: shim/src/stubs.c (GetGPUFrameTime, TiledMultiRes*, etc.).
• Texture handoff: xrr_get_layer_texture (~1230) — UE renders into colorImages[stage].

Open side-bug (parked)

One-time Seated-mode height bug: spawned very high at title, height oscillated high/normal in-game; not replicable. Tracking-origin/recenter race at init (ovrp_SetTrackingOriginType2, ovrp_GetLocalTrackingSpaceRecenterCount stubbed). Note ReorientHMDOnControllerRecenter is stubbed.