daniel-lynch--ovrplugin-openxr-shim/docs/research/ghost-fix-2026-06-27.md

# P4 passthru — NATIVE ground-truth (2026-06-27, title screen, seated)

Captured via `debug.re4vr.passthru=1` (real libOVRPlugin owns the session through our shim).
Boots clean to the title screen, no ghost — so these are the CORRECT values to match.

## Eye layer desc (CalculateEyeLayerDesc2)
- size: **1440 x 1584** per eye
- FovL: U 1.111  D 1.192  L 0.933  R 1.000   (tangents)
- FovR: U 1.111  D 1.192  L 1.000  R 0.933
- => asymmetric (U≠D) AND canted stereo (per-eye L/R mirrored: inner edge = 1.000, outer = 0.933)

## EndFrame4 submit
- 1 layer, id=1 (first 3 frames) then id=2, **flags=0x4**, pose = all-zero (0,0,0)/(0,0,0,0)
- flags 0x4 = bit2 (NOT HeadLocked=0x1)

## Eye poses (GetNodePoseState3, head tilted on table)
- eye0 (L): pos=(-0.0934, 1.2170, 0.2354)  quat=(0.0314, 0.1109, -0.0207, -0.9931)
- eye1 (R): pos=(-0.0271, 1.2203, 0.2503)  quat=(same as L)
- both eyes share orientation; offset L->R ≈ (+0.066, +0.003, +0.015) m in world (IPD ~66mm)

## OUR shim values (captured passthru=0, title screen) — DIFF RESULT
- eye size: **1440x1584** — MATCHES native (the 1728x1900 was an old supersample config).
- FOV: ours via OpenXR xrLocateViews, logged as angles (rad). Steady-state L.fov(l,r,u,d)=
  (-0.750,0.785,0.838,-0.873) -> tangents U1.110 D1.190 L0.932 R0.991 — MATCHES native
  (U1.111 D1.192 L0.933 R1.000). The game does NOT call GetNodeFrustum2; FOV comes from
  CalculateEyeLayerDesc2 in BOTH paths.
- IPD: 0.065-0.068 — MATCHES native ~0.066.
- eye orientation: e0.q==e1.q==head.q in both — MATCHES.
- submit flags: eye-fov LayerId flags=0x4 — MATCHES native (0x14 later = menu-quad layer
  present, same game logic).

## CONCLUSION (title screen)
Static eye geometry (size/fov/ipd/orientation/flags) is IDENTICAL native vs shim at the title
screen. The seated in-game hand-deform ghost is therefore NOT a static-geometry mismatch.
ONE anomaly: our shim's FIRST ~5s report a WIDER outer FOV (L outer angle -0.855, tan 1.149)
that settles to native's -0.750/0.933 — a cold-start transient (matches title-ghost-is-coldload
-reprojection-judder; self-resolves warm). Not the persistent in-game ghost.

## CONFIRMED: passthru (native) has ZERO ghost — hands + title (user, 2026-06-27)
=> the ghost is in OUR shim's path, not the game/headset.

## *** ACTUAL ROOT CAUSE: DROPPED FRAMES -> compositor reprojection multiples (2026-06-27 PM) ***
The submit-ordering theory below was WRONG (submithook=2 present-on-submit AND submithook=3
+full device-wait-idle completion BOTH still ghost). Added always-on ANOMALY logging of the
OpenXR runtime RESPONSES (not our inputs, which all check out): locateViews validity,
WaitSwapchainImage timeout, xrEndFrame errors, frame-pacing. In-game ghost capture result:
  - frame-pacing: 42 MISS / 3986 frames (~1%), gaps 21-137ms vs 13.9ms period
  - locateViews invalid: 0   WaitSwapchain timeout: 0   xrEndFrame err: 0
=> the ONLY fault is DROPPED FRAMES. We miss the display deadline; the compositor reprojects the
held frame to fill the gap; during motion that = the flashing multiples. Scales exactly with user's
report: 21ms gap (1 drop)=mild, 137ms (~10 drops)=violent; load/position-dependent; non-deterministic;
clean dumped eye textures (pixels fine, just late). User CONFIRMED dump "no ghost" was the dump's
crawling fps freezing reprojection, not a fix — consistent.
KEY OPEN Q: are the drops OURS (shim latency, e.g. per-frame xrr_vk_flush_wait) or the game's load
(native hits them too but its compositor rides them out cleanly)? Added ENDFRAME-PACE log at
ovrp_EndFrame4 ENTRY (core.c) that runs in BOTH modes -> compare NATIVE vs shim gap rate.
  - if NATIVE also hits 137ms gaps clean -> fix = match native compositor frame-timing/handling
    (ExtraLatencyMode / phase sync / how late frames are submitted), NOT eliminate hitches.
  - if NATIVE smooth -> our shim adds the latency -> reduce it (flush-wait is prime suspect).
NOTE for Steam Frame: passthru is Meta-only (vrapi); the OpenXR shim is the only cross-platform
path, so this drop/repro fix is what matters for portability.

## *** SUCCESS: game-thread pacing FIXED the ghost (2026-06-27, user-confirmed) ***
User: title ghost GONE, seated ghost GONE, standing black-flash GONE, "way better... feels like a
great success." Logs: BC (render-thread stall) 148ms->~3ms (FIXED); frame-pacing drops 0.79%
(180/22862, was ~1%+10%-in-motion, native 0.34%); WAITPACE steady 13.92ms. The ROOT CAUSE was: our
shim left the game thread UNPACED (WaitToBeginFrame no-op) and paced the render thread instead, which
desynced UE's internal pipeline and stalled the render thread during motion -> dropped frames ->
compositor judder = the "ghost". Fix = pace the game thread (xrWaitFrame in xrr_wait_frame) + FIFO
frameState handoff to render thread. Keep ffr=-1 (game FFR) + blackcount=0. RESIDUAL (tunable):
occasional "BeginFrame too many times" -> XR_FRAME_DISCARDED = brief hiccup; from the ring DROPPING
oldest frameState when render falls behind (desyncs wait/begin 1:1). FIX: block the game thread for
ring space (back-pressure) instead of dropping. THEN: strip diagnostic scaffolding (FLOOP/STALL/
JUDDER/WAITPACE/burst-dump/submithook leftovers), make ffr=-1+blackcount=0 defaults, commit.

## (attempt that became the fix) game-thread pacing rework
FLOOP trace proved the loop: WAIT(N) on GAME thread (tid A) runs 1 frame ahead of BEGIN/END(N-1)
on RENDER thread (tid B); steady BEGIN->END ~1ms. Old shim: WaitToBeginFrame=no-op, xrWaitFrame on
RENDER thread inside ovrp_BeginFrame4 (a workaround for "BeginFrame too many times"). REWORK (matches
vrapi + OpenXR's recommended pipelined model): xrWaitFrame now runs in xrr_wait_frame on the GAME
thread (blocks=paces it), NOT under g_xrlock; the frameState is handed to the render thread via a 1:1
FIFO ring (g_fsRing/g_fsHead/g_fsTail + g_fsCond). xrr_begin_frame pops the frameState (cond_timedwait
20ms) instead of calling xrWaitFrame; xrBeginFrame/xrEndFrame stay on the render thread. Theory: the
game thread was unpaced (no-op wait) so UE's internal pipeline desynced and the render thread stalled
~85ms during motion -> dropped frames -> judder. Pacing the game thread should fix it. RISK: wait/
begin must stay 1:1 to the runtime; if a begin is rejected (inFrame) after a wait was pushed, could
desync -> watch for xrWaitFrame/xrBeginFrame xrfail. Keep ffr=-1 + blackcount=0 (both help). Testing.

## *** REFINED: BC (UE render thread) BLOCKS ~85ms at only ~14ms GPU = architectural (2026-06-27 latest) ***
After FFR=-1 (applied ffr=1, GPU fed~13.7ms) AND blackcount/lumagate off (luma readback gone):
ghost PERSISTS. STALL split now: BC (begin_frame->end_frame = UE render) = 47-101ms while GPU does
only ~14ms of work => UE's RENDER THREAD is BLOCKING ~85ms, not computing. A(end->begin)~0.3-1.8ms,
our end-frame phases <25ms. FFR helped (BC was 5ms one run) but BC hitch is non-deterministic and
returns. Native at the SAME ~14ms GPU + same FFR = smooth (0.34% 1-frame drops). So the cause is
NOT GPU load, NOT our per-frame overhead, NOT submit-timing — it's UE's render thread intermittently
stalling under our OpenXR frame-loop. RULED OUT for the stall: xrWaitFrame (<25ms, 1 hit),
lock-wait, flush, xrEndFrame, WaitSwapchainImage (all <25ms). The block is in UE's own
render-recording window (begin_frame return -> EndFrame4 call).
HYPOTHESIS (user's "vrapi lazy / openxr eager"): our frame loop paces the RENDER thread (xrWaitFrame
in ovrp_BeginFrame4) and makes the GAME thread's ovrp_WaitToBeginFrame a NO-OP — opposite of vrapi,
which paces the GAME thread. So the game thread runs unpaced/eager and the render-thread back-pressure
(xrWaitFrame + frames-in-flight + compositor holding our 3 swapchain images during reproj) makes UE's
render thread block intermittently => drops => judder. Comment at xrr_wait_frame says game-thread
pacing was tried and caused "BeginFrame too many times" (XR_ERROR_CALL_ORDER_INVALID) -> they
worked around by render-thread pacing. The REAL fix is likely an architectural frame-loop rework:
pace the GAME thread (like vrapi) with correct xrWaitFrame/Begin/End 1:1:1 ordering across the two
threads. Non-trivial, real risk. Other cheap-ish probes: more swapchain images (UE frames-in-flight
stall if compositor holds our 3); check UE's own RHI frame-pacing/dynamic-res CVars.

## (helped, secondary) full-res render (FFR forced OFF) raised GPU load -> drop judder
Stall localization (per-phase + gap-split probes in xr_runtime.c) showed the 105-250ms stalls are
NOT in any of our blocking calls (xrWaitFrame/lock/flush/xrEndFrame/WaitSwapchainImage all <25ms,
A=end->begin ~0.3-1.4ms) — they're in BC = begin_frame->end_frame = UE's own render. Matched-motion:
native = 0.34% drops (all 1-frame); shim = more drops + 137-264ms stalls. So OUR shim makes UE's
render hitch. WHY: GPU-TIME log shows `fed=14ms gameLevel=1 dynamic=0 -> applied ffr=0` — the GAME
requests foveation (TiledMultiRes level 1, which native honors) but our shim had debug.re4vr.ffr=0
FORCING foveation OFF -> UE renders FULL RES -> GPU pinned at ~14ms (right at the 13.9ms/72Hz budget,
zero headroom) -> any head-motion load spike pushes GPU over budget -> render thread stalls on GPU ->
dropped frame -> compositor timewarp judder = the ghost. Native applies the game's FFR -> headroom ->
smooth. FIX (quality-neutral, matches native): debug.re4vr.ffr=-1 (game-driven) so we apply the
game's requested foveation level. Testing now. If confirmed, make ffr=-1 (game-driven) the default
in code (not 0). FFR maps game TiledMultiRes -> XR_FB_foveation (xr_runtime.c apply_foveation /
foveation_entrypoints, ~L1766+).

## *** VISUALLY CONFIRMED: whole-frame TEMPORAL JUDDER (2026-06-27 late) ***
Pulled the user's on-device recordings (/sdcard/Oculus/VideoShots/*.mp4, 30fps mono). Blending 3
consecutive frames (ImageMagick -evaluate-sequence mean) of the title screen shows the "Resident
Evil" banner + candle flames DOUBLED — two sharp offset copies (diagonal shift), WHOLE frame, not
just close objects. = real temporal judder (two distinct positions), not motion blur, not stereo.
Matches user: "blend frames shows it / not just close objects / mild-violent / random."
Tooling that works: adb pull the VideoShots mp4 (adb screenrecord gives 0 bytes — Quest blocks the
VR surface); ffmpeg extract frames; `compare -metric MAE` to find motion spikes; `convert
-evaluate-sequence mean` to blend & reveal judder; Read the PNG to view it.
Key: frame-pacing shows we DO present ~72fps (not half-rate), yet consecutive frames land at TWO
positions => the predicted-display-time or submitted pose ALTERNATES/jitters frame-to-frame and the
compositor timewarp snaps between spots. Added JUDDER probe: per-frame predictedDisplayTime delta in
xrr_begin_frame (after xrWaitFrame) — steady ~13.9ms = pose source; alternating/jittery = timing.
Mechanism candidate: our frame loop splits ovrp_WaitToBeginFrame(game thread, no-op) from
xrWaitFrame+LocateViews+Begin (render thread, in xrr_begin_frame) — this nonstandard pacing can give
the compositor jittery predicted times => judder. Native (vrapi) is phase-locked => smooth.

## (SUPERSEDED) render/composite SUBMIT-ORDERING race theory
Chain of elimination, all by in-MOTION data (static title was a red herring — geometry matches
statically; ghost only shows in motion):
- FOV: render (CalculateEyeLayerDesc2) == composite (g_xr.views) == native, per-frame, stable.
  (Earlier "inner-fov mismatch" was MY arithmetic error: tan(0.785 rad)=1.000, not 0.991.)
- IPD ~0.065, eye orientation == head, head == eye-mid, no LAYER MISMATCH (stage==acquired),
  full viewport, 3-image swapchain. ALL geometry/composition intrinsics correct.
- DECISIVE: a 60-frame eye-texture burst dump (debug.re4vr.dump=N) showed EVERY frame CLEAN
  (single hand) AND the user saw NO ghost while the dump ran. The dump adds a per-frame GPU
  fence-wait (synchronous readback) that stalls the game thread enough that UE's eye render
  (on its own RHI-thread queue, NOT our s_queue — qwait was a no-op, confirming separate queue)
  lands before we release+xrEndFrame. => the ghost is: WE RELEASE THE SWAPCHAIN / xrEndFrame
  BEFORE UE SUBMITS ITS EYE RENDER. OpenXR then syncs the compositor against incomplete/previous
  content => flashing per-eye double on fast/close motion. NOT geometry, NOT depth, NOT reproject.
- NEXT: test the built submit-hook (debug.re4vr.submithook=2 = present/release-on-submit; patches
  UE's global vkQueueSubmit PFN) — it orders our release AFTER UE's eye submit. Was refuted for
  BLACK but the ghost is a different artifact. If it fixes the ghost, refine to minimize latency.
  If not, try a completion fence injected at the hooked submit, or device-wait before release.

## Full call census (PTC, native returns) — 41 PT_FWD'd fns, all match our shim's returns
All getters return success/same values in native and shim. Only diff: GetMixedRealityInitialized
native=1 vs ours=0 (MR irrelevant to eye render). GetSystemDisplayFrequency2/PerfMetrics have
pre-init transient failures (-1002/-1008) then succeed — same as ours. CONCLUSION: the game makes
the same calls and gets the same answers in both modes => the ghost is NOT a getter-return diff;
it's in COMPOSITION (our OpenXR layer submit vs native vrapi compositor), which is not a game call.

## *** KEY DIFFERENCE: native submits eye-fov with ReverseZ depth reprojection ***
Native EndFrame4 eye-fov layer flags=**0x4 = ovrpLayerSubmitFlag_ReverseZ** (1<<2). NOT NoDepth(0x8).
=> native composites with DEPTH-BASED POSITIONAL TIMEWARP, reverse-Z convention. And the game DOES
request a depth buffer: our CalculateEyeLayerDesc2 logged depthFormat=10. Depth-aware reprojection
is exactly what corrects close-object parallax under head translation — its absence = "deform/swim
on close objects" = the seated hand ghost. PRIME SUSPECT.

Our shim CAN chain XrCompositionLayerDepthInfoKHR (xr_runtime.c build_composition L896, reverse-z via
g_depthRevZ; depth swapchain created L2012 gated on `depth_wanted()` + game depthFormat). But memory
says "depth on didn't fix it" — so VERIFY whether the game actually RENDERS valid depth into OUR
depth swapchain (UE only renders depth if GetLayerTexture2 returns a depth handle AND its RHI targets
it). If our depth image is empty/garbage, positional timewarp is a no-op (or worse) => ghost persists
even with depth=1. That's the next probe.

## NEXT probe: verify our depth reprojection is functionally live (passthru=0, depth=1)
1. "setup_layer: DEPTH swapchain ..." present? (swapchain created)
2. does GetLayerTexture2 return a depth handle to the game (outDepthTex non-null path)?
3. is depth chained each frame in build_composition (diag==0 && !pipeline && depthSwapchain)?
4. is the depth IMAGE actually written by UE (dump min/max; all-1.0 or all-0 = not rendered)?
If depth never reaches our swapchain -> that's the fix (wire UE's depth -> our depth image), and it
would explain native(ReverseZ)=clean vs ours=ghost.

## (superseded) NEXT: in-game passthru (the actual repro)
Title doesn't exercise the seated hand-deform. Run passthru=1, load save, play SEATED:
(a) does native eliminate the hand-deform? If yes -> ghost is in our submission/render path,
    not geometry (since geometry matches). Capture in-game native EndFrame4/pose/fov, diff vs
    our in-game STEREO/views/HEADvsEYE logs at the same moment.
(b) if native ALSO deforms -> not our shim's fault (game/headset).