diff --git a/OPENXR.md b/OPENXR.md index 2d243a2..097b79d 100644 --- a/OPENXR.md +++ b/OPENXR.md @@ -320,6 +320,19 @@ short-lived immutable stereo packet. Each sealed GX frame and immersive packet c policy-generation tag; a mismatch is rendered in mono and the acquired XR frame is canceled, so an asynchronous menu/race transition cannot replay race transforms over unsafe content. +With interpolation off, the standalone (Vulkan) backend paces render-first: the pacing thread +locates the views for an estimated display time (two periods past the last one the compositor +predicted), hands Aurora the packet and its shared-buffer targets with no compositor frame open, +and waits for Aurora to render the eyes at its next seal, repeating the retained layer if that +takes more than 50 ms. Only then does it call xrWaitFrame and xrBeginFrame, copy the eyes into the +freshly acquired swapchain images and end the frame with the packet's render poses. A headset +frame therefore stays open for the copy alone instead of for the next 60 Hz game frame plus the +whole encode, which on a 72 or 90 Hz display used to make every second cycle span two display +slots (about 45 headset frames per second while the game rendered 60). The compositor reprojects +the rendered pose to the frame it lands in. The D3D12 backend keeps the frame-first order below, +as does interpolation on either backend, since interpolation renders for the frame's own +predicted display time. + With VR interpolation enabled, Aurora retains each sealed race's command stream and matched previous/current transform uniforms. New OpenXR packets wake the frame worker between game frames. It interpolates at the requested display time, then applies that packet's head pose and diff --git a/docs/quest-port.md b/docs/quest-port.md index eccd422..4e934f4 100644 --- a/docs/quest-port.md +++ b/docs/quest-port.md @@ -661,12 +661,31 @@ stereo frame cost 13.2 ms, of which the native render was 5.7 ms, the eyes 3.5 and 3.8, copies and gaps 0.4. That native render is a 1280x720 image nobody sees during an immersive race, so it now stops after the last pass whose EFB copy the eyes sample: `mono` fell to 0.15 ms and a Luigi Circuit start at 0.5 -renders in 5.5 to 10 ms of GPU per frame. What remains is the headset pacing: -with the display at 72 or 90 Hz, each headset frame stays open for the next -60 Hz game frame plus the whole encode (`open` 16 ms in the pacing summary), -so cycles span one to two display slots and the headset gets 40 to 60 frames -per second while the game renders 60. Reworking that pacing (encode the newest -sealed frame at once, repeat the layer otherwise) is the next step. +renders in 5.5 to 10 ms of GPU per frame. The last limiter was the headset +pacing: with the display at 72 or 90 Hz, each headset frame stayed open for +the next 60 Hz game frame plus the whole encode (`open` 16 ms in the pacing +summary), so cycles spanned one to two display slots and the headset got 40 to +60 frames per second while the game rendered 60. The Vulkan backend now paces +render-first (`PreparePacket`, `BeginFrameForPacket`, `CopyRenderedEyes` in +`openxr_vulkan.cpp`; see `OPENXR.md`): the packet is located and handed to +Aurora with no compositor frame open, and the frame is begun only once the +eyes exist, for the copy alone. On the same automated start at 0.75 the +summary reads `cycles=60 skipped-slots=12 late=0 layers new=60 repeat=0 +open=5.5 end-gap=16.7`, the compositor shows 60 to 61 of 72 with the +inherent 12 stale slots, app-to-compositor latency fell from 51 to 9 to 13 ms, +and the frame worker's encode fell from 8 to 2.7 ms because the eye copy and +its fence wait moved off the worker onto the pacing thread. + +Retro Rewind tracks then showed a game-thread limit of their own: on Athens +Dash (a Mario Kart Tour port) the display-list index scan +(`WalkDisplayList`) was 11.5% of the thread while the base +game's tracks spend 0.3% there. The scan cache in `gx_dl.cpp` refused lists +above 64 KiB, so that track's large shape lists were scanned again on every +call; the cap is now 4 MiB. With it the scan is 0.2%, the game rate on Athens +Dash went from 47 to 51 fps to 50 to 58, and the thread splits into 62% game +plus mod code, 9% GX HLE, 6% FIFO decode, 4% memory copies, 3.5% dispatch and +the rest. What remains on such tracks is the game's own code plus the mod's, +which no host change shrinks; a GX thread could move about 20% of it. Verified on device since: the menus on the virtual screen, controller input (the user has driven races), and an immersive Grand Prix start with all 12 diff --git a/runtime/include/vr/openxr_runtime.h b/runtime/include/vr/openxr_runtime.h index 896ca1d..df05d7c 100644 --- a/runtime/include/vr/openxr_runtime.h +++ b/runtime/include/vr/openxr_runtime.h @@ -153,6 +153,11 @@ public: OpenXRFrameStatus WaitFrame(OpenXRFrame& frame); bool BeginFrame(const OpenXRFrame& frame); bool LocateViews(OpenXRFrame& frame); + // Locates the views for `display_time` outside the frame protocol, for a + // packet whose eyes are rendered before the compositor frame that will show + // them is begun (the standalone backend's render-first pacing). Fills the + // frame's display time, views, flags and validity; requires a running session. + bool LocateViewsAt(XrTime display_time, OpenXRFrame& frame); bool EndFrame( const OpenXRFrame& frame, const XrCompositionLayerBaseHeader* const* layers, @@ -224,6 +229,7 @@ private: bool EnumerateSwapchainFormats(); bool HandleSessionStateChanged(const XrEventDataSessionStateChanged& event); bool IsFrameTokenCurrent(const OpenXRFrame& frame, FramePhase expected) const; + bool LocateViewsForFrame(OpenXRFrame& frame); void ResetFrameState(); void DestroyReferenceSpaces(); void ResetSessionState(); diff --git a/runtime/include/vr/openxr_vulkan.h b/runtime/include/vr/openxr_vulkan.h index 2b23399..e185bbc 100644 --- a/runtime/include/vr/openxr_vulkan.h +++ b/runtime/include/vr/openxr_vulkan.h @@ -58,6 +58,20 @@ public: bool RepeatFrame(const OpenXRBackendFrame& frame); bool FinishFrame(OpenXRBackendFrame& frame, bool submit_layer); + // Render-first pacing, used while VR interpolation is off. A packet is prepared with the + // views located for an estimated display time and handed to Aurora without a compositor + // frame open; once Aurora has rendered the eyes into the shared buffers, the compositor frame + // is begun, the eyes are copied into its swapchain images and it is ended at once. A headset + // frame therefore never waits for a game frame: it stays open for the copy alone. + OpenXRBeginStatus PreparePacket(const OpenXRPresentation& presentation, OpenXRBackendFrame& packet); + bool TryCancelPendingPacket(OpenXRBackendFrame& packet); + OpenXRBeginStatus BeginFrameForPacket(const OpenXRBackendFrame& packet, OpenXRBackendFrame& frame); + OpenXRSubmissionStatus CopyRenderedEyes(const OpenXRBackendFrame& frame); + // One compositor cycle that resubmits the retained layer (or nothing), with no frame left + // active: keeps the runtime fed while the eyes are still being rendered and learns the + // display timing the next packet is located for. + OpenXRBeginStatus KeepAliveCycle(); + // Drains this backend's own queue before tearing down. Returns false only // when the private device could not be waited on, in which case the caller // retains the backend and runtime for the process lifetime. diff --git a/runtime/src/hle/gx/gx_dl.cpp b/runtime/src/hle/gx/gx_dl.cpp index 4a984a7..5a6bdd8 100644 --- a/runtime/src/hle/gx/gx_dl.cpp +++ b/runtime/src/hle/gx/gx_dl.cpp @@ -28,9 +28,13 @@ using GxCpDecode::SameVtxAttrFmt; // Small display lists dominate the in-race call count. Cache them as well, but // cap both individual entries and aggregate copied command bytes so malformed // guest input cannot turn this optimization into unbounded host allocation. -constexpr uint32_t kDlScanCacheMaxEntryBytes = 64u * 1024u; +// The entry cap was 64 KiB: a list above it was never cached and its index +// scan ran on every call, which on a Retro Rewind track with large shape lists +// was 11% of the game thread on a Quest 3. Only lists that need flattening +// store a copy, and the aggregate cap still bounds those. +constexpr uint32_t kDlScanCacheMaxEntryBytes = 4u * 1024u * 1024u; constexpr size_t kDlScanCacheMaxEntries = 8192; -constexpr size_t kDlScanCacheMaxStoredBytes = 8u * 1024u * 1024u; +constexpr size_t kDlScanCacheMaxStoredBytes = 32u * 1024u * 1024u; // Display-list write tracking (audit F6a): re-digesting every list every call is the // costliest step of GX__CallDisplayList, and wasted on BRRES shape lists that are written diff --git a/runtime/src/vr/openxr_integration.cpp b/runtime/src/vr/openxr_integration.cpp index f66aef4..11522cf 100644 --- a/runtime/src/vr/openxr_integration.cpp +++ b/runtime/src/vr/openxr_integration.cpp @@ -802,6 +802,19 @@ private: const uint32_t interpolation_target = frame_interpolation_fps_.load(std::memory_order_relaxed); SetInterpolationActive(immersive && FrameInterpolationAvailable() && interpolation_target != 0); +#if !defined(_WIN32) + // Standalone (Vulkan) backend: with interpolation off, the eyes are rendered before + // the compositor frame that shows them is begun, so that frame never waits for a + // game frame. Interpolation keeps the frame-first order below: it renders for the + // frame's own predicted display time. + if (!aurora_get_stereo_frame_interpolation()) { + if (!RenderFirstCycle(presentation, policy, immersive, consecutive_skips, + immersive_submission_logged)) { + fatal = true; + } + continue; + } +#endif OpenXRBackendFrame frame{}; const OpenXRBeginStatus begin = backend_->BeginFrame(presentation, frame); if (begin == OpenXRBeginStatus::SessionNotRunning) { @@ -961,6 +974,164 @@ private: ShutdownOrRetainGraphicsObjects(); } +#if !defined(_WIN32) + // One compositor cycle on the retained layer, with no frame left active. False on a fatal + // backend or runtime failure (the error is recorded). + bool KeepAlive() { + const OpenXRBeginStatus status = backend_->KeepAliveCycle(); + if (status == OpenXRBeginStatus::SessionNotRunning) { + MkwVRPolicySetSessionActive(false); + return true; + } + if (status == OpenXRBeginStatus::ExitRequested) { + SetError("OpenXR runtime requested session exit; continuing on the mirror output"); + return false; + } + if (status == OpenXRBeginStatus::Error) { + SetError(backend_->LastError()); + return false; + } + return true; + } + + // Render-first pacing (see OpenXRVulkanBackend::PreparePacket). Returns false on a fatal + // failure; a cycle that ends without a layer returns true and the loop tries again. + bool RenderFirstCycle(OpenXRPresentation presentation, const MkwVRPolicySnapshot& policy, bool immersive, + uint32_t& consecutive_skips, bool& immersive_submission_logged) { + OpenXRBackendFrame packet{}; + const OpenXRBeginStatus prepared = backend_->PreparePacket(presentation, packet); + if (prepared == OpenXRBeginStatus::SessionNotRunning) { + MkwVRPolicySetSessionActive(false); + return true; + } + if (prepared == OpenXRBeginStatus::ExitRequested) { + SetError("OpenXR runtime requested session exit; continuing on the mirror output"); + return false; + } + if (prepared == OpenXRBeginStatus::Error) { + SetError(backend_->LastError()); + return false; + } + // The head pose this packet was located with places the screens and aims the pointer. + ServiceRecenterRequest(); + UpdateVirtualScreenPose(packet); + if (input_ != nullptr) { + input_->Sync(packet.xr_frame.predicted_display_time, PointerScreen(packet, policy, immersive), + SettingsPanelScreen(packet, policy, immersive)); + } + if (!packet.expects_gpu_submission) { + // Nothing to render (no rendering requested or no tracking): keep the compositor fed. + return KeepAlive(); + } + { + std::lock_guard lock(published_mutex_); + BuildPublishedFrame(packet, immersive, policy.EffectiveUnitsPerMeter(), policy.content_tag); + diagnostics::OnPacketPublished(); + published_.store(&published_frame_, std::memory_order_release); + } + aurora_notify_stereo_frame(); + + // Aurora renders the eyes at its next seal. Meanwhile the compositor keeps showing the + // retained layer; a 50 ms stall repeats it explicitly and withdraws the packet. + OpenXRSubmissionStatus submission = OpenXRSubmissionStatus::Timeout; + bool canceled_before_encode = false; + const auto cancel_after = std::chrono::steady_clock::now() + std::chrono::milliseconds(50); + while (!stop_.load(std::memory_order_acquire) && submission == OpenXRSubmissionStatus::Timeout) { + submission = backend_->WaitForSubmission(packet, 50); + if (submission == OpenXRSubmissionStatus::Timeout) { + if (std::chrono::steady_clock::now() >= cancel_after) { + WithdrawPublishedFrame(); + canceled_before_encode = backend_->TryCancelPendingPacket(packet); + if (canceled_before_encode) { + diagnostics::OnPacketCanceled(); + break; + } + } + diagnostics::OnKeepaliveRepeat(); + if (!KeepAlive()) { + return false; + } + } + } + WithdrawPublishedFrame(); + if (stop_.load(std::memory_order_acquire) || canceled_before_encode || + submission == OpenXRSubmissionStatus::ShuttingDown) { + return true; + } + if (submission != OpenXRSubmissionStatus::Success) { + // Nothing reached the shared buffers (Skipped) or Aurora failed after queuing GPU + // work (Failed): same accounting as the frame-first path, on a keep-alive cycle. + diagnostics::OnSubmission(false); + if (submission == OpenXRSubmissionStatus::Failed) { + SetError(std::string("Aurora's ") + kGraphicsBackendName + + " stereo copy failed; continuing on the mirror output"); + return false; + } + ++consecutive_skips; + if (consecutive_skips == 1 || consecutive_skips % 60 == 0) { + RT_LOG(RT_TAG_RUNTIME) << "OpenXR: eye copy skipped (" << consecutive_skips + << " in a row): " << backend_->LastError() << std::endl; + } + if (consecutive_skips >= kMaxConsecutiveSkips) { + SetError(std::string("Aurora's ") + kGraphicsBackendName + + " stereo copy keeps failing; continuing on the mirror output"); + return false; + } + return KeepAlive(); + } + + // The eyes are in the shared buffers: begin the compositor frame, copy, end. + OpenXRBackendFrame frame{}; + const OpenXRBeginStatus begin = backend_->BeginFrameForPacket(packet, frame); + if (begin == OpenXRBeginStatus::SessionNotRunning) { + MkwVRPolicySetSessionActive(false); + return true; + } + if (begin == OpenXRBeginStatus::ExitRequested) { + SetError("OpenXR runtime requested session exit; continuing on the mirror output"); + return false; + } + if (begin == OpenXRBeginStatus::Error) { + SetError(backend_->LastError()); + return false; + } + UpdateFrameTiming(frame.xr_frame); + if (diagnostics::Enabled()) { + NoteFrameDiagnostics(frame, immersive); + } + const OpenXRSubmissionStatus copy = + frame.expects_gpu_submission ? backend_->CopyRenderedEyes(frame) : OpenXRSubmissionStatus::Skipped; + const bool submit = copy == OpenXRSubmissionStatus::Success; + diagnostics::OnSubmission(submit); + if (!backend_->FinishFrame(frame, submit)) { + SetError(backend_->LastError()); + return false; + } + if (copy == OpenXRSubmissionStatus::Failed) { + SetError(std::string("Aurora's ") + kGraphicsBackendName + + " stereo copy failed; continuing on the mirror output"); + return false; + } + if (!submit) { + ++consecutive_skips; + if (consecutive_skips == 1 || consecutive_skips % 60 == 0) { + RT_LOG(RT_TAG_RUNTIME) << "OpenXR: eye copy skipped (" << consecutive_skips + << " in a row): " << backend_->LastError() << std::endl; + } + return consecutive_skips < kMaxConsecutiveSkips; + } + consecutive_skips = 0; + ++timing_submissions_; + if (immersive && !immersive_submission_logged) { + immersive_submission_logged = true; + RT_LOG(RT_TAG_RUNTIME) << "[mkw-vr] first immersive packet consumed and submitted as " + "an OpenXR projection layer" + << std::endl; + } + return true; + } +#endif + void BuildPublishedFrame(const OpenXRBackendFrame& source, bool immersive, float units_per_meter, uint64_t content_tag) noexcept { ApplyPendingReferenceSpaceChange(source.xr_frame); diff --git a/runtime/src/vr/openxr_runtime.cpp b/runtime/src/vr/openxr_runtime.cpp index 386b1d9..fb183b2 100644 --- a/runtime/src/vr/openxr_runtime.cpp +++ b/runtime/src/vr/openxr_runtime.cpp @@ -741,6 +741,19 @@ bool OpenXRRuntime::LocateViews(OpenXRFrame& frame) { return Fail(XR_ERROR_CALL_ORDER_INVALID, "xrLocateViews", "frame token is stale or xrBeginFrame was not called"); } + return LocateViewsForFrame(frame); +} + +bool OpenXRRuntime::LocateViewsAt(XrTime display_time, OpenXRFrame& frame) { + ClearError(); + if (!HasSession() || !m_session_running) { + return Fail(XR_ERROR_SESSION_NOT_RUNNING, "xrLocateViews", "the session is not running"); + } + frame.predicted_display_time = display_time; + return LocateViewsForFrame(frame); +} + +bool OpenXRRuntime::LocateViewsForFrame(OpenXRFrame& frame) { frame.views_valid = false; frame.view_state_flags = 0; if (!frame.should_render) { diff --git a/runtime/src/vr/openxr_vulkan.cpp b/runtime/src/vr/openxr_vulkan.cpp index 8da3592..5783766 100644 --- a/runtime/src/vr/openxr_vulkan.cpp +++ b/runtime/src/vr/openxr_vulkan.cpp @@ -482,6 +482,139 @@ public: return true; } + // ---- Render-first pacing (see openxr_vulkan.h) --------------------------------------------- + + void DiscardPendingReleasesLocked() noexcept { + if (!have_pending_releases_) { + return; + } + for (auto& release : pending_releases_) { + CloseFd(release.releaseFenceFd); + release = {-1, VK_IMAGE_LAYOUT_UNDEFINED}; + } + have_pending_releases_ = false; + } + + OpenXRBeginStatus PreparePacket(const OpenXRPresentation& presentation, OpenXRBackendFrame& packet) { + packet = {}; + packet.presentation = presentation; + if (!bound_ || runtime_ == nullptr) { + Fail("PreparePacket called before the Vulkan backend was bound"); + return OpenXRBeginStatus::Error; + } + if (frame_active_) { + Fail("PreparePacket called while an OpenXR frame is active"); + return OpenXRBeginStatus::Error; + } + if (runtime_->ShouldExit()) { + return OpenXRBeginStatus::ExitRequested; + } + if (!runtime_->IsSessionRunning()) { + return OpenXRBeginStatus::SessionNotRunning; + } + if (last_display_period_ <= 0) { + // No display timing yet: one compositor cycle learns it. + const OpenXRBeginStatus primed = KeepAliveCycle(); + if (primed != OpenXRBeginStatus::Ready) { + return primed; + } + } + packet.xr_frame.serial = next_packet_serial_++; + // The eyes are ready after at most one game frame plus the encode and show at the first + // display slot after that: two periods past the last predicted display time. + packet.xr_frame.predicted_display_time = last_display_time_ + 2 * last_display_period_; + packet.xr_frame.predicted_display_period = last_display_period_; + packet.xr_frame.should_render = last_should_render_; + for (uint32_t eye = 0; eye < kOpenXREyeCount; ++eye) { + packet.render_width[eye] = eye_swapchains_[eye].width; + packet.render_height[eye] = eye_swapchains_[eye].height; + } + if (!packet.xr_frame.should_render) { + return OpenXRBeginStatus::Ready; + } + if (!runtime_->LocateViewsAt(packet.xr_frame.predicted_display_time, packet.xr_frame)) { + Fail("xrLocateViews failed for a packet"); + return OpenXRBeginStatus::Error; + } + if (!packet.xr_frame.views_valid) { + return OpenXRBeginStatus::Ready; + } + + const uint32_t target_count = + presentation.mode == OpenXRFrameMode::VirtualScreen ? 1u : kOpenXREyeCount; + if (target_count == 1) { + packet.render_width[1] = packet.render_width[0]; + packet.render_height[1] = packet.render_height[0]; + } + std::array targets{}; + const uint32_t slot = next_slot_; + { + std::lock_guard lock(vk_mutex_); + DiscardPendingReleasesLocked(); + for (uint32_t eye = 0; eye < target_count; ++eye) { + EyeSlot& eye_slot = slots_[eye][slot]; + targets[eye] = { + eye_slot.buffer, + eye_slot.width, + eye_slot.height, + static_cast(aurora_format_), + DupFd(eye_slot.pending_acquire_fd), + static_cast(eye_slot.layout), + }; + } + // The compositor images are acquired by BeginFrameForPacket, once the eyes exist. + pending_copy_ = {packet.xr_frame.serial, slot, target_count, {}}; + deferred_copy_ = true; + } + { + std::lock_guard lock(submission_mutex_); + awaiting_token_ = packet.xr_frame.serial; + submitted_token_ = 0; + submission_arrived_ = false; + submission_success_ = false; + submission_unsafe_ = false; + } + if (!aurora_vulkan_set_stereo_targets(packet.xr_frame.serial, targets.data(), target_count)) { + for (uint32_t eye = 0; eye < target_count; ++eye) { + CloseFd(targets[eye].acquireFenceFd); + } + { + std::lock_guard lock(submission_mutex_); + awaiting_token_ = 0; + } + { + std::lock_guard lock(vk_mutex_); + deferred_copy_ = false; + } + Fail("Aurora rejected the AHardwareBuffer stereo targets"); + return OpenXRBeginStatus::Error; + } + next_slot_ = (slot + 1) % kSlotCount; + packet.expects_gpu_submission = true; + return OpenXRBeginStatus::Ready; + } + + bool TryCancelPendingPacket(OpenXRBackendFrame& packet) { + if (!packet.expects_gpu_submission || !aurora_vulkan_cancel_stereo_targets(packet.xr_frame.serial)) { + return false; + } + { + std::lock_guard lock(submission_mutex_); + awaiting_token_ = 0; + submitted_token_ = 0; + submission_arrived_ = false; + submission_success_ = false; + submission_unsafe_ = false; + } + { + std::lock_guard lock(vk_mutex_); + DiscardPendingReleasesLocked(); + deferred_copy_ = false; + } + packet.expects_gpu_submission = false; + return true; + } + bool FinishFrame(OpenXRBackendFrame& frame, bool submit_layer) { if (!frame_active_ || runtime_ == nullptr || frame.xr_frame.serial != active_frame_serial_) { @@ -531,6 +664,12 @@ public: active_frame_serial_ = 0; active_frame_ = {}; frame.expects_gpu_submission = false; + { + // Eyes rendered for a packet that this frame did not copy are dropped with it. + std::lock_guard lock(vk_mutex_); + DiscardPendingReleasesLocked(); + deferred_copy_ = false; + } { std::lock_guard lock(submission_mutex_); awaiting_token_ = 0; @@ -541,6 +680,137 @@ public: return release_ok && end_ok; } + OpenXRBeginStatus BeginFrameForPacket(const OpenXRBackendFrame& packet, OpenXRBackendFrame& frame) { + frame = {}; + frame.presentation = packet.presentation; + frame.render_width = packet.render_width; + frame.render_height = packet.render_height; + if (!bound_ || runtime_ == nullptr) { + Fail("BeginFrameForPacket called before the Vulkan backend was bound"); + return OpenXRBeginStatus::Error; + } + if (frame_active_) { + Fail("BeginFrameForPacket called while another OpenXR frame is active"); + return OpenXRBeginStatus::Error; + } + const OpenXRFrameStatus status = runtime_->WaitFrame(frame.xr_frame); + if (status != OpenXRFrameStatus::Ready) { + if (status == OpenXRFrameStatus::Error) { + Fail(BeginStatusOperation(status)); + } + return status == OpenXRFrameStatus::SessionNotRunning ? OpenXRBeginStatus::SessionNotRunning + : status == OpenXRFrameStatus::ExitRequested ? OpenXRBeginStatus::ExitRequested + : OpenXRBeginStatus::Error; + } + NoteDisplayTiming(frame.xr_frame); + if (!runtime_->BeginFrame(frame.xr_frame)) { + Fail("xrBeginFrame failed"); + return OpenXRBeginStatus::Error; + } + frame_active_ = true; + active_frame_serial_ = frame.xr_frame.serial; + render_session_serial_ = runtime_->SessionRunSerial(); + render_space_serial_ = runtime_->LastReferenceSpaceChange().serial; + // The layer shows the eyes as they were rendered: it carries the packet's located views. + frame.xr_frame.views = packet.xr_frame.views; + frame.xr_frame.view_state_flags = packet.xr_frame.view_state_flags; + frame.xr_frame.views_valid = packet.xr_frame.views_valid; + active_frame_ = frame.xr_frame; + frame.expects_gpu_submission = packet.expects_gpu_submission; + if (!frame.xr_frame.should_render || !frame.expects_gpu_submission || !frame.xr_frame.views_valid) { + // No swapchain image is acquired for this frame, so the rendered eyes cannot be + // copied; FinishFrame drops them with the frame. + frame.expects_gpu_submission = false; + return OpenXRBeginStatus::Ready; + } + + const uint32_t target_count = + presentation_target_count(frame.presentation); + const diagnostics::Stopwatch acquire_timer; + for (uint32_t eye = 0; eye < target_count; ++eye) { + if (!AcquireSwapchain(eye_swapchains_[eye])) { + ReleaseAcquiredSwapchains(); + EndActiveFrameWithoutLayers(frame.xr_frame); + return OpenXRBeginStatus::Error; + } + } + diagnostics::OnSwapchainAcquire(acquire_timer); + { + std::lock_guard lock(vk_mutex_); + for (uint32_t eye = 0; eye < target_count; ++eye) { + pending_copy_.swapchain_images[eye] = + eye_swapchains_[eye].images[eye_swapchains_[eye].acquired_index].image; + } + } + return OpenXRBeginStatus::Ready; + } + + OpenXRSubmissionStatus CopyRenderedEyes(const OpenXRBackendFrame& frame) { + CopyOutcome outcome = CopyOutcome::Skipped; + { + std::lock_guard lock(vk_mutex_); + if (!frame_active_ || !have_pending_releases_ || !deferred_copy_) { + return OpenXRSubmissionStatus::Skipped; + } + outcome = RecordAndSubmitCopyLocked(pending_releases_); + have_pending_releases_ = false; + deferred_copy_ = false; + } + { + // FinishFrame judges the queue's safety by this frame's token. + std::lock_guard lock(submission_mutex_); + submitted_token_ = frame.xr_frame.serial; + submission_arrived_ = true; + submission_success_ = outcome == CopyOutcome::Submitted; + submission_unsafe_ = outcome == CopyOutcome::Unsafe; + } + return outcome == CopyOutcome::Submitted ? OpenXRSubmissionStatus::Success + : outcome == CopyOutcome::Unsafe ? OpenXRSubmissionStatus::Failed + : OpenXRSubmissionStatus::Skipped; + } + + OpenXRBeginStatus KeepAliveCycle() { + if (!bound_ || runtime_ == nullptr || frame_active_) { + Fail("KeepAliveCycle called with a frame active or before binding"); + return OpenXRBeginStatus::Error; + } + OpenXRFrame cycle{}; + const OpenXRFrameStatus status = runtime_->WaitFrame(cycle); + if (status != OpenXRFrameStatus::Ready) { + if (status == OpenXRFrameStatus::Error) { + Fail(BeginStatusOperation(status)); + } + return status == OpenXRFrameStatus::SessionNotRunning ? OpenXRBeginStatus::SessionNotRunning + : status == OpenXRFrameStatus::ExitRequested ? OpenXRBeginStatus::ExitRequested + : OpenXRBeginStatus::Error; + } + NoteDisplayTiming(cycle); + if (!runtime_->BeginFrame(cycle)) { + Fail("xrBeginFrame failed for a keep-alive cycle"); + return OpenXRBeginStatus::Error; + } + active_frame_ = cycle; + frame_active_ = true; + const bool end_ok = EndRetainedFrame(false); + frame_active_ = false; + active_frame_ = {}; + if (!end_ok) { + Fail("OpenXR could not resubmit the retained frame"); + return OpenXRBeginStatus::Error; + } + return OpenXRBeginStatus::Ready; + } + + void NoteDisplayTiming(const OpenXRFrame& frame) noexcept { + last_display_time_ = frame.predicted_display_time; + last_display_period_ = frame.predicted_display_period; + last_should_render_ = frame.should_render; + } + + static uint32_t presentation_target_count(const OpenXRPresentation& presentation) noexcept { + return presentation.mode == OpenXRFrameMode::VirtualScreen ? 1u : kOpenXREyeCount; + } + bool RepeatFrame(const OpenXRBackendFrame& frame) { if (!frame_active_ || runtime_ == nullptr || frame.xr_frame.serial != active_frame_serial_) { @@ -1082,6 +1352,7 @@ private: void DestroySlots() { std::lock_guard lock(vk_mutex_); + DiscardPendingReleasesLocked(); if (vk_device_ != VK_NULL_HANDLE) { vkDeviceWaitIdle(vk_device_); } @@ -1136,7 +1407,16 @@ private: expected = token == self->awaiting_token_ && token == self->pending_copy_.token; } if (expected && success && release_count >= self->pending_copy_.target_count) { - outcome = self->RecordAndSubmitCopyLocked(owned); + if (self->deferred_copy_) { + // No compositor frame is open yet: keep Dawn's release fences for + // CopyRenderedEyes, which records the copy once the frame is begun. + self->DiscardPendingReleasesLocked(); + self->pending_releases_ = owned; + self->have_pending_releases_ = true; + outcome = CopyOutcome::Submitted; + } else { + outcome = self->RecordAndSubmitCopyLocked(owned); + } } else { for (auto& release : owned) { CloseFd(release.releaseFenceFd); @@ -1594,6 +1874,15 @@ private: uint32_t next_submission_ = 0; uint32_t next_slot_ = 0; PendingCopy pending_copy_{}; + // Render-first pacing (PreparePacket): Aurora's release fences arrive while no compositor + // frame is active, so the copy is recorded later by CopyRenderedEyes. + bool deferred_copy_ = false; + std::array pending_releases_{}; + bool have_pending_releases_ = false; + uint64_t next_packet_serial_ = 1ull << 40; // never collides with the runtime's frame serials + XrTime last_display_time_ = 0; + XrDuration last_display_period_ = 0; + bool last_should_render_ = false; std::mutex submission_mutex_; std::condition_variable submission_cv_; @@ -1638,6 +1927,21 @@ OpenXRSubmissionStatus OpenXRVulkanBackend::WaitForSubmission(const OpenXRBacken uint32_t timeout_ms) { return m_impl->WaitForSubmission(frame, timeout_ms); } +OpenXRBeginStatus OpenXRVulkanBackend::PreparePacket(const OpenXRPresentation& presentation, + OpenXRBackendFrame& packet) { + return m_impl->PreparePacket(presentation, packet); +} +bool OpenXRVulkanBackend::TryCancelPendingPacket(OpenXRBackendFrame& packet) { + return m_impl->TryCancelPendingPacket(packet); +} +OpenXRBeginStatus OpenXRVulkanBackend::BeginFrameForPacket(const OpenXRBackendFrame& packet, + OpenXRBackendFrame& frame) { + return m_impl->BeginFrameForPacket(packet, frame); +} +OpenXRSubmissionStatus OpenXRVulkanBackend::CopyRenderedEyes(const OpenXRBackendFrame& frame) { + return m_impl->CopyRenderedEyes(frame); +} +OpenXRBeginStatus OpenXRVulkanBackend::KeepAliveCycle() { return m_impl->KeepAliveCycle(); } bool OpenXRVulkanBackend::TryCancelPendingFrame(OpenXRBackendFrame& frame) { return m_impl->TryCancelPendingFrame(frame);