diff --git a/OPENXR.md b/OPENXR.md index c5ccf87..16c6a9a 100644 --- a/OPENXR.md +++ b/OPENXR.md @@ -258,16 +258,42 @@ of you. How it is drawn: `settings_overlay.cpp` builds the panel with a second Dear ImGui context of its own, a 1440 × 1080 canvas at twice the desktop menu's scale with its own font atlas, fed by the pointer that `openxr_input.cpp` publishes through `vr/openxr_settings_panel.h`. Aurora renders that draw data -into a panel texture once per sealed frame and lays it over each eye after the eye is finished -(`aurora-main/lib/stereo_overlay.cpp`): through the eye's frustum and `viewFromCenter` onto the -screen rectangle for an immersive eye (including headset-rate interpolated eyes, which reuse the -texture), and as a centred rectangle on a virtual-screen eye image. The eye images the OpenXR -backends already submit carry it, so no extra swapchain or composition layer is involved. The -ImGui backend keeps a single projection uniform, so the panel's pass is submitted on its own command -buffer before the desktop's ImGui pass of the same frame is recorded. +into a panel texture once per sealed frame (`aurora-main/lib/stereo_overlay.cpp`). The ImGui backend +keeps a single projection uniform, so the panel's pass is submitted on its own command buffer before +the desktop's ImGui pass of the same frame is recorded. + +The panel is shown as a compositor quad layer of its own, submitted over the scene's projection or +menu quad layer. The compositor samples the 1440 × 1080 canvas directly, so its text stays sharp +whatever `render_scale` gives the eyes. Every backend (D3D12, Windows Vulkan, Quest) makes the +panel's swapchain pair the first time the panel opens (two 1440 × 1080 swapchains, plus two shared +buffers on the Quest) and keeps it for the session. Until then nothing is allocated, and while the +panel is closed nothing is copied or submitted. While it is open, each frame hands Aurora one more +target after the eyes: the stereo bridge copies the panel texture into it with the eyes (or a +transparent image on a frame where the panel is not drawn). The layer follows the eyes' swapchain +pairing: the image a frame wrote is shown only once that frame is submitted, so a cancelled frame +never shows an unwritten panel. The quad hangs exactly where the pointer's hits are tested +(`SettingsPanelScreen` in `openxr_integration.cpp`). ImGui's premultiplied output is blended with +`XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT`, and Aurora leaves the panel out of the eyes +(`aurora_set_stereo_panel_layer`). + +If a backend cannot make the panel's swapchains, it logs that once and the panel is drawn into the +eye images instead: through the eye's frustum and `viewFromCenter` onto the screen rectangle for an +immersive eye (including headset-rate interpolated eyes, which reuse the texture), and as a centred +rectangle on a virtual-screen eye image. On the Quest, `adb shell setprop +debug.wiicompiled.panel_layer 0` switches to that path at run time, to compare the two. + +Measured on a Quest 3 (base game, a Grand Prix start with the player idle, `render_scale = 0.8`, +60 FPS, eight interleaved rounds per state), the layer costs nothing while the panel is closed. While +it is open, the app's GPU time is 10.5 ms per frame with the layer, against 9.7 ms drawn into the eyes +(9.4 ms closed). GPU load is 74% against 67%, and the compositor's time 1.05 ms against 0.75 ms. Game +and headset frame rates did not change. The compositor redraws the layer at display rate, so the +panel stays steady even when the game drops frames. `mkw_vr_settings_panel_tests` covers the panel button in both controller modes, the release latch, -selection, scrolling and the canvas mapping; `gx_fifo_tests` covers where the panel lands in each eye. +selection, scrolling and the canvas mapping; `gx_fifo_tests` covers where the panel lands in each eye +on the fallback path. `mkw_openxr_replay_tests` and `mkw_openxr_vulkan_replay_tests` cover the layer: +nothing made before the panel opens, the panel image of a cancelled frame never shown, no layer while +the panel is closed or has no place yet, and render-first pacing. ## The first-person camera @@ -731,9 +757,8 @@ ends, including mid-frame flushes, so live setting changes cannot invalidate pen - The cockpit seat, the turning wheel and hand steering have not yet been validated in a headset on this build: the native wheel's match against the race camera's view, bikes and Quacker, and the Quest. Hand steering needs analog grips (Touch); the simple controller profile cannot grab. -- The headset settings panel is drawn into the eye images rather than submitted as its own quad - layer, so its text is resampled once more than a compositor layer's would be. It has no laser - beam, only the cursor on the panel itself, and text fields cannot be typed into without a keyboard. +- The headset settings panel has no laser beam, only the cursor on the panel itself, and text fields + cannot be typed into without a keyboard. - The desktop window remains available as a mirror/fallback. OpenXR diagnostics are written to the normal run log under diff --git a/aurora-main/include/aurora/aurora.h b/aurora-main/include/aurora/aurora.h index 5ac54a7..a4f19d5 100644 --- a/aurora-main/include/aurora/aurora.h +++ b/aurora-main/include/aurora/aurora.h @@ -150,6 +150,14 @@ typedef struct { float weights[4]; } AuroraVRHandVertex; +/** + * Shows the headset settings panel (aurora_imgui_set_stereo_overlay) as the + * OpenXR backend's own compositor quad layer instead of drawing it into the + * eyes, so the eye resolution no longer limits its text. The backend then asks + * for the panel image as an extra stereo target. Any thread. + */ +void aurora_set_stereo_panel_layer(bool enabled); + // Copies optional runtime-provided hand meshes (XR_FB_hand_tracking_mesh, 26 // joints). Null clears to the procedural glove. Bind poses: x,y,z,w,px,py,pz. void aurora_set_vr_hand_mesh(uint32_t hand, const AuroraVRHandVertex* vertices, uint32_t vertexCount, diff --git a/aurora-main/include/aurora/d3d12_interop.h b/aurora-main/include/aurora/d3d12_interop.h index 6aff39a..455f6ae 100644 --- a/aurora-main/include/aurora/d3d12_interop.h +++ b/aurora-main/include/aurora/d3d12_interop.h @@ -64,6 +64,17 @@ bool aurora_d3d12_set_stereo_targets(uint64_t frameToken, const AuroraD3D12StereoTarget* targets, uint32_t targetCount); +/** + * The same, plus the headset settings panel's quad-layer image when `panel` is + * not null (aurora_set_stereo_panel_layer): Aurora copies the panel into it, or + * a transparent image while the panel is not showing, with the eyes and under + * the same completion callback. + */ +bool aurora_d3d12_set_stereo_targets_with_panel(uint64_t frameToken, + const AuroraD3D12StereoTarget* targets, + uint32_t targetCount, + const AuroraD3D12StereoTarget* panel); + /** * Withdraws frameToken only while its target has not been encoded. This is * safe to race with Aurora's frame worker: false means the worker already owns diff --git a/aurora-main/include/aurora/vulkan_interop.h b/aurora-main/include/aurora/vulkan_interop.h index 59a7c30..f902b14 100644 --- a/aurora-main/include/aurora/vulkan_interop.h +++ b/aurora-main/include/aurora/vulkan_interop.h @@ -27,6 +27,8 @@ extern "C" { */ enum { AURORA_VULKAN_STEREO_MAX_TARGETS = 2 }; +// The eyes, then the settings panel's quad-layer image when one was given. +enum { AURORA_VULKAN_STEREO_MAX_RELEASES = AURORA_VULKAN_STEREO_MAX_TARGETS + 1 }; /** * Borrowed facts about Aurora's Dawn Vulkan device. colorVkFormat is the @@ -109,6 +111,18 @@ bool aurora_vulkan_set_stereo_targets(uint64_t frameToken, const AuroraVulkanStereoTarget* targets, uint32_t targetCount); +/** + * The same, plus the headset settings panel's quad-layer buffer when `panel` is + * not null (aurora_set_stereo_panel_layer): Aurora copies the panel into it, or + * a transparent image while the panel is not showing, with the eyes. Its + * release entry follows the eyes' in the submitted callback, whose + * releaseCount then counts it too. + */ +bool aurora_vulkan_set_stereo_targets_with_panel(uint64_t frameToken, + const AuroraVulkanStereoTarget* targets, + uint32_t targetCount, + const AuroraVulkanStereoTarget* panel); + /** * Withdraws frameToken only while its targets have not been encoded. Semantics * match aurora_d3d12_cancel_stereo_targets: false means the worker already diff --git a/aurora-main/include/aurora/vulkan_win32_interop.h b/aurora-main/include/aurora/vulkan_win32_interop.h index 82db145..fbecb5c 100644 --- a/aurora-main/include/aurora/vulkan_win32_interop.h +++ b/aurora-main/include/aurora/vulkan_win32_interop.h @@ -11,6 +11,10 @@ bool aurora_vulkan_win32_configure(const AuroraDawnVulkanHooks* hooks); bool aurora_vulkan_win32_get_handles(AuroraDawnVulkanHandles* handles, int64_t* colorFormat); bool aurora_vulkan_win32_enable(AuroraD3D12StereoSubmittedCallback submitted, void* userdata); bool aurora_vulkan_win32_set_targets(uint64_t token, const AuroraD3D12StereoTarget* targets, uint32_t count); +// Plus the headset settings panel's quad-layer image when panel is not null, as +// aurora_d3d12_set_stereo_targets_with_panel. +bool aurora_vulkan_win32_set_targets_with_panel(uint64_t token, const AuroraD3D12StereoTarget* targets, uint32_t count, + const AuroraD3D12StereoTarget* panel); bool aurora_vulkan_win32_cancel(uint64_t token); bool aurora_vulkan_win32_disable(); void* aurora_vulkan_win32_lock_queue(); diff --git a/aurora-main/lib/imgui.cpp b/aurora-main/lib/imgui.cpp index 5714d28..f3854d4 100644 --- a/aurora-main/lib/imgui.cpp +++ b/aurora-main/lib/imgui.cpp @@ -13,6 +13,7 @@ #include "fs_helper.hpp" #include "internal.hpp" +#include "stereo_overlay.hpp" #include "webgpu/gpu.hpp" #include "window.hpp" @@ -348,6 +349,8 @@ ImTextureID aurora_imgui_add_texture(uint32_t width, uint32_t height, const void return aurora::imgui::add_texture(width, height, static_cast(rgba8)); } +void aurora_set_stereo_panel_layer(bool enabled) { aurora::stereo_overlay::set_layer_mode(enabled); } + void aurora_imgui_set_stereo_overlay(ImDrawData* drawData, float widthFraction) { std::lock_guard lock(aurora::imgui::g_stereoOverlayMutex); aurora::imgui::g_stereoOverlay = { diff --git a/aurora-main/lib/stereo_overlay.cpp b/aurora-main/lib/stereo_overlay.cpp index 4395d69..99dc794 100644 --- a/aurora-main/lib/stereo_overlay.cpp +++ b/aurora-main/lib/stereo_overlay.cpp @@ -11,6 +11,7 @@ #include "tracy/Tracy.hpp" #include +#include #include namespace aurora::stereo_overlay { @@ -74,8 +75,12 @@ struct State { std::array bindGroups; float widthFraction = 0.f; bool visible = false; + // Stands in for the panel in a layer while it is not showing. + webgpu::TextureWithSampler transparent; + bool transparentCleared = false; }; State g_state; +std::atomic_bool g_layerMode{false}; bool ensure_pipeline() { auto& state = g_state; @@ -314,7 +319,7 @@ wgpu::CommandBuffer prepare(ImDrawData* drawData, float widthFraction) noexcept void composite_immersive(const wgpu::CommandEncoder& encoder, const wgpu::TextureView& eye, const Mat4x4& eyeFrustum, const Mat3x4& viewFromCenter, uint32_t eyeIndex) noexcept { - if (!g_state.visible) { + if (!g_state.visible || layer_mode()) { return; } float screenWidth = 0.f; @@ -331,7 +336,7 @@ void composite_immersive(const wgpu::CommandEncoder& encoder, const wgpu::Textur void composite_flat(const wgpu::CommandEncoder& encoder, const wgpu::TextureView& eye, const wgpu::Extent3D& size, uint32_t eyeIndex) noexcept { - if (!g_state.visible || size.width == 0 || size.height == 0) { + if (!g_state.visible || layer_mode() || size.width == 0 || size.height == 0) { return; } const float imageAspect = static_cast(size.width) / static_cast(size.height); @@ -340,6 +345,52 @@ void composite_flat(const wgpu::CommandEncoder& encoder, const wgpu::TextureView eyeIndex); } +void set_layer_mode(bool enabled) noexcept { g_layerMode.store(enabled, std::memory_order_release); } + +bool layer_mode() noexcept { return g_layerMode.load(std::memory_order_acquire); } + +bool layer_source(const wgpu::CommandEncoder& encoder, uint32_t width, uint32_t height, stereo::EyeImage& out) noexcept { + auto& state = g_state; + const auto format = webgpu::g_graphicsConfig.surfaceConfiguration.format; + if (width == 0 || height == 0) { + return false; + } + if (state.visible && state.panel.texture && state.panel.size.width == width && state.panel.size.height == height && + state.panel.format == format) { + out = {.texture = &state.panel.texture, .view = &state.panel.view, .size = state.panel.size, .format = format}; + return true; + } + if (!state.transparent.texture || state.transparent.size.width != width || + state.transparent.size.height != height || state.transparent.format != format) { + state.transparent = webgpu::create_render_texture(width, height, false); + state.transparentCleared = false; + if (state.transparent.size.width != width || state.transparent.size.height != height) { + state.transparent = {}; + return false; + } + } + if (!state.transparentCleared) { + const std::array attachments{ + wgpu::RenderPassColorAttachment{ + .view = state.transparent.view, + .loadOp = wgpu::LoadOp::Clear, + .storeOp = wgpu::StoreOp::Store, + .clearValue = {.r = 0.0, .g = 0.0, .b = 0.0, .a = 0.0}, + }, + }; + const wgpu::RenderPassDescriptor descriptor{ + .label = "Headset panel layer clear", + .colorAttachmentCount = attachments.size(), + .colorAttachments = attachments.data(), + }; + encoder.BeginRenderPass(&descriptor).End(); + state.transparentCleared = true; + } + out = {.texture = &state.transparent.texture, .view = &state.transparent.view, .size = state.transparent.size, + .format = format}; + return true; +} + void shutdown() noexcept { g_state = {}; } } // namespace aurora::stereo_overlay diff --git a/aurora-main/lib/stereo_overlay.hpp b/aurora-main/lib/stereo_overlay.hpp index b21eb77..0c10129 100644 --- a/aurora-main/lib/stereo_overlay.hpp +++ b/aurora-main/lib/stereo_overlay.hpp @@ -3,6 +3,8 @@ #include #include +#include "stereo.hpp" + #include struct ImDrawData; @@ -28,6 +30,20 @@ void composite_immersive(const wgpu::CommandEncoder& encoder, const wgpu::Textur void composite_flat(const wgpu::CommandEncoder& encoder, const wgpu::TextureView& eye, const wgpu::Extent3D& size, uint32_t eyeIndex) noexcept; +// Layer mode: the OpenXR backend shows the panel as its own compositor quad +// layer, sharp at any eye resolution, so it is no longer drawn into the eyes +// (both composite functions do nothing). Any thread; read by the frame worker. +void set_layer_mode(bool enabled) noexcept; +bool layer_mode() noexcept; + +// Frame worker, inside a stereo sink: the image to copy into a panel layer of +// width x height in the eyes' format. That is the panel while it is showing at +// exactly that size, and otherwise a transparent image of that size, so a layer +// asked for before the panel's first frame (or after it closed) shows nothing. +// A transparent image is cleared once, by a pass recorded into `encoder`. +// False only when no image can be made. +bool layer_source(const wgpu::CommandEncoder& encoder, uint32_t width, uint32_t height, stereo::EyeImage& out) noexcept; + void shutdown() noexcept; } // namespace aurora::stereo_overlay diff --git a/aurora-main/lib/webgpu/d3d12_interop.cpp b/aurora-main/lib/webgpu/d3d12_interop.cpp index 9d38040..d401c7b 100644 --- a/aurora-main/lib/webgpu/d3d12_interop.cpp +++ b/aurora-main/lib/webgpu/d3d12_interop.cpp @@ -2,6 +2,7 @@ #include "../internal.hpp" #include "../stereo.hpp" +#include "../stereo_overlay.hpp" #include "gpu.hpp" #if defined(_WIN32) && defined(WEBGPU_DAWN) && defined(DAWN_ENABLE_BACKEND_D3D12) @@ -126,6 +127,11 @@ struct SharedFenceDxgiHandleWire { void* handle = nullptr; }; +// The eyes, then the settings panel's layer image in a slot of its own so the +// eye intermediates are never resized for it. +constexpr uint32_t kPanelIndex = AURORA_D3D12_STEREO_MAX_TARGETS; +constexpr uint32_t kMaxImages = AURORA_D3D12_STEREO_MAX_TARGETS + 1; + struct IntermediateEye { ComPtr resource; wgpu::SharedTextureMemory memory; @@ -153,8 +159,8 @@ struct InFlightCommand { // both sides of every copy alive until this submission's fence completes; // an eye-size change may otherwise replace the bridge intermediate while // the GPU is still reading it. - std::array, AURORA_D3D12_STEREO_MAX_TARGETS> sources; - std::array, AURORA_D3D12_STEREO_MAX_TARGETS> destinations; + std::array, kMaxImages> sources; + std::array, kMaxImages> destinations; }; class StereoBridge final { @@ -209,38 +215,51 @@ public: return WaitForGpuLocked(); } - bool SetTargets(uint64_t token, const AuroraD3D12StereoTarget* targets, - uint32_t targetCount) noexcept { + bool SetTargets(uint64_t token, const AuroraD3D12StereoTarget* targets, uint32_t targetCount, + const AuroraD3D12StereoTarget* panel) noexcept { if (token == 0 || targets == nullptr || targetCount == 0 || targetCount > AURORA_D3D12_STEREO_MAX_TARGETS) { return false; } + const auto valid = [](const AuroraD3D12StereoTarget& target) { + if (target.resource == nullptr || target.width == 0 || target.height == 0 || + target.dxgiFormat == DXGI_FORMAT_UNKNOWN) { + return false; + } + const D3D12_RESOURCE_DESC desc = static_cast(target.resource)->GetDesc(); + return desc.Dimension == D3D12_RESOURCE_DIMENSION_TEXTURE2D && desc.Width >= target.width && + desc.Height >= target.height && desc.DepthOrArraySize == 1 && desc.MipLevels == 1 && + desc.SampleDesc.Count == 1 && + same_copy_family(desc.Format, static_cast(target.dxgiFormat)); + }; std::lock_guard lock(m_mutex); if (m_framePending || m_encoded) { return false; } for (uint32_t eye = 0; eye < targetCount; ++eye) { - if (targets[eye].resource == nullptr || targets[eye].width == 0 || - targets[eye].height == 0 || targets[eye].dxgiFormat == DXGI_FORMAT_UNKNOWN) { + if (!valid(targets[eye])) { return false; } - auto* resource = static_cast(targets[eye].resource); - const D3D12_RESOURCE_DESC desc = resource->GetDesc(); - if (desc.Dimension != D3D12_RESOURCE_DIMENSION_TEXTURE2D || - desc.Width < targets[eye].width || desc.Height < targets[eye].height || - desc.DepthOrArraySize != 1 || desc.MipLevels != 1 || desc.SampleDesc.Count != 1 || - !same_copy_family(desc.Format, static_cast(targets[eye].dxgiFormat))) { - return false; - } - m_targets[eye] = { - .resource = resource, - .width = targets[eye].width, - .height = targets[eye].height, - .format = static_cast(targets[eye].dxgiFormat), - }; } - for (uint32_t eye = targetCount; eye < m_targets.size(); ++eye) { - m_targets[eye] = {}; + if (panel != nullptr && !valid(*panel)) { + return false; + } + m_targets = {}; + m_imageCount = 0; + const auto add = [&](uint32_t index, const AuroraD3D12StereoTarget& target) { + m_targets[index] = { + .resource = static_cast(target.resource), + .width = target.width, + .height = target.height, + .format = static_cast(target.dxgiFormat), + }; + m_images[m_imageCount++] = index; + }; + for (uint32_t eye = 0; eye < targetCount; ++eye) { + add(eye, targets[eye]); + } + if (panel != nullptr) { + add(kPanelIndex, *panel); } m_frameToken = token; m_targetCount = targetCount; @@ -307,7 +326,7 @@ private: if (source.texture == nullptr || sourceFormat == DXGI_FORMAT_UNKNOWN || source.size.width != m_targets[eye].width || source.size.height != m_targets[eye].height || !same_copy_family(sourceFormat, m_targets[eye].format)) { - Log.error("Stereo eye {} does not match its OpenXR D3D12 target", eye); + Log.error("Stereo image {} does not match its OpenXR D3D12 target", eye); return false; } if (intermediate.texture && intermediate.width == source.size.width && @@ -350,7 +369,9 @@ private: wire.resource = intermediate.resource; const wgpu::SharedTextureMemoryDescriptor memoryDescriptor{ .nextInChain = &wire.chain, - .label = eye == 0 ? "OpenXR left eye intermediate" : "OpenXR right eye intermediate", + .label = eye == 0 ? "OpenXR left eye intermediate" + : eye == 1 ? "OpenXR right eye intermediate" + : "OpenXR panel intermediate", }; intermediate.memory = webgpu::g_device.ImportSharedTextureMemory(&memoryDescriptor); if (!intermediate.memory) { @@ -368,7 +389,9 @@ private: return false; } const wgpu::TextureDescriptor textureDescriptor{ - .label = eye == 0 ? "OpenXR left eye shared texture" : "OpenXR right eye shared texture", + .label = eye == 0 ? "OpenXR left eye shared texture" + : eye == 1 ? "OpenXR right eye shared texture" + : "OpenXR panel shared texture", .usage = wgpu::TextureUsage::CopyDst, .dimension = wgpu::TextureDimension::e2D, .size = {source.size.width, source.size.height, 1}, @@ -391,12 +414,22 @@ private: bool EncodeLocked(wgpu::CommandEncoder& encoder, const stereo::SinkFrame& frame) noexcept { CollectCompletedCommandsLocked(); - for (uint32_t eye = 0; eye < m_targetCount; ++eye) { - if (!EnsureIntermediate(eye, frame.eyes[eye])) { + std::array sources{}; + for (uint32_t n = 0; n < m_imageCount; ++n) { + const uint32_t eye = m_images[n]; + if (eye == kPanelIndex) { + if (!stereo_overlay::layer_source(encoder, m_targets[eye].width, m_targets[eye].height, sources[eye])) { + return false; + } + } else { + sources[eye] = frame.eyes[eye]; + } + if (!EnsureIntermediate(eye, sources[eye])) { return false; } } - for (uint32_t eye = 0; eye < m_targetCount; ++eye) { + for (uint32_t n = 0; n < m_imageCount; ++n) { + const uint32_t eye = m_images[n]; auto& intermediate = m_intermediates[eye]; const std::array fences{m_webgpuFence}; const std::array values{m_lastExternalFenceValue}; @@ -410,7 +443,8 @@ private: } if (intermediate.memory.BeginAccess(intermediate.texture, &begin) != wgpu::Status::Success) { Log.error("Dawn BeginAccess failed for stereo eye {}", eye); - for (uint32_t begunEye = 0; begunEye < eye; ++begunEye) { + for (uint32_t m = 0; m < n; ++m) { + const uint32_t begunEye = m_images[m]; wgpu::SharedTextureMemoryEndAccessState end{}; m_intermediates[begunEye].memory.EndAccess(m_intermediates[begunEye].texture, &end); m_intermediates[begunEye].initialized = end.initialized; @@ -424,10 +458,11 @@ private: // to one. If a later BeginAccess fails, the rollback above can therefore // end the earlier accesses without leaving an unsubmitted copy that uses // a texture after its access interval. - for (uint32_t eye = 0; eye < m_targetCount; ++eye) { + for (uint32_t n = 0; n < m_imageCount; ++n) { + const uint32_t eye = m_images[n]; const auto& intermediate = m_intermediates[eye]; const wgpu::TexelCopyTextureInfo source{ - .texture = *frame.eyes[eye].texture, + .texture = *sources[eye].texture, .mipLevel = 0, .origin = {}, .aspect = wgpu::TextureAspect::All, @@ -446,7 +481,8 @@ private: bool EndAccessLocked() noexcept { bool success = true; - for (uint32_t eye = 0; eye < m_targetCount; ++eye) { + for (uint32_t n = 0; n < m_imageCount; ++n) { + const uint32_t eye = m_images[n]; auto& intermediate = m_intermediates[eye]; if (!intermediate.accessBegun) { success = false; @@ -467,8 +503,8 @@ private: bool EnqueueNativeCopyLocked() noexcept { ComPtr allocator; ComPtr list; - std::array, AURORA_D3D12_STEREO_MAX_TARGETS> sources; - std::array, AURORA_D3D12_STEREO_MAX_TARGETS> destinations; + std::array, kMaxImages> sources; + std::array, kMaxImages> destinations; if (FAILED(m_device->CreateCommandAllocator(D3D12_COMMAND_LIST_TYPE_DIRECT, IID_PPV_ARGS(&allocator))) || FAILED(m_device->CreateCommandList(0, D3D12_COMMAND_LIST_TYPE_DIRECT, allocator.Get(), @@ -477,7 +513,8 @@ private: return false; } - for (uint32_t eye = 0; eye < m_targetCount; ++eye) { + for (uint32_t n = 0; n < m_imageCount; ++n) { + const uint32_t eye = m_images[n]; const auto& source = m_intermediates[eye]; const auto& destination = m_targets[eye]; sources[eye] = source.resource; @@ -564,6 +601,7 @@ private: } m_frameToken = 0; m_targetCount = 0; + m_imageCount = 0; m_framePending = false; m_encoded = false; } @@ -625,8 +663,11 @@ private: ComPtr m_queue; ComPtr m_fence; wgpu::SharedFence m_webgpuFence; - std::array m_intermediates{}; - std::array m_targets{}; + std::array m_intermediates{}; + std::array m_targets{}; + // The slots of m_targets this frame copies into, eyes first. + std::array m_images{}; + uint32_t m_imageCount = 0; std::vector m_commands; AuroraD3D12StereoSubmittedCallback m_callback = nullptr; void* m_userdata = nullptr; @@ -701,7 +742,13 @@ bool aurora_d3d12_set_stereo_targets(uint64_t frameToken, const AuroraD3D12StereoTarget* targets, uint32_t targetCount) { using namespace aurora::d3d12_interop; - return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount); + return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount, nullptr); +} + +bool aurora_d3d12_set_stereo_targets_with_panel(uint64_t frameToken, const AuroraD3D12StereoTarget* targets, + uint32_t targetCount, const AuroraD3D12StereoTarget* panel) { + using namespace aurora::d3d12_interop; + return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount, panel); } bool aurora_d3d12_cancel_stereo_targets(uint64_t frameToken) { @@ -744,6 +791,11 @@ bool aurora_d3d12_set_stereo_targets(uint64_t, const AuroraD3D12StereoTarget*, u return false; } +bool aurora_d3d12_set_stereo_targets_with_panel(uint64_t, const AuroraD3D12StereoTarget*, uint32_t, + const AuroraD3D12StereoTarget*) { + return false; +} + bool aurora_d3d12_cancel_stereo_targets(uint64_t) { return false; } bool aurora_d3d12_disable_stereo_bridge() { return true; } diff --git a/aurora-main/lib/webgpu/vulkan_interop.cpp b/aurora-main/lib/webgpu/vulkan_interop.cpp index 0b41da8..2ecd8b7 100644 --- a/aurora-main/lib/webgpu/vulkan_interop.cpp +++ b/aurora-main/lib/webgpu/vulkan_interop.cpp @@ -2,6 +2,7 @@ #include "../internal.hpp" #include "../stereo.hpp" +#include "../stereo_overlay.hpp" #include "gpu.hpp" #if defined(__ANDROID__) && defined(WEBGPU_DAWN) @@ -105,6 +106,11 @@ struct Import { bool accessBegun = false; }; +// The eyes, then the settings panel's layer image in a slot of its own. +constexpr uint32_t kPanelIndex = AURORA_VULKAN_STEREO_MAX_TARGETS; +constexpr uint32_t kMaxImages = AURORA_VULKAN_STEREO_MAX_RELEASES; +using Releases = std::array; + struct PendingTarget { AHardwareBuffer* buffer = nullptr; uint32_t width = 0; @@ -146,8 +152,8 @@ public: return true; } - bool SetTargets(uint64_t token, const AuroraVulkanStereoTarget* targets, - uint32_t targetCount) noexcept { + bool SetTargets(uint64_t token, const AuroraVulkanStereoTarget* targets, uint32_t targetCount, + const AuroraVulkanStereoTarget* panel) noexcept { if (token == 0 || targets == nullptr || targetCount == 0 || targetCount > AURORA_VULKAN_STEREO_MAX_TARGETS) { return false; @@ -157,25 +163,36 @@ public: return false; } const int64_t auroraFormat = to_vk_format(m_auroraFormat); + const auto valid = [&](const AuroraVulkanStereoTarget& target) { + return target.buffer != nullptr && target.width != 0 && target.height != 0 && + same_copy_family(target.vkFormat, auroraFormat); + }; for (uint32_t eye = 0; eye < targetCount; ++eye) { - const auto& target = targets[eye]; - if (target.buffer == nullptr || target.width == 0 || target.height == 0 || - !same_copy_family(target.vkFormat, auroraFormat)) { + if (!valid(targets[eye])) { return false; } } - for (uint32_t eye = 0; eye < targetCount; ++eye) { - m_targets[eye] = { - .buffer = targets[eye].buffer, - .width = targets[eye].width, - .height = targets[eye].height, - .vkFormat = targets[eye].vkFormat, - .acquireFenceFd = targets[eye].acquireFenceFd, - .acquireImageLayout = targets[eye].acquireImageLayout, - }; + if (panel != nullptr && !valid(*panel)) { + return false; } - for (uint32_t eye = targetCount; eye < m_targets.size(); ++eye) { - m_targets[eye] = {}; + m_targets = {}; + m_imageCount = 0; + const auto add = [&](uint32_t index, const AuroraVulkanStereoTarget& target) { + m_targets[index] = { + .buffer = target.buffer, + .width = target.width, + .height = target.height, + .vkFormat = target.vkFormat, + .acquireFenceFd = target.acquireFenceFd, + .acquireImageLayout = target.acquireImageLayout, + }; + m_images[m_imageCount++] = index; + }; + for (uint32_t eye = 0; eye < targetCount; ++eye) { + add(eye, targets[eye]); + } + if (panel != nullptr) { + add(kPanelIndex, *panel); } m_frameToken = token; m_targetCount = targetCount; @@ -202,7 +219,7 @@ public: if (!m_framePending || !m_encoded || frame.frameToken != m_frameToken) { return; } - std::array releases{}; + Releases releases{}; const bool success = EndAccessLocked(releases); NotifyLocked(frame.frameToken, success, true, releases); ClearFrameLocked(); @@ -215,7 +232,7 @@ public: } const uint64_t token = m_frameToken; const bool encoded = m_encoded; - std::array releases{}; + Releases releases{}; if (encoded) { EndAccessLocked(releases); for (auto& release : releases) { @@ -242,7 +259,7 @@ private: const auto& target = m_targets[eye]; if (source.texture == nullptr || source.format != m_auroraFormat || source.size.width != target.width || source.size.height != target.height) { - Log.error("Stereo eye {} does not match its OpenXR Vulkan target ({}x{} vs {}x{})", eye, + Log.error("Stereo image {} does not match its OpenXR Vulkan target ({}x{} vs {}x{})", eye, source.size.width, source.size.height, target.width, target.height); return nullptr; } @@ -264,7 +281,9 @@ private: ahb.handle = target.buffer; const wgpu::SharedTextureMemoryDescriptor memoryDescriptor{ .nextInChain = &ahb, - .label = eye == 0 ? "OpenXR left eye AHardwareBuffer" : "OpenXR right eye AHardwareBuffer", + .label = eye == 0 ? "OpenXR left eye AHardwareBuffer" + : eye == 1 ? "OpenXR right eye AHardwareBuffer" + : "OpenXR panel AHardwareBuffer", }; import.memory = webgpu::g_device.ImportSharedTextureMemory(&memoryDescriptor); if (!import.memory) { @@ -291,7 +310,9 @@ private: return nullptr; } const wgpu::TextureDescriptor textureDescriptor{ - .label = eye == 0 ? "OpenXR left eye shared texture" : "OpenXR right eye shared texture", + .label = eye == 0 ? "OpenXR left eye shared texture" + : eye == 1 ? "OpenXR right eye shared texture" + : "OpenXR panel shared texture", .usage = wgpu::TextureUsage::CopyDst, .dimension = wgpu::TextureDimension::e2D, .size = {target.width, target.height, 1}, @@ -313,19 +334,29 @@ private: } bool EncodeLocked(wgpu::CommandEncoder& encoder, const stereo::SinkFrame& frame) noexcept { - std::array imports{}; - for (uint32_t eye = 0; eye < m_targetCount; ++eye) { - imports[eye] = EnsureImport(eye, frame.eyes[eye]); + std::array sources{}; + std::array imports{}; + for (uint32_t n = 0; n < m_imageCount; ++n) { + const uint32_t eye = m_images[n]; + if (eye == kPanelIndex) { + if (!stereo_overlay::layer_source(encoder, m_targets[eye].width, m_targets[eye].height, sources[eye])) { + return false; + } + } else { + sources[eye] = frame.eyes[eye]; + } + imports[eye] = EnsureImport(eye, sources[eye]); if (imports[eye] == nullptr) { return false; } } - for (uint32_t eye = 0; eye < m_targetCount; ++eye) { + for (uint32_t n = 0; n < m_imageCount; ++n) { + const uint32_t eye = m_images[n]; auto& target = m_targets[eye]; auto& import = *imports[eye]; if (import.accessBegun) { Log.error("AHardwareBuffer for eye {} is still under a previous access", eye); - RollbackAccesses(imports, eye); + RollbackAccesses(imports, n); return false; } // The OpenXR side's release barrier leaves the image in acquireImageLayout; @@ -350,7 +381,7 @@ private: close_fd(target.acquireFenceFd); if (!acquireFence) { Log.error("Dawn could not import the OpenXR copy-out fence for eye {}", eye); - RollbackAccesses(imports, eye); + RollbackAccesses(imports, n); return false; } } @@ -370,15 +401,16 @@ private: } if (import.memory.BeginAccess(import.texture, &begin) != wgpu::Status::Success) { Log.error("Dawn BeginAccess failed for stereo eye {}", eye); - RollbackAccesses(imports, eye); + RollbackAccesses(imports, n); return false; } import.accessBegun = true; } - for (uint32_t eye = 0; eye < m_targetCount; ++eye) { + for (uint32_t n = 0; n < m_imageCount; ++n) { + const uint32_t eye = m_images[n]; const auto& import = *imports[eye]; const wgpu::TexelCopyTextureInfo source{ - .texture = *frame.eyes[eye].texture, + .texture = *sources[eye].texture, .mipLevel = 0, .origin = {}, .aspect = wgpu::TextureAspect::All, @@ -396,9 +428,10 @@ private: return true; } - void RollbackAccesses(const std::array& imports, - uint32_t count) noexcept { - for (uint32_t eye = 0; eye < count; ++eye) { + // Ends the accesses begun for the first `count` images of this frame. + void RollbackAccesses(const std::array& imports, uint32_t count) noexcept { + for (uint32_t n = 0; n < count; ++n) { + const uint32_t eye = m_images[n]; if (imports[eye] != nullptr && imports[eye]->accessBegun) { wgpu::SharedTextureMemoryEndAccessState end{}; imports[eye]->memory.EndAccess(imports[eye]->texture, &end); @@ -411,13 +444,15 @@ private: } } - bool EndAccessLocked( - std::array& releases) noexcept { + // Fills one release per image of this frame, in order: the eyes, then the panel. + bool EndAccessLocked(Releases& releases) noexcept { bool success = true; for (auto& release : releases) { release = {.releaseFenceFd = -1, .releasedImageLayout = VK_IMAGE_LAYOUT_UNDEFINED}; } - for (uint32_t eye = 0; eye < m_targetCount; ++eye) { + for (uint32_t n = 0; n < m_imageCount; ++n) { + const uint32_t eye = m_images[n]; + auto& release = releases[n]; Import* import = m_encodedImports[eye]; if (import == nullptr || !import->accessBegun) { success = false; @@ -431,7 +466,7 @@ private: success = false; } else { import->initialized = end.initialized; - releases[eye].releasedImageLayout = layout.newLayout; + release.releasedImageLayout = layout.newLayout; for (size_t i = 0; i < end.fenceCount; ++i) { wgpu::SharedFenceSyncFDExportInfo syncFd{}; wgpu::SharedFenceExportInfo info{}; @@ -441,16 +476,16 @@ private: // The fence keeps its descriptor; hand the caller an independent one. const int duplicate = ::dup(syncFd.handle); if (duplicate >= 0) { - if (releases[eye].releaseFenceFd >= 0) { + if (release.releaseFenceFd >= 0) { // Dawn normally returns exactly one fence per access. Both must // be honoured and one descriptor cannot express two fences, so // the earlier one is retired on the CPU before handing over the // latest. Log.warn("Dawn returned several release fences for eye {}; merging on the CPU", eye); - wait_sync_fd(releases[eye].releaseFenceFd); - close_fd(releases[eye].releaseFenceFd); + wait_sync_fd(release.releaseFenceFd); + close_fd(release.releaseFenceFd); } - releases[eye].releaseFenceFd = duplicate; + release.releaseFenceFd = duplicate; } } else { Log.error("Dawn returned a non-sync-fd fence for eye {}", eye); @@ -482,12 +517,13 @@ private: m_encodedImports = {}; m_frameToken = 0; m_targetCount = 0; + m_imageCount = 0; m_framePending = false; m_encoded = false; } void PublishAndClearFrameLocked(uint64_t token, bool success, bool gpuWorkQueued) noexcept { - std::array releases{}; + Releases releases{}; for (auto& release : releases) { release = {.releaseFenceFd = -1, .releasedImageLayout = VK_IMAGE_LAYOUT_UNDEFINED}; } @@ -496,10 +532,9 @@ private: } void NotifyLocked(uint64_t token, bool success, bool gpuWorkQueued, - const std::array& - releases) noexcept { + const Releases& releases) noexcept { if (m_callback != nullptr) { - m_callback(token, success, gpuWorkQueued, releases.data(), m_targetCount, m_userdata); + m_callback(token, success, gpuWorkQueued, releases.data(), m_imageCount, m_userdata); } else { for (auto release : releases) { close_fd(release.releaseFenceFd); @@ -509,8 +544,11 @@ private: std::mutex m_mutex; std::unordered_map m_imports; - std::array m_targets{}; - std::array m_encodedImports{}; + std::array m_targets{}; + std::array m_encodedImports{}; + // The slots of m_targets this frame copies into, eyes first. + std::array m_images{}; + uint32_t m_imageCount = 0; wgpu::TextureFormat m_auroraFormat = wgpu::TextureFormat::Undefined; AuroraVulkanStereoSubmittedCallback m_callback = nullptr; void* m_userdata = nullptr; @@ -575,7 +613,13 @@ bool aurora_vulkan_set_stereo_targets(uint64_t frameToken, const AuroraVulkanStereoTarget* targets, uint32_t targetCount) { using namespace aurora::vulkan_interop; - return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount); + return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount, nullptr); +} + +bool aurora_vulkan_set_stereo_targets_with_panel(uint64_t frameToken, const AuroraVulkanStereoTarget* targets, + uint32_t targetCount, const AuroraVulkanStereoTarget* panel) { + using namespace aurora::vulkan_interop; + return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount, panel); } bool aurora_vulkan_cancel_stereo_targets(uint64_t frameToken) { @@ -615,6 +659,11 @@ bool aurora_vulkan_set_stereo_targets(uint64_t, const AuroraVulkanStereoTarget*, return false; } +bool aurora_vulkan_set_stereo_targets_with_panel(uint64_t, const AuroraVulkanStereoTarget*, uint32_t, + const AuroraVulkanStereoTarget*) { + return false; +} + bool aurora_vulkan_cancel_stereo_targets(uint64_t) { return false; } bool aurora_vulkan_disable_stereo_bridge() { return true; } diff --git a/aurora-main/lib/webgpu/vulkan_win32_interop.cpp b/aurora-main/lib/webgpu/vulkan_win32_interop.cpp index 43cdacd..b420199 100644 --- a/aurora-main/lib/webgpu/vulkan_win32_interop.cpp +++ b/aurora-main/lib/webgpu/vulkan_win32_interop.cpp @@ -2,6 +2,7 @@ #include #include "../internal.hpp" #include "../stereo.hpp" +#include "../stereo_overlay.hpp" #include "gpu.hpp" #if defined(_WIN32) && defined(WEBGPU_DAWN) && defined(DAWN_ENABLE_BACKEND_VULKAN) #include @@ -53,27 +54,38 @@ bool CopyCompatible(int64_t a, int64_t b) { ((a == 44 || a == 50) && (b == 44 || b == 50)); } struct Import { uint64_t image; uint32_t width, height; wgpu::TextureFormat format; wgpu::Texture texture; }; +// The eyes, then the settings panel's layer image after them. +constexpr uint32_t kMaxImages = 3; class Bridge { public: std::mutex mutex; std::vector imports; - std::array targets{}; - std::array active{}; + std::array targets{}; + std::array active{}; uint64_t token = 0; uint32_t count = 0; + // Images this frame copies: `count` eyes, plus the panel when `panel` is set. + uint32_t images = 0; + bool panel = false; bool encoded = false; AuroraD3D12StereoSubmittedCallback callback; void* userdata; Bridge(AuroraD3D12StereoSubmittedCallback cb, void* data) : callback(cb), userdata(data) {} - bool Set(uint64_t next, const AuroraD3D12StereoTarget* data, uint32_t n) { + bool Set(uint64_t next, const AuroraD3D12StereoTarget* data, uint32_t n, const AuroraD3D12StereoTarget* panelTarget) { if (!next || !data || !n || n > 2) return false; std::lock_guard guard(mutex); if (token) return false; + const auto valid = [](const AuroraD3D12StereoTarget& target) { + return target.resource && target.width && target.height; + }; for (uint32_t i = 0; i < n; ++i) { - if (!data[i].resource || !data[i].width || !data[i].height) return false; - targets[i] = data[i]; + if (!valid(data[i])) return false; } - token = next; count = n; return true; + if (panelTarget && !valid(*panelTarget)) return false; + for (uint32_t i = 0; i < n; ++i) targets[i] = data[i]; + panel = panelTarget != nullptr; + if (panel) targets[n] = *panelTarget; + token = next; count = n; images = n + (panel ? 1u : 0u); return true; } bool Cancel(uint64_t wanted) { std::unique_lock guard(mutex, std::try_to_lock); @@ -83,9 +95,13 @@ public: bool Encode(wgpu::CommandEncoder& encoder, const stereo::SinkFrame& frame) { std::lock_guard guard(mutex); if (!token || encoded || frame.frameToken != token) return false; + std::array sources{}; + for (uint32_t i = 0; i < count; ++i) sources[i] = frame.eyes[i]; + if (panel && !stereo_overlay::layer_source(encoder, targets[count].width, targets[count].height, sources[count])) + return false; // Validate/import every target before recording any copy. - for (uint32_t i = 0; i < count; ++i) { - const auto& eye = frame.eyes[i]; + for (uint32_t i = 0; i < images; ++i) { + const auto& eye = sources[i]; const auto& target = targets[i]; if (!eye.texture || eye.size.width != target.width || eye.size.height != target.height || !CopyCompatible(VkFormat(eye.format), target.dxgiFormat)) return false; @@ -112,9 +128,9 @@ public: } active[i] = it->texture; } - for (uint32_t i = 0; i < count; ++i) { + for (uint32_t i = 0; i < images; ++i) { wgpu::TexelCopyTextureInfo source, destination; - source.texture = *frame.eyes[i].texture; + source.texture = *sources[i].texture; destination.texture = active[i]; wgpu::Extent3D size{targets[i].width, targets[i].height, 1}; encoder.CopyTextureToTexture(&source, &destination, &size); @@ -125,11 +141,11 @@ public: void Submitted(const stereo::SinkFrame& frame) { std::lock_guard guard(mutex); if (!token || !encoded || token != frame.frameToken) return; - std::array textures{}; - for (uint32_t i = 0; i < count; ++i) textures[i] = active[i].Get(); + std::array textures{}; + for (uint32_t i = 0; i < images; ++i) textures[i] = active[i].Get(); // Append the COLOR_ATTACHMENT_OPTIMAL release barriers to Dawn's queue, // flush them under its device guard, then allow the XR thread to release. - const bool success = api.release(webgpu::g_device.Get(), textures.data(), count) != 0; + const bool success = api.release(webgpu::g_device.Get(), textures.data(), images) != 0; const auto completed = token; token = 0; encoded = false; callback(completed, success, userdata); @@ -160,7 +176,12 @@ bool aurora_vulkan_win32_enable(AuroraD3D12StereoSubmittedCallback cb, void* dat } bool aurora_vulkan_win32_set_targets(uint64_t token, const AuroraD3D12StereoTarget* targets, uint32_t count) { using namespace aurora::vulkan_win32; - return bridge && bridge->Set(token, targets, count); + return bridge && bridge->Set(token, targets, count, nullptr); +} +bool aurora_vulkan_win32_set_targets_with_panel(uint64_t token, const AuroraD3D12StereoTarget* targets, uint32_t count, + const AuroraD3D12StereoTarget* panel) { + using namespace aurora::vulkan_win32; + return bridge && bridge->Set(token, targets, count, panel); } bool aurora_vulkan_win32_cancel(uint64_t token) { using namespace aurora::vulkan_win32; @@ -190,6 +211,8 @@ bool aurora_vulkan_win32_get_handles(AuroraDawnVulkanHandles* handles, int64_t* } bool aurora_vulkan_win32_enable(AuroraD3D12StereoSubmittedCallback, void*) { return false; } bool aurora_vulkan_win32_set_targets(uint64_t, const AuroraD3D12StereoTarget*, uint32_t) { return false; } +bool aurora_vulkan_win32_set_targets_with_panel(uint64_t, const AuroraD3D12StereoTarget*, uint32_t, + const AuroraD3D12StereoTarget*) { return false; } bool aurora_vulkan_win32_cancel(uint64_t) { return false; } bool aurora_vulkan_win32_disable() { return true; } void* aurora_vulkan_win32_lock_queue() { return nullptr; } diff --git a/docs/quest-port.md b/docs/quest-port.md index a1cb647..192afc8 100644 --- a/docs/quest-port.md +++ b/docs/quest-port.md @@ -614,6 +614,7 @@ the app: | --- | --- | | `debug.wiicompiled.vtxpad 0` | Turns the stride padding off, to re-check a driver update | | `debug.wiicompiled.validation 1` | Keeps WebGPU validation and robustness on in release builds | +| `debug.wiicompiled.panel_layer 0` | Draws the headset settings panel into the eye images instead of on its own quad layer (`OPENXR.md`, Settings in the headset); read about once a second, so it can be switched while the panel is open | | `debug.wiicompiled.inject :