steam-frame/mitch030504--Wiicompiled_VR_Frame
1
0 Fork 0
mirror of https://github.com/mitch030504/Wiicompiled_VR_Frame.git synced 2026-10-06 06:00:25 +02:00
Code Issues Packages Projects Releases 3 Wiki Activity

Show the headset settings panel as its own OpenXR quad layer

Browse Source 
The panel was drawn into the eye images, so below a 1.00x OpenXR
resolution its 1440x1080 canvas was minified into a small eye region and
the text became hard to read. It is now submitted as a quad layer of its
own over the scene's projection or menu quad, which the compositor samples
directly at any render scale.

Each backend (D3D12, Windows Vulkan, Quest) makes the panel's swapchain
pair (plus two shared buffers on the Quest) the first time the panel
opens. While it is open, the frame hands Aurora one more stereo target
after the eyes, which the bridge fills with the panel texture or a
transparent image. The panel image follows the eyes' displayed/retained
pairing, so a cancelled frame never shows an unwritten panel. The quad
hangs where the pointer's hits are tested. Aurora leaves the panel out of
the eyes in layer mode, and a backend that cannot make the layer falls
back to drawing it into the eyes.

On the Quest, debug.wiicompiled.panel_layer 0 selects the old path at
run time. Measured there at a race start (render_scale 0.8), the layer
costs nothing while the panel is closed; while it is open, app GPU time is
10.5 ms against 9.7 ms drawn into the eyes, with unchanged frame rates.

The replay tests cover lazy creation, cancelled frames, closed and
unplaced panels, and render-first pacing on D3D12 and Windows Vulkan.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
iChris4andClaude Opus 5 committed 2026-09-22 16:26:28 +02:00
1 parent dd7f046214
commit 54c07c982b
22 files changed
+1060 -257
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

OPENXR.md
+36 -11
View File
@@ -258,16 +258,42 @@ of you.
How it is drawn: `settings_overlay.cpp` builds the panel with a second Dear ImGui context of its own,
a 1440 × 1080 canvas at twice the desktop menu's scale with its own font atlas, fed by the pointer
that `openxr_input.cpp` publishes through `vr/openxr_settings_panel.h`. Aurora renders that draw data
into a panel texture once per sealed frame and lays it over each eye after the eye is finished
(`aurora-main/lib/stereo_overlay.cpp`): through the eye's frustum and `viewFromCenter` onto the
screen rectangle for an immersive eye (including headset-rate interpolated eyes, which reuse the
texture), and as a centred rectangle on a virtual-screen eye image. The eye images the OpenXR
backends already submit carry it, so no extra swapchain or composition layer is involved. The
ImGui backend keeps a single projection uniform, so the panel's pass is submitted on its own command
buffer before the desktop's ImGui pass of the same frame is recorded.
into a panel texture once per sealed frame (`aurora-main/lib/stereo_overlay.cpp`). The ImGui backend
keeps a single projection uniform, so the panel's pass is submitted on its own command buffer before
the desktop's ImGui pass of the same frame is recorded.
The panel is shown as a compositor quad layer of its own, submitted over the scene's projection or
menu quad layer. The compositor samples the 1440 × 1080 canvas directly, so its text stays sharp
whatever `render_scale` gives the eyes. Every backend (D3D12, Windows Vulkan, Quest) makes the
panel's swapchain pair the first time the panel opens (two 1440 × 1080 swapchains, plus two shared
buffers on the Quest) and keeps it for the session. Until then nothing is allocated, and while the
panel is closed nothing is copied or submitted. While it is open, each frame hands Aurora one more
target after the eyes: the stereo bridge copies the panel texture into it with the eyes (or a
transparent image on a frame where the panel is not drawn). The layer follows the eyes' swapchain
pairing: the image a frame wrote is shown only once that frame is submitted, so a cancelled frame
never shows an unwritten panel. The quad hangs exactly where the pointer's hits are tested
(`SettingsPanelScreen` in `openxr_integration.cpp`). ImGui's premultiplied output is blended with
`XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT`, and Aurora leaves the panel out of the eyes
(`aurora_set_stereo_panel_layer`).
If a backend cannot make the panel's swapchains, it logs that once and the panel is drawn into the
eye images instead: through the eye's frustum and `viewFromCenter` onto the screen rectangle for an
immersive eye (including headset-rate interpolated eyes, which reuse the texture), and as a centred
rectangle on a virtual-screen eye image. On the Quest, `adb shell setprop
debug.wiicompiled.panel_layer 0` switches to that path at run time, to compare the two.
Measured on a Quest 3 (base game, a Grand Prix start with the player idle, `render_scale = 0.8`,
60 FPS, eight interleaved rounds per state), the layer costs nothing while the panel is closed. While
it is open, the app's GPU time is 10.5 ms per frame with the layer, against 9.7 ms drawn into the eyes
(9.4 ms closed). GPU load is 74% against 67%, and the compositor's time 1.05 ms against 0.75 ms. Game
and headset frame rates did not change. The compositor redraws the layer at display rate, so the
panel stays steady even when the game drops frames.
`mkw_vr_settings_panel_tests` covers the panel button in both controller modes, the release latch,
selection, scrolling and the canvas mapping; `gx_fifo_tests` covers where the panel lands in each eye.
selection, scrolling and the canvas mapping; `gx_fifo_tests` covers where the panel lands in each eye
on the fallback path. `mkw_openxr_replay_tests` and `mkw_openxr_vulkan_replay_tests` cover the layer:
nothing made before the panel opens, the panel image of a cancelled frame never shown, no layer while
the panel is closed or has no place yet, and render-first pacing.
## The first-person camera
@@ -731,9 +757,8 @@ ends, including mid-frame flushes, so live setting changes cannot invalidate pen
- The cockpit seat, the turning wheel and hand steering have not yet been validated in a headset on
this build: the native wheel's match against the race camera's view, bikes and Quacker, and the
Quest. Hand steering needs analog grips (Touch); the simple controller profile cannot grab.
- The headset settings panel is drawn into the eye images rather than submitted as its own quad
layer, so its text is resampled once more than a compositor layer's would be. It has no laser
beam, only the cursor on the panel itself, and text fields cannot be typed into without a keyboard.
- The headset settings panel has no laser beam, only the cursor on the panel itself, and text fields
cannot be typed into without a keyboard.
- The desktop window remains available as a mirror/fallback.
OpenXR diagnostics are written to the normal run log under
aurora-main/include/aurora/aurora.h
+8
View File
@@ -150,6 +150,14 @@ typedef struct {
float weights[4];
} AuroraVRHandVertex;
/**
* Shows the headset settings panel (aurora_imgui_set_stereo_overlay) as the
* OpenXR backend's own compositor quad layer instead of drawing it into the
* eyes, so the eye resolution no longer limits its text. The backend then asks
* for the panel image as an extra stereo target. Any thread.
*/
void aurora_set_stereo_panel_layer(bool enabled);
// Copies optional runtime-provided hand meshes (XR_FB_hand_tracking_mesh, 26
// joints). Null clears to the procedural glove. Bind poses: x,y,z,w,px,py,pz.
void aurora_set_vr_hand_mesh(uint32_t hand, const AuroraVRHandVertex* vertices, uint32_t vertexCount,
aurora-main/include/aurora/d3d12_interop.h
+11
View File
@@ -64,6 +64,17 @@ bool aurora_d3d12_set_stereo_targets(uint64_t frameToken,
const AuroraD3D12StereoTarget* targets,
uint32_t targetCount);
/**
* The same, plus the headset settings panel's quad-layer image when `panel` is
* not null (aurora_set_stereo_panel_layer): Aurora copies the panel into it, or
* a transparent image while the panel is not showing, with the eyes and under
* the same completion callback.
*/
bool aurora_d3d12_set_stereo_targets_with_panel(uint64_t frameToken,
const AuroraD3D12StereoTarget* targets,
uint32_t targetCount,
const AuroraD3D12StereoTarget* panel);
/**
* Withdraws frameToken only while its target has not been encoded. This is
* safe to race with Aurora's frame worker: false means the worker already owns
aurora-main/include/aurora/vulkan_interop.h
+14
View File
@@ -27,6 +27,8 @@ extern "C" {
*/
enum { AURORA_VULKAN_STEREO_MAX_TARGETS = 2 };
// The eyes, then the settings panel's quad-layer image when one was given.
enum { AURORA_VULKAN_STEREO_MAX_RELEASES = AURORA_VULKAN_STEREO_MAX_TARGETS + 1 };
/**
* Borrowed facts about Aurora's Dawn Vulkan device. colorVkFormat is the
@@ -109,6 +111,18 @@ bool aurora_vulkan_set_stereo_targets(uint64_t frameToken,
const AuroraVulkanStereoTarget* targets,
uint32_t targetCount);
/**
* The same, plus the headset settings panel's quad-layer buffer when `panel` is
* not null (aurora_set_stereo_panel_layer): Aurora copies the panel into it, or
* a transparent image while the panel is not showing, with the eyes. Its
* release entry follows the eyes' in the submitted callback, whose
* releaseCount then counts it too.
*/
bool aurora_vulkan_set_stereo_targets_with_panel(uint64_t frameToken,
const AuroraVulkanStereoTarget* targets,
uint32_t targetCount,
const AuroraVulkanStereoTarget* panel);
/**
* Withdraws frameToken only while its targets have not been encoded. Semantics
* match aurora_d3d12_cancel_stereo_targets: false means the worker already
aurora-main/include/aurora/vulkan_win32_interop.h
+4
View File
@@ -11,6 +11,10 @@ bool aurora_vulkan_win32_configure(const AuroraDawnVulkanHooks* hooks);
bool aurora_vulkan_win32_get_handles(AuroraDawnVulkanHandles* handles, int64_t* colorFormat);
bool aurora_vulkan_win32_enable(AuroraD3D12StereoSubmittedCallback submitted, void* userdata);
bool aurora_vulkan_win32_set_targets(uint64_t token, const AuroraD3D12StereoTarget* targets, uint32_t count);
// Plus the headset settings panel's quad-layer image when panel is not null, as
// aurora_d3d12_set_stereo_targets_with_panel.
bool aurora_vulkan_win32_set_targets_with_panel(uint64_t token, const AuroraD3D12StereoTarget* targets, uint32_t count,
const AuroraD3D12StereoTarget* panel);
bool aurora_vulkan_win32_cancel(uint64_t token);
bool aurora_vulkan_win32_disable();
void* aurora_vulkan_win32_lock_queue();
aurora-main/lib/imgui.cpp
+3
View File
@@ -13,6 +13,7 @@
#include "fs_helper.hpp"
#include "internal.hpp"
#include "stereo_overlay.hpp"
#include "webgpu/gpu.hpp"
#include "window.hpp"
@@ -348,6 +349,8 @@ ImTextureID aurora_imgui_add_texture(uint32_t width, uint32_t height, const void
return aurora::imgui::add_texture(width, height, static_cast<const uint8_t*>(rgba8));
}
void aurora_set_stereo_panel_layer(bool enabled) { aurora::stereo_overlay::set_layer_mode(enabled); }
void aurora_imgui_set_stereo_overlay(ImDrawData* drawData, float widthFraction) {
std::lock_guard lock(aurora::imgui::g_stereoOverlayMutex);
aurora::imgui::g_stereoOverlay = {
aurora-main/lib/stereo_overlay.cpp
+53 -2
View File
@@ -11,6 +11,7 @@
#include "tracy/Tracy.hpp"
#include <array>
#include <atomic>
#include <cmath>
namespace aurora::stereo_overlay {
@@ -74,8 +75,12 @@ struct State {
std::array<wgpu::BindGroup, AURORA_STEREO_EYE_COUNT> bindGroups;
float widthFraction = 0.f;
bool visible = false;
// Stands in for the panel in a layer while it is not showing.
webgpu::TextureWithSampler transparent;
bool transparentCleared = false;
};
State g_state;
std::atomic_bool g_layerMode{false};
bool ensure_pipeline() {
auto& state = g_state;
@@ -314,7 +319,7 @@ wgpu::CommandBuffer prepare(ImDrawData* drawData, float widthFraction) noexcept
void composite_immersive(const wgpu::CommandEncoder& encoder, const wgpu::TextureView& eye,
const Mat4x4<float>& eyeFrustum, const Mat3x4<float>& viewFromCenter,
uint32_t eyeIndex) noexcept {
if (!g_state.visible) {
if (!g_state.visible || layer_mode()) {
return;
}
float screenWidth = 0.f;
@@ -331,7 +336,7 @@ void composite_immersive(const wgpu::CommandEncoder& encoder, const wgpu::Textur
void composite_flat(const wgpu::CommandEncoder& encoder, const wgpu::TextureView& eye, const wgpu::Extent3D& size,
uint32_t eyeIndex) noexcept {
if (!g_state.visible || size.width == 0 || size.height == 0) {
if (!g_state.visible || layer_mode() || size.width == 0 || size.height == 0) {
return;
}
const float imageAspect = static_cast<float>(size.width) / static_cast<float>(size.height);
@@ -340,6 +345,52 @@ void composite_flat(const wgpu::CommandEncoder& encoder, const wgpu::TextureView
eyeIndex);
}
void set_layer_mode(bool enabled) noexcept { g_layerMode.store(enabled, std::memory_order_release); }
bool layer_mode() noexcept { return g_layerMode.load(std::memory_order_acquire); }
bool layer_source(const wgpu::CommandEncoder& encoder, uint32_t width, uint32_t height, stereo::EyeImage& out) noexcept {
auto& state = g_state;
const auto format = webgpu::g_graphicsConfig.surfaceConfiguration.format;
if (width == 0 || height == 0) {
return false;
}
if (state.visible && state.panel.texture && state.panel.size.width == width && state.panel.size.height == height &&
state.panel.format == format) {
out = {.texture = &state.panel.texture, .view = &state.panel.view, .size = state.panel.size, .format = format};
return true;
}
if (!state.transparent.texture || state.transparent.size.width != width ||
state.transparent.size.height != height || state.transparent.format != format) {
state.transparent = webgpu::create_render_texture(width, height, false);
state.transparentCleared = false;
if (state.transparent.size.width != width || state.transparent.size.height != height) {
state.transparent = {};
return false;
}
}
if (!state.transparentCleared) {
const std::array attachments{
wgpu::RenderPassColorAttachment{
.view = state.transparent.view,
.loadOp = wgpu::LoadOp::Clear,
.storeOp = wgpu::StoreOp::Store,
.clearValue = {.r = 0.0, .g = 0.0, .b = 0.0, .a = 0.0},
},
};
const wgpu::RenderPassDescriptor descriptor{
.label = "Headset panel layer clear",
.colorAttachmentCount = attachments.size(),
.colorAttachments = attachments.data(),
};
encoder.BeginRenderPass(&descriptor).End();
state.transparentCleared = true;
}
out = {.texture = &state.transparent.texture, .view = &state.transparent.view, .size = state.transparent.size,
.format = format};
return true;
}
void shutdown() noexcept { g_state = {}; }
} // namespace aurora::stereo_overlay
aurora-main/lib/stereo_overlay.hpp
+16
View File
@@ -3,6 +3,8 @@
#include <aurora/math.hpp>
#include <webgpu/webgpu_cpp.h>
#include "stereo.hpp"
#include <cstdint>
struct ImDrawData;
@@ -28,6 +30,20 @@ void composite_immersive(const wgpu::CommandEncoder& encoder, const wgpu::Textur
void composite_flat(const wgpu::CommandEncoder& encoder, const wgpu::TextureView& eye, const wgpu::Extent3D& size,
uint32_t eyeIndex) noexcept;
// Layer mode: the OpenXR backend shows the panel as its own compositor quad
// layer, sharp at any eye resolution, so it is no longer drawn into the eyes
// (both composite functions do nothing). Any thread; read by the frame worker.
void set_layer_mode(bool enabled) noexcept;
bool layer_mode() noexcept;
// Frame worker, inside a stereo sink: the image to copy into a panel layer of
// width x height in the eyes' format. That is the panel while it is showing at
// exactly that size, and otherwise a transparent image of that size, so a layer
// asked for before the panel's first frame (or after it closed) shows nothing.
// A transparent image is cleared once, by a pass recorded into `encoder`.
// False only when no image can be made.
bool layer_source(const wgpu::CommandEncoder& encoder, uint32_t width, uint32_t height, stereo::EyeImage& out) noexcept;
void shutdown() noexcept;
} // namespace aurora::stereo_overlay
aurora-main/lib/webgpu/d3d12_interop.cpp
+87 -35
View File
@@ -2,6 +2,7 @@
#include "../internal.hpp"
#include "../stereo.hpp"
#include "../stereo_overlay.hpp"
#include "gpu.hpp"
#if defined(_WIN32) && defined(WEBGPU_DAWN) && defined(DAWN_ENABLE_BACKEND_D3D12)
@@ -126,6 +127,11 @@ struct SharedFenceDxgiHandleWire {
void* handle = nullptr;
};
// The eyes, then the settings panel's layer image in a slot of its own so the
// eye intermediates are never resized for it.
constexpr uint32_t kPanelIndex = AURORA_D3D12_STEREO_MAX_TARGETS;
constexpr uint32_t kMaxImages = AURORA_D3D12_STEREO_MAX_TARGETS + 1;
struct IntermediateEye {
ComPtr<ID3D12Resource> resource;
wgpu::SharedTextureMemory memory;
@@ -153,8 +159,8 @@ struct InFlightCommand {
// both sides of every copy alive until this submission's fence completes;
// an eye-size change may otherwise replace the bridge intermediate while
// the GPU is still reading it.
std::array<ComPtr<ID3D12Resource>, AURORA_D3D12_STEREO_MAX_TARGETS> sources;
std::array<ComPtr<ID3D12Resource>, AURORA_D3D12_STEREO_MAX_TARGETS> destinations;
std::array<ComPtr<ID3D12Resource>, kMaxImages> sources;
std::array<ComPtr<ID3D12Resource>, kMaxImages> destinations;
};
class StereoBridge final {
@@ -209,38 +215,51 @@ public:
return WaitForGpuLocked();
}
bool SetTargets(uint64_t token, const AuroraD3D12StereoTarget* targets,
uint32_t targetCount) noexcept {
bool SetTargets(uint64_t token, const AuroraD3D12StereoTarget* targets, uint32_t targetCount,
const AuroraD3D12StereoTarget* panel) noexcept {
if (token == 0 || targets == nullptr || targetCount == 0 ||
targetCount > AURORA_D3D12_STEREO_MAX_TARGETS) {
return false;
}
const auto valid = [](const AuroraD3D12StereoTarget& target) {
if (target.resource == nullptr || target.width == 0 || target.height == 0 ||
target.dxgiFormat == DXGI_FORMAT_UNKNOWN) {
return false;
}
const D3D12_RESOURCE_DESC desc = static_cast<ID3D12Resource*>(target.resource)->GetDesc();
return desc.Dimension == D3D12_RESOURCE_DIMENSION_TEXTURE2D && desc.Width >= target.width &&
desc.Height >= target.height && desc.DepthOrArraySize == 1 && desc.MipLevels == 1 &&
desc.SampleDesc.Count == 1 &&
same_copy_family(desc.Format, static_cast<DXGI_FORMAT>(target.dxgiFormat));
};
std::lock_guard lock(m_mutex);
if (m_framePending || m_encoded) {
return false;
}
for (uint32_t eye = 0; eye < targetCount; ++eye) {
if (targets[eye].resource == nullptr || targets[eye].width == 0 ||
targets[eye].height == 0 || targets[eye].dxgiFormat == DXGI_FORMAT_UNKNOWN) {
if (!valid(targets[eye])) {
return false;
}
auto* resource = static_cast<ID3D12Resource*>(targets[eye].resource);
const D3D12_RESOURCE_DESC desc = resource->GetDesc();
if (desc.Dimension != D3D12_RESOURCE_DIMENSION_TEXTURE2D ||
desc.Width < targets[eye].width || desc.Height < targets[eye].height ||
desc.DepthOrArraySize != 1 || desc.MipLevels != 1 || desc.SampleDesc.Count != 1 ||
!same_copy_family(desc.Format, static_cast<DXGI_FORMAT>(targets[eye].dxgiFormat))) {
}
if (panel != nullptr && !valid(*panel)) {
return false;
}
m_targets[eye] = {
.resource = resource,
.width = targets[eye].width,
.height = targets[eye].height,
.format = static_cast<DXGI_FORMAT>(targets[eye].dxgiFormat),
m_targets = {};
m_imageCount = 0;
const auto add = [&](uint32_t index, const AuroraD3D12StereoTarget& target) {
m_targets[index] = {
.resource = static_cast<ID3D12Resource*>(target.resource),
.width = target.width,
.height = target.height,
.format = static_cast<DXGI_FORMAT>(target.dxgiFormat),
};
m_images[m_imageCount++] = index;
};
for (uint32_t eye = 0; eye < targetCount; ++eye) {
add(eye, targets[eye]);
}
for (uint32_t eye = targetCount; eye < m_targets.size(); ++eye) {
m_targets[eye] = {};
if (panel != nullptr) {
add(kPanelIndex, *panel);
}
m_frameToken = token;
m_targetCount = targetCount;
@@ -307,7 +326,7 @@ private:
if (source.texture == nullptr || sourceFormat == DXGI_FORMAT_UNKNOWN ||
source.size.width != m_targets[eye].width || source.size.height != m_targets[eye].height ||
!same_copy_family(sourceFormat, m_targets[eye].format)) {
Log.error("Stereo eye {} does not match its OpenXR D3D12 target", eye);
Log.error("Stereo image {} does not match its OpenXR D3D12 target", eye);
return false;
}
if (intermediate.texture && intermediate.width == source.size.width &&
@@ -350,7 +369,9 @@ private:
wire.resource = intermediate.resource;
const wgpu::SharedTextureMemoryDescriptor memoryDescriptor{
.nextInChain = &wire.chain,
.label = eye == 0 ? "OpenXR left eye intermediate" : "OpenXR right eye intermediate",
.label = eye == 0 ? "OpenXR left eye intermediate"
: eye == 1 ? "OpenXR right eye intermediate"
: "OpenXR panel intermediate",
};
intermediate.memory = webgpu::g_device.ImportSharedTextureMemory(&memoryDescriptor);
if (!intermediate.memory) {
@@ -368,7 +389,9 @@ private:
return false;
}
const wgpu::TextureDescriptor textureDescriptor{
.label = eye == 0 ? "OpenXR left eye shared texture" : "OpenXR right eye shared texture",
.label = eye == 0 ? "OpenXR left eye shared texture"
: eye == 1 ? "OpenXR right eye shared texture"
: "OpenXR panel shared texture",
.usage = wgpu::TextureUsage::CopyDst,
.dimension = wgpu::TextureDimension::e2D,
.size = {source.size.width, source.size.height, 1},
@@ -391,12 +414,22 @@ private:
bool EncodeLocked(wgpu::CommandEncoder& encoder, const stereo::SinkFrame& frame) noexcept {
CollectCompletedCommandsLocked();
for (uint32_t eye = 0; eye < m_targetCount; ++eye) {
if (!EnsureIntermediate(eye, frame.eyes[eye])) {
std::array<stereo::EyeImage, kMaxImages> sources{};
for (uint32_t n = 0; n < m_imageCount; ++n) {
const uint32_t eye = m_images[n];
if (eye == kPanelIndex) {
if (!stereo_overlay::layer_source(encoder, m_targets[eye].width, m_targets[eye].height, sources[eye])) {
return false;
}
} else {
sources[eye] = frame.eyes[eye];
}
if (!EnsureIntermediate(eye, sources[eye])) {
return false;
}
}
for (uint32_t eye = 0; eye < m_targetCount; ++eye) {
for (uint32_t n = 0; n < m_imageCount; ++n) {
const uint32_t eye = m_images[n];
auto& intermediate = m_intermediates[eye];
const std::array fences{m_webgpuFence};
const std::array values{m_lastExternalFenceValue};
@@ -410,7 +443,8 @@ private:
}
if (intermediate.memory.BeginAccess(intermediate.texture, &begin) != wgpu::Status::Success) {
Log.error("Dawn BeginAccess failed for stereo eye {}", eye);
for (uint32_t begunEye = 0; begunEye < eye; ++begunEye) {
for (uint32_t m = 0; m < n; ++m) {
const uint32_t begunEye = m_images[m];
wgpu::SharedTextureMemoryEndAccessState end{};
m_intermediates[begunEye].memory.EndAccess(m_intermediates[begunEye].texture, &end);
m_intermediates[begunEye].initialized = end.initialized;
@@ -424,10 +458,11 @@ private:
// to one. If a later BeginAccess fails, the rollback above can therefore
// end the earlier accesses without leaving an unsubmitted copy that uses
// a texture after its access interval.
for (uint32_t eye = 0; eye < m_targetCount; ++eye) {
for (uint32_t n = 0; n < m_imageCount; ++n) {
const uint32_t eye = m_images[n];
const auto& intermediate = m_intermediates[eye];
const wgpu::TexelCopyTextureInfo source{
.texture = *frame.eyes[eye].texture,
.texture = *sources[eye].texture,
.mipLevel = 0,
.origin = {},
.aspect = wgpu::TextureAspect::All,
@@ -446,7 +481,8 @@ private:
bool EndAccessLocked() noexcept {
bool success = true;
for (uint32_t eye = 0; eye < m_targetCount; ++eye) {
for (uint32_t n = 0; n < m_imageCount; ++n) {
const uint32_t eye = m_images[n];
auto& intermediate = m_intermediates[eye];
if (!intermediate.accessBegun) {
success = false;
@@ -467,8 +503,8 @@ private:
bool EnqueueNativeCopyLocked() noexcept {
ComPtr<ID3D12CommandAllocator> allocator;
ComPtr<ID3D12GraphicsCommandList> list;
std::array<ComPtr<ID3D12Resource>, AURORA_D3D12_STEREO_MAX_TARGETS> sources;
std::array<ComPtr<ID3D12Resource>, AURORA_D3D12_STEREO_MAX_TARGETS> destinations;
std::array<ComPtr<ID3D12Resource>, kMaxImages> sources;
std::array<ComPtr<ID3D12Resource>, kMaxImages> destinations;
if (FAILED(m_device->CreateCommandAllocator(D3D12_COMMAND_LIST_TYPE_DIRECT,
IID_PPV_ARGS(&allocator))) ||
FAILED(m_device->CreateCommandList(0, D3D12_COMMAND_LIST_TYPE_DIRECT, allocator.Get(),
@@ -477,7 +513,8 @@ private:
return false;
}
for (uint32_t eye = 0; eye < m_targetCount; ++eye) {
for (uint32_t n = 0; n < m_imageCount; ++n) {
const uint32_t eye = m_images[n];
const auto& source = m_intermediates[eye];
const auto& destination = m_targets[eye];
sources[eye] = source.resource;
@@ -564,6 +601,7 @@ private:
}
m_frameToken = 0;
m_targetCount = 0;
m_imageCount = 0;
m_framePending = false;
m_encoded = false;
}
@@ -625,8 +663,11 @@ private:
ComPtr<ID3D12CommandQueue> m_queue;
ComPtr<ID3D12Fence> m_fence;
wgpu::SharedFence m_webgpuFence;
std::array<IntermediateEye, AURORA_D3D12_STEREO_MAX_TARGETS> m_intermediates{};
std::array<PendingTarget, AURORA_D3D12_STEREO_MAX_TARGETS> m_targets{};
std::array<IntermediateEye, kMaxImages> m_intermediates{};
std::array<PendingTarget, kMaxImages> m_targets{};
// The slots of m_targets this frame copies into, eyes first.
std::array<uint32_t, kMaxImages> m_images{};
uint32_t m_imageCount = 0;
std::vector<InFlightCommand> m_commands;
AuroraD3D12StereoSubmittedCallback m_callback = nullptr;
void* m_userdata = nullptr;
@@ -701,7 +742,13 @@ bool aurora_d3d12_set_stereo_targets(uint64_t frameToken,
const AuroraD3D12StereoTarget* targets,
uint32_t targetCount) {
using namespace aurora::d3d12_interop;
return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount);
return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount, nullptr);
}
bool aurora_d3d12_set_stereo_targets_with_panel(uint64_t frameToken, const AuroraD3D12StereoTarget* targets,
uint32_t targetCount, const AuroraD3D12StereoTarget* panel) {
using namespace aurora::d3d12_interop;
return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount, panel);
}
bool aurora_d3d12_cancel_stereo_targets(uint64_t frameToken) {
@@ -744,6 +791,11 @@ bool aurora_d3d12_set_stereo_targets(uint64_t, const AuroraD3D12StereoTarget*, u
return false;
}
bool aurora_d3d12_set_stereo_targets_with_panel(uint64_t, const AuroraD3D12StereoTarget*, uint32_t,
const AuroraD3D12StereoTarget*) {
return false;
}
bool aurora_d3d12_cancel_stereo_targets(uint64_t) { return false; }
bool aurora_d3d12_disable_stereo_bridge() { return true; }
aurora-main/lib/webgpu/vulkan_interop.cpp
+97 -48
View File
@@ -2,6 +2,7 @@
#include "../internal.hpp"
#include "../stereo.hpp"
#include "../stereo_overlay.hpp"
#include "gpu.hpp"
#if defined(__ANDROID__) && defined(WEBGPU_DAWN)
@@ -105,6 +106,11 @@ struct Import {
bool accessBegun = false;
};
// The eyes, then the settings panel's layer image in a slot of its own.
constexpr uint32_t kPanelIndex = AURORA_VULKAN_STEREO_MAX_TARGETS;
constexpr uint32_t kMaxImages = AURORA_VULKAN_STEREO_MAX_RELEASES;
using Releases = std::array<AuroraVulkanStereoRelease, kMaxImages>;
struct PendingTarget {
AHardwareBuffer* buffer = nullptr;
uint32_t width = 0;
@@ -146,8 +152,8 @@ public:
return true;
}
bool SetTargets(uint64_t token, const AuroraVulkanStereoTarget* targets,
uint32_t targetCount) noexcept {
bool SetTargets(uint64_t token, const AuroraVulkanStereoTarget* targets, uint32_t targetCount,
const AuroraVulkanStereoTarget* panel) noexcept {
if (token == 0 || targets == nullptr || targetCount == 0 ||
targetCount > AURORA_VULKAN_STEREO_MAX_TARGETS) {
return false;
@@ -157,25 +163,36 @@ public:
return false;
}
const int64_t auroraFormat = to_vk_format(m_auroraFormat);
const auto valid = [&](const AuroraVulkanStereoTarget& target) {
return target.buffer != nullptr && target.width != 0 && target.height != 0 &&
same_copy_family(target.vkFormat, auroraFormat);
};
for (uint32_t eye = 0; eye < targetCount; ++eye) {
const auto& target = targets[eye];
if (target.buffer == nullptr || target.width == 0 || target.height == 0 ||
!same_copy_family(target.vkFormat, auroraFormat)) {
if (!valid(targets[eye])) {
return false;
}
}
for (uint32_t eye = 0; eye < targetCount; ++eye) {
m_targets[eye] = {
.buffer = targets[eye].buffer,
.width = targets[eye].width,
.height = targets[eye].height,
.vkFormat = targets[eye].vkFormat,
.acquireFenceFd = targets[eye].acquireFenceFd,
.acquireImageLayout = targets[eye].acquireImageLayout,
};
if (panel != nullptr && !valid(*panel)) {
return false;
}
for (uint32_t eye = targetCount; eye < m_targets.size(); ++eye) {
m_targets[eye] = {};
m_targets = {};
m_imageCount = 0;
const auto add = [&](uint32_t index, const AuroraVulkanStereoTarget& target) {
m_targets[index] = {
.buffer = target.buffer,
.width = target.width,
.height = target.height,
.vkFormat = target.vkFormat,
.acquireFenceFd = target.acquireFenceFd,
.acquireImageLayout = target.acquireImageLayout,
};
m_images[m_imageCount++] = index;
};
for (uint32_t eye = 0; eye < targetCount; ++eye) {
add(eye, targets[eye]);
}
if (panel != nullptr) {
add(kPanelIndex, *panel);
}
m_frameToken = token;
m_targetCount = targetCount;
@@ -202,7 +219,7 @@ public:
if (!m_framePending || !m_encoded || frame.frameToken != m_frameToken) {
return;
}
std::array<AuroraVulkanStereoRelease, AURORA_VULKAN_STEREO_MAX_TARGETS> releases{};
Releases releases{};
const bool success = EndAccessLocked(releases);
NotifyLocked(frame.frameToken, success, true, releases);
ClearFrameLocked();
@@ -215,7 +232,7 @@ public:
}
const uint64_t token = m_frameToken;
const bool encoded = m_encoded;
std::array<AuroraVulkanStereoRelease, AURORA_VULKAN_STEREO_MAX_TARGETS> releases{};
Releases releases{};
if (encoded) {
EndAccessLocked(releases);
for (auto& release : releases) {
@@ -242,7 +259,7 @@ private:
const auto& target = m_targets[eye];
if (source.texture == nullptr || source.format != m_auroraFormat ||
source.size.width != target.width || source.size.height != target.height) {
Log.error("Stereo eye {} does not match its OpenXR Vulkan target ({}x{} vs {}x{})", eye,
Log.error("Stereo image {} does not match its OpenXR Vulkan target ({}x{} vs {}x{})", eye,
source.size.width, source.size.height, target.width, target.height);
return nullptr;
}
@@ -264,7 +281,9 @@ private:
ahb.handle = target.buffer;
const wgpu::SharedTextureMemoryDescriptor memoryDescriptor{
.nextInChain = &ahb,
.label = eye == 0 ? "OpenXR left eye AHardwareBuffer" : "OpenXR right eye AHardwareBuffer",
.label = eye == 0 ? "OpenXR left eye AHardwareBuffer"
: eye == 1 ? "OpenXR right eye AHardwareBuffer"
: "OpenXR panel AHardwareBuffer",
};
import.memory = webgpu::g_device.ImportSharedTextureMemory(&memoryDescriptor);
if (!import.memory) {
@@ -291,7 +310,9 @@ private:
return nullptr;
}
const wgpu::TextureDescriptor textureDescriptor{
.label = eye == 0 ? "OpenXR left eye shared texture" : "OpenXR right eye shared texture",
.label = eye == 0 ? "OpenXR left eye shared texture"
: eye == 1 ? "OpenXR right eye shared texture"
: "OpenXR panel shared texture",
.usage = wgpu::TextureUsage::CopyDst,
.dimension = wgpu::TextureDimension::e2D,
.size = {target.width, target.height, 1},
@@ -313,19 +334,29 @@ private:
}
bool EncodeLocked(wgpu::CommandEncoder& encoder, const stereo::SinkFrame& frame) noexcept {
std::array<Import*, AURORA_VULKAN_STEREO_MAX_TARGETS> imports{};
for (uint32_t eye = 0; eye < m_targetCount; ++eye) {
imports[eye] = EnsureImport(eye, frame.eyes[eye]);
std::array<stereo::EyeImage, kMaxImages> sources{};
std::array<Import*, kMaxImages> imports{};
for (uint32_t n = 0; n < m_imageCount; ++n) {
const uint32_t eye = m_images[n];
if (eye == kPanelIndex) {
if (!stereo_overlay::layer_source(encoder, m_targets[eye].width, m_targets[eye].height, sources[eye])) {
return false;
}
} else {
sources[eye] = frame.eyes[eye];
}
imports[eye] = EnsureImport(eye, sources[eye]);
if (imports[eye] == nullptr) {
return false;
}
}
for (uint32_t eye = 0; eye < m_targetCount; ++eye) {
for (uint32_t n = 0; n < m_imageCount; ++n) {
const uint32_t eye = m_images[n];
auto& target = m_targets[eye];
auto& import = *imports[eye];
if (import.accessBegun) {
Log.error("AHardwareBuffer for eye {} is still under a previous access", eye);
RollbackAccesses(imports, eye);
RollbackAccesses(imports, n);
return false;
}
// The OpenXR side's release barrier leaves the image in acquireImageLayout;
@@ -350,7 +381,7 @@ private:
close_fd(target.acquireFenceFd);
if (!acquireFence) {
Log.error("Dawn could not import the OpenXR copy-out fence for eye {}", eye);
RollbackAccesses(imports, eye);
RollbackAccesses(imports, n);
return false;
}
}
@@ -370,15 +401,16 @@ private:
}
if (import.memory.BeginAccess(import.texture, &begin) != wgpu::Status::Success) {
Log.error("Dawn BeginAccess failed for stereo eye {}", eye);
RollbackAccesses(imports, eye);
RollbackAccesses(imports, n);
return false;
}
import.accessBegun = true;
}
for (uint32_t eye = 0; eye < m_targetCount; ++eye) {
for (uint32_t n = 0; n < m_imageCount; ++n) {
const uint32_t eye = m_images[n];
const auto& import = *imports[eye];
const wgpu::TexelCopyTextureInfo source{
.texture = *frame.eyes[eye].texture,
.texture = *sources[eye].texture,
.mipLevel = 0,
.origin = {},
.aspect = wgpu::TextureAspect::All,
@@ -396,9 +428,10 @@ private:
return true;
}
void RollbackAccesses(const std::array<Import*, AURORA_VULKAN_STEREO_MAX_TARGETS>& imports,
uint32_t count) noexcept {
for (uint32_t eye = 0; eye < count; ++eye) {
// Ends the accesses begun for the first `count` images of this frame.
void RollbackAccesses(const std::array<Import*, kMaxImages>& imports, uint32_t count) noexcept {
for (uint32_t n = 0; n < count; ++n) {
const uint32_t eye = m_images[n];
if (imports[eye] != nullptr && imports[eye]->accessBegun) {
wgpu::SharedTextureMemoryEndAccessState end{};
imports[eye]->memory.EndAccess(imports[eye]->texture, &end);
@@ -411,13 +444,15 @@ private:
}
}
bool EndAccessLocked(
std::array<AuroraVulkanStereoRelease, AURORA_VULKAN_STEREO_MAX_TARGETS>& releases) noexcept {
// Fills one release per image of this frame, in order: the eyes, then the panel.
bool EndAccessLocked(Releases& releases) noexcept {
bool success = true;
for (auto& release : releases) {
release = {.releaseFenceFd = -1, .releasedImageLayout = VK_IMAGE_LAYOUT_UNDEFINED};
}
for (uint32_t eye = 0; eye < m_targetCount; ++eye) {
for (uint32_t n = 0; n < m_imageCount; ++n) {
const uint32_t eye = m_images[n];
auto& release = releases[n];
Import* import = m_encodedImports[eye];
if (import == nullptr || !import->accessBegun) {
success = false;
@@ -431,7 +466,7 @@ private:
success = false;
} else {
import->initialized = end.initialized;
releases[eye].releasedImageLayout = layout.newLayout;
release.releasedImageLayout = layout.newLayout;
for (size_t i = 0; i < end.fenceCount; ++i) {
wgpu::SharedFenceSyncFDExportInfo syncFd{};
wgpu::SharedFenceExportInfo info{};
@@ -441,16 +476,16 @@ private:
// The fence keeps its descriptor; hand the caller an independent one.
const int duplicate = ::dup(syncFd.handle);
if (duplicate >= 0) {
if (releases[eye].releaseFenceFd >= 0) {
if (release.releaseFenceFd >= 0) {
// Dawn normally returns exactly one fence per access. Both must
// be honoured and one descriptor cannot express two fences, so
// the earlier one is retired on the CPU before handing over the
// latest.
Log.warn("Dawn returned several release fences for eye {}; merging on the CPU", eye);
wait_sync_fd(releases[eye].releaseFenceFd);
close_fd(releases[eye].releaseFenceFd);
wait_sync_fd(release.releaseFenceFd);
close_fd(release.releaseFenceFd);
}
releases[eye].releaseFenceFd = duplicate;
release.releaseFenceFd = duplicate;
}
} else {
Log.error("Dawn returned a non-sync-fd fence for eye {}", eye);
@@ -482,12 +517,13 @@ private:
m_encodedImports = {};
m_frameToken = 0;
m_targetCount = 0;
m_imageCount = 0;
m_framePending = false;
m_encoded = false;
}
void PublishAndClearFrameLocked(uint64_t token, bool success, bool gpuWorkQueued) noexcept {
std::array<AuroraVulkanStereoRelease, AURORA_VULKAN_STEREO_MAX_TARGETS> releases{};
Releases releases{};
for (auto& release : releases) {
release = {.releaseFenceFd = -1, .releasedImageLayout = VK_IMAGE_LAYOUT_UNDEFINED};
}
@@ -496,10 +532,9 @@ private:
}
void NotifyLocked(uint64_t token, bool success, bool gpuWorkQueued,
const std::array<AuroraVulkanStereoRelease, AURORA_VULKAN_STEREO_MAX_TARGETS>&
releases) noexcept {
const Releases& releases) noexcept {
if (m_callback != nullptr) {
m_callback(token, success, gpuWorkQueued, releases.data(), m_targetCount, m_userdata);
m_callback(token, success, gpuWorkQueued, releases.data(), m_imageCount, m_userdata);
} else {
for (auto release : releases) {
close_fd(release.releaseFenceFd);
@@ -509,8 +544,11 @@ private:
std::mutex m_mutex;
std::unordered_map<AHardwareBuffer*, Import> m_imports;
std::array<PendingTarget, AURORA_VULKAN_STEREO_MAX_TARGETS> m_targets{};
std::array<Import*, AURORA_VULKAN_STEREO_MAX_TARGETS> m_encodedImports{};
std::array<PendingTarget, kMaxImages> m_targets{};
std::array<Import*, kMaxImages> m_encodedImports{};
// The slots of m_targets this frame copies into, eyes first.
std::array<uint32_t, kMaxImages> m_images{};
uint32_t m_imageCount = 0;
wgpu::TextureFormat m_auroraFormat = wgpu::TextureFormat::Undefined;
AuroraVulkanStereoSubmittedCallback m_callback = nullptr;
void* m_userdata = nullptr;
@@ -575,7 +613,13 @@ bool aurora_vulkan_set_stereo_targets(uint64_t frameToken,
const AuroraVulkanStereoTarget* targets,
uint32_t targetCount) {
using namespace aurora::vulkan_interop;
return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount);
return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount, nullptr);
}
bool aurora_vulkan_set_stereo_targets_with_panel(uint64_t frameToken, const AuroraVulkanStereoTarget* targets,
uint32_t targetCount, const AuroraVulkanStereoTarget* panel) {
using namespace aurora::vulkan_interop;
return g_bridge && g_bridge->SetTargets(frameToken, targets, targetCount, panel);
}
bool aurora_vulkan_cancel_stereo_targets(uint64_t frameToken) {
@@ -615,6 +659,11 @@ bool aurora_vulkan_set_stereo_targets(uint64_t, const AuroraVulkanStereoTarget*,
return false;
}
bool aurora_vulkan_set_stereo_targets_with_panel(uint64_t, const AuroraVulkanStereoTarget*, uint32_t,
const AuroraVulkanStereoTarget*) {
return false;
}
bool aurora_vulkan_cancel_stereo_targets(uint64_t) { return false; }
bool aurora_vulkan_disable_stereo_bridge() { return true; }
aurora-main/lib/webgpu/vulkan_win32_interop.cpp
+37 -14
View File
@@ -2,6 +2,7 @@
#include <aurora/vulkan_win32_interop.h>
#include "../internal.hpp"
#include "../stereo.hpp"
#include "../stereo_overlay.hpp"
#include "gpu.hpp"
#if defined(_WIN32) && defined(WEBGPU_DAWN) && defined(DAWN_ENABLE_BACKEND_VULKAN)
#include <windows.h>
@@ -53,27 +54,38 @@ bool CopyCompatible(int64_t a, int64_t b) {
((a == 44 || a == 50) && (b == 44 || b == 50));
}
struct Import { uint64_t image; uint32_t width, height; wgpu::TextureFormat format; wgpu::Texture texture; };
// The eyes, then the settings panel's layer image after them.
constexpr uint32_t kMaxImages = 3;
class Bridge {
public:
std::mutex mutex;
std::vector<Import> imports;
std::array<AuroraD3D12StereoTarget, 2> targets{};
std::array<wgpu::Texture, 2> active{};
std::array<AuroraD3D12StereoTarget, kMaxImages> targets{};
std::array<wgpu::Texture, kMaxImages> active{};
uint64_t token = 0;
uint32_t count = 0;
// Images this frame copies: `count` eyes, plus the panel when `panel` is set.
uint32_t images = 0;
bool panel = false;
bool encoded = false;
AuroraD3D12StereoSubmittedCallback callback;
void* userdata;
Bridge(AuroraD3D12StereoSubmittedCallback cb, void* data) : callback(cb), userdata(data) {}
bool Set(uint64_t next, const AuroraD3D12StereoTarget* data, uint32_t n) {
bool Set(uint64_t next, const AuroraD3D12StereoTarget* data, uint32_t n, const AuroraD3D12StereoTarget* panelTarget) {
if (!next || !data || !n || n > 2) return false;
std::lock_guard guard(mutex);
if (token) return false;
const auto valid = [](const AuroraD3D12StereoTarget& target) {
return target.resource && target.width && target.height;
};
for (uint32_t i = 0; i < n; ++i) {
if (!data[i].resource || !data[i].width || !data[i].height) return false;
targets[i] = data[i];
if (!valid(data[i])) return false;
}
token = next; count = n; return true;
if (panelTarget && !valid(*panelTarget)) return false;
for (uint32_t i = 0; i < n; ++i) targets[i] = data[i];
panel = panelTarget != nullptr;
if (panel) targets[n] = *panelTarget;
token = next; count = n; images = n + (panel ? 1u : 0u); return true;
}
bool Cancel(uint64_t wanted) {
std::unique_lock guard(mutex, std::try_to_lock);
@@ -83,9 +95,13 @@ public:
bool Encode(wgpu::CommandEncoder& encoder, const stereo::SinkFrame& frame) {
std::lock_guard guard(mutex);
if (!token || encoded || frame.frameToken != token) return false;
std::array<stereo::EyeImage, kMaxImages> sources{};
for (uint32_t i = 0; i < count; ++i) sources[i] = frame.eyes[i];
if (panel && !stereo_overlay::layer_source(encoder, targets[count].width, targets[count].height, sources[count]))
return false;
// Validate/import every target before recording any copy.
for (uint32_t i = 0; i < count; ++i) {
const auto& eye = frame.eyes[i];
for (uint32_t i = 0; i < images; ++i) {
const auto& eye = sources[i];
const auto& target = targets[i];
if (!eye.texture || eye.size.width != target.width || eye.size.height != target.height ||
!CopyCompatible(VkFormat(eye.format), target.dxgiFormat)) return false;
@@ -112,9 +128,9 @@ public:
}
active[i] = it->texture;
}
for (uint32_t i = 0; i < count; ++i) {
for (uint32_t i = 0; i < images; ++i) {
wgpu::TexelCopyTextureInfo source, destination;
source.texture = *frame.eyes[i].texture;
source.texture = *sources[i].texture;
destination.texture = active[i];
wgpu::Extent3D size{targets[i].width, targets[i].height, 1};
encoder.CopyTextureToTexture(&source, &destination, &size);
@@ -125,11 +141,11 @@ public:
void Submitted(const stereo::SinkFrame& frame) {
std::lock_guard guard(mutex);
if (!token || !encoded || token != frame.frameToken) return;
std::array<void*, 2> textures{};
for (uint32_t i = 0; i < count; ++i) textures[i] = active[i].Get();
std::array<void*, kMaxImages> textures{};
for (uint32_t i = 0; i < images; ++i) textures[i] = active[i].Get();
// Append the COLOR_ATTACHMENT_OPTIMAL release barriers to Dawn's queue,
// flush them under its device guard, then allow the XR thread to release.
const bool success = api.release(webgpu::g_device.Get(), textures.data(), count) != 0;
const bool success = api.release(webgpu::g_device.Get(), textures.data(), images) != 0;
const auto completed = token;
token = 0; encoded = false;
callback(completed, success, userdata);
@@ -160,7 +176,12 @@ bool aurora_vulkan_win32_enable(AuroraD3D12StereoSubmittedCallback cb, void* dat
}
bool aurora_vulkan_win32_set_targets(uint64_t token, const AuroraD3D12StereoTarget* targets, uint32_t count) {
using namespace aurora::vulkan_win32;
return bridge && bridge->Set(token, targets, count);
return bridge && bridge->Set(token, targets, count, nullptr);
}
bool aurora_vulkan_win32_set_targets_with_panel(uint64_t token, const AuroraD3D12StereoTarget* targets, uint32_t count,
const AuroraD3D12StereoTarget* panel) {
using namespace aurora::vulkan_win32;
return bridge && bridge->Set(token, targets, count, panel);
}
bool aurora_vulkan_win32_cancel(uint64_t token) {
using namespace aurora::vulkan_win32;
@@ -190,6 +211,8 @@ bool aurora_vulkan_win32_get_handles(AuroraDawnVulkanHandles* handles, int64_t*
}
bool aurora_vulkan_win32_enable(AuroraD3D12StereoSubmittedCallback, void*) { return false; }
bool aurora_vulkan_win32_set_targets(uint64_t, const AuroraD3D12StereoTarget*, uint32_t) { return false; }
bool aurora_vulkan_win32_set_targets_with_panel(uint64_t, const AuroraD3D12StereoTarget*, uint32_t,
const AuroraD3D12StereoTarget*) { return false; }
bool aurora_vulkan_win32_cancel(uint64_t) { return false; }
bool aurora_vulkan_win32_disable() { return true; }
void* aurora_vulkan_win32_lock_queue() { return nullptr; }
docs/quest-port.md
+1
View File
@@ -614,6 +614,7 @@ the app:
| --- | --- |
| `debug.wiicompiled.vtxpad 0` | Turns the stride padding off, to re-check a driver update |
| `debug.wiicompiled.validation 1` | Keeps WebGPU validation and robustness on in release builds |
| `debug.wiicompiled.panel_layer 0` | Draws the headset settings panel into the eye images instead of on its own quad layer (`OPENXR.md`, Settings in the headset); read about once a second, so it can be switched while the panel is open |
| `debug.wiicompiled.inject <n>:<button>` | Presses `a`, `b`, `x`, `y`, `start`, `up`, `down`, `left` or `right` for 12 XR frames each time `<n>` changes. As a Wii Remote, `x`/`y`/`start` are 1/2/+, the directions push the Nunchuk stick, and `home`, `c` and `z` also exist. `panel` presses the settings panel's button (left Y, or both thumbsticks as a gamepad), opening or closing it (see `OPENXR.md`) |
| `debug.wiicompiled.fpslog 1` | Logs the game's rendered frame rate every 5 s, with per-frame averages of the producer's waits for the frame worker's DONE and SEALED phases and of the worker's seal, permit wait, prepare and encode stretches. A third line reports the GX thread's command ring (records, waits, busy share). A second line gives the GPU time per frame from timestamp queries on every pass (`mono` native render, `eyeL`/`eyeR` replays, `screen`, `panel`, `efbcopy`, `palette`, `peek`, plus `passes-span` from the first pass begin to the last pass end and `between-passes` for copies and idle gaps). The compositor's `VrApi` log line gives headset FPS, `GPU%`, `CPU%`, clock levels and app GPU time (`App=`) |
runtime/include/vr/openxr_backend.h
+41
View File
@@ -5,6 +5,7 @@
#if defined(MKW_ENABLE_OPENXR)
#include "vr/openxr_runtime.h"
#include "vr/openxr_settings_panel.h"
#include <array>
#include <cstdint>
@@ -42,6 +43,44 @@ enum class OpenXRSubmissionStatus {
ShuttingDown,
};
// The headset settings panel as a compositor quad layer of its own, over the
// eyes or the menu screen, so the eye resolution never limits its text. Its
// image is rendered with the frame's eyes (Aurora's panel stereo target) into a
// swapchain of the panel canvas's own size. Nothing is allocated or copied
// until the panel first opens, and nothing is submitted while it is closed.
struct OpenXRPanelLayer {
// The frame renders the panel's image: set by the pacing thread when the
// panel is open, cleared by a backend that could not provide the layer.
bool requested = false;
// Where it hangs in the application space, once the head pose is known.
bool placed = false;
XrPosef pose{{0.0f, 0.0f, 0.0f, 1.0f}, {0.0f, 0.0f, 0.0f}};
float width_meters = 0.0f;
float height_meters = 0.0f;
};
// The panel image's size, which is the settings panel canvas's.
inline constexpr uint32_t kOpenXRPanelLayerWidth = static_cast<uint32_t>(kSettingsPanelWidthPixels);
inline constexpr uint32_t kOpenXRPanelLayerHeight = static_cast<uint32_t>(kSettingsPanelHeightPixels);
// The panel's layer, submitted after (so over) the scene's.
inline XrCompositionLayerQuad OpenXRPanelQuadLayer(const OpenXRPanelLayer& panel, XrSpace space,
XrSwapchain swapchain) noexcept {
XrCompositionLayerQuad quad{XR_TYPE_COMPOSITION_LAYER_QUAD};
// ImGui leaves premultiplied colour in the cleared panel image.
quad.layerFlags = XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT;
quad.space = space;
quad.eyeVisibility = XR_EYE_VISIBILITY_BOTH;
quad.subImage.swapchain = swapchain;
quad.subImage.imageRect = {{0, 0},
{static_cast<int32_t>(kOpenXRPanelLayerWidth),
static_cast<int32_t>(kOpenXRPanelLayerHeight)}};
quad.subImage.imageArrayIndex = 0;
quad.pose = panel.pose;
quad.size = {panel.width_meters, panel.height_meters};
return quad;
}
struct OpenXRPresentation {
OpenXRFrameMode mode = OpenXRFrameMode::ImmersiveProjection;
@@ -56,6 +95,8 @@ struct OpenXRPresentation {
// head pose good enough to anchor against.
bool quad_anchored = false;
XrPosef quad_pose{{0.0f, 0.0f, 0.0f, 1.0f}, {0.0f, 0.0f, 0.0f}};
OpenXRPanelLayer panel;
};
struct OpenXRBackendFrame {
runtime/include/vr/openxr_d3d12.h
+3
View File
@@ -98,6 +98,9 @@ public:
bool Shutdown();
bool IsBound() const;
// False once the settings panel's own layer could not be set up; the panel
// is then drawn into the eyes again.
bool PanelLayerAvailable() const;
const OpenXRD3D12GraphicsRequirements& GraphicsRequirements() const;
int64_t SwapchainFormat() const;
const std::string& LastError() const;
runtime/include/vr/openxr_vulkan.h
+3
View File
@@ -78,6 +78,9 @@ public:
bool Shutdown();
bool IsBound() const;
// False once the settings panel's own layer could not be set up; the panel
// is then drawn into the eyes again.
bool PanelLayerAvailable() const;
const OpenXRVulkanGraphicsRequirements& GraphicsRequirements() const;
int64_t SwapchainFormat() const;
const std::string& LastError() const;
runtime/include/vr/openxr_vulkan_win32.h
+3
View File
@@ -95,6 +95,9 @@ public:
bool Shutdown();
bool IsBound() const;
// False once the settings panel's own layer could not be set up; the panel
// is then drawn into the eyes again.
bool PanelLayerAvailable() const;
const OpenXRWindowsVulkanGraphicsRequirements& GraphicsRequirements() const;
int64_t SwapchainFormat() const;
const std::string& LastError() const;
runtime/include/vr/openxr_windows.h
+3
View File
@@ -54,6 +54,9 @@ public:
bool Shutdown() {
return std::visit([&](auto& backend) -> bool { return backend->Shutdown(); }, backend_);
}
bool PanelLayerAvailable() {
return std::visit([&](auto& backend) -> bool { return backend->PanelLayerAvailable(); }, backend_);
}
int64_t SwapchainFormat() {
return std::visit([&](auto& backend) -> int64_t { return backend->SwapchainFormat(); }, backend_);
}
runtime/src/vr/openxr_d3d12.cpp
+121 -36
View File
@@ -312,6 +312,23 @@ public:
static_cast<int64_t>(swapchain_format_),
};
}
// The settings panel's layer image, rendered with the eyes while it is open.
AuroraD3D12StereoTarget panel_target{};
const bool panel = frame.presentation.panel.requested && EnsurePanelSwapchains();
frame.presentation.panel.requested = panel;
if (panel) {
if (!AcquireSwapchain(panel_swapchain_)) {
ReleaseAcquiredSwapchains();
EndActiveFrameWithoutLayers(frame.xr_frame);
return OpenXRD3D12BeginStatus::Error;
}
panel_target = {
panel_swapchain_.images[panel_swapchain_.acquired_index].texture,
panel_swapchain_.width,
panel_swapchain_.height,
static_cast<int64_t>(swapchain_format_),
};
}
diagnostics::OnSwapchainAcquire(acquire_timer);
{
@@ -323,7 +340,8 @@ public:
submission_unsafe_ = false;
}
if (!diagnostics::Measure(diagnostics::Stage::SetTargets, [&] {
return aurora_d3d12_set_stereo_targets(frame.xr_frame.serial, targets.data(), target_count);
return aurora_d3d12_set_stereo_targets_with_panel(frame.xr_frame.serial, targets.data(), target_count,
panel ? &panel_target : nullptr);
})) {
{
std::lock_guard lock(submission_mutex_);
@@ -564,6 +582,10 @@ public:
// swapchain, not an explicit image index. Keep the displayed pair
// separate from the pair Aurora can write or cancel next.
std::swap(eye_swapchains_, retained_swapchains_);
if (frame.presentation.panel.requested) {
std::swap(panel_swapchain_, retained_panel_swapchain_);
}
retained_panel_valid_ = frame.presentation.panel.requested;
retained_frame_ = frame;
retained_session_serial_ = render_session_serial_;
retained_space_serial_ = render_space_serial_;
@@ -663,9 +685,7 @@ public:
quad.size.width = std::max(0.25f, frame.presentation.quad_width_meters);
quad.size.height = quad.size.width * static_cast<float>(retained_swapchains_[0].height) /
static_cast<float>(retained_swapchains_[0].width);
const XrCompositionLayerBaseHeader* layers[] = {
reinterpret_cast<const XrCompositionLayerBaseHeader*>(&quad)};
return runtime_->EndFrame(active_frame_, layers, 1);
return EndFrameWithPanel(frame, reinterpret_cast<const XrCompositionLayerBaseHeader*>(&quad));
} else {
std::array<XrCompositionLayerProjectionView, kOpenXREyeCount> views{};
for (uint32_t eye = 0; eye < kOpenXREyeCount; ++eye) {
@@ -685,12 +705,24 @@ public:
projection.space = runtime_->AppSpace();
projection.viewCount = kOpenXREyeCount;
projection.views = views.data();
const XrCompositionLayerBaseHeader* layers[] = {
reinterpret_cast<const XrCompositionLayerBaseHeader*>(&projection)};
return runtime_->EndFrame(active_frame_, layers, 1);
return EndFrameWithPanel(frame, reinterpret_cast<const XrCompositionLayerBaseHeader*>(&projection));
}
}
// Ends the compositor frame with the scene's layer and, while the retained
// frame rendered it, the settings panel's layer over it.
bool EndFrameWithPanel(const OpenXRBackendFrame& frame, const XrCompositionLayerBaseHeader* scene) {
const auto& panel = frame.presentation.panel;
XrCompositionLayerQuad panel_quad{};
const XrCompositionLayerBaseHeader* layers[2] = {scene, nullptr};
uint32_t count = 1;
if (retained_panel_valid_ && panel.requested && panel.placed) {
panel_quad = OpenXRPanelQuadLayer(panel, runtime_->AppSpace(), retained_panel_swapchain_.handle);
layers[count++] = reinterpret_cast<const XrCompositionLayerBaseHeader*>(&panel_quad);
}
return runtime_->EndFrame(active_frame_, layers, count);
}
bool Shutdown() {
if (shutdown_unsafe_) {
return false;
@@ -742,6 +774,7 @@ public:
}
bool IsBound() const { return bound_; }
bool PanelLayerAvailable() const { return !panel_layer_failed_; }
const OpenXRD3D12GraphicsRequirements& GraphicsRequirements() const { return requirements_; }
int64_t SwapchainFormat() const { return static_cast<int64_t>(swapchain_format_); }
const std::string& LastError() const { return last_error_; }
@@ -782,9 +815,17 @@ private:
bool CreateSwapchainPair(std::array<EyeSwapchain, kOpenXREyeCount>& pair) {
for (uint32_t eye = 0; eye < kOpenXREyeCount; ++eye) {
const auto& view = runtime_->ViewConfiguration()[eye];
auto& swapchain = pair[eye];
swapchain.width = view.render_width;
swapchain.height = view.render_height;
if (!CreateSwapchain(pair[eye], view.render_width, view.render_height,
eye == 0 ? "left eye" : "right eye")) {
return false;
}
}
return true;
}
bool CreateSwapchain(EyeSwapchain& swapchain, uint32_t width, uint32_t height, const char* what) {
swapchain.width = width;
swapchain.height = height;
XrSwapchainCreateInfo create{XR_TYPE_SWAPCHAIN_CREATE_INFO};
create.usageFlags = XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT |
@@ -805,7 +846,7 @@ private:
ObserveResult(result);
if (XR_FAILED(result)) {
std::ostringstream message;
message << "xrCreateSwapchain failed for D3D12 eye " << eye << " (" << result << ')';
message << "xrCreateSwapchain failed for the D3D12 " << what << " swapchain (" << result << ')';
return Fail(message.str());
}
@@ -826,7 +867,6 @@ private:
if (XR_FAILED(result)) {
return Fail("xrEnumerateSwapchainImages failed for a D3D12 eye swapchain");
}
}
return true;
}
@@ -861,58 +901,93 @@ private:
bool ReleaseAcquiredSwapchains() {
bool success = true;
for (auto& swapchain : eye_swapchains_) {
success = ReleaseSwapchain(swapchain) && success;
}
return ReleaseSwapchain(panel_swapchain_) && success;
}
bool ReleaseSwapchain(EyeSwapchain& swapchain) {
if (!swapchain.acquired || swapchain.handle == XR_NULL_HANDLE) {
continue;
return true;
}
if (!swapchain.waited) {
// OpenXR only permits release after a successful wait. Keep the
// image acquired and let session teardown destroy the child.
success = false;
Log(OpenXRLogLevel::Warning,
"cannot release an OpenXR D3D12 image whose wait did not complete");
continue;
Log(OpenXRLogLevel::Warning, "cannot release an OpenXR D3D12 image whose wait did not complete");
return false;
}
if (swapchain.release_forbidden) {
// Aurora reported a failed submission after it may already
// have queued native D3D12 work. Without a trustworthy fence,
// xrReleaseSwapchainImage could race that work. Leave the image
// acquired and let xrDestroySession reclaim the child instead.
success = false;
Log(OpenXRLogLevel::Warning,
"deferring an OpenXR D3D12 image after an unsafe GPU submission");
continue;
// Aurora reported a failed submission after it may already have
// queued GPU work. Without a trustworthy fence the release could race
// that work, so leave the image acquired for xrDestroySession.
Log(OpenXRLogLevel::Warning, "deferring an OpenXR D3D12 image after an unsafe GPU submission");
return false;
}
XrSwapchainImageReleaseInfo release{XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
const XrResult result = xrReleaseSwapchainImage(swapchain.handle, &release);
ObserveResult(result);
if (XR_FAILED(result)) {
success = false;
Fail("xrReleaseSwapchainImage failed for a D3D12 eye swapchain");
continue;
return Fail("xrReleaseSwapchainImage failed for an D3D12 swapchain");
}
swapchain.acquired = false;
swapchain.waited = false;
}
return success;
return true;
}
void AbandonAcquiredSwapchains() noexcept {
for (auto& swapchain : eye_swapchains_) {
if (swapchain.acquired) {
swapchain.release_forbidden = true;
for (auto* swapchain : {&eye_swapchains_[0], &eye_swapchains_[1], &panel_swapchain_}) {
if (swapchain->acquired) {
swapchain->release_forbidden = true;
}
}
}
void AllowAcquiredSwapchainsAfterGpuDrain() noexcept {
for (auto& swapchain : eye_swapchains_) {
if (swapchain.acquired) {
swapchain.release_forbidden = false;
for (auto* swapchain : {&eye_swapchains_[0], &eye_swapchains_[1], &panel_swapchain_}) {
if (swapchain->acquired) {
swapchain->release_forbidden = false;
}
}
}
// The settings panel's swapchain pair, made the first time the panel opens.
// A failure is logged once and the panel is drawn into the eyes again.
bool EnsurePanelSwapchains() {
if (panel_swapchains_ready_) {
return true;
}
if (panel_layer_failed_) {
return false;
}
if (CreateSwapchain(panel_swapchain_, kOpenXRPanelLayerWidth, kOpenXRPanelLayerHeight, "settings panel") &&
CreateSwapchain(retained_panel_swapchain_, kOpenXRPanelLayerWidth, kOpenXRPanelLayerHeight,
"settings panel")) {
panel_swapchains_ready_ = true;
Log(OpenXRLogLevel::Info, "OpenXR settings panel layer ready");
return true;
}
DestroyPanelSwapchains();
panel_layer_failed_ = true;
Log(OpenXRLogLevel::Warning, "the settings panel could not get its own OpenXR layer; drawing it into the eyes");
return false;
}
void DestroyPanelSwapchains() {
for (auto* swapchain : {&panel_swapchain_, &retained_panel_swapchain_}) {
if (swapchain->handle != XR_NULL_HANDLE && !swapchain->acquired) {
xrDestroySwapchain(swapchain->handle);
} else if (swapchain->acquired) {
Log(OpenXRLogLevel::Warning,
"D3D12 panel swapchain still owns an acquired image; deferring its destruction to xrDestroySession");
}
*swapchain = {};
}
panel_swapchains_ready_ = false;
retained_panel_valid_ = false;
}
void DestroySwapchains() {
DestroyPanelSwapchains();
DestroySwapchainPair(eye_swapchains_);
DestroySwapchainPair(retained_swapchains_);
have_retained_frame_ = false;
@@ -988,6 +1063,14 @@ private:
OpenXRD3D12GraphicsRequirements requirements_{};
std::array<EyeSwapchain, kOpenXREyeCount> eye_swapchains_{};
std::array<EyeSwapchain, kOpenXREyeCount> retained_swapchains_{};
// The settings panel's layer: written like the eyes into panel_swapchain_,
// shown from retained_panel_swapchain_ (see FinishFrame).
EyeSwapchain panel_swapchain_{};
EyeSwapchain retained_panel_swapchain_{};
bool panel_swapchains_ready_ = false;
bool panel_layer_failed_ = false;
// The retained frame rendered the panel's image into retained_panel_swapchain_.
bool retained_panel_valid_ = false;
OpenXRD3D12Frame retained_frame_{};
uint64_t retained_session_serial_ = 0;
uint64_t retained_space_serial_ = 0;
@@ -1076,6 +1159,8 @@ bool OpenXRD3D12Backend::Shutdown() { return m_impl->Shutdown(); }
bool OpenXRD3D12Backend::IsBound() const { return m_impl->IsBound(); }
bool OpenXRD3D12Backend::PanelLayerAvailable() const { return m_impl->PanelLayerAvailable(); }
const OpenXRD3D12GraphicsRequirements& OpenXRD3D12Backend::GraphicsRequirements() const {
return m_impl->GraphicsRequirements();
}
runtime/src/vr/openxr_integration.cpp
+54 -2
View File
@@ -40,6 +40,7 @@
#elif defined(__ANDROID__)
#include "vr/openxr_android.h"
#include "vr/openxr_vulkan.h"
#include <sys/system_properties.h>
#include <time.h>
#include <unistd.h>
#define XR_USE_TIMESPEC
@@ -828,6 +829,12 @@ private:
: OpenXRFrameMode::VirtualScreen;
presentation.quad_distance_meters = policy.config.hud_distance_meters;
presentation.quad_width_meters = policy.config.hud_width_meters;
// The settings panel gets a compositor layer of its own while it is
// open, and Aurora leaves it out of the eyes. A backend that could
// not make that layer has the panel drawn into the eyes instead.
const bool panel_layer = backend_->PanelLayerAvailable() && !PanelLayerForcedOff();
aurora_set_stereo_panel_layer(panel_layer);
presentation.panel.requested = panel_layer && OpenXRSettingsPanelOpen();
// Pipeline caches are stored where their stall is invisible: once when a race ends,
// and by the compiler itself while the headset shows the virtual screen. Never
@@ -888,12 +895,14 @@ private:
// before FinishFrame submits a layer built from it.
ServiceRecenterRequest();
UpdateVirtualScreenPose(frame);
const OpenXRPointerScreen panel_screen = SettingsPanelScreen(frame, policy, immersive);
PlacePanelLayer(frame, panel_screen);
if (input_ != nullptr) {
const diagnostics::ScopedStage input_timer(diagnostics::Stage::InputSync);
// After the screen is placed, so the pointer aims at this
// frame's screen rather than the previous one's.
input_->Sync(frame.xr_frame.predicted_display_time, PointerScreen(frame, policy, immersive),
SettingsPanelScreen(frame, policy, immersive), InputSeatFrame(immersive));
panel_screen, InputSeatFrame(immersive));
}
if (!frame.expects_gpu_submission) {
@@ -1018,6 +1027,7 @@ private:
SetInterpolationActive(false);
aurora_set_pipeline_cache_idle_store(false);
aurora_set_stereo_panel_layer(false);
running_.store(false, std::memory_order_release);
MkwVRPolicySetSessionActive(false);
if (!stop_.load(std::memory_order_acquire)) {
@@ -1071,10 +1081,12 @@ private:
// The head pose this packet was located with places the screens and aims the pointer.
ServiceRecenterRequest();
UpdateVirtualScreenPose(packet);
const OpenXRPointerScreen panel_screen = SettingsPanelScreen(packet, policy, immersive);
PlacePanelLayer(packet, panel_screen);
if (input_ != nullptr) {
const diagnostics::ScopedStage input_timer(diagnostics::Stage::InputSync);
input_->Sync(packet.xr_frame.predicted_display_time, PointerScreen(packet, policy, immersive),
SettingsPanelScreen(packet, policy, immersive), InputSeatFrame(immersive));
panel_screen, InputSeatFrame(immersive));
}
if (!packet.expects_gpu_submission) {
// Nothing to render (no rendering requested or no tracking): keep the compositor fed.
@@ -1393,6 +1405,42 @@ private:
return screen;
}
// Android: `adb shell setprop debug.wiicompiled.panel_layer 0` draws the settings panel into
// the eyes again, to compare the two ways or to rule out a runtime's quad layers. Read about
// once a second.
bool PanelLayerForcedOff() noexcept {
#if defined(__ANDROID__)
if (panel_layer_poll_ == 0) {
panel_layer_poll_ = 72;
char value[PROP_VALUE_MAX] = {};
const bool off =
__system_property_get("debug.wiicompiled.panel_layer", value) > 0 && value[0] == '0';
if (off != panel_layer_forced_off_) {
panel_layer_forced_off_ = off;
RT_LOG(RT_TAG_RUNTIME) << "OpenXR: settings panel "
<< (off ? "drawn into the eyes" : "on its own layer")
<< " (debug.wiicompiled.panel_layer)" << std::endl;
}
}
--panel_layer_poll_;
return panel_layer_forced_off_;
#else
return false;
#endif
}
// The settings panel's layer hangs exactly where its pointer hits are
// tested, the rectangle it used to cover in the eyes.
static void PlacePanelLayer(OpenXRBackendFrame& frame, const OpenXRPointerScreen& screen) noexcept {
OpenXRPanelLayer& panel = frame.presentation.panel;
panel.placed = panel.requested && screen.valid;
if (panel.placed) {
panel.pose = screen.pose;
panel.width_meters = 2.0f * screen.half_width_meters;
panel.height_meters = 2.0f * screen.half_height_meters;
}
}
// Centre of the race's 2D screen. ViewFromBase maps a point p of the
// recorded centre-eye space (in metres) to base + lean * p in the
// application space, so the screen Aurora hangs hud_distance_meters ahead
@@ -1601,6 +1649,10 @@ private:
OpenXRLogCallback logger_;
std::unique_ptr<OpenXRRuntime> runtime_;
std::unique_ptr<GraphicsBackend> backend_;
#if defined(__ANDROID__)
uint32_t panel_layer_poll_ = 0;
bool panel_layer_forced_off_ = false;
#endif
std::unique_ptr<OpenXRInput> input_;
std::thread pacing_thread_;
std::atomic_bool stop_{false};
runtime/src/vr/openxr_vulkan.cpp
+232 -70
View File
@@ -37,6 +37,10 @@ namespace {
// other into the compositor image. The sync-fd handshake orders the two
// devices on each buffer, so a deeper ring only adds latency.
constexpr uint32_t kSlotCount = 2;
// Images one frame copies into the compositor: the eyes, then the settings
// panel's layer image, in the order of Aurora's release entries.
constexpr uint32_t kMaxCopies = AURORA_VULKAN_STEREO_MAX_RELEASES;
static_assert(kMaxCopies == kOpenXREyeCount + 1);
// Copy command buffers in flight before the oldest fence is waited on.
constexpr uint32_t kSubmissionRingSize = 3;
constexpr uint64_t kFenceTimeoutNanos = 2'000'000'000ull;
@@ -154,10 +158,10 @@ public:
struct Submission {
VkFence fence = VK_NULL_HANDLE;
VkCommandBuffer command_buffer = VK_NULL_HANDLE;
// Dawn's release fences are imported here, one semaphore per eye. They belong to the
// Dawn's release fences are imported here, one semaphore per image. They belong to the
// submission rather than the slot: importing into a semaphore whose previous wait is
// still pending is invalid, and only the submission's fence proves that wait completed.
std::array<VkSemaphore, kOpenXREyeCount> wait_semaphores{};
std::array<VkSemaphore, kMaxCopies> wait_semaphores{};
bool busy = false;
};
@@ -173,7 +177,11 @@ public:
uint64_t token = 0;
uint32_t slot = 0;
uint32_t target_count = 0;
std::array<VkImage, kOpenXREyeCount> swapchain_images{};
// The settings panel's layer image follows the eyes.
bool panel = false;
std::array<VkImage, kMaxCopies> swapchain_images{};
uint32_t Count() const noexcept { return target_count + (panel ? 1u : 0u); }
};
bool QueryGraphicsRequirements(OpenXRRuntime& runtime) {
@@ -383,6 +391,9 @@ public:
frame.render_height[1] = frame.render_height[0];
}
// The settings panel's layer image, rendered with the eyes while it is open.
const bool panel = frame.presentation.panel.requested && EnsurePanelResources();
frame.presentation.panel.requested = panel;
const diagnostics::Stopwatch acquire_timer;
for (uint32_t eye = 0; eye < target_count; ++eye) {
if (!AcquireSwapchain(eye_swapchains_[eye])) {
@@ -391,28 +402,33 @@ public:
return OpenXRBeginStatus::Error;
}
}
if (panel && !AcquireSwapchain(panel_swapchain_)) {
ReleaseAcquiredSwapchains();
EndActiveFrameWithoutLayers(frame.xr_frame);
return OpenXRBeginStatus::Error;
}
diagnostics::OnSwapchainAcquire(acquire_timer);
std::array<AuroraVulkanStereoTarget, kOpenXREyeCount> targets{};
AuroraVulkanStereoTarget panel_target{};
const uint32_t slot = next_slot_;
{
std::lock_guard lock(vk_mutex_);
for (uint32_t eye = 0; eye < target_count; ++eye) {
EyeSlot& eye_slot = slots_[eye][slot];
targets[eye] = {
eye_slot.buffer,
eye_slot.width,
eye_slot.height,
static_cast<int64_t>(aurora_format_),
DupFd(eye_slot.pending_acquire_fd),
static_cast<int32_t>(eye_slot.layout),
};
targets[eye] = SlotTargetLocked(slots_[eye][slot]);
}
pending_copy_ = {frame.xr_frame.serial, slot, target_count, {}};
if (panel) {
panel_target = SlotTargetLocked(panel_slots_[slot]);
}
pending_copy_ = {frame.xr_frame.serial, slot, target_count, panel, {}};
for (uint32_t eye = 0; eye < target_count; ++eye) {
pending_copy_.swapchain_images[eye] =
eye_swapchains_[eye].images[eye_swapchains_[eye].acquired_index].image;
}
if (panel) {
pending_copy_.swapchain_images[target_count] =
panel_swapchain_.images[panel_swapchain_.acquired_index].image;
}
}
{
std::lock_guard lock(submission_mutex_);
@@ -422,11 +438,15 @@ public:
submission_success_ = false;
submission_unsafe_ = false;
}
if (!aurora_vulkan_set_stereo_targets(frame.xr_frame.serial, targets.data(), target_count)) {
if (!aurora_vulkan_set_stereo_targets_with_panel(frame.xr_frame.serial, targets.data(), target_count,
panel ? &panel_target : nullptr)) {
// The duplicates were not taken; the slots keep their own descriptors.
for (uint32_t eye = 0; eye < target_count; ++eye) {
CloseFd(targets[eye].acquireFenceFd);
}
if (panel) {
CloseFd(panel_target.acquireFenceFd);
}
{
std::lock_guard lock(submission_mutex_);
awaiting_token_ = 0;
@@ -546,24 +566,23 @@ public:
packet.render_width[1] = packet.render_width[0];
packet.render_height[1] = packet.render_height[0];
}
// The settings panel's layer image, rendered with the eyes while it is open.
const bool panel = packet.presentation.panel.requested && EnsurePanelResources();
packet.presentation.panel.requested = panel;
std::array<AuroraVulkanStereoTarget, kOpenXREyeCount> targets{};
AuroraVulkanStereoTarget panel_target{};
const uint32_t slot = next_slot_;
{
std::lock_guard lock(vk_mutex_);
DiscardPendingReleasesLocked();
for (uint32_t eye = 0; eye < target_count; ++eye) {
EyeSlot& eye_slot = slots_[eye][slot];
targets[eye] = {
eye_slot.buffer,
eye_slot.width,
eye_slot.height,
static_cast<int64_t>(aurora_format_),
DupFd(eye_slot.pending_acquire_fd),
static_cast<int32_t>(eye_slot.layout),
};
targets[eye] = SlotTargetLocked(slots_[eye][slot]);
}
if (panel) {
panel_target = SlotTargetLocked(panel_slots_[slot]);
}
// The compositor images are acquired by BeginFrameForPacket, once the eyes exist.
pending_copy_ = {packet.xr_frame.serial, slot, target_count, {}};
pending_copy_ = {packet.xr_frame.serial, slot, target_count, panel, {}};
deferred_copy_ = true;
}
{
@@ -574,10 +593,14 @@ public:
submission_success_ = false;
submission_unsafe_ = false;
}
if (!aurora_vulkan_set_stereo_targets(packet.xr_frame.serial, targets.data(), target_count)) {
if (!aurora_vulkan_set_stereo_targets_with_panel(packet.xr_frame.serial, targets.data(), target_count,
panel ? &panel_target : nullptr)) {
for (uint32_t eye = 0; eye < target_count; ++eye) {
CloseFd(targets[eye].acquireFenceFd);
}
if (panel) {
CloseFd(panel_target.acquireFenceFd);
}
{
std::lock_guard lock(submission_mutex_);
awaiting_token_ = 0;
@@ -653,6 +676,10 @@ public:
// swapchain, so keep the displayed pair separate from the pair
// Aurora may write next.
std::swap(eye_swapchains_, retained_swapchains_);
if (frame.presentation.panel.requested) {
std::swap(panel_swapchain_, retained_panel_swapchain_);
}
retained_panel_valid_ = frame.presentation.panel.requested;
retained_frame_ = frame;
retained_session_serial_ = render_session_serial_;
retained_space_serial_ = render_space_serial_;
@@ -726,6 +753,7 @@ public:
const uint32_t target_count =
presentation_target_count(frame.presentation);
const bool panel = frame.presentation.panel.requested;
const diagnostics::Stopwatch acquire_timer;
for (uint32_t eye = 0; eye < target_count; ++eye) {
if (!AcquireSwapchain(eye_swapchains_[eye])) {
@@ -734,6 +762,11 @@ public:
return OpenXRBeginStatus::Error;
}
}
if (panel && !AcquireSwapchain(panel_swapchain_)) {
ReleaseAcquiredSwapchains();
EndActiveFrameWithoutLayers(frame.xr_frame);
return OpenXRBeginStatus::Error;
}
diagnostics::OnSwapchainAcquire(acquire_timer);
{
std::lock_guard lock(vk_mutex_);
@@ -741,6 +774,10 @@ public:
pending_copy_.swapchain_images[eye] =
eye_swapchains_[eye].images[eye_swapchains_[eye].acquired_index].image;
}
if (panel) {
pending_copy_.swapchain_images[target_count] =
panel_swapchain_.images[panel_swapchain_.acquired_index].image;
}
}
return OpenXRBeginStatus::Ready;
}
@@ -875,9 +912,7 @@ public:
quad.size.width = std::max(0.25f, frame.presentation.quad_width_meters);
quad.size.height = quad.size.width * static_cast<float>(retained_swapchains_[0].height) /
static_cast<float>(retained_swapchains_[0].width);
const XrCompositionLayerBaseHeader* layers[] = {
reinterpret_cast<const XrCompositionLayerBaseHeader*>(&quad)};
return runtime_->EndFrame(active_frame_, layers, 1);
return EndFrameWithPanel(frame, reinterpret_cast<const XrCompositionLayerBaseHeader*>(&quad));
}
std::array<XrCompositionLayerProjectionView, kOpenXREyeCount> views{};
for (uint32_t eye = 0; eye < kOpenXREyeCount; ++eye) {
@@ -897,9 +932,21 @@ public:
projection.space = runtime_->AppSpace();
projection.viewCount = kOpenXREyeCount;
projection.views = views.data();
const XrCompositionLayerBaseHeader* layers[] = {
reinterpret_cast<const XrCompositionLayerBaseHeader*>(&projection)};
return runtime_->EndFrame(active_frame_, layers, 1);
return EndFrameWithPanel(frame, reinterpret_cast<const XrCompositionLayerBaseHeader*>(&projection));
}
// Ends the compositor frame with the scene's layer and, while the retained
// frame rendered it, the settings panel's layer over it.
bool EndFrameWithPanel(const OpenXRBackendFrame& frame, const XrCompositionLayerBaseHeader* scene) {
const auto& panel = frame.presentation.panel;
XrCompositionLayerQuad panel_quad{};
const XrCompositionLayerBaseHeader* layers[2] = {scene, nullptr};
uint32_t count = 1;
if (retained_panel_valid_ && panel.requested && panel.placed) {
panel_quad = OpenXRPanelQuadLayer(panel, runtime_->AppSpace(), retained_panel_swapchain_.handle);
layers[count++] = reinterpret_cast<const XrCompositionLayerBaseHeader*>(&panel_quad);
}
return runtime_->EndFrame(active_frame_, layers, count);
}
bool Shutdown() {
@@ -953,6 +1000,7 @@ public:
}
bool IsBound() const { return bound_; }
bool PanelLayerAvailable() const { return !panel_layer_failed_; }
const OpenXRVulkanGraphicsRequirements& GraphicsRequirements() const { return requirements_; }
int64_t SwapchainFormat() const { return static_cast<int64_t>(swapchain_format_); }
const std::string& LastError() const { return last_error_; }
@@ -1208,7 +1256,7 @@ private:
VkSemaphoreCreateInfo semaphore_info{VK_STRUCTURE_TYPE_SEMAPHORE_CREATE_INFO};
for (VkSemaphore& semaphore : submission.wait_semaphores) {
if (vkCreateSemaphore(vk_device_, &semaphore_info, nullptr, &semaphore) != VK_SUCCESS) {
return Fail("vkCreateSemaphore failed for an eye copy wait semaphore");
return Fail("vkCreateSemaphore failed for a copy wait semaphore");
}
}
}
@@ -1263,6 +1311,44 @@ private:
return true;
}
// Aurora's view of a shared buffer: it waits on the slot's last copy-out.
AuroraVulkanStereoTarget SlotTargetLocked(const EyeSlot& slot) const noexcept {
return {
slot.buffer,
slot.width,
slot.height,
static_cast<int64_t>(aurora_format_),
DupFd(slot.pending_acquire_fd),
static_cast<int32_t>(slot.layout),
};
}
// The settings panel's swapchain pair and shared buffers, made the first
// time it opens and kept for the session.
bool EnsurePanelResources() {
if (!EnsurePanelSwapchains()) {
return false;
}
std::lock_guard lock(vk_mutex_);
if (panel_slots_ready_) {
return true;
}
for (EyeSlot& slot : panel_slots_) {
if (!AllocateSlot(slot, kOpenXRPanelLayerWidth, kOpenXRPanelLayerHeight)) {
for (EyeSlot& allocated : panel_slots_) {
DestroySlotLocked(allocated);
}
DestroyPanelSwapchains();
panel_layer_failed_ = true;
Log(OpenXRLogLevel::Warning,
"the settings panel's shared buffers could not be allocated; drawing it into the eyes");
return false;
}
}
panel_slots_ready_ = true;
return true;
}
bool AllocateSlot(EyeSlot& slot, uint32_t width, uint32_t height) {
AHardwareBuffer_Desc desc{};
desc.width = width;
@@ -1358,6 +1444,16 @@ private:
}
for (auto& eye : slots_) {
for (EyeSlot& slot : eye) {
DestroySlotLocked(slot);
}
}
for (EyeSlot& slot : panel_slots_) {
DestroySlotLocked(slot);
}
panel_slots_ready_ = false;
}
void DestroySlotLocked(EyeSlot& slot) noexcept {
CloseFd(slot.pending_acquire_fd);
if (vk_device_ != VK_NULL_HANDLE) {
if (slot.signal_semaphore != VK_NULL_HANDLE) {
@@ -1375,8 +1471,6 @@ private:
}
slot = {};
}
}
}
// ---- The copy into the compositor image -----------------------------------
@@ -1384,7 +1478,7 @@ private:
const AuroraVulkanStereoRelease* releases, uint32_t release_count,
void* userdata) {
auto* self = static_cast<Impl*>(userdata);
std::array<AuroraVulkanStereoRelease, kOpenXREyeCount> owned{};
std::array<AuroraVulkanStereoRelease, kMaxCopies> owned{};
for (auto& release : owned) {
release = {-1, VK_IMAGE_LAYOUT_UNDEFINED};
}
@@ -1406,7 +1500,7 @@ private:
std::lock_guard submission_lock(self->submission_mutex_);
expected = token == self->awaiting_token_ && token == self->pending_copy_.token;
}
if (expected && success && release_count >= self->pending_copy_.target_count) {
if (expected && success && release_count >= self->pending_copy_.Count()) {
if (self->deferred_copy_) {
// No compositor frame is open yet: keep Dawn's release fences for
// CopyRenderedEyes, which records the copy once the frame is begun.
@@ -1444,7 +1538,12 @@ private:
self->submission_cv_.notify_all();
}
CopyOutcome RecordAndSubmitCopyLocked(std::array<AuroraVulkanStereoRelease, kOpenXREyeCount>& releases) {
// The slot image `n` of a copy reads: an eye's, or after the eyes the panel's.
EyeSlot& CopySlotLocked(const PendingCopy& copy, uint32_t n) noexcept {
return n < copy.target_count ? slots_[n][copy.slot] : panel_slots_[copy.slot];
}
CopyOutcome RecordAndSubmitCopyLocked(std::array<AuroraVulkanStereoRelease, kMaxCopies>& releases) {
const auto close_releases = [&releases] {
for (auto& release : releases) {
CloseFd(release.releaseFenceFd);
@@ -1467,19 +1566,19 @@ private:
return CopyOutcome::Skipped;
}
std::array<VkSemaphore, kOpenXREyeCount> waits{};
std::array<VkPipelineStageFlags, kOpenXREyeCount> wait_stages{};
std::array<VkSemaphore, kOpenXREyeCount> signals{};
std::array<VkSemaphore, kMaxCopies> waits{};
std::array<VkPipelineStageFlags, kMaxCopies> wait_stages{};
std::array<VkSemaphore, kMaxCopies> signals{};
uint32_t wait_count = 0;
uint32_t signal_count = 0;
constexpr VkImageSubresourceRange kColorRange{VK_IMAGE_ASPECT_COLOR_BIT, 0, 1, 0, 1};
for (uint32_t eye = 0; eye < copy.target_count; ++eye) {
EyeSlot& slot = slots_[eye][copy.slot];
AuroraVulkanStereoRelease& release = releases[eye];
for (uint32_t n = 0; n < copy.Count(); ++n) {
EyeSlot& slot = CopySlotLocked(copy, n);
AuroraVulkanStereoRelease& release = releases[n];
if (release.releaseFenceFd >= 0) {
const VkSemaphore wait_semaphore = submission->wait_semaphores[eye];
const VkSemaphore wait_semaphore = submission->wait_semaphores[n];
VkImportSemaphoreFdInfoKHR import{VK_STRUCTURE_TYPE_IMPORT_SEMAPHORE_FD_INFO_KHR};
import.semaphore = wait_semaphore;
import.flags = VK_SEMAPHORE_IMPORT_TEMPORARY_BIT;
@@ -1529,7 +1628,7 @@ private:
to_destination.newLayout = VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL;
to_destination.srcQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
to_destination.dstQueueFamilyIndex = VK_QUEUE_FAMILY_IGNORED;
to_destination.image = copy.swapchain_images[eye];
to_destination.image = copy.swapchain_images[n];
to_destination.subresourceRange = kColorRange;
vkCmdPipelineBarrier(cmd, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, VK_PIPELINE_STAGE_TRANSFER_BIT, 0, 0,
nullptr, 0, nullptr, 1, &acquire);
@@ -1542,7 +1641,7 @@ private:
region.srcSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1};
region.dstSubresource = {VK_IMAGE_ASPECT_COLOR_BIT, 0, 0, 1};
region.extent = {slot.width, slot.height, 1};
vkCmdCopyImage(cmd, slot.image, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, copy.swapchain_images[eye],
vkCmdCopyImage(cmd, slot.image, VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL, copy.swapchain_images[n],
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, 1, &region);
// Back to the layout the compositor expects, and hand the shared
@@ -1600,8 +1699,8 @@ private:
// Exporting a sync fd from a binary semaphore resets it, so the same
// semaphore serves the next copy. Dawn waits on this before it writes
// the buffer again.
for (uint32_t eye = 0; eye < copy.target_count; ++eye) {
EyeSlot& slot = slots_[eye][copy.slot];
for (uint32_t n = 0; n < copy.Count(); ++n) {
EyeSlot& slot = CopySlotLocked(copy, n);
VkSemaphoreGetFdInfoKHR get_fd{VK_STRUCTURE_TYPE_SEMAPHORE_GET_FD_INFO_KHR};
get_fd.semaphore = slot.signal_semaphore;
get_fd.handleType = VK_EXTERNAL_SEMAPHORE_HANDLE_TYPE_SYNC_FD_BIT;
@@ -1676,9 +1775,17 @@ private:
bool CreateSwapchainPair(std::array<EyeSwapchain, kOpenXREyeCount>& pair) {
for (uint32_t eye = 0; eye < kOpenXREyeCount; ++eye) {
const auto& view = runtime_->ViewConfiguration()[eye];
auto& swapchain = pair[eye];
swapchain.width = view.render_width;
swapchain.height = view.render_height;
if (!CreateSwapchain(pair[eye], view.render_width, view.render_height,
eye == 0 ? "left eye" : "right eye")) {
return false;
}
}
return true;
}
bool CreateSwapchain(EyeSwapchain& swapchain, uint32_t width, uint32_t height, const char* what) {
swapchain.width = width;
swapchain.height = height;
XrSwapchainCreateInfo create{XR_TYPE_SWAPCHAIN_CREATE_INFO};
create.usageFlags = XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT | XR_SWAPCHAIN_USAGE_TRANSFER_DST_BIT;
@@ -1693,7 +1800,7 @@ private:
ObserveResult(result);
if (XR_FAILED(result)) {
std::ostringstream message;
message << "xrCreateSwapchain failed for Vulkan eye " << eye << " (" << result << ')';
message << "xrCreateSwapchain failed for the Vulkan " << what << " swapchain (" << result << ')';
return Fail(message.str());
}
@@ -1714,7 +1821,6 @@ private:
if (XR_FAILED(result)) {
return Fail("xrEnumerateSwapchainImages failed for a Vulkan eye swapchain");
}
}
return true;
}
@@ -1748,50 +1854,93 @@ private:
bool ReleaseAcquiredSwapchains() {
bool success = true;
for (auto& swapchain : eye_swapchains_) {
success = ReleaseSwapchain(swapchain) && success;
}
return ReleaseSwapchain(panel_swapchain_) && success;
}
bool ReleaseSwapchain(EyeSwapchain& swapchain) {
if (!swapchain.acquired || swapchain.handle == XR_NULL_HANDLE) {
continue;
return true;
}
if (!swapchain.waited) {
success = false;
// OpenXR only permits release after a successful wait. Keep the
// image acquired and let session teardown destroy the child.
Log(OpenXRLogLevel::Warning, "cannot release an OpenXR Vulkan image whose wait did not complete");
continue;
return false;
}
if (swapchain.release_forbidden) {
success = false;
// Aurora reported a failed submission after it may already have
// queued GPU work. Without a trustworthy fence the release could race
// that work, so leave the image acquired for xrDestroySession.
Log(OpenXRLogLevel::Warning, "deferring an OpenXR Vulkan image after an unsafe GPU submission");
continue;
return false;
}
XrSwapchainImageReleaseInfo release{XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
const XrResult result = xrReleaseSwapchainImage(swapchain.handle, &release);
ObserveResult(result);
if (XR_FAILED(result)) {
success = false;
Fail("xrReleaseSwapchainImage failed for a Vulkan eye swapchain");
continue;
return Fail("xrReleaseSwapchainImage failed for an Vulkan swapchain");
}
swapchain.acquired = false;
swapchain.waited = false;
}
return success;
return true;
}
void AbandonAcquiredSwapchains() noexcept {
for (auto& swapchain : eye_swapchains_) {
if (swapchain.acquired) {
swapchain.release_forbidden = true;
for (auto* swapchain : {&eye_swapchains_[0], &eye_swapchains_[1], &panel_swapchain_}) {
if (swapchain->acquired) {
swapchain->release_forbidden = true;
}
}
}
void AllowAcquiredSwapchainsAfterGpuDrain() noexcept {
for (auto& swapchain : eye_swapchains_) {
if (swapchain.acquired) {
swapchain.release_forbidden = false;
for (auto* swapchain : {&eye_swapchains_[0], &eye_swapchains_[1], &panel_swapchain_}) {
if (swapchain->acquired) {
swapchain->release_forbidden = false;
}
}
}
// The settings panel's swapchain pair, made the first time the panel opens.
// A failure is logged once and the panel is drawn into the eyes again.
bool EnsurePanelSwapchains() {
if (panel_swapchains_ready_) {
return true;
}
if (panel_layer_failed_) {
return false;
}
if (CreateSwapchain(panel_swapchain_, kOpenXRPanelLayerWidth, kOpenXRPanelLayerHeight, "settings panel") &&
CreateSwapchain(retained_panel_swapchain_, kOpenXRPanelLayerWidth, kOpenXRPanelLayerHeight,
"settings panel")) {
panel_swapchains_ready_ = true;
Log(OpenXRLogLevel::Info, "OpenXR settings panel layer ready");
return true;
}
DestroyPanelSwapchains();
panel_layer_failed_ = true;
Log(OpenXRLogLevel::Warning, "the settings panel could not get its own OpenXR layer; drawing it into the eyes");
return false;
}
void DestroyPanelSwapchains() {
for (auto* swapchain : {&panel_swapchain_, &retained_panel_swapchain_}) {
if (swapchain->handle != XR_NULL_HANDLE && !swapchain->acquired) {
xrDestroySwapchain(swapchain->handle);
} else if (swapchain->acquired) {
Log(OpenXRLogLevel::Warning,
"Vulkan panel swapchain still owns an acquired image; deferring its destruction to xrDestroySession");
}
*swapchain = {};
}
panel_swapchains_ready_ = false;
retained_panel_valid_ = false;
}
void DestroySwapchains() {
DestroyPanelSwapchains();
DestroySwapchainPair(eye_swapchains_);
DestroySwapchainPair(retained_swapchains_);
have_retained_frame_ = false;
@@ -1849,6 +1998,14 @@ private:
OpenXRVulkanGraphicsRequirements requirements_{};
std::array<EyeSwapchain, kOpenXREyeCount> eye_swapchains_{};
std::array<EyeSwapchain, kOpenXREyeCount> retained_swapchains_{};
// The settings panel's layer: written like the eyes into panel_swapchain_,
// shown from retained_panel_swapchain_ (see FinishFrame).
EyeSwapchain panel_swapchain_{};
EyeSwapchain retained_panel_swapchain_{};
bool panel_swapchains_ready_ = false;
bool panel_layer_failed_ = false;
// The retained frame rendered the panel's image into retained_panel_swapchain_.
bool retained_panel_valid_ = false;
OpenXRBackendFrame retained_frame_{};
uint64_t retained_session_serial_ = 0;
uint64_t retained_space_serial_ = 0;
@@ -1870,6 +2027,9 @@ private:
PFN_vkGetSemaphoreFdKHR pfn_get_semaphore_fd_ = nullptr;
PFN_vkGetAndroidHardwareBufferPropertiesANDROID pfn_get_ahb_properties_ = nullptr;
std::array<std::array<EyeSlot, kSlotCount>, kOpenXREyeCount> slots_{};
// The settings panel's shared buffers, allocated when it first opens.
std::array<EyeSlot, kSlotCount> panel_slots_{};
bool panel_slots_ready_ = false;
std::array<Submission, kSubmissionRingSize> submissions_{};
uint32_t next_submission_ = 0;
uint32_t next_slot_ = 0;
@@ -1877,7 +2037,7 @@ private:
// Render-first pacing (PreparePacket): Aurora's release fences arrive while no compositor
// frame is active, so the copy is recorded later by CopyRenderedEyes.
bool deferred_copy_ = false;
std::array<AuroraVulkanStereoRelease, kOpenXREyeCount> pending_releases_{};
std::array<AuroraVulkanStereoRelease, kMaxCopies> pending_releases_{};
bool have_pending_releases_ = false;
uint64_t next_packet_serial_ = 1ull << 40; // never collides with the runtime's frame serials
XrTime last_display_time_ = 0;
@@ -1959,6 +2119,8 @@ bool OpenXRVulkanBackend::Shutdown() { return m_impl->Shutdown(); }
bool OpenXRVulkanBackend::IsBound() const { return m_impl->IsBound(); }
bool OpenXRVulkanBackend::PanelLayerAvailable() const { return m_impl->PanelLayerAvailable(); }
const OpenXRVulkanGraphicsRequirements& OpenXRVulkanBackend::GraphicsRequirements() const {
return m_impl->GraphicsRequirements();
}
runtime/src/vr/openxr_vulkan_win32.cpp
+121 -36
View File
@@ -334,6 +334,23 @@ public:
static_cast<int64_t>(swapchain_format_),
};
}
// The settings panel's layer image, rendered with the eyes while it is open.
AuroraD3D12StereoTarget panel_target{};
const bool panel = frame.presentation.panel.requested && EnsurePanelSwapchains();
frame.presentation.panel.requested = panel;
if (panel) {
if (!AcquireSwapchain(panel_swapchain_)) {
ReleaseAcquiredSwapchains();
EndActiveFrameWithoutLayers(frame.xr_frame);
return OpenXRWindowsVulkanBeginStatus::Error;
}
panel_target = {
reinterpret_cast<void*>(panel_swapchain_.images[panel_swapchain_.acquired_index].image),
panel_swapchain_.width,
panel_swapchain_.height,
static_cast<int64_t>(swapchain_format_),
};
}
diagnostics::OnSwapchainAcquire(acquire_timer);
{
@@ -345,7 +362,8 @@ public:
submission_unsafe_ = false;
}
if (!diagnostics::Measure(diagnostics::Stage::SetTargets, [&] {
return aurora_vulkan_win32_set_targets(frame.xr_frame.serial, targets.data(), target_count);
return aurora_vulkan_win32_set_targets_with_panel(frame.xr_frame.serial, targets.data(), target_count,
panel ? &panel_target : nullptr);
})) {
{
std::lock_guard lock(submission_mutex_);
@@ -586,6 +604,10 @@ public:
// swapchain, not an explicit image index. Keep the displayed pair
// separate from the pair Aurora can write or cancel next.
std::swap(eye_swapchains_, retained_swapchains_);
if (frame.presentation.panel.requested) {
std::swap(panel_swapchain_, retained_panel_swapchain_);
}
retained_panel_valid_ = frame.presentation.panel.requested;
retained_frame_ = frame;
retained_session_serial_ = render_session_serial_;
retained_space_serial_ = render_space_serial_;
@@ -685,9 +707,7 @@ public:
quad.size.width = std::max(0.25f, frame.presentation.quad_width_meters);
quad.size.height = quad.size.width * static_cast<float>(retained_swapchains_[0].height) /
static_cast<float>(retained_swapchains_[0].width);
const XrCompositionLayerBaseHeader* layers[] = {
reinterpret_cast<const XrCompositionLayerBaseHeader*>(&quad)};
return runtime_->EndFrame(active_frame_, layers, 1);
return EndFrameWithPanel(frame, reinterpret_cast<const XrCompositionLayerBaseHeader*>(&quad));
} else {
std::array<XrCompositionLayerProjectionView, kOpenXREyeCount> views{};
for (uint32_t eye = 0; eye < kOpenXREyeCount; ++eye) {
@@ -707,12 +727,24 @@ public:
projection.space = runtime_->AppSpace();
projection.viewCount = kOpenXREyeCount;
projection.views = views.data();
const XrCompositionLayerBaseHeader* layers[] = {
reinterpret_cast<const XrCompositionLayerBaseHeader*>(&projection)};
return runtime_->EndFrame(active_frame_, layers, 1);
return EndFrameWithPanel(frame, reinterpret_cast<const XrCompositionLayerBaseHeader*>(&projection));
}
}
// Ends the compositor frame with the scene's layer and, while the retained
// frame rendered it, the settings panel's layer over it.
bool EndFrameWithPanel(const OpenXRBackendFrame& frame, const XrCompositionLayerBaseHeader* scene) {
const auto& panel = frame.presentation.panel;
XrCompositionLayerQuad panel_quad{};
const XrCompositionLayerBaseHeader* layers[2] = {scene, nullptr};
uint32_t count = 1;
if (retained_panel_valid_ && panel.requested && panel.placed) {
panel_quad = OpenXRPanelQuadLayer(panel, runtime_->AppSpace(), retained_panel_swapchain_.handle);
layers[count++] = reinterpret_cast<const XrCompositionLayerBaseHeader*>(&panel_quad);
}
return runtime_->EndFrame(active_frame_, layers, count);
}
bool Shutdown() {
if (shutdown_unsafe_) {
return false;
@@ -769,6 +801,7 @@ public:
}
bool IsBound() const { return bound_; }
bool PanelLayerAvailable() const { return !panel_layer_failed_; }
const OpenXRWindowsVulkanGraphicsRequirements& GraphicsRequirements() const { return requirements_; }
int64_t SwapchainFormat() const { return static_cast<int64_t>(swapchain_format_); }
const std::string& LastError() const { return last_error_; }
@@ -809,9 +842,17 @@ private:
bool CreateSwapchainPair(std::array<EyeSwapchain, kOpenXREyeCount>& pair) {
for (uint32_t eye = 0; eye < kOpenXREyeCount; ++eye) {
const auto& view = runtime_->ViewConfiguration()[eye];
auto& swapchain = pair[eye];
swapchain.width = view.render_width;
swapchain.height = view.render_height;
if (!CreateSwapchain(pair[eye], view.render_width, view.render_height,
eye == 0 ? "left eye" : "right eye")) {
return false;
}
}
return true;
}
bool CreateSwapchain(EyeSwapchain& swapchain, uint32_t width, uint32_t height, const char* what) {
swapchain.width = width;
swapchain.height = height;
XrSwapchainCreateInfo create{XR_TYPE_SWAPCHAIN_CREATE_INFO};
create.usageFlags = XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT |
@@ -831,7 +872,7 @@ private:
ObserveResult(result);
if (XR_FAILED(result)) {
std::ostringstream message;
message << "xrCreateSwapchain failed for Vulkan eye " << eye << " (" << result << ')';
message << "xrCreateSwapchain failed for the Vulkan " << what << " swapchain (" << result << ')';
return Fail(message.str());
}
@@ -852,7 +893,6 @@ private:
if (XR_FAILED(result)) {
return Fail("xrEnumerateSwapchainImages failed for a Vulkan eye swapchain");
}
}
return true;
}
@@ -893,58 +933,93 @@ private:
const auto queue_guard = runtime_ ? runtime_->LockGraphicsQueue() : OpenXRRuntime::GraphicsQueueGuard{nullptr, nullptr};
bool success = true;
for (auto& swapchain : eye_swapchains_) {
success = ReleaseSwapchain(swapchain) && success;
}
return ReleaseSwapchain(panel_swapchain_) && success;
}
bool ReleaseSwapchain(EyeSwapchain& swapchain) {
if (!swapchain.acquired || swapchain.handle == XR_NULL_HANDLE) {
continue;
return true;
}
if (!swapchain.waited) {
// OpenXR only permits release after a successful wait. Keep the
// image acquired and let session teardown destroy the child.
success = false;
Log(OpenXRLogLevel::Warning,
"cannot release an OpenXR Vulkan image whose wait did not complete");
continue;
Log(OpenXRLogLevel::Warning, "cannot release an OpenXR Vulkan image whose wait did not complete");
return false;
}
if (swapchain.release_forbidden) {
// Aurora reported a failed submission after it may already
// have queued native Vulkan work. Without a trustworthy fence,
// xrReleaseSwapchainImage could race that work. Leave the image
// acquired and let xrDestroySession reclaim the child instead.
success = false;
Log(OpenXRLogLevel::Warning,
"deferring an OpenXR Vulkan image after an unsafe GPU submission");
continue;
// Aurora reported a failed submission after it may already have
// queued GPU work. Without a trustworthy fence the release could race
// that work, so leave the image acquired for xrDestroySession.
Log(OpenXRLogLevel::Warning, "deferring an OpenXR Vulkan image after an unsafe GPU submission");
return false;
}
XrSwapchainImageReleaseInfo release{XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
const XrResult result = xrReleaseSwapchainImage(swapchain.handle, &release);
ObserveResult(result);
if (XR_FAILED(result)) {
success = false;
Fail("xrReleaseSwapchainImage failed for a Vulkan eye swapchain");
continue;
return Fail("xrReleaseSwapchainImage failed for an Vulkan swapchain");
}
swapchain.acquired = false;
swapchain.waited = false;
}
return success;
return true;
}
void AbandonAcquiredSwapchains() noexcept {
for (auto& swapchain : eye_swapchains_) {
if (swapchain.acquired) {
swapchain.release_forbidden = true;
for (auto* swapchain : {&eye_swapchains_[0], &eye_swapchains_[1], &panel_swapchain_}) {
if (swapchain->acquired) {
swapchain->release_forbidden = true;
}
}
}
void AllowAcquiredSwapchainsAfterGpuDrain() noexcept {
for (auto& swapchain : eye_swapchains_) {
if (swapchain.acquired) {
swapchain.release_forbidden = false;
for (auto* swapchain : {&eye_swapchains_[0], &eye_swapchains_[1], &panel_swapchain_}) {
if (swapchain->acquired) {
swapchain->release_forbidden = false;
}
}
}
// The settings panel's swapchain pair, made the first time the panel opens.
// A failure is logged once and the panel is drawn into the eyes again.
bool EnsurePanelSwapchains() {
if (panel_swapchains_ready_) {
return true;
}
if (panel_layer_failed_) {
return false;
}
if (CreateSwapchain(panel_swapchain_, kOpenXRPanelLayerWidth, kOpenXRPanelLayerHeight, "settings panel") &&
CreateSwapchain(retained_panel_swapchain_, kOpenXRPanelLayerWidth, kOpenXRPanelLayerHeight,
"settings panel")) {
panel_swapchains_ready_ = true;
Log(OpenXRLogLevel::Info, "OpenXR settings panel layer ready");
return true;
}
DestroyPanelSwapchains();
panel_layer_failed_ = true;
Log(OpenXRLogLevel::Warning, "the settings panel could not get its own OpenXR layer; drawing it into the eyes");
return false;
}
void DestroyPanelSwapchains() {
for (auto* swapchain : {&panel_swapchain_, &retained_panel_swapchain_}) {
if (swapchain->handle != XR_NULL_HANDLE && !swapchain->acquired) {
xrDestroySwapchain(swapchain->handle);
} else if (swapchain->acquired) {
Log(OpenXRLogLevel::Warning,
"Vulkan panel swapchain still owns an acquired image; deferring its destruction to xrDestroySession");
}
*swapchain = {};
}
panel_swapchains_ready_ = false;
retained_panel_valid_ = false;
}
void DestroySwapchains() {
DestroyPanelSwapchains();
DestroySwapchainPair(eye_swapchains_);
DestroySwapchainPair(retained_swapchains_);
have_retained_frame_ = false;
@@ -1027,6 +1102,14 @@ private:
OpenXRWindowsVulkanGraphicsRequirements requirements_{};
std::array<EyeSwapchain, kOpenXREyeCount> eye_swapchains_{};
std::array<EyeSwapchain, kOpenXREyeCount> retained_swapchains_{};
// The settings panel's layer: written like the eyes into panel_swapchain_,
// shown from retained_panel_swapchain_ (see FinishFrame).
EyeSwapchain panel_swapchain_{};
EyeSwapchain retained_panel_swapchain_{};
bool panel_swapchains_ready_ = false;
bool panel_layer_failed_ = false;
// The retained frame rendered the panel's image into retained_panel_swapchain_.
bool retained_panel_valid_ = false;
OpenXRWindowsVulkanFrame retained_frame_{};
uint64_t retained_session_serial_ = 0;
uint64_t retained_space_serial_ = 0;
@@ -1115,6 +1198,8 @@ bool OpenXRWindowsVulkanBackend::Shutdown() { return m_impl->Shutdown(); }
bool OpenXRWindowsVulkanBackend::IsBound() const { return m_impl->IsBound(); }
bool OpenXRWindowsVulkanBackend::PanelLayerAvailable() const { return m_impl->PanelLayerAvailable(); }
const OpenXRWindowsVulkanGraphicsRequirements& OpenXRWindowsVulkanBackend::GraphicsRequirements() const {
return m_impl->GraphicsRequirements();
}
runtime/tests/openxr_d3d12_replay_tests.cpp
+112 -3
View File
@@ -15,6 +15,7 @@
#define OpenXRD3D12FrameMode OpenXRWindowsVulkanFrameMode
#define aurora_d3d12_enable_stereo_bridge aurora_vulkan_win32_enable
#define aurora_d3d12_set_stereo_targets aurora_vulkan_win32_set_targets
#define aurora_d3d12_set_stereo_targets_with_panel aurora_vulkan_win32_set_targets_with_panel
#define aurora_d3d12_cancel_stereo_targets aurora_vulkan_win32_cancel
#define aurora_d3d12_disable_stereo_bridge aurora_vulkan_win32_disable
#else
@@ -52,6 +53,8 @@ struct Swapchain {
bool released = false;
};
std::vector<AuroraD3D12StereoTarget> targets;
AuroraD3D12StereoTarget panel_target{};
bool has_panel_target = false;
AuroraD3D12StereoSubmittedCallback callback = nullptr;
void* callback_data = nullptr;
uint64_t pending_token = 0;
@@ -71,6 +74,9 @@ uint32_t live_swapchains = 0;
XrTime display_time = 0;
XrStructureType layer_type = XR_TYPE_UNKNOWN;
XrPosef quad_pose{};
uint64_t panel_content = 0; // what the settings panel's layer showed, 0 without one
XrPosef panel_pose{};
XrExtent2Df panel_size{};
void Complete(bool success = true) {
Require(pending_token != 0);
@@ -78,6 +84,8 @@ void Complete(bool success = true) {
if (success) {
for (const auto& target : targets)
reinterpret_cast<Image*>(target.resource)->content = pending_token;
if (has_panel_target)
reinterpret_cast<Image*>(panel_target.resource)->content = pending_token;
}
callback(pending_token, success, callback_data);
pending_token = 0;
@@ -151,13 +159,19 @@ bool aurora_d3d12_enable_stereo_bridge(AuroraD3D12StereoSubmittedCallback cb, vo
callback_data = data;
return true;
}
bool aurora_d3d12_set_stereo_targets(uint64_t token, const AuroraD3D12StereoTarget* data,
uint32_t count) {
bool aurora_d3d12_set_stereo_targets_with_panel(uint64_t token, const AuroraD3D12StereoTarget* data,
uint32_t count, const AuroraD3D12StereoTarget* panel) {
Require(pending_token == 0);
pending_token = token;
targets.assign(data, data + count);
has_panel_target = panel != nullptr;
panel_target = panel ? *panel : AuroraD3D12StereoTarget{};
return true;
}
bool aurora_d3d12_set_stereo_targets(uint64_t token, const AuroraD3D12StereoTarget* data,
uint32_t count) {
return aurora_d3d12_set_stereo_targets_with_panel(token, data, count, nullptr);
}
bool aurora_d3d12_cancel_stereo_targets(uint64_t token) {
if (encoded || token != pending_token) return false;
pending_token = 0;
@@ -301,8 +315,9 @@ bool OpenXRRuntime::EndFrame(const OpenXRFrame& frame,
Require(frame.predicted_display_time > display_time);
display_time = frame.predicted_display_time;
layer_count = count;
panel_content = 0;
if (count) {
Require(frame.should_render && count == 1);
Require(frame.should_render && count <= 2);
layer_type = layers[0]->type;
const auto check_image = [](const XrSwapchainSubImage& subimage) {
const auto& chain = *reinterpret_cast<Swapchain*>(subimage.swapchain);
@@ -325,6 +340,17 @@ bool OpenXRRuntime::EndFrame(const OpenXRFrame& frame,
displayed_content = check_image(quad.subImage);
quad_pose = quad.pose;
}
if (count == 2) {
// The settings panel, over the scene.
Require(layers[1]->type == XR_TYPE_COMPOSITION_LAYER_QUAD);
const auto& panel = *reinterpret_cast<const XrCompositionLayerQuad*>(layers[1]);
Require(panel.layerFlags == XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT);
Require(panel.subImage.imageRect.extent.width == static_cast<int32_t>(kOpenXRPanelLayerWidth));
Require(panel.subImage.imageRect.extent.height == static_cast<int32_t>(kOpenXRPanelLayerHeight));
panel_content = check_image(panel.subImage);
panel_pose = panel.pose;
panel_size = panel.size;
}
}
m_frame_phase = FramePhase::Idle;
compositor_open = false;
@@ -473,8 +499,91 @@ void TestRenderFirst() {
display_time = 0;
}
// The settings panel's quad layer: made when it first opens, rendered with the
// eyes, and shown from the image the last submitted frame wrote, never from one
// a cancelled frame released unwritten.
void TestPanelLayer() {
display_time = 0;
OpenXRRuntime runtime;
OpenXRD3D12Backend backend;
Require(backend.QueryGraphicsRequirements(runtime) && backend.BindAurora(runtime));
Require(live_swapchains == 4 && backend.PanelLayerAvailable()); // Nothing is made while it is closed.
OpenXRPresentation presentation;
OpenXRBackendFrame frame;
const auto begin = [&] {
Require(backend.BeginFrame(presentation, frame) == OpenXRBeginStatus::Ready);
};
const auto finish = [&] {
Complete();
Require(backend.WaitForSubmission(frame, 0) == OpenXRSubmissionStatus::Success);
Require(backend.FinishFrame(frame, true));
};
begin();
Require(!has_panel_target);
finish();
Require(layer_count == 1);
presentation.panel.requested = true;
presentation.panel.placed = true;
presentation.panel.pose.position.z = -2;
presentation.panel.width_meters = 1.0f;
presentation.panel.height_meters = 0.75f;
begin();
Require(has_panel_target && live_swapchains == 6);
Require(panel_target.width == kOpenXRPanelLayerWidth && panel_target.height == kOpenXRPanelLayerHeight);
finish();
const auto shown = frame.xr_frame.serial;
Require(layer_count == 2 && displayed_content == shown && panel_content == shown);
Require(panel_pose.position.z == -2 && panel_size.width == 1.0f && panel_size.height == 0.75f);
begin(); // Canceled: its panel image is released unwritten.
Require(has_panel_target);
Require(backend.RepeatFrame(frame) && layer_count == 2 && panel_content == shown);
Require(backend.TryCancelPendingFrame(frame) && backend.FinishFrame(frame, false));
Require(layer_count == 2 && panel_content == shown);
presentation.panel.placed = false; // No head pose to hang it from: rendered, not shown.
begin();
finish();
Require(layer_count == 1 && displayed_content == frame.xr_frame.serial);
presentation.panel.placed = true;
begin();
finish();
Require(layer_count == 2 && panel_content == frame.xr_frame.serial);
presentation.panel.requested = false; // Closed: neither rendered nor shown, even when repeated.
begin();
Require(!has_panel_target);
finish();
Require(layer_count == 1);
begin();
Require(backend.RepeatFrame(frame) && layer_count == 1);
Require(backend.TryCancelPendingFrame(frame) && backend.FinishFrame(frame, false));
Require(layer_count == 1);
presentation.panel.requested = true; // Reopened on the same swapchains.
begin();
finish();
Require(layer_count == 2 && live_swapchains == 6 && panel_content == frame.xr_frame.serial);
expect_render_first = true;
OpenXRBackendFrame packet;
Require(backend.PreparePacket(presentation, packet) == OpenXRBeginStatus::Ready);
Require(has_panel_target && packet.presentation.panel.requested);
Complete();
Require(backend.WaitForSubmission(packet, 0) == OpenXRSubmissionStatus::Success);
Require(backend.BeginFrameForPacket(packet, frame) == OpenXRBeginStatus::Ready);
Require(backend.CopyRenderedEyes(frame) == OpenXRSubmissionStatus::Success);
Require(backend.FinishFrame(frame, true));
Require(layer_count == 2 && panel_content == packet.xr_frame.serial);
expect_render_first = false;
Require(backend.Shutdown() && live_swapchains == 0);
display_time = 0;
}
int main() {
TestRenderFirst();
TestPanelLayer();
OpenXRRuntime runtime;
OpenXRD3D12Backend backend;
Require(backend.QueryGraphicsRequirements(runtime) && backend.BindAurora(runtime));