diff --git a/OPENXR.md b/OPENXR.md index 1780aa6..b279af4 100644 --- a/OPENXR.md +++ b/OPENXR.md @@ -46,6 +46,7 @@ hud_distance_meters = 2.0 hud_width_meters = 2.4 hud_virtual_screen = true flat_screen = false +immersive_window = false stop_at_display_copy = true skip_copy_clears = true single_pass_eyes = true @@ -146,14 +147,20 @@ immersive stereo: the whole race, 3D world and HUD alike, is the game's own pict in DolphinXR's Flat Screen mode. The first-person camera, hand steering, the lean-back angle, VR frame interpolation and `hud_virtual_screen` shape only the immersive race view, so none of them apply while it is on; the right-thumbstick first-person toggle is ignored rather than changing the saved -setting. It is live, as **F10 → VR → Flat Screen mode** (the headset panel's VR tab) and the Quest -launcher's Settings page, and turning it on or off mid-race switches on the next frame through the -presentation policy's safety generation. +setting. It is live, as **F10 → VR → Race view → Flat screen** (the headset panel's VR tab) and the +Quest launcher's Settings page, and turning it on or off mid-race switches on the next frame through +the presentation policy's safety generation. +`immersive_window` (default off) is the third race view, between the two: the race keeps its +immersive stereo view, head tracking and all, but is seen only through a window, with the room +around it on the Quest; see [The immersive window](#the-immersive-window). `flat_screen` wins when +both are set. The settings present the three as one choice, **Race view**: Immersive, Immersive +window or Flat screen. `passthrough` (Quest only, default on) shows the room through the headset's cameras around the menu screen and every other virtual screen, instead of black: an `XR_FB_passthrough` reconstruction layer submitted under the screen's quad, as PPSSPP VR does, with the blend mode -left `OPAQUE`. An immersive race never shows it, and the cameras are paused for the race; a race in -`flat_screen` is a virtual screen like the menus, so the room shows around it too. It is +left `OPAQUE`. A fully immersive race never shows it, and the cameras are paused for the race; a +race in `flat_screen` is a virtual screen like the menus, so the room shows around it too, and so it +does around the immersive window. It is live, from the headset panel's VR tab or the launcher's Settings page. The app declares `com.oculus.feature.PASSTHROUGH`, without which Horizon OS composites nothing for that layer. So that the room frames the picture rather than black bands, the Quest's menu quad shows only the @@ -529,6 +536,8 @@ The runtime deliberately fails safe instead of guessing which Mario Kart camera however complete the observations are. Changing it advances the safety generation like any other change of presentation, and the pacing thread still treats that race as a race: pipeline caches are not stored mid-race on the virtual screen either. +- `immersive_window` is not a policy state: the race is `ImmersiveRace` either way, and only the + packet and the layer that shows it carry the window. Aurora records the original GX frame once and replays it for both OpenXR eyes. Perspective GX draws receive asymmetric headset projections, while the game's 2D layer goes on a fixed virtual screen @@ -619,6 +628,41 @@ Kart Wii's minimap are treated as game art and remain eligible for the screen. A uses the full eye viewport and scissor because its recorded rectangle no longer describes where it ended up; its original viewport is folded into the projection instead. +## The immersive window + +`immersive_window` shows the immersive race through a window rather than all around you. The window +is the race's 2D-layer screen: `hud_width_meters` across and `hud_distance_meters` ahead of the +race origin latched at the race start, turned by the lean-back angle, its height following the +picture's aspect. That is where the HUD, the Wii Remote pointer and the settings panel already sit +in an immersive race, so the HUD lies on the window's plane and the pointer aims at it. The window +always carries the 2D layer, whatever `hud_virtual_screen` says: stretched across the eye, the HUD +would be cut by the window's edges. It sits straight ahead of the race's forward direction, as the +HUD does, which is the reference space's forward rather than the heading the menu screen was +anchored at, so after an in-game recenter while facing sideways the two can differ. + +Inside the window nothing changes: the eyes are the immersive race's, with their per-eye frusta, +head tracking, first person, hand steering and VR frame interpolation. Moving your head therefore +shifts the view through the window like a real window's, and geometry nearer than the window's +plane is still cut by its edges, as a stereo picture's frame cuts it. + +How it is drawn: the pacing thread marks the stereo packet (`AuroraStereoFrame::window`), and after an +eye's last draw Aurora covers the eye with one full-screen triangle (`aurora-main/lib/gfx/window_mask.hpp`) +that keeps the colour inside the window with alpha 1 and leaves transparent black outside it, with a +one-pixel ramp at the edge. The triangle carries, at each corner, where that pixel's ray meets the +window's plane in homogeneous window coordinates (`stereo_replay::window_mask`), which interpolate +exactly across the image. With `single_pass_eyes` it is drawn in the eye's own last render pass, so it +adds no pass and no tile load; otherwise it takes a pass of its own, as the cockpit overlay does. The +game still renders the whole eye: only its alpha changes. On the Quest the backend submits the +passthrough layer, then the projection layer with `XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT` +(premultiplied alpha), then the settings panel. The flag travels with the packet, so the eyes Aurora +masked and the layer that blends them always belong to the same frame, and switching the race view +mid-race needs no safety generation: presentation stays `ImmersiveRace`. The PC backends keep their +projection layer opaque, so the window is surrounded by black there. + +`gx_fifo_tests` covers the window's geometry (its corners through an asymmetric eye frustum, its +agreement with the HUD's placement, an eye turned away or beyond the window, a sideways step), and +`mkw_vr_config_tests` how the two keys read as one race view. + ## Foveated rendering On the Quest, `foveation` shades the edges of the immersive race view in 2x2, then 4x4 pixel @@ -892,6 +936,9 @@ ends, including mid-frame flushes, so live setting changes cannot invalidate pen analog grips (Touch); the simple controller profile cannot grab. - The headset settings panel has no laser beam, only the cursor on the panel itself, and text fields cannot be typed into without a keyboard. +- The immersive window renders the whole eye and only masks it, so it costs what a fully immersive + race costs, plus the passthrough's compositing. Hands and a separate VR wheel are masked with the + rest of the eye, so outside the window they are not seen. - The desktop window remains available as a mirror/fallback. OpenXR diagnostics are written to the normal run log under diff --git a/android/app/src/main/java/org/wiicompiled/quest/launcher/SettingsPage.kt b/android/app/src/main/java/org/wiicompiled/quest/launcher/SettingsPage.kt index 2a3b899..9ea09e3 100644 --- a/android/app/src/main/java/org/wiicompiled/quest/launcher/SettingsPage.kt +++ b/android/app/src/main/java/org/wiicompiled/quest/launcher/SettingsPage.kt @@ -116,14 +116,28 @@ class SettingsPage( private fun buildVr() { // Flat Screen mode keeps races on the menu screen, which none of the race view rows reach. val immersive = { c: TomlConfig -> !(c.bool("vr", "flat_screen") ?: false) } + // The immersive window is still the stereo race view, seen through the race HUD's screen. + val window = { c: TomlConfig -> immersive(c) && (c.bool("vr", "immersive_window") ?: false) } val firstPerson = { c: TomlConfig -> immersive(c) && (c.bool("vr", "first_person") ?: false) } // The steering wheel and hand steering belong to the cockpit seat. val cockpit = { c: TomlConfig -> firstPerson(c) && stringIndex(c, "vr", "first_person_seat", SEATS) == 0 } section(R.string.section_vr_camera) { - toggle( - R.string.vr_flat_screen, R.string.vr_flat_screen_helper, - read = { !immersive(it) }, - write = { c, value -> c.setBool("vr", "flat_screen", value) }, + // One setting in two keys, read as runtime_config.h's VrRaceView reads them: Flat + // Screen mode wins over the immersive window. + choice( + R.string.vr_race_view, R.string.vr_race_view_helper, + listOf(R.string.vr_race_view_immersive, R.string.vr_race_view_window, R.string.vr_race_view_flat), + read = { + when { + !immersive(it) -> 2 + window(it) -> 1 + else -> 0 + } + }, + write = { c, index -> + c.setBool("vr", "flat_screen", index == 2) + c.setBool("vr", "immersive_window", index == 1) + }, ) choice( R.string.vr_camera, R.string.vr_camera_helper, @@ -217,11 +231,12 @@ class SettingsPage( ) } section(R.string.section_vr_screen) { + // The immersive window is that screen and always carries the HUD. toggle( R.string.vr_hud_screen, R.string.vr_hud_screen_helper, read = { it.bool("vr", "hud_virtual_screen") ?: true }, write = { c, value -> c.setBool("vr", "hud_virtual_screen", value) }, - enabledIf = immersive, + enabledIf = { immersive(it) && !window(it) }, ) slider( R.string.vr_hud_distance, R.string.vr_hud_distance_helper, 0.5, 5.0, 0.1, diff --git a/android/app/src/main/res/values/strings.xml b/android/app/src/main/res/values/strings.xml index 0643c13..e38df9a 100644 --- a/android/app/src/main/res/values/strings.xml +++ b/android/app/src/main/res/values/strings.xml @@ -508,8 +508,11 @@ Camera - Flat Screen mode - Plays races on the same flat screen as the menus instead of all around you. The camera settings below do not apply while it is on. + Race view + Immersive plays races all around you in stereo. Immersive window keeps that stereo view but shows it only through a window where the menu screen sits, with your room around it. Flat screen plays races on the menu screen; the camera settings below do not apply to it. + Immersive + Immersive window + Flat screen Camera Ride behind the kart like the game, or sit in the driver\'s seat. Chase camera @@ -562,7 +565,7 @@ Screen width How wide the menu screen and race HUD are. Passthrough around the menu screen - Shows your room through the headset\'s cameras around the menus instead of black. Races stay fully virtual, except in Flat Screen mode. + Shows your room through the headset\'s cameras around the menus instead of black. Immersive races stay fully virtual; the immersive window and the flat screen race have the room around them too. Rendering diff --git a/aurora-main/include/aurora/aurora.h b/aurora-main/include/aurora/aurora.h index 6b77cde..c3441f0 100644 --- a/aurora-main/include/aurora/aurora.h +++ b/aurora-main/include/aurora/aurora.h @@ -180,6 +180,13 @@ typedef struct { uint64_t displayTimeNanos; // Optional; inactive when zero-initialised. AuroraCockpit cockpit; + // Immersive replay only: shows the race through a window rather than all + // around. Each eye keeps what it sees through the 2D layer's screen + // (aurora_set_stereo_hud_screen's rectangle, on which the 2D layer is then + // always placed), with premultiplied alpha 1 there, and is transparent black + // everywhere else, for the host's compositor to show its own background + // (the room, on a headset with passthrough) around it. + bool window; } AuroraStereoFrame; /** diff --git a/aurora-main/lib/aurora.cpp b/aurora-main/lib/aurora.cpp index d9d2f2d..38fe083 100644 --- a/aurora-main/lib/aurora.cpp +++ b/aurora-main/lib/aurora.cpp @@ -815,6 +815,7 @@ gfx::StereoReplayFrame make_stereo_replay_frame(const AuroraStereoFrame& input, std::memcpy(&anchorFromScene, sceneAnchor.anchorFromScene.data(), sizeof(anchorFromScene)); gfx::StereoReplayFrame replay{}; replay.cockpit = input.cockpit; + replay.window = input.mode == AURORA_STEREO_FRAME_IMMERSIVE_REPLAY && input.window; // The sealed guest frame owns its scale. The packet may have been sampled // just before a change of scale (a character swap, a lightning strike), so // only its head/IPD translation is rescaled to the frame's. @@ -2216,6 +2217,10 @@ std::vector encode_sealed_frame(gfx::SealedFrame& sealedFrame, encode_virtual_screen_eye(encoder, completedMono, eye); const auto& output = g_stereoEyeTargets[eye].output(); stereo_overlay::composite_flat(encoder, output.view, output.size, eye); + // An immersive packet that could not be replayed still goes out as a windowed layer. + if (ctx.stereoReplay->window) { + gfx::mask_stereo_eye_output(sealedFrame, encoder, *ctx.stereoReplay, eye, output.view, output.size); + } } if (mirrorPlan == MirrorPlan::Black && !headsetOnly) { encode_presentation_snapshot(encoder, ctx.presentSource, *finalImage, true, MirrorPlan::Black, diff --git a/aurora-main/lib/gfx/common.cpp b/aurora-main/lib/gfx/common.cpp index c2bf36e..43dfbb0 100644 --- a/aurora-main/lib/gfx/common.cpp +++ b/aurora-main/lib/gfx/common.cpp @@ -11,6 +11,7 @@ #include "pipeline_cache.hpp" #include "stereo_replay.hpp" #include "cockpit.hpp" +#include "window_mask.hpp" #include "tex_copy_conv.hpp" #include "tex_palette_conv.hpp" #include "texture_replacement.hpp" @@ -355,6 +356,8 @@ struct SealedFrameData { std::vector passes; LateStereoData stereo; uint32_t localPlayerCount = 1; + // The immersive window's screen as this frame's 2D layer was placed on it. + stereo_replay::HudScreen windowScreen{}; }; SealedFrame::SealedFrame() : m_data(std::make_unique()) {} @@ -1083,6 +1086,7 @@ void initialize() { void shutdown() { cockpit::shutdown(); + window_mask::shutdown(); shutdown_pipeline_cache(); gx::clear_shader_module_cache(); efb_ram::shutdown(); @@ -1331,11 +1335,9 @@ static void log_stereo_display_source_region(ClipRect region, bool foundDisplayC // ratio the game is currently presenting at: 4:3 while VILockAspectRatio holds // it there, otherwise the mirror window's own aspect, which is what Mario Kart // Wii's dynamic widescreen builds its projections from. Matching it keeps the -// HUD unstretched on the screen. -static stereo_replay::HudScreen stereo_hud_screen() noexcept { - if (!g_stereoHudScreenEnabled.load(std::memory_order_relaxed)) { - return {}; - } +// HUD unstretched on the screen. The immersive window is this same screen, and +// has it whether or not the 2D layer is set to go on it. +static stereo_replay::HudScreen stereo_window_screen() noexcept { const float width = g_stereoHudScreenWidth.load(std::memory_order_relaxed); const float distance = g_stereoHudScreenDistance.load(std::memory_order_relaxed); float aspect = 0.f; @@ -1350,6 +1352,13 @@ static stereo_replay::HudScreen stereo_hud_screen() noexcept { }; } +static stereo_replay::HudScreen stereo_hud_screen() noexcept { + if (!g_stereoHudScreenEnabled.load(std::memory_order_relaxed)) { + return {}; + } + return stereo_window_screen(); +} + // Shared by the normal seal and headset-deadline replay. Head transforms are // composed after scene interpolation, so free look never inherits its delay. static void write_stereo_uniform(std::span uniform, const gx::UniformReplayLayout& layout, @@ -1441,7 +1450,9 @@ static bool prepare_stereo_replay_uniforms(const StereoReplayFrame& stereoFrame, // This is the producer-side preparation path; eye replay can query the pure // helper concurrently without touching this diagnostic state. log_stereo_display_source_region(displayRegion, displaySource.foundDisplayCopy); - const stereo_replay::HudScreen hudScreen = stereo_hud_screen(); + // The immersive window always carries the 2D layer: stretched across the eye, + // it would be cut by the window's edges. + const stereo_replay::HudScreen hudScreen = stereoFrame.window ? stereo_window_screen() : stereo_hud_screen(); // A draw is replayed per eye when it carries the game camera (perspective) or // when it is 2D content the virtual screen is claiming. const auto replayed = [&](const gx::UniformReplayLayout& layout) noexcept { @@ -1782,6 +1793,11 @@ struct RenderInvocation { bool* sceneDrawn = nullptr; // An eye replay laid out by eye_pass_plan, replacing the one-render-pass-per-recorded-pass loop. const eye_pass_plan::Plan* eyePlan = nullptr; + // The immersive window's mask, drawn by a planned eye at the end of its last render pass + // (render_stereo_eye draws it otherwise). + const StereoReplayFrame* windowFrame = nullptr; + stereo_replay::HudScreen windowScreen{}; + bool* windowDrawn = nullptr; }; static void render_pass_impl(const wgpu::RenderPassEncoder& pass, const std::vector& passes, u32 idx, @@ -1859,6 +1875,11 @@ static void render_eye_planned(std::vector& renderPasses, wgpu::Comm invocation.cockpitDepth, &pass); *invocation.cockpitDrawn = true; } + // Last of all, over everything the eye draws, in the same render pass. + if (invocation.windowFrame != nullptr && !*invocation.windowDrawn) { + window_mask::draw(pass, *invocation.windowFrame, invocation.stereoEye, invocation.windowScreen); + *invocation.windowDrawn = true; + } pass.End(); } @@ -2061,6 +2082,7 @@ void seal_frame(SealedFrame& out) noexcept { recycle_render_passes(passes); passes.swap(g_renderPasses); g_currentRenderPass = UINT32_MAX; + out.data().windowScreen = stereo_window_screen(); } void render(SealedFrame& frame, wgpu::CommandEncoder& cmd, int32_t interpolatedFrame, bool finalize, @@ -2211,6 +2233,7 @@ void render_stereo_eye(SealedFrame& frame, wgpu::CommandEncoder& cmd, const Ster } bool cockpitDrawn = false; bool sceneDrawn = false; + bool windowDrawn = false; const bool cockpitActive = stereoFrame.cockpit.active && cockpitDepth.valid; const bool skipCopyClears = get_stereo_skip_copy_clears(); const eye_pass_plan::Plan* plan = @@ -2234,6 +2257,9 @@ void render_stereo_eye(SealedFrame& frame, wgpu::CommandEncoder& cmd, const Ster .cockpitDrawn = &cockpitDrawn, .sceneDrawn = &sceneDrawn, .eyePlan = plan, + .windowFrame = stereoFrame.window ? &stereoFrame : nullptr, + .windowScreen = frame.data().windowScreen, + .windowDrawn = &windowDrawn, }); // A frame without a virtual-screen draw after its world still gets the // overlay, in a pass of its own over the finished eye. A planned eye draws it @@ -2241,6 +2267,16 @@ void render_stereo_eye(SealedFrame& frame, wgpu::CommandEncoder& cmd, const Ster if (cockpitActive && !cockpitDrawn) { cockpit::render(cmd, stereoFrame, eye, cockpitDepth); } + // Likewise the window's mask, which has to come after the cockpit. + if (stereoFrame.window && !windowDrawn) { + window_mask::render(cmd, stereoFrame, eye, frame.data().windowScreen); + } +} + +void mask_stereo_eye_output(const SealedFrame& frame, wgpu::CommandEncoder& cmd, const StereoReplayFrame& stereoFrame, + uint32_t eye, const wgpu::TextureView& output, wgpu::Extent3D size) { + CHECK(eye < AURORA_STEREO_EYE_COUNT, "invalid stereo eye {}", eye); + window_mask::render_output(cmd, stereoFrame, eye, output, size, frame.data().windowScreen); } void render(wgpu::CommandEncoder& cmd, int32_t interpolatedFrame, bool finalize) { diff --git a/aurora-main/lib/gfx/common.hpp b/aurora-main/lib/gfx/common.hpp index 7f9a8c8..9e9d808 100644 --- a/aurora-main/lib/gfx/common.hpp +++ b/aurora-main/lib/gfx/common.hpp @@ -321,6 +321,9 @@ struct StereoReplayFrame { std::array eyes; // VR hands and synthetic wheel, drawn per eye after the world (gfx/cockpit.hpp). AuroraCockpit cockpit{}; + // The immersive window (AuroraStereoFrame::window): each eye is masked to the + // 2D layer's screen after its last draw (gfx/window_mask.hpp). + bool window = false; }; void end_frame(const wgpu::CommandEncoder& cmd); @@ -380,6 +383,12 @@ int32_t last_pass_feeding_replay(const SealedFrame& frame) noexcept; // mutated by stereo replay. void render_stereo_eye(SealedFrame& frame, wgpu::CommandEncoder& cmd, const StereoReplayFrame& stereoFrame, uint32_t eye, bool finalize = false); +// The immersive window's mask on an eye image that holds the duplicated mono +// picture instead of a replay (a windowed frame whose stereo replay could not +// be prepared), so the compositor never blends an undefined alpha channel. +void mask_stereo_eye_output(const SealedFrame& frame, wgpu::CommandEncoder& cmd, + const StereoReplayFrame& stereoFrame, uint32_t eye, + const wgpu::TextureView& output, wgpu::Extent3D size); // Encode the frame that is still being recorded. Only for the synchronous // EFB-readback split path, which runs on the producer thread. diff --git a/aurora-main/lib/gfx/stereo_replay.hpp b/aurora-main/lib/gfx/stereo_replay.hpp index 45a97c8..84e7662 100644 --- a/aurora-main/lib/gfx/stereo_replay.hpp +++ b/aurora-main/lib/gfx/stereo_replay.hpp @@ -1,6 +1,7 @@ #pragma once #include +#include #include namespace aurora::gfx::stereo_replay { @@ -364,4 +365,108 @@ inline Mat4x4 overlay_panel_flat_projection(float widthFraction, float pa return out; } +// The immersive window (AuroraStereoFrame::window): the 2D layer's screen as an +// opening each eye sees the race through, the rest of the eye left transparent. +// +// Each row, applied to (x, y, 1) for a point of the eye image at NDC (x, y), +// gives one of the homogeneous coordinates (u, v, w) of where that pixel's ray +// meets the screen's plane: the point (u / w, v / w), in units of the screen's +// half extents, so the screen covers -1..1 on both axes, lying in front of the +// eye exactly when w > 0. The rows are linear in NDC, so a full-screen triangle +// carrying their values at its corners interpolates them exactly. An eye on or +// behind the screen's plane gets all-zero rows and sees nothing through it. +struct WindowMask { + Vec3 u; + Vec3 v; + Vec3 w; + + [[nodiscard]] Vec3 at(float x, float y) const noexcept { + return {u.x * x + u.y * y + u.z, v.x * x + v.y * y + v.z, w.x * x + w.y * y + w.z}; + } +}; + +// The screen is the one compose_hud_screen_projection places the 2D layer on: +// halfWidth by halfHeight, `distance` straight ahead in the recorded center-eye +// view space, reached through viewFromCenter and the eye frustum's four terms. +inline WindowMask window_mask(const Mat4x4& eyeFrustum, const Mat3x4& viewFromCenter, + const HudScreen& screen) noexcept { + const float sx = eyeFrustum.m0[0]; + const float sy = eyeFrustum.m1[1]; + if (!screen.valid() || sx == 0.0f || sy == 0.0f) { + return {}; + } + // The inverse of viewFromCenter's linear part L, by its adjugate. + const auto& r0 = viewFromCenter.m0; + const auto& r1 = viewFromCenter.m1; + const auto& r2 = viewFromCenter.m2; + float inverse[3][3] = { + {r1[1] * r2[2] - r1[2] * r2[1], r0[2] * r2[1] - r0[1] * r2[2], r0[1] * r1[2] - r0[2] * r1[1]}, + {r1[2] * r2[0] - r1[0] * r2[2], r0[0] * r2[2] - r0[2] * r2[0], r0[2] * r1[0] - r0[0] * r1[2]}, + {r1[0] * r2[1] - r1[1] * r2[0], r0[1] * r2[0] - r0[0] * r2[1], r0[0] * r1[1] - r0[1] * r1[0]}, + }; + const float determinant = r0[0] * inverse[0][0] + r0[1] * inverse[1][0] + r0[2] * inverse[2][0]; + if (determinant == 0.0f) { + return {}; + } + for (auto& row : inverse) { + for (float& value : row) { + value /= determinant; + } + } + // Window coordinates of an eye-space point p are q = A p + b: back into the + // center-eye space, moved to the screen's centre and divided by its half + // extents. The screen's plane is q.z = 0, its front facing the camera. + const float scale[3] = {1.0f / screen.halfWidth, 1.0f / screen.halfHeight, 1.0f}; + const float t[3] = {r0[3], r1[3], r2[3]}; + float A[3][3]; + float b[3]; + for (int row = 0; row < 3; ++row) { + float back = 0.0f; + for (int column = 0; column < 3; ++column) { + A[row][column] = inverse[row][column] * scale[row]; + back += inverse[row][column] * t[column]; + } + b[row] = (-back + (row == 2 ? screen.distance : 0.0f)) * scale[row]; + } + if (!(b[2] > 0.0f)) { + return {}; + } + // The pixel's ray is d = K (x, y, 1) with d.z = -1: the frustum maps an eye + // point to clip x = sx * x + m0[2] * z, y = sy * y + m1[2] * z, w = -z. + const float K[3][3] = { + {1.0f / sx, 0.0f, eyeFrustum.m0[2] / sx}, + {0.0f, 1.0f / sy, eyeFrustum.m1[2] / sy}, + {0.0f, 0.0f, -1.0f}, + }; + // a = M (x, y, 1) is the ray in window coordinates. It meets the plane at + // q = b + s a with s = -b.z / a.z, in front of the eye when s > 0, i.e. when + // a.z < 0, so (u, v, w) = (b.z a.x - b.x a.z, b.z a.y - b.y a.z, -a.z). + float M[3][3]; + for (int row = 0; row < 3; ++row) { + for (int column = 0; column < 3; ++column) { + M[row][column] = A[row][0] * K[0][column] + A[row][1] * K[1][column] + A[row][2] * K[2][column]; + } + } + float rows[3][3]; + float largest = 0.0f; + for (int column = 0; column < 3; ++column) { + rows[0][column] = b[2] * M[0][column] - b[0] * M[2][column]; + rows[1][column] = b[2] * M[1][column] - b[1] * M[2][column]; + rows[2][column] = -M[2][column]; + for (const auto& row : rows) { + largest = std::max(largest, std::abs(row[column])); + } + } + if (!(largest > 0.0f)) { + return {}; + } + // Only the ratios matter; a positive scale keeps the values near one whatever + // the world units are. + const float normalize = 1.0f / largest; + const auto out = [&](const float (&row)[3]) { + return Vec3{row[0] * normalize, row[1] * normalize, row[2] * normalize}; + }; + return {.u = out(rows[0]), .v = out(rows[1]), .w = out(rows[2])}; +} + } // namespace aurora::gfx::stereo_replay diff --git a/aurora-main/lib/gfx/window_mask.hpp b/aurora-main/lib/gfx/window_mask.hpp new file mode 100644 index 0000000..6cce708 --- /dev/null +++ b/aurora-main/lib/gfx/window_mask.hpp @@ -0,0 +1,208 @@ +// SPDX-License-Identifier: GPL-3.0-or-later +// +// The immersive window (AuroraStereoFrame::window): after an eye's last draw, +// one full-screen triangle keeps what the eye sees through the 2D layer's +// screen and makes the rest transparent black, so the compositor shows its own +// background (the room, with passthrough) around the race. See OPENXR.md, "The +// immersive window". +#pragma once + +#include "common.hpp" +#include "stereo_replay.hpp" +#include "../webgpu/gpu.hpp" + +#include +#include +#include + +namespace aurora::gfx::window_mask { + +struct Vertex { + float position[2]; + // stereo_replay::WindowMask's (u, v, w) at this corner. + float window[3]; +}; + +struct PipelineKey { + wgpu::TextureFormat format = wgpu::TextureFormat::Undefined; + // Undefined: a pass with no depth attachment. + wgpu::TextureFormat depthFormat = wgpu::TextureFormat::Undefined; + uint32_t samples = 0; + + bool operator==(const PipelineKey&) const = default; +}; + +// The eye passes (with depth) use one attachment layout and the mono fallback (without) another; +// each keeps its own pipeline, rebuilt when its format or sample count changes. +inline std::array, 2> pipelines; +inline std::array vertexBuffers; + +inline void shutdown() { + pipelines = {}; + vertexBuffers = {}; +} + +inline const wgpu::RenderPipeline& pipeline(const PipelineKey& key) { + for (const auto& [cachedKey, cached] : pipelines) { + if (cached && cachedKey == key) { + return cached; + } + } + wgpu::ShaderSourceWGSL source{}; + source.code = R"( + struct Out { @builtin(position) position: vec4f, @location(0) window: vec3f }; + @vertex fn vs(@location(0) position: vec2f, @location(1) window: vec3f) -> Out { + var o: Out; + o.position = vec4f(position, 0.5, 1.0); + o.window = window; + return o; + } + @fragment fn fs(i: Out) -> @location(0) vec4f { + // Where this pixel's ray meets the screen's plane, in its half extents. + let p = i.window.xy / i.window.z; + let edge = 1.0 - abs(p); + // One pixel of coverage ramp at the edge, from the screen-space rate of change. + let ramp = clamp(edge / max(fwidth(edge), vec2f(1e-6)) + 0.5, vec2f(0.0), vec2f(1.0)); + // Behind the eye (w <= 0), or far off the screen where the plane nears the horizon. + let seen = i.window.z > 0.0 && all(abs(p) < vec2f(2.0)); + return vec4f(0.0, 0.0, 0.0, select(0.0, ramp.x * ramp.y, seen)); + } + )"; + wgpu::ShaderModuleDescriptor moduleDescriptor{}; + moduleDescriptor.nextInChain = &source; + moduleDescriptor.label = "VR immersive window mask"; + const auto shader = webgpu::g_device.CreateShaderModule(&moduleDescriptor); + const wgpu::VertexAttribute attributes[] = { + {.format = wgpu::VertexFormat::Float32x2, .offset = 0, .shaderLocation = 0}, + {.format = wgpu::VertexFormat::Float32x3, .offset = offsetof(Vertex, window), .shaderLocation = 1}, + }; + const wgpu::VertexBufferLayout layout{.arrayStride = sizeof(Vertex), .attributeCount = 2, .attributes = attributes}; + // Premultiplied alpha: the colour is scaled by the coverage the fragment returns as alpha, and the + // alpha becomes that coverage, whatever the game left there. + const wgpu::BlendState blend{ + .color = {.operation = wgpu::BlendOperation::Add, + .srcFactor = wgpu::BlendFactor::Zero, + .dstFactor = wgpu::BlendFactor::SrcAlpha}, + .alpha = {.operation = wgpu::BlendOperation::Add, + .srcFactor = wgpu::BlendFactor::One, + .dstFactor = wgpu::BlendFactor::Zero}, + }; + const wgpu::ColorTargetState color{.format = key.format, .blend = &blend}; + const wgpu::FragmentState fragment{.module = shader, .entryPoint = "fs", .targetCount = 1, .targets = &color}; + const wgpu::DepthStencilState depth{ + .format = key.depthFormat, + .depthWriteEnabled = false, + .depthCompare = wgpu::CompareFunction::Always, + .stencilReadMask = 0, + .stencilWriteMask = 0, + }; + wgpu::RenderPipelineDescriptor descriptor{}; + descriptor.label = "VR immersive window mask"; + descriptor.vertex = {.module = shader, .entryPoint = "vs", .bufferCount = 1, .buffers = &layout}; + descriptor.fragment = &fragment; + descriptor.depthStencil = key.depthFormat == wgpu::TextureFormat::Undefined ? nullptr : &depth; + descriptor.multisample.count = key.samples; + descriptor.primitive.topology = wgpu::PrimitiveTopology::TriangleList; + auto& slot = key.depthFormat == wgpu::TextureFormat::Undefined ? pipelines[1] : pipelines[0]; + slot = {key, webgpu::g_device.CreateRenderPipeline(&descriptor)}; + return slot.second; +} + +// Writes the eye's triangle: NDC corners (-1, -1), (3, -1) and (-1, 3) cover the whole image, with +// the mask's rows evaluated at each so they interpolate across it exactly. +inline const wgpu::Buffer& eye_vertices(uint32_t eye, const stereo_replay::WindowMask& mask) { + static constexpr std::array, 3> kCorners{{{-1.0f, -1.0f}, {3.0f, -1.0f}, {-1.0f, 3.0f}}}; + std::array vertices{}; + for (size_t i = 0; i < vertices.size(); ++i) { + const auto h = mask.at(kCorners[i][0], kCorners[i][1]); + vertices[i] = {{kCorners[i][0], kCorners[i][1]}, {h.x, h.y, h.z}}; + } + auto& buffer = vertexBuffers[eye]; + if (!buffer) { + const wgpu::BufferDescriptor descriptor{ + .label = "VR immersive window mask vertices", + .usage = wgpu::BufferUsage::Vertex | wgpu::BufferUsage::CopyDst, + .size = sizeof(vertices), + }; + buffer = webgpu::g_device.CreateBuffer(&descriptor); + } + webgpu::g_queue.WriteBuffer(buffer, 0, vertices.data(), sizeof(vertices)); + return buffer; +} + +// Masks one eye inside a render pass already open on its attachments, after everything else it draws. +inline void draw(const wgpu::RenderPassEncoder& pass, const StereoReplayFrame& frame, uint32_t eye, + const stereo_replay::HudScreen& screen) { + const auto& view = frame.eyes[eye]; + const auto& target = view.target; + const PipelineKey key{ + .format = webgpu::g_graphicsConfig.surfaceConfiguration.format, + .depthFormat = target.depthFormat, + .samples = target.msaaSamples, + }; + const auto& buffer = eye_vertices(eye, stereo_replay::window_mask(view.projection, view.viewFromCenter, screen)); + pass.SetViewport(0.0f, 0.0f, static_cast(target.size.width), static_cast(target.size.height), 0.0f, + 1.0f); + pass.SetScissorRect(0, 0, target.size.width, target.size.height); + pass.SetPipeline(pipeline(key)); + pass.SetVertexBuffer(0, buffer); + pass.Draw(3); +} + +// The same in a render pass of its own, over the finished eye's attachments. +inline void render(wgpu::CommandEncoder& cmd, const StereoReplayFrame& frame, uint32_t eye, + const stereo_replay::HudScreen& screen) { + const auto& target = frame.eyes[eye].target; + const bool stencil = target.depthFormat == wgpu::TextureFormat::Depth24PlusStencil8; + const wgpu::RenderPassColorAttachment color{ + .view = target.colorView, + .resolveTarget = target.resolveView, + .loadOp = wgpu::LoadOp::Load, + .storeOp = wgpu::StoreOp::Store, + }; + const wgpu::RenderPassDepthStencilAttachment depth{ + .view = target.depthView, + .depthLoadOp = wgpu::LoadOp::Load, + .depthStoreOp = wgpu::StoreOp::Store, + .stencilLoadOp = stencil ? wgpu::LoadOp::Load : wgpu::LoadOp::Undefined, + .stencilStoreOp = stencil ? wgpu::StoreOp::Store : wgpu::StoreOp::Undefined, + }; + const wgpu::RenderPassDescriptor descriptor{ + .label = "VR immersive window mask", + .colorAttachmentCount = 1, + .colorAttachments = &color, + .depthStencilAttachment = &depth, + }; + const auto pass = cmd.BeginRenderPass(&descriptor); + draw(pass, frame, eye, screen); + pass.End(); +} + +// Masks an eye image that holds the duplicated mono picture (a frame whose stereo replay could not +// be prepared): only the resolved output exists there, with no depth. +inline void render_output(wgpu::CommandEncoder& cmd, const StereoReplayFrame& frame, uint32_t eye, + const wgpu::TextureView& output, wgpu::Extent3D size, + const stereo_replay::HudScreen& screen) { + const auto& view = frame.eyes[eye]; + const PipelineKey key{.format = webgpu::g_graphicsConfig.surfaceConfiguration.format, .samples = 1}; + const auto& buffer = eye_vertices(eye, stereo_replay::window_mask(view.projection, view.viewFromCenter, screen)); + const wgpu::RenderPassColorAttachment color{ + .view = output, + .loadOp = wgpu::LoadOp::Load, + .storeOp = wgpu::StoreOp::Store, + }; + const wgpu::RenderPassDescriptor descriptor{ + .label = "VR immersive window mask", + .colorAttachmentCount = 1, + .colorAttachments = &color, + }; + const auto pass = cmd.BeginRenderPass(&descriptor); + pass.SetViewport(0.0f, 0.0f, static_cast(size.width), static_cast(size.height), 0.0f, 1.0f); + pass.SetScissorRect(0, 0, size.width, size.height); + pass.SetPipeline(pipeline(key)); + pass.SetVertexBuffer(0, buffer); + pass.Draw(3); + pass.End(); +} + +} // namespace aurora::gfx::window_mask diff --git a/aurora-main/tests/CMakeLists.txt b/aurora-main/tests/CMakeLists.txt index 2d85298..0275736 100644 --- a/aurora-main/tests/CMakeLists.txt +++ b/aurora-main/tests/CMakeLists.txt @@ -34,6 +34,12 @@ if (AURORA_GPU_SMOKE_TESTS AND AURORA_ENABLE_GX AND WIN32) target_compile_definitions(cockpit_gpu_smoke PRIVATE AURORA TARGET_PC WEBGPU_DAWN) target_link_libraries(cockpit_gpu_smoke PRIVATE fmt::fmt xxhash absl::flat_hash_map absl::btree dawn::webgpu_dawn dawn::dawncpp_headers TracyClient ${AURORA_SDL3_TARGET}) + # The immersive window's mask over a filled eye, read back. Standalone like the cockpit's. + add_executable(window_mask_gpu_smoke window_mask_gpu_smoke.cpp) + target_include_directories(window_mask_gpu_smoke PRIVATE ../include ../lib) + target_compile_definitions(window_mask_gpu_smoke PRIVATE AURORA TARGET_PC WEBGPU_DAWN) + target_link_libraries(window_mask_gpu_smoke PRIVATE fmt::fmt xxhash absl::flat_hash_map absl::btree + dawn::webgpu_dawn dawn::dawncpp_headers TracyClient ${AURORA_SDL3_TARGET}) endif () if (NOT TARGET gtest) diff --git a/aurora-main/tests/stereo_replay_test.cpp b/aurora-main/tests/stereo_replay_test.cpp index f84c54f..168f295 100644 --- a/aurora-main/tests/stereo_replay_test.cpp +++ b/aurora-main/tests/stereo_replay_test.cpp @@ -511,5 +511,134 @@ TEST(StereoReplayTest, MirroringKeepsEachEyeOnItsOwnSide) { EXPECT_LT(ndcX[1], 0.0f); } +Mat4x4 asymmetric_eye_frustum() { + Mat4x4 eyeFrustum{}; + eyeFrustum.m0 = {1.15f, 0.0f, 0.08f, 0.0f}; + eyeFrustum.m1 = {0.0f, 1.02f, -0.03f, 0.0f}; + return eyeFrustum; +} + +// A head turned and pitched a little, offset from the recorded center eye. +Mat3x4 turned_head() { + const float yaw = 0.3f; + const float pitch = -0.12f; + const float cy = std::cos(yaw); + const float sy = std::sin(yaw); + const float cp = std::cos(pitch); + const float sp = std::sin(pitch); + // Pitch about X after yaw about Y. + Mat3x4 m{}; + m.m0 = {cy, 0.0f, sy, 15.0f}; + m.m1 = {sp * sy, cp, -sp * cy, -4.0f}; + m.m2 = {-cp * sy, sp, cp * cy, 7.0f}; + return m; +} + +// Where a point of the center-eye space lands in the eye image, in NDC. +std::array eye_ndc(const Mat4x4& eyeFrustum, const Mat3x4& viewFromCenter, + const Vec4& centerPoint) { + const float eyeX = dot4(viewFromCenter.m0, centerPoint); + const float eyeY = dot4(viewFromCenter.m1, centerPoint); + const float eyeZ = dot4(viewFromCenter.m2, centerPoint); + EXPECT_LT(eyeZ, 0.0f); + return {(eyeFrustum.m0[0] * eyeX + eyeFrustum.m0[2] * eyeZ) / -eyeZ, + (eyeFrustum.m1[1] * eyeY + eyeFrustum.m1[2] * eyeZ) / -eyeZ}; +} + +TEST(StereoReplayTest, WindowMaskFindsTheScreenThroughTheEye) { + const auto eyeFrustum = asymmetric_eye_frustum(); + const auto viewFromCenter = turned_head(); + const HudScreen screen{.halfWidth = 600.0f, .halfHeight = 337.5f, .distance = 1000.0f}; + const auto mask = window_mask(eyeFrustum, viewFromCenter, screen); + + // Corners, centre, and points just inside and outside the edges, in half extents. + const std::array, 9> points{{ + {-1.0f, 1.0f}, + {1.0f, 1.0f}, + {-1.0f, -1.0f}, + {1.0f, -1.0f}, + {0.0f, 0.0f}, + {0.95f, 0.2f}, + {1.05f, 0.2f}, + {-0.3f, -0.97f}, + {-0.3f, -1.04f}, + }}; + for (const auto& point : points) { + const Vec4 centerPoint{point[0] * screen.halfWidth, point[1] * screen.halfHeight, -screen.distance, 1.0f}; + const auto ndc = eye_ndc(eyeFrustum, viewFromCenter, centerPoint); + const auto h = mask.at(ndc[0], ndc[1]); + ASSERT_GT(h.z, 0.0f); + EXPECT_NEAR(h.x / h.z, point[0], 1e-4f); + EXPECT_NEAR(h.y / h.z, point[1], 1e-4f); + } +} + +TEST(StereoReplayTest, WindowMaskCoincidesWithTheHudOnTheSameScreen) { + // The 2D layer is drawn on the window, so a HUD vertex must land at its own + // game NDC on the window: the mask and compose_hud_screen_projection agree. + const auto game = game_orthographic_projection(); + const auto eyeFrustum = asymmetric_eye_frustum(); + const auto viewFromCenter = turned_head(); + const HudScreen screen{.halfWidth = 600.0f, .halfHeight = 337.5f, .distance = 1000.0f}; + const auto composed = compose_hud_screen_projection(eyeFrustum, viewFromCenter, screen, game); + const auto mask = window_mask(eyeFrustum, viewFromCenter, screen); + for (const auto& v : kVertices) { + const float w = dot4(composed.m3, v); + ASSERT_GT(w, 0.0f); + const auto h = mask.at(dot4(composed.m0, v) / w, dot4(composed.m1, v) / w); + ASSERT_GT(h.z, 0.0f); + EXPECT_NEAR(h.x / h.z, dot4(game.m0, v), 1e-4f); + EXPECT_NEAR(h.y / h.z, dot4(game.m1, v), 1e-4f); + } +} + +TEST(StereoReplayTest, WindowMaskShowsNothingBehindTheEye) { + const auto eyeFrustum = asymmetric_eye_frustum(); + const HudScreen screen{.halfWidth = 600.0f, .halfHeight = 337.5f, .distance = 1000.0f}; + + // Turned right round: the screen is behind the eye, so no pixel's ray meets it. + Mat3x4 turnedAway{}; + turnedAway.m0 = {-1.0f, 0.0f, 0.0f, 0.0f}; + turnedAway.m1 = {0.0f, 1.0f, 0.0f, 0.0f}; + turnedAway.m2 = {0.0f, 0.0f, -1.0f, 0.0f}; + const auto away = window_mask(eyeFrustum, turnedAway, screen); + for (float x = -1.0f; x <= 1.0f; x += 0.25f) { + for (float y = -1.0f; y <= 1.0f; y += 0.25f) { + EXPECT_LE(away.at(x, y).z, 0.0f); + } + } + + // Walked through the screen: every row is zero, so every pixel is outside. + Mat3x4 beyond = identity3x4(); + beyond.m2[3] = screen.distance + 100.0f; + const auto through = window_mask(eyeFrustum, beyond, screen); + EXPECT_EQ(through.w, Vec3{}); + EXPECT_EQ(through.u, Vec3{}); + + // No screen at all: nothing either. + const auto none = window_mask(eyeFrustum, identity3x4(), HudScreen{}); + EXPECT_EQ(none.w, Vec3{}); +} + +TEST(StereoReplayTest, WindowMaskStaysFixedInSpaceAsTheEyeMoves) { + // Stepping sideways moves the screen across the eye image the other way, as + // a real window would: the screen's centre is no longer at the image centre. + const auto eyeFrustum = eye_frustum(0.0f); + const HudScreen screen{.halfWidth = 600.0f, .halfHeight = 337.5f, .distance = 1000.0f}; + Mat3x4 stepRight = identity3x4(); + stepRight.m0[3] = -200.0f; // The eye moves +200 to the right, so the world moves back. + const auto mask = window_mask(eyeFrustum, stepRight, screen); + const auto centre = eye_ndc(eyeFrustum, stepRight, {0.0f, 0.0f, -screen.distance, 1.0f}); + EXPECT_LT(centre[0], 0.0f); + const auto h = mask.at(centre[0], centre[1]); + ASSERT_GT(h.z, 0.0f); + EXPECT_NEAR(h.x / h.z, 0.0f, 1e-4f); + EXPECT_NEAR(h.y / h.z, 0.0f, 1e-4f); + // The image centre now looks through the right part of the screen. + const auto straight = mask.at(0.0f, 0.0f); + ASSERT_GT(straight.z, 0.0f); + EXPECT_NEAR(straight.x / straight.z, 200.0f / screen.halfWidth, 1e-4f); +} + } // namespace } // namespace aurora::gfx::stereo_replay diff --git a/aurora-main/tests/window_mask_gpu_smoke.cpp b/aurora-main/tests/window_mask_gpu_smoke.cpp new file mode 100644 index 0000000..5b9a8ff --- /dev/null +++ b/aurora-main/tests/window_mask_gpu_smoke.cpp @@ -0,0 +1,180 @@ +// SPDX-License-Identifier: GPL-3.0-or-later +// Draws the immersive window's mask (gfx/window_mask.hpp) over a filled eye on a real GPU, in the +// eye's own render pass and in passes of its own, 1x and 4x MSAA, and reads the image back: the +// window keeps its colour with alpha 1, everything outside it becomes transparent black. +#include "../lib/gfx/window_mask.hpp" + +#include +#include +#include +#include + +namespace aurora::webgpu { +wgpu::Device g_device; +wgpu::Queue g_queue; +GraphicsConfig g_graphicsConfig{}; +} // namespace aurora::webgpu + +namespace { +std::atomic errors = 0; +constexpr uint32_t kSize = 256; + +enum class Path { InPass, OwnPass, Output }; + +struct Pixel { + int r, g, b, a; +}; +} // namespace + +int main() { + using namespace aurora; + using namespace aurora::webgpu; + wgpu::InstanceDescriptor instanceDescriptor{}; + const wgpu::InstanceFeatureName timed = wgpu::InstanceFeatureName::TimedWaitAny; + instanceDescriptor.requiredFeatureCount = 1; + instanceDescriptor.requiredFeatures = &timed; + auto instance = wgpu::CreateInstance(&instanceDescriptor); + wgpu::Adapter adapter; + wgpu::RequestAdapterOptions options{.backendType = wgpu::BackendType::D3D12}; + auto future = instance.RequestAdapter(&options, wgpu::CallbackMode::WaitAnyOnly, + [&](wgpu::RequestAdapterStatus status, wgpu::Adapter a, wgpu::StringView message) { + if (status == wgpu::RequestAdapterStatus::Success) + adapter = std::move(a); + else + std::cerr << std::string_view(message) << '\n'; + }); + if (instance.WaitAny(future, 5000000000) != wgpu::WaitStatus::Success || !adapter) + return 1; + wgpu::DeviceDescriptor deviceDescriptor{}; + deviceDescriptor.SetUncapturedErrorCallback([](const wgpu::Device&, wgpu::ErrorType, wgpu::StringView message) { + ++errors; + std::cerr << std::string_view(message) << '\n'; + }); + future = adapter.RequestDevice(&deviceDescriptor, wgpu::CallbackMode::WaitAnyOnly, + [&](wgpu::RequestDeviceStatus status, wgpu::Device device, wgpu::StringView message) { + if (status == wgpu::RequestDeviceStatus::Success) + g_device = std::move(device); + else + std::cerr << std::string_view(message) << '\n'; + }); + if (instance.WaitAny(future, 5000000000) != wgpu::WaitStatus::Success || !g_device) + return 1; + g_queue = g_device.GetQueue(); + g_graphicsConfig.surfaceConfiguration.format = wgpu::TextureFormat::RGBA8Unorm; + + // A 90-degree eye looking at a screen 1 unit ahead, 1 across and 0.5 high: the window covers NDC + // x in -0.5..0.5 and y in -0.25..0.25, pixels 64..192 across and 96..160 down. + const gfx::stereo_replay::HudScreen screen{.halfWidth = 0.5f, .halfHeight = 0.25f, .distance = 1.0f}; + for (const Path path : {Path::InPass, Path::OwnPass, Path::Output}) + for (const uint32_t samples : {1u, 4u}) + for (const bool turnedAway : {false, true}) + for (const uint32_t eyeIndex : {0u, 1u}) { + if (path == Path::Output && samples != 1) + continue; + gfx::StereoReplayFrame frame{}; + frame.window = true; + wgpu::TextureDescriptor textureDescriptor{ + .usage = wgpu::TextureUsage::RenderAttachment | wgpu::TextureUsage::CopySrc, + .size = {kSize, kSize, 1}, + .format = wgpu::TextureFormat::RGBA8Unorm, + .sampleCount = 1, + }; + auto output = g_device.CreateTexture(&textureDescriptor); + textureDescriptor.sampleCount = samples; + textureDescriptor.usage = wgpu::TextureUsage::RenderAttachment; + auto color = g_device.CreateTexture(&textureDescriptor); + textureDescriptor.format = wgpu::TextureFormat::Depth24PlusStencil8; + auto depth = g_device.CreateTexture(&textureDescriptor); + auto& eye = frame.eyes[eyeIndex]; + eye.target.colorView = samples == 1 ? output.CreateView() : color.CreateView(); + if (samples > 1) + eye.target.resolveView = output.CreateView(); + eye.target.depthView = depth.CreateView(); + eye.target.depthFormat = wgpu::TextureFormat::Depth24PlusStencil8; + eye.target.size = {kSize, kSize, 1}; + eye.target.msaaSamples = samples; + eye.projection.m0[0] = 1.0f; + eye.projection.m1[1] = 1.0f; + eye.viewFromCenter.m0 = {turnedAway ? -1.0f : 1.0f, 0.0f, 0.0f, 0.0f}; + eye.viewFromCenter.m1 = {0.0f, 1.0f, 0.0f, 0.0f}; + eye.viewFromCenter.m2 = {0.0f, 0.0f, turnedAway ? -1.0f : 1.0f, 0.0f}; + + // The finished eye, with an alpha the game might leave anywhere. + auto encoder = g_device.CreateCommandEncoder(); + const wgpu::RenderPassColorAttachment fill{.view = eye.target.colorView, + .resolveTarget = eye.target.resolveView, + .loadOp = wgpu::LoadOp::Clear, + .storeOp = wgpu::StoreOp::Store, + .clearValue = {0.5, 0.25, 0.75, 0.3}}; + const wgpu::RenderPassDepthStencilAttachment fillDepth{.view = eye.target.depthView, + .depthLoadOp = wgpu::LoadOp::Clear, + .depthStoreOp = wgpu::StoreOp::Store, + .depthClearValue = 1.0f, + .stencilLoadOp = wgpu::LoadOp::Clear, + .stencilStoreOp = wgpu::StoreOp::Store}; + const wgpu::RenderPassDescriptor fillPass{ + .colorAttachmentCount = 1, .colorAttachments = &fill, .depthStencilAttachment = &fillDepth}; + auto pass = encoder.BeginRenderPass(&fillPass); + if (path == Path::InPass) + gfx::window_mask::draw(pass, frame, eyeIndex, screen); + pass.End(); + if (path == Path::OwnPass) + gfx::window_mask::render(encoder, frame, eyeIndex, screen); + if (path == Path::Output) + gfx::window_mask::render_output(encoder, frame, eyeIndex, output.CreateView(), {kSize, kSize, 1}, screen); + + const wgpu::BufferDescriptor bufferDescriptor{.usage = wgpu::BufferUsage::CopyDst | wgpu::BufferUsage::MapRead, + .size = kSize * kSize * 4}; + auto readback = g_device.CreateBuffer(&bufferDescriptor); + const wgpu::TexelCopyTextureInfo source{.texture = output}; + const wgpu::TexelCopyBufferInfo destination{.layout = {.bytesPerRow = kSize * 4, .rowsPerImage = kSize}, + .buffer = readback}; + const wgpu::Extent3D extent{kSize, kSize, 1}; + encoder.CopyTextureToBuffer(&source, &destination, &extent); + auto commands = encoder.Finish(); + g_queue.Submit(1, &commands); + bool mapped = false; + future = readback.MapAsync(wgpu::MapMode::Read, 0, kSize * kSize * 4, wgpu::CallbackMode::WaitAnyOnly, + [&](wgpu::MapAsyncStatus status, wgpu::StringView) { + mapped = status == wgpu::MapAsyncStatus::Success; + }); + if (instance.WaitAny(future, 5000000000) != wgpu::WaitStatus::Success || !mapped) + return 1; + const auto* bytes = static_cast(readback.GetConstMappedRange()); + const auto at = [&](uint32_t x, uint32_t y) { + const auto* p = bytes + (y * kSize + x) * 4; + return Pixel{p[0], p[1], p[2], p[3]}; + }; + const auto near = [](int value, int expected) { return std::abs(value - expected) <= 2; }; + const auto check = [&](uint32_t x, uint32_t y, bool inside) { + const auto p = at(x, y); + const bool ok = inside ? near(p.r, 128) && near(p.g, 64) && near(p.b, 191) && p.a == 255 + : p.r == 0 && p.g == 0 && p.b == 0 && p.a == 0; + if (!ok) { + std::cerr << "path " << static_cast(path) << ", " << samples << "x, eye " << eyeIndex + << (turnedAway ? ", turned away" : "") << ": pixel (" << x << ", " << y << ") is (" << p.r + << ", " << p.g << ", " << p.b << ", " << p.a << "), expected " + << (inside ? "the eye's colour, opaque" : "transparent black") << '\n'; + ++errors; + } + }; + const bool seen = !turnedAway; + check(128, 128, seen); + check(70, 100, seen); + check(186, 155, seen); + check(5, 5, false); + check(58, 128, false); + check(198, 128, false); + check(128, 90, false); + check(128, 166, false); + check(250, 250, false); + readback.Unmap(); + std::cout << "path " << static_cast(path) << ", " << samples << "x MSAA, eye " << eyeIndex + << (turnedAway ? ", turned away" : "") << ": checked\n"; + } + gfx::window_mask::shutdown(); + g_queue = nullptr; + g_device.Destroy(); + g_device = nullptr; + return errors ? 1 : 0; +} diff --git a/docs/quest-port.md b/docs/quest-port.md index 68b3707..3615217 100644 --- a/docs/quest-port.md +++ b/docs/quest-port.md @@ -250,7 +250,10 @@ suggested for `oculus/touch_controller` and `khr/simple_controller`. (created on first use) or pauses it as each presentation arrives, and submits it first, under the virtual screen's quad, or alone while there is no image yet (startup, a recenter). An immersive race never submits it and pauses the - cameras; a `flat_screen` race is a virtual screen, so it keeps the room. + cameras; a `flat_screen` race is a virtual screen, so it keeps the room, and + so does an `immersive_window` race, whose projection layer is submitted over + it with `XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT` (Aurora leaves + each eye transparent outside the window; see OPENXR.md, "The immersive window"). The quad is cropped to the snapshot Aurora letterboxes into the nearly square eye image (`OpenXRVirtualScreenContentRect`), or its black bands would frame the picture against the room. The manifest's `com.oculus.feature.PASSTHROUGH` is what lets Horizon @@ -268,7 +271,8 @@ The app opens on `LauncherActivity` (`android/app/src/main/java/org/wiicompiled/ a 2D Horizon OS panel modelled on the PC launcher, WheelWizard VR, and using its palette. **Home** has the Play button and reports a missing or incomplete `DATA` (the check is the runtime's own `IsDvdDataRoot`: `files/` and `sys/fst.bin`). -**Settings** edits `Config.toml` in tabs: VR (Flat Screen mode, camera, rotation, driver hiding, +**Settings** edits `Config.toml` in tabs: VR (race view: immersive, immersive window or flat +screen, camera, rotation, driver hiding, seat, hand steering, lean back, render scale, VR interpolation, virtual screen size and distance), Graphics (resolution, widescreen, bloom, shader stutter), Controls (controller diff --git a/runtime/include/runtime_config.h b/runtime/include/runtime_config.h index 218453f..13bc546 100644 --- a/runtime/include/runtime_config.h +++ b/runtime/include/runtime_config.h @@ -58,6 +58,7 @@ struct RuntimeUserConfig { std::optional vrHudWidthMeters; std::optional vrHudVirtualScreen; std::optional vrFlatScreen; + std::optional vrImmersiveWindow; std::optional vrPassthrough; std::optional vrStopAtDisplayCopy; std::optional vrSkipCopyClears; @@ -522,6 +523,11 @@ inline void EnsureConfigFile() { "# first-person camera or hand steering. Changeable live from the\n" "# F10 menu.\n" "flat_screen = false\n" + "# The immersive window keeps the stereo race view but shows it\n" + "# only through that screen, with the room around it on the\n" + "# Quest (black elsewhere). Flat Screen mode wins over it.\n" + "# Changeable live from the F10 menu.\n" + "immersive_window = false\n" "# EFB replay controls for the per-eye views, changeable live\n" "# from the F10 menu. stop_at_display_copy ends each eye at the\n" "# frame's final GXCopyDisp; skip_copy_clears drops the EFB\n" @@ -778,6 +784,7 @@ inline RuntimeUserConfig ParseConfigDocument(const toml::value& document) { } config.vrHudVirtualScreen = FindConfigValue(document, "vr", "hud_virtual_screen"); config.vrFlatScreen = FindConfigValue(document, "vr", "flat_screen"); + config.vrImmersiveWindow = FindConfigValue(document, "vr", "immersive_window"); config.vrPassthrough = FindConfigValue(document, "vr", "passthrough"); config.vrStopAtDisplayCopy = FindConfigValue(document, "vr", "stop_at_display_copy"); config.vrSkipCopyClears = FindConfigValue(document, "vr", "skip_copy_clears"); @@ -1102,6 +1109,11 @@ inline bool SetVrFlatScreen(bool value) { return WriteSetting("vr", "flat_screen", value ? "true" : "false"); } +inline bool SetVrImmersiveWindow(bool value) { + Mutable().vrImmersiveWindow = value; + return WriteSetting("vr", "immersive_window", value ? "true" : "false"); +} + inline bool SetVrPassthrough(bool value) { Mutable().vrPassthrough = value; return WriteSetting("vr", "passthrough", value ? "true" : "false"); @@ -1579,10 +1591,41 @@ inline bool VrFlatScreen(bool fallback = false) { return Get().vrFlatScreen.value_or(fallback); } +// Races in stereo as usual, but seen only through the screen the race's 2D +// layer sits on, with the room around it. Flat Screen mode wins over it. +inline bool VrImmersiveWindow(bool fallback = false) { + return Get().vrImmersiveWindow.value_or(fallback); +} + +// The race view the settings present as one choice, kept in the two keys +// above so that a file without immersive_window reads as it always did. +enum class VrRaceView : int { + Immersive = 0, + ImmersiveWindow = 1, + FlatScreen = 2, +}; + +inline VrRaceView VrRaceViewOf(const RuntimeUserConfig& config) { + if (config.vrFlatScreen.value_or(false)) { + return VrRaceView::FlatScreen; + } + return config.vrImmersiveWindow.value_or(false) ? VrRaceView::ImmersiveWindow : VrRaceView::Immersive; +} + +inline VrRaceView GetVrRaceView() { + return VrRaceViewOf(Get()); +} + +inline bool SetVrRaceView(VrRaceView view) { + const bool flat = SetVrFlatScreen(view == VrRaceView::FlatScreen); + const bool window = SetVrImmersiveWindow(view == VrRaceView::ImmersiveWindow); + return flat && window; +} + // The room, through the headset's cameras, around the menu screen and every -// other virtual screen, a Flat Screen race included (never an immersive -// race). Only the Quest offers it; the launcher's Settings page shows the same -// default. +// other virtual screen, a Flat Screen race included, and around the immersive +// window (never a fully immersive race). Only the Quest offers it; the +// launcher's Settings page shows the same default. inline bool VrPassthrough(bool fallback = true) { return Get().vrPassthrough.value_or(fallback); } diff --git a/runtime/include/vr/openxr_backend.h b/runtime/include/vr/openxr_backend.h index 6946055..db9e70d 100644 --- a/runtime/include/vr/openxr_backend.h +++ b/runtime/include/vr/openxr_backend.h @@ -132,9 +132,15 @@ struct OpenXRPresentation { bool quad_anchored = false; XrPosef quad_pose{{0.0f, 0.0f, 0.0f, 1.0f}, {0.0f, 0.0f, 0.0f}}; - // Show the room through the headset's cameras around the virtual screen - // (OpenXRPassthrough). Taken when the presentation is handed to the backend, - // which starts or pauses the view then; a backend without one ignores it. + // Used only by ImmersiveProjection: Aurora left each eye transparent outside + // the race's 2D-layer screen (AuroraStereoFrame::window), so the projection + // layer is blended by its alpha over whatever is under it. + bool immersive_window = false; + + // Show the room through the headset's cameras around the virtual screen or + // the immersive window (OpenXRPassthrough). Taken when the presentation is + // handed to the backend, which starts or pauses the view then; a backend + // without one ignores it. bool passthrough = false; OpenXRPanelLayer panel; diff --git a/runtime/include/vr/openxr_integration.h b/runtime/include/vr/openxr_integration.h index 8b36204..4ff8e57 100644 --- a/runtime/include/vr/openxr_integration.h +++ b/runtime/include/vr/openxr_integration.h @@ -61,11 +61,19 @@ void OpenXRRequestRecenter() noexcept; void OpenXRSetLeanBackDegrees(float degrees) noexcept; // Shows the room through the headset's cameras around the menu screen and every -// other virtual screen, never during an immersive race. Only the standalone -// (Quest) backend offers it; elsewhere this changes nothing. Callable from any -// thread; applied on the XR pacing thread's next frame. +// other virtual screen, and around the immersive window, never during a fully +// immersive race. Only the standalone (Quest) backend offers it; elsewhere this +// changes nothing. Callable from any thread; applied on the XR pacing thread's +// next frame. void OpenXRSetPassthrough(bool enabled) noexcept; +// The immersive window: an immersive race keeps its stereo view but is seen +// only through the screen its 2D layer sits on, with the room (or, without +// passthrough, black) around it. Callable from any thread; applied to the next +// published frame. Flat Screen mode, which keeps races off the immersive path +// altogether, makes it moot. +void OpenXRSetImmersiveWindow(bool enabled) noexcept; + // Live scene interpolation at the headset's own display deadlines. // 0 = Off, 1 = Auto, otherwise 72/90/120 as a rendering-rate ceiling. void OpenXRSetFrameInterpolationFps(uint32_t target) noexcept; diff --git a/runtime/src/settings_overlay.cpp b/runtime/src/settings_overlay.cpp index 3b7a98e..6fdb074 100644 --- a/runtime/src/settings_overlay.cpp +++ b/runtime/src/settings_overlay.cpp @@ -142,7 +142,11 @@ bool g_vrStopAtDisplayCopy = RuntimeConfigFile::VrStopAtDisplayCopy(true); bool g_vrSkipCopyClears = RuntimeConfigFile::VrSkipCopyClears(true); bool g_vrSinglePassEyes = RuntimeConfigFile::VrSinglePassEyes(true); bool g_vrHudVirtualScreen = RuntimeConfigFile::VrHudVirtualScreen(true); -bool g_vrFlatScreen = RuntimeConfigFile::VrFlatScreen(); +// Race view: Immersive, Immersive window or Flat screen (RuntimeConfigFile::VrRaceView), and +// Flat Screen mode as the flag the race view rows below are disabled by. +int g_vrRaceView = static_cast(RuntimeConfigFile::GetVrRaceView()); +bool g_vrFlatScreen = g_vrRaceView == static_cast(RuntimeConfigFile::VrRaceView::FlatScreen); +constexpr std::array kVrRaceViewLabels{"Immersive", "Immersive window", "Flat screen"}; #if defined(__ANDROID__) bool g_vrPassthrough = RuntimeConfigFile::VrPassthrough(); // Menu labels for the foveation levels, index-matched to RuntimeConfigFile::kVrFoveationLevels and to @@ -1444,7 +1448,8 @@ void DrawVrSettings() { } ImGui::Separator(); ImGui::Text("VR 2D layer"); - ImGui::BeginDisabled(g_vrFlatScreen); + ImGui::BeginDisabled( + g_vrFlatScreen || g_vrRaceView == static_cast(RuntimeConfigFile::VrRaceView::ImmersiveWindow)); if (ImGui::Checkbox("2D layer on a virtual screen", &g_vrHudVirtualScreen)) { ApplyVrHudVirtualScreen(); RuntimeConfigFile::SetVrHudVirtualScreen(g_vrHudVirtualScreen); @@ -1455,7 +1460,8 @@ void DrawVrSettings() { "Puts the minimap, race position, item roulette and the rest of the race HUD on a " "screen fixed ahead of the kart camera. Turn off to leave them stretched across " "the whole view. Its size and distance are the [vr] hud_width_meters and " - "hud_distance_meters read at launch."); + "hud_distance_meters read at launch. The immersive window is that screen, and " + "always carries them."); } ImGui::Separator(); ImGui::Text("VR view"); @@ -1514,8 +1520,8 @@ void DrawVrSettings() { ImGui::SetTooltip( "Shows your room through the headset's cameras around the menu screen and every " "other screen outside an immersive race, instead of black. Immersive races stay " - "fully virtual; the Flat Screen race has the room around it too. " - "Applies immediately."); + "fully virtual; the immersive window and the Flat Screen race have the room " + "around them too. Applies immediately."); } // Shows the live level, which debug.wiicompiled.foveation can override. g_vrFoveation = static_cast(aurora_get_stereo_foveation()); @@ -1539,16 +1545,23 @@ void DrawVrSettings() { #endif ImGui::Separator(); ImGui::Text("VR camera"); - if (ImGui::Checkbox("Flat Screen mode", &g_vrFlatScreen)) { - RuntimeConfigFile::SetVrFlatScreen(g_vrFlatScreen); + if (ImGui::Combo("Race view", &g_vrRaceView, kVrRaceViewLabels.data(), + static_cast(kVrRaceViewLabels.size()))) { + const auto view = static_cast(g_vrRaceView); + g_vrFlatScreen = view == RuntimeConfigFile::VrRaceView::FlatScreen; + RuntimeConfigFile::SetVrRaceView(view); mkw::vr::MkwVRPolicySetImmersiveRaces(!g_vrFlatScreen); + mkw::vr::OpenXRSetImmersiveWindow(view == RuntimeConfigFile::VrRaceView::ImmersiveWindow); mkw::vr::MkwVRFirstPersonApplyConfiguredSettings(); } if (ImGui::IsItemHovered()) { ImGui::SetTooltip( - "Plays races on the same flat screen as the menus, through the game's own camera, " - "instead of all around you in stereo. The first-person camera, hand steering and " - "the race view settings do not apply while it is on. Applies immediately."); + "Immersive plays races all around you in stereo. Immersive window keeps that " + "stereo view but shows it only through a window where the menu screen sits, with " + "your room around it on the Quest (black elsewhere); look through it from another " + "angle and the view shifts as through a real window. Flat screen plays races on " + "the menu screen through the game's own camera; the first-person camera, hand " + "steering and the race view settings do not apply to it. Applies immediately."); } // Everything below shapes the immersive race view, which Flat Screen mode replaces. ImGui::BeginDisabled(g_vrFlatScreen); @@ -1704,7 +1717,8 @@ void DrawVrSettings() { } // The right-thumbstick click: flips the first-person camera exactly as its -// checkbox does, so it does nothing in Flat Screen mode either. Game thread. +// checkbox does, so it does nothing in Flat Screen mode either (it does in the +// immersive window, which is still the stereo race view). Game thread. void ToggleFirstPersonCamera() { if (g_vrFlatScreen) { return; diff --git a/runtime/src/vr/openxr_integration.cpp b/runtime/src/vr/openxr_integration.cpp index ac2e343..c9f2cca 100644 --- a/runtime/src/vr/openxr_integration.cpp +++ b/runtime/src/vr/openxr_integration.cpp @@ -568,6 +568,10 @@ public: passthrough_.store(enabled, std::memory_order_relaxed); } + void SetImmersiveWindow(bool enabled) noexcept { + immersive_window_.store(enabled, std::memory_order_relaxed); + } + void SetLeanBackDegrees(float degrees) noexcept { lean_back_degrees_.store( std::clamp(degrees, -RuntimeConfigFile::kVrLeanBackDegreesLimit, @@ -846,10 +850,15 @@ private: aurora_get_stereo_screen_aspects(&picture_aspect, &snapshot_aspect)) { presentation.quad_content_aspect = snapshot_aspect; } - // The room around the menu screen and every other virtual screen, a - // Flat Screen race included; an immersive race is fully virtual, and the - // cameras are paused for it. - presentation.passthrough = !immersive && passthrough_.load(std::memory_order_relaxed); + // The immersive window: the race's stereo view seen through its 2D layer's screen. + // The flag travels with the packet, so the eyes Aurora masks and the layer the + // backend blends always belong to the same frame. + presentation.immersive_window = immersive && immersive_window_.load(std::memory_order_relaxed); + // The room around the menu screen and every other virtual screen, a Flat Screen + // race included, and around the immersive window; a fully immersive race is + // virtual all round, and the cameras are paused for it. + presentation.passthrough = + (!immersive || presentation.immersive_window) && passthrough_.load(std::memory_order_relaxed); // The settings panel gets a compositor layer of its own while it is // open, and Aurora leaves it out of the eyes. A backend that could // not make that layer has the panel drawn into the eyes instead. @@ -1247,6 +1256,7 @@ private: destination.displayTimeNanos = DisplayTimeNanos(source.xr_frame.predicted_display_time); destination.mode = immersive ? AURORA_STEREO_FRAME_IMMERSIVE_REPLAY : AURORA_STEREO_FRAME_VIRTUAL_SCREEN; + destination.window = immersive && source.presentation.immersive_window; for (uint32_t eye = 0; eye < kOpenXREyeCount; ++eye) { destination.eyes[eye].width = source.render_width[eye]; destination.eyes[eye].height = source.render_height[eye]; @@ -1378,7 +1388,8 @@ private: // that space (in metres) to base + lean * p in the application space, so the // screen sits at base + lean * (0, 0, -distance), turned by the lean, its // height following the picture aspect as stereo_hud_screen's does. With the - // 2D layer stretched across the eyes there is no screen to point at. + // 2D layer stretched across the eyes there is no screen to point at, except + // in the immersive window, which is that screen and always carries the layer. OpenXRPointerScreen PointerScreen(const OpenXRBackendFrame& frame, const MkwVRPolicySnapshot& policy, bool immersive) const noexcept { OpenXRPointerScreen screen{}; @@ -1389,8 +1400,9 @@ private: } if (immersive) { - if (!aurora_get_stereo_hud_screen_enabled() || !(policy.config.hud_width_meters > 0.0f) || - !RaceScreenPose(frame, policy, screen.pose)) { + // The immersive window always carries the 2D layer. + const bool on_screen = aurora_get_stereo_hud_screen_enabled() || frame.presentation.immersive_window; + if (!on_screen || !(policy.config.hud_width_meters > 0.0f) || !RaceScreenPose(frame, policy, screen.pose)) { return screen; } screen.half_width_meters = 0.5f * policy.config.hud_width_meters; @@ -1756,6 +1768,7 @@ private: std::atomic_bool recenter_requested_{false}; std::atomic lean_back_degrees_{RuntimeConfigFile::VrLeanBackDegrees()}; std::atomic_bool passthrough_{RuntimeConfigFile::VrPassthrough()}; + std::atomic_bool immersive_window_{RuntimeConfigFile::VrImmersiveWindow()}; std::atomic_uint32_t frame_interpolation_fps_{RuntimeConfigFile::VrFrameInterpolationFps()}; std::atomic_bool interpolation_available_{false}; std::mutex interpolation_mutex_; @@ -1877,6 +1890,14 @@ void OpenXRSetPassthrough(bool enabled) noexcept { #endif } +void OpenXRSetImmersiveWindow(bool enabled) noexcept { +#if MKW_OPENXR_GRAPHICS_BACKEND + OpenXRIntegration::Get().SetImmersiveWindow(enabled); +#else + (void)enabled; +#endif +} + void OpenXRSetFrameInterpolationFps(uint32_t target) noexcept { #if MKW_OPENXR_GRAPHICS_BACKEND OpenXRIntegration::Get().SetFrameInterpolationFps(target); diff --git a/runtime/src/vr/openxr_vulkan.cpp b/runtime/src/vr/openxr_vulkan.cpp index 44adc22..8c1f146 100644 --- a/runtime/src/vr/openxr_vulkan.cpp +++ b/runtime/src/vr/openxr_vulkan.cpp @@ -942,7 +942,10 @@ public: views[eye].subImage.imageArrayIndex = 0; } XrCompositionLayerProjection projection{XR_TYPE_COMPOSITION_LAYER_PROJECTION}; - projection.layerFlags = 0; + // The immersive window's eyes are transparent outside the window (premultiplied alpha), so + // the room shows around it; otherwise the race covers the whole view and alpha is ignored. + projection.layerFlags = + frame.presentation.immersive_window ? XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT : 0; projection.space = runtime_->AppSpace(); projection.viewCount = kOpenXREyeCount; projection.views = views.data(); @@ -950,16 +953,18 @@ public: } // Ends the compositor frame with the scene's layer: over the room's camera - // view while that runs and the scene is the virtual screen (never under the - // race's projection, which covers the whole view), and, while the retained - // frame rendered it, under the settings panel's layer. + // view while that runs and the scene is the virtual screen or the immersive + // window (never under a fully immersive race's projection, which covers the + // whole view), and, while the retained frame rendered it, under the settings + // panel's layer. bool EndFrameWithPanel(const OpenXRBackendFrame& frame, const XrCompositionLayerBaseHeader* scene) { const auto& panel = frame.presentation.panel; XrCompositionLayerQuad panel_quad{}; const XrCompositionLayerBaseHeader* layers[3] = {}; uint32_t count = 0; if (const XrCompositionLayerBaseHeader* passthrough = passthrough_.Layer(); - passthrough != nullptr && frame.presentation.mode == OpenXRFrameMode::VirtualScreen) { + passthrough != nullptr && (frame.presentation.mode == OpenXRFrameMode::VirtualScreen || + frame.presentation.immersive_window)) { layers[count++] = passthrough; } layers[count++] = scene; diff --git a/runtime/tests/vr_config_tests.cpp b/runtime/tests/vr_config_tests.cpp index bf18d13..8b493fe 100644 --- a/runtime/tests/vr_config_tests.cpp +++ b/runtime/tests/vr_config_tests.cpp @@ -34,5 +34,18 @@ int main() { Require(Parse("[vr]\nsingle_pass_eyes = true\n").vrSinglePassEyes == true); Require(Parse("[vr]\nsingle_pass_eyes = false\n").vrSinglePassEyes == false); Require(!Parse("[vr]\n").vrSinglePassEyes.has_value()); + + // [vr] immersive_window and flat_screen: one race view in two keys, Flat + // Screen mode winning, so a file that predates the window reads as before. + using RuntimeConfigFile::VrRaceView; + using RuntimeConfigFile::VrRaceViewOf; + Require(Parse("[vr]\nimmersive_window = true\n").vrImmersiveWindow == true); + Require(!Parse("[vr]\n").vrImmersiveWindow.has_value()); + Require(VrRaceViewOf(Parse("[vr]\n")) == VrRaceView::Immersive); + Require(VrRaceViewOf(Parse("[vr]\nflat_screen = false\n")) == VrRaceView::Immersive); + Require(VrRaceViewOf(Parse("[vr]\nflat_screen = true\n")) == VrRaceView::FlatScreen); + Require(VrRaceViewOf(Parse("[vr]\nimmersive_window = true\n")) == VrRaceView::ImmersiveWindow); + Require(VrRaceViewOf(Parse("[vr]\nflat_screen = true\nimmersive_window = true\n")) == VrRaceView::FlatScreen); + Require(VrRaceViewOf(Parse("[vr]\nflat_screen = false\nimmersive_window = false\n")) == VrRaceView::Immersive); return 0; }