mirror of
https://github.com/mitch030504/Wiicompiled_VR_Frame.git
synced 2026-10-06 01:00:14 +02:00
feat: added immersive window support for VR race views
- Introduced a new configuration option for immersive window mode in runtime_config.h. - Updated the parsing and setting functions to handle the immersive window state. - Modified the OpenXR backend to support rendering with the immersive window, blending the race view with the surrounding environment. - Enhanced the settings overlay to allow users to select between immersive, immersive window, and flat screen race views. - Implemented GPU rendering logic for the immersive window mask, ensuring correct visual output in various rendering paths. - Added tests to validate the immersive window functionality and its interaction with existing race view settings.
This commit is contained in:
iChris4 committed
1 parent
1969dda0bf
commit
1cf9389d69
20 files changed
+917
-53
No files matched your search
@@ -46,6 +46,7 @@ hud_distance_meters = 2.0
|
||||
hud_width_meters = 2.4
|
||||
hud_virtual_screen = true
|
||||
flat_screen = false
|
||||
immersive_window = false
|
||||
stop_at_display_copy = true
|
||||
skip_copy_clears = true
|
||||
single_pass_eyes = true
|
||||
@@ -146,14 +147,20 @@ immersive stereo: the whole race, 3D world and HUD alike, is the game's own pict
|
||||
in DolphinXR's Flat Screen mode. The first-person camera, hand steering, the lean-back angle, VR
|
||||
frame interpolation and `hud_virtual_screen` shape only the immersive race view, so none of them
|
||||
apply while it is on; the right-thumbstick first-person toggle is ignored rather than changing the saved
|
||||
setting. It is live, as **F10 → VR → Flat Screen mode** (the headset panel's VR tab) and the Quest
|
||||
launcher's Settings page, and turning it on or off mid-race switches on the next frame through the
|
||||
presentation policy's safety generation.
|
||||
setting. It is live, as **F10 → VR → Race view → Flat screen** (the headset panel's VR tab) and the
|
||||
Quest launcher's Settings page, and turning it on or off mid-race switches on the next frame through
|
||||
the presentation policy's safety generation.
|
||||
`immersive_window` (default off) is the third race view, between the two: the race keeps its
|
||||
immersive stereo view, head tracking and all, but is seen only through a window, with the room
|
||||
around it on the Quest; see [The immersive window](#the-immersive-window). `flat_screen` wins when
|
||||
both are set. The settings present the three as one choice, **Race view**: Immersive, Immersive
|
||||
window or Flat screen.
|
||||
`passthrough` (Quest only, default on) shows the room through the headset's cameras around the
|
||||
menu screen and every other virtual screen, instead of black: an `XR_FB_passthrough`
|
||||
reconstruction layer submitted under the screen's quad, as PPSSPP VR does, with the blend mode
|
||||
left `OPAQUE`. An immersive race never shows it, and the cameras are paused for the race; a race in
|
||||
`flat_screen` is a virtual screen like the menus, so the room shows around it too. It is
|
||||
left `OPAQUE`. A fully immersive race never shows it, and the cameras are paused for the race; a
|
||||
race in `flat_screen` is a virtual screen like the menus, so the room shows around it too, and so it
|
||||
does around the immersive window. It is
|
||||
live, from the headset panel's VR tab or the launcher's Settings page. The app declares
|
||||
`com.oculus.feature.PASSTHROUGH`, without which Horizon OS composites nothing for that layer.
|
||||
So that the room frames the picture rather than black bands, the Quest's menu quad shows only the
|
||||
@@ -529,6 +536,8 @@ The runtime deliberately fails safe instead of guessing which Mario Kart camera
|
||||
however complete the observations are. Changing it advances the safety generation like any
|
||||
other change of presentation, and the pacing thread still treats that race as a race: pipeline
|
||||
caches are not stored mid-race on the virtual screen either.
|
||||
- `immersive_window` is not a policy state: the race is `ImmersiveRace` either way, and only the
|
||||
packet and the layer that shows it carry the window.
|
||||
|
||||
Aurora records the original GX frame once and replays it for both OpenXR eyes. Perspective GX draws
|
||||
receive asymmetric headset projections, while the game's 2D layer goes on a fixed virtual screen
|
||||
@@ -619,6 +628,41 @@ Kart Wii's minimap are treated as game art and remain eligible for the screen. A
|
||||
uses the full eye viewport and scissor because its recorded rectangle no longer describes where it
|
||||
ended up; its original viewport is folded into the projection instead.
|
||||
|
||||
## The immersive window
|
||||
|
||||
`immersive_window` shows the immersive race through a window rather than all around you. The window
|
||||
is the race's 2D-layer screen: `hud_width_meters` across and `hud_distance_meters` ahead of the
|
||||
race origin latched at the race start, turned by the lean-back angle, its height following the
|
||||
picture's aspect. That is where the HUD, the Wii Remote pointer and the settings panel already sit
|
||||
in an immersive race, so the HUD lies on the window's plane and the pointer aims at it. The window
|
||||
always carries the 2D layer, whatever `hud_virtual_screen` says: stretched across the eye, the HUD
|
||||
would be cut by the window's edges. It sits straight ahead of the race's forward direction, as the
|
||||
HUD does, which is the reference space's forward rather than the heading the menu screen was
|
||||
anchored at, so after an in-game recenter while facing sideways the two can differ.
|
||||
|
||||
Inside the window nothing changes: the eyes are the immersive race's, with their per-eye frusta,
|
||||
head tracking, first person, hand steering and VR frame interpolation. Moving your head therefore
|
||||
shifts the view through the window like a real window's, and geometry nearer than the window's
|
||||
plane is still cut by its edges, as a stereo picture's frame cuts it.
|
||||
|
||||
How it is drawn: the pacing thread marks the stereo packet (`AuroraStereoFrame::window`), and after an
|
||||
eye's last draw Aurora covers the eye with one full-screen triangle (`aurora-main/lib/gfx/window_mask.hpp`)
|
||||
that keeps the colour inside the window with alpha 1 and leaves transparent black outside it, with a
|
||||
one-pixel ramp at the edge. The triangle carries, at each corner, where that pixel's ray meets the
|
||||
window's plane in homogeneous window coordinates (`stereo_replay::window_mask`), which interpolate
|
||||
exactly across the image. With `single_pass_eyes` it is drawn in the eye's own last render pass, so it
|
||||
adds no pass and no tile load; otherwise it takes a pass of its own, as the cockpit overlay does. The
|
||||
game still renders the whole eye: only its alpha changes. On the Quest the backend submits the
|
||||
passthrough layer, then the projection layer with `XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT`
|
||||
(premultiplied alpha), then the settings panel. The flag travels with the packet, so the eyes Aurora
|
||||
masked and the layer that blends them always belong to the same frame, and switching the race view
|
||||
mid-race needs no safety generation: presentation stays `ImmersiveRace`. The PC backends keep their
|
||||
projection layer opaque, so the window is surrounded by black there.
|
||||
|
||||
`gx_fifo_tests` covers the window's geometry (its corners through an asymmetric eye frustum, its
|
||||
agreement with the HUD's placement, an eye turned away or beyond the window, a sideways step), and
|
||||
`mkw_vr_config_tests` how the two keys read as one race view.
|
||||
|
||||
## Foveated rendering
|
||||
|
||||
On the Quest, `foveation` shades the edges of the immersive race view in 2x2, then 4x4 pixel
|
||||
@@ -892,6 +936,9 @@ ends, including mid-frame flushes, so live setting changes cannot invalidate pen
|
||||
analog grips (Touch); the simple controller profile cannot grab.
|
||||
- The headset settings panel has no laser beam, only the cursor on the panel itself, and text fields
|
||||
cannot be typed into without a keyboard.
|
||||
- The immersive window renders the whole eye and only masks it, so it costs what a fully immersive
|
||||
race costs, plus the passthrough's compositing. Hands and a separate VR wheel are masked with the
|
||||
rest of the eye, so outside the window they are not seen.
|
||||
- The desktop window remains available as a mirror/fallback.
|
||||
|
||||
OpenXR diagnostics are written to the normal run log under
|
||||
|
||||
@@ -116,14 +116,28 @@ class SettingsPage(
|
||||
private fun buildVr() {
|
||||
// Flat Screen mode keeps races on the menu screen, which none of the race view rows reach.
|
||||
val immersive = { c: TomlConfig -> !(c.bool("vr", "flat_screen") ?: false) }
|
||||
// The immersive window is still the stereo race view, seen through the race HUD's screen.
|
||||
val window = { c: TomlConfig -> immersive(c) && (c.bool("vr", "immersive_window") ?: false) }
|
||||
val firstPerson = { c: TomlConfig -> immersive(c) && (c.bool("vr", "first_person") ?: false) }
|
||||
// The steering wheel and hand steering belong to the cockpit seat.
|
||||
val cockpit = { c: TomlConfig -> firstPerson(c) && stringIndex(c, "vr", "first_person_seat", SEATS) == 0 }
|
||||
section(R.string.section_vr_camera) {
|
||||
toggle(
|
||||
R.string.vr_flat_screen, R.string.vr_flat_screen_helper,
|
||||
read = { !immersive(it) },
|
||||
write = { c, value -> c.setBool("vr", "flat_screen", value) },
|
||||
// One setting in two keys, read as runtime_config.h's VrRaceView reads them: Flat
|
||||
// Screen mode wins over the immersive window.
|
||||
choice(
|
||||
R.string.vr_race_view, R.string.vr_race_view_helper,
|
||||
listOf(R.string.vr_race_view_immersive, R.string.vr_race_view_window, R.string.vr_race_view_flat),
|
||||
read = {
|
||||
when {
|
||||
!immersive(it) -> 2
|
||||
window(it) -> 1
|
||||
else -> 0
|
||||
}
|
||||
},
|
||||
write = { c, index ->
|
||||
c.setBool("vr", "flat_screen", index == 2)
|
||||
c.setBool("vr", "immersive_window", index == 1)
|
||||
},
|
||||
)
|
||||
choice(
|
||||
R.string.vr_camera, R.string.vr_camera_helper,
|
||||
@@ -217,11 +231,12 @@ class SettingsPage(
|
||||
)
|
||||
}
|
||||
section(R.string.section_vr_screen) {
|
||||
// The immersive window is that screen and always carries the HUD.
|
||||
toggle(
|
||||
R.string.vr_hud_screen, R.string.vr_hud_screen_helper,
|
||||
read = { it.bool("vr", "hud_virtual_screen") ?: true },
|
||||
write = { c, value -> c.setBool("vr", "hud_virtual_screen", value) },
|
||||
enabledIf = immersive,
|
||||
enabledIf = { immersive(it) && !window(it) },
|
||||
)
|
||||
slider(
|
||||
R.string.vr_hud_distance, R.string.vr_hud_distance_helper, 0.5, 5.0, 0.1,
|
||||
|
||||
@@ -508,8 +508,11 @@
|
||||
|
||||
<!-- VR tab -->
|
||||
<string name="section_vr_camera">Camera</string>
|
||||
<string name="vr_flat_screen">Flat Screen mode</string>
|
||||
<string name="vr_flat_screen_helper">Plays races on the same flat screen as the menus instead of all around you. The camera settings below do not apply while it is on.</string>
|
||||
<string name="vr_race_view">Race view</string>
|
||||
<string name="vr_race_view_helper">Immersive plays races all around you in stereo. Immersive window keeps that stereo view but shows it only through a window where the menu screen sits, with your room around it. Flat screen plays races on the menu screen; the camera settings below do not apply to it.</string>
|
||||
<string name="vr_race_view_immersive">Immersive</string>
|
||||
<string name="vr_race_view_window">Immersive window</string>
|
||||
<string name="vr_race_view_flat">Flat screen</string>
|
||||
<string name="vr_camera">Camera</string>
|
||||
<string name="vr_camera_helper">Ride behind the kart like the game, or sit in the driver\'s seat.</string>
|
||||
<string name="vr_camera_chase">Chase camera</string>
|
||||
@@ -562,7 +565,7 @@
|
||||
<string name="vr_hud_width">Screen width</string>
|
||||
<string name="vr_hud_width_helper">How wide the menu screen and race HUD are.</string>
|
||||
<string name="vr_passthrough">Passthrough around the menu screen</string>
|
||||
<string name="vr_passthrough_helper">Shows your room through the headset\'s cameras around the menus instead of black. Races stay fully virtual, except in Flat Screen mode.</string>
|
||||
<string name="vr_passthrough_helper">Shows your room through the headset\'s cameras around the menus instead of black. Immersive races stay fully virtual; the immersive window and the flat screen race have the room around them too.</string>
|
||||
|
||||
<!-- Graphics tab -->
|
||||
<string name="section_graphics">Rendering</string>
|
||||
|
||||
@@ -180,6 +180,13 @@ typedef struct {
|
||||
uint64_t displayTimeNanos;
|
||||
// Optional; inactive when zero-initialised.
|
||||
AuroraCockpit cockpit;
|
||||
// Immersive replay only: shows the race through a window rather than all
|
||||
// around. Each eye keeps what it sees through the 2D layer's screen
|
||||
// (aurora_set_stereo_hud_screen's rectangle, on which the 2D layer is then
|
||||
// always placed), with premultiplied alpha 1 there, and is transparent black
|
||||
// everywhere else, for the host's compositor to show its own background
|
||||
// (the room, on a headset with passthrough) around it.
|
||||
bool window;
|
||||
} AuroraStereoFrame;
|
||||
|
||||
/**
|
||||
|
||||
@@ -815,6 +815,7 @@ gfx::StereoReplayFrame make_stereo_replay_frame(const AuroraStereoFrame& input,
|
||||
std::memcpy(&anchorFromScene, sceneAnchor.anchorFromScene.data(), sizeof(anchorFromScene));
|
||||
gfx::StereoReplayFrame replay{};
|
||||
replay.cockpit = input.cockpit;
|
||||
replay.window = input.mode == AURORA_STEREO_FRAME_IMMERSIVE_REPLAY && input.window;
|
||||
// The sealed guest frame owns its scale. The packet may have been sampled
|
||||
// just before a change of scale (a character swap, a lightning strike), so
|
||||
// only its head/IPD translation is rescaled to the frame's.
|
||||
@@ -2216,6 +2217,10 @@ std::vector<PresentationJob> encode_sealed_frame(gfx::SealedFrame& sealedFrame,
|
||||
encode_virtual_screen_eye(encoder, completedMono, eye);
|
||||
const auto& output = g_stereoEyeTargets[eye].output();
|
||||
stereo_overlay::composite_flat(encoder, output.view, output.size, eye);
|
||||
// An immersive packet that could not be replayed still goes out as a windowed layer.
|
||||
if (ctx.stereoReplay->window) {
|
||||
gfx::mask_stereo_eye_output(sealedFrame, encoder, *ctx.stereoReplay, eye, output.view, output.size);
|
||||
}
|
||||
}
|
||||
if (mirrorPlan == MirrorPlan::Black && !headsetOnly) {
|
||||
encode_presentation_snapshot(encoder, ctx.presentSource, *finalImage, true, MirrorPlan::Black,
|
||||
|
||||
@@ -11,6 +11,7 @@
|
||||
#include "pipeline_cache.hpp"
|
||||
#include "stereo_replay.hpp"
|
||||
#include "cockpit.hpp"
|
||||
#include "window_mask.hpp"
|
||||
#include "tex_copy_conv.hpp"
|
||||
#include "tex_palette_conv.hpp"
|
||||
#include "texture_replacement.hpp"
|
||||
@@ -355,6 +356,8 @@ struct SealedFrameData {
|
||||
std::vector<RenderPass> passes;
|
||||
LateStereoData stereo;
|
||||
uint32_t localPlayerCount = 1;
|
||||
// The immersive window's screen as this frame's 2D layer was placed on it.
|
||||
stereo_replay::HudScreen windowScreen{};
|
||||
};
|
||||
|
||||
SealedFrame::SealedFrame() : m_data(std::make_unique<SealedFrameData>()) {}
|
||||
@@ -1083,6 +1086,7 @@ void initialize() {
|
||||
|
||||
void shutdown() {
|
||||
cockpit::shutdown();
|
||||
window_mask::shutdown();
|
||||
shutdown_pipeline_cache();
|
||||
gx::clear_shader_module_cache();
|
||||
efb_ram::shutdown();
|
||||
@@ -1331,11 +1335,9 @@ static void log_stereo_display_source_region(ClipRect region, bool foundDisplayC
|
||||
// ratio the game is currently presenting at: 4:3 while VILockAspectRatio holds
|
||||
// it there, otherwise the mirror window's own aspect, which is what Mario Kart
|
||||
// Wii's dynamic widescreen builds its projections from. Matching it keeps the
|
||||
// HUD unstretched on the screen.
|
||||
static stereo_replay::HudScreen stereo_hud_screen() noexcept {
|
||||
if (!g_stereoHudScreenEnabled.load(std::memory_order_relaxed)) {
|
||||
return {};
|
||||
}
|
||||
// HUD unstretched on the screen. The immersive window is this same screen, and
|
||||
// has it whether or not the 2D layer is set to go on it.
|
||||
static stereo_replay::HudScreen stereo_window_screen() noexcept {
|
||||
const float width = g_stereoHudScreenWidth.load(std::memory_order_relaxed);
|
||||
const float distance = g_stereoHudScreenDistance.load(std::memory_order_relaxed);
|
||||
float aspect = 0.f;
|
||||
@@ -1350,6 +1352,13 @@ static stereo_replay::HudScreen stereo_hud_screen() noexcept {
|
||||
};
|
||||
}
|
||||
|
||||
static stereo_replay::HudScreen stereo_hud_screen() noexcept {
|
||||
if (!g_stereoHudScreenEnabled.load(std::memory_order_relaxed)) {
|
||||
return {};
|
||||
}
|
||||
return stereo_window_screen();
|
||||
}
|
||||
|
||||
// Shared by the normal seal and headset-deadline replay. Head transforms are
|
||||
// composed after scene interpolation, so free look never inherits its delay.
|
||||
static void write_stereo_uniform(std::span<uint8_t> uniform, const gx::UniformReplayLayout& layout,
|
||||
@@ -1441,7 +1450,9 @@ static bool prepare_stereo_replay_uniforms(const StereoReplayFrame& stereoFrame,
|
||||
// This is the producer-side preparation path; eye replay can query the pure
|
||||
// helper concurrently without touching this diagnostic state.
|
||||
log_stereo_display_source_region(displayRegion, displaySource.foundDisplayCopy);
|
||||
const stereo_replay::HudScreen hudScreen = stereo_hud_screen();
|
||||
// The immersive window always carries the 2D layer: stretched across the eye,
|
||||
// it would be cut by the window's edges.
|
||||
const stereo_replay::HudScreen hudScreen = stereoFrame.window ? stereo_window_screen() : stereo_hud_screen();
|
||||
// A draw is replayed per eye when it carries the game camera (perspective) or
|
||||
// when it is 2D content the virtual screen is claiming.
|
||||
const auto replayed = [&](const gx::UniformReplayLayout& layout) noexcept {
|
||||
@@ -1782,6 +1793,11 @@ struct RenderInvocation {
|
||||
bool* sceneDrawn = nullptr;
|
||||
// An eye replay laid out by eye_pass_plan, replacing the one-render-pass-per-recorded-pass loop.
|
||||
const eye_pass_plan::Plan* eyePlan = nullptr;
|
||||
// The immersive window's mask, drawn by a planned eye at the end of its last render pass
|
||||
// (render_stereo_eye draws it otherwise).
|
||||
const StereoReplayFrame* windowFrame = nullptr;
|
||||
stereo_replay::HudScreen windowScreen{};
|
||||
bool* windowDrawn = nullptr;
|
||||
};
|
||||
|
||||
static void render_pass_impl(const wgpu::RenderPassEncoder& pass, const std::vector<RenderPass>& passes, u32 idx,
|
||||
@@ -1859,6 +1875,11 @@ static void render_eye_planned(std::vector<RenderPass>& renderPasses, wgpu::Comm
|
||||
invocation.cockpitDepth, &pass);
|
||||
*invocation.cockpitDrawn = true;
|
||||
}
|
||||
// Last of all, over everything the eye draws, in the same render pass.
|
||||
if (invocation.windowFrame != nullptr && !*invocation.windowDrawn) {
|
||||
window_mask::draw(pass, *invocation.windowFrame, invocation.stereoEye, invocation.windowScreen);
|
||||
*invocation.windowDrawn = true;
|
||||
}
|
||||
pass.End();
|
||||
}
|
||||
|
||||
@@ -2061,6 +2082,7 @@ void seal_frame(SealedFrame& out) noexcept {
|
||||
recycle_render_passes(passes);
|
||||
passes.swap(g_renderPasses);
|
||||
g_currentRenderPass = UINT32_MAX;
|
||||
out.data().windowScreen = stereo_window_screen();
|
||||
}
|
||||
|
||||
void render(SealedFrame& frame, wgpu::CommandEncoder& cmd, int32_t interpolatedFrame, bool finalize,
|
||||
@@ -2211,6 +2233,7 @@ void render_stereo_eye(SealedFrame& frame, wgpu::CommandEncoder& cmd, const Ster
|
||||
}
|
||||
bool cockpitDrawn = false;
|
||||
bool sceneDrawn = false;
|
||||
bool windowDrawn = false;
|
||||
const bool cockpitActive = stereoFrame.cockpit.active && cockpitDepth.valid;
|
||||
const bool skipCopyClears = get_stereo_skip_copy_clears();
|
||||
const eye_pass_plan::Plan* plan =
|
||||
@@ -2234,6 +2257,9 @@ void render_stereo_eye(SealedFrame& frame, wgpu::CommandEncoder& cmd, const Ster
|
||||
.cockpitDrawn = &cockpitDrawn,
|
||||
.sceneDrawn = &sceneDrawn,
|
||||
.eyePlan = plan,
|
||||
.windowFrame = stereoFrame.window ? &stereoFrame : nullptr,
|
||||
.windowScreen = frame.data().windowScreen,
|
||||
.windowDrawn = &windowDrawn,
|
||||
});
|
||||
// A frame without a virtual-screen draw after its world still gets the
|
||||
// overlay, in a pass of its own over the finished eye. A planned eye draws it
|
||||
@@ -2241,6 +2267,16 @@ void render_stereo_eye(SealedFrame& frame, wgpu::CommandEncoder& cmd, const Ster
|
||||
if (cockpitActive && !cockpitDrawn) {
|
||||
cockpit::render(cmd, stereoFrame, eye, cockpitDepth);
|
||||
}
|
||||
// Likewise the window's mask, which has to come after the cockpit.
|
||||
if (stereoFrame.window && !windowDrawn) {
|
||||
window_mask::render(cmd, stereoFrame, eye, frame.data().windowScreen);
|
||||
}
|
||||
}
|
||||
|
||||
void mask_stereo_eye_output(const SealedFrame& frame, wgpu::CommandEncoder& cmd, const StereoReplayFrame& stereoFrame,
|
||||
uint32_t eye, const wgpu::TextureView& output, wgpu::Extent3D size) {
|
||||
CHECK(eye < AURORA_STEREO_EYE_COUNT, "invalid stereo eye {}", eye);
|
||||
window_mask::render_output(cmd, stereoFrame, eye, output, size, frame.data().windowScreen);
|
||||
}
|
||||
|
||||
void render(wgpu::CommandEncoder& cmd, int32_t interpolatedFrame, bool finalize) {
|
||||
|
||||
@@ -321,6 +321,9 @@ struct StereoReplayFrame {
|
||||
std::array<StereoReplayEye, AURORA_STEREO_EYE_COUNT> eyes;
|
||||
// VR hands and synthetic wheel, drawn per eye after the world (gfx/cockpit.hpp).
|
||||
AuroraCockpit cockpit{};
|
||||
// The immersive window (AuroraStereoFrame::window): each eye is masked to the
|
||||
// 2D layer's screen after its last draw (gfx/window_mask.hpp).
|
||||
bool window = false;
|
||||
};
|
||||
|
||||
void end_frame(const wgpu::CommandEncoder& cmd);
|
||||
@@ -380,6 +383,12 @@ int32_t last_pass_feeding_replay(const SealedFrame& frame) noexcept;
|
||||
// mutated by stereo replay.
|
||||
void render_stereo_eye(SealedFrame& frame, wgpu::CommandEncoder& cmd,
|
||||
const StereoReplayFrame& stereoFrame, uint32_t eye, bool finalize = false);
|
||||
// The immersive window's mask on an eye image that holds the duplicated mono
|
||||
// picture instead of a replay (a windowed frame whose stereo replay could not
|
||||
// be prepared), so the compositor never blends an undefined alpha channel.
|
||||
void mask_stereo_eye_output(const SealedFrame& frame, wgpu::CommandEncoder& cmd,
|
||||
const StereoReplayFrame& stereoFrame, uint32_t eye,
|
||||
const wgpu::TextureView& output, wgpu::Extent3D size);
|
||||
|
||||
// Encode the frame that is still being recorded. Only for the synchronous
|
||||
// EFB-readback split path, which runs on the producer thread.
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
#pragma once
|
||||
|
||||
#include <aurora/math.hpp>
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
|
||||
namespace aurora::gfx::stereo_replay {
|
||||
@@ -364,4 +365,108 @@ inline Mat4x4<float> overlay_panel_flat_projection(float widthFraction, float pa
|
||||
return out;
|
||||
}
|
||||
|
||||
// The immersive window (AuroraStereoFrame::window): the 2D layer's screen as an
|
||||
// opening each eye sees the race through, the rest of the eye left transparent.
|
||||
//
|
||||
// Each row, applied to (x, y, 1) for a point of the eye image at NDC (x, y),
|
||||
// gives one of the homogeneous coordinates (u, v, w) of where that pixel's ray
|
||||
// meets the screen's plane: the point (u / w, v / w), in units of the screen's
|
||||
// half extents, so the screen covers -1..1 on both axes, lying in front of the
|
||||
// eye exactly when w > 0. The rows are linear in NDC, so a full-screen triangle
|
||||
// carrying their values at its corners interpolates them exactly. An eye on or
|
||||
// behind the screen's plane gets all-zero rows and sees nothing through it.
|
||||
struct WindowMask {
|
||||
Vec3<float> u;
|
||||
Vec3<float> v;
|
||||
Vec3<float> w;
|
||||
|
||||
[[nodiscard]] Vec3<float> at(float x, float y) const noexcept {
|
||||
return {u.x * x + u.y * y + u.z, v.x * x + v.y * y + v.z, w.x * x + w.y * y + w.z};
|
||||
}
|
||||
};
|
||||
|
||||
// The screen is the one compose_hud_screen_projection places the 2D layer on:
|
||||
// halfWidth by halfHeight, `distance` straight ahead in the recorded center-eye
|
||||
// view space, reached through viewFromCenter and the eye frustum's four terms.
|
||||
inline WindowMask window_mask(const Mat4x4<float>& eyeFrustum, const Mat3x4<float>& viewFromCenter,
|
||||
const HudScreen& screen) noexcept {
|
||||
const float sx = eyeFrustum.m0[0];
|
||||
const float sy = eyeFrustum.m1[1];
|
||||
if (!screen.valid() || sx == 0.0f || sy == 0.0f) {
|
||||
return {};
|
||||
}
|
||||
// The inverse of viewFromCenter's linear part L, by its adjugate.
|
||||
const auto& r0 = viewFromCenter.m0;
|
||||
const auto& r1 = viewFromCenter.m1;
|
||||
const auto& r2 = viewFromCenter.m2;
|
||||
float inverse[3][3] = {
|
||||
{r1[1] * r2[2] - r1[2] * r2[1], r0[2] * r2[1] - r0[1] * r2[2], r0[1] * r1[2] - r0[2] * r1[1]},
|
||||
{r1[2] * r2[0] - r1[0] * r2[2], r0[0] * r2[2] - r0[2] * r2[0], r0[2] * r1[0] - r0[0] * r1[2]},
|
||||
{r1[0] * r2[1] - r1[1] * r2[0], r0[1] * r2[0] - r0[0] * r2[1], r0[0] * r1[1] - r0[1] * r1[0]},
|
||||
};
|
||||
const float determinant = r0[0] * inverse[0][0] + r0[1] * inverse[1][0] + r0[2] * inverse[2][0];
|
||||
if (determinant == 0.0f) {
|
||||
return {};
|
||||
}
|
||||
for (auto& row : inverse) {
|
||||
for (float& value : row) {
|
||||
value /= determinant;
|
||||
}
|
||||
}
|
||||
// Window coordinates of an eye-space point p are q = A p + b: back into the
|
||||
// center-eye space, moved to the screen's centre and divided by its half
|
||||
// extents. The screen's plane is q.z = 0, its front facing the camera.
|
||||
const float scale[3] = {1.0f / screen.halfWidth, 1.0f / screen.halfHeight, 1.0f};
|
||||
const float t[3] = {r0[3], r1[3], r2[3]};
|
||||
float A[3][3];
|
||||
float b[3];
|
||||
for (int row = 0; row < 3; ++row) {
|
||||
float back = 0.0f;
|
||||
for (int column = 0; column < 3; ++column) {
|
||||
A[row][column] = inverse[row][column] * scale[row];
|
||||
back += inverse[row][column] * t[column];
|
||||
}
|
||||
b[row] = (-back + (row == 2 ? screen.distance : 0.0f)) * scale[row];
|
||||
}
|
||||
if (!(b[2] > 0.0f)) {
|
||||
return {};
|
||||
}
|
||||
// The pixel's ray is d = K (x, y, 1) with d.z = -1: the frustum maps an eye
|
||||
// point to clip x = sx * x + m0[2] * z, y = sy * y + m1[2] * z, w = -z.
|
||||
const float K[3][3] = {
|
||||
{1.0f / sx, 0.0f, eyeFrustum.m0[2] / sx},
|
||||
{0.0f, 1.0f / sy, eyeFrustum.m1[2] / sy},
|
||||
{0.0f, 0.0f, -1.0f},
|
||||
};
|
||||
// a = M (x, y, 1) is the ray in window coordinates. It meets the plane at
|
||||
// q = b + s a with s = -b.z / a.z, in front of the eye when s > 0, i.e. when
|
||||
// a.z < 0, so (u, v, w) = (b.z a.x - b.x a.z, b.z a.y - b.y a.z, -a.z).
|
||||
float M[3][3];
|
||||
for (int row = 0; row < 3; ++row) {
|
||||
for (int column = 0; column < 3; ++column) {
|
||||
M[row][column] = A[row][0] * K[0][column] + A[row][1] * K[1][column] + A[row][2] * K[2][column];
|
||||
}
|
||||
}
|
||||
float rows[3][3];
|
||||
float largest = 0.0f;
|
||||
for (int column = 0; column < 3; ++column) {
|
||||
rows[0][column] = b[2] * M[0][column] - b[0] * M[2][column];
|
||||
rows[1][column] = b[2] * M[1][column] - b[1] * M[2][column];
|
||||
rows[2][column] = -M[2][column];
|
||||
for (const auto& row : rows) {
|
||||
largest = std::max(largest, std::abs(row[column]));
|
||||
}
|
||||
}
|
||||
if (!(largest > 0.0f)) {
|
||||
return {};
|
||||
}
|
||||
// Only the ratios matter; a positive scale keeps the values near one whatever
|
||||
// the world units are.
|
||||
const float normalize = 1.0f / largest;
|
||||
const auto out = [&](const float (&row)[3]) {
|
||||
return Vec3<float>{row[0] * normalize, row[1] * normalize, row[2] * normalize};
|
||||
};
|
||||
return {.u = out(rows[0]), .v = out(rows[1]), .w = out(rows[2])};
|
||||
}
|
||||
|
||||
} // namespace aurora::gfx::stereo_replay
|
||||
@@ -0,0 +1,208 @@
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
//
|
||||
// The immersive window (AuroraStereoFrame::window): after an eye's last draw,
|
||||
// one full-screen triangle keeps what the eye sees through the 2D layer's
|
||||
// screen and makes the rest transparent black, so the compositor shows its own
|
||||
// background (the room, with passthrough) around the race. See OPENXR.md, "The
|
||||
// immersive window".
|
||||
#pragma once
|
||||
|
||||
#include "common.hpp"
|
||||
#include "stereo_replay.hpp"
|
||||
#include "../webgpu/gpu.hpp"
|
||||
|
||||
#include <array>
|
||||
#include <cstddef>
|
||||
#include <utility>
|
||||
|
||||
namespace aurora::gfx::window_mask {
|
||||
|
||||
struct Vertex {
|
||||
float position[2];
|
||||
// stereo_replay::WindowMask's (u, v, w) at this corner.
|
||||
float window[3];
|
||||
};
|
||||
|
||||
struct PipelineKey {
|
||||
wgpu::TextureFormat format = wgpu::TextureFormat::Undefined;
|
||||
// Undefined: a pass with no depth attachment.
|
||||
wgpu::TextureFormat depthFormat = wgpu::TextureFormat::Undefined;
|
||||
uint32_t samples = 0;
|
||||
|
||||
bool operator==(const PipelineKey&) const = default;
|
||||
};
|
||||
|
||||
// The eye passes (with depth) use one attachment layout and the mono fallback (without) another;
|
||||
// each keeps its own pipeline, rebuilt when its format or sample count changes.
|
||||
inline std::array<std::pair<PipelineKey, wgpu::RenderPipeline>, 2> pipelines;
|
||||
inline std::array<wgpu::Buffer, AURORA_STEREO_EYE_COUNT> vertexBuffers;
|
||||
|
||||
inline void shutdown() {
|
||||
pipelines = {};
|
||||
vertexBuffers = {};
|
||||
}
|
||||
|
||||
inline const wgpu::RenderPipeline& pipeline(const PipelineKey& key) {
|
||||
for (const auto& [cachedKey, cached] : pipelines) {
|
||||
if (cached && cachedKey == key) {
|
||||
return cached;
|
||||
}
|
||||
}
|
||||
wgpu::ShaderSourceWGSL source{};
|
||||
source.code = R"(
|
||||
struct Out { @builtin(position) position: vec4f, @location(0) window: vec3f };
|
||||
@vertex fn vs(@location(0) position: vec2f, @location(1) window: vec3f) -> Out {
|
||||
var o: Out;
|
||||
o.position = vec4f(position, 0.5, 1.0);
|
||||
o.window = window;
|
||||
return o;
|
||||
}
|
||||
@fragment fn fs(i: Out) -> @location(0) vec4f {
|
||||
// Where this pixel's ray meets the screen's plane, in its half extents.
|
||||
let p = i.window.xy / i.window.z;
|
||||
let edge = 1.0 - abs(p);
|
||||
// One pixel of coverage ramp at the edge, from the screen-space rate of change.
|
||||
let ramp = clamp(edge / max(fwidth(edge), vec2f(1e-6)) + 0.5, vec2f(0.0), vec2f(1.0));
|
||||
// Behind the eye (w <= 0), or far off the screen where the plane nears the horizon.
|
||||
let seen = i.window.z > 0.0 && all(abs(p) < vec2f(2.0));
|
||||
return vec4f(0.0, 0.0, 0.0, select(0.0, ramp.x * ramp.y, seen));
|
||||
}
|
||||
)";
|
||||
wgpu::ShaderModuleDescriptor moduleDescriptor{};
|
||||
moduleDescriptor.nextInChain = &source;
|
||||
moduleDescriptor.label = "VR immersive window mask";
|
||||
const auto shader = webgpu::g_device.CreateShaderModule(&moduleDescriptor);
|
||||
const wgpu::VertexAttribute attributes[] = {
|
||||
{.format = wgpu::VertexFormat::Float32x2, .offset = 0, .shaderLocation = 0},
|
||||
{.format = wgpu::VertexFormat::Float32x3, .offset = offsetof(Vertex, window), .shaderLocation = 1},
|
||||
};
|
||||
const wgpu::VertexBufferLayout layout{.arrayStride = sizeof(Vertex), .attributeCount = 2, .attributes = attributes};
|
||||
// Premultiplied alpha: the colour is scaled by the coverage the fragment returns as alpha, and the
|
||||
// alpha becomes that coverage, whatever the game left there.
|
||||
const wgpu::BlendState blend{
|
||||
.color = {.operation = wgpu::BlendOperation::Add,
|
||||
.srcFactor = wgpu::BlendFactor::Zero,
|
||||
.dstFactor = wgpu::BlendFactor::SrcAlpha},
|
||||
.alpha = {.operation = wgpu::BlendOperation::Add,
|
||||
.srcFactor = wgpu::BlendFactor::One,
|
||||
.dstFactor = wgpu::BlendFactor::Zero},
|
||||
};
|
||||
const wgpu::ColorTargetState color{.format = key.format, .blend = &blend};
|
||||
const wgpu::FragmentState fragment{.module = shader, .entryPoint = "fs", .targetCount = 1, .targets = &color};
|
||||
const wgpu::DepthStencilState depth{
|
||||
.format = key.depthFormat,
|
||||
.depthWriteEnabled = false,
|
||||
.depthCompare = wgpu::CompareFunction::Always,
|
||||
.stencilReadMask = 0,
|
||||
.stencilWriteMask = 0,
|
||||
};
|
||||
wgpu::RenderPipelineDescriptor descriptor{};
|
||||
descriptor.label = "VR immersive window mask";
|
||||
descriptor.vertex = {.module = shader, .entryPoint = "vs", .bufferCount = 1, .buffers = &layout};
|
||||
descriptor.fragment = &fragment;
|
||||
descriptor.depthStencil = key.depthFormat == wgpu::TextureFormat::Undefined ? nullptr : &depth;
|
||||
descriptor.multisample.count = key.samples;
|
||||
descriptor.primitive.topology = wgpu::PrimitiveTopology::TriangleList;
|
||||
auto& slot = key.depthFormat == wgpu::TextureFormat::Undefined ? pipelines[1] : pipelines[0];
|
||||
slot = {key, webgpu::g_device.CreateRenderPipeline(&descriptor)};
|
||||
return slot.second;
|
||||
}
|
||||
|
||||
// Writes the eye's triangle: NDC corners (-1, -1), (3, -1) and (-1, 3) cover the whole image, with
|
||||
// the mask's rows evaluated at each so they interpolate across it exactly.
|
||||
inline const wgpu::Buffer& eye_vertices(uint32_t eye, const stereo_replay::WindowMask& mask) {
|
||||
static constexpr std::array<std::array<float, 2>, 3> kCorners{{{-1.0f, -1.0f}, {3.0f, -1.0f}, {-1.0f, 3.0f}}};
|
||||
std::array<Vertex, 3> vertices{};
|
||||
for (size_t i = 0; i < vertices.size(); ++i) {
|
||||
const auto h = mask.at(kCorners[i][0], kCorners[i][1]);
|
||||
vertices[i] = {{kCorners[i][0], kCorners[i][1]}, {h.x, h.y, h.z}};
|
||||
}
|
||||
auto& buffer = vertexBuffers[eye];
|
||||
if (!buffer) {
|
||||
const wgpu::BufferDescriptor descriptor{
|
||||
.label = "VR immersive window mask vertices",
|
||||
.usage = wgpu::BufferUsage::Vertex | wgpu::BufferUsage::CopyDst,
|
||||
.size = sizeof(vertices),
|
||||
};
|
||||
buffer = webgpu::g_device.CreateBuffer(&descriptor);
|
||||
}
|
||||
webgpu::g_queue.WriteBuffer(buffer, 0, vertices.data(), sizeof(vertices));
|
||||
return buffer;
|
||||
}
|
||||
|
||||
// Masks one eye inside a render pass already open on its attachments, after everything else it draws.
|
||||
inline void draw(const wgpu::RenderPassEncoder& pass, const StereoReplayFrame& frame, uint32_t eye,
|
||||
const stereo_replay::HudScreen& screen) {
|
||||
const auto& view = frame.eyes[eye];
|
||||
const auto& target = view.target;
|
||||
const PipelineKey key{
|
||||
.format = webgpu::g_graphicsConfig.surfaceConfiguration.format,
|
||||
.depthFormat = target.depthFormat,
|
||||
.samples = target.msaaSamples,
|
||||
};
|
||||
const auto& buffer = eye_vertices(eye, stereo_replay::window_mask(view.projection, view.viewFromCenter, screen));
|
||||
pass.SetViewport(0.0f, 0.0f, static_cast<float>(target.size.width), static_cast<float>(target.size.height), 0.0f,
|
||||
1.0f);
|
||||
pass.SetScissorRect(0, 0, target.size.width, target.size.height);
|
||||
pass.SetPipeline(pipeline(key));
|
||||
pass.SetVertexBuffer(0, buffer);
|
||||
pass.Draw(3);
|
||||
}
|
||||
|
||||
// The same in a render pass of its own, over the finished eye's attachments.
|
||||
inline void render(wgpu::CommandEncoder& cmd, const StereoReplayFrame& frame, uint32_t eye,
|
||||
const stereo_replay::HudScreen& screen) {
|
||||
const auto& target = frame.eyes[eye].target;
|
||||
const bool stencil = target.depthFormat == wgpu::TextureFormat::Depth24PlusStencil8;
|
||||
const wgpu::RenderPassColorAttachment color{
|
||||
.view = target.colorView,
|
||||
.resolveTarget = target.resolveView,
|
||||
.loadOp = wgpu::LoadOp::Load,
|
||||
.storeOp = wgpu::StoreOp::Store,
|
||||
};
|
||||
const wgpu::RenderPassDepthStencilAttachment depth{
|
||||
.view = target.depthView,
|
||||
.depthLoadOp = wgpu::LoadOp::Load,
|
||||
.depthStoreOp = wgpu::StoreOp::Store,
|
||||
.stencilLoadOp = stencil ? wgpu::LoadOp::Load : wgpu::LoadOp::Undefined,
|
||||
.stencilStoreOp = stencil ? wgpu::StoreOp::Store : wgpu::StoreOp::Undefined,
|
||||
};
|
||||
const wgpu::RenderPassDescriptor descriptor{
|
||||
.label = "VR immersive window mask",
|
||||
.colorAttachmentCount = 1,
|
||||
.colorAttachments = &color,
|
||||
.depthStencilAttachment = &depth,
|
||||
};
|
||||
const auto pass = cmd.BeginRenderPass(&descriptor);
|
||||
draw(pass, frame, eye, screen);
|
||||
pass.End();
|
||||
}
|
||||
|
||||
// Masks an eye image that holds the duplicated mono picture (a frame whose stereo replay could not
|
||||
// be prepared): only the resolved output exists there, with no depth.
|
||||
inline void render_output(wgpu::CommandEncoder& cmd, const StereoReplayFrame& frame, uint32_t eye,
|
||||
const wgpu::TextureView& output, wgpu::Extent3D size,
|
||||
const stereo_replay::HudScreen& screen) {
|
||||
const auto& view = frame.eyes[eye];
|
||||
const PipelineKey key{.format = webgpu::g_graphicsConfig.surfaceConfiguration.format, .samples = 1};
|
||||
const auto& buffer = eye_vertices(eye, stereo_replay::window_mask(view.projection, view.viewFromCenter, screen));
|
||||
const wgpu::RenderPassColorAttachment color{
|
||||
.view = output,
|
||||
.loadOp = wgpu::LoadOp::Load,
|
||||
.storeOp = wgpu::StoreOp::Store,
|
||||
};
|
||||
const wgpu::RenderPassDescriptor descriptor{
|
||||
.label = "VR immersive window mask",
|
||||
.colorAttachmentCount = 1,
|
||||
.colorAttachments = &color,
|
||||
};
|
||||
const auto pass = cmd.BeginRenderPass(&descriptor);
|
||||
pass.SetViewport(0.0f, 0.0f, static_cast<float>(size.width), static_cast<float>(size.height), 0.0f, 1.0f);
|
||||
pass.SetScissorRect(0, 0, size.width, size.height);
|
||||
pass.SetPipeline(pipeline(key));
|
||||
pass.SetVertexBuffer(0, buffer);
|
||||
pass.Draw(3);
|
||||
pass.End();
|
||||
}
|
||||
|
||||
} // namespace aurora::gfx::window_mask
|
||||
@@ -34,6 +34,12 @@ if (AURORA_GPU_SMOKE_TESTS AND AURORA_ENABLE_GX AND WIN32)
|
||||
target_compile_definitions(cockpit_gpu_smoke PRIVATE AURORA TARGET_PC WEBGPU_DAWN)
|
||||
target_link_libraries(cockpit_gpu_smoke PRIVATE fmt::fmt xxhash absl::flat_hash_map absl::btree
|
||||
dawn::webgpu_dawn dawn::dawncpp_headers TracyClient ${AURORA_SDL3_TARGET})
|
||||
# The immersive window's mask over a filled eye, read back. Standalone like the cockpit's.
|
||||
add_executable(window_mask_gpu_smoke window_mask_gpu_smoke.cpp)
|
||||
target_include_directories(window_mask_gpu_smoke PRIVATE ../include ../lib)
|
||||
target_compile_definitions(window_mask_gpu_smoke PRIVATE AURORA TARGET_PC WEBGPU_DAWN)
|
||||
target_link_libraries(window_mask_gpu_smoke PRIVATE fmt::fmt xxhash absl::flat_hash_map absl::btree
|
||||
dawn::webgpu_dawn dawn::dawncpp_headers TracyClient ${AURORA_SDL3_TARGET})
|
||||
endif ()
|
||||
|
||||
if (NOT TARGET gtest)
|
||||
|
||||
@@ -511,5 +511,134 @@ TEST(StereoReplayTest, MirroringKeepsEachEyeOnItsOwnSide) {
|
||||
EXPECT_LT(ndcX[1], 0.0f);
|
||||
}
|
||||
|
||||
Mat4x4<float> asymmetric_eye_frustum() {
|
||||
Mat4x4<float> eyeFrustum{};
|
||||
eyeFrustum.m0 = {1.15f, 0.0f, 0.08f, 0.0f};
|
||||
eyeFrustum.m1 = {0.0f, 1.02f, -0.03f, 0.0f};
|
||||
return eyeFrustum;
|
||||
}
|
||||
|
||||
// A head turned and pitched a little, offset from the recorded center eye.
|
||||
Mat3x4<float> turned_head() {
|
||||
const float yaw = 0.3f;
|
||||
const float pitch = -0.12f;
|
||||
const float cy = std::cos(yaw);
|
||||
const float sy = std::sin(yaw);
|
||||
const float cp = std::cos(pitch);
|
||||
const float sp = std::sin(pitch);
|
||||
// Pitch about X after yaw about Y.
|
||||
Mat3x4<float> m{};
|
||||
m.m0 = {cy, 0.0f, sy, 15.0f};
|
||||
m.m1 = {sp * sy, cp, -sp * cy, -4.0f};
|
||||
m.m2 = {-cp * sy, sp, cp * cy, 7.0f};
|
||||
return m;
|
||||
}
|
||||
|
||||
// Where a point of the center-eye space lands in the eye image, in NDC.
|
||||
std::array<float, 2> eye_ndc(const Mat4x4<float>& eyeFrustum, const Mat3x4<float>& viewFromCenter,
|
||||
const Vec4<float>& centerPoint) {
|
||||
const float eyeX = dot4(viewFromCenter.m0, centerPoint);
|
||||
const float eyeY = dot4(viewFromCenter.m1, centerPoint);
|
||||
const float eyeZ = dot4(viewFromCenter.m2, centerPoint);
|
||||
EXPECT_LT(eyeZ, 0.0f);
|
||||
return {(eyeFrustum.m0[0] * eyeX + eyeFrustum.m0[2] * eyeZ) / -eyeZ,
|
||||
(eyeFrustum.m1[1] * eyeY + eyeFrustum.m1[2] * eyeZ) / -eyeZ};
|
||||
}
|
||||
|
||||
TEST(StereoReplayTest, WindowMaskFindsTheScreenThroughTheEye) {
|
||||
const auto eyeFrustum = asymmetric_eye_frustum();
|
||||
const auto viewFromCenter = turned_head();
|
||||
const HudScreen screen{.halfWidth = 600.0f, .halfHeight = 337.5f, .distance = 1000.0f};
|
||||
const auto mask = window_mask(eyeFrustum, viewFromCenter, screen);
|
||||
|
||||
// Corners, centre, and points just inside and outside the edges, in half extents.
|
||||
const std::array<std::array<float, 2>, 9> points{{
|
||||
{-1.0f, 1.0f},
|
||||
{1.0f, 1.0f},
|
||||
{-1.0f, -1.0f},
|
||||
{1.0f, -1.0f},
|
||||
{0.0f, 0.0f},
|
||||
{0.95f, 0.2f},
|
||||
{1.05f, 0.2f},
|
||||
{-0.3f, -0.97f},
|
||||
{-0.3f, -1.04f},
|
||||
}};
|
||||
for (const auto& point : points) {
|
||||
const Vec4<float> centerPoint{point[0] * screen.halfWidth, point[1] * screen.halfHeight, -screen.distance, 1.0f};
|
||||
const auto ndc = eye_ndc(eyeFrustum, viewFromCenter, centerPoint);
|
||||
const auto h = mask.at(ndc[0], ndc[1]);
|
||||
ASSERT_GT(h.z, 0.0f);
|
||||
EXPECT_NEAR(h.x / h.z, point[0], 1e-4f);
|
||||
EXPECT_NEAR(h.y / h.z, point[1], 1e-4f);
|
||||
}
|
||||
}
|
||||
|
||||
TEST(StereoReplayTest, WindowMaskCoincidesWithTheHudOnTheSameScreen) {
|
||||
// The 2D layer is drawn on the window, so a HUD vertex must land at its own
|
||||
// game NDC on the window: the mask and compose_hud_screen_projection agree.
|
||||
const auto game = game_orthographic_projection();
|
||||
const auto eyeFrustum = asymmetric_eye_frustum();
|
||||
const auto viewFromCenter = turned_head();
|
||||
const HudScreen screen{.halfWidth = 600.0f, .halfHeight = 337.5f, .distance = 1000.0f};
|
||||
const auto composed = compose_hud_screen_projection(eyeFrustum, viewFromCenter, screen, game);
|
||||
const auto mask = window_mask(eyeFrustum, viewFromCenter, screen);
|
||||
for (const auto& v : kVertices) {
|
||||
const float w = dot4(composed.m3, v);
|
||||
ASSERT_GT(w, 0.0f);
|
||||
const auto h = mask.at(dot4(composed.m0, v) / w, dot4(composed.m1, v) / w);
|
||||
ASSERT_GT(h.z, 0.0f);
|
||||
EXPECT_NEAR(h.x / h.z, dot4(game.m0, v), 1e-4f);
|
||||
EXPECT_NEAR(h.y / h.z, dot4(game.m1, v), 1e-4f);
|
||||
}
|
||||
}
|
||||
|
||||
TEST(StereoReplayTest, WindowMaskShowsNothingBehindTheEye) {
|
||||
const auto eyeFrustum = asymmetric_eye_frustum();
|
||||
const HudScreen screen{.halfWidth = 600.0f, .halfHeight = 337.5f, .distance = 1000.0f};
|
||||
|
||||
// Turned right round: the screen is behind the eye, so no pixel's ray meets it.
|
||||
Mat3x4<float> turnedAway{};
|
||||
turnedAway.m0 = {-1.0f, 0.0f, 0.0f, 0.0f};
|
||||
turnedAway.m1 = {0.0f, 1.0f, 0.0f, 0.0f};
|
||||
turnedAway.m2 = {0.0f, 0.0f, -1.0f, 0.0f};
|
||||
const auto away = window_mask(eyeFrustum, turnedAway, screen);
|
||||
for (float x = -1.0f; x <= 1.0f; x += 0.25f) {
|
||||
for (float y = -1.0f; y <= 1.0f; y += 0.25f) {
|
||||
EXPECT_LE(away.at(x, y).z, 0.0f);
|
||||
}
|
||||
}
|
||||
|
||||
// Walked through the screen: every row is zero, so every pixel is outside.
|
||||
Mat3x4<float> beyond = identity3x4();
|
||||
beyond.m2[3] = screen.distance + 100.0f;
|
||||
const auto through = window_mask(eyeFrustum, beyond, screen);
|
||||
EXPECT_EQ(through.w, Vec3<float>{});
|
||||
EXPECT_EQ(through.u, Vec3<float>{});
|
||||
|
||||
// No screen at all: nothing either.
|
||||
const auto none = window_mask(eyeFrustum, identity3x4(), HudScreen{});
|
||||
EXPECT_EQ(none.w, Vec3<float>{});
|
||||
}
|
||||
|
||||
TEST(StereoReplayTest, WindowMaskStaysFixedInSpaceAsTheEyeMoves) {
|
||||
// Stepping sideways moves the screen across the eye image the other way, as
|
||||
// a real window would: the screen's centre is no longer at the image centre.
|
||||
const auto eyeFrustum = eye_frustum(0.0f);
|
||||
const HudScreen screen{.halfWidth = 600.0f, .halfHeight = 337.5f, .distance = 1000.0f};
|
||||
Mat3x4<float> stepRight = identity3x4();
|
||||
stepRight.m0[3] = -200.0f; // The eye moves +200 to the right, so the world moves back.
|
||||
const auto mask = window_mask(eyeFrustum, stepRight, screen);
|
||||
const auto centre = eye_ndc(eyeFrustum, stepRight, {0.0f, 0.0f, -screen.distance, 1.0f});
|
||||
EXPECT_LT(centre[0], 0.0f);
|
||||
const auto h = mask.at(centre[0], centre[1]);
|
||||
ASSERT_GT(h.z, 0.0f);
|
||||
EXPECT_NEAR(h.x / h.z, 0.0f, 1e-4f);
|
||||
EXPECT_NEAR(h.y / h.z, 0.0f, 1e-4f);
|
||||
// The image centre now looks through the right part of the screen.
|
||||
const auto straight = mask.at(0.0f, 0.0f);
|
||||
ASSERT_GT(straight.z, 0.0f);
|
||||
EXPECT_NEAR(straight.x / straight.z, 200.0f / screen.halfWidth, 1e-4f);
|
||||
}
|
||||
|
||||
} // namespace
|
||||
} // namespace aurora::gfx::stereo_replay
|
||||
@@ -0,0 +1,180 @@
|
||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||
// Draws the immersive window's mask (gfx/window_mask.hpp) over a filled eye on a real GPU, in the
|
||||
// eye's own render pass and in passes of its own, 1x and 4x MSAA, and reads the image back: the
|
||||
// window keeps its colour with alpha 1, everything outside it becomes transparent black.
|
||||
#include "../lib/gfx/window_mask.hpp"
|
||||
|
||||
#include <atomic>
|
||||
#include <cstdlib>
|
||||
#include <iostream>
|
||||
#include <string_view>
|
||||
|
||||
namespace aurora::webgpu {
|
||||
wgpu::Device g_device;
|
||||
wgpu::Queue g_queue;
|
||||
GraphicsConfig g_graphicsConfig{};
|
||||
} // namespace aurora::webgpu
|
||||
|
||||
namespace {
|
||||
std::atomic<int> errors = 0;
|
||||
constexpr uint32_t kSize = 256;
|
||||
|
||||
enum class Path { InPass, OwnPass, Output };
|
||||
|
||||
struct Pixel {
|
||||
int r, g, b, a;
|
||||
};
|
||||
} // namespace
|
||||
|
||||
int main() {
|
||||
using namespace aurora;
|
||||
using namespace aurora::webgpu;
|
||||
wgpu::InstanceDescriptor instanceDescriptor{};
|
||||
const wgpu::InstanceFeatureName timed = wgpu::InstanceFeatureName::TimedWaitAny;
|
||||
instanceDescriptor.requiredFeatureCount = 1;
|
||||
instanceDescriptor.requiredFeatures = &timed;
|
||||
auto instance = wgpu::CreateInstance(&instanceDescriptor);
|
||||
wgpu::Adapter adapter;
|
||||
wgpu::RequestAdapterOptions options{.backendType = wgpu::BackendType::D3D12};
|
||||
auto future = instance.RequestAdapter(&options, wgpu::CallbackMode::WaitAnyOnly,
|
||||
[&](wgpu::RequestAdapterStatus status, wgpu::Adapter a, wgpu::StringView message) {
|
||||
if (status == wgpu::RequestAdapterStatus::Success)
|
||||
adapter = std::move(a);
|
||||
else
|
||||
std::cerr << std::string_view(message) << '\n';
|
||||
});
|
||||
if (instance.WaitAny(future, 5000000000) != wgpu::WaitStatus::Success || !adapter)
|
||||
return 1;
|
||||
wgpu::DeviceDescriptor deviceDescriptor{};
|
||||
deviceDescriptor.SetUncapturedErrorCallback([](const wgpu::Device&, wgpu::ErrorType, wgpu::StringView message) {
|
||||
++errors;
|
||||
std::cerr << std::string_view(message) << '\n';
|
||||
});
|
||||
future = adapter.RequestDevice(&deviceDescriptor, wgpu::CallbackMode::WaitAnyOnly,
|
||||
[&](wgpu::RequestDeviceStatus status, wgpu::Device device, wgpu::StringView message) {
|
||||
if (status == wgpu::RequestDeviceStatus::Success)
|
||||
g_device = std::move(device);
|
||||
else
|
||||
std::cerr << std::string_view(message) << '\n';
|
||||
});
|
||||
if (instance.WaitAny(future, 5000000000) != wgpu::WaitStatus::Success || !g_device)
|
||||
return 1;
|
||||
g_queue = g_device.GetQueue();
|
||||
g_graphicsConfig.surfaceConfiguration.format = wgpu::TextureFormat::RGBA8Unorm;
|
||||
|
||||
// A 90-degree eye looking at a screen 1 unit ahead, 1 across and 0.5 high: the window covers NDC
|
||||
// x in -0.5..0.5 and y in -0.25..0.25, pixels 64..192 across and 96..160 down.
|
||||
const gfx::stereo_replay::HudScreen screen{.halfWidth = 0.5f, .halfHeight = 0.25f, .distance = 1.0f};
|
||||
for (const Path path : {Path::InPass, Path::OwnPass, Path::Output})
|
||||
for (const uint32_t samples : {1u, 4u})
|
||||
for (const bool turnedAway : {false, true})
|
||||
for (const uint32_t eyeIndex : {0u, 1u}) {
|
||||
if (path == Path::Output && samples != 1)
|
||||
continue;
|
||||
gfx::StereoReplayFrame frame{};
|
||||
frame.window = true;
|
||||
wgpu::TextureDescriptor textureDescriptor{
|
||||
.usage = wgpu::TextureUsage::RenderAttachment | wgpu::TextureUsage::CopySrc,
|
||||
.size = {kSize, kSize, 1},
|
||||
.format = wgpu::TextureFormat::RGBA8Unorm,
|
||||
.sampleCount = 1,
|
||||
};
|
||||
auto output = g_device.CreateTexture(&textureDescriptor);
|
||||
textureDescriptor.sampleCount = samples;
|
||||
textureDescriptor.usage = wgpu::TextureUsage::RenderAttachment;
|
||||
auto color = g_device.CreateTexture(&textureDescriptor);
|
||||
textureDescriptor.format = wgpu::TextureFormat::Depth24PlusStencil8;
|
||||
auto depth = g_device.CreateTexture(&textureDescriptor);
|
||||
auto& eye = frame.eyes[eyeIndex];
|
||||
eye.target.colorView = samples == 1 ? output.CreateView() : color.CreateView();
|
||||
if (samples > 1)
|
||||
eye.target.resolveView = output.CreateView();
|
||||
eye.target.depthView = depth.CreateView();
|
||||
eye.target.depthFormat = wgpu::TextureFormat::Depth24PlusStencil8;
|
||||
eye.target.size = {kSize, kSize, 1};
|
||||
eye.target.msaaSamples = samples;
|
||||
eye.projection.m0[0] = 1.0f;
|
||||
eye.projection.m1[1] = 1.0f;
|
||||
eye.viewFromCenter.m0 = {turnedAway ? -1.0f : 1.0f, 0.0f, 0.0f, 0.0f};
|
||||
eye.viewFromCenter.m1 = {0.0f, 1.0f, 0.0f, 0.0f};
|
||||
eye.viewFromCenter.m2 = {0.0f, 0.0f, turnedAway ? -1.0f : 1.0f, 0.0f};
|
||||
|
||||
// The finished eye, with an alpha the game might leave anywhere.
|
||||
auto encoder = g_device.CreateCommandEncoder();
|
||||
const wgpu::RenderPassColorAttachment fill{.view = eye.target.colorView,
|
||||
.resolveTarget = eye.target.resolveView,
|
||||
.loadOp = wgpu::LoadOp::Clear,
|
||||
.storeOp = wgpu::StoreOp::Store,
|
||||
.clearValue = {0.5, 0.25, 0.75, 0.3}};
|
||||
const wgpu::RenderPassDepthStencilAttachment fillDepth{.view = eye.target.depthView,
|
||||
.depthLoadOp = wgpu::LoadOp::Clear,
|
||||
.depthStoreOp = wgpu::StoreOp::Store,
|
||||
.depthClearValue = 1.0f,
|
||||
.stencilLoadOp = wgpu::LoadOp::Clear,
|
||||
.stencilStoreOp = wgpu::StoreOp::Store};
|
||||
const wgpu::RenderPassDescriptor fillPass{
|
||||
.colorAttachmentCount = 1, .colorAttachments = &fill, .depthStencilAttachment = &fillDepth};
|
||||
auto pass = encoder.BeginRenderPass(&fillPass);
|
||||
if (path == Path::InPass)
|
||||
gfx::window_mask::draw(pass, frame, eyeIndex, screen);
|
||||
pass.End();
|
||||
if (path == Path::OwnPass)
|
||||
gfx::window_mask::render(encoder, frame, eyeIndex, screen);
|
||||
if (path == Path::Output)
|
||||
gfx::window_mask::render_output(encoder, frame, eyeIndex, output.CreateView(), {kSize, kSize, 1}, screen);
|
||||
|
||||
const wgpu::BufferDescriptor bufferDescriptor{.usage = wgpu::BufferUsage::CopyDst | wgpu::BufferUsage::MapRead,
|
||||
.size = kSize * kSize * 4};
|
||||
auto readback = g_device.CreateBuffer(&bufferDescriptor);
|
||||
const wgpu::TexelCopyTextureInfo source{.texture = output};
|
||||
const wgpu::TexelCopyBufferInfo destination{.layout = {.bytesPerRow = kSize * 4, .rowsPerImage = kSize},
|
||||
.buffer = readback};
|
||||
const wgpu::Extent3D extent{kSize, kSize, 1};
|
||||
encoder.CopyTextureToBuffer(&source, &destination, &extent);
|
||||
auto commands = encoder.Finish();
|
||||
g_queue.Submit(1, &commands);
|
||||
bool mapped = false;
|
||||
future = readback.MapAsync(wgpu::MapMode::Read, 0, kSize * kSize * 4, wgpu::CallbackMode::WaitAnyOnly,
|
||||
[&](wgpu::MapAsyncStatus status, wgpu::StringView) {
|
||||
mapped = status == wgpu::MapAsyncStatus::Success;
|
||||
});
|
||||
if (instance.WaitAny(future, 5000000000) != wgpu::WaitStatus::Success || !mapped)
|
||||
return 1;
|
||||
const auto* bytes = static_cast<const unsigned char*>(readback.GetConstMappedRange());
|
||||
const auto at = [&](uint32_t x, uint32_t y) {
|
||||
const auto* p = bytes + (y * kSize + x) * 4;
|
||||
return Pixel{p[0], p[1], p[2], p[3]};
|
||||
};
|
||||
const auto near = [](int value, int expected) { return std::abs(value - expected) <= 2; };
|
||||
const auto check = [&](uint32_t x, uint32_t y, bool inside) {
|
||||
const auto p = at(x, y);
|
||||
const bool ok = inside ? near(p.r, 128) && near(p.g, 64) && near(p.b, 191) && p.a == 255
|
||||
: p.r == 0 && p.g == 0 && p.b == 0 && p.a == 0;
|
||||
if (!ok) {
|
||||
std::cerr << "path " << static_cast<int>(path) << ", " << samples << "x, eye " << eyeIndex
|
||||
<< (turnedAway ? ", turned away" : "") << ": pixel (" << x << ", " << y << ") is (" << p.r
|
||||
<< ", " << p.g << ", " << p.b << ", " << p.a << "), expected "
|
||||
<< (inside ? "the eye's colour, opaque" : "transparent black") << '\n';
|
||||
++errors;
|
||||
}
|
||||
};
|
||||
const bool seen = !turnedAway;
|
||||
check(128, 128, seen);
|
||||
check(70, 100, seen);
|
||||
check(186, 155, seen);
|
||||
check(5, 5, false);
|
||||
check(58, 128, false);
|
||||
check(198, 128, false);
|
||||
check(128, 90, false);
|
||||
check(128, 166, false);
|
||||
check(250, 250, false);
|
||||
readback.Unmap();
|
||||
std::cout << "path " << static_cast<int>(path) << ", " << samples << "x MSAA, eye " << eyeIndex
|
||||
<< (turnedAway ? ", turned away" : "") << ": checked\n";
|
||||
}
|
||||
gfx::window_mask::shutdown();
|
||||
g_queue = nullptr;
|
||||
g_device.Destroy();
|
||||
g_device = nullptr;
|
||||
return errors ? 1 : 0;
|
||||
}
|
||||
+6
-2
@@ -250,7 +250,10 @@ suggested for `oculus/touch_controller` and `khr/simple_controller`.
|
||||
(created on first use) or pauses it as each presentation arrives, and submits
|
||||
it first, under the virtual screen's quad, or alone while there is no image
|
||||
yet (startup, a recenter). An immersive race never submits it and pauses the
|
||||
cameras; a `flat_screen` race is a virtual screen, so it keeps the room.
|
||||
cameras; a `flat_screen` race is a virtual screen, so it keeps the room, and
|
||||
so does an `immersive_window` race, whose projection layer is submitted over
|
||||
it with `XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT` (Aurora leaves
|
||||
each eye transparent outside the window; see OPENXR.md, "The immersive window").
|
||||
The quad is cropped to the snapshot Aurora letterboxes into the
|
||||
nearly square eye image (`OpenXRVirtualScreenContentRect`), or its black
|
||||
bands would frame the picture against the room. The manifest's `com.oculus.feature.PASSTHROUGH` is what lets Horizon
|
||||
@@ -268,7 +271,8 @@ The app opens on `LauncherActivity` (`android/app/src/main/java/org/wiicompiled/
|
||||
a 2D Horizon OS panel modelled on the PC launcher, WheelWizard VR, and using its
|
||||
palette. **Home** has the Play button and reports a missing or incomplete `DATA`
|
||||
(the check is the runtime's own `IsDvdDataRoot`: `files/` and `sys/fst.bin`).
|
||||
**Settings** edits `Config.toml` in tabs: VR (Flat Screen mode, camera, rotation, driver hiding,
|
||||
**Settings** edits `Config.toml` in tabs: VR (race view: immersive, immersive window or flat
|
||||
screen, camera, rotation, driver hiding,
|
||||
seat, hand steering, lean back, render scale, VR interpolation, virtual screen
|
||||
size and distance),
|
||||
Graphics (resolution, widescreen, bloom, shader stutter), Controls (controller
|
||||
|
||||
@@ -58,6 +58,7 @@ struct RuntimeUserConfig {
|
||||
std::optional<float> vrHudWidthMeters;
|
||||
std::optional<bool> vrHudVirtualScreen;
|
||||
std::optional<bool> vrFlatScreen;
|
||||
std::optional<bool> vrImmersiveWindow;
|
||||
std::optional<bool> vrPassthrough;
|
||||
std::optional<bool> vrStopAtDisplayCopy;
|
||||
std::optional<bool> vrSkipCopyClears;
|
||||
@@ -522,6 +523,11 @@ inline void EnsureConfigFile() {
|
||||
"# first-person camera or hand steering. Changeable live from the\n"
|
||||
"# F10 menu.\n"
|
||||
"flat_screen = false\n"
|
||||
"# The immersive window keeps the stereo race view but shows it\n"
|
||||
"# only through that screen, with the room around it on the\n"
|
||||
"# Quest (black elsewhere). Flat Screen mode wins over it.\n"
|
||||
"# Changeable live from the F10 menu.\n"
|
||||
"immersive_window = false\n"
|
||||
"# EFB replay controls for the per-eye views, changeable live\n"
|
||||
"# from the F10 menu. stop_at_display_copy ends each eye at the\n"
|
||||
"# frame's final GXCopyDisp; skip_copy_clears drops the EFB\n"
|
||||
@@ -778,6 +784,7 @@ inline RuntimeUserConfig ParseConfigDocument(const toml::value& document) {
|
||||
}
|
||||
config.vrHudVirtualScreen = FindConfigValue<bool>(document, "vr", "hud_virtual_screen");
|
||||
config.vrFlatScreen = FindConfigValue<bool>(document, "vr", "flat_screen");
|
||||
config.vrImmersiveWindow = FindConfigValue<bool>(document, "vr", "immersive_window");
|
||||
config.vrPassthrough = FindConfigValue<bool>(document, "vr", "passthrough");
|
||||
config.vrStopAtDisplayCopy = FindConfigValue<bool>(document, "vr", "stop_at_display_copy");
|
||||
config.vrSkipCopyClears = FindConfigValue<bool>(document, "vr", "skip_copy_clears");
|
||||
@@ -1102,6 +1109,11 @@ inline bool SetVrFlatScreen(bool value) {
|
||||
return WriteSetting("vr", "flat_screen", value ? "true" : "false");
|
||||
}
|
||||
|
||||
inline bool SetVrImmersiveWindow(bool value) {
|
||||
Mutable().vrImmersiveWindow = value;
|
||||
return WriteSetting("vr", "immersive_window", value ? "true" : "false");
|
||||
}
|
||||
|
||||
inline bool SetVrPassthrough(bool value) {
|
||||
Mutable().vrPassthrough = value;
|
||||
return WriteSetting("vr", "passthrough", value ? "true" : "false");
|
||||
@@ -1579,10 +1591,41 @@ inline bool VrFlatScreen(bool fallback = false) {
|
||||
return Get().vrFlatScreen.value_or(fallback);
|
||||
}
|
||||
|
||||
// Races in stereo as usual, but seen only through the screen the race's 2D
|
||||
// layer sits on, with the room around it. Flat Screen mode wins over it.
|
||||
inline bool VrImmersiveWindow(bool fallback = false) {
|
||||
return Get().vrImmersiveWindow.value_or(fallback);
|
||||
}
|
||||
|
||||
// The race view the settings present as one choice, kept in the two keys
|
||||
// above so that a file without immersive_window reads as it always did.
|
||||
enum class VrRaceView : int {
|
||||
Immersive = 0,
|
||||
ImmersiveWindow = 1,
|
||||
FlatScreen = 2,
|
||||
};
|
||||
|
||||
inline VrRaceView VrRaceViewOf(const RuntimeUserConfig& config) {
|
||||
if (config.vrFlatScreen.value_or(false)) {
|
||||
return VrRaceView::FlatScreen;
|
||||
}
|
||||
return config.vrImmersiveWindow.value_or(false) ? VrRaceView::ImmersiveWindow : VrRaceView::Immersive;
|
||||
}
|
||||
|
||||
inline VrRaceView GetVrRaceView() {
|
||||
return VrRaceViewOf(Get());
|
||||
}
|
||||
|
||||
inline bool SetVrRaceView(VrRaceView view) {
|
||||
const bool flat = SetVrFlatScreen(view == VrRaceView::FlatScreen);
|
||||
const bool window = SetVrImmersiveWindow(view == VrRaceView::ImmersiveWindow);
|
||||
return flat && window;
|
||||
}
|
||||
|
||||
// The room, through the headset's cameras, around the menu screen and every
|
||||
// other virtual screen, a Flat Screen race included (never an immersive
|
||||
// race). Only the Quest offers it; the launcher's Settings page shows the same
|
||||
// default.
|
||||
// other virtual screen, a Flat Screen race included, and around the immersive
|
||||
// window (never a fully immersive race). Only the Quest offers it; the
|
||||
// launcher's Settings page shows the same default.
|
||||
inline bool VrPassthrough(bool fallback = true) {
|
||||
return Get().vrPassthrough.value_or(fallback);
|
||||
}
|
||||
|
||||
@@ -132,9 +132,15 @@ struct OpenXRPresentation {
|
||||
bool quad_anchored = false;
|
||||
XrPosef quad_pose{{0.0f, 0.0f, 0.0f, 1.0f}, {0.0f, 0.0f, 0.0f}};
|
||||
|
||||
// Show the room through the headset's cameras around the virtual screen
|
||||
// (OpenXRPassthrough). Taken when the presentation is handed to the backend,
|
||||
// which starts or pauses the view then; a backend without one ignores it.
|
||||
// Used only by ImmersiveProjection: Aurora left each eye transparent outside
|
||||
// the race's 2D-layer screen (AuroraStereoFrame::window), so the projection
|
||||
// layer is blended by its alpha over whatever is under it.
|
||||
bool immersive_window = false;
|
||||
|
||||
// Show the room through the headset's cameras around the virtual screen or
|
||||
// the immersive window (OpenXRPassthrough). Taken when the presentation is
|
||||
// handed to the backend, which starts or pauses the view then; a backend
|
||||
// without one ignores it.
|
||||
bool passthrough = false;
|
||||
|
||||
OpenXRPanelLayer panel;
|
||||
|
||||
@@ -61,11 +61,19 @@ void OpenXRRequestRecenter() noexcept;
|
||||
void OpenXRSetLeanBackDegrees(float degrees) noexcept;
|
||||
|
||||
// Shows the room through the headset's cameras around the menu screen and every
|
||||
// other virtual screen, never during an immersive race. Only the standalone
|
||||
// (Quest) backend offers it; elsewhere this changes nothing. Callable from any
|
||||
// thread; applied on the XR pacing thread's next frame.
|
||||
// other virtual screen, and around the immersive window, never during a fully
|
||||
// immersive race. Only the standalone (Quest) backend offers it; elsewhere this
|
||||
// changes nothing. Callable from any thread; applied on the XR pacing thread's
|
||||
// next frame.
|
||||
void OpenXRSetPassthrough(bool enabled) noexcept;
|
||||
|
||||
// The immersive window: an immersive race keeps its stereo view but is seen
|
||||
// only through the screen its 2D layer sits on, with the room (or, without
|
||||
// passthrough, black) around it. Callable from any thread; applied to the next
|
||||
// published frame. Flat Screen mode, which keeps races off the immersive path
|
||||
// altogether, makes it moot.
|
||||
void OpenXRSetImmersiveWindow(bool enabled) noexcept;
|
||||
|
||||
// Live scene interpolation at the headset's own display deadlines.
|
||||
// 0 = Off, 1 = Auto, otherwise 72/90/120 as a rendering-rate ceiling.
|
||||
void OpenXRSetFrameInterpolationFps(uint32_t target) noexcept;
|
||||
|
||||
@@ -142,7 +142,11 @@ bool g_vrStopAtDisplayCopy = RuntimeConfigFile::VrStopAtDisplayCopy(true);
|
||||
bool g_vrSkipCopyClears = RuntimeConfigFile::VrSkipCopyClears(true);
|
||||
bool g_vrSinglePassEyes = RuntimeConfigFile::VrSinglePassEyes(true);
|
||||
bool g_vrHudVirtualScreen = RuntimeConfigFile::VrHudVirtualScreen(true);
|
||||
bool g_vrFlatScreen = RuntimeConfigFile::VrFlatScreen();
|
||||
// Race view: Immersive, Immersive window or Flat screen (RuntimeConfigFile::VrRaceView), and
|
||||
// Flat Screen mode as the flag the race view rows below are disabled by.
|
||||
int g_vrRaceView = static_cast<int>(RuntimeConfigFile::GetVrRaceView());
|
||||
bool g_vrFlatScreen = g_vrRaceView == static_cast<int>(RuntimeConfigFile::VrRaceView::FlatScreen);
|
||||
constexpr std::array<const char*, 3> kVrRaceViewLabels{"Immersive", "Immersive window", "Flat screen"};
|
||||
#if defined(__ANDROID__)
|
||||
bool g_vrPassthrough = RuntimeConfigFile::VrPassthrough();
|
||||
// Menu labels for the foveation levels, index-matched to RuntimeConfigFile::kVrFoveationLevels and to
|
||||
@@ -1444,7 +1448,8 @@ void DrawVrSettings() {
|
||||
}
|
||||
ImGui::Separator();
|
||||
ImGui::Text("VR 2D layer");
|
||||
ImGui::BeginDisabled(g_vrFlatScreen);
|
||||
ImGui::BeginDisabled(
|
||||
g_vrFlatScreen || g_vrRaceView == static_cast<int>(RuntimeConfigFile::VrRaceView::ImmersiveWindow));
|
||||
if (ImGui::Checkbox("2D layer on a virtual screen", &g_vrHudVirtualScreen)) {
|
||||
ApplyVrHudVirtualScreen();
|
||||
RuntimeConfigFile::SetVrHudVirtualScreen(g_vrHudVirtualScreen);
|
||||
@@ -1455,7 +1460,8 @@ void DrawVrSettings() {
|
||||
"Puts the minimap, race position, item roulette and the rest of the race HUD on a "
|
||||
"screen fixed ahead of the kart camera. Turn off to leave them stretched across "
|
||||
"the whole view. Its size and distance are the [vr] hud_width_meters and "
|
||||
"hud_distance_meters read at launch.");
|
||||
"hud_distance_meters read at launch. The immersive window is that screen, and "
|
||||
"always carries them.");
|
||||
}
|
||||
ImGui::Separator();
|
||||
ImGui::Text("VR view");
|
||||
@@ -1514,8 +1520,8 @@ void DrawVrSettings() {
|
||||
ImGui::SetTooltip(
|
||||
"Shows your room through the headset's cameras around the menu screen and every "
|
||||
"other screen outside an immersive race, instead of black. Immersive races stay "
|
||||
"fully virtual; the Flat Screen race has the room around it too. "
|
||||
"Applies immediately.");
|
||||
"fully virtual; the immersive window and the Flat Screen race have the room "
|
||||
"around them too. Applies immediately.");
|
||||
}
|
||||
// Shows the live level, which debug.wiicompiled.foveation can override.
|
||||
g_vrFoveation = static_cast<int>(aurora_get_stereo_foveation());
|
||||
@@ -1539,16 +1545,23 @@ void DrawVrSettings() {
|
||||
#endif
|
||||
ImGui::Separator();
|
||||
ImGui::Text("VR camera");
|
||||
if (ImGui::Checkbox("Flat Screen mode", &g_vrFlatScreen)) {
|
||||
RuntimeConfigFile::SetVrFlatScreen(g_vrFlatScreen);
|
||||
if (ImGui::Combo("Race view", &g_vrRaceView, kVrRaceViewLabels.data(),
|
||||
static_cast<int>(kVrRaceViewLabels.size()))) {
|
||||
const auto view = static_cast<RuntimeConfigFile::VrRaceView>(g_vrRaceView);
|
||||
g_vrFlatScreen = view == RuntimeConfigFile::VrRaceView::FlatScreen;
|
||||
RuntimeConfigFile::SetVrRaceView(view);
|
||||
mkw::vr::MkwVRPolicySetImmersiveRaces(!g_vrFlatScreen);
|
||||
mkw::vr::OpenXRSetImmersiveWindow(view == RuntimeConfigFile::VrRaceView::ImmersiveWindow);
|
||||
mkw::vr::MkwVRFirstPersonApplyConfiguredSettings();
|
||||
}
|
||||
if (ImGui::IsItemHovered()) {
|
||||
ImGui::SetTooltip(
|
||||
"Plays races on the same flat screen as the menus, through the game's own camera, "
|
||||
"instead of all around you in stereo. The first-person camera, hand steering and "
|
||||
"the race view settings do not apply while it is on. Applies immediately.");
|
||||
"Immersive plays races all around you in stereo. Immersive window keeps that "
|
||||
"stereo view but shows it only through a window where the menu screen sits, with "
|
||||
"your room around it on the Quest (black elsewhere); look through it from another "
|
||||
"angle and the view shifts as through a real window. Flat screen plays races on "
|
||||
"the menu screen through the game's own camera; the first-person camera, hand "
|
||||
"steering and the race view settings do not apply to it. Applies immediately.");
|
||||
}
|
||||
// Everything below shapes the immersive race view, which Flat Screen mode replaces.
|
||||
ImGui::BeginDisabled(g_vrFlatScreen);
|
||||
@@ -1704,7 +1717,8 @@ void DrawVrSettings() {
|
||||
}
|
||||
|
||||
// The right-thumbstick click: flips the first-person camera exactly as its
|
||||
// checkbox does, so it does nothing in Flat Screen mode either. Game thread.
|
||||
// checkbox does, so it does nothing in Flat Screen mode either (it does in the
|
||||
// immersive window, which is still the stereo race view). Game thread.
|
||||
void ToggleFirstPersonCamera() {
|
||||
if (g_vrFlatScreen) {
|
||||
return;
|
||||
|
||||
@@ -568,6 +568,10 @@ public:
|
||||
passthrough_.store(enabled, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
void SetImmersiveWindow(bool enabled) noexcept {
|
||||
immersive_window_.store(enabled, std::memory_order_relaxed);
|
||||
}
|
||||
|
||||
void SetLeanBackDegrees(float degrees) noexcept {
|
||||
lean_back_degrees_.store(
|
||||
std::clamp(degrees, -RuntimeConfigFile::kVrLeanBackDegreesLimit,
|
||||
@@ -846,10 +850,15 @@ private:
|
||||
aurora_get_stereo_screen_aspects(&picture_aspect, &snapshot_aspect)) {
|
||||
presentation.quad_content_aspect = snapshot_aspect;
|
||||
}
|
||||
// The room around the menu screen and every other virtual screen, a
|
||||
// Flat Screen race included; an immersive race is fully virtual, and the
|
||||
// cameras are paused for it.
|
||||
presentation.passthrough = !immersive && passthrough_.load(std::memory_order_relaxed);
|
||||
// The immersive window: the race's stereo view seen through its 2D layer's screen.
|
||||
// The flag travels with the packet, so the eyes Aurora masks and the layer the
|
||||
// backend blends always belong to the same frame.
|
||||
presentation.immersive_window = immersive && immersive_window_.load(std::memory_order_relaxed);
|
||||
// The room around the menu screen and every other virtual screen, a Flat Screen
|
||||
// race included, and around the immersive window; a fully immersive race is
|
||||
// virtual all round, and the cameras are paused for it.
|
||||
presentation.passthrough =
|
||||
(!immersive || presentation.immersive_window) && passthrough_.load(std::memory_order_relaxed);
|
||||
// The settings panel gets a compositor layer of its own while it is
|
||||
// open, and Aurora leaves it out of the eyes. A backend that could
|
||||
// not make that layer has the panel drawn into the eyes instead.
|
||||
@@ -1247,6 +1256,7 @@ private:
|
||||
destination.displayTimeNanos = DisplayTimeNanos(source.xr_frame.predicted_display_time);
|
||||
destination.mode = immersive ? AURORA_STEREO_FRAME_IMMERSIVE_REPLAY
|
||||
: AURORA_STEREO_FRAME_VIRTUAL_SCREEN;
|
||||
destination.window = immersive && source.presentation.immersive_window;
|
||||
for (uint32_t eye = 0; eye < kOpenXREyeCount; ++eye) {
|
||||
destination.eyes[eye].width = source.render_width[eye];
|
||||
destination.eyes[eye].height = source.render_height[eye];
|
||||
@@ -1378,7 +1388,8 @@ private:
|
||||
// that space (in metres) to base + lean * p in the application space, so the
|
||||
// screen sits at base + lean * (0, 0, -distance), turned by the lean, its
|
||||
// height following the picture aspect as stereo_hud_screen's does. With the
|
||||
// 2D layer stretched across the eyes there is no screen to point at.
|
||||
// 2D layer stretched across the eyes there is no screen to point at, except
|
||||
// in the immersive window, which is that screen and always carries the layer.
|
||||
OpenXRPointerScreen PointerScreen(const OpenXRBackendFrame& frame, const MkwVRPolicySnapshot& policy,
|
||||
bool immersive) const noexcept {
|
||||
OpenXRPointerScreen screen{};
|
||||
@@ -1389,8 +1400,9 @@ private:
|
||||
}
|
||||
|
||||
if (immersive) {
|
||||
if (!aurora_get_stereo_hud_screen_enabled() || !(policy.config.hud_width_meters > 0.0f) ||
|
||||
!RaceScreenPose(frame, policy, screen.pose)) {
|
||||
// The immersive window always carries the 2D layer.
|
||||
const bool on_screen = aurora_get_stereo_hud_screen_enabled() || frame.presentation.immersive_window;
|
||||
if (!on_screen || !(policy.config.hud_width_meters > 0.0f) || !RaceScreenPose(frame, policy, screen.pose)) {
|
||||
return screen;
|
||||
}
|
||||
screen.half_width_meters = 0.5f * policy.config.hud_width_meters;
|
||||
@@ -1756,6 +1768,7 @@ private:
|
||||
std::atomic_bool recenter_requested_{false};
|
||||
std::atomic<float> lean_back_degrees_{RuntimeConfigFile::VrLeanBackDegrees()};
|
||||
std::atomic_bool passthrough_{RuntimeConfigFile::VrPassthrough()};
|
||||
std::atomic_bool immersive_window_{RuntimeConfigFile::VrImmersiveWindow()};
|
||||
std::atomic_uint32_t frame_interpolation_fps_{RuntimeConfigFile::VrFrameInterpolationFps()};
|
||||
std::atomic_bool interpolation_available_{false};
|
||||
std::mutex interpolation_mutex_;
|
||||
@@ -1877,6 +1890,14 @@ void OpenXRSetPassthrough(bool enabled) noexcept {
|
||||
#endif
|
||||
}
|
||||
|
||||
void OpenXRSetImmersiveWindow(bool enabled) noexcept {
|
||||
#if MKW_OPENXR_GRAPHICS_BACKEND
|
||||
OpenXRIntegration::Get().SetImmersiveWindow(enabled);
|
||||
#else
|
||||
(void)enabled;
|
||||
#endif
|
||||
}
|
||||
|
||||
void OpenXRSetFrameInterpolationFps(uint32_t target) noexcept {
|
||||
#if MKW_OPENXR_GRAPHICS_BACKEND
|
||||
OpenXRIntegration::Get().SetFrameInterpolationFps(target);
|
||||
|
||||
@@ -942,7 +942,10 @@ public:
|
||||
views[eye].subImage.imageArrayIndex = 0;
|
||||
}
|
||||
XrCompositionLayerProjection projection{XR_TYPE_COMPOSITION_LAYER_PROJECTION};
|
||||
projection.layerFlags = 0;
|
||||
// The immersive window's eyes are transparent outside the window (premultiplied alpha), so
|
||||
// the room shows around it; otherwise the race covers the whole view and alpha is ignored.
|
||||
projection.layerFlags =
|
||||
frame.presentation.immersive_window ? XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT : 0;
|
||||
projection.space = runtime_->AppSpace();
|
||||
projection.viewCount = kOpenXREyeCount;
|
||||
projection.views = views.data();
|
||||
@@ -950,16 +953,18 @@ public:
|
||||
}
|
||||
|
||||
// Ends the compositor frame with the scene's layer: over the room's camera
|
||||
// view while that runs and the scene is the virtual screen (never under the
|
||||
// race's projection, which covers the whole view), and, while the retained
|
||||
// frame rendered it, under the settings panel's layer.
|
||||
// view while that runs and the scene is the virtual screen or the immersive
|
||||
// window (never under a fully immersive race's projection, which covers the
|
||||
// whole view), and, while the retained frame rendered it, under the settings
|
||||
// panel's layer.
|
||||
bool EndFrameWithPanel(const OpenXRBackendFrame& frame, const XrCompositionLayerBaseHeader* scene) {
|
||||
const auto& panel = frame.presentation.panel;
|
||||
XrCompositionLayerQuad panel_quad{};
|
||||
const XrCompositionLayerBaseHeader* layers[3] = {};
|
||||
uint32_t count = 0;
|
||||
if (const XrCompositionLayerBaseHeader* passthrough = passthrough_.Layer();
|
||||
passthrough != nullptr && frame.presentation.mode == OpenXRFrameMode::VirtualScreen) {
|
||||
passthrough != nullptr && (frame.presentation.mode == OpenXRFrameMode::VirtualScreen ||
|
||||
frame.presentation.immersive_window)) {
|
||||
layers[count++] = passthrough;
|
||||
}
|
||||
layers[count++] = scene;
|
||||
|
||||
@@ -34,5 +34,18 @@ int main() {
|
||||
Require(Parse("[vr]\nsingle_pass_eyes = true\n").vrSinglePassEyes == true);
|
||||
Require(Parse("[vr]\nsingle_pass_eyes = false\n").vrSinglePassEyes == false);
|
||||
Require(!Parse("[vr]\n").vrSinglePassEyes.has_value());
|
||||
|
||||
// [vr] immersive_window and flat_screen: one race view in two keys, Flat
|
||||
// Screen mode winning, so a file that predates the window reads as before.
|
||||
using RuntimeConfigFile::VrRaceView;
|
||||
using RuntimeConfigFile::VrRaceViewOf;
|
||||
Require(Parse("[vr]\nimmersive_window = true\n").vrImmersiveWindow == true);
|
||||
Require(!Parse("[vr]\n").vrImmersiveWindow.has_value());
|
||||
Require(VrRaceViewOf(Parse("[vr]\n")) == VrRaceView::Immersive);
|
||||
Require(VrRaceViewOf(Parse("[vr]\nflat_screen = false\n")) == VrRaceView::Immersive);
|
||||
Require(VrRaceViewOf(Parse("[vr]\nflat_screen = true\n")) == VrRaceView::FlatScreen);
|
||||
Require(VrRaceViewOf(Parse("[vr]\nimmersive_window = true\n")) == VrRaceView::ImmersiveWindow);
|
||||
Require(VrRaceViewOf(Parse("[vr]\nflat_screen = true\nimmersive_window = true\n")) == VrRaceView::FlatScreen);
|
||||
Require(VrRaceViewOf(Parse("[vr]\nflat_screen = false\nimmersive_window = false\n")) == VrRaceView::Immersive);
|
||||
return 0;
|
||||
}
|
||||
Reference in new issue
Block a user