Repeat the last frame on every refresh the game has no new frame for

SteamVR on the Steam Frame ran the game at half the 120 Hz display rate,
since render-first pacing submits only when a game frame is sealed, and
filled every other refresh itself even with Motion Smoothing off, which
doubled the HUD and the menu screen while the head turned. With the new
[vr] repeat_frames (default on for the Frame, off elsewhere, live), the
pacing thread waits a millisecond for the eyes and otherwise spends the
refresh on a keep-alive cycle that resubmits the retained layer with its
rendered poses, paced by xrWaitFrame.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HBRGKTE1GnN2ah8gcZKr3

OPENXR.md

@@ -41,6 +41,7 @@ mirror_view = "normal"
 controller_mode = "wii_remote"
 frame_interpolation_fps = 0
 refresh_rate = 0
+repeat_frames = false
 render_scale = 1.0
 world_units_per_meter = 500.0
 hud_distance_meters = 2.0
@@ -231,6 +232,10 @@ Frame) to leave the headset's own. The game renders 60 frames a second, so 120 H
 for exactly two refreshes. A rate the runtime does not list, or declines, is logged and leaves its
 own; setting `0` again restores the rate the session started at. It is live from F10 / the headset
 panel (*Headset refresh rate*) and the Quest launcher; runtimes without the extension ignore it.
+`repeat_frames` (default off, on for the Steam Frame) submits the last frame again, with the poses it
+was rendered for, on each refresh the game has no new frame for, so a runtime sees the app at the
+display's rate and does not halve it and fill refreshes itself (SteamVR on the Frame did, doubling the
+HUD while the head turned). Render-first pacing only; live from F10 / the headset panel.
 
 ## Controllers
 

docs/steam-frame.md

@@ -159,6 +159,7 @@ build; its launch and manifest are Lepton's.
 | `[vr] refresh_rate` default | `0` (the headset's own) | `120` |
 | `[vr] passthrough` | default on (`XR_FB_passthrough`) | not asked for, default off, setting hidden |
 | `[vr] eye_tracked_foveation` default | off | on |
+| `[vr] repeat_frames` default | off | on |
 
 The application ID stays `org.wiicompiled.quest`, so the storage paths in `docs/quest-port.md` hold
 as they are. The kit's CPU string differs from the Quest ones, which gives the Frame its own kit
@@ -223,9 +224,17 @@ so nothing changes on a Quest.
 The request uses the runtime's own value within half a hertz of the setting (runtimes report 119.98
 for 120). Setting it back to `0` restores the rate the session started at. The game renders 60 frames
 a second, so at 120 Hz each frame shows for exactly two refreshes. At 72 or 90 Hz some frames show
-for one refresh and others for two, which judders. The Frame starts at 120. Render-first pacing
+for one refresh and others for two, which judders. The Frame starts at 120.
-(`docs/quest-port.md`) already waits for each sealed game frame, so on the Frame the pacing summary
+
-should read about 60 `skipped-slots` a second with no `late` cycles.
+Render-first pacing (`docs/quest-port.md`) submits a frame once the game has sealed one, 60 times a
+second. On the Frame, SteamVR answered that by running the app at half rate (the pacing summary read
+`predicted-rate=60.0Hz`) and filling every other refresh itself, even with Motion Smoothing off, which
+doubled the HUD and the menu screen while the head turned. `[vr] repeat_frames` (default on for the
+Frame, off elsewhere, live in the headset panel's VR tab) therefore submits the retained layer, with
+the poses it was rendered for, on every refresh the next eyes are not ready for: the pacing thread
+waits a millisecond for them and otherwise spends the refresh on a keep-alive cycle, which
+xrWaitFrame paces. The summary should then read `predicted-rate=120.0Hz`, about 60 `keepalive` a
+second and 60 `new` layers.
 
 Lepton may decline the request (frame-control found SteamVR keeping its own rate there). The session
 log then says `display refresh rate 120 Hz refused` with the rates it offers, and nothing else

runtime/include/runtime_config.h

@@ -100,6 +100,7 @@ struct RuntimeUserConfig {
     std::optional<std::string> vrPerformanceLevel;
     std::optional<std::string> vrFoveation;
     std::optional<bool> vrEyeTrackedFoveation;
+    std::optional<bool> vrRepeatFrames;
     std::optional<std::string> vrRecenterKey;
     std::optional<float> vrLeanBackDegrees;
     // F10 > Diagnostics: OpenXR pacing and presentation logging in console.log.
@@ -338,6 +339,19 @@ inline constexpr bool kVrEyeTrackedFoveationDefault = true;
 inline constexpr bool kVrEyeTrackedFoveationDefault = false;
 #endif
 
+// Repeated frames: while the game has no new frame for a display refresh, the last one is submitted
+// again with the head pose it was rendered for, which the compositor turns to the current one. A
+// runtime then sees the app at the display's rate instead of at the game's 60 FPS, and does not
+// halve the app's rate and fill every other refresh itself: SteamVR's filled frames double the HUD
+// and the menu screen while the head turns. On by default on the Steam Frame. Live.
+#if defined(MKW_HEADSET_STEAM_FRAME)
+inline constexpr bool kVrRepeatFramesDefault = true;
+#define MKW_VR_REPEAT_FRAMES_DEFAULT_TOML "true"
+#else
+inline constexpr bool kVrRepeatFramesDefault = false;
+#define MKW_VR_REPEAT_FRAMES_DEFAULT_TOML "false"
+#endif
+
 // The level aurora_set_stereo_foveation takes; anything unknown is off.
 inline uint32_t VrFoveationLevelIndex(std::string_view value) {
     const auto it = std::find(kVrFoveationLevels.begin(), kVrFoveationLevels.end(), value);
@@ -590,6 +604,9 @@ inline void EnsureConfigFile() {
               "# at 60, so 120 shows every frame twice. Only runtimes that let apps\n"
               "# choose (XR_FB_display_refresh_rate) take it. Live.\n"
               "refresh_rate = " MKW_VR_REFRESH_RATE_DEFAULT_TEXT "\n"
+              "# Submit the last frame again for each refresh the game has no new\n"
+              "# frame for, so the runtime does not fill those refreshes itself. Live.\n"
+              "repeat_frames = " MKW_VR_REPEAT_FRAMES_DEFAULT_TOML "\n"
               "render_scale = " MKW_VR_RENDER_SCALE_DEFAULT_TEXT "\n"
               "world_units_per_meter = 500.0\n"
               "hud_distance_meters = 2.0\n"
@@ -910,6 +927,7 @@ inline RuntimeUserConfig ParseConfigDocument(const toml::value& document) {
         config.vrFoveation = *value;
     }
     config.vrEyeTrackedFoveation = FindConfigValue<bool>(document, "vr", "eye_tracked_foveation");
+    config.vrRepeatFrames = FindConfigValue<bool>(document, "vr", "repeat_frames");
     if (auto value = FindConfigValue<std::string>(document, "vr", "mirror_view");
         value && IsSupportedVrMirrorView(*value)) {
         config.vrMirrorView = *value;
@@ -1322,6 +1340,11 @@ inline bool SetVrFoveation(std::string value) {
     return WriteSetting("vr", "foveation", FormatString(value));
 }
 
+inline bool SetVrRepeatFrames(bool value) {
+    Mutable().vrRepeatFrames = value;
+    return WriteSetting("vr", "repeat_frames", value ? "true" : "false");
+}
+
 inline bool SetVrEyeTrackedFoveation(bool value) {
     Mutable().vrEyeTrackedFoveation = value;
     return WriteSetting("vr", "eye_tracked_foveation", value ? "true" : "false");
@@ -1853,6 +1876,10 @@ inline std::string VrFoveation(std::string fallback = kVrFoveationDefault) {
     return value && IsSupportedVrFoveation(*value) ? *value : std::move(fallback);
 }
 
+inline bool VrRepeatFrames(bool fallback = kVrRepeatFramesDefault) {
+    return Get().vrRepeatFrames.value_or(fallback);
+}
+
 inline bool VrEyeTrackedFoveation(bool fallback = kVrEyeTrackedFoveationDefault) {
     return Get().vrEyeTrackedFoveation.value_or(fallback);
 }

runtime/src/settings_overlay.cpp

@@ -157,6 +157,7 @@ constexpr std::array<const char*, 4> kVrFoveationLabels{"Off", "Low", "Medium",
 static_assert(kVrFoveationLabels.size() == RuntimeConfigFile::kVrFoveationLevels.size());
 int g_vrFoveation = static_cast<int>(RuntimeConfigFile::VrFoveationLevelIndex(RuntimeConfigFile::VrFoveation()));
 bool g_vrEyeTrackedFoveation = RuntimeConfigFile::VrEyeTrackedFoveation();
+bool g_vrRepeatFrames = RuntimeConfigFile::VrRepeatFrames();
 #endif
 bool g_vrFirstPerson = RuntimeConfigFile::VrFirstPerson(false);
 bool g_vrFirstPersonToggleClick = RuntimeConfigFile::VrFirstPersonToggleClick();
@@ -1557,6 +1558,16 @@ void DrawVrSettings() {
                 "immediately.");
         }
     }
+    if (ImGui::Checkbox("Repeat frames at the headset's rate", &g_vrRepeatFrames)) {
+        RuntimeConfigFile::SetVrRepeatFrames(g_vrRepeatFrames);
+    }
+    if (ImGui::IsItemHovered()) {
+        ImGui::SetTooltip(
+            "Shows the last frame again, turned to where you now look, on each refresh the game has "
+            "no new frame for. The headset's runtime then does not drop the game to half its rate and "
+            "fill the gaps itself, which doubles the HUD and the menu screen as you turn your head "
+            "(SteamVR on the Steam Frame). Applies immediately.");
+    }
     if (ImGui::Combo("VR frame interpolation (experimental)", &g_vrFrameInterpolationMode,
                      kVrInterpolationLabels.data(), static_cast<int>(kVrInterpolationLabels.size()))) {
         const auto target = kVrInterpolationFps[static_cast<size_t>(g_vrFrameInterpolationMode)];

runtime/src/vr/openxr_integration.cpp

@@ -710,6 +710,9 @@ private:
     // Skipped eye copies tolerated back to back before the session is given up: a few seconds
     // at the headset's refresh rate.
     static constexpr uint32_t kMaxConsecutiveSkips = 300;
+    // With [vr] repeat_frames, how long render-first pacing waits for the eyes before it spends the
+    // next display refresh on the retained layer. The repeat's xrWaitFrame does the actual pacing.
+    static constexpr uint32_t kRepeatFramePollMs = 1;
 
     static float ClampRenderScale(float scale) noexcept {
         return std::clamp(scale, RuntimeConfigFile::kVrRenderScaleMin, RuntimeConfigFile::kVrRenderScaleMax);
@@ -1335,13 +1338,17 @@ private:
         aurora_notify_stereo_frame();
 
         // Aurora renders the eyes at its next seal. Meanwhile the compositor keeps showing the
-        // retained layer; a 50 ms stall repeats it explicitly and withdraws the packet.
+        // retained layer; a 50 ms stall repeats it explicitly and withdraws the packet. With
+        // [vr] repeat_frames the retained layer is also submitted for every display refresh the
+        // eyes are not ready for, each cycle paced by xrWaitFrame, so the runtime sees the app at
+        // the display's rate rather than the game's and never fills refreshes in itself.
+        const uint32_t wait_ms = RuntimeConfigFile::VrRepeatFrames() ? kRepeatFramePollMs : 50;
         OpenXRSubmissionStatus submission = OpenXRSubmissionStatus::Timeout;
         bool canceled_before_encode = false;
         const auto cancel_after = std::chrono::steady_clock::now() + std::chrono::milliseconds(50);
         while (!stop_.load(std::memory_order_acquire) && submission == OpenXRSubmissionStatus::Timeout) {
             submission = diagnostics::Measure(diagnostics::Stage::SubmissionWait, [&] {
-                return backend_->WaitForSubmission(packet, 50);
+                return backend_->WaitForSubmission(packet, wait_ms);
             });
             if (submission == OpenXRSubmissionStatus::Timeout) {
                 if (std::chrono::steady_clock::now() >= cancel_after) {

runtime/tests/vr_config_tests.cpp

@@ -79,6 +79,19 @@ int main() {
     Require(!RuntimeConfigFile::kVrEyeTrackedFoveationDefault);
 #endif
 
+    // [vr] repeat_frames: the retained layer fills the refreshes the game has no frame for.
+    Require(Parse("[vr]\nrepeat_frames = true\n").vrRepeatFrames == true);
+    Require(Parse("[vr]\nrepeat_frames = false\n").vrRepeatFrames == false);
+    Require(!Parse("[vr]\nrepeat_frames = 1\n").vrRepeatFrames.has_value());
+    Require(!Parse("[vr]\n").vrRepeatFrames.has_value());
+#if defined(MKW_HEADSET_STEAM_FRAME)
+    Require(RuntimeConfigFile::kVrRepeatFramesDefault);
+    Require(std::string_view(MKW_VR_REPEAT_FRAMES_DEFAULT_TOML) == "true");
+#else
+    Require(!RuntimeConfigFile::kVrRepeatFramesDefault);
+    Require(std::string_view(MKW_VR_REPEAT_FRAMES_DEFAULT_TOML) == "false");
+#endif
+
     // [vr] passthrough: Horizon OS's room view, which the Steam Frame build does not offer.
     Require(Parse("[vr]\npassthrough = false\n").vrPassthrough == false);
     Require(!Parse("[vr]\n").vrPassthrough.has_value());