Repeat a frame only when the new eyes miss the runtime's next wake

The first repeat_frames waited a millisecond for the eyes before spending a
refresh on the retained layer, so nearly every packet also cost a needless
repeat, the next packet missed the game's next frame, and the Steam Frame
showed about 35 new frames a second. The pacing thread now waits until
1.5 ms before the next wake, a period after the last xrWaitFrame returned,
which OpenXRRuntime records.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HBRGKTE1GnN2ah8gcZKr3

docs/steam-frame.md

@@ -232,9 +232,11 @@ second. On the Frame, SteamVR answered that by running the app at half rate (the
 doubled the HUD and the menu screen while the head turned. `[vr] repeat_frames` (default on for the
 Frame, off elsewhere, live in the headset panel's VR tab) therefore submits the retained layer, with
 the poses it was rendered for, on every refresh the next eyes are not ready for: the pacing thread
-waits a millisecond for them and otherwise spends the refresh on a keep-alive cycle, which
-xrWaitFrame paces. The summary should then read `predicted-rate=120.0Hz`, about 60 `keepalive` a
-second and 60 `new` layers.
+waits for them until 1.5 ms before the runtime's next wake (a period after the last xrWaitFrame
+returned) and otherwise spends that refresh on a keep-alive cycle, which xrWaitFrame paces. The
+summary should then read `predicted-rate=120.0Hz`, about 60 `keepalive` a second and 60 `new`
+layers. A first version waited only a millisecond, so every packet also spent a refresh on a repeat
+it did not need, the next packet missed the game's next frame, and `new` fell to about 35 a second.
 
 Lepton may decline the request (frame-control found SteamVR keeping its own rate there). The session
 log then says `display refresh rate 120 Hz refused` with the rates it offers, and nothing else

runtime/include/vr/openxr_runtime.h

@@ -6,6 +6,7 @@
 
 #include <algorithm>
 #include <array>
+#include <chrono>
 #include <cmath>
 #include <cstdint>
 #include <functional>
@@ -192,6 +193,10 @@ public:
     // BeginFrame and EndFrame using the same token. LocateViews is optional when
     // should_render is false; it is otherwise normally called after BeginFrame.
     OpenXRFrameStatus WaitFrame(OpenXRFrame& frame);
+    // When the last xrWaitFrame returned and the display period it predicted (0 before the first).
+    // The runtime wakes the app about once a period, so the next wake is due a period after it.
+    std::chrono::steady_clock::time_point LastWaitFrameReturn() const noexcept { return m_last_wait_return; }
+    XrDuration LastWaitFramePeriod() const noexcept { return m_last_wait_period; }
     bool BeginFrame(const OpenXRFrame& frame);
     bool LocateViews(OpenXRFrame& frame);
     // Locates the views for `display_time` outside the frame protocol, for a
@@ -312,6 +317,8 @@ private:
     uint64_t m_next_frame_serial = 1;
     uint64_t m_active_frame_serial = 0;
     XrTime m_active_frame_display_time = 0;
+    std::chrono::steady_clock::time_point m_last_wait_return{};
+    XrDuration m_last_wait_period = 0;
 
     OpenXRRuntimeInfo m_runtime_info;
     std::array<OpenXRViewConfiguration, kOpenXREyeCount> m_view_configuration{};

runtime/src/vr/openxr_integration.cpp

@@ -710,8 +710,10 @@ private:
     // Skipped eye copies tolerated back to back before the session is given up: a few seconds
     // at the headset's refresh rate.
     static constexpr uint32_t kMaxConsecutiveSkips = 300;
-    // With [vr] repeat_frames, how long render-first pacing waits for the eyes before it spends the
-    // next display refresh on the retained layer. The repeat's xrWaitFrame does the actual pacing.
+    // With [vr] repeat_frames, render-first pacing waits for the eyes until this long before the
+    // runtime's next wake, then spends that refresh on the retained layer; before the first wake,
+    // it waits this many milliseconds. The repeat's xrWaitFrame does the actual pacing.
+    static constexpr std::chrono::microseconds kRepeatFrameMargin{1500};
     static constexpr uint32_t kRepeatFramePollMs = 1;
 
     static float ClampRenderScale(float scale) noexcept {
@@ -1340,15 +1342,26 @@ private:
         // Aurora renders the eyes at its next seal. Meanwhile the compositor keeps showing the
         // retained layer; a 50 ms stall repeats it explicitly and withdraws the packet. With
         // [vr] repeat_frames the retained layer is also submitted for every display refresh the
-        // eyes are not ready for, each cycle paced by xrWaitFrame, so the runtime sees the app at
-        // the display's rate rather than the game's and never fills refreshes in itself.
-        const uint32_t wait_ms = RuntimeConfigFile::VrRepeatFrames() ? kRepeatFramePollMs : 50;
+        // eyes are not ready for, so the runtime sees the app at the display's rate rather than the
+        // game's and never fills refreshes in itself. The eyes are waited for until shortly before
+        // the runtime's next wake: a repeat begun earlier would spend a refresh the new eyes could
+        // still have made, and hold back the next packet past the game's next frame.
+        const bool repeat_frames = RuntimeConfigFile::VrRepeatFrames();
+        const auto wait_ms = [&]() -> uint32_t {
+            if (!repeat_frames) return 50;
+            const XrDuration period = runtime_->LastWaitFramePeriod();
+            if (period <= 0) return kRepeatFramePollMs;
+            const auto wake = runtime_->LastWaitFrameReturn() + std::chrono::nanoseconds(period);
+            const auto left = std::chrono::duration_cast<std::chrono::milliseconds>(
+                wake - kRepeatFrameMargin - std::chrono::steady_clock::now());
+            return static_cast<uint32_t>(std::clamp<int64_t>(left.count(), 0, 50));
+        };
         OpenXRSubmissionStatus submission = OpenXRSubmissionStatus::Timeout;
         bool canceled_before_encode = false;
         const auto cancel_after = std::chrono::steady_clock::now() + std::chrono::milliseconds(50);
         while (!stop_.load(std::memory_order_acquire) && submission == OpenXRSubmissionStatus::Timeout) {
             submission = diagnostics::Measure(diagnostics::Stage::SubmissionWait, [&] {
-                return backend_->WaitForSubmission(packet, wait_ms);
+                return backend_->WaitForSubmission(packet, wait_ms());
             });
             if (submission == OpenXRSubmissionStatus::Timeout) {
                 if (std::chrono::steady_clock::now() >= cancel_after) {

runtime/src/vr/openxr_runtime.cpp

@@ -710,6 +710,8 @@ OpenXRFrameStatus OpenXRRuntime::WaitFrame(OpenXRFrame& frame) {
         return OpenXRFrameStatus::Error;
     }
     diagnostics::OnWaitFrame(wait_timer, state.predictedDisplayTime, state.predictedDisplayPeriod);
+    m_last_wait_return = std::chrono::steady_clock::now();
+    m_last_wait_period = state.predictedDisplayPeriod;
 
     frame = {};
     frame.serial = m_next_frame_serial++;