Widen the eye-tracked foveation region and keep more gaze maps

On the Steam Frame the full-density region was too small and lagged the
eyes. Gaze-centred maps now widen both of the level's rings by 8 degrees,
covering a saccade until the next frame's map is bound, the tracker's error
and Turnip's per-bin density, and each eye keeps 128 maps instead of 32 so
fewer glances wait for an upload. The fixed, forward maps are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019HBRGKTE1GnN2ah8gcZKr3
This commit is contained in:
Claude committed 2026-10-04 15:49:18 +00:00
1 parent 2a72421926
commit 60fdb4b447
5 files changed
+30 -14

No files matched your search

+3 -2
View File
@@ -1088,8 +1088,9 @@ did (39 to 41.5 FPS).
With `eye_tracked_foveation`, a runtime that offers `XR_EXT_eye_gaze_interaction` and reports an eye
tracker (the Steam Frame's SteamVR) has its gaze pose located for each packet's display time and
turned into tangents of each eye's view (`vr/eye_gaze.h`), which `AuroraStereoFrame` carries as
`gaze`/`gazeValid`. Aurora centres the level's rings on the gaze snapped to a cell of two map texels
(about 3 degrees), keeping up to 32 maps per eye, one per cell looked at, and binds a new one once
`gaze`/`gazeValid`. Aurora centres the level's rings, each widened by 8 degrees to cover the lag
and error of tracking, on the gaze snapped to a cell of two map texels (about 3 degrees), keeping up
to 128 maps per eye, one per cell looked at, and binds a new one once
its upload completes, the previous map staying bound meanwhile. Without a tracked gaze (a blink, no
tracker, the setting off) the map is the forward one above, unchanged. Details and the Steam Frame
checks are in `docs/steam-frame.md`.
+3 -2
View File
@@ -615,8 +615,9 @@ struct EyeDensityMap {
uint64_t map = 0;
uint64_t lastUse = 0;
};
// The gaze cells' maps an eye keeps: a few glances' worth, each 2 bytes per 32x32 pixels.
constexpr size_t kEyeDensityMapCacheSize = 32;
// The gaze cells' maps an eye keeps, each 2 bytes per 32x32 pixels in a memory block of its own:
// most of the cells a session's glances reach, so a glance rarely waits for a map to upload.
constexpr size_t kEyeDensityMapCacheSize = 128;
struct StereoEyeTarget {
webgpu::TextureWithSampler color;
+12 -3
View File
@@ -138,8 +138,16 @@ inline Gaze cell_gaze(uint32_t eyeWidth, uint32_t eyeHeight, uint32_t texel, con
.tanY = fov.tanUp + (fov.tanDown - fov.tanUp) * v};
}
inline uint8_t density(Level level, float eccentricity) noexcept {
const Rings ring = rings(level);
// Eye-tracked maps widen both rings by this much. The full-density region has to cover where the eye
// lands after a saccade until the next game frame's map is bound, the tracker's error, and the
// snapping of the gaze to cells and of the density to the driver's tiles (Turnip shades a whole
// render-pass bin at one density).
inline constexpr float kTrackedMarginDegrees = 8.0f;
inline uint8_t density(Level level, float eccentricity, float margin = 0.0f) noexcept {
Rings ring = rings(level);
ring.full += margin;
ring.half += margin;
if (eccentricity < ring.full) {
return kFullDensity;
}
@@ -176,7 +184,8 @@ inline void build(uint32_t eyeWidth, uint32_t eyeHeight, uint32_t texel, const E
static_cast<float>(eyeWidth);
const float tanX = fov.tanLeft + (fov.tanRight - fov.tanLeft) * u;
const uint8_t value =
density(level, forward ? eccentricity_degrees(tanX, tanY) : angle_from_gaze_degrees(tanX, tanY, gaze));
forward ? density(level, eccentricity_degrees(tanX, tanY))
: density(level, angle_from_gaze_degrees(tanX, tanY, gaze), kTrackedMarginDegrees);
uint8_t* texelBytes = &map.rg8[(static_cast<size_t>(y) * map.width + x) * 2];
texelBytes[0] = value;
texelBytes[1] = value;
+3 -2
View File
@@ -199,8 +199,9 @@ TEST(Foveation, TheForwardGazeKeepsTheFixedMap) {
TEST(Foveation, TheFullDensityRegionFollowsTheGaze) {
const EyeFov fov = left_eye();
// Down and to the right, well off the forward direction.
const Gaze gaze{.tanX = std::tan(20.0f * kDegrees), .tanY = std::tan(-15.0f * kDegrees)};
// Down and to the right, off the forward direction, with the widened full-density region still
// inside the eye so its centre is not pulled in by the edge.
const Gaze gaze{.tanX = std::tan(12.0f * kDegrees), .tanY = std::tan(-10.0f * kDegrees)};
Map map;
foveation::build(1344, 1408, 32, fov, Level::High, map, gaze);
double sumX = 0.0;
+9 -5
View File
@@ -243,13 +243,17 @@ region to where the player looks:
2. `vr/eye_gaze.h` turns the gaze into tangents of each eye's own view, which may be canted.
`AuroraStereoFrame` carries them as `gaze` and `gazeValid`, appended after its existing fields.
3. Aurora snaps the gaze to a cell of two map texels (64 pixels, about 3 degrees) and builds that
cell's density map with the level's rings centred on the gaze (`gfx/foveation.hpp`).
cell's density map with the level's rings centred on the gaze (`gfx/foveation.hpp`). Both rings
are 8 degrees wider than the fixed map's: the full-density region has to cover where a saccade
lands until the next game frame's map is bound, the tracker's error, and Turnip shading a whole
render-pass bin at one density.
Each eye keeps up to 32 maps, one per cell looked at, so a glance back reuses its map instead of
uploading a new one. A new map is bound once its upload has completed, and until then the eye keeps
Each eye keeps up to 128 maps, one per cell looked at, so a glance back reuses its map instead of
uploading a new one. Each map has a memory block of its own: Turnip reads a map through a host
mapping of its memory, and Dawn's buffer uploads unmap the shared blocks they sub-allocate from. A new map is bound once its upload has completed, and until then the eye keeps
the map it had. A blink, lost tracking or the setting turned off returns to the map centred on the
forward direction, which is byte-identical to the fixed foveation map. No change to the Dawn patch
was needed: maps stay immutable, and the patch already lets a view be rebound to another map.
forward direction, which is byte-identical to the fixed foveation map. Maps stay immutable, and the
Dawn patch lets a view be rebound to another map.
The session log reports `OpenXR eye gaze: available` (or that the runtime has no eye tracker),
`OpenXR eye gaze: tracking` at the first tracked sample, and `eye foveation medium following the