mirror of
https://github.com/mitch030504/Wiicompiled_VR_Frame.git
synced 2026-10-06 05:00:27 +02:00
Widen the eye-tracked foveation region and keep more gaze maps
On the Steam Frame the full-density region was too small and lagged the eyes. Gaze-centred maps now widen both of the level's rings by 8 degrees, covering a saccade until the next frame's map is bound, the tracker's error and Turnip's per-bin density, and each eye keeps 128 maps instead of 32 so fewer glances wait for an upload. The fixed, forward maps are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019HBRGKTE1GnN2ah8gcZKr3
This commit is contained in:
5 files changed
+30
-14
No files matched your search
@@ -1088,8 +1088,9 @@ did (39 to 41.5 FPS).
|
||||
With `eye_tracked_foveation`, a runtime that offers `XR_EXT_eye_gaze_interaction` and reports an eye
|
||||
tracker (the Steam Frame's SteamVR) has its gaze pose located for each packet's display time and
|
||||
turned into tangents of each eye's view (`vr/eye_gaze.h`), which `AuroraStereoFrame` carries as
|
||||
`gaze`/`gazeValid`. Aurora centres the level's rings on the gaze snapped to a cell of two map texels
|
||||
(about 3 degrees), keeping up to 32 maps per eye, one per cell looked at, and binds a new one once
|
||||
`gaze`/`gazeValid`. Aurora centres the level's rings, each widened by 8 degrees to cover the lag
|
||||
and error of tracking, on the gaze snapped to a cell of two map texels (about 3 degrees), keeping up
|
||||
to 128 maps per eye, one per cell looked at, and binds a new one once
|
||||
its upload completes, the previous map staying bound meanwhile. Without a tracked gaze (a blink, no
|
||||
tracker, the setting off) the map is the forward one above, unchanged. Details and the Steam Frame
|
||||
checks are in `docs/steam-frame.md`.
|
||||
|
||||
@@ -615,8 +615,9 @@ struct EyeDensityMap {
|
||||
uint64_t map = 0;
|
||||
uint64_t lastUse = 0;
|
||||
};
|
||||
// The gaze cells' maps an eye keeps: a few glances' worth, each 2 bytes per 32x32 pixels.
|
||||
constexpr size_t kEyeDensityMapCacheSize = 32;
|
||||
// The gaze cells' maps an eye keeps, each 2 bytes per 32x32 pixels in a memory block of its own:
|
||||
// most of the cells a session's glances reach, so a glance rarely waits for a map to upload.
|
||||
constexpr size_t kEyeDensityMapCacheSize = 128;
|
||||
|
||||
struct StereoEyeTarget {
|
||||
webgpu::TextureWithSampler color;
|
||||
|
||||
@@ -138,8 +138,16 @@ inline Gaze cell_gaze(uint32_t eyeWidth, uint32_t eyeHeight, uint32_t texel, con
|
||||
.tanY = fov.tanUp + (fov.tanDown - fov.tanUp) * v};
|
||||
}
|
||||
|
||||
inline uint8_t density(Level level, float eccentricity) noexcept {
|
||||
const Rings ring = rings(level);
|
||||
// Eye-tracked maps widen both rings by this much. The full-density region has to cover where the eye
|
||||
// lands after a saccade until the next game frame's map is bound, the tracker's error, and the
|
||||
// snapping of the gaze to cells and of the density to the driver's tiles (Turnip shades a whole
|
||||
// render-pass bin at one density).
|
||||
inline constexpr float kTrackedMarginDegrees = 8.0f;
|
||||
|
||||
inline uint8_t density(Level level, float eccentricity, float margin = 0.0f) noexcept {
|
||||
Rings ring = rings(level);
|
||||
ring.full += margin;
|
||||
ring.half += margin;
|
||||
if (eccentricity < ring.full) {
|
||||
return kFullDensity;
|
||||
}
|
||||
@@ -176,7 +184,8 @@ inline void build(uint32_t eyeWidth, uint32_t eyeHeight, uint32_t texel, const E
|
||||
static_cast<float>(eyeWidth);
|
||||
const float tanX = fov.tanLeft + (fov.tanRight - fov.tanLeft) * u;
|
||||
const uint8_t value =
|
||||
density(level, forward ? eccentricity_degrees(tanX, tanY) : angle_from_gaze_degrees(tanX, tanY, gaze));
|
||||
forward ? density(level, eccentricity_degrees(tanX, tanY))
|
||||
: density(level, angle_from_gaze_degrees(tanX, tanY, gaze), kTrackedMarginDegrees);
|
||||
uint8_t* texelBytes = &map.rg8[(static_cast<size_t>(y) * map.width + x) * 2];
|
||||
texelBytes[0] = value;
|
||||
texelBytes[1] = value;
|
||||
|
||||
@@ -199,8 +199,9 @@ TEST(Foveation, TheForwardGazeKeepsTheFixedMap) {
|
||||
|
||||
TEST(Foveation, TheFullDensityRegionFollowsTheGaze) {
|
||||
const EyeFov fov = left_eye();
|
||||
// Down and to the right, well off the forward direction.
|
||||
const Gaze gaze{.tanX = std::tan(20.0f * kDegrees), .tanY = std::tan(-15.0f * kDegrees)};
|
||||
// Down and to the right, off the forward direction, with the widened full-density region still
|
||||
// inside the eye so its centre is not pulled in by the edge.
|
||||
const Gaze gaze{.tanX = std::tan(12.0f * kDegrees), .tanY = std::tan(-10.0f * kDegrees)};
|
||||
Map map;
|
||||
foveation::build(1344, 1408, 32, fov, Level::High, map, gaze);
|
||||
double sumX = 0.0;
|
||||
|
||||
+9
-5
@@ -243,13 +243,17 @@ region to where the player looks:
|
||||
2. `vr/eye_gaze.h` turns the gaze into tangents of each eye's own view, which may be canted.
|
||||
`AuroraStereoFrame` carries them as `gaze` and `gazeValid`, appended after its existing fields.
|
||||
3. Aurora snaps the gaze to a cell of two map texels (64 pixels, about 3 degrees) and builds that
|
||||
cell's density map with the level's rings centred on the gaze (`gfx/foveation.hpp`).
|
||||
cell's density map with the level's rings centred on the gaze (`gfx/foveation.hpp`). Both rings
|
||||
are 8 degrees wider than the fixed map's: the full-density region has to cover where a saccade
|
||||
lands until the next game frame's map is bound, the tracker's error, and Turnip shading a whole
|
||||
render-pass bin at one density.
|
||||
|
||||
Each eye keeps up to 32 maps, one per cell looked at, so a glance back reuses its map instead of
|
||||
uploading a new one. A new map is bound once its upload has completed, and until then the eye keeps
|
||||
Each eye keeps up to 128 maps, one per cell looked at, so a glance back reuses its map instead of
|
||||
uploading a new one. Each map has a memory block of its own: Turnip reads a map through a host
|
||||
mapping of its memory, and Dawn's buffer uploads unmap the shared blocks they sub-allocate from. A new map is bound once its upload has completed, and until then the eye keeps
|
||||
the map it had. A blink, lost tracking or the setting turned off returns to the map centred on the
|
||||
forward direction, which is byte-identical to the fixed foveation map. No change to the Dawn patch
|
||||
was needed: maps stay immutable, and the patch already lets a view be rebound to another map.
|
||||
forward direction, which is byte-identical to the fixed foveation map. Maps stay immutable, and the
|
||||
Dawn patch lets a view be rebound to another map.
|
||||
|
||||
The session log reports `OpenXR eye gaze: available` (or that the runtime has no eye tracker),
|
||||
`OpenXR eye gaze: tracking` at the first tracked sample, and `eye foveation medium following the
|
||||
|
||||
Reference in new issue
Block a user