commit 8c7dfc322ffa0d7b68a9db4e6918e53a95656ad7 Author: bod09 <2652540+bod09@users.noreply.github.com> Date: Thu Oct 1 23:00:45 2026 +0100 frame-autopass: automatic colour/IR passthrough for the Steam Frame Co-Authored-By: Claude Opus 5.5 diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml new file mode 100644 index 0000000..6b460c6 --- /dev/null +++ b/.github/workflows/build.yml @@ -0,0 +1,36 @@ +# Build and test every push; publish a release for every v* tag. +# The release package is built the same way on every push, so a tag never +# ships something that has not been built before. +name: build + +on: + push: + pull_request: + +permissions: + contents: write + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 # git describe needs the tags for the version + - name: Fetch toolchain and SDK headers + run: scripts/fetch-deps.sh + - name: Host tests + run: make test + - name: Release package + run: make dist + - uses: actions/upload-artifact@v4 + with: + name: frame-autopass-${{ github.sha }} + path: dist/ + - name: Publish release + if: startsWith(github.ref, 'refs/tags/v') + env: + GH_TOKEN: ${{ github.token }} + run: | + gh release create "$GITHUB_REF_NAME" --title "$GITHUB_REF_NAME" --generate-notes \ + dist/frame-autopass-aarch64.tar.gz dist/frame-autopass-aarch64.tar.gz.sha256 dist/install.sh diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..0ec0318 --- /dev/null +++ b/.gitignore @@ -0,0 +1,7 @@ +ref/ +build*/ +dist/ +third_party/openvr/ +toolchain/ +research/ +__pycache__/ diff --git a/FINDINGS.md b/FINDINGS.md new file mode 100644 index 0000000..7866c99 --- /dev/null +++ b/FINDINGS.md @@ -0,0 +1,1530 @@ +# Findings + +The research log behind frame-autopass, kept in the order things were found +(2026-09-29 to 2026-10-01), mistakes and dead ends included. It was written +during development with an AI assistant (see the disclaimer in the README), +so it uses the working names: **fapd** is today's `autopassd` and **fapctl** +is `autopass`. "The user" is the person who directed the project and wore +the headset for every test. Rules described in early sections were often +replaced later; the README describes what ships. + +Running log of discovery results. Everything in Step 1 was gathered +read-only: sysfs/procfs reads, log files, `strings` on binaries, and +reading the reference app's source. No device node was opened, no binary +from the reference app was built or run, nothing on the headset was +written. + +## 1. Reference app (KominoVR/frame-passthrough-shortcuts, v0.1.0) + +Cloned to `ref/frame-passthrough-shortcuts` (gitignored). MIT licence, +copyright "Frame Passthrough Shortcuts contributors". Vendors OpenVR SDK +(BSD-3) and nlohmann/json (MIT). + +### Camera source switch + +- Gets a private interface: `vr::VR_GetGenericInterface("IVRCameraPassthroughInternal_001")`. + Present in `/opt/steamvr/bin/linuxarm64/vrclient.so` on our build. +- Calls raw vtable slots on it (no header exists): + - slot 9: `bool get(void* self, uint8_t cfg[5])` + - slot 10: `void set(void* self, const uint8_t cfg[5])` +- The 5-byte config is: `[0] enabled/override, [1] stereo, [2] RGB source, + [3] disable devignetting, [4] sharpening`. Each byte must be 0 or 1. +- Switch = read all 5 bytes, flip byte 2 only, write back, read again to + verify. Refuses to write if byte 0 is 0 (it never turns the camera on). +- The code comment says it was verified on Frame firmware build 20260918. + Our headset is on BUILD_ID 20260922.6101926, so it's close but not the + same build. Vtable slot numbers are the fragile part: a SteamVR update + that reorders the interface would make slot 10 call the wrong function. +- Translucent/opaque is done through public settings instead: + `camera.roomViewStyle` (3 = translucent, 4 = opaque) via `IVRSettings`. + +### Arcturus presence detection + +- Walks `/sys/class/video4linux/*/name` for `arcimx616 *`, then opens + `/dev/v4l-subdevN` **O_RDWR** and issues a private ioctl `0x800456c1` + (reads a u32 "sensor connected" flag). It does this every 250 ms. +- The subdevs are held open by XRService. The ioctl is a read, but it + comes from a second process while XRService is streaming those sensors. + We should prefer not to copy this: sysfs `led_status` on the arcimx616 + i2c device already reports `present=1 powered=1` without opening any + device node (see section 3). + +### Autostart / registration + +- Runs as `VRApplication_Overlay` but never creates an overlay surface. +- `--install` writes `~/.config/frame-passthrough-shortcuts/app.vrmanifest` + (app key `local.frame.passthrough.shortcuts`, `is_dashboard_overlay: + true`, both `binary_path_linux` and `binary_path_linux_arm`), calls + `AddApplicationManifest` then `SetApplicationAutoLaunch(key, true)`. + SteamVR autolaunches dashboard-overlay apps with autolaunch set. +- Installs under `~/devkit-game/Frame_Passthrough_Shortcuts/` (also a + Devkit "Non-Steam" library entry). Uses a second, byte-identical + executable `frame-passthrough-control` for management commands because + SteamVR identifies apps by executable path. +- Controller input via SteamVR Input action manifest (thumbstick + double-press / 1 s hold), priority `k_nActionSetOverlayGlobalPriorityMax`. + Blocked while the dashboard is open. + +### Conflict risk with our app (not applicable) + +The user has never installed the reference app, so coexistence is out of +scope. Kept for the record only. + + +The reference app's main loop calls `reconcile_camera_source()` every +20 ms. If it has a saved preference (`camera-preference.json`, written the +first time the user toggles), it **re-asserts that source** whenever byte 2 +differs. Running both apps at once means they would fight and flicker. + +Mitigations, in order of preference: +1. Detect it (`VRApplications()->GetApplicationProcessId("local.frame.passthrough.shortcuts")` + or its `runtime.json`) and back off / warn. Our app then owns only the + auto mode and leaves manual RGB toggles to theirs, or +2. Tell the user to use ours instead of theirs for RGB/mono (theirs is still + fine for translucent/opaque; that path doesn't touch byte 2), or +3. Remove its saved preference file, which makes its reconcile a no-op + until the next manual toggle. (Touches their state; not preferred.) + +Status on this headset: **reference app is not installed** (no +`~/devkit-game`, no config dir, no vrappconfig). + +## 2. OS / runtime + +| Item | Value | +| --- | --- | +| OS | SteamOS `holo`, VARIANT_ID `vr`, VERSION_ID 0.3.0, BUILD_ID 20260922.6101926 | +| Kernel | 6.18.0-gfbdbca41fd45, aarch64 | +| SoC | Qualcomm SM8650 (Snapdragon 8 Gen 3), 8 cores, 15.6 GB RAM | +| SteamVR | `/opt/steamvr` (part of the read-only OS image), `bin/version.txt` = 1789606310 | +| OpenXR | `~/.config/openxr/1/active_runtime.json` -> `/opt/steamvr/steamxr_linuxarm64.json` (SteamVR) | +| OpenVR | `~/.config/openvr/openvrpaths.vrpath`, runtime `/opt/steamvr`, no external drivers | +| User | `steamos` (uid 1000), groups include `video`, `render`, `cdsp` | +| CV driver | `/opt/steamvr/drivers/cv` (driver_cv.so + XRService + libArcturusPerception.so) | + +Notable: XRService (Valve's tracking / passthrough service) is built on +the `arcturus-xr` codebase, and it names the colour module +`"Arcturus Camera V3"`. The colour module is therefore handled natively by +Valve's stack, not by a third-party driver. + +Public OpenVR camera API present in vrclient: `IVRTrackedCamera_006`. +driver_cv implements `IVRCameraComponent_003`. + +## 3. Cameras + +All camera sensors sit behind Qualcomm CAMSS (`/dev/media0`). V4L2 nodes: + +| Subdev | Sensor | I2C | Role (inferred) | +| --- | --- | --- | --- | +| v4l-subdev28 | arcimx616 0-0010 | cci0 i2c-0 | Arcturus colour, one eye | +| v4l-subdev29 | arcimx616 0-001a | cci0 i2c-0 | Arcturus colour, other eye (owns LED effect/status attrs) | +| v4l-subdev30 | og01a1bx 4-0060 | cci1 i2c-4 | OmniVision OG01A1B mono global shutter, supplies "l0" | +| v4l-subdev31 | og01a1bx 4-0036 | cci1 i2c-4 | same, "r0" | +| v4l-subdev32 | og0ve10x 5-0060 | cci1 i2c-5 | OmniVision OG0VE1B mono global shutter, "l1" | +| v4l-subdev33 | og0ve10x 5-003e | cci1 i2c-5 | same, "r1" | + +Four mono sensors (two pairs, l0/r0 and l1/r1) plus two colour. Which mono +pair feeds passthrough vs tracking-only is not yet confirmed; the log says +"Using 4 camera positions for CAD<>CAL alignment". All sensors report +runtime PM `active`. + +Ownership: **XRService (pid 2574) holds `/dev/media0`, all six sensor +subdevs, and capture nodes video0/3/6/7/9/13.** Nobody else touches the +sensors. Frames go from XRService to vrcompositor as dma-bufs (XRService +holds ~290 dmabuf fds) and are drawn by `CTrackedCamera` in vrcompositor +with shaders `tracked_camera_reprojection_simplified_{rgb,monochrome}_nv12` +(so both feeds arrive as NV12). Passthrough is reprojected onto a room +depth mesh ("Augmented passthrough geometry", "RoomView mesh" block queues). + +Other video nodes: +- `/dev/video99` "SteamVR": v4l2loopback, fed by `/opt/steamvr/bin/linuxarm64/v4l2cam --output=99`. + Format RGB3 1920x1080@30. v4l2cam uses `IVRHeadsetView`, so this is the + **composited headset view** (a virtual webcam), not a raw camera. + Readable by group `video` (we are in it). +- video22/23: Iris hardware video decoder/encoder. + +Arcturus i2c sysfs attributes (readable without opening a device node): +`0-001a/led_status` = `present=1 powered=1 effect=static ... last_error=0 override=0`. +This is a cheap, non-invasive presence check. + +Direct V4L2 capture from our own process is **not viable**: the sensors +and the CAMSS pipeline are exclusively in use by XRService, and grabbing +them would risk tracking. Frames must come through SteamVR. + +## 4. Light sensing + +### Hardware ALS: exists, probably unusable for room light + +`iio:device2` = Vishay **VCNL4040** (ALS + proximity) at i2c-6 0x60, on +the same bus as the two speaker amps. `in_illuminance_raw` and +`in_proximity_raw` are world-readable in sysfs. Nothing holds +`/dev/iio:device2`; `proxmicmute` references the proximity path. + +Current readings (headset idle, display backlight 0): illuminance 0, +proximity ~3 (near-level 220). This is almost certainly the face-presence +sensor pointing into the facial interface, which would read ~0 whenever +the headset is worn. **Unconfirmed**; needs a simple test (point the +front at a lamp vs the lenses at a lamp, reading sysfs). + +### Camera exposure / gain metadata: best candidate + +- XRService logs `Left/Right camera is now too dark or covered / + well-exposed / too bright` for the mono cameras, and `HmdAnalogGainMax + 1.25`. +- XRService carries colour metadata (`CameraColorFrameMetadata.h`) with + fields `exposureMs`, `isoSpeed`, `redGain/greenGain/blueGain`, + `whiteBalanceGains`, `useAutoExposure`, `useAutoWhiteBalance`. + Session settings push `captureSessionSettings.passthroughCameras.useAutoExposure`. +- Public route: `IVRTrackedCamera_006` frame header + (`CameraVideoStreamFrameHeader_t`) has `ulFrameExposureTime` plus + frame sequence and pose. Whether that is populated on Frame, and whether + it describes the colour or mono feed, is **untested**. +- Auto-exposure time alone saturates in the dark (it hits max exposure, + then gain rises). A usable light metric is `exposure x gain` or, failing + gain, exposure plus mean frame brightness. + +### Mean frame brightness: fallback + +Via `IVRTrackedCamera` frame buffers if accessible. `/dev/video99` also +works in principle but it is the composited view (content and overlays +bias it), so only a last resort. + +### Does the RGB feed stay readable while mono is selected? + +The reference README says the RGB sensor stays powered. Consistent with +what we see (both arcimx616 `rpm=active`, `powered=1`). Whether its +frames are still reachable from a client while byte 2 = 0 is **untested**. +That determines whether we can detect "the lights came back on" while in +IR mode (otherwise we need the mono feed's exposure for that direction). + +## 5. Phase 2 signals + +- Camera frames: only via SteamVR (`IVRTrackedCamera_006`), or not at all. + Unknown whether it exposes mono, colour, or both, at what resolution, + and whether the frame header pose is populated. +- Compositing: passthrough is drawn inside vrcompositor with fixed + shaders from the read-only image. Controls available: `roomView`, + `roomViewStyle` (translucent/opaque), the private 5-byte config + (stereo/RGB/devignette/sharpen), and `camera.monochromeTintHue` (0.67 + default, a single global tint for the mono feed). There is no public + per-pixel blend mode; `IVROverlay` layers are alpha-blended only. +- SteamVR already maintains a room depth mesh for passthrough + reprojection and XRService keeps a SLAM map (`serializedmap/`, 144 + keyframes / 2115 map points in the stats dump), both internal. + +## Open questions (need live but non-mutating tests, pending approval) + +1. VCNL4040 placement: read sysfs while the user shines a light at the + front vs into the lenses. +2. `IVRTrackedCamera_006` probe as a background OpenVR client: + `HasCamera`, frame sizes per frame type, acquire a stream for a few + seconds, log header (exposure, sequence, pose), mean luma, which feed it + is, and whether it changes when the source switches. +3. `IVRCameraPassthroughInternal_001` slot 9 read-only (get config) to + confirm the 5-byte layout on this build. + +## 6. Build and deploy setup (2026-09-29) + +- Cross-compiling on the PC with Zig 0.16.0 (`zig c++ -target + aarch64-linux-gnu.2.35`; the headset has glibc 2.39). libc++ is linked + statically, so binaries only need glibc. `scripts/fetch-deps.sh` fetches + Zig (sha256-checked) and the OpenVR SDK 2.15.6 (commit 0924064) into + gitignored dirs. `make test` runs host tests, `make device` builds for + the headset. +- Deploy target on the headset: `~/fap/bin/`. +- SteamVR runtime reports itself as **2.17.10** in client logs. + +## 7. Public tracked-camera API, first live probe (headset idle) + +Run with `camera_probe` as a Background client while the headset was +idle (display backlight 0, not worn, passthrough not showing): + +- `Prop_HasCamera_Bool` true, but `Prop_NumCameras_Int32` = 0 and + frame layout / stream format / firmware properties are unknown. +- `IVRTrackedCamera::HasCamera` true. `GetCameraFrameSize` and + `GetVideoStreamTextureSize` fail (`OperationFailed`) for all three + frame types, before and after acquiring. +- `AcquireVideoStreamingService` succeeds, but + `GetVideoStreamFrameBuffer` on that handle returns `InvalidHandle`. +- vrserver logged nothing about the camera request. +- Settings as stored: `camera.enableCamera` true, `roomView` 2, + `roomViewStyle` 0, `enableConstructRoomView` true, + `monochromeTintHue` 0.67. + +Not conclusive yet: it may only work while passthrough is actually +running. Next: repeat with the headset worn and passthrough visible. If +it still fails, the public frame API is not wired up on Frame and light +sensing has to come from elsewhere. + +## 8. Light sensor torch test and worn probe (2026-09-29 21:15) + +VCNL4040 sampled at 5 Hz for 60 s while the user wore the headset in a +lit room, then shone a phone torch at the front and then the inside: + +- Normal lit room, worn: raw 0-5 (0-0.5 lux at scale 0.1). +- Torch spike 1 (~15-20 s): raw up to 2200 (220 lux). +- Torch spike 2 (~33 s): raw up to 223. +- Proximity sat around 12-13 while worn and dropped to ~2-5 while the + headset was moved about. It never came near the 220 near-level. + +Verdict: whichever way it faces, a lit room reads under 1 lux with the +fixed 80 ms integration time (changing it needs root). **Not usable as +a room-light sensor.** Dropped from the plan. + +The "worn" camera_probe rerun happened around the moment passthrough +was closed ("Passthrough cameras paused" 21:16:57), so it does not +settle whether IVRTrackedCamera works while passthrough runs. Same result +as idle: sizes fail, frame read returns InvalidHandle. Needs one clean +retry. + +XRService's own log (`~/.local/share/Steam/logs/xrservice.txt`, a +symlink to the current session log) is live and readable. It records +`Passthrough cameras paused/resumed`, `Tracking cameras streaming +paused/resumed` (headset taken off / put on), and, under the torch, bursts +of `Max number of iteration reached when estimating the CCT from grey +world gains` (colour AWB struggling). No periodic exposure values. + +## 9. Private interface mapped from this headset's vrclient.so + +Copied `/opt/steamvr/bin/linuxarm64/vrclient.so` to the PC (read-only) +and located the implementation through RTTI and R_AARCH64_RELATIVE +relocations, without using the reference app's code: + +- Class `CVRCameraPassthroughInternal` (typeinfo 0x616948), implementing + `vr::IVRCameraPassthroughInternal`, interface string + `IVRCameraPassthroughInternal_001`. +- Vtable at 0x60cc90 with **17 methods**: + +| idx | addr | approx size | +| --- | --- | --- | +| 0 | 0x1a4728 | | +| 1 | 0x1a5670 | | +| 2 | 0x1a4a78 | | +| 3 | 0x1a4cd0 | | +| 4 | 0x1a4df8 | | +| 5 | 0x1a4f30 | ~1.4 KB (largest) | +| 6 | 0x1a44b0 | | +| 7 | 0x1a46d0 | | +| 8 | 0x1a4b38 | | +| 9 | 0x1a49d8 | ~160 B | +| 10 | 0x1a4bf8 | ~216 B | +| 11 | 0x1a4708 | | +| 12 | 0x1a4b98 | | +| 13 | 0x1a4968 | | +| 14 | 0x1a5498 | | +| 15 | 0x1a4498 | | +| 16 | 0x1a45c0 | | + +- Strings owned by the class: `Failed to import the camera frame + dma-buf`, `Failed to reference the shared image resource`, `...shared + spline distortion resource`, `...spline distortion resource file + descriptor`, plus shared-memory names `VR_CameraPassthroughState` and + `VR_CameraPassthroughMutex`. + +So this private interface is not only the source toggle: it also hands +out **camera frames as dma-bufs** together with the lens distortion +(spline) data. That is potentially the frame access we need for the +light level (Phase 1) and for Phase 2. The config state lives in a shared +memory block guarded by a named mutex. + +Next: disassemble the 17 methods to learn their signatures. The host has +no arm64 disassembler yet. + +## 10. The public camera stream blanks the user's passthrough (21:23) + +Clean retry with the headset worn and passthrough running (XRService +logged `Passthrough cameras resumed` at 21:23:02; probe ran 21:23:11 to +21:23:16): + +- Same result: frame sizes fail, frame reads return `InvalidHandle`. + **The public IVRTrackedCamera frame API is not usable on Frame.** +- Side effect reported by the user: every probe run turned their + passthrough **black** until they toggled passthrough off and on. + vrcompositor logged, at the moment the probe disconnected: + `[CTrackedCamera] : Depth mesh block queues disconnected successfully.` + The compositor's own passthrough renderer (CTrackedCamera) shares the + tracked-camera service; a client's acquire/release (or its disconnect + while holding the stream) tears that renderer down. The earlier 21:16 + `Passthrough cameras paused` was very likely caused by the probe too. +- Tracking was not affected (no SLAM or VIO resets in the stats). + +Action taken: the probe binary was deleted from the headset and the +stream acquisition code removed from `tools/camera_probe.cpp`; it now +only reads properties and settings. + +Rule from here on: any live call into SteamVR's camera stack is assumed +able to disturb the user's view. Before running one, say so, have the +user ready to toggle passthrough, and prefer analysing the binaries +offline first. + +## 11. Private interface: get/set config confirmed by disassembly + +`tools/disasm.py` (capstone, in `toolchain/venv`) disassembles the +headset's own vrclient.so. `this+0x18` holds the shared-state accessor; +helpers at 0x258500 / 0x258578 / 0x258508 are lock / get-pointer / unlock +of the `VR_CameraPassthroughState` block (guarded by +`VR_CameraPassthroughMutex`). + +- Slot 9, `bool GetConfig(uint8_t cfg[5])`: under the lock, copies + state bytes 2..6 into `cfg`, returns `state[2] != 0` (the enabled + byte). With `cfg == nullptr` it returns that flag without the copy. +- Slot 10, `void SetConfig(const uint8_t cfg[5])`: under the lock, + compares `cfg` with state bytes 2..6 and returns without doing anything + if equal. Otherwise writes all 5 bytes, unlocks, and broadcasts an event + built from the rodata pair (812, 0xffffffff): event type 812, device + index -1 (all), via a client singleton's vtable +0xc0. +- So the layout is: state[2] enabled, [3] stereo, [4] RGB source, + [5] disable devignetting, [6] sharpening (byte meanings from the earlier + analysis; the code itself only shows a 5-byte block starting at +2). +- Writing is a no-op when unchanged, so a periodic re-assert is cheap but + still wasteful; our app should only write on a decision change. + +## 12. VR_CameraPassthroughState is a readable tmpfs file + +- vrclient opens it (at 0x1a5adc) with size **0x6200 = 25088 bytes**, + plus a named mutex `VR_CameraPassthroughMutex`. +- On the headset it is `/dev/shm/u1000-Shm_78c2379b` (mode 0777, owner + steamos), mapped by vrcompositor, vrserver and v4l2cam. The hash in the + name is not yet derived, so tools find it by its unique size. +- Reading the file is completely passive: no SteamVR call, no lock, no + device. It cannot affect passthrough or tracking. + +Static layout (snapshot at 21:26 with passthrough paused): + +| offset | content | +| --- | --- | +| 0x00-0x01 | 01 00 (header) | +| 0x02-0x06 | the 5-byte config: enabled, stereo, RGB, disable devignetting, sharpening. Read as `01 01 01 00 01` (RGB active, as expected with the Arcturus attached) | +| 0x14 | stream 0 (mono): count 4, 1056 x 1024. Per-frame records mostly empty in this snapshot | +| 0x20e8 | stream 1 (colour): count 4, 2464 x 2464. Per-camera blocks 0x1040 apart, per-frame records 0x104 (260) bytes apart | + +Colour per-frame record fields found so far (relative to the float +1/2.2 gamma marker at record start + 0xd8): + +- record start: frame ids, two float64 timestamps (~5200 s, same clock), + fx fy cx cy (~1478, 1479, 1369, 1250 for 2464 px), 4 distortion + coefficients, rotation matrices, an identity 3x3. +- after the marker: 0, 0, 1.0, 1.0, **a per-frame value (0.861, 0.843, + 0.597)**, an int 1, **0.000331492 twice (candidate: left/right exposure + time in seconds, 331 us)**, 1.0, 1.0 (candidate gains), 0, then what + looks like the next record's owner pid (0x0a12 = 2578, vrcompositor) + and a sequence number. + +Hypotheses to test live: g6/g7 = exposure time, g8/g9 = gain, g4 = +something scene-dependent (AE target, brightness or a white balance +figure). Also: do the mono records fill while colour is selected (that +decides whether we can see the IR side's exposure)? + +`tools/shm_sampler.py` samples the file at 20 Hz and logs per-record +fields to CSV, plus optional raw snapshots for offline re-parsing. + +## 13. Live capture: a light signal in shared memory (21:32-21:34) + +95 s passive capture (`shm_sampler.py`, 20 Hz, raw snapshots kept) with +the headset worn and colour passthrough running throughout (XRService +resumed 21:31:53, no pause during the capture). User: ~20 s lit, ~30 s +lights off, then lights on. + +- Colour per-frame records update at ~13 Hz per slot. Config stayed + `01 01 01 00 01` (RGB) the whole time. +- **g4 (float at gamma marker + 0x14; e.g. 0x220c, 0x324c) tracks room + light:** 0.8-1.0 lit (saturates at 1.0), fell from 0.8 to 0.0 over + t=13-18 s, stayed 0.00-0.03 in the dark (one 0.14-0.37 blip at + t=35-38 s, likely a screen or light leak), rose from 0.2 to 0.8 over + t=48-56 s, back to 0.85-0.95. Identical in both camera blocks. +- A second field at 0x2414 / 0x3454 behaves like a slow, sparsely + updated version of the same (0.81 lit, 0 dark, 0.67-0.80 lit again, + updates every few seconds). +- The candidate "exposure" pair (0.000331) and "gains" (1.0) never + changed: that guess was wrong. +- Several rotation-matrix elements also separated lit from dark, but + those are head pose (the user moved differently in the dark), not + light. Treat any pose-derived field as a confound. +- **The mono stream region (0x14-0x20e8) never changed at all**: mono + records are not published while RGB is selected. + +What g4 is, exactly, is unknown. It behaves like a normalised "enough +light for colour" figure (clamped 0..1), which is exactly the colour to +IR trigger we want. It has no gradation below the point where colour +fails, so it cannot on its own tell "dim" from "pitch black". + +Open question that decides the IR to colour direction: do the colour +records keep updating while mono is selected? That needs a real source +switch (private SetConfig), which changes what the user sees. + +## 14. fapctl: our own reader and switch + +- `src/passthrough_state.*`: passive parser of the shared state (config + bytes, newest colour record's timestamp and light value). Tested on two + real snapshots from the live capture (`tests/fixtures/state_{lit,dark}.bin`: + light 0.884 and 0.000). +- Trap found while writing it: the gamma marker is stored as + `0x3ee8ba2f` (1/2.2 in double, rounded to float). `1.0f / 2.2f` in C++ + gives `0x3ee8ba2e`, so the marker must be matched by its bit pattern. +- `src/private_camera.*`: get/set through `IVRCameraPassthroughInternal_001` + slots 9/10. Before calling, the first 6 and 7 instruction words of the + two methods are compared with those in this headset's vrclient.so + (position-independent words only). Any SteamVR update that changes them + makes the tool refuse instead of calling into unknown code. +- `tools/fapctl`: `status [SECONDS]` (passive), `connect-test` (VR_Init + as Background, wait, shut down), `get`, `set rgb|mono` (changes only + the RGB byte, refuses if the camera is not enabled, verifies read-back). +- Only `status` has been run on the headset so far (21:36): config + `enabled=1 stereo=1 rgb=1 disable_devignetting=0 sharpening=1`. + +## 15. First real switch test (21:41:48-21:42:56, user approved) + +Headset worn, colour passthrough running (resumed 21:41:26). Timeline +from the run start: 0 s `fapctl connect-test`, 10 s `fapctl set mono`, +55 s `fapctl set rgb`, passive sampler throughout. User: lights on, off +at ~25 s, back on ~5 s early (~35 s). + +- Both writes succeeded and read back exactly; only the RGB byte changed. +- **No blanking**: vrcompositor logged only `[CTrackedCamera] Late + initialization successful` at each switch (21:41:49, 21:42:34), and none + of the teardown seen with the IVRTrackedCamera probe. Connecting and + disconnecting as a Background client alone did nothing visible in logs. +- XRService logged `Transition to IMUFallback` at 21:41:54.8 (6 s after + the mono switch), recovered at 21:41:58.3. Not proven related: the same + transition occurs 14 times in today's log, including at 21:32:58 during + the passive capture with no switching, and on every headset off/on. + Needs watching in future tests. +- **In mono the colour records stop updating** (newest colour timestamp + frozen at the switch) and the mono region starts updating. Back on RGB, + colour records resume immediately (g4 0.72 then 0.40-0.72). +- **Nothing in the shared state tracks light while mono is showing.** + Across all 118 words that varied during mono, the only lit/dark + differences are pose elements (separation < 1.4). Mono records carry no + gamma marker and no exposure-like field; a pair at 0x138/0x23c/0x1178/0x127c + is 0 except a blip at ~30 s. +- XRService's colour white-balance warnings (`estimating the CCT from grey + world gains`) continued through the mono period (21:42:04-21:42:33), so + XRService keeps processing colour frames while mono is displayed; it just + does not publish them to this block. + +Consequence for Phase 1: colour to IR is solved (g4 from shared memory, +passive). IR to colour has no signal yet. Options: +1. Find the colour statistics in XRService's other IPC (the world-writable + `/dev/shm/XR_ServerResponse_*` / `XR_ClientRequest_*` buffers), still + passive. +2. A brief periodic colour "peek" (switch to RGB for ~1-2 s, read g4, + switch back). Visible flash; last resort. +3. Only auto-switch colour to IR, and return by controller. + +## 16. Hunting a light signal usable in IR mode + +User saw nothing unusual during the 21:41 switch test (no blink, no view +jump); the IMUFallback at 21:41:54 is logged as unrelated for now (user's +guess: a controller idling off), still to be watched. + +The `XR_*` IPC buffers are mapped by XRService, vrserver and vrcompositor +(`XR_ServerResponse_High_Data` holds JSON status like `TRACKING_LEDS` +occurrences). XRService maps **73 shm files, 67.7 MB** in total. + +Tools: +- `tools/shm_capture.py`: passively snapshots every shm file a process + maps (small files every 0.25 s, >1 MB every 2 s, zlib-compressed), + plus `--also` extra files. ~16% of one core, ~1 MB/s on disk. +- `tools/shm_analyse.py`: offline; takes g4 from the passthrough state + as ground truth during the RGB phase, correlates every 32-bit word of + every other buffer with it, and prints the best candidates' ranges in + the RGB and mono phases. Verified on a synthetic capture with one + planted signal among 1024 random words (found at r = 1.000, nothing + else above 0.8). + +Plan: one run with an RGB phase (lit / dark / lit) then a mono phase +(lit / dark / lit), then back to RGB. + +## 17. Whole-process capture 1 (21:52-21:54): pose confound again + +Run: user lights on 0-30 s / off 30-60 s / on 60-90 s; `fapctl set +mono` at 45 s, `set rgb` at 75 s (fixed times, not detection). Capture +of all 73 XRService shm files plus the passthrough state: 105 MB, 380 +state snapshots. No IMUFallback during the run. + +- g4 ground truth (RGB phase): ~0.9 lit, ramp 0.54 to 0 over 20-28 s, + 0 while dark. +- 112 words correlate with g4 at |r| >= 0.7. A stricter test (must step + the same way at lights-off in RGB and at lights-on in mono, and be + steady within segments) still ranks pose-like values top: + `u1000-Shm_bc4005bc+0xef194` / `u1000-Shm_4c4bddef+0xa700,+0xb724,+0xbb00` + (8.08 lit, 6.44 dark, 6.35 mono dark, 8.06 mono lit: looks like an + unwrapped angle, 2pi..2.6pi), `u1000-Shm_7f04d645+0x1e0` and copies, + `XR_ServerRequest_VLow_Data+0x90/+0x110`, plus rotation-matrix blocks. +- Cause: the user stands at the light switch (facing it) whenever it is + dark and elsewhere when lit, so head pose is perfectly confounded with + light in both phases. The step test cannot separate them. + +Next capture must decouple pose from light: the user stays still at the +switch, looking at one spot, for the whole run. + +## 18. Capture 2 (22:01-22:02): head still, light switched by Home Assistant + +The user sat still looking at one spot. I toggled the room A light (LIFX +Mini, via the user's Home Assistant in their Chrome): **off 22:01:31 +(T0+30.4 s), on 22:02:03 (T0+61.9 s)**, times from HA's activity log. +`set mono` at T0+45, `set rgb` at T0+75. No IMUFallback. + +- g4 went 1.00 to 0.44 (not 0: the room kept other light), back to 1.00. +- With the head still, pose-derived words no longer rank (score < 2). +- One group stood out, stepping exactly with the light in **both** RGB + and mono: xyz triplets in `u1000-Shm_7f04d645` (24,002,048 bytes, about + 4 x the "RoomView mesh data block queue of size 6000000"), e.g. +0x5c728c + (0.128, -0.371, 0.421) lit vs (0.100, -0.289, 0.328) dark. Fell over + 30-34 s and returned at 64 s while mono was displayed. +- **It is the passthrough depth mesh, not a light meter.** The block is + smoothly varying vertex triplets; 174,758 of ~207k nonzero words changed + >2% between lit and dark (dark/lit ratio median 0.95, 5-95% 0.71-1.11). + XRService's stereo depth estimate changes with the lighting, so the whole + mesh shifts. Scene- and pose-dependent, so not usable as a light signal, + although it shows the mono pipeline does react to room light. + +Static look at the colour sensor driver (built into the kernel): symbols +`arcimx616_get_ctrl`, `arcimx616_set_ctrl`, `arcimx616_ctrl_read_ops`, +`arcimx616_ctrl_temperature`, `arcimx616_private_ioctl`, modes +3820x2464 and 1640x1232 binned. So the subdev has V4L2 controls; whether +exposure/gain are among them (and are updated by XRService's AE while mono +is shown) is unknown without querying the device. + +## 19. Colour sensor V4L2 controls (read-only query, 22:08:57) + +User approved opening the Arcturus sensor subdevs read-only. +`tools/subdev_ctrls` opens the node O_RDONLY and issues only +VIDIOC_QUERY_EXT_CTRL / VIDIOC_G_EXT_CTRLS. Headset idle, passthrough +paused since 22:03:49. No XRService or kernel log entries followed. + +Both `arcimx616` subdevs (28 = 0-0010, 29 = 0-001a) expose the same 5: + +| id | name | type | range | flags | value (28 / 29) | +| --- | --- | --- | --- | --- | --- | +| 0x00980900 | Brightness | int | 0..16777215 | 0x1020 (slider, not volatile) | 77841 (0x13011) / 12305 (0x3011) | +| 0x0098f900 | Temperature | int | 0..2^32-1 | 0x1080 (volatile) | 37 / 37 | +| 0x009f0901 | Link Frequency | intmenu | 0..0 | 0x1004 (read-only) | 0 | + +No standard Exposure or Analogue Gain control. "Brightness" is a 24-bit +non-volatile control, so it holds the last value someone set with +S_CTRL, presumably XRService. The two sensors differ only in bit 16. +Hypothesis: XRService packs exposure and/or gain into it. Needs a live +test to see whether it moves with light, and whether XRService keeps +writing it while mono is displayed. + +## 20. Run 3 (22:12:29-22:14:04): sensor controls do not track light + +Head still; HA light off 22:13:02 (T0+33), on 22:13:31 (T0+62); mono at +T0+45, RGB at T0+75. Both arcimx616 subdevs polled read-only at 5 Hz. +No IMUFallback, no log noise. + +- **Brightness stayed constant** on both sensors for the whole run + (77841 / 12305) through light, dark, colour and mono. It is static + configuration, not auto-exposure. Temperature rose 47 to 54 C (warm-up). +- So XRService does not run AE through standard V4L2 controls; it must go + through `arcimx616_private_ioctl` or the ISP. Reading those would need + reverse-engineering the driver and touching the sensor bus while + XRService streams: not pursued. + +## 21. Peek latency: colour light value is valid immediately after a switch + +From capture 2 (state file every 0.25 s): after `set rgb` at T0+74.8, the +very first snapshot already held a fresh colour record (timestamp jumped +from the frozen 7337.302 to 7367.388) with the correct light value 1.000, +and it stayed consistent afterwards. There is no AE settling after the +switch, which fits XRService keeping colour auto-exposure running while +mono is displayed (its colour AWB log lines also continue in mono). + +So a "peek" (switch to RGB, wait for the first fresh colour record, read +g4, switch back) could be short: bounded by the switch plus one colour +frame (~13 Hz, so roughly 80 ms) plus compositor re-init. Exact latency +still to be measured at high polling rate. + +Status of IR-to-colour detection, all passive routes tried: +- shared passthrough state: colour records frozen in mono (section 15) +- all 73 XRService shm buffers: only the depth mesh reacts (section 18) +- sensor V4L2 controls: static (this section) +- ALS: too insensitive (section 8); public camera API: dead and harmful + (section 10) + +## 22. Is there an official way? (web search, 2026-09-29) + +No. Checked Valve's Steamworks Steam Frame docs, SteamVR 2.17 release +coverage, Arcturus Vision's site, the Road to VR review and a VR.org +developer piece: + +- Valve's Steam Frame developer docs (setup, Unity, Unreal, Godot, custom + engines) never mention passthrough, environment blend modes, camera + state or whether a colour module is attached. Developer mode offers + SSH/ADB/RDP, beta branches, debugging and performance overlays. +- SteamVR does not implement OpenXR passthrough extensions; the only + camera path is OpenVR's tracked-camera interface, which returns nothing + on Frame (section 10). The community OpenXR passthrough layer + (Rectus/openxr-steamvr-passthrough) relies on that same interface. +- SteamVR 2.17 (released 2026-09-11) added a Quick Access slider for + environment and camera brightness: a control, not a meter. +- Arcturus Vision: no public SDK/API. The module "instantly takes over" + passthrough on plug-in; reviews and the site describe no low-light + fallback or colour/mono switch. + +Everything this project does (the source switch included) is therefore +through private interfaces, and will need re-verification after SteamVR +updates (fapctl's fingerprint check exists for that reason). + +Sources: partner.steamgames.com/doc/steamhardware/steamframe/setup, +vr.org/articles/steam-frame-color-passthrough-arcturus-vision-149-mixed-reality-developers-2026, +roadtovr.com/arcturus-vision-camera-review-steam-frame-passthrough-add-on/, +gamingonlinux.com/2026/09/steamvr-2-17-arrives-ready-to-go-for-the-steam-frame/, +arcturus.vision/howto, github.com/Rectus/openxr-steamvr-passthrough + +## 23. Private interface: producer and consumer halves (offline RE) + +`tools/disasm.py` gained `--until ADDR` (linear disassembly of a range), +since loops end the heuristic walk early. + +- **Slot 0** `SetGraphicsDevice(desc*)`: `desc->type` must be 2 (Vulkan; + otherwise logs "only Vulkan graphics devices are supported"); + `desc->data` points at {VkInstance, VkPhysicalDevice, VkDevice, VkQueue, + u32 queueFamilyIndex}, stored at this+0x50..0x70; sets this+0x74 once + the device is usable. Returns 0 on success, 1 on bad arguments. +- **Slot 1** is the **producer**: `(stream, desc*, images*)`, requires + slot 0 first (returns 2 otherwise), `desc` starts with a count <= 16, + per-image structs are 0x68 bytes. For each image it sets up a "shared + camera frame" (0x1a4808) and the "shared spline distortion images" + (0x1a5140), then under the state mutex copies a new 0x20d4-byte stream + block into the shared state (stream 0 at +0x14, stream 1 at +0x20e8) and + increments the stream's generation counter. The records' owner pid is + vrcompositor, so **the compositor publishes its camera frames to other + clients through this interface**. +- **Consumers** (all take `stream` 0 = mono, 1 = colour, and check a + per-stream "imported" flag at this+0x78 / this+0xc0): + slot 2 `(stream, u8* out)` returns the stream's available flag (+0x40); + slot 3 `(stream, eye <= 1, out*, size)`; slot 4 `(stream, eye <= 1, + frame < count)`; slot 6 `(stream, eye <= 1, out*)` calls four virtuals + on the imported per-eye image object (vtable +0xb0, +0xb8, +0x40, + +0x48) and writes the results (two handles, then width/height-like + values) to `out`. + +So a Vulkan client can import the frames of the stream that is currently +displayed. Only one stream is published at a time (section 15), so in IR +mode we would receive mono frames. + +Before building the Vulkan import, a cheap feasibility test: does the +**mono passthrough image** itself change measurably when the room light +comes on, or does XRService's auto-exposure flatten it? `/dev/video99` +(SteamVR's virtual webcam of the headset view, RGB3 1920x1080@30, group +`video`) shows the displayed passthrough. Reading it consumes an existing +output and changes no SteamVR state. + +`tools/view_grab` reads `/dev/video99` as a normal V4L2 capture consumer +(mmap buffers) and prints mean / p5 / p50 / p95 luma and the fraction of +clipped pixels, plus optional 1/8-scale PGM thumbnails. First run (22:24, +headset idle): 1920x1080 RGB24 frames arrive at the expected rate, all +black because nothing is displayed. No SteamVR log lines resulted. + +## 24. Run 4 (2026-09-30 22:40-22:42): image grain detects light while in IR + +Mono selected at T0+5 s, back to RGB at T0+115 s. Room A light via HA: +off 22:40:48, on 22:41:18, off 22:41:38, on 22:41:58 (T0+31.8, 61.8, +81.8, 101.8). `view_grab` read `/dev/video99` at 5 Hz with a 1/8-scale +point-sampled thumbnail every 5 s. No IMUFallback. The user had Steam +panels open in the middle of the view, and their content changed during +the run. + +Whole-frame brightness (1 s means) in mono: +- Lights off: a sharp ~1 s dip (p50 55 to 11, 41 to 17), auto-exposure + recovers within ~2 s. Lights on: a short spike, then settles. +- Steady state: mean is largely equalised by AE, but contrast differs: + lit p5/p95 ~7-8 / 83-86, IR only ~14 / 65. +- Panel content changes (t=11-30 s) move the whole-frame stats as much as + the light does, so whole-frame brightness is not reliable on its own. + +**Grain (median absolute Laplacian) in the top 35% of the thumbnail +(ceiling and upper walls, clear of panels):** + +| condition | samples | grain | +| --- | --- | --- | +| lit, mono (5-31 s) | 6 | 7, 7, 7, 8, 7, 7 | +| dark, mono (36-62 s) | 6 | 15, 15, 15, 14, 15, 15 | +| lit, mono (67-82 s) | 4 | 6, 6, 6, 6 | +| dark, mono (87-98 s) | 3 | 14, 14, 14 | +| just after on (102.8 s) | 1 | 9 (transient) | +| lit, mono (108-113 s) | 2 | 6, 6 | + +A clean 2x separation, reproduced over two cycles. Cause: with only +the IR illuminators the mono sensors run high analogue gain, which shows +as grain; AE can equalise mean brightness but not noise. This is a +**usable IR-to-colour signal**, obtained passively from the displayed view. + +Caveats to resolve before relying on it: +- Measured on thumbnails of the composited view (reprojected, sharpened, + with overlays). Needs a proper full-resolution estimator and a choice + of region robust to panels and head motion. +- Only two light levels tested (room A light on/off with some other light + present). Needs a calibration sweep of intermediate levels to map grain + to the colour-side light value g4, so the thresholds in both directions + meet with sensible hysteresis. +- `/dev/video99` shows what is displayed: it only reflects passthrough + while passthrough is visible. That matches when switching matters, but + translucent passthrough over an app would mix content in. + +## 25. Calibration sweep (2026-09-30 22:54-22:59) + +Room A light stepped through 100/60/35/20/12/7/4/2/1/off, 12 s per step, +first in colour, then in mono, by `hass.callService` from the Home +Assistant page (64 ms per call; HA's own last_changed matched my logged +call time within 0.11 s; PC and headset clocks agree within 0.1 s). No +IMUFallback. The user looked at a bed and a sloped wall this time, with +panels closed (a different scene from run 4). + +Medians per step (first 4 s of each step skipped): + +| light | colour: view mean | colour: g4 | mono: view mean | mono: grain (g8) | +| --- | --- | --- | --- | --- | +| 100% | 47.8 | 1.00 | 47.0 | 4 | +| 60% | 32 | 0.87 | 46.6 | 8 | +| 35% | 19 | 0.435 | 45.9 | 16 | +| 20% | 10.7 | 0.435 | 45.5 | 10-11 | +| 12% | 7.4 | 0.435 | 43.7 | 11 | +| 7% | 6.0 | 0.435 | 44.4 | 11 | +| 4% | 5.2 | 0.435 | 43.9 | 11 | +| 2% | 4.9 | 0.435 | 43.6 | 11 | +| 1% | 4.9 | 0.435 | 43.1 | 11 | +| off | 4.0 | 0.435 | 42.2 | 11 | + +Findings: +- **Colour AE barely compensates**: the colour image darkens steadily + (47.8 to 4.0). Colour passthrough is visibly poor from ~35% down. +- **g4 is not a linear light meter**: 1.0 at full light, 0.87 at 60%, + then a floor of 0.435 from 35% down (in capture 1, fully dark, it went + to 0.00). As a "colour is struggling" flag it works in every run: + **g4 < ~0.6** when light <= 35%, >= 0.85 when light >= 60%. +- **Mono grain is non-monotonic**: 4 at 100%, 8 at 60%, peaks at 16 at 35% + (visibly grainiest frame: maximum sensor gain), then 10-11 from 20% down + where the image takes on the flat, hazy IR-illuminated look (XRService + evidently leans on the IR illuminators and lowers gain). Every dim state + is >= 10 and every bright one <= 8, so **grain <= ~7** means "light is + back to at least ~60-100%". +- Mono view mean hardly moves (47 to 42), as expected with AE. + +Resulting Phase 1 rule (to be validated in other rooms and views): +- colour to IR: g4 < 0.6 for the confirmation time; +- IR to colour: grain <= 7 for the confirmation time; +- the 35-60% band is a natural hysteresis gap: whichever mode is active + stays. + +Caveats: one room, two views; grain depends on scene texture (a busy +bookshelf would read higher than a plain wall); the estimator runs on +the composited view, so it is only meaningful while passthrough is +displayed (which is also the only time switching matters). + +## 26. fapd first runs (observe-only), liveness bug, standby behaviour + +Both streams carry a per-frame float64 timestamp at record+0x10 (records +at stream+0x38 + cam*0x1040 + rec*0x104). Only the displayed stream's +timestamps advance; a paused stream leaves its last value in place. + +- **Bug found in the first observe-only run (23:12):** fapd treated the + first timestamp it read as "advancing", so a stale value counted as a + live frame and it began confirming a switch on stale data. Fixed with a + baseline-first rule, now in `src/liveness.hpp` with tests (a mutant + restoring the bug fails 22 checks). +- **Connecting as a `VRApplication_Overlay` client wakes the headset from + standby.** 23:13:22: vrserver "leaving standby", displays on, tracking + and passthrough cameras resumed, the instant fapd connected. Standby + returned 5 s later (23:14:30) **while fapd was still connected**, so + fapd does not keep the headset awake. Disconnecting an overlay client + also causes one brief wake (23:15:25). A `VRApplication_Background` + client (fapctl connect-test, 23:14:09) does not wake it. + With autostart, fapd connects when SteamVR starts and disconnects when + it quits, so neither wake happens in normal use. +- The second "passthrough in use" seen at 23:13:25 was real: our own + connection had woken the headset. + +## 27. Towards event-driven operation + +User requirement: feel instant, stay lightweight, and do nothing at all +while passthrough is not in use (no 2 Hz idle polling). + +OpenVR candidates (openvr.h): `VREvent_RoomViewShown` (526) / +`VREvent_RoomViewHidden` (527) ("for scene apps only - not for construct +or transient bounds"), `VREvent_EnterStandbyMode` / `LeaveStandbyMode` +(106/107), `VREvent_DashboardActivated/Deactivated` (502/503), +`VREvent_CameraSettingsHaveChanged` (851). Which of these fire on the +Frame when passthrough is toggled has to be measured. +`tools/event_probe` (Background client, does not wake the headset) logs +every event with its name. + +### Measured (2026-09-30 23:18, event_probe, user toggling passthrough) + +Passthrough toggled on/off three times (xrservice: resumed 23:18:25.6, +paused 30.6, resumed 35.5, paused 40.5, resumed 45.5, paused 50.5): + +- **`VREvent_CameraSettingsHaveChanged` (851) fired on every toggle, on + and off**, within ~50 ms, together with undocumented event 816. +- `VREvent_RoomViewShown/Hidden` (526/527) never fired, as the SDK + comment warns. +- Opening the dashboard: `VREvent_DashboardActivated` (502). +- Taking the headset off: `VREvent_EnterStandbyMode` (106) at 23:19:12.9, + the same moment xrservice logged onEnterStandby. + +fapd is now event-driven: Idle (no reads at all; SteamVR event queue +checked once a second, mode file via inotify) -> Checking (after 851, +LeaveStandbyMode, a mode change or start-up; stream timestamps read at +4 Hz for up to 2 s) -> Active (passthrough frames arriving; 4 Hz loop, +grain at 2 Hz) -> Idle again on standby or when frames stop for 1.5 s +(with a grace period after fapd's own switch). OpenVR has no blocking +event wait, so the once-a-second queue check is the floor. + +Measured idle cost (23:21, 20 s window, observe-only): 1 CPU tick +(10 ms, 0.05%), 20 voluntary wakeups (1/s), 1 thread, 16 MB RSS. + +## 28. First live test with switching (2026-09-30 23:26-23:28) + +fapd run manually with switching enabled, user wearing the headset with +passthrough on, room A light via HA: + +- Light off 23:26:25 -> **switched to IR 23:26:29** (4 s). IR grain + smoothed 12.75. +- Light on 23:26:54 -> **switched to colour 23:26:58** (4 s). +- No IMUFallback, no errors; compositor logged its usual + `Late initialization successful` at each switch. +- Controller: holding the left thumbstick cycled auto -> colour -> IR -> + auto as designed (23:27:06, :08, :13, :19). Going back to auto after a + forced IR waited ~6 s for the anti-flicker dwell; fixed (manual switches + no longer start the dwell), with a test. +- **False switch: at 23:27:47, with the light on, fapd switched to IR** + (the user forced colour back at 23:28:11). After restart, colour light + read ~0.63 at full light, versus 0.90-1.0 in the earlier views: g4 + depends on the scene, and 0.6 is too close. g4 is not reliable enough + as the colour-to-IR trigger. +- Going to standby took fapd to idle immediately (23:28:27, 23:28:33). + +## 29. XRService's IR emitter state is a better light signal + +XRService logs `SLAMConsole: [IREmitters] IR Emitters Turned On/Off` +(mode `Auto` at session start). Lined up with every light change so far: + +| test | light | emitters | +| --- | --- | --- | +| run 4 | off 22:40:48 | on 22:40:49.3 | +| run 4 | on 22:41:18 | off 22:41:24.2 | +| run 4 | off 22:41:38 | on 22:41:39.3 | +| run 4 | on 22:41:58 | off 22:42:04.2 | +| sweep, colour | 35% -> 20% at 22:55:07 | on 22:55:07.3 | +| sweep, colour | -> 100% at 22:56:31 | off 22:56:36.3 | +| sweep, mono | 35% -> 20% at 22:57:17 | on 22:57:17.3 | +| sweep, mono | -> 100% at 22:58:43 | off 22:58:48.2 | +| live test | off 23:26:25 | on 23:26:25.4 | +| live test | on 23:26:54 | off 23:26:59.2 | +| false switch | light on 23:27:47 | stayed off | + +- Turns on within ~0-1.3 s of darkness, between 35% and 20% light (the + same step in colour and mono mode), which is where colour passthrough + becomes poor (colour view mean 19 at 35%, 10.7 at 20%). +- Turns off ~5-6 s after full light returns (XRService's own hysteresis). +- Independent of where the user looks (it is XRService's judgement for its + tracking cameras), and it would not have made the 23:27:47 false switch. +- One unexplained on/off at 22:50:36-22:51:01 with the light on, probably + the cameras briefly covered or facing somewhere dark. +- Source is a log line: event-driven via inotify on the log, but the + format could change with a SteamVR update, so fapd must notice when the + signal disappears and fall back. + +## 30. Redesign: fast switching, adaptive cooldown, emitter-led evidence + +User feedback: switching must feel instant, 10 s is too long, and it must +still never flicker; sampling should happen only when needed. + +- LightPolicy now takes Evidence (Dark / Bright / Neutral / Unknown) and + owns only timing: confirm_s 0.5, settle_s 1.5, stale_s 3, and an + **adaptive cooldown** between automatic switches: 2 s, doubling for + each switch within 60 s of the previous one, capped at 30 s, reset + after a quiet minute. Manual switches never start it. Simulated light + flickering every second for 2 minutes: 7 switches (1.5, 4.5, 9.5, 18.5, + 35.5, 66.5, 97.5 s) instead of ~120. +- Sensors (`src/sensors.*`): + - colour shown: Dark when XRService's IR emitters are on AND the colour + light value is below 0.6 (the colour value clears instantly when a + hand leaves the cameras, while the emitters lag ~5 s; the emitters + stop dim-looking views in good light from counting as dark); + - IR shown: Bright when the emitters are off, or when mono grain is at + or below 7 (fast path: grain drops within ~1 s, emitters take ~5 s); + - fallbacks when the emitter line is missing: colour light < 0.5 -> + Dark; grain <= 7 -> Bright. +- End-to-end simulation (measured sensor behaviour): lights off at 10 s -> + IR at 11.5 s; lights on at 40 s -> colour at 41.0 s; 1.5 s hand over the + cameras -> no switch; good light with changing views -> no switch. +- `src/emitter_watch.*` follows `~/.local/share/Steam/logs/xrservice.txt` + (symlink to the session log) with inotify: reads the last emitter line + once, then only appended bytes; follows XRService restarts. Tested + against a fake logs directory, including a split line and a symlink + rotation. +- fapd waits in poll() on the mode-file and emitter-log inotify handles. + Wake intervals: idle 1 s (SteamVR event queue); active with nothing + pending 1 s (liveness); while a decision could be pending 250 ms + (500 ms with grain in IR). The view sampler runs only while IR is shown + and the emitters are still on. +- Mutation checks: all 8 safeguards (colour check with emitters, grain + fast path, unknown-vs-neutral, escalation, cooldown, reset, manual + switches exempt, confirmation) are caught by the tests. + +## 31. Walk-through test (2026-09-30 23:47-23:50): XRService crash, pitch-black false switch + +The user walked through the house with fapd (section 30 build) running. + +- 23:47:51 IR emitters on, colour light 0.01 -> switched to IR (correct). +- 23:48:39 switched to colour on "IR grain 5.3" (correct, lights on). +- 23:49:49 IR emitters on, colour light 0.00 -> switched to IR (correct). +- **23:49:55 switched to colour on "IR grain 2.3" while the user stood in + pitch black.** +- Root cause chain, from vrserver.txt and coredumpctl: + - XRService logged its last line at 23:49:49.269, then fell silent; + - vrserver at 23:49:55.68: "No valid tracker state for 1.0 seconds", + switched to 3DoF; + - XRService **crashed with SIGSEGV** (core at 23:49:54, pid 2336) and + was restarted by vrserver at 23:49:58 (new session log + XRService-23-49-58.log). +- Crash stack (symbols recovered from nearby strings in the headset's own + XRService binary): `WorldQueue::onFramePostTracked` -> + `[SparsePipeline] runNextTask` -> local bundle adjustment -> + `countRedundantAndTotalObservationForFrame` -> crash. That is SLAM + mapping, not passthrough. It happened 5 s after the room went dark and + the emitters came on; the same emitter-on + switch at 23:47:51 and ~12 + earlier switches did not crash. Best reading: a Valve SLAM bug in + near-total darkness, not caused by fapd. Not proven; watch for repeats + (`coredumpctl list`). +- fapd's wrong switch: with XRService stalled or the room pitch black, + the IR view was blank or crushed to black, which has almost no grain + (2.3), and the grain fast path read that as "bright". + +Fixes: +- Grain only counts when the measured band is lit: mean >= 30 and at most + 20% near-black pixels (every IR frame in earlier tests: mean 44-89, + <= 2% near-black). A dark frame also resets the grain smoothing. +- Grain only counts if the mono stream produced a new camera frame since + the previous grain sample (a stalled XRService can leave a blank or + frozen view). +- While SteamVR reports the HMD pose as not valid or not Running_OK, + fapd holds (evidence Unknown) and logs it. +- Tests: pitch-black and blank-image cases, end-to-end "walk into a + pitch-black room" replay (switches once, stays IR), black/grey frame + stats; mutation removing the brightness gate fails 5 checks. + +### Crash guard (added after section 31) + +XRService crash history on this headset (`coredumpctl list`): SIGABRT on +2026-04-13 (before this project) and the SIGSEGV on 2026-09-30 23:49:54. +Rare, and tonight's is the only segfault, so a link to fapd's switch 5 s +earlier cannot be ruled out. + +fapd now records every XRService restart that comes within 10 s of one of +its switches in `~/.config/frame-adaptive-passthrough/crash_guard` +(restart time taken from the new session log's name, so a restart during +an idle period is still timed correctly). With two entries fapd starts in +observe-only mode and says why; `fapctl guard` shows the entries and +`fapctl guard reset` clears them. Tonight's crash was entered by hand as +the first entry. + +## 32. Decision: Phase 2 dropped (2026-09-30) + +The user decided to drop the colourised-IR overlay as not worth it, and +the evidence supports that: passthrough is composited by vrcompositor with +fixed shaders and no blend control (section 5), so colourising would mean +rendering the full passthrough in our own process (stereo, reprojection, +latency, losing SteamVR's depth-mesh warp); IR reflectance does not map to +visible colour and a learned colour map goes stale for anything that +moves; and it would depend on the private frame-import interface every +frame, beside an XRService that crashed once during testing (section 31). +The project continues as Phase 1 only (fapd). The only built-in colour +control in the dark remains `camera.monochromeTintHue`, a single global +tint. + +## 33. Lean rewrite: no SteamVR connection, no controller, systemd autostart + +User direction (2026-10-01): purely automatic, no controller shortcuts, as +lightweight as possible. + +Measured before the rewrite (fapd connected permanently as an overlay): +RSS 16 MB, but mostly shared libraries pulled in by SteamVR's client +library (vrclient.so 4.4 MB, libstdc++, libcrypto, libGL...); proportional +share (Pss) ~3 MB, private dirty 1.8 MB; fapd's own code 0.6 MB; 1 wakeup/s +while idle. + +Facts behind the rewrite: +- XRService writes nothing to its log while the headset is idle (0 lines a + minute over 20 minutes of standby) and ~1-4 lines a second in use, so + following the log with inotify gives zero idle wakeups. +- Its log carries everything fapd needs: `[DeckardCaptureSource] + Passthrough cameras resumed/paused` (every toggle, confirmed by the event + test), `[UserPresence] Received onEnterStandby/onLeaveStandby`, the IR + emitter lines, and `Transition to IMUFallback` / `[IMUFallback] + Disabled` for visual tracking loss. +- A transient background connection (`fapctl get`) takes 10-18 ms and does + not wake the headset. +- SteamVR runs as the systemd user unit `steamvr.service`; Valve's own + `steamvr-v4l2cam.service` is a separate unit with + `After=`/`Requires=steamvr.service`. + +New design: +- fapd no longer links or connects to SteamVR. `src/xrservice_log.*` + follows the session log (replacing the emitter watcher) and fapd sleeps + in poll() on it. Active only while passthrough is shown and not in + standby; while active it wakes only when a decision could be pending + (250 ms), for grain in IR darkness (500 ms), or at 1 s when colour is + shown with the emitters on; otherwise it sleeps until the log changes. +- Tracking loss now comes from the IMUFallback lines (fapd holds). +- Switching runs `fapctl set rgb|mono --quiet` (exit 0 ok, 3 interface + changed, 4 camera not enabled); fapd disables switching on 3. +- Controller action, haptics, mode file and `fapctl mode` removed; + `observe_only` in fapd.conf and `fapctl guard` remain for support. +- Autostart: `fapctl install` writes + `~/.config/systemd/user/frame-adaptive-passthrough.service` + (`After=` and `BindsTo=steamvr.service`, `WantedBy=steamvr.service`, + Nice=10) and enables it; `fapctl uninstall` removes it and switches back + to colour. +- Release builds: -Os, gc-sections, stripped: fapd 342 KB (was 8.5 MB), + fapctl 550 KB. + +### Deployed and installed (2026-10-01 00:08) + +- `fapctl install` wrote the unit and the `steamvr.service.wants/` symlink; + systemd started fapd (Memory 292K, peak 1.7M, CPU 4 ms at start). +- Bug fixed on deploy: the first loop pass slept before evaluating the + initial state, so passthrough already shown at start-up would have gone + unnoticed until XRService's next log line. The first pass now runs at + once. +- Idle measurement, headset in standby (systemd-started fapd, 30 s): **0 CPU + ticks, 0 voluntary wakeups; Pss 534 kB, Private_Dirty 220 kB, RSS 2 MB.** +- Not yet exercised with this build: a live switch, and SteamVR stopping + and starting (the unit uses BindsTo=/WantedBy=steamvr.service). Pending + the user's walk-through. + +## 34. Walk-through 2 (2026-10-01 00:12-00:13): a close wall fools every IR-side signal + +The user deliberately held the headset close to a wall in a dark room: + +| time | fapd | XRService | +| --- | --- | --- | +| 00:12:15 | | IR emitters on | +| 00:12:16 | IR (emitters on, colour 0.00) | | +| 00:12:26 | colour: "IR grain 4.4, image mean 100" | | +| 00:12:31 | IR (cooldown safety net) | | +| 00:12:33-43 | | tracking lost (IMU fallback), too close | +| 00:12:48 | colour: "IR grain 5.5, image mean 106" | | +| 00:13:04 | IR (colour 0.30) | | +| 00:13:33 | | **IR emitters off, room still dark** | +| 00:13:34 | colour: "IR emitters off", **stuck** | | + +- A surface close to the headset, lit by the emitters, gives a bright + (mean 100+), low-grain IR image, and can make XRService itself turn its + emitters off. Every signal from the IR side can be fooled this way; only + the colour camera truly knows whether the room is lit, and it can only + be read while colour is shown. +- It stayed on colour because the colour side required "emitters on" to + call it dark. +- "Emitters on" has meant dark in every test (user's point); "emitters + off" is the unreliable direction. + +Changes (sensors.hpp): +- Grain removed entirely: while the emitters are on it is dark, whatever + the IR image looks like. fapd no longer reads /dev/video99. Cost: IR to + colour waits for XRService to turn the emitters off (~5 s after the + light returns), so ~6 s instead of ~1-2 s. +- Colour shown, Dark also when g4 < 0.35 for 1.5 s regardless of the + emitters. +- Verification: for 3 s after an automatic switch to colour, g4 < 0.35 is + urgent Dark (confirm 0.25 s, skips the cooldown, still escalates it) and + "emitters off" is distrusted until XRService turns the emitters on again. +- Settle after a switch to colour 0.3 s (colour's value is valid on its + first frame); confirm 1 s for normal switches (a 1.5 s hand over the + cameras was otherwise enough to switch once the dark colour value reads + 0.0, as it does in a truly dark room). +- Without the emitter signal nothing switches either way (no reliable + way back from IR). + +End-to-end replays: lights off -> IR at 2.0 s, on -> colour at 6.0 s; the +wall case -> one 0.75 s colour flash, then IR stays, also after leaving +the wall; emitters never on in the dark -> the same; hand for 1.5 s, view +changes in good light and 45% dimming -> no switch. Eight mutations of the +new safeguards are all caught. Also removed: the two room thumbnails in +tests/fixtures. + +Deployed 00:22 (fapd 338 KB; Memory 320K per systemd; Pss 558 kB). + +## 35. Walk-through 3 (2026-10-01 00:28-00:30): emitters stay on in bright rooms + +Deployed build: emitter-only return to colour (section 34). + +- 00:28:57 XRService turned its IR emitters on; fapd switched to IR at + 00:28:58 ("colour light 0.04"), correct. +- The user then walked into bright rooms. **The emitters stayed on until + standby at 00:30:06** (70 s). fapd never returned to colour. +- The wall test caused no problem (the verification design held). +- So "emitters off" is not a dependable "light is back" signal either: in + the room A the emitters went off ~5 s after the LIFX came to 100% every + time, but ordinary room lighting did not make XRService turn them off. + +Change: the grain path is back for IR to colour, with everything learned +since. Bright when grain <= 7 in a normally lit view (30 <= mean <= 90, at +most 20% near-black) held for 2 s, from a new camera frame; a colour check +after every automatic switch to colour; a grain-caused mistake locks grain +out for 60 s, doubling to 10 min. Emitters off still counts (fastest when +it happens). fapd samples the view (2 Hz) only while IR is shown and the +emitters are on (or "off" is distrusted), and logs "IR view: grain, mean, +near-black, emitters" every 10 s, so the next walk-through measures the +bright-room-emitters-on case directly (no data for it yet; thresholds come +from the room A sweeps). + +End-to-end replays: off/on -> IR at 2.0 s, colour at 3.5 s; bright room +with emitters staying on -> colour at 3.5 s; dark with emitters on for +3 min -> no return; close wall -> one 0.75 s flash; medium-distance wall +for 3 min -> two 0.75 s flashes a minute apart. Six mutations of the grain +safeguards all caught. Deployed 00:34 (fapd 345 KB, 316K memory). + +## 36. Faster return to colour (not yet deployed) + +User: colour came back after walk-through 3's fix, "but took a bit". +Return latency was grain hold 2 s + confirmation 1 s (~3.5 s). Since every +automatic switch to colour is verified by the colour camera and undone in +under a second, that direction can be quick: grain_hold_s 0.5 and a +separate confirm_to_colour_s 0.25 (confirm_s stays 1 s towards IR, which +keeps a hand over the cameras from switching). Replays: lights on -> +colour 1.5 s after the light; close and medium walls unchanged (single +sub-second flashes). Built; deploy pending, headset switched off for the +night. The fapd.log "IR view" lines from that walk-through were not read +(headset off). + +### Deployed 2026-10-01 13:28 + +The faster-return build is installed (Pss 516 kB). Last night's single +logged sample before the slow return (00:35:25: grain 5, mean 42, emitters +on) was within the "lit" rule, yet colour only returned at 00:35:33 via +"emitters off"; with one sample per 10 s logged, the most likely reading +is grain hovering around 7 and resetting the 2 s hold (now 0.5 s). Each +return to colour now logs the view samples of the previous 10 s so the +next slow return can be diagnosed from the log. + +## 37. Walk-through 4 (2026-10-01 17:07): faster return works + +User: "it all worked with no issues". Two dark/lit cycles: + +| Time | Event | +| --- | --- | +| 17:07:45.6 | XRService: IR emitters on | +| 17:07:46 | fapd to IR (emitters on, colour 0.02) | +| 17:07:58 | fapd to colour by grain (6.6, mean 58); emitters stay on | +| 17:08:07 | fapd to IR (emitters on, colour 0.06) | +| 17:08:22 | fapd to colour by grain (3.6, mean 68) | +| 17:08:24.4 | XRService: IR emitters off | + +The "IR view before the switch" samples show the light coming on cleanly: +grain 13-21 with mean 14-53 in the dark, dropping to 5-8 with mean 58-93 +within half a second of the light (first cycle: 1.8 s before the switch; +second: about 1.5 s before). Notably the emitters never turned off during +the first lit period (about 9 s) and turned off 2.4 s after fapd's second +return, about 4 s after the light. So the grain path, not the emitters, is +what brings colour back, and ir_max_grain 7 separates these rooms well +(lit 3-8, dark 9-22 with the gates). + +Emitter timing logging added (deployed 17:11): on each emitter change fapd +logs how long after the cameras first saw the room go dark (colour light +below colour_dark_light) or lit (colour light at or above colour_min_light, +or a usable IR view at or below ir_max_grain) it happened. Resolution is +the sampling rate: 1 s for the colour camera, 0.5 s for the IR view. + +## 38. What turns XRService's IR emitters on and off (disassembly) + +Read offline from XRService (`/opt/steamvr/drivers/cv/bin/linuxarm64/XRService`, +SteamVR 2.17.10) with capstone; nothing run on the headset. Addresses are +file virtual addresses in that build. + +Auto mode is part of the tracking cameras' auto-exposure controller +(`PE::ActiveExposureController`), not a separate light sensor: + +- Input: the median of a 256-bin luma histogram per tracking camera + (ISP histogram when available). Ratio r = targetMedian / median + (deadband 0.95..1.05). Desired gain Gd = r * current gain. Of the first + two tracking cameras, the one needing the least correction wins + (`0xda0480`). Analysis runs at most every 33 ms. +- State machine on exposure (`0xd9d2b0`): below 0.5 ms, up to 8.33 ms, + 8.33 ms, 16.66 ms (state 4, the longest exposure). +- On (`0xd9f498`): in state 4 with the emitters off and Gd > 1.2 * mid + gain, IR pulse 0.5 ms, switched on at once. From a lit room this takes + about three analyses (~100 ms): why "emitters on" is fast and reliable. +- While on (`0xd9fc80`), the pulse width (0.02..4 ms) follows the gain + demand: the emitters dim themselves as the scene gets brighter. +- Off (`0xd9fcb8`): only after 5.0 s continuously in a low state (exposure + below 8.33 ms, or 8.33 ms with Gd < 0.33 * (min + max gain)). Any return + to state 4 or high gain resets the timer. +- Logged On/Off is the pulse ramp (`0xd9d7f0`) crossing zero; it also + writes `led_state` 1/0 in the `scene_illum_ir_led` sysfs device. + +Inferences that match the observations: +- Slow or no "off" in lit rooms (sections 35, 37): the emitters light the + scene themselves, and in a dim-but-lit room the gain rarely stays low for + 5 s without interruption, so the timer keeps resetting. +- "Off" in the dark next to a close wall (section 33): the emitter-lit wall + drives the median up, the state drops for 5 s, the emitters go off, and + about 100 ms later darkness turns them back on. + +Usable by fapd: only the on/off result is visible outside the process (the +log line we already follow, and `led_state`). Exposure, gain, the median +and the AE state are not published anywhere passive. The per-frame IR +pulse width is stored in each tracking frame's metadata (`0xf67b0c`, NaN +when off); not traced to shared memory (`VR_CameraPassthroughState` is not +created by XRService). If it did reach a passive reader, a pulse shrinking +towards 0.02 ms would be an early "the room is lit" signal, seconds before +the 5 s off timer. + +Tunable only through `captureSessionSettings.trackingCameras.targetMedian` +(default 60) and `maxIrEmitterPulseWidth`, which would change tracking +exposure itself: not something to touch. The thresholds are hard-coded. + +Conclusion: "emitters on" stays the trusted dark signal; "emitters off" is +late by design (5 s minimum) and can be prevented by the emitters' own +light, so the IR image check remains the main way back to colour. + +## 39. Dusk false switch; the emitters become the judge of "dark" + +2026-10-01 18:41-18:45, headset on the desk (face sensor covered), no +lamp (the LIFX was offline), daylight fading. The user: the room "never +went dark", yet fapd switched to IR and stayed there. + +| Time | fapd | +| --- | --- | +| 18:41:58 | IR: colour light 0.14 for 1.5 s (emitters off) | +| 18:42:00 | colour: emitters off | +| 18:43:33 | IR: colour light 0.01 | +| 18:43:35 | colour: emitters off | +| 18:43:36 | IR: verify failed (0.01), "emitters off" distrusted | +| 18:43:55 | colour: IR grain 5.0, mean 34 | +| 18:43:56 | IR: verify failed (0.15), grain locked out 60 s | +| then | IR view grain 20, mean 48, emitters off: stuck in IR | + +The colour camera reads 0.01-0.15 at dusk, as low as a dark room (the +00:13 wall case even read 0.30), while XRService's tracking cameras need +no emitters. With section 38's mechanism in mind (emitters on only at the +longest exposure with gain to spare), "emitters off" means the room has +usable light, and the IR view then has no advantage. The IR view without +emitters was also too noisy for the grain check, and the distrust only +ended when the emitters came on, so nothing could bring colour back. + +Changes (sensors.hpp; user agreed): +- The "colour dark for 1.5 s whatever the emitters say" rule is gone. + Colour to IR needs the emitters on (and colour below 0.6). +- Verification after an automatic switch to colour only fails if the + emitters are on and colour is below 0.35. +- Distrust of "emitters off" now expires: 60 s, doubling per repeat up to + 600 s (`emitters_off_distrust_s`, `_max_s`), like the grain lockout. + It used to last until the emitters came back on. +- `colour_dark_hold_s` removed. +- Accepted cost: the deliberate face-at-a-wall case in the dark (emitters + fooled off) shows dark colour until the emitters come back on after + leaving the wall (replay: 19 s at the wall, then IR). + +Replays: dusk -> no switch; dark then dusk -> IR, then colour once the +emitters go off, and it stays; the earlier scenarios unchanged. Mutants +(no emitter gate, no distrust, no doubling) caught. Deployed 18:51. + +First emitter timing line: 18:51:07 "IR emitters turned on 1.2 s after +the cameras first saw the room dark" (colour sampled at 1 Hz). + +## 40. Cooldown only escalates for real flicker + +Walk-through 18:55-18:57 (the dusk build): every switch was right, but +each took longer. The user toggled the lights by hand every 4-15 s, and +the old rule doubled the cooldown for any switch within 60 s of the +previous one: 18:56:06 emitters on -> IR only at 18:56:19 (cooldown 16 s), +18:56:39 emitters off -> colour at 18:56:49. The user: the cooldown is +only meant to stop strobing or flashing lights. + +New rule (light_policy): a switch is flicker only if it comes within +`flicker_slack_s` (1.5 s) of the moment the cooldown allowed it; that +doubles the cooldown (cap 30 s). A slower switch steps it down one level, +and `cooldown_reset_s` (now 30 s) of quiet beyond the cooldown clears it. +Urgent reverts (a failed colour check) still escalate, as they come well +inside the cooldown. Test: lights toggled every 5 s for a minute -> all +12 switches within 1.25 s of the change, cooldown still 2 s; the 1 s +flicker test still backs off to 30 s gaps. The old rule fails the new +test. Deployed 18:59. + +Emitter timing from the same walk-through (fapd's new log lines): on 0.9 +to 1.2 s after the colour camera first read dark (1 s sampling, so +effectively immediate); off 0.4 to 0.6 s after the IR view first looked +lit (0.5 s sampling), much faster than the 5 s minimum suggests: the +5 s timer had evidently been running before the view crossed fapd's +grain threshold. + +## 41. Unattended light tests (room B switch); the IR camera's light value + +2026-10-01 21:43-21:58. Headset on a desk in the room B, face sensor +covered, passthrough shown; the room B ceiling light (Aqara H2 wall +switch, on/off only) toggled from Home Assistant (`hass.callService`). Light +times below are HA's `last_changed`. A passive recorder +(`tools/shm_records.py`) logged every new per-frame record of both +passthrough streams. + +**Tracking drops in the dark.** 0.1-0.6 s after the light went out, +XRService entered IMU fallback (tracking lost), and with the headset +still it often stayed lost for the whole dark period and beyond (once from +21:51:28 until 21:57:45, through several lit periods). fapd held all +switching while tracking was lost, so it stayed on dark colour (21:43:51). +After allowing switches to IR it got stuck in IR with the light on instead +(21:51:43 onwards), since only the grain check could bring colour back and +the IR view is frozen while tracking is lost (identical grain/mean for +10 s in the "before the switch" lists). + +**The IR camera's light value.** Each mono-stream record carries a float at ++0xEC, the same record offset as the colour stream's g4 (gamma marker ++0x14; mono records have 0.5 there instead of the 1/2.2 marker). It is: +- exactly 0.000 in the dark with the emitters on, for 70 s straight; +- 0.13-0.34 within 0.19-0.8 s of the light coming on (steps of 1/96), + settling to ~0.22-0.30 as the emitters dim; +- unaffected by tracking loss (rose 0.45 s after the light while tracking + stayed lost for another 3 s); +- only updated while IR is shown (like everything in the mono stream). +Not yet measured: a wall lit by the emitters from close by, and other +rooms. + +Emitter timing (8 cycles): on 0.12-0.26 s after the light goes off; off +4.7-5.8 s after it comes on, every time (the 5 s timer of section 38). + +Changes: +- `PassthroughSnapshot::mono_light`: the newest mono record's +0xEC. +- New IR-to-colour rule: IR light >= `ir_min_light` (0.1) for + `ir_light_hold_s` (0.25 s), fresh frames only. A failed colour check + after it locks it out together with grain (60 s, doubling). +- Tracking loss now only blocks the grain rule; going to IR, the IR light + value and "emitters off" still work. +- The "IR view" log line includes the IR light value. + +Results with the new build (21:55-21:58, 7 cycles, including 8 s on/off +toggling, tracking lost for most of it): every light-on returned to colour +within ~1 s ("IR camera light 0.30-0.33"), every light-off went to IR in +1.7-2.4 s. No wrong switches. + +## 42. User walk-through: wall test with the IR light value + +2026-10-01 22:03-22:04, user wearing the headset. +- 22:03:25 vrserver's XRServiceManager killed XRService "because it used + too much memory" (SIGTERM, journal); SteamVR restarted it at 22:03:27. + Not fapd-related (fapd's last switch was 344 s earlier; nothing of ours + runs inside XRService); possibly map growth after many tracking losses + in the unattended tests. The crash guard correctly did not count it. +- Wall test: at an IR-lit wall in the dark the IR light value read 0.10 + (exactly the threshold) for 0.25 s -> colour at 22:04:09, colour 0.24 + -> urgent revert to IR at 22:04:10, IR light locked out for 60 s. The + user saw one brief black blink, then IR stayed. +- 22:04:33 emitters went off (18 s after the first lit reading); colour + at 22:04:34, not reverted (the user was presumably away from the wall). + +Change: `ir_min_light` 0.10 -> 0.15. Lit rooms read 0.22-0.34 (first +frame after the light 0.13-0.19, so at most one extra frame of delay); the +wall read 0.10. Deployed 22:06. + +## 43. The image check is gone + +User question: is the image (grain) check still needed? No. Since section +41 every return to colour came from the IR light value (~1 s after the +light), never from grain, and grain was the heaviest and most fragile part +of fapd: a 2 Hz V4L2 capture of /dev/video99, frozen while tracking is lost, +fooled by surfaces lit by the emitters. + +Removed: src/view_grain.*, its tests, the grain rules and config keys +(`ir_max_grain`, `ir_min_mean`, `ir_max_mean`, `ir_max_dark_fraction`, +`grain_hold_s`; `grain_lockout_s`/`_max_s` became `ir_light_lockout_s`/ +`_max_s`). The tracking-loss special case went with it: no remaining signal +depends on tracking. fapd now only reads shared memory and XRService's log; +in IR it polls the IR light value at 2 Hz (a 25 KB tmpfs read; SteamVR +writes the segment without any notification, so it cannot be event-driven) +and logs it every 10 s; returns to colour log the last 10 s of readings. +The diagnostic probes that opened camera devices during discovery are not part of the published repository. + +Known gap: a lit room whose IR light value stays below 0.15 (a dim lamp, +not measured) returns to colour only when the emitters go off, ~5 s after +the light instead of ~1 s. + +Measured after deploying (22:14): fapd 342 KB, Pss 490 kB, no video device +open. + +## 44. Walk-through: wall at touching distance (corrected with HA history) + +2026-10-01 22:18-22:19, user wearing the headset in the room B, face at +and briefly touching a wall; the user then turned the ceiling light on and +off. Home Assistant history (`/api/history/period`) gives the light times, +local: + +| Time | Event | +| --- | --- | +| 22:18:28-29 | emitters on, fapd to IR (colour 0.01) | +| 22:18:56.1 | room B light on | +| 22:18:57-22:19:01 | IR light value 0.03 then 0.00 (facing the wall) | +| 22:19:02 | emitters off (5.9 s after the light); fapd to colour: correct | +| 22:19:04.9 | room B light off | +| 22:19:05-06 | emitters on; fapd to IR (colour 0.09): correct | +| 22:19:16 | fapd to colour on IR light 0.23-0.25; the room B light was still off, so another light source (probably the user leaving the room) | + +I first read 22:19:02 as the wall switching the emitters off in the dark, +added a 5 s hold on "emitters off" (19a5176) and deployed it. The HA +timeline refutes that, so it was reverted (deployed again 22:23): the hold +fixed nothing observed and would add 5 s to the return whenever the IR light +value cannot see the room light. + +New fact: **facing a wall from centimetres away, the IR light value reads +about 0 even with the room light on** (0.00-0.03 at 22:18:57-22:19:01, while +lit rooms read 0.22-0.34). The emitters lighting the wall dominate what the +IR camera sees. In that case only "emitters off" (~5 s after the light) +brings colour back; this is why "emitters off" must stay a prompt way back. + +Lesson: check the light's actual timeline (HA history) before explaining a +switch; an explanation from fapd's log alone was wrong here. + +## 45. Faster switching, measured to the frame + +2026-10-01 22:27-22:42, headset on a desk in the room B, ceiling light +toggled through Home Assistant; switch times taken from the passthrough +config byte in shared memory (`tools/shm_records.py` at 100 Hz), light times +from HA history, emitter times from XRService's log. Seconds after the +light change: + +| | emitters | signal | fapd switched | +| --- | --- | --- | --- | +| off, old build (4 cycles) | on 0.18-0.21 | colour g4 < 0.6 at 0.69-0.71 | 2.22-2.27 | +| on, old build (4) | off 5.27-5.31 | IR light >= 0.15 at 0.35-0.40 | 1.15-1.59 | +| off, new build (7) | on 0.19-0.25 | colour g4 < 0.6 at 0.64-0.73 | 1.33-1.63 | +| on, new build (7) | off 5.22-5.47 | IR light >= 0.15 at 0.31-0.56 | 0.67-0.94 | + +Where the time went: the colour camera's g4 is already smoothed by the +camera (0.89 to 0.44 over ~1.5 s after the light goes out), and fapd's own +0.5 s EMA on top roughly doubled the lag; then 1 s of confirmation. The IR +light value steps 0 -> 0.33 within ~0.1 s, but was polled at 2 Hz and held +0.25 s on top of the 0.25 s confirmation. + +Changes: `smoothing_s` 0.5 -> 0 (no extra smoothing), `confirm_s` 1.0 -> +0.5 (switching to IR also needs the emitters on, which a hand over the +colour camera does not cause), `ir_light_hold_s` 0.25 -> 0, IR light polled +at 4 Hz while it matters. All 14 switches correct, no flicker. + +What is left before IR: the colour value needs ~0.7 s to fall below 0.6 +(the camera's own smoothing), plus 0.5 s confirmation. Emitters are on at +~0.2 s, but they can also turn on in a lit room (a hand or a dark spot, +2026-10-01 18:11:09), so they are not enough on their own. + +Testing note: the Home Assistant page suspends its websocket while its tab +is hidden (`callService` rejects with code 3); the REST API from the same +page session (`fetch('/api/services/light/turn_on')` with the session's +access token) works regardless. + +## 46. IR light value at a wall in the dark; threshold stays 0.15 + +2026-10-01 22:46 manual test: one slow return to colour (22:46:31, via +"emitters off"): the IR light value read only 0.12 in that lit spot, under +the 0.15 threshold. Other returns read 0.17-0.19 (dimmer than the room B +ceiling light's 0.3). + +22:49:30-22:50:08, user staring at a wall with the IR emitters on in the +dark (room B light off per HA), sampled every frame (20 Hz): almost all +0.00, peaks 0.07 and 0.13 (22:49:49), plus 0.19 on the first frames right +after the switch to IR (covered by settle_s). 22:50:00 (0.30) was the user +walking back into the lit room A. + +So a wall in the dark reaches 0.13 and a dim lit room can read 0.12: the +value alone cannot separate them in that band. Decision (user): keep 0.15; +dim rooms fall back to "emitters off" (~5 s), which is acceptable since +colour is the direction you can still see in. diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..afc642c --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 bod09 + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..a005b96 --- /dev/null +++ b/Makefile @@ -0,0 +1,101 @@ +# Host targets (tests) build with the system compiler. Device targets are +# cross-compiled for the headset (aarch64, glibc) with the Zig toolchain +# fetched by scripts/fetch-deps.sh. + +CXX ?= c++ +CXXFLAGS ?= -std=c++17 -O2 -Wall -Wextra -Wpedantic -Werror +BUILD := build/host + +ZIG := toolchain/zig-x86_64-linux-0.16.0/zig +# The headset runs glibc 2.39; target a little lower so an OS update that +# holds glibc back does not strand the binaries. +TARGET := aarch64-linux-gnu.2.35 +DEV := build/aarch64 +DEVCXX := $(ZIG) c++ -target $(TARGET) +DEVFLAGS := -std=c++17 -O2 -g -Wall -Wextra + +OPENVR := third_party/openvr +OPENVR_SRC := $(addprefix $(OPENVR)/src/,openvr_api_public.cpp jsoncpp.cpp \ + vrcore/dirtools_public.cpp vrcore/envvartools_public.cpp vrcore/pathtools_public.cpp \ + vrcore/sharedlibtools_public.cpp vrcore/hmderrors_public.cpp \ + vrcore/vrpathregistry_public.cpp vrcore/strtools_public.cpp) +OPENVR_OBJ := $(patsubst $(OPENVR)/src/%.cpp,$(DEV)/openvr/%.o,$(OPENVR_SRC)) +OPENVR_DEFS := -DVR_API_PUBLIC -DOPENVR_BUILD_STATIC -DLINUX -DPOSIX -DLINUXARM64 -DVRCORE_NO_PLATFORM + +.PHONY: dist test device deploy clean + +HOST_TESTS := test_light_policy test_sensors test_xrservice_log test_daemon_config test_passthrough_state + +test: $(addprefix $(BUILD)/,$(HOST_TESTS)) + @for t in $(HOST_TESTS); do $(BUILD)/$$t || exit 1; done + +$(BUILD)/test_light_policy: tests/test_light_policy.cpp src/light_policy.cpp src/light_policy.hpp + @mkdir -p $(BUILD) + $(CXX) $(CXXFLAGS) -Isrc -o $@ tests/test_light_policy.cpp src/light_policy.cpp + +$(BUILD)/test_sensors: tests/test_sensors.cpp src/sensors.cpp src/sensors.hpp src/xrservice_log.cpp src/xrservice_log.hpp src/light_policy.cpp src/light_policy.hpp + @mkdir -p $(BUILD) + $(CXX) $(CXXFLAGS) -Isrc -o $@ tests/test_sensors.cpp src/sensors.cpp src/xrservice_log.cpp src/light_policy.cpp + +$(BUILD)/test_xrservice_log: tests/test_xrservice_log.cpp src/xrservice_log.cpp src/xrservice_log.hpp + @mkdir -p $(BUILD) + $(CXX) $(CXXFLAGS) -Isrc -o $@ tests/test_xrservice_log.cpp src/xrservice_log.cpp + +$(BUILD)/test_daemon_config: tests/test_daemon_config.cpp src/daemon_config.cpp src/daemon_config.hpp src/light_policy.cpp src/sensors.hpp + @mkdir -p $(BUILD) + $(CXX) $(CXXFLAGS) -Isrc -o $@ tests/test_daemon_config.cpp src/daemon_config.cpp src/light_policy.cpp + +$(BUILD)/test_passthrough_state: tests/test_passthrough_state.cpp src/passthrough_state.cpp src/passthrough_state.hpp + @mkdir -p $(BUILD) + $(CXX) $(CXXFLAGS) -Isrc -o $@ tests/test_passthrough_state.cpp src/passthrough_state.cpp + +# Device builds. autopassd does not link SteamVR at all; only autopass does (for +# the brief background connection that performs a switch). +device: $(DEV)/autopassd $(DEV)/autopass + +$(ZIG) $(OPENVR)/headers/openvr.h: + scripts/fetch-deps.sh + +$(DEV)/openvr/%.o: $(OPENVR)/src/%.cpp | $(ZIG) + @mkdir -p $(dir $@) + $(DEVCXX) -std=c++17 -O2 -w -Wno-nullability-completeness $(OPENVR_DEFS) -I$(OPENVR)/headers -I$(OPENVR)/src -I$(OPENVR)/src/vrcore -c -o $@ $< + +$(DEV)/libopenvr_loader.a: $(OPENVR_OBJ) + $(ZIG) ar rcs $@ $^ + +DAEMON_SRC := src/autopassd.cpp src/daemon_config.cpp src/app_paths.cpp src/passthrough_state.cpp \ + src/light_policy.cpp src/sensors.cpp src/xrservice_log.cpp +CTL_SRC := tools/autopass.cpp src/app_paths.cpp src/passthrough_state.cpp src/private_camera.cpp +VERSION ?= $(shell git describe --tags --always --dirty 2>/dev/null || echo dev) +RELEASE_FLAGS := -DAUTOPASS_VERSION='"$(VERSION)"' -std=c++17 -Os -Wall -Wextra -Wno-nullability-completeness -ffunction-sections -fdata-sections -Wl,--gc-sections -s + +$(DEV)/autopassd: $(DAEMON_SRC) $(wildcard src/*.hpp) | $(ZIG) + @mkdir -p $(DEV) + $(DEVCXX) $(RELEASE_FLAGS) -Isrc -o $@ $(DAEMON_SRC) + +$(DEV)/autopass: $(CTL_SRC) $(wildcard src/*.hpp) $(DEV)/libopenvr_loader.a + $(DEVCXX) $(RELEASE_FLAGS) -DOPENVR_BUILD_STATIC -Isrc -isystem $(OPENVR)/headers -o $@ $(CTL_SRC) $(DEV)/libopenvr_loader.a -ldl + +# Copies autopassd and autopass to the headset's install directory, replacing +# running binaries safely (rename). Does not install or start the unit: +# run `autopass install` on the headset for that. +deploy: $(DEV)/autopassd $(DEV)/autopass + ssh frame 'mkdir -p ~/.local/share/frame-autopass' + scp -q $(DEV)/autopassd frame:.local/share/frame-autopass/autopassd.new + scp -q $(DEV)/autopass frame:.local/share/frame-autopass/autopass.new + ssh frame 'cd ~/.local/share/frame-autopass && mv autopassd.new autopassd && mv autopass.new autopass' + +# Release package: dist/frame-autopass-aarch64.tar.gz (+ .sha256) and +# dist/install.sh. Always a clean build, so VERSION is baked in. +dist: + rm -rf dist $(DEV)/autopassd $(DEV)/autopass + $(MAKE) $(DEV)/autopassd $(DEV)/autopass + mkdir -p dist/frame-autopass + cp $(DEV)/autopassd $(DEV)/autopass install.sh README.md LICENSE dist/frame-autopass/ + tar -C dist -czf dist/frame-autopass-aarch64.tar.gz frame-autopass + cd dist && sha256sum frame-autopass-aarch64.tar.gz > frame-autopass-aarch64.tar.gz.sha256 + cp install.sh dist/install.sh + rm -rf dist/frame-autopass + +clean: + rm -rf build dist diff --git a/README.md b/README.md new file mode 100644 index 0000000..6c8e7e2 --- /dev/null +++ b/README.md @@ -0,0 +1,156 @@ +# frame-autopass + +> **AI disclaimer:** built with the help of AI and tested on a single +> headset. Treat it as experimental. + +Automatic passthrough switching for the **Steam Frame with the Arcturus +Vision colour module**: colour passthrough in good light, the Frame's own +infrared (IR) passthrough in the dark. Turn the lights off and you can see +in IR about 1.5 s later; turn them on and colour is back in about a second. +There is nothing to press and nothing to configure. + +It runs entirely on the headset, never talks to the network, and does +nothing at all while passthrough is not showing. + +## Requirements + +- A Steam Frame with the Arcturus Vision colour passthrough module attached. +- Tested on SteamOS 0.3.0 with SteamVR 2.17.10. Other versions may work; if + SteamVR changes the interface it uses, it stops switching safely (see + [After a SteamVR update](#after-a-steamvr-update)). +- Desktop Mode with a terminal (Konsole). Developer mode was enabled on the + test headset; it may not be required. + +## Install + +On the headset, switch to Desktop Mode, open **Konsole** and paste: + +```sh +curl -fsSL https://github.com/bod09/frame-autopass/releases/latest/download/install.sh | bash +``` + +That's it. It starts automatically whenever SteamVR runs, from now on. + +The installer checks that the Arcturus module is present, downloads the +latest release, verifies its checksum, puts two programs in +`~/.local/share/frame-autopass` and registers a systemd user service that +starts and stops with SteamVR. Everything stays in your home folder, so it +survives SteamOS updates and needs no root access. + +Prefer not to pipe a script into bash? Download +`frame-autopass-aarch64.tar.gz` from the +[releases page](https://github.com/bod09/frame-autopass/releases), unpack +it, read `install.sh`, and run `./install.sh` from that folder. + +## Use + +Just use passthrough as normal. The commands below are optional (run them +in Konsole): + +```sh +~/.local/share/frame-autopass/autopass status # running? what is it showing, and why +~/.local/share/frame-autopass/autopass update # install the latest release +~/.local/share/frame-autopass/autopass report # write ~/frame-autopass-report.txt for a bug report +~/.local/share/frame-autopass/autopass uninstall # remove the service, back to plain colour passthrough +``` + +`uninstall` stops the service and removes it; delete +`~/.local/share/frame-autopass` and `~/.config/frame-autopass` as well to +remove every trace. + +## How it decides + +It reads three signals, none of which needs an image or a camera device: + +- **XRService's IR emitters.** Valve's tracking service turns the IR + emitters on when its tracking cameras run out of light (about 0.2 s after + the room goes dark) and off once there is light again (at least 5 s + later). It says so in its log, which this follows. +- **The colour camera's light value**, published by SteamVR for every + colour frame. +- **The IR camera's light value**, published for every IR frame: 0 in the + dark even with the emitters on, about 0.2 to 0.35 in a lit room. + +From these: + +- **To IR:** the emitters are on and the colour camera reads dim. The + emitters decide what "dark" is, so a dim room at dusk stays in colour. +- **To colour:** the IR camera sees light, or the emitters have turned off. +- **Mistakes are undone fast:** for 3 s after switching to colour, if the + emitters are on and the colour camera sees darkness, it goes straight back + to IR and ignores whatever misled it for a minute (doubling if it repeats). + At worst you see one sub-second blink. +- **No flicker:** after a switch it waits at least 2 s before switching + back; that wait only grows (up to 30 s) if the light keeps changing back + the moment a switch is allowed, as a flashing light would. +- Losing tracking (which happens in the dark if you stand still) does not + stop it working. + +Known limits: in a dimly lit room, or with your face right up against a +wall, the IR camera may not see enough light, so colour comes back when the +emitters turn off, about 5 s after the light instead of 1 s. It never +leaves you in the dark for that: the slow direction is always back to +colour. + +The research behind every rule, with measurements, is in +[FINDINGS.md](FINDINGS.md). + +## After a SteamVR update + +There is no public way to switch the passthrough camera, so frame-autopass +uses a private SteamVR interface. Before every switch it checks that +SteamVR's code still matches what it was built against. If an update +changes it, frame-autopass stops switching (passthrough keeps working +normally, just without automatic switching) and `autopass status` says so. +Run `autopass update`; if no fixed release exists yet, please open an issue +with the output of `autopass report`. + +## Safety + +- It never opens a camera, never writes outside your home folder, and does + not connect to the internet (except `update`, when you run it). +- It only reads SteamVR's shared memory and logs. To switch, it connects to + SteamVR for about 15 ms as a background client, which never wakes the + headset or starts SteamVR. +- Crash guard: if XRService (which also runs tracking) restarts within 10 s + of a switch twice, switching pauses until you run `autopass guard reset`. + This exists because XRService once crashed during early testing; the + crash was most likely unrelated, but switching should not continue if it + ever repeats. +- Measured cost on the headset: about 0.5 MB of memory, no CPU or wake-ups + while passthrough is hidden. + +## Tuning + +Optional: put `key = value` lines in `~/.config/frame-autopass/autopass.conf` +and restart SteamVR. The keys and defaults are listed at the top of +[src/daemon_config.hpp](src/daemon_config.hpp). Unknown keys are rejected +and logged, so a typo never silently does nothing. + +## Building from source + +On an x86_64 Linux PC: + +```sh +scripts/fetch-deps.sh # Zig toolchain and OpenVR SDK headers, into gitignored folders +make test # host tests +make dist # cross-compiled release package in dist/ +``` + +`make deploy` copies a build to a headset reachable as `ssh frame`. +GitHub Actions builds and tests every push and publishes a release for every +`v*` tag. + +`tools/` holds the research tools used to find all of this: a passive +shared-memory recorder and sampler, and an annotated ARM64 disassembler for +SteamVR's binaries. + +## Credits + +The existence and name of SteamVR's private passthrough camera interface +were first learned from +[KominoVR/frame-passthrough-shortcuts](https://github.com/KominoVR/frame-passthrough-shortcuts), +which toggles the camera with controller gestures. No code from it is used; +everything here was re-derived from the headset's own binaries. + +MIT licence, see [LICENSE](LICENSE). diff --git a/install.sh b/install.sh new file mode 100755 index 0000000..01dace4 --- /dev/null +++ b/install.sh @@ -0,0 +1,65 @@ +#!/bin/bash +# frame-autopass installer and updater for the Steam Frame. +# +# curl -fsSL https://github.com/bod09/frame-autopass/releases/latest/download/install.sh | bash +# +# or, from an unpacked release folder: ./install.sh +# +# Installs two programs into ~/.local/share/frame-autopass and a systemd +# user unit that starts them with SteamVR. Nothing outside your home +# directory is touched, so it survives SteamOS updates. Running it again +# updates to the latest release. Remove with `autopass uninstall`. +set -euo pipefail + +REPO="bod09/frame-autopass" +DEST="${XDG_DATA_HOME:-$HOME/.local/share}/frame-autopass" +TARBALL="frame-autopass-aarch64.tar.gz" + +say() { printf '%s\n' "$*"; } +fail() { printf 'frame-autopass: %s\n' "$*" >&2; exit 1; } + +[ "$(uname -m)" = "aarch64" ] || fail "this is for the Steam Frame (aarch64); this machine is $(uname -m)." +command -v systemctl >/dev/null || fail "systemd not found." + +if ! grep -qs arcimx616 /sys/class/video4linux/*/name; then + fail "the Arcturus Vision colour module was not found. frame-autopass switches between that module +and the built-in IR cameras, so it needs the module attached." +fi + +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT + +# Use the files next to this script when run from an unpacked release, +# otherwise download the latest release and check its checksum. +here="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" 2>/dev/null && pwd || true)" +if [ -n "$here" ] && [ -x "$here/autopassd" ] && [ -x "$here/autopass" ]; then + src="$here" +else + base="https://github.com/$REPO/releases/latest/download" + say "Downloading the latest frame-autopass release..." + curl -fsSL -o "$work/$TARBALL" "$base/$TARBALL" || fail "download failed ($base/$TARBALL)." + curl -fsSL -o "$work/$TARBALL.sha256" "$base/$TARBALL.sha256" || fail "checksum download failed." + (cd "$work" && sha256sum -c --quiet "$TARBALL.sha256") || fail "checksum mismatch; not installing." + tar -C "$work" -xzf "$work/$TARBALL" + src="$work/frame-autopass" +fi + +new_version="$("$src/autopass" version 2>/dev/null || echo unknown)" +old_version="$("$DEST/autopass" version 2>/dev/null || echo none)" + +# Replace the programs while the service is stopped, then (re)install the +# unit, which starts it again if SteamVR is running. +systemctl --user stop frame-autopass.service 2>/dev/null || true +mkdir -p "$DEST" +for f in autopassd autopass; do + install -m 0755 "$src/$f" "$DEST/$f.new" + mv -f "$DEST/$f.new" "$DEST/$f" +done +"$DEST/autopass" install + +if [ "$old_version" = "none" ]; then + say "frame-autopass $new_version installed. It runs whenever SteamVR runs; nothing else to do." +else + say "frame-autopass updated: $old_version -> $new_version." +fi +say "Check it any time with: $DEST/autopass status" diff --git a/scripts/fetch-deps.sh b/scripts/fetch-deps.sh new file mode 100755 index 0000000..5eef94b --- /dev/null +++ b/scripts/fetch-deps.sh @@ -0,0 +1,29 @@ +#!/bin/sh +# Fetch the build dependencies into gitignored directories. Nothing is +# installed system-wide. +set -eu +cd "$(dirname "$0")/.." + +ZIG_VERSION=0.16.0 +ZIG_SHA256=70e49664a74374b48b51e6f3fdfbf437f6395d42509050588bd49abe52ba3d00 +OPENVR_COMMIT=0924064316de3effbcd1acf1e309182a2deb1c05 # OpenVR SDK 2.15.6 + +if [ ! -x toolchain/zig-x86_64-linux-$ZIG_VERSION/zig ]; then + mkdir -p toolchain + tarball=zig-x86_64-linux-$ZIG_VERSION.tar.xz + curl -sSfL -o toolchain/$tarball https://ziglang.org/download/$ZIG_VERSION/$tarball + echo "$ZIG_SHA256 toolchain/$tarball" | sha256sum -c + tar -C toolchain -xf toolchain/$tarball + rm toolchain/$tarball +fi + +if [ ! -f third_party/openvr/headers/openvr.h ]; then + mkdir -p third_party/openvr + git -C third_party/openvr init -q + git -C third_party/openvr fetch -q --depth 1 https://github.com/ValveSoftware/openvr $OPENVR_COMMIT + git -C third_party/openvr checkout -q FETCH_HEAD + # The SDK repo ships prebuilt binaries for every platform; only the + # headers and the loader source are needed. + rm -rf third_party/openvr/bin third_party/openvr/lib third_party/openvr/samples \ + third_party/openvr/unity_package third_party/openvr/controller_callouts third_party/openvr/.git +fi diff --git a/src/app_paths.cpp b/src/app_paths.cpp new file mode 100644 index 0000000..f1847d6 --- /dev/null +++ b/src/app_paths.cpp @@ -0,0 +1,85 @@ +#include "app_paths.hpp" + +#include + +#include +#include +#include +#include +#include +#include +#include + +namespace autopass { +namespace { + +std::string home() { + const char* h = std::getenv("HOME"); + return h && *h ? h : "/tmp"; +} + +std::string xdg(const char* var, const char* fallback) { + const char* v = std::getenv(var); + if (v && *v == '/') return v; + return home() + fallback; +} + +} // namespace + +std::string config_dir() { return xdg("XDG_CONFIG_HOME", "/.config") + "/frame-autopass"; } +std::string install_dir() { return xdg("XDG_DATA_HOME", "/.local/share") + "/frame-autopass"; } + +bool make_dirs(const std::string& path) { + std::string partial; + std::stringstream ss(path); + std::string part; + if (!path.empty() && path[0] == '/') partial = "/"; + while (std::getline(ss, part, '/')) { + if (part.empty()) continue; + partial += part + "/"; + if (::mkdir(partial.c_str(), 0755) != 0 && errno != EEXIST) return false; + } + return true; +} + +bool write_file_atomic(const std::string& path, const std::string& contents) { + const std::string tmp = path + ".tmp"; + { + std::ofstream f(tmp, std::ios::binary | std::ios::trunc); + if (!(f << contents) || !f.flush()) return false; + } + return std::rename(tmp.c_str(), path.c_str()) == 0; +} + +bool read_file(const std::string& path, std::string* contents) { + std::ifstream f(path, std::ios::binary); + if (!f) return false; + std::stringstream ss; + ss << f.rdbuf(); + *contents = ss.str(); + return true; +} + +bool colour_module_present() { + const std::string base = "/sys/class/video4linux"; + DIR* dir = ::opendir(base.c_str()); + if (!dir) return false; + bool found = false; + while (const dirent* e = ::readdir(dir)) { + if (e->d_name[0] == '.') continue; + std::string name; + if (read_file(base + "/" + e->d_name + "/name", &name) && name.find("arcimx616") != std::string::npos) { + found = true; + break; + } + } + ::closedir(dir); + return found; +} + +#ifndef AUTOPASS_VERSION +#define AUTOPASS_VERSION "dev" +#endif +const char* version() { return AUTOPASS_VERSION; } + +} // namespace autopass diff --git a/src/app_paths.hpp b/src/app_paths.hpp new file mode 100644 index 0000000..e869fc6 --- /dev/null +++ b/src/app_paths.hpp @@ -0,0 +1,38 @@ +#pragma once + +// Where autopassd and autopass keep their files. Everything lives in the user's +// home directory; nothing is written to system locations. + +#include + +namespace autopass { + +// ~/.config/frame-autopass (honours XDG_CONFIG_HOME). +std::string config_dir(); +// ~/.local/share/frame-autopass (honours XDG_DATA_HOME): the +// installed binaries (autopassd, autopass). +std::string install_dir(); + +inline std::string status_path() { return config_dir() + "/status.json"; } +inline std::string log_path() { return config_dir() + "/autopassd.log"; } +inline std::string conf_path() { return config_dir() + "/autopass.conf"; } +inline std::string lock_path() { return config_dir() + "/autopassd.lock"; } +// Crash guard: XRService restarts soon after an autopassd switch, one line each. +inline std::string crash_guard_path() { return config_dir() + "/crash_guard"; } + +// Creates the directory and its parents. Returns false on failure. +bool make_dirs(const std::string& path); +// Writes a file atomically (temporary file, then rename). +bool write_file_atomic(const std::string& path, const std::string& contents); +// Reads a whole small file; returns false if it cannot be read. +bool read_file(const std::string& path, std::string* contents); + +// Whether the Arcturus Vision colour module is attached: its sensors +// (arcimx616) are listed by name in /sys/class/video4linux. Only reads +// sysfs names; never opens a camera device. +bool colour_module_present(); + +// The release version, set at build time (Makefile VERSION). +const char* version(); + +} // namespace autopass diff --git a/src/autopassd.cpp b/src/autopassd.cpp new file mode 100644 index 0000000..d242164 --- /dev/null +++ b/src/autopassd.cpp @@ -0,0 +1,429 @@ +// autopassd: adaptive passthrough for the Steam Frame. +// +// Shows the Arcturus colour feed in good light and the built-in mono IR +// feed in low light, fully automatically: a fast switch, and an adaptive +// cooldown so it never flickers (light_policy.hpp). +// +// Designed to cost nothing when passthrough is not in use: +// - It does not connect to SteamVR. It follows XRService's session log +// with inotify (xrservice_log.hpp): passthrough shown/hidden, standby, +// IR emitters, tracking loss. XRService writes nothing while the headset +// is idle, so autopassd is not woken at all then. +// - While passthrough is shown in colour it reads the colour camera's light +// value from shared memory once a second (4 Hz while a decision is +// pending). While IR is shown with the IR emitters on, it reads the IR +// camera's light value from the same shared memory 4 times a second to +// recognise a lit room (sensors.hpp), and logs it every 10 s. It never +// captures images. +// - To switch, it runs `autopass set rgb|mono`, which connects to SteamVR as +// a background client for ~15 ms (it does not wake the headset), checks +// the private interface's fingerprint, switches and exits. +// +// Started and stopped with SteamVR by a systemd user unit (`autopass install`). +// Usage: autopassd [--observe] [--verbose] + +#include "app_paths.hpp" +#include "daemon_config.hpp" +#include "light_policy.hpp" +#include "passthrough_state.hpp" +#include "sensors.hpp" +#include "xrservice_log.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +extern char** environ; + +namespace { + +using autopass::Evidence; +using autopass::Source; + +volatile std::sig_atomic_t g_stop = 0; +void on_signal(int) { g_stop = 1; } + +bool g_verbose = false; +std::FILE* g_log = nullptr; + +std::uint64_t now_ms() { + return static_cast(std::chrono::duration_cast( + std::chrono::steady_clock::now().time_since_epoch()).count()); +} + +std::string wall_time() { + char buf[32]; + const std::time_t t = std::time(nullptr); + std::strftime(buf, sizeof buf, "%Y-%m-%d %H:%M:%S", std::localtime(&t)); + return buf; +} + +void log(const std::string& msg) { + const std::string line = wall_time() + " " + msg + "\n"; + if (g_log) { + std::fputs(line.c_str(), g_log); + std::fflush(g_log); + struct stat st{}; + if (::stat(autopass::log_path().c_str(), &st) == 0 && st.st_size > 512 * 1024) { + std::fclose(g_log); + std::rename(autopass::log_path().c_str(), (autopass::log_path() + ".1").c_str()); + g_log = std::fopen(autopass::log_path().c_str(), "a"); + } + } else { + std::fputs(line.c_str(), stderr); + } +} + +std::string json_escape(const std::string& s) { + std::string out; + for (char c : s) { + if (c == '"' || c == '\\') out += '\\'; + out += c; + } + return out; +} + +const char* tri(const std::optional& v, const char* yes, const char* no) { + return !v ? "unknown" : *v ? yes : no; +} + +// Exit codes of `autopass set` (tools/autopass.cpp). +constexpr int kSetOk = 0; +constexpr int kSetMismatch = 3; // SteamVR's private interface changed +constexpr int kSetNotEnabled = 4; // passthrough camera not enabled +constexpr int kSetNoModule = 5; // no Arcturus colour module + +// Runs `autopass set rgb|mono --quiet` and waits up to 5 s. Returns its exit +// code, or -1 if it could not be run or did not finish. +int run_switch_helper(bool rgb) { + const std::string helper = autopass::install_dir() + "/autopass"; + char arg0[] = "autopass", arg1[] = "set", arg_rgb[] = "rgb", arg_mono[] = "mono", arg3[] = "--quiet"; + char* argv[] = {arg0, arg1, rgb ? arg_rgb : arg_mono, arg3, nullptr}; + pid_t pid = 0; + if (::posix_spawn(&pid, helper.c_str(), nullptr, nullptr, argv, environ) != 0) return -1; + for (int i = 0; i < 100; ++i) { + int status = 0; + const pid_t r = ::waitpid(pid, &status, WNOHANG); + if (r == pid) return WIFEXITED(status) ? WEXITSTATUS(status) : -1; + if (r < 0 && errno != EINTR) return -1; + std::this_thread::sleep_for(std::chrono::milliseconds(50)); + } + ::kill(pid, SIGKILL); + ::waitpid(pid, nullptr, 0); + return -1; +} + +} // namespace + +int main(int argc, char** argv) { + bool observe_flag = false; + for (int i = 1; i < argc; ++i) { + const std::string a = argv[i]; + if (a == "--observe") observe_flag = true; + else if (a == "--verbose") g_verbose = true; + else if (a == "--help") { + std::printf("autopassd [--observe] [--verbose]\n"); + return 0; + } else { + std::fprintf(stderr, "unknown argument %s\n", a.c_str()); + return 2; + } + } + std::signal(SIGINT, on_signal); + std::signal(SIGTERM, on_signal); + + if (!autopass::make_dirs(autopass::config_dir())) { + std::fprintf(stderr, "cannot create %s\n", autopass::config_dir().c_str()); + return 1; + } + const int lock_fd = ::open(autopass::lock_path().c_str(), O_CREAT | O_RDWR | O_CLOEXEC, 0600); + if (lock_fd < 0 || ::flock(lock_fd, LOCK_EX | LOCK_NB) != 0) { + std::fprintf(stderr, "autopassd is already running\n"); + return 1; + } + g_log = std::fopen(autopass::log_path().c_str(), "a"); + + autopass::DaemonConfig cfg; + std::string conf_text; + if (autopass::read_file(autopass::conf_path(), &conf_text)) { + const auto problem = autopass::parse_daemon_config(conf_text, &cfg); + if (!problem.empty()) { + log("autopass.conf rejected (" + problem + "); using defaults"); + cfg = {}; + } + } + + // Crash guard (FINDINGS.md 31): two XRService restarts within 10 s of an + // autopassd switch stop switching until `autopass guard reset`. + std::string guard_text; + int guard_hits = 0; + if (autopass::read_file(autopass::crash_guard_path(), &guard_text)) + for (char c : guard_text) guard_hits += c == '\n'; + // Why switching is off, if it is (shown by `autopass status`): empty, or + // observe-only, crash-guard, steamvr-changed, no-colour-module. + std::string blocked = observe_flag || cfg.observe_only ? "observe-only" : guard_hits >= 2 ? "crash-guard" : ""; + if (blocked == "crash-guard") log("crash guard: XRService restarted twice soon after switches; observe-only until `autopass guard reset`"); + else if (!blocked.empty()) log("observe-only: decisions are logged, never applied"); + + // Follow XRService's log. SteamVR may still be starting: retry quietly. + autopass::XrServiceLog xr; + bool warned = false; + while (!g_stop && !xr.start()) { + if (!warned) log("waiting for XRService's log (" + xr.error() + ")"); + warned = true; + std::this_thread::sleep_for(std::chrono::seconds(3)); + } + if (g_stop) return 0; + + autopass::PassthroughState state; + Source initial = Source::Color; + if (const auto snap = state.read(); snap && !snap->config.rgb) initial = Source::Ir; + autopass::LightPolicy policy(cfg.policy, initial); + autopass::EvidenceBuilder evidence(cfg.sensors); + log(std::string("autopassd ") + autopass::version() + " started: showing " + autopass::to_string(initial)); + + double last_colour_ts = -1; + double mono_ts_at_last_light = 0; + std::uint64_t last_ir_log_ms = 0; + // The last 10 s of IR light readings (4 a second), written to the log + // when autopassd returns to colour, so a slow return shows what it read. + struct LightNote { + std::uint64_t ms; + double light; + }; + std::deque recent_ir; + // When the cameras first saw the room go dark or lit, to log how long + // XRService takes to turn its IR emitters on or off after that. + std::optional dark_seen_ms, lit_seen_ms; + std::optional last_emitters; + long long last_switch_wall = 0; + std::uint64_t expect_source_until_ms = 0, last_state_retry_ms = 0; + bool was_active = false; + std::string last_reason, last_status; + + autopass::Cause last_cause = autopass::Cause::None; + const auto apply = [&](Source target, const std::string& reason, std::uint64_t now) { + if (!blocked.empty()) { + log(std::string("would switch to ") + autopass::to_string(target) + " (" + reason + ") [" + blocked + "]"); + return; + } + const int rc = run_switch_helper(target == Source::Color); + if (rc == kSetOk) { + log(std::string("switched to ") + autopass::to_string(target) + ": " + reason); + if (target == Source::Color && !recent_ir.empty()) { + std::string notes = "IR light before the switch (s ago: value):"; + for (const auto& n : recent_ir) { + char item[32]; + std::snprintf(item, sizeof item, " %.1f:%.2f", (now - n.ms) / 1000.0, n.light); + notes += item; + } + log(notes); + } + recent_ir.clear(); + expect_source_until_ms = now + 3000; + last_switch_wall = static_cast(std::time(nullptr)); + evidence.on_switched(now, target, true, last_cause); + } else if (rc == kSetMismatch) { + log("SteamVR's camera interface changed (SteamVR update?); switching disabled until frame-autopass " + "is updated (`autopass update`)"); + blocked = "steamvr-changed"; + } else if (rc == kSetNoModule) { + log("Arcturus colour module not detected: not switching"); + blocked = "no-colour-module"; + } else if (rc == kSetNotEnabled) { + log("passthrough camera not enabled; not switching"); + } else { + log("switch helper failed (code " + std::to_string(rc) + ")"); + } + // Whatever happened, follow what is actually shown. + if (const auto snap = state.read()) policy.adopt(snap->config.rgb ? Source::Color : Source::Ir); + if (rc != kSetOk) evidence.reset(); + }; + + // First pass runs at once, so a passthrough already shown at start-up is + // handled without waiting for XRService's next log line. Afterwards -1 + // means: sleep until XRService logs something. + int wait_ms = 0; + while (!g_stop) { + pollfd pfd{xr.fd(), POLLIN, 0}; + if (::poll(&pfd, 1, wait_ms) < 0 && errno != EINTR) break; + if (g_stop) break; + const std::uint64_t now = now_ms(); + + const bool xr_changed = xr.update(); + const autopass::XrState& xs = xr.state(); + if (xr.take_restarted()) { + const long long restart = xr.session_start(); + const long long since = last_switch_wall && restart ? restart - last_switch_wall : -1; + log("XRService restarted" + (since >= 0 ? " " + std::to_string(since) + " s after autopassd's last switch" + : std::string(" (no recent autopassd switch)"))); + if (since >= 0 && since <= 10) { + std::string text; + autopass::read_file(autopass::crash_guard_path(), &text); + text += wall_time() + " XRService restart " + std::to_string(since) + " s after a switch\n"; + autopass::write_file_atomic(autopass::crash_guard_path(), text); + log("crash guard: recorded; switching stops after two such restarts"); + } + } + if (g_verbose && xr_changed) + log(std::string("xrservice: passthrough ") + tri(xs.passthrough, "shown", "hidden") + ", standby " + + tri(xs.standby, "yes", "no") + ", emitters " + tri(xs.emitters, "on", "off") + ", tracking " + + tri(xs.tracking_lost, "lost", "ok")); + + if (xs.emitters != last_emitters) { + if (last_emitters && xs.emitters) { + const auto& seen = *xs.emitters ? dark_seen_ms : lit_seen_ms; + char msg[160]; + if (seen) + std::snprintf(msg, sizeof msg, "IR emitters turned %s %.1f s after the cameras first saw the room %s", + *xs.emitters ? "on" : "off", (now - *seen) / 1000.0, *xs.emitters ? "dark" : "lit"); + else + std::snprintf(msg, sizeof msg, "IR emitters turned %s (no light change seen before it)", + *xs.emitters ? "on" : "off"); + log(msg); + } + last_emitters = xs.emitters; + } + + const bool active = xs.passthrough == true && xs.standby != true; + if (active != was_active) { + log(active ? "passthrough shown: sensing" : "passthrough not shown: idle"); + was_active = active; + // Without the Arcturus module there is no colour feed to switch to. + // Checked each time passthrough appears (a sysfs read). + if (active) { + const bool module = autopass::colour_module_present(); + if (!module && blocked.empty()) { + log("Arcturus colour module not detected: not switching"); + blocked = "no-colour-module"; + } else if (module && blocked == "no-colour-module") { + log("Arcturus colour module detected: switching again"); + blocked.clear(); + } + } + evidence.reset(); + last_colour_ts = -1; + dark_seen_ms.reset(); + lit_seen_ms.reset(); + } + + Evidence ev = Evidence::Unknown; + Source shown = policy.source(); + bool want_ir = false; + if (active) { + if (state.path().empty() && now - last_state_retry_ms > 5000) { + last_state_retry_ms = now; + state = autopass::PassthroughState(); + } + std::optional colour_light; + std::optional ir_light; + if (const auto snap = state.read()) { + shown = snap->config.rgb ? Source::Color : Source::Ir; + // The IR camera's light value counts only from a new frame: + // a stalled XRService leaves the last one in place. + if (shown == Source::Ir && snap->mono_light && snap->mono_timestamp > mono_ts_at_last_light) { + mono_ts_at_last_light = snap->mono_timestamp; + ir_light = *snap->mono_light; + } + if (shown != policy.source() && now > expect_source_until_ms) { + log(std::string("source changed outside autopassd to ") + autopass::to_string(shown)); + policy.adopt(shown); + evidence.reset(); + } + if (snap->colour) { + const bool fresh = last_colour_ts >= 0 && snap->colour->timestamp != last_colour_ts; + last_colour_ts = snap->colour->timestamp; + if (shown == Source::Color && fresh) colour_light = snap->colour->light; + } + } + + want_ir = shown == Source::Ir && evidence.wants_ir_light(now, xs.emitters); + if (want_ir && ir_light) { + recent_ir.push_back({now, *ir_light}); + while (!recent_ir.empty() && now - recent_ir.front().ms > 10000) recent_ir.pop_front(); + if (now - last_ir_log_ms >= 10000) { + last_ir_log_ms = now; + char msg[96]; + std::snprintf(msg, sizeof msg, "IR light %.2f, emitters %s, tracking %s", *ir_light, + tri(xs.emitters, "on", "off"), tri(xs.tracking_lost, "lost", "ok")); + log(msg); + } + } + + // Emitter timing (log only): the colour camera knows dark from lit; + // in IR, the IR light value is the first sign of light. + const auto saw = [&](bool lit) { + auto& mark = lit ? lit_seen_ms : dark_seen_ms; + if (!mark) mark = now; + (lit ? dark_seen_ms : lit_seen_ms).reset(); + }; + if (colour_light) { + if (*colour_light < cfg.sensors.colour_dark_light) saw(false); + else if (*colour_light >= cfg.sensors.colour_min_light) saw(true); + } + if (ir_light) { + if (*ir_light >= cfg.sensors.ir_min_light) saw(true); + else lit_seen_ms.reset(); // not lit (yet); "dark" is the colour camera's call + } + + // Tracking loss does not hold anything: in a dark room tracking + // can drop the moment the light goes out and stay lost while the + // headset is still (2026-10-01 21:43-21:53), and neither the + // emitters nor the IR light value depend on it (FINDINGS.md 41). + const autopass::EvidenceResult er = evidence.evaluate(now, shown, xs.emitters, colour_light, ir_light); + ev = er.evidence; + last_cause = er.cause; + const auto d = policy.update(now, ev, er.why, er.urgent); + if (d.changed) apply(d.source, d.reason, now); + if (g_verbose && d.reason != last_reason && d.reason != "missing reading") log("decision: " + d.reason); + last_reason = d.reason; + } + // Next wake-up. Idle, or nothing that could change our mind: sleep + // until XRService logs something. + const bool pending = last_reason == "confirming" || last_reason == "confirmed, waiting for cooldown" || + last_reason == "settling after switch" || last_reason == "missing reading"; + + // Colour shown: read the colour camera's light value once a second + // (XRService's "emitters on" line wakes autopassd at once when it gets + // dark). IR shown: read the IR camera's light value 4 times a second + // while it matters, otherwise wait for XRService's log. + if (!active) wait_ms = -1; + else if (pending) wait_ms = 250; + else if (want_ir) wait_ms = 250; + else if (shown == Source::Color) wait_ms = 1000; + else wait_ms = -1; + + char buf[512]; + std::snprintf(buf, sizeof buf, + "{\"pid\": %d, \"passthrough\": \"%s\", \"standby\": \"%s\", \"showing\": \"%s\", " + "\"ir_emitters\": \"%s\", \"tracking\": \"%s\", \"evidence\": \"%s\", \"cooldown_s\": %.0f, " + "\"can_switch\": %s, \"blocked\": \"%s\", \"reason\": \"%s\", \"version\": \"%s\"}\n", + static_cast(::getpid()), tri(xs.passthrough, "shown", "hidden"), tri(xs.standby, "yes", "no"), + autopass::to_string(policy.source()), tri(xs.emitters, "on", "off"), tri(xs.tracking_lost, "lost", "ok"), + autopass::to_string(ev), policy.cooldown_ms() / 1000.0, blocked.empty() ? "true" : "false", + blocked.c_str(), json_escape(active ? last_reason : "").c_str(), autopass::version()); + if (buf != last_status) { + autopass::write_file_atomic(autopass::status_path(), buf); + last_status = buf; + } + } + + xr.stop(); + log("autopassd stopped"); + if (g_log) std::fclose(g_log); + return 0; +} diff --git a/src/daemon_config.cpp b/src/daemon_config.cpp new file mode 100644 index 0000000..58988c0 --- /dev/null +++ b/src/daemon_config.cpp @@ -0,0 +1,92 @@ +#include "daemon_config.hpp" + +#include +#include +#include +#include + +namespace autopass { +namespace { + +std::string trim(const std::string& s) { + const auto b = s.find_first_not_of(" \t\r"); + if (b == std::string::npos) return {}; + const auto e = s.find_last_not_of(" \t\r"); + return s.substr(b, e - b + 1); +} + +bool parse_number(const std::string& text, double* out) { + if (text.empty()) return false; + char* end = nullptr; + errno = 0; + const double v = std::strtod(text.c_str(), &end); + if (errno || !end || *end != '\0' || !std::isfinite(v)) return false; + *out = v; + return true; +} + +} // namespace + +std::string parse_daemon_config(const std::string& text, DaemonConfig* config) { + DaemonConfig result = *config; + std::istringstream in(text); + std::string line; + int number = 0; + while (std::getline(in, line)) { + ++number; + const auto hash = line.find('#'); + if (hash != std::string::npos) line.erase(hash); + line = trim(line); + if (line.empty()) continue; + const auto eq = line.find('='); + const std::string where = "line " + std::to_string(number) + ": "; + if (eq == std::string::npos) return where + "expected key = value"; + const std::string key = trim(line.substr(0, eq)); + const std::string value = trim(line.substr(eq + 1)); + + if (key == "observe_only") { + if (value == "true") result.observe_only = true; + else if (value == "false") result.observe_only = false; + else return where + "observe_only must be true or false"; + continue; + } + double* target = nullptr; + if (key == "confirm_s") target = &result.policy.confirm_s; + else if (key == "urgent_confirm_s") target = &result.policy.urgent_confirm_s; + else if (key == "confirm_to_colour_s") target = &result.policy.confirm_to_colour_s; + else if (key == "cooldown_base_s") target = &result.policy.cooldown_base_s; + else if (key == "cooldown_max_s") target = &result.policy.cooldown_max_s; + else if (key == "cooldown_reset_s") target = &result.policy.cooldown_reset_s; + else if (key == "flicker_slack_s") target = &result.policy.flicker_slack_s; + else if (key == "settle_s") target = &result.policy.settle_s; + else if (key == "settle_to_colour_s") target = &result.policy.settle_to_colour_s; + else if (key == "stale_s") target = &result.policy.stale_s; + else if (key == "colour_min_light") target = &result.sensors.colour_min_light; + else if (key == "colour_dark_light") target = &result.sensors.colour_dark_light; + else if (key == "verify_s") target = &result.sensors.verify_s; + else if (key == "ir_min_light") target = &result.sensors.ir_min_light; + else if (key == "ir_light_hold_s") target = &result.sensors.ir_light_hold_s; + else if (key == "ir_light_lockout_s") target = &result.sensors.ir_light_lockout_s; + else if (key == "ir_light_lockout_max_s") target = &result.sensors.ir_light_lockout_max_s; + else if (key == "emitters_off_distrust_s") target = &result.sensors.emitters_off_distrust_s; + else if (key == "emitters_off_distrust_max_s") target = &result.sensors.emitters_off_distrust_max_s; + else if (key == "smoothing_s") target = &result.sensors.smoothing_s; + else return where + "unknown key '" + key + "'"; + if (!parse_number(value, target)) return where + "'" + value + "' is not a number"; + } + const std::string problem = validate(result.policy); + if (!problem.empty()) return problem; + const auto& s = result.sensors; + if (!(s.colour_min_light > 0 && s.colour_min_light <= 1) || + !(s.colour_dark_light > 0 && s.colour_dark_light <= s.colour_min_light)) + return "colour light thresholds must be in (0, 1], colour_dark_light <= colour_min_light"; + if (!(s.verify_s >= 0) || !(s.smoothing_s >= 0) || !(s.ir_light_hold_s >= 0) || + !(s.ir_min_light > 0 && s.ir_min_light <= 1) || !(s.ir_light_lockout_s >= 0) || + !(s.ir_light_lockout_max_s >= s.ir_light_lockout_s) || !(s.emitters_off_distrust_s >= 0) || + !(s.emitters_off_distrust_max_s >= s.emitters_off_distrust_s)) + return "sensor durations must be non-negative (each _max_s at least its base)"; + *config = result; + return {}; +} + +} // namespace autopass diff --git a/src/daemon_config.hpp b/src/daemon_config.hpp new file mode 100644 index 0000000..65b17ee --- /dev/null +++ b/src/daemon_config.hpp @@ -0,0 +1,48 @@ +#pragma once + +// autopass.conf: optional `key = value` lines overriding the policy defaults. +// Blank lines and lines starting with '#' are ignored. Unknown keys and +// malformed values are errors, reported with their line number, so a typo +// never silently leaves a default in place. +// +// confirm_s = 0.5 # evidence must persist this long (to IR) +// confirm_to_colour_s = 0.25 # ...to colour (checked by the colour camera) +// urgent_confirm_s = 0.25 # ...when undoing a mistaken switch +// cooldown_base_s = 2 # anti-flicker cooldown: starts at base, +// cooldown_max_s = 30 # doubles for a switch that comes within +// flicker_slack_s = 1.5 # flicker_slack_s of being allowed, +// cooldown_reset_s = 30 # steps down for slower ones, clears after quiet +// settle_s = 1.5 # after a switch to IR +// settle_to_colour_s = 0.3 # after a switch to colour +// stale_s = 3.0 +// colour_min_light = 0.6 # with IR emitters on: dark below this +// colour_dark_light = 0.35 # emitters on and colour this dark right after a +// # switch to colour: that switch was a mistake +// verify_s = 3 # check colour right after switching to it +// ir_min_light = 0.15 # IR shown: the IR camera's light value says lit +// ir_light_hold_s = 0 # for this long (on top of confirm_to_colour_s) +// ir_light_lockout_s = 60 # ignore the IR light value after it misled us, doubling +// ir_light_lockout_max_s = 600 # up to this +// emitters_off_distrust_s = 60 # ignore "emitters off" after it misled us, doubling +// emitters_off_distrust_max_s = 600 # up to this +// smoothing_s = 0 # extra smoothing of the colour light value (the camera already smooths it) +// observe_only = false # log decisions but never switch + +#include "light_policy.hpp" +#include "sensors.hpp" + +#include + +namespace autopass { + +struct DaemonConfig { + PolicyConfig policy; + SensorConfig sensors; + bool observe_only = false; +}; + +// Parses `text` over `config` (so absent keys keep their current value). +// Returns an empty string on success, otherwise "line N: reason". +std::string parse_daemon_config(const std::string& text, DaemonConfig* config); + +} // namespace autopass diff --git a/src/light_policy.cpp b/src/light_policy.cpp new file mode 100644 index 0000000..3008e81 --- /dev/null +++ b/src/light_policy.cpp @@ -0,0 +1,155 @@ +#include "light_policy.hpp" + +#include +#include +#include + +namespace autopass { + +const char* to_string(Source source) { + return source == Source::Color ? "colour" : "ir"; +} + +const char* to_string(Mode mode) { + switch (mode) { + case Mode::Auto: return "auto"; + case Mode::ForceColor: return "colour"; + case Mode::ForceIr: return "ir"; + } + return "unknown"; +} + +const char* to_string(Evidence e) { + switch (e) { + case Evidence::Unknown: return "unknown"; + case Evidence::Neutral: return "neutral"; + case Evidence::Dark: return "dark"; + case Evidence::Bright: return "bright"; + } + return "unknown"; +} + +bool parse_mode(const std::string& text, Mode* mode) { + if (text == "auto") *mode = Mode::Auto; + else if (text == "colour" || text == "color") *mode = Mode::ForceColor; + else if (text == "ir" || text == "mono") *mode = Mode::ForceIr; + else return false; + return true; +} + +std::string validate(const PolicyConfig& c) { + for (double v : {c.confirm_s, c.confirm_to_colour_s, c.urgent_confirm_s, c.cooldown_base_s, c.cooldown_max_s, c.cooldown_reset_s, c.flicker_slack_s, c.settle_s, + c.settle_to_colour_s, c.stale_s}) + if (!std::isfinite(v) || v < 0) return "durations must be finite and non-negative"; + if (c.stale_s <= 0) return "stale_s must be positive"; + if (c.cooldown_max_s < c.cooldown_base_s) return "cooldown_max_s must be at least cooldown_base_s"; + return {}; +} + +namespace { +std::uint64_t to_ms(double seconds) { + return static_cast(std::llround(seconds * 1000.0)); +} +} // namespace + +LightPolicy::LightPolicy(PolicyConfig config, Source initial) + : config_(config), source_(initial) {} + +void LightPolicy::reset_confirmation() { + pending_ = false; + pending_since_ms_ = 0; +} + +bool LightPolicy::settling(std::uint64_t now_ms) const { + const double settle = source_ == Source::Color ? config_.settle_to_colour_s : config_.settle_s; + return switched_ && now_ms - last_switch_ms_ < to_ms(settle); +} + +std::uint64_t LightPolicy::cooldown_ms() const { + double s = config_.cooldown_base_s; + for (int i = 0; i < flicker_level_ && s < config_.cooldown_max_s; ++i) s *= 2; + return to_ms(std::min(s, config_.cooldown_max_s)); +} + +void LightPolicy::adopt(Source actual) { + if (actual == source_) return; + source_ = actual; + reset_confirmation(); +} + +Decision LightPolicy::decide(std::uint64_t now_ms, Source target, std::string reason, bool automatic) { + const bool changed = target != source_; + if (changed) { + source_ = target; + switched_ = true; + last_switch_ms_ = now_ms; + if (automatic) { + // Only a switch that came about as soon as the cooldown allowed + // it is flicker; slower ones step the cooldown back down. + if (auto_switched_) { + const std::uint64_t gap = now_ms - last_auto_switch_ms_, allowed = cooldown_ms(); + if (gap < allowed + to_ms(config_.flicker_slack_s)) + flicker_level_ = std::min(flicker_level_ + 1, 16); + else if (gap >= allowed + to_ms(config_.cooldown_reset_s)) + flicker_level_ = 0; + else if (flicker_level_ > 0) + --flicker_level_; + } + auto_switched_ = true; + last_auto_switch_ms_ = now_ms; + } + reset_confirmation(); + } + return {source_, changed, std::move(reason)}; +} + +Decision LightPolicy::set_mode(std::uint64_t now_ms, Mode mode) { + mode_ = mode; + reset_confirmation(); + switch (mode) { + case Mode::ForceColor: return decide(now_ms, Source::Color, "manual: force colour", false); + case Mode::ForceIr: return decide(now_ms, Source::Ir, "manual: force IR", false); + case Mode::Auto: break; + } + return decide(now_ms, source_, "manual: auto, holding current source until light confirms a change", false); +} + +Decision LightPolicy::update(std::uint64_t now_ms, Evidence evidence, const std::string& why, bool urgent) { + if (now_ms < last_tick_ms_) now_ms = last_tick_ms_; // never run time backwards + last_tick_ms_ = now_ms; + + if (settling(now_ms)) return {source_, false, "settling after switch"}; + + if (evidence == Evidence::Unknown) { + if (last_known_ms_ == 0 || now_ms - last_known_ms_ >= to_ms(config_.stale_s)) { + reset_confirmation(); + return {source_, false, "no reading, holding"}; + } + return {source_, false, "missing reading"}; + } + last_known_ms_ = now_ms; + + if (mode_ != Mode::Auto) return {source_, false, std::string("manual: ") + to_string(mode_)}; + + const bool want_ir = source_ == Source::Color && evidence == Evidence::Dark; + const bool want_color = source_ == Source::Ir && evidence == Evidence::Bright; + if (!want_ir && !want_color) { + reset_confirmation(); + return {source_, false, "no change needed"}; + } + + if (!pending_) { + pending_ = true; + pending_since_ms_ = now_ms; + } + const double confirm = urgent ? config_.urgent_confirm_s : want_color ? config_.confirm_to_colour_s : config_.confirm_s; + if (now_ms - pending_since_ms_ < to_ms(confirm)) return {source_, false, "confirming"}; + if (!urgent && auto_switched_ && now_ms - last_auto_switch_ms_ < cooldown_ms()) + return {source_, false, "confirmed, waiting for cooldown"}; + + const std::string suffix = why.empty() ? "" : " (" + why + ")"; + return want_ir ? decide(now_ms, Source::Ir, "auto: too dark for colour" + suffix, true) + : decide(now_ms, Source::Color, "auto: bright enough for colour" + suffix, true); +} + +} // namespace autopass diff --git a/src/light_policy.hpp b/src/light_policy.hpp new file mode 100644 index 0000000..0e131e0 --- /dev/null +++ b/src/light_policy.hpp @@ -0,0 +1,123 @@ +#pragma once + +// Decides which passthrough source to show. Pure logic: no SteamVR, no +// clock, no I/O, so it can be tested on the host with synthetic signals. +// +// Sensors (see sensors.hpp) turn their readings into Evidence each tick: +// Dark (too dark for colour passthrough), Bright (bright enough for +// colour), Neutral (no reason to change), or Unknown (no reading). This +// class owns the timing rules: the first switch is fast (a short +// confirmation), and an adaptive cooldown stops flicker. The cooldown +// before the next automatic switch starts short and doubles each time +// switches follow each other quickly, resetting after a quiet period. +// There is also a settle window after any switch and holding when +// readings go stale. + +#include +#include + +namespace autopass { + +enum class Source { Color, Ir }; +enum class Mode { Auto, ForceColor, ForceIr }; +enum class Evidence { Unknown, Neutral, Dark, Bright }; + +const char* to_string(Source source); +const char* to_string(Mode mode); +const char* to_string(Evidence evidence); +bool parse_mode(const std::string& text, Mode* mode); + +struct PolicyConfig { + // Evidence for a change must persist this long before a switch fires, + // so a hand passing over the cameras does not. Urgent evidence (undoing + // a switch that turned out wrong) needs only urgent_confirm_s. + // Switching to colour is checked by the colour camera straight away + // (sensors.hpp), so a mistake there costs under a second: it can be + // quicker (confirm_to_colour_s) than the switch to IR, where confirm_s + // keeps a hand over the cameras from counting. 0.5 s is enough there + // since switching to IR also needs XRService's emitters on, which a + // hand over the colour camera does not cause (FINDINGS.md 45). + double confirm_s = 0.5; + double confirm_to_colour_s = 0.25; + double urgent_confirm_s = 0.25; + // Cooldown after an automatic switch before the next automatic one. It + // only grows for flicker: a switch that comes within flicker_slack_s of + // the moment the cooldown allowed it (the light changing back as soon + // as it could, as a flashing light or a fooled sensor does) doubles it, + // up to cooldown_max_s. Anything slower, such as someone turning lights + // on and off, steps it back down one level per switch, and + // cooldown_reset_s of quiet clears it. Manual switches do not count: + // the cooldown exists to stop automatic flip-flopping, not to delay the + // user. + double cooldown_base_s = 2.0; + double cooldown_max_s = 30.0; + double cooldown_reset_s = 30.0; + double flicker_slack_s = 1.5; + // Evidence this soon after a switch is ignored while the other + // camera's pipeline settles. The colour camera's light value is valid + // on its first frame (FINDINGS.md 21), so after a switch to colour a + // much shorter settle lets a mistaken switch be caught quickly. + double settle_s = 1.5; + double settle_to_colour_s = 0.3; + // With no reading for this long, hold the current source and forget + // partial confirmation. + double stale_s = 3.0; +}; + +// Returns an empty string when the config is usable, otherwise the reason. +std::string validate(const PolicyConfig& config); + +struct Decision { + Source source; + bool changed; // source differs from the previous decision + std::string reason; // human-readable, for logs and the status file +}; + +class LightPolicy { +public: + LightPolicy(PolicyConfig config, Source initial); + + // Feed the evidence for this tick. Timestamps are monotonic + // milliseconds. `why` is appended to the reason of an automatic switch. + // `urgent` evidence (a switch that turned out to be a mistake) still + // needs confirmation but skips the cooldown; it counts towards the + // cooldown's escalation like any automatic switch. + Decision update(std::uint64_t now_ms, Evidence evidence, const std::string& why = {}, bool urgent = false); + + // Manual override. Leaving a forced mode for Auto keeps the current + // source and requires fresh confirmation before any switch. + Decision set_mode(std::uint64_t now_ms, Mode mode); + + // The source actually shown changed without us. Adopt it without + // counting it as a switch. + void adopt(Source actual); + + Mode mode() const { return mode_; } + Source source() const { return source_; } + // True within settle_s of the last switch (sensors should reset). + bool settling(std::uint64_t now_ms) const; + // Current cooldown after the last automatic switch, in milliseconds. + std::uint64_t cooldown_ms() const; + +private: + Decision decide(std::uint64_t now_ms, Source target, std::string reason, bool automatic); + void reset_confirmation(); + + PolicyConfig config_; + Mode mode_ = Mode::Auto; + Source source_; + + std::uint64_t last_tick_ms_ = 0; + std::uint64_t last_known_ms_ = 0; + + bool switched_ = false; // any switch (drives the settle window) + std::uint64_t last_switch_ms_ = 0; + bool auto_switched_ = false; // automatic switch (drives the cooldown) + std::uint64_t last_auto_switch_ms_ = 0; + int flicker_level_ = 0; // cooldown doublings in effect + + bool pending_ = false; + std::uint64_t pending_since_ms_ = 0; +}; + +} // namespace autopass diff --git a/src/passthrough_state.cpp b/src/passthrough_state.cpp new file mode 100644 index 0000000..a661180 --- /dev/null +++ b/src/passthrough_state.cpp @@ -0,0 +1,146 @@ +#include "passthrough_state.hpp" + +#include +#include +#include +#include +#include +#include +#include + +namespace autopass { +namespace { + +constexpr std::size_t kConfigOffset = 2; +// Stream blocks: mono at 0x14, colour at 0x20e8. Inside each, per-camera +// blocks 0x1040 apart hold per-frame records 0x104 apart starting at +0x38; +// a record's frame timestamp (float64) is at +0x10. +constexpr std::size_t kMonoStream = 0x14; +constexpr std::size_t kFirstRecord = 0x38; +constexpr std::size_t kCameraStride = 0x1040; +constexpr std::size_t kRecordStride = 0x104; +constexpr int kCameras = 2; +constexpr int kRecordsPerCamera = 4; +// The colour stream block starts here; everything before is the mono one. +constexpr std::size_t kColourStream = 0x20e8; +// Per-frame records are found by a float32 1/2.2 gamma marker. +constexpr std::size_t kRecordBeforeMarker = 0xd8; +constexpr std::size_t kTimestampInRecord = 0x10; +constexpr std::size_t kLightAfterMarker = 0x14; +// The same light value, by fixed offset: mono records carry no gamma marker. +constexpr std::size_t kLightInRecord = kRecordBeforeMarker + kLightAfterMarker; + +float read_f32(const std::uint8_t* p) { + float v; + std::memcpy(&v, p, sizeof v); + return v; +} + +double read_f64(const std::uint8_t* p) { + double v; + std::memcpy(&v, p, sizeof v); + return v; +} + +double newest_timestamp(const std::uint8_t* buf, std::size_t stream) { + double newest = 0; + for (int cam = 0; cam < kCameras; ++cam) + for (int rec = 0; rec < kRecordsPerCamera; ++rec) { + const std::size_t off = stream + kFirstRecord + cam * kCameraStride + rec * kRecordStride + kTimestampInRecord; + const double ts = read_f64(buf + off); + if (std::isfinite(ts) && ts > newest) newest = ts; + } + return newest; +} + +} // namespace + +bool CameraConfig::valid() const { + for (auto b : bytes()) + if (b > 1) return false; + return true; +} + +std::string CameraConfig::describe() const { + std::string s; + s += "enabled=" + std::to_string(enabled); + s += " stereo=" + std::to_string(stereo); + s += " rgb=" + std::to_string(rgb); + s += " disable_devignetting=" + std::to_string(disable_devignetting); + s += " sharpening=" + std::to_string(sharpening); + return s; +} + +std::string PassthroughState::locate() { + std::string found; + int matches = 0; + DIR* dir = ::opendir("/dev/shm"); + if (!dir) return {}; + while (auto* entry = ::readdir(dir)) { + const std::string name = entry->d_name; + if (name.rfind("u", 0) != 0 || name.find("-Shm_") == std::string::npos) continue; + const std::string path = "/dev/shm/" + name; + struct stat st{}; + if (::stat(path.c_str(), &st) == 0 && static_cast(st.st_size) == kSize) { + found = path; + ++matches; + } + } + ::closedir(dir); + return matches == 1 ? found : std::string(); +} + +PassthroughState::PassthroughState(std::string path) : path_(std::move(path)) {} + +std::optional PassthroughState::read() const { + if (path_.empty()) return std::nullopt; + const int fd = ::open(path_.c_str(), O_RDONLY | O_CLOEXEC); + if (fd < 0) return std::nullopt; + std::vector buf(kSize); + std::size_t got = 0; + while (got < kSize) { + const auto n = ::pread(fd, buf.data() + got, kSize - got, static_cast(got)); + if (n <= 0) break; + got += static_cast(n); + } + ::close(fd); + if (got != kSize) return std::nullopt; + return parse(buf.data()); +} + +PassthroughSnapshot PassthroughState::parse(const std::uint8_t* buf) { + PassthroughSnapshot snap; + snap.config = CameraConfig::from(buf + kConfigOffset); + snap.mono_timestamp = newest_timestamp(buf, kMonoStream); + snap.colour_timestamp = newest_timestamp(buf, kColourStream); + double newest_mono = 0; + for (int cam = 0; cam < kCameras; ++cam) + for (int rec = 0; rec < kRecordsPerCamera; ++rec) { + const std::uint8_t* record = buf + kMonoStream + kFirstRecord + cam * kCameraStride + rec * kRecordStride; + const double ts = read_f64(record + kTimestampInRecord); + const float light = read_f32(record + kLightInRecord); + if (std::isfinite(ts) && ts > newest_mono && light >= 0.0f && light <= 1.0f) { + newest_mono = ts; + snap.mono_light = light; + } + } + + // 1/2.2 computed in double and rounded to float by the writer. A float + // division (1.0f / 2.2f) rounds to 0x3ee8ba2e and would never match. + const std::uint32_t gamma_bits = 0x3ee8ba2f; + std::uint8_t marker[4]; + std::memcpy(marker, &gamma_bits, sizeof marker); + + for (std::size_t pos = kColourStream + kRecordBeforeMarker; pos + kLightAfterMarker + 4 <= kSize; pos += 4) { + if (std::memcmp(buf + pos, marker, 4) != 0) continue; + const std::uint8_t* record = buf + pos - kRecordBeforeMarker; + ColourFrameInfo info; + info.timestamp = read_f64(record + kTimestampInRecord); + info.light = read_f32(buf + pos + kLightAfterMarker); + if (!(info.timestamp > 0) || !(info.light >= 0.0f && info.light <= 1.0f)) continue; + if (!snap.colour || info.timestamp > snap.colour->timestamp) snap.colour = info; + } + return snap; +} + +} // namespace autopass diff --git a/src/passthrough_state.hpp b/src/passthrough_state.hpp new file mode 100644 index 0000000..e39edbc --- /dev/null +++ b/src/passthrough_state.hpp @@ -0,0 +1,73 @@ +#pragma once + +// Passive reader for SteamVR's VR_CameraPassthroughState shared memory +// (see FINDINGS.md sections 12 and 13). Reads a tmpfs file only: no +// SteamVR calls, no locks, no devices, so it cannot disturb passthrough. + +#include +#include +#include +#include + +namespace autopass { + +// The 5-byte passthrough camera config, as stored at state offset 2. +struct CameraConfig { + std::uint8_t enabled = 0; + std::uint8_t stereo = 0; + std::uint8_t rgb = 0; + std::uint8_t disable_devignetting = 0; + std::uint8_t sharpening = 0; + + std::array bytes() const { + return {enabled, stereo, rgb, disable_devignetting, sharpening}; + } + static CameraConfig from(const std::uint8_t* p) { return {p[0], p[1], p[2], p[3], p[4]}; } + bool valid() const; // every byte is 0 or 1 + std::string describe() const; +}; + +struct ColourFrameInfo { + double timestamp = 0; // seconds, SteamVR clock + float light = 0; // "g4": ~1 bright, 0 when too dark for colour +}; + +struct PassthroughSnapshot { + CameraConfig config; + // Newest colour-stream frame, if any record is present. + std::optional colour; + // Newest frame timestamp of each stream (0 if none). Only the stream + // being displayed advances, so a timestamp that keeps moving means + // that passthrough source is live (FINDINGS.md section 26). + double mono_timestamp = 0; + double colour_timestamp = 0; + // Light value of the newest mono frame, at the record offset where the + // colour stream keeps g4 (FINDINGS.md 41): 0 in the dark even with the + // IR emitters on, about 0.2-0.35 in a lit room. Only advances while IR + // is shown. + std::optional mono_light; +}; + +class PassthroughState { +public: + static constexpr std::size_t kSize = 0x6200; + + // Finds the state file by its unique size under /dev/shm. Returns an + // empty string when there is not exactly one candidate. + static std::string locate(); + + explicit PassthroughState(std::string path = locate()); + const std::string& path() const { return path_; } + + // Reads and parses one snapshot. Returns nullopt if the file cannot be + // read or is not the expected size. + std::optional read() const; + + // Exposed for tests: parse a raw buffer of kSize bytes. + static PassthroughSnapshot parse(const std::uint8_t* buf); + +private: + std::string path_; +}; + +} // namespace autopass diff --git a/src/private_camera.cpp b/src/private_camera.cpp new file mode 100644 index 0000000..75ceeea --- /dev/null +++ b/src/private_camera.cpp @@ -0,0 +1,102 @@ +#include "private_camera.hpp" + +#include + +#include +#include +#include +#include + +namespace autopass { +namespace { + +constexpr const char* kInterface = "IVRCameraPassthroughInternal_001"; +constexpr std::size_t kGetConfig = 9; +constexpr std::size_t kSetConfig = 10; + +// First instructions of CVRCameraPassthroughInternal's GetConfig and +// SetConfig in SteamVR 2.17.10 (vrclient.so 0x1a49d8 and 0x1a4bf8). Only +// position-independent words are included: the prologue, argument moves, +// the `add x1, x0, #0x18` that reaches the shared-state accessor, and a +// short relative branch. A call instruction would change whenever the +// library is relinked, so the fingerprints stop before the first `bl`. +constexpr std::array kGetFingerprint = { + 0xa9bc7bfd, // stp x29, x30, [sp, #-0x40]! + 0x910003fd, // mov x29, sp + 0xa90153f3, // stp x19, x20, [sp, #0x10] + 0xaa0103f3, // mov x19, x1 (cfg) + 0x91006001, // add x1, x0, #0x18 (shared state) + 0xb40002b3, // cbz x19, ... (null cfg returns only the flag) +}; +constexpr std::array kSetFingerprint = { + 0xa9ba7bfd, // stp x29, x30, [sp, #-0x60]! + 0x910003fd, // mov x29, sp + 0xa90153f3, // stp x19, x20, [sp, #0x10] + 0xaa0103f3, // mov x19, x1 (cfg) + 0x91006001, // add x1, x0, #0x18 (shared state) + 0x910083f4, // add x20, sp, #0x20 (lock guard) + 0xaa1403e0, // mov x0, x20 +}; + +template +bool matches(const void* fn, const std::array& expected) { + std::uint32_t words[N]; + std::memcpy(words, fn, sizeof words); + return std::memcmp(words, expected.data(), sizeof words) == 0; +} + +using GetConfigFn = bool (*)(void* self, std::uint8_t* cfg); +using SetConfigFn = void (*)(void* self, const std::uint8_t* cfg); + +} // namespace + +PrivateCamera::PrivateCamera() { + vr::EVRInitError err = vr::VRInitError_None; + void* instance = vr::VR_GetGenericInterface(kInterface, &err); + if (!instance || err != vr::VRInitError_None) { + error_ = std::string("interface unavailable: ") + vr::VR_GetVRInitErrorAsEnglishDescription(err); + return; + } + auto** methods = *static_cast(instance); + if (!methods || !methods[kGetConfig] || !methods[kSetConfig]) { + error_ = "interface has no config methods"; + return; + } + if (!matches(methods[kGetConfig], kGetFingerprint) || !matches(methods[kSetConfig], kSetFingerprint)) { + error_ = "SteamVR's camera interface does not match the verified build (2.17.10); refusing to call it"; + return; + } + instance_ = instance; + methods_ = methods; +} + +std::optional PrivateCamera::get() { + if (!ok()) return std::nullopt; + std::array raw; + raw.fill(0xff); + reinterpret_cast(methods_[kGetConfig])(instance_, raw.data()); + const auto config = CameraConfig::from(raw.data()); + if (!config.valid()) { + error_ = "config read back non-boolean bytes: " + config.describe(); + return std::nullopt; + } + return config; +} + +bool PrivateCamera::set(const CameraConfig& config) { + if (!ok()) return false; + if (!config.valid()) { + error_ = "refusing to write non-boolean config"; + return false; + } + const auto raw = config.bytes(); + reinterpret_cast(methods_[kSetConfig])(instance_, raw.data()); + const auto back = get(); + if (!back || back->bytes() != raw) { + error_ = "config did not read back as written"; + return false; + } + return true; +} + +} // namespace autopass diff --git a/src/private_camera.hpp b/src/private_camera.hpp new file mode 100644 index 0000000..a2bbfb0 --- /dev/null +++ b/src/private_camera.hpp @@ -0,0 +1,39 @@ +#pragma once + +// Access to SteamVR's private IVRCameraPassthroughInternal_001 interface, +// limited to reading and writing the 5-byte camera config. Requires an +// initialised OpenVR client (VR_Init) in the calling process. +// +// The interface has no public header. Its layout was mapped from the +// headset's own vrclient.so (FINDINGS.md sections 9 and 11). Before any +// call, the code of the two methods is compared with the instructions +// seen in that binary; if SteamVR has changed them, nothing is called. + +#include "passthrough_state.hpp" + +#include +#include + +namespace autopass { + +class PrivateCamera { +public: + // Looks up the interface and verifies the method fingerprints. On + // failure, error() says why and every other call returns failure. + PrivateCamera(); + + bool ok() const { return methods_ != nullptr; } + const std::string& error() const { return error_; } + + std::optional get(); + // Writes the config and reads it back. Returns false (with error()) + // on any mismatch. + bool set(const CameraConfig& config); + +private: + void* instance_ = nullptr; + void** methods_ = nullptr; + std::string error_; +}; + +} // namespace autopass diff --git a/src/sensors.cpp b/src/sensors.cpp new file mode 100644 index 0000000..9b12f2a --- /dev/null +++ b/src/sensors.cpp @@ -0,0 +1,152 @@ +#include "sensors.hpp" + +#include +#include +#include + +namespace autopass { +namespace { + +std::uint64_t to_ms(double seconds) { return static_cast(std::llround(seconds * 1000.0)); } + +} // namespace + +double Ema::add(std::uint64_t now_ms, double value) { + if (!has_ || tau_s_ <= 0) { + value_ = value; + has_ = true; + } else { + const double dt = static_cast(now_ms > last_ms_ ? now_ms - last_ms_ : 0) / 1000.0; + value_ += (1.0 - std::exp(-dt / tau_s_)) * (value - value_); + } + last_ms_ = now_ms; + return value_; +} + +const char* to_string(Cause c) { + switch (c) { + case Cause::None: return "none"; + case Cause::EmittersOff: return "IR emitters off"; + case Cause::IrLight: return "IR camera sees light"; + case Cause::EmittersOnDim: return "IR emitters on, colour dim"; + case Cause::VerifyFailed: return "colour dark right after switching"; + } + return "?"; +} + +void EvidenceBuilder::on_switched(std::uint64_t now_ms, Source to, bool automatic, Cause cause) { + reset(); + if (to == Source::Color && automatic) { + verify_until_ms_ = now_ms + to_ms(config_.verify_s); + verify_cause_ = cause; + } else { + verify_until_ms_ = 0; + verify_cause_ = Cause::None; + } +} + +EvidenceResult EvidenceBuilder::ir_evidence(std::uint64_t now_ms, std::optional emitters_on, + std::optional ir_light) { + EvidenceResult r; + char buf[128]; + if (!*emitters_on && now_ms >= distrust_until_ms_) { + r.evidence = Evidence::Bright; + r.cause = Cause::EmittersOff; + r.why = "IR emitters off"; + return r; + } + r.why = *emitters_on ? "IR emitters on" : "IR emitters off, not trusted for a while after a failed colour check"; + if (now_ms < ir_light_lockout_until_ms_) { + r.evidence = Evidence::Neutral; + return r; + } + if (!ir_light) return r; // Unknown between samples + if (*ir_light >= config_.ir_min_light) { + if (!ir_lit_since_ms_) ir_lit_since_ms_ = now_ms; + } else { + ir_lit_since_ms_.reset(); + } + std::snprintf(buf, sizeof buf, "IR camera light %.2f", *ir_light); + r.why = buf; + if (ir_lit_since_ms_ && now_ms - *ir_lit_since_ms_ >= to_ms(config_.ir_light_hold_s)) { + r.evidence = Evidence::Bright; + r.cause = Cause::IrLight; + } else { + r.evidence = Evidence::Neutral; + } + return r; +} + +EvidenceResult EvidenceBuilder::evaluate(std::uint64_t now_ms, Source shown, std::optional emitters_on, + std::optional colour_light, std::optional ir_light) { + EvidenceResult r; + char buf[128]; + if (colour_light) colour_.add(now_ms, *colour_light); + // Without the emitter signal there is no reliable way back from IR, so + // nothing switches either way. + if (!emitters_on) { + r.evidence = Evidence::Neutral; + r.why = "IR emitter state unknown"; + return r; + } + + if (shown == Source::Ir) return ir_evidence(now_ms, emitters_on, ir_light); + + if (!colour_.has()) { + if (*emitters_on) { + r.evidence = Evidence::Dark; + r.cause = Cause::EmittersOnDim; + r.why = "IR emitters on, no colour reading"; + } + return r; + } + // No fresh colour sample this tick is "unknown", not "neutral": neutral + // would reset the policy's confirmation between samples. + if (!colour_light) return r; + const double v = colour_.value(); + + // XRService's emitters decide what is dark (FINDINGS.md 38, 39): its + // tracking cameras turn them on only at their longest exposure. With + // the emitters off, a low colour reading is a dim room (dusk), not a + // dark one, and the IR view would be no better. + if (!*emitters_on) { + std::snprintf(buf, sizeof buf, "IR emitters off, colour light %.2f", v); + r.evidence = Evidence::Neutral; + r.why = buf; + return r; + } + // Right after an automatic switch to colour, emitters on and colour + // clearly dark: that switch was a mistake. Go back at once and stop + // believing whatever misled it for a while (doubling each time). + if (now_ms < verify_until_ms_ && v < config_.colour_dark_light) { + std::snprintf(buf, sizeof buf, "colour light %.2f right after switching (was: %s)", v, to_string(verify_cause_)); + if (verify_cause_ == Cause::EmittersOff) { + const double s = next_distrust_s_ > 0 ? next_distrust_s_ : config_.emitters_off_distrust_s; + distrust_until_ms_ = now_ms + to_ms(s); + next_distrust_s_ = std::min(s * 2, config_.emitters_off_distrust_max_s); + } else if (verify_cause_ == Cause::IrLight) { + const double s = next_ir_light_lockout_s_ > 0 ? next_ir_light_lockout_s_ : config_.ir_light_lockout_s; + ir_light_lockout_until_ms_ = now_ms + to_ms(s); + next_ir_light_lockout_s_ = std::min(s * 2, config_.ir_light_lockout_max_s); + } + verify_cause_ = Cause::None; // count this mistake once + r.evidence = Evidence::Dark; + r.cause = Cause::VerifyFailed; + r.urgent = true; + r.why = buf; + return r; + } + if (v < config_.colour_min_light) { + std::snprintf(buf, sizeof buf, "IR emitters on, colour light %.2f", v); + r.evidence = Evidence::Dark; + r.cause = Cause::EmittersOnDim; + r.why = buf; + return r; + } + std::snprintf(buf, sizeof buf, "colour light %.2f", v); + r.evidence = Evidence::Neutral; + r.why = buf; + return r; +} + +} // namespace autopass diff --git a/src/sensors.hpp b/src/sensors.hpp new file mode 100644 index 0000000..718b628 --- /dev/null +++ b/src/sensors.hpp @@ -0,0 +1,137 @@ +#pragma once + +// Turn raw readings into Evidence for LightPolicy (FINDINGS.md 29, 34, 38-43). +// +// Three signals, none of them needing an image: +// - XRService's IR emitter state (from its log, xrservice_log.hpp). Its +// tracking cameras' auto-exposure turns the emitters on only at their +// longest exposure with gain to spare, within ~0.1 s of darkness, and off +// only after 5 s of comfortable exposure (often much later, as the +// emitters light the room themselves). So "on" means dark, and "off" +// means the room has light, even if a dim one. +// - The colour camera's light value g4 (shared memory, only while colour +// is shown). Low in a dark room, but also at dusk, when the room is +// still fine to see in (2026-10-01 18:41), so on its own it never +// switches. +// - The IR camera's light value (shared memory, the same record field as +// g4, only while IR is shown). 0 in the dark even with the emitters on, +// ~0.2-0.35 within half a second of a light coming on, also while +// tracking is lost (FINDINGS.md 41). +// +// Colour shown, Dark when the emitters are on and: +// - g4 < `colour_min_light`; +// - within `verify_s` of an automatic switch to colour, g4 < +// `colour_dark_light`: that switch was a mistake. Urgent (skips the +// cooldown), and what misled it is not believed for a while: +// "emitters off" for `emitters_off_distrust_s`, the IR light value for +// `ir_light_lockout_s`, each doubling per repeat up to its maximum. +// With the emitters off nothing in colour switches to IR. The cost: if the +// emitters are fooled off in a dark room (a wall centimetres away, +// 00:13:33), colour stays until they come back on, which they do as soon +// as the wall is no longer lit by them. +// +// IR shown, Bright when either: +// - the emitters are off (and not distrusted). XRService keeps them on for +// at least 5 s after the light returns, often longer; +// - the IR light value is at least `ir_min_light` for `ir_light_hold_s`: +// what normally brings colour back, about a second after the light. A +// wall lit by the emitters from close by read 0.10 (22:04:09), below the +// threshold; if one fools it anyway, the colour check above undoes it. +// +// Without the emitter signal (for example a SteamVR update changed the log +// line) nothing switches in either direction. + +#include "light_policy.hpp" + +#include +#include +#include + +namespace autopass { + +// Exponential moving average over irregular samples. +class Ema { +public: + explicit Ema(double time_constant_s = 1.0) : tau_s_(time_constant_s) {} + double add(std::uint64_t now_ms, double value); + void reset() { has_ = false; } + bool has() const { return has_; } + double value() const { return value_; } + +private: + double tau_s_; + bool has_ = false; + double value_ = 0; + std::uint64_t last_ms_ = 0; +}; + +struct SensorConfig { + double colour_min_light = 0.6; // with emitters on: dim for colour below this + double colour_dark_light = 0.35; // clearly dark: a switch to colour was a mistake + double verify_s = 3.0; // check colour right after switching to it + double ir_min_light = 0.15; // IR shown: the IR camera's light value says lit... + double ir_light_hold_s = 0.0; // ...for this long (on top of the policy's confirmation) + double ir_light_lockout_s = 60.0; // ignore the IR light value after it misled us, doubling... + double ir_light_lockout_max_s = 600.0; + double emitters_off_distrust_s = 60.0; // ignore "emitters off" after it misled us, doubling... + double emitters_off_distrust_max_s = 600.0; + // Extra smoothing of the colour light value. 0: the camera already + // smooths it (0.89 to 0.44 over ~1.5 s when the light goes out). + double smoothing_s = 0.0; +}; + +// Which signal produced a Bright or Dark. +enum class Cause { None, EmittersOff, IrLight, EmittersOnDim, VerifyFailed }; +const char* to_string(Cause cause); + +struct EvidenceResult { + Evidence evidence = Evidence::Unknown; + Cause cause = Cause::None; + bool urgent = false; // may skip the cooldown (a mistaken switch) + std::string why; // short description for logs +}; + +// Combines the signals into one Evidence for the source being shown. +class EvidenceBuilder { +public: + explicit EvidenceBuilder(SensorConfig config = {}) + : config_(config), colour_(config.smoothing_s) {} + + // Forget smoothed readings (passthrough hidden, source changed). + void reset() { + colour_.reset(); + ir_lit_since_ms_.reset(); + } + + // Tell the builder about a switch that happened, so it can verify an + // automatic switch to colour and learn from a mistaken one. + void on_switched(std::uint64_t now_ms, Source to, bool automatic, Cause cause); + + // `emitters_on`: known emitter state, or nullopt. `colour_light` / + // `ir_light`: fresh readings for this tick, if any. + EvidenceResult evaluate(std::uint64_t now_ms, Source shown, std::optional emitters_on, + std::optional colour_light, std::optional ir_light = std::nullopt); + + bool distrusting_emitters_off(std::uint64_t now_ms) const { return now_ms < distrust_until_ms_; } + // Whether autopassd should poll the IR light value (IR shown, and it could + // matter right now). + bool wants_ir_light(std::uint64_t now_ms, std::optional emitters_on) const { + return now_ms >= ir_light_lockout_until_ms_ && (emitters_on != false || now_ms < distrust_until_ms_); + } + +private: + EvidenceResult ir_evidence(std::uint64_t now_ms, std::optional emitters_on, + std::optional ir_light); + + SensorConfig config_; + Ema colour_; + std::optional ir_lit_since_ms_; + std::uint64_t ir_light_lockout_until_ms_ = 0; + double next_ir_light_lockout_s_ = 0; // 0: use ir_light_lockout_s + std::uint64_t verify_until_ms_ = 0; + Cause verify_cause_ = Cause::None; + std::uint64_t distrust_until_ms_ = 0; + double next_distrust_s_ = 0; // 0: use emitters_off_distrust_s +}; + +} // namespace autopass diff --git a/src/xrservice_log.cpp b/src/xrservice_log.cpp new file mode 100644 index 0000000..f7b19a1 --- /dev/null +++ b/src/xrservice_log.cpp @@ -0,0 +1,190 @@ +#include "xrservice_log.hpp" + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace autopass { +namespace { + +constexpr const char* kLink = "xrservice.txt"; +// The session log is a few hundred KB; read at most this much of its tail +// when looking for the latest state. +constexpr long long kMaxInitialRead = 16LL * 1024 * 1024; + +} // namespace + +std::string XrServiceLog::default_logs_dir() { + const char* home = std::getenv("HOME"); + return std::string(home && *home ? home : "/tmp") + "/.local/share/Steam/logs"; +} + +XrServiceLog::XrServiceLog(std::string logs_dir) : logs_dir_(std::move(logs_dir)) {} + +XrServiceLog::~XrServiceLog() { stop(); } + +bool apply_xrservice_line(const std::string& line, XrState* s) { + const auto has = [&line](const char* text) { return line.find(text) != std::string::npos; }; + if (has("[DeckardCaptureSource] Passthrough cameras resumed")) s->passthrough = true; + else if (has("[DeckardCaptureSource] Passthrough cameras paused")) s->passthrough = false; + else if (has("[UserPresence] Received onEnterStandby")) s->standby = true; + else if (has("[UserPresence] Received onLeaveStandby")) s->standby = false; + else if (has("[IREmitters] IR Emitters Turned On")) s->emitters = true; + else if (has("[IREmitters] IR Emitters Turned Off")) s->emitters = false; + else if (has("[IREmitters] IR emitters mode changed to Auto")) s->emitters = false; + else if (has("Transition to IMUFallback")) s->tracking_lost = true; + else if (has("[IMUFallback] Disabled")) s->tracking_lost = false; + else return false; + return true; +} + +void apply_xrservice_text(const std::string& text, XrState* state) { + std::size_t start = 0; + while (start < text.size()) { + std::size_t end = text.find('\n', start); + if (end == std::string::npos) end = text.size(); + apply_xrservice_line(text.substr(start, end - start), state); + start = end + 1; + } +} + +long long XrServiceLog::parse_session_start(const std::string& path) { + int y, mo, d, h, mi, s; + const auto slash = path.rfind("/XRService-"); + if (slash == std::string::npos || slash < 11) return 0; + const auto dir = path.rfind("XRService-", slash - 1); + if (dir == std::string::npos) return 0; + if (std::sscanf(path.c_str() + dir, "XRService-%d.%d.%d/XRService-%d-%d-%d.log", &y, &mo, &d, &h, &mi, &s) != 6) + return 0; + std::tm tm{}; + tm.tm_year = y - 1900; + tm.tm_mon = mo - 1; + tm.tm_mday = d; + tm.tm_hour = h; + tm.tm_min = mi; + tm.tm_sec = s; + tm.tm_isdst = -1; + return static_cast(std::mktime(&tm)); +} + +void XrServiceLog::stop() { + if (file_fd_ >= 0) ::close(file_fd_); + if (inotify_fd_ >= 0) ::close(inotify_fd_); + file_fd_ = inotify_fd_ = -1; + dir_watch_ = file_watch_ = -1; + partial_.clear(); +} + +bool XrServiceLog::start() { + stop(); + inotify_fd_ = ::inotify_init1(IN_NONBLOCK | IN_CLOEXEC); + if (inotify_fd_ < 0) { + error_ = std::string("inotify: ") + std::strerror(errno); + return false; + } + dir_watch_ = ::inotify_add_watch(inotify_fd_, logs_dir_.c_str(), IN_CREATE | IN_MOVED_TO); + if (!open_log()) { + stop(); + return false; + } + return true; +} + +bool XrServiceLog::open_log() { + if (file_fd_ >= 0) ::close(file_fd_); + if (file_watch_ >= 0) ::inotify_rm_watch(inotify_fd_, file_watch_); + file_fd_ = file_watch_ = -1; + partial_.clear(); + + char resolved[PATH_MAX]; + const std::string link = logs_dir_ + "/" + kLink; + if (!::realpath(link.c_str(), resolved)) { + error_ = "cannot resolve " + link + ": " + std::strerror(errno); + return false; + } + // A different session log than the last one seen (even across a + // stop/start) means XRService restarted in between. + if (!log_path_.empty() && log_path_ != resolved) restarted_ = true; + log_path_ = resolved; + session_start_ = parse_session_start(log_path_); + file_fd_ = ::open(log_path_.c_str(), O_RDONLY | O_CLOEXEC); + if (file_fd_ < 0) { + error_ = "cannot open " + log_path_ + ": " + std::strerror(errno); + return false; + } + file_watch_ = ::inotify_add_watch(inotify_fd_, log_path_.c_str(), IN_MODIFY); + + // Initial state from the tail of the log. + struct stat st{}; + ::fstat(file_fd_, &st); + const long long size = st.st_size; + const long long from = size > kMaxInitialRead ? size - kMaxInitialRead : 0; + std::string text(static_cast(size - from), '\0'); + long long got = 0; + while (got < size - from) { + const auto n = ::pread(file_fd_, text.data() + got, static_cast(size - from - got), from + got); + if (n <= 0) break; + got += n; + } + text.resize(static_cast(got)); + // Everything we track comes from this session's log. + state_ = {}; + apply_xrservice_text(text, &state_); + offset_ = from + got; + error_.clear(); + return true; +} + +void XrServiceLog::read_new_bytes() { + struct stat st{}; + if (file_fd_ < 0 || ::fstat(file_fd_, &st) != 0) return; + if (st.st_size < offset_) offset_ = 0; // truncated + std::vector buf(64 * 1024); + for (;;) { + const auto n = ::pread(file_fd_, buf.data(), buf.size(), offset_); + if (n <= 0) break; + offset_ += n; + partial_.append(buf.data(), static_cast(n)); + std::size_t start = 0, nl; + while ((nl = partial_.find('\n', start)) != std::string::npos) { + apply_xrservice_line(partial_.substr(start, nl - start), &state_); + start = nl + 1; + } + partial_.erase(0, start); + } +} + +bool XrServiceLog::update() { + if (inotify_fd_ < 0) return false; + const XrState before = state_; + bool file_changed = false, relink = false; + alignas(inotify_event) char buf[4096]; + for (;;) { + const auto n = ::read(inotify_fd_, buf, sizeof buf); + if (n <= 0) break; + for (char* p = buf; p < buf + n;) { + const auto* e = reinterpret_cast(p); + if (e->wd == file_watch_) file_changed = true; + if (e->wd == dir_watch_ && e->len && std::strcmp(e->name, kLink) == 0) relink = true; + p += sizeof(inotify_event) + e->len; + } + } + if (relink) { + char resolved[PATH_MAX]; + const std::string link = logs_dir_ + "/" + kLink; + if (::realpath(link.c_str(), resolved) && log_path_ != resolved) open_log(); + } + if (file_changed) read_new_bytes(); + return state_.passthrough != before.passthrough || state_.standby != before.standby || + state_.emitters != before.emitters || state_.tracking_lost != before.tracking_lost; +} + +} // namespace autopass diff --git a/src/xrservice_log.hpp b/src/xrservice_log.hpp new file mode 100644 index 0000000..9621f97 --- /dev/null +++ b/src/xrservice_log.hpp @@ -0,0 +1,86 @@ +#pragma once + +// Follows XRService's session log for the few facts autopassd needs +// (FINDINGS.md sections 29 and 33). Lines, as logged on the headset: +// +// [DeckardCaptureSource] Passthrough cameras resumed | paused +// passthrough is being shown | not shown (every toggle, and standby) +// [UserPresence] Received onEnterStandby | onLeaveStandby +// [IREmitters] IR Emitters Turned On | Off +// XRService's own "too dark for the cameras" judgement +// Transition to IMUFallback | [IMUFallback] Disabled +// visual tracking lost | back +// +// ~/.local/share/Steam/logs/xrservice.txt is a symlink to the current +// session log. start() reads the log once for the latest state, then +// inotify delivers only appended bytes. XRService writes nothing while the +// headset is idle, so an idle autopassd is never woken. A new session log (the +// symlink changes, also across stop/start) means XRService restarted. + +#include +#include + +namespace autopass { + +struct XrState { + std::optional passthrough; // cameras resumed (shown) / paused + std::optional standby; // headset in standby + std::optional emitters; // IR emitters on + std::optional tracking_lost; // IMU fallback active +}; + +// Applies one log line to `state`. Returns true if it was a line we track. +bool apply_xrservice_line(const std::string& line, XrState* state); +// Applies every complete or trailing line of `text` in order. +void apply_xrservice_text(const std::string& text, XrState* state); + +class XrServiceLog { +public: + explicit XrServiceLog(std::string logs_dir = default_logs_dir()); + ~XrServiceLog(); + XrServiceLog(const XrServiceLog&) = delete; + XrServiceLog& operator=(const XrServiceLog&) = delete; + + static std::string default_logs_dir(); + + bool start(); // false with error() if the log cannot be opened + void stop(); + bool running() const { return inotify_fd_ >= 0; } + int fd() const { return inotify_fd_; } + const std::string& error() const { return error_; } + + // Handles pending inotify events without blocking. Returns true if any + // tracked state changed. + bool update(); + const XrState& state() const { return state_; } + + // True once after XRService started a new session log since the one + // seen before, including across stop()/start(). Cleared by reading. + bool take_restarted() { + const bool r = restarted_; + restarted_ = false; + return r; + } + // Session start (Unix time) parsed from the log path, or 0. + long long session_start() const { return session_start_; } + static long long parse_session_start(const std::string& path); + +private: + bool open_log(); + void read_new_bytes(); + + std::string logs_dir_; + std::string log_path_; + int inotify_fd_ = -1; + int dir_watch_ = -1; + int file_watch_ = -1; + int file_fd_ = -1; + long long offset_ = 0; + std::string partial_; + XrState state_; + bool restarted_ = false; + long long session_start_ = 0; + std::string error_; +}; + +} // namespace autopass diff --git a/tests/fixtures/state_dark.bin b/tests/fixtures/state_dark.bin new file mode 100644 index 0000000..e443648 Binary files /dev/null and b/tests/fixtures/state_dark.bin differ diff --git a/tests/fixtures/state_lit.bin b/tests/fixtures/state_lit.bin new file mode 100644 index 0000000..522b40a Binary files /dev/null and b/tests/fixtures/state_lit.bin differ diff --git a/tests/fixtures/state_mono.bin b/tests/fixtures/state_mono.bin new file mode 100644 index 0000000..01f4a81 Binary files /dev/null and b/tests/fixtures/state_mono.bin differ diff --git a/tests/fixtures/state_mono_dark.bin b/tests/fixtures/state_mono_dark.bin new file mode 100644 index 0000000..0e5db42 Binary files /dev/null and b/tests/fixtures/state_mono_dark.bin differ diff --git a/tests/fixtures/state_mono_lit.bin b/tests/fixtures/state_mono_lit.bin new file mode 100644 index 0000000..edae8ca Binary files /dev/null and b/tests/fixtures/state_mono_lit.bin differ diff --git a/tests/test_daemon_config.cpp b/tests/test_daemon_config.cpp new file mode 100644 index 0000000..dd1adc7 --- /dev/null +++ b/tests/test_daemon_config.cpp @@ -0,0 +1,53 @@ +// Host-side tests for the autopass.conf parser. + +#include "daemon_config.hpp" + +#include + +using namespace autopass; + +namespace { +int failures = 0; +#define CHECK(expr) do { if (!(expr)) { std::printf("FAIL line %d: %s\n", __LINE__, #expr); ++failures; } } while (0) +} // namespace + +int main() { + DaemonConfig c; + CHECK(parse_daemon_config("", &c).empty()); + CHECK(c.sensors.colour_min_light == 0.6 && c.sensors.colour_dark_light == 0.35 && !c.observe_only); + CHECK(c.policy.confirm_s == 0.5 && c.policy.urgent_confirm_s == 0.25 && c.policy.cooldown_base_s == 2.0); + + CHECK(parse_daemon_config("# comment\n\n colour_min_light = 0.5 # inline\nverify_s=4\nobserve_only = true\n", &c).empty()); + CHECK(c.sensors.colour_min_light == 0.5 && c.sensors.verify_s == 4.0 && c.observe_only); + CHECK(parse_daemon_config("cooldown_base_s = 1\ncooldown_max_s = 20\n", &c).empty()); + CHECK(c.policy.cooldown_base_s == 1.0 && c.policy.cooldown_max_s == 20.0); + + // Errors leave the config untouched and name the line. + DaemonConfig d; + const auto e1 = parse_daemon_config("confirm_s = 2\nverfy_s = 5\n", &d); + CHECK(e1.find("line 2") != std::string::npos && e1.find("unknown key") != std::string::npos); + CHECK(d.policy.confirm_s == 0.5); + // Removed keys are rejected, not silently ignored. + CHECK(parse_daemon_config("ir_max_grain = 7\n", &d).find("unknown key") != std::string::npos); + CHECK(parse_daemon_config("grain_lockout_s = 60\n", &d).find("unknown key") != std::string::npos); + CHECK(parse_daemon_config("ir_min_light = 0.2\nir_light_hold_s = 0.5\nconfirm_to_colour_s = 0.5\n", &d).empty()); + CHECK(d.policy.confirm_to_colour_s == 0.5); + CHECK(d.sensors.ir_min_light == 0.2 && d.sensors.ir_light_hold_s == 0.5); + CHECK(!parse_daemon_config("ir_min_light = 0\n", &d).empty()); + CHECK(!parse_daemon_config("ir_light_lockout_max_s = 10\n", &d).empty()); // below ir_light_lockout_s + // colour_dark_light above colour_min_light makes no sense. + CHECK(!parse_daemon_config("colour_dark_light = 0.7\n", &d).empty()); + CHECK(parse_daemon_config("cooldown_base_s = ten\n", &d).find("not a number") != std::string::npos); + CHECK(parse_daemon_config("cooldown_base_s = 10s\n", &d).find("not a number") != std::string::npos); + CHECK(parse_daemon_config("min_dwell_s = 10\n", &d).find("unknown key") != std::string::npos); + CHECK(!parse_daemon_config("cooldown_base_s = 40\n", &d).empty()); // above cooldown_max_s + CHECK(parse_daemon_config("observe_only = yes\n", &d).find("true or false") != std::string::npos); + CHECK(parse_daemon_config("just words\n", &d).find("key = value") != std::string::npos); + // Values that parse but fail policy validation are rejected too. + CHECK(!parse_daemon_config("colour_min_light = 2\n", &d).empty()); + CHECK(d.sensors.colour_min_light == 0.6); + + if (failures) { std::printf("%d check(s) failed\n", failures); return 1; } + std::printf("all autopassd config tests passed\n"); + return 0; +} diff --git a/tests/test_light_policy.cpp b/tests/test_light_policy.cpp new file mode 100644 index 0000000..5a34445 --- /dev/null +++ b/tests/test_light_policy.cpp @@ -0,0 +1,273 @@ +// Host-side tests for LightPolicy: evidence in, switching decisions out. +// Sensor behaviour is tested in test_sensors.cpp. + +#include "light_policy.hpp" + +#include +#include +#include + +using namespace autopass; + +namespace { + +int failures = 0; + +void check(bool ok, const char* what, int line) { + if (!ok) { + std::printf("FAIL line %d: %s\n", line, what); + ++failures; + } +} +#define CHECK(expr) check((expr), #expr, __LINE__) + +// Feeds evidence(t) every `step_ms` from `from_ms` to `to_ms`; returns the +// switch times. +std::vector run(LightPolicy& p, std::uint64_t from_ms, std::uint64_t to_ms, + const std::function& evidence, std::uint64_t step_ms = 250) { + std::vector switches; + for (std::uint64_t t = from_ms; t <= to_ms; t += step_ms) + if (p.update(t, evidence(t)).changed) switches.push_back(t); + return switches; +} + +// Evidence for the source currently shown in a room that is dark when +// `dark(t)`: Dark while colour is shown in the dark, Bright while IR is +// shown in the light, otherwise Neutral. +std::function room(LightPolicy& p, const std::function& dark) { + return [&p, dark](std::uint64_t t) { + const bool d = dark(t); + if (p.source() == Source::Color) return d ? Evidence::Dark : Evidence::Neutral; + return d ? Evidence::Neutral : Evidence::Bright; + }; +} + +// The timing rules are tested with a 0.5 s confirmation; the defaults are +// covered by test_sensors' end-to-end replays. +PolicyConfig cfg() { + PolicyConfig c; + c.confirm_s = 0.5; + return c; +} + +void test_validate_and_strings() { + CHECK(validate(PolicyConfig{}).empty()); + PolicyConfig c; + c.confirm_s = -1; + CHECK(!validate(c).empty()); + c = {}; + c.stale_s = 0; + CHECK(!validate(c).empty()); + c = {}; + c.cooldown_max_s = 1; // below the 2 s base + CHECK(!validate(c).empty()); + Mode m; + CHECK(parse_mode("auto", &m) && m == Mode::Auto); + CHECK(parse_mode("colour", &m) && m == Mode::ForceColor); + CHECK(parse_mode("color", &m) && m == Mode::ForceColor); + CHECK(parse_mode("ir", &m) && m == Mode::ForceIr); + CHECK(!parse_mode("bright", &m)); +} + +void test_first_switch_is_fast() { + // Lights off in a room: the switch comes after confirm_s (0.5 s), not + // after any cooldown. + LightPolicy p(cfg(), Source::Color); + const auto s = run(p, 0, 20000, room(p, [](std::uint64_t t) { return t >= 5000; })); + CHECK(s.size() == 1); + if (!s.empty()) CHECK(s[0] >= 5500 && s[0] <= 5750); +} + +void test_normal_off_then_on_is_fast_both_ways() { + LightPolicy p(cfg(), Source::Color); + const auto s = run(p, 0, 60000, room(p, [](std::uint64_t t) { return t >= 5000 && t < 30000; })); + CHECK(s.size() == 2); + if (s.size() == 2) { + CHECK(s[0] <= 5750); + CHECK(s[1] >= 30000 && s[1] <= 32250); // settle 1.5 s already long past; confirm 0.5 s + } +} + +void test_quick_reversal_waits_for_base_cooldown() { + // Light off, then straight back on after 1 s: the reversal waits for + // the 2 s cooldown, not longer. + LightPolicy p(cfg(), Source::Color); + const auto s = run(p, 0, 20000, room(p, [](std::uint64_t t) { return t >= 5000 && t < 6000; })); + CHECK(s.size() == 2); + if (s.size() == 2) CHECK(s[1] - s[0] >= 2000 && s[1] - s[0] <= 2750); +} + +void test_flickering_light_escalates_cooldown() { + // A light flickering on/off every second for two minutes. Without the + // cooldown this would switch ~120 times; with it, the gaps double. + LightPolicy p(cfg(), Source::Color); + const auto s = run(p, 0, 120000, room(p, [](std::uint64_t t) { return (t / 1000) % 2 == 1; })); + CHECK(s.size() >= 3 && s.size() <= 10); + for (std::size_t i = 2; i < s.size(); ++i) CHECK(s[i] - s[i - 1] >= s[i - 1] - s[i - 2] - 1000); + // Gaps reach the 30 s cap and never exceed cap + one flicker period. + if (s.size() >= 2) CHECK(s.back() - s[s.size() - 2] <= 32000); + std::printf("flicker: %zu switches in 120 s:", s.size()); + for (auto t : s) std::printf(" %.1f", t / 1000.0); + std::printf("\n"); +} + +void test_lights_toggled_by_hand_stay_fast() { + // Someone switching the lights every 5 s for a minute (2026-10-01 18:55: + // the old rule made each switch slower than the last). Every change is + // followed within confirm time; the cooldown never grows. + LightPolicy p(cfg(), Source::Color); + const auto s = run(p, 0, 62000, room(p, [](std::uint64_t t) { return t >= 2000 && (t - 2000) / 5000 % 2 == 0; })); + CHECK(s.size() == 12); + for (std::size_t i = 0; i < s.size(); ++i) { + const std::uint64_t change = 2000 + 5000 * i; + CHECK(s[i] >= change && s[i] <= change + 1250); + } + CHECK(p.cooldown_ms() == 2000); +} + +void test_cooldown_resets_after_quiet_period() { + LightPolicy p(cfg(), Source::Color); + // Four quick reversals escalate the cooldown... + run(p, 0, 30000, room(p, [](std::uint64_t t) { return (t / 1000) % 2 == 1; })); + CHECK(p.cooldown_ms() > 2000); + // ...then a quiet minute in steady light, and a normal off/on is fast + // again. + const bool steady_dark = p.source() == Source::Ir; // keep the light matching what is shown + run(p, 30250, 100000, room(p, [steady_dark](std::uint64_t) { return steady_dark; })); + const bool start_dark = p.source() == Source::Ir; + const auto s = run(p, 100250, 110000, room(p, [start_dark](std::uint64_t) { return !start_dark; })); + CHECK(s.size() == 1); + if (!s.empty()) CHECK(s[0] <= 101000); +} + +void test_brief_evidence_ignored() { + // Dark evidence for less than confirm_s (a hand passing) never switches. + LightPolicy p(cfg(), Source::Color); + CHECK(run(p, 0, 30000, room(p, [](std::uint64_t t) { return t >= 5000 && t < 5250; })).empty()); + // Evidence flickering faster than confirm_s never switches either. + LightPolicy q(cfg(), Source::Color); + int changes = 0; + for (std::uint64_t t = 0; t < 60000; t += 250) + if (q.update(t, (t / 250) % 2 ? Evidence::Dark : Evidence::Neutral).changed) ++changes; + CHECK(changes == 0); +} + +void test_unknown_between_samples_keeps_confirming() { + // A sensor sampling slower than the loop produces Unknown in between; + // confirmation keeps accumulating across those ticks. + LightPolicy p(cfg(), Source::Color); + const auto s = run(p, 0, 5000, [](std::uint64_t t) { return (t / 250) % 2 ? Evidence::Unknown : Evidence::Dark; }); + CHECK(s.size() == 1); +} + +void test_stale_readings_hold_and_reset() { + LightPolicy p(cfg(), Source::Color); + run(p, 0, 2000, [](auto) { return Evidence::Neutral; }); + // Dark for 0.25 s (not yet confirmed), then readings stop for 5 s. + CHECK(run(p, 2250, 2500, [](auto) { return Evidence::Dark; }).empty()); + CHECK(run(p, 2750, 7750, [](auto) { return Evidence::Unknown; }).empty()); + CHECK(p.source() == Source::Color); + // Dark again: confirmation restarts from zero. + const auto s = run(p, 8000, 12000, [](auto) { return Evidence::Dark; }); + CHECK(s.size() == 1); + if (!s.empty()) CHECK(s[0] >= 8500); +} + +void test_manual_override() { + LightPolicy p(cfg(), Source::Color); + auto d = p.set_mode(1000, Mode::ForceIr); + CHECK(d.changed && d.source == Source::Ir); + CHECK(run(p, 1250, 60000, [](auto) { return Evidence::Bright; }).empty()); + d = p.set_mode(60000, Mode::Auto); + CHECK(!d.changed && d.source == Source::Ir); + d = p.set_mode(61000, Mode::ForceColor); + CHECK(d.changed && d.source == Source::Color); + CHECK(!p.set_mode(62000, Mode::ForceColor).changed); +} + +void test_manual_switch_does_not_delay_auto() { + // Seen live 23:27: force IR in bright light, back to auto; colour + // returns after settle + confirm, with no cooldown. A long cooldown + // makes the difference visible past the settle window. + PolicyConfig c = cfg(); + c.cooldown_base_s = 10; + LightPolicy p(c, Source::Color); + CHECK(p.set_mode(5000, Mode::ForceIr).changed); + p.set_mode(6000, Mode::Auto); + const auto s = run(p, 6250, 20000, [](auto) { return Evidence::Bright; }); + CHECK(s.size() == 1); + if (!s.empty()) CHECK(s[0] <= 7250); +} + +void test_settle_ignores_transition_evidence() { + LightPolicy p(cfg(), Source::Color); + CHECK(p.set_mode(1000, Mode::ForceIr).changed); + p.set_mode(1100, Mode::Auto); + for (std::uint64_t t = 1250; t < 2500; t += 250) + CHECK(p.update(t, Evidence::Bright).reason == "settling after switch"); + CHECK(p.update(2600, Evidence::Bright).reason == "confirming"); +} + +void test_adopt_is_not_a_switch() { + LightPolicy p(cfg(), Source::Color); + p.adopt(Source::Ir); + CHECK(p.source() == Source::Ir && !p.settling(0)); + const auto s = run(p, 0, 5000, [](auto) { return Evidence::Bright; }); + CHECK(s.size() == 1); + if (!s.empty()) CHECK(s[0] <= 750); +} + +void test_urgent_skips_cooldown_with_short_confirm() { + // A switch to colour turned out wrong (it is dark): undoing it must not + // wait for the cooldown, and needs only urgent_confirm_s (0.25 s). + PolicyConfig c = cfg(); + c.cooldown_base_s = 10; + LightPolicy p(c, Source::Ir); + std::uint64_t to_colour = 0, back = 0; + for (std::uint64_t t = 0; t < 5000 && !to_colour; t += 250) + if (p.update(t, Evidence::Bright).changed) to_colour = t; + CHECK(to_colour > 0); + for (std::uint64_t t = to_colour + 50; t < 20000 && !back; t += 50) + if (p.update(t, Evidence::Dark, "", true).changed) back = t; + CHECK(back && back - to_colour <= 650); // settle 0.3 s + urgent confirm 0.25 s + // Non-urgent evidence afterwards respects the escalated cooldown (the + // quick reversal doubled it to 20 s). + CHECK(p.cooldown_ms() == 20000); + const auto s2 = run(p, back + 250, back + 40000, [](auto) { return Evidence::Bright; }); + CHECK(s2.size() == 1); + if (!s2.empty()) CHECK(s2[0] - back >= 20000); +} + +void test_time_going_backwards_is_safe() { + LightPolicy p(cfg(), Source::Color); + run(p, 0, 5000, [](auto) { return Evidence::Neutral; }); + CHECK(!p.update(4000, Evidence::Neutral).changed); + CHECK(!p.update(100, Evidence::Dark).changed); +} + +} // namespace + +int main() { + test_validate_and_strings(); + test_first_switch_is_fast(); + test_normal_off_then_on_is_fast_both_ways(); + test_quick_reversal_waits_for_base_cooldown(); + test_flickering_light_escalates_cooldown(); + test_lights_toggled_by_hand_stay_fast(); + test_cooldown_resets_after_quiet_period(); + test_brief_evidence_ignored(); + test_unknown_between_samples_keeps_confirming(); + test_stale_readings_hold_and_reset(); + test_manual_override(); + test_manual_switch_does_not_delay_auto(); + test_settle_ignores_transition_evidence(); + test_adopt_is_not_a_switch(); + test_urgent_skips_cooldown_with_short_confirm(); + test_time_going_backwards_is_safe(); + if (failures) { + std::printf("%d check(s) failed\n", failures); + return 1; + } + std::printf("all light policy tests passed\n"); + return 0; +} diff --git a/tests/test_passthrough_state.cpp b/tests/test_passthrough_state.cpp new file mode 100644 index 0000000..1b47c5f --- /dev/null +++ b/tests/test_passthrough_state.cpp @@ -0,0 +1,83 @@ +// Parses real VR_CameraPassthroughState snapshots captured on the headset +// (tests/fixtures, taken from the 21:32 live capture: one lit, one dark). + +#include "passthrough_state.hpp" + +#include +#include +#include +#include +#include + +using namespace autopass; + +namespace { + +int failures = 0; +#define CHECK(expr) do { if (!(expr)) { std::printf("FAIL line %d: %s\n", __LINE__, #expr); ++failures; } } while (0) + +std::vector load(const char* path) { + std::ifstream f(path, std::ios::binary); + return {std::istreambuf_iterator(f), {}}; +} + +} // namespace + +int main() { + const auto lit = load("tests/fixtures/state_lit.bin"); + const auto dark = load("tests/fixtures/state_dark.bin"); + CHECK(lit.size() == PassthroughState::kSize); + CHECK(dark.size() == PassthroughState::kSize); + if (failures) return 1; + + const auto a = PassthroughState::parse(lit.data()); + CHECK(a.config.valid()); + CHECK(a.config.enabled == 1 && a.config.rgb == 1); + CHECK(a.colour.has_value()); + if (a.colour) CHECK(a.colour->light > 0.7f && a.colour->timestamp > 0); + + const auto b = PassthroughState::parse(dark.data()); + CHECK(b.config.rgb == 1); + CHECK(b.colour.has_value()); + if (b.colour) CHECK(b.colour->light < 0.05f); + + // Liveness timestamps: in RGB mode the colour stream is the newest; + // in mono mode (capture 2, t=60 s) the mono stream is newer and the + // colour records are frozen at the moment of the switch. + CHECK(a.colour_timestamp > 0 && a.colour_timestamp > a.mono_timestamp); + if (a.colour) CHECK(a.colour_timestamp >= a.colour->timestamp); + const auto mono = load("tests/fixtures/state_mono.bin"); + CHECK(mono.size() == PassthroughState::kSize); + if (mono.size() == PassthroughState::kSize) { + const auto m = PassthroughState::parse(mono.data()); + CHECK(m.config.enabled == 1 && m.config.rgb == 0); + CHECK(m.mono_timestamp > m.colour_timestamp); + CHECK(m.mono_timestamp - m.colour_timestamp > 10.0); + } + + // The IR camera's light value (FINDINGS.md 41), captured with IR shown + // in room B 2026-10-01: light on 0.219, light off (emitters on) 0. + for (const auto& [file, lit] : {std::pair{"tests/fixtures/state_mono_lit.bin", true}, + std::pair{"tests/fixtures/state_mono_dark.bin", false}}) { + const auto raw = load(file); + CHECK(raw.size() == PassthroughState::kSize); + if (raw.size() != PassthroughState::kSize) continue; + const auto m = PassthroughState::parse(raw.data()); + CHECK(m.config.rgb == 0 && m.mono_light.has_value()); + if (m.mono_light) CHECK(lit ? *m.mono_light > 0.2f && *m.mono_light < 0.25f : *m.mono_light == 0.0f); + } + + // An all-zero buffer has no records and an all-off config. + std::vector zero(PassthroughState::kSize, 0); + const auto z = PassthroughState::parse(zero.data()); + CHECK(!z.colour.has_value()); + CHECK(z.config.valid() && !z.config.enabled); + CHECK(z.mono_timestamp == 0 && z.colour_timestamp == 0); + CHECK(!z.mono_light.has_value()); + + if (a.colour && b.colour) + std::printf("lit light=%.3f dark light=%.3f\n", a.colour->light, b.colour->light); + if (failures) { std::printf("%d check(s) failed\n", failures); return 1; } + std::printf("all passthrough state tests passed\n"); + return 0; +} diff --git a/tests/test_sensors.cpp b/tests/test_sensors.cpp new file mode 100644 index 0000000..3322a9d --- /dev/null +++ b/tests/test_sensors.cpp @@ -0,0 +1,384 @@ +// Host-side tests for the sensor layer: XRService log parsing and the +// evidence rules (FINDINGS.md sections 29, 34 and 39), plus an end-to-end +// replay of the scenarios seen live, run through LightPolicy. + +#include "light_policy.hpp" +#include "sensors.hpp" +#include "xrservice_log.hpp" + +#include +#include +#include +#include + +using namespace autopass; + +namespace { + +int failures = 0; +#define CHECK(expr) do { if (!(expr)) { std::printf("FAIL line %d: %s\n", __LINE__, #expr); ++failures; } } while (0) + +// Real XRService lines from 2026-09-30. +const char* kModeAuto = "Wed Sep 30 2026 22:39:15.067651 INFO: SLAMConsole: [IREmitters] IR emitters mode changed to Auto"; +const char* kOn = "Wed Sep 30 2026 22:40:49.267834 INFO: SLAMConsole: [IREmitters] IR Emitters Turned On"; +const char* kOff = "Wed Sep 30 2026 22:41:24.230440 INFO: SLAMConsole: [IREmitters] IR Emitters Turned Off"; +const char* kNoise = "Wed Sep 30 2026 22:39:23.152963 WARNING: [DeckardCaptureSource] Max number of iteration reached when estimating the CCT from grey world gains"; + +void test_parsing() { + XrState s; + CHECK(apply_xrservice_line(kOn, &s) && s.emitters == std::optional(true)); + CHECK(apply_xrservice_line(kOff, &s) && s.emitters == std::optional(false)); + CHECK(apply_xrservice_line(kModeAuto, &s) && s.emitters == std::optional(false)); + CHECK(!apply_xrservice_line(kNoise, &s)); + CHECK(!apply_xrservice_line("IR Emitters Turned On", &s)); // must carry the [IREmitters] tag + + // Real lines from 2026-09-30 for the other tracked facts. + CHECK(apply_xrservice_line("Wed Sep 30 2026 22:39:15.083409 INFO: [DeckardCaptureSource] Passthrough cameras resumed", &s)); + CHECK(s.passthrough == std::optional(true)); + CHECK(apply_xrservice_line("Wed Sep 30 2026 22:44:35.826667 INFO: [DeckardCaptureSource] Passthrough cameras paused", &s)); + CHECK(s.passthrough == std::optional(false)); + CHECK(apply_xrservice_line("Wed Sep 30 2026 22:44:35.792844 INFO: [UserPresence] Received onEnterStandby from SteamVR. Setting UserPresenceDetected to 0.", &s)); + CHECK(s.standby == std::optional(true)); + CHECK(apply_xrservice_line("Wed Sep 30 2026 22:49:17.425068 INFO: [UserPresence] Received onLeaveStandby from SteamVR. Setting UserPresenceDetected to 1.", &s)); + CHECK(s.standby == std::optional(false)); + CHECK(apply_xrservice_line("Wed Sep 30 2026 22:44:35.852332 INFO: Transition to IMUFallback. Latest tracked pose: ref=Cam0", &s)); + CHECK(s.tracking_lost == std::optional(true)); + CHECK(apply_xrservice_line("Wed Sep 30 2026 22:39:15.336452 INFO: [DCU] [IMUFallback] Disabled with latest prediction: ", &s)); + CHECK(s.tracking_lost == std::optional(false)); + // "Tracking cameras streaming paused" is not the passthrough line. + XrState t; + CHECK(!apply_xrservice_line("Wed Sep 30 2026 22:44:35.885592 INFO: [DeckardCaptureSource] Tracking cameras streaming paused", &t)); + CHECK(!t.passthrough); + + // Text: last line of each kind wins; a trailing partial line counts. + const std::string nl = "\n"; + XrState u; + apply_xrservice_text(std::string(kModeAuto) + nl + kOn + nl + kNoise + nl + kOff + nl + kOn, &u); + CHECK(u.emitters == std::optional(true)); +} + +void test_ema() { + Ema e(1.0); + CHECK(!e.has()); + CHECK(e.add(0, 10) == 10); + const double v = e.add(1000, 0); // one time constant later: ~37% left + CHECK(v > 3.5 && v < 3.9); + e.reset(); + CHECK(!e.has()); +} + +void test_colour_rules() { + EvidenceBuilder b; + // Emitters on and colour dim: dark. + auto r = b.evaluate(0, Source::Color, true, 0.435); + CHECK(r.evidence == Evidence::Dark && r.cause == Cause::EmittersOnDim && !r.urgent); + // Emitters on but colour bright (a hand just left the cameras; the + // emitters lag ~5 s): not dark. + b.reset(); + CHECK(b.evaluate(0, Source::Color, true, 0.95).evidence == Evidence::Neutral); + // Emitters off, dim-looking view in good light (the 23:27:47 case): not dark. + b.reset(); + CHECK(b.evaluate(0, Source::Color, false, 0.63).evidence == Evidence::Neutral); + // Emitters off and colour reads dark (dusk, 2026-10-01 18:41): XRService's + // cameras have light enough, so no switch however long it lasts. + b.reset(); + for (std::uint64_t t = 0; t <= 60000; t += 1000) + CHECK(b.evaluate(t, Source::Color, false, 0.01).evidence == Evidence::Neutral); + // Emitters on and no colour reading ever: dark. + b.reset(); + CHECK(b.evaluate(0, Source::Color, true, std::nullopt).evidence == Evidence::Dark); + // Colour seen before but no fresh sample this tick: unknown. + b.reset(); + b.evaluate(0, Source::Color, true, 0.9); + CHECK(b.evaluate(250, Source::Color, true, std::nullopt).evidence == Evidence::Unknown); + // Emitter state unknown: never switch either way. + b.reset(); + CHECK(b.evaluate(0, Source::Color, std::nullopt, 0.0).evidence == Evidence::Neutral); + CHECK(b.evaluate(0, Source::Ir, std::nullopt, std::nullopt).evidence == Evidence::Neutral); +} + +void test_ir_rules() { + EvidenceBuilder b; + auto r = b.evaluate(0, Source::Ir, false, std::nullopt); + CHECK(r.evidence == Evidence::Bright && r.cause == Cause::EmittersOff); + CHECK(b.evaluate(0, Source::Ir, true, std::nullopt).evidence == Evidence::Unknown); + + // Emitters on and a dark IR light reading: not bright. + CHECK(b.evaluate(500, Source::Ir, true, std::nullopt, 0.0).evidence == Evidence::Neutral); +} + +void test_ir_light() { + // The IR camera's own light value (FINDINGS.md 41): a lit reading is + // bright at once (it steps 0 -> 0.33 within 0.1 s; the policy's + // confirmation does the rest). + EvidenceBuilder a; + auto r = a.evaluate(0, Source::Ir, true, std::nullopt, 0.3); + CHECK(r.evidence == Evidence::Bright && r.cause == Cause::IrLight); + CHECK(a.evaluate(250, Source::Ir, true, std::nullopt, 0.14).evidence == Evidence::Neutral); + // With a hold configured, it must last that long. + SensorConfig held; + held.ir_light_hold_s = 0.25; + EvidenceBuilder b(held); + CHECK(b.evaluate(0, Source::Ir, true, std::nullopt, 0.3).evidence == Evidence::Neutral); + r = b.evaluate(250, Source::Ir, true, std::nullopt, 0.3); + CHECK(r.evidence == Evidence::Bright && r.cause == Cause::IrLight); + // Dark (0 with the emitters on, 70 s in room B): never. + EvidenceBuilder c; + for (std::uint64_t t = 0; t <= 70000; t += 500) + CHECK(c.evaluate(t, Source::Ir, true, std::nullopt, 0.0).evidence != Evidence::Bright); + // A dark reading resets the hold. + EvidenceBuilder d(held); + d.evaluate(0, Source::Ir, true, std::nullopt, 0.3); + d.evaluate(125, Source::Ir, true, std::nullopt, 0.0); + CHECK(d.evaluate(250, Source::Ir, true, std::nullopt, 0.3).evidence != Evidence::Bright); + // If it misled us (colour dark right after), it is locked out for a while. + EvidenceBuilder e; + e.on_switched(0, Source::Color, true, Cause::IrLight); + CHECK(e.evaluate(300, Source::Color, true, 0.0).cause == Cause::VerifyFailed); + e.on_switched(600, Source::Ir, true, Cause::VerifyFailed); + for (std::uint64_t t = 1000; t < 60000; t += 250) + CHECK(e.evaluate(t, Source::Ir, true, std::nullopt, 0.3).evidence != Evidence::Bright); + e.evaluate(61000, Source::Ir, true, std::nullopt, 0.3); + CHECK(e.evaluate(61250, Source::Ir, true, std::nullopt, 0.3).evidence == Evidence::Bright); +} + +void test_ir_light_lockout_escalates() { + // The IR light value misled us (assumed: a surface at medium distance, + // lit by the emitters, in the dark): after each failed colour check it + // is ignored for 60 s, then 120 s, ... + EvidenceBuilder b; + b.on_switched(10000, Source::Color, true, Cause::IrLight); + CHECK(b.evaluate(10300, Source::Color, true, 0.0).cause == Cause::VerifyFailed); + b.on_switched(10600, Source::Ir, true, Cause::VerifyFailed); + CHECK(!b.wants_ir_light(11000, true)); + for (std::uint64_t t = 11000; t < 70000; t += 500) + CHECK(b.evaluate(t, Source::Ir, true, std::nullopt, 0.2).evidence != Evidence::Bright); + CHECK(b.wants_ir_light(70400, true)); + b.on_switched(80000, Source::Color, true, Cause::IrLight); + b.evaluate(80300, Source::Color, true, 0.0); + b.on_switched(80600, Source::Ir, true, Cause::VerifyFailed); + CHECK(!b.wants_ir_light(80300 + 119000, true)); + CHECK(b.wants_ir_light(80300 + 121000, true)); + // Emitters off never needs it unless "off" is distrusted. + EvidenceBuilder c; + CHECK(!c.wants_ir_light(0, false)); + CHECK(c.wants_ir_light(0, true)); +} + +void test_verify_and_distrust() { + // "Emitters off" took us to colour, but they are back on and colour is + // dark (a wall fooled XRService): back to IR at once, and "off" is not + // believed for 60 s, then 120 s, ... + EvidenceBuilder b; + b.on_switched(10000, Source::Color, true, Cause::EmittersOff); + auto r = b.evaluate(10300, Source::Color, true, 0.0); + CHECK(r.evidence == Evidence::Dark && r.cause == Cause::VerifyFailed && r.urgent); + CHECK(b.distrusting_emitters_off(10300)); + b.on_switched(10800, Source::Ir, true, Cause::VerifyFailed); + CHECK(b.evaluate(11000, Source::Ir, false, std::nullopt).evidence != Evidence::Bright); + CHECK(b.evaluate(70000, Source::Ir, false, std::nullopt).evidence != Evidence::Bright); + CHECK(b.wants_ir_light(70000, false)); // the IR light value stands in meanwhile + // It expires by itself: never stuck in IR while the emitters stay off. + CHECK(!b.distrusting_emitters_off(70400)); + r = b.evaluate(70400, Source::Ir, false, std::nullopt); + CHECK(r.evidence == Evidence::Bright && r.cause == Cause::EmittersOff); + // A second mistake doubles it. + b.on_switched(80000, Source::Color, true, Cause::EmittersOff); + b.evaluate(80300, Source::Color, true, 0.0); + CHECK(b.distrusting_emitters_off(80300 + 119000)); + CHECK(!b.distrusting_emitters_off(80300 + 120000)); + + // Emitters off and colour dark right after switching (dusk): fine, no revert. + EvidenceBuilder f; + f.on_switched(0, Source::Color, true, Cause::EmittersOff); + r = f.evaluate(300, Source::Color, false, 0.01); + CHECK(r.evidence == Evidence::Neutral && !r.urgent && !f.distrusting_emitters_off(300)); + // A correct switch to colour (it is light) passes the check quietly, + // also while the emitters lag behind. + EvidenceBuilder c; + c.on_switched(0, Source::Color, true, Cause::IrLight); + r = c.evaluate(300, Source::Color, true, 0.9); + CHECK(r.evidence == Evidence::Neutral && !r.urgent); + // After the verify window, emitters on and dark colour take the normal + // (not urgent) route. + EvidenceBuilder d; + d.on_switched(0, Source::Color, true, Cause::EmittersOff); + d.evaluate(3500, Source::Color, true, 0.9); + r = d.evaluate(4000, Source::Color, true, 0.0); + CHECK(r.evidence == Evidence::Dark && r.cause == Cause::EmittersOnDim && !r.urgent); + // A manual switch to colour is not verified. + EvidenceBuilder e; + e.on_switched(0, Source::Color, false, Cause::None); + CHECK(!e.evaluate(300, Source::Color, true, 0.0).urgent); +} + +// End to end: a simulated room, XRService's emitters as measured (on ~1 s +// after darkness, off ~5 s after light returns, fooled off after ~3 s with +// a wall close to the headset, off at dusk), the colour camera, sensors, +// policy. +enum class Wall { None, Close, Medium }; + +struct World { + bool emitters = false; + bool sticky = false; // XRService keeps its emitters on once on (00:28-00:30) + std::uint64_t dark_since = 0, bright_since = 0, wall_since = 0; + void step(std::uint64_t t, double light, Wall wall_kind) { + const bool wall = wall_kind == Wall::Close; + if (wall) { + if (!wall_since) wall_since = t ? t : 1; + if (t - wall_since >= 3000) emitters = false; // reflected IR fools XRService + return; + } + wall_since = 0; + if (light < 0.3 && !dusk(light)) { + if (!dark_since) dark_since = t ? t : 1; + bright_since = 0; + if (t - dark_since >= 1000) emitters = true; + } else if (light >= 0.6 || dusk(light)) { + if (!bright_since) bright_since = t ? t : 1; + dark_since = 0; + if (t - bright_since >= 5000 && !sticky) emitters = false; + } else { + dark_since = bright_since = 0; + } + } + // Dusk (light 0.1..0.3): the colour camera reads ~0 like a dark room, + // but XRService's cameras manage without emitters. + static bool dusk(double light) { return light >= 0.1 && light < 0.3; } + static double colour(double light) { return light >= 0.6 ? 0.9 : light >= 0.3 ? 0.435 : 0.0; } + // The IR camera's light value (FINDINGS.md 41-42): 0 dark, ~0.3 lit, + // 0.10 at a close wall lit by the emitters (measured). A surface at + // medium distance is assumed to fool it (not measured). + static double ir_light(double light, Wall wall) { + if (wall == Wall::Close) return 0.10; + if (wall == Wall::Medium) return 0.2; + return light >= 0.6 ? 0.3 : 0.0; + } +}; + +struct Replay { + std::vector switches; + std::vector to; + std::uint64_t colour_in_dark_ms = 0; // time spent showing colour while it was dark +}; + +Replay replay(double (*light)(std::uint64_t), Wall (*wall)(std::uint64_t), std::uint64_t end_ms, + bool emitters_stuck_off = false, bool sticky = false) { + LightPolicy p({}, Source::Color); + EvidenceBuilder b; + World w; + w.sticky = sticky; + Replay out; + for (std::uint64_t t = 0; t <= end_ms; t += 250) { + const double l = light(t); + const Wall wk = wall(t); + w.step(t, l, wk); + if (emitters_stuck_off) w.emitters = false; + const bool colour_shown = p.source() == Source::Color; + if (colour_shown && l < 0.1) out.colour_in_dark_ms += 250; + // autopassd reads the IR light value at 2 Hz, only when it could matter. + std::optional ir; + if (!colour_shown && b.wants_ir_light(t, w.emitters) && t % 500 == 0) ir = World::ir_light(l, wk); + const auto r = b.evaluate(t, p.source(), w.emitters, + colour_shown ? std::optional(World::colour(l)) : std::nullopt, ir); + const auto d = p.update(t, r.evidence, r.why, r.urgent); + if (d.changed) { + out.switches.push_back(t); + out.to.push_back(d.source); + b.on_switched(t, d.source, true, r.cause); + } + } + return out; +} + +Wall no_wall(std::uint64_t) { return Wall::None; } + +void print(const char* what, const Replay& r) { + std::printf("%s: switches at", what); + for (std::size_t i = 0; i < r.switches.size(); ++i) + std::printf(" %.2f(%s)", r.switches[i] / 1000.0, r.to[i] == Source::Color ? "colour" : "ir"); + std::printf("; colour shown in the dark %.2f s\n", r.colour_in_dark_ms / 1000.0); +} + +void test_end_to_end() { + // Lights off at 10 s, on at 40 s: IR within ~2 s; colour within ~1 s + // (the IR light value, held 0.25 s), before XRService's emitters go off. + auto r = replay([](std::uint64_t t) { return t >= 10000 && t < 40000 ? 0.0 : 1.0; }, no_wall, 80000); + print("off/on", r); + CHECK(r.switches.size() == 2); + if (r.switches.size() == 2) { + CHECK(r.switches[0] >= 11000 && r.switches[0] <= 12000); + CHECK(r.switches[1] >= 40500 && r.switches[1] <= 41750); + } + + // The 00:28 walk-through: XRService keeps its emitters on through bright + // rooms. Colour must still come back. + r = replay([](std::uint64_t t) { return t >= 10000 && t < 40000 ? 0.0 : 1.0; }, no_wall, 80000, false, true); + print("bright room, emitters stay on", r); + CHECK(r.switches.size() == 2); + if (r.switches.size() == 2) CHECK(r.switches[1] >= 40500 && r.switches[1] <= 41750); + + // Dark throughout with the emitters on: never back to colour. + r = replay([](std::uint64_t t) { return t >= 5000 ? 0.0 : 1.0; }, no_wall, 180000); + CHECK(r.switches.size() == 1); + + // A hand over the cameras for 1.5 s in a lit room: no switch. + r = replay([](std::uint64_t t) { return t >= 10000 && t < 11500 ? 0.0 : 1.0; }, no_wall, 40000); + CHECK(r.switches.empty()); + + // Good light, changing views: never switches. + r = replay([](std::uint64_t t) { return (t / 7000) % 2 ? 1.0 : 0.7; }, no_wall, 120000); + CHECK(r.switches.empty()); + + // Dimmed to 45%: nothing changes. + r = replay([](std::uint64_t) { return 0.45; }, no_wall, 60000); + CHECK(r.switches.empty()); + + // Dusk (2026-10-01 18:41): the colour camera reads dark, XRService's + // emitters stay off. Never switches. + r = replay([](std::uint64_t t) { return t >= 10000 ? 0.2 : 1.0; }, no_wall, 180000); + print("dusk", r); + CHECK(r.switches.empty()); + + // Lights off (IR), then back to dusk level: the emitters go off, colour + // returns and stays (the dark colour reading is not a failed check). + r = replay([](std::uint64_t t) { return t >= 10000 && t < 40000 ? 0.0 : 0.2; }, no_wall, 180000); + print("dark then dusk", r); + CHECK(r.switches.size() == 2 && !r.to.empty() && r.to.back() == Source::Color); + + // Dark room, face near a wall from 20 s to 40 s. XRService turns its + // emitters off at ~23 s: colour (dark) until they come back on after + // the wall, then IR stays. The accepted cost of trusting the emitters. + r = replay([](std::uint64_t) { return 0.0; }, + [](std::uint64_t t) { return t >= 20000 && t < 40000 ? Wall::Close : Wall::None; }, 120000); + print("close wall in the dark", r); + CHECK(!r.to.empty() && r.to.back() == Source::Ir); + CHECK(r.switches.size() <= 3); + CHECK(r.colour_in_dark_ms <= 22000); // the initial ~1.5 s, plus the time at the wall + + // A wall at medium distance in the dark for three minutes: the view + // is assumed to fool the IR light value. Each mistake is a sub-second + // colour flash, then it is ignored for 60 s, 120 s, ...: a few flashes. + r = replay([](std::uint64_t) { return 0.0; }, + [](std::uint64_t t) { return t >= 20000 && t < 200000 ? Wall::Medium : Wall::None; }, 240000); + print("medium wall in the dark", r); + CHECK(!r.to.empty() && r.to.back() == Source::Ir); + CHECK(r.switches.size() <= 7); + CHECK(r.colour_in_dark_ms <= 5000); +} + +} // namespace + +int main() { + test_parsing(); + test_ema(); + test_colour_rules(); + test_ir_rules(); + test_ir_light(); + test_ir_light_lockout_escalates(); + test_verify_and_distrust(); + test_end_to_end(); + if (failures) { std::printf("%d check(s) failed\n", failures); return 1; } + std::printf("all sensor tests passed\n"); + return 0; +} diff --git a/tests/test_xrservice_log.cpp b/tests/test_xrservice_log.cpp new file mode 100644 index 0000000..6a09aa0 --- /dev/null +++ b/tests/test_xrservice_log.cpp @@ -0,0 +1,139 @@ +// Host-side test for XrServiceLog against a fake Steam logs directory: +// initial state from an existing log, appended lines (including one split +// across two writes), unrelated noise, and an XRService restart that +// replaces the xrservice.txt symlink. + +#include "xrservice_log.hpp" + +#include +#include +#include +#include +#include +#include + +using autopass::XrServiceLog; + +namespace { + +int failures = 0; +#define CHECK(expr) do { if (!(expr)) { std::printf("FAIL line %d: %s\n", __LINE__, #expr); ++failures; } } while (0) + +const char* kAuto = "Wed Sep 30 2026 22:39:15.067651 INFO: SLAMConsole: [IREmitters] IR emitters mode changed to Auto\n"; +const char* kOn = "Wed Sep 30 2026 22:40:49.267834 INFO: SLAMConsole: [IREmitters] IR Emitters Turned On\n"; +const char* kOff = "Wed Sep 30 2026 22:41:24.230440 INFO: SLAMConsole: [IREmitters] IR Emitters Turned Off\n"; +const char* kNoise = "Wed Sep 30 2026 22:39:23.152963 WARNING: [DeckardCaptureSource] Max number of iteration reached\n"; + +void append(const std::string& path, const std::string& text) { + std::ofstream f(path, std::ios::app | std::ios::binary); + f << text; +} + +} // namespace + +int main() { + char tmpl[] = "/tmp/autopass_emitter_XXXXXX"; + const std::string dir = ::mkdtemp(tmpl); + const std::string log1 = dir + "/XRService-1.log", log2 = dir + "/XRService-2.log", link = dir + "/xrservice.txt"; + const std::string log3 = dir + "/XRService-3.log"; + + append(log1, std::string(kAuto) + kNoise + kOn + kNoise); + CHECK(::symlink(log1.c_str(), link.c_str()) == 0); + + XrServiceLog w(dir); + CHECK(w.start()); + CHECK(w.state().emitters == std::optional(true)); // last line in the existing log + CHECK(!w.update()); // nothing new + CHECK(!w.take_restarted()); + + append(log1, kNoise); + CHECK(!w.update()); + CHECK(w.state().emitters == std::optional(true)); + + // A line written in two parts is only parsed once complete. + const std::string off = kOff; + append(log1, off.substr(0, 40)); + w.update(); + CHECK(w.state().emitters == std::optional(true)); + append(log1, off.substr(40)); + CHECK(w.update()); + CHECK(w.state().emitters == std::optional(false)); + + // XRService restarts: a new session log, symlink replaced atomically. + append(log2, std::string(kAuto) + kNoise); + const std::string tmp_link = dir + "/xrservice.txt.new"; + CHECK(::symlink(log2.c_str(), tmp_link.c_str()) == 0); + CHECK(std::rename(tmp_link.c_str(), link.c_str()) == 0); + w.update(); + CHECK(w.state().emitters == std::optional(false)); // "mode changed to Auto" with no On: off + CHECK(w.take_restarted()); + CHECK(!w.take_restarted()); + append(log2, kOn); + CHECK(w.update()); + CHECK(w.state().emitters == std::optional(true)); + // The old log is no longer followed. + append(log1, kOff); + w.update(); + CHECK(w.state().emitters == std::optional(true)); + + // A restart while the watcher is stopped (passthrough off) is still + // noticed on the next start. + w.stop(); + CHECK(!w.running()); + append(log3, kAuto); + const std::string tmp3 = dir + "/xrservice.txt.3"; + CHECK(::symlink(log3.c_str(), tmp3.c_str()) == 0); + CHECK(std::rename(tmp3.c_str(), link.c_str()) == 0); + CHECK(w.start()); + CHECK(w.take_restarted()); + // ...but not when it is the same session. + w.stop(); + CHECK(w.start()); + CHECK(!w.take_restarted()); + w.stop(); + std::remove(log3.c_str()); + std::remove(link.c_str()); + std::remove(log1.c_str()); + std::remove(log2.c_str()); + ::rmdir(dir.c_str()); + + // Session start from the real path layout. + const long long t0 = XrServiceLog::parse_session_start( + "/home/steamos/.local/share/Steam/logs/XRService-2026.09.30/XRService-23-49-58.log"); + CHECK(t0 > 0); + { + const std::time_t tt = static_cast(t0); + const std::tm* lt = std::localtime(&tt); + CHECK(lt->tm_year == 126 && lt->tm_mon == 8 && lt->tm_mday == 30 && lt->tm_hour == 23 && lt->tm_min == 49 && + lt->tm_sec == 58); + } + CHECK(XrServiceLog::parse_session_start("/tmp/whatever.log") == 0); + + // Passthrough and standby lines in the same stream. + { + char tmpl2[] = "/tmp/autopass_xrlog_XXXXXX"; + const std::string d2 = ::mkdtemp(tmpl2); + const std::string lg = d2 + "/XRService-1.log", lk = d2 + "/xrservice.txt"; + append(lg, "x INFO: [DeckardCaptureSource] Passthrough cameras resumed\n"); + CHECK(::symlink(lg.c_str(), lk.c_str()) == 0); + XrServiceLog x(d2); + CHECK(x.start()); + CHECK(x.state().passthrough == std::optional(true)); + append(lg, "x INFO: [UserPresence] Received onEnterStandby from SteamVR.\nx INFO: [DeckardCaptureSource] Passthrough cameras paused\n"); + CHECK(x.update()); + CHECK(x.state().passthrough == std::optional(false) && x.state().standby == std::optional(true)); + x.stop(); + std::remove(lk.c_str()); + std::remove(lg.c_str()); + ::rmdir(d2.c_str()); + } + + // No log at all: start() fails cleanly. + XrServiceLog missing("/nonexistent/autopass/logs"); + CHECK(!missing.start()); + CHECK(!missing.error().empty()); + + if (failures) { std::printf("%d check(s) failed\n", failures); return 1; } + std::printf("all XRService log tests passed\n"); + return 0; +} diff --git a/tools/autopass.cpp b/tools/autopass.cpp new file mode 100644 index 0000000..7d9bbe5 --- /dev/null +++ b/tools/autopass.cpp @@ -0,0 +1,310 @@ +// autopass: install, inspect and support autopassd. +// +// autopass install install the systemd user unit that starts autopassd +// with SteamVR and stops it with SteamVR; start it +// autopass uninstall stop and remove the unit, switch back to colour +// autopass status autopassd's status and whether the unit is active +// autopass update download and install the latest release +// autopass report write ~/frame-autopass-report.txt for a bug report +// autopass version print the version +// autopass guard [reset] show or clear the crash guard (XRService restarts +// soon after autopassd switches) +// autopass state [SECONDS] passive: camera config and colour light value +// from shared memory, no SteamVR calls +// autopass get read the camera config (private interface) +// autopass set rgb|mono [--quiet] +// switch the camera source (used by autopassd). Exit +// codes: 0 done or already so, 1 error, 3 SteamVR's +// interface changed, 4 camera not enabled, 5 no +// Arcturus colour module. +// +// `get` and `set` connect to SteamVR as a background client for a few +// milliseconds; that never starts SteamVR and does not wake the headset. + +#include "app_paths.hpp" +#include "passthrough_state.hpp" +#include "private_camera.hpp" + +#include + +#include +#include +#include +#include +#include +#include +#include + +namespace { + +constexpr const char* kUnit = "frame-autopass.service"; +constexpr const char* kInstallScript = "https://github.com/bod09/frame-autopass/releases/latest/download/install.sh"; + +int usage() { + std::fprintf(stderr, + "usage: autopass install | uninstall | status | update | report | version |\n" + " guard [reset] | state [SECONDS] | get | set rgb|mono [--quiet]\n"); + return 2; +} + +std::string unit_path() { + const char* xdg = std::getenv("XDG_CONFIG_HOME"); + const char* home = std::getenv("HOME"); + const std::string base = xdg && *xdg == '/' ? xdg : std::string(home ? home : "/tmp") + "/.config"; + return base + "/systemd/user/" + kUnit; +} + +// Runs `systemctl --user ARGS...`, returns its exit code. +int systemctl(std::initializer_list args, bool quiet = false) { + std::string cmd = "systemctl --user"; + for (const char* a : args) cmd += std::string(" ") + a; + if (quiet) cmd += " >/dev/null 2>&1"; + const int rc = std::system(cmd.c_str()); + return WIFEXITED(rc) ? WEXITSTATUS(rc) : 1; +} + +std::string unit_text() { + return std::string( + "# Installed by `autopass install`. Starts autopassd with SteamVR and stops it\n" + "# with SteamVR. Remove with `autopass uninstall`.\n" + "[Unit]\n" + "Description=Adaptive passthrough (colour in good light, IR in the dark)\n" + "After=steamvr.service\n" + "BindsTo=steamvr.service\n" + "\n" + "[Service]\n" + "Type=simple\n" + "ExecStart=") + autopass::install_dir() + "/autopassd\n" + "Restart=on-failure\n" + "RestartSec=5\n" + "Slice=session.slice\n" + "Nice=10\n" + "\n" + "[Install]\n" + "WantedBy=steamvr.service\n"; +} + +int install() { + const std::string dir = autopass::install_dir(); + if (::access((dir + "/autopassd").c_str(), X_OK) != 0 || ::access((dir + "/autopass").c_str(), X_OK) != 0) { + std::fprintf(stderr, "autopassd and autopass must be in %s first\n", dir.c_str()); + return 1; + } + if (!autopass::colour_module_present()) { + std::fprintf(stderr, + "The Arcturus Vision colour module was not found (no arcimx616 camera in\n" + "/sys/class/video4linux). frame-autopass switches between that module and the\n" + "built-in IR cameras, so it needs the module attached. Not installing.\n"); + return 1; + } + const std::string path = unit_path(); + if (!autopass::make_dirs(path.substr(0, path.rfind('/'))) || !autopass::write_file_atomic(path, unit_text())) { + std::fprintf(stderr, "cannot write %s\n", path.c_str()); + return 1; + } + if (systemctl({"daemon-reload"}) != 0 || systemctl({"enable", "--now", kUnit}) != 0) { + std::fprintf(stderr, "systemctl failed; see `systemctl --user status %s`\n", kUnit); + return 1; + } + std::printf("installed %s: autopassd now starts and stops with SteamVR\n", path.c_str()); + return 0; +} + +int switch_source(bool want_rgb, bool quiet); + +int uninstall() { + systemctl({"disable", "--now", kUnit}, true); + std::remove(unit_path().c_str()); + systemctl({"daemon-reload"}, true); + std::printf("removed %s\n", kUnit); + // Leave the headset showing colour, the stock behaviour with the module. + switch_source(true, true); + return 0; +} + +int status() { + std::string text; + std::printf("unit: %s, %s\n", systemctl({"is-enabled", "--quiet", kUnit}, true) == 0 ? "installed" : "not installed", + systemctl({"is-active", "--quiet", kUnit}, true) == 0 ? "running" : "not running"); + std::printf("version: %s\n", autopass::version()); + if (autopass::read_file(autopass::status_path(), &text)) { + std::printf("status: %s", text.c_str()); + if (text.find("\"blocked\": \"steamvr-changed\"") != std::string::npos) + std::printf("\nSteamVR has changed the camera interface frame-autopass uses, so it has\n" + "stopped switching (passthrough itself works normally). Run `autopass update`;\n" + "if that does not help, a new release is needed.\n"); + else if (text.find("\"blocked\": \"no-colour-module\"") != std::string::npos) + std::printf("\nThe Arcturus Vision colour module is not attached, so there is nothing to\n" + "switch to; it resumes when the module is back.\n"); + else if (text.find("\"blocked\": \"crash-guard\"") != std::string::npos) + std::printf("\nXRService restarted twice soon after a switch, so switching is paused as a\n" + "precaution. `autopass guard reset`, then restart SteamVR, to resume.\n"); + } + if (autopass::read_file(autopass::crash_guard_path(), &text) && !text.empty()) + std::printf("crash guard entries (switching stops at 2):\n%s", text.c_str()); + return 0; +} + +int guard(bool reset) { + std::string text; + const bool present = autopass::read_file(autopass::crash_guard_path(), &text) && !text.empty(); + if (reset) { + std::remove(autopass::crash_guard_path().c_str()); + std::printf("crash guard cleared; restart autopassd (or SteamVR) to resume switching\n"); + return 0; + } + std::printf(present ? "crash guard entries:\n%s" : "crash guard: no entries\n", text.c_str()); + return 0; +} + +int state(double seconds) { + const autopass::PassthroughState st; + if (st.path().empty()) { + std::fprintf(stderr, "passthrough state file not found in /dev/shm\n"); + return 1; + } + const auto end = std::chrono::steady_clock::now() + std::chrono::duration(seconds); + do { + const auto s = st.read(); + if (!s) { + std::fprintf(stderr, "cannot read %s\n", st.path().c_str()); + return 1; + } + std::printf("config: %s", s->config.describe().c_str()); + if (s->colour) std::printf(" | colour light %.3f", s->colour->light); + std::printf(" | newest mono %.3f colour %.3f\n", s->mono_timestamp, s->colour_timestamp); + if (seconds > 0) std::this_thread::sleep_for(std::chrono::milliseconds(500)); + } while (std::chrono::steady_clock::now() < end); + return 0; +} + +int update() { + std::printf("fetching %s\n", kInstallScript); + const std::string cmd = std::string("curl -fsSL ") + kInstallScript + " | bash"; + const int rc = std::system(cmd.c_str()); + return WIFEXITED(rc) ? WEXITSTATUS(rc) : 1; +} + +// Appends a command's output to the report. +void section(std::string* out, const char* title, const std::string& cmd) { + *out += std::string("\n== ") + title + " ==\n"; + if (FILE* p = ::popen((cmd + " 2>&1").c_str(), "r")) { + char buf[4096]; + std::size_t n; + while ((n = std::fread(buf, 1, sizeof buf, p)) > 0) out->append(buf, n); + ::pclose(p); + } +} + +int report() { + const char* home = std::getenv("HOME"); + const std::string path = std::string(home ? home : "/tmp") + "/frame-autopass-report.txt"; + std::string out = std::string("frame-autopass report, version ") + autopass::version() + "\n"; + out += std::string("Arcturus colour module: ") + (autopass::colour_module_present() ? "found" : "NOT found") + "\n"; + section(&out, "date", "date"); + section(&out, "SteamOS", "grep -E '^(VERSION_ID|BUILD_ID)=' /etc/os-release"); + section(&out, "SteamVR", "cat /opt/steamvr/bin/version.txt; grep -m1 -E 'cv: version [0-9]' ~/.local/share/Steam/logs/vrserver.txt"); + section(&out, "unit", std::string("systemctl --user status --no-pager ") + kUnit + " | head -5"); + section(&out, "status", "cat " + autopass::status_path()); + section(&out, "config", "cat " + autopass::conf_path() + " 2>/dev/null || echo '(defaults)'"); + section(&out, "crash guard", "cat " + autopass::crash_guard_path() + " 2>/dev/null || echo '(empty)'"); + section(&out, "autopassd log (last 300 lines)", "tail -n 300 " + autopass::log_path()); + section(&out, "XRService (last 150 relevant lines)", + "grep -hE 'IREmitters|Passthrough cameras|UserPresence|IMUFallback\\] (En|Dis)|Transition to IMU' " + "~/.local/share/Steam/logs/xrservice.txt | cut -c1-140 | tail -n 150"); + if (!autopass::write_file_atomic(path, out)) { + std::fprintf(stderr, "cannot write %s\n", path.c_str()); + return 1; + } + std::printf("wrote %s\nIt contains your SteamOS and SteamVR versions and recent logs (no personal\n" + "data that I know of; have a look before sharing it). Attach it to a GitHub issue.\n", + path.c_str()); + return 0; +} + +struct Session { + bool ok = false; + Session() { + vr::EVRInitError err = vr::VRInitError_None; + vr::VR_Init(&err, vr::VRApplication_Background); + ok = err == vr::VRInitError_None; + if (!ok) std::fprintf(stderr, "SteamVR unavailable: %s\n", vr::VR_GetVRInitErrorAsEnglishDescription(err)); + } + ~Session() { + if (ok) vr::VR_Shutdown(); + } +}; + +int get_config() { + Session session; + if (!session.ok) return 1; + autopass::PrivateCamera camera; + if (!camera.ok()) { + std::fprintf(stderr, "%s\n", camera.error().c_str()); + return 3; + } + const auto c = camera.get(); + if (!c) { + std::fprintf(stderr, "%s\n", camera.error().c_str()); + return 1; + } + std::printf("%s\n", c->describe().c_str()); + return 0; +} + +int switch_source(bool want_rgb, bool quiet) { + if (!autopass::colour_module_present()) { + if (!quiet) std::fprintf(stderr, "Arcturus colour module not found; not changing anything\n"); + return 5; + } + Session session; + if (!session.ok) return 1; + autopass::PrivateCamera camera; + if (!camera.ok()) { + std::fprintf(stderr, "%s\n", camera.error().c_str()); + return 3; + } + auto c = camera.get(); + if (!c) { + std::fprintf(stderr, "%s\n", camera.error().c_str()); + return 1; + } + if (!c->enabled) { + if (!quiet) std::fprintf(stderr, "passthrough camera is not enabled; not changing anything\n"); + return 4; + } + if (c->rgb == static_cast(want_rgb)) { + if (!quiet) std::printf("already %s\n", want_rgb ? "colour" : "IR"); + return 0; + } + c->rgb = want_rgb ? 1 : 0; + if (!camera.set(*c)) { + std::fprintf(stderr, "%s\n", camera.error().c_str()); + return 1; + } + if (!quiet) std::printf("switched to %s\n", want_rgb ? "colour" : "IR"); + return 0; +} + +} // namespace + +int main(int argc, char** argv) { + if (argc < 2) return usage(); + const std::string cmd = argv[1]; + if (cmd == "install" && argc == 2) return install(); + if (cmd == "uninstall" && argc == 2) return uninstall(); + if (cmd == "status" && argc == 2) return status(); + if (cmd == "update" && argc == 2) return update(); + if (cmd == "report" && argc == 2) return report(); + if ((cmd == "version" || cmd == "--version") && argc == 2) return std::printf("%s\n", autopass::version()) < 0; + if (cmd == "guard" && argc == 2) return guard(false); + if (cmd == "guard" && argc == 3 && std::string(argv[2]) == "reset") return guard(true); + if (cmd == "state" && argc <= 3) return state(argc == 3 ? std::atof(argv[2]) : 0); + if (cmd == "get" && argc == 2) return get_config(); + if (cmd == "set" && (argc == 3 || (argc == 4 && std::string(argv[3]) == "--quiet"))) { + const std::string which = argv[2]; + if (which != "rgb" && which != "mono") return usage(); + return switch_source(which == "rgb", argc == 4); + } + return usage(); +} diff --git a/tools/disasm.py b/tools/disasm.py new file mode 100644 index 0000000..4b146a2 --- /dev/null +++ b/tools/disasm.py @@ -0,0 +1,201 @@ +#!/usr/bin/env python3 +"""Annotated arm64 disassembly of functions in a stripped shared object. + +Used to learn the signatures of SteamVR's private interfaces from the +headset's own binaries. Resolves adrp+add/ldr pairs to strings or data, +PLT stubs to imported symbol names, and RELATIVE relocations in data. + +Usage: disasm.py LIB ADDR [ADDR ...] [--max N] [--until ADDR] + disasm.py --xref LIB TARGET [TARGET ...] +Addresses are hex. Disassembles linearly from ADDR until a `ret` that is +not skipped over by an earlier forward branch, or N instructions. +""" + +import re +import struct +import subprocess +import sys + +import capstone +from capstone import arm64_const as A + + +class Elf: + def __init__(self, path): + self.path = path + self.data = open(path, "rb").read() + self.sections = [] + out = subprocess.run(["readelf", "-SW", path], capture_output=True, text=True).stdout + for m in re.finditer(r"\]\s+(\S+)\s+\S+\s+([0-9a-f]+)\s+([0-9a-f]+)\s+([0-9a-f]+)", out): + name, addr, off, size = m.group(1), int(m.group(2), 16), int(m.group(3), 16), int(m.group(4), 16) + self.sections.append((name, addr, off, size)) + self.relative = {} + self.symbolic = {} + out = subprocess.run(["readelf", "-rW", path], capture_output=True, text=True).stdout + for line in out.splitlines(): + m = re.match(r"\s*([0-9a-f]+)\s+[0-9a-f]+\s+(R_AARCH64_\w+)\s+(\S+)?\s*(.*)", line) + if not m: + continue + where, kind = int(m.group(1), 16), m.group(2) + if kind == "R_AARCH64_RELATIVE": + self.relative[where] = int(m.group(3), 16) + elif kind in ("R_AARCH64_JUMP_SLOT", "R_AARCH64_GLOB_DAT", "R_AARCH64_ABS64"): + rest = (m.group(4) or "").strip() + sym = rest.split("+")[0].strip() if rest else "" + self.symbolic[where] = sym.split("@")[0] + self.plt = self._map_plt() + + def section_of(self, addr): + for name, a, off, size in self.sections: + if a and a <= addr < a + size: + return name, a, off + return None + + def read(self, addr, n): + s = self.section_of(addr) + if not s or s[0] == ".bss": + return None + return self.data[addr - s[1] + s[2]: addr - s[1] + s[2] + n] + + def cstring(self, addr, limit=160): + raw = self.read(addr, limit) + if not raw: + return None + end = raw.find(b"\0") + if end <= 0: + return None + text = raw[:end] + if all(32 <= c < 127 or c in (9, 10) for c in text): + return text.decode() + return None + + def u64(self, addr): + raw = self.read(addr, 8) + return struct.unpack(" 1: + got = page + ins.operands[1].mem.disp + sym = self.symbolic.get(got) + if sym: + plt[ins.address - 4] = sym + page = None + return plt + + def describe(self, addr): + """Best-effort label for an address used as data.""" + if addr in self.plt: + return "plt:" + self.plt[addr] + s = self.cstring(addr) + if s: + return repr(s) + if addr in self.relative: + target = self.relative[addr] + s = self.cstring(target) + return f"-> {target:#x}" + (f" {s!r}" if s else "") + if addr in self.symbolic: + return "got:" + self.symbolic[addr] + sec = self.section_of(addr) + return sec[0] if sec else None + + +def disassemble(elf, start, max_ins, until=None): + md = capstone.Cs(capstone.CS_ARCH_ARM64, capstone.CS_MODE_ARM) + md.detail = True + code = elf.read(start, max_ins * 4) + regs_page = {} + furthest = start + for ins in md.disasm(code, start): + note = "" + ops = ins.operands + if ins.id == A.ARM64_INS_ADRP: + regs_page[ops[0].reg] = ops[1].imm + elif ins.id == A.ARM64_INS_ADD and len(ops) == 3 and ops[1].reg in regs_page and ops[2].type == A.ARM64_OP_IMM: + target = regs_page.pop(ops[1].reg) + ops[2].imm + note = f"{target:#x} {elf.describe(target) or ''}" + elif ins.id in (A.ARM64_INS_LDR, A.ARM64_INS_STR) and len(ops) > 1 and ops[1].type == A.ARM64_OP_MEM \ + and ops[1].mem.base in regs_page: + target = regs_page[ops[1].mem.base] + ops[1].mem.disp + note = f"[{target:#x}] {elf.describe(target) or ''}" + elif ins.id in (A.ARM64_INS_BL, A.ARM64_INS_B) and ops and ops[0].type == A.ARM64_OP_IMM: + target = ops[0].imm + label = elf.plt.get(target) + note = f"-> {label}" if label else "" + if ins.id == A.ARM64_INS_B and target > furthest: + furthest = target + elif ins.group(capstone.CS_GRP_JUMP) and ops and ops[-1].type == A.ARM64_OP_IMM: + if ops[-1].imm > furthest: + furthest = ops[-1].imm + print(f" {ins.address:#x}: {ins.mnemonic:8} {ins.op_str:40} {note}".rstrip()) + if until is not None: + if ins.address + 4 >= until: + break + continue + if ins.id == A.ARM64_INS_RET and ins.address >= furthest: + break + if ins.id == A.ARM64_INS_B and ops and ops[0].type == A.ARM64_OP_IMM and ins.address >= furthest \ + and ops[0].imm < ins.address: + break + + +def xrefs(elf, target): + """Addresses in .text whose adrp+add (or adrp+ldr) pair forms `target`.""" + name, addr, off, size = [s for s in elf.sections if s[0] == ".text"][0] + words = struct.unpack_from(f"<{size // 4}I", elf.data, off) + hits = [] + page_of = {} + for i, w in enumerate(words): + pc = addr + i * 4 + if (w & 0x9F000000) == 0x90000000: # adrp + rd = w & 0x1F + immlo = (w >> 29) & 3 + immhi = (w >> 5) & 0x7FFFF + imm = (immhi << 2) | immlo + if imm & (1 << 20): + imm -= 1 << 21 + page_of[rd] = ((pc & ~0xFFF) + (imm << 12), pc) + elif (w & 0xFF800000) == 0x91000000: # add (immediate, 64-bit, no shift) + rn = (w >> 5) & 0x1F + if rn in page_of and pc - page_of[rn][1] < 64: + if page_of[rn][0] + ((w >> 10) & 0xFFF) == target: + hits.append(pc) + return hits + + +def main(): + args = sys.argv[1:] + if args and args[0] == "--xref": + elf = Elf(args[1]) + for a in args[2:]: + print(a, [hex(h) for h in xrefs(elf, int(a, 16))]) + return + max_ins = 400 + until = None + if "--until" in args: + i = args.index("--until") + until = int(args[i + 1], 16) + del args[i:i + 2] + if "--max" in args: + i = args.index("--max") + max_ins = int(args[i + 1]) + del args[i:i + 2] + elf = Elf(args[0]) + for a in args[1:]: + start = int(a, 16) + print(f"== {start:#x}") + disassemble(elf, start, max_ins if until is None else (until - start) // 4 + 1, until) + + +if __name__ == "__main__": + main() diff --git a/tools/shm_analyse.py b/tools/shm_analyse.py new file mode 100644 index 0000000..542dfe7 --- /dev/null +++ b/tools/shm_analyse.py @@ -0,0 +1,120 @@ +#!/usr/bin/env python3 +"""Offline analysis of a shm_capture.py recording (runs on the PC). + +Ground truth is the colour light value (g4) read from the +VR_CameraPassthroughState file, available only while RGB is selected. +Every 32-bit word of every other captured buffer is correlated with it +during the RGB phase; the best candidates are then printed across the +whole run so we can see whether they still follow the lights while mono +is selected. + +Usage: shm_analyse.py CAPTURE [--top N] [--min-r R] +""" + +import argparse +import struct +import zlib +from collections import defaultdict + +import numpy as np + +STATE_SIZE = 0x6200 +GAMMA = bytes.fromhex("2fbae83e") + + +def load(path): + series = defaultdict(list) + with open(path, "rb") as f: + data = f.read() + pos = 0 + while pos < len(data): + (n,) = struct.unpack_from(" best[0]: + best = (ts, light) + return best + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("capture") + ap.add_argument("--top", type=int, default=25) + ap.add_argument("--min-r", type=float, default=0.8) + args = ap.parse_args() + + series = load(args.capture) + state_name = next(n for n, s in series.items() if len(s[0][1]) == STATE_SIZE) + state = series.pop(state_name) + + # Ground truth: g4 while RGB is selected and the colour record is fresh. + truth_t, truth_v, mono_t = [], [], [] + last_ts = None + for t, buf in state: + rgb = buf[4] + if not rgb: + mono_t.append(t) + continue + cl = colour_light(buf) + if cl and cl[0] != last_ts: + truth_t.append(t) + truth_v.append(cl[1]) + last_ts = cl[0] + truth_t = np.array(truth_t) + truth_v = np.array(truth_v) + print(f"state file {state_name}: {len(state)} snapshots, {len(truth_t)} fresh RGB light samples, " + f"mono from {min(mono_t, default=float('nan')):.1f} to {max(mono_t, default=float('nan')):.1f} s") + print("light over time (RGB phase):", " ".join(f"{t:.0f}s={v:.2f}" for t, v in zip(truth_t[::8], truth_v[::8]))) + + candidates = [] + for name, snaps in series.items(): + times = np.array([t for t, _ in snaps]) + rgb_idx = [i for i, t in enumerate(times) if truth_t.size and truth_t[0] <= t <= truth_t[-1] + and not any(abs(t - m) < 0.5 for m in mono_t[:1] + mono_t[-1:]) + and not (mono_t and mono_t[0] - 1 <= t <= mono_t[-1] + 1)] + if len(rgb_idx) < 6: + continue + size = len(snaps[0][1]) // 4 * 4 + mat = np.frombuffer(b"".join(snaps[i][1][:size] for i in rgb_idx), dtype=" 0) & finite, (m * r0[:, None]).sum(axis=0) / denom, 0) + for w in np.argsort(-np.abs(r))[:args.top]: + if abs(r[w]) >= args.min_r: + candidates.append((abs(r[w]), r[w], name, int(w) * 4)) + + candidates.sort(reverse=True) + print(f"\n{len(candidates)} words with |r| >= {args.min_r} against the RGB light value") + for absr, r, name, off in candidates[:args.top]: + snaps = series[name] + vals = [struct.unpack_from("= args.seconds: + break + do_slow = now >= next_slow + if do_slow: + next_slow = now + args.slow + for p in paths: + if sizes[p] > args.large and not do_slow: + continue + try: + with open(p, "rb") as f: + data = f.read() + except OSError: + continue + z = zlib.compress(data, 1) + name = p.encode() + out.write(struct.pack(" best[0]): + best = (ts, rec, off) + if best and last.get((name, cam)) != best[0]: + last[(name, cam)] = best[0] + ts, rec, off = best + out.write(f"{now:.3f} {name} {cam} {rec} {ts:.6f} {buf[off:off + REC_SIZE].hex()}\n") + time.sleep(1 / a.hz) + + +if __name__ == "__main__": + main() diff --git a/tools/shm_sampler.py b/tools/shm_sampler.py new file mode 100644 index 0000000..be8d12f --- /dev/null +++ b/tools/shm_sampler.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python3 +"""Passively sample SteamVR's VR_CameraPassthroughState shared memory. + +Only reads a tmpfs file. It makes no SteamVR calls, takes no locks and +opens no devices, so it cannot disturb passthrough or tracking. Readings +may occasionally tear (the writer holds a mutex we deliberately ignore); +that is acceptable for exploration. + +Per-frame records are found by their gamma marker (1/2.2 as float32). +For each record whose frame counter changed, one CSV row is written with +the raw fields around the marker, so we can see which ones track light. + +With --raw PATH, every snapshot is also appended to PATH as +[float64 time][STATE_SIZE bytes] so it can be re-parsed offline. + +Usage: shm_sampler.py [--seconds N] [--hz N] [--out PATH] [--raw PATH] [--shm PATH] +""" + +import argparse +import glob +import os +import struct +import sys +import time + +GAMMA = struct.pack("= 0: + yield start, pos + pos = buf.find(GAMMA, pos + 4) + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--seconds", type=float, default=120) + ap.add_argument("--hz", type=float, default=20) + ap.add_argument("--out", default="shm_samples.csv") + ap.add_argument("--raw") + ap.add_argument("--shm") + args = ap.parse_args() + path = args.shm or find_state_file() + + header = ["t", "offset", "stream", "id", "frame", "ts0", "ts1", "fx", "fy", "cx", "cy"] + header += [f"g{i}" for i in range(FIELDS_AFTER_GAMMA)] + ["owner", "seq", "config"] + last_frame = {} + t0 = time.monotonic() + raw = open(args.raw, "wb") if args.raw else None + with open(args.out, "w") as out, open(path, "rb") as f: + out.write(",".join(header) + "\n") + while time.monotonic() - t0 < args.seconds: + f.seek(0) + buf = f.read(STATE_SIZE) + now = time.monotonic() - t0 + if raw: + raw.write(struct.pack("= 0x20E8 else 0 + row = [f"{now:.3f}", hex(start), str(stream), str(ident), str(frame), f"{ts0:.4f}", f"{ts1:.4f}", + f"{fx:.1f}", f"{fy:.1f}", f"{cx:.1f}", f"{cy:.1f}"] + row += [f"{v:.6g}" for v in fields] + [str(owner), str(seq), config] + out.write(",".join(row) + "\n") + time.sleep(1 / args.hz) + if raw: + raw.close() + + +if __name__ == "__main__": + main()