mirror of
https://github.com/hakumaguro/vrcft-steam-frame.git
synced 2026-10-06 07:00:22 +02:00
Review fixes: safer calibration and config, neutral eyes without data, pinned frameeyeosc (0.2.2)
- tune.py: reject reversed/flat per-eye ranges and keep the previous config; recommend wink assist; readiness check before recording; skip malformed trace lines - module: invalid steamframe-config.json is not applied (last good config kept, configError reported); neutral eyes when no source is fresh; trace written with invariant culture - setup: sandbox mode leaves real SteamVR settings and firewall alone; block rules handled separately; headset target only counts connected adapters - headset-setup.sh: build a pinned, reviewed frameeyeosc revision and move existing checkouts to it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
f7a9140208
commit
f5ec1458fe
10 files changed
+230
-42
No files matched your search
@@ -5,7 +5,7 @@ Eye tracking for the Steam Frame in VRChat via VRCFaceTracking (VRCFT). Read `RE
|
||||
|
||||
## Current state (2026-09-27)
|
||||
|
||||
- Latest release **v0.2.1** (2026-09-27; fixes the Module Registry crash via a complete `module.json`), with a prebuilt-DLL zip. Working end to end on the author's setup: per-eye gaze, blinks, both
|
||||
- Module **0.2.2** committed locally (review fixes below), not yet pushed or released. Latest release **v0.2.1** (2026-09-27; fixes the Module Registry crash via a complete `module.json`), with a prebuilt-DLL zip. Working end to end on the author's setup: per-eye gaze, blinks, both
|
||||
winks, wink assist; frameeyeosc auto-starts on the headset and survived a real reboot; `doctor.ps1` reports "All good".
|
||||
- `setup.ps1 -Headset` (menu 2) was run for real on 2026-09-27 against an already-set-up Frame: SSH login, target kept, build no-op,
|
||||
unit refreshed, service active, doctor "All good"; a second run changed nothing. Still never exercised: the first-time path on a fresh
|
||||
@@ -13,6 +13,22 @@ Eye tracking for the Steam Frame in VRChat via VRCFaceTracking (VRCFT). Read `RE
|
||||
- Open ideas: verify the gaze scale (frameeyeosc +-1 == +-45 deg -> radians) against a reference; a first-run check in `Start Here.cmd`
|
||||
that VRCFT is running (users forget to start it from Steam each session; the doctor catches it).
|
||||
|
||||
## Review fixes in 0.2.2 (2026-09-27, from an external review; all verified against the code first)
|
||||
|
||||
- `tune.py analyze` rejects a calibration whose per-eye open-closed range is < 0.15 or reversed (was a ZeroDivisionError or bad
|
||||
values), keeps the previous config, and says so. It now also recommends `wink.assist` + `assistMin` (other eye's both-closed p75 + 0.08).
|
||||
- `tune.py record` checks first (status file) that VRCFT runs and frameeyeosc data is fresh, and speaks what is missing.
|
||||
- `ModuleConfig.TryLoad` + `Validate`: a broken or invalid `steamframe-config.json` is not applied; the module keeps the last good
|
||||
config and reports `configError` (status file, log, doctor). Previously it silently fell back to defaults.
|
||||
- Without fresh data from either source the module sets neutral eyes (open, centred) once, instead of replaying stale Steam Link values.
|
||||
- Trace lines are written with `FormattableString.Invariant` (decimal-comma locales broke the CSV); `tune.py` skips malformed lines.
|
||||
- `setup.ps1` sandbox mode (`-VrcftData`) no longer touches the real SteamVR settings (unless `-SteamVrSettings <file>`) or the firewall.
|
||||
- Firewall: a *block* rule is handled separately (disable it + add allow, elevated); an allow rule alone does not beat a block rule.
|
||||
- `headset-setup.sh` pins frameeyeosc to a reviewed commit (`REV`, override `FRAMEEYEOSC_REV`) and moves existing checkouts to it;
|
||||
`FRAMEEYEOSC_NO_BUILD=1` tests only the checkout logic (works locally with `HOME=<tmp>`).
|
||||
- `Select-HeadsetTarget` only counts addresses of connected adapters (Windows lists a disconnected adapter's IP).
|
||||
Not done yet: offline replay tests for the eyelid pipeline (the replays in this history were ad-hoc scripts).
|
||||
|
||||
## Working with the user
|
||||
|
||||
The user is in VR most of the time and cannot read text in the headset. Prefer spoken output (`Say` in `common.ps1`, `speak` in `tune.py`),
|
||||
@@ -51,6 +67,9 @@ frameeyeosc sends `/avatar/parameters/FT/v2/{EyeLeftX,EyeLeftY,EyeRightX,EyeRigh
|
||||
- Tracker levels differ per eye: closed about L 0.14 / R 0.32, open about L 0.65 / R 0.75 (calibrations 2026-09-26).
|
||||
- Many blinks are reported lopsided: one lid ~0 while the other is pinned at raw ~1.0, up to ~350 ms. Real winks keep the open eye ~0.7-0.85.
|
||||
- Closing one eye tightens the other: during a right wink the left lid reads ~0.23-0.45 (vs ~0.14 when both are closed).
|
||||
- The Frame can reach the PC over two paths: its own Wi-Fi AP (PC "Wi-Fi 2", 10.35.78.x) or the home network. Which one Steam Link
|
||||
uses can change between sessions; the headset's route to the AP subnet disappears when the PC's Wi-Fi 2 is disconnected, and packets to
|
||||
the AP address then go to the home router and vanish. Target the PC's home-network address (the `$SSH_CLIENT` address) when both exist.
|
||||
- After a headset reboot the network comes up after the service starts; frameeyeosc exits with "Network is unreachable" and systemd
|
||||
`Restart=always` retries until it works (observed 2026-09-27). This is expected.
|
||||
- Avatars can link the eyelids themselves: Haku_FT_Comfy has a Bool `FT/EyeSync` toggle; with it on, a wink closes both eyes. Check the
|
||||
|
||||
@@ -82,6 +82,8 @@ and `~/.cargo/bin/rustup self uninstall` for Rust.
|
||||
| Winking closes **both** avatar eyes | The avatar links its eyelids: open the avatar menu and turn off **Eye Sync** / "Link eyes". |
|
||||
| Status says "no eye data" | Wear the headset and connect Steam Link. For eyelids, the headset part must run: menu **2** sets it up again. |
|
||||
| Status says "only the Steam Link fallback" | Gaze works, eyelids are guessed. The headset part is not sending: menu **2**, or on the headset `systemctl --user status frameeyeosc`. |
|
||||
| Eyelids stopped after a headset reboot or network change ("only the Steam Link fallback") | The headset may now reach the PC over a different network. Run menu **2** again: it picks the address that works and keeps the rest. |
|
||||
| Status says "your settings file has an error" | The module ignored the broken `steamframe-config.json` and kept the previous settings. Fix it, restore `steamframe-config.json.bak`, or calibrate again (menu 4). |
|
||||
| Status says "started without its modules" | Close VRCFaceTracking and start it from Steam. |
|
||||
| Status mentions the old SteamLink module / port 9015 | Menu **1** moves the stock SteamLink module aside (menu **7** puts it back). |
|
||||
| Eyes look wrong after a Frame update | Menu **2** again; if the headset reports a new eye-data version, the headset part needs an update upstream. |
|
||||
|
||||
+39
-7
@@ -90,16 +90,48 @@ public sealed class ModuleConfig
|
||||
DefaultIgnoreCondition = JsonIgnoreCondition.Never,
|
||||
};
|
||||
|
||||
public static ModuleConfig Load(string path)
|
||||
/// <summary>Load and validate. Returns null (with a reason) when the file is unreadable or invalid, so the caller can keep
|
||||
/// the configuration it already has instead of silently dropping the user's calibration. A missing file gets the defaults.</summary>
|
||||
public static ModuleConfig? TryLoad(string path, out string error)
|
||||
{
|
||||
error = "";
|
||||
try
|
||||
{
|
||||
if (File.Exists(path))
|
||||
return JsonSerializer.Deserialize<ModuleConfig>(File.ReadAllText(path), Json) ?? new ModuleConfig();
|
||||
var cfg = new ModuleConfig();
|
||||
File.WriteAllText(path, JsonSerializer.Serialize(cfg, Json));
|
||||
return cfg;
|
||||
if (!File.Exists(path))
|
||||
{
|
||||
var def = new ModuleConfig();
|
||||
File.WriteAllText(path, JsonSerializer.Serialize(def, Json));
|
||||
return def;
|
||||
}
|
||||
var cfg = JsonSerializer.Deserialize<ModuleConfig>(File.ReadAllText(path), Json);
|
||||
if (cfg == null) { error = "the file is empty or null"; return null; }
|
||||
error = cfg.Validate();
|
||||
return error == "" ? cfg : null;
|
||||
}
|
||||
catch { return new ModuleConfig(); }
|
||||
catch (Exception e) { error = e.Message; return null; }
|
||||
}
|
||||
|
||||
/// <summary>Empty string when usable, otherwise the first problem found.</summary>
|
||||
public string Validate()
|
||||
{
|
||||
if (Lid == null || Wink == null || Blink == null || Gaze == null) return "a section (lid, wink, blink or gaze) is missing or null";
|
||||
bool Bad(float v, float lo, float hi) => !float.IsFinite(v) || v < lo || v > hi;
|
||||
if (Bad(Lid.Deadband, 0f, 0.45f)) return "lid.deadband must be 0..0.45";
|
||||
if (Bad(Lid.Smoothing, 0f, 1f) || Bad(Lid.Tau, 0f, 1f)) return "lid.smoothing and lid.tau must be 0..1";
|
||||
if (Bad(Lid.MaxFloor, 0f, 1f) || Bad(Lid.MinCeil, 0f, 1f) || Bad(Lid.MinRange, 0.01f, 1f)) return "lid.maxFloor/minCeil/minRange out of range";
|
||||
foreach (var (c, o, eye) in new[] { (Lid.LeftClosed, Lid.LeftOpen, "left"), (Lid.RightClosed, Lid.RightOpen, "right") })
|
||||
{
|
||||
if (c is float fc && Bad(fc, 0f, 1f)) return $"lid.{eye}Closed must be 0..1";
|
||||
if (o is float fo && Bad(fo, 0f, 1f)) return $"lid.{eye}Open must be 0..1";
|
||||
if (c is float a && o is float b && b - a < 0.05f) return $"lid.{eye}Open must be above lid.{eye}Closed";
|
||||
}
|
||||
if (Bad(Wink.Threshold, 0f, 1f) || Bad(Wink.Range, 0.01f, 2f) || Bad(Wink.Strength, 0f, 1f)) return "wink.threshold/range/strength out of range";
|
||||
if (Bad(Wink.AssistOpen, 0f, 1f) || Bad(Wink.AssistClosed, 0f, 1f) || Bad(Wink.AssistMin, 0f, 1f)) return "wink.assist* levels must be 0..1";
|
||||
if (Wink.AssistPersistMs < 0 || Wink.AssistReleaseMs < 0) return "wink.assist*Ms must not be negative";
|
||||
if (Blink.HoldMs < 0 || Blink.CoupleMs < 0 || Blink.GlitchMinMs < 0) return "blink.*Ms must not be negative";
|
||||
if (Bad(Blink.ReleasePerSec, 0f, 1000f) || Bad(Blink.AsymClosed, 0f, 1f) || Bad(Blink.AsymOpen, 0f, 1f) || Bad(Blink.SaturatedRaw, 0f, 1f))
|
||||
return "blink.* values out of range";
|
||||
if (Bad(Gaze.Scale, 0f, 10f)) return "gaze.scale must be 0..10";
|
||||
return "";
|
||||
}
|
||||
}
|
||||
@@ -41,6 +41,8 @@ public class SteamFrameVRCFTModule : ExtTrackingModule
|
||||
private string _source = "";
|
||||
private long _lastStatus;
|
||||
private string _startError = "";
|
||||
private string _configError = "";
|
||||
private bool _neutralSent;
|
||||
|
||||
private OscReceiver? _osc, _frame;
|
||||
private long _lastTrace, _lastCfgCheck, _lastLidTick;
|
||||
@@ -173,7 +175,18 @@ public class SteamFrameVRCFTModule : ExtTrackingModule
|
||||
{
|
||||
var stamp = File.Exists(_cfgPath) ? File.GetLastWriteTimeUtc(_cfgPath) : DateTime.MinValue;
|
||||
if (!force && stamp == _cfgStamp) return;
|
||||
_cfg = ModuleConfig.Load(_cfgPath);
|
||||
_cfgStamp = stamp;
|
||||
var next = ModuleConfig.TryLoad(_cfgPath, out var error);
|
||||
if (next == null)
|
||||
{
|
||||
// keep the configuration that was working; say why the new one was not used
|
||||
_configError = $"steamframe-config.json was not applied ({error}); still using the previous settings";
|
||||
FileLog(_configError);
|
||||
Logger.LogWarning("{0}", _configError);
|
||||
return;
|
||||
}
|
||||
_cfg = next;
|
||||
_configError = "";
|
||||
_cfgStamp = File.Exists(_cfgPath) ? File.GetLastWriteTimeUtc(_cfgPath) : DateTime.MinValue;
|
||||
FileLog("config loaded");
|
||||
}
|
||||
@@ -196,7 +209,8 @@ public class SteamFrameVRCFTModule : ExtTrackingModule
|
||||
}
|
||||
bool steamLinkFresh = _osc != null && _osc.LastPacketTicks != 0 && t0 - _osc.LastPacketTicks < 1000;
|
||||
SetSource(steamLinkFresh ? "steamlink" : "none");
|
||||
if (_osc != null) UpdateFromSteamLink();
|
||||
if (steamLinkFresh) { _neutralSent = false; UpdateFromSteamLink(); }
|
||||
else SetNeutralEyes();
|
||||
}
|
||||
|
||||
// ---- source 1: frameeyeosc (per-eye) -------------------------------------------------------------
|
||||
@@ -253,7 +267,8 @@ public class SteamFrameVRCFTModule : ExtTrackingModule
|
||||
if (c.Trace && nowMs - _lastTrace >= 30) // off by default; tools/tune.py switches it on while it needs samples
|
||||
{
|
||||
_lastTrace = nowMs;
|
||||
AppendCapped(TracePath, $"{DateTime.Now:HH:mm:ss.fff},FRAME,{f.lx:F3},{f.ly:F3},{f.rx:F3},{f.ry:F3},{f.ol:F3},{f.or:F3},{mol:F3},{mor:F3},{_calL.Lo:F2},{_calL.Hi:F2},{_calR.Lo:F2},{_calR.Hi:F2}{Environment.NewLine}", MaxTraceBytes);
|
||||
// invariant culture: with a decimal comma the CSV would gain extra columns and tune.py would misread it
|
||||
AppendCapped(TracePath, FormattableString.Invariant($"{DateTime.Now:HH:mm:ss.fff},FRAME,{f.lx:F3},{f.ly:F3},{f.rx:F3},{f.ry:F3},{f.ol:F3},{f.or:F3},{mol:F3},{mor:F3},{_calL.Lo:F2},{_calL.Hi:F2},{_calR.Lo:F2},{_calR.Hi:F2}") + Environment.NewLine, MaxTraceBytes);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -299,12 +314,27 @@ public class SteamFrameVRCFTModule : ExtTrackingModule
|
||||
eye._maxDilation = 10; eye._minDilation = 0;
|
||||
}
|
||||
|
||||
/// <summary>No fresh data from either source: look straight ahead with open eyes instead of freezing on the last reading
|
||||
/// (which could be mid-blink). Sent once per no-data period.</summary>
|
||||
private void SetNeutralEyes()
|
||||
{
|
||||
if (_neutralSent) return;
|
||||
_neutralSent = true;
|
||||
_closed = 0f; _gx = 0f; _gy = 0f;
|
||||
var e = UnifiedTracking.Data.Eye;
|
||||
e.Left.Gaze.x = 0f; e.Left.Gaze.y = 0f; e.Right.Gaze.x = 0f; e.Right.Gaze.y = 0f;
|
||||
e.Left.Openness = 1f; e.Right.Openness = 1f;
|
||||
e.Left.PupilDiameter_MM = 5f; e.Right.PupilDiameter_MM = 5f;
|
||||
e._maxDilation = 10; e._minDilation = 0;
|
||||
}
|
||||
|
||||
// ---- status for tools/tune.py doctor ---------------------------------------------------------------
|
||||
|
||||
private void SetSource(string s)
|
||||
{
|
||||
if (s == _source) return;
|
||||
_source = s;
|
||||
if (s != "none") _neutralSent = false;
|
||||
FileLog(s switch
|
||||
{
|
||||
"frameeyeosc" => "eye data: frameeyeosc (per-eye gaze and eyelids)",
|
||||
@@ -331,6 +361,7 @@ public class SteamFrameVRCFTModule : ExtTrackingModule
|
||||
["trace"] = _cfg.Trace,
|
||||
["config"] = _cfgPath,
|
||||
["startError"] = _startError,
|
||||
["configError"] = _configError,
|
||||
};
|
||||
try { File.WriteAllText(StatusPath, System.Text.Json.JsonSerializer.Serialize(status)); } catch { }
|
||||
}
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<AssemblyName>SteamFrameVRCFTModule</AssemblyName>
|
||||
<Version>0.2.1</Version>
|
||||
<Version>0.2.2</Version>
|
||||
<VrcftDir Condition="'$(VrcftDir)' == ''">C:\Program Files (x86)\Steam\steamapps\common\VRCFaceTracking</VrcftDir>
|
||||
</PropertyGroup>
|
||||
<ItemGroup>
|
||||
|
||||
+8
-3
@@ -4,7 +4,7 @@
|
||||
$script:SteamAppId = "3329480" # VRCFaceTracking on Steam
|
||||
$script:ModuleDll = "SteamFrameVRCFTModule.dll"
|
||||
$script:StockDll = "SteamLinkVRCFTModule.dll"
|
||||
$script:ModuleVersion = "0.2.1"
|
||||
$script:ModuleVersion = "0.2.2"
|
||||
$script:RepoUrl = "https://github.com/hakumaguro/vrcft-steam-frame"
|
||||
|
||||
function Get-VrcftDataDir {
|
||||
@@ -102,7 +102,8 @@ function Test-ModuleFirewall {
|
||||
$exe = Join-Path $VrcftDir "VRCFaceTracking.ModuleProcess.exe"
|
||||
$rules = @(Get-NetFirewallApplicationFilter -ErrorAction SilentlyContinue | Where-Object { $_.Program -ieq $exe } |
|
||||
Get-NetFirewallRule -ErrorAction SilentlyContinue | Where-Object { "$($_.Enabled)" -eq "True" -and "$($_.Direction)" -eq "Inbound" })
|
||||
if ($rules | Where-Object { "$($_.Action)" -eq "Block" }) { return @{ State = "blocked"; Exe = $exe } }
|
||||
$block = @($rules | Where-Object { "$($_.Action)" -eq "Block" })
|
||||
if ($block.Count -gt 0) { return @{ State = "blocked"; Exe = $exe; BlockRules = @($block | ForEach-Object { $_.Name }) } }
|
||||
$allow = @($rules | Where-Object { "$($_.Action)" -eq "Allow" })
|
||||
$cats = @(Get-NetConnectionProfile -ErrorAction SilentlyContinue | ForEach-Object { ("$($_.NetworkCategory)" -replace 'DomainAuthenticated', 'Domain') } | Select-Object -Unique)
|
||||
$missing = @()
|
||||
@@ -159,7 +160,11 @@ function Find-DotnetSdk10 {
|
||||
# addresses (DHCP can change it); otherwise uses the address the headset's SSH session came from.
|
||||
function Select-HeadsetTarget {
|
||||
param([string]$Override, [string]$Existing, [string]$Client, [string[]]$LocalIps)
|
||||
if (-not $LocalIps) { $LocalIps = @(Get-NetIPAddress -AddressFamily IPv4 -ErrorAction SilentlyContinue | ForEach-Object { $_.IPAddress }) }
|
||||
if (-not $LocalIps) {
|
||||
# connected adapters only: Windows keeps listing the address of a disconnected adapter (seen with the Frame's own Wi-Fi)
|
||||
$up = @(Get-NetAdapter -ErrorAction SilentlyContinue | Where-Object { $_.Status -eq "Up" } | ForEach-Object { $_.ifIndex })
|
||||
$LocalIps = @(Get-NetIPAddress -AddressFamily IPv4 -ErrorAction SilentlyContinue | Where-Object { $up -contains $_.InterfaceIndex } | ForEach-Object { $_.IPAddress })
|
||||
}
|
||||
if ($Override) { return @{ Target = $Override; Reason = "as requested" } }
|
||||
$old = ""
|
||||
if ($Existing) { $old = ($Existing -split ':')[0] }
|
||||
|
||||
@@ -42,6 +42,7 @@ if ($vrcftRunning -and $moduleProc) {
|
||||
else { Fail "the Steam Frame module is not active (status is $([int]($st.AgeMs/1000)) s old)" "The Steam Frame module is not active." "check the Modules page in VRCFaceTracking; if another module claims eye tracking, disable it" }
|
||||
} else {
|
||||
if ($st.startError) { Warn "module reports: $($st.startError)" "The module reported a start problem. See the screen." "" }
|
||||
if ($st.configError) { Warn "$($st.configError)" "Your settings file has an error, so the previous settings are still used." "fix the file (menu 6 opens its folder) or restore steamframe-config.json.bak" }
|
||||
switch ($st.source) {
|
||||
"frameeyeosc" { Write-Step ok "eye data from the headset (per-eye gaze and eyelids), $($st.frameAgeMs) ms old" }
|
||||
"steamlink" { Warn "only the Steam Link fallback is sending: gaze works, eyelids are guessed" "Only the Steam Link fallback is working. Eyelids are guessed. The headset part is not sending." "on the headset: systemctl --user status frameeyeosc (or run setup with -Headset)" }
|
||||
|
||||
@@ -1,13 +1,31 @@
|
||||
#!/bin/bash
|
||||
# Run ON the Steam Frame (SteamOS, aarch64): installs Rust for the current user and builds frameeyeosc.
|
||||
# Run ON the Steam Frame (SteamOS, aarch64): installs Rust for the current user and builds frameeyeosc at a pinned,
|
||||
# reviewed revision. Re-running moves an existing checkout to that revision, so updating this script updates the headset.
|
||||
# Nothing outside $HOME is touched. Undo with: ~/.cargo/bin/rustup self uninstall; rm -rf ~/frameeyeosc
|
||||
set -euo pipefail
|
||||
if ! command -v ~/.cargo/bin/cargo >/dev/null 2>&1; then
|
||||
|
||||
# frameeyeosc revision this project was tested with (review the upstream diff before changing it)
|
||||
REV="${FRAMEEYEOSC_REV:-b9f0c017a0b5c08d2139891bdcdabe78746a5245}"
|
||||
REPO="https://github.com/konsti219/frameeyeosc"
|
||||
SRC="$HOME/frameeyeosc"
|
||||
|
||||
if [ -z "${FRAMEEYEOSC_NO_BUILD:-}" ] && ! command -v ~/.cargo/bin/cargo >/dev/null 2>&1; then
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain stable
|
||||
fi
|
||||
|
||||
if [ ! -d "$SRC/.git" ]; then
|
||||
git init -q "$SRC"
|
||||
git -C "$SRC" remote add origin "$REPO"
|
||||
fi
|
||||
if [ "$(git -C "$SRC" rev-parse -q --verify HEAD 2>/dev/null || true)" != "$REV" ]; then
|
||||
git -C "$SRC" fetch -q --depth 1 origin "$REV"
|
||||
git -C "$SRC" checkout -q --detach FETCH_HEAD
|
||||
fi
|
||||
[ "$(git -C "$SRC" rev-parse HEAD)" = "$REV" ] || { echo "frameeyeosc checkout is not at $REV"; exit 1; }
|
||||
echo "frameeyeosc commit $REV"
|
||||
|
||||
[ -n "${FRAMEEYEOSC_NO_BUILD:-}" ] && exit 0 # testing the checkout logic only
|
||||
. ~/.cargo/env
|
||||
[ -d ~/frameeyeosc ] || git clone --depth 1 https://github.com/konsti219/frameeyeosc ~/frameeyeosc
|
||||
cd ~/frameeyeosc
|
||||
git log -1 --format='frameeyeosc commit %H'
|
||||
cd "$SRC"
|
||||
cargo build --release
|
||||
ls -l target/release/frameeyeosc
|
||||
+34
-15
@@ -27,13 +27,16 @@ param(
|
||||
[string]$KeyFile = "$env:USERPROFILE\.ssh\id_ed25519_frame",
|
||||
[switch]$NoSpeech,
|
||||
[switch]$NoRestart, # do not stop/start VRCFaceTracking (for testing against -VrcftData)
|
||||
[string]$VrcftData # VRCFT data folder override (testing)
|
||||
[string]$VrcftData, # VRCFT data folder override (testing). Sandbox mode: SteamVR settings and firewall are not touched
|
||||
[string]$SteamVrSettings # steamvr.vrsettings to use instead of Steam's (testing; with -VrcftData only this file is edited)
|
||||
)
|
||||
$ErrorActionPreference = "Continue" # native tools (ssh, dotnet) write to stderr; each change below stops on its own errors
|
||||
. "$PSScriptRoot\common.ps1"
|
||||
$script:NoSpeech = $NoSpeech
|
||||
$root = Split-Path -Parent $PSScriptRoot
|
||||
$data = Get-VrcftDataDir $VrcftData
|
||||
$sandbox = [bool]$VrcftData
|
||||
if ($sandbox) { Write-Host "SANDBOX: VRCFT data in $VrcftData; SteamVR settings only if -SteamVrSettings is given; firewall skipped.`n" -ForegroundColor Cyan }
|
||||
$parked = Join-Path $data "disabled-modules"
|
||||
$changed = @() # human-readable list of changes made
|
||||
$needVrcftRestart = $false
|
||||
@@ -137,11 +140,13 @@ if ($fixOutPort) { Write-Step warn "VRCFaceTracking sends to port $outPort (VRCh
|
||||
elseif ($outPort) { Write-Step ok "VRCFaceTracking sends to VRChat on port 9000" }
|
||||
|
||||
# ---- 5. Steam Link driver settings ---------------------------------------------------------------------
|
||||
$vrPath = Get-SteamVrSettingsPath
|
||||
$vrPath = $SteamVrSettings
|
||||
if (-not $vrPath -and -not $sandbox) { $vrPath = Get-SteamVrSettingsPath }
|
||||
$vr = Get-VrlinkSettings $vrPath
|
||||
if ($vr.Ok) { Write-Step ok "Steam Link shares eye data on port 9015" }
|
||||
if ($sandbox -and -not $vrPath) { Write-Step skip "sandbox: the real SteamVR settings are not touched" }
|
||||
elseif ($vr.Ok) { Write-Step ok "Steam Link shares eye data on port 9015" }
|
||||
elseif (-not $vrPath -or -not (Test-Path $vrPath)) { Write-Step warn "steamvr.vrsettings not found (start SteamVR once, then run setup again)" }
|
||||
elseif (Test-SteamVrRunning) {
|
||||
elseif ((-not $SteamVrSettings) -and (Test-SteamVrRunning)) {
|
||||
Write-Step warn "Steam Link OSC settings need changing, but SteamVR is running (it would overwrite the change)."
|
||||
Write-Host " Close SteamVR and run setup again, or in SteamVR: Settings > Advanced > Video > Steam Link: OSC Output Port 9015, share eye tracking on." -ForegroundColor DarkYellow
|
||||
} else {
|
||||
@@ -214,22 +219,36 @@ if ($needVrcftRestart) {
|
||||
}
|
||||
|
||||
# ---- 7. firewall --------------------------------------------------------------------------------------
|
||||
$fw = Test-ModuleFirewall $vrcft
|
||||
if ($fw.State -eq "ok") { Write-Step ok "firewall allows the module process" }
|
||||
if ($sandbox) { Write-Step skip "sandbox: firewall not checked" }
|
||||
else {
|
||||
$why = "blocked"
|
||||
if ($fw.State -eq "missing") { $why = "no allow rule on: $($fw.Missing) network" }
|
||||
Write-Step warn "firewall: $why (headset data on UDP 9020 may not arrive)"
|
||||
if ($DryRun) { Write-Step would "add an inbound UDP 9020 allow rule for the module process (admin prompt)" }
|
||||
elseif (Ask "Add a firewall rule allowing UDP 9020 to the module? (Windows will ask for admin)") {
|
||||
$cmd = "New-NetFirewallRule -DisplayName 'Steam Frame eye tracking (VRCFT module)' -Direction Inbound -Action Allow -Protocol UDP -LocalPort 9020 -Program '$($fw.Exe)' -Profile Any"
|
||||
$fw = Test-ModuleFirewall $vrcft
|
||||
$allowCmd = "New-NetFirewallRule -DisplayName 'Steam Frame eye tracking (VRCFT module)' -Direction Inbound -Action Allow -Protocol UDP -LocalPort 9020 -Program '$($fw.Exe)' -Profile Any"
|
||||
$fwCmd = $null; $fwQuestion = $null
|
||||
if ($fw.State -eq "ok") { Write-Step ok "firewall allows the module process" }
|
||||
elseif ($fw.State -eq "blocked") {
|
||||
# A block rule wins over any allow rule, so adding an allow rule alone would not help.
|
||||
Write-Step warn "firewall: a rule BLOCKS the module process (Windows adds one when its network prompt is answered with Cancel)"
|
||||
$names = ($fw.BlockRules | ForEach-Object { "'" + $_ + "'" }) -join ","
|
||||
$fwCmd = "Get-NetFirewallRule -Name $names | Disable-NetFirewallRule; $allowCmd"
|
||||
$fwQuestion = "Disable the blocking rule(s) for the module and add an allow rule for UDP 9020? (Windows will ask for admin)"
|
||||
} else {
|
||||
Write-Step warn "firewall: no allow rule on: $($fw.Missing) network (headset data on UDP 9020 may not arrive)"
|
||||
$fwCmd = $allowCmd
|
||||
$fwQuestion = "Add a firewall rule allowing UDP 9020 to the module? (Windows will ask for admin)"
|
||||
}
|
||||
if ($fwCmd) {
|
||||
if ($DryRun) { Write-Step would ($fwQuestion -replace '\?.*$', '') }
|
||||
elseif (Ask $fwQuestion) {
|
||||
# -EncodedCommand: Start-Process joins arguments without quoting, which breaks paths with spaces
|
||||
$enc = [Convert]::ToBase64String([Text.Encoding]::Unicode.GetBytes($cmd))
|
||||
$enc = [Convert]::ToBase64String([Text.Encoding]::Unicode.GetBytes($fwCmd))
|
||||
Start-Process powershell -Verb RunAs -Wait -ArgumentList "-NoProfile", "-EncodedCommand", $enc
|
||||
if ((Test-ModuleFirewall $vrcft).State -eq "ok") { Write-Step done "firewall rule added"; $changed += "firewall rule" }
|
||||
else { Write-Step warn "firewall rule not confirmed (admin prompt declined?)" }
|
||||
$after = (Test-ModuleFirewall $vrcft).State
|
||||
if ($after -eq "ok") { Write-Step done "firewall now allows the module"; $changed += "firewall" }
|
||||
elseif ($after -eq "blocked") { Write-Step warn "a blocking rule is still active (admin prompt declined, or the rule is set by policy); check Windows Defender Firewall > Inbound Rules" }
|
||||
else { Write-Step warn "no allow rule yet (admin prompt declined?)" }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ---- 8. headset (optional) ----------------------------------------------------------------------------
|
||||
if ($Headset) {
|
||||
|
||||
+68
-7
@@ -47,7 +47,8 @@ PROTOCOL = [
|
||||
("open", 2, "Done."),
|
||||
]
|
||||
GUARD_START, GUARD_END = 1.3, 0.2
|
||||
SPOKEN = [] # short sentences for the spoken result of analyze # seconds skipped at each segment edge (prompt + reaction time)
|
||||
SPOKEN = [] # short sentences for the spoken result of analyze
|
||||
MIN_LID_RANGE = 0.15 # raw open level must be at least this far above the closed level, per eye # seconds skipped at each segment edge (prompt + reaction time)
|
||||
|
||||
|
||||
# ---- helpers ---------------------------------------------------------------------------------------
|
||||
@@ -141,14 +142,20 @@ def speak_wait(text):
|
||||
|
||||
def read_trace(path):
|
||||
"""FRAME rows: time,FRAME,lx,ly,rx,ry,rawL,rawR,outL,outR,..."""
|
||||
rows = []
|
||||
rows, bad = [], 0
|
||||
with open(path, newline="", encoding="utf-8", errors="replace") as f:
|
||||
for p in csv.reader(f):
|
||||
if len(p) >= 10 and p[1] == "FRAME":
|
||||
try:
|
||||
rows.append((tod(p[0]), float(p[6]), float(p[7]), float(p[8]), float(p[9])))
|
||||
except ValueError:
|
||||
pass
|
||||
if len(p) < 2 or p[1] != "FRAME":
|
||||
continue
|
||||
if len(p) not in (12, 14): # e.g. numbers written with a decimal comma split into extra columns
|
||||
bad += 1
|
||||
continue
|
||||
try:
|
||||
rows.append((tod(p[0]), float(p[6]), float(p[7]), float(p[8]), float(p[9])))
|
||||
except ValueError:
|
||||
bad += 1
|
||||
if bad:
|
||||
print(f"warning: skipped {bad} malformed trace lines in {path}")
|
||||
return rows
|
||||
|
||||
|
||||
@@ -159,7 +166,30 @@ def pct(v, p):
|
||||
|
||||
# ---- record ----------------------------------------------------------------------------------------
|
||||
|
||||
STATUS = os.path.join(os.environ.get("TEMP", "."), "steamframe-status.json")
|
||||
|
||||
|
||||
def ready_or_exit():
|
||||
"""Before asking the user to sit through a recording, check VRCFT and fresh headset data; say what is missing."""
|
||||
msg = None
|
||||
try:
|
||||
stt = json.load(open(STATUS, encoding="utf-8"))
|
||||
age = time.time() * 1000 - stt.get("updated", 0)
|
||||
except Exception:
|
||||
stt, age = {}, 1e12
|
||||
if age > 5000:
|
||||
msg = ("VRCFaceTracking is not running, or the Steam Frame module is not active. Start VRCFaceTracking from Steam.",
|
||||
"VRCFaceTracking / the module is not running (no fresh status). Start VRCFaceTracking from Steam, then run the status check.")
|
||||
elif stt.get("source") != "frameeyeosc":
|
||||
msg = ("No eyelid data from the headset. Put the headset on, and check the headset part with the status check.",
|
||||
f"No frameeyeosc data (source: {stt.get('source')}). Wear the headset; run scripts\\doctor.ps1 if it persists.")
|
||||
if msg:
|
||||
speak_wait(msg[0])
|
||||
sys.exit(msg[1])
|
||||
|
||||
|
||||
def cmd_record(a):
|
||||
ready_or_exit()
|
||||
with Tracing():
|
||||
if not wait_for_samples():
|
||||
speak_wait("No eye data from the headset, so calibration cannot start. Run the status check.")
|
||||
@@ -246,6 +276,13 @@ def cmd_analyze(a):
|
||||
print(f" left : open p20 {cal['leftOpen']:.2f} (median {st.median(L(g['open'])):.2f}) closed p80 {cal['leftClosed']:.2f} (median {st.median(L(g['closed'])):.2f})")
|
||||
print(f" right: open p20 {cal['rightOpen']:.2f} (median {st.median(R(g['open'])):.2f}) closed p80 {cal['rightClosed']:.2f} (median {st.median(R(g['closed'])):.2f})")
|
||||
warn = []
|
||||
bad = [eye for eye in ("left", "right") if cal[eye + "Open"] - cal[eye + "Closed"] < MIN_LID_RANGE]
|
||||
if bad:
|
||||
which = " and ".join(bad)
|
||||
print(f"\nCalibration rejected: the {which} eye's open and closed readings are too close together or reversed "
|
||||
f"(need open at least {MIN_LID_RANGE} above closed). Nothing was changed; your previous settings are kept.")
|
||||
speak_wait(f"Calibration failed for the {which} eye, so nothing was changed. Keep the eye fully open and fully closed when asked, and try again.")
|
||||
sys.exit(2)
|
||||
for eye in ("left", "right"):
|
||||
if cal[eye + "Open"] - cal[eye + "Closed"] < 0.25:
|
||||
warn.append(f"{eye} eye: open and closed levels are only {cal[eye + 'Open'] - cal[eye + 'Closed']:.2f} apart, the tracker barely sees it close")
|
||||
@@ -281,6 +318,30 @@ def cmd_analyze(a):
|
||||
rec["wink"]["range"] = round(max(0.10, min(0.40, (st.median(sep) if sep else 0.4) * 0.7)), 3)
|
||||
print(f" open-eye L/R mismatch p95 {base:.2f} -> wink.threshold {rec['wink']['threshold']}, wink.range {rec['wink']['range']}")
|
||||
|
||||
# Wink assist: does the other eye squint while one eye is closed? Compare it (in the module's mapped units) with that eye's
|
||||
# level when both eyes are closed; if the two separate, the assist can tell a wink from a real closure.
|
||||
mapped = lambda n: max(0.0, min(1.0, (n - 0.06) / 0.88)) # module: deadband 0.06
|
||||
mL = lambda x: mapped(nL(x[1]))
|
||||
mR = lambda x: mapped(nR(x[2]))
|
||||
# per eye, its level while both eyes are closed (p75: the both-closed step also contains ceiling-pinned glitch frames)
|
||||
closed_lvl = {"L": pct([mL(x) for x in g["closed"]], 0.75), "R": pct([mR(x) for x in g["closed"]], 0.75)}
|
||||
squint, mins = [], []
|
||||
for name, winking, other, oe in (("wink_left", mL, mR, "R"), ("wink_right", mR, mL, "L")):
|
||||
seg = [x for x in g.get(name, []) if winking(x) < 0.06]
|
||||
if len(seg) < 10:
|
||||
continue
|
||||
med = st.median(other(x) for x in seg)
|
||||
if 0.05 < med < 0.85:
|
||||
squint.append(name)
|
||||
if med > closed_lvl[oe] + 0.10:
|
||||
mins.append(closed_lvl[oe] + 0.08)
|
||||
else:
|
||||
warn.append(f"{name}: the other eye squints almost as much as when both eyes are closed; wink assist cannot separate them")
|
||||
if mins:
|
||||
rec["wink"].update({"assist": True, "assistMin": round(min(0.3, max(0.03, max(mins))), 3)})
|
||||
print(f" the other eye squints during {', '.join(squint)} -> wink.assist on, assistMin {rec['wink']['assistMin']}")
|
||||
SPOKEN.append("Wink assist is on, so the open eye stays open while you wink.")
|
||||
|
||||
bl = g.get("blinks", [])
|
||||
if len(bl) >= 30:
|
||||
# A blink shows up as any lid closing (the tracker often reports it lopsided: one lid ~0, the other ~1).
|
||||
|
||||
Reference in new issue
Block a user