mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-06 07:00:14 +02:00
Compare commits
7
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
33fb3bb611 | ||
|
|
6cae2e0afe | ||
|
|
57617d4a14 | ||
|
|
b7dfe4759e | ||
|
|
5b6cfcd746 | ||
|
|
e1ee5ef395 | ||
|
|
3e7248a04e |
No files matched your search
@@ -144,7 +144,8 @@ This is an early release, tested on one Steam Frame (SteamOS 0.3.0 build 2026092
|
|||||||
- Dragging something from one panel to another (a screen and a floating window) works, but the dragged item's icon doesn't show while the pointer is between panels.
|
- Dragging something from one panel to another (a screen and a floating window) works, but the dragged item's icon doesn't show while the pointer is between panels.
|
||||||
- Gaze mode is only as good as its calibration, and that depends on how the headset sits on your face. If the pointer lands off after you adjust the headset, run Quick check or Calibrate on the Gaze page of Frametop Input Settings.
|
- Gaze mode is only as good as its calibration, and that depends on how the headset sits on your face. If the pointer lands off after you adjust the headset, run Quick check or Calibrate on the Gaze page of Frametop Input Settings.
|
||||||
- On SteamVR's Settings page, the 3D mouse shows a laser beam and a larger hit dot, like a controller. SteamVR doesn't tell other programs where that page is (unlike Steam's pages, such as Library), so the mouse used to miss most of it: clicks went through to a desktop screen behind, and the dot disappeared. As a workaround, on that page only, the laser starts near your eye and SteamVR finds the page itself. See docs/design.md.
|
- On SteamVR's Settings page, the 3D mouse shows a laser beam and a larger hit dot, like a controller. SteamVR doesn't tell other programs where that page is (unlike Steam's pages, such as Library), so the mouse used to miss most of it: clicks went through to a desktop screen behind, and the dot disappeared. As a workaround, on that page only, the laser starts near your eye and SteamVR finds the page itself. See docs/design.md.
|
||||||
- Remote desktop over VNC (Frametop Remote Access in the app menu, or `./desktops.sh remote on`) needs Tailscale on the Frame. It shows the primary screen only. The app turns it on and off, shows the address, and shows, copies, or changes the VNC password. The password is made at random on the Frame and kept in `~/.config/frametop-remote` (only you can read it); VNC limits it to 8 characters, and the tailnet encrypts the connection. Turning it on in a desktop that started with it off takes a desktop restart.
|
- Remote desktop over VNC (Frametop Remote Access in the app menu, or `./desktops.sh remote on`) needs Tailscale on the Frame. It shows the primary screen only. The app turns it on and off, shows the address, and shows, copies, or changes the VNC password. The password is made at random on the Frame and kept in `~/.config/frametop-remote` (only you can read it); VNC limits it to 8 characters, and the tailnet encrypts the connection. Turning it on in a desktop that started with it off takes a desktop restart. It costs almost nothing until a viewer connects; the picture then takes a few seconds to appear.
|
||||||
|
- The desktop has no blur behind panels and menus, and no window animations, so it leaves the headset's GPU to SteamVR. Turn them back on in the Frametop desktop's System Settings (Desktop Effects, and Animation speed under General Behavior); Frametop won't turn them off again.
|
||||||
- Turning the displays off on a stand only turns their backlight off. SteamVR has no way for other programs to put the headset in standby, so tracking and rendering keep running, and the headset draws nearly its full power.
|
- Turning the displays off on a stand only turns their backlight off. SteamVR has no way for other programs to put the headset in standby, so tracking and rendering keep running, and the headset draws nearly its full power.
|
||||||
|
|
||||||
## Reporting problems
|
## Reporting problems
|
||||||
|
|||||||
@@ -177,6 +177,12 @@ The private runtime directory also moves the session's document portal to `$XDG_
|
|||||||
|
|
||||||
A podman container's monitor process (conmon) stays in the cgroup of whatever started the container, and `distrobox enter` starts it on demand. When a Frametop service happened to start the `dev` container, stopping that service stopped the container and everything in it, including the desktop's compositor. `scripts/container-up.sh` starts the container in a systemd scope of its own before anything enters it. It then waits for distrobox-init to log `container_setup_done`, as `distrobox enter` does only for containers it starts itself. A new container's first start takes a minute or more (it installs distrobox's dependencies and sets up passwordless sudo), and an install that entered right away met a sudo password prompt with no terminal to answer it ([#9](https://github.com/DeeJanuz/frametop/issues/9)).
|
A podman container's monitor process (conmon) stays in the cgroup of whatever started the container, and `distrobox enter` starts it on demand. When a Frametop service happened to start the `dev` container, stopping that service stopped the container and everything in it, including the desktop's compositor. `scripts/container-up.sh` starts the container in a systemd scope of its own before anything enters it. It then waits for distrobox-init to log `container_setup_done`, as `distrobox enter` does only for containers it starts itself. A new container's first start takes a minute or more (it installs distrobox's dependencies and sets up passwordless sudo), and an install that entered right away met a sudo password prompt with no terminal to answer it ([#9](https://github.com/DeeJanuz/frametop/issues/9)).
|
||||||
|
|
||||||
|
KWin renders with OpenGL through zink on Turnip, Vulkan on the same GPU vrcompositor needs to hit its frame time, and on the Frame that costs CPU too. The nested session started with KWin's defaults: blur and background contrast on (no `[Plugins]` group in its kwinrc) and animations at full length. Blur re-renders what's behind every translucent panel and menu each time it changes, and every animated frame is one more frame for KWin and ft-screens to draw and send. They're off by default in the Frametop desktop. The session script writes them before KWin starts, only where the desktop's own config has no value, once: System Settings deletes a setting put back to KDE's default rather than writing it, so without the marker in `frametoprc` a user who turned blur back on would lose it at the next start. The effect ids (`blur`, `contrast`) are the ones built into KWin 6.2.5 on SteamOS; KWin reads `<id>Enabled` from `[Plugins]`.
|
||||||
|
|
||||||
|
The nested session also runs the system's XDG autostart entries, being a KDE session. Discover's update notifier started `plasma-discover --mode update` in it (520 to 620 MB resident and about 9% of a core, plus `flatpak-system-helper` and AppStream downloads), and IBus started a daemon, the kimpanel panel and its GTK extension that nothing can use: KWin hands text input to the one input method it starts (`ft-textinput`), and the session drops `QT_IM_MODULE`, `GTK_IM_MODULE` and `XMODIFIERS`. The session hides both for this desktop only, with `Hidden=true` copies in its own autostart folder. The geoclue demo agent stays: it's what answers apps' location requests to Geoclue outside GNOME, and it costs nothing while idle. Orca's entry only starts in GNOME-family desktops.
|
||||||
|
|
||||||
|
Remote desktop is a chain (krdp, then FreeRDP inside Xvnc) because nothing on SteamOS serves KWin over VNC directly. Kept connected all the time, it cost about a core with nobody watching: krdpserver 55 to 78% (it encodes H.264 in software with openh264: VA-API finds no driver for the Frame's GPU in the container), FreeRDP 16 to 27%, Xvnc 6 to 11%, and the bridge's layout check every 5 seconds another 4%. krdp creates its screencast session per RDP connection and drops it when the connection closes (`SessionController::onNewConnection` in krdp 6.7), so an idle krdpserver costs nothing and can stay up; only the RDP connection has to go. The bridge connects FreeRDP when a VNC client appears and disconnects 45 seconds after the last one leaves. Xvnc has no hook for its clients, so the bridge counts established connections to its port with `ss`, woken early by Xvnc's log output; looking with `ss` once a second cost about 0.9% of a core in bash, against about 0.1% this way. `Xvnc -inetd` from a systemd socket would start a server per connection and lose sharing between viewers. The layout check (`ft-layout remote-view`, which scans `/proc` for plasmashell and runs `kscreen-doctor -j`) now runs only while FreeRDP runs, and then only after `kwinoutputconfig.json` or `frametop-layout.json` changes, with one check a minute in case a change touched neither.
|
||||||
|
|
||||||
Program names stay within 15 characters, because Linux truncates process names there and the scripts find programs with `pgrep -x` and `pkill -x`. That's why the prefix is `ft-`.
|
Program names stay within 15 characters, because Linux truncates process names there and the scripts find programs with `pgrep -x` and `pkill -x`. That's why the prefix is `ft-`.
|
||||||
|
|
||||||
## Displays off on a stand
|
## Displays off on a stand
|
||||||
|
|||||||
@@ -18,6 +18,16 @@ desktops.sh start | stop | restart | status | log [lines]
|
|||||||
|
|
||||||
When the VR launcher starts the desktop, it inherits the Steam client's environment. The session script drops the client's runtime from it (`LD_LIBRARY_PATH`, the `STEAM_*` settings, and the Steam overlay's Vulkan layer), so apps in the desktop use the system's libraries, including its video codecs, just as they would after a normal login.
|
When the VR launcher starts the desktop, it inherits the Steam client's environment. The session script drops the client's runtime from it (`LD_LIBRARY_PATH`, the `STEAM_*` settings, and the Steam overlay's Vulkan layer), so apps in the desktop use the system's libraries, including its video codecs, just as they would after a normal login.
|
||||||
|
|
||||||
|
KWin's blur and background contrast effects and its animations are off in this desktop, because KWin draws on the headset's GPU, which SteamVR needs. The session script turns them off once, the first time it starts (it leaves a setting you already have alone, and marks it done in `~/.config/frametop/frametoprc`), so turning them back on sticks. In the Frametop desktop, System Settings → Window Management → Desktop Effects has Blur and Background Contrast, and General Behavior has Animation speed. Or from a terminal, then restart the desktop:
|
||||||
|
|
||||||
|
```
|
||||||
|
kwriteconfig6 --file ~/.config/frametop/kwinrc --group Plugins --key blurEnabled true
|
||||||
|
kwriteconfig6 --file ~/.config/frametop/kwinrc --group Plugins --key contrastEnabled true
|
||||||
|
kwriteconfig6 --file ~/.config/frametop/kdeglobals --group KDE --key AnimationDurationFactor 1
|
||||||
|
```
|
||||||
|
|
||||||
|
Two of the system's autostart programs don't start in this desktop: Discover's update notifier (`org.kde.discover.notifier`), which starts Discover to check for updates, and IBus (`ibus`), which can't reach the desktop's apps because KWin's input method is `input/ft-textinput`. The session script puts copies with `Hidden=true` in `~/.config/frametop/autostart` once (marked in `frametoprc`), and skips a name you already have a file for. Delete a copy to start that program again.
|
||||||
|
|
||||||
Settings are in two files, and Frametop Display Settings edits both. The screens (resolution, width in metres, scale, curve, which one has the taskbar) and their layout are in `~/.config/frametop-layout.json`. The backend, remote desktop, and pointer settings are in `~/.config/frametop.conf`; `session/frametop.conf.example` lists every key.
|
Settings are in two files, and Frametop Display Settings edits both. The screens (resolution, width in metres, scale, curve, which one has the taskbar) and their layout are in `~/.config/frametop-layout.json`. The backend, remote desktop, and pointer settings are in `~/.config/frametop.conf`; `session/frametop.conf.example` lists every key.
|
||||||
|
|
||||||
Restarting the desktop closes its windows. Before the unit stops, `session/keep-apps.sh` moves every program started in the desktop into a systemd scope of its own, so background work such as servers, tmux, and builds keeps running. An app that shuts down its own helper processes when its window closes will still lose them; run that kind of work outside the desktop, for example as a systemd user service.
|
Restarting the desktop closes its windows. Before the unit stops, `session/keep-apps.sh` moves every program started in the desktop into a systemd scope of its own, so background work such as servers, tmux, and builds keeps running. An app that shuts down its own helper processes when its window closes will still lose them; run that kind of work outside the desktop, for example as a systemd user service.
|
||||||
@@ -259,6 +269,8 @@ It listens on port 5900 on the Frame's Tailscale address only, not the LAN, so i
|
|||||||
|
|
||||||
No VNC server can capture KWin on SteamOS directly: `krfb` needs `xdg-desktop-portal-kde`, which SteamOS doesn't ship, and `wayvnc` only works with wlroots compositors. So `session/remote-desktop.sh` captures the desktop with KDE's `krdpserver --plasma` on `127.0.0.1:3390`, and `session/vnc-bridge.sh` runs TigerVNC's `Xvnc` on display `:20` with a FreeRDP client inside it and serves that. Both run in the `dev` container, and the extra hop adds a little latency. krdp streams every screen; the VNC screen is the primary's size, and the FreeRDP window is shifted so the primary fills it (`ft-layout remote-view` gives the offset). krdp's own `--monitor` would stream just one screen, but it maps the pointer as if that screen sat at 0,0, so clicks would miss. When the layout changes, the VNC screen resizes and FreeRDP reconnects within a few seconds.
|
No VNC server can capture KWin on SteamOS directly: `krfb` needs `xdg-desktop-portal-kde`, which SteamOS doesn't ship, and `wayvnc` only works with wlroots compositors. So `session/remote-desktop.sh` captures the desktop with KDE's `krdpserver --plasma` on `127.0.0.1:3390`, and `session/vnc-bridge.sh` runs TigerVNC's `Xvnc` on display `:20` with a FreeRDP client inside it and serves that. Both run in the `dev` container, and the extra hop adds a little latency. krdp streams every screen; the VNC screen is the primary's size, and the FreeRDP window is shifted so the primary fills it (`ft-layout remote-view` gives the offset). krdp's own `--monitor` would stream just one screen, but it maps the pointer as if that screen sat at 0,0, so clicks would miss. When the layout changes, the VNC screen resizes and FreeRDP reconnects within a few seconds.
|
||||||
|
|
||||||
|
FreeRDP runs only while a VNC viewer is connected, because while it's connected krdp captures and encodes every redraw. With no viewer, krdp has no RDP connection and so captures nothing, and Xvnc shows a black screen. When a viewer connects, the bridge starts FreeRDP, and the desktop appears about 3 seconds later; FreeRDP stops 45 seconds after the last viewer leaves (`VNC_IDLE_SEC` in the bridge's environment). The bridge looks for viewers with `ss` whenever Xvnc logs something, as it does for every connection, and every 5 seconds otherwise. While a viewer is connected, the bridge asks ft-screens to draw every screen at full rate (`watch 15` on `@ft_screens`, renewed every 5 seconds), so screens you aren't looking at in the headset, or a headset on a stand, don't stream at a low rate. It reads the primary screen's place again (`ft-layout remote-view`) only while FreeRDP runs, after `~/.config/frametop/kwinoutputconfig.json` or `~/.config/frametop-layout.json` changes, and once a minute.
|
||||||
|
|
||||||
With remote access on, the nested KWin runs with `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1` and `KWIN_SCREENSHOT_NO_PERMISSION_CHECKS=1`, so any app in the Frametop desktop could capture its screens or inject input. The second one lets scripts take screenshots through KWin's `org.kde.KWin.ScreenShot2` D-Bus interface. This applies only to that desktop, not the stock one. Port 3389 is SteamOS's own `xrdp`, which starts a separate X11 session rather than showing the VR desktop.
|
With remote access on, the nested KWin runs with `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1` and `KWIN_SCREENSHOT_NO_PERMISSION_CHECKS=1`, so any app in the Frametop desktop could capture its screens or inject input. The second one lets scripts take screenshots through KWin's `org.kde.KWin.ScreenShot2` D-Bus interface. This applies only to that desktop, not the stock one. Port 3389 is SteamOS's own `xrdp`, which starts a separate X11 session rather than showing the VR desktop.
|
||||||
|
|
||||||
## Limits
|
## Limits
|
||||||
|
|||||||
+2
-2
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
The Steam Frame's eye tracking as pointer input: a gaze mode for the 3D mouse (the pointer goes where you look, and the mouse does the last bit), and the tools to calibrate and measure it.
|
The Steam Frame's eye tracking as pointer input: a gaze mode for the 3D mouse (the pointer goes where you look, and the mouse does the last bit), and the tools to calibrate and measure it.
|
||||||
|
|
||||||
- `ft-gaze` (C++, OpenVR, runs in the dev container) reads the eye tracker and prints one JSON line per sample (90 Hz). For each source, it gives the gaze direction relative to the head and the Frametop screen pixel it lands on. The gaze service asks it for only the sources it uses (`--sources`, and `sources LIST` on its stdin): our tracker and mmap set 1 with Own tracker, about 700 bytes a line instead of 1.3 KB, and every source while a check or the calibration runs. So SteamVR's gaze action, the only source that calls into vrserver (twice a sample), is read only then. The probe gets them all.
|
- `ft-gaze` (C++, OpenVR, runs in the dev container) reads the eye tracker and prints one JSON line per sample (90 Hz). For each source, it gives the gaze direction relative to the head and the Frametop screen pixel it lands on.
|
||||||
- `gazecal.py` has what the probe and the gaze service share: the correction models, filters, and the reader for SteamVR's eye tracking log.
|
- `gazecal.py` has what the probe and the gaze service share: the correction models, filters, and the reader for SteamVR's eye tracking log.
|
||||||
- `tracker/` is our own eye tracker, an alternative to SteamVR's: `ft-eyes` finds the pupils and glints in the eye-camera frames that `ft-eyegrab` (a small root service) copies out of SteamVR's tracker. See "Our own eye tracker" below.
|
- `tracker/` is our own eye tracker, an alternative to SteamVR's: `ft-eyes` finds the pupils and glints in the eye-camera frames that `ft-eyegrab` (a small root service) copies out of SteamVR's tracker. See "Our own eye tracker" below.
|
||||||
- `probe/ft-gazeprobe` (GTK 4, host Python) is a fullscreen playground, for developing the gaze tracking: day to day, the calibration and the checks run in the headset panel (Quick check, Calibrate, and Check headset fit on the Gaze page). It runs ft-gaze, draws where you're looking, measures accuracy, and tries out hold-to-adjust clicking with a calibration that learns from your adjustments.
|
- `probe/ft-gazeprobe` (GTK 4, host Python) is a fullscreen playground, for developing the gaze tracking: day to day, the calibration and the checks run in the headset panel (Quick check, Calibrate, and Check headset fit on the Gaze page). It runs ft-gaze, draws where you're looking, measures accuracy, and tries out hold-to-adjust clicking with a calibration that learns from your adjustments.
|
||||||
@@ -63,7 +63,7 @@ Ground rules, for anyone changing it:
|
|||||||
- **Clean room.** Nothing of Valve's goes in: we don't decompile, disassemble, or patch the `eyetracking` binary or its network weights, and we don't copy their code or weights. Its public output (eye-server.mmap, read-only) is fair game as a baseline and as labels, and so are published papers and openly licensed pupil detectors (check each one's license: PuRe, PuReST, ElSe, and ExCuSe are non-commercial only).
|
- **Clean room.** Nothing of Valve's goes in: we don't decompile, disassemble, or patch the `eyetracking` binary or its network weights, and we don't copy their code or weights. Its public output (eye-server.mmap, read-only) is fair game as a baseline and as labels, and so are published papers and openly licensed pupil detectors (check each one's license: PuRe, PuReST, ElSe, and ExCuSe are non-commercial only).
|
||||||
- **Root only reads.** ft-eyegrab never writes to, stops, or signals the `eyetracking` process, vrserver, or vrcompositor, never opens `/dev/adsp`, `/dev/cdsp`, or `/dev/spidev0.1`, and never writes to `/dev/shm/eye-server.mmap` (it also carries calibration clicks into SteamVR's tracker), `/opt`, or `/persist`.
|
- **Root only reads.** ft-eyegrab never writes to, stops, or signals the `eyetracking` process, vrserver, or vrcompositor, never opens `/dev/adsp`, `/dev/cdsp`, or `/dev/spidev0.1`, and never writes to `/dev/shm/eye-server.mmap` (it also carries calibration clicks into SteamVR's tracker), `/opt`, or `/persist`.
|
||||||
- **Eye images are biometric data.** Recordings live outside the repo, in `~/.local/share/frametop/eyes/captures` (0700), and `.gitignore` catches stray frame dumps. They go nowhere but the machine that runs your offline jobs.
|
- **Eye images are biometric data.** Recordings live outside the repo, in `~/.local/share/frametop/eyes/captures` (0700), and `.gitignore` catches stray frame dumps. They go nowhere but the machine that runs your offline jobs.
|
||||||
- **Mind the headset's budget.** Finding a pupil takes about 0.4 ms a frame while ft-eyes follows it, and 1.4-2.1 ms when it searches the whole frame. ft-eyes keeps OpenCV and numpy to one thread: their pools of one per core spun idle workers at about a quarter of a core, for frames this small. It also runs at nice 10 with SCHED_BATCH, and ft-gaze at nice 5 (not batch, since each sample goes on to the pointer): both run in the dev container's podman scope, out of reach of the gaze service's unit, on the cores vrcompositor and vrserver use at nice 0. Replays, scoring, and training go to a PC.
|
- **Mind the headset's budget.** Finding a pupil takes about 0.4 ms a frame while ft-eyes follows it, and 1.4-2.1 ms when it searches the whole frame. Replays, scoring, and training go to a PC.
|
||||||
|
|
||||||
## Headset fit
|
## Headset fit
|
||||||
|
|
||||||
|
|||||||
+16
-85
@@ -4,14 +4,7 @@
|
|||||||
// Every eye tracker sample (90 Hz) becomes one JSON line on stdout with each gaze source
|
// Every eye tracker sample (90 Hz) becomes one JSON line on stdout with each gaze source
|
||||||
// hit-tested against the Frametop screens:
|
// hit-tested against the Frametop screens:
|
||||||
//
|
//
|
||||||
// Options: -v (log action errors), --watch-stdin (quit when stdin closes; until then, a line
|
// Options: -v (log action errors), --watch-stdin (quit when stdin closes).
|
||||||
// "sources LIST" on stdin switches the sources as --sources does), --sources LIST (comma-
|
|
||||||
// separated: action, mmap1, mmap2, left, right, own, and eye for EYE; or all, the default).
|
|
||||||
// Sources left out are read not at all and print as {"ok":0} ("eye" as null), so every line
|
|
||||||
// keeps the same keys. The gaze service asks for the ones it uses (own and mmap1 with our
|
|
||||||
// tracker), and all of them while a check or the calibration runs. Only the action costs
|
|
||||||
// SteamVR anything (two calls into vrserver per sample), and a line with every source is
|
|
||||||
// about 1.5 KB, 130 KB a second through podman's relay.
|
|
||||||
//
|
//
|
||||||
// {"t":<sample time, CLOCK_MONOTONIC_RAW s>,"age":<ms old when read>,"n":<sample counter>,
|
// {"t":<sample time, CLOCK_MONOTONIC_RAW s>,"age":<ms old when read>,"n":<sample counter>,
|
||||||
// "head":{"yaw":..,"pitch":..,"hit":HIT}, head forward ray (for head nudging)
|
// "head":{"yaw":..,"pitch":..,"hit":HIT}, head forward ray (for head nudging)
|
||||||
@@ -65,7 +58,6 @@
|
|||||||
|
|
||||||
#include <algorithm>
|
#include <algorithm>
|
||||||
#include <atomic>
|
#include <atomic>
|
||||||
#include <cerrno>
|
|
||||||
#include <chrono>
|
#include <chrono>
|
||||||
#include <climits>
|
#include <climits>
|
||||||
#include <cmath>
|
#include <cmath>
|
||||||
@@ -80,7 +72,6 @@
|
|||||||
|
|
||||||
#include <fcntl.h>
|
#include <fcntl.h>
|
||||||
#include <sys/mman.h>
|
#include <sys/mman.h>
|
||||||
#include <sys/resource.h>
|
|
||||||
#include <sys/socket.h>
|
#include <sys/socket.h>
|
||||||
#include <sys/stat.h>
|
#include <sys/stat.h>
|
||||||
#include <sys/un.h>
|
#include <sys/un.h>
|
||||||
@@ -398,30 +389,6 @@ std::string SrcJson(const std::vector<Screen> &screens, const vr::HmdMatrix34_t
|
|||||||
return buf + extra + "\"hit\":" + HitJson(screens, head, yaw, pitch) + "}";
|
return buf + extra + "\"hit\":" + HitJson(screens, head, yaw, pitch) + "}";
|
||||||
}
|
}
|
||||||
|
|
||||||
// Sources (--sources, "sources LIST" on stdin): a bit each.
|
|
||||||
enum : unsigned { kAction = 1, kMmap1 = 2, kMmap2 = 4, kLeft = 8, kRight = 16, kOwn = 32, kEye = 64, kAll = 127 };
|
|
||||||
|
|
||||||
bool ParseSources(const std::string &list, unsigned &mask) {
|
|
||||||
static const struct {
|
|
||||||
const char *name;
|
|
||||||
unsigned bit;
|
|
||||||
} names[] = {{"action", kAction}, {"mmap1", kMmap1}, {"mmap2", kMmap2}, {"left", kLeft},
|
|
||||||
{"right", kRight}, {"own", kOwn}, {"eye", kEye}, {"all", kAll}};
|
|
||||||
unsigned m = 0;
|
|
||||||
size_t at = 0;
|
|
||||||
while (at <= list.size()) {
|
|
||||||
const size_t comma = std::min(list.find(',', at), list.size());
|
|
||||||
const std::string name = list.substr(at, comma - at);
|
|
||||||
bool known = false;
|
|
||||||
for (const auto &n : names)
|
|
||||||
if (name == n.name) m |= n.bit, known = true;
|
|
||||||
if (!known) return false;
|
|
||||||
at = comma + 1;
|
|
||||||
}
|
|
||||||
mask = m;
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Head poses of the last half second, so a sample can use the pose at its own time.
|
// Head poses of the last half second, so a sample can use the pose at its own time.
|
||||||
class PoseHistory {
|
class PoseHistory {
|
||||||
public:
|
public:
|
||||||
@@ -459,42 +426,17 @@ std::string ExeDir() {
|
|||||||
|
|
||||||
int main(int argc, char **argv) {
|
int main(int argc, char **argv) {
|
||||||
bool verbose = false, watchStdin = false;
|
bool verbose = false, watchStdin = false;
|
||||||
std::atomic<unsigned> sources{kAll};
|
|
||||||
for (int i = 1; i < argc; ++i) {
|
for (int i = 1; i < argc; ++i) {
|
||||||
if (std::strcmp(argv[i], "-v") == 0) verbose = true;
|
if (std::strcmp(argv[i], "-v") == 0) verbose = true;
|
||||||
if (std::strcmp(argv[i], "--watch-stdin") == 0) watchStdin = true;
|
if (std::strcmp(argv[i], "--watch-stdin") == 0) watchStdin = true;
|
||||||
if (std::strcmp(argv[i], "--sources") == 0 && i + 1 < argc) {
|
|
||||||
unsigned m;
|
|
||||||
if (ParseSources(argv[++i], m))
|
|
||||||
sources = m;
|
|
||||||
else
|
|
||||||
std::fprintf(stderr, "ft-gaze: --sources %s: unknown source (all used)\n", argv[i]);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
// Nice 5, before any thread starts (they inherit it): we run in the dev container's podman
|
|
||||||
// scope, beside vrcompositor and vrserver at nice 0, and the gaze service's unit doesn't
|
|
||||||
// reach us. Not SCHED_BATCH, as ft-eyes is: that would let each wakeup wait out another
|
|
||||||
// task's turn, and each sample goes on to the pointer.
|
|
||||||
errno = 0;
|
|
||||||
const int nice0 = getpriority(PRIO_PROCESS, 0);
|
|
||||||
if (errno == 0 && nice0 < 5 && setpriority(PRIO_PROCESS, 0, 5) != 0)
|
|
||||||
std::fprintf(stderr, "ft-gaze: nice: %s\n", std::strerror(errno));
|
|
||||||
// --watch-stdin: quit when stdin closes. The probe runs us through distrobox, which
|
// --watch-stdin: quit when stdin closes. The probe runs us through distrobox, which
|
||||||
// passes neither its signals nor a closed stdout on to us, but does pass stdin's end.
|
// passes neither its signals nor a closed stdout on to us, but does pass stdin's end.
|
||||||
// Lines on stdin until then: "sources LIST".
|
|
||||||
std::atomic<bool> stdinClosed{false};
|
std::atomic<bool> stdinClosed{false};
|
||||||
if (watchStdin)
|
if (watchStdin)
|
||||||
std::thread([&stdinClosed, &sources] {
|
std::thread([&stdinClosed] {
|
||||||
char c[256];
|
char c[256];
|
||||||
std::string line;
|
while (read(0, c, sizeof c) > 0) {
|
||||||
ssize_t n;
|
|
||||||
while ((n = read(0, c, sizeof c)) > 0) {
|
|
||||||
line.append(c, size_t(n));
|
|
||||||
for (size_t nl; (nl = line.find('\n')) != std::string::npos; line.erase(0, nl + 1)) {
|
|
||||||
unsigned m;
|
|
||||||
if (line.compare(0, 8, "sources ") == 0 && ParseSources(line.substr(8, nl - 8), m)) sources = m;
|
|
||||||
}
|
|
||||||
if (line.size() > 4096) line.clear(); // no newline in sight: not ours
|
|
||||||
}
|
}
|
||||||
stdinClosed = true;
|
stdinClosed = true;
|
||||||
}).detach();
|
}).detach();
|
||||||
@@ -567,7 +509,6 @@ int main(int argc, char **argv) {
|
|||||||
|
|
||||||
if (fresh && hp.bPoseIsValid) {
|
if (fresh && hp.bPoseIsValid) {
|
||||||
lastEmit = now;
|
lastEmit = now;
|
||||||
const unsigned want = sources;
|
|
||||||
const auto list = screens.Get();
|
const auto list = screens.Get();
|
||||||
const vr::HmdMatrix34_t &headNow = hp.mDeviceToAbsoluteTracking;
|
const vr::HmdMatrix34_t &headNow = hp.mDeviceToAbsoluteTracking;
|
||||||
vr::HmdMatrix34_t headThen = headNow;
|
vr::HmdMatrix34_t headThen = headNow;
|
||||||
@@ -576,7 +517,7 @@ int main(int argc, char **argv) {
|
|||||||
// SteamVR's action: a room-space origin and fixation point, turned into the head
|
// SteamVR's action: a room-space origin and fixation point, turned into the head
|
||||||
// frame so every source reports the same kind of angles.
|
// frame so every source reports the same kind of angles.
|
||||||
std::string action = "{\"ok\":0}";
|
std::string action = "{\"ok\":0}";
|
||||||
if (!inGame && (want & kAction)) {
|
if (!inGame) {
|
||||||
vr::VRActiveActionSet_t active{};
|
vr::VRActiveActionSet_t active{};
|
||||||
active.ulActionSet = set;
|
active.ulActionSet = set;
|
||||||
active.nPriority = vr::k_nActionSetOverlayGlobalPriorityMin;
|
active.nPriority = vr::k_nActionSetOverlayGlobalPriorityMin;
|
||||||
@@ -619,33 +560,26 @@ int main(int argc, char **argv) {
|
|||||||
return std::string(b);
|
return std::string(b);
|
||||||
};
|
};
|
||||||
char extra[256];
|
char extra[256];
|
||||||
if (want & kMmap1) {
|
std::snprintf(extra, sizeof extra, "\"dist\":%.3f,\"open\":[%.3f,%.3f],\"lr\":%.3f,", Length(s.fix1),
|
||||||
std::snprintf(extra, sizeof extra, "\"dist\":%.3f,\"open\":[%.3f,%.3f],\"lr\":%.3f,", Length(s.fix1),
|
s.open[0], s.open[1], lr(s.left1, s.right1));
|
||||||
s.open[0], s.open[1], lr(s.left1, s.right1));
|
m1 = SrcJson(list, headThen, s.left1 + s.right1, extra + eyes(s.left1, s.right1) + unc(s.var1));
|
||||||
m1 = SrcJson(list, headThen, s.left1 + s.right1, extra + eyes(s.left1, s.right1) + unc(s.var1));
|
std::snprintf(extra, sizeof extra, "\"lr\":%.3f,", lr(s.left2, s.right2));
|
||||||
}
|
m2 = SrcJson(list, headThen, s.left2 + s.right2, extra + eyes(s.left2, s.right2) + unc(s.var2));
|
||||||
if (want & kMmap2) {
|
left = SrcJson(list, headThen, s.left2);
|
||||||
std::snprintf(extra, sizeof extra, "\"lr\":%.3f,", lr(s.left2, s.right2));
|
right = SrcJson(list, headThen, s.right2);
|
||||||
m2 = SrcJson(list, headThen, s.left2 + s.right2, extra + eyes(s.left2, s.right2) + unc(s.var2));
|
|
||||||
}
|
|
||||||
if (want & kLeft) left = SrcJson(list, headThen, s.left2);
|
|
||||||
if (want & kRight) right = SrcJson(list, headThen, s.right2);
|
|
||||||
// "new" compares with the last sample, so the last measurement is kept either way.
|
|
||||||
const float *m = s.meas;
|
const float *m = s.meas;
|
||||||
const bool newL = m[0] != lastMeas[0] || m[1] != lastMeas[1];
|
const bool newL = m[0] != lastMeas[0] || m[1] != lastMeas[1];
|
||||||
const bool newR = m[2] != lastMeas[2] || m[3] != lastMeas[3];
|
const bool newR = m[2] != lastMeas[2] || m[3] != lastMeas[3];
|
||||||
std::memcpy(lastMeas, m, sizeof lastMeas);
|
std::memcpy(lastMeas, m, sizeof lastMeas);
|
||||||
if (want & kEye) {
|
std::snprintf(extra, sizeof extra, "{\"q\":[%.3g,%.3g],\"m\":[[%.4f,%.4f],[%.4f,%.4f]],\"new\":[%d,%d]}",
|
||||||
std::snprintf(extra, sizeof extra, "{\"q\":[%.3g,%.3g],\"m\":[[%.4f,%.4f],[%.4f,%.4f]],\"new\":[%d,%d]}",
|
(m[4] + m[5]) / 2, (m[6] + m[7]) / 2, m[0], m[1], m[2], m[3], int(newL), int(newR));
|
||||||
(m[4] + m[5]) / 2, (m[6] + m[7]) / 2, m[0], m[1], m[2], m[3], int(newL), int(newR));
|
eye = extra;
|
||||||
eye = extra;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Our tracker: its own sample time picks the head pose, like the mmap's.
|
// Our tracker: its own sample time picks the head pose, like the mmap's.
|
||||||
std::string own = "{\"ok\":0}";
|
std::string own = "{\"ok\":0}";
|
||||||
OwnSample o;
|
OwnSample o;
|
||||||
if ((want & kOwn) && ownFile.Read(o) && now - o.t < 0.1) {
|
if (ownFile.Read(o) && now - o.t < 0.1) {
|
||||||
vr::HmdMatrix34_t headOwn = headNow;
|
vr::HmdMatrix34_t headOwn = headNow;
|
||||||
history.At(o.t, headOwn);
|
history.At(o.t, headOwn);
|
||||||
auto pair = [](bool ok, float a, float b) {
|
auto pair = [](bool ok, float a, float b) {
|
||||||
@@ -690,10 +624,7 @@ int main(int argc, char **argv) {
|
|||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
if (stdinClosed) break;
|
if (stdinClosed) break;
|
||||||
// 250 passes a second: a new sample is printed within 4 ms (2 on average) of appearing,
|
std::this_thread::sleep_for(std::chrono::milliseconds(2));
|
||||||
// and the pose history has a pose within 2 ms of any sample's time (a 0.2 degree head
|
|
||||||
// turn at 100 degrees a second). Every 2 ms read the pose and the events twice as often.
|
|
||||||
std::this_thread::sleep_for(std::chrono::milliseconds(4));
|
|
||||||
}
|
}
|
||||||
screens.Stop();
|
screens.Stop();
|
||||||
vr::VR_Shutdown();
|
vr::VR_Shutdown();
|
||||||
|
|||||||
+3
-38
@@ -76,12 +76,6 @@ off. A check asked for while idle waits for the tracker to start (at most WAKE_S
|
|||||||
tracker's next click after it starts again re-seats it, as after the headset was off, so turning
|
tracker's next click after it starts again re-seats it, as after the headset was off, so turning
|
||||||
gaze mode on after a while opens the quick check.
|
gaze mode on after a while opens the quick check.
|
||||||
|
|
||||||
ft-gaze prints only the sources this uses ("sources LIST" on its stdin, see wanted_sources):
|
|
||||||
own and mmap1 with our tracker, left, right and mmap1 with SteamVR's eyes, and every source
|
|
||||||
while a check or the calibration runs or is asked for, since those record them all. So
|
|
||||||
SteamVR's gaze action, the one source that calls into vrserver, is read only then (or with
|
|
||||||
--source action).
|
|
||||||
|
|
||||||
Checks and calibration (gaze/gazecheck.py): a one-dot quick check when the headset goes on or
|
Checks and calibration (gaze/gazecheck.py): a one-dot quick check when the headset goes on or
|
||||||
our tracker asks for a click, and the full calibration when gaze mode comes on without one,
|
our tracker asks for a click, and the full calibration when gaze mode comes on without one,
|
||||||
both in a panel fixed to the headset (gaze/panel/ft-gazepanel, which this service runs too).
|
both in a panel fixed to the headset (gaze/panel/ft-gazepanel, which this service runs too).
|
||||||
@@ -266,7 +260,6 @@ class Service:
|
|||||||
self.sel.register(self.eyes_sock, selectors.EVENT_READ, "own")
|
self.sel.register(self.eyes_sock, selectors.EVENT_READ, "own")
|
||||||
self.checks = Checks(self, self.sel)
|
self.checks = Checks(self, self.sel)
|
||||||
self.proc = None
|
self.proc = None
|
||||||
self.proc_sources = None # what ft-gaze was last told to print
|
|
||||||
self.buf = b""
|
self.buf = b""
|
||||||
self.restart_at = 0.0
|
self.restart_at = 0.0
|
||||||
self.running = True
|
self.running = True
|
||||||
@@ -466,31 +459,6 @@ class Service:
|
|||||||
|
|
||||||
# --- ft-gaze ---
|
# --- ft-gaze ---
|
||||||
|
|
||||||
def wanted_sources(self):
|
|
||||||
"""The sources ft-gaze should print (its --sources): what on_sample and the checks read.
|
|
||||||
mmap1 always (blinks, lost eyes, and the headset going on, from its openness and
|
|
||||||
variances); everything while a check runs or waits, since they record every source."""
|
|
||||||
c = self.checks
|
|
||||||
if c.active or c.pending:
|
|
||||||
return "all"
|
|
||||||
kind = self.kind
|
|
||||||
if kind == "own":
|
|
||||||
return "own,mmap1"
|
|
||||||
if kind == "eyes":
|
|
||||||
return "left,right,mmap1"
|
|
||||||
return ",".join(dict.fromkeys((self.source, "mmap1", "mmap2"))) # mmap2: each eye, for the fallback
|
|
||||||
|
|
||||||
def sync_sources(self):
|
|
||||||
want = self.wanted_sources()
|
|
||||||
if not self.proc or want == self.proc_sources:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
self.proc.stdin.write(f"sources {want}\n".encode())
|
|
||||||
self.proc.stdin.flush()
|
|
||||||
except (OSError, ValueError):
|
|
||||||
return # it's stopping: read_stdout notices
|
|
||||||
self.proc_sources = want
|
|
||||||
|
|
||||||
def start_helper(self):
|
def start_helper(self):
|
||||||
if not HELPER.exists():
|
if not HELPER.exists():
|
||||||
log(f"ft-gaze isn't built: run {REPO}/gaze/build.sh")
|
log(f"ft-gaze isn't built: run {REPO}/gaze/build.sh")
|
||||||
@@ -501,11 +469,9 @@ class Service:
|
|||||||
subprocess.run([str(REPO / "scripts" / "container-up.sh")], env=env, check=False)
|
subprocess.run([str(REPO / "scripts" / "container-up.sh")], env=env, check=False)
|
||||||
distrobox = Path.home() / ".local" / "bin" / "distrobox"
|
distrobox = Path.home() / ".local" / "bin" / "distrobox"
|
||||||
# ft-gaze quits when its stdin closes: the one thing distrobox passes on.
|
# ft-gaze quits when its stdin closes: the one thing distrobox passes on.
|
||||||
sources = self.wanted_sources()
|
self.proc = subprocess.Popen([str(distrobox), "enter", "dev", "--", str(HELPER), "--watch-stdin"], env=env,
|
||||||
self.proc = subprocess.Popen([str(distrobox), "enter", "dev", "--", str(HELPER), "--watch-stdin",
|
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
|
||||||
"--sources", sources], env=env, stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
start_new_session=True)
|
||||||
stderr=subprocess.PIPE, start_new_session=True)
|
|
||||||
self.proc_sources = sources
|
|
||||||
os.set_blocking(self.proc.stdout.fileno(), False)
|
os.set_blocking(self.proc.stdout.fileno(), False)
|
||||||
os.set_blocking(self.proc.stderr.fileno(), False)
|
os.set_blocking(self.proc.stderr.fileno(), False)
|
||||||
self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
|
self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
|
||||||
@@ -910,7 +876,6 @@ class Service:
|
|||||||
elif key.data == "stderr" and self.proc:
|
elif key.data == "stderr" and self.proc:
|
||||||
self.read_stderr()
|
self.read_stderr()
|
||||||
self.checks.tick()
|
self.checks.tick()
|
||||||
self.sync_sources()
|
|
||||||
if now >= next_periodic:
|
if now >= next_periodic:
|
||||||
self.periodic()
|
self.periodic()
|
||||||
next_periodic = now + 1.0
|
next_periodic = now + 1.0
|
||||||
|
|||||||
@@ -47,7 +47,6 @@
|
|||||||
#include <drm_fourcc.h>
|
#include <drm_fourcc.h>
|
||||||
#include <fcntl.h>
|
#include <fcntl.h>
|
||||||
#include <gbm.h>
|
#include <gbm.h>
|
||||||
#include <poll.h>
|
|
||||||
#include <sys/socket.h>
|
#include <sys/socket.h>
|
||||||
#include <sys/un.h>
|
#include <sys/un.h>
|
||||||
#include <unistd.h>
|
#include <unistd.h>
|
||||||
@@ -525,11 +524,7 @@ int main(int argc, char **argv) {
|
|||||||
if (!shown) ov->ShowOverlay(h), shown = true;
|
if (!shown) ov->ShowOverlay(h), shown = true;
|
||||||
dirty = false;
|
dirty = false;
|
||||||
}
|
}
|
||||||
// Until a command comes, or 10 ms while shown (SteamVR's events). Hidden, it waits up to a
|
std::this_thread::sleep_for(std::chrono::milliseconds(visible ? 10 : 50));
|
||||||
// second: it woke 20 to 30 times a second for nothing, the main cost left with gaze idle.
|
|
||||||
// A closed stdin (--watch-stdin) wakes it too, so quitting doesn't wait.
|
|
||||||
pollfd fds[2] = {{sock, POLLIN, 0}, {0, POLLIN, 0}};
|
|
||||||
poll(fds, watchStdin ? 2 : 1, visible ? 10 : 1000);
|
|
||||||
}
|
}
|
||||||
ov->DestroyOverlay(h);
|
ov->DestroyOverlay(h);
|
||||||
buffers.Drop();
|
buffers.Drop();
|
||||||
|
|||||||
+6
-26
@@ -37,26 +37,14 @@ gazed.read_settings = lambda: ("steam", "steam", "auto", 55.0)
|
|||||||
logs = []
|
logs = []
|
||||||
gazed.log = gazecheck.log = lambda msg: logs.append(msg)
|
gazed.log = gazecheck.log = lambda msg: logs.append(msg)
|
||||||
|
|
||||||
# The fake ft-gaze: 90 samples a second, both eyes seen, until its stdin closes. It logs the
|
# The fake ft-gaze: 90 samples a second, both eyes seen, until its stdin closes.
|
||||||
# sources it was asked for: "argv LIST" (--sources), then "line LIST" for each "sources LIST".
|
|
||||||
tmp = tempfile.mkdtemp(prefix="ft-gaze-idle-test-")
|
tmp = tempfile.mkdtemp(prefix="ft-gaze-idle-test-")
|
||||||
FAKE = os.path.join(tmp, "ft-gaze")
|
FAKE = os.path.join(tmp, "ft-gaze")
|
||||||
SOURCES_LOG = os.path.join(tmp, "sources.log")
|
|
||||||
with open(FAKE, "w") as f:
|
with open(FAKE, "w") as f:
|
||||||
f.write('''import json, os, select, sys, time
|
f.write('''import json, select, sys, time
|
||||||
log = open(sys.argv[1], "a", buffering=1)
|
|
||||||
log.write("argv " + (sys.argv[sys.argv.index("--sources") + 1] if "--sources" in sys.argv else "-") + "\\n")
|
|
||||||
buf = b""
|
|
||||||
while True:
|
while True:
|
||||||
if select.select([sys.stdin], [], [], 1 / 90)[0]:
|
if select.select([sys.stdin], [], [], 1 / 90)[0] and not sys.stdin.read(1):
|
||||||
data = os.read(0, 4096)
|
break
|
||||||
if not data:
|
|
||||||
break
|
|
||||||
buf += data
|
|
||||||
while b"\\n" in buf:
|
|
||||||
line, buf = buf.split(b"\\n", 1)
|
|
||||||
if line.startswith(b"sources "):
|
|
||||||
log.write("line " + line[8:].decode() + "\\n")
|
|
||||||
eye = {"hy": 1.0, "hp": 2.0}
|
eye = {"hy": 1.0, "hp": 2.0}
|
||||||
print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
|
print(json.dumps({"t": time.monotonic(), "src": {"mmap1": {"hy": 1.0, "hp": 2.0, "unc": [0.001, 0.001],
|
||||||
"open": [0.8, 0.8]}, "left": eye, "right": eye}}), flush=True)
|
"open": [0.8, 0.8]}, "left": eye, "right": eye}}), flush=True)
|
||||||
@@ -67,9 +55,8 @@ started = []
|
|||||||
def start_helper(self):
|
def start_helper(self):
|
||||||
"""ft-gaze, straight from here instead of the dev container."""
|
"""ft-gaze, straight from here instead of the dev container."""
|
||||||
import selectors
|
import selectors
|
||||||
self.proc = subprocess.Popen([sys.executable, FAKE, SOURCES_LOG, "--sources", self.wanted_sources()],
|
self.proc = subprocess.Popen([sys.executable, FAKE], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
||||||
stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
stderr=subprocess.PIPE)
|
||||||
self.proc_sources = self.wanted_sources()
|
|
||||||
os.set_blocking(self.proc.stdout.fileno(), False)
|
os.set_blocking(self.proc.stdout.fileno(), False)
|
||||||
os.set_blocking(self.proc.stderr.fileno(), False)
|
os.set_blocking(self.proc.stderr.fileno(), False)
|
||||||
self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
|
self.sel.register(self.proc.stdout, selectors.EVENT_READ, "stdout")
|
||||||
@@ -155,24 +142,17 @@ check("lease over (2 s + IDLE_AFTER): idle", wait(lambda: not svc.awake, 4.5), T
|
|||||||
|
|
||||||
time.sleep(0.5)
|
time.sleep(0.5)
|
||||||
logs.clear()
|
logs.clear()
|
||||||
open(SOURCES_LOG, "w").close()
|
|
||||||
count = len(started)
|
count = len(started)
|
||||||
check("quick check while idle: queued, waking", ask("quickcal"), "ok waking the eye tracker first")
|
check("quick check while idle: queued, waking", ask("quickcal"), "ok waking the eye tracker first")
|
||||||
check("it woke", wait(lambda: svc.awake and len(started) > count, 1.5), True)
|
check("it woke", wait(lambda: svc.awake and len(started) > count, 1.5), True)
|
||||||
check("once the tracker sends, it ran (and says why it couldn't open)",
|
check("once the tracker sends, it ran (and says why it couldn't open)",
|
||||||
wait(lambda: any("quickcal, asked for while idle: the panel isn't running" in m for m in logs), 3), True)
|
wait(lambda: any("quickcal, asked for while idle: the panel isn't running" in m for m in logs), 3), True)
|
||||||
check("nothing left pending", svc.checks.pending, None)
|
check("nothing left pending", svc.checks.pending, None)
|
||||||
sources = open(SOURCES_LOG).read().split("\n")
|
|
||||||
check("ft-gaze started with every source for the check", sources[0], "argv all")
|
|
||||||
in_use = svc.wanted_sources()
|
|
||||||
check("then only those in use (SteamVR's tracker: not the action, not own)",
|
|
||||||
(f"line {in_use}" in sources, in_use != "all", "action" in in_use, "own" in in_use), (True, True, False, False))
|
|
||||||
check("idle again after", wait(lambda: not svc.awake, 3), True)
|
check("idle again after", wait(lambda: not svc.awake, 3), True)
|
||||||
|
|
||||||
print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
|
print("FAILED: " + ", ".join(failures) if failures else "all passed", flush=True)
|
||||||
svc.running = False
|
svc.running = False
|
||||||
time.sleep(0.7)
|
time.sleep(0.7)
|
||||||
os.remove(FAKE)
|
os.remove(FAKE)
|
||||||
os.remove(SOURCES_LOG)
|
|
||||||
os.rmdir(tmp)
|
os.rmdir(tmp)
|
||||||
os._exit(1 if failures else 0)
|
os._exit(1 if failures else 0)
|
||||||
@@ -9,12 +9,7 @@ the search runs again with a smaller closing.
|
|||||||
import cv2
|
import cv2
|
||||||
import numpy as np
|
import numpy as np
|
||||||
|
|
||||||
# One thread: OpenCV's pool of one per core costs more than it saves on a frame this small
|
DARK = 30 # pupil pixels are below this (the face around it is 40-180)
|
||||||
# (a 140-240 px window while it follows the pupil). Its idle workers spun and yielded about
|
|
||||||
# 14,000 times a second each, a quarter of a core, beside SteamVR's compositor.
|
|
||||||
cv2.setNumThreads(1)
|
|
||||||
|
|
||||||
DARK = 30 # pupil pixels are below this (the face around it is 40-180)
|
|
||||||
MIN_AREA = 150 # pupil area range in pixels
|
MIN_AREA = 150 # pupil area range in pixels
|
||||||
MAX_AREA = 20000
|
MAX_AREA = 20000
|
||||||
MIN_FILL = 0.75 # blob area / fitted-ellipse area
|
MIN_FILL = 0.75 # blob area / fitted-ellipse area
|
||||||
|
|||||||
@@ -63,13 +63,6 @@ reading only.
|
|||||||
frame when its camera starts the frame after next (no slot is rewritten sooner than three
|
frame when its camera starts the frame after next (no slot is rewritten sooner than three
|
||||||
frames), ignores late writes to the frame just finished, and saves from a separate
|
frames), ignores late writes to the frame just finished, and saves from a separate
|
||||||
thread. fit1 may hold a few percent of torn frames.
|
thread. fit1 may hold a few percent of torn frames.
|
||||||
- `--share` (2026-10-03) checks only the slot each camera writes next, from the two before
|
|
||||||
(the orders above, learned again if they change; all four slots for 2 s after a frame turns
|
|
||||||
up elsewhere), sleeps until 2.5 ms before the next frame is due, then looks every 1 ms with
|
|
||||||
0.5 ms of timer slack. Against the 0.3 ms poll of all eight slots, on simulated cameras:
|
|
||||||
207 wakeups a second instead of 1,486, 0.8% of a core instead of 3.6% (more on the real
|
|
||||||
DMA-BUF memory), no torn or skipped frames, and a frame's start seen 1.35 ms late on
|
|
||||||
average instead of 0.76. `--rec` still polls all slots every 0.3 ms, for its times.
|
|
||||||
- Eye tracking stops when the headset is off ("HMD off, stopping eye tracking"), so
|
- Eye tracking stops when the headset is off ("HMD off, stopping eye tracking"), so
|
||||||
recordings are empty then.
|
recordings are empty then.
|
||||||
- **Which camera is which eye** (capture fit1, 2026-09-29, closing one eye at a time):
|
- **Which camera is which eye** (capture fit1, 2026-09-29, closing one eye at a time):
|
||||||
|
|||||||
+16
-73
@@ -46,7 +46,6 @@
|
|||||||
#include <stdlib.h>
|
#include <stdlib.h>
|
||||||
#include <string.h>
|
#include <string.h>
|
||||||
#include <sys/mman.h>
|
#include <sys/mman.h>
|
||||||
#include <sys/prctl.h>
|
|
||||||
#include <sys/stat.h>
|
#include <sys/stat.h>
|
||||||
#include <sys/syscall.h>
|
#include <sys/syscall.h>
|
||||||
#include <time.h>
|
#include <time.h>
|
||||||
@@ -289,53 +288,20 @@ static int eye_buffer(void) {
|
|||||||
|
|
||||||
// Calls done(frame, slot, time) for every complete eye-camera frame until `seconds` pass
|
// Calls done(frame, slot, time) for every complete eye-camera frame until `seconds` pass
|
||||||
// (forever if negative), a stop signal comes, the tracker process goes away, or keep()
|
// (forever if negative), a stop signal comes, the tracker process goes away, or keep()
|
||||||
// (checked about every 0.25 s, when given) says to stop. Looks every `poll_us`.
|
// (checked about every 0.25 s, when given) says to stop.
|
||||||
//
|
//
|
||||||
// A frame lands over several milliseconds, in bursts, and its last bursts can come after
|
// A frame lands over several milliseconds, in bursts, and its last bursts can come after
|
||||||
// the camera has started its next frame. A slot isn't rewritten until at least three frames
|
// the camera has started its next frame. A slot isn't rewritten until at least three frames
|
||||||
// later (camera 0 cycles 3,0,1,2; camera 1 7,5,4,6,5,7,6,4), so a frame is passed on when
|
// later (camera 0 cycles 3,0,1,2; camera 1 7,5,4,6,5,7,6,4), so a frame is passed on when
|
||||||
// its camera starts the frame after next. Changes to the slot just finished are late bursts,
|
// its camera starts the frame after next. Changes to the slot just finished are late bursts,
|
||||||
// not a new frame. A frame's time is when its slot first changed.
|
// not a new frame. A frame's time is when its slot first changed.
|
||||||
//
|
|
||||||
// Each look checks only the slot each camera writes next: the one that followed the last two
|
|
||||||
// before (after[][], seeded with the orders above and learned as frames come). Every check
|
|
||||||
// reads 256 words spread over a frame in DMA-BUF memory, so all eight slots each time was
|
|
||||||
// most of the cost. A camera with nothing in its expected slot for 1.5 frames, or with no
|
|
||||||
// order known yet, has all its slots checked until a frame comes. When that finds a frame
|
|
||||||
// somewhere else (the order changed, or a frame was missed), all its slots are checked for
|
|
||||||
// SCAN_AFTER_MISS, as before, while after[][] learns the new order. A slot's fingerprint is
|
|
||||||
// taken again when it stops being one of the two in use, so a check later sees only a new frame.
|
|
||||||
// Between frames it sleeps until FRAME_EARLY before the next one is due (the cameras run at
|
|
||||||
// 90 fps, within a ms of each other), then looks every `poll_us`.
|
|
||||||
#define FRAME_DUE (1.5 / 90) // s: a camera that hasn't started a frame by then gets all its slots checked
|
|
||||||
#define SCAN_AFTER_MISS 2.0 // s of checking all slots after a frame came in an unexpected one
|
|
||||||
#define CAMS_IDLE 0.5 // s without a frame from either camera: look 4 times less often
|
|
||||||
#define FRAME_EARLY 0.0025 // s before a frame is due to start looking for it
|
|
||||||
|
|
||||||
// When to start looking for the next frame: FRAME_EARLY before the first camera's is due. A
|
|
||||||
// camera already late (it lost its order, or stopped) means now.
|
|
||||||
static double next_due(const double last[2], double t) {
|
|
||||||
double due = 1e300;
|
|
||||||
for (int cam = 0; cam < 2; cam++) {
|
|
||||||
double d = last[cam] + 1.0 / 90 - FRAME_EARLY;
|
|
||||||
if (t - last[cam] > CAMS_IDLE) continue; // stopped: the other one sets the pace
|
|
||||||
if (d < due) due = d;
|
|
||||||
}
|
|
||||||
return due < 1e300 ? due : t;
|
|
||||||
}
|
|
||||||
|
|
||||||
static void poll_frames(int b, double seconds, int pid, void (*done)(const uint8_t *, int, double),
|
static void poll_frames(int b, double seconds, int pid, void (*done)(const uint8_t *, int, double),
|
||||||
int (*keep)(void), unsigned poll_us) {
|
int (*keep)(void)) {
|
||||||
static const int order[2][8] = {{3, 0, 1, 2, 3, 0, 1, 2}, {7, 5, 4, 6, 5, 7, 6, 4}};
|
|
||||||
int after[EYE_SLOTS][EYE_SLOTS];
|
|
||||||
memset(after, -1, sizeof after);
|
|
||||||
for (int c = 0; c < 2; c++)
|
|
||||||
for (int i = 0; i < 8; i++) after[order[c][i]][order[c][(i + 1) % 8]] = order[c][(i + 2) % 8];
|
|
||||||
uint64_t sig[EYE_SLOTS];
|
uint64_t sig[EYE_SLOTS];
|
||||||
double first[EYE_SLOTS];
|
double first[EYE_SLOTS];
|
||||||
int cur[2] = {-1, -1}, prev[2] = {-1, -1};
|
int cur[2] = {-1, -1}, prev[2] = {-1, -1};
|
||||||
for (int k = 0; k < EYE_SLOTS; k++) sig[k] = frame_sig(bufs[b].p + slot_start(k)), first[k] = 0;
|
for (int k = 0; k < EYE_SLOTS; k++) sig[k] = frame_sig(bufs[b].p + slot_start(k)), first[k] = 0;
|
||||||
double start = now(), checked = start, kept = start, last[2] = {start, start}, scan_until[2] = {0, 0};
|
double start = now(), checked = start, kept = start;
|
||||||
char proc[64];
|
char proc[64];
|
||||||
snprintf(proc, sizeof proc, "/proc/%d", pid);
|
snprintf(proc, sizeof proc, "/proc/%d", pid);
|
||||||
while ((seconds < 0 || now() - start < seconds) && !stop_rec) {
|
while ((seconds < 0 || now() - start < seconds) && !stop_rec) {
|
||||||
@@ -349,36 +315,18 @@ static void poll_frames(int b, double seconds, int pid, void (*done)(const uint8
|
|||||||
if (!keep()) return;
|
if (!keep()) return;
|
||||||
kept = t;
|
kept = t;
|
||||||
}
|
}
|
||||||
for (int cam = 0; cam < 2; cam++) {
|
for (int k = 0; k < EYE_SLOTS; k++) {
|
||||||
int next = prev[cam] >= 0 ? after[prev[cam]][cur[cam]] : -1;
|
uint64_t s = frame_sig(bufs[b].p + slot_start(k));
|
||||||
int all = next < 0 || t - last[cam] > FRAME_DUE || t < scan_until[cam];
|
if (s == sig[k]) continue;
|
||||||
int from = all ? cam * 4 : next, to = all ? cam * 4 + 4 : next + 1;
|
sig[k] = s;
|
||||||
for (int k = from; k < to; k++) {
|
int cam = k >= 4;
|
||||||
uint64_t s = frame_sig(bufs[b].p + slot_start(k));
|
if (k == cur[cam] || k == prev[cam]) continue; // landing, or a late burst
|
||||||
if (s == sig[k]) continue;
|
if (prev[cam] >= 0) done(bufs[b].p + slot_start(prev[cam]), prev[cam], first[prev[cam]]);
|
||||||
sig[k] = s;
|
prev[cam] = cur[cam];
|
||||||
if (k == cur[cam] || k == prev[cam]) continue; // landing, or a late burst
|
cur[cam] = k;
|
||||||
if (next >= 0 && k != next) scan_until[cam] = t + SCAN_AFTER_MISS;
|
first[k] = t;
|
||||||
if (prev[cam] >= 0) {
|
|
||||||
done(bufs[b].p + slot_start(prev[cam]), prev[cam], first[prev[cam]]);
|
|
||||||
after[prev[cam]][cur[cam]] = k;
|
|
||||||
// Out of use from now on: later changes are a new frame.
|
|
||||||
sig[prev[cam]] = frame_sig(bufs[b].p + slot_start(prev[cam]));
|
|
||||||
}
|
|
||||||
prev[cam] = cur[cam];
|
|
||||||
cur[cam] = k;
|
|
||||||
first[k] = t;
|
|
||||||
last[cam] = t;
|
|
||||||
next = prev[cam] >= 0 ? after[prev[cam]][cur[cam]] : -1;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
double wait = poll_us * 1e-6;
|
usleep(300);
|
||||||
if (t - last[0] > CAMS_IDLE && t - last[1] > CAMS_IDLE) {
|
|
||||||
wait *= 4;
|
|
||||||
} else if (next_due(last, t) - t > wait) {
|
|
||||||
wait = next_due(last, t) - t;
|
|
||||||
}
|
|
||||||
usleep((useconds_t)(wait * 1e6));
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -408,7 +356,7 @@ static int rec(double seconds, const char *dir, int pid) {
|
|||||||
pthread_t writer;
|
pthread_t writer;
|
||||||
pthread_create(&writer, NULL, ring_writer, NULL);
|
pthread_create(&writer, NULL, ring_writer, NULL);
|
||||||
double start = now();
|
double start = now();
|
||||||
poll_frames(b, seconds, pid, rec_frame, NULL, 300); // 0.3 ms: recordings' times
|
poll_frames(b, seconds, pid, rec_frame, NULL);
|
||||||
pthread_mutex_lock(&ring.mu);
|
pthread_mutex_lock(&ring.mu);
|
||||||
ring.done = 1;
|
ring.done = 1;
|
||||||
pthread_cond_signal(&ring.cv);
|
pthread_cond_signal(&ring.cv);
|
||||||
@@ -435,10 +383,6 @@ static int rec(double seconds, const char *dir, int pid) {
|
|||||||
#define SHARE_SLOTS 8
|
#define SHARE_SLOTS 8
|
||||||
#define SHARE_MAGIC 0x31434546u // "FEC1"
|
#define SHARE_MAGIC 0x31434546u // "FEC1"
|
||||||
#define WANT_FRESH 3.0 // seconds a touch of the --want file lasts
|
#define WANT_FRESH 3.0 // seconds a touch of the --want file lasts
|
||||||
// How often --share looks for frames, with a timer slack that lets the kernel group the
|
|
||||||
// wakeups: a frame reaches ft-eyes 1 to 1.5 ms after its camera starts the next, not 0.3.
|
|
||||||
#define SHARE_POLL_US 1000
|
|
||||||
#define SHARE_SLACK_NS 500000
|
|
||||||
|
|
||||||
typedef struct {
|
typedef struct {
|
||||||
uint32_t magic, version, width, height, slots, entry_size;
|
uint32_t magic, version, width, height, slots, entry_size;
|
||||||
@@ -518,7 +462,6 @@ static int share_loop(const char *path) {
|
|||||||
.slots = SHARE_SLOTS, .entry_size = (uint32_t)esize};
|
.slots = SHARE_SLOTS, .entry_size = (uint32_t)esize};
|
||||||
signal(SIGINT, on_stop);
|
signal(SIGINT, on_stop);
|
||||||
signal(SIGTERM, on_stop);
|
signal(SIGTERM, on_stop);
|
||||||
if (prctl(PR_SET_TIMERSLACK, SHARE_SLACK_NS, 0, 0, 0) != 0) perror("PR_SET_TIMERSLACK");
|
|
||||||
fprintf(stderr, "ft-eyegrab: sharing frames in %s%s%s\n", path, want_path ? " while wanted by " : "",
|
fprintf(stderr, "ft-eyegrab: sharing frames in %s%s%s\n", path, want_path ? " while wanted by " : "",
|
||||||
want_path ? want_path : "");
|
want_path ? want_path : "");
|
||||||
int pid = -1, idle = -1, missing = 0;
|
int pid = -1, idle = -1, missing = 0;
|
||||||
@@ -543,7 +486,7 @@ static int share_loop(const char *path) {
|
|||||||
if (idle != 0 || missing) fprintf(stderr, "ft-eyegrab: copying frames from eyetracking %d\n", pid);
|
if (idle != 0 || missing) fprintf(stderr, "ft-eyegrab: copying frames from eyetracking %d\n", pid);
|
||||||
idle = 0, missing = 0;
|
idle = 0, missing = 0;
|
||||||
h->tracker_pid = pid;
|
h->tracker_pid = pid;
|
||||||
poll_frames(eye_buffer(), -1, pid, share_frame, wanted, SHARE_POLL_US);
|
poll_frames(eye_buffer(), -1, pid, share_frame, wanted);
|
||||||
struct stat st;
|
struct stat st;
|
||||||
char proc[64];
|
char proc[64];
|
||||||
snprintf(proc, sizeof proc, "/proc/%d", pid);
|
snprintf(proc, sizeof proc, "/proc/%d", pid);
|
||||||
|
|||||||
+4
-42
@@ -50,7 +50,6 @@ import json
|
|||||||
import math
|
import math
|
||||||
import mmap
|
import mmap
|
||||||
import os
|
import os
|
||||||
import select
|
|
||||||
import socket
|
import socket
|
||||||
import struct
|
import struct
|
||||||
import sys
|
import sys
|
||||||
@@ -59,12 +58,7 @@ import time
|
|||||||
from collections import deque
|
from collections import deque
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
# One thread for numpy's BLAS and OpenMP, set before numpy loads: it would start one per core
|
import numpy as np
|
||||||
# (8 here) for small arrays that never need them. eyes_pupil keeps OpenCV to one as well.
|
|
||||||
for _var in ("OPENBLAS_NUM_THREADS", "OMP_NUM_THREADS"):
|
|
||||||
os.environ.setdefault(_var, "1")
|
|
||||||
|
|
||||||
import numpy as np # noqa: E402
|
|
||||||
|
|
||||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||||
import eyes_model # noqa: E402
|
import eyes_model # noqa: E402
|
||||||
@@ -88,15 +82,6 @@ GAP = 3.0 # s without frames: the headset was off, and may sit diffe
|
|||||||
CLICK_BEFORE = 0.3 # a click's frames: the 300 ms before it (like the probe's fixation)
|
CLICK_BEFORE = 0.3 # a click's frames: the 300 ms before it (like the probe's fixation)
|
||||||
CALIB_MIN = 15 # frames an eye needs in a calibration dot's window
|
CALIB_MIN = 15 # frames an eye needs in a calibration dot's window
|
||||||
CALIB_SPREAD = 4.0 # px: more than this and the eye moved during the dot
|
CALIB_SPREAD = 4.0 # px: more than this and the eye moved during the dot
|
||||||
# Waiting for frames. They come only as a counter in shared memory, so there's nothing to block
|
|
||||||
# on: sleep until a camera's next frame is due, then look every POLL. ft-eyegrab passes each one
|
|
||||||
# on within a ms or two of when its camera starts the one after, so they arrive 11.1 ms apart,
|
|
||||||
# give or take that.
|
|
||||||
PERIOD = 1 / 90 # s between a camera's frames
|
|
||||||
EARLY = 0.002 # start looking this long before a frame is due
|
|
||||||
POLL = 0.001 # then this often until it comes
|
|
||||||
STALLED = 0.1 # s without a frame: that camera stopped (headset off), and isn't waited for
|
|
||||||
IDLE_POLL = 0.02 # how often to look while both are stopped
|
|
||||||
|
|
||||||
|
|
||||||
class Cams:
|
class Cams:
|
||||||
@@ -367,7 +352,7 @@ def wait_for_cams(sock, tracker, want):
|
|||||||
return Cams()
|
return Cams()
|
||||||
except (OSError, ValueError, RuntimeError):
|
except (OSError, ValueError, RuntimeError):
|
||||||
serve(sock, tracker)
|
serve(sock, tracker)
|
||||||
select.select([sock], [], [], 0.2)
|
time.sleep(0.2)
|
||||||
|
|
||||||
|
|
||||||
def serve(sock, tracker):
|
def serve(sock, tracker):
|
||||||
@@ -387,24 +372,7 @@ def serve(sock, tracker):
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
|
|
||||||
def below_steamvr():
|
|
||||||
"""Nice 10 and SCHED_BATCH, for this thread and those it starts. ft-eyes runs in the dev
|
|
||||||
container's podman scope, where the gaze service's unit doesn't reach it, so it ran at
|
|
||||||
nice 0 on the cores vrcompositor and vrserver use. Batch also lets a waking ft-eyes wait
|
|
||||||
for the running task's turn instead of taking the core: a frame a few ms late costs the
|
|
||||||
gaze little, a late compositor frame costs a dropped frame in the headset."""
|
|
||||||
try:
|
|
||||||
os.setpriority(os.PRIO_PROCESS, 0, max(os.getpriority(os.PRIO_PROCESS, 0), 10))
|
|
||||||
except OSError as e:
|
|
||||||
print(f"ft-eyes: nice: {e}", file=sys.stderr, flush=True)
|
|
||||||
try:
|
|
||||||
os.sched_setscheduler(0, os.SCHED_BATCH, os.sched_param(0))
|
|
||||||
except (OSError, AttributeError) as e:
|
|
||||||
print(f"ft-eyes: SCHED_BATCH: {e}", file=sys.stderr, flush=True)
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
below_steamvr()
|
|
||||||
verbose = "-v" in sys.argv
|
verbose = "-v" in sys.argv
|
||||||
if "--watch-stdin" in sys.argv:
|
if "--watch-stdin" in sys.argv:
|
||||||
# Run by ft-gazed through distrobox, which doesn't pass a stop on: quit when our
|
# Run by ft-gazed through distrobox, which doesn't pass a stop on: quit when our
|
||||||
@@ -430,7 +398,6 @@ def main():
|
|||||||
print("ft-eyes: frames found, tracking", file=sys.stderr, flush=True)
|
print("ft-eyes: frames found, tracking", file=sys.stderr, flush=True)
|
||||||
seen = [cams.count(0), cams.count(1)]
|
seen = [cams.count(0), cams.count(1)]
|
||||||
report = time.monotonic()
|
report = time.monotonic()
|
||||||
arrived = [0.0, 0.0] # when each camera's newest frame was seen (monotonic)
|
|
||||||
while True:
|
while True:
|
||||||
serve(sock, tracker)
|
serve(sock, tracker)
|
||||||
want()
|
want()
|
||||||
@@ -440,7 +407,6 @@ def main():
|
|||||||
if n == seen[c]:
|
if n == seen[c]:
|
||||||
continue
|
continue
|
||||||
seen[c] = n
|
seen[c] = n
|
||||||
arrived[c] = time.monotonic()
|
|
||||||
got = cams.frame(c, n - 1) # only the newest: never fall behind
|
got = cams.frame(c, n - 1) # only the newest: never fall behind
|
||||||
if got:
|
if got:
|
||||||
eyes[c].feed(*got)
|
eyes[c].feed(*got)
|
||||||
@@ -459,6 +425,8 @@ def main():
|
|||||||
shifts += [float(v) for v in e.shift.value]
|
shifts += [float(v) for v in e.shift.value]
|
||||||
pupils += [e.gaze[3], e.gaze[4]] if fresh else [math.nan, math.nan]
|
pupils += [e.gaze[3], e.gaze[4]] if fresh else [math.nan, math.nan]
|
||||||
out.write(latest, (yaw, pitch), flags, per, shifts, pupils)
|
out.write(latest, (yaw, pitch), flags, per, shifts, pupils)
|
||||||
|
else:
|
||||||
|
time.sleep(0.001)
|
||||||
now = time.monotonic()
|
now = time.monotonic()
|
||||||
if now - report >= 5:
|
if now - report >= 5:
|
||||||
if verbose:
|
if verbose:
|
||||||
@@ -476,12 +444,6 @@ def main():
|
|||||||
print("ft-eyes: frames went away; waiting", file=sys.stderr, flush=True)
|
print("ft-eyes: frames went away; waiting", file=sys.stderr, flush=True)
|
||||||
cams = wait_for_cams(sock, tracker, want)
|
cams = wait_for_cams(sock, tracker, want)
|
||||||
seen = [cams.count(0), cams.count(1)]
|
seen = [cams.count(0), cams.count(1)]
|
||||||
# Until the next frame is due (a command on the socket wakes us sooner).
|
|
||||||
wait = IDLE_POLL
|
|
||||||
for c in (0, 1):
|
|
||||||
if now - arrived[c] < STALLED:
|
|
||||||
wait = min(wait, max(arrived[c] + PERIOD - EARLY - now, POLL))
|
|
||||||
select.select([sock], [], [], wait)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
|
|||||||
+72
-64
@@ -306,14 +306,19 @@ int main() {
|
|||||||
std::printf("displays on: %s\n", why.c_str());
|
std::printf("displays on: %s\n", why.c_str());
|
||||||
};
|
};
|
||||||
|
|
||||||
|
// SteamVR is asked every 100 ms (events, the activity level, poses: IPC calls to
|
||||||
|
// vrserver). Input wakes the loop too, to wake the displays at once, but a moving mouse
|
||||||
|
// sends hundreds of events a second, so those wakes only drain the devices.
|
||||||
|
constexpr auto kTick = std::chrono::milliseconds(100);
|
||||||
|
auto lastVr = now - kTick;
|
||||||
std::vector<pollfd> fds;
|
std::vector<pollfd> fds;
|
||||||
while (!stopSignal) {
|
while (!stopSignal) {
|
||||||
// Sleep until input or 100 ms: input wakes the displays at once, poses every tick.
|
|
||||||
fds.clear();
|
fds.clear();
|
||||||
fds.push_back({ctl, POLLIN, 0});
|
fds.push_back({ctl, POLLIN, 0});
|
||||||
for (auto &[path, d] : inputs)
|
for (auto &[path, d] : inputs)
|
||||||
if (d.fd >= 0) fds.push_back({d.fd, POLLIN, 0});
|
if (d.fd >= 0) fds.push_back({d.fd, POLLIN, 0});
|
||||||
poll(fds.data(), fds.size(), 100);
|
const auto wait = std::chrono::ceil<std::chrono::milliseconds>(kTick - (Clock::now() - lastVr));
|
||||||
|
poll(fds.data(), fds.size(), std::clamp<int>(wait.count(), 0, kTick.count()));
|
||||||
now = Clock::now();
|
now = Clock::now();
|
||||||
const bool grace = now < graceUntil;
|
const bool grace = now < graceUntil;
|
||||||
std::string use; // why this tick counts as use
|
std::string use; // why this tick counts as use
|
||||||
@@ -329,71 +334,74 @@ int main() {
|
|||||||
lastConf = now;
|
lastConf = now;
|
||||||
}
|
}
|
||||||
|
|
||||||
vr::VREvent_t ev;
|
if (now - lastVr >= kTick) {
|
||||||
while (sys->PollNextEvent(&ev, sizeof ev)) {
|
lastVr = now;
|
||||||
if (ev.eventType == vr::VREvent_Quit) {
|
vr::VREvent_t ev;
|
||||||
stopSignal = SIGTERM;
|
while (sys->PollNextEvent(&ev, sizeof ev)) {
|
||||||
sys->AcknowledgeQuit_Exiting();
|
if (ev.eventType == vr::VREvent_Quit) {
|
||||||
} else if (ev.eventType == vr::VREvent_TrackedDeviceActivated ||
|
stopSignal = SIGTERM;
|
||||||
ev.eventType == vr::VREvent_TrackedDeviceDeactivated) {
|
sys->AcknowledgeQuit_Exiting();
|
||||||
lastClasses = now - std::chrono::hours(1);
|
} else if (ev.eventType == vr::VREvent_TrackedDeviceActivated ||
|
||||||
}
|
ev.eventType == vr::VREvent_TrackedDeviceDeactivated) {
|
||||||
}
|
lastClasses = now - std::chrono::hours(1);
|
||||||
if (now - lastClasses > std::chrono::seconds(5)) {
|
|
||||||
for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i)
|
|
||||||
classes[i] = sys->GetTrackedDeviceClass(i);
|
|
||||||
lastClasses = now;
|
|
||||||
}
|
|
||||||
|
|
||||||
// SteamVR's standby (headset taken off) turns the displays off itself; putting the
|
|
||||||
// headset on again (or the mount covering the sensor again) counts as use.
|
|
||||||
// A change counts once it has held for a second (the first reading after connecting
|
|
||||||
// is Idle).
|
|
||||||
const auto level = sys->GetTrackedDeviceActivityLevel(vr::k_unTrackedDeviceIndex_Hmd);
|
|
||||||
const bool awayNow = level == vr::k_EDeviceActivityLevel_Idle || level == vr::k_EDeviceActivityLevel_Standby ||
|
|
||||||
level == vr::k_EDeviceActivityLevel_Idle_Timeout;
|
|
||||||
if (awayNow == away) awaySince = now;
|
|
||||||
if (awayNow != away && now - awaySince >= std::chrono::seconds(1)) {
|
|
||||||
away = awayNow;
|
|
||||||
std::printf("SteamVR: headset %s\n", away ? "off (SteamVR's standby has the displays)" : "on");
|
|
||||||
if (!away) use = "headset on again";
|
|
||||||
}
|
|
||||||
|
|
||||||
// Raw poses: recentering or a new play area doesn't move anything.
|
|
||||||
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseRawAndUncalibrated, 0, poses, vr::k_unMaxTrackedDeviceCount);
|
|
||||||
for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) {
|
|
||||||
const auto c = classes[i];
|
|
||||||
if (c != vr::TrackedDeviceClass_HMD && c != vr::TrackedDeviceClass_Controller &&
|
|
||||||
c != vr::TrackedDeviceClass_GenericTracker)
|
|
||||||
continue;
|
|
||||||
const auto &p = poses[i];
|
|
||||||
if (!p.bPoseIsValid || p.eTrackingResult != vr::TrackingResult_Running_OK) {
|
|
||||||
anchors[i].set = false; // tracking back or a controller turned on: start over
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
if (!anchors[i].set || grace || now - anchors[i].at > kMoveWindow) {
|
|
||||||
SetAnchor(anchors[i], p.mDeviceToAbsoluteTracking, now);
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
double metres, degrees;
|
|
||||||
Distance(anchors[i], p.mDeviceToAbsoluteTracking, metres, degrees);
|
|
||||||
if (metres > moveMetres || degrees > moveDegrees) {
|
|
||||||
SetAnchor(anchors[i], p.mDeviceToAbsoluteTracking, now);
|
|
||||||
if (use.empty()) {
|
|
||||||
char why[96];
|
|
||||||
std::snprintf(why, sizeof why, "%s %u moved %.1f mm, %.2f deg",
|
|
||||||
c == vr::TrackedDeviceClass_HMD ? "headset" : "device", i, metres * 1000, degrees);
|
|
||||||
use = why;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
if (now - lastClasses > std::chrono::seconds(5)) {
|
||||||
|
for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i)
|
||||||
|
classes[i] = sys->GetTrackedDeviceClass(i);
|
||||||
|
lastClasses = now;
|
||||||
|
}
|
||||||
|
|
||||||
if (off && ReadInt(kBacklight, -1) > 0) {
|
// SteamVR's standby (headset taken off) turns the displays off itself; putting the
|
||||||
// Something else lit them (SteamVR leaving standby, the brightness setting).
|
// headset on again (or the mount covering the sensor again) counts as use.
|
||||||
off = false;
|
// A change counts once it has held for a second (the first reading after connecting
|
||||||
std::remove(SavedPath().c_str());
|
// is Idle).
|
||||||
use = "turned on elsewhere";
|
const auto level = sys->GetTrackedDeviceActivityLevel(vr::k_unTrackedDeviceIndex_Hmd);
|
||||||
std::printf("displays on: turned on elsewhere\n");
|
const bool awayNow = level == vr::k_EDeviceActivityLevel_Idle || level == vr::k_EDeviceActivityLevel_Standby ||
|
||||||
|
level == vr::k_EDeviceActivityLevel_Idle_Timeout;
|
||||||
|
if (awayNow == away) awaySince = now;
|
||||||
|
if (awayNow != away && now - awaySince >= std::chrono::seconds(1)) {
|
||||||
|
away = awayNow;
|
||||||
|
std::printf("SteamVR: headset %s\n", away ? "off (SteamVR's standby has the displays)" : "on");
|
||||||
|
if (!away) use = "headset on again";
|
||||||
|
}
|
||||||
|
|
||||||
|
// Raw poses: recentering or a new play area doesn't move anything.
|
||||||
|
sys->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseRawAndUncalibrated, 0, poses, vr::k_unMaxTrackedDeviceCount);
|
||||||
|
for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) {
|
||||||
|
const auto c = classes[i];
|
||||||
|
if (c != vr::TrackedDeviceClass_HMD && c != vr::TrackedDeviceClass_Controller &&
|
||||||
|
c != vr::TrackedDeviceClass_GenericTracker)
|
||||||
|
continue;
|
||||||
|
const auto &p = poses[i];
|
||||||
|
if (!p.bPoseIsValid || p.eTrackingResult != vr::TrackingResult_Running_OK) {
|
||||||
|
anchors[i].set = false; // tracking back or a controller turned on: start over
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (!anchors[i].set || grace || now - anchors[i].at > kMoveWindow) {
|
||||||
|
SetAnchor(anchors[i], p.mDeviceToAbsoluteTracking, now);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
double metres, degrees;
|
||||||
|
Distance(anchors[i], p.mDeviceToAbsoluteTracking, metres, degrees);
|
||||||
|
if (metres > moveMetres || degrees > moveDegrees) {
|
||||||
|
SetAnchor(anchors[i], p.mDeviceToAbsoluteTracking, now);
|
||||||
|
if (use.empty()) {
|
||||||
|
char why[96];
|
||||||
|
std::snprintf(why, sizeof why, "%s %u moved %.1f mm, %.2f deg",
|
||||||
|
c == vr::TrackedDeviceClass_HMD ? "headset" : "device", i, metres * 1000, degrees);
|
||||||
|
use = why;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (off && ReadInt(kBacklight, -1) > 0) {
|
||||||
|
// Something else lit them (SteamVR leaving standby, the brightness setting).
|
||||||
|
off = false;
|
||||||
|
std::remove(SavedPath().c_str());
|
||||||
|
use = "turned on elsewhere";
|
||||||
|
std::printf("displays on: turned on elsewhere\n");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if (!use.empty()) {
|
if (!use.empty()) {
|
||||||
lastUse = now;
|
lastUse = now;
|
||||||
|
|||||||
@@ -83,6 +83,7 @@ class Window(Adw.ApplicationWindow):
|
|||||||
self.status = {}
|
self.status = {}
|
||||||
self.revealed = False
|
self.revealed = False
|
||||||
self.setting = False # the switch is being set from the status, not by the user
|
self.setting = False # the switch is being set from the status, not by the user
|
||||||
|
self.asking = False # a status check is running
|
||||||
|
|
||||||
self.toasts = Adw.ToastOverlay()
|
self.toasts = Adw.ToastOverlay()
|
||||||
view = Adw.ToolbarView()
|
view = Adw.ToolbarView()
|
||||||
@@ -139,16 +140,27 @@ class Window(Adw.ApplicationWindow):
|
|||||||
"/tmp/frametop-remote.log and /tmp/frametop-vnc.log.")
|
"/tmp/frametop-remote.log and /tmp/frametop-vnc.log.")
|
||||||
page.add(about)
|
page.add(about)
|
||||||
|
|
||||||
|
# Each check runs remote-ctl.sh (bash, curl, and python3 for the tailnet name), so
|
||||||
|
# it runs every 5 seconds, when the window comes to the front, and after a change.
|
||||||
self.refresh()
|
self.refresh()
|
||||||
GLib.timeout_add_seconds(2, self.refresh)
|
GLib.timeout_add_seconds(5, self.refresh)
|
||||||
|
self.connect("notify::is-active", lambda win, _param: win.is_active() and win.refresh())
|
||||||
|
|
||||||
# --- status ---
|
# --- status ---
|
||||||
def refresh(self):
|
def refresh(self):
|
||||||
proc = Gio.Subprocess.new([CTL, "status"], Gio.SubprocessFlags.STDOUT_PIPE | Gio.SubprocessFlags.STDERR_SILENCE)
|
if not self.asking:
|
||||||
proc.communicate_utf8_async(None, None, self.on_status)
|
self.asking = True
|
||||||
|
proc = Gio.Subprocess.new([CTL, "status"], Gio.SubprocessFlags.STDOUT_PIPE | Gio.SubprocessFlags.STDERR_SILENCE)
|
||||||
|
proc.communicate_utf8_async(None, None, self.on_status)
|
||||||
return True
|
return True
|
||||||
|
|
||||||
|
def refresh_soon(self):
|
||||||
|
"""Again in a moment: remote access takes a second or two to start or stop."""
|
||||||
|
self.refresh()
|
||||||
|
GLib.timeout_add_seconds(2, lambda: self.refresh() and False)
|
||||||
|
|
||||||
def on_status(self, proc, result):
|
def on_status(self, proc, result):
|
||||||
|
self.asking = False
|
||||||
try:
|
try:
|
||||||
_, out, _ = proc.communicate_utf8_finish(result)
|
_, out, _ = proc.communicate_utf8_finish(result)
|
||||||
except GLib.Error:
|
except GLib.Error:
|
||||||
@@ -197,7 +209,7 @@ class Window(Adw.ApplicationWindow):
|
|||||||
self.toast("Remote access turns on at the next desktop restart")
|
self.toast("Remote access turns on at the next desktop restart")
|
||||||
else:
|
else:
|
||||||
self.toast("Remote access on")
|
self.toast("Remote access on")
|
||||||
self.refresh()
|
self.refresh_soon()
|
||||||
|
|
||||||
def on_reveal(self, button):
|
def on_reveal(self, button):
|
||||||
self.revealed = button.get_active()
|
self.revealed = button.get_active()
|
||||||
@@ -221,7 +233,7 @@ class Window(Adw.ApplicationWindow):
|
|||||||
if self.status.get("running") == "1":
|
if self.status.get("running") == "1":
|
||||||
subprocess.run([CTL, "restart"], stdin=subprocess.DEVNULL, capture_output=True)
|
subprocess.run([CTL, "restart"], stdin=subprocess.DEVNULL, capture_output=True)
|
||||||
self.toast("New password set" + (": remote access restarted" if self.status.get("running") == "1" else ""))
|
self.toast("New password set" + (": remote access restarted" if self.status.get("running") == "1" else ""))
|
||||||
self.refresh()
|
self.refresh_soon()
|
||||||
|
|
||||||
def copy(self, text, done):
|
def copy(self, text, done):
|
||||||
if text:
|
if text:
|
||||||
|
|||||||
+14
-6
@@ -46,8 +46,9 @@ PACKAGES = {
|
|||||||
"deckard-steamvr-rel": "the 3D mouse on the dashboard, on SteamVR Settings, and on a SteamVR "
|
"deckard-steamvr-rel": "the 3D mouse on the dashboard, on SteamVR Settings, and on a SteamVR "
|
||||||
"window's grab bar; picking up a controller; mapped controller buttons; gaze",
|
"window's grab bar; picking up a controller; mapped controller buttons; gaze",
|
||||||
"kwin": "clicks near the far edge of a screen whose scale isn't 1; every screen comes back "
|
"kwin": "clicks near the far edge of a screen whose scale isn't 1; every screen comes back "
|
||||||
"after a desktop restart; floating a window",
|
"after a desktop restart; floating a window; no blur behind the taskbar's menus",
|
||||||
"plasma-workspace": "the taskbar and panels after a desktop restart",
|
"plasma-workspace": "the taskbar and panels after a desktop restart; no DiscoverNotifier or "
|
||||||
|
"ibus-daemon inside the desktop",
|
||||||
"gamescope": "the headset's volume buttons with nothing focused; typing goes where you last clicked",
|
"gamescope": "the headset's volume buttons with nothing focused; typing goes where you last clicked",
|
||||||
"bluez": "a Bluetooth mouse reconnecting after it sleeps",
|
"bluez": "a Bluetooth mouse reconnecting after it sleeps",
|
||||||
}
|
}
|
||||||
@@ -176,14 +177,21 @@ def check_host():
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
with open("/usr/bin/kwin_wayland", "rb") as f:
|
with open("/usr/bin/kwin_wayland", "rb") as f:
|
||||||
# Qt keeps the option name as a UTF-16 string literal.
|
kwin = f.read()
|
||||||
output_count = "output-count".encode("utf-16-le") in f.read()
|
|
||||||
except OSError:
|
except OSError:
|
||||||
output_count = False
|
kwin = b""
|
||||||
if output_count:
|
# Qt keeps the option name as a UTF-16 string literal.
|
||||||
|
if "output-count".encode("utf-16-le") in kwin:
|
||||||
report("ok", "KWin", "has --output-count (one output per screen)")
|
report("ok", "KWin", "has --output-count (one output per screen)")
|
||||||
else:
|
else:
|
||||||
report("FAIL", "KWin", "no --output-count option: the desktop gets one screen at most")
|
report("FAIL", "KWin", "no --output-count option: the desktop gets one screen at most")
|
||||||
|
# The session turns these built-in effects off by id (blurEnabled, contrastEnabled in kwinrc).
|
||||||
|
gone = [e for e in ("blur", "contrast") if b"KWin::%s_factory" % e.encode() not in kwin]
|
||||||
|
if not gone:
|
||||||
|
report("ok", "KWin effects", "blur and contrast, which the desktop turns off, keep their ids")
|
||||||
|
else:
|
||||||
|
report("warn", "KWin effects", f"no built-in {', '.join(gone)} effect: renamed? The desktop's "
|
||||||
|
"kwinrc may no longer turn it off (session/frametop-session.sh)")
|
||||||
|
|
||||||
if systemctl("cat", "steamvr.service"):
|
if systemctl("cat", "steamvr.service"):
|
||||||
report("ok", "steamvr.service", "Frametop's services start and stop with it")
|
report("ok", "steamvr.service", "Frametop's services start and stop with it")
|
||||||
|
|||||||
@@ -234,6 +234,40 @@ elif [ "$(kreadconfig6 --file "$kwinrc" --group org.kde.kdecoration2 --key theme
|
|||||||
kwriteconfig6 --file "$kwinrc" --group org.kde.kdecoration2 --key theme --delete
|
kwriteconfig6 --file "$kwinrc" --group org.kde.kdecoration2 --key theme --delete
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# KWin draws through zink on the headset's GPU, which vrcompositor needs, so the blur and
|
||||||
|
# background contrast behind panels and menus and the window animations cost frames for
|
||||||
|
# little. They're off by default in this desktop. Each is written once, before KWin first
|
||||||
|
# reads it, and only if this desktop's config doesn't have it yet; the marker keeps it
|
||||||
|
# from coming back, since System Settings deletes a setting put back to KDE's default.
|
||||||
|
# docs/reference.md says how to turn them on again.
|
||||||
|
default() { # file group key value
|
||||||
|
[ -n "$(kreadconfig6 --file "$1" --group "$2" --key "$3")" ] || kwriteconfig6 --file "$1" --group "$2" --key "$3" "$4"
|
||||||
|
}
|
||||||
|
frametoprc=$XDG_CONFIG_HOME/frametoprc
|
||||||
|
if [ "$(kreadconfig6 --file "$frametoprc" --group Defaults --key effects)" != 1 ]; then
|
||||||
|
default "$kwinrc" Plugins blurEnabled false
|
||||||
|
default "$kwinrc" Plugins contrastEnabled false
|
||||||
|
default "$XDG_CONFIG_HOME/kdeglobals" KDE AnimationDurationFactor 0
|
||||||
|
kwriteconfig6 --file "$frametoprc" --group Defaults --key effects 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# System autostart entries this desktop doesn't need, hidden for it alone by a copy with
|
||||||
|
# Hidden=true in its own autostart folder, once and unless there's a file of that name
|
||||||
|
# already. Discover's update notifier starts Discover itself to check for updates (about
|
||||||
|
# 600 MB and a share of a core, with Flatpak's helper and AppStream behind it); updates
|
||||||
|
# come with SteamOS and from Discover in the stock desktop. IBus can't reach the
|
||||||
|
# desktop's apps: KWin's input method is ft-textinput, and the session drops the
|
||||||
|
# variables that point apps at IBus or XIM. Deleting the copy brings an entry back.
|
||||||
|
if [ "$(kreadconfig6 --file "$frametoprc" --group Defaults --key autostart)" != 1 ]; then
|
||||||
|
for entry in org.kde.discover.notifier ibus; do
|
||||||
|
src=/etc/xdg/autostart/$entry.desktop dst=$XDG_CONFIG_HOME/autostart/$entry.desktop
|
||||||
|
[ -r "$src" ] && [ ! -e "$dst" ] || continue
|
||||||
|
mkdir -p "$(dirname "$dst")"
|
||||||
|
sed '/^\[Desktop Entry\]$/a Hidden=true' "$src" > "$dst"
|
||||||
|
done
|
||||||
|
kwriteconfig6 --file "$frametoprc" --group Defaults --key autostart 1
|
||||||
|
fi
|
||||||
|
|
||||||
# Profiles reopen apps (docs/profiles.md), so Plasma's own session restore stays off here;
|
# Profiles reopen apps (docs/profiles.md), so Plasma's own session restore stays off here;
|
||||||
# with both, apps would open twice.
|
# with both, apps would open twice.
|
||||||
kwriteconfig6 --file "$XDG_CONFIG_HOME/ksmserverrc" --group General --key loginMode emptySession
|
kwriteconfig6 --file "$XDG_CONFIG_HOME/ksmserverrc" --group General --key loginMode emptySession
|
||||||
|
|||||||
+105
-15
@@ -11,6 +11,16 @@
|
|||||||
# screen placed lower or further right. Instead the VNC screen is the primary's size, and
|
# screen placed lower or further right. Instead the VNC screen is the primary's size, and
|
||||||
# the workspace-sized RDP window inside it is shifted so the primary fills it. The pointer
|
# the workspace-sized RDP window inside it is shifted so the primary fills it. The pointer
|
||||||
# maps 1:1. When the layout changes, the VNC screen resizes and the RDP client reconnects.
|
# maps 1:1. When the layout changes, the VNC screen resizes and the RDP client reconnects.
|
||||||
|
#
|
||||||
|
# The RDP client runs only while someone watches. While connected, krdp captures and
|
||||||
|
# H.264-encodes every redraw in software, about 60% of a core, with FreeRDP and Xvnc adding
|
||||||
|
# about 30% more, even with no VNC viewer. krdp starts its capture per RDP connection and
|
||||||
|
# stops it when the connection closes, so it idles without one. So FreeRDP starts when a
|
||||||
|
# VNC client connects (the screen is black for the few seconds that takes) and stops
|
||||||
|
# VNC_IDLE_SEC (45) seconds after the last one leaves. Xvnc has no hook for clients, so ss
|
||||||
|
# counts them: whenever Xvnc writes to its log (it logs each connection), every second
|
||||||
|
# while FreeRDP runs, and every 5 seconds otherwise. The log only wakes this script up;
|
||||||
|
# what it says doesn't matter.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
here=$(dirname "$(readlink -f "$0")")
|
here=$(dirname "$(readlink -f "$0")")
|
||||||
@@ -57,16 +67,51 @@ stop_rdp() { pkill -f "[x]freerdp /v:127.0.0.1:$rdp_port " 2>/dev/null || true;
|
|||||||
trap 'stop_rdp; pkill -f "[X]vnc $display " 2>/dev/null || true' EXIT
|
trap 'stop_rdp; pkill -f "[X]vnc $display " 2>/dev/null || true' EXIT
|
||||||
|
|
||||||
box bash -c 'vncpasswd -f < "$1/vnc-password" > "$1/vnc-passwd.bin" && chmod 600 "$1/vnc-passwd.bin"' - "$creds"
|
box bash -c 'vncpasswd -f < "$1/vnc-password" > "$1/vnc-passwd.bin" && chmod 600 "$1/vnc-passwd.bin"' - "$creds"
|
||||||
box Xvnc "$display" -geometry "${w}x${h}" -depth 24 \
|
exec {xlog}< <(box Xvnc "$display" -geometry "${w}x${h}" -depth 24 \
|
||||||
-interface "$addr" -rfbport "$vnc_port" \
|
-interface "$addr" -rfbport "$vnc_port" \
|
||||||
-SecurityTypes VncAuth -PasswordFile "$creds/vnc-passwd.bin" \
|
-SecurityTypes VncAuth -PasswordFile "$creds/vnc-passwd.bin" \
|
||||||
-AlwaysShared -desktop "Steam Frame (Frametop)" &
|
-AlwaysShared -desktop "Steam Frame (Frametop)" 2>&1)
|
||||||
xvnc=$!
|
xvnc=$!
|
||||||
sleep 2
|
|
||||||
|
|
||||||
# Keep an RDP connection open inside the VNC screen. Reconnect if it drops or the layout changes.
|
# Wait up to $1 seconds, less if Xvnc logs something; its lines go on to this log.
|
||||||
|
nap() {
|
||||||
|
local line rc=0
|
||||||
|
IFS= read -rt "$1" -u "$xlog" line || rc=$?
|
||||||
|
if [ $rc -eq 0 ]; then
|
||||||
|
printf '%s\n' "$line"
|
||||||
|
while IFS= read -rt 0.1 -u "$xlog" line; do printf '%s\n' "$line"; done
|
||||||
|
elif [ $rc -le 128 ]; then
|
||||||
|
sleep 1 # Xvnc's output closed: it's exiting
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
nap 2
|
||||||
|
|
||||||
|
# A connected VNC client: an established TCP connection to Xvnc's port. Any connection
|
||||||
|
# counts, authenticated or not; it's on the tailnet only.
|
||||||
|
clients() { [ -n "$(ss -Htn state established "( sport = :$vnc_port )" 2>/dev/null)" ]; }
|
||||||
|
|
||||||
|
# ft-screens drops screens you aren't looking at to a low frame rate, and krdp would
|
||||||
|
# stream that. "watch SECONDS" asks it for full rate on every screen for that long: sent
|
||||||
|
# when a client connects and renewed every few seconds while one stays, so it lapses by
|
||||||
|
# itself if this script dies. An older ft-screens just answers that it doesn't know it.
|
||||||
|
watch() {
|
||||||
|
if command -v socat >/dev/null; then
|
||||||
|
printf 'watch 15' | socat -u - ABSTRACT-SENDTO:ft_screens 2>/dev/null
|
||||||
|
else
|
||||||
|
python3 -c 'import socket; socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM).sendto(b"watch 15", "\0ft_screens")' 2>/dev/null
|
||||||
|
fi || true
|
||||||
|
}
|
||||||
|
|
||||||
|
# What remote-view depends on: KWin's saved outputs (positions, scales, primary) and
|
||||||
|
# Frametop's layout (screen sizes). It's read again only after one of these changes, and
|
||||||
|
# once a minute in case a change didn't touch them, and only while FreeRDP runs.
|
||||||
|
layout_files=("$HOME/.config/frametop/kwinoutputconfig.json" "$HOME/.config/frametop-layout.json")
|
||||||
|
stamp() { stat -c %y "${layout_files[@]}" 2>/dev/null || true; }
|
||||||
|
|
||||||
|
# Start FreeRDP inside the VNC screen, sized and shifted for $v.
|
||||||
# /cert:ignore is fine here: the connection never leaves this host.
|
# /cert:ignore is fine here: the connection never leaves this host.
|
||||||
while kill -0 $xvnc 2>/dev/null; do
|
start_rdp() {
|
||||||
read -r x y w h ww wh <<< "$v"
|
read -r x y w h ww wh <<< "$v"
|
||||||
box env DISPLAY=$display bash -c '
|
box env DISPLAY=$display bash -c '
|
||||||
size=$1 x=$2 y=$3 ww=$4 wh=$5 creds=$6 rdp_port=$7
|
size=$1 x=$2 y=$3 ww=$4 wh=$5 creds=$6 rdp_port=$7
|
||||||
@@ -80,6 +125,7 @@ while kill -0 $xvnc 2>/dev/null; do
|
|||||||
rdp=$!
|
rdp=$!
|
||||||
# FreeRDP takes no negative position, so move its window once it is up.
|
# FreeRDP takes no negative position, so move its window once it is up.
|
||||||
for _ in $(seq 60); do
|
for _ in $(seq 60); do
|
||||||
|
kill -0 $rdp 2>/dev/null || break
|
||||||
win=$(xdotool search --class xfreerdp 2>/dev/null | tail -1)
|
win=$(xdotool search --class xfreerdp 2>/dev/null | tail -1)
|
||||||
[ -n "$win" ] && break
|
[ -n "$win" ] && break
|
||||||
sleep 0.5
|
sleep 0.5
|
||||||
@@ -88,14 +134,58 @@ while kill -0 $xvnc 2>/dev/null; do
|
|||||||
wait $rdp
|
wait $rdp
|
||||||
' vnc-rdp "${w}x$h" "$x" "$y" "$ww" "$wh" "$creds" "$rdp_port" || true &
|
' vnc-rdp "${w}x$h" "$x" "$y" "$ww" "$wh" "$creds" "$rdp_port" || true &
|
||||||
rdp=$!
|
rdp=$!
|
||||||
while kill -0 $rdp 2>/dev/null; do
|
}
|
||||||
sleep 5
|
|
||||||
now=$(view) || continue
|
idle_sec=${VNC_IDLE_SEC:-45}
|
||||||
[ -n "$now" ] && [ "$now" != "$v" ] || continue
|
rdp= # FreeRDP's job while it runs
|
||||||
echo "layout changed: $v -> $now"
|
seen=0 # when a client was last seen ($SECONDS)
|
||||||
v=$now
|
watched=-99 # when "watch" was last sent
|
||||||
stop_rdp
|
stamp_v= # stamp() when $v was read
|
||||||
done
|
recheck=0 # read the layout every 2 seconds until then
|
||||||
wait $rdp 2>/dev/null || true
|
next_view=0 # next time to read it
|
||||||
sleep 2
|
while kill -0 $xvnc 2>/dev/null; do
|
||||||
|
if clients; then
|
||||||
|
if [ $((SECONDS - watched)) -ge 5 ]; then watch; watched=$SECONDS; fi
|
||||||
|
seen=$SECONDS
|
||||||
|
if [ -z "$rdp" ]; then
|
||||||
|
echo "VNC client connected, starting the RDP client"
|
||||||
|
now=$(view) && [ -n "$now" ] && v=$now
|
||||||
|
stamp_v=$(stamp) recheck=0 next_view=$((SECONDS + 60))
|
||||||
|
start_rdp
|
||||||
|
fi
|
||||||
|
elif [ "$seen" -ne 0 ]; then
|
||||||
|
watched=-99
|
||||||
|
if [ $((SECONDS - seen)) -ge "$idle_sec" ]; then
|
||||||
|
seen=0
|
||||||
|
if [ -n "$rdp" ]; then
|
||||||
|
echo "no VNC client for ${idle_sec}s, stopping the RDP client"
|
||||||
|
stop_rdp
|
||||||
|
wait "$rdp" 2>/dev/null || true
|
||||||
|
rdp=
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
if [ -n "$rdp" ] && ! kill -0 "$rdp" 2>/dev/null; then
|
||||||
|
# It dropped, or the layout changed: reconnect next round if a client is still there.
|
||||||
|
wait "$rdp" 2>/dev/null || true
|
||||||
|
rdp=
|
||||||
|
nap 2
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
if [ -n "$rdp" ]; then
|
||||||
|
# A layout change shows up in KWin's outputs a few seconds after the files change.
|
||||||
|
s=$(stamp)
|
||||||
|
[ "$s" = "$stamp_v" ] || { stamp_v=$s; recheck=$((SECONDS + 10)) next_view=$SECONDS; }
|
||||||
|
if [ "$SECONDS" -ge "$next_view" ]; then
|
||||||
|
next_view=$((SECONDS + 60))
|
||||||
|
[ "$SECONDS" -ge "$recheck" ] || next_view=$((SECONDS + 2))
|
||||||
|
now=$(view) || now=
|
||||||
|
if [ -n "$now" ] && [ "$now" != "$v" ]; then
|
||||||
|
echo "layout changed: $v -> $now"
|
||||||
|
v=$now
|
||||||
|
stop_rdp
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
if [ -n "$rdp" ] || [ "$seen" -ne 0 ]; then nap 1; else nap 5; fi
|
||||||
done
|
done
|
||||||
Reference in new issue
Block a user