mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-04 22:00:05 +02:00
Defer hand tracking; the README leads with what Frametop does now
- install.sh no longer offers hand tracking (it's heavy on the CPU and needs more work); hands/ still builds and installs by hand - the build container drops python3-opencv and python3-numpy, which only hand tracking's tools used: Fedora's OpenCV pulls in over a GB - README: a new opening and feature list, the Use section by topic (screens, mouse, floating windows, profiles, gaze), and new known limitations Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
c257000a23
commit
7bcda94267
5 files changed
+88
-54
No files matched your search
@@ -1,14 +1,15 @@
|
||||
# Frametop
|
||||
|
||||
Frametop puts a multi-monitor KDE Plasma desktop into SteamVR on the Valve Steam Frame, and lets a Bluetooth mouse drive all of SteamVR. It installs and runs on the headset itself.
|
||||
Frametop is a desktop for the Steam Frame that runs on the headset itself, with no PC. Put several monitors around you, pull app windows out to float on their own, and point with a mouse, or with your eyes.
|
||||
|
||||
Each screen is its own monitor with its own resolution, so you can have an ultrawide in the middle and two portrait screens beside it, at whatever size and distance you like. The screens come back to your saved layout when the desktop starts. You can move, resize, curve, and roll them, pin one to your wrist, and put them all back with a shortcut.
|
||||
- **Screens that are real monitors.** Each KDE Plasma screen has its own resolution and shape: an ultrawide in front, portrait screens beside it. Move, resize, curve, and roll them, or pin one to your wrist or your head. They come back to your layout when the desktop starts.
|
||||
- **Windows that float on their own.** Take any app window off the screens into a panel of its own with Meta+Shift+F, its title bar button, or Launch as Standalone. It stays part of the desktop, so drag and drop and the clipboard still work between floating windows and the screens.
|
||||
- **Profiles.** Save where your screens are, which ones show, and which apps are open where. Switch to a profile from Display Settings, a key, or its launcher entry, or start the desktop in one.
|
||||
- **One mouse for all of SteamVR.** A Bluetooth mouse drives a small dot anchored in the room. It works the screens, the dashboard, Steam, and overlays, and hands the laser back when you pick up a controller.
|
||||
- **Look and click (experimental).** In gaze mode the pointer goes where you look. Meta+J and Meta+K, or the mouse buttons, click and fine-tune, and each correction teaches the tracker. Calibration runs in the headset.
|
||||
- **Made for long sessions.** The displays turn off when the headset isn't used, even on a stand that makes it seem worn. It stays awake on the charger, and you can reach the desktop remotely over VNC.
|
||||
|
||||
The mouse shows up as a small dot anchored in the room. It works on the SteamVR dashboard, Steam, overlays, and the desktop, and it hands the laser back to your controllers when you pick one up.
|
||||
|
||||
It comes with two settings apps, Frametop Display Settings for the screens and Frametop Input Settings for mice, keyboards, and button mappings, plus fixes that let Bluetooth LE mice and keyboards like the Swiftpoint Z3 reconnect after they sleep.
|
||||
|
||||
When you're not wearing the headset, Frametop can turn its displays off and keep it awake on the charger, so you can still reach it remotely. This works even on a stand or mount that makes the headset seem worn.
|
||||
Two settings apps come with it: Frametop Display Settings for the screens, profiles, and power, and Frametop Input Settings for mice, keyboards, gaze, and button mappings. Optional fixes let Bluetooth LE mice and keyboards like the Swiftpoint Z3 reconnect after they sleep. Install it with one command: see [Install on the headset](#install-on-the-headset).
|
||||
|
||||
Frametop is an independent project, not made by or affiliated with Valve.
|
||||
|
||||
@@ -26,7 +27,7 @@ You need a Steam Frame with an internet connection, a keyboard (Bluetooth, or th
|
||||
|
||||
It asks which version you want: stable (the `main` branch, tested releases) or experimental (the `experimental` branch, the newest features, less tested). Then it clones the repo into `~/frametop` and runs `install.sh`. To choose without the question, add `-s -- --stable` or `-s -- --experimental` after `bash`. By hand, the same is `git clone https://github.com/DeeJanuz/frametop.git ~/frametop`, then `cd ~/frametop` and `./install.sh` (add `--branch experimental` to the clone for experimental).
|
||||
|
||||
The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks you four things along the way: whether to install gaze mode (experimental, yes by default), the Bluetooth fixes, and hand tracking (experimental), then whether to restart SteamVR. The Bluetooth fixes and hand tracking need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
|
||||
The installer sets up distrobox in your home folder (the system files aren't touched), a Fedora build container, and everything else. The first run downloads 1–2 GB. It asks you three things along the way: whether to install gaze mode (experimental, yes by default) and the Bluetooth fixes, then whether to restart SteamVR. The Bluetooth fixes need your `sudo` password; if you've never set one, run `passwd` first, or skip them for now. SteamVR has to restart once at the end, which closes everything open in VR, including the terminal. Rebooting the headset works too.
|
||||
|
||||
After the restart, Launch a program → Desktop opens the multi-screen desktop, with its screens arranged around where you're facing. Frametop Display Settings and Frametop Input Settings are in the desktop's application menu, under Settings.
|
||||
|
||||
@@ -40,11 +41,10 @@ If you work in the desktop for long stretches, or leave the headset on a stand,
|
||||
|
||||
## Use
|
||||
|
||||
### Screens
|
||||
|
||||
| Do this | To get this |
|
||||
| --- | --- |
|
||||
| Move the mouse | The dot moves around you and snaps onto whatever panel it's over |
|
||||
| Click, right-click, scroll | Acts on the panel under the dot |
|
||||
| Pick up a controller | The controller gets its laser back; move the mouse to take over again |
|
||||
| Point near the bottom of a screen | Its controls fade in: the bar, the curve and roll buttons, and the resize tab on the corner |
|
||||
| Drag the bar under a screen | Moves the screen; scroll while dragging to push it away or pull it closer. With the mouse, hold right while dragging to tilt it |
|
||||
| Drag the tab on a screen's bottom right corner | Resizes the screen |
|
||||
@@ -52,25 +52,57 @@ If you work in the desktop for long stretches, or leave the headset on a stand,
|
||||
| Drag the roll button sideways, or scroll on it | Rolls the screen; it snaps level near straight |
|
||||
| While carrying a screen, sweep its laser across your other controller's ring, then let go | Pins it to that wrist, at its size and distance, as you hold it when you let go; it shows while you see its front. Grab its bar to adjust it (it stays pinned); sweep across the ring again to take it off |
|
||||
| Set a screen to On your head (Frametop Display Settings, Visibility & pins) | Pins it to your head where it is, like a HUD. Grab its bar to move it; it stays on your head |
|
||||
| Save as profile… (Frametop Display Settings, Layout & profiles) | Saves where the screens are, with their sizes and pins, which ones are hidden, and the open apps and where their windows are (on a screen or floating), under a name. Pick it under Arrangement and press Open profile to switch to it: the screens move, open windows of its apps go to their places, and the apps that aren't open start. Nothing closes |
|
||||
| Pick a profile under Start in profile, or run its entry (Frametop: NAME) from SteamVR's Launch a program list | The desktop starts in that profile, or switches to it if it's running. A profile can also go on a key combination, mouse button, or controller button in Frametop Input Settings |
|
||||
| Meta+Shift+R in the desktop | Puts the screens back in their layout (also in the menu as Reset Screen Layout, and mappable to a mouse button) |
|
||||
| Meta+Shift+F over a desktop window | Floats that window in VR as a panel of its own, or puts it back on its screen if it floats. It acts on the window under the pointer, or the active one if the pointer is over the wallpaper. Rebind it, or map it to a mouse or controller button, in Frametop Input Settings (Keyboard page, or Buttons and Controllers as Float window in VR). Float in VR is also in every window's menu (Alt+F3), and the button left of Close in a window's title bar does the same (apps that draw their own title bar, like Chromium and Electron apps, don't have it) |
|
||||
| Right-click an app in the Application Launcher (or the taskbar) and pick Launch as Standalone | Starts the app with its window floating in VR, where that app last floated, or in front of you the first time. From a terminal: `float/ft-float launch org.kde.dolphin`, or `float/ft-float run <command>` |
|
||||
| Meta+Shift+H in the desktop | Hides or shows all screens (also in the menu as Hide/Show Screens, and mappable). The Visibility & pins tab of Frametop Display Settings can instead show them only with the dashboard open, or while you look at your wrist |
|
||||
| Switch a screen to Hidden (Frametop Display Settings, Visibility & pins → Screens shown) | Hides just that screen until you switch it back, whatever the other visibility settings say; new windows that would open on it float instead |
|
||||
| Leave the headset on a stand | Its displays turn off once it has gone unused for the time set in Frametop Display Settings → Power, even if the stand covers its proximity sensor. Pick it up, or use any mouse, keyboard, or button, and they come back on |
|
||||
| Play a VR game | The screens hide and your controllers stay in the game. Open the SteamVR dashboard, or press Meta+Shift+H, to see and use them. To keep them visible over games, change During VR games on the Visibility & pins tab; the controllers still stay in the game, and you use the screens with the mouse or the dashboard |
|
||||
|
||||
You can map the mouse's extra buttons to actions such as Toggle SteamVR dashboard, Recenter pointer, or Head follow on/off on the Buttons page of Frametop Input Settings, and the Frame controllers' buttons on its Controllers page. Pointer speed, dot size, and the rest are on its Pointer page and take effect immediately. If a panel you only look at, such as a performance overlay that follows your view, keeps catching the dot, tick it (or its whole app) on the Ignored panels page, and the pointer passes through it. Head follow, which is experimental and off by default, makes the pointer come along when you turn your head: it stays put until your head turns past the leash angle, then glides back to its place in your view, and a leash of 0 keeps it fixed in your view. It's only lightly tested and not polished; tuning its settings, or improving how it feels, is open to anyone who wants to take it further.
|
||||
|
||||
Restarting the desktop (Restart desktop in Frametop Display Settings) closes its windows, but background work you started in it, such as servers, tmux sessions, or builds, keeps running.
|
||||
|
||||
### Experimental: gaze and hand tracking
|
||||
### Mouse and controllers
|
||||
|
||||
Gaze mode makes the pointer go where you look. The installer offers it (or run `gaze/run.sh install` later); turn it on and calibrate on the Gaze page of Frametop Input Settings. Meta+J left-clicks and Meta+K right-clicks where you look; hold the key and turn your head to correct the aim, then let go. The mouse's buttons work the same way, with the mouse doing the correcting, and the corrections teach the gaze tracker.
|
||||
| Do this | To get this |
|
||||
| --- | --- |
|
||||
| Move the mouse | The dot moves around you and snaps onto whatever panel it's over |
|
||||
| Click, right-click, scroll | Acts on the panel under the dot |
|
||||
| Pick up a controller | The controller gets its laser back; move the mouse to take over again |
|
||||
|
||||
Hand tracking, which the installer offers, shows your hands through the screens. Turn it on with `ft-handsctl on` and off with `ft-handsctl off`. With `POINTER_HANDS=1` in `~/.config/frametop.conf`, a pinch clicks and a grip drags. [docs/reference.md](docs/reference.md) has the details of both.
|
||||
You can map the mouse's extra buttons to actions such as Toggle SteamVR dashboard, Recenter pointer, or Head follow on/off on the Buttons page of Frametop Input Settings, and the Frame controllers' buttons on its Controllers page. Pointer speed, dot size, and the rest are on its Pointer page and take effect immediately. If a panel you only look at, such as a performance overlay that follows your view, keeps catching the dot, tick it (or its whole app) on the Ignored panels page, and the pointer passes through it. Head follow, which is experimental and off by default, makes the pointer come along when you turn your head: it stays put until your head turns past the leash angle, then glides back to its place in your view, and a leash of 0 keeps it fixed in your view. It's only lightly tested and not polished; tuning its settings, or improving how it feels, is open to anyone who wants to take it further.
|
||||
|
||||
### Floating windows
|
||||
|
||||
Any desktop window can float in VR as a panel of its own. It stays a window of the same desktop, so drag and drop and the clipboard work between floating windows and the screens, and it's still in the taskbar and Alt+Tab.
|
||||
|
||||
| Do this | To get this |
|
||||
| --- | --- |
|
||||
| Meta+Shift+F over a desktop window | Floats that window, or puts it back on its screen if it floats. It acts on the window under the pointer, or the active one if the pointer is over the wallpaper. Rebind it, or map it to a mouse or controller button, in Frametop Input Settings (Keyboard page, or Buttons and Controllers as Float window in VR) |
|
||||
| Click the float button, left of Close in a window's title bar | The same. Float in VR is also in every window's menu (Alt+F3). Apps that draw their own title bar, like Chromium and Electron apps, don't have the button: use Meta+Shift+F |
|
||||
| Right-click an app in the Application Launcher (or the taskbar) and pick Launch as Standalone | Starts the app with its window floating, where that app last floated, or in front of you the first time. From a terminal: `float/ft-float launch org.kde.dolphin`, or `float/ft-float run <command>` |
|
||||
| Meta+scroll over a floating window | Scales it up or down |
|
||||
|
||||
### Profiles
|
||||
|
||||
A profile is a named setup: where the screens are, with their sizes and pins, which ones are hidden, and which apps are open and where their windows are, on a screen or floating.
|
||||
|
||||
| Do this | To get this |
|
||||
| --- | --- |
|
||||
| Save as profile… (Frametop Display Settings, Layout & profiles) | Saves the current setup under a name, or updates the profile you're in |
|
||||
| Pick a profile under Arrangement and press Open profile | Switches to it: the screens move, open windows of its apps go to their places, and the apps that aren't open start. Nothing closes |
|
||||
| Pick a profile under Start in profile, or run its entry (Frametop: NAME) from SteamVR's Launch a program list | The desktop starts in that profile, or switches to it if it's running. A profile can also go on a key combination, mouse button, or controller button in Frametop Input Settings |
|
||||
|
||||
### Gaze mode (experimental)
|
||||
|
||||
In gaze mode the pointer goes where you look, and the mouse or the keyboard does the last bit. The installer offers it (or run `gaze/run.sh install` later). Turn it on and calibrate it on the Gaze page of Frametop Input Settings.
|
||||
|
||||
| Do this | To get this |
|
||||
| --- | --- |
|
||||
| Tap Meta+J, or Meta+K | A left or right click where you look |
|
||||
| Hold Meta+J, turn your head onto what you meant, and let go | A click there. Held still for half a second, it becomes a press, and turning your head drags |
|
||||
| Hold the left mouse button, move the mouse onto what you meant, and let go | A click there. The right button does the same for a right click. Held still, the left button drags |
|
||||
| Double right click (or double Meta+K) while dragging a screen's bar | Pans and tilts the screen |
|
||||
| Put the headset on | A quick check: look at the dot for a moment, and the pointer lines up again |
|
||||
|
||||
By default the mouse only corrects: while the gaze has the pointer, moving the mouse does nothing until you hold a button. Each correction before a click teaches the tracker where it was off. A correction past the learning limit (55 degrees by default, about half of what you can see) starts a quick check instead. Calibrate, on the Gaze page, runs a full calibration in a panel in front of you: look at each dot and click. Check headset fit shows how well the eye tracker sees your eyes. [gaze/README.md](gaze/README.md) has the details.
|
||||
|
||||
### Leave the headset on a stand and reach it remotely
|
||||
|
||||
@@ -91,6 +123,9 @@ This is an early release, tested on one Steam Frame (SteamOS 0.3.0 build 2026092
|
||||
- Flatscreen games aren't detected as games. If your controllers end up working the screens instead of the game, set Controllers on the screens to "Only with the SteamVR dashboard open" (Frametop Display Settings, Visibility & pins tab).
|
||||
- Typing follows your last click. A controller click on a panel other than the screens (the dashboard, a Steam app) doesn't move typing there; click it with the mouse, or click a screen to bring typing back.
|
||||
- The screens don't draw a mouse cursor of their own. The 3D mouse's dot or SteamVR's laser shows where you're pointing.
|
||||
- Profiles reopen apps, not what the apps had open. Tabs, files, and folders are left to each app's own restore.
|
||||
- Dragging something from one panel to another (a screen and a floating window) works, but the dragged item's icon doesn't show while the pointer is between panels.
|
||||
- Gaze mode is only as good as its calibration, and that depends on how the headset sits on your face. If the pointer lands off after you adjust the headset, run Quick check or Calibrate on the Gaze page of Frametop Input Settings.
|
||||
- On SteamVR's Settings page, the 3D mouse shows a laser beam and a larger hit dot, like a controller. SteamVR doesn't tell other programs where that page is (unlike Steam's pages, such as Library), so the mouse used to miss most of it: clicks went through to a desktop screen behind, and the dot disappeared. As a workaround, on that page only, the laser starts near your eye and SteamVR finds the page itself. See docs/design.md.
|
||||
- Remote desktop over VNC (Frametop Remote Access in the app menu, or `./desktops.sh remote on`) needs Tailscale on the Frame. It shows the primary screen only. The app turns it on and off, shows the address, and shows, copies, or changes the VNC password. The password is made at random on the Frame and kept in `~/.config/frametop-remote` (only you can read it); VNC limits it to 8 characters, and the tailnet encrypts the connection. Turning it on in a desktop that started with it off takes a desktop restart.
|
||||
- Turning the displays off on a stand only turns their backlight off. SteamVR has no way for other programs to put the headset in standby, so tracking and rendering keep running, and the headset draws nearly its full power.
|
||||
@@ -127,7 +162,7 @@ input-settings/install.sh uninstall
|
||||
display-settings/install.sh uninstall
|
||||
remote/install.sh uninstall
|
||||
setup/bluetooth/install.sh uninstall # if you installed the Bluetooth fixes
|
||||
hands/run.sh uninstall # if you installed hand tracking
|
||||
hands/run.sh uninstall # if you installed hand tracking by hand
|
||||
gaze/run.sh uninstall # if you installed the gaze service
|
||||
gaze/tracker/install.sh uninstall # if you installed our own eye tracker's frame grabber
|
||||
gaze/probe/install.sh uninstall # if you installed the gaze probe
|
||||
@@ -151,7 +186,7 @@ A Plasma session runs nested inside ft-screens (`screens/`), a small Wayland com
|
||||
| `pointer/` | The 3D mouse: SteamVR driver, helper service, and a probe tool. |
|
||||
| `power/` | ft-powerd: turns the displays off while the headset isn't used. |
|
||||
| `gaze/` | Gaze mode (experimental): the gaze service, its calibration panel, our own eye tracker, and the gaze probe. See [gaze/README.md](gaze/README.md). |
|
||||
| `hands/` | Hand tracking (experimental): the camera broker and the tracker, for the hands over the screens and pinch clicks. See [hands/README.md](hands/README.md). |
|
||||
| `hands/` | Hand tracking (experimental, deferred: the installer doesn't offer it). See [hands/README.md](hands/README.md). |
|
||||
| `display-settings/`, `input-settings/` | The two settings apps (Kirigami, Python). |
|
||||
| `remote/` | Frametop Remote Access, the app that turns remote desktop over VNC on and off. |
|
||||
| `setup/` | The build container and the Bluetooth fixes. See [setup/README.md](setup/README.md). |
|
||||
@@ -171,7 +206,7 @@ The scripts also work from a Linux or WSL PC over SSH, which is easier for editi
|
||||
IdentityFile ~/.ssh/<your-key>
|
||||
```
|
||||
|
||||
3. The Bluetooth fixes and hand tracking need `sudo` on the Frame. The installer asks for the password in your terminal (over `ssh -t`). To skip the question, or to install with no terminal, put it in `.env` at the repo root instead. It's gitignored and never synced:
|
||||
3. The Bluetooth fixes need `sudo` on the Frame. The installer asks for the password in your terminal (over `ssh -t`). To skip the question, or to install with no terminal, put it in `.env` at the repo root instead. It's gitignored and never synced:
|
||||
|
||||
```
|
||||
steamos_root_pwd="<password>"
|
||||
|
||||
+4
-2
@@ -208,9 +208,11 @@ In gaze mode the 3D mouse's pointer goes where you look, and the mouse or the ke
|
||||
|
||||
[gaze/README.md](../gaze/README.md) has the details, our own eye tracker, and the gaze probe, a development tool.
|
||||
|
||||
## Hand tracking (experimental)
|
||||
## Hand tracking (experimental, deferred)
|
||||
|
||||
Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen. The same tracker detects pinches and grips, and with `POINTER_HANDS=1` in `~/.config/frametop.conf` they work the pointer. In gaze mode a pinch clicks where you look when it opens; hold it and move the hand to correct the pointer first. Without gaze mode a pinch is a press like the mouse's button, so a held pinch drags. A grip (closing the hand) presses and drags. It's optional: `hands/run.sh install`, or the last question of `install.sh`.
|
||||
Deferred: it costs a lot of the headset's CPU and needs more work, so `install.sh` doesn't offer it. It still builds and runs, installed by hand, for working on it.
|
||||
|
||||
Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen. The same tracker detects pinches and grips, and with `POINTER_HANDS=1` in `~/.config/frametop.conf` they work the pointer. In gaze mode a pinch clicks where you look when it opens; hold it and move the hand to correct the pointer first. Without gaze mode a pinch is a press like the mouse's button, so a held pinch drags. A grip (closing the hand) presses and drags. To install it: `hands/run.sh install`.
|
||||
|
||||
- `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`.
|
||||
- `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches and grips, read by the pointer helper) next to the ring.
|
||||
|
||||
+6
-4
@@ -1,6 +1,8 @@
|
||||
# Hands (experimental)
|
||||
# Hands (experimental, deferred)
|
||||
|
||||
Hand tracking from the headset's own cameras. It serves two things in Frametop:
|
||||
Hand tracking from the headset's own cameras. It's deferred: it costs a lot of the headset's CPU and needs more work, so `install.sh` doesn't offer it and the README doesn't list it. It still builds and runs, installed by hand (below), for working on it.
|
||||
|
||||
It serves two things in Frametop:
|
||||
|
||||
- **Hand cutouts:** where your hand is between an eye and a screen, that eye sees the room through the screen (ft-screens, `screens/handcut.cpp`), so your hands show over the screens the way they do on a Vision Pro.
|
||||
- **Pinches and grips:** with `POINTER_HANDS=1`, the pointer helper takes them as clicks and drags. Look at something and pinch to click it, with the eye tracker doing the looking (`gaze/`), or close your hand to press and drag what the pointer is on. See "Pinches and grips in the pointer" below.
|
||||
@@ -10,7 +12,7 @@ Two programs, each a user service that stops when SteamVR does:
|
||||
- `ft-camd` (`camd/`, C) borrows XRService's camera buffers and publishes the four IR tracking cameras' frames to a shared-memory ring. It runs on the host.
|
||||
- `ft-hands` (`track/`, C++) finds hands in those frames with MediaPipe's palm and landmark models on ncnn, triangulates them, and publishes them. It runs in the dev container.
|
||||
|
||||
They don't start with SteamVR. `hands/run.sh install` builds them, gives ft-camd its capabilities, installs both services disabled, and links `hands/ft-handsctl` into `~/.local/bin`. Then `ft-handsctl on` starts hand tracking and `ft-handsctl off` stops it. `install.sh` offers the install as its last, optional step.
|
||||
They don't start with SteamVR. `hands/run.sh install` builds them, gives ft-camd its capabilities, installs both services disabled, and links `hands/ft-handsctl` into `~/.local/bin`. Then `ft-handsctl on` starts hand tracking and `ft-handsctl off` stops it. `install.sh` doesn't install it.
|
||||
|
||||
```
|
||||
ft-handsctl on | off # on the Frame: start or stop hand tracking (SteamVR must be running)
|
||||
@@ -188,7 +190,7 @@ hands/build/ft-handreplay ~/.local/share/frametop/hands/rec-20260929-120000 --co
|
||||
|
||||
## Tools
|
||||
|
||||
Python, with NumPy and OpenCV (in the dev container: `python3-numpy`, `python3-opencv`, which `setup/dev-container.sh` installs). Off the Frame, `FRAME_JOB_DEVICE_ROOT` can point at a folder with copies of the headset's calibration files.
|
||||
Python, with NumPy and OpenCV. `setup/dev-container.sh` doesn't install them, because Fedora's `python3-opencv` pulls in over a gigabyte; in the dev container, run `sudo dnf install python3-numpy python3-opencv` once. Off the Frame, `FRAME_JOB_DEVICE_ROOT` can point at a folder with copies of the headset's calibration files.
|
||||
|
||||
- `tools/check_sides.py --ring` (or a recording): are the side cameras named right?
|
||||
- `tools/check_color.py REC`: how the colour module's calibration maps onto its images.
|
||||
|
||||
+14
-21
@@ -1,13 +1,13 @@
|
||||
#!/usr/bin/env bash
|
||||
# Install everything on the Steam Frame: the build container, Frametop (multi-screen
|
||||
# desktop, input relay, universal 3D mouse, settings app), and optionally gaze mode, the
|
||||
# Bluetooth fixes, and hand tracking. Run it on the headset in a terminal, from this repo.
|
||||
# It's safe to re-run, for example after `git pull`.
|
||||
# desktop, input relay, universal 3D mouse, settings app), and optionally gaze mode and the
|
||||
# Bluetooth fixes. Run it on the headset in a terminal, from this repo. It's safe to re-run,
|
||||
# for example after `git pull`. (Hand tracking, hands/, is deferred: it isn't offered here.)
|
||||
# (It also works from a PC over SSH; see "Developing from a PC" in the README.)
|
||||
#
|
||||
# Usage: ./install.sh [--yes] [--no-bluetooth]
|
||||
# --yes don't ask; installs gaze mode, skips the Bluetooth fixes, hand tracking,
|
||||
# and the SteamVR restart
|
||||
# --yes don't ask; installs gaze mode, skips the Bluetooth fixes and the SteamVR
|
||||
# restart
|
||||
# --no-bluetooth don't offer the Bluetooth fixes
|
||||
set -euo pipefail
|
||||
|
||||
@@ -50,7 +50,7 @@ if [ "$FRAME_LOCAL" = 0 ] || [ -n "${SSH_CONNECTION:-}" ]; then
|
||||
echo "that need SteamVR start with it if it isn't running now."
|
||||
fi
|
||||
|
||||
step "1/10 distrobox (container tool, installed in your home folder)"
|
||||
step "1/9 distrobox (container tool, installed in your home folder)"
|
||||
if on_frame 'test -x ~/.local/bin/distrobox'; then
|
||||
echo "already installed: $(on_frame '~/.local/bin/distrobox version | head -1')"
|
||||
else
|
||||
@@ -60,25 +60,25 @@ else
|
||||
cd ~/dev/src/distrobox && ./install --prefix ~/.local'
|
||||
fi
|
||||
|
||||
step "2/10 build container (Fedora 44 'dev', about 1-2 GB the first time)"
|
||||
step "2/9 build container (Fedora 44 'dev', about 1-2 GB the first time)"
|
||||
"$root/setup/dev-container.sh"
|
||||
|
||||
step "3/10 input relay (keeps Bluetooth mice working in SteamVR, device roles, button maps)"
|
||||
step "3/9 input relay (keeps Bluetooth mice working in SteamVR, device roles, button maps)"
|
||||
"$root/desktops.sh" relay install
|
||||
|
||||
step "4/10 3D mouse: SteamVR driver"
|
||||
step "4/9 3D mouse: SteamVR driver"
|
||||
"$root/pointer/driver/build.sh"
|
||||
"$root/pointer/driver/install.sh" install 2>&1 | grep -v xdg-open
|
||||
|
||||
step "5/10 3D mouse: pointer helper service"
|
||||
step "5/9 3D mouse: pointer helper service"
|
||||
"$root/pointer/helper/build.sh"
|
||||
"$root/pointer/helper/run.sh" install
|
||||
|
||||
step "6/10 power service (turns the displays off while the headset isn't used, even on a stand)"
|
||||
step "6/9 power service (turns the displays off while the headset isn't used, even on a stand)"
|
||||
"$root/power/build.sh"
|
||||
"$root/power/run.sh" install
|
||||
|
||||
step "7/10 multi-screen desktop (ft-screens), Frametop Input Settings, and Frametop Display Settings"
|
||||
step "7/9 multi-screen desktop (ft-screens), Frametop Input Settings, and Frametop Display Settings"
|
||||
"$root/screens/build.sh"
|
||||
"$root/desktops.sh" install >/dev/null
|
||||
"$root/input-settings/install.sh"
|
||||
@@ -87,27 +87,20 @@ step "7/10 multi-screen desktop (ft-screens), Frametop Input Settings, and Frame
|
||||
on_frame "sed -i 's/^POINTER=0/POINTER=1/' ~/.config/frametop.conf; grep -q '^POINTER=' ~/.config/frametop.conf || echo 'POINTER=1' >> ~/.config/frametop.conf"
|
||||
echo "the launcher's Desktop entry now opens the multi-screen desktop; 3D mouse on (POINTER=1 in ~/.config/frametop.conf)"
|
||||
|
||||
step "8/10 gaze mode (optional, experimental: the pointer goes where you look)"
|
||||
step "8/9 gaze mode (optional, experimental: the pointer goes where you look)"
|
||||
if ask "Install gaze mode? You turn it on and calibrate it in Frametop Input Settings, on the Gaze page." y; then
|
||||
"$root/gaze/run.sh" install
|
||||
else
|
||||
echo "skipped. Install later with: gaze/run.sh install"
|
||||
fi
|
||||
|
||||
step "9/10 Bluetooth fixes (optional; they let LE mice and keyboards like the Swiftpoint Z3 reconnect)"
|
||||
step "9/9 Bluetooth fixes (optional; they let LE mice and keyboards like the Swiftpoint Z3 reconnect)"
|
||||
if [ "$bluetooth" = 1 ] && ask "Install the Bluetooth fixes? They need your password (sudo)." n; then
|
||||
"$root/setup/bluetooth/install.sh" install
|
||||
else
|
||||
echo "skipped. Install later with: setup/bluetooth/install.sh install"
|
||||
fi
|
||||
|
||||
step "10/10 hand tracking (optional, experimental: your hands show over the screens)"
|
||||
if ask "Install hand tracking? It needs your password (sudo) to let its camera service read the headset cameras." n; then
|
||||
"$root/hands/run.sh" install
|
||||
else
|
||||
echo "skipped. Install later with: hands/run.sh install"
|
||||
fi
|
||||
|
||||
step "Done"
|
||||
cat <<'EOF'
|
||||
SteamVR has to restart once, to load the 3D mouse driver and to start the input relay
|
||||
|
||||
@@ -18,9 +18,11 @@ packages=(
|
||||
mesa-libgbm-devel wayland-devel vulkan-loader-devel vulkan-headers plasma-wayland-protocols wlroots-devel
|
||||
# ft_pointer SteamVR driver: static C++ runtime (the host has an older glibc)
|
||||
libstdc++-static
|
||||
# hand tracking: ft-hands reads the calibration with jsoncpp; ft-camd runs on the host, linked
|
||||
# statically; the Python tools (hands/tools) need NumPy and OpenCV
|
||||
jsoncpp-devel glibc-static python3-numpy python3-opencv
|
||||
# hand tracking (deferred: not installed by install.sh; hands/run.sh install builds it): ft-hands
|
||||
# reads the calibration with jsoncpp; ft-camd runs on the host, linked statically. Its Python
|
||||
# tools (hands/tools) need NumPy and OpenCV, which aren't here: Fedora's python3-opencv pulls
|
||||
# in over a gigabyte (hands/README.md says how to get them)
|
||||
jsoncpp-devel glibc-static
|
||||
# Frametop Input Settings app (Kirigami, PySide6)
|
||||
python3-pyside6 kf6-kirigami kf6-qqc2-desktop-style qt6-qtwayland breeze-icon-theme plasma-breeze
|
||||
# Frametop remote desktop (VNC bridge through krdp)
|
||||
|
||||
Reference in new issue
Block a user