steam-frame/DeeJanuz--frametop
1
0 Fork 0
mirror of https://github.com/DeeJanuz/frametop.git synced 2026-10-06 06:00:09 +02:00
Code Issues Packages Projects Releases 4 Wiki Activity

Defer hand tracking; the README leads with what Frametop does now

Browse Source 
- install.sh no longer offers hand tracking (it's heavy on the CPU and
  needs more work); hands/ still builds and installs by hand
- the build container drops python3-opencv and python3-numpy, which
  only hand tracking's tools used: Fedora's OpenCV pulls in over a GB
- README: a new opening and feature list, the Use section by topic
  (screens, mouse, floating windows, profiles, gaze), and new known
  limitations

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DeeJanuzandClaude Opus 5.5 committed 2026-10-01 15:20:39 -06:00
1 parent c257000a23
commit 7bcda94267
5 files changed
+88 -54
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/reference.md
+4 -2
View File
@@ -208,9 +208,11 @@ In gaze mode the 3D mouse's pointer goes where you look, and the mouse or the ke
[gaze/README.md](../gaze/README.md) has the details, our own eye tracker, and the gaze probe, a development tool.
## Hand tracking (experimental)
## Hand tracking (experimental, deferred)
Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen. The same tracker detects pinches and grips, and with `POINTER_HANDS=1` in `~/.config/frametop.conf` they work the pointer. In gaze mode a pinch clicks where you look when it opens; hold it and move the hand to correct the pointer first. Without gaze mode a pinch is a press like the mouse's button, so a held pinch drags. A grip (closing the hand) presses and drags. It's optional: `hands/run.sh install`, or the last question of `install.sh`.
Deferred: it costs a lot of the headset's CPU and needs more work, so `install.sh` doesn't offer it. It still builds and runs, installed by hand, for working on it.
Your hands show over the screens: where a tracked hand is between an eye and a screen, ft-screens lets that eye see the room through the screen. The same tracker detects pinches and grips, and with `POINTER_HANDS=1` in `~/.config/frametop.conf` they work the pointer. In gaze mode a pinch clicks where you look when it opens; hold it and move the hand to correct the pointer first. Without gaze mode a pinch is a press like the mouse's button, so a held pinch drags. A grip (closing the hand) presses and drags. To install it: `hands/run.sh install`.
- `ft-camd` borrows XRService's camera buffers and publishes the four IR tracking cameras to `/run/user/UID/frametop-hands/cam-ring`. It runs on the host as `frametop-camd.service`, with file capabilities that `hands/run.sh install` sets through sudo, and it drops them once set up. A rebuild clears them: `hands/run.sh caps`.
- `ft-hands` runs in the `dev` container as `frametop-hands.service`. It finds and triangulates the hands, and publishes `hands` (read by ft-screens' cutouts) and `gestures` (pinches and grips, read by the pointer helper) next to the ring.