mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-06 01:00:06 +02:00
main
33
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
9d7c8a0b91 |
Experimental (#29)
* Input relay: typing with the pointer helper down no longer ends the relay Typing on a pass-through keyboard tells the helper "typing". With the helper not running (SteamVR off), that send raised ConnectionRefusedError and the relay exited, dropping every grab until systemd restarted it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ft-steam: open Steam's menu through Steam's own UI steam/ft-steam menu opens the SteamVR dashboard on Steam's menu, or closes the dashboard if it's up, without pointer mode: it asks Steam's UI over its debugging port to show its dashboard overlay (ShowVROverlay, what Steam calls itself) and focus the Steam frame's left menu (MenuStore.OpenMainMenu). ft-steam check says whether those calls still exist, and update-check.py runs it, since a Steam client update can rename them. The CDP client moves from display-settings/steam_settings.py to steam/steamui.py, so both use it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Shortcuts: Steam menu, commands, and modifier taps Key combinations (and mouse and controller buttons) get two new actions: - steam_menu: Open Steam menu / close dashboard (steam/ft-steam menu). - command:CMD: run CMD with sh -c, as the relay's service, with layout/, float/ and steam/ on its PATH. Input Settings offers it for key combinations as Run a command... Both work without pointer mode. A modifier on its own is now a key combination too: a tap, pressed and released with no other key, mouse button, or scroll in between. A bound tap sends the desktop F24 before the release, so Plasma's launcher stays shut. The defaults gain a Meta tap for the Steam menu; this replaces META_DASHBOARD, which only worked in pointer mode. input/test/keys-test.py runs the relay against fake devices with every outgoing socket renamed, so it's safe next to the live relay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Relay: share a key combination's Meta release with frame-voice A Meta+key combination (Meta+J for gaze_left, say) hides Meta's release from the desktop, and the relay skipped share_key for it too. frame-voice saw Meta go down on @frametop_keys and never come up, so it held all dictated text back, waiting for that release. Keys of grabbed keyboards are now shared as pressed, before key_binding() decides what the desktop gets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: ft-cutouts, the hand cutouts without pinches and grips hands/ft-cutouts on|off|status starts ft-camd and ft-hands as transient user units with ft-hands' new --no-gestures: hands are published for ft-screens' cutouts, but no pinch or grip is detected, so nothing clicks or drags and a closing hand doesn't raise the tracking rate. It needs a build and ft-camd's capabilities, not hands/run.sh install. Its units conflict with ft-handsctl's, so each stops the other, and they stop with SteamVR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ft-cutouts status: only the current run's tracker lines Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Gaze: say why gaze mode can't work yet, and open the calibration whenever it's missing Turning gaze mode on without a calibration opened Calibrate only on the off-to-on change, and only if it could open right then. With the headset off, the eye tracker silent, or the panel not built, or with gaze mode already on when the gaze service started, nothing opened and nothing said why: the pointer just stayed a mouse. - The gaze service now checks every second: gaze mode on, no calibration for the tracker in use, eyes seen -> the full calibration opens. One that closes unfinished opens again only after the headset comes off and on, gaze mode off and on, or Calibrate, so it doesn't loop. A start that fails retries every 10 s. - Its status says why gaze mode can't work yet (checks.problem): not calibrated and opening, open, closed unfinished, or can't open and why. - Input Settings shows that under the Gaze pointer switch, along with the gaze service not installed or not running and our tracker missing its frame grabber. - ft-gazectl on notes a missing calibration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Gaze: install our eye tracker, prefer it, and say why a calibration dot wasn't taken A user on a fresh install got "Calibration failed: only 0 of 21 dots" with no reason. The installer never installed our tracker, so gaze used SteamVR's, and the only way SteamVR's tracker rejects a dot is losing an eye for most of the look. The panel just showed a red ring. - install.sh: step 9/10 installs our tracker (gaze/tracker/install.sh) after gaze mode, yes by default; it needs sudo, so --yes runs it only when sudo won't prompt. If it fails, gaze keeps SteamVR's tracker. Configs that still say GAZE_TRACKER=steam (the old template) are asked whether to switch. - GAZE_TRACKER=auto, the new default: ours when it's installed (the frame grabber, its unit, and ft-eyes' Python), else SteamVR's. ft-gazed rechecks every second, so installing it switches over. Input Settings lists Own tracker first as recommended, and says how to install it when it's missing (checking the host's /etc through /run/host from the dev container). - The calibration panel has a note line, orange over the instructions: why a dot wasn't taken (an eye lost, a blink, the eyes disagreeing for SteamVR's tracker, from steady_samples' new drop counts; ft-eyes' reply for ours), what a click is still waiting for after 1.5 s, and a failed calibration's most common reason, which the Gaze page shows too. steady_samples keeps the same samples as before (checked on 2037 windows of recordings); a lost eye is named before a blink, since its openness reads 0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * README: link the Frametop Discord Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Wait for a new container to finish setting up before entering it container-up.sh starts the dev container in a scope of its own, so distrobox enter finds it running and skips its wait for distrobox-init. On a fresh install, init was still setting up passwordless sudo when dev-container.sh ran sudo dnf install, and sudo asked for a password with no terminal to read it from. container-up.sh now waits for container_setup_done itself, and the container's sudo calls use -n, so a password prompt fails at once with a clear message. Fixes #9 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Prevent small desktop overlay pointer movements from starting a drag * Clear reported drag state on controller release * Click stability: only a hand controller's press starts it The 3D mouse drives SteamVR's laser through the ft_pointer virtual controller, so its events reach the screens the same way a controller's do. The filter held every press, which turned the mouse's short drags (selecting a character or two, nudging a slider) into clicks. Mark button events from hand controllers and start the filter only on those. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Input relay: retry a new device until udev gives it to the input group A new /dev/input node is root:root 0600 until udev applies GROUP=input. The scan probed each new node once and marked it seen even when the open failed, so a node caught in that gap was never opened. Behind a KVM, a switch brings back a hub of devices at once: on the Frame, four nodes failed with EACCES in one switch, the keyboard was never grabbed, and its keys went to gamescope instead of the desktop screens. A node that isn't readable yet now waits for the next scan. * Screens: take a screen's overlays from one copy in the catcher While a button pressed on a screen is held, UpdateCatcher checks every tick whether the laser is still on one of the screen's overlays. It built that list from s.All().begin() and s.All().end(), but All() returns a std::array by value: iterators into two different temporaries, which is undefined behaviour. A clang build of ft-screens got a garbage length, threw std::length_error, and aborted on the first click, taking KWin and the desktop with it. * Gaze: leave SteamVR's gaze action alone during VR games From curiousjtuber's PR #13: with the gaze service running, SteamVR restarted its eye tracker every 10 to 13 s of Beat Saber, as if the headset came off, and each restart took input focus from the game. The PR stopped every read in a game. Only the action path reaches SteamVR (UpdateActionState on the gaze set at overlay-global priority, then GetEyeTrackingDataRelativeToNow); the mmap and our tracker are read-only files. So only the action is skipped while a scene app runs, and gaze keeps moving the pointer over the dashboard in a game. The action source is only used with --source action. Co-Authored-By: CuriousJ <curious.j.tuber@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: --record-hz, and the hand recorder's design (hands/rec/DESIGN.md) ft-hands --record-hz N records at most N frame sets a second, for the hand recorder (10). DESIGN.md lays out the recorder: the headset panel, the session runner and its script, the files, review and export, consent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: ft-handpanel, the hand recorder's headset panel A head-locked SteamVR overlay for the hand recorder (hands/rec/DESIGN.md): 1.2 m ahead, 12 degrees up, 36 degrees wide, drawn with stb_truetype into three shared DMA-BUFs as ft-gazepanel does. It shows the title, step, wrapped instruction, note, countdown, hand chips, near/far bar and a "Paused" cover, driven over @ft_handpanel. It also places the touch target, a 2 cm dot in its own overlay fixed in the room where the head was at the first command for that point, and logs head and controller poses to poses.jsonl at 250 Hz from a thread of its own. Both threads take one lock around OpenVR calls. --no-vr prints each picture's state to stdout (and --dump writes the pictures), for testing without a headset. hands/rec/build.sh builds it in the dev container. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: the recorder's worn check goes by the panel's backlight, as frame-job does Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: the hand recorder's session runner and guided script hands/rec/session.py runs a recording session from script.json: it starts ft-camd and a tracking ft-hands as transient units only if they aren't running, records each section as one take (ft-hands --record-only at 10 sets/s, a new sets-N.bin after each pause), drives ft-handpanel, and writes session.json, calibration.json (identifying fields removed), prompts.jsonl and take.json. Feedback comes from the live hands file and, in the controller sections, from the panel's device poll. It also runs from the command line (--dry-run, --speed, --ring, --no-start). hands/rec/script.json: 11 sections, about 9 minutes without the object and controller sections. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: the hand recorder's window, review and export hands/rec/ft_handrec.py + main.qml (Kirigami, dev container; host launcher hands/rec/ft-handrec): consent (CONSENT.md, asked again when its version changes; profile.json with a random contributor id), the before-you-start checklist with the lighting and free-space checks, the session controls (Space pauses, Esc stops), review with a frame-set viewer that deletes ranges, takes and sessions, export with progress and cancel (warns while the headset is worn), and the upload page (UPLOAD.md, the huggingface-cli command; HF_DATASET is a placeholder). --dry-run runs sessions without processes, for testing. hands/rec/takes.py (standard library): indexes sets.bin and sets-N.bin without reading pixels, reads one set's cameras, keeps deleted ranges in take.json, and exports: deleted sets left out, zstd -10 -T2 at nice 19, manifest.json and SHA256SUMS, nothing left behind on cancel. CONSENT.md and UPLOAD.md are drafts pending a legal review; the window says contributions aren't open yet. The dev container gains zstd. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Hands: upload from the hand recorder's window, export checks, a rehearsal hands/rec/validate.py (standard library; Linux and Windows, Python 3.12+) checks an export before upload and when it's received: SHA256SUMS, an allow-list of files, the manifest's schema and keys, the consent version, a uuid4 contributor, no identifying fields in calibration.json or device.json, every sets.bin.zst decompressed to its end as a stream with each FHSET01 header checked against the manifest, jsonl lines, the total size. It decompresses with compression.zstd, zstandard or the zstd program. validate.py DIR [--json]. hands/rec/hub.py uploads an export with huggingface_hub, as a pull request to contributions/<contributor>/<session>: validate first, refuse a repeat of the same export, check the login (whoami) and access (auth_check), upload_folder(create_pr=True) with the manifest summary as the description, then record the PR under "uploads" in session.json. Errors are explained (terms not accepted, not found, 401/403, network). --dry-run makes no network calls. FT_HANDREC_DATASET overrides HF_DATASET (DeeJanuz/frametop-hands); while the texts are drafts a real upload needs FT_HANDREC_ALLOW_UPLOAD=1. The Upload page shows the login with "Check again" and how to run hf auth login in a terminal (the token never enters the window), then Upload with a phase, progress and Cancel (hub.py as a child process), the PR link, and a warning for an export uploaded before. The manual command stays as the fallback. ft-handrec --hub-dry-run. session.py also saves device.json: cv.cad_from_cal and head from /persist/device_config.json, the labeller's shape, nothing identifying; export copies it. Session ids with a -N suffix are accepted everywhere. hands/rec/rehearse.sh runs it all without the headset: ft-ringplay plays 30 s of a capture into a ring, session.py records a short test script with ft-handpanel --no-vr and a tracker, then export, validate and a dry-run upload (--repo ID uploads for real). It runs in one frame-job scope, deletes its data and stops its processes, also on Ctrl+C. hands/rec/tests/test_validate.py covers good and broken exports and hub.py without the network. The dev container gains python3-huggingface-hub. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * README: Frametop doesn't work on the SteamOS beta yet Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit |
||
|
DeeJanuzandClaude Opus 5.5
|
0ac2b10cd7 |
Remove leftovers before a release
- gaze/tracker/lab/eyes_track.py: nothing used it - Input Settings: the pointer role backend, whose page was removed - ft-screens: no log line for every floating-window resize - hands: ft-hands --help gives the palm-down default (1: off), the uninstall removes ft-handsctl's link, .frame-job is ignored - Display Settings: "Save as profile…", not "Save current arrangement" - two stale comments (ft-pointer's grabprobe, ft-gaze) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
9d91ecbcaa |
Hide screens one at a time
ft-screens: "conceal <screen|all>" hides a screen on its own, whatever the
visibility mode or the hotkey says, until "reveal"; "concealed" lists them.
(Not "hide N": older builds read anything starting with "hide" as the hotkey.)
ft-layout keeps it per screen ("hidden" in the layout), applies it when it
arranges the screens, and has hide/show N|all and hidden. Display Settings
gets a Shown switch per screen on the Visibility tab. ft-floatd floats a new
window that opens on a hidden screen, and puts a stray window on a screen
that shows. Profiles (next) use it to show only some screens.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
DeeJanuzandClaude Opus 5.5
|
8bc6739dca |
Merge hands-migration into experimental
Hand tracking (ft-camd, ft-hands, and their tools) joins the desktop. The hands file and ring move to /run/user/UID/frametop-hands/, which ft-screens' hand cutouts now read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
e2abaa06b6 |
Hands: fixes from the first headset test
- The runtime files move to /run/user/UID/frametop-hands/: the desktop session deletes /run/user/UID/frametop at every start. - The cutout copy shader runs at highp: mediump (16-bit on Adreno) stepped 1.7 texels across a 3440-pixel screen. - One hand no longer pinches both sides after its left/right call flips mid-pinch, and --pinch-palm-down (0.6) holds back pinches with the palm facing down (typing on a lap keyboard). - ft-camd judges a colour frame fresh by its luma rows only, and logs per-buffer changes at stale colour frames with FT_CAMD_DEBUG=1. - hands/run.sh caps skips the setcap when ft-camd already has them. - The replay tool dumps poses (--poses), and its pinch events carry the hand id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
4493b789fd |
Merge floating-windows into experimental
Floating windows join the keyboard and the hand cutouts. Floating panels don't get cutouts yet: their panel and popups show crops of the client buffer (texture bounds), which the side-by-side cutout buffer doesn't match. KWin gets both the spare outputs and our input method, and the pointer helper's frametop. prefix already covers the float panels. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
5714978e65 |
Add a keyboard for text fields on the desktop
Fixes #7: SteamVR's keyboard never came up for the desktop's apps, and opening it for our panels doesn't work well on the Frame (it's Steam's own panel, mounted in the dashboard's scene, it follows the laser between panels, and it takes the controllers over to SteamVR's laser). - KWin starts input/ft-textinput as the desktop's input method. It tells the input relay when a text field gains or loses focus, and the relay asks ft-screens to open or close the keyboard. The session drops the QT_IM_MODULE=xim and GTK_IM_MODULE=xim that the gamescope session sets, or Qt and GTK apps never report text fields. - The keyboard is ft-screens' own panel (screens/keyboard.cpp): a US laptop layout, typed with a controller's laser or the 3D mouse. It opens 0.7 m in front of you, below your eyes and facing you. It has a grab bar to move it, a Close key, latching Shift, Ctrl and Alt, and repeat on a held key. It's drawn into shared DMA-BUFs, so it doesn't flicker. Its keys reach the focused screen as key presses, so every app takes them. - It steps aside while the Steam menu or Steam's own keyboard is up and comes back after. A layout reset closes it, and it doesn't open without a head pose. - Frametop Input Settings has a Keyboard page: open it for every text field, only while no keyboard is connected (the default), only from a mapped button (the new Open/close keyboard action, for mice and controllers), or never. A switch keeps it open until you close it. - The pointer helper treats every frametop.* overlay as a real panel. The keyboard's shared texture reports 0x0 like SteamVR's scene-graph controls, and the helper had given it their wide catch radius. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
c8fc6351bb |
Floating windows: fix the scale crash, the click offset, and placement
- KWin's nested backend makes an output the size it's configured to times its scale, so after Meta+scroll every size ft-floatd sent was multiplied again, and an odd result disconnected KWin (buffer not divisible by its scale). ft-floatd now asks for sizes in the output's scaled terms (kwin_size), and asks again after each scale change. - SteamVR reports mouse positions on a panel with texture bounds in the whole texture, not the crop, so clicks on a floating window landed up to ~200 px off. The mouse scale is now the buffer's size, as on a screen. - A floated window starts 30 cm in front of its screen (was 5 cm), so it's easy to point at apart from the screen behind it. - No 1 s wait before a spare turns on (a disabled output never commits), and the login splash on the spares isn't taken for floating windows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Nikita Koptelov
|
4e5131d8ce |
Start Frametop's SteamVR clients only once SteamVR is up (#6)
The pointer and gaze services need steamvr.service to be running (Requisite=), and ft-pointer, ft-screens, and ft-gaze connect as a background app before switching to overlay, so they never start a vrserver of their own. One started from the dev container never finds the headset, which left a reboot stuck in a loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit
|
||
|
Nikita Koptelov
|
2a9fbdebb4 |
Start Frametop's SteamVR clients only once SteamVR is up (#6)
The pointer and gaze services need steamvr.service to be running (Requisite=), and ft-pointer, ft-screens, and ft-gaze connect as a background app before switching to overlay, so they never start a vrserver of their own. One started from the dev container never finds the headset, which left a reboot stuck in a loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit
|
||
|
Nikita Koptelov
|
3aea571496 |
Start Frametop's SteamVR clients only once SteamVR is up (#6)
The pointer and gaze services need steamvr.service to be running (Requisite=), and ft-pointer, ft-screens, and ft-gaze connect as a background app before switching to overlay, so they never start a vrserver of their own. One started from the dev container never finds the headset, which left a reboot stuck in a loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit
|
||
|
DeeJanuzandClaude Opus 5.5
|
369736f0ca |
Read the hands file through the shared header, at its Frametop path
ft-screens' hand cutouts read /run/user/UID/frametop/hands through hands/include/fh_hands.h instead of their own copy of its offsets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
691b66cd87 |
Stop ft-screens without an abort
wlroots asserts that nothing still listens to its xdg-shell and decoration globals when the display goes, so every desktop stop ended with ft-screens aborting and leaving a core dump. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
6122b7eb15 |
Floating windows: full screen, per-window scale, and drags across panels
Full screen fills the window's own panel (the margin drops to zero). Meta+scroll over a floating window changes its scale at the same size in pixels. Spare outputs are sized to a multiple of KWin's buffer scale, and screens to even sizes: an odd buffer at a fractional scale is a protocol error that disconnected KWin. The 3D mouse's drag lock now crosses onto other Frametop panels unless the pressed one is being carried. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
c2e5e861c8 |
Show floating windows as panels of their own
ft-screens makes a panel for each spare output (frametop.float.N), hidden until ft-floatd floats a window on it. The panel shows only the window's rectangle of the buffer at the density of the screen it came from, its popups and dialogs get small panels over it, pressing its title bar carries it while KWin's pointer stays put, the corner tab resizes the window in pixels, and two more buttons close it and put it back on the desktop. The session adds FLOAT_SLOTS spare outputs to KWin and starts ft-floatd; ft-layout arranges only the screens' outputs, and the pointer helper treats the new panels like screens. Not yet tried in the headset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
87a402c68d |
Add ft-floatd and the frametop-float KWin script
ft-floatd loads the script into the desktop's KWin, which reports windows over D-Bus and takes commands through a long poll. Floating a window (the window menu's Float in VR, Meta+Shift+F, or ft-float) turns on a spare output sized to the window plus a margin, moves the window onto it, and tells ft-screens the panel's crop, density, and place; docking puts it back and turns the spare off. Tested on the headless test desktop; the panel side in ft-screens comes next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
4859412126 |
Put the first click after crossing onto a screen where the pointer is
KWin's nested backend ignores the position in wl_pointer.enter, and wlroots drops a motion to the position it entered at, so KWin kept its old pointer until the next move. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
42a755b9f4 |
Add a headless test mode to ft-screens and a throwaway test desktop
ft-screens --no-vr runs without SteamVR and leaves the input relay alone; --control names its control socket; "toplevels" lists KWin's windows with their titles and sizes; "input" feeds pointer events as if from a panel. screens/test/headless.sh starts it with a bare nested KWin next to the running desktop, and loads KWin scripts, runs apps, and takes screenshots. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
0a88a6b7c9 |
Release a button held on a screen even when the laser lets go between panels
While a button is held on a screen, the laser leaving it no longer takes KWin's pointer, and an invisible catcher overlay sits on the laser whenever it's off every panel, so the release reaches KWin at the pointer's last spot. The pointer helper also reports the mouse's left release as a backstop. Not yet tested in the headset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
a31d42b25c |
Move hand cutouts ahead to where the hands will be
The tracked hands arrive 30-60 ms after the cameras saw them and reach the displays later still, so holes trailed moving hands. Track each hand's palm velocity in the room and move its capsules ahead to about when the frame is on the displays, every tick, so the holes also move smoothly between tracker updates. Slow hands aren't moved (their velocity is noise). The control socket gets cutouts predict on|off and cutouts lead <ms>. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
094a7b27f0 |
Don't cut out hand parts right in front of the eyes
A capsule end near the eyes' plane projects far across a screen with a huge radius, so one bad hand estimate there tore a hole through the screens for a moment. Clip capsules 12 cm in front of the eye, as the tracker now does too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
b7cbd9fe16 |
Cut tracked hands out of screens so you see them through it (work in progress)
Where frame-hands tracks a hand between an eye and a screen, that eye sees the room through the screen. handcut.cpp draws the screen's buffer side by side (one half per eye) with the hands cut out, only while a hand is in front of it. The cutouts command turns it on or off. ft-handtest tries it on a test panel. Not yet tested in the headset. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
0be802c7e1 |
Add named layouts and a head pin for screens
Named layouts: Save current arrangement in Frametop Display Settings now asks for a name, and saved layouts are listed with the presets under Arrangement, with rename and delete next to the list. A named layout is the custom arrangement under a name: each screen's place relative to your head, width, curve, and pin, but not resolution or scale. Using one copies it into the custom arrangement, so desktop start, Meta+Shift+R, and Arrange now apply it unchanged; "active" remembers the name, and a plain capture clears it. ft-layout gains save, use, layouts, rename, and delete. A layout saved with fewer screens than there are now leaves the others where they were saved last, or where the preset puts them. Head pin: ft-screens' pin command takes "head" as well as left and right, and pins the screen to the headset (device 0) where it is, like a HUD. A head-pinned screen skips the wrist facing rule and shows whenever the screens do. Carrying it re-pins it to the head on release, like a wrist pin, so it can be adjusted in VR. The Visibility tab (now Visibility & pins) sets each screen's pin: in the room, either wrist, or your head, and the pin command now rejects anything but left, right, or head (it used to take anything else as left). This removes the "no HUD" limit from the docs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
18aa0fec3a |
Put clicks where the cursor is on scaled desktop screens
With a screen's scale set to anything but 100%, clicks landed away from the cursor, further off the further from the top left (#3). ft-screens hands KWin panel positions in buffer pixels, and KWin's nested Wayland backend (6.2.5, WaylandInputDevice) adds surface coordinates to its output's logical position without dividing by the output's scale. At 125% a click at the middle of a 3440x1440 screen, (1720, 720), reached KWin as logical (1720, 720), pixel (2150, 900). ft-screens now keeps a scale per screen and divides pointer positions by it. ft-layout sends each screen's scale, as KWin reports it after applying, with a new "scale N s" command, whenever it applies scales: at desktop start and from Frametop Display Settings. Screens default to 1, so an ft-layout that never sends it keeps the old behaviour. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
e84ac22313 |
Arrange the desktop's outputs the way the screens are around you
KWin's output positions now follow where the Frametop screens are in the room instead of their numbers: a screen you see to the left of another is to its left in Plasma, so the pointer and dragged windows cross straight to it. Screens one above the other stack, and wrist-pinned screens go last. ft-layout scale works this out from ft-screens' head pose and screen poses, and runs after arranging, capturing, or pinning; ft-screens runs it half a second after a screen is let go. With no head pose (headset off) KWin's order is kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
8f051db083 |
Send typing to the panel clicked last
An ungrabbed keyboard reached both sides at once: gamescope, which in VR reads every input device itself (the SteamOS build's InputStealer), typed into its focused app, and ft-screens typed into the desktop. Space in the desktop paused Spotify on the dashboard, including every space frame-voice dictated. And while the SteamVR dashboard was open, the desktop got no keys at all. Typing now follows the last click. ft-screens sees clicks on its own screens; the pointer helper reports the panel under the dot on each mouse press, so a click on any other panel sends typing to Steam. While typing goes to the desktop and the screens are showing, ft-screens tells the relay every second, and the relay grabs pass-through keyboards. A grab waits until no key is down, and the relay lets go if ft-screens stops reporting. Controller clicks on other panels aren't visible to overlay apps, so they don't move typing (noted in the README). A program that reads every keyboard for a hotkey loses a grabbed one. Repeating the keys on another input device doesn't work, since gamescope reads that too, so with SHARE_KEYS=1 the relay sends them to @frametop_keys as datagrams with the device name. It's off by default: any local process that binds that name first would get every key typed into the desktop. The docs now say gamescope reads keyboards and the headset's buttons itself; they said SteamVR passed keys on to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
eee4aba554 |
Let key releases reach the desktop while the dashboard is open
ft-screens dropped every key while the SteamVR dashboard was open or no screen had focus, releases included. A modifier held as the dashboard opened stayed down in the desktop, so later keys launched shortcuts (T opened Konsole as Ctrl+Alt+T). The release of a key the desktop got the press for now always goes through. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
951fe139b1 |
Keep a screen's controls with it when the layout moves it
ft-screens read a screen's pose back from SteamVR right after setting it to place its controls. SteamVR could still return the old pose, so after Arrange now (or a layout reset) the bar and buttons stayed where the screen had been. ft-screens now keeps each screen's pose itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
a4499c1c16 |
Reveal a screen's controls when a laser lands on them
Hidden controls were taken out of SteamVR, so only ft-screens' own proximity test could bring them back, and that test didn't match every controller's laser. They now stay in place fully transparent while hidden, and SteamVR's hover event on one reveals them, for any laser. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
54b1e5cd4b |
Aim controller rays along the laser, not the controller
The Frame controllers' laser comes from their render model's tip, 40 degrees below the controller pose's forward axis. ft-screens tested rays along the pose, so a controller's laser near a screen's controls didn't reveal them, and dragging the resize tab or the roll knob with a controller followed the wrong point. Rays now start from the tip. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
b9dc805e75 |
Hide the screens during VR games unless the dashboard is open
A new setting, During VR games: with Always, the screens hide while a VR game runs and show while the SteamVR dashboard is open (the default), or stay visible over the game. The hotkey still shows them; a game starting or stopping resets it. Socket command: ingames hide|visible. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
c18f6c732a |
Keep the controllers in VR games while the screens stay up
Visible screens kept SteamVR's laser mouse on, which takes the controllers away from a VR game. Now, by default, that's off while a scene app runs: the screens stay over the game and the 3D mouse or the dashboard works them. Frametop Display Settings has the choice (always, except during VR games, only with the dashboard open); the socket command is controllers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
DeeJanuzandClaude Opus 5.5
|
d439bc3f25 |
Frametop: a multi-screen desktop and universal 3D mouse for the Steam Frame
Several KDE Plasma screens floating in SteamVR, each a real monitor of any resolution and shape, shown by our own compositor (ft-screens), with a layout, wrist pinning, and visibility modes; a Bluetooth mouse that drives all of SteamVR as a room-anchored 3D pointer (input relay, ft-pointer helper, ft_pointer SteamVR driver); two settings apps; and Bluetooth LE fixes. Installs on the headset with ./install.sh. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |