Conflicts with experimental's pause_toggle, steam_menu and command: actions and its
AnnounceOverlay: both kept. The spin bindings join the Meta tap in the defaults, spinning
doesn't need pointer mode, and like other actions it does nothing while Frametop is paused.
AnnounceOverlay now sends through the PR's SendPointer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GetComponentStateForDevicePath with no input source handle fails for
every render model component while a VR game runs (checked 2026-10-03
with a game up: all 21 components of frame_controller_right). TipOffset
then fell back to the controller's pose, which aims 40 degrees above the
Frame controller's laser. In games, pointing at a screen's middle missed
it and pointing below it hit, so the new aim-to-laser only worked from
the bottom; the controls' reveal and pin/roll aim were off the same way.
GetComponentState still answers then, with the same tip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SteamVR's own floating windows take the laser while a controller points
at them in a game and give it back when it points away. Frametop's
panels didn't: with the controllers left to the game (outside_games, the
default, or dashboard), they couldn't be clicked without the dashboard.
ft-screens now sets MakeOverlaysInteractiveIfVisible on a screen or
floating window while a hand controller's laser pose meets it, its
controls, or its popups (UpdateAim; curved screens hit on their
cylinder), and clears it 0.3 s after the aim leaves a wider margin. A
drag or a held button keeps it on. The keyboard, one overlay, uses
ComputeOverlayIntersection and now follows the mode when a game starts
or ends while it's open. This replaces the reset button's own aim zone.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each desktop screen gets a reset button left of its bar (a reticle). It
puts every screen back in its layout around where you are now, like
Meta+Shift+R (ft-layout apply).
In a VR game the screens leave the controllers to the game (the
outside_games and dashboard modes), so a controller couldn't click any
of their controls. Aiming a hand controller at the reset button now sets
MakeOverlaysInteractiveIfVisible on that button's overlay alone, so the
trigger clicks it; the flag clears half a second after the aim leaves a
zone twice as wide, and the game gets the controllers back. The aim
comes from the laser poses ft-screens already reads to show the controls.
The ft-layout spawn is now RunLayout(cmd), shared with the arrange.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- ft-screens "spin next|prev|<degrees>": every unpinned screen and
floating window turns together about a vertical axis through your
head (0.3 s, eased), so the next panel on the right or left comes to
straight ahead; the arrangement stays as it is. Taps during a spin
add to it, from where the panels are headed; grabbing a panel or
placing it (ft-layout, ft-floatd) takes it out of the spin
- when a spin settles, the panel in front gets the pointer (recenter),
typing (as after a click), and KWin's active window: its floating
window, or the top window on a screen (ft-floatd "front N", the KWin
script's activate-output). KWin's outputs follow the screens'
new places (ft-layout scale), as after a move
- the input relay: spin_next and spin_prev actions, Meta+Alt+Tab and
Meta+Alt+Shift+Tab by default; Frametop Input Settings lists them.
Not Meta+Tab: that's Cmd+Tab on a Mac reached through a remote
desktop like RustDesk, and the relay would take the Mac's app
switcher. Meta+Alt+Tab (Cmd+Option+Tab) is unused on macOS,
Windows, and KDE
Used on the Frame (SteamOS 0.3.0 build 20260922) with one screen and
three or four floating windows, through RustDesk to a Mac.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The helper now reads SteamVR's list of panels every 20 s instead of every second, so a panel
made in between (a floating window's menu, frametop.float.N.sub.K, or the Frametop keyboard
the first time it opens) couldn't be clicked with the mouse until the next read. ft-screens
now sends "overlay <key>" to @ft_pointer_helper right after it makes one, and the helper adds
it to its list at once (only frametop.* keys). An older helper ignores it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-screens gave KWin a frame callback for every committed screen on each tick, and the tick
was an 11 ms timer set again after each run, so it slid through the display's frame and came
about 85 times a second at 90 Hz: the desktop repeated a frame several times a second (judder
in scrolling and video), and KWin drew every screen in one burst at a random point of
vrcompositor's frame. Hidden screens got the same 90 Hz unless Frametop was paused for a game.
- Ticks run on a timerfd at absolute times, once per display frame, 1 ms after the vsync
(IVRSystem::GetTimeSinceLastVsync and the HMD's display frequency, read once a second), so
KWin gets its callbacks early in the frame. Measured with --no-vr: 91 wakeups a second
instead of about 85. On the Frame the vsync times SteamVR reports lie on a 90 Hz grid.
- Each screen's callbacks come at a rate for how much of it you see (vr.cpp,
UpdateAttention): every frame while focused (within 12 degrees of where your head points,
a laser or the mouse on it in the last 1.5 s, carried, or typed on), 15 a second for the
rest of what you see (within 60 degrees), and 1 a second when hidden, behind you, or
paused. Levels rise at once and fall after 1.5 s (focused) or 0.5 s (in view). KWin draws
a screen only after its callback and its apps wait for theirs, so this throttles the apps
too. A screen where nothing changes costs nothing at any rate, as before.
- A video in view keeps every frame: 8 commits in a row that each redraw 6% or more of the
screen, at 10 a second or more, count as one (from the surface's buffer damage).
- "rates F V H" / --rates set the three rates (default 0 15 1, 0 = every frame), "rates?"
shows them and each screen's level, "watch S" gives everything full rate for S seconds for
a remote viewer (vnc-bridge.sh renews it), and "phase MS" moves the ticks for tuning.
- ft-screens' main thread runs at nice -5 after the session starts: SteamOS allows down to
-8 once the soft RLIMIT_NICE is raised, and KWin waits on these ticks. It had spent nearly
3 times as long waiting to run as running.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Frametop kept using the headset during games: with gaze mode off, our eye tracker still took
about 60% of a core, remote desktop about 2 cores while on, and KWin kept drawing hidden
screens because ft-screens sent their frame callbacks at 90 Hz. Pausing gives that back, and
resuming brings back only what pausing stopped. It's also a way to keep the gaze service and
our eye tracker off during games, which PR #13 asked for.
Paused (input/game_pause.py, run by the input relay):
- frametop-gaze stops (ft-eyegrab then idles by itself), and hand tracking and remote desktop
stop if they run
- the desktop hides and slows down: ft-screens "pause on" hides every panel and sends KWin a
frame callback once a second; or, with pause_desktop "close", the desktop closes and starts
again on resume
- the relay lets go of the 3D mouse, typing goes to Steam, and mapped buttons and key
combinations do only pause_toggle, steam_menu and commands
Toggled by both thumbsticks clicked together twice (configurable), read passively from
vrserver's web socket (input/vrws.py) so it works in games and takes nothing from them; by the
new pause_toggle action; by input/ft-pause; and, with pause_auto (default on), by a VR game
starting and ending, which the pointer helper now reports ("vrgame 1|0"). Frametop Input
Settings has a Games page for it. update-check.py checks the web socket, and doesn't count a
paused gaze service as failed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While a button pressed on a screen is held, UpdateCatcher checks every tick
whether the laser is still on one of the screen's overlays. It built that list
from s.All().begin() and s.All().end(), but All() returns a std::array by
value: iterators into two different temporaries, which is undefined behaviour.
A clang build of ft-screens got a garbage length, threw std::length_error, and
aborted on the first click, taking KWin and the desktop with it.
The 3D mouse drives SteamVR's laser through the ft_pointer virtual
controller, so its events reach the screens the same way a controller's
do. The filter held every press, which turned the mouse's short drags
(selecting a character or two, nudging a slider) into clicks. Mark
button events from hand controllers and start the filter only on those.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- gaze/tracker/lab/eyes_track.py: nothing used it
- Input Settings: the pointer role backend, whose page was removed
- ft-screens: no log line for every floating-window resize
- hands: ft-hands --help gives the palm-down default (1: off), the
uninstall removes ft-handsctl's link, .frame-job is ignored
- Display Settings: "Save as profile…", not "Save current arrangement"
- two stale comments (ft-pointer's grabprobe, ft-gaze)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-screens: "conceal <screen|all>" hides a screen on its own, whatever the
visibility mode or the hotkey says, until "reveal"; "concealed" lists them.
(Not "hide N": older builds read anything starting with "hide" as the hotkey.)
ft-layout keeps it per screen ("hidden" in the layout), applies it when it
arranges the screens, and has hide/show N|all and hidden. Display Settings
gets a Shown switch per screen on the Visibility tab. ft-floatd floats a new
window that opens on a hidden screen, and puts a stray window on a screen
that shows. Profiles (next) use it to show only some screens.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hand tracking (ft-camd, ft-hands, and their tools) joins the desktop. The
hands file and ring move to /run/user/UID/frametop-hands/, which ft-screens'
hand cutouts now read.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Floating windows join the keyboard and the hand cutouts. Floating panels
don't get cutouts yet: their panel and popups show crops of the client
buffer (texture bounds), which the side-by-side cutout buffer doesn't
match. KWin gets both the spare outputs and our input method, and the
pointer helper's frametop. prefix already covers the float panels.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes#7: SteamVR's keyboard never came up for the desktop's apps, and
opening it for our panels doesn't work well on the Frame (it's Steam's own
panel, mounted in the dashboard's scene, it follows the laser between
panels, and it takes the controllers over to SteamVR's laser).
- KWin starts input/ft-textinput as the desktop's input method. It tells
the input relay when a text field gains or loses focus, and the relay
asks ft-screens to open or close the keyboard. The session drops the
QT_IM_MODULE=xim and GTK_IM_MODULE=xim that the gamescope session sets,
or Qt and GTK apps never report text fields.
- The keyboard is ft-screens' own panel (screens/keyboard.cpp): a US laptop
layout, typed with a controller's laser or the 3D mouse. It opens 0.7 m
in front of you, below your eyes and facing you. It has a grab bar to
move it, a Close key, latching Shift, Ctrl and Alt, and repeat on a held
key. It's drawn into shared DMA-BUFs, so it doesn't flicker. Its keys
reach the focused screen as key presses, so every app takes them.
- It steps aside while the Steam menu or Steam's own keyboard is up and
comes back after. A layout reset closes it, and it doesn't open without
a head pose.
- Frametop Input Settings has a Keyboard page: open it for every text
field, only while no keyboard is connected (the default), only from a
mapped button (the new Open/close keyboard action, for mice and
controllers), or never. A switch keeps it open until you close it.
- The pointer helper treats every frametop.* overlay as a real panel. The
keyboard's shared texture reports 0x0 like SteamVR's scene-graph
controls, and the helper had given it their wide catch radius.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- KWin's nested backend makes an output the size it's configured to times its
scale, so after Meta+scroll every size ft-floatd sent was multiplied again,
and an odd result disconnected KWin (buffer not divisible by its scale).
ft-floatd now asks for sizes in the output's scaled terms (kwin_size), and
asks again after each scale change.
- SteamVR reports mouse positions on a panel with texture bounds in the whole
texture, not the crop, so clicks on a floating window landed up to ~200 px
off. The mouse scale is now the buffer's size, as on a screen.
- A floated window starts 30 cm in front of its screen (was 5 cm), so it's
easy to point at apart from the screen behind it.
- No 1 s wait before a spare turns on (a disabled output never commits), and
the login splash on the spares isn't taken for floating windows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The pointer and gaze services need steamvr.service to be running (Requisite=), and ft-pointer, ft-screens, and ft-gaze connect as a background app before switching to overlay, so they never start a vrserver of their own. One started from the dev container never finds the headset, which left a reboot stuck in a loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 51b4e79318)
The pointer and gaze services need steamvr.service to be running (Requisite=), and ft-pointer, ft-screens, and ft-gaze connect as a background app before switching to overlay, so they never start a vrserver of their own. One started from the dev container never finds the headset, which left a reboot stuck in a loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 51b4e79318)
The pointer and gaze services need steamvr.service to be running (Requisite=), and ft-pointer, ft-screens, and ft-gaze connect as a background app before switching to overlay, so they never start a vrserver of their own. One started from the dev container never finds the headset, which left a reboot stuck in a loop.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 51b4e79318)
ft-screens' hand cutouts read /run/user/UID/frametop/hands through
hands/include/fh_hands.h instead of their own copy of its offsets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-screens makes a panel for each spare output (frametop.float.N), hidden
until ft-floatd floats a window on it. The panel shows only the window's
rectangle of the buffer at the density of the screen it came from, its
popups and dialogs get small panels over it, pressing its title bar
carries it while KWin's pointer stays put, the corner tab resizes the
window in pixels, and two more buttons close it and put it back on the
desktop. The session adds FLOAT_SLOTS spare outputs to KWin and starts
ft-floatd; ft-layout arranges only the screens' outputs, and the pointer
helper treats the new panels like screens. Not yet tried in the headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-screens --no-vr runs without SteamVR and leaves the input relay alone;
--control names its control socket; "toplevels" lists KWin's windows with
their titles and sizes; "input" feeds pointer events as if from a panel.
screens/test/headless.sh starts it with a bare nested KWin next to the
running desktop, and loads KWin scripts, runs apps, and takes screenshots.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While a button is held on a screen, the laser leaving it no longer takes
KWin's pointer, and an invisible catcher overlay sits on the laser whenever
it's off every panel, so the release reaches KWin at the pointer's last
spot. The pointer helper also reports the mouse's left release as a
backstop. Not yet tested in the headset.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tracked hands arrive 30-60 ms after the cameras saw them and reach the
displays later still, so holes trailed moving hands. Track each hand's palm
velocity in the room and move its capsules ahead to about when the frame is
on the displays, every tick, so the holes also move smoothly between tracker
updates. Slow hands aren't moved (their velocity is noise). The control
socket gets cutouts predict on|off and cutouts lead <ms>.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Where frame-hands tracks a hand between an eye and a screen, that eye sees the
room through the screen. handcut.cpp draws the screen's buffer side by side
(one half per eye) with the hands cut out, only while a hand is in front of it.
The cutouts command turns it on or off. ft-handtest tries it on a test panel.
Not yet tested in the headset.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Named layouts: Save current arrangement in Frametop Display Settings now
asks for a name, and saved layouts are listed with the presets under
Arrangement, with rename and delete next to the list. A named layout is
the custom arrangement under a name: each screen's place relative to
your head, width, curve, and pin, but not resolution or scale. Using one
copies it into the custom arrangement, so desktop start, Meta+Shift+R,
and Arrange now apply it unchanged; "active" remembers the name, and a
plain capture clears it. ft-layout gains save, use, layouts, rename, and
delete. A layout saved with fewer screens than there are now leaves the
others where they were saved last, or where the preset puts them.
Head pin: ft-screens' pin command takes "head" as well as left and
right, and pins the screen to the headset (device 0) where it is, like a
HUD. A head-pinned screen skips the wrist facing rule and shows whenever
the screens do. Carrying it re-pins it to the head on release, like a
wrist pin, so it can be adjusted in VR. The Visibility tab (now
Visibility & pins) sets each screen's pin: in the room, either wrist, or
your head, and the pin command now rejects anything but left, right, or
head (it used to take anything else as left). This removes the "no HUD"
limit from the docs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
KWin's output positions now follow where the Frametop screens are in the room
instead of their numbers: a screen you see to the left of another is to its
left in Plasma, so the pointer and dragged windows cross straight to it.
Screens one above the other stack, and wrist-pinned screens go last.
ft-layout scale works this out from ft-screens' head pose and screen poses,
and runs after arranging, capturing, or pinning; ft-screens runs it half a
second after a screen is let go. With no head pose (headset off) KWin's
order is kept.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An ungrabbed keyboard reached both sides at once: gamescope, which in VR
reads every input device itself (the SteamOS build's InputStealer), typed
into its focused app, and ft-screens typed into the desktop. Space in the
desktop paused Spotify on the dashboard, including every space frame-voice
dictated. And while the SteamVR dashboard was open, the desktop got no
keys at all.
Typing now follows the last click. ft-screens sees clicks on its own
screens; the pointer helper reports the panel under the dot on each mouse
press, so a click on any other panel sends typing to Steam. While typing
goes to the desktop and the screens are showing, ft-screens tells the
relay every second, and the relay grabs pass-through keyboards. A grab
waits until no key is down, and the relay lets go if ft-screens stops
reporting. Controller clicks on other panels aren't visible to overlay
apps, so they don't move typing (noted in the README).
A program that reads every keyboard for a hotkey loses a grabbed one.
Repeating the keys on another input device doesn't work, since gamescope
reads that too, so with SHARE_KEYS=1 the relay sends them to
@frametop_keys as datagrams with the device name. It's off by default:
any local process that binds that name first would get every key typed
into the desktop.
The docs now say gamescope reads keyboards and the headset's buttons
itself; they said SteamVR passed keys on to it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ft-screens read a screen's pose back from SteamVR right after setting it
to place its controls. SteamVR could still return the old pose, so after
Arrange now (or a layout reset) the bar and buttons stayed where the screen
had been. ft-screens now keeps each screen's pose itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hidden controls were taken out of SteamVR, so only ft-screens' own
proximity test could bring them back, and that test didn't match every
controller's laser. They now stay in place fully transparent while
hidden, and SteamVR's hover event on one reveals them, for any laser.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Frame controllers' laser comes from their render model's tip, 40
degrees below the controller pose's forward axis. ft-screens tested rays
along the pose, so a controller's laser near a screen's controls didn't
reveal them, and dragging the resize tab or the roll knob with a
controller followed the wrong point. Rays now start from the tip.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new setting, During VR games: with Always, the screens hide while a VR
game runs and show while the SteamVR dashboard is open (the default), or
stay visible over the game. The hotkey still shows them; a game starting or
stopping resets it. Socket command: ingames hide|visible.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Visible screens kept SteamVR's laser mouse on, which takes the controllers
away from a VR game. Now, by default, that's off while a scene app runs:
the screens stay over the game and the 3D mouse or the dashboard works
them. Frametop Display Settings has the choice (always, except during VR
games, only with the dashboard open); the socket command is controllers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Several KDE Plasma screens floating in SteamVR, each a real monitor of any
resolution and shape, shown by our own compositor (ft-screens), with a
layout, wrist pinning, and visibility modes; a Bluetooth mouse that drives
all of SteamVR as a room-anchored 3D pointer (input relay, ft-pointer
helper, ft_pointer SteamVR driver); two settings apps; and Bluetooth LE
fixes. Installs on the headset with ./install.sh.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
