From 0af087c7762d8998c05eb45dfb60aab423240536 Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 22:36:21 -0600 Subject: [PATCH 01/13] Plan floating windows: any desktop app in a VR panel of its own Co-Authored-By: Claude Opus 5.5 --- docs/floating-windows.md | 154 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 154 insertions(+) create mode 100644 docs/floating-windows.md diff --git a/docs/floating-windows.md b/docs/floating-windows.md new file mode 100644 index 0000000..6af6d5f --- /dev/null +++ b/docs/floating-windows.md @@ -0,0 +1,154 @@ +# Floating windows (plan) + +Status: plan only, on the `floating-windows` branch. Nothing here is built yet. + +The goal is to let any desktop app float in VR in a panel of its own, like SteamVR's floating windows, while it stays part of the Frametop desktop. That means drag and drop, the clipboard, and focus keep working between floating windows and the screens. + +- There are two ways to get a floating window. Launch the app floating, or drag a desktop window by its title bar off a screen and let go in the air. +- There are two ways to put one back. Drag it onto a screen, or press its "back to desktop" button. +- Files, text, and images drag between any two floating windows, and between floating windows and the screens. + +## The approach: each floating window gets a KWin output of its own + +Drag and drop and the clipboard only work between windows of the same compositor. A Wayland window can't move from one compositor to another. So a floating window has to stay a KWin window. + +ft-screens already shows each KWin output as a panel. It sets the output's size with an `xdg_toplevel` configure, and KWin resizes the output to match. So a floating window can get an output of its own, sized to fit it, and ft-screens shows that output as a panel with its own controls. To KWin this is an ordinary desktop with more monitors. Dragging between two floating windows is the same as dragging between two monitors, which KWin already handles. ft-screens already moves the pointer between panels in the middle of a drag: `handle_vr_event` moves pointer focus to another KWin window even while a button is held. + +Alternatives we considered: + +- **Run floating apps directly on ft-screens.** It's a wlroots compositor, so apps could connect to it and get a panel per window. But they would get no drag and drop or clipboard with desktop apps unless we wrote a bridge. Also, a window that's already on the desktop could never be torn off, because a Wayland client can't change compositors. Rejected. +- **One large hidden "canvas" output.** Every floating window would sit on one big output, and each panel would show a crop of it (`SetOverlayTextureBounds`). That needs only one extra output, with no copies, and popups could extend past their window. But an 8K canvas uses about 128 MB per buffer, with two or three buffers in KWin's swapchain. It would also have to repack windows whenever one resized. Maximize and fullscreen would fill the whole canvas, and every window would share one scale. This is the fallback if per-window outputs don't work. +- **Screencast single windows** (`zkde_screencast` `stream_window`, over PipeWire). This adds copies and latency, and the window still needs a real place in KWin's layout to receive input. Rejected. +- **SteamOS's own floating windows** (Launch a program from the dashboard). Those apps run in gamescope, outside KWin, so they can't drag and drop with the desktop. + +### Where the extra outputs come from + +KWin's nested backend opens its outputs at start (`--output-count`). There are two ways to get more: + +1. **Spare outputs (try first).** Start KWin with the screen count plus `FLOAT_SLOTS` outputs (default 6). Park each spare output until it's needed. If the nested backend supports it, disable the spare with `kscreen-doctor`. Otherwise make it tiny (64 × 64), put it far away in KWin's layout, and don't show its panel. Floating a window takes a spare, and docking the window gives the spare back. `FLOAT_SLOTS` limits how many windows can float at once, and changing it means restarting the desktop. +2. **Virtual outputs (fallback).** KWin 6.2.5's nested backend implements `createVirtualOutput` and `removeVirtualOutput` (checked in `libkwin.so`). A client reaches them through `zkde_screencast_unstable_v1`'s `stream_virtual_output` request. There's no limit on count, and the output gets whatever name we choose. But that request belongs to a screencast protocol, and only privileged clients may use it. A helper in the container can't be matched to a desktop file, so we would need `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1`. That turns off the permission checks for every client in the session. design.md also records a KWin 6.2.5 crash (`textureForOutput`) when such an output was streamed. It might be safe if nothing ever consumes the PipeWire stream, but that needs a test. + +ft-screens has to tell a floating window's output from a screen's. KWin titles each nested output window `KDE Wayland Compositor %1`. If `%1` turns out to be the output name (to confirm), ft-screens can read the title. Otherwise it can go by creation order, since spare outputs come after the screens. + +## How the parts fit together + +``` + KWin script "frametop-float" ft-floatd (host, Python) ft-screens + window events, moves, menus ── D-Bus ──▶ window ↔ output ↔ panel table ── @ft_screens ──▶ panels, controls, + runs commands ◀─ long poll ─ spare outputs (kscreen-doctor) ◀─ @frametop_float ─ lasers, ghost, catcher +``` + +- **KWin script `frametop-float`** (JavaScript). A script keeps working across KWin updates. A C++ effect would have to match the host's exact KWin build, and our build container is Fedora, not SteamOS. The script watches windows (`windowAdded`/`windowRemoved`, `interactiveMoveResizeStarted`/`Stepped`/`Finished`, `outputChanged`, `minimizedChanged`, `windowActivated`). It runs commands: move a window to an output, maximize it, put it on all virtual desktops, and restore it. It adds "Float in VR" to the window menu (`registerUserActionsMenu`) and registers a shortcut (`registerShortcut`, Meta+Shift+F). KWin scripts can call D-Bus but can't serve it, so commands come back through a long poll. The script calls ft-floatd's `NextCommand`, which answers when a command is ready, and then the script calls it again. The fallback is loading one-shot scripts through `org.kde.kwin.Scripting`, the way kdotool does. All of these API names are present in the host's KWin 6.2.5. +- **ft-floatd** (Python). The host has dbus-python and PyGObject. It owns `org.frametop.Float` on the session's private bus, and it keeps the table of which window is on which output and panel. It hands out spare outputs and sets their size, scale, and position with `kscreen-doctor`, as ft-layout does. It tells ft-screens where each floating window goes and tells the script which window goes where. It also remembers each app's placement, keyed by desktop file name. +- **ft-screens.** `Screen` becomes a panel with a kind: screen or floating window. Floating panels get the same bar, curve, roll, resize tab, and wrist pin, plus close and "back to desktop" buttons. New parts are the tear-off ghost, the catcher, carrying a panel during a KWin move, and the dock target highlight. `MAX_SCREENS` goes from 8 to 16. Commands arrive on `@ft_screens`. Events go out to `@frametop_float` from an unbound socket, the same way ft-screens talks to the input relay. +- **Session script.** Adds the spare outputs to the count, starts ft-floatd, and enables the KWin script in the session's `kwinrc`. +- **ft-layout.** Arranges only the screens' outputs. Today it arranges everything in `kscreen-doctor -j`, so it has to skip floating windows' outputs. +- **ft-pointer.** Changes to the drag lock (see "Drag and drop between panels"). +- **Frametop Display Settings.** Gets a Floating windows section. + +Program names stay within 15 characters (`ft-floatd`). Overlay keys are `frametop.float.N` and `frametop.float.N.bar`, and so on. + +## A floating window + +- **Size and scale.** Its output is the window's size, at the scale of the screen it came from. The panel's width is its pixel width times that screen's metres per pixel, so text stays the same size in VR. +- **Window state.** The window is maximized on its output and set to show on all virtual desktops. It keeps its title bar: Breeze drops the borders of a maximized window but keeps the title bar. Apps that draw their own title bar (GTK, Chromium) keep theirs. +- **Moving.** Press the title bar. KWin starts an interactive move and the script reports it. ft-screens then stops forwarding pointer motion to KWin, so KWin's pointer stays at the press point, the window moves by nothing, and it doesn't come out of maximize. Meanwhile ft-screens carries the panel with the pressing device, the same way the bar does today: it follows rigidly, scroll pushes and pulls, and the 3D mouse's right-drag tilts. When the button comes up, KWin gets the release at the press point. The bar under the panel works too. +- **Resizing.** The corner tab changes the window's size in pixels at the same density, so the app lays itself out again. A screen's tab only scales the panel. Resizing is throttled to about 20 updates a second, with a minimum of 320 × 200, like screens. +- **Buttons.** The close button closes the window. "Back to desktop" docks it where it came from. +- **Minimize.** Minimizing, from the title bar or the taskbar, hides the panel, and restoring it shows the panel again. Floating windows stay in the desktop's taskbar and in Alt+Tab. If one is activated while it's out of view, it can come around in front of you (optional). +- **New windows.** A dialog of a floating window (`transientFor`) floats in front of its parent. Other new windows open on the screens: if KWin puts a new window on a floating window's output, the script moves it to the screen used last. +- **Popups.** KWin keeps menus and tooltips inside their output, so a menu can't reach past the window's edges. The first version accepts that. A later version pads the output and crops the panel to the window plus any open popups (`SetOverlayTextureBounds`, with mouse positions offset by the crop). + +## Tearing a window off a screen + +1. Press a desktop window's title bar and drag it. KWin starts a move, and the script tells ft-floatd, which tells ft-screens: `move-start `. +2. While the button is held, the laser leaves every Frametop panel. A second trigger is also possible: scrolling toward you while dragging pulls the window out of the screen. +3. ft-screens shows a ghost: an overlay showing the screen's live buffer cropped to the window (`SetOverlayTextureBounds`, no copy). It's at the screen's pixel density and distance, on the laser, facing you, with the point you grabbed under the laser. The ghost takes mouse input, so SteamVR's laser lands on it and the release comes to ft-screens. +4. Go back onto a screen before letting go, and the ghost disappears. It's an ordinary move again. +5. Let go on the ghost, and ft-screens releases the button in KWin, which ends the move. It then reports the tear-off to ft-floatd, with the window, the ghost's pose, and the density. ft-floatd takes a spare output and has ft-screens size it to the window and put its panel at the ghost's pose. Then it has the script move the window onto that output and maximize it. The ghost stays until the new panel's first frame at the right size arrives, so nothing blinks. + +## Putting it back + +- **Button.** "Back to desktop" returns the window to the screen, position, and size it had before it was torn off (saved at tear-off). +- **Dragging.** Carry the floating window, by its title bar or its bar, until the spot you're pointing at is on a screen. Then push it flush with the screen, within about 10 cm of its surface: scroll away with the mouse, or move the controller forward. The screen shows where the window will land, and letting go docks it. A carried panel keeps its distance, so moving a floating window in front of a screen never docks it by accident. +- Docking gives the output back (disabled or parked) and removes the panel. + +## Launching an app floating + +- **In the desktop.** Use "Float in VR" in any window's menu, or press Meta+Shift+F for the active window. +- **From a command.** `ft-float run ` and `ft-float launch ` start an app and float its first window. ft-floatd records the process it started, and the script matches new windows by PID, including child processes. Some single-instance apps (Firefox, D-Bus-activated apps) open the window from a process that was already running. Those are matched by desktop file name within a few seconds, or by `XDG_ACTIVATION_TOKEN` where the app honors it. +- **From the headset without the desktop open.** Add one new launcher entry, "Frametop Apps", instead of one entry per app, since per-app entries would need setting up for every new app. The entry starts the Frametop session with the screens hidden (floats-only mode) and opens an app picker as a floating window. Picking an app launches it floating. The screens are still there: Meta+Shift+H shows them. Everything runs in one session, so dragging between a standalone app and a desktop app works. If the desktop is already running, the entry just opens the picker. +- **The picker.** Either KRunner, floated, or a small Kirigami app like the settings apps, with a grid of apps and their icons. +- **Visibility in floats-only mode.** The screens stay hidden and floating windows show. See open question 2 for how floating windows follow the visibility modes otherwise. + +## Drag and drop between panels + +KWin handles the protocols: Wayland, X11 through Xwayland, and the portal's file transfer. Frametop has to get the pointer right between panels. + +- **Crossing panels.** When the laser moves onto another panel mid-drag, ft-screens gives that panel's KWin window pointer focus. KWin puts its cursor at that output's position, and the drop target gets enter and motion events. Floating windows add nothing new here, but they make gaps between panels the normal case. +- **Gaps.** While the laser is between panels, none of our overlays get its events. If you let go in empty space, the release never reaches KWin, and the drag or move stays stuck until the next click. The fix: while a button is held on a Frametop panel and the laser leaves all of them, ft-screens puts an invisible catcher overlay on the laser (the tear-off ghost is the same thing with a picture on it). A release on the catcher releases in KWin wherever the pointer last was. Dropping in a gap cancels, just as dropping outside any window does. The pointer helper also tells ft-screens when the mouse's left button comes up, in case the catcher misses it. This should also fix window moves that end off a panel today (to check). +- **The 3D mouse's drag lock.** While the button is held, the drag lock keeps the cursor at its distance and stops hit tests. So a drag onto a nearer panel passes behind it, and a farther panel works only if SteamVR's laser happens to reach it. The change: while the button is held, keep testing the other panels (not the one pressed on) and move onto a panel the ray meets. Off the edge of the pressed panel, keep that panel's plane, as now, so moves and resizes past the edge still work. +- **Drag icon.** KWin 6 draws the drag icon as part of its scene, so it should show on the panel under the laser (to check). In a gap it stops at the source panel's edge. A later version can show a small drag proxy on the catcher. +- **Flatpak apps.** Dropping files into a sandboxed app goes through the document portal, the same path that the session script's file-picker fix covers. Test it explicitly, for example Dolphin to Brave. + +## Things that must keep working + +- **Typing follows the last click.** A click on a floating panel counts as a click on the desktop, since the window is a KWin window. +- **VR games.** Floating windows follow the screens' rules: they're hidden during a game unless the dashboard is open, and controllers' lasers are off in games. +- **Headset standby.** Nothing new may poll SteamVR with new clients, so no new `vrcmd` loops. +- **The pointer helper's overlay list.** The helper learns about overlays by running `vrcmd --overlays` in the background, so a new floating panel appears in its next listing. Check the delay after a tear-off. If it's too long, ft-screens can send the helper new overlay keys directly. +- **Plasma.** Each enabled floating output gets a desktop view (wallpaper) under its window. Plasma doesn't add panels to new outputs by default. A floating output must never become primary. With spare outputs, the output count stays the same, which avoids the lost-taskbar problem in design.md's open questions. +- **Show Desktop.** Meta+D would hide floating windows. The script can leave them out, or we accept it. +- **Restarting the desktop** closes every window, floating ones included. Each app's placement is remembered, so an app launched floating again comes back where it was. + +## Plan + +### Phase 0: find out + +Each item has a pass condition. Items that need a desktop restart with extra outputs wait until the headset is free, or run in the headless test mode from 0.1. + +- **0.1 Headless test mode.** `ft-screens --no-vr` runs the compositor without SteamVR. It logs toplevels, titles, and sizes, and answers commands on a separate control socket name. This lets KWin and output experiments run without the headset, and without touching the running desktop. +- **0.2 Outputs.** Start KWin with spare outputs, then disable and re-enable one with `kscreen-doctor`. Pass: we know whether its toplevel is destroyed and recreated or stays. Also confirm that the title carries the output name, that resizing a spare through configure works, and that outputs with gaps between them are accepted. Fallback test: `stream_virtual_output` with no PipeWire consumer creates a toplevel and doesn't crash. +- **0.3 KWin script API on 6.2.5.** Move signals fire for moves from both KWin's title bars and apps' own. `sendClientToScreen` and `setMaximize` work on another output, the `callDBus` long poll works, and `registerUserActionsMenu` works. Also check whether popups show up in `windowAdded` (needed for the crop later). The observers can load into the running desktop through `org.kde.kwin.Scripting` and move nothing, so this is safe while the headset is in use. +- **0.4 Frozen-pointer move.** Pass: a KWin move on a maximized window with no pointer movement neither unmaximizes nor shifts it. +- **0.5 SteamVR's laser.** Find out which overlay gets MouseMove and ButtonUp when a held laser moves from overlay A to overlay B, and when it's released over nothing, for both a controller and the 3D mouse. Pass: an interactive overlay placed on the laser reliably catches the release. +- **0.6 Texture bounds.** Pass: `SetOverlayTextureBounds` crops a DMA-BUF (`SharedTextureHandle`) overlay correctly, and mouse positions map to the cropped area. + +### Phase 1: float a window from its menu + +Build the KWin script, ft-floatd, and floating panels in ft-screens: controls, moving by the title bar, resizing, close, and back to desktop. Add drag and drop across panels, with the catcher and the pointer helper change. + +Done when: + +- Dolphin and Kate float from "Float in VR". +- A file drags from the floating Dolphin to the floating Kate, to a screen, and back. +- The clipboard works between them. +- Closing and docking give the output back. + +### Phase 2: tear off and dock by dragging + +The ghost and tear-off, and the dock highlight with push-flush docking. + +### Phase 3: launch floating + +`ft-float run` and `ft-float launch`, window matching, the Frametop Apps launcher entry, floats-only mode, the picker, and remembered placement. + +### Phase 4: polish + +Popups past the window's edges (pad and crop), the drag proxy, bringing a window into view when it's activated, the Display Settings section, and docs (design.md, reference.md, and the Use table in the README). + +## Open questions + +1. **Dock gesture.** Push flush (recommended), hover and wait, or the button only? +2. **Visibility.** Should floating windows follow the hide hotkey and the visibility modes like the screens, or have their own setting? +3. **Title bars.** Keep them on floating windows (recommended), or go borderless with only Frametop's controls? +4. **Standalone entry.** One Frametop Apps entry (recommended), or add each app to Launch a program? +5. **Count.** How many windows may float at once? This matters only if virtual outputs don't work out (default 6 spares). +6. **Windows opened by a floating app.** Float them too, or open them on the screens? Dialogs float either way. + +## Risks + +- If the nested backend can't disable outputs, parked spares still work, but each one keeps a Plasma desktop view. +- A SteamOS update can change KWin's script API or its nested backend. The script and the output handling are the parts to recheck after one. +- GPU memory: each floating output has its own swapchain of two or three buffers. A 1600 × 1000 window is about 6.4 MB per buffer. +- Frame pacing with many panels hasn't been measured (already an open question in design.md). Each output is a separate render pass in KWin. From 389878b9fd148c960a2058af95e264819caa37de Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:01:46 -0600 Subject: [PATCH 02/13] Settle the floating-windows design with the user Co-Authored-By: Claude Opus 5.5 --- docs/floating-windows.md | 132 ++++++++++++++++++++++++--------------- 1 file changed, 83 insertions(+), 49 deletions(-) diff --git a/docs/floating-windows.md b/docs/floating-windows.md index 6af6d5f..df0c9d5 100644 --- a/docs/floating-windows.md +++ b/docs/floating-windows.md @@ -1,13 +1,43 @@ # Floating windows (plan) -Status: plan only, on the `floating-windows` branch. Nothing here is built yet. +Status: design settled 2026-09-29 (see "Decisions"); being built on the `floating-windows` branch. The goal is to let any desktop app float in VR in a panel of its own, like SteamVR's floating windows, while it stays part of the Frametop desktop. That means drag and drop, the clipboard, and focus keep working between floating windows and the screens. - There are two ways to get a floating window. Launch the app floating, or drag a desktop window by its title bar off a screen and let go in the air. -- There are two ways to put one back. Drag it onto a screen, or press its "back to desktop" button. +- There are two ways to put one back. Push it flush against a screen and let go, or press its "back to desktop" button. - Files, text, and images drag between any two floating windows, and between floating windows and the screens. +## Decisions + +Settled with the user on 2026-09-29. The sections below follow them. + +| # | Question | Decision | +|---|---|---| +| 1 | How many windows can float at once | 8 spare outputs by default, configurable (`FLOAT_SLOTS`, and Display Settings); a change needs a desktop restart | +| 2 | Menus and dropdowns | Each floating output has a margin around the window. The panel shows only the window, and each open popup gets a small overlay of its own, cut from the same buffer | +| 3 | Tearing off | Drag the title bar past a screen's edge and let go in the air, with a small dead zone past the edge | +| 4 | Docking by dragging | Push the window flush against a screen (within about 10 cm), with the landing spot highlighted, and let go | +| 5 | Visibility | Floating windows follow the same rules as the screens: the hide hotkey, the visibility modes, and the games rule | +| 6 | Windows a floating app opens | They float too | +| 7 | Launching floating from the headset | One "Frametop Apps" launcher entry with a picker | +| 8 | Build order | The catcher first, as a fix that stands on its own; `pointer-ignore` and `layouts-headpin` merged before phase 1; the hand cutouts stay out | +| 9 | Show Desktop (Meta+D) | Floating windows stay | +| 10 | Frametop Apps and visibility | The entry starts the desktop with each screen hidden on its own (the existing per-screen hide), so only floating windows show. No special mode | +| 11 | Window frame | KWin's title bar and border stay. Frametop's bar, close, and "back to desktop" are extras | +| 12 | Resizing | The window's own edges and Frametop's corner tab both change the size in pixels at the same density; the output follows | +| 13 | Margin | 300 px on each side, configurable | +| 14 | All spares in use | The window opens on the screens, with a notification | +| 15 | Where a floating app's new windows go | Where that app's windows went last time; otherwise to the parent's right, curving around you | +| 16 | Where the code is written | A branch in the PC's clone of the repo | +| 17 | Size when docked by dragging | The current floating size in pixels, shrunk to fit the screen | +| 18 | Bigger text | A scale for each window (KWin's output scale): Meta+scroll over the window, or +/- on its bar. Remembered for each app | +| 19 | Switching to a window you can't see | It's focused, and a glow at the edge of your view points to it. Moving it in front of you is a setting | +| 20 | Full screen | The window fills its own panel. The margin drops to zero while it's full screen, and the panel keeps its size and place | +| 21 | Named layouts | They cover the screens only. Floating windows use the placement remembered for each app | + +Also assumed: floating windows get the wrist pin, the head pin, and pass-through (`pointer-ignore`) like screens. Every gesture works with the controllers as well as the 3D mouse. A window launched floating uses the primary screen's density. VNC shows only the primary screen, as now. Anything that restarts the live desktop waits for the user's OK. + ## The approach: each floating window gets a KWin output of its own Drag and drop and the clipboard only work between windows of the same compositor. A Wayland window can't move from one compositor to another. So a floating window has to stay a KWin window. @@ -17,18 +47,22 @@ ft-screens already shows each KWin output as a panel. It sets the output's size Alternatives we considered: - **Run floating apps directly on ft-screens.** It's a wlroots compositor, so apps could connect to it and get a panel per window. But they would get no drag and drop or clipboard with desktop apps unless we wrote a bridge. Also, a window that's already on the desktop could never be torn off, because a Wayland client can't change compositors. Rejected. -- **One large hidden "canvas" output.** Every floating window would sit on one big output, and each panel would show a crop of it (`SetOverlayTextureBounds`). That needs only one extra output, with no copies, and popups could extend past their window. But an 8K canvas uses about 128 MB per buffer, with two or three buffers in KWin's swapchain. It would also have to repack windows whenever one resized. Maximize and fullscreen would fill the whole canvas, and every window would share one scale. This is the fallback if per-window outputs don't work. +- **One large hidden "canvas" output.** Every floating window would sit on one big output, and each panel would show a crop of it (`SetOverlayTextureBounds`). That needs only one extra output, with no copies. But an 8K canvas uses about 128 MB per buffer, with two or three buffers in KWin's swapchain. It would also have to repack windows whenever one resized, full screen would fill the whole canvas, and every window would share one scale. This is the fallback if per-window outputs don't work. - **Screencast single windows** (`zkde_screencast` `stream_window`, over PipeWire). This adds copies and latency, and the window still needs a real place in KWin's layout to receive input. Rejected. - **SteamOS's own floating windows** (Launch a program from the dashboard). Those apps run in gamescope, outside KWin, so they can't drag and drop with the desktop. -### Where the extra outputs come from +### Where the extra outputs come from: spare outputs -KWin's nested backend opens its outputs at start (`--output-count`). There are two ways to get more: +KWin's nested backend opens its outputs at start (`--output-count`). The session starts KWin with the screen count plus `FLOAT_SLOTS` outputs (default 8). Each spare is disabled until it's needed, with `kscreen-doctor` (or in the session's `kwinoutputconfig.json`, so it starts disabled). Floating a window enables a spare, and docking the window disables it again. `FLOAT_SLOTS` limits how many windows can float at once, and changing it means restarting the desktop. -1. **Spare outputs (try first).** Start KWin with the screen count plus `FLOAT_SLOTS` outputs (default 6). Park each spare output until it's needed. If the nested backend supports it, disable the spare with `kscreen-doctor`. Otherwise make it tiny (64 × 64), put it far away in KWin's layout, and don't show its panel. Floating a window takes a spare, and docking the window gives the spare back. `FLOAT_SLOTS` limits how many windows can float at once, and changing it means restarting the desktop. -2. **Virtual outputs (fallback).** KWin 6.2.5's nested backend implements `createVirtualOutput` and `removeVirtualOutput` (checked in `libkwin.so`). A client reaches them through `zkde_screencast_unstable_v1`'s `stream_virtual_output` request. There's no limit on count, and the output gets whatever name we choose. But that request belongs to a screencast protocol, and only privileged clients may use it. A helper in the container can't be matched to a desktop file, so we would need `KWIN_WAYLAND_NO_PERMISSION_CHECKS=1`. That turns off the permission checks for every client in the session. design.md also records a KWin 6.2.5 crash (`textureForOutput`) when such an output was streamed. It might be safe if nothing ever consumes the PipeWire stream, but that needs a test. +Checked in KWin 6.2.5's source (`src/backends/wayland/`, 2026-09-29): -ft-screens has to tell a floating window's output from a screen's. KWin titles each nested output window `KDE Wayland Compositor %1`. If `%1` turns out to be the output name (to confirm), ft-screens can read the title. Otherwise it can go by creation order, since spare outputs come after the screens. +- Disabling a nested output keeps its host window. `Output::applyChanges` only flips `enabled`, KWin stops rendering it, and Plasma drops its desktop view. So ft-screens keeps the same toplevel, and its screen numbers stay put. +- Each output's host window is titled `KDE Wayland Compositor WL-`, with `- Output disabled` appended while it's disabled (`WaylandOutput::updateWindowTitle`, on every `enabledChanged`). ft-screens reads the title to tell screens (`WL-0` to `WL-`) from spares, and to see a spare turn on and off. +- Pointer positions reach KWin only through motion events: the output's position in the layout plus the position on its window. When ft-screens stops sending motion, KWin's pointer stays put. +- **Virtual outputs don't work.** `createVirtualOutput` makes an output window but never adds it to the backend's `m_outputs`, so `findOutput()` returns null when the pointer enters it, and the next line dereferences it (`Q_ASSERT` is compiled out). A click on such a panel would crash KWin. This rules out the virtual-output fallback (`stream_virtual_output`) without a patched KWin. + +ft-screens creates a `screen` for each toplevel in the order they appear, and indexes its settings by that order. Spares come after the screens, so they get indices `SCREENS` and up. Their panels are hidden while their output is disabled. ## How the parts fit together @@ -38,55 +72,59 @@ ft-screens has to tell a floating window's output from a screen's. KWin titles e runs commands ◀─ long poll ─ spare outputs (kscreen-doctor) ◀─ @frametop_float ─ lasers, ghost, catcher ``` -- **KWin script `frametop-float`** (JavaScript). A script keeps working across KWin updates. A C++ effect would have to match the host's exact KWin build, and our build container is Fedora, not SteamOS. The script watches windows (`windowAdded`/`windowRemoved`, `interactiveMoveResizeStarted`/`Stepped`/`Finished`, `outputChanged`, `minimizedChanged`, `windowActivated`). It runs commands: move a window to an output, maximize it, put it on all virtual desktops, and restore it. It adds "Float in VR" to the window menu (`registerUserActionsMenu`) and registers a shortcut (`registerShortcut`, Meta+Shift+F). KWin scripts can call D-Bus but can't serve it, so commands come back through a long poll. The script calls ft-floatd's `NextCommand`, which answers when a command is ready, and then the script calls it again. The fallback is loading one-shot scripts through `org.kde.kwin.Scripting`, the way kdotool does. All of these API names are present in the host's KWin 6.2.5. -- **ft-floatd** (Python). The host has dbus-python and PyGObject. It owns `org.frametop.Float` on the session's private bus, and it keeps the table of which window is on which output and panel. It hands out spare outputs and sets their size, scale, and position with `kscreen-doctor`, as ft-layout does. It tells ft-screens where each floating window goes and tells the script which window goes where. It also remembers each app's placement, keyed by desktop file name. -- **ft-screens.** `Screen` becomes a panel with a kind: screen or floating window. Floating panels get the same bar, curve, roll, resize tab, and wrist pin, plus close and "back to desktop" buttons. New parts are the tear-off ghost, the catcher, carrying a panel during a KWin move, and the dock target highlight. `MAX_SCREENS` goes from 8 to 16. Commands arrive on `@ft_screens`. Events go out to `@frametop_float` from an unbound socket, the same way ft-screens talks to the input relay. -- **Session script.** Adds the spare outputs to the count, starts ft-floatd, and enables the KWin script in the session's `kwinrc`. -- **ft-layout.** Arranges only the screens' outputs. Today it arranges everything in `kscreen-doctor -j`, so it has to skip floating windows' outputs. +- **KWin script `frametop-float`** (JavaScript). A script keeps working across KWin updates. A C++ effect would have to match the host's exact KWin build, and our build container is Fedora, not SteamOS. The script watches windows (`windowAdded`/`windowRemoved`, `interactiveMoveResizeStarted`/`Stepped`/`Finished`, `outputChanged`, `minimizedChanged`, `windowActivated`, `fullScreenChanged`). It runs commands: move a window to an output, set its geometry, put it on all virtual desktops, and restore it. It adds "Float in VR" to the window menu (`registerUserActionsMenu`) and registers a shortcut (`registerShortcut`, Meta+Shift+F). KWin scripts can call D-Bus but can't serve it, so commands come back through a long poll. The script calls ft-floatd's `NextCommand`, which answers when a command is ready, and then the script calls it again. The fallback is loading one-shot scripts through `org.kde.kwin.Scripting`, the way kdotool does. All of these API names are present in the host's KWin 6.2.5. +- **ft-floatd** (Python). The host has dbus-python and PyGObject. It owns `org.frametop.Float` on the session's private bus, and it keeps the table of which window is on which output and panel. It enables and disables spare outputs and sets their size, scale, and position with `kscreen-doctor`, as ft-layout does. It tells ft-screens where each floating window goes and tells the script which window goes where. It also remembers each app's placement and scale, keyed by desktop file name. +- **ft-screens.** `Screen` becomes a panel with a kind: screen or floating window. Floating panels get the same bar, curve, roll, resize tab, and wrist and head pins, plus close, "back to desktop", and scale buttons. New parts are the tear-off ghost, the catcher, popup overlays, carrying a panel during a KWin move, and the dock target highlight. `MAX_SCREENS` goes from 8 to 16. Commands arrive on `@ft_screens`. Events go out to `@frametop_float` from an unbound socket, the same way ft-screens talks to the input relay. +- **Session script.** Adds `FLOAT_SLOTS` to the output count, starts ft-floatd, and enables the KWin script in the session's `kwinrc`. +- **ft-layout.** Arranges only the screens' outputs. Today it arranges everything in `kscreen-doctor -j`, so it has to skip the spares (`WL-` and up), enabled or not. - **ft-pointer.** Changes to the drag lock (see "Drag and drop between panels"). -- **Frametop Display Settings.** Gets a Floating windows section. +- **Frametop Display Settings.** Gets a Floating windows section: slots, margin, and "bring a window in front of you when it's activated". Program names stay within 15 characters (`ft-floatd`). Overlay keys are `frametop.float.N` and `frametop.float.N.bar`, and so on. ## A floating window -- **Size and scale.** Its output is the window's size, at the scale of the screen it came from. The panel's width is its pixel width times that screen's metres per pixel, so text stays the same size in VR. -- **Window state.** The window is maximized on its output and set to show on all virtual desktops. It keeps its title bar: Breeze drops the borders of a maximized window but keeps the title bar. Apps that draw their own title bar (GTK, Chromium) keep theirs. -- **Moving.** Press the title bar. KWin starts an interactive move and the script reports it. ft-screens then stops forwarding pointer motion to KWin, so KWin's pointer stays at the press point, the window moves by nothing, and it doesn't come out of maximize. Meanwhile ft-screens carries the panel with the pressing device, the same way the bar does today: it follows rigidly, scroll pushes and pulls, and the 3D mouse's right-drag tilts. When the button comes up, KWin gets the release at the press point. The bar under the panel works too. -- **Resizing.** The corner tab changes the window's size in pixels at the same density, so the app lays itself out again. A screen's tab only scales the panel. Resizing is throttled to about 20 updates a second, with a minimum of 320 × 200, like screens. +- **Output and margin.** Its output is the window's frame plus a margin on each side (`FLOAT_MARGIN`, default 300 px). KWin keeps a Wayland popup inside its parent's output (`XdgPopupWindow::updateRelativePlacement` uses the output's placement area), so the margin gives menus and dropdowns room past the window's edges. X11 apps place their own menus within the monitor, so the same applies. Enabled spares sit apart from the screens and from each other in KWin's layout, so nothing spills from one to the next. Memory: a 1600 × 1000 window with a 300 px margin is about 14 MB per buffer, 42 MB for three. +- **What the panel shows.** Only the window's frame: ft-screens crops the output's buffer with `SetOverlayTextureBounds` and maps mouse positions through the crop. Each open popup gets a small overlay of its own, cut from the same buffer and placed a few millimetres in front of the window, so the main panel never changes size. KWin tells scripts about popups as windows of their own (`windowAdded` with `popupWindow`), so the script reports their rectangles. +- **Size and scale.** The panel's width is the window's pixel width times the source screen's metres per pixel, so text stays the same size in VR. A window launched floating uses the primary screen's density. Each window also has a scale (KWin's output scale), changed with Meta+scroll over the window or +/- on its bar and remembered for each app. A bigger scale makes the content bigger at the same panel size. +- **Window state.** An ordinary window, not maximized, placed inside its output with the margin around it, and set to show on all virtual desktops. It keeps its title bar and border. Apps that draw their own title bar (GTK, Chromium) keep theirs. +- **Moving.** Press the title bar. KWin starts an interactive move and the script reports it. ft-screens then stops forwarding pointer motion to KWin, so KWin's pointer stays at the press point and the window moves by nothing. Meanwhile ft-screens carries the panel with the pressing device, the same way the bar does today: it follows rigidly, scroll pushes and pulls, and the 3D mouse's right-drag tilts. When the button comes up, KWin gets the release at the press point. The bar under the panel works too. +- **Resizing.** The window's own edges (inside the margin, so KWin's resize works as on the desktop) and Frametop's corner tab both change the window's size in pixels at the same density, so the app lays itself out again. ft-floatd resizes the output to keep the margin, and the panel grows or shrinks around the window's top-left corner. A screen's tab only scales the panel. Resizing is throttled to about 20 updates a second, with a minimum of 320 × 200, like screens. +- **Full screen.** The window fills its own panel: the margin drops to zero while it's full screen, and the output is the panel's size in pixels. The panel keeps its size and place. On leaving full screen, the margin comes back. - **Buttons.** The close button closes the window. "Back to desktop" docks it where it came from. -- **Minimize.** Minimizing, from the title bar or the taskbar, hides the panel, and restoring it shows the panel again. Floating windows stay in the desktop's taskbar and in Alt+Tab. If one is activated while it's out of view, it can come around in front of you (optional). -- **New windows.** A dialog of a floating window (`transientFor`) floats in front of its parent. Other new windows open on the screens: if KWin puts a new window on a floating window's output, the script moves it to the screen used last. -- **Popups.** KWin keeps menus and tooltips inside their output, so a menu can't reach past the window's edges. The first version accepts that. A later version pads the output and crops the panel to the window plus any open popups (`SetOverlayTextureBounds`, with mouse positions offset by the crop). +- **Minimize.** Minimizing, from the title bar or the taskbar, hides the panel, and restoring it shows the panel again. Floating windows stay in the desktop's taskbar and in Alt+Tab. +- **Activated out of view.** When a floating window is activated (taskbar, Alt+Tab, a notification) and it's more than about 60° from where you're looking, it's focused and a glow at the edge of your view points to it. A setting moves it in front of you instead. +- **New windows.** A dialog of a floating window (`transientFor`) floats in front of its parent. Other new windows of a floating app float too: where that app's windows went last time, otherwise to the parent's right at the same distance, curving around you, and to its left if that's taken. When every spare is in use, the window opens on the screen used last and a notification says so. +- **Show Desktop.** Meta+D leaves floating windows alone. ## Tearing a window off a screen 1. Press a desktop window's title bar and drag it. KWin starts a move, and the script tells ft-floatd, which tells ft-screens: `move-start `. -2. While the button is held, the laser leaves every Frametop panel. A second trigger is also possible: scrolling toward you while dragging pulls the window out of the screen. +2. While the button is held, the laser leaves every Frametop panel by more than a small dead zone (a few centimetres past the edge). Letting go inside the dead zone is an ordinary drop. 3. ft-screens shows a ghost: an overlay showing the screen's live buffer cropped to the window (`SetOverlayTextureBounds`, no copy). It's at the screen's pixel density and distance, on the laser, facing you, with the point you grabbed under the laser. The ghost takes mouse input, so SteamVR's laser lands on it and the release comes to ft-screens. 4. Go back onto a screen before letting go, and the ghost disappears. It's an ordinary move again. -5. Let go on the ghost, and ft-screens releases the button in KWin, which ends the move. It then reports the tear-off to ft-floatd, with the window, the ghost's pose, and the density. ft-floatd takes a spare output and has ft-screens size it to the window and put its panel at the ghost's pose. Then it has the script move the window onto that output and maximize it. The ghost stays until the new panel's first frame at the right size arrives, so nothing blinks. +5. Let go on the ghost, and ft-screens releases the button in KWin, which ends the move. It then reports the tear-off to ft-floatd, with the window, the ghost's pose, and the density. ft-floatd enables a spare output and has ft-screens size it to the window plus the margin and put its panel at the ghost's pose. Then it has the script move the window onto that output. The ghost stays until the new panel's first frame at the right size arrives, so nothing blinks. ## Putting it back - **Button.** "Back to desktop" returns the window to the screen, position, and size it had before it was torn off (saved at tear-off). -- **Dragging.** Carry the floating window, by its title bar or its bar, until the spot you're pointing at is on a screen. Then push it flush with the screen, within about 10 cm of its surface: scroll away with the mouse, or move the controller forward. The screen shows where the window will land, and letting go docks it. A carried panel keeps its distance, so moving a floating window in front of a screen never docks it by accident. -- Docking gives the output back (disabled or parked) and removes the panel. +- **Dragging.** Carry the floating window, by its title bar or its bar, until the spot you're pointing at is on a screen. Then push it flush with the screen, within about 10 cm of its surface: scroll away with the mouse, or move the controller forward. The screen shows where the window will land, and letting go docks it there at its current size in pixels, shrunk to fit if the screen is smaller. A carried panel keeps its distance, so moving a floating window in front of a screen never docks it by accident. +- Docking disables the output and removes the panel. ## Launching an app floating - **In the desktop.** Use "Float in VR" in any window's menu, or press Meta+Shift+F for the active window. - **From a command.** `ft-float run ` and `ft-float launch ` start an app and float its first window. ft-floatd records the process it started, and the script matches new windows by PID, including child processes. Some single-instance apps (Firefox, D-Bus-activated apps) open the window from a process that was already running. Those are matched by desktop file name within a few seconds, or by `XDG_ACTIVATION_TOKEN` where the app honors it. -- **From the headset without the desktop open.** Add one new launcher entry, "Frametop Apps", instead of one entry per app, since per-app entries would need setting up for every new app. The entry starts the Frametop session with the screens hidden (floats-only mode) and opens an app picker as a floating window. Picking an app launches it floating. The screens are still there: Meta+Shift+H shows them. Everything runs in one session, so dragging between a standalone app and a desktop app works. If the desktop is already running, the entry just opens the picker. +- **From the headset without the desktop open.** One new launcher entry, "Frametop Apps". It starts the Frametop session with each screen hidden on its own (the per-screen hide that already exists), and opens an app picker as a floating window. Picking an app launches it floating. The hide hotkey and visibility modes still apply to everything, and a screen comes back with one click. Everything runs in one session, so dragging between a standalone app and a desktop app works. If the desktop is already running, the entry just opens the picker. - **The picker.** Either KRunner, floated, or a small Kirigami app like the settings apps, with a grid of apps and their icons. -- **Visibility in floats-only mode.** The screens stay hidden and floating windows show. See open question 2 for how floating windows follow the visibility modes otherwise. +- **Remembered placement.** Each app's last floating pose, size, and scale, keyed by desktop file name. Named layouts don't include floating windows. ## Drag and drop between panels KWin handles the protocols: Wayland, X11 through Xwayland, and the portal's file transfer. Frametop has to get the pointer right between panels. - **Crossing panels.** When the laser moves onto another panel mid-drag, ft-screens gives that panel's KWin window pointer focus. KWin puts its cursor at that output's position, and the drop target gets enter and motion events. Floating windows add nothing new here, but they make gaps between panels the normal case. -- **Gaps.** While the laser is between panels, none of our overlays get its events. If you let go in empty space, the release never reaches KWin, and the drag or move stays stuck until the next click. The fix: while a button is held on a Frametop panel and the laser leaves all of them, ft-screens puts an invisible catcher overlay on the laser (the tear-off ghost is the same thing with a picture on it). A release on the catcher releases in KWin wherever the pointer last was. Dropping in a gap cancels, just as dropping outside any window does. The pointer helper also tells ft-screens when the mouse's left button comes up, in case the catcher misses it. This should also fix window moves that end off a panel today (to check). +- **Gaps (the catcher).** While the laser is between panels, none of our overlays get its events, and ft-screens clears pointer focus on `FT_LEAVE` even with a button held. If you let go in empty space, the release never reaches KWin, and the drag or move stays stuck until the next click. The fix: while a button is held on a Frametop panel and the laser leaves all of them, ft-screens puts an invisible catcher overlay on the laser (the tear-off ghost is the same thing with a picture on it). A release on the catcher releases in KWin wherever the pointer last was. Dropping in a gap cancels, just as dropping outside any window does. The pointer helper also tells ft-screens when the mouse's left button comes up, in case the catcher misses it. This also fixes window moves and drags that end off a panel today. - **The 3D mouse's drag lock.** While the button is held, the drag lock keeps the cursor at its distance and stops hit tests. So a drag onto a nearer panel passes behind it, and a farther panel works only if SteamVR's laser happens to reach it. The change: while the button is held, keep testing the other panels (not the one pressed on) and move onto a panel the ray meets. Off the edge of the pressed panel, keep that panel's plane, as now, so moves and resizes past the edge still work. - **Drag icon.** KWin 6 draws the drag icon as part of its scene, so it should show on the panel under the laser (to check). In a gap it stops at the source panel's edge. A later version can show a small drag proxy on the catcher. - **Flatpak apps.** Dropping files into a sandboxed app goes through the document portal, the same path that the session script's file-picker fix covers. Test it explicitly, for example Dolphin to Brave. @@ -94,36 +132,42 @@ KWin handles the protocols: Wayland, X11 through Xwayland, and the portal's file ## Things that must keep working - **Typing follows the last click.** A click on a floating panel counts as a click on the desktop, since the window is a KWin window. -- **VR games.** Floating windows follow the screens' rules: they're hidden during a game unless the dashboard is open, and controllers' lasers are off in games. +- **Visibility.** Floating windows follow the screens' rules: the hide hotkey, the visibility modes, and hiding during a VR game unless the dashboard is open. Controllers' lasers are off in games. - **Headset standby.** Nothing new may poll SteamVR with new clients, so no new `vrcmd` loops. - **The pointer helper's overlay list.** The helper learns about overlays by running `vrcmd --overlays` in the background, so a new floating panel appears in its next listing. Check the delay after a tear-off. If it's too long, ft-screens can send the helper new overlay keys directly. -- **Plasma.** Each enabled floating output gets a desktop view (wallpaper) under its window. Plasma doesn't add panels to new outputs by default. A floating output must never become primary. With spare outputs, the output count stays the same, which avoids the lost-taskbar problem in design.md's open questions. -- **Show Desktop.** Meta+D would hide floating windows. The script can leave them out, or we accept it. +- **Plasma.** An enabled floating output gets a desktop view (wallpaper) under its window, hidden by the crop. Plasma doesn't add panels to new outputs by default. A floating output must never become primary. With spare outputs, the output count stays the same, which avoids the lost-taskbar problem in design.md's open questions. - **Restarting the desktop** closes every window, floating ones included. Each app's placement is remembered, so an app launched floating again comes back where it was. ## Plan +### Step 1: the catcher and the branches + +- The catcher (see "Gaps"), as a fix that stands on its own, so it can go to `main` by itself. +- Merge `pointer-ignore` and `layouts-headpin`. + ### Phase 0: find out Each item has a pass condition. Items that need a desktop restart with extra outputs wait until the headset is free, or run in the headless test mode from 0.1. - **0.1 Headless test mode.** `ft-screens --no-vr` runs the compositor without SteamVR. It logs toplevels, titles, and sizes, and answers commands on a separate control socket name. This lets KWin and output experiments run without the headset, and without touching the running desktop. -- **0.2 Outputs.** Start KWin with spare outputs, then disable and re-enable one with `kscreen-doctor`. Pass: we know whether its toplevel is destroyed and recreated or stays. Also confirm that the title carries the output name, that resizing a spare through configure works, and that outputs with gaps between them are accepted. Fallback test: `stream_virtual_output` with no PipeWire consumer creates a toplevel and doesn't crash. -- **0.3 KWin script API on 6.2.5.** Move signals fire for moves from both KWin's title bars and apps' own. `sendClientToScreen` and `setMaximize` work on another output, the `callDBus` long poll works, and `registerUserActionsMenu` works. Also check whether popups show up in `windowAdded` (needed for the crop later). The observers can load into the running desktop through `org.kde.kwin.Scripting` and move nothing, so this is safe while the headset is in use. -- **0.4 Frozen-pointer move.** Pass: a KWin move on a maximized window with no pointer movement neither unmaximizes nor shifts it. +- **0.2 Outputs.** Start KWin with spare outputs, then disable and re-enable one with `kscreen-doctor`. Pass: the toplevel stays and its title changes (as the source says), a disabled output costs no frames, resizing a spare through configure works, and outputs with gaps between them are accepted. Also: an output larger than its window with the window placed inside, and a popup placed in the margin. +- **0.3 KWin script API on 6.2.5.** Move signals fire for moves from both KWin's title bars and apps' own. `sendClientToScreen` and `frameGeometry` work on another output, the `callDBus` long poll works, and `registerUserActionsMenu` works. Popups show up in `windowAdded` with their geometry. The observers can load into the running desktop through `org.kde.kwin.Scripting` and move nothing, so this is safe while the headset is in use. +- **0.4 Frozen-pointer move.** Pass: a KWin move with no pointer movement doesn't shift the window. - **0.5 SteamVR's laser.** Find out which overlay gets MouseMove and ButtonUp when a held laser moves from overlay A to overlay B, and when it's released over nothing, for both a controller and the 3D mouse. Pass: an interactive overlay placed on the laser reliably catches the release. -- **0.6 Texture bounds.** Pass: `SetOverlayTextureBounds` crops a DMA-BUF (`SharedTextureHandle`) overlay correctly, and mouse positions map to the cropped area. +- **0.6 Texture bounds.** Pass: `SetOverlayTextureBounds` crops a DMA-BUF (`SharedTextureHandle`) overlay correctly, mouse positions map to the cropped area, and two overlays can show different crops of one buffer. ### Phase 1: float a window from its menu -Build the KWin script, ft-floatd, and floating panels in ft-screens: controls, moving by the title bar, resizing, close, and back to desktop. Add drag and drop across panels, with the catcher and the pointer helper change. +Build the KWin script, ft-floatd, and floating panels in ft-screens: the margin and popup overlays, controls, moving by the title bar, resizing (edges and tab), scale, full screen, close, and back to desktop. Add drag and drop across panels, with the pointer helper change. Done when: - Dolphin and Kate float from "Float in VR". - A file drags from the floating Dolphin to the floating Kate, to a screen, and back. - The clipboard works between them. +- A menu near a floating window's edge opens past the edge. - Closing and docking give the output back. +- Frame pacing and GPU memory are measured with several floating windows. ### Phase 2: tear off and dock by dragging @@ -131,24 +175,14 @@ The ghost and tear-off, and the dock highlight with push-flush docking. ### Phase 3: launch floating -`ft-float run` and `ft-float launch`, window matching, the Frametop Apps launcher entry, floats-only mode, the picker, and remembered placement. +`ft-float run` and `ft-float launch`, window matching, new windows of floating apps, the Frametop Apps launcher entry, the picker, remembered placement, and the notification when every spare is in use. ### Phase 4: polish -Popups past the window's edges (pad and crop), the drag proxy, bringing a window into view when it's activated, the Display Settings section, and docs (design.md, reference.md, and the Use table in the README). - -## Open questions - -1. **Dock gesture.** Push flush (recommended), hover and wait, or the button only? -2. **Visibility.** Should floating windows follow the hide hotkey and the visibility modes like the screens, or have their own setting? -3. **Title bars.** Keep them on floating windows (recommended), or go borderless with only Frametop's controls? -4. **Standalone entry.** One Frametop Apps entry (recommended), or add each app to Launch a program? -5. **Count.** How many windows may float at once? This matters only if virtual outputs don't work out (default 6 spares). -6. **Windows opened by a floating app.** Float them too, or open them on the screens? Dialogs float either way. +The drag proxy, the glow toward a window activated out of view (and the setting to bring it in front), the Display Settings section, and docs (design.md, reference.md, and the Use table in the README). ## Risks -- If the nested backend can't disable outputs, parked spares still work, but each one keeps a Plasma desktop view. - A SteamOS update can change KWin's script API or its nested backend. The script and the output handling are the parts to recheck after one. -- GPU memory: each floating output has its own swapchain of two or three buffers. A 1600 × 1000 window is about 6.4 MB per buffer. +- GPU memory: each floating output has its own swapchain of two or three buffers, including the margin. The Frame has 16 GB shared, with about 4 GB free in normal use (2026-09-29). - Frame pacing with many panels hasn't been measured (already an open question in design.md). Each output is a separate render pass in KWin. From 0a88a6b7c906ed24f2c59db915c914279d92d491 Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:05:54 -0600 Subject: [PATCH 03/13] Release a button held on a screen even when the laser lets go between panels While a button is held on a screen, the laser leaving it no longer takes KWin's pointer, and an invisible catcher overlay sits on the laser whenever it's off every panel, so the release reaches KWin at the pointer's last spot. The pointer helper also reports the mouse's left release as a backstop. Not yet tested in the headset. Co-Authored-By: Claude Opus 5.5 --- pointer/helper/ft-pointer.cpp | 5 ++ screens/vr.cpp | 149 ++++++++++++++++++++++++++++++++-- 2 files changed, 149 insertions(+), 5 deletions(-) diff --git a/pointer/helper/ft-pointer.cpp b/pointer/helper/ft-pointer.cpp index dbd6cb8..bba6729 100644 --- a/pointer/helper/ft-pointer.cpp +++ b/pointer/helper/ft-pointer.cpp @@ -103,6 +103,8 @@ // press and collision is frozen, so dragging past a panel's edge (resizing, moving) // doesn't jump the cursor to free space or swap in the laser-catching dot, which made // SteamVR's resize snap back. +// A left release also goes to ft-screens ("up"), which releases a button held on its +// screens in KWin if SteamVR gave the release to some other overlay. // // Head follow (experimental, off by default; POINTER_FOLLOW=1, or the relay's "follow toggle"): the cursor // is carried by a reference direction, where the head faced when it last settled, and turns @@ -691,6 +693,9 @@ int main() { // Hold the drag pose (tilt, frozen distance) while SteamVR finishes the drop. dropHoldUntil = Clock::now() + std::chrono::milliseconds(500); SendTo(out, "ft_pointer", "btn trigger 0"); + // ft-screens releases a button held on its screens in KWin even when SteamVR hands + // the release to some other overlay (its catcher usually gets it; this is the backstop). + SendTo(out, "ft_screens", "up"); if (gazeBack) gazeOwns = true, gazeBack = false; }; // The left button, from the relay's "btn trigger", with gaze mode's held-back press (see diff --git a/screens/vr.cpp b/screens/vr.cpp index 264bea0..5011962 100644 --- a/screens/vr.cpp +++ b/screens/vr.cpp @@ -40,6 +40,8 @@ // own; also for flatscreen games, which aren't scene apps). // - during a VR game the screens hide unless the dashboard is open (g_inGames, default), // or stay visible over it; the hotkey still shows them. +// - the catcher: a button pressed on a screen is released in KWin even when the laser +// lets go between panels (UpdateCatcher). // OpenVR has no overlay-relative transforms here (openvr v2.15.6), so the bar, button, // and handle are placed whenever their screen moves. #include "vr.h" @@ -1008,6 +1010,95 @@ void Push(Screen &s, double notches) { s.dragRel = Mul(Inverse(d), p); } +// ---------------------------------------------------------------- the catcher + +// A button pressed on a screen belongs to KWin until it comes up, wherever the laser is by +// then: a window move or a drag and drop can end between panels. SteamVR sends the release +// only to an overlay under the laser, so while the pressing laser is on none of our panels, +// an invisible catcher sits on it, at the distance where it last met one, and a release +// there goes to KWin at the pointer's last spot. While a button is held, the laser leaving +// a screen doesn't take KWin's pointer away either, as with a real mouse; crossing onto +// another screen still moves it there. +struct Press { + uint32_t buttons = 0; // held, as bits (1 << (BTN_* - BTN_LEFT)) + vr::TrackedDeviceIndex_t device = kNone; // the laser that pressed them + int screen = -1; // where KWin's pointer is: the last screen the laser was on + double x = 0, y = 0; // ...and where on it, in buffer pixels + double distance = 1; // from the laser's start to the last panel it met + long upAt = -1; // the pointer helper saw left come up: release it at this tick +}; +Press g_press; +vr::VROverlayHandle_t g_catcher = vr::k_ulOverlayHandleInvalid; +bool g_catcherShown = false; + +uint32_t ButtonBit(uint32_t linuxButton) { return 1u << (linuxButton - BTN_LEFT); } + +void PressDown(vr::TrackedDeviceIndex_t dev, uint32_t button, int screen, double x, double y) { + if (!g_press.buttons) g_press.device = dev; + g_press.buttons |= ButtonBit(button); + g_press.screen = screen, g_press.x = x, g_press.y = y; +} + +// A button came up somewhere that isn't a screen (the catcher, a control, or the helper's +// word): release it in KWin where its pointer is, and once nothing is held, the laser is +// off the screens, so KWin's pointer leaves. +void ReleaseAway(uint32_t button, void (*handle)(const struct ft_event *, void *), void *data) { + if (!(g_press.buttons & ButtonBit(button))) return; + g_press.buttons &= ~ButtonBit(button); + if (!g_press.buttons) g_press.upAt = -1; + if (g_press.screen < 0) return; + ft_event e{}; + e.type = FT_BUTTON; + e.screen = g_press.screen; + e.button = button; + e.pressed = false; + e.x = g_press.x, e.y = g_press.y; + handle(&e, data); + std::printf("caught a release off the screens (button %u)\n", button); + if (g_press.buttons) return; + e = ft_event{}; + e.type = FT_LEAVE; + e.screen = g_press.screen; + handle(&e, data); +} + +void ShowCatcher(bool on) { + if (on == g_catcherShown || g_catcher == vr::k_ulOverlayHandleInvalid) return; + g_catcherShown = on; + if (on) vr::VROverlay()->ShowOverlay(g_catcher); + else vr::VROverlay()->HideOverlay(g_catcher); +} + +// Every tick: while a button is held, find what the pressing laser is on. On one of our +// panels or controls, note how far away; on none, put the catcher across it there. +void UpdateCatcher() { + Mat l; + if (!g_press.buttons || g_catcher == vr::k_ulOverlayHandleInvalid || !LaserPose(g_press.device, &l)) { + ShowCatcher(false); + return; + } + vr::VROverlayIntersectionParams_t params{}; + params.eOrigin = vr::TrackingUniverseStanding; + for (int k = 0; k < 3; ++k) params.vSource.v[k] = l.m[k][3], params.vDirection.v[k] = -l.m[k][2]; + for (auto &[i, s] : g_screens) { + if (!s.visible) continue; + for (auto o : s.All()) { + vr::VROverlayIntersectionResults_t hit; + if (vr::VROverlay()->ComputeOverlayIntersection(o, ¶ms, &hit)) { + g_press.distance = std::max(0.05, double(hit.fDistance)); + ShowCatcher(false); + return; + } + } + } + const double d = g_press.distance; + const double pt[3] = {l.m[0][3] - l.m[0][2] * d, l.m[1][3] - l.m[1][2] * d, l.m[2][3] - l.m[2][2] * d}; + const Mat m = FacingPose(pt, l); // across the laser, facing its start + vr::VROverlay()->SetOverlayTransformAbsolute(g_catcher, vr::TrackingUniverseStanding, &m); + vr::VROverlay()->SetOverlayWidthInMeters(g_catcher, float(std::max(0.5, 2 * d))); + ShowCatcher(true); +} + Screen *Find(int one_based) { auto it = g_screens.find(one_based - 1); return it == g_screens.end() ? nullptr : &it->second; @@ -1021,6 +1112,12 @@ uint32_t LinuxButton(uint32_t vrButton) { } } +// Any of the holding laser's buttons coming up on one of our controls or the catcher. +void ReleaseAwayBy(vr::TrackedDeviceIndex_t dev, uint32_t vrButton, void (*handle)(const struct ft_event *, void *), + void *data) { + if (g_press.buttons && dev == g_press.device) ReleaseAway(LinuxButton(vrButton), handle, data); +} + const char *LasersName() { switch (g_lasers) { case Lasers::Always: return "always"; @@ -1054,10 +1151,22 @@ bool ft_vr_init(void) { return false; } RefreshPoses(); + // The catcher (see UpdateCatcher): clear and invisible, but the laser lands on it, and + // it keeps SteamVR's laser mouse on while it's up. + if (vr::VROverlay()->CreateOverlay("frametop.catcher", "Frametop: release catcher", &g_catcher) == + vr::VROverlayError_None) { + static std::vector clear(4 * 4 * 4, 0); + vr::VROverlay()->SetOverlayRaw(g_catcher, clear.data(), 4, 4, 4); + vr::VROverlay()->SetOverlayInputMethod(g_catcher, vr::VROverlayInputMethod_Mouse); + vr::VROverlay()->SetOverlayAlpha(g_catcher, 0); + vr::VROverlay()->SetOverlayFlag(g_catcher, vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible, true); + } return true; } void ft_vr_shutdown(void) { + if (g_catcher != vr::k_ulOverlayHandleInvalid) vr::VROverlay()->DestroyOverlay(g_catcher); + g_catcher = vr::k_ulOverlayHandleInvalid; for (auto &[i, s] : g_screens) for (auto o : s.All()) vr::VROverlay()->DestroyOverlay(o); for (auto &[dev, g] : g_guides) @@ -1187,6 +1296,7 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { e.type = FT_MOTION; e.x = ev.data.mouse.x; e.y = s.height - ev.data.mouse.y; // OpenVR's mouse origin is bottom left + if (g_press.buttons) g_press.screen = index, g_press.x = e.x, g_press.y = e.y; break; case vr::VREvent_MouseButtonDown: case vr::VREvent_MouseButtonUp: @@ -1196,6 +1306,12 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { e.pressed = ev.eventType == vr::VREvent_MouseButtonDown; e.x = ev.data.mouse.x; e.y = s.height - ev.data.mouse.y; + if (e.pressed) { + PressDown(ev.trackedDeviceIndex, e.button, index, e.x, e.y); + } else { + g_press.buttons &= ~ButtonBit(e.button); + if (!g_press.buttons) g_press.upAt = -1; + } break; case vr::VREvent_ScrollDiscrete: e.type = FT_SCROLL; @@ -1203,6 +1319,7 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { e.dy = -ev.data.scroll.ydelta; break; case vr::VREvent_FocusLeave: + if (g_press.buttons) continue; // KWin keeps the pointer while a button is held e.type = FT_LEAVE; break; default: @@ -1221,8 +1338,10 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { hover(0); if (ev.eventType == vr::VREvent_MouseButtonDown && ev.data.mouse.button == vr::VRMouseButton_Left) StartDrag(s, Drag::Move, ev.trackedDeviceIndex); - else if (ev.eventType == vr::VREvent_MouseButtonUp) + else if (ev.eventType == vr::VREvent_MouseButtonUp) { EndDragsBy(ev.trackedDeviceIndex); + ReleaseAwayBy(ev.trackedDeviceIndex, ev.data.mouse.button, handle, data); + } else if (ev.eventType == vr::VREvent_ScrollDiscrete && s.drag == Drag::Move) Push(s, ev.data.scroll.ydelta); } @@ -1231,24 +1350,30 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { hover(3); if (ev.eventType == vr::VREvent_MouseButtonDown && ev.data.mouse.button == vr::VRMouseButton_Left) StartDrag(s, Drag::Resize, ev.trackedDeviceIndex); - else if (ev.eventType == vr::VREvent_MouseButtonUp) + else if (ev.eventType == vr::VREvent_MouseButtonUp) { EndDragsBy(ev.trackedDeviceIndex); + ReleaseAwayBy(ev.trackedDeviceIndex, ev.data.mouse.button, handle, data); + } } // The curve button. while (vr::VROverlay()->PollNextOverlayEvent(s.curveButton, &ev, sizeof ev)) { hover(1); if (ev.eventType == vr::VREvent_MouseButtonDown && ev.data.mouse.button == vr::VRMouseButton_Left) ToggleCurve(s); - else if (ev.eventType == vr::VREvent_MouseButtonUp) + else if (ev.eventType == vr::VREvent_MouseButtonUp) { EndDragsBy(ev.trackedDeviceIndex); + ReleaseAwayBy(ev.trackedDeviceIndex, ev.data.mouse.button, handle, data); + } } // The roll button: drag around like a knob, or scroll. while (vr::VROverlay()->PollNextOverlayEvent(s.rollButton, &ev, sizeof ev)) { hover(2); if (ev.eventType == vr::VREvent_MouseButtonDown && ev.data.mouse.button == vr::VRMouseButton_Left) StartDrag(s, Drag::Roll, ev.trackedDeviceIndex); - else if (ev.eventType == vr::VREvent_MouseButtonUp) + else if (ev.eventType == vr::VREvent_MouseButtonUp) { EndDragsBy(ev.trackedDeviceIndex); + ReleaseAwayBy(ev.trackedDeviceIndex, ev.data.mouse.button, handle, data); + } else if (ev.eventType == vr::VREvent_ScrollDiscrete && s.drag == Drag::None) { Mat p; if (!ScreenPose(s, &p)) continue; @@ -1258,8 +1383,14 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { } if (s.drag != Drag::None) UpdateDrag(s, index); } - RefreshChrome(); + // A release on the catcher, or the pointer helper's word that left came up (see "up"). vr::VREvent_t ev; + while (g_catcher != vr::k_ulOverlayHandleInvalid && + vr::VROverlay()->PollNextOverlayEvent(g_catcher, &ev, sizeof ev)) + if (ev.eventType == vr::VREvent_MouseButtonUp) + ReleaseAwayBy(ev.trackedDeviceIndex, ev.data.mouse.button, handle, data); + if (g_press.upAt >= 0 && g_tick >= g_press.upAt) ReleaseAway(BTN_LEFT, handle, data); + RefreshChrome(); while (vr::VRSystem()->PollNextEvent(&ev, sizeof ev)) { if (ev.eventType == vr::VREvent_Quit) { ft_event e{}; @@ -1276,6 +1407,7 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { UpdateLasers(); UpdateControls(); UpdateGuides(); + UpdateCatcher(); } // Control commands (datagrams on @ft_screens, replies to the sender): @@ -1298,6 +1430,9 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { // controllers always|outside_games|dashboard when controllers' lasers work the screens // ingames hide|visible during a VR game, "always" acts like "only with the dashboard" // (hide), or stays as it is (visible) +// up the pointer helper: the mouse's left button came up. If SteamVR +// hasn't delivered that release to one of our overlays within +// ~100 ms (it landed on something else), KWin gets it anyway // state -> "ok // " // (size and key are handled in compositor.c.) Screens are @@ -1429,6 +1564,10 @@ void ft_vr_command(const char *cmd, char *reply, int size) { else return (void)std::snprintf(reply, size, "error modes: always outside_games dashboard"); UpdateLasers(); std::snprintf(reply, size, "ok %s", LasersName()); + } else if (std::strcmp(cmd, "up") == 0) { + if ((g_press.buttons & ButtonBit(BTN_LEFT)) && g_press.device != kNone && !IsHandController(g_press.device)) + g_press.upAt = g_tick + 9; + std::snprintf(reply, size, "ok"); } else if (std::strncmp(cmd, "state", 5) == 0) { std::snprintf(reply, size, "ok %s %d %.0f %s %.0f %s %d %s", ModeName(), g_manual ? 1 : 0, g_wristAngle, g_gestureHand.c_str(), g_gestureAngle, LasersName(), g_gameRunning ? 1 : 0, From 42a755b9f4f1bd1a0b37a33c9ba7ea9007ca52d9 Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:14:09 -0600 Subject: [PATCH 04/13] Add a headless test mode to ft-screens and a throwaway test desktop ft-screens --no-vr runs without SteamVR and leaves the input relay alone; --control names its control socket; "toplevels" lists KWin's windows with their titles and sizes; "input" feeds pointer events as if from a panel. screens/test/headless.sh starts it with a bare nested KWin next to the running desktop, and loads KWin scripts, runs apps, and takes screenshots. Co-Authored-By: Claude Opus 5.5 --- screens/compositor.c | 88 ++++++++++++++++++--- screens/test/headless.sh | 160 +++++++++++++++++++++++++++++++++++++++ screens/test/observe.js | 22 ++++++ screens/vr.cpp | 14 +++- 4 files changed, 272 insertions(+), 12 deletions(-) create mode 100755 screens/test/headless.sh create mode 100644 screens/test/observe.js diff --git a/screens/compositor.c b/screens/compositor.c index 3555531..f6932a8 100644 --- a/screens/compositor.c +++ b/screens/compositor.c @@ -12,8 +12,13 @@ // buffers are in pixels, so a panel position in pixels is divided by the screen's KWin // scale first ("scale ", from ft-layout). // -// Usage: ft-screens [--socket NAME] [--screen WxH@METRES]... [-- COMMAND ARGS...] +// Usage: ft-screens [--socket NAME] [--control NAME] [--no-vr] [--screen WxH@METRES]... +// [-- COMMAND ARGS...] // --socket Wayland socket name in $XDG_RUNTIME_DIR (default ft-screens-0) +// --control the control socket's abstract name (default ft_screens) +// --no-vr run without SteamVR, for tests next to the running desktop: no panels, no +// input, and nothing sent to the input relay. Commands still work, and +// "toplevels" shows what KWin opened. // --screen one per screen, in KWin's order (default: 3440x1440@2.4) // COMMAND run with WAYLAND_DISPLAY set to our socket (e.g. the Frametop session) // Runs in the dev container (wlroots 0.20); KWin connects from the host. @@ -68,8 +73,10 @@ struct screen { struct wlr_xdg_toplevel *toplevel; struct wlr_buffer *held; // on the panel now; unlocked when the next one arrives bool frame_pending; // a commit waits for its frame callback + unsigned commits; // buffers committed, and the last one's size ("toplevels") + int buffer_width, buffer_height; struct wlr_xdg_toplevel_decoration_v1 *decoration; // answered on the first commit - struct wl_listener commit, destroy, decoration_destroy; + struct wl_listener commit, destroy, decoration_destroy, set_title; }; // Per client buffer: forget its import when it goes away. @@ -102,6 +109,7 @@ struct server { int relay_fd; // unbound, so the relay can't reply into our control socket uint32_t relay_sent; // when the relay last heard from us (ms) unsigned ticks; + bool vr; // connected to SteamVR (not --no-vr) }; static uint32_t now_ms(void) { @@ -153,6 +161,8 @@ static void screen_commit(struct wl_listener *l, void *data) { } if (!buffer) return; sc->frame_pending = true; + ++sc->commits; + sc->buffer_width = buffer->width, sc->buffer_height = buffer->height; if (buffer == sc->held) return; struct wlr_dmabuf_attributes a; if (!wlr_buffer_get_dmabuf(buffer, &a)) { @@ -187,9 +197,17 @@ static void screen_destroy(struct wl_listener *l, void *data) { sc->server->screens[sc->index] = NULL; wl_list_remove(&sc->commit.link); wl_list_remove(&sc->destroy.link); + wl_list_remove(&sc->set_title.link); free(sc); } +// KWin titles each window "KDE Wayland Compositor ", with "- Output disabled" +// after it while that output is off. +static void screen_title(struct wl_listener *l, void *data) { + struct screen *sc = wl_container_of(l, sc, set_title); + wlr_log(WLR_INFO, "screen %d: \"%s\"", sc->index + 1, sc->toplevel->title ? sc->toplevel->title : ""); +} + static void new_toplevel(struct wl_listener *l, void *data) { struct server *s = wl_container_of(l, s, new_toplevel); struct wlr_xdg_toplevel *toplevel = data; @@ -211,6 +229,8 @@ static void new_toplevel(struct wl_listener *l, void *data) { wl_signal_add(&toplevel->base->surface->events.commit, &sc->commit); sc->destroy.notify = screen_destroy; wl_signal_add(&toplevel->events.destroy, &sc->destroy); + sc->set_title.notify = screen_title; + wl_signal_add(&toplevel->events.set_title, &sc->set_title); } // KWin asks for server-side decorations for its screens; we draw none. It asks before its @@ -294,6 +314,7 @@ static void handle_vr_event(const struct ft_event *e, void *data) { // gamescope's focus) doesn't get the keys too. Without word from us for a few seconds, // the relay gives the keyboards back, so a closed desktop doesn't keep them. static void keys_update(struct server *s) { + if (!s->vr) return; // a test instance leaves the running desktop's keyboards alone const bool desktop = s->keys_clicked && ft_vr_screens_shown(); const uint32_t t = now_ms(); if (desktop == s->keys_desktop && t - s->relay_sent < 1000) return; @@ -366,6 +387,9 @@ static void handle_key(struct server *s, uint32_t code, int value, char *reply, // Control socket: abstract datagram @ft_screens. Here: "size " (a new // resolution, live), "scale " (KWin's scale for it, from ft-layout), +// "toplevels" (-> "ok " and a line per KWin window: " x +// "), "input <screen> move|down|up|leave [x y [left|right|middle]]" (pointer input +// as if from that panel, x and y in its pixels; for tests, mostly with --no-vr), // "key <code> <value>", and "click <overlay key>" (from the pointer // helper: a mouse click landed on that panel, "-" for none); the rest is in vr.cpp // (ft_vr_command). @@ -402,6 +426,34 @@ static int control_readable(int fd, uint32_t mask, void *data) { handle_key(s, code, value, reply, sizeof reply); len = sizeof from; continue; // no reply: keys are fire-and-forget + } else if (strncmp(buf, "input ", 6) == 0) { + char what[8] = "", button[8] = "left"; + double x = 0, y = 0; + const int got = sscanf(buf, "input %d %7s %lf %lf %7s", &index, what, &x, &y, button); + struct ft_event e = {.screen = index - 1, .x = x, .y = y, .button = BTN_LEFT}; + if (strcmp(button, "right") == 0) e.button = BTN_RIGHT; + else if (strcmp(button, "middle") == 0) e.button = BTN_MIDDLE; + if (got >= 2 && strcmp(what, "leave") == 0) e.type = FT_LEAVE; + else if (got >= 4 && strcmp(what, "move") == 0) e.type = FT_MOTION; + else if (got >= 4 && (strcmp(what, "down") == 0 || strcmp(what, "up") == 0)) + e.type = FT_BUTTON, e.pressed = what[0] == 'd'; + else index = 0; + if (index < 1 || index > MAX_SCREENS || !s->screens[index - 1]) { + snprintf(reply, sizeof reply, "error input <screen> move|down|up|leave [x y [button]]"); + } else { + handle_vr_event(&e, s); + snprintf(reply, sizeof reply, "ok"); + } + } else if (strcmp(buf, "toplevels") == 0) { + int n = 0, at = 0; + for (int i = 0; i < MAX_SCREENS; ++i) n += s->screens[i] != NULL; + at = snprintf(reply, sizeof reply, "ok %d", n); + for (int i = 0; i < MAX_SCREENS && at < (int)sizeof reply; ++i) { + const struct screen *sc = s->screens[i]; + if (!sc) continue; + at += snprintf(reply + at, sizeof reply - at, "\n%d %dx%d %u %s", i + 1, sc->buffer_width, + sc->buffer_height, sc->commits, sc->toplevel->title ? sc->toplevel->title : ""); + } } else if (strncmp(buf, "click ", 6) == 0) { // A click on another panel takes typing to Steam. Our own panels (the screens, // their controls) leave it: a click on a screen arrives as a panel event. @@ -418,13 +470,18 @@ static int control_readable(int fd, uint32_t mask, void *data) { return 0; } -static int open_control_socket(void) { +static int open_control_socket(const char *name) { int fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0); struct sockaddr_un addr = {.sun_family = AF_UNIX}; - const char name[] = "ft_screens"; - memcpy(addr.sun_path + 1, name, sizeof name - 1); - if (bind(fd, (struct sockaddr *)&addr, offsetof(struct sockaddr_un, sun_path) + 1 + sizeof name - 1) != 0) { - wlr_log(WLR_ERROR, "can't bind @ft_screens (another ft-screens running?)"); + const size_t len = strlen(name); + if (len == 0 || len >= sizeof addr.sun_path - 1) { + wlr_log(WLR_ERROR, "bad control socket name"); + close(fd); + return -1; + } + memcpy(addr.sun_path + 1, name, len); + if (bind(fd, (struct sockaddr *)&addr, offsetof(struct sockaddr_un, sun_path) + 1 + len) != 0) { + wlr_log(WLR_ERROR, "can't bind @%s (another ft-screens running?)", name); close(fd); return -1; } @@ -467,11 +524,16 @@ static bool setup_dmabuf(struct server *s) { int main(int argc, char **argv) { struct server s = {0}; for (int i = 0; i < MAX_SCREENS; ++i) s.scale[i] = 1; - const char *socket_name = "ft-screens-0"; + const char *socket_name = "ft-screens-0", *control_name = "ft_screens"; char **command = NULL; + s.vr = true; for (int i = 1; i < argc; ++i) { if (strcmp(argv[i], "--socket") == 0 && i + 1 < argc) { socket_name = argv[++i]; + } else if (strcmp(argv[i], "--control") == 0 && i + 1 < argc) { + control_name = argv[++i]; + } else if (strcmp(argv[i], "--no-vr") == 0) { + s.vr = false; } else if (strcmp(argv[i], "--screen") == 0 && i + 1 < argc && s.n_config < MAX_SCREENS) { struct config *c = &s.config[s.n_config]; c->metres = 0; @@ -485,7 +547,10 @@ int main(int argc, char **argv) { command = &argv[i + 1]; break; } else { - fprintf(stderr, "usage: %s [--socket NAME] [--screen WxH@METRES]... [-- COMMAND ARGS...]\n", argv[0]); + fprintf(stderr, + "usage: %s [--socket NAME] [--control NAME] [--no-vr] [--screen WxH@METRES]... " + "[-- COMMAND ARGS...]\n", + argv[0]); return 2; } } @@ -493,7 +558,8 @@ int main(int argc, char **argv) { setvbuf(stdout, NULL, _IOLBF, 0); // vr.cpp prints to stdout; keep it in order with the log wlr_log_init(WLR_INFO, NULL); - if (!ft_vr_init()) return 1; + if (s.vr && !ft_vr_init()) return 1; + if (!s.vr) wlr_log(WLR_INFO, "--no-vr: running without SteamVR"); s.display = wl_display_create(); s.loop = wl_display_get_event_loop(s.display); @@ -530,7 +596,7 @@ int main(int argc, char **argv) { snprintf(path, sizeof path, "%s/%s", getenv("XDG_RUNTIME_DIR") ? getenv("XDG_RUNTIME_DIR") : "/tmp", socket_name); wlr_log(WLR_INFO, "listening on %s; %d screen(s) configured", path, s.n_config); - const int control = open_control_socket(); + const int control = open_control_socket(control_name); if (control < 0) return 1; s.relay_fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0); wl_event_loop_add_fd(s.loop, control, WL_EVENT_READABLE, control_readable, &s); diff --git a/screens/test/headless.sh b/screens/test/headless.sh new file mode 100755 index 0000000..a0952c5 --- /dev/null +++ b/screens/test/headless.sh @@ -0,0 +1,160 @@ +#!/usr/bin/env bash +# A throwaway desktop for experiments, next to the running one: ft-screens --no-vr (its own +# Wayland socket, control socket @ft_screens_test, and runtime folder) with a bare nested +# KWin inside it, on its own D-Bus and config folder. Nothing touches SteamVR, the input +# relay, or the real desktop's settings. Run it from the PC (it runs itself on the Frame) +# or on the Frame. Build ft-screens first (screens/build.sh). +# +# headless.sh start [SCREENS] [SPARES] (default 2 screens, 1920x1080 and 1080x1920, and +# 3 spare 800x600 outputs, disabled once KWin is up) +# headless.sh stop +# headless.sh ask '<ft-screens command>' e.g. toplevels, "input 1 down 400 20" +# headless.sh kd <kscreen-doctor args> e.g. -o, output.WL-2.enable +# headless.sh run <command> [args] start an app in the test KWin (in the background) +# headless.sh script <file.js> [name] load a KWin script into it and run it +# headless.sh unscript <name> +# headless.sh js '<javascript>' run a one-off KWin script; its print()s come back +# headless.sh shot [OUTPUT] a JPEG of one output (default WL-0); from the PC it +# lands in captures/ and its path is printed +# headless.sh log [kwin|screens|apps] [N] the last N lines (default 30) +set -euo pipefail +here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) +. "$here/../../scripts/_env.sh" + +if [ "$FRAME_LOCAL" != 1 ]; then + "$REPO_ROOT/scripts/sync.sh" >/dev/null + if [ "${1:-}" = shot ]; then + mkdir -p "$REPO_ROOT/captures" + out=$REPO_ROOT/captures/ft-test-${2:-WL-0}-$(date +%H%M%S).jpg + on_frame "screens/test/headless.sh $(printf '%q ' "$@")" > "$out" + echo "$out" + else + on_frame "screens/test/headless.sh $(printf '%q ' "$@")" + fi + exit +fi + +rt=/run/user/$(id -u)/ft-test +cfg=/tmp/ft-test-config +bus=unix:path=$rt/bus +logs=/tmp/ft-test +mkdir -p "$logs" + +kwin_env() { + env XDG_RUNTIME_DIR="$rt" XDG_CONFIG_HOME="$cfg" DBUS_SESSION_BUS_ADDRESS="$bus" \ + WAYLAND_DISPLAY=wayland-0 QT_QPA_PLATFORM=wayland "$@" +} + +ask() { + python3 - "$1" <<'EOF' +import socket, sys +s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) +s.bind("") +s.settimeout(2) +s.sendto(sys.argv[1].encode(), "\0ft_screens_test") +print(s.recv(8192).decode()) +EOF +} + +kd() { kwin_env kscreen-doctor "$@" 2>&1 | sed 's/\x1b\[[0-9;]*m//g'; } + +stop() { + pkill -f -- "--control ft_screens_test" 2>/dev/null || true + [ -f "$rt/dbus.pid" ] && kill "$(cat "$rt/dbus.pid")" 2>/dev/null || true + sleep 0.5 +} + +load_script() { # file name + local id + id=$(kwin_env gdbus call --session -d org.kde.KWin -o /Scripting \ + -m org.kde.kwin.Scripting.loadScript "$1" "$2" | tr -dc '0-9-') + [ "$id" -ge 0 ] 2>/dev/null || { echo "loadScript failed ($id)" >&2; return 1; } + kwin_env gdbus call --session -d org.kde.KWin -o "/Scripting/Script$id" -m org.kde.kwin.Script.run >/dev/null +} + +unload_script() { + kwin_env gdbus call --session -d org.kde.KWin -o /Scripting -m org.kde.kwin.Scripting.unloadScript "$1" >/dev/null +} + +case "${1:-}" in + start) + screens=${2:-2} spares=${3:-3} + stop + rm -rf "$rt" "$cfg" + mkdir -p "$rt" "$cfg" + chmod 700 "$rt" + args=(--screen 1920x1080@1.6 --screen 1080x1920@0.9) + for ((i = 2; i < screens; i++)); do args+=(--screen 1920x1080@1.6); done + args=("${args[@]:0:$((screens * 2))}") + for ((i = 0; i < spares; i++)); do args+=(--screen 800x600); done + cd "$REPO_ROOT" + nohup "$HOME/.local/bin/distrobox" enter "$FRAME_BOX" -- env XDG_RUNTIME_DIR="$rt" \ + ./screens/build/ft-screens --no-vr --socket ft-test-0 --control ft_screens_test "${args[@]}" \ + > "$logs/screens.log" 2>&1 & + for _ in $(seq 100); do [ -S "$rt/ft-test-0" ] && break; sleep 0.2; done + [ -S "$rt/ft-test-0" ] || { echo "ft-screens didn't start:"; tail "$logs/screens.log"; exit 1; } + dbus-daemon --session --address="$bus" --fork --print-pid > "$rt/dbus.pid" + # KWin's screenshot interface without a permission check, so tests can look. + nohup env XDG_RUNTIME_DIR="$rt" XDG_CONFIG_HOME="$cfg" DBUS_SESSION_BUS_ADDRESS="$bus" \ + WAYLAND_DISPLAY=ft-test-0 KWIN_SCREENSHOT_NO_PERMISSION_CHECKS=1 QT_LOGGING_RULES="kwin_scripting=true" QT_FORCE_STDERR_LOGGING=1 \ + kwin_wayland --output-count $((screens + spares)) --width 800 --height 600 --no-lockscreen \ + > "$logs/kwin.log" 2>&1 & + for _ in $(seq 100); do [ -S "$rt/wayland-0" ] && break; sleep 0.2; done + [ -S "$rt/wayland-0" ] || { echo "KWin didn't start:"; tail "$logs/kwin.log"; exit 1; } + sleep 1 + off=() + for ((i = screens; i < screens + spares; i++)); do off+=("output.WL-$i.disable"); done + [ ${#off[@]} -eq 0 ] || kd "${off[@]}" >/dev/null + ask toplevels + ;; + stop) stop ;; + ask) ask "$2" ;; + kd) shift; kd "$@" ;; + run) + shift + nohup env XDG_RUNTIME_DIR="$rt" XDG_CONFIG_HOME="$cfg" DBUS_SESSION_BUS_ADDRESS="$bus" \ + WAYLAND_DISPLAY=wayland-0 QT_QPA_PLATFORM=wayland "$@" >> "$logs/apps.log" 2>&1 & + echo "started $1 (pid $!)" + ;; + script) load_script "$(realpath "$2")" "${3:-$(basename "$2" .js)}" ;; + unscript) unload_script "$2" ;; + js) + f=$(mktemp /tmp/ft-test-XXXX.js) + printf '%s\n' "$2" > "$f" + name=ft-test-js-$$ + mark=$(wc -l < "$logs/kwin.log") + load_script "$f" "$name" + sleep 0.5 + unload_script "$name" || true + rm -f "$f" + tail -n +$((mark + 1)) "$logs/kwin.log" | sed -n 's/^js: //p' + ;; + shot) + raw=$(mktemp /tmp/ft-test-shot.XXXXXX) + meta=$(kwin_env python3 - "${2:-WL-0}" "$raw" <<'PY' +import os, sys, dbus +name, raw = sys.argv[1:3] +bus = dbus.bus.BusConnection(os.environ['DBUS_SESSION_BUS_ADDRESS']) +shot = dbus.Interface(bus.get_object('org.kde.KWin', '/org/kde/KWin/ScreenShot2'), 'org.kde.KWin.ScreenShot2') +r, w = os.pipe() +fd = dbus.types.UnixFd(w) +os.close(w) +opts = dbus.Dictionary({'include-cursor': True, 'native-resolution': True}, signature='sv') +res = shot.CaptureScreen(name, opts, fd) +del fd # our copy of the write end, so the read ends when KWin is done +with os.fdopen(r, 'rb') as src, open(raw, 'wb') as dst: + while chunk := src.read(1 << 20): + dst.write(chunk) +print(int(res['width']), int(res['height']), int(res['stride']), int(res['format'])) +PY +) + read -r width height stride format <<<"$meta" + layout=bgra # QImage formats 4-6: (A)RGB32, stored B,G,R,A; 16-18: RGBA8888 + case $format in 16|17|18) layout=rgba ;; esac + "$HOME/.local/bin/distrobox" enter "$FRAME_BOX" -- magick -size "$((stride / 4))x$height" -depth 8 \ + "$layout:$raw" -crop "${width}x${height}+0+0" +repage -alpha off -resize '1280x1280>' -quality 80 jpg:- + rm -f "$raw" + ;; + log) tail -n "${3:-30}" "$logs/${2:-kwin}.log" ;; + *) sed -n '2,20p' "$0"; exit 2 ;; +esac diff --git a/screens/test/observe.js b/screens/test/observe.js new file mode 100644 index 0000000..56b898c --- /dev/null +++ b/screens/test/observe.js @@ -0,0 +1,22 @@ +// A KWin script for experiments (screens/test/headless.sh script ...): prints what happens +// to windows, for checking the script API floating windows rely on. +function desc(w) { + const g = w.frameGeometry; + return String(w.internalId).slice(1, 9) + " " + w.resourceClass + " pid=" + w.pid + " popup=" + w.popupWindow + + " transient=" + w.transient + " normal=" + w.normalWindow + " dialog=" + w.dialog + + " out=" + (w.output ? w.output.name : "?") + " geo=" + g.x + "," + g.y + " " + g.width + "x" + g.height + + " '" + w.caption + "'"; +} +function watch(w) { + print("added " + desc(w)); + w.interactiveMoveResizeStarted.connect(() => print("move-start move=" + w.move + " resize=" + w.resize + " " + desc(w))); + w.interactiveMoveResizeStepped.connect(g => print("move-step " + g.x + "," + g.y + " " + g.width + "x" + g.height)); + w.interactiveMoveResizeFinished.connect(() => print("move-end " + desc(w))); + w.outputChanged.connect(() => print("output " + desc(w))); + w.fullScreenChanged.connect(() => print("fullscreen " + w.fullScreen + " " + desc(w))); + w.minimizedChanged.connect(() => print("minimized " + w.minimized)); +} +workspace.windowList().forEach(watch); +workspace.windowAdded.connect(watch); +workspace.windowRemoved.connect(w => print("removed " + desc(w))); +workspace.windowActivated.connect(w => print("activated " + (w ? w.resourceClass : "none"))); diff --git a/screens/vr.cpp b/screens/vr.cpp index 5011962..04c4af9 100644 --- a/screens/vr.cpp +++ b/screens/vr.cpp @@ -206,6 +206,7 @@ constexpr double kRollStep = 5; // degrees per scroll notch on the roll but constexpr float kChromeIdle = 0.55f; // the controls' opacity without a laser on them constexpr long kControlsLinger = 35; // ticks (~0.4 s) the controls stay after a laser leaves long g_tick = 0; // ft_vr_poll calls +bool g_vr = false; // connected to SteamVR (ft-screens --no-vr runs without it) constexpr vr::TrackedDeviceIndex_t kNone = vr::k_unTrackedDeviceIndexInvalid; struct Screen { @@ -1150,6 +1151,7 @@ bool ft_vr_init(void) { std::fprintf(stderr, "openvr: no IVRIPCResourceManagerClient (SteamVR too old?)\n"); return false; } + g_vr = true; RefreshPoses(); // The catcher (see UpdateCatcher): clear and invisible, but the laser lands on it, and // it keeps SteamVR's laser mouse on while it's up. @@ -1165,6 +1167,7 @@ bool ft_vr_init(void) { } void ft_vr_shutdown(void) { + if (!g_vr) return; if (g_catcher != vr::k_ulOverlayHandleInvalid) vr::VROverlay()->DestroyOverlay(g_catcher); g_catcher = vr::k_ulOverlayHandleInvalid; for (auto &[i, s] : g_screens) @@ -1179,14 +1182,20 @@ void ft_vr_shutdown(void) { } int ft_vr_modifiers(uint32_t format, uint64_t *out, int max) { + if (!g_vr) { // --no-vr: nothing imports the buffers, so any layout KWin can draw + if (max < 1) return 0; + out[0] = 0; // DRM_FORMAT_MOD_LINEAR + return 1; + } uint32_t n = uint32_t(max); if (!vr::VRIPCResourceManager()->GetDmabufModifiers(vr::VRApplication_Overlay, format, &n, out)) return 0; return int(n < uint32_t(max) ? n : uint32_t(max)); } -bool ft_vr_screens_shown(void) { return ModeVisible(); } +bool ft_vr_screens_shown(void) { return g_vr && ModeVisible(); } void ft_vr_screen_create(int index, double metres, int count) { + if (!g_vr) return; Screen &s = g_screens[index]; s.metres = metres; char key[64], name[64]; @@ -1237,6 +1246,7 @@ void ft_vr_screen_destroy(int index) { } bool ft_vr_screen_present(int index, const void *key, const struct ft_dmabuf *b) { + if (!g_vr) return false; auto sit = g_screens.find(index); if (sit == g_screens.end()) return false; Screen &s = sit->second; @@ -1284,6 +1294,7 @@ void ft_vr_forget(const void *key) { } void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { + if (!g_vr) return; RefreshPoses(); for (auto &[index, s] : g_screens) { vr::VREvent_t ev; @@ -1438,6 +1449,7 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { // (size <screen> <w> <h> and key <code> <value> are handled in compositor.c.) Screens are // numbered from 1 here, like everywhere the user sees them. void ft_vr_command(const char *cmd, char *reply, int size) { + if (!g_vr) return (void)std::snprintf(reply, size, "error no SteamVR (--no-vr)"); RefreshPoses(); int n; double x, y, z, yaw, pitch, roll, w; From 4859412126abf95c9a2dc7b7125285b0a6577a02 Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:14:09 -0600 Subject: [PATCH 05/13] Put the first click after crossing onto a screen where the pointer is KWin's nested backend ignores the position in wl_pointer.enter, and wlroots drops a motion to the position it entered at, so KWin kept its old pointer until the next move. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- screens/compositor.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/screens/compositor.c b/screens/compositor.c index f6932a8..ac48cfc 100644 --- a/screens/compositor.c +++ b/screens/compositor.c @@ -272,7 +272,12 @@ static void handle_vr_event(const struct ft_event *e, void *data) { case FT_MOTION: case FT_BUTTON: if (s->pointer_focus != sc) { - wlr_seat_pointer_notify_enter(s->seat, surface, x, y); + // KWin's nested backend ignores the position in wl_pointer.enter, and wlroots + // drops a motion to the position it entered at, so KWin would keep its old + // pointer until the next move: a press right after crossing onto another + // screen landed where the pointer had been. Entering one unit off makes the + // motion below go through. + wlr_seat_pointer_notify_enter(s->seat, surface, x + 1, y); s->pointer_focus = sc; } wlr_seat_pointer_notify_motion(s->seat, t, x, y); From fcc8d96246610791d273c52db90056241907975e Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:14:22 -0600 Subject: [PATCH 06/13] Record the headless phase 0 results Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- docs/floating-windows.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/floating-windows.md b/docs/floating-windows.md index df0c9d5..77ac815 100644 --- a/docs/floating-windows.md +++ b/docs/floating-windows.md @@ -156,6 +156,17 @@ Each item has a pass condition. Items that need a desktop restart with extra out - **0.5 SteamVR's laser.** Find out which overlay gets MouseMove and ButtonUp when a held laser moves from overlay A to overlay B, and when it's released over nothing, for both a controller and the 3D mouse. Pass: an interactive overlay placed on the laser reliably catches the release. - **0.6 Texture bounds.** Pass: `SetOverlayTextureBounds` crops a DMA-BUF (`SharedTextureHandle`) overlay correctly, mouse positions map to the cropped area, and two overlays can show different crops of one buffer. +#### Results (2026-09-29, headless, KWin 6.2.5) + +`screens/test/headless.sh` runs these: ft-screens `--no-vr` with a bare nested KWin next to the running desktop, with an `input` command that feeds pointer events as if from a panel, KWin scripts loaded over D-Bus, and screenshots through ScreenShot2. + +- **0.1 passes.** `--no-vr`, `--control`, `toplevels`, and `input` are in ft-screens. +- **0.2 passes.** Disabling a spare with `kscreen-doctor` keeps its toplevel, its title gets `- Output disabled`, and it stops committing; enabling it resumes on the same toplevel. A spare resized (`size`) while disabled comes up at the new size on its first frame, so a tear-off needn't blink. Live resizing works, output scale works (1.5: KWin lays out 1067 × 667 on a 1600 × 1000 buffer), and outputs with gaps between them (x = 5000, 8000, 10000) are accepted. A window placed inside a 1600 × 1200 output with a 300 px margin opens a context menu past its bottom and right edges, into the margin. +- **0.3 mostly passes.** `workspace.screens`, `sendClientToScreen`, `windowList`, `frameGeometry` (set; it applies asynchronously), `callDBus`, `registerShortcut`, `registerUserActionsMenu`, and `readConfig` exist. `windowAdded` reports popups (`popupWindow` true, `transient` true) with their geometry. Move signals fire for KWin's title bars (`interactiveMoveResizeStarted` with `move` true, `Stepped` with the geometry, `Finished`). Not yet checked: apps' own title bars, the `callDBus` long poll, and the window menu entry. `globalThis` isn't defined in KWin's script engine. `print` goes to the journal unless `QT_FORCE_STDERR_LOGGING=1`. +- **0.4: KWin starts the move on the press itself,** before any motion. So ft-screens freezes pointer motion as soon as a press lands in a floating window's title bar band (from the frame and client rectangles ft-floatd sends it), with no round trip. For apps that draw their own title bars, the script reports the move and ft-screens freezes then; the script puts back any few pixels the window slipped before that. +- **Found and fixed:** KWin's nested backend ignores the position in `wl_pointer.enter`, and wlroots drops a motion to the position it entered at, so the first click after crossing onto another screen landed where KWin's pointer had been. ft-screens now enters one unit off. +- **0.5 and 0.6** need SteamVR and the headset. + ### Phase 1: float a window from its menu Build the KWin script, ft-floatd, and floating panels in ft-screens: the margin and popup overlays, controls, moving by the title bar, resizing (edges and tab), scale, full screen, close, and back to desktop. Add drag and drop across panels, with the pointer helper change. From 87a402c68df7135cfeeda3654a78839aafe64189 Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:20:26 -0600 Subject: [PATCH 07/13] Add ft-floatd and the frametop-float KWin script ft-floatd loads the script into the desktop's KWin, which reports windows over D-Bus and takes commands through a long poll. Floating a window (the window menu's Float in VR, Meta+Shift+F, or ft-float) turns on a spare output sized to the window plus a margin, moves the window onto it, and tells ft-screens the panel's crop, density, and place; docking puts it back and turns the spare off. Tested on the headless test desktop; the panel side in ft-screens comes next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- float/frametop-float.js | 166 +++++++++++++ float/ft-float | 25 ++ float/ft-floatd | 3 + float/ft_floatd.py | 523 +++++++++++++++++++++++++++++++++++++++ screens/test/headless.sh | 16 +- 5 files changed, 732 insertions(+), 1 deletion(-) create mode 100644 float/frametop-float.js create mode 100755 float/ft-float create mode 100755 float/ft-floatd create mode 100644 float/ft_floatd.py diff --git a/float/frametop-float.js b/float/frametop-float.js new file mode 100644 index 0000000..8d12c0f --- /dev/null +++ b/float/frametop-float.js @@ -0,0 +1,166 @@ +// frametop-float: the KWin side of floating windows (see docs/floating-windows.md). ft-floatd +// loads it into the desktop's KWin over D-Bus (org.kde.kwin.Scripting) and talks to it: +// - events go to ft-floatd as JSON strings (org.frametop.Float.Event), for the windows it +// cares about: floating windows (the ones on a spare output, WL-<screens> and up), their +// popups and dialogs, new windows, and "Float in VR" requests; +// - commands come back through a long poll: the script calls NextCommand, ft-floatd +// answers when it has one (or after a while with nothing), and the script calls again. +// KWin scripts can call D-Bus but can't serve it, hence the poll. Window ids are KWin's +// internalId (a UUID string). + +const SERVICE = "org.frametop.Float", PATH = "/Float", IFACE = "org.frametop.Float"; +let screens = 0; // outputs WL-0 .. WL-<screens - 1> are screens; the rest are spares +let polling = false; +const watched = {}; // id -> true once its signals are connected + +function send(ev) { + callDBus(SERVICE, PATH, IFACE, "Event", JSON.stringify(ev)); +} + +function outputIndex(o) { + const m = o ? /^WL-(\d+)$/.exec(o.name) : null; + return m ? parseInt(m[1]) : -1; +} +function isSpare(o) { + return screens > 0 && outputIndex(o) >= screens; +} +function rect(g) { + return {x: g.x, y: g.y, w: g.width, h: g.height}; +} +function byId(id) { + const all = workspace.windowList(); + for (let i = 0; i < all.length; ++i) + if (String(all[i].internalId) === id) return all[i]; + return null; +} +function outputByName(name) { + const all = workspace.screens; + for (let i = 0; i < all.length; ++i) + if (all[i].name === name) return all[i]; + return null; +} +function info(w) { + const o = w.output; + return { + id: String(w.internalId), pid: w.pid, cls: String(w.resourceClass), app: String(w.desktopFileName), + caption: String(w.caption), output: o ? o.name : "", outputRect: o ? rect(o.geometry) : null, + frame: rect(w.frameGeometry), client: rect(w.clientGeometry), popup: w.popupWindow, + transient: w.transient, parent: w.transientFor ? String(w.transientFor.internalId) : "", + normal: w.normalWindow, dialog: w.dialog, fullScreen: w.fullScreen, minimized: w.minimized, + onAllDesktops: w.onAllDesktops + }; +} + +function report(type, w) { + const ev = info(w); + ev.ev = type; + send(ev); +} + +// Floating windows, and popups and dialogs on a spare output: tell ft-floatd about changes. +function watch(w) { + const id = String(w.internalId); + if (watched[id]) return; + watched[id] = true; + const onSpare = () => isSpare(w.output); + w.frameGeometryChanged.connect(() => { if (onSpare()) report("geometry", w); }); + w.outputChanged.connect(() => report("output", w)); + w.interactiveMoveResizeStarted.connect(() => { + if (onSpare()) send({ev: "move-start", id: id, move: w.move, resize: w.resize, frame: rect(w.frameGeometry)}); + }); + w.interactiveMoveResizeFinished.connect(() => { if (onSpare()) report("move-end", w); }); + w.fullScreenChanged.connect(() => { if (onSpare()) report("fullscreen", w); }); + w.minimizedChanged.connect(() => { if (onSpare()) report("minimized", w); }); + w.maximizedChanged.connect(() => { + // A floating window stays an ordinary window: its output is its size plus a margin. + if (onSpare() && w.normalWindow && !w.fullScreen) w.setMaximize(false, false); + }); +} + +workspace.windowAdded.connect(w => { + watch(w); + report("added", w); +}); +workspace.windowRemoved.connect(w => { + send({ev: "removed", id: String(w.internalId)}); + delete watched[String(w.internalId)]; +}); +workspace.windowActivated.connect(w => { + if (w && isSpare(w.output)) send({ev: "activated", id: String(w.internalId)}); +}); +workspace.windowList().forEach(watch); + +function requestFloat(w) { + if (!w || !w.normalWindow || w.popupWindow) return; + report(isSpare(w.output) ? "dock-request" : "float-request", w); +} + +registerUserActionsMenu(w => { + if (!w.normalWindow || w.popupWindow) return null; + const floating = isSpare(w.output); + return { + text: floating ? "Back to Desktop" : "Float in VR", + icon: floating ? "window-restore" : "window-new", + triggered: () => requestFloat(w) + }; +}); +registerShortcut("Frametop Float Window", "Frametop: Float Window in VR (or put it back)", "Meta+Shift+F", + () => requestFloat(workspace.activeWindow)); + +function run(c) { + const w = c.id ? byId(c.id) : null; + switch (c.cmd) { + case "config": + screens = c.screens; + workspace.windowList().forEach(w => report("window", w)); + break; + case "place": { // onto an output, at a frame rectangle (logical, global) + if (!w) break; + const o = outputByName(c.output); + if (!o) break; + if (w.fullScreen && !c.keepFullScreen) w.fullScreen = false; + w.setMaximize(false, false); + workspace.sendClientToScreen(w, o); + w.frameGeometry = {x: c.x, y: c.y, width: c.w, height: c.h}; + if (c.onAllDesktops !== undefined) w.onAllDesktops = c.onAllDesktops; + break; + } + case "geometry": + if (w) w.frameGeometry = {x: c.x, y: c.y, width: c.w, height: c.h}; + break; + case "close": + if (w) w.closeWindow(); + break; + case "activate": + if (w) workspace.activeWindow = w; + break; + case "minimize": + if (w) w.minimized = c.on; + break; + case "info": + if (w) report("window", w); + break; + case "request-active": // ft-float float|dock active: like the shortcut + requestFloat(workspace.activeWindow); + break; + } +} + +function poll() { + if (polling) return; + polling = true; + callDBus(SERVICE, PATH, IFACE, "NextCommand", reply => { + polling = false; + if (reply) { + try { + JSON.parse(reply).forEach(run); + } catch (e) { + print("frametop-float: bad command " + reply + ": " + e); + } + } + poll(); + }); +} + +send({ev: "hello"}); +poll(); diff --git a/float/ft-float b/float/ft-float new file mode 100755 index 0000000..ebd83c9 --- /dev/null +++ b/float/ft-float @@ -0,0 +1,25 @@ +#!/usr/bin/env python3 +"""ft-float: talk to ft-floatd (floating windows in the Frametop desktop). + + ft-float float [ID|active] float a window (default: the active one; for it, a toggle) + ft-float dock [ID|active] put a floating window back on the desktop + ft-float close ID close a window + ft-float list the spare outputs and what floats on them +FT_FLOAT_SOCKET names ft-floatd's socket (default frametop_float). +""" +import os +import socket +import sys + +if len(sys.argv) < 2 or sys.argv[1] in ("-h", "--help"): + sys.exit(__doc__) +s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) +s.bind("") +s.settimeout(5) +try: + s.sendto(" ".join(sys.argv[1:]).encode(), "\0" + os.environ.get("FT_FLOAT_SOCKET", "frametop_float")) + reply = s.recv(8192).decode() +except OSError as e: + sys.exit(f"ft-floatd didn't answer ({e}); is the Frametop desktop running?") +print(reply) +sys.exit(0 if reply.startswith("ok") else 1) diff --git a/float/ft-floatd b/float/ft-floatd new file mode 100755 index 0000000..3b32693 --- /dev/null +++ b/float/ft-floatd @@ -0,0 +1,3 @@ +#!/bin/bash +# ft-floatd on the Frame host, inside the Frametop desktop's session (see ft_floatd.py). +exec python3 "$(dirname "$(readlink -f "$0")")/ft_floatd.py" "$@" diff --git a/float/ft_floatd.py b/float/ft_floatd.py new file mode 100644 index 0000000..caf5c36 --- /dev/null +++ b/float/ft_floatd.py @@ -0,0 +1,523 @@ +#!/usr/bin/env python3 +"""ft-floatd: floating windows for the Frametop desktop (see docs/floating-windows.md). + +Runs inside the desktop's Plasma session (its D-Bus and Wayland). It keeps the table of +which window floats on which spare output and panel, and connects three parts: + - the KWin script frametop-float (float/frametop-float.js), which it loads into KWin. The + script sends events over D-Bus (org.frametop.Float.Event) and fetches commands with a + long poll (NextCommand). + - ft-screens, through its control socket (@ft_screens): the spare output's size, and the + floating panel's crop, density, place, and popups. + - kscreen-doctor, to turn spare outputs on and off and place them in KWin's layout. +Commands and ft-screens' events arrive as datagrams on @frametop_float (ft-float is the +command-line side). Replies go to the sender: + float [ID|active] dock [ID|active] close ID list quit (ft-float) + dock N | close N | resize N W H | carried N (ft-screens, N = its screen) + +Spare outputs are WL-<screens> .. WL-<screens + slots - 1>. A floating window's output is its +frame plus a margin on each side (FLOAT_MARGIN pixels), so menus have room; the panel shows +only the window. Spares are placed apart from the screens and from each other in KWin's +layout, all within Xwayland's 32767-pixel limit. + +Usage: ft-floatd [--screens N] [--slots N] [--margin PX] [--control NAME] [--socket NAME] +Defaults: FT_SCREEN_COUNT (from the session) or the layout's count, FLOAT_SLOTS and +FLOAT_MARGIN from ~/.config/frametop.conf (8 and 300), @ft_screens, @frametop_float. +""" +import argparse +import json +import math +import os +import re +import socket +import subprocess +import sys +import time + +import dbus +import dbus.mainloop.glib +import dbus.service +from gi.repository import GLib + +HERE = os.path.dirname(os.path.realpath(__file__)) +sys.path.insert(0, os.path.join(HERE, "..", "layout")) +import ft_layout # noqa: E402 (config and layout) + +SERVICE = IFACE = "org.frametop.Float" +PATH = "/Float" +SCRIPT = "frametop-float" +POLL_SECONDS = 20 # NextCommand answers empty after this (KWin's D-Bus timeout is 25 s) +SPARE_X, SPARE_CELL = 12000, 5000 # spares in KWin's layout: a grid from here, 4 across +PULL_OUT = 0.05 # a floated window starts this far in front of its screen (metres) +DEFAULT_MPP = 1.6 / 1920 # metres per pixel when ft-screens can't say (no SteamVR) + + +def log(*args): + print(time.strftime("%H:%M:%S"), *args, flush=True) + + +class Screens: + """ft-screens' control socket.""" + + def __init__(self, name): + self.address = "\0" + name + self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) + self.sock.bind("") + + def ask(self, text, quiet=False): + self.sock.settimeout(2.0) + try: + self.sock.sendto(text.encode(), self.address) + reply = self.sock.recv(8192).decode() + except OSError as e: + reply = f"error no answer ({e})" + if not reply.startswith("ok") and not quiet: + log(f"ft-screens: {text}: {reply}") + return reply + + +def kscreen(*args): + try: + r = subprocess.run(["kscreen-doctor", *args], capture_output=True, text=True, timeout=20) + return r.stdout + except (OSError, subprocess.TimeoutExpired) as e: + log(f"kscreen-doctor: {e}") + return "" + + +def output_scales(): + try: + data = json.loads(kscreen("-j") or "{}") + except ValueError: + return {} + return {o["name"]: float(o.get("scale", 1)) for o in data.get("outputs", []) if o.get("name")} + + +class Float: + """A floating window.""" + + def __init__(self, wid, slot, saved): + self.id = wid + self.slot = slot # Slot + self.saved = saved # where it came from: output, frame, onAllDesktops + self.scale = 1.0 # its output's scale (pixels per logical unit) + self.mpp = DEFAULT_MPP # metres per pixel on its panel + self.frame = None # last frame (logical, global) + self.client = None + self.full = False # full screen: no margin + self.move_from = None # frame when a title-bar move started (put back after) + self.subs = {} # popup or dialog id -> number on the panel + + +class Slot: + def __init__(self, k, screens): + self.k = k + self.output = f"WL-{screens + k}" + self.index = screens + k + 1 # ft-screens' number (1-based) + self.pos = (SPARE_X + (k % 4) * SPARE_CELL, (k // 4) * SPARE_CELL) + self.size = None # its output's size in pixels, as last set + self.window = None # Float + + +class Daemon: + def __init__(self, args): + self.screens_n = args.screens + self.margin = args.margin + self.slots = [Slot(k, args.screens) for k in range(args.slots)] + self.floats = {} # window id -> Float + self.windows = {} # window id -> last info from the script + self.pending = [] # commands for the script + self.waiter = None # (reply callback, timeout source) while the script waits + self.screens = Screens(args.control) + self.sub_numbers = {} # popup/dialog id -> (window id, number) + self.next_sub = 1 + + # ------------------------------------------------------------ the script + + def command(self, **cmd): + self.pending.append(cmd) + self.flush() + + def flush(self): + if not self.waiter or not self.pending: + return + reply, source = self.waiter + self.waiter = None + GLib.source_remove(source) + text, self.pending = json.dumps(self.pending), [] + reply(text) + + def wait(self, reply): + if self.waiter: # a stale poll (the script reloaded): let it go + old, source = self.waiter + GLib.source_remove(source) + old("") + + def timeout(): + if self.waiter and self.waiter[0] is reply: + self.waiter = None + reply("") + return False + self.waiter = (reply, GLib.timeout_add_seconds(POLL_SECONDS, timeout)) + self.flush() + + def load_script(self, bus): + kwin = dbus.Interface(bus.get_object("org.kde.KWin", "/Scripting"), "org.kde.kwin.Scripting") + if kwin.isScriptLoaded(SCRIPT): + kwin.unloadScript(SCRIPT) + sid = int(kwin.loadScript(os.path.join(HERE, "frametop-float.js"), SCRIPT, signature="ss")) + if sid < 0: + raise RuntimeError("KWin didn't load the script") + bus.get_object("org.kde.KWin", f"/Scripting/Script{sid}").run(dbus_interface="org.kde.kwin.Script") + log(f"script loaded ({sid})") + + # ------------------------------------------------------------ events from the script + + def on_event(self, ev): + kind = ev.get("ev") + wid = ev.get("id", "") + if kind == "hello": + self.command(cmd="config", screens=self.screens_n) + # The script reports every window after "config"; spares nothing floats on are off. + GLib.timeout_add(1500, self.disable_unused) + return + if kind == "removed": + self.windows.pop(wid, None) + if wid in self.floats: + log(f"{wid[:9]} closed") + self.release(self.floats.pop(wid)) + self.drop_sub(wid) + return + if wid: + self.windows[wid] = ev + f = self.floats.get(wid) + if kind == "float-request": + self.float_window(ev) + elif kind == "dock-request": + if f: + self.dock(f) + elif kind in ("added", "window", "output"): + self.seen(ev) + elif kind == "geometry": + if f: + self.follow(f, ev) + else: + self.sub(ev) + elif kind == "move-start" and f and ev.get("move"): + f.move_from = ev["frame"] + self.screens.ask(f"carry {f.slot.index}", quiet=True) + elif kind == "move-end" and f and f.move_from: + # The panel carried the window; KWin may have slipped it a few pixels first. + m, f.move_from = f.move_from, None + if (m["x"], m["y"]) != (ev["frame"]["x"], ev["frame"]["y"]): + self.command(cmd="geometry", id=f.id, x=m["x"], y=m["y"], w=ev["frame"]["w"], h=ev["frame"]["h"]) + elif kind == "fullscreen" and f: + f.full = bool(ev.get("fullScreen")) + self.follow(f, ev, refit=True) + elif kind == "minimized" and f: + self.screens.ask(f"minimized {f.slot.index} {1 if ev.get('minimized') else 0}", quiet=True) + + def spare_slot(self, output): + for s in self.slots: + if s.output == output: + return s + return None + + def seen(self, ev): + """A window the script told us about: is it somewhere it shouldn't be?""" + wid = ev["id"] + slot = self.spare_slot(ev.get("output", "")) + f = self.floats.get(wid) + if f and f.slot is not slot and ev["ev"] == "output": + # Left its spare (KWin moved it, or docking): it's not floating any more. + if slot is None: + log(f"{wid[:9]} left its floating output") + del self.floats[wid] + self.release(f) + return + if slot is None or f: + return + if ev.get("popup") or (ev.get("transient") and ev.get("parent") in self.floats): + self.sub(ev) + return + if not ev.get("normal"): + return + if slot.window is None: + # Floating when ft-floatd (re)started: take it over where it is. + f = Float(wid, slot, None) + slot.window = f + self.floats[wid] = f + f.scale = output_scales().get(slot.output, 1.0) + log(f"{wid[:9]} already floats on {slot.output}") + self.follow(f, ev, refit=True) + return + # A new window that opened on a floating window's output: windows of floating apps + # float too; anything else goes to the screens. + if ev["ev"] == "added" and any(o.saved is not None and self.windows.get(o.id, {}).get("pid") == ev.get("pid") + for o in self.floats.values()): + self.float_window(ev) + else: + self.command(cmd="place", id=wid, output="WL-0", x=ev["frame"]["x"] % 400 + 100, + y=ev["frame"]["y"] % 300 + 100, w=ev["frame"]["w"], h=ev["frame"]["h"]) + + # ------------------------------------------------------------ floating and docking + + def disable_unused(self): + off = [f"output.{s.output}.disable" for s in self.slots if s.window is None] + if off: + kscreen(*off) + return False + + def free_slot(self): + for s in self.slots: + if s.window is None: + return s + return None + + def screen_mpp(self, output): + """Metres per pixel on the screen showing this output, from ft-screens.""" + m = re.match(r"WL-(\d+)$", output or "") + reply = self.screens.ask("screens", quiet=True) + if m and reply.startswith("ok"): + for part in reply.split()[2:]: + idx, size, metres = part.split(":") + if int(idx) == int(m.group(1)) + 1: + return float(metres) / max(1, int(size.split("x")[0])) + return DEFAULT_MPP + + def float_window(self, ev): + wid = ev["id"] + if wid in self.floats: + return + slot = self.free_slot() + if slot is None: + self.notify(f"All {len(self.slots)} floating windows are in use. Put one back on the desktop " + "to float another.") + return + f = Float(wid, slot, {"output": ev["output"], "frame": ev["frame"], "onAllDesktops": ev.get("onAllDesktops")}) + slot.window = f + self.floats[wid] = f + f.scale = output_scales().get(ev["output"], 1.0) + f.mpp = self.screen_mpp(ev["output"]) + fr, s, m = ev["frame"], f.scale, self.margin + w, h = round(fr["w"] * s), round(fr["h"] * s) + slot.size = (w + 2 * m, h + 2 * m) + log(f"{wid[:9]} ({ev.get('cls')}) floats on {slot.output}: {w}x{h} px, scale {s:g}") + # The spare's size first (while it's off, so its first frame is right), then its panel, + # then turn it on, then the window. + self.screens.ask(f"size {slot.index} {slot.size[0]} {slot.size[1]}") + self.set_panel(f, (m, m, w, h), title=round((ev["client"]["y"] - fr["y"]) * s)) + self.place_panel(f, ev) + kscreen(f"output.{slot.output}.enable", f"output.{slot.output}.scale.{s:g}", + f"output.{slot.output}.position.{slot.pos[0]},{slot.pos[1]}") + self.command(cmd="place", id=wid, output=slot.output, x=slot.pos[0] + m / s, y=slot.pos[1] + m / s, + w=fr["w"], h=fr["h"], onAllDesktops=True) + + def set_panel(self, f, crop, title=0): + x, y, w, h = crop + self.screens.ask(f"float {f.slot.index} {f.mpp:.7f} {x} {y} {w} {h} {title}") + + def place_panel(self, f, ev): + """Put the panel where the window was on its screen, a little in front of it.""" + m = re.match(r"WL-(\d+)$", ev["output"]) + reply = self.screens.ask(f"get {int(m.group(1)) + 1}", quiet=True) if m else "" + if not reply.startswith("ok"): + return + g = ft_layout.parse_get(reply) + c, xa, ya, za = g["center"], g["x"], g["y"], g["z"] + out, fr, s = ev["outputRect"], ev["frame"], f.scale + # The window's centre relative to the screen's, in panel pixels, then metres. + dx = ((fr["x"] - out["x"]) + fr["w"] / 2 - out["w"] / 2) * s * f.mpp + dy = ((fr["y"] - out["y"]) + fr["h"] / 2 - out["h"] / 2) * s * f.mpp + p = [c[k] + xa[k] * dx - ya[k] * dy + za[k] * PULL_OUT for k in range(3)] + rows = [f"{xa[k]:.5f} {ya[k]:.5f} {za[k]:.5f} {p[k]:.4f}" for k in range(3)] + self.screens.ask(f"pose {f.slot.index} {' '.join(rows)}") + + def follow(self, f, ev, refit=False): + """The window moved or resized on its output: crop the panel to it, and keep the + output its size plus the margin.""" + slot, s = f.slot, f.scale + fr, cl, out = ev["frame"], ev["client"], ev["outputRect"] + f.frame, f.client = fr, cl + m = 0 if f.full else self.margin + w, h = round(fr["w"] * s), round(fr["h"] * s) + want = (w + 2 * m, h + 2 * m) + if f.full: + want = slot.size or want + if refit or want != slot.size: + if not f.full and want != slot.size: + slot.size = want + self.screens.ask(f"size {slot.index} {want[0]} {want[1]}") + x0, y0 = slot.pos[0] + m / s, slot.pos[1] + m / s + if not f.full and (abs(fr["x"] - x0) > 0.5 or abs(fr["y"] - y0) > 0.5): + self.command(cmd="geometry", id=f.id, x=x0, y=y0, w=fr["w"], h=fr["h"]) + return # the next geometry event crops the panel + x, y = round((fr["x"] - out["x"]) * s), round((fr["y"] - out["y"]) * s) + self.set_panel(f, (x, y, w, h), title=0 if f.full else round((cl["y"] - fr["y"]) * s)) + + def dock(self, f, frame=None): + """Back where it came from (or onto screen 1 if we don't know).""" + saved = f.saved or {"output": "WL-0", "frame": dict(f.frame or {"x": 100, "y": 100, "w": 800, "h": 600}), + "onAllDesktops": False} + fr = frame or saved["frame"] + log(f"{f.id[:9]} back to {saved['output']}") + self.command(cmd="place", id=f.id, output=saved["output"], x=fr["x"], y=fr["y"], w=fr["w"], h=fr["h"], + onAllDesktops=bool(saved.get("onAllDesktops"))) + + def release(self, f): + """Its window left: hide the panel and turn the spare off.""" + slot = f.slot + if slot.window is f: + slot.window = None + for sub_id in list(f.subs): + self.drop_sub(sub_id) + self.screens.ask(f"unfloat {slot.index}", quiet=True) + kscreen(f"output.{slot.output}.disable") + + # ------------------------------------------------------------ popups and dialogs + + def sub(self, ev): + parent = self.floats.get(ev.get("parent", "")) + if parent is None: + # A popup of a popup: its top-level parent is the floating window. + known = self.sub_numbers.get(ev.get("parent", "")) + parent = self.floats.get(known[0]) if known else None + if parent is None: + return + wid = ev["id"] + if wid not in self.sub_numbers: + self.sub_numbers[wid] = (parent.id, self.next_sub) + parent.subs[wid] = self.next_sub + self.next_sub += 1 + n = self.sub_numbers[wid][1] + fr, out, s = ev["frame"], ev["outputRect"], parent.scale + x, y = round((fr["x"] - out["x"]) * s), round((fr["y"] - out["y"]) * s) + self.screens.ask(f"sub {parent.slot.index} {n} {x} {y} {round(fr['w'] * s)} {round(fr['h'] * s)}", quiet=True) + + def drop_sub(self, wid): + known = self.sub_numbers.pop(wid, None) + if not known: + return + parent = self.floats.get(known[0]) + if parent: + parent.subs.pop(wid, None) + self.screens.ask(f"sub {parent.slot.index} {known[1]} off", quiet=True) + + # ------------------------------------------------------------ requests on @frametop_float + + def by_panel(self, index): + for s in self.slots: + if s.index == index: + return s.window + return None + + def request(self, text): + words = text.split() + if not words: + return "error empty" + cmd, rest = words[0], words[1:] + if cmd == "list": + return "ok " + " ".join(f"{s.output}:{s.window.id if s.window else '-'}" for s in self.slots) + if cmd == "quit": + GLib.idle_add(self.loop.quit) + return "ok" + if cmd in ("float", "dock") and (not rest or rest[0] == "active"): + self.command(cmd="request-active") + return "ok" + if cmd in ("dock", "close", "resize", "carried") and rest and rest[0].isdigit(): + f = self.by_panel(int(rest[0])) + if not f: + return f"error no floating window on screen {rest[0]}" + if cmd == "dock": + self.dock(f) + elif cmd == "close": + self.command(cmd="close", id=f.id) + elif cmd == "resize" and len(rest) == 3 and f.frame: + w, h = max(320, int(rest[1])), max(200, int(rest[2])) + self.command(cmd="geometry", id=f.id, x=f.frame["x"], y=f.frame["y"], w=w / f.scale, h=h / f.scale) + return "ok" + if cmd in ("float", "dock", "close") and rest: + ev = self.windows.get(rest[0]) + if not ev: + return f"error no window {rest[0]}" + if cmd == "float": + self.float_window(ev) + elif cmd == "dock" and rest[0] in self.floats: + self.dock(self.floats[rest[0]]) + elif cmd == "close": + self.command(cmd="close", id=rest[0]) + return "ok" + return "error unknown command" + + def notify(self, text): + log(text) + try: + n = dbus.Interface(dbus.SessionBus().get_object("org.freedesktop.Notifications", + "/org/freedesktop/Notifications"), + "org.freedesktop.Notifications") + n.Notify("Frametop", 0, "window-new", "Floating windows", text, [], {}, 5000) + except dbus.DBusException as e: + log(f"notification: {e.get_dbus_message()}") + + +class Service(dbus.service.Object): + def __init__(self, bus, daemon): + super().__init__(dbus.service.BusName(SERVICE, bus), PATH) + self.daemon = daemon + + @dbus.service.method(IFACE, in_signature="s", out_signature="") + def Event(self, text): + try: + self.daemon.on_event(json.loads(text)) + except (ValueError, KeyError, TypeError) as e: + log(f"bad event {text[:200]}: {e!r}") + + @dbus.service.method(IFACE, in_signature="", out_signature="s", async_callbacks=("reply", "error")) + def NextCommand(self, reply, error): + self.daemon.wait(reply) + + +def main(): + conf = ft_layout.read_conf() + p = argparse.ArgumentParser(description="Floating windows for the Frametop desktop") + p.add_argument("--screens", type=int, default=int(os.environ.get("FT_SCREEN_COUNT") or 0)) + p.add_argument("--slots", type=int, default=int(conf.get("FLOAT_SLOTS") or 8)) + p.add_argument("--margin", type=int, default=int(conf.get("FLOAT_MARGIN") or 300)) + p.add_argument("--control", default="ft_screens") + p.add_argument("--socket", default="frametop_float") + args = p.parse_args() + if args.screens <= 0: + args.screens = ft_layout.screen_count() + args.slots = max(0, min(16, args.slots)) + args.margin = max(0, min(1000, args.margin)) + + dbus.mainloop.glib.DBusGMainLoop(set_as_default=True) + bus = dbus.SessionBus() + daemon = Daemon(args) + service = Service(bus, daemon) # noqa: F841 (keeps the name) + daemon.loop = GLib.MainLoop() + + sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM) + sock.bind("\0" + args.socket) + sock.setblocking(False) + + def readable(*_): + while True: + try: + data, sender = sock.recvfrom(4096) + except BlockingIOError: + return True + reply = daemon.request(data.decode(errors="replace").strip()) + if sender: + try: + sock.sendto(reply.encode(), sender) + except OSError: + pass + GLib.io_add_watch(sock.fileno(), GLib.IO_IN, readable) + + log(f"{args.screens} screens, {args.slots} floating slots (WL-{args.screens} and up), margin {args.margin} px") + daemon.load_script(bus) + daemon.loop.run() + + +if __name__ == "__main__": + main() diff --git a/screens/test/headless.sh b/screens/test/headless.sh index a0952c5..d2f5bc1 100755 --- a/screens/test/headless.sh +++ b/screens/test/headless.sh @@ -16,7 +16,9 @@ # headless.sh js '<javascript>' run a one-off KWin script; its print()s come back # headless.sh shot [OUTPUT] a JPEG of one output (default WL-0); from the PC it # lands in captures/ and its path is printed -# headless.sh log [kwin|screens|apps] [N] the last N lines (default 30) +# headless.sh floatd start ft-floatd in it (socket @frametop_float_test) +# headless.sh float <ft-float args> e.g. "float active", list +# headless.sh log [kwin|screens|apps|floatd] [N] the last N lines (default 30) set -euo pipefail here=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd) . "$here/../../scripts/_env.sh" @@ -83,6 +85,7 @@ case "${1:-}" in rm -rf "$rt" "$cfg" mkdir -p "$rt" "$cfg" chmod 700 "$rt" + echo "$screens $spares" > "$rt/counts" args=(--screen 1920x1080@1.6 --screen 1080x1920@0.9) for ((i = 2; i < screens; i++)); do args+=(--screen 1920x1080@1.6); done args=("${args[@]:0:$((screens * 2))}") @@ -108,6 +111,17 @@ case "${1:-}" in ask toplevels ;; stop) stop ;; + floatd) + pkill -f -- "--socket frametop_float_test" 2>/dev/null || true + read -r screens spares < "$rt/counts" + nohup env XDG_RUNTIME_DIR="$rt" XDG_CONFIG_HOME="$cfg" DBUS_SESSION_BUS_ADDRESS="$bus" \ + WAYLAND_DISPLAY=wayland-0 "$REPO_ROOT/float/ft-floatd" --screens "$screens" --slots "$spares" \ + --control ft_screens_test --socket frametop_float_test \ + > "$logs/floatd.log" 2>&1 & + sleep 2 + tail -3 "$logs/floatd.log" + ;; + float) shift; FT_FLOAT_SOCKET=frametop_float_test "$REPO_ROOT/float/ft-float" "$@" ;; ask) ask "$2" ;; kd) shift; kd "$@" ;; run) From c2e5e861c893443cbed0920308752489dfe0dfe9 Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:26:20 -0600 Subject: [PATCH 08/13] Show floating windows as panels of their own ft-screens makes a panel for each spare output (frametop.float.N), hidden until ft-floatd floats a window on it. The panel shows only the window's rectangle of the buffer at the density of the screen it came from, its popups and dialogs get small panels over it, pressing its title bar carries it while KWin's pointer stays put, the corner tab resizes the window in pixels, and two more buttons close it and put it back on the desktop. The session adds FLOAT_SLOTS spare outputs to KWin and starts ft-floatd; ft-layout arranges only the screens' outputs, and the pointer helper treats the new panels like screens. Not yet tried in the headset. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- float/ft_floatd.py | 1 + layout/ft_layout.py | 7 +- pointer/helper/ft-pointer.cpp | 8 +- screens/compositor.c | 34 ++- screens/test/headless.sh | 4 +- screens/vr.cpp | 463 ++++++++++++++++++++++++++++++---- screens/vr.h | 4 + session/frametop-session.sh | 32 ++- session/frametop.conf.example | 2 + 9 files changed, 480 insertions(+), 75 deletions(-) diff --git a/float/ft_floatd.py b/float/ft_floatd.py index caf5c36..3137822 100644 --- a/float/ft_floatd.py +++ b/float/ft_floatd.py @@ -305,6 +305,7 @@ class Daemon: # The spare's size first (while it's off, so its first frame is right), then its panel, # then turn it on, then the window. self.screens.ask(f"size {slot.index} {slot.size[0]} {slot.size[1]}") + self.screens.ask(f"scale {slot.index} {s:g}") # for pointer positions (KWin's units) self.set_panel(f, (m, m, w, h), title=round((ev["client"]["y"] - fr["y"]) * s)) self.place_panel(f, ev) kscreen(f"output.{slot.output}.enable", f"output.{slot.output}.scale.{s:g}", diff --git a/layout/ft_layout.py b/layout/ft_layout.py index 601a817..50b42d1 100755 --- a/layout/ft_layout.py +++ b/layout/ft_layout.py @@ -684,13 +684,18 @@ def nested_env(): def outputs(env): - """KWin's outputs, in screen order (WL-0, WL-1, ...).""" + """KWin's outputs for the screens, in screen order (WL-0, WL-1, ...).""" try: data = json.loads(subprocess.run(["kscreen-doctor", "-j"], capture_output=True, text=True, env=env, timeout=10).stdout) except (OSError, ValueError, subprocess.TimeoutExpired): return [] outs = [o for o in data.get("outputs", []) if o.get("connected")] + if backend() == "screens": + # Outputs after the screens are spares for floating windows (ft-floatd places them). + count = screen_count() + outs = [o for o in outs if not re.fullmatch(r"WL-(\d+)", o.get("name", "")) or + int(o["name"][3:]) < count] return sorted(outs, key=lambda o: [int(t) if t.isdigit() else t for t in re.split(r"(\d+)", o.get("name", ""))]) diff --git a/pointer/helper/ft-pointer.cpp b/pointer/helper/ft-pointer.cpp index bba6729..a41b5e7 100644 --- a/pointer/helper/ft-pointer.cpp +++ b/pointer/helper/ft-pointer.cpp @@ -371,6 +371,7 @@ private: if (rest.find("Thumbnail") != std::string::npos || rest.find("Subview") != std::string::npos) continue; if (key.rfind("system.pointer", 0) == 0 || key.rfind("system.cursor", 0) == 0 || key.rfind("frametop.pointer", 0) == 0 || key.rfind("frametop.guide", 0) == 0 || + key == "frametop.catcher" || // ft-screens' release catcher: only on a laser mid-drag key == "system.HeadsetView" || key == "system.toast") continue; // The name can hold quotes; it ends at the last "', " (the size and state follow). @@ -1255,10 +1256,11 @@ int main() { overlay->GetOverlayTextureSize(h, &tw, &th); vr::VROverlayTransformType tt = vr::VROverlayTransform_Invalid; overlay->GetOverlayTransformType(h, &tt); - // ft-screens' panels (frametop.screen.N) are 0x0 and absolute too (a shared - // texture), but they're real panels of any size. + // ft-screens' panels (frametop.screen.N, and floating windows with their + // popups, frametop.float.N...) are 0x0 and absolute too (a shared texture), + // but they're real panels of any size. sceneGraph[key] = (tw == 0 || th == 0) && tt == vr::VROverlayTransform_Absolute && - key.rfind("frametop.screen.", 0) != 0; + key.rfind("frametop.screen.", 0) != 0 && key.rfind("frametop.float.", 0) != 0; } ours = vr::k_unTrackedDeviceIndexInvalid; for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) { diff --git a/screens/compositor.c b/screens/compositor.c index ac48cfc..10affcc 100644 --- a/screens/compositor.c +++ b/screens/compositor.c @@ -13,13 +13,15 @@ // scale first ("scale <screen> <s>", from ft-layout). // // Usage: ft-screens [--socket NAME] [--control NAME] [--no-vr] [--screen WxH@METRES]... -// [-- COMMAND ARGS...] +// [--spares N] [-- COMMAND ARGS...] // --socket Wayland socket name in $XDG_RUNTIME_DIR (default ft-screens-0) // --control the control socket's abstract name (default ft_screens) // --no-vr run without SteamVR, for tests next to the running desktop: no panels, no // input, and nothing sent to the input relay. Commands still work, and // "toplevels" shows what KWin opened. // --screen one per screen, in KWin's order (default: 3440x1440@2.4) +// --spares KWin's outputs after the screens: spares for floating windows (ft-floatd +// turns them on and sizes them; see docs/floating-windows.md) // COMMAND run with WAYLAND_DISPLAY set to our socket (e.g. the Frametop session) // Runs in the dev container (wlroots 0.20); KWin connects from the host. #define _GNU_SOURCE @@ -57,7 +59,7 @@ #include "vr.h" -#define MAX_SCREENS 8 +#define MAX_SCREENS 24 // screens and spare outputs struct config { int width, height; @@ -97,7 +99,8 @@ struct server { struct screen *screens[MAX_SCREENS]; struct config config[MAX_SCREENS]; double scale[MAX_SCREENS]; // KWin's scale for each screen (panel pixels per logical unit) - int n_config, n_screens; + int n_config, n_screens; // n_config: the screens; the outputs after them are spares + int spares; struct wl_list buffers; // tracked_buffer struct wl_event_source *tick; struct screen *pointer_focus; @@ -145,7 +148,8 @@ static void screen_commit(struct wl_listener *l, void *data) { struct wlr_xdg_surface *xdg = sc->toplevel->base; if (xdg->initial_commit) { // First commit: tell KWin the size of this screen. - const struct config *c = &sc->server->config[sc->index < sc->server->n_config ? sc->index : 0]; + const struct config spare = {640, 480, 0.5}; // until ft-floatd sizes it + const struct config *c = sc->index < sc->server->n_config ? &sc->server->config[sc->index] : &spare; wlr_xdg_toplevel_set_size(sc->toplevel, c->width, c->height); wlr_xdg_toplevel_set_activated(sc->toplevel, true); if (sc->decoration) @@ -205,7 +209,9 @@ static void screen_destroy(struct wl_listener *l, void *data) { // after it while that output is off. static void screen_title(struct wl_listener *l, void *data) { struct screen *sc = wl_container_of(l, sc, set_title); - wlr_log(WLR_INFO, "screen %d: \"%s\"", sc->index + 1, sc->toplevel->title ? sc->toplevel->title : ""); + const char *title = sc->toplevel->title ? sc->toplevel->title : ""; + wlr_log(WLR_INFO, "screen %d: \"%s\"", sc->index + 1, title); + if (sc->index >= sc->server->n_config) ft_vr_float_output(sc->index, !strstr(title, "Output disabled")); } static void new_toplevel(struct wl_listener *l, void *data) { @@ -222,9 +228,15 @@ static void new_toplevel(struct wl_listener *l, void *data) { sc->index = index; sc->toplevel = toplevel; s->screens[index] = sc; - const struct config *c = &s->config[index < s->n_config ? index : 0]; - wlr_log(WLR_INFO, "screen %d: KWin window, %dx%d, %.2f m wide", index + 1, c->width, c->height, c->metres); - ft_vr_screen_create(index, c->metres, s->n_config > index + 1 ? s->n_config : index + 1); + if (index >= s->n_config) { + wlr_log(WLR_INFO, "screen %d: KWin window, a spare output (floating window %d)", index + 1, + index - s->n_config + 1); + ft_vr_float_create(index, index - s->n_config + 1); + } else { + const struct config *c = &s->config[index]; + wlr_log(WLR_INFO, "screen %d: KWin window, %dx%d, %.2f m wide", index + 1, c->width, c->height, c->metres); + ft_vr_screen_create(index, c->metres, s->n_config); + } sc->commit.notify = screen_commit; wl_signal_add(&toplevel->base->surface->events.commit, &sc->commit); sc->destroy.notify = screen_destroy; @@ -537,6 +549,8 @@ int main(int argc, char **argv) { socket_name = argv[++i]; } else if (strcmp(argv[i], "--control") == 0 && i + 1 < argc) { control_name = argv[++i]; + } else if (strcmp(argv[i], "--spares") == 0 && i + 1 < argc) { + s.spares = atoi(argv[++i]); } else if (strcmp(argv[i], "--no-vr") == 0) { s.vr = false; } else if (strcmp(argv[i], "--screen") == 0 && i + 1 < argc && s.n_config < MAX_SCREENS) { @@ -554,7 +568,7 @@ int main(int argc, char **argv) { } else { fprintf(stderr, "usage: %s [--socket NAME] [--control NAME] [--no-vr] [--screen WxH@METRES]... " - "[-- COMMAND ARGS...]\n", + "[--spares N] [-- COMMAND ARGS...]\n", argv[0]); return 2; } @@ -599,7 +613,7 @@ int main(int argc, char **argv) { } char path[256]; snprintf(path, sizeof path, "%s/%s", getenv("XDG_RUNTIME_DIR") ? getenv("XDG_RUNTIME_DIR") : "/tmp", socket_name); - wlr_log(WLR_INFO, "listening on %s; %d screen(s) configured", path, s.n_config); + wlr_log(WLR_INFO, "listening on %s; %d screen(s) configured, %d spare(s)", path, s.n_config, s.spares); const int control = open_control_socket(control_name); if (control < 0) return 1; diff --git a/screens/test/headless.sh b/screens/test/headless.sh index d2f5bc1..7870fb5 100755 --- a/screens/test/headless.sh +++ b/screens/test/headless.sh @@ -6,7 +6,7 @@ # or on the Frame. Build ft-screens first (screens/build.sh). # # headless.sh start [SCREENS] [SPARES] (default 2 screens, 1920x1080 and 1080x1920, and -# 3 spare 800x600 outputs, disabled once KWin is up) +# 3 spare outputs, disabled once KWin is up) # headless.sh stop # headless.sh ask '<ft-screens command>' e.g. toplevels, "input 1 down 400 20" # headless.sh kd <kscreen-doctor args> e.g. -o, output.WL-2.enable @@ -89,7 +89,7 @@ case "${1:-}" in args=(--screen 1920x1080@1.6 --screen 1080x1920@0.9) for ((i = 2; i < screens; i++)); do args+=(--screen 1920x1080@1.6); done args=("${args[@]:0:$((screens * 2))}") - for ((i = 0; i < spares; i++)); do args+=(--screen 800x600); done + args+=(--spares "$spares") cd "$REPO_ROOT" nohup "$HOME/.local/bin/distrobox" enter "$FRAME_BOX" -- env XDG_RUNTIME_DIR="$rt" \ ./screens/build/ft-screens --no-vr --socket ft-test-0 --control ft_screens_test "${args[@]}" \ diff --git a/screens/vr.cpp b/screens/vr.cpp index 04c4af9..755f058 100644 --- a/screens/vr.cpp +++ b/screens/vr.cpp @@ -42,6 +42,15 @@ // or stay visible over it; the hotkey still shows them. // - the catcher: a button pressed on a screen is released in KWin even when the laser // lets go between panels (UpdateCatcher). +// - floating windows (docs/floating-windows.md): KWin's spare outputs, after the screens, +// are panels too, for one window each. ft-floatd sizes the output to the window plus a +// margin and tells us the window's rectangle ("float"): the panel shows only that crop of +// the buffer (SetOverlayTextureBounds), at the density of the screen it came from, and +// each popup or dialog gets a small panel of its own over it, cut from the same buffer +// ("sub"). Pressing its title bar carries the panel like the bar does, while KWin's +// pointer stays put, so the window doesn't move on its output. The corner tab resizes the +// window (in pixels, at the same density) instead of scaling the panel, and two more +// buttons close it and put it back on the desktop (both through ft-floatd). // OpenVR has no overlay-relative transforms here (openvr v2.15.6), so the bar, button, // and handle are placed whenever their screen moves. #include "vr.h" @@ -49,6 +58,8 @@ #include <openvr.h> #include <fcntl.h> +#include <sys/socket.h> +#include <sys/un.h> #include <linux/input-event-codes.h> #include <limits.h> #include <spawn.h> @@ -59,6 +70,7 @@ extern char **environ; // for posix_spawn #include <chrono> #include <array> #include <cmath> +#include <cstddef> #include <cstdio> #include <cstdlib> #include <cstring> @@ -209,10 +221,17 @@ long g_tick = 0; // ft_vr_poll calls bool g_vr = false; // connected to SteamVR (ft-screens --no-vr runs without it) constexpr vr::TrackedDeviceIndex_t kNone = vr::k_unTrackedDeviceIndexInvalid; +// A popup or dialog of a floating window: a small panel over it, cut from the same buffer. +struct Sub { + vr::VROverlayHandle_t overlay = vr::k_ulOverlayHandleInvalid; + int x = 0, y = 0, w = 0, h = 0; // in the output's buffer, pixels +}; + struct Screen { vr::VROverlayHandle_t overlay = vr::k_ulOverlayHandleInvalid, bar = vr::k_ulOverlayHandleInvalid, handle = vr::k_ulOverlayHandleInvalid, curveButton = vr::k_ulOverlayHandleInvalid, - rollButton = vr::k_ulOverlayHandleInvalid; + rollButton = vr::k_ulOverlayHandleInvalid, dockButton = vr::k_ulOverlayHandleInvalid, + closeButton = vr::k_ulOverlayHandleInvalid; // the last two: floating windows int width = 0, height = 0; // current buffer size (mouse scale) double metres = 1; double curve = 0; // cylinder radius in metres; 0 = flat @@ -228,7 +247,7 @@ struct Screen { double grabX = 0, grabY = 0; // resize: the grab point relative to the corner Mat rollFrom = Identity(); // roll: the pose at the press (pinRel when pinned) double rollAngle = 0; // roll: the laser's angle around the centre then - bool hover[4] = {}; // a laser is on the bar, curve, roll, resize control + bool hover[6] = {}; // a laser is on the bar, curve, roll, resize, dock, close control bool lasers = true; // MakeOverlaysInteractiveIfVisible is set float controls = 0; // the controls' fade, 0 (hidden) .. 1 bool controlsUp = false; // the controls' overlays are shown @@ -238,9 +257,35 @@ struct Screen { bool barLit = false; double chrome = 0.3; // the bar's width; the other controls follow it (ChromeSize) double grip = 0.04; // the corner tab's and the round buttons' size - double heightMetres() const { return width > 0 ? metres * height / width : metres * 9 / 16; } - std::array<vr::VROverlayHandle_t, 4> Controls() const { return {bar, curveButton, rollButton, handle}; } - std::array<vr::VROverlayHandle_t, 5> All() const { return {overlay, bar, curveButton, rollButton, handle}; } + // A floating window's panel (see the top): the window's rectangle in the buffer, its + // title bar's height there, and the density. + bool floating = false; // a spare output's panel + bool floatOn = false; // ft-floatd has a window on it ("float" .. "unfloat") + bool outputOn = false; // KWin has the spare output turned on + bool minimized = false; + int cropX = 0, cropY = 0, cropW = 0, cropH = 0; + int titleH = 0; + double mpp = 0; // metres per buffer pixel + bool titleCarry = false; // carried by its title bar: KWin's pointer stays at carryX, carryY + double carryX = 0, carryY = 0; + long resizeSent = 0; // g_tick of the last resize request (they're throttled) + int resizeW = 0, resizeH = 0; // ...and its size + std::map<int, Sub> subs; + double heightMetres() const { + if (floating && cropW > 0) return metres * cropH / cropW; + return width > 0 ? metres * height / width : metres * 9 / 16; + } + // Buffer pixels from OpenVR's mouse position on the panel (its origin is bottom left). + void ToBuffer(double mx, double my, double *x, double *y) const { + if (floating && cropW > 0) *x = cropX + mx, *y = cropY + cropH - my; + else *x = mx, *y = height - my; + } + std::array<vr::VROverlayHandle_t, 6> Controls() const { + return {bar, curveButton, rollButton, handle, dockButton, closeButton}; + } + std::array<vr::VROverlayHandle_t, 7> All() const { + return {overlay, bar, curveButton, rollButton, handle, dockButton, closeButton}; + } }; std::map<int, Screen> g_screens; std::map<const void *, vr::SharedTextureHandle_t> g_imports; @@ -348,6 +393,29 @@ std::vector<uint8_t> RollTexture(int n) { DiscRim); } +std::vector<uint8_t> CloseTexture(int n) { + // A cross: "close this window". + return ControlTexture( + n, InDisc, + [](double u, double v) { + return std::max(std::fabs(u), std::fabs(v)) < 0.42 && + (std::fabs(u - v) < 0.12 || std::fabs(u + v) < 0.12); + }, + DiscRim); +} + +std::vector<uint8_t> DockTexture(int n) { + // An arrow down onto a line: "back to the desktop". + return ControlTexture( + n, InDisc, + [](double u, double v) { + if (std::fabs(u) < 0.5 && v > -0.52 && v < -0.38) return true; // the line + if (std::fabs(u) < 0.08 && v > -0.1 && v < 0.5) return true; // the shaft + return v >= -0.3 && v <= -0.05 && std::fabs(u) <= (v + 0.3) * 1.2; // the head, point down + }, + DiscRim); +} + vr::VROverlayHandle_t MakeChrome(const char *key, const char *name, const std::vector<uint8_t> &px, int w, int h) { vr::VROverlayHandle_t o = vr::k_ulOverlayHandleInvalid; if (vr::VROverlay()->CreateOverlay(key, name, &o) != vr::VROverlayError_None) return o; @@ -500,13 +568,37 @@ Mat BarOffset(const Screen &s) { return OnSurface(s, 0, BarY(s), 0.003); } // Put the bar, the curve button, and the corner tab under the screen (same parent: the // room or the controller), sized for the screen and its distance, and on its surface. -// Where each control sits, relative to the screen: bar, curve, roll, resize tab. -std::array<Mat, 4> ControlOffsets(const Screen &s) { +// Where each control sits, relative to the screen: bar, curve, roll, resize tab, and a +// floating window's dock and close buttons (left of the bar). +std::array<Mat, 6> ControlOffsets(const Screen &s) { const double h = s.heightMetres(), bar = s.chrome, button = s.grip, gap = bar * 0.06; return {BarOffset(s), OnSurface(s, bar / 2 + gap + button / 2, BarY(s), 0.003), OnSurface(s, bar / 2 + gap * 2 + button * 1.5, BarY(s), 0.003), // The tab's top left corner is the screen's bottom right corner. - OnSurface(s, s.metres / 2 + s.grip / 2, -(h / 2 + s.grip / 2), 0.003)}; + OnSurface(s, s.metres / 2 + s.grip / 2, -(h / 2 + s.grip / 2), 0.003), + OnSurface(s, -(bar / 2 + gap + button / 2), BarY(s), 0.003), + OnSurface(s, -(bar / 2 + gap * 2 + button * 1.5), BarY(s), 0.003)}; +} + +// A floating window's popups and dialogs, a few millimetres in front of it, where they are +// in the buffer relative to the window. +void PlaceSubs(const Screen &s) { + if (s.subs.empty() || s.cropW <= 0) return; + Mat p; + if (s.pinned == kNone && !ScreenPose(s, &p)) return; + for (const auto &[k, sub] : s.subs) { + const double u = (sub.x + sub.w / 2.0 - (s.cropX + s.cropW / 2.0)) * s.mpp; + const double v = -(sub.y + sub.h / 2.0 - (s.cropY + s.cropH / 2.0)) * s.mpp; + const Mat off = OnSurface(s, u, v, 0.005); + vr::VROverlay()->SetOverlayWidthInMeters(sub.overlay, float(std::max(0.01, sub.w * s.mpp))); + if (s.pinned != kNone) { + const Mat m = Mul(s.pinRel, off); + vr::VROverlay()->SetOverlayTransformTrackedDeviceRelative(sub.overlay, s.pinned, &m); + } else { + const Mat m = Mul(p, off); + vr::VROverlay()->SetOverlayTransformAbsolute(sub.overlay, vr::TrackingUniverseStanding, &m); + } + } } void PlaceChrome(Screen &s) { @@ -517,12 +609,19 @@ void PlaceChrome(Screen &s) { vr::VROverlay()->SetOverlayWidthInMeters(s.curveButton, float(button)); vr::VROverlay()->SetOverlayWidthInMeters(s.rollButton, float(button)); vr::VROverlay()->SetOverlayWidthInMeters(s.handle, float(s.grip)); + if (s.floating) { + vr::VROverlay()->SetOverlayWidthInMeters(s.dockButton, float(button)); + vr::VROverlay()->SetOverlayWidthInMeters(s.closeButton, float(button)); + } // Curved, the bar bends with the screen's bottom edge. vr::VROverlay()->SetOverlayCurvature(s.bar, s.curve > 0 ? float(std::min(1.0, bar / (2 * M_PI * s.curve))) : 0.f); const std::pair<vr::VROverlayHandle_t, Mat> parts[] = { - {s.bar, offsets[0]}, {s.curveButton, offsets[1]}, {s.rollButton, offsets[2]}, {s.handle, offsets[3]}}; + {s.bar, offsets[0]}, {s.curveButton, offsets[1]}, {s.rollButton, offsets[2]}, + {s.handle, offsets[3]}, {s.dockButton, offsets[4]}, {s.closeButton, offsets[5]}}; + PlaceSubs(s); if (s.pinned != kNone) { for (const auto &[o, off] : parts) { + if (o == vr::k_ulOverlayHandleInvalid) continue; const Mat m = Mul(s.pinRel, off); vr::VROverlay()->SetOverlayTransformTrackedDeviceRelative(o, s.pinned, &m); } @@ -531,6 +630,7 @@ void PlaceChrome(Screen &s) { Mat p; if (!ScreenPose(s, &p)) return; for (const auto &[o, off] : parts) { + if (o == vr::k_ulOverlayHandleInvalid) continue; const Mat m = Mul(p, off); vr::VROverlay()->SetOverlayTransformAbsolute(o, vr::TrackingUniverseStanding, &m); } @@ -637,11 +737,13 @@ bool ModeVisible() { // it's being dragged. void ApplyAlpha(const Screen &s) { vr::VROverlay()->SetOverlayAlpha(s.overlay, s.alpha); - const bool active[4] = {s.hover[0] || s.drag == Drag::Move, s.hover[1], s.hover[2] || s.drag == Drag::Roll, - s.hover[3] || s.drag == Drag::Resize}; + for (const auto &[k, sub] : s.subs) vr::VROverlay()->SetOverlayAlpha(sub.overlay, s.alpha); + const bool active[6] = {s.hover[0] || s.drag == Drag::Move, s.hover[1], s.hover[2] || s.drag == Drag::Roll, + s.hover[3] || s.drag == Drag::Resize, s.hover[4], s.hover[5]}; const auto controls = s.Controls(); - for (int k = 0; k < 4; ++k) - vr::VROverlay()->SetOverlayAlpha(controls[k], s.alpha * s.controls * (active[k] ? 1.f : kChromeIdle)); + for (int k = 0; k < 6; ++k) + if (controls[k] != vr::k_ulOverlayHandleInvalid) + vr::VROverlay()->SetOverlayAlpha(controls[k], s.alpha * s.controls * (active[k] ? 1.f : kChromeIdle)); } void SetVisible(Screen &s, bool visible, float alpha) { @@ -653,11 +755,14 @@ void SetVisible(Screen &s, bool visible, float alpha) { s.visible = visible; if (visible) { vr::VROverlay()->ShowOverlay(s.overlay); + for (const auto &[k, sub] : s.subs) vr::VROverlay()->ShowOverlay(sub.overlay); return; } // Hidden: the controls go at once (UpdateControls brings them back). vr::VROverlay()->HideOverlay(s.overlay); - for (auto o : s.Controls()) vr::VROverlay()->HideOverlay(o); + for (const auto &[k, sub] : s.subs) vr::VROverlay()->HideOverlay(sub.overlay); + for (auto o : s.Controls()) + if (o != vr::k_ulOverlayHandleInvalid) vr::VROverlay()->HideOverlay(o); s.controls = 0, s.controlsUp = false; } @@ -667,6 +772,9 @@ void UpdateVisibility() { const bool haveHead = DevicePose(vr::k_unTrackedDeviceIndex_Hmd, &head); for (auto &[i, s] : g_screens) { bool visible = s.shown && (shared || s.drag != Drag::None); + // A floating window's panel: while a window floats on it, its output is on, and the + // window isn't minimized (and once it has a crop). + if (s.floating) visible = visible && s.floatOn && s.outputOn && !s.minimized && s.cropW > 0; float alpha = 1; Mat p; if (visible && s.pinned != kNone && s.pinned != vr::k_unTrackedDeviceIndex_Hmd && s.drag == Drag::None && @@ -712,7 +820,9 @@ void UpdateControls() { const auto offsets = ControlOffsets(s); for (double f : {-0.5, -0.25, 0.0, 0.25, 0.5}) spots.push_back(Mul(p, Mul(offsets[0], Translation(f * s.chrome, 0, 0)))); - for (int k = 1; k < 4; ++k) spots.push_back(Mul(p, offsets[k])); + const auto controls = s.Controls(); + for (int k = 1; k < 6; ++k) + if (controls[k] != vr::k_ulOverlayHandleInvalid) spots.push_back(Mul(p, offsets[k])); const double reach = std::max(s.grip * 1.5, s.chrome * 0.12); for (const Mat &d : lasers) { const double o[3] = {d.m[0][3], d.m[1][3], d.m[2][3]}, dir[3] = {-d.m[0][2], -d.m[1][2], -d.m[2][2]}; @@ -730,13 +840,14 @@ void UpdateControls() { } } } - const bool inUse = s.drag != Drag::None || s.hover[0] || s.hover[1] || s.hover[2] || s.hover[3]; + const bool inUse = s.drag != Drag::None || std::any_of(std::begin(s.hover), std::end(s.hover), [](bool h) { return h; }); const bool want = s.visible && (inUse || g_tick < s.nearUntil); // The controls stay shown while their screen is, just fully transparent when not // wanted: SteamVR's laser still hits them, and the hover event brings them in, for // any device's laser, whatever its shape. if (s.visible && !s.controlsUp) { - for (auto o : s.Controls()) vr::VROverlay()->ShowOverlay(o); + for (auto o : s.Controls()) + if (o != vr::k_ulOverlayHandleInvalid) vr::VROverlay()->ShowOverlay(o); s.controlsUp = true; ApplyAlpha(s); } @@ -748,6 +859,19 @@ void UpdateControls() { // ---------------------------------------------------------------- moving, resizing, pinning +// To ft-floatd (@frametop_float), for floating windows: dock, close, resize. From an unbound +// socket, so its replies go nowhere. +void SendFloat(const std::string &msg) { + static const int fd = socket(AF_UNIX, SOCK_DGRAM | SOCK_CLOEXEC | SOCK_NONBLOCK, 0); + sockaddr_un addr{}; + addr.sun_family = AF_UNIX; + const char name[] = "frametop_float"; + std::memcpy(addr.sun_path + 1, name, sizeof name - 1); + sendto(fd, msg.data(), msg.size(), MSG_DONTWAIT, reinterpret_cast<sockaddr *>(&addr), + socklen_t(offsetof(sockaddr_un, sun_path) + 1 + sizeof name - 1)); + std::printf("to ft-floatd: %s\n", msg.c_str()); +} + // Where a device's ray meets the screen's plane, in the screen's x (right) and y (up), // metres from its centre. bool RayOnPlane(const Mat &p, const Mat &d, double *x, double *y) { @@ -905,7 +1029,7 @@ void FinishDrag(Screen &s, int index) { EndDrag(s); Mat c, p; if (!moved) return; - ArrangeDesktopSoon(); + if (!s.floating) ArrangeDesktopSoon(); if (target != kNone && DevicePose(target, &c) && ScreenPose(s, &p)) { Pin(s, target, Mul(Inverse(c), p)); if (target == vr::k_unTrackedDeviceIndex_Hmd) std::printf("screen %d: pinned to the head\n", index + 1); @@ -992,6 +1116,19 @@ void UpdateDrag(Screen &s, int index) { double hx, hy; Mat l; if (!LaserPose(s.dragDevice, &l) || !RayOnScreen(s, l, &hx, &hy)) return; + if (s.floating) { + // A floating window: the corner goes where the laser is, in both directions, and the + // window gets that many pixels at the same density (ft-floatd resizes it, and the + // new crop comes back as "float", with the top left corner kept where it is). + if (s.mpp <= 0 || g_tick - s.resizeSent < 4) return; // about 20 a second + const double left = -s.metres / 2, top = s.heightMetres() / 2; + const int w = std::max(320, int(std::lround((hx - s.grabX - left) / s.mpp))); + const int h = std::max(200, int(std::lround((top - (hy - s.grabY)) / s.mpp))); + if (w == s.resizeW && h == s.resizeH) return; + s.resizeW = w, s.resizeH = h, s.resizeSent = g_tick; + SendFloat("resize " + std::to_string(index + 1) + " " + std::to_string(w) + " " + std::to_string(h)); + return; + } const double a = s.width > 0 ? double(s.height) / s.width : 9.0 / 16; const double cx = hx - s.grabX, cy = hy - s.grabY; // where the corner should be SetWidth(s, 2 * (cx - a * cy) / (1 + a * a)); @@ -1011,6 +1148,92 @@ void Push(Screen &s, double notches) { s.dragRel = Mul(Inverse(d), p); } +// ---------------------------------------------------------------- floating windows + +// Show the window's rectangle of the buffer, and map the mouse to it (see ToBuffer). +// Texture bounds are fractions of the buffer, v from the top. +void CropOverlay(vr::VROverlayHandle_t o, const Screen &s, int x, int y, int w, int h) { + if (s.width <= 0 || s.height <= 0 || w <= 0 || h <= 0) return; + vr::VRTextureBounds_t b = {float(x) / s.width, float(y) / s.height, float(x + w) / s.width, + float(y + h) / s.height}; + vr::VROverlay()->SetOverlayTextureBounds(o, &b); + vr::HmdVector2_t scale = {float(w), float(h)}; + vr::VROverlay()->SetOverlayMouseScale(o, &scale); +} + +void ApplyCrop(Screen &s) { + CropOverlay(s.overlay, s, s.cropX, s.cropY, s.cropW, s.cropH); + for (const auto &[k, sub] : s.subs) CropOverlay(sub.overlay, s, sub.x, sub.y, sub.w, sub.h); +} + +// ft-floatd's "float": the window's rectangle, its title bar, and the density. The panel's +// top left corner stays where it is when the window changes size. +void SetFloat(Screen &s, double mpp, int x, int y, int w, int h, int title) { + const bool first = !s.floatOn || s.cropW <= 0; + const double oldW = s.metres, oldH = s.heightMetres(); + s.floatOn = true; + s.mpp = mpp; + s.cropX = x, s.cropY = y, s.cropW = w, s.cropH = h, s.titleH = title; + s.metres = w * mpp; + vr::VROverlay()->SetOverlayWidthInMeters(s.overlay, float(s.metres)); + ApplyCurve(s); + ApplyCrop(s); + const double dx = (s.metres - oldW) / 2, dy = -(s.heightMetres() - oldH) / 2; + if (!first && (std::fabs(dx) > 1e-6 || std::fabs(dy) > 1e-6)) { + if (s.pinned != kNone) Pin(s, s.pinned, Mul(s.pinRel, Translation(dx, dy, 0))); + else SetAbsolute(s, Mul(s.pose, Translation(dx, dy, 0))); + } else { + PlaceChrome(s); + } +} + +void Unfloat(Screen &s) { + if (s.drag != Drag::None) EndDrag(s); + for (auto &[k, sub] : s.subs) vr::VROverlay()->DestroyOverlay(sub.overlay); + s.subs.clear(); + s.floatOn = s.minimized = s.titleCarry = false; + s.cropW = s.cropH = 0; + s.resizeW = s.resizeH = 0; +} + +// A popup or dialog (number k) at x, y, w, h in the buffer; w = 0 takes it away. +void SetSub(Screen &s, int index, int k, int x, int y, int w, int h) { + auto it = s.subs.find(k); + if (w <= 0 || h <= 0) { + if (it != s.subs.end()) { + vr::VROverlay()->DestroyOverlay(it->second.overlay); + s.subs.erase(it); + } + return; + } + if (it == s.subs.end()) { + Sub sub; + char key[80], name[64]; + std::snprintf(key, sizeof key, "frametop.float.%d.sub.%d", index + 1, k); + std::snprintf(name, sizeof name, "Floating window menu %d", k); + if (vr::VROverlay()->CreateOverlay(key, name, &sub.overlay) != vr::VROverlayError_None) return; + vr::VROverlay()->SetOverlayInputMethod(sub.overlay, vr::VROverlayInputMethod_Mouse); + vr::VROverlay()->SetOverlayFlag(sub.overlay, vr::VROverlayFlags_IgnoreTextureAlpha, true); + vr::VROverlay()->SetOverlayFlag(sub.overlay, vr::VROverlayFlags_SendVRDiscreteScrollEvents, true); + vr::VROverlay()->SetOverlayFlag(sub.overlay, vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible, s.lasers); + vr::VROverlay()->SetOverlaySortOrder(sub.overlay, 5); + it = s.subs.emplace(k, sub).first; + if (s.shown) { + auto imp = g_imports.find(s.shown); + if (imp != g_imports.end()) { + vr::SharedTextureHandle_t handle = imp->second; + vr::Texture_t tex = {&handle, vr::TextureType_SharedTextureHandle, vr::ColorSpace_Gamma}; + vr::VROverlay()->SetOverlayTexture(sub.overlay, &tex); + } + } + vr::VROverlay()->SetOverlayAlpha(sub.overlay, s.alpha); + if (s.visible) vr::VROverlay()->ShowOverlay(sub.overlay); + } + it->second.x = x, it->second.y = y, it->second.w = w, it->second.h = h; + CropOverlay(it->second.overlay, s, x, y, w, h); + PlaceSubs(s); +} + // ---------------------------------------------------------------- the catcher // A button pressed on a screen belongs to KWin until it comes up, wherever the laser is by @@ -1083,9 +1306,11 @@ void UpdateCatcher() { for (int k = 0; k < 3; ++k) params.vSource.v[k] = l.m[k][3], params.vDirection.v[k] = -l.m[k][2]; for (auto &[i, s] : g_screens) { if (!s.visible) continue; - for (auto o : s.All()) { + std::vector<vr::VROverlayHandle_t> parts(s.All().begin(), s.All().end()); + for (const auto &[k, sub] : s.subs) parts.push_back(sub.overlay); + for (auto o : parts) { vr::VROverlayIntersectionResults_t hit; - if (vr::VROverlay()->ComputeOverlayIntersection(o, ¶ms, &hit)) { + if (o != vr::k_ulOverlayHandleInvalid && vr::VROverlay()->ComputeOverlayIntersection(o, ¶ms, &hit)) { g_press.distance = std::max(0.05, double(hit.fDistance)); ShowCatcher(false); return; @@ -1194,18 +1419,21 @@ int ft_vr_modifiers(uint32_t format, uint64_t *out, int max) { bool ft_vr_screens_shown(void) { return g_vr && ModeVisible(); } -void ft_vr_screen_create(int index, double metres, int count) { - if (!g_vr) return; - Screen &s = g_screens[index]; - s.metres = metres; +} // extern "C" + +namespace { + +// A panel and its controls. `prefix` names the overlays (frametop.screen.N, +// frametop.float.N), `label` is what SteamVR shows ("Screen 2", "Floating window 1"). +bool MakePanel(Screen &s, const char *prefix, const char *label) { char key[64], name[64]; - std::snprintf(key, sizeof key, "frametop.screen.%d", index + 1); - std::snprintf(name, sizeof name, "Screen %d", index + 1); + std::snprintf(key, sizeof key, "%s", prefix); + std::snprintf(name, sizeof name, "%s", label); if (vr::VROverlay()->CreateOverlay(key, name, &s.overlay) != vr::VROverlayError_None) { std::fprintf(stderr, "openvr: can't create overlay %s\n", key); - return; + return false; } - vr::VROverlay()->SetOverlayWidthInMeters(s.overlay, float(metres)); + vr::VROverlay()->SetOverlayWidthInMeters(s.overlay, float(s.metres)); vr::VROverlay()->SetOverlayInputMethod(s.overlay, vr::VROverlayInputMethod_Mouse); vr::VROverlay()->SetOverlayFlag(s.overlay, vr::VROverlayFlags_IgnoreTextureAlpha, true); vr::VROverlay()->SetOverlayFlag(s.overlay, vr::VROverlayFlags_SendVRDiscreteScrollEvents, true); @@ -1213,21 +1441,39 @@ void ft_vr_screen_create(int index, double metres, int count) { static const auto corner = CornerTexture(64); static const auto curve = CurveTexture(64); static const auto roll = RollTexture(64); - std::snprintf(key, sizeof key, "frametop.screen.%d.bar", index + 1); - std::snprintf(name, sizeof name, "Screen %d: move", index + 1); - s.bar = MakeChrome(key, name, BarTexture(false), 256, 24); + auto chrome = [&](const char *part, const char *what, const std::vector<uint8_t> &px, int w, int h) { + std::snprintf(key, sizeof key, "%s.%s", prefix, part); + std::snprintf(name, sizeof name, "%s: %s", label, what); + return MakeChrome(key, name, px, w, h); + }; + s.bar = chrome("bar", "move", BarTexture(false), 256, 24); vr::VROverlay()->SetOverlayFlag(s.bar, vr::VROverlayFlags_SendVRDiscreteScrollEvents, true); - std::snprintf(key, sizeof key, "frametop.screen.%d.curve", index + 1); - std::snprintf(name, sizeof name, "Screen %d: curve", index + 1); - s.curveButton = MakeChrome(key, name, curve, 64, 64); - std::snprintf(key, sizeof key, "frametop.screen.%d.roll", index + 1); - std::snprintf(name, sizeof name, "Screen %d: roll", index + 1); - s.rollButton = MakeChrome(key, name, roll, 64, 64); + s.curveButton = chrome("curve", "curve", curve, 64, 64); + s.rollButton = chrome("roll", "roll", roll, 64, 64); vr::VROverlay()->SetOverlayFlag(s.rollButton, vr::VROverlayFlags_SendVRDiscreteScrollEvents, true); - std::snprintf(key, sizeof key, "frametop.screen.%d.resize", index + 1); - std::snprintf(name, sizeof name, "Screen %d: resize", index + 1); - s.handle = MakeChrome(key, name, corner, 64, 64); + s.handle = chrome("resize", "resize", corner, 64, 64); + if (s.floating) { + static const auto dock = DockTexture(64); + static const auto close = CloseTexture(64); + s.dockButton = chrome("dock", "back to the desktop", dock, 64, 64); + s.closeButton = chrome("close", "close", close, 64, 64); + } ApplyAlpha(s); + return true; +} + +} // namespace + +extern "C" { + +void ft_vr_screen_create(int index, double metres, int count) { + if (!g_vr) return; + Screen &s = g_screens[index]; + s.metres = metres; + char prefix[64], label[64]; + std::snprintf(prefix, sizeof prefix, "frametop.screen.%d", index + 1); + std::snprintf(label, sizeof label, "Screen %d", index + 1); + if (!MakePanel(s, prefix, label)) return; // Until the layout places it: 2 m ahead of the head, in a row, screen 1 on the left. RefreshPoses(); Mat head; @@ -1238,10 +1484,28 @@ void ft_vr_screen_create(int index, double metres, int count) { SetAbsolute(s, PanelPose(head.m[0][3] + dx * 2, head.m[1][3], head.m[2][3] + dz * 2, yaw, 0, 0)); } +// A spare output's panel (number `slot` from 1): hidden until a window floats on it. +void ft_vr_float_create(int index, int slot) { + if (!g_vr) return; + Screen &s = g_screens[index]; + s.floating = true; + char prefix[64], label[64]; + std::snprintf(prefix, sizeof prefix, "frametop.float.%d", slot); + std::snprintf(label, sizeof label, "Floating window %d", slot); + MakePanel(s, prefix, label); +} + +void ft_vr_float_output(int index, bool on) { + auto it = g_screens.find(index); + if (it != g_screens.end()) it->second.outputOn = on; +} + void ft_vr_screen_destroy(int index) { auto it = g_screens.find(index); if (it == g_screens.end()) return; - for (auto o : it->second.All()) vr::VROverlay()->DestroyOverlay(o); + for (auto o : it->second.All()) + if (o != vr::k_ulOverlayHandleInvalid) vr::VROverlay()->DestroyOverlay(o); + for (auto &[k, sub] : it->second.subs) vr::VROverlay()->DestroyOverlay(sub.overlay); g_screens.erase(it); } @@ -1274,14 +1538,19 @@ bool ft_vr_screen_present(int index, const void *key, const struct ft_dmabuf *b) } if (b->width != s.width || b->height != s.height) { s.width = b->width, s.height = b->height; - vr::HmdVector2_t scale = {float(s.width), float(s.height)}; - vr::VROverlay()->SetOverlayMouseScale(s.overlay, &scale); + if (s.floating) { + ApplyCrop(s); + } else { + vr::HmdVector2_t scale = {float(s.width), float(s.height)}; + vr::VROverlay()->SetOverlayMouseScale(s.overlay, &scale); + } PlaceChrome(s); // the height changed std::printf("screen %d: %dx%d\n", index + 1, s.width, s.height); } vr::SharedTextureHandle_t handle = it->second; vr::Texture_t tex = {&handle, vr::TextureType_SharedTextureHandle, vr::ColorSpace_Gamma}; vr::VROverlay()->SetOverlayTexture(s.overlay, &tex); + for (const auto &[k, sub] : s.subs) vr::VROverlay()->SetOverlayTexture(sub.overlay, &tex); s.shown = key; // UpdateVisibility shows it on the next tick return true; } @@ -1299,14 +1568,19 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { for (auto &[index, s] : g_screens) { vr::VREvent_t ev; // The screen itself: input for KWin. - while (vr::VROverlay()->PollNextOverlayEvent(s.overlay, &ev, sizeof ev)) { + // The screen itself (and a floating window's popups): input for KWin. + auto panelEvent = [&](const vr::VREvent_t &ev, bool sub, const Sub *from) { ft_event e{}; e.screen = index; + auto at = [&] { + if (from) e.x = from->x + ev.data.mouse.x, e.y = from->y + from->h - ev.data.mouse.y; + else s.ToBuffer(ev.data.mouse.x, ev.data.mouse.y, &e.x, &e.y); + }; switch (ev.eventType) { case vr::VREvent_MouseMove: + if (s.titleCarry) return; // KWin's pointer stays where the title bar was pressed e.type = FT_MOTION; - e.x = ev.data.mouse.x; - e.y = s.height - ev.data.mouse.y; // OpenVR's mouse origin is bottom left + at(); if (g_press.buttons) g_press.screen = index, g_press.x = e.x, g_press.y = e.y; break; case vr::VREvent_MouseButtonDown: @@ -1315,10 +1589,17 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { e.type = FT_BUTTON; e.button = LinuxButton(ev.data.mouse.button); e.pressed = ev.eventType == vr::VREvent_MouseButtonDown; - e.x = ev.data.mouse.x; - e.y = s.height - ev.data.mouse.y; + at(); + if (!e.pressed && s.titleCarry) e.x = s.carryX, e.y = s.carryY, s.titleCarry = false; if (e.pressed) { PressDown(ev.trackedDeviceIndex, e.button, index, e.x, e.y); + // A floating window's title bar: carry the panel, and KWin (which starts + // moving the window on the press) sees no motion until the release. + if (!sub && s.floating && e.button == BTN_LEFT && s.titleH > 0 && e.y >= s.cropY && + e.y < s.cropY + s.titleH && s.drag == Drag::None) { + s.titleCarry = true, s.carryX = e.x, s.carryY = e.y; + StartDrag(s, Drag::Move, ev.trackedDeviceIndex); + } } else { g_press.buttons &= ~ButtonBit(e.button); if (!g_press.buttons) g_press.upAt = -1; @@ -1330,14 +1611,18 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { e.dy = -ev.data.scroll.ydelta; break; case vr::VREvent_FocusLeave: - if (g_press.buttons) continue; // KWin keeps the pointer while a button is held + if (g_press.buttons) return; // KWin keeps the pointer while a button is held + if (sub) return; // off a popup is usually onto its window e.type = FT_LEAVE; break; default: - continue; + return; } handle(&e, data); - } + }; + while (vr::VROverlay()->PollNextOverlayEvent(s.overlay, &ev, sizeof ev)) panelEvent(ev, false, nullptr); + for (const auto &[k, sub] : s.subs) + while (vr::VROverlay()->PollNextOverlayEvent(sub.overlay, &ev, sizeof ev)) panelEvent(ev, true, &sub); // The controls light up under a laser. auto hover = [&](int k) { const bool on = ev.eventType == vr::VREvent_MouseMove || ev.eventType == vr::VREvent_FocusEnter; @@ -1376,6 +1661,20 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { ReleaseAwayBy(ev.trackedDeviceIndex, ev.data.mouse.button, handle, data); } } + // A floating window's buttons: back to the desktop, and close (ft-floatd does both). + for (int k : {4, 5}) { + const vr::VROverlayHandle_t o = s.Controls()[k]; + if (o == vr::k_ulOverlayHandleInvalid) continue; + while (vr::VROverlay()->PollNextOverlayEvent(o, &ev, sizeof ev)) { + hover(k); + if (ev.eventType == vr::VREvent_MouseButtonDown && ev.data.mouse.button == vr::VRMouseButton_Left) { + SendFloat((k == 4 ? "dock " : "close ") + std::to_string(index + 1)); + } else if (ev.eventType == vr::VREvent_MouseButtonUp) { + EndDragsBy(ev.trackedDeviceIndex); + ReleaseAwayBy(ev.trackedDeviceIndex, ev.data.mouse.button, handle, data); + } + } + } // The roll button: drag around like a knob, or scroll. while (vr::VROverlay()->PollNextOverlayEvent(s.rollButton, &ev, sizeof ev)) { hover(2); @@ -1446,8 +1745,18 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { // ~100 ms (it landed on something else), KWin gets it anyway // state -> "ok <mode> <manual 0|1> <wrist deg> <gesture hand> <gesture deg> // <controllers> <game running 0|1> <ingames>" +// Floating windows (from ft-floatd; <screen> is the spare output's number, after the screens): +// float <screen> <metres per pixel> <x> <y> <w> <h> <title> the window's rectangle in the +// buffer and its title bar's height (pixels); shows the panel +// unfloat <screen> hides it +// pose <screen> <12 numbers> its place in the room (rows of a 3x4, standing universe) +// sub <screen> <k> <x> <y> <w> <h> | sub <screen> <k> off popup or dialog k over it +// minimized <screen> 0|1 +// carry <screen> the window's own title bar was pressed (an app that draws its +// own): carry the panel with the pressing laser until the release // (size <screen> <w> <h> and key <code> <value> are handled in compositor.c.) Screens are -// numbered from 1 here, like everywhere the user sees them. +// numbered from 1 here, like everywhere the user sees them. "screens" and "all" leave out +// floating windows. void ft_vr_command(const char *cmd, char *reply, int size) { if (!g_vr) return (void)std::snprintf(reply, size, "error no SteamVR (--no-vr)"); RefreshPoses(); @@ -1457,7 +1766,8 @@ void ft_vr_command(const char *cmd, char *reply, int size) { float r[12]; auto each = [&](const char *which, auto fn) -> bool { // "all" or a screen number if (std::strcmp(which, "all") == 0) { - for (auto &[i, s] : g_screens) fn(s); + for (auto &[i, s] : g_screens) + if (!s.floating) fn(s); return true; } Screen *s = Find(std::atoi(which)); @@ -1528,9 +1838,10 @@ void ft_vr_command(const char *cmd, char *reply, int size) { for (int k = 0; k < 12 && len < size; ++k) len += std::snprintf(reply + len, size - len, " %.5f", s->pinRel.m[k / 4][k % 4]); } else if (std::strncmp(cmd, "screens", 7) == 0) { - int len = std::snprintf(reply, size, "ok %zu", g_screens.size()); + const size_t count = std::count_if(g_screens.begin(), g_screens.end(), [](auto &e) { return !e.second.floating; }); + int len = std::snprintf(reply, size, "ok %zu", count); for (auto &[i, s] : g_screens) - if (len < size) + if (len < size && !s.floating) len += std::snprintf(reply + len, size - len, " %d:%dx%d:%.3f", i + 1, s.width, s.height, s.metres); } else if (std::strncmp(cmd, "head", 4) == 0) { Mat m; @@ -1576,6 +1887,48 @@ void ft_vr_command(const char *cmd, char *reply, int size) { else return (void)std::snprintf(reply, size, "error modes: always outside_games dashboard"); UpdateLasers(); std::snprintf(reply, size, "ok %s", LasersName()); + } else if (int x0, y0, w0, h0, t0; std::sscanf(cmd, "float %d %lf %d %d %d %d %d", &n, &w, &x0, &y0, &w0, &h0, &t0) == 7) { + Screen *s = Find(n); + if (!s || !s->floating) return (void)std::snprintf(reply, size, "error no floating window panel %d", n); + if (!(w > 1e-5 && w < 0.01) || w0 < 1 || h0 < 1) return (void)std::snprintf(reply, size, "error bad float"); + SetFloat(*s, w, x0, y0, w0, h0, std::max(0, t0)); + std::snprintf(reply, size, "ok"); + } else if (std::sscanf(cmd, "unfloat %d", &n) == 1) { + Screen *s = Find(n); + if (!s || !s->floating) return (void)std::snprintf(reply, size, "error no floating window panel %d", n); + Unfloat(*s); + UpdateVisibility(); + std::snprintf(reply, size, "ok"); + } else if (std::sscanf(cmd, "pose %d %f %f %f %f %f %f %f %f %f %f %f %f", &n, &r[0], &r[1], &r[2], &r[3], &r[4], + &r[5], &r[6], &r[7], &r[8], &r[9], &r[10], &r[11]) == 13) { + Screen *s = Find(n); + if (!s) return (void)std::snprintf(reply, size, "error no screen %d", n); + Mat m{}; + for (int k = 0; k < 12; ++k) m.m[k / 4][k % 4] = r[k]; + EndDrag(*s); + SetAbsolute(*s, m); + std::snprintf(reply, size, "ok"); + } else if (int k0; std::sscanf(cmd, "sub %d %d %d %d %d %d", &n, &k0, &x0, &y0, &w0, &h0) == 6 || + (std::sscanf(cmd, "sub %d %d %15s", &n, &k0, word) == 3 && !std::strcmp(word, "off"))) { + Screen *s = Find(n); + if (!s || !s->floating) return (void)std::snprintf(reply, size, "error no floating window panel %d", n); + if (std::strstr(cmd, " off")) w0 = h0 = 0; + SetSub(*s, n - 1, k0, x0, y0, w0, h0); + std::snprintf(reply, size, "ok"); + } else if (int on; std::sscanf(cmd, "minimized %d %d", &n, &on) == 2) { + Screen *s = Find(n); + if (!s || !s->floating) return (void)std::snprintf(reply, size, "error no floating window panel %d", n); + s->minimized = on != 0; + UpdateVisibility(); + std::snprintf(reply, size, "ok"); + } else if (std::sscanf(cmd, "carry %d", &n) == 1) { + Screen *s = Find(n); + if (!s || !s->floating) return (void)std::snprintf(reply, size, "error no floating window panel %d", n); + if (!(g_press.buttons & ButtonBit(BTN_LEFT)) || g_press.screen != n - 1 || s->drag != Drag::None) + return (void)std::snprintf(reply, size, "error not pressed there"); + s->titleCarry = true, s->carryX = g_press.x, s->carryY = g_press.y; + StartDrag(*s, Drag::Move, g_press.device); + std::snprintf(reply, size, "ok"); } else if (std::strcmp(cmd, "up") == 0) { if ((g_press.buttons & ButtonBit(BTN_LEFT)) && g_press.device != kNone && !IsHandController(g_press.device)) g_press.upAt = g_tick + 9; diff --git a/screens/vr.h b/screens/vr.h index 876fc1e..7b32186 100644 --- a/screens/vr.h +++ b/screens/vr.h @@ -37,6 +37,10 @@ bool ft_vr_screens_shown(void); // A panel for screen `index`, width in metres, placed in a row in front of the head. void ft_vr_screen_create(int index, double metres, int count); void ft_vr_screen_destroy(int index); +// A spare output's panel, for floating windows (slot numbers from 1): hidden until +// ft-floatd floats a window on it and KWin has the output turned on. +void ft_vr_float_create(int index, int slot); +void ft_vr_float_output(int index, bool on); // Show a client buffer (identified by `key`) on the screen's panel. False if SteamVR // can't import it. bool ft_vr_screen_present(int index, const void *key, const struct ft_dmabuf *buf); diff --git a/session/frametop-session.sh b/session/frametop-session.sh index 068cbee..813771c 100755 --- a/session/frametop-session.sh +++ b/session/frametop-session.sh @@ -30,7 +30,7 @@ for var in $(compgen -e); do done conf=$HOME/.config/frametop.conf -BACKEND=screens SCREENS=2 WIDTH=1920 HEIGHT=1080 PHYS_WIDTH=1.6 REMOTE=0 +BACKEND=screens SCREENS=2 WIDTH=1920 HEIGHT=1080 PHYS_WIDTH=1.6 REMOTE=0 FLOAT_SLOTS=8 FLOAT_MARGIN=300 # shellcheck disable=SC1090 [ -f "$conf" ] && . "$conf" backend=${FT_BACKEND:-$BACKEND} @@ -39,6 +39,13 @@ width=${FT_WIDTH:-$WIDTH} height=${FT_HEIGHT:-$HEIGHT} phys_width=${FT_PHYS_WIDTH:-$PHYS_WIDTH} remote=${FT_REMOTE:-$REMOTE} +# Floating windows (screens backend): KWin gets this many spare outputs after the screens, +# and ft-floatd floats a window on each (docs/floating-windows.md). Changing it takes a +# desktop restart. +float_slots=${FT_FLOAT_SLOTS:-$FLOAT_SLOTS} +[[ $float_slots =~ ^[0-9]+$ ]] || float_slots=8 +[ "$float_slots" -le 16 ] || float_slots=16 +[ "$backend" = screens ] || float_slots=0 if [ "$backend" = gamescope ] && [ $((width * height)) -gt $((1920 * 1080)) ]; then # gamescope's VR backend aborts above 1920x1080 worth of pixels (its upload buffer; # see docs/design.md). Shrink a bigger size to fit, keeping its shape. @@ -80,10 +87,10 @@ if [ "${1:-}" != --inner ]; then # Plasma stay on the host and connect to its socket. socket=ft-screens-0 read -ra screen_args <<< "$("$here/../layout/ft-layout" screen-args)" - export FT_SCREEN_COUNT=$(( ${#screen_args[@]} / 2 )) + export FT_SCREEN_COUNT=$(( ${#screen_args[@]} / 2 )) FT_FLOAT_SLOTS=$float_slots "$here/../scripts/container-up.sh" # not owned by this desktop, or stopping it would stop the container "$HOME/.local/bin/distrobox" enter dev -- "$here/../screens/build/ft-screens" --socket "$socket" \ - "${screen_args[@]}" > /tmp/frametop-screens.log 2>&1 < /dev/null & + "${screen_args[@]}" --spares "$float_slots" > /tmp/frametop-screens.log 2>&1 < /dev/null & stop_screens() { pkill -x ft-screens 2>/dev/null || true; } trap stop_screens EXIT for _ in $(seq 100); do [ -S "$XDG_RUNTIME_DIR/$socket" ] && break; sleep 0.2; done @@ -135,7 +142,7 @@ fi # With ft-screens the size is only the starting one: ft-screens sets each screen's own. cat > "$runtime/bin/kwin_wayland_wrapper" <<EOF #!/bin/sh -exec /usr/bin/kwin_wayland_wrapper --width $width --height $height --output-count $screens --no-lockscreen "\$@" +exec /usr/bin/kwin_wayland_wrapper --width $width --height $height --output-count $((screens + float_slots)) --no-lockscreen "\$@" EOF chmod +x "$runtime/bin/kwin_wayland_wrapper" export PATH=$runtime/bin:$PATH @@ -163,4 +170,21 @@ if [ "$remote" = 1 ]; then "$here/vnc-bridge.sh" "$width" "$height" > /tmp/frametop-vnc.log 2>&1 & fi +# ft-floatd (floating windows) runs inside the Plasma session, on its D-Bus: started from +# the session's autostart, which only this desktop reads (XDG_CONFIG_HOME above). +autostart=$XDG_CONFIG_HOME/autostart/frametop-floatd.desktop +if [ "$float_slots" -gt 0 ]; then + mkdir -p "$(dirname "$autostart")" + cat > "$autostart" <<EOF +[Desktop Entry] +Type=Application +Name=Frametop floating windows +Exec=sh -c 'exec "$here/../float/ft-floatd" --screens $screens --slots $float_slots > /tmp/frametop-floatd.log 2>&1' +X-KDE-autostart-phase=2 +NoDisplay=true +EOF +else + rm -f "$autostart" +fi + dbus-run-session startplasma-wayland diff --git a/session/frametop.conf.example b/session/frametop.conf.example index 3ea46cb..194d4eb 100644 --- a/session/frametop.conf.example +++ b/session/frametop.conf.example @@ -7,6 +7,8 @@ WIDTH=1920 # gamescope: pixels per screen (at most 1920x1080 worth) HEIGHT=1080 PHYS_WIDTH=1.6 # gamescope: panel width in metres, docked REMOTE=0 # 1 = serve the desktop over VNC on the tailnet (port 5900; see README) +FLOAT_SLOTS=8 # screens backend: how many windows can float in VR at once (spare outputs; 0 turns it off; restart the desktop after a change) +FLOAT_MARGIN=300 # floating windows: pixels around each window on its output, so menus have room past its edges POINTER=0 # 1 = the mouse drives the universal 3D pointer (ft_pointer driver) instead of a plain mouse POINTER_SENSITIVITY=0.03 # degrees per mouse count POINTER_IDLE=30 # seconds without mouse use before the controllers get their laser back From 6122b7eb15f04b50af12440de145280e350bf8ab Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:31:23 -0600 Subject: [PATCH 09/13] Floating windows: full screen, per-window scale, and drags across panels Full screen fills the window's own panel (the margin drops to zero). Meta+scroll over a floating window changes its scale at the same size in pixels. Spare outputs are sized to a multiple of KWin's buffer scale, and screens to even sizes: an odd buffer at a fractional scale is a protocol error that disconnected KWin. The 3D mouse's drag lock now crosses onto other Frametop panels unless the pressed one is being carried. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- float/ft_floatd.py | 96 ++++++++++++++++++++++++++++------- pointer/helper/ft-pointer.cpp | 58 +++++++++++++++++++++ screens/compositor.c | 16 ++++++ screens/test/headless.sh | 2 +- 4 files changed, 153 insertions(+), 19 deletions(-) diff --git a/float/ft_floatd.py b/float/ft_floatd.py index 3137822..5f8327a 100644 --- a/float/ft_floatd.py +++ b/float/ft_floatd.py @@ -12,7 +12,7 @@ which window floats on which spare output and panel, and connects three parts: Commands and ft-screens' events arrive as datagrams on @frametop_float (ft-float is the command-line side). Replies go to the sender: float [ID|active] dock [ID|active] close ID list quit (ft-float) - dock N | close N | resize N W H | carried N (ft-screens, N = its screen) + dock N | close N | resize N W H | scale N STEPS (ft-screens, N = its screen) Spare outputs are WL-<screens> .. WL-<screens + slots - 1>. A floating window's output is its frame plus a margin on each side (FLOAT_MARGIN pixels), so menus have room; the panel shows @@ -49,6 +49,15 @@ POLL_SECONDS = 20 # NextCommand answers empty after this (KWin's D-Bus ti SPARE_X, SPARE_CELL = 12000, 5000 # spares in KWin's layout: a grid from here, 4 across PULL_OUT = 0.05 # a floated window starts this far in front of its screen (metres) DEFAULT_MPP = 1.6 / 1920 # metres per pixel when ft-screens can't say (no SteamVR) +DEBUG = os.environ.get("FT_FLOAT_DEBUG") == "1" # log every event from the script + + +def fit(px, scale): + """An output size KWin can draw at this scale: its nested backend gives the buffer a whole + buffer scale (1.2 -> 2), and a buffer that isn't a multiple of it is a protocol error that + disconnects KWin. So round up to a multiple; the margin takes the extra pixels.""" + k = max(1, math.ceil(scale - 1e-6)) + return (px + k - 1) // k * k def log(*args): @@ -104,6 +113,8 @@ class Float: self.frame = None # last frame (logical, global) self.client = None self.full = False # full screen: no margin + self.normal = None # its size in pixels when last not full screen + self.unfull_until = 0.0 # left full screen just now (see follow) self.move_from = None # frame when a title-bar move started (put back after) self.subs = {} # popup or dialog id -> number on the panel @@ -174,6 +185,9 @@ class Daemon: def on_event(self, ev): kind = ev.get("ev") + if DEBUG: + log("event", {k: v for k, v in ev.items() if k in ("ev", "id", "output", "frame", "fullScreen", "popup", + "parent", "move")}) wid = ev.get("id", "") if kind == "hello": self.command(cmd="config", screens=self.screens_n) @@ -211,8 +225,9 @@ class Daemon: if (m["x"], m["y"]) != (ev["frame"]["x"], ev["frame"]["y"]): self.command(cmd="geometry", id=f.id, x=m["x"], y=m["y"], w=ev["frame"]["w"], h=ev["frame"]["h"]) elif kind == "fullscreen" and f: - f.full = bool(ev.get("fullScreen")) - self.follow(f, ev, refit=True) + if not ev.get("fullScreen"): + f.unfull_until = time.monotonic() + 1.0 + self.follow(f, ev) elif kind == "minimized" and f: self.screens.ask(f"minimized {f.slot.index} {1 if ev.get('minimized') else 0}", quiet=True) @@ -248,7 +263,7 @@ class Daemon: self.floats[wid] = f f.scale = output_scales().get(slot.output, 1.0) log(f"{wid[:9]} already floats on {slot.output}") - self.follow(f, ev, refit=True) + self.follow(f, ev) return # A new window that opened on a floating window's output: windows of floating apps # float too; anything else goes to the screens. @@ -261,6 +276,18 @@ class Daemon: # ------------------------------------------------------------ floating and docking + def sized(self, slot, size, timeout=1.0): + """Wait (briefly) until KWin has taken the spare's new size, before a scale that needs it.""" + end = time.monotonic() + timeout + while time.monotonic() < end: + for line in self.screens.ask("toplevels", quiet=True).splitlines()[1:]: + f = line.split() + if len(f) >= 2 and f[0] == str(slot.index) and f[1] == f"{size[0]}x{size[1]}": + return True + time.sleep(0.03) + log(f"{slot.output} didn't take {size[0]}x{size[1]} in time") + return False + def disable_unused(self): off = [f"output.{s.output}.disable" for s in self.slots if s.window is None] if off: @@ -300,11 +327,12 @@ class Daemon: f.mpp = self.screen_mpp(ev["output"]) fr, s, m = ev["frame"], f.scale, self.margin w, h = round(fr["w"] * s), round(fr["h"] * s) - slot.size = (w + 2 * m, h + 2 * m) + slot.size = (fit(w + 2 * m, s), fit(h + 2 * m, s)) log(f"{wid[:9]} ({ev.get('cls')}) floats on {slot.output}: {w}x{h} px, scale {s:g}") # The spare's size first (while it's off, so its first frame is right), then its panel, # then turn it on, then the window. self.screens.ask(f"size {slot.index} {slot.size[0]} {slot.size[1]}") + self.sized(slot, slot.size) self.screens.ask(f"scale {slot.index} {s:g}") # for pointer positions (KWin's units) self.set_panel(f, (m, m, w, h), title=round((ev["client"]["y"] - fr["y"]) * s)) self.place_panel(f, ev) @@ -333,27 +361,57 @@ class Daemon: rows = [f"{xa[k]:.5f} {ya[k]:.5f} {za[k]:.5f} {p[k]:.4f}" for k in range(3)] self.screens.ask(f"pose {f.slot.index} {' '.join(rows)}") - def follow(self, f, ev, refit=False): + def follow(self, f, ev): """The window moved or resized on its output: crop the panel to it, and keep the - output its size plus the margin.""" + output its size plus the margin. Full screen: no margin, and the output keeps the + window's size from before, so the window fills its own panel.""" slot, s = f.slot, f.scale fr, cl, out = ev["frame"], ev["client"], ev["outputRect"] f.frame, f.client = fr, cl - m = 0 if f.full else self.margin + # KWin sizes a window to its output before it reports it full screen; with a margin, a + # window that fills its output is going full screen. (Not just after it left full + # screen: then it fills the output until the output grows back.) + fills = self.margin > 0 and (fr["x"], fr["y"], fr["w"], fr["h"]) == (out["x"], out["y"], out["w"], out["h"]) + full = bool(ev.get("fullScreen")) or (fills and time.monotonic() > f.unfull_until) + f.full = full w, h = round(fr["w"] * s), round(fr["h"] * s) - want = (w + 2 * m, h + 2 * m) - if f.full: - want = slot.size or want - if refit or want != slot.size: - if not f.full and want != slot.size: - slot.size = want - self.screens.ask(f"size {slot.index} {want[0]} {want[1]}") + if not full: + f.normal = (w, h) + m = 0 if full else self.margin + want = f.normal if full and f.normal else (w + 2 * m, h + 2 * m) + want = (fit(want[0], s), fit(want[1], s)) + if want != slot.size: + slot.size = want + self.screens.ask(f"size {slot.index} {want[0]} {want[1]}") + if not full: x0, y0 = slot.pos[0] + m / s, slot.pos[1] + m / s - if not f.full and (abs(fr["x"] - x0) > 0.5 or abs(fr["y"] - y0) > 0.5): + if abs(fr["x"] - x0) > 0.5 or abs(fr["y"] - y0) > 0.5: self.command(cmd="geometry", id=f.id, x=x0, y=y0, w=fr["w"], h=fr["h"]) return # the next geometry event crops the panel x, y = round((fr["x"] - out["x"]) * s), round((fr["y"] - out["y"]) * s) - self.set_panel(f, (x, y, w, h), title=0 if f.full else round((cl["y"] - fr["y"]) * s)) + self.set_panel(f, (x, y, w, h), title=0 if full else round((cl["y"] - fr["y"]) * s)) + + def rescale(self, f, steps): + """Meta+scroll: the window's content bigger or smaller, at the same size in pixels, so its + panel stays the same size (KWin's output scale, in steps of 10%).""" + if not f.frame or f.full or steps == 0: + return + s = min(3.0, max(0.5, round(f.scale * 1.1 ** steps * 20) / 20)) + if s == f.scale: + return + w, h = round(f.frame["w"] * f.scale), round(f.frame["h"] * f.scale) + slot, m = f.slot, self.margin + log(f"{f.id[:9]} scale {f.scale:g} -> {s:g}") + f.scale = s + # The output's size must suit the new scale before KWin draws at it (see fit). + size = (fit(slot.size[0], s), fit(slot.size[1], s)) + if size != slot.size: + slot.size = size + self.screens.ask(f"size {slot.index} {size[0]} {size[1]}") + self.sized(slot, size) + kscreen(f"output.{slot.output}.scale.{s:g}") + self.screens.ask(f"scale {slot.index} {s:g}") + self.command(cmd="geometry", id=f.id, x=slot.pos[0] + m / s, y=slot.pos[1] + m / s, w=w / s, h=h / s) def dock(self, f, frame=None): """Back where it came from (or onto screen 1 if we don't know).""" @@ -424,7 +482,7 @@ class Daemon: if cmd in ("float", "dock") and (not rest or rest[0] == "active"): self.command(cmd="request-active") return "ok" - if cmd in ("dock", "close", "resize", "carried") and rest and rest[0].isdigit(): + if cmd in ("dock", "close", "resize", "scale") and rest and rest[0].isdigit(): f = self.by_panel(int(rest[0])) if not f: return f"error no floating window on screen {rest[0]}" @@ -432,6 +490,8 @@ class Daemon: self.dock(f) elif cmd == "close": self.command(cmd="close", id=f.id) + elif cmd == "scale" and len(rest) == 2: + self.rescale(f, int(rest[1])) elif cmd == "resize" and len(rest) == 3 and f.frame: w, h = max(320, int(rest[1])), max(200, int(rest[2])) self.command(cmd="geometry", id=f.id, x=f.frame["x"], y=f.frame["y"], w=w / f.scale, h=h / f.scale) diff --git a/pointer/helper/ft-pointer.cpp b/pointer/helper/ft-pointer.cpp index a41b5e7..ea5864f 100644 --- a/pointer/helper/ft-pointer.cpp +++ b/pointer/helper/ft-pointer.cpp @@ -105,6 +105,11 @@ // SteamVR's resize snap back. // A left release also goes to ft-screens ("up"), which releases a button held on its // screens in KWin if SteamVR gave the release to some other overlay. +// Across Frametop's panels (a drag and drop, or a window moved from one screen or floating +// window to another), the lock gives way: while the left button is held on one of ft-screens' +// panels, the ray is still tested against ft-screens' panels, and the cursor goes onto +// whichever it meets first. Not while that panel is being carried (its title bar or its bar): +// then the ray would find what's behind it. // // Head follow (experimental, off by default; POINTER_FOLLOW=1, or the relay's "follow toggle"): the cursor // is carried by a reference direction, where the head faced when it last settled, and turns @@ -285,6 +290,19 @@ std::string ExeDir() { return p.substr(0, p.rfind('/')); } +// One of ft-screens' panels showing a desktop: a screen (frametop.screen.N), a floating +// window (frametop.float.N), or a floating window's popup (frametop.float.N.sub.K), not a +// control of theirs. +bool FramePanel(const std::string &key) { + for (const char *prefix : {"frametop.screen.", "frametop.float."}) { + if (key.rfind(prefix, 0) != 0) continue; + const std::string rest = key.substr(std::strlen(prefix)); + const size_t dot = rest.find('.'); + return dot == std::string::npos || rest.compare(dot, 5, ".sub.") == 0; + } + return false; +} + void SendTo(int fd, const char *name, const std::string &msg) { sockaddr_un addr{}; addr.sun_family = AF_UNIX; @@ -621,6 +639,9 @@ int main() { Clock::time_point dropHoldUntil{}; // after a left release: keep the drag pose this long bool debug = false; std::string lastHit; + // The ft-screens panel the left button was pressed on, and where it was then (see the top). + std::string pressKey; + vr::HmdMatrix34_t pressPose{}; auto lastDebug = Clock::now(); vr::VROverlayHandle_t systemPointer = vr::k_ulOverlayHandleInvalid; overlay->FindOverlay("system.pointer", &systemPointer); @@ -683,6 +704,14 @@ int main() { nudging = false; leftHeld = true; dragDistance = lastDistance; + pressKey.clear(); + if (FramePanel(lastHit)) { + vr::ETrackingUniverseOrigin uo; + auto it = handles.find(lastHit); + if (it != handles.end() && + overlay->GetOverlayTransformAbsolute(it->second, &uo, &pressPose) == vr::VROverlayError_None) + pressKey = lastHit; + } tiltYaw = tiltPitch = 0; // a new drag starts untilted dropHoldUntil = {}; pulseAt = Clock::now(); @@ -1366,6 +1395,35 @@ int main() { } } } + // Held on one of ft-screens' panels that stays where it is: onto whichever of them + // the ray meets (see the top). + if (leftHeld && !pressKey.empty()) { + vr::ETrackingUniverseOrigin uo; + vr::HmdMatrix34_t now{}; + auto it = handles.find(pressKey); + bool still = it != handles.end() && + overlay->GetOverlayTransformAbsolute(it->second, &uo, &now) == vr::VROverlayError_None; + for (int i = 0; still && i < 3; ++i) + for (int j = 0; j < 4; ++j) + if (std::fabs(now.m[i][j] - pressPose.m[i][j]) > 0.001f) still = false; + if (still) { + Hit h; + for (const auto &[key, handle] : handles) { + if (!visible[key] || !FramePanel(key)) continue; + vr::VROverlayIntersectionParams_t params{}; + params.vSource = {float(anchor.x), float(anchor.y), float(anchor.z)}; + params.vDirection = {float(dir.x), float(dir.y), float(dir.z)}; + params.eOrigin = vr::TrackingUniverseStanding; + vr::VROverlayIntersectionResults_t r{}; + if (overlay->ComputeOverlayIntersection(handle, ¶ms, &r) && r.fDistance > 0.05f && + r.fDistance < h.along) + h.along = r.fDistance, h.key = key; + } + if (h.along < 1e8) dragDistance = h.along, lastHit = h.key; + } else { + pressKey.clear(); // carried: the lock holds for the rest of this press + } + } // While dragging: keep the press-time distance and show the non-interactive marker. // On a scene-graph plane or a panel's edge: the laser-catching dot goes 5 cm behind it. double distance = dragging ? dragDistance : (best < 1e8 ? best : freeDistance); diff --git a/screens/compositor.c b/screens/compositor.c index 10affcc..fe1bf6c 100644 --- a/screens/compositor.c +++ b/screens/compositor.c @@ -305,6 +305,18 @@ static void handle_vr_event(const struct ft_event *e, void *data) { break; case FT_SCROLL: if (s->pointer_focus != sc) break; + if (sc->index >= s->n_config && e->dy != 0 && + (wlr_keyboard_get_modifiers(&s->keyboard) & WLR_MODIFIER_LOGO)) { + // Meta+scroll on a floating window: its scale, bigger or smaller (ft-floatd). + char msg[48]; + snprintf(msg, sizeof msg, "scale %d %d", sc->index + 1, e->dy < 0 ? 1 : -1); + struct sockaddr_un addr = {.sun_family = AF_UNIX}; + const char name[] = "frametop_float"; + memcpy(addr.sun_path + 1, name, sizeof name - 1); + sendto(s->relay_fd, msg, strlen(msg), MSG_DONTWAIT, (struct sockaddr *)&addr, + offsetof(struct sockaddr_un, sun_path) + 1 + sizeof name - 1); + break; + } if (e->dy != 0) wlr_seat_pointer_notify_axis(s->seat, t, WL_POINTER_AXIS_VERTICAL_SCROLL, e->dy * 15, (int32_t)(e->dy * 120), WL_POINTER_AXIS_SOURCE_WHEEL, @@ -427,6 +439,10 @@ static int control_readable(int fd, uint32_t mask, void *data) { h > 16384) { snprintf(reply, sizeof reply, "error bad screen or size"); } else { + // A screen's size is even: KWin's nested backend gives a scaled screen a whole + // buffer scale (1.5 -> 2), and a buffer that isn't a multiple of it is a protocol + // error that disconnects KWin. (ft-floatd rounds spares' sizes for their scale.) + if (index - 1 < s->n_config) w += w & 1, h += h & 1; if (index - 1 < s->n_config) s->config[index - 1].width = w, s->config[index - 1].height = h; wlr_xdg_toplevel_set_size(s->screens[index - 1]->toplevel, w, h); snprintf(reply, sizeof reply, "ok"); diff --git a/screens/test/headless.sh b/screens/test/headless.sh index 7870fb5..5f02a25 100755 --- a/screens/test/headless.sh +++ b/screens/test/headless.sh @@ -115,7 +115,7 @@ case "${1:-}" in pkill -f -- "--socket frametop_float_test" 2>/dev/null || true read -r screens spares < "$rt/counts" nohup env XDG_RUNTIME_DIR="$rt" XDG_CONFIG_HOME="$cfg" DBUS_SESSION_BUS_ADDRESS="$bus" \ - WAYLAND_DISPLAY=wayland-0 "$REPO_ROOT/float/ft-floatd" --screens "$screens" --slots "$spares" \ + WAYLAND_DISPLAY=wayland-0 FT_FLOAT_DEBUG=1 "$REPO_ROOT/float/ft-floatd" --screens "$screens" --slots "$spares" \ --control ft_screens_test --socket frametop_float_test \ > "$logs/floatd.log" 2>&1 & sleep 2 From 691b66cd87715ae4d49ef2478ad0612acea39a07 Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:31:54 -0600 Subject: [PATCH 10/13] Stop ft-screens without an abort wlroots asserts that nothing still listens to its xdg-shell and decoration globals when the display goes, so every desktop stop ended with ft-screens aborting and leaving a core dump. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- screens/compositor.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/screens/compositor.c b/screens/compositor.c index fe1bf6c..b177d30 100644 --- a/screens/compositor.c +++ b/screens/compositor.c @@ -657,6 +657,9 @@ int main(int argc, char **argv) { wlr_log(WLR_INFO, "stopping"); if (s.child > 0) kill(s.child, SIGTERM); wl_display_destroy_clients(s.display); + // wlroots asserts that nothing still listens to its globals when they go. + wl_list_remove(&s.new_toplevel.link); + wl_list_remove(&s.new_decoration.link); ft_vr_shutdown(); wl_display_destroy(s.display); return 0; From fc15a8a8a6ba21e36011ce6edcf515b228cac600 Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Tue, 29 Sep 2026 23:32:03 -0600 Subject: [PATCH 11/13] Record what's built of floating windows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- docs/floating-windows.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/floating-windows.md b/docs/floating-windows.md index 77ac815..8bd1d3f 100644 --- a/docs/floating-windows.md +++ b/docs/floating-windows.md @@ -2,6 +2,16 @@ Status: design settled 2026-09-29 (see "Decisions"); being built on the `floating-windows` branch. +Built so far (2026-09-29; the KWin side tested on the headless test desktop, `screens/test/headless.sh`; nothing yet tried in the headset): + +- The catcher (a release off every panel still reaches KWin), and the pointer helper's "up" backstop. +- `float/frametop-float.js` (the KWin script), `float/ft-floatd`, and `float/ft-float`. "Float in VR" is in the window menu under Extensions, and Meta+Shift+F toggles the active window. Floating, docking (back where it came from), closing, full screen, per-window scale (Meta+scroll), popups reported with their rectangles, windows of a floating app floating too, and the notification when every spare is in use. +- ft-screens: a panel per spare output (`frametop.float.N`) with the crop, density, popups and dialogs as small panels over it, title-bar carrying, the corner tab resizing the window, and dock and close buttons. `--spares`, and the commands `float`, `unfloat`, `pose`, `sub`, `minimized`, `carry`. +- The session adds `FLOAT_SLOTS` spares and starts ft-floatd from the desktop's autostart; ft-layout leaves the spares alone. +- The 3D mouse's drag lock crosses onto other Frametop panels (not while carrying one). + +Not built yet: phase 2 (the ghost, tear-off by dragging, push-flush docking), phase 3 (launching floating, the Frametop Apps entry and picker, remembered placement), and phase 4. Frametop Apps (decision 10) needs a per-screen hide, which ft-screens doesn't have yet: its hide and show are for all screens. + The goal is to let any desktop app float in VR in a panel of its own, like SteamVR's floating windows, while it stays part of the Frametop desktop. That means drag and drop, the clipboard, and focus keep working between floating windows and the screens. - There are two ways to get a floating window. Launch the app floating, or drag a desktop window by its title bar off a screen and let go in the air. From 2a9fbdebb4bb2b407d1a212a6724b810723ae069 Mon Sep 17 00:00:00 2001 From: Nikita Koptelov <utrack@users.noreply.github.com> Date: Wed, 30 Sep 2026 18:02:24 +0200 Subject: [PATCH 12/13] Start Frametop's SteamVR clients only once SteamVR is up (#6) The pointer and gaze services need steamvr.service to be running (Requisite=), and ft-pointer, ft-screens, and ft-gaze connect as a background app before switching to overlay, so they never start a vrserver of their own. One started from the dev container never finds the headset, which left a reboot stuck in a loop. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 51b4e7931899800ef8e6c5e4f5feee2c73061bed) --- docs/design.md | 2 ++ gaze/frametop-gaze.service | 1 + gaze/ft-gaze.cpp | 6 +++++- pointer/helper/frametop-pointer.service | 1 + pointer/helper/ft-pointer.cpp | 6 +++++- screens/vr.cpp | 6 +++++- 6 files changed, 19 insertions(+), 3 deletions(-) diff --git a/docs/design.md b/docs/design.md index f99046a..fb2ef2f 100644 --- a/docs/design.md +++ b/docs/design.md @@ -70,6 +70,8 @@ SteamVR's dashboard and every overlay it hosts are driven by the vrcompositor `l Driver poses are in SteamVR's raw tracking space, and client programs work in the standing universe, which on the Frame is about 1.6 m above raw. Mixing them up put the laser's origin 1.6 m above your head. The helper converts using the headset's pose in both spaces every frame. +Frametop's SteamVR clients (ft-pointer, ft-screens, ft-gaze) connect as a background app first and switch to an overlay app only once that works. `VR_Init` as an overlay app starts vrserver itself when none is running, and one started that way from the dev container never finds the headset. At a boot where the gamescope session timed out, systemd dropped `steamvr.service`'s start job, the pointer service (ordered only `After=` it) started anyway, and its vrserver made every SteamVR launch fail with `HmdNotFound`. SteamOS's health check then kept resetting the Steam client and tried to fall back to the previous OS slot. The units also say `Requisite=steamvr.service`, so they don't start at all when SteamVR's start fails. + The driver starts disconnected, because holding the right-hand role while SteamVR starts leaves the Steam UI stuck on its loading icon. It connects when the mouse is used and claims the right hand. SteamVR keeps a hand role reserved for a disconnected device that still asks for it, so the driver switches its role hint between right hand (connected) and opt-out (not connected). ### The cursor diff --git a/gaze/frametop-gaze.service b/gaze/frametop-gaze.service index a19dc24..9b0c096 100644 --- a/gaze/frametop-gaze.service +++ b/gaze/frametop-gaze.service @@ -5,6 +5,7 @@ Documentation=file://@REPO@/gaze/README.md # Needs SteamVR's IPC (ft-gaze is an overlay client); it starts and stops with SteamVR. After=steamvr.service frametop-pointer.service PartOf=steamvr.service +Requisite=steamvr.service [Service] # Host Python; it runs ft-gaze in the dev container (distrobox enter), which quits when diff --git a/gaze/ft-gaze.cpp b/gaze/ft-gaze.cpp index efcf41e..756f968 100644 --- a/gaze/ft-gaze.cpp +++ b/gaze/ft-gaze.cpp @@ -441,7 +441,11 @@ int main(int argc, char **argv) { stdinClosed = true; }).detach(); vr::EVRInitError err = vr::VRInitError_None; - vr::VR_Init(&err, vr::VRApplication_Overlay); + vr::VR_Init(&err, vr::VRApplication_Background); + if (err == vr::VRInitError_None) { + vr::VR_Shutdown(); + vr::VR_Init(&err, vr::VRApplication_Overlay); + } if (err != vr::VRInitError_None) { std::fprintf(stderr, "ft-gaze: SteamVR: %s\n", vr::VR_GetVRInitErrorAsEnglishDescription(err)); return 1; diff --git a/pointer/helper/frametop-pointer.service b/pointer/helper/frametop-pointer.service index 3f180ea..5ffbbed 100644 --- a/pointer/helper/frametop-pointer.service +++ b/pointer/helper/frametop-pointer.service @@ -5,6 +5,7 @@ Documentation=file://@REPO@/README.md # Needs SteamVR's IPC; it starts and stops with SteamVR. After=steamvr.service frametop-input-relay.service PartOf=steamvr.service +Requisite=steamvr.service [Service] # Runs in the dev container (built there against its libraries). The helper process diff --git a/pointer/helper/ft-pointer.cpp b/pointer/helper/ft-pointer.cpp index ea5864f..10c922a 100644 --- a/pointer/helper/ft-pointer.cpp +++ b/pointer/helper/ft-pointer.cpp @@ -567,7 +567,11 @@ int main() { vr::EVRInitError err = vr::VRInitError_None; while (true) { - vr::VR_Init(&err, vr::VRApplication_Overlay); + vr::VR_Init(&err, vr::VRApplication_Background); + if (err == vr::VRInitError_None) { + vr::VR_Shutdown(); + vr::VR_Init(&err, vr::VRApplication_Overlay); + } if (err == vr::VRInitError_None) break; std::fprintf(stderr, "waiting for SteamVR: %s\n", vr::VR_GetVRInitErrorAsEnglishDescription(err)); std::this_thread::sleep_for(std::chrono::seconds(2)); diff --git a/screens/vr.cpp b/screens/vr.cpp index 755f058..f02c6e4 100644 --- a/screens/vr.cpp +++ b/screens/vr.cpp @@ -1367,7 +1367,11 @@ extern "C" { bool ft_vr_init(void) { vr::EVRInitError err = vr::VRInitError_None; - vr::VR_Init(&err, vr::VRApplication_Overlay); + vr::VR_Init(&err, vr::VRApplication_Background); + if (err == vr::VRInitError_None) { + vr::VR_Shutdown(); + vr::VR_Init(&err, vr::VRApplication_Overlay); + } if (err != vr::VRInitError_None) { std::fprintf(stderr, "openvr: %s\n", vr::VR_GetVRInitErrorAsEnglishDescription(err)); return false; From c8fc6351bb51a123b18be191b044e89885e3654a Mon Sep 17 00:00:00 2001 From: DeeJanuz <45082401+DeeJanuz@users.noreply.github.com> Date: Wed, 30 Sep 2026 10:57:36 -0600 Subject: [PATCH 13/13] Floating windows: fix the scale crash, the click offset, and placement - KWin's nested backend makes an output the size it's configured to times its scale, so after Meta+scroll every size ft-floatd sent was multiplied again, and an odd result disconnected KWin (buffer not divisible by its scale). ft-floatd now asks for sizes in the output's scaled terms (kwin_size), and asks again after each scale change. - SteamVR reports mouse positions on a panel with texture bounds in the whole texture, not the crop, so clicks on a floating window landed up to ~200 px off. The mouse scale is now the buffer's size, as on a screen. - A floated window starts 30 cm in front of its screen (was 5 cm), so it's easy to point at apart from the screen behind it. - No 1 s wait before a spare turns on (a disabled output never commits), and the login splash on the spares isn't taken for floating windows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --- float/ft_floatd.py | 90 +++++++++++++++++++++++++++++++------------- screens/compositor.c | 6 ++- screens/vr.cpp | 25 +++++------- 3 files changed, 78 insertions(+), 43 deletions(-) diff --git a/float/ft_floatd.py b/float/ft_floatd.py index 5f8327a..ecef8e1 100644 --- a/float/ft_floatd.py +++ b/float/ft_floatd.py @@ -47,17 +47,32 @@ PATH = "/Float" SCRIPT = "frametop-float" POLL_SECONDS = 20 # NextCommand answers empty after this (KWin's D-Bus timeout is 25 s) SPARE_X, SPARE_CELL = 12000, 5000 # spares in KWin's layout: a grid from here, 4 across -PULL_OUT = 0.05 # a floated window starts this far in front of its screen (metres) +PULL_OUT = 0.3 # a floated window starts this far in front of its screen (metres), clear of it DEFAULT_MPP = 1.6 / 1920 # metres per pixel when ft-screens can't say (no SteamVR) DEBUG = os.environ.get("FT_FLOAT_DEBUG") == "1" # log every event from the script -def fit(px, scale): - """An output size KWin can draw at this scale: its nested backend gives the buffer a whole - buffer scale (1.2 -> 2), and a buffer that isn't a multiple of it is a protocol error that - disconnects KWin. So round up to a multiple; the margin takes the extra pixels.""" - k = max(1, math.ceil(scale - 1e-6)) - return (px + k - 1) // k * k +def whole(*scales): + """The multiple a spare's size in pixels must be at these scales: KWin's nested backend gives + the buffer a whole buffer scale (1.2 -> 2), and a buffer that isn't a multiple of it is a + protocol error that disconnects KWin.""" + k = 1 + for s in scales: + k = math.lcm(k, max(1, math.ceil(s - 1e-6))) + return k + + +def kwin_size(px, scale, k): + """What to ask ft-screens for so a spare comes out at least px pixels, a multiple of k. KWin + makes a nested output the size it's configured to times its scale, rounded. Returns (the + size to ask for, the pixels it comes out); the margin takes the extra pixels.""" + n = max(1, math.floor(px / scale)) + while True: + exact = n * scale + p = math.floor(exact + 0.5) + if p >= px and p % k == 0 and abs(exact - math.floor(exact) - 0.5) > 1e-6: + return n, p + n += 1 def log(*args): @@ -126,6 +141,8 @@ class Slot: self.index = screens + k + 1 # ft-screens' number (1-based) self.pos = (SPARE_X + (k % 4) * SPARE_CELL, (k // 4) * SPARE_CELL) self.size = None # its output's size in pixels, as last set + self.want = None # the size in pixels asked for (size is at least that) + self.kscale = 1.0 # its output's scale in KWin, as last set self.window = None # Float @@ -257,12 +274,14 @@ class Daemon: if not ev.get("normal"): return if slot.window is None: + if ev.get("cls") == "ksplashqml": # the login splash, on every output at first + return # Floating when ft-floatd (re)started: take it over where it is. f = Float(wid, slot, None) slot.window = f self.floats[wid] = f - f.scale = output_scales().get(slot.output, 1.0) - log(f"{wid[:9]} already floats on {slot.output}") + f.scale = slot.kscale = output_scales().get(slot.output, 1.0) + log(f"{wid[:9]} ({ev.get('cls')}) already floats on {slot.output}") self.follow(f, ev) return # A new window that opened on a floating window's output: windows of floating apps @@ -288,6 +307,15 @@ class Daemon: log(f"{slot.output} didn't take {size[0]}x{size[1]} in time") return False + def set_size(self, slot, want, k=None): + """Size a spare's output to at least want (pixels) at its current scale in KWin.""" + if want == slot.want: + return + k = k or whole(slot.kscale) + (w, pw), (h, ph) = kwin_size(want[0], slot.kscale, k), kwin_size(want[1], slot.kscale, k) + slot.want, slot.size = want, (pw, ph) + self.screens.ask(f"size {slot.index} {w} {h}") + def disable_unused(self): off = [f"output.{s.output}.disable" for s in self.slots if s.window is None] if off: @@ -323,21 +351,24 @@ class Daemon: f = Float(wid, slot, {"output": ev["output"], "frame": ev["frame"], "onAllDesktops": ev.get("onAllDesktops")}) slot.window = f self.floats[wid] = f - f.scale = output_scales().get(ev["output"], 1.0) + scales = output_scales() + f.scale = scales.get(ev["output"], 1.0) + slot.kscale = scales.get(slot.output, slot.kscale) f.mpp = self.screen_mpp(ev["output"]) fr, s, m = ev["frame"], f.scale, self.margin w, h = round(fr["w"] * s), round(fr["h"] * s) - slot.size = (fit(w + 2 * m, s), fit(h + 2 * m, s)) log(f"{wid[:9]} ({ev.get('cls')}) floats on {slot.output}: {w}x{h} px, scale {s:g}") - # The spare's size first (while it's off, so its first frame is right), then its panel, - # then turn it on, then the window. - self.screens.ask(f"size {slot.index} {slot.size[0]} {slot.size[1]}") - self.sized(slot, slot.size) + # The spare's size first (while it's off, so its first frame is right; it's sized at + # its old scale, for pixels that suit the new one), then its panel, then turn it on, + # then the window. + slot.want = None + self.set_size(slot, (w + 2 * m, h + 2 * m), whole(slot.kscale, s)) self.screens.ask(f"scale {slot.index} {s:g}") # for pointer positions (KWin's units) self.set_panel(f, (m, m, w, h), title=round((ev["client"]["y"] - fr["y"]) * s)) self.place_panel(f, ev) kscreen(f"output.{slot.output}.enable", f"output.{slot.output}.scale.{s:g}", f"output.{slot.output}.position.{slot.pos[0]},{slot.pos[1]}") + self.rescaled(slot, s) self.command(cmd="place", id=wid, output=slot.output, x=slot.pos[0] + m / s, y=slot.pos[1] + m / s, w=fr["w"], h=fr["h"], onAllDesktops=True) @@ -378,11 +409,7 @@ class Daemon: if not full: f.normal = (w, h) m = 0 if full else self.margin - want = f.normal if full and f.normal else (w + 2 * m, h + 2 * m) - want = (fit(want[0], s), fit(want[1], s)) - if want != slot.size: - slot.size = want - self.screens.ask(f"size {slot.index} {want[0]} {want[1]}") + self.set_size(slot, f.normal if full and f.normal else (w + 2 * m, h + 2 * m)) if not full: x0, y0 = slot.pos[0] + m / s, slot.pos[1] + m / s if abs(fr["x"] - x0) > 0.5 or abs(fr["y"] - y0) > 0.5: @@ -403,16 +430,26 @@ class Daemon: slot, m = f.slot, self.margin log(f"{f.id[:9]} scale {f.scale:g} -> {s:g}") f.scale = s - # The output's size must suit the new scale before KWin draws at it (see fit). - size = (fit(slot.size[0], s), fit(slot.size[1], s)) - if size != slot.size: - slot.size = size - self.screens.ask(f"size {slot.index} {size[0]} {size[1]}") - self.sized(slot, size) + # The output's size in pixels must suit the new scale before KWin draws at it (see whole). + k = whole(slot.kscale, s) + if slot.size[0] % k or slot.size[1] % k: + slot.want = None + self.set_size(slot, slot.size, k) + self.sized(slot, slot.size) kscreen(f"output.{slot.output}.scale.{s:g}") self.screens.ask(f"scale {slot.index} {s:g}") + self.rescaled(slot, s) self.command(cmd="geometry", id=f.id, x=slot.pos[0] + m / s, y=slot.pos[1] + m / s, w=w / s, h=h / s) + def rescaled(self, slot, s): + """KWin has the spare at scale s now: ask for its size again in the new scale's terms, + or the next configure (any size, or KWin's own) would make it the old size times s.""" + if s == slot.kscale: + return + slot.kscale = s + want, slot.want = slot.size, None + self.set_size(slot, want) + def dock(self, f, frame=None): """Back where it came from (or onto screen 1 if we don't know).""" saved = f.saved or {"output": "WL-0", "frame": dict(f.frame or {"x": 100, "y": 100, "w": 800, "h": 600}), @@ -427,6 +464,7 @@ class Daemon: slot = f.slot if slot.window is f: slot.window = None + slot.want = None for sub_id in list(f.subs): self.drop_sub(sub_id) self.screens.ask(f"unfloat {slot.index}", quiet=True) diff --git a/screens/compositor.c b/screens/compositor.c index b177d30..364e29f 100644 --- a/screens/compositor.c +++ b/screens/compositor.c @@ -434,8 +434,10 @@ static int control_readable(int fd, uint32_t mask, void *data) { int value, index, w, h; double scale; if (sscanf(buf, "size %d %d %d", &index, &w, &h) == 3) { - // A new resolution for a screen, live: KWin resizes the screen to match. - if (index < 1 || index > MAX_SCREENS || !s->screens[index - 1] || w < 320 || h < 200 || w > 16384 || + // A new resolution for a screen, live: KWin resizes the screen to match. (KWin makes + // it this size times its scale; ft-floatd sends spares' sizes divided by theirs.) + const int min_w = index - 1 < s->n_config ? 320 : 64, min_h = index - 1 < s->n_config ? 200 : 64; + if (index < 1 || index > MAX_SCREENS || !s->screens[index - 1] || w < min_w || h < min_h || w > 16384 || h > 16384) { snprintf(reply, sizeof reply, "error bad screen or size"); } else { diff --git a/screens/vr.cpp b/screens/vr.cpp index f02c6e4..7847971 100644 --- a/screens/vr.cpp +++ b/screens/vr.cpp @@ -276,10 +276,8 @@ struct Screen { return width > 0 ? metres * height / width : metres * 9 / 16; } // Buffer pixels from OpenVR's mouse position on the panel (its origin is bottom left). - void ToBuffer(double mx, double my, double *x, double *y) const { - if (floating && cropW > 0) *x = cropX + mx, *y = cropY + cropH - my; - else *x = mx, *y = height - my; - } + // A cropped panel too: SteamVR gives the position in the whole texture, not the crop. + void ToBuffer(double mx, double my, double *x, double *y) const { *x = mx, *y = height - my; } std::array<vr::VROverlayHandle_t, 6> Controls() const { return {bar, curveButton, rollButton, handle, dockButton, closeButton}; } @@ -1150,14 +1148,15 @@ void Push(Screen &s, double notches) { // ---------------------------------------------------------------- floating windows -// Show the window's rectangle of the buffer, and map the mouse to it (see ToBuffer). -// Texture bounds are fractions of the buffer, v from the top. +// Show the window's rectangle of the buffer. Texture bounds are fractions of the buffer, v +// from the top. SteamVR reports mouse positions in the whole texture (the bounds applied), so +// the mouse scale is the buffer's size, as on a screen (see ToBuffer). void CropOverlay(vr::VROverlayHandle_t o, const Screen &s, int x, int y, int w, int h) { if (s.width <= 0 || s.height <= 0 || w <= 0 || h <= 0) return; vr::VRTextureBounds_t b = {float(x) / s.width, float(y) / s.height, float(x + w) / s.width, float(y + h) / s.height}; vr::VROverlay()->SetOverlayTextureBounds(o, &b); - vr::HmdVector2_t scale = {float(w), float(h)}; + vr::HmdVector2_t scale = {float(s.width), float(s.height)}; vr::VROverlay()->SetOverlayMouseScale(o, &scale); } @@ -1571,15 +1570,11 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { RefreshPoses(); for (auto &[index, s] : g_screens) { vr::VREvent_t ev; - // The screen itself: input for KWin. // The screen itself (and a floating window's popups): input for KWin. - auto panelEvent = [&](const vr::VREvent_t &ev, bool sub, const Sub *from) { + auto panelEvent = [&](const vr::VREvent_t &ev, bool sub) { ft_event e{}; e.screen = index; - auto at = [&] { - if (from) e.x = from->x + ev.data.mouse.x, e.y = from->y + from->h - ev.data.mouse.y; - else s.ToBuffer(ev.data.mouse.x, ev.data.mouse.y, &e.x, &e.y); - }; + auto at = [&] { s.ToBuffer(ev.data.mouse.x, ev.data.mouse.y, &e.x, &e.y); }; switch (ev.eventType) { case vr::VREvent_MouseMove: if (s.titleCarry) return; // KWin's pointer stays where the title bar was pressed @@ -1624,9 +1619,9 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) { } handle(&e, data); }; - while (vr::VROverlay()->PollNextOverlayEvent(s.overlay, &ev, sizeof ev)) panelEvent(ev, false, nullptr); + while (vr::VROverlay()->PollNextOverlayEvent(s.overlay, &ev, sizeof ev)) panelEvent(ev, false); for (const auto &[k, sub] : s.subs) - while (vr::VROverlay()->PollNextOverlayEvent(sub.overlay, &ev, sizeof ev)) panelEvent(ev, true, &sub); + while (vr::VROverlay()->PollNextOverlayEvent(sub.overlay, &ev, sizeof ev)) panelEvent(ev, true); // The controls light up under a laser. auto hover = [&](int k) { const bool on = ev.eventType == vr::VREvent_MouseMove || ev.eventType == vr::VREvent_FocusEnter;