Put clicks where the cursor is on scaled desktop screens

With a screen's scale set to anything but 100%, clicks landed away from
the cursor, further off the further from the top left (#3). ft-screens
hands KWin panel positions in buffer pixels, and KWin's nested Wayland
backend (6.2.5, WaylandInputDevice) adds surface coordinates to its
output's logical position without dividing by the output's scale. At
125% a click at the middle of a 3440x1440 screen, (1720, 720), reached
KWin as logical (1720, 720), pixel (2150, 900).

ft-screens now keeps a scale per screen and divides pointer positions by
it. ft-layout sends each screen's scale, as KWin reports it after
applying, with a new "scale N s" command, whenever it applies scales:
at desktop start and from Frametop Display Settings. Screens default to
1, so an ft-layout that never sends it keeps the old behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

docs/reference.md

@@ -53,12 +53,14 @@ In the last three modes the hotkey shows the screens anyway. Two more settings o
 
 Input from the lasers reaches KWin through ft-screens' own seat. Keys come from the input relay, from pass-through keyboards and any key a pointer device passes through. Typing follows your last click: after a click on a screen it goes to the desktop, even with the SteamVR dashboard open, and after a mouse click on any other panel (the dashboard, Steam, an app like Spotify) it goes there instead. While it goes to the desktop, the relay grabs pass-through keyboards so gamescope, which reads every keyboard itself, doesn't type them into the Steam app too. A program that watches every keyboard for a hotkey loses a grabbed one; with `SHARE_KEYS=1` in `~/.config/frametop.conf`, their keys also go to `@frametop_keys` for it. That's off by default, since any local process that binds the name first would get everything typed into the desktop. Hidden screens don't take typing.
 
+KWin's nested backend doesn't undo a screen's scale on pointer input, so ft-screens divides panel positions (in pixels) by it. `ft-layout` sends it each screen's scale as KWin reports it (`scale N s`) whenever it applies scales: at desktop start and from Frametop Display Settings. A scale changed only in Plasma's own display settings is put back to the Frametop layout's the next time `ft-layout` runs.
+
 ft-screens listens for datagrams on the abstract socket `@ft_screens` and replies to the sender:
 
 ```
 place N x y z yaw pitch roll     width N metres          curve N radius|on|off
 pin N|all left|right [matrix]    unpin N|all             size N w h
-get N    screens    head    state    key code value
+get N    screens    head    state    key code value    scale N s
 visibility always|dashboard|gesture|toggle    wrist degrees    gesture left|right degrees
 hide | show | toggle    controllers always|outside_games|dashboard    ingames hide|visible
 ```

layout/ft_layout.py

@@ -645,6 +645,21 @@ def arrangement(count):
     return [[b["i"] for b in sorted(c, key=lambda b: -b["pitch"])] for c in columns] + [[i] for i in pinned]
 
 
+def send_scales(outs):
+    """KWin's scale for each screen, as it is now, to ft-screens: KWin's nested backend
+    doesn't undo its scale on pointer input, so ft-screens does (panel pixels / scale)."""
+    if backend() != "screens":
+        return
+    try:
+        sock = screens_socket()
+        for i, o in enumerate(outs):
+            reply = sock.ask(f"scale {i + 1} {float(o.get('scale', 1)):g}")
+            if not reply.startswith("ok"):
+                log(f"screen {i + 1}: scale: {reply}")
+    except RuntimeError as e:
+        log(f"scale: {e}")
+
+
 def apply_scales():
     """Per-screen scale and rotation, positions side by side, and the primary screen (the
     taskbar goes there) to KWin, which keeps them in the session's config."""
@@ -670,6 +685,7 @@ def apply_scales():
     # Laid out as you see the screens around you (arrangement), centred on one line, so
     # the pointer and dragged windows cross to the screen you see next to this one.
     outs = outputs(env)
+    send_scales(outs)
     # kscreen's "size" is in pixels (already turned for a rotation); positions are in
     # logical units, the pixels divided by the scale (KWin rounds up).
     sizes = [(math.ceil(o["size"]["width"] / float(o.get("scale", 1)) - 1e-6),

screens/compositor.c

@@ -7,7 +7,10 @@
 //   - KWin renders into DMA-BUFs and hands them to us (linux-dmabuf). We draw nothing:
 //     each buffer goes to SteamVR as the panel's texture (vr.cpp, ImportDmabuf).
 //   - Pointer input from the panels (controller lasers, the 3D mouse) goes to KWin through
-//     our seat, as if we were a normal desktop.
+//     our seat, as if we were a normal desktop. KWin's nested backend adds our surface
+//     coordinates to its output's logical position without undoing its own scale, and its
+//     buffers are in pixels, so a panel position in pixels is divided by the screen's KWin
+//     scale first ("scale <screen> <s>", from ft-layout).
 //
 // Usage: ft-screens [--socket NAME] [--screen WxH@METRES]... [-- COMMAND ARGS...]
 //   --socket    Wayland socket name in $XDG_RUNTIME_DIR (default ft-screens-0)
@@ -86,6 +89,7 @@ struct server {
     struct wl_listener new_toplevel, new_decoration;
     struct screen *screens[MAX_SCREENS];
     struct config config[MAX_SCREENS];
+    double scale[MAX_SCREENS];  // KWin's scale for each screen (panel pixels per logical unit)
     int n_config, n_screens;
     struct wl_list buffers;  // tracked_buffer
     struct wl_event_source *tick;
@@ -243,14 +247,15 @@ static void handle_vr_event(const struct ft_event *e, void *data) {
     struct screen *sc = s->screens[e->screen];
     struct wlr_surface *surface = sc->toplevel->base->surface;
     const uint32_t t = now_ms();
+    const double x = e->x / s->scale[e->screen], y = e->y / s->scale[e->screen];  // KWin's units
     switch (e->type) {
         case FT_MOTION:
         case FT_BUTTON:
             if (s->pointer_focus != sc) {
-                wlr_seat_pointer_notify_enter(s->seat, surface, e->x, e->y);
+                wlr_seat_pointer_notify_enter(s->seat, surface, x, y);
                 s->pointer_focus = sc;
             }
-            wlr_seat_pointer_notify_motion(s->seat, t, e->x, e->y);
+            wlr_seat_pointer_notify_motion(s->seat, t, x, y);
             if (e->type == FT_BUTTON) {
                 wlr_seat_pointer_notify_button(s->seat, t, e->button,
                                                e->pressed ? WL_POINTER_BUTTON_STATE_PRESSED
@@ -360,7 +365,8 @@ static void handle_key(struct server *s, uint32_t code, int value, char *reply,
 }
 
 // Control socket: abstract datagram @ft_screens. Here: "size <screen> <w> <h>" (a new
-// resolution, live), "key <code> <value>", and "click <overlay key>" (from the pointer
+// resolution, live), "scale <screen> <s>" (KWin's scale for it, from ft-layout),
+// "key <code> <value>", and "click <overlay key>" (from the pointer
 // helper: a mouse click landed on that panel, "-" for none); the rest is in vr.cpp
 // (ft_vr_command).
 static int control_readable(int fd, uint32_t mask, void *data) {
@@ -373,6 +379,7 @@ static int control_readable(int fd, uint32_t mask, void *data) {
         buf[n] = 0;
         unsigned code;
         int value, index, w, h;
+        double scale;
         if (sscanf(buf, "size %d %d %d", &index, &w, &h) == 3) {
             // A new resolution for a screen, live: KWin resizes the screen to match.
             if (index < 1 || index > MAX_SCREENS || !s->screens[index - 1] || w < 320 || h < 200 || w > 16384 ||
@@ -383,6 +390,14 @@ static int control_readable(int fd, uint32_t mask, void *data) {
                 wlr_xdg_toplevel_set_size(s->screens[index - 1]->toplevel, w, h);
                 snprintf(reply, sizeof reply, "ok");
             }
+        } else if (sscanf(buf, "scale %d %lf", &index, &scale) == 2) {
+            if (index < 1 || index > MAX_SCREENS || !(scale >= 0.25 && scale <= 8)) {
+                snprintf(reply, sizeof reply, "error bad screen or scale");
+            } else {
+                if (s->scale[index - 1] != scale) wlr_log(WLR_INFO, "screen %d: KWin scale %g", index, scale);
+                s->scale[index - 1] = scale;
+                snprintf(reply, sizeof reply, "ok");
+            }
         } else if (sscanf(buf, "key %u %d", &code, &value) == 2) {
             handle_key(s, code, value, reply, sizeof reply);
             len = sizeof from;
@@ -451,6 +466,7 @@ static bool setup_dmabuf(struct server *s) {
 
 int main(int argc, char **argv) {
     struct server s = {0};
+    for (int i = 0; i < MAX_SCREENS; ++i) s.scale[i] = 1;
     const char *socket_name = "ft-screens-0";
     char **command = NULL;
     for (int i = 1; i < argc; ++i) {