Add a keyboard for text fields on the desktop

Fixes #7: SteamVR's keyboard never came up for the desktop's apps, and opening it for our panels doesn't work well on the Frame (it's Steam's own panel, mounted in the dashboard's scene, it follows the laser between panels, and it takes the controllers over to SteamVR's laser). - KWin starts input/ft-textinput as the desktop's input method. It tells the input relay when a text field gains or loses focus, and the relay asks ft-screens to open or close the keyboard. The session drops the QT_IM_MODULE=xim and GTK_IM_MODULE=xim that the gamescope session sets, or Qt and GTK apps never report text fields. - The keyboard is ft-screens' own panel (screens/keyboard.cpp): a US laptop layout, typed with a controller's laser or the 3D mouse. It opens 0.7 m in front of you, below your eyes and facing you. It has a grab bar to move it, a Close key, latching Shift, Ctrl and Alt, and repeat on a held key. It's drawn into shared DMA-BUFs, so it doesn't flicker. Its keys reach the focused screen as key presses, so every app takes them. - It steps aside while the Steam menu or Steam's own keyboard is up and comes back after. A layout reset closes it, and it doesn't open without a head pose. - Frametop Input Settings has a Keyboard page: open it for every text field, only while no keyboard is connected (the default), only from a mapped button (the new Open/close keyboard action, for mice and controllers), or never. A switch keeps it open until you close it. - The pointer helper treats every frametop.* overlay as a real panel. The keyboard's shared texture reports 0x0 like SteamVR's scene-graph controls, and the helper had given it their wide catch radius. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 00:00:07 +02:00 · 2026-09-30 13:41:47 -06:00
1 parent 3aea571496
commit 5714978e65
@@ -117,6 +117,10 @@ An ungrabbed keyboard reaches both sides at once. In VR, gamescope reads every i

 Volume keys must never reach gamescope. With the openvr backend, gamescope sends volume up and down to Steam by moving keyboard focus to Steam for the key and then back to the previously focused surface. When nothing had focus, the one it moves back to is null, and wlroots aborts on a null focus surface (`wlr_seat_keyboard_notify_enter: Assertion 'surface' failed`), which ends the whole VR session. Keyboard focus is often empty while you work in VR, so one press of the headset's volume button could take everything down. gamescope reads the headset's buttons and every keyboard itself (`InputStealer`), as do SteamVR's processes, so the relay has to stop volume keys at the device. Grabbing `gpio-keys` would also take the headset's click button, so the relay remaps the volume entries in each device's keymap (`EVIOCSKEYCODE`) and handles the stand-in codes itself. That fix covers every device at once, including keyboards that aren't grabbed.

+Frametop's keyboard opens by itself for a text field on the desktop. The apps run inside the nested KWin, so only KWin knows when a text field has focus, and the way it tells anyone is its input method protocol (`zwp_input_method_v1`): KWin starts one input method program and activates it whenever the focused app turns on text input. `input/ft-textinput` is that program, speaking the Wayland wire protocol directly so it needs nothing but Python on the host. It only reports focus. The gamescope session puts `QT_IM_MODULE=xim` and `GTK_IM_MODULE=xim` in the systemd user environment; with those, Qt and GTK apps use X input methods and never turn on Wayland text input, so the session script drops them.
+
+The keyboard itself is ft-screens' own panel (`screens/keyboard.cpp`). We tried SteamVR's first (`ShowKeyboardForOverlay`), and on the Frame it doesn't fit a desktop. It's Steam's own panel (`valve.steam.gamepadui.keyboard`), which SteamVR mounts in the dashboard's scene, so with the dashboard closed it opened but wasn't drawn. Placing it in the room ourselves (`SetKeyboardTransformAbsolute`) made it show, but SteamVR moves it to whichever overlay the laser goes to and mounts it again, and while it's open the controllers switch to SteamVR's own laser. Our panel is an overlay like the screens' controls: any laser or the 3D mouse clicks it, nothing moves it, and its keys go out as key presses on ft-screens' seat rather than as text handed back to the input method. So nothing typed leaves ft-screens (a socket to the input method could be claimed by any local process, like `@frametop_keys`), apps without text input (X11, Electron) take the keys too, and they mean what the desktop's keyboard layout says. It's drawn on the CPU and uploaded with `SetOverlayRaw` when a key's look changes; the labels come from stb_truetype, so the container needs no text rendering stack.
+
 The Frame controllers can be mapped like mouse buttons, but they aren't input devices on the host: they reach SteamVR over the headset's own radio, and no evdev or hidraw node exists for them. So only a SteamVR client can read them. Overlay apps normally get controller input only while they have input focus, which a background helper never has. SteamVR's experimental global action set priority (`steamvr/globalActionSetPriority`, "Enable global input from overlays") lets an overlay's action set receive input anyway, and takes the inputs it binds from the scene app. Binding every button would take them all from games, so the helper's action manifest puts each button in an action set of its own, and it activates only the sets of mapped buttons. The mapping itself stays in the relay, which does the action, so mice and controllers share one list of actions.

 ## The desktop session
@@ -55,6 +55,8 @@ In the last three modes the hotkey shows the screens anyway. Two more settings o

 Input from the lasers reaches KWin through ft-screens' own seat. Keys come from the input relay, from pass-through keyboards and any key a pointer device passes through. Typing follows your last click: after a click on a screen it goes to the desktop, even with the SteamVR dashboard open, and after a mouse click on any other panel (the dashboard, Steam, an app like Spotify) it goes there instead. While it goes to the desktop, the relay grabs pass-through keyboards so gamescope, which reads every keyboard itself, doesn't type them into the Steam app too. A program that watches every keyboard for a hotkey loses a grabbed one; with `SHARE_KEYS=1` in `~/.config/frametop.conf`, their keys also go to `@frametop_keys` for it. That's off by default, since any local process that binds the name first would get everything typed into the desktop. Hidden screens don't take typing.

+Frametop's keyboard opens by itself when a text field on the desktop gets focus, and stays open until its Close key, a layout reset, or a mapped button closes it (or, with Keep it open off in Frametop Input Settings, until the text field loses focus). While the Steam menu (the dashboard) or Steam's own keyboard is up, it steps aside, and it comes back where it was when they're gone; one asked for meanwhile appears then. In the "only with the dashboard" visibility mode, the dashboard doesn't count. It doesn't open without a head pose (the headset in standby). It's a panel of keys (a US laptop layout, with Esc where Caps Lock would be, arrows, and a Close key) that ft-screens shows 0.7 m in front of you and below your eyes, facing you. It stays where it opened, and its grab bar (the pill along the top) moves it like a screen's. Type on it with a controller's laser or the 3D mouse. Shift, Ctrl and Alt latch for the next key, and a held key repeats. KWin starts `input/ft-textinput` as the desktop's input method, and KWin activates it whenever the focused app turns on text input for a field. It tells the relay (`textfield 1` or `0`), the relay decides by the Keyboard setting in Frametop Input Settings, and ft-screens opens the keyboard for the screen that has keyboard focus (`vrkeyboard show`, `hide`, or `toggle` from a mapped button). Its keys reach the focused screen as key presses, so it works in every app, but only apps that use Wayland text input (Qt, GTK, Firefox) open it by themselves; Chromium, Electron and X11 apps need the button. The session drops the `QT_IM_MODULE=xim` and `GTK_IM_MODULE=xim` that the gamescope session sets, or Qt and GTK apps wouldn't use Wayland text input either.
+
 KWin's nested backend doesn't undo a screen's scale on pointer input, so ft-screens divides panel positions (in pixels) by it. `ft-layout` sends it each screen's scale as KWin reports it (`scale N s`) whenever it applies scales: at desktop start and from Frametop Display Settings. A scale changed only in Plasma's own display settings is put back to the Frametop layout's the next time `ft-layout` runs.

 ft-screens listens for datagrams on the abstract socket `@ft_screens` and replies to the sender:
@@ -62,7 +64,7 @@ ft-screens listens for datagrams on the abstract socket `@ft_screens` and replie
 ```
 place N x y z yaw pitch roll     width N metres          curve N radius|on|off
 pin N|all left|right|head [matrix]    unpin N|all        size N w h
-get N    screens    head    state    key code value    scale N s
+get N    screens    head    state    key code value    scale N s    vrkeyboard show|hide|toggle
 visibility always|dashboard|gesture|toggle    wrist degrees    gesture left|right degrees
 hide | show | toggle    controllers always|outside_games|dashboard    ingames hide|visible
 ```
@@ -106,11 +108,12 @@ The pointer settings are in `~/.config/frametop.conf`: `POINTER_SENSITIVITY`, `P

 ## Frametop Input Settings

-A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has seven pages:
+A Kirigami app with a Python backend, in the Plasma menu under Settings. It runs in the `dev` container and talks to the relay over its control socket, `@frametop_relay`. It has eight pages:

 - Devices lists every USB and Bluetooth mouse and keyboard, with a light that flashes when the device is used. Each device gets a role: 3D pointer (grabbed, drives the pointer; the default for anything with a mouse), Pass through (grabbed only while typing goes to the desktop; the default for keyboards, where a Meta tap toggles the dashboard if `META_DASHBOARD=1` is in `~/.config/frametop.conf`), or Ignore. A device is identified by its Bluetooth address, or its USB ids and name, so all of its input nodes share one role. Forget drops everything saved for a device.
- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, head follow on or off, gaze pointer on or off, faster or slower, pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
+- Buttons maps a pointer device's buttons. Choose Capture a button, press the button or key, then pick an action: a click, back, scroll, toggle dashboard, recenter, pointer on or off, open or close the keyboard, head follow on or off, gaze pointer on or off, faster or slower, pass the key through, or nothing. Devices with saved mappings are listed even while they're asleep.
 - Controllers maps the Frame controllers' buttons (every button but the system button) to the same actions, except passing a key through. Capture a button and press it on a controller, or pick it from the list. The controllers aren't input devices on the host; only SteamVR sees them. So the pointer helper reads them with SteamVR input (`pointer/helper/vrbuttons.h`, `pointer/helper/actions/`) and sends presses to the relay (`vrbtn right/a 1`), which does the mapped action. The helper only takes the buttons that are mapped (the relay tells it with `vrbind`), at an overlay-global priority, and only while no game (scene application) runs, so games keep every button; with In games on (`controller_in_games`), a mapped button is taken from games too. That needs SteamVR's "Enable global input from overlays (Experimental)" setting (`steamvr/globalActionSetPriority`), which the page's Global input switch turns on and off. Mappings are saved as `controller_buttons` in `~/.config/frametop-input.json`.
+- Keyboard sets when Frametop's keyboard opens: whenever a text field is selected; only while no pass-through keyboard is connected (the default; keyboards other programs make through uinput, like frame-voice's, don't count); only with a mouse or controller button mapped to Open/close keyboard; or never, which turns the button off too. Keep it open (on by default, `vr_keyboard_persist`) leaves it open after the text field loses focus. The mode is saved as `vr_keyboard` in `~/.config/frametop-input.json`, and the page lists the keyboards that count as connected.
 - Pointer has a Head follow switch and sliders for the pointer settings, which apply immediately, and a Recenter button.
 - Ignored panels lists the SteamVR overlays that are showing, grouped by app (the first two parts of the overlay key, such as `sasaken.frame-perf-overlay`), from the pointer helper (`overlays`). Tick a panel, or Ignore the whole app, and the pointer passes through it to what's behind. It's for panels you only look at, like a performance overlay that follows your view. The list is saved as `POINTER_IGNORE` in `~/.config/frametop.conf`: comma-separated overlay keys, where a shell pattern like `vendor.app*` covers a whole app, including panels it opens later. The helper reloads at once. Frametop's own screens aren't listed, and entries for apps that aren't open are listed below, to remove.
 - Gaze has the gaze pointer switch (on now and from now on; a mapped button toggles it until the helper restarts), the gaze mode sliders, the gaze service's state (headset, samples per second, how often the tracker is losing each eye, the calibration, the nudges learned), and Calibrate (opens the gaze probe), Check headset fit (opens the probe's Headset fit mode), Reload calibration, and Forget nudges.
@@ -62,9 +62,18 @@ ACTION_LABELS = {
    "follow_toggle": "Head follow on/off (experimental)", "gaze_toggle": "Gaze pointer on/off (experimental)",
    "sens_up": "Faster pointer",
    "sens_down": "Slower pointer", "layout_reset": "Reset desktop screen layout",
-    "screens_toggle": "Hide/show desktop screens", "key": "Pass through as key",
+    "screens_toggle": "Hide/show desktop screens", "keyboard_toggle": "Open/close keyboard",
+    "key": "Pass through as key",
    "none": "Do nothing",
 }
+# When Frametop's keyboard opens ("vr_keyboard" in the rules; the relay's
+# VR_KEYBOARD_MODES). "no_keyboard" is the default.
+VR_KEYBOARD_MODES = {
+    "always": "When a text field is selected",
+    "no_keyboard": "When a text field is selected and no keyboard is connected",
+    "button": "Only with a mapped mouse or controller button",
+    "never": "Never",
+}
 ROLE_LABELS = {"pointer": "3D pointer", "passthrough": "Pass through", "ignore": "Ignore"}
 # Frame controller buttons the pointer helper can read (pointer/helper/vrbuttons.h). The
 # system button stays SteamVR's.
@@ -363,7 +372,8 @@ class Backend(QObject):
        grouped = {}
        for n in self._nodes:
            d = grouped.setdefault(n["id"], {"id": n["id"], "name": n["name"], "bus": n["bus"], "kinds": [],
-                                             "nodes": [], "role": n["role"], "grabbed": False})
+                                             "nodes": [], "role": n["role"], "grabbed": False,
+                                             "uinput": n.get("uinput", False)})
            d["nodes"].append(n["path"])
            d["kinds"] = sorted(set(d["kinds"]) | set(n["kinds"]))
            d["grabbed"] = d["grabbed"] or n["grabbed"]
@@ -601,6 +611,36 @@ class Backend(QObject):
    def removeIgnore(self, pattern):
        self._save_ignore([p for p in self._ignore_list() if p != pattern], f"{pattern}: no longer ignored")

+    # --- Frametop's keyboard ---
+    @Property("QVariantList", constant=True)
+    def vrKeyboardModes(self):
+        return [{"value": k, "text": v} for k, v in VR_KEYBOARD_MODES.items()]
+
+    @Property(str, notify=mappingsChanged)
+    def vrKeyboard(self):
+        mode = read_json(RULES_PATH).get("vr_keyboard")
+        return mode if mode in VR_KEYBOARD_MODES else "no_keyboard"
+
+    @Property(bool, notify=mappingsChanged)
+    def vrKeyboardPersist(self):
+        return bool(read_json(RULES_PATH).get("vr_keyboard_persist", True))
+
+    @Slot(bool)
+    def setVrKeyboardPersist(self, on):
+        rules = read_json(RULES_PATH)
+        rules["vr_keyboard_persist"] = bool(on)
+        self._save_rules(rules)
+        self.message.emit("Keyboard: " + ("stays open until you hide it" if on else "closes with the text field"), False)
+
+    @Slot(str)
+    def setVrKeyboard(self, mode):
+        if mode not in VR_KEYBOARD_MODES:
+            return
+        rules = read_json(RULES_PATH)
+        rules["vr_keyboard"] = mode
+        self._save_rules(rules)
+        self.message.emit(f"Keyboard: {VR_KEYBOARD_MODES[mode].lower()}", False)
+
    # --- controllers ---
    @Property("QVariantList", constant=True)
    def controllerActions(self):
@@ -19,6 +19,7 @@ Kirigami.ApplicationWindow {
            Kirigami.Action { text: "Devices"; icon.name: "input-mouse"; onTriggered: root.show(devicesPage) },
            Kirigami.Action { text: "Buttons"; icon.name: "input-keyboard"; onTriggered: root.show(buttonsPage) },
            Kirigami.Action { text: "Controllers"; icon.name: "input-gamepad"; onTriggered: root.show(controllersPage) },
+            Kirigami.Action { text: "Keyboard"; icon.name: "input-keyboard-virtual"; onTriggered: root.show(keyboardPage) },
            Kirigami.Action { text: "Pointer"; icon.name: "transform-move"; onTriggered: root.show(pointerPage) },
            Kirigami.Action { text: "Ignored panels"; icon.name: "view-hidden"; onTriggered: root.show(ignorePage) },
            Kirigami.Action { text: "Gaze"; icon.name: "view-visible"; onTriggered: root.show(gazePage) },
@@ -55,9 +56,10 @@ Kirigami.ApplicationWindow {
        pageStack.push(page)
    }

-    // FT_INPUT_PAGE=buttons|controllers|pointer|ignore|gaze|bluetooth opens the app on that page.
-    pageStack.initialPage: ({ buttons: buttonsPage, controllers: controllersPage, pointer: pointerPage,
-                              ignore: ignorePage, gaze: gazePage, bluetooth: bluetoothPage })[startPage] || devicesPage
+    // FT_INPUT_PAGE=buttons|controllers|keyboard|pointer|ignore|gaze|bluetooth opens the app on that page.
+    pageStack.initialPage: ({ buttons: buttonsPage, controllers: controllersPage, keyboard: keyboardPage,
+                              pointer: pointerPage, ignore: ignorePage, gaze: gazePage,
+                              bluetooth: bluetoothPage })[startPage] || devicesPage

    Connections {
        target: backend
@@ -505,6 +507,54 @@ Kirigami.ApplicationWindow {
        }
    }

+    // ---------------------------------------------------------------- Keyboard
+    Component {
+        id: keyboardPage
+        Kirigami.ScrollablePage {
+            id: kpage
+            title: "Keyboard"
+            // Pass-through keyboards connected now: with "no_keyboard", the keyboard waits for none.
+            // A program's uinput keyboard (frame-voice's, say) doesn't count.
+            property var keyboards: backend.devices.filter(d => d.connected && d.role === "passthrough"
+                                                                && d.kinds.indexOf("keyboard") >= 0 && !d.uinput)
+
+            Kirigami.FormLayout {
+                Controls.ComboBox {
+                    Kirigami.FormData.label: "Show the keyboard:"
+                    model: backend.vrKeyboardModes
+                    textRole: "text"
+                    valueRole: "value"
+                    Component.onCompleted: currentIndex = indexOfValue(backend.vrKeyboard)
+                    onActivated: backend.setVrKeyboard(currentValue)
+                }
+                Controls.Switch {
+                    Kirigami.FormData.label: "Keep it open:"
+                    text: "Until you press its Close key or your keyboard button, not only while the text field has focus"
+                    checked: backend.vrKeyboardPersist
+                    enabled: backend.vrKeyboard !== "never"
+                    onToggled: backend.setVrKeyboardPersist(checked)
+                }
+                Controls.Label {
+                    Kirigami.FormData.label: "Keyboards connected:"
+                    text: kpage.keyboards.length ? kpage.keyboards.map(d => d.name).join(", ") : "none"
+                }
+            }
+
+            footer: Controls.Label {
+                padding: Kirigami.Units.largeSpacing
+                wrapMode: Text.Wrap
+                opacity: 0.7
+                text: "Frametop's keyboard opens in front of you, below your eyes, and closes with its Close key or a layout reset "
+                      + "(or when the text field loses focus, with Keep it open off). It steps aside while the Steam "
+                      + "menu or Steam's own keyboard is up. Type on it with a laser or the 3D mouse. It types into any "
+                      + "app, but only Qt, GTK and Firefox apps say when a text field is selected; for the rest "
+                      + "(Chromium, Electron and X11 apps), map Open/close keyboard to a button on the Buttons or "
+                      + "Controllers page. Keyboards set to Ignore, and keyboards other programs make (like "
+                      + "frame-voice's), don't count as connected."
+            }
+        }
+    }
+
    // ---------------------------------------------------------------- Pointer
    Component {
        id: pointerPage
@@ -0,0 +1,92 @@
+#!/usr/bin/env python3
+"""ft-textinput: Frametop's input method, which tells the input relay when a text field
+on the desktop has keyboard focus.
+
+KWin starts it (--inputmethod, from session/frametop-session.sh) and activates it
+whenever the focused app enables text input on a field (text-input v1/v2/v3: Qt, GTK,
+and Firefox apps do; Xwayland apps and most Electron apps don't). This only passes that
+on: "textfield 1" or "textfield 0" to the input relay (@frametop_relay), which opens or
+closes Frametop's keyboard (ft-screens' own key panel), depending on the keyboard setting
+in Frametop Input Settings. Typing on it reaches the app as key presses from ft-screens,
+not through this input method, so nothing typed passes through here.
+
+Speaks the Wayland wire protocol itself (zwp_input_method_v1 only), so it needs nothing
+but Python on the SteamOS host.
+"""
+import os
+import socket
+import struct
+import sys
+
+RELAY = "\0frametop_relay"
+DISPLAY, REGISTRY, INPUT_METHOD = 1, 2, 3  # our object ids
+
+
+def string(text):
+    data = text.encode() + b"\0"
+    return struct.pack("=I", len(data)) + data + b"\0" * (-len(data) % 4)
+
+
+def read_string(body, at):
+    """(text, offset after it) for a string argument at `at`."""
+    (length,) = struct.unpack_from("=I", body, at)
+    text = body[at + 4:at + 4 + length - 1].decode(errors="replace")
+    return text, at + 4 + length + (-length % 4)
+
+
+def main():
+    fd = os.environ.pop("WAYLAND_SOCKET", None)
+    if fd is None:
+        sys.exit("ft-textinput: no WAYLAND_SOCKET (KWin starts this as its input method)")
+    conn = socket.socket(fileno=int(fd))
+    relay = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM | socket.SOCK_NONBLOCK)
+
+    def request(obj, opcode, payload=b""):
+        conn.sendall(struct.pack("=II", obj, (8 + len(payload)) << 16 | opcode) + payload)
+
+    def report(on):
+        try:
+            relay.sendto(f"textfield {int(on)}".encode(), RELAY)
+        except OSError:
+            pass  # the relay isn't running
+
+    request(DISPLAY, 1, struct.pack("=I", REGISTRY))  # wl_display.get_registry
+    contexts = set()  # KWin makes one per activation
+    buf = b""
+    while True:
+        data = conn.recv(65536)
+        if not data:
+            return  # KWin went away
+        buf += data
+        while len(buf) >= 8:
+            obj, word = struct.unpack_from("=II", buf)
+            size, opcode = word >> 16, word & 0xFFFF
+            if size < 8 or len(buf) < size:
+                break
+            body, buf = buf[8:size], buf[size:]
+            if obj == DISPLAY and opcode == 0:  # error
+                bad, code = struct.unpack_from("=II", body)
+                message, _ = read_string(body, 8)
+                sys.exit(f"ft-textinput: Wayland error {code} on object {bad}: {message}")
+            elif obj == REGISTRY and opcode == 0:  # global
+                (name,) = struct.unpack_from("=I", body)
+                interface, _ = read_string(body, 4)
+                if interface == "zwp_input_method_v1":
+                    request(REGISTRY, 0, struct.pack("=I", name) + string(interface) + struct.pack("=II", 1, INPUT_METHOD))
+            elif obj == INPUT_METHOD and opcode == 0:  # activate: a text field has focus
+                contexts.add(struct.unpack_from("=I", body)[0])
+                report(True)
+            elif obj == INPUT_METHOD and opcode == 1:  # deactivate: it lost focus
+                (context,) = struct.unpack_from("=I", body)
+                if context in contexts:
+                    contexts.discard(context)
+                    request(context, 0)  # zwp_input_method_context_v1.destroy
+                report(bool(contexts))
+            # Everything else (the contexts' surrounding text, content type, ...) is unused.
+
+
+if __name__ == "__main__":
+    try:
+        main()
+    except KeyboardInterrupt:
+        pass
@@ -21,8 +21,8 @@ Buttons and keys of pointer devices go through a per-device map to actions
 (left, right, middle, back, scroll_up, scroll_down, dashboard, recenter,
 pointer_toggle, follow_toggle = head follow on or off, gaze_toggle = gaze mode on or off
 (the pointer goes where you look; see pointer/helper/ft-pointer.cpp), sens_up, sens_down,
-layout_reset = put the desktop screens back in their saved layout, screens_toggle = hide or show the desktop screens, key = pass
-through as a key, none).
+layout_reset = put the desktop screens back in their saved layout, screens_toggle = hide or show the desktop screens,
+keyboard_toggle = open or close Frametop's keyboard, key = pass through as a key, none).

 Frame controller buttons can be mapped too ("controller_buttons": {"right/a": action} in the
 rules file; any action but key). The controllers aren't input devices here, only SteamVR sees
@@ -47,6 +47,16 @@ a grabbed keyboard's keys also go out as "key <code> <value> <device name>" data
 It's off by default: any local process that binds that name first gets every key typed
 into the desktop.

+Frametop's keyboard (ft-screens' key panel): the desktop's input method (input/ft-textinput)
+says "textfield 1|0" when a text field on the desktop gains or loses keyboard focus, and
+the relay tells ft-screens to open ("vrkeyboard show") or close ("vrkeyboard hide") the
+keyboard, depending on "vr_keyboard" in the rules file: "always", "no_keyboard" (the
+default: only while no pass-through keyboard is connected; a program's uinput keyboard
+doesn't count), "button" (only the keyboard_toggle action opens it), or "never"
+(keyboard_toggle does nothing either). With "vr_keyboard_persist" (the default), it stays
+open when the text field loses focus, until its Close key, keyboard_toggle, or a layout reset
+(ft-layout apply) closes it.
+
 Volume keys, from every device that has them (the headset's own buttons included),
 are handled here: wpctl steps the default output. Nothing else may see a volume key,
 because gamescope aborts on one when no window has keyboard focus, which ends the
@@ -67,6 +77,7 @@ Control socket (abstract datagram @frametop_relay, JSON replies to the sender):
  vrcapture <s>     take every controller button for s seconds (0: stop), so the settings
                    app can capture one; watchers see them as events with id frame_controller
  vrbtn, vrhello, gazeawake   from the pointer helper (above)
+  textfield 1|0     from the desktop's input method (above)

 Runs on the Frame host as a user service (frametop-input-relay.service). The
 virtual devices are parked in systemd's file descriptor store, so a relay
@@ -156,7 +167,8 @@ VIRTUAL_PREFIX = "frametop virtual"
 RULES_PATH = os.path.expanduser("~/.config/frametop-input.json")
 ACTIONS = ("left", "right", "middle", "back", "scroll_up", "scroll_down", "dashboard", "recenter",
           "pointer_toggle", "follow_toggle", "gaze_toggle", "sens_up", "sens_down", "layout_reset", "screens_toggle",
-           "key", "none")
+           "keyboard_toggle", "key", "none")
+VR_KEYBOARD_MODES = ("always", "no_keyboard", "button", "never")  # when Frametop's keyboard opens
 HELPER = "\0ft_pointer_helper"
 # Frame controller buttons the pointer helper can read (pointer/helper/vrbuttons.h).
 VR_BUTTONS = ("left/view", "left/dpad_up", "left/dpad_down", "left/dpad_left", "left/dpad_right", "left/bumper",
@@ -260,7 +272,8 @@ def read_config(path=os.path.expanduser("~/.config/frametop.conf")):

 def read_rules(path=RULES_PATH):
    """{"devices": {id: {"role", "name"}}, "buttons": {id: {"<code>": action}},
-    "controller_buttons": {"<hand>/<button>": action}}."""
+    "controller_buttons": {"<hand>/<button>": action}, "controller_in_games": bool,
+    "vr_keyboard": one of VR_KEYBOARD_MODES, "vr_keyboard_persist": bool}."""
    try:
        with open(path) as f:
            rules = json.load(f)
@@ -502,8 +515,9 @@ class Node:
    or any other device with volume keys (candidate False, role "volume")."""

    def __init__(self, path, fd, name, bus, vendor, product, uniq, is_mouse, is_keyboard,
-                 candidate=True, volume_keys=False, only_volume=False):
+                 candidate=True, volume_keys=False, only_volume=False, uinput=False):
        self.path, self.fd, self.name = path, fd, name
+        self.uinput = uinput  # made by a program (frame-voice's keyboard, say), not a real device
        self.bus, self.vendor, self.product, self.uniq = bus, vendor, product, uniq
        self.is_mouse, self.is_keyboard = is_mouse, is_keyboard
        self.candidate, self.volume_keys, self.only_volume = candidate, volume_keys, only_volume
@@ -520,7 +534,7 @@ class Node:
        kinds = [k for k, on in (("mouse", self.is_mouse), ("keyboard", self.is_keyboard)) if on]
        return {"path": self.path, "name": self.name, "id": self.id, "uniq": self.uniq,
                "bus": {BUS_USB: "usb", BUS_BLUETOOTH: "bluetooth"}.get(self.bus, str(self.bus)),
-                "kinds": kinds, "role": self.role, "grabbed": self.grabbed}
+                "kinds": kinds, "role": self.role, "grabbed": self.grabbed, "uinput": self.uinput}


 def probe(path):
@@ -552,8 +566,10 @@ def probe(path):
        volume_keys = bool(keys & VOLUME_CODES)  # stand-ins too: kept from before a relay restart
        if not (candidate or volume_keys):
            raise ValueError
+        # uinput devices live here (Bluetooth LE ones come through uhid, under virtual/misc).
+        sysfs = os.path.realpath(f"/sys/class/input/{os.path.basename(path)}/device")
        return Node(path, fd, name, bus, vendor, product, uniq, is_mouse, is_keyboard,
-                    candidate, volume_keys, keys <= VOLUME_CODES)
+                    candidate, volume_keys, keys <= VOLUME_CODES, sysfs.startswith("/sys/devices/virtual/input/"))
    except (OSError, ValueError):
        os.close(fd)
        return None
@@ -624,6 +640,35 @@ def main():
                             "type": "vr", "code": button, "value": value})
        action = state["rules"]["controller_buttons"].get(button)
        if state["pointer"] and action in ACTIONS and action not in ("key", "none"):
+            do_action(action, value, now)
+
+    def vr_keyboard_mode():
+        mode = state["rules"].get("vr_keyboard")
+        return mode if mode in VR_KEYBOARD_MODES else "no_keyboard"
+
+    def vr_keyboard(command):
+        """Open or close Frametop's keyboard (ft-screens)."""
+        try:
+            screens_sock.sendto(f"vrkeyboard {command}".encode(), SCREENS)
+        except OSError:
+            pass  # ft-screens not running
+
+    def text_field(focused):
+        """A text field on the desktop gained or lost keyboard focus (the input method)."""
+        mode = vr_keyboard_mode()
+        if not focused:
+            if not state["rules"].get("vr_keyboard_persist", True):
+                vr_keyboard("hide")  # ft-screens closes it only if it opened it for a text field
+        elif mode == "always" or (mode == "no_keyboard" and not any(
+                n.candidate and n.is_keyboard and n.role == "passthrough" and not n.uinput for n in nodes.values())):
+            vr_keyboard("show")
+
+    def do_action(action, value, now):
+        """A mapped mouse or controller button (pointer mode only)."""
+        if action == "keyboard_toggle":
+            if value == 1 and vr_keyboard_mode() != "never":
+                vr_keyboard("toggle")
+        else:
            state["pointer"].action(action, value, now)


@@ -789,6 +834,9 @@ def main():
            if cmd == "vrhello":
                vr_bind(now)
                continue
+            if cmd == "textfield" and len(words) == 2:
+                text_field(words[1] == "1")
+                continue
            if cmd == "gazeawake" and len(words) == 2:
                if state["pointer"]:
                    state["pointer"].gaze_awake_until = now + 12.0 if words[1] == "1" else 0.0
@@ -930,7 +978,7 @@ def main():
                if etype == EV_KEY:
                    action = buttons.get(str(code), DEFAULT_BUTTONS.get(code, "key"))
                    if pointer and action not in ("key", "none"):
-                        pointer.action(action, value, now)
+                        do_action(action, value, now)
                        continue
                    if action == "none":
                        continue
@@ -440,6 +440,10 @@ def apply_screens(wait=0):
                sock.ask(f"pin {i + 1} {pin['hand']} " + " ".join(f"{v:.5f}" for v in pin["rel"]))
            except RuntimeError as e:
                log(f"screen {i + 1}: {e}")  # that controller isn't on
+    try:
+        sock.ask("vrkeyboard close")  # the keyboard, if open, goes too: a reset starts over
+    except RuntimeError:
+        pass  # an older ft-screens
    log(f"arranged {count} screen(s)")
    return results

@@ -1254,10 +1254,10 @@ int main() {
                overlay->GetOverlayTextureSize(h, &tw, &th);
                vr::VROverlayTransformType tt = vr::VROverlayTransform_Invalid;
                overlay->GetOverlayTransformType(h, &tt);
-                // ft-screens' panels (frametop.screen.N) are 0x0 and absolute too (a shared
-                // texture), but they're real panels of any size.
+                // ft-screens' panels (frametop.screen.N, the keyboard) are 0x0 and absolute too
+                // (a shared texture), but they're real panels of any size.
                sceneGraph[key] = (tw == 0 || th == 0) && tt == vr::VROverlayTransform_Absolute &&
-                                  key.rfind("frametop.screen.", 0) != 0;
+                                  key.rfind("frametop.", 0) != 0;
            }
            ours = vr::k_unTrackedDeviceIndexInvalid;
            for (vr::TrackedDeviceIndex_t i = 0; i < vr::k_unMaxTrackedDeviceCount; ++i) {
@@ -5,20 +5,24 @@
 # vr.cpp needs OpenVR's IVRIPCResourceManagerClient (ImportDmabuf), which the header
 # shipped with SteamVR on the Frame predates, so the build uses the public header from
 # Valve's openvr repo (pinned; the Frame's runtime supports its interface versions).
+# keyboard.cpp draws its key labels with stb_truetype (public domain, one header, pinned).
 set -euo pipefail
 root=$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)
 "$root/scripts/sync.sh" >/dev/null
 exec "$root/scripts/frame.sh" -C screens 'set -e; mkdir -p build/include
 openvr=v2.15.6
 [ -f build/include/openvr-$openvr ] || { curl -fsSL "https://raw.githubusercontent.com/ValveSoftware/openvr/$openvr/headers/openvr.h" -o build/include/openvr.h && touch build/include/openvr-$openvr; }
+stb=2c980bb59875b0d32144a71867fbdebb2f77cd20
+[ -f build/include/stb-$stb ] || { curl -fsSL "https://raw.githubusercontent.com/nothings/stb/$stb/stb_truetype.h" -o build/include/stb_truetype.h && touch build/include/stb-$stb; }
 gcc -std=c11 -O2 -Wall -Wno-unused-parameter -c -o build/compositor.o compositor.c \
  $(pkg-config --cflags wlroots-0.20 wayland-server xkbcommon libdrm pixman-1)
 cxx="g++ -std=c++17 -O2 -Wall -Wno-missing-field-initializers -Ibuild/include $(pkg-config --cflags egl glesv2 gbm libdrm)"
 $cxx -c -o build/vr.o vr.cpp
+$cxx -c -o build/keyboard.o keyboard.cpp
 $cxx -c -o build/handcut.o handcut.cpp
 $cxx -c -o build/handtest.o handtest.cpp
 vrlibs="$(pkg-config --libs egl glesv2 gbm) -L/opt/steamvr/bin/linuxarm64 -lopenvr_api -Wl,-rpath,/opt/steamvr/bin/linuxarm64"
-g++ -o build/ft-screens build/compositor.o build/vr.o build/handcut.o \
+g++ -o build/ft-screens build/compositor.o build/vr.o build/keyboard.o build/handcut.o \
  $(pkg-config --libs wlroots-0.20 wayland-server xkbcommon) $vrlibs
 g++ -o build/ft-handtest build/handtest.o build/handcut.o $vrlibs
 echo "built build/ft-screens build/ft-handtest"'
@@ -102,6 +102,10 @@ struct server {
    int relay_fd;         // unbound, so the relay can't reply into our control socket
    uint32_t relay_sent;  // when the relay last heard from us (ms)
    unsigned ticks;
+    // Our keyboard ("vrkeyboard" commands from the input relay; see keyboard_command).
+    int kb_screen;         // the screen it's open for, -1 closed
+    bool kb_auto;          // opened for a focused text field (not by a button)
+    unsigned kb_close_at;  // ticks: close it then (a text field lost focus), 0 not
 };

 static uint32_t now_ms(void) {
@@ -237,12 +241,24 @@ static void new_decoration(struct wl_listener *l, void *data) {

 // ---------------------------------------------------------------- input from the panels

+static void panel_key(struct server *s, uint32_t code, bool pressed);
+
 static void handle_vr_event(const struct ft_event *e, void *data) {
    struct server *s = data;
    if (e->type == FT_QUIT) {
        wl_display_terminate(s->display);
        return;
    }
+    if (e->type == FT_KEY) {
+        panel_key(s, e->key, e->pressed);
+        return;
+    }
+    if (e->type == FT_KEYBOARD_CLOSED) {
+        if (s->kb_screen >= 0) wlr_log(WLR_INFO, "keyboard closed");
+        s->kb_screen = -1;
+        s->kb_close_at = 0;
+        return;
+    }
    if (e->screen < 0 || e->screen >= MAX_SCREENS || !s->screens[e->screen]) return;
    struct screen *sc = s->screens[e->screen];
    struct wlr_surface *surface = sc->toplevel->base->surface;
@@ -313,6 +329,13 @@ static int tick(void *data) {
    struct server *s = data;
    ft_vr_poll(handle_vr_event, s);
    if (++s->ticks % 9 == 0) keys_update(s);
+    if (s->kb_close_at && s->ticks >= s->kb_close_at) {
+        s->kb_close_at = 0;
+        if (s->kb_screen >= 0) {
+            ft_vr_keyboard_hide();
+            s->kb_screen = -1;
+        }
+    }
    struct timespec now;
    clock_gettime(CLOCK_MONOTONIC, &now);
    for (int i = 0; i < MAX_SCREENS; ++i) {
@@ -344,6 +367,17 @@ static bool key_held(const struct wlr_keyboard *kb, uint32_t code) {
    return false;
 }

+// One key to the focused screen, through the seat's keyboard so its xkb state and
+// modifiers stay right.
+static void send_key(struct server *s, uint32_t code, int pressed) {
+    struct wlr_keyboard_key_event ev = {
+        .time_msec = now_ms(), .keycode = code, .update_state = true,
+        .state = pressed ? WL_KEYBOARD_KEY_STATE_PRESSED : WL_KEYBOARD_KEY_STATE_RELEASED};
+    wlr_keyboard_notify_key(&s->keyboard, &ev);
+    wlr_seat_keyboard_notify_modifiers(s->seat, &s->keyboard.modifiers);
+    wlr_seat_keyboard_notify_key(s->seat, ev.time_msec, code, ev.state);
+}
+
 // Keys from the input relay (physical keyboards): "key <evdev code> <1 press|0 release>".
 // They go to the screen KWin has keyboard focus on (the last one clicked), while typing
 // goes to the desktop (keys_update). The release of a key the desktop got the press for
@@ -355,20 +389,67 @@ static void handle_key(struct server *s, uint32_t code, int value, char *reply,
        if (!s->seat->keyboard_state.focused_surface) return (void)snprintf(reply, size, "ok no focus");
        if (!s->keys_desktop) return (void)snprintf(reply, size, "ok typing goes to Steam");
    }
-    struct wlr_keyboard_key_event ev = {
-        .time_msec = now_ms(), .keycode = code, .update_state = true,
-        .state = value ? WL_KEYBOARD_KEY_STATE_PRESSED : WL_KEYBOARD_KEY_STATE_RELEASED};
-    wlr_keyboard_notify_key(&s->keyboard, &ev);  // keeps the xkb state and modifiers
-    wlr_seat_keyboard_notify_modifiers(s->seat, &s->keyboard.modifiers);
-    wlr_seat_keyboard_notify_key(s->seat, ev.time_msec, code, ev.state);
+    send_key(s, code, value);
    snprintf(reply, size, "ok");
 }

+// Our keyboard (keyboard.cpp). The input relay sends "vrkeyboard show" when a text field
+// on the desktop gets focus (and its setting allows), "vrkeyboard hide" when it loses it,
+// and "vrkeyboard toggle" for a mapped button; ft-layout sends "vrkeyboard close" when it
+// resets the layout. It opens for the screen KWin has keyboard focus on. A hide closes only a keyboard a text field opened, and waits a moment, so
+// moving between text fields doesn't close and reopen it.
+static int focused_screen(struct server *s) {
+    struct wlr_surface *focus = s->seat->keyboard_state.focused_surface;
+    for (int i = 0; i < MAX_SCREENS; ++i)
+        if (s->screens[i] && s->screens[i]->toplevel->base->surface == focus) return i;
+    if (s->pointer_focus) return s->pointer_focus->index;
+    for (int i = 0; i < MAX_SCREENS; ++i)
+        if (s->screens[i]) return i;
+    return -1;
+}
+
+static void keyboard_command(struct server *s, const char *what, char *reply, int size) {
+    const bool open = s->kb_screen >= 0;
+    if (strcmp(what, "hide") == 0) {
+        if (open && s->kb_auto && !s->kb_close_at) s->kb_close_at = s->ticks + 30;  // ~1/3 s
+        return (void)snprintf(reply, size, "ok");
+    }
+    if (strcmp(what, "close") == 0) {  // however it opened, now (ft-layout apply: a reset)
+        if (open) wlr_log(WLR_INFO, "keyboard closed (reset)");
+        ft_vr_keyboard_hide();
+        s->kb_screen = -1;
+        s->kb_close_at = 0;
+        return (void)snprintf(reply, size, "ok");
+    }
+    const bool toggle = strcmp(what, "toggle") == 0;
+    if (!toggle && strcmp(what, "show") != 0) return (void)snprintf(reply, size, "error show|hide|toggle|close");
+    s->kb_close_at = 0;
+    if (open && toggle) {
+        ft_vr_keyboard_hide();
+        s->kb_screen = -1;
+        return (void)snprintf(reply, size, "ok closed");
+    }
+    if (open) return (void)snprintf(reply, size, "ok open");
+    if (!ft_vr_screens_shown()) return (void)snprintf(reply, size, "ok screens hidden");
+    const int screen = focused_screen(s);
+    if (screen < 0 || !ft_vr_keyboard_show(screen)) return (void)snprintf(reply, size, "error not shown");
+    wlr_log(WLR_INFO, "keyboard open for screen %d (%s)", screen + 1, toggle ? "button" : "text field");
+    s->kb_screen = screen;
+    s->kb_auto = !toggle;
+    snprintf(reply, size, "ok opened");
+}
+
+// A key from our keyboard, for the focused screen. Its release always goes through, so
+// no key stays held.
+static void panel_key(struct server *s, uint32_t code, bool pressed) {
+    if (pressed ? s->seat->keyboard_state.focused_surface != NULL : key_held(&s->keyboard, code))
+        send_key(s, code, pressed);
+}
 // Control socket: abstract datagram @ft_screens. Here: "size <screen> <w> <h>" (a new
 // resolution, live), "scale <screen> <s>" (KWin's scale for it, from ft-layout),
-// "key <code> <value>", and "click <overlay key>" (from the pointer
-// helper: a mouse click landed on that panel, "-" for none); the rest is in vr.cpp
-// (ft_vr_command).
+// "key <code> <value>", "vrkeyboard show|hide|toggle|close" (our keyboard, from the input
+// relay), and "click <overlay key>" (from the pointer helper: a mouse click landed on
+// that panel, "-" for none); the rest is in vr.cpp (ft_vr_command).
 static int control_readable(int fd, uint32_t mask, void *data) {
    struct server *s = data;
    char buf[512], reply[2048];
@@ -402,6 +483,8 @@ static int control_readable(int fd, uint32_t mask, void *data) {
            handle_key(s, code, value, reply, sizeof reply);
            len = sizeof from;
            continue;  // no reply: keys are fire-and-forget
+        } else if (strncmp(buf, "vrkeyboard ", 11) == 0) {
+            keyboard_command(s, buf + 11, reply, sizeof reply);
        } else if (strncmp(buf, "click ", 6) == 0) {
            // A click on another panel takes typing to Steam. Our own panels (the screens,
            // their controls) leave it: a click on a screen arrives as a panel event.
@@ -467,6 +550,7 @@ static bool setup_dmabuf(struct server *s) {
 int main(int argc, char **argv) {
    struct server s = {0};
    for (int i = 0; i < MAX_SCREENS; ++i) s.scale[i] = 1;
+    s.kb_screen = -1;
    const char *socket_name = "ft-screens-0";
    char **command = NULL;
    for (int i = 1; i < argc; ++i) {
@@ -0,0 +1,544 @@
+// Frametop's keyboard: the panel ft-screens shows for a text field on the desktop (see
+// ft_vr_keyboard_show in vr.cpp). A US laptop layout in five rows, with Esc where Caps Lock
+// would be and a Close key by the arrows.
+//   - The keys send linux key codes, which ft-screens types into the focused screen through
+//     its seat, so they mean what they'd mean on a real keyboard. A key goes down on the
+//     press and up on the release, so KWin repeats a held key.
+//   - Shift, Ctrl and Alt latch: tap one and it's held for the next key; tap it again to let
+//     go. The labels follow Shift.
+//   - A grab bar along the top edge moves it, like a screen's bar: press it with any laser
+//     (or the 3D mouse) and the keyboard follows that device rigidly until the release.
+//   - The panel is drawn on the CPU: rounded keys, labels from Noto Sans (the container's,
+//     found with fc-match) rasterized with stb_truetype, and arrows drawn as triangles
+//     (Noto Sans has no arrow glyphs). It's drawn again when the key under a pointer, a
+//     pressed key, or a latch changes, into the next of three shared buffers (linear
+//     DMA-BUFs SteamVR imported once), and the panel switches to it. SetOverlayRaw, which
+//     uploads a new texture each time, left the panel blank for a moment on every change;
+//     it's only the fallback if the buffers can't be made.
+#include "keyboard.h"
+
+#define STB_TRUETYPE_IMPLEMENTATION
+#include "stb_truetype.h"
+
+#include <drm_fourcc.h>
+#include <fcntl.h>
+#include <gbm.h>
+#include <linux/input-event-codes.h>
+#include <unistd.h>
+
+#include <algorithm>
+#include <cmath>
+#include <cstdio>
+#include <cstring>
+#include <map>
+#include <string>
+#include <vector>
+
+namespace keyboard {
+namespace {
+
+constexpr int kW = 1280, kH = 470;     // texture pixels
+constexpr int kGrip = 40;              // the grab bar's band along the top
+constexpr double kWidthMetres = 0.7;   // the panel's width in the room
+constexpr int kMargin = 12, kGap = 7;  // around the keys, between keys (pixels)
+constexpr int kRows = 5, kColumns = 15;  // row width in key units
+constexpr uint32_t kHide = 0;          // the Close key's code
+
+struct KeyDef {
+    uint32_t code;
+    const char *label, *shifted;  // shifted: nullptr = the same
+    double units;                 // width
+};
+
+// One row after another, 15 units each.
+const std::vector<std::vector<KeyDef>> kLayout = {
+    {{KEY_GRAVE, "`", "~", 1}, {KEY_1, "1", "!", 1}, {KEY_2, "2", "@", 1}, {KEY_3, "3", "#", 1},
+     {KEY_4, "4", "$", 1}, {KEY_5, "5", "%", 1}, {KEY_6, "6", "^", 1}, {KEY_7, "7", "&", 1},
+     {KEY_8, "8", "*", 1}, {KEY_9, "9", "(", 1}, {KEY_0, "0", ")", 1}, {KEY_MINUS, "-", "_", 1},
+     {KEY_EQUAL, "=", "+", 1}, {KEY_BACKSPACE, "Backspace", nullptr, 2}},
+    {{KEY_TAB, "Tab", nullptr, 1.5}, {KEY_Q, "q", "Q", 1}, {KEY_W, "w", "W", 1}, {KEY_E, "e", "E", 1},
+     {KEY_R, "r", "R", 1}, {KEY_T, "t", "T", 1}, {KEY_Y, "y", "Y", 1}, {KEY_U, "u", "U", 1},
+     {KEY_I, "i", "I", 1}, {KEY_O, "o", "O", 1}, {KEY_P, "p", "P", 1}, {KEY_LEFTBRACE, "[", "{", 1},
+     {KEY_RIGHTBRACE, "]", "}", 1}, {KEY_BACKSLASH, "\\", "|", 1.5}},
+    {{KEY_ESC, "Esc", nullptr, 1.75}, {KEY_A, "a", "A", 1}, {KEY_S, "s", "S", 1}, {KEY_D, "d", "D", 1},
+     {KEY_F, "f", "F", 1}, {KEY_G, "g", "G", 1}, {KEY_H, "h", "H", 1}, {KEY_J, "j", "J", 1},
+     {KEY_K, "k", "K", 1}, {KEY_L, "l", "L", 1}, {KEY_SEMICOLON, ";", ":", 1},
+     {KEY_APOSTROPHE, "'", "\"", 1}, {KEY_ENTER, "Enter", nullptr, 2.25}},
+    {{KEY_LEFTSHIFT, "Shift", nullptr, 1.25}, {KEY_Z, "z", "Z", 1}, {KEY_X, "x", "X", 1},
+     {KEY_C, "c", "C", 1}, {KEY_V, "v", "V", 1}, {KEY_B, "b", "B", 1}, {KEY_N, "n", "N", 1},
+     {KEY_M, "m", "M", 1}, {KEY_COMMA, ",", "<", 1}, {KEY_DOT, ".", ">", 1}, {KEY_SLASH, "/", "?", 1},
+     {KEY_RIGHTSHIFT, "Shift", nullptr, 1.75}, {KEY_UP, "", nullptr, 1}, {KEY_DELETE, "Del", nullptr, 1}},
+    {{KEY_LEFTCTRL, "Ctrl", nullptr, 1.5}, {KEY_LEFTALT, "Alt", nullptr, 1.5}, {KEY_SPACE, "", nullptr, 7},
+     {kHide, "\u00d7  Close", nullptr, 2}, {KEY_LEFT, "", nullptr, 1}, {KEY_DOWN, "", nullptr, 1},
+     {KEY_RIGHT, "", nullptr, 1}},
+};
+
+struct Key {
+    KeyDef def;
+    int x0, y0, x1, y1;  // texture pixels, top left origin
+};
+
+enum Mod { Shift, Ctrl, Alt, kMods };
+const uint32_t kModCode[kMods] = {KEY_LEFTSHIFT, KEY_LEFTCTRL, KEY_LEFTALT};
+
+int ModOf(uint32_t code) {
+    switch (code) {
+        case KEY_LEFTSHIFT:
+        case KEY_RIGHTSHIFT: return Shift;
+        case KEY_LEFTCTRL: return Ctrl;
+        case KEY_LEFTALT: return Alt;
+        default: return -1;
+    }
+}
+
+vr::VROverlayHandle_t g_overlay = vr::k_ulOverlayHandleInvalid;
+bool g_shown = false;
+std::vector<Key> g_keys;
+std::vector<uint8_t> g_px;
+int g_hover = -1;              // key under a pointer
+bool g_hoverGrip = false;      // the grab bar is under a pointer
+vr::TrackedDeviceIndex_t g_dragDevice = vr::k_unTrackedDeviceIndexInvalid;  // carrying it
+vr::HmdMatrix34_t g_dragRel{};   // device -> panel, while carried
+vr::HmdMatrix34_t g_pose{};      // where it is
+int g_down = -1;               // key held down
+bool g_latched[kMods] = {};    // Shift, Ctrl, Alt held for the next key
+bool g_dirty = true;           // draw again
+
+// ---------------------------------------------------------------- labels
+
+stbtt_fontinfo g_font;
+std::vector<unsigned char> g_fontData;
+bool g_fontOk = false;
+
+struct Glyph {
+    std::vector<unsigned char> bitmap;
+    int w = 0, h = 0, xoff = 0, yoff = 0, advance = 0;
+};
+std::map<std::pair<uint32_t, int>, Glyph> g_glyphs;  // (codepoint, pixel size)
+
+void LoadFont() {
+    std::string path;
+    if (FILE *p = popen("fc-match -f '%{file}' 'Noto Sans' 2>/dev/null", "r")) {
+        char buf[512];
+        if (std::fgets(buf, sizeof buf, p)) path = buf;
+        pclose(p);
+    }
+    if (path.empty()) path = "/usr/share/fonts/google-noto-vf/NotoSans[wght].ttf";
+    if (FILE *f = std::fopen(path.c_str(), "rb")) {
+        std::fseek(f, 0, SEEK_END);
+        g_fontData.resize(size_t(std::ftell(f)));
+        std::fseek(f, 0, SEEK_SET);
+        g_fontOk = std::fread(g_fontData.data(), 1, g_fontData.size(), f) == g_fontData.size() &&
+                   stbtt_InitFont(&g_font, g_fontData.data(), stbtt_GetFontOffsetForIndex(g_fontData.data(), 0));
+        std::fclose(f);
+    }
+    if (!g_fontOk) std::printf("keyboard: no font (%s); the keys have no labels\n", path.c_str());
+}
+
+const Glyph &GetGlyph(uint32_t cp, int size) {
+    auto [it, fresh] = g_glyphs.try_emplace({cp, size});
+    Glyph &g = it->second;
+    if (fresh) {
+        const float scale = stbtt_ScaleForPixelHeight(&g_font, float(size));
+        unsigned char *b = stbtt_GetCodepointBitmap(&g_font, 0, scale, int(cp), &g.w, &g.h, &g.xoff, &g.yoff);
+        if (b) g.bitmap.assign(b, b + size_t(g.w) * g.h), stbtt_FreeBitmap(b, nullptr);
+        int adv, lsb;
+        stbtt_GetCodepointHMetrics(&g_font, int(cp), &adv, &lsb);
+        g.advance = int(std::lround(adv * scale));
+    }
+    return g;
+}
+
+std::vector<uint32_t> Codepoints(const char *s) {
+    std::vector<uint32_t> out;
+    for (const unsigned char *p = (const unsigned char *)s; *p;) {
+        uint32_t c = *p++;
+        int more = c >= 0xF0 ? 3 : c >= 0xE0 ? 2 : c >= 0xC0 ? 1 : 0;
+        if (more) c &= 0x3Fu >> more;
+        for (; more && (*p & 0xC0) == 0x80; --more) c = c << 6 | (*p++ & 0x3F);
+        out.push_back(c);
+    }
+    return out;
+}
+
+// Text centred on (cx, cy), white at `alpha`.
+void DrawText(const char *text, int size, int cx, int cy, uint8_t alpha) {
+    if (!g_fontOk || !*text) return;
+    const auto cps = Codepoints(text);
+    int width = 0;
+    for (uint32_t cp : cps) width += GetGlyph(cp, size).advance;
+    int ascent, descent, gap;
+    stbtt_GetFontVMetrics(&g_font, &ascent, &descent, &gap);
+    const float scale = stbtt_ScaleForPixelHeight(&g_font, float(size));
+    int x = cx - width / 2;
+    const int baseline = cy + int(std::lround((ascent + descent) * scale / 2));
+    for (uint32_t cp : cps) {
+        const Glyph &g = GetGlyph(cp, size);
+        for (int gy = 0; gy < g.h; ++gy)
+            for (int gx = 0; gx < g.w; ++gx) {
+                const int px = x + g.xoff + gx, py = baseline + g.yoff + gy;
+                if (px < 0 || py < 0 || px >= kW || py >= kH) continue;
+                const double a = g.bitmap[size_t(gy) * g.w + gx] / 255.0 * alpha / 255.0;
+                uint8_t *p = &g_px[(size_t(py) * kW + px) * 4];
+                for (int c = 0; c < 3; ++c) p[c] = uint8_t(std::lround(p[c] + (255 - p[c]) * a));
+            }
+        x += g.advance;
+    }
+}
+
+// ---------------------------------------------------------------- drawing
+
+// A white triangle pointing along (dx, dy) (one of them 0), centred on (cx, cy), `size` across.
+void DrawArrow(int cx, int cy, int size, int dx, int dy, uint8_t alpha) {
+    const double h = size / 2.0;
+    const double pts[3][2] = {{h, 0}, {-h, -h}, {-h, h}};  // tip and base, pointing right
+    double v[3][2];
+    for (int i = 0; i < 3; ++i) {
+        v[i][0] = cx + pts[i][0] * dx - pts[i][1] * dy;
+        v[i][1] = cy + pts[i][0] * dy + pts[i][1] * dx;
+    }
+    auto inside = [&](double x, double y) {
+        bool pos = false, neg = false;
+        for (int i = 0; i < 3; ++i) {
+            const double *a = v[i], *b = v[(i + 1) % 3];
+            const double c = (b[0] - a[0]) * (y - a[1]) - (b[1] - a[1]) * (x - a[0]);
+            pos |= c > 0, neg |= c < 0;
+        }
+        return !(pos && neg);
+    };
+    for (int y = int(cy - h) - 1; y <= int(cy + h) + 1; ++y)
+        for (int x = int(cx - h) - 1; x <= int(cx + h) + 1; ++x) {
+            if (x < 0 || y < 0 || x >= kW || y >= kH) continue;
+            int n = 0;  // 4x4 samples, for smooth edges
+            for (int sy = 0; sy < 4; ++sy)
+                for (int sx = 0; sx < 4; ++sx) n += inside(x + (sx + 0.5) / 4, y + (sy + 0.5) / 4);
+            if (!n) continue;
+            const double a = n / 16.0 * alpha / 255.0;
+            uint8_t *p = &g_px[(size_t(y) * kW + x) * 4];
+            for (int c = 0; c < 3; ++c) p[c] = uint8_t(std::lround(p[c] + (255 - p[c]) * a));
+        }
+}
+
+void Layout() {
+    g_keys.clear();
+    const double unit = double(kW - 2 * kMargin) / kColumns, row = double(kH - kGrip - kMargin) / kRows;
+    for (int r = 0; r < kRows; ++r) {
+        double x = kMargin;
+        for (const KeyDef &d : kLayout[r]) {
+            Key k{d, int(std::lround(x + kGap / 2.0)), int(std::lround(kGrip + r * row + kGap / 2.0)),
+                  int(std::lround(x + d.units * unit - kGap / 2.0)),
+                  int(std::lround(kGrip + (r + 1) * row - kGap / 2.0))};
+            g_keys.push_back(k);
+            x += d.units * unit;
+        }
+    }
+}
+
+// A rounded rectangle over what's there (straight alpha, like the other panels).
+void FillRounded(int x0, int y0, int x1, int y1, double radius, const uint8_t rgba[4]) {
+    for (int y = y0; y < y1; ++y)
+        for (int x = x0; x < x1; ++x) {
+            const double cx = std::clamp(x + 0.5, x0 + radius, x1 - radius);
+            const double cy = std::clamp(y + 0.5, y0 + radius, y1 - radius);
+            const double cover = std::clamp(radius - std::hypot(x + 0.5 - cx, y + 0.5 - cy) + 0.5, 0.0, 1.0);
+            if (cover <= 0) continue;
+            uint8_t *p = &g_px[(size_t(y) * kW + x) * 4];
+            const double a = rgba[3] / 255.0 * cover, below = p[3] / 255.0;
+            const double out = a + below * (1 - a);
+            for (int c = 0; c < 3; ++c)
+                p[c] = uint8_t(std::lround(out > 0 ? (rgba[c] * a + p[c] * below * (1 - a)) / out : 0));
+            p[3] = uint8_t(std::lround(out * 255));
+        }
+}
+
+// ---------------------------------------------------------------- buffers
+
+struct Buffer {
+    gbm_bo *bo = nullptr;
+    int fd = -1;
+    vr::SharedTextureHandle_t handle = 0;
+};
+int g_drm = -1;
+gbm_device *g_gbm = nullptr;
+Buffer g_buffers[3];
+int g_next = 0;
+bool g_useBuffers = false;
+
+void DropBuffers() {
+    for (Buffer &b : g_buffers) {
+        if (b.handle) vr::VRIPCResourceManager()->UnrefResource(b.handle);
+        if (b.fd >= 0) close(b.fd);
+        if (b.bo) gbm_bo_destroy(b.bo);
+        b = Buffer{};
+    }
+    if (g_gbm) gbm_device_destroy(g_gbm);
+    if (g_drm >= 0) close(g_drm);
+    g_gbm = nullptr, g_drm = -1, g_useBuffers = false;
+}
+
+bool MakeBuffers() {
+    g_drm = open("/dev/dri/renderD128", O_RDWR | O_CLOEXEC);
+    if (g_drm >= 0) g_gbm = gbm_create_device(g_drm);
+    for (Buffer &b : g_buffers) {
+        // ABGR8888 is R, G, B, A in memory, like g_px.
+        if (g_gbm) b.bo = gbm_bo_create(g_gbm, kW, kH, GBM_FORMAT_ABGR8888, GBM_BO_USE_RENDERING | GBM_BO_USE_LINEAR);
+        if (!b.bo || (b.fd = gbm_bo_get_fd(b.bo)) < 0) break;
+        vr::DmabufAttributes_t a{};
+        a.unWidth = kW, a.unHeight = kH;
+        a.unDepth = a.unMipLevels = a.unArrayLayers = a.unSampleCount = 1;
+        a.unFormat = DRM_FORMAT_ABGR8888;
+        a.ulModifier = DRM_FORMAT_MOD_LINEAR;
+        a.unPlaneCount = 1;
+        a.plane[0].unOffset = gbm_bo_get_offset(b.bo, 0);
+        a.plane[0].unStride = gbm_bo_get_stride(b.bo);
+        a.plane[0].nFd = b.fd;
+        if (!vr::VRIPCResourceManager()->ImportDmabuf(vr::VRApplication_Overlay, &a, &b.handle)) b.handle = 0;
+        if (!b.handle) break;
+    }
+    g_useBuffers = g_buffers[2].handle != 0;
+    if (!g_useBuffers) {
+        std::printf("keyboard: no shared buffers; falling back to SetOverlayRaw (the panel flickers)\n");
+        DropBuffers();
+    }
+    return g_useBuffers;
+}
+
+// g_px to the panel: into the next buffer (premultiplied, as the flag says), then shown.
+void Present() {
+    if (!g_useBuffers) {
+        vr::VROverlay()->SetOverlayRaw(g_overlay, g_px.data(), kW, kH, 4);
+        return;
+    }
+    Buffer &b = g_buffers[g_next];
+    g_next = (g_next + 1) % 3;
+    uint32_t stride = 0;
+    void *mapping = nullptr;
+    auto *dst = static_cast<uint8_t *>(gbm_bo_map(b.bo, 0, 0, kW, kH, GBM_BO_TRANSFER_WRITE, &stride, &mapping));
+    if (!dst) return;
+    for (int y = 0; y < kH; ++y) {
+        const uint8_t *src = &g_px[size_t(y) * kW * 4];
+        uint8_t *row = dst + size_t(y) * stride;
+        for (int x = 0; x < kW * 4; x += 4) {
+            const unsigned a = src[x + 3];
+            row[x] = uint8_t(src[x] * a / 255), row[x + 1] = uint8_t(src[x + 1] * a / 255);
+            row[x + 2] = uint8_t(src[x + 2] * a / 255), row[x + 3] = uint8_t(a);
+        }
+    }
+    gbm_bo_unmap(b.bo, mapping);
+    vr::Texture_t tex = {&b.handle, vr::TextureType_SharedTextureHandle, vr::ColorSpace_Gamma};
+    vr::VROverlay()->SetOverlayTexture(g_overlay, &tex);
+}
+
+void Draw() {
+    g_px.assign(size_t(kW) * kH * 4, 0);
+    const uint8_t back[4] = {20, 22, 26, 225};
+    FillRounded(0, 0, kW, kH, 22, back);
+    const bool grip = g_hoverGrip || g_dragDevice != vr::k_unTrackedDeviceIndexInvalid;
+    const uint8_t bar[4] = {220, 222, 228, uint8_t(grip ? 235 : 120)};
+    FillRounded(kW / 2 - 130, kGrip / 2 - 5, kW / 2 + 130, kGrip / 2 + 7, 6, bar);
+    const bool shift = g_latched[Shift];
+    const int row = (kH - kGrip - kMargin) / kRows;
+    for (size_t i = 0; i < g_keys.size(); ++i) {
+        const Key &k = g_keys[i];
+        const int mod = ModOf(k.def.code);
+        const bool latched = mod >= 0 && g_latched[mod];
+        uint8_t face[4] = {58, 62, 70, 255};
+        if (latched) face[0] = 26, face[1] = 115, face[2] = 232;  // the blue the other controls use when armed
+        if (int(i) == g_down) face[0] = 26, face[1] = 115, face[2] = 232;
+        else if (int(i) == g_hover) for (int c = 0; c < 3; ++c) face[c] = uint8_t(std::min(255, face[c] + 38));
+        FillRounded(k.x0, k.y0, k.x1, k.y1, 10, face);
+        const int cx = (k.x0 + k.x1) / 2, cy = (k.y0 + k.y1) / 2;
+        switch (k.def.code) {
+            case KEY_UP: DrawArrow(cx, cy, row / 3, 0, -1, 235); continue;
+            case KEY_DOWN: DrawArrow(cx, cy, row / 3, 0, 1, 235); continue;
+            case KEY_LEFT: DrawArrow(cx, cy, row / 3, -1, 0, 235); continue;
+            case KEY_RIGHT: DrawArrow(cx, cy, row / 3, 1, 0, 235); continue;
+        }
+        const char *label = shift && k.def.shifted ? k.def.shifted : k.def.label;
+        const bool word = Codepoints(label).size() > 1;
+        DrawText(label, int(row * (word ? 0.3 : 0.46)), cx, cy, 235);
+    }
+    Present();
+    g_dirty = false;
+}
+
+int KeyAt(double x, double y) {
+    for (size_t i = 0; i < g_keys.size(); ++i) {
+        const Key &k = g_keys[i];
+        if (x >= k.x0 - kGap / 2.0 && x < k.x1 + kGap / 2.0 && y >= k.y0 - kGap / 2.0 && y < k.y1 + kGap / 2.0)
+            return int(i);
+    }
+    return -1;
+}
+
+// ---------------------------------------------------------------- carrying it
+
+vr::HmdMatrix34_t Mul(const vr::HmdMatrix34_t &a, const vr::HmdMatrix34_t &b) {
+    vr::HmdMatrix34_t r{};
+    for (int i = 0; i < 3; ++i)
+        for (int j = 0; j < 4; ++j) {
+            double v = j == 3 ? a.m[i][3] : 0;
+            for (int k = 0; k < 3; ++k) v += a.m[i][k] * b.m[k][j];
+            r.m[i][j] = float(v);
+        }
+    return r;
+}
+
+vr::HmdMatrix34_t Inverse(const vr::HmdMatrix34_t &a) {  // rigid: R^T, -R^T t
+    vr::HmdMatrix34_t r{};
+    for (int i = 0; i < 3; ++i)
+        for (int j = 0; j < 3; ++j) r.m[i][j] = a.m[j][i];
+    for (int i = 0; i < 3; ++i) r.m[i][3] = -(r.m[i][0] * a.m[0][3] + r.m[i][1] * a.m[1][3] + r.m[i][2] * a.m[2][3]);
+    return r;
+}
+
+bool DevicePose(vr::TrackedDeviceIndex_t dev, vr::HmdMatrix34_t *out) {
+    vr::TrackedDevicePose_t poses[vr::k_unMaxTrackedDeviceCount];
+    vr::VRSystem()->GetDeviceToAbsoluteTrackingPose(vr::TrackingUniverseStanding, 0, poses, vr::k_unMaxTrackedDeviceCount);
+    if (dev >= vr::k_unMaxTrackedDeviceCount || !poses[dev].bPoseIsValid) return false;
+    *out = poses[dev].mDeviceToAbsoluteTracking;
+    return true;
+}
+
+void Place(const vr::HmdMatrix34_t &pose) {
+    g_pose = pose;
+    vr::VROverlay()->SetOverlayTransformAbsolute(g_overlay, vr::TrackingUniverseStanding, &g_pose);
+}
+
+void StopCarrying() {
+    if (g_dragDevice == vr::k_unTrackedDeviceIndexInvalid) return;
+    g_dragDevice = vr::k_unTrackedDeviceIndexInvalid;
+    g_dirty = true;
+}
+
+bool Create() {
+    if (g_overlay != vr::k_ulOverlayHandleInvalid) return true;
+    if (vr::VROverlay()->CreateOverlay("frametop.keyboard", "Frametop keyboard", &g_overlay) != vr::VROverlayError_None)
+        return false;
+    LoadFont();
+    Layout();
+    vr::VROverlay()->SetOverlayWidthInMeters(g_overlay, float(kWidthMetres));
+    vr::VROverlay()->SetOverlayInputMethod(g_overlay, vr::VROverlayInputMethod_Mouse);
+    vr::HmdVector2_t scale{{float(kW), float(kH)}};
+    vr::VROverlay()->SetOverlayMouseScale(g_overlay, &scale);
+    vr::VROverlay()->SetOverlaySortOrder(g_overlay, 15);  // over the screens and their controls
+    if (MakeBuffers()) vr::VROverlay()->SetOverlayFlag(g_overlay, vr::VROverlayFlags_IsPremultiplied, true);
+    return true;
+}
+
+}  // namespace
+
+bool Show(const vr::HmdMatrix34_t &pose) {
+    if (!Create()) return false;
+    Place(pose);
+    if (!g_shown) {
+        std::fill(std::begin(g_latched), std::end(g_latched), false);
+        g_hover = g_down = -1;
+        g_hoverGrip = false;
+        g_dragDevice = vr::k_unTrackedDeviceIndexInvalid;
+        Draw();
+        vr::VROverlay()->ShowOverlay(g_overlay);
+        g_shown = true;
+    }
+    return true;
+}
+
+const vr::HmdMatrix34_t &Pose() { return g_pose; }
+
+void EndDragBy(uint32_t device) {
+    if (device == g_dragDevice) StopCarrying();
+}
+
+void Hide() {
+    StopCarrying();
+    if (!g_shown) return;
+    vr::VROverlay()->HideOverlay(g_overlay);
+    g_shown = false;
+}
+
+bool Shown() { return g_shown; }
+
+void SetLasers(bool on) {
+    if (Create()) vr::VROverlay()->SetOverlayFlag(g_overlay, vr::VROverlayFlags_MakeOverlaysInteractiveIfVisible, on);
+}
+
+void Poll(void (*handle)(const Event &, void *), void *data) {
+    if (g_overlay == vr::k_ulOverlayHandleInvalid) return;
+    auto send = [&](uint32_t code, bool pressed) { handle(Event{Event::Key, code, pressed}, data); };
+    // Let go of the key held down: it, then the latched modifiers it was typed with.
+    auto release = [&]() {
+        if (g_down < 0) return;
+        const uint32_t code = g_keys[g_down].def.code;
+        g_down = -1;
+        g_dirty = true;
+        if (code == kHide || ModOf(code) >= 0) return;
+        send(code, false);
+        for (int m = kMods - 1; m >= 0; --m)
+            if (g_latched[m]) send(kModCode[m], false), g_latched[m] = false;
+    };
+    vr::VREvent_t ev;
+    while (vr::VROverlay()->PollNextOverlayEvent(g_overlay, &ev, sizeof ev)) {
+        const double x = ev.data.mouse.x, y = kH - ev.data.mouse.y;  // OpenVR's mouse origin is bottom left
+        switch (ev.eventType) {
+            case vr::VREvent_MouseMove: {
+                const int k = KeyAt(x, y);
+                if (k != g_hover) g_hover = k, g_dirty = true;
+                if ((y < kGrip) != g_hoverGrip) g_hoverGrip = y < kGrip, g_dirty = true;
+                break;
+            }
+            case vr::VREvent_MouseButtonDown: {
+                if (ev.data.mouse.button != vr::VRMouseButton_Left) break;
+                release();
+                vr::HmdMatrix34_t dev;
+                if (y < kGrip && DevicePose(ev.trackedDeviceIndex, &dev)) {
+                    g_dragDevice = ev.trackedDeviceIndex;
+                    g_dragRel = Mul(Inverse(dev), g_pose);
+                    g_dirty = true;
+                    break;
+                }
+                const int k = KeyAt(x, y);
+                if (k < 0) break;
+                g_down = k;
+                g_dirty = true;
+                const uint32_t code = g_keys[k].def.code;
+                const int mod = ModOf(code);
+                if (code == kHide) {
+                    Hide();
+                    handle(Event{Event::Closed, 0, false}, data);
+                } else if (mod >= 0) {
+                    g_latched[mod] = !g_latched[mod];
+                } else {
+                    for (int m = 0; m < kMods; ++m)
+                        if (g_latched[m]) send(kModCode[m], true);
+                    send(code, true);
+                }
+                break;
+            }
+            case vr::VREvent_MouseButtonUp:
+                if (ev.data.mouse.button == vr::VRMouseButton_Left) release(), EndDragBy(ev.trackedDeviceIndex);
+                break;
+            case vr::VREvent_FocusLeave:
+                release();
+                if (g_hover >= 0) g_hover = -1, g_dirty = true;
+                if (g_hoverGrip) g_hoverGrip = false, g_dirty = true;
+                break;
+            default:
+                break;
+        }
+    }
+    vr::HmdMatrix34_t dev;
+    if (g_dragDevice != vr::k_unTrackedDeviceIndexInvalid) {
+        if (DevicePose(g_dragDevice, &dev)) Place(Mul(dev, g_dragRel));
+        else StopCarrying();  // the device went away
+    }
+    if (g_dirty && g_shown) Draw();
+}
+
+void Destroy() {
+    if (g_overlay != vr::k_ulOverlayHandleInvalid) vr::VROverlay()->DestroyOverlay(g_overlay);
+    DropBuffers();
+    g_overlay = vr::k_ulOverlayHandleInvalid;
+    g_shown = false;
+}
+
+}  // namespace keyboard
@@ -0,0 +1,33 @@
+// Frametop's keyboard (keyboard.cpp): a panel of keys that ft-screens shows for a text
+// field on the desktop. Any laser or the 3D mouse types on it, as on the screens.
+#pragma once
+
+#include <openvr.h>
+
+#include <cstdint>
+
+namespace keyboard {
+
+// What happened on the panel since the last Poll.
+struct Event {
+    enum Type { Key, Closed } type;
+    uint32_t code;  // Key: linux KEY_*
+    bool pressed;   // Key: down or up
+};
+
+// Show the panel at `pose` (standing universe; its front faces +z), creating it the first
+// time. False if SteamVR won't make the overlay.
+bool Show(const vr::HmdMatrix34_t &pose);
+void Hide();
+bool Shown();
+// Where it is now (it can be carried by its grab bar).
+const vr::HmdMatrix34_t &Pose();
+// A button came up on `device` somewhere else: stop carrying the panel if it was.
+void EndDragBy(uint32_t device);
+// Controllers' lasers work the panel with the dashboard closed, like the screens'.
+void SetLasers(bool on);
+// The panel's input: key presses and releases, and Closed for its Close key.
+void Poll(void (*handle)(const Event &, void *), void *data);
+void Destroy();
+
+}  // namespace keyboard
@@ -49,6 +49,7 @@
 #include "vr.h"

 #include "handcut.h"
+#include "keyboard.h"

 #include <drm_fourcc.h>
 #include <openvr.h>
@@ -931,6 +932,7 @@ void FinishDrag(Screen &s, int index) {
 // A button release on any of our panels ends that device's drags (it may be over another
 // screen by then).
 void EndDragsBy(vr::TrackedDeviceIndex_t dev) {
+    keyboard::EndDragBy(dev);
    for (auto &[index, s] : g_screens)
        if (s.drag != Drag::None && s.dragDevice == dev) FinishDrag(s, index);
 }
@@ -1142,6 +1144,37 @@ void UpdateCutouts() {
    }
 }

+// Steam in front: the dashboard (the Steam menu) is open, or Steam's own keyboard is up
+// (valve.steam.gamepadui.keyboard, for text fields in Steam and the dashboard). Our
+// keyboard steps aside then, and comes back where it was when Steam is out of the way; one
+// asked for meanwhile appears then. Checked every 9 ticks; Steam makes its keyboard's
+// overlay again now and then, so it's looked up each time.
+bool g_steamInFront = false;
+bool g_keyboardAside = false;   // ours is waiting for Steam to get out of the way
+Mat g_asidePose = Identity();   // ...and goes here then
+
+bool SteamInFront() {
+    vr::VROverlayHandle_t h = vr::k_ulOverlayHandleInvalid;
+    // In the dashboard mode the screens only show with the dashboard, so it doesn't count.
+    return (g_mode != Mode::Dashboard && vr::VROverlay()->IsDashboardVisible()) ||
+           (vr::VROverlay()->FindOverlay("valve.steam.gamepadui.keyboard", &h) == vr::VROverlayError_None &&
+            vr::VROverlay()->IsOverlayVisible(h));
+}
+
+void UpdateSteamInFront() {
+    const bool front = SteamInFront();
+    if (front == g_steamInFront) return;
+    g_steamInFront = front;
+    if (front && keyboard::Shown()) {
+        g_asidePose = keyboard::Pose();
+        keyboard::Hide();
+        g_keyboardAside = true;
+    } else if (!front && g_keyboardAside) {
+        g_keyboardAside = false;
+        keyboard::Show(g_asidePose);
+    }
+}
+
 }  // namespace

 extern "C" {
@@ -1173,6 +1206,7 @@ void ft_vr_shutdown(void) {
    for (auto &[dev, g] : g_guides)
        for (auto o : {g.ring.overlay, g.dot.overlay}) vr::VROverlay()->DestroyOverlay(o);
    for (auto &[k, h] : g_imports) vr::VRIPCResourceManager()->UnrefResource(h);
+    keyboard::Destroy();
    g_guides.clear();
    g_screens.clear();
    g_imports.clear();
@@ -1386,6 +1420,30 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) {
        // A carrying controller that goes away drops its screen.
        if (ev.eventType == vr::VREvent_TrackedDeviceDeactivated) EndDragsBy(ev.trackedDeviceIndex);
    }
+    // Our keyboard: its keys, and its Close key. It goes when the screens do.
+    struct Forward {
+        void (*handle)(const struct ft_event *, void *);
+        void *data;
+    } forward{handle, data};
+    keyboard::Poll(
+        [](const keyboard::Event &k, void *f) {
+            ft_event e{};
+            e.screen = -1;
+            e.type = k.type == keyboard::Event::Key ? FT_KEY : FT_KEYBOARD_CLOSED;
+            e.key = k.code;
+            e.pressed = k.pressed;
+            static_cast<Forward *>(f)->handle(&e, static_cast<Forward *>(f)->data);
+        },
+        &forward);
+    if (g_tick % 9 == 0) UpdateSteamInFront();
+    if ((keyboard::Shown() || g_keyboardAside) && !ModeVisible()) {
+        g_keyboardAside = false;
+        keyboard::Hide();
+        ft_event e{};
+        e.type = FT_KEYBOARD_CLOSED;
+        e.screen = -1;
+        handle(&e, data);
+    }
    ++g_tick;
    UpdateGame();
    UpdateArrange();
@@ -1396,6 +1454,41 @@ void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data) {
    UpdateCutouts();
 }

+// Our keyboard (keyboard.cpp) for a screen. It's placed where you'll reach it, not on the
+// screen: kKeyboardAhead in front of you (the way your head faces, level) and
+// kKeyboardBelow under your eyes, turned to face your eyes, and it stays where it opened
+// (or where its grab bar carries it). With Steam in front (UpdateSteamInFront), it waits.
+// Without a head pose (the headset is in standby, say) it doesn't open: anywhere else could
+// be out of sight or reach. The next text field opens it.
+constexpr double kKeyboardAhead = 0.7, kKeyboardBelow = 0.35;
+
+bool ft_vr_keyboard_show(int index) {
+    if (g_screens.find(index) == g_screens.end()) return false;
+    RefreshPoses();
+    Mat head;
+    if (!DevicePose(vr::k_unTrackedDeviceIndex_Hmd, &head)) {
+        std::printf("keyboard: no head pose, not opened\n");
+        return false;
+    }
+    const double fx = -head.m[0][2], fz = -head.m[2][2], n = std::sqrt(fx * fx + fz * fz) + 1e-9;
+    const double at[3] = {head.m[0][3] + fx / n * kKeyboardAhead, head.m[1][3] - kKeyboardBelow,
+                          head.m[2][3] + fz / n * kKeyboardAhead};
+    keyboard::SetLasers(g_lasers == Lasers::Always || (g_lasers == Lasers::OutsideGames && !g_gameRunning));
+    g_steamInFront = SteamInFront();
+    if (g_steamInFront) {
+        g_asidePose = FacingPose(at, head);
+        g_keyboardAside = true;
+        std::printf("keyboard: waiting for Steam to close\n");
+        return true;
+    }
+    return keyboard::Show(FacingPose(at, head));
+}
+
+void ft_vr_keyboard_hide(void) {
+    g_keyboardAside = false;
+    keyboard::Hide();
+}
+
 // Control commands (datagrams on @ft_screens, replies to the sender):
 //   place <screen> <x> <y> <z> <yaw> <pitch> <roll>   centre (standing universe) and facing
 //   width <screen> <metres>
@@ -17,7 +17,7 @@ struct ft_dmabuf {
    int fd[4];
 };

-enum ft_event_type { FT_MOTION, FT_BUTTON, FT_SCROLL, FT_LEAVE, FT_QUIT };
+enum ft_event_type { FT_MOTION, FT_BUTTON, FT_SCROLL, FT_LEAVE, FT_QUIT, FT_KEY, FT_KEYBOARD_CLOSED };

 struct ft_event {
    enum ft_event_type type;
@@ -26,6 +26,7 @@ struct ft_event {
    uint32_t button;  // FT_BUTTON: linux BTN_*
    bool pressed;
    double dx, dy;    // FT_SCROLL: notches (positive dy: scroll down)
+    uint32_t key;     // FT_KEY: linux KEY_* from our keyboard (pressed: down or up)
 };

 bool ft_vr_init(void);
@@ -44,6 +45,12 @@ bool ft_vr_screen_present(int index, const void *key, const struct ft_dmabuf *bu
 void ft_vr_forget(const void *key);
 // Poll panel input and SteamVR events.
 void ft_vr_poll(void (*handle)(const struct ft_event *, void *), void *data);
+// Our keyboard (keyboard.cpp), for typing on screen `index`: its keys arrive as FT_KEY
+// events, and FT_KEYBOARD_CLOSED when its Close key is pressed or the screens hide. False if
+// it can't be shown or there's no head pose. With the Steam menu or Steam's own keyboard
+// up, it waits and appears when they're gone.
+bool ft_vr_keyboard_show(int index);
+void ft_vr_keyboard_hide(void);
 // A command from the control socket (@ft_screens); writes the reply (see vr.cpp).
 void ft_vr_command(const char *command, char *reply, int reply_size);

@@ -21,11 +21,15 @@ here=$(dirname "$(readlink -f "$0")")
 # it should see the system as a normal login does, so drop the client's runtime: its
 # LD_LIBRARY_PATH put Steam's own libraries ahead of the system's (Steam's libavcodec has no
 # H.264 decoder, so VLC couldn't play most videos), and its overlay and launch settings are
-# meant for games. SteamOS's own defaults (/usr/share/deckard/mesavars.sh) stay.
+# meant for games. SteamOS's own defaults (/usr/share/deckard/mesavars.sh) stay. The
+# gamescope session also puts QT_IM_MODULE=xim and GTK_IM_MODULE=xim in the systemd user
+# environment, and with those, Qt and GTK apps never tell KWin a text field has focus, so
+# Frametop's keyboard (input/ft-textinput) never opens for them.
 for var in $(compgen -e); do
  case $var in
    LD_LIBRARY_PATH | LD_PRELOAD | STEAM_* | Steam* | SRT_* | PRESSURE_VESSEL_* | MANGOHUD_* | \
-      ENABLE_VK_LAYER_VALVE_steam_overlay_* | STEAMVIDEOTOKEN) unset "$var" ;;
+      ENABLE_VK_LAYER_VALVE_steam_overlay_* | STEAMVIDEOTOKEN | QT_IM_MODULE | GTK_IM_MODULE | \
+      XMODIFIERS) unset "$var" ;;
  esac
 done

@@ -133,9 +137,13 @@ fi

 # plasma-session starts KWin through kwin_wayland_wrapper. Shadow it to add our outputs.
 # With ft-screens the size is only the starting one: ft-screens sets each screen's own.
+# Our input method tells the input relay when a text field has focus, for SteamVR's
+# keyboard (input/ft-textinput).
+textinput=$(readlink -f "$here/../input/ft-textinput")
 cat > "$runtime/bin/kwin_wayland_wrapper" <<EOF
 #!/bin/sh
-exec /usr/bin/kwin_wayland_wrapper --width $width --height $height --output-count $screens --no-lockscreen "\$@"
+exec /usr/bin/kwin_wayland_wrapper --width $width --height $height --output-count $screens --no-lockscreen \\
+  --inputmethod $textinput "\$@"
 EOF
 chmod +x "$runtime/bin/kwin_wayland_wrapper"
 export PATH=$runtime/bin:$PATH