mirror of
https://github.com/lhns/steam-frame-nix.git
synced 2026-10-06 03:00:13 +02:00
Pierre Kisters
b2babb3859
Steam sends Tab as the text "\t" and drops Shift. With Shift, Tab now goes to the xdotool helper (shift+Tab, ISO_Left_Tab on any keymap); the suggestions text model resets on it like on Shift+arrows. The helper's allowlist moves to allowlist.mjs with a test (flake check vr-keyboard-extra-keys).
170 lines
8.1 KiB
Markdown
170 lines
8.1 KiB
Markdown
# VR keyboard
|
||
|
||
Two patches of Steam's VR keyboard: [extra keys](#extra-keys) and
|
||
[swipe and suggestions](#swipe-and-suggestions). They work together or
|
||
alone. Options: [README, Options](../README.md#options) (`keyboard.vr.*`).
|
||
The keyboard layout of the Steam session itself is a separate fix
|
||
([keyboard layout](session.md#keyboard-layout)).
|
||
|
||
## Extra keys
|
||
|
||
`keyboard.vr.extraKeys.enable`, module `vr-keyboard-extra-keys` (sources in
|
||
`modules/vr-keyboard-extra-keys/`).
|
||
|
||
|
||
|
||
**Problem:** Steam's VR keyboard has no Ctrl, Alt or Esc, can't press real
|
||
keys, and its text emulation only maps plain ASCII: non-ASCII and
|
||
AltGr/dead-key characters on the German keymap (`| @ { [ ] } \ ~ ^`,
|
||
backtick, `ä ö ü €`) come out as `1`.
|
||
|
||
**What you get:**
|
||
|
||
- Bottom row: `Esc Ctrl Alt [Space] AltGr ← ↑ ↓ → Close`, stable with Shift
|
||
or AltGr. Esc, Ctrl and Alt have Steam's dark special-key face, like its
|
||
AltGr and layout keys.
|
||
- AltGr + arrows: Home, End, Page Up, Page Down (hinted on the keys);
|
||
Shift + arrows select text.
|
||
- Shift + Tab moves the focus backwards (stock Steam drops the Shift);
|
||
Ctrl + Shift + Tab works too (e.g. previous Firefox tab). A locked Shift
|
||
is released after it, like a tapped one. Shift + Enter and
|
||
Shift + Backspace stay plain Enter and Backspace.
|
||
- AltGr + the key left of Backspace (`´` on German, `=` on US): Delete,
|
||
labelled like Steam's Delete key in its language (`Entf`; `Del` if that is
|
||
longer), hinted on the key without AltGr; repeats while held. Layouts with
|
||
an AltGr character on that key get none.
|
||
- Layouts without AltGr (US, Dvorak, Colemak, Bulgarian, Chinese, Japanese,
|
||
Korean) get an `Fn` key right of the space bar: Steam's AltGr toggle
|
||
(tap: once, tap twice: locked, hold), for Delete and Home/End/Page Up/Down.
|
||
- Ctrl/Alt chords and Esc are real key presses in the VR-selected window; a
|
||
toggled Ctrl/Alt is held down while the keyboard is open (e.g.
|
||
Ctrl+scroll).
|
||
- Characters Steam would type as `1` are typed correctly; everything else
|
||
goes through Steam as before.
|
||
- Enter always types Return (stock Steam may send it to a Steam search box).
|
||
|
||
Applies right away: no reboot or Steam restart needed, and it is re-applied
|
||
after Steam restarts. Turning it off reverts the keyboard.
|
||
|
||
**Layouts:** the character routing targets the German keymap; on others it
|
||
is harmless, and Esc/Ctrl/Alt/arrows work regardless.
|
||
|
||
**Security:** the helper service that presses the keys only accepts
|
||
single-key Ctrl/Alt chords, the extra keys, Shift + Tab, Ctrl/Alt
|
||
hold/release and single non-ASCII/AltGr characters; it cannot type ASCII
|
||
text or press Enter (`allowlist.mjs`, tested by the flake check
|
||
`vr-keyboard-extra-keys`).
|
||
|
||
**Caveat:** depends on Steam UI internals; after a Steam update that changes
|
||
them the keyboard stays stock and the journal says why (see
|
||
[after a Steam update](ui-patches.md#after-a-steam-update)). Tested with
|
||
Steam client 1790377368 (UI build 11041156).
|
||
|
||
**Remove when** Steam's VR keyboard gets these keys.
|
||
|
||
### How it works
|
||
|
||
- The `steam-keyboard-patch` user service
|
||
(`modules/vr-keyboard-extra-keys/xdotool-helper.mjs`) injects a patch into
|
||
Steam's `SharedJSContext` over DevTools (`127.0.0.1:8080`) and re-injects
|
||
it after Steam restarts; stopping it (or disabling the option)
|
||
runs the unpatch.
|
||
- Ctrl/Alt chords, Esc, Shift + arrows and Shift + Tab are sent with
|
||
`xdotool key` on `:0` (focus follows the VR-selected window); a toggled
|
||
Ctrl/Alt is held down with `xdotool`. Shift is never held as a real key
|
||
(Steam types capitals itself); `shift+Tab` is `ISO_Left_Tab` on any
|
||
keymap.
|
||
- Problem characters (non-ASCII, AltGr/dead-key characters on the German
|
||
keymap) are typed with `xdotool type`; everything else goes through
|
||
Steam's own text emulation. On other keymaps those characters are still
|
||
typed by xdotool, which is why the routing is harmless there.
|
||
- The new keys use Steam's key type Meta (the dark face); the Delete key
|
||
takes Backspace's key type.
|
||
- Found by signature (see
|
||
[finders and signatures](ui-patches.md#finders-and-signatures)).
|
||
|
||
## Swipe and suggestions
|
||
|
||
`keyboard.vr.*`, module `vr-keyboard`.
|
||
|
||
|
||
|
||
**Problem:** Steam's VR keyboard is tap-only: no swipe typing, no
|
||
suggestions, and deleting more than a few characters means many Backspace
|
||
taps.
|
||
|
||
**What you get** with `keyboard.vr.enable` (the sub-features `swipe`,
|
||
`autocorrect`, `completions`, `backspaceDrag` and `haptics` are on by
|
||
default):
|
||
|
||
- **Swipe:** press the trigger on the first letter, sweep over the others,
|
||
release on the last. The word is typed with a space before it if needed
|
||
(`text.autoSpace`); alternatives show in the strip. `'` and `-` are typed,
|
||
not swiped.
|
||
- **Suggestions** never change text by themselves: a finished tapped word
|
||
that isn't in the dictionary gets corrections (itself first;
|
||
`autocorrect`), a word being tapped gets completions (the typed letters
|
||
first; `completions`). A pick replaces exactly what it typed and can be
|
||
switched again.
|
||
- **Backspace drag:** drag Backspace left to delete one character per
|
||
`pixelsPerChar`, with a detent (`wordDetentPixels`) at each word border
|
||
and at the start of what the keyboard typed; drag back right to retype.
|
||
- **Strip** (`suggestions.position`): a SteamVR dashboard panel below or
|
||
above the keyboard, or inside the keyboard over its number row. Its
|
||
buttons take the keyboard's key style. Below/above uses the
|
||
[SteamVR debugger](steamvr-debugger.md), turned on automatically.
|
||
- **Haptics:** light ticks for drag steps and picks, a Snap at word detents.
|
||
|
||
**Dictionary** (`dictionary.*`): by default the `keyboard.layout` language
|
||
(de, fr, es, it, nl, pt, sv) plus English, else English only. Add words
|
||
(`extraWords`, `extraWordFiles`), remove some (`excludeWords`) or configure
|
||
the languages:
|
||
|
||
```nix
|
||
steamFrame.keyboard.vr = {
|
||
enable = true;
|
||
suggestions.position = "inside"; # no SteamVR panel
|
||
dictionary.extraWords = [ "SteamOS" "Nix" ];
|
||
dictionary.excludeWords = [ "teh" ];
|
||
backspaceDrag.pixelsPerChar = 20;
|
||
};
|
||
```
|
||
|
||
**Text memory:** the keyboard can't read the text field, so it remembers
|
||
what it typed itself (`text.bufferChars`); anything it can't follow (Enter,
|
||
arrows, extraKeys' keys, another field, `text.resetAfterIdleSeconds`) resets
|
||
that, and suggestions only replace text the memory proves intact. Works with
|
||
and without [extra keys](#extra-keys) (with it, non-ASCII words are typed
|
||
via its xdotool helper).
|
||
|
||
**Caveats:** depends on Steam/SteamVR UI internals; after an update that
|
||
changes them the keyboard stays stock (see
|
||
[after a Steam update](ui-patches.md#after-a-steam-update)). Accented words
|
||
of other languages are in the dictionary but only swipable where the layout
|
||
has the letters.
|
||
|
||
**Remove when** Steam's VR keyboard gets swipe typing and suggestions.
|
||
|
||
### How it works
|
||
|
||
- A Steam UI patch (`vr-keyboard`, injected by `steam-ui-patches` like the
|
||
other [UI patches](ui-patches.md)); sources in `modules/vr-keyboard/`.
|
||
- Swiped words are matched by shape (SHARK2-style template matching)
|
||
against a dictionary built at build time from wordfreq frequency lists
|
||
and Hunspell, both from nixpkgs (per language the `words` most frequent
|
||
wordfreq entries, shifted by `frequencyOffset`, filtered by Hunspell
|
||
except words at or above `keepFrequentAbove`).
|
||
- The strip below/above the keyboard is a SteamVR dashboard panel
|
||
(`suggestions-panel/patch.js`, a patch of SteamVR's `systemui` page on
|
||
port 8087), fed by the `vr-keyboard-relay` user service
|
||
(`suggestions-panel/relay.mjs`) between the two pages.
|
||
With `inside` neither the panel nor the relay runs.
|
||
- Found by signature (entries `vr-keyboard`, `vr-keyboard-panel`).
|
||
|
||
**Tests:** `nix flake check` (checks `vr-keyboard`: text model, corrector,
|
||
decoder accuracy on German + English) and `keyboard.vr.checks` for the
|
||
configured dictionary.
|
||
|
||
**Debugging:** `window.__sfuiSwipeLog` and `__sfuiSwipePaths` in Steam's
|
||
SharedJSContext (replay swipes with `scripts/vr-keyboard-replay.mjs`).
|