mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-14 15:00:53 +02:00
voice/ft-voiced (was frame_voice.py) runs in Frametop's container as frametop-voice.service. It keeps the mic, the wake word and live text, and hands the keyboard side to the input relay: - push-to-talk keys are key combinations (dictate, dictate_hold); the relay sends "toggle|start|stop" to @frametop_voice, taken only from this user - text goes to the relay as "type TEXT" (ASCII, in pieces of 2000 bytes), so it lands where typing goes; the relay holds it back while a modifier is held - controller buttons are read from vrserver's web socket with input/vrws.py, in a thread, so the websockets package goes No own uinput keyboard, fd store, evdev, or controller service any more. whisper-server runs only while the wake word is on, started and stopped by ft-voiced. Voice commands and hooks run on the host through distrobox-host-exec (env passed through env(1)); their FRAME_VOICE_* variables keep frame-voice's names. Settings: ~/.config/frametop-voice.json (voice_settings.py, standard library), wake word off by default, "advanced" for the old config.toml tunables. voice/run.sh install carries frame-voice's settings over once (its keys become key combinations) and turns the standalone units off. Install: install.sh step 8/11 asks (yes by default; --no-voice through get.sh, install-release.sh and a FrameDrop checkbox). voice/build.sh builds whisper-server (whisper.cpp d09f61a, Vulkan, static) and build/venv; voice/models.sh fetches the 322 MB of models into ~/.local/share/frametop/voice by fixed URL and sha256. The image gets glslc, spirv-headers-devel and pipewire-utils, uv.lock a voice group (sherpa-onnx with its core pinned: uv dropped it), CI smoke whisper-server and the venv. uninstall.sh asks about the models; update-check and report.sh know the service (report.sh leaves out the lines with what was said). Tests: voice/test (settings, carrying over, typing through a stand-in relay, push-to-talk, controller buttons from a stand-in web socket) in just test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
194 lines
9.4 KiB
Bash
Executable File
194 lines
9.4 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Install everything on the Steam Frame: the build container, Frametop (multi-screen
|
|
# desktop, input relay, universal 3D mouse, settings app), and optionally voice dictation,
|
|
# gaze mode, our own eye tracker for it, and the Bluetooth fixes. Run it on the headset in a terminal, from this repo. It's safe to re-run,
|
|
# for example after `git pull`. (Hand tracking, hands/, is deferred: it isn't offered here.)
|
|
# (It also works from a PC over SSH; see "Developing from a PC" in the README.)
|
|
# In a release (pack/install-release.sh), the programs come built from its image: this installs the
|
|
# distrobox the release brings, makes the release's container from its image, and builds nothing.
|
|
#
|
|
# Usage: ./install.sh [--yes] [--no-voice] [--no-eye-tracker] [--no-bluetooth | --bluetooth]
|
|
# --yes don't ask; installs voice dictation, gaze mode, and our eye tracker if sudo can
|
|
# run without a password prompt; skips the Bluetooth fixes and the SteamVR restart
|
|
# --no-voice don't offer voice dictation (Frametop Voice: it downloads about 320 MB)
|
|
# --no-eye-tracker don't offer our own eye tracker
|
|
# --no-bluetooth don't offer the Bluetooth fixes
|
|
# --bluetooth install the Bluetooth fixes without asking (with --yes: if sudo can run
|
|
# without a password prompt)
|
|
set -euo pipefail
|
|
|
|
root=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
|
. "$root/scripts/_env.sh"
|
|
|
|
assume_yes=0 bluetooth=1 eye_tracker=1 voice=1
|
|
for arg in "$@"; do
|
|
case $arg in
|
|
--yes) assume_yes=1 ;;
|
|
--no-bluetooth) bluetooth=0 ;;
|
|
--bluetooth) bluetooth=2 ;;
|
|
--no-eye-tracker) eye_tracker=0 ;;
|
|
--no-voice) voice=0 ;;
|
|
-h|--help) sed -n '2,18p' "$0"; exit 0 ;;
|
|
*) echo "unknown option: $arg" >&2; exit 2 ;;
|
|
esac
|
|
done
|
|
|
|
# Only the questions read from the terminal (or whatever stdin is); the build steps get
|
|
# no input, so they can't swallow typed-ahead or piped answers.
|
|
exec 3<&0 </dev/null
|
|
|
|
step() { printf '\n\033[1m== %s\033[0m\n' "$*"; }
|
|
ask() { # ask "question" default(y|n)
|
|
[ "$assume_yes" = 1 ] && [ "$2" = y ] && return 0
|
|
[ "$assume_yes" = 1 ] && return 1
|
|
local hint answer
|
|
hint=$([ "$2" = y ] && echo "Y/n" || echo "y/N")
|
|
read -r -p "$1 [$hint] " answer <&3 || answer=
|
|
answer=${answer:-$2}
|
|
[[ $answer =~ ^[Yy] ]]
|
|
}
|
|
# Can sudo run here without a password prompt (SUDO_ASKPASS, the repo's .env, a recent sudo)?
|
|
# See frame_sudo in scripts/_env.sh.
|
|
sudo_quiet() {
|
|
{ [ "$FRAME_LOCAL" = 1 ] && [ -n "${SUDO_ASKPASS:-}" ]; } && return 0
|
|
grep -q '^steamos_root_pwd=.' "$REPO_ROOT/.env" 2>/dev/null && return 0
|
|
on_frame 'sudo -n true' 2>/dev/null
|
|
}
|
|
# A release never builds (FRAME_RELEASE, scripts/_env.sh): its programs come in its image.
|
|
build() { [ "$FRAME_RELEASE" = 1 ] || "$@"; }
|
|
|
|
if [ "$FRAME_RELEASE" = 1 ]; then
|
|
echo "Installing Frametop $(sed -n 's/^VERSION=//p' "$root/.frametop-release") on this Steam Frame from $FRAME_REPO"
|
|
elif [ "$FRAME_LOCAL" = 1 ]; then
|
|
echo "Installing on this Steam Frame from $FRAME_REPO"
|
|
else
|
|
echo "Installing on $FRAME_HOST over SSH (repo copy at $FRAME_REPO)"
|
|
"$root/scripts/sync.sh" >/dev/null
|
|
fi
|
|
if [ "$FRAME_LOCAL" = 0 ] || [ -n "${SSH_CONNECTION:-}" ]; then
|
|
echo "Over SSH: if the connection drops, run this again (it keeps what it downloaded). Services"
|
|
echo "that need SteamVR start with it if it isn't running now."
|
|
fi
|
|
|
|
step "1/11 distrobox (container tool, installed in your home folder)"
|
|
if on_frame 'test -x ~/.local/bin/distrobox'; then
|
|
echo "already installed: $(on_frame '~/.local/bin/distrobox version | head -1')"
|
|
elif [ "$FRAME_RELEASE" = 1 ]; then
|
|
# The release's image brings the distrobox it was tested with (pack/Containerfile).
|
|
on_frame 'cd pack/build/distrobox && ./install --prefix ~/.local'
|
|
else
|
|
on_frame 'set -e; mkdir -p ~/dev/src
|
|
# A tested release, so upstream changes cannot break new installs.
|
|
[ -d ~/dev/src/distrobox ] || git clone --depth 1 --branch 1.8.2.5 https://github.com/89luca89/distrobox.git ~/dev/src/distrobox
|
|
cd ~/dev/src/distrobox && ./install --prefix ~/.local'
|
|
fi
|
|
|
|
if [ "$FRAME_RELEASE" = 1 ]; then
|
|
step "2/11 Frametop's container (from this release's image)"
|
|
"$root/pack/release-box.sh"
|
|
else
|
|
step "2/11 build container (Fedora 44 'dev', about 1-2 GB the first time)"
|
|
"$root/setup/dev-container.sh"
|
|
fi
|
|
|
|
step "3/11 input relay (keeps Bluetooth mice working in SteamVR, device roles, button maps)"
|
|
"$root/desktops.sh" relay install
|
|
|
|
step "4/11 3D mouse: SteamVR driver"
|
|
build "$root/pointer/driver/build.sh"
|
|
"$root/pointer/driver/install.sh" install
|
|
|
|
step "5/11 3D mouse: pointer helper service"
|
|
build "$root/pointer/helper/build.sh"
|
|
"$root/pointer/helper/run.sh" install
|
|
|
|
step "6/11 power service (turns the displays off while the headset isn't used, even on a stand)"
|
|
build "$root/power/build.sh"
|
|
"$root/power/run.sh" install
|
|
|
|
step "7/11 multi-screen desktop (ft-screens), Frametop Input Settings, Display Settings, and Remote Displays"
|
|
build "$root/screens/build.sh"
|
|
build "$root/stream/build.sh" # ft-stream: remote displays (Frametop Remote Displays)
|
|
"$root/desktops.sh" install >/dev/null
|
|
"$root/input-settings/install.sh"
|
|
"$root/display-settings/install.sh"
|
|
"$root/remote-displays/install.sh"
|
|
"$root/remote/install.sh"
|
|
on_frame "sed -i 's/^POINTER=0/POINTER=1/' ~/.config/frametop.conf; grep -q '^POINTER=' ~/.config/frametop.conf || echo 'POINTER=1' >> ~/.config/frametop.conf"
|
|
echo "the launcher's Desktop entry now opens the multi-screen desktop; 3D mouse on (POINTER=1 in ~/.config/frametop.conf)"
|
|
"$root/scripts/conf-migrate.sh"
|
|
|
|
step "8/11 voice dictation (optional: speak, and it's typed where you type; nothing leaves the headset)"
|
|
if [ "$voice" = 0 ]; then
|
|
echo "skipped (--no-voice). Install later with: voice/run.sh install"
|
|
elif ask "Install voice dictation? A Right Ctrl tap or holding the left d-pad down starts it; it downloads about 320 MB of speech models." y; then
|
|
build "$root/voice/build.sh"
|
|
"$root/voice/run.sh" install ||
|
|
echo "voice dictation: not installed (above). Try again later with: voice/run.sh install"
|
|
else
|
|
echo "skipped. Install later with: voice/run.sh install"
|
|
fi
|
|
|
|
step "9/11 gaze mode (optional, experimental: the pointer goes where you look)"
|
|
gaze=0
|
|
if ask "Install gaze mode? You turn it on and calibrate it in Frametop Input Settings, on the Gaze page." y; then
|
|
"$root/gaze/run.sh" install
|
|
gaze=1
|
|
else
|
|
echo "skipped. Install later with: gaze/run.sh install"
|
|
fi
|
|
|
|
# Ours is gaze mode's default once it's installed (GAZE_TRACKER=auto). Without it, gaze mode
|
|
# uses SteamVR's eye tracker.
|
|
step "10/11 our own eye tracker for gaze mode (recommended: more accurate than SteamVR's)"
|
|
if [ "$gaze" = 0 ]; then
|
|
echo "skipped: gaze mode isn't installed. Install it later with: gaze/tracker/install.sh"
|
|
elif [ "$eye_tracker" = 0 ]; then
|
|
echo "skipped. Install it later with: gaze/tracker/install.sh"
|
|
elif [ "$assume_yes" = 1 ] && ! sudo_quiet; then
|
|
echo "skipped: it needs your password (sudo), and --yes doesn't ask. Install it later with: gaze/tracker/install.sh"
|
|
elif ask "Install our own eye tracker? Gaze mode then uses it instead of SteamVR's. Its frame grabber is a small system service, so it needs your password (sudo)$([ "$FRAME_RELEASE" = 1 ] || echo ", and it downloads about 165 MB (numpy, OpenCV)")." y; then
|
|
if "$root/gaze/tracker/install.sh" install; then
|
|
# Configs made before GAZE_TRACKER=auto say steam, which keeps SteamVR's.
|
|
tracker=$(on_frame "sed -n 's/^GAZE_TRACKER=\([a-z]*\).*/\1/p' ~/.config/frametop.conf | tail -1")
|
|
if [ "$tracker" = steam ] &&
|
|
ask "Your settings pick SteamVR's eye tracker (GAZE_TRACKER=steam, the old default). Use ours instead?" y; then
|
|
on_frame "sed -i 's~^GAZE_TRACKER=steam\b.*~GAZE_TRACKER=auto # gaze service: auto = our own eye tracker when installed, else SteamVR | own | steam~' ~/.config/frametop.conf"
|
|
tracker=auto
|
|
fi
|
|
if [ "$tracker" = steam ]; then
|
|
echo "installed; your settings keep SteamVR's eye tracker (pick Own tracker on the Gaze page to use ours)"
|
|
else
|
|
echo "gaze mode uses our eye tracker: calibrate it once, with Calibrate on the Gaze page"
|
|
fi
|
|
else
|
|
echo "our eye tracker didn't install, so gaze mode uses SteamVR's. Try again with: gaze/tracker/install.sh"
|
|
fi
|
|
else
|
|
echo "skipped: gaze mode uses SteamVR's eye tracker. Install ours later with: gaze/tracker/install.sh"
|
|
fi
|
|
|
|
step "11/11 Bluetooth fixes (optional; they let LE mice and keyboards like the Swiftpoint Z3 reconnect)"
|
|
if [ "$bluetooth" = 2 ] && [ "$assume_yes" = 1 ] && ! sudo_quiet; then
|
|
echo "skipped: they need your password (sudo), and --yes doesn't ask. Install later with: setup/bluetooth/install.sh install"
|
|
elif [ "$bluetooth" = 2 ] || { [ "$bluetooth" = 1 ] && ask "Install the Bluetooth fixes? They need your password (sudo)." n; }; then
|
|
"$root/setup/bluetooth/install.sh" install
|
|
else
|
|
echo "skipped. Install later with: setup/bluetooth/install.sh install"
|
|
fi
|
|
|
|
step "Done"
|
|
cat <<'EOF'
|
|
SteamVR has to restart once, to load the 3D mouse driver and to start the input relay
|
|
before it. Restarting SteamVR closes everything open in VR, including this terminal if
|
|
it's in a VR desktop. Rebooting the headset works too.
|
|
|
|
Recommended: in Frametop Display Settings > Power, choose when the displays turn off
|
|
while the headset isn't used (for a stand or mount that covers its proximity sensor), and
|
|
turn on Stay awake while plugged in, so Steam doesn't put the headset to sleep while it
|
|
charges. The displays still turn off when you take the headset off.
|
|
EOF
|
|
if ask "Restart SteamVR now?" n; then
|
|
on_frame 'systemctl --user restart steamvr.service'
|
|
fi
|