diff --git a/README.md b/README.md index 54a5e7a..94c7da0 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ Standalone OpenVR overlay; no Steam store AppID or sudo. First v0.1 release is i ## Requirements - Steam Frame with usable SteamVR/OpenVR and Gamescope for the **native** overlay and text delivery; Linux ARM64/glibc for the current installer payload format. -- For voice recognition, separately provision a compatible **CPU Python runtime** (moondream 2.4.0 / Kestrel 0.8.0 and dependencies) and the pinned local Parakeet Redux model. Neither is bundled or installed with pip by the current native-only installer. There is no fallback ASR service. +- For voice recognition, a **CPU Python runtime** (moondream 2.4.0 / Kestrel 0.8.0, CPU Torch 2.8.0) and the pinned local Parakeet Redux model. Neither is bundled; each has its own explicit install step: `sh install.sh --install-model --backend redux --yes` and `sh install.sh --install-runtime --yes` (needs Python 3.10–3.13 with venv; about 200 MB download). Use `--print-plan` first to see exactly what each fetches. There is no fallback ASR service. - A local source build needs CMake 3.20+, C++20 and explicit native libraries/SDK; the default hardware-free build needs only CMake and C++20. See [build requirements](docs/build.md). ## Install (local artifacts only) diff --git a/TODO.md b/TODO.md index 4990131..efed8cd 100644 --- a/TODO.md +++ b/TODO.md @@ -50,8 +50,8 @@ accepted; local code/tests cannot establish a fixed delivery regression. ## A. First release (v0.1) blockers -Remaining release gates (2026-09-25): implement D4 (pip runtime install), then a -fresh clean install on Frame that also checks the missing-runtime panel message +Remaining release gates (2026-09-25): D4 is implemented; next a +fresh clean install on Frame (install → `--install-model` → `--install-runtime`) that also checks the missing-runtime panel message (H), then tag `v0.1.` and update the README install section. - [x] **A8. UI polish batch (local; deploy pending).** Removed the fixed @@ -132,10 +132,12 @@ fresh clean install on Frame that also checks the missing-runtime panel message fixtures only, not a released archive or an installed Frame UI handoff. - [ ] **D3. Publish a prebuilt ARM64 archive.** (M) **Deferred.** v0.1 is source-only. Revisit after source-build acceptance. -- [ ] **D4. Source install fetches the Python runtime with pip.** (M) Proposed, not - implemented: the installer creates a user-local venv and installs pinned - moondream/Kestrel with the CPU Torch wheel, on an explicit flag/confirmation. - The installer does not run pip today. +- [x] **D4. Installer fetches the Python runtime with pip.** (M) Implemented + 2026-09-25 as explicit `install.sh --install-runtime --yes`: user-local venv, + CPU `torch==2.8.0` from PyTorch's CPU index, `moondream==2.4.0` from PyPI, + import/no-CUDA verification, then `python=` in `paths.conf`. Offline tests + mock pip; a real run passed on x86-64 Python 3.12. Needs the clean Frame + install to confirm on ARM64. ## E. Naming and versioning @@ -209,5 +211,6 @@ this app is future design and out of scope for v0.1. ## Open questions -Source-install runtime provisioning (D4), P1 real-target behavior -and live headset validation are unresolved gates, not implied by checked source tasks. +The clean-account Frame install (including D4's ARM64 pip run and the +missing-runtime message) is the remaining unresolved gate; checked source tasks +do not imply it. diff --git a/docs/build.md b/docs/build.md index 3a035da..ee537c1 100644 --- a/docs/build.md +++ b/docs/build.md @@ -63,7 +63,8 @@ initialize OpenVR, open a microphone, run ASR, download files or inject input. source-build mode, safe extraction, atomic current-version selection, retained rollback, runtime/install lock, foreign-file refusal and explicit unregister-before-uninstall acknowledgement. Source mode needs a local - compiler, SDK and libraries; the installer does not yet pip-install a runtime. + compiler, SDK and libraries. An explicit `--install-runtime --yes` step + pip-installs the pinned CPU Python runtime into a user-local venv. ## Deliberately not claimed @@ -103,7 +104,8 @@ compatible offline runtime and independent tests. Native `--run` flags `--backen `--manifest-dir /absolute/manifests` are wired through the installed launcher as an explicit override, not a provisioning command. Inference uses the `moondream` Python package and Kestrel runtime, separately -provisioned in your own environment; builds/tests/installer do not pip-install them. See +provisioned in your own environment or by the explicit `install.sh --install-runtime --yes`; +builds and tests never pip-install them. See [third-party notes](third-party.md). ## Developer native build diff --git a/docs/install-design.md b/docs/install-design.md index 741f7e8..de8ff84 100644 --- a/docs/install-design.md +++ b/docs/install-design.md @@ -4,9 +4,10 @@ Goal: a one-command, pinned GitHub release install for Steam Frame without a Ste store AppID, sudo or end-user compiler. **No public archive or verified clean install is published. Do not advertise a `curl | sh` command as functional.** The local installer has binary-archive and explicitly provisioned source-build -modes, machine-readable plans/results and an attended TTY path. The current -native-only artifact does not include or pip-install an ASR runtime; it is not -a one-command voice-typing experience. See [packaging](packaging.md) for exact +modes, machine-readable plans/results and an attended TTY path. The native-only +artifact does not include an ASR runtime; a separate explicit +`--install-runtime --yes` step pip-installs the pinned CPU runtime into a +user-local venv. Install, model and runtime are still three commands. See [packaging](packaging.md) for exact flags and [third-party notes](third-party.md). `scripts/install-preflight.sh` is a read-only Linux ARM64/glibc/bootstrap check; @@ -52,7 +53,7 @@ already be available for registration; never start/restart it for installation. second **Confirm Install** click; the installed/native UI route still needs clean-target and headset acceptance. `--without-model` permits an archive install without bundled model files. Neither operation installs Torch, - moondream, Kestrel or an interpreter. Launch paths to an independently + moondream or Kestrel; only `--install-runtime --yes` does (see packaging). Launch paths to an independently authorized runtime/model can be set in `paths.conf`. - **Noninteractive**: supply flags and `--yes` for network/model consent. `--print-plan` is read-only; `--json` provides structured results/errors and @@ -66,7 +67,7 @@ listing, unrelated application dependency, microphone recording, input injection or automatic update daemon. Hashes detect accidental/unauthorized alteration of a downloaded artifact but do not authenticate a compromised publisher; release metadata needs independent trust. No installer operation silently runs -pip or launches inference. Native-only archives support overlay checks but need +pip or launches inference; pip runs only under explicit `--install-runtime --yes`. Native-only archives support overlay checks but need an externally provisioned runtime/weights before voice typing. Selecting an uninstalled backend does not authorize a download or supply its inference engine. diff --git a/docs/packaging.md b/docs/packaging.md index fdf5221..8067fa7 100644 --- a/docs/packaging.md +++ b/docs/packaging.md @@ -102,8 +102,21 @@ installed `redux.json` bytes: under the model lock a mismatch fails **before** a model directory is created or any network request. In the locally implemented Models UI, Install displays source, rounded size, license text, attribution and the fingerprint; only the second Confirm Install click passes that fingerprint -through the backend helper to the installer. Installation alone does not +through the backend helper to the installer. Model installation does not provision a CPU Python runtime; no in-panel flow has been accepted on Frame. + +`sh install.sh --install-runtime --yes` explicitly creates a user-local venv +under `$XDG_DATA_HOME/frameyap/runtimes/` from the invoking `python3` (3.10–3.13), +pip-installs `torch==2.8.0` from PyTorch's CPU index, then `moondream==2.4.0` +(which pins kestrel 0.8.0, kestrel-native 0.1.8, kestrel-kernels 0.7.0) from PyPI +with Torch constrained, binary wheels only and `pip --isolated`. It verifies the +imports and that Torch has no CUDA, then sets `python=` in `paths.conf` (other +lines kept) and removes superseded FrameYap-managed runtimes. A failure removes +only the new venv and leaves `paths.conf` unchanged. `--print-plan --json` shows +the packages, index and size (about 200 MB of wheels, roughly 1–1.5 GB on disk) +without touching anything. The app must be closed (install lock). Transitive +dependencies are not pinned. Validated end to end on x86-64 Python 3.12 +(2026-09-25); a clean Frame install is still pending. In the source tree, `python3 scripts/model-status.py --list-models` or `--check-model redux --model-dir /absolute/model` hash-checks local files; the native `frameyap --list-models` / `--check-model` entry points use the @@ -151,7 +164,7 @@ model=/absolute/path/to/pinned/local/model The launcher reads these as **literal absolute paths**, not shell code, and does not follow a symlink to the config file. Environment variables override either setting for an explicit launch. This `paths.conf` survives upgrade/uninstall; -the installer does not populate it or bundle an unauthorized runtime. No +only `--install-runtime` writes its `python=` line, and no runtime is bundled. No pip/bootstrap/model download or fallback is invoked by the launcher. Without paths, the native-only artifact cannot perform voice inference; its default runtime and model locations do not exist. A new installer accepts only the exact diff --git a/docs/worker.md b/docs/worker.md index b17451d..2507cb2 100644 --- a/docs/worker.md +++ b/docs/worker.md @@ -80,8 +80,9 @@ flags do not prove every third-party internal is unable to access a network. Runtime/build/tests perform no downloads; the separate explicit setup utility `scripts/fetch-model.py` can provision the public pinned weights. -Current native-only archives do not bundle or pip-install the runtime; see -[third-party notes](third-party.md). No public runtime bundle has been released. +Native-only archives do not bundle the runtime; the explicit +`install.sh --install-runtime --yes` step pip-installs it into a user-local venv +(see [packaging](packaging.md) and [third-party notes](third-party.md)). No public runtime bundle has been released. Limited ARM64 measurements were taken during development; they are not a claim of complete headset acceptance. diff --git a/install.sh b/install.sh index e932077..e1ee418 100755 --- a/install.sh +++ b/install.sh @@ -70,6 +70,20 @@ CONFIG_DEFAULTS = { "insert": "/user/hand/right/input/a", "enter": "", "quick_chat": "/user/hand/right/input/y"}, } +# Explicit --install-runtime only. CPU Torch comes from PyTorch's CPU index first; +# an unconstrained PyPI resolve selects CUDA/NVIDIA wheels. moondream 2.4.0 pins +# kestrel 0.8.0, kestrel-native 0.1.8 and kestrel-kernels 0.7.0. +RUNTIME_TORCH = "torch==2.8.0" +RUNTIME_TORCH_INDEX = "https://download.pytorch.org/whl/cpu" +RUNTIME_REQUIREMENT = "moondream==2.4.0" +RUNTIME_DOWNLOAD = "about 200 MB of prebuilt wheels (CPU Torch is about 100 MB), roughly 1-1.5 GB on disk; no compilation" +RUNTIME_PYTHON_RANGE = ((3, 10), (3, 14)) # moondream allows <3.15; CPU Torch 2.8.0 wheels stop at 3.13 +RUNTIME_NAME_RE = re.compile(r"cpu-[0-9]{14}\Z") +RUNTIME_VERIFY = ( + "import importlib.metadata as m, torch\n" + "assert torch.version.cuda is None, 'CUDA Torch was installed; CPU build required'\n" + "import moondream\n" + "print('moondream', m.version('moondream'), 'kestrel', m.version('kestrel'), 'torch', torch.__version__)\n") COLOR_RE = re.compile(r"#[0-9a-fA-F]{6}\Z") BUTTON_RE = re.compile(r"/user/hand/(left|right)/input/[A-Za-z0-9_]+\Z") BACKEND_RE = re.compile(r"[a-z][a-z0-9_-]{0,47}\Z", re.ASCII) @@ -103,6 +117,26 @@ def config_path(): return base / "frameyap/config.json" +def paths_config_path(): + return config_path().parent / "paths.conf" + + +def set_runtime_python(python): + """Point the managed launcher at `python`; other paths.conf lines are kept verbatim.""" + path = paths_config_path() + if "\n" in str(python) or not python.is_absolute(): + fail(f"unsafe runtime path: {python!r}") + owned_dir(path.parent) + if path.is_symlink(): + fail(f"refusing symlink paths config: {path}") + lines = (path.read_text().splitlines() if path.exists() else + ["# FrameYap literal paths (python=/abs, model=/abs); never shell code"]) + previous = [line[len("python="):] for line in lines if line.startswith("python=")] + kept = [line for line in lines if not line.startswith("python=")] + atomic_write(path, ("\n".join(kept + [f"python={python}"]) + "\n").encode()) + return previous[-1] if previous else None + + def unique_pairs(pairs): result = {} for key, value in pairs: @@ -581,11 +615,11 @@ def model_target(args, root): @contextlib.contextmanager -def model_download_cancellation(): +def model_download_cancellation(message="model installation cancelled"): # Install-model runs on the main thread. Raising unwinds the owned temp's # finally block; unlike SIGKILL this does not leave a partial download. def terminate(_signal, _frame): - raise ValueError("model installation cancelled") + raise ValueError(message) previous = signal.signal(signal.SIGTERM, terminate) try: @@ -652,6 +686,52 @@ def install_model(args, root): print(f"Backend {backend.id} model verified at {dest}; license {backend.license_id}. Configure runtime/model paths explicitly for launch.") +def runtime_step(args, name, command): + if args.json: + print(json.dumps({"ok": True, "event": "runtime_step", "step": name}), flush=True) + else: + print(f"[runtime] {name}", flush=True) + # Tool output goes to stderr so --json stdout stays one event per line. + subprocess.run(command, check=True, stdout=sys.stderr, stdin=subprocess.DEVNULL) + + +def install_runtime(args, root): + low, high = RUNTIME_PYTHON_RANGE + if not low <= sys.version_info[:2] < high: + fail("the runtime needs Python 3.10-3.13 (CPU Torch 2.8.0 wheels); found " + platform.python_version()) + if importlib.util.find_spec("venv") is None or importlib.util.find_spec("ensurepip") is None: + fail("python3 cannot create a virtual environment (venv/ensurepip missing); no packages installed") + runtimes = root / "runtimes" + owned_dir(runtimes) + target = runtimes / datetime.now().strftime("cpu-%Y%m%d%H%M%S") + if target.exists() or target.is_symlink(): + fail(f"runtime directory already exists: {target}") + python = target / "bin/python3" + # --isolated ignores user pip config/env indexes; binary-only avoids compilers. + pip = [str(python), "-m", "pip", "--isolated", "--disable-pip-version-check", "--no-input", + "install", "--only-binary=:all:"] + try: + with model_download_cancellation("runtime installation cancelled"): + runtime_step(args, "create virtual environment", [sys.executable, "-m", "venv", str(target)]) + runtime_step(args, "install CPU Torch", pip + ["--index-url", RUNTIME_TORCH_INDEX, RUNTIME_TORCH]) + constraints = target / "frameyap-constraints.txt" + constraints.write_text(RUNTIME_TORCH + "\n") + runtime_step(args, "install moondream/Kestrel", pip + ["--constraint", str(constraints), RUNTIME_REQUIREMENT]) + runtime_step(args, "verify CPU runtime imports", [str(python), "-c", RUNTIME_VERIFY]) + except BaseException: + shutil.rmtree(target, ignore_errors=True) + raise + previous = set_runtime_python(python) + # Superseded managed runtimes are removed; anything else is never touched. + for item in runtimes.iterdir(): + if item != target and item.is_dir() and not item.is_symlink() and RUNTIME_NAME_RE.fullmatch(item.name): + shutil.rmtree(item) + if not args.json: + print(f"CPU Python runtime installed at {target}; {paths_config_path()} now has python={python}.") + if previous and not previous.startswith(str(runtimes) + "/"): + print(f"Previous python={previous} was replaced; restore it in paths.conf to switch back.") + + def source_preflight(args): """Read-only checks before the installer creates its install root.""" source = Path(args.source).expanduser().resolve(strict=True) @@ -798,7 +878,7 @@ def do_install(args, root, launcher): select(root, "current", f"versions/{version}") print(f"FrameYap {version} installed. OpenVR registration is NOT automatic; see docs/packaging.md.") if json.loads((target / "release.json").read_text())["runtime"] == "external-authorized-python": - print("ASR runtime is NOT included. Supply an independently authorized environment with FRAMEYAP_PYTHON or --python; no packages are downloaded.") + print("ASR runtime is NOT included. Run install.sh --install-runtime --yes to pip-install it, or supply one with FRAMEYAP_PYTHON/--python or paths.conf.") def uninstall(root, launcher): @@ -840,7 +920,8 @@ def uninstall(root, launcher): manifest.unlink() if desktop.exists(): desktop.unlink() - print("FrameYap removed; config and saved models preserved. OpenVR unregister acknowledgement was required.") + print("FrameYap removed; config, saved models and any runtimes/ directory preserved. " + "OpenVR unregister acknowledgement was required.") class UsageError(ValueError): @@ -859,6 +940,8 @@ def resolve_args(argv): action.add_argument("--rollback", action="store_true") action.add_argument("--uninstall", action="store_true") action.add_argument("--install-model", action="store_true", help="explicit model provisioning for installed backend") + action.add_argument("--install-runtime", action="store_true", + help="explicit pip install of the pinned CPU Python runtime into a user-local venv") parser.add_argument("--mode", choices=("binary", "source"), default="binary") parser.add_argument("--source", help="explicit local FrameYap source tree (source mode only)") parser.add_argument("--openvr-root", help="local OpenVR SDK root (source mode)") @@ -884,7 +967,7 @@ def resolve_args(argv): if args.mode == "source": if not args.source or any(not getattr(args, name) for name in source_inputs): parser.error("--mode source requires --source and explicit --openvr-root, --openvr-library, --openvr-license, --sdl-library, --sdl-license") - if args.archive or args.sha256 or args.rollback or args.uninstall or args.install_model: + if args.archive or args.sha256 or args.rollback or args.uninstall or args.install_model or args.install_runtime: parser.error("source mode cannot combine with binary archive or lifecycle operations") elif args.source or any(getattr(args, name) for name in source_inputs): parser.error("source inputs require --mode source") @@ -894,7 +977,10 @@ def resolve_args(argv): parser.error("--sha256 must be a 64-character hex SHA-256") if not args.archive and args.sha256: parser.error("--sha256 only applies to --archive") - if not (args.rollback or args.uninstall or args.install_model) and not args.version: + if args.install_runtime and (args.version or args.without_model or args.autolaunch is not None or + args.backend != "redux" or args.model_dir or args.expected_manifest_sha256): + parser.error("--install-runtime is independent of release/model flags") + if not (args.rollback or args.uninstall or args.install_model or args.install_runtime) and not args.version: parser.error("--version VERSION is required; no moving/latest release") if args.uninstall != args.unregistered: parser.error("--uninstall requires --unregistered after explicit OpenVR unregister") @@ -923,7 +1009,7 @@ def resolve_args(argv): def plan(args): operation = ("uninstall" if args.uninstall else "rollback" if args.rollback else - "install-model" if args.install_model else "install") + "install-model" if args.install_model else "install-runtime" if args.install_runtime else "install") return {"operation": operation, "mode": args.mode, "version": args.version, "tag": "v" + args.version if args.version else None, "archive": str(args.archive) if args.archive else None, @@ -931,7 +1017,8 @@ def plan(args): "backend": args.backend, "model_dir": str(args.model_dir) if args.model_dir else None, "expected_manifest_sha256": (args.expected_manifest_sha256.lower() if args.expected_manifest_sha256 else None), "without_model": args.without_model, "autolaunch": args.autolaunch, - "network": bool(args.install_model or (operation == "install" and not args.archive and not args.source)), + "network": bool(args.install_model or args.install_runtime or + (operation == "install" and not args.archive and not args.source)), "registration": ("explicit --register --autostart" if args.autolaunch is True else "explicit --register (autostart off)" if args.autolaunch is False else "none")} @@ -947,6 +1034,12 @@ def main(argv=None): check_expected_manifest(args, manifest_sha) result.update(model_dir=str(model_target(args, root)), model=backend.description(), installed_manifest_sha256=manifest_sha) + if args.install_runtime: + data = Path(os.environ.get("XDG_DATA_HOME") or Path.home() / ".local/share").expanduser().absolute() + result.update(runtime_dir=str(data / "frameyap/runtimes"), base_python=sys.executable, + base_python_version=platform.python_version(), + packages=[RUNTIME_TORCH + " from " + RUNTIME_TORCH_INDEX, RUNTIME_REQUIREMENT + " from PyPI"], + download=RUNTIME_DOWNLOAD, paths_config=str(paths_config_path())) if args.json: print(json.dumps({"ok": True, "event": "plan", **result}, sort_keys=True)) else: @@ -996,6 +1089,8 @@ def main(argv=None): print(f"Rolled back to {previous}") elif args.install_model: install_model(args, root) + elif args.install_runtime: + install_runtime(args, root) else: do_install(args, root, launcher) # The native registration helper takes this same install lock. Never run it @@ -1065,7 +1160,7 @@ def cli(argv=None): try: if structured and "--print-plan" in argv: main(argv) # already emits one JSON plan - elif structured and "--install-model" in argv: + elif structured and ("--install-model" in argv or "--install-runtime" in argv): main(argv) # streaming per-file JSON events for a UI child print(json.dumps({"ok": True, "event": "complete"})) elif structured: diff --git a/scripts/install_payload.py b/scripts/install_payload.py index daaa16a..fc7cbe4 100644 --- a/scripts/install_payload.py +++ b/scripts/install_payload.py @@ -50,6 +50,20 @@ CONFIG_DEFAULTS = { "insert": "/user/hand/right/input/a", "enter": "", "quick_chat": "/user/hand/right/input/y"}, } +# Explicit --install-runtime only. CPU Torch comes from PyTorch's CPU index first; +# an unconstrained PyPI resolve selects CUDA/NVIDIA wheels. moondream 2.4.0 pins +# kestrel 0.8.0, kestrel-native 0.1.8 and kestrel-kernels 0.7.0. +RUNTIME_TORCH = "torch==2.8.0" +RUNTIME_TORCH_INDEX = "https://download.pytorch.org/whl/cpu" +RUNTIME_REQUIREMENT = "moondream==2.4.0" +RUNTIME_DOWNLOAD = "about 200 MB of prebuilt wheels (CPU Torch is about 100 MB), roughly 1-1.5 GB on disk; no compilation" +RUNTIME_PYTHON_RANGE = ((3, 10), (3, 14)) # moondream allows <3.15; CPU Torch 2.8.0 wheels stop at 3.13 +RUNTIME_NAME_RE = re.compile(r"cpu-[0-9]{14}\Z") +RUNTIME_VERIFY = ( + "import importlib.metadata as m, torch\n" + "assert torch.version.cuda is None, 'CUDA Torch was installed; CPU build required'\n" + "import moondream\n" + "print('moondream', m.version('moondream'), 'kestrel', m.version('kestrel'), 'torch', torch.__version__)\n") COLOR_RE = re.compile(r"#[0-9a-fA-F]{6}\Z") BUTTON_RE = re.compile(r"/user/hand/(left|right)/input/[A-Za-z0-9_]+\Z") BACKEND_RE = re.compile(r"[a-z][a-z0-9_-]{0,47}\Z", re.ASCII) @@ -83,6 +97,26 @@ def config_path(): return base / "frameyap/config.json" +def paths_config_path(): + return config_path().parent / "paths.conf" + + +def set_runtime_python(python): + """Point the managed launcher at `python`; other paths.conf lines are kept verbatim.""" + path = paths_config_path() + if "\n" in str(python) or not python.is_absolute(): + fail(f"unsafe runtime path: {python!r}") + owned_dir(path.parent) + if path.is_symlink(): + fail(f"refusing symlink paths config: {path}") + lines = (path.read_text().splitlines() if path.exists() else + ["# FrameYap literal paths (python=/abs, model=/abs); never shell code"]) + previous = [line[len("python="):] for line in lines if line.startswith("python=")] + kept = [line for line in lines if not line.startswith("python=")] + atomic_write(path, ("\n".join(kept + [f"python={python}"]) + "\n").encode()) + return previous[-1] if previous else None + + def unique_pairs(pairs): result = {} for key, value in pairs: @@ -561,11 +595,11 @@ def model_target(args, root): @contextlib.contextmanager -def model_download_cancellation(): +def model_download_cancellation(message="model installation cancelled"): # Install-model runs on the main thread. Raising unwinds the owned temp's # finally block; unlike SIGKILL this does not leave a partial download. def terminate(_signal, _frame): - raise ValueError("model installation cancelled") + raise ValueError(message) previous = signal.signal(signal.SIGTERM, terminate) try: @@ -632,6 +666,52 @@ def install_model(args, root): print(f"Backend {backend.id} model verified at {dest}; license {backend.license_id}. Configure runtime/model paths explicitly for launch.") +def runtime_step(args, name, command): + if args.json: + print(json.dumps({"ok": True, "event": "runtime_step", "step": name}), flush=True) + else: + print(f"[runtime] {name}", flush=True) + # Tool output goes to stderr so --json stdout stays one event per line. + subprocess.run(command, check=True, stdout=sys.stderr, stdin=subprocess.DEVNULL) + + +def install_runtime(args, root): + low, high = RUNTIME_PYTHON_RANGE + if not low <= sys.version_info[:2] < high: + fail("the runtime needs Python 3.10-3.13 (CPU Torch 2.8.0 wheels); found " + platform.python_version()) + if importlib.util.find_spec("venv") is None or importlib.util.find_spec("ensurepip") is None: + fail("python3 cannot create a virtual environment (venv/ensurepip missing); no packages installed") + runtimes = root / "runtimes" + owned_dir(runtimes) + target = runtimes / datetime.now().strftime("cpu-%Y%m%d%H%M%S") + if target.exists() or target.is_symlink(): + fail(f"runtime directory already exists: {target}") + python = target / "bin/python3" + # --isolated ignores user pip config/env indexes; binary-only avoids compilers. + pip = [str(python), "-m", "pip", "--isolated", "--disable-pip-version-check", "--no-input", + "install", "--only-binary=:all:"] + try: + with model_download_cancellation("runtime installation cancelled"): + runtime_step(args, "create virtual environment", [sys.executable, "-m", "venv", str(target)]) + runtime_step(args, "install CPU Torch", pip + ["--index-url", RUNTIME_TORCH_INDEX, RUNTIME_TORCH]) + constraints = target / "frameyap-constraints.txt" + constraints.write_text(RUNTIME_TORCH + "\n") + runtime_step(args, "install moondream/Kestrel", pip + ["--constraint", str(constraints), RUNTIME_REQUIREMENT]) + runtime_step(args, "verify CPU runtime imports", [str(python), "-c", RUNTIME_VERIFY]) + except BaseException: + shutil.rmtree(target, ignore_errors=True) + raise + previous = set_runtime_python(python) + # Superseded managed runtimes are removed; anything else is never touched. + for item in runtimes.iterdir(): + if item != target and item.is_dir() and not item.is_symlink() and RUNTIME_NAME_RE.fullmatch(item.name): + shutil.rmtree(item) + if not args.json: + print(f"CPU Python runtime installed at {target}; {paths_config_path()} now has python={python}.") + if previous and not previous.startswith(str(runtimes) + "/"): + print(f"Previous python={previous} was replaced; restore it in paths.conf to switch back.") + + def source_preflight(args): """Read-only checks before the installer creates its install root.""" source = Path(args.source).expanduser().resolve(strict=True) @@ -778,7 +858,7 @@ def do_install(args, root, launcher): select(root, "current", f"versions/{version}") print(f"FrameYap {version} installed. OpenVR registration is NOT automatic; see docs/packaging.md.") if json.loads((target / "release.json").read_text())["runtime"] == "external-authorized-python": - print("ASR runtime is NOT included. Supply an independently authorized environment with FRAMEYAP_PYTHON or --python; no packages are downloaded.") + print("ASR runtime is NOT included. Run install.sh --install-runtime --yes to pip-install it, or supply one with FRAMEYAP_PYTHON/--python or paths.conf.") def uninstall(root, launcher): @@ -820,7 +900,8 @@ def uninstall(root, launcher): manifest.unlink() if desktop.exists(): desktop.unlink() - print("FrameYap removed; config and saved models preserved. OpenVR unregister acknowledgement was required.") + print("FrameYap removed; config, saved models and any runtimes/ directory preserved. " + "OpenVR unregister acknowledgement was required.") class UsageError(ValueError): @@ -839,6 +920,8 @@ def resolve_args(argv): action.add_argument("--rollback", action="store_true") action.add_argument("--uninstall", action="store_true") action.add_argument("--install-model", action="store_true", help="explicit model provisioning for installed backend") + action.add_argument("--install-runtime", action="store_true", + help="explicit pip install of the pinned CPU Python runtime into a user-local venv") parser.add_argument("--mode", choices=("binary", "source"), default="binary") parser.add_argument("--source", help="explicit local FrameYap source tree (source mode only)") parser.add_argument("--openvr-root", help="local OpenVR SDK root (source mode)") @@ -864,7 +947,7 @@ def resolve_args(argv): if args.mode == "source": if not args.source or any(not getattr(args, name) for name in source_inputs): parser.error("--mode source requires --source and explicit --openvr-root, --openvr-library, --openvr-license, --sdl-library, --sdl-license") - if args.archive or args.sha256 or args.rollback or args.uninstall or args.install_model: + if args.archive or args.sha256 or args.rollback or args.uninstall or args.install_model or args.install_runtime: parser.error("source mode cannot combine with binary archive or lifecycle operations") elif args.source or any(getattr(args, name) for name in source_inputs): parser.error("source inputs require --mode source") @@ -874,7 +957,10 @@ def resolve_args(argv): parser.error("--sha256 must be a 64-character hex SHA-256") if not args.archive and args.sha256: parser.error("--sha256 only applies to --archive") - if not (args.rollback or args.uninstall or args.install_model) and not args.version: + if args.install_runtime and (args.version or args.without_model or args.autolaunch is not None or + args.backend != "redux" or args.model_dir or args.expected_manifest_sha256): + parser.error("--install-runtime is independent of release/model flags") + if not (args.rollback or args.uninstall or args.install_model or args.install_runtime) and not args.version: parser.error("--version VERSION is required; no moving/latest release") if args.uninstall != args.unregistered: parser.error("--uninstall requires --unregistered after explicit OpenVR unregister") @@ -903,7 +989,7 @@ def resolve_args(argv): def plan(args): operation = ("uninstall" if args.uninstall else "rollback" if args.rollback else - "install-model" if args.install_model else "install") + "install-model" if args.install_model else "install-runtime" if args.install_runtime else "install") return {"operation": operation, "mode": args.mode, "version": args.version, "tag": "v" + args.version if args.version else None, "archive": str(args.archive) if args.archive else None, @@ -911,7 +997,8 @@ def plan(args): "backend": args.backend, "model_dir": str(args.model_dir) if args.model_dir else None, "expected_manifest_sha256": (args.expected_manifest_sha256.lower() if args.expected_manifest_sha256 else None), "without_model": args.without_model, "autolaunch": args.autolaunch, - "network": bool(args.install_model or (operation == "install" and not args.archive and not args.source)), + "network": bool(args.install_model or args.install_runtime or + (operation == "install" and not args.archive and not args.source)), "registration": ("explicit --register --autostart" if args.autolaunch is True else "explicit --register (autostart off)" if args.autolaunch is False else "none")} @@ -927,6 +1014,12 @@ def main(argv=None): check_expected_manifest(args, manifest_sha) result.update(model_dir=str(model_target(args, root)), model=backend.description(), installed_manifest_sha256=manifest_sha) + if args.install_runtime: + data = Path(os.environ.get("XDG_DATA_HOME") or Path.home() / ".local/share").expanduser().absolute() + result.update(runtime_dir=str(data / "frameyap/runtimes"), base_python=sys.executable, + base_python_version=platform.python_version(), + packages=[RUNTIME_TORCH + " from " + RUNTIME_TORCH_INDEX, RUNTIME_REQUIREMENT + " from PyPI"], + download=RUNTIME_DOWNLOAD, paths_config=str(paths_config_path())) if args.json: print(json.dumps({"ok": True, "event": "plan", **result}, sort_keys=True)) else: @@ -976,6 +1069,8 @@ def main(argv=None): print(f"Rolled back to {previous}") elif args.install_model: install_model(args, root) + elif args.install_runtime: + install_runtime(args, root) else: do_install(args, root, launcher) # The native registration helper takes this same install lock. Never run it @@ -1045,7 +1140,7 @@ def cli(argv=None): try: if structured and "--print-plan" in argv: main(argv) # already emits one JSON plan - elif structured and "--install-model" in argv: + elif structured and ("--install-model" in argv or "--install-runtime" in argv): main(argv) # streaming per-file JSON events for a UI child print(json.dumps({"ok": True, "event": "complete"})) elif structured: diff --git a/tests/test_installer.py b/tests/test_installer.py index d29c400..cd880ee 100644 --- a/tests/test_installer.py +++ b/tests/test_installer.py @@ -1113,6 +1113,92 @@ with patch.object(module, "check_host"), patch.object(module.urllib.request, "ur self.assertIn("mismatched", json.loads(output.getvalue())["message"]) fetch.assert_not_called() + def test_runtime_install_is_explicit_cpu_pinned_and_updates_only_python_path(self): + # subprocess is mocked: no venv, pip or network is touched. + root = self.data / "frameyap" + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(installer.cli(["--install-runtime", "--print-plan", "--json"]), 0) + plan = json.loads(output.getvalue()) + self.assertEqual((plan["operation"], plan["network"]), ("install-runtime", True)) + self.assertIn("torch==2.8.0 from https://download.pytorch.org/whl/cpu", plan["packages"]) + self.assertFalse(root.exists()) + for extra in (["--version", "0.1.202609241530"], ["--backend", "other"], ["--without-model"]): + output = io.StringIO() + with contextlib.redirect_stdout(output): + self.assertEqual(installer.cli(["--install-runtime", "--json"] + extra), 2) + output = io.StringIO() + with contextlib.redirect_stdout(output), patch.object(installer.subprocess, "run") as run: + self.assertEqual(installer.cli(["--install-runtime", "--json"]), 1) + run.assert_not_called() + self.assertIn("--yes", json.loads(output.getvalue())["message"]) + self.assertFalse((root / "runtimes").exists()) + + config = self.home / ".config/frameyap/paths.conf" + config.parent.mkdir(parents=True) + config.write_text("# mine\npython=/opt/old/bin/python3\nmodel=/opt/model\n") + old = root / "runtimes/cpu-20260101000000" + old.mkdir(parents=True) + foreign = root / "runtimes/keep-me" + foreign.mkdir() + commands = [] + + def fake(command, **kwargs): + commands.append(command) + if command[1:3] == ["-m", "venv"]: + python = Path(command[3]) / "bin/python3" + python.parent.mkdir(parents=True) + python.write_text("") + return SimpleNamespace(returncode=0) + + with patch.object(installer, "RUNTIME_PYTHON_RANGE", ((3, 0), (3, 1))), \ + patch.object(installer.subprocess, "run") as run: + with contextlib.redirect_stdout(io.StringIO()) as unsupported: + self.assertEqual(installer.cli(["--install-runtime", "--yes", "--json"]), 1) + self.assertIn("Python 3.10-3.13", json.loads(unsupported.getvalue())["message"]) + run.assert_not_called() + self.env_range = patch.object(installer, "RUNTIME_PYTHON_RANGE", ((3, 0), (4, 0))) + self.env_range.start() + self.addCleanup(self.env_range.stop) + output = io.StringIO() + with contextlib.redirect_stdout(output), patch.object(installer.subprocess, "run", side_effect=fake): + self.assertEqual(installer.cli(["--install-runtime", "--yes", "--json"]), 0) + events = [json.loads(line) for line in output.getvalue().splitlines()] + self.assertEqual([e.get("step") for e in events if e["event"] == "runtime_step"], + ["create virtual environment", "install CPU Torch", "install moondream/Kestrel", + "verify CPU runtime imports"]) + self.assertEqual(events[-1]["event"], "complete") + torch, packages = commands[1], commands[2] + for command in (torch, packages): + self.assertIn("--isolated", command) + self.assertIn("--only-binary=:all:", command) + self.assertEqual(torch[-3:], ["--index-url", "https://download.pytorch.org/whl/cpu", "torch==2.8.0"]) + self.assertEqual(packages[-1], "moondream==2.4.0") + self.assertIn("--constraint", packages) + runtimes = [item for item in (root / "runtimes").iterdir() if item.name.startswith("cpu-")] + self.assertEqual(len(runtimes), 1) + self.assertNotEqual(runtimes[0], old) + self.assertTrue(foreign.is_dir()) + self.assertEqual(config.read_text(), + f"# mine\nmodel=/opt/model\npython={runtimes[0]}/bin/python3\n") + + # A failed pip step removes only its own new venv and leaves paths.conf alone. + before = config.read_text() + + def failing(command, **kwargs): + if "moondream==2.4.0" in command: + raise subprocess.CalledProcessError(1, command) + return fake(command, **kwargs) + + output = io.StringIO() + with contextlib.redirect_stdout(output), patch.object(installer.subprocess, "run", side_effect=failing), \ + patch.object(installer, "datetime") as clock: + clock.now.return_value.strftime.return_value = "cpu-20991231235959" + self.assertEqual(installer.cli(["--install-runtime", "--yes", "--json"]), 1) + self.assertFalse((root / "runtimes/cpu-20991231235959").exists()) + self.assertTrue(runtimes[0].is_dir()) + self.assertEqual(config.read_text(), before) + if __name__ == "__main__": unittest.main()