mirror of
https://github.com/baketnk/frame-yap.git
synced 2026-10-06 01:00:04 +02:00
installer: add explicit --install-runtime for the pinned CPU Python runtime (D4)
Creates a user-local venv under $XDG_DATA_HOME/frameyap/runtimes from the invoking python3 (3.10-3.13), installs CPU torch==2.8.0 from PyTorch's CPU index and moondream==2.4.0 from PyPI (binary wheels only, pip --isolated, Torch constrained), verifies imports and no CUDA, then sets python= in paths.conf and removes superseded managed runtimes. Requires --yes; has a read-only --print-plan/--json. A failure removes only the new venv. Offline tests mock pip; a real run passed on x86-64 Python 3.12. ARM64 Frame install is still pending. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
70ea838a59
commit
e5b998eb6c
9 files changed
+334
-38
No files matched your search
@@ -7,7 +7,7 @@ Standalone OpenVR overlay; no Steam store AppID or sudo. First v0.1 release is i
|
||||
## Requirements
|
||||
|
||||
- Steam Frame with usable SteamVR/OpenVR and Gamescope for the **native** overlay and text delivery; Linux ARM64/glibc for the current installer payload format.
|
||||
- For voice recognition, separately provision a compatible **CPU Python runtime** (moondream 2.4.0 / Kestrel 0.8.0 and dependencies) and the pinned local Parakeet Redux model. Neither is bundled or installed with pip by the current native-only installer. There is no fallback ASR service.
|
||||
- For voice recognition, a **CPU Python runtime** (moondream 2.4.0 / Kestrel 0.8.0, CPU Torch 2.8.0) and the pinned local Parakeet Redux model. Neither is bundled; each has its own explicit install step: `sh install.sh --install-model --backend redux --yes` and `sh install.sh --install-runtime --yes` (needs Python 3.10–3.13 with venv; about 200 MB download). Use `--print-plan` first to see exactly what each fetches. There is no fallback ASR service.
|
||||
- A local source build needs CMake 3.20+, C++20 and explicit native libraries/SDK; the default hardware-free build needs only CMake and C++20. See [build requirements](docs/build.md).
|
||||
|
||||
## Install (local artifacts only)
|
||||
|
||||
@@ -50,8 +50,8 @@ accepted; local code/tests cannot establish a fixed delivery regression.
|
||||
|
||||
## A. First release (v0.1) blockers
|
||||
|
||||
Remaining release gates (2026-09-25): implement D4 (pip runtime install), then a
|
||||
fresh clean install on Frame that also checks the missing-runtime panel message
|
||||
Remaining release gates (2026-09-25): D4 is implemented; next a
|
||||
fresh clean install on Frame (install → `--install-model` → `--install-runtime`) that also checks the missing-runtime panel message
|
||||
(H), then tag `v0.1.<timestamp>` and update the README install section.
|
||||
|
||||
- [x] **A8. UI polish batch (local; deploy pending).** Removed the fixed
|
||||
@@ -132,10 +132,12 @@ fresh clean install on Frame that also checks the missing-runtime panel message
|
||||
fixtures only, not a released archive or an installed Frame UI handoff.
|
||||
- [ ] **D3. Publish a prebuilt ARM64 archive.** (M) **Deferred.** v0.1 is
|
||||
source-only. Revisit after source-build acceptance.
|
||||
- [ ] **D4. Source install fetches the Python runtime with pip.** (M) Proposed, not
|
||||
implemented: the installer creates a user-local venv and installs pinned
|
||||
moondream/Kestrel with the CPU Torch wheel, on an explicit flag/confirmation.
|
||||
The installer does not run pip today.
|
||||
- [x] **D4. Installer fetches the Python runtime with pip.** (M) Implemented
|
||||
2026-09-25 as explicit `install.sh --install-runtime --yes`: user-local venv,
|
||||
CPU `torch==2.8.0` from PyTorch's CPU index, `moondream==2.4.0` from PyPI,
|
||||
import/no-CUDA verification, then `python=` in `paths.conf`. Offline tests
|
||||
mock pip; a real run passed on x86-64 Python 3.12. Needs the clean Frame
|
||||
install to confirm on ARM64.
|
||||
|
||||
## E. Naming and versioning
|
||||
|
||||
@@ -209,5 +211,6 @@ this app is future design and out of scope for v0.1.
|
||||
|
||||
## Open questions
|
||||
|
||||
Source-install runtime provisioning (D4), P1 real-target behavior
|
||||
and live headset validation are unresolved gates, not implied by checked source tasks.
|
||||
The clean-account Frame install (including D4's ARM64 pip run and the
|
||||
missing-runtime message) is the remaining unresolved gate; checked source tasks
|
||||
do not imply it.
|
||||
+4
-2
@@ -63,7 +63,8 @@ initialize OpenVR, open a microphone, run ASR, download files or inject input.
|
||||
source-build mode, safe extraction, atomic current-version selection, retained
|
||||
rollback, runtime/install lock, foreign-file refusal and explicit
|
||||
unregister-before-uninstall acknowledgement. Source mode needs a local
|
||||
compiler, SDK and libraries; the installer does not yet pip-install a runtime.
|
||||
compiler, SDK and libraries. An explicit `--install-runtime --yes` step
|
||||
pip-installs the pinned CPU Python runtime into a user-local venv.
|
||||
|
||||
## Deliberately not claimed
|
||||
|
||||
@@ -103,7 +104,8 @@ compatible offline runtime and independent tests. Native `--run` flags `--backen
|
||||
`--manifest-dir /absolute/manifests` are wired through
|
||||
the installed launcher as an explicit override, not a provisioning command.
|
||||
Inference uses the `moondream` Python package and Kestrel runtime, separately
|
||||
provisioned in your own environment; builds/tests/installer do not pip-install them. See
|
||||
provisioned in your own environment or by the explicit `install.sh --install-runtime --yes`;
|
||||
builds and tests never pip-install them. See
|
||||
[third-party notes](third-party.md).
|
||||
|
||||
## Developer native build
|
||||
|
||||
@@ -4,9 +4,10 @@ Goal: a one-command, pinned GitHub release install for Steam Frame without a Ste
|
||||
store AppID, sudo or end-user compiler. **No public archive or verified clean
|
||||
install is published. Do not advertise a `curl | sh` command as functional.**
|
||||
The local installer has binary-archive and explicitly provisioned source-build
|
||||
modes, machine-readable plans/results and an attended TTY path. The current
|
||||
native-only artifact does not include or pip-install an ASR runtime; it is not
|
||||
a one-command voice-typing experience. See [packaging](packaging.md) for exact
|
||||
modes, machine-readable plans/results and an attended TTY path. The native-only
|
||||
artifact does not include an ASR runtime; a separate explicit
|
||||
`--install-runtime --yes` step pip-installs the pinned CPU runtime into a
|
||||
user-local venv. Install, model and runtime are still three commands. See [packaging](packaging.md) for exact
|
||||
flags and [third-party notes](third-party.md).
|
||||
|
||||
`scripts/install-preflight.sh` is a read-only Linux ARM64/glibc/bootstrap check;
|
||||
@@ -52,7 +53,7 @@ already be available for registration; never start/restart it for installation.
|
||||
second **Confirm Install** click; the installed/native UI route still needs
|
||||
clean-target and headset acceptance. `--without-model` permits an
|
||||
archive install without bundled model files. Neither operation installs Torch,
|
||||
moondream, Kestrel or an interpreter. Launch paths to an independently
|
||||
moondream or Kestrel; only `--install-runtime --yes` does (see packaging). Launch paths to an independently
|
||||
authorized runtime/model can be set in `paths.conf`.
|
||||
- **Noninteractive**: supply flags and `--yes` for network/model consent.
|
||||
`--print-plan` is read-only; `--json` provides structured results/errors and
|
||||
@@ -66,7 +67,7 @@ listing, unrelated application dependency, microphone recording, input injection
|
||||
or automatic update daemon. Hashes detect accidental/unauthorized alteration
|
||||
of a downloaded artifact but do not authenticate a compromised publisher;
|
||||
release metadata needs independent trust. No installer operation silently runs
|
||||
pip or launches inference. Native-only archives support overlay checks but need
|
||||
pip or launches inference; pip runs only under explicit `--install-runtime --yes`. Native-only archives support overlay checks but need
|
||||
an externally provisioned runtime/weights before voice typing. Selecting an
|
||||
uninstalled backend does not authorize a download or supply its inference engine.
|
||||
|
||||
|
||||
+15
-2
@@ -102,8 +102,21 @@ installed `redux.json` bytes: under the model lock a mismatch fails **before**
|
||||
a model directory is created or any network request. In the locally implemented
|
||||
Models UI, Install displays source, rounded size, license text, attribution and
|
||||
the fingerprint; only the second Confirm Install click passes that fingerprint
|
||||
through the backend helper to the installer. Installation alone does not
|
||||
through the backend helper to the installer. Model installation does not
|
||||
provision a CPU Python runtime; no in-panel flow has been accepted on Frame.
|
||||
|
||||
`sh install.sh --install-runtime --yes` explicitly creates a user-local venv
|
||||
under `$XDG_DATA_HOME/frameyap/runtimes/` from the invoking `python3` (3.10–3.13),
|
||||
pip-installs `torch==2.8.0` from PyTorch's CPU index, then `moondream==2.4.0`
|
||||
(which pins kestrel 0.8.0, kestrel-native 0.1.8, kestrel-kernels 0.7.0) from PyPI
|
||||
with Torch constrained, binary wheels only and `pip --isolated`. It verifies the
|
||||
imports and that Torch has no CUDA, then sets `python=` in `paths.conf` (other
|
||||
lines kept) and removes superseded FrameYap-managed runtimes. A failure removes
|
||||
only the new venv and leaves `paths.conf` unchanged. `--print-plan --json` shows
|
||||
the packages, index and size (about 200 MB of wheels, roughly 1–1.5 GB on disk)
|
||||
without touching anything. The app must be closed (install lock). Transitive
|
||||
dependencies are not pinned. Validated end to end on x86-64 Python 3.12
|
||||
(2026-09-25); a clean Frame install is still pending.
|
||||
In the source tree, `python3 scripts/model-status.py --list-models` or
|
||||
`--check-model redux --model-dir /absolute/model` hash-checks local files;
|
||||
the native `frameyap --list-models` / `--check-model` entry points use the
|
||||
@@ -151,7 +164,7 @@ model=/absolute/path/to/pinned/local/model
|
||||
The launcher reads these as **literal absolute paths**, not shell code, and does
|
||||
not follow a symlink to the config file. Environment variables override either
|
||||
setting for an explicit launch. This `paths.conf` survives upgrade/uninstall;
|
||||
the installer does not populate it or bundle an unauthorized runtime. No
|
||||
only `--install-runtime` writes its `python=` line, and no runtime is bundled. No
|
||||
pip/bootstrap/model download or fallback is invoked by the launcher. Without
|
||||
paths, the native-only artifact cannot perform voice inference; its default
|
||||
runtime and model locations do not exist. A new installer accepts only the exact
|
||||
|
||||
+3
-2
@@ -80,8 +80,9 @@ flags do not prove every third-party internal is unable to access a network.
|
||||
Runtime/build/tests perform no downloads; the separate explicit setup utility
|
||||
`scripts/fetch-model.py` can provision the public pinned weights.
|
||||
|
||||
Current native-only archives do not bundle or pip-install the runtime; see
|
||||
[third-party notes](third-party.md). No public runtime bundle has been released.
|
||||
Native-only archives do not bundle the runtime; the explicit
|
||||
`install.sh --install-runtime --yes` step pip-installs it into a user-local venv
|
||||
(see [packaging](packaging.md) and [third-party notes](third-party.md)). No public runtime bundle has been released.
|
||||
Limited ARM64 measurements were taken during development; they are not a claim
|
||||
of complete headset acceptance.
|
||||
|
||||
|
||||
+104
-9
@@ -70,6 +70,20 @@ CONFIG_DEFAULTS = {
|
||||
"insert": "/user/hand/right/input/a", "enter": "",
|
||||
"quick_chat": "/user/hand/right/input/y"},
|
||||
}
|
||||
# Explicit --install-runtime only. CPU Torch comes from PyTorch's CPU index first;
|
||||
# an unconstrained PyPI resolve selects CUDA/NVIDIA wheels. moondream 2.4.0 pins
|
||||
# kestrel 0.8.0, kestrel-native 0.1.8 and kestrel-kernels 0.7.0.
|
||||
RUNTIME_TORCH = "torch==2.8.0"
|
||||
RUNTIME_TORCH_INDEX = "https://download.pytorch.org/whl/cpu"
|
||||
RUNTIME_REQUIREMENT = "moondream==2.4.0"
|
||||
RUNTIME_DOWNLOAD = "about 200 MB of prebuilt wheels (CPU Torch is about 100 MB), roughly 1-1.5 GB on disk; no compilation"
|
||||
RUNTIME_PYTHON_RANGE = ((3, 10), (3, 14)) # moondream allows <3.15; CPU Torch 2.8.0 wheels stop at 3.13
|
||||
RUNTIME_NAME_RE = re.compile(r"cpu-[0-9]{14}\Z")
|
||||
RUNTIME_VERIFY = (
|
||||
"import importlib.metadata as m, torch\n"
|
||||
"assert torch.version.cuda is None, 'CUDA Torch was installed; CPU build required'\n"
|
||||
"import moondream\n"
|
||||
"print('moondream', m.version('moondream'), 'kestrel', m.version('kestrel'), 'torch', torch.__version__)\n")
|
||||
COLOR_RE = re.compile(r"#[0-9a-fA-F]{6}\Z")
|
||||
BUTTON_RE = re.compile(r"/user/hand/(left|right)/input/[A-Za-z0-9_]+\Z")
|
||||
BACKEND_RE = re.compile(r"[a-z][a-z0-9_-]{0,47}\Z", re.ASCII)
|
||||
@@ -103,6 +117,26 @@ def config_path():
|
||||
return base / "frameyap/config.json"
|
||||
|
||||
|
||||
def paths_config_path():
|
||||
return config_path().parent / "paths.conf"
|
||||
|
||||
|
||||
def set_runtime_python(python):
|
||||
"""Point the managed launcher at `python`; other paths.conf lines are kept verbatim."""
|
||||
path = paths_config_path()
|
||||
if "\n" in str(python) or not python.is_absolute():
|
||||
fail(f"unsafe runtime path: {python!r}")
|
||||
owned_dir(path.parent)
|
||||
if path.is_symlink():
|
||||
fail(f"refusing symlink paths config: {path}")
|
||||
lines = (path.read_text().splitlines() if path.exists() else
|
||||
["# FrameYap literal paths (python=/abs, model=/abs); never shell code"])
|
||||
previous = [line[len("python="):] for line in lines if line.startswith("python=")]
|
||||
kept = [line for line in lines if not line.startswith("python=")]
|
||||
atomic_write(path, ("\n".join(kept + [f"python={python}"]) + "\n").encode())
|
||||
return previous[-1] if previous else None
|
||||
|
||||
|
||||
def unique_pairs(pairs):
|
||||
result = {}
|
||||
for key, value in pairs:
|
||||
@@ -581,11 +615,11 @@ def model_target(args, root):
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def model_download_cancellation():
|
||||
def model_download_cancellation(message="model installation cancelled"):
|
||||
# Install-model runs on the main thread. Raising unwinds the owned temp's
|
||||
# finally block; unlike SIGKILL this does not leave a partial download.
|
||||
def terminate(_signal, _frame):
|
||||
raise ValueError("model installation cancelled")
|
||||
raise ValueError(message)
|
||||
|
||||
previous = signal.signal(signal.SIGTERM, terminate)
|
||||
try:
|
||||
@@ -652,6 +686,52 @@ def install_model(args, root):
|
||||
print(f"Backend {backend.id} model verified at {dest}; license {backend.license_id}. Configure runtime/model paths explicitly for launch.")
|
||||
|
||||
|
||||
def runtime_step(args, name, command):
|
||||
if args.json:
|
||||
print(json.dumps({"ok": True, "event": "runtime_step", "step": name}), flush=True)
|
||||
else:
|
||||
print(f"[runtime] {name}", flush=True)
|
||||
# Tool output goes to stderr so --json stdout stays one event per line.
|
||||
subprocess.run(command, check=True, stdout=sys.stderr, stdin=subprocess.DEVNULL)
|
||||
|
||||
|
||||
def install_runtime(args, root):
|
||||
low, high = RUNTIME_PYTHON_RANGE
|
||||
if not low <= sys.version_info[:2] < high:
|
||||
fail("the runtime needs Python 3.10-3.13 (CPU Torch 2.8.0 wheels); found " + platform.python_version())
|
||||
if importlib.util.find_spec("venv") is None or importlib.util.find_spec("ensurepip") is None:
|
||||
fail("python3 cannot create a virtual environment (venv/ensurepip missing); no packages installed")
|
||||
runtimes = root / "runtimes"
|
||||
owned_dir(runtimes)
|
||||
target = runtimes / datetime.now().strftime("cpu-%Y%m%d%H%M%S")
|
||||
if target.exists() or target.is_symlink():
|
||||
fail(f"runtime directory already exists: {target}")
|
||||
python = target / "bin/python3"
|
||||
# --isolated ignores user pip config/env indexes; binary-only avoids compilers.
|
||||
pip = [str(python), "-m", "pip", "--isolated", "--disable-pip-version-check", "--no-input",
|
||||
"install", "--only-binary=:all:"]
|
||||
try:
|
||||
with model_download_cancellation("runtime installation cancelled"):
|
||||
runtime_step(args, "create virtual environment", [sys.executable, "-m", "venv", str(target)])
|
||||
runtime_step(args, "install CPU Torch", pip + ["--index-url", RUNTIME_TORCH_INDEX, RUNTIME_TORCH])
|
||||
constraints = target / "frameyap-constraints.txt"
|
||||
constraints.write_text(RUNTIME_TORCH + "\n")
|
||||
runtime_step(args, "install moondream/Kestrel", pip + ["--constraint", str(constraints), RUNTIME_REQUIREMENT])
|
||||
runtime_step(args, "verify CPU runtime imports", [str(python), "-c", RUNTIME_VERIFY])
|
||||
except BaseException:
|
||||
shutil.rmtree(target, ignore_errors=True)
|
||||
raise
|
||||
previous = set_runtime_python(python)
|
||||
# Superseded managed runtimes are removed; anything else is never touched.
|
||||
for item in runtimes.iterdir():
|
||||
if item != target and item.is_dir() and not item.is_symlink() and RUNTIME_NAME_RE.fullmatch(item.name):
|
||||
shutil.rmtree(item)
|
||||
if not args.json:
|
||||
print(f"CPU Python runtime installed at {target}; {paths_config_path()} now has python={python}.")
|
||||
if previous and not previous.startswith(str(runtimes) + "/"):
|
||||
print(f"Previous python={previous} was replaced; restore it in paths.conf to switch back.")
|
||||
|
||||
|
||||
def source_preflight(args):
|
||||
"""Read-only checks before the installer creates its install root."""
|
||||
source = Path(args.source).expanduser().resolve(strict=True)
|
||||
@@ -798,7 +878,7 @@ def do_install(args, root, launcher):
|
||||
select(root, "current", f"versions/{version}")
|
||||
print(f"FrameYap {version} installed. OpenVR registration is NOT automatic; see docs/packaging.md.")
|
||||
if json.loads((target / "release.json").read_text())["runtime"] == "external-authorized-python":
|
||||
print("ASR runtime is NOT included. Supply an independently authorized environment with FRAMEYAP_PYTHON or --python; no packages are downloaded.")
|
||||
print("ASR runtime is NOT included. Run install.sh --install-runtime --yes to pip-install it, or supply one with FRAMEYAP_PYTHON/--python or paths.conf.")
|
||||
|
||||
|
||||
def uninstall(root, launcher):
|
||||
@@ -840,7 +920,8 @@ def uninstall(root, launcher):
|
||||
manifest.unlink()
|
||||
if desktop.exists():
|
||||
desktop.unlink()
|
||||
print("FrameYap removed; config and saved models preserved. OpenVR unregister acknowledgement was required.")
|
||||
print("FrameYap removed; config, saved models and any runtimes/ directory preserved. "
|
||||
"OpenVR unregister acknowledgement was required.")
|
||||
|
||||
|
||||
class UsageError(ValueError):
|
||||
@@ -859,6 +940,8 @@ def resolve_args(argv):
|
||||
action.add_argument("--rollback", action="store_true")
|
||||
action.add_argument("--uninstall", action="store_true")
|
||||
action.add_argument("--install-model", action="store_true", help="explicit model provisioning for installed backend")
|
||||
action.add_argument("--install-runtime", action="store_true",
|
||||
help="explicit pip install of the pinned CPU Python runtime into a user-local venv")
|
||||
parser.add_argument("--mode", choices=("binary", "source"), default="binary")
|
||||
parser.add_argument("--source", help="explicit local FrameYap source tree (source mode only)")
|
||||
parser.add_argument("--openvr-root", help="local OpenVR SDK root (source mode)")
|
||||
@@ -884,7 +967,7 @@ def resolve_args(argv):
|
||||
if args.mode == "source":
|
||||
if not args.source or any(not getattr(args, name) for name in source_inputs):
|
||||
parser.error("--mode source requires --source and explicit --openvr-root, --openvr-library, --openvr-license, --sdl-library, --sdl-license")
|
||||
if args.archive or args.sha256 or args.rollback or args.uninstall or args.install_model:
|
||||
if args.archive or args.sha256 or args.rollback or args.uninstall or args.install_model or args.install_runtime:
|
||||
parser.error("source mode cannot combine with binary archive or lifecycle operations")
|
||||
elif args.source or any(getattr(args, name) for name in source_inputs):
|
||||
parser.error("source inputs require --mode source")
|
||||
@@ -894,7 +977,10 @@ def resolve_args(argv):
|
||||
parser.error("--sha256 must be a 64-character hex SHA-256")
|
||||
if not args.archive and args.sha256:
|
||||
parser.error("--sha256 only applies to --archive")
|
||||
if not (args.rollback or args.uninstall or args.install_model) and not args.version:
|
||||
if args.install_runtime and (args.version or args.without_model or args.autolaunch is not None or
|
||||
args.backend != "redux" or args.model_dir or args.expected_manifest_sha256):
|
||||
parser.error("--install-runtime is independent of release/model flags")
|
||||
if not (args.rollback or args.uninstall or args.install_model or args.install_runtime) and not args.version:
|
||||
parser.error("--version VERSION is required; no moving/latest release")
|
||||
if args.uninstall != args.unregistered:
|
||||
parser.error("--uninstall requires --unregistered after explicit OpenVR unregister")
|
||||
@@ -923,7 +1009,7 @@ def resolve_args(argv):
|
||||
|
||||
def plan(args):
|
||||
operation = ("uninstall" if args.uninstall else "rollback" if args.rollback else
|
||||
"install-model" if args.install_model else "install")
|
||||
"install-model" if args.install_model else "install-runtime" if args.install_runtime else "install")
|
||||
return {"operation": operation, "mode": args.mode, "version": args.version,
|
||||
"tag": "v" + args.version if args.version else None,
|
||||
"archive": str(args.archive) if args.archive else None,
|
||||
@@ -931,7 +1017,8 @@ def plan(args):
|
||||
"backend": args.backend, "model_dir": str(args.model_dir) if args.model_dir else None,
|
||||
"expected_manifest_sha256": (args.expected_manifest_sha256.lower() if args.expected_manifest_sha256 else None),
|
||||
"without_model": args.without_model, "autolaunch": args.autolaunch,
|
||||
"network": bool(args.install_model or (operation == "install" and not args.archive and not args.source)),
|
||||
"network": bool(args.install_model or args.install_runtime or
|
||||
(operation == "install" and not args.archive and not args.source)),
|
||||
"registration": ("explicit --register --autostart" if args.autolaunch is True else
|
||||
"explicit --register (autostart off)" if args.autolaunch is False else "none")}
|
||||
|
||||
@@ -947,6 +1034,12 @@ def main(argv=None):
|
||||
check_expected_manifest(args, manifest_sha)
|
||||
result.update(model_dir=str(model_target(args, root)), model=backend.description(),
|
||||
installed_manifest_sha256=manifest_sha)
|
||||
if args.install_runtime:
|
||||
data = Path(os.environ.get("XDG_DATA_HOME") or Path.home() / ".local/share").expanduser().absolute()
|
||||
result.update(runtime_dir=str(data / "frameyap/runtimes"), base_python=sys.executable,
|
||||
base_python_version=platform.python_version(),
|
||||
packages=[RUNTIME_TORCH + " from " + RUNTIME_TORCH_INDEX, RUNTIME_REQUIREMENT + " from PyPI"],
|
||||
download=RUNTIME_DOWNLOAD, paths_config=str(paths_config_path()))
|
||||
if args.json:
|
||||
print(json.dumps({"ok": True, "event": "plan", **result}, sort_keys=True))
|
||||
else:
|
||||
@@ -996,6 +1089,8 @@ def main(argv=None):
|
||||
print(f"Rolled back to {previous}")
|
||||
elif args.install_model:
|
||||
install_model(args, root)
|
||||
elif args.install_runtime:
|
||||
install_runtime(args, root)
|
||||
else:
|
||||
do_install(args, root, launcher)
|
||||
# The native registration helper takes this same install lock. Never run it
|
||||
@@ -1065,7 +1160,7 @@ def cli(argv=None):
|
||||
try:
|
||||
if structured and "--print-plan" in argv:
|
||||
main(argv) # already emits one JSON plan
|
||||
elif structured and "--install-model" in argv:
|
||||
elif structured and ("--install-model" in argv or "--install-runtime" in argv):
|
||||
main(argv) # streaming per-file JSON events for a UI child
|
||||
print(json.dumps({"ok": True, "event": "complete"}))
|
||||
elif structured:
|
||||
|
||||
+104
-9
@@ -50,6 +50,20 @@ CONFIG_DEFAULTS = {
|
||||
"insert": "/user/hand/right/input/a", "enter": "",
|
||||
"quick_chat": "/user/hand/right/input/y"},
|
||||
}
|
||||
# Explicit --install-runtime only. CPU Torch comes from PyTorch's CPU index first;
|
||||
# an unconstrained PyPI resolve selects CUDA/NVIDIA wheels. moondream 2.4.0 pins
|
||||
# kestrel 0.8.0, kestrel-native 0.1.8 and kestrel-kernels 0.7.0.
|
||||
RUNTIME_TORCH = "torch==2.8.0"
|
||||
RUNTIME_TORCH_INDEX = "https://download.pytorch.org/whl/cpu"
|
||||
RUNTIME_REQUIREMENT = "moondream==2.4.0"
|
||||
RUNTIME_DOWNLOAD = "about 200 MB of prebuilt wheels (CPU Torch is about 100 MB), roughly 1-1.5 GB on disk; no compilation"
|
||||
RUNTIME_PYTHON_RANGE = ((3, 10), (3, 14)) # moondream allows <3.15; CPU Torch 2.8.0 wheels stop at 3.13
|
||||
RUNTIME_NAME_RE = re.compile(r"cpu-[0-9]{14}\Z")
|
||||
RUNTIME_VERIFY = (
|
||||
"import importlib.metadata as m, torch\n"
|
||||
"assert torch.version.cuda is None, 'CUDA Torch was installed; CPU build required'\n"
|
||||
"import moondream\n"
|
||||
"print('moondream', m.version('moondream'), 'kestrel', m.version('kestrel'), 'torch', torch.__version__)\n")
|
||||
COLOR_RE = re.compile(r"#[0-9a-fA-F]{6}\Z")
|
||||
BUTTON_RE = re.compile(r"/user/hand/(left|right)/input/[A-Za-z0-9_]+\Z")
|
||||
BACKEND_RE = re.compile(r"[a-z][a-z0-9_-]{0,47}\Z", re.ASCII)
|
||||
@@ -83,6 +97,26 @@ def config_path():
|
||||
return base / "frameyap/config.json"
|
||||
|
||||
|
||||
def paths_config_path():
|
||||
return config_path().parent / "paths.conf"
|
||||
|
||||
|
||||
def set_runtime_python(python):
|
||||
"""Point the managed launcher at `python`; other paths.conf lines are kept verbatim."""
|
||||
path = paths_config_path()
|
||||
if "\n" in str(python) or not python.is_absolute():
|
||||
fail(f"unsafe runtime path: {python!r}")
|
||||
owned_dir(path.parent)
|
||||
if path.is_symlink():
|
||||
fail(f"refusing symlink paths config: {path}")
|
||||
lines = (path.read_text().splitlines() if path.exists() else
|
||||
["# FrameYap literal paths (python=/abs, model=/abs); never shell code"])
|
||||
previous = [line[len("python="):] for line in lines if line.startswith("python=")]
|
||||
kept = [line for line in lines if not line.startswith("python=")]
|
||||
atomic_write(path, ("\n".join(kept + [f"python={python}"]) + "\n").encode())
|
||||
return previous[-1] if previous else None
|
||||
|
||||
|
||||
def unique_pairs(pairs):
|
||||
result = {}
|
||||
for key, value in pairs:
|
||||
@@ -561,11 +595,11 @@ def model_target(args, root):
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def model_download_cancellation():
|
||||
def model_download_cancellation(message="model installation cancelled"):
|
||||
# Install-model runs on the main thread. Raising unwinds the owned temp's
|
||||
# finally block; unlike SIGKILL this does not leave a partial download.
|
||||
def terminate(_signal, _frame):
|
||||
raise ValueError("model installation cancelled")
|
||||
raise ValueError(message)
|
||||
|
||||
previous = signal.signal(signal.SIGTERM, terminate)
|
||||
try:
|
||||
@@ -632,6 +666,52 @@ def install_model(args, root):
|
||||
print(f"Backend {backend.id} model verified at {dest}; license {backend.license_id}. Configure runtime/model paths explicitly for launch.")
|
||||
|
||||
|
||||
def runtime_step(args, name, command):
|
||||
if args.json:
|
||||
print(json.dumps({"ok": True, "event": "runtime_step", "step": name}), flush=True)
|
||||
else:
|
||||
print(f"[runtime] {name}", flush=True)
|
||||
# Tool output goes to stderr so --json stdout stays one event per line.
|
||||
subprocess.run(command, check=True, stdout=sys.stderr, stdin=subprocess.DEVNULL)
|
||||
|
||||
|
||||
def install_runtime(args, root):
|
||||
low, high = RUNTIME_PYTHON_RANGE
|
||||
if not low <= sys.version_info[:2] < high:
|
||||
fail("the runtime needs Python 3.10-3.13 (CPU Torch 2.8.0 wheels); found " + platform.python_version())
|
||||
if importlib.util.find_spec("venv") is None or importlib.util.find_spec("ensurepip") is None:
|
||||
fail("python3 cannot create a virtual environment (venv/ensurepip missing); no packages installed")
|
||||
runtimes = root / "runtimes"
|
||||
owned_dir(runtimes)
|
||||
target = runtimes / datetime.now().strftime("cpu-%Y%m%d%H%M%S")
|
||||
if target.exists() or target.is_symlink():
|
||||
fail(f"runtime directory already exists: {target}")
|
||||
python = target / "bin/python3"
|
||||
# --isolated ignores user pip config/env indexes; binary-only avoids compilers.
|
||||
pip = [str(python), "-m", "pip", "--isolated", "--disable-pip-version-check", "--no-input",
|
||||
"install", "--only-binary=:all:"]
|
||||
try:
|
||||
with model_download_cancellation("runtime installation cancelled"):
|
||||
runtime_step(args, "create virtual environment", [sys.executable, "-m", "venv", str(target)])
|
||||
runtime_step(args, "install CPU Torch", pip + ["--index-url", RUNTIME_TORCH_INDEX, RUNTIME_TORCH])
|
||||
constraints = target / "frameyap-constraints.txt"
|
||||
constraints.write_text(RUNTIME_TORCH + "\n")
|
||||
runtime_step(args, "install moondream/Kestrel", pip + ["--constraint", str(constraints), RUNTIME_REQUIREMENT])
|
||||
runtime_step(args, "verify CPU runtime imports", [str(python), "-c", RUNTIME_VERIFY])
|
||||
except BaseException:
|
||||
shutil.rmtree(target, ignore_errors=True)
|
||||
raise
|
||||
previous = set_runtime_python(python)
|
||||
# Superseded managed runtimes are removed; anything else is never touched.
|
||||
for item in runtimes.iterdir():
|
||||
if item != target and item.is_dir() and not item.is_symlink() and RUNTIME_NAME_RE.fullmatch(item.name):
|
||||
shutil.rmtree(item)
|
||||
if not args.json:
|
||||
print(f"CPU Python runtime installed at {target}; {paths_config_path()} now has python={python}.")
|
||||
if previous and not previous.startswith(str(runtimes) + "/"):
|
||||
print(f"Previous python={previous} was replaced; restore it in paths.conf to switch back.")
|
||||
|
||||
|
||||
def source_preflight(args):
|
||||
"""Read-only checks before the installer creates its install root."""
|
||||
source = Path(args.source).expanduser().resolve(strict=True)
|
||||
@@ -778,7 +858,7 @@ def do_install(args, root, launcher):
|
||||
select(root, "current", f"versions/{version}")
|
||||
print(f"FrameYap {version} installed. OpenVR registration is NOT automatic; see docs/packaging.md.")
|
||||
if json.loads((target / "release.json").read_text())["runtime"] == "external-authorized-python":
|
||||
print("ASR runtime is NOT included. Supply an independently authorized environment with FRAMEYAP_PYTHON or --python; no packages are downloaded.")
|
||||
print("ASR runtime is NOT included. Run install.sh --install-runtime --yes to pip-install it, or supply one with FRAMEYAP_PYTHON/--python or paths.conf.")
|
||||
|
||||
|
||||
def uninstall(root, launcher):
|
||||
@@ -820,7 +900,8 @@ def uninstall(root, launcher):
|
||||
manifest.unlink()
|
||||
if desktop.exists():
|
||||
desktop.unlink()
|
||||
print("FrameYap removed; config and saved models preserved. OpenVR unregister acknowledgement was required.")
|
||||
print("FrameYap removed; config, saved models and any runtimes/ directory preserved. "
|
||||
"OpenVR unregister acknowledgement was required.")
|
||||
|
||||
|
||||
class UsageError(ValueError):
|
||||
@@ -839,6 +920,8 @@ def resolve_args(argv):
|
||||
action.add_argument("--rollback", action="store_true")
|
||||
action.add_argument("--uninstall", action="store_true")
|
||||
action.add_argument("--install-model", action="store_true", help="explicit model provisioning for installed backend")
|
||||
action.add_argument("--install-runtime", action="store_true",
|
||||
help="explicit pip install of the pinned CPU Python runtime into a user-local venv")
|
||||
parser.add_argument("--mode", choices=("binary", "source"), default="binary")
|
||||
parser.add_argument("--source", help="explicit local FrameYap source tree (source mode only)")
|
||||
parser.add_argument("--openvr-root", help="local OpenVR SDK root (source mode)")
|
||||
@@ -864,7 +947,7 @@ def resolve_args(argv):
|
||||
if args.mode == "source":
|
||||
if not args.source or any(not getattr(args, name) for name in source_inputs):
|
||||
parser.error("--mode source requires --source and explicit --openvr-root, --openvr-library, --openvr-license, --sdl-library, --sdl-license")
|
||||
if args.archive or args.sha256 or args.rollback or args.uninstall or args.install_model:
|
||||
if args.archive or args.sha256 or args.rollback or args.uninstall or args.install_model or args.install_runtime:
|
||||
parser.error("source mode cannot combine with binary archive or lifecycle operations")
|
||||
elif args.source or any(getattr(args, name) for name in source_inputs):
|
||||
parser.error("source inputs require --mode source")
|
||||
@@ -874,7 +957,10 @@ def resolve_args(argv):
|
||||
parser.error("--sha256 must be a 64-character hex SHA-256")
|
||||
if not args.archive and args.sha256:
|
||||
parser.error("--sha256 only applies to --archive")
|
||||
if not (args.rollback or args.uninstall or args.install_model) and not args.version:
|
||||
if args.install_runtime and (args.version or args.without_model or args.autolaunch is not None or
|
||||
args.backend != "redux" or args.model_dir or args.expected_manifest_sha256):
|
||||
parser.error("--install-runtime is independent of release/model flags")
|
||||
if not (args.rollback or args.uninstall or args.install_model or args.install_runtime) and not args.version:
|
||||
parser.error("--version VERSION is required; no moving/latest release")
|
||||
if args.uninstall != args.unregistered:
|
||||
parser.error("--uninstall requires --unregistered after explicit OpenVR unregister")
|
||||
@@ -903,7 +989,7 @@ def resolve_args(argv):
|
||||
|
||||
def plan(args):
|
||||
operation = ("uninstall" if args.uninstall else "rollback" if args.rollback else
|
||||
"install-model" if args.install_model else "install")
|
||||
"install-model" if args.install_model else "install-runtime" if args.install_runtime else "install")
|
||||
return {"operation": operation, "mode": args.mode, "version": args.version,
|
||||
"tag": "v" + args.version if args.version else None,
|
||||
"archive": str(args.archive) if args.archive else None,
|
||||
@@ -911,7 +997,8 @@ def plan(args):
|
||||
"backend": args.backend, "model_dir": str(args.model_dir) if args.model_dir else None,
|
||||
"expected_manifest_sha256": (args.expected_manifest_sha256.lower() if args.expected_manifest_sha256 else None),
|
||||
"without_model": args.without_model, "autolaunch": args.autolaunch,
|
||||
"network": bool(args.install_model or (operation == "install" and not args.archive and not args.source)),
|
||||
"network": bool(args.install_model or args.install_runtime or
|
||||
(operation == "install" and not args.archive and not args.source)),
|
||||
"registration": ("explicit --register --autostart" if args.autolaunch is True else
|
||||
"explicit --register (autostart off)" if args.autolaunch is False else "none")}
|
||||
|
||||
@@ -927,6 +1014,12 @@ def main(argv=None):
|
||||
check_expected_manifest(args, manifest_sha)
|
||||
result.update(model_dir=str(model_target(args, root)), model=backend.description(),
|
||||
installed_manifest_sha256=manifest_sha)
|
||||
if args.install_runtime:
|
||||
data = Path(os.environ.get("XDG_DATA_HOME") or Path.home() / ".local/share").expanduser().absolute()
|
||||
result.update(runtime_dir=str(data / "frameyap/runtimes"), base_python=sys.executable,
|
||||
base_python_version=platform.python_version(),
|
||||
packages=[RUNTIME_TORCH + " from " + RUNTIME_TORCH_INDEX, RUNTIME_REQUIREMENT + " from PyPI"],
|
||||
download=RUNTIME_DOWNLOAD, paths_config=str(paths_config_path()))
|
||||
if args.json:
|
||||
print(json.dumps({"ok": True, "event": "plan", **result}, sort_keys=True))
|
||||
else:
|
||||
@@ -976,6 +1069,8 @@ def main(argv=None):
|
||||
print(f"Rolled back to {previous}")
|
||||
elif args.install_model:
|
||||
install_model(args, root)
|
||||
elif args.install_runtime:
|
||||
install_runtime(args, root)
|
||||
else:
|
||||
do_install(args, root, launcher)
|
||||
# The native registration helper takes this same install lock. Never run it
|
||||
@@ -1045,7 +1140,7 @@ def cli(argv=None):
|
||||
try:
|
||||
if structured and "--print-plan" in argv:
|
||||
main(argv) # already emits one JSON plan
|
||||
elif structured and "--install-model" in argv:
|
||||
elif structured and ("--install-model" in argv or "--install-runtime" in argv):
|
||||
main(argv) # streaming per-file JSON events for a UI child
|
||||
print(json.dumps({"ok": True, "event": "complete"}))
|
||||
elif structured:
|
||||
|
||||
@@ -1113,6 +1113,92 @@ with patch.object(module, "check_host"), patch.object(module.urllib.request, "ur
|
||||
self.assertIn("mismatched", json.loads(output.getvalue())["message"])
|
||||
fetch.assert_not_called()
|
||||
|
||||
def test_runtime_install_is_explicit_cpu_pinned_and_updates_only_python_path(self):
|
||||
# subprocess is mocked: no venv, pip or network is touched.
|
||||
root = self.data / "frameyap"
|
||||
output = io.StringIO()
|
||||
with contextlib.redirect_stdout(output):
|
||||
self.assertEqual(installer.cli(["--install-runtime", "--print-plan", "--json"]), 0)
|
||||
plan = json.loads(output.getvalue())
|
||||
self.assertEqual((plan["operation"], plan["network"]), ("install-runtime", True))
|
||||
self.assertIn("torch==2.8.0 from https://download.pytorch.org/whl/cpu", plan["packages"])
|
||||
self.assertFalse(root.exists())
|
||||
for extra in (["--version", "0.1.202609241530"], ["--backend", "other"], ["--without-model"]):
|
||||
output = io.StringIO()
|
||||
with contextlib.redirect_stdout(output):
|
||||
self.assertEqual(installer.cli(["--install-runtime", "--json"] + extra), 2)
|
||||
output = io.StringIO()
|
||||
with contextlib.redirect_stdout(output), patch.object(installer.subprocess, "run") as run:
|
||||
self.assertEqual(installer.cli(["--install-runtime", "--json"]), 1)
|
||||
run.assert_not_called()
|
||||
self.assertIn("--yes", json.loads(output.getvalue())["message"])
|
||||
self.assertFalse((root / "runtimes").exists())
|
||||
|
||||
config = self.home / ".config/frameyap/paths.conf"
|
||||
config.parent.mkdir(parents=True)
|
||||
config.write_text("# mine\npython=/opt/old/bin/python3\nmodel=/opt/model\n")
|
||||
old = root / "runtimes/cpu-20260101000000"
|
||||
old.mkdir(parents=True)
|
||||
foreign = root / "runtimes/keep-me"
|
||||
foreign.mkdir()
|
||||
commands = []
|
||||
|
||||
def fake(command, **kwargs):
|
||||
commands.append(command)
|
||||
if command[1:3] == ["-m", "venv"]:
|
||||
python = Path(command[3]) / "bin/python3"
|
||||
python.parent.mkdir(parents=True)
|
||||
python.write_text("")
|
||||
return SimpleNamespace(returncode=0)
|
||||
|
||||
with patch.object(installer, "RUNTIME_PYTHON_RANGE", ((3, 0), (3, 1))), \
|
||||
patch.object(installer.subprocess, "run") as run:
|
||||
with contextlib.redirect_stdout(io.StringIO()) as unsupported:
|
||||
self.assertEqual(installer.cli(["--install-runtime", "--yes", "--json"]), 1)
|
||||
self.assertIn("Python 3.10-3.13", json.loads(unsupported.getvalue())["message"])
|
||||
run.assert_not_called()
|
||||
self.env_range = patch.object(installer, "RUNTIME_PYTHON_RANGE", ((3, 0), (4, 0)))
|
||||
self.env_range.start()
|
||||
self.addCleanup(self.env_range.stop)
|
||||
output = io.StringIO()
|
||||
with contextlib.redirect_stdout(output), patch.object(installer.subprocess, "run", side_effect=fake):
|
||||
self.assertEqual(installer.cli(["--install-runtime", "--yes", "--json"]), 0)
|
||||
events = [json.loads(line) for line in output.getvalue().splitlines()]
|
||||
self.assertEqual([e.get("step") for e in events if e["event"] == "runtime_step"],
|
||||
["create virtual environment", "install CPU Torch", "install moondream/Kestrel",
|
||||
"verify CPU runtime imports"])
|
||||
self.assertEqual(events[-1]["event"], "complete")
|
||||
torch, packages = commands[1], commands[2]
|
||||
for command in (torch, packages):
|
||||
self.assertIn("--isolated", command)
|
||||
self.assertIn("--only-binary=:all:", command)
|
||||
self.assertEqual(torch[-3:], ["--index-url", "https://download.pytorch.org/whl/cpu", "torch==2.8.0"])
|
||||
self.assertEqual(packages[-1], "moondream==2.4.0")
|
||||
self.assertIn("--constraint", packages)
|
||||
runtimes = [item for item in (root / "runtimes").iterdir() if item.name.startswith("cpu-")]
|
||||
self.assertEqual(len(runtimes), 1)
|
||||
self.assertNotEqual(runtimes[0], old)
|
||||
self.assertTrue(foreign.is_dir())
|
||||
self.assertEqual(config.read_text(),
|
||||
f"# mine\nmodel=/opt/model\npython={runtimes[0]}/bin/python3\n")
|
||||
|
||||
# A failed pip step removes only its own new venv and leaves paths.conf alone.
|
||||
before = config.read_text()
|
||||
|
||||
def failing(command, **kwargs):
|
||||
if "moondream==2.4.0" in command:
|
||||
raise subprocess.CalledProcessError(1, command)
|
||||
return fake(command, **kwargs)
|
||||
|
||||
output = io.StringIO()
|
||||
with contextlib.redirect_stdout(output), patch.object(installer.subprocess, "run", side_effect=failing), \
|
||||
patch.object(installer, "datetime") as clock:
|
||||
clock.now.return_value.strftime.return_value = "cpu-20991231235959"
|
||||
self.assertEqual(installer.cli(["--install-runtime", "--yes", "--json"]), 1)
|
||||
self.assertFalse((root / "runtimes/cpu-20991231235959").exists())
|
||||
self.assertTrue(runtimes[0].is_dir())
|
||||
self.assertEqual(config.read_text(), before)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in new issue
Block a user