Add idempotent native release installer with external ASR boundary

This commit is contained in:
baketnk committed 2026-09-24 11:44:37 -04:00
1 parent 0cb2991f02
commit 73dc4fda06
7 files changed
+1500

No files matched your search

+124
View File
@@ -0,0 +1,124 @@
# Release packaging and idempotent user-local installer
**No GitHub release is published.** Native-only local artifacts have been installed
and reinstalled on Frame. The end-to-end bundled-ASR release remains blocked on
runtime permission; see [third-party notes](third-party.md). The installer never
pretends the proprietary runtime is included when it is not.
## Producer
Default builds stay offline. Producers explicitly provision native dependencies,
build with `FRAMEYAP_NATIVE=ON`, and stage this layout (regular files, no links):
```
bin/frameyap
lib/* # compatible bundled native libraries
assets/actions.json # and adjacent controller binding JSON
fonts/font.ttf
python/frameyap/*.py
licenses/THIRD_PARTY_NOTICES.txt
model/* # optional pinned public weights + attribution
runtime/bin/python3 # ONLY for an authorized bundled-runtime artifact
```
For the current **external-runtime** POC, `scripts/stage-native-poc.py --help`
documents explicit inputs. It invokes `cmake --install` on an existing native build,
copies SDL/OpenVR and an explicitly licensed font, and retains notices. It does
not build, download, run the app, or copy a proprietary ASR runtime. The native
POC relies on Frame's system Wayland, FreeType, libstdc++ and glibc; audit `ldd`
on the installed binary. SDL/OpenVR resolve inside its own `lib/`, not a producer
prefix. ARM64/glibc packaging is not a claim of compatibility with arbitrary Linux.
```sh
python3 scripts/package-release.py --stage /path/to/stage --output /existing/output \
--version v0.1.0-poc --arch linux-aarch64 \
--model-revision fad622f25f303105c20d70e201bcc477c88b620c --external-runtime
```
`--external-runtime` refuses a runtime directory and records
`runtime: external-authorized-python` in `release.json`. The installer explicitly
reports that ASR is not supplied. Without that flag, a complete independently
licensed, compatible isolated CPU Python runtime is required. **Do not use that
bundled route for Kestrel without permission covering redistribution.** Staging
validation is not a license grant or an inference test.
The producer refuses overwrites and emits `frameyap-VERSION-linux-aarch64.tar.gz`
plus `.sha256` containing `HASH FILENAME`. Archive extraction rejects traversal,
links/special files, duplicate members, oversized metadata/payloads and invalid
layout. Checksums detect corruption, not a malicious/compromised publisher;
authenticate release metadata independently. No packaging/installation model fetch.
## Consumer
Bootstrap: Linux ARM64/glibc, Python 3.12+, curl, sha256sum and tar. **No compiler,
sudo, Steam store AppID or engine checkout.** Download/inspect a pinned installer
before running it. Current local artifact route:
```sh
sh install.sh --archive /path/to/frameyap-VERSION-linux-aarch64.tar.gz \
--sha256 64_HEX_DIGIT_HASH --version VERSION
```
After an actual vetted release exists, `sh install.sh --version TAG` retrieves
that GitHub release and its versioned checksum; no `latest` or moving-branch
lookup. A pipe invocation is supported, never prompts on stdin, and must also
pin a real published tag. **There is no functional public download command yet.**
`--without-model` omits bundled model files from staging, never deletes a current
model on rerun, and records the choice. Same digest/version/choice is idempotent
and repairs missing managed wrappers. Different digest or model choice for the
same version is refused. External model provisioning is always deliberate.
The installed launcher defaults to `--run`. For a native-only package, supply
`FRAMEYAP_PYTHON=/absolute/authorized/python` and `FRAMEYAP_MODEL=/absolute/model`,
or override `--python`/`--model` on an explicit `--run`. No pip/bootstrap/fallback
is invoked by the launcher. Missing runtime means visible failure, not recording.
Environment configuration is not automatically persisted for SteamVR autolaunch.
Without any ASR runtime, these checks work directly through the installed launcher:
```sh
~/.local/bin/frameyap --check-input
~/.local/bin/frameyap --check-overlay --head
~/.local/bin/frameyap --check-controls --head
```
These modes cannot record or type. `GAMESCOPE_SOCKET` or `--socket` selects the
input backend; default is `GAMESCOPE_WAYLAND_DISPLAY`, then `gamescope-0`.
## Lifecycle
Install root: `$XDG_DATA_HOME/frameyap` (default `~/.local/share/frameyap`);
launcher: `~/.local/bin/frameyap`; existing config is untouched. The launcher
passes a stable install-root lock identity and sets `PYTHONDONTWRITEBYTECODE=1`.
Runtime/check/registration modes and installer share an exclusive nonblocking
`.lock`; no upgrade/rollback/uninstall kills a running app or any other process.
Selection of a completed `current` is atomic; `previous` is retained.
`sh install.sh --rollback` switches to the prior validated version. Foreign/modified
wrappers, untracked install files and inconsistent ownership metadata are refused.
Same-user malicious concurrent filesystem mutation is outside the POC threat model.
The generated `frameyap.vrmanifest` uses `local.frameyap.overlay`, **not a store
AppID**. Linux ARM requires `binary_path_linux_arm`; both Linux fields are written.
Installation does not initialize OpenVR, register, autostart, record or type.
With SteamVR ready, explicitly register:
```sh
~/.local/bin/frameyap --register "$HOME/.local/share/frameyap/frameyap.vrmanifest"
# Substitute XDG_DATA_HOME if customized. Add --autostart only if deliberately wanted.
```
Registration checks actual OpenVR installed state rather than trusting Add's return
alone. Repeated registration/removal were exercised on Frame, autolaunch off.
Actual menu launch and cold-runtime behavior remain separate acceptance checks.
Before uninstall, explicitly `frameyap --unregister /absolute/manifest/path` and
verify success, then run `sh install.sh --uninstall --unregistered`. The latter is
an acknowledgement, not a hidden SteamVR edit. Only owned files are removed;
config stays, models move to `saved-models/VERSION`, conflicts/untracked files abort.
Offline tests: `python3 -m unittest discover -s tests -p test_installer.py`.
They use temporary homes/local fixtures. Dated live-device observations are in the
[POC record](evidence/poc-cpu-overlay-2026-09-24.md). When editing the Python helper,
run `python3 scripts/sync-installer.py`; tests enforce embedded installer parity.
Executable
+468
View File
@@ -0,0 +1,468 @@
#!/bin/sh
# FrameYap pinned release installer. No implicit version, downloads, or runtime launch.
set -eu
for tool in python3 curl sha256sum tar; do
command -v "$tool" >/dev/null 2>&1 || { echo "frameyap installer: missing prerequisite: $tool" >&2; exit 1; }
done
python3 -c 'import sys; sys.exit(0 if sys.version_info >= (3, 12) else 1)' || {
echo 'frameyap installer: Python 3.12+ required for bootstrap only (release bundles runtime)' >&2
exit 1
}
exec python3 - "$@" <<'PY'
"""FrameYap installer implementation. Embedded verbatim in install.sh for piped installs."""
import argparse
import fcntl
import hashlib
import json
import os
from pathlib import Path
import platform
import re
import shutil
import subprocess
import sys
import tarfile
import tempfile
from urllib.parse import quote
KEY = "local.frameyap.overlay"
MARKER = "# FrameYap managed launcher v1\n"
ARCHIVE_LIMIT = 12 * 1024**3
MEMBER_LIMIT = 50000
VERSION_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,95}\Z")
DIGEST_RE = re.compile(r"[a-fA-F0-9]{64}\Z")
def fail(message):
raise ValueError(message)
def json_atomic(path, value):
atomic_write(path, (json.dumps(value, indent=2, sort_keys=True) + "\n").encode())
def atomic_write(path, content, mode=0o600):
fd, name = tempfile.mkstemp(prefix=".frameyap-", dir=path.parent)
try:
os.fchmod(fd, mode)
with os.fdopen(fd, "wb") as stream:
stream.write(content)
stream.flush()
os.fsync(stream.fileno())
os.replace(name, path)
finally:
if os.path.exists(name):
os.unlink(name)
def owned_dir(path):
if path.is_symlink():
fail(f"refusing symlink directory: {path}")
path.mkdir(mode=0o700, parents=True, exist_ok=True)
if not path.is_dir():
fail(f"not a directory: {path}")
def check_host():
if sys.platform != "linux" or platform.machine().lower() not in ("aarch64", "arm64"):
fail("release requires Linux AArch64 (ARM64); no cross-architecture install")
if platform.libc_ver()[0] != "glibc":
fail("release requires glibc Linux; libc compatibility still requires native testing")
def digest_file(path):
h = hashlib.sha256()
with open(path, "rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
h.update(block)
return h.hexdigest()
def check_digest(value):
if not DIGEST_RE.fullmatch(value):
fail("--sha256 must be a 64-character hex SHA-256")
return value.lower()
def check_version(value):
if not VERSION_RE.fullmatch(value) or value in (".", ".."):
fail("invalid release version/tag")
return value
def release_name(version):
return f"frameyap-{version}-linux-aarch64.tar.gz"
def download(url, destination):
subprocess.run(["curl", "--fail", "--silent", "--show-error", "--location", "--proto", "=https",
"--proto-redir", "=https", "--tlsv1.2", "--max-filesize",
str(4096 if url.endswith(".sha256") else ARCHIVE_LIMIT),
"--output", str(destination), url], check=True)
def verify_members(archive):
seen = set()
total = 0
count = 0
for member in archive:
count += 1
if count > MEMBER_LIMIT:
fail("too many archive members")
name = member.name
parts = name.rstrip("/").split("/")
if (not name or name.startswith("/") or len(name) > 1024 or len(parts) > 32
or any(x in ("", ".", "..") for x in parts)
or "\\" in name or "\x00" in name):
fail(f"unsafe archive path: {name!r}")
if name in seen:
fail(f"duplicate archive member: {name}")
seen.add(name)
if not (member.isfile() or member.isdir()):
fail(f"archive links and special files forbidden: {name}")
if member.size < 0 or member.size > ARCHIVE_LIMIT:
fail("oversized archive member")
if name == "release.json" and member.size > 8192:
fail("oversized release metadata")
total += member.size
if total > ARCHIVE_LIMIT:
fail("archive uncompressed limit exceeded")
if (parts[0] not in ("release.json", "bin", "lib", "assets", "python", "runtime", "model", "fonts", "licenses")
or (parts[0] == "release.json" and (len(parts) != 1 or not member.isfile()))
or (len(parts) == 1 and parts[0] != "release.json" and not member.isdir())):
fail(f"unexpected archive path: {name}")
def extract(archive_path, destination):
with tarfile.open(archive_path, "r:gz") as archive:
verify_members(archive)
with tarfile.open(archive_path, "r:gz") as archive:
for member in archive:
target = destination.joinpath(*member.name.rstrip("/").split("/"))
if member.isdir():
owned_dir(target)
else:
owned_dir(target.parent)
# Never follow a pre-existing link, including one created by the archive.
with target.open("xb") as out, archive.extractfile(member) as source:
shutil.copyfileobj(source, out, 1024 * 1024)
# Retain executable helpers/shared libraries; discard all other mode bits.
target.chmod(0o700 if member.mode & 0o111 else 0o600)
def validate_payload(root, version, without_model=False, installed=False):
meta = json.loads((root / "release.json").read_text())
if not isinstance(meta, dict):
fail("invalid release metadata")
if meta.get("schema") != 1 or meta.get("version") != version or meta.get("arch") != "linux-aarch64":
fail("release metadata version/architecture/schema mismatch")
if meta.get("runtime") not in ("bundled-cpu-python", "external-authorized-python") or not isinstance(meta.get("model_revision"), str) or not meta["model_revision"]:
fail("missing runtime/model revision declaration")
has_model = meta.get("includes_model")
if not isinstance(has_model, bool) or (not installed and not has_model and (root / "model").exists()):
fail("inconsistent model declaration")
if has_model and not (root / "model").is_dir() and not (installed and without_model and not (root / "model").exists()):
fail("missing declared model")
if meta["runtime"] == "external-authorized-python" and (root / "runtime").exists():
fail("external runtime payload must not include a runtime")
for name in ("bin/frameyap", "runtime/bin/python3", "python/frameyap/worker.py", "assets/actions.json", "fonts/font.ttf"):
if name == "runtime/bin/python3" and meta["runtime"] == "external-authorized-python":
continue
if not (root / name).is_file():
fail(f"missing payload file: {name}")
for name in ("lib", "fonts"):
if not (root / name).is_dir() or not any((root / name).iterdir()):
fail(f"missing payload directory: {name}")
return meta
def selected(root, link):
path = root / link
if not path.is_symlink():
if path.exists():
fail(f"refusing foreign selection path: {path}")
return None
value = os.readlink(path)
if not re.fullmatch(r"versions/[A-Za-z0-9][A-Za-z0-9._-]{0,95}", value):
fail(f"invalid {link} selection")
if not (root / value).is_dir():
fail(f"broken {link} selection")
return value
def select(root, link, target):
path = root / link
temp = root / ("." + link + "-" + str(os.getpid()))
try:
os.symlink(target, temp)
os.replace(temp, path)
finally:
temp.unlink(missing_ok=True)
def desired_launcher(root):
import shlex
q = lambda path: shlex.quote(str(path))
font = root / "current/fonts/font.ttf"
base = root / "current"
flags = ["--assets", base / "assets", "--font", font,
"--worker", base / "python/frameyap/worker.py"]
args = " ".join(q(item) for item in flags)
return ("#!/bin/sh\n" + MARKER + 'export PYTHONDONTWRITEBYTECODE=1\n'
+ f'export FRAMEYAP_INSTALL_ROOT={q(root)}\n'
+ f'export LD_LIBRARY_PATH={q(base / "lib")}${{LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}}\n'
+ f'PYTHON=${{FRAMEYAP_PYTHON:-{q(base / "runtime/bin/python3")}}}\n'
+ f'MODEL=${{FRAMEYAP_MODEL:-{q(base / "model")}}}\n'
+ 'if [ "$#" -eq 0 ]; then set -- --run; fi\n'
+ 'case "$1" in\n'
+ ' --run)\n'
+ ' if [ -n "${GAMESCOPE_SOCKET:-}" ]; then set -- "$@" --socket "$GAMESCOPE_SOCKET"; fi\n'
+ f" shift; exec {q(base / 'bin/frameyap')} --run {args} --python \"$PYTHON\" --model \"$MODEL\" \"$@\";;\n"
+ f" --check-overlay|--check-controls) mode=$1; shift; exec {q(base / 'bin/frameyap')} \"$mode\" --assets {q(base / 'assets')} --font {q(font)} \"$@\";;\n"
+ f" *) exec {q(base / 'bin/frameyap')} \"$@\";;\n"
+ 'esac\n').encode()
def desired_manifest(launcher):
return {"source": "builtin", "applications": [{"app_key": KEY, "launch_type": "binary",
"binary_path_linux": str(launcher), "binary_path_linux_arm": str(launcher),
"is_dashboard_overlay": True,
"strings": {"en_us": {"name": "FrameYap"}}}]}
def check_owned_file(path, expected):
if path.is_symlink():
fail(f"refusing foreign symlink: {path}")
if path.exists():
if not path.is_file():
fail(f"refusing foreign path: {path}")
data = path.read_bytes()
if data != expected:
fail(f"refusing to replace modified/foreign file: {path}")
def check_wrappers(root, launcher):
manifest = root / "frameyap.vrmanifest"
desired = (json.dumps(desired_manifest(launcher), sort_keys=True, indent=2) + "\n").encode()
check_owned_file(launcher, desired_launcher(root))
check_owned_file(manifest, desired)
return manifest, desired
def install_files(root, installed, launcher):
owned_dir(launcher.parent)
manifest, desired = check_wrappers(root, launcher)
atomic_write(launcher, desired_launcher(root), 0o755)
atomic_write(manifest, desired)
def receipt(path):
marker = path / ".archive-sha256"
if marker.is_symlink() or not marker.is_file() or not DIGEST_RE.fullmatch(marker.read_text().strip()):
fail(f"refusing unowned installation directory: {path}")
return marker.read_text().strip()
def inventory(path):
return sorted(p.relative_to(path).as_posix() for p in path.rglob("*")
if p.relative_to(path).parts[0] != "model"
and p.relative_to(path).as_posix() not in (".archive-sha256", ".installed-files.json", ".install-options.json"))
def check_inventory(path):
index = path / ".installed-files.json"
if not index.is_file() or index.is_symlink():
fail(f"missing installation inventory: {path}")
declared = json.loads(index.read_text())
if not isinstance(declared, list) or inventory(path) != declared:
fail(f"refusing to remove untracked files in: {path}")
for p in path.rglob("*"):
if p.is_symlink() and p.relative_to(path).parts[0] != "model":
fail(f"refusing link inside installation: {p}")
def installed_choice(path):
receipt(path)
options = path / ".install-options.json"
if not options.is_file() or options.is_symlink():
fail(f"missing installation options: {path}")
choice = json.loads(options.read_text())
if not isinstance(choice, dict) or set(choice) != {"without_model"} or not isinstance(choice["without_model"], bool):
fail(f"invalid installation options: {path}")
return choice["without_model"]
def do_install(args, root, launcher):
version = check_version(args.version)
versions = root / "versions"
owned_dir(versions)
current = selected(root, "current")
selected(root, "previous")
# Refuse foreign wrappers/launcher directories before installing a new version.
owned_dir(launcher.parent)
check_wrappers(root, launcher)
if args.archive:
archive = Path(args.archive).expanduser().resolve(strict=True)
expected = check_digest(args.sha256)
do_download = False
else:
do_download = True
archive = None
with tempfile.TemporaryDirectory(prefix=".download-", dir=root) as td:
if do_download:
name = release_name(version)
url = f"https://github.com/{args.repo}/releases/download/{quote(version)}/{name}"
archive = Path(td) / name
sidecar = Path(td) / (name + ".sha256")
download(url + ".sha256", sidecar)
if sidecar.stat().st_size > 4096:
fail("oversized SHA-256 sidecar")
line = sidecar.read_text().strip()
match = re.fullmatch(r"([a-fA-F0-9]{64}) " + re.escape(name), line)
if not match:
fail("invalid versioned SHA-256 sidecar")
expected = match.group(1).lower()
download(url, archive)
if archive.stat().st_size > ARCHIVE_LIMIT:
fail("compressed archive exceeds limit")
if not do_download:
local_copy = Path(td) / "local-archive.tar.gz"
shutil.copyfile(archive, local_copy)
archive = local_copy
if digest_file(archive) != expected:
fail("archive SHA-256 mismatch")
target = versions / version
if target.exists() or target.is_symlink():
if target.is_symlink() or receipt(target) != expected:
fail("same version already installed with different archive digest")
if installed_choice(target) != args.without_model:
fail("same version already installed with different --without-model choice")
check_inventory(target)
validate_payload(target, version, args.without_model, installed=True)
else:
temp = Path(tempfile.mkdtemp(prefix=".staging-", dir=versions))
try:
extract(archive, temp)
validate_payload(temp, version)
if args.without_model and (temp / "model").exists():
shutil.rmtree(temp / "model")
json_atomic(temp / ".installed-files.json", inventory(temp))
json_atomic(temp / ".install-options.json", {"without_model": args.without_model})
(temp / ".archive-sha256").write_text(expected + "\n")
os.replace(temp, target)
finally:
if temp.exists():
shutil.rmtree(temp)
install_files(root, target, launcher)
if current == f"versions/{version}":
print(f"FrameYap {version}: already installed (same digest); wrappers verified")
return
if current and current != f"versions/{version}":
select(root, "previous", current)
select(root, "current", f"versions/{version}")
print(f"FrameYap {version} installed. OpenVR registration is NOT automatic; see docs/packaging.md.")
if json.loads((target / "release.json").read_text())["runtime"] == "external-authorized-python":
print("ASR runtime is NOT included. Supply an independently authorized environment with FRAMEYAP_PYTHON or --python; no packages are downloaded.")
def uninstall(root, launcher):
current = selected(root, "current")
previous = selected(root, "previous")
manifest = root / "frameyap.vrmanifest"
desired = (json.dumps(desired_manifest(launcher), sort_keys=True, indent=2) + "\n").encode()
check_owned_file(manifest, desired)
check_owned_file(launcher, desired_launcher(root))
versions = root / "versions"
if versions.exists():
if versions.is_symlink():
fail("refusing symlink versions directory")
for item in versions.iterdir():
if not item.is_dir() or item.is_symlink() or item.name.startswith("."):
fail(f"unexpected versions entry: {item}")
choice = installed_choice(item)
check_inventory(item)
validate_payload(item, item.name, choice, installed=True)
if (item / "model").exists() or (item / "model").is_symlink():
saved = root / "saved-models"
owned_dir(saved)
target = saved / item.name
if target.exists() or target.is_symlink():
fail(f"saved model already exists: {target}")
for item in versions.iterdir():
if (item / "model").exists() or (item / "model").is_symlink():
os.replace(item / "model", root / "saved-models" / item.name)
shutil.rmtree(item)
for link in ("current", "previous"):
(root / link).unlink(missing_ok=True)
if launcher.exists():
launcher.unlink()
if manifest.exists():
manifest.unlink()
print("FrameYap removed; config and saved models preserved. OpenVR unregister acknowledgement was required.")
def main(argv=None):
parser = argparse.ArgumentParser(prog="install.sh", description="User-local FrameYap release installer (no SteamVR actions)")
mode = parser.add_mutually_exclusive_group()
mode.add_argument("--archive", help="local release archive (requires --sha256 and --version)")
mode.add_argument("--rollback", action="store_true")
mode.add_argument("--uninstall", action="store_true")
parser.add_argument("--sha256")
parser.add_argument("--version")
parser.add_argument("--repo", default="baketnk/frame-yap")
parser.add_argument("--without-model", action="store_true")
parser.add_argument("--unregistered", action="store_true", help="acknowledge explicit OpenVR removal before uninstall")
args = parser.parse_args(argv)
if args.repo != "baketnk/frame-yap":
parser.error("only the pinned baketnk/frame-yap release repository is supported")
if args.archive and (not args.sha256 or not args.version):
parser.error("--archive requires --sha256 and --version")
if not args.archive and args.sha256:
parser.error("--sha256 only applies to --archive")
if not (args.rollback or args.uninstall or args.archive) and not args.version:
parser.error("--version TAG is required; no moving/latest release")
if args.uninstall and not args.unregistered:
parser.error("uninstall requires --unregistered after explicit OpenVR unregister")
if args.unregistered and not args.uninstall:
parser.error("--unregistered only applies to --uninstall")
if args.version:
check_version(args.version)
check_host()
data = Path(os.environ.get("XDG_DATA_HOME") or Path.home() / ".local/share").expanduser().absolute()
root = data / "frameyap"
launcher = Path.home() / ".local/bin/frameyap"
owned_dir(root)
lock = root / ".lock"
if lock.is_symlink():
fail("refusing symlink lock")
with lock.open("a+b") as fd:
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
fail("FrameYap installer or application is running (.lock held); try again later")
if args.uninstall:
uninstall(root, launcher)
elif args.rollback:
owned_dir(root / "versions")
current, previous = selected(root, "current"), selected(root, "previous")
if not current or not previous:
fail("no previous installation to roll back to")
check_wrappers(root, launcher)
choice = installed_choice(root / previous)
check_inventory(root / previous)
validate_payload(root / previous, Path(previous).name, choice, installed=True)
select(root, "current", previous)
select(root, "previous", current)
print(f"Rolled back to {previous}")
else:
do_install(args, root, launcher)
if __name__ == "__main__":
try:
main()
except (ValueError, OSError, subprocess.CalledProcessError, tarfile.TarError, json.JSONDecodeError) as exc:
print(f"frameyap installer: {exc}", file=sys.stderr)
sys.exit(1)
PY
+455
View File
@@ -0,0 +1,455 @@
"""FrameYap installer implementation. Embedded verbatim in install.sh for piped installs."""
import argparse
import fcntl
import hashlib
import json
import os
from pathlib import Path
import platform
import re
import shutil
import subprocess
import sys
import tarfile
import tempfile
from urllib.parse import quote
KEY = "local.frameyap.overlay"
MARKER = "# FrameYap managed launcher v1\n"
ARCHIVE_LIMIT = 12 * 1024**3
MEMBER_LIMIT = 50000
VERSION_RE = re.compile(r"[A-Za-z0-9][A-Za-z0-9._-]{0,95}\Z")
DIGEST_RE = re.compile(r"[a-fA-F0-9]{64}\Z")
def fail(message):
raise ValueError(message)
def json_atomic(path, value):
atomic_write(path, (json.dumps(value, indent=2, sort_keys=True) + "\n").encode())
def atomic_write(path, content, mode=0o600):
fd, name = tempfile.mkstemp(prefix=".frameyap-", dir=path.parent)
try:
os.fchmod(fd, mode)
with os.fdopen(fd, "wb") as stream:
stream.write(content)
stream.flush()
os.fsync(stream.fileno())
os.replace(name, path)
finally:
if os.path.exists(name):
os.unlink(name)
def owned_dir(path):
if path.is_symlink():
fail(f"refusing symlink directory: {path}")
path.mkdir(mode=0o700, parents=True, exist_ok=True)
if not path.is_dir():
fail(f"not a directory: {path}")
def check_host():
if sys.platform != "linux" or platform.machine().lower() not in ("aarch64", "arm64"):
fail("release requires Linux AArch64 (ARM64); no cross-architecture install")
if platform.libc_ver()[0] != "glibc":
fail("release requires glibc Linux; libc compatibility still requires native testing")
def digest_file(path):
h = hashlib.sha256()
with open(path, "rb") as stream:
for block in iter(lambda: stream.read(1024 * 1024), b""):
h.update(block)
return h.hexdigest()
def check_digest(value):
if not DIGEST_RE.fullmatch(value):
fail("--sha256 must be a 64-character hex SHA-256")
return value.lower()
def check_version(value):
if not VERSION_RE.fullmatch(value) or value in (".", ".."):
fail("invalid release version/tag")
return value
def release_name(version):
return f"frameyap-{version}-linux-aarch64.tar.gz"
def download(url, destination):
subprocess.run(["curl", "--fail", "--silent", "--show-error", "--location", "--proto", "=https",
"--proto-redir", "=https", "--tlsv1.2", "--max-filesize",
str(4096 if url.endswith(".sha256") else ARCHIVE_LIMIT),
"--output", str(destination), url], check=True)
def verify_members(archive):
seen = set()
total = 0
count = 0
for member in archive:
count += 1
if count > MEMBER_LIMIT:
fail("too many archive members")
name = member.name
parts = name.rstrip("/").split("/")
if (not name or name.startswith("/") or len(name) > 1024 or len(parts) > 32
or any(x in ("", ".", "..") for x in parts)
or "\\" in name or "\x00" in name):
fail(f"unsafe archive path: {name!r}")
if name in seen:
fail(f"duplicate archive member: {name}")
seen.add(name)
if not (member.isfile() or member.isdir()):
fail(f"archive links and special files forbidden: {name}")
if member.size < 0 or member.size > ARCHIVE_LIMIT:
fail("oversized archive member")
if name == "release.json" and member.size > 8192:
fail("oversized release metadata")
total += member.size
if total > ARCHIVE_LIMIT:
fail("archive uncompressed limit exceeded")
if (parts[0] not in ("release.json", "bin", "lib", "assets", "python", "runtime", "model", "fonts", "licenses")
or (parts[0] == "release.json" and (len(parts) != 1 or not member.isfile()))
or (len(parts) == 1 and parts[0] != "release.json" and not member.isdir())):
fail(f"unexpected archive path: {name}")
def extract(archive_path, destination):
with tarfile.open(archive_path, "r:gz") as archive:
verify_members(archive)
with tarfile.open(archive_path, "r:gz") as archive:
for member in archive:
target = destination.joinpath(*member.name.rstrip("/").split("/"))
if member.isdir():
owned_dir(target)
else:
owned_dir(target.parent)
# Never follow a pre-existing link, including one created by the archive.
with target.open("xb") as out, archive.extractfile(member) as source:
shutil.copyfileobj(source, out, 1024 * 1024)
# Retain executable helpers/shared libraries; discard all other mode bits.
target.chmod(0o700 if member.mode & 0o111 else 0o600)
def validate_payload(root, version, without_model=False, installed=False):
meta = json.loads((root / "release.json").read_text())
if not isinstance(meta, dict):
fail("invalid release metadata")
if meta.get("schema") != 1 or meta.get("version") != version or meta.get("arch") != "linux-aarch64":
fail("release metadata version/architecture/schema mismatch")
if meta.get("runtime") not in ("bundled-cpu-python", "external-authorized-python") or not isinstance(meta.get("model_revision"), str) or not meta["model_revision"]:
fail("missing runtime/model revision declaration")
has_model = meta.get("includes_model")
if not isinstance(has_model, bool) or (not installed and not has_model and (root / "model").exists()):
fail("inconsistent model declaration")
if has_model and not (root / "model").is_dir() and not (installed and without_model and not (root / "model").exists()):
fail("missing declared model")
if meta["runtime"] == "external-authorized-python" and (root / "runtime").exists():
fail("external runtime payload must not include a runtime")
for name in ("bin/frameyap", "runtime/bin/python3", "python/frameyap/worker.py", "assets/actions.json", "fonts/font.ttf"):
if name == "runtime/bin/python3" and meta["runtime"] == "external-authorized-python":
continue
if not (root / name).is_file():
fail(f"missing payload file: {name}")
for name in ("lib", "fonts"):
if not (root / name).is_dir() or not any((root / name).iterdir()):
fail(f"missing payload directory: {name}")
return meta
def selected(root, link):
path = root / link
if not path.is_symlink():
if path.exists():
fail(f"refusing foreign selection path: {path}")
return None
value = os.readlink(path)
if not re.fullmatch(r"versions/[A-Za-z0-9][A-Za-z0-9._-]{0,95}", value):
fail(f"invalid {link} selection")
if not (root / value).is_dir():
fail(f"broken {link} selection")
return value
def select(root, link, target):
path = root / link
temp = root / ("." + link + "-" + str(os.getpid()))
try:
os.symlink(target, temp)
os.replace(temp, path)
finally:
temp.unlink(missing_ok=True)
def desired_launcher(root):
import shlex
q = lambda path: shlex.quote(str(path))
font = root / "current/fonts/font.ttf"
base = root / "current"
flags = ["--assets", base / "assets", "--font", font,
"--worker", base / "python/frameyap/worker.py"]
args = " ".join(q(item) for item in flags)
return ("#!/bin/sh\n" + MARKER + 'export PYTHONDONTWRITEBYTECODE=1\n'
+ f'export FRAMEYAP_INSTALL_ROOT={q(root)}\n'
+ f'export LD_LIBRARY_PATH={q(base / "lib")}${{LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}}\n'
+ f'PYTHON=${{FRAMEYAP_PYTHON:-{q(base / "runtime/bin/python3")}}}\n'
+ f'MODEL=${{FRAMEYAP_MODEL:-{q(base / "model")}}}\n'
+ 'if [ "$#" -eq 0 ]; then set -- --run; fi\n'
+ 'case "$1" in\n'
+ ' --run)\n'
+ ' if [ -n "${GAMESCOPE_SOCKET:-}" ]; then set -- "$@" --socket "$GAMESCOPE_SOCKET"; fi\n'
+ f" shift; exec {q(base / 'bin/frameyap')} --run {args} --python \"$PYTHON\" --model \"$MODEL\" \"$@\";;\n"
+ f" --check-overlay|--check-controls) mode=$1; shift; exec {q(base / 'bin/frameyap')} \"$mode\" --assets {q(base / 'assets')} --font {q(font)} \"$@\";;\n"
+ f" *) exec {q(base / 'bin/frameyap')} \"$@\";;\n"
+ 'esac\n').encode()
def desired_manifest(launcher):
return {"source": "builtin", "applications": [{"app_key": KEY, "launch_type": "binary",
"binary_path_linux": str(launcher), "binary_path_linux_arm": str(launcher),
"is_dashboard_overlay": True,
"strings": {"en_us": {"name": "FrameYap"}}}]}
def check_owned_file(path, expected):
if path.is_symlink():
fail(f"refusing foreign symlink: {path}")
if path.exists():
if not path.is_file():
fail(f"refusing foreign path: {path}")
data = path.read_bytes()
if data != expected:
fail(f"refusing to replace modified/foreign file: {path}")
def check_wrappers(root, launcher):
manifest = root / "frameyap.vrmanifest"
desired = (json.dumps(desired_manifest(launcher), sort_keys=True, indent=2) + "\n").encode()
check_owned_file(launcher, desired_launcher(root))
check_owned_file(manifest, desired)
return manifest, desired
def install_files(root, installed, launcher):
owned_dir(launcher.parent)
manifest, desired = check_wrappers(root, launcher)
atomic_write(launcher, desired_launcher(root), 0o755)
atomic_write(manifest, desired)
def receipt(path):
marker = path / ".archive-sha256"
if marker.is_symlink() or not marker.is_file() or not DIGEST_RE.fullmatch(marker.read_text().strip()):
fail(f"refusing unowned installation directory: {path}")
return marker.read_text().strip()
def inventory(path):
return sorted(p.relative_to(path).as_posix() for p in path.rglob("*")
if p.relative_to(path).parts[0] != "model"
and p.relative_to(path).as_posix() not in (".archive-sha256", ".installed-files.json", ".install-options.json"))
def check_inventory(path):
index = path / ".installed-files.json"
if not index.is_file() or index.is_symlink():
fail(f"missing installation inventory: {path}")
declared = json.loads(index.read_text())
if not isinstance(declared, list) or inventory(path) != declared:
fail(f"refusing to remove untracked files in: {path}")
for p in path.rglob("*"):
if p.is_symlink() and p.relative_to(path).parts[0] != "model":
fail(f"refusing link inside installation: {p}")
def installed_choice(path):
receipt(path)
options = path / ".install-options.json"
if not options.is_file() or options.is_symlink():
fail(f"missing installation options: {path}")
choice = json.loads(options.read_text())
if not isinstance(choice, dict) or set(choice) != {"without_model"} or not isinstance(choice["without_model"], bool):
fail(f"invalid installation options: {path}")
return choice["without_model"]
def do_install(args, root, launcher):
version = check_version(args.version)
versions = root / "versions"
owned_dir(versions)
current = selected(root, "current")
selected(root, "previous")
# Refuse foreign wrappers/launcher directories before installing a new version.
owned_dir(launcher.parent)
check_wrappers(root, launcher)
if args.archive:
archive = Path(args.archive).expanduser().resolve(strict=True)
expected = check_digest(args.sha256)
do_download = False
else:
do_download = True
archive = None
with tempfile.TemporaryDirectory(prefix=".download-", dir=root) as td:
if do_download:
name = release_name(version)
url = f"https://github.com/{args.repo}/releases/download/{quote(version)}/{name}"
archive = Path(td) / name
sidecar = Path(td) / (name + ".sha256")
download(url + ".sha256", sidecar)
if sidecar.stat().st_size > 4096:
fail("oversized SHA-256 sidecar")
line = sidecar.read_text().strip()
match = re.fullmatch(r"([a-fA-F0-9]{64}) " + re.escape(name), line)
if not match:
fail("invalid versioned SHA-256 sidecar")
expected = match.group(1).lower()
download(url, archive)
if archive.stat().st_size > ARCHIVE_LIMIT:
fail("compressed archive exceeds limit")
if not do_download:
local_copy = Path(td) / "local-archive.tar.gz"
shutil.copyfile(archive, local_copy)
archive = local_copy
if digest_file(archive) != expected:
fail("archive SHA-256 mismatch")
target = versions / version
if target.exists() or target.is_symlink():
if target.is_symlink() or receipt(target) != expected:
fail("same version already installed with different archive digest")
if installed_choice(target) != args.without_model:
fail("same version already installed with different --without-model choice")
check_inventory(target)
validate_payload(target, version, args.without_model, installed=True)
else:
temp = Path(tempfile.mkdtemp(prefix=".staging-", dir=versions))
try:
extract(archive, temp)
validate_payload(temp, version)
if args.without_model and (temp / "model").exists():
shutil.rmtree(temp / "model")
json_atomic(temp / ".installed-files.json", inventory(temp))
json_atomic(temp / ".install-options.json", {"without_model": args.without_model})
(temp / ".archive-sha256").write_text(expected + "\n")
os.replace(temp, target)
finally:
if temp.exists():
shutil.rmtree(temp)
install_files(root, target, launcher)
if current == f"versions/{version}":
print(f"FrameYap {version}: already installed (same digest); wrappers verified")
return
if current and current != f"versions/{version}":
select(root, "previous", current)
select(root, "current", f"versions/{version}")
print(f"FrameYap {version} installed. OpenVR registration is NOT automatic; see docs/packaging.md.")
if json.loads((target / "release.json").read_text())["runtime"] == "external-authorized-python":
print("ASR runtime is NOT included. Supply an independently authorized environment with FRAMEYAP_PYTHON or --python; no packages are downloaded.")
def uninstall(root, launcher):
current = selected(root, "current")
previous = selected(root, "previous")
manifest = root / "frameyap.vrmanifest"
desired = (json.dumps(desired_manifest(launcher), sort_keys=True, indent=2) + "\n").encode()
check_owned_file(manifest, desired)
check_owned_file(launcher, desired_launcher(root))
versions = root / "versions"
if versions.exists():
if versions.is_symlink():
fail("refusing symlink versions directory")
for item in versions.iterdir():
if not item.is_dir() or item.is_symlink() or item.name.startswith("."):
fail(f"unexpected versions entry: {item}")
choice = installed_choice(item)
check_inventory(item)
validate_payload(item, item.name, choice, installed=True)
if (item / "model").exists() or (item / "model").is_symlink():
saved = root / "saved-models"
owned_dir(saved)
target = saved / item.name
if target.exists() or target.is_symlink():
fail(f"saved model already exists: {target}")
for item in versions.iterdir():
if (item / "model").exists() or (item / "model").is_symlink():
os.replace(item / "model", root / "saved-models" / item.name)
shutil.rmtree(item)
for link in ("current", "previous"):
(root / link).unlink(missing_ok=True)
if launcher.exists():
launcher.unlink()
if manifest.exists():
manifest.unlink()
print("FrameYap removed; config and saved models preserved. OpenVR unregister acknowledgement was required.")
def main(argv=None):
parser = argparse.ArgumentParser(prog="install.sh", description="User-local FrameYap release installer (no SteamVR actions)")
mode = parser.add_mutually_exclusive_group()
mode.add_argument("--archive", help="local release archive (requires --sha256 and --version)")
mode.add_argument("--rollback", action="store_true")
mode.add_argument("--uninstall", action="store_true")
parser.add_argument("--sha256")
parser.add_argument("--version")
parser.add_argument("--repo", default="baketnk/frame-yap")
parser.add_argument("--without-model", action="store_true")
parser.add_argument("--unregistered", action="store_true", help="acknowledge explicit OpenVR removal before uninstall")
args = parser.parse_args(argv)
if args.repo != "baketnk/frame-yap":
parser.error("only the pinned baketnk/frame-yap release repository is supported")
if args.archive and (not args.sha256 or not args.version):
parser.error("--archive requires --sha256 and --version")
if not args.archive and args.sha256:
parser.error("--sha256 only applies to --archive")
if not (args.rollback or args.uninstall or args.archive) and not args.version:
parser.error("--version TAG is required; no moving/latest release")
if args.uninstall and not args.unregistered:
parser.error("uninstall requires --unregistered after explicit OpenVR unregister")
if args.unregistered and not args.uninstall:
parser.error("--unregistered only applies to --uninstall")
if args.version:
check_version(args.version)
check_host()
data = Path(os.environ.get("XDG_DATA_HOME") or Path.home() / ".local/share").expanduser().absolute()
root = data / "frameyap"
launcher = Path.home() / ".local/bin/frameyap"
owned_dir(root)
lock = root / ".lock"
if lock.is_symlink():
fail("refusing symlink lock")
with lock.open("a+b") as fd:
try:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
fail("FrameYap installer or application is running (.lock held); try again later")
if args.uninstall:
uninstall(root, launcher)
elif args.rollback:
owned_dir(root / "versions")
current, previous = selected(root, "current"), selected(root, "previous")
if not current or not previous:
fail("no previous installation to roll back to")
check_wrappers(root, launcher)
choice = installed_choice(root / previous)
check_inventory(root / previous)
validate_payload(root / previous, Path(previous).name, choice, installed=True)
select(root, "current", previous)
select(root, "previous", current)
print(f"Rolled back to {previous}")
else:
do_install(args, root, launcher)
if __name__ == "__main__":
try:
main()
except (ValueError, OSError, subprocess.CalledProcessError, tarfile.TarError, json.JSONDecodeError) as exc:
print(f"frameyap installer: {exc}", file=sys.stderr)
sys.exit(1)
+107
View File
@@ -0,0 +1,107 @@
#!/usr/bin/env python3
"""Produce an offline release archive from an independently vetted ARM64 staging tree.
This tool does not build/download a runtime, model, native libraries, or licenses.
"""
import argparse
import hashlib
import io
import json
import os
from pathlib import Path
import re
import sys
import tarfile
import tempfile
ARCH = "linux-aarch64"
ALLOWED = {"bin", "lib", "assets", "python", "runtime", "model", "fonts", "licenses"}
REQUIRED = ("bin/frameyap", "runtime/bin/python3", "python/frameyap/worker.py",
"assets/actions.json", "fonts/font.ttf", "licenses/THIRD_PARTY_NOTICES.txt")
def main(argv=None):
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--stage", type=Path, required=True, help="vetted standalone payload directory")
p.add_argument("--output", type=Path, required=True, help="existing output directory")
p.add_argument("--version", required=True)
p.add_argument("--arch", choices=[ARCH], required=True)
p.add_argument("--model-revision", required=True, help="exact vetted model revision identifier")
p.add_argument("--external-runtime", action="store_true", help="omit ASR runtime; user must supply an independently authorized Python environment")
args = p.parse_args(argv)
if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9._-]{0,95}", args.version) or args.version in (".", ".."):
p.error("invalid version")
if not args.model_revision.strip():
p.error("model revision must be nonempty")
stage = args.stage.resolve(strict=True)
if not stage.is_dir() or args.stage.is_symlink():
p.error("stage must be a real directory")
if not args.output.is_dir() or args.output.is_symlink():
p.error("output must be an existing real directory")
if any(item.name not in ALLOWED for item in stage.iterdir()):
p.error("stage has unknown root entries or preexisting release.json")
for name in REQUIRED:
if args.external_runtime and name == "runtime/bin/python3":
continue
if not (stage / name).is_file() or (stage / name).is_symlink():
p.error(f"missing file: {name}")
if args.external_runtime and (stage / "runtime").exists():
p.error("external-runtime package must not contain a runtime directory")
for name in (("bin/frameyap",) if args.external_runtime else ("bin/frameyap", "runtime/bin/python3")):
if not os.access(stage / name, os.X_OK):
p.error(f"not executable: {name}")
for name in ("lib", "fonts", "licenses"):
if not (stage / name).is_dir() or not any((stage / name).iterdir()):
p.error(f"missing directory or empty: {name}")
if not (stage / "licenses/THIRD_PARTY_NOTICES.txt").read_text().strip():
p.error("third-party notices must not be empty; review redistribution licenses")
if (stage / "model").exists() and not any((stage / "model").rglob("*")):
p.error("model directory is empty")
entries = sorted(stage.rglob("*"), key=lambda item: item.relative_to(stage).as_posix())
if len(entries) > 49999:
p.error("too many payload members")
total = 0
for entry in entries:
name = entry.relative_to(stage).as_posix()
if (len(name) > 1024 or len(name.split("/")) > 32 or "\\" in name
or name.endswith("/") or any(part in ("", ".", "..") for part in name.split("/"))):
p.error(f"unsupported archive member path: {name}")
if entry.is_symlink() or not (entry.is_dir() or entry.is_file()):
p.error(f"links and special files forbidden (copy vetted files into stage): {entry}")
if entry.is_file():
total += entry.stat().st_size
if total > 12 * 1024**3:
p.error("payload exceeds 12 GiB uncompressed limit")
name = f"frameyap-{args.version}-{ARCH}.tar.gz"
target = args.output / name
sidecar = args.output / (name + ".sha256")
if target.exists() or sidecar.exists() or target.is_symlink() or sidecar.is_symlink():
p.error("release output exists; refusing to replace it")
meta = {"schema": 1, "version": args.version, "arch": ARCH,
"runtime": "external-authorized-python" if args.external_runtime else "bundled-cpu-python", "model_revision": args.model_revision,
"includes_model": (stage / "model").is_dir()}
fd, temp = tempfile.mkstemp(prefix=".frameyap-package-", dir=args.output)
os.close(fd)
try:
with tarfile.open(temp, "w:gz", format=tarfile.PAX_FORMAT, compresslevel=1) as tar:
payload = (json.dumps(meta, sort_keys=True, indent=2) + "\n").encode()
info = tarfile.TarInfo("release.json")
info.size = len(payload)
info.mode = 0o600
tar.addfile(info, io.BytesIO(payload))
for entry in entries:
tar.add(entry, arcname=entry.relative_to(stage).as_posix(), recursive=False)
digest = hashlib.sha256()
with open(temp, "rb") as stream:
for chunk in iter(lambda: stream.read(1024 * 1024), b""):
digest.update(chunk)
os.replace(temp, target)
sidecar.write_text(f"{digest.hexdigest()} {name}\n")
print(f"Wrote {target} and {sidecar}")
finally:
if os.path.exists(temp):
os.unlink(temp)
if __name__ == "__main__":
main()
+51
View File
@@ -0,0 +1,51 @@
#!/usr/bin/env python3
"""Stage a native-only POC from an explicit native build and licensed files.
No downloads, compiler invocation, proprietary ASR runtime, registration or launch.
System Wayland/FreeType/libstdc++/glibc remain platform prerequisites.
"""
import argparse
from pathlib import Path
import shutil
import subprocess
def main():
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--build", type=Path, required=True)
p.add_argument("--destination", type=Path, required=True, help="new staging directory")
for name in ("openvr-library", "openvr-license", "sdl-library", "sdl-license", "font", "font-license"):
p.add_argument("--" + name, type=Path, required=True)
args = p.parse_args()
root = Path(__file__).resolve().parents[1]
dest = args.destination.absolute()
if dest.exists() or dest.is_symlink():
p.error("destination already exists; use a new staging directory")
for name in ("openvr_library", "openvr_license", "sdl_library", "sdl_license", "font", "font_license"):
if not getattr(args, name).is_file():
p.error(f"missing explicit input {name}")
cache = (args.build / "CMakeCache.txt").read_text()
if "FRAMEYAP_NATIVE:BOOL=ON" not in cache:
p.error("build must explicitly enable FRAMEYAP_NATIVE")
dest.mkdir(mode=0o700, parents=True)
subprocess.run(["cmake", "--install", str(args.build.absolute()), "--prefix", str(dest)], check=True)
for directory in ("lib", "fonts", "licenses"):
(dest / directory).mkdir()
shutil.copyfile(args.openvr_library, dest / "lib/libopenvr_api.so", follow_symlinks=True)
shutil.copyfile(args.sdl_library, dest / "lib/libSDL3.so.0", follow_symlinks=True)
shutil.copyfile(args.font, dest / "fonts/font.ttf")
notices = ["FrameYap native-only POC. No ASR runtime or model is included.\n",
"Original FrameYap code: MIT. System Wayland/FreeType/libstdc++/glibc are not bundled.\n",
"Bundled libraries: Valve OpenVR and unmodified SDL3; font supplied explicitly below.\n",
"This package does not grant any rights to kestrel-kernels or provide a functioning ASR environment.\n"]
for label, file in (("FrameYap", root / "LICENSE"), ("OpenVR", args.openvr_license),
("SDL3", args.sdl_license), ("Font", args.font_license)):
notices.extend([f"\n--- {label} ---\n", file.read_text()])
notices.extend(["\n--- Gamescope protocol: embedded copyright/license ---\n",
(root / "protocol/gamescope-input-method.xml").read_text()])
(dest / "licenses/THIRD_PARTY_NOTICES.txt").write_text("\n".join(notices))
print(f"Native-only stage: {dest}. Package with --external-runtime; do not claim bundled ASR.")
if __name__ == "__main__":
main()
+10
View File
@@ -0,0 +1,10 @@
#!/usr/bin/env python3
"""Developer helper: synchronize the standalone piped installer payload."""
from pathlib import Path
root = Path(__file__).resolve().parents[1]
wrapper = root / "install.sh"
header = wrapper.read_text().split("<<'PY'\n", 1)[0] + "<<'PY'\n"
payload = (root / "scripts/install_payload.py").read_text()
if "\nPY\n" in payload:
raise ValueError("heredoc delimiter appears in payload")
wrapper.write_text(header + payload + "\nPY\n")
+285
View File
@@ -0,0 +1,285 @@
"""Offline installer/packager checks: temporary HOME only, no hardware/network."""
import contextlib
import fcntl
import hashlib
import importlib.util
import io
import json
import os
from pathlib import Path
import subprocess
import sys
import tarfile
import tempfile
import unittest
from unittest.mock import patch
REPO = Path(__file__).resolve().parents[1]
SPEC = importlib.util.spec_from_file_location("install_payload", REPO / "scripts/install_payload.py")
installer = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(installer)
class InstallTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.addCleanup(self.temp.cleanup)
self.base = Path(self.temp.name)
self.home = self.base / "home"
self.home.mkdir()
self.data = self.base / "data"
self.output = self.base / "out"
self.output.mkdir()
self.stage = self.base / "stage"
for name in ("bin/frameyap", "runtime/bin/python3", "runtime/bin/helper",
"python/frameyap/worker.py", "assets/actions.json", "fonts/font.ttf", "lib/libtest.so",
"licenses/THIRD_PARTY_NOTICES.txt", "model/weights.bin"):
path = self.stage / name
path.parent.mkdir(exist_ok=True, parents=True)
path.write_bytes((name + " fixture\n").encode())
for name in ("bin/frameyap", "runtime/bin/python3", "runtime/bin/helper", "lib/libtest.so"):
(self.stage / name).chmod(0o755)
self.env = patch.dict(os.environ, {"HOME": str(self.home), "XDG_DATA_HOME": str(self.data)})
self.env.start()
self.addCleanup(self.env.stop)
self.host = patch.object(installer, "check_host")
self.host.start()
self.addCleanup(self.host.stop)
def package(self, version, *extra):
subprocess.run([sys.executable, str(REPO / "scripts/package-release.py"), "--stage", str(self.stage),
"--output", str(self.output), "--arch", "linux-aarch64", "--version", version,
"--model-revision", "test-revision", *extra], check=True, capture_output=True)
archive = self.output / installer.release_name(version)
return archive, hashlib.sha256(archive.read_bytes()).hexdigest()
def install(self, version, archive, sha, *extra):
with contextlib.redirect_stdout(io.StringIO()):
installer.main(["--archive", str(archive), "--sha256", sha, "--version", version, *extra])
def test_embedded_installer_is_current(self):
wrapper = (REPO / "install.sh").read_text()
self.assertEqual(wrapper.split("<<'PY'\n", 1)[1].removesuffix("\nPY\n"),
(REPO / "scripts/install_payload.py").read_text())
def test_install_idempotence_upgrade_rollback_uninstall(self):
a1, h1 = self.package("v1")
self.install("v1", a1, h1)
root = self.data / "frameyap"
self.assertEqual(os.readlink(root / "current"), "versions/v1")
launcher = self.home / ".local/bin/frameyap"
self.assertIn("--run", launcher.read_text())
self.assertIn("PYTHONDONTWRITEBYTECODE=1", launcher.read_text())
self.assertTrue(os.access(root / "versions/v1/runtime/bin/helper", os.X_OK))
self.assertTrue(os.access(root / "versions/v1/lib/libtest.so", os.X_OK))
manifest = json.loads((root / "frameyap.vrmanifest").read_text())
self.assertEqual(manifest["applications"][0]["app_key"], "local.frameyap.overlay")
self.assertEqual(manifest["applications"][0]["binary_path_linux"], str(launcher))
self.assertEqual(manifest["applications"][0]["binary_path_linux_arm"], str(launcher))
self.assertNotIn("binary_path", manifest["applications"][0])
(root / "config-untouched").write_text("keep")
self.install("v1", a1, h1)
(self.stage / "bin/frameyap").write_text("next binary")
a2, h2 = self.package("v2")
self.install("v2", a2, h2)
self.assertEqual(os.readlink(root / "current"), "versions/v2")
self.assertEqual(os.readlink(root / "previous"), "versions/v1")
(root / "frameyap.vrmanifest").write_text("foreign")
with self.assertRaisesRegex(ValueError, "foreign file"):
installer.main(["--rollback"])
self.assertEqual(os.readlink(root / "current"), "versions/v2")
(root / "frameyap.vrmanifest").unlink()
with contextlib.redirect_stdout(io.StringIO()):
installer.main(["--rollback"])
self.assertEqual(os.readlink(root / "current"), "versions/v1")
self.assertEqual(os.readlink(root / "previous"), "versions/v2")
with contextlib.redirect_stderr(io.StringIO()), self.assertRaises(SystemExit):
installer.main(["--uninstall"])
with contextlib.redirect_stdout(io.StringIO()):
installer.main(["--uninstall", "--unregistered"])
self.assertEqual((root / "config-untouched").read_text(), "keep")
self.assertTrue((root / "saved-models/v1/weights.bin").exists())
self.assertTrue((root / "saved-models/v2/weights.bin").exists())
self.assertFalse(launcher.exists())
def test_digest_and_same_version_mismatch_leave_previous(self):
a, h = self.package("v1")
self.install("v1", a, h)
root = self.data / "frameyap"
with self.assertRaisesRegex(ValueError, "SHA-256 mismatch"):
self.install("v1", a, "0" * 64)
a.unlink()
(self.output / (a.name + ".sha256")).unlink()
(self.stage / "bin/frameyap").write_text("changed")
a, h2 = self.package("v1")
with self.assertRaisesRegex(ValueError, "different archive digest"):
self.install("v1", a, h2)
self.assertEqual(os.readlink(root / "current"), "versions/v1")
def test_without_model_lock_and_foreign_launcher(self):
a, h = self.package("v1")
lock = self.data / "frameyap/.lock"
lock.parent.mkdir(parents=True)
with lock.open("a+b") as fd:
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
with self.assertRaisesRegex(ValueError, "running"):
self.install("v1", a, h)
self.install("v1", a, h, "--without-model")
root = self.data / "frameyap"
self.assertFalse((root / "versions/v1/model").exists())
self.assertTrue((self.stage / "model/weights.bin").exists())
self.assertEqual(json.loads((root / "versions/v1/.install-options.json").read_text()),
{"without_model": True})
self.install("v1", a, h, "--without-model")
self.assertFalse((root / "versions/v1/model").exists())
with self.assertRaisesRegex(ValueError, "different --without-model choice"):
self.install("v1", a, h)
self.assertFalse((root / "versions/v1/model").exists())
(root / "frameyap.vrmanifest").unlink()
launcher = self.home / ".local/bin/frameyap"
launcher.unlink()
self.install("v1", a, h, "--without-model")
self.assertTrue(launcher.exists())
self.assertTrue((root / "frameyap.vrmanifest").exists())
(root / "frameyap.vrmanifest").write_text("foreign")
with self.assertRaisesRegex(ValueError, "foreign file"):
self.install("v1", a, h, "--without-model")
(root / "frameyap.vrmanifest").unlink()
self.install("v1", a, h, "--without-model")
(self.home / ".local/bin/frameyap").write_text("#!/bin/sh\n" + installer.MARKER + "echo foreign\n")
(self.stage / "bin/frameyap").write_text("new")
a2, h2 = self.package("v2")
with self.assertRaisesRegex(ValueError, "foreign file"):
self.install("v2", a2, h2)
self.assertEqual(os.readlink(self.data / "frameyap/current"), "versions/v1")
self.assertFalse((self.data / "frameyap/versions/v2").exists())
def test_traversal_and_link_archives_rejected(self):
a, h = self.package("v1")
self.install("v1", a, h)
for badname, kind in (("../outside", "file"), ("bin/escape", "symlink"),
("/absolute", "file"), ("bin/escape", "hardlink")):
with self.subTest(badname=badname, kind=kind):
bad = self.base / "bad.tar.gz"
with tarfile.open(bad, "w:gz") as tar:
info = tarfile.TarInfo(badname)
if kind != "file":
info.type = tarfile.SYMTYPE if kind == "symlink" else tarfile.LNKTYPE
info.linkname = "../../outside"
tar.addfile(info)
else:
info.size = 1
tar.addfile(info, io.BytesIO(b"x"))
sha = hashlib.sha256(bad.read_bytes()).hexdigest()
with self.assertRaisesRegex(ValueError, "unsafe archive|forbidden"):
self.install("v2", bad, sha)
self.assertEqual(os.readlink(self.data / "frameyap/current"), "versions/v1")
self.assertFalse((self.base / "outside").exists())
def test_unexpected_root_entries_and_oversized_metadata_rejected(self):
for name, size in (("release.json/child", 1), ("bin", 1), ("release.json", 8193)):
with self.subTest(name=name, size=size):
bad = self.base / "bad.tar.gz"
with tarfile.open(bad, "w:gz") as tar:
info = tarfile.TarInfo(name)
info.size = size
tar.addfile(info, io.BytesIO(b"x" * size))
sha = hashlib.sha256(bad.read_bytes()).hexdigest()
with self.assertRaisesRegex(ValueError, "unexpected archive path|oversized release metadata"):
self.install("v1", bad, sha)
def test_uninstall_refuses_untracked_files(self):
a, h = self.package("v1")
self.install("v1", a, h)
root = self.data / "frameyap"
extra = root / "versions/v1/user.txt"
extra.write_text("preserve")
with self.assertRaisesRegex(ValueError, "untracked files"):
with contextlib.redirect_stdout(io.StringIO()):
installer.main(["--uninstall", "--unregistered"])
self.assertTrue(extra.exists())
self.assertEqual(os.readlink(root / "current"), "versions/v1")
def test_release_metadata_mismatch_retains_current(self):
a, h = self.package("v1")
self.install("v1", a, h)
a2, h2 = self.package("v2")
with self.assertRaisesRegex(ValueError, "metadata version/architecture/schema mismatch"):
self.install("v3", a2, h2)
self.assertEqual(os.readlink(self.data / "frameyap/current"), "versions/v1")
def test_launcher_defaults_run_but_forwards_registration(self):
path = self.stage / "bin/frameyap"
path.write_text('#!/bin/sh\nprintf "%s\\n" "$@"\nprintf "ENV:%s\\n" "${PYTHONDONTWRITEBYTECODE:-}"\n')
path.chmod(0o755)
a, h = self.package("v1")
self.install("v1", a, h)
launcher = self.home / ".local/bin/frameyap"
reg = subprocess.run([str(launcher), "--register", "test-manifest"], capture_output=True, text=True, check=True)
self.assertEqual(reg.stdout.splitlines(), ["--register", "test-manifest", "ENV:1"])
run = subprocess.run([str(launcher)], capture_output=True, text=True, check=True,
env={**os.environ, "GAMESCOPE_SOCKET": "fixture-socket"})
self.assertEqual(run.stdout.splitlines()[0], "--run")
self.assertIn("--assets", run.stdout)
self.assertIn("--socket\nfixture-socket\n", run.stdout)
self.assertIn("ENV:1", run.stdout)
self.assertEqual(json.loads((self.data / "frameyap/versions/v1/.install-options.json").read_text()),
{"without_model": False})
with self.assertRaisesRegex(ValueError, "different --without-model choice"):
self.install("v1", a, h, "--without-model")
self.assertTrue((self.data / "frameyap/versions/v1/model/weights.bin").exists())
def test_no_model_reinstall_preserves_provisioned_model_and_uninstall(self):
a, h = self.package("v1")
self.install("v1", a, h, "--without-model")
root = self.data / "frameyap"
model = root / "versions/v1/model"
model.mkdir()
(model / "provided.bin").write_text("user-provided")
self.install("v1", a, h, "--without-model")
with contextlib.redirect_stdout(io.StringIO()):
installer.main(["--uninstall", "--unregistered"])
self.assertEqual((root / "saved-models/v1/provided.bin").read_text(), "user-provided")
def test_version_switch_rejects_untracked_installed_files(self):
a, h = self.package("v1")
self.install("v1", a, h)
(self.data / "frameyap/versions/v1/untracked").write_text("keep")
a2, h2 = self.package("v2")
with self.assertRaisesRegex(ValueError, "untracked files"):
self.install("v1", a, h)
# Upgrade does not delete the old directory, but uninstall must still refuse it.
self.install("v2", a2, h2)
with self.assertRaisesRegex(ValueError, "untracked files"):
installer.main(["--uninstall", "--unregistered"])
def test_external_authorized_runtime_is_explicit(self):
import shutil
shutil.rmtree(self.stage / "runtime")
native = self.stage / "bin/frameyap"
native.write_text('#!/bin/sh\nprintf "%s\\n" "$@"\n')
native.chmod(0o755)
a, h = self.package("external", "--external-runtime")
self.install("external", a, h)
root = self.data / "frameyap/current"
self.assertEqual(json.loads((root / "release.json").read_text())["runtime"], "external-authorized-python")
self.assertFalse((root / "runtime").exists())
launcher = self.home / ".local/bin/frameyap"
run = subprocess.run([str(launcher)], capture_output=True, text=True, check=True,
env={**os.environ, "FRAMEYAP_PYTHON": "/authorized/python", "FRAMEYAP_MODEL": "/local/model"})
self.assertIn("--python\n/authorized/python\n--model\n/local/model\n", run.stdout)
probe = subprocess.run([str(launcher), "--check-controls", "--head"], capture_output=True, text=True, check=True)
self.assertTrue(probe.stdout.startswith("--check-controls\n--assets\n"))
self.assertNotIn("--python", probe.stdout)
def test_package_rejects_symlink(self):
(self.stage / "lib/link.so").symlink_to("libtest.so")
result = subprocess.run([sys.executable, str(REPO / "scripts/package-release.py"), "--stage", str(self.stage),
"--output", str(self.output), "--arch", "linux-aarch64", "--version", "v1",
"--model-revision", "test"], capture_output=True, text=True)
self.assertNotEqual(result.returncode, 0)
self.assertIn("links and special files forbidden", result.stderr)
if __name__ == "__main__":
unittest.main()