diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c652dd..742b179 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -396,6 +396,64 @@ code the task overlay's existing password prompt already reacts to. - End-to-end validation of the built ELF on a real console. +## [v1.9.2] — 2026-09-05 + +**Version-string-only re-release: the tag now points at the tree that produced +the published binary.** + +The `v1.9.1` tag sat four commits behind the tree its ELF was built from, so +cloning that tag could not rebuild the published artifact. v1.9.2 is cut from +the right commit. It is functionally identical to the v1.9.1 binary — the only +source delta is the version literal itself (`VERSION_TAG` in the Makefile, plus +the UI footer fallback in `assets/main.js`) — and the build is reproducible: +reverting those two literals reproduces the v1.9.1 ELF byte for byte. + +Release artifact: `web-file-mgr-v1.9.2.elf` — 870 488 bytes (~850 KiB), sha256 +`177e90fecf93a0251e83f67884fba4551051be248330b0d70fda8ea732f88e84`. + +## [v1.9.1] — 2026-09-05 + +**7z extraction — a third engine — plus a size and throughput pass.** + +Added: + +- `src/sevenz_extract.{c,h}` — the 7z engine, built on the LZMA SDK 26.03 + decode subset plus the project's own pull-based codec chain + (`src/sevenz_chain.c`). The SDK's own `SzArEx` path only understands folders + of up to four coders, which cannot express BCJ2's five — hence the + self-parsed folder table and the pull-based chain. Dispatch is by extension + in `src/extract.c`; the three-phase model, the limit profiles and the + conflict policy are shared with ZIP and RAR, so `.7z` files get the same + **Extract** button as `.zip` and `.rar`. +- `src/sevenz_volstream.{c,h}` — `.7z.001` / `.z01` byte-split volume sets, + stitched by name; open the first volume. +- 7zAES content decryption (AES-256-CBC). The frontend asks for the password + *up front* here, so an unencrypted archive does not pay for a wasted scan. + +Performance (decode-only, no functional change): + +- LZMA SDK assembly decoder (`Asm/x86/LzmaDecOpt.asm` assembled with jwasm, + with an automatic pure-C fallback) ≈ 1.26×. +- Single-coder pure-LZMA2 folders decode multi-threaded + (`Lzma2DecMt` via `src/sevenz_mt.c`, 8 threads) ≈ 1.37×. +- The extract path drops its per-entry `fsync` — publish is rename-only and + there is no resume feature to protect (≥ 14× measured on an 8000-file + archive; see `docs/EXTRACTION-PERF.md`). + +Build and size: + +- `VERSION_TAG` v1.9.1. `src/demangle_stub.c` keeps libc++abi's Itanium name + demangler (105 KiB, reachable only from the uncaught-exception path) out of + the link, and `-Wl,--icf=all` folds identical functions: −15.8% overall with + no functional or throughput change, 1 017 864 B → 870 488 B. Measured in + `docs/SIZE-OPTIMIZATION.md`. + +Known gap at the time: 7z `-mhe=on` encrypted headers — closed in v1.9.3M with +`src/sevenz_header.c`. + +Tests: **163 checks** (ZIP 108 + RAR 27 + 7z 28), 0 failures, plus a successful +PS5 cross-compile. + ## [v1.9] — 2026-09-05 **RAR engine replaced: rarlab UnRAR 7.20.1 (v6 / multi-volume / decryption-capable).** @@ -761,7 +819,9 @@ python3 .build/check-elf-gzip.py ./web-file-mgr.elf # 7/7 v1.7 keys + 1 v A long-form technical write-up of this upgrade lives in [`docs/UPGRADE-v1.8-rar-support.md`](./docs/UPGRADE-v1.8-rar-support.md). The vendoring decision tree (and the v1.9 plan) is in -[`third_party/unrar/VENDORED.md`](./third_party/unrar/VENDORED.md). +[`third_party/unrar7/VENDORED.md`](./third_party/unrar7/VENDORED.md) — v1.8 +shipped it at `third_party/unrar/VENDORED.md`; the directory was renamed in +v1.9 when the engine was replaced. ### Credits diff --git a/HANDOVER.md b/HANDOVER.md index 92dbb5d..abb6401 100644 --- a/HANDOVER.md +++ b/HANDOVER.md @@ -1,6 +1,6 @@ # 交接文档 — ps5-web-file-manager 工作进度 -> 交接时间：2026-09-24 · 分支 `main` · 最新提交 **`ba668ad`**（tag + Release **`v1.9.3M`**，已推送）——此前那一轮加密改动已全部提交并发布 +> 交接时间：2026-09-25 · 分支 `main` · 最新提交 **`770dcb8`**（「docs: mark v1.9.3M as published」）——tag + Release **`v1.9.3M`** 指向发布提交 `ba668ad`，三者均已推送 > > **主线（用户 2026-09-12 指令）**：「先从 zip 分卷开始吧，然后把六种组合打齐，并把密码通道补齐，注意一些报错信息提示的时候尽量详细准确」 > **状态：主线全部闭合，格式面已无已知缺口，且已发版。** 六种组合（ZIP/RAR/7z × 单卷/分卷）+ 密码通道（ZIP ZipCrypto/AES + RAR `-p`/`-hp` + 7zAES **含 `-mhe=on` 加密头**）+ 报错详细信息，全部落地、测试全绿、PS5 ELF 构建成功。 @@ -19,6 +19,7 @@ | 上一个发布产物（回滚点） | `web-file-mgr-v1.9.2.elf` · 870,488 B · sha256 `177e90fecf93a0251e83f67884fba4551051be248330b0d70fda8ea732f88e84`（不含加密改动） | | GitHub | `main` = `ba668ad`、tag `v1.9.3M`、Release `v1.9.3M` 三者均已推送/发布，资产 sha256 与本地逐字节一致 | | 已知功能缺口 | **无**（`-mhe=on` 已于 2026-09-23 补齐） | +| README | 2026-09-25 **重写为中英双语「功能导向」结构**（原为上游式的逐版本累加稿）：删去六段 `What's new in vX.Y.Z`，改为 *功能 / 压缩包支持矩阵 / 限额与安全 / 快速上手 / 构建 / 使用 / 校验 / 测试 / 项目结构 / **与上游的差异** / 备注 / FAQ / 版本历史*；逐版本正文已迁入 `CHANGELOG.md`（并补上了原先缺正文的 `[v1.9.1]` `[v1.9.2]` 两节）。**注意**：上游 README 里的 `.elf` 一键启动（`localhost:9021`）**本仓源码中已不存在**，重写时未回填 | ### 发布命令（v1.9.2） diff --git a/README.md b/README.md index 89023a3..97d4711 100644 --- a/README.md +++ b/README.md @@ -4,216 +4,50 @@ # PS5 Web File Manager -> Homebrew HTTP file manager for jailbroken PS5 consoles. Browse, edit, upload, download and extract ZIPs through any browser on the same network — single self-contained ELF payload, no external services, no telemetry. +

+ Latest release + License + Target platform: x86_64-sie-ps5 + Total downloads +

+ +

+ Download the ELF payload + Beginner's guide (Chinese) +

+ +> A homebrew HTTP file manager for jailbroken PS5 consoles. Browse, edit, upload, +> download and extract archives from any browser on the same network — one +> self-contained ELF payload, no external helper file, no telemetry. **Version:** v1.9.3M · **Title ID:** `FMGR88888` · **License:** GPLv3+ · **Target:** `x86_64-sie-ps5` +**Download:** [latest release](https://github.com/LisherSong/ps5-web-file-manager/releases/latest) · **First time here?** [Beginner's guide (中文)](docs/USER-GUIDE-zh-CN.md) + --- -## Overview +## What it is -A payload ELF that runs an HTTP file manager inside a jailbroken PS5. Open `http://:8888/` from any browser on the LAN — including the PS5 browser itself — to manage files on attached USB storage and the user partition. Designed for safely copying game-dump folders from USB to internal storage, but it also handles general file management, in-place text editing, PKG preview/install, image preview, and ZIP extraction with built-in zip-bomb protection. +A single payload ELF that runs an HTTP file manager inside a jailbroken PS5. +Send it to the console's ELF loader, and the console starts an HTTP service on +port `8888` (it walks up to the next free port if that one is taken). Open +`http://:8888/` from any browser on the LAN — including the PS5's own +browser — and manage files on attached USB storage and the user partition. -The same source tree builds a Linux binary for development and a PS5 payload ELF for deployment — see `make linux` below. +It was written to make one job safe and fast: **copying game-dump folders from +USB storage to internal storage.** Everything else it does — browsing, sorting, +permissions, in-place text editing, image preview, PKG install, multi-select +copy/move/delete, upload and download — exists to make that job practical. On +top of that this fork adds **native archive extraction** for ZIP, RAR and 7z, +with the safety rails ("zip bomb", path traversal, disk-full) that the upstream +helper approach does not have. -## What's new in v1.9.3M +The same source tree also builds a Linux binary, so the whole UI can be worked +on without a console or the PS5 SDK: -- **Encrypted ZIP extraction works end to end** — both schemes: - - traditional PKWARE ("ZipCrypto", what `zip -e` writes), and - - WinZip AES-128/192/256 (compression method `99` plus the `0x9901` extra - field, what `7z -mem=AES256` and WinZip write), for stored and deflated - entries. - - A missing or wrong password is reported as `extract_password` — the code the - task overlay already knew how to translate, even though until now nothing - could produce it for a ZIP. The SHA-1, - HMAC-SHA1 and AES primitives live in a new local minizip-ng backend, - `third_party/minizip-ng/src/mz_crypt_wfm.c` (PBKDF2 comes from the vendored - `mz_crypt.c`); its header explains why they are implemented in-tree instead - of delegating to another vendored library. -- **Encrypted RAR is actually wired up.** v1.9 vendored an engine that *could* - decrypt (`RARSetPassword`) but never called it, so encrypted archives were - rejected. The password now reaches the engine, including for `-hp` - header-encrypted archives, and a wrong password returns `extract_password` - so the prompt can retry. -- **Build fix: compiler-flag changes now invalidate objects.** `make` cannot - see flag changes, so adding `-DHAVE_WZAES -DHAVE_PKCRYPT` left the existing - `mz_zip.o` / `mz_crypt.o` in place — and because nothing referenced the new - streams any more, `--gc-sections` quietly dropped the encryption code again - while the link still "succeeded". The Makefile now records the third-party - flag set in `ps5-obj/.third_party_cflags` and rebuilds only when it really - changes. This is the same trap the `LzmaDec.o` rule was working around. -- **The UI can now retry with a password.** An `extract_password` failure no - longer ends in an error box: the attempt is re-sent with whatever the user - types, up to three times, keeping the conflict policy and the large-file - opt-in of the original request. Cancelling or submitting an empty box falls - back to the original failure report. 7z keeps its up-front prompt, since an - encrypted 7z header would otherwise cost a wasted scan. -- **Encrypted 7z headers (`-mhe=on`) now open.** This was the last known format - gap: with `-mhe=on` the file names, the folder table *and* every entry size - sit inside the encrypted header, so the vendored SDK gives up with - `SZ_ERROR_UNSUPPORTED` before it can list a single entry. A new module, - `src/sevenz_header.c`, reads the header record, decodes its one folder with - the project's own 7zAES path (`src/sevenz_chain.c`) and then hands the SDK a - small virtual stream in which the encrypted record has been replaced by the - plaintext — so the SDK goes on parsing exactly the archive it always did, and - nothing on disk is touched. Archives whose header is merely *compressed* - (`-mhc=on`, the default) are not modified in any way, and a wrong password - comes back as `extract_password` like every other encrypted archive. -- `tests/make-zip-enc-fixtures.bat` and three real fixtures under - `tests/fixtures-real/` (`enc-zipcrypto.zip`, `enc-aes256.zip`, - `enc-aes256-store.zip`, password `secret123`). -- Host checks: **140 ZIP + 37 RAR = 177** (`tests/run-tests.sh`). The new - encrypted-ZIP cases run against real archives in `tests/fixtures-real/` - generated by the new `tests/make-zip-enc-fixtures.bat`. -- A RAR archive that asks for a dictionary larger than this build supports no - longer reports as a per-entry size problem: it gets its own - `extract_dict_too_large` code and a message that names the required and the - supported dictionary size. The build's behaviour is unchanged — such an - archive is still refused, because honouring it would mean allocating the - whole window in one shot, which is exactly what rarlab's own CLI refuses to - do by default and what a 16 GB shared-memory console cannot afford. -- 7z checks: **27 cases, 0 failures** (`tests/run-sevenz-tests.sh`), and the - `KNOWN_GAPS` list that held `aeshe` is now empty — the encrypted-header - fixture passes through both the folder decoder and the extraction facade. -- The frontend retry flow has a headless check of its own — - `node .build/ui_retry_test.mjs` loads the real `assets/main.js` into a stubbed - DOM and asserts the remembered request, the retry cap and the give-up paths, - including the regression case for a non-ASCII folder: **40 checks, 0 failures**. - `node .build/ui_upload_menu_test.mjs` covers the markup side — every - `data-i18n` key exists in both languages, the upload menu is wired to the - right handlers, the classes it uses are styled, and the row-highlight rules - are scoped so they cannot lose the cascade to a generic button rule, and that - the extract button is never hidden — only disabled, with a message per reason: - **40 checks, 0 failures**. -- The tree builds to **903 448 B**, sha256 - `8ca47d5aaca75085b32641300cce30fadb7df7749cb6b53d04f129bcecc286b7`, - `e_machine` `0x003e`. Rebuilt from the same tree, byte-identical both times; - the built ELF was checked to contain the new frontend code, which is only - reachable after gunzipping the embedded assets. Same 20 sections and **no new - dynamic symbols**. The encrypted-archive work added +5 712 bytes of content - (`.text` +4 880, `.rodata` +640, `.eh_frame*` +192); the fork marker below - then added a further +0x100 (256 B), the corrected - `err_extract_unsupported` copy another +0x40 (64 B), the upload menu +0x980 - (2 432 B), the frontend copy and CSS of the first fix round +0x100 (256 B) - and the scoped row-highlight rules +0x180 (384 B), then the always-on extract - button +0x240 (576 B) — every one of them to - `.rodata` **and to no other section**, so the file is still 903 448 B across - all six builds. An unchanged size is not evidence of an unchanged binary — - compare sections with `readelf -SW`. -- **The version string carries a fork marker: `v1.9.3M`.** Upstream releases are - plain `vX.Y.Z`, so the trailing `M` (Modified) is what tells you which of the - two projects a build came from. It is part of `VERSION_TAG`, so `/api/version`, - the PS5 start-up notification, the stdout banner, the UI footer and the ELF - file name all carry it at once, and the UI footer spells it out on hover. The - file name changing also means a fork build can no longer shadow an upstream - artifact of the same upstream version. See Credits. -- Validated end to end on a real console before release. - -> This section describes **v1.9.3M**, which is released: -> . -> The previous release, `v1.9.2`, contains none of it. - -## What's new in v1.9.2 - -Version-string-only re-release. The `v1.9.1` tag sat four commits behind the -tree that produced its binary, so the tag could not rebuild the published -artifact; v1.9.2 is cut from the right commit. It is functionally identical to -the v1.9.1 binary — the only change is the baked-in version string. - -## What's new in v1.9 - -- **RAR engine replaced with the official rarlab UnRAR 7.20.1** - (`third_party/unrar7/`, replacing dmc_unrar). This is what actually - makes RAR extraction work on real files: dmc_unrar could not decode - archives written by **WinRAR 6.x/7.x** (RAR5 "v6" compression) and had - no multi-volume support — both now work. -- **RAR5 "v6" archives extract** (the v1.8-era "corrupt archive" report - on WinRAR 6/7 files is gone). -- **Multi-volume RAR** (`.part01.rar` chains): unrar stitches the parts by - name when the full set sits next to the volume you open. -- Engine can decrypt encrypted RAR (`RARSetPassword`) — password UI / - API plumbing still pending, encrypted archives are rejected for now. -- Host tests now run real archives (v6 / encrypted / 3-volume fixtures - committed under `tests/fixtures-real/`): **70 ZIP + 24 RAR = 94 checks**. - -## What's new in v1.8 - -- **Single-volume RAR extraction** via the vendored FLOSS library - [`dmc_unrar`](https://github.com/DrMcCoy/dmc_unrar) (GPL-2.0-or-later). - RAR 1.5, 2.x, 3.x, 4.x and 5.x archives are supported. `.rar` files - appear in the file list with the **Extract** button enabled; the button - is greyed out on `.part02+.rar` sub-volumes with the tooltip "select - the main volume instead" — v1.8 cannot stitch multi-volume RARs (see - the [RAR extraction](#rar-extraction) section below). -- **Shared extraction protocol** between the new `src/rar_extract.c` - engine and the existing `src/zip_extract.c` engine: same `zipx_status_t` - codes, same `zipx_limits_t` profile (default / `large=1`), same - three-phase model (`scan → extract → publish → cleanup`), same staging - directory layout, same conflict policy, same error mapping into the - task UI. The dispatcher in `src/extract.c` is one tiny - `ends_with_ci(…)` switch. -- **14 new host-side C tests** (`tests/test_rar_extract.c`) wired into - the existing `tests/run-tests.sh`. Coverage: format dispatch, error - translation across every `DMC_UNRAR_*` code that affects RAR users, - limit-profile handoff. Total host checks: **69 ZIP + 14 RAR = 83**. -- **Documentation**: [`CHANGELOG.md`](./CHANGELOG.md), - [`docs/UPGRADE-v1.8-rar-support.md`](./docs/UPGRADE-v1.8-rar-support.md) - and the vendoring decision tree at - [`third_party/unrar7/VENDORED.md`](./third_party/unrar7/VENDORED.md) - (v1.8 shipped it as `third_party/unrar/VENDORED.md`). -- See the [dedicated section](#rar-extraction) below for scope and the - limitations that come from using dmc_unrar (no multi-volume, no - encryption in v1.8 — both lift in v1.9 when the library is replaced). - -## What's new in v1.8.1 - -- **Default ZIP limits relaxed** (companion to v1.7's large profile). - v1.7 shipped with a 64 GiB default per-entry cap, which was too - aggressive for typical PS5 system-backup ZIPs (200-300 GiB). v1.8.1 - raises the default profile to **1 TiB total / 256 GiB per entry / - 500 : 1 ratio**, with the `large=1` opt-in kept at 2 TiB / 1 TiB / - 1000 : 1. The frontend threshold rises from 60 GiB to 240 GiB so - common system-backup archives no longer trigger the prompt. -- RAR extraction inherits the new defaults (rar_extract.c threads - `c->limits` from the engine — no engine change required). -- Rationale: the real zip-bomb defence is `check_space()` (statvfs-based - real disk-space check before staging) + `max_ratio` (declared - compression ratio cap). The size caps are a UX guard, not a security - boundary. - -## What's new in v1.8.2 - -- **Default ZIP limits relaxed again** for the 3A-game single-file case. - A single ~300 GiB uncompressed file inside an archive was still - silently rejected by v1.8.1 (the default scan returns - `ZIPX_ERR_LIMIT_FILE_SIZE` before the request ever reaches the - frontend confirmation prompt). v1.8.2 raises the default profile to - **2 TiB total / 512 GiB per entry / 500 : 1 ratio**, with the `large=1` - opt-in bumped to 4 TiB / 1 TiB / 1000 : 1. Frontend threshold rises - from 240 GiB to 480 GiB. -- **Two PS5-only build fixes** discovered when cross-compiling for the - PS5 target. The host-side test suite (`tests/run-tests.sh`) had - silently accepted both because it links the same sources but uses - gcc rather than clang 18 and a different include path: - - `Makefile` CFLAGS: add `-Ithird_party/unrar` so `src/rar_extract.c` - can find the project-authored `dmc_unrar_api.h` facade header. - - `src/extract.c`: move `extract_progress()` definition above - `extract_dispatch()` so the implicit function declaration is not - flagged by `-Werror=implicit-function-declaration` (clang 18 in the - PS5 SDK is stricter than the host gcc used by tests). -- **Release artifact** for v1.8.2: `web-file-mgr.elf` — 509 704 bytes, - sha256 `1b2c3d68b35e32737105f17d14a80a3c159ceca0cabd274ee168cbcd81906f65`, - ELF class 64, little-endian, e_machine `0x003e` (x86_64-sie-ps5). -- Tests: **84 host-side checks** (70 ZIP + 14 RAR), 0 failures. PS5 - cross-compile succeeds end-to-end. - -## What's new in v1.7 - -- **ZIP large-file profile** (opt-in via the new `large=1` argument on `/api/extract`): relaxed caps of **2 TiB** archive total, **1 TiB** per entry, **1000 : 1** compression ratio. The frontend prompts for confirmation whenever the archive on disk is larger than **60 GiB**; the server only activates the profile when the user explicitly agrees. -- **Stricter default ZIP profile** stays safe: **1 TiB** total / **256 GiB** per entry / **500 : 1** ratio. A 4 MiB compressed payload that expands to 800 GiB still gets rejected before any output file is opened. -- **69 host-side C tests** (`tests/run-tests.sh`) now cover path traversal, ZIP64, encryption rejection, ratios, conflict policies and the new large-file profile (`tests/test_zip_extract.c`). -- Earlier refinements — see `git log` since v1.6. +```sh +make linux && ./web-file-mgr-linux-v1.9.3M +``` ## Screenshots @@ -228,53 +62,229 @@ the v1.9.1 binary — the only change is the baked-in version string. ## Features -- **Browse** — list files and folders; sort by name, type, size, mtime or permissions. Last sort mode persists in `localStorage`. -- **Permissions** — toggle read/write/execute with checkboxes, or paste a validated four-digit octal mode. -- **Operations** — copy, move, delete (recursive, no recycle bin), rename, create files and folders. -- **Editor** — in-place UTF-8 text editor for files ≤ 1 MiB across a curated extension list: `.txt .json .xml .ini .cfg .conf .md .log .lua .js .css .html .htm .c .h .cpp .hpp .sh .csv .yaml .yml .shn`. +**Files and folders** + +- **Browse and sort** — list files and folders; sort by name, type, size, + modified time or permissions. The chosen sort mode persists in + `localStorage`. +- **Permissions** — toggle read / write / execute from the permissions column + with checkboxes, or paste a validated four-digit octal mode. +- **Copy and move** — the two-step, clipboard-style flow: select the sources, + then paste (copy) or move them into the folder you browse to next. Conflict + prompts appear for overwriting files and for merging folders. +- **Delete** — recursive and permanent; there is no recycle bin. +- **Create** — new folders and new empty text files. - **Multi-select** — copy, move, delete or tar-download many items in one go. -- **Upload** — single files or folder trees from any device on the LAN (hidden in the PS5 browser). Atomic temp + rename. -- **Download** — single file as raw bytes, or folders/multi-select as a streaming `.tar`. Hidden in the PS5 browser. -- **Tasks** — full-screen overlay with delayed show, live progress, throughput, ETA, cancel, and recovery if the browser is closed and reopened mid-task. -- **Archive extraction** — ZIP, RAR and 7z, all behind the same zip-bomb / traversal / ratio protection. ZIP covers stored / deflated / ZIP64 plus **encrypted** entries (ZipCrypto and WinZip AES-128/192/256); RAR covers RAR4 + RAR5 including WinRAR 6/7 "v6", multi-volume, and `-p` / `-hp` encryption; 7z covers Copy / LZMA / LZMA2 / PPMd, the Delta and BCJ2 filters, `.7z.001` volumes, 7zAES, and `-mhe=on` encrypted headers. See the [ZIP extraction](#zip-extraction) and [RAR extraction](#rar-extraction) sections below for scope. -- **Encrypted archives** — a wrong or missing password is reported as `err_extract_password` and retried through a password prompt, up to three times, with the original conflict policy and large-file opt-in preserved. 7z asks for the password up front instead, so an encrypted header does not cost a wasted scan. -- **PKG** — install and preview `.pkg` files. -- **Images** — preview `.png .jpg .jpeg .gif .bmp .webp`. -- **Localization** — English + Simplified Chinese, auto-selected from `navigator.languages`. -- **Mobile-friendly** — responsive layout with wrapped toolbars and horizontally scrollable file lists. +- **Copied and moved files are chmod'ed `0777`** where the filesystem supports + Unix permissions. FAT/exFAT-style filesystems may ignore the chmod — that is + the filesystem's answer, not an error. + +**Content** + +- **Text editor** — in-place UTF-8 editing for files up to 1 MiB, across a + curated extension list: `.txt .json .xml .ini .cfg .conf .md .log .lua .js + .css .html .htm .c .h .cpp .hpp .sh .csv .yaml .yml .shn`. Non-UTF-8 and + oversized files are refused rather than mangled. +- **Image preview** — `.png .jpg .jpeg .gif .bmp .webp`, served straight from + the console. +- **PKG** — install `.pkg` files and preview their metadata. + +**Moving data in and out** + +- **Upload** — a "Upload ▾" menu offering *single file* and *folder tree*; + full-page drag and drop works too, and the footer says so. Files are written + to a temporary name and renamed into place when the transfer completes. + Hidden in the PS5 browser, since the point is to drive the console from + another device. +- **Download** — a single file as raw bytes, or folders / multi-selection as a + streamed `.tar` that is never written to console storage first. Hidden in the + PS5 browser. +- **Upload and extract** — pick an archive, choose "extract after upload", and + the extraction starts as soon as the upload lands. If it turns out to be + encrypted, the password prompt appears immediately. + +**Archive extraction** — see [Archive support](#archive-support) for the full +matrix. In short: ZIP, RAR and 7z, plain or encrypted, single or split, all +behind the same size / ratio / traversal / disk-space protection, and all +implemented inside this payload — no second file to install. + +**Everything else** + +- **Task overlay** — a full-screen overlay with delayed appearance, live + progress, throughput, ETA, cancel, and recovery of the active task display if + the browser is closed and reopened while the payload keeps running. +- **Localization** — English and Simplified Chinese, selected from + `navigator.languages` / `navigator.language` (`zh*` → Chinese, everything + else → English). +- **Mobile-friendly** — responsive layout with wrapped toolbars and + horizontally scrollable file lists. +- **Start-up notification and home-screen launcher** — the notification shows + the app name, the version and the actual listen port; on first start the + payload installs a "PS5 Web File Manager" shortcut in the Media category + without overwriting launcher files that already exist. The launcher icon and + the browser favicon are the same embedded `icon0.png`, so the icon is stored + once in the ELF. +- **Filenames survive mixed encodings** — names are transported as UTF-8 over + the web API, but the payload also preserves the byte-oriented names returned + by mounted filesystems, so a USB stick holding GBK names still displays and + operates correctly. (This is a fork fix; see [Notes](#notes).) + +## Archive support + +Three engines, dispatched by extension in `src/extract.c`, sharing one +three-phase pipeline (`scan → extract to staging → publish by rename`) and one +set of limit profiles and conflict policies. Vendoring decisions and the +per-library licence position are in +[`third_party/unrar7/VENDORED.md`](third_party/unrar7/VENDORED.md) and +[`THIRD_PARTY_NOTICES`](THIRD_PARTY_NOTICES). + +| | ZIP | RAR | 7z | +|---|---|---|---| +| Engine | `src/zip_extract.{c,h}` | `src/rar_extract.{c,h}` | `src/sevenz_extract.{c,h}` | +| Backend | vendored minizip-ng 4.2.2 + zlib | vendored **rarlab UnRAR 7.20.1** (official source) | LZMA SDK 26.03 decode subset + self-written codec chain | +| Stored / deflated | ✅ | n/a | ✅ (Copy / LZMA / LZMA2 / PPMd) | +| 64-bit sizes | ✅ ZIP64 | ✅ | ✅ | +| Filters / converters | — | — | ✅ Delta, BCJ2, PPC / IA64 / ARM / ARMT / SPARC | +| Multi-volume | ✅ parts offered by the engine | ✅ unrar stitches by name | ✅ | +| Traditional password | ✅ PKWARE "ZipCrypto" (`zip -e`) | ✅ `-p` | — | +| AES encryption | ✅ WinZip AES-128/192/256 | ✅ | ✅ 7zAES (AES-256-CBC) | +| Encrypted file names | — | ✅ `-hp` header encryption | ✅ `-mhe=on` encrypted header | +| Password prompt | on failure, retried | on failure, retried | asked up front | + +**Volume naming that works** + +| Format | Accepted | Note | +|---|---|---| +| ZIP | `name.zip.001…` (7-Zip), `name.part1.zip…` (WinRAR), `name.z01…` + `name.zip` (Info-ZIP) | Any part can be selected; the engine finds the rest in the same folder | +| RAR | `name.part1.rar` / `name.part01.rar` (first volume) | Select the **first** volume. Other volumes are greyed out with a hint | +| 7z | `name.7z.001…` | Any volume works; the engine walks the directory for the rest | + +### Size and safety limits + +Two profiles. The default is shipped safe; the large profile is engaged **only** +when the request carries `large=1`, and the UI asks for that opt-in through a +confirmation prompt. + +| Limit | Default | Large (`large=1`) | +|---|---|---| +| `max_entries` | 200 000 | 500 000 | +| `max_total_bytes` (uncompressed) | 2 TiB | 4 TiB | +| `max_file_bytes` (per entry) | 512 GiB | 1 TiB | +| `max_ratio` (uncompressed ÷ compressed) | 500 : 1 | 1000 : 1 | +| `max_depth` (folder nesting) | 32 | 32 | +| `max_name_len` / `max_path_len` | 255 / 1024 | 255 / 1024 | + +The default caps are sized for the console's real workload: a 3A title packed +as one ~300 GiB file inside an archive extracts without any prompt. + +### Security checks + +Extraction refuses, before creating a single output file: + +- **Path traversal** — `..` segments, absolute POSIX paths, Windows drive + letters, `\` treated as a separator inside RAR. +- **Special files** — symbolic links, devices, FIFOs, sockets + (`ZIPX_ERR_SPECIAL`). +- **Duplicate entries**, and directory / file name clashes inside one archive. +- **Limit breaches** — expanded size, entry count, nesting depth, name length or + compression ratio over the active profile. +- **Disk space** — `check_space()` consults `statvfs` for the *expanded* total + before staging begins, so a download that cannot finish is never started. + +Handing over a password does **not** skip the scan phase: an encrypted archive +gets the same limits as a plain one. + +### Conflict policy + +Passed as `conflict=` on `/api/extract`: + +- `fail` (default) — refuse to overwrite anything that already exists. +- `overwrite` — replace existing files, merge into existing folders. +- `merge` — keep existing files, add the new ones. + +### Password handling + +A missing or wrong password comes back as `ZIPX_ERR_PASSWORD` +(`err_extract_password` in the UI). The frontend shows a password box and +re-sends **the same request** — same conflict policy, same large-file opt-in — +up to three times; cancelling or submitting an empty box falls back to the +original failure report. The first failure says the archive is encrypted rather +than blaming a password you were never asked for. + +7z is the exception: because `-mhe=on` hides the file names inside the header, +the prompt comes **up front**, before the scan — otherwise an encrypted 7z would +cost a wasted scan before anyone could ask. + +### Tuning the large-file prompt + +The frontend threshold lives in `assets/main.js`: + +```js +const LARGE_FILE_THRESHOLD_BYTES = 480 * 1024 * 1024 * 1024; // 480 GiB +``` + +An archive larger than this on disk triggers the confirmation prompt. Set it to +`Infinity` to silence the prompt, lower it to be more conservative, or remove +the call — the server honours `large=1` regardless of what the frontend does. + +### What this build refuses, on purpose + +- **Formats other than ZIP / RAR / 7z.** `.tar`, `.tar.gz` / `.tgz`, `.gz`, + `.xz`, `.bz2`, `.zst`, `.cab`, `.arj`, `.lzh`, `.cpio`, `.xar` and the rest of + the long tail are not recognised. Upstream covers ~30 extensions by shipping + a full 7-Zip as an external helper process; this fork deliberately does not — + see [How this fork differs from upstream](#how-this-fork-differs-from-upstream). +- **ZIP entries using a compression method other than stored / deflated**, 7z + folders with an unsupported coder, RAR older than 1.4. +- **RAR volume sets named `x.rar.001`.** unrar chains its own `x.partN.rar` + naming; rename the parts (`.rar.001` → `.part1.rar`, `.002` → `.part2.rar`, …) + and it works. Split ZIP and 7z sets accept the `.001` style directly. +- **RAR dictionaries above 4 GiB.** Such an archive is refused with its own + `err_extract_dict_too_large` code and a message naming both the required and + the supported size. Honouring it would mean one single allocation of the whole + dictionary window — what rarlab's own CLI refuses by default and what a 16 GB + shared-memory console cannot afford. (RAR5 caps the header field at 4 GiB, so + this can only come from the newer RAR7 header format.) A wrong password is + **not** in this category, and neither is a multi-volume set. ## Quickstart -1. **Build** the ELF: +1. **Build** the payload: ```sh - export PS5_PAYLOAD_SDK=/opt/ps5-payload-sdk # see "Build" for SDK setup + export PS5_PAYLOAD_SDK=/opt/ps5-payload-sdk # see Build below make ``` -2. **Send** the payload to the PS5 (default ELF-loader port `9021`): + +2. **Send** it to the console (the usual ELF-loader port is `9021`): ```sh - nc -q0 "$PS5_HOST" 9021 < web-file-mgr.elf + nc -q0 "$PS5_HOST" 9021 < web-file-mgr-v1.9.3M.elf ``` -3. **Read** the on-screen PS5 notification — it prints the actual listen port (default `8888`). -4. **Open** `http://:/` in any browser on the same LAN — the PS5 browser works too. -5. On first run, the payload also writes a **Media**-category home-screen launcher; existing launcher files are not overwritten. + +3. **Read** the on-screen notification — it prints the actual listen port + (usually `8888`). +4. **Open** `http://:/` in any browser on the same LAN. +5. On first start the payload also writes a **Media**-category home-screen + launcher; existing launcher files are left alone. ## Build -Requires [ps5-payload-dev/sdk](https://github.com/ps5-payload-dev/sdk#quick-start): +Requires the [ps5-payload-dev/sdk](https://github.com/ps5-payload-dev/sdk#quick-start): ```sh export PS5_PAYLOAD_SDK=/opt/ps5-payload-sdk ``` -This project links against `libmicrohttpd`. `make` checks for it before building and runs the installer automatically when missing: +The project links against `libmicrohttpd`. `make` checks for it and runs the +installer automatically when it is missing: ```sh make ``` -If the build host has no network access, drop the libmicrohttpd tarball in advance and run the installer manually: +On a host without network access, drop the tarball in place and install it +manually first: ```sh LIBMICROHTTPD_TARBALL=/path/to/libmicrohttpd-1.0.1.tar.gz \ @@ -285,272 +295,108 @@ make Output: ```text -web-file-mgr.elf (~several hundred KiB, larger in v1.9 with unrar; x86_64-sie-ps5) +web-file-mgr-v1.9.3M.elf # x86_64-sie-ps5, ~882 KiB ``` -For pure UI/JS work without the PS5 toolchain: +The version string is part of `VERSION_TAG` and therefore of the output **file +name**, so a build cannot silently shadow another version's artifact. Override +it when needed: + +```sh +make VERSION_TAG=v1.9.4M +``` + +For UI/JS work without the PS5 toolchain: ```sh make linux -./web-file-mgr-linux +./web-file-mgr-linux-v1.9.3M ``` -The Linux build does **not** include the PS5 home-screen launcher installer. +The Linux build does not include the PS5 home-screen launcher installer. ## Usage -Start an ELF loader on the PS5 (port `9021` is common). Send the payload: +Start an ELF loader on the console (port `9021` is the common one) and send the +payload: ```sh export PS5_HOST=ps5_ip_address -nc -q0 "$PS5_HOST" 9021 < web-file-mgr.elf +nc -q0 "$PS5_HOST" 9021 < web-file-mgr-v1.9.3M.elf ``` -After the payload starts, the PS5 notification shows the app name, version and actual listen port. Open the URL it prints, for example: +After it starts, the notification shows the app name, the version and the actual +listen port. Open the URL it prints: ```text http://${PS5_IP_ADDRESS}:8888/ ``` -If the payload had to fall back to a different port (e.g. `8889`), use whatever port the notification shows — the URL is not hard-coded. +If `8888` was already in use the payload walked up to the next free port — use +whatever the notification says, the URL is not hard-coded. On first start it +installs a `PS5 Web File Manager` shortcut in the Media category when needed; +missing launcher files are written, existing ones are preserved. -On first startup, the payload installs a `PS5 Web File Manager` shortcut in the Media category when needed. Existing launcher files are preserved; only missing ones are written. - -## ZIP extraction - -Plain and encrypted ZIPs — stored / deflated / ZIP64, in the clear or with either encryption scheme (traditional PKWARE "ZipCrypto" and WinZip AES-128/192/256). The engine is a standalone three-phase module (`scan → extract → publish → cleanup`) at `src/zip_extract.{c,h}`, with a separate host-side C test suite. Each entry is first written into a staging directory (`*.wfm-part-*`), then atomically renamed into the destination. There is deliberately **no per-entry `fsync`** anywhere in the extract path — the whole pipeline is "sync nothing, rename everything", because publish is rename-only and there is no resume feature to protect (measured ≥14x on an 8000-file archive; see `docs/EXTRACTION-PERF.md`). Any failure mid-archive rolls back partial changes; cancel and fatal errors always clean up staging. - -### Limits - -| Limit | Default profile | Large profile (`ZIPX_LIMITS_LARGE`) | -|---|---|---| -| `max_entries` | 200 000 | 500 000 | -| `max_total_bytes` (uncompressed) | 2 TiB | 4 TiB | -| `max_file_bytes` (per entry) | 512 GiB | 1 TiB | -| `max_ratio` (uncompressed / compressed) | 500 : 1 | 1000 : 1 | -| `max_depth` (folder nesting) | 32 | 32 | -| `max_name_len` / `max_path_len` | 255 / 1024 | 255 / 1024 | - -The **default profile** is shipped safe: a 4 MiB compressed blob that decodes to 800 GiB is rejected before any output file is opened. The **large profile** is engaged **only** when the request includes `large=1` — the archive dialog prompts the user automatically whenever the archive on disk is larger than `LARGE_FILE_THRESHOLD_BYTES` (480 GiB by default; configurable in `assets/main.js`). Confirming the prompt is the user's explicit opt-in; the server still records nothing extra on its own. - -### Security checks - -The engine refuses to extract: - -- Path traversal (`..` segments, absolute POSIX paths, Windows drive letters). -- Symbolic links, devices, FIFOs, sockets (`ZIPX_ERR_SPECIAL`). -- Duplicate entries or directory/file name clashes inside the same archive. -- Archives whose expanded size, entry count, depth, name length or compression ratio breach the active profile. - -Encrypted entries are no longer a refusal: the password arrives as `password=` -on `/api/extract`, and a missing or wrong one comes back as `ZIPX_ERR_PASSWORD` -(`err_extract_password` in the UI) so the prompt can retry. The scan phase still -runs for encrypted archives — handing over a password does not skip the limits. - -### Conflict policy - -Passed as `conflict=` on `/api/extract`: - -- `fail` (default) — refuse to overwrite any existing target. -- `overwrite` — replace existing files; merge into existing folders. -- `merge` — keep existing files, add new ones. - -### Tuning the threshold - -The 480 GiB frontend threshold lives in `assets/main.js`: - -```js -const LARGE_FILE_THRESHOLD_BYTES = 480 * 1024 * 1024 * 1024; -``` - -Set it to `Infinity` to silence the prompt, lower it to be more conservative, or remove the call entirely — the server still respects `large=1` regardless of the threshold. - -## RAR extraction - -A RAR extraction engine (`src/rar_extract.{c,h}`) backed by the **official -rarlab UnRAR source** (`third_party/unrar7/`, version 7.20.1, compiled as a -static library and driven through its C-compatible DLL API). Files with the -extension `.rar` get the same **Extract** button as `.zip` files; the engine -is dispatched by `src/extract.c` based on extension. - -> v1.9 replaced the v1.8 engine (dmc_unrar 1.7.0). dmc_unrar could not -> decode archives written by WinRAR 6.x/7.x (RAR5 "v6" compression) and had -> no multi-volume support; unrar handles both natively. - -### Scope - -| Format | Support | Notes | -|---|---|---| -| RAR 1.5 → 4.x (incl. 2.9 / 3.6 / 4.0) | ✅ | | -| RAR 5.0 and **5.0 "v6"** (WinRAR 6.x / 7.x) | ✅ | The v1.9 trigger | -| Solid blocks, dictionary up to 1 GiB | ✅ | | -| PPMd decompression (RAR 3.0+) | ✅ | | -| **Multi-volume** (`.part01.rar` + `.part02.rar` + …) | ✅ | unrar stitches parts by name when the whole set sits next to the volume you open. Select the first volume (`name.part1.rar` / `name.part01.rar`); non-first volumes are still greyed out in the UI with a hint. | -| **Encrypted RAR** | ✅ | Both `-p` data encryption and `-hp` header encryption. The password reaches the engine as `password=` on `/api/extract` (`RARSetPassword` runs after `RAROpenArchiveEx` and before the first `RARReadHeaderEx`); a missing or wrong one returns `ZIPX_ERR_PASSWORD` so the prompt can retry. | -| Symbolic links / FIFOs / sockets / devices | ❌ | Rejected with `ZIPX_ERR_SPECIAL` (mirrors ZIP behaviour) | -| RAR 1.3 (pre-1.4) | ❌ | Rejected upstream by unrar | - -When an archive is rejected, the user gets an `extract_unsupported` -failure with the file name as the detail argument. The frontend already -shows this with the typical bilingual retry guidance. - -### Limits - -The RAR engine re-uses the ZIP limits table verbatim — there is no RAR -profile table on top. Defaults and the `large=1` opt-in are identical: - -| Limit | Default profile | Large profile (`large=1`) | -|---|---|---| -| `max_entries` | 200 000 | 500 000 | -| `max_total_bytes` (uncompressed) | 2 TiB | 4 TiB | -| `max_file_bytes` (per entry) | 512 GiB | 1 TiB | -| `max_ratio` (uncompressed / compressed) | 500 : 1 | 1000 : 1 | -| `max_depth` (folder nesting) | 32 | 32 | -| `max_name_len` / `max_path_len` | 255 / 1024 | 255 / 1024 | - -Large-profile RAR extraction uses the same `LARGE_FILE_THRESHOLD_BYTES` -(480 GiB) prompt as ZIP — the frontend treats `.rar` and `.zip` the same -way for the prompt, and the server only ever activates the large caps -when the request carries `large=1` (opt-in). - -### Security checks - -The RAR engine applies the same checks as the ZIP engine — re-uses -`zipx_status_t` codes, so the task UI's `err_extract_unsafe_name`, -`err_extract_too_deep`, `err_extract_ratio`, etc. all fire identically: - -- Path traversal (`..` segments, absolute POSIX paths, Windows drive - letters, `\` treated as a path separator after a `Rar!\x1a\x07…` - header, etc.). -- Symbolic links, FIFOs, sockets, devices. -- Duplicate entries or directory/file name clashes inside the archive. -- Archive size, entry count, depth, name length or compression ratio - breaches of the active profile. - -### Vendoring and licence - -`third_party/unrar7/` is a verbatim copy of the official **rarlab UnRAR -source** (7.20.1), mirrored by -[`opello/unrar`](https://github.com/opello/unrar) at commit `97e1780`. It is -distributed under the **UnRAR freeware licence** (see -`third_party/unrar7/license.txt`): it may be used in any software to handle -RAR archives, but may not be used to develop a RAR-compatible *archiver* or -re-create the RAR compression algorithm. The project-authored facade -`third_party/unrar7/unrar_c_api.h` carries the project's own licence. - -> The v1.8 engine `third_party/unrar/dmc_unrar.c` (DrMcCoy/dmc_unrar 1.7.0, -> GPL-2.0-or-later) was removed in v1.9; its notice lives in git history. - -### Encrypted RAR - -The password channel is complete: `password=` on `/api/extract` is handed to -the engine, and `RARSetPassword` runs after `RAROpenArchiveEx` and before the -first `RARReadHeaderEx` — the order unrar needs to decrypt a `-hp` header. A -missing or wrong password comes back as `ZIPX_ERR_PASSWORD` -(`err_extract_password` in the UI), which is what raises the password prompt -and re-sends the original request, up to three times. - -## 7z extraction - -The 7z engine (`src/sevenz_extract.{c,h}`) is built on the LZMA SDK decode -subset plus the project's own pull-based codec chain -(`src/sevenz_chain.c`). Files ending in `.7z` get the same **Extract** button as -`.zip` and `.rar`; `src/extract.c` dispatches by extension, and the engine -re-uses the same three-phase model, limit profiles and conflict policy. - -> Added in v1.9.1. The SDK's own `SzArEx` path only understands folders with up -> to four coders, which cannot express BCJ2's five — hence the self-parsed -> folder table and the pull-based chain. - -A note on the header. 7-Zip keeps the archive header at the end of the file and -compresses it when it grows (`-mhc=on`, the default), which is why the header -region normally starts with an `k7zIdEncodedHeader` record describing one -folder. With `-mhe=on` that folder is *also* encrypted, and since it holds the -file names, the folder table and every entry size, the vendored SDK gives up on -the whole archive before listing anything. `src/sevenz_header.c` handles that -case: it reads the record, decodes its folder through the same 7zAES path the -content uses, and then presents the SDK with a virtual stream whose header -region is the plaintext — the archive on disk is never written to, and an -archive whose header is merely compressed is not touched at all. - -### Scope - -| Format | Support | Notes | -|---|---|---| -| Copy / LZMA / LZMA2 (incl. ZIP64-style sizes) | ✅ | A single-coder pure-LZMA2 folder decodes multi-threaded (`src/sevenz_mt.c`, 8 threads) | -| BCJ2 (x86 branch converter) | ✅ | Through the self-written chain; not expressible in the SDK's `SzArEx` | -| Multi-coder folders, Delta filter, PPC / IA64 / ARM / ARMT / SPARC converters | ✅ | Parsed by `src/sevenz_chain.c` | -| **Volumes** (`.7z.001` / `.z01` chains) | ✅ | `src/sevenz_volstream.c` stitches by name; open the first volume | -| **Content encryption** (7zAES, AES-256-CBC) | ✅ | The engine decrypts; the frontend asks for the password up front (so an unencrypted archive does not pay a wasted scan), passes it as `password=`, and retries on `ZIPX_ERR_PASSWORD` | -| **`-mhe=on` (encrypted header)** | ✅ | `src/sevenz_header.c` decodes the header record itself (through the same 7zAES path) and hands the SDK a virtual stream carrying the plaintext; a wrong password reports `ZIPX_ERR_PASSWORD`, so the prompt retries like any other encrypted archive | -| `-mhc=off` (uncompressed header) | ✅ | Plain headers were always readable; they are now read one byte at a time and left alone | - -### Limits - -The 7z engine re-uses the ZIP limits table verbatim — see -[ZIP extraction → Limits](#limits). - -### Security checks - -The same `zipx_status_t` codes and the same checks as ZIP and RAR: path -traversal, special files, duplicate/clashing entries, and size, entry-count, -depth, name-length or ratio breaches of the active profile. - -## Verification - -After `make`, sanity-check the produced ELF: +## Verifying the build ```sh -ls -la web-file-mgr.elf # size grew in v1.9 (unrar static library); ~509 KiB was v1.8.3 -sha256sum web-file-mgr.elf # record the digest in your release notes -file web-file-mgr.elf # expect "ELF 64-bit LSB pie executable, x86-64" -od -An -tx1 -N20 web-file-mgr.elf | head -2 # magic 7f45 4c46 0201 + e_machine 003e +ls -la web-file-mgr-v1.9.3M.elf # ~882 KiB +sha256sum web-file-mgr-v1.9.3M.elf # 8ca47d5a…c9bb for v1.9.3M +file web-file-mgr-v1.9.3M.elf # "ELF 64-bit LSB pie executable, x86-64" +od -An -tx1 -N20 web-file-mgr-v1.9.3M.elf | head -2 # magic 7f45 4c46 0201, e_machine 003e ``` -The `e_machine = 0x003e` confirms the PS5 target triple `x86_64-sie-ps5`. The `e_type = 3` (`ET_DYN`) confirms the position-independent payload expected by ELF loaders. +`e_machine = 0x003e` confirms the PS5 target triple `x86_64-sie-ps5`; +`e_type = 3` (`ET_DYN`) confirms the position-independent payload an ELF loader +expects. + +The JS/CSS/HTML assets are **gzip-compressed and embedded** in the ELF, so a +plain `strings` search for anything from `assets/` returns nothing useful. Use +the helper script instead: + +```sh +python3 .build/check-elf-gzip.py ./web-file-mgr-v1.9.3M.elf uploadMenu extractRetryKey +``` ## Tests -A POSIX/host-side C test suite covers the ZIP, RAR and 7z engines and runs on -any Linux / macOS / MSYS shell without the PS5 SDK: +A POSIX / host-side C suite covers the ZIP, RAR and 7z engines and runs on any +Linux / macOS / MSYS shell without the PS5 SDK: ```sh -cd tests && bash run-tests.sh # ZIP + RAR suites -bash run-sevenz-tests.sh # 7z suite (needs MinGW gcc + a 7-Zip binary) +cd tests && bash run-tests.sh # ZIP + RAR suites +bash run-sevenz-tests.sh # 7z suite (needs MinGW gcc and a 7-Zip binary) ``` -Output is a per-case `check`-style report — **177 checks** on the current `main` -(140 ZIP + 37 RAR), 0 failures. Coverage: +Current `main`: **177 checks** (140 ZIP + 37 RAR), 0 failures, plus **27 7z +cases**, 0 failures. Coverage: -- ZIP entry parsing (stored + deflated + ZIP64) -- Path traversal, absolute paths, backslash, Windows drive letters -- Symbolic links, FIFOs, bad CRC, truncated archives, non-ZIP files -- Limits: `entries`, `total_bytes`, `file_bytes`, `ratio`, `depth`, `name_len` -- Conflict policies: `fail` / `overwrite` / `merge` -- Cancellation in every phase -- **Encrypted archives** — each real fixture is run four ways (no password, - empty password and wrong password → `ZIPX_ERR_PASSWORD`; correct password → - success with a byte-level content check): `enc-zipcrypto.zip`, - `enc-aes256.zip` and `enc-aes256-store.zip` on the ZIP side, `enc-v6.rar` on - the RAR side. Two further cases prove the limits still apply once a password - has been handed over, and that the failing paths publish nothing. -- **Large-file profile** — `medium_bomb.zip` (ratio ≈ 238) is rejected under default caps and accepted under large caps; lowered large caps still enforce. -- **RAR engine** (`tests/test_rar_extract.c`, 37 checks) — format - dispatch (renamed ZIP rejected, junk blob rejected), error translation - across every reachable engine code, limits handoff (the - `large=1` opt-in flows into `rar_extract()` unchanged), the oversized - dictionary path above, plus the real-archive coverage above. -- **7z engine** (`tests/test_sevenz_extract.c` + `tests/run-sevenz-tests.sh`) — - byte-for-byte comparison against real `.7z` fixtures, encrypted-header - rejection, conflicts under every policy, cancellation, limits, a missing - destination parent, and the guarantee that a failure publishes nothing and - cleans up its staging tree. +- ZIP entry parsing (stored, deflated, ZIP64), and a byte-for-byte comparison + of extracted content against real archives +- Path traversal, absolute paths, backslashes, Windows drive letters +- Symbolic links, FIFOs, bad CRC, truncated archives, non-ZIP input +- Every limit (entries, total bytes, file bytes, ratio, depth, name length) +- Conflict policies `fail` / `overwrite` / `merge` +- Cancellation in every phase, and the guarantee that a failure publishes + nothing and cleans up its staging tree +- **Encrypted archives** — each real fixture is run four ways: no password, + empty password and wrong password all yield `ZIPX_ERR_PASSWORD`, correct + password succeeds with a byte-level content check. Two further cases prove the + limits still apply once a password has been handed over. Fixtures: + `enc-zipcrypto.zip`, `enc-aes256.zip`, `enc-aes256-store.zip` (ZIP), + `enc-v6.rar` (RAR), `aeshe.7z` (7z, encrypted header) +- **Large-file profile** — `medium_bomb.zip` (ratio ≈ 238) is rejected under the + default caps and accepted under the large ones +- **Format dispatch** — a renamed ZIP and a junk blob are both refused -The frontend retry flow has its own headless check — -`node .build/ui_retry_test.mjs` loads the real `assets/main.js` into a stubbed -DOM and asserts the remembered request, the retry cap and the give-up paths: -**27 checks, 0 failures**. It lives in `.build/` (outside the gitignore -whitelist), so it is a development-time script rather than a committed test. +Three frontend/served-page harnesses live in `.build/` (a development-time +directory, outside the gitignore whitelist): + +| Script | Covers | Checks | +|---|---|---| +| `ui_retry_test.mjs` | the password retry flow in the real `assets/main.js` against a stubbed DOM: remembered request, retry cap, give-up paths, and the regression case for a non-ASCII folder | 40 | +| `ui_upload_menu_test.mjs` | the markup side: every `data-i18n` key exists in both languages, all 117 `t("…")` keys used in `main.js` are translated, the upload menu is wired to the right handlers, the classes it uses are styled, the row-highlight rules keep their panel scope, and the extract button is never hidden — only disabled | 40 | +| `preview_check.mjs` | the real page against a fixture API in headless Chromium: menu hidden at rest / opens / focus / reaches the file input / closes, footer layout, and the pinned toolbar wrap thresholds | 12 assertions | ## Project layout @@ -582,71 +428,130 @@ whitelist), so it is a development-time script rather than a committed test. │ ├── sevenz_chain_e2e.c sevenz_e2e.c bigfile_e2e.c │ ├── make_fixtures.py make_sevenz_fixtures.py make_split_fixtures.py │ ├── run-tests.sh # one-shot runner (ZIP + RAR suites) -│ ├── run-sevenz-tests.sh # 7z suite, carries the KNOWN_GAPS list +│ ├── run-sevenz-tests.sh # 7z suite │ ├── bench_driver.py bench_formats.py # throughput benchmarks │ ├── compat/ # tiny Win32/MSYS shims │ └── fixtures/ fixtures-7z/ fixtures-real/ ├── docs/ -│ ├── HANDOVER.md # v1.8-era playbook, historical — see the root HANDOVER.md +│ ├── USER-GUIDE-zh-CN.md # beginner's walkthrough (Chinese) +│ ├── DEVICE-TEST-v1.9.3M.md # the acceptance checklist run before release │ ├── SIZE-OPTIMIZATION.md # ELF size analysis + per-symbol ledger │ ├── EXTRACTION-PERF.md # decompression benchmarks +│ ├── REAL-CONSOLE-PROFILE.md # measured on-device throughput +│ ├── UPSTREAM-V1.8-COMPARISON.md # this fork vs upstream's helper approach │ ├── REWRITE-FEASIBILITY.md # engine-extraction study -│ ├── UPSTREAM-V1.8-COMPARISON.md │ ├── UPGRADE-v1.7-zip-large-file-profile.md │ ├── UPGRADE-v1.8-rar-support.md │ └── screenshots/ # README screenshot images +├── CHANGELOG.md # per-release history ├── THIRD_PARTY_NOTICES # per-library licence summary ├── HANDOVER.md # current engineering handover ├── LICENSE # GPLv3+ └── README.md ``` +## How this fork differs from upstream + +This project is a fork of +[owendswang/ps5-web-file-manager](https://github.com/owendswang/ps5-web-file-manager). +The web UI, the task model and the PS5 packaging all originate upstream, and the +upstream author's release under GPL-3.0 is what makes this derivative work +possible. From v1.8 onward, upstream outsources extraction to a **separate +helper process** — a full 7-Zip shipped as `wfm-7zip-helper.elf`, which the user +must install at `/data/wfm/` themselves. This fork takes the opposite route: the +decoders are vendored *into* the payload. + +| | Upstream | This fork | +|---|---|---| +| Extraction architecture | external `wfm-7zip-helper.elf` (~100 MB, distributed separately, fixed path `/data/wfm/`), driven over a Unix-socket IPC protocol | the engines live **inside the payload**; there is no second file and no IPC | +| Deployment | two files; a missing/misplaced helper means extraction is dead (`archive_helper_not_running`) | one ELF, no external dependency | +| Formats | ~30 extensions (`.tar`, `.gz`, `.xz`, `.bz2`, `.zst`, `.cab`, `.arj`, `.lzh`, `.cpio`, …) | `.zip` / `.rar` / `.7z` and their volume forms — three, each complete | +| Zip-bomb and ratio defence | none | entry count, total size, per-file size, compression ratio, and a 1 GiB exemption so small files are not falsely flagged | +| Disk-space pre-check | none | `statvfs` against the expanded total before staging | +| Path-traversal defence | delegated to 7-Zip | implemented here, with a dedicated test group | +| Failure residue | can leave a half-extracted directory | staging directory + rename; a failure or cancel cleans up and publishes nothing | +| Password prompts | the helper's IPC protocol carries a `PASSWORD_REQUIRED` message | prompt + retry (capped at three attempts) reported as `extract_password`; 7z asks up front | +| Task survivability across a payload restart | ✅ the helper is a separate process, so a job survives | ❌ a restart loses the running task | +| Memory isolation | ✅ extraction runs in its own process | ❌ shares the address space (the LZMA2 dictionary is capped instead) | +| Version identity | plain `vX.Y.Z` | `vX.Y.ZM` — the trailing `M` marks a fork build | + +The measurement and the reasoning behind this trade-off are in +[`docs/UPSTREAM-V1.8-COMPARISON.md`](docs/UPSTREAM-V1.8-COMPARISON.md). In one +line: upstream wins on format breadth and process architecture, this fork wins +on safety, deployment and error quality. The format gap is incremental work +inside the existing architecture, not a reason to go back. + ## Notes -- Copy, move, delete, upload and download run as single background tasks. While one task is running, other file operations are rejected. +- Copy, move, delete, upload and download run as single background tasks. While + one task is running, other file operations are rejected. - Delete is recursive and permanent. There is no recycle bin. -- Copy/move tasks can be canceled. A partially copied single file is removed, but partially copied folders are left in place to avoid deleting pre-existing files when merging into an existing target folder. -- Upload tasks can be canceled. A partially uploaded temporary file is removed when possible. -- Downloading a folder or multiple selected items produces a tar stream. The tar archive is generated by the payload and is not written to PS5 storage first. -- The UI can recover the active task display if the browser is closed and reopened while the payload process is still running. -- Text editing is limited to the curated extension list above. Non-UTF-8 and oversized files are rejected. -- File names are transmitted as UTF-8 through the web API. The payload also preserves legacy byte-oriented names returned by mounted filesystems so mixed USB filename encodings still display and operate correctly. +- Copy/move tasks can be cancelled. A partially copied single file is removed; + partially copied **folders** are left in place, to avoid deleting pre-existing + files when merging into an existing target folder. +- Upload tasks can be cancelled; a partially uploaded temporary file is removed + when possible. +- Downloading a folder or a multi-selection produces a tar stream generated on + the fly — it is not written to console storage first. +- The UI recovers the active-task display if the browser is closed and reopened + while the payload is still running. +- Text editing is limited to the extension list above; non-UTF-8 and oversized + files are refused. +- **Filename encoding:** names travel over the web API as UTF-8, while a mounted + filesystem may hand back legacy byte sequences (a GBK USB stick, for example). + Rather than losing those bytes, the API maps every byte ≥ `0x80` to `\u00XX` + and restores it on the way back, and the frontend decodes to GBK/gb18030 for + display. The practical consequence is that the same directory has two + different string representations — the page's and the server's — which is why + nothing in the frontend may use a path as a cross-request key. ## FAQ -- **This is a homebrew app and should not intentionally modify system processes or kernel memory.** If you hit a kernel panic, make sure you are using a recent jailbreak method and ELF loader, or revert to the stable method you normally use. -- **P2JB users** — if this payload triggers a kernel panic, avoid using it on that setup. Stability matters more than convenience when each retry is expensive. -- **The preparing stage can take a while** when a folder contains many files — it sums folder size and checks free space, which helps avoid starting a copy / move / upload / download that cannot finish safely. -- **`err_extract_entry_too_large`** — default archive caps are 512 GiB per - entry / 500:1 ratio (covers a typical 3A-game archive with one ~300 GiB - uncompressed file). If you exceed the default, confirm the large-file - prompt (appears for archives > 480 GiB on disk), split the archive, or - pass `large=1` directly to the API. -- **`err_extract_unsupported`** — the archive is one this build cannot read: - a file that is neither `.zip` nor `.rar` nor `.7z`, a ZIP entry using a - compression method other than stored/deflated, a 7z folder with an - unsupported coder, a split set whose naming is not recognised (a RAR set - named `x.rar.001` must be renamed to `x.part1.rar`, `x.part2.rar`, …), or a - RAR older than 1.4. Encrypted and multi-volume archives are **not** in this - category — both are supported. The backend's own sentence is appended in - parentheses and names the actual cause. -- **`err_extract_dict_too_large`** — a RAR archive declares a compression - dictionary larger than this build supports (4096 MiB) and unrar asked for - permission to exceed it. The message states both the size the archive needs - and the size the build allows. This is refused on purpose: the alternative is - a single allocation of the entire dictionary window, which rarlab's own CLI - rejects by default and which a 16 GB shared-memory console cannot sustain. - Recompress the file on a PC with a dictionary of 4 GiB or less (`-md`), or - extract it there. Note that the RAR5 format itself caps the field at 4 GiB, - so this can only come from an archive written in the newer RAR7 header - format. +- **This is homebrew software and does not intentionally modify system processes + or kernel memory.** If you hit a kernel panic, make sure you are on a recent + jailbreak method and ELF loader, or go back to the setup you normally use. +- **P2JB users** — if this payload triggers a kernel panic on that setup, do not + use it there. Stability matters more than convenience when every retry is + expensive. +- **The "preparing" stage can take a while** on a folder with many files — it + sums the folder size and checks free space, which is what stops a copy, move, + upload or download that could not finish safely from starting at all. +- **`err_extract_unsupported`** — the archive is one this build cannot read: a + file that is not `.zip` / `.rar` / `.7z`, a ZIP entry using a compression + method other than stored/deflated, a 7z folder with an unsupported coder, a + split set whose naming is not recognised (a RAR set named `x.rar.001` must be + renamed to `x.part1.rar`, `x.part2.rar`, …), or a RAR older than 1.4. + **Encrypted and multi-volume archives are not in this category** — both are + supported. The backend's own sentence is appended in parentheses and names the + actual cause. +- **`err_extract_entry_too_large`** — the archive exceeds the default caps + (512 GiB per entry / 500:1 ratio). Confirm the large-file prompt (which + appears for archives over 480 GiB on disk), split the archive, or pass + `large=1` to the API directly. +- **`err_extract_dict_too_large`** — the RAR archive declares a compression + dictionary larger than this build supports (4096 MiB). Recompress it on a PC + with `-md` at or below 4 GiB, or extract it there. - **`err_extract_password`** — the archive is encrypted and the password was - missing or wrong. That includes a 7z archive with an encrypted header - (`-mhe=on`): the file names and entry sizes live inside the header, so - nothing at all can be listed until the header decrypts. ZIP and RAR raise a - password prompt on failure and retry the same request with what you type (up - to three times; cancel or an empty box gives up); 7z asks before it starts, - since an encrypted 7z header would otherwise cost a wasted scan. + missing or wrong. That includes a 7z with an encrypted header (`-mhe=on`), + where the file names and entry sizes live inside the header, so nothing can be + listed until it decrypts. + +## Version history + +Per-release detail — artefacts, digests, section-size deltas, test counts — lives +in [`CHANGELOG.md`](./CHANGELOG.md). + +| Release | Date | Headline | +|---|---|---| +| `v1.9.3M` | 2026-09-24 | Encrypted archives end to end (ZIP ZipCrypto + WinZip AES, RAR `-p`/`-hp`, 7z 7zAES incl. `-mhe=on`), dictionary reporting, and a UI pass (upload menu, drag hint, always-visible extract button) | +| `v1.9.2` | 2026-09-05 | Version-string-only re-release; tag re-cut so tag = source = binary | +| `v1.9.1` | 2026-09-05 | 7z engine, volume sets, 7zAES, and a −15.8 % size / throughput pass | +| `v1.9` | 2026-09-05 | RAR engine replaced with rarlab UnRAR 7.20.1 (RAR5 "v6", multi-volume) | +| `v1.8.3` | 2026-09-05 | "Upload and extract" accepts `.rar` | +| `v1.8.2` | 2026-09-05 | Per-entry cap raised for 3A single-file archives; two PS5-only build fixes | +| `v1.8.1` | 2026-09-05 | Default ZIP caps relaxed for system-backup archives | +| `v1.8` | 2026-09-05 | First RAR support (dmc_unrar), shared extraction protocol | +| `v1.7` | 2026-09-04 | ZIP large-file profile (`large=1`) | ## Credits @@ -657,8 +562,8 @@ author's release under GPL-3.0 is what makes this derivative work possible. **Telling a fork build from an upstream one:** since v1.9.3 the version string carries an `M` suffix (`vX.Y.ZM`) — *M* for *Modified*. Upstream owendswang releases are plain `vX.Y.Z`. So `v1.9.2` is upstream/fork-shared numbering while -`v1.9.3M` can only have come from this repository; the same letter appears in -the ELF file name, the PS5 start-up notification, `/api/version` and the web UI +`v1.9.3M` can only have come from this repository; the same letter appears in the +ELF file name, the PS5 start-up notification, `/api/version` and the web UI footer. Releases before v1.9.3M predate the convention and keep their plain numbers. @@ -676,6 +581,7 @@ Built with reference to these projects: - **[zlib](https://www.zlib.net/):** Compression backend for minizip-ng. Vendored under `third_party/zlib/`. License: zlib. - **[rarlab UnRAR](https://www.rarlab.com/rar_add.htm)** — RAR reader used by the `/api/extract` endpoint since v1.9. Vendored under `third_party/unrar7/` (version 7.20.1, the RARDLL source set). License: **UnRAR freeware license** — see `third_party/unrar7/license.txt`. Note this is a restricted licence rather than a FLOSS one: it permits using the source to handle RAR archives but forbids using it to build a RAR-compatible compressor. - **[opello/unrar](https://github.com/opello/unrar)** — the mirror the vendored rarlab sources were fetched from (commit `97e1780`). +- **[LZMA SDK](https://www.7-zip.org/sdk.html)** (7-Zip / Igor Pavlov) — 7z decoder used by the `/api/extract` endpoint since v1.9.1, vendored as a decode subset under `third_party/7z/`. License: public domain. - **[DrMcCoy/dmc_unrar](https://github.com/DrMcCoy/dmc_unrar)** — RAR engine shipped in v1.8 only, superseded in v1.9 by rarlab UnRAR (it could not decode RAR5 "v6" archives or multi-volume sets). Removed from the tree; its licence was GPL-2.0-or-later. ## License diff --git a/README.zh-CN.md b/README.zh-CN.md index 6154024..acddb8b 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -4,122 +4,43 @@ # PS5 网页文件管理器（PS5 Web File Manager） -> 面向已越狱 PS5 主机的自制 HTTP 文件管理器。通过同一局域网内的任意浏览器（包括 PS5 自带浏览器）即可浏览、编辑、上传、下载并解压 ZIP / RAR / 7z 压缩包——单个自包含 ELF 载荷，无外部服务、无遥测上报。 +

+ 最新发布版 + 许可证 + 目标平台：x86_64-sie-ps5 + 总下载量 +

+ +

+ 下载 ELF 载荷 + 新手使用说明 +

+ +> 面向已越狱 PS5 主机的自制 HTTP 文件管理器。通过同一局域网内的任意浏览器即可浏览、编辑、上传、 +> 下载并解压压缩包——单个自包含 ELF 载荷，不需要任何外部 helper 文件，不上报任何遥测。 **版本：** v1.9.3M · **标题 ID：** `FMGR88888` · **许可证：** GPLv3+ · **目标平台：** `x86_64-sie-ps5` +**下载：** [最新发布版](https://github.com/LisherSong/ps5-web-file-manager/releases/latest) · **第一次用？** [《新手使用说明》](docs/USER-GUIDE-zh-CN.md) + --- -## 概述 +## 这是什么 -一个在已越狱 PS5 上运行的 HTTP 文件管理器载荷。从局域网内任意浏览器（含 PS5 浏览器本身）打开 `http://:8888/`，即可管理外接 USB 存储与用户分区的文件。设计初衷是安全地把游戏 dump 文件夹从 USB 拷贝到内置存储，但它同时也支持常规文件管理、原地文本编辑、PKG 预览/安装、图片预览，以及内置防 zip 炸弹保护的解压功能。 +一个单文件载荷 ELF，在已越狱的 PS5 上跑起一个 HTTP 文件管理器。把它发给主机的 ELF 加载器，主机会在 +`8888` 端口启动 HTTP 服务（该端口被占用时自动往上找下一个空闲端口）。在局域网内任意浏览器（包括 +PS5 自带浏览器）打开 `http://:8888/`，即可管理外接 USB 存储与用户分区上的文件。 -同一套源码树可构建出供开发用的 Linux 二进制，以及供部署的 PS5 载荷 ELF——见下方 `make linux`。 +它只为把一件事做安全、做快而写：**把游戏 dump 文件夹从 USB 拷进内置存储。** 其余能力——浏览、排序、 +改权限、原地编辑文本、预览图片、安装 PKG、多选复制/移动/删除、上传与下载——都是为了让这件事在 +实际操作中行得通。在这之上，本仓又加了 **ZIP / RAR / 7z 的原生解压**，并配上了上游那套 helper 路线 +所没有的安全护栏（防压缩炸弹、防路径穿越、防写满磁盘）。 -> **第一次用、不想看技术细节？** 直接看 -> [《新手使用说明》](docs/USER-GUIDE-zh-CN.md)：怎么装、怎么传文件、怎么解压（含带密码与分卷的包）、 -> 界面上每句话是什么意思，以及与上游原版的差别——全部用大白话写。 +同一套源码树也能编出 Linux 二进制，因此整个前端界面不依赖主机、也不需要 PS5 SDK 就能开发： -## v1.9.3M 新增内容 - -**加密归档现在可以端到端解压——ZIP（两种方案）、RAR，以及带头加密的 7z 都已打通。** - -此前所有加密归档都会被提前拒绝，尽管密码输入框、失败提示与 `extract_password` 文案从 v1.9 起就已就位。真正的缺口在引擎侧，而不在 UI： - -- **ZIP**：vendored 的 minizip-ng 在裁剪时把 crypto 后端一起裁掉了，于是（未被改动的）`mz_zip.c` 里那些 `-DHAVE_WZAES` / `-DHAVE_PKCRYPT` 分支没有实现可调。 -- **RAR**：rarlab UnRAR 本身能解密，但 `RARSetPassword` 从未被调用。 -- **7z**：`-mhe=on` 把文件名与 folder 表放进了加密头，归档连列出都做不到。 - -三者现在都已接线。密码缺失或错误会统一报为 `extract_password`（引擎层即 `ZIPX_ERR_PASSWORD`），也就是任务浮层已有的密码提示所响应的那个错误码。 - -### 变更 - -- **版本号加改版标记：`v1.9.3` → `v1.9.3M`。** 上游 owendswang 的发布版是纯 `vX.Y.Z`，因此这个 `M`（Modified，改版）就是「上游原版还是本仓改版」的判据。它是 `VERSION_TAG` 的一部分，所以 `/api/version`、PS5 启动通知、stdout 横幅、网页右下角**与 ELF 文件名**会一次性全部带上；网页右下角另加悬浮提示（`versionTooltip`，中英各一）解释这个字母的含义，免得没读过发行说明的人无从判断。产物名随之改变，也顺带让本仓产物再不可能与上游同版本号的资产同名相撞。 - -### 新增 - -- **加密 ZIP** —— 传统 PKWARE（「ZipCrypto」，即 `zip -e` 写出的格式）与 WinZip AES-128/192/256（压缩方法 `99` + `0x9901` 扩展字段，即 `7z -mem=AES256` 写出的格式），stored 与 deflated 条目均支持。 -- **加密 RAR** —— `-p` 内容加密与 `-hp` 头加密。`RARSetPassword` 现在在 `RAROpenArchiveEx` 之后、首次 `RARReadHeaderEx` 之前调用，这正是 unrar 解密 RAR5 头所需的顺序。 -- `/api/extract` 的 `password=` 现在对两个引擎都能真正走到解密路径。空值或缺失视为「无密码」，因此表单原值可以直接透传。 -- `third_party/minizip-ng/src/mz_crypt_wfm.c` —— 为裁剪后的 minizip-ng 提供的本地 crypto 后端：SHA-1、HMAC-SHA1、AES-128/192/256；S-box 与 GF(2^8) 表在首次使用时推导，因此二进制不新增任何 `.rodata` 查表。PBKDF2 复用 vendored 的 `mz_crypt.c`；随机数直接读 `/dev/urandom`（不再是 `mz_os_rand()`），从而把 `rand`/`srand` 排除在导入表之外。以下文件按上游 4.2.2 原样恢复：`mz_strm_wzaes.{c,h}`、`mz_strm_pkcrypt.{c,h}`。 -- **前端在密码失败后可直接重试**：`extract_password` 失败不再只是弹一个错误框，而是弹出密码输入框并按原参数（冲突策略、大文件选配）重新发起同一次解压，最多重试 3 次；取消或留空则回落到原有的失败提示。 -- **加密 7z 头（`-mhe=on`）现在可以打开。** 这是最后一个已知的格式缺口：`-mhe=on` 时文件名、folder 表**与每个条目的尺寸**全都在加密头里，因此 vendored SDK 在能列出任何条目之前就以 `SZ_ERROR_UNSUPPORTED` 退出。新模块 `src/sevenz_header.c` 读出该头部记录，用它自己的那一个 folder 走项目自研的 7zAES 路径（`src/sevenz_chain.c`）解码，然后交给 SDK 一个虚拟流——把加密头所在区域替换成明文。SDK 于是照常解析它一向解析的那个归档，磁盘上的文件完全不被改动；头部只是被*压缩*（`-mhc=on`，默认）的归档完全不受影响；密码错误则与其它加密归档一样返回 `extract_password`。 -- `tests/make-zip-enc-fixtures.bat`，以及 `tests/fixtures-real/` 下的三个真实 fixture（`enc-zipcrypto.zip`、`enc-aes256.zip`、`enc-aes256-store.zip`，密码 `secret123`）。 - -### 修复 - -- **编译选项变化现在会使目标文件失效。** `make` 察觉不到编译选项变化，因此加上 `-DHAVE_WZAES -DHAVE_PKCRYPT` 后旧的 `mz_zip.o` / `mz_crypt.o` 原样保留——又因为此时已没有任何代码引用新流，`--gc-sections` 会在链接「成功」的同时把加密代码再次丢掉（本次改动的第一次构建产物与已发布的 release 逐字节相同）。Makefile 现在把第三方编译选项集记录进 `ps5-obj/.third_party_cflags` / `linux-obj/.third_party_cflags`，只在真正变化时重编——这正是早年 `LzmaDec.o` 规则所规避的同一个陷阱，现已通用化。 -- `ZIPX_ERR_UNSUPPORTED` 不再涵盖加密，现在仅表示「多卷或不受支持的压缩方法」。 -- 顺带把 `tests/test_sevenz_extract.c` 里一处会导致截断告警的 `snprintf` 缓冲区调足。 - -### 测试 - -- `tests/test_zip_extract.c` 对每个加密 fixture 跑四种情况（无密码 → `PASSWORD`、空密码 → `PASSWORD`、错密码 → `PASSWORD`、正确密码 → `ZIPX_OK` 并逐字节校验内容），另加一项证明「提供密码后限额依旧生效」。 -- `tests/test_rar_extract.c` 对 `enc-v6.rar` 做同样的四种情况验证，包括失败路径绝不发布任何文件。 -- `tests/test_sevenz_extract.c` 对 `aeshe.7z` 跑三种情况：无密码 → `ZIPX_ERR_PASSWORD`、错密码 → `ZIPX_ERR_PASSWORD`、正确密码 → 成功且逐字节一致，并证明失败后不留下 staging 目录。 -- 前端重试流程有一份无头检查（`.build/ui_retry_test.mjs`，把 `assets/main.js` 载入桩 DOM）：**40 项检查**，覆盖参数记忆、重试上限、取消与空密码的回落，以及「非 ASCII 目录下必须仍然弹出密码框」的回归用例。另有一份 `.build/ui_upload_menu_test.mjs`（**40 项检查**）盯标记侧：i18n 键在两份语言文件里都存在、上传菜单接对了回调、用到的 class 确实有样式、菜单行高亮规则必须带面板作用域（否则会输给通用按钮规则而静默失效），以及**解压按钮不许被隐藏、只许被置灰**（顺带扫 `main.js` 里 117 个 `t("...")` 键是否双语齐全）。 -- 主机端合计：**140 ZIP + 37 RAR = 177 项检查**，0 失败。 -- 请求的字典超过本构建支持上限的 RAR 归档不再被误报成「单条目过大」：它有独立的 `extract_dict_too_large` 编码，报错文案同时给出归档需要的字典与构建支持的上限。构建行为**未变** —— 这类归档仍然被拒，因为放行意味着一次性分配整个字典窗口，而这正是 rarlab 自家 CLI 默认拒绝、16 GB 共享内存的主机也承受不了的。 -- 7z 套件：**27 项用例，0 失败**（`tests/run-sevenz-tests.sh`），且原先登记 `aeshe` 的 `KNOWN_GAPS` 列表现已**清空**——加密头 fixture 同时通过 folder 解码器与解压 façade 两条路径。 -- 当前源码树构建产物 **903 448 B**，sha256 `8ca47d5aaca75085b32641300cce30fadb7df7749cb6b53d04f129bcecc286b7`，`e_machine` 为 `0x003e`；同一源码树构建两次逐字节一致。产物内已确认包含新的前端代码（前端资源是 gzip 内嵌的，需先解压才能在二进制里检索到）。段数仍为 20，**动态符号零新增**。加密归档那批改动净增 5 712 字节正文（`.text` +4 880、`.rodata` +640、`.eh_frame*` +192）；加 `M` 标记再让 `.rodata` 涨 0x100（256 B），修正 `err_extract_unsupported` 文案再涨 0x40（64 B），上传菜单再涨 0x980（2 432 B），第一轮修复的文案与 CSS 再涨 0x100（256 B），菜单行高亮的收敛规则再涨 0x180（384 B），解压按钮常显（去掉 `hidden`、换短标签、三条禁用理由、`.extract-action:disabled`）再涨 0x240（576 B），**其余段尺寸一个都没变**，因此文件总尺寸仍是 903 448 B。**尺寸没变不等于内容没变** —— 判断只看 `readelf -SW` 的段尺寸。 - -### 已完成 - -- 已在真机上做端到端验证（加密包上传即解压弹口令、菜单高亮、解压按钮灰/亮等全部通过）。 - -> 本节描述的是 **v1.9.3M**，该版本**已发布**： -> 。 -> 上一个发布版 `v1.9.2` 的二进制**不含**上述内容。 - -## v1.9.2 与 v1.9.1 新增内容 - -> **v1.9.2 与 v1.9.1 的功能完全相同，只换了内嵌版本号。** 原因是原先的 `v1.9.1` tag 指在产出发布二进制的提交**之前 4 个提交**，tag 与产物对不上（clone 该 tag 无法重建出发布的那份 ELF）；v1.9.2 重新从产出该二进制的提交上打，使 tag = 源码 = 二进制。 - -- **7z 解压引擎**（`src/sevenz_extract.{c,h}`）：自研解码子集 + 拉式 codec 链（`src/sevenz_chain.c`，覆盖 LZMA2 / BCJ2 等），由 `src/extract.c` 按扩展名分派，与 ZIP / RAR 共用同一套三阶段模型与限额档位。`.7z` 文件在文件列表中同样带「解压」按钮。 -- **7zAES 内容解密**（AES-256-CBC）：引擎可解密带密码的 7z 内容，密码经 `/api/extract` 的 `password=` 传入；解压 7z 时前端会提前询问密码。ZIP / RAR 的加密当时尚未打通（引擎侧缺口，见顶部「v1.9.3M 新增内容」），v1.9.1 时对它们仍会报 `extract_unsupported`。 -- **7z 分卷**：`.7z.001` / `.z01` 等链式分卷由 `src/sevenz_volstream.c` 按名拼接，打开首个分卷即可。 -- **性能三项**（纯解码提速，不影响功能面）： - - SDK 汇编 LZMA 解码器（`Asm/x86/LzmaDecOpt.asm` + jwasm，无 jwasm 自动回退纯 C）≈ 1.26×。 - - 纯 LZMA2 文件夹多线程解码（`src/sevenz_mt.c` + `Lzma2DecMt`，8 线程）≈ 1.37×。 - - 移除 ZIP 逐条目 fsync，减少 staging 重命名前的写盘开销。 -- **当时唯一缺口**：7z `-mhe=on` 加密头（独立单元，读取需自研头解析器），其余 7z 特性均已支持。已在顶部「v1.9.3M 新增内容」中补上。 - -## v1.9 新增内容 - -- **RAR 引擎替换为官方 rarlab UnRAR 7.20.1**（`third_party/unrar7/`，取代 dmc_unrar）。这正是让 RAR 解压在真实文件上可用的一步：dmc_unrar 无法解码 **WinRAR 6.x/7.x** 写出的归档（RAR5「v6」压缩），也不支持多卷；两者现在都能工作。 -- **RAR5「v6」归档可解压**（v1.8 时代在 WinRAR 6/7 文件上报「归档损坏」的问题已消失）。 -- **多卷 RAR**（`.part01.rar` 链）：当完整卷集与被打开的卷放在同一目录时，unrar 按文件名拼接各部分。 -- 引擎可解密加密 RAR（`RARSetPassword`）——发 v1.9 时密码 UI / API 接线尚未完成，加密归档会被拒绝；该接线已在顶部「v1.9.3M 新增内容」中补齐。 -- 主机测试现用真实归档（v6 / 加密 / 3 卷 fixture，提交于 `tests/fixtures-real/`）：**70 ZIP + 24 RAR = 94 项检查**。 - -## v1.8 新增内容 - -- **单卷 RAR 解压**，基于内置的 FLOSS 库 [`dmc_unrar`](https://github.com/DrMcCoy/dmc_unrar)（GPL-2.0-or-later）。支持 RAR 1.5、2.x、3.x、4.x、5.x 归档。`.rar` 文件出现在文件列表中且「解压」按钮可用；`.part02+.rar` 子卷上的按钮置灰，提示「请选择主卷」——v1.8 无法拼接多卷 RAR（见下方 [RAR 解压](#rar-解压) 章节）。 -- 新引擎 `src/rar_extract.c` 与既有 `src/zip_extract.c` 之间**共享解压协议**：相同的 `zipx_status_t` 状态码、相同的 `zipx_limits_t` 档位（默认 / `large=1`）、相同的三阶段模型（`scan → extract → publish → cleanup`）、相同的 staging 目录布局、相同的冲突策略、相同的错误映射到任务 UI。`src/extract.c` 中的分派器只是一个微小的 `ends_with_ci(…)` 判断。 -- **14 个新增主机端 C 测试**（`tests/test_rar_extract.c`）接入现有 `tests/run-tests.sh`。覆盖：格式分派、每个影响 RAR 用户的 `DMC_UNRAR_*` 错误码翻译、限额档位交接。主机检查总数：**69 ZIP + 14 RAR = 83**。 -- **文档**：[`CHANGELOG.md`](./CHANGELOG.md)、[`docs/UPGRADE-v1.8-rar-support.md`](./docs/UPGRADE-v1.8-rar-support.md)，以及 `third_party/unrar7/VENDORED.md` 中的 vendoring 决策树（v1.8 时为 `third_party/unrar/VENDORED.md`）。 - -## v1.8.1 新增内容 - -- **放宽默认 ZIP 限额**（配合 v1.7 的大档案档位）。v1.7 默认单条目上限为 64 GiB，对典型 PS5 系统备份 ZIP（200–300 GiB）过于激进。v1.8.1 将默认档位提高到 **总量 1 TiB / 单条目 256 GiB / 500:1 比率**，保留 `large=1` 选项为 2 TiB / 1 TiB / 1000:1。前端阈值从 60 GiB 提升到 240 GiB，使常见系统备份归档不再触发确认提示。 -- RAR 解压继承这些新默认值（`rar_extract.c` 直接从引擎透传 `c->limits`，无需改引擎）。 -- 理由：真正的防 zip 炸弹防线是 `check_space()`（staging 前基于 statvfs 的真实磁盘空间检查）+ `max_ratio`（声明的压缩比上限）。尺寸上限只是 UX 护栏，而非安全边界。 - -## v1.8.2 新增内容 - -- **再次放宽默认 ZIP 限额**，针对 3A 游戏单文件场景。v1.8.1 仍会静默拒绝归档内单个约 300 GiB 的未压缩文件（默认扫描在请求到达前端确认提示之前就返回 `ZIPX_ERR_LIMIT_FILE_SIZE`）。v1.8.2 将默认档位提高到 **总量 2 TiB / 单条目 512 GiB / 500:1 比率**，`large=1` 选件提到 4 TiB / 1 TiB / 1000:1。前端阈值从 240 GiB 提升到 480 GiB。 -- **两个 PS5 专属构建修复**，在交叉编译 PS5 目标时发现。主机端测试套件（`tests/run-tests.sh`）曾静默接受二者，因为它链接相同源码但使用 gcc 而非 clang 18，且包含路径不同： - - `Makefile` CFLAGS：加入 `-Ithird_party/unrar`，使 `src/rar_extract.c` 能找到项目自有的 `dmc_unrar_api.h` 门面头文件。 - - `src/extract.c`：把 `extract_progress()` 定义移到 `extract_dispatch()` 之前，避免被 `-Werror=implicit-function-declaration` 标记（PS5 SDK 的 clang 18 比测试用的主机 gcc 更严格）。 -- v1.8.2 发布产物：`web-file-mgr.elf` —— 509 704 字节，sha256 `1b2c3d68b35e32737105f17d14a80a3c159ceca0cabd274ee168cbcd81906f65`，ELF 64 位小端，e_machine `0x003e`（x86_64-sie-ps5）。 -- 测试：**84 项主机端检查**（70 ZIP + 14 RAR），0 失败。PS5 交叉编译端到端成功。 - -## v1.7 新增内容 - -- **ZIP 大文件档位**（通过在 `/api/extract` 传入新的 `large=1` 参数选配启用）：放宽的限额为 **总量 2 TiB** / **单条目 1 TiB** / **1000:1 压缩比**。当磁盘上归档大于 **60 GiB** 时前端会提示确认；仅当用户明确同意时服务器才启用该档位。 -- **更严格的默认 ZIP 档位**保持安全：**总量 1 TiB** / **单条目 256 GiB** / **500:1 比率**。一个 4 MiB 压缩包解压到 800 GiB 仍会在打开任何输出文件之前被拒绝。 -- **69 项主机端 C 测试**（`tests/run-tests.sh`）现已覆盖路径穿越、ZIP64、加密拒绝、压缩比、冲突策略与新增大文件档位（`tests/test_zip_extract.c`）。 -- 更早的细化——见 v1.6 以来的 `git log`。 +```sh +make linux && ./web-file-mgr-linux-v1.9.3M +``` ## 截图 @@ -134,37 +55,174 @@ ## 功能 -- **浏览** —— 列出文件与文件夹；按名称、类型、大小、修改时间或权限排序。上次排序方式持久化在 `localStorage`。 -- **权限** —— 用复选框切换读/写/执行，或粘贴经过校验的四位八进制模式。 -- **操作** —— 复制、移动、删除（递归、无回收站）、重命名、创建文件与文件夹。 -- **编辑器** —— 针对 ≤ 1 MiB 的文件，跨精选扩展名列表的原地 UTF-8 文本编辑器：`.txt .json .xml .ini .cfg .conf .md .log .lua .js .css .html .htm .c .h .cpp .hpp .sh .csv .yaml .yml .shn`。 +**文件与目录** + +- **浏览与排序** —— 列出文件与文件夹；按名称、类型、大小、修改时间或权限排序。上次选的排序方式 + 持久化在 `localStorage` 里。 +- **权限** —— 在权限列用复选框切换读 / 写 / 执行，或粘贴一个经过校验的四位八进制模式。 +- **复制与移动** —— 两步式「剪贴板」流程：先选中要处理的项，再浏览到目标目录粘贴（复制）或移入 + （移动）。覆盖文件与合并文件夹时都会弹出冲突确认。 +- **删除** —— 递归且永久，没有回收站。 +- **新建** —— 新建文件夹与新建空文本文件。 - **多选** —— 一次性复制、移动、删除或打包下载多个项目。 -- **上传** —— 从局域网内任意设备上传单文件或文件夹树（在 PS5 浏览器中隐藏）。原子化的临时文件 + 重命名。 -- **下载** —— 单文件以原始字节下载，或文件夹/多选以流式 `.tar` 下载。在 PS5 浏览器中隐藏。 -- **任务** —— 全屏覆盖层，带延迟显示、实时进度、吞吐率、ETA、取消，以及浏览器中途关闭重开后的恢复能力。 -- **归档解压** —— ZIP、RAR、7z 三种引擎，均带防 zip 炸弹 / 路径穿越 / 压缩比保护。ZIP 覆盖 stored / deflated / ZIP64 以及**加密**条目（ZipCrypto 与 WinZip AES-128/192/256）；RAR 覆盖 RAR4 + RAR5（含 WinRAR 6/7「v6」）、多卷，以及 `-p` / `-hp` 加密；7z 覆盖 LZMA / LZMA2 / PPMd、Delta 与 BCJ2、`.7z.001` 分卷、7zAES 与 `-mhe=on` 加密头。详见下方 [ZIP 解压](#zip-解压)、[RAR 解压](#rar-解压)、[7z 解压](#7z-解压)。 -- **加密归档重试** —— 解压遇到加密归档时，会弹出密码输入框并按原参数自动重试（最多 3 次）；也可以在解压 7z 时提前输入密码以免白跑一次扫描。 -- **PKG** —— 安装并预览 `.pkg` 文件。 -- **图片** —— 预览 `.png .jpg .jpeg .gif .bmp .webp`。 -- **本地化** —— 英文 + 简体中文，根据 `navigator.languages` 自动选择。 +- **复制/移动后的文件会被 chmod 成 `0777`**（前提是文件系统支持 Unix 权限）。FAT/exFAT 类文件系统 + 可能忽略 chmod——那是文件系统自己的答复，不是出错。 + +**内容** + +- **文本编辑器** —— 对 ≤ 1 MiB 的文件做原地 UTF-8 编辑，覆盖一份精选扩展名列表：`.txt .json .xml + .ini .cfg .conf .md .log .lua .js .css .html .htm .c .h .cpp .hpp .sh .csv .yaml .yml .shn`。 + 非 UTF-8 与超大文件会被直接拒绝，而不是改坏。 +- **图片预览** —— `.png .jpg .jpeg .gif .bmp .webp`，直接由主机串出。 +- **PKG** —— 安装 `.pkg` 文件，并预览其元信息。 + +**数据的进出** + +- **上传** —— 工具条上的「上传 ▾」菜单里选**单文件**或**文件夹树**；整页拖拽上传同样可用，页脚也 + 写明了这一点。文件先写临时名，传输完成后重命名就位。在 PS5 浏览器中隐藏——它的用途是让你从 + 另一台设备去驱动主机。 +- **下载** —— 单文件按原始字节下载；文件夹或多选则打成流式 `.tar`，不会先写进主机存储。在 PS5 + 浏览器中隐藏。 +- **上传并解压** —— 选中一个压缩包并勾选「上传后解压」，上传一落盘就开始解压；若发现是加密包， + 密码框会立刻弹出。 + +**压缩包解压** —— 完整支持矩阵见 [压缩包支持](#压缩包支持)。一句话：ZIP、RAR、7z，明文或加密、 +单卷或分卷，全都走同一套尺寸 / 压缩比 / 路径穿越 / 磁盘空间保护，而且全部实现在本载荷内部—— +不需要再装第二个文件。 + +**其他** + +- **任务浮层** —— 全屏浮层，延迟显示、实时进度、吞吐率、ETA、取消，并能在浏览器中途关闭重开、 + 而载荷进程仍在运行时恢复活动任务的显示。 +- **本地化** —— 英文与简体中文，依 `navigator.languages` / `navigator.language` 自动选择 + （`zh*` → 中文，其余 → 英文）。 - **移动端友好** —— 响应式布局，工具栏自动换行，文件列表可横向滚动。 +- **启动通知与主屏启动器** —— 通知会显示应用名、版本与实际监听端口；首次启动时载荷会在 Media + 分类安装一个「PS5 Web File Manager」快捷方式，且不覆盖已存在的启动器文件。启动器图标与浏览器 + favicon 用的是同一份内嵌 `icon0.png`，所以图标在 ELF 里只存一份。 +- **文件名不因编码混杂而丢失** —— 名字经 Web API 以 UTF-8 传输，但载荷也会保留挂载文件系统返回的 + 字节序名称，因此一块装着 GBK 文件名的 U 盘仍能正确显示与操作。（这是本仓的修复，见[备注](#备注)。） + +## 压缩包支持 + +三个引擎，由 `src/extract.c` 按扩展名分派，共用同一条三阶段流水线 +（`scan → 解压到 staging → 按 rename 发布`）与同一套限额档位、冲突策略。 +vendoring 决策与逐库许可证立场见 +[`third_party/unrar7/VENDORED.md`](third_party/unrar7/VENDORED.md) 与 +[`THIRD_PARTY_NOTICES`](THIRD_PARTY_NOTICES)。 + +| | ZIP | RAR | 7z | +|---|---|---|---| +| 引擎 | `src/zip_extract.{c,h}` | `src/rar_extract.{c,h}` | `src/sevenz_extract.{c,h}` | +| 后端 | vendored minizip-ng 4.2.2 + zlib | vendored **rarlab UnRAR 7.20.1**（官方源码） | LZMA SDK 26.03 解码子集 + 自研 codec 链 | +| stored / deflated | ✅ | 不适用 | ✅（Copy / LZMA / LZMA2 / PPMd） | +| 64 位尺寸 | ✅ ZIP64 | ✅ | ✅ | +| 过滤器 / 转换器 | — | — | ✅ Delta、BCJ2、PPC / IA64 / ARM / ARMT / SPARC | +| 分卷 | ✅ 引擎自行找齐各卷 | ✅ unrar 按名拼接 | ✅ | +| 传统密码 | ✅ PKWARE「ZipCrypto」（`zip -e`） | ✅ `-p` | — | +| AES 加密 | ✅ WinZip AES-128/192/256 | ✅ | ✅ 7zAES（AES-256-CBC） | +| 加密文件名 | — | ✅ `-hp` 头加密 | ✅ `-mhe=on` 加密头 | +| 密码询问时机 | 失败后询问并重试 | 失败后询问并重试 | 解压前提前询问 | + +**可识别的分卷命名** + +| 格式 | 接受 | 说明 | +|---|---|---| +| ZIP | `name.zip.001…`（7-Zip）、`name.part1.zip…`（WinRAR）、`name.z01…` + `name.zip`（Info-ZIP） | 任意一卷都可选，引擎会自己在同目录找齐其余分卷 | +| RAR | `name.part1.rar` / `name.part01.rar`（首卷） | 请选**首卷**；其余卷在界面中置灰并带提示 | +| 7z | `name.7z.001…` | 任意一卷均可，引擎会遍历目录取齐其余分卷 | + +### 尺寸与安全限额 + +两档档位。默认档位出厂即安全；大档案档位**仅**在请求携带 `large=1` 时才启用，而界面会通过一次 +确认提示让用户做出这个选择。 + +| 限额 | 默认 | 大档案（`large=1`） | +|---|---|---| +| `max_entries` | 200 000 | 500 000 | +| `max_total_bytes`（未压缩） | 2 TiB | 4 TiB | +| `max_file_bytes`（单条目） | 512 GiB | 1 TiB | +| `max_ratio`（未压缩 ÷ 压缩） | 500 : 1 | 1000 : 1 | +| `max_depth`（文件夹嵌套） | 32 | 32 | +| `max_name_len` / `max_path_len` | 255 / 1024 | 255 / 1024 | + +默认上限是按主机上的真实工作量定的：一个 3A 作品打成「单个约 300 GiB 文件」的归档，无需任何提示 +即可解出。 + +### 安全检查 + +在创建任何一个输出文件**之前**，解压就会拒绝以下情况： + +- **路径穿越** —— `..` 段、绝对 POSIX 路径、Windows 盘符、RAR 内把 `\` 当分隔符。 +- **特殊文件** —— 符号链接、设备、FIFO、套接字（`ZIPX_ERR_SPECIAL`）。 +- **重复条目**，以及同一归档内目录与文件同名冲突。 +- **突破限额** —— 解压后总尺寸、条目数、嵌套深度、名称长度或压缩比超出当前档位。 +- **磁盘空间** —— `check_space()` 在开始写 staging 之前就按**解压后总量**查 `statvfs`，因此一个 + 不可能完成的解压根本不会启动。 + +提交密码**不会**跳过 scan 阶段：加密归档与明文归档受同一套限额约束。 + +### 冲突策略 + +通过 `/api/extract` 上的 `conflict=` 传入： + +- `fail`（默认）—— 只要目标已存在就失败。 +- `overwrite` —— 覆盖已存在文件，合并进已存在文件夹。 +- `merge` —— 保留已存在文件，只新增其余文件。 + +### 密码处理 + +密码缺失或错误会返回 `ZIPX_ERR_PASSWORD`（界面上的 `err_extract_password`）。前端会弹出密码框， +并**按原请求**重新发起——冲突策略、大文件选配全部沿用——最多三次；取消或留空则回落到最初的失败 +提示。首次失败时的文案说的是「此压缩包已加密」，而不会去责怪一个你压根还没被问过的密码。 + +7z 是例外：因为 `-mhe=on` 把文件名藏在加密头里，密码框会**提前**出现、早于 scan——否则一个加密 +的 7z 会先白跑一遍扫描，才轮到有人问你要密码。 + +### 调整大文件提示阈值 + +前端阈值位于 `assets/main.js`： + +```js +const LARGE_FILE_THRESHOLD_BYTES = 480 * 1024 * 1024 * 1024; // 480 GiB +``` + +磁盘上大于该值的归档会触发确认提示。设为 `Infinity` 可静音提示，调低则更保守，或干脆删掉该调用 +——无论前端如何，服务器始终遵循 `large=1`。 + +### 本构建刻意不做的部分 + +- **ZIP / RAR / 7z 之外的格式。** `.tar`、`.tar.gz` / `.tgz`、`.gz`、`.xz`、`.bz2`、`.zst`、`.cab`、 + `.arj`、`.lzh`、`.cpio`、`.xar` 以及长尾里的其他格式都不识别。上游是靠把一整个 7-Zip 当作外部 + helper 进程分发，从而覆盖约 30 种后缀；本仓刻意不走这条路——原因见 + [与上游的差异](#与上游的差异)。 +- **ZIP 中 stored / deflated 之外的压缩方法**、7z 中使用了不受支持 coder 的 folder、早于 RAR 1.4 的归档。 +- **命名成 `x.rar.001` 的 RAR 分卷集。** unrar 只认它自己的 `x.partN.rar` 命名；把分卷改名 + （`.rar.001` → `.part1.rar`、`.002` → `.part2.rar`……）即可正常解压。ZIP 与 7z 的分卷集可以直接吃 + `.001` 风格。 +- **字典超过 4 GiB 的 RAR。** 这类归档会被独立地报成 `err_extract_dict_too_large`，文案同时给出 + 归档需要的尺寸与构建支持的尺寸。放行意味着**一次性分配整个字典窗口**——正是 rarlab 自家 CLI + 默认拒绝、16 GB 共享内存的主机也承受不起的那件事。（RAR5 头字段本身卡在 4 GiB，所以这种情况只 + 可能来自更新版的 RAR7 头格式。）密码错误**不属于**这一类，多卷归档也不属于。 ## 快速上手 -1. **构建** ELF： +1. **构建**载荷： ```sh - export PS5_PAYLOAD_SDK=/opt/ps5-payload-sdk # 见「构建」章节的 SDK 配置 + export PS5_PAYLOAD_SDK=/opt/ps5-payload-sdk # SDK 配置见下方「构建」 make ``` -2. **发送** 载荷到 PS5（默认 ELF 加载器端口 `9021`）： + +2. **发送**到主机（ELF 加载器常用端口 `9021`）： ```sh - nc -q0 "$PS5_HOST" 9021 < web-file-mgr.elf + nc -q0 "$PS5_HOST" 9021 < web-file-mgr-v1.9.3M.elf ``` -3. **读取** PS5 屏幕上的通知——它会打印实际监听端口（默认 `8888`）。 -4. 在**同一局域网**内的任意浏览器中打开 `http://:/`——PS5 浏览器也可以。 -5. 首次运行时，载荷还会写入一个 **Media** 分类的主屏启动器；已有的启动器文件不会被覆盖。 + +3. **读取**主机屏幕上的通知——它会打印实际监听端口（通常 `8888`）。 +4. 在同一局域网内任意浏览器中**打开** `http://:/`。 +5. 首次启动时载荷还会写入一个 **Media** 分类的主屏启动器；已有的启动器文件不会被改动。 ## 构建 @@ -174,13 +232,13 @@ export PS5_PAYLOAD_SDK=/opt/ps5-payload-sdk ``` -本项目链接 `libmicrohttpd`。`make` 在构建前会检查它，缺失时自动运行安装器： +本项目链接 `libmicrohttpd`。`make` 会先检查它，缺失时自动运行安装器： ```sh make ``` -若构建主机无网络访问，可提前放入 libmicrohttpd 源码包并手动运行安装器： +若构建主机没有外网，可提前放入源码包并手动装一次： ```sh LIBMICROHTTPD_TARBALL=/path/to/libmicrohttpd-1.0.1.tar.gz \ @@ -191,204 +249,95 @@ make 输出： ```text -web-file-mgr.elf （约数百 KiB，v1.9.1 含 unrar7 + 7z 后更大；x86_64-sie-ps5） +web-file-mgr-v1.9.3M.elf # x86_64-sie-ps5，约 882 KiB ``` -若只想做纯 UI/JS 开发而不需要 PS5 工具链： +版本号是 `VERSION_TAG` 的一部分，因此也是输出**文件名**的一部分——一次构建不可能悄悄顶替掉另一个 +版本的产物。需要时可以直接覆盖： + +```sh +make VERSION_TAG=v1.9.4M +``` + +只想做纯 UI / JS 开发、不需要 PS5 工具链时： ```sh make linux -./web-file-mgr-linux +./web-file-mgr-linux-v1.9.3M ``` Linux 构建**不包含** PS5 主屏启动器安装器。 ## 使用 -在 PS5 上启动一个 ELF 加载器（端口 `9021` 常见）。发送载荷： +在主机上启动一个 ELF 加载器（常用端口 `9021`），发送载荷： ```sh export PS5_HOST=ps5_ip_address -nc -q0 "$PS5_HOST" 9021 < web-file-mgr.elf +nc -q0 "$PS5_HOST" 9021 < web-file-mgr-v1.9.3M.elf ``` -载荷启动后，PS5 通知会显示应用名、版本与实际监听端口。打开它打印的 URL，例如： +启动后，通知会显示应用名、版本与实际监听端口。打开它打印的 URL： ```text http://${PS5_IP_ADDRESS}:8888/ ``` -若载荷不得不回退到其它端口（如 `8889`），请以通知显示的端口为准——URL 并未硬编码。 +如果 `8888` 已被占用，载荷会往上走到下一个空闲端口——请以通知显示的端口为准，URL 并未硬编码。 +首次启动时它会在需要时于 Media 分类安装一个 `PS5 Web File Manager` 快捷方式；缺失的启动器文件会 +被写入，已存在的会被保留。 -首次启动时，载荷会在需要时于 Media 分类安装一个 `PS5 Web File Manager` 快捷方式。已有的启动器文件会被保留；只补写缺失的文件。 - -## ZIP 解压 - -支持普通 ZIP 与加密 ZIP——stored / deflated / ZIP64，传统 PKWARE（ZipCrypto）与 WinZip AES-128/192/256 两种加密方案。引擎是一个独立的三阶段模块（`scan → extract → publish → cleanup`），位于 `src/zip_extract.{c,h}`，配有独立的主机端 C 测试套件。每个条目先写入 staging 目录（`*.wfm-part-*`），再原子重命名到目标位置。解压路径里**刻意不做逐条目 `fsync`**——整条流水线是「不 sync、只 rename」，因为 publish 只是 rename、也没有续解功能需要保护（8000 文件档实测 ≥14×，见 `docs/EXTRACTION-PERF.md`）。归档中途任何失败都会回滚部分改动；取消与致命错误总会清理 staging。 - -### 限额 - -| 限额 | 默认档位 | 大档案档位（`ZIPX_LIMITS_LARGE`） | -|---|---|---| -| `max_entries` | 200 000 | 500 000 | -| `max_total_bytes`（未压缩） | 2 TiB | 4 TiB | -| `max_file_bytes`（单条目） | 512 GiB | 1 TiB | -| `max_ratio`（未压缩 / 压缩） | 500 : 1 | 1000 : 1 | -| `max_depth`（文件夹嵌套） | 32 | 32 | -| `max_name_len` / `max_path_len` | 255 / 1024 | 255 / 1024 | - -**默认档位**出厂即安全：一个解压到 800 GiB 的 4 MiB 压缩块会在打开任何输出文件之前被拒绝。**大档案档位**仅在请求携带 `large=1` 时才启用——当磁盘上归档大于 `LARGE_FILE_THRESHOLD_BYTES`（默认 480 GiB；可在 `assets/main.js` 配置）时，解压对话框会自动提示用户。确认提示即为用户的明确选配；服务器自身不会额外记录任何内容。 - -### 安全检查 - -引擎拒绝解压以下归档： - -- 路径穿越（`..` 段、绝对 POSIX 路径、Windows 盘符）。 -- 符号链接、设备、FIFO、套接字（`ZIPX_ERR_SPECIAL`）。 -- 同一归档内的重复条目或目录/文件名冲突。 -- 解压后尺寸、条目数、嵌套深度、名称长度或压缩比突破当前档位。 - -加密条目**不再**属于拒绝项：密码通过 `/api/extract` 的 `password=` 传入，缺失或错误时返回 `zipx` 层的 `ZIPX_ERR_PASSWORD`（前端对应 `err_extract_password`），由界面提示后重试。scan 阶段对加密条目同样生效——限额不会因为提供了密码而被跳过。 - -### 冲突策略 - -通过 `/api/extract` 上的 `conflict=` 传入： - -- `fail`（默认）—— 拒绝覆盖任何已存在的目标。 -- `overwrite` —— 替换已存在文件；合并进已存在文件夹。 -- `merge` —— 保留已存在文件，新增其余文件。 - -### 调整阈值 - -480 GiB 的前端阈值位于 `assets/main.js`： - -```js -const LARGE_FILE_THRESHOLD_BYTES = 480 * 1024 * 1024 * 1024; -``` - -设为 `Infinity` 可静音提示，调低则更保守，或干脆删掉该调用——无论阈值如何，服务器始终遵循 `large=1`。 - -## RAR 解压 - -RAR 解压引擎（`src/rar_extract.{c,h}`）由 **官方 rarlab UnRAR 源码** 支撑（`third_party/unrar7/`，版本 7.20.1，编译为静态库并通过其 C 兼容的 DLL API 驱动）。扩展名为 `.rar` 的文件与 `.zip` 文件一样拥有**解压**按钮；引擎由 `src/extract.c` 按扩展名分派。 - -> v1.9 替换了 v1.8 的引擎（dmc_unrar 1.7.0）。dmc_unrar 无法解码 WinRAR 6.x/7.x 写出的归档（RAR5「v6」压缩）且不支持多卷；unrar 原生支持两者。 - -### 支持范围 - -| 格式 | 支持 | 备注 | -|---|---|---| -| RAR 1.5 → 4.x（含 2.9 / 3.6 / 4.0） | ✅ | | -| RAR 5.0 及 **5.0「v6」**（WinRAR 6.x / 7.x） | ✅ | v1.9 的触发点 | -| Solid 块、最大 1 GiB 字典 | ✅ | | -| PPMd 解压（RAR 3.0+） | ✅ | | -| **多卷**（`.part01.rar` + `.part02.rar` + …） | ✅ | 当完整卷集与被打开的卷同处一目录时，unrar 按名拼接。选择首个卷（`name.part1.rar` / `name.part01.rar`）；非首卷在 UI 中仍置灰并给出提示。 | -| **加密 RAR** | ✅ | `-p` 内容加密与 `-hp` 头加密均可。密码经 `/api/extract` 的 `password=` 传入引擎（`RARSetPassword` 在 `RAROpenArchiveEx` 之后、首次 `RARReadHeaderEx` 之前调用）；缺失或错误返回 `ZIPX_ERR_PASSWORD`，界面提示后重试。 | -| 符号链接 / FIFO / 套接字 / 设备 | ❌ | 以 `ZIPX_ERR_SPECIAL` 拒绝（与 ZIP 行为一致） | -| RAR 1.3（1.4 之前） | ❌ | 被 unrar 上游拒绝 | - -当某归档被拒绝时，用户会收到 `extract_unsupported` 失败，文件名作为详情参数。前端已用典型的双语重试指引显示该错误。 - -### 限额 - -RAR 引擎原样复用 ZIP 的限额表——其上并无额外的 RAR 档位表。默认值与 `large=1` 选配完全相同： - -| 限额 | 默认档位 | 大档案档位（`large=1`） | -|---|---|---| -| `max_entries` | 200 000 | 500 000 | -| `max_total_bytes`（未压缩） | 2 TiB | 4 TiB | -| `max_file_bytes`（单条目） | 512 GiB | 1 TiB | -| `max_ratio`（未压缩 / 压缩） | 500 : 1 | 1000 : 1 | -| `max_depth`（文件夹嵌套） | 32 | 32 | -| `max_name_len` / `max_path_len` | 255 / 1024 | 255 / 1024 | - -大档案档位的 RAR 解压使用与 ZIP 相同的 `LARGE_FILE_THRESHOLD_BYTES`（480 GiB）提示——前端对 `.rar` 与 `.zip` 的提示处理相同，且服务器仅在请求携带 `large=1`（选配）时才启用大限额。 - -### 安全检查 - -RAR 引擎应用与 ZIP 引擎相同的检查——复用 `zipx_status_t` 状态码，因此任务 UI 的 `err_extract_unsafe_name`、`err_extract_too_deep`、`err_extract_ratio` 等会一致触发： - -- 路径穿越（`..` 段、绝对 POSIX 路径、Windows 盘符、`\` 在 `Rar!\x1a\x07…` 头之后被视为路径分隔符等）。 -- 符号链接、FIFO、套接字、设备。 -- 归档内重复条目或目录/文件名冲突。 -- 归档尺寸、条目数、深度、名称长度或压缩比突破当前档位。 - -### Vendoring 与许可 - -`third_party/unrar7/` 是官方 **rarlab UnRAR 源码**（7.20.1）的逐字副本，由 [`opello/unrar`](https://github.com/opello/unrar) 在提交 `97e1780` 处镜像。它依 **UnRAR 免费软件许可** 分发（见 `third_party/unrar7/license.txt`）：可于任何软件中用于处理 RAR 归档，但不得用于开发 RAR 兼容的*归档器*或重新实现 RAR 压缩算法。项目自有的门面 `third_party/unrar7/unrar_c_api.h` 携带项目自身许可。 - -> v1.8 引擎 `third_party/unrar/dmc_unrar.c`（DrMcCoy/dmc_unrar 1.7.0，GPL-2.0-or-later）已在 v1.9 移除；其声明留存于 git 历史。 - -### 加密 RAR - -密码通道现已完整接通：`/api/extract` 的 `password=` 会被透传给引擎，并在 `RAROpenArchiveEx` 之后、首次 `RARReadHeaderEx` 之前通过 `RARSetPassword` 交给 unrar（这个顺序是解密 `-hp` 加密头的前提）。密码缺失或错误一律返回 `ZIPX_ERR_PASSWORD`（前端 `err_extract_password`），界面据此弹出密码框并按原参数重试，最多 3 次。 - -## 7z 解压 - -7z 解压引擎（`src/sevenz_extract.{c,h}`）基于 SDK 解码子集（LZMA2 / LZMA / BCJ2 等）加上项目自研的拉式 codec 链（`src/sevenz_chain.c`，位于 `src/sevenz_chain.h`）。扩展名为 `.7z` 的文件与 ZIP / RAR 一样拥有**解压**按钮；引擎由 `src/extract.c` 按扩展名分派，并复用同一套三阶段模型、限额档位与冲突策略。 - -> v1.9.1 新增。SDK 自带的 `SzArEx` 路径仅覆盖 4 个 coder 的文件夹，不足以装下 BCJ2 的 5 coder；本项目改为自研 folder 解析 + 拉式 codec 链，从而原生支持 BCJ2 与多 coder 组合。 - -关于头部：7-Zip 把归档头放在文件末尾，并在头部变大时把它压缩（`-mhc=on`，默认行为），所以头部区域通常以一条 `k7zIdEncodedHeader` 记录开头、描述一个 folder。`-mhe=on` 时那个 folder **也被加密**，而它装着文件名、folder 表与每个条目的尺寸，于是 vendored SDK 在能列出任何条目之前就对整个归档放弃。`src/sevenz_header.c` 负责这种情况：读出该记录，用与内容完全相同的那条 7zAES 路径解出它的 folder，再把一个「头部区域是明文」的虚拟流交给 SDK——磁盘上的归档从不被写入，仅仅被*压缩*过头的归档也完全不受影响。 - -### 支持范围 - -| 格式 | 支持 | 备注 | -|---|---|---| -| LZMA2 / LZMA（含 ZIP64 式大尺寸） | ✅ | 单 coder 纯 LZMA2 走多线程解码（`src/sevenz_mt.c`，8 线程） | -| BCJ2（x86 反汇编后处理） | ✅ | 经自研拉式链；SDK `SzArEx` 装不下 5 coder 时由本项目承载 | -| 多 coder 组合文件夹 | ✅ | 自研 `sevenz_chain.c` 解析 | -| **分卷**（`.7z.001` / `.z01` 链） | ✅ | `src/sevenz_volstream.c` 按名拼接；打开首个分卷 | -| **内容加密**（7zAES，AES-256-CBC） | ✅ | 引擎可解密；解压 7z 时前端会**提前**询问密码（避免为无密码归档白跑一次扫描 + folder 解析），密码经 `password=` 传给引擎，缺失或错误返回 `ZIPX_ERR_PASSWORD` 并可重试 | -| **`-mhe=on` 加密头** | ✅ | `src/sevenz_header.c` 自行解码该头部记录（复用同一条 7zAES 路径），再把一个携带明文的虚拟流交给 SDK；密码错误返回 `ZIPX_ERR_PASSWORD`，与其它加密归档一样由提示重试 | -| `-mhc=off`（未压缩头） | ✅ | 明文头一向可读；现在按字节探测后完全不做干预 | - -当某归档被拒绝时，用户同样收到 `extract_unsupported` 失败，UI 显示双语重试指引。 - -### 限额 - -7z 引擎复用与 ZIP / RAR 完全相同的限额表；默认档位与 `large=1` 选配一致（见 [ZIP 解压 → 限额](#限额)）。 - -### 安全检查 - -7z 引擎复用相同的 `zipx_status_t` 错误码与检查集合：路径穿越、特殊文件、重复条目/名冲突、以及突破当前档位的尺寸/条目数/深度/名称长度/压缩比。coder 的 `out_size` 取自 `coder_unpack_sizes[index]`（而非文件夹尺寸），`SzArEx` 失败时重置 `blockIndex` 以避免伪 CRC。 - -## 校验 - -`make` 之后，对生成的 ELF 做健全性检查： +## 校验产物 ```sh -ls -la web-file-mgr.elf # v1.9.1 因含 unrar7 + 7z 体积更大；v1.8.3 约 509 KiB -sha256sum web-file-mgr.elf # 把摘要记录进你的发布说明 -file web-file-mgr.elf # 期望 "ELF 64-bit LSB pie executable, x86-64" -od -An -tx1 -N20 web-file-mgr.elf | head -2 # 魔数 7f45 4c46 0201 + e_machine 003e +ls -la web-file-mgr-v1.9.3M.elf # 约 882 KiB +sha256sum web-file-mgr-v1.9.3M.elf # v1.9.3M 应为 8ca47d5a…c9bb +file web-file-mgr-v1.9.3M.elf # 期望 "ELF 64-bit LSB pie executable, x86-64" +od -An -tx1 -N20 web-file-mgr-v1.9.3M.elf | head -2 # 魔数 7f45 4c46 0201，e_machine 003e ``` -`e_machine = 0x003e` 确认了 PS5 目标三元组 `x86_64-sie-ps5`。`e_type = 3`（`ET_DYN`）确认了 ELF 加载器期望的位置无关载荷。 +`e_machine = 0x003e` 确认了 PS5 目标三元组 `x86_64-sie-ps5`；`e_type = 3`（`ET_DYN`）确认了 ELF +加载器期望的位置无关载荷。 + +前端资源（JS / CSS / HTML）是 **gzip 压缩后内嵌** 进 ELF 的，所以拿 `strings` 去搜 `assets/` 里的 +任何东西都不会有命中——这是压缩所致，不是内容缺失。请改用附带脚本： + +```sh +python3 .build/check-elf-gzip.py ./web-file-mgr-v1.9.3M.elf uploadMenu extractRetryKey +``` ## 测试 -一套 POSIX / 主机端 C 测试套件覆盖 ZIP、RAR 与 7z 三个引擎，可在任意 Linux / macOS / MSYS shell 下、无需 PS5 SDK 运行： +一套 POSIX / 主机端 C 测试套件覆盖 ZIP、RAR 与 7z 三个引擎，可在任意 Linux / macOS / MSYS shell +下、无需 PS5 SDK 运行： ```sh -cd tests && bash run-tests.sh # ZIP + RAR 套件 -bash run-sevenz-tests.sh # 7z 套件（需 MinGW gcc 与 7-Zip 二进制） +cd tests && bash run-tests.sh # ZIP + RAR 套件 +bash run-sevenz-tests.sh # 7z 套件（需 MinGW gcc 与 7-Zip 二进制） ``` -输出为逐用例的 `check` 风格报告。当前 `main` 上为 **177 项检查**（140 ZIP + 37 RAR），0 失败；7z 套件另有 **27 项用例**，同样 0 失败。覆盖： +当前 `main`：**177 项检查**（140 ZIP + 37 RAR），0 失败；7z 套件另有 **27 项用例**，0 失败。覆盖： -- ZIP 条目解析（stored + deflated + ZIP64） +- ZIP 条目解析（stored、deflated、ZIP64），并与真实归档做逐字节内容比对 - 路径穿越、绝对路径、反斜杠、Windows 盘符 -- 符号链接、FIFO、坏 CRC、截断归档、非 ZIP 文件 -- 限额：`entries`、`total_bytes`、`file_bytes`、`ratio`、`depth`、`name_len` -- 冲突策略：`fail` / `overwrite` / `merge` -- 每个阶段的取消 -- **加密档案** —— `tests/fixtures-real/` 下的真实归档各跑四种情况（无密码 / 空密码 / 错密码 → `ZIPX_ERR_PASSWORD`；正确密码 → 成功并逐字节校验内容）：ZIP 侧覆盖 `enc-zipcrypto.zip`、`enc-aes256.zip`、`enc-aes256-store.zip`，RAR 侧覆盖 `enc-v6.rar`；另验证「提供密码后限额依旧生效」与「失败路径绝不发布文件」 -- **大文件档位** —— `medium_bomb.zip`（比率 ≈ 238）在默认限额下被拒、在大档位下通过；降低后的大档位仍生效 -- **RAR 引擎**（`tests/test_rar_extract.c`，37 项检查）—— 格式分派（改名的 ZIP / 垃圾数据均被拒）、每个可达引擎错误码的翻译、限额交接（`large=1` 原样传入 `rar_extract()`）、超过上限的字典路径，以及真实归档覆盖 -- **7z 引擎**（`tests/test_sevenz_extract.c` + `tests/run-sevenz-tests.sh`）—— 真实 `.7z` fixture 逐字节比对、加密头（三种情况：无密码 / 错密码 / 正确密码）、各策略下的冲突、取消、限额、缺失目标父目录，以及失败时绝不发布且 staging 树被清理的保证 +- 符号链接、FIFO、坏 CRC、截断归档、非 ZIP 输入 +- 全部限额（条目数、总字节、单文件字节、压缩比、深度、名称长度） +- 冲突策略 `fail` / `overwrite` / `merge` +- 每个阶段的取消，以及「失败绝不发布任何文件、并清理自己的 staging 树」这一保证 +- **加密归档** —— 每个真实 fixture 各跑四种情况：无密码、空密码、错密码都得 + `ZIPX_ERR_PASSWORD`，正确密码则成功并做逐字节内容校验。另有两项证明「提供密码后限额仍然生效」。 + fixture：`enc-zipcrypto.zip`、`enc-aes256.zip`、`enc-aes256-store.zip`（ZIP）、 + `enc-v6.rar`（RAR）、`aeshe.7z`（7z，加密头） +- **大档案档位** —— `medium_bomb.zip`（压缩比 ≈ 238）在默认档位下被拒、在大档案档位下通过 +- **格式分派** —— 改名的 ZIP 与垃圾数据块都会被拒 -前端还有一份无头检查 `.build/ui_retry_test.mjs`（`node .build/ui_retry_test.mjs`）：把 `assets/main.js` 载入桩 DOM，验证加密失败后的密码重试流程——参数记忆、重试上限、取消与空密码的回落，共 27 项检查。它位于 `.build/`（gitignore 白名单之外），属于开发期验证脚本。 +三份前端 / 真页面验证脚本位于 `.build/`（开发期目录，不在 gitignore 白名单内）： + +| 脚本 | 覆盖内容 | 检查数 | +|---|---|---| +| `ui_retry_test.mjs` | 真实 `assets/main.js` 载入桩 DOM 后的密码重试流程：参数记忆、重试上限、取消 / 空密码的回落，以及「非 ASCII 目录必须仍弹口令框」的回归用例 | 40 | +| `ui_upload_menu_test.mjs` | 标记侧：`index.html` 里每个 `data-i18n` 键在两份语言文件中都存在、`main.js` 里 117 个 `t("…")` 键全部有译文、上传菜单接对了回调、用到的 class 确实有样式、菜单行高亮规则保住了面板作用域，以及解压按钮**绝不隐藏、只置灰** | 40 | +| `preview_check.mjs` | 真页面 + 桩 API + 无头 Chromium：菜单静止时隐藏 / 点击打开 / 焦点落位 / 真能点到 file input / 关闭，页脚布局，以及被钉死的工具栏换行阈值 | 12 项断言 | ## 项目结构 @@ -400,67 +349,130 @@ bash run-sevenz-tests.sh # 7z 套件（需 MinGW gcc 与 7-Zip 二 ├── assets/ # HTML / CSS / JS / 图标 / param.json ├── src/ # C 载荷源码 │ ├── main.c websrv.c filemgr.c # 入口、HTTP 前端、任务模型 -│ ├── upload.c download.c # 流处理 +│ ├── upload.c download.c text.c # 流处理与原地编辑 │ ├── extract.c # /api/extract 分派器（ZIP + RAR + 7z） -│ ├── zip_extract.{c,h} zipx_common.c # ZIP 引擎 +│ ├── zip_extract.{c,h} zipx_common.c # ZIP 引擎（minizip-ng 后端） │ ├── zipx_volume.c zipx_volstream.c # ZIP 分卷探测 + 拼接流 -│ ├── rar_extract.{c,h} # RAR 引擎（unrar7 后端） -│ ├── sevenz_extract.{c,h} # 7z 引擎 -│ ├── sevenz_chain.{c,h} # 7z 拉式 codec 链（BCJ2 等） +│ ├── rar_extract.{c,h} # RAR 引擎（rarlab UnRAR 7.20.1 后端） +│ ├── sevenz_extract.{c,h} sevenz_chain.{c,h} # 7z 引擎，自解析 codec 链 │ ├── sevenz_header.{c,h} # 7z 头部读取 / `-mhe=on` 解密 -│ ├── sevenz_mt.{c,h} # 7z 多线程 LZMA2 解码 -│ ├── sevenz_volstream.{c,h} # 7z 分卷流拼接 -│ └── app_installer.c # PS5 Media 启动器安装器 -├── third_party/ # vendored：zlib、minizip-ng、unrar7、7z(SDK 子集) -│ ├── unrar7/ # rarlab UnRAR 7.20.1，静态库 + C API 门面 -│ ├── minizip-ng/ # ZIP 读取器 -│ ├── zlib/ # minizip-ng 的压缩后端 -│ └── 7z/ # LZMA SDK 解码子集 -├── tests/ # POSIX / 主机测试套件 -│ ├── test_zip_extract.c -│ ├── test_rar_extract.c -│ ├── test_sevenz_extract.c # 7z 用例驱动 -│ ├── make_fixtures.py # 重新生成测试 fixture -│ ├── run-tests.sh # 一次性运行器（ZIP + RAR） -│ ├── run-sevenz-tests.sh # 7z 运行器 -│ ├── compat/ # 小型 Win32 / MSYS 垫片 -│ └── fixtures/ fixtures-7z/ fixtures-real/ # 生成的测试归档 +│ ├── sevenz_mt.c sevenz_volstream.c # 多线程 LZMA2 + `.7z.001` 分卷 +│ ├── app_installer.c pkg_installer.c pkg_info.c # PS5 PKG 预览 / 安装 +│ └── demangle_stub.c cpu_support_stub.c # 体积 / 可移植性桩 +├── third_party/ # vendored 库 +│ ├── unrar7/ # rarlab UnRAR 7.20.1 —— RAR 引擎 +│ ├── minizip-ng/ # 4.2.2，裁剪到只留读取路径 +│ ├── 7z/ # LZMA SDK 26.03 解码子集 +│ └── zlib/ # minizip 的压缩后端 +├── tests/ # POSIX / 主机端测试套件 +│ ├── test_zip_extract.c test_rar_extract.c test_sevenz_extract.c +│ ├── sevenz_chain_e2e.c sevenz_e2e.c bigfile_e2e.c +│ ├── make_fixtures.py make_sevenz_fixtures.py make_split_fixtures.py +│ ├── run-tests.sh # 一次性运行器（ZIP + RAR） +│ ├── run-sevenz-tests.sh # 7z 套件 +│ ├── bench_driver.py bench_formats.py # 吞吐基准 +│ ├── compat/ # 小型 Win32 / MSYS 垫片 +│ └── fixtures/ fixtures-7z/ fixtures-real/ ├── docs/ -│ ├── HANDOVER.md # v1.8 时代的开发手册（历史存档，现行见根目录 HANDOVER.md） +│ ├── USER-GUIDE-zh-CN.md # 新手使用说明（中文） +│ ├── DEVICE-TEST-v1.9.3M.md # 发布前跑过的真机验收清单 +│ ├── SIZE-OPTIMIZATION.md # ELF 体积分析 + 逐符号台账 +│ ├── EXTRACTION-PERF.md # 解压基准 +│ ├── REAL-CONSOLE-PROFILE.md # 真机实测吞吐 +│ ├── UPSTREAM-V1.8-COMPARISON.md # 本仓 vs 上游 helper 路线 +│ ├── REWRITE-FEASIBILITY.md # 引擎抽取可行性研究 │ ├── UPGRADE-v1.7-zip-large-file-profile.md │ ├── UPGRADE-v1.8-rar-support.md -│ └── screenshots/ # README 截图 +│ └── screenshots/ # README 截图 +├── CHANGELOG.md # 逐版本变更记录 ├── THIRD_PARTY_NOTICES # 捆绑库署名 +├── HANDOVER.md # 现行开发交接文档 ├── LICENSE # GPLv3+ └── README.md ``` +## 与上游的差异 + +本项目 **fork 自 [owendswang/ps5-web-file-manager](https://github.com/owendswang/ps5-web-file-manager)**。 +Web UI、任务模型与 PS5 打包方式均源自该项目；上游作者以 GPL-3.0 发布，是本衍生作品得以存在的前提。 +从 v1.8 起，上游把解压**外包给一个独立 helper 进程**——一整个 7-Zip，做成 +`wfm-7zip-helper.elf`，由用户自行安装到 `/data/wfm/`。本仓走的是相反的路：解码器 vendor **进** +载荷内部。 + +| | 上游 | 本仓 | +|---|---|---| +| 解压架构 | 外部 `wfm-7zip-helper.elf`（约百 MB 量级，单独分发，路径写死 `/data/wfm/`），用 Unix socket IPC 协议驱动 | 引擎就在**载荷内部**；没有第二个文件，没有 IPC | +| 部署 | 两个文件；helper 缺失或放错位置，解压功能全废（`archive_helper_not_running`） | 单 ELF，零外部依赖 | +| 格式 | 约 30 种后缀（`.tar`、`.gz`、`.xz`、`.bz2`、`.zst`、`.cab`、`.arj`、`.lzh`、`.cpio`……） | `.zip` / `.rar` / `.7z` 及其分卷形态——三种，但每一种都完整 | +| 防压缩炸弹 / 压缩比 | 无 | 条目数、总尺寸、单文件尺寸、压缩比筛查，并对 1 GiB 以下小文件豁免以免误判 | +| 磁盘空间预检 | 无 | 写 staging 前按解压后总量查 `statvfs` | +| 路径穿越防护 | 交给 7-Zip | 本仓实现，并有专项测试组 | +| 失败残留 | 可能留下解压了一半的目录 | staging 目录 + rename；失败或取消都会清理，且不发布任何文件 | +| 密码提示 | helper 的 IPC 协议里带 `PASSWORD_REQUIRED` 消息 | 失败后弹框重试（上限三次），统一报为 `extract_password`；7z 提前询问 | +| 载荷重启后的任务存活 | ✅ helper 是独立进程，解压任务不会丢 | ❌ 重启会丢掉正在跑的任务 | +| 内存隔离 | ✅ 解压在独立进程里 | ❌ 共享地址空间（改为对 LZMA2 字典封顶） | +| 版本标识 | 纯 `vX.Y.Z` | `vX.Y.ZM`——尾部的 `M` 标记本仓改版 | + +这套取舍的实测数据与推理过程在 +[`docs/UPSTREAM-V1.8-COMPARISON.md`](docs/UPSTREAM-V1.8-COMPARISON.md)。一句话： +**上游赢在格式广度与进程架构，本仓赢在安全、部署与错误质量。** 格式覆盖的差距是现有架构里可以 +增量补的活，不构成推倒重来的理由。 + ## 备注 -- 复制、移动、删除、上传、下载作为单个后台任务运行。一个任务运行时，其它文件操作会被拒绝。 -- 删除是递归且永久的。没有回收站。 -- 复制/移动任务可取消。单个文件的部分拷贝会被移除，但部分拷贝的文件夹会保留在原地，以避免在合并进已存在目标文件夹时误删既有文件。 -- 上传任务可取消。尽可能移除部分上传的临时文件。 -- 下载文件夹或多个选中项会生成 tar 流。tar 归档由载荷生成，不会先写入 PS5 存储。 -- 若浏览器在载荷进程仍在运行时被关闭重开，UI 可恢复活动任务显示。 -- 文本编辑仅限于上述精选扩展名列表。非 UTF-8 与超大文件会被拒绝。 -- 文件名通过 Web API 以 UTF-8 传输。载荷也会保留挂载文件系统返回的遗留字节序名称，以便混合 USB 文件名编码仍能正确显示与操作。 +- 复制、移动、删除、上传、下载都作为单个后台任务运行。一个任务运行时，其它文件操作会被拒绝。 +- 删除是递归且永久的，没有回收站。 +- 复制 / 移动任务可取消。单个文件的部分拷贝会被移除；部分拷贝的**文件夹**会保留在原地，以免在 + 合并进已存在的目标文件夹时误删既有文件。 +- 上传任务可取消；尽可能移除部分上传的临时文件。 +- 下载文件夹或多选会生成 tar 流，就地生成——不会先写入主机存储。 +- 若浏览器在载荷进程仍在运行时被关闭重开，界面可恢复活动任务的显示。 +- 文本编辑仅限上述扩展名列表；非 UTF-8 与超大文件会被拒绝。 +- **文件名编码：** 名字经 Web API 以 UTF-8 传输，而挂载的文件系统可能返回遗留字节序列（比如一块 + GBK 的 U 盘）。为了不丢这些字节，API 会把每个 ≥ `0x80` 的字节映射成 `\u00XX`、回程再还原， + 前端显示时按 GBK / gb18030 解码。实际后果是：同一个目录在「页面手里」与「服务端手里」是两串 + 不同的字符串——这就是为什么前端里任何东西都不能拿路径当跨请求的键。 ## 常见问题 -- **这是自制应用，不应故意修改系统进程或内核内存。** 若遇到内核崩溃（kernel panic），请确保使用较新的越狱方法与 ELF 加载器，或回退到你惯用的稳定方法。 -- **P2JB 用户** —— 若此载荷触发内核崩溃，请避免在该环境下使用。当每次重试代价高昂时，稳定性比便利更重要。 -- **准备阶段可能耗时较久** —— 当文件夹含大量文件时，它会累加文件夹大小并检查剩余空间，这有助于避免启动一个无法安全完成的复制 / 移动 / 上传 / 下载。 -- **`err_extract_entry_too_large`** —— 默认归档上限为单条目 512 GiB / 500:1 比率（覆盖典型 3A 游戏归档中单个约 300 GiB 未压缩文件）。若超过默认，请确认大文件提示（磁盘上 > 480 GiB 的归档会出现），拆分归档，或直接向 API 传入 `large=1`。 -- **`err_extract_unsupported`** —— 这个包本机读不了：既非 `.zip` / `.rar` / `.7z` 的文件；ZIP 条目用了 stored / deflated 之外的压缩方法；7z 用了不支持的 coder；分卷命名不被识别（RAR 分卷若叫 `x.rar.001`，需改名为 `x.part1.rar`、`x.part2.rar` ……）；或早于 RAR 1.4 的归档。**加密归档与多卷归档不属于这一类**——两者都支持。界面会在括号里附上后端原文，指明具体原因。 -- **`err_extract_dict_too_large`** —— RAR 归档声明的压缩字典超过本构建支持的上限（4096 MiB），且 unrar 请求允许超额。报错文案会同时给出归档需要的尺寸与构建允许的尺寸。这是**刻意拒绝**：另一条路是一次性分配整个字典窗口，rarlab 自家 CLI 默认也会拒绝，16 GB 共享内存的主机更是承受不起。请在 PC 上用不超过 4 GiB 的字典重新压缩（`-md`），或在 PC 上解压。注意 RAR5 格式本身把这个字段卡在 4 GiB，所以这种情况只可能来自更新版 RAR7 头格式写出的归档。 -- **`err_extract_password`** —— 归档已加密，而本次提交的密码缺失或错误。这也包括带加密头（`-mhe=on`）的 7z：文件名与条目尺寸都在头部里，头解密之前连条目列表都读不出来。ZIP / RAR（以及现在的 7z 加密头）在失败后会弹出密码框（取消或留空即放弃），可用正确密码按原参数重试，最多 3 次；解压 7z 时仍会提前询问一次密码。 +- **这是自制软件，不会有意修改系统进程或内核内存。** 若遇到内核崩溃（kernel panic），请确认使用 + 较新的越狱方法与 ELF 加载器，或回到你惯用的稳定方案。 +- **P2JB 用户** —— 若此载荷在该环境下触发内核崩溃，请勿在此环境使用。当每次重试代价都很高时， + 稳定性比便利更重要。 +- **「准备阶段」在文件很多的目录下可能耗时较久** —— 它会累加目录大小并检查剩余空间，这正是让一个 + 无法安全完成的复制 / 移动 / 上传 / 下载从一开始就不会启动的原因。 +- **`err_extract_unsupported`** —— 这个包本机读不了：既非 `.zip` / `.rar` / `.7z` 的文件；ZIP 条目 + 用了 stored / deflated 之外的压缩方法；7z 用了不支持的 coder；分卷命名不被识别（RAR 分卷若叫 + `x.rar.001`，需改名为 `x.part1.rar`、`x.part2.rar`……）；或早于 RAR 1.4 的归档。**加密归档与 + 多卷归档不属于这一类**——两者都支持。界面会在括号里附上后端原文，指明具体原因。 +- **`err_extract_entry_too_large`** —— 归档超出默认上限（单条目 512 GiB / 500:1 比率）。确认那个 + 大文件提示（磁盘上 > 480 GiB 的归档会出现）、拆分归档，或直接向 API 传入 `large=1`。 +- **`err_extract_dict_too_large`** —— RAR 归档声明的压缩字典超过本构建支持的上限（4096 MiB）。 + 请在 PC 上用不超过 4 GiB 的字典（`-md`）重新压缩，或直接在 PC 上解压。 +- **`err_extract_password`** —— 归档已加密，而密码缺失或错误。这也包括带加密头（`-mhe=on`）的 7z： + 文件名与条目尺寸都在头部里，头解密之前连条目列表都读不出来。 + +## 版本历史 + +逐版本的产物、摘要、段尺寸增量与测试计数见 [`CHANGELOG.md`](./CHANGELOG.md)。 + +| 版本 | 日期 | 一句话 | +|---|---|---| +| `v1.9.3M` | 2026-09-24 | 加密归档端到端打通（ZIP ZipCrypto + WinZip AES、RAR `-p`/`-hp`、7z 7zAES 含 `-mhe=on`）、字典超限独立报错，以及一轮 UI（上传菜单、拖拽提示、解压按钮常显置灰） | +| `v1.9.2` | 2026-09-05 | 只改版本号的重发版；重新打 tag 使 tag = 源码 = 二进制 | +| `v1.9.1` | 2026-09-05 | 7z 引擎、分卷、7zAES，以及 −15.8% 体积 / 吞吐优化 | +| `v1.9` | 2026-09-05 | RAR 引擎换成 rarlab UnRAR 7.20.1（RAR5「v6」、多卷） | +| `v1.8.3` | 2026-09-05 | 「上传并解压」开始接受 `.rar` | +| `v1.8.2` | 2026-09-05 | 为 3A 单文件归档放宽单条目上限；两个 PS5 专属构建修复 | +| `v1.8.1` | 2026-09-05 | 为系统备份归档放宽默认 ZIP 上限 | +| `v1.8` | 2026-09-05 | 首个 RAR 支持（dmc_unrar），共享解压协议 | +| `v1.7` | 2026-09-04 | ZIP 大文件档位（`large=1`） | ## 署名 本项目 **fork 自 [owendswang/ps5-web-file-manager](https://github.com/owendswang/ps5-web-file-manager)**（GPL-3.0）。Web UI、任务模型与 PS5 打包方式均源自该项目；上游作者以 GPL-3.0 发布，是本衍生作品得以存在的前提。 -**怎么区分上游原版与本仓改版：** 自 v1.9.3 起版本号带 `M` 后缀（`vX.Y.ZM`），*M* 即 *Modified*（改版）；上游 owendswang 的发布版是纯 `vX.Y.Z`。因此 `v1.9.3M` 只可能出自本仓，而这个字母同时出现在 ELF 文件名、PS5 启动通知、`/api/version` 与网页右下角。v1.9.3M 之前的发布早于该约定，保留原本的无后缀编号。 +**怎么区分上游原版与本仓改版：** 自 v1.9.3 起版本号带 `M` 后缀（`vX.Y.ZM`），*M* 即 *Modified*（改版）；上游 owendswang 的发布版是纯 `vX.Y.Z`。因此 `v1.9.2` 是上游 / 本仓共用的编号，而 `v1.9.3M` 只可能出自本仓；这个字母同时出现在 ELF 文件名、PS5 启动通知、`/api/version` 与网页右下角。v1.9.3M 之前的发布早于该约定，保留原本的无后缀编号。 本项目另参考了以下项目构建： @@ -474,7 +486,10 @@ bash run-sevenz-tests.sh # 7z 套件（需 MinGW gcc 与 7-Zip 二 - **[ezremote](https://github.com/cy33hc/ps5-ezremote-client)：** PKG 预览功能的参考出处。许可证：**GPL-2.0-only** —— 其源文件未声明 "or later"，因此**无法**与本项目的 GPL-3.0 代码组合。**未取其任何代码**：`src/pkg_info.c` 是独立的 C99 实现（它还负责 `.pkg` 条目表与 `param.json` 字段，而 ezremote 根本没有 `.pkg` 解析器；JSON 走的是 `src/json_util.c` 里自写的分词器，不是 json-c）。详见 `docs/REWRITE-FEASIBILITY.md` §2.2。 - **[zlib-ng/minizip-ng](https://github.com/zlib-ng/minizip-ng)：** `/api/extract` 端点使用的 ZIP 读取器。vendored 于 `third_party/minizip-ng/`。许可证：zlib。 - **[zlib](https://www.zlib.net/)：** minizip-ng 的压缩后端。vendored 于 `third_party/zlib/`。许可证：zlib。 -- **[rarlab UnRAR (opello/unrar)](https://github.com/opello/unrar)：** v1.9 起 `/api/extract` 使用的 RAR 读取器（7.20.1）。vendored 于 `third_party/unrar7/`。许可证：UnRAR 免费软件许可。 +- **[rarlab UnRAR](https://www.rarlab.com/rar_add.htm)** —— v1.9 起 `/api/extract` 使用的 RAR 读取器（7.20.1，RARDLL 源文件集）。vendored 于 `third_party/unrar7/`。许可证：**UnRAR 免费软件许可**（见 `third_party/unrar7/license.txt`）。注意这是受限许可而非 FLOSS 许可：它允许用源码处理 RAR 归档，但禁止用它开发 RAR 兼容的压缩器。 +- **[opello/unrar](https://github.com/opello/unrar)** —— vendored 的 rarlab 源码取自该镜像（提交 `97e1780`）。 +- **[LZMA SDK](https://www.7-zip.org/sdk.html)**（7-Zip / Igor Pavlov）—— v1.9.1 起 `/api/extract` 使用的 7z 解码器，以解码子集形式 vendored 于 `third_party/7z/`。许可证：公有领域。 +- **[DrMcCoy/dmc_unrar](https://github.com/DrMcCoy/dmc_unrar)** —— 仅 v1.8 使用的 RAR 引擎，v1.9 被 rarlab UnRAR 取代（它无法解码 RAR5「v6」归档，也不支持多卷）。已从树中移除；其许可证为 GPL-2.0-or-later。 ## 许可证 @@ -482,7 +497,9 @@ bash run-sevenz-tests.sh # 7z 套件（需 MinGW gcc 与 7-Zip 二 第三方项目保留各自许可证。请勿在未保留相应许可证声明的情况下，将署名项目的资源或源码复制到其它发行版中。 -若分发二进制，除本项目 GPL 许可外，还需遵守 `libmicrohttpd` 的 LGPL 条款。vendored 的 `zlib` 与 `minizip-ng` 源码以 zlib 许可分发；再分发用此特性构建的二进制时，保留 `third_party/zlib/LICENSE` 与 `third_party/minizip-ng/LICENSE` 中的版权声明。vendored 的 `unrar7`（RAR 引擎）依 UnRAR 免费软件许可分发；再分发用 v1.9 或更高版本构建的二进制时，保留 `third_party/unrar7/license.txt` 中的声明，且不得用其开发 RAR 兼容归档器或重新实现 RAR 压缩算法。 +若分发二进制，除本项目 GPL 许可外，还需遵守 `libmicrohttpd` 的 LGPL 条款。vendored 的 `zlib` 与 `minizip-ng` 源码以 zlib 许可分发；再分发用此特性构建的二进制时，保留 `third_party/zlib/LICENSE` 与 `third_party/minizip-ng/LICENSE` 中的版权声明。 + +vendored 的 `third_party/unrar7/`（rarlab UnRAR —— `src/rar_extract.c` 背后的 RAR 引擎）**不是** GPL：它依 UnRAR 免费软件许可分发（见 `third_party/unrar7/license.txt`），该许可禁止用它开发 RAR 兼容的压缩器。再分发时请保留该声明与限制。`THIRD_PARTY_NOTICES` 载有逐库完整摘要。 ## 免责声明 diff --git a/docs/FORUM-POST-v1.9.2.md b/docs/FORUM-POST-v1.9.2.md index 474b830..36eebd4 100644 --- a/docs/FORUM-POST-v1.9.2.md +++ b/docs/FORUM-POST-v1.9.2.md @@ -74,8 +74,8 @@ ### 已知缺口（诚实列出） > 本节描述的是 **v1.9.2 发布时**的状态。此后补齐的两条缺口 —— 加密 ZIP/RAR 与 -> 7z `-mhe=on` 加密头 —— 已随 **v1.9.3M** 发布。详见 README「v1.9.3M 新增内容」与 -> `HANDOVER.md` §十一 / §十二。 +> 7z `-mhe=on` 加密头 —— 已随 **v1.9.3M** 发布。详见 README 的「压缩包支持」一节、 +> [`CHANGELOG.md`](../CHANGELOG.md) 的 `[v1.9.3M]` 段，以及 `HANDOVER.md` §十一 / §十二。 - 带密码的 ZIP / RAR / 7z：**拒绝解压**（引擎有解密能力，但密码输入 UI/API 还没接，临时先挡掉）。 - 7z `-mhe=on` **加密头**：暂不支持（需要自研头解析器）。这是 7z 侧唯一已知缺口。 diff --git a/docs/UPGRADE-v1.8-rar-support.md b/docs/UPGRADE-v1.8-rar-support.md index 05c36bb..1d42d65 100644 --- a/docs/UPGRADE-v1.8-rar-support.md +++ b/docs/UPGRADE-v1.8-rar-support.md @@ -2,11 +2,12 @@ > This is the long-form maintainer's manual for the v1.8 archive-engine > expansion. It is written for the next developer, not the user. The -> user-facing description lives in [`README.md → RAR extraction`](../README.md#rar-extraction); +> user-facing description lives in [`README.md → Archive support`](../README.md#archive-support); > the release notes are in [`CHANGELOG.md`](../CHANGELOG.md). The vendoring > decision tree (and the v1.9 upgrade path) is at -> [`third_party/unrar/VENDORED.md`](../third_party/unrar/VENDORED.md) — most -> of the "why" questions are answered there, not here. +> [`third_party/unrar7/VENDORED.md`](../third_party/unrar7/VENDORED.md) — most +> of the "why" questions are answered there, not here. (v1.8 shipped that file +> as `third_party/unrar/VENDORED.md`; the directory was renamed in v1.9.) --- @@ -54,7 +55,8 @@ in `third_party/unrar/`, add a CXX link step to `Makefile`, switch `src/rar_extract.c` to the `RAROpenArchiveEx` / `RARSetPassword` DLL API. **The `rar_extract()` signature, the dispatch layer and the host tests do not need to change.** Full step-by-step recipe is in -[`third_party/unrar/VENDORED.md`](../third_party/unrar/VENDORED.md). +[`third_party/unrar7/VENDORED.md`](../third_party/unrar7/VENDORED.md) (the v1.8 +original was `third_party/unrar/VENDORED.md`). --- @@ -509,7 +511,7 @@ git -c core.autocrlf=false commit -m "v1.8: RAR4/RAR5 single-volume unencrypted ### 10.1 v1.9 — full RAR (multi-volume + encrypted) -See [`third_party/unrar/VENDORED.md`](../third_party/unrar/VENDORED.md) +See [`third_party/unrar7/VENDORED.md`](../third_party/unrar7/VENDORED.md) §"Upgrading to a fuller library (v1.9 plan)" for the migration recipe. The public `rar_extract()` signature and the dispatch layer do **not** need to change; only: