A split ZIP is several files that only form an archive together. Until now a
`.zip.001` was rejected as an unsupported format and a `.z01`/`.partN.zip`
member could not be resolved, so these downloads were unusable.
Three naming conventions are recognised, from any member of the set:
name.zip.001, name.zip.002, ... byte split (7-Zip)
name.part1.zip, name.part2.zip byte split (WinRAR)
name.z01, ..., name.zip zip split disks (Info-ZIP / PKZIP)
The parts are served through a new stream that makes them look like one
archive, so nothing is copied to disk first (a 160 GiB set would otherwise
need a second 160 GiB scratch copy):
- src/zipx_volstream.c implements an mz_stream over the ordered part list.
Byte splits keep absolute offsets; zip split disks carry offsets relative to
the disk an entry starts on, so that mode honours
MZ_STREAM_PROP_DISK_NUMBER the way mz_zip_entry_seek_local_header drives it
and starts on the last volume, where the central directory lives.
- src/zipx_volume.c groups the set, verifies the numbering is complete and
reports exactly which volume is missing.
- zip_extract.c tries the layout implied by the naming first and the other one
as a fallback, so tools that disagree about offset bases still work.
- extract.c dispatches volume members to the engines and, when the caller
asked for the source to be deleted, removes every volume instead of leaving
orphaned parts behind.
Error messages are specific on purpose: an incomplete set reports the missing
file name (e.g. "gap.zip.002 is missing"), a lone first volume says the
remaining volumes are missing, `.rar.001` sets explain the rename that makes
them work, and 7z volumes say 7z is not supported yet.
Tests: tests/make_split_fixtures.py builds all three layouts (the disk set is
written with per-disk offsets exactly as APPNOTE 4.4.11 describes) plus two
broken sets; test_zip_extract.c extracts each layout from a different member
and byte-compares the nested binary entries. 108 ZIP + 27 RAR checks, 0
failures.
Two issues found in real-machine testing of v1.9:
1. RAR multi-volume extraction never advanced the progress bar. The engine
only accounted bytes_done after RARProcessFile() returned for a whole
entry, so a multi-GB entry spanning volumes froze the UI for its entire
duration. Wire RARSetCallback/UCM_PROCESSDATA, which unrar fires per
decompressed chunk during disk extraction, and fold each chunk into
bytes_done + a throttled progress report. (DLL-mode break handling is
never armed, so cancellation stays entry-granular; returning -1 would
be ignored.)
2. A 95k-file game zip was refused as a "compression bomb" because one
small entry exceeded the per-entry uncompressed/compressed ratio cap
(500/1000). Such entries are common and harmless in legitimate archives
(zero-filled placeholders, sparse blobs): actual bytes written are
bounded by the declared size (enforced during extract) and check_space()
verifies the full declared total against real free space before any
writes. Ratio screening now only applies to entries >= 1 GiB
(ratio_min_bytes, default in both profiles); sub-GiB high-ratio entries
are accepted. Error text now includes compressed -> uncompressed sizes.
Tests updated: small high-ratio bomb.zip now extracts OK under both
profiles and is rejected again when ratio_min_bytes is lowered below its
size; RAR multi-volume fixture asserts mid-entry progress events and that
final bytes_done reaches bytes_total. 73 ZIP + 27 RAR checks, 0 failures.
Host-side verification that the ZIP engine can extract a >4GiB zip64
entry end to end (4.7 GiB fixture, content byte-compared):
- tests/compat/sys/statvfs.h: resolve the path before taking the drive
letter (relative paths used to fail) and scale f_bavail by 4096 since
`unsigned long` is 32-bit on Windows and silently truncated ~250GB
down to ~2.3GB; real 64-bit hosts and the PS5 (LP64) are unaffected.
- tests/bigfile_e2e.c: standalone driver to extract one large archive
and print the engine result (verification via cmp/sha256 by caller).
- tests/run-tests.sh: incremental clean instead of `rm -rf` of the whole
build tree.
- Makefile: force -DHAVE_FSEEKO so minizip uses fseeko/ftello instead of
falling back to fseek/ftell on libcs without the probe.
Result: 4.7 GiB zip64 extracts OK on host in ~6s, output identical to
source. Engine 64-bit write path confirmed clean; PS5 failure needs the
strerror passthrough build for real-machine errno capture.
Host (MinGW) tests default to 32-bit off_t, so minizip lseek overflowed
on the >4 GiB zip64 fixture ('Invalid argument' opening big.zip). The PS5
Makefile already defines _FILE_OFFSET_BITS=64; this makes the host engine
matches production and lets the >4 GiB path actually be tested here.
Generated once with tests/make-rar-fixtures.bat (WinRAR RAR 7.23) so
checkouts without WinRAR still exercise the real-archive happy paths.
basic-rar4.rar is absent because RAR 7.x dropped RAR4 writing; that
check SKIPs.
unrar in RAR_OM_EXTRACT mode presents a file that spans volumes as
several header segments with the SAME name, flagged RHDF_SPLITBEFORE/
SPLITAFTER. The scan dedup rejected the continuation segments as
"duplicate entry name" and the whole multi-volume extract failed.
Fix: segments after the first (RHDF_SPLITBEFORE set) skip the
normalize/dedup/size/limit accounting in scan (the first segment already
carries the full size) and, in extract, still call RARProcessFile(EXTRACT)
to drive the volume chain but are not counted or size-accumulated.
Verified against real fixtures (tests/fixtures-real/, generated by
tests/make-rar-fixtures.bat on the user's RAR 7.23): a 512 KB file split
across vol.part1/2/3.rar extracts whole (status ZIPX_OK, big.bin + root.txt).
Host suite now 70 ZIP + 24 RAR checks, 0 failures.
-epl + absolute stage paths stored dir\nested.txt as nested.txt, so the
v6/volume happy-path checks found no dir\nested.txt. Generator now cd's
into the stage dir and passes relative names without -ep1.
tests/make_fixtures.py rmtree()s tests/fixtures on every run-tests.sh
invocation, silently deleting the real RAR fixtures. The WinRAR generator
now writes tests/fixtures-real/; run-tests.sh passes it as a third
argument to test-rar-extract, which resolves v6/volume/encrypted checks
against it and falls back to SKIP when absent.
Naming the target vol.part1.rar made RAR 7 append its own numbering onto
the whole stem -> vol.part1.partN.rar. Plain vol.rar splits into the
standard vol.part1/2/3.rar the test expects.
rar a updates an existing archive instead of re-splitting it, so the
single-volume vol.part1.rar left by earlier runs (small payload) was
grown to 512KB without ever producing part2/3. Remove vol.part* first.
The earlier -v200k split never produced part2 because the payload was a
few hundred bytes. Now generates a 512KB incompressible file (fsutil +
-m0 store) so vol.part1/2/3.rar actually exist for the volume test.
Write tool wrote UTF-8 Chinese comments which cmd.exe parsed under the
ANSI codepage as garbage ('M' is not a command). This version is pure
ASCII with CRLF line endings. RAR4 (-ma4) creation fails on newer RAR
builds; that step now warns and continues instead of aborting the whole
run. Requires no behavioural change from the engine side.
tests/make-rar-fixtures.bat produces real RAR fixtures on a machine with
WinRAR (Rar.exe) so the RAR engine is finally exercised against genuine
archives instead of 11-byte placeholders (v1.8 shipped without ever
running a real RAR through the happy path):
- basic-v6.rar RAR5 with WinRAR 6/7 "v6" compression (the format
dmc_unrar could not decode — the v1.9 trigger)
- vol.part1+2.rar multi-volume (200 KB split)
- enc-v6.rar encrypted (password secret123)
- basic-rar4.rar legacy RAR4
tests/test_rar_extract.c gains test_real_archives(): asserts v6 single
volume and RAR4 extract OK with expected files on disk, multi-volume
auto-merges from the same directory, and encrypted archives are rejected
up front with ZIPX_ERR_UNSUPPORTED (password channel lands separately).
Each check SKIPs cleanly when the fixture is absent, so plain CI
checkouts stay green.
Run: tests/make-rar-fixtures.bat (WinRAR required), then rerun
tests/run-tests.sh to flip the SKIPs into real assertions.
Replace the dmc_unrar 1.7.0 backend (third_party/unrar/) with the rarlab
UnRAR 7.20.1 source vendored in third_party/unrar7/ (commit c68c0de),
driven through its C-compatible DLL API.
Why: dmc_unrar only dispatches RAR5 compression version 5
(switch(file->version) case 0x5000). WinRAR 6.x/7.x archives (algorithm
"v6") land on the default branch -> DMC_UNRAR_FILE_UNSUPPORTED_VERSION
-> user-visible "corrupt archive", confirmed on a real v6:8M archive on
2026-09-05. unrar 7.20.1 natively handles v6, multi-volume (.partNN.rar
auto-merge) and encrypted archives.
Engine changes (src/rar_extract.c):
- Facade include swapped to third_party/unrar7/unrar_c_api.h; all calls
now go through RAROpenArchiveEx/RARReadHeaderEx/RARProcessFile/
RARCloseArchive. scan + extract each re-open the archive (the DLL API
is sequential, unlike dmc's index-based access).
- Error translation rewritten for the ERAR_* code set.
- extract no longer builds each staging path by hand (open_parent_dirs/
extract_one removed): unrar writes the whole tree under the staging
root; names were validated during scan.
- Bug fix: normalize_name() cleared the caller-provided *is_dir (it set
*is_dir = 0), turning directory entries into files and tripping the
duplicate detector whenever an explicit directory header followed
files beneath it (real v6 archive had exactly this). RHDF_DIRECTORY
is now honored end to end.
- Encrypted RAR still fails up front (ZIPX_ERR_UNSUPPORTED): unrar can
decrypt, but password plumbing (API/UI) lands in a later commit.
Build:
- Makefile: C++ compiler vars (CXX/HOST_CXX, prospero-clang++ defaults
to -stdlib=libc++), unrar7 RARDLL source set (49 files, mirrors
UnRARDll.vcxproj), .cpp pattern rules, link through the C++ driver.
- tests/run-tests.sh: compiles unrar7 objects with $CXX, links the RAR
test with g++ and -lpowrprof on Windows.
- third_party/unrar/ (dmc_unrar) removed; see git history.
Verified on host (MinGW): full engine run on the user's real v6 archive
(24.9 MB, 170 headers) -> ZIPX_OK, 170/170 entries, 133 files + 37 dirs,
71,514,931 bytes. Host suite: 70 ZIP + 14 RAR checks green.
Frontend password/multi-volume UX and release plumbing (CHANGELOG,
THIRD_PARTY_NOTICES, README, tag) follow in the v1.9 release commit.
A 3A-game archive with a single ~300 GiB uncompressed file was silently
rejected by the v1.8.1 default profile (scan_archive returns
ZIPX_ERR_LIMIT_FILE_SIZE before the request ever reaches the frontend
confirmation prompt, so the user just sees "卡壳"). The 256 GiB default
cap was tuned for PS5 system backups (many smaller entries) and was wrong
for the 3A-game single-file case.
Limits changes:
- src/zip_extract.c k_default_limits: 1 TiB / 256 GiB / 500 -> 2 TiB / 512 GiB / 500
- src/zip_extract.c k_large_limits: 2 TiB / 1 TiB / 1000 -> 4 TiB / 1 TiB / 1000 (must stay > default)
- assets/main.js LARGE_FILE_THRESHOLD_BYTES: 240 GiB -> 480 GiB
- assets/lang-{en,zh}.js extractLargeAsk copy reflects new numbers
- tests/test_zip_extract.c: large.max_total_bytes advertised-number assertion updated to 4 TiB
- README.md / docs/HANDOVER.md: limit tables, FAQ, threshold snippets updated
- CHANGELOG.md: v1.8.2 entry added above v1.8.1
PS5-only build fixes (caught by WSL cross-compile, host tests pass without):
- Makefile CFLAGS: add -Ithird_party/unrar so src/rar_extract.c can find
the dmc_unrar_api.h facade header (THIRD_PARTY_CFLAGS already had it
but the main CFLAGS for project sources did not).
- src/extract.c: move extract_progress() definition before
extract_dispatch() so the implicit function declaration is not flagged
by -Werror=implicit-function-declaration (clang 18 in the PS5 SDK is
stricter than the host gcc used by tests/run-tests.sh).
Release artifact: web-file-mgr.elf 509 704 bytes,
sha256 1b2c3d68b35e32737105f17d14a80a3c159ceca0cabd274ee168cbcd81906f65,
e_machine 0x003e (x86_64-sie-ps5).
Safety argument is unchanged from v1.8.1: zip-bomb defence is
check_space() (statvfs-based real disk-space check before staging) +
max_ratio (declared compression ratio cap). The size caps are a UX guard,
not a security boundary.
RAR extraction inherits the new defaults automatically (rar_extract.c
threads c->limits from the engine).
84 host-side checks (70 ZIP + 14 RAR), 0 failures. PS5 cross-compile
succeeds; ELF size grew from 427 656 (v1.8) to 509 704 (v1.8.2) due to
the dmc_unrar vendor TU being linked in.
User feedback: the previous default limits were an over-cautious UX
guard, not a security guard. check_space() already enforces available
>= bytes_total before staging begins, and max_ratio already rejects
classic zip bombs at any declared ratio above the cap. A user with a
multi-hundred-GiB PS5 system image shouldn't have to click through a
confirmation prompt for an obviously safe archive.
k_default_limits (src/zip_extract.c:36-44) relaxed:
- max_total_bytes: 512 GiB -> 1 TiB
- max_file_bytes: 64 GiB -> 256 GiB
- max_ratio: 200 -> 500
k_large_limits unchanged (1 TiB / 1 TiB / 1000).
Frontend threshold LARGE_FILE_THRESHOLD_BYTES (assets/main.js:838)
bumped 60 GiB -> 240 GiB to track the new default. The 240 GiB value
keeps the same 4x head-room over the default cap as the previous 60
GiB did, so the confirm() prompt only triggers for archives that
truly warrant the user paying attention.
assets/lang-{en,zh}.js extractLargeAsk copy updated to reflect the
new default-profile numbers (format-agnostic; same string for .zip and
.rar).
README.md: 'Stricter default' line under "What's new in v1.7", both
limit tables (the ZIP section and the RAR section), the "Tuning the
threshold" snippet, and the err_extract_entry_too_large FAQ entry all
updated. CHANGELOG.md gets a new [v1.8.1] - 2026-09-05 section
documenting the relaxation, the rationale (check_space + max_ratio
are the real guards), and explicit migration notes.
docs/HANDOVER.md and docs/UPGRADE-v1.8-rar-support.md numeric
references updated. The historical v1.7 upgrade document
(docs/UPGRADE-v1.7-zip-large-file-profile.md) is deliberately left
unchanged so the v1.7 -> v1.8.1 evolution remains traceable from git.
src/rar_extract.c is untouched: it already threads c->limits from
the engine and picks up the new defaults for free.
tests/test_zip_extract.c test_large_profile() rewritten for the new
ratio cap:
- medium_bomb.zip (ratio ~238) is now accepted by both default and
large profiles (showing real-world high-ratio archives aren't
artificially blocked)
- bomb.zip (ratio ~1027) is rejected by BOTH default and large
profiles (a bomb is a bomb regardless of which profile you opt
into)
- User-lowered tight ratio cap (200) still rejects medium_bomb
(proves caps are enforced, not just nominal)
- User-lowered tight file cap still rejects zip64 entries
Final test count: 70 ZIP + 14 RAR = 84 checks, 0 failures (was 69+14).
Migration:
- Forward-compatible: existing v1.7/v1.8 deployments that never
triggered err_extract_entry_too_large see no difference
- No data loss: relaxation only widens accepted archives
- Real guards (check_space, max_ratio, path traversal) unchanged
Tests/make_fixtures.py now patches zipfile.time so Python 3.13's
zipfile.writestr no longer embeds wall-clock date_time into each entry
(the module uses time.localtime(time.time())[:6] when the caller
supplies a string arcname, which silently made every fixture diff on
each regeneration and polluted git with thousands of unrelated byte
changes). The patch fakes localtime/time to always return
(1980, 1, 1, 0, 0, 0).
All .zip fixtures are regenerated against the new deterministic
generator and committed. notar.rar is regenerated alongside because
test_rar_extract.c's main() recreates it from basic.zip at run time;
including it here keeps the source fixture and the rar fixture in
lock-step.
Verified: 2 consecutive python3 make_fixtures.py runs produce identical
md5 for all 19 .zip fixtures; full test suite stays green
(69 ZIP + 14 RAR = 83 checks, 0 failures).
tests/test_rar_extract.c — 14 checks, zero dependency on host having
rar or 7z installed:
test_engine_dispatch (4):
- not-a-RAR rejected: notar.rar (renamed basic.zip) -> FORMAT
- junk blob rejected: junk.rar (1024 random bytes) -> FORMAT
- missing source -> OPEN
- null rar_path / dst -> INTERNAL
test_format_translation (6):
- rar_translate_error mapping table covers every dmc_unrar_return
code rar_extract can emit, verified against fixture files
test_limits_handoff (4):
- default vs large limits profile passed through unchanged
- max_entries / max_total_bytes / max_file_bytes / max_ratio
Uses tests/posix_compat.h (open/close/mkdirat renames) via gcc -include
for host builds -- never compiled into the PS5 payload.
tests/make_fixtures.py — adds rar_fixtures() that prefers host 'rar'
(non-free) and falls back to host '7z' (LGPL). If neither is present,
emits an 11-byte 'placeholder' basic.rar so the negative tests still
fire (basic.rar is positive-path, but is only read when a RAR writer
exists). Wired into the fn list in main().
tests/run-tests.sh:
- Builds third_party/unrar/dmc_unrar.o with -DDMC_UNRAR_DISABLE_BE32TOH_BE64TOH=1
- Builds rar_extract.o with -Ithird_party/unrar and posix_compat shim
- Builds test_rar_extract.o against both rar_extract.o and zip_extract.o
(the latter is required: rar_extract.c references zipx_status_string,
zipx_default_limits, zipx_limits_profile for error/log/output)
- Links test-rar-extract, then runs both test-zip-extract and
test-rar-extract in sequence.
Fixtures committed:
- basic.rar -- 11-byte placeholder (host has no RAR writer)
- notar.rar -- basic.zip renamed, for format-rejection test
- junk.rar -- 1024 random bytes, for format-rejection test
- All other .rar fixtures (encrypted, multi-volume, symlink, ...) are
generated on demand when run on a host with rar/7z installed.
Final test count on a writer-less host: 69 ZIP checks + 14 RAR checks
= 83 total, 0 failures (re-verified post-commit).
Per user decision on 2026-09-04 21:01 GMT+8, the v1.8 plan no longer
covers the legacy split-archive format (name.rar + .r00/.r01/.r02...).
Rationale: PS5 users almost exclusively ship the modern RAR5
part-volume format (name.part01.rar, name.part02.rar, ...); the
legacy .rNN format is from the 2005-2010 era. Removing it cuts:
- Frontend: isRarSubVolume() no longer needs /\.r\d{2,}$/ branch
- Fixtures: drop rar4_multi_old.rar generation
- Tests: drop test_rar4_multi_volume_old()
- Backend: unrar API surface still supports legacy .rNN, but we just
never feed it one
Documentation sweep:
- Section 5.1: range updated
- Section 6.4.1: isExtractableArchive/isRarSubVolume simplified
- Section 6.5.1: fixture generation script updated
- Section 6.5.3: test_rar4_multi_volume_old removed
- Section 8.4: decision rationale no longer mentions .rNN
- Section 6.6.4: CHANGELOG / README example updated
- Section 11.2: test matrix updated
- Section 6.4.9: D4 manual test list no longer tests .r00
Homebrew HTTP file manager payload for jailbroken PS5 consoles (x86_64-sie-ps5,
title id FMGR88888). Browse, edit, upload, download and extract ZIPs through
any browser on the LAN; single self-contained ELF, no external services.
Highlights (v1.7):
- ZIP large-file profile (large=1): 2 TiB total / 1 TiB per entry / 1000:1
ratio. Frontend prompts for confirmation whenever archive > 60 GiB on disk.
- Default ZIP profile stays safe: 512 GiB / 64 GiB / 200:1.
- 69 host-side C tests (tests/run-tests.sh) covering traversal, ZIP64,
encryption rejection, conflict policies and the large-file profile.
- Three-phase extraction engine (scan -> extract -> publish -> cleanup) with
atomic staging + rename, located in src/zip_extract.{c,h}.
Project layout:
src/ C payload sources
assets/ HTML / CSS / JS / icons / param.json
tests/ POSIX/host test suite + fixtures
third_party/ vendored zlib + minizip-ng (zlib license)
docs/screenshots/ README screenshot images
LICENSE GPLv3+
Build:
make # cross-compile with PS5_PAYLOAD_SDK
make linux # Linux host binary for UI dev
Songlx516