Most 7z folders are a single plain LZMA2 coder (7-Zip default -m0=lzma2
-ms=on). For that shape the facade now drives the SDK's own parallel
decoder (Lzma2DecMt, vendored with MtDec/Threads) instead of the
single-threaded chain walk; the chain stays responsible for every other
shape (BCJ2 stacks, encrypted folders, exotic methods) and as the
fallback when the platform cannot provide threads (SZ_ERROR_THREAD
downgrades, never fails).
Adapters: ISeqInStream over the read_at callback, ISeqOutStream into the
staging sink (runs on the calling thread, so the sink contract is
unchanged), ICompressProgress polling the cancel hook. The folder CRC is
taken over the decoded stream as before.
329 MiB fixture, internal timing: 1050 ms (asm, 1 thread) -> 930 ms
(4 threads) -> 765 ms (8 threads + 1 MiB inBufSize_MT), i.e. 1.37x,
now at parity with 7za -mmt=off (898 ms); 7za -mmt=8 is 485 ms. The PS5
count is 8 Zen 2 cores like the dev host; SZX_MT_THREADS=8.
Test matrices: 7z 28 + ZIP 108 + RAR 27 checks green.
UI regression: a 7z with fewer than 4096 entries spun a "scanning archive"
label and never replaced it until extraction had been running long enough
for the byte-level callback to fire, on a slow PS5 that meant "looks hung".
Lower the per-entry barrier in scan_entries to 256 entries, force a final
report after the scan so entries_total / bytes_total always reach the UI,
and force a report at the start of precheck_folders so the throttled
"between scan end and first extraction byte" window is not silent either.
Error regression: when the password was wrong the engine returned
ZIPX_ERR_PASSWORD with an empty detail field; extract_set_error copied it
into task->error_arg verbatim, so the i18n template's {arg} expanded to ""
and the user saw "Wrong password: ". Carry the original archive path into
the error detail so the toast reads "Wrong password: myfile.7z".
Languages: drop the stale "only ZIP/RAR" line from err_extract_unsupported
(7z has been supported since v1.9), and add the new err_extract_password
key in both en/zh.
Tests: lock down both regressions.
* progress recorder: bytes_done monotonic, total > 0, final byte count
reaches >= 90% of the declared total.
* wrong password: r.detail contains the archive basename.
7z 28 + ZIP 108 + RAR 27 = 163 checks, 0 failures.
The 7z chain decoder (commit 2748b38) handles bytes; what the /api/extract
task needed was a third sibling of zip_extract.c and rar_extract.c that
runs that chain over a real archive, mirrors their staging / publish /
rollback / name-validation machinery, and fills in zipx_result_t the same
way so the dispatcher and the web UI can treat every format alike.
* src/sevenz_extract.[ch] -- one pass over the archive (scan: validate every
name, sum bytes, drop archives the chain cannot drive), then folder by
folder through the chain. The publish phase recurses with OVERWRITE and
MERGE alike for matching directories, and only differs at the file
leaves -- a strict non-recursive OVERWRITE would have made the policy
useless for any archive that overlaps a directory already on disk.
* src/zipx_common.c -- the limits profiles and zipx_status_string() that
the three engines share. Pulled out of zip_extract.c so every consumer
gets the same error text and the same MAX_* numbers.
* src/zip_extract.[ch] -- ZIPX_ERR_PASSWORD joined the contract (7zAES).
* tests/run-sevenz-tests.sh -- the engine matrix now also runs the facade
over every fixture, plus 22 error and policy checks against
test_sevenz_extract. KNOWN_GAPS holds only "aeshe" (encrypted header);
the plain 7zAES fixture is in the matrix.
* tests/test_sevenz_extract.c -- end-to-end driver for the facade.
The POSIX shim the host test runner injects needed three more pieces so
the tests pass on a CP936 host:
* lstat/stat -> wfm_stat (the MinGW ANSI entry points can't see a UTF-8
filename in CP936; without the redirect a non-ASCII entry causes the
publish phase to lstat() a mangled path and fail with ENOENT).
* opendir/readdir/closedir -> the wide variants, so readdir() hands us
real UTF-8 names instead of CP936 bytes that nothing can round-trip.
* fopen -> _wfopen, for the test helpers that read a non-ASCII file
back to verify it.
ZIP 108 checks / RAR 27 checks / 7z 26 checks, all green; the AES
fixture extracts with the password "Secret123" and rejects wrong / absent
passwords with "password required or wrong" / "wrong password, or the
archive is damaged".
tests/fixtures/ gains the volume-set fixtures (broken.zip.001, gap.zip.*,
disks.*, parts.part*.zip, plain.zip.*, split_single.zip) that were
generated by tests/make_split_fixtures.py in earlier sessions but never
made it into the index.
Next commit wires sevenz_extract() into the dispatch in src/extract.c and
recognises 7z archives in assets/main.js.
7-Zip wraps the packed stream of a password protected archive in a 7zAES
coder (method 0x06F10701). The bundled LZMA SDK has no C implementation of
it, so add one to the folder chain:
* aes_props_layout() splits the property bytes into numCyclesPower, salt
and IV exactly as CDecoder::SetDecoderProperties2() does, and rejects a
block whose declared lengths do not match its size;
* aes_derive_key() runs the 7-Zip KDF: SHA-256 over
(salt || password_utf16le || counter_le64) repeated 1 << numCyclesPower
times, with the 0x3F power meaning "no derivation", the key being the
salt and password copied into 32 bytes;
* utf8_to_utf16le() converts the caller's password, since that is the
encoding 7-Zip hashes;
* the SZ_N_AES node decrypts one block at a time with Aes_SetKey_Dec() /
AesCbc_Init() / g_AesCbc_Decode(), keeping the trailing partial block in
the node so the declared (unpadded) unpack size is what gets delivered.
sz_chain_decode() gains a `password` argument, and sz_chain_needs_password()
lets a caller ask for one before it touches the filesystem. A folder that
needs a password and did not get one fails as SZ_CHAIN_ERR_PASSWORD with a
message naming 7zAES, rather than as a generic corrupt archive.
Because a wrong password decrypts to plausible looking rubbish, a failure
from a folder that carried a 7zAES coder gets "(a wrong password looks like
this)" appended, so the UI can offer a retry instead of calling the file
damaged.
numCyclesPower comes from the archive and drives 2^n SHA-256 passes, so it is
capped by the limits profile (max_aes_cycles, 2^24 for both the default and
the large profile).
An encrypted *header* (-mhe=on) is a different problem: the archive header is
encrypted with the same coder and has to be decrypted before any folder
exists. That stays out of scope and is reported as such.
tests/run-sevenz-tests.sh: aes leaves KNOWN_GAPS and passes; aeshe stays
there with the reason spelled out.
A split 7z (`name.7z.001`, `.002`, ...) only contains the first slice of the
archive, so SzArEx_Open() followed the header offsets off the end of the file
and gave up with SZ_ERROR_INPUT_EOF -- the set was unusable even though the
file listing of each part looked fine.
src/sevenz_volstream.c backs ISeekInStream with the ordered part list, so the
SDK sees one continuous archive and never learns it was split. Nothing is
merged on disk: a 160 GiB set would otherwise need a second 160 GiB scratch
copy. A 7z split is a plain byte split, so offsets stored inside the archive
are already absolute and the Stream/Seek callbacks stay trivial.
The ordered list comes from zipx_volume, which already understands
`name.7z.001` naming, so an incomplete set fails before any decoding starts
and names the missing part. Two further checks report a set that could not
have worked anyway: a lone first volume, and a part whose size differs from
the earlier ones (7-Zip cuts equal sized parts and lets only the last one be
short).
To reuse the split-layout constants without dragging the ZIP stream header
into the 7z build, ZIPX_VOL_MODE_* moved to zipx_volume.h: they describe the
set, and every consumer of zipx_volume_t needs them.
The e2e driver now opens its input through the volume stream, so the packed
data reads go through it too.
Matrix: all 10 single-volume fixtures plus vol.7z.001 are byte-identical
(13 passed, 0 failed). Remaining known gaps: 7zAES (aes, aeshe).
Regression: ZIP 108 / RAR 27 checks, 0 failures.
The LZMA SDK's CSzFolder caps a folder at four coders, so anything 7-Zip
writes for BCJ2 (4 packed streams + 5 coders) comes back as
SZ_ERROR_UNSUPPORTED from SzAr_DecodeFolder. The header scanner has no such
limit (64 coders), which makes the failure look like corrupt data instead.
src/sevenz_chain.c parses the folder descriptor itself and drives the codec
chain as a pull pipeline: every decoder hands up to N bytes to its consumer
and asks upstream for more when it needs it, so a folder is decoded straight
into the caller's sink without staging the whole thing. Covered here:
- Copy / LZMA / LZMA2 / PPMd, Delta, and the x86, PPC, IA64, ARM, ARMT,
SPARC branch converters
- BCJ2, whose three side streams have to be materialised while MAIN keeps
streaming
- per coder output sizes taken from UNPACK_INFO, which is what keeps the
chains exact
tests/sevenz_chain_e2e.c decodes each folder once and splits the byte stream
across the entries that share it, mirroring the real extraction path, and
checks the per-entry and per-folder CRCs on the way past.
tests/make_sevenz_fixtures.py builds the fixture matrix (including
non-solid `-ms=off` folders), and tests/run-sevenz-tests.sh runs it while
keeping a KNOWN_GAPS list that fails loudly once a gap closes.
Two real bugs surfaced while bringing the matrix up:
- build_coder() gave every coder node the *folder's* unpack size instead of
its own entry in UNPACK_INFO. In a BCJ2 folder MAIN is regularly larger
than the folder's final size (300066 vs 300000 in the fixture), so the
node silently truncated its own output and the chain came up short by 21
bytes per LZMA2 layer. Only archives where a coder is bigger than the
folder tripped it, which is why single-folder BCJ2 passed and the same
archive split into per-file folders did not.
- the e2e driver kept `written` across folders, so a folder that failed
mid-entry made every later entry look half-written and the failures
cascaded.
Scripts no longer delete anything: objects are overwritten and each case gets
a fresh output directory, so the suite runs under a guarded shell.
Matrix: 10/10 single-volume fixtures byte-identical (store, lzma2, lzma,
ppmd, bcj, delta, utf8, bcj2, solidoff, bcj2off). Still open and listed as
known gaps: 7zAES (aes, aeshe) and multi-volume input (vol.7z.001).
Groundwork for 7z support (the second of the six format x volume
combinations). No engine code yet: this lands the vendor subset, real
fixtures and the end-to-end driver, so the engine has something to be
built and checked against.
Vendor (third_party/7z, LZMA SDK 26.03, public domain):
- decoder-only subset: 7zArcIn/7zDec container, Lzma/Lzma2/Ppmd7/Bcj2/
Bra/Delta codecs, Aes/Sha256 for the password channel to come
- the encoder half (LzmaEnc, Xz*, Sort, Threads, ...) is deliberately
not vendored; the engine never encodes
- README.md records two SDK limitations that shape the engine design:
* CSzFolder caps a folder at 4 coders / 3 bonds, so 7-Zip's own BCJ2
chain (BCJ2 + 4xLZMA2 = 5 coders) is refused by SzAr_DecodeFolder
even though SzArEx_Open parses it happily - the engine therefore
parses folder blobs and drives the codec chain itself
* the C decoder ships no 7zAES coder at all, so both -p and -mhe=on
archives are rejected until the engine implements it on top of the
vendored Aes.c / Sha256.c
Tests:
- make_sevenz_fixtures.py builds real archives with an actual 7-Zip
binary (7za from the Extra package; falls back to the reduced 7zr,
which has no PPMd encoder): store/lzma/lzma2/ppmd/bcj/delta/bcj2,
AES with plain and with encrypted headers, a 7-part -v100k volume
set, and UTF-8 entry names
- sevenz_e2e.c extracts an archive and validates every entry against
the stored CRC; it calls the wide-character file APIs on Windows, so
non-ASCII entry names are really created instead of silently failing
- run-sevenz-tests.sh builds the subset and the driver, then runs the
fixture matrix with an explicit KNOWN_GAPS list that fails loudly
when one of the gaps starts passing
Status: store, lzma, lzma2, ppmd, bcj, delta and utf8 all extract
byte-identical to the source tree (7/10). bcj2, aes, aeshe and
vol.7z.001 are the known gaps, each mapped to a specific piece of
engine work still to come.
A split ZIP is several files that only form an archive together. Until now a
`.zip.001` was rejected as an unsupported format and a `.z01`/`.partN.zip`
member could not be resolved, so these downloads were unusable.
Three naming conventions are recognised, from any member of the set:
name.zip.001, name.zip.002, ... byte split (7-Zip)
name.part1.zip, name.part2.zip byte split (WinRAR)
name.z01, ..., name.zip zip split disks (Info-ZIP / PKZIP)
The parts are served through a new stream that makes them look like one
archive, so nothing is copied to disk first (a 160 GiB set would otherwise
need a second 160 GiB scratch copy):
- src/zipx_volstream.c implements an mz_stream over the ordered part list.
Byte splits keep absolute offsets; zip split disks carry offsets relative to
the disk an entry starts on, so that mode honours
MZ_STREAM_PROP_DISK_NUMBER the way mz_zip_entry_seek_local_header drives it
and starts on the last volume, where the central directory lives.
- src/zipx_volume.c groups the set, verifies the numbering is complete and
reports exactly which volume is missing.
- zip_extract.c tries the layout implied by the naming first and the other one
as a fallback, so tools that disagree about offset bases still work.
- extract.c dispatches volume members to the engines and, when the caller
asked for the source to be deleted, removes every volume instead of leaving
orphaned parts behind.
Error messages are specific on purpose: an incomplete set reports the missing
file name (e.g. "gap.zip.002 is missing"), a lone first volume says the
remaining volumes are missing, `.rar.001` sets explain the rename that makes
them work, and 7z volumes say 7z is not supported yet.
Tests: tests/make_split_fixtures.py builds all three layouts (the disk set is
written with per-disk offsets exactly as APPNOTE 4.4.11 describes) plus two
broken sets; test_zip_extract.c extracts each layout from a different member
and byte-compares the nested binary entries. 108 ZIP + 27 RAR checks, 0
failures.
Two issues found in real-machine testing of v1.9:
1. RAR multi-volume extraction never advanced the progress bar. The engine
only accounted bytes_done after RARProcessFile() returned for a whole
entry, so a multi-GB entry spanning volumes froze the UI for its entire
duration. Wire RARSetCallback/UCM_PROCESSDATA, which unrar fires per
decompressed chunk during disk extraction, and fold each chunk into
bytes_done + a throttled progress report. (DLL-mode break handling is
never armed, so cancellation stays entry-granular; returning -1 would
be ignored.)
2. A 95k-file game zip was refused as a "compression bomb" because one
small entry exceeded the per-entry uncompressed/compressed ratio cap
(500/1000). Such entries are common and harmless in legitimate archives
(zero-filled placeholders, sparse blobs): actual bytes written are
bounded by the declared size (enforced during extract) and check_space()
verifies the full declared total against real free space before any
writes. Ratio screening now only applies to entries >= 1 GiB
(ratio_min_bytes, default in both profiles); sub-GiB high-ratio entries
are accepted. Error text now includes compressed -> uncompressed sizes.
Tests updated: small high-ratio bomb.zip now extracts OK under both
profiles and is rejected again when ratio_min_bytes is lowered below its
size; RAR multi-volume fixture asserts mid-entry progress events and that
final bytes_done reaches bytes_total. 73 ZIP + 27 RAR checks, 0 failures.
Host-side verification that the ZIP engine can extract a >4GiB zip64
entry end to end (4.7 GiB fixture, content byte-compared):
- tests/compat/sys/statvfs.h: resolve the path before taking the drive
letter (relative paths used to fail) and scale f_bavail by 4096 since
`unsigned long` is 32-bit on Windows and silently truncated ~250GB
down to ~2.3GB; real 64-bit hosts and the PS5 (LP64) are unaffected.
- tests/bigfile_e2e.c: standalone driver to extract one large archive
and print the engine result (verification via cmp/sha256 by caller).
- tests/run-tests.sh: incremental clean instead of `rm -rf` of the whole
build tree.
- Makefile: force -DHAVE_FSEEKO so minizip uses fseeko/ftello instead of
falling back to fseek/ftell on libcs without the probe.
Result: 4.7 GiB zip64 extracts OK on host in ~6s, output identical to
source. Engine 64-bit write path confirmed clean; PS5 failure needs the
strerror passthrough build for real-machine errno capture.
Host (MinGW) tests default to 32-bit off_t, so minizip lseek overflowed
on the >4 GiB zip64 fixture ('Invalid argument' opening big.zip). The PS5
Makefile already defines _FILE_OFFSET_BITS=64; this makes the host engine
matches production and lets the >4 GiB path actually be tested here.
Generated once with tests/make-rar-fixtures.bat (WinRAR RAR 7.23) so
checkouts without WinRAR still exercise the real-archive happy paths.
basic-rar4.rar is absent because RAR 7.x dropped RAR4 writing; that
check SKIPs.
unrar in RAR_OM_EXTRACT mode presents a file that spans volumes as
several header segments with the SAME name, flagged RHDF_SPLITBEFORE/
SPLITAFTER. The scan dedup rejected the continuation segments as
"duplicate entry name" and the whole multi-volume extract failed.
Fix: segments after the first (RHDF_SPLITBEFORE set) skip the
normalize/dedup/size/limit accounting in scan (the first segment already
carries the full size) and, in extract, still call RARProcessFile(EXTRACT)
to drive the volume chain but are not counted or size-accumulated.
Verified against real fixtures (tests/fixtures-real/, generated by
tests/make-rar-fixtures.bat on the user's RAR 7.23): a 512 KB file split
across vol.part1/2/3.rar extracts whole (status ZIPX_OK, big.bin + root.txt).
Host suite now 70 ZIP + 24 RAR checks, 0 failures.
-epl + absolute stage paths stored dir\nested.txt as nested.txt, so the
v6/volume happy-path checks found no dir\nested.txt. Generator now cd's
into the stage dir and passes relative names without -ep1.
tests/make_fixtures.py rmtree()s tests/fixtures on every run-tests.sh
invocation, silently deleting the real RAR fixtures. The WinRAR generator
now writes tests/fixtures-real/; run-tests.sh passes it as a third
argument to test-rar-extract, which resolves v6/volume/encrypted checks
against it and falls back to SKIP when absent.
Naming the target vol.part1.rar made RAR 7 append its own numbering onto
the whole stem -> vol.part1.partN.rar. Plain vol.rar splits into the
standard vol.part1/2/3.rar the test expects.
rar a updates an existing archive instead of re-splitting it, so the
single-volume vol.part1.rar left by earlier runs (small payload) was
grown to 512KB without ever producing part2/3. Remove vol.part* first.
The earlier -v200k split never produced part2 because the payload was a
few hundred bytes. Now generates a 512KB incompressible file (fsutil +
-m0 store) so vol.part1/2/3.rar actually exist for the volume test.
Write tool wrote UTF-8 Chinese comments which cmd.exe parsed under the
ANSI codepage as garbage ('M' is not a command). This version is pure
ASCII with CRLF line endings. RAR4 (-ma4) creation fails on newer RAR
builds; that step now warns and continues instead of aborting the whole
run. Requires no behavioural change from the engine side.
tests/make-rar-fixtures.bat produces real RAR fixtures on a machine with
WinRAR (Rar.exe) so the RAR engine is finally exercised against genuine
archives instead of 11-byte placeholders (v1.8 shipped without ever
running a real RAR through the happy path):
- basic-v6.rar RAR5 with WinRAR 6/7 "v6" compression (the format
dmc_unrar could not decode — the v1.9 trigger)
- vol.part1+2.rar multi-volume (200 KB split)
- enc-v6.rar encrypted (password secret123)
- basic-rar4.rar legacy RAR4
tests/test_rar_extract.c gains test_real_archives(): asserts v6 single
volume and RAR4 extract OK with expected files on disk, multi-volume
auto-merges from the same directory, and encrypted archives are rejected
up front with ZIPX_ERR_UNSUPPORTED (password channel lands separately).
Each check SKIPs cleanly when the fixture is absent, so plain CI
checkouts stay green.
Run: tests/make-rar-fixtures.bat (WinRAR required), then rerun
tests/run-tests.sh to flip the SKIPs into real assertions.
Replace the dmc_unrar 1.7.0 backend (third_party/unrar/) with the rarlab
UnRAR 7.20.1 source vendored in third_party/unrar7/ (commit c68c0de),
driven through its C-compatible DLL API.
Why: dmc_unrar only dispatches RAR5 compression version 5
(switch(file->version) case 0x5000). WinRAR 6.x/7.x archives (algorithm
"v6") land on the default branch -> DMC_UNRAR_FILE_UNSUPPORTED_VERSION
-> user-visible "corrupt archive", confirmed on a real v6:8M archive on
2026-09-05. unrar 7.20.1 natively handles v6, multi-volume (.partNN.rar
auto-merge) and encrypted archives.
Engine changes (src/rar_extract.c):
- Facade include swapped to third_party/unrar7/unrar_c_api.h; all calls
now go through RAROpenArchiveEx/RARReadHeaderEx/RARProcessFile/
RARCloseArchive. scan + extract each re-open the archive (the DLL API
is sequential, unlike dmc's index-based access).
- Error translation rewritten for the ERAR_* code set.
- extract no longer builds each staging path by hand (open_parent_dirs/
extract_one removed): unrar writes the whole tree under the staging
root; names were validated during scan.
- Bug fix: normalize_name() cleared the caller-provided *is_dir (it set
*is_dir = 0), turning directory entries into files and tripping the
duplicate detector whenever an explicit directory header followed
files beneath it (real v6 archive had exactly this). RHDF_DIRECTORY
is now honored end to end.
- Encrypted RAR still fails up front (ZIPX_ERR_UNSUPPORTED): unrar can
decrypt, but password plumbing (API/UI) lands in a later commit.
Build:
- Makefile: C++ compiler vars (CXX/HOST_CXX, prospero-clang++ defaults
to -stdlib=libc++), unrar7 RARDLL source set (49 files, mirrors
UnRARDll.vcxproj), .cpp pattern rules, link through the C++ driver.
- tests/run-tests.sh: compiles unrar7 objects with $CXX, links the RAR
test with g++ and -lpowrprof on Windows.
- third_party/unrar/ (dmc_unrar) removed; see git history.
Verified on host (MinGW): full engine run on the user's real v6 archive
(24.9 MB, 170 headers) -> ZIPX_OK, 170/170 entries, 133 files + 37 dirs,
71,514,931 bytes. Host suite: 70 ZIP + 14 RAR checks green.
Frontend password/multi-volume UX and release plumbing (CHANGELOG,
THIRD_PARTY_NOTICES, README, tag) follow in the v1.9 release commit.
A 3A-game archive with a single ~300 GiB uncompressed file was silently
rejected by the v1.8.1 default profile (scan_archive returns
ZIPX_ERR_LIMIT_FILE_SIZE before the request ever reaches the frontend
confirmation prompt, so the user just sees "卡壳"). The 256 GiB default
cap was tuned for PS5 system backups (many smaller entries) and was wrong
for the 3A-game single-file case.
Limits changes:
- src/zip_extract.c k_default_limits: 1 TiB / 256 GiB / 500 -> 2 TiB / 512 GiB / 500
- src/zip_extract.c k_large_limits: 2 TiB / 1 TiB / 1000 -> 4 TiB / 1 TiB / 1000 (must stay > default)
- assets/main.js LARGE_FILE_THRESHOLD_BYTES: 240 GiB -> 480 GiB
- assets/lang-{en,zh}.js extractLargeAsk copy reflects new numbers
- tests/test_zip_extract.c: large.max_total_bytes advertised-number assertion updated to 4 TiB
- README.md / docs/HANDOVER.md: limit tables, FAQ, threshold snippets updated
- CHANGELOG.md: v1.8.2 entry added above v1.8.1
PS5-only build fixes (caught by WSL cross-compile, host tests pass without):
- Makefile CFLAGS: add -Ithird_party/unrar so src/rar_extract.c can find
the dmc_unrar_api.h facade header (THIRD_PARTY_CFLAGS already had it
but the main CFLAGS for project sources did not).
- src/extract.c: move extract_progress() definition before
extract_dispatch() so the implicit function declaration is not flagged
by -Werror=implicit-function-declaration (clang 18 in the PS5 SDK is
stricter than the host gcc used by tests/run-tests.sh).
Release artifact: web-file-mgr.elf 509 704 bytes,
sha256 1b2c3d68b35e32737105f17d14a80a3c159ceca0cabd274ee168cbcd81906f65,
e_machine 0x003e (x86_64-sie-ps5).
Safety argument is unchanged from v1.8.1: zip-bomb defence is
check_space() (statvfs-based real disk-space check before staging) +
max_ratio (declared compression ratio cap). The size caps are a UX guard,
not a security boundary.
RAR extraction inherits the new defaults automatically (rar_extract.c
threads c->limits from the engine).
84 host-side checks (70 ZIP + 14 RAR), 0 failures. PS5 cross-compile
succeeds; ELF size grew from 427 656 (v1.8) to 509 704 (v1.8.2) due to
the dmc_unrar vendor TU being linked in.
User feedback: the previous default limits were an over-cautious UX
guard, not a security guard. check_space() already enforces available
>= bytes_total before staging begins, and max_ratio already rejects
classic zip bombs at any declared ratio above the cap. A user with a
multi-hundred-GiB PS5 system image shouldn't have to click through a
confirmation prompt for an obviously safe archive.
k_default_limits (src/zip_extract.c:36-44) relaxed:
- max_total_bytes: 512 GiB -> 1 TiB
- max_file_bytes: 64 GiB -> 256 GiB
- max_ratio: 200 -> 500
k_large_limits unchanged (1 TiB / 1 TiB / 1000).
Frontend threshold LARGE_FILE_THRESHOLD_BYTES (assets/main.js:838)
bumped 60 GiB -> 240 GiB to track the new default. The 240 GiB value
keeps the same 4x head-room over the default cap as the previous 60
GiB did, so the confirm() prompt only triggers for archives that
truly warrant the user paying attention.
assets/lang-{en,zh}.js extractLargeAsk copy updated to reflect the
new default-profile numbers (format-agnostic; same string for .zip and
.rar).
README.md: 'Stricter default' line under "What's new in v1.7", both
limit tables (the ZIP section and the RAR section), the "Tuning the
threshold" snippet, and the err_extract_entry_too_large FAQ entry all
updated. CHANGELOG.md gets a new [v1.8.1] - 2026-09-05 section
documenting the relaxation, the rationale (check_space + max_ratio
are the real guards), and explicit migration notes.
docs/HANDOVER.md and docs/UPGRADE-v1.8-rar-support.md numeric
references updated. The historical v1.7 upgrade document
(docs/UPGRADE-v1.7-zip-large-file-profile.md) is deliberately left
unchanged so the v1.7 -> v1.8.1 evolution remains traceable from git.
src/rar_extract.c is untouched: it already threads c->limits from
the engine and picks up the new defaults for free.
tests/test_zip_extract.c test_large_profile() rewritten for the new
ratio cap:
- medium_bomb.zip (ratio ~238) is now accepted by both default and
large profiles (showing real-world high-ratio archives aren't
artificially blocked)
- bomb.zip (ratio ~1027) is rejected by BOTH default and large
profiles (a bomb is a bomb regardless of which profile you opt
into)
- User-lowered tight ratio cap (200) still rejects medium_bomb
(proves caps are enforced, not just nominal)
- User-lowered tight file cap still rejects zip64 entries
Final test count: 70 ZIP + 14 RAR = 84 checks, 0 failures (was 69+14).
Migration:
- Forward-compatible: existing v1.7/v1.8 deployments that never
triggered err_extract_entry_too_large see no difference
- No data loss: relaxation only widens accepted archives
- Real guards (check_space, max_ratio, path traversal) unchanged
Tests/make_fixtures.py now patches zipfile.time so Python 3.13's
zipfile.writestr no longer embeds wall-clock date_time into each entry
(the module uses time.localtime(time.time())[:6] when the caller
supplies a string arcname, which silently made every fixture diff on
each regeneration and polluted git with thousands of unrelated byte
changes). The patch fakes localtime/time to always return
(1980, 1, 1, 0, 0, 0).
All .zip fixtures are regenerated against the new deterministic
generator and committed. notar.rar is regenerated alongside because
test_rar_extract.c's main() recreates it from basic.zip at run time;
including it here keeps the source fixture and the rar fixture in
lock-step.
Verified: 2 consecutive python3 make_fixtures.py runs produce identical
md5 for all 19 .zip fixtures; full test suite stays green
(69 ZIP + 14 RAR = 83 checks, 0 failures).
tests/test_rar_extract.c — 14 checks, zero dependency on host having
rar or 7z installed:
test_engine_dispatch (4):
- not-a-RAR rejected: notar.rar (renamed basic.zip) -> FORMAT
- junk blob rejected: junk.rar (1024 random bytes) -> FORMAT
- missing source -> OPEN
- null rar_path / dst -> INTERNAL
test_format_translation (6):
- rar_translate_error mapping table covers every dmc_unrar_return
code rar_extract can emit, verified against fixture files
test_limits_handoff (4):
- default vs large limits profile passed through unchanged
- max_entries / max_total_bytes / max_file_bytes / max_ratio
Uses tests/posix_compat.h (open/close/mkdirat renames) via gcc -include
for host builds -- never compiled into the PS5 payload.
tests/make_fixtures.py — adds rar_fixtures() that prefers host 'rar'
(non-free) and falls back to host '7z' (LGPL). If neither is present,
emits an 11-byte 'placeholder' basic.rar so the negative tests still
fire (basic.rar is positive-path, but is only read when a RAR writer
exists). Wired into the fn list in main().
tests/run-tests.sh:
- Builds third_party/unrar/dmc_unrar.o with -DDMC_UNRAR_DISABLE_BE32TOH_BE64TOH=1
- Builds rar_extract.o with -Ithird_party/unrar and posix_compat shim
- Builds test_rar_extract.o against both rar_extract.o and zip_extract.o
(the latter is required: rar_extract.c references zipx_status_string,
zipx_default_limits, zipx_limits_profile for error/log/output)
- Links test-rar-extract, then runs both test-zip-extract and
test-rar-extract in sequence.
Fixtures committed:
- basic.rar -- 11-byte placeholder (host has no RAR writer)
- notar.rar -- basic.zip renamed, for format-rejection test
- junk.rar -- 1024 random bytes, for format-rejection test
- All other .rar fixtures (encrypted, multi-volume, symlink, ...) are
generated on demand when run on a host with rar/7z installed.
Final test count on a writer-less host: 69 ZIP checks + 14 RAR checks
= 83 total, 0 failures (re-verified post-commit).
Per user decision on 2026-09-04 21:01 GMT+8, the v1.8 plan no longer
covers the legacy split-archive format (name.rar + .r00/.r01/.r02...).
Rationale: PS5 users almost exclusively ship the modern RAR5
part-volume format (name.part01.rar, name.part02.rar, ...); the
legacy .rNN format is from the 2005-2010 era. Removing it cuts:
- Frontend: isRarSubVolume() no longer needs /\.r\d{2,}$/ branch
- Fixtures: drop rar4_multi_old.rar generation
- Tests: drop test_rar4_multi_volume_old()
- Backend: unrar API surface still supports legacy .rNN, but we just
never feed it one
Documentation sweep:
- Section 5.1: range updated
- Section 6.4.1: isExtractableArchive/isRarSubVolume simplified
- Section 6.5.1: fixture generation script updated
- Section 6.5.3: test_rar4_multi_volume_old removed
- Section 8.4: decision rationale no longer mentions .rNN
- Section 6.6.4: CHANGELOG / README example updated
- Section 11.2: test matrix updated
- Section 6.4.9: D4 manual test list no longer tests .r00
Homebrew HTTP file manager payload for jailbroken PS5 consoles (x86_64-sie-ps5,
title id FMGR88888). Browse, edit, upload, download and extract ZIPs through
any browser on the LAN; single self-contained ELF, no external services.
Highlights (v1.7):
- ZIP large-file profile (large=1): 2 TiB total / 1 TiB per entry / 1000:1
ratio. Frontend prompts for confirmation whenever archive > 60 GiB on disk.
- Default ZIP profile stays safe: 512 GiB / 64 GiB / 200:1.
- 69 host-side C tests (tests/run-tests.sh) covering traversal, ZIP64,
encryption rejection, conflict policies and the large-file profile.
- Three-phase extraction engine (scan -> extract -> publish -> cleanup) with
atomic staging + rename, located in src/zip_extract.{c,h}.
Project layout:
src/ C payload sources
assets/ HTML / CSS / JS / icons / param.json
tests/ POSIX/host test suite + fixtures
third_party/ vendored zlib + minizip-ng (zlib license)
docs/screenshots/ README screenshot images
LICENSE GPLv3+
Build:
make # cross-compile with PS5_PAYLOAD_SDK
make linux # Linux host binary for UI dev
Songlx516