9 Commits
Author SHA1 Message Date
Songlx516 ba668ade50 release: v1.9.3M -- encrypted archives, plus the UI round that followed
First release under the fork-marker convention: VERSION_TAG carries a trailing
`M`, so /api/version, the PS5 start-up notification, the stdout banner, the UI
footer and the ELF file name all read "v1.9.3M" in one move -- and a fork build
can no longer collide with an upstream artifact of the same version, a mix-up
that already happened twice. The footer carries a tooltip spelling the marker
out.

Two bodies of work.

1. Encrypted archives (ZIP / RAR / 7z)

   - ZIP: ZipCrypto (traditional PKWARE) and WinZip AES-256, through
     minizip-ng plus a vendored crypto layer (mz_crypt_wfm.c,
     mz_strm_pkcrypt.c, mz_strm_wzaes.c).
   - RAR: RARSetPassword, wired after RAROpenArchiveEx and before the first
     RARReadHeaderEx. The ordering is load-bearing, not stylistic.
   - 7z: 7zAES including -mhe=on encrypted headers, via a virtual
     ISeekInStream that splices a pseudo-header + the real archive + the
     decrypted header, so no offset stored inside the archive has to move.

   A wrong password is reported as ZIPX_ERR_PASSWORD, and a failed attempt
   leaves no staging directory behind.

2. Reporting, and the UI round that on-device testing produced

   - A RAR whose dictionary exceeds what the build supports now gets its own
     extract_dict_too_large code instead of being mis-reported as "entry too
     large"; the message names both the required and the supported size. The
     behaviour is deliberately unchanged -- such archives are still refused,
     because admitting one means allocating the whole window up front, which
     is why rarlab's own CLI refuses them by default.
   - The upload entry is a menu again: one "Upload" button opening "Upload
     files / Upload folder". The previous main-button-plus-small-arrow made
     "upload folder" effectively undiscoverable.
   - A drag-and-drop hint sits in the footer (hidden on the console browser,
     where drag is not how anyone uploads).
   - The extract button is now always present and merely disabled until
     exactly one archive is selected, instead of appearing out of nowhere.
   - Upload-and-extract on an encrypted archive now prompts for the password
     directly. The retry table used to be keyed by PATH, and for a non-ASCII
     directory the string the page holds and the string the server reports
     are not the same bytes -- the lookup missed, so the user got a bare
     error box and had to press Extract by hand before the prompt appeared.
     It is now keyed by task id, which the server assigns and echoes back
     verbatim.
   - Error text passes through decodeFsText(), so a GBK entry name no longer
     surfaces as `â®…ç§.psd`.
   - Local names are encoded with encodeFsText() before being joined onto a
     server-side path. fs_path_value() declines to rewrite a path if ANY code
     point exceeds 0xFF, so concatenating a local name onto a server directory
     produced a mixed representation and a silently dead path.
   - The menu row highlight was losing the cascade to the generic button rule
     (identical specificity, later in the file) while inheriting the toolbar's
     3px focus ring, which overflowed a 46px row. Both rules are now scoped to
     the panel and the keyboard cue is an inset ring, so it cannot escape the
     row at any line height.
   - The footer status line is clamped to a single line; a long
     "uploading 3/12: some-name.zip" used to wrap out of the 46px footer.
   - A first failed password attempt now says the archive is encrypted,
     instead of blaming a password the user was never asked for.

Artifact
  web-file-mgr-v1.9.3M.elf
  903,448 B
  sha256 8ca47d5aaca75085b32641300cce30fadb7df7749cb6b53d04f129bcecc286b7
  e_machine 0x003e (x86-64 / PS5)

  The file size is identical to the four builds before it, and every one of
  them carries a different sha256: only .rodata moved, and by less than the
  16 KiB section alignment absorbs. Compare sections with `readelf -SW` --
  never infer "nothing changed" from the byte count.

Verification
  - host suites: 140 ZIP + 37 RAR = 177 checks, 0 failures
  - 7z suite: 27 cases, 0 failures. The `aeshe` entry that used to sit in
    KNOWN_GAPS is gone -- the -mhe=on fixture now passes both the folder
    decoder and the extraction facade
  - frontend: .build/ui_retry_test.mjs (40 checks), .build/ui_upload_menu_test.mjs
    (40 checks), .build/preview_check.mjs (12 assertions in headless Chromium
    against the real page and a fixture API). Two layout regressions and the
    highlight cascade bug were caught by the last one and by nothing else --
    reading the source, both CSS rules "look correct"
  - built twice from this tree: byte-identical (cmp clean). rsync refreshes
    every asset mtime, so this is a genuine recompile, not make short-circuiting
    on unchanged sources
  - embedded assets verified in place with .build/check-elf-gzip.py, because
    gen-asset-module.py gzips them and plain `strings` finds none of their text
  - exercised end to end on a real PS5; the checklist is
    docs/DEVICE-TEST-v1.9.3M.md

Docs
  - docs/USER-GUIDE-zh-CN.md (new, simplified Chinese user guide)
  - docs/DEVICE-TEST-v1.9.3M.md (new, on-device acceptance checklist)
  - docs/REAL-CONSOLE-PROFILE.md (new, measured console behaviour)
  - docs/archive/HANDOVER-v1.8-planning.md (superseded v1.8 design notes)
  - CHANGELOG / README (both languages) / HANDOVER updated with the artifact
    fingerprint, the section deltas and the new test counts
2026-09-24 21:07:12 +08:00
Songlx516 5cc493d152 perf: multithread the plain-LZMA2 7z folders via Lzma2DecMt
Most 7z folders are a single plain LZMA2 coder (7-Zip default -m0=lzma2
-ms=on). For that shape the facade now drives the SDK's own parallel
decoder (Lzma2DecMt, vendored with MtDec/Threads) instead of the
single-threaded chain walk; the chain stays responsible for every other
shape (BCJ2 stacks, encrypted folders, exotic methods) and as the
fallback when the platform cannot provide threads (SZ_ERROR_THREAD
downgrades, never fails).

Adapters: ISeqInStream over the read_at callback, ISeqOutStream into the
staging sink (runs on the calling thread, so the sink contract is
unchanged), ICompressProgress polling the cancel hook. The folder CRC is
taken over the decoded stream as before.

329 MiB fixture, internal timing: 1050 ms (asm, 1 thread) -> 930 ms
(4 threads) -> 765 ms (8 threads + 1 MiB inBufSize_MT), i.e. 1.37x,
now at parity with 7za -mmt=off (898 ms); 7za -mmt=8 is 485 ms. The PS5
count is 8 Zen 2 cores like the dev host; SZX_MT_THREADS=8.

Test matrices: 7z 28 + ZIP 108 + RAR 27 checks green.
2026-09-16 14:30:29 +08:00
Songlx516 fd48232b6c perf: enable the SDK's assembly LZMA decoder (1.26x on LZMA2) 2026-09-16 13:07:56 +08:00
Songlx516 8bee84bd49 feat(7z): vendor LZMA SDK and add the 7z fixture/verification harness
Groundwork for 7z support (the second of the six format x volume
combinations). No engine code yet: this lands the vendor subset, real
fixtures and the end-to-end driver, so the engine has something to be
built and checked against.

Vendor (third_party/7z, LZMA SDK 26.03, public domain):

- decoder-only subset: 7zArcIn/7zDec container, Lzma/Lzma2/Ppmd7/Bcj2/
  Bra/Delta codecs, Aes/Sha256 for the password channel to come
- the encoder half (LzmaEnc, Xz*, Sort, Threads, ...) is deliberately
  not vendored; the engine never encodes
- README.md records two SDK limitations that shape the engine design:
  * CSzFolder caps a folder at 4 coders / 3 bonds, so 7-Zip's own BCJ2
    chain (BCJ2 + 4xLZMA2 = 5 coders) is refused by SzAr_DecodeFolder
    even though SzArEx_Open parses it happily - the engine therefore
    parses folder blobs and drives the codec chain itself
  * the C decoder ships no 7zAES coder at all, so both -p and -mhe=on
    archives are rejected until the engine implements it on top of the
    vendored Aes.c / Sha256.c

Tests:

- make_sevenz_fixtures.py builds real archives with an actual 7-Zip
  binary (7za from the Extra package; falls back to the reduced 7zr,
  which has no PPMd encoder): store/lzma/lzma2/ppmd/bcj/delta/bcj2,
  AES with plain and with encrypted headers, a 7-part -v100k volume
  set, and UTF-8 entry names
- sevenz_e2e.c extracts an archive and validates every entry against
  the stored CRC; it calls the wide-character file APIs on Windows, so
  non-ASCII entry names are really created instead of silently failing
- run-sevenz-tests.sh builds the subset and the driver, then runs the
  fixture matrix with an explicit KNOWN_GAPS list that fails loudly
  when one of the gaps starts passing

Status: store, lzma, lzma2, ppmd, bcj, delta and utf8 all extract
byte-identical to the source tree (7/10). bcj2, aes, aeshe and
vol.7z.001 are the known gaps, each mapped to a specific piece of
engine work still to come.
2026-09-12 17:17:45 +08:00
Songlx516 ae41434089 fix(build): restore POSIX type shims in unrar_c_api.h
The _WIN32-only include of windows.h left PS5/prospero builds (no _WIN32,
no _UNIX for project sources) with PASCAL/HANDLE/LONG undeclared when
rar_extract.c pulled in dll.hpp. Restore the _UNIX mirror block that was
lost in an earlier edit.
2026-09-05 23:31:38 +08:00
Songlx516 a0482a41ed feat(extract): swap RAR engine to unrar 7.20.1 (v1.9, v6/multi-volume)
Replace the dmc_unrar 1.7.0 backend (third_party/unrar/) with the rarlab
UnRAR 7.20.1 source vendored in third_party/unrar7/ (commit c68c0de),
driven through its C-compatible DLL API.

Why: dmc_unrar only dispatches RAR5 compression version 5
(switch(file->version) case 0x5000). WinRAR 6.x/7.x archives (algorithm
"v6") land on the default branch -> DMC_UNRAR_FILE_UNSUPPORTED_VERSION
-> user-visible "corrupt archive", confirmed on a real v6:8M archive on
2026-09-05. unrar 7.20.1 natively handles v6, multi-volume (.partNN.rar
auto-merge) and encrypted archives.

Engine changes (src/rar_extract.c):
- Facade include swapped to third_party/unrar7/unrar_c_api.h; all calls
  now go through RAROpenArchiveEx/RARReadHeaderEx/RARProcessFile/
  RARCloseArchive. scan + extract each re-open the archive (the DLL API
  is sequential, unlike dmc's index-based access).
- Error translation rewritten for the ERAR_* code set.
- extract no longer builds each staging path by hand (open_parent_dirs/
  extract_one removed): unrar writes the whole tree under the staging
  root; names were validated during scan.
- Bug fix: normalize_name() cleared the caller-provided *is_dir (it set
  *is_dir = 0), turning directory entries into files and tripping the
  duplicate detector whenever an explicit directory header followed
  files beneath it (real v6 archive had exactly this). RHDF_DIRECTORY
  is now honored end to end.
- Encrypted RAR still fails up front (ZIPX_ERR_UNSUPPORTED): unrar can
  decrypt, but password plumbing (API/UI) lands in a later commit.

Build:
- Makefile: C++ compiler vars (CXX/HOST_CXX, prospero-clang++ defaults
  to -stdlib=libc++), unrar7 RARDLL source set (49 files, mirrors
  UnRARDll.vcxproj), .cpp pattern rules, link through the C++ driver.
- tests/run-tests.sh: compiles unrar7 objects with $CXX, links the RAR
  test with g++ and -lpowrprof on Windows.
- third_party/unrar/ (dmc_unrar) removed; see git history.

Verified on host (MinGW): full engine run on the user's real v6 archive
(24.9 MB, 170 headers) -> ZIPX_OK, 170/170 entries, 133 files + 37 dirs,
71,514,931 bytes. Host suite: 70 ZIP + 14 RAR checks green.

Frontend password/multi-volume UX and release plumbing (CHANGELOG,
THIRD_PARTY_NOTICES, README, tag) follow in the v1.9 release commit.
2026-09-05 22:40:58 +08:00
Songlx516 c68c0def35 vendor(unrar): add unrar 7.20.1 (RARLab source, opello mirror)
New RAR engine for v1.9, replacing dmc_unrar 1.7.0. dmc_unrar only
dispatches RAR5 compression version 5 (switch(file->version) case
0x5000); WinRAR 6.x/7.x writes version 6 -> DMC_UNRAR_FILE_UNSUPPORTED_VERSION
-> user-facing "corrupt archive" on real v6 archives (diagnosed 2026-09-05
from a v6:8M:m0:m3 test file). unrar 7.20.1 natively supports v6,
encrypted and multi-volume RAR.

Vendored into third_party/unrar7/ (new dir) so the tree stays buildable
while src/rar_extract.c still references the dmc facade. Contents are a
verbatim copy of opello/unrar @97e1780 (159 files, v7.20.1) plus two
project files:
  - unrar_c_api.h  (C facade shim: platform types + dll.hpp re-export)
  - VENDORED.md    (source pin, build model, license notes)

Build model verified on host (MinGW g++ 16.2, Windows):
  - all 49 UnRARDll.vcxproj sources compile with
    -std=c++17 -DRARDLL -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
  - links + runs: RARGetDllVersion() = 9 (smoke binary in .build/, ignored)
  - Windows link additionally needs -lpowrprof

License: UnRAR freeware license (not GPL) - free to use for handling RAR
archives; cannot be used to build a RAR-compatible archiver. THIRD_PARTY_NOTICES
update lands with the engine swap commit.

Next (v1.9 work items): engine swap in rar_extract.c, Makefile .cpp rules,
fixtures + host tests, frontend password/multi-volume, then ELF build.
2026-09-05 21:57:15 +08:00
Songlx516 0d24359d80 vendor: add DrMcCoy/dmc_unrar 1.7.0 (GPL-2.0, single-file UnRAR)
Vendored verbatim from upstream tag 'v1.7.0' (commit d0f3efe, 2024-09-14):
  - dmc_unrar.c (375310 bytes, upstream unmodified)
  - example.c (upstream reference, not used in build)
  - COPYING (GPL-2.0-or-later)
  - README.md (upstream README)

Project-authored additions:
  - dmc_unrar_api.h: facade header exposing only the symbols rar_extract.c
    uses (opaque archive struct + init/close/open/get_*_file/extract/*_to_path).
    Avoids inlining dmc_unrar.c into rar_extract.c, which would pollute the
    dmc_unrar_io_handler struct field names (open/close) with the host-test
    posix_compat.h macros (wfm_open/wfm_close).
  - VENDORED.md: rationale for choosing dmc_unrar over opello/unrar, supported
    formats (RAR4/RAR5, single-volume, unencrypted), explicit limitations
    (no volumes, no encryption, no symlinks, no RAR1.4/1.5), and the full
    v1.9 upgrade recipe (vendor opello/unrar C++, swap RAROpenArchiveEx API,
    rar_extract.c signature unchanged).

GPL-2.0 license obligation: web-file-mgr.elf must ship GPL notice + source
(THIRD_PARTY_NOTICES + this directory; both added in the next commit batch).
2026-09-05 15:09:03 +08:00
Songlx516 5cb0b76e7a Initial import of PS5 Web File Manager v1.7
Homebrew HTTP file manager payload for jailbroken PS5 consoles (x86_64-sie-ps5,
title id FMGR88888). Browse, edit, upload, download and extract ZIPs through
any browser on the LAN; single self-contained ELF, no external services.

Highlights (v1.7):
- ZIP large-file profile (large=1): 2 TiB total / 1 TiB per entry / 1000:1
  ratio. Frontend prompts for confirmation whenever archive > 60 GiB on disk.
- Default ZIP profile stays safe: 512 GiB / 64 GiB / 200:1.
- 69 host-side C tests (tests/run-tests.sh) covering traversal, ZIP64,
  encryption rejection, conflict policies and the large-file profile.
- Three-phase extraction engine (scan -> extract -> publish -> cleanup) with
  atomic staging + rename, located in src/zip_extract.{c,h}.

Project layout:
  src/          C payload sources
  assets/       HTML / CSS / JS / icons / param.json
  tests/        POSIX/host test suite + fixtures
  third_party/  vendored zlib + minizip-ng (zlib license)
  docs/screenshots/  README screenshot images
  LICENSE       GPLv3+

Build:
  make         # cross-compile with PS5_PAYLOAD_SDK
  make linux   # Linux host binary for UI dev
2026-09-04 14:21:15 +08:00