queue_buffer leaked the MUST_FREE payload (every queue_json_owned/
build_*_json/strdup(resp)) when MHD_create_response_from_buffer hit OOM.
The MUST_FREE contract hands ownership to MHD only on success — on the
NULL return path the caller still owns the buffer, so free it before
bailing. Critical under memory pressure where the first OOM turns into
a cascade.
patchdl_websrv_stop walked the verxml thread + workers but never freed
the remaining dl_job_t entries or g_debug_json. Today main never calls
the function, but the early-failure path inside patchdl_websrv_start
does, and any later graceful-shutdown work would hit the same leak.
Drain g_pool.jobs through free_job_locked under the pool lock; free
g_debug_json under g_mutex.
RAM caps:
- MHD: CONNECTION_LIMIT 64 -> 8 (single-user UI), and
THREAD_STACK_SIZE = 512 KB. THREAD_PER_CONNECTION on libc's default
pthread stack (multi-MB) was reserving hundreds of MB of VM per
burst; the largest stack frame in any handler is the 8 KB sidecar
buffer, so 512 KB is generous even with curl + openssl in the path.
- patchdl_buf_t caps: manifest 64 -> 16 MiB, version.xml 16 -> 4 MiB.
Real PS5 manifests are 1-2 MiB; the old caps allowed 64 MiB per
fetch with multiple fetches possible in flight.
- patchdl_install_status_json: ai_install_status_t (2 KB pad) moves
from the MHD worker stack to a calloc/free pair so a polling browser
doesn't keep committing pages on each /api/installstatus tick.
- seed_from_sidecar: 16 KB stack buf -> 8 KB. 4096-piece cap fits with
the JSON wrapper inside 8 KB.
No functional changes; download/install/scan/tile paths unaffected.
Real regression from c9d721f: pinning CURLOPT_PROTOCOLS_STR /
REDIR_PROTOCOLS_STR to "https" on the piece downloader (and the simple
file streamer) broke real-world Sony patch downloads. Banishers reliably
aborted after ~73 MB and Last of Us Part I after ~127 MB — the Sony CDN
appears to 302 the piece URL to an http:// signed edge inside its own
infrastructure, and refusing those redirects killed the transfer mid-piece.
The smaller patchdl_http_get path (version.xml + manifest JSON) keeps the
HTTPS pin since those payloads always come back from the public https
endpoints. The defence still holds elsewhere: host_allowed gates every
URL to the Sony CDN allowlist, TLS verifies against the pinned SCEI root
even on a 302, and NOSIGNAL stays so a connection RST can't smuggle a
SIGPIPE back into the worker thread.
For future regressions of this class /api/downloads now exposes the last
CURLcode + HTTP status per job (last_curl_rc, last_http_code) so we don't
have to instrument the binary again to find out what curl returned.
Write param.json + icon0.png into /user/app/<TITLE_ID>/sce_sys/, then
call sceAppInstUtilAppInstallTitleDir to register the directory as an
app. No package, no code signing — the installer reads the metadata
files directly.
The tile uses TITLE_ID PTDL00001 and deeplinkUri
http://127.0.0.1:12880/, so tapping it opens PatchDL's own UI in the
on-console browser. Only useful while the ELF is running.
Asset pipeline: param.json + icon0.png live under assets/ and get
.incbin'd straight into .rodata. Makefile lists them as TILE_ASSETS so
a touch on either forces a relink.
Stat-guard: file_matches() diffs each asset against the on-disk copy
first; when nothing changed the install API isn't called at all. Keeps
repeated payload starts from re-registering the app. Repeated
/api/install_tile calls return "already installed and up to date".
Backend lazy-load: AppInstUtil isn't mapped until something pokes it,
so the helper polls patchdl_install_backend_check() for up to ~15 s
before resolving sceAppInstUtilAppInstallTitleDir.
Wiring: /api/install_tile POST triggers the install on demand;
patchdl_websrv_start() runs it at startup when home_shortcut is on;
and flipping the toggle from off to on in /api/config also fires it.
All three paths share the stat-guarded helper so they're safe to
repeat.
The sidebar tag was a literal "v0.0.3" string in index.html, so a
release bump left the running UI lying about which build it was. Server
now publishes PATCHDL_VERSION via /api/status; renderStatus() drops it
into a #brandVersion span. No more chasing a hardcoded version on every
release.
Update all skips shadowmount titles. They pass the download policy but
not the install policy (their app slot has no real source medium), so
sweeping them in would burn tens of GB on a download AppInstUtil would
refuse. Single shadowmount downloads via the per-title button still work
for the "pop the disc in later" workflow.
The three other curl_easy code paths (downloader, piece pool, net_diag)
now set CURLOPT_NOSIGNAL=1L plus PROTOCOLS_STR/REDIR_PROTOCOLS_STR =
"https" — matching what patchdl_http_get already does. SIGPIPE on a
broken connection in a worker thread previously could crash the process;
the protocol pin keeps a redirect from sliding off https.
patchdl_proc_kill_others is now bounded to 8 iterations. If kill returns
success but the process never exits (zombie / unusual proc-table state),
sleep(1) × N would otherwise stall startup indefinitely.
lookup_tsv rejects URLs that don't start with https://. The TSV file
lives under /data/patchdl and is writable by anyone with /data access;
patchdl_http_get's host_allowed gate still applies, but failing earlier
keeps a poisoned line from even reaching the network layer.
json_get_str now decodes the common JSON escapes (\" \\ \/ \n \r \t \b
\f) and collapses \uXXXX to '?'. Previously \" terminated the value
early and \\ was copied literal, so a body containing escapes turned
into garbage at the install backend.
default_policy is constrained to "allow" or "deny" before being stored,
so a malformed POST can't write an arbitrary string into config.json
and round-trip it back out of /api/config as broken JSON.
/api/manifest/<tid>, /api/pkgverify/<tid>, /api/pkgmeta/<tid> now run
path_segment_safe(tid) explicitly. The lookup gate they relied on
(get_title_action_info) is defense-by-coincidence — a future refactor
that populates g_titles via another path would lose the check.
patchdl_scan and patchdl_scan_debug_json perform a process-wide vnode
swap that is only safe single-threaded. patchdl_scan_lock() is now
called once right after MHD_start_daemon; subsequent calls return -1 /
NULL instead of racing the worker threads.
EVP_DigestFinal_ex return code is now checked. A failed final left dig
uninitialized; hex_encode would have produced empty hex and a silent
-2 with no diagnostic. patchdl_sha256_fd_region switches its inner
sprintf to snprintf — same effect, no -Wformat-security warning.
g_stage and g_err are now _Atomic. The backend init thread publishes
stage transitions and function-pointer assignments; HTTP request
handlers read g_stage to decide whether to call the Sony API. With the
old `volatile int` reads, nothing in the C memory model ordered the
function-pointer loads against the stage check — a stage==5 sighting
could (in theory) come before the pointer stores were visible. Default
seq_cst on _Atomic gives us the acquire/release pairing for free.
/api/pkgdiag returned g_pkg_diag_json directly with RESPMEM_PERSISTENT,
so MHD's writer thread could read the buffer while record_pkg_diag was
mid-snprintf — torn JSON or a missing NUL terminator. Snapshot under
g_mutex into a heap copy and queue with RESPMEM_MUST_FREE instead.
patchdl_net.c gets fopen_safe(): open() with O_NOFOLLOW|O_CLOEXEC and
mode 0600, then fdopen. The old fopen("wb") follows symlinks (a
malicious symlink at dest_path could redirect the write) and creates
mode 0666 (libc default). Both download paths now use it.
patchdl_appdb opens SQLite with SQLITE_OPEN_FULLMUTEX. Today the scan
runs on the startup thread only, but a future rescan triggered from the
HTTP thread would otherwise race the handle.
POST handler: cap accumulated body at PATCHDL_POST_MAX_BYTES (64 KiB).
A LAN client streaming gigabytes into /api/config would otherwise grow
the per-connection buffer until OOM-kill. Past the cap, further chunks
are dropped and the final call returns 413.
title_pkg_path: the basename comes from the patch_url and ultimately
from version.xml via the Sony CDN. A poisoned manifest with a basename
like ".." or one containing delimiters would compose a dest path that
escapes /data/patchdl/<tid>/. Validate the basename through
path_segment_safe and fall back to "<title_id>.pkg" on rejection.
cleanup_installed_download now routes through the same helper instead
of doing its own basename extraction.
ai_install_by_package's playgo struct is 0x2700 bytes — comfortably
fine on its own, but on the MHD worker stack alongside meta/pkg/uris
buffers and Sony's own frame use it leaves little headroom. Move it to
the heap in both patchdl_install_local_pkg and patchdl_install_by_uri.
Manifest JSON parser used substring scans with no per-piece scope; a key
defined in a later piece could be misattributed to the current one, and
the escape handling silently dropped the byte after a backslash even for
unknown escapes. json_string_after/json_u64_after now take an optional
limit pointer (NULL = legacy unbounded), and the manifest loops pass
obj_end so per-piece reads can't leak across pieces. JSON escapes are
decoded properly: \" \\ \/ \n \r \t \b \f; unknown \X drops the
backslash and keeps the payload byte.
Sanity caps on assembled manifests: PATCHDL_MAX_PIECES (4096),
PATCHDL_MAX_PIECE_BYTES (8 GiB), PATCHDL_MAX_TOTAL_BYTES (200 GiB).
A malformed manifest with a single multi-TB piece or millions of entries
is now rejected before any disk activity.
patchdl_buf_t gains an optional `max` field; write_cb fails the transfer
when growth would exceed it. patchdl_http_get preserves the caller-set
max across its internal memset(). verxml_query caps at 16 MiB,
fetch_manifest and download_manifest at 64 MiB.
host_allowed switches to strcasecmp (DNS is case-insensitive; an upstream
redirect could otherwise drop out of the list). CURLOPT_PROTOCOLS_STR /
REDIR_PROTOCOLS_STR pin all traffic and redirects to HTTPS.
CURLOPT_NOSIGNAL=1 prevents libcurl from raising SIGPIPE in a worker.
DNS label parser bounds-checks the length byte before incrementing pos,
so a malformed response with a 0xFF label near the end can no longer
read past the receive buffer.
extract_title_id used `p[8]` as the loop guard, which crossed the NUL
terminator on strings shorter than 9 chars (UB). Replaced with a
strlen-based bound.
Sony's GetTitleIdFromPkg, GetContentIdFromPkg and GetInstallStatus take
output buffers with no length hint. We sized them to the visible id
length (0x30 / 0x40), but the firmware may NUL-pad more — that class of
bug already crashed the process once (commit 970c7d8). Switch all three
calls to padded AI_*_OUT_SIZE temporaries and copy_bounded() the safe
portion back into the right-sized destination.
patchdl_install_local_pkg extracts title_id and file_base from the
caller's local_path and splices them into http://127.0.0.1:.../api/pkg/
and the LAN equivalent that get fed to InstallByPackage. A path with
CRLF or '/' embedded in the basename would inject into Sony's HTTP
request line. install_id_safe() now gates both before they reach the
URI builders; on failure the loop / LAN URI is simply omitted (the
direct sdk_path and file:// URIs still run).
title_id_eq9() replaces strncmp(...,9): PS4/PS5 ids are exactly 9
chars (4 letters + 5 digits), and a prefix match would let PPSA12345
collide with PPSA12345EVIL when a future caller passes a longer string.
/api/install_aip path must be PATCHDL_DL_DIR/<safe-title-id>/<safe-filename>,
rejecting attempts to point AppInstUtil at arbitrary on-disk PKGs (e.g.
/system/..., /user/uploads/...). local_install_path_safe enforces the
shape and reuses path_segment_safe for both segments.
/api/install_uri requires https:// to a Sony CDN host (subdomain match
against sgst/gst/gs2.*.playstation.net). file://, http://, and arbitrary
hosts are refused. The CDN list duplicates patchdl_net.c's ALLOWED_HOSTS
deliberately — both layers gate independently, both must stay in sync.
content_id / title_id from both endpoints now go through path_segment_safe
before reaching the install backend, so they cannot smuggle delimiters or
control chars into Sony's HTTP fetch.
MHD gets CONNECTION_LIMIT=64 (was unbounded with THREAD_PER_CONNECTION),
PER_IP_CONNECTION_LIMIT=8, CONNECTION_TIMEOUT=30s. A noisy LAN client
can no longer exhaust pthreads on the PS5.
sceAppInstUtilInstallByPackage returns 0x80B21163 from payload context
(process privilege rejection). sceAppInstUtilAppInstallPkg accepts the
same PKG with rc=0 and does install it. Switch all install paths to use
AppInstallPkg.
do_install (cross-region): require assembled PKG from the manifest
download; InstallByPackage and DP.pkg fallbacks are removed since both
are dead ends in this process context. do_install (same-region): also
switched to AppInstallPkg. do_download: removed the DP.pkg shortcut so
the manifest-assembled full PKG is downloaded as before.
sceAppInstUtilGetInstallStatus ABI: first arg is an output buffer for
the current install's content_id, not an input query. Passing our
tracking buffer there was overwriting it with zeros (disc game has no
explicit content_id). Fix: use a fresh output buffer; keep `cid` from
g_last_content_id untouched. Also add 2048-byte padding to
ai_install_status_t against firmware struct size variance.
New APIs: patchdl_install_by_uri, patchdl_install_app_pkg,
patchdl_install_debug_state. New endpoints: /api/install_uri,
/api/install_aip, /api/debug_install. delta_url propagated through
verxml → scan → websrv for future DP.pkg tracking.
Verified on device (FW 11.60, BD-JB+PPPwn): Dead Island 2 PPSA03099
updated from 01.000.001 to 01.000.011.
Sony path-allowlists the URI given to sceAppInstUtilInstallByPackage —
/user/data/ and /mnt/usb are accepted, a bare /data/... path is rejected with
0x80B2116F (confirmed by the ps5upload project). PatchDL stored the pkg under
/data/patchdl and passed that /data path, so every install was rejected at the
path stage. Pass the /user/data view of the same file instead (the code already
computed it as sdk_path; it was only used for AppInstallPkg before).
Also report each URI's individual rc instead of only the last attempt's, which
revealed the real wall: via file:// the installer reaches header parsing and
rejects with 0x80B21106 — the assembled file is a valid but DEBUG-magic PKG
(\x7FFIH, not retail \x7FCNT), the format Sony's system updater consumes rather
than the retail-pkg format InstallByPackage expects.
Three read-only endpoints (no install, no writes) to inspect a downloaded
package on-device:
- GET /api/manifest/<title_id> — re-fetch the patch manifest (PatchDL bypasses
the DNS block) and dump each piece's offset/size/SHA-256.
- GET /api/pkgverify/<title_id> — SHA-256 every piece of the assembled .pkg
against the manifest hashes, on-device (SSD, no multi-GB transfer), and report
per-piece pass/fail. Proves whether the file is byte-correct.
- GET /api/pkgmeta/<title_id> — read the pkg's embedded content id + title id
(GetContentIdFromPkg) vs the target ids, to expose cross-region linkage.
Supporting code: patchdl_sha256_fd_region() (pread + OpenSSL EVP) in the net
layer, and bind sceAppInstUtilGetContentIdFromPkg in the install backend.
Used to diagnose the Dead Island 2 install: the 61.6 GB package verifies
byte-perfect (17/17 pieces) and its content id matches the target, so the
0x80B2116F install rejection is a Sony install-method limitation, not the data.
Resume: write the sidecar after every completed piece instead of batching
every 8. Each piece's bytes are already fdatasync'd and the sidecar write is
a tiny atomic tmp+rename, so an unclean kill now re-downloads only the pieces
still in flight, not a batch of up-to-8 already-finished ones. Drops the now
-unused 'unpersisted' counter.
UI: the "parallel download connections" control is now a stepper — two large
54px -/+ buttons around a tabular value, in a row beside its label, instead of
a full-width number field for a 1-16 value. Big targets and a clear green focus
ring suit controller navigation (the UI is driven by the PS5 pad via the home
tile). Tapping -/+ updates and auto-saves that field alone (debounced), applying
live on the server.
The pool now spawns the full worker set at startup and gates each worker by
its slot against a live active_conns limit, instead of spawning exactly
max_connections threads once. Saving a new value in Settings updates the
limit and broadcasts: idle workers wake to pull pieces, and a lowered limit
parks the extra workers after they finish their current piece. No restart,
and no thread creation/teardown at runtime.
Verified on device: max_connections changed 4 -> 8 -> 16 -> 4 through the API
while a download stayed active throughout. (Throughput did not scale with
connections on this CDN, which caps aggregate bandwidth per source IP; 4 is a
sensible default.)
Replace the single sequential transfer with a pool of N worker threads
that pull pieces of one manifest in parallel, lifting the per-connection
~7 MB/s ceiling. One job runs at a time; the rest queue. The connection
count is configurable (1-16, default 4) and applies on the next start.
Resume is tracked per piece in a sidecar bitmap that survives a reboot,
and a one-time migration recognises a partial written by the old
sequential build (a piece-aligned contiguous prefix on disk) and marks
those pieces done so an in-progress download is not restarted from zero.
Pause keeps the partial; Cancel deletes it. Both, plus Resume, are
available at any point in a download's life.
Concurrency review fixes folded in:
- a job is published as the active (claimable) job only after its
manifest/state/fd are attached, so a half-built job can no longer be
settled to "done" before any bytes are fetched
- cancel/pause during the admit I/O window only flag the job; admit_next
is the sole finalizer, closing a use-after-free and a lost-pause race
- resuming a paused job frees the stale per-job buffers and zeroes the
committed counters before re-seeding, fixing a leak and a double-count
- the background version.xml thread is joined on shutdown before the
title list is freed
- verify_downloads is snapshotted under its own lock before the pool lock
- the web UI keeps Resume/Cancel after a failed transfer and bounds the
local "downloading" bridge flag so a card cannot wedge
Split the single morphing button into a green/amber play-pause (Update →
Pause → Resume) and a red stop (Cancel). Pause aborts the download but
keeps the partial (resumable); Cancel aborts and deletes. Backend gets a
separate pause flag distinct from cancel.
Resume now continues WITHIN a part: the partially-written piece is fetched
from its last byte via an HTTP byte range (with a safe fall back to
re-fetching the whole piece if the CDN ignores the range), instead of
re-downloading the whole part. A title is resumable as soon as any bytes
are on disk.
Version bumped to 0.0.3 (no release tagged).
An interrupted download (cancel excepted) now keeps its partial package on
disk instead of deleting it, and records the manifest it belongs to in a
sidecar (state.json). On the next start the title is flagged resumable and
the UI shows a "Paused — X downloaded" note with a Resume button.
Resume refetches the manifest, skips every piece already fully on disk, and
re-fetches only the one partially-written piece (piece-granular, no HTTP
range needed), appending the rest. The fresh-download path is unchanged. A
partial belonging to a different/older manifest is dropped and the download
starts clean; a corrupt (failed SHA-256) download is not kept.
Survives a reboot: a killed payload runs no cleanup, so the partial and its
sidecar persist under /data/patchdl until resumed, completed, or deleted.
Replace the single-page layout with a view-based one (Games / Settings /
Logs via the left nav) and fold all download UI into the game tile — the
separate download queue is gone.
Per tile while downloading: an in-tile progress bar with auto-scaled size
(B/KB/MB/GB/TB), live transfer speed and ETA from the poll deltas, and a
green "Downloading" marker. One fixed-width action button that no longer
reflows with its label: a blue Update/Download/Install that morphs into an
amber Cancel while the download runs, plus a ghost Delete for a finished
package.
Settings and Logs moved to their own pages. Mobile-first responsive layout
(rail collapses to a top bar, icon-only nav, single-column tiles, >=44px
touch targets, 16px inputs). Real free space (statvfs) is shown auto-scaled
in the rail and status strip.
New "Home-screen shortcut" toggle (default on, persisted as home_shortcut
in config.json). The actual PS5 tile install is not wired yet: it needs a
prebuilt deeplinkUri stub PKG installed via sceAppInstUtil.
Fixes from an adversarial review pass:
- Reconcile in-flight downloads from /api/downloads onto the cards, so
progress + Cancel appear after a reload or a download started elsewhere,
and stop the per-poll full-grid rebuild.
- Clear the downloaded flag on install and once the server reports the
title up to date, so a patched title no longer shows Install/Delete
forever.
- Zero a stale speed/ETA if the byte counter goes backwards.
- a11y: nav buttons keep an accessible name when the label is hidden on
small screens; visible focus ring on the search box; filter group is
role=group with aria-pressed; drop the noisy grid-level aria-live; fold
the transient "checking" state into a visible filter bucket.
Defense in depth around the only operations that touch the filesystem,
so no request can escape /data/patchdl or leave the process able to
write to a system path:
- Validate the HTTP title_id with path_segment_safe at the top of the
title-action route, and again inside remove_title_dir and
cleanup_installed_download. The delete primitives are now self-
protecting instead of relying only on the "title exists in the scan"
guard, so a future refactor cannot reintroduce a /data-wiping
traversal (a title_id of ".." would otherwise resolve the dir to
/data).
- Only swap the process root vnode when the current root was captured
and can be restored, in both the scan and the debug dump. Otherwise
the process could be left rooted at the system root, sending later
absolute-path writes to the wrong place.
Reviewed and confirmed safe with no change needed: the installer only
delegates to Sony's signed AppInstUtil service (it never writes or
redirects to system paths itself), app.db is opened read-only and
immutable, every write targets /data/patchdl, the patch picker never
selects an update that needs a newer firmware, and the /api/pkg file
server already blocks path traversal.
Each Sony manifest piece carries a SHA-256 (hashValue). When the new
"Verify downloaded pieces" setting is on, every piece is hashed while it
streams to disk (OpenSSL EVP, already linked) and compared against the
manifest value; a mismatch aborts the download, deletes the partial, and
reports piece_verify_failed instead of handing a corrupt 60 GB package to
the installer.
Off by default: TLS already protects the bytes in transit and the PS5
installer verifies the whole packageDigest before applying, so this is a
fail-fast belt-and-suspenders check. It is also unverified on hardware
yet, so it stays opt-in (persisted in config.json) until confirmed
on-device; the hex compare is case-insensitive since Sony mixes cases.
Cancel a running download (the worker aborts mid-piece and the partial
file is removed) or delete a finished package, from the queue or the
title card. The progress callback now returns an abort signal that
reaches libcurl and the manifest merge loop.
Manifest merge: bound the piece scan to the "pieces" array so a later
"url" key (e.g. playgoChunkCrcUrl) can't be appended as a bogus piece,
and require each piece's fileOffset to match the bytes written so far so
an out-of-order manifest fails instead of silently producing a corrupt
package.
Report real free space on the download partition via statvfs; it was a
hardcoded 0.
Fixes found in review:
- scan: bound the SFO entry table to the bytes actually read and require
the key to be NUL-terminated before strcmp (OOB read on a crafted
param.sfo from a shadow-mounted dir).
- proc: bound the kinfo_proc walk and the name compare to the record and
the buffer.
- install: publish the API probe under the lock (data race with the MHD
worker thread) and initialize rc2.
- verxml: reject a truncated attribute value instead of returning it as
valid.
- web: keep download/install/downloaded flags across a refresh, stop the
queue poll only after repeated empty results, coerce the progress
number, and treat a cancelled download (HTTP 200, ok:false) as
not-downloaded.
Standalone PS5 payload with an embedded web UI on :12880, no etaHEN.
- Scans installed titles and classifies the source (genuine install,
ShadowMountPlus mount, preinstall, unknown) from the on-disk layout.
- Reads name, installed version and the Sony version.xml URL from the PS5
app database (vendored SQLite).
- Resolves version.xml past nanoDNS via a raw DNS query, TLS pinned to the
SCEI DNAS root.
- Filters patches to the newest one compatible with the current firmware.
- Downloads the patch and installs it through AppInstUtil (sysmodule loaded
at runtime), gated to genuine installs with a package title-id match guard.
- Action-based UI filters and an on-screen startup notification with the URL.
Knutwurst