Found this through introspection, only a real problem with -O0.
We were moving a fextl::vector in to a shared pointer so it can be long
living, this causes a change in ownership which was causing the
`BlobData` pointer to be stale. This was then passed to the work queue
to be consumed. In -O1 and above a std::move of a fextl::vector is a
proper move, but -O0 it turns in to a copy.
Easy fix is to just create the shared_ptr early and reference that, so
the pointer is long lived and no references become stale.
This peephole removes unnecessary masking operations when the mask is
provably all 1s. This improves performance in GB7 'Asset Compression' by
about 2% with SVE and about 4% without SVE.
This doesn't fully fix#5912 but gets a step closer. Instead of just
hashing the filename, hash in the build-id as well when it exists.
This isn't all encompassing because the build-id may not exist in all
cases. Crypt Of the Necrodancer for example doesn't ship with the
build-id on their 64-bit build. Although their legacy 32-bit build had
it.
The build-id is always a hash, depending on tool it is either 64-bit or
160-bit in all the executables I found. Although it can be anything so
make sure to be flexible enough to support everything.
StealMemoryRegion guards the open() with LogMan::Throw::AFmt, which
compiles out in release builds. With no procfs mounted the fd is -1 and
CollectMemoryGaps then loops on read(-1) forever at 100% CPU.
This is hit whenever FEX is the x86_64 binfmt handler and something runs
an x86 binary in a root without /proc. BuildKit does exactly that: its
emulator probe execs a static x86_64 binary in an empty chroot, so every
BuildKit start on an Arm host with FEX registered hangs.
Reserve nothing instead: StealMemoryRegion returns no regions and
Setup48BitAllocatorIfExists skips the 48-bit allocator. The guest then
runs without the upper VA hidden, which only matters in environments
that previously never got past startup.
A bit messy but it basically duplicates the Linux side. We don't have a
way to include NT internal defines in to FEXCore currently so the two
required symbols are just forward declared.
Adds a config option to disable pruning if desired.
Had to change FEXCore::HostFeatures slightly to remove the HostType from
the Hash for caching, as we don't want the bucket to inadvertently
delete wow64,arm64ec,linux buckets when executing in the same namespace.
Minor changes to the DiskCache::Init code to handle returning two
values, moving the HostType to be a part of the "ProcessBucketHash"
instead of the "MachineBucketHash".
Then uses the directory walking function added in the previous commit to
find mismatched/old "MachineBucketHash" directories and remove them.
The base directory is now the machine bucket, which is everything that affects
codegen that you can ask FEX about offline: FEX compiler version, host features.
The DB files now carry a process bucket, which is everything else that can
affect codegen, on the user/process side: 32-bitness and config.
Both together are still the bucket hash that gets laced into every lookup key,
so we still won't load mismatching code because a file is in the wrong spot.
In order to reuse code buffers, when invalidating we will no longer be
able to invalidate code while inside the code we are invalidating.
Changes the IR operation to be a block ender itself and control jumping
to the new RIP. This way it can invalidate from the dispatcher and just
restart JIT execution.
Have it treated as a block terminator IR and instead of doing
syscall+exitfunction, just do syscall and jump in to the dispatcher.
This is necessary for code invalidation and reuse for code that is stuck
in long running syscalls. We'll need to do something similar for thunks
later, although the recursive nature of thunk+callback can make that a
little squirrely.
For some reason `perf top` prefers VMA names over perf symbols. So
disable them when any JIT symbol option is enabled. Breaks WTF tracking
for those buffers but because it's debug, we kind of have to.
Ryan Houdek