Every time we are calling a function in `FEXCore::Allocator::` this is a
pointer indirection. Which means on x86 it is always a `call [rdi]` and
on AArch64 it is a `ldr x17, [x0]; blr x17;`.
Instead of doing this, use inline functions in the header that call the
correct allocation function directly. This function gets inlined and is
no longer an indirect call.
When compiling with jemalloc, we forward declare the jemalloc function
definitions so we don't have to pull in the entire jemalloc interface in
to the public header definitions.
This is done by consuming a single page at the end of the current sbrk
memory region. Then consuming any remaining bytes that could have
potentially ended up in it.
This ensures that glibc won't be able to return 64-bit pointers to
32-bit thunks once the remaining work is in place.
Optimize `FileLoad` by not using fstream.
For some reason fstream is just really bad.
Switching over to raw pread cuts the amount of time it takes to read
files by a quarter of CPU time.
If we have a fixed file to read, we can read it in three syscalls.
fstream is...weirdly slow in the other implementation.
Theoretically in the future this can be improved to a single syscall if
the `readfile` syscall ends up in upstream Linux.
- Some minor cleanups in the VMARegion struct type.
- Switches over to using the FlexBitSet range scanning, based off this
implementation.
- Move memory region allocation to its own function instead of
constructor
- This will be used by a new constructor later for 48-bit host-side
allocations
- Minor optimization to keep track of Munmap.
- We were burning a bunch of time on backward scanning for free
regions even though we never did a munmap to free anything.
- Now only do backward scanning if a munmap occured.
- Saves a bunch of CPU time
These were currently living in the 64BitAllocator class but can be moved
directly to the FlexBitSet.
Ideally in the future these routines can be optimized so our allocator
is faster but for now these are just moved.
This creates a generic interface that FEXCore can use for timeline
profiling. This allows us to create a generic interface which the
backend details are hidden so we can support multiple timeline profile
APIs.
The only API supported right now is ftrace/gpuvis. Which is extremely
lightweight of an interface with minimal overhead.
We must be careful here since in most cases will will have dozens of
FEX instances running at any given time. So a timeline profiler like
Microprofiler can have major issues since that only ever expects a
single process at a time.
Not enabled by default but just needs the `ENABLE_FEXCORE_PROFILER`
cmake option set to enable.
Due to how the 64-bit allocator previously worked, it was never subjected to memory
regions larger than 64GB to be tracked. With the change in
PR #1885, this has changed to have regions that will hit sizes larger
than 170TB on some platforms.
Better yet, even with smaller regions it still had a performance issue,
it just wasn't as visible.
First problem: We used MemSet instead of MemClear for the live page
clearing. This caused pages to be claimed as "always in use".
This would cause us to always scan the entire region on allocation, find
that it didn't work and allocate a fresh region on every slab
allocation.
jemalloc saving us here since it allocates slabs from the OS fairly
aggressively.
Second problem: We used MemSet (now changed to MemClear) to "clear" the
state tracking for pages.
This causes ~600MB of memory to be used purely for state tracking.
This was physically backed since we were writing to every bit of
tracking for handling 256TB of VA.
This had a fault dance with the kernel for every new page being hit
here.
Instead of clearing the the bits with a memset, clear it with madvise so
it doesn't consume physical pages at all.
This means we use significantly less physical memory for 32-bit
applications.
With this change, pressure-vessel startup time goes from 24 seconds down
to 17 seconds. 70% of the original startup time.
But really the main savings here comes from the memory reduction that
PR #1885 ballooned, but has been an unseen problem before.
Before that PR we were burning 2MB of physical memory per region for no
reason.
After that PR we were burning up to 600MB of physical memory per region
for no reason. Changing a bit depending on how large the region ended up
being.
This now ends up being 2 pages starting out and grows as more pages are
are used. A significant improvement.
While we were getting the application name for the application layer, we
were failing to store the filename for telemetry.
Save the filename we get for application layers and store it for the
telemetry file.
Otherwise these were just alway ending up as wine or wine-preloader.
In the case of an AArch64 builder is using 16kb or 64kb pages like is
common on servers then it would fail to compile, even if the resulting
application would only ever run on 4k page hosts.
Resolve this by removing the build check and hardcoding 4kb pages for
each of our uses. We still require 4kb pages to run, so this mostly just
removes the weirdness where it is 16kb builder + 4k runner. Would have
broken some of our assumptions when running.
Mai