We still need to hook glibc for thunks to work with
`IsHostHeapAllocation`.
So now we link in two jemalloc allocators in different namespaces.
As usual we have multiple heap allocators that we need to be careful about.
1. jemalloc with `je_` namespace.
- This is FEX's regular heap allocator and what gets used for all the
fextl objects.
- This allocator is the one that the FEX mmap/munmap hooks hook in to
- This mmap hooking gives this allocator the full 48-bit VA even in
32-bit space.
2. jemalloc with `glibc_je_` namespace.
- This is the allocator that overrides the glibc allocator routines
- This is the allocator that thunks will use.
- This is what `IsHostHeapAllocation` will check for.
3. Guest glibc allocator
- We don't touch this one. But it is distinct from the host side
allocators.
- The guest side of thunks will use this heap allocator.
4. Host glibc allocator
- #2 replaces this one unless explicitly disabled.
- Always expected to override the allocator, so this configuration
isn't expected.
Already tested this with Dota Underlords to ensure this works with
thunks.
The LOAD_LIB and EXPORTS macros behave slightly differently in this regard:
* Use LOAD_LIB(libwayland-client) in Guest.cpp (library name with dash)
* Use EXPORTS(libwayland_client) in Host.cpp (library name with underscore)
This has been a long time coming. The C interface has been a thorn in
our side for no reason for a long time.
The purpose of this step is to remove the C interface without changing
behaviour as much as possible. This means that with this commit there
are still some bad practices but the remaining issues will be solved
with followup PRs.
Primarily, we still have a `DestroyContext(CTX)` static function which calls
the Context implementation's `DestroyContext` and does a raw C++ delete.
Follow up PR will remove that, but I didn't want to touch it yet since
it'll require checking to ensure the unique_ptr changes play nice with
our allocator hooking. Which this is already a huge PR without trying to
change behaviour.
* BUGFIX:Adding the nullptr check to FinalizeHostTrampolineForGuestFunction. It will lead crash on interpreter code set function nullptr callback. eg. XSetErrorHandler(NULL)
* fix#2189
This is a bit of a large commit since it is an all or nothing sort of
change.
Instead of doing LoadContext and StoreContext for GPRs and FPRs, any
that are statically allocated (GPR and XMM) should use
{Load,Store}Register directly.
This is now enforced that {Load,Store}Context{,Indexed} will assert if
trying to access these ranges.
This is the first step towards accelerating our JIT compile times in
that it removes the need for the SRA pass to convert all the IR over.
This ensures that from the Dispatcher directly we are handling SRA
accesses as "registers", ensuring that future changes don't need to run
in to this problem.
Even though it wasn't a target of this change, in a simple test
application this removes ~12% of the total JIT compile time.
x86-64 has five symbols within VDSO that we need to emulate.
Pass this through either glibc or host vdso if the symbol exists.
AArch64 doesn't have the time or getcpu vdso interface, so fall down
glibc instead.
This changes how host trampolines for guest functions are created. Instead
of doing this purely on the guest-side, it's either the host-side that
creates them in a single step *or* a cooperative two-step initialization
process must be used. In the latter, trampolines are allocated and partially
initialized on the guest and must be finalized on the host before use.
This function must be able to handle both guest heap pointers *and* host heap
pointers, so it only forwards to the native host library for the latter.
This is because Xlibint users allocate memory using internal macros aliasing
to libc's malloc but then they free using the function XFree. For libX11,
this is not a problem since the allocation happens in a thunked API function
(and hence on the host heap), but if a function from an unthunked library
accesses Xlibint, it will allocate on the guest heap.
One notable example where this was encountered is XF86VidModeGetAllModeLines.
Dota Underlords hit this when querying Vulkan two different symbols
that resolve to the same function. Ignoring the error in that specific
case is safe, since the linked guest functions have the same implementation.
std::string_view was sticking around for longer than libraries being
loaded.
This was causing a crash.
Change this to a std::string directly until we support cleanly removing
this data on library unload.
Interface definitions must enable this functionality by enclosing functions
that may be called through function pointers in a namespace annotated with
`fexgen::indirect_guest_calls`. The guest thunk must further link any host
function pointers to a guest-side instance of CallHostThunkFromRuntimePointer.
Previously, the host thunk would seemingly succeed to load even when
the corresponding host library could not be loaded, leading to obscure
crashes later throughout execution.
Migrates lingering instances of the old logger over to fmt where
applicable. This allows removing some of the old defines and functions.
The only remaining usages of the printf-based variant of the logger is
in Tests/LinuxSyscalls/Syscalls.cpp for the strace handling.
Keeping this in the commit history to go back to later.
While these are required for static-pie builds to work, with the glibc
bug we can't use those yet.
Disable for now since it is is unnecessary.
If the thunk configuration is enabled then preemptively load the
libraries. Since they will need to be loaded for any thunking library.
This is because we need to ALWAYS share the allocators to the thunks.
This isn't quite a 100% clean sweep of IWYU.
There are some false positives where clang fails.
Additionally there are still a few missed in the frontend side of things
that I didn't get to
This was using the implicit thread TLS object. All users of this
have access to the thread object directly.
Use that instead. Fixes a subtle bug were the frontend could be trying
to do a callback and TLS sections weren't correctly set.
Configuration mapping was duplicated between three different tables.
Additionally default configuration values were strewn about. Making it confusing as to what the default value would end up being
Adds a new ConfigValues.inl header that defines a few things right next to each other.
Defines the enum name as usual.
Defines the JSON config option name.
Defines the Environment config option name
Defines the default value that the configuration should be
Sadly these things can't be split without breaking functionality so it
turns in to a bit of a mess.
SyscallHandler is very much something that is a Linux only construct and
shouldn't be in FEXCore itself. Lets the frontend register a
Syscallhandler with FEXCore. FEXCore itself is then aware of the current
syscall ABI and handles the ABI in an optimal fashion.
So it is not a 100% clean break otherwise we would lose performance.
The SignalDelegator then needs to move to the frontend since the
SyscallHandler requires it for signal based syscalls.
The CPU backend signal handling still needs to happen in FEXCore because
it is a very tight coupling with the CPU backend.
Once we need to support more Signal handling we can give the backends
cleaner support to select which specific OS handler to handle.
If we want to execve files directly we need to ensure that the
binfmt_misc interpreter path is installed.
If it isn't installed then we need to fall down the regular path of
pushing FEX to the front of the argument list.
fstatfs64 and statfs64 was wrong, the syscall doesn't quite match the
glibc definition
Fixes recvmsg. Specifically the aux data was failing, thus SCM_RIGHTS
wasn't getting passed correctly.
Fixes a couple of time syscalls where their arguments are optional.
This implements a bunch of new 32bit syscalls which almost gets FEX to
be able to run Steam.
The main missing syscalls are mremap, shmat, and ioctl for supporting
Steam execution.
Ryan Houdek