With the previous stack leak fix, RUINER reduced its memory leaking down
to around 50MB/s. The remaining memory leaks come from the pthread stack
that we are required to allocate (128KB per thread) and some internal
DTV tracking structures.
The problems come in the fact that glibc/pthread only tears down its
internal state for these if the pthread function actually returns! We
can **technically** switch the initial stack over to a "user" stack but
that introduces more problems around internal dtv tracking that we
already fixed months ago, so we can't actually do that in practice.
This leaves us no choice, we effectively are mandated to return from the
pthread function in order to free the memory from glibc. The only way we
can safely do this is with a long jump and deferring some data structure
management until that case.
This is all incredibly sucky but it's necessary to work. With these
changes, RUINER is no longer leaking memory (Hovering at around 3GB used
while in-game) and even Steam is consuming less memory.
It doesn't solve the problem that thread creation and teardown could
likely be faster, but not many applications are creating 720
threads/second.
Fixes an issue where a thread that exits with the `exit` syscall never
actually frees its pivot stack. This is common practice and it was
missed when I was fixing the previous stack pivot leak.
This was uncovered when looking at the game
[RUINER](https://store.steampowered.com/agecheck/app/464060/?curator_clanid=4777282)
for timing bugs. Turns out the Linux build of the game creates and
destroys a VLC object every tick of the engine. Creating this VLC
object creates six threads behind the scenes. At what I assume the
default tick of the engine is of 120Hz(?) this would mean it is
attempting to create and destroy 720 threads per second, quickly
leading to memory exhaustion under FEX.
While this hits the biggest memory leak we have around thread creation,
this game is still hitting thread creation so hard that I can see other
leaks that I need to track down still.
In regular x86 programs, when a signal occurs, the signal will not be handled within the signal handler. However, under FEX's defer signal mechanism, the signal is not immediately masked when it is deferred. When returning to the location that receives the signal and continues processing, the signal might be received again, causing inconsistency between the emulation and the actual program.
Here is an unit test for this patch from ltp:
https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/syscalls/timer_settime/timer_settime03.c
Instead of hardcoding the bogomips calculation, more closely match what
the Linux kernel does for bogomips. There are some applications out
there that use bogomips for silly timing calculations, so this is a
better version.
`llseek` returns only ever 0 or errno in the return register. This is in
contrast to `lseek` which returns the result (or errno) in the return
register.
We were accidentally returning the result on non-error conditions which
could freak out some software. Thanks to
[OFFTKP](https://github.com/OFFTKP) for pointing out this issue
This changes the instcountCI code to consistently load test data in to
RIP 0x1'0000 so we don't have any spurious changes due to json changes.
This has been a minor annoyance where if a test was added, it had the
potential to shift the rest of the data in the tests. This now ensures
it is consistent.
Avoids the non-fatal error: "[ERROR] Close closing FEX FD 2"
that happens when the guest program executes a syscall to close fd 2,
and it's not in fex tracked set.
These are completely unused since this ELFContainer is significantly
less utilized than original expectations.
More of this code is dead and can be removed in the future.
Telemetry value address generation was forcing an indirection at all
times which was unnecessary. These values live in the BSS, zero
initialized at process start and is unnecessary.
Instead change the wrapper defines to directly operate on the enum
passed in which saves an indirection on all of these telemetry
operations (except for the ones in the JIT which are required to be PIC
compliant).
This also fixes an annoying warning about
`FEXCORE_TELEMETRY_STATIC_INIT` causing initialization and destruction
order being unspecified, so two wins.
Since this syscall doesn't exist, we need to convert it to the
equivalent utimensat like the kernel does internally.
This is fairly trivial but there are some safety nets in place.
Instead of keeping the vlaue as a string array in the MetaLayer, convert
the value to its final type once.
Improves performance in some hotpaths that were doing config based
string conversion in a relatively high frequency.
Previously, new VMA entries were always prepended to the list of the
associated MappedResource. This usually made FirstVMA erroneously point to
the *highest* VMA instead of the lowest.
The Linux kernel clears these flags on signal, DF is particularly
dangerous because it would break ABI if a signal happened to occur in
the middle of a memory operation that changed the direction of copy.
Because of how frequently wine uses signals, this is actually fairly
likely to occur inside of a memcpy/memset function.
Shout out to BlinkDagger on Discord who found that we forgot to do this.
With the previous fixes in place, we can now stop burning a fextl::list
in every single config option. This list is only required for strarray
options so reserve it for those entirely.
We also don't need to save the config option enum for each, so these
actually go from ~32 bytes per object down to their base type for most
everything.
I kept finding I needed `./fex_shm_stats_read `FEXpidof Celeste.exe``
but FEXpidof wasn't ever wired up to find FEX in the face of emulating
wine and arm64 wine.
This adds two new features basically:
- If x86 wine is being emulated, then walk the argument list just like
our config options to see what the program executable name is.
- If it is arm64 wine using FEX, then we need to detect that, and walk
the arguments in a similar fashion
The detection is the main thing here in that the only way to detect FEX
for arm64 wine is checking the applications mapped files and seeing if
it is mapping arm64ecfex.dll or wow64fex.dll.
x86 Wine is easy since that's just skipping the wine{64,}{-preloader,}
arguments to get to the executable name.
Ryan Houdek