This is a new procfs symlink path that changes behaviour of binfmt_misc
when exposed. We need to check both procfs/exe and procfs/interpreter
and see if they exist AND also differ.
Once/if they do then we can disable a bunch of checking of paths once
they do. The fallback when none of this is supported has the same
behaviour has previously where it still does all the regular checking.
During binfmt_misc install cmake will check the kernel version for the
raw binfmt_misc writing. Which will never pass until we have a real
kernel version that it is upstreamed in.
For update-binfmts we add a new optional argument where the tool will
drop the flag if the host kernel version isn't new enough to handle the
option.
When a fork occurs FEX needs to be incredibly careful as any thread
(that isn't forking) that holds a lock will vanish when the fork occurs.
At this point if the newly forked process tries to use these mutexes
then the process hangs indefinitely.
The three major mutexes that need to be held during a fork:
- Code Invalidation mutex
- This is the highest priority and causes us to hang frequently.
- This is highly likely to occur when one thread is loading shared
libraries and another thread is forking.
- Happens frequently with Wine and steam.
- VMA tracking mutex
- This one happens when one thread is allocating memory while a fork
occurs.
- This closely relates to the code invalidation mutex, just happens at
the syscall layer instead of the FEXCore layer.
- Happens as frequently as the code invalidation mutex.
- Allocation mutex
- This mutex is used for FEX's 64-bit Allocator, this happens when FEX
is allocating memory on one thread and a fork occurs.
- Fairly infrequent because jemalloc doesn't allocate VMA regions that
often.
While this likely doesn't hit all of the FEX mutexes, this hits the ones
that are burning fires and are happening frequently.
- FEXCore: Adds forkable mutex/locks
Necessary since we have a few locations in FEX that need to be locked
before and after a fork.
When a fork occurs the locks must be locked prior to the fork. Then
afterwards they either need to unlock or be set to default
initialization state.
- Parent
- Does an unlock
- Child
- Sets the lock to default initialization state
- This is because it pthreads does TID based ownership checking on
unique locks and refcount based waiting for shared locks.
- No way to "unlock" after fork in this case other than default
initializing.
istringstream is a very slow way to parse this, let's make it a bit
quicker.
Some implementation numbers:
1. Original implementation - 1833556 calculations per second
2. std::strtoul implementation - 4666818 calculations per second
- 2.54x the istringstream implementation
3. str::from_chars implementation - 5120718 calculations per second
- 1.09x the std::strtoul implementation
- 2.79x th istringstream implementation
This message is complaining each time VFORK was using with clone, but we
are handling VFORK here now.
This is just causing debug messages for no reason.
Remove the message and remove the flag removal option.
I originally wrote this emulation prior to me fully understanding how
the syscall works. So there are two optimizations here.
1) No need to consume the incoming buffer at all.
- Originally I thought the incoming dirent structures were used to
calculate offset.
- This is not the case, the FD's file position is used instead.
- This means we can remove the incoming buffer consuming overhead
entirely.
2) No need to allocate a temporary buffer at all.
- With getdents and getdents64 we are guaranteed to be dealing with
structures that are the same size or smaller than the host
structure.
- This lets us encode the real host dirents in to the provided
buffer.
- After the `getdents64` host syscall, we then iterate forward
through the list, modifying as we go.
- Need to make sure to shift the elements of the structure in order.
- Need to make sure to use memmove on the `d_name` member since the
movement region can overlap.
These two optimizations significantly reduce the amount of time spent in
getdents, which has a noticeable impact on load times.
Side-tangent: I noticed a fun quirk of how NFS operates with getdents.
If the FSCache hasn't populated the metadata for that folder, then it
will early return with "some" data, not fully maxing out the buffer. The
kernel will start prefetching metadata assuming directory iterating is
happening. The next `getdents` happens and it should return a larger
number of elements.
Very neat.
This is not an attempt to clean up the various issues with the pthread
logic, instead just moving the pthread specific logic out of FEXCore in
to FEXLoader.
FEXCore needs to know how to create threads in an agnostic way. Which is
why we obfuscate the details with this inteface.
Initially this was implemented with the pthread handlers in FEXCore and
expected eventually for those to get moved to the frontend. This is the
time when it has been moved.
This is the first step towards compiling with llvm-mingw.
Still a long way to go.
Ryan Houdek