6722 Commits
Author SHA1 Message Date
Ryan Houdek cc85a6a722 Docs: Update for release FEX-2307 FEX-2307 2023-07-07 08:33:30 -07:00
Ryan Houdek e72fa02897 Merge pull request #2739 from Sonicadvance1/fork_mutexes
Linux: Fixes hangs due to mutexes locked while fork happens.
2023-07-05 15:01:05 -07:00
Mai 8a4c5bcc65 Merge pull request #2741 from Sonicadvance1/workaround_stdc++_bug
FHU: Workaround libstdc++ version 13+ bug
2023-07-05 17:20:35 -04:00
Ryan Houdek 7a13a24c05 FHU: Workaround libstdc++ version 13+ bug
In libstdc++ version 13, they moved the implementation of
`polymorphic_allocator` to `bits/memory_resource.h`.
In doing so they forgot to move the template's default argument to that
header. This causes the problem that `bits/memory_resource.h` is
included first without the template's default argument defined. This
breaking the automatic type deducation of `std::byte`.

Still broken in
[upstream](https://github.com/gcc-mirror/gcc/blob/be240fc6acc9714e66afbfbe6dc193844bfcba05/libstdc%2B%2B-v3/include/std/memory_resource#L79-L83)
and is unlikely to be fixed and backported. Since this is the only place
we use this type, just fix it here.
2023-07-05 13:52:23 -07:00
Mai 5a53931b92 Merge pull request #2738 from Sonicadvance1/xattr_emulatedpath
Linux: Handle xattr syscalls with emulated paths.
2023-07-05 15:19:18 -04:00
Ryan Houdek f9b352a093 Linux: Fixes hangs due to mutexes locked while fork happens.
When a fork occurs FEX needs to be incredibly careful as any thread
(that isn't forking) that holds a lock will vanish when the fork occurs.

At this point if the newly forked process tries to use these mutexes
then the process hangs indefinitely.

The three major mutexes that need to be held during a fork:
- Code Invalidation mutex
  - This is the highest priority and causes us to hang frequently.
  - This is highly likely to occur when one thread is loading shared
    libraries and another thread is forking.
     - Happens frequently with Wine and steam.
- VMA tracking mutex
  - This one happens when one thread is allocating memory while a fork
    occurs.
  - This closely relates to the code invalidation mutex, just happens at
    the syscall layer instead of the FEXCore layer.
  - Happens as frequently as the code invalidation mutex.
- Allocation mutex
  - This mutex is used for FEX's 64-bit Allocator, this happens when FEX
    is allocating memory on one thread and a fork occurs.
  - Fairly infrequent because jemalloc doesn't allocate VMA regions that
    often.

While this likely doesn't hit all of the FEX mutexes, this hits the ones
that are burning fires and are happening frequently.

- FEXCore: Adds forkable mutex/locks

Necessary since we have a few locations in FEX that need to be locked
before and after a fork.

When a fork occurs the locks must be locked prior to the fork. Then
afterwards they either need to unlock or be set to default
initialization state.
- Parent
   - Does an unlock
- Child
   - Sets the lock to default initialization state
   - This is because it pthreads does TID based ownership checking on
     unique locks and refcount based waiting for shared locks.
   - No way to "unlock" after fork in this case other than default
     initializing.
2023-07-04 02:13:06 -07:00
Mai f444b03317 Merge pull request #2740 from Sonicadvance1/faccessat2
Linux: Stop using faccessat2 for faccessat emulation
2023-07-03 14:35:31 -04:00
Ryan Houdek ed05846dd0 Linux: Stop using faccessat2 for faccessat emulation
This can can issues when running on devices with kernel older than 5.8.
2023-07-02 17:22:15 -07:00
Ryan Houdek f609990f90 Linux: Handle xattr syscalls with emulated paths.
Fixes a spurious `No such file or directory` error when `ls` is trying
to query a path's xattributes that come from the emulated rootfs.

These syscalls don't support the *at variants, so it can't use the optimized `GetEmulatedFDPath` implementation.
It must also return an error on a found file path, which makes their
implementation be slightly different than the other user of of
`GetEmulatedPath`. In the case of error, it must only return an error
from the emulated path if it is /not/ ENOENT.

Before:
```
$ FEXInterpreter /usr/bin/ls -alth /usr/bin/wine-stable
/usr/bin/ls: /usr/bin/wine-stable: No such file or directory
-rwxr-xr-x 1 ryanh ryanh 1.1K Sep 24  2022 /usr/bin/wine-stable
```

After:
```
$ FEXInterpreter /usr/bin/ls -alth /usr/bin/wine-stable
-rwxr-xr-x 1 ryanh ryanh 1.1K Sep 24  2022 /usr/bin/wine-stable
```
2023-07-01 16:47:19 -07:00
Ryan Houdek d2032da452 Merge pull request #2737 from bylaws/main
Some small fixes for android building
2023-07-01 14:59:18 -07:00
Mai 8047007a7a Merge pull request #2734 from Sonicadvance1/add_cssc
Emitter: Adds support for CSSC
2023-07-01 17:58:31 -04:00
Billy Laws 1f7e82ea09 CMake: Allow for disabling FEXConfig building
It's useful even in non-termux builds to be able to disable FEXConfig due to its build-time dependencies.
2023-07-01 22:21:17 +01:00
Billy Laws 35c52f20f9 AllocatorHooks: Avoid referencing valloc on Android
This is not implemented in bionic, so follow the MINGW approach and implement it with _aligned_alloc.
2023-07-01 22:21:16 +01:00
Billy Laws 17c82c22a6 JitSymbols: Store symbol mappings in /data/local/tmp on Android 2023-07-01 22:13:44 +01:00
Ryan Houdek df03a7b101 unittests/Emitter: Adds CSSC tests 2023-06-30 19:34:35 -07:00
Ryan Houdek c859540d7e Emitter: Adds support for CSSC
Not used currently but will be used in the future.
2023-06-30 19:34:35 -07:00
Ryan Houdek 20794593e7 unittests/Emitter: Update tests for updated vixl
Output in vixl changed for some of these. Most for the better but not
all of them.
2023-06-30 19:34:35 -07:00
Ryan Houdek a80a2bf569 External/vixl: Update 2023-06-30 19:11:22 -07:00
Mai e86a792189 Merge pull request #2731 from Sonicadvance1/serverfd_cloexec
FEXServerClient: Ensure server socket is created with SOCK_CLOEXEC
2023-06-30 17:59:36 -04:00
Mai d6c9b549df Merge pull request #2725 from Sonicadvance1/catchsegv_check
Scripts: Disable using catchsegv if it doesn't exist
2023-06-30 17:58:44 -04:00
Mai 1a4d5a1abb Merge pull request #2733 from Sonicadvance1/fix_jemalloc_checks
External/jemalloc: Updates external jemallocs
2023-06-30 17:57:29 -04:00
Mai c3e123df25 Merge pull request #2732 from Sonicadvance1/remove_dead_interface
Context: Removes dead `AddVirtualMemoryMapping` function
2023-06-30 17:57:06 -04:00
Mai cac798574a Merge pull request #2729 from Sonicadvance1/remove_warning
Linux: Remove warning that isn't necessary anymore
2023-06-30 17:56:35 -04:00
Mai 1506a19229 Merge pull request #2735 from Sonicadvance1/optimize_host_kernel_version_calculate
Linux: Optimize CalculateHostKernelVersion
2023-06-30 17:56:01 -04:00
Ryan Houdek 51861234bc Linux: Optimize CalculateHostKernelVersion
istringstream is a very slow way to parse this, let's make it a bit
quicker.

Some implementation numbers:
1. Original implementation - 1833556 calculations per second
2. std::strtoul implementation - 4666818 calculations per second
   - 2.54x the istringstream implementation
3. str::from_chars implementation - 5120718 calculations per second
   - 1.09x the std::strtoul implementation
   - 2.79x th istringstream implementation
2023-06-28 14:31:56 -07:00
Ryan Houdek 677b72c9a5 External/jemalloc: Updates external jemallocs
Fixes their `malloc_usable_size` checks.
2023-06-28 09:26:45 -07:00
Ryan Houdek 71a8c66c95 Context: Removes dead AddVirtualMemoryMapping function
This has been around since the initial commit. Bad idea that wasn't ever
thought through. Something about remapping guest virtual and host
virtual memory which will never be a thing.
2023-06-28 09:18:36 -07:00
Ryan Houdek 3372e9bdbb FEXServerClient: Ensure server socket is created with SOCK_CLOEXEC
To make sure we don't have dangling FDs when an application calls
execve, enable this flag.
2023-06-28 09:17:20 -07:00
Ryan Houdek df0723e14b Linux: Remove warning that isn't necessary anymore
This message is complaining each time VFORK was using with clone, but we
are handling VFORK here now.
This is just causing debug messages for no reason.
Remove the message and remove the flag removal option.
2023-06-26 13:41:33 -07:00
Ryan Houdek 7ee6fc0d7f Merge pull request #2726 from lioncash/fmtup
Externals: Update fmt to 10.0.0
2023-06-18 17:59:17 -07:00
Lioncache bf773452ac IR: Add missing formatters
Currently RegisterClassType and FenceType are passed into logs, which
fmt 10.0.0 is more strict about. Adds the formatters that were missing
so that compilation can succeed without needing to change all log sites.
2023-06-17 09:42:31 -04:00
Lioncache 95dbccc0ab Externals: Update fmt to 10.0.0
Keeps ourselves up to date with the latest major release.
2023-06-17 09:25:20 -04:00
Ryan Houdek e5189d63a2 Merge pull request #2708 from Sonicadvance1/fix_paranoidtso
Arm64: Fixes paranoidtso option for CPUs that support LRCPC/2
2023-06-16 13:32:43 -07:00
Ryan Houdek 7d5442357a Scripts: Disable using catchsegv if it doesn't exist
Fixes #2724

If catchsegv doesn't exist then just remove it from the execution
environment.

While nice to have, this shouldn't be mandatory especially with Debian
no longer shipping it.
2023-06-16 13:31:00 -07:00
Ryan Houdek 9dcc1deec0 Merge pull request #2722 from Sonicadvance1/rip_reconstruction
JIT: Implement support for per-instruction RIP reconstruction
2023-06-16 13:02:14 -07:00
Ryan Houdek 66d4206cd7 Merge pull request #2719 from lioncash/flags
OpcodeDispatcher: Ensure MXCSR is saved/restored with FXSAVE/FXRSTOR
2023-06-16 13:01:56 -07:00
Ryan Houdek f39163b1e1 Merge pull request #2723 from lioncash/str64
IR: Move VPCMPESTRX REX handling to OpcodeDispatcher
2023-06-16 12:59:03 -07:00
Lioncache 01837b3ad6 IR: Remove HasSideEffects for VPCMPXSTRX ops
This is a leftover from early on and not necessary, since we
don't operate on any state other than what is provided to the
IR op itself.
2023-06-16 11:53:31 -04:00
Lioncache bdb68840e3 IR: Move VPCMPESTRX REX handling to OpcodeDispatcher
We can handle this in the dispatcher itself, so that we don't need to pass along
the register size as a member of the opcode. This gets rid of some unnecessary duplication
of functionality in the backends and makes it so potential backends don't need to deal
with this.
2023-06-16 11:49:36 -04:00
Lioncache 4e2dcf3298 OpcodeDispatcher: Ensure MXCSR is saved/restored with FXSAVE/FXRSTOR
Previously, the bits that we support in the MXCSR weren't being saved,
which means that some opcode patterns may fail to restore the rounding mode
properly.

e.g. FXSAVE, followed by FNINIT, followed by FXRSTOR wouldn't restore the
     rounding mode properly

This fixes that.
2023-06-16 09:25:53 -04:00
Ryan Houdek 628f825416 JIT: Implement support for per-instruction RIP reconstruction
FEX's current implementation of RIP reconstruction is limited to the
entrypoint that a single block has. This will cause the RIP to be
incorrect past the first instruction in that block.

While this is fine for a decent number of games, especially since fault
handling isn't super common. This doesn't work for all situations.

When testing Ultimate Chicken Horse, we found out that changing the
block size to 1 worked around an early crash in the game's startup.
This game is likely relying on Mono/Unity's AOT compilation step, which
does some more robust faulting that the runtime JIT. Needing the RIP to
be correct since they do some sort of checking for what the code came
from.

This fixes Ultimate Chicken Horse specifically, but will likely fix
other games that are built the same way.
2023-06-14 17:28:56 -07:00
Ryan Houdek a80327f6df X86Tables: Adds some missing MEM_ACCESS flags to REP instructions 2023-06-14 17:04:50 -07:00
Ryan Houdek 16f7002222 Merge pull request #2720 from Sonicadvance1/fix_rapair_allocation
Arm64: Fixes GPR pair allocation to get one pair back
2023-06-14 17:00:02 -07:00
Ryan Houdek c9712e45cb Arm64: Fixes GPR pair allocation to get one pair back
When executing a 32-bit application we were failing to allocate a single
GPR pair. This meant we only have 7 pairs when we could have had 8.

This was because r30 was ending up in the middle of the allocation
arrays so we couldn't safely create a sequential pair of registers.

Organize the register allocation arrays to be unique for each bitness
being executed and then access them through spans instead.

Also works around bug where the RA validation doesn't understand when pair
indexes don't correlate directly to GPR indexes. So while the previous
PR fixed the RA pass, it didn't fix the RA validation pass.

Noticed this when pr57018 32-bit gcc test was run with the #2700 PR
which improved the RA allocation a bit.
2023-06-13 20:04:51 -07:00
Ryan Houdek a082161d72 Merge pull request #2717 from lioncash/xsave
OpcodeDispatcher: Handle XSAVE/XRSTOR
2023-06-13 16:32:53 -07:00
Lioncache 9017325c95 CPUID: Signify support for XSAVE if AVX is enabled
Now that XSAVE and XRSTOR are implemented, we can enable the
CPUID bits for them when AVX support is enabled.
2023-06-13 19:21:14 -04:00
Lioncache ae536e44d7 OpcodeDispatcher: Handle XRSTOR 2023-06-13 17:47:45 -04:00
Lioncache 7679485cc3 OpcodeDispatcher: Handle XSAVE 2023-06-13 15:01:33 -04:00
Ryan Houdek cb8bf1add6 Merge pull request #2716 from lioncash/helper
unittests: Add include search path for asm tests
2023-06-13 11:57:25 -07:00
Lioncache a69c457715 unittests: Add include search path for includes
Allows us to have a place to put helper includes and files that contain
macro utilities. This will be nice for making macro files that cut down
on verbosity across tests (e.g. Making tests for XSAVE would be way less
copy-pastey).
2023-06-13 14:40:43 -04:00