The LookupKey based on data available at Lookup time gives a list of possible
candidate entries, which may have different guest code sizes/footprints.
If only one candidate, it's stored in-line in the map like before - if we grow
past that, an additional (multi)map is allocated, sorted by footprint.
The footprint sorting lets us reduce the amount of hashes performed at Lookup
to the strict minimum.
Anon entries keys are based on a limited size decoded prefix, so this is an
important part of geting the best hit rate possible out of all the other work
with anon keys, as there's often multiple candidates with the same prefix.
Cap the max amount of entries per bucket to limit growth.
This allows WTF to catch the allocations just like on Linux. Punch our
unixlib path all the way through to rpmalloc so it gets named and
tracked properly.
Otherwise GuestSize ends up different across compiles of the same valid code
extents, including the copy of it baked in JITCodeTail of the host code itself.
JIT invalidation around syscalls and kernel calls is mighty fickle.
This is why the previous code path was born. In order to properly handle
this case we need strong coordination between Wine and FEX around
invalidating code and memory protections. This doesn't quite exist today
so we're kind of stuck with a kludge solution. Instead of forcing the
Persona 5 code invalidation on to every process, only do it on P5R.
This fixes a hang in msiexec with PhysX trying to do a blocking read and
jitting code or creating threads, while also maintaining the P5R
approach. The full comment is in the file about the reasoning.
Return the same as the NonExecutableRange for now. I have a unittest
that is mostly correct with this change, just need to fixup some of the
supplementary data.
For now push this change to get a bad assert removed that can happen
during code discovery. I'll fix up the remaining details afterwards.
Moves cache data from %APPDATA% to wherever `STEAM_COMPAT_SHADER_PATH`
points. Turns out this was easier than expected.
Basically moving disk cache from:
- compatdata/<AppID>/pfx/drive_c/users/steamuser/AppData/Local/fex-emu/DiskCache
to:
- shadercache/<AppID>/fex-emu/DiskCache/
Fixes#5913
Just be very explicit about this rather than questioning why MMX moves
can't be a nop in the regular vector move code.
Doesn't change codegen so binarycacheversion doesn't need to change.
All buffers should be disowned leaving their respective compilation
sites, and reowning a buffer should never have the flag already be
owned.
Throw an assert in both cases because that would be a programming error
and result in some squirrely buffer handling
UpdateTopForPop_Slow() now invalidates ST(0)'s tag by default,
so every slow-path StackPop() does this consistently instead of
the previous special case only in OP_POPSTACKDESTROY.
FINCSTP is an exception as it only moves the stack pointer without
invalidating the tag.
Decode a few bytes in advance to get a hashable prefix to use as key.
Generate touched pages dynamically since they can be misaligned now, as the
cached-hit guest code isn't necessarily in the same spot as the store was.
PR #5902 technically introduced a bug where we would read past the end
of bounds for thunk instructions when full smc was enabled. Luckily this
never occurs in practice as the Mono hacks never are on VDSO boundaries,
and no one is expected to enable full smc detection really.
Switch this path over to using crc32 unconditionally. This raises our
minspec technically to armv8-a+crc, but nothing that matters shipped
without crc so it's fine.
This also is a minor speed and JIT size reduction due less branches
polluting the BTB. But really only for mono/unity games.
Requires revving the DiskCache version again.
Ensures that whenever a file handle is transferred anywhere, that the
moved from instance won't end up closing the file handle when the
destructor runs.
Does less hashing, improves hit rate when there's data adjacent to code,
and/or when the SMC check makes us rebuild code that hasn't actually been
changed.
The JIT was doing a bunch of additional work where it was saving and
restoring registers and then juggling the arguments back in to a stack
frame. All of this is nonsensical without the optimization where we
could call syscalls inline without a stack frame.
Instead remove this optimization entirely and behave like a "generic"
syscall path always. The Linux syscall handler now pulls the arguments
out of the CPU context directly and stores the result back in to RAX
directly as well.
This has knock-on effects where technically syscalls are
going to be slightly faster because no stack frame setup for the
arguments, but additionally we are going to be able to have syscalls be
proper serialization points where we can interrupt the syscall and
long-jump out without problems.
Bumps the DiskCache version again because it causes codegen to change.
Two added failure modes here. If the disk cache version has changed then
the json files must be updated to the new version to ensure correct
tracking.
Additional failure mode is that codegen actually changed but the disk
cache version hasn't. This is the expected common failure mode and we
need to do additional work before updating json results. This just means
incrementing the disk cache version before updating the instcountci
results. Have a fairly lengthy error message to showcase how much of an
impact this might have.
Everything that we care about supports the unixlib path now. Also turns
out we were doing `svc #0` on Windows when unixlib didn't exist which is
kind of funny.
Remove the legacy hacky path as it is no longer necessary.
When a guest executable is invoked through two or more levels of symlink,
FEX expands $ORIGIN in its DT_RPATH/DT_RUNPATH to the directory of the
intermediate symlink rather than the directory of the fully-resolved binary.
Libraries referenced relative to $ORIGIN then fail to load.
This PR fixes the issue by continuously chasing the symlinks until we find
the actual executable name.
This is unfiltered data that ends up in HostFeatures. If it was
serialized when set then it would effectively never allow the code
serialization to be used.
If we're shrinking the L1 cache then we just deleted the entry that we
just looked up. Add it back to ensure we don't get yet another lookup
for this entry.
CMAKE_CXX_FLAGS is a space-separated string rather than a semicolon-separated
list. Passing ${CMAKE_CXX_FLAGS} directly to execute_process(COMMAND ...)
passes the entire multi-flag string as a single argv argument to the compiler,
causing option parsing to fail when multiple flags are present (such as
flags configured via the CXXFLAGS environment variable).
Use separate_arguments() to convert CMAKE_CXX_FLAGS into a list so each flag
is passed as an individual argument.
Growing widens L1PointerMask without touching the table, so an entry whose
address has the new mask bit set sits where InvalidateCache no longer looks,
and a later shrink hands it back to the JIT. Wipe [0, old) on grow, the way
the shrink wipes [new, MAX); the still-valid entries go with it.
Compute a bucket hash and use it in the cache path to avoid grouping entries
that will never make sense together. Don't trust the path, though, and also
lace it into the keys themselves, so that eg. a RO cache miss can never turn
into corruption.
Keep a readable metadata entry at the beginning of the cache, with readable
version, bitness and serialized config.
Use printable characters in FOZ key names as intended.
Turns out packed enum classes without specifying an underlying type
causes problems. Declare its underlying type as uint32_t to match
everything else here.
We were assuming that SHLD undefined behaviour matches SHL, but the
specification actually changes a `ge` comparison to `gt`, which means a
shift of 16 isn't UB!
Thanks to the impeccable @OFFTKP in #5842 for bringing this up as it took a bit
for me to figure out what was actually wrong here. I modified their
unittest to cover more just to ensure we don't break it.
As long as the hash is smaller than 64-bits we can just return the bits
encoded directly. Codegen slightly changes with this packed
representation, but doesn't really matter.
Also removes ICacheLineSize as that doesn't actually affect codegen for
us. Once we add 27 more HostFeatures we can switch the hash over to
XXH3.
This was using a one second alarm which could catch the JIT while it is
still busy. Instead wait for it to let an alarm thread that it is ready,
then tgkill the thread. This removes the race that this was hitting.
We actually never use this anymore, we instead always pass zero for
both, and then rely on the thread inheritance model or setting the
values manually. Now that we expose visibility of the
InternalThreadState to the frontend they just access it directly.
Just a smidge of cleanup, NFC.
Serializes every option, even ones that are set to default to ensure
validation that if any config option value is added or changed that they
are captured.
Skips a handful of options that are either meta options, environment
options that don't matter, or HostFeatures which is handled elsewhere.
(HostFeatures will be controlling bucketing rather than the remaining
options).
This serialization is currently 1413 bytes and generates in 17580ns on
my A1A. So it's not the fastest, definitely don't want to be generating
it constantly per process.
FEX Relocations now live at an offset from the `CodeData.BlockBegin` of the
code. Regardless of where the relocation moves to, it should always be
relative to that address. This is what makes it PIC compatible.
We were preemptively offsetting the relocation location to be relative
to the memory base in the buffer, which is unnecessary and causes code
caching to basically relocate twice to get the real location.
So in JIT.cpp, stop relocating the offsets, they're already relative to
`BlockBegin`, which is offset 0.
Then when storing the relocation, stop relocating offsets AGAIN because it's
already relative to the code being serialized.
Then when loading the relocations in `CodeCache::ApplyCodeRelocations`
stop relocating offsets YET ANOTHER TIME.
All this is to say that relocation offsets are already PIC and relative
to offset 0, so we don't need to do it three times.
I accidentally replaced a couple usages of `AllocatedSize` with
`GetAllocatedSize()`. This resulted in a JIT buffer that ran out of
space would actually allocate a slightly smaller buffer each time, and
then it cascades downwards resulting in catastrophic performance.
Fix the use in `SharedCodeBufferManager.cpp` and `Core.cpp` which were
incorrect and renames the function to be more explicit.
Noticed while taking a look at the relocations that we were technically
not doing alignment before writing down code size.
- Make sure Align16B isn't used with unaligned code with assert
- Switch an `Align` over to `Align(16)` to force 16-byte alignment
- Without NOP insertion, as this is data at this point, so just zeros.
- Record data size after that alignment
- Remove the `Align16B` that occurred afterwards
- Previous query between alignments would leave us with up to 12 bytes
unaccounted for.
- Ensure everything is using the correct sizes by not querying again
- Ensure that emission buffer abuse can't happen by zeroing the buffer.
The various places that were using the CodeBuffer object were using
internal implementation details that are changing as we move over to a
bitmap allocator.
Preempt this by hiding some of the implementation details early without
changing behaviour. `GetBufferBase` is still technically leaking some of
the internal details, but it needs changes around how relocations are
being handled and how the disk cache validation works in order to handle
that right now.
Should be no functional change.
With all Stores happening on the same thread now, we can also make locking
more granular for extra perf. Move to positioned IO for everything, as we
can't reliably track the cursor with that faster locking model.
Add some bounds checking to index population to protect against corruption.
It's soon going to change how these buffers are managed, where the
CodeBuffer is going to manage its own allocations soon once it changes
over to the bitmap allocator. Additionally the Manager class is actually
going to do proper management, pooling, and invalidation handling.
Split the task preemptively before we switch to the bitmap allocator to
reduce churn. A little change in the CodeCache where it needs to query
the codebuffer directly rather than the context, but fairly safe.
Shouldn't be any real behaviour change.
Serializes code blocks to disk - only blocks coming from known regions, for now
Disabled by default, key and versioning still needs work, but works for testing
This test is trying to execute "invalid" code from the last two pages of
the address space to the first two pages of the address space. But
failed to noticed that the last two pages of the 32-bit x86 address
space are actually valid, usually containing VDSO things.
If `STEAM_COMPAT_FEX_CONFIG` is missing options, then instead of having
an opinion about what those options should be, just leave them unset.
This allows FEX's regular default option handling to kick in for missing
configuration options.
Where previously if an option was missing from the config, it would
default to boolean false, which may or may not be the default depending
on option.
This removes the fairly long lived lock that the buffer allocator held
while doing significantly more work than intended while holding that
lock.
As the first step towards moving over to the atomic bitmap allocator,
change this to be atomic to closer match what the new allocator is
doing. Since we are just doing linear allocations, this is an easy
convert and should give a good stutter improvement.
This is tailored towards our needs for our JIT and eventually replacing
the linear allocator. Allowing us to reallocate memory for code blocks
that have been invalidated, letting us keep a single code buffer around
for longer and using less memory overall.
In particular, high-invalidation games that ship anti-tamper tend to
emit millions of ~128-byte blocks in just a handful of minutes which
causes our current linear allocator to consume gigabytes very quickly.
This will allow us to more aggressively reuse the allocation space and
reduce the memory load in those situations.
There's some additional resize tuning that needs some work whence it is
in situ which doesn't need to be done now.
This thing is a bit intense, so some requirements from the start:
- It needs to be lock-free and thread-safe
- It needs to support contiguous range allocations
- It needs to support allocations larger than a single atomic word
These requirements kind of fly in the face of most bitset allocators
where they will support some parts of these requirements, or just throw
a mutex in front of the whole thing.
Some implementation details:
- If allocating only 1-bit, trivial and always succeeds if there is space
- If allocating <= 64-bit, then always succeeds if there is at least
those many contiguous bits within a single atomic word
- Allocation can fail if there are cross-word contiguous bits of the
size available
- Introduces some sparsity
- If allocating > 64-bits then it falls down the longer scan path.
- Searches for contiguous bits of free space between multiple atomic
words.
- If found, will attempt to allocate tracking which bits were allocated
- If allocation fails, unwind bits already acquired and continue
scanning
Some downsides to this implementation:
- Allocations can fail if sparsity builds up
- Heavily contended allocations can be worse than a lock
- If larger than atomic word allocations are in flight.
- Unwinding larger than word allocations and continuing scanning adds
overhead, a lock would have won at that point.
- A small bit of false sharing where an atomic word is read without
acquire semantics for scanning can technically overlook some
allocations that no longer exist.
- Slower than a linear allocator, but that's not unexpected.
Most of these downsides are okay for our use case, which is code buffer
allocations with the ability to do partial invalidation. If the atomic
bitset fails to fit an allocation, we can throw away the code buffer
like we currently do.
The bitmap allocator that uses this lock-free atomic bitset is still
in-flight but this is one complex container that can land independently.
Useful for removing integer division instructions when we know the
source value is aligned to be power of two. As integer division is quite
slow, we want to use this when possible.
The two page-size extensions are a nop so might as well as enable them.
For the debug flag, we already set the duplicated flag in 8000_0001.edx, but missed this one.
Doesn't add anything new for the FEX side, but Burnout Paradise (and
remastered) is incorrectly checking for SSE2 support by checking if this is set.
Closes#5805 although their (ML?) write-up was incorrect.
While it would be better to fix the error in the source, it shouldn't be
a case of blocking release. Print the line that was failed to parse and
then continue onwards.
In particular hit by `ERROR:root:Failure to parse Thread shared code buffer management`
This fixes compilation with libc++ 23, which has removed a number
of unnecessary transitive includes in its headers.
Include <cstdlib> in StringConv.h for std::strtoll and std::strtoull.
Include <cstdlib> for the declarations of malloc/free/realloc/calloc
in Alloc.cpp. (Without this, the functions we define end up with
C++ name mangling.)
Include <stdarg.h> in IO.cpp for va_start/va_end.
NFC
- Renames CodeBufferManager to SharedCodeBufferManager to be more
explicit about it being shared between threads
- Renames `CodeBuffers` to `SharedCodeBuffers` to make it more explicit
about sharing these buffers between threads.
- Separates the Manager to its own file so it is distinct from the rest
of the CPUBackend code
Makes it easier to parse ownership and lifetime semantics of these
buffers.
Now that we have VMA region naming enabled on JIT buffers, this is no
longer used. Confirming a region is a JIT buffer is now just a case of
comparing the name that shows up in `/procfs/maps` rather than dumping
the first bytes of an unknown region.
`TempAllocator` was a bit too opaque as to what the allocator was for,
so I kept needing to lookup its usage every couple of months. Rename it
to `TempCodeBufferAllocator` so I can remember that it is a temporary
allocator for the staging JIT code buffer more easily.
NFC
Gets rid of potential extraneous copies. We also add handling for cases
where two passes with the same name are unintentionally added.
Previously we'd blindly overwrite the mapping.
We don't conditionally add any passes, so we can simplify the interface
so that we just add all existing passes at once. Makes the core
initialization process a little more straightforward.
This both take a void*, so the casting is unnecessary to begin with,
since this would occur anyway without it. We can also avoid a
duplication to reduce line noise.
This needs to be enabled when *generating* caches, not at runtime when we're
loading them (unless we're compiling for validation).
Previous code would incorrectly disable NOP padding in FEXOfflineCompiler and
instead enable it at runtime when it wasn't needed.
Previously this wouldn't have worked, since .first isn't a valid member.
The only reason it wasn't caught is because the function is never
instantiated.
We've accumulated a bunch of forward declarations that are no longer
necessary. We also don't need to pass the signal delegator as a
reference, since we're not modifying the pointer itself, it's just
passed in to register a signal handler.
Makes sure prototypes are visible to their implementation. Also marks
functions internally linked where applicable.
Also makes it a little more visibly obvious which bits are exposed for
use elsewhere.
Just lets you get a broad overview all at once instead of needing to
type out every long command.
Now it's easier to be lazy and just pass "-e", or "--all-emu-info".
STLXR cannot use the same register as both the status register and the
value register, otherwise it's architecturally unpredictable
behavior.
Only applies to hardware without FEAT_LSE, so this only meaningfully
affects hardware using the v8.0 spec, since FEAT_LSE becomes mandatory
in v8.1 and newer.
ShouldClose was never being set in the event we opened a regular file.
The only time it was set (to false) is when it's used to encapsulate
stderr and stdout.
So anything opened by a File instance was essentially held open.
Same functional behavior, just makes it visually match the store size
below. Technically also avoids delegating off to the 64-bit element
path if a 128-bit constant zero is already loaded.
This used to be used for the intrusively allocated `LiveVMARegion` but
that is all handled internally to the object now, making this
unnecessary. It was always receiving zero and doing nothing so just
remove it.
Lets us shave off an instruction and also avoid using a temporary
register in some cases. We can also tweak our worst case that requires a
predicate to eliminate the temporary as well.
We can also expand our cmpps cases, so that we can reflect the
BSL2N usages in instcountci.
This needs to divide by 8 to get a proper byte size for all type sizes.
The only usage of this is currently a uint64_t, so it worked by
coincidence, since sizeof(uint64_t) == 8.
Because our bitset type is uint64_t, then that means Memory + ManagedSize
is more like: Memory + (ManagedSize * 8), which is way larger of a base
than we need.
According to POSIX open docs, this is a completely separate flag that
isn't a combination of O_RDONLY and O_WRONLY, so we need to handle this
separately.
Makes the codepath behaviorally symmetric with the Windows one.
This just means the end of the file is being used as the base offset.
Also note that according to the documentation for SetFilePositionEx,
that setting the position beyond the current file size is not considered
an error as far as the API is concerned.
Since this was a reference, this would end up overwriting Region[0] with
whatever the smallest region was instead of just being a running pointer
to what happened to be the current smallest region.
We can switch over to a pointer to avoid obliterating the first memory
region.
Ensures that even if someone threw bogus constants in the upper bits of
the immediate, that the special-cased comparison types would still be
handled properly.
We can move the masking in the AVX variant too, just to be consistent.
Previously this would result in a larger amount of words specified than
necessary.
e.g. Given nfds = 1:
With AlignUp(1, 32) / 4, we'd end up with supposedly eight words, when it
should only be one word.
On the other extreme, given a full fd set of 1024 fds, then we'd end up
with 256 words, when it should only be 32.
CopyToUser doesn't return the number of bytes copied, but rather returns
0 to indicate success, otherwise a fault has occurred (and the SIGSEGV
handler has set X0 to EFAULT)
Element size doesn't really mean anything here, considering all bits are
acted upon independently of segmentation.
Makes using these ops a little bit less noisy.
If during multiblock code discovery a RDTSCP/RDPID instruction was
encountered then ProcessorID has an assert at JIT compile time. Make
sure to early exit with an illegal instruction encoding early instead.
Also make sure to correctly report RDPID support in CPUID, it's
technically a different bit than RDTSCP.
Fixes a crash in Crusader Kings 3's Paradox Launcher installer. Although
the installer seems to fail otherwise for some reason.
We can massage a given selector into a valid predicate register bitmask
and then simply perform a merging move, which eliminates most busywork
around optimizing 256-bit blends.
In the future, once we drop SVE2.1 support in, we can use PMOV to
eliminate the load from memory and related constant management.
Instead of burning roughly a million watts, put this spinloop on a WFE.
This tends to occur on a crash during shutdown that isn't fully able to
be avoided. The least we can do is not consume all the power in the
world.
The lazy code loading refactor replaced LoadData with the new
LoadCache/EnableLoadedSection API but left the Windows path as TODOs.
Implement the wiring: LoadAOTImages now calls LoadCache +
RegisterMappedCodeBuffer for each mapped cache file, and HandleImageMap
calls EnableLoadedSection (with nullptr thread since lazy mapping is not
yet implemented on Windows).
`DetermineVASize` does not return the size of VA, but the number of bits
it can use. Change the naming to make it more self explanatory.
In the mean time also move `HostVASize` global into `GetHostVABits`
since it is not and should not be used directly.
Commit abf9724 ("Allocator: Fix and optimize VA range detection")
removed assignment to the `HostVASize` global thus making each call to
`DetermineVASize` redo all the work with potential to produce incorrect
results. Set the global again to fix this.
While unlikely to be used in practice over other kind of
shuffling and blending, these should also have stress tests
to make sure they do the right thing.
Currently we split these into two 128-bit operations since VIXL doesn't
have support for the unified SVE operations yet.
Now we fully support VAES on SVE256.
This was accidentally setting `CurrentSize` instead of just returning
the newly allocated size to the frontend. This was causing the frontend
to then fail to detect the reallocation actually occured and no longer
get stats for new threads.
Also happened to not use `NewSize` but instead `CurrentSize * 2` which
didn't matter as it matched the growth pattern, but was technically
incorrect.
Fixes SHM stats since the introduction of the unixlib, ezpz.
We really don't want to do this, but in the case that the AF_UNIX path
is longer than the 108-byte limit that sun_path provides we don't really
have a choice. The alternative choice would be to switch /entirely/ away
from AF_UNIX and instead use pipes. We need a bandage fix for now, so
throw the socket in to a temp folder if the path is too long.
CPUFeature detection is marginally different between Linux and Windows.
FOC was only using the Linux path which had two broken things happening
to it.
- Feature detection was incorrect and enabling/disabling features
differently from wow64/arm64ec .dll files
- HostType was being set as Linux even though it was generating code for
WINE
Ensure that when built for Win32 that it uses the correct feature
fetching.
One thing that is still incorrect is that 64-bit or 32-bit is determined
at compile time on win32, whereas the Linux side parses an ELF and
determines bitness at runtime. This doesn't fix that remaining problem
there.
If the first source and destination alias, then it's fine to use the
register in destructive operations, since the source data doesn't need
to be preserved.
Tiny saving, but reduces overall register use in some cases.
Centralizes all the nasty behaviour that will end up breaking when WINE
eventually turns on userspace syscall dispatch. Pushes all of the logic
in to the UnixLib. Support both paths until everyone is migrated to
supporting the UnixLib, then we can delete the bit of code duplication
between the PE side and UnixLib side.
Helpful that everything that gets punched through the UnixLib is
optional, so worst case some optional bits can break for a while.
Mainly a correctness change more than anything. The COMISX and
UCOMISX group of operations only ever load 128 bits when given
a vector source.
No change in codegen, but ensures this doesn't change if any backing
handling changes.
Since the registers are non-aliasing, this is about the best we can do
now. These are just holdovers from the initial bring-up of the 256-bit
SVE path where optimization wasn't as strong a concern as getting everything
in place and running properly.
Performs the same thing as what we've done for VMOVMSKPS. Though,
the effect isn't as drastic, given we're only operating on a max
of four elements as opposed to 8.
Currently we can trivially split this up and join the results, which
is much nicer than iterating all the elements individually and shifting
their sign bit over.
This was handled in both the Module.cpp files and also the common
TSOHandlerConfig on accident. Wouldn't have caused an issue but it was
definitely a bit weird.
Including fallback to non-unixlib path because we need to support both.
Showcases how these are going to be implemented without throwing the
entire world at it right away. Next PR will be implementing the
remaining four necessary unixlib handlers that we will require:
- Kernel unaligned atomic control
- shm_stats thing
- madvise operation
- prctl vma naming
Currently does nothing other than load it (as the library also doesn't
do anything yet). Ensured it was working by temporarily creating a test
entrypoint and doing `Call` on to it.
Next step after this is to reimplement some of the nasty hacks FEX is
doing inside the unixlib code itself.
Newer WINE has a better mechanism for asking to load unix libraries.
Older WINE like what is in Proton doesn't have this yet. Add definitions
for both so we can try either one.
We are going to need a unix library. Going to take this one step at a
time without AI/ML so I fully understand all the pieces of the puzzle,
and to ensure we don't lose any functionality before we're ready.
This only ensures that we are building the Linux facing .so files for
arm64ec and wow64, but they are empty today. Next PR will be
initializing it on the PE side.
Because these compile options change codegen, we need to make sure these
are runtime selected rather than compile-time selected. Will reduce
code-cache variance.
When we have a 3 element identical permutation followed by a single
unique outlier, we can simplify the whole operation into a single
broadcast followed by an insert.
e.g.
0b00'00'00'01
0b00'01'01'01
0b11'11'00'11
are all examples of cases where we can broadcast and then insert.
Apparently some compilers or libstdc++ or libc++ takes offence to
constexpr std::array that gets filled by GOT. My compiler this generates
the same code regardless but I guess this'll probably fix#5582.
In the slower case, if our iteration index and the selector index match,
then all that means is that we'd be inserting the same data that already exists
at that location, so we can skip the insertion in that case.
These functions need to clean at least 64 bytes of cache since this is
the default on x86_64, but previously they could clean less if
DCacheLineSize was smaller than 64 bytes.
Arbitrary insertion of an element requires the use of a predicate
register. Since we only care about a particular element in the vector,
being replicated, we can broadcast that element instead of doing an
insert, which is effectively the same thing without excessive busywork.
Lets the AVX implementation get all the optimizations that the SSE
variant has, reducing the overhead a little.
Even with the individual lane handling, this is still leagues better
than all of the individual inserts that are pretty beefy with SVE.
For example:
vpshufd ymm0, ymm1, 0b00000011
drops from 50 instructions to 9
We don't need to broadcast if we're inserting across registers into the
equivalent position, since we already have a predicate around that can
satisfy that.
Lets us reduce inserts by seeing which bits in the selector mask
indicates a particular source is used more than the other one, and
then just uses that as the base to be inserted into, cutting down
on overall insertion overhead.
In some cases, this can be quite drastic, like with:
vpblendw ymm0, ymm1, ymm2, 0b00000001
being cut down from 98 instructions to 14.
These are old paths still around from when StoreResult used to
automatically perform truncating moves.
These aren't necessary anymore, since the AdvSIMD operation already
ensures zero-extension.
(See #3799)
I had a feeling #5569 was a little overkill, but was just getting
everything up to a functional baseline at the time. Now, with the tests
added in #5584 to test all SSE paths, I was able to see which comparisons
in particular were the ones that would have deviating behavior (NLT and NLE)
This lets us safely restore the behavior without the excessive moves on
hardware that makes use of FEAT_AFP.
The compiler is smart enough to use the zero register for atomic
operations. Our JIT never generated code like this so it was unexpected.
Make sure handle zero register in all the cases where it matters.
See #3799
These were technically already covered when the work was done to make
SSE insertion behavior conform to hardware, so this just adds tests
that ensure that behavior holds over time.
See #3799 for the bulk of the issue explanation. Ensures that emulated
SSE operation on aarch64 don't end up obliterating the upper 128-bit
lane when SVE-256 is present (Adv. SIMD operations zero-extend)
Knocks out quite a few SSE instructions right off the jump.
We had a few places that were using Bind, and a few other places
that were using specializations as a means to composing the instruction
tables. Instead, we can just use Bind consistently, which lets us tidy
up a bunch of the implementations (and gets rid of some unnecessary
codegen).
Implements an offline JIT compiler backend that compiles x86 code blocks
from a given code map into an ARM64 code cache on Windows. Separate
binaries are built for WOW64 (32-bit) and ARM64EC (64-bit) targets.
NOTE: This patch originally added a separate binary; instead the new
functionality is added to the existing FEXOfflineCompiler and will
be properly integrated in the next patches.
Now that NX is tracked in the frontend, we need to ensure that adjusted
RIP pages are tracked correctly. Keep around both instruction stream
pointers, validate the the original RIP is executable, and read from the
adjusted RIP as appropriate.
Fixes#5544
This operation will take care of any pending code cache operations:
* import new code maps from `CACHE_DIR/codemap/new` and process them to `CACHE_DIR/codemap/ready`
* generate caches for updated code maps with new blocks
* ensure caches already exist for all other code maps (and generate them if needed)
When the LOCK prefix is on an instruction that doesn't support LOCK then
it raises a SIGILL. Make sure to pass that up.
Additionally if the instruction does support lock prefix, has a lock
prefix, but the destination is not memory then that is also invalid.
If the override isn't handled then it'll fall back to either using the
wrong address and getting the wrong data or crashing depending on how it
is broken.
This is enough to get less complex cuda applications running, and is a
good starting spot to slowly finish off the remaining implementation.
Some information:
- 429 functions in total
- 153 only compiled for 64-bit (35.6%)
- 7 functions disabled entirely (1.6%)
The main thing /not/ working with this initial implementation is .cu
files compiled in to an ELF using the static cuda runtime. This is due
to the `cuGetExportTable` function being stubbed out and the static cuda
RT requires at least two interfaces from that function before it
continues.
This function isn't publicly documented by NVIDIA but has been publicly
reverse engineered to be fairly trivial. It's just a jump table with the
first element being the size of the table in bytes.
That will be the next step of the implementation.
building on musl fails with:
`error: Unsupported parameter type 'snd_htimestamp_t *' (aka 'timespec *')`
due to empty padding members in `alltypes.h`:
```
STRUCT timespec {
time_t tv_sec;
int :8*(sizeof(time_t)-sizeof(long))*(__BYTE_ORDER==4321);
long tv_nsec;
int :8*(sizeof(time_t)-sizeof(long))*(__BYTE_ORDER!=4321);
};
```
add (missing) annotation to libasound interface
fix: #5513
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
We can merge the lsr+and in to a single lsr by 18 and then use ldr with
LSL of 3 to accomplish the same result. Modern Cortex doesn't even
generate an additional integer pipeline uop for this ldr+lsl instruction
anymore.
We were accidentally allocating some things without this flag and it was
causing us to dump memory in to the lower 32-bits on arm64ec.
This was causing the game
[Below](https://store.steampowered.com/app/250680/BELOW/) to run out of
memory to allocate for its LUA JIT and causes it to crash.
building using musl fails due to missing defintions:
```
Source/Tools/LinuxEmulation/LinuxSyscalls/Syscalls.cpp:913:23: error: no member named 'sleep_for' in namespace 'std::this_thread'
913 | std::this_thread::sleep_for(std::chrono::milliseconds {10});
| ^~~~~~~~~
1 error generated.
```
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
**Faulty Behaviour:**
When not building on Ubuntu `unittests/ThunkLibs` fails to find
c++ header and fails:
```
gen_input.cpp:2:10: fatal error: 'cstddef' file not found
2 | #include <cstddef>
| ^~~~~~~~~
1 error generated.
```
This also includes building with nix-shell:
```
nix-shell ../Data/nix/LibraryForwarding/shell.nix --run "cmake --build . --target thunkgen_tests"
```
**Root Cause**:
`X86_DEV_ROOTFS` is not used, but header paths are hard
coded to Ubuntu's multilib layout (`/usr/i686-linux-gnu/include/`,
`/usr/x86_64-linux-gnu/include/`). It works on Ubuntu but the
mechanism to use to another directory is broken and the include
paths always point to the host.
**Solution**:
set `--sysroot ${X86_DEV_ROOTFS}` for guest builds and remove
hard coded include paths.
Fix: #5515
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
Ccache won't attempt to cache files that use __DATE__/__TIME__ since their
contents would always be out of date. Setting sloppiness disables this
behavior, which works for us since we don't use __DATE__/__TIME__ for anything
that needs accurate values.
If the simulator was selected, we would still capture CTR and MIDR on
the host AArch64 system. Potentially modifying codegen in unexpected
ways.
Ensure we return 0/0 like under x86 with simulator to simulate
"unknown".
This optimization was only written for Ampere1A where it showed a
noticable performance improvement in #5321. On Cortex it didn't matter.
Turns out this actually hits a bad case on Oryon CPUs where `dc zva` is
actually dramatically slower in the face of memory barriers and
overlapping stores in flight.
So now just detect Ampere and only use the optimization on that hardware
and send everyone else down the regular path.
microbench A1A:
```
Cycle counter frequency: 1000000000
Cycle counter granularity: 20
ns in cycle: 1
suite: memory
Test, Total Cycles, Iterations, Cycles Average, Iter Time Average, iterations/Second
dc zva - vzeroupper, 723390880, 363855872, 1.99, 1.99 nanosecond, 502986534.75
dc zva - vzeroall, 571708060, 161742848, 3.53, 3.53 nanosecond, 282911610.52
dc zva (stp emu) - vzeroupper, 541543980, 107872256, 5.02, 5.02 nanosecond, 199193897.42
dc zva (stp emu) - vzeroall, 722548940, 71958528, 10.04, 10.04 nanosecond, 99589832.63
```
microbench X2E:
```
Cycle counter frequency: 19200000
Cycle counter granularity: 1
ns in cycle: 52.083333333333336
suite: memory
Test, Total Cycles, Iterations, Cycles Average, Iter Time Average, iterations/Second
dc zva - memset 0, 12065162, 49, 246227.80, 12.82 millisecond, 77.98
dc zva - vzeroupper, 12098598, 4325376, 2.80, 145.68 nanosecond, 6864201.89
dc zva - vzeroall, 12031459, 4325376, 2.78, 144.87 nanosecond, 6902506.11
dc zva (stp emu) - vzeroupper, 13899441, 363855872, 0.04, 1.99 nanosecond, 502612496.60
dc zva (stp emu) - vzeroall, 12389283, 161742848, 0.08, 3.99 nanosecond, 250657175.37
```
We are actually quite close to a single page of CPU state per thread and
any additional changes are likely to cause it to overflow which would
hit these asserts. As we saw with the libc++ implementation of mutexes,
just one object type changing size could push it over the edge.
Future proof this by ensuring we can have this be sixteen pages per
thread before needing to hit more complex implementations. Which I don't
see us getting that large of CPU context tracking.
Implicitly adding "-c" breaks argument passing for scripts. For example, the
command "FEXBash ./steam.sh -silent" will process steam.sh but the script
wouldn't see the "-silent" argument previously.
coredump applications aren't smart enough to only dump resident pages,
so explicitly mark our 128TB and other mapped VA ranges as DONTDUMP.
This will speed up coredumps.
To not have this environment variable accidently be enabled on arm64
native Wine games, we need to set it from inside of FEX.
Requires the FEX dlls to set them directly rather than launch scripts.
EL0 registers are readable without going through the registry, but we
were failing to populate this register, which was causing clzero to not
be supported.
building with clang/musl causes these warnings:
```
FEX/Source/Tools/FEXServer/ProcessPipe.cpp:100:96: warning: format specifies type 'ssize_t' (aka 'long') but the argument has type 'rlim_t' (aka 'unsigned long long') [-Wformat]
```
- cast platform specific MaxFDs members to uintmax_t and print as PRIuMAX
- use %zu for GetNumFilesOpen (size_t)
fix: #5471
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
build on clang/musl fails with:
```
FEX/unittests/APITests/Allocator.cpp:17:5: error: use of undeclared identifier 'close'
FEX/unittests/APITests/Allocator.cpp:23:5: error: use of undeclared identifier 'lseek'; did you mean 'fseek'?
FEX/unittests/APITests/Allocator.cpp:23:11: error: cannot initialize a parameter of type 'FILE *' (aka 'struct _IO_FILE *') with an lvalue of type 'int'
FEX/unittests/APITests/Allocator.cpp:24:5: error: use of undeclared identifier 'write'; did you mean '_IO_cookie_io_functions_t::write'?
FEX/unittests/APITests/Allocator.cpp:24:5: error: invalid use of non-static data member 'write'
```
include header to provide defintions
fix: #5476
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
building using libc++ failes due to shared_mutex ObjectCacheRefCounter
inflating InternalThreadState beyond FEX_PAGE_SIZE, thus triggering
the static assert:
```
FEXCore/Debug/InternalThreadState.h:133:15: error: static assertion failed
FEX/FEXCore/include/FEXCore/Debug/InternalThreadState.h:133:145: note: expression evaluates to '7680 < 4096'
FEX/FEXCore/include/FEXCore/Debug/InternalThreadState.h:136:58: note: expression evaluates to '12288 == 8192'
```
remove `ObjectCacheRefCounter` as it is not used anywhere.
fix: #5456
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
musl build fails due to access to internal glibc member:
```
FEX/Source/Tools/LinuxEmulation/LinuxSyscalls/SignalDelegator.cpp:655:39: error: no member named '__val' in '__sigset_t'
655 | .SigMask = _context->uc_sigmask.__val[0],
| ~~~~~~~~~~~~~~~~~~~~ ^
1 error generated.
```
add check to determine private glibc or musl member name or throw an error.
fixes: #5461
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
building on clang/musl fails with:
```
FEX/Source/Tools/LinuxEmulation/LinuxSyscalls/GdbServer.cpp:1174:5: error: use of undeclared identifier 'tgkill'
1174 | tgkill(::getpid(), ::getpid(), SIGKILL);
| ^~~~~~
1 error generated.
```
include and use `FHU::Syscalls::tgkill`.
fixes: #5463
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
using <sys/prctl.h> and <linux/prctl.h> simultaneously causes clang to
fail:
```
In file included from FEX/FEXCore/Source/Utils/AllocatorHooks.cpp:6:
/usr/include/sys/prctl.h:88:8: error: redefinition of 'prctl_mm_map'
88 | struct prctl_mm_map {
| ^
/usr/include/linux/prctl.h:134:8: note: previous definition is here
134 | struct prctl_mm_map {
| ^
1 error generated.
```
prefer <sys/prctl.h> and do not include <linux/prctl.h>
fix: #5454
Signed-off-by: Pepper Gray <hello@peppergray.xyz>
In portable mode, FEXOfflineCompiler may not be in PATH (and if it is, it's
most likely not a compatible version). Instead, use the executable next to
the FEXServer binary.
Both the default F80 softfloat wrappers (FXTRACT_SIG/FXTRACT_EXP) and
the reduced-precision F64 dispatcher fell through to the generic
exponent/significand extraction for Inf and NaN inputs, producing
finite garbage. The F80 wrapper returns input unchanged in the
significand slot and +Inf (or NaN) in the exponent slot; the F64
dispatcher detects the exponent-all-ones case and selects the proper
Inf/NaN result before the existing zero-case fold.
The lhs==0 short-circuit in X80SoftFloat::FSCALE returned lhs
unchanged without calling extF80_mul, so the 0*Inf invalid-operation
case never set softfloat_flag_invalid. Detect +Inf rhs explicitly
in the zero-lhs path and raise the flag, returning QNaN to match
hardware.
The special-value check masked the biased exponent with 0x7fff and
then used TestNZ, so it triggered on Exp==0 (denormals) instead of
Exp==0x7fff (NaN/Inf). Replace the TestNZ with SubWithFlags against
0x7fff so EQ only fires for genuine NaN/Inf.
fcmgt returns false on NaN, so the existing polarity in the non-SVE
fcmgt+bit sequences and in the SVE predicate-merge picked the wrong
source on NaN/tie. Swap the compare operands and flip bit<->bif / add
a predicate not to match x86 second source wins behaviour.
The previous check site would easily fail when loading caches for binaries
with multiple executable sections.
It makes much more sense to refuse generating caches anyway: The condition
effectively checked for invalid code map entries, so FEXOfflineCompiler
should reject them as bad inputs.
Noticed this while benchmarking that the FIST* operations were
converting to a GPR, and then storing to memory using an atomic TSO
operation. This should be instead listening to the vector TSO
configuration option. This gives a 3.8x - 6.05x improvement in my
microbench.
Additionally when possible, make sure to use vector conversion
instructions when possible. It's lower cost to avoid the FPR->GPR
transfer, but we can only use it for 64-bit FIST operations. Microbench
couldn't show a difference for that on my platform, but that's because
it's float pipeline bounded regardless. Should help X-class Cortex and
newer Cortex-A.
Only ensures we are pushing and popping the correct size, not any of the
selector data within it, as 64-bit systems with the FSGSBase extension
don't use them selectors anyway.
Can't test the 32-bit side currently because we would corrupt FS/GS in
CI and the host testharnessrunner can't fix that right now.
Graphics provider needs to be a path to a json file in the root of the
rootfs. Make sure to strip the filepath off to get the directory.
Misunderstood the assignment before.
- RNDRRS is still broken on this CPU
- Fault granularity checking needs to use loads
- stlxp does monitor check before alignment check, use loads for all
for consistency
- Fault granularity is still 16B which matches LSE2 requirements.
- X2E still only ships a 19.2Mhz cycle counter, so still only
ARMv9.0-a hardware
- Adds product name to CPUID
- Oryon-3 being CPU PartID 2 isn't a mistake.
- No distinction between Oryon-1 and Oryon-2, both are partid 1.
cpuinfo:
```
processor : 0
BogoMIPS : 38.40
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm uscat ilrcpc flagm ssbs sb paca pacg dcpodp sve2 sveaes svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 rng ecv afp rpres
CPU implementer : 0x51
CPU architecture: 8
CPU variant : 0x1
CPU part : 0x002
CPU revision : 1
```
We had a bug where nop encoded prefetch instructions were getting
flagged as illegal instructions erroneously. Fix that and add a unittest
for ensuring execution.
Fixes `Devil May Cry 4`
Docker's seccomp filter fails to follow AAPCS64 and SysV zero-extension
rules. For values smaller than 64-bit they were required in their
seccomp filters to truncate the value to the specific size but do not.
Instead they do a 64-bit comparison operation against smaller arguments
(in this case 32-bit). This means 64-bit -1 and 32-bit -1 passed through
have different values for this `personality` syscall.
The real fix would be for Docker to audit their seccomp filter rules and
ensure they zero-extend every argument that is smaller than 64-bit, but
we don't control that. So there is likely to be more bugs in their
filter that we encounter, this is just an easy one to resolve.
This was causing a surprisingly high amount of branch mispredicts in
Death Stranding. Suspend doorbell is fairly rare so just invert the
check and move the target down out of the hot path. Then the doorbell
handling code will trampoline to the correct location still.
Death Stranding 2 is using this instruction instead of `vbroadcastss`
for some reason. Optimize its specific case and add a note that when we
know sources match that we can optimize more patterns easily.
Allows WTF to work (mostly) with Wine by letting us VirtualName things,
and also allows madvise control of THP, which significantly cuts back
memory usage.
This works around the problem of Wine not giving us control of this by
using raw syscalls when wine is detected.
Based on top of #5362 so the THP disable controls are in.
This wasn't quite wired up exactly how we wanted it. It was previously
matching against the opaque file config handle, which can be anything.
Instead compare it to the appname that now gets passed over to it for
matching.
This allows us to do the following:
```
{
"Config": {
"ProfileStats": "1",
"X87ReducedPrecision": "1",
"TSOEnabled": "1",
"VectorTSOEnabled": "0",
"MemcpySetTSOEnabled": "0",
"HalfBarrierTSOEnabled":"1",
"MaxInst": "500",
"Multiblock": "1"
},
"AppOverrides" : {
"setup*" : {
"Comment": [
"292030 - The Witcher 3: Wild Hunt"
],
"X87ReducedPrecision": "0"
}
}
}
```
Based on #121 which needs to get merged first.
Code Review
Code Review: Class deletion
Add support for question mark and plus mark in regex, supply testing for star
Added more characters to the regex alphabets, add more test case
Added support for regex matching of configs, awaiting reviews
Rename variable to CamelCase
Addresses PR reviews
Remove unnecessary features and test cases
Rewrite to naive regex with dp
Addresses PR reviews
Build fixes
Code Review
The previous `ForkableSharedMutex` using `std::shared_mutex` was showing
up as significant CPU time on arm64ec. In particular it was showing up
upwards of 700ms/S of CPU time for read-contented workloads at only
~2700 locks per second. The libc++ implementation for arm64ec must be
particularly gnarly for this to be so slow.
With this swapped over, it's now only spending around 56ms/S on the
contended shared lock, but at ~8000 locks per second. So a significant
uplift.
Don't use pread, instead map the file and madvise larger blocks. This
removes copying overhead as its just mapping file pages in instead.
Also splits the implementation of file reading from hashing to make
tinkering less involved, as if I want more performance out of this (say
due to live hashing) then it's easier to tinker.
Improves hashing performance from ~2.2GB/s to ~3.6GB/s on my system,
which is CPU bounded by xxhash here.
I utilize this functionality quite heavily when debugging and I need
bread crumbs spread around. Instead of reimplementing it a dozen times,
just have it upstreamed.
This was causing an unfortunate set of circumstances where Dark Souls
III was modifying MXCSR and we weren't saving it, cause the value to change
from 0x9fc0 to 0.
This "enabled" float exceptions by unmasking the exception masks in
MXCSR. This in turn had Dark Souls III's `expf` function to fault out,
as it checks if the MXCSR exception masks are set or not for determining
if underflow should assert or not.
Wow64/arm64ec has a similar problem where it always sets back to default
on signal. Which means game lose DAZ, but I'm not fixing that bug right
now.
Fixes#5391
ELF headers were read unconditionally because doing so was assumed to be cheap
(as the guest app would read them anyway shortly after). However, relocation
parsing was added since then, which has less predictable performance due to
crossing page boundaries and reading larger amounts of memory. It might be
possible to make the underlying code more efficient, but until that's done
it's better to skip this logic unless needed.
Closes#5390
We needed this handling on old kernels that didn't understand the
NOREPLACE flag. We no longer support kernels this old, so remove some of
this vestigial code.
People keep forgetting to set this and have a worse experience.
Default to a Release build, which ensures optimizations are enabled and
assertions are disabled.
Disables THP on some key locations that are fairly sparse
- rpmalloc
- This is the big one as this allocates some heavy sparse buffers.
- CallRet stacks
- These get in the hundreds of megabytes, while not being sparse they
trend towards only using a handful of pages and ballooning to 2MB
per thread is quite heavy.
- Lookup cache
- L1 specifically gets hit here which adds a decent chunk of overhead
due to sparsity.
Win32 for all of these also aren't handled, but that will need to be a
followup.
Dramatically reduces memory consumption of FEX's per-thread lookup
structures. Primarily because L2 cache entirely goes away which can end
up reaching hundreds of megabytes or over a gigabyte of memory in some
cases, but also because L1 cache dynamically scales based on load.
Useful for conserving memory on systems with less than 16GB of RAM and
are UMA, like Asahi users inside of muvm.
While not a leak in the traditional sense, we were causing pool
allocations to never become free until the thread was closed.
This meant in the case of a game running with >200 threads or so, these
would add up very quickly. So some minor reworking so the IREmitter
doesn't allocate a buffer until first JIT, and making sure to actually
disown the buffer on dispatch error resolved the problems.
Fixes an edge case where Ender Lilies was consuming 409MB with THP
enabled on my desktop, and now it is something like 6MB once idling for
a bit to have the pool allocations do its magic.
Add a no-op for zero length in ChangeProtectionFlags.
This fixes AMD Vivado 2025.2 which tries to mprotect with 0 as size and merge strategies fails:
```
Unexpected ChangeProtectionFlags Merge strategy! [0x400000, 0x401000) Versus [0x0, 0x0)
```
Fixes b4a87d8c0b
`FEX_APP_CONFIG_LOCATION` wasn't overriding paths properly anymore once
that commit landed. Instead legacy `~/.fex-emu/` path would get returned
if it existed first.
Ensures that it returns first, before `STEAM_COMPAT_DATA_PATH` even.
If/When this works, the amount of iTLB misses drop dramatically, which
reduces L2 TLB pressure, which just improves performance for our CPU
cores that have itty-bitty L1 iTLB entry counts.
We can't use `mmap(MAP_HUGETLB)` directly for terrible reasons, so we
are required to lean on madvise instead.
Totally forgot to update this when I opened that PR. oops!
FYI I would recommend merge-committing if I PR to other repos since it
preserves my GPG signature
Signed-off-by: crueter <crueter@eden-emu.dev>
We were passing through 32-bit signed values to the 64-bit handler,
which isn't valid and we need to make sure to sign extend.
I don't think this fixes anything because negative values aren't valid,
but a negative could have been interpreted as a large positive without
this.
So WTF can see their usage. Previously didn't add this due to some weird
madvise bug that delete anon names, but that seems to be fixed in a
newer kernel.
Fixes#5230
Basically just stores it as an array of `uint8_t`, and automatically
pads it out to 24 bytes. Hopefully FEX never reaches SHA1 collisions so
this should (TM) never collide.
Note: I have no idea how fmt will handle that format string with an
array of uint8_t.
Signed-off-by: crueter <crueter@eden-emu.dev>
The AMD documentation about this instruction is very vague and
misleading in multiple ways. While the Intel documentation is much
cleaner and explains how we need to implement these.
8 of these "new" operations are just inverted signaling versions of the
original 8 SSE versions.
The remaining 16 new operations fill gaps in the original x86 version of
the instructions, exposing the 5 bit truth tables directly, which is why
we also have a "true" and "false" version as well.
Both scalar and vector wide.
Fixes#5326
Presumably this was done as a hack to highlight the line in red when rendering
documentation to markdown. Since it's only used for two instructions, drop this
use to ease generation of C++ docstrings.
For the upper-half of the registers it is more efficient to zero the
context with `dc zva` on Ampere1A hardware, while Cortex implements this
as equivalent uops in their store pipeline and aren't affected one way
or the other. ARM C1-Pro and newer with FEAT_MOPS also match `dc zva`
performance with 64B/c, but theoretically slightly fewer instructions.
C1-Nano on the other hand, clearly loses to `dc zva`, where mops can
only do 16B/c, but `dc zva` does 64B/c. So we'll need to benchmark or
not if MOPS is a clear win once hardware is actually shipping.
The current CodeBuffer regrowth code discards any existing contents. This is
undesirable with code caching, since those contents can't be re-fetched from
the disk cache and instead need to be recompiled at runtime.
Additionally, FEXOfflineCompiler obviously should never discard compiled code.
Using a large enough CodeBuffer right away reduces the likelihood that FEX
runs into such scenarios.
When thunks are jumping out, games are jumping /entirely/ out of their
controlled code, which means we don't need to save and restore
NZCV,PF,AF.
Some CPUs don't fully rename direct accesses to this register which adds
up during thunking. FPCR is also in the same situation where it'll force
pipeline flushes and isn't renamed away, but we can't really avoid that.
Improves performance at least in Detroit: Become human where the game
spends ~48% CPU time inside of the thunk trampoline for
`vkUpdateDescriptorSets`.
I plan on a follow-up PR where I converge all these options into a
struct argument instead, but that's a follow-up since I don't want to
burn a bunch of time right now.
No need for individual calls here. While doing this I also realized
xxhash has this at one point, I guess I'll PR that there later
Signed-off-by: crueter <crueter@eden-emu.dev>
Uses ALIAS targets to propagate include directories to targets rather
than polluting the preprocessor with an extra include directory it
usually doesn't need.
Signed-off-by: crueter <crueter@eden-emu.dev>
rpmalloc is currently very aggressively configured which causes
significant reductions in resident memory over jemalloc.
In Bayonetta's title screen it went from 963MB down to 834MB resident.
Turns out casting a vector of data to a string is a bad idea, who knew?!
This bug was exposed by the rpmalloc PR because the
`/run/host/container-manager` file isn't null terminated. So the cast
fextl::string through std::vector::data had the /potential/ to not be
null terminated.
This was just a bug hiding in the code, convert it over to use
fextl::string directly and avoid the entire dance.
Fixes#5236
Should just work (TM). Because unordered_dense is in my system include
dir I actually can't test comp with external (since it finds the right
include file anyways), so plez test on systems w/o unordered dense
Signed-off-by: crueter <crueter@eden-emu.dev>
Otherwise we hit an assert in FEXCore backend with code discovery
hitting things that look like AVX.
Fixes a crash in Uplay.
Also adds a test to just ensure that the instruction faults out and is
captured instead of crashing in FEX itself.
This breaks relocations currently due to not handling negatives and also
an interesting overwriting problem.
Not that big of a deal, it's only a minor optimization anyway.
Fixes#5227
Mesa uses this to determine what the executable name is in a process for
application profiles. Without thunking, the guest glibc sets this
correctly, with thunking mesa would pick up `FEX` instead of the app.
With this fixed, it fixes Dead Island rendering when thunks are enabled.
Plenty of other games in Mesa's application profiles that it would fix
as well.
This is a fairly trivial test to ensure that signals correlate to their
expected order. I keep getting spooked that signal numbers on x86 don't
always correlate to signal numbers on other architectures. So slam
through all 64 signals to ensure they are correct.
On an architecture that doesn't match signals 1:1, or an emulator that
doesn't properly remap when that occurs, then the order would be
incorrect.
As requested by neobrain. Makes log parsing easier because you can see
the "noisy" and usually unchanging targets in one step
(which usually succeed, I hope), and the more error-prone "the rest of FEX"
build in separate steps. Makes parsing FEX build errors easier
Signed-off-by: crueter <crueter@eden-emu.dev>
- Makes all scripts POSIX-compliant
- General formatting and such
- shellcheck :)
Shell script is love. Shell script is life.
Signed-off-by: crueter <crueter@eden-emu.dev>
When an anonymous FD is passed to execveat that has the CLOEXEC flag
set, then binfmt_misc fails with ENOENT.
Workaround this limitation by duplicating the FD, stripping its CLOEXEC
flag in the process.
For #5234
6 steps use this verbatim, and steamrt4 does as well (minus rootfs
stuff). Less LoC and whatnot, you get the drill
Signed-off-by: crueter <crueter@eden-emu.dev>
On 36-bit VA systems the stack was ending up /wherever/ when it should
be at the top of the VA space (usually).
Additionally VDSO was getting mapped anywhere on 64-bit, so push that to
the top of the VA space as well.
Also removes a check for old kernels not supporting MAP_FIXED_NOREPLACE.
This was a nasty race condition where each thread could be accessing the
DRM cache at any given moment. Move it over to a per thread object that
is only allocated once it gets used.
Also removes an `atexit` registration that contributes to crashing on
exit.
Checkout and upload-artifact. They contain general bug fixes and update
the Node runtime such that they should be generally faster. Note that
this requires Node.js 24, if that's a dealbreaker for the runners feel
free to close.
Signed-off-by: crueter <crueter@eden-emu.dev>
Significantly reduces the constant duplication of test run steps. Port
of #5246, but differs in that the composite action now truncates the log
file in-place.
CC: This is rebased on top of #5252 because I do NOT want to deal with
these merge conflicts again.
Signed-off-by: crueter <crueter@eden-emu.dev>
CMake already handles the mkdir step implicitly with `-B build`. Same
with build type, where setting `CMAKE_BUILD_TYPE` will propagate it as
such to the build commands. The only exception to this is the MSBuild
generator, which we don't use here.
Signed-off-by: crueter <crueter@eden-emu.dev>
Much like the prior xxhash PR. This one also has the advantage of making
it completely trivial to handle distros that only install zy{dis,core}-config.cmake
and not the PkgConfig files (Gentoo, Arch)
Working on both Gentoo and Arch (CMake), and Fedora (PkgConfig). Does
work on Ubuntu 22.04 though it gets rejected due to being too old, since
we require 4.0 or newer.
Signed-off-by: crueter <crueter@eden-emu.dev>
Uses `git --no-pager diff --exit-code HEAD` instead of the two-step check used before. Also shows the diff if it exists so you can see changes at a quick glance.
Signed-off-by: crueter <crueter@eden-emu.dev>
Bash is already the default on Linux runners.
Just a bit cleaner and less lines. Also cleaned out some unneeded
comments from the template.
Signed-off-by: crueter <crueter@eden-emu.dev>
Vixl has migrated its upstream repo to https://gitlab.arm.com/runtimes/vixl
without telling anyone. Rebase on their current main rather than the
dead project.
At the very least removes the obnoxious `operator""_h` warnings. Not too
much has changed since our last rebased.
Fixes#5237
Previously, FEX would use `$HOME/.fex-emu` for its data and config if
`XDG_CONFIG_HOME` and/or `XDG_DATA_HOME` were unset. This doesn't follow
XDG, so instead we do a fallback to `$HOME/.config` and
`$HOME/.local/share` respectively if XDG env vars are unset. Also,
pre-emptively creates those directories since `~/.local/share` and
`~/.config` existing is technically not a guarantee if XDG dirs are unset.
Signed-off-by: crueter <crueter@eden-emu.dev>
ankerl::unordered_dense is faster on average and has less memory usage
than tsl::robin_map. It is pretty significantly faster than std but
we'll keep that as is for now.
Obviously, this will need a lot of testing.
Signed-off-by: crueter <crueter@eden-emu.dev>
These three objects are easy enough to move out of the static namespace.
They were causing atexit registrations for deallocating memory.
Removes three `atexit` registrations that contributes to crashing on
exit.
Integrate Zydis as an optional dependency to enable x86/x86-64 guest
instruction disassembly during JIT compilation.
Build with -DENABLE_ZYDIS=TRUE.
Use FEX_X86DISASSEMBLE=1 at runtime to output guest x86 instructions
for each compiled block.
Applications can use this to ensure they've grabbed the whole XSTATE
correctly. UML uses it to determine if the FPState was correctly saved.
Also adds a unittest to ensure the same correct behaviour.
For #5206
Turns out the Linux kernel's definition of `MINSIGSTKSZ` and glibc's
definition of `MINSIGSTKSZ` don't match.
The linux kernel has a massive comment about it in `arch/x86/kernel/signal.c`.
Also adds a unittest for it.
For #5206
Previously, changes to externals were also tracked, resulting in
absolutely hilarious abominations like https://github.com/FEX-Emu/FEX/actions/runs/20498228017/job/58901276463?pr=5162
So rather than dealing with that weird action, just fetch the main
branch and run a `diff --name-only` with the PR's merge base with main.
This is how I've done several dozen diff-based scripts (license headers,
clang-format, etc) and it works perfectly, so let's just use this.
Should run quicker too but don't put any money on it.
I also went ahead and removed a bunch of unnecessary quotations cuz
wynaut.
Signed-off-by: crueter <crueter@eden-emu.dev>
This is fundamentally a frontend only problem, and also Linux only.
Moves it to the frontend where it belongs.
There's likely more things in Allocator.cpp that can be moved to the
frontend but this is the first thing.
NFC
UML disables ASLR using a personality. FEX in the ELFCodeLoader was
accidentally erasing personalities. This caused UML to do a bootloop
trying to repeat disable ASLR.
It now gets ASLR disabled, but fails shortly afterwards.
For #5206
eb95a959bc (#5171) did not keep the interface
file in sync with the output of DefinitionExtract.py. Some definitions were
reordered in the Vulkan headers.
There also is no "conflict" about CUDA extensions; instead, the previous
update didn't reflect that the new headers fully hide beta extensions.
There's no longer a distinction between AArch64 and x86 and everything
effectively falls under "Common" now. This means flattening the entire
structure just cleans it up.
NFC. (Although instcountCI will update because of a couple pointer
offsets changing)
Fixes crash in thunks that use callbacks, introduced in #5148.
The dispatcher would call the syscallhandler to get the VDSO thunk
callback. But due to reordering initialization, the VDSO thunk would
have not been loaded at that point. This would cause thunks that use
callbacks to crash with a nullptr exception.
Instead, defer the thunk callback pointer loading until the thread
starts executing, and load the pointer in to our thread state's pointer
struct instead.
Didn't get caught in my initial test sweep since I didn't run a Wine
game with thunks.
This wasn't handling negatives correctly which was causing xalia.exe to
assert. Disable for now rather than further changing logic, with a TODO that it
should get fixed in the future.
Steam started taking advantage of newer fcntl commands, in particular
`F_CREATED_QUERY`. This was causing our fcntl handler for 32-bit
processes to assert out.
Fixes the handler so it passes all other fcntl commands forward, just
like the Linux kernel does. Splits the 32-bit handler to explicitly
ignore a handful of commands just like the Linux kernel.
Fixes an assert that Steam was hitting.
Prevent suspension when flushing thread state to the WOW64 context
during BTCpuSetContext as suspension itself performs a flush. If
suspension were to occur halfway through FlushThreadStateContext then
the fact that the WOW64 context has been dirtied would be missed and the
new context would be partially overwritten with stale data from the FEX
thread state. Additionally if the context were to be dirtied by
suspension at any point during context merging in BTCpuSetContext the
input context could be lost.
Fixes occasional crashes when starting AC4.
Handling this safely requires blocking all compilation for the duration
of the read, as fault-based tracking doesn't work when wine's unix side
itself is the one performing the write into the RWX region (we cannot
catch unix faults).
Fixes a startup crash in Persona 5.
We have run out of EmulatorData slots on ARM64EC, so introduce this for
less-frequently accessed TLS data for which the additional cost doesn't
matter.
Execstack enables `READ_IMPLIES_EXEC` and we were accidentally
decoupling our expected persona due to the reordering of initialization.
This was causing the smc-1-dynamic test to fail because it's testing
execstack and our VMA tracking wasn't returning read-write VMAs as
executable once these decoupled.
Easy fix.
This allows the context and parent thread objects to be created earlier,
allowing the VDSO and ELFCodeLoader mapping functions to have a thread
object for tracking memory mappings through the regular guest routines.
This means that we no longer need to do any form of deferred handling
for code caching as all the state is ready early in the initialization
process.
A little bit of care needed to be taken to ensure we still close the
ELFCodeLoader's FDs later and that VDSO unmapping happens before tearing
down the parent thread, but overall this is mostly just passing the
InternalThreadState object around as normal.
I couldn't find any functional regression from this change alongside
code caching, but it would be good for @neobrain to double check this.
FEX argument will no longer exist in `/proc/<pid>/cmdline` after this
PR. Ensure this tool still works.
After #5097 is merged, one can use regular `pidof` for Linux
applications, but this is still useful as it finds Wine applications
running with FEX as well.
Theoretically, another thread could run after the reprotection but
before the invalidation, write some code and jump to it but end up
jumping to old code as the invalidation is yet to occur. Fix this by
locking the invalidation mutex, invalidating and only then reprotecting.
Opcode handlers are written with the assumption that LoadSource will not
touch flags and this would be an annoying assumption to change. As this
is such an edge case anyway just don't defer flags and force a load of
the saved value before _TelemetrySetValue (which are implicitly saved
before it).
Fixes the following snippet in upc.exe:
AND word ptr [ESP + ECX*0x1 + 0x80000000],DX
BTR CX,DX
ADC CX,word ptr SS:[EAX + ECX*0x1 + 0x80000000]
Windows will populate Size/Address with those of the underlying free
region. This is important for cross-process free operations, where the
the 'before' callback is still called after the memory is freed so
querying the size of the section (which is not freed) fails.
For now, a simple directory structure with filename based mapping is
used:
LOCALAPPDATA/cache/<main image id>/{<dll 1 image id>, <dll 2 image id>}
All cache blobs are loaded once at the start for simplicity, this will
also be useful in the future as metadata validation for settings etc can
be done just once.
Before compiler/architecture checks, we check:
- `sizeof(void*) == 8`: 64-bit systems have an 8-byte void pointer,
whereas 32-bit systems (should) have a 4-byte pointer; since 32-bit
hosts are completely unsupported might as well check for it just in
case
- Only Windows and Linux are supported, so let's add an early check for
that as well.
Signed-off-by: crueter <crueter@eden-emu.dev>
I may have gotten carried away.
- I missed some stuff for end parenthesis because I accidentally
searched within project files instead of the entire directory (so some
thunk/test/windows stuff was missed), cleaned those up.
- `INTERFACE`, `PUBLIC`, `PRIVATE`, `RUNTIME`, `LIBRARY` should be on
the same line as the target name. (I should really invest in making a
style guide...)
- Some short statements were unnecessarily split across multiple
lines--cleaned those up
- Made a common `LinkerGC` module that applies gc-sections etc. to a
target in Release mode
- Usually for functions you want to have something on the first line,
e.g. `FILES`/`DIRECTORY` for install, or the target/a positional
argument, etc etc. Not always though, notably for some custom_command
calls
TODO:
- What's with the `list(APPEND LIBS...)` stuff? It's used really
inconsistently, sometimes not at all, sometimes it looks like there're
duplicates? A more thorough cleanup is in order there.
Signed-off-by: crueter <crueter@eden-emu.dev>
Most constants don't need to be padded for relocations. So now that
these have all been audited, switch to defaulting to NoPad to reduce
verbosity.
The number of constant that need to be explicitly padded are now marked
and with all the prior changes, this allows bisecting if something has
gone wrong.
- Do compiler/architecture checks EARLY, don't waste time doing random
configuration stuff if the user can't even compile in the first place
- MSVC is unsupported, I assume? So add a check to disallow. There's
literally no MSVC or MSC_VER checks anywhere, so...
- Rather than using the MSVC architecture definitions, use our own
`ARCHITECTURE_arm64` et al. Hijacking existing "standard" definitions
is a very bad idea. Also makes it more readable in CMake
- Change the x86 host check to `x86|amd64`. Some systems still refer to
themselves as x86 despite being 64-bit for... reasons, and I saw one a
very long time ago that referred to it as amd64. This should
basically never come up, nor is it really relevant given that FEX is
for arm64... but it kinda annoyed me so whatever.
TODOs:
- Should we check `CMAKE_SIZEOF_VOID_P (equal) 64`? I don't think anyone
is even trying to compile this thing on armv7 or older, but might as
well? maybe?
- What's the status of *BSD, Solaris, macOS? Technically macOS does
support Wine, not sure about the others.
Signed-off-by: crueter <crueter@eden-emu.dev>
Just my brain spinning as I try and determine what is causing some
hanging. Seems to be WINE specific so might not even be in FEX code.
Good to have some more documentation so when I read this again I don't
need to make some more logic deductions.
Port of #5159
Find modules are preferred for pkgconfig/otherwise non-CMake libraries.
Let's use that here for xxHash.
Signed-off-by: crueter <crueter@eden-emu.dev>
Trivial variables like SRCS, NAME, etc. actually do more harm than good.
They *will* make your IDE mad, and are also less readable. Remember:
verbosity is not a bad thing! Usually
Also: did a few tiny cleanups that I missed from my `endpara` PR.
Signed-off-by: crueter <crueter@eden-emu.dev>
Command names shouldn't be uppercase. A lot of LSPs scold you if you do
this, and apparently it's the official recommendation from the CMake
team to absolutely never, ever do this.
Signed-off-by: crueter <crueter@eden-emu.dev>
The Race:
1. A Reader sets `READ_WAITER_BIT` (Bit 15) and sleeps on the High 16 bits (`Futex+2`).
2. Writer A unlocks. It clears `READ_WAITER_BIT` (in Low 16 bits) and `WRITE_OWNED` (in High 16 bits).
3. Writer B immediately steals the lock. It sets `WRITE_OWNED` but preserves the now-cleared `READ_WAITER_BIT`.
4. The Reader, checking `Futex+2`, sees `WRITE_OWNED` is set. Since it cannot see that Bit 15 was unset (as it is watching High 16 bits), it assumes its wait signal is still valid and sleeps.
5. Writer B unlocks. It sees no `READ_WAITER_BIT` and wakes nobody. Deadlock.
The Fix:
Move `READ_WAITER_BIT` to Bit 30 (High 16 bits).
Now, when Writer A clears the flag, the High 16 bits change value which will prevent the wait from occurring within WaitForAddress
- Some CMake LSPs have aneurysms when you put the end parenthesis on a
different line. Annoying? Yes, but this is all we can really do about
it for now.
- `set`, `option`, and `message` should not have spaces before their
opening parenthesis.
Signed-off-by: crueter <crueter@eden-emu.dev>
- `AddObject` doesn't need an additional Type parameter since it's
already assumed to be an object library
- The static and shared libraries don't need any explicit compile
options, as the object already handled this.
- They also don't need to be linked to FEXCore_Base. Object libraries
already handle that for us since the symbols already get pulled in
anyways.
- The object library doesn't need an output name. Only the user-facing
libraries do
Signed-off-by: crueter <crueter@eden-emu.dev>
Rather than individually adding `${CMAKE_SOURCE_DIR}/Source` as
an include directory to each target, just use `include_directories` once
in the Tools directory and each subsequent target will have this
propagated down.
Also removed a seemingly unnecessary `-I` in LinuxEmulation--maybe
needed? But I can't test compilation right now as I don't have an ARM
development environment on hand for the next day or two.
Signed-off-by: crueter <crueter@eden-emu.dev>
CMake has had the `MINGW` builtin to describe MinGW targets since at
least version 3.2, so it can safely be used. This variable is also set
for the MSYS2 environments, so CLANGARM64 also correctly sets `MINGW`.
Note that this depends on https://github.com/FEX-Emu/jemalloc/pull/11.
Signed-off-by: crueter <crueter@eden-emu.dev>
Windows caches can only really work safely if they are exclusive to a
prefix due to the state for things like sharing modes that is tied to
the per-prefix wineserver and codemaps encoding prefix-local paths for
system DLLs.
Recent CPUs do nop fusion with the following instruction, this gives the
CPU the best chance to do fusion with something that actually does work.
Very trivial, doesn't do this for the more complex handling below these
as counting the number of moves before nop emitting is messy.
Instead of just a trivial pad being on or off, support a tri-state
on/off/auto where on will always pad, off will never pad, and auto will
pad only if code caching is enabled.
Further augment this by allowing a byte-width to be passed in, which can
be used with pointers to force a 48-bit VA width to only ever pad to
three instructions, reducing the common worst-case situation from 4
instructions to 3. This works because we're not going to expose a VA
width larger than 47-bit to the guest.
Fixes the handful of use-cases that explicitly chose their NOP padding,
and a bug in Arm64Relocations.cpp where it was incorrectly asking to not
receive padding even though it requires it.
and use it to commonise volatile metadata code (adding WOW64 image
voltmd support in the process). This will be extended to track mapped
images for code caching.
When relocations are loaded, all immediates read from memory are checked
against the relocation map and transformed into an appropriately
sign-extended entrypoint-relative variant of the specific operands
addressing mode. As almost every case of an unhandled relocation will
lead to a later, likely harder to debug, crash at runtime just bail out
early if any such cases are encountered. Note that while this
handles/detects all cases of relocated immediates, if relocations were
applied to instructions themselves (occurs in some malware variants)
these would be missed without any errors reported.
When compiling code at runtime there is no harm to including jumps to
different sections within a multiblock, when enforcing as such would
introduce a lookup cost for every decode invocation (or some caching).
However when compiling offline as each cache blob is tied to a specific
library these boundaries should be enforced.
In order to support code caching of 32-bit libraries, any library-base
relative relocations on the guest must be transformed into FEX
relocations so e.g. absolute jumps or loads refer to the correct
location when the library is loaded at a different base address.
I noticed that we weren't actually testing 32-bit displacement on
64-bit. This is an edge case on 64-bit where you can have 32-bit
displacement WITHOUT it being RIP relative. This is very uncommonly used
but is technically a possibility where 64-bit code can access an
absolute address in the lower 2GB.
With negative addresses you could technically access canonical kernel
addresses in the lower 2GB if they were mapped in userspace, but since
they aren't this is untestable (FEX also doesn't handle canonical kernel
addresses correctly anyway).
`MemoryData.asm` accidentally tested this but it didn't test both load
and store sides. So let's be a bit more stringent on this.
When loading code caches, these constants get patched up for the new guest
address. The new value may be larger than the original, so the padding bytes
ensure the maximum of 16 bytes of encoding space is always available.
Closing the original stdout is how we signal to a larger compatibility
tool (in practice pressure-vessel) that either we are ready, or we have
crashed: when every process that held the write end open has closed it,
the compatibility tool sees EOF on the pipe. If the FEXServer inherits
stdout and holds it open, then that state will never be reached.
steamrt/tasks#870
Signed-off-by: Simon McVittie <smcv@collabora.com>
The ELF tracking thing has an expectation that only portions of ELF
files that are described in the program headers will be mapped
executable. This doesn't hold true as programs will remap random
portions of ELF files as executable. In the case that this occurs, don't
assert out and instead print a warning.
This was discovered as Node.js remaps a portion of itself executable
that isn't described as such in the program headers. I also have a local
unittest that exposes the same problem. I had discovered same problem in
some other program with #5038.
Also fixes a bug where sometimes completely anonymously mapped
executable sneak in and cause a crash, which is kind of silly.
Always enables FEXServer log thread when built for Steam so that clients
can be controlled with `STEAM_FEX_LOG=1`. FEX logs will then always go
to FEXServer and those can get directed to wherever pressure-vessel
chooses.
This is fairly simple. Needs to be installed alongside FEXServer, so
that it can start it in a portable config.
- Starts a FEXServer
- Tells pressure-vessel when FEXServer is ready
- Keeps FEXServer alive with the `watch_fd` as long as the process lives
- Listens for pressure-vessel to be shutting down
- Exits once pressure-vessel exits, also letting FEXServer shutdown if
no FEX instances are alive.
When the realpath of a program path can't be resolved, we weren't setting
the config option. This was cascading to be a crashing in codemaps where
it was unconditionally using the optional value (with assert checks),
and causing things to crash.
Pass in the path that can't be resolved to work around a crash in PV
that can happen.
This was a requested feature. To make sure that FEXServer is running and
managed by a parent process, we need to have a way to tell FEXServer to
keep alive without any FEX clients. The best way to do this is to pass
FEXServer a Pipe (like FEX does when a client starts it), but instead of
FEXServer signaling to FEXInterpreter that it's ready. FEXServer listens
to the pipe to see if the management process is still alive.
The expectation here is that the management process passes FEXServer the
read end of a pipe, and when the management software is done (or gets
killed by the kernel!) then the write end of the pipe is closed, and
FEXServer naturally closes (As long as there's no FEX processes
remaining).
Checks to see if the package is installed first before trying to use it.
Fixes an issue where fresh users don't have this package installed and
the script fails.
This will be useful to fix the mutex locking mess that occurs currently
when forks occur. Instead of needing to be /very/ meticulous with many
futexes, we can instead have working threads shared_lock this one, then
when a fork occurs just only have the forker themselves unique_lock and
let the readers drain out. Since it's write-priority it'll happen quite
quickly, letting the fork get in and out relatively easily.
This is going to take some massaging to get the frontend and FEXCore to
a place that this works but we can get this simple change in early.
Saves power and responds faster. Pass in the atomic to `WaitPred` with
the predicate checking if the buffer has been flushed yet. Same
behaviour as previous code but more efficient on our hardware.
This simplifies the loop a bit and moves the non-predicated exact
matching version to use the predicated version.
We will need a predicated version for the next commit.
Default to enabled because this is the config we expect by default.
In the future will get some benchmarking in various games like
Assassin's Creed, and Call of Duty.
Managing code maps in FEXServer rather than in FEXInterpreter makes it
easier to handle multiple concurrent processes sharing code caches for
the main executable and libraries.
A few games were generating "Can't handle adddress size".
I implemented 0x67 prefix handling for CMPSOp and SCASOP and improved
the error messages for the remainder. This will implement the address
modifier on 64bit systems, and keep issuing an error on 32bits.
Shared code buffer support introduced the concept of having a single
GuestToHostMaps shared across many threads. In the common case all
threads will share one however if e.g. a resize recently occured and
specific thread is yet to compile any code with the new codebuffer it
will still use the old GuestToHostMap. The current invalidation
approach handles this by repeatedly calling erase for every single
thread's GuestToHostMap, even if it is repeated. An accumulator is used
to ensure when two threads share a map, the L1/L2 cache entries in the
second thread will still be invalidated even if the the iteration for
the first thread removed them from the map.
Unfortunately this is incredibly slow in cases with many threads, as
a significant number of redundant map lookups and L1/L2 cache erasures
on threads that never even observed a given block can occur. Solve this
by introducing a two-pass model:
- First, all active codebuffers (and their associated GuestToHostMaps)
have their entries invalidated for the given range, these codebuffers
are tracked internally within FEXCore. It is at this point that delinking
callbacks are ran.
- Second, each thread will have its caches invalidated. But rather than
naively invalidating the L1/L2 caches for every invalidated block for
every thread, threads now track on their own what specific entries
have been potentially fetched into their L1/L2 caches. This is
aided by GuestToHostMap now tracking the pages each block touches. (an
inverse CodePages so to speak).
We currently rely on the frontend to keep track of threads and then
iterate over all threads to perform per-codebuffer operations. However
as codebuffers are shared between many threads (the common case is a
single code buffer across all) this ends up being inefficient. Introduce
a list of codebuffers to solve that (new codebuffers are very rare, so a
vector is plenty fine here for erasing invalid weak refs).
The sender might provide all requested message bytes but no FD. The receiver
interface has no simple way of indicating this scenario yet, so just assert
out for now to ensure it never happens in the first place.
If needed, this can be changed to return a new error code to indicate partial
read in the future.
This allows using read_some for incoming messages with an optional FD.
Doing so fits the purpose of read_some more closely, which is to read *any*
non-empty amount of data.
Adds it to the VDSO handling, it's not necessarily a VDSO function but
it behaves as such as it is in every single process. This means we get
to reuse the mapped page for every process when thunks are built,
shaving a page out of 32-bit processes.
Also, fixes a bug in guest VDSO symbol loading where clang sticks all
symbols in to `.dynsym` where gcc sticks them in to `.symtab`. Search
both. This effectively meant the couple of guest VDSO symbols were
always failing to get found, causing us to allocate yet another page on
32-bit. So effectively three pages stolen.
This also means we can remove the Linux specific X86HelperGen stuff from
FEXCore, only passing a single "VDSO" function pointer to the backend
for the dispatcher. Once again moving the Linux stuff to the frontend is
good.
Fixes an assert about about untracked noexec code `NoExec
instruction in entry block: FFFFE000` whenever thunk callbacks were
used.
Fixes a bug in guest VDSO symbol loading where clang sticks all
symbols in to .dynsym where gcc sticks them in to .symtab. Search
both. This effectively meant the couple of guest VDSO symbols were
always failing to get found, causing us to allocate yet another page on
32-bit. So effectively three pages stolen.
Peeled out of #5039
FEX_CONFIG_OPT can only be used as a standalone statement, which is
inconvenient for config values that are only used once. The new functions
(e.g. Get_DUMPIR()) can be used in conditions or other expressions.
Enables memcpy optimization of 80bit floats on reduced precision.
Also uncovered a bug where if we had done 80bit memcpy
optimization, we wouldn't have properly stored the 80bits.
This was caught by the existing tests when we enabled the optimization.
This is going to be necessary for the offline compiler work.
Also allow FEXGetConfig to print the same registers in the correct
format for easy fetching.
This allows using read_some for incoming messages with an optional FD.
Doing so fits the purpose of read_some more closely, which is to read *any*
non-empty amount of data.
Previously this was overriding the cached value in the local Value object.
The global SilentLog variable never got updated, so a stale value would be
used.
This is only ever used on the path where we'd want to handle something
like this (in EmitEntryPoint()), so we can just extend the long handler
type instead of introducing a new type to handle this.
At runtime, glxtest is mapped as follows:
0x000055fd9a030000 0x000055fd9a034000 0x4000 0x0 r--p glxtest
0x000055fd9a034000 0x000055fd9a038000 0x4000 0x3000 r-xp glxtest
0x000055fd9a038000 0x000055fd9a039000 0x1000 0x6000 rw-p glxtest
0x000055fd9a039000 0x000055fd9a03a000 0x1000 0x6000 rw-p glxtest
The problem here is that the last two sections can't be distinguished solely
by their mmap parameters. This would cause the wrong base address to be
inferred for the last mapping. To fix this, we can be more permissive by
allowing multiple candidates to be returned.
In practice, this only affects non-code sections, so it's not a big issue
either way.
This is the only usage of LSE atomics that isn't the fetch variety.
[This article](https://www.phoronix.com/news/Linux-6.18-ARM64-Atomics-Issue)
reminded me that this was a thing and that I should double check the IR.
This was the only IR operation remaining that still didn't use the fetch
variety. Convert it over to the fetch to avoid the expectation that it
can be a "remote atomic". Change is going to fall in to noise, but might
as well as be consistent.
While this worked great for the singular unit test. I remembered thatour
pool allocator returns the minimum working size asked for but will
return larger sizes if exact fitment couldn't occur.
Because we are dealing with guard pages, we need to return the full
buffer size to the "client" so they can tell the frontend where the
guard page actually lives. Otherwise the JIT will tell the frontend the
guard page is at the end of the requested size, blow past the limit,
and fault in a completely different location.
With a bit of logging I saw in a multithreaded environment that we were
basically always getting a larger requested buffer while Steam was
starting up.
Primary fix here is returning the current CPU index in function 01h.
Intel Quartus uses this alongside affinity setting to check if all cores
can be used for its calculation. Since we had hardcoded apicid 0 here,
it assumed to only have one core and never generated worker threads.
Additional fix for apicid size. This is the size of the bitmask required
for apic ids, we weren't calculating this correctly at all. This mask is
a "maximum" number of APICs that the CPU reserves in power of two.
Say the core supports 256 APICs, but the processor only supports 16, or
any other combination.
Turns out I was reading six year old code for Wine's implementation for
SRWLocks. It actually /doesn't/ use WAIT_BITSET in their implementation.
It's still write-priority but it's actually significantly slower than I
was expecting due to futex queue usage and some other implementation
details.
Instead of using Wine's implementation, use win32's Wait/Wake on address
functionality and reuse all our other mechanism for implementing this
futex. This grants us our regular low-overhead codepath that I tested on
Linux, while the fallback is the only "slow" path. This also allows us
to still support a pseudo `WAIT_BITSET` code-path that reduces
stampeding even on Win32. The reader side just waits on the upper-half
of the futex (the writer bits) and the `WaitOnAddress` means only the
exact match address will be woken. We also get the regular
reader<->writer hand-offs working.
While this path still uses the futex
queue, the majority of the time our mutexes get acquired in the WFE loop
already, so it's a significant win.
Dark Souls Remastered before:
```
$RDLck Time: 4.531100 ms/second (0.04 percent)
$WRLck Time: 2.122560 ms/second (0.02 percent)
```
after:
```
$RDLck Time: 1.441620 ms/second (0.01 percent)
$WRLck Time: 0.963720 ms/second (0.01 percent)
```
When the JIT CodeBuffer overflows, we will now catch accesses to the
guard page and longjump while restarting the JIT with a larger buffer
request.
Fixes#4877
ARM64 branches have fairly small relative distances they can encode.
These can be +-1MB, or even +-32KB. The largest relative branch is
+-128MB, which we already set as an upper limit of our block JIT cache
size.
We have for a long time just compiled these without checking with the
expectation that things just happen to work. We didn't hit the asserts
so it was relatively low priority. Apparently now with Steam and a
MaxInst limit of 5000, we are now hitting an assert where we are
encoding too large of a range.
Implement support for long jumping from anywhere in the JIT for when a
long jump tries to be encoded and fails, allowing us to restart the JIT
at any moment. This is implemented as a long jump when this singular
feature could have gotten away with some sort of invasive check and
early exit path for two reasons. For one, that would be even more
invasive, effectively doing try-catch logic manually. And two, the next
step is supporting JIT buffer overflow for when our block size heuristic
fails.
This next step will mandate longjump on SIGSEGV (with cooperative
interaction with the frontend) from effectively /anywhere/ in the JIT.
One of the design goals of the CodeEmitter is that every code emission
function doesn't do a size remaining check to allow the compiler to do
some very effective optimization of emitting code blocks to memory (and
it works!).
But we lose the ability to sanely size check. When writing the emitter I
knew we were going to need to write this cooperative guard page handler,
and we're finally at a point where it needs to be done. This will be in
the next PR although.
When MAP_FIXED is used, if it was larger than the VMA region it was
trying to fit in to, then it would overallocate, corruption memory
adjacent to the VMA region. This was due to a typo in the LiveRegion
range checking.
Fix the typo, add a unittest that tries to overallocate space. Would
assert out without this bug fix.
In the case that overlapping `MAP_FIXED` mmap functions were used, we
were incorrectly tracking the full mapped regions size as new
allocation. We instead need to track which pages have already been
previously allocated and only track those. Would behave like FEX was
running out of memory, but we were just mapping the same location many
times.
Adds a unittest to track this.
Now that our Lookup cache mutex is no longer recursive, we can safely
use a shared_mutex instead. The problem with a c++ std::shared_mutex is
that it doesn't guarantee any form of priority, so tens of thousands of
read-locks per second can cause a writer to never acquire the lock, or
take too much time.
The bad news is that C++ doesn't provide us a primitive with
write-priority, so we need to construct our own that is still compatible
with Linux futex. So this is what we do.
- Windows: Uses an SRWLock instead.
- Only way for WINE to provide us a futex fallback that priorities
write-priority without stampeding.
A couple things here, we were never returning the last searched element,
either the last or first depending on search direction.
Also the backward scan would return incorrect indexes in some cases.
Also scanning beyond its page bounds.
Additionally some minorly incorrect assertions.
Adds a new unit test that ensures that we can allocate in to every
location, and that we get the correct indexes back. Also allocated
within guarded pages to ensure it doesn't read outside the bounds.
Fixes a spurious crash in Ender Magnolia.
Just helps when an unhandled ESR occurs, it was always a case of needing
to go in to the ARM ARM to decode it which was a bit of a pain. Add a
textual representation of it.
Fixes crash that occurs in applications that use both GL and Vulkan,
Like UE5 Vulkan native games. Fixes Ender Magnolia.
The issue here is that UE5 loads libGL first, which initializes our
libGL thunks, setting its X11Manager's functions.
It then loads libvulkan, which calls our oninit constructor, which
because of the symbol conflict, calls in to the libGL thunk's host
functions to reinitialize its function pointers, never initializing the
Vulkan X11Manager's functions. It would then crash as soon as an X11
function was used.
Give them unique symbol names so we don't accidentally look up the
incorrect symbol.
QEmu 10.2 is going to expose MIDR with Apple's vendor ID with variant 0.
That's the best they can do because they don't can't pin threads to
particular cores. So give a string for it, and detect it in the fit
script.
This reverts commit e1a45a2720, reversing
changes made to bd7edd8651.
The change rendered pressure-vessel non-functional on muvm-based setups
like Fedora Asahi Remix.
The InterpretAsFloat was never properly made use of. There's a couple of issues
that are fixed more easily with this gone, so lets remove it.
If there's a specific optimization that requires this, we can bring it back
at a later time. This should not have any effect on the current code generation.
This was missed before, where the non-repeating strings instructions
were still using TSO even when the memcpy/set config option was
disabled. Make sure it listens to the config option and disable TSO in
those instances.
Noticed this while profiling Dishonored, and WINE's `sse2_memmove`
function was showing up as a high amount of CPU time. This is due to
them using non-repeating string operations on the header and tail of
their memmove to align to 16-byte.
With this fixed, it causes the game to go from ~62FPS to ~67FPS,
becoming bottlenecked by x87 emulation instead of memmove. Doing about
23 million soft-float operations per second, because it needs full
precision to remove some flickering artifacts.
- Cache miss counts
- Useful for determining if L2 cache or dynamic cache could help
- Cache read/write lock contention times
- Useful to see if threads are blocking each other on contention
- Read lock is the case where a read-lock is beneficial, even if we
currently use a write lock.
- JIT count
- Useful to see if any new JIT blocks are generating
On top of #4951 because it fiddles with the cache stuff.
rpmalloc is currently very aggressively configured which causes
significant reductions in resident memory over jemalloc.
In Bayonetta's title screen it went from 963MB down to 834MB resident.
With our flags being optimized, this does even less than when it was
introduced. It's a hack, people are tinkering with it thinking it'll do
something. Get rid of it.
L1 cache residency can get quite large. Solution, start out small and
scale quickly on L1 cache misses but L2/L3 cache hits.
Some stats on L1 cache residency change:
- Teardown: 40MB -> 16MB (40%)
- Ender Lilies: 79MB -> 32MB (40.5%)
- Death Stranding: 186MB -> 93MB (50%)
- Steam: 75MB -> 7MB (9.3%)
The cost of this option is effectively free in our JIT. It changes a
single LDR to be a single LDP, which on Cortex CPUs cost the same. We do
this by moving the L1 pointer mask in to the CPUState object, making it
dynamic so it lives next to the L1 pointer. We then use that directly
rather than having the hardcoded value.
The lookup cache does a little bit of additional tracking and heuristics
to determine when the current L1 cache should increase or decrease in
size. From 128KB to 16MB per thread, allocating the full VA range as
previously.
Once the heuristic determines that L1 should be increased, it simply
changes the max and the L1 pointer size to compensate, the kernel will
fault in whichever pages are necessary.
Decreasing the size is a little bit more complex, as we want to madvise
the resulting L1 range to ensure we don't have that memory as resident
anymore. Same heuristic but going in the opposite direction otherwise.
Tends to be the case that L1 cache increases a bit on loading screens
then backs down once in-game.
These heuristic values are exposed for increasing and decreasing because
while I think I've picked reasonable values, we will likely need some
more fine tuning over time. Kind of expert user toggles at that point.
Based on #4940 as a base which needs to be merged first.
Full tracked stats from steam as an example of where we are:
```
Total (1000 millisecond sample period):
JIT Time: 0.486630 ms/second (0.00 percent)
Signal Time: 0.065880 ms/second (0.00 percent)
SIGBUS Cnt: 38 (38.160780 per second)
SMC Cnt: 0
Softfloat Cnt: 0
FEX JIT Load: 0.004585 (cycles: 552510)
Total FEX Anon memory resident: 368 mB
JIT resident: 95 mB
OpDispatcher resident: 38 mB
Frontend resident: 8 mB
CPUBackend resident: 624 kB
Lookup cache resident: 0 (null)
Lookup L1 cache resident: 7 mB
ThreadStates resident: 460 kB
Unaccounted resident: 217 mB
```
This mode has been broken for a long time because it's mostly untested.
Barriers, and backpatching while slow have proven that they work.
Maintain the one TSO path, at least until all ARM hardware gains support for
x86-TSO memory model mode.
Test behaviour themselves not changed at all, just data moved or
aligned.
For tests that aren't explicitly testing out SMC behaviour, we were
accidentally relying on some aggressive SMC tracking by mixing data and
code in the same page. To fix this just align the test's data to the
next page boundary which means FEX's SMC tracking won't get triggered
since it is no longer living in the same page.
This has been a thorn for a while, so just get rid of it. We obviously
still have ASM tests that still exist that /do/ rely on SMC, and those
are still expected to work.
If called back-to-back, the compiler can't optimize the object creation
resulting in multiple indirections. Save the creation and pass it
around, allowing the compiler to merge loadstores, and remove redundant
loads.
NFC
The file offset of a file mapping doesn't necessarily match its address
offset in virtual memory from the base file mapping. Indeed, most libraries
violate this assumption.
Now that the MappedResource::FirstVMA reliably identifies the base memory
mapping for a given library (even when that library is mapped multiple times),
this can easily be fixed.
PE/ELF binaries are sometimes mapped multiple times in the same process.
If this happens, there is no longer a unique base virtual address per file.
This breaks assumptions required for code caching: Any time a file mapping
is created, FEX must be able to unambiguously determine the base virtual
address of the mapped library.
This becomes possible by creating a separate MappedResource each time an
ELF header is re-mapped.
We want to take advantage of 16-byte single-copy atomicity. Which I am
relying on, but didn't codify it the first time.
Additionally add comments to explain that new members should be added to
the end to allow tools time to gain support gradually. This will allow
me to add new members without fully breaking mangohud, they'll just not
display the new information until support is added.
We're not guaranteeing backwards compatibility, just an attempt not to
constantly churn the format unless necessary. This way if we do break
compatibility, the tool will have an upper bound on supported versions
before needing to rewrite code.
Currently FEX doesn't properly support partial decoded instructions,
which behave slightly differently than full noexec or invalid
instruction decodings. Before this commit we didn't even have a way to
detect the difference.
Primary difference is that the faulting RIP is the beginning of
instruction decode, while the fault address is the first byte that
couldn't be fetched due to memory permissions. This shows up as a
difference between the RIP in mcontext and si_addr in siginfo in the
Linux signal handler.
Right now just change the log so we can determine if we need to support
this edge case.
This saves a whole bunch of memory. Cutting `Just Cause 2`'s title
screen from 1132MB anonymous FEX memory down to 438MB. 629MB in L2
alone.
L2 is primarily a means to reduce overhead in map queries, so it's all
about performance. But because it consumes a lot of people it's kind of
hard.
One idea is that the L2 lookups can be moved to shared data structures,
since we already pull the shared lock when doing an L2 lookup this is
already halfway there.
Side note, we're using unique locks even with read-only code paths
which we can't use the shared lock because this terrible recursive
mutex!
Instead of outright changing L2 behaviour and potentially wrecking
havoc, add a config option for now so testing can happen over time.
before:
```
Total FEX Anon memory resident: 1132 mB
JIT resident: 60 mB
OpDispatcher resident: 97 mB
Frontend resident: 37 mB
CPUBackend resident: 500 kB
Lookup cache resident: 629 mB
Lookup L1 cache resident: 108 mB
ThreadStates resident: 436 kB
```
after:
```
Total FEX Anon memory resident: 438 mB
JIT resident: 62 mB
OpDispatcher resident: 56 mB
Frontend resident: 22 mB
CPUBackend resident: 496 kB
Lookup cache resident: 0 (null)
Lookup L1 cache resident: 109 mB
ThreadStates resident: 436 kB
```
Trivial fix, if a symlink gets deleted between checking if it is a
symlink versus getting a path of it then it resulted in a crash.
Happened periodically for me.
Every time I see this recursive mutex I glare at it. Remove the last one
so that we no longer need to deal with it.
The only reason why this recursive mutex still existed today was because
it is fairly intertwined with the ContextImpl and tracing it all was a
pain.
Peel back the layers and follow the idiom to have ContextImpl pull the
write mutex when requiredand pass it through by reference to ensure it stays alive.
This allows us to entirely give rid of the recursive nature of the
mutex, which means that `FindBlock` can eventually be switched over to a
read-lock to improve multiple threads reading the caches at the same
time.
I didn't do that exercise since that can be followed up in a subsequent
PR.
Use getuid() instead of geteuid() when determining FEXServer socket names.
Ensures setuid binaries (like chrome-sandbox from Discord) connect to their parent
user's FEXServer instance instead of trying to spawn a separate server.
Fixes connection errors when running applications that spawn setuid children.
Fix a typo in unittests/ASM/Secondary/09_XX_07.asm which caused it to
jump back to the test_32bit label instead of test_64bit.
On Arm systems with FEAT_RNG support, a significant amount of time may
be required before successive uses of RNDRRS. This is because it returns
a random number with fresh full entropy, and it can take a while to
collect the new entropy. That time may be hundreds or thousands of
instructions, so by jumping back to test_32bit the 64-bit test will
alway fail because an RNDRRS has been executed too recently.
Signed-off-by: Rebecca Cran <rebecca@bsdio.com>
This captures the remaining FEX allocations that /aren't/ coming from
JEMalloc, allowing us to separate our mapped regions versus just
jemalloc allocations.
With some additional naming in jemalloc (which I'm not adding here) this
gets us interesting results:
```
Misc resident: 54 MiB
JEMalloc resident: 208 MiB
```
So 208MB of active jemalloc allocations in this particular case. These will be able to be tracked in heaptrack-like applications if careful.
This should let us target down whatever live allocations we're keeping
large amounts of data around if possible.
These don't require being bound to class state directly, and so the list
management can be completely opaque to the outside (also means less
rebuilding if these change)
With the gather overflow fixes in place, I've been having this running
for a while. Now that we just kicked out a release, enable AVX even on
32-bit.
We'll need to eventually create a list of games that explode with AVX
enabled, but that same list would match what happens on real x86 hosts,
so there can be some collaboration there.
The frontend did a quirky widening check which was accidentally working
in this case, but it is supposed to be for the couple of GPR handling
AVX instructions.
Correct the implementation to use the correct register size for FMA.
Turns out a simpler way was added to the docs at some point and I never
noticed.
Before:
text data bss dec hex filename
4159895 1471360 4336824 9968079 9819cf Bin/FEX
After:
text data bss dec hex filename
4157159 1471360 4336824 9965343 980f1f Bin/FEX
Just a trivial rearrangement, Drops the struct down to 40 bytes.
We can also make some functions take it by const reference so we
aren't churning some copies.
Before:
text data bss dec hex filename
4160559 1471360 4336824 9968743 981c67 Bin/FEX
After:
text data bss dec hex filename
4159927 1471360 4336824 9968111 9819ef Bin/FEX
SHM stats and CallRet stack was accidentally using system mmap which
means it would take VA space from the guest. Ensure it uses the
Allocator helpers so that doesn't happen.
Gives 32-bit guests 8-ish megabyte of VA space back.
Avoids actively doing this wonky thing where we're passing
iInvalid all over the place to mean variable alignment depending
on store element size or GPR size.
Makes using the API a little more visibly straightforward and makes
cases where alignment matters more explicit.
Will allow external tools to track how much memory FEX allocates.
Necessary since we can't use traditional memory usage tools to track FEX
memory allocation independently of guest allocations. Plus most tools
like heaptrack hook allocation symbols, which break under jemalloc.
Using this information I can see with Steam loaded with my library that
FEX consumes ~825MB. Total process resident memory is 1204M, accounting
for around 379MB being used by steam itself. This is /relatively/ close
to my desktop running steam at around 261MB. There's a bit of variance
due to what Steam chooses to do at startup.
This tracking will be the first step towards seeing where our memory
usage is going.
rbit is useful in generic algorithms.
`MaskGenerateFromBitWidth` is only really useful for SSE4a, but
implemented in the OpcodeDispatcher is rough, so add an operation.
This isn't actually used anywhere in the IR header, so we can
move it to where it's actually used.
Now the IR interface header doesn't have anything related to the
independent passes in it.
Reduces a bunch of noise related to the register classes and hoists them
out so that converting the classes over to enums should be fairly
straightforward.
This allows easily moving the default case out of the switch, making it
easier for compilers to warn about missing values in switches if any
enum members are added in the future but aren't added to the
formatters.
Instead of having this sort of odd indirection through a struct type,
we can add support for defining custom enums in the IR description.
This lets us both get strong typing (and allow for weak typing, should
any enum in the future need it), without needing a struct for a basic
value type.
Even then, if we do need a struct for anything in the future, then
we still allow strong typing for values themselves while allowing
them to be used in various ways.
Allows applications running entirely under Linux to use the same
extended volatile metadata as Windows.
For example `FEX_EXTENDEDVOLATILEMETADATA=iw4sp.exe\;0xe9da0-0xe9ec7`
this configuration works for both wow64 and Linux to disable the TSO
emulation on the memcpy routine in that game that consumes around 80% of
CPU time in TSO emulation.
Works with Linux native games as well of course.
This adds a new mode X87StrictReducedPrecision.
The strict reduced precision is like the reduced precision but adds extra checks,
like the currently implemented nan and snan propagations.
Fix for __builtin_issignaling() test of SPEC2017 classify test.
These get stored based on the current rotation of TOP. So we need to be
a bit careful with how we do this storing. A smidge of overhead, but
nothing unexpected.
This allows Known_Failures files to contain either:
- Partial paths (existing): Test_X87_F64/SomeTest.asm (affects all configs)
- Full paths (new): jit_1/Test_64Bit_X87_F64/SomeTest.asm (specific config)
For example: Mark only the jit_1 (-n 1) configuration as failing:
jit_1/Test_64Bit_X87_F64/Memcopy_int_F64.asm
Two bug fixes here.
- We weren't reporting a correctl ErrorRegister. Needs to contain
PF_INSTR at least.
- We weren't setting siginfo->si_addr in the case of `FaultToTopAndGeneratedException`
Fixes Mafia 2 (Classic)
We need to read the current x87 TOP location and rotate the values when
saving and restoring the context state on Linux. Wasn't visible in the
ASM tests since we read the FEX's CPU state directly there.
Still creates a copy of FEXInterpreter from FEX for downstream projects
to have some time to get off the old name. Creating a symlink is kind of
a pain in cmake so just doing an install copy is easy.
We were paying a large cost per Literal type that we can special case
for the two class of instructions that use a 64-bit literal.
If we packed this would get to a further 62 bytes but probably not worth
it.
This is entirely unused as FEXServer only ever reads the environment and
other configs.
There was /technically/ a weird conflict with FEXServer arguments but
that was an accident if it ever happened.
x87 80-bit loads, both BCD and regular tword were loading 128-bits of
data when reduced precision was enabled. This was an oversight from the
previous fix a while ago.
Adds a specific reduced precision test for this, and updates the current
test to ensure stores are still tested as well.
This is now cached internally in the X87 pass, and all operations
outside of that that use it are rare so can afford loading/storing
directly from context after flushing x87 regs.
Flushing other regs is not necessary, and breaks any ConvertNZCVToX87 use
which relies previously saved NZCV values as the flag-setting NZCV op after
the save could trigger a flush of NZCV.
We had quite a bit of dependencies that were currently being indirectly relied upon.
We can forward declare the relevant ones and add the includes for the ones that are
explicit.
We don't really need to make this potentially return a null pointer when we could always return a valid instance
that would just happen to be empty if unused.
We already have an equivalent header within FEXCore that's header only,
so we can adapt it to conform for both cases, allowing for removal of
one of them.
Moves the IR include into one of the more specific headers, which avoids dumping the IR header into any core bits that use the interface.
Also uncovered a missing header guard.
We definitely don't want the boolean null test to be able to be implicitly converted
(e.g. to an int or whatever else).
For example a non-explicit bool operator allows for silly things like:
NonMovableUniquePtr<...> ptr;
// ...
auto k = 5 + ptr;
to build without issue, which we should really force the user to be explicit about if it's *really* a desired behavior.
We can reduce includes, such as logging by specifying a concrete size for logging levels,
allowing the enum to be forward declared. We can also move FillHeader into the cpp file,
allowing the syscalls header to be removed.
Profiling shows this is by far the hottest path, as every DX call will
hit this (as arm64ec functions in a vtable). L2 hits are much rarer and
L1 almost never hits since that is looked up inline in most cases.
The dispatcher/block linker will handle this, but if the instruction
following a POPF flag doesn't otherwise trigger one of those the
interrupt would be missed.
Unity games crash with TSO disabled due to the SPSC GfxDevice
ThreadedStreamBuffer read/write pointer updates missing acq/rel
semantics. Rather than attempting to use heuristics to match cases like
this, which could be overzealous and hit more accesses than necessary, just
target the problem directly as this is consist across 32/64 bit Unity versions
for at least the past 10 years. Gate this behind the existing Unity mono
hacks to avoid false-positives in non-Unity games.
nasm doesn't let you encode all of these but we had already supported
them in FEX.
It's a bit of a weird edge case behaviour, since it's supposed to be
sign extended a 32-bit value in to a 64-bit register but with prefixes
(or lack of) it can be a move with zero-extend or a 16-bit insert.
Make sure we're actually testing these edge cases.
Fix SoftFloat IsNan - custom detection matches IEEE754 semantics.
Sets Invalid Operation flags properly for NaN comparisons.
Fixes: GCC-C-execute-ieee-fp-cmp-8l test
__builtin_isunordered() now returns correct values for both NaN and normal operands
This isn't used outside of the IR emitter. Plus, IsBlockExit() is already a more general interface to use,
since it handles the fragment exit case as well.
Since all of the information comes from the Dispatcher, we can have the dispatcher
provide that information. This way we can also eliminate a bunch of now-redundant
public interface members and simplify the config setup within InitCoreImpl().
Conveniently, this also allows making all members of the dispatcher non-public.
The name itself is already qualified with SignalDelegator, so this can reasonably be outside the class itself.
This also allows for forward declarations of the config struct
One set of tables for 128-bit and one set of tables for 256-bit.
This one took a bit longer since I needed to convert a few handlers over
to `Bind`. With this all of our x86 tables are costexpr so they end up
in RO mapped memory which is great.
This matches kernel behaviour for how brk works. I initially implemented
this as a way to speed up some early applications that relied heavily on
brk. We're long past that time and we should match Linux behaviour
instead.
This fixes issues where applications (FEXLoader really) can see we've
allocated a larger granule and breaks self-hosting. There's no reason to
be smarter around this, if applications want more perf then they'll
switch to a better allocator than brk.
Also need to make sure that after the brk region has been reserved, to
unmap its initial mapping to allow future mmap syscalls to overwrite it.
We need to reserve it early to ensure the region initially exists and we
don't accidentally map other things in that space. Then it is up to the
guest if they don't want to overwrite it.
Implied the maximum size that brk could grow to. Which isn't correct, it
is the current maximum size mapped which is page size, versus the
current brk offset which is byte ranged.
A wild use case of union over a variant because we don't want to
increase the encoding size from 128-bit to 256-bit (because of padding).
The type of operation is encoded with the table operation type, so a
variant is unnecessary and we get to keep the 128-bit encoding.
This allows us to have "recursive" x86 table descriptions. But in
reality this is going to only be one layer deep. As this will allow the
Frontend decoder to select instruction encodings based on arch bitness
once the tables are generated correctly.
Noticed some runners had generated a .fex-emu/Config.json, run the tests
in portable mode to ensure they don't pick up a spurious config option.
FEXLinuxTests still ran non-portable because it's required for the thunk
tests.
A prevalent pattern in the FEX codebase is to compute some data and store it
in a maybe_unused variable that's only ever passed to LOGMAN_THROW_A_FMT.
Besides few exceptions, we never compute expensive data in the macro
arguments themselves, so we can remove a lot of code noise by unconditionally
evaluating the condition even in assertion-disabled builds.
When loading an ELF file (usually PT_EXEC), we would have BSS regions
that ended up in the high pages of the address space. We would then
attempt mapping BRK after whatever the highest address ending up being.
This was problematic because we used `MAP_FIXED` which means that the
BRK region could cross in to 64-bit address space, overwrite VDSO,
overwrite the stack, vsyscall, maybe a couple of other things.
Instead of letting that happen, reorder some of the logic so that in
`LoadElfFile` will ensure the full `BRK_SIZE` is allocated (or return
zero) and then we can ensure we're never overwriting other various
memory regions.
This doesn't fix two fundamental issues that FEX has with brk:
* If BRK gets mapped below the ELF, that should mean it isn't mapped at all
* Linux-isms, doubt this breaks new software
* We still map the full 8MB, which isn't correct
* I'm working on fixing this issue, which is why I encountered this.
Only wired up for wow64 and arm64ec. Gives more granular control over
TSO enabling and disabling. Matches arm64ec volatile metadata except
with one more additional feature that whole modules can be disabled at a
time.
Once we know the mapped size of files in Linux then we'll be able to do
the same thing there, but there's not a full mechanism wired up for that
yet.
This nearly gets FEX's TestHarnessRunner to be self-hosting inside of
FEX. The only thing blocking it currently is that our SBRK emulation
reserves the whole region, when it should be "soft-reserved" and mmap
with MAP_FIXED_NOREPLACE can override it. Plus an assert in
OpcodeDispatcher preventing any 32-bit code from running from a 64-bit
process.
In the most simple terms, gdt and ldt are setup to be unique per thread,
and modify_ldt then modifies that thread's ldt entry. On thread
creation, these values get inherited as a copy.
This allows installation of 32-bit code entries, which with the previous
PRs merged allows the code to attempt jumping to that 32-bit code entry.
It then will immediately explode with an assert in our OpcodeDispatcher.
We can't allow 32-bit code jumping yet until our OpDispatcher/X86Tables
allows runtime selection of 64-bit and 32-bit code entries which is
still a ways away.
With the assert removed and the SBRK code handling hacked out,
/technically/ the TestHarnessRunner can run some code, albeit anything
that changes behaviour between bitness is completely incorrect.
These x87 f64 reduced precision operations don't use FCW so we don't
need to load it from the context. So just remove loading it. This falls
within noise while benchmarking.
Effectively NFC today while we don't allow installing LDT entries, but
is required to have signal handlers execute in the correct bitness if
32-bit code segments are enabled.
Nothing too crazy, just that four segments are stuffed in to the
`REG_CSGSFS` data value, and then CS/SS gets restored on restore.
FS and GS are ignored in this instance because edge case behaviour where
those don't get reset on signal entry, so if they get changed then it is
on the userspace application to fix it.
Gets another change out of my stashes.
Similar to the previous far jmp, if the CS changes operating mode then
things will still explode with other FEX asserts. But this gets another
change out of my stashes.
These instructions go hand-in-hand obviously so they get implemented as
a pair.
Taking this very slowly because this is very fickle code. The frontend
needs to manage GDT and LDT, but before we get there, we need to
actually add support for LDT in the backend. Split the segments to two
arrays so the JIT can actually update their cached values correctly.
Still treats GDT and LDT as mirrors like how the JIT previously did (By
it ignoring the selector's TI bit).
While most applications will just create ALL_ACCESS handles which
support the additional operations FEX needs over the regular windows syscall
(usually just QUERY_INFORMATION to lookup the thread object), it is
valid for them to pass in the minimal set required for each operation
and windows/wine will reject any other operations (e.g. querying the TEB
base on a handle with only the SUSPEND right). Windows permits
duplicating handles to extend their permissions given the process itself
has such permissions, so do that as necessary for the operations FEX
needs.
In the terminate case, if we don't return early then we will delete the
FEX thread object but the actual terminate call that follows will
fail, leaving the thread in a bad state.
We don't need to be informed of our own mappings, avoids a deadlock that
occurs if the ThreadCreationMutex lifetime is extended when creating a
thread.
Hashmaps are super expensive and there's no reason not to use a vector - we
already have compact block IDs so we don't benefit from the sparseness. Huge win
for very little effort.
Spotted when profiling FEX. CondJump() in the JIT was almost 4% of our time (?!)
and all because of map slowness. Easy fix.
Difference at 95.0% confidence
-0.0196494 +/- 0.00194956
-3.92827% +/- 0.389753%
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
..and be consistent about signedness.
clang doesn't seem able to do this itself.
Difference at 95.0% confidence, n=100
-0.272326% +/- 0.242353%
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We are onyl wanting to test the FTW, which is 16-bit but we were reading
a whole 32-bits. The upper 16-bits was ending up in the reserved
section, which is undefined as to what it contains. In FEX it'll contain
0 but on a Zen 4 CPU it contains 0xFFFF.
Only read the bits we care about.
now obsolete!
Results for the whole series are excellent:
Difference at 95.0% confidence
-3.97603% +/- 0.254656%
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Augment IREmitter to inline constants as we generate code, rather than
needing a later clean up pass. This replaces the last function of ConstProp.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Now executable page tracking is implemented, this limit is technically
unnecessary and effectively never hit in normal code. Keep a reasonable
limit however to avoid accidentally inlining tail calls and exploring
dead branches in obfuscated code.
We were failing to save the entire 64-bit register, cutting off the top
32-bits which contain SS and CS. This ...happened to not cause issues
but is a bug that I encountered with far jump handling.
This patch does cover up all v4l2 syscalls in i386. It just works
well with the poor v4l2 driver of wine32.
Signed-off-by: Changwei Miao <chanthmiao@outlook.com>
In b8dd5d95b ("OpcodeDispatcher: optimize X87FTWTag"), we optimized
X87FTWTag using an efficient Morton interleave operation. Here, we do the
inverse, optimizing SetX87FTW using an efficient Morton deinterleave
operation.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
set(X86_32_TOOLCHAIN_FILE"${CMAKE_CURRENT_SOURCE_DIR}/Data/CMake/toolchain_x86_32.cmake"CACHEFILEPATH"Toolchain file for the (cross-)compiler targeting i686")
set(X86_64_TOOLCHAIN_FILE"${CMAKE_CURRENT_SOURCE_DIR}/Data/CMake/toolchain_x86_64.cmake"CACHEFILEPATH"Toolchain file for the (cross-)compiler targeting x86_64")
set(X86_DEV_ROOTFS"/"CACHEFILEPATH"Path to the sysroot used for cross-compiling for i686 and x86_64")
set(DATA_DIRECTORY""CACHEPATH"Global data directory (override)")
set(HOSTLIBS_DATA_DIRECTORY""CACHEPATH"Global data directory (override)")
set(X86_32_TOOLCHAIN_FILE"${CMAKE_CURRENT_SOURCE_DIR}/Data/CMake/toolchain_x86_32.cmake"CACHEFILEPATH"Toolchain file for the (cross-)compiler targeting i686")
set(X86_64_TOOLCHAIN_FILE"${CMAKE_CURRENT_SOURCE_DIR}/Data/CMake/toolchain_x86_64.cmake"CACHEFILEPATH"Toolchain file for the (cross-)compiler targeting x86_64")
set(X86_DEV_ROOTFS"/"CACHEFILEPATH"Path to the sysroot used for cross-compiling for i686 and x86_64")
set(DATA_DIRECTORY""CACHEPATH"Global data directory (override)")
set(HOSTLIBS_DATA_DIRECTORY""CACHEPATH"Global data directory (override)")
FEX allows you to run x86 applications on ARM64 Linux devices. It offers broad compatibility with both 32-bit and 64-bit binaries, and it can be used alongside Wine/Proton to play Windows games.
* A manager that coordinates access to the CodeBuffer used for compiling new code across threads.
*
* The CodeBuffer is managed as a partially persistent data structure:
* - Exactly one CodeBuffer is now designated as "active", which means data can be appended to it
* - Lossy modifications to the active CodeBuffer will not invalidate any data in use by other threads (which is what enables save CodeBuffer sharing across threads)
* - Instead, such lossy modifications trigger a new "version" of the data in the modifying thread. Old versions of the CodeBuffer persist as read-only data for use by the other threads.
* - The other threads can update their version of the CodeBuffer. This will decrease the reference count and eventually trigger deallocation of the old version
*/
classCodeBufferManager{
public:
// Get the CodeBuffer that was most recently allocated.
// This is the only CodeBuffer that data may be written to.
fextl::shared_ptr<CodeBuffer>GetLatest();
// Allocate a new CodeBuffer with geometric growth up to an internal maximum.
// Subsequent calls to GetLatest will point to the returned buffer.
Loaded 100 of 1628 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Ryan Houdek