This flag breaks FEX heavily for now.
glibc 2.38 started using this flag as an optimization for posix_spawn.
It will fall back to a "non-optimized" implementation if the clone
syscall returns EINVAL. For now do this while we investigate a more
proper implementation.
Should be backported to 2312.1.
In some situations TestNZ is generated with a constant that is using a
constant that can't fit inside of the tst instruction.
This was found in libGLX with virgl, crashing invalid instruction
generation and crashing steamwebhelper
These functions only want the GPRs returned for SRA. This is because the
signal handler needs this map to relation between x86 GPRs and AArch64
GPRs.
When we added AF and PF to the SRA array we accidentally started
returning two more GPRs to the frontend. This caused the signal
delegator to start corrupting the members after GPRs in FEX's CoreState.
Corrupting 16-bytes after the gregs[] array.
This included corrupting:
- es_idx, cs_idx, ss_idx, ds_idx, gs_idx, fs_idx, _pad[]
they're all copypastes of each other, unify into one general "bit test & perform
action" template. this means most of the wins from the previous commits now
apply for bt* without more copypaste.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
if the shift is < N, and we grab bit 0 after, we only need to consider <=N
bits of the source. this lets us use 32-bit lsr for 32-bit bt, which will
reduce masking in the next commit.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Split from #3284 without changing ownership semantics while I reduce the
debugging surface here.
Removes one usage of ParentThread from FEXCore. Which can be done since
it is no longer an opaque structure, we can read the StatusCode
directly.
No functional change.
When COND_AL was added it wasn't added to this helper. Since there is a
gap between the last condition, just early check the value.
Fixes reading beyond the end of the array
Since we're invoking curl directly, we don't need to wrap it in `sh -c`
with this function.
Fixes an issue where curl downloads to a non-escaped path weren't
working. Now they do.
As I was poking around erofs-utils documentation, I found out that
fsck.erofs actually provides an option for extracting erofs images
without using fuse.
This finally puts the erofs handling on feature parity with squashfs.
O_TMPFILE has a few minor problems that I have been thinking about for a
while. I just recently got reminded about this and remembered that most
problems get resolved by using memfd_create.
- O_TMPFILE is only supported on some filesystems.
- Supported filesystem must be the one mounted to the pathname being
opened.
- Only a minor inconvenience as tmpfs and all related filesystems
support this.
- An inode is actually created on whatever filesystem is backing the
folder.
- `/tmp/` must exist as a directory
- If this folder happened to not be mounted then these temporary
files wouldn't have been created.
- memfd_create doesn't have a folder that needs to exist.
- We were leaving the files open as read/write
- While we were rewinding the file offset, an misbehaving application
could have wrote garbage to the temp file.
- memfd sealing allows us to open the FD as RW and then seal its
capabilities, making it a read-only FD.
- We were leaking FDs opened with O_CLOEXEC
- We could have just opened the O_TMPFILE with O_CLOEXEC
- memfd also just supports this flag, so use it.
- No real issues, just nice to be sanitary here.
Overall this doesn't really change any behaviour, but it is nice to
cleanup some of the edges there.
These are only used by gdbserver for filling out its XML data structures
so just remove them from FEXCore.
Also fixes the ordering on RegNames to match the definition of the enum
class definition in CoreState. This has been out of correct order since
we reordered registers months ago.
As we are moving more and more OS specific code to the frontend, this is
another set of functions that can be moved to FEXLoader from FEXCore.
No functional change here, only code moved from protected to private and
to FEXLoader's SignalDelegator.
Once more thread handling is moved to the frontend we can move even more
out of FEXCore. As follows:
- CheckXIDHandler can get moved.
- First pthread FEX makes would just call this.
- Register/UnregisterTLSState
- This can happen in the clone/thread handler once the frontend
handles it.
This leaves very little in the backend and is mostly an interface for
passing signal data to the frontend that it needs once a signal has
occured.
It additionally also is used for `SignalThread`.
The frontend needs to be in control of how threads are created. This is
inherent to the fact that OS threads are OS specific. We currently have
this weird split that when initializing the FEXCore context, we create a
parent thread at all times.
This does some initial cleanup that gets the core initialization nearly
decoupled.
This has long since been unused. Originally implemented for some fuzzing
tests but has been abandoned and that should likely be implemented some
other way.
When a 32-bit imul was being executed it had a chance of returning
garbage data in the upper 32-bits of the 64-bit result.
While this didn't typically cause problems, this gets exacerbated from
32-bit applications executing multiplies for address calculations.
A combination of commits 7146691360 and
d01b457727 exposed this problem where
previously there would be multiple moves between the calculation and
data use which would have zero'd the upper bits for us previously.
Now that we are no longer doing that, we need to make sure the opcode
dispatcher doesn't generate broken code instead.
Fixes Dungeon Defenders, which hasn't worked since FEX-2308.
Adds an ASM test that ensures we don't break it again.
This is an option that has been long overdue for removal. It's original
intention was primarily to lie to the guest application about the number
of cores in the system. This allowed us to say that the system was only
single threaded which worked around some threading bugs that we had
early on.
This is no longer the case and now it is a confusing remnant of the past
that people think they need to set. Incorrectly assuming that "0" by
default means that FEX is doing some sort of disabling of threading and
forcing all emulation down one CPU core. This is not the case and has
never been the case, so removing the option makes that idea go away.
Stop lying to the application about getcpu, sched_getaffinity, and
sched_setaffinity.
- getcpu would wrap the cpu result modulo the count of cores
- sched_setaffinity wouldn't work at all
- sched_getaffinity lied and always reported full affinity of config
option
Split off from #3282 to reduce burden.
We can read the data member directly now since it isn't opaque. In fact
we already do in the signal handlers. Removes these redundant helpers.
Removes one usage of ParentThread in FEXCore.
This usually happens on backwards memcpy where we know the direction of
the copy because the code will typically do as follows:
```
std
rep movsb
cld
```
This is because the direction flag is part of the ABI and needs to be
set back to the forward direction if it was modified.
This typically doesn't get picked up on forward copies because we won't
have visibility of a cld instruction in the block.
This optimization allows us to only emit half of the code for the memcpy
if it is a compile time constant.
There's definitely some future task that could assume forward direction
if unknown and recompile the code if the assumption has failed, but not
doing that here.
While auditing our JIT to see if we have any AFP issues I noticed this.
RPRES has a hard dependency that AFP exists in order to be used, we were
hitting a case where RPRES would be enabled, but AFP is disabled by
default.
RPRES only changes behaviour when FPCR.AH is set which requires AFP.
While this doesn't affect any hardware today, it likely will in the
future.
Removes new warning from clang:
```
FEXCore/include/FEXCore/Core/CoreState.h:119:35: warning: not packing field 'DeferredSignalRefCount' as it is non-POD for the purposes of layout [-Wpacked-non-pod]
119 | NonAtomicRefCounter<uint64_t> DeferredSignalRefCount;
| ^
```
We already ensure sane data layout that this is unnecessary anyway.
Suspend may be called on a thread before it has finished WOW64 initialisation,
keep track of all initialized threads and fallback to direct
NtSuspendThread when this is the case.
The frontend shouldn't need to know any information about how to
reconstruct eflags. Just give us the information we need and it'll work
out.
There are still some inherit limitations of this and some edge cases
that might give invalid data, but it is roughly as close as it was
before.
Just provide if the PC was in the JIT, the host GPRs, and the PState object from the signal
information and FEXCore does the rest.
We don't need to change the signature for `SetFlagsFromCompactedEFLAGS`
because during reloading of register state automatically does this for
us.
Goal is to improve FEX performance with real world games. Let's start with
Sonic Mania. Two hot blocks from #2681.
I'm only bothering to track flagm for this because I'm super biased.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Many flag-generating instructions like cmp need to save calculations for
deferred PF and AF flag calculation. Currently, they require a store per flag,
which is prohibitively expensive for hot instructions like cmp. By instead
pinning PF/AF temporary results to registers (x26/x27 by convention here), we
eliminate many stores altogether and turn the rest into zero-cycle moves (on
64-bit at least, this isn't optimal for 32-bit emulation due to CTX->GetGPRSize
shenanigans, need to check if this requirement can be lifted..).
To implement, we model as SRA and then the existing SRA code is able to generate
good code with little manual tuning. (Future work will get us to excellent code
with more tuning ;) ).
The tradeoff is reducing the working dynamic GPR set by 2 registers, which might
increase spilling in some cases. I think it's worth it in practice, though.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
- sha1nexte
- Takes advantage of sha1h if supported
- Does the operation in a vector otherwise
- sha1msg2
- Instead of dumping everything to GPRs, we can do this with vectors
- Mostly matches ARM's sha1su1 instruction, but it is /just/
different enough to be annoying.
- sha256msg1
- Directly matches sha256u0
- Leaves the previous implementation alone
Not supposed to touch flags at all, so don't! instead of making a terrible mess
of csels. a lot less instructions, and probably faster because the branch should
be predicted correctly in practice in hot loops.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Instruction count CI has transformed the way we work on FEX… I love the system
and want to make it better. there’s one part of instruction count CI that isn’t
so lovable: the problematic “optimal” flag on instructions.
There are several issues with this flag, both philosophical and practical.
– it is tedious to update the optimal flag when making an implementation
optimal. The effect of that is discouraging people from making instructions,
optimal, or encouraging people to fail to update the flag, and dilute the value
of it. Either way, since we care far more about optimal implementations, then we
do about updating the flag, clearly we should prioritize the implementation and
not the flag. This issue was not obvious at the outset, when instruction count,
CI was introduced, and still quite small. The problem magnified when we started
duplicating instructions in bulk for different combinations of CPU features
(flagm, AFP, etc.) that intern multiplies the manual work required to update the
flags by the corresponding constant factor. if it comes down to a choice between
removing this extra coverage and removing the flag, I think we all agree that
removing the flag is the lesser evil.
– The definition of “optimal” is fundamentally problematic. I have often
improved the instruction count of an instruction that was already “optimal”.
This is all kinds of silly, and calls into question whether there’s any value
whatsoever in the existing classifications of the flag. Furthermore, it is often
unknowable, whether an implementation really is optimal. Is it possible to
implement BZHI (with flag calculations) in fewer than eight instructions? We
don’t know, and it’s silly to pretend that we do.
– as a consequence of the problematic definitions , there are so many errors in
both directions that I don’t think there’s much value in preserving the existing
classification at the expense of +progress. Being able to say “32% of
instructions are translated optimally” is neat, but it really doesn’t tell us
anything whatsoever when you dig a little deeper.
So, as the flag is misleading at best and perhaps harmful at worst, let’s remove
it and make the instruction count CI, more useful overall. let’s let the
expected count and the assembly speak for themselves, and cut away the chaff. if
we want a meaningless number to report to management, we can instead calculate
the average blowup factor ;-)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
If a bind fails then this is usually because PID reuse happened and the
unix domain socket path still existed on the filesystem. Alternatively
the process did an execve which left a dangling unix domain file.
Unlink the file and try again in these cases. It will almost certainly
work the second time.
Instead of relying on sleeping in accept, use poll. This allows us to
early shutdown instead of leaving a thread hanging stuck in accept.
Doesn't change behaviour with FEX gdbserver waiting for attach, but in
the future when we all full process trees to have gdbserver running this
will be more important.
Using the server mount folder works most of the time, but when running
under pressure-vessel this stacks directories in a weird way because the
mount folder has some tricks applied to it.
Expose the temp folder being used directly instead.
These are setup to be nullptr by default. Instead of providing no-op
lock instructions just have them be nullptr.
It's already part of the API that they need to be nullptr checked before
calling and this matches behaviour of the real libX11 library. Removes
some spam that is unnecessary.
Unused since TestNZ+NZCVSelect accomplishes the same and good riddance. Might
bring them back later for tbz/tbnz, but certainly not in this Selectful form.
(I added them when I thought we were going to RA the flags. With the more
effective static approach, we don't need this for that.)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Now unused. If we bring it back, it should be brought back as CSSC only. On
non-CSSC platforms, an explicit cmp + predicated neg can be better.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Separate out the NZCV bits from the more complex stuff so we can specially
optimize the branches.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
In this mode, rather than the branch comparing its arguments and then jumping
based on the result, the branch simply jumps by the native comparison based on
the NZCV value. This allows us to map x86 branches to arm64 branches 1:1.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Resolving the issue that we can only ever have one gdbserver process
running consuming port 8086 (even if the port number is cute).
Doesn't give us anything yet but in the future will allow us to have
whole process trees running gdbservers that we can attach to.
Usually better in practice... some rotates are slightly regressed by this but
they were already terrible.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
via the 0 reg. We really need a more generalized approach to taking advantage of
wzr, but this optimizes the special case I care about for seta (saving a move to
make the impl optimal).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This will replace Select soon, as it lets us take advantage of
NZCV-generating instructions and it doesn't clobber NZCV.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Do it explicitly for sve-256 and punt on optimizing, so we avoid regressing code
gen otherwise.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Rather than the context. Effectively a static register allocation scheme for
flags. This will let us optimize out a LOT of flag handling code, keeping things
in NZCV rather than needing to copy between NZCV and memory all the time.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Some opcodes only clobber NZCV under certain circumstances, we don't yet have
a good way of encoding that. In the mean time this hot fixes some would-be
instcountci regressions.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Again we need to handle this one specially because the dispatcher can't insert
restore code after the branch. It should be optimized in the near future, don't
worry.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Semantics differ markedly from the non-NZCV flags, splitting this out makes it a
lot easier to do things correctly imho. Gets the dest/src size correct
(important for spilling), as well as makes our existing opt passes skip this
which is needed for correctness at the moment anyway.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This fixes an issue where CPU tunables were ending up in the thunk
generator which means if your CPU doesn't support all the features on
the *Builder* then it would crash with SIGILL. This was happening with
Canonical's runners because they typically only support ARMv8.2 but we
are compiling packages to run on ARMv8.4 devices.
cc: FEX-2311.1
SHA instructions are very large right now and cause register spilling
due to their codegen. Ender Lilies has a really large block in a
function called `sha1_block_data_order` that was causing FEX to spill
NZCV flags incorrectly. The assumption which held true before NZCV
optimizations were a thing was that all flags were either 1-bit in an
8-bit container, or just 8-bit (x87 TOP flag).
NZCV host flags broke this assumption by making its flags 32-bit which
ended up breaking when encounting spilling situations.
Replace every instance of the Op overwrite pattern, and ban that anti-pattern
from the codebase in the future. This will prevent piles of NZCV related
regressions.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
The "create op with wrong opcode, then change the opcode" pattern is REALLY
dangerous. This does not address that. But when we start doing NZCV trickery, it
will get /more/ dangerous, and so it's time to add a helper and make the
convenient thing the safe(r) thing. This helper correctly saves NZCV /before/
the instruction like the real builders would. It also provides a spot for future
safety asserts if someone is motivated.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
It looks like currently FEX has a bug around implicit flag clobbering
with this pull request where an IR operation that implicitly clobbers
flags isn't correctly saving the NZCV flags before doing the operation.
Adds a unit test that specifically captures this issue. RAX will be 1 or
0 depending on if the flags are clobbered incorrectly or not.
When attempting to debug #3162 I had noticed spurious behaviour around
what I assumed to be eflags getting corrupt around inlined syscalls.
This turned out to be a red herring but to ensure we are still testing
this, create a fully fleshed out unit test.
This test ensures a couple of things.
1) A flag that is set or unset before a syscall doesn't have its data
corrupt
2) An inline syscall doesn't corrupt the eflags, checking the eflag
result after returning from the syscall.
3) A signal occuring while in an inline syscall returns the correct
eflags information in the signal handler information
This test gets accomplished by setting or unsetting a particular flag
and then calling the futex syscall in a way that is guaranteed to be
inlined and also wait forever. Then the parent thread will signal with a
SIGTERM and read back the signal information. It does this multiple
times for each flag we care about.
While tracking issues in #3162, I had encountered a random crash that I
started hunting. It was very quickly apparent that this crash was
unrelated to that PR. I just happened to be running a unittest that was
creating and tearing down a bunch of threads that exacerbated the
problem.
See as follows with the strace output:
```
[pid 269497] munmap(0x7fffde1ff000, 16777216) = 0
[pid 269497] munmap(0x7fffde1ff000, 16777216 <unfinished ...>
[pid 268982] mmap(NULL, 16777216, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fffde1ff000
[pid 269497] <... munmap resumed>) = 0
```
One thread is freeing some memory with munmap, another one then does a mmap and gets the same address back.
Nothing too crazy at initial glance, but taking a closelier look, we can
see that there are two strange oddities:
1) We are double unmapping the same address range through munmap
2) The second munmap is interrupted and returns AFTER the mmap.
This has the unfortunate side-effect that the mmap that just returned
the same address has actually just been unmapped! This was resulting in
spurious crashes around thread creation that was SUPER hard to nail
down.
The problem comes down to how code buffer objects are managed, in
particular how the Arm64Emitter and Dispatcher handled its buffers.
Arm64Emitter is inherited by two classes; Dispatcher, and Arm64JITCore.
On class destruction the emitter would free its internal tracking
buffer. Additionally on destruction, the Arm64JITCore would walk through
all of its CodeBuffers and free them. The problem ends up being that in
the Arm64JITCore, it would free its code buffers which also ended up
being the current active buffer bound to the Arm64Emitter. Thus causing
the Arm64Emitter to come back around and try to free the same buffer
again.
This is a double-free problem! and was only visible on thread exiting!
Can't track double frees with mmap and munmap with current tooling!
This problem typically didn't occur because of how fast the destruction
usually takes and jemalloc inbetween also typically means the problem
doesn't occur. Initially thinking this was a threaded pool allocator bug
because typically the new allocation would end up in there once a new
thread was spinning up.
Now we change behaviour, Arm64Emitter doesn't do any buffer management
itself, instead just passing an initial buffer on to its internal buffer
tracking if given one up front.
This leaves the Dispatcher and the Arm64JITCore to do their buffer
management and ensuring there is no double free.
The day is saved!
This mask was being used incorrectly, it's a GPR spill mask for host
GPRs not an index in to the SRA array. Search the array of SRA registers
for the first one in the mask first to use as a temporary.
Fixes an issue with 32-bit inline syscalls where the first register
being spilled was r8, which was beyond the size of SRA registers on
32-bit processes. This would cause FEX to read the value just after
x32::SRA which is x32::RA. This would mean it would use r20 as a
temporary, corrupting the register in the process.
I noticed this while poking at #3162, but also when I was looking at a
memory buffer ownership problem.
Requires #3249 to be merged first
Library alerting has been disabled for now, and storing IR while
gdbserver is running is removed.
Otherwise no functional change.
When attempting to read files that aren't backed by a filesystem then
our current read file helpers fail since they query the file size
upfront.
Change the helper so that it doesn't query the size and just reads the file if it
can be opened. This lets us read `/proc/self/maps` using helpers.
GDBServer is inherently OS specific which is why all this code is
removed when compiling for mingw/win32. This should get moved to the
frontend before we start landing more work to clean this interface up.
Not really any functional change.
Changes:
FEXCore/Context: Adds new public interfaces, these were previously
private.
- WaitForIdle
- If `Pause` was called or the process is shutting down then this
will wait until all threads have paused or exited.
- WaitForThreadsToRun
- If `Pause` was previously called and then `Run` was called to get
them running again, this waits until all the threads have come out
of idle to avoid races.
- GetThreads
- Returns the `InternalThreadData` for all the current threads.
- GDBServer needs to know all the internal thread data state when the
threads are paused which is what this gives it.
GDBServer:
- Removes usages of internal data structures where possible.
- This gets it clean enough that moving it out of FEXCore is now
possible.
These should always be used in the dispatcher rather than the raw jumps they
translate to, as they ensure that flags are flushed. Eliminates a class of bugs
that will become a lot easier to hit with the new nzcv work.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Requires #3238 to be merged first since this uses the tbx IR operation.
Worst case is now a three instruction sequence of ldr+ldr+tbx.
Some operations are special-cased, which definitely doesn't cover all
possible cases we could use without tbx, but as a worst case improvement
this is a significant improvement.
We can't sanely test all 256 swizzle masks, so walk the array of random
data for each swizzle and CRC the results each step along the way.
This will give us a final result rather than each individual step.
A bunch of blendps swizzles weren't optimal. This optimizes all swizzles
to be optimal.
Two instructions can be more optimal without a tbx but the rest required
tbx to be optimal since they don't match ARM's swizzle mechanics.
gdbserver is currently entirely broken so this doesn't change behaviour.
The gdb pause check that we originally had an excessive amount of
overhead.
Instead use the pending interrupt fault check that was wired
up for wine.
This makes the check very lightweight and makes it more reasonable to
implement a way to have gdbserver support attaching to a process.
gdb gets angry if we return text with `<No Name>` in an xml text field.
Instead only return a name if we have one and gdb will take care of the
rest.
Additionally change the formatting of the return packet, it doesn't need
the xml version header.
Messed up when originally implementing this, substr's second argument is
requested substring length, not the ending position.
Noticed this while trying to parse multiple FEX_HOSTFEATURES options.
Everytime I want to quickly output a value for testing I tend to use
Print which didn't work under the simulator.
Give this a quick fix to wire up the jump to the vixl sim.
Currently we don't get why an IR emit failed in the assert message. Put
the code in to the message so it is easier to see.
This also resolved the issue that when in RelWithDebInfo the assert line
would typically be the end of the IR emission function, so you couldn't
see which assert actually triggered. Now since the message is printed
this is easier
Before:
```
[ASSERT]
```
After:
```
[ASSERT] Size == FEXCore::IR::OpSize::i32Bit || Size == FEXCore::IR::OpSize::i64Bit
```
This is a lot easier and better data than what #3227 proposed.
If the fetch and nonfetch versions mismatched then the DCE optimization
which changes the IR operation would break handily.
Add a comment as a reminder so if anyone touches this they will
understand.
When the result of an atomic fetch operation is unused then we can
safely convert it to a non-fetching version of the operation.
This happens hundreds of times per process as far as I can tell. No idea
if this actually helps any hardware, but theoretically it can allow CPUs
to not stall waiting for writeback of the atomic operation.
Couldn't detect any performance improvements in the various little
things I was poking at least. Very trivial to support so add it.
Unaligned variants are already handled in our unaligned fault handler
since the only difference is the acquire semantic is dropped and the
destination register is the zero register.
Secondary ALU operations were missed and when the operation is 4-bytes
in size we can also allow garbage upper bits since the JIT will emit a
32-bit operation for this instruction which is safe.
Optimizes some bad codegen around 32-bit ALU operations.
Instead of using VInsElement in pi2fw and pf2iw, just use uzp1 to ensure
we don't unintentionally add to RA pressure.
Additionally we can generate the constant needed for pmulhrw directly
using the movi instruction. Converts two instructions in to one.
Under FEX's current constraints this makes all 3DNow! instructions
optimal.
This PR has a bug around flags calculation and REP LODS{B,W,D,Q}.
This currently passes on main but fails on #3162.
Bug only occurs in 32-bit instead of 64-bit with the same test. Should
help diagnose the bugs in #3162.
When SubShift (LSL) occurs with both sources constant then optimize away
the calculation.
Additionally if add is found to have one immediate constant where the
inverse of the constant fits in to ImmAddSub range, then invert the
constant and change it in to a sub.
This optimizes the cases when direction flag is known upfront in an
instruction.
Previously this moved two constant, did a compare and a csel. Four
instructions in total. It also corrupts NZCV which we want to use for
other things.
This new codegen emits one constant and one subtract instruction, two
instructions total and doesn't touch NZCV.
More optimal!
Audit the code base and mark any instruction that implicitly clobbers flags so
it can get special handling in the dispatcher to spill NZCV ahead of emitting.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Lots of instructions clobber NZCV inadvertently but are not intended to write to
the host flags from the IR point-of-view. As an example, Abs logically has no
side effects but physically clobbers NZCV due to its cmp/csneg impl on non-CSSC
hw. Add infrastructure to model this in the IR so we can deal with it when we
start using NZCV for things.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
So we don't need to mark VInsertElement as implicit clobber in the common case.
Only afects sve256 which doesn't exist yet.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This lets most of the ASM tests run on 16K Linux hosts which is good because I
have a Mac and I'm bad at computer.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
With the previous RCLSE pass optimization that fixes store->load
forwarding, this pass started optimizing harder.
This hit a bug with this vmov removal that previously didn't get hit.
In particular this would eliminate vmov IR operations even if they were
zero extending a vector.
Since we have dramatically cleaned up the amount of vmov IR operations
we are generating, remove this optimization entirely. In the games I
tested, the only game that hit this "optimization" was Ender Lilies and
it started generating broken code for the single block of instructions
that did.
Adds a unit test for this case just in-case it comes back in the future
for some reason.
Fixes an issue where Ender Lilies would flash the screen to black every
time an enemy hit the player character.
These instructions aren't super amazing due to the fact that they have
both a source mask and a destination duplication mask.
Setup a case where we can generate more optimal code in /most/ cases.
There are a few that still fall down a "bad" path for the result
broadcast but in most cases they are optimal. Still to be seen what
games typically use the broadcast mask as.
AVX in its infinite wisdom expanded DPPS to 256-bit, while leaving DPPD
to only support 128-bit still. This leaves the original implementation
alone for 256-bit DPPS since I don't want to break it.
This is another instruction that gets a free optimization when
SVE-128bit is supported!
If no registers alias, then we can move the first source directly into the
destination and then perform the FCADD operation as opposed to using a
temporary.
These annotations allow for a given type or parameter to be treated as
"compatible" even if data layout analysis can't infer this automatically.
assume_compatible_data_layout is more powerful than is_opaque, since it
allows for structs containing members of a certain type to be automatically
inferred as "compatible".
Conversely however, is_opaque enforces that the underlying data is never
accessed directly, since non-pointer uses of the type would still be
detected as "incompatible".
This annotation can be used for data types that can't be repacked
automatically even with custom repack annotations. With ptr_passthrough,
the types are wrapped in guest_layout and passed to the host like that.
Previously, two functions with the same signature would always be wrapped
in the same logic. This change allows customizing one function with
annotations while leaving the other one unchanged.
We can perform less moves by checking for scenarios where aliasing
occurs. Since addition is commutative (usually, general-case anyway),
order of inputs doesn't strictly matter here.
In the event no source vectors alias the destination,
we can just move the first source vector into it and
then perform the divide without needing to move afterword.
When a syscall from the *at series is provided an FD but the path is
absolute then dirfd should be ignored. We weren't correctly doing this.
Now if the path is absolute, but set the argument to the special
AT_FDCWD..
Fixes#3204
We can avoid needing to use movprfx here by moving
directly into the destination when possible and just
doing the UMAX directly.
Also expands the unsigned max tests to test values with
the sign bit set to ensure all behavior is caught.
Since SMAX performs a comparison and returns the max value regardless
of how the operands are provided, we can check for when the second
input aliases the destination.
Removes the truncating move that we perform inside the StoreResult
function and instead delegates the responsibility to the instruction
implementations themselves.
This removes a lot of redundant moves that occur on 128-bit variants
of AVX instructions.
Also fixes a weird case where we were handling 128-bit SVE
in VBroadcastFromMem when we already have AdvSIMD instructions
that will perfom the zero-extension behavior for us.
Allows for easier expansion without needing to expand the function definitons.
Also makes a few usages significantly less verbose and makes specifying
options a little more declarative, rather than having to memorize what
each argument is specifying.
Notably this bugfix version also introduces support for formatting
std::atomic types and std::atomic_flag.
Also, of course keeps our tracked external up to date.
A couple of games were hitting these. Not sure how they were missed in
PR #3159 but adds the missing one.
Small rearrangement to make this easier as well. Hopefully thunk stuff
lands sooner rather than later to automate this for Vulkan.
Maybe `-isystem` instead of `-I` needs to be used unlike what #2076,
might depend on what is installed on the host system.
Some simple tests to showcase instructions that we can optimize.
- Back to back pushes could be optimized
- Back to back scalar vector operations can be optimized
- Show with AFP that back to back scalar is already optimal
- Also ensures we don't break this stage.
We don't need zero in the upper bits for a push.
Makes a couple variants optimal.
Adds missing tests to the 32-bit file, since only 32-bit can push a
32-bit register.
These IR operations are required to support AFP's NEP mode which does
vector insert in to the destination register. Additionally it gives us
tracking information to allow optimizing out redundant inserts on
devices that don't support AFP natively.
In order to match x86 semantics we need to support binary and unary
scalar operations that do a final insert in to a vector. With optional
zeroing of the top 128-bits for AVX variants.
A tricky thing is that in binary operations this means that the
destination and first source have an intrinsically linked property
depending on if it is SSE or AVX.
SSE example:
- addss xmm0, xmm1
- xmm0 is both the destination and the first source.
- This means xmm0[31:0] = xmm0[31:0] + xmm1[31:0]
- Bits [127:32] are UNMODIFIED.
FEX's JIT jumps through some hoops so that if the destination register
equals the first source register, then it hits the optimal path the
AFP.NEP will insert in to the result. AVX throws a small wrench in to
this due to changed behaviour
AVX example:
- vaddss xmm0, xmm1, xmm2
- xmm0 is ONLY the destination, xmm1 and xmm2 are the sources
- This operation copies the bits above the scalar result from the
first source (xmm1).
- Additionally this will zero bits above the original 128-bit xmm
register.
- xmm0[31:0] = xmm1[31:0] + xmm2[31:0]
- xmm0[127:32] = xmm1[127:32]
- ymm0[255:127] = 0
This causes these instructions to support a fairly large table depending
on if the instruction is an SSE or AVX instruction, plus if the host CPU
supports AFP or not.
So while fairly complex, it's handling all the edge cases and gives us
optimization opportunities as we move forward. Currently on non-AFP
supporting devices this has a minor benefit that these IR operations
remove one temporary register, lowering the Register Allocation
overhead.
In the coming weeks I am likely to introduce an optimization pass that
removes redundant inserts because FEX currently does /really/ badly with
scalar code loops.
Needs #3184 merged first.
When FEX is in the JIT we need to make sure to enable NEP and AH and
then disable when leaving.
Explicitly disabled when the vixl simulator is used since even
attempting to set the bits will cause it to fault out. Ensures
InstCountCI keeps working.
There are some cases where we want to test multiple instructions where
we can do optimizations that would overwise be hard to see.
eg:
```asm
; Can be optimized to a single stp
push eax
push ebx
; Can remove half of the copy since we know the direction
cld
rep movsb
; Can remove a redundant insert
addss xmm0, xmm1
addss xmm0, xmm2
```
This lets us have arbitrary sized code in instruction count CI, with the
original json key becoming only a label if the instruction array is
provided.
There are still some major limitations to this, instructions that
generate side-effects might have "garbage" after the end of the block
that isn't correctly accounted for. So care must be taken.
Example in the json
```json
"push ax, bx": {
"ExpectedInstructionCount": 4,
"Optimal": "No",
"Comment": "0x50",
"x86Insts": [
"push ax",
"push bx"
],
"ExpectedArm64ASM": [
"uxth w20, w4",
"strh w20, [x8, #-2]!",
"uxth w20, w7",
"strh w20, [x8, #-2]!"
]
}
```
This adds all the missing atomic tests in to their own tests files.
This includes all of them except a few choice ones that are in their
original files.
- BTC, BTR, BTS are in their Secondary/SecondaryGroup files
- CMPXCHG, CMPXCHG8B, CMPXCHG16B are in their Secondary/SecondaryGroup
files
- These always imply lock semantics even without the prefix.
Six of the EFLAGS can't be used directly in a bitmask because they are
either contained in a different flags location or has multiple bits
stored in it.
SF, ZF, CF, OF are stored in ARM's NZCV format in offset 24.
PF calculation is deferred but stored in the regular offset.
AF is also deferred in relation to the PF but stored in the regular
offset.
These /need/ to be reconstructed using the `ReconstructCompactedEFLAGS`
function when wanting to read the EFLAGS.
When setting these flags they /need/ to be set using
`SetFlagsFromCompactedEFLAGS`.
If either of these functions are not used when managing EFLAGs then the
internal representation will get mangled and the state will be
corrupted.
Having a little `_RAW` on these to signify that these aren't just
regular single bit representations like the other flags in EFLAGS should
make us puzzle about this issue before writing more broken code that
tries accessing it directly.
This allows us to use reciprocal instructions which matches precision of
what x86 expects rather than converting everything to float divides.
Currently no hardware supports this, and even the upcoming X4/A720/A520
won't support it, but it was trivial to implement so wire it up.
Suggested by Alyssa. Adding an IR operation can be a little tedious
since you need to add the definition to JIT.cpp for the dispatch switch,
JITClass.h for the function declared, and then actually defining the
implementation in the correct file.
Instead support the common case where an IR operation just gets
dispatched through to the regular handler. This lets the developer just
put the function definition in to the json and the relevent cpp file and
it just gets picked up.
Some minor things:
- Needs to support dynamic dispatch for {Load,Store}Register and
{Load,Store}Mem
- This is just a bool in the json
- It needs to not output JIT dispatch for some IR operations
- SSE4.2 string instructions and x87 operations
- These go down the "Unhandled" path
- Needs to support a Dispatcher function override
- This is just for handling NoOp IR operations that get used for
other reasons.
- Finally removes VSMul and VUMul, consolidating to VMul
- Unlike V{U,S}Mull, signed or unsigned doesn't change behaviour here
- Fixed a couple random handler names not matching the IR operation
name.
This syscall requires a valid pointer otherwise it returns EFAULT.
When going through the glibc helper it can crash before reaching the raw
syscall even.
Enables in InstCountCI so Pi users can run InstCountCI can run the tests
without breaking on crypto operations.
When crypto is enabled or disabled just wholesale change AES, CRC32, and
PMULL 128-bit in one step. We don't really care about partial support
here.
The motivation towards just having a pointer array in CpuState was that
initialization was fairly cheap and that we have limited space inside
the encoding depending on what we want to do.
Initialization cost is still a concern but doing a memcpy of 128-bytes
isn't that big of a deal.
Limited space in CpuState, while a concern isn't a significant one.
- Needs to currently be less than 1 page in size
- Needs to be under the architectural offset limitations of loadstore
scaled offsets. Which is 65KB for 128-bit vectors
Still keeps the pointer array around for cases when we would need
synthesize an address offset and it's just easier to load the
process-wide table.
The performance improvement here is removing the dependency in the
ldr+ldr chain. In microbenchmarks this has shown to have an improvement
of ~4% by removing this dependency chain on Cortex-X1C.
This is the cause of a bunch of redundant moves that shows up in
InstCountCI. Fixing this aliasing and pre-colouring issue causes a ton
of 256-bit operations to become optimal.
Using the cached zero value is less efficient than loading it in to the
register for all these cases.
Lets us use rename hardware more efficiently and removes a dependency
chain on a single register.
Original:
```
movi v2.2d, #0x0
mov z16.d, p7/m, z2.d
<... 16 more times>
mov z31.d, p7/m, z2.d
```
Result:
```
movi v16.2d, #0x0
<... 16 more times>
movi v31.2d, #0x0
```
This is a quality of life improvement for people that want to tinker
with the InstCountCI but they may not necessarily have an Arm64 device
available immediately for poking.
As long as the vixl disassembler is enabled then the InstCountCI tests
can run and get bit-accurate encodings just like on an Arm64 device.
This also ensures that behaviour is consistent with or without the vixl
simulator enabled which is very important when running on x86 hosts.
This runs the data layout analysis pass added in the previous change twice:
Once for the host architecture and once for the guest architecture. This
allows the new DataLayoutCompareAction to query architecture differences for
each type, which can then be used to instruct code generation accordingly.
Currently, type compatibility is classified into 3 categories:
* Fully compatible (same size/alignment for the type itself and any members)
* Repackable (incompatibility can be resolved with emission of automatable
repacking code, e.g. when struct members are located at differing offsets
due to padding bytes)
* Incompatible
The set of these types is tracked in AnalysisAction, to which extensive
verification logic is added to detect potential incompatibilities and to
enforce use of annotatations where needed.
This was only required on x86 devices trying to escape the emulation.
Since x86 is now remove, this is entirely unnecessary.
When Steam launches applications with `/bin/sh`, this will remain under
the emulation and not escape these days.
With the removal of the x86 JIT, there is no need to have these be
independent classes.
Merges the Arm64Dispatcher in to the base Dispatcher class.
No functional change, just moving code.
Similar to previous tests, vpgatherqq and vgatherqpd are equivalent
instructions. So the tests are the same with the mnemonic changed.
This adds tests for an additional two sets of instructions. Getting us
full coverage of all eight instructions if we include the tests from
PR #3167 and #3166
Tests the same things as described in #3165
In addition, since these tests use 64-bit indices for address
calculation, we can easily generate and indice vector that tests
overflow. So every test at every displacement ALSO gains an additional
overflow test to ensure correct behaviour around pointer overflow
calculation.
Similar to previous tests, vgatherqd and vgatherqps are equivalent
instructions. So the tests are the same with the mnemonic changed.
This adds tests for an additional two sets of instructions, Getting us
up to six total over the eight if we include the tests from #3166.
Tests the same things as described in #3165
In addition, since these tests use 64-bit indices for address
calculation, we can easily generate and indice vector that tests
overflow. So every test at every displacement ALSO gains and additional
overflow test to ensure correct behaviour around pointer overflow
calculation.
Just like the previous tests, vpgatherdq and vgatherpq are equivalent
instructions. So the tests are the same except for the instruction
mnemonic again.
This adds unittests for two more of the eight gather instructions.
Getting us up to testing four in total.
Specifically this adds tests for 32-bit indices while loading 64-bit
element instructions.
Same thing as PR #3165 for what it tests versus doesn't.
vpgatherdd and vgatherps are effectively the same instructions, so the
tests are the same except for the instruction mnemonic.
This adds unit tests for two of the eight gather instructions.
Specifically this adds tests for the 32-bit indices loading 32-bit
elements instructions.
What it tests:
- Tests all displacement scales
- Tests multiple mask arrangements
- Ensures the mask register is zero'd after the instruction
What it doesn't test:
- Doesn't test address size calculation overflow
- Only would happen on 32-bit with 32-bit indices, or /really/ high
base addresses
- The instruction should behave as a mask to the address size
- Effectively behaves like `(uint64_t)(base + index << ilog2(scale))`
- Better idea is to just not expose AVX to 32-bit applications
- Doesn't test VSIB immediate displacement
- This just ends up being base_addr + imm so it isn't too interesting
- We can add more tests in the future if we think we messed that up
- Doesn't test partial fault behaviour
- Because that's a nightmare.
Specifically keeps each instruction test small and isolated so if a
single register fails it is very easily to nail down which operation did
it.
I know some of our ASM tests do a chunk of work and spit out a result at
the end which can be difficult to debug in some cases. Didn't want to do
that which is why the tests are spread out across 16 files for these
single class of instructions.
If we ever get around to fusing ops with shifts in the ConstProp optimizer (may
or may not be worthwhile), this will delete an instruction from things like "or
al, bh".
Even though lsr is the same speed as bfe on Firestorm, I feel if you ask for
garbage you should get garbage C:
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Pointless, upper bits ignored anyway. Deletes piles of uxt and even some 32-bit
instruction moves.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
To load 8-bit sources without bfe'ing for al/bl/cl if the caller knows it
doesn't need masking behaviour, but without lying about the size so the extract
for ah/bh/ch will still work properly.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
For the GPR result, the masking already happens as part of the bfi. So the only
point of masking is for the flag calculation. But actually, every flag except
carry will ignore the upper bits anyway. And the carry calculation actually
WANTS the upper bit as a faster impl.
Deletes a pile of code both in FEX and the output :-)
ADC/SBC could probably get similar treatment later.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Now unused, its former users all prefer LoadPFRaw since they can fold in some of
this math into the use.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Use the raw popcount rather than the final PF and use some sneaky bit math to
come out 1 instruction ahead.
Closes#3117
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Mostly copypaste of Orlshl... we really should deduplicate this mess somehow.
Maybe a shift enum on the core Or op?
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
If we const-prop the required functions and leafs then we can directly
encode the CPUID information rather than jumping out of the JIT.
In testing almost all CPUID executions const-prop which function is
getting called. Worst case that I found was only 85% const-prop rate.
This isn't quite 100% optimal since we need to call the RCLSE and
Constprop passes after we optimize these, which would remove some
redundant moves.
Sadly there seems to be a bug in the constprop pass that starts crashing
applications if that is done.
Easily enough tested by running Half-Life 2 and it immediately hitting
SIGILL.
Even without this optimization, this is stil a significant savings since
we aren't jumping out of the JIT anymore for these optimized CPUIDs.
Most CPUID routines return constant data, there are four that don't.
Some CPUID functions also need the leaf descriptor, so we need to
describe that as well.
Functions that don't return constant data:
- function 1Ah - Returns different data depending on current CPU core
- function 8000_000{2,3,4} - Different data based on CPU core
Functions that need leaf constprop:
- 4h, 7h, Dh, 4000_0001h, 8000_001Dh
Gets us the constant source optimization without more code duplication. And
honestly I prefer the combined presentation.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This option was disabled a few months ago when we switched the server
socket from a filesystem unix socket to an abstract socket.
This partially broke our chroot scripts which relied on this option
existing.
Readds support for an explicitly named abstract socket named from
config.
This is a workaround for dealing with chroots that change users.
They end up changing a user while doing operations and then can't
connect to the FEXServer anymore because environment variables have been
wiped away.
movprfx is invalid to use when the source register matches the movprfx
destination.
This was getting picked up on by `TwoByte/0F_D1.asm` now that RCLSE is
working better now.
The bug that was causing crashes with this was due to inline syscalls.
Now that this is fixed we can re-enable store->load operations.
This allows constant propagation to work significantly better, which
means inline syscalls start working again. This can significantly
improve syscall performance in some cases.
This is most likely to improve performance in dxsetup and vc_redist but
hard to get a real profile.
Additionally this will let us inline cpuid results in the future which
is pretty nice.
Ever since we reordered registers in `X86Enums.h` this has silently been
broken. This wasn't hit because RCLSE has been broken ever since SRA was
added, so inlinesyscalls just weren't ever happening.
Quick fix while I think of a way to more strictly correlate these
registers so it doesn't happen again.
The range was slightly incorrect which mostly wouldn't have caused
issues.
The lowest byte would have just generated slightly less optimal code.
The upper byte could have generated broken code, which our CI couldn't
catch since TSO instructions only get enabled when multiple threads are
in-flight.
Easy enough to fix.
This would have caused core to try and initialize a custom core on
Arm64, which causes a std::function assert because it doesn't support
that.
Users would likely get hit by this immediately since we deleted the
interpreter and shifted all the core numbers.
Originally this was going to use setf8/setf16, but it looks like the approach of
shift-and-test turns out to be faster. As a bonus this is a nice delete-the-code
win :-)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
The only reason we need to XOR arguments for AF is to get bit 4 correct. But if
the operand in question is known to have bit 4 clear, the XOR will be an
effective no-op and can be skipped. This saves an instruction in a bunch of
common cases, like inc/dec. If we dedicated a register to AF to eliminate the
store, we would not save an instruction from this but would still come out ahead
due to an eor turning into a (zero cycle?) mov that can be handled by the
renamer.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Add new synthetic condition codes that do an AND as their relational operator,
testing the result. This is 1 IR op for things like
(A & B) == 0 ? C : D
This can translate to
tst A, B
csel A, B, eq
In the future, if A is the NZCV register and B is a supported immediate, eg
(NZCV & 0x80000000) == 0 ? C : D
this will be able to translate to a single instruction with the appropriate
condition
csel A, B, pl
but that needs RA support.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This provides more robust handling than a signal based approach, as the
suspender is able to wait for the suspendee to reach a suitable position and
flush its context to memory before returning.
This should support most simple cases of SMC, however programs which make use
of separate shared memory mappings for writing and execution are not handled.
The overall approach is the same as is done for linux, where RWX mappings are
protected to RX and then when a write occurs the signal handler invalidates the
faulting page and reprotects it to RWX until code in that page is jitted again.
When an exception occurs, pretend that we were just at the point of JIT entry
so the stack can be unwound to the wow64 SEH handler, which then handles
dispatching the exception to the x86 guest with the restored context.
This allows for running x86 applications under wine without having to run all
of wine under FEX. The JIT is invoked when running application code and then
left when handling NT syscalls or unix calls to e.g. the Vulkan driver.
The MinGW supplied import libraries are incomplete and miss a lot of
functions necessary to implement lower level windows code. To avoid
needing to many resolve every function, pull in .def files from wine
that detail the entire ntdll and wow64 APIs.
These are cut down versions of wine headers containing only what is necessary
for WOW. This shouldn't carry any license implications for FEX, as per the
LGPLv3 license:
```
The object code form of an Application may incorporate material from a header
file that is part of the Library. You may convey such object code under terms
of your choice, provided that, if the incorporated material is not limited to
numerical parameters, data structure layouts and accessors, or small macros,
inline functions and templates (ten or fewer lines in length), you do both of
the following:
a) Give prominent notice with each copy of the object code that the Library is
used in it and that the Library and its use are covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
```
This is blocking performance improvements. This backend is almost
unilaterally unused except for when I'm testing if games run on Radeon
video drivers.
Hopefully AmpereOne and Orin/Grace can fulfill this role when they
launch next year.
This is an atomicFetchCLR, removes two mvn instructions that are back to
back negating the source.
We didn't have this instruction combination in InstCountCI so will be a
bit hard to see.
It is scarcely used today, and like the x86 jit, it is a significant
maintainence burden complicating work on FEXCore and arm64 optimization. Remove
it, bringing us down to 2 backends.
1 down, 1 to go.
Some interpreter scaffolding remains for x87 fallbacks. That is not a problem
here.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
While this interface is usually pretty fast because it is a write and
forget operation, this has issues when there are multiple threads
hitting the perf map file at the same time. In particular this interface
becomes a bottleneck due to a locking mutex on writes in the kernel.
The situations when this bottleneck occurs is when a bunch of threads
get spawned and they are all jitting code as quickly as possible. In
particular Geekbench's clang benchmark hits this hard where each CPU
thread spends ~40% CPU time on all eight CPU threads because they are
stalled waiting for this mutex to unlock.
To work around this issue, buffer the writes a small amount. Either up
to a page-ish of data or 100ms of time. This completely eliminates
threads waiting on the kernel mutex.
- Around a page of buffer space was chosen by profiling Geekbench's
clang benchmark and seeing how frequently it was still writing.
- 1024 bytes was still fairly aggressive, 4096 seemed fine.
- 100ms was chosen to ensure we don't wait /too/ long to write JIT
symbols.
- In most cases 100ms is enough that you won't notice the blip in
perf.
One thing of note is that with profiling enabled and checking the time
on every JIT block still ends up with 2-3% CPUtime in vdso
clock_gettime. We can improve this by using the cyclecounter directly
since that is still guaranteed to be monotonic. Maybe we'll come back to
that if it is actually an issue here.
We can cut down on a few of the generated moves. For
the case where both destinations alias one another,
we can just calculate the high part instead of both of them.
Allows viewing the codegen for cases where conditional zeroing is performed.
Also fixes up the vperm2f variants shorthanding one of the registers
to make everything a little more explicit.
cmov was quite terrible in its implementation. Some things of note:
- NZCV cache would cause store for no reason
- {16,32}-bit would zero extend sources for no reason
- 16-bit would zero extend result for no reason
A bunch of flag testing is still doing a ubfx plus compare against zero
when it could end up being a tst instead, but this is a step in the
right direction and switches over to explicit sized selects.
If we are going to throw away the updated value of CF anyway there is no point
wasting an instruction to invert CF. Add an IR toggle for that so the arm64 JIT
can make better choices.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Changes the helper which all the source uses to still calculate the size
implicitly. This is going to take a while to convert all implicit uses
over to the explicit operation.
Get us started by at least having the IR operation itself be explicit.
Wide shifts under SVE use 64-bit source elements. If a smaller element
overlaps the 64-bit shift element then it uses that shift
eg:
- Src1[15:0] >> Shift[63:0]
- Src1[31:16] >> Shift[63:0]
- Src1[47:32] >> Shift[63:0]
- Src1[63:48] >> Shift[63:0]
- After this point it will switch to the next 64-bit shift element
- Src1[79:64] >> Shift[127:64]
- Src1[95:80] >> Shift[127:64]
- Src1[111:96] >> Shift[127:64]
- Src1[127:112] >> Shift[127:64]
As seen, we can skip the duplication of the scalar element if the OpSize
is 64-bit, this makes MMX emulation slightly more optimal here.
This also means that a few instructions that weren't claimed to be
optimal actually are since they need the duplication operation (which
vixl always labels as a mov).
Hades and the vcruntime hits this very hard in memmove.
`86.56% [JIT] tid 458574 [.] JIT_0x18000c375_0x7fffc94790c8`
```asm
0x00007fffc94790f8: ldaprb w3, [x2]
0x00007fffc94790fc: stlrb w3, [x1]
0x00007fffc9479100: add x1, x1, #0x1
0x00007fffc9479104: add x2, x2, #0x1
0x00007fffc9479108: sub x0, x0, #0x1
0x00007fffc947910c: cbnz x0, 0x7fffc94790f8
```
This performance is terrible because Cortex's LRCPC performance is bottom-tier.
Work around the performance issue by forcing things to do larger moves with vector moves instead.
The only version of this instruction that was generating optimal code
was the one with 64-bit destination and source.
Optimizes the rest of the operating sizes so that they are all optimal
at one instruction translations
16-bit MOVBE is a bit of a special case where it loads 16-bits in to the
bottom of the GPR without clearing the upper bits of the register.
Which means 32-bits or 64-bits depending on operating mode.
Arm64 doesn't support a 16-bit bswap so it needs to operate at 32-bits
instead. We then can insert the resulting bits of the 32-bit rev with a
bfxil in to the lower bits of the resulting destination register.
This allows 16-bit movbe to be optimal now.
Optimal blendps is worst case 2 instructions.
FEX's RA doesn't quite get there since it can't see through multiple
instructions with SRA destinations. That'll be fixed in the future.
Optimal blendps is always one instruction, one is a no-op.
We always hit this.
Cleans up the code which had special cased some 32-bit optimization
which is unnecessary now that both 8-bit and 16-bit are also optimized.
When FEX does a VExtractToGPR, the result is zero extended to the full
GPR register size. This means we don't need to do a zero extend when
storing to a guest GPR.
Makes pextr{b,w} optimal now.
Needs #3088 merged first.
If CI faults out due to a bug then we would have no log as to which
instruction caused the issue.
I find myself adding this each time an assert fires to see what
instruction it was working on. Just add it directly.
I managed to miss a whole section of instructions from the secondary
opsize tables. This resulted in four instructions missing from the
database.
Adds cmppd, pinsrw, pextrw, and shufpd which are all non-optimal
instruction implementations.
In the case that source registers are sequential then this turns in to a
load of the vector constant (2 instructions) and the single tbl
instruction.
If the registers aren't sequential then the tbl turns in to 2 moves and
then the single tbl, which with zero-cycle rename isn't too bad.
Since this is a worst case option this is significantly better than the
previous implementation doing a bunch of inserts which was always 9
instructions.
We should still strive to implement faster versions without the use of
TBL2 if possible but this makes it less of a concern.
Skips implementing it for the x86 JIT because that's a bit of a
nightmare to think about.
The ARM64 implementation requires sequential registers which means if
the incoming sources aren't sequential then we need to move the sources
in to the two vector temporaries. This is fine since we have zero-cycle
vector renames and the alternative is slower.
Hits a whole bunch of common cases, most of which then emit optimal code
generation.
Two cases that use VInsElement hit the RA quirk where the SRA
destination is dead but RA doesn't see it, so it ends up doing a couple
moves. If RA gets fixed then those two moves will go away.
There are definitely still cases that we could emit more optimal code.
Additionally we could implement a TBL2 IR operation to do a LUT approach
for ones we don't cover.
Problem with implementing a TBL2 ir operation is that we have no way to
ensure registers are sequential so we would need to always do moves
```asm
ldr v2, <LUT Table>
mov v0, v16
mov v1, v18
tbl v16.16b, { v0.16b, v1.16b }, v2.16b
```
Which to be fair isn't terrible, and if we're lucky that the guest uses
sequential registers we can naturally get the more optimal code path.
Ideally our RA could push some operations in to sequential registers but
that's not possible currently.
I'll do a follow-up PR that implements TBL2.
Move instruction to itself here is a nop.
Need to be careful about AVX operations which use a different handler
since those might actually zero the upper bits on 128-bit move
{Load,Store}RegisterSRA always loads or stores GPRSize. 8-bit and 16-bit
are vestigial and all OpcodeDispatcher usage will load the full GPR size
(32-bit or 64-bit) and then extract or insert as necessary.
This cleans up a few bits of codegen in InstCountCI.
Currently FEX will always jump out of the JIT any time FCW was getting
written to, ensuring that the softfloat state is setup to rounding at
the time of FCW getting written.
This has the unintended side-effect that even in "x87 reduced precision"
mode we were jumping out of the JIT.
This hit a real world use case of an installer reloading FCW after every
x87 operation and generating a block with 2297 instructions.
Instead when jumping out of the JIT for handling x87 operations, load
FCW and pass it as the first argument of the handler. Setting the
softfloat state at that point.
This helps the installer's hottest block by cutting it down to 1477
instructions. 64.3% of the original size. The code block is still
burning 90% of the CPU time of the installer but the performance is
significantly better while it is doing its decompression.
In order to optimize this installer's block of code more then we will
likely need to optimize out x87 stack usage.
This leaves us with two temporary vectors that the JIT can use.
As of last month we stopped using v2 and v3 as temporaries and these can
now be given back to the JIT.
Ensures that the registers are still sequentially ordered and adds
support for spilling the FPR counts that are aligned by 2 instead of 4.
Adds a couple of instructions to filling and spilling but isn't that big
of an issue.
InstcountCI has some ridiculously large changes just because RA is
starting at a new register number.
This wasn't implemented initially for the interpreter and x86 JIT.
This meant we are maintaining two codepaths. Implement these operations
in the interpreter and x86 JIT so we no longer need to do that.
The emitted code in the x86 JIT is hot garbage, but it's only necessary
for correctness testing, not performance testing there.
When flags are invalidated but we're going to insert a new flag we end
up in a situation where we loaded the prior value from memory, claimed
unknown cache status (they were all invalid!), and then did an insert.
These instructions set all the flags to undefined and moves the
resulting bit in to CF. No need to calculate the deferred flags when
we are about to write over them.
We have supported this since #163 but we haven't been exposing the
feature in hwcap2.
We have exposed it in CPUID this entire time, just not in hwcap2.
Arm64 store with writeback when source register is the same register as
the address is undefined behaviour.
Depending on hardware details this can do a whole bunch of things.
This situation happens when the x86 code does `push rsp` which is quite
common for applications to do. We would then convert this to a `str x8, [x8, #-8]!`
Which results in undefined behaviour.
Now that redundant loads are optimized this showed up as an issue. Adds
a unit test to ensure we don't hit this again.
When the destination overlaps one of the sources we must be careful to
follow a movprfx rule.
```
The destination register must not refer to architectural register state
referenced by any other source operand register of this instruction.
```
We ended up in a situation in the vpmulh{u,}w AVX tests where zm was
overlapping the destination which violated that rule. This also
generated invalid code for this instruction.
```
[INFO] movprfx z6, z4
[INFO] umulh z6.h, p6/m, z6.h, z6.h
```
As seen, we were overwriting one of the sources because the destination
overlapped it. Now instead check if each individual overlap so invalid
code isn't generated.
InstCountCI results aren't affected since this only happens in
situations with multiple instructions.
Now that the RCLSE pass finally optimizes redundant loads again this
optimization that lives in the OpcodeDispatcher can be removed.
With InstCountCI reran, the pblendvb results don't change at all, as
expected.
This is taking steps to start fixing RCLSE which was started by #2700.
Same situation as that PR, since #2170 when we converted
{Load,Store}Context in to {Load,Store}Register we broke this pass
entirely. It hasn't been doing anything for redundant GPRs and FPRs
since at least November of last year.
Technically it was potentially still optimizing redundant MMX
accesses, but it is so broken that it doesn't matter.
Instead of going all in like #2700 did, tear down the pass and start
again. We are now /only/ optimizing redundant context/register loads.
This fixes an issue that comes up commonly where the same register used
as sources was getting loaded twice, causing redundant moves.
`packsswb xmm0, xmm0` for example was generating a four instruction
sequence instead of three instructions because we weren't eliminating
the redundant load.
Going to take reimplementing all the optimizations that this pass does
in steps. This way we can track any regression in the independent steps
unlike what happened in #2700.
Confirmed that Proton/Sonic Mania still works after this.
This is a new procfs symlink path that changes behaviour of binfmt_misc
when exposed. We need to check both procfs/exe and procfs/interpreter
and see if they exist AND also differ.
Once/if they do then we can disable a bunch of checking of paths once
they do. The fallback when none of this is supported has the same
behaviour has previously where it still does all the regular checking.
During binfmt_misc install cmake will check the kernel version for the
raw binfmt_misc writing. Which will never pass until we have a real
kernel version that it is upstreamed in.
For update-binfmts we add a new optional argument where the tool will
drop the flag if the host kernel version isn't new enough to handle the
option.
It turns out that pure SSA isn't a great choice for the sort of emulation we do.
On one hand, it discards information from the guest binary's register allocation
that would let us skip stuff. On the other hand, it doesn't have nearly as many
benefits in this setting as in a traditional compiler... We really *don't* want
to do global RA or really any global optimization. We assume the guest optimizer
did its job for x86, we just need to clean up the mess left from going x86 ->
arm. So we just need enough SSA to peephole optimize.
My concrete IR proposals are that:
* SSA values must be killed in the same block that they are defined.
* Explicit LoadGPR/StoreGPR instructions can be used for global persistence.
* LoadGPR/StoreGPR are eliminated in favour of SSA within a block.
This has a lot of nice properties for our setting:
* Except for some internal REP instruction emulation (etc), we already have
registers for everything that escapes block boundaries, so this form is very
easy to go into -- straightforward local value numbering, not a full into
SSA pass.
* Spilling is entirely local (if it happens at all), since everything is in
registers at block boundaries. This is excellent, because Belady's algorithm
lets us spill nearly optimally in linear-time for individual blocks. (And
the global version of Belady's algorithm is massively more complicated...)
A nice fit for a JIT.
Relatedly, it turns out allowing spilling is probably a decent decision,
since the same spiller code can be used to rematerialize constants in a
straightforward way. This is an issue with the current RA.
* Register assignment is entirely local. For the same reason, we can assign
registers "optimally" in linear time & memory (e.g. with linear scan). And
the impl is massively simpler than a full blown SSA-based tree scan RA. For
example, we don't have to worry about parallel copies or coalescing phis or
anything. Massively nicer algorithm to deal with.
* SSA value names can be block local which makes the validation implicit :~)
It also has remarkably few drawbacks, because we didn't want to do CFG global
optimization anyway given our time budget and the diminishng returns. The few
global optimizations we might want (flag escape analysis?) don't necessarily
benefit from pure SSA anyway.
Anyway, we explicitly don't want phi nodes in any of this. They're currently
unused. Let's just remove them so nobody gets the bright idea of changing that.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Now the calculation of PF is entirely deferred, by inverting our internal
representation of PF. All the (e.g.) logical op needs to do is store the low
8-bits of the result.
This is a bit of a mixed bag. Primary ALU ops all save an instruction, by
skip the XOR. Loading PF takes an extra instruction, that's expected. The tricky
cases are:
* Zeroing PF. This now requires writing 1 instead of 0, which may require an
extra move for the constant. Some of this will go away when we merge PF+AF
into a single register, which is next up on the list. In that case, the
two stores will turn into 1 `or`. So if we need to write a 1 to PF (zeroing
x86 view of PF), that will get absorbed into the or, if we also write AF. If
we leave AF undefined and need to write a 1, that's a single mov instruction
and we couldn't do better anyway if not inverted (since we'd still have a mov
wzr even then). So in view of the future work, this isn't something I'm
concerned about.
* Float comparisons that put Unordered into PF. These require an extra invert to
match the new convention. These are already so unnecessarily bloated that I'm
not convinced I'm making things materially worse here. But we realistically
need multiple destination support in the IR to fix this particular mess.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Now that PF calculation is deferred, the cost of calculating PF correctly should
be tolerable. Remove the speed hack to skip PF. It's fundamentally broken, and
there are enough broken things in FEX as it is that we don't need to maintain
this one ;-)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
AF is calculated as:
((Src1 ^ Src2) ^ Res)[4]
Due to the extract, this is equivalent to
((Src1 ^ Src2) ^ (Res ^ 1))[4]
We already store (Res ^ 1) as the PF byte. So, it suffices to store
AF Byte = Src1 ^ Src2
and then we can recover the flag value
AF = (AF Byte ^ PF Byte)[4]
This saves an instruction from the AF calculation. It does couple PF/AF writes.
In practice, most instructions fall into one of these categories:
* Both PF and AF written together, the coupling is correct.
* PF written but AF invalidated, irrelevant.
* Both invalidated, irrelevant.
None of these require special handling. Where we do need special handling is
when we want to write them separately, in which case we can fix-up the value of
AF as appropriate.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Logical ops leave AF undefined so we can't expect it to be zero after. Mask the
result of lahf to avoid testing UB. These unit tests would regress from the work
in this MR otherwise.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Use an explicit invalidate, so we can zero easily enough if we need to for
debugging later but we can save the instrs ordinarily.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
The AF calculation is a Bfe of an XOR result. We can't defer the XOR (since it
combines multiple inputs into one), but we can & should defer the Bfe. Since AF
is written much more often than it is read, this should come out ahead.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
For now these are trivial to let us refactor without functional changes. Later
in this series, they will be made nontrivial to let us defer AF calculation.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
FEX has a problem with large blocks that uses a ton of constants spread
throughout the block. Once a block gets large enough with enough
constants that have large live ranges, FEX slows down to unusable speeds
due to the register allocator spending more time calculating node
interferences than anything else in the program.
This adds a little heuristic to ensure that constants aren't reused if
the previous value is past a certain distance threshold. This threshold
works well enough that XeSS's pedantic initialization code doesn't have
issues now. See https://github.com/FEX-Emu/FEX/issues/2688 for more
information about that.
FEX itself should work to remove bad constant usages to make this pass
less necessary anyway. In most cases we are materializing duplicated 0,
1, and masks which could be done without a constant entirely.
Maybe once we've improve that enough we could remove this constant
pooling entirely.
To note, this doesn't fix the issue that XeSS causes our register
allocator, this is purely a heuristic workaround.
This class is very expensive to initialize so if you happen to have the
disassembler configuration enabled you were eating a very bad
initialization cost for no reason.
Only initialize the data member if any disassembler runtime option is
enabled, this completely removes the overhead.
With Or, Orlshl, Bfe, and Bfi there were some assumptions made that i8
and i16 operations made sense. Which required us to disable the IR
validation for these operations when it was just added.
This removes the final assumptions about these IR operations supporting
these small operating sizes allowing us to enable the IR validation.
Also a very minor optimization by moving a couple extracts from source
before trying to BFI from it, making RA more optimal.
When moving everything away from implicit size handling, I kept this the
same codegen even though it was uglier.
Now that implicit stuff is mostly done, switch this over to 32-bit
operations. The behaviour of these changes is no functional change, just
cleans up the operations.
Currently in main today, FEX fails to compact OF/CF/ZF/SF and PF.
This is due to recent optimizations with flag calculations on each of
these. Now that we have a centralized location where we compact and set
our internal representation of flags we can do this in one location.
The FXSAVE and FSAVE tag words are written out in different formats,
with FXSAVE using an abridged version that lacks the zero/special/valid
distinction. Switch to using this abridged version internally for
simplicity, and to allow the calculation of zero/special/valid
distinction to be deferred until an fxsave instruction (in the future,
currently the distinction is ignored and only valid/empty states are
possible).
Currently FEX's internal EFLAGS representation is a perfect 1:1 mapping
between bit offset and byte offset. This is going to change with #3038.
There should be no reason that the frontend needs to understand how to
reconstruct the compacted flags from the internal representation.
Adds context helpers and moves all the logic to FEXCore. The locations
that previously needed to handle this have been converted over to use
this.
32-bit or 64-bit addition without carry-in. This matches the baseline hardware
semantic. Generalizing to support other cases can come later, this should be a
win already.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
For correct carry/overflow behaviour, we need to use a compare of the right size. The existing logic
to look at the source sizes doesn't work for this, since a 32-bit NEG instruction will compare a
32-bit source with a 64-bit _Constant(0) .. which needs a 32-bit compare but the existing logic
would use a 64-bit compare. This is not yet a bug fix, since the overflow code is currently in
software for 32-bit negates so it's irrelevant. But it should prevent regressions from using native
compares later in this series. Presumably this was intended all along but left as-is to avoid
disturbing instcountci once noticed. Time to disturb CI!
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
AddNZCV is a new op to return the NZCV for an addition directly, which lets us
skip software flag calculation in some cases. In the future it would be nice to
fuse this into the Add itself as a second destination to avoid repeating the
addition, but that's a very involved change and right now I'm building FEX on an
old Chromebook because my M1 kernel is FUBAR.
Similarly, SubNZCV returns flags for Sub. This has the extra twist of needing to
invert the carry bit due to the inverted definition between arm64 and x86_64.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This will let us reuse it in some cases. In some of these
implementations there is a bad code smell around using the zero register
but that isn't going to get solved in this commit.
A bunch of the AES operations take a zero register upfront and we
currently materialize it for each instruction.
Considering that most AES operations are used back to back, we can
eliminate these materializations by caching it between instructions.
Additionally removes a move in the optimal case when destination matches
the state register, which is exactly what the SSE operation ends up
doing.
AESKeyGenAssist has an edge case that if the destination RA overlaps the
zero register then we still need to eat a move, hopefully doesn't happen
too frequently in practice. This is also the lesser used instruction so
it isn't a big deal. RA constraints could solve that still.
This instruction has xmm0 be one of the implicit sources. We were
loading xmm0 twice. #2700 would also fix this but that breaks other
things for some reason.
We can load the swizzle table from our constant pool now. This removes
the only usage of VTMP3 from our Arm64 JIT.
I would say the this is now optimal for the version without RCON set.
With RCON we could technically make some of the move of the constant
more optimal.
Saw a few locations in here that we operate things at 64-bit
unconditionally around pointer calculation. Will be coming back for
those when running in 32-bit mode.
This is the last of the implicit sized ALU operations! After this I'll
be going through the IR more individually to try and remove any
stragglers.
Then should be able to start cleaning up and actually optimizing GPR
operations.
These functions are called a lot....a lot a lot.
Optimize these in to a couple of ALU operations instead of a whole
table lookup. Confirming with output assembly that this becomes more
optimal.
This now improves the instruction implementation from 17 instructions
down to 5 or 6 depending on if the host supports SVE.
I would say this is now optimal.
The range check and clamping is necessary in the cases of passing x86
shift amounts directly through VUSHL/VSSHR.
Some AVX operations are still using these with range clamping. A future
investigation task should be the check if they can be switched over to
the wide variants that we implemented for the SSE instructions.
When consuming our own controlled data, we don't want the range clamping
to be enabled.
The number of times the implicit size calculation in GPR operations has
bit us is immeasurable and was a mistake from the start of the project.
The vector based operations never had this problem since they were
explicitly sized for a long time now.
This converts the base IR operations to be explicitly sized, but adds
implicit sized helpers for the moment while we work on removing implicit
usage from the OpcodeDispatcher.
Should be NFC at this moment but it is a big enough change that I want
it in before the "real" work starts.
Noticed that we hadn't ever enabled this, which was a concern when our
GPR operations weren't as strict about leaving garbage in the upper bits
when operating as a 32-bit operation.
Now that our ALU operations are more strict about enforcing upper bit
zeroing we can enable this.
This causes Half-Life: Source FPS to get to > 200FPS finally. Causes
significant performance improvements for 32-bit games because we're no
longer redundantly moving registers before and after every operation.
Causing a bunch of 3-4 instruction sequences to convert to 1.
RAValidation was making an assumption that GPR register class would only
have up to 16 registers for either SRA or dynamic registers.
When running a 32-bit application we allow 17 GPRs to be dynamically
allocated, since we can take 8 back from SRA in that case.
Just split the two classes in the RAValidation pass since they will
never overlap their allocation.
Fixes validation in `32Bit_Secondary/15_XX_0.asm` locally that changed
behaviour due to tinkering.
When the application calls exit_group we no longer need to care about
cleanup because the entire process group is leaving.
Just immediately call exit group and get out. Might revisit this in the
future.
Fixes#2752
This previously used `Round_Nearest` which had a bug on Arm64 that it
actually was always using `Round_Host` aka frinti.
Ever since 393cea2e8ba47a15a3ce31d07a6088a2ff91653c[1] this has been fixed
so that `Round_Nearest` actually uses frintn for neaest.
This instruction actually wants to use the host rounding mode.
Once issue with this is that x87 and SSE have different rounding mode
flags and currently we conflate the two in our JIT. This will need to be
fixed in the future.
In the meantime this restores behaviour that it actually uses the host
rounding mode, which fixes black screen and broken vertices in Grim
Fandango Remastered.
[1] e89321dc60 for scalar.
1) In the case that we are converted a GPR, don't zero extend it first.
2) In the case that the scalar comes from memory, load it first in an
FPR and converted it in-place.
These are now optimal in the case of AFP is unsupported.
This extension was added with seemingly Cortex-A710 and turns this
instruction in to two instructions which is quite good.
Needs #2994 merged first.
Huge thanks to @dougallj for the optimization idea!
If the named constant of that size gets used multiple times then just
use the previous value if it was in scope.
Makes addsubp{s,d} and phminposuw more optimal for each that are in a
block.
Needs #2993 merged first.
Use a named constant for loading the sign inversion, then EOR the second
source and just FAdd it all.
In a vacuum it isn't a significant improvement, but as soon as more than
one instruction is in a block it will eventually get optimized with
named constant caching and be a significant win.
Thanks to @rygorous for the idea!
This takes the two independent VSXT{U}N{2,} operations and merges them
in to a single IR operations.
In some cases this can result in a more optimal implementation since
there is no need for moves inbetween.
Given the operation is:
Dst = Vector1 / Vector2
If Dst and Vector1 alias one another, then we can just perform
the division as is without any moving of data around.
Zero-extension will already occur if necessary upon storing.
Also we can join the AVX and SSE implementations together and get
rid of some template instantiations, now that the only differing
behavior is removed.
This now uses the new load element IR operation and makes these
instructions optimal.
LRPCPC3 will introduce instructions in the future for TSO emulation to
help these operations, but that doesn't exist today.
This can be done without storing any data to memory and also
compressing the amount of instructions being used.
Thanks to @dougallj for the optimization suggestions.
VRev32 matches Arm64 semantics directly.
LoadNamedVectorConstant allows FEX to quickly load "named constants".
This will allow us to have specific hardcoded vector constant values
that we can load with a ldr(State)+ldr(Value) and will be more abused in
the future.
This also allows us to do a very simple optimization in the future where
we can optimize away redundant loads of these loads if they are used
multiple times in the same block. (Not implemented here).
Two optimizations here:
1) The final VInsElement was generating three instructions
- This itself could have been change to vzip, which would have
removed two instructions.
2) Optimize how the pairwise elements are calculated to shave one
instruction off the calculation.
- addp odd elements and even elemnts first
- Then transpose those elements
- Then use one final addp to generate the result in the correct
order.
The ext+uabdl+addp pairs of operations could be reordered to shave off
one temporary register usage if we really care later.
Can be slightly more optimal with a slightly change algorithm but will
require implementing some IR ops which can be put off. It's only about
an instruction savings.
Even on platforms without SVE these have improved slightly but the real
improvement comes from SVE.
Adds some new InstCountCI files for SVE128 enabled testing.
Also enables SVE128 in the VEX maps. Host features should probably
enable SVE128 when SVE256/AVX is enabled, but that isn't the case today.
This matches x86 vector shift behaviour closely for ps{rl,ra,ll}{w,d,q}
where the vector is shifted by a scalar value that is 64-bits wide.
Anything larger than the element size will set that element to zero.
With SVE we have some new wide element shifts that match this behaviour
exactly (except supports wide shift sources rather than scalar).
This is a significant improvement even on platforms that only support
128-bit SVE.
With ASIMD FEX would never optimize BSL out of fear if some registers
overlapped it would break things. So it had previously always moved to a
temporary first and then moved the result back out when done.
Now instead check upfront if any of the source registers overlap the
destination. If the destination register overlaps any of the three
sources we can bsl, bit, or bif depending on which register gets
overlapped.
Worst case the destination doesn't overlap any of the source registers
and still needs these moves.
When clearing multiple flags it is more optimal to load the mask
constant in to a register and then clear with a single and/bic.
Back to back bfi is actually less optimal due to dependency tracking.
With #2911, this is a total win since this hits an edge case with
constant loading that #2911 fixes.
Since all we're going to be doing is an insert as the final operation,
in the cases where our source is a vector, we can specify the size of
the vector rather than the size of the element to avoid doing unnecessary
zero-extending.
When dealing with source vectors, we can use the vector length
rather than using a smaller size and zero extending the register,
especially since the resulting value is just inserted into another
vector.
We can specify the full vector length when dealing with a source vector
to avoid zero-extending the vector unnecessarily. When dealing with a
memory operand, however, we only want to load the exact source size.
These have the same behavior and only differ based on element size,
so we can join the implementations together instead of duplicating
them across both functions.
Like the changes made to the xmm to xmm case, since we're going to be storing
a 64-bit value, we don't directly need to zero-extend the vector on a load.
In the event that we have a full length vector, we can just load and move
from it, which gets rid of a little bit of mov noise. Since all we intend
to do is perform an insert from one vector into another, we don't need the
zero-extending behavior that an 64-bit vector load would do.
For a bunch of cases that act as broadcasts (where all
indices in the imm8 specify the same element), we
can use VDupElement here rather than iterating through.
This paves the way to optimizing pushes in to both push operations and
push pair operations to more optimally match Arm64 push support.
While this does the first step for supporting the base push, we'll leave
optimizing push pairs to future work.
This is a bit of tricky operation where due to our our usage of SSA, the
incoming source isn't guaranteed to end its live-range at this
instruction.
This gives us a behaviour where to be optimal we need to take different
paths depending on if the incoming address register is the same as the
destination node.
Once we have form of RA constraints or non-SSA IR form that can
guarantee this restriction then this will go away.
FEX doesn't use the platform register on wine platforms so there is no
reason to save and restore it.
On Linux we can still use it at some point but for now it isn't part of
our RA.
Changes the idiom used for constant mask generation to a ternary.
This pattern is definitely used elsewhere in code but we can get rid of
all instances here.
This takes a similar approach to deferred signal handling and allows any given
thread to be interrupted while running JIT code by protecting the appropriate
page as RO. When the thread then enters a new block, it will try to acccess
that page and segfault. This is safer than just sending a signal to the thread
as that could stop in a place where JIT context couldn't be recovered correctly.
With WOW, all allocations from 64-bit code use the full address space
and limiting is handled on the syscall thunk side so theres need to
worry about STL allocations stealing AS.
This relies on wine's behaviour passing through linux paths and env vars,
so that the config in the user's home directory can be accessed outside
of the wine prefix.
Due to Intel dropping support for legacy segment registers[1] there is a
concern that this will break legacy 32-bit software that is doing some
magic segment register handling.
Adds some simple telemetry for 32-bit applications that when they
encounter an instruction that sets the segment register or uses a
segment register that the JIT will do a /relatively/ quick four
instruction check to see if it is not a null segment.
It's not enough to just check if the segment index is 0 or not, 32-bit
Linux software starts with non-zero segment register indexes but the LDT
for each segment index is a null-descriptor.
Once the segment address is loaded, the IR operation will do a quick
check against zero and if it /isn't/ zero then set the telemetry value.
A very minor optimization that segment registers only get checked once
per block to ensure overhead stays low.
[1] https://www.intel.com/content/www/us/en/developer/articles/technical/envisioning-future-simplified-architecture.html
- 3.6 - Restricted Subset of Segmentation
- `Bases are supported for FS, GS, GDT, IDT, LDT, and TSS
registers; the base for CS, DS, ES, and SS is ignored for 32-bit
mode, same as 64-bit mode (treated as zero).`
- 4.2.17 - MOV to Segment Register
- Will fault if SS is written (Breaking anything that writes to
SS).
- Will not fault if CS, DS, ES are written (Thus it sets the
segment but gets ignored due to 3.6).
While it would be bizarre if this actually occurred frequently
in practice, we can still tune it so there's no subpar assembly
output in the cases it actually does happen.
It is not an external component, and it makes paths needlessly long.
Ryan seemed amenable to this when we discussed on IRC earlier.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Now all vbroadcast implementations go down the more optimal path.
For non-SVE 128-bit cases where we only have 128-bit wide registers,
we behave like ld1rqb and just act as a normal 128-bit load for
interface convenience.
In the case of running on a 128-bit SVE system this predicate wasn't
setup. Since we never had any predicate usage before this wasn't an
issue. Now that #2914 is using the 128-bit predicate we need to make
sure that we are generating it.
Allows the implementations of the vbroadcast instructions to perform the
load and broadcast in one operation as opposed to doing the load and then
broadcast separately.
Notably, the broadcasting loads can also be used on systems that have SVE 128-bit
support as well, not only 256-bit.
On non-SVE systems, we use the equivalent AdvSIMD instructions.
Instead of allocating a temporary copy of the string, return a view of
it instead. Should improve the performance of system calls that take
file paths. Since it was allocating a string for every single syscall
that uses them in this case.
For config values that were string objects we were unnecessary creating
copies each time the string was accessed.
Convert the () operator over to returning a reference.
The current implementation uses orr excessively. This has FEX missing
hardware optimization opportunities where some CPU cores will zero-cycle
move constants that fit in to the 16-bits of movz/movk.
First evaluate up front if the number of 16-bit segments is > 1, in
those cases we should check if it is a bitfield that can be moved in one
instruction with orr.
After that point we will use movz for 16-bit constant moves.
Additionally this optimizes the case where a constant of zero is loaded
to be a `mov <reg>, zr` which gets renamed in most hardware.
Commonly we are doing a BFI into a 32-bit register, which is hitting the
ubfx (lsr alias) path.
In the case of 32-bit destination we can also do a regular move, which
will take advantage of CPU's rename functionality and give a minor speed
boost.
Didn't notice this in the previous PR, When DUMPIR=stderr without and
selection of where to place it in PASSMANAGERDUMPIR it was supposed to
put the dumper at the end of the passes.
We need to make sure that it it placed at the end of the passes rather
than current `it`.
This will allow investigating the Arm64 directly next to the test, plus
publicly linking directly to badly behaving tests.
Perfect for nerdsniping implementations.
We can perform the SQRT first and then broadcast 1.0 into the destination
since all the intermediary work is done, meaning we don't have to worry
about Dst and Vector aliasing one another.
Pretty sure this is why CI is unhappy. If a test in a different file has
the same name then it is highly likely to conflict when nasm is
generating files and will overwrite and erase, causing CI to break.
Include the incoming json filename as part of the asm keys so it can't
conflict here.
If DumpIR is enabled but the PassManagerDumpIR option isn't enabled then
this currently does nothing.
As a convenience, enable dumping the final optimized IR if an option
hasn't been specified.
None of the instructions here are optimal, a couple are close.
Will be splitting each of the map tables in to their own json files
since each one can get fairly large.
We need to move the modifier enum out of the SVEMemOperand class
since it's also used with adr. Plus, this can also be convenient
not being tied down to the class itself.
This also makes accessing modifiers less noisy, since the class
When a 32-bit adcx instruction was encountered, it was getting treated
as a 16-bit adcx instruction instead. This is because of the 0x66 prefix
required to handle this instruction.
Adds a unit test to ensure it doesn't break again.
Implements CI for tracking instruction counts for generate blocks of
code when transforming from x86 to ARM64 assembly.
This will end up encompassing every instruction in our instruction
tables similarly to how our assembly tests try to test everything in our
instruction tables.
Incidentally, the data for this CI is generated using our assembly
tests. By enabling disassembly and instruction stats when executing a
suite of instructions, this gives the stats that can be added to a json
file.
The current implementation only implements the SecondGroup table of
instructions because it is a relatively small table and has known
inefficiencies in the instruction implementations. As this gets merged I
will be adding more tables of instructions to additional json files for
testing.
These JSON files will support adjusting CPU features regardless of the
host features so it can test implementations depending on different CPU
features. This will let us test things like one instruction having
different "optimal" implementations depending on if it supports SVE128,
SVE256, SVEI8MM, etc.
This initial instruction auditing is what found the bug in our vector
shift instructions by size of zero. If inspecting the result of the CI
run, you can tell that these instructions still aren't "optimal" because
they are doing loads and stores that can be eliminated.
The "Optimal" in the JSON is purely for human readable and grepping
ability to see what is optimal versus not. Same with the "Comment"
section.
According to my auditing spreadsheet, the total number of instructions
that will end up in these json files will be about 1000, but we will
likely end up with more since there will be edge cases that can be more
optimal depending on arguments.
This was confusingly split between Arm64Emitter, Arm64Dispatcher, and
Arm64JIT.
- Arm64JIT objects were unnecessary and free to be deleted.
- Arm64Dispatcher simulator and decoder moved to Arm64Emitter
- Arm64Emitter disassembler and decoder renamed
- Dropped usage of the PrintDisassembler since it is hardcoded to go
through a FILE* type
- We instead want its output to go through LogMan, which means using a
split Decoder+Disassembler object pair.
- Can't reuse the object from the vixl simulator since the simulator
registers the decoder as a visitor, causing the simulator to execute
while disassembling instructions if reused.
- Disassembly output for blocks and dispatcher now output through Logman
- Blocks wrapped in Begin/End text for tracking purposes for CI.
We don't currently have a device in CI that can run SVE with 128-bit
width registers. Until we have a device with this, make sure the vixl
simulator is also running the ASM tests in this width.
This was causing test failure locally where some values were set to
uninitialized data. Ensure that gregs, YMM, and MMX registers are all
zero initialized.
Requires the IR headerop to house the number of host instructions this
code is translating for the stats.
Fixes compiling with disassembly enabled, will be used with the
instruction count CI.
This is incredibly useful and I find myself hacking this feature in
every time I am optimizing IR. Adds a new configuration option which
allows dumping IR at various times.
Before any optimization passes has happened
After all optimizations passes have happened
Before and After each IRPass to see what is breaking something.
Needs #2864 merged first
This is a /very/ simple optimization purely because of a choice that ARM
made with SVE in latest Cortex.
Cortex-A715:
- sxtl/sxtl2/uxtl/uxtl2 can execute 1 instruction per cycle.
- sunpklo/sunpkhi/uunpklo/uunpkhi can execute 2 instructions per cycle.
Cortex-X3:
- sxtl/sxtl2/uxtl/uxtl2 can execute 2 instruction per cycle.
- sunpklo/sunpkhi/uunpklo/uunpkhi can execute 4 instructions per cycle.
This is fairly quirky since this optimization only works on SVE systems
with 128-bit Vector length. Which since it is all of the current
consumer platforms, it will work.
We need to know the difference between the host supporting SVE with
128-bit registers versus 256-bit registers. Ensure we know the
difference.
No functional change here.
This allows use to both enable and disable regardless of what the host
supports. This replaces the old `EnableAVX` option.
Unlike the old EnableAVX option which was a binary option which could
only disable, each of these options are technically trinary states.
Not setting an option gives you the default detection, while explicitly
enabling or disabling will toggle the option regardless of what the host
supports.
This will be used by the instruction count CI in the future.
Moves the dummy handlers over to this library. This will end up getting
used for more than the mingw test harness runner once the instruction
count CI is operational.
This was a debug LoadConstant that would load the entry in to a temprary
register to make it easier to see what RIP a block was in.
This was implemented when FEX stopped storing the RIP in the CPU state
for every block. This is now no longer necessary since FEX stores the
in the tail data of the block.
This was affecting instructioncountci when in a debug build.
I use this locally when looking for optimization opportunities in the
JIT.
The instruction count CI in the future will use this as well.
Just get it upstreamed right away.
`eor <reg>, <reg>, <reg>` is not the optimal way to zero a vector
register on ARM CPUs. Instead we should move by constant or zero
register to take advantage of zero-latency moves.
While the ENABLE_LLD and ENABLE_MOLD options are nice, they don't handle
the case when the linker of `lld` or `mold` doesn't match the compiler.
This particularly crops up when overriding the C compiler to a new
version of clang but the globally installed `ld.lld` is still the old
clang version.
This then causes clang to fail with unusual errors when upstream breaks
compatibility with itself.
Easy enough to use by passing the linker to cmake:
`-DUSE_LINKER=/usr/bin/ld.lld-15`
This also removes the ENABLE_LLD and ENABLE_MOLD options to use
USE_LINKER directly.
- ldd: `-DUSE_LINKER=lld`
- mold: `-DUSE_LINKER=mold`
Example of compiler failure when built with clang-15 but attempting to
link with ld.lld 14:
```bash
ld.lld-14: error: unittests/APITests/CMakeFiles/Filesystem.dir/Filesystem.cpp.o: Opaque pointers are only supported in -opaque-pointers mode (Producer: 'LLVM15.0.7' Reader: 'LLVM 14.0.6')
```
This needs to default to 64-bit addresses, this was previously
defaulting to 32-bit which was meaning the destination address was
getting truncated. In a 32-bit process the address is still 32-bit.
I'm actually surprised this hasn't caused spurious SIGSEGV before this
point.
Adds a 32-bit test to ensure that side is tested as well.
This is more obvious. llvm-mca says TST is half the cycle count of CMN
for whatever it's defaulting to. dougallj's reference shows both as the
same performance.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
In the non-immediate cases, we can amortize some work between the two
flags to come out 1 instruction ahead.
In the immediate case, costs us an extra 2 instructions compared to
before we packed NZCV flags, but this mitigates a bigger instr count
regression that this PR would otherwise have. Coming out ahead will
require FlagM and smarter RA, but is doable.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Same technique as the left shifts. Gets rid of all our COND_FLAG_SET
use, which is good because it's a performance footgun.
Overall saves 17 instructions (!!!!) from the flag calculation code for
`sar eax, cl`.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Similar to the immediate case, but now we select between the entire old
and new NZCV registers. This is faster than selecting each bit
independently. Saves 11 instructions for calculating flags for "shl eax,
cl".
It is undefined in this case. We prefer to zero (rather than preserve
the existing value) as it avoids a costly RMW of the NZCV register.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We need to be careful to preserve V if needed. For `shl 1` and `shr 1`,
saves 2 instruction overall compared to before the PR. For `sar 1`,
saves 3 overall.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We can just return zero, no need to do a pointless Bfe. Saves yet
another instruction for GetPackedRLAG in a test I'm looking at.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We know that bit 0 is CF, so we can do CF first and then avoid setting
Original to 2 (for reserved) with a silly `or xzr, #2` instruction.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Faster sign/negate testing for 32-bit/64-bit inputs. This could maybe be
extended to 8/16-bit if we have FlagM but that's for later.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
If we can prove that a flag bit could not possibly be set, we can use
orlshl rather than bfi, which can be more efficient.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
In some cases we just want to insert in one bit at a time, add a helper
to zero the 4 flags together so we can avoid the extra RMW cycle at the
beginning.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We can set N more efficiently with some bit math, and zero ZCV at the
same time. In the future we'll be able to use TST for this to make it
even faster.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Later, this will let us take advantage of the arm64 flags. For
now, this just turns some strb's into bfi's for dubious benefit.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
If we read or write NZCV flags we need to call CalculateDeferredFlags on
block boundaries, if only to flush out the cached copy.
Also, when leaving a block we call it to flush out. This is annoyingly
invasive but I don't know of a better way to do this that doesn't
involve rearchitecting the dispatcher.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We'll add an extra caching layer in a moment so can't call _LoadFlag
directly and expect correct results.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Maps to arm64 tst, except properly SSA. This will need some RA support
to avoid redundant mrs/msr sequences.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
On arm64, orr (with a shifted register) is maybe fewer cycles and
definitely easier on the RA than bfi (=> fewer moves generated). So,
it's preferred when we know the corresponding bit is 0 in the
destination.
It's not useful on other targets, so it's gated behind a backend feature
bit.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reserve 4 bytes of "flags" to model the 32-bit arm64 NZCV register, so
we can start porting FEX's flag handling code over to using NZCV without
needing the whole compiler to be aware of instructions that might
clobber host flags.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This IR operation was limited to GPR only previously for the values
getting compared.
This adds support for GPRPair (and technically FPR) so that it can be
used directly with GPR pairs. I say technically FPR because the
IREmitter disallowed FPRs for the comparison, but this was already
supported in all of the backends, we just didn't ever use it.
Some minor changes to the constant prop pass to ensure that we don't try
to propagate a select in to a CondJump, otherwise pair comparisons would
be duplicated. This code is expecting to be able to merge a simple
comparison in to a `cbnz`, which doesn't happen with GPR pairs.
When cmake's `install` function is invoked with a relative path, then it
is interpreted as being relative to the `CMAKE_INSTALL_PREFIX` variable.
This variable follows both `DESTDIR` and `CMAKE_INSTALL_PREFIX` so it is
best to use relative addresses in the install path.
Thanks for the report Mike!
Fixes#2849
Currently only does a build, doing a CI run means figuring out why
TestHarnessRunner doesn't find libraries correctly.
I want to ensure we don't break building at least while sorting out the
rest of this.
Inspired by #2832 by going through the source and removing uses of
`assert`.
assert doesn't work for us in debug builds because some games will
capture SIGABRT and continue running. So we need to use FEX's built in
assert handlers which call `FEX_TRAP_EXECUTION` which will take down the
FEX process as expected.
There wasn't too much usage of this in the source, so this is relatively
straightforward.
We can move asserts and the base opcode into the implementing function.
While we're at it, we can add an assert to ensure predicate registers are in range.
We can centralize the base opcode and some of the asserts in the implementing function.
We can also add an assert that validates the predicate register range
and also make the offset assertions much more informative.
Since everything we need is already in the ARMEmitter namespace, we
don't need to qualify all type usages, which reduces the verbosity
a little during reading.
This is actually already implemented but was grouped with the
SVE floating-point convert to integer group. We can extract this
out to be organized a little nicer.
We can move the op into the implementing function and get rid of
an unnecessary function.
We can also add an assert to ensure the predicates are valid as well.
This test was written to test SMC where one thread is doing execution
while the other thread is modifying.
According to the printf documentation it is supposed to "wait for code
to be modified" but actually it was testing a race between a printf on
one thread and the primary thread modifying the code.
Fix this test so it is actually waiting for code modification to happen
rather than testing a race condition. This is likely what the original
author intended.
CI is hitting this flake more frequently now because it is even faster
it seems, so fixing this test is necessary to resolve these flakes.
insert() will only ever perform an insertion if the relevant element
doesn't exist within the set, so we were doing an unnecessary lookup
in two spots.
We don't use emplace() here because it will need to construct the
key for the element inside the allocated node (since the key may
be non-copyable/non-movable), causing an allocation even if an insert
doesn't actually occur.
Conversely, insert() will not need to allocate and construct a node
ahead of time if an element already exists in the set.
Every time I look at this I can never remember which node is the value
node and which one is the store node.
Rename the opaque `Node` to `ValueNode` and have documentation comments
to explain which one it is. This way I won't forget again.
Tests for the regression from 7e6bb04db ("OpcodeDispatcher: Extract
CalculatePF"). This fails on main but passes with this PR.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Inspired from: https://github.com/dotnet/runtime/issues/8072
Currently FEX is /very/ heavy handed with our backpatching where we wrap
every backpatched loadstore with `dmb ish`.
This can be relaxed slightly according to the linked issue.
For TSO load instructions the instruction sequence changes to:
ldr <args>;
dmb ld; <-- Slightly less strict dmb
For TSO store instructions the instruction sequence changes to:
dmb ish; <-- Still the all encompassing dmb
str <args>;
For backpatching loadstores this does the same thing where only one side
needs the nop and it uses the same instruction sequence when
backpatched.
The minor change is that on load backpatching, we are no longer backing
up a single instruction, instead just re-executing the instruction we
patched directly.
Took a long time to come back to this (Last looked in August 2020).
Previously when I was implementing this idea it didn't work, but that
was because our CompareExchange operation was broken back then. With the
CAS now, it should just work.
We expect that PF is written more often than it's read, so we want to
get the expensive popcount out of the hot path. (Thank you to Dougall
for suggesting that.)
There are two cases:
1. PF is written by an integer instruction. In this case, we calculate
with the formula `popcount(x ^ 1) & 1`.
2. PF is written by a float instruction, copying a host flag.
What we really want is to defer the relatively expensive popcount. So,
to unify these cases, we have integer instructions write `x ^ 1` and
(unchanged) float instructions write the host flag. Then, when reading
PF, we do `popcount(value) & 1` on the byte read in.
If PF is written but not read, this saves the expensive popcount and
leaves only the cheap xor.
If PF is written by an integer op and read, this maybe shuffles some
code but does not materially change anything.
If PF is written by a float op and read, this is worse because now we're
doing an extra pointless popcount. This is a tradeoff... However, this
is only relevant to unordered float comparisons, which I expect to be
obscure for games. So this should be worth it over all (for games, if
not weird numerical computing workloads).
How does this connect to my register zeroing quest? The constant folding
code doesn't currently deal with FPRs and I'm not in a mood to change
this. So before, a block ending with `xor eax, eax` would still do a
popcount for PF. Now it just writes a constant 1 since the xor constant
folds and the popcount never happens at all.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
For flag calculation after moving a constant. This cleans up the code
generated for zeroing at the end of a block.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Lets us deduplicate the asserts and put them in one spot. We can also
improve the assert message to also indicate the valid range.
While we're in the area, we can collapse a few case paths
as a result of this assert movement.
Usually uploading of results takes about two seconds.
Sometimes github's connection to the runner flakes and it stalls out the
upload action for some reason.
Github's default timeout is SIX HOURS.
Change this to a one-minute timeout on the upload step so it quickly
goes away when Github's internet flakes out.
For #2804 so it can compare if GPRs match more easily.
Only adding to GPR and Literal types, since its ambiguous what this
would mean for the memory accessing types. Going to leave those other
ones alone for now.
Two bugs here that caused thunking X11 thunking in Wine/Proton to not
work.
The easier of the two. The various variadic functions that we thunk
actually take key:value pairs where the first is a string pointer, and
the value can be various things.
We need to handle these as true key:value pairs rather than finding the
first nullptr and dropping the remainder.
Additionally, there are 12 keys that specify a callback that FEX needs
to catch and convert to host callable. Wine is the first application
that I have seen that actually uses this. If these callbacks aren't
wired up then it it can miss events.
The harder of the two problems is the `libX11_Variadic_u64` function was
subtly incorrect. Nothing had previously truly exercised this and my
test program didn't notice anything wrong while writing it.
The first incorrect thing was that it was subtracting the nullptr ender
variable before the stack size calculation, causing the value to
overwrite the stack if the number of remaining elements was event.
Secondly the assembly that was storing two elements per step was
decrementing the counter by 8 instead of two. Didn't pick this up before
since I believe the code was only hitting the non-pair path before.
This gets Proton thunking working under FEX now.
I was hitting an issue where thunking in Wine+Vulkan applications was breaking
unless both OpenGL and Vulkan was enabled.
Turns out this was because Vulkan enabled only XCB, which didn't enable
X11. So when XCB is thunked but X11 isn't, this causes weird issues
where X11 calls in to XCB functions and gets in desync'ed state. Causing
hangs to appear in xcb_take_socket.
Now we can enabled just `Vulkan` as a thunk and it'll work fine.
```
(gdb) bt
from target:/usr/lib/fex-emu/HostThunks//libvulkan-host.so
```
Now that we have the helper for encoding immediate shifts,
we can trivially implement the remaining missing instruction
in the bitwise logical unpredicated group.
Prevents the code invalidation mutex from being locked as shared recursively,
since it is locked before entering ThreadExitFunctionLink and would end up
being locked again by ThreadAddBlockLink.
This fixes a deadlock on Windows.
This lets us deduplicate the behavior rather than open-coding it everywhere
and also makes it nicer to implement instructions that make use of this
encoding pattern.
Fixes#2754
These panicking fallbacks are at times not ending up in as plt calls
for some reason that I haven't been able to reproduce locally.
So far the only way I can reproduce is building with Canonical's PPA
build system, since rebuilding locally didn't resolve the issue.
This will change the failure mode from these panicking asserts happening
at call time, to dlopen failing during relocation when loading the
thunk. Which LD_DEBUG=all can be used for debugging relocation failure
in that case.
Allows us to consume an array of strings and convert it to an mask of
enum values. This is a quality of life change that allows us to specify
a mask of options.
The first configuration option added to support this is to control the
vixl disassembler. Now by default the vixl disassembler doesn't
disassemble any blocks and needs to be enabled individually.
eg:
```
FEXLoader --disassemble=blocks <args>
FEXLoader --disassemble=dispatcher <args>
FEXLoader --disassemble=blocks,dispatcher <args>
```
Has the additional convenience option of just passing in numbers as
well.
```
FEXLoader --disassemble=2 <args>
FEXLoader --disassemble=1 <args>
FEXLoader --disassemble=3 <args>
```
Also of course all of this works through environment variables.
```
FEX_DISASSEMBLE=blocks FEXInterpreter <args>
FEX_DISASSEMBLE=dispatcher FEXInterpreter <args>
FEX_DISASSEMBLE=blocks,dispatcher FEXInterpreter <args>
```
While only used fairly sparingly now, this is likely to have some
additional configurations using this in the future. Since we already
have some configs that are basically using enums, but just by doing
string comparisons.
This was asked for by a developer, so I figured I would throw it
together quick.
Ensures that we handle the AVX2 VSIB byte in a decent way.
As is, we can't compute the [index * scale] variant portion
of the entire address operand, since the scale needs to act
on every element of the vector after sign extension.
What we can do though, is compute the base address and add
the displacement to it ahead of time though.
All of these IR operations were being fairly inefficient in their
address calculation. All of these are known using power of 2 stride
indexing. So all of these can be converted from three instructions to
one.
These are always used for x87 stack accesses so each one gets an
improvement.
Before:
```asm
0x0000ffff6a800248 d2800200 mov x0, #0x10
0x0000ffff6a80024c 9b007e80 mul x0, x20, x0
0x0000ffff6a800250 8b000380 add x0, x28, x0
0x0000ffff6a800254 fd417805 ldr d5, [x0, #752]
```
After:
```asm
0x0000ffff91e80240 8b141380 add x0, x28, x20, lsl #4
0x0000ffff91e80244 fd417805 ldr d5, [x0, #752]
```
Currently we're clearing icache including the data that lives on the
tail of the block. Instead only clear the code that the was emitted and
not tail data.
Additionally only disasm the code rather than all the tail data as well,
as it gets unwieldy if viewing.
If we are loading exactly the flags we need from the RFLAGS (ensuring we
don't load the reserved flag in bit 1) then we don't need to do a mask
on the result.
Additionally there is some bad code-motion around selects that was
causing SBFE operations to occur on constants. Ensure that we const-prop
any SBFE operations to clean this up.
This PR along with #2783 causes FMOV blow-up to go from 41 instruction
to 31 instructions.
Noticed this when inspecting some code that was moving constant
`0x80808080` in to a register. Was using two move instructions when it
could have used a single bitmask move.
This now checks to see if a constant can be 32-bit encoded in a logical
bitmask move and uses that.
This instruction doesn't match ARM semantics very well since it returns
the position of the minimum element.
But at the very least the insert in to the final instruction can be a
bit more optimal, Converts an 5 inst eor+mov+mov+mov+mov in to 2 inst
mov+mov.
This works because `VUMinV` already zero extends the vector so the
position only needs to be inserted at the end.
32-bit and 64-bit SH{L,R}D matches behaviour of EXTR. Optimize to using
this op in that case.
This converts the lsl+lsr+orr sequence in to a single extr instruction.
16-bit still goes down the old path.
Weirdly this code manages to have a bad insert for no reason? But
unrelated since this happens in the old code as well.
```
%4(GPRFixed3) i64 = LoadRegister #0x0, #0x20, GPR, GPRFixed, u8:Tmp:Size
%5(GPR0) i64 = LoadRegister #0x0, #0x8, GPR, GPRFixed, u8:Tmp:Size
%6(GPRFixed0) i64 = Extr %5(GPR0) i64, %4(GPRFixed3) i64, #0x3e
```
Not sure why the SRA fails on that second LoadRegister.
There was one holdout variable that was in a TLS object in FEXCore. Move
it to the frontend with the rest of the TLS variables.
Allows us to remove "Frontend" TLS management to be the only TLS
management.
This pass is currently doing nothing in main.
Ever since we have enforced that LoadContext/StoreContext doesn't touch
GPRs and FPRs, this has only been eliminating flags.
Remove that usage of LoadContext/StoreContext and replace with their
their replacement of LoadRegister/StoreRegister for tracking GPR and FPR
accesses.
Stripped from #2700 since this is safe to merge.
Noticed while looking at #2700.
Testing doesn't currently see this as a bug but will once #2700 starts
optimizing StoreRegister+LoadRegister pairs.
Doesn't fix the issues in that PR, but this is one.
While we're in the area implementing the Scalar + Vector variants,
we may as well cross off the Vector + Immediate variants and
complete all of the load variants for the regular LD1{*} loads
Our regex would only ever capture a single digit, so versions that had
more than one digit per section would lose additional digits.
Fixes and moves the helper to a cmake file to be shared between
GuestLibs and HostLibs.
Uses the fix in xcb because Fedora ships an older version that doesn't
have some of FEX's newer symbols.
Removes the @PREFIX_ARCH@ replacement string in the thunks path.
The library prefix paths now get generated upfront and everything gets
replaced to handle the differences between multiarch distros.
Fixes Thunks on Arch and Fedora.
This is less noisy with no loss of clarity, and follows the notation
used by both LLVM IR and NIR. (So, it should be familiar.)
Change done with:
sed -i -e 's/%ssa/%/g' $(git grep -l '%ssa')
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This does duplicate the _Constant(1) but it doesn't matter because it
gets inlined into the eor anyway. There is no functional change here.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We store garbage in the upper bits. That's ok, but it means we need to
mask on read for correct behaviour.
Closes#2767
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We can fold the Not into the And. This requires flipping the arguments
to Andn, but we do not flip the order of the assignments since that
requires an extra register in a test I'm looking at.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
WIN32 has a define already called `GetObject` and will cause our
symbol to have an A appended to it and break linking.
Just rename it to `GetTelemetryValue`
Noticed during introspection that we were generating zero constants
redundantly. Bunch of single cycle hits or zero-register renames.
Every time a `SetRFLAG` helper was called, it was /always/ doing a BFE
on everything passed in to extract the lowest bit. In nearly all cases
the data getting passed in is already only the lowest bit.
Instead, stop the helper from doing this BFE, and ensure the
OpcodeDispatcher does BFE in the couple of cases it still needs to do.
As I was skimming through all these to ensure BFE isn't necessary, I did
notice that some of the BCD instructions are wrong or questionable. So I
left a comment on those so we can come back to it.
These address calculations were failing to understand that they can be
optimized. When TSO emulation is disabled these were fine, but with TSO
we were eating one more instruction.
Before:
```
add x20, x12, #0x4 (4)
dmb ish
ldr s16, [x20]
dmb ish
```
After:
```
dmb ish
ldr s16, [x12, #4]
dmb ish
```
Also left a note that once LRCPC3 is supported in hardware that we can do a similar optimization there.
When this instruction returns the index in to the ecx register, this is
defined as a 32-bit result. This means it actually gets zero-extended to
the full 64-bit GPR size on 64-bit processes.
Previously FEX was doing a 32-bit insert which leaves garbage data in
the upper 32-bits of the RCX register.
Adds a unit test to ensure the result is zero extended.
Fixes running Java games under FEX now that SSE4.2 is exposed.
ARM64 BFI doesn't allow you to encode two source registers here to match
our SSA semantics. Also since we don't support RA constraints to ensure
that these match, just do the optimal case in the backend.
Leave a comment for future RA contraint excavators to make this more
optimal
In libstdc++ version 13, they moved the implementation of
`polymorphic_allocator` to `bits/memory_resource.h`.
In doing so they forgot to move the template's default argument to that
header. This causes the problem that `bits/memory_resource.h` is
included first without the template's default argument defined. This
breaking the automatic type deducation of `std::byte`.
Still broken in
[upstream](https://github.com/gcc-mirror/gcc/blob/be240fc6acc9714e66afbfbe6dc193844bfcba05/libstdc%2B%2B-v3/include/std/memory_resource#L79-L83)
and is unlikely to be fixed and backported. Since this is the only place
we use this type, just fix it here.
When a fork occurs FEX needs to be incredibly careful as any thread
(that isn't forking) that holds a lock will vanish when the fork occurs.
At this point if the newly forked process tries to use these mutexes
then the process hangs indefinitely.
The three major mutexes that need to be held during a fork:
- Code Invalidation mutex
- This is the highest priority and causes us to hang frequently.
- This is highly likely to occur when one thread is loading shared
libraries and another thread is forking.
- Happens frequently with Wine and steam.
- VMA tracking mutex
- This one happens when one thread is allocating memory while a fork
occurs.
- This closely relates to the code invalidation mutex, just happens at
the syscall layer instead of the FEXCore layer.
- Happens as frequently as the code invalidation mutex.
- Allocation mutex
- This mutex is used for FEX's 64-bit Allocator, this happens when FEX
is allocating memory on one thread and a fork occurs.
- Fairly infrequent because jemalloc doesn't allocate VMA regions that
often.
While this likely doesn't hit all of the FEX mutexes, this hits the ones
that are burning fires and are happening frequently.
- FEXCore: Adds forkable mutex/locks
Necessary since we have a few locations in FEX that need to be locked
before and after a fork.
When a fork occurs the locks must be locked prior to the fork. Then
afterwards they either need to unlock or be set to default
initialization state.
- Parent
- Does an unlock
- Child
- Sets the lock to default initialization state
- This is because it pthreads does TID based ownership checking on
unique locks and refcount based waiting for shared locks.
- No way to "unlock" after fork in this case other than default
initializing.
Fixes a spurious `No such file or directory` error when `ls` is trying
to query a path's xattributes that come from the emulated rootfs.
These syscalls don't support the *at variants, so it can't use the optimized `GetEmulatedFDPath` implementation.
It must also return an error on a found file path, which makes their
implementation be slightly different than the other user of of
`GetEmulatedPath`. In the case of error, it must only return an error
from the emulated path if it is /not/ ENOENT.
Before:
```
$ FEXInterpreter /usr/bin/ls -alth /usr/bin/wine-stable
/usr/bin/ls: /usr/bin/wine-stable: No such file or directory
-rwxr-xr-x 1 ryanh ryanh 1.1K Sep 24 2022 /usr/bin/wine-stable
```
After:
```
$ FEXInterpreter /usr/bin/ls -alth /usr/bin/wine-stable
-rwxr-xr-x 1 ryanh ryanh 1.1K Sep 24 2022 /usr/bin/wine-stable
```
istringstream is a very slow way to parse this, let's make it a bit
quicker.
Some implementation numbers:
1. Original implementation - 1833556 calculations per second
2. std::strtoul implementation - 4666818 calculations per second
- 2.54x the istringstream implementation
3. str::from_chars implementation - 5120718 calculations per second
- 1.09x the std::strtoul implementation
- 2.79x th istringstream implementation
This has been around since the initial commit. Bad idea that wasn't ever
thought through. Something about remapping guest virtual and host
virtual memory which will never be a thing.
This message is complaining each time VFORK was using with clone, but we
are handling VFORK here now.
This is just causing debug messages for no reason.
Remove the message and remove the flag removal option.
Currently RegisterClassType and FenceType are passed into logs, which
fmt 10.0.0 is more strict about. Adds the formatters that were missing
so that compilation can succeed without needing to change all log sites.
Fixes#2724
If catchsegv doesn't exist then just remove it from the execution
environment.
While nice to have, this shouldn't be mandatory especially with Debian
no longer shipping it.
We can handle this in the dispatcher itself, so that we don't need to pass along
the register size as a member of the opcode. This gets rid of some unnecessary duplication
of functionality in the backends and makes it so potential backends don't need to deal
with this.
Previously, the bits that we support in the MXCSR weren't being saved,
which means that some opcode patterns may fail to restore the rounding mode
properly.
e.g. FXSAVE, followed by FNINIT, followed by FXRSTOR wouldn't restore the
rounding mode properly
This fixes that.
FEX's current implementation of RIP reconstruction is limited to the
entrypoint that a single block has. This will cause the RIP to be
incorrect past the first instruction in that block.
While this is fine for a decent number of games, especially since fault
handling isn't super common. This doesn't work for all situations.
When testing Ultimate Chicken Horse, we found out that changing the
block size to 1 worked around an early crash in the game's startup.
This game is likely relying on Mono/Unity's AOT compilation step, which
does some more robust faulting that the runtime JIT. Needing the RIP to
be correct since they do some sort of checking for what the code came
from.
This fixes Ultimate Chicken Horse specifically, but will likely fix
other games that are built the same way.
When executing a 32-bit application we were failing to allocate a single
GPR pair. This meant we only have 7 pairs when we could have had 8.
This was because r30 was ending up in the middle of the allocation
arrays so we couldn't safely create a sequential pair of registers.
Organize the register allocation arrays to be unique for each bitness
being executed and then access them through spans instead.
Also works around bug where the RA validation doesn't understand when pair
indexes don't correlate directly to GPR indexes. So while the previous
PR fixed the RA pass, it didn't fix the RA validation pass.
Noticed this when pr57018 32-bit gcc test was run with the #2700 PR
which improved the RA allocation a bit.
Allows us to have a place to put helper includes and files that contain
macro utilities. This will be nice for making macro files that cut down
on verbosity across tests (e.g. Making tests for XSAVE would be way less
copy-pastey).
When FEX was updated to reclaim 64-bit registers in #2494, I had
mistakenly messed up pair register class conflicts.
The problem is that FEX has r30 stuck in the middle of the RA which
causes the paired registers to need to offset their index half way.
This meant that the conflict index being incorrect was always broken on
32-bit applications ever since that PR.
Keep the intersection indexes in their own array so to can be correctly
indexed at runtime.
Thanks to @asahilina finding out that Osmos started crashing a few
months ago and I finally just got around to bisecting what the problem
was.
This now fixes Osmos from crashing, although the motes are still
invisible on the 32-bit application. Not sure what other havok this has
been causing.
Not sure how this ever managed to work before this point actually. We
were returning a 64-bit pointer when we were supposed to be returning a
32-bit pointer.
Seemingly this was overwriting the len stack variable so then Steam's
chromehtml.so library was checking the results thoroughly and detecting
that the robust list wasn't setup before this point.
SOMEHOW this worked if FEX was built locally, but broke from the PPA
builders? Not sure how that happened, but theoretically on the next PPA
release this is now fixed and Steam can run from those builds.
Also when setting the robust list, make sure to return EINVAL if the
size doesn't match what's expected there.
So, uh, this was a little silly to track down. So, having the upper limit
as unsigned was a mistake, since this would cause negative valid lengths to
convert into an unsigned value within the first two flag comparison cases
A -1 valid length can occur if one of the strings starts with a null character
in a vector's first element. (It will be zero and we then subtract it to
make the length zero-based).
Fixes this edge-case up and expands a test to check for this in the future.
Allows us to generate a header at compile time for OS specific features.
Should fix compiling on Android since they have a different function
declaration for `malloc_usable_size` compared to Linux.
We spent a bit of effort removing 8-bits from this header to get it down
to three bytes. This ended up in PRs #2319 and #2320
There was no explicit need to go down to three bytes, the other two
arguments we were removing were just better served to be lookups instead
of adding IR overhead for each operation.
This now introduced alignment issues that was brought up in #2472.
Apparently the Android NDK's clang will pad nested structs like this,
maybe to match alignment? Regardless we should just make it be 32-bit.
This fixes Android execution of FEXCore.
This fixes#2472
Pros:
- Initialization now turns in to a single str because it's 32-bit
- We have 8-bits more space that we can abuse in the IR op now
- If we need more than 64-bit and 128-bit are easy bumps in the
future
Cons:
- Each IR operation takes at minimum 25% more space in the intrusive
allocators
- Not really that big of a deal since we are talking 3 bytes versus
4.
FEXCore has no need to understand how to load these layers. Which
requires json parsing.
Move these to the frontend which is already doing the configuration
layer setup and initialization tasks anyway.
Means FEXCore itself no longer needs to link to tiny-json which can be
left to the frontend.
Regular LoadStoreTSO operations have gained support for LRCPC and LRCPC2
which changes the semantics of the operation by letting it support
immediate offsets.
The paranoid version of these operations didn't support the immediate
offsets yet which was causing incorrect memory loadstores.
Bring over the new semantics from the regular LoadStoreTSO but without
any nop padding.
This is just confusing end users these days and no longer matters as a
debug option.
Remove from the GUI initially, maybe afterwards we will even remove
setting this at all and always auto-detect.
This basically just means that we detect ArchLinux and set a flag that
it is a rolling release, skipping doing the version check for an "exact"
match in that instance.
`eor <reg>, <reg>, #-1` can't be encoded as an instruction. Instead use
mvn which does the same thing.
Removes a single instruction from each OF calculation for ADC and ADD.
Also no reason to use a switch statement for the source size, just use
_Bfe and calculate the offset based on operation size.
SBB caught in the crossfire to ensure it also isn't using a switch
statement.
This is part of FEXCore since it pulls in InternalThreadData, but is
related to the FHU signal mutex class.
Necessary to allow deferring signals in C++ code rather than right in
the JIT.
When a signal handler is not installed and is a terminal failure, make
sure to save telemetry before faulting.
We know when an application is going down in this case so we can make
sure to have the telemetry data saved.
Adds a telemetry signal mask data point as well to know which signal
took it down.
Noticed this while debugging Proton Experimental hanging and thought
this could be related. Didn't solve that issue but this should be merged
anyway.
vfork doesn't fork the host's process space in to the child process.
Saving Copy-On-Write overhead problems. It also puts the parent process
to sleep until the fork terminates or executes.
This is a major issue under FEX where we can't emulate vfork correctly
because we need to do other work before this process terminates or
executes a new process. We have been treating `vfork` as a `fork` this
entire time.
This can likely cause problems for applications that actually use vfork
to wait for a process to complete. So let's actually emulate that
feature by using a pipe with poll to determine when that FD gets
removed.
FEX can't use waitpid to wait for this process to terminate since we
would affect the guest also wanted to use a waitpid.
These two extensions rely on AVX being supported to be used. Primarily
because they are VEX encoded.
GTA5 is using these flags to determine if it should enable its AVX
support.
Some code in FEX's Arm64 emitter was making an assumption that once
SpillStaticRegs was called that it was safe to still use the SRA
register state.
This wasn't actually true since FEX was using one SRA register to
optimize FPR stores. Assuming that the SRA registers were safe to use
since they were just saved and no longer necessary.
Correct this assumption hell by forcing users of the function to provide
the temporary register directly. In all cases the users have a temporary
available that it can use.
Probably fixes some very weird edge case bugs.
This returns the `XFEATURE_ENABLED_MASK` register which reports what
features are enabled on the CPU.
This behaves similarly to CPUID where it uses an index register in ecx.
This is a prerequisite to enabling XSAVE/XRSTOR and AVX since
applications will expect this to exist.
xsetbv is a privileged instruction and doesn't need to be implemented.
Removes some annotation warnings that have been showing up on the
actions results page.
v2 is deprecated so going to v3 is necessary. Apparently this upgrades
from Node.js 12 to 16.
I originally wrote this emulation prior to me fully understanding how
the syscall works. So there are two optimizations here.
1) No need to consume the incoming buffer at all.
- Originally I thought the incoming dirent structures were used to
calculate offset.
- This is not the case, the FD's file position is used instead.
- This means we can remove the incoming buffer consuming overhead
entirely.
2) No need to allocate a temporary buffer at all.
- With getdents and getdents64 we are guaranteed to be dealing with
structures that are the same size or smaller than the host
structure.
- This lets us encode the real host dirents in to the provided
buffer.
- After the `getdents64` host syscall, we then iterate forward
through the list, modifying as we go.
- Need to make sure to shift the elements of the structure in order.
- Need to make sure to use memmove on the `d_name` member since the
movement region can overlap.
These two optimizations significantly reduce the amount of time spent in
getdents, which has a noticeable impact on load times.
Side-tangent: I noticed a fun quirk of how NFS operates with getdents.
If the FSCache hasn't populated the metadata for that folder, then it
will early return with "some" data, not fully maxing out the buffer. The
kernel will start prefetching metadata assuming directory iterating is
happening. The next `getdents` happens and it should return a larger
number of elements.
Very neat.
I forgot that x11 was part of the custom ABI of thunks. #2672 had broken
thunks on ARM64. I thought I had tested a game with them enabled but
apparently I tested the wrong game.
Not a full revert since we can still ldr with a literal, but we also
still need to adr x11 and nop pad. At least removes the data dependency
on x11 from the ldr.
Currently WINE's longjump doesn't work, so instead set a flag that if
HLT is attempted, just exit the JIT.
This will get our unittests executing at least.
InferFromOS doesn't work under WINE.
InferFromIDRegisters doesn't work under Windows but it will under Wine.
Since we don't support Windows, just use InferFromIDRegisters.
Needs to be alligned to allocation size. Which is a page on Linux, or
64k on Windows.
In order to map at `0x1'0000` on Wine, we need to use a special case DOS
area allocation path.
New versions of CEF rely on this existing. It will get this value and
run strdup on it, even if it is nullptr.
Fixes a steamwebhelper process constantly crashing with the Steam Beta
client.
Only missing auxv values now
- AT_PAGESZ
- AT_EXECFD (for execveat?)
- AT_PHDR
- All the random cache information values.
Some of the unit tests we run will leak shm regions. Presumably this is
because they never called `shmctl(IPC_RMID)` so the ID is laked forever.
This can be seen by querying `/proc/sysvipc/shm` to see a list of old
shm regions that eventually hit the maximum capacity of 4096 shm ids.
Once CI is done running, run a utility application that all it does is
check for SHM IDs that have zero attachments (thus unused), it was
created by the UID of the runner, and it is older than ten minutes. At
which point it will erase it.
This will fix spurious failures in our CI caused by running out of SHM
IDs, previously I had a cron job setup to restart the CI runners every
hour or so which caused its own spurious failure problems.
FINALLY this bug was triaged which has been annoying us for...years?
No need to use adr for getting the PC relative literal, we can use LDR
(literal) to load the PC relative address directly.
Reduces trampline instructions from 3 to 2, also reduces trampoline size
from 24-bytes to 16-bytes.
Wine syscalls need to end the code block at the point of the syscall.
This is because syscalls may update RIP which means the JIT loop needs
to immediately restart.
Additionally since they can update CPU state, make wine syscalls not
return a result and instead refill the register state from the CPU
state. This will mean the syscall handler will need to update their
result register (RAX?) before returning.
Disabling SRA has been broken a quite a while. Disabling this was
instrumental in figuring out the VC redistributable crash.
Ensure it works by reintroducing non-SRA load/store register handlers,
and by supporting runtime selectable dispatch pointers for the JIT.
Side-bonus, moves the {LOAD,STORE}MEMTSO ops over to this dispatch as
well to make it consistent and probably slightly quicker.
From https://github.com/AsahiLinux/linux/commits/bits/220-tso
This fails gracefully in the case the upstream kernel doesn't support
this feature, so can go in early.
This feature allows FEX to use hardware's TSO emulation capability to
reduce emulation overhead from our atomic/lrcpc implementation.
In the case that the TSO emulation feature is enabled in FEX, we will
check if the hardware supports this feature and then enable it.
If the hardware feature is supported it will then use regular memory
accesses with the expectation that these are x86-TSO in strength.
The only hardware that anyone cares about that supports this is Apple's
M class SoCs. Theoretically NVIDIA Denver/Carmel supports sequentially
consistent, which isn't quite the same thing. I haven't cared to check
if multithreaded SC has as strong of guarantees. But also since
Carmel/Denver hardware is fairly rare, it's hard to care about for our
use case.
This can be done in an OS agnostic fashion. FEXCore knows the details of
its JIT and should be done in FEXCore itself.
The frontend is only necessary to inform FEXCore where the fault occured
and provide the array of GPRs for accessing and modifying the signal
state.
This is necessary for supporting both Linux and Wine signal contexts
with their unaligned access handlers.
We don't have a sane way to query cpu index under wine. We could
technically still use the syscall since we know that we are still
executing under Linux, but that seems a bit terrible.
Disable for now until something can be worked out. Not like it is used
heavily anyway.
Forgot to initialize CBDone to false before the helper thread is
started.
This fixes an issue where an XCB context is created, then stopped, then
another is created but immediately exits because CBDone was still true
from the previous run.
Also adds the handler for `xcb_connect_to_fd` so we don't miss that
usage.
This will be used with the TestHarnessRunner in the future to map
specific memory regions.
This is only used as a hint rather than exact placement with failure on
inability to map. This also hits the fun quirk of 64k allocation
granularity which developers need to be careful about.
Related to #2659 but not necessary directly.
Currently x30(LR) is unused in our RA. In all locations that call out to
code, we are already preserving LR and bringing it back after the fact.
This was just a missed opportunity since we aren't doing any call-ret
stack manipulations that would facilitate LR needing to stick around.
Since x18 is a reserved platform register on win32, we can replace its
usage with r19, and then replace r19 usage with x30 and everything just
works happily. Now x18 is the unused register instead of x30 and we can
come back in the future to gain one more register for RA on Linux
platforms.
All code paths to this are already guaranteed to own the lock.
The rest of the codepaths haven't been vetted to actually need
recursive_mutex yet, but seems likely that it will be able to get
converted to a regular mutex with some more work.
All variants of the PCMPXSTRX instructions will take their arguments in
the same manner, so we don't need to specify them for each handler.
We can also rename the function to PCMPXSTRXOpImpl, since this will
be extended to handle the masking variants of the string instructions.
In the case that an unknown drm device shows up, send it down the
default handler. This handler is just a passthrough and assumes that the
kernel doesn't have any compat handlers for that device.
This is nicer than crashing or returning EPERM, since then downstream
drm drivers like Xe, Asahi, and PVR and still try to run.
We of course still want to run their struct definitions through CI once
they go upstream.
These strings don't actually null-terminate and previous checks were
working just because it would usually be null terminated due to
initialization.
Since this isn't guaranteed, I noticed a failure to determine drm device
due to some trailing garbage in the string.
Fixes#2560
This was a forgotten member of the context that needs to be saved and
restored when jumping around the signal state.
When FEX was receiving signals back to back, there was a chance that the
signals would ride the edge of having set `InSyscallInfo` in the JIT,
which meant the SRA state would get saved once, then another signal
would occur with the previous SRA data, saving SRA again. Then when
unwinding the frames it would corrupt the SRA registers. This would
result in trying to load SRA state that was no longer valid, looking
like a crash in the JIT that was hard to see what happens.
Should also make Mono games a little less crash happy.
Cleans up CPUState memcpy as well, since it was a little weird looking.
There are a handful of siginfo layout types. To determine which layout
to use we need to check a combination of si_code and signal number
because each one in isolation doesn't explain the layout type.
Once the layout is calculated then calculate the siginfo using that
layout type. Adds a couple of different layout types to our guest
siginfo_t to handle the previously missing types.
In the case that the symlink following would immediately fail (Due to
the first query not existing or not being a symlink), then FEX would
immediately return the first query. Then the resulting syscall using
this result would try to run the syscall on that file, immediately
error, and try again on the file outside of the rootfs. This gives us
three syscalls for the price of one.
This adds an additional syscall cost to every syscall that that takes
the `FollowSymlink` code path, which is quite a few.
Instead now, check to see if the filepath exists at all, and if the
filepath doesn't even exist for the first fstatat, return a `NoEntry`
immediately. This will cause the resulting syscall waiting for the
result to skip the EmulatedFDPath check and run regular syscall.
If the filepath is symlink it will still loop to track through the
entire symlink tree to find the one that is in the rootfs.
If the filepath is just a file, then the `IsLink` step will fail,
returning the filepath to the file inside of the rootfs. (Getting this
step wrong will make wine/proton immediately break).
With this resolved, this converts a lot of three syscall operations down
to two.
This is a very OS specific operation and it living in FEXCore doesn't
make much sense. This still requires some strong collaboration between
FEXCore and the frontend but it is now split between the locations.
There's still a bit more cleanup work that can be done after this is
merged, but we need to get this burning fire out of the way.
This is necessary for llvm-mingw, this requires all previous PRs to be
merged first.
After this is merged, most of the llvm-mingw work is complete, just some
minor cleanups.
To be merged first:
- #2602
- #2604
- #2605
- #2607
- #2610
- #2615
- #2619
- #2621
- #2622
- #2624
- #2625
- #2626
- #2627
- #2628
- #2629
We can reuse the same helper we have for handling VMASKMOVPD and VMASKMOVPS,
though we need to move some handling around to account for the fact that
VPMASKMOVD and VPMASKMOVQ 'hijack' the REX.W bit to signify the element
size of the operation.
This was only used for the unit test fuzzing framework. Which has been
removed and unused for pretty much its entire lifespan.
These can now be internal only.
This is supposed to be a fuzzing based test but has been unused and not
fully supported for a long time.
Just remove it since we won't be coming back to this.
Adds in the handling of destination type size differences with AVX.
Also fixes cases where the SSE operations would load 128-bit vectors
from meory, rather than only loading 64-bit vectors with VCVTPS2PD.
In order to implement the SSE4.2 string instructions in a reasonable
manner, we can make use of a fallback implementation for the time
being.
This implementation just returns the intermediate result and leaves it
up to the function making use of it to derive the final result from said
intermediate result. This is fine, considering we have the immediate
control byte that tells us exactly what is desired as far as output
formats go.
Given that the result of this IR op will never take up more than
16-bits, we store the flags we need to set in the upper 16 bits of the
result to avoid needing to implement multiple return values in the JIT.
Also, since the IR op just returns the intermediate result, this can be
used to implement all of the explicit string instructions with a single IR op.
The implementation is pretty heavily documented to help make heads or
tails of these monster instructions.
This will use the correct `__cpuid` define, either in cpuid.h or
self-defined depending on environment.
Otherwise we would need to define our own cpuid helpers to match the
difference between mingw and linux.
These use std::filesystem and should be moved over to WIN32 specific
code.
Added comments to explain that these are currently placeholders and
since we don't do mingw+glibc fault testing these won't get picked up
anyway.
This lets all the path generation for the config to be in the frontend.
This then informs FEXCore where things should live.
This is for llvm-mingw. While paths aren't quite generated correctly,
this gets the code closer to compiling.
This is not an attempt to clean up the various issues with the pthread
logic, instead just moving the pthread specific logic out of FEXCore in
to FEXLoader.
FEXCore needs to know how to create threads in an agnostic way. Which is
why we obfuscate the details with this inteface.
Initially this was implemented with the pthread handlers in FEXCore and
expected eventually for those to get moved to the frontend. This is the
time when it has been moved.
This is the first step towards compiling with llvm-mingw.
Still a long way to go.
Somewhere between kernel 5.15 and 6.0 this syscall's behaviour has
changed.
The unit test expects -ESPIPE result but new kernels will return 0
(noop_llseek in the source).
The new solidrun board is our first CI machine to be running kernel 6.1
so this needs to be merged before that CI machine can be enabled.
This is based on our regular CI runner file with a couple of things
stripped out.
- gvisor tests removed because they cause our CI machines pain with
tmp/shm mounts
- Thunks disabled since glibc fault testing is incompatible with it
- ARMEmitter tests removed since we don't want to test vixl.
All glibc conversion PRs will need to be merged before this and then
this needs to be rebased.
We still need to hook glibc for thunks to work with
`IsHostHeapAllocation`.
So now we link in two jemalloc allocators in different namespaces.
As usual we have multiple heap allocators that we need to be careful about.
1. jemalloc with `je_` namespace.
- This is FEX's regular heap allocator and what gets used for all the
fextl objects.
- This allocator is the one that the FEX mmap/munmap hooks hook in to
- This mmap hooking gives this allocator the full 48-bit VA even in
32-bit space.
2. jemalloc with `glibc_je_` namespace.
- This is the allocator that overrides the glibc allocator routines
- This is the allocator that thunks will use.
- This is what `IsHostHeapAllocation` will check for.
3. Guest glibc allocator
- We don't touch this one. But it is distinct from the host side
allocators.
- The guest side of thunks will use this heap allocator.
4. Host glibc allocator
- #2 replaces this one unless explicitly disabled.
- Always expected to override the allocator, so this configuration
isn't expected.
Already tested this with Dota Underlords to ensure this works with
thunks.
FEX might not ever make it to its cleanup routines when exit is used. So
make sure to disable the fault checking if exit is used.
Opened #2594 to track this investigation.
The dispatcher was saving AVX state even though FEX doesn't support it
currently. This is due to it checking for the config option rather than
the HostFeatures option.
The `EnableAVX` config option is supposed to be used to inform FEXCore
if we want AVX disabled or not when the host supports the feature. In
this case it is universally enabled because we haven't encountered any
games that have issues with AVX state being saved with signals. (We know
they exist, we just don't have configurations for them).
The HostFeatures option `SupportsAVX` is the option that is supposed to
be getting used for determining if the runtime AVX feature is enabled.
This also had an issue though that this was **also** always enabled if
running on an x86 host with AVX, or an ARM host with SVE2-256bit.
It was then disabled if the config option was disabled; But, since
FEX-Emu doesn't support AVX fully yet, we need to ensure this isn't yet
enabled.
But this only solves half the problem. In order for our CI to test AVX
features before fully supporting AVX, it needs to be able to enable AVX
so that the CPU state is correctly saved.
So we need to change the default configuration option to be false, and
have CI enable it for the tests that matter before AVX is fully
implemented.
Every time we are calling a function in `FEXCore::Allocator::` this is a
pointer indirection. Which means on x86 it is always a `call [rdi]` and
on AArch64 it is a `ldr x17, [x0]; blr x17;`.
Instead of doing this, use inline functions in the header that call the
correct allocation function directly. This function gets inlined and is
no longer an indirect call.
When compiling with jemalloc, we forward declare the jemalloc function
definitions so we don't have to pull in the entire jemalloc interface in
to the public header definitions.
`std::stoul` and `std::stroull` take a std::string which was converting
the string_view to a std::string first. Causing glibc fault testing to
catch this since not much uses this.
These will be added to the documentation.
fwrite allocates some backing memory for buffering outputs which FEX
can't track.
Switch to using `fileno` to get the fd from the FILE and write directly.
This will need to be changed for llvm-mingw support but that will come after
this.
This will be added to the documentation that we can't use fwrite.
This unique_ptr would end up getting allocated through jemalloc then
fread through glibc.
Convert over to fextl::unique_ptr to ensure it gets allocated and
deallocated through jemalloc correctly.
This is done by consuming a single page at the end of the current sbrk
memory region. Then consuming any remaining bytes that could have
potentially ended up in it.
This ensures that glibc won't be able to return 64-bit pointers to
32-bit thunks once the remaining work is in place.
Fixes a crash in xcb thunks since more stuff on FEX side has moved over
to jemalloc.
Destructors don't actually get called when a shared library is removed.
It's some weirdo quirk that we can't work around. Instead refcount
Display connections being created and disconnected. Creating the thread
on the first display creation, and tearing down on final display
teardown.
Must be merged before #2564 otherwise that PR will break thunks.
The LOAD_LIB and EXPORTS macros behave slightly differently in this regard:
* Use LOAD_LIB(libwayland-client) in Guest.cpp (library name with dash)
* Use EXPORTS(libwayland_client) in Host.cpp (library name with underscore)
The previous log in the frontend is super useful when an instruction
decoding wasn't supported.
Now that most of AVX is covered, a game will crash on SIGILL (and
usually catch it) and close without any indication.
Now if the instruction is decoded but it is invalid for the
configuration, still output a message as a good indicator that the game
is using instructions that the host doesn't support.
Will let us still pick up on games crashing due to lack of SVE very
easily.
This is an undocumented but supported instruction. It behaves just like
an `sbb al, al` but doesn't set flags and is one byte shorter.
The end result is that al is set to 0xFF or 0 depending on if CF is set
or not.
And with that, we support all of the AVX1-only instructions.
The remaining instructions for full AVX1 support is now just the SSE4.2
string instructions.
Will be used to implement the load variants of VMASKMOVP{D, S} and
VPMASKMOV{D, Q}
Particularly useful, since with SVE this behavior can be collapsed into
two instructions (CMPGT followed by the relevant LD1 load instruction)
These conditionals were accidentally inverted and were treating 32-bit
elements as 64-bit ones, when this is unintended.
Also add missing tests to ensure this doesn't slip through in the
future.
When the textual field is changed, that order changes and there would be
a partial resort where the labels would misalign compared to the text
entries.
Fixes that.
We can slightly modify the table flags for VPSRAVD in order to
consolidate the variable shift code into one place.
Useful, since we still need to implement VPSLLVD and VPSLLVQ.
Also in the event we start supporting AVX-512, then the code can just be
extended from a single point.
We can just use the movprfx to move the data to shift into the
destination and then perform the shift directly with Dst as an operand.
This is safe, since no original vectors are used in the ASR operation,
only temporaries are used.
Lets us eliminate a temporary and keep things simpler.
When building with the simulator enabled, these aren't used and cause
some unused funtion/variable warnings.
We can silence these with [[maybe_unused]], since these are used in
non-simulator builds.
This dependency was on FEXCore only. Add it to the same dependency
definitions that CONFIG_INC is in so that FEXCore_Base picks it up.
Might solve a compile error from FEX Common missing the dependency being
generated.
Things like VPINSRB and co. use this pattern, as well as VCVTSI2S{D,S}.
No other VEX-encoded instructions expect the first source to be a GPR
aside from some of the BMI1/BMI2 instructions, so we'll need to check
for this case.
Causes Portal to go from 85FPS to 120FPS on Lenovo X13s.
When running in 32-bit mode we were wasting 8 GPRs and 8 FPRs by still
allocating the top 8 registers of each even though 32-bit can't use
them.
Reallocate them to be register allocated registers when running 32-bit
applications which reduces spills and lowers the cost of
spilling/filling SRA registers.
Quite a significant speed boost for a little bit of work.
Completely unused right now, this will very quickly spread throughout
the entire codebase.
Currently only containers with FEX's global allocator usage.
Any STL class that is allocation free and has been vetted to fit within
the guidelines of not allocating memory will get an alias inside of
`fextl`.
This means that any std:: namespace usage going forward will need to be
scrutinized heavily. As aliasing it will mean it is vetted, otherwise it
needs to be checked with FEX CI (once in place) to ensure it doesn't
allocate memory behind our back.
Aliases everything we allow from std:: reduces mental burden from
needing to check if it is okay to use or not. Additionally it will allow
us to quickly pivot to the EASTL if we find as some point during this
transition that this will in-fact not work.
Once this is committed, we are going to be moving very quickly to
piecemeal convert /everything/ in the codebase over to fextl:: namespace
from std::.
I am sorry for the hellscape that will follow.
Right now cmake doesn't even get past the configuration stage when mingw
is used.
If mingw is detected, setup cmake so it can at least configure itself.
Will be necessary for cleaning up the rest of the codebase.
Even without this being an optimal implementation, enabling this option
reduces the amount of time spent in memmove in Hollow Knight.
Before:
```
10.07% [JIT] tid 284422 [.] /usr/lib/x86_64-linux-gnu/libc.so.6+0x17d780 (0x7fbcef8d6dfd)
```
After:
```
0.53% [JIT] tid 288916 [.] /usr/lib/x86_64-linux-gnu/libc.so.6+0x17d780 (0x7f0cbe0dc608)
```
Alwa's Awakening had an issue where it was spamming clock_nanosleep and
getting EINVAL, consuming a CPU thread to 100%.
This is due to two things in the previous implementation
1) glibc helper does some data munging and validation of inputs
2) We were updating remain even if TIMER_ABSTIME was used
This only cropped up because this error would only occur when EINTR
occured. Since the game is a mono game, the thread that would be
sleeping would get interrupted and then spin forever with corrupted
data.
Just like the `REP STOS` instruction, this removes a fairly nasty bit of
code generation from our dispatcher. This instruction behaves like a
memcpy/memmove when not dealing with the faulting behaviour around the
instruction.
With a microbench things improves this instruction's throughput by 3.5%
to 25.5% depending on operating size.
There is a TODO for implementing this instruction with ARM's future MOPS
instructions which will accelerate this implementation further.
Now that our register types are strongly typed and don't implicitly convert
across one another, we can get rid of the templating on some member
functions, turning them into normal member functions.
We can collapse the array into a shift, and the others can just use 8 as
the starting value to be shifted instead of multiplying by 8.
Aside from the array change, everything else is likely already constant
folded, but it does make things a little more straightforward
Switches to the new CDN which is significantly faster and has other
benefits.
In order to make sure we don't break old clients, switch to the new link
for a few months while leaving the old one operational.
The links file in the old CDN still points to the new rootfs links so
they get the performance improvement on old clients still.
Need to ensure that dirfd is AT_FDCWD and also need to check flags
correctly.
Flags were incorrectly checking mode for O_WRONLY and also we should
check for O_APPEND. Split it out to a helper function just so it is
easier to see what is going on.
Fixes the issue of proton not finding `/lib64/ld-linux-x86-64.so.2`
x86's REP STOS instruction is a memset (with element size!) with the
ability to choose a direction of execution.
Additionally it has a feature where if it faults part-way through the
copy, an application can catch the fault and continue afterwards to know
how many bytes got copied.
RCX is the counter which decrements for each element, and RDI is the
memory pointer. On fault these will reflect the last location that was
attempted to be written. FEX doesn't support this behaviour which makes
our lives easier.
Without supporting that feature, this turns in to a directional memset
by element size. Let's remove all the multiple blocks and just emit a
single IR operation to improve performance of the JIT.
Our generated code here was terrible, the IR was terrible, multiblock is
always slow with RA. Just a general overall improvement.
With profiling pressure-vessel this change deletes the hottest block that
appeared in the trace. This instruction is very commonly used for
memsetting a region to zero so it should be quite fast.
We can also optimize REP MOVS in the future with a Memcpy IR operation
in a similar fashion.
Additionally in the future these can be optimized to use ARM's new MOPS
instructions since the most common case is memset by byte. Which is when
we should expose the "Fast REP STOS" CPUID bit. Both setp/setm/sete and
cpyfp/cpyfm/cpyfe match `REP STOS` and `REP MOVS` respectively.
This operation directly matches what the x86 STOS instruction does
without supporting its faulting behaviour.
STOS faulting behaviour is that RCX and RDI get updated to the last word
written. Which is something that FEX hasn't ever supported.
In the case that a file is getting opened to be created or writable then
skip EmuFD and rootfs searching for this file.
This fixes an edge case where if FEX was run with an unpacked rootfs
that was writable then pressure-vessel would break.
Fixes pressure-vessel with unpacked rootfs.
Like with the vector registers, we can remove all implicit conversion
operators except the ones that convert down to the base PRegister class.
With this, all of the registers are now adequately constrained, so we
shouldn't have any wonky implicit conversions happening anymore.
Ensures that we're always explicit about the size of a register when
using APIs that enforce it.
The only implicit conversions we keep are conversions that convert down
to Register, but not anything that converts up the hierarchy or across
it.
A few vector instructions that take register lists often require
vector registers within the list to be sequential in the form of an
increasing list modulo the register file size.
For example:
v1, v2, v3, v4
v31, v0, v1, v2
both fit these requirements.
This will be used to enforce this restriction within the asserts from a
single place.
A large amount of size handling in this category is just decrementing
the size by 1, so we can tidy up a bunch of conditionals by just doing
that instead.
When FEXServer is daemonizing through an instance of FEXLoader or
FEXInterpreter, it would leave a zombie process which was waiting for us
to read the process status.
Since we don't care about the child status and don't want to get blocked
by waitpid, just ignore the signal.
This tells the kernel that we don't care about the signal and will kill
the zombie process immediately.
Didn't notice this before since FEXServer started failing to daemonize.
It looks like `SYSTEMD_EXEC_PID` can leak through to the executable
environment in regular situations. Instead let's key off of
`INVOCATATION_ID` which doesn't leak through.
Fixes an edge case behaviour where FEXServer wouldn't daemonize in some
systemd environments.
Fixes#2443
I found out with some profiling that this we were spending a decent
amount of time with the `openat` syscall in heavily utilized situations.
While not super common in active gameplay situations, it matters
significantly in loading screens that this is fairly optimal.
The bulk of the time is spent in the emulated files handler to ensure
that whatever path we are given, we can capture file paths that we need
to emulate. The largest contributor being the std::filesystem::canonical
function call.
A couple of optimizations in place here.
1) Do a quick hashmap check right at the start to see if we exactly fit
2) Change from `std::fs::canonical` to `realpath`
3) Switch `GetEmulatedFDPath` to not use optional so it stops building
on the stack
I'm still not super happy with the performance of `realpath` and also
not happy that we still need to use `lexically_normal` in one code path.
But short of writing a super hand-optimized `realpath` that fits our
constraints, I don't think we can do better.
Micro benchmark needs to test four different situations due to this
optimization.
1) Non-EmuFD path
2) Non-EmuFD path with dirfs
3) EmuFD path
4) EmuFD path with dirfs
And the performance improvement for each situation respectively
1) 12% performance improvement
- 213413 openat syscalls/s -> 238999 syscalls/s
2) 17% performance improvement
- 202085 openat syscalls/s -> 237309 syscalls/s
3) 17% performance improvement (/proc/cpuinfo)
- 56616 openat syscalls/s -> 66231 syscalls/s
- Includes overhead of generating temp FD and close syscall
4) 5% performance improvement (/proc/cpuinfo)
- 51080 openat syscalls/s -> 53956 syscalls/s
- Includes overhead of generating temp FD and close syscall
And for sake of comparison to the non-emulated system; My test system
can hit around 1-1.1 million openat syscalls per second in the same
microbench.
Nice little performance uplift.
This is known to cause issues in some cases. We hit the first game that
checks for this bit and early exits if it is found.
Lets the MMORPG Tibia run in non-VM situations.
Looks like they have more checks for VMs other than hypervisor bit, so
running under Parallels still won't work. Running on bare Linux is fine.
Centralizes all the shifting behavior and whatnot into a single
function, making everything much more localized.
Also gets rid of a lot of magic constants related to the encoding limits
of immediates.
When we were taking a signal that had a misaligned stack, we would store
the host stack at a weird offset.
After that point when we were trying to sigreturn we wouldn't know the
alignment of the stack coming back and we would try loading the host
stack from the wrong offset. Easy fix is to just align the host stack
location.
Fixes Ender Lilies, which was consistently crashing from a SIGCHLD due
to having a misaligned stack.
Side-change: Move the cookie check to the start of the restore. Doesn't
make sense to check the cookie after restoring state since it could be
quite wrong.
Now that we've moved MOVHPS over to sharing the implementation of
MOVHPD, the FLAGS_SF_HIGH_XMM_REG is now unused.
Since we're supporting AVX, this flag is kind of weird in terms of
behavior, since what determines the high part of a register is now
situationally different.
Also it's much more explicit to perform the insert directly in the
implementation of instructions, than relying on a flag to do it for us.
So, instead of keeping it around, we can reclaim it as unused for use
with any necessary behavior that we would require in the future.
These instructions essentially have the same behavior. This also allows
us to remove the only used instance of FLAGS_SF_HIGH_XMM_REG, which,
given that we now support AVX, has ambiguous use.
While we're at it, we can expand the tests to make use of the store to
memory variant.
Also removes an erroneous copy-pasted comment about ZEXTing. This is
from the MOVQ implementation function. MOVHPS/MOVHPD don't do any
ZEXTing, they either store to memory or insert into a register.
Will catch cases where registers aren't consecutive in the constructive
variant. While we're at it, we can also amend EXT's similar but slightly wrong
consecutive check.
Also adds tests to ensure these corner-cases hold.
Performance stats up front:
This improves pressure-vessel startup time on my test device by 10.1%
Improving the startup time from 9.71425 seconds to 8.7421 seconds.
Most filesystem based syscalls support a file descriptor version with an
*at suffix. This allows us to do these syscalls with pathnames that are
relative to the directory FD that is passed to the syscall.
This is pretty much exactly what we want when we are searching for files
inside of our rootfs. The only quirk ends up being that we are getting
passed absolute paths. This ends up being very simple to workaround by
stripping off the front '/' character. Doing this is just offsetting the
pointer passed to the syscall by one byte.
This does require having two temporary buffers of size PATH_MAX passed
to the handler since just like in the other implementation, we need to
keep the previous result around. The difference being now that we aren't
doing a bunch of std::string temporary manipulation and now we are
returning one of the passed in buffers back depending on the result.
Moves some instruction helpers that existed outside of the private
section of the class back into them, so that we're not exposing
unnecessary things in the interface.
When executing a non-inlined syscall, we spill all static registers.
We weren't storing in to the thread context that we have done this.
If a signal occured between FEX returning from the syscall (after the
blr) and before the `FillStaticRegs` then the signal handler would get
the incorrect register state.
This typically manifested as Steam getting a SIGCHLD, trying to recover
the guest stack pointer, and it that pointer would be zero or some other
corrupt value. Thus crashing inside of the signal handler.
Surprising that we hadn't hit this way more before this point, must have
needed hardware that tickled the race condition *just* right.
This has been a long time coming. The C interface has been a thorn in
our side for no reason for a long time.
The purpose of this step is to remove the C interface without changing
behaviour as much as possible. This means that with this commit there
are still some bad practices but the remaining issues will be solved
with followup PRs.
Primarily, we still have a `DestroyContext(CTX)` static function which calls
the Context implementation's `DestroyContext` and does a raw C++ delete.
Follow up PR will remove that, but I didn't want to touch it yet since
it'll require checking to ensure the unique_ptr changes play nice with
our allocator hooking. Which this is already a huge PR without trying to
change behaviour.
Cortex-X1C and A78C are relatively minor changes to their non-C
counterparts. Support classifying them in case clang understands them.
Fixes a minor perf regression noticed on the Lenovo X13s while testing.
Take a leaf from the Vector ops and have the jump entry choose the IR
op.
Also generate one atomic op and modify the IR type in the locked memory
type just like the non locked memory path.
This class of instructions in the number one instruction type percentage
wise, so making this more optimal will be a win.
It's a fairly minor optimization so it should be a small impact.
This allows us to not update RIP on block entry, but still allow
reconstructing the RIP up until that point.
While still not full RIP reconstruction, this lets us update the signal
context's RIP just like the `x86dec_SynchronizeRIPOnAllBlocks` without
eating the cost of writing to RIP on block entry.
We can move the initial move down by SPLICE, which not only lets us turn
it into a MOVPRFX, but also we can safely move into the final
destination register directly, since we can be sure there's no
potential dependencies at this point
Fixes#2410
We were pulling the guest RSP before spilling static registers back to
the state.
Move this to after we spill SRA state to fix this bug.
Thanks to @ifquant for diving in, identifying, and finding the exact bug.
Because the binaries have no metadata, we allow a .json file to be
placed alongside a test indicating required features in a requirements
directory
We also check if the system itself supports those features and run tests
based off of that.
This will allow us to support operating on 256-bit vectors.
Currently only sets up the bits and pieces on the x86-64 side, since
facilities for testing the 256-bit operations on ARM isn't set up yet.
With this in place, a _VMov isn't necessary for variable blends anymore,
since the vector upper lanes are guaranteed to be zeroed out in the 128-bit case.
Currently unused (The full data gets thrown away after CompileCode is
called), but allows us to separate code and data in what `CompileCode`
returns.
This will allow us put a header on JIT blocks which will fix a long
outstanding bug where RIP isn't always synchronized on block entry, but
since it only needs to synchronize on signal we can rebuild in the
handler. This future task will remove the `86dec_SynchronizeRIPOnAllBlocks`
config option, but the data will also end up being used for more things
in the future.
SelectCC shift wasn't necessary since we just need to ensure the final
result is zero when or'd together.
Also operations calculating SF can just use a BFE instead of a shifts
with a constant. BFE by immediate is more efficiently encoded in our IR.
When used in conjuction with #2345 this is a useful way to enable
libSegFault in applications using application profiles.
Very useful for applications and games that use launcher scripts that
set LD_PRELOAD to nothing prior to launch.
A user was wanting this.
For the case that the 32-bit VDSO thunk library isn't available, have a
fallback that can work as well.
Otherwise 32-bit applications will just straight up crash on signal
return.
VDSO behaves like a raw syscall which doesn't set errno.
posix tests are testing that errno is set correctly.
Our VDSO handlers weren't wired up to return errors from VDSO correctly.
To handle this we need to have different handlers depending on if the
syscall being used comes from glibc or true VDSO.
This wasn't being uncovered previously since CI wasn't running with VDSO
thunks enabled, but now that it is this needs to be handled or CI will
fail.
This is required for backtrace to work correctly.
If we are using our custom instruction for returning from a signal, then
backtrace tries to read PC for the sigreturn code and finds our code,
breaking it.
Instead we now /correctly/ support using rt_sigreturn/sigreturn and the
restorer provided from the user.
To facilitate this, we now store a single 64-bit value on the stack to
return our host stack pointer to the correct location from before the
signal.
With cookie checking in place, we can know if an application betrays our
expectations and tries to pass its own signal frames.
If an application in the future /does/ try to pass its own signal
frames, that's unsafe and we cna deal with it then.
This fixes a bug where we are falling back down the default signal
delegator after a fatal error.
We need to reraise the event in the case that it didn't come from the
kernel.
Fixes backtrace crashing with incorrect signal when it tries to reraise
the signal that it handled using tgkill.
These need to be bit-exact following exactly what is shown in the
assembly.
libunwind parses where EIP is to see if it is in a stack frame.
Also needsto live in VDSO otherwise backtrace doesn't work.
The instructions allow specifying 64-bit element sizes.
With this, we can also completely remove the size checking from the
functions, since the general SVE integer reduction operation already
checks for invalid sizes for us.
LogMan::Throw::AFmt(Thread->ThreadManager.GetTID()==FHU::Syscalls::gettid(),"Must be called from owning thread {}, not {}",Thread->ThreadManager.GetTID(),FHU::Syscalls::gettid());
Copyright (C) 2018 Ryan Houdek <Sonicadvance1@gmail.com>
Everyone is permitted to copy and distribute verbatim or modified
copies of this license document, and changing it is allowed as long
as the name is changed.
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. You just DO WHAT THE FUCK YOU WANT TO.
Loaded 100 of 1101 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Ryan Houdek