Now that all the threading behaviour has been correctly separated/moved
to the frontend, these functions serve no purpose.
- Instead of using RunUntilExit, all threads can use `ExecuteThread`
directly, since there's nothing special about the primary thread now.
- This also removes the public function definition of `ExecutionThread` since that was only used for threading logic.
- Instead of using an exit handler, just do the same cleanup after
`ExecuteThread` has returned.
- Just make gdbserver is cleaned up early if it exists since it may
want to send some things to the connected gdb instance before
threads are exited.
These are all frontend constructs with mostly deprecated constraints.
WaitingToStart isn't used anymore, Running is effectively always true
(and behaviour has changed that if a thread is alive, it's running).
The only one that remains is `ThreadSleeping` which is only handled in
the frontend, and there was some conflation between ThreadSleeping and
Running which was hard to gauge. So delete `Running` and
`WaitingToStart`, but move `ThreadSleeping` to the frontend.
Now that most of the thread tracking is in the frontend, change this
over to building the thread execution handler on the parent thread.
Removes a memory allocation/free pair, and removes the copy of each
variable in the child thread.
We were using this variable for two things, letting the frontend signal
to the backend that it wants to start executing once the thread is
created, and also for handling thread pausing. These two features are
conflated with one another and actually makes things more confusing.
- Move StartRunning/StartPaused to the frontend, because its a construct
that only needs to exist in the frontend
- Adds a FEX::HLE::ThreadStateObject CV for handling pausing, which only
needs to exist for gdbserver
FEXCore hasn't been returning anything other than EXIT_SHUTDOWN for a
long time, so this ended up just moving data around for no reason.
This isn't going to be used for further GdbServer work anyway, so just
completely remove it.
This is a Linux construct, move it to the frontend.
This is going to need some changes in the future since exit_group and
exit syscalls are supposed to behave differently than how FEX implements
it. For now just move it to the frontend.
GDB has two ways to read the registers. One way is reading the full
GDBContextDefinition, which matches the layout in `BuildTargetXML`.
The other way is to read the individual elements out of
GDBContextDefinition.
These two code paths were independently implemented. Instead generate in
one location and use in either location.
NFC
This currently happens in two locations, so split it out.
There's some behaviour here where if the TID isn't found, then it
returns the ParentThread of the process. This is working around a bug in
either FEX's gdbserver or binaryninja. Leave it currently before we
figure out what's wrong.
NFC
Previous `S AA` logic is legacy for non-multithreaded applications. This
newer command gives more information about what occured and in what
thread id.
Alloc::OSAllocator uses a TLS variable of the thread object so it can
use a forkable mutex plus a deferring signal section. This was setup
when the FEXCore "ExecutionThread" function is called, which is a bit
awkward and is an artifact from when the thread creation was mixed
between the frontend and the backend.
Instead let the frontend inform the backend when to install the TLS
variable.
This is one step required to make GdbServer work correctly again since
the thread initialization and pausing is awkward today.
This was working around an edge case in the GdbServer where a thread was
getting created while the process was shutting down. This edge case is
getting removed so get rid of it.
Previously, running ctest with BUILD_TESTS=OFF would discover and execute
leftover tests from a previous build. This change ensures CTestTestfile.cmake
gets regenerated so that ctest will see an empty test list in that case.
ltrim had the issue that it would always consume the left-most character
even if it wasn't whitespace. So `FEXLoader` would turn in to
`EXLoader`, even without any whitespace in the string.
Adds a test to ensure this doesn't occur again.
this is a pain to track and, it turns out, buys us virtually nothing on flagm
systems. rip it out.
this fixes a bug with failing to set in all the right places.
on non-flagm systems there's a slight instcountci impact, but that is mostly
mitigated by the earlier patches in the series. so overall a wash there but
worth it for making the codebase easier
to reason about.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Ensures reads don't go past the end of the page boundary.
SVE masked loads can make this more effective but `VLoadVectorMasked`
isn't setup to be efficient for this case yet.
Abstract sockets have one limitation: they are bound to a network
namespace. Chromium/CEF sandboxes using a new netns, which breaks
connecting to the FEXServer.
To work around this, use and try *both* abstract and named sockets. As
long as either the filesystem or the network is unsandboxed, things will
work. If both are sandboxed, there isn't much we can do... but at that
point we shouldn't be reinitializing the FEXServer connection anyway
since the FS should be available on FEXInterpreter startup.
Needed by Discord, part of the Chromium sandbox code. The warning still
triggers because Chromium asks for CLONE_VM on x86_64, but that can be
safely ignored (CLONE_FS is the one that matters).
Apparently Chromium/CEF can chroot or otherwise sandbox the filesystem
away before forking and checking for directory FDs, making /proc
inaccessible, which means we can't stat it for our inode check, breaking
the hiding.
So, double down on things and do what Chromium does: open an fd to /proc
ahead of time, so that continues to work. Then we use it to update the
inode of our RootFS fd instead, and finally, also do the /proc fd itself
to hide that one too.
We also don't need to check the st_dev of /proc more than once, since
that's not expected to change anyway.
Fixes cefsimple.
NFC
Finally converts the IR operations themselves to store the OpSize for
the IR operation size and element sizes.
This also finally, FINALLY, converts that remaining `_Constant` helper
to stop using a size field that is specified in bits rather than bytes
like all the other IR op handlers. That thing was so confusing and now
it's gone.
Chromium/CEF has code that iterates through all open FDs and bails if
any are directories (apparently a sandboxing sanity check). To avoid
this check, we need to hide the RootFS FD. This requires hooking all the
getdents variants to skip that entry.
To keep the runtime cost low, we keep track of the inode of
/proc/self/fd/<rootfs fd> (note: not the RootFS inode, the inode of the
magic symlink in /proc), and first do a quick check on that. If it
matches, then we stat the dirfd we are reading and check against the
procfs device, to complete the inode equality check.
As an extra benefit, this also fixes code that tries to iterate and
close all/extra FDs and ends up closing the RootFS fd.
The push operation already truncates the result, there's no need to bfe
it. Noticed this while cleaning up in #4134. Removes one instruction for
16-bit and 32-bit pushf instructions.
these are used in places:
* bsf/bsr
* pcmpblabla
* x87 fild
In all cases we explicitly check for zero and change the behaviour accordingly.
So weaken the IR op to let us optimize. No sense checking twice.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Allows the AVX128 implementation to use the same implementation as the
128-bit SSE implementation, because it works per 128-bit lane just like
SSE. Is a minor optimization.
vblendvps, vblendvpd, vpblendvb all broke because of failing to zext the
128-bit register correctly.
vinsertps broke because it was accidentally using the wrong
implementation.
Extends each of their unittests to handle these cases.
Fixes fxtract by returning the correct values for 0.0 and -0.0. We moved the split of fxtract into _sig and _exp, to the opcode dispatcher, to ease some comparisons.
Also removed the IR node F80XTRACTStack which is not needed anymore.
This was brought up by #3831 but I finally got the courage to look at
the hard problem.
Although I'm only tackling half of the problem with this PR, which is
that FEXLoader needs to strip the rootfs path from the executed path if
it begins with the rootfs, plus some changes to the surrounding code.
The primary concern here is that when an application has been executed
under FEX, specifically through binfmt_misc, then FEX needs to prepend
the full rootfs path otherwise Linux can't find the program.
Additionally execveat with an FD will resolve a full path to the rootfs.
So past FEX's initial setup, we need to strip off the rootfs path to
provide an "absolute" path that is visible to the guest application
later. Which is kind of funny since we have a `RootFSRedirect` function
which did the exact opposite. This was due to legacy problems in the
original ELFLoader that couldn't handle symlinks correctly, which has
since been resolved, so that no longer needs to exist.
There was also some weirdness in `GetApplicationNames` where the passed
in argument list was modifying Args[0] and then saving the Program as
well. Which I just got rid of. Also stopped passing in the arguments by
value because....why did I write it like that?
In InterpreterHandler we now need to check if we can open the path
inside the rootfs or fallback without it. Plus I had to change the
shebang handling so it stopped prefixing the rootfs AGAIN. Took the time
to change the shebang handling there so it stops creating string copies
and instead just generates views.
Overall this fixes a fairly major flaw with how we were representing
`/proc/self` to the application, which was breaking wine since it would
prefix the rootfs multiple times, which was weird.
It doesn't address the remaining problem in #3831, which is that
applications can still see some of the leaky abstractions with symlinks
through the rootfs, but I want to get at least this step in.
FEX's ASM unitests had the problem that they were copy and pasted
templates and MemoryRegion was copied in to almost all tests.
Very few tests actually use the MemoryRegion they were asking for and
instead used none, or the hardcoded memory regions that the
TestHarnessRunner provides.
This is entirely a sed replacement and minor fixups plus reverts for the
few tests that actually use the region asked for.
When writing `FEX_bugs/tls_vector_element.asm` I had to switch to using
GS segment instead of FS segment because the unittests didn't correctly
restore FS after running. This is because GS is unused on Linux
applications, but FS would become broken and break glibc cleanup on
shutdown.
Now that xbyak has been updated to v7.09, it now supports
{rd,wr}{fs,gs}base which allows us to save and restore the segments
correctly. This lets us drop in TLS tests in to unittests more easily
now.
Ensured this works by modifying the test temporarily to use fs instead
of gs again, seeing it crash before HostRunner changes, and work after
HostRunner changes.
Fixes#4104
The instruction definition only allows these instructions to load/store
94 or 108 bytes, not affecting any bytes afterwards. This is a bit
awkward because 80-bit x87 registers are stored at the end.
FEX has an optimization today where it uses overlapping loads and stores
for the first seven x87 registers, and a split loadstore for the final
register. This ensures that we get the correct data while reducing the
number of loadstores.
We didn't have a unittest in place to ensure we only ever write the
correct amount of data, so changes like in #4107 which look correct from
an initial glance, would have resulted in broken behaviour.
This unittest ensures both that the instructions don't try to access
beyond the end of the page, and also ensures that they don't overwrite
subsequent data. Making sure that potentially broken behaviour doesn't
make its way in.
The size of the input_event struct differs between 32 bits applications
and 64 bits applications. To deal with this, the kernel implements a
compat variant for the input syscalls, but it's only enabled for 32 bit
processes.
In libkrunfw we're introducing a prctl that enables a 64 bit process to
request the kernel to enable the compat variant for the input syscalls.
This commit makes use of that interface for enabling/disabling the
compat input variant as required.
The visible effect is that input devices such as gamepads work properly
on emulated 32 bit applications.
Signed-off-by: Sergio Lopez <slp@redhat.com>
Just requires a thread pointer check to be fixed in FEXCore.
This doesn't need unique pages to exist for the file mapping and can be
shared since it's readonly mapped.
These had completely forgotten to load segment prefixes on the element
loadstore operations. This had caused a crash on the Linux version of
"Halls of Tormet" (steamid 2218750).
This game has TLS vector data which it was loading with a movhps and hit
this path.
fixes incorrect signs on FPREM. in turn should fix end-to-end failures logging
into Steam.
Thank you to Sergio Lopez for tracking down the JavaScript fail, and Ryan for
finding the bug.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
every pattern costs us JIT time, but not every pattern is doing anything.
especially with the flag rework of 2023-2024, a lot of patterns just don't make
sense anymore. constant folding in particular isn't too useful now.
only instcountci change is AAD which i'm contractually forbidden from caring
about.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
hash tables have high overhead! but individual blocks are small, so the
quadratic search is _much_ faster in practice. knocks 8% off node.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is an interesting vdso implementation because it only exists in
x86-64 with kernel v6.11. For Arm64 the implementation is likely to land
in a couple kernel versions from now.
Some differences with vdso_getrandom versus the regular getrandom
syscall
- Has two additional arguments, opaque_state and opaque_len
- Expects the userspace to mmap/munmap this opaque state structure
- Lets userspace query information about this structure upfront
- If the opaque data structure isn't provided then it falls back to
regular SYS_getrandom
With the "glibc" implementation, we can tell the interface to allocate a
single page that gets unused (otherwise glibc ends up not using the
interface), and fallback to the regular SYS_getrandom.
In the case of the vdso interface, currently the arguments just get
passed through.
Keeping this as a WIP until the ARM64 kernel patches land and I can
actually test the things. Running the "glibc" path works today with the
selftest, but the vdso path is currently completely untested.
Between this version and our previous version v1.3.278 this adds a few
extensions.
- VK_EXT_device_generated_commands
- VK_EXT_depth_clamp_control
- VK_AMD_anti_lag
- VK_KHR_pipeline_binary
These extensions are almost immediately getting picked up by mesa and
the D3D layers, so we want to support them as soon as possible.
Once again updated as described in #2076. This is starting to take more
time as more features and extensions get added to Vulkan. We probably
want to update the `DefinitionExtract.py` script at some point to
contain all the tertiary data in some json so everything gets generated
automatically. Not doing that today but something to think about.
Fixes#4083
From prep commit 511103ee5dcc9474b0b7468c05f61ce10fea4393.
Now that the frontend is setup, we can remove the temporary TLS
variables and use the ThreadObject directly.
Moves from FEXCore to FEX::HLE. Also moves the ThunkFunctions that get
exposed to a namespace to make it more obvious that these are
thunkhandlers rather than just static functions.
With the new systemd support, we had forgotten to wrap this check in an
arm64 check. I had done an install and broke my x86 machine.
Add the check back so we stop breaking x86 machines...again. Took me
only about three hours to find the issue this time.
Global rootfs doesn't support a named rootfs (only local supports that).
So when it is a global rootfs option just give the full path to the
config path.
Just needs some special handling to ensure the full path is retained for
global paths.
A reimplementation of #4067 that doesn't hardcode paths and routes the
global data path through FEXCore.
Using select initially was a mistake, it has issues with FD limits and
is less efficient due to fd_set. It's recommended to switch select using
over to polling, so do it.
we had a bug with max of nans. since x86 behaviour is weird, add a unit test to
try all the weird things. this fails on main without AFP
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We were creating a copy of the FEXCore::Core::CPUState object when we
didn't need to. We can pass the host thread's CPUState frame through to
the creation handlers since it's read-only (so modify it to be const).
We then just move the RAX and RSP setting to /after/ the CreateThread
handling instead of before.
This reduces stack usage from ~1392 bytes to ~80 bytes.
NaN and signed zero handling was busted. Pretty nasty.
Fixes the following Vulkan CTS test run under emulation without thunking with a clang-built Mesa:
dEQP-VK.spirv_assembly.instruction.graphics.float_controls.fp32.generated_args.denorm_nmax_nan_flush_to_zero_vert
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
EnvLoader::Load was generating strings on the stack and then copying
them to the FEXCore::Config state. FEXCore::Config::Set supports
string_views directly which will emplace in to the map directly.
This improves this functions stack usage from ~4464 bytes to ~288 bytes.
Since the frontend has changed to informing the backend if AVX is
supported, there is no reason to feed that configuration back in to
SignalDelegator from the backend.
Instead inform the SignalDelegator directly in the frontend instead of
this now weird round-about path.
This was a legacy interface that has long lost its implementation and
has long lost its use. This interface was similar to an older
Dolphin-Emu interface that was useful in that project, but isn't really
useful in a world with hundreds of threads and processes running at the
same time.
Previously reported as some old CPU without AVX and SSE4 and other
things.
Start advertising as something more modern that actually shipped with
AVX2 and other features. Should help some modern libraries that do bad
family and model checks rather that CPUID features checks.
Also removes the silly `(ES)` tag from CPU-Z.
Also moves generation in to a constexpr function that can actually range
check these 4-bit and 8-bit values.
Only doing the single table for review purposes. Once reviewed I will
hammer out the remaining tables.
Similar to #3320, most of the OpcodeDispatcher tables can be consteval
and made to be a compile time constant. This just requires shuffling the
code slightly. The idea is to get almost all of the table setup out of
the `InstallOpcodeHandlers` function and instead only install the
handlers that change based on 32-bit or 64-bit, just like the x86 tables
we also did.
Upstream has rejected this flag which would have let FEX close the gap
in functionality between binfmt_misc interpreters and PT_INTERP
interpreters for how `/proc/exe` is handled. Since we are unable to
change their opinions, just remove the code from FEX since it's never
going to happen.
Removes a little bit of confusing behaviour in execveat and
filemanagement handling. Leaving us with only the regular confusing
behaviour of trying to track accesses to `/proc/exe` instead.
This was causing problems in a typical use case of piping stdout/stderr
to another application. So if FEXInterpreter was started, with stdout
piped to another application, and FEXServer wasn't running so a new
instance needed to be started. FEXServer would write a null to every
pipe then close it. This includes the stdout that is being piped to the
new application!
This writing was legacy behaviour before FEX was properly listening to
POLLHUP and is no longer necessary. Just remove the writing completely
to fix this issue.
Easiest test case was just `./Bin/FEXInterpreter `which ls` / | xxd | head -n 3`
Returned:
```
$ ./Bin/FEXInterpreter `which ls` / | xxd | head -n 3
00000000: 0000 0000 6269 6e0a 6269 6e2e 7573 722d ....bin.bin.usr-
00000010: 6973 2d6d 6572 6765 640a 626f 6f74 0a64 is-merged.boot.d
00000020: 6576 0a65 7463 0a68 6f6d 650a 6c69 620a ev.etc.home.lib.
```
That first null shouldn't be there.
CEF on Windows patches these before FEX is even loaded, so this needs to
be setup as early as possible using some direct syscalls which skip any
such patches.
No functional change here.
- CoreRunningMode enum and variable wasn't used anymore.
- Code was moved to the frontend
- CustomCPUFactory wasn't used anymore
- All special signal handling and various features were moved to
TestHarnessRunner
- We also don't want to support actual custom CPU cores.
- TestHarnessRunner just runs as a host runner if compiled on an
x86-64 device if vixl sim isn't enabled now.
- Removes the Core config option entirely.
- Moves VDSOPointers struct to the frontend
- Every use of this lives in the Linux frontend instead now
This is one of the most expensive headers in FEX, averaging 1.2 seconds
of compile time per include.
We include this header 31 times around the project. Remove
the five instances where it is unused to help this issue. Next step
would be to make the header lighter if possible.
Since `HadError` is a lambda that handles FD closing this was a double
FD close. Just move the xxhash state freeing in there as well and remove
the FD and hash freeing. Fixes the double free.
Like how FEXCore disabled vixl linking, make sure we aren't accidentally
compiling and including it when vixl stuff is disabled. Noticed this in
the build logs.
We have been relying on the
`YesIKnowImNotSupposedToUseTheGlibcAllocator` fault avoidance for a long
while now. The plan was to rewrite the symbol fetching in the future to
avoid this since VDSO is kind of special anyway.
That time is now, I have had this VDSO symbol parsing code living in a
different project for a few months now and it is working great.
The basic things here are that the Linux kernel provides the VDSO
mapping base pointer through the auxv value `AT_SYSINFO_EHDR`. We then
need a minimal ELF parser that /only/ parses the dynamic symbol header
(and accompanying string header).
Then it's a simple case of walking the symbol table and recording the
pointers. Confirmed this fetches all the correct symbols (As I've been
using it for a while already I already knew it worked.)
This lets us stop allocating memory through the glibc allocator due to
dlopen.
`Extract` was never set to false even if the user said not to overwrite.
Ensure that if the user decided not to overwrite an existing folder,
that it returns false so it correctly doesn't overwrite the folder.
When the shift amount is >= 16-bytes then we need to zero the register.
We had a bug where we were assigning `Result.High` to itself, which
effectively made the top 128-bits of the ymm register not modify itself.
Adds a unit test to ensure that doesn't happen again.
more efficient to pool the output of constant folding and such. but we need to
avoid that becoming accidentally quadratic, so rework that too
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
The previous approach assumed that ntdll wouldn't have been patched
before FEX was loaded, but that isn't necessarily the case thanks to
CEF. This takes the same approach used by xtajit which wine recently gained
support for wine. The overhead to this isn't ideal but most syscalls
aren't directly done through x86 code anyway and in the future FEX could
forward unpatched thunks to their ARM variants at JIT time.
it's only really load bearing for pf/af, which is handled as a global flag opt
now. this mitigates some of the compile time hit from globalizing flag opts.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
another global CFG-based optimization -- if we know that the raw PF is already
1-bit we can skip parity evaluation, saving work with floating point compares.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This works fine on real Windows and is relied on by wine as SystemCall
is set to 1 in KUSER_SHARED_DATA, which causes the ntdll thunks to use
it over `syscall`
The pointer is allowed to be null or garbage if the number of nfds
passed in is zero.
Unify the x86 and x86-64 implementations to ensure consistency.
Fixes Warhammer 40k: Relics of War
Because we already pooled the _Constants, there's no benefit to also pooling inline constants. the robin map to do so just adds extra overhead for no benefit - drop it.
without multiblock, shaves around 2% off node. with multiblock, a bit less than
1% but still a win.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
These will differ depending on which runner they are running on. Just
disable them but keep the instructions around so show how bad rdtscp is,
but how good rdtsc is.
When I implemented TSC scaling originally, I chose a scale factor of 128
because it basically covered the range of devices we cared about without
going too high. I also only tested devices that had a TSC scale factor
from 19.2Mhz to 34Mhz. Turns out there is hardware that also has a 48Mhz
cycle counter, which cause them to effectively have a 6.1Ghz cycle
counter, which is kind of absurd.
Instead of a fixed scale, just calculate the amount of scaling we need
to get >= the minimum threshold of 1Ghz. This will change the shift from
7 to 5 or 6 for the faster cycle counter devices.
Of course if someone wants to know the scale factor they can still use
cpuid function 15h to know it.
Fixes#4026
Seccomp is a relatively complex feature that was added to Linux back in
2005, and was further extended in 2013 to support BPF based protections.
Once seccomp is enabled, you can no longer disable seccomp but
additional protections can be placed on top of existing seccomp filters.
Additionally seccomp filters are inherited in child processes, which
ensures the process tree can't escape from the secure computing
environment through child processes.
The basis of this feature is a shim that lives between userspace and the
kernel at the syscall entrypoint.
In "strict" mode, seccomp only allows read, write, exit, exit_group, and {rt_,}sigreturn to function.
When in "filter" mode, a BPF filter is run on syscall entrypoint and
returns state about if the syscall should be allowed or not. Multiple
filters can be installed in this mode, all of which get executed. The
result that is the most restricted is the action that occurs at the end.
There are some significant limitations in filter mode that must be
adhered to which makes executing this code inside of kernel space a
non-issue and effectively limits how much cpu time is spent in the filters.
Although these filters are free to do basically anything with the
provided data, just can't do any loops.
FEX needs to implement seccomp because there are multiple applications
using the feature, the primary one being Chromium which some games embed
without disabling the sandbox. WINE also uses seccomp for capturing
games that do raw Windows system calls. Apparently Red Dead Redemption
is one of the games that requires this.
While FEX implements seccomp, it is not yet all encompassing, which is
one of the reasons why it isn't enabled by default and requires a config
option.
**seccomp_unotify is not implemented**
This is a relatively new feature for seccomp which lets the seccomp
filter signal an FD for multiple things. Luckily Chromium and WINE don't
use this. This will be tricky to implement under FEX since it
requires ioctl trapping and some other behaviour
**ptrace isn't supported**
One feature of seccomp is that it can raise ptrace events. Since FEX
doesn't support ptrace at all, this isn't handled. Again Chromium and
WINE don't use this.
**kill-thread not quite correct**
This isn't directly related to seccomp but more about how we do thread
shutdown in FEX. This will require some more changes around thread state
tracking before fully supporting this. Chromium and WINE don't use this.
kill-process also falls under this
Features that are supported:
- Strict mode and seccomp-bpf mode supported
- All BFP instructions that seccomp-bpf understands
- Inheriting seccomp through execve
- This means we serialize and deserialize the calling thread's
seccomp filters
- An execve that escapes FEX will also escape seccomp. Not much we
can do about it
- TSync - Allowing post-mortem seccomp insertion which allows threads to
synchronize seccomp filters after the fact
Features that are not supported:
- Different arch qualifiers depending on syscall entrypoint
- Just like our syscall handler, we are hardcoded to the arch that the
application starts with
- user_notif
- ptrace
- Runtime code cache invalidation when seccomp is installed
- Currently we must ensure all syscalls go through the frontend
syscall handler
- Runtime invalidation of code cache with inline syscalls will get
fixed in the future.
This currently isn't enabled by default because of the minor feature
problems that haven't been resolved. Currently the Linux Kernel's test
application works for the features that FEX supports, and WINE's usage
can be handled by FEX. Chromium's sandbox doesn't yet work with this PR,
but it only fails due to features unrelated to seccomp.
Having this open for merging now so we can work to resolve the remaining
issues without this bitrotting.
This must have happened during a refactor or something, but since we're
making a copy of the function pointers, it would have only gotten the
version /prior/ to loading host VDSO symbols.
Moves the VDSO thunk definition setting to the end after VDSO symbol
definitions in order to get host VDSO symbols working again.
Two primary things here:
- Remove the static `GlobalDelegator`
- Move the thread_local SignalDelegator::ThreadState information
directly in to ThreadStateObject
Having the ThreadStateObject and the SignalDelegator information
disjoint was confusing but was required when we didn't have any object
in the frontend that could have its own independent data. Since we fixed
this with the `ThreadStateObject` type we can now move this over.
The `GlobalDelegator` object is now instead stored in
`ThreadStateObject` instead.
Instead of using a thread_local variable, we now just consume 8-bytes of
the signal alt-stack since the kernel gives us that information about
where it lives. This then converts all the thread_local usage to use
either the passed in CPU state if it exists, or fetching it from the
alt-stack offset.
Very minor changes in behaviour here, will help when trying to improve
FEX's behaviour around signals.
The Ultra-class SoCs are two Max-class SoCs connected via
Apple's fabric, and thus use the same core revisions as the
Max-class SoCs for both big and LITTLE cores.
Signed-off-by: James Calligeros <jcalligeros99@gmail.com>
These are a Linux construct and should live here. Removes a weird
passthrough API from FEXCore and keeps it in the frontend instead.
This isn't even typically allocated in a real setup, as it's only a
fallback for if VDSO isn't loaded.
The CallbackReturn function stays in FEXCore because it would have
caused an API in the other direction instead.
This variable can't be constexpr initialized since it requires linker
fix-ups, which changes it in to a global static initializer instead.
While this benign as it doesn't allocate any memory, just move it next
to its single use. Removes the static initializer that was there for no
reason.
This is useful for some people.
This doesn't allow building thunks due to the cmake limitation about
tools binaries also getting cross compiled, we need to switch to meson
if we want that, since they recently started fixing this
https://github.com/mesonbuild/meson/pull/12022
But that's an okay limitation for now.
For the remaining syscalls we need to introduce some new concepts around
checking if pointers and string arrays are readable and I don't want to
do that yet.
No functional change, just moving the code.
This is entirely freestanding from the rest of the signal delegator
handling and mostly gets in the way when I'm working with the rest of
the signal handling. Separate it out to improve readability.
Our JIT will happily consume incorrectly formed 256-bit vector operations in a lot of cases when the host CPU doesn't support 256-bit SVE.
This is what caused the bug in #4006. For every vector operation that
can consume a 256-bit size, add an assert that always checks if 256-bit
SVE is supported in those cases.
This will ensure that #4006 doesn't happen again.
Just like the bug in #4006, we were incorrectly passing a 256-bit
comparison in to the float compare IR operations. Once again it would
"safely" decompose in to 128-bit operations without issue.
PR #4007 added asserts to ensure that 256-bit operations aren't emitted
when the host CPU doesn't support 256-bit SVE and detected this.
This handler was incorrectly using 256-bit IR operation sizes. Due to a
quirk with our IR handling, this would "safely" fall back to a 128-bit
operation and work "correctly".
The problem encountered is that since the IR operation is claiming to be
256-bit, when the value got spilled due to register pressure then a
true 256-bit store and load operation would be generated. This would
then emit an SVE load and store, with the expectation of 256-bit SVE
loadstores. This caused a SIGILL on Oryon since it doesn't support SVE,
but even would generate an invalid predicated loadstore on SVE 128-bit
hardware.
Fixes Aperture Desk Job in FEX.
A source of overhead with multiblock is hitting instructions through a
conditional branch that can never be executed. Usually AVX512
instructions in glibc. This causes us to emit partial blocks for a ton
of targets that will never get executed.
Instead, when we have multiblock enabled, if a block hits an instruction
encoding we don't support, then remove all the decoded instructions from
the block and early terminate it if it isn't the entry block. This
resolves the issue of emitting a bunch of IR and code for blocks never
executed.
If the block of code has an invalid instruction in the entry block for
decoding then it'll still emit code up to the invalid instruction and
raise a SIGILL. This has the potential for generating some additional
blocks of code if a game is abusing SIGILL, but since that's unlikely
it's a good trade-off.
Also removes a few log instructions that don't really provide anything
anymore and just show up as confusing messages when multiblock is
enabled.
For atomics that cross the 16-byte or 64-byte granularity, we need to
lock a mutex to ensure strict emulation of split-locks.
I took another look at these when I found out that Zen3 actually
implements split-locks. Not sure which architecture actually added
support for from them, but I wanted to ensure we have the ability to
handle this.
One thing that we can't handle in user-space is cross-process
split-locks through shared memory. This requires a kernel SIGBUS handler
to ensure a crashing/SIGKILL'd process doesn't lock all FEX processes in
the system.
This fixes a little split-lock abusing test that I have locally. It's a
bit flakey so it isn't viable to run in CI. Considering it is explicitly
testing a race problem.
Required by newer insider preview versions, I noticed many crashes with
this exception number and QueryPeformanceCounter in the backtrace,
testing with XTA found it to not be passed through and instead write
the host CNTVCT (unscaled) into RAX. No other registers seem to be
affected.
The MIDR querying is inherently OS specific and needs a bit of special
casing. Instead let the frontend inform FEXCore how many CPU cores there
are and their MIDRs instead.
This lets us keep the Linux specific code in the frontend.
Fairly trivial because we already supported deferring these signals. We
had just failed the final step of blocking the signal if we can't block
the signal (like with SIGSEGV, SIGBUS, etc).
Once the guest unblocks the signal mask with sigprocmask, the signal
will fire again.
Apparently older glibc relied on this behaviour for signal raising which
this fixes.
There is a common arch bit that is included in each individual mask,
explicitly test for equality against the whole mask to avoid just
testing for this.
When #3899 refactored some of this code, it had split an if-else chain
in to two independent if-else chains. Turns out there was a minor
dependency between them. Add back the few checks necessary to ensure
that FEAT_LRCPC falls through before hitting the FEAT_LSE checks.
This is necessary because their instructions share an encoding so the
masks between the two instruction classes overlap slightly.
useful for checking blow up ratios. I keep calculating this by hand so let's
just add it to the json
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
PR #3980 is adding a feature to merge loadstores in to paired
loadstores, but it was using the incorrect atomic check to determine if
it can safely merge them or not. It was using the GPR atomic check
instead of the vector atomic check. While this would improve performance
on Apple Silicon with its hardware TSO implementation, it would have had
zero impact on Cortex and Oryon.
Instead split out the three config options to live as a boolean check in
the ContextImpl similar to how we disable "AtomicTSOEmulation". Removing
the various configs in the JIT and CPUID so that it queries from the
same context. This makes it clearer that if you are wanting the current
active configuration for memcpy, vector, or general atomic TSO
emulation, you should query one of those three getters.
This also fixes a weird edge case bug in the arm64 JIT where you could
have TSO emulation disable, but still have vector TSO enabled partially.
Just because half a config wasn't checked in {Load,Store}MemTSO for
vectors. If the global "TSOEnabled" option is disabled then TSO should
always be disabled.
Alyssa will be able to pull this in to #3980 once merged and get the
performance uplift on Cortex and Oryon, since our default configuration
is to have vector and memcpy TSO emulation disabled.
Adds support for binfmt_misc through systemd configuration paths. Their
configuration files are basically the raw kernel interface description
in a .conf file, quite a bit more simple than the legacy debian path.
Default enable this path since systemd is the expected default
arrangement these days.
Fixes#2417
rotate right by less than 8:
ror(________________7654321076543120, ...)
rotate left by less than 8:
rol(76543210________________76543210, ...)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
avoid masking. apparently even modern compilers will do cute tricks with 8-bit
math in hot loops ...
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
With recent bug fixes, WFE now can sleep for roughly as long as the
programmed architecture timer of 100 microseconds. Still nowhere near as
close as what x86 CPUs can get with waitx and waitpkg, because those
don't get spuriously woken up by an architecture timer.
100 microseconds is a significantly improvement over 52ns although.
not used. we'll probably rip the whole thing out at some point but for now, no
reason to pollute user systems with this.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Instead of clearing a hardcoded 16 bytes, adjust for the actual number
of instructions modified. The implementation will still only clear a
single cacheline so it doesn't change behaviour.
This used to exist in the FEXCore header since the unaligned handler was
done in the frontend. Once it got moved in to FEXCore it had stayed
there. Move it over now.
In the case of a visibility tear when one thread is backpatching while
another is executing. The executing thread can /potentially/ see the
writing of instructions depending on coherency rules or filling of
cachelines.
By ensuring the DMB instructions are backpatched over the NOP
instructions first, this ensures correct atomic visibility even on tear.
When code buffers are shared between threads, FEX needs to be careful
around backpatching its code buffers, since one thread might have
backpatched the code that another thread was also planning on
backpatching.
To handle this case, when the handler fails to find a backpatchable
instruction, check if it was already backpatched. This can be determined
by atomically reading the instructions back and seeing if they have
turned in to the non-atomic variants.
In most cases we can just return saying that it has been handled, in the
case of a store we need to back the PC up 4 bytes to ensure the DMB is
executed before the non-atomic store.
These handlers don't do any code backpatching so locking the spinlock
futex isn't necessary. Move them before the lock to make them a bit more
efficient once code buffers get shared.
If we query the CPU flags ourselves then vixl is no longer a
compile-time or runtime dependency unless the vixl disassembler or
simulator is built.
A bit spicy from all the feature bits we need to load up.
they are both tiny and seemingly unmaintained upstream, let's bring them
under our own umbrella in tree.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Somewhere there was an assumption made that INC and DEC supported the
repeat prefix. This isn't actually the case, while the prefix can be
encoded, it is a nop and should only expect to be used for padding.
Adds a unittest to ensure that behaviour is as expected.
With the previous Copy{To,From}User helpers we need to actually
implement the handlers correctly. We want something that is a bit
lighter so we don't need to implement the faulting path in the syscall
handlers.
Implements a handful of helpers that just check for readable and
writable capability which can be thrown in to an assertion handler that
is zero cost in release mode.
Readable is checked by just attempting to read all bytes.
Writable is checked by attempting to read each byte and writing it back
to the same location.
Uses these helpers in x64/FD.cpp to showcase how they will be used to
detect EFAULT. Tested locally that they work correctly by writing some
small tests for the syscalls that expect EFAULT.
We now have two types of destinations:
* regular destinations. These are SSA. You get exactly 1 per instruction. This
is what almost every instruction should use.
* special destinations, introduced here. These are *not* SSA. They must be
allocated with a special instruction (added later in this PR), and then they
are mutated by the instruction. There are two types, either pure destinations
("out") or read-modify-write source+destinations ("in-out"). The former are
useful for instructions that return multiple destinations, like Memcpy. The
latter are useful for instructions that need a source tied with a special
destination (currently just Pop, introduced later in this series).
Special destinations reuse the mechanism of sources, to get around the
limitations on regular destinations in our current IR. Ops with special
destinations desugar to ops with no destination but extra sources prefixed Out
or Inout.
They further require HasSideEffects so we don't optimize ourselves into corners.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
- We can have the SyscallFunctionDefinitions be the correct size out of
the gate. Both tables are always 512 entries in size.
- In the RegisterSyscall_{32,64} handlers, just get the reference using
operator[]. We always know we will be under the size of the array, add
a an assert to check. Removes a bit of vector range checking overhead.
- Namespace 32-bit syscalls like 64-bit syscalls and include in the
regular header like 64-bit. This was just an oversight
- Use std::fill for the syscall gap for the invalid syscall, just a
minor cleanup.
No functional change.
I don't know what I was thinking when I wrote that code. Drop the silly logic
and let ConstProp inline the immediates. This fixes a lot of silly code
generated for 32-bit.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
When a suspend request occurs on a running thread, Windows gives the JIT
~10s to pause and NtContinue with the x86 context. Resumption is then
done through BeginSimulation.
ARM64EC doesn't have an explicit callback for suspend, instead this task
is delegated to the kernel setting a bool at a JIT-defined memory
location, as such the faulting based approach as was done for WOW64
cannot be used.
After thread creation, the WOW64 CPU area context needs to be flushed
into the FEX state before entering the JIT. Wine explicitly calls
BTCpuSetContext to trigger this but Windows doesn't.
Some programs will hook the NTDLL exports that FEX depends on, the
regular ARM64EC call checker will detect such patches and invoke the
JIT to run them, which leads to infinite recursion if those same
exports are used during code compilation. Fix this by resolving all
patchable FFSs to their native ARM implementations for all indirect
calls performed by FEX, skipping any x86 patches.
The class constructor for ContextImpl::CPUID requires HostFeatures to be
available at construction time. Pass the host features struct directly
through during construction time instead, which cleans up the interface
slightly and fixes that issue.
These are only missing if using the hostrunner and the CI machine
doesn't support that particular feature. FEX otherwise always supports
these feature flags so they don't need to exist as options.
Just check the feature bit directly in the HostRunner frontend for these
bits.
These are already linked in by default with clang, sp having these set
here only served to prevent the -nostdlib compiler option from having
any effect, as CMake will always explicitly include the libs in the
compiler cmdline.
It is dangerous for FEX to rely on the system CRT as calls can have side
effects that are also visible to the running application, and if patches
are applied to any CRT exports used during compilation the call checker
would trigger a reentry into the JIT to compile the patch and hence
deadlock. Only functions that FEX actively uses are implemented, with
the rest triggering an abort.
This moves the CPU feature querying to the frontend. The primary purpose
here is for the wow64 frontend to not require linux-isms for querying
these features. This is required since non-Linux environments don't have
the "CPUID" feature for reading EL1 MSRs in EL0.
Wiring up the remaining wow64 registry querying is left for a future
exercise.
This also technically removes an xbyak requirement from FEXCore for when
building the x86 Test harness runner, but that doesn't really matter for
regular use cases.
Started by cherry-picking some cases from the variants that appeared when running
Steam, games, AV1 convolve tests, openssl, ffmpeg, libjpeg-turbo,
openh264, libvpx, gemmlowp, libyuv, and dav1d.
Then turned it around and optimized them all since all variants end up
needing to be split in to two halves, that effectively means we need to
have 16 implementations, plus a couple of special cases for duplicated
results.
Fixes#3795
These two maps used for environment lookup translations were getting
globally initialized and then registers with atexit handlers.
Switch over to a constexpr array and just do linear scans. This plus
short-circuiting the environment loader so it skips all entries that
don't start with `FEX_` has the side benefit of cutting the CPU time to
1/10th the time.
This plus #3917 removes the global static initializers entirely from
this file.
An exception in JIT code acts as a transition to ARM64EC code (in
NTDLL for exception handling etc) as such, much like ExitFunction,
InSimulation must be unset. InSyscallCallback is unset for robustness
against exception in the JIT itself.
Only NZCV and TF are passed through to BeginSimulation as the rest are
lost when converting to a native context and back on the ntdll side. To
prevent thread suspension from wiping out the rest of the flags, only
copy these specific flags into the current JIT EFlags state.
Most syscalls on Windows are done by calling into their NTDLL thunks,
however some DRMs parse out their numbers from NTDLL and directly call
them. Support this by redirecting to their entry thunks in the FEX
syscall handler.
To prevent FEX from redirecting to x86 code when NTDLL exports it calls
into are patched, a custom call checker will be used that checks this LUT
to redirect calls rather than the FFS itself.
- IsImmLogical already existed in our CodeEmitter. We just forgot to
allow nullptr arguments and to use it.
- Adds an equivalent IsImmAddSub helper and uses it
This gets us closer to removing vixl's global initializers from FEXCore.
Saw this vector was getting initialized at runtime, sticking around, and
installing an atexit handler. This is completely unnecessary, just use
the OPT_BASE handler directly to walk the environment variable names.
This is required for handling SMC with the ResetToConsistentState
arguments as used in Windows, as using the NTDLL exported NtContinue
would wipe out any reserved registers in the ARM64EC ABI.
For Windows the syscall numbers are somewhat stable, and the SVC
instruction can be called directly. Since wine doesn't handle that on
ARM64, hardcode the system call number and manually call into wine
dispatcher. Once wine gains proper syscall thunks, those can be
parsed to get the number and the hardcoding dropped.
OutputDebugString etc are exception based and thus don't really work for
FEX's needs as often times logs can happen in places where exceptions
cannot be thrown.
The prior approach using the L2 cache was flawed as it assumed L2
page entries had a 1-1 correspondence with actual pages. While the L2
cache could be extended to handle aliases with EC, this could lead to
thrashing etc. The cost of a lookup in the actual EC code bitmap is
cheap enough to perform every time considering the infrequency of calls
to ARM64EC code when compare to X86 L2 hits.
In the environment section this was causing the next environment
variable line to be merged with the strenum options
Also makes it so strenum options doesn't have a spurious comma at the
end of the list.
Different approach from #3579
Instead of completely ddropping the deprecated path, support the new
path and the old path using python try-except import exceptions.
This allows us to continue using old packages in CI, while supporting
the future API once pkg_resources gets deprecated and removed. Best of
both worlds.
Using a brute force solver to add in more optimized code paths
- Adds 12 single VInsElement implementations
- Adds 4 two IR operation implementations
Not adding any of the two or three IR operation implementations that use
VInsElement because SRA interacts badly and becomes worse than the VTBX
implementation.
This has been leaked state to FEXCore for quite a while. FEXCore never
actually needed this information, moves the bits to the frontend that
are necessary.
Minor behaviour change that `RunUntilExit` now just assumes the primary
thread is using it. This behaviour is on the chopping block to get
removed next anyway.
Instead of passing the TID back to the exit handler, just pass the whole
thread object. This will allow some cleanups with the frontend thread
tracking soon
NFC
Runs through all immediate encodings for vpblendw and crcs the results
to ensure correct behaviour. This was just a concern because of the typo
in documentation. But it is also good to have.
Optimizes the AVX128 blends by reusing the prior SSE4.1 implementation.
Only difference is the destination register isn't reused as a source
register.
One confusing thing is that Felix Cloutier's documentation has a typo on
the 256-bit VPBLENDW instruction where it had the top 128-bit lane
reusing the destination instead of sources. So I wrote a unittest to
ensure correctness.
Fixes#3796
stop prefixing the arguments when we generate allocate ops (in particular), this
is more convenient and simpler. in exchange we need to prefix Op to avoid a
collision on fcmpscalarinsert which has an argument named Op, but that's a local
change at least.
came up when experimenting with new IR, but I think this is probably a win by
itself.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
now that we do everything via NZCV, this is mostly vestigial. DF/x87 flags are
sufficiently rare to be "don't care"s here, and we don't even have multiblock
enabled yet!
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is no longer necessary and it also no longer provides us any useful
information. Since we expose the AVX CPUID flag, basically everything
uses VEX encoding now, so it is basically always set.
Some locations could end up with SRA registers that only spilled one
register.
Allow passing in temporaries from the call site.
Fixes rpid and syscalls asserting.
When AFP is supported then we can actually support DAZ. This might also
fix the audio corruption in Animal Well but I can't test it until Steam
is running on Oryon. Requires a bit of plumbing for MXCSR which we were
hacking around before but now we actually want to store the value.
Fixes#3856
A disowned buffer could be unclaimed or claimed by a different thread in
the time between the !IsFree check and locking the allocation mutex.
Fix this and prevent such errors in the future by always checking
ownership with the allocator locked before attempting to unclaim
buffers.
We support leaf functions now, so add the few that were calling for it.
We will be gaining support for the xsave ones relatively soon, so its
good to have them supported.
Also deletes a couple of cdt/cqm things that aren't exposed and we won't
be supporting.
With a brute force search of methods between 1-3 instructions we cover a
lot more cases more optimally.
There's definitely still more cases (and probably some that can reduce
from 3 instruction to 2), but covering 44 cases is a pretty good margin
already.
VPSHUFD and VPERMILPS are aliases of each other.
Reuses the implementation path from the PSHUFD implementation which has
a few swizzles and then a table lookup.
VPERMILPD is a very simple swizzle per 128-bit lane.
Fixes#3797Fixes#3784
nothing is optimizing around this, it's just adding pointless complexity. if we
want to actually optimize F80Cmp, the right way would be to lift the
implementation into the OpcodeDispatcher or JIT. it wouldn't be terribly
difficult. This kludge doesn't get us closer there.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Disabled_Tests was mostly a copy of Known_Failures. Leave only the race
condition on SIGPROF test (mcount_pic.c).
Also remove pr57275.c from known failures. It passes now that we support
AVX.
Also if a test is disabled, just skip it
We were only running SVE256 and SVE128 with AVX disabled.
Enable asm tests with SVE256, SVE128, and ASIMD, all running with AVX
enabled to hit all the tests.
FEX emulates faulting instructions (e.g. ud2 or int 2d) by jumping to
the dispatcher and filling out a structure with fault details in the
thread context. Parse this out into a windows exception record structure
so the correct fault information can be seen by the guest.
As the exception dispatcher is initially invoked on the emulator stack,
control needs to be transferred to the dispatcher on the guest stack
after recovering the x86 RSP to allow for invoking x86 exception
handlers.
This is used by the kernel (or UNIX side of ntdll in wine) to jump into
x86 code with the given context as is necessary when e.g. returning from
an exception.
FEX is unable to deal with reentrant compilation of any x64 hotpatches
so they need to be ignored by bypassing FFSs and calling directly into
the native target.
ARM64 requires that SP is always 16-byte aligned for memory accesses,
but ARM64EC shares the SP between x64 code and ARM64 code, the former
of which doesn't enforce such a restriction. This causes crashes in
programs such as HITMAN 3 that don't correctly follow the Windows ABI
and call into system library functions with SP only 8-byte-aligned.
Fixup stack alignment in such cases by leaving the 8-byte return
address on the stack and returning to a lone 'ret' instruction instead.
This allows for running x64 applications under wine without having to run all
of wine under FEX. The JIT is invoked when ARM64EC code performs an indirect
branch to x64 code, and left whenever the x64 code calls into ARM64EC
code.
Removes a global initializer and atexit registration
Ownership of this data has always been the frontend and the config
system, we just used these static vectors as a side-channel.
Removes a global static initializer for the vector and its atexit
handler.
This handler array can be consteval similar to the x86 tables so it can
be generated entirely at compile time.
This causes a global initializer that registers an atexit handler.
Be smarter, use an std::array and pass its data around using a span
instead.
Removes the global initializer and removes the atexit installation
We never use more than one logging method at a time so this was
overengineered for what it is doing.
Instead only allow one handler for messages and throw messages each
which just is a pointer.
Removes a global initializer and an atexit handler being installed
This is the initial split to decouple AVX256 composed operations from
their MMX/SSE counterparts. This is to work around the subtle
differences with AVX/SSE zext/insert behaviour.
This was doing a 128-bit load from memory and then a 64-bit zero extend
which looked like a spurious move but it was trying to match the
behaviour of vmovq where it needed the zero extend.
Also adds a unit test to ensure that we aren't loading too much data by
loading right up against a page boundary.
Fixes#3787
Also fixes a bug where it was failing to zero the upper bits of the
destination register in the AVX128 implementation. Which the updated
unit tests now check against.
Fixes a minor precision issue that was reported in #2995. We still don't
return correct values for overflow. x86 always returns maximum negative
int32_t on overflow, ARM will return maximum negative or positive
depending on sign of the double.
The codepath from #3826 was only targeting 256-bit sized operations.
This missed the vpgatherdq/vgatherdpd 128-bit operations. By extending
the codepath to understand 128-bit operations, we now hit these
instruction variants.
With this PR, we now have SVE128 codepaths that handle ALL variants of
x86 gather instructions! There are zero ASIMD fallbacks used in this
case!
Of course depending on the instruction, the performance still leaves a
lot to be desired, and there is no way to emulate x86 TSO behaviour
without an ASIMD fallback, which we will likely need to add as a
fallback at some point.
Based on #3836 until that is merged.
With the introduction of the wide gathers in #3828 this has opened new
avenues for optimizing these cases that would typically fall back to
ASIMD. In the cases that 32-bit SVE scaling doesn't fit, we can instead
sign extend the elements in to double-width address registers.
This then feeds naturally in to the SVE path even though we end up
needing to allocate 512-bits worth of address registers. This ends up
being significantly better than the ASIMD path still.
Relies on #3828 to be merged first
Fixes#3829
SVE has a special version of their gather instruction that gets similar
behaviour to x86's VGATHERQPS/VPGATHERQD instructions.
The quirk of these instructions that the previous SVE implementation
didn't handle and required ASIMD fallback, was that most gather
instructions require the data element size and address element size to
match. This x86 instruction uses a 64-bit address size while loading 32-bit
elements. This matches this specific variant of the SVE instruction, but
the data is zero-extended once loaded, requiring us to shuffle the data
after it is loaded.
This isn't the worst but the implementation is different enough that
stuffing it in to the other gather load will cause headaches.
Basically gets 32 instruction variants to use the SVE version!
Fixes#3827
If the destination isn't any of the incoming sources then we can avoid
one of the moves at the end. This half works around the problem proposed
in #3794, but doesn't solve the entire problem.
To solve the other half of the moving problem means we need to solve the
SRA allocation problem for this temporary register with addsub/subadd, so it gets allocated
for both the FMA operation and the XOR operation.
When loading 256-bits of data with only 128-bits of address indices, we
can sign extend the source indices to be 64-bit. Thus falling down the
ideal path for SVE where each 128-bit lane is loading the data to
addresses in a 1:1 element ratio.
This means we use the SVE path more often because of this.
Based on top of #3825 because the prescaling behaviour was introduced
there. This implements its own prescaling when the sign extension occurs
because ARM's SSHLL{,2} instruction gives us that for free.
This additionally fixes a bug where we were accidentally loading the top
128-bit half of the addresses for gathers when it was unnecessary, and
on the AVX256 side it was duplicating and doing some additional work
when it shouldn't have.
It'll be good to walk the commits when looking at this one, as there are
a couple of incremental changes that are easier to follow that way.
Fixes#3806
Was missing a clamp on the high half when encounting a 128-bit gather
instruction. Was causing us to unconditionally load the top half when it
was unncessary.
FEX allocations can get in the way of allocations that are 4gb-limited
even in 65-bit mode (i.e. those from LuaJIT), so allocate starting from
the top of the AS to prevent conflicts.
If the host supports SVE128, if the address element size and data size is 64-bit, and the scale is not one of the two that is supported by SVE; Then prescale the addresses.
64-bit address overflow masks the top bits so is well defined that we
can scale the vector elements and still execute the SVE code path in
that case. Removing the ASIMD code paths from a lot of gathers.
Fixes#3805
the special case here lines up with the special case of using a larger shift for
a smaller result, so we can just grab CF from the larger result.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
The two halves are provided as two uint64_t values that shouldn't be
sign extended between them. Treat them as uint64_t until combined in to
a single int128_t. Fixes long signed divide.
For all the 32-bit rotates (except for RORX) we were failing to zero
extend the 32-bit result to the destination register when the rotate was
masked to zero.
Ensure we do this.
this fails on current main with blocksize=500 due to mentioned RA bug. passes
with blocksize=1.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
At the moment we always run ctest with max number of cpus. If
undefined, it will keep current behaviour, otherwise it will
honour TEST_JOB_COUNT.
Therefore to run ctest one test at a time, use
`cmake ... -DTEST_JOB_COUNT=1`
We had failed to enable these implementations for the
`ExtendedMemOperand` helpers. We had already implemented the non-helper
forms, which are already tested in CI. These helpers just weren't
updated?
Noticed this when running libaom's SSE4.1 tests, where it managed to
execute a pmovzxbq instruction with reg+reg memory source and was
breaking the test results.
There are /very/ few vector register operations that access only 8-bit
or 16-bit in vectors so this flew under the radar for quite a while.
Fixes their unit tests.
Also adds a unittest using sse4.1 pmovzxbq to ensure we support the
reg+reg case, and also a few other instructions to test 8-bit and 16-bit
vector loads and stores.
We didn't have a unit test for this and we weren't implementing it at
all.
We treated it as vmovhps/vmovhpd accidentally. Once again caught by the
libaom Intrinsics unit tests.
This is a very minor performance change. On Cortex CPUs that support
SVE, they do movprfx+<instruction> fusion to remove two cycles and a
dependency from the backend.
This is a minor win to convert from ASIMD mov+bsl to SVE movprfx+bsl
because of this, saving two cycles and a dependency on Cortex A710 and
A715. This is slightly less of a win on Cortex-A720/A725 because it supports
zero-cycle vector register renames, but it is still a win on Cortex-X925
because that is an older core design that doesn't support zero-cycle
vector register renames.
Very silly little thing.
The canonical way to generate a zero register vector in x86 is to xor
itself. Capture this can convert it to canonical zero register instead.
Can get zero-cycle renamed on latest CPUs.
x86 doesn't have a lot of non-temporal vector stores but we do have a
few of them.
- MMX: MOVNTQ
- SSE2: MOVNTDQ, MOVNTPS, MOVNTPD
- AVX: VMOVNTDQ (128-bit & 256-bit), VMOVNTPD
Additionally SSE4a adds 32-bit and 64-bit scalar vector non-temporal
stores, which we keep as regular stores. Since ARM doesn't have matching
semantics for those.
Additionally SSE4.1 adds non-temporal vector LOADS which this doesn't
touch.
- SSE4.1: MOVNTDQA
- AVX: VMOVNTDQA (128-bit)
- AVX2: VMOVNTDQA (256-bit)
Fixes#3364
Arm64's SVE load instruction can be minorly optimized in the case that a
base GPR register isn't provided, as it has a version of the instruction
that doesn't require one.
The limitation of this instruction is that it doesn't support scaling at
all so it only works if the offset scale is 1.
When FEX hits the optimal case that the destination isn't one of the
incoming sources (other than the incomingDest source) then we can
optimize out two moves per 128-bit lane.
Cuts 256-bit non-SVE gather loads from 50 instructions down to 46.
Some applications don't measure rdtsc correctly and instead use cpuinfo
to get the CPU core's base clock speed. Which for most x86 CPUs is the
base clock speed which also matches their cycle counter speed.
Did this as a quick test to see if this would help `Unbound: Worlds
Apart` stuttering while BinaryNinja was disassembling the binary.
Turns out the game doesn't use cpuinfo for its cycle counter speed
determination, but it is good to implement this regardless.
We can support a few combinations of guest and host vector sizes
Host: 128-bit or 256-bit
Guest: 128-bit or 256-bit
The typical case is Host = 128-bit and Guest = 256-bit now that AVX is
implemented.
On 32-bit this changes to Host=128-bit and Guest=128-bit because we
disable AVX.
In the vixl simulator 32-bit turns in to Host=256-bit and Guest=128-bit.
And then in the vixl sim 64-bit turns in to Host=256-bit and
Guest=256-bit.
We cover all four combinations of guest and host vector register sizes!
Fixes a few assumptions that SVE256 = AVX256 basically.
In regular SIB land the index register encoding of 0b100 encodes to "no
register", this feature lets you get SIB encodings without an index
register for flexibility.
In VSIB encoding this isn't expected behaviour and instead there are no
encodings where an index register is missing. Allowing you to encode all
sixteen registers as an index register.
This was causing an abort in `AVX128_LoadVSIB` because the index turned
in to an invalid register.
Working instruction:
`vgatherdps ymm2, dword [eax+ymm5*4], ymm7`
Broken instruction:
`vgatherdps ymm0, dword [eax+ymm4*4], ymm7`
This fixes a crash in libfmod where it is using gathers in the wild.
Fixing a crash in Ender Lilies.
When the source or destination is a register, the address size override
doesn't apply. We were accidentally applying it on all sources
regardless of type which was causing us to zero extend on operations
that aren't affected by address size override.
This fixes the OpenSSL cert error in every application, but most
importantly Steam.
During refactoring this was missed and rerunning unittests locally
caught it. 256-bit operations get their shift only from the lower half
of the vector register.
We can avoid a LOT of pointless work with some dedicated IR ops for specifically
overriding the round mode.
Small behaviour change here: we no longer reset FTZ. I think this is a bug fix?
But if it's not it's not hard to fix.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This does a gather load three ways, SVE256, SVE128, and ASIMD.
This operation is a bit special since it it can't quite handle all
gather loadstores in the 256-bit case and requires the frontend to
decompose the operation in the case that the striding hits a mode that
SVE doesn't support!
The 128-bit case is a lot simpler since both support all the cases where
stride doesn't match. I find this to be a nice compromise while there
aren't any SVE256 products on the market.
In the 128-bit case there is an SVE path which is utilized if the passed
in stride supports what SVE understands, otherwise it falls back to an
ASIMD implementation which manually emulates everything that is
necessary.
This instruction is very explicitly doing basically exactly what AVX
gather instructions want, because it's complex enough that we don't want
to try and make this a generic solution.
A bit confusing because the instruction encoding is the same between
VMOVHLPS and VMOVLPS so this unittest was missed.
Implement the test to ensure it stays working
The TestHarness infrastructure doesn't understand the difference between
converged versus split view.
So fetch the split view immediately and reconverge the view manually
inside of the state object so it continues working with the split ymm
view.
Because we have two views of the YMM registers depending on if the host
supports SVE256 or not, add helper functions to fetch them correctly.
We fetch them in the way that Linux desires them in signal handlers, if
we want to return the converged view directly, that is easy to add
support for. It's unnecessary for now.
Any time we get the value of Literal, we want to assert that it's actually a
literal. We've been open coding this pattern sporadically throughout the
opcodedispatcher. Let's add an ergonomic helper to fetch the value of literal,
asserting that the value is indeed literal.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Previously we could always tell the size of the operation depending on
how this effects the operating size of the instruction. Converting
64-bit down to 32-bit as an example.
AVX gather instructions are the first instruction class that can't infer
this information. The element load size is determined by the W flag but
the operating size of 128-bit or 256-bit is determined by other means.
Expose this flag so we can determine this difference. The FMA
instructions are going to need this flag as well.
This instruction is a little weird.
When accessing memory, the 128-bit operating size of the instruction
only loads 64-bits.
Meanwhile the 256-bit operating size of the instruction fetches a full
256-bits.
Theoretically the hardware could get away with two 64-bit loads or a
wacky 24-byte load, but it looks like to simplify hardware they just
spec'd it that the 256-bit version will always load the full range.
add a bit of coverage for this funny addressing corner. We do handle this
optimally but I had to write this to check ;)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Only installs the tables if SVE256 isn't supported yet AVX is explicitly
enabled with HostFeatures, to protect accidental enablement early.
- Only implements 85 instructions starting out
- Basic vector moves
- Basic vector unary operations
- Basic vector binary operations
- VZeroUpper/VZeroAll
The bulk of the implementation is currently the handling for loading and
storing the halves of the registers from the context or from memory.
This means the load/store helpers must always return a pair unless only
requesting the bottom half of the register, which occurs with 128-bit
AVX operations. The store side then needing to consume the named zero
register if it occurs since those cases will zero the upper bits.
This implementation approach has a few benefits.
- I can pound this out extremely quickly
- SSE implementations are unaffected and don't need to deal with the
insert behaviour of SVE256.
- We still keep the SVE256 implementation for the inevitable future when
hardware vendors actually do implement it (Give it 8 years or
something).
- We can actually unit test this path in CI once it is complete.
- We can partially optimize some paths with SVE128 (Gathers) and support
a full ASIMD path if necessary.
One downside is that I can't enable this in CI yet because it can't pass
all unittests. but that's a non-issue since it is going to be in heavy
flux as I'm hammering out the implementation. It'll get switched on at
the end when it's passing all 1265 AVX unittests. Currently at 1001 on
this.
This is a different feature flag than regular AES as the default AES+AVX
only operates on 128-bit wide vectors.
With the newer `VAES` extension this is expanded to 256-bit.
Fixes#3690
When doing scalar insertions, upper bits come from different arguments
depending on the operation. These are listed in the ARM spec under the
NEP bit documentation.
The Oryon is the first CPU we know of that implemented support for the
RNG extension. It also has an errata where reading the RNDRRS register
never returns success. X86's RDSEED guarantees forward progress with
enough retries.
When an x86 processor messed this up at one point, some Linux systems
would infinite loop (presumably when something in boot was filling an
entropy pool). This required a microcode change to fix that processor.
The rdseed unittest infinite loops on this platform if RNG was exposed.
to be consistent with the scalar _Andn opcode, which is specifically named _Andn
and not _Bic.
noticed while reviewing AVX patches
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Fixes#3691
We weren't checking if the file was empty before using its `at` function
member. This was causing an early crash if the config file existed but
was empty.
Consolidates the three locations that copy and pasted the json allocator
tools and adds an empty check for all of them.
Also adds two missing checks to the ThunksDB handler that could have
resulted in the same crash if ThunksDB was an empty file.
Needed something inbetween the `InlineJITBlockHeader` and `avx_high` in
order to match alignment requirements of 16-byte for avx_high. Chose the
`DeferredSignalRefCount` because we hit it quite frequently and it is
basically the only 64-bit variable that we end up touching
significantly.
In the future the CPUState object is going to need to change its view of
the object depending on if the device supports SVE256 or not, but we
don't need to frontload the work right now. It'll become significantly
easier to support that path once the RCLSE pass gets deleted.
This is required to be less than the maximum range for LDP and STP in
the Arm64 Dispatcher otherwise it breaks. Necessary to ensure this when
reorganizing the CoreState.
(except for when we explicitly enable AFP).
Since AFP gets saved/restored, we get `msr fpcr` garbage in random instructions
when AFP is enabled. Explicitly disable everywhere since it's not worth our time
to triage which files might hit that path. Fixes instcountci on AFP-supporting
hosts now that we have AFP enabled.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is helpful for devs working on FEXCore, I've been using this locally but it
might make sense to stick it in tree.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
In quite a few locations we are mixing the case that SVE256 == AVX or
that AVX means the guest register size is 256-bit.
While this is true today, this is entanglement is going to change very
quickly and cause confusion in follow-up PRs.
Now we have SVE128, SVE256, and SVE2 HostFeatures to disambiguate the
different features which mean different things.
This PR keeps the alias that `SupportsAVX` = `SupportsSVE256 && SupportsSVE2`
but that alias is going to very quickly change its definition.
This currently doesn't do much but soon this will be very important to
ensure the data prefetcher of Cortex keeps the cachelines following this
variable in L1.
Allows a nice way to get information about how TSO emulation is occuring
on the individual's hardware. In particular, the more subtle details
around the implementation rather than just a hard on or off toggle.
Currently our minimum clang version requirement is 12.0 but soon will
require at least 13 or 14. Add a new version check in cmake to ensure
minimum version requirements.
Makes it easier to determine why a build is failing due to old compiler.
Pulled from the seccomp WIP PR where it pulls this object more frequently.
Since it is an opaque frontend pointer it needs to be cast and we
already have a few locations that use it.
No functional change.
No instcountci changes yet, since nothing currently spills in instcountci. This
mitigates spilling later seen with #3703, and should help for certain
pathological blocks even without those changes (maybe we should try to get some
of those blocks in instcountci?).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Fixes#3675
This was the first time I've ever actually dived in to this code and this
function melted my brain a bit while reading it. It was trying to be too
smart in tracking VMA splits, but if it split right at the end of a VMA
range it would then add a new range at the end where one already
existed. This then caused us to have overlapping VMA ranges and it would
completely break our SMC tracking since all tracking in the map must not
overlap.
Instead of being too smart, just break it down in to 4 merge strategies,
three of which are one shot. This is significantly easier to reason
about and each strategy is mostly self-contained. The fourth strategy in
the list is the most complex since it requires multiple steps since it
needs to walk multiple VMAs.
To help test this I added some sanity checking code that proved
invaluable to ensure everything was correct. That's not getting merged
since the overhead is too much to run, but good to have available. Diff
for this is at
https://gist.github.com/Sonicadvance1/1ee60101ed0742b971a476fadbb51083
FEX had a bug with this instruction where it was incorrectly using both
the address size override and operand size override to truncate the
immediate offset. This isn't how the instruction should behave as it
should actually ignore the address size override.
This now puts it correctly inline with how the jump instruction works
and adds a unit test to ensure it doesn't break again.
This fixes a crash from the Arch rootfs from the glibc dynamic linker
being compiling in a way where a call instruction was getting aligned
using this prefix (Since the compiler knew it does nothing).
The vixl disassembler has gotten more strict about certain instruction types, so these tests
aren't really needed.
Alternatively, we could mark them as unallocated, but we can opt to remove them here.
FEX_ENABLEAVX was removed. Settings for host features should use
FEX_HOSTFEATURES but actually FEX_HOSTFEATURES=enableavx has different
behaviour, so we don't enable it.
Installing PPA with the script now installs a different
URL. This means that GetPPAStatus always returns false.
For the sake of backwards compatibility match the end
of the line to check.
dougallj mentioned that adding these tests might expose
a bug in bextr. Since bextr implementation was changed
apparently it now works correctly, that's good.
slightly worse for compile time, slightly better output, honestly I'll take the
win because this is easier to reason about.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
I don't get the point, it should be handled by a combination of existing
passes/techniques just fine. no instcountci changes.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
No reason to have a separate pass for this, merging should be a bit faster since
it eliminates an IR walk.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
find-and-replace across the tree, excluding IR.h itself.
also excluded IRValidation because its treatment of blocks blows up and will be
reformed in the new IR anyway.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
deduplicate all the things.
functional change:
hit by sse4_1-pmaxuw.c.gcc-target-test-64.jit.gcc-target-64
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
In preparation for seccomp execve inheritance where we need to extract
another FD from a different environment variable.
- Small function to extract the FD and also unset the environment
variable in the same place.
- Keeping the fetch and unset together instead of spreading to
another location in the source.
- Extract the FD upfront instead of passing the string_view around,
since we are unsetting the environment variable at the same place.
Future seccomp inheritance will get the FD just after the FEXFD
- `int FEXSeccompFD {GetFEXFDFromEnv("FEX_SECCOMPFD")};`
When asked to not follow the symlink, FEX needs to return data about the
symlink itself rather than following to the target executable. In that
case we need to return symlink information otherwise games that sanity
check can break.
This is what happened with Darwinia in #3662.
We return the FEXInterpreter symlink information in this case since it
doesn't return any information that is relevent to leaking emulator
state. Once the application asks to follow through to the symlink target
is when we will replace.
Also adds a unit test to ensure we don't break it.
This was causing us to generate invalid code in Darwinia, resulting in a
crash. With assertions enabled this would be picked up in the emitter.
Only implement AddShift optimizations for now because I don't want to do
the remaining optimizations in a bug fix PR.
Fixes Darwinia.
Accidentally we were swapping which sources were the base and which was
the one getting shifted. This wasn't super common so it usually didn't
matter.
Fixes one crash in Darwinia.
Doesn't handle all 127 combinations of the control immediate for all
four instructions. Although supplies the instruction control instruction
that A Hat in Time abuses heavily.
The SSE2 implementation of the function in vcruntime140 is likely faster
than our currently implementation but we should be able to get something
comparable. Not bad considering this is a required extension and this is
the first game we found that abuses the instruction heavily.
This was a bit confusing to read and I had always expected to change
this at some point.
Previous:
```
[INFO][1579518391560577][1601857.1601857] clone: Unsupported flags w/o CLONE_THREAD (Shared Resources), 4100
```
Now:
```
[INFO][1590468.992593376][1629501.1629501] clone: Unsupported flags w/o CLONE_THREAD (Shared Resources), 4100
```
use a vec. block indices will be dense in the new IR. This is memory intensive
but seems faster in practice.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Flag DCE needs to do general DCE anyway to converge in one pass. So we can move
the special syscall/atomic logic over to flag DCE and then drop the second DCE
pass altogether. Now local dead code of both is eliminated in a single pass.
Flag DCE is carefully written to converge in a single iteration which makes this
scheme work.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
If both the destination and the flags are dead for an AddWithFlags, we need to
eliminate it in one pass. If we only replace without elimiating, we would need a
second DCE pass to eliminate. We want DCE to finish in one pass, so fix this.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This has been deadcode since 2020. Drop it so we can focus on what *does* work
and what does matter.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Generally, there are three reasons to track progress:
* Conditional optimizations. E.g. only run DCE if ConstProp succeeds.
* Fixed point optimizations. E.g. keep running the opt loop until convergence.
* Metadata shenianigans.
None of these apply to FEX. We explicitly do not want a nonlinear pass ordering,
instead we want just a few passes that each converge in a single iteration. We
expect them all to make progress when run. As such, tracking progress is a waste
of CPU cycles. Stop doing it.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
New RA does not need it for correctness, and the slight slow down to new RA from
not compacting first is much smaller than the cost of compaction. Overall speeds
up node.js start time by ~6% on top of new RA.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
I recommend viewing the new source file as the diff is quite messy.
---
The old RA commits every "how not to write an RA" sin in the book.
Chaitin spill-one loop? Check.
Potential spilling caused by alignment issues since there's no live range
splitting? Check.
Panic spilling? Check.
Generating an interference graph with linear live ranges, so you get the code
quality of linear scan with the cost of graph colouring? Check.
...
It is wholly unsuitable to any application, and specifically unsuitable for FEX.
---
The new RA exploits a key IR invariant unique to FEX: no values are live across
block boundaries. This is validated.
Because of this invariant, all RA is block local. This lets us use a dead simple
2 pass RA that generates ~optimal code in linear time.
The first pass walks the IR backwards, analyzing the IR. This is a souped up
analogue to liveness analysis.
The second pass walks the IR forward, blasting out registers. If necessary, it
will insert spill and/or shuffle code on the fly. Spilling uses the well-known
furthest-first heuristic, which has excellent results for straight line code.
That's it :-)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Fails with the new RA when copies are inserted, since RAValidation loses
visibility. Nontrivial to fix, but this particular assert hopefully isn't buying
us a ton. Weaken it for now, we can revisit later.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
To implement GPRPair "properly", we need to be able to shuffle scalars around
the register file. That means we need explicit copy/swap instructions that RA
can generate them. Add some.
Swap is split in a really sketchy way, because of the 1 instruction = 1 dest
requirement. Hopefully that requirement is lifted in the future and then this
goes away.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
this was basically practice for using perf with FEX, but a few things do stand
out in the assembly as suboptimal.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
After two months of testing I finally have enough confidence that these
default options getting changed is safe enough that most games won't
notice the difference. But the performance differences can be wild.
Might want to come back and reenable this on platforms that support
hardware TSO and LRCPC3 (for the vector feature), but we can care once
those platforms come up to speed.
Useful smoke test for the asymptoptic behaviour of our constant pooling code.
Not useful for correctness testing, so skip in CI.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
It doesn't write to its node. Fixes spurious
%7: Arg[0] expects reg0 to contain %4, but it actually contains %16
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
now that we've eliminated cross block liveness, we can do our validation locally
too for a massive simplification.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
All we actually need to validate is that each source has been previously defined
within the block. That checks everything we care about now.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
terser and will aid coalescing, as well as eventual transition to multidest
extracts which is what we'll actually want.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This avoids a bunch of sharp edges for RA at a small cost when obscure
segment registers are used.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
From a theoretical perspective, we should not need to run RCLSE more than once.
If there are convergence issues with the current implementation, they should be
fixed instead of bandaged around. Fortunately, this has no instcountci changes.
Brings RCLSE cost down from like 12% to 5%.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
IRListView is now purely a view type. Instead, ownership is managed on-demand
by a separate interface (IRStorageBase). Materialization of IRListViews to
owning types is moved to this interface as well.
This also avoids unneeded copies of the data.
doesn't matter for left shifts (we mask off the garbage), or 32-bit shifts, or
shifts where we explicitly sbfe after.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
In preparation for seccomp execve inheritance.
We are going to need to add a new environment variable earlier in the
execve sequence to handle inheritance in the case of binfmt_misc.
No functional change in regards to envp handling.
Minor change around execveat with FD without binfmt_misc. In the case
that execveat returned an error and we did a `dup` of the FD then we
would have an FD leak. Make sure to close the duplicated FD in that
instance.
alternative to #3638. this is theoretically better for side-by-side diffs. in
practice it may make other diffs worse since all the \'s change when part of the
macro change.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Fixed offset x86 code doesn't quite solve the issue, so adjust this
heuristic just to get instcounci to stop flaking.
This code is going to heavily change soon anyway so +50 doesn't change
much.
Depending on where the assembly was getting loaded in to memory it was
causing slight code generation differences.
Map the entire file to the same fixed offset as our ASM tests to ensure
consistency and removing flakes in CI.
Sometimes github or the CI runner times out trying to checkout the
source and stays timing out forever.
Give it a three minute timeout otherwise the CI runner will stall
forever.
undefined instructions are expected to return SIGILL, while implemented
instructions that aren't available in CPL-3 are expected to SIGSEGV.
Noticed this while testing out CPU-Z, it installs a kernel module and
does a bunch of `RDMSR` and `OUTS` instructions. Decided to walk through
the rest of the instructions in the `System Instruction Reference`
section.
Turns out there's a bunch of oddities in there that we don't support.
First step is to go through all the explicitl SIGILL and SIGSEGV and
implement a test for them.
Next step will be implementing the remaining operations that are
considered "System" operations but are still available in CPL-3.
This list includes:
- lar
- lgdt
- lsl
- sidt
- sldt
- stac
- clac
- verr
- verw
Arm64ec introduced the InterruptFaultPage which is lower overhead since
instead of ldr+str it just turns in to a single str. We were already
allocating the space, FEXCore and the frontend signal delegator just
needed to be updated to understand the new location.
We can additionally use this in the future if we want to make deferred
async signals INSIDE the JIT only cost a single str as well.
RA should not depend on whether we support AVX, that's a huge layering
violation! and fortunately, it does not.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Aside from its own self-test, the parser is unused and should remain that way,
since it's a maintenance burden with no real benefit. Burn it.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
SRA is fundamentally about hardware registers, not stores into a
software-defined context. So, it should take a register instead of an offset.
This makes all the unaligned special cases unrepresentable (by design).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
While the ARM64EC ABI mostly matches FEX's SRA, the stack still needs to
be switched to the emulator stack and target RIP stored into the FEX
context before jumping to the dispatcher loop.
These callbacks are used for code invalidation and setting the right
emulated CPU features, neither of which are necessary for syscalls made
from within FEX. Avoid calling them to prevent deadlocks caused by
nested locks during compilation.
This is required by recent wine changes to use longjmp for user
callbacks. Switch to saving the context at every simulate call and
setting the unwind SP/PC to that context with a small SEH trampoline
for the syscall handler.
A bit of refactoring necessary before we can move the remaining Linux
specific code to the frontend.
Most of this taken from #3535 but attempting to be NFC as much as
possible.
From `man 2 open`:
> The mode argument must be supplied if O_CREAT or O_TMPFILE is
> specified in flags; if it is not supplied, some arbitrary bytes
> from the stack will be applied as the file mode.
The implementation of this has been brittle and is architecturally
incompatible with 32-bit guests. It's unlikely this could be fixed with
incremental improvements.
Since libGL and libvulkan can be forwarded independently of libX11 now,
these libX11 bits can be dropped without negative impact on compatibility.
Some applications create multiple Vulkan instances with different sets of
extensions. We might hence miss some of these pointers during the initial
function pointer query.
X11 displays and xcb connections managed by the guest libX11 can't be used by
the host, but we can create intermediary objects using the host libX11. This
allows to connect guest-managed objects to the host window system integration
APIs in OpenGL/Vulkan.
Moves it to the hypervisor leafs.
Before:
```bash
$ FEXBash 'cat /proc/cpuinfo | grep "model name"'
model name : FEX-2404-101-gf9effcb Cortex-A78C
model name : FEX-2404-101-gf9effcb Cortex-A78C
model name : FEX-2404-101-gf9effcb Cortex-A78C
model name : FEX-2404-101-gf9effcb Cortex-A78C
model name : FEX-2404-101-gf9effcb Cortex-X1C
model name : FEX-2404-101-gf9effcb Cortex-X1C
model name : FEX-2404-101-gf9effcb Cortex-X1C
model name : FEX-2404-101-gf9effcb Cortex-X1C
```
After:
```bash
$ FEXBash 'cat /proc/cpuinfo | grep "model name"'
model name : Cortex-A78C
model name : Cortex-A78C
model name : Cortex-A78C
model name : Cortex-A78C
model name : Cortex-X1C
model name : Cortex-X1C
model name : Cortex-X1C
model name : Cortex-X1C
```
Now the FEX string is in the hypervisor functions as a leaf, so if some
utility wants the FEX version they can query that directly
Ex:
```bash
$ ./Bin/FEXInterpreter get_cpuid_fex
Maximum 4000_0001h sub-leaf: 2
We are running under FEX on host: 2
FEX version string is: 'FEX-2404-113-g820494d'
```
We were previously genrating nonsense code if the destination != source:
faddp v2.4s, v4.4s, v4.4s
faddp s2, v4.2s
The result of the first faddp is ignored, so the second merely calculates the
sum of the first 2 sources (not all 4 as needed).
The correct fix is to feed the first add into the second, regardless of the
final destination:
faddp v2.4s, v4.4s, v4.4s
faddp s2, v2.2s
Hit in an ASM test with new RA.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
A feature of FEX's JIT is that when an unaligned atomic load/store
operation occurs, the instructions will be backpatched in to a barrier
plus a non-atomic memory instruction. This is the half-barrier technique
that still ensures correct visibility of loadstores in an unaligned
context.
The problem with this approach is that the dmb instructions are HEAVY,
because they effectively stop the world until all memory operations in
flight are visible. But it is a necessary evil since unaligned atomics
aren't a thing on ARM processors. FEAT_LSE only gives you unaligned
atomics inside of a 16-byte granularity, which doesn't match x86
behaviour of cacheline size (effectively always 64B).
This adds a new TSO option to disable the half-barrier on unaligned
atomic and instead only convert it to a regular loadstore instruction,
ommiting the half-barrier. This gives more insight in to how well a
CPU's LRCPC implementation is by not stalling on DMB instructions when
possible.
Originally implemented as a test to see if this makes Sonic Adventure 2
run full speed with TSO enabled (but all available TSO options disabled)
on NVIDIA Orin. Unfortunately this basically makes the code no longer
stall on dmb instructions and instead just showing how bad the LRCPC
implementation is, since the stalls show up on `ldapur` instructions
instead.
Tested Sonic Adventure 2 on X13s and it ran at 60FPS there without the
hack anyway.
The library's soname is changed to libX11.so.6 and the CMake target is
renamed to libPlaceholderX11. This fixes two issues:
* Steam and mangohud can't find libX11 during startup if the library doesn't
include a version suffix.
* Calling the CMake target libX11 overrode the true host X11 library used by
unrelated targets (such as FEXConfig), which could cause link errors
Instead of only enabling enhanced rep movs if software TSO is disabled,
Enable it if software tso is disabled OR memcpysettso is disabled. This
is because now we hit the fast path when memcpysettso is disabled alone
but global TSO is disabled.
Retested Hades and performance was fine in this configuration.
Unmapping a section will unmap the whole size initially allocated,
irrespective of how their protections are changed afterwards. Make sure
to follow this logic for invalidation too.
When thread management was moved to the frontend, invalidation moved
from being a global operation to per-thread but the WOW64 backend wasn't
updated to account for this. Now for any invalidation event loop over
all threads tracked by the frontend and invalidate the appropriate
range.
Found out that Far Cry uses this instruction and it is viable to use in
CPL-3. This only returns constant data but its behaviour is a little
quirky.
This instruction has a weird behaviour that the 32-bit operation does an
insert in to the 64-bit destination, which might be an Intel versus AMD
behaviour. I don't have an Intel machine available to test if that
theory is true although. This assumption would match similar behaviour
where segment registers are inserted instead of zext.
Gets the game farther but then it crashes in a `___ascii_strnicmp`
function where the arguments end up being `___ascii_strnicmp(nullptr, "Color", 5);`.
Functional revert of 92f31648b ("RCLSE: optimize out pointless stores"), which
reportedly regressed some titles due to RA doom. We'll revisit later, leaving in
the code for when RA is ready to light this up.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Adds paths to ignore to clang-format, .clang-format file.
Also a wrapper script to clang-format to read .clang-format-ignore.
To format the whole tree at the root of the repository run:
`find . -iname '*.h' -o -iname '*.cpp' -exec python3 Scripts/clang-format.py -i \{\} \;`
Add reformat target to reformat the whole tree
This app bypasses the glX functions exported by libGL and instead sends GLX
requests directly via xcb. FEX won't support forwarding this usage pattern in
the foreseeable future.
Adapted from LLVM version of pr-code-format.yml.
Copies a few scripts from LLVM to External/.
Runs self-hosted on X64.
Assumes clang-format 16.0.6 for formatting.
These were broken to varying degrees across the board on win32,
just switch to their std::filesystem as windows doesn't have the
same allocator issues that require their reimplementation.
There's quite a few places where the segment offset appending is open-coded
throughout the opcode dispatcher, but we can pull these out into a few
helpers to make the sites a little more compact and declarative.
Now that we have successfully eliminated crossblock liveness from the IR we
generate, validate as much to ensure it doesn't come back. We will take
advantage of this new invariant in RA in the future.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
this ensures we put the StoreNZCV in the right block, which will fix validation
fails later in the series.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Rather than checking the actual EC bitmap in the dispatcher (~6 instrs), this
indirection through the code cache allows just 1 instr for the hot path
of calling repeated EC code/x64 code.
The frontend will provide the return logic via ExitFunctionEC, which
will be jumped to whenever there is an indirect branch/return to an addr
such that RtlIsEcCode(addr) returns true.
Executable mapped memory is treated as x86 code by default when
running under EC, VirtualAlloc2 needs to be used together with a
special flag to map JIT arm64 code.
Generates flags for a variable shift as a dedicated IR op. This lets us optimize
around it (without generating control flow, relying on deferred flag infra,
etc). And it neatly solves our RA problem for shifts.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is something the new shift flag code will do. Backporting the opt since
that's stalled and this reduces the diff.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This environment variable had an incorrect priority on the configuration
system. The expectation was higher priority than most other layers.
Now the only layer that has higher priority is the environment
variables.
1. pull flag calculation out of the loop body for perf
2. fully rotate the inner loop to save an instruction per iteration
3. hoist the rcx=0 jump to avoid computing df when rcx=0
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
single unified implementation for ROL & ROR (instead of 4 cases). no more
deferred flags because it's easy to shoot ourselves in the foot with deferred
flags w.r.t the new RA design, and rotates are rare enough with very efficient
flag calculations such that the extra JIT overhead should be minimal to DCE the
resulting calculations later.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Just like #3508, clang-18 complains about VLA usage.
This vector is relatively small, only around 18 elements but is
semi-dynamic depending on arch and if FEXCore is targeting Linux or
Win32.
It has been a long time coming that FEX no longer needed to leak IR
implementation details to the frontend, this was legacy due to IR CI and
various other problems.
Now that the last bits of IR leaking has been removed, move everything
that we can internally to the implementation.
We still have a couple of minor details in the exposed IR.h to the
frontend, but these are limited to a few enums and some thunking struct
information rather than all the implementation details.
No functional change with this, just moving headers around.
FEXCore includes was including an FHU header which would result in
compilation failure for external projects trying to link to libFEXCore.
Moves it over to fix this, it was the only FHU usage in FEXCore/include
NFC
This is no longer necessary to be part of the public API. Moves the
header internally.
Needed to pass through `IsAddressInCodeBuffer` from CPUBackend through
the Context object, but otherwise no functional change.
In the old case:
* if we take the branch, 1 instruction
* if we don't take the branch, 3 instruction
* branch predictor fun
* 3 instructions of icache pressure
In the new case:
* unconditionally 2 instructions
* no branch predictor dependence
* 2 instructions of icache pressure
This should not be non-neglibly worse, and it simplifies things for RA.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
The Vulkan specification states that querying "global commands" like
vkCreateInstance with a non-NULL instance is undefined behavior. Indeed, some
implementations will return null pointers in such cases.
Instead, we can drop the query from DoSetupWithInstance altogether, since
the library initializer will load the function pointer using dlsym instead.
Fixes#3519.
exhaustively checked against the Intel pseudocode since this is tricky:
def intel(AL, CF, AF):
old_AL = AL
old_CF = CF
CF = False
if (AL & 0x0F) > 9 or AF:
Borrow = AL < 6
AL = (AL - 6) & 0xff
CF = old_CF or Borrow
AF = True
else:
AF = False
if (old_AL > 0x99) or old_CF:
AL = (AL - 0x60) & 0xff
CF = True
return (AL & 0xff, CF, AF)
def fex(AL, CF, AF):
AF = AF | ((AL & 0xf) > 9)
CF = CF | (AL > 0x99)
NewCF = CF | (AF if (AL < 6) else CF)
AL = (AL - 6) if AF else AL
AL = (AL - 0x60) if CF else AL
return (AL & 0xff, NewCF, AF)
for AL in range(256):
for CF in [False, True]:
for AF in [False, True]:
ref = intel(AL, CF, AF)
test = fex(AL, CF, AF)
print(AL, "CF" if CF else "", "AF" if AF else "", ref, test)
assert(ref == test)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Based on https://www.righto.com/2023/01/
New implementation is branchless, which is theoretically easier to RA. It's also
massively simpler which is good for a demon opcode.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Same situation as the last stack leak memory fix, this is fairly tricky
since it is dealing with stack pivoting. Fixes the memory leak around
pthread stack allocations, making memory usage lower for applications
that constantly spin-up and destroy threads (Like Steam).
We need to let glibc allocate a minimum sized stack (128KB and we can't
control it) to work around a race condition with DTV/TLS regions. This
means we need to do a stack pivot once the thread starts executing.
We also need to be careful because the `PThread` object is deleted
inside of the execution thread, which was resulting in a use-after-free
bug.
There are definitely some more memory leaks that I'm still fighting, and I have
noticed in my abusive thread creation program that we might want to
change some jemalloc options to more aggressively cut down on residency.
This is just one out of many.
I remember seeing some application last year where they closed a FEX
owned FD but now I don't remember what it was. This can really mess us
up so add some debug tracking so we can try and find it again.
Might be something specifically around flatpack, appimage, or chrome's
sandbox. I have some ideas about how to work around these problems if
they crop up but need to find the problem applications again.
Since we do an immediate overwrite of the file we are copying, we can
instead do a rename. Failure on rename is fine, will either mean the
telemetry file didn't exist initially, or some other permission error so
the telemetry will get lost regardless.
This may be useful for tracking TSO faulting when it manages to fetch
stale data. While most TSO crashes are due to nullptr dereferences, this
can still check for the corruption case.
In 64-bit mode, the LOOP instruction's RCX register usage is 64-bit or
32-bit.
In 32-bit mode, the LOOP instruction's RCX register usage is 32-bit or
16-bit.
FEX wasn't handling the 16-bit case at all which was causing the LOOP
instruction to effectively always operate at 32-bit size. Now this is
correctly supported, and it also stops treating the operation as 64-bit.
This was a funny joke that this was here, but it is fundamentally
incompatible with what we're doing. All those users are running proot
anyway because of how broken running under termux directly is.
Just remove this from here.
Take e.g a forward rep movsb copy from addr 0 to 1, the expected
behaviour since this is a bytewise copy is:
before: aaabbbb...
after: aaaaaaa...
but by copying in 32-byte chunks we end up with:
after: aaaabbbb...
due to the self overwrites not occuring within a single 32 bit copy.
When TSO is disabled, vector LDP/STP can be used for a two
instruction 32 byte memory copy which is significantly faster than the
current byte-by-byte copy. Performing two such copies directly after
oneanother also marginally increases copy speed for all sizes >=64.
I was looking at some other JIT overheads and this cropped up as some
overhead. Instead of materializing a constant using mov+movk+movk+movk,
load it from the named vector constant array.
In a micro-benchmark this improved performance by 34%.
In bytemark this improved on subbench by 0.82%
Missed this instruction when implementing rdtscp. Returns the same ID
result in a register just like rdtscp, but without the cycle counter
results. Doesn't touch any flags just like rdtscp.
This unit test recreates the error condition that #3478 causes.
With a string operation that is a backwards copy then the optimization
will read past the end of the page and result in a crash.
Seemingly only happens with backwards string operations, but test
forward and backward in this test.
x86 has a few prefetch instructions.
- prefetch - One of two classic 3DNow! instructions
- Prefetch in to L1 data cache
- prefetchw - One of two classic 3DNow! instructions
- Implies prefetch in to L1 data cache
- Prefetch cacheline with intent to write and exclusive ownership
- prefetchnta
- Prefetch non-temporal data in respect to /all/ cache levels
- Assumes inclusive caches?
- prefetch{t0,t1,t2}
- Prefetch data with respect to each cache level
- T0 = L1 and higher
- T1 = L2 and higher
- T2 = L3 and higher
**Some silly duplicates**
- prefetchwt1
- Duplicate of prefetchw but explicitly L1 data cache
- prefetch_exclusive
- Duplicate of prefetch
God Of War 2018 uses prefetchw as a hint for exclusive ownership of the
cacheline in some very aggressive spin-loops. Let's implement the
operations to help it along.
This function can be unit-tested more easily, and the stack special is more
cleanly handled as a post-collection step.
There is a minor functional change: The stack special case didn't trigger
previously if the range end was within the stack mapping. This is now fixed.
can help a lot of x86 code because x86 is 2-address and a64 is 3-address, so x86
ends up with piles of movs that end up dead after translation
It's not a win across the board because our RA isn't aware of tied registers so
sometimes we regress moves. But it's a win on average, and the RA bits can be
improved with time.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Patch written by Sonicadvance1. Unclear how this wasn't already broken, but we
need this to keep CI happy with the rest of this series.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Arguments and conditional doesn't get optimized out in release builds
for the inline function call versus the define.
Was showing up an annoying amount of time when testing.
The NVIF ioctl isn't publicly described in the nouveau headers and it is
required for anything to work with Nouveau.
Pass the ioctl command through without modification and hope that this
ioctl is architecture agnostic.
Folds reg+const memory address into addressing mode,
if the constant is within 16Kb.
Update instcountci files.
Add test 32Bit_ASM/FEX_bugs/SubAddrBug.asm
Fixes#3479
As of the Steam update on Feb 29 2024, this is no longer necessary and
actually causes steamwebhelper to abort.
Steam has started using SLR to launch steamwebhelper, which already
passes in --no-sandbox. Adding this argument twice seemingly breaks the
application since it starts complaining that it doesn't understand the
argument and closes with a SIGABRT.
With this removed Steam starts working again.
Fixes an issue where TestHarnessRunner was managing to reserve the space
below stack again, resulting in stack growth breaking. Would typically
only show up when using the vixl simulator under gdb for some reason.
This is likely the last bandage on this code before it gets completely
rewritten to be more readable.
ldswpal doesn't overwrite the source register and only reads the bits
required for the sized operation.
Not sure exactly why we were doing a copy here.
Removing it means improving Skyrim's hottest code block, as seen in #3472
This reduces pointless constant usage. For now, it's no net change to
instcountci, but it should make it easier to get wins later. Hopefully.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Doesn't quite match the libc code directly because it uses `[gs:eax]`
with both having the sign bit set and we can't deal with that with ASM
tests. So match the behaviour in a different way.
This is what we'll actually ship (I hope), so that's the config we want to
track long-term. It's also a lot more managable resulting asm.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Reimagining of #3355 without any json generators or new concepts.
Fixes some mislabeling of system calls. Some getting inlined when they
shouldn't be, a lot not getting inlined when they can be.
This really cleans up the syscall implementation, all syscalls that can
be passthrough implementations require a very small two line
declaration.
Additionally cleans up a bit of implementation cruft where some
passthrough syscalls were using the glibc syscall handler, and some were
using the glibc implementation. We have had multiple issues in the past
where the glibc implementation does something subtly different than the
raw syscall and breaks things. Now all passthrough handlers do a system
call directly, removing at least one indirection and some ambiguity.
This makes it significantly easier to add new passthrough syscalls as
well. Only need to do a version check and add the three lines per
syscall. Which there are new syscalls incoming that we will want to add.
Tangible improvements:
- Syscalls are lower overhead than ever.
- When I'm adding more syscalls I have less chance of mucking it up.
Doing very little work with a bunch of instructions.
Hottest block in the Windows version of Psychonauts, it's just doing a
matrix swizzle but in the worst possible way.
this gets rid of the awkward non-flag SBB case, which streamlines SBB. while
getting better codegen for the demon opcode (-:
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
When the source arguments for LoadMem/StoreMem have bit 31 set then they
are incorrectly sign extending in some instances.
Detected this when testing #3421 but I don't have a proper fix for it.
The usual tricks, also requires introducing a bare adc op to optimize adcs to,
but we wanted that anyway!
Also support a zero source, so we can calculate "foo + CF" in one instruction to
optimize the "lock adc" cases.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
It's the same, stop copypasting. This gets our flag and arithmetic opts (current
and future) applied to secondary ALU too.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
we don't need the result, we can use subs and come out ahead in practice. also a
step towards better fusion
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is required to ensure single-iteration convergence with a sequence like:
write C
whatever = load C
rmif C, whatever
invalidate C
avoids regressing the "DEC dead" case with future work in the series.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
FEXCore doesn't need track the TLS state of the SignalDelegator, this is
a frontend concept.
Removes the tracking from the backend and keeps it in the frontend.
Previously: Would keep one clone thread's stack active for teardown
delaying.
With aggressive cloning and teardown, this was unsafe.
Only reap the stack when told it is safe to do so.
Spurred on by #3421. To ensure that applications don't take advantage of
small address wrap around, allocate the second 4GB of virtual memory.
Some context. Linux always reserves the first 16KB of virtual address
space (unless you tinker with some settings which nobody should do).
Example of 32-bit code:
lea eax, [0xffff_0000]
mov ebx, [eax + 0x1_0000]
The address calculated by the mov will wrap around to 0x0 which will
result in SIGSEGV. If FEX messes up zero extensions then it would try to
access 0x1_0000_0000 instead.
This could result in a 32-bit application potentially accessing some FEX
memory instead of crashing.
Add this safety net which will still SIGSEGV and we will be able to see
the crash.
Spurred on by #3421
This does a bunch of GPR and vector loads to showcase addressing
limitations between ARM and x86.
Tests:
- 8/16/32/64-bit GPR loads
- Both 32-bit and 64-bit addressing modes
- 32/64/128-bit Vector loads
- Both 32-bit and 64-bit addressing modes
- Duplicate the tests for 32-bit addressing mode with a 32-bit process
- Since it should change behaviour.
Untested:
- 8/16-bit vector loadstores since those don't exist on x86
- 16-bit x87 integer load exists but that doesn't go through a vector
load in FEX.
- Res is unused
- SrcSize doesn't matter since we ignore the high bits, might as well always use
32-bit, it doesn't matter
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
ARM64EC has a shared SRA mapping between ARM64 and X64 code, so there
needs to be a public way to enter the dispatcher without refilling SRA
from the in-memory context struct.
The pointer tracked internally by Wayland can be queried via
wl_proxy_get_listener, so we don't need our own bookkeeping.
This also changes the callback table's element type to a fixed-size uint64_t,
which makes it work for 32-bit guests.
The creation mutex could have been held if the parent thread was in the
middle of creating a thread when forking. This would result in a
deadlock once the fork child attempted to create another thread.
Forcefully dropping the lock in the fork child works around this
deadlock. This comes at the expense of potentially leaving resources
guarded by the thread creation mutex in an invalid state. Crashes caused
by this are easier to reason about than a delayed deadlock, though.
Moves the CTX LockBeforeFork in to the Syscallhandler's LockBeforeFork.
This lets the syscall handler just call its own LockBeforeFork and
UnlockAfterFork functions rather than two on each call site.
Also moves the CTX->UnlockAfterFork in to the SyscallHandler's to be
consistent with the LockBeforeFork half.
No functional change.
When code invalidation is happening we currently have the issue that a
thread can acquire the code invalidation mutex in the middle of
invalidation. This is due to us acquiring and releasing the mutex
between each thread's code invalidation.
We need to hold the mutex for the entire duration for all thread's code
invalidation.
This fixes a rare hang on proton startup and resolves a consistent hang
on Proton application shutdown.
This now puts us on par with FEX-2312.1 with hanging.
This does not fix a relatively rare hang on fork (which also existed with FEX-2312.1).
This also does not fix the issue that the intersection of our mutexes
between frontend and backend are very convoluted. In part of the work
that is going to fix the rare fork mutex hang will change more of this.
Now that lots of instructions have optimal flag calculation in isolation, we
need to look at sequences of instructions together. These cases test various
interesting cases where concatenating the optimal instruction translations gives
something terrible for the whole block. These cases exercise the
new flag optimization pass.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
RCLSE ignores NZCV and doesn't optimize stores which doesn't help us with PF/AF
either. So, we add a new pass for dead flag elimination (cannibalizing the old
and broken dead flag elimination pass). This is a simple local optimizer that
walks each block backwards, converging in linear time & constant space in a
single iteration.
Right now, it doesn't do a ton (other than a nice reduction in silliness in
the hot Sonic block), but it provides the framework to fuse comparisons.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
these are useful x86 instructions for e.g. branch-if-not-zero, and have
opportunities for clever translations. Add the instcountci cases to make sure we
do something reasonable for them.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is required for host-side calls to guest functions on 32-bit guests.
Since the host stack is allocated before FEX blocks memory inaccessible to
the guest, the guest would otherwise fail to read the packed argument data.
ARM64, x86 (64-bit), and x86 (32-bit) each have different alignment
requirements, so this change ensures that consistent data layout is
used for packing and unpacking.
If the thread object is added to the tracking vector immediately then
there ends up being a race condition before the thread manages to fill
out the thread-specific data that only occurs at the start of the new
thread.
This manifests in a crash when a thread is allocating memory while
another thread is getting constructed. Easy fix is to defer the tracking
until the thread has setup its state.
With a contended unique lock, we forgot to reset the `Expected` value to
zero. This was causing a contended mutex to incorrectly succeed.
Noticed this when converting some pthread mutexes over to spinloops to
remove strace noise.
The reference wfe_mutex library I wrote didn't have this problem since
the implementation is slightly different.
We only need each part of W extracted in the corresponding round, so sink the
extract into the round to reduce pressure.
Further, W and E are added and then never used again. So, by reassociating we
can do the add upfront, killing W and E at the start and further reducing
pressure.
Eliminates spilling in sha1rnds4.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This is used for instcountci to ensure instruction counts don't change
when a compiler supports this feature or not. Always runtime disable
when running in instcountci.
CMake option from #3394 can still be useful so leaving that in place.
When 4d109c9ce0 fixed parsing strenum
types in the json, it also added `ArgumentHandler` types to the json
parsing. This was incorrect as those types are already stored in the
json in their decoded numerical format.
Without this change, all config options with `ArgumentHandler` will
decode as "0" which is incorrect. The main killer here is that SMCChecks
gets disabled (visible in both FEXConfig and when applications are
running) which was causing spurious failures.
Forces disabling use of __attribute__((preserve_all)).
Until CI uses clang17, where this attribute was added, instcountci fails
when FEX is compiled with clang>=17.
as far as flags go, they're identical: set ZF for zero output, set CF for output
= DestSize, undef the rest. merge the impls, so we get the optimized lzcnt impl
for tzcnt.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
this skips the constant folding, which saves the branching in the rotate
immediate implementations.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
JIT code and loaded code memory offsets aren't consistent run to run
which causes code generation consistency issues.
Instead use a register plus an immediate that can't fit in a single
16-bit encoding to get similar register allocation to RIP relative
addresses.
At some point it will probably be nice to support consistent placement
of both guest and host code so RIP relative can be used, but we don't have
support in our backend to do that.
Generally, implicit integer conversions are prohibited for data wrapped in
guest_layout/host_layout, but a few types are exceptional:
* char vs signed char vs unsigned char vs other 8-bit ints
* wchar_t vs other 32-bit ints
* size_t vs uint32_t (32-bit only)
* long long vs other 64-bit ints
* long vs long long (64-bit only)
These combinations have the same data size, so conversions between them are
explicitly allowed now.
Some types (notably size_t on 32-bit) have different sizes on the guest than on
the host. This template function must be aware of these differences, so a
second parameter list with fixed-size types must be provided to describe the
guest types.
Note that this information can't be queried through type traits: To a C++
compiler, size_t is indistuingishable from uint64_t. For this reason, the
correct guest type must indeed be provided externally.
This block of code causes our RA to panic spill and it gets hit my
steamwebhelper.
We should be able to spill without panic on this code at some point.
- use better algorithm that is O(# set bits) instead of O(# total bits)
- eliminate spilling by careful management of our temporaries
- fix nzcv clobber bug (whoops)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
We are required in our syscall emulation to handle cases where pointers
are invalid. This means we need to pessimistically assume a memcpy will
fault when reading application memory.
This implements a signal handler based approach to catching the SIGSEGV
on memcpy and returning an EFAULT if it faults.
The ARM64EC SRA layout will use x0-3 for x86_64 registers, as such any
arguments passed to C ABI functions need to proxy their arguments
through the temporaries and move as appropriate.
The -deps target is the wrong target to add this to, since its compile flags
are propagated to both Guest.cpp and Host.cpp. Instead, define the flag only
when processing files within a guest context.
I added the `pkg-config` that is missing in the Dockerfile, and make the git clone the repository direct from the image, so the user don't need to clone it manually and after that pulls inside the Dockerfile!
We had a chance of doing an additional bogus wfe if the expected value
was hit in one iteration of a loop. Not the biggest problem on current
hardware where WFE only ever sleeps for 1-4 system cycles, but on future
hardware where WFE might actually sleep for longer then this could have
been an issue.
This can be used to allow automatically handling structures that require
special behavior for one member but are automatically repackable otherwise.
The feature is enabled using the new custom_repack annotation and requires
additional repacking functions to be defined in the host file for each
customized member.
Noticed this while writing #3342.
Fixes#3343
The syscall instruction is defined in the documentation that it will set
RCX to the next instruction's RIP and R11 to be RFLAGS. We entirely
skipped this which I noticed while writing unit tests.
Adds unittests to test both 32-bit and 64-bit behaviour because our
helper shares code with both.
I don't know if anything actually relied on this behaviour but we should
definitely support it.
This behaves exactly like pidof but only searches for FEX applications.
This fixes a long standing annoyance of mine that pidof doesn't work for
FEX. This behaves exactly like pidof but knows how to decode the command
line options to pull out the program data.
If the Linux kernel ever accepts the patches for binfmt_misc to change
how the interpreter is handled then this will become redundant, but
until that happens here is a utility that I want.
cmpxchg handles rax specially, so cmpxchg with dest=rax is a special case. test
also the general case.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
To avoid performance traps, several conditions must hold for exit repacking
to apply:
* Argument must be a pointer
* The pointee type must have differing data layout between guest and host
* The pointee type must be non-const
Arguments that don't meet the first two conditions are safe *not* to repack
on exit, since they're either passed by copy or have consistent data layout.
The third condition is a heuristic: In principle, an API function could modify
data through nested pointers even if the argument pointer is const. However,
automatic repacking is not supported for such types anyway, so this is a safe
heuristic to use.
Primary goal for this is to ensure that the delinker doesn't need to
allocate any memory. This delinker can end up getting hit heavily with
JIT code so we don't want it to be allocating memory.
Currently all uses of the forward label calls in to jemalloc to allocate
memory. This allows a forward label that doesn't require any memory
allocation, which is the common case in FEX.
The delinker step of the JIT was using std::function with capture
lambdas that required memory allocation when unnecessary.
Because the compiler can't see through our std::function usage it could
never decompose these by itself.
By passing the Thread's frame and record to the function as arguments
then we can have the signature be a raw function pointer.
This fixes an area of concern from:
https://github.com/FEX-Emu/FEX/blob/main/docs/ProgrammingConcerns.md#stdfunction-and-lambdas
If the Dst register is allocated as VectorIndices or VectorTable,
using Dst as an operand to perform the tbx operation will result in an error.
For example:
%131(FPR0) i128 = LoadNamedVectorIndexedConstant u8:Tmp:RegisterSize, #0x6, #0xaa0
%132(FPR0) i128 = VTBX1 u8:Tmp:RegisterSize, %129(FPRFixed6) i32v4, %126(FPRFixed10) i16v8, %131(FPR0) i128
Since the tbx instruction's destination register is also the original operand,
this is consistent with the semantics of VTBX1. Therefore,
directly using VectorSrcDst as the destination operand for the tbx instruction is safe.
While locking a shared_lock and doing an empty table lookup is fairly
fast, just remove them from the hot path entirely if no custom IR
handlers are installed.
This is only used for our IRLoader, which is losing its importance
significantly and should probably be removed anyway.
Pointer types inherently cause data layout compatibility issues, so they're
worth special-casing here. The wrappers will type-pun pointers to 32-bit or
64-bit integers (matching the guest architecture) to avoid direct host-side
use of guest pointers without consideration.
The guest_layout wrapper provides an architecture-agnostic representation of
the guest data layout of each struct used in a thunked library. A constructor
is added to host_layout to allow conversion of the data to the host layout.
For types that are already fully compatible, both layout wrappers are simple
type aliases to minimize overhead.
This unit test hasn't really served any purpose for a while now and
mostly just causes pain when reworking things in the IR.
Just remove the IRLoader, its unit tests, the github action steps and
the public FEXCore interface to it. Since it isn't used by anything
other than Thunks.
Also moves some IR definitions from the public API to the backend.
Need #3348 merged first.
As I was casually thinking, this code made me realize that it was quite
branch heavy and could likely be optimized to logic.
The previous code generated some fairly nasty branch heavy code. This
can be optimized to be branchless and take roughly five instructions
per flag. Using a bitfield for each feature would turn each calculation
in to 3-4 instructions but that seems overkill.
Very minor thing.
This allows future changes to emit interdependent helper structures in the
same order.
The sort algorithm was chosen for simplicity rather than performance. It's
fast enough in practice even for APIs as large as Vulkan.
We only used this so that our Xavier CI system which were running old
kernels could run unit tests. We have now removed the Xaviers from CI
and this is no longer necessary.
Stop pretending that we support kernels older than 5.0 and allowing this
fallback.
The 32-bit allocator is still used for the MAP_32BIT mmap flag, so the
load bearing code can't be fully removed. Just remove the config and the
frontend things using it.
Currently no functional change but public API breaks should come early.
The thread state object will be used for looking up thread specific
codebuffers in the future when we support MDWE with code mirrors.
We can safely call virtual functions through the JIT with a little bit
of work.
FEX's JIT has quite a few steps before it gets to a syscall handler.
Before this commit:
JIT->static HandleSyscall->SyscallHandler::HandleSyscall->SyscallHandler
After this commit:
JIT->SyscallHandler::HandleSyscall->SyscallHandler
A bit hard to notice this when this interface can spin at 67-million
calls per second though.
This has the Frontend and OpcodeDispatcher select their operating mode
depending on the incoming code segment long-mode flag.
Adds some asserts since currently it is unexpected if the configuration
changes at runtime.
This is fairly straightforward for an initial setup but isn't fully
fleshed out.
Right now FEX's x86 tables aren't setup in a way to support choosing a
different instruction decoding depending on runtime operating mode
change, so that would break in interesting ways.
Primarily this just gets FEX setup to start piping the operating mode
through from the frontend to the backend. This is a long term task, so
it is going to take a long time to iron out all the issues.
Previously we were only storing the 32-bit base address which isn't
actually how segment descriptors work.
In reality segment descriptors are 64-bit descriptors that are laid out
in a particular layout depending on the 4-bit type value. In reality we
only care about code and data segment layouts since the rest are
bonkers.
Describe these descriptors correctly and setup a default code descriptor
for the operating mode that FEX is starting in.
This will result in FEX not being able to allocate executable memory.
We can use shared memory in the future to work around this but for now
we don't support that as a fix.
Lots going on here.
This moves OS thread object lifetime management and internal thread
state lifetime management to the frontend. This causes a bunch of thread
handling to move from the FEXCore Context to the frontend.
Looking at `FEXCore/include/FEXCore/Core/Context.h` really shows how
much of the API has moved to the frontend that FEXCore no longer needs
to manage. Primarily this makes FEXCore itself no longer need to care
about most of the management of the emulation state.
A large amount of the behaviour moved wholesale from Core.cpp to
LinuxEmulation's ThreadManager.cpp. Which this manages the lifetimes of
both the OS threads and the FEXCore thread state objects.
One feature lost was the instruction capability, but this was already
buggy and is going to be rewritten/fixed when gdbserver work continues.
Now that all of this management is moved to the frontend, the gdbserver
can start improving since it can start managing all thread state
directly.
FEX was incorrectly invalidating RBX and RBP when a new
thread/clone/fork was created. Only RAX should be set to zero to
signify that it is the child process.
I tried to find some reference as to why this was being done in FEX, and
I seem to recall some code somewhere that was setting up this state.
Sadly I can't seem to find any reference to this being required now by
the linux kernel when it creates new processes, so maybe I was just
gaslighting myself.
Fixes crash in bubblewrap where it was expecting a valid RBP to exist
after it does a clone syscall.
I have unit tests coming for this but it is uncovering some more bugs in
FEX's thread handling. Some of which are getting fixed with the current
PRs that are moving the thread handling to the frontend.
But since proton is broken now, this needs to get in.
This was left over and was using the wrong argument.
This would cause the stack for the child thread/process to be at the
start instead of the end.
Fixes a crash in bubblewrap.
Similar to #3284 but works around some of the bugs that one introduced.
This is the minimal amount of changes to move the ownership from FEXCore
to the frontend. Since the frontends don't yet have a full thread state
tracking, there is an opaque pointer that needs to be managed.
In the followup commits this will be changed to have the syscall handler
to be the thread object manager.
Now that only FEXLoader is under the FEXLoader folder, the mingw build
check can be moved up one level.
No functional change, just moving the check and re-aligning. Might be
good to view without whitespace changes.
Which requires moving LinuxEmulation to its own independent folder as
well. Since both IRLoader and FEXLoader rely on it.
No functional change, just moves the the code around.
This was a temporary header to help with when this header was migrated
to our public API headers.
It's temporary nature is no longer necessary, just get rid of it.
No need to wait for initialization on for this anymore.
Ever since Init was refactored to do basically no work, this hasn't been
necessary.
CPUID does need to still be initialized after HostFeatures though, so
need to ensure correct member ordering there.
When the address calculation for SIB has both index and base then we can
optimize this to an add with a shifted register. This will convert a
three instruction sequence in to one instruction in most cases.
When originally written I was testing symbol loading on an x86 host
assuming that the vdso symbol names were the same between x86 and
aarch64.
Turns out this is not true and aarch64 uses `__kernel_` prefix instead
of x86's `__vdso_`.
AArch64 VDSO only provides us with four symbols right now
- __kernel_rt_sigreturn
- __kernel_gettimeofday
- __kernel_clock_gettime
- __kernel_clock_getres
This means that currently AArch64 is missing getcpu and time.
time is unlikely to ever be implemented so the vdso scanning for that is
unlikely to ever get a pointer and will need to go down the glibc path.
getcpu is likely to get an implementation at some point. There's even an
old patch series that implemented it[1]. Currently the patch has been
dropped on the floor for whatever reason but hopefully they will come
back to it.
Benchmarks on Cortex-X1C:
- Native AArch64 clock_gettime bench
- VDSO: 33.3ns per call
- glibc (which uses vdso): 33.3ns per call
- Roughly equivalent
- Emulated x86_64 clock_gettime bench
- Prior to fix:
- 49ns per call for both VDSO and glibc
- After fix:
- VDSO: ~46ns per call
- glibc: ~49ns per call
- Emulated i386 clock_gettime bench
- Only tested after fix
- VDSO: 50ns per call
- glibc: 65ns per call
The only way to improve this further would be to optimize thunks to push
certain function signature arguments in registers instead of stack.
Which is a future optimization as time goes on.
[1] https://patchwork.kernel.org/project/linux-kselftest/list/?series=335203
While we were calling this function, its asserting nature hasn't been
used for a long time.
This used to trigger more frequently when CompileBlock would fail to
compile code, either due to not being able to decode an instruction or
hitting an instruction that FEX doesn't understand.
When these cases are hit today we still generate code blocks which
generate SIGILL. This means that this code was actually never hit.
Completely remove this function and have the JIT's dispatcher call the
CompileBlock function directly. Signature is slightly different since we
need to set x3 to be 0.
Reduces the ELF's VM size from 9.8MB down to 9.37MB and should reduce
initialization time a smidge.
Slammed this out while waiting for other PRs to get reviewed.
Fairly lightweight since it is almost 1:1 transplanting the code from
FEXCore in to the SyscallHandler's thread creation code.
Minor changes:
- ExecutionThreadHandler gets freed before executing the thread
- Saves 16-bytes of memory per thread
- Start all threads paused by default
- Since I moved the code to the frontend, I noticed we needed to do
some post thread-creation setup.
- Without the pause we were racing code execution with TLS setup and
a few other things.
clone needs to allocate memory before the fork locks are held, otherwise
they will hang forever waiting on a locked mutex.
This wasn't previously seen since nothing was using fork with clone3.
GdbServer object was getting deleted before the socket thread was
shutdown which was causing a crash on detach.
Now on destructor it will wait for the thread to thread to exit.
A dictionary lookup was only ever returning the .ero result since we
were adding multiple values to the same key. Adds them as a list now and
walks them.
Additionally if we forget to add a new version again, have a fallback to
try checking using our file format layout.
Fixes an issue where a user running 23.10 was faulting the python
script.
This flag breaks FEX heavily for now.
glibc 2.38 started using this flag as an optimization for posix_spawn.
It will fall back to a "non-optimized" implementation if the clone
syscall returns EINVAL. For now do this while we investigate a more
proper implementation.
Should be backported to 2312.1.
In some situations TestNZ is generated with a constant that is using a
constant that can't fit inside of the tst instruction.
This was found in libGLX with virgl, crashing invalid instruction
generation and crashing steamwebhelper
These functions only want the GPRs returned for SRA. This is because the
signal handler needs this map to relation between x86 GPRs and AArch64
GPRs.
When we added AF and PF to the SRA array we accidentally started
returning two more GPRs to the frontend. This caused the signal
delegator to start corrupting the members after GPRs in FEX's CoreState.
Corrupting 16-bytes after the gregs[] array.
This included corrupting:
- es_idx, cs_idx, ss_idx, ds_idx, gs_idx, fs_idx, _pad[]
they're all copypastes of each other, unify into one general "bit test & perform
action" template. this means most of the wins from the previous commits now
apply for bt* without more copypaste.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
if the shift is < N, and we grab bit 0 after, we only need to consider <=N
bits of the source. this lets us use 32-bit lsr for 32-bit bt, which will
reduce masking in the next commit.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Split from #3284 without changing ownership semantics while I reduce the
debugging surface here.
Removes one usage of ParentThread from FEXCore. Which can be done since
it is no longer an opaque structure, we can read the StatusCode
directly.
No functional change.
When COND_AL was added it wasn't added to this helper. Since there is a
gap between the last condition, just early check the value.
Fixes reading beyond the end of the array
The XID signal handler is OS specific and needs to be delegated to being
handled in the frontend. This only happens when we create a thread with
pthread rather than clone/fork/vfork.
The first created pthread will reset the XID signal handler in glibc,
which we need to check the first time this occurs. A little bit disjoint
while the actual thread creation still happens in the backend but will
come together once the thread creation gets moved to the frontend.
The only game I am aware of that requires this to work is SOMA since its
one of its middleware libraries tries asking for root privileges.
Retested the game to make sure it still works as expected.
Since we're invoking curl directly, we don't need to wrap it in `sh -c`
with this function.
Fixes an issue where curl downloads to a non-escaped path weren't
working. Now they do.
As I was poking around erofs-utils documentation, I found out that
fsck.erofs actually provides an option for extracting erofs images
without using fuse.
This finally puts the erofs handling on feature parity with squashfs.
O_TMPFILE has a few minor problems that I have been thinking about for a
while. I just recently got reminded about this and remembered that most
problems get resolved by using memfd_create.
- O_TMPFILE is only supported on some filesystems.
- Supported filesystem must be the one mounted to the pathname being
opened.
- Only a minor inconvenience as tmpfs and all related filesystems
support this.
- An inode is actually created on whatever filesystem is backing the
folder.
- `/tmp/` must exist as a directory
- If this folder happened to not be mounted then these temporary
files wouldn't have been created.
- memfd_create doesn't have a folder that needs to exist.
- We were leaving the files open as read/write
- While we were rewinding the file offset, an misbehaving application
could have wrote garbage to the temp file.
- memfd sealing allows us to open the FD as RW and then seal its
capabilities, making it a read-only FD.
- We were leaking FDs opened with O_CLOEXEC
- We could have just opened the O_TMPFILE with O_CLOEXEC
- memfd also just supports this flag, so use it.
- No real issues, just nice to be sanitary here.
Overall this doesn't really change any behaviour, but it is nice to
cleanup some of the edges there.
These are only used by gdbserver for filling out its XML data structures
so just remove them from FEXCore.
Also fixes the ordering on RegNames to match the definition of the enum
class definition in CoreState. This has been out of correct order since
we reordered registers months ago.
As we are moving more and more OS specific code to the frontend, this is
another set of functions that can be moved to FEXLoader from FEXCore.
No functional change here, only code moved from protected to private and
to FEXLoader's SignalDelegator.
Once more thread handling is moved to the frontend we can move even more
out of FEXCore. As follows:
- CheckXIDHandler can get moved.
- First pthread FEX makes would just call this.
- Register/UnregisterTLSState
- This can happen in the clone/thread handler once the frontend
handles it.
This leaves very little in the backend and is mostly an interface for
passing signal data to the frontend that it needs once a signal has
occured.
It additionally also is used for `SignalThread`.
The frontend needs to be in control of how threads are created. This is
inherent to the fact that OS threads are OS specific. We currently have
this weird split that when initializing the FEXCore context, we create a
parent thread at all times.
This does some initial cleanup that gets the core initialization nearly
decoupled.
This has long since been unused. Originally implemented for some fuzzing
tests but has been abandoned and that should likely be implemented some
other way.
When a 32-bit imul was being executed it had a chance of returning
garbage data in the upper 32-bits of the 64-bit result.
While this didn't typically cause problems, this gets exacerbated from
32-bit applications executing multiplies for address calculations.
A combination of commits 7146691360 and
d01b457727 exposed this problem where
previously there would be multiple moves between the calculation and
data use which would have zero'd the upper bits for us previously.
Now that we are no longer doing that, we need to make sure the opcode
dispatcher doesn't generate broken code instead.
Fixes Dungeon Defenders, which hasn't worked since FEX-2308.
Adds an ASM test that ensures we don't break it again.
This is an option that has been long overdue for removal. It's original
intention was primarily to lie to the guest application about the number
of cores in the system. This allowed us to say that the system was only
single threaded which worked around some threading bugs that we had
early on.
This is no longer the case and now it is a confusing remnant of the past
that people think they need to set. Incorrectly assuming that "0" by
default means that FEX is doing some sort of disabling of threading and
forcing all emulation down one CPU core. This is not the case and has
never been the case, so removing the option makes that idea go away.
Stop lying to the application about getcpu, sched_getaffinity, and
sched_setaffinity.
- getcpu would wrap the cpu result modulo the count of cores
- sched_setaffinity wouldn't work at all
- sched_getaffinity lied and always reported full affinity of config
option
Split off from #3282 to reduce burden.
We can read the data member directly now since it isn't opaque. In fact
we already do in the signal handlers. Removes these redundant helpers.
Removes one usage of ParentThread in FEXCore.
This usually happens on backwards memcpy where we know the direction of
the copy because the code will typically do as follows:
```
std
rep movsb
cld
```
This is because the direction flag is part of the ABI and needs to be
set back to the forward direction if it was modified.
This typically doesn't get picked up on forward copies because we won't
have visibility of a cld instruction in the block.
This optimization allows us to only emit half of the code for the memcpy
if it is a compile time constant.
There's definitely some future task that could assume forward direction
if unknown and recompile the code if the assumption has failed, but not
doing that here.
While auditing our JIT to see if we have any AFP issues I noticed this.
RPRES has a hard dependency that AFP exists in order to be used, we were
hitting a case where RPRES would be enabled, but AFP is disabled by
default.
RPRES only changes behaviour when FPCR.AH is set which requires AFP.
While this doesn't affect any hardware today, it likely will in the
future.
Removes new warning from clang:
```
FEXCore/include/FEXCore/Core/CoreState.h:119:35: warning: not packing field 'DeferredSignalRefCount' as it is non-POD for the purposes of layout [-Wpacked-non-pod]
119 | NonAtomicRefCounter<uint64_t> DeferredSignalRefCount;
| ^
```
We already ensure sane data layout that this is unnecessary anyway.
Suspend may be called on a thread before it has finished WOW64 initialisation,
keep track of all initialized threads and fallback to direct
NtSuspendThread when this is the case.
The frontend shouldn't need to know any information about how to
reconstruct eflags. Just give us the information we need and it'll work
out.
There are still some inherit limitations of this and some edge cases
that might give invalid data, but it is roughly as close as it was
before.
Just provide if the PC was in the JIT, the host GPRs, and the PState object from the signal
information and FEXCore does the rest.
We don't need to change the signature for `SetFlagsFromCompactedEFLAGS`
because during reloading of register state automatically does this for
us.
Goal is to improve FEX performance with real world games. Let's start with
Sonic Mania. Two hot blocks from #2681.
I'm only bothering to track flagm for this because I'm super biased.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Many flag-generating instructions like cmp need to save calculations for
deferred PF and AF flag calculation. Currently, they require a store per flag,
which is prohibitively expensive for hot instructions like cmp. By instead
pinning PF/AF temporary results to registers (x26/x27 by convention here), we
eliminate many stores altogether and turn the rest into zero-cycle moves (on
64-bit at least, this isn't optimal for 32-bit emulation due to CTX->GetGPRSize
shenanigans, need to check if this requirement can be lifted..).
To implement, we model as SRA and then the existing SRA code is able to generate
good code with little manual tuning. (Future work will get us to excellent code
with more tuning ;) ).
The tradeoff is reducing the working dynamic GPR set by 2 registers, which might
increase spilling in some cases. I think it's worth it in practice, though.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
- sha1nexte
- Takes advantage of sha1h if supported
- Does the operation in a vector otherwise
- sha1msg2
- Instead of dumping everything to GPRs, we can do this with vectors
- Mostly matches ARM's sha1su1 instruction, but it is /just/
different enough to be annoying.
- sha256msg1
- Directly matches sha256u0
- Leaves the previous implementation alone
Not supposed to touch flags at all, so don't! instead of making a terrible mess
of csels. a lot less instructions, and probably faster because the branch should
be predicted correctly in practice in hot loops.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Instruction count CI has transformed the way we work on FEX… I love the system
and want to make it better. there’s one part of instruction count CI that isn’t
so lovable: the problematic “optimal” flag on instructions.
There are several issues with this flag, both philosophical and practical.
– it is tedious to update the optimal flag when making an implementation
optimal. The effect of that is discouraging people from making instructions,
optimal, or encouraging people to fail to update the flag, and dilute the value
of it. Either way, since we care far more about optimal implementations, then we
do about updating the flag, clearly we should prioritize the implementation and
not the flag. This issue was not obvious at the outset, when instruction count,
CI was introduced, and still quite small. The problem magnified when we started
duplicating instructions in bulk for different combinations of CPU features
(flagm, AFP, etc.) that intern multiplies the manual work required to update the
flags by the corresponding constant factor. if it comes down to a choice between
removing this extra coverage and removing the flag, I think we all agree that
removing the flag is the lesser evil.
– The definition of “optimal” is fundamentally problematic. I have often
improved the instruction count of an instruction that was already “optimal”.
This is all kinds of silly, and calls into question whether there’s any value
whatsoever in the existing classifications of the flag. Furthermore, it is often
unknowable, whether an implementation really is optimal. Is it possible to
implement BZHI (with flag calculations) in fewer than eight instructions? We
don’t know, and it’s silly to pretend that we do.
– as a consequence of the problematic definitions , there are so many errors in
both directions that I don’t think there’s much value in preserving the existing
classification at the expense of +progress. Being able to say “32% of
instructions are translated optimally” is neat, but it really doesn’t tell us
anything whatsoever when you dig a little deeper.
So, as the flag is misleading at best and perhaps harmful at worst, let’s remove
it and make the instruction count CI, more useful overall. let’s let the
expected count and the assembly speak for themselves, and cut away the chaff. if
we want a meaningless number to report to management, we can instead calculate
the average blowup factor ;-)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
If a bind fails then this is usually because PID reuse happened and the
unix domain socket path still existed on the filesystem. Alternatively
the process did an execve which left a dangling unix domain file.
Unlink the file and try again in these cases. It will almost certainly
work the second time.
Instead of relying on sleeping in accept, use poll. This allows us to
early shutdown instead of leaving a thread hanging stuck in accept.
Doesn't change behaviour with FEX gdbserver waiting for attach, but in
the future when we all full process trees to have gdbserver running this
will be more important.
Using the server mount folder works most of the time, but when running
under pressure-vessel this stacks directories in a weird way because the
mount folder has some tricks applied to it.
Expose the temp folder being used directly instead.
These are setup to be nullptr by default. Instead of providing no-op
lock instructions just have them be nullptr.
It's already part of the API that they need to be nullptr checked before
calling and this matches behaviour of the real libX11 library. Removes
some spam that is unnecessary.
Unused since TestNZ+NZCVSelect accomplishes the same and good riddance. Might
bring them back later for tbz/tbnz, but certainly not in this Selectful form.
(I added them when I thought we were going to RA the flags. With the more
effective static approach, we don't need this for that.)
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Now unused. If we bring it back, it should be brought back as CSSC only. On
non-CSSC platforms, an explicit cmp + predicated neg can be better.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Separate out the NZCV bits from the more complex stuff so we can specially
optimize the branches.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
In this mode, rather than the branch comparing its arguments and then jumping
based on the result, the branch simply jumps by the native comparison based on
the NZCV value. This allows us to map x86 branches to arm64 branches 1:1.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Resolving the issue that we can only ever have one gdbserver process
running consuming port 8086 (even if the port number is cute).
Doesn't give us anything yet but in the future will allow us to have
whole process trees running gdbservers that we can attach to.
Usually better in practice... some rotates are slightly regressed by this but
they were already terrible.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
via the 0 reg. We really need a more generalized approach to taking advantage of
wzr, but this optimizes the special case I care about for seta (saving a move to
make the impl optimal).
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This will replace Select soon, as it lets us take advantage of
NZCV-generating instructions and it doesn't clobber NZCV.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Do it explicitly for sve-256 and punt on optimizing, so we avoid regressing code
gen otherwise.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Rather than the context. Effectively a static register allocation scheme for
flags. This will let us optimize out a LOT of flag handling code, keeping things
in NZCV rather than needing to copy between NZCV and memory all the time.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Some opcodes only clobber NZCV under certain circumstances, we don't yet have
a good way of encoding that. In the mean time this hot fixes some would-be
instcountci regressions.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Again we need to handle this one specially because the dispatcher can't insert
restore code after the branch. It should be optimized in the near future, don't
worry.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Semantics differ markedly from the non-NZCV flags, splitting this out makes it a
lot easier to do things correctly imho. Gets the dest/src size correct
(important for spilling), as well as makes our existing opt passes skip this
which is needed for correctness at the moment anyway.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
This fixes an issue where CPU tunables were ending up in the thunk
generator which means if your CPU doesn't support all the features on
the *Builder* then it would crash with SIGILL. This was happening with
Canonical's runners because they typically only support ARMv8.2 but we
are compiling packages to run on ARMv8.4 devices.
cc: FEX-2311.1
SHA instructions are very large right now and cause register spilling
due to their codegen. Ender Lilies has a really large block in a
function called `sha1_block_data_order` that was causing FEX to spill
NZCV flags incorrectly. The assumption which held true before NZCV
optimizations were a thing was that all flags were either 1-bit in an
8-bit container, or just 8-bit (x87 TOP flag).
NZCV host flags broke this assumption by making its flags 32-bit which
ended up breaking when encounting spilling situations.
Replace every instance of the Op overwrite pattern, and ban that anti-pattern
from the codebase in the future. This will prevent piles of NZCV related
regressions.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
The "create op with wrong opcode, then change the opcode" pattern is REALLY
dangerous. This does not address that. But when we start doing NZCV trickery, it
will get /more/ dangerous, and so it's time to add a helper and make the
convenient thing the safe(r) thing. This helper correctly saves NZCV /before/
the instruction like the real builders would. It also provides a spot for future
safety asserts if someone is motivated.
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
Requires #3249 to be merged first
Library alerting has been disabled for now, and storing IR while
gdbserver is running is removed.
Otherwise no functional change.
"Allows FEX to run without installation. Global locations for configuration and binfmt_misc are ignored. These files are instead read from <FEXInterpreterPath>/fex-emu/ by default.",
"For further customization, see FEX_APP_CONFIG_LOCATION and FEX_APP_DATA_LOCATION."
Loaded 100 of 2136 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Ryan Houdek