Compare commits

..
991 Commits
Author SHA1 Message Date
Ryan Houdek 20caf69951 Docs: Update for release FEX-2412 2024-12-03 11:02:44 -08:00
Ryan Houdek 731e4d6271 Merge pull request #4168 from pmatos/SVEStoreInstCountCI
instcountci: testing multiple 80bit ldst using SVE
2024-12-02 07:56:12 -08:00
Paulo Matos 48beb18f29 instcountci: testing multiple 80bit ldst using SVE
In preparation for #4166 which should improve on these results.
2024-12-02 11:37:11 +01:00
LC 41c8731443 Merge pull request #4188 from Sonicadvance1/fexcore_remove_unnecessary
FEXCore: Removes ExitHandler and RunUntilExit
2024-12-01 15:06:57 -05:00
Ryan Houdek efb276f489 FEXCore: Removes ExitHandler and RunUntilExit
Now that all the threading behaviour has been correctly separated/moved
to the frontend, these functions serve no purpose.

- Instead of using RunUntilExit, all threads can use `ExecuteThread`
  directly, since there's nothing special about the primary thread now.
  - This also removes the public function definition of `ExecutionThread` since that was only used for threading logic.
- Instead of using an exit handler, just do the same cleanup after
  `ExecuteThread` has returned.
  - Just make gdbserver is cleaned up early if it exists since it may
    want to send some things to the connected gdb instance before
    threads are exited.
2024-12-01 10:45:38 -08:00
LC 9febddefa3 Merge pull request #4190 from Sonicadvance1/remove_old_gprsize
FEXCore: Removes GetGPRSize and convert all uses to GetGPROpSize
2024-12-01 13:13:21 -05:00
Ryan Houdek ce9a860335 OpcodeDispatcher: Also remove unused CacheIndexToSize 2024-12-01 05:38:11 -08:00
Ryan Houdek 0123946ed1 FEXCore: Removes GetGPRSize and convert all uses to GetGPROpSize
Only a few remaining uses left, easy enough to convert. This finally
switches the final few uses over.

NFC
2024-12-01 05:38:09 -08:00
LC c7098d0da1 Merge pull request #4189 from Sonicadvance1/fexcore_remove_definition
FEXCore: Removes stale function definition
2024-12-01 08:29:11 -05:00
LC 65a162bdf9 Merge pull request #4187 from Sonicadvance1/fexcore_remove_coreshuttingdown
FEXCore: Removes CoreShuttingDown from ContextImpl
2024-12-01 08:28:32 -05:00
Ryan Houdek 2de485d02a FEXCore: Removes stale function definition
`CopyMemoryMapping` was removed a long time ago, the definition happened
to remain. Remove the definition.
2024-11-30 23:20:07 -08:00
Ryan Houdek bf64facaf6 FEXCore: Removes CoreShuttingDown from ContextImpl
This is unused now.

NFC
2024-11-30 22:10:37 -08:00
LC 2e7fc60dbf Merge pull request #4169 from Sonicadvance1/x87_loadstore_tests
unittests/ASM: Fixes x87 80-bit loads on the edge of page boundaries.
2024-11-30 10:43:10 -05:00
LC baddfe00b1 Merge pull request #4186 from Sonicadvance1/remove_remaining_runningevents
FEXCore: Removes remaining RunningEvents from InternalThreadState
2024-11-29 20:00:00 -05:00
Ryan Houdek e7e59204d3 FEXCore: Removes remaining RunningEvents from InternalThreadState
These are all frontend constructs with mostly deprecated constraints.
WaitingToStart isn't used anymore, Running is effectively always true
(and behaviour has changed that if a thread is alive, it's running).

The only one that remains is `ThreadSleeping` which is only handled in
the frontend, and there was some conflation between ThreadSleeping and
Running which was hard to gauge. So delete `Running` and
`WaitingToStart`, but move `ThreadSleeping` to the frontend.
2024-11-29 14:08:55 -08:00
Ryan Houdek 95d5b14f99 Merge pull request #4183 from Sonicadvance1/move_fexcore_executionthread
FEXCore: Moves InternalThreadState ExecutionThread to the frontend
2024-11-29 14:06:42 -08:00
Ryan Houdek 802eaee9c8 FEXCore: Moves InternalThreadState ExecutionThread to the frontend
Once again this is another frontend construct, so move it to
ThreadStateObject
2024-11-29 13:33:56 -08:00
LC e89f48f237 Merge pull request #4182 from Sonicadvance1/move_start_paused
FEXCore: Move InternalThreadState StartRunning to frontend
2024-11-29 16:31:37 -05:00
Ryan Houdek e771e25632 LinuxSyscalls/Thread: Build child thread arguments on parent stack
Now that most of the thread tracking is in the frontend, change this
over to building the thread execution handler on the parent thread.

Removes a memory allocation/free pair, and removes the copy of each
variable in the child thread.
2024-11-29 09:38:36 -08:00
Ryan Houdek 25c202575e FEXCore: Move InternalThreadState StartRunning to frontend
We were using this variable for two things, letting the frontend signal
to the backend that it wants to start executing once the thread is
created, and also for handling thread pausing. These two features are
conflated with one another and actually makes things more confusing.

- Move StartRunning/StartPaused to the frontend, because its a construct
  that only needs to exist in the frontend
- Adds a FEX::HLE::ThreadStateObject CV for handling pausing, which only
  needs to exist for gdbserver
2024-11-29 09:38:24 -08:00
Ryan Houdek f59fc0f747 Merge pull request #4181 from Sonicadvance1/remove_exitreason
FEXCore: Removes ExitReason from InternalThreadState
2024-11-29 09:36:33 -08:00
Ryan Houdek f7a076e00c FEXCore: Removes ExitReason from InternalThreadState
FEXCore hasn't been returning anything other than EXIT_SHUTDOWN for a
long time, so this ended up just moving data around for no reason.

This isn't going to be used for further GdbServer work anyway, so just
completely remove it.
2024-11-29 09:25:44 -08:00
Ryan Houdek 56fadecdaf Merge pull request #4179 from Sonicadvance1/move_thread_waiting_start
FEXCore: Moves ThreadWaiting to the frontend
2024-11-29 09:23:26 -08:00
Ryan Houdek 9f681f9e41 FEXCore: Moves ThreadWaiting to the frontend
Only in one location does the frontend actually care about this, the
backend doesn't care at all.
2024-11-29 09:10:36 -08:00
Ryan Houdek 1b11f2f184 Merge pull request #4185 from asahilina/fix-autoshutdown-regression
FEXServer: Fix auto-shutdown regression
2024-11-29 09:03:53 -08:00
LC b2e61c37be Merge pull request #4170 from Sonicadvance1/gdbserver_work
GdbServer: Minor work
2024-11-29 08:24:43 -05:00
Asahi Lina 7c0cf51f09 FEXServer: Fix auto-shutdown regression
Fixes: #4184
2024-11-29 20:56:07 +09:00
Ryan Houdek 649a49488b Merge pull request #4180 from Sonicadvance1/fexcore_const_ptr_ctx
FEXCore: Constify CTX ptr in InternalThreadState
2024-11-28 17:02:16 -08:00
Ryan Houdek aa2180d494 Merge pull request #4178 from Sonicadvance1/move_statuscode_frontend
FEXCore: Moves StatusCode to the frontend
2024-11-28 16:51:33 -08:00
Ryan Houdek 969cae581c FEXCore: Constify CTX ptr in InternalThreadState
The CTX pointer in the InternalThreadState object will not and must not
change, since it is associated with that CTX object.

Contify it to codify it.
2024-11-28 15:57:36 -08:00
Ryan Houdek 1bf7e2544a FEXCore: Moves StatusCode to the frontend
This is a Linux construct, move it to the frontend.

This is going to need some changes in the future since exit_group and
exit syscalls are supposed to behave differently than how FEX implements
it. For now just move it to the frontend.
2024-11-28 15:55:46 -08:00
Ryan Houdek fad22144a2 Merge pull request #4177 from Sonicadvance1/move_deferred_signal_state
FEXCore: Moves DeferredSignalFrames to the frontend
2024-11-28 15:55:02 -08:00
Ryan Houdek b440e176fb Merge pull request #4176 from Sonicadvance1/move_signalreason
FEXCore: Moves SignalThread/SignalEvent to Frontend
2024-11-28 15:54:21 -08:00
Ryan Houdek 56c6b0d2cb Merge pull request #4175 from Sonicadvance1/gdbserver_remove_earlyexit
FEXCore: Removes EarlyExit running event
2024-11-28 15:53:17 -08:00
Ryan Houdek 0596a963e1 Merge pull request #4174 from Sonicadvance1/gdbserver_move_alloc_tls
FEXCore: Moves TLS initialization for Alloc::OSAllocator
2024-11-28 15:49:19 -08:00
Ryan Houdek 357cc04940 GdbServer: Splits Multi-letter v command handler
Just breaks out the two commands we support and leaves TODOs for
implementing the remaining commands.

NFC
2024-11-28 15:32:50 -08:00
Ryan Houdek 7c6e836865 GdbServer: Split out GDB context definition generation to its own function
GDB has two ways to read the registers. One way is reading the full
GDBContextDefinition, which matches the layout in `BuildTargetXML`.

The other way is to read the individual elements out of
GDBContextDefinition.

These two code paths were independently implemented. Instead generate in
one location and use in either location.

NFC
2024-11-28 15:32:50 -08:00
Ryan Houdek 54a7317312 GdbServer: Split out function searching for thread by TID
This currently happens in two locations, so split it out.

There's some behaviour here where if the TID isn't found, then it
returns the ParentThread of the process. This is working around a bug in
either FEX's gdbserver or binaryninja. Leave it currently before we
figure out what's wrong.

NFC
2024-11-28 15:32:50 -08:00
Ryan Houdek 1fb20710e6 GdbServer: Switch to thread specific stopping break logic
Previous `S AA` logic is legacy for non-multithreaded applications. This
newer command gives more information about what occured and in what
thread id.
2024-11-28 15:32:50 -08:00
Ryan Houdek 811ea093b5 GdbServer: Split out qXfer handlers
NFC, just making this easier to track for me.
2024-11-28 15:32:50 -08:00
Ryan Houdek 71fe9aee21 GdbServer: Fixes thread name setting
When parsing `comm`, by default it will have a newline which breaks gdb
in some cases. Strip out the whitespace to fix that issue.
2024-11-28 15:32:50 -08:00
Ryan Houdek 38c834e731 FEXCore: Removes global StartPaused check for gdb
This doesn't behave properly anymore now that thread management was
moved to the frontend.
2024-11-28 15:32:50 -08:00
Ryan Houdek 740ff60a71 GdbServer: Reorganize packet command handlers
Makes these consistent in the handling and documents the commands in a
way that is easier to parse while working on this.

NFC
2024-11-28 15:32:50 -08:00
Ryan Houdek ee69b9f650 GdbServer: Reconstruct XMM/YMM registers using FEXCore helpers
Previously this would have corrupted data in the upper 128-bits of the
YMM register.
2024-11-28 15:32:50 -08:00
Ryan Houdek a4565ce783 GdbServer: Pass through FCW
We have supported this for a while, just wasn't passed through gdbserver
since it usually doesn't matter.
2024-11-28 15:32:50 -08:00
Ryan Houdek 3131ee4de1 GdbServer: Moves information fetching to independent files
NFC
2024-11-28 15:32:50 -08:00
Ryan Houdek 3ecc66fbcf FEXCore: Moves TLS initialization for Alloc::OSAllocator
Alloc::OSAllocator uses a TLS variable of the thread object so it can
use a forkable mutex plus a deferring signal section. This was setup
when the FEXCore "ExecutionThread" function is called, which is a bit
awkward and is an artifact from when the thread creation was mixed
between the frontend and the backend.

Instead let the frontend inform the backend when to install the TLS
variable.

This is one step required to make GdbServer work correctly again since
the thread initialization and pausing is awkward today.
2024-11-28 02:46:06 -08:00
Ryan Houdek e322e84785 FEXCore: Moves DeferredSignalFrames to the frontend
Deferred signal frames are a frontend construct. Move it there.
2024-11-28 01:14:55 -08:00
Ryan Houdek f0fa7a5b6a FEXCore: Moves SignalThread/SignalEvent to Frontend
This is purely a Linux frontend construct now, move it.
2024-11-28 00:57:50 -08:00
Ryan Houdek 718221be71 FEXCore: Removes EarlyExit running event
This was working around an edge case in the GdbServer where a thread was
getting created while the process was shutting down. This edge case is
getting removed so get rid of it.
2024-11-28 00:46:42 -08:00
Ryan Houdek ee592ba03c Merge pull request #4173 from neobrain/fix_ctest_list
CMake: Generate test list even when testing is disabled
2024-11-27 13:52:32 -08:00
Tony Wasserka 56947f3a94 CMake: Generate test list even when testing is disabled
Previously, running ctest with BUILD_TESTS=OFF would discover and execute
leftover tests from a previous build. This change ensures CTestTestfile.cmake
gets regenerated so that ctest will see an empty test list in that case.
2024-11-27 11:02:54 +01:00
Ryan Houdek f41b9bc514 Merge pull request #4172 from alyssarosenzweig/jit/cf
OpcodeDispatcher: drop PossiblySetNZCV
2024-11-26 15:20:44 -08:00
Ryan Houdek 0463512c6c Merge pull request #4171 from Sonicadvance1/fix_ltrim
Utils/StringUtil: Fixes ltrim and adds a unittest
2024-11-26 13:09:22 -08:00
Ryan Houdek 47369d058e Utils/StringUtil: Fixes ltrim and adds a unittest
ltrim had the issue that it would always consume the left-most character
even if it wasn't whitespace. So `FEXLoader` would turn in to
`EXLoader`, even without any whitespace in the string.

Adds a test to ensure this doesn't occur again.
2024-11-26 12:57:48 -08:00
Billy Laws b6f34fa209 unittests: Add test for carry inversion bug 2024-11-26 09:02:23 -05:00
Alyssa Rosenzweig 03e0ca9833 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-11-26 09:02:01 -05:00
Alyssa Rosenzweig d19473160d OpcodeDispatcher: drop InvalidateDeferredFlags
it is now useless.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-11-26 09:02:01 -05:00
Alyssa Rosenzweig aeb2c98cbf OpcodeDispatcher: drop PossiblySetNZCV
this is a pain to track and, it turns out, buys us virtually nothing on flagm
systems. rip it out.

this fixes a bug with failing to set in all the right places.

on non-flagm systems there's a slight instcountci impact, but that is mostly
mitigated by the earlier patches in the series. so overall a wash there but
worth it for making the codebase easier
to reason about.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-11-26 08:59:19 -05:00
Alyssa Rosenzweig 2350ae5a07 OpcodeDispatcher: optimize BTC on !flagm
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-11-26 08:59:19 -05:00
Alyssa Rosenzweig 6076d1747e OpcodeDispatcher: use NZV invalidate CF set for BT
this is similar perf on flagm and better on not flagm.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-11-26 08:59:19 -05:00
Alyssa Rosenzweig f4ce6fb621 OpcodeDispatcher: optimize AAS/AAD flag
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-11-26 08:59:19 -05:00
Ryan Houdek f0d25d413b InstcounCI: Update 2024-11-24 00:38:54 -08:00
Ryan Houdek c84503c271 unittests/ASM: Adds x87 loadstore tests for edge of pages
These tests ensure that FEX's x87 80-bit loadstores don't read/write
past the end of the page.
2024-11-24 00:36:50 -08:00
Ryan Houdek 1465df874b OpcodeDispatcher: Fixes 80-bit loads
Ensures reads don't go past the end of the page boundary.
SVE masked loads can make this more effective but `VLoadVectorMasked`
isn't setup to be efficient for this case yet.
2024-11-24 00:36:50 -08:00
Ryan Houdek 60c52e3826 Merge pull request #4167 from pmatos/CheckLDPathNonEmpty
Check that LDPath is not empty
2024-11-22 15:47:42 -08:00
Paulo Matos 13b806130b Check that LDPath is not empty
This ensures we don't underflow at --RootFSLength.
2024-11-22 10:24:14 +01:00
Ryan Houdek 22058c06a1 Merge pull request #4164 from bylaws/swwow
WOW64: Set the software CPU area flag
2024-11-19 09:39:33 -08:00
Ryan Houdek d5c96555f1 Merge pull request #4159 from asahilina/fexserver-two-sockets
FEXServer: Listen on both abstract & named sockets
2024-11-19 09:28:48 -08:00
Asahi Lina c3e8cd8d30 docs: Document that std::filesystem::temp_directory_path() is unsafe 2024-11-20 02:05:32 +09:00
Asahi Lina d761fc44f4 FEXServer: Listen on both abstract & named sockets
Abstract sockets have one limitation: they are bound to a network
namespace. Chromium/CEF sandboxes using a new netns, which breaks
connecting to the FEXServer.

To work around this, use and try *both* abstract and named sockets. As
long as either the filesystem or the network is unsandboxed, things will
work. If both are sandboxed, there isn't much we can do... but at that
point we shouldn't be reinitializing the FEXServer connection anyway
since the FS should be available on FEXInterpreter startup.
2024-11-20 01:58:17 +09:00
Asahi Lina a1aa2547ce FEXServerClient: Do not use strerror() in ConnectToServer()
This triggers glibc allocation.

Signed-off-by: Asahi Lina <lina@asahilina.net>
2024-11-20 01:58:17 +09:00
Asahi Lina 44213c3968 FEXServerClient: Switch GetTempFolder to not use temp_directory_path()
Apparently this causes allocations which are banned in some paths?
2024-11-19 01:46:33 +09:00
Billy Laws 830bd347c5 WOW64: Set the software CPU area flag
This is required for wow64.dll to setup the cross-process queue
that is used to pass through e.g. memory unmap events.
2024-11-18 16:18:31 +00:00
Ryan Houdek bcfdf39d63 Merge pull request #4157 from asahilina/fix-chromium-sandbox
Support CLONE_FS and CLONE_FILES with fork() semantics
2024-11-18 06:42:15 -08:00
Asahi Lina bfed21870f Support CLONE_FS and CLONE_FILES with fork() semantics
Needed by Discord, part of the Chromium sandbox code. The warning still
triggers because Chromium asks for CLONE_VM on x86_64, but that can be
safely ignored (CLONE_FS is the one that matters).
2024-11-18 22:09:33 +09:00
Ryan Houdek 4278c48791 Merge pull request #4158 from asahilina/hide-rootfs-fd-take2
FileManagement: Hide the FEX RootFS fd from /proc/self/fd take 2
2024-11-17 20:27:59 -08:00
Ryan Houdek c1db7a78b1 Merge pull request #4153 from pmatos/VixlSkip
Add some more tests as unsupported by vixl
2024-11-17 19:56:56 -08:00
Ryan Houdek 1bf06f8946 Merge pull request #4162 from pmatos/WarningAvoid
Avoid warning on assertionless builds
2024-11-17 19:03:31 -08:00
Ryan Houdek 09bfe58827 Merge pull request #4160 from asahilina/align-stack
FEXLoader: Align stack base
2024-11-17 18:27:25 -08:00
Paulo Matos 474c780399 Avoid warning on assertionless builds
This was causing unused variable warning due
to the variable only being used in an assertion.
2024-11-13 15:17:36 +01:00
Asahi Lina 4a67893f1d FEXLoader: Align stack base
This ensures that __libc_stack_end is aligned, the same way it is on
native.
2024-11-13 03:47:43 +09:00
Asahi Lina 73ffaa1e18 FileManagement: Hide the FEX RootFS fd from /proc/self/fd take 2
Apparently Chromium/CEF can chroot or otherwise sandbox the filesystem
away before forking and checking for directory FDs, making /proc
inaccessible, which means we can't stat it for our inode check, breaking
the hiding.

So, double down on things and do what Chromium does: open an fd to /proc
ahead of time, so that continues to work. Then we use it to update the
inode of our RootFS fd instead, and finally, also do the /proc fd itself
to hide that one too.

We also don't need to check the st_dev of /proc more than once, since
that's not expected to change anyway.

Fixes cefsimple.
2024-11-13 01:26:42 +09:00
Ryan Houdek e675f4241a Merge pull request #4154 from Liamolucko/check-home
Check if a candidate home directory exists before using it
2024-11-05 20:07:30 -08:00
Liam Murphy 7a61d9d2b4 Check if a candidate home directory exists before using it
This allows FEX to be used in situations where `HOME` is set to
something invalid, e.g. inside Nix builds.
2024-11-06 11:16:53 +11:00
Paulo Matos 06b950a9cd Rounding test doesn't need to be skipped 2024-11-04 19:02:12 +01:00
Paulo Matos de70651406 Add some more tests as unsupported by vixl
It seems a form of `mrs` is unsupported as well as the hint `wfe` used for `pause`.
Remove skipping Rounding(Neg|Pos).asm as they are passing.
2024-11-04 18:52:48 +01:00
LC 5ad7fdb2f3 Merge pull request #4149 from Sonicadvance1/iropsize_convert_class
IR: Convert OpSize over to enum class
2024-10-30 23:55:55 -04:00
Ryan Houdek 9b6cc8f7e0 IR: Convert OpSize over to enum class
NFC

Do the final mopping up to convert the OpSize enum to an enum class!
2024-10-29 16:52:16 -07:00
LC 5c6de4ed14 Merge pull request #4147 from Sonicadvance1/iropsize_convert_irops
IR: Converts base IR operations to store OpSize sizes
2024-10-29 12:02:11 -04:00
Ryan Houdek 82f936cb6d IR: Converts base IR operations to store OpSize sizes
NFC

Finally converts the IR operations themselves to store the OpSize for
the IR operation size and element sizes.

This also finally, FINALLY, converts that remaining `_Constant` helper
to stop using a size field that is specified in bits rather than bytes
like all the other IR op handlers. That thing was so confusing and now
it's gone.
2024-10-28 21:26:59 -07:00
LC 493b952e3f Merge pull request #4146 from Sonicadvance1/iropsize_implicit_jit
JIT: Remove implicit OpSize conversions
2024-10-28 23:47:36 -04:00
Ryan Houdek 460a21625e JIT: Remove implicit OpSize conversions
NFC
2024-10-28 19:48:40 -07:00
LC 00ab3f8440 Merge pull request #4145 from Sonicadvance1/iropsize_various_implicit
OpcodeDispatcher: Various missed OpSize implicit cast fixes
2024-10-28 22:38:41 -04:00
Ryan Houdek 034b62292b Passes/x86StackOptimization: Fixes implicit conversion of OpSize 2024-10-28 19:26:02 -07:00
Ryan Houdek 7b615a07d0 OpcodeDispatcher: Various missed OpSize implicit cast fixes
NFC
Probably more of these around, just tracking the few I found.
2024-10-28 19:18:36 -07:00
LC 704841f004 Merge pull request #4144 from Sonicadvance1/iropsize_addrsize
OpcodeDispatcher: Convert address size helpers to use OpSize
2024-10-28 22:12:40 -04:00
LC c122f3faf9 Merge pull request #4143 from Sonicadvance1/iropsize_flags
OpcodeDispatcher: Convert flags helpers over to OpSize
2024-10-28 22:10:19 -04:00
Ryan Houdek f74f276d64 OpcodeDispatcher: Convert address size helpers to use OpSize
NFC
2024-10-28 19:02:34 -07:00
Ryan Houdek 4b10cbdafd OpcodeDispatcher: Convert flags helpers over to OpSize
NFC

Plus the tertiary bits that require changing to support it.
2024-10-28 18:56:42 -07:00
LC 04c701e912 Merge pull request #4142 from Sonicadvance1/irsize_loadstoregpr
OpcodeDispatcher: Convert {Load,Store}GPRRegister to OpSize
2024-10-28 21:04:01 -04:00
LC 0c29f8faad Merge pull request #4141 from Sonicadvance1/missing_ir_sizes
IR: Fix some missing OpSize conversions
2024-10-28 21:02:46 -04:00
Ryan Houdek 5ed82fa0f6 OpcodeDispatcher: Convert {Load,Store}GPRRegister to OpSize
Trivial but quite a few places pass in a raw integer

NFC
2024-10-28 16:35:23 -07:00
Ryan Houdek 6cca007817 IR: Fix some missing OpSize conversions
Missed these in the previous PR.
2024-10-28 16:25:36 -07:00
LC c0a9463700 Merge pull request #4140 from Sonicadvance1/enforce_irsize
Convert all of the IR operations to use OpSize
2024-10-28 18:20:02 -04:00
Ryan Houdek 55b3d67eb4 Merge pull request #4138 from asahilina/hide-rootfs-fd
FileManagement: Hide the FEX RootFS fd from /proc/self/fd
2024-10-28 14:24:55 -07:00
Ryan Houdek 65ddae1b71 IR: Change F80VBSLStack to use IR::OpSize 2024-10-28 02:25:17 -07:00
Ryan Houdek 063f524084 IR: Change F80CVTToInt to use IR::OpSize 2024-10-28 02:24:25 -07:00
Ryan Houdek 2dd0a82059 IR: Change F80CVTTo to use IR::OpSize 2024-10-28 02:23:47 -07:00
Ryan Houdek b810070e9f IR: Change F80CVTInt to use IR::OpSize 2024-10-28 02:07:20 -07:00
Ryan Houdek 4c7ac17f7d IR: Change F80CVT to use IR::OpSize 2024-10-28 02:06:48 -07:00
Ryan Houdek 84767c8b20 IR: Change PushStack to use IR::OpSize 2024-10-28 02:05:35 -07:00
Ryan Houdek eccfb53bd5 IR: Change StoreStackMemory to use IR::OpSize 2024-10-28 02:01:47 -07:00
Ryan Houdek f4e930262f IR: Change PCLMUL to use IR::OpSize 2024-10-28 01:50:24 -07:00
Ryan Houdek d26d9e7e03 IR: Change CRC32 to use IR::OpSize 2024-10-28 01:50:24 -07:00
Ryan Houdek 51c1998d70 IR: Change VAESDecLast to use IR::OpSize 2024-10-28 01:50:24 -07:00
Ryan Houdek 87f818249d IR: Change VAESDec to use IR::OpSize 2024-10-28 01:50:23 -07:00
Ryan Houdek d81f92f5e2 IR: Change VAESEncLast to use IR::OpSize 2024-10-28 01:50:23 -07:00
Ryan Houdek 29ffe02afe IR: Change VAESEnc to use IR::OpSize 2024-10-28 01:50:23 -07:00
Ryan Houdek dfe4076fe4 IR: Change Vector_F64ToI32 to use IR::OpSize 2024-10-28 01:50:23 -07:00
Ryan Houdek 44f9df062e IR: Change Vector_FToI to use IR::OpSize 2024-10-28 01:50:23 -07:00
Ryan Houdek 6f98ef8cbb IR: Change VFCVTN2 to use IR::OpSize 2024-10-28 01:50:23 -07:00
Ryan Houdek d54888a4c6 IR: Change VFCVTL2 to use IR::OpSize 2024-10-28 01:50:23 -07:00
Ryan Houdek 38c58706da IR: Change Vector_FToF to use IR::OpSize 2024-10-28 01:50:23 -07:00
Ryan Houdek 8ad9286bd4 IR: Change Vector_FToZS to use IR::OpSize 2024-10-28 01:29:32 -07:00
Ryan Houdek 03bc962564 IR: Change Vector_FToS to use IR::OpSize 2024-10-28 01:28:56 -07:00
Ryan Houdek 947b7ae6fe IR: Change Vector_SToF to use IR::OpSize 2024-10-28 01:23:20 -07:00
Ryan Houdek 30317ac979 IR: Change Float_FToF to use IR::OpSize 2024-10-28 01:22:45 -07:00
Ryan Houdek 4544e7c1af IR: Change Float_FromGPR_S to use IR::OpSize 2024-10-28 01:21:48 -07:00
Ryan Houdek 34050431ff IR: Change VDupFromGPR to use IR::OpSize 2024-10-28 01:20:06 -07:00
Ryan Houdek 65439956bf IR: Change VCastFromGPR to use IR::OpSize 2024-10-28 01:18:21 -07:00
Ryan Houdek a6cbce4fd7 IR: Change VFNMLS to use IR::OpSize 2024-10-28 01:16:11 -07:00
Ryan Houdek 17b851d4f3 IR: Change VFNMLA to use IR::OpSize 2024-10-28 01:15:47 -07:00
Ryan Houdek 362b5728be IR: Change VFMLS to use IR::OpSize 2024-10-28 01:15:25 -07:00
Ryan Houdek bd5159c7d5 IR: Change VFMLA to use IR::OpSize 2024-10-28 01:15:01 -07:00
Ryan Houdek 660dfcd1f9 IR: Change VFCADD to use IR::OpSize 2024-10-28 01:14:21 -07:00
Ryan Houdek 4e21177988 IR: Change VBSL to use IR::OpSize 2024-10-28 01:13:53 -07:00
Ryan Houdek 5a83a65905 IR: Change VTBX1 to use IR::OpSize 2024-10-28 01:13:15 -07:00
Ryan Houdek 061fc44923 IR: Change VTBL2 to use IR::OpSize 2024-10-28 01:13:00 -07:00
Ryan Houdek 0c7afa0672 IR: Change VTBL1 to use IR::OpSize 2024-10-28 01:12:43 -07:00
Ryan Houdek 30bf0d5767 IR: Change VFCMPUNO to use IR::OpSize 2024-10-28 01:12:22 -07:00
Ryan Houdek ee8e3127d2 IR: Change VFCMPORD to use IR::OpSize 2024-10-28 01:11:54 -07:00
Ryan Houdek e8f64f2976 IR: Change VFCMPLE to use IR::OpSize 2024-10-28 01:11:31 -07:00
Ryan Houdek 0a4b21da87 IR: Change VFCMPGT to use IR::OpSize 2024-10-28 01:11:11 -07:00
Ryan Houdek 074777bc75 IR: Change VFCMPLT to use IR::OpSize 2024-10-28 01:10:50 -07:00
Ryan Houdek 47c403998a IR: Change VFCMPNEQ to use IR::OpSize 2024-10-28 01:10:28 -07:00
Ryan Houdek 0fa095cec5 IR: Change VFCMPEQ to use IR::OpSize 2024-10-28 01:10:07 -07:00
Ryan Houdek cfa4e7f165 IR: Change VCMPGT to use IR::OpSize 2024-10-28 01:09:23 -07:00
Ryan Houdek 75b8226e7d IR: Change VCMPEQ to use IR::OpSize 2024-10-28 01:09:04 -07:00
Ryan Houdek 9e3c50ca2c IR: Change VExtr to use IR::OpSize 2024-10-28 01:08:35 -07:00
Ryan Houdek 365d8b9508 IR: Change VInsGPR to use IR::OpSize 2024-10-28 01:07:16 -07:00
Ryan Houdek 02ebe06496 IR: Change VInsElement to use IR::OpSize 2024-10-28 01:05:34 -07:00
Ryan Houdek 34d5e70e6b IR: Change VUShlSWide to use IR::OpSize 2024-10-28 00:59:33 -07:00
Ryan Houdek 2d8bd7b59d IR: Change VSShrSWide to use IR::OpSize 2024-10-28 00:57:02 -07:00
Ryan Houdek 3a6f5e638b IR: Change VUShrSWide to use IR::OpSize 2024-10-28 00:54:45 -07:00
Ryan Houdek c06274066f IR: Change VSShrS to use IR::OpSize 2024-10-28 00:51:19 -07:00
Ryan Houdek 820b0be9f2 IR: Change VUShrS to use IR::OpSize 2024-10-28 00:50:57 -07:00
Ryan Houdek 3e74817dd9 IR: Change VUShlS to use IR::OpSize 2024-10-28 00:50:36 -07:00
Ryan Houdek 3b9a2d2141 IR: Change VSShr to use IR::OpSize 2024-10-28 00:50:15 -07:00
Ryan Houdek 9ba431a51d IR: Change VUShr to use IR::OpSize 2024-10-28 00:49:52 -07:00
Ryan Houdek d0d2229db6 IR: Change VUShl to use IR::OpSize 2024-10-28 00:49:26 -07:00
Ryan Houdek 286258a2f2 IR: Change VUABDL2 to use IR::OpSize 2024-10-28 00:48:59 -07:00
Ryan Houdek ac14a88647 IR: Change VUABDL to use IR::OpSize 2024-10-28 00:48:32 -07:00
Ryan Houdek bf19578673 IR: Change VSMulH to use IR::OpSize 2024-10-28 00:47:44 -07:00
Ryan Houdek c5f396d889 IR: Change VUMulH to use IR::OpSize 2024-10-28 00:47:08 -07:00
Ryan Houdek 22325500d9 IR: Change VSMull2 to use IR::OpSize 2024-10-28 00:46:12 -07:00
Ryan Houdek 51eede080c IR: Change VUMull2 to use IR::OpSize 2024-10-28 00:45:50 -07:00
Ryan Houdek 503c86d47d IR: Change VSMull to use IR::OpSize 2024-10-28 00:45:23 -07:00
Ryan Houdek 0e0887181d IR: Change VUMull to use IR::OpSize 2024-10-28 00:44:20 -07:00
Ryan Houdek 974cc591bd IR: Change VMul to use IR::OpSize 2024-10-28 00:43:57 -07:00
Ryan Houdek 844afdb653 IR: Change VFMax to use IR::OpSize 2024-10-28 00:43:36 -07:00
Ryan Houdek c3c643d9b7 IR: Change VFMin to use IR::OpSize 2024-10-28 00:43:14 -07:00
Ryan Houdek b51b0c5f62 IR: Change VFDiv to use IR::OpSize 2024-10-28 00:42:54 -07:00
Ryan Houdek 96bdefddc9 IR: Change VFMul to use IR::OpSize 2024-10-28 00:42:32 -07:00
Ryan Houdek ba74a6a252 IR: Change VFSub to use IR::OpSize 2024-10-28 00:42:09 -07:00
Ryan Houdek 79427097e1 IR: Change VFAddV to use IR::OpSize 2024-10-28 00:41:46 -07:00
Ryan Houdek 7ec21b7121 IR: Change VFAddP to use IR::OpSize 2024-10-28 00:41:18 -07:00
Ryan Houdek 29115b3185 IR: Change VFAdd to use IR::OpSize 2024-10-28 00:38:55 -07:00
Ryan Houdek 33b3814642 IR: Change VTrn2 to use IR::OpSize 2024-10-28 00:37:44 -07:00
Ryan Houdek 85431a8132 IR: Change VTrn to use IR::OpSize 2024-10-28 00:35:12 -07:00
Ryan Houdek 2d9ef56a8b IR: Change VUnZip2 to use IR::OpSize 2024-10-28 00:31:48 -07:00
Ryan Houdek b2ae829731 IR: Change VUnZip to use IR::OpSize 2024-10-28 00:31:23 -07:00
Ryan Houdek a3c544a9a1 IR: Change VZip2 to use IR::OpSize 2024-10-28 00:27:28 -07:00
Ryan Houdek 9c2292289b IR: Change VZip to use IR::OpSize 2024-10-28 00:26:34 -07:00
Ryan Houdek b514548ca2 IR: Change VSMax to use IR::OpSize 2024-10-28 00:22:02 -07:00
Ryan Houdek 692a4a8fcd IR: Change VSMin to use IR::OpSize 2024-10-28 00:21:39 -07:00
Ryan Houdek 3a07cf7d70 IR: Change VUMax to use IR::OpSize 2024-10-28 00:21:22 -07:00
Ryan Houdek 7f5421fc26 IR: Change VUMin to use IR::OpSize 2024-10-28 00:21:00 -07:00
Ryan Houdek 04d62cd269 IR: Change VURAvg to use IR::OpSize 2024-10-28 00:20:35 -07:00
Ryan Houdek ac55e468a7 IR: Change VAddP to use IR::OpSize 2024-10-28 00:20:07 -07:00
Ryan Houdek e20db7dc88 IR: Change VSQSub to use IR::OpSize 2024-10-28 00:18:03 -07:00
Ryan Houdek 235bee9191 IR: Change VSQAdd to use IR::OpSize 2024-10-28 00:17:20 -07:00
Ryan Houdek 7a429b01c7 IR: Change VUQSub to use IR::OpSize 2024-10-28 00:16:31 -07:00
Ryan Houdek fbf5b14933 IR: Change VUQAdd to use IR::OpSize 2024-10-28 00:16:01 -07:00
Ryan Houdek 0dfd5dd96f IR: Change VXor to use IR::OpSize 2024-10-28 00:15:33 -07:00
Ryan Houdek 6d5acec958 IR: Change VOr to use IR::OpSize 2024-10-28 00:13:06 -07:00
Ryan Houdek beed43e577 IR: Change VAndn to use IR::OpSize 2024-10-28 00:12:37 -07:00
Ryan Houdek fc04b9113e IR: Change VAnd to use IR::OpSize 2024-10-28 00:11:44 -07:00
Ryan Houdek 68c038085a IR: Change VSub to use IR::OpSize 2024-10-28 00:10:21 -07:00
Ryan Houdek 176f5a2860 IR: Change VAdd to use IR::OpSize 2024-10-28 00:07:30 -07:00
Ryan Houdek e7c9623aa9 IR: Change VRev64 to use IR::OpSize 2024-10-27 23:36:03 -07:00
Ryan Houdek 53ca2ac378 IR: Change VRev32 to use IR::OpSize 2024-10-27 23:34:51 -07:00
Ryan Houdek d140cb4450 IR: Change VSQSHL to use IR::OpSize 2024-10-27 23:34:22 -07:00
Ryan Houdek 0ba501636e IR: Change VSRSHR to use IR::OpSize 2024-10-27 23:33:49 -07:00
Ryan Houdek ceca9fff17 IR: Change VSQXTUNPair to use IR::OpSize 2024-10-27 23:30:02 -07:00
Ryan Houdek 3e31abb645 IR: Change VSQXTUN2 to use IR::OpSize 2024-10-27 23:27:53 -07:00
Ryan Houdek 6c07cd319b IR: Change VSQXTUN to use IR::OpSize 2024-10-27 23:27:27 -07:00
Ryan Houdek 5247b7124f IR: Change VSQXTNPair to use IR::OpSize 2024-10-27 23:26:55 -07:00
Ryan Houdek 6cec557855 IR: Change VSQXTN2 to use IR::OpSize 2024-10-27 23:24:50 -07:00
Ryan Houdek 1868bd6777 IR: Change VSQXTN to use IR::OpSize 2024-10-27 23:24:25 -07:00
Ryan Houdek 7c7efeda82 IR: Change VUXTL2 to use IR::OpSize 2024-10-27 23:23:58 -07:00
Ryan Houdek 414486f1dd IR: Change VUXTL to use IR::OpSize 2024-10-27 23:23:12 -07:00
Ryan Houdek b8bc9659d4 IR: Change VSSHLL2 to use IR::OpSize 2024-10-27 23:22:47 -07:00
Ryan Houdek d52a6e6fc4 IR: Change VSSHLL to use IR::OpSize 2024-10-27 23:22:19 -07:00
Ryan Houdek 5ab41056ab IR: Change VSXTL2 to use IR::OpSize 2024-10-27 23:21:49 -07:00
Ryan Houdek 0692b34192 IR: Change VSXTL to use IR::OpSize 2024-10-27 23:21:17 -07:00
Ryan Houdek 3636c332ff IR: Change VUShrNI2 to use IR::OpSize 2024-10-27 23:05:41 -07:00
Ryan Houdek cb18963ded IR: Change VUShrNI to use IR::OpSize 2024-10-27 23:05:08 -07:00
Ryan Houdek 8cf92d3303 IR: Change VSShrI to use IR::OpSize 2024-10-27 23:03:21 -07:00
Ryan Houdek cdc5c15b4b IR: Change VUShraI to use IR::OpSize 2024-10-27 22:58:35 -07:00
Ryan Houdek ed313edd07 IR: Change VUShrI to use IR::OpSize 2024-10-27 22:57:57 -07:00
Ryan Houdek 9b981a4f61 IR: Change VShlI to use IR::OpSize 2024-10-27 22:55:08 -07:00
Ryan Houdek c791893b4a IR: Change VDupElement to use IR::OpSize 2024-10-27 22:50:47 -07:00
Ryan Houdek 2605c7e0b3 IR: Change VCMPLTZ to use IR::OpSize 2024-10-27 22:44:45 -07:00
Ryan Houdek f315948028 IR: Change VCMPGTZ to use IR::OpSize 2024-10-27 22:44:14 -07:00
Ryan Houdek 869367f7e2 IR: Change VCMPEQZ to use IR::OpSize 2024-10-27 22:43:42 -07:00
Ryan Houdek 3a2c7e8edd IR: Change VFRSqrt to use IR::OpSize 2024-10-27 22:43:12 -07:00
Ryan Houdek 0a34a43976 IR: Change VFSqrt to use IR::OpSize 2024-10-27 22:42:37 -07:00
Ryan Houdek 78cd21d78f IR: Change VFRecp to use IR::OpSize 2024-10-27 22:33:53 -07:00
Ryan Houdek 9f18de0196 IR: Change VFNeg to use IR::OpSize 2024-10-27 22:33:25 -07:00
Ryan Houdek 0bffdc4e27 IR: Change VFAbs to use IR::OpSize 2024-10-27 22:32:37 -07:00
Ryan Houdek 0f5ff53386 IR: Change VUMaxV to use IR::OpSize 2024-10-27 22:31:31 -07:00
Ryan Houdek 21611fc1ad IR: Change VUMinV to use IR::OpSize 2024-10-27 22:31:04 -07:00
Ryan Houdek 99e1eb5452 IR: Change VAddv to use IR::OpSize 2024-10-27 22:30:30 -07:00
Ryan Houdek 8c3ca44c57 IR: Change VPopcount to use IR::OpSize 2024-10-27 22:28:51 -07:00
Ryan Houdek 7a85e17d14 IR: Change VAbs to use IR::OpSize 2024-10-27 22:28:24 -07:00
Ryan Houdek b533dcd86d IR: Change VNot to use IR::OpSize 2024-10-27 22:27:28 -07:00
Ryan Houdek a379ce6fed IR: Change VNeg to use IR::OpSize 2024-10-27 22:09:02 -07:00
Ryan Houdek efd5c51110 IR: Change LoadNamedVectorIndexedConstant to use IR::OpSize 2024-10-27 22:08:27 -07:00
Ryan Houdek 886db4ffca IR: Change LoadNamedVectorConstant to use IR::OpSize 2024-10-27 22:05:42 -07:00
Ryan Houdek e6f6ee2bcd IR: Change VectorImm to use IR::OpSize 2024-10-27 21:58:34 -07:00
Ryan Houdek 4a6b5d4ec7 IR: Change VMov to use IR::OpSize 2024-10-27 21:52:11 -07:00
Ryan Houdek 027e7624cb IR: Change VFNMLSScalarInsert to use IR::OpSize 2024-10-27 18:37:30 -07:00
Ryan Houdek 0e31077735 IR: Change VFNMLAScalarInsert to use IR::OpSize 2024-10-27 18:36:53 -07:00
Ryan Houdek c8a9dd0d0a IR: Change VFMLSScalarInsert to use IR::OpSize 2024-10-27 18:36:14 -07:00
Ryan Houdek 2bd7ddaa31 IR: Change VFMLAScalarInsert to use IR::OpSize 2024-10-27 18:35:40 -07:00
Ryan Houdek 5566b4455b IR: Change VFCMPScalarInsert to use IR::OpSize 2024-10-27 18:35:03 -07:00
Ryan Houdek fed2c13521 IR: Change VFToIScalarInsert to use IR::OpSize 2024-10-27 18:32:11 -07:00
Ryan Houdek 37d092aab8 IR: Change VSToFGPRInsert to use IR::OpSize 2024-10-27 18:29:34 -07:00
Ryan Houdek 5626f4e50a IR: Change VSToFVectorInsert to use IR::OpSize 2024-10-27 18:27:39 -07:00
Ryan Houdek efbc42dac3 IR: Change VFToFScalarInsert to use IR::OpSize 2024-10-27 18:25:13 -07:00
Ryan Houdek 160934884d IR: Change VFRecpScalarInsert to use IR::OpSize 2024-10-27 18:21:05 -07:00
Ryan Houdek af1cfcb9bd IR: Change VFRSqrtScalarInsert to use IR::OpSize 2024-10-27 18:20:35 -07:00
Ryan Houdek d6f726fc23 IR: Change VFSqrtScalarInsert to use IR::OpSize 2024-10-27 18:20:06 -07:00
Ryan Houdek 92ee071eb2 IR: Change VFMaxScalarInsert to use IR::OpSize 2024-10-27 18:17:43 -07:00
Ryan Houdek bdfa8ad4f3 IR: Change VFMinScalarInsert to use IR::OpSize 2024-10-27 18:17:17 -07:00
Ryan Houdek 37540f4927 IR: Change VFDivScalarInsert to use IR::OpSize 2024-10-27 18:16:30 -07:00
Ryan Houdek 000ab5ff19 IR: Change VFMulScalarInsert to use IR::OpSize 2024-10-27 18:15:58 -07:00
Ryan Houdek f054274948 IR: Change VFSubScalarInsert to use IR::OpSize 2024-10-27 18:15:22 -07:00
Ryan Houdek 081907e168 IR: Change VFAddScalarInsert to use IR::OpSize 2024-10-27 18:14:37 -07:00
Ryan Houdek 1a115a8ce6 IR: Change FCmp to use IR::OpSize 2024-10-27 18:06:34 -07:00
Ryan Houdek fd9158c75f IR: Change Float_ToGPR_ZS to use IR::OpSize 2024-10-27 18:03:18 -07:00
Ryan Houdek 1bde30a196 IR: Change Float_ToGPR_S to use IR::OpSize 2024-10-27 18:02:45 -07:00
Ryan Houdek 764aacaa8f IR: Change VExtractToGPR to use IR::OpSize 2024-10-27 17:58:53 -07:00
Ryan Houdek a848211926 IR: Change NZCVSelectV to use IR::OpSize 2024-10-27 17:53:15 -07:00
Ryan Houdek f1a42869d5 IR: Change CondJump to use IR::OpSize 2024-10-27 17:50:51 -07:00
Ryan Houdek 97a6ba9931 IR: Change VLoadNonTemporal to use IR::OpSize 2024-10-27 17:45:29 -07:00
Ryan Houdek f4744f1e79 IR: Change VStoreNonTemporalPair to use IR::OpSize 2024-10-27 17:44:57 -07:00
Ryan Houdek 321f686108 IR: Change VStoreNonTemporal to use IR::OpSize 2024-10-27 17:44:23 -07:00
Ryan Houdek 90340350fa IR: Change MemCpy to use IR::OpSize 2024-10-27 17:43:10 -07:00
Ryan Houdek 6f4fd4467b IR: Change MemSet to use IR::OpSize 2024-10-27 17:42:39 -07:00
Ryan Houdek b31ce13f68 IR: Change Pop to use IR::OpSize 2024-10-27 17:41:30 -07:00
Ryan Houdek 260d3b0b4e IR: Change Push to use IR::OpSize 2024-10-27 17:39:21 -07:00
Ryan Houdek c8c7ffbf05 IR: Change VBroadcastFromMem to use IR::OpSize 2024-10-27 17:35:56 -07:00
Ryan Houdek 4b03185b77 IR: Change VStoreVectorElement to use IR::OpSize 2024-10-27 17:32:25 -07:00
Ryan Houdek 52ec572db3 IR: Change VLoadVectorElement to use IR::OpSize 2024-10-27 17:27:42 -07:00
Ryan Houdek dc31cf83c6 IR: Change VLoadVectorGatherMaskedQPS to use IR::OpSize 2024-10-27 17:22:07 -07:00
Ryan Houdek 8a4f51257d IR: Change VLoadVectorGatherMasked to use IR::OpSize 2024-10-27 17:21:33 -07:00
Ryan Houdek 051469fa16 IR: Change VStoreVectorMasked to use IR::OpSize 2024-10-27 17:20:56 -07:00
Ryan Houdek 3f6cdc2e03 IR: Change VLoadVectorMasked to use IR::OpSize 2024-10-27 17:20:18 -07:00
Ryan Houdek f3449f2b00 IR: Change StoreMemTSO to use IR::OpSize 2024-10-27 17:17:26 -07:00
Ryan Houdek d7691d9a25 IR: Change LoadMemTSO to use IR::OpSize 2024-10-27 17:16:23 -07:00
Ryan Houdek f414d4934c IR: Change StoreMemPair to use IR::OpSize 2024-10-27 17:15:00 -07:00
Ryan Houdek 014917301a IR: Change StoreMem to use IR::OpSize 2024-10-27 17:14:18 -07:00
Ryan Houdek cc483acbde IR: Change LoadMemPair to use IR::OpSize 2024-10-27 16:33:52 -07:00
Ryan Houdek 07f8a4eadd IR: Change LoadMem to use IR::OpSize 2024-10-27 16:33:14 -07:00
Ryan Houdek 5fd127b53a IR: Change StoreContextIndexed to use IR::OpSize 2024-10-27 15:51:30 -07:00
Ryan Houdek ece89ddeab IR: Change LoadContextIndexed to use IR::OpSize 2024-10-27 15:50:03 -07:00
Ryan Houdek a1565a7d99 IR: Change StoreContextPair to use IR::OpSize 2024-10-27 15:47:08 -07:00
Ryan Houdek 2f9b0de742 IR: Change StoreContext to use IR::OpSize 2024-10-27 15:46:32 -07:00
Ryan Houdek 7e5f1b5859 IR: Change LoadContextPair to use IR::OpSize 2024-10-27 15:42:39 -07:00
Ryan Houdek 40fd4bbb66 IR: Change LoadContext to use IR::OpSize 2024-10-27 15:42:07 -07:00
Ryan Houdek e4143352c9 IR: Change Store{PF,AF} to use IR::OpSize 2024-10-27 15:37:41 -07:00
Ryan Houdek c045e14837 IR: Change StoreRegister to use IR::OpSize 2024-10-27 15:37:05 -07:00
Ryan Houdek f0f3c215ce IR: Change Load{PF,AF} to use IR::OpSize 2024-10-27 15:35:26 -07:00
Ryan Houdek 8f4113d859 IR: Change LoadRegister to use IR::OpSize 2024-10-27 15:34:46 -07:00
Ryan Houdek 4cfc2ac1a4 IR: Change AllocateFPR to use IR::OpSize 2024-10-27 15:30:11 -07:00
LC d2aa5217dc Merge pull request #4132 from Sonicadvance1/fix_irsize
Fix IR operation usage to use OpSize when possible
2024-10-27 17:26:11 -04:00
Ryan Houdek e8baf4a28c OpcodeDispatcher: Ensure IR ops use OpSize
NFC
2024-10-27 14:11:25 -07:00
Ryan Houdek e438d32879 OpcodeDispatcher/Vector: Ensure IR ops use OpSize
NFC
2024-10-27 14:11:07 -07:00
Ryan Houdek 32ef10b273 OpcodeDispatcher/AVX128: Ensure IR ops use OpSize
NFC
2024-10-27 14:11:07 -07:00
Ryan Houdek ad296051b7 OpcodeDispatcher/Crypto: Ensure IR ops use OpSize
NFC
2024-10-27 14:11:07 -07:00
Ryan Houdek e603136918 OpcodeDispatcher/Flags: Ensure IR ops use OpSize
NFC
2024-10-27 14:11:07 -07:00
Ryan Houdek f8a61f7d7e OpcodeDispatcher/X87: Ensure IR ops use OpSize
NFC
2024-10-27 14:11:07 -07:00
Ryan Houdek cb5ba8baae OpcodeDispatcher/X87F64: Ensure IR ops use OpSize
NFC
2024-10-27 14:11:07 -07:00
LC 079e70fc4e Merge pull request #4134 from Sonicadvance1/move_jit
JIT: Moves Arm64 JIT up one folder
2024-10-27 17:10:26 -04:00
Asahi Lina 3d701f5fcf FileManagement: Hide the FEX RootFS fd from /proc/self/fd
Chromium/CEF has code that iterates through all open FDs and bails if
any are directories (apparently a sandboxing sanity check). To avoid
this check, we need to hide the RootFS FD. This requires hooking all the
getdents variants to skip that entry.

To keep the runtime cost low, we keep track of the inode of
/proc/self/fd/<rootfs fd> (note: not the RootFS inode, the inode of the
magic symlink in /proc), and first do a quick check on that. If it
matches, then we stat the dirfd we are reading and check against the
procfs device, to complete the inode equality check.

As an extra benefit, this also fixes code that tries to iterate and
close all/extra FDs and ends up closing the RootFS fd.
2024-10-27 07:05:00 +09:00
Ryan Houdek b31e4a3c27 JIT: Moves Arm64 JIT up one folder
We have only one JIT, there is no reason to subfolder this. Move it up
one folder.

NFC
2024-10-25 18:44:20 -07:00
LC 992d6e8477 Merge pull request #4136 from Sonicadvance1/support_tpidrro
FEXCore: Adds support for CPU Index through TPIDRRO
2024-10-25 21:43:33 -04:00
LC f6cdb165a3 Merge pull request #4137 from Sonicadvance1/minor_pushf_opt
OpcodeDispatcher: Minor optimization to small pushf
2024-10-25 21:42:45 -04:00
LC f60388d160 Merge pull request #4135 from Sonicadvance1/remove_xop
X86Tables: Removes XOP tables
2024-10-25 19:41:31 -04:00
Ryan Houdek 96fa2ad8eb InstcountCI: Update 2024-10-25 15:43:10 -07:00
Ryan Houdek f143462ebe OpcodeDispatcher: Minor optimization to small pushf
The push operation already truncates the result, there's no need to bfe
it. Noticed this while cleaning up in #4134. Removes one instruction for
16-bit and 32-bit pushf instructions.
2024-10-25 15:41:22 -07:00
Ryan Houdek 51fa61a1cd InstcountCI: Adds missing pushf implementations
pushf was aliasing to pushfq, needed an o16 prefix.
We also weren't testing the 32-bit path, which only exists on 32-bit, so
add that as well.
2024-10-25 15:40:44 -07:00
Ryan Houdek 048e967546 FEXCore: Adds support for CPU Index through TPIDRRO 2024-10-25 15:07:57 -07:00
Ryan Houdek 608fd49ac3 CodeEmitter/unittests: Add support for TPIDRRO_EL0 2024-10-25 15:07:29 -07:00
Ryan Houdek bb630797b5 CodeEmitter: Add support for TPIDRRO_EL0 2024-10-25 15:07:15 -07:00
Ryan Houdek 01a6e914f2 X86Tables: Removes XOP tables
These weren't even wired up to the frontend. We aren't going to support
XOP, so just remove the tables.
2024-10-25 14:32:13 -07:00
Ryan Houdek 0190e1a00b Merge pull request #4130 from pmatos/X87F64Simp
X87 Code Simplification
2024-10-24 10:57:12 -07:00
Paulo Matos 11a87c22f9 instcountci: X87 code simplification 2024-10-24 18:17:48 +02:00
Paulo Matos 5f6c0d2245 X87 code simplification
Merges some of the code from reduced precision into the main path
since they are practically the same.
2024-10-24 18:15:49 +02:00
Ryan Houdek caaacb6c15 Merge pull request #4127 from alyssarosenzweig/opt/masking
Optimize bsf, bsr, register cmpxchg, pcmpistri
2024-10-23 07:28:29 -07:00
Ryan Houdek 767c61c08b Merge pull request #4129 from pmatos/RPRESInstcountci
Disable RPRES in instcounci files
2024-10-23 07:27:57 -07:00
Paulo Matos dc93e30451 Disable RPRES in instcounci files
This was giving false changes on RPRES enabled HW.
2024-10-23 15:09:17 +02:00
Ryan Houdek 368162df87 Merge pull request #4128 from ahoneybun/update-ubuntu-support
add Ubuntu 24.10 and remove unsupported releases
2024-10-22 14:43:22 -07:00
Aaron Honeycutt 9eb2106ed2 update supported list 2024-10-22 15:35:41 -06:00
Aaron Honeycutt cfc05b78fe add Ubuntu 24.10 2024-10-22 15:27:21 -06:00
Alyssa Rosenzweig d2a42c0038 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-22 13:39:21 -04:00
Alyssa Rosenzweig 58a3d174ec OpcodeDispatcher: explain why we provide defined bsf behaviour
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-22 13:36:24 -04:00
Alyssa Rosenzweig 9c605e7333 OpcodeDispatcher: optimize bsf/bsr
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-22 13:31:49 -04:00
Alyssa Rosenzweig 1fd7e88ffd OpcodeDispatcher: optimize cmp in cmpxchg
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-22 13:29:47 -04:00
Alyssa Rosenzweig c5e7da0631 OpcodeDispatcher: optimize more cmpxchg mask
none of it matters.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-22 13:29:47 -04:00
Alyssa Rosenzweig 68f58e415f OpcodeDispatcher: optimize cmpxchg masking
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-22 13:29:47 -04:00
Alyssa Rosenzweig 698abec25c JIT: drop FindMSB zero handling
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-22 13:29:47 -04:00
Alyssa Rosenzweig 1578f5ed47 JIT: drop FindLSB masking
consequence of the UB

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-22 13:29:47 -04:00
Alyssa Rosenzweig 80d7b5a5c9 JIT: drop FindLSB zero handling
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-22 13:29:47 -04:00
Alyssa Rosenzweig eb023ceb51 IR: make FindLSB/FindMSB undefined for zero
these are used in places:

* bsf/bsr
* pcmpblabla
* x87 fild

In all cases we explicitly check for zero and change the behaviour accordingly.
So weaken the IR op to let us optimize. No sense checking twice.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-22 13:29:47 -04:00
LC 5d1fda7d7f Merge pull request #4125 from Sonicadvance1/unify_pshuflhw
OpcodeDispatcher: Unify PSHUF{L,H}W implementations
2024-10-21 14:37:39 -04:00
Ryan Houdek fafc04a59e OpcodeDispatcher/AVX128: Fixes glibc allocation 2024-10-21 09:34:59 -07:00
Ryan Houdek ddcca58f64 InstcountCI: Update for PSHUF{L,H}W changes 2024-10-21 09:25:58 -07:00
Ryan Houdek fe8f5c745d OpcodeDispatcher/AVX128: Use unified PSHUF{L,H}W implementation
Allows the AVX128 implementation to use the same implementation as the
128-bit SSE implementation, because it works per 128-bit lane just like
SSE. Is a minor optimization.
2024-10-21 09:24:38 -07:00
Ryan Houdek d876224358 OpcodeDispatcher: Unify MMX and SSE PSHUF{L,H}W implementations
No functional change.
2024-10-21 09:11:05 -07:00
LC c4306f2f0a Merge pull request #4122 from Sonicadvance1/avx_fixes
AVX128: Fixes some AVX bugs
2024-10-18 19:34:28 -04:00
Ryan Houdek 8b7a227820 InstcountCI: Update 2024-10-18 15:49:25 -07:00
Ryan Houdek d7afcee622 AVX128: Fixes asome AVX bugs
vblendvps, vblendvpd, vpblendvb all broke because of failing to zext the
128-bit register correctly.

vinsertps broke because it was accidentally using the wrong
implementation.

Extends each of their unittests to handle these cases.
2024-10-18 15:49:25 -07:00
Ryan Houdek a421ff1105 Merge pull request #4093 from pmatos/FXtractFix
Fix FXTRACT for 0.0 and -0.0
2024-10-17 05:31:34 -07:00
Paulo Matos 5997030c97 instcountci: Fix FXTRACT for 0.0 and -0.0 2024-10-17 09:13:40 +02:00
Paulo Matos 3c8086373b FXTRACT fix ASM tests 2024-10-17 09:05:17 +02:00
Paulo Matos 10ec6b63b6 Fix FXTRACT for 0.0 and -0.0
Fixes fxtract by returning the correct values for 0.0 and -0.0. We moved the split of fxtract into _sig and _exp, to the opcode dispatcher, to ease some comparisons.

Also removed the IR node F80XTRACTStack which is not needed anymore.
2024-10-17 09:05:10 +02:00
Paulo Matos 49087007be Implement NZCVSelectV for selection on FPRs
Behaves like NZCVSelect for FPRs.
2024-10-17 08:42:50 +02:00
Ryan Houdek 0897cd8777 Merge pull request #4116 from Sonicadvance1/personality_handling
LinuxEmulation: Personality handling
2024-10-16 15:52:35 -07:00
Ryan Houdek 4b945a9041 Merge pull request #4105 from pmatos/X87MMXState
Implement explicit state switch between X87 and MMX
2024-10-16 15:52:27 -07:00
Ryan Houdek d66ed71bc6 Merge pull request #4082 from Sonicadvance1/fix_wine
FEXLoader: Fixes newer wine versions and Fedora
2024-10-16 15:52:19 -07:00
LC b5b34df155 Merge pull request #4119 from Sonicadvance1/unaligned_lock
unittests/ASM: Adds missing unaligned atomic tests
2024-10-15 12:16:37 -04:00
LC ff51435747 Merge pull request #4118 from Sonicadvance1/wfe_for_pause
FEXCore: Change yield implementation to use wfe
2024-10-15 12:15:11 -04:00
Paulo Matos def561986b instcountci: Implements explicit state switch between X87 and MMX 2024-10-15 17:58:59 +02:00
Paulo Matos 5c258d4a2a ASM Test: Implements explicit state switch between X87 and MMX
Tags is set to all valid in FEX, but in host it's set to all valid _and_
reinterpreted. Adding this to known failures in the host runner.
2024-10-15 17:58:59 +02:00
Paulo Matos 0d53f2b45c Implements explicit state switch between X87 and MMX
Fixes #3850
2024-10-15 17:58:53 +02:00
Ryan Houdek 4f03044fe7 unittests/ASM: Adds missing unaligned atomic tests
Fixes #2670

Walked through all the unaligned atomic tests to find which ones were
missing. Turns out it was only ADC, NEG, NOT, and SBB.
2024-10-15 06:59:38 -07:00
Ryan Houdek b967538435 InstcountCI: Add pause instruction 2024-10-15 05:52:41 -07:00
Ryan Houdek e53f3969e9 FEXCore: Change yield implementation to use wfe
According to
https://github.com/rust-lang/rust/commit/c064b6560b7ce0adeb9bbf5d7dcf12b1acb0c807
turns out that the arm yield instruction is effectively a nop on all
reasonably new CPUs.
Instead switch over to wfe because it matches x86 `PAUSE` semantics more
closely.
2024-10-15 05:50:26 -07:00
Ryan Houdek 5026bf8247 Merge pull request #4117 from pmatos/NoTest
Remove file since FXAM_Simple is not a Linux test
2024-10-11 18:17:09 -07:00
Paulo Matos 09cb4f5fc5 Remove file since FXAM_Simple is not a Linux test
Already properly skipped in the right place.
File added accidentally.
2024-10-11 16:55:07 +02:00
Paulo Matos ddd7a550e4 Remove check on top 16bits
Makes it uniform among all 3DNow tests instead of
some checking and some don't.
2024-10-11 16:18:43 +02:00
Ryan Houdek 3398f22c16 FEXLinuxTests: Adds personality test
These would have failed before the prior changes.
2024-10-11 05:03:28 -07:00
Ryan Houdek 7f17519fbf LinuxEmulation/personality: Support PER_LINUX32 2024-10-11 04:52:23 -07:00
Ryan Houdek a65884f9ae LinuxEmulation/personality: Support UNAME26 2024-10-11 04:52:21 -07:00
Ryan Houdek d70766f4c8 LinuxEmulation: Support personality tracking
Doesn't handle the emulation of it, but handle passing it to the host
kernel, tracking the value, and inheriting it through new threads.
2024-10-11 04:52:21 -07:00
Ryan Houdek 1365aa8881 FEXLinuxTests: Update to c++20 2024-10-11 04:52:21 -07:00
Ryan Houdek fe5bc02682 FEXLoader: Fixes newer wine versions and Fedora
This was brought up by #3831 but I finally got the courage to look at
the hard problem.

Although I'm only tackling half of the problem with this PR, which is
that FEXLoader needs to strip the rootfs path from the executed path if
it begins with the rootfs, plus some changes to the surrounding code.

The primary concern here is that when an application has been executed
under FEX, specifically through binfmt_misc, then FEX needs to prepend
the full rootfs path otherwise Linux can't find the program.
Additionally execveat with an FD will resolve a full path to the rootfs.

So past FEX's initial setup, we need to strip off the rootfs path to
provide an "absolute" path that is visible to the guest application
later. Which is kind of funny since we have a `RootFSRedirect` function
which did the exact opposite. This was due to legacy problems in the
original ELFLoader that couldn't handle symlinks correctly, which has
since been resolved, so that no longer needs to exist.

There was also some weirdness in `GetApplicationNames` where the passed
in argument list was modifying Args[0] and then saving the Program as
well. Which I just got rid of. Also stopped passing in the arguments by
value because....why did I write it like that?

In InterpreterHandler we now need to check if we can open the path
inside the rootfs or fallback without it. Plus I had to change the
shebang handling so it stopped prefixing the rootfs AGAIN. Took the time
to change the shebang handling there so it stops creating string copies
and instead just generates views.

Overall this fixes a fairly major flaw with how we were representing
`/proc/self` to the application, which was breaking wine since it would
prefix the rootfs multiple times, which was weird.

It doesn't address the remaining problem in #3831, which is that
applications can still see some of the leaky abstractions with symlinks
through the rootfs, but I want to get at least this step in.
2024-10-11 01:40:36 -07:00
Ryan Houdek 6f096e7c4b FHU: Add StringArgumentParser function
Split this out so we can unittest it.

Adds a unittest to handle specific edge cases.
2024-10-11 01:40:36 -07:00
LC 389ad737e6 Merge pull request #4103 from Sonicadvance1/shared_vdso_mmap
VDSOEmulation: Support loading VDSO thunk as shared
2024-10-10 01:25:12 -04:00
LC c00f7813a2 Merge pull request #4108 from Sonicadvance1/ensure_x87_size_save_restore
unittests/ASM: Ensures FNSAVE and FRSTOR only store as much data as required
2024-10-10 01:00:54 -04:00
LC e5ceaa182d Merge pull request #4110 from Sonicadvance1/remove_unused_memory_regions
unitests/ASM: Removes unused MemoryRegion configs
2024-10-10 00:59:31 -04:00
LC eeb8eb1824 Merge pull request #4109 from Sonicadvance1/fix_fsgs_testharness
TestHarnessRunner: Fixes FS/GS usage in tests
2024-10-10 00:56:20 -04:00
Ryan Houdek 7c6444c37c unitests/ASM: Removes unused MemoryRegion configs
FEX's ASM unitests had the problem that they were copy and pasted
templates and MemoryRegion was copied in to almost all tests.

Very few tests actually use the MemoryRegion they were asking for and
instead used none, or the hardcoded memory regions that the
TestHarnessRunner provides.

This is entirely a sed replacement and minor fixups plus reverts for the
few tests that actually use the region asked for.
2024-10-08 16:07:25 -07:00
Ryan Houdek 4a179c8f87 TestHarnessRunner: Fixes FS/GS usage in tests
When writing `FEX_bugs/tls_vector_element.asm` I had to switch to using
GS segment instead of FS segment because the unittests didn't correctly
restore FS after running. This is because GS is unused on Linux
applications, but FS would become broken and break glibc cleanup on
shutdown.

Now that xbyak has been updated to v7.09, it now supports
{rd,wr}{fs,gs}base which allows us to save and restore the segments
correctly. This lets us drop in TLS tests in to unittests more easily
now.

Ensured this works by modifying the test temporarily to use fs instead
of gs again, seeing it crash before HostRunner changes, and work after
HostRunner changes.

Fixes #4104
2024-10-08 15:42:44 -07:00
Ryan Houdek 2b3895a514 Update xbyak to v7.09 2024-10-08 15:38:00 -07:00
Ryan Houdek 0bb0f9cec7 unittests/ASM: Ensures FNSAVE and FRSTOR only store as much data as required
The instruction definition only allows these instructions to load/store
94 or 108 bytes, not affecting any bytes afterwards. This is a bit
awkward because 80-bit x87 registers are stored at the end.

FEX has an optimization today where it uses overlapping loads and stores
for the first seven x87 registers, and a split loadstore for the final
register. This ensures that we get the correct data while reducing the
number of loadstores.

We didn't have a unittest in place to ensure we only ever write the
correct amount of data, so changes like in #4107 which look correct from
an initial glance, would have resulted in broken behaviour.

This unittest ensures both that the instructions don't try to access
beyond the end of the page, and also ensures that they don't overwrite
subsequent data. Making sure that potentially broken behaviour doesn't
make its way in.
2024-10-08 15:31:36 -07:00
Ryan Houdek 6a07ea73a8 Merge pull request #4106 from slp/compat-input-prctl
FEXCore: adds support for compat input prctl
2024-10-08 13:22:19 -07:00
Sergio Lopez 5c51c54ccc FEXCore: adds support for compat input prctl
The size of the input_event struct differs between 32 bits applications
and 64 bits applications. To deal with this, the kernel implements a
compat variant for the input syscalls, but it's only enabled for 32 bit
processes.

In libkrunfw we're introducing a prctl that enables a 64 bit process to
request the kernel to enable the compat variant for the input syscalls.
This commit makes use of that interface for enabling/disabling the
compat input variant as required.

The visible effect is that input devices such as gamepads work properly
on emulated 32 bit applications.

Signed-off-by: Sergio Lopez <slp@redhat.com>
2024-10-08 11:38:42 +02:00
Ryan Houdek cac3767c20 Docs: Update for release FEX-2410 2024-10-07 11:14:53 -07:00
Ryan Houdek e82d2c70c3 Merge pull request #4095 from alyssarosenzweig/opt/jit-time
speed up the JIT
2024-10-04 09:00:03 -07:00
LC 9716fc73d3 Merge pull request #4099 from Sonicadvance1/v6.11
LinuxSyscalls: Update max reported kernel version to 6.11
2024-10-04 11:58:20 -04:00
LC aaa8eef9f1 Merge pull request #4098 from Sonicadvance1/linux_v6.11_syscalls
LinuxEmulation: Update syscalls for v6.11
2024-10-04 11:57:52 -04:00
LC c036868938 Merge pull request #4097 from Sonicadvance1/linux_v6.11_drm
LinuxEmulation: Update DRM for v6.11
2024-10-04 11:57:21 -04:00
LC 2726f35a10 Merge pull request #4102 from Sonicadvance1/fix_vector_element_loadstore
OpcodeDispatcher: Fixes segment prefixing on vector element loadstores
2024-10-04 11:56:15 -04:00
Ryan Houdek c740801ea5 VDSOEmulation: Support loading VDSO thunk as shared
Just requires a thread pointer check to be fixed in FEXCore.
This doesn't need unique pages to exist for the file mapping and can be
shared since it's readonly mapped.
2024-10-03 21:06:35 -07:00
Ryan Houdek 407dc5d4dd unittests: Adds TLS vector element loadstore test
Hits all the instructions that would have previously crashed.
2024-10-03 20:53:28 -07:00
Ryan Houdek 7dda75646d OpcodeDispatcher: Fixes segment prefixing on vector element loadstores
These had completely forgotten to load segment prefixes on the element
loadstore operations. This had caused a crash on the Linux version of
"Halls of Tormet" (steamid 2218750).

This game has TLS vector data which it was loading with a movhps and hit
this path.
2024-10-03 20:51:37 -07:00
Ryan Houdek b89f5b8a03 Merge pull request #4101 from Sonicadvance1/fprem_fixes
Fixes fprem
2024-10-03 15:49:22 -07:00
Ryan Houdek 16869d8954 unittests: fprem tests work since 688af49485 2024-10-03 14:03:51 -07:00
Ryan Houdek e353ae8408 unittests: Adds x87 fprem test
Exposes a bug in FEX's x87 FPREM implementation.
2024-10-03 14:03:51 -07:00
Alyssa Rosenzweig aa3c963df1 SoftFloat: fix FPREM
fixes incorrect signs on FPREM. in turn should fix end-to-end failures logging
into Steam.

Thank you to Sergio Lopez for tracking down the JavaScript fail, and Ryan for
finding the bug.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-03 13:56:14 -07:00
Ryan Houdek 0c8cfa7bb2 IoctlEmulation: Update AssignDeviceTypeToFD
Adds known good drivers to remove a log.
2024-10-02 19:29:45 -07:00
Ryan Houdek dd837fa693 LinuxSyscalls: Update max reported kernel version to 6.11
Investigated the uapi differences and nothing terrifying leapt out that
we haven't already implemented.
2024-10-02 15:36:47 -07:00
Ryan Houdek eeff198ff1 LinuxEmulation: Update syscalls for v6.11
Only thing added was uretprobe on x86-64. We can't support this so just
return ENOSYS.
2024-10-02 14:40:11 -07:00
Ryan Houdek 0cd6371c1b IoctlEmulation: Add panthor_drm 2024-10-02 14:27:24 -07:00
Ryan Houdek 5aff16f72b IoctlEmulation: Update xe_drm 2024-10-02 14:26:38 -07:00
Ryan Houdek 2c7896bce4 IoctlEmulation: Update v3d_drm 2024-10-02 14:26:30 -07:00
Ryan Houdek 6dd72a09e5 Scripts: Fixes newer clang-python
It provided this in the newer clang-python
2024-10-02 14:25:59 -07:00
Ryan Houdek 4e6a4d9b69 Linux: Update drm-headers to v6.11 2024-10-02 13:43:43 -07:00
Alyssa Rosenzweig 44484a7b05 ConstProp: drop non-loadbearing opts
every pattern costs us JIT time, but not every pattern is doing anything.
especially with the flag rework of 2023-2024, a lot of patterns just don't make
sense anymore. constant folding in particular isn't too useful now.

only instcountci change is AAD which i'm contractually forbidden from caring
about.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 14:01:32 -04:00
Alyssa Rosenzweig 6ac7ba388f ConstProp: rm leftover
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 36d0a67070 ConstProp: don't pool with a hash table
hash tables have high overhead! but individual blocks are small, so the
quadratic search is _much_ faster in practice. knocks 8% off node.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 0650dd1992 ConstProp: defer initialization
for blocks that don't need pooling. hopefully that's most of them.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig e2d58809ed OpcodeDispatcher: rm dead
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 967a74cda9 OpcodeDispatcher: manually inline constants
less work for constprop.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 6cc6181261 OpcodeDispatcher/Flags: dont emit zero
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig f175b525f4 OpcodeDispatcher/Flags: inline constants
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 319f1e66cb RegisterAllocationPass: eliminate a silly sxtb
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig d16969bdce RegisterAllocationPass: simplify
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig c17858bfeb RegisterAllocationPass: don't lookup sources
lot of silly chasing. not needed.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig ec24fc3d5d RegisterAllocationPass: infer AnyRemapped
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 8819fa88d5 RegisterAllocationPass: avoid allocating remaps
shaves 6ms off.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 13:38:48 -04:00
Alyssa Rosenzweig 5e9c2110db Merge pull request #4096 from pmatos/x87Cleanup
x87 small cleanups; NFC
2024-10-02 12:43:13 -04:00
Alyssa Rosenzweig 5aaa18e7a2 RegisterAllocationPass: remove Class->Allocated
not needed anymore.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 12:41:37 -04:00
Alyssa Rosenzweig 8a551b9e64 RegisterAllocationPass: rm leftover comment
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 12:41:37 -04:00
Alyssa Rosenzweig aa548bd19c RegisterAllocationPass: track if we need a remap
should be faster on average.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 12:41:37 -04:00
Alyssa Rosenzweig 9565f16d84 RegisterAllocationPass: simplify
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-10-02 12:41:37 -04:00
Paulo Matos eb76dbdf4d x87 small cleanups; NFC 2024-10-02 18:27:51 +02:00
Ryan Houdek 967c04e252 Merge pull request #4088 from bylaws/winesys
ARM64EC: Dynamically determine syscall numbers under wine.
2024-10-01 09:07:35 -07:00
Ryan Houdek 49de1fac59 Merge pull request #4028 from neobrain/refactor_jemallocless_tools
Build host tools without jemalloc
2024-09-30 11:59:45 -07:00
Ryan Houdek 3c205eb35e Merge pull request #4094 from Sonicadvance1/strict_unittest_args
unittests: Be more strict with -- separator
2024-09-30 11:38:30 -07:00
Tony Wasserka 682b8ef705 Build host tools without jemalloc
Jemalloc blocks use on platforms with 16K pages.
2024-09-30 20:38:24 +02:00
Tony Wasserka 8c47a40625 Update jemalloc 2024-09-30 20:35:46 +02:00
Ryan Houdek ee5c6af868 unittests: Be more strict with -- separator 2024-09-30 11:30:56 -07:00
Ryan Houdek 1670c89b5e Merge pull request #4090 from Sonicadvance1/update_vulkan_headers
Thunks/Vulkan: Updates thunks to v1.3.296
2024-09-30 09:54:15 -07:00
Ryan Houdek e9d34c7ded Merge pull request #4089 from bylaws/wineavx
ARM64EC: AVX register save/restore support under wine
2024-09-30 09:46:33 -07:00
Ryan Houdek c2f2c58367 Merge pull request #4060 from bylaws/ucrt
Windows: Switch supported MinGW CRT to UCRT
2024-09-30 09:31:27 -07:00
Billy Laws 38acd9e18c ARM64EC: Dynamically determine syscall numbers under wine
Takes advantage of the consistent alphabetical syscall ordering that
wine follows.
2024-09-29 19:42:49 +00:00
Billy Laws 21604708ca Windows: Commonise the linker loadcfg and support CFGuard toolchains 2024-09-29 19:42:13 +00:00
Billy Laws 5fec6faeb6 Windows: Switch supported MinGW CRT to UCRT
MSVCRT is deprecated and the reasons FEX used it no longer apply, so
support UCRT and UCRT only.
2024-09-29 19:42:13 +00:00
Billy Laws b659701cef ARM64EC: AVX register save/restore support under wine
Requires corresponding a wine patch to function:
https://github.com/bylaws/wine/commit/7d411bd74bd03ba90cf202623560733cb229cc7d

Since AVX still works in almost all cases without this, I chose not to
e.g. detect the patch and disable AVX support when it is missing.
2024-09-29 18:59:22 +00:00
LC e902ad5278 Merge pull request #3946 from Sonicadvance1/vdso_getrandom
VDSO: Implements v6.11 vdso getrandom
2024-09-28 15:26:35 -04:00
Ryan Houdek 0f54369f9b VDSO: Implements v6.11 vdso getrandom
This is an interesting vdso implementation because it only exists in
x86-64 with kernel v6.11. For Arm64 the implementation is likely to land
in a couple kernel versions from now.

Some differences with vdso_getrandom versus the regular getrandom
syscall
- Has two additional arguments, opaque_state and opaque_len
- Expects the userspace to mmap/munmap this opaque state structure
- Lets userspace query information about this structure upfront
- If the opaque data structure isn't provided then it falls back to
  regular SYS_getrandom

With the "glibc" implementation, we can tell the interface to allocate a
single page that gets unused (otherwise glibc ends up not using the
interface), and fallback to the regular SYS_getrandom.

In the case of the vdso interface, currently the arguments just get
passed through.

Keeping this as a WIP until the ARM64 kernel patches land and I can
actually test the things. Running the "glibc" path works today with the
selftest, but the vdso path is currently completely untested.
2024-09-28 01:06:56 -07:00
LC f640dcc7a4 Merge pull request #4075 from Sonicadvance1/move_thunkhandler
Thunks: Move to the frontend
2024-09-28 03:26:10 -04:00
Ryan Houdek b59da0e049 Thunks/Vulkan: Updates thunks to v1.3.296
Between this version and our previous version v1.3.278 this adds a few
extensions.
- VK_EXT_device_generated_commands
- VK_EXT_depth_clamp_control
- VK_AMD_anti_lag
- VK_KHR_pipeline_binary

These extensions are almost immediately getting picked up by mesa and
the D3D layers, so we want to support them as soon as possible.

Once again updated as described in #2076. This is starting to take more
time as more features and extensions get added to Vulkan. We probably
want to update the `DefinitionExtract.py` script at some point to
contain all the tertiary data in some json so everything gets generated
automatically. Not doing that today but something to think about.

Fixes #4083
2024-09-27 17:59:31 -07:00
Ryan Houdek 4ae3aef502 Vulkan-Headers: Update to v1.3.296 2024-09-27 17:17:50 -07:00
Ryan Houdek 926fa3c24c Thunks: Just move code around for a minor cleanup
NFC
2024-09-27 15:50:23 -07:00
Ryan Houdek 8c1740f592 Thunks: Remove the temporary TLS variables
From prep commit 511103ee5dcc9474b0b7468c05f61ce10fea4393.
Now that the frontend is setup, we can remove the temporary TLS
variables and use the ThreadObject directly.
2024-09-27 15:50:23 -07:00
Ryan Houdek 611aa0a5b9 Thunks: Wire up TLS handling in the frontend
Because it was moved from the backend to the frontend, re-wire up the
TLS handling which requires going through our syscall handler.
2024-09-27 15:50:21 -07:00
Ryan Houdek 0b8b5108b9 Thunks: Moves AppendThunkDefinitions function to the frontend
Backend doesn't need access to this at all
2024-09-27 15:48:31 -07:00
Ryan Houdek e409a0afec Thunks: Shift namespace name
Moves from FEXCore to FEX::HLE. Also moves the ThunkFunctions that get
exposed to a namespace to make it more obvious that these are
thunkhandlers rather than just static functions.
2024-09-27 15:48:31 -07:00
Ryan Houdek 24211f8523 FEXCore: Move ThunksHandler class to FEXLoader
With as little changes as possible, because this is fairly tricky.
2024-09-27 15:48:31 -07:00
Ryan Houdek 94462e4dd0 Thunks: Temporarily add more TLS variables while moving
Prep commit to give Thunks the data it needs before moving.
2024-09-27 15:48:31 -07:00
Ryan Houdek aa44668a41 FEXCore/Thunks: Moves the implementation of LinkAddressToGuestFunction to ContextImpl
No functional change, just moving code here.
2024-09-27 15:48:31 -07:00
Ryan Houdek 33a675624c FEXCore/Thunks: Remove lock passing in AddCustomIREntrypoint
This was a legacy feature that is no longer necessary every since we
removed the frontend IRLoader.
2024-09-27 15:48:31 -07:00
LC 5745b419d9 Merge pull request #4045 from Sonicadvance1/stop_using_dlopen_for_vdso
VDSOEmulation: Stop using dlopen for VDSO
2024-09-27 18:20:55 -04:00
LC 9100235041 Merge pull request #4077 from Sonicadvance1/frontend_informed_supportsavx
SignalDelegator: Let the frontend inform AVX support
2024-09-27 18:15:13 -04:00
LC 4d62e75d5a Merge pull request #4085 from pmatos/FScaleZero
Fix FScale'ing zero that should return zero
2024-09-27 18:13:07 -04:00
LC b6d3df0e54 Merge pull request #4086 from Sonicadvance1/stop_breaking_x86
Stop installing binfmt_misc on x86
2024-09-27 03:34:41 -04:00
Ryan Houdek d5d1230422 Stop installing binfmt_misc on x86
With the new systemd support, we had forgotten to wrap this check in an
arm64 check. I had done an install and broke my x86 machine.

Add the check back so we stop breaking x86 machines...again. Took me
only about three hours to find the issue this time.
2024-09-26 15:49:48 -07:00
Paulo Matos f35a212f06 Fix FScale'ing zero that should return zero
Added tests will fail without the current patch.
This will properly check if we return the correct sign of zero.
2024-09-26 17:57:39 +02:00
Ryan Houdek f49d82deb3 Merge pull request #4074 from Sonicadvance1/global_rootfs
FEXConfig: Add the ability to watch and configure global rootfs
2024-09-24 03:30:47 -07:00
Ryan Houdek 77b801b8ae FEXConfig: Add the ability to watch and configure global rootfs
Global rootfs doesn't support a named rootfs (only local supports that).
So when it is a global rootfs option just give the full path to the
config path.

Just needs some special handling to ensure the full path is retained for
global paths.

A reimplementation of #4067 that doesn't hardcode paths and routes the
global data path through FEXCore.
2024-09-24 01:17:45 -07:00
Ryan Houdek 163a3fc899 FEXConfig: Convert select over to ppoll usage
Using select initially was a mistake, it has issues with FD limits and
is less efficient due to fd_set. It's recommended to switch select using
over to polling, so do it.
2024-09-24 01:17:45 -07:00
Ryan Houdek 9b627e8743 FEXCore: Add global data directory support
Currently unused as all data still comes from local installs.
2024-09-24 01:17:43 -07:00
LC 9d6865f62d Merge pull request #4073 from Sonicadvance1/constexpr_all_tables
OpcodeDispatcher: Constexpr-ify all the tables possible
2024-09-23 21:40:37 -04:00
LC d547f2b8f2 Merge pull request #4080 from Sonicadvance1/improve_createthread_stack
Linux: Optimize CreateNewThread and HandleNewClone stack usage
2024-09-23 21:28:56 -04:00
Ryan Houdek b5c9eb3463 Merge pull request #4081 from Sonicadvance1/smarter_meta_reload
Config: Be a bit smarter in ReloadMetaLayer
2024-09-23 04:52:25 -07:00
Ryan Houdek 00a3793814 Config: Be a bit smarter in ReloadMetaLayer
Reduces stack usage from ~1024 bytes to ~704 bytes and stops making a
bunch of string copies when they can just be pointers instead.
2024-09-23 04:21:10 -07:00
LC 4f8d7cb89c Merge pull request #4078 from Sonicadvance1/fix_envloader_stackusage
Config: Fixes stack usage in EnvLoader
2024-09-22 19:19:46 -04:00
LC 9310ac59f4 Merge pull request #4076 from Sonicadvance1/remove_block_sampling
FEXCore: Remove BlockSamplingData
2024-09-22 19:18:37 -04:00
Ryan Houdek 9e4de3bfe8 Merge pull request #4079 from alyssarosenzweig/bug/maxss-nan
Fix (v)maxss on non-AFP platforms
2024-09-22 11:45:30 -07:00
Alyssa Rosenzweig 598b99fe58 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-22 14:06:27 -04:00
Alyssa Rosenzweig 9dda76ebe6 unittests: add min/max tests
we had a bug with max of nans. since x86 behaviour is weird, add a unit test to
try all the weird things. this fails on main without AFP

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-22 14:06:27 -04:00
Ryan Houdek f5def7ae1c Linux: Also optimize HandleNewClone stack usage
Drops from ~1392 bytes of stack usage to ~80 bytes
2024-09-22 10:51:25 -07:00
Ryan Houdek 58bcf691d6 Linux: Optimize CreateNewThread stack usage
We were creating a copy of the FEXCore::Core::CPUState object when we
didn't need to. We can pass the host thread's CPUState frame through to
the creation handlers since it's read-only (so modify it to be const).

We then just move the RAX and RSP setting to /after/ the CreateThread
handling instead of before.
This reduces stack usage from ~1392 bytes to ~80 bytes.
2024-09-22 10:45:21 -07:00
Alyssa Rosenzweig 33186b803d JIT: fix scalar fmax on non-AFP platforms
NaN and signed zero handling was busted.  Pretty nasty.

Fixes the following Vulkan CTS test run under emulation without thunking with a clang-built Mesa:

  dEQP-VK.spirv_assembly.instruction.graphics.float_controls.fp32.generated_args.denorm_nmax_nan_flush_to_zero_vert

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-22 12:14:21 -04:00
Alyssa Rosenzweig bb1d7d0750 JIT: rm broken dead code
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-22 12:14:21 -04:00
Ryan Houdek 4897c1e80c Config: Fixes stack usage in EnvLoader
EnvLoader::Load was generating strings on the stack and then copying
them to the FEXCore::Config state. FEXCore::Config::Set supports
string_views directly which will emplace in to the map directly.

This improves this functions stack usage from ~4464 bytes to ~288 bytes.
2024-09-22 08:45:02 -07:00
Ryan Houdek e7605c94dd SignalDelegator: Let the frontend inform AVX support
Since the frontend has changed to informing the backend if AVX is
supported, there is no reason to feed that configuration back in to
SignalDelegator from the backend.

Instead inform the SignalDelegator directly in the frontend instead of
this now weird round-about path.
2024-09-22 08:23:06 -07:00
Ryan Houdek ab5a10374b FEXCore: Remove BlockSamplingData
This was a legacy interface that has long lost its implementation and
has long lost its use. This interface was similar to an older
Dolphin-Emu interface that was useful in that project, but isn't really
useful in a world with hundreds of threads and processes running at the
same time.
2024-09-22 08:04:39 -07:00
Ryan Houdek dc2a26582f FEXCore/OpcodeDispatcher: Fixes missing static on some tables
This was causing egregious stack usage in these two functions
2024-09-16 18:52:55 -07:00
Ryan Houdek e6512fbe4d X86Tables: Remove function wraps from VEX tables 2024-09-16 18:52:55 -07:00
Ryan Houdek dbca441573 X86Tables: Remove function wraps from DDD table 2024-09-16 18:52:55 -07:00
Ryan Houdek 31adc953b7 X86Tables: Remove function wraps from H0F38/H0F3A table 2024-09-16 18:52:55 -07:00
Ryan Houdek 0dd7f5e2f7 X86Tables: Remove function wraps from SecondaryModRM table 2024-09-16 18:52:54 -07:00
Ryan Houdek 36556c4705 X86Tables: Remove function wraps from Group tables 2024-09-16 18:52:54 -07:00
Ryan Houdek e29fac3f25 X86Tables: Remove function wraps from Secondary tables 2024-09-16 18:52:54 -07:00
Ryan Houdek c984bdb42a OpcodeDispatcher: Remove previous template instantiantions and use Bind 2024-09-16 18:52:54 -07:00
Ryan Houdek 2645b374a7 OpcodeDispatcher: Deduplicate InstallToTable helper 2024-09-16 18:52:54 -07:00
Ryan Houdek 10192eebe5 X86Tables: Remove EVEX Table
Still track the telemetry if EVEX is attempted, but remove the tables
which we won't be supporting.
2024-09-16 17:40:20 -07:00
Ryan Houdek 0e57cbf5b9 OpcodeDispatcher: Remove EVEX table install
This hasn't been necessary for quite a while as we handle
invalid/unsupported opcodes in the frontend now.
2024-09-16 17:40:20 -07:00
Ryan Houdek c7413d96ad OpcodeDispatcher: Constexpr-ify VEX & VEXGroup tables 2024-09-16 17:40:20 -07:00
Ryan Houdek f3ba8cb33c OpcodeDispatcher: Constexpr-ify DDD tables 2024-09-16 17:40:20 -07:00
Ryan Houdek e714933b10 OpcodeDispatcher: Constexpr-ify H0F3A tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 9a0f83cbf6 OpcodeDispatcher: Constexpr-ify H0F38 tables 2024-09-16 17:40:20 -07:00
Ryan Houdek c4d30e8437 OpcodeDispatcher: Constexpr-ify SecondaryModRM tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 7b30df8a56 OpcodeDispatcher: Constexpr-ify SecondaryGroup tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 86aa459dd1 OpcodeDispatcher: Constexpr-ify PrimaryGroup tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 6c9a47fc71 OpcodeDispatcher: Constexpr-ify Secondary OpSizeMod tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 7a72cf631b OpcodeDispatcher: Constexpr-ify Secondary RepNEMod tables 2024-09-16 17:40:20 -07:00
Ryan Houdek 3d98eef7b8 OpcodeDispatcher: Constexpr-ify Secondary RepMod tables 2024-09-16 17:40:20 -07:00
Ryan Houdek f2011b0b79 OpcodeDispatcher: Constexpr-ify Secondary base tables 2024-09-16 17:40:19 -07:00
Ryan Houdek c0b8a0d9c3 Merge pull request #4066 from Sonicadvance1/remove_imgui
Tools: Delete imgui FEXConfig
2024-09-16 17:25:46 -07:00
Ryan Houdek 5bd0afa307 Merge pull request #4068 from Sonicadvance1/update_cpuid
CPUID: Update to something a little more modern
2024-09-16 06:33:04 -07:00
Ryan Houdek 20fb0da7d9 Merge pull request #4048 from Sonicadvance1/constexpr_more_tables
FEXCore: Convert Base tables over to constexpr
2024-09-16 06:32:23 -07:00
Ryan Houdek 35cb1f710c FEXQConfig->FEXConfig 2024-09-16 02:11:27 -07:00
Ryan Houdek 7c7b99f9c5 FEXConfig: Remove 2024-09-16 02:09:25 -07:00
Ryan Houdek 09879bd962 CPUID: Update to something a little more modern
Previously reported as some old CPU without AVX and SSE4 and other
things.
Start advertising as something more modern that actually shipped with
AVX2 and other features. Should help some modern libraries that do bad
family and model checks rather that CPUID features checks.

Also removes the silly `(ES)` tag from CPU-Z.

Also moves generation in to a constexpr function that can actually range
check these 4-bit and 8-bit values.
2024-09-16 01:42:35 -07:00
Ryan Houdek c8f3fe3788 Merge pull request #4071 from Sonicadvance1/unittests_incorrect_arg
unittests: Fixes incorrect argument
2024-09-15 15:20:54 -07:00
Ryan Houdek 88e3164b55 unittests: Fixes incorrect argument
Turns out our unittests have been passing an incorrect argument for
years.
2024-09-14 23:15:52 -07:00
Ryan Houdek bebd7400c0 Merge pull request #4063 from Sonicadvance1/fallback_if_no_zenity
FEXRootFSFetcher: Fallback to TTY if zenity isn't installed
2024-09-14 16:14:32 -07:00
Ryan Houdek e190d029dc Merge pull request #4064 from Sonicadvance1/remove_binfmt_flag
FEXLoader: Drop the binfmt_misc `I` flag
2024-09-13 12:51:24 -07:00
Ryan Houdek 37a70e2ec6 FEXCore: Convert Base tables over to constexpr
Only doing the single table for review purposes. Once reviewed I will
hammer out the remaining tables.

Similar to #3320, most of the OpcodeDispatcher tables can be consteval
and made to be a compile time constant. This just requires shuffling the
code slightly. The idea is to get almost all of the table setup out of
the `InstallOpcodeHandlers` function and instead only install the
handlers that change based on 32-bit or 64-bit, just like the x86 tables
we also did.
2024-09-13 11:39:09 -07:00
Ryan Houdek 6ff073ac11 FEXRootFSFetcher: convert vector usage over to array and span
Most uses of vector could actually be converted to array and a lot of
argument passing could actually be converted over to spans instead.
2024-09-13 11:29:44 -07:00
Ryan Houdek b6e4c47abc FEXLoader: Drop the binfmt_misc I flag
Upstream has rejected this flag which would have let FEX close the gap
in functionality between binfmt_misc interpreters and PT_INTERP
interpreters for how `/proc/exe` is handled. Since we are unable to
change their opinions, just remove the code from FEX since it's never
going to happen.

Removes a little bit of confusing behaviour in execveat and
filemanagement handling. Leaving us with only the regular confusing
behaviour of trying to track accesses to `/proc/exe` instead.
2024-09-12 19:58:35 -07:00
Ryan Houdek daf8b409f3 FEXRootFSFetcher: Fallback to TTY if zenity isn't installed
If packaged improperly or if someone has just did a source install then
they may not have zenity. Fallback to TTY path if zenity isn't
installed.
2024-09-11 17:08:20 -07:00
Ryan Houdek 0a35029d9a Merge pull request #4062 from Sonicadvance1/fix_fexserver_writing_to_pipe
FEXServer/PipeScanner: Stop writing null to incoming pipes
2024-09-11 13:59:24 -07:00
Ryan Houdek e8b51ac0a4 FEXServer/PipeScanner: Stop writing null to incoming pipes
This was causing problems in a typical use case of piping stdout/stderr
to another application. So if FEXInterpreter was started, with stdout
piped to another application, and FEXServer wasn't running so a new
instance needed to be started. FEXServer would write a null to every
pipe then close it. This includes the stdout that is being piped to the
new application!

This writing was legacy behaviour before FEX was properly listening to
POLLHUP and is no longer necessary. Just remove the writing completely
to fix this issue.

Easiest test case was just `./Bin/FEXInterpreter `which ls` / | xxd | head -n 3`

Returned:
```
$ ./Bin/FEXInterpreter `which ls` / | xxd | head -n 3
00000000: 0000 0000 6269 6e0a 6269 6e2e 7573 722d  ....bin.bin.usr-
00000010: 6973 2d6d 6572 6765 640a 626f 6f74 0a64  is-merged.boot.d
00000020: 6576 0a65 7463 0a68 6f6d 650a 6c69 620a  ev.etc.home.lib.
```

That first null shouldn't be there.
2024-09-11 12:52:57 -07:00
Ryan Houdek f20e626fda Merge pull request #4059 from bylaws/log
Windows: Don't assume the log file was successfully opened
2024-09-11 10:16:14 -07:00
Ryan Houdek 2672e06aa5 Merge pull request #4058 from bylaws/cef
ARM64EC: Patch the call checker before calling any syscall exports
2024-09-11 10:16:02 -07:00
Ryan Houdek 4a2b4be59d Merge pull request #4057 from bylaws/overcommit
Support for emulated overcommit on Windows
2024-09-11 10:15:43 -07:00
Ryan Houdek 05be9445cd Merge pull request #4056 from Sonicadvance1/context_remove_unused_features
FEXCore/Context: Removes unused features
2024-09-11 10:09:24 -07:00
Ryan Houdek 4cb4c0e277 Merge pull request #4055 from Sonicadvance1/remove_header
FEXCore: Remove Context header include when unneeded
2024-09-11 10:09:14 -07:00
Billy Laws 0135e2d78c LookupCache: Use emulated overcommit if necessary
Allocates uncommited memory and calls the frontend callbacks so it can
commit pages on faults as necessary.
2024-09-10 16:08:45 +00:00
Billy Laws 1266a5a5ce Windows: Support emulated memory overcommit 2024-09-10 16:08:45 +00:00
Billy Laws 7255850e9d ARM64EC: Patch the call checker before calling any syscall exports
CEF on Windows patches these before FEX is even loaded, so this needs to
be setup as early as possible using some direct syscalls which skip any
such patches.
2024-09-10 16:05:37 +00:00
Billy Laws 002a8bebd8 Windows: Cleanup exception handling logic 2024-09-10 16:05:02 +00:00
Billy Laws 452c9fd19b AllocatorHooks: Make commiting allocated memory optional
Ignored for linux as it supports overcommit.
2024-09-10 16:05:02 +00:00
Billy Laws 0b0e15c9b4 Windows: Don't assume the log file was successfully opened 2024-09-10 15:58:03 +00:00
Ryan Houdek ae9db336e7 FEXCore/Context: Removes unused features
No functional change here.
- CoreRunningMode enum and variable wasn't used anymore.
   - Code was moved to the frontend
- CustomCPUFactory wasn't used anymore
   - All special signal handling and various features were moved to
     TestHarnessRunner
   - We also don't want to support actual custom CPU cores.
   - TestHarnessRunner just runs as a host runner if compiled on an
     x86-64 device if vixl sim isn't enabled now.
   - Removes the Core config option entirely.
- Moves VDSOPointers struct to the frontend
   - Every use of this lives in the Linux frontend instead now
2024-09-09 18:38:33 -07:00
Ryan Houdek 360ea538b7 Merge pull request #4054 from Sonicadvance1/fix_xxhash
FEXRootFSFetcher/XXHash: Fix double fd close
2024-09-09 18:03:55 -07:00
Ryan Houdek f99900bdc5 Merge pull request #4053 from Sonicadvance1/disable_vixl_compiling
CMake: Disable vixl compiling if not enabled
2024-09-09 18:03:46 -07:00
Ryan Houdek 34f2bce203 FEXCore: Remove Context header include when unneeded
This is one of the most expensive headers in FEX, averaging 1.2 seconds
of compile time per include.
We include this header 31 times around the project. Remove
the five instances where it is unused to help this issue. Next step
would be to make the header lighter if possible.
2024-09-09 17:43:35 -07:00
Ryan Houdek a05d6ae082 FEXRootFSFetcher/XXHash: Check for errors in two locations that were missed 2024-09-09 14:07:23 -07:00
Ryan Houdek 6a4eb434f7 FEXRootFSFetcher/XXHash: Fix double fd close
Since `HadError` is a lambda that handles FD closing this was a double
FD close. Just move the xxhash state freeing in there as well and remove
the FD and hash freeing. Fixes the double free.
2024-09-09 14:00:44 -07:00
Ryan Houdek 0c3eb41f57 CMake: Disable vixl compiling if not enabled
Like how FEXCore disabled vixl linking, make sure we aren't accidentally
compiling and including it when vixl stuff is disabled. Noticed this in
the build logs.
2024-09-09 13:54:35 -07:00
Ryan Houdek e4bfbda008 VDSOEmulation: Stop using dlopen for VDSO
We have been relying on the
`YesIKnowImNotSupposedToUseTheGlibcAllocator` fault avoidance for a long
while now. The plan was to rewrite the symbol fetching in the future to
avoid this since VDSO is kind of special anyway.

That time is now, I have had this VDSO symbol parsing code living in a
different project for a few months now and it is working great.

The basic things here are that the Linux kernel provides the VDSO
mapping base pointer through the auxv value `AT_SYSINFO_EHDR`. We then
need a minimal ELF parser that /only/ parses the dynamic symbol header
(and accompanying string header).

Then it's a simple case of walking the symbol table and recording the
pointers. Confirmed this fetches all the correct symbols (As I've been
using it for a while already I already knew it worked.)

This lets us stop allocating memory through the glibc allocator due to
dlopen.
2024-09-09 13:26:06 -07:00
Ryan Houdek b4c797c199 Merge pull request #4052 from Sonicadvance1/various_warnings
Bunch of little things reported by coverity
2024-09-09 12:54:22 -07:00
Ryan Houdek ebae1bf71d Merge pull request #4051 from Sonicadvance1/fix_warning
Arm64: Fix warning
2024-09-09 08:24:48 -07:00
Ryan Houdek fa293e5e8f GDBServer: Fix buffer overrun in x87 register copying 2024-09-08 20:24:17 -07:00
Ryan Houdek 720c89e648 GDBServer: Make it clear we're copying all 512-bytes to clear a warning 2024-09-08 20:23:03 -07:00
Ryan Houdek abc5f5abd4 GDBServer: Use std::move for move assignment 2024-09-08 20:22:39 -07:00
Ryan Houdek 22ab16b217 Linux/x32/ipc: Fixes version check for msgrcv
Also adds an EINVAL return if version 1 is attempted with SHMAT,
mirroring what the kernel returns.
2024-09-08 20:18:11 -07:00
Ryan Houdek b80f05dd86 Seccomp/BPFEmitter: Reorder check to avoid integer overflow warning 2024-09-08 20:13:14 -07:00
Ryan Houdek 1642a0f4d9 Seccomp/BPFEmitter: Initialize Func,FuncSize just to make coverity happy 2024-09-08 20:10:44 -07:00
Ryan Houdek 96e990dade SignalDelegator: Use const auto ref to avoid copy 2024-09-08 20:07:45 -07:00
Ryan Houdek 3423a12ab6 SignalDelegator: Remove unused function copying for delegators 2024-09-08 20:06:38 -07:00
Ryan Houdek c4a0fb6609 Ioctl/drm: Fix missing member copy 2024-09-08 20:02:51 -07:00
Ryan Houdek 8cf23eba9c FEXLoader/ELFCodeLoader: Use const ref auto to avoid copy 2024-09-08 20:02:51 -07:00
Ryan Houdek fbe817f1c4 Config: Use std::move on strings to avoid copy 2024-09-08 20:02:51 -07:00
Ryan Houdek 3b61394548 FEXRootFSFetcher: Fixes some extract logic
`Extract` was never set to false even if the user said not to overwrite.
Ensure that if the user decided not to overwrite an existing folder,
that it returns false so it correctly doesn't overwrite the folder.
2024-09-08 20:02:51 -07:00
Ryan Houdek 8ed6b36181 FileManagement: Use const ref for AppConfigName 2024-09-08 20:02:51 -07:00
Ryan Houdek 9141322666 Ioctl/drm: Adds missing flags member copying 2024-09-08 20:02:51 -07:00
Ryan Houdek 2d91c5441e Ioctl/Radeon: Adds missing func member copying 2024-09-08 20:02:51 -07:00
Ryan Houdek 2ff1589c35 FileManagement: Use const ref for Filename 2024-09-08 20:02:51 -07:00
Ryan Houdek a308b9edbe Linux/GenerateMap: Fixes FD leak
This FD was unused
2024-09-08 19:34:38 -07:00
Ryan Houdek f6a8e595df Linux/Syscalls: Add todo for futimesat 2024-09-08 19:34:38 -07:00
Ryan Houdek eaaf62e4b7 X87: Make it a bit clearer that 80-bit floats can't hit the memory path
Makes the static analysis happy.
2024-09-08 18:03:20 -07:00
Ryan Houdek dc5fc57e19 FEXServer: Check if prctl errors 2024-09-08 18:03:20 -07:00
Ryan Houdek c3c0bbb060 F80Fallbacks: Fix Uninitialized variable warning that can't occur 2024-09-08 18:03:20 -07:00
Ryan Houdek 98a91d242d FEXRootFSFetcher: Fixes resource leak on XXHash error 2024-09-08 18:03:20 -07:00
Ryan Houdek 8da6b1e729 FEXCore/Allocator: Fixes uninitialized scalar variable warning that can't occur 2024-09-08 18:03:20 -07:00
Ryan Houdek d260bce31e FEXRootFSFetcher: Fixes uninitialized variable 2024-09-08 18:03:20 -07:00
Ryan Houdek a94aaca0ec OpcodeDispatcher: Add default FEX_UNREACHABLE
This can't happen but if it changes then make sure we capture it.
2024-09-08 18:03:20 -07:00
Ryan Houdek 2ac3ca3ead pidof: Fix uninitialized variable warning 2024-09-08 18:03:20 -07:00
Ryan Houdek c559445b62 Linux/IoctlEmulation: Fixes incorrect direction copy for a few ioctls
Plus convert remaining handlers over to using CPYF/CPYT handlers so
ensure this doesn't happen again.
2024-09-08 18:03:20 -07:00
Ryan Houdek 8ea4e4f663 Linux: Adds some missing brace initializers on conversion operators 2024-09-08 18:03:20 -07:00
Ryan Houdek 2da700fe82 CodeEmitter: Removes some ternaries that are unnecessary
NFC
2024-09-08 18:03:20 -07:00
Ryan Houdek 0a32ba9b29 instcountCI: Fixes for shifts 2024-09-08 18:03:20 -07:00
Ryan Houdek f51832ca6b FEXCore: Fixes a bug with VPSRLDQ/VPSLLDQ with >= 16-byte shifts
When the shift amount is >= 16-bytes then we need to zero the register.
We had a bug where we were assigning `Result.High` to itself, which
effectively made the top 128-bits of the ymm register not modify itself.

Adds a unit test to ensure that doesn't happen again.
2024-09-08 15:39:22 -07:00
Ryan Houdek 5f9a30af7a FileManagement: Use const ref for RootFSPath
No need to make a copy
2024-09-08 15:20:04 -07:00
Ryan Houdek 453ef0b94c Linux/ExecveHandler: Move a path rather than copy
We can do move assignment instead of copy assignment here.
2024-09-08 15:17:04 -07:00
Ryan Houdek 4779e64cf5 Linux/EmulatedFiles: Check return for seal
This shouldn't be possible to occur, but have a log message just in
case.
2024-09-08 15:15:38 -07:00
Ryan Houdek e6025cc087 Arm64: Fix warning
This variable is only used in a log message no, so move it there.
2024-09-08 11:15:46 -07:00
Ryan Houdek cd4c224dc0 Merge pull request #4050 from alyssarosenzweig/sra/move-size
JIT: always use 64-bit moves for SRA
2024-09-08 10:30:17 -07:00
Ryan Houdek 07f117b957 Merge pull request #4047 from bylaws/arm64ec-sysv2
ARM64EC: Switch to exception-based native syscall dispatch
2024-09-08 10:30:05 -07:00
Ryan Houdek 36e4f9a070 Merge pull request #4049 from alyssarosenzweig/opt/constprop-merges
ConstProp: speed it up
2024-09-08 10:13:23 -07:00
Alyssa Rosenzweig 67c751d3e5 JIT: always use 64-bit moves for SRA
I see no reason not to. Simpler code and it's slightly faster on Firestorm.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 13:02:57 -04:00
Ryan Houdek 1c59bfeb9f Merge pull request #4027 from alyssarosenzweig/opt/global-flag
Global flag optimizations
2024-09-08 09:42:18 -07:00
Alyssa Rosenzweig 2b0041a291 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:27:20 -04:00
Alyssa Rosenzweig 39cc1e5116 ConstProp: merge folding+inlining
2% off nodejs

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Alyssa Rosenzweig f46856c090 ConstProp: template constant inlining
DRY

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Alyssa Rosenzweig b7859052da ConstProp: don't mask shifts
done in the emitter now

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Alyssa Rosenzweig 3b30d8103f ConstProp: combine like cases
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Alyssa Rosenzweig d19fc36f8f ConstProp: delay constant pooling
more efficient to pool the output of constant folding and such. but we need to
avoid that becoming accidentally quadratic, so rework that too

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Alyssa Rosenzweig 5fbf76ee9e CodeEmitter: mask immediate shifts
more convenient for constprop and not really any more expensive.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-08 10:26:58 -04:00
Ryan Houdek 5f79761fe4 Merge pull request #4046 from Sonicadvance1/cpubackend_remove_unused
CPUBackend: Remove unused functions
2024-09-07 13:11:32 -07:00
Billy Laws 621af9e7e8 ARM64EC: Switch to exception-based native syscall dispatch
The previous approach assumed that ntdll wouldn't have been patched
before FEX was loaded, but that isn't necessarily the case thanks to
CEF. This takes the same approach used by xtajit which wine recently gained
support for wine. The overhead to this isn't ideal but most syscalls
aren't directly done through x86 code anyway and in the future FEX could
forward unpatched thunks to their ARM variants at JIT time.
2024-09-07 15:30:00 +00:00
Billy Laws f4d107c8f0 ARM64EC: Split out FEX -> packed EC context conversion from reconstruction 2024-09-07 15:29:48 +00:00
Ryan Houdek 59643db331 CPUBackend: Remove unused functions
Some of these were bad ideas, some ideas were something we effectively
grew out of.
2024-09-07 08:07:00 -07:00
Alyssa Rosenzweig 46f36dc89d InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:41 -04:00
Alyssa Rosenzweig 384882744c DeadStoreElimination: drop now that it's redundant
it's only really load bearing for pf/af, which is handled as a global flag opt
now. this mitigates some of the compile time hit from globalizing flag opts.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig b36d1f7e7b RedundantFlagCalculationElimination: optimize parity
another global CFG-based optimization -- if we know that the raw PF is already
1-bit we can skip parity evaluation, saving work with floating point compares.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig dd7c66db2e RedundantFlagCalculationElimination: use LUT for flags
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 47ac9edd7a RedundantFlagCalculationElimination: select testz
this saves uops.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 6e6d640fec RedundantFlagCalculationElimination: fold compares/axflag into branches
now that we know whether flags are killed on the edge, we can improve branch
isel

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig eb88366614 RedundantFlagCalculationElimination: globalize
Gather a control flow graph and use it to propagate flags throughout the
program.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 50d6ddd591 RedundantFlagCalculationElimination: extract per-block logic
nfc

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 249de7c758 RedundantFlagCalculationElimination: drop unneeded bools
nonzero <==> not dummy

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig c350888d70 RedundantFlagCalculationElimination: add missing adczero case
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig f23bef1653 RedundantFlagCalculationElimination: add missing testnz case
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig d17f33a922 OpcodeDispatcher: don't emit fake 0 for condjump
not needed and getting in the way

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 50a3ca0d6d IR: introduce dedicated PF/AF instructions
this makes reasoning about them a little easier, e.g. for flags.  about 1% win
in nodejs.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 10:59:05 -04:00
Alyssa Rosenzweig 8745455a5b IR: track whether parity is read
so we can gate optimizations efficiently

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Alyssa Rosenzweig e9ab514962 IR: push parity evaluation down
so we can optimize it globally

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Alyssa Rosenzweig 4eb0948451 IR: push down AXFLAG lowering
so we can get the new axflag optimizations on billy's x13s.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Alyssa Rosenzweig 8d9f19bd73 IR: add TestZ op
more optimized than TestNZ if we don't care about the sign bit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Alyssa Rosenzweig 9fe7bc5818 InstCountCI: add ucomiss+pf opt case
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Alyssa Rosenzweig 5a590a9b11 InstCountCI: add 8-bit test cases
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-07 08:26:04 -04:00
Ryan Houdek a4acd64246 Merge pull request #4042 from neobrain/refactor_system_libs
CMake: Compile with system libraries for xxhash, Catch2, and fmt, if available
2024-09-06 11:48:12 -07:00
Ryan Houdek 304b5de1af Merge pull request #4038 from Sonicadvance1/move_aotir_debugdata
AOTIR: Move debugdata structure to internal header
2024-09-06 10:50:42 -07:00
Ryan Houdek ca2fe0b301 AOTIR: Move debugdata structure to internal header
This definition doesn't need to be exposed in the public API.
2024-09-06 10:19:45 -07:00
Ryan Houdek bbef4d762c Merge pull request #4044 from bylaws/int2e
OpcodeDispatcher: Do not forbid INT 2E syscalls on 64-bit Windows
2024-09-06 10:18:18 -07:00
Ryan Houdek f77841d784 Merge pull request #4041 from Sonicadvance1/fix_poll
LinuxSyscalls: With poll syscall, ensure fds is writable only if nfds is not zero
2024-09-06 10:17:55 -07:00
Ryan Houdek 219a4777c6 Merge pull request #4040 from Sonicadvance1/delete_threadsstate
FEXCore: Delete ThreadsState struct
2024-09-06 10:16:28 -07:00
Ryan Houdek 29f0e9b4d6 Merge pull request #4039 from Sonicadvance1/move_customir_entry
FEXCore: Move `CustomIRResult` to internal header
2024-09-06 10:16:18 -07:00
Ryan Houdek b75efc42bf Merge pull request #4037 from Sonicadvance1/move_symbol
FEXCore: Move `JITSymbolBuffer` to internal header
2024-09-06 10:15:54 -07:00
Tony Wasserka 90dbd4766d Merge pull request #4043 from alyssarosenzweig/gitignore/build
gitignore: ignore build symlink
2024-09-06 18:08:47 +02:00
Billy Laws f7b911ca43 OpcodeDispatcher: Do not forbid INT 2E syscalls on 64-bit Windows
This works fine on real Windows and is relied on by wine as SystemCall
is set to 1 in KUSER_SHARED_DATA, which causes the ntdll thunks to use
it over `syscall`
2024-09-06 15:58:17 +00:00
Alyssa Rosenzweig 6de0333708 gitignore: ignore build symlink
I symlink build -> Build for YouCompleteMe.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-06 11:22:31 -04:00
Tony Wasserka ab5d3ab22b CMake: Don't use pkgconfig when cross-compiling 2024-09-06 10:30:48 +02:00
Tony Wasserka 355c3428c0 CMake: Don't use system libraries when compiling with mingw 2024-09-06 10:30:24 +02:00
Tony Wasserka c76b7bfe8d CMake: Compile with system libraries for xxhash, Catch2, and fmt, if available 2024-09-06 09:47:49 +02:00
Ryan Houdek 64c5362580 LinuxSyscalls: With poll syscall, ensure fds is writable only if nfds is not zero
The pointer is allowed to be null or garbage if the number of nfds
passed in is zero.
Unify the x86 and x86-64 implementations to ensure consistency.

Fixes Warhammer 40k: Relics of War
2024-09-05 14:33:52 -07:00
Ryan Houdek 77469de247 FEXCore: Delete ThreadsState struct
All uses of this have been removed.
2024-09-05 13:39:02 -07:00
Ryan Houdek 48fd827004 FEXCore: Move CustomIRResult to internal header
This definition doesn't need to be exposed in the public API.

Stopped needing to be public once we removed our IR CI thing.
2024-09-05 13:36:15 -07:00
Ryan Houdek f09d511ac8 FEXCore: Move JITSymbolBuffer to internal header
This definition doesn't need to be exposed in the public API.
2024-09-05 13:28:45 -07:00
Ryan Houdek d5db8948db Docs: Update for release FEX-2409 2024-09-05 12:43:45 -07:00
Ryan Houdek 5013b8a0db Merge pull request #4036 from alyssarosenzweig/opt/dont-inline-pool
ConstProp: stop pooling inline constants
2024-09-05 07:30:39 -07:00
Alyssa Rosenzweig ac65deed6c ConstProp: drop CreateInlineConstant
bit of cleanup

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-05 09:48:59 -04:00
Alyssa Rosenzweig d1d4d2d876 ConstProp: stop pooling inline constants
Because we already pooled the _Constants, there's no benefit to also pooling inline constants. the robin map to do so just adds extra overhead for no benefit - drop it.

without multiblock, shaves around 2% off node. with multiblock, a bit less than
1% but still a win.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-05 09:47:35 -04:00
Ryan Houdek e234e118e0 Merge pull request #4035 from neobrain/fix_fexqonfic_polish
FEXQonfig: Fix minor saving/loading quirks
2024-09-05 06:18:51 -07:00
Tony Wasserka 1cc54312fa FEXQonfig: Recognize file: URLs in addition to file://
QUrl::fromLocalFile produces this format for relative paths.
2024-09-05 12:54:55 +02:00
Tony Wasserka de9ab6a023 FEXQonfig: Fix loading local config files by command line argument
ParentPath doesn't return a usable value for "Config.json", so just use
std::filesystem::absolute/parent_path instead.
2024-09-05 12:53:44 +02:00
Tony Wasserka 463a0b5ed4 FEXQonfig: Set a default save location when Config.json does not exist 2024-09-05 12:37:01 +02:00
Tony Wasserka 7b0365b377 FEXQonfig: Reset dirty indicator after loading a file 2024-09-05 12:36:05 +02:00
Tony Wasserka 803501526e FEXQonfig: Fix false "Failed to load config file" error when saving to a new file 2024-09-05 12:36:05 +02:00
Tony Wasserka a63a3a47e3 FEXQonfig: Clearly indicate save dialogs as such 2024-09-05 12:36:05 +02:00
Ryan Houdek a66fac614b Merge pull request #4034 from alyssarosenzweig/fix-tied-fma
IR: fix scalar FMA tied sources
2024-09-04 09:14:24 -07:00
Alyssa Rosenzweig 6d4693cbc1 IR: fix scalar FMA tied sources
needs to be modelled explicitly or else we lose information when translating

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-09-04 07:42:44 -04:00
Ryan Houdek 46a2a0608b Merge pull request #4033 from Sonicadvance1/dynamic_scale_support
FEXCore: Dynamically scale TSC
2024-09-03 16:11:52 -07:00
Ryan Houdek 49b40b71a6 Merge pull request #4030 from bylaws/eccfg
Windows: Load per-application configs
2024-09-03 16:11:42 -07:00
Ryan Houdek ca8f347902 InstcountCI: Skip rdtsc and rdtscp
These will differ depending on which runner they are running on. Just
disable them but keep the instructions around so show how bad rdtscp is,
but how good rdtsc is.
2024-09-03 13:50:43 -07:00
Ryan Houdek e415b9443f Merge pull request #4032 from bylaws/small
Windows: Fix missing pragma and license text
2024-09-03 13:47:43 -07:00
Ryan Houdek fd4f6b8020 FEXCore: Dynamically scale TSC
When I implemented TSC scaling originally, I chose a scale factor of 128
because it basically covered the range of devices we cared about without
going too high. I also only tested devices that had a TSC scale factor
from 19.2Mhz to 34Mhz. Turns out there is hardware that also has a 48Mhz
cycle counter, which cause them to effectively have a 6.1Ghz cycle
counter, which is kind of absurd.

Instead of a fixed scale, just calculate the amount of scaling we need
to get >= the minimum threshold of 1Ghz. This will change the shift from
7 to 5 or 6 for the faster cycle counter devices.

Of course if someone wants to know the scale factor they can still use
cpuid function 15h to know it.

Fixes #4026
2024-09-03 13:41:31 -07:00
Ryan Houdek 7a9eb01573 Merge pull request #4031 from bylaws/midr
Windows: Fill in per-core MIDR information
2024-09-03 13:40:30 -07:00
Billy Laws b0474816c7 Windows: Fix missing pragma and license text 2024-09-03 20:30:07 +00:00
Billy Laws 941b065da6 Windows: Fill in per-core MIDR information 2024-09-03 20:29:37 +00:00
Billy Laws ccba993ca9 Windows: Load per-application configs 2024-09-03 20:25:21 +00:00
Billy Laws e664f61da8 Config: Split out command line parsing from config init 2024-09-03 20:25:21 +00:00
Ryan Houdek c74df6a59b Merge pull request #4029 from neobrain/refactor_fexqonfic_followups
FEXQonfig: Minor followup changes
2024-09-03 11:53:01 -07:00
Tony Wasserka fc76d4e8c3 CMake: Suppress warning about Qt 6 not being found 2024-09-03 16:48:22 +02:00
Tony Wasserka 099211f39c FEXQonfig: Tune padding a little
The leftPadding on the radio buttons wasn't needed. Adding a little bit
of padding below them makes the grouping visually more clear however.
2024-09-03 16:48:21 +02:00
Tony Wasserka b327d7ae19 FEXQonfig: Drop superfluous include 2024-09-03 16:48:21 +02:00
Tony Wasserka 0e6a22febe FEXQonfig: Add missing license identifiers 2024-09-03 16:48:21 +02:00
Alyssa Rosenzweig b368223d50 Merge pull request #3628 from Sonicadvance1/seccomp
LinuxEmulation: Implement support for seccomp
2024-09-03 08:49:16 -04:00
Ryan Houdek 8fe1e9562d Merge pull request #4025 from Sonicadvance1/split_lock_config_fexqconfig
FEXQConfig: Add strict split-lock option
2024-09-02 14:44:15 -07:00
Ryan Houdek ae545b8cf3 FEXQConfig: Add strict split-lock option
This was missed in the initial FEXQConfig implementation since it was
implemented at the same time.
2024-09-02 14:11:04 -07:00
Ryan Houdek ac32876e4e LinuxEmulation: Implement support for seccomp
Seccomp is a relatively complex feature that was added to Linux back in
2005, and was further extended in 2013 to support BPF based protections.
Once seccomp is enabled, you can no longer disable seccomp but
additional protections can be placed on top of existing seccomp filters.
Additionally seccomp filters are inherited in child processes, which
ensures the process tree can't escape from the secure computing
environment through child processes.

The basis of this feature is a shim that lives between userspace and the
kernel at the syscall entrypoint.
In "strict" mode, seccomp only allows read, write, exit, exit_group, and {rt_,}sigreturn to function.
When in "filter" mode, a BPF filter is run on syscall entrypoint and
returns state about if the syscall should be allowed or not. Multiple
filters can be installed in this mode, all of which get executed. The
result that is the most restricted is the action that occurs at the end.

There are some significant limitations in filter mode that must be
adhered to which makes executing this code inside of kernel space a
non-issue and effectively limits how much cpu time is spent in the filters.
Although these filters are free to do basically anything with the
provided data, just can't do any loops.

FEX needs to implement seccomp because there are multiple applications
using the feature, the primary one being Chromium which some games embed
without disabling the sandbox. WINE also uses seccomp for capturing
games that do raw Windows system calls. Apparently Red Dead Redemption
is one of the games that requires this.

While FEX implements seccomp, it is not yet all encompassing, which is
one of the reasons why it isn't enabled by default and requires a config
option.

**seccomp_unotify is not implemented**
This is a relatively new feature for seccomp which lets the seccomp
filter signal an FD for multiple things. Luckily Chromium and WINE don't
use this. This will be tricky to implement under FEX since it
requires ioctl trapping and some other behaviour

**ptrace isn't supported**
One feature of seccomp is that it can raise ptrace events. Since FEX
doesn't support ptrace at all, this isn't handled. Again Chromium and
WINE don't use this.

**kill-thread not quite correct**
This isn't directly related to seccomp but more about how we do thread
shutdown in FEX. This will require some more changes around thread state
tracking before fully supporting this. Chromium and WINE don't use this.
kill-process also falls under this

Features that are supported:
- Strict mode and seccomp-bpf mode supported
- All BFP instructions that seccomp-bpf understands
- Inheriting seccomp through execve
   - This means we serialize and deserialize the calling thread's
     seccomp filters
   - An execve that escapes FEX will also escape seccomp. Not much we
     can do about it
- TSync - Allowing post-mortem seccomp insertion which allows threads to
  synchronize seccomp filters after the fact

Features that are not supported:
- Different arch qualifiers depending on syscall entrypoint
  - Just like our syscall handler, we are hardcoded to the arch that the
    application starts with
- user_notif
- ptrace
- Runtime code cache invalidation when seccomp is installed
  - Currently we must ensure all syscalls go through the frontend
    syscall handler
  - Runtime invalidation of code cache with inline syscalls will get
    fixed in the future.

This currently isn't enabled by default because of the minor feature
problems that haven't been resolved. Currently the Linux Kernel's test
application works for the features that FEX supports, and WINE's usage
can be handled by FEX. Chromium's sandbox doesn't yet work with this PR,
but it only fails due to features unrelated to seccomp.

Having this open for merging now so we can work to resolve the remaining
issues without this bitrotting.
2024-09-02 14:07:53 -07:00
Ryan Houdek 9336e35052 Merge pull request #3929 from Sonicadvance1/portable
FEXInterpreter: Support portable installs
2024-09-02 10:13:07 -07:00
Ryan Houdek 0754affb98 Merge pull request #4024 from Sonicadvance1/fix_vdso_on_vdso_host
VDSO: Fixes a pretty nasty bug where we were never using the host VDSO
2024-09-02 09:49:24 -07:00
Ryan Houdek c413d7950b FEXInterpreter: Support portable installs 2024-09-02 09:37:21 -07:00
Ryan Houdek f8eaf9c14f VDSO: Fixes a pretty nasty bug where we were never using the host VDSO
This must have happened during a refactor or something, but since we're
making a copy of the function pointers, it would have only gotten the
version /prior/ to loading host VDSO symbols.

Moves the VDSO thunk definition setting to the end after VDSO symbol
definitions in order to get host VDSO symbols working again.
2024-09-02 09:18:37 -07:00
Ryan Houdek fc677eaabf CMake: Update minimum clang version to 13
Seccomp emulator uses lambda expressions in an unevaluated operand,
which was only added in clang-13
2024-09-02 07:38:23 -07:00
Alyssa Rosenzweig 114112a716 Merge pull request #3997 from Sonicadvance1/refactor_signal_information
SignalDelegator: Refactor how thread local data is stored
2024-09-02 10:14:52 -04:00
Ryan Houdek 9056d9b9de SignalDelegator: Refactor how thread local data is stored
Two primary things here:
- Remove the static `GlobalDelegator`
- Move the thread_local SignalDelegator::ThreadState information
  directly in to ThreadStateObject

Having the ThreadStateObject and the SignalDelegator information
disjoint was confusing but was required when we didn't have any object
in the frontend that could have its own independent data. Since we fixed
this with the `ThreadStateObject` type we can now move this over.

The `GlobalDelegator` object is now instead stored in
`ThreadStateObject` instead.

Instead of using a thread_local variable, we now just consume 8-bytes of
the signal alt-stack since the kernel gives us that information about
where it lives. This then converts all the thread_local usage to use
either the passed in CPU state if it exists, or fetching it from the
alt-stack offset.

Very minor changes in behaviour here, will help when trying to improve
FEX's behaviour around signals.
2024-09-02 06:46:19 -07:00
Alyssa Rosenzweig 74e95df661 Merge pull request #3974 from Sonicadvance1/strict_inprocess_splitlocks
Arm64: Implement support for strict in-process split-locks
2024-09-02 09:26:24 -04:00
Alyssa Rosenzweig d9544e7e02 Merge pull request #4018 from Sonicadvance1/move_guest_frame_manage
LinuxEmulation: Moves guest signal frame generation to its own file
2024-09-02 09:21:06 -04:00
Alyssa Rosenzweig 62e1767ee0 Merge pull request #4019 from Sonicadvance1/more_efault_handlers
More EFAULT handlers
2024-09-02 09:20:55 -04:00
Alyssa Rosenzweig 4baeffe84f Merge pull request #4022 from Sonicadvance1/move_sigreturn_to_frontend
FEX: Moves sigreturn symbols to frontend
2024-09-02 09:20:24 -04:00
Ryan Houdek b4a67a6178 Merge pull request #4021 from Sonicadvance1/remove_global_initializer_thunks
Thunks: Removes global static initializer
2024-08-31 22:54:22 -07:00
Ryan Houdek 8296bfc7de Merge pull request #4023 from chadmed/apple-partnums
CPUID: add missing Apple core part numbers
2024-08-31 19:45:30 -07:00
James Calligeros f588304b12 CPUID: add missing Apple core part numbers
The Ultra-class SoCs are two Max-class SoCs connected via
Apple's fabric, and thus use the same core revisions as the
Max-class SoCs for both big and LITTLE cores.

Signed-off-by: James Calligeros <jcalligeros99@gmail.com>
2024-09-01 12:14:39 +10:00
Ryan Houdek c748dbf0e3 FEX: Moves sigreturn symbols to frontend
These are a Linux construct and should live here. Removes a weird
passthrough API from FEXCore and keeps it in the frontend instead.
This isn't even typically allocated in a real setup, as it's only a
fallback for if VDSO isn't loaded.

The CallbackReturn function stays in FEXCore because it would have
caused an API in the other direction instead.
2024-08-31 07:43:04 -07:00
Mai 06497fdfad Merge pull request #4020 from Sonicadvance1/add_cross_compile_toolchain
CMake: Adds an AArch64 cross-compile toolchain file
2024-08-31 05:04:41 -04:00
Ryan Houdek e2a7fef742 Thunks: Removes global static initializer
This variable can't be constexpr initialized since it requires linker
fix-ups, which changes it in to a global static initializer instead.

While this benign as it doesn't allocate any memory, just move it next
to its single use. Removes the static initializer that was there for no
reason.
2024-08-31 01:58:35 -07:00
Ryan Houdek 17692d6eef CMake: Adds an AArch64 cross-compile toolchain file
This is useful for some people.

This doesn't allow building thunks due to the cmake limitation about
tools binaries also getting cross compiled, we need to switch to meson
if we want that, since they recently started fixing this
https://github.com/mesonbuild/meson/pull/12022

But that's an okay limitation for now.
2024-08-31 00:38:59 -07:00
Tony Wasserka 3020a0db2b Merge pull request #3982 from neobrain/feature_fexqonfic
Add Qt-based config editor
2024-08-30 10:43:07 +02:00
Ryan Houdek 92ddc0041b Merge pull request #4003 from Sonicadvance1/shortcircuit_invalid_inst
Frontend: short-circuit code generation on invalid instructions with multiblock
2024-08-29 04:52:14 -07:00
Tony Wasserka 8a5388f514 CMake: Build Dear ImGui only if needed 2024-08-29 12:14:06 +02:00
Tony Wasserka a444db7dac FEXQonfig: Use consistent orthography 2024-08-29 11:53:15 +02:00
Tony Wasserka 74da2eb5fc FEXQonfig: Fix unnecessary prompts for file selection on save 2024-08-29 11:53:15 +02:00
Tony Wasserka 539d5ed26f FEXQonfig: Fix Qt6 runtime-warnings caused by unannotated return types 2024-08-29 11:53:15 +02:00
Ryan Houdek cf7ee98831 x32/Thread: Adds EFAULT handlers 2024-08-28 04:31:06 -07:00
Ryan Houdek 5529948479 x32/Time: Adds EFAULT handlers 2024-08-28 04:25:13 -07:00
Ryan Houdek 91b6aeffe1 x32/Timer: Adds EFAULT handlers 2024-08-28 04:19:45 -07:00
Ryan Houdek 3a79b61f58 x64/Thread: Adds EFAULT handlers for two syscalls
For the remaining syscalls we need to introduce some new concepts around
checking if pointers and string arrays are readable and I don't want to
do that yet.
2024-08-28 04:14:16 -07:00
Ryan Houdek e92b24302f LinuxEmulation: Moves guest signal frame generation to its own file
No functional change, just moving the code.

This is entirely freestanding from the rest of the signal delegator
handling and mostly gets in the way when I'm working with the rest of
the signal handling. Separate it out to improve readability.
2024-08-28 03:23:00 -07:00
Tony Wasserka cc9ccf881b FEXQonfig: Use Unicode/locale-aware sorting for RootFS list
This orders the entries more like a normal file browser would.
2024-08-28 09:56:20 +02:00
Tony Wasserka 72d74ae482 FEXQonfig: Set a maximum height for RootFS list and make it scrollable 2024-08-28 09:52:46 +02:00
Tony Wasserka e88c57bec7 FEXQonfig: Move fallback RootFS entry to the top of the list
This provides better visibility when the RootFS list is large.
2024-08-28 09:51:27 +02:00
Ryan Houdek ac814ac015 Merge pull request #4017 from davide125/gdb
Install GDBSymbol integration in the correct location
2024-08-27 22:04:37 -07:00
Ryan Houdek 90f7cc925d Merge pull request #4007 from Sonicadvance1/ensure_no_256bit_operations
Arm64: Ensure 256-bit operations always assert without 256-bit SVE
2024-08-27 20:14:06 -07:00
Ryan Houdek 2039950762 Merge pull request #4016 from alyssarosenzweig/opt/umul-mov
Eliminate a move in 64-bit umul
2024-08-27 20:13:49 -07:00
Ryan Houdek b526c60c73 Merge pull request #4004 from Sonicadvance1/more_syscall_checks
Adds more syscall memory access checks
2024-08-27 20:13:21 -07:00
Davide Cavalca 1bfdd031d3 Install GDBSymbol integration in the correct location 2024-08-27 14:38:36 -07:00
Alyssa Rosenzweig 49ee8ba3be InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-27 12:33:07 -04:00
Alyssa Rosenzweig 335cd9180e OpcodeDispatcher: optimize mul rax
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-27 12:32:43 -04:00
Tony Wasserka ca9a94d572 CMake: Build only one FEXConfig and make Qt/ImGui exclusive choices 2024-08-27 18:10:06 +02:00
Alyssa Rosenzweig 03832b2523 Merge pull request #4012 from alyssarosenzweig/opt/pf-af-kill
DeadStoreElimination: handle PF/AF invalidate
2024-08-27 08:01:54 -04:00
Alyssa Rosenzweig 812224a0ef Merge pull request #4011 from bylaws/arm64-mb
Fix multiblock on ARM64EC
2024-08-27 08:01:26 -04:00
Alyssa Rosenzweig 42f2851575 Merge pull request #4009 from alyssarosenzweig/opt/axflag
Optimize AXFLAG-less systems
2024-08-27 08:01:05 -04:00
Ryan Houdek 9c7f44f8d4 x32/Signals: Add EFAULT checks 2024-08-27 03:04:42 -07:00
Ryan Houdek 5117ba351e x32/Semaphore: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek 441fdb689d x32/Sched: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek e786dfc998 x32/Msg: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek a1d3183c14 x32/Info: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek c5ef0910c5 x32/IO: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek affa1d0efc x32/FD: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek a83dc27a42 x32/EPoll: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek 129ec63e92 x64/Time: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek 3459369c6e x64/Signals: Add EFAULT checks 2024-08-27 03:04:24 -07:00
Ryan Houdek 7a490a3811 x64/Semaphore: Add EFAULT checks 2024-08-27 03:04:23 -07:00
Ryan Houdek ee17fe239a x64/EPoll: Add EFAULT checks 2024-08-27 03:04:23 -07:00
Tony Wasserka 0416950aaa Merge pull request #4015 from Sonicadvance1/fix_bug
x32/Signals: Fixes bug in the sigqueue syscalls
2024-08-27 12:03:55 +02:00
Ryan Houdek 6e46383cdb x32/Signals: Fixes bug in the sigqueue syscalls
In rt_sigqueueinfo and rt_tgsigqueueinfo we were failiny to pass along
the provided siginfo_t information to the syscall.

Fix that
2024-08-27 02:54:27 -07:00
Alyssa Rosenzweig faf1b85904 DeadStoreElimination: handle PF/AF invalidate
lets us kill some AF calculations with multiblock.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-26 11:28:16 -04:00
Billy Laws bec5f4fe2a DeadStoreElimination: Fix uint64 typing for register bitmask
unsigned longs are 32 bits wide on Windows.
2024-08-26 13:12:58 +00:00
Billy Laws fe6dbbfa63 Frontend: Don't explore native ARM64EC jump targets with multiblock 2024-08-26 13:01:06 +00:00
Ryan Houdek b19440c78f Arm64: Ensure 256-bit operations always assert without 256-bit SVE
Our JIT will happily consume incorrectly formed 256-bit vector operations in a lot of cases when the host CPU doesn't support 256-bit SVE.
This is what caused the bug in #4006. For every vector operation that
can consume a 256-bit size, add an assert that always checks if 256-bit
SVE is supported in those cases.

This will ensure that #4006 doesn't happen again.
2024-08-25 18:16:31 -07:00
Alyssa Rosenzweig abf9700475 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-25 18:38:03 -04:00
Alyssa Rosenzweig 8d6b454455 OpcodeDispatcher: optimize AXFLAG emulation
this should help on x13s which has flagm but not flagm2.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-25 18:38:03 -04:00
Alyssa Rosenzweig 205ec3e14d OpcodeDispatcher: refactor axflag
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-25 18:35:27 -04:00
Mai f897579593 Merge pull request #4005 from Sonicadvance1/fault_safe_assert
SignalDelegator: Changes AFmt to ERROR_AND_DIE_FMT
2024-08-25 13:15:04 -04:00
Mai 2478abba29 Merge pull request #4008 from Sonicadvance1/fix_avx128_vfcmp
AVX128: Fixes 256-bit float compares
2024-08-25 12:37:04 -04:00
Mai a4db585664 Merge pull request #4006 from Sonicadvance1/fix_avx128_convert
AVX128: Fixes incorrect size usage in AVX128_Vector_CVT_Int_To_Float
2024-08-25 12:34:24 -04:00
Ryan Houdek 8bf4a124c8 AVX128: Fixes 256-bit float compares
Just like the bug in #4006, we were incorrectly passing a 256-bit
comparison in to the float compare IR operations. Once again it would
"safely" decompose in to 128-bit operations without issue.

PR #4007 added asserts to ensure that 256-bit operations aren't emitted
when the host CPU doesn't support 256-bit SVE and detected this.
2024-08-25 06:33:31 -07:00
Ryan Houdek 13c3b65732 unittests/asm: Adds test for previous bug fix 2024-08-25 05:22:30 -07:00
Ryan Houdek 7e6ba184f9 AVX128: Fixes incorrect size usage in AVX128_Vector_CVT_Int_To_Float
This handler was incorrectly using 256-bit IR operation sizes. Due to a
quirk with our IR handling, this would "safely" fall back to a 128-bit
operation and work "correctly".

The problem encountered is that since the IR operation is claiming to be
256-bit, when the value got spilled due to register pressure then a
true 256-bit store and load operation would be generated. This would
then emit an SVE load and store, with the expectation of 256-bit SVE
loadstores. This caused a SIGILL on Oryon since it doesn't support SVE,
but even would generate an invalid predicated loadstore on SVE 128-bit
hardware.

Fixes Aperture Desk Job in FEX.
2024-08-25 05:18:53 -07:00
Ryan Houdek f6cb914a2a SignalDelegator: Changes AFmt to ERROR_AND_DIE_FMT
To ensure that it actually dies after logging.
2024-08-25 04:27:59 -07:00
Tony Wasserka 6c92f94ee8 FEXQonfig: Remove unused code from CMakeLists.txt 2024-08-24 11:28:52 +02:00
Tony Wasserka bebf4209d6 FEXQonfig: Fix Qt 6 build 2024-08-24 11:01:16 +02:00
Ryan Houdek c82a683987 Frontend: short-circuit code generation on invalid instructions with multiblock
A source of overhead with multiblock is hitting instructions through a
conditional branch that can never be executed. Usually AVX512
instructions in glibc. This causes us to emit partial blocks for a ton
of targets that will never get executed.

Instead, when we have multiblock enabled, if a block hits an instruction
encoding we don't support, then remove all the decoded instructions from
the block and early terminate it if it isn't the entry block. This
resolves the issue of emitting a bunch of IR and code for blocks never
executed.

If the block of code has an invalid instruction in the entry block for
decoding then it'll still emit code up to the invalid instruction and
raise a SIGILL. This has the potential for generating some additional
blocks of code if a game is abusing SIGILL, but since that's unlikely
it's a good trade-off.

Also removes a few log instructions that don't really provide anything
anymore and just show up as confusing messages when multiblock is
enabled.
2024-08-23 17:22:33 -07:00
Ryan Houdek 886d40ccfe FEXGetConfig: Add new logs for split-lock emulation 2024-08-23 16:18:20 -07:00
Ryan Houdek 8f33e56e21 FEXConfig: Add new option for strict split-lock emulation 2024-08-23 16:18:19 -07:00
Ryan Houdek b1634680fe SpinWaitLock: Ensure mutexes are move-only
This is to prevent issues with copying mutex classes around.
2024-08-23 16:18:19 -07:00
Ryan Houdek 54d332935e Arm64: Implement support for strict in-process split-locks
For atomics that cross the 16-byte or 64-byte granularity, we need to
lock a mutex to ensure strict emulation of split-locks.

I took another look at these when I found out that Zen3 actually
implements split-locks. Not sure which architecture actually added
support for from them, but I wanted to ensure we have the ability to
handle this.

One thing that we can't handle in user-space is cross-process
split-locks through shared memory. This requires a kernel SIGBUS handler
to ensure a crashing/SIGKILL'd process doesn't lock all FEX processes in
the system.

This fixes a little split-lock abusing test that I have locally. It's a
bit flakey so it isn't viable to run in CI. Considering it is explicitly
testing a race problem.
2024-08-23 16:18:19 -07:00
Mai 2829ad56a1 Merge pull request #3996 from Sonicadvance1/more_bind
OpcodeDispatcher: Convert more template handlers to Bind handlers
2024-08-23 18:27:59 -04:00
Ryan Houdek fbf62f1296 Merge pull request #3998 from Sonicadvance1/move_midr_fetch
HostFeatures: Moves MIDR querying to the frontend
2024-08-23 15:19:18 -07:00
Ryan Houdek e6aa268093 Merge pull request #4002 from bylaws/vct
OpcodeDispatcher: Allow x86 code to read CNTVCT on ARM64EC
2024-08-23 15:19:05 -07:00
Ryan Houdek cc589ba7e6 Merge pull request #4001 from bylaws/sim
ARM64EC: Always use the CPU area context for BeginSimulation
2024-08-23 15:08:43 -07:00
Ryan Houdek f009a00986 Merge pull request #4000 from bylaws/binfmt
CMake: Don't install binfmts for MinGW builds
2024-08-23 15:08:30 -07:00
Ryan Houdek 8617150a42 Merge pull request #3999 from Sonicadvance1/fix_defer_signal_mask
SignalDelegator: Make sure to defer a signal if the guest signal mask desires
2024-08-23 15:08:06 -07:00
Billy Laws ef823ce82b OpcodeDispatcher: Allow x86 code to read CNTVCT on ARM64EC
Required by newer insider preview versions, I noticed many crashes with
this exception number and QueryPeformanceCounter in the backtrace,
testing with XTA found it to not be passed through and instead write
the host CNTVCT (unscaled) into RAX. No other registers seem to be
affected.
2024-08-23 21:46:11 +00:00
Billy Laws fc2917117e ARM64EC: Always use the CPU area context for BeginSimulation
New wine tests suggest x0 holding the pointer to the correct context is
just a coincidence so avoid relying on it.
2024-08-23 21:43:09 +00:00
Tony Wasserka 68abe400a7 FEXQonfig: Use better icon for RootFS file selection 2024-08-23 17:36:56 +02:00
Tony Wasserka df86d80a85 FEXQonfig: Clean up object ownership 2024-08-23 17:36:56 +02:00
Tony Wasserka 58cff72d38 FEXQonfig: Use FolderDialog when selecting library forwarding paths 2024-08-23 17:36:56 +02:00
Tony Wasserka fd1f5643c7 FEXQonfig: Rename header to Main.h 2024-08-23 17:28:28 +02:00
Billy Laws 0ea29dfbdf CMake: Don't install binfmts for MinGW builds 2024-08-23 12:09:02 +00:00
Tony Wasserka f4fca4482f FEXQonfig: Implement environment variable editing 2024-08-23 13:30:38 +02:00
Tony Wasserka 86d1ce7f00 FEXQonfig: Implement RootFS monitoring via inotify 2024-08-23 12:03:29 +02:00
Tony Wasserka 15fef1794c FEXQonfig: Create config and RootFS folders on startup if needed 2024-08-23 12:02:20 +02:00
Ryan Houdek 8b5d9d8fdf FEXLinuxTests: Adds unit test to ensure FEX defers a signal until unmask 2024-08-23 02:32:35 -07:00
Ryan Houdek 0ec724cf1f HostFeatures: Moves MIDR querying to the frontend
The MIDR querying is inherently OS specific and needs a bit of special
casing. Instead let the frontend inform FEXCore how many CPU cores there
are and their MIDRs instead.

This lets us keep the Linux specific code in the frontend.
2024-08-23 00:53:53 -07:00
Ryan Houdek 0dc0117e86 SignalDelegator: Make sure to defer a signal if the guest signal mask desires
Fairly trivial because we already supported deferring these signals. We
had just failed the final step of blocking the signal if we can't block
the signal (like with SIGSEGV, SIGBUS, etc).

Once the guest unblocks the signal mask with sigprocmask, the signal
will fire again.

Apparently older glibc relied on this behaviour for signal raising which
this fixes.
2024-08-22 19:48:23 -07:00
Ryan Houdek 1d00ad6030 OpcodeDispatcher: Convert VectorVariableBlend to Bind handler 2024-08-22 15:09:44 -07:00
Ryan Houdek 23a076c313 OpcodeDispatcher: Convert packed vector shifts to Bind handler 2024-08-22 15:07:09 -07:00
Ryan Houdek 57eacab654 OpcodeDispatcher: Convert VBROADCASTOp to Bind handler 2024-08-22 14:59:25 -07:00
Ryan Houdek b4093a8888 OpcodeDispatcher: Convert packed HSub to Bind handler 2024-08-22 14:57:40 -07:00
Ryan Houdek 7c5a9b5d6a OpcodeDispatcher: Convert AVXVectorVariableBlend to Bind handler 2024-08-22 14:55:34 -07:00
Ryan Houdek 12b3c82d83 OpcodeDispatcher: Convert PExtr to Bind handler 2024-08-22 14:54:26 -07:00
Ryan Houdek 66520bce0a OpcodeDispatcher: Convert VPACK{U,S}S to Bind handler 2024-08-22 14:52:41 -07:00
Ryan Houdek 25cc2bdcb8 OpcodeDispatcher: Convert VPERMILImm to Bind handler 2024-08-22 14:51:02 -07:00
Ryan Houdek f98b18800c OpcodeDispatcher: Convert MOVMSK to Bind handler 2024-08-22 14:49:57 -07:00
Ryan Houdek 0dd687a7a1 OpcodeDispatcher: Convert SHUFOp to Bind handler 2024-08-22 14:47:30 -07:00
Ryan Houdek 1aff3acbb9 OpcodeDispatcher: Convert PSHUFW to Bind handler 2024-08-22 14:45:04 -07:00
Ryan Houdek a6ab2ca30d OpcodeDispatcher: Convert PUNPCKH to Bind handler 2024-08-22 14:42:24 -07:00
Ryan Houdek ca43e2a61c OpcodeDispatcher: Convert PUNPCKL to Bind handler 2024-08-22 14:39:45 -07:00
Mai 4f404160d0 Merge pull request #3995 from Sonicadvance1/missing_header
LinuxSyscalls: Adds missing header
2024-08-22 17:22:32 -04:00
Ryan Houdek 5edc69b692 LinuxSyscalls: Adds missing header 2024-08-22 14:02:17 -07:00
Ryan Houdek b6cb897896 Merge pull request #3994 from bylaws/iofix
Windows: Fix some file operations on actual Windows
2024-08-22 14:01:56 -07:00
Ryan Houdek 47b3637452 Merge pull request #3993 from bylaws/flags
ARM64EC: Fix ContextFlag member tests
2024-08-22 14:01:43 -07:00
Ryan Houdek 5b65f30c8f Merge pull request #3992 from Sonicadvance1/rootfs_fetcher_choose_ui
FEXRootFSFetcher: Allow UI override through options
2024-08-22 13:47:54 -07:00
Ryan Houdek 23572539f8 Merge pull request #3991 from Sonicadvance1/fix_feat_lrcpc_sigbus
Arm64: Fixes SIGBUS handler for FEAT_LRCPC
2024-08-22 13:47:39 -07:00
Tony Wasserka 7176c717e5 FEXQonfig: Handle load failure 2024-08-22 20:51:19 +02:00
Tony Wasserka 3a05b760d4 FEXQonfig: Further improve UI responsiveness
clear() emits the modelReset signal, which is redundant since we're
about to add new data anyway.
2024-08-22 18:42:09 +02:00
Tony Wasserka 1e0273d570 FEXQonfig: Improve UI responsiveness 2024-08-22 18:32:22 +02:00
Tony Wasserka 8232be6302 FEXQonfig: Implement RootFS selection from file dialog 2024-08-22 18:11:04 +02:00
Alyssa Rosenzweig 43cd897cf4 Merge pull request #3988 from alyssarosenzweig/opt/idea
Optimize `adc x, 0`
2024-08-22 11:05:16 -04:00
Alyssa Rosenzweig e593807856 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-22 10:51:06 -04:00
Alyssa Rosenzweig ce8e6e5c0c OpcodeDispatcher: optimize adc 0
clang generates this.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-22 10:51:06 -04:00
Alyssa Rosenzweig 5b9a7c7845 InstCountCI: add IDEA blocks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-22 10:51:06 -04:00
Alyssa Rosenzweig 0e5f5a2db9 Merge pull request #3990 from alyssarosenzweig/opt/tst-al-1
Optimise `test al, 1`
2024-08-22 10:48:44 -04:00
Alyssa Rosenzweig 50a9cea16e Merge pull request #3989 from alyssarosenzweig/ici/x8count
InstCountCI: include x86 instruction count
2024-08-22 10:44:44 -04:00
Billy Laws 6aa95d82f2 Windows: Query FileStandardInformation for getting file EOF info
FileEOFInformation is write-only on actual windows.
2024-08-22 14:34:45 +00:00
Billy Laws 7c34f449d1 Windows: Translate file open flags passed to NtCreateFile 2024-08-22 14:34:45 +00:00
Billy Laws e9435203a0 Windows: Fix access() implementation
Had the result inverted.
2024-08-22 14:33:06 +00:00
Billy Laws 8d9ba0e102 ARM64EC: Fix ContextFlag member tests
There is a common arch bit that is included in each individual mask,
explicitly test for equality against the whole mask to avoid just
testing for this.
2024-08-22 14:31:11 +00:00
Tony Wasserka 3fcfd2ccde FEXQonfig: Enable file creation on save 2024-08-22 15:07:26 +02:00
Tony Wasserka aa4205c2e8 FEXQonfig: Implement "Save As" 2024-08-22 14:55:11 +02:00
Tony Wasserka 5d5b411611 FEXQonfig: Tune window size constraints 2024-08-22 14:52:34 +02:00
Tony Wasserka 5f5a06fb3b FEXQonfig: Fix formatting 2024-08-22 14:52:34 +02:00
Tony Wasserka 3077addcf8 FEXQonfig: Unindent after refactor 2024-08-22 14:52:34 +02:00
Tony Wasserka 8b6fe0c4ff FEXQonfig: Make pages scrollable 2024-08-22 14:52:34 +02:00
Tony Wasserka 29fd62e9ba FEXQonfig: Drop leftover debug log 2024-08-22 14:52:34 +02:00
Ryan Houdek 2631b113da FEXRootFSFetcher: Allow UI override through options
For #3987
2024-08-21 18:45:07 -07:00
Ryan Houdek da152031b3 unittest: Adds unittest for unaligned loadstores hitting the correct SIGBUS handler 2024-08-21 18:05:31 -07:00
Ryan Houdek be9c5678c0 Arm64: Fixes SIGBUS handler for FEAT_LRCPC
When #3899 refactored some of this code, it had split an if-else chain
in to two independent if-else chains. Turns out there was a minor
dependency between them. Add back the few checks necessary to ensure
that FEAT_LRCPC falls through before hitting the FEAT_LSE checks.

This is necessary because their instructions share an encoding so the
masks between the two instruction classes overlap slightly.
2024-08-21 18:01:55 -07:00
Alyssa Rosenzweig fdd370dd1a InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Alyssa Rosenzweig 6951284924 OpcodeDispatcher: optimize ADCX/ADOX
rewrite to use native adcs with flag fixups.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Alyssa Rosenzweig b03c613fbf OpcodeDispatcher: optimize JP/JNP
fuse and+cbnz into tbz/tbnz.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Alyssa Rosenzweig b893bdb8df OpcodeDispatcher: optimize fcmovu
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Alyssa Rosenzweig db45f6eec8 OpcodeDispatcher: optimize test with small immediate
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Alyssa Rosenzweig d53e689e22 IR: model tbz/tbnz
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 17:52:22 -04:00
Tony Wasserka f55378257a FEXQonfig: Delay saving until after the user selects a filename 2024-08-21 21:55:23 +02:00
Tony Wasserka 6a7914ac56 FEXQonfig: Fix double-save due to Qt 5 bug 2024-08-21 21:54:16 +02:00
Alyssa Rosenzweig d935d25f0c InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 11:48:03 -04:00
Alyssa Rosenzweig 84cf1d2fd2 InstCountCI: include x86 instruction count
useful for checking blow up ratios. I keep calculating this by hand so let's
just add it to the json

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-21 11:47:55 -04:00
Tony Wasserka ed0c045c17 FEXQonfig: Add Qt 6 support 2024-08-21 13:21:08 +02:00
Tony Wasserka d585063e60 FEXQonfig: Save CacheObjectCodeCompilation on modification 2024-08-21 11:30:56 +02:00
Tony Wasserka 3077fec9ab FEXQonfig: Add status bar 2024-08-21 11:30:56 +02:00
Tony Wasserka 9500842efc FEXQonfig: Strip file:// prefixes when displaying file paths 2024-08-21 11:30:56 +02:00
Tony Wasserka a6f9c51317 FEXQonfig: Load config from default location on startup 2024-08-21 10:04:35 +02:00
Ryan Houdek cfa2ad8423 Merge pull request #3986 from alyssarosenzweig/opt/test-x-x
Optimize test
2024-08-20 19:20:57 -07:00
Alyssa Rosenzweig 19010491da InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 21:17:08 -04:00
Alyssa Rosenzweig 5d613e8716 OpcodeDispatcher: optimize test x, x
fewer uops now that we invert carry

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 21:17:08 -04:00
Alyssa Rosenzweig 894aaa980f Merge pull request #3975 from Sonicadvance1/update_wfe_comment
SpinWaitLock: Update comment about WFE spurious wakeups
2024-08-20 20:55:30 -04:00
Ryan Houdek 877b2f4fef Merge pull request #3955 from alyssarosenzweig/opt/pop-final
Rearrange SRA to let us coalesce cmpxchg moves
2024-08-20 17:36:11 -07:00
Ryan Houdek 2a170cfdec Merge pull request #3985 from alyssarosenzweig/opt/tso
OpcodeDispatcher: fix tso checks
2024-08-20 17:18:21 -07:00
Alyssa Rosenzweig a299d6b1a5 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 20:17:06 -04:00
Alyssa Rosenzweig ffb85e6305 ArchHelpers: rearrange SRA layout to coalesce cmpxchg
linux only for now, arm64ec should do something similar.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 20:14:08 -04:00
Alyssa Rosenzweig d7a20fa28f OpcodeDispatcher: fix tso checks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 20:11:06 -04:00
Ryan Houdek 5ac7d5dfcd Merge pull request #3980 from alyssarosenzweig/opt/avx
Optimize AVX load/store with ldp/stp
2024-08-20 16:29:55 -07:00
Alyssa Rosenzweig 75644b33df InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:36:50 -04:00
Alyssa Rosenzweig 4c4c6e7807 OpcodeDispatcher: pair loads on cortex
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:36:50 -04:00
Alyssa Rosenzweig a8c9c71ce3 OpcodeDispatcher: use loadcontextpair
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 8a4bd5f22c OpcodeDispatcher: pair avx128 save
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 138a36c69a OpcodeDispatcher: pair avx128 restore
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 3400ca5d42 OpcodeDispatcher: pair avx restore
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 5d8164da4e OpcodeDispatcher: pair restore x87
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 8b89b30a6f OpcodeDispatcher: pair restore SSE
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig e66d7cfd6c OpcodeDispatcher: pair mm save
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 63afa29dae OpcodeDispatcher: pair save AVX
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 926a9b40e2 OpcodeDispatcher: pair save mxcsr
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:41 -04:00
Alyssa Rosenzweig 9bb43264b5 OpcodeDispatcher: pair SSE store
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:33:37 -04:00
Alyssa Rosenzweig 32d6daf558 OpcodeDispatcher: use ldp/stp for AVX load/store
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig bebcb73c68 OpcodeDispatcher: pair AVX high writes
reduces instr count with AVX-128

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig d0e040514f IR: add LoadContextPair
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 850c027d52 IR: add AllocateFPR
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 5df90563e2 IR: add LoadMemPair
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 7a489d18c6 IR: add StoreMemPair
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 1db092e96f IR: add StoreContextPair
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 65a4de221b InstructionCountCI: add AddressingLimitations for AVX
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Alyssa Rosenzweig 9c8df79dfb InstructionCountCI: add bytemark stringsort blocks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 18:31:19 -04:00
Ryan Houdek 689b461d7b Merge pull request #3984 from Sonicadvance1/atomic_tso_subchecks
FEXCore: Splits up atomic enablement checks
2024-08-20 15:16:54 -07:00
Ryan Houdek 92c951c81f Merge pull request #3983 from Sonicadvance1/direct_sra_correlation
Arm64: Allow directly correlating an ARM register back to an x86 register
2024-08-20 15:16:46 -07:00
Ryan Houdek 9c8438f264 FEXCore: Splits up atomic enablement checks
PR #3980 is adding a feature to merge loadstores in to paired
loadstores, but it was using the incorrect atomic check to determine if
it can safely merge them or not. It was using the GPR atomic check
instead of the vector atomic check. While this would improve performance
on Apple Silicon with its hardware TSO implementation, it would have had
zero impact on Cortex and Oryon.

Instead split out the three config options to live as a boolean check in
the ContextImpl similar to how we disable "AtomicTSOEmulation". Removing
the various configs in the JIT and CPUID so that it queries from the
same context. This makes it clearer that if you are wanting the current
active configuration for memcpy, vector, or general atomic TSO
emulation, you should query one of those three getters.

This also fixes a weird edge case bug in the arm64 JIT where you could
have TSO emulation disable, but still have vector TSO enabled partially.
Just because half a config wasn't checked in {Load,Store}MemTSO for
vectors. If the global "TSOEnabled" option is disabled then TSO should
always be disabled.

Alyssa will be able to pull this in to #3980 once merged and get the
performance uplift on Cortex and Oryon, since our default configuration
is to have vector and memcpy TSO emulation disabled.
2024-08-20 14:34:34 -07:00
Ryan Houdek caf7ad53e6 Arm64: Allow directly correlating an ARM register back to an x86 register
Fixes a bug that is getting introduced in to #3955 when it rearranged
register allocation.
2024-08-20 14:17:13 -07:00
Tony Wasserka 47f0fec2f2 Add Qt-based config editor 2024-08-20 20:54:34 +02:00
Ryan Houdek eadb502059 Merge pull request #3978 from bylaws/create
Windows: Implement CreateDirectoryW CRT function
2024-08-20 11:39:57 -07:00
Ryan Houdek 8e1695afa3 Merge pull request #3977 from bylaws/wowex
WOW64: Improve exception handling
2024-08-20 11:39:22 -07:00
Ryan Houdek a7138f26b6 Merge pull request #3981 from alyssarosenzweig/opt/btc
Optimize BTC
2024-08-20 11:38:43 -07:00
Alyssa Rosenzweig f2a9ce9d4c InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 09:26:20 -04:00
Alyssa Rosenzweig 84e4960e52 OpcodeDispatcher: optimize btc
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-20 09:24:02 -04:00
Ryan Houdek 99afd876ba Merge pull request #3976 from alyssarosenzweig/opt/huffman-
Improvements from bytemark "huffman"
2024-08-19 16:10:29 -07:00
Ryan Houdek 1caa31c5cb Merge pull request #3962 from Sonicadvance1/remove_vixl_requirement
HostFeatures: Removes vixl usage
2024-08-19 14:31:13 -07:00
Tony Wasserka d2c82ba707 Merge pull request #3968 from Sonicadvance1/binfmt_misc_systemd
binfmt_misc: Support systemd binfmt_misc
2024-08-19 22:33:52 +02:00
Billy Laws 9067f3513a Windows: Implement CreateDirectoryW CRT function
Used when the .fex-emu config dir doesn't exist. Also fixes up
CreateFile to not leak memory.
2024-08-19 19:11:17 +00:00
Billy Laws 409d691877 WOW64: Only handle unaligned atomics in JIT code 2024-08-19 19:07:30 +00:00
Billy Laws 1762ff0393 WOW64: Spill EFlags into the FEX state when reconstructing JIT context 2024-08-19 19:07:30 +00:00
Billy Laws 4d26178f25 Windows: Commonise guest exception handling logic and use for WOW64 2024-08-19 19:07:29 +00:00
Ryan Houdek c6582a1ce5 binfmt_misc: Support systemd binfmt_misc
Adds support for binfmt_misc through systemd configuration paths. Their
configuration files are basically the raw kernel interface description
in a .conf file, quite a bit more simple than the legacy debian path.

Default enable this path since systemd is the expected default
arrangement these days.

Fixes #2417
2024-08-19 11:15:18 -07:00
Alyssa Rosenzweig a51be56c9e InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 10:40:53 -04:00
Alyssa Rosenzweig e5cb583cc0 OpcodeDispatcher: optimize 8-bit rol
rotate right by less than 8:

  ror(________________7654321076543120, ...)

rotate left by less than 8:

  rol(76543210________________76543210, ...)

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 10:40:53 -04:00
Alyssa Rosenzweig 3ef83a9e23 OpcodeDispatcher: optimize ror al, cl
rely on the masking.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 10:40:53 -04:00
Alyssa Rosenzweig 7de6a5cb07 OpcodeDispatcher: optimize rotate flags
introduce an IR op for it so we can reason about flags across rotate
instructions easily.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 10:40:53 -04:00
Alyssa Rosenzweig e06b3a4186 ConstProp: optimize and x, -1
mitigates regression from previous patch

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 08:54:47 -04:00
Alyssa Rosenzweig c495f82f4f OpcodeDispatcher: optimize 8/16-bit bitwise with constants
avoid masking. apparently even modern compilers will do cute tricks with 8-bit
math in hot loops ...

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 08:54:47 -04:00
Alyssa Rosenzweig 49e4426ed5 RedundantFlagCalculationElimination: drop register writes
this allows us to form real cmp instructions when we know PF is dead.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 08:54:47 -04:00
Alyssa Rosenzweig c4e2436885 ConstProp: optimize add eax, -1 and friends
need to sign extend to get the constant inlined.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 08:54:47 -04:00
Alyssa Rosenzweig f078b25c7d InstructionCountCI: add bytemark huffman blocks
this test case has our biggest delta to native by far. the trash code we emit
makes clear why!

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-19 08:54:47 -04:00
Tony Wasserka e5149fba57 Merge pull request #3965 from Sonicadvance1/followup_3964
Library Forwarding: Follow up from #3964
2024-08-19 11:22:43 +02:00
Tony Wasserka 96055cbde7 Merge pull request #3972 from Sonicadvance1/update_robin_map_1_3_0
External: Update robin-map from 1.2.1 to 1.3.0
2024-08-19 10:53:40 +02:00
Tony Wasserka 4abac0cac7 Merge pull request #3970 from Sonicadvance1/stop_installing_static
FEXCore: Stop installing static library
2024-08-19 10:30:26 +02:00
Tony Wasserka 1e1bcc4af2 Merge pull request #3971 from alyssarosenzweig/cleanup/aotir
FEXLoader: don't install FEXUpdateAOTIRCache
2024-08-19 10:29:34 +02:00
Tony Wasserka f1d7879365 Merge pull request #3973 from Sonicadvance1/update_fmt_11_0_2
External: Update fmt from 10.1.1 to 11.0.2
2024-08-19 10:26:22 +02:00
Ryan Houdek 0ea3de95e0 SpinWaitLock: Update comment about WFE spurious wakeups
With recent bug fixes, WFE now can sleep for roughly as long as the
programmed architecture timer of 100 microseconds. Still nowhere near as
close as what x86 CPUs can get with waitx and waitpkg, because those
don't get spuriously woken up by an architecture timer.

100 microseconds is a significantly improvement over 52ns although.
2024-08-18 17:50:20 -07:00
Ryan Houdek 33cef7c8cd CRT: New symbols that fmt uses 2024-08-18 13:21:17 -07:00
Ryan Houdek b0fd220f3e External: Update fmt from 10.1.1 to 11.0.2
Required adding a new header to FEXCore/fextl/fmt.h and Thunks gen.cpp
2024-08-18 13:13:41 -07:00
Ryan Houdek 1e3539537d External: Update robin-map from 1.2.1 to 1.3.0
Carrying forward our patch to remove their static initializer.
2024-08-18 12:38:24 -07:00
Alyssa Rosenzweig 1b13a3dd4e FEXLoader: don't install FEXUpdateAOTIRCache
not used. we'll probably rip the whole thing out at some point but for now, no
reason to pollute user systems with this.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-17 11:33:46 -04:00
Ryan Houdek d1ec242e4f FEXCore: Stop installing static library
This hasn't ever mattered. Our API isn't even stable enough to really
install the shared library.
2024-08-16 19:55:00 -07:00
Ryan Houdek 21f9841e9d Merge pull request #3967 from bylaws/lib
Windows: Install libraries to CMAKE_INSTALL_LIBDIR
2024-08-16 17:04:52 -07:00
Billy Laws ea6c05a46a Windows: Install libraries to CMAKE_INSTALL_LIBDIR 2024-08-16 23:43:55 +00:00
Ryan Houdek 226f5e2f23 Merge pull request #3899 from Sonicadvance1/thread_safe_backpatch
Arm64: Handle backpatching in a thread-safe manner
2024-08-16 10:28:09 -07:00
Ryan Houdek a82fcdecd7 PR review comments 2024-08-16 10:21:18 -07:00
Ryan Houdek 27acbe305d ArchHelpers: Adjust ClearICache for its usage
Instead of clearing a hardcoded 16 bytes, adjust for the actual number
of instructions modified. The implementation will still only clear a
single cacheline so it doesn't change behaviour.
2024-08-16 10:21:18 -07:00
Ryan Houdek cd0739a534 Arm64Helpers: Moves instruction definitions to implementation
This used to exist in the FEXCore header since the unaligned handler was
done in the frontend. Once it got moved in to FEXCore it had stayed
there. Move it over now.
2024-08-16 10:21:18 -07:00
Ryan Houdek f1055d0713 Arm64: On backpatch ensure DMB instructions get patched in first
In the case of a visibility tear when one thread is backpatching while
another is executing. The executing thread can /potentially/ see the
writing of instructions depending on coherency rules or filling of
cachelines.

By ensuring the DMB instructions are backpatched over the NOP
instructions first, this ensures correct atomic visibility even on tear.
2024-08-16 10:21:18 -07:00
Ryan Houdek 7875b20594 Arm64: Handle backpatching in a thread-safe manner
When code buffers are shared between threads, FEX needs to be careful
around backpatching its code buffers, since one thread might have
backpatched the code that another thread was also planning on
backpatching.

To handle this case, when the handler fails to find a backpatchable
instruction, check if it was already backpatched. This can be determined
by atomically reading the instructions back and seeing if they have
turned in to the non-atomic variants.

In most cases we can just return saying that it has been handled, in the
case of a store we need to back the PC up 4 bytes to ensure the DMB is
executed before the non-atomic store.
2024-08-16 10:21:18 -07:00
Ryan Houdek 25679cd319 Arm64: Moves some unaligned atomic handling before mutex lock
These handlers don't do any code backpatching so locking the spinlock
futex isn't necessary. Move them before the lock to make them a bit more
efficient once code buffers get shared.
2024-08-16 10:21:18 -07:00
Ryan Houdek fa45ec32db Arm64: Early exit in unaligned handler if not on arm64
NFC, just removes a level of indent
2024-08-16 10:21:18 -07:00
Ryan Houdek 40d12c4d0e HostFeatures: Removes vixl usage
If we query the CPU flags ourselves then vixl is no longer a
compile-time or runtime dependency unless the vixl disassembler or
simulator is built.

A bit spicy from all the feature bits we need to load up.
2024-08-16 08:36:52 -07:00
Ryan Houdek 5e706dff64 Merge pull request #3961 from Sonicadvance1/fexcore_disable_vixl
FEXCore: Disable vixl linking if vixl disasm or simulator is disabled
2024-08-16 08:34:55 -07:00
Ryan Houdek a4860d6f87 Thunks: Follow up from #3964
Thunk host libraries were still hardcoded to lib/, use the same
mechanism as #3964 for installing to the correct location.
2024-08-16 07:39:55 -07:00
Ryan Houdek ef4c4f6e9b FEXCore: Disable vixl linking if vixl disasm or simulator is disabled
This was mostly there, just needed to remove some extraneous headers and
only insert vixl in to the library list if the options were enabled.
2024-08-16 07:29:41 -07:00
Ryan Houdek abbd65549a Merge pull request #3964 from davide125/lib64
Install libraries in the correct location
2024-08-16 07:26:02 -07:00
Ryan Houdek 00ef1baead Merge pull request #3963 from alyssarosenzweig/vendor/json
Clean up our JSON dependencies
2024-08-16 07:19:56 -07:00
Davide Cavalca 75687070a5 Install libraries in the correct location 2024-08-16 10:04:25 -04:00
Alyssa Rosenzweig 9b1496c327 tiny-json: collect includes
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:29:58 -04:00
Alyssa Rosenzweig 27cce9d9b7 tiny-json: drop nosprintf handling
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:29:55 -04:00
Alyssa Rosenzweig 08b66bf827 External: merge json headers
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:26:53 -04:00
Alyssa Rosenzweig 0aa5807482 External: merge json source files
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:25:46 -04:00
Alyssa Rosenzweig 7a2d8c5c01 External: merge json targets
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:24:44 -04:00
Alyssa Rosenzweig 1b8b1a24b0 External: merge json-maker into tiny-json
They are functionally one library.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:23:40 -04:00
Alyssa Rosenzweig 86eb2b13b9 External: hard vendor json libraries
they are both tiny and seemingly unmaintained upstream, let's bring them
under our own umbrella in tree.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-15 21:20:20 -04:00
Mai c634c53434 Merge pull request #3960 from Sonicadvance1/remove_bad_rep_assumption
OpcodeDispatcher: Remove old bad assumption in INC/DEC
2024-08-15 15:42:47 -04:00
Ryan Houdek 2eb7a9ff28 OpcodeDispatcher: Remove old bad assumption in INC/DEC
Somewhere there was an assumption made that INC and DEC supported the
repeat prefix. This isn't actually the case, while the prefix can be
encoded, it is a nop and should only expect to be used for padding.

Adds a unittest to ensure that behaviour is as expected.
2024-08-15 10:22:31 -07:00
Ryan Houdek 933c65d805 Merge pull request #3956 from alyssarosenzweig/opt/pop-return
small optimizations for returns
2024-08-15 03:30:23 -07:00
Ryan Houdek df0ecad15b Merge pull request #3933 from bylaws/arm64-suspend
Support cooperative suspend on ARM64EC
2024-08-15 01:22:28 -07:00
Ryan Houdek ce88f5f948 Merge pull request #3941 from Sonicadvance1/fault_assertion_checking
LinuxSyscalls: Implements less invasive assertion only EFAULT handlers
2024-08-15 01:14:15 -07:00
Ryan Houdek 27cd399d6f Merge pull request #3944 from Sonicadvance1/minor_syscall_changes
LinuxSyscalls: Some minor cleanups
2024-08-15 00:55:06 -07:00
Ryan Houdek 7b70925acf unittests/gvisor: Updates known failures test for poll 2024-08-15 00:48:57 -07:00
Ryan Houdek 2a3e0b93e8 unittests: Update syscalls_efault test 2024-08-15 00:48:57 -07:00
Ryan Houdek 2b26d0fff5 LinuxSyscalls: Implements less invasive assertion only EFAULT handlers
With the previous Copy{To,From}User helpers we need to actually
implement the handlers correctly. We want something that is a bit
lighter so we don't need to implement the faulting path in the syscall
handlers.

Implements a handful of helpers that just check for readable and
writable capability which can be thrown in to an assertion handler that
is zero cost in release mode.

Readable is checked by just attempting to read all bytes.
Writable is checked by attempting to read each byte and writing it back
to the same location.

Uses these helpers in x64/FD.cpp to showcase how they will be used to
detect EFAULT. Tested locally that they work correctly by writing some
small tests for the syscalls that expect EFAULT.
2024-08-15 00:48:57 -07:00
Alyssa Rosenzweig 129f676610 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 16:59:34 -04:00
Alyssa Rosenzweig 2dc92c122e BranchOps: micro-optimize ExitFunction
this should be slightly faster on Firestorm and no worse on recent cortex

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 16:59:34 -04:00
Alyssa Rosenzweig 0db17bd58a OpcodeDispatcher: optimize IRET
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 16:59:34 -04:00
Alyssa Rosenzweig aac16493f1 OpcodeDispatcher: optimize RET
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 16:59:34 -04:00
Ryan Houdek 86e5e1a15e Merge pull request #3954 from Sonicadvance1/fix_missing_comment
HostFeatures: fix clang reformat constantly with missing comment
2024-08-14 12:03:10 -07:00
Ryan Houdek aa5d2ff31c Merge pull request #3951 from alyssarosenzweig/opt/pops
Add a hack for multiple destinations & make good use of it
2024-08-14 12:03:00 -07:00
Ryan Houdek 30dd5f5c75 HostFeatures: fix clang reformat constantly with missing comment 2024-08-14 10:00:28 -07:00
Alyssa Rosenzweig f5bc06476a InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig c70e44cf05 OpcodeDispatcher: extract Push helper
Mirrors the Pop helper we added. This cleans up a bunch

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig ede13e37d9 OpcodeDispatcher: optimize Thunk
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig a7dada457a OpcodeDispatcher: optimize POPF
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 6367554d30 OpcodeDispatcher: optimize LEAVE
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 81969a684e OpcodeDispatcher: optimize pop segment
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig ee339b5960 OpcodeDispatcher: optimize POPA
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 881c940693 OpcodeDispatcher: optimize POP
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 900c62fa7b OpcodeDispatcher: add Pop helpers
hide away the allocate dance

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 200c6c054f IR: introduce POP operation
rmw on a source, kind of terrible.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig bc0927b7b1 JIT: optimize moves for cmpxchg
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 231c28395c RegisterAllocationPass: clean up after pair removal
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 64a45c0d29 IR: remove pairs
They're now unused. And won't be missed.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig cab02be637 IR: remove unused pair create/extract
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig 74f341bc0e IR: remove pair from cpuid
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig feaa1af1a8 IR: remove pair from XGetBV
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:37:06 -04:00
Alyssa Rosenzweig d59d040b4e IR: add design doc breadcrumb
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig a9c26cbf71 IR: add coalescing heuristics for pair replacements
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig f4b5c4e69a OpcodeDispatcher: allow upper garbage for cmpxchg
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig b8cac9f7d5 JIT: avoid some moves with caspal
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 13974df204 IR: drop CASPair pair result
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig fa6fe9bf06 IR: drop cmppairz pairs
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 746be0824e IR: drop CASPair source pairs
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 93120cabbb IR: drop pair from memcpy
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 04ae05f4ce IR: add hack for multiple destinations
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 83773dddc7 IR: fix size for cmppairz
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Alyssa Rosenzweig 9813553f02 json_ir_generator: introduce multidestination hack
We now have two types of destinations:

* regular destinations. These are SSA. You get exactly 1 per instruction. This
  is what almost every instruction should use.

* special destinations, introduced here. These are *not* SSA. They must be
  allocated with a special instruction (added later in this PR), and then they
  are mutated by the instruction. There are two types, either pure destinations
  ("out") or read-modify-write source+destinations ("in-out"). The former are
  useful for instructions that return multiple destinations, like Memcpy. The
  latter are useful for instructions that need a source tied with a special
  destination (currently just Pop, introduced later in this series).

Special destinations reuse the mechanism of sources, to get around the
limitations on regular destinations in our current IR. Ops with special
destinations desugar to ops with no destination but extra sources prefixed Out
or Inout.

They further require HasSideEffects so we don't optimize ourselves into corners.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-14 09:17:23 -04:00
Ryan Houdek 924723d433 LinuxSyscalls: Some minor cleanups
- We can have the SyscallFunctionDefinitions be the correct size out of
  the gate. Both tables are always 512 entries in size.
- In the RegisterSyscall_{32,64} handlers, just get the reference using
  operator[]. We always know we will be under the size of the array, add
  a an assert to check. Removes a bit of vector range checking overhead.
- Namespace 32-bit syscalls like 64-bit syscalls and include in the
  regular header like 64-bit. This was just an oversight
- Use std::fill for the syscall gap for the invalid syscall, just a
  minor cleanup.

No functional change.
2024-08-13 14:06:16 -07:00
Ryan Houdek 33558e63c4 Merge pull request #3950 from alyssarosenzweig/bug/flagm-ci
InstructionCountCI: explicitly enable flagm for multiinst
2024-08-13 13:49:11 -07:00
Ryan Houdek 97c229d5eb Merge pull request #3935 from bylaws/win-feats
Windows: Support CPU feature detection from ID registry keys
2024-08-13 13:40:54 -07:00
Billy Laws 16b007df33 Windows: Partially implement the registry read API 2024-08-13 13:31:32 +00:00
Billy Laws 3cbc421c7e Windows: Detect CPU features using the registry
Native Windows does not support ID instruction emulation, instead it
populates registry keys containing their values.
2024-08-13 13:31:32 +00:00
Billy Laws 890e5e1f0f FEXCore: Support disabling host cacheline clean/clear operations 2024-08-13 13:25:34 +00:00
Billy Laws 6b98454f03 HostFeatures: Split out vixl feature and CTR/MIDR detection 2024-08-13 13:25:34 +00:00
Alyssa Rosenzweig b05329c8df InstructionCountCI: explicitly enable flagm for multiinst
Closes: https://github.com/FEX-Emu/FEX/issues/3949
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 09:09:07 -04:00
Ryan Houdek 6f43c8ffac Merge pull request #3947 from alyssarosenzweig/opt/littles
Little opcodedispatcher optimizations
2024-08-13 05:54:47 -07:00
Alyssa Rosenzweig ffac98051f InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Alyssa Rosenzweig 40812efaae OpcodeDispatcher: better handle SIB indexing
if we have shift and a constant, we can save an instruction

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Alyssa Rosenzweig 3429321d59 OpcodeDispatcher: allow upper garbage for MOVGPR
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Alyssa Rosenzweig 6cddd6cbe7 OpcodeDispatcher: allow upper garbage for a2/a3
stores mask.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Alyssa Rosenzweig 23d07d7d0c OpcodeDispatcher: fix folding negative offsets for 32-bit
I don't know what I was thinking when I wrote that code. Drop the silly logic
and let ConstProp inline the immediates. This fixes a lot of silly code
generated for 32-bit.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Alyssa Rosenzweig 1351575713 InstructionCountCI: add fpemu blocks
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-13 08:41:16 -04:00
Ryan Houdek 8aa7d1a278 Merge pull request #3939 from alyssarosenzweig/opt/cfinv
Invert carry flag internally
2024-08-13 01:05:54 -07:00
Alyssa Rosenzweig 3383786205 InstCountCI: Update
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 15:21:08 -04:00
Alyssa Rosenzweig 91f4c54768 OpcodeDispatcher: optimize RDRAND on flagm
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 15:21:08 -04:00
Alyssa Rosenzweig d9c779289c OpcodeDispatcher: simplify RDRAND
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 15:18:00 -04:00
Alyssa Rosenzweig 5631ff4fd5 OpcodeDispatcher: optimize variable shifts
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig 34301319bf OpcodeDispatcher: optimize IncrementByCarry
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig 8eac3198b6 OpcodeDispatcher: use carry increment for ADC
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig 832edd4da3 OpcodeDispatcher: use carry increment for SBC
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig 5823e74bcd OpcodeDispatcher: use increment carry for atomics
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig a4545f493e OpcodeDispatcher: add IncrementByCarry helper
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig 3b8cd44ca4 IR: add NZCVSelectIncrement
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:43 -04:00
Alyssa Rosenzweig f138d7d9b8 OpcodeDispatcher: optimize JA/JNA
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 3bb9d44bf5 OpcodeDispatcher: optimize SetCFDirect
generally same # of instructions, but potentially fewer cycles:

old:
        "rmif x4, #63, #nzCv",
        "cfinv"

new:
        "xor x20, x4, #1",
        "rmif x20, #63, #nzCv"

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 06e6a1b19e OpcodeDispatcher: optimize logic ops
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 256d166126 OpcodeDispatcher: optimize DAA
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 630285c589 OpcodeDispatcher: optimize SetPackedRFLAG
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 9621eca677 OpcodeDispatcher: fix setrflag masking
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 7eee50d929 OpcodeDispatcher: optimize mul/umul
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 1dc22e33ae OpcodeDispatcher: inline some flag calculations
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 0ef8aaebeb OpcodeDispatcher: optimize BZHI
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig d17c427e47 OpcodeDispatcher: optimize non-flagm2 comiss
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig f73fb62c6e OpcodeDispatcher: optimize V(P)TEST
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 5976b712ca OpcodeDispatcher: optimize bl*
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 63f5e64adb OpcodeDispatcher: optimize add/sub
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 34ee3bb8aa OpcodeDispatcher: optimize clc/stc
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 79c745929a FEXCore: invert CF internally
Flag day change to the ABI.

Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:06:42 -04:00
Alyssa Rosenzweig 8330cc6876 OpcodeDispatcher: defer carry inverts
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 13:05:21 -04:00
Alyssa Rosenzweig 16e6163677 OpcodeDispatcher: drop unused NZCVIndexMask
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 11:26:12 -04:00
Alyssa Rosenzweig bbbc0dc9dc OpcodeDispatcher: fix weird formatting
Signed-off-by: Alyssa Rosenzweig <alyssa@rosenzweig.io>
2024-08-10 11:26:12 -04:00
Billy Laws ea4004ec9d ARM64EC: Support cooperative suspend
When a suspend request occurs on a running thread, Windows gives the JIT
~10s to pause and NtContinue with the x86 context. Resumption is then
done through BeginSimulation.
2024-08-09 14:34:17 +00:00
Billy Laws a469047e7a FEXCore: Support cooperative suspend on ARM64EC
ARM64EC doesn't have an explicit callback for suspend, instead this task
is delegated to the kernel setting a bool at a JIT-defined memory
location, as such the faulting based approach as was done for WOW64
cannot be used.
2024-08-09 14:34:17 +00:00
Billy Laws 2cc0a867a9 ARM64EC: Set the InSimulation TEB flag when inside the JIT
This is used by the kernel to determine whether to perform cooperative
suspend.
2024-08-09 11:52:03 +00:00
1335 changed files with 68040 additions and 61963 deletions

No files matched your search

+1 -1
View File
@@ -4,7 +4,7 @@ compile_commands.json
vim_rc
Config.json
[Bb]uild*/
[Bb]uild*
[Bb]in/
out/
.vscode/
-9
View File
@@ -5,15 +5,6 @@
[submodule "External/cpp-optparse"]
path = Source/Common/cpp-optparse
url = https://github.com/Sonicadvance1/cpp-optparse
[submodule "External/imgui"]
path = External/imgui
url = https://github.com/Sonicadvance1/imgui.git
[submodule "External/json-maker"]
path = External/json-maker
url = https://github.com/Sonicadvance1/json-maker.git
[submodule "External/tiny-json"]
path = External/tiny-json
url = https://github.com/Sonicadvance1/tiny-json.git
[submodule "External/xbyak"]
shallow = true
path = External/xbyak
+38 -26
View File
@@ -7,7 +7,7 @@ CHECK_INCLUDE_FILES ("gdb/jit-reader.h" HAVE_GDB_JIT_READER_H)
option(BUILD_TESTS "Build unit tests to ensure sanity" TRUE)
option(BUILD_FEX_LINUX_TESTS "Build FEXLinuxTests, requires x86 compiler" FALSE)
option(BUILD_THUNKS "Build thunks" FALSE)
option(BUILD_FEXCONFIG "Build FEXConfig, requires SDL2 and X11" TRUE)
option(BUILD_FEXCONFIG "Build FEXConfig" TRUE)
option(ENABLE_CLANG_THUNKS "Build thunks with clang" FALSE)
option(ENABLE_IWYU "Enables include what you use program" FALSE)
option(ENABLE_LTO "Enable LTO with compilation" TRUE)
@@ -28,6 +28,7 @@ option(ENABLE_LIBCXX "Enables LLVM libc++" FALSE)
option(ENABLE_CCACHE "Enables ccache for compile caching" TRUE)
option(ENABLE_VIXL_SIMULATOR "Forces the FEX JIT to use the VIXL simulator" FALSE)
option(ENABLE_VIXL_DISASSEMBLER "Enables debug disassembler output with VIXL" FALSE)
option(USE_LEGACY_BINFMTMISC "Uses legacy method of setting up binfmt_misc" FALSE)
option(COMPILE_VIXL_DISASSEMBLER "Compiles the vixl disassembler in to vixl" FALSE)
option(ENABLE_FEXCORE_PROFILER "Enables use of the FEXCore timeline profiling capabilities" FALSE)
set (FEXCORE_PROFILER_BACKEND "gpuvis" CACHE STRING "Set which backend you want to use for the FEXCore profiler")
@@ -48,7 +49,7 @@ endif()
if (NOT MINGW_BUILD)
message (STATUS "Clang version ${CMAKE_CXX_COMPILER_VERSION}")
set (CLANG_MINIMUM_VERSION 12.0)
set (CLANG_MINIMUM_VERSION 13.0)
if (CMAKE_CXX_COMPILER_VERSION VERSION_LESS ${CLANG_MINIMUM_VERSION})
message (FATAL_ERROR "Clang version too old for FEX. Need at least ${CLANG_MINIMUM_VERSION} but has ${CMAKE_CXX_COMPILER_VERSION}")
endif()
@@ -231,7 +232,6 @@ if (ENABLE_JEMALLOC_GLIBC_ALLOC)
# The glibc jemalloc subproject which hooks the glibc allocator.
# Required for thunks to work.
# All host native libraries will use this allocator, while *most* other FEX internal allocations will use the other jemalloc allocator.
add_definitions(-DENABLE_JEMALLOC_GLIBC=1)
add_subdirectory(External/jemalloc_glibc/)
elseif (NOT MINGW_BUILD)
message (STATUS
@@ -243,9 +243,7 @@ endif()
if (ENABLE_JEMALLOC)
# The jemalloc subproject that all FEXCore fextl objects allocate through.
add_definitions(-DENABLE_JEMALLOC=1)
add_subdirectory(External/jemalloc/)
include_directories(External/jemalloc/pregen/include/)
elseif (NOT MINGW_BUILD)
message (STATUS
" jemalloc disabled!\n"
@@ -272,8 +270,10 @@ if (BUILD_TESTS)
set(COMPILE_VIXL_DISASSEMBLER TRUE)
endif()
add_subdirectory(External/vixl/)
include_directories(SYSTEM External/vixl/src/)
if (COMPILE_VIXL_DISASSEMBLER OR ENABLE_VIXL_SIMULATOR)
add_subdirectory(External/vixl/)
include_directories(SYSTEM External/vixl/src/)
endif()
if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
# This means we were attempted to get compiled with GCC
@@ -283,31 +283,38 @@ endif()
find_package(PkgConfig REQUIRED)
find_package(Python 3.0 REQUIRED COMPONENTS Interpreter)
set(XXHASH_BUNDLED_MODE TRUE)
set(XXHASH_BUILD_XXHSUM FALSE)
set(BUILD_SHARED_LIBS OFF)
add_subdirectory(External/xxhash/cmake_unofficial/)
pkg_search_module(xxhash IMPORTED_TARGET xxhash libxxhash)
if (TARGET PkgConfig::xxhash AND NOT CMAKE_CROSSCOMPILING)
add_library(xxHash::xxhash ALIAS PkgConfig::xxhash)
else()
set(XXHASH_BUNDLED_MODE TRUE)
set(XXHASH_BUILD_XXHSUM FALSE)
add_subdirectory(External/xxhash/cmake_unofficial/)
endif()
add_definitions(-Wno-trigraphs)
add_definitions(-DGLOBAL_DATA_DIRECTORY="${DATA_DIRECTORY}/")
if (BUILD_TESTS)
add_subdirectory(External/Catch2/)
find_package(Catch2 QUIET)
if (NOT Catch2_FOUND)
add_subdirectory(External/Catch2/)
# Pull in catch_discover_tests definition
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/External/Catch2/contrib/")
endif()
# Pull in catch_discover_tests definition
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/External/Catch2/contrib/")
include(Catch)
endif()
# Disable fmt install
set(FMT_INSTALL OFF)
add_subdirectory(External/fmt/)
add_subdirectory(External/imgui/)
include_directories(External/imgui/)
add_subdirectory(External/json-maker/)
include_directories(External/json-maker/)
find_package(fmt QUIET)
if (NOT fmt_FOUND)
# Disable fmt install
set(FMT_INSTALL OFF)
add_subdirectory(External/fmt/)
endif()
add_subdirectory(External/tiny-json/)
include_directories(External/tiny-json/)
@@ -405,10 +412,13 @@ configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/include/Config.h.in
${CMAKE_BINARY_DIR}/generated/ConfigDefines.h)
include(CTest)
if (BUILD_TESTS)
include(CTest)
enable_testing()
message(STATUS "Unit tests are enabled")
if (NOT BUILD_TESTING)
# CMake checks this variable before generating CTestTestfile.cmake
message(SEND_ERROR "Unit tests require BUILD_TESTING to be enabled")
endif()
set (TEST_JOB_COUNT "" CACHE STRING "Override number of parallel jobs to use while running tests")
if (TEST_JOB_COUNT)
@@ -423,8 +433,10 @@ add_subdirectory(FEXHeaderUtils/)
add_subdirectory(CodeEmitter/)
add_subdirectory(FEXCore/)
# Binfmt_misc files must be installed prior to Source/ installs
add_subdirectory(Data/binfmts/)
if (_M_ARM_64 AND NOT MINGW_BUILD)
# Binfmt_misc files must be installed prior to Source/ installs
add_subdirectory(Data/binfmts/)
endif()
add_subdirectory(Source/)
add_subdirectory(Data/AppConfig/)
+35
View File
@@ -0,0 +1,35 @@
# This is a reference AArch64 cross compile script
# Pass in to cmake when building:
# eg: cmake -DCMAKE_TOOLCHAIN_FILE=../CMakeToolchains/AArch64.cmake ..
if (NOT DEFINED ENV{SYSROOT})
message(FATAL_ERROR "Need to have SYSROOT environment variable set")
endif()
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(CMAKE_CROSSCOMPILING TRUE)
# Target triple needs to match the binutils exactly
set(TARGET_TRIPLE aarch64-linux-gnu)
set(CMAKE_C_COMPILER "clang")
set(CMAKE_CXX_COMPILER "clang++")
set(CMAKE_C_COMPILER_AR "llvm-ar")
set(CMAKE_CXX_COMPILER_AR "llvm-ar")
set(CMAKE_C_COMPILER_RANLIB "llvm-ranlib")
set(CMAKE_CXX_COMPILER_RANLIB "llvm-ranlib")
set(CMAKE_LINKER "ld.lld")
set(CMAKE_C_COMPILER_TARGET ${TARGET_TRIPLE})
set(CMAKE_CXX_COMPILER_TARGET ${TARGET_TRIPLE})
# Set the environment variable SYSROOT to the aarch64 rootfs
set(CMAKE_FIND_ROOT_PATH "$ENV{SYSROOT}")
set(CMAKE_SYSROOT "$ENV{SYSROOT}")
list(APPEND CMAKE_PREFIX_PATH "$ENV{SYSROOT}/usr/lib/${TARGET_TRIPLE}/cmake/")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)
+10 -8
View File
@@ -301,8 +301,9 @@ public:
xbfiz_helper(true, s, rd, rn, lsb, width);
}
void asr(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn, uint32_t shift) {
LOGMAN_THROW_A_FMT(shift <= RegSizeInBits(s), "Tried to asr a region larger than the register");
sbfm(s, rd, rn, shift, RegSizeInBits(s) - 1);
const auto RegSize_m1 = RegSizeInBits(s) - 1;
shift &= RegSize_m1;
sbfm(s, rd, rn, shift, RegSize_m1);
}
void uxtb(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn) {
@@ -325,14 +326,14 @@ public:
}
void lsl(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn, uint32_t shift) {
const auto RegSize = RegSizeInBits(s);
LOGMAN_THROW_A_FMT(shift < RegSize, "Tried to lsl a region larger than the register");
ubfm(s, rd, rn, (RegSize - shift) % RegSize, RegSize - shift - 1);
const auto RegSize_m1 = RegSizeInBits(s) - 1;
shift &= RegSize_m1;
ubfm(s, rd, rn, (RegSizeInBits(s) - shift) & RegSize_m1, RegSize_m1 - shift);
}
void lsr(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn, uint32_t shift) {
const auto RegSize = RegSizeInBits(s);
LOGMAN_THROW_A_FMT(shift < RegSize, "Tried to lsr a region larger than the register");
ubfm(s, rd, rn, shift, RegSize - 1);
const auto RegSize_m1 = RegSizeInBits(s) - 1;
shift &= RegSize_m1;
ubfm(s, rd, rn, shift, RegSize_m1);
}
void ubfx(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn, uint32_t lsb, uint32_t width) {
LOGMAN_THROW_A_FMT(width > 0, "ubfx needs width > 0");
@@ -368,6 +369,7 @@ public:
}
void ror(ARMEmitter::Size s, ARMEmitter::Register rd, ARMEmitter::Register rn, uint32_t Imm) {
Imm &= RegSizeInBits(s) - 1;
extr(s, rd, rn, rn, Imm);
}
+12 -12
View File
@@ -1070,7 +1070,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i16Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc(Op, 0, ConvertedSize, 0b10110, rd.D(), rn.D());
}
@@ -1082,7 +1082,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i16Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc(Op, 0, ConvertedSize, 0b10110, rd.Q(), rn.Q());
}
@@ -1095,7 +1095,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc(Op, 0, ConvertedSize, 0b10111, rd.D(), rn.D());
}
@@ -1107,7 +1107,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc(Op, 0, ConvertedSize, 0b10111, rd.Q(), rn.Q());
}
@@ -1123,7 +1123,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11000, rd, rn);
}
@@ -1138,7 +1138,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11001, rd, rn);
}
@@ -1154,7 +1154,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11010, rd, rn);
}
@@ -1169,7 +1169,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11011, rd, rn);
}
@@ -1184,7 +1184,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11100, rd, rn);
}
@@ -1199,7 +1199,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11101, rd, rn);
}
@@ -1214,7 +1214,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11110, rd, rn);
}
@@ -1229,7 +1229,7 @@ public:
constexpr uint32_t Op = 0b0000'1110'0010'0000'0000'10 << 10;
const auto ConvertedSize =
size == ARMEmitter::SubRegSize::i64Bit ? ARMEmitter::SubRegSize::i16Bit :
size == ARMEmitter::SubRegSize::i32Bit ? ARMEmitter::SubRegSize::i8Bit : ARMEmitter::SubRegSize::i8Bit;
ARMEmitter::SubRegSize::i8Bit;
ASIMD2RegMisc<T>(Op, 0, ConvertedSize, 0b11111, rd, rn);
}
+1
View File
@@ -354,6 +354,7 @@ enum class SystemRegister : uint32_t {
RNDRRS = GenSystemReg<0b11, 0b011, 0b0010, 0b0100, 0b001>(),
NZCV = GenSystemReg<0b11, 0b011, 0b0100, 0b0010, 0b000>(),
FPCR = GenSystemReg<0b11, 0b011, 0b0100, 0b0100, 0b000>(),
TPIDRRO_EL0 = GenSystemReg<0b11, 0b011, 0b1101, 0b0000, 0b011>(),
CNTFRQ_EL0 = GenSystemReg<0b11, 0b011, 0b1110, 0b0000, 0b000>(),
CNTVCT_EL0 = GenSystemReg<0b11, 0b011, 0b1110, 0b0000, 0b010>(),
};
+11 -2
View File
@@ -13,5 +13,14 @@ function(GenBinFmt Name)
DESTINATION ${CMAKE_INSTALL_PREFIX}/share/binfmts/)
endfunction()
GenBinFmt(FEX-x86.in)
GenBinFmt(FEX-x86_64.in)
if (NOT USE_LEGACY_BINFMTMISC)
configure_file(FEX-x86.conf.in ${CMAKE_BINARY_DIR}/Data/binfmts/FEX-x86.conf)
configure_file(FEX-x86_64.conf.in ${CMAKE_BINARY_DIR}/Data/binfmts/FEX-x86_64.conf)
install(
FILES ${CMAKE_BINARY_DIR}/Data/binfmts/FEX-x86.conf ${CMAKE_BINARY_DIR}/Data/binfmts/FEX-x86_64.conf
DESTINATION ${CMAKE_INSTALL_PREFIX}/lib/binfmt.d/)
else()
GenBinFmt(FEX-x86.in)
GenBinFmt(FEX-x86_64.in)
endif()
+1
View File
@@ -0,0 +1 @@
:FEX-x86:M:0:\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00:\xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff:@CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter:POCF
-1
View File
@@ -6,4 +6,3 @@ mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xf
credentials yes
fix_binary yes
preserve yes
expose_interpreter optional
+1
View File
@@ -0,0 +1 @@
:FEX-x86_64:M:0:\x7fELF\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x3e\x00:\xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xff\xff:@CMAKE_INSTALL_PREFIX@/bin/FEXInterpreter:POCF
-1
View File
@@ -6,4 +6,3 @@ mask \xff\xff\xff\xff\xff\xfe\xfe\x00\x00\x00\x00\xff\xff\xff\xff\xff\xfe\xff\xf
credentials yes
fix_binary yes
preserve yes
expose_interpreter optional
+1 -1
-1
Submodule External/imgui deleted from 4c986ecb8d.
Submodule External/json-maker deleted from 8ecb8ecc34.
Submodule External/tiny-json deleted from 9d09127f87.
+3
View File
@@ -0,0 +1,3 @@
set(NAME tiny-json)
set(SRCS tiny-json.c)
add_library(${NAME} ${SRCS})
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2018 Rafa Garcia
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+647
View File
@@ -0,0 +1,647 @@
/*
<https://github.com/rafagafe/tiny-json>
Licensed under the MIT License <http://opensource.org/licenses/MIT>.
SPDX-License-Identifier: MIT
Copyright (c) 2016-2018 Rafa Garcia <rafagarcia77@gmail.com>.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include <stdio.h>
#include <string.h>
#include <ctype.h>
#include <stddef.h> // For NULL
#include "tiny-json.h"
/** Structure to handle a heap of JSON properties. */
typedef struct jsonStaticPool_s {
json_t* const mem; /**< Pointer to array of json properties. */
unsigned int const qty; /**< Length of the array of json properties. */
unsigned int nextFree; /**< The index of the next free json property. */
jsonPool_t pool;
} jsonStaticPool_t;
/* Search a property by its name in a JSON object. */
json_t const* json_getProperty( json_t const* obj, char const* property ) {
json_t const* sibling;
for( sibling = obj->u.c.child; sibling; sibling = sibling->sibling )
if ( sibling->name && !strcmp( sibling->name, property ) )
return sibling;
return 0;
}
/* Search a property by its name in a JSON object and return its value. */
char const* json_getPropertyValue( json_t const* obj, char const* property ) {
json_t const* field = json_getProperty( obj, property );
if ( !field ) return 0;
jsonType_t type = json_getType( field );
if ( JSON_ARRAY >= type ) return 0;
return json_getValue( field );
}
/* Internal prototypes: */
static char* goBlank( char* str );
static char* goNum( char* str );
static json_t* poolInit( jsonPool_t* pool );
static json_t* poolAlloc( jsonPool_t* pool );
static char* objValue( char* ptr, json_t* obj, jsonPool_t* pool );
static char* setToNull( char* ch );
static bool isEndOfPrimitive( char ch );
/* Parse a string to get a json. */
json_t const* json_createWithPool( char *str, jsonPool_t *pool ) {
char* ptr = goBlank( str );
if ( !ptr || *ptr != '{' ) return 0;
json_t* obj = pool->init( pool );
obj->name = 0;
obj->sibling = 0;
obj->u.c.child = 0;
ptr = objValue( ptr, obj, pool );
if ( !ptr ) return 0;
return obj;
}
/* Parse a string to get a json. */
json_t const* json_create( char* str, json_t mem[], unsigned int qty ) {
jsonStaticPool_t spool = {
.mem = mem,
.qty = qty,
.pool = {
.init = poolInit,
.alloc = poolAlloc
}
};
return json_createWithPool( str, &spool.pool );
}
/** Get a special character with its escape character. Examples:
* 'b' -> '\b', 'n' -> '\n', 't' -> '\t'
* @param ch The escape character.
* @return The character code. */
static char getEscape( char ch ) {
static struct { char ch; char code; } const pair[] = {
{ '\"', '\"' }, { '\\', '\\' },
{ '/', '/' }, { 'b', '\b' },
{ 'f', '\f' }, { 'n', '\n' },
{ 'r', '\r' }, { 't', '\t' },
};
unsigned int i;
for( i = 0; i < sizeof pair / sizeof *pair; ++i )
if ( pair[i].ch == ch )
return pair[i].code;
return '\0';
}
/** Parse 4 characters.
* @Param str Pointer to first digit.
* @retval '?' If the four characters are hexadecimal digits.
* @retcal '\0' In other cases. */
static unsigned char getCharFromUnicode( unsigned char const* str ) {
unsigned int i;
for( i = 0; i < 4; ++i )
if ( !isxdigit( str[i] ) )
return '\0';
return '?';
}
/** Parse a string and replace the scape characters by their meaning characters.
* This parser stops when finds the character '\"'. Then replaces '\"' by '\0'.
* @param str Pointer to first character.
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* parseString( char* str ) {
unsigned char* head = (unsigned char*)str;
unsigned char* tail = (unsigned char*)str;
for( ; *head >= ' '; ++head, ++tail ) {
if ( *head == '\"' ) {
*tail = '\0';
return (char*)++head;
}
if ( *head == '\\' ) {
if ( *++head == 'u' ) {
char const ch = getCharFromUnicode( ++head );
if ( ch == '\0' ) return 0;
*tail = ch;
head += 3;
}
else {
char const esc = getEscape( *head );
if ( esc == '\0' ) return 0;
*tail = esc;
}
}
else *tail = *head;
}
return 0;
}
/** Parse a string to get the name of a property.
* @param str Pointer to first character.
* @param property The property to assign the name.
* @retval Pointer to first of property value. If success.
* @retval Null pointer if any error occur. */
static char* propertyName( char* ptr, json_t* property ) {
property->name = ++ptr;
ptr = parseString( ptr );
if ( !ptr ) return 0;
ptr = goBlank( ptr );
if ( !ptr ) return 0;
if ( *ptr++ != ':' ) return 0;
return goBlank( ptr );
}
/** Parse a string to get the value of a property when its type is JSON_TEXT.
* @param str Pointer to first character ('\"').
* @param property The property to assign the name.
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* textValue( char* ptr, json_t* property ) {
++property->u.value;
ptr = parseString( ++ptr );
if ( !ptr ) return 0;
property->type = JSON_TEXT;
return ptr;
}
/** Compare two strings until get the null character in the second one.
* @param ptr sub string
* @param str main string
* @retval Pointer to next character.
* @retval Null pointer if any error occur. */
static char* checkStr( char* ptr, char const* str ) {
while( *str )
if ( *ptr++ != *str++ )
return 0;
return ptr;
}
/** Parser a string to get a primitive value.
* If the first character after the value is different of '}' or ']' is set to '\0'.
* @param str Pointer to first character.
* @param property Property handler to set the value and the type, (true, false or null).
* @param value String with the primitive literal.
* @param type The code of the type. ( JSON_BOOLEAN or JSON_NULL )
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* primitiveValue( char* ptr, json_t* property, char const* value, jsonType_t type ) {
ptr = checkStr( ptr, value );
if ( !ptr || !isEndOfPrimitive( *ptr ) ) return 0;
ptr = setToNull( ptr );
property->type = type;
return ptr;
}
/** Parser a string to get a true value.
* If the first character after the value is different of '}' or ']' is set to '\0'.
* @param str Pointer to first character.
* @param property Property handler to set the value and the type, (true, false or null).
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* trueValue( char* ptr, json_t* property ) {
return primitiveValue( ptr, property, "true", JSON_BOOLEAN );
}
/** Parser a string to get a false value.
* If the first character after the value is different of '}' or ']' is set to '\0'.
* @param str Pointer to first character.
* @param property Property handler to set the value and the type, (true, false or null).
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* falseValue( char* ptr, json_t* property ) {
return primitiveValue( ptr, property, "false", JSON_BOOLEAN );
}
/** Parser a string to get a null value.
* If the first character after the value is different of '}' or ']' is set to '\0'.
* @param str Pointer to first character.
* @param property Property handler to set the value and the type, (true, false or null).
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* nullValue( char* ptr, json_t* property ) {
return primitiveValue( ptr, property, "null", JSON_NULL );
}
/** Analyze the exponential part of a real number.
* @param str Pointer to first character.
* @retval Pointer to first non numerical after the string. If success.
* @retval Null pointer if any error occur. */
static char* expValue( char* ptr ) {
if ( *ptr == '-' || *ptr == '+' ) ++ptr;
if ( !isdigit( *ptr ) ) return 0;
ptr = goNum( ++ptr );
return ptr;
}
/** Analyze the decimal part of a real number.
* @param str Pointer to first character.
* @retval Pointer to first non numerical after the string. If success.
* @retval Null pointer if any error occur. */
static char* fraqValue( char* ptr ) {
if ( !isdigit( *ptr ) ) return 0;
ptr = goNum( ++ptr );
if ( !ptr ) return 0;
return ptr;
}
/** Parser a string to get a numerical value.
* If the first character after the value is different of '}' or ']' is set to '\0'.
* @param str Pointer to first character.
* @param property Property handler to set the value and the type: JSON_REAL or JSON_INTEGER.
* @retval Pointer to first non white space after the string. If success.
* @retval Null pointer if any error occur. */
static char* numValue( char* ptr, json_t* property ) {
if ( *ptr == '-' ) ++ptr;
if ( !isdigit( *ptr ) ) return 0;
if ( *ptr != '0' ) {
ptr = goNum( ptr );
if ( !ptr ) return 0;
}
else if ( isdigit( *++ptr ) ) return 0;
property->type = JSON_INTEGER;
if ( *ptr == '.' ) {
ptr = fraqValue( ++ptr );
if ( !ptr ) return 0;
property->type = JSON_REAL;
}
if ( *ptr == 'e' || *ptr == 'E' ) {
ptr = expValue( ++ptr );
if ( !ptr ) return 0;
property->type = JSON_REAL;
}
if ( !isEndOfPrimitive( *ptr ) ) return 0;
if ( JSON_INTEGER == property->type ) {
char const* value = property->u.value;
bool const negative = *value == '-';
static char const min[] = "-9223372036854775808";
static char const max[] = "9223372036854775807";
unsigned int const maxdigits = ( negative? sizeof min: sizeof max ) - 1;
unsigned int const len = ptr - value;
if ( len > maxdigits ) return 0;
if ( len == maxdigits ) {
char const tmp = *ptr;
*ptr = '\0';
char const* const threshold = negative ? min: max;
if ( 0 > strcmp( threshold, value ) ) return 0;
*ptr = tmp;
}
}
ptr = setToNull( ptr );
return ptr;
}
/** Add a property to a JSON object or array.
* @param obj The handler of the JSON object or array.
* @param property The handler of the property to be added. */
static void add( json_t* obj, json_t* property ) {
property->sibling = 0;
if ( !obj->u.c.child ){
obj->u.c.child = property;
obj->u.c.last_child = property;
} else {
obj->u.c.last_child->sibling = property;
obj->u.c.last_child = property;
}
}
/** Parser a string to get a json object value.
* @param str Pointer to first character.
* @param pool The handler of a json pool for creating json instances.
* @retval Pointer to first character after the value. If success.
* @retval Null pointer if any error occur. */
static char* objValue( char* ptr, json_t* obj, jsonPool_t* pool ) {
obj->type = JSON_OBJ;
obj->u.c.child = 0;
obj->sibling = 0;
ptr++;
for(;;) {
ptr = goBlank( ptr );
if ( !ptr ) return 0;
if ( *ptr == ',' ) {
++ptr;
continue;
}
char const endchar = ( obj->type == JSON_OBJ )? '}': ']';
if ( *ptr == endchar ) {
*ptr = '\0';
json_t* parentObj = obj->sibling;
if ( !parentObj ) return ++ptr;
obj->sibling = 0;
obj = parentObj;
++ptr;
continue;
}
json_t* property = pool->alloc( pool );
if ( !property ) return 0;
if( obj->type != JSON_ARRAY ) {
if ( *ptr != '\"' ) return 0;
ptr = propertyName( ptr, property );
if ( !ptr ) return 0;
}
else property->name = 0;
add( obj, property );
property->u.value = ptr;
switch( *ptr ) {
case '{':
property->type = JSON_OBJ;
property->u.c.child = 0;
property->sibling = obj;
obj = property;
++ptr;
break;
case '[':
property->type = JSON_ARRAY;
property->u.c.child = 0;
property->sibling = obj;
obj = property;
++ptr;
break;
case '\"': ptr = textValue( ptr, property ); break;
case 't': ptr = trueValue( ptr, property ); break;
case 'f': ptr = falseValue( ptr, property ); break;
case 'n': ptr = nullValue( ptr, property ); break;
default: ptr = numValue( ptr, property ); break;
}
if ( !ptr ) return 0;
}
}
/** Initialize a json pool.
* @param pool The handler of the pool.
* @return a instance of a json. */
static json_t* poolInit( jsonPool_t* pool ) {
jsonStaticPool_t *spool = json_containerOf( pool, jsonStaticPool_t, pool );
spool->nextFree = 1;
return spool->mem;
}
/** Create an instance of a json from a pool.
* @param pool The handler of the pool.
* @retval The handler of the new instance if success.
* @retval Null pointer if the pool was empty. */
static json_t* poolAlloc( jsonPool_t* pool ) {
jsonStaticPool_t *spool = json_containerOf( pool, jsonStaticPool_t, pool );
if ( spool->nextFree >= spool->qty ) return 0;
return spool->mem + spool->nextFree++;
}
/** Checks whether an character belongs to set.
* @param ch Character value to be checked.
* @param set Set of characters. It is just a null-terminated string.
* @return true or false there is membership or not. */
static bool isOneOfThem( char ch, char const* set ) {
while( *set != '\0' )
if ( ch == *set++ )
return true;
return false;
}
/** Increases a pointer while it points to a character that belongs to a set.
* @param str The initial pointer value.
* @param set Set of characters. It is just a null-terminated string.
* @return The final pointer value or null pointer if the null character was found. */
static char* goWhile( char* str, char const* set ) {
for(; *str != '\0'; ++str ) {
if ( !isOneOfThem( *str, set ) )
return str;
}
return 0;
}
/** Set of characters that defines a blank. */
static char const* const blank = " \n\r\t\f";
/** Increases a pointer while it points to a white space character.
* @param str The initial pointer value.
* @return The final pointer value or null pointer if the null character was found. */
static char* goBlank( char* str ) {
return goWhile( str, blank );
}
/** Increases a pointer while it points to a decimal digit character.
* @param str The initial pointer value.
* @return The final pointer value or null pointer if the null character was found. */
static char* goNum( char* str ) {
for( ; *str != '\0'; ++str ) {
if ( !isdigit( *str ) )
return str;
}
return 0;
}
/** Set of characters that defines the end of an array or a JSON object. */
static char const* const endofblock = "}]";
/** Set a char to '\0' and increase its pointer if the char is different to '}' or ']'.
* @param ch Pointer to character.
* @return Final value pointer. */
static char* setToNull( char* ch ) {
if ( !isOneOfThem( *ch, endofblock ) ) *ch++ = '\0';
return ch;
}
/** Indicate if a character is the end of a primitive value. */
static bool isEndOfPrimitive( char ch ) {
return ch == ',' || isOneOfThem( ch, blank ) || isOneOfThem( ch, endofblock );
}
/** Add a character at the end of a string.
* @param dest Pointer to the null character of the string
* @param ch Value to be added.
* @return Pointer to the null character of the destination string. */
static char* chtoa( char* dest, char ch ) {
*dest = ch;
*++dest = '\0';
return dest;
}
/** Copy a null-terminated string.
* @param dest Destination memory block.
* @param src Source string.
* @return Pointer to the null character of the destination string. */
static char* atoa( char* dest, char const* src ) {
for( ; *src != '\0'; ++dest, ++src )
*dest = *src;
*dest = '\0';
return dest;
}
/* Open a JSON object in a JSON string. */
char* json_objOpen( char* dest, char const* name ) {
if ( NULL == name )
dest = chtoa( dest, '{' );
else {
dest = chtoa( dest, '\"' );
dest = atoa( dest, name );
dest = atoa( dest, "\":{" );
}
return dest;
}
/* Close a JSON object in a JSON string. */
char* json_objClose( char* dest ) {
if ( dest[-1] == ',' )
--dest;
return atoa( dest, "}," );
}
/* Open an array in a JSON string. */
char* json_arrOpen( char* dest, char const* name ) {
if ( NULL == name )
dest = chtoa( dest, '[' );
else {
dest = chtoa( dest, '\"' );
dest = atoa( dest, name );
dest = atoa( dest, "\":[" );
}
return dest;
}
/* Close an array in a JSON string. */
char* json_arrClose( char* dest ) {
if ( dest[-1] == ',' )
--dest;
return atoa( dest, "]," );
}
/** Add the name of a text property.
* @param dest Destination memory.
* @param name The name of the property.
* @return Pointer to the next char. */
static char* strname( char* dest, char const* name ) {
dest = chtoa( dest, '\"' );
if ( NULL != name ) {
dest = atoa( dest, name );
dest = atoa( dest, "\":\"" );
}
return dest;
}
/** Get the hexadecimal digit of the least significant nibble of a integer. */
static int nibbletoch( int nibble ) {
return "0123456789ABCDEF"[ nibble % 16u ];
}
/** Get the escape character of a non-printable.
* @param ch Character source.
* @return The escape character or null character if error. */
static int escape( int ch ) {
static struct { char code; char ch; } const pair[] = {
{ '\"', '\"' }, { '\\', '\\' }, { '/', '/' }, { 'b', '\b' },
{ 'f', '\f' }, { 'n', '\n' }, { 'r', '\r' }, { 't', '\t' },
};
for( int i = 0; i < sizeof pair / sizeof *pair; ++i )
if ( ch == pair[i].ch )
return pair[i].code;
return '\0';
}
/** Copy a null-terminated string inserting escape characters if needed.
* @param dest Destination memory block.
* @param src Source string.
* @return Pointer to the null character of the destination string. */
static char* atoesc( char* dest, char const* src ) {
for( ; *src != '\0'; ++dest, ++src ) {
if ( *src >= ' ' && *src != '\"' && *src != '\\' && *src != '/' )
*dest = *src;
else {
*dest++ = '\\';
int const esc = escape( *src );
if ( esc )
*dest = esc;
else {
*dest++ = 'u';
*dest++ = '0';
*dest++ = '0';
*dest++ = nibbletoch( *src / 16 );
*dest++ = nibbletoch( *src );
}
}
}
*dest = '\0';
return dest;
}
/* Add a text property in a JSON string. */
char* json_str( char* dest, char const* name, char const* value ) {
dest = strname( dest, name );
dest = atoesc( dest, value );
dest = atoa( dest, "\"," );
return dest;
}
/** Add the name of a primitive property.
* @param dest Destination memory.
* @param name The name of the property.
* @return Pointer to the next char. */
static char* primitivename( char* dest, char const* name ) {
if( NULL == name )
return dest;
dest = chtoa( dest, '\"' );
dest = atoa( dest, name );
dest = atoa( dest, "\":" );
return dest;
}
/* Add a boolean property in a JSON string. */
char* json_bool( char* dest, char const* name, int value ) {
dest = primitivename( dest, name );
dest = atoa( dest, value ? "true," : "false," );
return dest;
}
/* Add a null property in a JSON string. */
char* json_null( char* dest, char const* name ) {
dest = primitivename( dest, name );
dest = atoa( dest, "null," );
return dest;
}
/* Used to finish the root JSON object. After call json_objClose(). */
char* json_end( char* dest ) {
if ( ',' == dest[-1] ) {
dest[-1] = '\0';
--dest;
}
return dest;
}
#define ALL_TYPES \
X( json_int, int, "%d" ) \
X( json_long, long, "%ld" ) \
X( json_uint, unsigned int, "%u" ) \
X( json_ulong, unsigned long, "%lu" ) \
X( json_verylong, long long, "%lld" ) \
X( json_double, double, "%g" ) \
#define json_num( funcname, type, fmt ) \
char* funcname( char* dest, char const* name, type value ) { \
dest = primitivename( dest, name ); \
dest += sprintf( dest, fmt, value ); \
dest = chtoa( dest, ',' ); \
return dest; \
}
#define X( name, type, fmt ) json_num( name, type, fmt )
ALL_TYPES
#undef X
+270
View File
@@ -0,0 +1,270 @@
/*
<https://github.com/rafagafe/tiny-json>
Licensed under the MIT License <http://opensource.org/licenses/MIT>.
SPDX-License-Identifier: MIT
Copyright (c) 2016-2018 Rafa Garcia <rafagarcia77@gmail.com>.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#ifndef _TINY_JSON_H_
#define _TINY_JSON_H_
#ifdef __cplusplus
extern "C" {
#endif
#include <stddef.h>
#include <stdlib.h>
#include <stdbool.h>
#include <stdint.h>
#define json_containerOf( ptr, type, member ) \
((type*)( (char*)ptr - offsetof( type, member ) ))
/** @defgroup tinyJson Tiny JSON parser.
* @{ */
/** Enumeration of codes of supported JSON properties types. */
typedef enum {
JSON_OBJ, JSON_ARRAY, JSON_TEXT, JSON_BOOLEAN,
JSON_INTEGER, JSON_REAL, JSON_NULL
} jsonType_t;
/** Structure to handle JSON properties. */
typedef struct json_s {
struct json_s* sibling;
const char* name;
union {
const char* value;
struct {
struct json_s* child;
struct json_s* last_child;
} c;
} u;
jsonType_t type;
} json_t;
/** Parse a string to get a json.
* @param str String pointer with a JSON object. It will be modified.
* @param mem Array of json properties to allocate.
* @param qty Number of elements of mem.
* @retval Null pointer if any was wrong in the parse process.
* @retval If the parser process was successfully a valid handler of a json.
* This property is always unnamed and its type is JSON_OBJ. */
const json_t* json_create(char* str, json_t mem[], unsigned int qty);
/** Get the name of a json property.
* @param json A valid handler of a json property.
* @retval Pointer to null-terminated if property has name.
* @retval Null pointer if the property is unnamed. */
static inline const char* json_getName(const json_t* json) {
return json->name;
}
/** Get the value of a json property.
* The type of property cannot be JSON_OBJ or JSON_ARRAY.
* @param json A valid handler of a json property.
* @return Pointer to null-terminated string with the value. */
static inline const char* json_getValue(const json_t* property) {
return property->u.value;
}
/** Get the type of a json property.
* @param json A valid handler of a json property.
* @return The code of type.*/
static inline jsonType_t json_getType(const json_t* json) {
return json->type;
}
/** Get the next sibling of a JSON property that is within a JSON object or array.
* @param json A valid handler of a json property.
* @retval The handler of the next sibling if found.
* @retval Null pointer if the json property is the last one. */
static inline const json_t* json_getSibling(const json_t* json) {
return json->sibling;
}
/** Search a property by its name in a JSON object.
* @param obj A valid handler of a json object. Its type must be JSON_OBJ.
* @param property The name of property to get.
* @retval The handler of the json property if found.
* @retval Null pointer if not found. */
const json_t* json_getProperty(const json_t* obj, const char* property);
/** Search a property by its name in a JSON object and return its value.
* @param obj A valid handler of a json object. Its type must be JSON_OBJ.
* @param property The name of property to get.
* @retval If found a pointer to null-terminated string with the value.
* @retval Null pointer if not found or it is an array or an object. */
const char* json_getPropertyValue(const json_t* obj, const char* property);
/** Get the first property of a JSON object or array.
* @param json A valid handler of a json property.
* Its type must be JSON_OBJ or JSON_ARRAY.
* @retval The handler of the first property if there is.
* @retval Null pointer if the json object has not properties. */
static inline const json_t* json_getChild(const json_t* json) {
return json->u.c.child;
}
/** Get the value of a json boolean property.
* @param property A valid handler of a json object. Its type must be JSON_BOOLEAN.
* @return The value stdbool. */
static inline bool json_getBoolean(const json_t* property) {
return *property->u.value == 't';
}
/** Get the value of a json integer property.
* @param property A valid handler of a json object. Its type must be JSON_INTEGER.
* @return The value stdint. */
static inline int64_t json_getInteger(const json_t* property) {
return atoll( property->u.value );
}
/** Get the value of a json real property.
* @param property A valid handler of a json object. Its type must be JSON_REAL.
* @return The value. */
static inline double json_getReal(const json_t* property) {
return atof( property->u.value );
}
/** Structure to handle a heap of JSON properties. */
typedef struct jsonPool_s jsonPool_t;
struct jsonPool_s {
json_t* (*init)( jsonPool_t* pool );
json_t* (*alloc)( jsonPool_t* pool );
};
/** Parse a string to get a json.
* @param str String pointer with a JSON object. It will be modified.
* @param pool Custom json pool pointer.
* @retval Null pointer if any was wrong in the parse process.
* @retval If the parser process was successfully a valid handler of a json.
* This property is always unnamed and its type is JSON_OBJ. */
const json_t* json_createWithPool(char* str, jsonPool_t* pool);
/** @ } */
/** @defgroup makejoson Make JSON.
* @{ */
/** Open a JSON object in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @return Pointer to the new end of JSON under construction. */
char* json_objOpen(char* dest, const char* name);
/** Close a JSON object in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @return Pointer to the new end of JSON under construction. */
char* json_objClose(char* dest);
/** Used to finish the root JSON object. After call json_objClose().
* @param dest Pointer to the end of JSON under construction.
* @return Pointer to the new end of JSON under construction. */
char* json_end(char* dest);
/** Open an array in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @return Pointer to the new end of JSON under construction. */
char* json_arrOpen(char* dest, const char* name);
/** Close an array in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @return Pointer to the new end of JSON under construction. */
char* json_arrClose(char* dest);
/** Add a text property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value A valid null-terminated string with the value.
* Backslash escapes will be added for special characters.
* @return Pointer to the new end of JSON under construction. */
char* json_str(char* dest, const char* name, const char* value);
/** Add a boolean property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Zero for false. Non zero for true.
* @return Pointer to the new end of JSON under construction. */
char* json_bool(char* dest, const char* name, int value);
/** Add a null property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @return Pointer to the new end of JSON under construction. */
char* json_null(char* dest, const char* name);
/** Add an integer property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_int(char* dest, const char* name, int value);
/** Add an unsigned integer property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_uint(char* dest, const char* name, unsigned int value);
/** Add a long integer property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_long(char* dest, const char* name, long int value);
/** Add an unsigned long integer property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_ulong(char* dest, const char* name, unsigned long int value);
/** Add a long long integer property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_verylong(char* dest, const char* name, long long int value);
/** Add a double precision number property in a JSON string.
* @param dest Pointer to the end of JSON under construction.
* @param name Pointer to null-terminated string or null for unnamed.
* @param value Value of the property.
* @return Pointer to the new end of JSON under construction. */
char* json_double(char* dest, const char* name, double value);
/** @ } */
#ifdef __cplusplus
}
#endif
#endif /* _TINY_JSON_H_ */
+1 -1
+9 -1
View File
@@ -217,6 +217,14 @@ def print_man_environment_tail():
],
"''", True)
print_man_env_option(
"FEX_PORTABLE",
[
"Allows FEX to run without installation. Global locations for configuration and binfmt_misc are ignored. These files are instead read from <FEXInterpreterPath>/fex-emu/ by default.",
"For further customization, see FEX_APP_CONFIG_LOCATION and FEX_APP_DATA_LOCATION."
],
"''", True)
def print_man_header():
header ='''.Dd {0}
.Dt FEX
@@ -393,7 +401,7 @@ def print_parse_argloader_options(options):
conversion_func = "FEXCore::Config::Handler::{0}(".format(op_vals["ArgumentHandler"])
if (value_type == "str"):
NeedsString = True
conversion_func = "("
conversion_func = "std::move("
if (value_type == "bool"):
# boolean values need a decimal specifier. Otherwise fmt prints strings.
conversion_func = "fextl::fmt::format(\"{:d}\", "
+40 -23
View File
@@ -125,21 +125,36 @@ def parse_ops(ops):
RHS = EqualSplit[0].strip()
if len(EqualSplit) > 1:
OpDef.HasDest = True
LHS = EqualSplit[0].strip()
RHS = EqualSplit[1].strip()
# Parse the destination, must be one type of SSA, GPR, or FPR
ResultType = EqualSplit[0].strip()
if ResultType == "SSA":
OpDef.DestType = "SSA" # We don't know this type right now
elif ResultType == "GPR":
OpDef.DestType = "GPR"
elif ResultType == "GPRPair":
OpDef.DestType = "GPRPair"
elif ResultType == "FPR":
OpDef.DestType = "FPR"
if ":" in LHS:
# Named destinations. This is a hack, but so is the entire
# multi-destination support bolten onto the old IR...
#
# Named destinations require side effects because they break
# SSA hard. Validate that.
assert("HasSideEffects" in op_val and op_val["HasSideEffects"])
for Dest in LHS.split(","):
Dest = Dest.strip()
DType, Name = Dest.split(":$")
# If the destination appears also as a source, it is
# read-modify-write.
if Dest in RHS:
# Turn RMW into an in/out source
RHS = RHS.replace(Dest.strip(), f"{DType}:$Inout{Name}")
else:
# Turn named destinations into an out source.
RHS += f", {DType}:$Out{Name}"
else:
ExitError("Unknown destination class type {}. Needs to be one of {SSA, GPR, GPRPair, FPR}".format(ResultType))
# Single anonymous destination
if LHS not in ["SSA", "GPR", "GPRPair", "FPR"]:
ExitError(f"Unknown destination class type {LHS}. Needs to be one of SSA, GPR, GPRPair, FPR")
OpDef.HasDest = True
OpDef.DestType = LHS
# IR Op needs to start with a name
RHS = RHS.split(" ", 1)
@@ -308,8 +323,8 @@ def print_ir_structs(defines):
output_file.write("struct __attribute__((packed)) IROp_Header {\n")
output_file.write("\tvoid* Data[0];\n")
output_file.write("\tIROps Op;\n\n")
output_file.write("\tuint8_t Size;\n")
output_file.write("\tuint8_t ElementSize;\n")
output_file.write("\tIR::OpSize Size;\n")
output_file.write("\tIR::OpSize ElementSize;\n")
output_file.write("\ttemplate<typename T>\n")
output_file.write("\tT const* C() const { return reinterpret_cast<T const*>(Data); }\n")
@@ -615,20 +630,19 @@ def print_ir_allocator_helpers():
output_file.write("\t\treturn IRPair<T>{Op, CreateNode(&Op->Header)};\n")
output_file.write("\t}\n\n")
output_file.write("\tuint8_t GetOpSize(const OrderedNode *Op) const {\n")
output_file.write("\tIR::OpSize GetOpSize(const OrderedNode *Op) const {\n")
output_file.write("\t\tauto HeaderOp = Op->Header.Value.GetNode(DualListData.DataBegin());\n")
output_file.write("\t\treturn HeaderOp->Size;\n")
output_file.write("\t}\n\n")
output_file.write("\tuint8_t GetOpElementSize(const OrderedNode *Op) const {\n")
output_file.write("\tIR::OpSize GetOpElementSize(const OrderedNode *Op) const {\n")
output_file.write("\t\tauto HeaderOp = Op->Header.Value.GetNode(DualListData.DataBegin());\n")
output_file.write("\t\treturn HeaderOp->ElementSize;\n")
output_file.write("\t}\n\n")
output_file.write("\tuint8_t GetOpElements(const OrderedNode *Op) const {\n")
output_file.write("\t\tauto HeaderOp = Op->Header.Value.GetNode(DualListData.DataBegin());\n")
output_file.write("\t\tLOGMAN_THROW_A_FMT(OpHasDest(Op), \"Op {} has no dest\\n\", GetName(HeaderOp->Op));\n")
output_file.write("\t\treturn HeaderOp->Size / HeaderOp->ElementSize;\n")
output_file.write("\t\tLOGMAN_THROW_A_FMT(OpHasDest(Op), \"Op {} has no dest\\n\", GetOpName(Op));\n")
output_file.write("\t\treturn IR::OpSizeToSize(GetOpSize(Op)) / IR::OpSizeToSize(GetOpElementSize(Op));\n")
output_file.write("\t}\n\n")
output_file.write("\tbool OpHasDest(const OrderedNode *Op) const {\n")
@@ -684,8 +698,12 @@ def print_ir_allocator_helpers():
# We gather the "has x87?" flag as we go. This saves the user from
# having to keep track of whether they emitted any x87.
# Also changes the mmx state to X87.
if op.LoweredX87:
output_file.write("\t\tRecordX87Use();\n")
output_file.write(
"\t\tif(MMXState == MMXState_MMX) ChgStateMMX_X87();\n"
)
output_file.write("\t\tauto _Op = AllocateOp<IROp_{}, IROps::OP_{}>();\n".format(op.Name, op.Name.upper()))
@@ -709,11 +727,11 @@ def print_ir_allocator_helpers():
# We can only infer a size if we have arguments
if op.DestSize == None:
# We need to infer destination size
output_file.write("\t\tuint8_t InferSize = 0;\n")
output_file.write("\t\tIR::OpSize InferSize = OpSize::iUnsized;\n")
if len(op.Arguments) != 0:
for arg in op.Arguments:
if arg.IsSSA:
output_file.write("\t\tuint8_t Size{} = GetOpSize({});\n".format(arg.Name, arg.Name))
output_file.write("\t\tauto Size{} = GetOpSize({});\n".format(arg.Name, arg.Name))
for arg in op.Arguments:
if arg.IsSSA:
output_file.write("\t\tInferSize = std::max(InferSize, Size{});\n".format(arg.Name))
@@ -726,7 +744,7 @@ def print_ir_allocator_helpers():
output_file.write("\t\t_Op.first->Header.Size = {};\n".format(op.DestSize))
if op.NumElements == None:
output_file.write("\t\t_Op.first->Header.ElementSize = _Op.first->Header.Size / ({});\n".format(1))
output_file.write("\t\t_Op.first->Header.ElementSize = _Op.first->Header.Size;\n")
else:
output_file.write("\t\t_Op.first->Header.ElementSize = _Op.first->Header.Size / ({});\n".format(op.NumElements))
@@ -811,4 +829,3 @@ print_ir_dispatcher_defs()
print_ir_dispatcher_dispatch()
output_dispatch_file.close()
+41 -30
View File
@@ -1,3 +1,4 @@
include(GNUInstallDirs)
set (MAN_DIR share/man CACHE PATH "MAN_DIR")
set (FEXCORE_BASE_SRCS
@@ -85,7 +86,6 @@ set (SRCS
Common/SoftFloat-3e/s_f32UIToCommonNaN.c
Interface/Context/Context.cpp
Interface/Core/LookupCache.cpp
Interface/Core/BlockSamplingData.cpp
Interface/Core/Core.cpp
Interface/Core/CPUBackend.cpp
Interface/Core/CPUID.cpp
@@ -105,20 +105,19 @@ set (SRCS
Interface/Core/ArchHelpers/Arm64Emitter.cpp
Interface/Core/Dispatcher/Dispatcher.cpp
Interface/Core/Interpreter/Fallbacks/InterpreterFallbacks.cpp
Interface/Core/JIT/Arm64/JIT.cpp
Interface/Core/JIT/Arm64/ALUOps.cpp
Interface/Core/JIT/Arm64/AtomicOps.cpp
Interface/Core/JIT/Arm64/BranchOps.cpp
Interface/Core/JIT/Arm64/ConversionOps.cpp
Interface/Core/JIT/Arm64/EncryptionOps.cpp
Interface/Core/JIT/Arm64/MemoryOps.cpp
Interface/Core/JIT/Arm64/MiscOps.cpp
Interface/Core/JIT/Arm64/MoveOps.cpp
Interface/Core/JIT/Arm64/VectorOps.cpp
Interface/Core/JIT/Arm64/Arm64Relocations.cpp
Interface/Core/JIT/JIT.cpp
Interface/Core/JIT/ALUOps.cpp
Interface/Core/JIT/AtomicOps.cpp
Interface/Core/JIT/BranchOps.cpp
Interface/Core/JIT/ConversionOps.cpp
Interface/Core/JIT/EncryptionOps.cpp
Interface/Core/JIT/MemoryOps.cpp
Interface/Core/JIT/MiscOps.cpp
Interface/Core/JIT/MoveOps.cpp
Interface/Core/JIT/VectorOps.cpp
Interface/Core/JIT/Arm64Relocations.cpp
Interface/Core/X86Tables/BaseTables.cpp
Interface/Core/X86Tables/DDDTables.cpp
Interface/Core/X86Tables/EVEXTables.cpp
Interface/Core/X86Tables/H0F38Tables.cpp
Interface/Core/X86Tables/H0F3ATables.cpp
Interface/Core/X86Tables/PrimaryGroupTables.cpp
@@ -127,8 +126,6 @@ set (SRCS
Interface/Core/X86Tables/SecondaryTables.cpp
Interface/Core/X86Tables/VEXTables.cpp
Interface/Core/X86Tables/X87Tables.cpp
Interface/Core/X86Tables/XOPTables.cpp
Interface/HLE/Thunks/Thunks.cpp
Interface/GDBJIT/GDBJIT.cpp
Interface/IR/AOTIR.cpp
Interface/IR/IRDumper.cpp
@@ -139,7 +136,6 @@ set (SRCS
Interface/IR/Passes/IRValidation.cpp
Interface/IR/Passes/RAValidation.cpp
Interface/IR/Passes/RedundantFlagCalculationElimination.cpp
Interface/IR/Passes/DeadStoreElimination.cpp
Interface/IR/Passes/RegisterAllocationPass.cpp
Interface/IR/Passes/x87StackOptimizationPass.cpp
Utils/Telemetry.cpp
@@ -181,7 +177,11 @@ endif()
# Some defines for the softfloat library
list(APPEND DEFINES "-DSOFTFLOAT_BUILTIN_CLZ")
set (LIBS fmt::fmt vixl xxHash::xxhash FEXHeaderUtils CodeEmitter)
set (LIBS fmt::fmt xxHash::xxhash FEXHeaderUtils CodeEmitter)
if (ENABLE_VIXL_DISASSEMBLER OR ENABLE_VIXL_SIMULATOR)
list (APPEND LIBS vixl)
endif()
if (NOT MINGW_BUILD)
list (APPEND LIBS dl)
@@ -192,14 +192,6 @@ else()
endif()
endif()
if (ENABLE_JEMALLOC)
list (APPEND LIBS FEX_jemalloc)
endif()
if (ENABLE_JEMALLOC_GLIBC_ALLOC)
list (APPEND LIBS FEX_jemalloc_glibc)
endif()
# Generate config
configure_file(
${CMAKE_CURRENT_SOURCE_DIR}/Interface/Config/Config.json.in
@@ -369,11 +361,30 @@ AddLibrary(${PROJECT_NAME} STATIC)
AddLibrary(${PROJECT_NAME}_shared SHARED)
if (NOT MINGW_BUILD)
install(TARGETS ${PROJECT_NAME} ${PROJECT_NAME}_shared
install(TARGETS ${PROJECT_NAME}_shared
LIBRARY
DESTINATION lib
COMPONENT Libraries
ARCHIVE
DESTINATION lib
DESTINATION ${CMAKE_INSTALL_LIBDIR}
COMPONENT Libraries)
endif()
# Meta-library to link jemalloc libraries enabled in the build configuration.
# Only needed for targets that run emulation. For others, use JemallocDummy.
add_library(JemallocLibs STATIC Utils/AllocatorHooks.cpp)
if (ENABLE_JEMALLOC)
target_compile_definitions(JemallocLibs PRIVATE ENABLE_JEMALLOC=1 JEMALLOC_NO_RENAME=1)
target_link_libraries(JemallocLibs PUBLIC FEX_jemalloc)
endif()
if (ENABLE_JEMALLOC_GLIBC_ALLOC)
set_source_files_properties(Interface/HLE/Thunks/Thunks.cpp PROPERTIES COMPILE_DEFINITIONS ENABLE_JEMALLOC_GLIBC=1)
target_link_libraries(JemallocLibs INTERFACE FEX_jemalloc_glibc)
endif()
if (NOT MINGW_BUILD)
# Dummy project to use for host tools.
# This overrides use of jemalloc in FEXCore with the normal glibc allocator.
add_library(JemallocDummy STATIC Utils/AllocatorHooks.cpp)
target_include_directories(JemallocDummy PRIVATE "${PROJECT_SOURCE_DIR}/include/")
endif()
# The shared library should always link enabled jemalloc libraries
target_link_libraries(${PROJECT_NAME}_shared JemallocLibs)
+4 -4
View File
@@ -55,7 +55,7 @@ void JITSymbols::RegisterJITSpace(const void* HostAddr, uint32_t CodeSize) {
}
// Buffered JIT symbols.
void JITSymbols::Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint64_t GuestAddr, uint32_t CodeSize) {
void JITSymbols::Register(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint64_t GuestAddr, uint32_t CodeSize) {
if (fd == -1) {
return;
}
@@ -79,7 +79,7 @@ void JITSymbols::Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, u
WriteBuffer(Buffer);
}
void JITSymbols::Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name, uintptr_t Offset) {
void JITSymbols::Register(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name, uintptr_t Offset) {
if (fd == -1) {
return;
}
@@ -104,7 +104,7 @@ void JITSymbols::Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, u
WriteBuffer(Buffer);
}
void JITSymbols::RegisterNamedRegion(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name) {
void JITSymbols::RegisterNamedRegion(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name) {
if (fd == -1) {
return;
}
@@ -128,7 +128,7 @@ void JITSymbols::RegisterNamedRegion(Core::JITSymbolBuffer* Buffer, const void*
WriteBuffer(Buffer);
}
void JITSymbols::WriteBuffer(Core::JITSymbolBuffer* Buffer, bool ForceWrite) {
void JITSymbols::WriteBuffer(FEXCore::JITSymbolBuffer* Buffer, bool ForceWrite) {
auto Now = std::chrono::steady_clock::now();
if (!ForceWrite) {
if (((Buffer->LastWrite - Now) < Buffer->MAXIMUM_THRESHOLD) && Buffer->Offset < Buffer->NEEDS_WRITE_DISTANCE) {
+26 -6
View File
@@ -11,6 +11,26 @@
#include <string_view>
namespace FEXCore {
// Buffered JIT symbol tracking.
struct JITSymbolBuffer {
// Maximum buffer size to ensure we are a page in size.
constexpr static size_t BUFFER_SIZE = 4096 - (8 * 2);
// Maximum distance until the end of the buffer to do a write.
constexpr static size_t NEEDS_WRITE_DISTANCE = BUFFER_SIZE - 64;
// Maximum time threshhold to wait before a buffer write occurs.
constexpr static std::chrono::milliseconds MAXIMUM_THRESHOLD {100};
JITSymbolBuffer()
: LastWrite {std::chrono::steady_clock::now()} {}
// stead_clock to ensure a monotonic increasing clock.
// In highly stressed situations this can still cause >2% CPU time in vdso_clock_gettime.
// If we need lower CPU time when JIT symbols are enabled then FEX can read the cycle counter directly.
std::chrono::steady_clock::time_point LastWrite {};
size_t Offset {};
char Buffer[BUFFER_SIZE] {};
};
static_assert(sizeof(JITSymbolBuffer) == 4096, "Ensure this is one page in size");
class JITSymbols final {
public:
JITSymbols();
@@ -21,16 +41,16 @@ public:
void RegisterJITSpace(const void* HostAddr, uint32_t CodeSize);
// Allocate JIT buffer.
static fextl::unique_ptr<Core::JITSymbolBuffer> AllocateBuffer() {
return fextl::make_unique<Core::JITSymbolBuffer>();
static fextl::unique_ptr<FEXCore::JITSymbolBuffer> AllocateBuffer() {
return fextl::make_unique<FEXCore::JITSymbolBuffer>();
}
void Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint64_t GuestAddr, uint32_t CodeSize);
void Register(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name, uintptr_t Offset);
void RegisterNamedRegion(Core::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name);
void Register(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint64_t GuestAddr, uint32_t CodeSize);
void Register(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name, uintptr_t Offset);
void RegisterNamedRegion(FEXCore::JITSymbolBuffer* Buffer, const void* HostAddr, uint32_t CodeSize, std::string_view Name);
private:
int fd {-1};
void WriteBuffer(Core::JITSymbolBuffer* Buffer, bool ForceWrite = false);
void WriteBuffer(FEXCore::JITSymbolBuffer* Buffer, bool ForceWrite = false);
};
} // namespace FEXCore
+39 -1
View File
@@ -160,7 +160,31 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
return extF80_rem(state, lhs, rhs);
/*
* FPREM is not an IEEE-754 remainder. From the spec:
*
* Computes the remainder obtained from dividing the value in the ST(0)
* register (the dividend) by the value in the ST(1) register (the divisor
* or modulus), and stores the result in ST(0). The remainder represents the
* following value:
*
* Remainder := ST(0) − (Q * ST(1))
*
* Here, Q is an integer value that is obtained by truncating the
* floating-point number quotient of [ST(0) / ST(1)] toward zero.
*
* We implement this sequence literally. softfloat_round_minMag means
* "truncate towards zero".
*/
extFloat80_t quotient = extF80_div(state, lhs, rhs);
extFloat80_t Q = extF80_roundToInt(state, quotient, softfloat_round_minMag, true);
bool Q_zero = Q.signif == 0 && (Q.signExp & ~(1 << 15)) == 0;
if (Q_zero) {
return lhs;
} else {
return extF80_sub(state, lhs, extF80_mul(state, Q, rhs));
}
#endif
}
@@ -209,6 +233,10 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
// Zero is a special case, the significand for +/- 0 is +/- zero.
if (lhs.Exponent == 0x0 && lhs.Significand == 0x0) {
return lhs;
}
X80SoftFloat Tmp = lhs;
Tmp.Exponent = 0x3FFF;
Tmp.Sign = lhs.Sign;
@@ -232,6 +260,12 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
// Zero is a special case, the exponent is always -inf
if (lhs.Exponent == 0x0 && lhs.Significand == 0x0) {
X80SoftFloat Result(1, 0x7FFFUL, 0x8000'0000'0000'0000UL);
return Result;
}
int32_t TrueExp = lhs.Exponent - ExponentBias;
return i32_to_extF80(TrueExp);
#endif
@@ -262,6 +296,10 @@ struct FEX_PACKED X80SoftFloat {
return Result;
#else
extFloat80_t Zero {0, 0};
if (extF80_eq(state, lhs, Zero)) {
return lhs;
}
X80SoftFloat Int = FRNDINT(state, rhs, softfloat_round_minMag);
LIBRARY_PRECISION Src2_d = Int.ToFMax(state);
Src2_d = exp2l(Src2_d);
+33 -53
View File
@@ -43,7 +43,8 @@ namespace DefaultValues {
} // namespace DefaultValues
enum Paths {
PATH_DATA_DIR = 0,
PATH_DATA_DIR_LOCAL = 0,
PATH_DATA_DIR_GLOBAL,
PATH_CONFIG_DIR_LOCAL,
PATH_CONFIG_DIR_GLOBAL,
PATH_CONFIG_FILE_LOCAL,
@@ -53,8 +54,8 @@ enum Paths {
};
static std::array<fextl::string, Paths::PATH_LAST> Paths;
void SetDataDirectory(const std::string_view Path) {
Paths[PATH_DATA_DIR] = Path;
void SetDataDirectory(const std::string_view Path, bool Global) {
Paths[PATH_DATA_DIR_LOCAL + Global] = Path;
}
void SetConfigDirectory(const std::string_view Path, bool Global) {
@@ -73,15 +74,15 @@ const fextl::string& GetTelemetryDirectory() {
Path = TelemetryDirectory;
Path += "/";
} else {
Path = Config::GetDataDirectory() + "Telemetry/";
Path = Config::GetDataDirectory(false) + "Telemetry/";
}
}
return Path;
}
const fextl::string& GetDataDirectory() {
return Paths[PATH_DATA_DIR];
const fextl::string& GetDataDirectory(bool Global) {
return Paths[PATH_DATA_DIR_LOCAL + Global];
}
const fextl::string& GetConfigDirectory(bool Global) {
@@ -230,27 +231,26 @@ void Load() {
}
}
fextl::string ExpandPath(const fextl::string& ContainerPrefix, fextl::string PathName) {
fextl::string ExpandPath(const fextl::string& ContainerPrefix, const fextl::string& PathName) {
if (PathName.empty()) {
return {};
}
// Expand home if it exists
if (FHU::Filesystem::IsRelative(PathName)) {
fextl::string Home = getenv("HOME") ?: "";
// Home expansion only works if it is the first character
// This matches bash behaviour
if (PathName.at(0) == '~') {
PathName.replace(0, 1, Home);
return PathName;
if (PathName.starts_with("~/")) {
Home.append(PathName.begin() + 1, PathName.end());
return Home;
}
// Expand relative path to absolute
char ExistsTempPath[PATH_MAX];
char* RealPath = FHU::Filesystem::Absolute(PathName.c_str(), ExistsTempPath);
if (RealPath) {
PathName = RealPath;
if (RealPath && FHU::Filesystem::Exists(RealPath)) {
return RealPath;
}
// Only return if it exists
@@ -318,81 +318,61 @@ fextl::string FindContainerPrefix() {
void ReloadMetaLayer() {
Meta->Load();
// Do configuration option fix ups after everything is reloaded
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_CORE)) {
// Sanitize Core option
FEX_CONFIG_OPT(Core, CORE);
#if (_M_X86_64)
constexpr uint32_t MaxCoreNumber = 1;
#else
constexpr uint32_t MaxCoreNumber = 0;
#endif
if (Core > MaxCoreNumber) {
// Sanitize the core option by setting the core to the JIT if invalid
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_CORE, fextl::fmt::format("{}", static_cast<uint32_t>(FEXCore::Config::CONFIG_IRJIT)));
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_CACHEOBJECTCODECOMPILATION)) {
FEX_CONFIG_OPT(CacheObjectCodeCompilation, CACHEOBJECTCODECOMPILATION);
FEX_CONFIG_OPT(Core, CORE);
}
fextl::string ContainerPrefix {FindContainerPrefix()};
auto ExpandPathIfExists = [&ContainerPrefix](FEXCore::Config::ConfigOption Config, fextl::string PathName) {
auto NewPath = ExpandPath(ContainerPrefix, PathName);
const fextl::string ContainerPrefix {FindContainerPrefix()};
auto ExpandPathIfExists = [&ContainerPrefix](FEXCore::Config::ConfigOption Config, const fextl::string& PathName) {
const auto NewPath = ExpandPath(ContainerPrefix, PathName);
if (!NewPath.empty()) {
FEXCore::Config::EraseSet(Config, NewPath);
}
};
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_ROOTFS)) {
FEX_CONFIG_OPT(PathName, ROOTFS);
auto ExpandedString = ExpandPath(ContainerPrefix, PathName());
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_ROOTFS);
const auto ExpandedString = ExpandPath(ContainerPrefix, *PathName);
if (!ExpandedString.empty()) {
// Adjust the path if it ended up being relative
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_ROOTFS, ExpandedString);
} else if (!PathName().empty()) {
} else if (!PathName->empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
fextl::string NamedRootFS = GetDataDirectory() + "RootFS/" + PathName();
fextl::string NamedRootFS = GetDataDirectory(false) + "RootFS/" + *PathName;
if (FHU::Filesystem::Exists(NamedRootFS)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_ROOTFS, NamedRootFS);
}
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_THUNKHOSTLIBS)) {
FEX_CONFIG_OPT(PathName, THUNKHOSTLIBS);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKHOSTLIBS, PathName());
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_THUNKHOSTLIBS);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKHOSTLIBS, *PathName);
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_THUNKGUESTLIBS)) {
FEX_CONFIG_OPT(PathName, THUNKGUESTLIBS);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKGUESTLIBS, PathName());
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_THUNKGUESTLIBS);
ExpandPathIfExists(FEXCore::Config::CONFIG_THUNKGUESTLIBS, *PathName);
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_THUNKCONFIG)) {
FEX_CONFIG_OPT(PathName, THUNKCONFIG);
auto ExpandedString = ExpandPath(ContainerPrefix, PathName());
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_THUNKCONFIG);
const auto ExpandedString = ExpandPath(ContainerPrefix, *PathName);
if (!ExpandedString.empty()) {
// Adjust the path if it ended up being relative
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, ExpandedString);
} else if (!PathName().empty()) {
} else if (!PathName->empty()) {
// If the filesystem doesn't exist then let's see if it exists in the fex-emu folder
fextl::string NamedConfig = GetDataDirectory() + "ThunkConfigs/" + PathName();
fextl::string NamedConfig = GetDataDirectory(false) + "ThunkConfigs/" + *PathName;
if (FHU::Filesystem::Exists(NamedConfig)) {
FEXCore::Config::EraseSet(FEXCore::Config::CONFIG_THUNKCONFIG, NamedConfig);
}
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_OUTPUTLOG)) {
FEX_CONFIG_OPT(PathName, OUTPUTLOG);
if (PathName() != "stdout" && PathName() != "stderr" && PathName() != "server") {
ExpandPathIfExists(FEXCore::Config::CONFIG_OUTPUTLOG, PathName());
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_OUTPUTLOG);
if (*PathName != "stdout" && *PathName != "stderr" && *PathName != "server") {
ExpandPathIfExists(FEXCore::Config::CONFIG_OUTPUTLOG, *PathName);
}
}
if (FEXCore::Config::Exists(FEXCore::Config::CONFIG_DUMPIR) && !FEXCore::Config::Exists(FEXCore::Config::CONFIG_PASSMANAGERDUMPIR)) {
// If DumpIR is set but no PassManagerDumpIR configuration is set, then default to `afteropt`
FEX_CONFIG_OPT(PathName, DUMPIR);
if (PathName() != "no") {
const auto PathName = *Meta->Get(FEXCore::Config::CONFIG_DUMPIR);
if (*PathName != "no") {
EraseSet(FEXCore::Config::ConfigOption::CONFIG_PASSMANAGERDUMPIR,
fextl::fmt::format("{}", static_cast<uint64_t>(FEXCore::Config::PassManagerDumpIR::AFTEROPT)));
}
+17 -15
View File
@@ -1,19 +1,6 @@
{
"Options": {
"CPU": {
"Core": {
"Type": "uint32",
"Default": "FEXCore::Config::ConfigCore::CONFIG_IRJIT",
"TextDefault": "irjit",
"ShortArg": "c",
"Choices": [ "irjit", "host" ],
"ArgumentHandler": "CoreHandler",
"Desc": [
"Which CPU core to use",
"host only exists on x86_64",
"[irjit, host]"
]
},
"Multiblock": {
"Type": "bool",
"Default": "false",
@@ -142,7 +129,7 @@
},
"ThunkHostLibs": {
"Type": "str",
"Default": "@CMAKE_INSTALL_PREFIX@/lib/fex-emu/HostThunks/",
"Default": "@CMAKE_INSTALL_PREFIX@/@CMAKE_INSTALL_LIBDIR@/fex-emu/HostThunks/",
"ShortArg": "t",
"Desc": [
"Folder to find the host-side thunking libraries."
@@ -158,7 +145,7 @@
},
"ThunkHostLibs32": {
"Type": "str",
"Default": "@CMAKE_INSTALL_PREFIX@/lib/fex-emu/HostThunks_32/",
"Default": "@CMAKE_INSTALL_PREFIX@/@CMAKE_INSTALL_LIBDIR@/fex-emu/HostThunks_32/",
"Desc": [
"Folder to find the 32-bit host-side thunking libraries."
]
@@ -422,6 +409,14 @@
"Can be dangerous due to aligned loadstores through the same code now become non-atomic."
]
},
"StrictInProcessSplitLocks": {
"Type": "bool",
"Default": "false",
"Desc": [
"Strict global lock when handling an unaligned atomic that crosses a 16-byte or cacheline granularity",
"This is required to ensure a split-lock doesn't tear inside the process"
]
},
"TSOAutoMigration": {
"Type": "bool",
"Default": "true",
@@ -509,6 +504,13 @@
"Desc": [
"Override for a FEXServer socket path. Only useful for chroots."
]
},
"NeedsSeccomp": {
"Type": "bool",
"Default": "false",
"Desc": [
"Disables inline syscalls in order to support seccomp handling"
]
}
}
},
+5 -12
View File
@@ -8,6 +8,7 @@
#include <FEXCore/Core/CPUID.h>
#include <FEXCore/Core/HostFeatures.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Core/Thunks.h>
#include "FEXCore/Debug/InternalThreadState.h"
#include <string.h>
@@ -23,14 +24,6 @@ fextl::unique_ptr<FEXCore::Context::Context> FEXCore::Context::Context::CreateNe
return fextl::make_unique<FEXCore::Context::ContextImpl>(Features);
}
void FEXCore::Context::ContextImpl::SetExitHandler(ExitHandler handler) {
CustomExitHandler = std::move(handler);
}
ExitHandler FEXCore::Context::ContextImpl::GetExitHandler() const {
return CustomExitHandler;
}
void FEXCore::Context::ContextImpl::CompileRIP(FEXCore::Core::InternalThreadState* Thread, uint64_t GuestRIP) {
CompileBlock(Thread->CurrentFrame, GuestRIP);
}
@@ -39,10 +32,6 @@ void FEXCore::Context::ContextImpl::CompileRIPCount(FEXCore::Core::InternalThrea
CompileBlock(Thread->CurrentFrame, GuestRIP, MaxInst);
}
void FEXCore::Context::ContextImpl::SetCustomCPUBackendFactory(CustomCPUFactoryType Factory) {
CustomCPUFactory = std::move(Factory);
}
void FEXCore::Context::ContextImpl::SetSignalDelegator(FEXCore::SignalDelegator* _SignalDelegation) {
SignalDelegation = _SignalDelegation;
}
@@ -52,6 +41,10 @@ void FEXCore::Context::ContextImpl::SetSyscallHandler(FEXCore::HLE::SyscallHandl
SourcecodeResolver = Handler->GetSourcecodeResolver();
}
void FEXCore::Context::ContextImpl::SetThunkHandler(FEXCore::ThunkHandler* Handler) {
ThunkHandler = Handler;
}
FEXCore::CPUID::FunctionResults FEXCore::Context::ContextImpl::RunCPUIDFunction(uint32_t Function, uint32_t Leaf) {
return CPUID.RunFunction(Function, Leaf);
}
+46 -85
View File
@@ -26,14 +26,8 @@
#include <stdint.h>
#include <atomic>
#include <condition_variable>
#include <functional>
#include <istream>
#include <memory>
#include <mutex>
#include <shared_mutex>
#include <stddef.h>
#include <queue>
namespace FEXCore {
class CodeLoader;
@@ -65,18 +59,21 @@ namespace Validation {
} // namespace FEXCore::IR
namespace FEXCore::Context {
enum CoreRunningMode {
MODE_RUN = 0,
MODE_SINGLESTEP = 1,
};
struct ExitFunctionLinkData {
uint64_t HostBranch;
uint64_t GuestRIP;
};
struct CustomIRResult {
void* Creator;
void* Data;
CustomIRResult(void* Creator, void* Data)
: Creator(Creator)
, Data(Data) {}
};
using BlockDelinkerFunc = void (*)(FEXCore::Core::CpuStateFrame* Frame, FEXCore::Context::ExitFunctionLinkData* Record);
constexpr uint32_t TSC_SCALE = 128;
constexpr uint32_t TSC_SCALE_MAXIMUM = 1'000'000'000; ///< 1Ghz
class ContextImpl final : public FEXCore::Context::Context {
@@ -84,18 +81,11 @@ public:
// Context base class implementation.
bool InitCore() override;
void SetExitHandler(ExitHandler handler) override;
ExitHandler GetExitHandler() const override;
ExitReason RunUntilExit(FEXCore::Core::InternalThreadState* Thread) override;
void ExecuteThread(FEXCore::Core::InternalThreadState* Thread) override;
void CompileRIP(FEXCore::Core::InternalThreadState* Thread, uint64_t GuestRIP) override;
void CompileRIPCount(FEXCore::Core::InternalThreadState* Thread, uint64_t GuestRIP, uint64_t MaxInst) override;
void SetCustomCPUBackendFactory(CustomCPUFactoryType Factory) override;
void HandleCallback(FEXCore::Core::InternalThreadState* Thread, uint64_t RIP) override;
uint64_t RestoreRIPFromHostPC(FEXCore::Core::InternalThreadState* Thread, uint64_t HostPC) override;
@@ -118,33 +108,29 @@ public:
* Usecases:
* Parent thread Creation:
* - Thread = CreateThread(InitialRIP, InitialStack, nullptr, 0);
* - CTX->RunUntilExit(Thread);
* - CTX->ExecuteThread(Thread);
* OS thread Creation:
* - Thread = CreateThread(0, 0, NewState, PPID);
* - Thread->ExecutionThread = FEXCore::Threads::Thread::Create(ThreadHandler, Arg);
* - ThreadHandler calls `CTX->ExecutionThread(Thread)`
* - ThreadHandler calls `CTX->ExecuteThread(Thread)`
* OS fork (New thread created with a clone of thread state):
* - clone{2, 3}
* - Thread = CreateThread(0, 0, CopyOfThreadState, PPID);
* - ExecutionThread(Thread); // Starts executing without creating another host thread
* - ExecuteThread(Thread); // Starts executing without creating another host thread
* Thunk callback executing guest code from native host thread
* - Thread = CreateThread(0, 0, NewState, PPID);
* - InitializeThreadTLSData(Thread);
* - HandleCallback(Thread, RIP);
*/
FEXCore::Core::InternalThreadState*
CreateThread(uint64_t InitialRIP, uint64_t StackPointer, FEXCore::Core::CPUState* NewThreadState, uint64_t ParentTID) override;
// Public for threading
void ExecutionThread(FEXCore::Core::InternalThreadState* Thread) override;
CreateThread(uint64_t InitialRIP, uint64_t StackPointer, const FEXCore::Core::CPUState* NewThreadState, uint64_t ParentTID) override;
/**
* @brief Destroys this FEX thread object and stops tracking it internally
*
* @param Thread The internal FEX thread state object
*/
void DestroyThread(FEXCore::Core::InternalThreadState* Thread, bool NeedsTLSUninstall) override;
void DestroyThread(FEXCore::Core::InternalThreadState* Thread) override;
#ifndef _WIN32
void LockBeforeFork(FEXCore::Core::InternalThreadState* Thread) override;
@@ -152,6 +138,7 @@ public:
#endif
void SetSignalDelegator(FEXCore::SignalDelegator* SignalDelegation) override;
void SetSyscallHandler(FEXCore::HLE::SyscallHandler* Handler) override;
void SetThunkHandler(FEXCore::ThunkHandler* Handler) override;
FEXCore::CPUID::FunctionResults RunCPUIDFunction(uint32_t Function, uint32_t Leaf) override;
FEXCore::CPUID::XCRResults RunXCRFunction(uint32_t Function) override;
@@ -191,9 +178,10 @@ public:
bool IsAddressInCodeBuffer(FEXCore::Core::InternalThreadState* Thread, uintptr_t Address) const override;
// returns false if a handler was already registered
CustomIRResult AddCustomIREntrypoint(uintptr_t Entrypoint, CustomIREntrypointHandler Handler, void* Creator = nullptr, void* Data = nullptr);
std::optional<CustomIRResult>
AddCustomIREntrypoint(uintptr_t Entrypoint, CustomIREntrypointHandler Handler, void* Creator = nullptr, void* Data = nullptr);
void AppendThunkDefinitions(std::span<const FEXCore::IR::ThunkDefinition> Definitions) override;
void AddThunkTrampolineIRHandler(uintptr_t Entrypoint, uintptr_t GuestThunkEntrypoint) override;
public:
friend class FEXCore::HLE::SyscallHandler;
@@ -204,8 +192,8 @@ public:
friend class FEXCore::IR::Validation::IRValidation;
struct {
CoreRunningMode RunningMode {CoreRunningMode::MODE_RUN};
uint64_t VirtualMemSize {1ULL << 36};
uint64_t TSCScale = 0;
// Used if the JIT needs to have its interrupt fault code emitted.
bool NeedsPendingInterruptFaultCheck {false};
@@ -216,17 +204,15 @@ public:
FEX_CONFIG_OPT(Is64BitMode, IS64BIT_MODE);
FEX_CONFIG_OPT(TSOEnabled, TSOENABLED);
FEX_CONFIG_OPT(TSOAutoMigration, TSOAUTOMIGRATION);
FEX_CONFIG_OPT(VectorTSOEnabled, VECTORTSOENABLED);
FEX_CONFIG_OPT(MemcpySetTSOEnabled, MEMCPYSETTSOENABLED);
FEX_CONFIG_OPT(ABILocalFlags, ABILOCALFLAGS);
FEX_CONFIG_OPT(AOTIRCapture, AOTIRCAPTURE);
FEX_CONFIG_OPT(AOTIRGenerate, AOTIRGENERATE);
FEX_CONFIG_OPT(AOTIRLoad, AOTIRLOAD);
FEX_CONFIG_OPT(SMCChecks, SMCCHECKS);
FEX_CONFIG_OPT(Core, CORE);
FEX_CONFIG_OPT(MaxInstPerBlock, MAXINST);
FEX_CONFIG_OPT(RootFSPath, ROOTFS);
FEX_CONFIG_OPT(ThunkHostLibsPath, THUNKHOSTLIBS);
FEX_CONFIG_OPT(ThunkHostLibsPath32, THUNKHOSTLIBS32);
FEX_CONFIG_OPT(ThunkConfigFile, THUNKCONFIG);
FEX_CONFIG_OPT(GlobalJITNaming, GLOBALJITNAMING);
FEX_CONFIG_OPT(LibraryJITNaming, LIBRARYJITNAMING);
FEX_CONFIG_OPT(BlockJITNaming, BLOCKJITNAMING);
@@ -237,28 +223,21 @@ public:
FEX_CONFIG_OPT(DisableTelemetry, DISABLETELEMETRY);
FEX_CONFIG_OPT(DisableVixlIndirectCalls, DISABLE_VIXL_INDIRECT_RUNTIME_CALLS);
FEX_CONFIG_OPT(SmallTSCScale, SMALLTSCSCALE);
FEX_CONFIG_OPT(StrictInProcessSplitLocks, STRICTINPROCESSSPLITLOCKS);
} Config;
std::atomic_bool CoreShuttingDown {false};
FEXCore::ForkableSharedMutex CodeInvalidationMutex;
uint32_t StrictSplitLockMutex {};
FEXCore::HostFeatures HostFeatures;
// CPUID depends on HostFeatures so needs to be initialized after that.
FEXCore::CPUIDEmu CPUID;
FEXCore::HLE::SyscallHandler* SyscallHandler {};
FEXCore::HLE::SourcecodeResolver* SourcecodeResolver {};
fextl::unique_ptr<FEXCore::ThunkHandler> ThunkHandler;
FEXCore::ThunkHandler* ThunkHandler {};
fextl::unique_ptr<FEXCore::CPU::Dispatcher> Dispatcher;
CustomCPUFactoryType CustomCPUFactory;
FEXCore::Context::ExitHandler CustomExitHandler;
#ifdef BLOCKSTATS
fextl::unique_ptr<FEXCore::BlockSamplingData> BlockData;
#endif
SignalDelegator* SignalDelegation {};
X86GeneratedCode X86CodeGen;
@@ -310,32 +289,12 @@ public:
CompileCodeResult CompileCode(FEXCore::Core::InternalThreadState* Thread, uint64_t GuestRIP, uint64_t MaxInst = 0);
uintptr_t CompileBlock(FEXCore::Core::CpuStateFrame* Frame, uint64_t GuestRIP, uint64_t MaxInst = 0);
// Used for thread creation from syscalls
/**
* @brief Initializes TID, PID and TLS data for a thread
*
* @param Thread The internal FEX thread state object
*/
void InitializeThreadTLSData(FEXCore::Core::InternalThreadState* Thread);
void CopyMemoryMapping(FEXCore::Core::InternalThreadState* ParentThread, FEXCore::Core::InternalThreadState* ChildThread);
uint8_t GetGPRSize() const {
return Config.Is64BitMode ? 8 : 4;
IR::OpSize GetGPROpSize() const {
return Config.Is64BitMode ? IR::OpSize::i64Bit : IR::OpSize::i32Bit;
}
FEXCore::JITSymbols Symbols;
void GetVDSOSigReturn(VDSOSigReturn* VDSOPointers) override {
if (VDSOPointers->VDSO_kernel_sigreturn == nullptr) {
VDSOPointers->VDSO_kernel_sigreturn = reinterpret_cast<void*>(X86CodeGen.sigreturn_32);
}
if (VDSOPointers->VDSO_kernel_rt_sigreturn == nullptr) {
VDSOPointers->VDSO_kernel_rt_sigreturn = reinterpret_cast<void*>(X86CodeGen.rt_sigreturn_32);
}
}
FEXCore::Utils::PooledAllocatorVirtual OpDispatcherAllocator;
FEXCore::Utils::PooledAllocatorVirtual FrontendAllocator;
@@ -344,27 +303,21 @@ public:
return AtomicTSOEmulationEnabled;
}
// If atomic-based TSO emulation is enabled for vector operations.
bool IsVectorAtomicTSOEnabled() const {
return VectorAtomicTSOEmulationEnabled;
}
// If atomic-based TSO emulation is enabled for memcpy operations.
bool IsMemcpyAtomicTSOEnabled() const {
return MemcpyAtomicTSOEmulationEnabled;
}
void SetHardwareTSOSupport(bool HardwareTSOSupported) override {
SupportsHardwareTSO = HardwareTSOSupported;
UpdateAtomicTSOEmulationConfig();
}
// Returns if Software TSO emulation is required.
// NOTE: This doesn't necessary return if Atomic-based TSO is currently enabled.
// This will still return true if on a single thread and TSO is currently disabled.
//
// This is to ensure that if early initialization checks CPU features and TSO /could/ be enabled, that
// we return consistent results.
//
// To check if Atomic TSO is currently enabled in the JIT, use `IsAtomicTSOEnabled` instead.
bool SoftwareTSORequired() const {
if (SupportsHardwareTSO) {
return false;
}
return Config.TSOEnabled;
}
void EnableExitOnHLT() override {
ExitOnHLT = true;
}
@@ -378,9 +331,15 @@ protected:
if (SupportsHardwareTSO) {
// If the hardware supports TSO then we don't need to emulate it through atomics.
AtomicTSOEmulationEnabled = false;
VectorAtomicTSOEmulationEnabled = false;
MemcpyAtomicTSOEmulationEnabled = false;
} else {
// Atomic TSO emulation only enabled if the config option is enabled.
AtomicTSOEmulationEnabled = (IsMemoryShared || !Config.TSOAutoMigration) && Config.TSOEnabled;
// Atomic vector TSO emulation only enabled if TSO emulation is enabled and also vector TSO is enabled.
VectorAtomicTSOEmulationEnabled = (IsMemoryShared || !Config.TSOAutoMigration) && Config.TSOEnabled && Config.VectorTSOEnabled;
// Atomic memcpy TSO emulation only enabled if TSO emulation is enabled and also memcpy TSO is enabled.
MemcpyAtomicTSOEmulationEnabled = (IsMemoryShared || !Config.TSOAutoMigration) && Config.TSOEnabled && Config.MemcpySetTSOEnabled;
}
}
@@ -399,10 +358,12 @@ private:
IR::AOTIRCaptureCache IRCaptureCache;
fextl::unique_ptr<FEXCore::CodeSerialize::CodeObjectSerializeService> CodeObjectCacheService;
bool StartPaused = false;
bool IsMemoryShared = false;
bool SupportsHardwareTSO = false;
bool AtomicTSOEmulationEnabled = true;
bool VectorAtomicTSOEmulationEnabled = false;
bool MemcpyAtomicTSOEmulationEnabled = false;
bool ExitOnHLT = false;
FEX_CONFIG_OPT(AppFilename, APP_FILENAME);
@@ -4,7 +4,6 @@
#include "FEXCore/Utils/AllocatorHooks.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Context/Context.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Utils/LogManager.h>
@@ -29,11 +28,12 @@ namespace FEXCore::CPU {
namespace x64 {
#ifndef _M_ARM_64EC
// All but x19 and x29 are caller saved
// Note that rax/rdx are rearranged here so we can coalesce cmpxchg.
constexpr std::array<ARMEmitter::Register, 18> SRA = {
ARMEmitter::Reg::r4,
ARMEmitter::Reg::r7,
ARMEmitter::Reg::r5,
ARMEmitter::Reg::r6,
ARMEmitter::Reg::r7,
ARMEmitter::Reg::r8,
ARMEmitter::Reg::r9,
ARMEmitter::Reg::r10,
@@ -194,12 +194,12 @@ namespace x64 {
} // namespace x64
namespace x32 {
// All but x19 and x29 are caller saved
// All but x19 and x29 are caller saved. eax/edx rearranged for cmpxchg.
constexpr std::array<ARMEmitter::Register, 10> SRA = {
ARMEmitter::Reg::r4,
ARMEmitter::Reg::r7,
ARMEmitter::Reg::r5,
ARMEmitter::Reg::r6,
ARMEmitter::Reg::r7,
ARMEmitter::Reg::r8,
ARMEmitter::Reg::r9,
ARMEmitter::Reg::r10,
@@ -373,6 +373,20 @@ Arm64Emitter::Arm64Emitter(FEXCore::Context::ContextImpl* ctx, void* EmissionPtr
}
}
FEXCore::X86State::X86Reg Arm64Emitter::GetX86RegRelationToARMReg(ARMEmitter::Register Reg) {
for (size_t i = 0; i < StaticRegisters.size(); ++i) {
const auto& RegI = StaticRegisters[i];
if (RegI == Reg) {
// X86 Registers are mapped linerally from the StaticRegisters span.
// Directly correlating Enum index to span index.
return static_cast<FEXCore::X86State::X86Reg>(FEXCore::ToUnderlying(FEXCore::X86State::X86Reg::REG_RAX) + i);
}
}
// Unmapped register.
return FEXCore::X86State::X86Reg::REG_INVALID;
}
void Arm64Emitter::LoadConstant(ARMEmitter::Size s, ARMEmitter::Register Reg, uint64_t Constant, bool NOPPad) {
bool Is64Bit = s == ARMEmitter::Size::i64Bit;
int Segments = Is64Bit ? 4 : 2;
@@ -4,11 +4,6 @@
#include "FEXCore/Utils/EnumUtils.h"
#include "Interface/Core/ObjectCache/Relocations.h"
#include <aarch64/assembler-aarch64.h>
#include <aarch64/constants-aarch64.h>
#include <aarch64/cpu-aarch64.h>
#include <aarch64/operands-aarch64.h>
#include <platform-vixl.h>
#ifdef VIXL_DISASSEMBLER
#include <aarch64/disasm-aarch64.h>
#endif
@@ -17,6 +12,7 @@
#include <aarch64/simulator-constants-aarch64.h>
#endif
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Config/Config.h>
#include <FEXCore/fextl/vector.h>
#include <CodeEmitter/Emitter.h>
@@ -106,6 +102,10 @@ protected:
void FillSpecialRegs(ARMEmitter::Register TmpReg, ARMEmitter::Register TmpReg2, bool SetFIZ, bool SetPredRegs);
// Correlate an ARM register back to an x86 register index.
// Returning REG_INVALID if there was no mapping.
FEXCore::X86State::X86Reg GetX86RegRelationToARMReg(ARMEmitter::Register Reg);
// NOTE: These functions WILL clobber the register TMP4 if AVX support is enabled
// and FPRs are being spilled or filled. If only GPRs are spilled/filled, then
// TMP4 is left alone.
@@ -1,51 +0,0 @@
// SPDX-License-Identifier: MIT
#include "Interface/Core/BlockSamplingData.h"
#include <FEXCore/Utils/LogManager.h>
#include <cstring>
#include <fstream>
#include <utility>
namespace FEXCore {
void BlockSamplingData::DumpBlockData() {
std::fstream Output;
Output.open("output.csv", std::fstream::out | std::fstream::binary);
if (!Output.is_open()) {
return;
}
Output << "Entry, Min, Max, Total, Calls, Average" << std::endl;
for (auto it : SamplingMap) {
if (!it.second->TotalCalls) {
continue;
}
Output << "0x" << std::hex << it.first << ", " << std::dec << it.second->Min << ", " << std::dec << it.second->Max << ", " << std::dec
<< it.second->TotalTime << ", " << std::dec << it.second->TotalCalls << ", " << std::dec
<< ((double)it.second->TotalTime / (double)it.second->TotalCalls) << std::endl;
}
Output.close();
LogMan::Msg::DFmt("Dumped {} blocks of sampling data", SamplingMap.size());
}
BlockSamplingData::BlockData* BlockSamplingData::GetBlockData(uint64_t RIP) {
auto it = SamplingMap.find(RIP);
if (it != SamplingMap.end()) {
return it->second;
}
BlockData* NewData = new BlockData {};
memset(NewData, 0, sizeof(BlockData));
NewData->Min = ~0ULL;
SamplingMap[RIP] = NewData;
return NewData;
}
BlockSamplingData::~BlockSamplingData() {
DumpBlockData();
for (auto it : SamplingMap) {
delete it.second;
}
SamplingMap.clear();
}
} // namespace FEXCore
@@ -1,25 +0,0 @@
// SPDX-License-Identifier: MIT
#pragma once
#include <cstdint>
#include <unordered_map>
namespace FEXCore {
class BlockSamplingData {
public:
struct BlockData {
uint64_t Start, End;
uint64_t Min, Max;
uint64_t TotalTime;
uint64_t TotalCalls;
};
BlockData* GetBlockData(uint64_t RIP);
~BlockSamplingData();
void DumpBlockData();
private:
std::unordered_map<uint64_t, BlockData*> SamplingMap;
};
} // namespace FEXCore
@@ -48,11 +48,6 @@ namespace CPU {
CPUBackend(FEXCore::Core::InternalThreadState* ThreadState, size_t InitialCodeSize, size_t MaxCodeSize);
virtual ~CPUBackend();
/**
* @return The name of this backend
*/
[[nodiscard]]
virtual fextl::string GetName() = 0;
struct CompiledCode {
// Where this code block begins.
@@ -124,9 +119,6 @@ namespace CPU {
*
* This is a thread specific compilation unit since there is one CPUBackend per guest thread
*
* If NeedsOpDispatch is returning false then IR and DebugData may be null and the expectation is that the code will still compile
* FEXCore::Core::ThreadState* is valid at the time of compilation.
*
* @param IR - IR that maps to the IR for this RIP
* @param DebugData - Debug data that is available for this IR indirectly
*
@@ -149,26 +141,6 @@ namespace CPU {
return nullptr;
}
/**
* @brief Function for mapping memory in to the CPUBackend's visible space. Allows setting up virtual mappings if required
*
* @return Currently unused
*/
[[nodiscard]]
virtual void* MapRegion(void* HostPtr, uint64_t GuestPtr, uint64_t Size) = 0;
/**
* @brief Lets FEXCore know if this CPUBackend needs IR and DebugData for CompileCode
*
* This is useful if the FEXCore Frontend hits an x86-64 instruction that isn't understood but can continue regardless
*
* This is useful for example, a VM based CPUbackend
*
* @return true if it needs the IR
*/
[[nodiscard]]
virtual bool NeedsOpDispatch() = 0;
virtual void ClearCache() {}
/**
+88 -45
View File
@@ -72,34 +72,63 @@ namespace ProductNames {
static const char ARM_Denver[] = "Nvidia Denver";
static const char ARM_Carmel[] = "Nvidia Carmel";
static const char ARM_Firestorm[] = "Apple Firestorm";
static const char ARM_Icestorm[] = "Apple Icestorm";
static const char ARM_Firestorm_M1[] = "Apple Firestorm (M1)";
static const char ARM_Icestorm_M1[] = "Apple Icestorm (M1)";
static const char ARM_Firestorm_M1Pro[] = "Apple Firestorm (M1 Pro)";
static const char ARM_Icestorm_M1Pro[] = "Apple Icestorm (M1 Pro)";
static const char ARM_Firestorm_M1Max[] = "Apple Firestorm (M1 Max)";
static const char ARM_Icestorm_M1Max[] = "Apple Icestorm (M1 Max)";
static const char ARM_Avalanche_M2[] = "Apple Avalanche (M2)";
static const char ARM_Blizzard_M2[] = "Apple Blizzard (M2)";
static const char ARM_Avalanche_M2Pro[] = "Apple Avalanche (M2 Pro)";
static const char ARM_Blizzard_M2Pro[] = "Apple Blizzard (M2 Pro)";
static const char ARM_Avalanche_M2Max[] = "Apple Avalanche (M2 Max)";
static const char ARM_Blizzard_M2Max[] = "Apple Blizzard (M2 Max)";
static const char ARM_ORYON_1[] = "Oryon-1";
#else
#endif
} // namespace ProductNames
static uint32_t GetCPUID() {
uint32_t GetCPUID_Syscall() {
uint32_t CPU {};
FHU::Syscalls::getcpu(&CPU, nullptr);
return CPU;
}
struct CPUFamily {
uint32_t Stepping : 4;
uint32_t Model : 4;
uint32_t ExtendedModel : 4;
uint32_t FamilyID : 4;
uint32_t ExtendedFamilyID : 8;
uint32_t ProcessorType : 4;
};
constexpr static uint32_t GenerateFamily(const CPUFamily Family) {
return Family.Stepping | (Family.Model << 4) | (Family.FamilyID << 8) | (Family.ProcessorType << 12) | (Family.ExtendedModel << 16) |
(Family.ExtendedFamilyID << 20);
}
#ifdef CPUID_AMD
constexpr uint32_t FAMILY_IDENTIFIER = 0 | // Stepping
(0xA << 4) | // Model
(0xF << 8) | // Family ID
(0 << 12) | // Processor type
(0 << 16) | // Extended model ID
(1 << 20); // Extended family ID
constexpr uint32_t FAMILY_IDENTIFIER = GenerateFamily(CPUFamily {
.Stepping = 0,
.Model = 0xA,
.ExtendedModel = 0,
.FamilyID = 0xF,
.ExtendedFamilyID = 1,
.ProcessorType = 0,
});
#else
constexpr uint32_t FAMILY_IDENTIFIER = 0 | // Stepping
(0x7 << 4) | // Model
(0x6 << 8) | // Family ID
(0 << 12) | // Processor type
(1 << 16) | // Extended model ID
(0x0 << 20); // Extended family ID
constexpr uint32_t FAMILY_IDENTIFIER = GenerateFamily(CPUFamily {
.Stepping = 1,
.Model = 6,
.ExtendedModel = 0xA,
.FamilyID = 6,
.ExtendedFamilyID = 0,
.ProcessorType = 0,
});
#endif
#ifdef _M_ARM_64
@@ -109,32 +138,27 @@ uint32_t GetCycleCounterFrequency() {
return Result;
}
uint32_t GetCPUID_TPIDRRO() {
uint64_t Result {};
__asm("mrs %[Res], TPIDRRO_EL0" : [Res] "=r"(Result));
return Result;
}
void CPUIDEmu::SetupHostHybridFlag() {
PerCPUData.resize(Cores);
uint64_t MIDR {};
for (size_t i = 0; i < Cores; ++i) {
std::error_code ec {};
fextl::string MIDRPath = fextl::fmt::format("/sys/devices/system/cpu/cpu{}/regs/identification/midr_el1", i);
std::array<char, 18> Data;
// Needs to be a fixed size since depending on kernel it will try to read a full page of data and fail
// Only read 18 bytes for a 64bit value prefixed with 0x
if (FEXCore::FileLoading::LoadFileToBuffer(MIDRPath, Data) == sizeof(Data)) {
uint64_t NewMIDR {};
std::string_view MIDRView(Data.data(), sizeof(Data));
if (FEXCore::StrConv::Conv(MIDRView, &NewMIDR)) {
if (MIDR != 0 && MIDR != NewMIDR) {
// CPU mismatch, claim hybrid
Hybrid = true;
}
// Truncate to 32-bits, top 32-bits are all reserved in MIDR
PerCPUData[i].ProductName = ProductNames::ARM_UNKNOWN;
PerCPUData[i].MIDR = NewMIDR;
MIDR = NewMIDR;
}
auto NewMIDR = CTX->HostFeatures.CPUMIDRs[i];
if (MIDR != 0 && MIDR != NewMIDR) {
// CPU mismatch, claim hybrid
Hybrid = true;
}
// Truncate to 32-bits, top 32-bits are all reserved in MIDR
PerCPUData[i].ProductName = ProductNames::ARM_UNKNOWN;
PerCPUData[i].MIDR = NewMIDR;
MIDR = NewMIDR;
}
struct CPUMIDR {
@@ -147,11 +171,16 @@ void CPUIDEmu::SetupHostHybridFlag() {
// CPU priority order
// This is mostly arbitrary but will sort by some sort of CPU priority by performance
// Relative list so things they will commonly end up in big.little configurations sort of relate
static constexpr std::array<CPUMIDR, 48> CPUMIDRs = {{
static constexpr std::array<CPUMIDR, 58> CPUMIDRs = {{
// Typically big CPU cores
{0x51, 0x001, 1, ProductNames::ARM_ORYON_1}, // Qualcomm Oryon-1
{0x61, 0x023, 1, ProductNames::ARM_Firestorm}, // Apple M1 Firestorm
{0x61, 0x039, 1, ProductNames::ARM_Avalanche_M2Max}, // Apple Avalanche (M2 Max)
{0x61, 0x035, 1, ProductNames::ARM_Avalanche_M2Pro}, // Apple Avalanche (M2 Pro)
{0x61, 0x033, 1, ProductNames::ARM_Avalanche_M2}, // Apple Avalanche (M2)
{0x61, 0x029, 1, ProductNames::ARM_Firestorm_M1Max}, // Apple Firestorm (M1 Max)
{0x61, 0x025, 1, ProductNames::ARM_Firestorm_M1Pro}, // Apple Firestorm (M1 Pro)
{0x61, 0x023, 1, ProductNames::ARM_Firestorm_M1}, // Apple Firestorm (M1)
{0x41, 0xd85, 1, ProductNames::ARM_X925}, // X925
{0x41, 0xd87, 1, ProductNames::ARM_A725}, // A725
@@ -193,7 +222,13 @@ void CPUIDEmu::SetupHostHybridFlag() {
{0x41, 0xd07, 1, ProductNames::ARM_A57}, // A57
// Typically Little CPU cores
{0x61, 0x022, 0, ProductNames::ARM_Icestorm}, // Apple M1 Icestorm
{0x61, 0x038, 0, ProductNames::ARM_Blizzard_M2Max}, // Apple Blizzard (M2 Max)
{0x61, 0x034, 0, ProductNames::ARM_Blizzard_M2Pro}, // Apple Blizzard (M2 Pro)
{0x61, 0x032, 0, ProductNames::ARM_Blizzard_M2}, // Apple Blizzard (M2)
{0x61, 0x028, 0, ProductNames::ARM_Icestorm_M1Max}, // Apple Icestorm (M1 Max)
{0x61, 0x024, 0, ProductNames::ARM_Icestorm_M1Pro}, // Apple Icestorm (M1 Pro)
{0x61, 0x022, 0, ProductNames::ARM_Icestorm_M1}, // Apple Icestorm (M1)
{0x41, 0xd80, 0, ProductNames::ARM_A520}, // A520
{0x41, 0xd46, 0, ProductNames::ARM_A510}, // A510
{0x41, 0xd06, 0, ProductNames::ARM_A65}, // A65
@@ -608,8 +643,8 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_07h(uint32_t Leaf) const {
// Disable Enhanced REP MOVS when TSO is enabled.
// vcruntime140 memmove will use `rep movsb` in this case which completely destroys perf in Hades(appId 1145360)
// This is due to LRCPC performance on Cortex being abysmal.
// Only enable EnhancedREPMOVS if SoftwareTSO isn't required OR if MemcpySetTSO is not enabled.
const uint32_t SupportsEnhancedREPMOVS = CTX->SoftwareTSORequired() == false || MemcpySetTSOEnabled() == false;
// Only enable EnhancedREPMOVS if atomic memcpy tso emulation isn't enabled.
const uint32_t SupportsEnhancedREPMOVS = CTX->IsMemcpyAtomicTSOEnabled() == false;
const uint32_t SupportsVPCLMULQDQ = CTX->HostFeatures.SupportsPMULL_128Bit && SupportsAVX();
// Number of subfunctions
@@ -772,7 +807,7 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_15h(uint32_t Leaf) const {
uint32_t FrequencyHz = GetCycleCounterFrequency();
if (FrequencyHz) {
Res.eax = 1;
Res.ebx = CTX->Config.SmallTSCScale() ? FEXCore::Context::TSC_SCALE : 1;
Res.ebx = 1U << CTX->Config.TSCScale;
Res.ecx = FrequencyHz;
}
return Res;
@@ -866,11 +901,11 @@ FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0000h(uint32_t Leaf) con
// Extended processor and feature bits
FEXCore::CPUID::FunctionResults CPUIDEmu::Function_8000_0001h(uint32_t Leaf) const {
// RDTSCP is disabled on WIN32/Wine because there is no sane way to query processor ID.
#ifndef _WIN32
constexpr uint32_t SUPPORTS_RDTSCP = 1;
#else
constexpr uint32_t SUPPORTS_RDTSCP = 0;
// RDTSCP under WIN32 is only supported if CPUIndex is available in TPIDRRO.
const uint32_t SUPPORTS_RDTSCP = SupportsCPUIndexInTPIDRRO;
#endif
FEXCore::CPUID::FunctionResults Res {};
@@ -1184,12 +1219,20 @@ FEXCore::CPUID::XCRResults CPUIDEmu::XCRFunction_0h() const {
}
CPUIDEmu::CPUIDEmu(const FEXCore::Context::ContextImpl* ctx)
: CTX {ctx} {
Cores = FEXCore::CPUInfo::CalculateNumberOfCPUs();
: CTX {ctx}
, SupportsCPUIndexInTPIDRRO {CTX->HostFeatures.SupportsCPUIndexInTPIDRRO}
, GetCPUID {GetCPUID_Syscall} {
Cores = CTX->HostFeatures.CPUMIDRs.size();
// Setup some state tracking
SetupHostHybridFlag();
SetupFeatures();
#ifdef _M_ARM_64
if (SupportsCPUIndexInTPIDRRO) {
GetCPUID = GetCPUID_TPIDRRO;
}
#endif
}
} // namespace FEXCore
+4 -2
View File
@@ -115,11 +115,10 @@ public:
private:
const FEXCore::Context::ContextImpl* CTX;
bool SupportsCPUIndexInTPIDRRO {};
bool Hybrid {};
uint32_t Cores {};
FEX_CONFIG_OPT(HideHypervisorBit, HIDEHYPERVISORBIT);
FEX_CONFIG_OPT(SmallTSCScale, SMALLTSCSCALE);
FEX_CONFIG_OPT(MemcpySetTSOEnabled, MEMCPYSETTSOENABLED);
// XFEATURE_ENABLED_MASK
// Mask that configures what features are enabled on the CPU.
@@ -512,5 +511,8 @@ private:
// 0x8000'001F: AMD Secure Encryption
{SupportsConstant::CONSTANT, NeedsLeafConstant::NOLEAFCONSTANT},
}};
using GetCPUIDPtr = uint32_t (*)();
GetCPUIDPtr GetCPUID;
};
} // namespace FEXCore
+101 -117
View File
@@ -9,18 +9,16 @@ $end_info$
*/
#include <cstdint>
#include "Interface/Context/Context.h"
#include "Interface/Core/ArchHelpers//Arm64Emitter.h"
#include "Interface/Core/ArchHelpers/Arm64Emitter.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/CPUBackend.h"
#include "Interface/Core/CPUID.h"
#include "Interface/Core/Frontend.h"
#include "Interface/Core/ObjectCache/ObjectCacheService.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include "Interface/Core/JIT/JITCore.h"
#include "Interface/Core/JIT/JITClass.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include "Interface/IR/IR.h"
#include "Interface/IR/IREmitter.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
@@ -29,11 +27,13 @@ $end_info$
#include "Interface/IR/RegisterAllocationData.h"
#include "Utils/Allocator.h"
#include "Utils/Allocator/HostAllocator.h"
#include "Utils/SpinWaitLock.h"
#include <FEXCore/Config/Config.h>
#include <FEXCore/Core/Context.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/SignalDelegator.h>
#include <FEXCore/Core/Thunks.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/HLE/SyscallHandler.h>
@@ -78,9 +78,6 @@ ContextImpl::ContextImpl(const FEXCore::HostFeatures& Features)
: HostFeatures {Features}
, CPUID {this}
, IRCaptureCache {this} {
#ifdef BLOCKSTATS
BlockData = std::make_unique<FEXCore::BlockSamplingData>();
#endif
if (Config.CacheObjectCodeCompilation() != FEXCore::Config::ConfigObjectCodeHandler::CONFIG_NONE) {
CodeObjectCacheService = fextl::make_unique<FEXCore::CodeSerialize::CodeObjectSerializeService>(this);
}
@@ -94,8 +91,13 @@ ContextImpl::ContextImpl(const FEXCore::HostFeatures& Features)
Symbols.InitFile();
}
if (FEXCore::GetCycleCounterFrequency() >= FEXCore::Context::TSC_SCALE_MAXIMUM) {
Config.SmallTSCScale = false;
uint64_t FrequencyCounter = FEXCore::GetCycleCounterFrequency();
if (FrequencyCounter && FrequencyCounter < FEXCore::Context::TSC_SCALE_MAXIMUM && Config.SmallTSCScale()) {
// Scale TSC until it is at the minimum required.
while (FrequencyCounter < FEXCore::Context::TSC_SCALE_MAXIMUM) {
FrequencyCounter <<= 1;
++Config.TSCScale;
}
}
// Track atomic TSO emulation configuration.
@@ -190,6 +192,9 @@ uint32_t ContextImpl::ReconstructCompactedEFLAGS(FEXCore::Core::InternalThreadSt
uint32_t ZF = (Packed_NZCV >> IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_ZF_RAW_LOC)) & 1;
uint32_t SF = (Packed_NZCV >> IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_SF_RAW_LOC)) & 1;
// CF is inverted in our representation, undo the invert here.
CF ^= 1;
// Pack in to EFLAGS
EFLAGS |= OF << X86State::RFLAG_OF_RAW_LOC;
EFLAGS |= CF << X86State::RFLAG_CF_RAW_LOC;
@@ -293,10 +298,10 @@ void ContextImpl::SetFlagsFromCompactedEFLAGS(FEXCore::Core::InternalThreadState
}
}
// Calculate packed NZCV
// Calculate packed NZCV. Note CF is inverted.
uint32_t Packed_NZCV {};
Packed_NZCV |= (EFLAGS & (1U << X86State::RFLAG_OF_RAW_LOC)) ? 1U << IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_OF_RAW_LOC) : 0;
Packed_NZCV |= (EFLAGS & (1U << X86State::RFLAG_CF_RAW_LOC)) ? 1U << IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_CF_RAW_LOC) : 0;
Packed_NZCV |= (EFLAGS & (1U << X86State::RFLAG_CF_RAW_LOC)) ? 0 : 1U << IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_CF_RAW_LOC);
Packed_NZCV |= (EFLAGS & (1U << X86State::RFLAG_ZF_RAW_LOC)) ? 1U << IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_ZF_RAW_LOC) : 0;
Packed_NZCV |= (EFLAGS & (1U << X86State::RFLAG_SF_RAW_LOC)) ? 1U << IR::OpDispatchBuilder::IndexNZCV(X86State::RFLAG_SF_RAW_LOC) : 0;
memcpy(&Frame->State.flags[X86State::RFLAG_NZCV_LOC], &Packed_NZCV, sizeof(Packed_NZCV));
@@ -313,8 +318,6 @@ bool ContextImpl::InitCore() {
// Set up the SignalDelegator config since core is initialized.
FEXCore::SignalDelegator::SignalDelegatorConfig SignalConfig {
.SupportsAVX = HostFeatures.SupportsAVX,
.DispatcherBegin = Dispatcher->Start,
.DispatcherEnd = Dispatcher->End,
@@ -343,17 +346,14 @@ bool ContextImpl::InitCore() {
SignalDelegation->SetConfig(SignalConfig);
#ifndef _WIN32
ThunkHandler = FEXCore::ThunkHandler::Create();
#else
// WIN32 always needs the interrupt fault check to be enabled.
#elif !defined(_M_ARM64EC)
// WOW64 always needs the interrupt fault check to be enabled.
Config.NeedsPendingInterruptFaultCheck = true;
#endif
if (Config.GdbServer) {
// If gdbserver is enabled then this needs to be enabled.
Config.NeedsPendingInterruptFaultCheck = true;
// FEX needs to start paused when gdb is enabled.
StartPaused = true;
}
return true;
@@ -363,32 +363,17 @@ void ContextImpl::HandleCallback(FEXCore::Core::InternalThreadState* Thread, uin
static_cast<ContextImpl*>(Thread->CTX)->Dispatcher->ExecuteJITCallback(Thread->CurrentFrame, RIP);
}
FEXCore::Context::ExitReason ContextImpl::RunUntilExit(FEXCore::Core::InternalThreadState* Thread) {
ExecutionThread(Thread);
CoreShuttingDown.store(true);
if (CustomExitHandler) {
CustomExitHandler(Thread, FEXCore::Context::ExitReason::EXIT_SHUTDOWN);
return Thread->ExitReason;
}
return FEXCore::Context::ExitReason::EXIT_SHUTDOWN;
}
void ContextImpl::ExecuteThread(FEXCore::Core::InternalThreadState* Thread) {
Dispatcher->ExecuteDispatch(Thread->CurrentFrame);
}
void ContextImpl::InitializeThreadTLSData(FEXCore::Core::InternalThreadState* Thread) {
// Let's do some initial bookkeeping here
if (ThunkHandler) {
ThunkHandler->RegisterTLSState(Thread);
{
// Ensure the Code Object Serialization service has fully serialized this thread's data before clearing the cache
// Use the thread's object cache ref counter for this
CodeSerialize::CodeObjectSerializeService::WaitForEmptyJobQueue(&Thread->ObjectCacheRefCounter);
}
#ifndef _WIN32
Alloc::OSAllocator::RegisterTLSData(Thread);
#endif
// If it is the parent thread that died then just leave
FEX_TODO("This doesn't make sense when the parent thread doesn't outlive its children");
}
void ContextImpl::InitializeCompiler(FEXCore::Core::InternalThreadState* Thread) {
@@ -403,29 +388,23 @@ void ContextImpl::InitializeCompiler(FEXCore::Core::InternalThreadState* Thread)
Dispatcher->InitThreadPointers(Thread);
Thread->CTX = this;
Thread->PassManager->AddDefaultPasses(this);
Thread->PassManager->AddDefaultValidationPasses();
Thread->PassManager->RegisterSyscallHandler(SyscallHandler);
// Create CPU backend
switch (Config.Core) {
case FEXCore::Config::CONFIG_IRJIT:
Thread->PassManager->InsertRegisterAllocationPass();
Thread->CPUBackend = FEXCore::CPU::CreateArm64JITCore(this, Thread);
break;
case FEXCore::Config::CONFIG_CUSTOM: Thread->CPUBackend = CustomCPUFactory(this, Thread); break;
default: ERROR_AND_DIE_FMT("Unknown core configuration"); break;
}
Thread->PassManager->InsertRegisterAllocationPass();
Thread->CPUBackend = FEXCore::CPU::CreateArm64JITCore(this, Thread);
Thread->PassManager->Finalize();
}
FEXCore::Core::InternalThreadState*
ContextImpl::CreateThread(uint64_t InitialRIP, uint64_t StackPointer, FEXCore::Core::CPUState* NewThreadState, uint64_t ParentTID) {
FEXCore::Core::InternalThreadState* Thread = new FEXCore::Core::InternalThreadState {};
ContextImpl::CreateThread(uint64_t InitialRIP, uint64_t StackPointer, const FEXCore::Core::CPUState* NewThreadState, uint64_t ParentTID) {
FEXCore::Core::InternalThreadState* Thread = new FEXCore::Core::InternalThreadState {
.CTX = this,
};
Thread->CurrentFrame->State.gregs[X86State::REG_RSP] = StackPointer;
Thread->CurrentFrame->State.rip = InitialRIP;
@@ -450,13 +429,7 @@ ContextImpl::CreateThread(uint64_t InitialRIP, uint64_t StackPointer, FEXCore::C
return Thread;
}
void ContextImpl::DestroyThread(FEXCore::Core::InternalThreadState* Thread, bool NeedsTLSUninstall) {
if (NeedsTLSUninstall) {
#ifndef _WIN32
Alloc::OSAllocator::UninstallTLSData(Thread);
#endif
}
void ContextImpl::DestroyThread(FEXCore::Core::InternalThreadState* Thread) {
FEXCore::Allocator::VirtualProtect(&Thread->InterruptFaultPage, sizeof(Thread->InterruptFaultPage),
Allocator::ProtectOptions::Read | Allocator::ProtectOptions::Write);
delete Thread;
@@ -468,8 +441,14 @@ void ContextImpl::UnlockAfterFork(FEXCore::Core::InternalThreadState* LiveThread
if (Child) {
CodeInvalidationMutex.StealAndDropActiveLocks();
if (Config.StrictInProcessSplitLocks) {
StrictSplitLockMutex = 0;
}
} else {
CodeInvalidationMutex.unlock();
if (Config.StrictInProcessSplitLocks) {
FEXCore::Utils::SpinWaitLock::unlock(&StrictSplitLockMutex);
}
return;
}
}
@@ -477,6 +456,9 @@ void ContextImpl::UnlockAfterFork(FEXCore::Core::InternalThreadState* LiveThread
void ContextImpl::LockBeforeFork(FEXCore::Core::InternalThreadState* Thread) {
CodeInvalidationMutex.lock();
Allocator::LockBeforeFork(Thread);
if (Config.StrictInProcessSplitLocks) {
FEXCore::Utils::SpinWaitLock::lock(&StrictSplitLockMutex);
}
}
#endif
@@ -581,7 +563,7 @@ ContextImpl::GenerateIR(FEXCore::Core::InternalThreadState* Thread, uint64_t Gue
Thread->OpDispatcher->BeginFunction(GuestRIP, CodeBlocks, BlockInfo->TotalInstructionCount);
const uint8_t GPRSize = GetGPRSize();
const auto GPRSize = GetGPROpSize();
for (size_t j = 0; j < CodeBlocks->size(); ++j) {
const FEXCore::Frontend::Decoder::DecodedBlocks& Block = CodeBlocks->at(j);
@@ -595,6 +577,11 @@ ContextImpl::GenerateIR(FEXCore::Core::InternalThreadState* Thread, uint64_t Gue
uint64_t InstsInBlock = Block.NumInstructions;
if (InstsInBlock == 0) {
// Special case for an empty instruction block.
Thread->OpDispatcher->ExitFunction(Thread->OpDispatcher->_EntrypointOffset(GPRSize, Block.Entry - GuestRIP));
}
for (size_t i = 0; i < InstsInBlock; ++i) {
const FEXCore::X86Tables::X86InstInfo* TableInfo {nullptr};
const FEXCore::X86Tables::DecodedInst* DecodedInfo {nullptr};
@@ -635,8 +622,7 @@ ContextImpl::GenerateIR(FEXCore::Core::InternalThreadState* Thread, uint64_t Gue
Thread->OpDispatcher->SetCurrentCodeBlock(CodeWasChangedBlock);
Thread->OpDispatcher->_ThreadRemoveCodeEntry();
Thread->OpDispatcher->ExitFunction(
Thread->OpDispatcher->_EntrypointOffset(IR::SizeToOpSize(GPRSize), Block.Entry + BlockInstructionsLength - GuestRIP));
Thread->OpDispatcher->ExitFunction(Thread->OpDispatcher->_EntrypointOffset(GPRSize, Block.Entry + BlockInstructionsLength - GuestRIP));
auto NextOpBlock = Thread->OpDispatcher->CreateNewCodeBlockAfter(CurrentBlock);
@@ -666,7 +652,7 @@ ContextImpl::GenerateIR(FEXCore::Core::InternalThreadState* Thread, uint64_t Gue
}
// Invalid instruction
Thread->OpDispatcher->InvalidOp(DecodedInfo);
Thread->OpDispatcher->ExitFunction(Thread->OpDispatcher->_EntrypointOffset(IR::SizeToOpSize(GPRSize), Block.Entry - GuestRIP));
Thread->OpDispatcher->ExitFunction(Thread->OpDispatcher->_EntrypointOffset(GPRSize, Block.Entry - GuestRIP));
}
const bool NeedsBlockEnd =
@@ -680,11 +666,8 @@ ContextImpl::GenerateIR(FEXCore::Core::InternalThreadState* Thread, uint64_t Gue
}
if (NeedsBlockEnd) {
const uint8_t GPRSize = GetGPRSize();
// We had some instructions. Early exit
Thread->OpDispatcher->ExitFunction(
Thread->OpDispatcher->_EntrypointOffset(IR::SizeToOpSize(GPRSize), Block.Entry + BlockInstructionsLength - GuestRIP));
Thread->OpDispatcher->ExitFunction(Thread->OpDispatcher->_EntrypointOffset(GPRSize, Block.Entry + BlockInstructionsLength - GuestRIP));
break;
}
@@ -879,45 +862,6 @@ uintptr_t ContextImpl::CompileBlock(FEXCore::Core::CpuStateFrame* Frame, uint64_
return (uintptr_t)CodePtr;
}
void ContextImpl::ExecutionThread(FEXCore::Core::InternalThreadState* Thread) {
Thread->ExitReason = FEXCore::Context::ExitReason::EXIT_WAITING;
InitializeThreadTLSData(Thread);
// Now notify the thread that we are initialized
Thread->ThreadWaiting.NotifyAll();
if (StartPaused || Thread->StartPaused) {
// Parent thread doesn't need to wait to run
Thread->StartRunning.Wait();
}
if (!Thread->RunningEvents.EarlyExit.load()) {
Thread->RunningEvents.WaitingToStart = false;
Thread->ExitReason = FEXCore::Context::ExitReason::EXIT_NONE;
Thread->RunningEvents.Running = true;
static_cast<ContextImpl*>(Thread->CTX)->Dispatcher->ExecuteDispatch(Thread->CurrentFrame);
Thread->RunningEvents.Running = false;
}
{
// Ensure the Code Object Serialization service has fully serialized this thread's data before clearing the cache
// Use the thread's object cache ref counter for this
CodeSerialize::CodeObjectSerializeService::WaitForEmptyJobQueue(&Thread->ObjectCacheRefCounter);
}
// If it is the parent thread that died then just leave
FEX_TODO("This doesn't make sense when the parent thread doesn't outlive its children");
#ifndef _WIN32
Alloc::OSAllocator::UninstallTLSData(Thread);
#endif
}
static void InvalidateGuestThreadCodeRange(FEXCore::Core::InternalThreadState* Thread, uint64_t Start, uint64_t Length) {
std::lock_guard<std::recursive_mutex> lk(Thread->LookupCache->WriteLock);
@@ -943,6 +887,10 @@ void ContextImpl::InvalidateGuestCodeRange(FEXCore::Core::InternalThreadState* T
}
void ContextImpl::MarkMemoryShared(FEXCore::Core::InternalThreadState* Thread) {
if (!Thread) {
return;
}
if (!IsMemoryShared) {
IsMemoryShared = true;
UpdateAtomicTSOEmulationConfig();
@@ -971,7 +919,8 @@ void ContextImpl::ThreadRemoveCodeEntry(FEXCore::Core::InternalThreadState* Thre
Thread->LookupCache->Erase(Thread->CurrentFrame, GuestRIP);
}
CustomIRResult ContextImpl::AddCustomIREntrypoint(uintptr_t Entrypoint, CustomIREntrypointHandler Handler, void* Creator, void* Data) {
std::optional<CustomIRResult>
ContextImpl::AddCustomIREntrypoint(uintptr_t Entrypoint, CustomIREntrypointHandler Handler, void* Creator, void* Data) {
LOGMAN_THROW_A_FMT(Config.Is64BitMode || !(Entrypoint >> 32), "64-bit Entrypoint in 32-bit mode {:x}", Entrypoint);
std::unique_lock lk(CustomIRMutex);
@@ -981,10 +930,51 @@ CustomIRResult ContextImpl::AddCustomIREntrypoint(uintptr_t Entrypoint, CustomIR
if (!InsertedIterator.second) {
const auto& [fn, Creator, Data] = InsertedIterator.first->second;
return CustomIRResult(std::move(lk), Creator, Data);
} else {
lk.unlock();
return CustomIRResult(std::move(lk), 0, 0);
return CustomIRResult(Creator, Data);
}
return std::nullopt;
}
void ContextImpl::AddThunkTrampolineIRHandler(uintptr_t Entrypoint, uintptr_t GuestThunkEntrypoint) {
LOGMAN_THROW_AA_FMT(Entrypoint, "Tried to link null pointer address to guest function");
LOGMAN_THROW_AA_FMT(GuestThunkEntrypoint, "Tried to link address to null pointer guest function");
if (!Config.Is64BitMode) {
LOGMAN_THROW_AA_FMT((Entrypoint >> 32) == 0, "Tried to link 64-bit address in 32-bit mode");
LOGMAN_THROW_AA_FMT((GuestThunkEntrypoint >> 32) == 0, "Tried to link 64-bit address in 32-bit mode");
}
LogMan::Msg::DFmt("Thunks: Adding guest trampoline from address {:#x} to guest function {:#x}", Entrypoint, GuestThunkEntrypoint);
auto Result = AddCustomIREntrypoint(
Entrypoint,
[this, GuestThunkEntrypoint](uintptr_t Entrypoint, FEXCore::IR::IREmitter* emit) {
auto IRHeader = emit->_IRHeader(emit->Invalid(), Entrypoint, 0, 0);
auto Block = emit->CreateCodeNode();
IRHeader.first->Blocks = emit->WrapNode(Block);
emit->SetCurrentCodeBlock(Block);
const auto GPRSize = GetGPROpSize();
if (GPRSize == IR::OpSize::i64Bit) {
emit->_StoreRegister(emit->_Constant(Entrypoint), X86State::REG_R11, IR::GPRClass, GPRSize);
} else {
emit->_StoreContext(GPRSize, IR::FPRClass, emit->_VCastFromGPR(IR::OpSize::i64Bit, IR::OpSize::i64Bit, emit->_Constant(Entrypoint)),
offsetof(Core::CPUState, mm[0][0]));
}
emit->_ExitFunction(emit->_Constant(GuestThunkEntrypoint));
},
ThunkHandler, (void*)GuestThunkEntrypoint);
if (Result.has_value()) {
if (Result->Creator != ThunkHandler) {
ERROR_AND_DIE_FMT("Input address for AddThunkTrampoline is already linked by another module");
}
if (Result->Data != (void*)GuestThunkEntrypoint) {
// NOTE: This may happen in Vulkan thunks if the Vulkan driver resolves two different symbols
// to the same function (e.g. vkGetPhysicalDeviceFeatures2/vkGetPhysicalDeviceFeatures2KHR)
LogMan::Msg::EFmt("Input address for AddThunkTrampoline is already linked elsewhere");
}
}
}
@@ -1007,12 +997,6 @@ void ContextImpl::UnloadAOTIRCacheEntry(IR::AOTIRCacheEntry* Entry) {
IRCaptureCache.UnloadAOTIRCacheEntry(Entry);
}
void ContextImpl::AppendThunkDefinitions(std::span<const FEXCore::IR::ThunkDefinition> Definitions) {
if (ThunkHandler) {
ThunkHandler->AppendThunkDefinitions(Definitions);
}
}
void ContextImpl::ConfigureAOTGen(FEXCore::Core::InternalThreadState* Thread, fextl::set<uint64_t>* ExternalBranches, uint64_t SectionMaxAddress) {
Thread->FrontendDecoder->SetExternalBranches(ExternalBranches);
Thread->FrontendDecoder->SetSectionMaxAddress(SectionMaxAddress);
+42 -20
View File
@@ -274,12 +274,10 @@ bool Decoder::NormalOp(const FEXCore::X86Tables::X86InstInfo* Info, uint16_t Op,
DecodeInst->TableInfo = Info;
if (Info->Type == FEXCore::X86Tables::TYPE_UNKNOWN) {
LogMan::Msg::DFmt("Unknown instruction: {} 0x{:04x} 0x{:x}", Info->Name ?: "UND", Op, DecodeInst->PC);
return false;
}
if (Info->Type == FEXCore::X86Tables::TYPE_INVALID) {
LogMan::Msg::DFmt("Invalid or Unknown instruction: {} 0x{:04x} 0x{:x}", Info->Name ?: "UND", Op, DecodeInst->PC);
return false;
}
@@ -557,12 +555,10 @@ bool Decoder::NormalOpHeader(const FEXCore::X86Tables::X86InstInfo* Info, uint16
DecodeInst->TableInfo = Info;
if (Info->Type == FEXCore::X86Tables::TYPE_UNKNOWN) {
LogMan::Msg::DFmt("Unknown instruction: {} 0x{:04x} 0x{:x}", Info->Name ?: "UND", Op, DecodeInst->PC);
return false;
}
if (Info->Type == FEXCore::X86Tables::TYPE_INVALID) {
LogMan::Msg::DFmt("Invalid or Unknown instruction: {} 0x{:04x} 0x{:x}", Info->Name ?: "UND", Op, DecodeInst->PC);
return false;
}
@@ -694,12 +690,8 @@ bool Decoder::NormalOpHeader(const FEXCore::X86Tables::X86InstInfo* Info, uint16
}
} else if (Info->Type == FEXCore::X86Tables::TYPE_GROUP_EVEX) {
FEXCORE_TELEMETRY_SET(EVEXOpTelem, 1);
/* uint8_t P1 = */ ReadByte();
/* uint8_t P2 = */ ReadByte();
/* uint8_t P3 = */ ReadByte();
uint8_t EVEXOp = ReadByte();
return NormalOp(&EVEXTableOps[EVEXOp], EVEXOp);
// EVEX unsupported
return false;
}
LOGMAN_MSG_A_FMT("Invalid instruction decoding type");
@@ -934,7 +926,7 @@ void Decoder::BranchTargetInMultiblockRange() {
// If the RIP setting is conditional AND within our symbol range then it can be considered for multiblock
uint64_t TargetRIP = 0;
const uint8_t GPRSize = CTX->GetGPRSize();
const auto GPRSize = CTX->GetGPROpSize();
bool Conditional = true;
switch (DecodeInst->OP) {
@@ -962,13 +954,19 @@ void Decoder::BranchTargetInMultiblockRange() {
default: return; break;
}
if (GPRSize == 4) {
if (GPRSize == IR::OpSize::i32Bit) {
// If we are running a 32bit guest then wrap around addresses that go above 32bit
TargetRIP &= 0xFFFFFFFFU;
}
// If the target RIP is within the symbol ranges then we are golden
if (TargetRIP >= SymbolMinAddress && TargetRIP < SymbolMaxAddress) {
// If the target RIP is x86 code within the symbol ranges then we are golden
bool ValidMultiblockMember = TargetRIP >= SymbolMinAddress && TargetRIP < SymbolMaxAddress;
#ifdef _M_ARM_64EC
ValidMultiblockMember = ValidMultiblockMember && !RtlIsEcCode(TargetRIP);
#endif
if (ValidMultiblockMember) {
// Update our conditional branch ranges before we return
if (Conditional) {
MaxCondBranchForward = std::max(MaxCondBranchForward, TargetRIP);
@@ -977,12 +975,12 @@ void Decoder::BranchTargetInMultiblockRange() {
// If we are conditional then a target can be the instruction past the conditional instruction
uint64_t FallthroughRIP = DecodeInst->PC + DecodeInst->InstSize;
if (!HasBlocks.contains(FallthroughRIP)) {
BlocksToDecode.insert(FallthroughRIP);
CurrentBlockTargets.insert(FallthroughRIP);
}
}
if (!HasBlocks.contains(TargetRIP)) {
BlocksToDecode.insert(TargetRIP);
CurrentBlockTargets.insert(TargetRIP);
}
} else {
if (ExternalBranches) {
@@ -997,13 +995,13 @@ bool Decoder::BranchTargetCanContinue(bool FinalInstruction) const {
}
uint64_t TargetRIP = 0;
const uint8_t GPRSize = CTX->GetGPRSize();
const auto GPRSize = CTX->GetGPROpSize();
if (DecodeInst->OP == 0xE8) { // Call - immediate target
const uint64_t NextRIP = DecodeInst->PC + DecodeInst->InstSize;
TargetRIP = DecodeInst->PC + DecodeInst->InstSize + DecodeInst->Src[0].Literal();
if (GPRSize == 4) {
if (GPRSize == IR::OpSize::i32Bit) {
// If we are running a 32bit guest then wrap around addresses that go above 32bit
TargetRIP &= 0xFFFFFFFFU;
}
@@ -1079,6 +1077,8 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
MaxInst = CTX->Config.MaxInstPerBlock;
}
bool EntryBlock {true};
while (!BlocksToDecode.empty()) {
auto BlockDecodeIt = BlocksToDecode.begin();
uint64_t RIPToDecode = *BlockDecodeIt;
@@ -1115,7 +1115,6 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
bool ErrorDuringDecoding = !DecodeInstruction(RIPToDecode + PCOffset);
if (ErrorDuringDecoding) [[unlikely]] {
LogMan::Msg::DFmt("Couldn't Decode something at 0x{:x}, Started at 0x{:x}", RIPToDecode + PCOffset, PC);
// Put an invalid instruction in the stream so the core can raise SIGILL if hit
CurrentBlockDecoding.HasInvalidInstruction = true;
// Error while decoding instruction. We don't know the table or instruction size
@@ -1123,6 +1122,14 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
DecodeInst->InstSize = 0;
}
if (!ErrorDuringDecoding) {
// If there wasn't an error during decoding but we have no dispatcher for the instruction then claim invalid instruction.
auto TableInfo = DecodedBuffer[BlockStartOffset + BlockNumberOfInstructions].TableInfo;
if (!TableInfo || !TableInfo->OpcodeDispatcher) {
CurrentBlockDecoding.HasInvalidInstruction = true;
}
}
DecodedMinAddress = std::min(DecodedMinAddress, RIPToDecode + PCOffset);
DecodedMaxAddress = std::max(DecodedMaxAddress, RIPToDecode + PCOffset + DecodeInst->InstSize);
++TotalInstructions;
@@ -1130,7 +1137,17 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
++DecodedSize;
// Can not continue this block at all on invalid instruction
if (CurrentBlockDecoding.HasInvalidInstruction) {
if (CurrentBlockDecoding.HasInvalidInstruction) [[unlikely]] {
if (!EntryBlock) {
// In multiblock configurations, we can early terminate any non-entrypoint blocks with the expectation that this won't get hit.
// Improves compile-times.
// Just need to undo additions that this block decoding has caused.
TotalInstructions -= CurrentBlockDecoding.NumInstructions;
DecodedSize = BlockStartOffset;
BlockNumberOfInstructions = 0;
InstStream -= PCOffset;
CurrentBlockTargets.clear();
}
break;
}
@@ -1160,6 +1177,9 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
InstStream += DecodeInst->InstSize;
}
BlocksToDecode.merge(CurrentBlockTargets);
CurrentBlockTargets.clear();
BlocksToDecode.erase(BlockDecodeIt);
HasBlocks.emplace(RIPToDecode);
@@ -1167,6 +1187,8 @@ void Decoder::DecodeInstructionsAtEntry(const uint8_t* _InstStream, uint64_t PC,
CurrentBlockDecoding.NumInstructions = BlockNumberOfInstructions;
CurrentBlockDecoding.DecodedInstructions = &DecodedBuffer[BlockStartOffset];
BlockInfo.TotalInstructionCount += BlockNumberOfInstructions;
EntryBlock = false;
}
for (auto CodePage : CodePages) {
+1
View File
@@ -104,6 +104,7 @@ private:
uint64_t SectionMaxAddress {~0ULL};
DecodedBlockInformation BlockInfo;
fextl::set<uint64_t> CurrentBlockTargets;
fextl::set<uint64_t> BlocksToDecode;
fextl::set<uint64_t> HasBlocks;
fextl::set<uint64_t>* ExternalBranches {nullptr};
@@ -7,7 +7,7 @@
namespace FEXCore::CPU {
FEXCORE_PRESERVE_ALL_ATTR static softfloat_state SoftFloatStateFromFCW(uint16_t FCW) {
softfloat_state State;
softfloat_state State {};
State.detectTininess = softfloat_tininess_afterRounding;
State.exceptionFlags = 0;
@@ -322,8 +322,11 @@ struct OpHandlers<IR::OP_F64FYL2X> {
template<>
struct OpHandlers<IR::OP_F64SCALE> {
static double handle(uint16_t FCW, double src1, double src2) {
double trunc = (double)(int64_t)(src2); // truncate
return src1 * exp2(trunc);
if (src1 == 0.0) { // src1 might be +/- zero
return src1; // this will return negative or positive zero if when appropriate
}
double trun = trunc(src2);
return src1 * exp2(trun);
}
};
@@ -79,17 +79,17 @@ void InterpreterOps::FillFallbackIndexPointers(uint64_t* Info) {
}
bool InterpreterOps::GetFallbackHandler(bool SupportsPreserveAllABI, const IR::IROp_Header* IROp, FallbackInfo* Info) {
uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
switch (IROp->Op) {
case IR::OP_F80CVTTO: {
auto Op = IROp->C<IR::IROp_F80CVTTo>();
switch (Op->SrcSize) {
case 4: {
case IR::OpSize::i32Bit: {
*Info = {FABI_F80_I16_F32, (void*)&FEXCore::CPU::OpHandlers<IR::OP_F80CVTTO>::handle4, Core::OPINDEX_F80CVTTO_4, SupportsPreserveAllABI};
return true;
}
case 8: {
case IR::OpSize::i64Bit: {
*Info = {FABI_F80_I16_F64, (void*)&FEXCore::CPU::OpHandlers<IR::OP_F80CVTTO>::handle8, Core::OPINDEX_F80CVTTO_8, SupportsPreserveAllABI};
return true;
}
@@ -99,11 +99,11 @@ bool InterpreterOps::GetFallbackHandler(bool SupportsPreserveAllABI, const IR::I
}
case IR::OP_F80CVT: {
switch (OpSize) {
case 4: {
case IR::OpSize::i32Bit: {
*Info = {FABI_F32_I16_F80, (void*)&FEXCore::CPU::OpHandlers<IR::OP_F80CVT>::handle4, Core::OPINDEX_F80CVT_4, SupportsPreserveAllABI};
return true;
}
case 8: {
case IR::OpSize::i64Bit: {
*Info = {FABI_F64_I16_F80, (void*)&FEXCore::CPU::OpHandlers<IR::OP_F80CVT>::handle8, Core::OPINDEX_F80CVT_8, SupportsPreserveAllABI};
return true;
}
@@ -115,7 +115,7 @@ bool InterpreterOps::GetFallbackHandler(bool SupportsPreserveAllABI, const IR::I
auto Op = IROp->C<IR::IROp_F80CVTInt>();
switch (OpSize) {
case 2: {
case IR::OpSize::i16Bit: {
if (Op->Truncate) {
*Info = {FABI_I16_I16_F80, (void*)&FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle2t, Core::OPINDEX_F80CVTINT_TRUNC2,
SupportsPreserveAllABI};
@@ -124,7 +124,7 @@ bool InterpreterOps::GetFallbackHandler(bool SupportsPreserveAllABI, const IR::I
}
return true;
}
case 4: {
case IR::OpSize::i32Bit: {
if (Op->Truncate) {
*Info = {FABI_I32_I16_F80, (void*)&FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle4t, Core::OPINDEX_F80CVTINT_TRUNC4,
SupportsPreserveAllABI};
@@ -133,7 +133,7 @@ bool InterpreterOps::GetFallbackHandler(bool SupportsPreserveAllABI, const IR::I
}
return true;
}
case 8: {
case IR::OpSize::i64Bit: {
if (Op->Truncate) {
*Info = {FABI_I64_I16_F80, (void*)&FEXCore::CPU::OpHandlers<IR::OP_F80CVTINT>::handle8t, Core::OPINDEX_F80CVTINT_TRUNC8,
SupportsPreserveAllABI};
@@ -156,11 +156,11 @@ bool InterpreterOps::GetFallbackHandler(bool SupportsPreserveAllABI, const IR::I
auto Op = IROp->C<IR::IROp_F80CVTToInt>();
switch (Op->SrcSize) {
case 2: {
case IR::OpSize::i16Bit: {
*Info = {FABI_F80_I16_I16, (void*)&FEXCore::CPU::OpHandlers<IR::OP_F80CVTTOINT>::handle2, Core::OPINDEX_F80CVTTOINT_2, SupportsPreserveAllABI};
return true;
}
case 4: {
case IR::OpSize::i32Bit: {
*Info = {FABI_F80_I16_I32, (void*)&FEXCore::CPU::OpHandlers<IR::OP_F80CVTTOINT>::handle4, Core::OPINDEX_F80CVTTOINT_4, SupportsPreserveAllABI};
return true;
}
@@ -5,9 +5,10 @@ tags: backend|arm64
$end_info$
*/
#include "CodeEmitter/Emitter.h"
#include "FEXCore/IR/IR.h"
#include "Interface/Context/Context.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/JIT/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
namespace FEXCore::CPU {
@@ -41,21 +42,6 @@ DEF_BINOP_WITH_CONSTANT(Lshl, lslv, lsl)
DEF_BINOP_WITH_CONSTANT(Lshr, lsrv, lsr)
DEF_BINOP_WITH_CONSTANT(Ror, rorv, ror)
DEF_OP(TruncElementPair) {
auto Op = IROp->C<IR::IROp_TruncElementPair>();
switch (IROp->Size) {
case 4: {
auto Dst = GetRegPair(Node);
auto Src = GetRegPair(Op->Pair.ID());
mov(ARMEmitter::Size::i32Bit, Dst.first, Src.first);
mov(ARMEmitter::Size::i32Bit, Dst.second, Src.second);
break;
}
default: LOGMAN_MSG_A_FMT("Unhandled Truncation size: {}", IROp->Size); break;
}
}
DEF_OP(Constant) {
auto Op = IROp->C<IR::IROp_Constant>();
auto Dst = GetReg(Node);
@@ -68,8 +54,8 @@ DEF_OP(EntrypointOffset) {
auto Constant = Entry + Op->Offset;
auto Dst = GetReg(Node);
uint64_t Mask = ~0ULL;
uint8_t OpSize = IROp->Size;
if (OpSize == 4) {
const auto OpSize = IROp->Size;
if (OpSize == IR::OpSize::i32Bit) {
Mask = 0xFFFF'FFFFULL;
}
@@ -106,10 +92,10 @@ DEF_OP(AddNZCV) {
uint64_t Const;
if (IsInlineConstant(Op->Src2, &Const)) {
LOGMAN_THROW_AA_FMT(IROp->Size >= 4, "Constant not allowed here");
LOGMAN_THROW_AA_FMT(IROp->Size >= IR::OpSize::i32Bit, "Constant not allowed here");
cmn(EmitSize, Src1, Const);
} else if (IROp->Size < 4) {
unsigned Shift = 32 - (8 * IROp->Size);
} else if (IROp->Size < IR::OpSize::i32Bit) {
unsigned Shift = 32 - IR::OpSizeAsBits(IROp->Size);
lsl(ARMEmitter::Size::i32Bit, TMP1, Src1, Shift);
cmn(EmitSize, TMP1, GetReg(Op->Src2.ID()), ARMEmitter::ShiftType::LSL, Shift);
@@ -130,6 +116,21 @@ DEF_OP(AdcWithFlags) {
adcs(ConvertSize48(IROp), GetReg(Node), GetZeroableReg(Op->Src1), GetReg(Op->Src2.ID()));
}
DEF_OP(AdcZeroWithFlags) {
auto Op = IROp->C<IR::IROp_AdcZeroWithFlags>();
auto Size = ConvertSize48(IROp);
cset(Size, TMP1, ARMEmitter::Condition::CC_CC);
adds(Size, GetReg(Node), GetReg(Op->Src1.ID()), TMP1);
}
DEF_OP(AdcZero) {
auto Op = IROp->C<IR::IROp_AdcZero>();
auto Size = ConvertSize48(IROp);
cinc(Size, GetReg(Node), GetReg(Op->Src1.ID()), ARMEmitter::Condition::CC_CC);
}
DEF_OP(Adc) {
auto Op = IROp->C<IR::IROp_Adc>();
@@ -164,7 +165,7 @@ DEF_OP(TestNZ) {
// Shift the sign bit into place, clearing out the garbage in upper bits.
// Adding zero does an effective test, setting NZ according to the result and
// zeroing CV.
if (IROp->Size < 4) {
if (IROp->Size < IR::OpSize::i32Bit) {
// Cheaper to and+cmn than to lsl+lsl+tst, so do the and ourselves if
// needed.
if (Op->Src1 != Op->Src2) {
@@ -178,7 +179,7 @@ DEF_OP(TestNZ) {
Src1 = TMP1;
}
unsigned Shift = 32 - (IROp->Size * 8);
unsigned Shift = 32 - IR::OpSizeAsBits(IROp->Size);
cmn(EmitSize, ARMEmitter::Reg::zr, Src1, ARMEmitter::ShiftType::LSL, Shift);
} else {
if (IsInlineConstant(Op->Src2, &Const)) {
@@ -190,6 +191,30 @@ DEF_OP(TestNZ) {
}
}
DEF_OP(TestZ) {
auto Op = IROp->C<IR::IROp_TestZ>();
LOGMAN_THROW_AA_FMT(IROp->Size < IR::OpSize::i32Bit, "TestNZ used at higher sizes");
const auto EmitSize = ARMEmitter::Size::i32Bit;
uint64_t Const;
uint64_t Mask = IROp->Size == IR::OpSize::i64Bit ? ~0ULL : ((1ull << IR::OpSizeAsBits(IROp->Size)) - 1);
auto Src1 = GetReg(Op->Src1.ID());
if (IsInlineConstant(Op->Src2, &Const)) {
// We can promote 8/16-bit tests to 32-bit since the constant is masked.
LOGMAN_THROW_AA_FMT(!(Const & ~Mask), "constant is already masked");
tst(EmitSize, Src1, Const);
} else {
const auto Src2 = GetReg(Op->Src2.ID());
if (Src1 == Src2) {
tst(EmitSize, Src1 /* Src2 */, Mask);
} else {
and_(EmitSize, TMP1, Src1, Src2);
tst(EmitSize, TMP1, Mask);
}
}
}
DEF_OP(SubShift) {
auto Op = IROp->C<IR::IROp_SubShift>();
@@ -198,25 +223,25 @@ DEF_OP(SubShift) {
DEF_OP(SubNZCV) {
auto Op = IROp->C<IR::IROp_SubNZCV>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
const auto EmitSize = ConvertSize(IROp);
uint64_t Const;
if (IsInlineConstant(Op->Src2, &Const)) {
LOGMAN_THROW_AA_FMT(OpSize >= 4, "Constant not allowed here");
LOGMAN_THROW_AA_FMT(OpSize >= IR::OpSize::i32Bit, "Constant not allowed here");
cmp(EmitSize, GetReg(Op->Src1.ID()), Const);
} else {
unsigned Shift = OpSize < 4 ? (32 - (8 * OpSize)) : 0;
unsigned Shift = OpSize < IR::OpSize::i32Bit ? (32 - IR::OpSizeAsBits(OpSize)) : 0;
ARMEmitter::Register ShiftedSrc1 = GetZeroableReg(Op->Src1);
// Shift to fix flags for <32-bit ops.
// Any shift of zero is still zero so optimize out silly zero shifts.
if (OpSize < 4 && ShiftedSrc1 != ARMEmitter::Reg::zr) {
if (OpSize < IR::OpSize::i32Bit && ShiftedSrc1 != ARMEmitter::Reg::zr) {
lsl(ARMEmitter::Size::i32Bit, TMP1, ShiftedSrc1, Shift);
ShiftedSrc1 = TMP1;
}
if (OpSize < 4) {
if (OpSize < IR::OpSize::i32Bit) {
cmp(EmitSize, ShiftedSrc1, GetReg(Op->Src2.ID()), ARMEmitter::ShiftType::LSL, Shift);
} else {
cmp(EmitSize, ShiftedSrc1, GetReg(Op->Src2.ID()));
@@ -232,10 +257,8 @@ DEF_OP(CmpPairZ) {
mrs(TMP1, ARMEmitter::SystemRegister::NZCV);
// Compare, setting Z and clobbering NzCV
const auto Src1 = GetRegPair(Op->Src1.ID());
const auto Src2 = GetRegPair(Op->Src2.ID());
cmp(EmitSize, Src1.first, Src2.first);
ccmp(EmitSize, Src1.second, Src2.second, ARMEmitter::StatusFlags::None, ARMEmitter::Condition::CC_EQ);
cmp(EmitSize, GetReg(Op->Src1Lo.ID()), GetReg(Op->Src2Lo.ID()));
ccmp(EmitSize, GetReg(Op->Src1Hi.ID()), GetReg(Op->Src2Hi.ID()), ARMEmitter::StatusFlags::None, ARMEmitter::Condition::CC_EQ);
// Restore NzCV
if (CTX->HostFeatures.SupportsFlagM) {
@@ -263,10 +286,10 @@ DEF_OP(SetSmallNZV) {
auto Op = IROp->C<IR::IROp_SetSmallNZV>();
LOGMAN_THROW_A_FMT(CTX->HostFeatures.SupportsFlagM, "Unsupported flagm op");
const uint8_t OpSize = IROp->Size;
LOGMAN_THROW_AA_FMT(OpSize == 1 || OpSize == 2, "Unsupported {} size: {}", __func__, OpSize);
const auto OpSize = IROp->Size;
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i8Bit || OpSize == IR::OpSize::i16Bit, "Unsupported {} size: {}", __func__, OpSize);
if (OpSize == 1) {
if (OpSize == IR::OpSize::i8Bit) {
setf8(GetReg(Op->Src.ID()).W());
} else {
setf16(GetReg(Op->Src.ID()).W());
@@ -274,8 +297,43 @@ DEF_OP(SetSmallNZV) {
}
DEF_OP(AXFlag) {
LOGMAN_THROW_A_FMT(CTX->HostFeatures.SupportsFlagM2, "Unsupported flagm2 op");
axflag();
if (CTX->HostFeatures.SupportsFlagM2) {
axflag();
} else {
// AXFLAG is defined in the Arm spec as
//
// gt: nzCv -> nzCv
// lt: Nzcv -> nzcv <==> 1 + 0
// eq: nZCv -> nZCv <==> 1 + (~0)
// un: nzCV -> nZcv <==> 0 + 0
//
// For the latter 3 cases, we therefore get the right NZCV by adding V_inv
// to (eq ? ~0 : 0). The remaining case is forced with ccmn.
auto V_inv = GetReg(IROp->Args[0].ID());
csetm(ARMEmitter::Size::i64Bit, TMP1, ARMEmitter::Condition::CC_EQ);
ccmn(ARMEmitter::Size::i64Bit, V_inv, TMP1, ARMEmitter::StatusFlags {0x2} /* nzCv */, ARMEmitter::Condition::CC_LE);
}
}
DEF_OP(Parity) {
auto Op = IROp->C<IR::IROp_Parity>();
auto Raw = GetReg(Op->Raw.ID());
auto Dest = GetReg(Node);
// Cascade to calculate parity of bottom 8-bits to bottom bit.
eor(ARMEmitter::Size::i32Bit, TMP1, Raw, Raw, ARMEmitter::ShiftType::LSR, 4);
eor(ARMEmitter::Size::i32Bit, TMP1, TMP1, TMP1, ARMEmitter::ShiftType::LSR, 2);
if (Op->Invert) {
eon(ARMEmitter::Size::i32Bit, Dest, TMP1, TMP1, ARMEmitter::ShiftType::LSR, 1);
} else {
eor(ARMEmitter::Size::i32Bit, Dest, TMP1, TMP1, ARMEmitter::ShiftType::LSR, 1);
}
// The above sequence leaves garbage in the upper bits.
if (Op->Mask) {
and_(ARMEmitter::Size::i32Bit, Dest, Dest, 1);
}
}
DEF_OP(CondAddNZCV) {
@@ -343,20 +401,20 @@ DEF_OP(Div) {
// Each source is OpSize in size
// So you can have up to a 128bit divide from x86-64
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
const auto EmitSize = ConvertSize(IROp);
const auto Dst = GetReg(Node);
auto Src1 = GetReg(Op->Src1.ID());
auto Src2 = GetReg(Op->Src2.ID());
if (OpSize == 1) {
if (OpSize == IR::OpSize::i8Bit) {
sxtb(EmitSize, TMP1, Src1);
sxtb(EmitSize, TMP2, Src2);
Src1 = TMP1;
Src2 = TMP2;
} else if (OpSize == 2) {
} else if (OpSize == IR::OpSize::i16Bit) {
sxth(EmitSize, TMP1, Src1);
sxth(EmitSize, TMP2, Src2);
@@ -372,20 +430,20 @@ DEF_OP(UDiv) {
// Each source is OpSize in size
// So you can have up to a 128bit divide from x86-64
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
const auto EmitSize = ConvertSize(IROp);
const auto Dst = GetReg(Node);
auto Src1 = GetReg(Op->Src1.ID());
auto Src2 = GetReg(Op->Src2.ID());
if (OpSize == 1) {
if (OpSize == IR::OpSize::i8Bit) {
uxtb(EmitSize, TMP1, Src1);
uxtb(EmitSize, TMP2, Src2);
Src1 = TMP1;
Src2 = TMP2;
} else if (OpSize == 2) {
} else if (OpSize == IR::OpSize::i16Bit) {
uxth(EmitSize, TMP1, Src1);
uxth(EmitSize, TMP2, Src2);
@@ -400,20 +458,20 @@ DEF_OP(Rem) {
auto Op = IROp->C<IR::IROp_Rem>();
// Each source is OpSize in size
// So you can have up to a 128bit divide from x86-64
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
const auto EmitSize = ConvertSize(IROp);
const auto Dst = GetReg(Node);
auto Src1 = GetReg(Op->Src1.ID());
auto Src2 = GetReg(Op->Src2.ID());
if (OpSize == 1) {
if (OpSize == IR::OpSize::i8Bit) {
sxtb(EmitSize, TMP1, Src1);
sxtb(EmitSize, TMP2, Src2);
Src1 = TMP1;
Src2 = TMP2;
} else if (OpSize == 2) {
} else if (OpSize == IR::OpSize::i16Bit) {
sxth(EmitSize, TMP1, Src1);
sxth(EmitSize, TMP2, Src2);
@@ -429,20 +487,20 @@ DEF_OP(URem) {
auto Op = IROp->C<IR::IROp_URem>();
// Each source is OpSize in size
// So you can have up to a 128bit divide from x86-64
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
const auto EmitSize = ConvertSize(IROp);
const auto Dst = GetReg(Node);
auto Src1 = GetReg(Op->Src1.ID());
auto Src2 = GetReg(Op->Src2.ID());
if (OpSize == 1) {
if (OpSize == IR::OpSize::i8Bit) {
uxtb(EmitSize, TMP1, Src1);
uxtb(EmitSize, TMP2, Src2);
Src1 = TMP1;
Src2 = TMP2;
} else if (OpSize == 2) {
} else if (OpSize == IR::OpSize::i16Bit) {
uxth(EmitSize, TMP1, Src1);
uxth(EmitSize, TMP2, Src2);
@@ -456,15 +514,15 @@ DEF_OP(URem) {
DEF_OP(MulH) {
auto Op = IROp->C<IR::IROp_MulH>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
LOGMAN_THROW_AA_FMT(OpSize == 4 || OpSize == 8, "Unsupported {} size: {}", __func__, OpSize);
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i32Bit || OpSize == IR::OpSize::i64Bit, "Unsupported {} size: {}", __func__, OpSize);
const auto Dst = GetReg(Node);
const auto Src1 = GetReg(Op->Src1.ID());
const auto Src2 = GetReg(Op->Src2.ID());
if (OpSize == 4) {
if (OpSize == IR::OpSize::i32Bit) {
sxtw(TMP1, Src1.W());
sxtw(TMP2, Src2.W());
mul(ARMEmitter::Size::i32Bit, Dst, TMP1, TMP2);
@@ -476,15 +534,15 @@ DEF_OP(MulH) {
DEF_OP(UMulH) {
auto Op = IROp->C<IR::IROp_UMulH>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
LOGMAN_THROW_AA_FMT(OpSize == 4 || OpSize == 8, "Unsupported {} size: {}", __func__, OpSize);
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i32Bit || OpSize == IR::OpSize::i64Bit, "Unsupported {} size: {}", __func__, OpSize);
const auto Dst = GetReg(Node);
const auto Src1 = GetReg(Op->Src1.ID());
const auto Src2 = GetReg(Op->Src2.ID());
if (OpSize == 4) {
if (OpSize == IR::OpSize::i32Bit) {
uxtw(ARMEmitter::Size::i64Bit, TMP1, Src1);
uxtw(ARMEmitter::Size::i64Bit, TMP2, Src2);
mul(ARMEmitter::Size::i64Bit, Dst, TMP1, TMP2);
@@ -535,7 +593,7 @@ DEF_OP(Ornror) {
DEF_OP(AndWithFlags) {
auto Op = IROp->C<IR::IROp_AndWithFlags>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
const auto EmitSize = ConvertSize(IROp);
uint64_t Const;
@@ -543,7 +601,7 @@ DEF_OP(AndWithFlags) {
auto Src1 = GetReg(Op->Src1.ID());
// See TestNZ
if (OpSize < 4) {
if (OpSize < IR::OpSize::i32Bit) {
if (IsInlineConstant(Op->Src2, &Const)) {
and_(EmitSize, Dst, Src1, Const);
} else {
@@ -556,7 +614,7 @@ DEF_OP(AndWithFlags) {
}
}
unsigned Shift = 32 - (OpSize * 8);
unsigned Shift = 32 - IR::OpSizeAsBits(OpSize);
cmn(EmitSize, ARMEmitter::Reg::zr, Dst, ARMEmitter::ShiftType::LSL, Shift);
} else {
if (IsInlineConstant(Op->Src2, &Const)) {
@@ -582,7 +640,7 @@ DEF_OP(XornShift) {
DEF_OP(Ashr) {
auto Op = IROp->C<IR::IROp_Ashr>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
const auto EmitSize = ConvertSize(IROp);
const auto Dst = GetReg(Node);
@@ -590,29 +648,29 @@ DEF_OP(Ashr) {
uint64_t Const;
if (IsInlineConstant(Op->Src2, &Const)) {
if (OpSize >= 4) {
if (OpSize >= IR::OpSize::i32Bit) {
asr(EmitSize, Dst, Src1, (unsigned int)Const);
} else {
sbfx(EmitSize, TMP1, Src1, 0, OpSize * 8);
sbfx(EmitSize, TMP1, Src1, 0, IR::OpSizeAsBits(OpSize));
asr(EmitSize, Dst, TMP1, (unsigned int)Const);
ubfx(EmitSize, Dst, Dst, 0, OpSize * 8);
ubfx(EmitSize, Dst, Dst, 0, IR::OpSizeAsBits(OpSize));
}
} else {
const auto Src2 = GetReg(Op->Src2.ID());
if (OpSize >= 4) {
if (OpSize >= IR::OpSize::i32Bit) {
asrv(EmitSize, Dst, Src1, Src2);
} else {
sbfx(EmitSize, TMP1, Src1, 0, OpSize * 8);
sbfx(EmitSize, TMP1, Src1, 0, IR::OpSizeAsBits(OpSize));
asrv(EmitSize, Dst, TMP1, Src2);
ubfx(EmitSize, Dst, Dst, 0, OpSize * 8);
ubfx(EmitSize, Dst, Dst, 0, IR::OpSizeAsBits(OpSize));
}
}
}
DEF_OP(ShiftFlags) {
auto Op = IROp->C<IR::IROp_ShiftFlags>();
const uint8_t OpSize = Op->Size;
const auto EmitSize = OpSize == 8 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto OpSize = Op->Size;
const auto EmitSize = OpSize == IR::OpSize::i64Bit ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto PFOutput = GetReg(Node);
const auto PFInput = GetReg(Op->PFInput.ID());
@@ -632,16 +690,16 @@ DEF_OP(ShiftFlags) {
// We need to mask the source before comparing it. We don't just skip flag
// updates for Src2=0 but anything that masks to zero.
and_(ARMEmitter::Size::i32Bit, TMP1, Src2, OpSize == 8 ? 0x3f : 0x1f);
and_(ARMEmitter::Size::i32Bit, TMP1, Src2, OpSize == IR::OpSize::i64Bit ? 0x3f : 0x1f);
ARMEmitter::SingleUseForwardLabel Done;
cbz(EmitSize, TMP1, &Done);
{
// PF/SF/ZF/OF
if (OpSize >= 4) {
if (OpSize >= IR::OpSize::i32Bit) {
ands(EmitSize, PFTemp, Dst, Dst);
} else {
unsigned Shift = 32 - (OpSize * 8);
unsigned Shift = 32 - (IR::OpSizeToSize(OpSize) * 8);
cmn(EmitSize, ARMEmitter::Reg::zr, Dst, ARMEmitter::ShiftType::LSL, Shift);
mov(ARMEmitter::Size::i64Bit, PFTemp, Dst);
}
@@ -651,18 +709,23 @@ DEF_OP(ShiftFlags) {
// Extract the last bit shifted in to CF
if (Op->Shift == IR::ShiftType::LSL) {
if (OpSize >= 4) {
if (OpSize >= IR::OpSize::i32Bit) {
neg(EmitSize, CFWord, Src2);
lsrv(EmitSize, CFWord, Src1, CFWord);
} else {
CFWord = Dst.X();
CFBit = (OpSize * 8);
CFBit = IR::OpSizeToSize(OpSize) * 8;
}
} else {
sub(ARMEmitter::Size::i64Bit, CFWord, Src2, 1);
lsrv(EmitSize, CFWord, Src1, CFWord);
}
if (Op->InvertCF) {
mvn(ARMEmitter::Size::i64Bit, TMP1, CFWord);
CFWord = TMP1;
}
bool SetOF = Op->Shift != IR::ShiftType::ASR;
if (SetOF) {
// Only defined when Shift is 1 else undefined
@@ -674,7 +737,7 @@ DEF_OP(ShiftFlags) {
rmif(CFWord, (CFBit - 1) % 64, (1 << 1) /* C */);
if (SetOF) {
rmif(TMP3, OpSize * 8 - 1, (1 << 0) /* V */);
rmif(TMP3, IR::OpSizeToSize(OpSize) * 8 - 1, (1 << 0) /* V */);
}
} else {
mrs(TMP2, ARMEmitter::SystemRegister::NZCV);
@@ -687,7 +750,7 @@ DEF_OP(ShiftFlags) {
bfi(ARMEmitter::Size::i32Bit, TMP2, CFWord, 29 /* C */, 1);
if (SetOF) {
lsr(EmitSize, TMP3, TMP3, OpSize * 8 - 1);
lsr(EmitSize, TMP3, TMP3, IR::OpSizeToSize(OpSize) * 8 - 1);
bfi(ARMEmitter::Size::i32Bit, TMP2, TMP3, 28 /* V */, 1);
}
@@ -702,6 +765,50 @@ DEF_OP(ShiftFlags) {
}
}
DEF_OP(RotateFlags) {
auto Op = IROp->C<IR::IROp_RotateFlags>();
const auto Result = GetReg(Op->Result.ID());
const auto Shift = GetReg(Op->Shift.ID());
const bool Left = Op->Left;
const auto EmitSize = Op->Size == IR::OpSize::i64Bit ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
// If shift=0, flags are unaffected. Wrap the whole implementation in a cbz.
ARMEmitter::SingleUseForwardLabel Done;
cbz(EmitSize, Shift, &Done);
{
// Extract the last bit shifted in to CF
const auto BitSize = IR::OpSizeToSize(Op->Size) * 8;
unsigned CFBit = Left ? 0 : BitSize - 1;
// For ROR, OF is the XOR of the new CF bit and the most significant bit of the result.
// For ROL, OF is the LSB and MSB XOR'd together.
// OF is architecturally only defined for 1-bit rotate.
eor(ARMEmitter::Size::i64Bit, TMP1, Result, Result, ARMEmitter::ShiftType::LSR, Left ? BitSize - 1 : 1);
unsigned OFBit = Left ? 0 : BitSize - 2;
// Invert result so we get inverted carry.
mvn(ARMEmitter::Size::i64Bit, TMP2, Result);
if (CTX->HostFeatures.SupportsFlagM) {
rmif(TMP2, (CFBit - 1) % 64, 1 << 1 /* nzCv */);
rmif(TMP1, OFBit, 1 << 0 /* nzcV */);
} else {
if (OFBit != 0) {
lsr(EmitSize, TMP1, TMP1, OFBit);
}
if (CFBit != 0) {
lsr(EmitSize, TMP2, TMP2, CFBit);
}
mrs(TMP3, ARMEmitter::SystemRegister::NZCV);
bfi(ARMEmitter::Size::i32Bit, TMP3, TMP1, 28 /* V */, 1);
bfi(ARMEmitter::Size::i32Bit, TMP3, TMP2, 29 /* C */, 1);
msr(ARMEmitter::SystemRegister::NZCV, TMP3);
}
}
Bind(&Done);
}
DEF_OP(Extr) {
auto Op = IROp->C<IR::IROp_Extr>();
const auto Dst = GetReg(Node);
@@ -790,7 +897,7 @@ DEF_OP(PDep) {
DEF_OP(PExt) {
auto Op = IROp->C<IR::IROp_PExt>();
const auto OpSize = IROp->Size;
const auto OpSizeBitsM1 = (OpSize * 8) - 1;
const auto OpSizeBitsM1 = IR::OpSizeAsBits(OpSize) - 1;
const auto EmitSize = ConvertSize48(IROp);
const auto Input = GetReg(Op->Input.ID());
@@ -845,8 +952,8 @@ DEF_OP(PExt) {
DEF_OP(LDiv) {
auto Op = IROp->C<IR::IROp_LDiv>();
const uint8_t OpSize = IROp->Size;
const auto EmitSize = OpSize >= 4 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto OpSize = IROp->Size;
const auto EmitSize = OpSize >= IR::OpSize::i32Bit ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto Dst = GetReg(Node);
const auto Upper = GetReg(Op->Upper.ID());
@@ -856,14 +963,14 @@ DEF_OP(LDiv) {
// Each source is OpSize in size
// So you can have up to a 128bit divide from x86-64
switch (OpSize) {
case 2: {
case IR::OpSize::i16Bit: {
uxth(EmitSize, TMP1, Lower);
bfi(EmitSize, TMP1, Upper, 16, 16);
sxth(EmitSize, TMP2, Divisor);
sdiv(EmitSize, Dst, TMP1, TMP2);
break;
}
case 4: {
case IR::OpSize::i32Bit: {
// TODO: 32-bit operation should be guaranteed not to leave garbage in the upper bits.
mov(EmitSize, TMP1, Lower);
bfi(EmitSize, TMP1, Upper, 32, 32);
@@ -871,7 +978,7 @@ DEF_OP(LDiv) {
sdiv(EmitSize, Dst, TMP1, TMP2);
break;
}
case 8: {
case IR::OpSize::i64Bit: {
ARMEmitter::SingleUseForwardLabel Only64Bit {};
ARMEmitter::SingleUseForwardLabel LongDIVRet {};
@@ -915,8 +1022,8 @@ DEF_OP(LDiv) {
DEF_OP(LUDiv) {
auto Op = IROp->C<IR::IROp_LUDiv>();
const uint8_t OpSize = IROp->Size;
const auto EmitSize = OpSize >= 4 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto OpSize = IROp->Size;
const auto EmitSize = OpSize >= IR::OpSize::i32Bit ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto Dst = GetReg(Node);
const auto Upper = GetReg(Op->Upper.ID());
@@ -926,20 +1033,20 @@ DEF_OP(LUDiv) {
// Each source is OpSize in size
// So you can have up to a 128bit divide from x86-64=
switch (OpSize) {
case 2: {
case IR::OpSize::i16Bit: {
uxth(EmitSize, TMP1, Lower);
bfi(EmitSize, TMP1, Upper, 16, 16);
udiv(EmitSize, Dst, TMP1, Divisor);
break;
}
case 4: {
case IR::OpSize::i32Bit: {
// TODO: 32-bit operation should be guaranteed not to leave garbage in the upper bits.
mov(EmitSize, TMP1, Lower);
bfi(EmitSize, TMP1, Upper, 32, 32);
udiv(EmitSize, Dst, TMP1, Divisor);
break;
}
case 8: {
case IR::OpSize::i64Bit: {
ARMEmitter::SingleUseForwardLabel Only64Bit {};
ARMEmitter::SingleUseForwardLabel LongDIVRet {};
@@ -979,8 +1086,8 @@ DEF_OP(LUDiv) {
DEF_OP(LRem) {
auto Op = IROp->C<IR::IROp_LRem>();
const uint8_t OpSize = IROp->Size;
const auto EmitSize = OpSize >= 4 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto OpSize = IROp->Size;
const auto EmitSize = OpSize >= IR::OpSize::i32Bit ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto Dst = GetReg(Node);
const auto Upper = GetReg(Op->Upper.ID());
@@ -990,7 +1097,7 @@ DEF_OP(LRem) {
// Each source is OpSize in size
// So you can have up to a 128bit divide from x86-64
switch (OpSize) {
case 2: {
case IR::OpSize::i16Bit: {
uxth(EmitSize, TMP1, Lower);
bfi(EmitSize, TMP1, Upper, 16, 16);
sxth(EmitSize, TMP2, Divisor);
@@ -998,7 +1105,7 @@ DEF_OP(LRem) {
msub(EmitSize, Dst, TMP3, TMP2, TMP1);
break;
}
case 4: {
case IR::OpSize::i32Bit: {
// TODO: 32-bit operation should be guaranteed not to leave garbage in the upper bits.
mov(EmitSize, TMP1, Lower);
bfi(EmitSize, TMP1, Upper, 32, 32);
@@ -1007,7 +1114,7 @@ DEF_OP(LRem) {
msub(EmitSize, Dst, TMP2, TMP3, TMP1);
break;
}
case 8: {
case IR::OpSize::i64Bit: {
ARMEmitter::SingleUseForwardLabel Only64Bit {};
ARMEmitter::SingleUseForwardLabel LongDIVRet {};
@@ -1053,8 +1160,8 @@ DEF_OP(LRem) {
DEF_OP(LURem) {
auto Op = IROp->C<IR::IROp_LURem>();
const uint8_t OpSize = IROp->Size;
const auto EmitSize = OpSize >= 4 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto OpSize = IROp->Size;
const auto EmitSize = OpSize >= IR::OpSize::i32Bit ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto Dst = GetReg(Node);
const auto Upper = GetReg(Op->Upper.ID());
@@ -1064,14 +1171,14 @@ DEF_OP(LURem) {
// Each source is OpSize in size
// So you can have up to a 128bit divide from x86-64
switch (OpSize) {
case 2: {
case IR::OpSize::i16Bit: {
uxth(EmitSize, TMP1, Lower);
bfi(EmitSize, TMP1, Upper, 16, 16);
udiv(EmitSize, TMP2, TMP1, Divisor);
msub(EmitSize, Dst, TMP2, Divisor, TMP1);
break;
}
case 4: {
case IR::OpSize::i32Bit: {
// TODO: 32-bit operation should be guaranteed not to leave garbage in the upper bits.
mov(EmitSize, TMP1, Lower);
bfi(EmitSize, TMP1, Upper, 32, 32);
@@ -1079,7 +1186,7 @@ DEF_OP(LURem) {
msub(EmitSize, Dst, TMP2, Divisor, TMP1);
break;
}
case 8: {
case IR::OpSize::i64Bit: {
ARMEmitter::SingleUseForwardLabel Only64Bit {};
ARMEmitter::SingleUseForwardLabel LongDIVRet {};
@@ -1131,30 +1238,30 @@ DEF_OP(Not) {
DEF_OP(Popcount) {
auto Op = IROp->C<IR::IROp_Popcount>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
const auto Dst = GetReg(Node);
const auto Src = GetReg(Op->Src.ID());
switch (OpSize) {
case 0x1:
case IR::OpSize::i8Bit:
fmov(ARMEmitter::Size::i32Bit, VTMP1.S(), Src);
// only use lowest byte
cnt(ARMEmitter::SubRegSize::i8Bit, VTMP1.D(), VTMP1.D());
break;
case 0x2:
case IR::OpSize::i16Bit:
fmov(ARMEmitter::Size::i32Bit, VTMP1.S(), Src);
cnt(ARMEmitter::SubRegSize::i8Bit, VTMP1.D(), VTMP1.D());
// only count two lowest bytes
addp(ARMEmitter::SubRegSize::i8Bit, VTMP1.D(), VTMP1.D(), VTMP1.D());
break;
case 0x4:
case IR::OpSize::i32Bit:
fmov(ARMEmitter::Size::i32Bit, VTMP1.S(), Src);
cnt(ARMEmitter::SubRegSize::i8Bit, VTMP1.D(), VTMP1.D());
// fmov has zero extended, unused bytes are zero
addv(ARMEmitter::SubRegSize::i8Bit, VTMP1.D(), VTMP1.D());
break;
case 0x8:
case IR::OpSize::i64Bit:
fmov(ARMEmitter::Size::i64Bit, VTMP1.D(), Src);
cnt(ARMEmitter::SubRegSize::i8Bit, VTMP1.D(), VTMP1.D());
// fmov has zero extended, unused bytes are zero
@@ -1173,34 +1280,27 @@ DEF_OP(FindLSB) {
const auto Dst = GetReg(Node);
const auto Src = GetReg(Op->Src.ID());
if (IROp->Size != 8) {
ubfx(EmitSize, TMP1, Src, 0, IROp->Size * 8);
cmp(EmitSize, TMP1, 0);
rbit(EmitSize, TMP1, TMP1);
} else {
rbit(EmitSize, TMP1, Src);
cmp(EmitSize, Src, 0);
}
// We assume the source is nonzero, so we can just rbit+clz without worrying
// about upper garbage for smaller types.
rbit(EmitSize, TMP1, Src);
clz(EmitSize, Dst, TMP1);
csinv(EmitSize, Dst, Dst, ARMEmitter::Reg::zr, ARMEmitter::Condition::CC_NE);
}
DEF_OP(FindMSB) {
auto Op = IROp->C<IR::IROp_FindMSB>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
LOGMAN_THROW_AA_FMT(OpSize == 2 || OpSize == 4 || OpSize == 8, "Unsupported {} size: {}", __func__, OpSize);
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i16Bit || OpSize == IR::OpSize::i32Bit || OpSize == IR::OpSize::i64Bit,
"Unsupported {} size: {}", __func__, OpSize);
const auto EmitSize = ConvertSize(IROp);
const auto Dst = GetReg(Node);
const auto Src = GetReg(Op->Src.ID());
movz(ARMEmitter::Size::i64Bit, TMP1, OpSize * 8 - 1);
movz(ARMEmitter::Size::i64Bit, TMP1, IR::OpSizeAsBits(OpSize) - 1);
if (OpSize == 2) {
if (OpSize == IR::OpSize::i16Bit) {
lsl(EmitSize, Dst, Src, 16);
orr(EmitSize, Dst, Dst, 0x8000);
clz(EmitSize, Dst, Dst);
} else {
clz(EmitSize, Dst, Src);
@@ -1211,9 +1311,10 @@ DEF_OP(FindMSB) {
DEF_OP(FindTrailingZeroes) {
auto Op = IROp->C<IR::IROp_FindTrailingZeroes>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
LOGMAN_THROW_AA_FMT(OpSize == 2 || OpSize == 4 || OpSize == 8, "Unsupported {} size: {}", __func__, OpSize);
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i16Bit || OpSize == IR::OpSize::i32Bit || OpSize == IR::OpSize::i64Bit,
"Unsupported {} size: {}", __func__, OpSize);
const auto EmitSize = ConvertSize(IROp);
const auto Dst = GetReg(Node);
@@ -1221,7 +1322,7 @@ DEF_OP(FindTrailingZeroes) {
rbit(EmitSize, Dst, Src);
if (OpSize == 2) {
if (OpSize == IR::OpSize::i16Bit) {
// This orr does two things. First, if the (masked) source is zero, it
// reverses to zero in the top so it forces clz to return 16. Second, it
// ensures garbage in the upper bits of the source don't affect clz, because
@@ -1235,15 +1336,16 @@ DEF_OP(FindTrailingZeroes) {
DEF_OP(CountLeadingZeroes) {
auto Op = IROp->C<IR::IROp_CountLeadingZeroes>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
LOGMAN_THROW_AA_FMT(OpSize == 2 || OpSize == 4 || OpSize == 8, "Unsupported {} size: {}", __func__, OpSize);
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i16Bit || OpSize == IR::OpSize::i32Bit || OpSize == IR::OpSize::i64Bit,
"Unsupported {} size: {}", __func__, OpSize);
const auto EmitSize = ConvertSize(IROp);
const auto Dst = GetReg(Node);
const auto Src = GetReg(Op->Src.ID());
if (OpSize == 2) {
if (OpSize == IR::OpSize::i16Bit) {
// Expressing as lsl+orr+clz clears away any garbage in the upper bits
// (alternatively could do uxth+clz+sub.. equal cost in total).
lsl(EmitSize, Dst, Src, 16);
@@ -1256,16 +1358,17 @@ DEF_OP(CountLeadingZeroes) {
DEF_OP(Rev) {
auto Op = IROp->C<IR::IROp_Rev>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
LOGMAN_THROW_AA_FMT(OpSize == 2 || OpSize == 4 || OpSize == 8, "Unsupported {} size: {}", __func__, OpSize);
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i16Bit || OpSize == IR::OpSize::i32Bit || OpSize == IR::OpSize::i64Bit,
"Unsupported {} size: {}", __func__, OpSize);
const auto EmitSize = ConvertSize(IROp);
const auto Dst = GetReg(Node);
const auto Src = GetReg(Op->Src.ID());
rev(EmitSize, Dst, Src);
if (OpSize == 2) {
if (OpSize == IR::OpSize::i16Bit) {
lsr(EmitSize, Dst, Dst, 16);
}
}
@@ -1291,10 +1394,10 @@ DEF_OP(Bfi) {
mov(EmitSize, TMP1, SrcDst);
bfi(EmitSize, TMP1, Src, Op->lsb, Op->Width);
if (IROp->Size >= 4) {
if (IROp->Size >= IR::OpSize::i32Bit) {
mov(EmitSize, Dst, TMP1.R());
} else {
ubfx(EmitSize, Dst, TMP1, 0, IROp->Size * 8);
ubfx(EmitSize, Dst, TMP1, 0, IR::OpSizeAsBits(IROp->Size));
}
}
}
@@ -1325,7 +1428,7 @@ DEF_OP(Bfxil) {
DEF_OP(Bfe) {
auto Op = IROp->C<IR::IROp_Bfe>();
LOGMAN_THROW_AA_FMT(IROp->Size <= 8, "OpSize is too large for BFE: {}", IROp->Size);
LOGMAN_THROW_AA_FMT(IROp->Size <= IR::OpSize::i64Bit, "OpSize is too large for BFE: {}", IROp->Size);
LOGMAN_THROW_AA_FMT(Op->Width != 0, "Invalid BFE width of 0");
const auto EmitSize = ConvertSize(IROp);
@@ -1335,7 +1438,7 @@ DEF_OP(Bfe) {
if (Op->lsb == 0 && Op->Width == 32) {
mov(ARMEmitter::Size::i32Bit, Dst, Src);
} else if (Op->lsb == 0 && Op->Width == 64) {
LOGMAN_THROW_AA_FMT(IROp->Size == 8, "Must be 64-bit wide register");
LOGMAN_THROW_AA_FMT(IROp->Size == IR::OpSize::i64Bit, "Must be 64-bit wide register");
mov(ARMEmitter::Size::i64Bit, Dst, Src);
} else {
ubfx(EmitSize, Dst, Src, Op->lsb, Op->Width);
@@ -1352,9 +1455,9 @@ DEF_OP(Sbfe) {
DEF_OP(Select) {
auto Op = IROp->C<IR::IROp_Select>();
const uint8_t OpSize = IROp->Size;
const auto OpSize = IROp->Size;
const auto EmitSize = ConvertSize(IROp);
const auto CompareEmitSize = Op->CompareSize == 8 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto CompareEmitSize = Op->CompareSize == IR::OpSize::i64Bit ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
uint64_t Const;
auto cc = MapCC(Op->Cond);
@@ -1368,15 +1471,10 @@ DEF_OP(Select) {
const auto Src2 = GetReg(Op->Cmp2.ID());
cmp(CompareEmitSize, Src1, Src2);
}
} else if (IsGPRPair(Op->Cmp1.ID())) {
const auto Src1 = GetRegPair(Op->Cmp1.ID());
const auto Src2 = GetRegPair(Op->Cmp2.ID());
cmp(EmitSize, Src1.first, Src2.first);
ccmp(EmitSize, Src1.second, Src2.second, ARMEmitter::StatusFlags::None, cc);
} else if (IsFPR(Op->Cmp1.ID())) {
const auto Src1 = GetVReg(Op->Cmp1.ID());
const auto Src2 = GetVReg(Op->Cmp2.ID());
fcmp(Op->CompareSize == 8 ? ARMEmitter::ScalarRegSize::i64Bit : ARMEmitter::ScalarRegSize::i32Bit, Src1, Src2);
fcmp(Op->CompareSize == IR::OpSize::i64Bit ? ARMEmitter::ScalarRegSize::i64Bit : ARMEmitter::ScalarRegSize::i32Bit, Src1, Src2);
} else {
LOGMAN_MSG_A_FMT("Select: Expected GPR or FPR");
}
@@ -1385,7 +1483,7 @@ DEF_OP(Select) {
bool is_const_true = IsInlineConstant(Op->TrueVal, &const_true);
bool is_const_false = IsInlineConstant(Op->FalseVal, &const_false);
uint64_t all_ones = OpSize == 8 ? 0xffff'ffff'ffff'ffffull : 0xffff'ffffull;
uint64_t all_ones = OpSize == IR::OpSize::i64Bit ? 0xffff'ffff'ffff'ffffull : 0xffff'ffffull;
ARMEmitter::Register Dst = GetReg(Node);
@@ -1414,7 +1512,7 @@ DEF_OP(NZCVSelect) {
bool is_const_true = IsInlineConstant(Op->TrueVal, &const_true);
bool is_const_false = IsInlineConstant(Op->FalseVal, &const_false);
uint64_t all_ones = IROp->Size == 8 ? 0xffff'ffff'ffff'ffffull : 0xffff'ffffull;
uint64_t all_ones = IROp->Size == IR::OpSize::i64Bit ? 0xffff'ffff'ffff'ffffull : 0xffff'ffffull;
ARMEmitter::Register Dst = GetReg(Node);
@@ -1433,26 +1531,41 @@ DEF_OP(NZCVSelect) {
}
}
DEF_OP(NZCVSelectV) {
auto Op = IROp->C<IR::IROp_NZCVSelectV>();
auto cc = MapCC(Op->Cond);
const auto SubRegSize = ConvertSubRegSizePair248(IROp);
fcsel(SubRegSize.Scalar, GetVReg(Node), GetVReg(Op->TrueVal.ID()), GetVReg(Op->FalseVal.ID()), cc);
}
DEF_OP(NZCVSelectIncrement) {
auto Op = IROp->C<IR::IROp_NZCVSelectIncrement>();
csinc(ConvertSize(IROp), GetReg(Node), GetReg(Op->TrueVal.ID()), GetZeroableReg(Op->FalseVal), MapCC(Op->Cond));
}
DEF_OP(VExtractToGPR) {
const auto Op = IROp->C<IR::IROp_VExtractToGPR>();
const auto OpSize = IROp->Size;
constexpr auto AVXRegBitSize = Core::CPUState::XMM_AVX_REG_SIZE * 8;
constexpr auto SSERegBitSize = Core::CPUState::XMM_SSE_REG_SIZE * 8;
const auto ElementSizeBits = Op->Header.ElementSize * 8;
const auto ElementSizeBits = IR::OpSizeAsBits(Op->Header.ElementSize);
const auto Offset = ElementSizeBits * Op->Index;
const auto Is256Bit = Offset >= SSERegBitSize;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Dst = GetReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
const auto PerformMove = [&](const ARMEmitter::VRegister reg, int index) {
switch (OpSize) {
case 1: umov<ARMEmitter::SubRegSize::i8Bit>(Dst, Vector, index); break;
case 2: umov<ARMEmitter::SubRegSize::i16Bit>(Dst, Vector, index); break;
case 4: umov<ARMEmitter::SubRegSize::i32Bit>(Dst, Vector, index); break;
case 8: umov<ARMEmitter::SubRegSize::i64Bit>(Dst, Vector, index); break;
case IR::OpSize::i8Bit: umov<ARMEmitter::SubRegSize::i8Bit>(Dst, Vector, index); break;
case IR::OpSize::i16Bit: umov<ARMEmitter::SubRegSize::i16Bit>(Dst, Vector, index); break;
case IR::OpSize::i32Bit: umov<ARMEmitter::SubRegSize::i32Bit>(Dst, Vector, index); break;
case IR::OpSize::i64Bit: umov<ARMEmitter::SubRegSize::i64Bit>(Dst, Vector, index); break;
default: LOGMAN_MSG_A_FMT("Unhandled ExtractElementSize: {}", OpSize); break;
}
};
@@ -1463,7 +1576,6 @@ DEF_OP(VExtractToGPR) {
// when acting on larger register sizes.
PerformMove(Vector, Op->Index);
} else {
LOGMAN_THROW_AA_FMT(HostSupportsSVE256, "Host doesn't support SVE. Cannot perform 256-bit operation.");
LOGMAN_THROW_AA_FMT(Is256Bit, "Can't perform 256-bit extraction with op side: {}", OpSize);
LOGMAN_THROW_AA_FMT(Offset < AVXRegBitSize, "Trying to extract element outside bounds of register. Offset={}, Index={}", Offset, Op->Index);
@@ -1478,10 +1590,10 @@ DEF_OP(VExtractToGPR) {
// upper half of the vector.
const auto SanitizedIndex = [OpSize, Op] {
switch (OpSize) {
case 1: return Op->Index - 16;
case 2: return Op->Index - 8;
case 4: return Op->Index - 4;
case 8: return Op->Index - 2;
case IR::OpSize::i8Bit: return Op->Index - 16;
case IR::OpSize::i16Bit: return Op->Index - 8;
case IR::OpSize::i32Bit: return Op->Index - 4;
case IR::OpSize::i64Bit: return Op->Index - 2;
default: LOGMAN_MSG_A_FMT("Unhandled OpSize: {}", OpSize); return 0;
}
}();
@@ -1497,7 +1609,7 @@ DEF_OP(Float_ToGPR_ZS) {
ARMEmitter::Register Dst = GetReg(Node);
ARMEmitter::VRegister Src = GetVReg(Op->Scalar.ID());
if (Op->SrcElementSize == 8) {
if (Op->SrcElementSize == IR::OpSize::i64Bit) {
fcvtzs(ConvertSize(IROp), Dst, Src.D());
} else {
fcvtzs(ConvertSize(IROp), Dst, Src.S());
@@ -1510,7 +1622,7 @@ DEF_OP(Float_ToGPR_S) {
ARMEmitter::Register Dst = GetReg(Node);
ARMEmitter::VRegister Src = GetVReg(Op->Scalar.ID());
if (Op->SrcElementSize == 8) {
if (Op->SrcElementSize == IR::OpSize::i64Bit) {
frinti(VTMP1.D(), Src.D());
fcvtzs(ConvertSize(IROp), Dst, VTMP1.D());
} else {
@@ -1521,7 +1633,7 @@ DEF_OP(Float_ToGPR_S) {
DEF_OP(FCmp) {
auto Op = IROp->C<IR::IROp_FCmp>();
const auto EmitSubSize = Op->ElementSize == 8 ? ARMEmitter::ScalarRegSize::i64Bit : ARMEmitter::ScalarRegSize::i32Bit;
const auto EmitSubSize = Op->ElementSize == IR::OpSize::i64Bit ? ARMEmitter::ScalarRegSize::i64Bit : ARMEmitter::ScalarRegSize::i32Bit;
ARMEmitter::VRegister Scalar1 = GetVReg(Op->Scalar1.ID());
ARMEmitter::VRegister Scalar2 = GetVReg(Op->Scalar2.ID());
@@ -1,68 +0,0 @@
// SPDX-License-Identifier: MIT
/*
$info$
tags: backend|arm64
$end_info$
*/
#include "Interface/Core/JIT/Arm64/JITClass.h"
namespace FEXCore::CPU {
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const* IROp, IR::NodeID Node)
DEF_OP(ExtractElementPair) {
auto Op = IROp->C<IR::IROp_ExtractElementPair>();
const auto Dst = GetReg(Node);
const auto Pair = GetRegPair(Op->Pair.ID());
const auto Src = Op->Element == 0 ? Pair.first : Pair.second;
if (Dst != Src) {
mov(ConvertSize48(IROp), Dst, Src);
}
}
DEF_OP(CreateElementPair) {
auto Op = IROp->C<IR::IROp_CreateElementPair>();
LOGMAN_THROW_AA_FMT(IROp->ElementSize == 4 || IROp->ElementSize == 8, "Invalid size");
std::pair<ARMEmitter::Register, ARMEmitter::Register> Dst = GetRegPair(Node);
ARMEmitter::Register RegFirst = GetReg(Op->Lower.ID());
ARMEmitter::Register RegSecond = GetReg(Op->Upper.ID());
ARMEmitter::Register RegTmp = TMP1.R();
const auto EmitSize = IROp->ElementSize == 8 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
if (Dst.first.Idx() != RegSecond.Idx()) {
mov(EmitSize, Dst.first, RegFirst);
mov(EmitSize, Dst.second, RegSecond);
} else if (Dst.second.Idx() != RegFirst.Idx()) {
mov(EmitSize, Dst.second, RegSecond);
mov(EmitSize, Dst.first, RegFirst);
} else {
mov(EmitSize, RegTmp, RegFirst);
mov(EmitSize, Dst.second, RegSecond);
mov(EmitSize, Dst.first, RegTmp);
}
}
DEF_OP(Copy) {
auto Op = IROp->C<IR::IROp_Copy>();
mov(ARMEmitter::Size::i64Bit, GetReg(Node), GetReg(Op->Source.ID()));
}
DEF_OP(Swap1) {
auto Op = IROp->C<IR::IROp_Swap1>();
auto A = GetReg(Op->A.ID()), B = GetReg(Op->B.ID());
LOGMAN_THROW_A_FMT(B == GetReg(Node), "Invariant");
mov(ARMEmitter::Size::i64Bit, TMP1, A);
mov(ARMEmitter::Size::i64Bit, A, B);
mov(ARMEmitter::Size::i64Bit, B, TMP1);
}
DEF_OP(Swap2) {
// Implemented above
}
#undef DEF_OP
} // namespace FEXCore::CPU
@@ -6,8 +6,9 @@ desc: relocation logic of the arm64 splatter backend
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/HLE/Thunks/Thunks.h"
#include "Interface/Core/JIT/JITClass.h"
#include <FEXCore/Core/Thunks.h>
namespace FEXCore::CPU {
@@ -7,27 +7,51 @@ $end_info$
#include "Interface/Context/Context.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/JIT/JITClass.h"
namespace FEXCore::CPU {
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const* IROp, IR::NodeID Node)
DEF_OP(CASPair) {
auto Op = IROp->C<IR::IROp_CASPair>();
LOGMAN_THROW_AA_FMT(IROp->ElementSize == 4 || IROp->ElementSize == 8, "Wrong element size");
LOGMAN_THROW_AA_FMT(IROp->ElementSize == IR::OpSize::i32Bit || IROp->ElementSize == IR::OpSize::i64Bit, "Wrong element size");
// Size is the size of each pair element
auto Dst = GetRegPair(Node);
auto Expected = GetRegPair(Op->Expected.ID());
auto Desired = GetRegPair(Op->Desired.ID());
auto Dst0 = GetReg(Op->OutLo.ID());
auto Dst1 = GetReg(Op->OutHi.ID());
auto Expected0 = GetReg(Op->ExpectedLo.ID());
auto Expected1 = GetReg(Op->ExpectedHi.ID());
auto Desired0 = GetReg(Op->DesiredLo.ID());
auto Desired1 = GetReg(Op->DesiredHi.ID());
auto MemSrc = GetReg(Op->Addr.ID());
const auto EmitSize = IROp->ElementSize == 8 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
const auto EmitSize = IROp->ElementSize == IR::OpSize::i64Bit ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
if (CTX->HostFeatures.SupportsAtomics) {
mov(EmitSize, TMP3, Expected.first);
mov(EmitSize, TMP4, Expected.second);
// RA has heuristics to try to pair sources, but we need to handle the cases
// where they fail. We do so by moving to temporaries. Note we use 64-bit
// moves here even for 32-bit cmpxchg, for the Firestorm register renamer.
if (Desired1.Idx() != (Desired0.Idx() + 1) || Desired0.Idx() & 1) {
mov(ARMEmitter::Size::i64Bit, TMP1, Desired0);
mov(ARMEmitter::Size::i64Bit, TMP2, Desired1);
Desired0 = TMP1;
Desired1 = TMP2;
}
caspal(EmitSize, TMP3, TMP4, Desired.first, Desired.second, MemSrc);
mov(EmitSize, Dst.first, TMP3.R());
mov(EmitSize, Dst.second, TMP4.R());
auto CaspalDst0 = Dst0;
auto CaspalDst1 = Dst1;
if (CaspalDst1.Idx() != (CaspalDst0.Idx() + 1) || CaspalDst0.Idx() & 1) {
CaspalDst0 = TMP3;
CaspalDst1 = TMP4;
}
// We can't clobber the source, these moves are inherently required due to
// ISA limitations. But by making them 64-bit, Firestorm can rename.
mov(ARMEmitter::Size::i64Bit, CaspalDst0, Expected0);
mov(ARMEmitter::Size::i64Bit, CaspalDst1, Expected1);
caspal(EmitSize, CaspalDst0, CaspalDst1, Desired0, Desired1, MemSrc);
if (CaspalDst0 != Dst0) {
mov(ARMEmitter::Size::i64Bit, Dst0, CaspalDst0);
mov(ARMEmitter::Size::i64Bit, Dst1, CaspalDst1);
}
} else {
// Save NZCV so we don't have to mark this op as clobbering NZCV (the
// SupportsAtomics does not clobber atomics and this !SupportsAtomics path
@@ -43,19 +67,19 @@ DEF_OP(CASPair) {
// This instruction sequence must be synced with HandleCASPAL_Armv8.
ldaxp(EmitSize, TMP2, TMP3, MemSrc);
cmp(EmitSize, TMP2, Expected.first);
ccmp(EmitSize, TMP3, Expected.second, ARMEmitter::StatusFlags::None, ARMEmitter::Condition::CC_EQ);
cmp(EmitSize, TMP2, Expected0);
ccmp(EmitSize, TMP3, Expected1, ARMEmitter::StatusFlags::None, ARMEmitter::Condition::CC_EQ);
b(ARMEmitter::Condition::CC_NE, &LoopNotExpected);
stlxp(EmitSize, TMP2, Desired.first, Desired.second, MemSrc);
stlxp(EmitSize, TMP2, Desired0, Desired1, MemSrc);
cbnz(EmitSize, TMP2, &LoopTop);
mov(EmitSize, Dst.first, Expected.first);
mov(EmitSize, Dst.second, Expected.second);
mov(EmitSize, Dst0, Expected0);
mov(EmitSize, Dst1, Expected1);
b(&LoopExpected);
Bind(&LoopNotExpected);
mov(EmitSize, Dst.first, TMP2.R());
mov(EmitSize, Dst.second, TMP3.R());
mov(EmitSize, Dst0, TMP2.R());
mov(EmitSize, Dst1, TMP3.R());
// exclusive monitor needs to be cleared here
// Might have hit the case where ldaxr was hit but stlxr wasn't
clrex();
@@ -88,9 +112,9 @@ DEF_OP(CAS) {
ARMEmitter::SingleUseForwardLabel LoopExpected;
Bind(&LoopTop);
ldaxr(SubEmitSize, TMP2, MemSrc);
if (IROp->Size == 1) {
if (IROp->Size == IR::OpSize::i8Bit) {
cmp(EmitSize, TMP2, Expected, ARMEmitter::ExtendedType::UXTB, 0);
} else if (IROp->Size == 2) {
} else if (IROp->Size == IR::OpSize::i16Bit) {
cmp(EmitSize, TMP2, Expected, ARMEmitter::ExtendedType::UXTH, 0);
} else {
cmp(EmitSize, TMP2, Expected);
@@ -249,18 +273,21 @@ DEF_OP(AtomicNeg) {
DEF_OP(AtomicSwap) {
auto Op = IROp->C<IR::IROp_AtomicSwap>();
uint8_t OpSize = IROp->Size;
LOGMAN_THROW_AA_FMT(OpSize == 8 || OpSize == 4 || OpSize == 2 || OpSize == 1, "Unexpected CAS size");
const auto OpSize = IROp->Size;
LOGMAN_THROW_AA_FMT(
OpSize == IR::OpSize::i64Bit || OpSize == IR::OpSize::i32Bit || OpSize == IR::OpSize::i16Bit || OpSize == IR::OpSize::i8Bit, "Unexpecte"
"d CAS "
"size");
auto MemSrc = GetReg(Op->Addr.ID());
auto Src = GetReg(Op->Value.ID());
const auto EmitSize = ConvertSize(IROp);
const auto SubEmitSize = OpSize == 8 ? ARMEmitter::SubRegSize::i64Bit :
OpSize == 4 ? ARMEmitter::SubRegSize::i32Bit :
OpSize == 2 ? ARMEmitter::SubRegSize::i16Bit :
OpSize == 1 ? ARMEmitter::SubRegSize::i8Bit :
ARMEmitter::SubRegSize::i8Bit;
const auto SubEmitSize = OpSize == IR::OpSize::i64Bit ? ARMEmitter::SubRegSize::i64Bit :
OpSize == IR::OpSize::i32Bit ? ARMEmitter::SubRegSize::i32Bit :
OpSize == IR::OpSize::i16Bit ? ARMEmitter::SubRegSize::i16Bit :
OpSize == IR::OpSize::i8Bit ? ARMEmitter::SubRegSize::i8Bit :
ARMEmitter::SubRegSize::i8Bit;
if (CTX->HostFeatures.SupportsAtomics) {
ldswpal(SubEmitSize, Src, GetReg(Node), MemSrc);
@@ -270,7 +297,7 @@ DEF_OP(AtomicSwap) {
ldaxr(SubEmitSize, TMP2, MemSrc);
stlxr(SubEmitSize, TMP4, Src, MemSrc);
cbnz(EmitSize, TMP4, &LoopTop);
ubfm(EmitSize, GetReg(Node), TMP2, 0, OpSize * 8 - 1);
ubfm(EmitSize, GetReg(Node), TMP2, 0, IR::OpSizeAsBits(OpSize) - 1);
}
}
@@ -9,13 +9,13 @@ $end_info$
#include "FEXCore/IR/IR.h"
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/JIT/JITClass.h"
#include <FEXCore/Core/Thunks.h>
#include <FEXCore/Core/X86Enums.h>
#include <FEXCore/Debug/InternalThreadState.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include <FEXCore/Utils/MathUtils.h>
#include <Interface/HLE/Thunks/Thunks.h>
namespace FEXCore::CPU {
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const* IROp, IR::NodeID Node)
@@ -78,8 +78,12 @@ DEF_OP(ExitFunction) {
// L1 Cache
ldr(TMP1, STATE, offsetof(FEXCore::Core::CpuStateFrame, Pointers.Common.L1Pointer));
and_(ARMEmitter::Size::i64Bit, TMP4, RipReg, LookupCache::L1_ENTRIES_MASK);
add(TMP1, TMP1, TMP4, ARMEmitter::ShiftType::LSL, 4);
// Calculate (tmp1 + ((ripreg & L1_ENTRIES_MASK) << 4)) for the address
// arithmetic. ubfiz+add is marginally faster on Firestorm than
// and+add(shift). Same performance on Cortex.
static_assert(LookupCache::L1_ENTRIES_MASK == ((1u << 20) - 1));
ubfiz(ARMEmitter::Size::i64Bit, TMP4, RipReg, 4, 20);
add(TMP1, TMP1, TMP4);
// Note: sub+cbnz used over cmp+br to preserve flags.
ldp<ARMEmitter::IndexType::OFFSET>(TMP2, TMP1, TMP1, 0);
@@ -112,20 +116,27 @@ DEF_OP(CondJump) {
[[maybe_unused]] uint64_t Const;
[[maybe_unused]] const bool isConst = IsInlineConstant(Op->Cmp2, &Const);
const auto Size = Op->CompareSize == 4 ? ARMEmitter::Size::i32Bit : ARMEmitter::Size::i64Bit;
auto Reg = GetReg(Op->Cmp1.ID());
const auto Size = Op->CompareSize == IR::OpSize::i32Bit ? ARMEmitter::Size::i32Bit : ARMEmitter::Size::i64Bit;
LOGMAN_THROW_A_FMT(IsGPR(Op->Cmp1.ID()), "CondJump: Expected GPR");
LOGMAN_THROW_A_FMT(isConst && Const == 0, "CondJump: Expected 0 source");
LOGMAN_THROW_A_FMT(Op->Cond.Val == FEXCore::IR::COND_EQ || Op->Cond.Val == FEXCore::IR::COND_NEQ, "CondJump: Expected simple "
"condition");
LOGMAN_THROW_A_FMT(isConst, "CondJump: Expected constant source");
if (Op->Cond.Val == FEXCore::IR::COND_EQ) {
cbz(Size, GetReg(Op->Cmp1.ID()), TrueTargetLabel);
LOGMAN_THROW_A_FMT(Const == 0, "CondJump: Expected 0 source");
cbz(Size, Reg, TrueTargetLabel);
} else if (Op->Cond.Val == FEXCore::IR::COND_NEQ) {
LOGMAN_THROW_A_FMT(Const == 0, "CondJump: Expected 0 source");
cbnz(Size, Reg, TrueTargetLabel);
} else if (Op->Cond.Val == FEXCore::IR::COND_TSTZ) {
LOGMAN_THROW_A_FMT(Const < 64, "CondJump: Expected valid bit source");
tbz(Reg, Const, TrueTargetLabel);
} else if (Op->Cond.Val == FEXCore::IR::COND_TSTNZ) {
LOGMAN_THROW_A_FMT(Const < 64, "CondJump: Expected valid bit source");
tbnz(Reg, Const, TrueTargetLabel);
} else {
cbnz(Size, GetReg(Op->Cmp1.ID()), TrueTargetLabel);
LOGMAN_THROW_A_FMT(false, "CondJump expected simple condition");
}
// TODO: Wire up tbz/tbnz
}
PendingTargetLabel = &JumpTargets.try_emplace(Op->FalseBlock.ID()).first->second;
@@ -255,15 +266,13 @@ DEF_OP(InlineSyscall) {
}
auto Reg = GetReg(Op->Header.Args[i].ID());
// In the case of intersection with x4, x5, or x8 then these are currently SRA
// for registers RAX, RBX, and RSI. Which have just been spilled
// Just load back from the context. Could be slightly smarter but this is fairly uncommon
if (Reg == ARMEmitter::Reg::r8) {
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RSP]));
} else if (Reg == ARMEmitter::Reg::r4) {
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RAX]));
} else if (Reg == ARMEmitter::Reg::r5) {
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[X86State::REG_RCX]));
if (SpillMask & (1U << Reg.Idx())) {
// In the case of intersection with x4, x5, or x8 then these are currently SRA
// for registers RAX, RDX, and RSP. Which have just been spilled
// Just load back from the context.
auto Correlation = GetX86RegRelationToARMReg(Reg);
LOGMAN_THROW_A_FMT(Correlation != X86State::REG_INVALID, "Invalid register mapping");
ldr(EmitSubSize, RegArgs[i].R(), STATE, offsetof(FEXCore::Core::CpuStateFrame, State.gregs[Correlation]));
} else {
mov(EmitSize, RegArgs[i].R(), Reg);
}
@@ -427,10 +436,11 @@ DEF_OP(CPUID) {
PopDynamicRegsAndLR();
// Results are in x0, x1
// Results want to be in a i64v2 vector
auto Dst = GetRegPair(Node);
mov(ARMEmitter::Size::i64Bit, Dst.first, TMP1);
mov(ARMEmitter::Size::i64Bit, Dst.second, TMP2);
// Results want to be 4xi32 scalars
mov(ARMEmitter::Size::i32Bit, GetReg(Op->OutEAX.ID()), TMP1);
mov(ARMEmitter::Size::i32Bit, GetReg(Op->OutECX.ID()), TMP2);
ubfx(ARMEmitter::Size::i64Bit, GetReg(Op->OutEBX.ID()), TMP1, 32, 32);
ubfx(ARMEmitter::Size::i64Bit, GetReg(Op->OutEDX.ID()), TMP2, 32, 32);
}
DEF_OP(XGetBV) {
@@ -459,11 +469,9 @@ DEF_OP(XGetBV) {
PopDynamicRegsAndLR();
// Results are in x0
// Results want to be in a i32v2 vector
auto Dst = GetRegPair(Node);
mov(ARMEmitter::Size::i32Bit, Dst.first, TMP1);
lsr(ARMEmitter::Size::i64Bit, Dst.second, TMP1, 32);
// Results are in x0, need to split into i32 parts
mov(ARMEmitter::Size::i32Bit, GetReg(Op->OutEAX.ID()), TMP1);
ubfx(ARMEmitter::Size::i64Bit, GetReg(Op->OutEDX.ID()), TMP1, 32, 32);
}
#undef DEF_OP
@@ -5,8 +5,7 @@ tags: backend|arm64
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/JIT/JITClass.h"
namespace FEXCore::CPU {
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const* IROp, IR::NodeID Node)
@@ -16,17 +15,18 @@ DEF_OP(VInsGPR) {
const auto DestIdx = Op->DestIdx;
const auto ElementSize = Op->Header.ElementSize;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Is256Bit = OpSize == IR::OpSize::i256Bit;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto SubEmitSize = ConvertSubRegSize8(IROp);
const auto ElementsPer128Bit = 16 / ElementSize;
const auto ElementsPer128Bit = IR::NumElements(IR::OpSize::i128Bit, ElementSize);
const auto Dst = GetVReg(Node);
const auto DestVector = GetVReg(Op->DestVector.ID());
const auto Src = GetReg(Op->Src.ID());
if (HostSupportsSVE256 && Is256Bit) {
const auto ElementSizeBits = ElementSize * 8;
const auto ElementSizeBits = IR::OpSizeAsBits(ElementSize);
const auto Offset = ElementSizeBits * DestIdx;
const auto SSEBitSize = Core::CPUState::XMM_SSE_REG_SIZE * 8;
@@ -90,16 +90,16 @@ DEF_OP(VCastFromGPR) {
auto Src = GetReg(Op->Src.ID());
switch (Op->Header.ElementSize) {
case 1:
case IR::OpSize::i8Bit:
uxtb(ARMEmitter::Size::i32Bit, TMP1, Src);
fmov(ARMEmitter::Size::i32Bit, Dst.S(), TMP1);
break;
case 2:
case IR::OpSize::i16Bit:
uxth(ARMEmitter::Size::i32Bit, TMP1, Src);
fmov(ARMEmitter::Size::i32Bit, Dst.S(), TMP1);
break;
case 4: fmov(ARMEmitter::Size::i32Bit, Dst.S(), Src); break;
case 8: fmov(ARMEmitter::Size::i64Bit, Dst.D(), Src); break;
case IR::OpSize::i32Bit: fmov(ARMEmitter::Size::i32Bit, Dst.S(), Src); break;
case IR::OpSize::i64Bit: fmov(ARMEmitter::Size::i64Bit, Dst.D(), Src); break;
default: LOGMAN_MSG_A_FMT("Unknown castGPR element size: {}", Op->Header.ElementSize);
}
}
@@ -111,7 +111,9 @@ DEF_OP(VDupFromGPR) {
const auto Dst = GetVReg(Node);
const auto Src = GetReg(Op->Src.ID());
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Is256Bit = OpSize == IR::OpSize::i256Bit;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto SubEmitSize = ConvertSubRegSize8(IROp);
if (HostSupportsSVE256 && Is256Bit) {
@@ -124,8 +126,8 @@ DEF_OP(VDupFromGPR) {
DEF_OP(Float_FromGPR_S) {
const auto Op = IROp->C<IR::IROp_Float_FromGPR_S>();
const uint16_t ElementSize = Op->Header.ElementSize;
const uint16_t Conv = (ElementSize << 8) | Op->SrcElementSize;
const uint16_t ElementSize = IR::OpSizeToSize(Op->Header.ElementSize);
const uint16_t Conv = (ElementSize << 8) | IR::OpSizeToSize(Op->SrcElementSize);
auto Dst = GetVReg(Node);
auto Src = GetReg(Op->Src.ID());
@@ -163,7 +165,7 @@ DEF_OP(Float_FromGPR_S) {
DEF_OP(Float_FToF) {
auto Op = IROp->C<IR::IROp_Float_FToF>();
const uint16_t Conv = (Op->Header.ElementSize << 8) | Op->SrcElementSize;
const uint16_t Conv = (IR::OpSizeToSize(Op->Header.ElementSize) << 8) | IR::OpSizeToSize(Op->SrcElementSize);
auto Dst = GetVReg(Node);
auto Src = GetVReg(Op->Scalar.ID());
@@ -203,7 +205,8 @@ DEF_OP(Vector_SToF) {
const auto ElementSize = Op->Header.ElementSize;
const auto SubEmitSize = ConvertSubRegSize248(IROp);
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Is256Bit = OpSize == IR::OpSize::i256Bit;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
@@ -212,15 +215,15 @@ DEF_OP(Vector_SToF) {
scvtf(Dst.Z(), SubEmitSize, Mask.Merging(), Vector.Z(), SubEmitSize);
} else {
if (OpSize == ElementSize) {
if (ElementSize == 8) {
if (ElementSize == IR::OpSize::i64Bit) {
scvtf(ARMEmitter::ScalarRegSize::i64Bit, Dst.D(), Vector.D());
} else if (ElementSize == 4) {
} else if (ElementSize == IR::OpSize::i32Bit) {
scvtf(ARMEmitter::ScalarRegSize::i32Bit, Dst.S(), Vector.S());
} else {
scvtf(ARMEmitter::ScalarRegSize::i16Bit, Dst.H(), Vector.H());
}
} else {
if (OpSize == 8) {
if (OpSize == IR::OpSize::i64Bit) {
scvtf(SubEmitSize, Dst.D(), Vector.D());
} else {
scvtf(SubEmitSize, Dst.Q(), Vector.Q());
@@ -235,7 +238,8 @@ DEF_OP(Vector_FToZS) {
const auto ElementSize = Op->Header.ElementSize;
const auto SubEmitSize = ConvertSubRegSize248(IROp);
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Is256Bit = OpSize == IR::OpSize::i256Bit;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
@@ -244,15 +248,15 @@ DEF_OP(Vector_FToZS) {
fcvtzs(Dst.Z(), SubEmitSize, Mask.Merging(), Vector.Z(), SubEmitSize);
} else {
if (OpSize == ElementSize) {
if (ElementSize == 8) {
if (ElementSize == IR::OpSize::i64Bit) {
fcvtzs(ARMEmitter::ScalarRegSize::i64Bit, Dst.D(), Vector.D());
} else if (ElementSize == 4) {
} else if (ElementSize == IR::OpSize::i32Bit) {
fcvtzs(ARMEmitter::ScalarRegSize::i32Bit, Dst.S(), Vector.S());
} else {
fcvtzs(ARMEmitter::ScalarRegSize::i16Bit, Dst.H(), Vector.H());
}
} else {
if (OpSize == 8) {
if (OpSize == IR::OpSize::i64Bit) {
fcvtzs(SubEmitSize, Dst.D(), Vector.D());
} else {
fcvtzs(SubEmitSize, Dst.Q(), Vector.Q());
@@ -265,7 +269,9 @@ DEF_OP(Vector_FToS) {
const auto Op = IROp->C<IR::IROp_Vector_FToS>();
const auto OpSize = IROp->Size;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Is256Bit = OpSize == IR::OpSize::i256Bit;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto SubEmitSize = ConvertSubRegSize248(IROp);
const auto Dst = GetVReg(Node);
@@ -278,7 +284,7 @@ DEF_OP(Vector_FToS) {
} else {
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
if (OpSize == 8) {
if (OpSize == IR::OpSize::i64Bit) {
frinti(SubEmitSize, Dst.D(), Vector.D());
fcvtzs(SubEmitSize, Dst.D(), Dst.D());
} else {
@@ -294,8 +300,10 @@ DEF_OP(Vector_FToF) {
const auto ElementSize = Op->Header.ElementSize;
const auto SubEmitSize = ConvertSubRegSize248(IROp);
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Conv = (ElementSize << 8) | Op->SrcElementSize;
const auto Is256Bit = OpSize == IR::OpSize::i256Bit;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Conv = (IR::OpSizeToSize(ElementSize) << 8) | IR::OpSizeToSize(Op->SrcElementSize);
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
@@ -395,7 +403,8 @@ DEF_OP(Vector_FToI) {
const auto ElementSize = Op->Header.ElementSize;
const auto SubEmitSize = ConvertSubRegSize248(IROp);
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Is256Bit = OpSize == IR::OpSize::i256Bit;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
@@ -418,15 +427,15 @@ DEF_OP(Vector_FToI) {
// frinti having AdvSIMD, AdvSIMD scalar, and an SVE version),
// we can't just use a lambda without some seriously ugly casting.
// This is fairly self-contained otherwise.
#define ROUNDING_FN(name) \
if (ElementSize == 2) { \
name(Dst.H(), Vector.H()); \
} else if (ElementSize == 4) { \
name(Dst.S(), Vector.S()); \
} else if (ElementSize == 8) { \
name(Dst.D(), Vector.D()); \
} else { \
FEX_UNREACHABLE; \
#define ROUNDING_FN(name) \
if (ElementSize == IR::OpSize::i16Bit) { \
name(Dst.H(), Vector.H()); \
} else if (ElementSize == IR::OpSize::i32Bit) { \
name(Dst.S(), Vector.S()); \
} else if (ElementSize == IR::OpSize::i64Bit) { \
name(Dst.D(), Vector.D()); \
} else { \
FEX_UNREACHABLE; \
}
switch (Op->Round) {
@@ -455,7 +464,8 @@ DEF_OP(Vector_F64ToI32) {
const auto OpSize = IROp->Size;
const auto Round = Op->Round;
const auto Is256Bit = OpSize == Core::CPUState::XMM_AVX_REG_SIZE;
const auto Is256Bit = OpSize == IR::OpSize::i256Bit;
LOGMAN_THROW_A_FMT(!Is256Bit || (Is256Bit && HostSupportsSVE256), "Need SVE256 support in order to use {} with 256-bit operation", __func__);
const auto Dst = GetVReg(Node);
const auto Vector = GetVReg(Op->Vector.ID());
@@ -5,7 +5,7 @@ tags: backend|arm64
$end_info$
*/
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/JIT/JITClass.h"
namespace FEXCore::CPU {
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const* IROp, IR::NodeID Node)
@@ -24,7 +24,7 @@ DEF_OP(VAESEnc) {
const auto State = GetVReg(Op->State.ID());
const auto ZeroReg = GetVReg(Op->ZeroReg.ID());
LOGMAN_THROW_AA_FMT(OpSize == Core::CPUState::XMM_SSE_REG_SIZE, "Currently only supports 128-bit operations.");
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i128Bit, "Currently only supports 128-bit operations.");
if (Dst == State && Dst != Key) {
// Optimal case in which Dst already contains the starting state.
@@ -49,7 +49,7 @@ DEF_OP(VAESEncLast) {
const auto State = GetVReg(Op->State.ID());
const auto ZeroReg = GetVReg(Op->ZeroReg.ID());
LOGMAN_THROW_AA_FMT(OpSize == Core::CPUState::XMM_SSE_REG_SIZE, "Currently only supports 128-bit operations.");
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i128Bit, "Currently only supports 128-bit operations.");
if (Dst == State && Dst != Key) {
// Optimal case in which Dst already contains the starting state.
@@ -72,7 +72,7 @@ DEF_OP(VAESDec) {
const auto State = GetVReg(Op->State.ID());
const auto ZeroReg = GetVReg(Op->ZeroReg.ID());
LOGMAN_THROW_AA_FMT(OpSize == Core::CPUState::XMM_SSE_REG_SIZE, "Currently only supports 128-bit operations.");
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i128Bit, "Currently only supports 128-bit operations.");
if (Dst == State && Dst != Key) {
// Optimal case in which Dst already contains the starting state.
@@ -97,7 +97,7 @@ DEF_OP(VAESDecLast) {
const auto State = GetVReg(Op->State.ID());
const auto ZeroReg = GetVReg(Op->ZeroReg.ID());
LOGMAN_THROW_AA_FMT(OpSize == Core::CPUState::XMM_SSE_REG_SIZE, "Currently only supports 128-bit operations.");
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i128Bit, "Currently only supports 128-bit operations.");
if (Dst == State && Dst != Key) {
// Optimal case in which Dst already contains the starting state.
@@ -152,10 +152,10 @@ DEF_OP(CRC32) {
const auto Src2 = GetReg(Op->Src2.ID());
switch (Op->SrcSize) {
case 1: crc32cb(Dst.W(), Src1.W(), Src2.W()); break;
case 2: crc32ch(Dst.W(), Src1.W(), Src2.W()); break;
case 4: crc32cw(Dst.W(), Src1.W(), Src2.W()); break;
case 8: crc32cx(Dst.X(), Src1.X(), Src2.X()); break;
case IR::OpSize::i8Bit: crc32cb(Dst.W(), Src1.W(), Src2.W()); break;
case IR::OpSize::i16Bit: crc32ch(Dst.W(), Src1.W(), Src2.W()); break;
case IR::OpSize::i32Bit: crc32cw(Dst.W(), Src1.W(), Src2.W()); break;
case IR::OpSize::i64Bit: crc32cx(Dst.X(), Src1.X(), Src2.X()); break;
default: LOGMAN_MSG_A_FMT("Unknown CRC32 size: {}", Op->SrcSize);
}
}
@@ -193,7 +193,7 @@ DEF_OP(PCLMUL) {
const auto Src1 = GetVReg(Op->Src1.ID());
const auto Src2 = GetVReg(Op->Src2.ID());
LOGMAN_THROW_AA_FMT(OpSize == Core::CPUState::XMM_SSE_REG_SIZE, "Currently only supports 128-bit operations.");
LOGMAN_THROW_AA_FMT(OpSize == IR::OpSize::i128Bit, "Currently only supports 128-bit operations.");
switch (Op->Selector) {
case 0b00000000: pmull(ARMEmitter::SubRegSize::i128Bit, Dst.D(), Src1.D(), Src2.D()); break;
@@ -16,7 +16,7 @@ $end_info$
#include "Interface/Core/LookupCache.h"
#include "Interface/Core/Dispatcher/Dispatcher.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/JIT/JITClass.h"
#include "Interface/IR/Passes/RegisterAllocationPass.h"
@@ -626,8 +626,8 @@ bool Arm64JITCore::IsInlineEntrypointOffset(const IR::OrderedNodeWrapper& WNode,
auto Op = OpHeader->C<IR::IROp_InlineEntrypointOffset>();
if (Value) {
uint64_t Mask = ~0ULL;
uint8_t OpSize = OpHeader->Size;
if (OpSize == 4) {
const auto Size = OpHeader->Size;
if (Size == IR::OpSize::i32Bit) {
Mask = 0xFFFF'FFFFULL;
}
*Value = (Entry + Op->Offset) & Mask;
@@ -654,12 +654,6 @@ bool Arm64JITCore::IsGPR(IR::NodeID Node) const {
return Class == IR::GPRClass || Class == IR::GPRFixedClass;
}
bool Arm64JITCore::IsGPRPair(IR::NodeID Node) const {
auto Class = GetRegClass(Node);
return Class == IR::GPRPairClass;
}
CPUBackend::CompiledCode Arm64JITCore::CompileCode(uint64_t Entry, const FEXCore::IR::IRListView* IR, FEXCore::Core::DebugData* DebugData,
const FEXCore::IR::RegisterAllocationData* RAData) {
FEXCORE_PROFILE_SCOPED("Arm64::CompileCode");
@@ -724,6 +718,16 @@ CPUBackend::CompiledCode Arm64JITCore::CompileCode(uint64_t Entry, const FEXCore
offsetof(FEXCore::Core::InternalThreadState, InterruptFaultPage) - offsetof(FEXCore::Core::InternalThreadState, BaseFrameState));
}
#ifdef _M_ARM_64EC
static constexpr uint16_t SuspendMagic {0xCAFE};
ldr(TMP2.W(), STATE_PTR(CpuStateFrame, SuspendDoorbell));
ARMEmitter::SingleUseForwardLabel l_NoSuspend;
cbz(ARMEmitter::Size::i32Bit, TMP2, &l_NoSuspend);
brk(SuspendMagic);
Bind(&l_NoSuspend);
#endif
SpillSlots = RAData->SpillSlots();
if (SpillSlots) {
@@ -14,9 +14,6 @@ $end_info$
#include "Interface/IR/IntrusiveIRList.h"
#include "Interface/IR/RegisterAllocationData.h"
#include <aarch64/assembler-aarch64.h>
#include <aarch64/disasm-aarch64.h>
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/IR/IR.h>
#include <FEXCore/fextl/map.h>
@@ -40,25 +37,10 @@ public:
explicit Arm64JITCore(FEXCore::Context::ContextImpl* ctx, FEXCore::Core::InternalThreadState* Thread);
~Arm64JITCore() override;
[[nodiscard]]
fextl::string GetName() override {
return "JIT";
}
[[nodiscard]]
CPUBackend::CompiledCode CompileCode(uint64_t Entry, const FEXCore::IR::IRListView* IR, FEXCore::Core::DebugData* DebugData,
const FEXCore::IR::RegisterAllocationData* RAData) override;
[[nodiscard]]
void* MapRegion(void* HostPtr, uint64_t, uint64_t) override {
return HostPtr;
}
[[nodiscard]]
bool NeedsOpDispatch() override {
return true;
}
void ClearCache() override;
void ClearRelocations() override {
@@ -67,8 +49,6 @@ public:
private:
FEX_CONFIG_OPT(ParanoidTSO, PARANOIDTSO);
FEX_CONFIG_OPT(VectorTSOEnabled, VECTORTSOENABLED);
FEX_CONFIG_OPT(MemcpySetTSOEnabled, MEMCPYSETTSOENABLED);
const bool HostSupportsSVE128 {};
const bool HostSupportsSVE256 {};
@@ -114,15 +94,6 @@ private:
FEX_UNREACHABLE;
}
[[nodiscard]]
std::pair<ARMEmitter::Register, ARMEmitter::Register> GetRegPair(IR::NodeID Node) const {
const auto Reg = GetPhys(Node);
LOGMAN_THROW_AA_FMT(Reg.Class == IR::GPRPairClass.Val, "Unexpected Class: {}", Reg.Class);
return std::make_pair(GeneralRegisters[Reg.Reg], GeneralRegisters[Reg.Reg + 1]);
}
[[nodiscard]]
FEXCore::IR::RegisterClassType GetRegClass(IR::NodeID Node) const;
@@ -158,23 +129,25 @@ private:
[[nodiscard]]
ARMEmitter::Size ConvertSize(const IR::IROp_Header* Op) {
return Op->Size == 8 ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
return Op->Size == IR::OpSize::i64Bit ? ARMEmitter::Size::i64Bit : ARMEmitter::Size::i32Bit;
}
[[nodiscard]]
ARMEmitter::Size ConvertSize48(const IR::IROp_Header* Op) {
LOGMAN_THROW_AA_FMT(Op->Size == 4 || Op->Size == 8, "Invalid size");
LOGMAN_THROW_AA_FMT(Op->Size == IR::OpSize::i32Bit || Op->Size == IR::OpSize::i64Bit, "Invalid size");
return ConvertSize(Op);
}
[[nodiscard]]
ARMEmitter::SubRegSize ConvertSubRegSize16(uint8_t ElementSize) {
LOGMAN_THROW_AA_FMT(ElementSize == 1 || ElementSize == 2 || ElementSize == 4 || ElementSize == 8 || ElementSize == 16, "Invalid size");
return ElementSize == 1 ? ARMEmitter::SubRegSize::i8Bit :
ElementSize == 2 ? ARMEmitter::SubRegSize::i16Bit :
ElementSize == 4 ? ARMEmitter::SubRegSize::i32Bit :
ElementSize == 8 ? ARMEmitter::SubRegSize::i64Bit :
ARMEmitter::SubRegSize::i128Bit;
ARMEmitter::SubRegSize ConvertSubRegSize16(IR::OpSize ElementSize) {
LOGMAN_THROW_AA_FMT(ElementSize == IR::OpSize::i8Bit || ElementSize == IR::OpSize::i16Bit || ElementSize == IR::OpSize::i32Bit ||
ElementSize == IR::OpSize::i64Bit || ElementSize == IR::OpSize::i128Bit,
"Invalid size");
return ElementSize == IR::OpSize::i8Bit ? ARMEmitter::SubRegSize::i8Bit :
ElementSize == IR::OpSize::i16Bit ? ARMEmitter::SubRegSize::i16Bit :
ElementSize == IR::OpSize::i32Bit ? ARMEmitter::SubRegSize::i32Bit :
ElementSize == IR::OpSize::i64Bit ? ARMEmitter::SubRegSize::i64Bit :
ARMEmitter::SubRegSize::i128Bit;
}
[[nodiscard]]
@@ -183,8 +156,8 @@ private:
}
[[nodiscard]]
ARMEmitter::SubRegSize ConvertSubRegSize8(uint8_t ElementSize) {
LOGMAN_THROW_AA_FMT(ElementSize != 16, "Invalid size");
ARMEmitter::SubRegSize ConvertSubRegSize8(IR::OpSize ElementSize) {
LOGMAN_THROW_AA_FMT(ElementSize != IR::OpSize::i128Bit, "Invalid size");
return ConvertSubRegSize16(ElementSize);
}
@@ -195,13 +168,13 @@ private:
[[nodiscard]]
ARMEmitter::SubRegSize ConvertSubRegSize4(const IR::IROp_Header* Op) {
LOGMAN_THROW_AA_FMT(Op->ElementSize != 8, "Invalid size");
LOGMAN_THROW_AA_FMT(Op->ElementSize != IR::OpSize::i64Bit, "Invalid size");
return ConvertSubRegSize8(Op);
}
[[nodiscard]]
ARMEmitter::SubRegSize ConvertSubRegSize248(const IR::IROp_Header* Op) {
LOGMAN_THROW_AA_FMT(Op->ElementSize != 1, "Invalid size");
LOGMAN_THROW_AA_FMT(Op->ElementSize != IR::OpSize::i8Bit, "Invalid size");
return ConvertSubRegSize8(Op);
}
@@ -212,13 +185,13 @@ private:
[[nodiscard]]
ARMEmitter::VectorRegSizePair ConvertSubRegSizePair8(const IR::IROp_Header* Op) {
LOGMAN_THROW_AA_FMT(Op->ElementSize != 16, "Invalid size");
LOGMAN_THROW_AA_FMT(Op->ElementSize != IR::OpSize::i128Bit, "Invalid size");
return ConvertSubRegSizePair16(Op);
}
[[nodiscard]]
ARMEmitter::VectorRegSizePair ConvertSubRegSizePair248(const IR::IROp_Header* Op) {
LOGMAN_THROW_AA_FMT(Op->ElementSize != 1, "Invalid size");
LOGMAN_THROW_AA_FMT(Op->ElementSize != IR::OpSize::i8Bit, "Invalid size");
return ConvertSubRegSizePair8(Op);
}
@@ -253,11 +226,9 @@ private:
bool IsFPR(IR::NodeID Node) const;
[[nodiscard]]
bool IsGPR(IR::NodeID Node) const;
[[nodiscard]]
bool IsGPRPair(IR::NodeID Node) const;
[[nodiscard]]
ARMEmitter::ExtendedMemOperand GenerateMemOperand(uint8_t AccessSize, ARMEmitter::Register Base, IR::OrderedNodeWrapper Offset,
ARMEmitter::ExtendedMemOperand GenerateMemOperand(IR::OpSize AccessSize, ARMEmitter::Register Base, IR::OrderedNodeWrapper Offset,
IR::MemOffsetType OffsetType, uint8_t OffsetScale);
// NOTE: Will use TMP1 as a way to encode immediates that happen to fall outside
@@ -266,7 +237,7 @@ private:
// TMP1 is safe to use again once this memory operand is used with its
// equivalent loads or stores that this was called for.
[[nodiscard]]
ARMEmitter::SVEMemOperand GenerateSVEMemOperand(uint8_t AccessSize, ARMEmitter::Register Base, IR::OrderedNodeWrapper Offset,
ARMEmitter::SVEMemOperand GenerateSVEMemOperand(IR::OpSize AccessSize, ARMEmitter::Register Base, IR::OrderedNodeWrapper Offset,
IR::MemOffsetType OffsetType, uint8_t OffsetScale);
[[nodiscard]]
@@ -347,18 +318,20 @@ private:
using ScalarFMAOpCaller =
std::function<void(ARMEmitter::VRegister Dst, ARMEmitter::VRegister Src1, ARMEmitter::VRegister Src2, ARMEmitter::VRegister Src3)>;
void VFScalarFMAOperation(uint8_t OpSize, uint8_t ElementSize, ScalarFMAOpCaller ScalarEmit, ARMEmitter::VRegister Dst,
ARMEmitter::VRegister Vector1, ARMEmitter::VRegister Vector2, ARMEmitter::VRegister Addend);
void VFScalarFMAOperation(IR::OpSize OpSize, IR::OpSize ElementSize, ScalarFMAOpCaller ScalarEmit, ARMEmitter::VRegister Dst,
ARMEmitter::VRegister Upper, ARMEmitter::VRegister Vector1, ARMEmitter::VRegister Vector2,
ARMEmitter::VRegister Addend);
using ScalarBinaryOpCaller = std::function<void(ARMEmitter::VRegister Dst, ARMEmitter::VRegister Src1, ARMEmitter::VRegister Src2)>;
void VFScalarOperation(uint8_t OpSize, uint8_t ElementSize, bool ZeroUpperBits, ScalarBinaryOpCaller ScalarEmit,
void VFScalarOperation(IR::OpSize OpSize, IR::OpSize ElementSize, bool ZeroUpperBits, ScalarBinaryOpCaller ScalarEmit,
ARMEmitter::VRegister Dst, ARMEmitter::VRegister Vector1, ARMEmitter::VRegister Vector2);
using ScalarUnaryOpCaller = std::function<void(ARMEmitter::VRegister Dst, std::variant<ARMEmitter::VRegister, ARMEmitter::Register> SrcVar)>;
void VFScalarUnaryOperation(uint8_t OpSize, uint8_t ElementSize, bool ZeroUpperBits, ScalarUnaryOpCaller ScalarEmit, ARMEmitter::VRegister Dst,
ARMEmitter::VRegister Vector1, std::variant<ARMEmitter::VRegister, ARMEmitter::Register> Vector2);
void VFScalarUnaryOperation(IR::OpSize OpSize, IR::OpSize ElementSize, bool ZeroUpperBits, ScalarUnaryOpCaller ScalarEmit,
ARMEmitter::VRegister Dst, ARMEmitter::VRegister Vector1,
std::variant<ARMEmitter::VRegister, ARMEmitter::Register> Vector2);
void Emulate128BitGather(size_t Size, size_t ElementSize, ARMEmitter::VRegister Dst, ARMEmitter::VRegister IncomingDst,
void Emulate128BitGather(IR::OpSize Size, IR::OpSize ElementSize, ARMEmitter::VRegister Dst, ARMEmitter::VRegister IncomingDst,
std::optional<ARMEmitter::Register> BaseAddr, ARMEmitter::VRegister VectorIndexLow,
std::optional<ARMEmitter::VRegister> VectorIndexHigh, ARMEmitter::VRegister MaskReg, size_t VectorIndexSize,
std::optional<ARMEmitter::VRegister> VectorIndexHigh, ARMEmitter::VRegister MaskReg, IR::OpSize VectorIndexSize,
size_t DataElementOffsetStart, size_t IndexElementOffsetStart, uint8_t OffsetScale);
// Runtime selection;
// Load and store TSO memory style
@@ -382,4 +355,7 @@ private:
#undef DEF_OP
};
[[nodiscard]]
fextl::unique_ptr<CPUBackend> CreateArm64JITCore(FEXCore::Context::ContextImpl* ctx, FEXCore::Core::InternalThreadState* Thread);
} // namespace FEXCore::CPU
@@ -1,21 +0,0 @@
// SPDX-License-Identifier: MIT
#pragma once
#include "Interface/Core/CPUBackend.h"
#include <FEXCore/fextl/memory.h>
namespace FEXCore::Context {
class ContextImpl;
}
namespace FEXCore::Core {
struct InternalThreadState;
}
namespace FEXCore::CPU {
class CPUBackend;
[[nodiscard]]
fextl::unique_ptr<CPUBackend> CreateArm64JITCore(FEXCore::Context::ContextImpl* ctx, FEXCore::Core::InternalThreadState* Thread);
} // namespace FEXCore::CPU
@@ -10,7 +10,7 @@ $end_info$
#endif
#include "Interface/Context/Context.h"
#include "Interface/Core/JIT/Arm64/JITClass.h"
#include "Interface/Core/JIT/JITClass.h"
#include "FEXCore/Debug/InternalThreadState.h"
#include <FEXCore/Core/SignalDelegator.h>
@@ -18,6 +18,10 @@ $end_info$
namespace FEXCore::CPU {
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const* IROp, IR::NodeID Node)
DEF_OP(AllocateGPR) {}
DEF_OP(AllocateGPRAfter) {}
DEF_OP(AllocateFPR) {}
DEF_OP(GuestOpcode) {
auto Op = IROp->C<IR::IROp_GuestOpcode>();
// metadata
@@ -188,8 +192,17 @@ DEF_OP(Print) {
PopDynamicRegsAndLR();
}
#ifndef _WIN32
DEF_OP(ProcessorID) {
if (CTX->HostFeatures.SupportsCPUIndexInTPIDRRO) {
mrs(GetReg(Node), ARMEmitter::SystemRegister::TPIDRRO_EL0);
return;
}
#ifdef _WIN32
else {
// If on Windows and TPIDRRO isn't supported (like in wine), then this is a programming error.
ERROR_AND_DIE_FMT("Unsupported");
}
#else
// We always need to spill x8 since we can't know if it is live at this SSA location
uint32_t SpillMask = 1U << 8;
@@ -244,32 +257,17 @@ DEF_OP(ProcessorID) {
// CPU is in w0
// Node is in w1
orr(ARMEmitter::Size::i64Bit, GetReg(Node), ARMEmitter::Reg::r0, ARMEmitter::Reg::r1, ARMEmitter::ShiftType::LSL, 12);
}
#else
DEF_OP(ProcessorID) {
ERROR_AND_DIE_FMT("Unsupported");
}
#endif
}
DEF_OP(RDRAND) {
auto Op = IROp->C<IR::IROp_RDRAND>();
// Results are in x0, x1
// Results want to be in a i64v2 vector
auto Dst = GetRegPair(Node);
if (Op->GetReseeded) {
mrs(Dst.first, ARMEmitter::SystemRegister::RNDRRS);
} else {
mrs(Dst.first, ARMEmitter::SystemRegister::RNDR);
}
// If the rng number is valid then NZCV is 0b0000, otherwise NZCV is 0b0100
cset(ARMEmitter::Size::i64Bit, Dst.second, ARMEmitter::Condition::CC_NE);
mrs(GetReg(Node), Op->GetReseeded ? ARMEmitter::SystemRegister::RNDRRS : ARMEmitter::SystemRegister::RNDR);
}
DEF_OP(Yield) {
yield();
wfe();
}
#undef DEF_OP
@@ -0,0 +1,43 @@
// SPDX-License-Identifier: MIT
/*
$info$
tags: backend|arm64
$end_info$
*/
#include "Interface/Core/JIT/JITClass.h"
namespace FEXCore::CPU {
#define DEF_OP(x) void Arm64JITCore::Op_##x(IR::IROp_Header const* IROp, IR::NodeID Node)
DEF_OP(Copy) {
auto Op = IROp->C<IR::IROp_Copy>();
mov(ARMEmitter::Size::i64Bit, GetReg(Node), GetReg(Op->Source.ID()));
}
DEF_OP(RMWHandle) {
auto Op = IROp->C<IR::IROp_RMWHandle>();
auto Dest = GetReg(Node);
auto Src = GetReg(Op->Value.ID());
if (Dest != Src) {
mov(ARMEmitter::Size::i64Bit, Dest, Src);
}
}
DEF_OP(Swap1) {
auto Op = IROp->C<IR::IROp_Swap1>();
auto A = GetReg(Op->A.ID()), B = GetReg(Op->B.ID());
LOGMAN_THROW_A_FMT(B == GetReg(Node), "Invariant");
mov(ARMEmitter::Size::i64Bit, TMP1, A);
mov(ARMEmitter::Size::i64Bit, A, B);
mov(ARMEmitter::Size::i64Bit, B, TMP1);
}
DEF_OP(Swap2) {
// Implemented above
}
#undef DEF_OP
} // namespace FEXCore::CPU
@@ -8,6 +8,7 @@ $end_info$
#include <FEXCore/Utils/Allocator.h>
#include <FEXCore/Utils/LogManager.h>
#include <FEXCore/HLE/SyscallHandler.h>
#include "Interface/Context/Context.h"
#include "Interface/Core/LookupCache.h"
@@ -34,7 +35,8 @@ LookupCache::LookupCache(FEXCore::Context::ContextImpl* CTX)
// Allocate a region of memory that we can use to back our block pointers
// We need one pointer per page of virtual memory
// At 64GB of virtual memory this will allocate 128MB of virtual memory space
PagePointer = reinterpret_cast<uintptr_t>(FEXCore::Allocator::VirtualAlloc(TotalCacheSize));
PagePointer = reinterpret_cast<uintptr_t>(FEXCore::Allocator::VirtualAlloc(TotalCacheSize, false, false));
CTX->SyscallHandler->MarkOvercommitRange(PagePointer, TotalCacheSize);
// Allocate our memory backing our pages
// We need 32KB per guest page (One pointer per byte)
@@ -52,8 +54,8 @@ LookupCache::LookupCache(FEXCore::Context::ContextImpl* CTX)
}
LookupCache::~LookupCache() {
const size_t TotalCacheSize = ctx->Config.VirtualMemSize / 4096 * 8 + CODE_SIZE + L1_SIZE;
FEXCore::Allocator::VirtualFree(reinterpret_cast<void*>(PagePointer), TotalCacheSize);
ctx->SyscallHandler->UnmarkOvercommitRange(PagePointer, TotalCacheSize);
// No need to free BlockLinks map.
// These will get freed when their memory allocators are deallocated.
@@ -63,7 +65,7 @@ void LookupCache::ClearL2Cache() {
std::lock_guard<std::recursive_mutex> lk(WriteLock);
// Clear out the page memory
// PagePointer and PageMemory are sequential with each other. Clear both at once.
FEXCore::Allocator::VirtualDontNeed(reinterpret_cast<void*>(PagePointer), ctx->Config.VirtualMemSize / 4096 * 8 + CODE_SIZE);
FEXCore::Allocator::VirtualDontNeed(reinterpret_cast<void*>(PagePointer), ctx->Config.VirtualMemSize / 4096 * 8 + CODE_SIZE, false);
AllocateOffset = 0;
}
@@ -71,7 +73,7 @@ void LookupCache::ClearCache() {
std::lock_guard<std::recursive_mutex> lk(WriteLock);
// Clear L1 and L2 by clearing the full cache.
FEXCore::Allocator::VirtualDontNeed(reinterpret_cast<void*>(PagePointer), TotalCacheSize);
FEXCore::Allocator::VirtualDontNeed(reinterpret_cast<void*>(PagePointer), TotalCacheSize, false);
// Allocate a new pointer from the BlockLinks pma again.
BlockLinks = BlockLinks_pma->new_object<BlockLinksMapType>();
// All code is gone, clear the block list
@@ -1,5 +1,4 @@
// SPDX-License-Identifier: MIT
#include "Interface/Context/Context.h"
#include "Interface/Core/ObjectCache/ObjectCacheService.h"
#include <FEXCore/Config/Config.h>
@@ -1,8 +1,6 @@
// SPDX-License-Identifier: MIT
#include "Interface/Context/Context.h"
#include "Interface/Core/ObjectCache/ObjectCacheService.h"
#include <FEXCore/fextl/memory.h>
#include <FEXCore/fextl/string.h>
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,107 @@
// SPDX-License-Identifier: MIT
#pragma once
#include "Interface/Core/OpcodeDispatcher.h"
namespace FEXCore::IR {
constexpr inline std::tuple<uint8_t, uint8_t, X86Tables::OpDispatchPtr> OpDispatch_BaseOpTable[] = {
// Instructions
{0x00, 6, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ALUOp, FEXCore::IR::IROps::OP_ADD, FEXCore::IR::IROps::OP_ATOMICFETCHADD, 0>},
{0x08, 6, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ALUOp, FEXCore::IR::IROps::OP_OR, FEXCore::IR::IROps::OP_ATOMICFETCHOR, 0>},
{0x10, 6, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ADCOp, 0>},
{0x18, 6, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SBBOp, 0>},
{0x20, 6, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ALUOp, FEXCore::IR::IROps::OP_ANDWITHFLAGS, FEXCore::IR::IROps::OP_ATOMICFETCHAND, 0>},
{0x28, 6, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ALUOp, FEXCore::IR::IROps::OP_SUB, FEXCore::IR::IROps::OP_ATOMICFETCHSUB, 0>},
{0x30, 6, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ALUOp, FEXCore::IR::IROps::OP_XOR, FEXCore::IR::IROps::OP_ATOMICFETCHXOR, 0>},
{0x38, 6, &OpDispatchBuilder::Bind<&OpDispatchBuilder::CMPOp, 0>},
{0x50, 8, &OpDispatchBuilder::PUSHREGOp},
{0x58, 8, &OpDispatchBuilder::POPOp},
{0x68, 1, &OpDispatchBuilder::PUSHOp},
{0x69, 1, &OpDispatchBuilder::IMUL2SrcOp},
{0x6A, 1, &OpDispatchBuilder::PUSHOp},
{0x6B, 1, &OpDispatchBuilder::IMUL2SrcOp},
{0x6C, 4, &OpDispatchBuilder::PermissionRestrictedOp},
{0x70, 16, &OpDispatchBuilder::CondJUMPOp},
{0x84, 2, &OpDispatchBuilder::Bind<&OpDispatchBuilder::TESTOp, 0>},
{0x86, 2, &OpDispatchBuilder::XCHGOp},
{0x88, 4, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVGPROp, 0>},
{0x8C, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVSegOp, false>},
{0x8D, 1, &OpDispatchBuilder::LEAOp},
{0x8E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVSegOp, true>},
{0x8F, 1, &OpDispatchBuilder::POPOp},
{0x90, 8, &OpDispatchBuilder::XCHGOp},
{0x98, 1, &OpDispatchBuilder::CDQOp},
{0x99, 1, &OpDispatchBuilder::CQOOp},
{0x9B, 1, &OpDispatchBuilder::NOPOp},
{0x9C, 1, &OpDispatchBuilder::PUSHFOp},
{0x9D, 1, &OpDispatchBuilder::POPFOp},
{0x9E, 1, &OpDispatchBuilder::SAHFOp},
{0x9F, 1, &OpDispatchBuilder::LAHFOp},
{0xA4, 2, &OpDispatchBuilder::MOVSOp},
{0xA6, 2, &OpDispatchBuilder::CMPSOp},
{0xA8, 2, &OpDispatchBuilder::Bind<&OpDispatchBuilder::TESTOp, 0>},
{0xAA, 2, &OpDispatchBuilder::STOSOp},
{0xAC, 2, &OpDispatchBuilder::LODSOp},
{0xAE, 2, &OpDispatchBuilder::SCASOp},
{0xB0, 16, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVGPROp, 0>},
{0xC2, 2, &OpDispatchBuilder::RETOp},
{0xC8, 1, &OpDispatchBuilder::EnterOp},
{0xC9, 1, &OpDispatchBuilder::LEAVEOp},
{0xCC, 2, &OpDispatchBuilder::INTOp},
{0xCF, 1, &OpDispatchBuilder::IRETOp},
{0xD7, 2, &OpDispatchBuilder::XLATOp},
{0xE0, 3, &OpDispatchBuilder::LoopOp},
{0xE3, 1, &OpDispatchBuilder::CondJUMPRCXOp},
{0xE4, 4, &OpDispatchBuilder::PermissionRestrictedOp},
{0xE8, 1, &OpDispatchBuilder::CALLOp},
{0xE9, 1, &OpDispatchBuilder::JUMPOp},
{0xEB, 1, &OpDispatchBuilder::JUMPOp},
{0xEC, 4, &OpDispatchBuilder::PermissionRestrictedOp},
{0xF1, 1, &OpDispatchBuilder::INTOp},
{0xF4, 1, &OpDispatchBuilder::INTOp},
{0xF5, 1, &OpDispatchBuilder::FLAGControlOp},
{0xF8, 2, &OpDispatchBuilder::FLAGControlOp},
{0xFA, 2, &OpDispatchBuilder::PermissionRestrictedOp},
{0xFC, 2, &OpDispatchBuilder::FLAGControlOp},
};
constexpr inline std::tuple<uint8_t, uint8_t, X86Tables::OpDispatchPtr> OpDispatch_BaseOpTable_64[] = {
{0x63, 1, &OpDispatchBuilder::MOVSXDOp},
{0xA0, 4, &OpDispatchBuilder::MOVOffsetOp},
};
constexpr inline std::tuple<uint8_t, uint8_t, X86Tables::OpDispatchPtr> OpDispatch_BaseOpTable_32[] = {
{0x06, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUSHSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_ES_PREFIX>},
{0x07, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::POPSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_ES_PREFIX>},
{0x0E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUSHSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_CS_PREFIX>},
{0x16, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUSHSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_SS_PREFIX>},
{0x17, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::POPSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_SS_PREFIX>},
{0x1E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUSHSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_DS_PREFIX>},
{0x1F, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::POPSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_DS_PREFIX>},
{0x27, 1, &OpDispatchBuilder::DAAOp},
{0x2F, 1, &OpDispatchBuilder::DASOp},
{0x37, 1, &OpDispatchBuilder::AAAOp},
{0x3F, 1, &OpDispatchBuilder::AASOp},
{0x40, 8, &OpDispatchBuilder::INCOp},
{0x48, 8, &OpDispatchBuilder::DECOp},
{0x60, 1, &OpDispatchBuilder::PUSHAOp},
{0x61, 1, &OpDispatchBuilder::POPAOp},
{0xA0, 4, &OpDispatchBuilder::MOVOffsetOp},
{0xCE, 1, &OpDispatchBuilder::INTOp},
{0xD4, 1, &OpDispatchBuilder::AAMOp},
{0xD5, 1, &OpDispatchBuilder::AADOp},
{0xD6, 1, &OpDispatchBuilder::SALCOp},
};
} // namespace FEXCore::IR
@@ -43,19 +43,19 @@ void OpDispatchBuilder::SHA1NEXTEOp(OpcodeArgs) {
auto Tmp = _VAdd(OpSize::i128Bit, OpSize::i32Bit, Src, RotatedNode);
auto Result = _VInsElement(OpSize::i128Bit, OpSize::i32Bit, 3, 3, Src, Tmp);
StoreResult(FPRClass, Op, Result, -1);
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::SHA1MSG1Op(OpcodeArgs) {
Ref Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags);
Ref Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Ref NewVec = _VExtr(16, 8, Dest, Src, 1);
Ref NewVec = _VExtr(OpSize::i128Bit, OpSize::i64Bit, Dest, Src, 1);
// [W0, W1, W2, W3] ^ [W2, W3, W4, W5]
Ref Result = _VXor(16, 1, Dest, NewVec);
Ref Result = _VXor(OpSize::i128Bit, OpSize::i8Bit, Dest, NewVec);
StoreResult(FPRClass, Op, Result, -1);
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::SHA1MSG2Op(OpcodeArgs) {
@@ -86,7 +86,7 @@ void OpDispatchBuilder::SHA1MSG2Op(OpcodeArgs) {
auto Result = _VInsElement(OpSize::i128Bit, OpSize::i32Bit, 0, 0, RotatedXor1, RotatedXorLower);
StoreResult(FPRClass, Op, Result, -1);
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::SHA1RNDS4Op(OpcodeArgs) {
@@ -121,25 +121,26 @@ void OpDispatchBuilder::SHA1RNDS4Op(OpcodeArgs) {
const uint64_t Imm8 = Op->Src[1].Literal() & 0b11;
const FnType Fn = fn_array[Imm8];
auto K = _Constant(32, k_array[Imm8]);
auto K = _Constant(OpSize::i32Bit, k_array[Imm8]);
Ref Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags);
Ref Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
auto W0E = _VExtractToGPR(16, 4, Src, 3);
auto W0E = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Src, 3);
using RoundResult = std::tuple<Ref, Ref, Ref, Ref, Ref>;
const auto Round0 = [&]() -> RoundResult {
auto A = _VExtractToGPR(16, 4, Dest, 3);
auto B = _VExtractToGPR(16, 4, Dest, 2);
auto C = _VExtractToGPR(16, 4, Dest, 1);
auto D = _VExtractToGPR(16, 4, Dest, 0);
auto A = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 3);
auto B = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 2);
auto C = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 1);
auto D = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 0);
auto A1 =
_Add(OpSize::i32Bit, _Add(OpSize::i32Bit, _Add(OpSize::i32Bit, Fn(*this, B, C, D), _Ror(OpSize::i32Bit, A, _Constant(32, 27))), W0E), K);
_Add(OpSize::i32Bit,
_Add(OpSize::i32Bit, _Add(OpSize::i32Bit, Fn(*this, B, C, D), _Ror(OpSize::i32Bit, A, _Constant(OpSize::i32Bit, 27))), W0E), K);
auto B1 = A;
auto C1 = _Ror(OpSize::i32Bit, B, _Constant(32, 2));
auto C1 = _Ror(OpSize::i32Bit, B, _Constant(OpSize::i32Bit, 2));
auto D1 = C;
auto E1 = D;
@@ -147,13 +148,14 @@ void OpDispatchBuilder::SHA1RNDS4Op(OpcodeArgs) {
};
const auto Round1To3 = [&](Ref A, Ref B, Ref C, Ref D, Ref E, Ref Src, unsigned W_idx) -> RoundResult {
// Kill W and E at the beginning
auto W = _VExtractToGPR(16, 4, Src, W_idx);
auto W = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Src, W_idx);
auto Q = _Add(OpSize::i32Bit, W, E);
auto ANext =
_Add(OpSize::i32Bit, _Add(OpSize::i32Bit, _Add(OpSize::i32Bit, Fn(*this, B, C, D), _Ror(OpSize::i32Bit, A, _Constant(32, 27))), Q), K);
_Add(OpSize::i32Bit,
_Add(OpSize::i32Bit, _Add(OpSize::i32Bit, Fn(*this, B, C, D), _Ror(OpSize::i32Bit, A, _Constant(OpSize::i32Bit, 27))), Q), K);
auto BNext = A;
auto CNext = _Ror(OpSize::i32Bit, B, _Constant(32, 2));
auto CNext = _Ror(OpSize::i32Bit, B, _Constant(OpSize::i32Bit, 2));
auto DNext = C;
auto ENext = D;
@@ -165,12 +167,12 @@ void OpDispatchBuilder::SHA1RNDS4Op(OpcodeArgs) {
auto [A3, B3, C3, D3, E3] = Round1To3(A2, B2, C2, D2, E2, Src, 1);
auto Final = Round1To3(A3, B3, C3, D3, E3, Src, 0);
auto Dest3 = _VInsGPR(16, 4, 3, Dest, std::get<0>(Final));
auto Dest2 = _VInsGPR(16, 4, 2, Dest3, std::get<1>(Final));
auto Dest1 = _VInsGPR(16, 4, 1, Dest2, std::get<2>(Final));
auto Dest0 = _VInsGPR(16, 4, 0, Dest1, std::get<3>(Final));
auto Dest3 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 3, Dest, std::get<0>(Final));
auto Dest2 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 2, Dest3, std::get<1>(Final));
auto Dest1 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 1, Dest2, std::get<2>(Final));
auto Dest0 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 0, Dest1, std::get<3>(Final));
StoreResult(FPRClass, Op, Dest0, -1);
StoreResult(FPRClass, Op, Dest0, OpSize::iInvalid);
}
void OpDispatchBuilder::SHA256MSG1Op(OpcodeArgs) {
@@ -183,52 +185,56 @@ void OpDispatchBuilder::SHA256MSG1Op(OpcodeArgs) {
Result = _VSha256U0(Dest, Src);
} else {
const auto Sigma0 = [this](Ref W) -> Ref {
return _Xor(OpSize::i32Bit, _Xor(OpSize::i32Bit, _Ror(OpSize::i32Bit, W, _Constant(32, 7)), _Ror(OpSize::i32Bit, W, _Constant(32, 18))),
_Lshr(OpSize::i32Bit, W, _Constant(32, 3)));
return _Xor(
OpSize::i32Bit,
_Xor(OpSize::i32Bit, _Ror(OpSize::i32Bit, W, _Constant(OpSize::i32Bit, 7)), _Ror(OpSize::i32Bit, W, _Constant(OpSize::i32Bit, 18))),
_Lshr(OpSize::i32Bit, W, _Constant(OpSize::i32Bit, 3)));
};
auto W4 = _VExtractToGPR(16, 4, Src, 0);
auto W3 = _VExtractToGPR(16, 4, Dest, 3);
auto W2 = _VExtractToGPR(16, 4, Dest, 2);
auto W1 = _VExtractToGPR(16, 4, Dest, 1);
auto W0 = _VExtractToGPR(16, 4, Dest, 0);
auto W4 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Src, 0);
auto W3 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 3);
auto W2 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 2);
auto W1 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 1);
auto W0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 0);
auto Sig3 = _Add(OpSize::i32Bit, W3, Sigma0(W4));
auto Sig2 = _Add(OpSize::i32Bit, W2, Sigma0(W3));
auto Sig1 = _Add(OpSize::i32Bit, W1, Sigma0(W2));
auto Sig0 = _Add(OpSize::i32Bit, W0, Sigma0(W1));
auto D3 = _VInsGPR(16, 4, 3, Dest, Sig3);
auto D2 = _VInsGPR(16, 4, 2, D3, Sig2);
auto D1 = _VInsGPR(16, 4, 1, D2, Sig1);
Result = _VInsGPR(16, 4, 0, D1, Sig0);
auto D3 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 3, Dest, Sig3);
auto D2 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 2, D3, Sig2);
auto D1 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 1, D2, Sig1);
Result = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 0, D1, Sig0);
}
StoreResult(FPRClass, Op, Result, -1);
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::SHA256MSG2Op(OpcodeArgs) {
const auto Sigma1 = [this](Ref W) -> Ref {
return _Xor(OpSize::i32Bit, _Xor(OpSize::i32Bit, _Ror(OpSize::i32Bit, W, _Constant(32, 17)), _Ror(OpSize::i32Bit, W, _Constant(32, 19))),
_Lshr(OpSize::i32Bit, W, _Constant(32, 10)));
return _Xor(
OpSize::i32Bit,
_Xor(OpSize::i32Bit, _Ror(OpSize::i32Bit, W, _Constant(OpSize::i32Bit, 17)), _Ror(OpSize::i32Bit, W, _Constant(OpSize::i32Bit, 19))),
_Lshr(OpSize::i32Bit, W, _Constant(OpSize::i32Bit, 10)));
};
Ref Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags);
Ref Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
auto W14 = _VExtractToGPR(16, 4, Src, 2);
auto W15 = _VExtractToGPR(16, 4, Src, 3);
auto W16 = _Add(OpSize::i32Bit, _VExtractToGPR(16, 4, Dest, 0), Sigma1(W14));
auto W17 = _Add(OpSize::i32Bit, _VExtractToGPR(16, 4, Dest, 1), Sigma1(W15));
auto W18 = _Add(OpSize::i32Bit, _VExtractToGPR(16, 4, Dest, 2), Sigma1(W16));
auto W19 = _Add(OpSize::i32Bit, _VExtractToGPR(16, 4, Dest, 3), Sigma1(W17));
auto W14 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Src, 2);
auto W15 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Src, 3);
auto W16 = _Add(OpSize::i32Bit, _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 0), Sigma1(W14));
auto W17 = _Add(OpSize::i32Bit, _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 1), Sigma1(W15));
auto W18 = _Add(OpSize::i32Bit, _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 2), Sigma1(W16));
auto W19 = _Add(OpSize::i32Bit, _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 3), Sigma1(W17));
auto D3 = _VInsGPR(16, 4, 3, Dest, W19);
auto D2 = _VInsGPR(16, 4, 2, D3, W18);
auto D1 = _VInsGPR(16, 4, 1, D2, W17);
auto D0 = _VInsGPR(16, 4, 0, D1, W16);
auto D3 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 3, Dest, W19);
auto D2 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 2, D3, W18);
auto D1 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 1, D2, W17);
auto D0 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 0, D1, W16);
StoreResult(FPRClass, Op, D0, -1);
StoreResult(FPRClass, Op, D0, OpSize::iInvalid);
}
Ref OpDispatchBuilder::BitwiseAtLeastTwo(Ref A, Ref B, Ref C) {
@@ -246,12 +252,12 @@ void OpDispatchBuilder::SHA256RNDS2Op(OpcodeArgs) {
return _Xor(OpSize::i32Bit, _And(OpSize::i32Bit, E, F), _Andn(OpSize::i32Bit, G, E));
};
const auto Sigma0 = [this](Ref A) -> Ref {
return _XorShift(OpSize::i32Bit, _XorShift(OpSize::i32Bit, _Ror(OpSize::i32Bit, A, _Constant(32, 2)), A, ShiftType::ROR, 13), A,
ShiftType::ROR, 22);
return _XorShift(OpSize::i32Bit, _XorShift(OpSize::i32Bit, _Ror(OpSize::i32Bit, A, _Constant(OpSize::i32Bit, 2)), A, ShiftType::ROR, 13),
A, ShiftType::ROR, 22);
};
const auto Sigma1 = [this](Ref E) -> Ref {
return _XorShift(OpSize::i32Bit, _XorShift(OpSize::i32Bit, _Ror(OpSize::i32Bit, E, _Constant(32, 6)), E, ShiftType::ROR, 11), E,
ShiftType::ROR, 25);
return _XorShift(OpSize::i32Bit, _XorShift(OpSize::i32Bit, _Ror(OpSize::i32Bit, E, _Constant(OpSize::i32Bit, 6)), E, ShiftType::ROR, 11),
E, ShiftType::ROR, 25);
};
Ref Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags);
@@ -259,64 +265,64 @@ void OpDispatchBuilder::SHA256RNDS2Op(OpcodeArgs) {
// Hardcoded to XMM0
auto XMM0 = LoadXMMRegister(0);
auto E0 = _VExtractToGPR(16, 4, Src, 1);
auto F0 = _VExtractToGPR(16, 4, Src, 0);
auto G0 = _VExtractToGPR(16, 4, Dest, 1);
auto E0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Src, 1);
auto F0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Src, 0);
auto G0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 1);
Ref Q0 = _Add(OpSize::i32Bit, Ch(E0, F0, G0), Sigma1(E0));
auto WK0 = _VExtractToGPR(16, 4, XMM0, 0);
auto WK0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, XMM0, 0);
Q0 = _Add(OpSize::i32Bit, Q0, WK0);
auto H0 = _VExtractToGPR(16, 4, Dest, 0);
auto H0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 0);
Q0 = _Add(OpSize::i32Bit, Q0, H0);
auto A0 = _VExtractToGPR(16, 4, Src, 3);
auto B0 = _VExtractToGPR(16, 4, Src, 2);
auto C0 = _VExtractToGPR(16, 4, Dest, 3);
auto A0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Src, 3);
auto B0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Src, 2);
auto C0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 3);
auto A1 = _Add(OpSize::i32Bit, _Add(OpSize::i32Bit, Q0, BitwiseAtLeastTwo(A0, B0, C0)), Sigma0(A0));
auto D0 = _VExtractToGPR(16, 4, Dest, 2);
auto D0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 2);
auto E1 = _Add(OpSize::i32Bit, Q0, D0);
Ref Q1 = _Add(OpSize::i32Bit, Ch(E1, E0, F0), Sigma1(E1));
auto WK1 = _VExtractToGPR(16, 4, XMM0, 1);
auto WK1 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, XMM0, 1);
Q1 = _Add(OpSize::i32Bit, Q1, WK1);
// Rematerialize G0. Costs a move but saves spilling, coming out ahead.
G0 = _VExtractToGPR(16, 4, Dest, 1);
G0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 1);
Q1 = _Add(OpSize::i32Bit, Q1, G0);
auto A2 = _Add(OpSize::i32Bit, _Add(OpSize::i32Bit, Q1, BitwiseAtLeastTwo(A1, A0, B0)), Sigma0(A1));
// Rematerialize C0. As with G0.
C0 = _VExtractToGPR(16, 4, Dest, 3);
C0 = _VExtractToGPR(OpSize::i128Bit, OpSize::i32Bit, Dest, 3);
auto E2 = _Add(OpSize::i32Bit, Q1, C0);
auto Res3 = _VInsGPR(16, 4, 3, Dest, A2);
auto Res2 = _VInsGPR(16, 4, 2, Res3, A1);
auto Res1 = _VInsGPR(16, 4, 1, Res2, E2);
auto Res0 = _VInsGPR(16, 4, 0, Res1, E1);
auto Res3 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 3, Dest, A2);
auto Res2 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 2, Res3, A1);
auto Res1 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 1, Res2, E2);
auto Res0 = _VInsGPR(OpSize::i128Bit, OpSize::i32Bit, 0, Res1, E1);
StoreResult(FPRClass, Op, Res0, -1);
StoreResult(FPRClass, Op, Res0, OpSize::iInvalid);
}
void OpDispatchBuilder::AESImcOp(OpcodeArgs) {
Ref Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Ref Result = _VAESImc(Src);
StoreResult(FPRClass, Op, Result, -1);
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::AESEncOp(OpcodeArgs) {
Ref Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags);
Ref Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Ref Result = _VAESEnc(16, Dest, Src, LoadZeroVector(16));
StoreResult(FPRClass, Op, Result, -1);
Ref Result = _VAESEnc(OpSize::i128Bit, Dest, Src, LoadZeroVector(OpSize::i128Bit));
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::VAESEncOp(OpcodeArgs) {
const auto DstSize = GetDstSize(Op);
[[maybe_unused]] const auto Is128Bit = DstSize == Core::CPUState::XMM_SSE_REG_SIZE;
const auto DstSize = OpSizeFromDst(Op);
[[maybe_unused]] const auto Is128Bit = DstSize == OpSize::i128Bit;
// TODO: Handle 256-bit VAESENC.
LOGMAN_THROW_A_FMT(Is128Bit, "256-bit VAESENC unimplemented");
@@ -325,19 +331,19 @@ void OpDispatchBuilder::VAESEncOp(OpcodeArgs) {
Ref Key = LoadSource(FPRClass, Op, Op->Src[1], Op->Flags);
Ref Result = _VAESEnc(DstSize, State, Key, LoadZeroVector(DstSize));
StoreResult(FPRClass, Op, Result, -1);
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::AESEncLastOp(OpcodeArgs) {
Ref Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags);
Ref Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Ref Result = _VAESEncLast(16, Dest, Src, LoadZeroVector(16));
StoreResult(FPRClass, Op, Result, -1);
Ref Result = _VAESEncLast(OpSize::i128Bit, Dest, Src, LoadZeroVector(OpSize::i128Bit));
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::VAESEncLastOp(OpcodeArgs) {
const auto DstSize = GetDstSize(Op);
[[maybe_unused]] const auto Is128Bit = DstSize == Core::CPUState::XMM_SSE_REG_SIZE;
const auto DstSize = OpSizeFromDst(Op);
[[maybe_unused]] const auto Is128Bit = DstSize == OpSize::i128Bit;
// TODO: Handle 256-bit VAESENCLAST.
LOGMAN_THROW_A_FMT(Is128Bit, "256-bit VAESENCLAST unimplemented");
@@ -346,19 +352,19 @@ void OpDispatchBuilder::VAESEncLastOp(OpcodeArgs) {
Ref Key = LoadSource(FPRClass, Op, Op->Src[1], Op->Flags);
Ref Result = _VAESEncLast(DstSize, State, Key, LoadZeroVector(DstSize));
StoreResult(FPRClass, Op, Result, -1);
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::AESDecOp(OpcodeArgs) {
Ref Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags);
Ref Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Ref Result = _VAESDec(16, Dest, Src, LoadZeroVector(16));
StoreResult(FPRClass, Op, Result, -1);
Ref Result = _VAESDec(OpSize::i128Bit, Dest, Src, LoadZeroVector(OpSize::i128Bit));
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::VAESDecOp(OpcodeArgs) {
const auto DstSize = GetDstSize(Op);
[[maybe_unused]] const auto Is128Bit = DstSize == Core::CPUState::XMM_SSE_REG_SIZE;
const auto DstSize = OpSizeFromDst(Op);
[[maybe_unused]] const auto Is128Bit = DstSize == OpSize::i128Bit;
// TODO: Handle 256-bit VAESDEC.
LOGMAN_THROW_A_FMT(Is128Bit, "256-bit VAESDEC unimplemented");
@@ -367,19 +373,19 @@ void OpDispatchBuilder::VAESDecOp(OpcodeArgs) {
Ref Key = LoadSource(FPRClass, Op, Op->Src[1], Op->Flags);
Ref Result = _VAESDec(DstSize, State, Key, LoadZeroVector(DstSize));
StoreResult(FPRClass, Op, Result, -1);
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::AESDecLastOp(OpcodeArgs) {
Ref Dest = LoadSource(FPRClass, Op, Op->Dest, Op->Flags);
Ref Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Ref Result = _VAESDecLast(16, Dest, Src, LoadZeroVector(16));
StoreResult(FPRClass, Op, Result, -1);
Ref Result = _VAESDecLast(OpSize::i128Bit, Dest, Src, LoadZeroVector(OpSize::i128Bit));
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::VAESDecLastOp(OpcodeArgs) {
const auto DstSize = GetDstSize(Op);
[[maybe_unused]] const auto Is128Bit = DstSize == Core::CPUState::XMM_SSE_REG_SIZE;
const auto DstSize = OpSizeFromDst(Op);
[[maybe_unused]] const auto Is128Bit = DstSize == OpSize::i128Bit;
// TODO: Handle 256-bit VAESDECLAST.
LOGMAN_THROW_A_FMT(Is128Bit, "256-bit VAESDECLAST unimplemented");
@@ -388,20 +394,20 @@ void OpDispatchBuilder::VAESDecLastOp(OpcodeArgs) {
Ref Key = LoadSource(FPRClass, Op, Op->Src[1], Op->Flags);
Ref Result = _VAESDecLast(DstSize, State, Key, LoadZeroVector(DstSize));
StoreResult(FPRClass, Op, Result, -1);
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
Ref OpDispatchBuilder::AESKeyGenAssistImpl(OpcodeArgs) {
Ref Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
const uint64_t RCON = Op->Src[1].Literal();
auto KeyGenSwizzle = LoadAndCacheNamedVectorConstant(16, NAMED_VECTOR_AESKEYGENASSIST_SWIZZLE);
return _VAESKeyGenAssist(Src, KeyGenSwizzle, LoadZeroVector(16), RCON);
auto KeyGenSwizzle = LoadAndCacheNamedVectorConstant(OpSize::i128Bit, NAMED_VECTOR_AESKEYGENASSIST_SWIZZLE);
return _VAESKeyGenAssist(Src, KeyGenSwizzle, LoadZeroVector(OpSize::i128Bit), RCON);
}
void OpDispatchBuilder::AESKeyGenAssist(OpcodeArgs) {
Ref Result = AESKeyGenAssistImpl(Op);
StoreResult(FPRClass, Op, Result, -1);
StoreResult(FPRClass, Op, Result, OpSize::iInvalid);
}
void OpDispatchBuilder::PCLMULQDQOp(OpcodeArgs) {
@@ -409,19 +415,19 @@ void OpDispatchBuilder::PCLMULQDQOp(OpcodeArgs) {
Ref Src = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
const auto Selector = static_cast<uint8_t>(Op->Src[1].Literal());
auto Res = _PCLMUL(16, Dest, Src, Selector & 0b1'0001);
StoreResult(FPRClass, Op, Res, -1);
auto Res = _PCLMUL(OpSize::i128Bit, Dest, Src, Selector & 0b1'0001);
StoreResult(FPRClass, Op, Res, OpSize::iInvalid);
}
void OpDispatchBuilder::VPCLMULQDQOp(OpcodeArgs) {
const auto DstSize = GetDstSize(Op);
const auto DstSize = OpSizeFromDst(Op);
Ref Src1 = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Ref Src2 = LoadSource(FPRClass, Op, Op->Src[1], Op->Flags);
const auto Selector = static_cast<uint8_t>(Op->Src[2].Literal());
Ref Res = _PCLMUL(DstSize, Src1, Src2, Selector & 0b1'0001);
StoreResult(FPRClass, Op, Res, -1);
StoreResult(FPRClass, Op, Res, OpSize::iInvalid);
}
} // namespace FEXCore::IR
@@ -0,0 +1,45 @@
// SPDX-License-Identifier: MIT
#pragma once
#include "Interface/Core/OpcodeDispatcher.h"
namespace FEXCore::IR {
constexpr std::tuple<uint8_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_DDDTable[] = {
{0x0C, 1, &OpDispatchBuilder::PI2FWOp},
{0x0D, 1, &OpDispatchBuilder::Vector_CVT_Int_To_Float<OpSize::i32Bit, false>},
{0x1C, 1, &OpDispatchBuilder::PF2IWOp},
{0x1D, 1, &OpDispatchBuilder::Vector_CVT_Float_To_Int<OpSize::i32Bit, false, false>},
{0x86, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VFRECP, OpSize::i32Bit>},
{0x87, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VFRSQRT, OpSize::i32Bit>},
{0x8A, 1, &OpDispatchBuilder::PFNACCOp},
{0x8E, 1, &OpDispatchBuilder::PFPNACCOp},
{0x90, 1, &OpDispatchBuilder::VPFCMPOp<1>},
{0x94, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFMIN, OpSize::i32Bit>},
{0x96, 1, &OpDispatchBuilder::VectorUnaryDuplicateOp<IR::OP_VFRECP, OpSize::i32Bit>},
{0x97, 1, &OpDispatchBuilder::VectorUnaryDuplicateOp<IR::OP_VFRSQRT, OpSize::i32Bit>},
{0x9A, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFSUB, OpSize::i32Bit>},
{0x9E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFADD, OpSize::i32Bit>},
{0xA0, 1, &OpDispatchBuilder::VPFCMPOp<2>},
{0xA4, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFMAX, OpSize::i32Bit>},
// Can be treated as a move
{0xA6, 1, &OpDispatchBuilder::MOVVectorUnalignedOp},
{0xA7, 1, &OpDispatchBuilder::MOVVectorUnalignedOp},
{0xAA, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUROp, IR::OP_VFSUB, OpSize::i32Bit>},
{0xAE, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFADDP, OpSize::i32Bit>},
{0xB0, 1, &OpDispatchBuilder::VPFCMPOp<0>},
{0xB4, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFMUL, OpSize::i32Bit>},
// Can be treated as a move
{0xB6, 1, &OpDispatchBuilder::MOVVectorUnalignedOp},
{0xB7, 1, &OpDispatchBuilder::PMULHRWOp},
{0xBB, 1, &OpDispatchBuilder::PSWAPDOp},
{0xBF, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VURAVG, OpSize::i8Bit>},
};
} // namespace FEXCore::IR
@@ -6,7 +6,6 @@ desc: Handles x86/64 flag generation
$end_info$
*/
#include "Interface/Context/Context.h"
#include "Interface/Core/OpcodeDispatcher.h"
#include "Interface/Core/X86Tables/X86Tables.h"
@@ -37,15 +36,14 @@ void OpDispatchBuilder::SetPackedRFLAG(bool Lower8, Ref Src) {
size_t NumFlags = FlagOffsets.size();
if (Lower8) {
// Calculate flags early.
// Could use InvalidateDeferredFlags() if we had masked invalidation.
// This is only a partial overwrite of flags since OF isn't stored here.
CalculateDeferredFlags();
NumFlags = 5;
} else {
// We are overwriting all RFLAGS. Invalidate the deferred flag state.
InvalidateDeferredFlags();
}
// PF and CF are both stored inverted, so hoist the invert.
auto SrcInverted = _Not(OpSize::i32Bit, Src);
for (size_t i = 0; i < NumFlags; ++i) {
const auto FlagOffset = FlagOffsets[i];
@@ -59,31 +57,28 @@ void OpDispatchBuilder::SetPackedRFLAG(bool Lower8, Ref Src) {
// So we write out the whole flags byte to AF without an extract.
static_assert(FEXCore::X86State::RFLAG_AF_RAW_LOC == 4);
SetRFLAG(Src, FEXCore::X86State::RFLAG_AF_RAW_LOC);
} else if (FlagOffset == FEXCore::X86State::RFLAG_PF_RAW_LOC) {
// PF is stored parity flipped
Ref Tmp = _Bfe(OpSize::i32Bit, 1, FlagOffset, Src);
Tmp = _Xor(OpSize::i32Bit, Tmp, _Constant(1));
SetRFLAG(Tmp, FlagOffset);
} else if (FlagOffset == FEXCore::X86State::RFLAG_PF_RAW_LOC || FlagOffset == FEXCore::X86State::RFLAG_CF_RAW_LOC) {
// PF and CF are both stored parity flipped.
SetRFLAG(SrcInverted, FlagOffset, FlagOffset, true);
} else {
SetRFLAG(Src, FlagOffset, FlagOffset, true);
}
}
CFInverted = true;
}
Ref OpDispatchBuilder::GetPackedRFLAG(uint32_t FlagsMask) {
// Calculate flags early.
CalculateDeferredFlags();
Ref Original = _Constant(0);
// SF/ZF and N/Z are together on both arm64 and x86_64, so we special case that.
bool GetNZ = (FlagsMask & (1 << FEXCore::X86State::RFLAG_SF_RAW_LOC)) && (FlagsMask & (1 << FEXCore::X86State::RFLAG_ZF_RAW_LOC));
// Handle CF first, since it's at bit 0 and hence doesn't need shift or OR.
if (FlagsMask & (1 << FEXCore::X86State::RFLAG_CF_RAW_LOC)) {
static_assert(FEXCore::X86State::RFLAG_CF_RAW_LOC == 0);
Original = GetRFLAG(FEXCore::X86State::RFLAG_CF_RAW_LOC);
}
LOGMAN_THROW_A_FMT(FlagsMask & (1 << FEXCore::X86State::RFLAG_CF_RAW_LOC), "CF always handled");
static_assert(FEXCore::X86State::RFLAG_CF_RAW_LOC == 0);
Ref Original = GetRFLAG(FEXCore::X86State::RFLAG_CF_RAW_LOC);
for (size_t i = 0; i < FlagOffsets.size(); ++i) {
const auto FlagOffset = FlagOffsets[i];
@@ -114,7 +109,7 @@ Ref OpDispatchBuilder::GetPackedRFLAG(uint32_t FlagsMask) {
// instead.
if (FlagsMask & (1 << FEXCore::X86State::RFLAG_PF_RAW_LOC)) {
// Set every bit except the bottommost.
auto OnesInvPF = _Or(OpSize::i64Bit, LoadPFRaw(false), _Constant(~1ull));
auto OnesInvPF = _Or(OpSize::i64Bit, LoadPFRaw(false, false), _InlineConstant(~1ull));
// Rotate the bottom bit to the appropriate location for PF, so we get
// something like 111P1111. Then invert that to get 000p0000. Then OR that
@@ -127,21 +122,21 @@ Ref OpDispatchBuilder::GetPackedRFLAG(uint32_t FlagsMask) {
if (GetNZ) {
static_assert(FEXCore::X86State::RFLAG_SF_RAW_LOC == (FEXCore::X86State::RFLAG_ZF_RAW_LOC + 1));
auto NZCV = GetNZCV();
auto NZ = _And(OpSize::i64Bit, NZCV, _Constant(0b11u << 30));
auto NZ = _And(OpSize::i64Bit, NZCV, _InlineConstant(0b11u << 30));
Original = _Orlshr(OpSize::i64Bit, Original, NZ, 31 - FEXCore::X86State::RFLAG_SF_RAW_LOC);
}
// The constant is OR'ed in at the end, to avoid a pointless or xzr, #2.
if ((1U << X86State::RFLAG_RESERVED_LOC) & FlagsMask) {
Original = _Or(OpSize::i64Bit, Original, _Constant(2));
Original = _Or(OpSize::i64Bit, Original, _InlineConstant(2));
}
return Original;
}
void OpDispatchBuilder::CalculateOF(uint8_t SrcSize, Ref Res, Ref Src1, Ref Src2, bool Sub) {
auto OpSize = SrcSize == 8 ? OpSize::i64Bit : OpSize::i32Bit;
uint64_t SignBit = (SrcSize * 8) - 1;
void OpDispatchBuilder::CalculateOF(IR::OpSize SrcSize, Ref Res, Ref Src1, Ref Src2, bool Sub) {
const auto OpSize = SrcSize == OpSize::i64Bit ? OpSize::i64Bit : OpSize::i32Bit;
const uint64_t SignBit = IR::OpSizeAsBits(SrcSize) - 1;
Ref Anded = nullptr;
// For add, OF is set iff the sources have the same sign but the destination
@@ -172,25 +167,15 @@ void OpDispatchBuilder::CalculateOF(uint8_t SrcSize, Ref Res, Ref Src1, Ref Src2
}
}
SetRFLAG<FEXCore::X86State::RFLAG_OF_RAW_LOC>(Anded, SrcSize * 8 - 1, true);
SetRFLAG<FEXCore::X86State::RFLAG_OF_RAW_LOC>(Anded, SignBit, true);
}
Ref OpDispatchBuilder::LoadPFRaw(bool Invert) {
// Read the stored byte. This is the original result (up to 64-bits), it needs
// parity calculated.
auto Result = GetRFLAG(FEXCore::X86State::RFLAG_PF_RAW_LOC);
Ref OpDispatchBuilder::LoadPFRaw(bool Mask, bool Invert) {
// Most blocks do not read parity, so PF optimization is gated on this flag.
CurrentHeader->ReadsParity = true;
// Cascade to calculate parity of bottom 8-bits to bottom bit.
Result = _XorShift(OpSize::i32Bit, Result, Result, ShiftType::LSR, 4);
Result = _XorShift(OpSize::i32Bit, Result, Result, ShiftType::LSR, 2);
if (Invert) {
Result = _XornShift(OpSize::i32Bit, Result, Result, ShiftType::LSR, 1);
} else {
Result = _XorShift(OpSize::i32Bit, Result, Result, ShiftType::LSR, 1);
}
return Result;
// Evaluate parity on the deferred raw value.
return _Parity(GetRFLAG(FEXCore::X86State::RFLAG_PF_RAW_LOC), Mask, Invert);
}
Ref OpDispatchBuilder::LoadAF() {
@@ -267,34 +252,42 @@ void OpDispatchBuilder::CalculateDeferredFlags() {
NZCVDirty = false;
}
Ref OpDispatchBuilder::CalculateFlags_ADC(uint8_t SrcSize, Ref Src1, Ref Src2) {
auto Zero = _Constant(0);
auto One = _Constant(1);
auto OpSize = SrcSize == 8 ? OpSize::i64Bit : OpSize::i32Bit;
Ref OpDispatchBuilder::IncrementByCarry(OpSize OpSize, Ref Src) {
// If CF not inverted, we use .cc since the increment happens when the
// condition is false. If CF inverted, invert to use .cs. A bit mindbendy.
return _NZCVSelectIncrement(OpSize, {CFInverted ? COND_UGE : COND_ULT}, Src, Src);
}
Ref OpDispatchBuilder::CalculateFlags_ADC(IR::OpSize SrcSize, Ref Src1, Ref Src2) {
auto Zero = _InlineConstant(0);
auto One = _InlineConstant(1);
auto OpSize = SrcSize == OpSize::i64Bit ? OpSize::i64Bit : OpSize::i32Bit;
Ref Res;
CalculateAF(Src1, Src2);
if (SrcSize >= 4) {
if (SrcSize >= OpSize::i32Bit) {
RectifyCarryInvert(false);
HandleNZCV_RMW();
Res = _AdcWithFlags(OpSize, Src1, Src2);
CFInverted = false;
} else {
// Need to zero-extend for correct comparisons below
Src2 = _Bfe(OpSize, SrcSize * 8, 0, Src2);
Src2 = _Bfe(OpSize, IR::OpSizeAsBits(SrcSize), 0, Src2);
// Note that we do not extend Src2PlusCF, since we depend on proper
// 32-bit arithmetic to correctly handle the Src2 = 0xffff case.
Ref Src2PlusCF = _Adc(OpSize, _Constant(0), Src2);
Ref Src2PlusCF = IncrementByCarry(OpSize, Src2);
// Need to zero-extend for the comparison.
Res = _Add(OpSize, Src1, Src2PlusCF);
Res = _Bfe(OpSize, SrcSize * 8, 0, Res);
Res = _Bfe(OpSize, IR::OpSizeAsBits(SrcSize), 0, Res);
// TODO: We can fold that second Bfe in (cmp uxth).
auto SelectCF = _Select(FEXCore::IR::COND_ULT, Res, Src2PlusCF, One, Zero);
auto SelectCFInv = _Select(FEXCore::IR::COND_UGE, Res, Src2PlusCF, One, Zero);
SetNZ_ZeroCV(SrcSize, Res);
SetRFLAG<FEXCore::X86State::RFLAG_CF_RAW_LOC>(SelectCF);
SetCFInverted(SelectCFInv);
CalculateOF(SrcSize, Res, Src1, Src2, false);
}
@@ -302,37 +295,35 @@ Ref OpDispatchBuilder::CalculateFlags_ADC(uint8_t SrcSize, Ref Src1, Ref Src2) {
return Res;
}
Ref OpDispatchBuilder::CalculateFlags_SBB(uint8_t SrcSize, Ref Src1, Ref Src2) {
auto Zero = _Constant(0);
auto One = _Constant(1);
auto OpSize = SrcSize == 8 ? OpSize::i64Bit : OpSize::i32Bit;
Ref OpDispatchBuilder::CalculateFlags_SBB(IR::OpSize SrcSize, Ref Src1, Ref Src2) {
auto Zero = _InlineConstant(0);
auto One = _InlineConstant(1);
auto OpSize = SrcSize == OpSize::i64Bit ? OpSize::i64Bit : OpSize::i32Bit;
CalculateAF(Src1, Src2);
Ref Res;
if (SrcSize >= 4) {
// Rectify input carry
CarryInvert();
if (SrcSize >= OpSize::i32Bit) {
// Arm's subtraction has inverted CF from x86, so rectify the input and
// invert the output.
RectifyCarryInvert(true);
HandleNZCV_RMW();
Res = _SbbWithFlags(OpSize, Src1, Src2);
// Rectify output carry
CarryInvert();
CFInverted = true;
} else {
// Zero extend for correct comparison behaviour with Src1 = 0xffff.
Src1 = _Bfe(OpSize, SrcSize * 8, 0, Src1);
Src2 = _Bfe(OpSize, SrcSize * 8, 0, Src2);
Src1 = _Bfe(OpSize, IR::OpSizeAsBits(SrcSize), 0, Src1);
Src2 = _Bfe(OpSize, IR::OpSizeAsBits(SrcSize), 0, Src2);
auto Src2PlusCF = _Adc(OpSize, _Constant(0), Src2);
auto Src2PlusCF = IncrementByCarry(OpSize, Src2);
Res = _Sub(OpSize, Src1, Src2PlusCF);
Res = _Bfe(OpSize, SrcSize * 8, 0, Res);
Res = _Bfe(OpSize, IR::OpSizeAsBits(SrcSize), 0, Res);
auto SelectCF = _Select(FEXCore::IR::COND_ULT, Src1, Src2PlusCF, One, Zero);
auto SelectCFInv = _Select(FEXCore::IR::COND_UGE, Src1, Src2PlusCF, One, Zero);
SetNZ_ZeroCV(SrcSize, Res);
SetRFLAG<FEXCore::X86State::RFLAG_CF_RAW_LOC>(SelectCF);
SetCFInverted(SelectCFInv);
CalculateOF(SrcSize, Res, Src1, Src2, true);
}
@@ -340,93 +331,99 @@ Ref OpDispatchBuilder::CalculateFlags_SBB(uint8_t SrcSize, Ref Src1, Ref Src2) {
return Res;
}
Ref OpDispatchBuilder::CalculateFlags_SUB(uint8_t SrcSize, Ref Src1, Ref Src2, bool UpdateCF) {
Ref OpDispatchBuilder::CalculateFlags_SUB(IR::OpSize SrcSize, Ref Src1, Ref Src2, bool UpdateCF) {
// Stash CF before stomping over it
auto OldCF = UpdateCF ? nullptr : GetRFLAG(FEXCore::X86State::RFLAG_CF_RAW_LOC);
auto OldCFInv = UpdateCF ? nullptr : GetRFLAG(FEXCore::X86State::RFLAG_CF_RAW_LOC, true);
HandleNZCVWrite();
CalculateAF(Src1, Src2);
Ref Res;
if (SrcSize >= 4) {
Res = _SubWithFlags(IR::SizeToOpSize(SrcSize), Src1, Src2);
if (SrcSize >= OpSize::i32Bit) {
Res = _SubWithFlags(SrcSize, Src1, Src2);
} else {
_SubNZCV(IR::SizeToOpSize(SrcSize), Src1, Src2);
_SubNZCV(SrcSize, Src1, Src2);
Res = _Sub(OpSize::i32Bit, Src1, Src2);
}
CalculatePF(Res);
// If we're updating CF, we need to invert it for correctness. If we're not
// updating CF, we need to restore the CF since we stomped over it.
// If we're updating CF, we need it to be inverted because SubNZCV is inverted
// from x86. If we're not updating CF, we need to restore the CF since we
// stomped over it.
if (UpdateCF) {
CarryInvert();
CFInverted = true;
} else {
SetRFLAG<FEXCore::X86State::RFLAG_CF_RAW_LOC>(OldCF);
SetCFInverted(OldCFInv);
}
return Res;
}
Ref OpDispatchBuilder::CalculateFlags_ADD(uint8_t SrcSize, Ref Src1, Ref Src2, bool UpdateCF) {
Ref OpDispatchBuilder::CalculateFlags_ADD(IR::OpSize SrcSize, Ref Src1, Ref Src2, bool UpdateCF) {
// Stash CF before stomping over it
auto OldCF = UpdateCF ? nullptr : GetRFLAG(FEXCore::X86State::RFLAG_CF_RAW_LOC);
auto OldCFInv = UpdateCF ? nullptr : GetRFLAG(FEXCore::X86State::RFLAG_CF_RAW_LOC, true);
HandleNZCVWrite();
CalculateAF(Src1, Src2);
Ref Res;
if (SrcSize >= 4) {
Res = _AddWithFlags(IR::SizeToOpSize(SrcSize), Src1, Src2);
if (SrcSize >= OpSize::i32Bit) {
Res = _AddWithFlags(SrcSize, Src1, Src2);
} else {
_AddNZCV(IR::SizeToOpSize(SrcSize), Src1, Src2);
_AddNZCV(SrcSize, Src1, Src2);
Res = _Add(OpSize::i32Bit, Src1, Src2);
}
CalculatePF(Res);
// We stomped over CF while calculation flags, restore it.
if (!UpdateCF) {
SetRFLAG<FEXCore::X86State::RFLAG_CF_RAW_LOC>(OldCF);
if (UpdateCF) {
// Adds match between x86 and arm64.
CFInverted = false;
} else {
SetCFInverted(OldCFInv);
}
return Res;
}
void OpDispatchBuilder::CalculateFlags_MUL(uint8_t SrcSize, Ref Res, Ref High) {
void OpDispatchBuilder::CalculateFlags_MUL(IR::OpSize SrcSize, Ref Res, Ref High) {
HandleNZCVWrite();
InvalidatePF_AF();
// CF and OF are set if the result of the operation can't be fit in to the destination register
// If the value can fit then the top bits will be zero
auto SignBit = _Sbfe(OpSize::i64Bit, 1, SrcSize * 8 - 1, Res);
auto SignBit = _Sbfe(OpSize::i64Bit, 1, IR::OpSizeAsBits(SrcSize) - 1, Res);
_SubNZCV(OpSize::i64Bit, High, SignBit);
// If High = SignBit, then sets to nZcv. Else sets to nzCV. Since SF/ZF
// undefined, this does what we need.
auto Zero = _Constant(0);
_CondAddNZCV(OpSize::i64Bit, Zero, Zero, CondClassType {COND_EQ}, 0x3 /* nzCV */);
// If High = SignBit, then sets to nZCv. Else sets to nzcV. Since SF/ZF
// undefined, this does what we need after inverting carry.
auto Zero = _InlineConstant(0);
_CondSubNZCV(OpSize::i64Bit, Zero, Zero, CondClassType {COND_EQ}, 0x1 /* nzcV */);
CFInverted = true;
}
void OpDispatchBuilder::CalculateFlags_UMUL(Ref High) {
HandleNZCVWrite();
InvalidatePF_AF();
auto Zero = _Constant(0);
OpSize Size = IR::SizeToOpSize(GetOpSize(High));
auto Zero = _InlineConstant(0);
const auto Size = GetOpSize(High);
// CF and OF are set if the result of the operation can't be fit in to the destination register
// The result register will be all zero if it can't fit due to how multiplication behaves
_SubNZCV(Size, High, Zero);
// If High = 0, then sets to nZcv. Else sets to nzCV. Since SF/ZF undefined,
// If High = 0, then sets to nZCv. Else sets to nzcV. Since SF/ZF undefined,
// this does what we need.
_CondAddNZCV(Size, Zero, Zero, CondClassType {COND_EQ}, 0x3 /* nzCV */);
_CondSubNZCV(Size, Zero, Zero, CondClassType {COND_EQ}, 0x1 /* nzcV */);
CFInverted = true;
}
void OpDispatchBuilder::CalculateFlags_Logical(uint8_t SrcSize, Ref Res, Ref Src1, Ref Src2) {
void OpDispatchBuilder::CalculateFlags_Logical(IR::OpSize SrcSize, Ref Res, Ref Src1, Ref Src2) {
InvalidateAF();
CalculatePF(Res);
@@ -435,13 +432,13 @@ void OpDispatchBuilder::CalculateFlags_Logical(uint8_t SrcSize, Ref Res, Ref Src
SetNZ_ZeroCV(SrcSize, Res);
}
void OpDispatchBuilder::CalculateFlags_ShiftLeftImmediate(uint8_t SrcSize, Ref UnmaskedRes, Ref Src1, uint64_t Shift) {
void OpDispatchBuilder::CalculateFlags_ShiftLeftImmediate(IR::OpSize SrcSize, Ref UnmaskedRes, Ref Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) {
return;
}
auto OpSize = SrcSize == 8 ? OpSize::i64Bit : OpSize::i32Bit;
auto OpSize = SrcSize == OpSize::i64Bit ? OpSize::i64Bit : OpSize::i32Bit;
SetNZ_ZeroCV(SrcSize, UnmaskedRes);
@@ -450,9 +447,9 @@ void OpDispatchBuilder::CalculateFlags_ShiftLeftImmediate(uint8_t SrcSize, Ref U
// Extract the last bit shifted in to CF. Shift is already masked, but for
// 8/16-bit it might be >= SrcSizeBits, in which case CF is cleared. There's
// nothing to do in that case since we already cleared CF above.
auto SrcSizeBits = SrcSize * 8;
const auto SrcSizeBits = IR::OpSizeAsBits(SrcSize);
if (Shift < SrcSizeBits) {
SetRFLAG<FEXCore::X86State::RFLAG_CF_RAW_LOC>(Src1, SrcSizeBits - Shift, true);
SetCFDirect(Src1, SrcSizeBits - Shift, true);
}
}
@@ -463,13 +460,13 @@ void OpDispatchBuilder::CalculateFlags_ShiftLeftImmediate(uint8_t SrcSize, Ref U
// In the case of left shift. OF is only set from the result of <Top Source Bit> XOR <Top Result Bit>
if (Shift == 1) {
auto Xor = _Xor(OpSize, UnmaskedRes, Src1);
SetRFLAG<FEXCore::X86State::RFLAG_OF_RAW_LOC>(Xor, SrcSize * 8 - 1, true);
SetRFLAG<FEXCore::X86State::RFLAG_OF_RAW_LOC>(Xor, IR::OpSizeAsBits(SrcSize) - 1, true);
} else {
// Undefined, we choose to zero as part of SetNZ_ZeroCV
}
}
void OpDispatchBuilder::CalculateFlags_SignShiftRightImmediate(uint8_t SrcSize, Ref Res, Ref Src1, uint64_t Shift) {
void OpDispatchBuilder::CalculateFlags_SignShiftRightImmediate(IR::OpSize SrcSize, Ref Res, Ref Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) {
return;
@@ -477,11 +474,8 @@ void OpDispatchBuilder::CalculateFlags_SignShiftRightImmediate(uint8_t SrcSize,
SetNZ_ZeroCV(SrcSize, Res);
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_RAW_LOC>(Src1, Shift - 1, true);
}
// Extract the last bit shifted in to CF
SetCFDirect(Src1, Shift - 1, true);
CalculatePF(Res);
InvalidateAF();
@@ -492,22 +486,19 @@ void OpDispatchBuilder::CalculateFlags_SignShiftRightImmediate(uint8_t SrcSize,
// already zeroed there's nothing to do here.
}
void OpDispatchBuilder::CalculateFlags_ShiftRightImmediateCommon(uint8_t SrcSize, Ref Res, Ref Src1, uint64_t Shift) {
void OpDispatchBuilder::CalculateFlags_ShiftRightImmediateCommon(IR::OpSize SrcSize, Ref Res, Ref Src1, uint64_t Shift) {
// Set SF and PF. Clobbers OF, but OF only defined for Shift = 1 where it is
// set below.
SetNZ_ZeroCV(SrcSize, Res);
// CF
{
// Extract the last bit shifted in to CF
SetRFLAG<FEXCore::X86State::RFLAG_CF_RAW_LOC>(Src1, Shift - 1, true);
}
// Extract the last bit shifted in to CF
SetCFDirect(Src1, Shift - 1, true);
CalculatePF(Res);
InvalidateAF();
}
void OpDispatchBuilder::CalculateFlags_ShiftRightImmediate(uint8_t SrcSize, Ref Res, Ref Src1, uint64_t Shift) {
void OpDispatchBuilder::CalculateFlags_ShiftRightImmediate(IR::OpSize SrcSize, Ref Res, Ref Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) {
return;
@@ -520,18 +511,18 @@ void OpDispatchBuilder::CalculateFlags_ShiftRightImmediate(uint8_t SrcSize, Ref
// Only defined when Shift is 1 else undefined
// Is set to the MSB of the original value
if (Shift == 1) {
SetRFLAG<FEXCore::X86State::RFLAG_OF_RAW_LOC>(Src1, SrcSize * 8 - 1, true);
SetRFLAG<FEXCore::X86State::RFLAG_OF_RAW_LOC>(Src1, IR::OpSizeAsBits(SrcSize) - 1, true);
}
}
}
void OpDispatchBuilder::CalculateFlags_ShiftRightDoubleImmediate(uint8_t SrcSize, Ref Res, Ref Src1, uint64_t Shift) {
void OpDispatchBuilder::CalculateFlags_ShiftRightDoubleImmediate(IR::OpSize SrcSize, Ref Res, Ref Src1, uint64_t Shift) {
// No flags changed if shift is zero
if (Shift == 0) {
return;
}
const auto OpSize = SrcSize == 8 ? OpSize::i64Bit : OpSize::i32Bit;
const auto OpSize = SrcSize == OpSize::i64Bit ? OpSize::i64Bit : OpSize::i32Bit;
CalculateFlags_ShiftRightImmediateCommon(SrcSize, Res, Src1, Shift);
// OF
@@ -541,70 +532,12 @@ void OpDispatchBuilder::CalculateFlags_ShiftRightDoubleImmediate(uint8_t SrcSize
// XOR of Result and Src1
if (Shift == 1) {
auto val = _Xor(OpSize, Src1, Res);
SetRFLAG<FEXCore::X86State::RFLAG_OF_RAW_LOC>(val, SrcSize * 8 - 1, true);
SetRFLAG<FEXCore::X86State::RFLAG_OF_RAW_LOC>(val, IR::OpSizeAsBits(SrcSize) - 1, true);
}
}
}
void OpDispatchBuilder::CalculateFlags_BEXTR(Ref Src) {
// ZF is set properly. CF and OF are defined as being set to zero. SF, PF, and
// AF are undefined.
SetNZ_ZeroCV(GetOpSize(Src), Src);
InvalidatePF_AF();
}
void OpDispatchBuilder::CalculateFlags_BLSI(uint8_t SrcSize, Ref Result) {
// CF is cleared if Src is zero, otherwise it's set. However, Src is zero iff
// Result is zero, so we can test the result instead. So, CF is just the
// inverted ZF.
//
// ZF/SF/OF set as usual.
SetNZ_ZeroCV(SrcSize, Result);
InvalidatePF_AF();
auto CFOp = GetRFLAG(X86State::RFLAG_ZF_RAW_LOC, true /* Invert */);
SetRFLAG<X86State::RFLAG_CF_RAW_LOC>(CFOp);
}
void OpDispatchBuilder::CalculateFlags_BLSMSK(uint8_t SrcSize, Ref Result, Ref Src) {
InvalidatePF_AF();
// CF set according to the Src
auto Zero = _Constant(0);
auto One = _Constant(1);
auto CFOp = _Select(IR::COND_EQ, Src, Zero, One, Zero);
// The output of BLSMSK is always nonzero, so TST will clear Z (along with C
// and O) while setting S.
SetNZ_ZeroCV(SrcSize, Result);
SetRFLAG<X86State::RFLAG_CF_RAW_LOC>(CFOp);
}
void OpDispatchBuilder::CalculateFlags_BLSR(uint8_t SrcSize, Ref Result, Ref Src) {
auto Zero = _Constant(0);
auto One = _Constant(1);
auto CFOp = _Select(IR::COND_EQ, Src, Zero, One, Zero);
SetNZ_ZeroCV(SrcSize, Result);
SetRFLAG<X86State::RFLAG_CF_RAW_LOC>(CFOp);
InvalidatePF_AF();
}
void OpDispatchBuilder::CalculateFlags_POPCOUNT(Ref Result) {
// We need to set ZF while clearing the rest of NZCV. The result of a popcount
// is in the range [0, 63]. In particular, it is always positive. So a
// combined NZ test will correctly zero SF/CF/OF while setting ZF.
SetNZ_ZeroCV(OpSize::i32Bit, Result);
ZeroPF_AF();
}
void OpDispatchBuilder::CalculateFlags_BZHI(uint8_t SrcSize, Ref Result, Ref Src) {
InvalidatePF_AF();
SetNZ_ZeroCV(SrcSize, Result);
SetRFLAG<X86State::RFLAG_CF_RAW_LOC>(Src);
}
void OpDispatchBuilder::CalculateFlags_ZCNT(uint8_t SrcSize, Ref Result) {
void OpDispatchBuilder::CalculateFlags_ZCNT(IR::OpSize SrcSize, Ref Result) {
// OF, SF, AF, PF all undefined
// Test ZF of result, SF is undefined so this is ok.
SetNZ_ZeroCV(SrcSize, Result);
@@ -612,17 +545,8 @@ void OpDispatchBuilder::CalculateFlags_ZCNT(uint8_t SrcSize, Ref Result) {
// Now set CF if the Result = SrcSize * 8. Since SrcSize is a power-of-two and
// Result is <= SrcSize * 8, we equivalently check if the log2(SrcSize * 8)
// bit is set. No masking is needed because no higher bits could be set.
unsigned CarryBit = FEXCore::ilog2(SrcSize * 8u);
SetRFLAG<FEXCore::X86State::RFLAG_CF_RAW_LOC>(Result, CarryBit);
}
void OpDispatchBuilder::CalculateFlags_RDRAND(Ref Src) {
// OF, SF, ZF, AF, PF all zero
ZeroNZCV();
ZeroPF_AF();
// CF is set to the incoming source
SetRFLAG<FEXCore::X86State::RFLAG_CF_RAW_LOC>(Src);
unsigned CarryBit = FEXCore::ilog2(IR::OpSizeAsBits(SrcSize));
SetCFDirect(Result, CarryBit);
}
} // namespace FEXCore::IR
@@ -0,0 +1,82 @@
// SPDX-License-Identifier: MIT
#pragma once
#include "Interface/Core/OpcodeDispatcher.h"
namespace FEXCore::IR {
#define OPD(prefix, opcode) (((prefix) << 8) | opcode)
constexpr uint16_t PF_38_NONE = 0;
constexpr uint16_t PF_38_66 = (1U << 0);
constexpr uint16_t PF_38_F3 = (1U << 2);
constexpr std::tuple<uint16_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_H0F38Table[] = {
{OPD(PF_38_NONE, 0x00), 1, &OpDispatchBuilder::PSHUFBOp},
{OPD(PF_38_66, 0x00), 1, &OpDispatchBuilder::PSHUFBOp},
{OPD(PF_38_NONE, 0x01), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADDP, OpSize::i16Bit>},
{OPD(PF_38_66, 0x01), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADDP, OpSize::i16Bit>},
{OPD(PF_38_NONE, 0x02), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADDP, OpSize::i32Bit>},
{OPD(PF_38_66, 0x02), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADDP, OpSize::i32Bit>},
{OPD(PF_38_NONE, 0x03), 1, &OpDispatchBuilder::PHADDS},
{OPD(PF_38_66, 0x03), 1, &OpDispatchBuilder::PHADDS},
{OPD(PF_38_NONE, 0x04), 1, &OpDispatchBuilder::PMADDUBSW},
{OPD(PF_38_66, 0x04), 1, &OpDispatchBuilder::PMADDUBSW},
{OPD(PF_38_NONE, 0x05), 1, &OpDispatchBuilder::PHSUB<OpSize::i16Bit>},
{OPD(PF_38_66, 0x05), 1, &OpDispatchBuilder::PHSUB<OpSize::i16Bit>},
{OPD(PF_38_NONE, 0x06), 1, &OpDispatchBuilder::PHSUB<OpSize::i32Bit>},
{OPD(PF_38_66, 0x06), 1, &OpDispatchBuilder::PHSUB<OpSize::i32Bit>},
{OPD(PF_38_NONE, 0x07), 1, &OpDispatchBuilder::PHSUBS},
{OPD(PF_38_66, 0x07), 1, &OpDispatchBuilder::PHSUBS},
{OPD(PF_38_NONE, 0x08), 1, &OpDispatchBuilder::PSIGN<OpSize::i8Bit>},
{OPD(PF_38_66, 0x08), 1, &OpDispatchBuilder::PSIGN<OpSize::i8Bit>},
{OPD(PF_38_NONE, 0x09), 1, &OpDispatchBuilder::PSIGN<OpSize::i16Bit>},
{OPD(PF_38_66, 0x09), 1, &OpDispatchBuilder::PSIGN<OpSize::i16Bit>},
{OPD(PF_38_NONE, 0x0A), 1, &OpDispatchBuilder::PSIGN<OpSize::i32Bit>},
{OPD(PF_38_66, 0x0A), 1, &OpDispatchBuilder::PSIGN<OpSize::i32Bit>},
{OPD(PF_38_NONE, 0x0B), 1, &OpDispatchBuilder::PMULHRSW},
{OPD(PF_38_66, 0x0B), 1, &OpDispatchBuilder::PMULHRSW},
{OPD(PF_38_66, 0x10), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorVariableBlend, OpSize::i8Bit>},
{OPD(PF_38_66, 0x14), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorVariableBlend, OpSize::i32Bit>},
{OPD(PF_38_66, 0x15), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorVariableBlend, OpSize::i64Bit>},
{OPD(PF_38_66, 0x17), 1, &OpDispatchBuilder::PTestOp},
{OPD(PF_38_NONE, 0x1C), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VABS, OpSize::i8Bit>},
{OPD(PF_38_66, 0x1C), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VABS, OpSize::i8Bit>},
{OPD(PF_38_NONE, 0x1D), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VABS, OpSize::i16Bit>},
{OPD(PF_38_66, 0x1D), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VABS, OpSize::i16Bit>},
{OPD(PF_38_NONE, 0x1E), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VABS, OpSize::i32Bit>},
{OPD(PF_38_66, 0x1E), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VABS, OpSize::i32Bit>},
{OPD(PF_38_66, 0x20), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i8Bit, OpSize::i16Bit, true>},
{OPD(PF_38_66, 0x21), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i8Bit, OpSize::i32Bit, true>},
{OPD(PF_38_66, 0x22), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i8Bit, OpSize::i64Bit, true>},
{OPD(PF_38_66, 0x23), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i16Bit, OpSize::i32Bit, true>},
{OPD(PF_38_66, 0x24), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i16Bit, OpSize::i64Bit, true>},
{OPD(PF_38_66, 0x25), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i32Bit, OpSize::i64Bit, true>},
{OPD(PF_38_66, 0x28), 1, &OpDispatchBuilder::PMULLOp<OpSize::i32Bit, true>},
{OPD(PF_38_66, 0x29), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPEQ, OpSize::i64Bit>},
{OPD(PF_38_66, 0x2A), 1, &OpDispatchBuilder::MOVVectorNTOp},
{OPD(PF_38_66, 0x2B), 1, &OpDispatchBuilder::PACKUSOp<OpSize::i32Bit>},
{OPD(PF_38_66, 0x30), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i8Bit, OpSize::i16Bit, false>},
{OPD(PF_38_66, 0x31), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i8Bit, OpSize::i32Bit, false>},
{OPD(PF_38_66, 0x32), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i8Bit, OpSize::i64Bit, false>},
{OPD(PF_38_66, 0x33), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i16Bit, OpSize::i32Bit, false>},
{OPD(PF_38_66, 0x34), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i16Bit, OpSize::i64Bit, false>},
{OPD(PF_38_66, 0x35), 1, &OpDispatchBuilder::ExtendVectorElements<OpSize::i32Bit, OpSize::i64Bit, false>},
{OPD(PF_38_66, 0x37), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPGT, OpSize::i64Bit>},
{OPD(PF_38_66, 0x38), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSMIN, OpSize::i8Bit>},
{OPD(PF_38_66, 0x39), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSMIN, OpSize::i32Bit>},
{OPD(PF_38_66, 0x3A), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUMIN, OpSize::i16Bit>},
{OPD(PF_38_66, 0x3B), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUMIN, OpSize::i32Bit>},
{OPD(PF_38_66, 0x3C), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSMAX, OpSize::i8Bit>},
{OPD(PF_38_66, 0x3D), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSMAX, OpSize::i32Bit>},
{OPD(PF_38_66, 0x3E), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUMAX, OpSize::i16Bit>},
{OPD(PF_38_66, 0x3F), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUMAX, OpSize::i32Bit>},
{OPD(PF_38_66, 0x40), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VMUL, OpSize::i32Bit>},
{OPD(PF_38_66, 0x41), 1, &OpDispatchBuilder::PHMINPOSUWOp},
{OPD(PF_38_NONE, 0xF0), 2, &OpDispatchBuilder::MOVBEOp},
{OPD(PF_38_66, 0xF0), 2, &OpDispatchBuilder::MOVBEOp},
{OPD(PF_38_66, 0xF6), 1, &OpDispatchBuilder::ADXOp},
{OPD(PF_38_F3, 0xF6), 1, &OpDispatchBuilder::ADXOp},
};
#undef OPD
} // namespace FEXCore::IR
@@ -0,0 +1,51 @@
// SPDX-License-Identifier: MIT
#pragma once
#include "Interface/Core/OpcodeDispatcher.h"
namespace FEXCore::IR {
#define OPD(REX, prefix, opcode) ((REX << 9) | (prefix << 8) | opcode)
#define PF_3A_NONE 0
#define PF_3A_66 1
constexpr std::tuple<uint16_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_H0F3ATable[] = {
{OPD(0, PF_3A_66, 0x08), 1, &OpDispatchBuilder::VectorRound<OpSize::i32Bit>},
{OPD(0, PF_3A_66, 0x09), 1, &OpDispatchBuilder::VectorRound<OpSize::i64Bit>},
{OPD(0, PF_3A_66, 0x0A), 1, &OpDispatchBuilder::InsertScalarRound<OpSize::i32Bit>},
{OPD(0, PF_3A_66, 0x0B), 1, &OpDispatchBuilder::InsertScalarRound<OpSize::i64Bit>},
{OPD(0, PF_3A_66, 0x0C), 1, &OpDispatchBuilder::VectorBlend<OpSize::i32Bit>},
{OPD(0, PF_3A_66, 0x0D), 1, &OpDispatchBuilder::VectorBlend<OpSize::i64Bit>},
{OPD(0, PF_3A_66, 0x0E), 1, &OpDispatchBuilder::VectorBlend<OpSize::i16Bit>},
{OPD(0, PF_3A_NONE, 0x0F), 1, &OpDispatchBuilder::PAlignrOp},
{OPD(0, PF_3A_66, 0x0F), 1, &OpDispatchBuilder::PAlignrOp},
{OPD(0, PF_3A_66, 0x14), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PExtrOp, OpSize::i8Bit>},
{OPD(0, PF_3A_66, 0x15), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PExtrOp, OpSize::i16Bit>},
{OPD(0, PF_3A_66, 0x16), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PExtrOp, OpSize::i32Bit>},
{OPD(0, PF_3A_66, 0x17), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PExtrOp, OpSize::i32Bit>},
{OPD(0, PF_3A_66, 0x20), 1, &OpDispatchBuilder::PINSROp<OpSize::i8Bit>},
{OPD(0, PF_3A_66, 0x21), 1, &OpDispatchBuilder::InsertPSOp},
{OPD(0, PF_3A_66, 0x22), 1, &OpDispatchBuilder::PINSROp<OpSize::i32Bit>},
{OPD(0, PF_3A_66, 0x40), 1, &OpDispatchBuilder::DPPOp<OpSize::i32Bit>},
{OPD(0, PF_3A_66, 0x41), 1, &OpDispatchBuilder::DPPOp<OpSize::i64Bit>},
{OPD(0, PF_3A_66, 0x42), 1, &OpDispatchBuilder::MPSADBWOp},
{OPD(0, PF_3A_66, 0x60), 1, &OpDispatchBuilder::VPCMPESTRMOp},
{OPD(0, PF_3A_66, 0x61), 1, &OpDispatchBuilder::VPCMPESTRIOp},
{OPD(0, PF_3A_66, 0x62), 1, &OpDispatchBuilder::VPCMPISTRMOp},
{OPD(0, PF_3A_66, 0x63), 1, &OpDispatchBuilder::VPCMPISTRIOp},
{OPD(0, PF_3A_NONE, 0xCC), 1, &OpDispatchBuilder::SHA1RNDS4Op},
};
constexpr std::tuple<uint16_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_H0F3ATable_64[] = {
{OPD(1, PF_3A_66, 0x0F), 1, &OpDispatchBuilder::PAlignrOp},
{OPD(1, PF_3A_66, 0x16), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PExtrOp, OpSize::i64Bit>},
{OPD(1, PF_3A_66, 0x22), 1, &OpDispatchBuilder::PINSROp<OpSize::i64Bit>},
};
#undef PF_3A_NONE
#undef PF_3A_66
#undef OPD
} // namespace FEXCore::IR
@@ -0,0 +1,129 @@
// SPDX-License-Identifier: MIT
#pragma once
#include "Interface/Core/OpcodeDispatcher.h"
namespace FEXCore::IR {
using X86Tables::OpToIndex;
#define OPD(group, prefix, Reg) (((group - FEXCore::X86Tables::TYPE_GROUP_1) << 6) | (prefix) << 3 | (Reg))
constexpr std::tuple<uint16_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_PrimaryGroupTables[] = {
// GROUP 1
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x80), 0), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x80), 1), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x80), 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ADCOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x80), 3), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SBBOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x80), 4), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x80), 5), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x80), 6), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x80), 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::CMPOp, 1>}, // CMP
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x81), 0), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x81), 1), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x81), 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ADCOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x81), 3), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SBBOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x81), 4), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x81), 5), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x81), 6), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x81), 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::CMPOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x83), 0), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x83), 1), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x83), 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ADCOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x83), 3), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SBBOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x83), 4), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x83), 5), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x83), 6), 1, &OpDispatchBuilder::SecondaryALUOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_1, OpToIndex(0x83), 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::CMPOp, 1>},
// GROUP 2
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC0), 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, true, true, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC0), 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, false, true, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC0), 2), 1, &OpDispatchBuilder::RCLOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC0), 3), 1, &OpDispatchBuilder::RCROp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC0), 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHLImmediateOp, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC0), 5), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHRImmediateOp, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC0), 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHLImmediateOp, false>}, // SAL
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC0), 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ASHROp, true, false>}, // SAR
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC1), 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, true, true, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC1), 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, false, true, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC1), 2), 1, &OpDispatchBuilder::RCLOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC1), 3), 1, &OpDispatchBuilder::RCROp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC1), 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHLImmediateOp, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC1), 5), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHRImmediateOp, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC1), 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHLImmediateOp, false>}, // SAL
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xC1), 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ASHROp, true, false>}, // SAR
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD0), 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, true, true, true>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD0), 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, false, true, true>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD0), 2), 1, &OpDispatchBuilder::RCLOp1Bit},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD0), 3), 1, &OpDispatchBuilder::RCROp8x1Bit},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD0), 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHLImmediateOp, true>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD0), 5), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHRImmediateOp, true>}, // 1Bit SHR
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD0), 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHLImmediateOp, true>}, // SAL
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD0), 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ASHROp, true, true>}, // SAR
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD1), 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, true, true, true>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD1), 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, false, true, true>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD1), 2), 1, &OpDispatchBuilder::RCLOp1Bit},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD1), 3), 1, &OpDispatchBuilder::RCROp1Bit},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD1), 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHLImmediateOp, true>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD1), 5), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHRImmediateOp, true>}, // 1Bit SHR
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD1), 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHLImmediateOp, true>}, // SAL
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD1), 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ASHROp, true, true>}, // SAR
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD2), 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, true, false, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD2), 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, false, false, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD2), 2), 1, &OpDispatchBuilder::RCLSmallerOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD2), 3), 1, &OpDispatchBuilder::RCRSmallerOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD2), 4), 1, &OpDispatchBuilder::SHLOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD2), 5), 1, &OpDispatchBuilder::SHROp}, // SHR by CL
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD2), 6), 1, &OpDispatchBuilder::SHLOp}, // SAL
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD2), 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ASHROp, false, false>}, // SAR
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD3), 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, true, false, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD3), 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::RotateOp, false, false, false>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD3), 2), 1, &OpDispatchBuilder::RCLOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD3), 3), 1, &OpDispatchBuilder::RCROp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD3), 4), 1, &OpDispatchBuilder::SHLOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD3), 5), 1, &OpDispatchBuilder::SHROp}, // SHR by CL
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD3), 6), 1, &OpDispatchBuilder::SHLOp}, // SAL
{OPD(FEXCore::X86Tables::TYPE_GROUP_2, OpToIndex(0xD3), 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::ASHROp, false, false>}, // SAR
// GROUP 3
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF6), 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::TESTOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF6), 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::TESTOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF6), 2), 1, &OpDispatchBuilder::NOTOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF6), 3), 1, &OpDispatchBuilder::NEGOp}, // NEG
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF6), 4), 1, &OpDispatchBuilder::MULOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF6), 5), 1, &OpDispatchBuilder::IMULOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF6), 6), 1, &OpDispatchBuilder::DIVOp}, // DIV
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF6), 7), 1, &OpDispatchBuilder::IDIVOp}, // IDIV
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF7), 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::TESTOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF7), 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::TESTOp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF7), 2), 1, &OpDispatchBuilder::NOTOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF7), 3), 1, &OpDispatchBuilder::NEGOp}, // NEG
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF7), 4), 1, &OpDispatchBuilder::MULOp}, // MUL
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF7), 5), 1, &OpDispatchBuilder::IMULOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF7), 6), 1, &OpDispatchBuilder::DIVOp}, // DIV
{OPD(FEXCore::X86Tables::TYPE_GROUP_3, OpToIndex(0xF7), 7), 1, &OpDispatchBuilder::IDIVOp}, // IDIV
// GROUP 4
{OPD(FEXCore::X86Tables::TYPE_GROUP_4, OpToIndex(0xFE), 0), 1, &OpDispatchBuilder::INCOp}, // INC
{OPD(FEXCore::X86Tables::TYPE_GROUP_4, OpToIndex(0xFE), 1), 1, &OpDispatchBuilder::DECOp}, // DEC
// GROUP 5
{OPD(FEXCore::X86Tables::TYPE_GROUP_5, OpToIndex(0xFF), 0), 1, &OpDispatchBuilder::INCOp}, // INC
{OPD(FEXCore::X86Tables::TYPE_GROUP_5, OpToIndex(0xFF), 1), 1, &OpDispatchBuilder::DECOp}, // DEC
{OPD(FEXCore::X86Tables::TYPE_GROUP_5, OpToIndex(0xFF), 2), 1, &OpDispatchBuilder::CALLAbsoluteOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_5, OpToIndex(0xFF), 4), 1, &OpDispatchBuilder::JUMPAbsoluteOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_5, OpToIndex(0xFF), 6), 1, &OpDispatchBuilder::PUSHOp},
// GROUP 11
{OPD(FEXCore::X86Tables::TYPE_GROUP_11, OpToIndex(0xC6), 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVGPROp, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_11, OpToIndex(0xC7), 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVGPROp, 1>},
};
#undef OPD
} // namespace FEXCore::IR
@@ -0,0 +1,161 @@
// SPDX-License-Identifier: MIT
#pragma once
#include "Interface/Core/OpcodeDispatcher.h"
namespace FEXCore::IR {
#define OPD(group, prefix, Reg) (((group - FEXCore::X86Tables::TYPE_GROUP_6) << 5) | (prefix) << 3 | (Reg))
constexpr uint16_t PF_NONE = 0;
constexpr uint16_t PF_F3 = 1;
constexpr uint16_t PF_66 = 2;
constexpr uint16_t PF_F2 = 3;
constexpr std::tuple<uint16_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_SecondaryGroupTables[] = {
// GROUP 6
{OPD(FEXCore::X86Tables::TYPE_GROUP_6, PF_NONE, 3), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_6, PF_F3, 3), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_6, PF_66, 3), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_6, PF_F2, 3), 1, &OpDispatchBuilder::PermissionRestrictedOp},
// GROUP 7
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_NONE, 0), 1, &OpDispatchBuilder::SGDTOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_F3, 0), 1, &OpDispatchBuilder::SGDTOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_66, 0), 1, &OpDispatchBuilder::SGDTOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_F2, 0), 1, &OpDispatchBuilder::SGDTOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_NONE, 3), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_F3, 3), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_66, 3), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_F2, 3), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_NONE, 4), 1, &OpDispatchBuilder::SMSWOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_F3, 4), 1, &OpDispatchBuilder::SMSWOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_66, 4), 1, &OpDispatchBuilder::SMSWOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_F2, 4), 1, &OpDispatchBuilder::SMSWOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_NONE, 6), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_F3, 6), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_66, 6), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_7, PF_F2, 6), 1, &OpDispatchBuilder::PermissionRestrictedOp},
// GROUP 8
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_NONE, 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTNone>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_F3, 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTNone>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_66, 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTNone>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_F2, 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTNone>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_NONE, 5), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTSet>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_F3, 5), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTSet>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_66, 5), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTSet>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_F2, 5), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTSet>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_NONE, 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTClear>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_F3, 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTClear>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_66, 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTClear>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_F2, 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTClear>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_NONE, 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTComplement>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_F3, 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTComplement>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_66, 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTComplement>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_8, PF_F2, 7), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 1, BTAction::BTComplement>},
// GROUP 9
{OPD(FEXCore::X86Tables::TYPE_GROUP_9, PF_NONE, 1), 1, &OpDispatchBuilder::CMPXCHGPairOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_9, PF_F3, 1), 1, &OpDispatchBuilder::CMPXCHGPairOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_9, PF_66, 1), 1, &OpDispatchBuilder::CMPXCHGPairOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_9, PF_F2, 1), 1, &OpDispatchBuilder::CMPXCHGPairOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_9, PF_F3, 7), 1, &OpDispatchBuilder::RDPIDOp},
// GROUP 12
{OPD(FEXCore::X86Tables::TYPE_GROUP_12, PF_NONE, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLI, OpSize::i16Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_12, PF_NONE, 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRAIOp, OpSize::i16Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_12, PF_NONE, 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLLI, OpSize::i16Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_12, PF_66, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLI, OpSize::i16Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_12, PF_66, 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRAIOp, OpSize::i16Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_12, PF_66, 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLLI, OpSize::i16Bit>},
// GROUP 13
{OPD(FEXCore::X86Tables::TYPE_GROUP_13, PF_NONE, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLI, OpSize::i32Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_13, PF_NONE, 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRAIOp, OpSize::i32Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_13, PF_NONE, 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLLI, OpSize::i32Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_13, PF_66, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLI, OpSize::i32Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_13, PF_66, 4), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRAIOp, OpSize::i32Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_13, PF_66, 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLLI, OpSize::i32Bit>},
// GROUP 14
{OPD(FEXCore::X86Tables::TYPE_GROUP_14, PF_NONE, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLI, OpSize::i64Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_14, PF_NONE, 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLLI, OpSize::i64Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_14, PF_66, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLI, OpSize::i64Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_14, PF_66, 3), 1, &OpDispatchBuilder::PSRLDQ},
{OPD(FEXCore::X86Tables::TYPE_GROUP_14, PF_66, 6), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLLI, OpSize::i64Bit>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_14, PF_66, 7), 1, &OpDispatchBuilder::PSLLDQ},
// GROUP 15
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_NONE, 0), 1, &OpDispatchBuilder::FXSaveOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_NONE, 1), 1, &OpDispatchBuilder::FXRStoreOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_NONE, 2), 1, &OpDispatchBuilder::LDMXCSR},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_NONE, 3), 1, &OpDispatchBuilder::STMXCSR},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_NONE, 4), 1, &OpDispatchBuilder::XSaveOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_NONE, 5), 1, &OpDispatchBuilder::LoadFenceOrXRSTOR}, // LFENCE (or XRSTOR)
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_NONE, 6), 1, &OpDispatchBuilder::MemFenceOrXSAVEOPT}, // MFENCE (or XSAVEOPT)
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_NONE, 7), 1, &OpDispatchBuilder::StoreFenceOrCLFlush}, // SFENCE (or CLFLUSH)
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_F3, 5), 1, &OpDispatchBuilder::UnimplementedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_F3, 6), 1, &OpDispatchBuilder::UnimplementedOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_66, 6), 1, &OpDispatchBuilder::CLWB},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_66, 7), 1, &OpDispatchBuilder::CLFLUSHOPT},
// GROUP 16
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_NONE, 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, true, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_NONE, 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_NONE, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 2>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_NONE, 3), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 3>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_NONE, 4), 4, &OpDispatchBuilder::NOPOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_F3, 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, true, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_F3, 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_F3, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 2>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_F3, 3), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 3>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_F3, 4), 4, &OpDispatchBuilder::NOPOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_66, 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, true, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_66, 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_66, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 2>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_66, 3), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 3>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_66, 4), 4, &OpDispatchBuilder::NOPOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_F2, 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, true, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_F2, 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_F2, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 2>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_F2, 3), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 3>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_16, PF_F2, 4), 4, &OpDispatchBuilder::NOPOp},
// GROUP P
{OPD(FEXCore::X86Tables::TYPE_GROUP_P, PF_NONE, 0), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, false, false, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_P, PF_NONE, 1), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, true, false, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_P, PF_NONE, 2), 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Prefetch, true, false, 1>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_P, PF_NONE, 3), 5, &OpDispatchBuilder::NOPOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_P, PF_F3, 0), 8, &OpDispatchBuilder::NOPOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_P, PF_66, 0), 8, &OpDispatchBuilder::NOPOp},
{OPD(FEXCore::X86Tables::TYPE_GROUP_P, PF_F2, 0), 8, &OpDispatchBuilder::NOPOp},
};
constexpr std::tuple<uint16_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_SecondaryGroupTables_64[] = {
// GROUP 15
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_F3, 0), 1,
&OpDispatchBuilder::Bind<&OpDispatchBuilder::ReadSegmentReg, OpDispatchBuilder::Segment::FS>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_F3, 1), 1,
&OpDispatchBuilder::Bind<&OpDispatchBuilder::ReadSegmentReg, OpDispatchBuilder::Segment::GS>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_F3, 2), 1,
&OpDispatchBuilder::Bind<&OpDispatchBuilder::WriteSegmentReg, OpDispatchBuilder::Segment::FS>},
{OPD(FEXCore::X86Tables::TYPE_GROUP_15, PF_F3, 3), 1,
&OpDispatchBuilder::Bind<&OpDispatchBuilder::WriteSegmentReg, OpDispatchBuilder::Segment::GS>},
};
#undef OPD
} // namespace FEXCore::IR
@@ -0,0 +1,22 @@
// SPDX-License-Identifier: MIT
#pragma once
#include "Interface/Core/OpcodeDispatcher.h"
namespace FEXCore::IR {
constexpr std::tuple<uint8_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_SecondaryModRMTables[] = {
// REG /1
{((0 << 3) | 0), 1, &OpDispatchBuilder::UnimplementedOp},
{((0 << 3) | 1), 1, &OpDispatchBuilder::UnimplementedOp},
// REG /2
{((1 << 3) | 0), 1, &OpDispatchBuilder::XGetBVOp},
// REG /3
{((2 << 3) | 7), 1, &OpDispatchBuilder::PermissionRestrictedOp},
// REG /7
{((3 << 3) | 0), 1, &OpDispatchBuilder::PermissionRestrictedOp},
{((3 << 3) | 1), 1, &OpDispatchBuilder::RDTSCPOp},
};
} // namespace FEXCore::IR
@@ -0,0 +1,329 @@
// SPDX-License-Identifier: MIT
#pragma once
#include "Interface/Core/OpcodeDispatcher.h"
namespace FEXCore::IR {
constexpr std::tuple<uint8_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_TwoByteOpTable[] = {
// Instructions
{0x06, 1, &OpDispatchBuilder::PermissionRestrictedOp},
{0x07, 1, &OpDispatchBuilder::PermissionRestrictedOp},
{0x0B, 1, &OpDispatchBuilder::INTOp},
{0x0E, 1, &OpDispatchBuilder::X87EMMS},
{0x19, 7, &OpDispatchBuilder::NOPOp}, // NOP with ModRM
{0x20, 4, &OpDispatchBuilder::PermissionRestrictedOp},
{0x30, 1, &OpDispatchBuilder::PermissionRestrictedOp},
{0x31, 1, &OpDispatchBuilder::RDTSCOp},
{0x32, 2, &OpDispatchBuilder::PermissionRestrictedOp},
{0x34, 3, &OpDispatchBuilder::UnimplementedOp},
{0x3F, 1, &OpDispatchBuilder::ThunkOp},
{0x40, 16, &OpDispatchBuilder::CMOVOp},
{0x6E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVBetweenGPR_FPR, OpDispatchBuilder::VectorOpType::MMX>},
{0x6F, 1, &OpDispatchBuilder::MOVQMMXOp},
{0x7E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVBetweenGPR_FPR, OpDispatchBuilder::VectorOpType::MMX>},
{0x7F, 1, &OpDispatchBuilder::MOVQMMXOp},
{0x80, 16, &OpDispatchBuilder::CondJUMPOp},
{0x90, 16, &OpDispatchBuilder::SETccOp},
{0xA2, 1, &OpDispatchBuilder::CPUIDOp},
{0xA3, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 0, BTAction::BTNone>}, // BT
{0xA4, 1, &OpDispatchBuilder::SHLDImmediateOp},
{0xA5, 1, &OpDispatchBuilder::SHLDOp},
{0xAB, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 0, BTAction::BTSet>}, // BTS
{0xAC, 1, &OpDispatchBuilder::SHRDImmediateOp},
{0xAD, 1, &OpDispatchBuilder::SHRDOp},
{0xAF, 1, &OpDispatchBuilder::IMUL1SrcOp},
{0xB0, 2, &OpDispatchBuilder::CMPXCHGOp}, // CMPXCHG
{0xB3, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 0, BTAction::BTClear>}, // BTR
{0xB6, 2, &OpDispatchBuilder::MOVZXOp},
{0xBB, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::BTOp, 0, BTAction::BTComplement>}, // BTC
{0xBC, 1, &OpDispatchBuilder::BSFOp}, // BSF
{0xBD, 1, &OpDispatchBuilder::BSROp}, // BSF
{0xBE, 2, &OpDispatchBuilder::MOVSXOp},
{0xC0, 2, &OpDispatchBuilder::XADDOp},
{0xC3, 1, &OpDispatchBuilder::MOVGPRNTOp},
{0xC4, 1, &OpDispatchBuilder::PINSROp<OpSize::i16Bit>},
{0xC5, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PExtrOp, OpSize::i16Bit>},
{0xC8, 8, &OpDispatchBuilder::BSWAPOp},
// SSE
{0x10, 2, &OpDispatchBuilder::MOVVectorUnalignedOp},
{0x12, 2, &OpDispatchBuilder::MOVLPOp},
{0x14, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKLOp, OpSize::i32Bit>},
{0x15, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKHOp, OpSize::i32Bit>},
{0x16, 2, &OpDispatchBuilder::MOVHPDOp},
{0x28, 2, &OpDispatchBuilder::MOVVectorAlignedOp},
{0x2A, 1, &OpDispatchBuilder::InsertMMX_To_XMM_Vector_CVT_Int_To_Float},
{0x2B, 1, &OpDispatchBuilder::MOVVectorNTOp},
{0x2C, 1, &OpDispatchBuilder::XMM_To_MMX_Vector_CVT_Float_To_Int<OpSize::i32Bit, false, false>},
{0x2D, 1, &OpDispatchBuilder::XMM_To_MMX_Vector_CVT_Float_To_Int<OpSize::i32Bit, false, true>},
{0x2E, 2, &OpDispatchBuilder::UCOMISxOp<OpSize::i32Bit>},
{0x50, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVMSKOp, OpSize::i32Bit>},
{0x51, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VFSQRT, OpSize::i32Bit>},
{0x52, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VFRSQRT, OpSize::i32Bit>},
{0x53, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VFRECP, OpSize::i32Bit>},
{0x54, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VAND, OpSize::i128Bit>},
{0x55, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUROp, IR::OP_VANDN, OpSize::i64Bit>},
{0x56, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VOR, OpSize::i128Bit>},
{0x57, 1, &OpDispatchBuilder::VectorXOROp},
{0x58, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFADD, OpSize::i32Bit>},
{0x59, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFMUL, OpSize::i32Bit>},
{0x5A, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Vector_CVT_Float_To_Float, OpSize::i64Bit, OpSize::i32Bit, false>},
{0x5B, 1, &OpDispatchBuilder::Vector_CVT_Int_To_Float<OpSize::i32Bit, false>},
{0x5C, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFSUB, OpSize::i32Bit>},
{0x5D, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFMIN, OpSize::i32Bit>},
{0x5E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFDIV, OpSize::i32Bit>},
{0x5F, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFMAX, OpSize::i32Bit>},
{0x60, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKLOp, OpSize::i8Bit>},
{0x61, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKLOp, OpSize::i16Bit>},
{0x62, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKLOp, OpSize::i32Bit>},
{0x63, 1, &OpDispatchBuilder::PACKSSOp<OpSize::i16Bit>},
{0x64, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPGT, OpSize::i8Bit>},
{0x65, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPGT, OpSize::i16Bit>},
{0x66, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPGT, OpSize::i32Bit>},
{0x67, 1, &OpDispatchBuilder::PACKUSOp<OpSize::i16Bit>},
{0x68, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKHOp, OpSize::i8Bit>},
{0x69, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKHOp, OpSize::i16Bit>},
{0x6A, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKHOp, OpSize::i32Bit>},
{0x6B, 1, &OpDispatchBuilder::PACKSSOp<OpSize::i32Bit>},
{0x70, 1, &OpDispatchBuilder::PSHUFW8ByteOp},
{0x74, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPEQ, OpSize::i8Bit>},
{0x75, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPEQ, OpSize::i16Bit>},
{0x76, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPEQ, OpSize::i32Bit>},
{0x77, 1, &OpDispatchBuilder::X87EMMS},
{0xC2, 1, &OpDispatchBuilder::VFCMPOp<OpSize::i32Bit>},
{0xC6, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHUFOp, OpSize::i32Bit>},
{0xD1, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLDOp, OpSize::i16Bit>},
{0xD2, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLDOp, OpSize::i32Bit>},
{0xD3, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLDOp, OpSize::i64Bit>},
{0xD4, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADD, OpSize::i64Bit>},
{0xD5, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VMUL, OpSize::i16Bit>},
{0xD7, 1, &OpDispatchBuilder::MOVMSKOpOne}, // PMOVMSKB
{0xD8, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUQSUB, OpSize::i8Bit>},
{0xD9, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUQSUB, OpSize::i16Bit>},
{0xDA, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUMIN, OpSize::i8Bit>},
{0xDB, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VAND, OpSize::i64Bit>},
{0xDC, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUQADD, OpSize::i8Bit>},
{0xDD, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUQADD, OpSize::i16Bit>},
{0xDE, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUMAX, OpSize::i8Bit>},
{0xDF, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUROp, IR::OP_VANDN, OpSize::i64Bit>},
{0xE0, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VURAVG, OpSize::i8Bit>},
{0xE1, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRAOp, OpSize::i16Bit>},
{0xE2, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRAOp, OpSize::i32Bit>},
{0xE3, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VURAVG, OpSize::i16Bit>},
{0xE4, 1, &OpDispatchBuilder::PMULHW<false>},
{0xE5, 1, &OpDispatchBuilder::PMULHW<true>},
{0xE7, 1, &OpDispatchBuilder::MOVVectorNTOp},
{0xE8, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSQSUB, OpSize::i8Bit>},
{0xE9, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSQSUB, OpSize::i16Bit>},
{0xEA, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSMIN, OpSize::i16Bit>},
{0xEB, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VOR, OpSize::i64Bit>},
{0xEC, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSQADD, OpSize::i8Bit>},
{0xED, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSQADD, OpSize::i16Bit>},
{0xEE, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSMAX, OpSize::i16Bit>},
{0xEF, 1, &OpDispatchBuilder::VectorXOROp},
{0xF1, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLL, OpSize::i16Bit>},
{0xF2, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLL, OpSize::i32Bit>},
{0xF3, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLL, OpSize::i64Bit>},
{0xF4, 1, &OpDispatchBuilder::PMULLOp<OpSize::i32Bit, false>},
{0xF5, 1, &OpDispatchBuilder::PMADDWD},
{0xF6, 1, &OpDispatchBuilder::PSADBW},
{0xF7, 1, &OpDispatchBuilder::MASKMOVOp},
{0xF8, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSUB, OpSize::i8Bit>},
{0xF9, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSUB, OpSize::i16Bit>},
{0xFA, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSUB, OpSize::i32Bit>},
{0xFB, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSUB, OpSize::i64Bit>},
{0xFC, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADD, OpSize::i8Bit>},
{0xFD, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADD, OpSize::i16Bit>},
{0xFE, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADD, OpSize::i32Bit>},
// FEX reserved instructions
{0x37, 1, &OpDispatchBuilder::CallbackReturnOp},
};
constexpr std::tuple<uint8_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_SecondaryRepModTables[] = {
{0x10, 2, &OpDispatchBuilder::MOVSSOp},
{0x12, 1, &OpDispatchBuilder::VMOVSLDUPOp},
{0x16, 1, &OpDispatchBuilder::VMOVSHDUPOp},
{0x2A, 1, &OpDispatchBuilder::InsertCVTGPR_To_FPR<OpSize::i32Bit>},
{0x2B, 1, &OpDispatchBuilder::MOVVectorNTOp},
{0x2C, 1, &OpDispatchBuilder::CVTFPR_To_GPR<OpSize::i32Bit, false>},
{0x2D, 1, &OpDispatchBuilder::CVTFPR_To_GPR<OpSize::i32Bit, true>},
{0x51, 1, &OpDispatchBuilder::VectorScalarUnaryInsertALUOp<IR::OP_VFSQRTSCALARINSERT, OpSize::i32Bit>},
{0x52, 1, &OpDispatchBuilder::VectorScalarUnaryInsertALUOp<IR::OP_VFRSQRTSCALARINSERT, OpSize::i32Bit>},
{0x53, 1, &OpDispatchBuilder::VectorScalarUnaryInsertALUOp<IR::OP_VFRECPSCALARINSERT, OpSize::i32Bit>},
{0x58, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFADDSCALARINSERT, OpSize::i32Bit>},
{0x59, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFMULSCALARINSERT, OpSize::i32Bit>},
{0x5A, 1, &OpDispatchBuilder::InsertScalar_CVT_Float_To_Float<OpSize::i64Bit, OpSize::i32Bit>},
{0x5B, 1, &OpDispatchBuilder::Vector_CVT_Float_To_Int<OpSize::i32Bit, false, false>},
{0x5C, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFSUBSCALARINSERT, OpSize::i32Bit>},
{0x5D, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFMINSCALARINSERT, OpSize::i32Bit>},
{0x5E, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFDIVSCALARINSERT, OpSize::i32Bit>},
{0x5F, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFMAXSCALARINSERT, OpSize::i32Bit>},
{0x6F, 1, &OpDispatchBuilder::MOVVectorUnalignedOp},
{0x70, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSHUFWOp, false>},
{0x7E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVQOp, OpDispatchBuilder::VectorOpType::SSE>},
{0x7F, 1, &OpDispatchBuilder::MOVVectorUnalignedOp},
{0xB8, 1, &OpDispatchBuilder::PopcountOp},
{0xBC, 1, &OpDispatchBuilder::TZCNT},
{0xBD, 1, &OpDispatchBuilder::LZCNT},
{0xC2, 1, &OpDispatchBuilder::InsertScalarFCMPOp<OpSize::i32Bit>},
{0xD6, 1, &OpDispatchBuilder::MOVQ2DQ<true>},
{0xE6, 1, &OpDispatchBuilder::Vector_CVT_Int_To_Float<OpSize::i32Bit, true>},
};
constexpr std::tuple<uint8_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_SecondaryRepNEModTables[] = {
{0x10, 2, &OpDispatchBuilder::MOVSDOp},
{0x12, 1, &OpDispatchBuilder::MOVDDUPOp},
{0x2A, 1, &OpDispatchBuilder::InsertCVTGPR_To_FPR<OpSize::i64Bit>},
{0x2B, 1, &OpDispatchBuilder::MOVVectorNTOp},
{0x2C, 1, &OpDispatchBuilder::CVTFPR_To_GPR<OpSize::i64Bit, false>},
{0x2D, 1, &OpDispatchBuilder::CVTFPR_To_GPR<OpSize::i64Bit, true>},
{0x51, 1, &OpDispatchBuilder::VectorScalarUnaryInsertALUOp<IR::OP_VFSQRTSCALARINSERT, OpSize::i64Bit>},
// x52 = Invalid
{0x58, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFADDSCALARINSERT, OpSize::i64Bit>},
{0x59, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFMULSCALARINSERT, OpSize::i64Bit>},
{0x5A, 1, &OpDispatchBuilder::InsertScalar_CVT_Float_To_Float<OpSize::i32Bit, OpSize::i64Bit>},
{0x5C, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFSUBSCALARINSERT, OpSize::i64Bit>},
{0x5D, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFMINSCALARINSERT, OpSize::i64Bit>},
{0x5E, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFDIVSCALARINSERT, OpSize::i64Bit>},
{0x5F, 1, &OpDispatchBuilder::VectorScalarInsertALUOp<IR::OP_VFMAXSCALARINSERT, OpSize::i64Bit>},
{0x70, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSHUFWOp, true>},
{0x7C, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFADDP, OpSize::i32Bit>},
{0x7D, 1, &OpDispatchBuilder::HSUBP<OpSize::i32Bit>},
{0xD0, 1, &OpDispatchBuilder::ADDSUBPOp<OpSize::i32Bit>},
{0xD6, 1, &OpDispatchBuilder::MOVQ2DQ<false>},
{0xC2, 1, &OpDispatchBuilder::InsertScalarFCMPOp<OpSize::i64Bit>},
{0xE6, 1, &OpDispatchBuilder::Vector_CVT_Float_To_Int<OpSize::i64Bit, true, true>},
{0xF0, 1, &OpDispatchBuilder::MOVVectorUnalignedOp},
};
constexpr std::tuple<uint8_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_SecondaryOpSizeModTables[] = {
{0x10, 2, &OpDispatchBuilder::MOVVectorUnalignedOp},
{0x12, 2, &OpDispatchBuilder::MOVLPOp},
{0x14, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKLOp, OpSize::i64Bit>},
{0x15, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKHOp, OpSize::i64Bit>},
{0x16, 2, &OpDispatchBuilder::MOVHPDOp},
{0x28, 2, &OpDispatchBuilder::MOVVectorAlignedOp},
{0x2A, 1, &OpDispatchBuilder::MMX_To_XMM_Vector_CVT_Int_To_Float},
{0x2B, 1, &OpDispatchBuilder::MOVVectorNTOp},
{0x2C, 1, &OpDispatchBuilder::XMM_To_MMX_Vector_CVT_Float_To_Int<OpSize::i64Bit, true, false>},
{0x2D, 1, &OpDispatchBuilder::XMM_To_MMX_Vector_CVT_Float_To_Int<OpSize::i64Bit, true, true>},
{0x2E, 2, &OpDispatchBuilder::UCOMISxOp<OpSize::i64Bit>},
{0x50, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVMSKOp, OpSize::i64Bit>},
{0x51, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorUnaryOp, IR::OP_VFSQRT, OpSize::i64Bit>},
{0x54, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VAND, OpSize::i128Bit>},
{0x55, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUROp, IR::OP_VANDN, OpSize::i64Bit>},
{0x56, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VOR, OpSize::i128Bit>},
{0x57, 1, &OpDispatchBuilder::VectorXOROp},
{0x58, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFADD, OpSize::i64Bit>},
{0x59, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFMUL, OpSize::i64Bit>},
{0x5A, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::Vector_CVT_Float_To_Float, OpSize::i32Bit, OpSize::i64Bit, false>},
{0x5B, 1, &OpDispatchBuilder::Vector_CVT_Float_To_Int<OpSize::i32Bit, false, true>},
{0x5C, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFSUB, OpSize::i64Bit>},
{0x5D, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFMIN, OpSize::i64Bit>},
{0x5E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFDIV, OpSize::i64Bit>},
{0x5F, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFMAX, OpSize::i64Bit>},
{0x60, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKLOp, OpSize::i8Bit>},
{0x61, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKLOp, OpSize::i16Bit>},
{0x62, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKLOp, OpSize::i32Bit>},
{0x63, 1, &OpDispatchBuilder::PACKSSOp<OpSize::i16Bit>},
{0x64, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPGT, OpSize::i8Bit>},
{0x65, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPGT, OpSize::i16Bit>},
{0x66, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPGT, OpSize::i32Bit>},
{0x67, 1, &OpDispatchBuilder::PACKUSOp<OpSize::i16Bit>},
{0x68, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKHOp, OpSize::i8Bit>},
{0x69, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKHOp, OpSize::i16Bit>},
{0x6A, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKHOp, OpSize::i32Bit>},
{0x6B, 1, &OpDispatchBuilder::PACKSSOp<OpSize::i32Bit>},
{0x6C, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKLOp, OpSize::i64Bit>},
{0x6D, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUNPCKHOp, OpSize::i64Bit>},
{0x6E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVBetweenGPR_FPR, OpDispatchBuilder::VectorOpType::SSE>},
{0x6F, 1, &OpDispatchBuilder::MOVVectorAlignedOp},
{0x70, 1, &OpDispatchBuilder::PSHUFDOp},
{0x74, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPEQ, OpSize::i8Bit>},
{0x75, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPEQ, OpSize::i16Bit>},
{0x76, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VCMPEQ, OpSize::i32Bit>},
{0x78, 1, nullptr}, // GROUP 17
{0x7C, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VFADDP, OpSize::i64Bit>},
{0x7D, 1, &OpDispatchBuilder::HSUBP<OpSize::i64Bit>},
{0x7E, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVBetweenGPR_FPR, OpDispatchBuilder::VectorOpType::SSE>},
{0x7F, 1, &OpDispatchBuilder::MOVVectorAlignedOp},
{0xC2, 1, &OpDispatchBuilder::VFCMPOp<OpSize::i64Bit>},
{0xC4, 1, &OpDispatchBuilder::PINSROp<OpSize::i16Bit>},
{0xC5, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PExtrOp, OpSize::i16Bit>},
{0xC6, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SHUFOp, OpSize::i64Bit>},
{0xD0, 1, &OpDispatchBuilder::ADDSUBPOp<OpSize::i64Bit>},
{0xD1, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLDOp, OpSize::i16Bit>},
{0xD2, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLDOp, OpSize::i32Bit>},
{0xD3, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRLDOp, OpSize::i64Bit>},
{0xD4, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADD, OpSize::i64Bit>},
{0xD5, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VMUL, OpSize::i16Bit>},
{0xD6, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::MOVQOp, OpDispatchBuilder::VectorOpType::SSE>},
{0xD7, 1, &OpDispatchBuilder::MOVMSKOpOne}, // PMOVMSKB
{0xD8, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUQSUB, OpSize::i8Bit>},
{0xD9, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUQSUB, OpSize::i16Bit>},
{0xDA, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUMIN, OpSize::i8Bit>},
{0xDB, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VAND, OpSize::i128Bit>},
{0xDC, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUQADD, OpSize::i8Bit>},
{0xDD, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUQADD, OpSize::i16Bit>},
{0xDE, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VUMAX, OpSize::i8Bit>},
{0xDF, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUROp, IR::OP_VANDN, OpSize::i64Bit>},
{0xE0, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VURAVG, OpSize::i8Bit>},
{0xE1, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRAOp, OpSize::i16Bit>},
{0xE2, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSRAOp, OpSize::i32Bit>},
{0xE3, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VURAVG, OpSize::i16Bit>},
{0xE4, 1, &OpDispatchBuilder::PMULHW<false>},
{0xE5, 1, &OpDispatchBuilder::PMULHW<true>},
{0xE6, 1, &OpDispatchBuilder::Vector_CVT_Float_To_Int<OpSize::i64Bit, true, false>},
{0xE7, 1, &OpDispatchBuilder::MOVVectorNTOp},
{0xE8, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSQSUB, OpSize::i8Bit>},
{0xE9, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSQSUB, OpSize::i16Bit>},
{0xEA, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSMIN, OpSize::i16Bit>},
{0xEB, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VOR, OpSize::i128Bit>},
{0xEC, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSQADD, OpSize::i8Bit>},
{0xED, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSQADD, OpSize::i16Bit>},
{0xEE, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSMAX, OpSize::i16Bit>},
{0xEF, 1, &OpDispatchBuilder::VectorXOROp},
{0xF1, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLL, OpSize::i16Bit>},
{0xF2, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLL, OpSize::i32Bit>},
{0xF3, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PSLL, OpSize::i64Bit>},
{0xF4, 1, &OpDispatchBuilder::PMULLOp<OpSize::i32Bit, false>},
{0xF5, 1, &OpDispatchBuilder::PMADDWD},
{0xF6, 1, &OpDispatchBuilder::PSADBW},
{0xF7, 1, &OpDispatchBuilder::MASKMOVOp},
{0xF8, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSUB, OpSize::i8Bit>},
{0xF9, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSUB, OpSize::i16Bit>},
{0xFA, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSUB, OpSize::i32Bit>},
{0xFB, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VSUB, OpSize::i64Bit>},
{0xFC, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADD, OpSize::i8Bit>},
{0xFD, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADD, OpSize::i16Bit>},
{0xFE, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::VectorALUOp, IR::OP_VADD, OpSize::i32Bit>},
};
constexpr std::tuple<uint8_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_TwoByteOpTable_64[] = {
{0x05, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::SyscallOp, true>},
{0xA0, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUSHSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_FS_PREFIX>},
{0xA1, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::POPSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_FS_PREFIX>},
{0xA8, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUSHSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_GS_PREFIX>},
{0xA9, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::POPSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_GS_PREFIX>},
};
constexpr std::tuple<uint8_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_TwoByteOpTable_32[] = {
{0x05, 1, &OpDispatchBuilder::NOPOp},
{0xA0, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUSHSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_FS_PREFIX>},
{0xA1, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::POPSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_FS_PREFIX>},
{0xA8, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::PUSHSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_GS_PREFIX>},
{0xA9, 1, &OpDispatchBuilder::Bind<&OpDispatchBuilder::POPSegmentOp, FEXCore::X86Tables::DecodeFlags::FLAG_GS_PREFIX>},
};
} // namespace FEXCore::IR
@@ -0,0 +1,26 @@
// SPDX-License-Identifier: MIT
#pragma once
#include "Interface/Core/OpcodeDispatcher.h"
namespace FEXCore::IR {
#define OPD(map_select, pp, opcode) (((map_select - 1) << 10) | (pp << 8) | (opcode))
constexpr std::tuple<uint16_t, uint8_t, FEXCore::X86Tables::OpDispatchPtr> OpDispatch_VEXTable[] = {
{OPD(2, 0b00, 0xF2), 1, &OpDispatchBuilder::ANDNBMIOp}, {OPD(2, 0b00, 0xF5), 1, &OpDispatchBuilder::BZHI},
{OPD(2, 0b10, 0xF5), 1, &OpDispatchBuilder::PEXT}, {OPD(2, 0b11, 0xF5), 1, &OpDispatchBuilder::PDEP},
{OPD(2, 0b11, 0xF6), 1, &OpDispatchBuilder::MULX}, {OPD(2, 0b00, 0xF7), 1, &OpDispatchBuilder::BEXTRBMIOp},
{OPD(2, 0b01, 0xF7), 1, &OpDispatchBuilder::BMI2Shift}, {OPD(2, 0b10, 0xF7), 1, &OpDispatchBuilder::BMI2Shift},
{OPD(2, 0b11, 0xF7), 1, &OpDispatchBuilder::BMI2Shift},
{OPD(3, 0b11, 0xF0), 1, &OpDispatchBuilder::RORX},
};
#undef OPD
#define OPD(group, pp, opcode) (((group - X86Tables::InstType::TYPE_VEX_GROUP_12) << 4) | (pp << 3) | (opcode))
constexpr std::tuple<uint8_t, uint8_t, X86Tables::OpDispatchPtr> OpDispatch_VEXGroupTable[] = {
{OPD(X86Tables::InstType::TYPE_VEX_GROUP_17, 0, 0b001), 1, &OpDispatchBuilder::BLSRBMIOp},
{OPD(X86Tables::InstType::TYPE_VEX_GROUP_17, 0, 0b010), 1, &OpDispatchBuilder::BLSMSKBMIOp},
{OPD(X86Tables::InstType::TYPE_VEX_GROUP_17, 0, 0b011), 1, &OpDispatchBuilder::BLSIBMIOp},
};
#undef OPD
} // namespace FEXCore::IR
File diff suppressed because it is too large. Load diff
@@ -26,7 +26,7 @@ class OrderedNode;
Ref OpDispatchBuilder::GetX87Top() {
// Yes, we are storing 3 bits in a single flag register.
// Deal with it
return _LoadContext(1, GPRClass, offsetof(FEXCore::Core::CPUState, flags) + FEXCore::X86State::X87FLAG_TOP_LOC);
return _LoadContext(OpSize::i8Bit, GPRClass, offsetof(FEXCore::Core::CPUState, flags) + FEXCore::X86State::X87FLAG_TOP_LOC);
}
Ref OpDispatchBuilder::GetX87Tag(Ref Value, Ref AbridgedFTW) {
@@ -56,17 +56,17 @@ void OpDispatchBuilder::SetX87FTW(Ref FTW) {
}
void OpDispatchBuilder::SetX87Top(Ref Value) {
_StoreContext(1, GPRClass, Value, offsetof(FEXCore::Core::CPUState, flags) + FEXCore::X86State::X87FLAG_TOP_LOC);
_StoreContext(OpSize::i8Bit, GPRClass, Value, offsetof(FEXCore::Core::CPUState, flags) + FEXCore::X86State::X87FLAG_TOP_LOC);
}
// Float LoaD operation with memory operand
void OpDispatchBuilder::FLD(OpcodeArgs, size_t Width) {
size_t ReadWidth = (Width == 80) ? 16 : Width / 8;
void OpDispatchBuilder::FLD(OpcodeArgs, IR::OpSize Width) {
const auto ReadWidth = (Width == OpSize::f80Bit) ? OpSize::i128Bit : Width;
Ref Data = LoadSource_WithOpSize(FPRClass, Op, Op->Src[0], ReadWidth, Op->Flags);
Ref Data = LoadSource_WithOpSize(FPRClass, Op, Op->Src[0], Width, Op->Flags);
Ref ConvertedData = Data;
// Convert to 80bit float
if (Width == 32 || Width == 64) {
if (Width == OpSize::i32Bit || Width == OpSize::i64Bit) {
ConvertedData = _F80CVTTo(Data, ReadWidth);
}
_PushStack(ConvertedData, Data, ReadWidth, true);
@@ -79,31 +79,31 @@ void OpDispatchBuilder::FLDFromStack(OpcodeArgs) {
void OpDispatchBuilder::FBLD(OpcodeArgs) {
// Read from memory
Ref Data = LoadSource_WithOpSize(FPRClass, Op, Op->Src[0], 16, Op->Flags);
Ref Data = LoadSource_WithOpSize(FPRClass, Op, Op->Src[0], OpSize::f80Bit, Op->Flags);
Ref ConvertedData = _F80BCDLoad(Data);
_PushStack(ConvertedData, Data, 16, true);
_PushStack(ConvertedData, Data, OpSize::i128Bit, true);
}
void OpDispatchBuilder::FBSTP(OpcodeArgs) {
Ref converted = _F80BCDStore(_ReadStackValue(0));
StoreResult_WithOpSize(FPRClass, Op, Op->Dest, converted, 10, 1);
StoreResult_WithOpSize(FPRClass, Op, Op->Dest, converted, OpSize::f80Bit, OpSize::i8Bit);
_PopStackDestroy();
}
void OpDispatchBuilder::FLD_Const(OpcodeArgs, NamedVectorConstant Constant) {
// Update TOP
Ref Data = LoadAndCacheNamedVectorConstant(16, Constant);
_PushStack(Data, Data, 16, true);
Ref Data = LoadAndCacheNamedVectorConstant(OpSize::i128Bit, Constant);
_PushStack(Data, Data, OpSize::i128Bit, true);
}
void OpDispatchBuilder::FILD(OpcodeArgs) {
size_t ReadWidth = GetSrcSize(Op);
const auto ReadWidth = OpSizeFromSrc(Op);
// Read from memory
Ref Data = LoadSource_WithOpSize(GPRClass, Op, Op->Src[0], ReadWidth, Op->Flags);
// Sign extend to 64bits
if (ReadWidth != 8) {
Data = _Sbfe(OpSize::i64Bit, ReadWidth * 8, 0, Data);
if (ReadWidth != OpSize::i64Bit) {
Data = _Sbfe(OpSize::i64Bit, IR::OpSizeAsBits(ReadWidth), 0, Data);
}
// We're about to clobber flags to grab the sign, so save NZCV.
@@ -123,14 +123,14 @@ void OpDispatchBuilder::FILD(OpcodeArgs) {
auto zeroed_exponent = _Select(COND_EQ, absolute, zero, zero, adjusted_exponent);
auto upper = _Or(OpSize::i64Bit, sign, zeroed_exponent);
Ref ConvertedData = _VCastFromGPR(16, 8, shifted);
ConvertedData = _VInsElement(16, 8, 1, 0, ConvertedData, _VCastFromGPR(16, 8, upper));
Ref ConvertedData = _VCastFromGPR(OpSize::i64Bit, OpSize::i64Bit, shifted);
ConvertedData = _VInsElement(OpSize::i128Bit, OpSize::i64Bit, 1, 0, ConvertedData, _VCastFromGPR(OpSize::i128Bit, OpSize::i64Bit, upper));
_PushStack(ConvertedData, Data, ReadWidth, false);
}
void OpDispatchBuilder::FST(OpcodeArgs, size_t Width) {
void OpDispatchBuilder::FST(OpcodeArgs, IR::OpSize Width) {
Ref Mem = LoadSource(GPRClass, Op, Op->Dest, Op->Flags, {.LoadData = false});
_StoreStackMemory(Mem, OpSize::i128Bit, true, Width / 8);
_StoreStackMemory(Mem, OpSize::i128Bit, true, Width);
if (Op->TableInfo->Flags & X86Tables::InstFlags::FLAGS_POP) {
_PopStackDestroy();
}
@@ -149,20 +149,18 @@ void OpDispatchBuilder::FSTToStack(OpcodeArgs) {
// Store integer to memory (possibly with truncation)
void OpDispatchBuilder::FIST(OpcodeArgs, bool Truncate) {
auto Size = GetSrcSize(Op);
// FIXME(pmatos): is there any advantage of using STORESTACKMEMORY here?
// Do we need STORESTACKMEMORY at all?
const auto Size = OpSizeFromSrc(Op);
Ref Data = _ReadStackValue(0);
Data = _F80CVTInt(Size, Data, Truncate);
StoreResult_WithOpSize(GPRClass, Op, Op->Dest, Data, Size, 1);
StoreResult_WithOpSize(GPRClass, Op, Op->Dest, Data, Size, OpSize::i8Bit);
if ((Op->TableInfo->Flags & X86Tables::InstFlags::FLAGS_POP) != 0) {
_PopStackDestroy();
}
}
void OpDispatchBuilder::FADD(OpcodeArgs, size_t Width, bool Integer, OpDispatchBuilder::OpResult ResInST0) {
void OpDispatchBuilder::FADD(OpcodeArgs, IR::OpSize Width, bool Integer, OpDispatchBuilder::OpResult ResInST0) {
if (Op->Src[0].IsNone()) { // Implicit argument case
auto Offset = Op->OP & 7;
auto St0 = 0;
@@ -177,23 +175,22 @@ void OpDispatchBuilder::FADD(OpcodeArgs, size_t Width, bool Integer, OpDispatchB
return;
}
LOGMAN_THROW_A_FMT(Width != OpSize::f80Bit, "No 80-bit floats from memory");
// We have one memory argument
Ref Arg {};
if (Width == 16 || Width == 32 || Width == 64) {
if (Integer) {
Arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
Arg = _F80CVTToInt(Arg, Width / 8);
} else {
Arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Arg = _F80CVTTo(Arg, Width / 8);
}
if (Integer) {
Arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
Arg = _F80CVTToInt(Arg, Width);
} else {
Arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Arg = _F80CVTTo(Arg, Width);
}
// top of stack is at offset zero
_F80AddValue(0, Arg);
}
void OpDispatchBuilder::FMUL(OpcodeArgs, size_t Width, bool Integer, OpDispatchBuilder::OpResult ResInST0) {
void OpDispatchBuilder::FMUL(OpcodeArgs, IR::OpSize Width, bool Integer, OpDispatchBuilder::OpResult ResInST0) {
if (Op->Src[0].IsNone()) { // Implicit argument case
auto offset = Op->OP & 7;
auto st0 = 0;
@@ -208,16 +205,15 @@ void OpDispatchBuilder::FMUL(OpcodeArgs, size_t Width, bool Integer, OpDispatchB
return;
}
LOGMAN_THROW_A_FMT(Width != OpSize::f80Bit, "No 80-bit floats from memory");
// We have one memory argument
Ref arg {};
if (Width == 16 || Width == 32 || Width == 64) {
if (Integer) {
arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
arg = _F80CVTToInt(arg, Width / 8);
} else {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
arg = _F80CVTTo(arg, Width / 8);
}
if (Integer) {
arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
arg = _F80CVTToInt(arg, Width);
} else {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
arg = _F80CVTTo(arg, Width);
}
// top of stack is at offset zero
@@ -228,7 +224,7 @@ void OpDispatchBuilder::FMUL(OpcodeArgs, size_t Width, bool Integer, OpDispatchB
}
}
void OpDispatchBuilder::FDIV(OpcodeArgs, size_t Width, bool Integer, bool Reverse, OpDispatchBuilder::OpResult ResInST0) {
void OpDispatchBuilder::FDIV(OpcodeArgs, IR::OpSize Width, bool Integer, bool Reverse, OpDispatchBuilder::OpResult ResInST0) {
if (Op->Src[0].IsNone()) {
const auto Offset = Op->OP & 7;
const auto St0 = 0;
@@ -246,16 +242,15 @@ void OpDispatchBuilder::FDIV(OpcodeArgs, size_t Width, bool Integer, bool Revers
return;
}
LOGMAN_THROW_A_FMT(Width != OpSize::f80Bit, "No 80-bit floats from memory");
// We have one memory argument
Ref arg {};
if (Width == 16 || Width == 32 || Width == 64) {
if (Integer) {
arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
arg = _F80CVTToInt(arg, Width / 8);
} else {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
arg = _F80CVTTo(arg, Width / 8);
}
if (Integer) {
arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
arg = _F80CVTToInt(arg, Width);
} else {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
arg = _F80CVTTo(arg, Width);
}
// top of stack is at offset zero
@@ -270,7 +265,7 @@ void OpDispatchBuilder::FDIV(OpcodeArgs, size_t Width, bool Integer, bool Revers
}
}
void OpDispatchBuilder::FSUB(OpcodeArgs, size_t Width, bool Integer, bool Reverse, OpDispatchBuilder::OpResult ResInST0) {
void OpDispatchBuilder::FSUB(OpcodeArgs, IR::OpSize Width, bool Integer, bool Reverse, OpDispatchBuilder::OpResult ResInST0) {
if (Op->Src[0].IsNone()) {
const auto Offset = Op->OP & 7;
const auto St0 = 0;
@@ -288,16 +283,15 @@ void OpDispatchBuilder::FSUB(OpcodeArgs, size_t Width, bool Integer, bool Revers
return;
}
LOGMAN_THROW_A_FMT(Width != OpSize::f80Bit, "No 80-bit floats from memory");
// We have one memory argument
Ref Arg {};
if (Width == 16 || Width == 32 || Width == 64) {
if (Integer) {
Arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
Arg = _F80CVTToInt(Arg, Width / 8);
} else {
Arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Arg = _F80CVTTo(Arg, Width / 8);
}
if (Integer) {
Arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
Arg = _F80CVTToInt(Arg, Width);
} else {
Arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Arg = _F80CVTTo(Arg, Width);
}
// top of stack is at offset zero
@@ -348,42 +342,42 @@ void OpDispatchBuilder::X87FNSTENV(OpcodeArgs) {
// Before we store anything we need to sync our stack to the registers.
_SyncStackToSlow();
auto Size = GetDstSize(Op);
const auto Size = OpSizeFromSrc(Op);
Ref Mem = LoadSource(GPRClass, Op, Op->Dest, Op->Flags, {.LoadData = false});
Mem = AppendSegmentOffset(Mem, Op->Flags);
{
auto FCW = _LoadContext(2, GPRClass, offsetof(FEXCore::Core::CPUState, FCW));
auto FCW = _LoadContext(OpSize::i16Bit, GPRClass, offsetof(FEXCore::Core::CPUState, FCW));
_StoreMem(GPRClass, Size, Mem, FCW, Size);
}
{ _StoreMem(GPRClass, Size, ReconstructFSW_Helper(), Mem, _Constant(Size * 1), Size, MEM_OFFSET_SXTX, 1); }
{ _StoreMem(GPRClass, Size, ReconstructFSW_Helper(), Mem, _Constant(IR::OpSizeToSize(Size) * 1), Size, MEM_OFFSET_SXTX, 1); }
auto ZeroConst = _Constant(0);
{
// FTW
_StoreMem(GPRClass, Size, GetX87FTW_Helper(), Mem, _Constant(Size * 2), Size, MEM_OFFSET_SXTX, 1);
_StoreMem(GPRClass, Size, GetX87FTW_Helper(), Mem, _Constant(IR::OpSizeToSize(Size) * 2), Size, MEM_OFFSET_SXTX, 1);
}
{
// Instruction Offset
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 3), Size, MEM_OFFSET_SXTX, 1);
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(IR::OpSizeToSize(Size) * 3), Size, MEM_OFFSET_SXTX, 1);
}
{
// Instruction CS selector (+ Opcode)
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 4), Size, MEM_OFFSET_SXTX, 1);
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(IR::OpSizeToSize(Size) * 4), Size, MEM_OFFSET_SXTX, 1);
}
{
// Data pointer offset
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 5), Size, MEM_OFFSET_SXTX, 1);
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(IR::OpSizeToSize(Size) * 5), Size, MEM_OFFSET_SXTX, 1);
}
{
// Data pointer selector
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 6), Size, MEM_OFFSET_SXTX, 1);
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(IR::OpSizeToSize(Size) * 6), Size, MEM_OFFSET_SXTX, 1);
}
}
@@ -406,26 +400,27 @@ Ref OpDispatchBuilder::ReconstructX87StateFromFSW_Helper(Ref FSW) {
void OpDispatchBuilder::X87LDENV(OpcodeArgs) {
_StackForceSlow();
auto Size = GetSrcSize(Op);
const auto Size = OpSizeFromSrc(Op);
Ref Mem = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags, {.LoadData = false});
Mem = AppendSegmentOffset(Mem, Op->Flags);
auto NewFCW = _LoadMem(GPRClass, 2, Mem, 2);
_StoreContext(2, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
auto NewFCW = _LoadMem(GPRClass, OpSize::i16Bit, Mem, OpSize::i16Bit);
_StoreContext(OpSize::i16Bit, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
Ref MemLocation = _Add(OpSize::i64Bit, Mem, _Constant(Size * 1));
Ref MemLocation = _Add(OpSize::i64Bit, Mem, _Constant(IR::OpSizeToSize(Size) * 1));
auto NewFSW = _LoadMem(GPRClass, Size, MemLocation, Size);
ReconstructX87StateFromFSW_Helper(NewFSW);
{
// FTW
Ref MemLocation = _Add(OpSize::i64Bit, Mem, _Constant(Size * 2));
Ref MemLocation = _Add(OpSize::i64Bit, Mem, _Constant(IR::OpSizeToSize(Size) * 2));
SetX87FTW(_LoadMem(GPRClass, Size, MemLocation, Size));
}
}
void OpDispatchBuilder::X87FNSAVE(OpcodeArgs) {
_SyncStackToSlow();
// 14 bytes for 16bit
// 2 Bytes : FCW
// 2 Bytes : FSW
@@ -444,60 +439,66 @@ void OpDispatchBuilder::X87FNSAVE(OpcodeArgs) {
// 2 bytes : Opcode
// 4 bytes : data pointer offset
// 4 bytes : data pointer selector
const auto Size = GetDstSize(Op);
const auto Size = OpSizeFromDst(Op);
Ref Mem = MakeSegmentAddress(Op, Op->Dest);
Ref Top = GetX87Top();
{
auto FCW = _LoadContext(2, GPRClass, offsetof(FEXCore::Core::CPUState, FCW));
auto FCW = _LoadContext(OpSize::i16Bit, GPRClass, offsetof(FEXCore::Core::CPUState, FCW));
_StoreMem(GPRClass, Size, Mem, FCW, Size);
}
{ _StoreMem(GPRClass, Size, ReconstructFSW_Helper(), Mem, _Constant(Size * 1), Size, MEM_OFFSET_SXTX, 1); }
{ _StoreMem(GPRClass, Size, ReconstructFSW_Helper(), Mem, _Constant(IR::OpSizeToSize(Size) * 1), Size, MEM_OFFSET_SXTX, 1); }
auto ZeroConst = _Constant(0);
{
// FTW
_StoreMem(GPRClass, Size, GetX87FTW_Helper(), Mem, _Constant(Size * 2), Size, MEM_OFFSET_SXTX, 1);
_StoreMem(GPRClass, Size, GetX87FTW_Helper(), Mem, _Constant(IR::OpSizeToSize(Size) * 2), Size, MEM_OFFSET_SXTX, 1);
}
{
// Instruction Offset
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 3), Size, MEM_OFFSET_SXTX, 1);
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(IR::OpSizeToSize(Size) * 3), Size, MEM_OFFSET_SXTX, 1);
}
{
// Instruction CS selector (+ Opcode)
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 4), Size, MEM_OFFSET_SXTX, 1);
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(IR::OpSizeToSize(Size) * 4), Size, MEM_OFFSET_SXTX, 1);
}
{
// Data pointer offset
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 5), Size, MEM_OFFSET_SXTX, 1);
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(IR::OpSizeToSize(Size) * 5), Size, MEM_OFFSET_SXTX, 1);
}
{
// Data pointer selector
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 6), Size, MEM_OFFSET_SXTX, 1);
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(IR::OpSizeToSize(Size) * 6), Size, MEM_OFFSET_SXTX, 1);
}
auto OneConst = _Constant(1);
auto SevenConst = _Constant(7);
const auto LoadSize = ReducedPrecisionMode ? OpSize::i64Bit : OpSize::i128Bit;
for (int i = 0; i < 7; ++i) {
auto data = _LoadContextIndexed(Top, 16, MMBaseOffset(), 16, FPRClass);
_StoreMem(FPRClass, 16, data, Mem, _Constant((Size * 7) + (10 * i)), 1, MEM_OFFSET_SXTX, 1);
Ref data = _LoadContextIndexed(Top, LoadSize, MMBaseOffset(), IR::OpSizeToSize(OpSize::i128Bit), FPRClass);
if (ReducedPrecisionMode) {
data = _F80CVTTo(data, OpSize::i64Bit);
}
_StoreMem(FPRClass, OpSize::i128Bit, data, Mem, _Constant((IR::OpSizeToSize(Size) * 7) + (10 * i)), OpSize::i8Bit, MEM_OFFSET_SXTX, 1);
Top = _And(OpSize::i32Bit, _Add(OpSize::i32Bit, Top, OneConst), SevenConst);
}
// The final st(7) needs a bit of special handling here
auto data = _LoadContextIndexed(Top, 16, MMBaseOffset(), 16, FPRClass);
Ref data = _LoadContextIndexed(Top, LoadSize, MMBaseOffset(), IR::OpSizeToSize(OpSize::i128Bit), FPRClass);
if (ReducedPrecisionMode) {
data = _F80CVTTo(data, OpSize::i64Bit);
}
// ST7 broken in to two parts
// Lower 64bits [63:0]
// upper 16 bits [79:64]
_StoreMem(FPRClass, 8, data, Mem, _Constant((Size * 7) + (7 * 10)), 1, MEM_OFFSET_SXTX, 1);
auto topBytes = _VDupElement(16, 2, data, 4);
_StoreMem(FPRClass, 2, topBytes, Mem, _Constant((Size * 7) + (7 * 10) + 8), 1, MEM_OFFSET_SXTX, 1);
_StoreMem(FPRClass, OpSize::i64Bit, data, Mem, _Constant((IR::OpSizeToSize(Size) * 7) + (7 * 10)), OpSize::i8Bit, MEM_OFFSET_SXTX, 1);
auto topBytes = _VDupElement(OpSize::i128Bit, OpSize::i16Bit, data, 4);
_StoreMem(FPRClass, OpSize::i16Bit, topBytes, Mem, _Constant((IR::OpSizeToSize(Size) * 7) + (7 * 10) + 8), OpSize::i8Bit, MEM_OFFSET_SXTX, 1);
// reset to default
FNINIT(Op);
@@ -505,17 +506,27 @@ void OpDispatchBuilder::X87FNSAVE(OpcodeArgs) {
void OpDispatchBuilder::X87FRSTOR(OpcodeArgs) {
_StackForceSlow();
const auto Size = GetSrcSize(Op);
const auto Size = OpSizeFromSrc(Op);
Ref Mem = MakeSegmentAddress(Op, Op->Src[0]);
auto NewFCW = _LoadMem(GPRClass, 2, Mem, 2);
_StoreContext(2, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
auto NewFCW = _LoadMem(GPRClass, OpSize::i16Bit, Mem, OpSize::i16Bit);
_StoreContext(OpSize::i16Bit, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
if (ReducedPrecisionMode) {
// ignore the rounding precision, we're always 64-bit in F64.
// extract rounding mode
Ref roundingMode = NewFCW;
auto roundShift = _Constant(10);
auto roundMask = _Constant(3);
roundingMode = _Lshr(OpSize::i32Bit, roundingMode, roundShift);
roundingMode = _And(OpSize::i32Bit, roundingMode, roundMask);
_SetRoundingMode(roundingMode, false, roundingMode);
}
auto NewFSW = _LoadMem(GPRClass, Size, Mem, _Constant(Size * 1), Size, MEM_OFFSET_SXTX, 1);
auto NewFSW = _LoadMem(GPRClass, Size, Mem, _Constant(IR::OpSizeToSize(Size) * 1), Size, MEM_OFFSET_SXTX, 1);
Ref Top = ReconstructX87StateFromFSW_Helper(NewFSW);
{
// FTW
SetX87FTW(_LoadMem(GPRClass, Size, Mem, _Constant(Size * 2), Size, MEM_OFFSET_SXTX, 1));
SetX87FTW(_LoadMem(GPRClass, Size, Mem, _Constant(IR::OpSizeToSize(Size) * 2), Size, MEM_OFFSET_SXTX, 1));
}
auto OneConst = _Constant(1);
@@ -523,15 +534,18 @@ void OpDispatchBuilder::X87FRSTOR(OpcodeArgs) {
auto low = _Constant(~0ULL);
auto high = _Constant(0xFFFF);
Ref Mask = _VCastFromGPR(16, 8, low);
Mask = _VInsGPR(16, 8, 1, Mask, high);
Ref Mask = _VCastFromGPR(OpSize::i128Bit, OpSize::i64Bit, low);
Mask = _VInsGPR(OpSize::i128Bit, OpSize::i64Bit, 1, Mask, high);
const auto StoreSize = ReducedPrecisionMode ? OpSize::i64Bit : OpSize::i128Bit;
for (int i = 0; i < 7; ++i) {
Ref Reg = _LoadMem(FPRClass, 16, Mem, _Constant((Size * 7) + (10 * i)), 1, MEM_OFFSET_SXTX, 1);
Ref Reg = _LoadMem(FPRClass, OpSize::i128Bit, Mem, _Constant((IR::OpSizeToSize(Size) * 7) + (10 * i)), OpSize::i8Bit, MEM_OFFSET_SXTX, 1);
// Mask off the top bits
Reg = _VAnd(16, 16, Reg, Mask);
_StoreContextIndexed(Reg, Top, 16, MMBaseOffset(), 16, FPRClass);
Reg = _VAnd(OpSize::i128Bit, OpSize::i128Bit, Reg, Mask);
if (ReducedPrecisionMode) {
// Convert to double precision
Reg = _F80CVT(OpSize::i64Bit, Reg);
}
_StoreContextIndexed(Reg, Top, StoreSize, MMBaseOffset(), IR::OpSizeToSize(OpSize::i128Bit), FPRClass);
Top = _And(OpSize::i32Bit, _Add(OpSize::i32Bit, Top, OneConst), SevenConst);
}
@@ -540,29 +554,31 @@ void OpDispatchBuilder::X87FRSTOR(OpcodeArgs) {
// ST7 broken in to two parts
// Lower 64bits [63:0]
// upper 16 bits [79:64]
Ref Reg = _LoadMem(FPRClass, 8, Mem, _Constant((Size * 7) + (10 * 7)), 1, MEM_OFFSET_SXTX, 1);
Ref RegHigh = _LoadMem(FPRClass, 2, Mem, _Constant((Size * 7) + (10 * 7) + 8), 1, MEM_OFFSET_SXTX, 1);
Reg = _VInsElement(16, 2, 4, 0, Reg, RegHigh);
_StoreContextIndexed(Reg, Top, 16, MMBaseOffset(), 16, FPRClass);
Ref Reg = _LoadMem(FPRClass, OpSize::i64Bit, Mem, _Constant((IR::OpSizeToSize(Size) * 7) + (10 * 7)), OpSize::i8Bit, MEM_OFFSET_SXTX, 1);
Ref RegHigh =
_LoadMem(FPRClass, OpSize::i16Bit, Mem, _Constant((IR::OpSizeToSize(Size) * 7) + (10 * 7) + 8), OpSize::i8Bit, MEM_OFFSET_SXTX, 1);
Reg = _VInsElement(OpSize::i128Bit, OpSize::i16Bit, 4, 0, Reg, RegHigh);
if (ReducedPrecisionMode) {
Reg = _F80CVT(OpSize::i64Bit, Reg); // Convert to double precision
}
_StoreContextIndexed(Reg, Top, StoreSize, MMBaseOffset(), IR::OpSizeToSize(OpSize::i128Bit), FPRClass);
}
// Load / Store Control Word
void OpDispatchBuilder::X87FSTCW(OpcodeArgs) {
auto FCW = _LoadContext(2, GPRClass, offsetof(FEXCore::Core::CPUState, FCW));
StoreResult(GPRClass, Op, FCW, -1);
auto FCW = _LoadContext(OpSize::i16Bit, GPRClass, offsetof(FEXCore::Core::CPUState, FCW));
StoreResult(GPRClass, Op, FCW, OpSize::iInvalid);
}
void OpDispatchBuilder::X87FLDCW(OpcodeArgs) {
// FIXME: Because loading control flags will affect several instructions in fast path, we might have
// to switch for now to slow mode whenever these are manually changed.
// Remove the next line and try DF_04.asm in fast path.
_StackForceSlow();
Ref NewFCW = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
_StoreContext(2, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
_StoreContext(OpSize::i16Bit, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
}
void OpDispatchBuilder::FXCH(OpcodeArgs) {
uint8_t Offset = Op->OP & 7;
// fxch st0, st0 is for us essentially a nop
@@ -575,15 +591,15 @@ void OpDispatchBuilder::FXCH(OpcodeArgs) {
void OpDispatchBuilder::X87FYL2X(OpcodeArgs, bool IsFYL2XP1) {
if (IsFYL2XP1) {
// create an add between top of stack and 1.
Ref One = ReducedPrecisionMode ? _VCastFromGPR(8, 8, _Constant(0x3FF0000000000000)) :
LoadAndCacheNamedVectorConstant(16, NamedVectorConstant::NAMED_VECTOR_X87_ONE);
Ref One = ReducedPrecisionMode ? _VCastFromGPR(OpSize::i64Bit, OpSize::i64Bit, _Constant(0x3FF0000000000000)) :
LoadAndCacheNamedVectorConstant(OpSize::i128Bit, NamedVectorConstant::NAMED_VECTOR_X87_ONE);
_F80AddValue(0, One);
}
_F80FYL2XStack();
}
void OpDispatchBuilder::FCOMI(OpcodeArgs, size_t Width, bool Integer, OpDispatchBuilder::FCOMIFlags WhichFlags, bool PopTwice) {
void OpDispatchBuilder::FCOMI(OpcodeArgs, IR::OpSize Width, bool Integer, OpDispatchBuilder::FCOMIFlags WhichFlags, bool PopTwice) {
Ref arg {};
Ref b {};
@@ -594,13 +610,13 @@ void OpDispatchBuilder::FCOMI(OpcodeArgs, size_t Width, bool Integer, OpDispatch
Res = _F80CmpStack(Offset);
} else {
// Memory arg
if (Width == 16 || Width == 32 || Width == 64) {
if (Width == OpSize::i16Bit || Width == OpSize::i32Bit || Width == OpSize::i64Bit) {
if (Integer) {
arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
b = _F80CVTToInt(arg, Width / 8);
b = _F80CVTToInt(arg, Width);
} else {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
b = _F80CVTTo(arg, Width / 8);
b = _F80CVTTo(arg, Width);
}
}
Res = _F80CmpValue(b);
@@ -618,11 +634,8 @@ void OpDispatchBuilder::FCOMI(OpcodeArgs, size_t Width, bool Integer, OpDispatch
SetRFLAG<FEXCore::X86State::X87FLAG_C2_LOC>(HostFlag_Unordered);
SetRFLAG<FEXCore::X86State::X87FLAG_C3_LOC>(HostFlag_ZF);
} else {
// Invalidate deferred flags early
// OF, SF, AF, PF all undefined
InvalidateDeferredFlags();
SetRFLAG<FEXCore::X86State::RFLAG_CF_RAW_LOC>(HostFlag_CF);
SetCFDirect(HostFlag_CF);
SetRFLAG<FEXCore::X86State::RFLAG_ZF_RAW_LOC>(HostFlag_ZF);
// PF is stored inverted, so invert from the host flag.
@@ -681,7 +694,6 @@ void OpDispatchBuilder::X87ModifySTP(OpcodeArgs, bool Inc) {
// Optionally we can pass a pre calculated value for Top, otherwise we calculate it
// during the function runtime.
Ref OpDispatchBuilder::ReconstructFSW_Helper(Ref T) {
// Start with the top value
auto Top = T ? T : GetX87Top();
Ref FSW = _Lshl(OpSize::i64Bit, Top, _Constant(11));
@@ -706,18 +718,21 @@ Ref OpDispatchBuilder::ReconstructFSW_Helper(Ref T) {
// There's no load Status Word instruction but you can load it through frstor
// or fldenv.
void OpDispatchBuilder::X87FNSTSW(OpcodeArgs) {
Ref TopValue = _SyncStackToSlow();
Ref StatusWord = ReconstructFSW_Helper(TopValue);
StoreResult(GPRClass, Op, StatusWord, -1);
StoreResult(GPRClass, Op, StatusWord, OpSize::iInvalid);
}
void OpDispatchBuilder::FNINIT(OpcodeArgs) {
auto Zero = _Constant(0);
if (ReducedPrecisionMode) {
_SetRoundingMode(Zero, false, Zero);
}
// Init FCW to 0x037F
auto NewFCW = _Constant(16, 0x037F);
_StoreContext(2, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
auto NewFCW = _Constant(OpSize::i16Bit, 0x037F);
_StoreContext(OpSize::i16Bit, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
// Set top to zero
SetX87Top(Zero);
@@ -782,13 +797,14 @@ void OpDispatchBuilder::X87FCMOV(OpcodeArgs) {
auto AllOneConst = _Constant(0xffff'ffff'ffff'ffffull);
Ref SrcCond = SelectCC(CC, OpSize::i64Bit, AllOneConst, ZeroConst);
Ref VecCond = _VDupFromGPR(16, 8, SrcCond);
_F80VBSLStack(16, VecCond, Op->OP & 7, 0);
Ref VecCond = _VDupFromGPR(OpSize::i128Bit, OpSize::i64Bit, SrcCond);
_F80VBSLStack(OpSize::i128Bit, VecCond, Op->OP & 7, 0);
}
void OpDispatchBuilder::X87FXAM(OpcodeArgs) {
auto a = _ReadStackValue(0);
Ref Result = ReducedPrecisionMode ? _VExtractToGPR(8, 8, a, 0) : _VExtractToGPR(16, 8, a, 1);
Ref Result =
ReducedPrecisionMode ? _VExtractToGPR(OpSize::i64Bit, OpSize::i64Bit, a, 0) : _VExtractToGPR(OpSize::i128Bit, OpSize::i64Bit, a, 1);
// Extract the sign bit
Result = ReducedPrecisionMode ? _Bfe(OpSize::i64Bit, 1, 63, Result) : _Bfe(OpSize::i64Bit, 1, 15, Result);
@@ -810,4 +826,14 @@ void OpDispatchBuilder::X87FXAM(OpcodeArgs) {
SetRFLAG<FEXCore::X86State::X87FLAG_C3_LOC>(C3);
}
void OpDispatchBuilder::X87FXTRACT(OpcodeArgs) {
auto Top = _ReadStackValue(0);
_PopStackDestroy();
auto Exp = _F80XTRACT_EXP(Top);
auto Sig = _F80XTRACT_SIG(Top);
_PushStack(Exp, Exp, OpSize::f80Bit, true);
_PushStack(Sig, Sig, OpSize::f80Bit, true);
}
} // namespace FEXCore::IR
@@ -8,6 +8,7 @@ $end_info$
#include "Interface/Core/OpcodeDispatcher.h"
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/IR/IR.h"
#include <FEXCore/Core/CoreState.h>
#include <FEXCore/Core/X86Enums.h>
@@ -22,58 +23,28 @@ class OrderedNode;
#define OpcodeArgs [[maybe_unused]] FEXCore::X86Tables::DecodedOp Op
// Functions in X87.cpp (no change required)
// GetX87Top
// SetX87ValidTag
// GetX87ValidTag
// GetX87Tag (will need changing once special tag handling is implemented)
// SetX87FTW
// GetX87FTW (will need changing once special tag handling is implemented)
// SetX87Top
// X87ModifySTP
// EMMS
// FFREE
// FNSTENV
// FSTCW
// LDSW
// FNSTSW
// FXCH
// FCMOV
// FST(register to register)
// FCHS
void OpDispatchBuilder::FNINITF64(OpcodeArgs) {
// Init host rounding mode to zero
auto Zero = _Constant(0);
_SetRoundingMode(Zero, false, Zero);
// Call generic version
FNINIT(Op);
}
void OpDispatchBuilder::X87LDENVF64(OpcodeArgs) {
_StackForceSlow();
const auto Size = GetSrcSize(Op);
const auto Size = OpSizeFromSrc(Op);
Ref Mem = MakeSegmentAddress(Op, Op->Src[0]);
auto NewFCW = _LoadMem(GPRClass, 2, Mem, 2);
auto NewFCW = _LoadMem(GPRClass, OpSize::i16Bit, Mem, OpSize::i16Bit);
// ignore the rounding precision, we're always 64-bit in F64.
// extract rounding mode
Ref roundingMode = _Bfe(OpSize::i32Bit, 3, 10, NewFCW);
_SetRoundingMode(roundingMode, false, roundingMode);
_StoreContext(2, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
_StoreContext(OpSize::i16Bit, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
auto NewFSW = _LoadMem(GPRClass, Size, Mem, _Constant(Size * 1), Size, MEM_OFFSET_SXTX, 1);
auto NewFSW = _LoadMem(GPRClass, Size, Mem, _Constant(IR::OpSizeToSize(Size)), Size, MEM_OFFSET_SXTX, 1);
ReconstructX87StateFromFSW_Helper(NewFSW);
{
// FTW
SetX87FTW(_LoadMem(GPRClass, Size, Mem, _Constant(Size * 2), Size, MEM_OFFSET_SXTX, 1));
SetX87FTW(_LoadMem(GPRClass, Size, Mem, _Constant(IR::OpSizeToSize(Size) * 2), Size, MEM_OFFSET_SXTX, 1));
}
}
void OpDispatchBuilder::X87FLDCWF64(OpcodeArgs) {
_StackForceSlow();
@@ -82,59 +53,59 @@ void OpDispatchBuilder::X87FLDCWF64(OpcodeArgs) {
// extract rounding mode
Ref roundingMode = _Bfe(OpSize::i32Bit, 3, 10, NewFCW);
_SetRoundingMode(roundingMode, false, roundingMode);
_StoreContext(2, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
_StoreContext(OpSize::i16Bit, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
}
// F64 ops
// Float load op with memory operand
void OpDispatchBuilder::FLDF64(OpcodeArgs, size_t Width) {
size_t ReadWidth = (Width == 80) ? 16 : Width / 8;
void OpDispatchBuilder::FLDF64(OpcodeArgs, IR::OpSize Width) {
const auto ReadWidth = (Width == OpSize::f80Bit) ? OpSize::i128Bit : Width;
Ref Data = LoadSource_WithOpSize(FPRClass, Op, Op->Src[0], ReadWidth, Op->Flags);
// Convert to 64bit float
Ref ConvertedData = Data;
if (Width == 32) {
ConvertedData = _Float_FToF(8, 4, Data);
} else if (Width == 80) {
ConvertedData = _F80CVT(8, Data);
if (Width == OpSize::i32Bit) {
ConvertedData = _Float_FToF(OpSize::i64Bit, OpSize::i32Bit, Data);
} else if (Width == OpSize::f80Bit) {
ConvertedData = _F80CVT(OpSize::i64Bit, Data);
}
_PushStack(ConvertedData, Data, ReadWidth, true);
}
void OpDispatchBuilder::FBLDF64(OpcodeArgs) {
// Read from memory
Ref Data = LoadSource_WithOpSize(FPRClass, Op, Op->Src[0], 16, Op->Flags);
Ref Data = LoadSource_WithOpSize(FPRClass, Op, Op->Src[0], OpSize::i128Bit, Op->Flags);
Ref ConvertedData = _F80BCDLoad(Data);
ConvertedData = _F80CVT(8, ConvertedData);
_PushStack(ConvertedData, Data, 8, true);
ConvertedData = _F80CVT(OpSize::i64Bit, ConvertedData);
_PushStack(ConvertedData, Data, OpSize::i64Bit, true);
}
void OpDispatchBuilder::FBSTPF64(OpcodeArgs) {
Ref converted = _F80CVTTo(_ReadStackValue(0), 8);
Ref converted = _F80CVTTo(_ReadStackValue(0), OpSize::i64Bit);
converted = _F80BCDStore(converted);
StoreResult_WithOpSize(FPRClass, Op, Op->Dest, converted, 10, 1);
StoreResult_WithOpSize(FPRClass, Op, Op->Dest, converted, OpSize::f80Bit, OpSize::i8Bit);
_PopStackDestroy();
}
void OpDispatchBuilder::FLDF64_Const(OpcodeArgs, uint64_t Num) {
auto Data = _VCastFromGPR(8, 8, _Constant(Num));
_PushStack(Data, Data, 8, true);
auto Data = _VCastFromGPR(OpSize::i64Bit, OpSize::i64Bit, _Constant(Num));
_PushStack(Data, Data, OpSize::i64Bit, true);
}
void OpDispatchBuilder::FILDF64(OpcodeArgs) {
size_t ReadWidth = GetSrcSize(Op);
const auto ReadWidth = OpSizeFromSrc(Op);
// Read from memory
Ref Data = LoadSource_WithOpSize(GPRClass, Op, Op->Src[0], ReadWidth, Op->Flags);
if (ReadWidth == 2) {
Data = _Sbfe(OpSize::i64Bit, ReadWidth * 8, 0, Data);
if (ReadWidth == OpSize::i16Bit) {
Data = _Sbfe(OpSize::i64Bit, IR::OpSizeAsBits(ReadWidth), 0, Data);
}
auto ConvertedData = _Float_FromGPR_S(8, ReadWidth == 4 ? 4 : 8, Data);
auto ConvertedData = _Float_FromGPR_S(OpSize::i64Bit, ReadWidth == OpSize::i32Bit ? OpSize::i32Bit : OpSize::i64Bit, Data);
_PushStack(ConvertedData, Data, ReadWidth, false);
}
void OpDispatchBuilder::FSTF64(OpcodeArgs, size_t Width) {
void OpDispatchBuilder::FSTF64(OpcodeArgs, IR::OpSize Width) {
Ref Mem = LoadSource(GPRClass, Op, Op->Dest, Op->Flags, {.LoadData = false});
_StoreStackMemory(Mem, OpSize::i64Bit, true, Width / 8);
_StoreStackMemory(Mem, OpSize::i64Bit, true, Width);
if (Op->TableInfo->Flags & X86Tables::InstFlags::FLAGS_POP) {
_PopStackDestroy();
@@ -142,22 +113,22 @@ void OpDispatchBuilder::FSTF64(OpcodeArgs, size_t Width) {
}
void OpDispatchBuilder::FISTF64(OpcodeArgs, bool Truncate) {
auto Size = GetSrcSize(Op);
const auto Size = OpSizeFromSrc(Op);
Ref data = _ReadStackValue(0);
if (Truncate) {
data = _Float_ToGPR_ZS(Size == 4 ? 4 : 8, 8, data);
data = _Float_ToGPR_ZS(Size == OpSize::i32Bit ? OpSize::i32Bit : OpSize::i64Bit, OpSize::i64Bit, data);
} else {
data = _Float_ToGPR_S(Size == 4 ? 4 : 8, 8, data);
data = _Float_ToGPR_S(Size == OpSize::i32Bit ? OpSize::i32Bit : OpSize::i64Bit, OpSize::i64Bit, data);
}
StoreResult_WithOpSize(GPRClass, Op, Op->Dest, data, Size, 1);
StoreResult_WithOpSize(GPRClass, Op, Op->Dest, data, Size, OpSize::i8Bit);
if ((Op->TableInfo->Flags & X86Tables::InstFlags::FLAGS_POP) != 0) {
_PopStackDestroy();
}
}
void OpDispatchBuilder::FADDF64(OpcodeArgs, size_t Width, bool Integer, OpDispatchBuilder::OpResult ResInST0) {
void OpDispatchBuilder::FADDF64(OpcodeArgs, IR::OpSize Width, bool Integer, OpDispatchBuilder::OpResult ResInST0) {
if (Op->Src[0].IsNone()) { // Implicit argument case
auto Offset = Op->OP & 7;
auto St0 = 0;
@@ -177,14 +148,14 @@ void OpDispatchBuilder::FADDF64(OpcodeArgs, size_t Width, bool Integer, OpDispat
if (Integer) {
arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
if (Width == 16) {
if (Width == OpSize::i16Bit) {
arg = _Sbfe(OpSize::i64Bit, 16, 0, arg);
}
arg = _Float_FromGPR_S(8, Width == 64 ? 8 : 4, arg);
} else if (Width == 32) {
arg = _Float_FromGPR_S(OpSize::i64Bit, Width == OpSize::i64Bit ? OpSize::i64Bit : OpSize::i32Bit, arg);
} else if (Width == OpSize::i32Bit) {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
arg = _Float_FToF(8, 4, arg);
} else if (Width == 64) {
arg = _Float_FToF(OpSize::i64Bit, OpSize::i32Bit, arg);
} else if (Width == OpSize::i64Bit) {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
}
@@ -193,7 +164,7 @@ void OpDispatchBuilder::FADDF64(OpcodeArgs, size_t Width, bool Integer, OpDispat
}
// FIXME: following is very similar to FADDF64
void OpDispatchBuilder::FMULF64(OpcodeArgs, size_t Width, bool Integer, OpDispatchBuilder::OpResult ResInST0) {
void OpDispatchBuilder::FMULF64(OpcodeArgs, IR::OpSize Width, bool Integer, OpDispatchBuilder::OpResult ResInST0) {
if (Op->Src[0].IsNone()) { // Implicit argument case
auto offset = Op->OP & 7;
auto st0 = 0;
@@ -213,14 +184,14 @@ void OpDispatchBuilder::FMULF64(OpcodeArgs, size_t Width, bool Integer, OpDispat
if (Integer) {
arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
if (Width == 16) {
if (Width == OpSize::i16Bit) {
arg = _Sbfe(OpSize::i64Bit, 16, 0, arg);
}
arg = _Float_FromGPR_S(8, Width == 64 ? 8 : 4, arg);
} else if (Width == 32) {
arg = _Float_FromGPR_S(OpSize::i64Bit, Width == OpSize::i64Bit ? OpSize::i64Bit : OpSize::i32Bit, arg);
} else if (Width == OpSize::i32Bit) {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
arg = _Float_FToF(8, 4, arg);
} else if (Width == 64) {
arg = _Float_FToF(OpSize::i64Bit, OpSize::i32Bit, arg);
} else if (Width == OpSize::i64Bit) {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
}
@@ -232,7 +203,7 @@ void OpDispatchBuilder::FMULF64(OpcodeArgs, size_t Width, bool Integer, OpDispat
}
}
void OpDispatchBuilder::FDIVF64(OpcodeArgs, size_t Width, bool Integer, bool Reverse, OpDispatchBuilder::OpResult ResInST0) {
void OpDispatchBuilder::FDIVF64(OpcodeArgs, IR::OpSize Width, bool Integer, bool Reverse, OpDispatchBuilder::OpResult ResInST0) {
if (Op->Src[0].IsNone()) {
const auto offset = Op->OP & 7;
const auto st0 = 0;
@@ -260,17 +231,17 @@ void OpDispatchBuilder::FDIVF64(OpcodeArgs, size_t Width, bool Integer, bool Rev
// We have one memory argument
Ref Arg {};
if (Width == 16 || Width == 32 || Width == 64) {
if (Width == OpSize::i16Bit || Width == OpSize::i32Bit || Width == OpSize::i64Bit) {
if (Integer) {
Arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
if (Width == 16) {
if (Width == OpSize::i16Bit) {
Arg = _Sbfe(OpSize::i64Bit, 16, 0, Arg);
}
Arg = _Float_FromGPR_S(8, Width == 64 ? 8 : 4, Arg);
} else if (Width == 32) {
Arg = _Float_FromGPR_S(OpSize::i64Bit, Width == OpSize::i64Bit ? OpSize::i64Bit : OpSize::i32Bit, Arg);
} else if (Width == OpSize::i32Bit) {
Arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
Arg = _Float_FToF(8, 4, Arg);
} else if (Width == 64) {
Arg = _Float_FToF(OpSize::i64Bit, OpSize::i32Bit, Arg);
} else if (Width == OpSize::i64Bit) {
Arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
}
}
@@ -287,7 +258,7 @@ void OpDispatchBuilder::FDIVF64(OpcodeArgs, size_t Width, bool Integer, bool Rev
}
}
void OpDispatchBuilder::FSUBF64(OpcodeArgs, size_t Width, bool Integer, bool Reverse, OpDispatchBuilder::OpResult ResInST0) {
void OpDispatchBuilder::FSUBF64(OpcodeArgs, IR::OpSize Width, bool Integer, bool Reverse, OpDispatchBuilder::OpResult ResInST0) {
if (Op->Src[0].IsNone()) {
const auto Offset = Op->OP & 7;
const auto St0 = 0;
@@ -315,17 +286,17 @@ void OpDispatchBuilder::FSUBF64(OpcodeArgs, size_t Width, bool Integer, bool Rev
// We have one memory argument
Ref arg {};
if (Width == 16 || Width == 32 || Width == 64) {
if (Width == OpSize::i16Bit || Width == OpSize::i32Bit || Width == OpSize::i64Bit) {
if (Integer) {
arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
if (Width == 16) {
if (Width == OpSize::i16Bit) {
arg = _Sbfe(OpSize::i64Bit, 16, 0, arg);
}
arg = _Float_FromGPR_S(8, Width == 64 ? 8 : 4, arg);
} else if (Width == 32) {
arg = _Float_FromGPR_S(OpSize::i64Bit, Width == OpSize::i64Bit ? OpSize::i64Bit : OpSize::i32Bit, arg);
} else if (Width == OpSize::i32Bit) {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
arg = _Float_FToF(8, 4, arg);
} else if (Width == 64) {
arg = _Float_FToF(OpSize::i64Bit, OpSize::i32Bit, arg);
} else if (Width == OpSize::i64Bit) {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
}
}
@@ -348,11 +319,10 @@ void OpDispatchBuilder::FTSTF64(OpcodeArgs) {
// Now we do our comparison.
_F80StackTest(0);
PossiblySetNZCVBits = ~0;
ConvertNZCVToX87();
}
void OpDispatchBuilder::FCOMIF64(OpcodeArgs, size_t Width, bool Integer, OpDispatchBuilder::FCOMIFlags WhichFlags, bool PopTwice) {
void OpDispatchBuilder::FCOMIF64(OpcodeArgs, IR::OpSize Width, bool Integer, OpDispatchBuilder::FCOMIFlags WhichFlags, bool PopTwice) {
Ref arg {};
Ref b {};
@@ -362,17 +332,17 @@ void OpDispatchBuilder::FCOMIF64(OpcodeArgs, size_t Width, bool Integer, OpDispa
b = _ReadStackValue(offset);
} else {
// Memory arg
if (Width == 16 || Width == 32 || Width == 64) {
if (Width == OpSize::i16Bit || Width == OpSize::i32Bit || Width == OpSize::i64Bit) {
if (Integer) {
arg = LoadSource(GPRClass, Op, Op->Src[0], Op->Flags);
if (Width == 16) {
if (Width == OpSize::i16Bit) {
arg = _Sbfe(OpSize::i64Bit, 16, 0, arg);
}
b = _Float_FromGPR_S(8, Width == 64 ? 8 : 4, arg);
} else if (Width == 32) {
b = _Float_FromGPR_S(OpSize::i64Bit, Width == OpSize::i64Bit ? OpSize::i64Bit : OpSize::i32Bit, arg);
} else if (Width == OpSize::i32Bit) {
arg = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
b = _Float_FToF(8, 4, arg);
} else if (Width == 64) {
b = _Float_FToF(OpSize::i64Bit, OpSize::i32Bit, arg);
} else if (Width == OpSize::i64Bit) {
b = LoadSource(FPRClass, Op, Op->Src[0], Op->Flags);
}
}
@@ -383,7 +353,6 @@ void OpDispatchBuilder::FCOMIF64(OpcodeArgs, size_t Width, bool Integer, OpDispa
GetNZCV();
_F80CmpValue(b);
PossiblySetNZCVBits = ~0;
ConvertNZCVToX87();
} else {
HandleNZCVWrite();
@@ -399,144 +368,37 @@ void OpDispatchBuilder::FCOMIF64(OpcodeArgs, size_t Width, bool Integer, OpDispa
}
}
// This function converts to F80 on save for compatibility
void OpDispatchBuilder::X87FNSAVEF64(OpcodeArgs) {
_SyncStackToSlow();
// 14 bytes for 16bit
// 2 Bytes : FCW
// 2 Bytes : FSW
// 2 bytes : FTW
// 2 bytes : Instruction offset
// 2 bytes : Instruction CS selector
// 2 bytes : Data offset
// 2 bytes : Data selector
void OpDispatchBuilder::X87FXTRACTF64(OpcodeArgs) {
// Split node into SIG and EXP while handling the special zero case.
// i.e. if val == 0.0, then sig = 0.0, exp = -inf
// if val == -0.0, then sig = -0.0, exp = -inf
// otherwise we just extract the 64-bit sig and exp as normal.
Ref Node = _ReadStackValue(0);
// 28 bytes for 32bit
// 4 bytes : FCW
// 4 bytes : FSW
// 4 bytes : FTW
// 4 bytes : Instruction pointer
// 2 bytes : instruction pointer selector
// 2 bytes : Opcode
// 4 bytes : data pointer offset
// 4 bytes : data pointer selector
Ref Gpr = _VExtractToGPR(OpSize::i64Bit, OpSize::i64Bit, Node, 0);
const auto Size = GetDstSize(Op);
Ref Mem = MakeSegmentAddress(Op, Op->Dest);
Ref Top = GetX87Top();
{
auto FCW = _LoadContext(2, GPRClass, offsetof(FEXCore::Core::CPUState, FCW));
_StoreMem(GPRClass, Size, Mem, FCW, Size);
}
// zero case
Ref ExpZV = _VCastFromGPR(OpSize::i64Bit, OpSize::i64Bit, _Constant(0xfff0'0000'0000'0000UL));
Ref SigZV = Node;
{ _StoreMem(GPRClass, Size, ReconstructFSW_Helper(), Mem, _Constant(Size * 1), Size, MEM_OFFSET_SXTX, 1); }
// non zero case
Ref ExpNZ = _Bfe(OpSize::i64Bit, 11, 52, Gpr);
ExpNZ = _Sub(OpSize::i64Bit, ExpNZ, _Constant(1023));
Ref ExpNZV = _Float_FromGPR_S(OpSize::i64Bit, OpSize::i64Bit, ExpNZ);
auto ZeroConst = _Constant(0);
Ref SigNZ = _And(OpSize::i64Bit, Gpr, _Constant(0x800f'ffff'ffff'ffffLL));
SigNZ = _Or(OpSize::i64Bit, SigNZ, _Constant(0x3ff0'0000'0000'0000LL));
Ref SigNZV = _VCastFromGPR(OpSize::i64Bit, OpSize::i64Bit, SigNZ);
{
// FTW
_StoreMem(GPRClass, Size, GetX87FTW_Helper(), Mem, _Constant(Size * 2), Size, MEM_OFFSET_SXTX, 1);
}
// Comparison and select to push onto stack
SaveNZCV();
_TestNZ(OpSize::i64Bit, Gpr, _Constant(0x7fff'ffff'ffff'ffffUL));
{
// Instruction Offset
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 3), Size, MEM_OFFSET_SXTX, 1);
}
Ref Sig = _NZCVSelectV(OpSize::i64Bit, {COND_EQ}, SigZV, SigNZV);
Ref Exp = _NZCVSelectV(OpSize::i64Bit, {COND_EQ}, ExpZV, ExpNZV);
{
// Instruction CS selector (+ Opcode)
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 4), Size, MEM_OFFSET_SXTX, 1);
}
{
// Data pointer offset
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 5), Size, MEM_OFFSET_SXTX, 1);
}
{
// Data pointer selector
_StoreMem(GPRClass, Size, ZeroConst, Mem, _Constant(Size * 6), Size, MEM_OFFSET_SXTX, 1);
}
auto OneConst = _Constant(1);
auto SevenConst = _Constant(7);
for (int i = 0; i < 7; ++i) {
Ref data = _LoadContextIndexed(Top, 8, MMBaseOffset(), 16, FPRClass);
data = _F80CVTTo(data, 8);
_StoreMem(FPRClass, 16, data, Mem, _Constant((Size * 7) + (i * 10)), 1, MEM_OFFSET_SXTX, 1);
Top = _And(OpSize::i32Bit, _Add(OpSize::i32Bit, Top, OneConst), SevenConst);
}
// The final st(7) needs a bit of special handling here
Ref data = _LoadContextIndexed(Top, 8, MMBaseOffset(), 16, FPRClass);
data = _F80CVTTo(data, 8);
// ST7 broken in to two parts
// Lower 64bits [63:0]
// upper 16 bits [79:64]
_StoreMem(FPRClass, 8, data, Mem, _Constant((Size * 7) + (7 * 10)), 1, MEM_OFFSET_SXTX, 1);
auto topBytes = _VDupElement(16, 2, data, 4);
_StoreMem(FPRClass, 2, topBytes, Mem, _Constant((Size * 7) + (7 * 10) + 8), 1, MEM_OFFSET_SXTX, 1);
// reset to default
FNINITF64(Op);
_PopStackDestroy();
_PushStack(Exp, Exp, OpSize::i64Bit, true);
_PushStack(Sig, Sig, OpSize::i64Bit, true);
}
// This function converts from F80 on load for compatibility
void OpDispatchBuilder::X87FRSTORF64(OpcodeArgs) {
_StackForceSlow();
const auto Size = GetSrcSize(Op);
Ref Mem = MakeSegmentAddress(Op, Op->Src[0]);
auto NewFCW = _LoadMem(GPRClass, 2, Mem, 2);
// ignore the rounding precision, we're always 64-bit in F64.
// extract rounding mode
Ref roundingMode = NewFCW;
auto roundShift = _Constant(10);
auto roundMask = _Constant(3);
roundingMode = _Lshr(OpSize::i32Bit, roundingMode, roundShift);
roundingMode = _And(OpSize::i32Bit, roundingMode, roundMask);
_SetRoundingMode(roundingMode, false, roundingMode);
_StoreContext(2, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
_StoreContext(2, GPRClass, NewFCW, offsetof(FEXCore::Core::CPUState, FCW));
auto NewFSW = _LoadMem(GPRClass, Size, Mem, _Constant(Size * 1), Size, MEM_OFFSET_SXTX, 1);
Ref Top = ReconstructX87StateFromFSW_Helper(NewFSW);
{
// FTW
SetX87FTW(_LoadMem(GPRClass, Size, Mem, _Constant(Size * 2), Size, MEM_OFFSET_SXTX, 1));
}
auto OneConst = _Constant(1);
auto SevenConst = _Constant(7);
auto low = _Constant(~0ULL);
auto high = _Constant(0xFFFF);
Ref Mask = _VCastFromGPR(16, 8, low);
Mask = _VInsGPR(16, 8, 1, Mask, high);
for (int i = 0; i < 7; ++i) {
Ref Reg = _LoadMem(FPRClass, 16, Mem, _Constant((Size * 7) + (i * 10)), 1, MEM_OFFSET_SXTX, 1);
// Mask off the top bits
Reg = _VAnd(16, 16, Reg, Mask);
// Convert to double precision
Reg = _F80CVT(8, Reg);
_StoreContextIndexed(Reg, Top, 8, MMBaseOffset(), 16, FPRClass);
Top = _And(OpSize::i32Bit, _Add(OpSize::i32Bit, Top, OneConst), SevenConst);
}
// The final st(7) needs a bit of special handling here
// ST7 broken in to two parts
// Lower 64bits [63:0]
// upper 16 bits [79:64]
Ref Reg = _LoadMem(FPRClass, 8, Mem, _Constant((Size * 7) + (7 * 10)), 1, MEM_OFFSET_SXTX, 1);
Ref RegHigh = _LoadMem(FPRClass, 2, Mem, _Constant((Size * 7) + (7 * 10) + 8), 1, MEM_OFFSET_SXTX, 1);
Reg = _VInsElement(16, 2, 4, 0, Reg, RegHigh);
Reg = _F80CVT(8, Reg); // Convert to double precision
_StoreContextIndexed(Reg, Top, 8, MMBaseOffset(), 16, FPRClass);
}
} // namespace FEXCore::IR
@@ -31,32 +31,9 @@ X86GeneratedCode::X86GeneratedCode() {
0x0F, 0x37, // CALLBACKRET FEX Instruction
};
// Signal return handlers need to be bit-exact to what the Linux kernel provides in VDSO.
// GDB and unwinding libraries key off of these instructions to understand if the stack frame is a signal frame or not.
// This two code sections match exactly what libSegFault expects.
//
// Typically this handlers are provided by the 32-bit VDSO thunk library, but that isn't available in all cases.
// Falling back to this generated code segment still allows a backtrace to work, just might not show
// the symbol as VDSO since there is no ELF to parse.
constexpr std::array<uint8_t, 9> sigreturn_32_code = {
0x58, // pop eax
0xb8, 0x77, 0x00, 0x00, 0x00, // mov eax, 0x77
0xcd, 0x80, // int 0x80
0x90, // nop
};
constexpr std::array<uint8_t, 7> rt_sigreturn_32_code = {
0xb8, 0xad, 0x00, 0x00, 0x00, // mov eax, 0xad
0xcd, 0x80, // int 0x80
};
CallbackReturn = reinterpret_cast<uint64_t>(CodePtr);
sigreturn_32 = CallbackReturn + SignalReturnCode.size();
rt_sigreturn_32 = sigreturn_32 + sigreturn_32_code.size();
memcpy(reinterpret_cast<void*>(CallbackReturn), &SignalReturnCode.at(0), SignalReturnCode.size());
memcpy(reinterpret_cast<void*>(sigreturn_32), &sigreturn_32_code.at(0), sigreturn_32_code.size());
memcpy(reinterpret_cast<void*>(rt_sigreturn_32), &rt_sigreturn_32_code.at(0), rt_sigreturn_32_code.size());
mprotect(CodePtr, CODE_SIZE, PROT_READ);
#endif
@@ -17,8 +17,6 @@ public:
~X86GeneratedCode();
uint64_t CallbackReturn {};
uint64_t sigreturn_32 {};
uint64_t rt_sigreturn_32 {};
private:
void* CodePtr {};
@@ -14,12 +14,14 @@ namespace FEXCore::X86Tables {
void InitializeBaseTables(Context::OperatingMode Mode);
void InitializeSecondaryTables(Context::OperatingMode Mode);
void InitializeSecondaryGroupTables(Context::OperatingMode Mode);
void InitializePrimaryGroupTables(Context::OperatingMode Mode);
void InitializeH0F3ATables(Context::OperatingMode Mode);
void InitializeInfoTables(Context::OperatingMode Mode) {
InitializeBaseTables(Mode);
InitializeSecondaryTables(Mode);
InitializeSecondaryGroupTables(Mode);
InitializePrimaryGroupTables(Mode);
InitializeH0F3ATables(Mode);
}
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/Core/OpcodeDispatcher/BaseTables.h"
#include <FEXCore/Core/Context.h>
@@ -236,6 +237,7 @@ std::array<X86InstInfo, MAX_PRIMARY_TABLE_SIZE> BaseOps = []() consteval {
};
GenerateTable(&Table.at(0), BaseOpTable, std::size(BaseOpTable));
IR::InstallToTable(Table, IR::OpDispatch_BaseOpTable);
return Table;
}();
@@ -301,9 +303,11 @@ void InitializeBaseTables(Context::OperatingMode Mode) {
if (Mode == Context::MODE_64BIT) {
GenerateTable(&BaseOps.at(0), BaseOpTable_64, std::size(BaseOpTable_64));
IR::InstallToTable(BaseOps, IR::OpDispatch_BaseOpTable_64);
}
else {
GenerateTable(&BaseOps.at(0), BaseOpTable_32, std::size(BaseOpTable_32));
IR::InstallToTable(BaseOps, IR::OpDispatch_BaseOpTable_32);
}
}
}
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/Core/OpcodeDispatcher/DDDTables.h"
#include <iterator>
@@ -54,6 +55,8 @@ std::array<X86InstInfo, MAX_3DNOW_TABLE_SIZE> DDDNowOps = []() consteval {
};
GenerateTable(&Table.at(0), DDDNowOpTable, std::size(DDDNowOpTable));
IR::InstallToTable(Table, IR::OpDispatch_DDDTable);
return Table;
}();
@@ -1,37 +0,0 @@
// SPDX-License-Identifier: MIT
/*
$info$
tags: frontend|x86-tables
$end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include <iterator>
namespace FEXCore::X86Tables {
using namespace InstFlags;
std::array<X86InstInfo, MAX_EVEX_TABLE_SIZE> EVEXTableOps = []() consteval {
std::array<X86InstInfo, MAX_EVEX_TABLE_SIZE> Table{};
constexpr U16U8InfoStruct EVEXTable[] = {
{0x10, 1, X86InstInfo{"VMOVUPS", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x11, 1, X86InstInfo{"VMOVUPS", TYPE_INST, FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x18, 1, X86InstInfo{"VBROADCASTSS", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x19, 1, X86InstInfo{"VBROADCASTD", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x1A, 1, X86InstInfo{"VBROADCASTSD", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x1B, 1, X86InstInfo{"VBROADCASTF64X4", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x28, 1, X86InstInfo{"VMOVAPS", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x29, 1, X86InstInfo{"VMOVAPS", TYPE_INST, FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x59, 1, X86InstInfo{"VBROADCASTQ", TYPE_INST, FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x6F, 1, X86InstInfo{"VMOVDQU64", TYPE_INST, FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x73, 1, X86InstInfo{"VPSLLDQ", TYPE_INST, FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{0x7F, 1, X86InstInfo{"VMOVDQU64", TYPE_INST, FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
{0xE7, 1, X86InstInfo{"VMOVNTDQ", TYPE_INST, FLAGS_MODRM | FLAGS_SF_MOD_DST | FLAGS_XMM_FLAGS, 0, nullptr}},
};
GenerateTable(&Table.at(0), EVEXTable, std::size(EVEXTable));
return Table;
}();
}
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/Core/OpcodeDispatcher/H0F38Tables.h"
#include <iterator>
#include <stdint.h>
@@ -119,6 +120,8 @@ std::array<X86InstInfo, MAX_0F_38_TABLE_SIZE> H0F38TableOps = []() consteval {
#undef OPD
GenerateTable(&Table.at(0), H0F38Table, std::size(H0F38Table));
IR::InstallToTable(Table, IR::OpDispatch_H0F38Table);
return Table;
}();
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/Core/OpcodeDispatcher/H0F3ATables.h"
#include <FEXCore/Core/Context.h>
@@ -55,6 +56,8 @@ std::array<X86InstInfo, MAX_0F_3A_TABLE_SIZE> H0F3ATableOps = []() consteval {
};
GenerateTable(&Table.at(0), H0F3ATable, std::size(H0F3ATable));
IR::InstallToTable(Table, IR::OpDispatch_H0F3ATable);
return Table;
}();
@@ -69,6 +72,7 @@ void InitializeH0F3ATables(Context::OperatingMode Mode) {
if (Mode == Context::MODE_64BIT) {
GenerateTable(&H0F3ATableOps.at(0), H0F3ATable_64, std::size(H0F3ATable_64));
IR::InstallToTable(H0F3ATableOps, IR::OpDispatch_H0F3ATable_64);
}
}
}
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/Core/OpcodeDispatcher/PrimaryGroupTables.h"
#include <FEXCore/Core/Context.h>
@@ -144,6 +145,8 @@ std::array<X86InstInfo, MAX_INST_GROUP_TABLE_SIZE> PrimaryInstGroupOps = []() co
};
GenerateTable(&Table.at(0), PrimaryGroupOpTable, std::size(PrimaryGroupOpTable));
IR::InstallToTable(Table, IR::OpDispatch_PrimaryGroupTables);
return Table;
}();
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/Core/OpcodeDispatcher/SecondaryGroupTables.h"
#include <iterator>
#include <stdint.h>
@@ -488,7 +489,15 @@ std::array<X86InstInfo, MAX_INST_SECOND_GROUP_TABLE_SIZE> SecondInstGroupOps = [
#undef OPD
GenerateTable(&Table.at(0), SecondaryExtensionOpTable, std::size(SecondaryExtensionOpTable));
IR::InstallToTable(Table, IR::OpDispatch_SecondaryGroupTables);
return Table;
}();
void InitializeSecondaryGroupTables(Context::OperatingMode Mode) {
if (Mode == Context::MODE_64BIT) {
IR::InstallToTable(SecondInstGroupOps, IR::OpDispatch_SecondaryGroupTables_64);
}
}
}
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/Core/OpcodeDispatcher/SecondaryModRMTables.h"
#include <iterator>
@@ -56,6 +57,8 @@ std::array<X86InstInfo, MAX_SECOND_MODRM_TABLE_SIZE> SecondModRMTableOps = []()
};
GenerateTable(&Table.at(0), SecondaryModRMExtensionOpTable, std::size(SecondaryModRMExtensionOpTable));
IR::InstallToTable(Table, IR::OpDispatch_SecondaryModRMTables);
return Table;
}();
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/Core/OpcodeDispatcher/SecondaryTables.h"
#include <FEXCore/Core/Context.h>
@@ -270,6 +271,8 @@ auto BaseOpsLambda = []() consteval {
GenerateTable(&Table.at(0), TwoByteOpTable, std::size(TwoByteOpTable));
IR::InstallToTable(Table, IR::OpDispatch_TwoByteOpTable);
return Table;
};
@@ -297,7 +300,7 @@ std::array<X86InstInfo, MAX_REP_MOD_TABLE_SIZE> RepModOps = []() consteval {
{0x2E, 2, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0x30, 16, X86InstInfo{"", TYPE_COPY_OTHER, FLAGS_NONE, 0, nullptr}},
{0x40, 16, X86InstInfo{"", TYPE_COPY_OTHER, FLAGS_NONE, 0, nullptr}},
{0x40, 16, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0x50, 1, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0x51, 1, X86InstInfo{"SQRTSS", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_32BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
@@ -359,6 +362,7 @@ std::array<X86InstInfo, MAX_REP_MOD_TABLE_SIZE> RepModOps = []() consteval {
GenerateTableWithCopy(&Table.at(0), RepModOpTable, std::size(RepModOpTable), &BaseOpsLambda().at(0));
IR::InstallToTable(Table, IR::OpDispatch_SecondaryRepModTables);
return Table;
}();
@@ -383,7 +387,7 @@ std::array<X86InstInfo, MAX_REPNE_MOD_TABLE_SIZE> RepNEModOps = []() consteval {
{0x2E, 2, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0x30, 16, X86InstInfo{"", TYPE_COPY_OTHER, FLAGS_NONE, 0, nullptr}},
{0x40, 16, X86InstInfo{"", TYPE_COPY_OTHER, FLAGS_NONE, 0, nullptr}},
{0x40, 16, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0x50, 1, X86InstInfo{"", TYPE_INVALID, FLAGS_NONE, 0, nullptr}},
{0x51, 1, X86InstInfo{"SQRTSD", TYPE_INST, GenFlagsSizes(SIZE_128BIT, SIZE_64BIT) | FLAGS_MODRM | FLAGS_XMM_FLAGS, 0, nullptr}},
@@ -440,6 +444,7 @@ std::array<X86InstInfo, MAX_REPNE_MOD_TABLE_SIZE> RepNEModOps = []() consteval {
GenerateTableWithCopy(&Table.at(0), RepNEModOpTable, std::size(RepNEModOpTable), &BaseOpsLambda().at(0));
IR::InstallToTable(Table, IR::OpDispatch_SecondaryRepNEModTables);
return Table;
}();
@@ -595,6 +600,7 @@ std::array<X86InstInfo, MAX_OPSIZE_MOD_TABLE_SIZE> OpSizeModOps = []() consteval
GenerateTableWithCopy(&Table.at(0), OpSizeModOpTable, std::size(OpSizeModOpTable), &BaseOpsLambda().at(0));
IR::InstallToTable(Table, IR::OpDispatch_SecondaryOpSizeModTables);
return Table;
}();
@@ -620,12 +626,16 @@ void InitializeSecondaryTables(Context::OperatingMode Mode) {
LateInitCopyTable(&RepModOps.at(0), TwoByteOpTable_64, std::size(TwoByteOpTable_64));
LateInitCopyTable(&RepNEModOps.at(0), TwoByteOpTable_64, std::size(TwoByteOpTable_64));
LateInitCopyTable(&OpSizeModOps.at(0), TwoByteOpTable_64, std::size(TwoByteOpTable_64));
IR::InstallToTable(SecondBaseOps, IR::OpDispatch_TwoByteOpTable_64);
}
else {
LateInitCopyTable(&SecondBaseOps.at(0), TwoByteOpTable_32, std::size(TwoByteOpTable_32));
LateInitCopyTable(&RepModOps.at(0), TwoByteOpTable_32, std::size(TwoByteOpTable_32));
LateInitCopyTable(&RepNEModOps.at(0), TwoByteOpTable_32, std::size(TwoByteOpTable_32));
LateInitCopyTable(&OpSizeModOps.at(0), TwoByteOpTable_32, std::size(TwoByteOpTable_32));
IR::InstallToTable(SecondBaseOps, IR::OpDispatch_TwoByteOpTable_32);
}
}
@@ -6,6 +6,7 @@ $end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include "Interface/Core/OpcodeDispatcher/VEXTables.h"
#include <iterator>
@@ -489,6 +490,8 @@ std::array<X86InstInfo, MAX_VEX_TABLE_SIZE> VEXTableOps = []() consteval {
#undef OPD
GenerateTable(&Table.at(0), VEXTable, std::size(VEXTable));
IR::InstallToTable(Table, IR::OpDispatch_VEXTable);
return Table;
}();
@@ -521,6 +524,7 @@ std::array<X86InstInfo, MAX_VEX_GROUP_TABLE_SIZE> VEXTableGroupOps = []() conste
GenerateTable(&Table.at(0), VEXGroupTable, std::size(VEXGroupTable));
IR::InstallToTable(Table, IR::OpDispatch_VEXGroupTable);
return Table;
}();
}
@@ -225,7 +225,6 @@ enum InstType {
TYPE_SECONDARY_TABLE_PREFIX,
TYPE_X87_TABLE_PREFIX,
TYPE_VEX_TABLE_PREFIX,
TYPE_XOP_TABLE_PREFIX,
TYPE_INST,
TYPE_X87 = TYPE_INST,
TYPE_INVALID,
@@ -466,16 +465,6 @@ constexpr size_t MAX_VEX_TABLE_SIZE = (1 << 13);
// group select (3 bits for now) | ModRM opcode (3 bits)
constexpr size_t MAX_VEX_GROUP_TABLE_SIZE = (1 << 7);
// XOP
// group (2 bits for now) | vex.pp (2 bits) | opcode (8bit)
constexpr size_t MAX_XOP_TABLE_SIZE = (1 << 13);
// XOP group ops
// group select (2 bits for now) | modrm opcode (3 bits)
constexpr size_t MAX_XOP_GROUP_TABLE_SIZE = (1 << 6);
constexpr size_t MAX_EVEX_TABLE_SIZE = 256;
extern std::array<X86InstInfo, MAX_PRIMARY_TABLE_SIZE> BaseOps;
extern std::array<X86InstInfo, MAX_SECOND_TABLE_SIZE> SecondBaseOps;
extern std::array<X86InstInfo, MAX_REP_MOD_TABLE_SIZE> RepModOps;
@@ -494,13 +483,6 @@ extern std::array<X86InstInfo, MAX_0F_3A_TABLE_SIZE> H0F3ATableOps;
extern std::array<X86InstInfo, MAX_VEX_TABLE_SIZE> VEXTableOps;
extern std::array<X86InstInfo, MAX_VEX_GROUP_TABLE_SIZE> VEXTableGroupOps;
// XOP
extern std::array<X86InstInfo, MAX_XOP_TABLE_SIZE> XOPTableOps;
extern std::array<X86InstInfo, MAX_XOP_GROUP_TABLE_SIZE> XOPTableGroupOps;
// EVEX
extern std::array<X86InstInfo, MAX_EVEX_TABLE_SIZE> EVEXTableOps;
template <typename OpcodeType>
struct X86TablesInfoStruct {
OpcodeType first;
@@ -518,7 +500,10 @@ constexpr static inline void GenerateTable(X86InstInfo *FinalTable, X86TablesInf
X86InstInfo const &Info = Op.Info;
for (uint32_t i = 0; i < Op.second; ++i) {
if (FinalTable[OpNum + i].Type != TYPE_UNKNOWN) {
ERROR_AND_DIE_FMT("Duplicate Entry {}->{}", FinalTable[OpNum + i].Name, Info.Name);
LOGMAN_MSG_A_FMT("Duplicate Entry {}->{}", FinalTable[OpNum + i].Name, Info.Name);
}
if (FinalTable[OpNum + i].OpcodeDispatcher) {
LOGMAN_MSG_A_FMT("Already installed an OpcodeDispatcher for 0x{:x}", OpNum + i);
}
FinalTable[OpNum + i] = Info;
}
@@ -533,7 +518,7 @@ constexpr static inline void GenerateTableWithCopy(X86InstInfo *FinalTable, X86T
X86InstInfo const &Info = Op.Info;
for (uint32_t i = 0; i < Op.second; ++i) {
if (FinalTable[OpNum + i].Type != TYPE_UNKNOWN) {
ERROR_AND_DIE_FMT("Duplicate Entry {}->{}", FinalTable[OpNum + i].Name, Info.Name);
LOGMAN_MSG_A_FMT("Duplicate Entry {}->{}", FinalTable[OpNum + i].Name, Info.Name);
}
if (Info.Type == TYPE_COPY_OTHER) {
FinalTable[OpNum + i] = OtherLocal[OpNum + i];
@@ -568,7 +553,7 @@ constexpr static inline void GenerateX87Table(X86InstInfo *FinalTable, X86Tables
X86InstInfo const &Info = Op.Info;
for (uint32_t i = 0; i < Op.second; ++i) {
if (FinalTable[OpNum + i].Type != TYPE_UNKNOWN) {
ERROR_AND_DIE_FMT("Duplicate Entry {}->{}", FinalTable[OpNum + i].Name, Info.Name);
LOGMAN_MSG_A_FMT("Duplicate Entry {}->{}", FinalTable[OpNum + i].Name, Info.Name);
}
if ((OpNum & 0b11'000'000) == 0b11'000'000) {
// If the mod field is 0b11 then it is a regular op
@@ -1,143 +0,0 @@
// SPDX-License-Identifier: MIT
/*
$info$
tags: frontend|x86-tables
$end_info$
*/
#include "Interface/Core/X86Tables/X86Tables.h"
#include <iterator>
#include <stdint.h>
namespace FEXCore::X86Tables {
using namespace InstFlags;
std::array<X86InstInfo, MAX_XOP_TABLE_SIZE> XOPTableOps = []() consteval {
std::array<X86InstInfo, MAX_XOP_TABLE_SIZE> Table{};
#define OPD(group, pp, opcode) ( (group << 10) | (pp << 8) | (opcode))
constexpr uint16_t XOP_GROUP_8 = 0;
constexpr uint16_t XOP_GROUP_9 = 1;
constexpr uint16_t XOP_GROUP_A = 2;
constexpr U16U8InfoStruct XOPTable[] = {
// Group 8
{OPD(XOP_GROUP_8, 0, 0x85), 1, X86InstInfo{"VPMAXSSWW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0x86), 1, X86InstInfo{"VPMACSSWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0x87), 1, X86InstInfo{"VPMAXSSDQL", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0x8E), 1, X86InstInfo{"VPMACSSDD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0x8F), 1, X86InstInfo{"VPMACSSDQH", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0x95), 1, X86InstInfo{"VPMAXSWW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0x96), 1, X86InstInfo{"VPMAXSWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0x97), 1, X86InstInfo{"VPMAXSDQL", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0x9E), 1, X86InstInfo{"VPMACSDD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0x9F), 1, X86InstInfo{"VPMACSDQH", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xA2), 1, X86InstInfo{"VPCMOV", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xA3), 1, X86InstInfo{"VPPERM", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xA6), 1, X86InstInfo{"VPMADCSSWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xB6), 1, X86InstInfo{"VPMADCSWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xC0), 1, X86InstInfo{"VPROTB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xC1), 1, X86InstInfo{"VPROTW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xC2), 1, X86InstInfo{"VPROTD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xC3), 1, X86InstInfo{"VPROTQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xCC), 1, X86InstInfo{"VPCOMccB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xCD), 1, X86InstInfo{"VPCOMccW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xCE), 1, X86InstInfo{"VPCOMccD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xCF), 1, X86InstInfo{"VPCOMccQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xEC), 1, X86InstInfo{"VPCOMccUB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xED), 1, X86InstInfo{"VPCOMccUW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xEE), 1, X86InstInfo{"VPCOMccUD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_8, 0, 0xEF), 1, X86InstInfo{"VPCOMccUQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
// Group 9
{OPD(XOP_GROUP_9, 0, 0x01), 1, X86InstInfo{"", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}}, // Group 1
{OPD(XOP_GROUP_9, 0, 0x02), 1, X86InstInfo{"", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}}, // Group 2
{OPD(XOP_GROUP_9, 0, 0x12), 1, X86InstInfo{"", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}}, // Group 3
{OPD(XOP_GROUP_9, 0, 0x80), 1, X86InstInfo{"VFRZPS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x81), 1, X86InstInfo{"VFRCZPD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x82), 1, X86InstInfo{"VFRCZSS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x83), 1, X86InstInfo{"VFRCZSD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x90), 1, X86InstInfo{"VPROTB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x91), 1, X86InstInfo{"VPROTW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x92), 1, X86InstInfo{"VPROTD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x93), 1, X86InstInfo{"VRPTOQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x94), 1, X86InstInfo{"VPSHLB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x95), 1, X86InstInfo{"VPSHLW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x96), 1, X86InstInfo{"VPSHLD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x97), 1, X86InstInfo{"VPSHLQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x98), 1, X86InstInfo{"VPSHAB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x99), 1, X86InstInfo{"VPSHAW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x9A), 1, X86InstInfo{"VPSHAD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0x9B), 1, X86InstInfo{"VPSHAQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xC1), 1, X86InstInfo{"VPHADDBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xC2), 1, X86InstInfo{"VPHADDBD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xC3), 1, X86InstInfo{"VPHADDBQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xC6), 1, X86InstInfo{"VPHADDWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xC7), 1, X86InstInfo{"VPHADDWQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xCB), 1, X86InstInfo{"VPHADDDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xD1), 1, X86InstInfo{"VPHADDUBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xD2), 1, X86InstInfo{"VPHADDUBD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xD3), 1, X86InstInfo{"VPHADDUBQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xD6), 1, X86InstInfo{"VPHADDUWD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xD7), 1, X86InstInfo{"VPHADDUWQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xDB), 1, X86InstInfo{"VPHADDUDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xE1), 1, X86InstInfo{"VPHSUBBW", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xE2), 1, X86InstInfo{"VPHSUBBD", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_9, 0, 0xE3), 1, X86InstInfo{"VPHSUBDQ", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
// Group A
{OPD(XOP_GROUP_A, 0, 0x10), 1, X86InstInfo{"BEXTR", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(XOP_GROUP_A, 0, 0x12), 1, X86InstInfo{"", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}}, // Group 4
};
#undef OPD
GenerateTable(&Table.at(0), XOPTable, std::size(XOPTable));
return Table;
}();
std::array<X86InstInfo, MAX_XOP_GROUP_TABLE_SIZE> XOPTableGroupOps = []() consteval {
std::array<X86InstInfo, MAX_XOP_GROUP_TABLE_SIZE> Table{};
#define OPD(subgroup, opcode) (((subgroup - 1) << 3) | (opcode))
constexpr U8U8InfoStruct XOPGroupTable[] = {
// Group 1
{OPD(1, 1), 1, X86InstInfo{"BLCFILL", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 2), 1, X86InstInfo{"BLSFILL", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 3), 1, X86InstInfo{"BLCS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 4), 1, X86InstInfo{"TZMSK", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 5), 1, X86InstInfo{"BLCIC", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 6), 1, X86InstInfo{"BLSIC", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(1, 7), 1, X86InstInfo{"T1MSKC", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
// Group 2
{OPD(2, 1), 1, X86InstInfo{"BLCMSK", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(2, 6), 1, X86InstInfo{"BLCI", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
// Group 3
{OPD(3, 0), 1, X86InstInfo{"LLWPCB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(3, 1), 1, X86InstInfo{"SLWPCB", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
// Group 4
{OPD(4, 0), 1, X86InstInfo{"LWPINS", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
{OPD(4, 1), 1, X86InstInfo{"LWPVAL", TYPE_UNDEC, FLAGS_NONE, 0, nullptr}},
};
#undef OPD
GenerateTable(&Table.at(0), XOPGroupTable, std::size(XOPGroupTable));
return Table;
}();
}
+27 -2
View File
@@ -17,9 +17,34 @@
#include <shared_mutex>
#include <FEXCore/HLE/SourcecodeResolver.h>
namespace FEXCore::CPU {
union Relocation;
} // namespace FEXCore::CPU
namespace FEXCore::Core {
struct DebugData;
}
struct DebugDataSubblock {
uint32_t HostCodeOffset;
uint32_t HostCodeSize;
};
struct DebugDataGuestOpcode {
uint64_t GuestEntryOffset;
ptrdiff_t HostEntryOffset;
};
/**
* @brief Contains debug data for a block of code for later debugger analysis
*
* Needs to remain around for as long as the code could be executed at least
*/
struct DebugData : public FEXCore::Allocator::FEXAllocOperators {
uint64_t HostCodeSize; ///< The size of the code generated in the host JIT
fextl::vector<DebugDataSubblock> Subblocks;
fextl::vector<DebugDataGuestOpcode> GuestOpcodes;
fextl::vector<FEXCore::CPU::Relocation>* Relocations;
};
} // namespace FEXCore::Core
namespace FEXCore::Context {
class ContextImpl;
}
+59 -1
View File
@@ -548,13 +548,16 @@ protected:
// This must directly match bytes to the named opsize.
// Implicit sized IR operations does math to get between sizes.
enum OpSize : uint8_t {
enum class OpSize : uint8_t {
iUnsized = 0,
i8Bit = 1,
i16Bit = 2,
i32Bit = 4,
i64Bit = 8,
f80Bit = 10,
i128Bit = 16,
i256Bit = 32,
iInvalid = 0xFF,
};
enum class FloatCompareOp : uint8_t {
@@ -578,16 +581,71 @@ enum class ShiftType : uint8_t {
// This is a nop operation and will be eliminated by the compiler.
static inline OpSize SizeToOpSize(uint8_t Size) {
switch (Size) {
case 0: return OpSize::iUnsized;
case 1: return OpSize::i8Bit;
case 2: return OpSize::i16Bit;
case 4: return OpSize::i32Bit;
case 8: return OpSize::i64Bit;
case 10: return OpSize::f80Bit;
case 16: return OpSize::i128Bit;
case 32: return OpSize::i256Bit;
case 0xFF: return OpSize::iInvalid;
default: FEX_UNREACHABLE;
}
}
// This is a nop operation and will be eliminated by the compiler.
static inline uint8_t OpSizeToSize(IR::OpSize Size) {
switch (Size) {
case OpSize::iUnsized: return 0;
case OpSize::i8Bit: return 1;
case OpSize::i16Bit: return 2;
case OpSize::i32Bit: return 4;
case OpSize::i64Bit: return 8;
case OpSize::f80Bit: return 10;
case OpSize::i128Bit: return 16;
case OpSize::i256Bit: return 32;
case OpSize::iInvalid: return 0xFF;
default: FEX_UNREACHABLE;
}
}
static inline uint16_t OpSizeAsBits(IR::OpSize Size) {
LOGMAN_THROW_A_FMT(Size != IR::OpSize::iInvalid, "Invalid Size");
return IR::OpSizeToSize(Size) * 8u;
}
template<typename T>
requires (std::is_integral_v<T>)
static inline OpSize operator<<(IR::OpSize Size, T Shift) {
LOGMAN_THROW_A_FMT(Size != IR::OpSize::iInvalid, "Invalid Size");
return IR::SizeToOpSize(IR::OpSizeToSize(Size) << Shift);
}
template<typename T>
requires (std::is_integral_v<T>)
static inline OpSize operator>>(IR::OpSize Size, T Shift) {
LOGMAN_THROW_A_FMT(Size != IR::OpSize::iInvalid, "Invalid Size");
return IR::SizeToOpSize(IR::OpSizeToSize(Size) >> Shift);
}
static inline OpSize operator/(IR::OpSize Size, IR::OpSize Divisor) {
LOGMAN_THROW_A_FMT(Size != IR::OpSize::iInvalid, "Invalid Size");
return IR::SizeToOpSize(IR::OpSizeToSize(Size) / IR::OpSizeToSize(Divisor));
}
template<typename T>
requires (std::is_integral_v<T>)
static inline OpSize operator/(IR::OpSize Size, T Divisor) {
LOGMAN_THROW_A_FMT(Size != IR::OpSize::iInvalid, "Invalid Size");
return IR::SizeToOpSize(IR::OpSizeToSize(Size) / Divisor);
}
static inline uint8_t NumElements(IR::OpSize RegisterSize, IR::OpSize ElementSize) {
LOGMAN_THROW_A_FMT(RegisterSize != IR::OpSize::iInvalid && ElementSize != IR::OpSize::iInvalid, "Invalid Size");
return IR::OpSizeToSize(RegisterSize) / IR::OpSizeToSize(ElementSize);
}
#define IROP_ENUM
#define IROP_STRUCTS
#define IROP_SIZES
File diff suppressed because it is too large. Load diff
Loaded 100 of 1335 files, more files were not shown because too many files have changed in this diff. Show more